Instructions for WordPress 2.1 or 2.2 and later
Open /wp-admin/admin-functions.php and look for the following code:
$max_side = apply_filters( 'wp_thumbnail_max_side_length', 128, $attachment_id, $file );
This works a bit differently than earlier versions of WordPress. The number specified here (128 by default) is the maximum size of either dimension. You can still just change this to whatever number you like though.
Reference (Actually more of a copy & paste)