Most Complete Guide EVER on Compressing and Resizing Video for iPod

The Complete Guide to iPod, Apple TV and iPhone Video Formats | iLounge

The Complete Guide to iPod, Apple TV and iPhone Video Formats

By Jesse David Hollington Contributing Editor Published: Friday, July 13, 2007 Last Update: Friday, July 13, 2007

digg this article  tell a friend  submit a news tip

In the fall of 2005, Apple took their first steps into the portable video market with the fifth-generation iPod with video capabilities. This original “video iPod” sported a 320x240 screen and supported playback of videos encoded only at that maximum resolution using very specific formats. At the same time, the iTunes Store began offering video content in these formats suitable for playing on the iPod.

The limited content available on the iTunes Store, and the limited availability of it outside of the U.S. meant that many iPod owners immediately turned to converting their own videos into a format suitable for viewing on their iPod, and a plethora of utilities became available to automate this process.

The following year came an upgrade to the fifth-generation iPod providing higher-resolution video playback. This was soon followed by the Apple TV to provide playback of video content on your home entertainment system, and then the iPhone, providing a more natural widescreen portable video experience.

Even today, however, the video formats supported by the iPod, Apple TV, and iPhone are relatively limited compared to the number of formats available on the market. This means that just about any content that you want to view on these devices is going to require some type of conversion process. The advent of these new devices has complicated the landscape even further, since more options are now available for both the viewing and encoding of videos, but with these changes come more considerations about how to encode video for the best possible viewing experience.

In later articles, we’ll look at some of the specific tools and options available for converting video to an iPod, iPhone or Apple TV ready format, and some of the pros and cons of each. Before going into that, however, it’s important to start with a discussion of the video formats that are actually supported by each device so that those looking to convert their own video content can make informed decisions about the options available and the best resolution and format to use before starting the lengthy encoding process.

Apple’s Choice of Video Formats

The iPod, Apple TV and iPhone will play back videos encoded using either the MPEG-4 or H.264 codecs. These are open-standard video formats, and not in any way proprietary to Apple, but at the same time do not represent a broad portion of the video content that is currently available outside of the iTunes Store. Further, this does not represent the video standard that is used by most video recording devices, TV recording devices, or commercial DVDs. The result is that finding video content from anywhere other than the iTunes Store that is already encoded in an Apple-ready format is going to be difficult, and much of this content will therefore need to be converted.

For example, most commercial video cameras use either uncompressed Digital Video (or “DV") or MPEG-2. Commercial DVDs also use MPEG-2 as their format. Videos downloaded from the Internet can be in any number of formats, including DivX, Windows Media Video (WMV) or QuickTime, among others.

Apple’s likely reason for these particular choices of codec is that they are an open, established standard, and they both provide a very high level of video and audio quality for a given file size. MPEG-4 has historically been very good in this regard to begin with, and the H.264 codec has only improved on the quality and file size efficiency.

As one would expect, when developing a portable video playback device, the quality-to-size ratio is very important both in terms of maximizing the amount of content that can fit on the more limited storage of a portable device, as well as maximizing the battery life of the device, as larger content can require additional processing power, thus shortening battery life. The H.264 codec appears to have been a natural fit to address both requirements, as well as providing a stable, open standard for Apple to use for their preferred video format.

Content on the iTunes Store uses the H.264 codec exclusively. Content you encode yourself can be encoded into either H.264 or MPEG-4, although H.264 will generally provide better quality for a given file-size, it also takes longer to encode.

Resolution and Bit-rate

Two other important considerations with video playback and the quality of the resulting file are the resolution and the bit-rate. The resolution simply refers to the dimension of the screen image, in terms of the number of pixels wide by the number of pixels high (ie, “640x480") while the bit-rate refers to the amount of data that is actually encoded to make up a single second of video playback, normally expressed in either kilobits per second or megabits per second. This is the same concept as bit-rates for audio formats such as AAC and MP3.

Naturally, the higher that both of these numbers are, the better the quality of the resulting image will be. These factors both work together, however, since the resolution simply specifies the number of pixels available in a specific frame, and the bit-rate specifies how much actual information is being used to generate those pixels.

Without getting into too much technical detail, modern lossy video compression formats actually work not by encoding every single pixel of every single frame of a video, but rather simply encoding the information that changes between frames. “Reference frames” are taken at certain points, and the remaining frames are built based on what changes between each frame.

To conceptualize this, imagine watching a pro golfer make a putt:  Most of the scenery, the trees, the sky, and the golf course itself remains relatively unchanged throughout the putt, with only some motion from the golfer and of course the little white ball rolling across the green. Encoding every single frame of this would take up a lot of storage space (the equivalent of several hundred high-resolution JPEG files), when in reality very little is changing between frames. As a result, only the information that is different in each frame actually needs to be stored, and this can later be applied to the original reference frame to build a smooth video playback motion.

The bottom line is that this makes for significantly more efficient file sizes and storage of video content, and in fact is the same technology that DVDs use with the MPEG-2 format. Without MPEG-2 compression, a 90-minute DVD movie would actually occupy well over 167 GB of space, rather than fitting comfortably onto a 4.7 GB DVD.

How it directly affects the concept of bit-rates, therefore, has not to do with how much data is actually being encoded for each frame, but how much data is available to record the differences in frames. A high-resolution video with a low bit-rate will produce a more blurry image as well as visible signs of “artifacting” (distortion, blocking effects and jagged edges) on high-motion action sequences.

So what resolution and bit-rate should you use to encode your video content?  Logic would suggest that one should always use the highest settings possible, but much of this has to do with what your source content is, and the limitations of the equipment you’ll be using to watch it.

For example, if you only ever intend to watch video content on the iPod’s 320x240 screen, it would obviously be a waste of disk space and encoding time to convert these videos to any resolution higher than this.

Further, if your original source video is already at a smaller resolution, there is absolutely no point in encoding it in any higher resolution or bit-rate, as you’re not going to magically gain any resolution that wasn’t there in the first place. In our experience, consumer-grade conversion tools that promise “upconversion” of video formats are generally not worth the effort.

Likewise, even though the Apple TV supports a high-definition quality of video output (1280 x 720), if you are converting standard-definition DVDs, there is no sense in encoding them in anything beyond their original resolution.

So what are the different resolutions and formats supported by Apple’s video playback devices?  The table below provides an outline of the maximum resolutions supported, as well as the output quality of the device itself:

Naturally if you want to encode content for playback on the iPod and iPhone, you will have to stick to the lower-resolution settings of these devices—640 x 480 at 1.5 mbps. Content encoded in the higher Apple TV resolutions will not play on anything but the Apple TV.

Note that the resolutions above for the iPod and iPhone are the officially supported ones, and have been simplified in Apple’s specs to represent the more standard resolutions. More advanced users have noted that it is possible to push these resolutions and bit-rates slightly, although iTunes itself will not necessarily support the transfer of these files to the iPod or iPhone. For maximum compatibility it is therefore always best to stay within the published specifications.

A Word on Frame Rates

The “frame rate” of a video clip refers to the number of frames, or still images, for each second of video, normally expressed in “frames-per-second” or fps for short. Normal North American TV (NTSC) is broadcast in approximately 30 fps, while theatrical movies are normally 24 fps due to being shot on film. This means that if you’re encoding a DVD movie, chances are you’ll be looking to use 24 fps, whereas most other video content you record, including your own camera content and broadcast television content will be 30 fps. With the exception of the higher Apple TV resolution noted above, all resolutions support a maximum frame rate of 30 fps.

Bit-rate and File Size

Another important consideration when encoding videos is the size of the resulting file. This will correspond directly to the bit-rate of the file rather than the resolution (although higher resolutions require higher bit-rates), and while the H.264 and MPEG-4 codecs do a very efficient job of producing high-quality video at lower bit-rates, you’re still going to ultimately be constrained by how much space you have available on your particular device and how much content you want to be able to store.

If you’re inclined to do a bit of math, you can get an easy approximation of the file-size of a converted video simply by multiplying the bit-rate by the duration of the video, using a formula like the following:


size = (bit-rate / 8) * (length in minutes * 60)


The resulting size will be in either kilobytes if a kbps bitrate was used, or megabytes if a mbps bit-rate was used. So, for example, to calculate the size of a 90-minute movie being encoded at 1500 kbps, you would plug the numbers in as follows:


1500 / 8 * 90 * 60
187.5 * 5400
= 1,012,500 (KB)
or approximately 1 GB.


A simple rule-of-thumb, however, is that for standard iPod-encoded content (1.5mbps) you’ll be looking at approximately 500MB per hour of video content. Apple TV content at 3 mbps will naturally be twice that size (1GB per hour).

If storage space is of primary concern, encoding at lower bit-rates will naturally save some space, but this will come at the cost of video quality. Any bit-rate less than 1 mbps for 640x480 content will generally be too low to produce reasonably viewable TV output, for example.

Consider the Source...

Keep in mind as well when determining the best bit-rate to use to always keep the source content in mind. Older TV Shows on DVD, for example, will seldom benefit from using an extremely high bit-rate, since the original video quality was relatively low to begin with. On the other hand, recent blockbuster movies with lots of action and motion should definitely be encoding with as high a bit-rate as reasonably possible for the target device.

This is particularly relevant when considering whether to encode in a bit-rate suitable for the iPod or the Apple TV. Older TV content and analog TV recordings will rarely benefit from the higher-quality settings available for the Apple TV.

Aspect Ratios

Another very important consideration when trying to decide on the optimal format in which to encode your content is the aspect ratio of the original source content.

Aspect ratio simply refers to the ratio of the width of the image to the height of the image as shown on screen. There are three common aspect ratios in use for commercial video content today:

• 4:3 used for almost all standard-definition TV broadcast content. This is sometimes also referred to as 1.33:1:
  
• 16:9 used for almost all high-definition TV content (HDTV) and many theatrical DVD releases. This is sometimes also referred to as 1.78:1:
  
• 2.35:1 used for “Cinemascope” or “Panavision” movies on DVD:
  

For commercial DVDs, you will generally find the aspect ratio indicated on the back. For other types of video content, you can determine the aspect ratio yourself simply by dividing the width of a video by its height.  For example a 640x480 video clip would have a 1.33:1 aspect ratio (640/480 = 1.33).

Since the aspect ratio of the content will not always match the aspect ratio of the actual output device, you will often get either “letterboxing” or cropping, depending on the device and its settings. Letterboxing is more common, which is the practice of adding black bars to the top and bottom of a widescreen video when playing it back on a standard TV.

A widescreen TV show, 16:9 aspect ratio, as shown on a 4:3 screen

A Cinemascope Movie, 2.35:1 aspect ratio, as shown on a 4:3 screen

Conversely, if you were to watch a standard 4:3 TV show on a widescreen TV, you will end up with something referred to as pillarboxing which places black bars at the sides of the image:

A standard 4:3 TV show as shown on a 16:9 widescreen TV

On the other hand, some TVs and other devices (such as the iPod itself) also provide the option for cropping content to fit the aspect ratio of the screen. As the name implies, cropping the content chops off the portion of the image that does not fit, expanding the image to the full size of the screen.

This will result in the loss of detail on the left and right sides when trying to display widescreen content on a 4:3 screen:

A widescreen TV show cropped for 4:3 display

A Cinemascope Movie, 2.35:1 aspect ratio, cropped for 4:3 display

Or, the top and bottom of the image in the case of trying to display 4:3 content to fill a 16:9 screen:

A standard 4:3 TV show cropped for 16:9 display

Anamorphic Encoding

Some newer video conversion tools now provide support for anamorphic encoding. This basically refers to encoding video content in one aspect ratio but setting it to display using another, through the use of non-square pixels.

A pixel is not necessarily always rendered as a perfectly square element of a picture, and many modern video playback applications and hardware support a pixel aspect ratio (PAR) setting. In fact, the aspect ratio of content as stored on a normal North American DVD is actually 1.5:1 (720 x 480), rather than either 16:9 or 4:3. A DVD player, however, produces an image in the appropriate aspect ratio by reading a flag within the content that tells the player what shape of pixels to use when playing back the video content. The result is that a 720x480 DVD image is actually rendered in 640 x 480 (4:3) or 854 x 480 (16:9).

Without getting into too much technical detail, the short explanation is that the use of anamorphic encoding can provide a proper widescreen presentation of a movie without having to force it to actually be encoded at the higher resolution. Since nothing is truly gained by increasing the encoded resolution of a video, it makes more sense to save the storage space and simply render the frame in its proper aspect ratio, in the same way that a DVD player does.

An anamorphically encoded video, when examined in Quicktime’s Show Movie Inspector dialog, will actually show two different resolutions:

The first number, 720 x 480, is the actual stored resolution of the video image. The second represents the aspect ratio that it will play back in. The 720 horizontal pixels will simply be stretched to fill the width of the screen, resulting in a proper 16:9 presentation.

Note that if you are viewing an anamorphically-encoded video on a device that doesn’t understand pixel aspect ratio (PAR), then the resulting video will appear distorted, as it will simply play back in its actual stored resolution.

At this point, the fifth-generation iPod and Apple TV support pixel aspect ratio and will therefore display an anamorphic video properly.  The iPhone does not presently handle pixel aspect ratio, and anamorphic videos will therefore look distorted when viewed on the iPhone.

Hard Letterboxing

One other problem that you may encounter is that many DVDs, particularly older ones, were actually in what was known as a “hard letterbox” format, rather than a proper anamorphic widescreen format. In this case, even though the original video format may have been widescreen, the DVD video is not actually encoded in a widescreen aspect ratio. Rather, “letterboxed” movies were basically encoded onto the DVD in a 4:3 aspect ratio with the black bars at the top and bottom actually added to the movie. This would allow these movies to play in a widescreen format even on older DVD players that did not have a widescreen mode.

Unfortunately, however, when these DVDs are converted, most video converters will leave them in their original 4:3 aspect ratio, with the black bars as part of the frame. In this case, the resulting video will be treated as a 4:3 video, with the black bars continuously shown at the top and bottom of the frame. Cropping will not be available on the iPod, and the Apple TV and iPhone will show these videos by default in a black frame:

In this case, the black bars at the top and bottom are not being generated by the output device, but are in reality part of the video image.

When dealing with DVDs, the description on the packaging can often be helpful in determining what the source format is. Although the terminology differs among the different studios, many older DVDs that used the term “Letterboxed” referred to this type of encoding: a 4:3 image with the black bars added to the frame. On the other hand, the term “anamorphic widescreen” almost always refers to videos that are properly encoded in 1.85:1 or 2.35:1 aspect ratio. Note that this will be a much less common issue with DVDs that have been more recently published.

You can also check whether or not your video content is in a proper widescreen format by viewing it in a window on your computer through iTunes or QuickTime. When playing video in a window, QuickTime will not add any black bars, but will rather show the window itself in the proper aspect ratio.

On the other hand, if a video is “letterboxed” and has been encoded in a 4:3 aspect ratio, the QuickTime window will be in a 4:3 aspect ratio as well, and the black bars will be apparent in the video.

Note that the same holds true for most DVD playback applications, so you can often check a DVD before encoding it by simply playing it back on your computer in a window through your software DVD player, such as Apple’s DVD Player app included with Mac OS X.

In this case, to produce a proper widescreen video, you would need to manually crop the video using your original encoder to remove the black bars from the top and bottom of the image.

Apple Devices and Aspect Ratios

All of this creates an additional consideration when encoding video content based on your choice of device:

• The 5G iPod has a screen that is a 4:3 aspect ratio (320 x 240) and will also only output to a TV in a 4:3 aspect ratio. Widescreen videos on the iPod or TV output will either be shown letterboxed or cropped, depending on the “Widescreen” setting on the iPod itself.
• The Apple TV on the other hand is designed to output to a 16:9 widescreen HDTV. Standard TV content will therefore be shown as pillarboxed by the Apple TV. Most high-definition TVs have a “zoom” feature that can used to crop the 4:3 output instead if this is preferred.
• The iPhone has a screen with a 1.5:1 aspect ratio (480 x 320), and no TV output capabilities at this time. 4:3 content will be pillarboxed by default, and 16:9 and 2.35:1 content will be letterboxed by default. The iPhone implements a “zoom” feature to crop the image to fill the screen in either aspect ratio.

Although the aspect ratio is going to be determined by the source content, this is an important consideration when deciding on the optimal resolution due to issues of maximum horizontal and vertical resolution.

For instance, the maximum resolution of a video encoded for iPod playback is 640 x 480, which is essentially standard-definition TV resolution in a 4:3 aspect ratio. Unfortunately, this maximum resolution means that 16:9 content gets encoded for the iPod at 640 x 360 and 2.35:1 content gets encoded at 640 x 272. Both of these are significantly below the resolution of a standard TV, and will look significantly deficient when played on an HDTV.

Therefore, while encoding standard 4:3 TV content for playback on both an iPod and Apple TV may be practical, since the maximum resolution of the source material is limited to 640 x 480 anyway, encoding movie content intended for playback on the Apple TV is a tougher decision. The Apple TV itself can display 16:9 widescreen content in full 854 x 480 resolution, taking full advantage of the original DVD quality, but this will of course limit these files to being playable only on the Apple TV.

The following table illustrates the maximum resolutions that can be played by each device for the three common aspect ratios:

Note that although the Apple TV is capable of playing 4:3 content in a much higher resolution than the iPod, most of the available 4:3 content will only be in a 640 x 480 resolution to begin with, and nothing is truly gained by increasing this resolution to anything higher.

Single-Device Viewing

Of course, it stands to reason that if you’re going to be encoding video solely for the purpose of watching on one specific portable device, and never intend to watch it anywhere else ("never" being a very long-term word), then you should encode it to the screen resolution of that device for the best possible quality and optimal file-size. The iPod and iPhone will scale down anything larger to their native screen resolution anyway, so encoding anything in a larger resolution is simply a waste of space.

Keep in mind, however, that the 5G iPod also offers the capability of outputting video content to a TV, and 320 x 240 content will look noticeably poor on anything but the smallest of TV screens. This alone may be a reason to use the higher 640 x 480 resolution, even for iPod content.

Recommended Resolutions for typical content

So, armed with all of this information, what are the recommended resolutions that should be used for the content that you’re likely to be encoding? 

Ultimately, the ideal resolution will be determined by the source material in question, the desired output quality, available storage space, and of course the devices on which you’ll be watching the content. However, a brief summary of the recommended maximum resolutions for compatibility with the various devices is shown in the following table:

With the exception of standard 4:3 TV content, videos encoded in the “Apple TV” resolutions noted above will only play back on the Apple TV. On the other hand, videos encoded in the lower resolutions supported by the iPhone and iPod will also play on an Apple TV, although the more limited vertical resolutions for widescreen and CinemaScope content will produce much lower-quality images when viewed on larger-screen TVs, making the decision on whether to trade off for mobile compatibility or TV viewing quality a difficult one for some.

Note as well that when encoding standard-definition DVDs for the Apple TV, considerable space can be saved when using a tool that supports anamorphic encoding, since increasing the size of a 720 x 480 video to the larger horizontal resolutions above may require a higher bit-rate to render and thereby unnecessarily increase the file size.

Conclusion

While perhaps not the most inexpensive method, the easiest way to get video content that is playable on the iPod is still to purchase it from the iTunes Store. However, this is not always practical for all users, and certainly not even available to many iPod users outside of the U.S. As a result, the only practical way for many to get video content in an Apple-ready format and build a library is to convert your own content.

Unfortunately, video conversion can be a time-consuming task, so it is important to have a good understanding of the various issues before spending many dozens of hours converting video only to discover that it is not in the most optimal format. Further, due to the time required, it is a good idea to try and create a video collection that takes into account the platforms and devices that you may want to use in the future.

In our next article we will be looking at some of the more popular conversion tools that are available for encoding video content onto the iPod from various sources and covering some tips on ways to best optimize video conversion using these tools.

Blogged with Flock