We first wrote about the Chroma Upsampling Error (CUE) back in April, 2001. At that time, we only told you half the story (because we had not worked out the second half yet). Now we’re back again to give you the latest information on what is commonly referred to in the A/V community as the “Chroma Bug.” If you are just now hearing about the Chroma Bug and you have come here looking for answers, you might be interested in how all of this started.

We were first introduced to this problem by Richard Ansell of Snell & Wilcox just a little over three years ago. Richard showed us the problem using an expensive boutique DVD player. The scene he used at the time was from “The Fifth Element” with a close-up of a big red button, and it looked really jagged. He didn’t know what was causing the artifact, and neither did we. But we started seeing it all over the place, not just in DVD players, but MPEG decoders in general. The problem definitely predates DVD, and from what we have learned, the satellite system used in Europe suffered from this problem before DVD came to market.

We have long been using “Toy Story” (Pictured above from the 3-Disc Ultimate Toy Box version) as our example because its highly saturated colors make the problem easy to spot and demonstrate to others, but it wasn’t the first one we discovered on our own. The first example we found after Richard shared the problem with us was from “Drop Zone” with Wesley Snipes. It was chapter 5 where Snipes gets dropped from the plane without a parachute. The red skydiving suit with black stripes that Yancy Butler wears really showed off the problem. (An example is at the end of the report.)

We had planned to cover this artifact back with the very first DVD Benchmark, but we did not know what caused it, and we had a hard time getting any answers. It was not until we sat down and read through the MPEG spec and talked to some MPEG experts that we understood what was happening.

We have just finished (December, 2002) looking at the latest crop of DVD players from several DVD manufacturers, and we are disappointed to learn that this problem still exists, even though it’s been well documented, by us and others, and written about in magazines of all kinds. The problem is not limited to DVD players. It can happen in any video compression codec that uses 4:2:0, which includes MPEG-2 and MPEG-4.

At this time, Denon is the only consumer electronic manufacturer to publicly acknowledge the existence of the problem in their players. Not only have they admitted the problem, but their engineers worked with the engineers at ESS to fix the problem. You can now get the DVD-3800 and DVD-9000 from Denon without the bug. Kudos to a company that admits a serious issue publicly and then fixes it. Matsushita (MEI, Panasonic) also deserves a pat on the back for getting it right the first time, as does Mediamatics (now owned by National Semiconductor).

Shortly after the first article was published, Dale Adams at Silicon Image wrote an algorithm to detect the chroma bug and remove it. Silicon Image has finally added a chroma filter to their latest box, the iScan Ultra. We had hoped this filter would have made it into the SiI504, but it did not. We hope that when Silicon Image produces the SiI505, they’ll integrate the chroma filter.

In this article update (December, 2002), we’ll not only cover the bug, we’ll show actual MPEG source code that people were using to do it right long ago, before DVD was even released. We’ll tell you why 4:2:0 is just fundamentally broken for interlaced video. And for any engineers reading, we’ll give specific examples of how to properly interpret the flag to determine the right upsampling mode to use, and how to write a filter that fixes both the chroma bug and the fundamental chroma flaw built into 4:2:0 encoding.

The Error in a Nutshell

The basic “Chroma Bug” manifests itself as streaky or spiky horizontal lines running through the chroma channel, most notably on diagonal edges. As mentioned above, this problem has been around for a long time. It’s only just now being noticed largely because one needs a good high-resolution display, such as a front projector and a six foot projection screen, to really see the problem clearly. In addition, the increasingly common use of large progressive displays has really allowed people to get up close to the screen and see every artifact magnified in great detail. Problems that might have gone unnoticed on a 20 inch interlaced TV suddenly hit you in the face. With the advent of relatively high resolution media like DVD, people are starting to compare the video image to the original film image, not to other forms of TV. And suddenly strange problems that people accepted in a TV picture, but would never be allowed on film, look out of place. The Chroma Bug is one of the most visible artifacts around, but because it’s specific to MPEG and 4:2:0 encoding, there was nothing written about it until very recently.

The underlying problem, stated simply, is this: many of the MPEG decoders in DVD players are not properly converting the 4:2:0 format chroma information on the DVD to the 4:2:2 or 4:4:4 format that video encoders need. The MPEG spec makes it clear that while there is a wide latitude for the MPEG decoder to choose an upsampling algorithm for 4:2:0 to 4:2:2 conversion, the decoder must use two different algorithms to be correct. One should be used for frames marked “interlaced”, and a different one should be used for frames marked “progressive.” Most decoders use only one algorithm for all frames, and that’s where they go wrong.

We’ll go into more detail later, but first you should understand what chroma formats like “4:2:0” and “4:2:2” mean.

Chroma Formats

For decades, video engineers have known that the eye is much less sensitive to color information than grayscale information. This is physiological in nature: the retina of the eye has more gray-sensitive cells, called “rods,” than color-sensitive cells, called “cones.” Your eyes can resolve much finer details in black and white than in color. For this reason, it’s not necessary to store full color resolution on a video recording system. Your eye can’t resolve it, so why waste space (money) for it?

Consumer videotape formats like Beta and VHS take advantage of this fact by devoting most of the tape bandwidth to the black-and-white signal, called “luma” or “luminance,” and less bandwidth to the color signal, called “chroma” or “chrominance.” (In fact, there are subtle differences between “luma” and “luminance” and between “chroma” and “chrominance,” but they aren’t important here. We’ll just refer to “chroma” and “luma” to keep it simple.)

The luma signal is always called “Y.” The chroma signal is actually two signals, modulated together to form a single waveform on a tape. Depending on the color system and format, the two chroma signals may be labeled “U” and “V,” or “Pb” and “Pr,” or “Cb” and “Cr.” These are all slightly different encoding formats, but they are all essentially the same in concept. On DVD, the chroma is stored as ‘Cb’ and ‘Cr’ (the C stands for color, b stands for blue, and r stands for red).

In MPEG-2 (the compression format used for DVD) the Y, ‘Cb’, and ‘Cr’ signals are stored separately (this is why component video needs three cables). Since the Y signal is the black-and-white signal, it is always stored at full resolution. But the color signals can be optionally stored at lesser resolutions, relying on the eye’s relative insensitivity to color in order to mask the resolution loss.

The highest resolution format is 4:4:4, which means that for every 4 samples of Y, there are also 4 samples of ‘Cb’ and 4 samples of ‘Cr’. In other words, the color signal has the same resolution as the black and white signal. This format is generally only used internally within a device to avoid degradation during processing. When the image is recorded to a master tape like D1 or D5, it is reduced to 4:2:2 (see below).

Next highest is 4:2:2, which means for every 4 samples of Y, there are 2 samples of ‘Cb’ and 2 samples of ‘Cr’. There are still the same number of scan lines in the luma and the chroma, but the chroma signal has half as many samples on each scan line. When a 4:2:2 signal is decoded, the “missing” samples are effectively interpolated from the samples on either side.

The lowest resolution format, and the one used for DVD, is 4:2:0. This is a confusing designation, as it suggests that for every 4 Y samples, there are 2 ‘Cb’ samples and 0 ‘Cr’ samples, which is not the case at all. What 4:2:0 means is that there are half as many samples of ‘Cb’ and ‘Cr’ on each scan line, and half as many scan lines of ‘Cb’ and ‘Cr’ as there are of Y. In other words, the resolution for chroma is half that of luma in both the horizontal and vertical directions. If the full image resolution is 720 x 480, then the chroma information is only stored at 360 x 240. In 4:2:0, not only are missing samples interpolated on each scan line, entire scan lines of chroma must be interpolated from the scan lines above and below. The reason this is done is for efficient use of space on the DVD. Sure, 4:4:4 would be nice, but the DVD might have to be two feet in diameter. So, it’s 4:2:0.

The tricky problem of 4:2:0 is that the method of interpreting the samples and interpolating new scan lines can be done two different ways. One way is used for interlaced images, and one for progressive images.

Picture Format

The images on DVD, whether sourced from interlaced video or from a film, are generally stored in full frames, not separate fields (for a refresher on fields vs. frames, please refer to Part 5 of the DVD benchmark, which explains progressive DVD). If the original source was interlaced, the MPEG encoder that writes the DVD takes pairs of fields, weaves them together into full frames, and stores those frames on the disc. If the original source was progressive, like a film or a computer image, the original frames are just written to disc as-is (in fact, under most circumstances the film is converted to interlaced somewhere along the line, but a good encoder will recognize the film cadence and reconstruct the original frames properly). The encoder sets the “progressive_frame” flag to True or False depending on whether the MPEG picture represents a single frame of film or two separate video fields.

On the player end, if the original video was interlaced, then it’s important that the chroma information for one field not leak into the other field. The decoder should split the MPEG picture into two fields, and then convert the chroma to 4:2:2 separately for each field. If, however, the frame was originally progressive, then the chroma information needs to be interpolated to 4:2:2 across the whole frame, then split into fields if necessary. This is a subtle difference, but it’s the root cause of the chroma upsampling problem.

Encoding 4:4:4 to 4:2:0

In Figure 1 (above), you can see the layout of a 4:4:4 encoded image. For every pixel in the image, there is one luma sample and one chroma sample (actually one pair of chroma samples, one for ‘Cb’ and one for ‘Cr’, but we’ll treat them as a single unit).

Figure 2 (above) shows the layout of a 4:2:2 encoded image. Here, half the chroma samples have been thrown away, and while there are luma samples for every pixel, there are chroma samples only for every other pixel. When this is displayed, the missing chroma information will be interpolated from the samples on either side. This doesn’t really cause a problem. As mentioned above, the eye is less sensitive to chroma, and most people can’t tell the difference between 4:2:2 and 4:4:4. Again, it would be nice to have 4:4:4 on the DVD, but there is not enough room.

Now it gets tricky. Figure 3 (above) shows how the luma and chroma samples are laid out, conceptually, for a progressive 4:2:0 frame. There are half as many chroma samples per line, and half as many lines as well. In total, there are one fourth as many chroma samples as luma samples. Note that the chroma samples are defined to be located between the scan lines. How can a sample be between scan lines? It’s simple: the chroma samples stored on the disc are averaged from the chroma information on the original scan lines above and below. The first line of chroma samples is created by averaging lines 1 and 2 from the original image, and the second line is created from image lines 3 and 4, and so forth.

Now look at Figure 4 (above). This is how the chroma values are laid out for two consecutive interlaced 4:2:0 fields, when they are stored together as a single MPEG picture. The chroma values are still located between the scan lines, but there’s an important difference. The first line of chroma values is averaged not from lines 1 and 2, but from lines 1 and 3. And (assuming the encoder is doing the correct thing), the sample should be averaged using 75% line 1 and 25% line 3, because it’s defined to be “virtually” closer to line 1 than line 3.

This is not an easy issue to understand at first, so look again. The first “virtual line” of chroma values is still between image lines 1 and 2. But, line 2 is not used for field 1; it’s part of field 2. So the chroma information is averaged from lines 1 and 3. The “virtual line” of chroma is 0.5 lines away from line 1, and 1.5 lines away from line 3. So Line 1, being 3 times closer, affects the eventual chroma sample three times as much as line 3. Or at least it should. Encoders sometimes play fast and loose with downsampling, so the encoder might just do a simple 50/50 average of lines 1 and 3. Or, it might just use line 1, and throw away line 3. We’ll assume, just to be charitable, that it does, in fact, do the math to generate a mathematically correct 4:2:0 sample.

Note, though, that picture line 2 is not involved in encoding the first chroma line at all. Line 2 is the first line of the next field, and you don’t want color from field 2 bleeding into field 1. There might be movement between the two fields, and it could produce ugly color artifacts.

In the next field, field 2, the second chroma line affects picture lines 2 and 4, but again, the chroma sample is closer to line 4 than line 2, so the sample should be calculated as 75% line 4’s chroma and 25% line 2’s chroma.

(In practice, more sophisticated algorithms than a simple 75/25 average, using more than 2 data points, are often used by MPEG encoders, but the point remains that the chroma information is not supposed to be calculated from a simple 50/50 average.) In any case, if you compare Figure 1 (4:4:4) with Figures 3 and 4 (4:2:0), you will see how much the MPEG decoding process in the player has to fill in the blanks to produce an image on your TV.

Converting 4:2:0 Back to 4:2:2 or 4:4:4

It’s not terribly important whether the output device converts the 4:2:0 information to 4:2:2 or 4:4:4. Many video encoders can accept either format. But it has to be in one of those two formats. The video encoder needs to get a line of chroma for every line of luma, at the same time, because it’s converting the digital video to analog video on the fly. It can’t “remember” what the chroma information was for the previous scan line, so it can’t interpolate missing chroma information. Only 4:2:2 and 4:4:4 have a 1:1 ratio of chroma and luma scan lines, so the MPEG decoder needs to output one or the other.

The MPEG pictures are stored in the same binary format whether they represent a progressive frame or two interlaced fields woven together. The only difference is the “progressive_frame” flag. That flag tells the decoder whether the chroma information should be interpolated across the whole picture, or separated into fields and interpolated separately for each one. Unfortunately, it appears that most MPEG decoders ignore the “progressive_frame” flag, and only do one kind of interpolation. It makes the chip simpler, which reduces cost, though we are beginning to think that the cost isn’t the real issue. The real problem is that many decoder manufacturers bought the core of their MPEG2 decoding code from one company, and that company just didn’t do it right.

The MPEG decoders that use only one algorithm all use the interlaced algorithm, because it’s the “safest.” “When in doubt, assume it’s interlaced” seems to be the motto of these decoders. When the interlaced algorithm is applied to progressive images, chroma samples that were supposed to be used for scan lines 1 and 2 are instead interpolated to scan lines 1 and 3, and the chroma samples for scan lines 3 and 4 instead are affecting 2 and 4. In the simplest case, where the chroma lines are just copied to the adjacent image lines, the end result is that scan line 2 gets the chroma information that should have gone to scan line 3, and vise-versa, all the way down the screen. Effectively, adjacent pairs of chroma scan lines get switched, which produces the characteristic jagged/streaky effect we see in the Chroma Upsampling Error (CUE).

In Figure 5 (above), you can see a simple red shape with a diagonal side on the left. In Figure 6 (also above), it has been converted to 4:2:0, and back to 4:2:2, correctly. The left edge is more jagged, because there are only half as many samples as before, in both directions. But it still looks pretty much like the original shape.

Now consider Figure 7 (above). The shape has been converted back incorrectly, using the simplest “copy each chroma line twice” algorithm. The chroma for line 2 has been swapped with line 3, and the chroma for line 6 has been swapped with line 7. Since the original line 7, which became line 6, didn’t have any red in it, there’s a gap between finished lines 5 and 7. This is the reason you will often see gaps on the top and bottom of colored objects and “ghost” lines floating above or below the objects, if your DVD player has this chroma upsampling problem.

Why This Varies from Player to Player

Different DVD players show the problem differently because they all use slightly different interpolation algorithms. In the simplest case, they just copy the data from the 4:2:0 samples twice to create the 4:2:2 data, as we show in Figures 5 through 7. This tends to make the chroma channel look blocky, and accentuates the jaggedness.

Other players do more sophisticated interpolation using bilinear filtering or multi-tap FIR filtering, which smoothes the chroma channel and can make the upsampling error less visible. But it can’t be completely hidden. It will just look like a smoother version of Figure 7, with the jaggedness and streaks on the edges less noticeable. We’ve looked at a lot of DVD players now, and one thing is clear: the very best, smoothest upsampling of the wrong source lines looks worse than the simplest, least smooth upsampling of the correct source lines. There is no substitute for doing it right in the first place.

A Real World Example

We recently saw the code below in some free MPEG decoder source code (available many places on the net including here in a package called mpeg2v12.zip). The code is from store.c, and shows the section where the code decides what upsampling algorithm to use.
Line     Code
1     /* vertical 1:2 interpolation filter */
2     void conv420to422(src,dst)
3     unsigned char *src,*dst;
4     {
5
6          if (progressive_frame)
7          {
8               /* intra frame */
9
10          }
11          else
12          {
13               /* intra field */
14
15               /* top field */
16
17               /* bottom field */
18
19          }
20     }

The source file is dated 1/9/96, which is well before DVD was released. The function that does the upsampling is called conver420to422 and it begins on line 467 in store.c.

The total amount of code required to do the upsampling correctly is 109 lines, including the ones above, and a clever rewrite could get it done in even fewer extra lines of code. If you look closely you will notice that they use a 6-tap FIR filter, which is better than what most MPEG decoders do. This creates a smooth chroma channel. All of our examples at the end were created using this MPEG decoder.

So, the CUE is not a difficult problem to fix. The above code shows that people have done it right for years. (Though actually this code doesn’t handle the alternating progressive_frame flag problem, about which more will be said later.) The MPEG decoder manufacturers that aren’t doing it right have no excuse, other than inertia. And you as a consumer have a right to expect that the DVD player you buy will have a bug-free decoder.

4:2:0 Interlaced: Fundamentally Broken

When we wrote the first chroma article, an issue we noticed but didn’t write about is that even when the content is correctly marked interlaced, using the interlaced upsampling algorithm always looked wrong. We assumed that this was implementation-dependent at the time, but we now realize that no player can produce a good looking chroma channel on interlaced content. It’s fundamentally a bad design. Below is a scene from “More Tales of the City“. This DVD, like many, has been encoded as . We have blown up several sections in that scene to show you the problem, and note that this is not the CUE. It is an entirely different problem. This was taken from a good MPEG decoder.

Here’s the problem: In 4:2:0 interlaced MPEG frames, the chroma channel is really two separate chroma channels interleaved. One channel is for field one, and the other is for field two. These two chroma fields are extremely low resolution. The chroma channel for a 480 line frame of 4:2:0 has already been cut down to 240 lines of vertical resolution, so a single field only has 120 lines of resolution. In order to convert to 4:2:2 interlaced video, those 120 lines must be upsampled to form two 240-line chroma fields. The upsampling process essentially fills in the missing scan lines of the image, without regard for the samples from the other field. So in the process, detail is left out of each field, and the upsampling process fills it in with what you could call a “best guess.” That guess is different for each field, because the two fields have a completely different set of data to work from. Most importantly, the positions of edge contours, especially diagonals, will always be calculated in slightly different locations in each field because the samples are in different locations in each field.

It’s this mismatch of edge contours that causes the problem. When the two fields are put back together later by a deinterlacer (or by your eye and brain, if you watch it on an interlaced TV), the relatively smooth gradations and contours of each field are broken up by a slightly different set of gradations and contours from the other field. If the image is moving, this isn’t a problem, because you’d never notice that the two fields’ chroma channels don’t match up. But on static content, something that looks very much like the chroma bug appears on the diagonal edges of bright-colored objects.

It’s important to note that this is not the chroma bug. First off, it’s not a bug. The decoder is doing exactly what it’s supposed to do, unlike the classic chroma bug case where it ignores the progressive_frame flag and upsamples with an incorrect algorithm. Secondly, the results are subtly different from the results of the chroma bug. From the viewer’s perspective, though, they look pretty much the same. Let’s call this second issue the 4:2:0 interlaced chroma problem (ICP).

All DVD players suffer from this second problem, and all MPEG decoders. On static interlaced scenes, the chroma channel is all messed up. And there’s nothing that can be done in the MPEG decoder to solve the problem, because it can’t know which parts of the scene are moving and which parts are static.

The obvious fix, one would think, would be to always use the progressive upsampling algorithm. But that causes more problems than it solves. On static scenes where nothing is moving, everything looks wonderful. But as soon as something moves, a strange artifact appears. The chroma information from field 1 leaks into field 2 and vice versa on every pair of fields, and the result is a bizarre pre- and post-echo in the chroma channel. On slow movements, it manifests as a smearing of all the chroma information, and on fast movement it looks like people and objects are split into two parts – a high-resolution luma object and a low-resolution chroma object. It’s difficult to describe. We were unable to get a screen shot for this article, but trust us when we say it’s just as distracting as the chroma bug, if not more so.

So what can be done about this second problem? In the MPEG decoder proper, very little. Very good upsampling with multi-tap FIR filters makes the problem less nasty, but it doesn’t go away completely. The MPEG decoder doesn’t have the information necessary to fix the problem. But in the deinterlacer, there is hope. Read on for more.

How to Fix the Problems (This Section is for the DVD Industry Engineers)

As we’ve seen, the basic chroma bug results from always using the interlaced upsampling algorithm. The fix for this should be simple: use a progressive algorithm that interpolates smoothly across the whole frame when the progressive_frame flag is set to “true,” and an interlaced algorithm that interpolates each field separately when the progressive_frame flag is set to “false.” There’s a small issue, though: the alternating progressive_frame encoding problem. This is a problem where the progressive_frame flag alternates between true and false every frame, when it really should have been set to “true” all the time.

There is a lot of content that is affected by this issue, because it’s caused by a dumb decision by a very big company that made a very popular MPEG encoder. They’ve since fixed the encoder, but there are apparently still authoring companies that use it, because the problem is on a lot of high-profile current discs, like Disney’s “Monsters Inc.” and “Beauty and the Beast”.

It’s not good enough to ignore this problem and chalk it up to an authoring error. First off, it’s not really an error. It’s a perfectly legal MPEG stream. (For more on this, read Part 5 of the DVD benchmark, where we explain why the progressive_frame flag is essentially always optional.) One could argue that it screws up the chroma channel, but the MPEG spec does not require that the chroma channel look good. If encoding companies want to encode progressive content as interlaced, that’s their choice.

Secondly, ignoring this problem produces lousy-looking video, and the user will not blame the authoring company. They will blame the player, and rightly so. Most MPEG decoders handle this issue fine.

The alternating flag issue has a couple of identifying hallmarks: first off, it only happens on 3-2 pulldown material. Secondly, the progressive_frame flag is only set to “true” when the repeat_first_field flag is also set to “true.” Given this, a simple and workable algorithm that will produce the correct upsampling result looks like this (in pseudo-code):

if the current frame’s progressive_frame flag is “true”
use progressive upsampling
else
if the previous frame’s progressive_frame AND repeat_first_field flags are “true”
use progressive upsampling
else
use interlaced upsampling

If the architecture of the MPEG decoder allows it, it’s worthwhile to look ahead to the next frame, as well as the previous frame. If both frames have both of the relevant flags set, then progressive upsampling should be used. However, the above logic will give the same results 99% of the time.

A few decoders only use progressive upsampling on 3-2 material, where the repeat_first_field flag is alternating. On 2-2 progressive material, the decoder switches to interlaced upsampling. We don’t think this is a good idea. There is much more properly flagged 2-2 content than improperly flagged 2-2 content. We’ve seen our improperly flagged 2-2 test material on a player that upsampled it progressively, and it looked fine. Also, in that case, one can blame it on the encoding, because while it’s not incorrect to leave the progressive_frame flag false on progressive content, it is incorrect to set it to true on content that isn’t progressive.

In any event, the most common issues with flags in MPEG streams are where they’ve been left false, not where they’ve been accidentally set to true. MPEG encoders tend to be very conservative with the flags.

Fixing Chroma Problems in a Deinterlacer

As we’ve mentioned before, the Faroudja-designed Sage/Genesis FLI2200 chip hides most evidence of the chroma bug quite well. It also hides the problem mentioned earlier with 4:2:0 interlaced material. Both effects are not by design, but are a side effect of two different circuits in the chip. First off, they only use one field of chroma to create the output chroma channel. This is really a cost saving measure, but it works out OK in most cases because of our eyes’ fundamental insensitivity to chroma. While it works pretty well, we don’t really advocate this approach. When there are sharp edges in the chroma channel, the constant switching between two dissimilar fields produces a flickering effect on the tops and bottoms of sharply defined color fields, such as the ones you find in animation.

The other thing that Faroudja does is a vertical low-pass filter on the chroma channel, deliberately reducing the final chroma resolution by half. This filter is mainly in place to remove chroma artifacts one might find in composite video, like laserdisc or VHS. The fact that it mostly fixes the chroma bug is icing on the cake. But it is this technique, more than any other, that produces the best results.

A vertical low-pass filter turns out to be the simplest and cheapest method of mitigating chroma problems. It doesn’t completely undo all the damage done in the MPEG decoder by the chroma bug, so for this reason we advocate both fixing the MPEG decoder and adding a filter in the deinterlacer. It does, however, completely fix the 4:2:0 interlaced chroma problem (ICP), because in that case all the chroma information is on the correct scan lines, it’s just not interleaved smoothly with the opposite field.

A moment’s thought will demonstrate that a vertical low-pass filter that reduces the effective resolution to 240 lines doesn’t really harm the chroma channel at all. There is only 240 lines worth of chroma information on the disc to begin with. When that 240 lines of information is separated into two 120-line fields, upsampled to two 240-line fields, and interleaved into a single 480-line field, false contour information is introduced into the image, something like the false frequencies that appear above the Nyquist limit when converting digital samples to an analog waveform. In both of those cases, the false information needs to be masked off, and the appropriate technique in both cases is a low-pass filter.

Because of humans’ basic insensitivity to chroma, a super-smooth filter with many taps is not really necessary. A three-tap vertical blur filter with 0.25, 0.5. and 0.25 coefficients will work just about as well as anything. Of course, many of these all-in-one MPEG decoders have amazingly good FIR filters built right in, so doing a final low-pass filter on the chroma channel post deinterlacing is trivial.

It’s important to do the filtering after the deinterlacing is complete, because if the filter is applied before deinterlacing, the artifacts mentioned earlier (where moving objects get chroma “ghosts”) start to crop up. One needs to separate the moving objects in the scene from the static areas. Luckily, that’s exactly what good video deinterlacing does. The deinterlacing algorithms analyze the image and separate the moving components from the static components. The parts of the image that are moving are interpolated from a single field (which effectively removes the chroma issues for those areas), while the static areas are woven from two fields, and hence will have chroma issues that need filtering. The final low-pass filter will clean up those static areas, without doing much at all to the moving areas, which were interpolated from a single field and thus don’t have high chroma detail to begin with.

Some simple deinterlacers just use “bob”, or single-field interpolation. These deinterlacers don’t need to worry about the 4:2:0 flaw, because they never “weave” the chroma fields together. Of course, they don’t deinterlace very well, so the fact that they are missing the chroma artifacts is a sort of Pyrrhic victory.

Other simple deinterlacers use a vertical filter to combine two fields and smooth out the combing artifacts that inevitably result. Some of the same comments from the previous paragraph apply here, but it’s worth noting that they still might get some benefit from filtering the chroma channel all the way down to 240 lines, while using a less aggressive filter on luma.

Wrap-Up

At this point, the problems with 4:2:0 chroma and decoding are well understood. We’ve provided specific techniques to fix these problems, and even some source code. There is no excuse for not fixing them, especially now that DVD players without the error are available for under $70. In the future, we will not be recommending any DVD player for video playback if it suffers from this bug. We also expect all future deinterlacing solutions to offer a chroma filter so that the 4:2:0 flaw (Interlaced Chroma Problem – ICP) can be corrected. If next generation deinterlacing solutions (chips) do not offer this, we will no longer be recommending them.

If you are interested in reproducing an accurate image, and your current DVD player has the bug, we ask that you please contact the manufacturer and let them know you are unhappy that your DVD player is incapable of properly reproducing the bits on the disc.

Already, several MPEG decoder manufacturers have fixed the bug in their chips, almost entirely because of public complaints about these issues. If the player manufacturers perceive this as an important issue, they will lean on the chip makers, and these issues will be fixed. You deserve nothing less for your hard-earned cash.

HERE is a list of DVD players with and without the Chroma Upsampling Error (CUE). (This is not an exhaustive list, and remember, all the players have the Interlaced Chroma Problem – ICP.)

Examples of CUE

Below are six examples that were captured using the the software MPEG decoder we referenced above. We want to thank Ron Economos for helping us capture these images. The MPEG decoder was modified to force correct and incorrect upsampling. If you are using a monitor at high resolution (1600 x 1200), it may be a little difficult to see the error, as the images will be smaller.

“Toy Story” Menu

First up is the legendary “Toy Story”. This shot is taken from the menu found only on the 3-disc set. This menu is actually encoded as 2-2. The film itself is encoded as 3-2.

You will notice there are four close-up shots. The shots on the left are using correct upsampling while the shots on the right are incorrect. The top shots were created using no filtering (worst case scenario), while the shots on the bottom are using a 6-tap FIR filter (best case scenario). We are showing you filtered vs. unfiltered to point out that the is a difference in terms of visibility based on filtering.

“Big Momma’s House”

The first time we saw the chroma streaks on HD content (1080i) was while watching this movie on HBO. At this time we were not sure if the error was caused by improper chroma upsampling (CUE) or the 4:2:0 interlaced chroma problem (ICP). We mention this because the first run of D-Theater tapes are all encoded as 4:2:0 interlaced. This DVD is encoded as 3-2.

“Drop Zone”

The first time we ever saw the chroma problem was with this film. This DVD is encoded as 3-2.

“Ghost World”

You may notice the common thread between the images, as the problem is most visible in red and along edges. This DVD is encoded as 3-2, and is good for checking the severity of the problem.

“Monsters, Inc.”

Animations are always great at showing the CUE, but we are not using “Monsters, Inc.” because it is an animation. We are using it because this is an example of 3-2 encoding that suffers from the alternating progressive_frame flag that early C-Cube encoders suffer from.

“Moulin Rouge”

The chroma error actually shows up more often in this film than it does in “Toy Story”. The red jacket shown here is an awesome chroma bug test pattern. You can see the streaks all throughout the image, not just in his jacket. The problem is the jacket is so severe that it draws your attention away from the rest.

Last Updated: 1/1/03

“Big Momma’s House” Image Copyright 2000, Fox
“Drop Zone” Image Copyright 1994, Paramount
“Ghost World” Image Copyright 2001, MGM
“Monsters, Inc.” Image Copyright 2001, Pixar and Disney
“More Tales of the City” Image Copyright 1998, Showtime Networks Inc. and DVD International
“Moulin Rouge” Image Copyright 2001, Fox
“Toy Story” Images Copyright 1995, Pixar and Disney

Note: We have heard of some corporations using our articles and graphics to teach courses to their staff without authorization. Secrets articles and graphics are original and copyrighted. If you wish to use them in your business, you MUST obtain permission from us first, which involves a license. Please contact the editor.