Digital Noise Reduction on DVD: Background and Examples

by Michel Hafner - mhafner@dial.eunet.ch (22 February 2002) (technical consultant: John G. DeGroof) (inspiration Bjoern Roy) (for a page on edge enhancement click here) (for DVD quality evaluation in general click here)

Introduction

Digital noise reduction (DNR) in the context of DVD mastering and film to digital video transfers (including High Definition) is a process which uses a digital filtering algorithm on the digital image data to reduce the amount of random noise (like film grain, electronic noise of the teleciné, comb filter artifacts in composite video sources, film speckles, dirt, scratches etc.). Why is this desirable, if at all?

To reduce the visibility of the random noise during playback of the video. Obvious film grain, scratches, dirt particles and the like are usually considered unwanted for aesthetic reasons if not annoying, hence the wish to get rid of it. Pleasing customers and avoiding complaints is the purpose here.
To reduce entropy (information content) in the images which allows for more efficient compression at a lower bit rate without unwanted and distracting compression artifacts. Here cost and bit saving is the desired goal. (Entropy can also be removed with a low pass filter that removes all fine detail beyond a certain frequency. This creates no noise reduction artifacts but also removes no noise, just blurrs it like the rest of the image.)
To restore a film on video to a state closer to its original state on film before degradation happened. Film restoration is the goal.

Obviously these goals are not necessarily compatible or only to some extent.

How is it done? Since random noise is random it's generally not possible to perfectly remove it and unveil what has been poluted or erased by the noise. But using the high degree of redundancy present within and in consecutive film/video frames and using knowledge about the nature of the noise (such as its expected value and standard deviation) mathematically derived/optimised filters have been suggested and are used to deal with different kinds of noise. These filters come in different flavours:

Hardware implemented real time filters versus computationally very costly non real time filters
Motion adaptive filters versus non motion adaptive filters
Motion compensated filters versus non motion compensated filters
Filters optimised for scratches, speckles or general random noise

People working with digital noise reduction in the video/film industry usually distinguish between noise reduction, which reduces film grain, electronic noise from sensors and other types of general random noise, and specialised algorithms that go for specific kinds of noise, such as scratch concealment going after scratches and dust busters going after dust particles, speckles and the like. Mathematically speaking it's all digital noise reduction/removal, though. And that's how the term is used on this page.

Motion is very important since the data redundancy comes either from within the same frame or consecutive frames in time. If the redundancy of consecutive frames is to be used one must pay attention to the fact that corresponding parts of the image are not generally at the same place in consecutive images since the represented objects do move in time and space and therefore also on the images. Finding the corresponding image parts by calculating the motion from frame to frame is in general a very difficult and computationally intensive task. Ignoring the motion or getting it not right inevitably creates incorrect filtering results when several frames are combined in a filtering operation. These filtering errors may be visible to the human eye or not, but they usually are, so care is required or random noise is traded in for new random noise generated by the filtering which may be more objectionable than the original noise. Naturally real time filters are limited in their precision and filters ignoring or miscalculating motion will create artifacts on moving image parts. So the best results can be expected from non real time filters with motion compensation. In DVD mastering facilities real time filters dominate that can not estimate motion correctly and therefore have to be carefully tuned to leave moving image parts alone or do not filter them much compared to non moving parts.

Noise Reduction Artifacts on DVDs

The purpose of this page is to document noise reduction artifacts on some DVDs to make a couple of points:

Noise reduction is widely applied to DVDs, like edge enhancement. Affected are DVDs (almost) everyone knows or owns.
Noise reduction adds artifacts to many DVDs that are as objectionable as edge enhancement if not more so in some cases.
Noise reduction can ruin the film look and give a DVD an unnatural digital video look that can be outright ugly.
Noise reduction can make a DVD look worse than it would look without any noise reduction (and compression at a proper bit rate suitable for the amount of noise present).
Noise reduction is visible even on simple regular TVs if you know what to look for. It affects everybody with an eye for quality, not just videophiles in their 'high end ivory towers'.

Noise reduction artifacts are pretty much artifacts caused by incorrect or missing motion estimation or misjudging image details for what they really are (treating window bars or highlights as scratches/speckles, for example). By combining image parts from several frames that do not belong together the following artifacts are usually created:

Image detail in textures that should move naturally starts flickering when moving
Image detail in textures that should move naturally gets smeared and blurred when moving (well beyond motion blur of film frames)
Image detail in textures that should move naturally disappears completely and reappears when motion stops.
Objects with a high contrast to the background create multiple echoes of their edges on this background when in (fast) motion.
Solid objects when moving get suddenly holes, break apart or are eaten into from the edges (such as letters in a text).
Textures look strangely noisy but differently than normal noise from film grain or sensors.
Image parts that belong naturally together (like sky and earth) move independently in different directions. The effect can be a subtle floating or a surreal effect with massive movement of the parts against each other.

Scratch removal attacking real image detail causes flickering as well and removal of parts or all of tiny objects like highlights or window bars. The type of filter used and the parametrisation define which artifacts are created. Median and other order statistic filters tend to create flickering, linear filters tend to smear and create echoes.

Noise reduction can be done during teleciné or before MPEG compression. The latter is far better since the transfer itself is not affected by noise reduction artifacts and the compressionist can use as much DNR as he needs given his bit budget (he might need none). As time goes by and he gets better hardware he can do a better job in subsequent versions of a DVD. Once the damage is done in the transfer there is no way back. You have to retransfer to fix the problem.

Methodology and Examples

Following are 11 examples from 9 DVDs. Presented are a full frame jpeg of the first frame in the animation and 3 animations. All animations are direct digital transfers from the original DVD without any processing except the choice of a suitable window of interest and recompression for animation one and two. Animation one is compressed with MPEG-1, animation two with QuickTime Photojpeg. Animation 3 is uncompressed QuickTime for skeptics who think the problems are caused by recompressing the DVD and need hard evidence that it ain't so. Naturally these uncompressed versions are huge despite lossless compression with gzip. So I do not advise to use them, but if you must you can. I have made sure that all 3 versions of every example are for all noise reduction demonstration purposes the same.

It's best to watch the animations with 24bit frame buffer hardware so full DVD resolution is given, and set the playback speed to normal 24 frames/s since these are progressive frames as on film without interlace motion artifacts. For some examples it might be useful to step through the clip frame by frame as well to see in detail what's going on.

Let's have a look at the examples one by one, shall we?

Terminator 2: Judgment Day (1991) (both NTSC versions, Artisan): Linda Hamilton, chapter 12. Watch how the facial textures look, especially during movement of the head. The effect is unnatural, as if she has two (noisy) skins that are moving a bit against each other. The effect is subtle enough that many won't notice it at all or think it's a MPEG compression issue. Looks rather weird on a big screen. This example shows more the DNR look than one specific artifact you can point a finger at.

Terminator 2: Judgment Day (1991) Still Mpeg-1 (2834048 bytes) QuickTime PhotoJpeg (2359062 bytes) QuickTime uncompressed (26965944 bytes)
Notti di Cabiria, Le (1957) (NTSC, Criterion): Chapter 9, the waiter. This is a rather crass case of flickering and smearing caused by noise reduction. Not subtle at all and easily visible on any kind of display. Black and white stripes like this are pure poison for noise reduction since the high frequencies show any error in motion compensation mercilessly. This should have never passed quality control. There is more of this kind on that DVD which is otherwise a very nice DVD. In this case I happen to know that the damage was done at the teleciné and was not the fault of the compressionist.

Notti di Cabiria, Le (1957) Still Mpeg-1 (1614044 bytes) QuickTime PhotoJpeg (1711606 bytes) QuickTime uncompressed (20092537 bytes)
Cleopatra (1963) (NTSC, Fox): Cleopatra's entrance into Rome. This transfer has been noise reduced throughout and there are artifacts to be found in every scene. Usually the flickering kind. The camera movement causes the horizontal elements in the arch to flicker. Pretty obvious on bigger screens. Too bad since this is a fine DVD otherwise. Ben-Hur is rather clean concerning DNR, Ten Commandments is not.

Cleopatra (1963) Still Mpeg-1 (1479548 bytes) QuickTime PhotoJpeg (1258406 bytes)
Cruel Intentions (1999) (NTSC, Columbia-Tristar): chapter 17 and chapter 24. The first example shows minor flickering of the car grill (and a strange blurring at one point), the second shows some smearing in the pink sweater. Both artifacts are subtle enough that many won't notice them at all. The trained eye sees them easily, though.

Cruel Intentions (1999) Still Mpeg-1 (2770228 bytes) QuickTime PhotoJpeg (1351046 bytes) QuickTime uncompressed (11915898 bytes)

Cruel Intentions (1999) Still Mpeg-1 (733728) QuickTime PhotoJpeg (931374 bytes) QuickTime uncompressed (11360138 bytes)
Hum Dil De Chuke Sanam (1999) (NTSC, DEI): Chapter 3. Indian DVDs are with almost no exception massively noise reduced and there are DNR artifacts galore to be found. The DNR is rather counterproductive as the film artifacts removed are less objectionable than the digital artifacts added. The example shows the complete elimination of sand dune details when the camera moves, and their reappearance. Pretty amazing, no? I appreciate Indian films for their magic, but not this kind, please.

Hum Dil De Chuke Sanam (1999) Still Mpeg-1 (1883260 bytes) QuickTime PhotoJpeg (1655878 bytes) QuickTime uncompressed (17217525 bytes)
Matrix, The (1999) (NTSC DVD, Warner): Chapter 15 and chapter 20. The matrix has been noise reduced by agent Smith. He did it rather carefully, but he won't deceive Morpheus nor Neo. The first example shows some heavy flickering due to (presumably) scratch removal attacking the pseudo scratches in the wall. The second example shows some smearing and loss/erasing of detail in the blue sweater. The first effect is crass when you look at the right place (and not at Morpheus). The second is subtler and easy to overlook. HBO's HD transfer is also noise reduced, by the way.

Matrix, The (1999) Still Mpeg-1 (1698572 bytes) QuickTime PhotoJpeg (1741734 bytes) QuickTime uncompressed (22922889 bytes)

Matrix, The (1999) Still Mpeg-1 (808376 bytes) QuickTime PhotoJpeg (723422 bytes) QuickTime uncompressed (5203046 bytes)
Meet Joe Black (1998) (NTSC DVD): Chapter 3. Look what the DNR is doing to the texture of the jacket. It gets messed up quite some. The effect is not subtle on a big screen. Nonetheless this DVD was applauded as a high quality DVD when it came out. It's full of DNR problems like this. It has the DNR look, as I call it. Not my idea of a high quality DVD.

Meet Joe Black (1998) Still Mpeg-1 (2185492 bytes) QuickTime PhotoJpeg (2038390 bytes) QuickTime uncompressed (25196385 bytes)
Pledge, The (2001) (NSTC, Warner): Chapter 8. This DVD has been called reference quality. I more or less agree. Nonetheless it does have minor DNR artifacts at times. The smearing in the shirt and jacket is subtle and hard to see at normal speed. Certainly not distracting. DNR of this quality is fine with me.

Pledge, The (2001) Still Mpeg-1 (1024988 bytes) QuickTime PhotoJpeg (963246 bytes) QuickTime uncompressed (9397033 bytes)
Witness (1985) (NSTC, Paramount): Chapter 3. This DVD made me very angry when it came out. It's DNR artifact city and has the ugly DNR look I can't stand. The example shows scratch removal attacking the lights in the ceiling. Distracting and entirely avoidable. The DVD has also tons of smearing due to DNR. The HBO HD transfer is pretty much free of these problems and so much better. It would be trivial to make a DVD out of it that blows this one away. An example of how NOT to do DNR on DVD.

Witness (1985) Still Mpeg-1 (5395388 bytes) QuickTime PhotoJpeg (3563302 bytes) QuickTime uncompressed (34719616 bytes)

Conclusion

I hope I have demonstrated a couple of points with this page:

Noise reduction and its artifacts can be found on many DVDs from all labels. It's widespread like edge enhancement.
Noise reduction artifacts go from almost invisible to easily visible.
Noise reduction artifacts come in various forms.
Noise reduction artifacts can become very distracting and the obvious ones have no place on a high quality DVD.
Sloppy noise reduction does not improve image quality, it degrades it, destroys the film look and replaces familiar film artifacts with less familiar digital processing artifacts. A very dubious trade.
The proper way to deal with a bad film master is to search for a better film master. If that's not possible noise reduction should be done with the best algorithms available to minimise artifacts. Simple film grain as authentic attribute of film should not be reduced at the cost of adding artifacts, but compressed with a suitably high bit rate.