Solution: Over-processed, blurry, smooth looking video from ATI 3870 HD with DXVA

[bastian74]

Someone in a rarely-used ATI user forum posted a solution to the poor quality video I've been getting from my ATI video card and output from the HD-PVR h.264 video files with DXVA enabled. If I dissabled DXVA then the texture quality improved but motion compensation was not nearly as good, and of course CPU usage was high.

Strangely, this did not seem to affect BD playback, but for files recorded from the HD PVR even at full quality the results I was getting were terrible. It was as if someone took each frame in photoshop and chose "de-noise" with the maximum setting. Texture was completely wiped out from people's skin, 5 0'clock shadow looked like a smooth tan, etc.

The improvement in picture quality when this TRDenoise setting was turned on and forced to 0 was night and day.

If you have an ATI card and use DXVA I highly suggest you experiment with setting TRDenoise to 0. (Reboot and confirm that it is still set to 0)

In the tool mentioned below the setting shows up when you enabled "all" options in the advanced menu, and it is called TRDenoise exactly.

Anyways the solution was posted here:
http://forums.amd.com/game/messagevi...hreadid=102638

My original complain and screen capture example are here:
http://forums.amd.com/game/messagevi...VIEWTMP=Single

And a mirrored copy of the text from the link is here:

The problem is excessive temporal denoise processing.

This typically occurs when temporal denoise is performed at multiple stages along the video pipeline such as the hardware set top box itself (STB), the TV Tuner card, combined with the video card.

The best work around IMO is to disable temporal denoise at the TV Tuner and if possible, at your hardware STB.

The easy work around is to disable temporal denoise at the ATI video card.

This can be done by adding the "TRDenoise=0" registry entry to the DXVA video section of the registry. Easiest way to do this is to download and run "DXVAChecker" (from the Japanese site http://translate.google.com/translat...23DXVAChecker; program is english), right click on anything and get into the advanced video acceleration settings. In there enable "TRDenoise" and set the value to 0. If you are running Vista, restart your application. If running XP, reboot.

ATI, if you are reading this, please add control of the TRDenoise registry entry from within CCC.

[Wirenut]

Uh, the new Cats have this setting in XP and Vista.

And.. you're right. Turn off TRDenoise and you'll get a great picture. I still use a little edge enhancement to sharpen the picture. ...which is also available in the Catalyst Control Center... under the advanced dialogue.

Still worth posting.

Wirenut

[bastian74]

I have the latest prod drivers and didn't see it

UPDATE: Ahh, I see it now. I did not even notice that under ATI CC Advanced -> AVIVO Video -> All Settings there is a vertical scroll-bar.
The page looks fixed and seamless, not like the kind of page you can scroll up and down on. The settings were hidden way down.

[hammers929]

Very good post.. never spotted these settings before in ccc.

[bastian74]

FYI the guy on the ATI forum who helped me find the temporal denoise settings said the slider in the control center is not the same denoise. Read below:

Originally posted by: bastian74
What is the TRDenoise even supposed to do? I didn't seem to LOOSE any quality by turning it off. Everything is better all around.

UPDATE: Ahh, I see they've added it now. I did not even notice that under ATI CC Advanced -> AVIVO Video -> All Settings there is a vertical scroll-bar. The page looks fixed and seamless, not like the kind of page you can scroll up and down on. The settings were hidden way down. Denoise and sharpness, though not color.


The TRDenoise registry setting allows you to completely turn off ATI's temporal denoise processing. Temporal denoise is where multiple video frames are analyzed and by comparing stationary and moving portions of the image from multiple frames, noise is removed. If you were watching a video source where there was a lot of noise in the image, you would find it looks much better with temporal denoise on (even DVDs often look better with it on).

The slider you are talking about controls spacial denoise, not temporal denoise. Spacial denoise is where a single frame is processed to "smooth out noise". Having said that, I'm not certain that this slider doesn't have a small amount of control over temporal denoise (but looking at the results it doesn't seem to).

Ideally what would be nice to have is separate slider for temporal denoise and another for spacial denoise. Most likely the current implementation doesn't allow a slider for temporal so probably the best we could ask for right now is a slider for spacial denoise and an off/off toggle for temporal denoise.

You might want to experiment a little with the denoise slider and edge enhancement with TRDenoise set to 0 (disabled) and again with it set to 1 (enabled). BTW, these sliders mainly affect SD video, not HD.

[alfi33]

Does this setting (or any of the settings under under ATI CC Advanced -> AVIVO Video, for that matter) apply if you are not using AVIVO as your decoder?

[bastian74]

The setting applies if you are using DXVA (hardware movie acceleration)

[Wirenut]

Most interesting. Did not know that. I will try the Registry tweak and see. I stopped applying the Reg tweak after the slider appeared in the Cats.

Makes sense.

Wirenut