Before anyone gets upset, I am making this post just for fun. This is NOT a real look into the way tensor cores work. I am just here to make fun of NVidia for a few minutes. Yes, I placed this in the Gaming section, because DLSS and NVidia are all about gaming. If anyone wants me to move this to the General Discussion forum, I absolutely will. Also, be aware that forum limitations will shrink all of the pictures to the same size. Each image is hotlinked so you are able to see them in full resolution. This is the same technique that NVidia used when UL Benchmarks introduced the
Port Royal DLSS Demo.
I wanted to do a really good look into how Tensor Cores render the image for DLSS and I did a lot of research. I haven't quite figured out RTX, so I won't go into that, but NVidia has really been driving DLSS as this revolutionary anti-aliasing process that is going to change how 4k gaming works. Now, the image I found wasn't at 4k, so I had to make do with what I could.
At this point, we know that Jensen Huang stood on stage and pushed DLSS as the next revolution in anti-aliaising, but he didn't go into the specifics because.. well.. why would any CEO start by telling you his company is working on a way to remove nearly/over half of your pixels from the image in favor of a few more FPS? He wouldn't even show FPS numbers, or full speed game play of the games he had on stage that day, so it all seemed a little strange. At this point, we know Digital Foundry did their deep dive into
DLSS with FFXV (YouTube) and showed that when DLSS is used, the image is actually scaled from 4k down to 1440p, then upscaled and reconstructed back to 4k. Well, after tons of research, I have found the answer to DLSS.
With all of the research I could possibly conjure up, what I found was very interesting. Take this image for example (again, note that the forum software is making all of the images the same size):
This image is intended for 1080p, with DLSS enabled. The image is actually 954x633, so just under half the resolution that it is intended for. In total, 603,882 pixels are rendered. 1920x1080 renders in 2,073,600 pixels, so the performance gain from rendering less than a third of the pixels will be massive, but how does it work? Let me show you!
Once the native resolution is diagnosed, and DLSS is enabled, the tensor cores open the image in Microsoft Paint. Yes, that old program most people completely ignore.. this is why windows 10 version 1809 was required, they loaded a new version of paint, even though it looks exactly the same. I pulled this image directly from the tensor core cache to show it to you, since NVidia hasn't shown you before:
Once the Tensor cores load the image into Microsoft Paint, the magic happens. The Tensor cores expand the image to 200%, making it grainy, but remember, you are rendering 603,882 pixels, not the traditional 2,073,600, so when the image is upscaled and reconstructed, the Tensor cores really hit the gym and start working. Here is the upscaled image directly from the tensor core cache:
After the image is upscaled, you will noticeably see some grainy texture to the image, due to quality loss. Now the Tensor cores MUST apply some "anti-aliasing". At this point, they open an nvidia proprietary smudge tool to blend the missing pixels with the rendered pixels which is why you were required to download the newest driver 418.91, they didn't want this trade secret getting out, so I had to sign some paperwork promising not to show the actual tool itself. Once the proprietary tool is utilized, the image is finally displayed back to your monitor in all of the DLSS glory! Unfortunately, I was only able to pull the image from the tensor core cache to show you, so I don't have the final rendered image without paint available, but here is the final pristine image from the cache:
That is right!!! 1080p DLSS in all of it's glory!
I hope you have all enjoyed this in depth look into how DLSS works. Hopefully you are pleased with the final image, because like NVidia has told the consumer in the last few days, you can have high resolution, crisp images, and lowered performance, or you can have lower resolutions, blurry images, and better performance.. You can have one or the other, but you can't have both.