Hi, I’m Andrew Edelsten, Technical Director of Deep Learning at NVIDIA. I’ve been working here since 2010, and for the last couple of years my team has been working with the folks at NVIDIA Research to create DLSS.
This week, we’re excited to launch DLSS for BattlefieldTM V and Metro Exodus, following launches in Final Fantasy XV: Windows Edition and 3DMark Port Royal. There have been a lot of questions, and I wanted to get some answers out to you on the most popular ones.
Q: What is DLSS?
A: Deep Learning Super Sampling (DLSS) is an NVIDIA RTX technology that uses the power of AI to boost your frame rates in games with graphically-intensive workloads. With DLSS, gamers can use higher resolutions and settings while still maintaining solid framerates.
Q: How does DLSS work?
A: The DLSS team first extracts many aliased frames from the target game, and then for each one we generate a matching “perfect frame” using either super-sampling or accumulation rendering. These paired frames are fed to NVIDIA’s supercomputer. The supercomputer trains the DLSS model to recognize aliased inputs and generate high quality anti-aliased images that match the “perfect frame” as closely as possible. We then repeat the process, but this time we train the model to generate additional pixels rather than applying AA. This has the effect of increasing the resolution of the input. Combining both techniques enables the GPU to render the full monitor resolution at higher frame rates.
Q: Where does DLSS provide the biggest benefit? And why isn’t it available for all resolutions?
A: The results of DLSS vary a bit, because each game has different characteristics based on the game engine, complexity of content, and the time spent on training. Our supercomputer never sleeps, and we continue to train and improve our deep learning neural network even after a game’s launch. When we have improvements to performance or image quality ready, we provide them to you via NVIDIA software updates.
DLSS is designed to boost frame rates at high GPU workloads (i.e. when your framerate is low and your GPU is working to its full capacity without bottlenecks or other limitations). If your game is already running at high frame rates, your GPU’s frame rendering time may be shorter than the DLSS execution time. In this case, DLSS is not available because it would not improve your framerate. However, if your game is heavily utilizing the GPU (e.g. FPS is below ~60), DLSS provides an optimal performance boost. You can crank up your settings to maximize your gains. (Note: 60 FPS is an approximation -- the exact number varies by game and what graphics settings are enabled)
To put it a bit more technically, DLSS requires a fixed amount of GPU time per frame to run the deep neural network. Thus, games that run at lower frame rates (proportionally less fixed workload) or higher resolutions (greater pixel shading savings), benefit more from DLSS. For games running at high frame rates or low resolutions, DLSS may not boost performance. When your GPU’s frame rendering time is shorter than what it takes to execute the DLSS model, we don’t enable DLSS. We only enable DLSS for cases where you will receive a performance gain. DLSS availability is game-specific, and depends on your GPU and selected display resolution.
Q: Some users mentioned blurry frames. Can you explain?
A: DLSS is a new technology and we are working hard to perfect it.
We built DLSS to leverage the Turing architecture’s Tensor Cores and to provide the largest benefit when GPU load is high. To this end, we concentrated on high resolutions during development (where GPU load is highest) with 4K (3840x2160) being the most common training target. Running at 4K is beneficial when it comes to image quality as the number of input pixels is high. Typically for 4K DLSS, we have around 3.5-5.5 million pixels from which to generate the final frame, while at 1920x1080 we only have around 1.0-1.5 million pixels. The less source data, the greater the challenge for DLSS to detect features in the input frame and predict the final frame.
We have seen the screenshots and are listening to the community’s feedback about DLSS at lower resolutions, and are focusing on it as a top priority. We are adding more training data and some new techniques to improve quality, and will continue to train the deep neural network so that it improves over time.
Q: Why don’t I just use upscaled TAA instead?
A: Depending on the resolution, quality settings, and game implementation, some may prefer TAA in one game and DLSS in another.
The game industry has used TAA for many years and we know that it can fall down in certain ways. TAA is generated from multiple frames and can suffer from high-motion ghosting and flickering that DLSS tends to handle better.
Q: When’s the next DLSS update for Battlefield V and Metro Exodus?
A: We are constantly working to improve image quality. Recently we updated the core of DLSS so that you get the latest model updates the moment you launch your game. So make sure you have our latest Game Ready Driver (418.91 or higher) installed.
For Battlefield V, we think DLSS delivers a big improvement in 4K and 2560x1440 performance -- up to 40% -- for the corresponding quality, but also hear the community. For the next push, we are focusing our testing and training to improve the image quality at 1920x1080 and also for ultrawide monitors (e.g. 3440x1440). The current experience at these resolutions is not where we want them.
For Metro Exodus, we’ve got an update coming that improves DLSS sharpness and overall image quality across all resolutions that didn’t make it into day of launch. We’re also training DLSS on a larger cross section of the game, and once these updates are ready you will see another increase in quality. Lastly, we are looking into a few other reported issues, such as with HDR, and will update as soon as we have fixes.
Thank you for all of your feedback to date, and please keep it coming via our feedback page!