NVIDIA RTX Remix To Open Source Creator Toolkit

NVIDIA RTX Remix empowers modders to remaster classic DirectX 8 and DirectX 9 games with full ray tracing, DLSS 3.5, highly detailed assets, and physically accurate materials. Built on NVIDIA Omniverse, RTX Remix consists of a runtime renderer and a toolkit app that facilitates the modding of game assets and materials.

Last year, NVIDIA open sourced the RTX Remix Runtime, allowing modders to expand game compatibility and advance rendering capabilities. Coming later in June, NVIDIA will be open sourcing the RTX Remix Toolkit, allowing modders to streamline how assets are replaced and scenes are relit, increase supported file formats for RTX Remix’s asset ingestor, and bolster RTX Remix’s AI Texture Tools with new models.

In addition, NVIDIA is making the capabilities of RTX Remix Toolkit accessible via REST API, allowing modders to livelink RTX Remix to other DCC tools such as Blender, modding tools such as Hammer, and generative AI apps such as ComfyUI. NVIDIA is also providing an SDK for the RTX Remix runtime to allow modders to deploy RTX Remix’s renderer into other applications and games beyond DirectX 8 and 9 classics.

Since RTX Remix Toolkit launched earlier this year, 20,000 modders have experimented with using it to mod classic games, resulting in over 100 RTX remasters in development on the RTX Remix Showcase Discord. By continuing to open up the RTX Remix platform, we aim to enable and inspire modders across the globe to build even more amazing RTX remasters.

The RTX Remix platform opens up with an open source Toolkit, Remix Runtime SDK, and REST API

Introducing RTX Remix’s REST API

Developers will be able to use the REST API to connect RTX Remix with other DCC applications. While the RTX Remix Toolkit runs in the background, applications can work in lockstep with any of its functions. For example, a developer could connect Blender to RTX Remix, and design a button in Blender that instantly imports an original game asset selected in RTX Remix’s viewport.

Once the asset is redesigned, it can automatically appear in the game once it has been saved in Blender, using RTX Remix REST API to automatically execute the replacement. Alternatively, modders with decades of experience in mod tools like Hammer could build a live link with RTX Remix, allowing them to upgrade Hammer’s asset replacement capabilities with RTX Remix’s instead.

To demonstrate the possibilities of the RTX Remix REST API, we’ve connected RTX Remix with the generative AI ecosystem of ComfyUI. RTX Remix modders will be able to download RTX Remix ComfyUI nodes, enabling modders to accelerate AI art operations significantly.

For small mod teams with modders that lack art experience or time, generative AI can be a life saver. It can help them remaster large swathes of a game’s textures in high resolution and with physically accurate materials that work in concert with RTX Remix’s full ray tracing, saving the modders time that they can spend hand crafting hero assets in impeccable quality. And for larger teams with dozens of artists, AI-generated textures make for perfect placeholders, allowing them to dial in lighting while crafting hand-made replacements for each asset.

Using ComfyUI, which is now accelerated by Tensor Cores on GeForce RTX and NVIDIA RTX GPUs, texture replacement is now significantly faster.

NVIDIA TensorRT accelerates ComfyUI image generation by 60%, making Stable Diffusion workflows even faster on RTX

Through our REST API integration, you can seamlessly export all game textures captured in RTX Remix to ComfyUI and enhance them in one big batch using upscaling or PBR-adding AI models. ComfyUI offers many super resolution and PBR models to choose from, unlocking the capability for 8X or more resolution increases, and PBR models that include metallic and height maps. Additionally, ComfyUI enables modders to use text prompts to generate new details in textures, or make grand stylistic departures by changing an entire scene’s look with a single text prompt.

Examples of workflows supported by Remix and ComfyUI via REST API

There’s nothing quite like selecting a handful of textures in RTX Remix, typing a prompt in ComfyUI, and watching the changes take place in game, to every instance of that asset, without needing to manage a single file.

Test textures from Half Life 2 processed by ComfyUI - click to load a 4K fullscreen comparison

In our test scene in Portal with RTX, we were able to remaster 5 textures throughout the entire game—a task that would have taken days or weeks with traditional mod tools. Some AI Models run even faster yielding results in seconds.

Test textures from Portal processed by ComfyUI - click to load a 4K fullscreen comparison

We expect RTX Remix mods that leverage AI to be made quicker, look better, and adopt many more visual styles than before, thanks to this new integration with ComfyUI and RTX Remix’s REST API.

First image is original Portal, while the next are style changes from text prompts via ComfyUI

Download The RTX Remix Beta Now

If you want to make your own ray-traced mod for a classic game, the NVIDIA RTX Remix Beta is available to download now, with a full set of Tutorial Videos to walk you through the process. The open source Toolkit, RTX Remix Rest API and ComfyUI integration will be updated later in June. Check out ModDB to find the latest RTX Remix mods and visit NVIDIA Studio to learn more about RTX-accelerated software, such as Adobe Substance and Blender.

We can’t wait to see how modders and developers build on top of the fully open RTX Remix platform, and expand the ecosystem of games and applications it works with. For news about future RTX Remix developments, stay tuned to GeForce.com. Also, be sure to check out our other Computex 2024 announcements to see how we’re advancing gaming and creating with AI and NVIDIA RTX technologies.