NVIDIA ACE & Digital Human Technologies Showcased In First Game, Mecha BREAK

NVIDIA ACE is a revolutionary suite of digital human technologies that brings digital humans to life with generative AI. Since its debut at Computex 2023 in the Ramen Shop tech demo, ACE’s capabilities have evolved rapidly.

At Gamescom 2024, we announced our first digital human technology on-device small language model improving the conversation abilities of game characters. We also announced that the first game to showcase these ACE and digital human technologies is Amazing Seasun Game’s Mecha BREAK, bringing its characters to life and providing a more dynamic and immersive gameplay experience on GeForce RTX AI PCs.

NVIDIA’s first digital human technology on-device small language model, called Nemotron-4 4B Instruct, is designed for role-playing, with leading retrieval-augmented generation and function-calling capabilities, allowing game characters to respond to gamers, more intuitively comprehend player instructions, and perform more accurate and relevant actions.

Nemotron-4 4B Instruct is available as an NVIDIA NIM for cloud and on-device deployment by game developers. The NIMs are optimized for low memory usage, offering faster response times and providing developers a way to take advantage of over 100 million GeForce RTX-powered PCs and laptops.

Mecha BREAK, developed by Amazing Seasun Games, a Kingsoft Corporation game subsidiary, has implemented the NVIDIA Nemotron-4 4B Instruct NIM running on-device in the first showcase of ACE-powered game interactions. Additionally, NVIDIA Audio2Face-3D NIM and Whisper, OpenAI’s automatic speech recognition model, provide facial animation and speech recognition running on device, while Elevenlabs powers the character’s voice through the cloud.

Mecha BREAK is a multiplayer mech game that allows players to choose from diverse mechs, customize appearances, and battle colossal war machines on treacherous terrain. In the ACE showcase demo, players can interact via natural language with game characters, most notably the player’s mechanic. Ask for advice on objectives, the ideal mech for the task, and more, before having their mech’s paint job quickly updated for maximum battlefield bling.

“We’re excited to showcase the power and potential of ACE NIMs in Mecha BREAK, using Audio2Face and NVIDIA Nemotron-4 4B Instruct to dramatically enhance in-game immersion,” said Kris Kwok, CEO of Amazing Seasun Games.

Mecha BREAK will launch in 2025, players can sign up for future playtests on the game’s Steam page. For a look at its action-packed gameplay enhanced with day one NVIDIA DLSS 3 support, check out our DLSS 3 gameplay reveal:

Perfect World Games Explores the Latest Digital Human Technologies

NVIDIA ACE and digital human technologies continue to expand their footprint in the gaming industry. Global game publisher and developer Perfect World Games is advancing its NVIDIA ACE and digital human technology demo, Legends, with new AI-powered vision capabilities.

Within the demo, the character Yun Ni can see gamers and identify people and objects in the real world using the computer’s camera powered by ChatGPT-4o, adding an augmented reality layer to the gameplay experience. These capabilities unlock a new level of immersion and accessibility for PC games.

NVIDIA ACE: Bringing Digital Characters To Life

The capabilities of ACE, NVIDIA NIMs, and our supporting technologies are constantly evolving, helping developers create richer, more immersive experiences, and smarter customer-facing tools. Learn more about NVIDIA ACE, future developments, and early access programs here. And stay tuned to GeForce.com for future updates about ACE adoption in games.

In the meantime, head here to learn about our new Gamescom 2024 announcements, including the reveal of 20 new games launching in the coming weeks and months that leverage DLSS and other RTX technologies to give GeForce RTX gamers the definitive experience. And we made several other announcements, too. Head here to get all of the details.