NVIDIA & Developers Pioneer Lifelike Digital Characters For Games And Applications With NVIDIA ACE

NVIDIA ACE (Avatar Cloud Engine) is a suite of technologies that helps developers bring digital avatars to life using generative AI. With ACE, generic non-playable characters (NPCs) can be turned into dynamic, interactive characters capable of striking up a conversation, or providing game knowledge to aid players in their quests.

At CES 2024 we are announcing the availability of ACE production microservices for NVIDIA Audio2Face (A2F) and NVIDIA Riva Automatic Speech Recognition (ASR), and that top digital avatar developers are embracing NVIDIA ACE.

We also released the latest version of the Kairos demo in collaboration with Convai, to showcase how next-generation AI NPCs will revolutionize gaming. Convai is an NPC developer platform that makes it easy for developers to enable characters in 3D worlds to have human-like conversation, perception, and action abilities. 

With ACE, middleware, tool and game developers can take state-of-the-art AI models, such as A2F and Riva ASR, and implement them into their digital avatar solution for games and applications.

Riva ASR enables the player to speak to the AI character via a microphone rather than typing to it. The goal of the ASR model is to enable the gamer to speak casually and naturally, enabling an immersive and dynamic conversation.

Audio2Face creates expressive facial animation for game characters from just an audio source, such as a text-to-speech model. A2F is used both for realtime facial expression and lip sync generation and offline creation of facial performances. Realtime becomes a hard requirement for AI NPCs because of the non-deterministic nature of the NPC’s vocal lines. Alongside freeform AI dialog, developers need models like A2F to generate realistic facial performances.

Top interactive avatar and game developers are pioneering how ACE and generative AI technologies can transform player to character interactions. In addition to Convai, other developers embracing ACE include Charisma.AI, Inworld, miHoYo, NetEase Games, OurPalm, Tencent, Ubisoft, and UneeQ.

“Generative AI-powered characters in virtual worlds unlock various use cases and experiences that were previously impossible,” said Purnendu Mukherjee, founder and CEO at Convai. “Convai is leveraging Riva ASR and A2F to enable lifelike NPCs with low-latency response times and high-fidelity natural animation.”

“This is a milestone moment for AI in games,” said Tencent Games. “NVIDIA ACE and Tencent Games will help lay the foundation that will bring digital avatars with individual, lifelike personalities and interactions to video games.”

“For years NVIDIA has been the Pied-piper of gaming technologies, delivering new and innovative ways to create games. NVIDIA is making games more intelligent and playable through the adoption of gaming AI technologies, which ultimately creates a more immersive experience.” - Zhipeng Hu, Sr. vice president of NetEase and head of LeiHuo business group

Open-ended conversations with NPCs open up a world of possibilities for interactivity in games. However, conversation should have consequences that could lead to potential actions. To carry out actions from NPCs, they need to be aware of the world around them and be able to interact dynamically. With our partner Convai and their latest releases, we take our collaborative demo to the next level, enabling these AI NPCs with the following new features:

• Spatial Awareness - enables game characters to identify and interact with objects and engage in conversations about their surroundings. For example, the player can point at items, and the character will talk about the item intelligently.
• Actions - enable game characters to interact with items in the game world based on the conversation. For example, delivering a bottle of sake when requested.
• NPC-to-NPC Interaction - enables game characters to interact with each other, bringing the world alive even when the player is idle.

In addition, Convai has integrated the new NVIDIA ACE microservices: Audio2Face and Riva ASR. With the addition of the new ACE microservices in Convai’s pipeline, game characters will have improved lip sync, better expression, and accurate speech detection when listening to the player.

Let’s take a look at the new Kairos demo.

Generative AI technologies will transform gaming, enabling NPCs to have meaningful interactions with players beyond the pre-recorded script. Your actions could influence the responses of civilians and key characters, and you’ll get a unique interaction rather than a scripted response.

NVIDIA is at the forefront of Generative AI for all industries, and at CES 2024 NVIDIA announced GeForce RTX SUPER desktop GPUs for supercharged generative AI performance, new AI laptops from every top manufacturer, and new NVIDIA RTX-accelerated AI software and tools for both developers and consumers, full details about which you can find here.

Subscribe to learn more about NVIDIA ACE, future developments, and early access programs. And discover more uses of NVIDIA RTX technology and GeForce RTX GPUs in the latest games by heading to GeForce.com.