Introducing Project G-Assist: A Preview Of How AI Assistants Can Enhance Games & Apps

At Computex 2024, we’re exhibiting Project G-Assist, an RTX-powered AI assistant technology demo that provides context-aware help for PC games and apps.

 

PC games offer vast universes to explore and intricate mechanics to challenge even the most dedicated gamer. Many of us spend hours scouring the internet for assistance to learn and master our favorite titles. The amount of knowledge available is immense, spanning millions of guides and countless words.

Project G-Assist aims to put game and system knowledge at players’ fingertips. Simply ask a question about your game or system performance, and the AI assistant provides context-aware answers.

How Gaming AI Assistants Work

Project G-Assist takes voice or text inputs from the player, along with a snapshot of what’s in the game window. The snapshot is fed into AI vision models that provide context awareness and app-specific understanding for the Large Language Model (LLM), which is connected to a database of game knowledge such as a wiki. The output of the LLM is an insightful and personalized response—either text, or speech from the AI —based on what’s happening in-game.

G-Assist’s vision and language models can be customized by developers for a specific game or app, providing a high degree of accuracy and insightfulness.

These custom AI models can then be run on the cloud, or are locally accelerated by GeForce RTX AI PCs and laptops.

Project G-Assist Gaming Demo, Featuring ARK: Survival Ascended

To demonstrate what’s possible with Project G-Assist, we’ve partnered with Studio Wildcard for a tech demonstration using ARK: Survival Ascended, their Unreal Engine 5, open-world, dinosaur-filled survival game.

Activated by pressing a hotkey, or using a wake phrase, the AI Assistant can help players with questions they often find themselves looking up online: whether it’s about quests, items, lore, or difficult to tackle bosses. In ARK: Survival Ascended, it may suggest the best early game weapon and where to find the necessary materials to craft it.

Leveraging AI vision models to understand what is happening in the game window, the AI Assistant can advise whether an on-screen dinosaur should be avoided, or different approaches to taming a particular beast.

And, because the assistant is context aware, it can personalize its recommendations to the user’s playthrough: it can analyze skill points, the crafting menu and currently locked engrams, for example, and suggest what to pick next to help players conquer higher-level areas and foes.

Should you wish to learn more about a particular answer, context-sensitive links can direct you to additional information online, from the official ARK wiki in this instance.

In an RPG or level-based game, the assistant could hint at the location of secret items and missable lore, before giving you step by step instructions to some or all, if asked. In an RTS, it could share the recommended early game build order, or resource management strategies to help you get on level terms with long-time players. And in an FPS, you could instantly discover the current loadout meta, along with all recommended attachments, leveling the field and ensuring you’re not at a competitive disadvantage.

These are just a few of many possibilities.

Project G-Assist Performance Tuning

Modern games demand peak performance from our systems for smooth, immersive gameplay. But determining the best PC and game settings for your specific setup often isn’t straightforward, and can require a ton of time-intensive tweaking and testing.

Project G-Assist can also be extremely useful in performance tuning. To demonstrate, let’s check out a GeForce RTX PC playing CD PROJEKT RED’s critically-acclaimed game, Cyberpunk 2077.

With a simple prompt, Project G-Assist can evaluate your system’s configuration and performance, and instantly tune it for an optimal experience. Apply NVIDIA app’s optimized game settings, based on extensive testing on thousands of hardware configurations; enable Performance Tuning to apply a safe GPU overclock; switch on NVIDIA Reflex to reduce system latency for more responsive gaming; plus so much more.

The AI Assistant can track and graph frame rates, power usage, PC latency, and other hardware statistics, suggesting ways to improve them when prompted.

It can intelligently find ways to increase performance per watt, such as dynamically undervolting the GPU while respecting a frame rate target. For laptop users, power usage is of particular importance, determining how long they can play before their battery is depleted.

There are dozens of game and hardware settings spread between game menus, apps and control panels. The AI Assistant can scan your system to see if there’s anything out of order, such as not taking full advantage of your display’s resolution, refresh rate, or other capabilities, including HDR and G-SYNC.

It can also describe what settings do, and what the expected impact on performance is. For instance, it can explain the difference between DLAA and DLSS, and between Ultra Low Latency Mode in the NVIDIA Control Panel and NVIDIA Reflex in-game.

These questions and more can be quickly answered with AI Assistants.

Project G-Assist: A Glimpse Into The Future

We envision AI assistants transforming the way we engage with our favorite games and apps. Imagine getting instant context-aware assistance with a complex creative workflow in a photo or video app. Or an AI coach analyzing your multiplayer replays, teaching you strategies to move up the ranks in competitive matches. Project G-Assist is a glimpse into that future.

Project G-Assist won’t play a game for you like we suggested in our prophetic 2017 April Fool’s video, but it can help you get more out of your favorite games, and find help for a troublesome boss. Takeout ordering, however… we could definitely add that to the roadmap.

Head over to GeForce.com to learn about our other Computex 2024 announcements, and stay tuned to hear about the AI innovations we’re working on next. If you’re a developer interested in learning more about Project G-Assist, sign up here.