NVIDIA Broadcast Features Now Integrated Directly In Top Streaming Applications

With the recent launch of GeForce RTX 30 Series GPUs we introduced the NVIDIA Broadcast app. It upgrades any room into a home broadcast studio by transforming standard webcams and microphones into smart devices through the power of AI.

The app provides Noise Removal, Virtual Background and webcam Auto Frame features to gamers, streamers, and those working from home. The positive feedback we’ve received so far has been overwhelming.

Today, the NVIDIA Broadcast Engine — a collection of SDKs with the underlying AI technology that powers these features — is being released broadly to our community of developers. This will allow app developers to implement AI effects easily while unlocking new use cases.

Three apps have adopted features from the NVIDIA Broadcast Engine: XSplit, Notch, and Xaymar, with a fourth, NewTek, coming soon.

XSplit has integrated the Audio Effects SDK into XSplit Broadcaster, the popular live streaming client, to enable their users to apply noise removal per audio source. The latest version can be downloaded here.

“NVIDIA Broadcast Engine allows developers to tap into NVIDIA's RTX-powered AI effects and the XSplit team has been working closely with NVIDIA to get these enhancements into the hands of our users. In particular, we've integrated their Noise Removal technology from their Audio Effects SDK, enabling broadcasters to consistently produce a clean and noise-free stream for their viewers. We are committed to continuously bring value to our customers and look forward to future collaborations with NVIDIA in their innovations.” - Miguel Malvar Molina - Director of Developer Relations, XSplit

Notch is using the AR SDK to enable advanced Face Tracking, available widely later this month, providing face tracking driven animation and advanced video effects with greatly reduced processing times while improving quality and stability compared to old CPU-based solutions. Tensor Cores achieve low latency and high responsiveness in face tracking allowing for use during live broadcasts.

Words don’t do it justice, just take a look at the video below.

NVIDIA’s AR SDK in Notch Enabling Advanced Face Tracking

“The speed and quality of the new (and easy-to-integrate!) AR SDK face tracker is incredible. Through the use of dedicated Tensor Cores, Notch users have access to sub-frame accurate face tracking with high-resolution cameras in a live environment. We're excited to see what this, as well as other NVIDIA Broadcast features, will bring to the live show and virtual production environment.” - Matt Swoboda, Founder and Director at Notch

Xaymar, the creator of the popular StreamFX plugin for OBS Studio that already uses the AR SDK, has now added Noise Removal to a VST3 filter with VoiceFX. This allows creators to use NVIDIA’s Noise Removal feature to remove background noise directly from video and audio editing applications such as Adobe Premiere Pro and Adobe Audition.

"The Audio Effects SDK is a game changer, allowing us to easily integrate Noise Removal into the VoiceFX VST3 plugin. Creators now have a simple filter to remove unwanted noise using popular apps such as Adobe Premiere Pro or Adobe Audition." - Michael 'Xaymar' Dirks, Author of VoiceFX

NDI, a part of the Vizrt Group, has integrated the Video Codec SDK into NDI, their popular solution to transmit video in real-time across a local network, replacing the need for a capture card. NDI now provides NVIDIA GPU acceleration for screen capture and encoding. By offloading the encoding to the GPU, it reduces the performance impact to the CPU to almost zero, maximizing the quality of gaming or broadcast production while also producing and sharing content. With this update, users will be able to transfer frames to their stream PC at up to 4K 120 FPS! The latest version is in development and coming soon, learn more here.

"Our mission is to make IP video easy to use and fun for anyone to make a great show," said Dr. Andrew Cross, head of Research and Development for the Vizrt Group and NDI®. “Offering full NVIDIA GPU accelerated support for NDI allows gamers to use applications like OBS Studio to share gaming experiences with higher quality, performance, and frame-rates than ever previously possible."

To try out the new integrations:

• Update your app to the latest beta versions.
• For XSplit, Notch and VoiceFX, you’ll also need to download the SDK redistributable files from our SDK Resources page.

Most importantly, we want to hear from you on how these improvements are working!

After using the new AI features, let us know what you think by sending us feedback. You can also send us a quick clip that’ll help us train and improve the AI networks.

Stream on!