Humanoid Robots

There’s a tremendous need for multimodal data to get a rich understanding of robot surroundings. Gathering extensive datasets for robotics is challenging and synthetic data can address this data gap. 

Create: Simulated assets such as visually diverse scenes, objects, and textures using GR00T-Gen workflow let you apply domain randomization, build effective robot policies in simulation, and deploy it in the real world.

Generate: Humanoids can also learn from observations and replication of actions from human demonstrations. With the GR00T-Teleop workflow, users can tap into the Apple Vision Pro to capture human actions in a digital twin. These human actions are mimicked by a robot in  simulation and recorded for use as ground truth. 

The GR00T-Mimic workflow then multiplies the captured human demonstration into a larger synthetic motion dataset.

Compress: The NVIDIA Cosmos Tokenizer suite of neural tokenizers helps you more efficiently process image and video data in real time and simulate future events in an environment. Now, you can maintain higher image quality, produce 8X the compression rate, and perform 12X better than current state-of-the-art methods. NVIDIA Nemo can then be used to train or fine-tune video foundation models.

Humanoid robot grasping functionality requires human-like dextrous object manipulation capabilities that can perform gross and fine-grained dexterous manipulation. GR00T-Dexterity is a suite of models and policies that uses a reinforcement learning-based approach and reference workflows to develop them.

Galbot is using Isaac Lab and Isaac Sim today to build and validate a vast number of grasps and demonstrate zero-shot, sim-to-real transfer.

General-purpose navigation in complex and dynamic environments requires extensive tuning. With the GR00T-Mobility novel reference workflow, you can create a mobility generalist for navigating across varied settings and robot embodiments.

Whole-body control of a humanoid robot can be challenging as it requires both stable dextrous manipulation and robust locomotion. GR00T-Control is a suite of advanced motion planning and control models, policies, and reference workflows to streamline the development of effective control systems for humanoid robots. With GR00T-Control, you can train robust, whole-body motion autonomous policies through imitation learning and demonstrate humanoid whole-body skill learning for dexterous manipulation and locomotion from teleoperated datasets.

To enhance situational awareness and interaction efficiency for humanoid robots, they need to have long-term memory with details about events, spaces, personalized settings, and context-aware responses. 

GR00T-Perception helps make this possible with a suite of perception libraries, foundation models, and reference workflows built using Isaac Sim and Isaac ROS.

The GR00T workflows can be orchestrated with NVIDIA OSMO a cloud-native orchestration service for scaling complex, multi-stage, and multi-container ‌robotics workloads across a hybrid infrastructure.