The below is a summary of a blog post NVIDIA made in October 2024, highlighting the possibilities for autonomous physical AI developed on its systems:
Generative AI has revolutionized digital work, but the physical world awaits its own breakthrough. This is where physical AI, the embodiment of AI in robots and industrial systems, comes in. Now, thanks to three powerful computers working together, physical AI is poised to transform industries like manufacturing, transportation, and logistics.
From Software 1.0 to Multimodal Physical AI
For decades, software development relied on human programmers writing code. Then came the breakthrough of deep learning with GPUs, ushering in the era of Software 2.0, where software writes software. However, current AI models struggle to understand and interact with the real, 3D world.
Physical AI bridges this gap. It utilizes generative AI and multimodal models to equip robots with the ability to perceive, reason, navigate, and interact with their environment. This paves the way for a future where robots seamlessly integrate into our lives.
The Three Computers Powering Physical AI
Developing robots with physical AI capabilities requires a unique approach. Here’s where NVIDIA’s three-computer system comes into play:
- Training: Supercomputers equipped with NVIDIA DGX platforms and NeMo software are used to train powerful foundation and generative AI models. Project GR00T, an initiative focused on these models, allows robots to understand natural language and mimic human movements.
- Simulation: NVIDIA Omniverse, running on OVX servers, provides a virtual world for testing and optimizing robots. Developers can use Isaac Sim to simulate robots and environments, generate synthetic data, and refine robot skills. Isaac Lab, an open-source framework, further accelerates robot learning through reinforcement and imitation techniques.
- Deployment: NVIDIA Jetson Thor computers, designed for on-board processing, deploy the trained AI models onto robots. These models, encompassing control policies, vision, and language capabilities, form the robot’s “brain.”
Building the Future with Autonomous Facilities
The culmination of these technologies leads to the creation of autonomous facilities. Imagine factories with teams of robots working alongside humans, managed through a network of sensors. These facilities will utilize “digital twins” – virtual replicas for layout planning, operation simulation, and most importantly, robot fleet testing.
NVIDIA’s Role in Empowering Developers
NVIDIA’s technology stack empowers a global ecosystem of robotics developers. Companies like Universal Robots and RGo Robotics are leveraging NVIDIA’s solutions to build intelligent robots. Additionally, leading humanoid robot makers like Boston Dynamics and Fourier are utilizing NVIDIA’s platform for development and training.
The era of physical AI is here, driven by advancements in computing power and new approaches to AI development. With its three-computer system, NVIDIA is at the forefront of this revolution, paving the way for a future filled with intelligent robots transforming industries and our world.
See the article here.
