NVIDIA announces huge GenAI update to Jetson Orin, greatly advancing human-robot interaction

NVIDIA has just rolled out what they're calling their biggest system software update ever to their Jetson ecosystem, which brings generative AI (GenAI) and Metropolis expansion to Jetson Orin and a new ROS 2.0 (Robot OS) for the Isaac robotics platform.

The major focus of this update is to bring about GenAI capabilities to computer vision processing, like what OpenAI did for text and images (ChatGPT and DALL-E). NVIDIA hopes that this will transform the way we interact with autonomous machines as GenAI on edge computing platforms like Jetson Orin will mean faster processing, real-time responsiveness and natural language human-robot interaction.

If you look at the video below, NVIDIA's GenAI-based vision transformer models outperform current CNN (Convolutional Neural Network) models in human and object detection. Zero-shot inference means that the GenAI model is also able to correctly classify objects like type of vehicles and colour without prior learning. Again, this is compared to current CNN models that are highly rule-based and require intensive data labelling and multiple neural networks for different models. With GenAI, development cycles are shorter and natural language input means having the ability to contextualise input and responses. According to NVIDIA, these new GenAI vision models are more flexible and will outperform CNN models in detection, segmentation, tracking, searching and even programming.

.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }

Play

Not only that, NVIDIA claims GenAI processing to be up to 1.7x faster on a Jetson AGX Orin 64GB as compared to an Intel Sapphire Rapids Xeon Platinum 8084+ CPU.

According to Deepu Talla, vice president of embedded and edge computing at NVIDIA, “Generative AI will significantly accelerate deployments of AI at the edge with better generalization, ease of use and higher accuracy than previously possible.” 

Besides adding GenAI, NVIDIA is also rolling out a more extensive Metropolis expansion into Jetson Orin as well. While there doesn't seem to be any new updates to Metropolis' features, this expansion means that more Metropilos services and APIs can be directly performed on Orin without going through the cloud. This update is expected to arrive by Dec 2023.

For developers, the JetPack 6 SDK is now slated for a Nov 2023 release, and one of the biggest quality-of-life improvements would be the wider choice for more Linux kernels and distros.

Our articles may contain affiliate links. If you buy through these links, we may earn a small commission.