Nvidia’s EgoScale teaches robots from human POV videos

Nvidia is developing EgoScale, a system that trains robots by learning from large-scale egocentric human video. The dataset includes more than 20,000 hours of POV footage labeled with actions and hand poses, covering tasks such as folding clothes, using tools, and handling everyday objects.

EgoScale uses a vision-language-action model that combines visual input, text instructions, and action outputs to teach robots how to perform complex, multi-step tasks. In demos, robots can follow natural-language commands to manipulate cards, open bottles, or fold shirts based on what the model learned from the POV data.

Nvidia plans to release code on GitHub, which could accelerate research into general-purpose household and industrial robots.