An independent developer is building a small autonomous robot, dubbed the Microduck, that can bow to release a cat treat from a 3D-printed compartment. The virtual version was coded and trained in only a few hours thanks to OpenAI’s newest models, which excel at interpreting three-dimensional environments and converting that understanding into executable code and control policies.
OpenAI introduced a model called Astra last September, and researchers at the University of California, Berkeley quickly explored its capabilities. Professor Ken Goldberg and his students fed a video of a real robot into Nvidia’s Newton tool and asked the model to recreate the scene. Goldberg reported that within an hour the system generated a complete 3D simulation, automatically configuring camera angles, lighting, background and material properties.
The same workflow was applied to the Microduck prototype. The creator combined GPT-6 Sol with an agentic framework named Hermes from Nous Research, running the process on an Nvidia DGX Spark workstation. The AI model leveraged Nvidia’s Isaac Sim platform to issue commands, evaluate video feedback, and propose adjustments to the training regimen, dramatically reducing the expertise normally required for robot programming.
Nvidia researcher Jim Fan, who has worked with Goldberg, explained that large language models now function like software agents that can command physical devices. He noted that visual-language-action models enable the same kind of interaction that programmers have with computer applications, but applied to robot actuators. Fan and his colleagues recently released an open-source framework called Aspire, which lets robots discover and apply new skills such as solving a Rubik’s cube with dual arms.
At the Conference on Robot Learning in Austin, Texas, Fan presented a demonstration where Astra controlled a robotic arm to paint an image of the Golden Gate Bridge using nearby brushes and paint. The showcase illustrated how an AI model can orchestrate perception, planning and actuation without extensive human intervention, reinforcing the claim that much of the tedious work of robot training can be automated.
Despite these advances, the Microduck’s vision system remains insufficient for commercial deployment, and reliable operation still demands substantial engineering effort. Nonetheless, the speed and accessibility of AI-driven simulation are attracting interest from manufacturing and e-commerce firms, and hobbyists anticipate a wave of “vibe robotics” where programming robots becomes as straightforward as coding software. If the developer perfects the design, the robot could reliably dispense treats for the cat Leona.