Why AI’s next frontier is learning to grip a tomato

A robot hand picks up a tomato, cracks an egg and lifts a fragile lab instrument. These tasks, featured in a Genesis AI video, are ones a child can do without thinking and ones that machines historically couldn’t do at all. Now startups are betting they can, and money is pouring in.

For the past three years, AI largely lived inside laptops and phones through chatbots, coding assistants and image generators. Companies such as Genesis AI, Tesla and Figure AI are now racing to build what they call “physical AI,” systems capable of understanding and manipulating the real world through robots. Genesis AI made its bid in early May, unveiling a robotic foundation model it calls GENE-26.5 and pitching it as the first AI brain to give robots human-level physical manipulation. Researchers at organizations, including IBM, are exploring related work around simulation, physics-based models and embodied intelligence.

“What is so special about the moment today is the technology,” Vivian Sun, Vice President of Commercial and Strategy at Genesis AI, told IBM Think in an interview. “The transformer-based foundation model is now giving us a new possibility of robots.”

Traditionally, these machines performed narrow, preprogrammed tasks. “Robots have been here for decades, with industrial robots, Roombas, etcetera, doing pretty particular tasks that we program them to do,” Sun said. The new wave of robotics reverses that. Inside labs and Silicon Valley boardrooms, executives are talking about bringing AI into factories, warehouses, hospitals, restaurants and homes.

Cooking dinner illustrates the challenge. A robot must grip slippery vegetables, crack fragile eggs, manipulate soft cloth, handle transparent containers and coordinate dozens of actions in sequence without failure. Humans do those tasks instinctively. Machines have historically struggled with them. But Genesis AI has been showing progress with those problems, releasing demonstrations of robots preparing smoothies, solving Rubik’s Cubes in midair and cleaning surfaces with towels.

“The brain and hand are the two most valuable and complex pieces of robotics, and today we are presenting the industry’s most advanced versions of both,” Zhou Xian, Cofounder and CEO of Genesis AI, said in the company announcement.

Much of the difficulty lies in the complexity of the tasks. Industrial robots usually repeat the same movement endlessly in carefully controlled settings, while human environments constantly shift. Kitchen counters are cluttered. Fabrics wrinkle and liquids tend to spill. Roboticists describe the harder feats as “long-horizon tasks,” meaning activities that require many coordinated steps over extended periods.

“If you saw the cooking case, it is ultra-long horizon with more than 20 subtasks,” Sun said.

Soft materials remain especially tricky, since they continuously change shape. Transparent objects can confuse computer vision systems, which struggle to interpret edges and depth.

The recent excitement around robotics traces largely back to transformers, the neural network architecture behind modern generative AI. The new wave of automation champions believes the same systems powering chatbots could help robots understand motion, predict outcomes and coordinate physical actions.

“Now robots can ingest a massive amount of data and have a prior knowledge, just like humans,” Sun said. “As a human, you’re trained to do different tasks, but then you don’t need to be trained to take up objects.”

The greatest obstacle, in Sun’s view, isn’t software, but data. Large language models (LLMs) are trained on vast amounts of internet text, but robotics companies lack anything comparable for physical actions.

“This is the biggest bottleneck that everybody faces,” Sun said.

Sun said Genesis AI has found a partial solution through hardware designed to more closely mimic human anatomy. The company designed a robotic hand modeled on the structure and movement of a human one. Workers wear sensor-equipped gloves while performing ordinary tasks, capturing detailed data about how people grasp, move and manipulate objects. Because the robotic hand closely mirrors a human hand, the company can use that data to train models that transfer those movements more directly to the robot.

Engineers often call the gap between human hands and robotic hardware the “embodiment gap,” and most machines still rely on simple grippers that are poorly suited for delicate manipulation.

“By building a robot hand that’s as close as possible to the human hand, and then [creating] a glove solution, you really create this one-to-one mapping,” Sun said. Genesis AI also draws on footage from head-mounted cameras and internet videos of people at work.

Sun said robotics may eventually follow the same arc as LLMs, where more information and more computing power steadily improve performance.

“We have discovered the scaling law in our current training,” Sun said. “The more data we provide, the fidelity of the model, the performance of the model and then the task execution all have a correlation with the size of the data.”

Interest in physical AI now stretches well beyond the startups. IBM researchers recently partnered with the Italian race-car manufacturer the Dallara Group to develop physics-based AI capable of accelerating the aerodynamic simulations used in vehicle design. Using simulation data, the companies trained a model that can evaluate airflow around race cars far faster than traditional computational fluid dynamics software does. They reported that it cut some simulations from hours to roughly 10 seconds while maintaining similar accuracy.

“The coordinates of Boston and New York may vary depending on whether you use GPS or a 2D map, but the actual physical distance between the two cities does not,” Mattia Rigotti, one of the IBM researchers behind the work, said in an article on the Dallara work. “Physical relationships, whether it’s distances between cities, or the structure of points and links in a graph, are gauge-invariant.”

Simulation has become another central piece of the push. Virtual environments let robots rehearse tasks millions of times before entering the physical world, and Genesis AI says its platform narrows the so-called “sim-to-real gap,” where machines perform well in simulation but fail once exposed to real conditions.

Factories and warehouses may serve as the first proving grounds. Such settings are more controlled than homes or restaurants, and they already face labor shortages and pressure to automate repetitive work.

“There is actually a USD 20 trillion economy based on labor,” Sun said. “We have twice as much physical labor as the mental labor that the LLM world is trying to tackle.”

The company expects industrial deployments to come first, with robots moving into service industries and homes later. “We would say the industrial use cases will be where we would start,” Sun said. “In the future, we see the robot going to the service industry, going to the Starbucks of the world, to become one of [the] critical parts of our society.”