MIT develops new technology to solve thorny challenge of robots making pizza dough

A new technique developed by researchers at the Massachusetts Institute of Technology (MIT) could allow robots to manipulate soft objects like pizza dough or soft materials like clothing. It is very simple for a pizza maker to handle the dough. However, handling deformable objects like dough is tricky for robots, because the shape of dough can change in many ways, which is difficult to express in equations. Also, creating a new shape from the dough requires multiple steps and the use of different tools.

Learning a manipulation task with a long sequence of steps is particularly difficult for a robot—there are many possible options—because learning is often by trial and error.

Researchers at MIT, Carnegie Mellon University and the University of California, San Diego have come up with a better way. They created a framework for a robotic manipulation system that uses a two-stage learning process, which enables robots to perform complex dough manipulation tasks over long timescales. A "teacher" algorithm addresses every step the robot must take to complete the task. It then trains a "student" machine learning model to learn abstractions about when and how to perform each skill required in a task, such as using a rolling pin. Armed with this knowledge, the system infers how to perform these skills to complete the entire task.

The researchers showed that the method, called DiffSkill, can perform complex manipulation tasks in simulations, such as dividing and spreading dough, or collecting dough pieces from around a cutting board, while outperforming other machine learning methods.

Aside from making pizza, this approach could be applied to other situations where robots need to manipulate deformable objects, such as nursing robots that feed, bath or dress the elderly or people with motor impairments.

"This approach is closer to the way we humans plan actions. When humans do a long-distance task, we're not writing down all the details. We have a higher-level planner that tells us roughly what the phases are what, and some intermediate goals that we need to achieve along the way, and then we execute them," said Yunzhu Li, a graduate student in MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) and one of the lead authors of the paper introducing DiffSkill.

The "teacher" in the DiffSkill framework is a trajectory optimization algorithm that can solve short-range tasks, where the initial state of an object is close to the target position. The trajectory optimizer works in a simulator that simulates real-world physics (called a differentiable physics simulator, which makes the "Diff" in "DiffSkill" a reality). The "teacher" algorithm uses the information in the simulator to learn how the dough must move at each stage, one at a time, and then outputs these trajectories.

The "student" neural network then learns to imitate the actions of the "teacher". As input, it uses images from two cameras, one showing the current state of the dough and the other showing the dough at the end of the task. The neural network generates a high-level plan to determine how to link different skills to achieve a goal. It then generates specific, short-range trajectories for each skill and sends instructions directly to the tool.

The researchers used this technique to experiment with three different simulated dough manipulation tasks. In one task, the robot uses a shovel to lift dough onto a cutting board and then uses a rolling pin to flatten it. In another task, the robot uses grippers to collect dough from all corners of the counter, place it on a spatula, and transfer it to a cutting board. In the third task, the robot cuts a pile of dough in half with a knife, and then uses a gripper to transport each piece to a different location.

DiffSkill was able to outperform popular techniques that rely on reinforcement learning, where robots learn tasks through trial and error. In fact, DiffSkill was the only method that successfully completed all three dough manipulation tasks. Interestingly, the researchers found that the "student" neural network was even able to outperform the "teacher" algorithm.

"Our framework provides a novel way for robots to acquire new skills. These skills can then be chained together to solve more complex tasks that are beyond the capabilities of previous robotic systems," the researchers said.

Because their method focuses on controlling tools (shovels, knives, rolling pins, etc.), it can be applied to different robots, but only if they use specific tools as defined by the researchers. In the future, they plan to integrate the shape of the tool into the reasoning of the "student" network, so that it can be applied to other devices.

The researchers intend to improve DiffSkill's performance by using 3D data as input, rather than using images that are difficult to transfer from simulation to the real world. They also hope to make the neural network planning process more efficient and collect more diverse training data to enhance DiffSkill's ability to generalize to new situations. In the long term, they hope to apply DiffSkill to more diverse tasks, including cloth manipulation and more.