The remote-controlled robot can do puzzles and hugs

The research group led by Stefano Dafara from the Italian Institute of Technology in Genoa (Italy) participated in the international robot avatar competition ANA Avatar XPrize with the iCub3. Scientists have now described the technology and the tasks performed in the journal Science Robotics.

“We present a versatile avatar system that enables people to be embodied in humanoid robots,” the researchers wrote. Information is exchanged in both directions: the operator allows the avatar to walk and make arm and hand movements. He communicates with people through his avatar using his voice and facial expressions, especially the eyebrows, mouth and eyes – the iCub3 is able to blink when exposed to bright light.

Conversely, the operator receives sensory impressions from the robot. Wearing an augmented virtual reality headset. The device shows him camera images of the avatar and allows him to hear recordings from the built-in microphone — all in real time. The headset also records the player's facial expressions and transmits them to the avatar. A full-body suit and special gloves allow the operator to feel the robot's touch. Torque force sensors allow weight to be recorded when an object is placed in the robot's hands.

These sensors are also integrated into the shoes worn by the operator on the walking platform. It measures the force and torque that the actuator exchanges with the ground and thus indicates the running motion to the avatar. However, since balance is always an issue in two-legged robots, the iCub3 has an automatic balancing system that is used, for example, when the robot is transporting a load.

Performances in front of large audiences

While the iCub3 traveled through the Venice Biennale art exhibition in November 2021, it was controlled in Genoa, 290 kilometers away. The operator got impressions of the show and spoke with one of his companions. Finally, the host hugged the robot, and the worker and his avatar exchanged hugs. In June 2022, the iCub3 performed in front of 2,000 spectators at a showcase at the We Make Future Festival in Rimini, Italy. Once again, it was controlled remotely from Genoa, this time at a distance of 300 km.

In the ANA Avatar XPrize science competition that same year, the iCub3 placed second in the semifinals. Among other things, she completed the tasks of placing the puzzle pieces, feeling the surface of the vase and estimating its weight, and attaching the glasses to the cup. In the final in November 2022 in Los Angeles, the iCub3 hit the goal post, fell and was eliminated early. Davara and his colleagues are still proud of their accomplishment: “Our avatar system was the only system to complete the tasks by eventually moving bipedally using a lightweight set of actuation devices consisting of commercial and custom wearable devices,” they wrote in the study.