Japanese Humanoid Robot Learns to Drive a Car, Follow Traffic Rules

Researchers at the Department of Mechano-Informatics, University of Tokyo have developed a humanoid robot that could potentially sit in the driver’s seat of a standard commercial vehicle and operate it safely. This demonstration could be an important step in establishing a middle ground between today’s cars and fully autonomous transportation solutions of the future. The robot, named Musashi, has a “musculoskeletal” structure with limbs and a head, and uses pneumatic actuators to simulate human muscle movements.

The paper’s authors cite previous work including entries to the DARPA Robotics Challenge, which have had specifically designed to drive a car, with specialised limbs and body structures. However, they envision a future humanoid robot that can perform multiple functions including menial housework and carrying heavy items, which means recreating the adaptability of the human body more closely.

The Musashi robot has 74 such “muscles” and 39 joints excluding its hands. Its limbs are constructed using metal, springs, soft foam, and synthetic fibre, as well as sensors that can measure force. Each joint’s stiffness can be adjusted, allowing Musashi to enter and exit the vehicle, control the steering wheel with both hands, pull the handbrake, grip and rotate the key, and operate various controls such as the turn indicator. It can apply force to the car’s pedals and even correct itself if its foot slips and gets stuck behind a pedal.

Musashi’s head has movable “eye” units with two colour autofocus cameras to simulate human abilities. The researchers say they consciously chose not to use radar, Lidar, or other such sensors. It is capable of panning and tilting, and can recognise objects within a wide field of view, including humans and objects seen through the car’s side mirrors. 

On the software side, the researchers focused on quick reflex actions as a safety mechanism. Using its cameras and audio sensors, Musashi is able to learn to recognise objects and conditions, and perform mechanical actions. It can recognise people, traffic signals, and car horns.

A small single-seater Toyota COMS commercial EV was modified for the actual driving experiments, which took place on the University’s campus. Musashi’s “eyes” were not able to read the car’s instruments and it cannot determine speed itself, so it had to be fed this information using sensors on the car. 

The research paper points out examples of Musashi stepping on the car’s brake pedal to stop the car in response to a horn and seeing a human in front of it. The team identified issues with tracking the speeds of other moving cars and turning the car’s steering wheel fully by crossing one hand over another to handle sharp turns. With further development, future robots could enable a reimagining of public transport as well as vehicle ownership. Interestingly, the team behind Musashi also sees it being used as an advanced crash test dummy, due to its humanoid muscle structure, reaction training, and ability to operate a vehicle much like a human.