Cruise Knew Its Self Driving Cars Couldn’t Recognise Children

In a startling revelation, documents reviewed by The Intercept have shown that Cruise, a self-driving car company backed by General Motors (GM), has been running its autonomous taxis despite knowing that the vehicles had difficulty recognising children on the roads. Cruise has been beta testing its robotaxis in some US cities.

Companies in the autonomous driving sector, such as Cruise and the Google-backed pioneer Waymo, assert they can replace ride-hailing entities like Uber, Ola, and Lyft with their robot cars, which operate independently of human drivers. It has long been believed that robots will be safer than human-driven vehicles.

However, this confidence was undermined when, on 17 October, the NHTSA in the US announced an investigation into the 600-vehicle fleet operated by Cruise due to risks posed to other vehicles and pedestrians. A week later, in San Francisco, where Cruise has been operating driverless cars since 2021, the company suspended its service following a series of malfunctions and accidents. The most recent incident saw a Cruise robot taxi strike a pedestrian, dragging her 20 feet along the road.

In the wake of this incident, Cruise also suspended its service in Phoenix and Austin. Consequently, the only operational Cruise vehicles are those with human drivers.

Subsequently, it has been disclosed that internal chat logs reviewed by The Intercept allege Cruise was aware of two critical safety issues: driverless Cruise vehicles struggled to detect large potholes and had so much trouble recognising children that they risked colliding with them.

"We have the lowest risk tolerance for contact with children and treat them with the highest safety priority. No vehicle — human-operated or autonomous — will have zero risk of collision," Cruise stated.

GM's acquisition of Cruise for $1.1 billion in 2016 remains a loss-making venture as driverless technology is perfected. These disclosures could threaten the business, particularly as GM is also heavily investing in electric vehicle technology as part of a global move towards sustainable mobility.

"Cruise AVs may not exercise additional care around children," reads one internal safety assessment. The company’s robotic cars "still need the ability to distinguish children from adults so we can display additional caution around children," said another assessment disclosed by The Intercept.

Its internal documents added that its software lacked a VRU classifier, which is machine learning software that could automatically detect child-shaped objects around the car and manoeuvre appropriately. Internally, Cruise was attempting to use human workers to manually offset this deficiency, as its software was unable to do it automatically.

"It is inaccurate to say that our AVs were not detecting or exercising appropriate caution around pedestrian children," Cruise retorted.

Cruise claimed that it had improved its technology to such an extent that there was only a one in 300 million chance of its fleet accidentally hitting a child, asserting that there had been no on-road collisions with children.

Yet a Cruise spokesperson later stated that the driving software had not failed to detect children but had failed to classify them as children.

The Intercept also reviewed a video of a Cruise driverless vehicle seemingly failing a test in which it was supposed to detect a hole at a construction site, filled with many workers, despite numerous safety signs. Although it did not fall into the hole, the vehicle only stopped inches from the edge after a worker waved a "Slow" sign in front of the driverless car's windshield.

"Enhancing our AV's ability to detect potential hazards around construction zones has been an area of focus, and over the last several years, we have conducted extensive human-supervised testing and simulations, resulting in continued improvements," said Erik Moser, Cruise’s Director of Communications.

"These improvements include enhanced cone detection, full avoidance of construction zones with digging or other complex operations, and the immediate activation of the AV’s Remote Assistance support/supervision by human observers," Moser added.