On Robots Killing People

The robot revolution began long ago, and so did the killing. One day in 1979, a robot at a Ford Motor Company casting plant malfunctioned—human workers determined that it was not going fast enough. And so twenty-five-year-old Robert Williams was asked to climb into a storage rack to help move things along. The one-ton robot continued to work silently, smashing into Williams’s head and instantly killing him. This was reportedly the first incident in which a robot killed a human; many more would follow.

At Kawasaki Heavy Industries in 1981, Kenji Urada died in similar circumstances. A malfunctioning robot he went to inspect killed him when he obstructed its path, according to Gabriel Hallevy in his 2013 book, When Robots Kill: Artificial Intelligence Under Criminal Law. As Hallevy puts it, the robot simply determined that “the most efficient way to eliminate the threat was to push the worker into an adjacent machine.” From 1992 to 2017, workplace robots were responsible for 41 recorded deaths in the United States—and that’s likely an underestimate, especially when you consider knock-on effects from automation, such as job loss. A robotic anti-aircraft cannon killed nine South African soldiers in 2007 when a possible software failure led the machine to swing itself wildly and fire dozens of lethal rounds in less than a second. In a 2018 trial, a medical robot was implicated in killing Stephen Pettitt during a routine operation that had occurred a few years earlier.

You get the picture. Robots—”intelligent” and not—have been killing people for decades. And the development of more advanced artificial intelligence has only increased the potential for machines to cause harm. Self-driving cars are already on American streets, and robotic "dogs" are being used by law enforcement. Computerized systems are being given the capabilities to use tools, allowing them to directly affect the physical world. Why worry about the theoretical emergence of an all-powerful, superintelligent program when more immediate problems are at our doorstep? Regulation must push companies toward safe innovation and innovation in safety. We are not there yet.

Historically, major disasters have needed to occur to spur regulation—the types of disasters we would ideally foresee and avoid in today’s AI paradigm. The 1905 Grover Shoe Factory disaster led to regulations governing the safe operation of steam boilers. At the time, companies claimed that large steam-automation machines were too complex to rush safety regulations. This, of course, led to overlooked safety flaws and escalating disasters. It wasn’t until the American Society of Mechanical Engineers demanded risk analysis and transparency that dangers from these huge tanks of boiling water, once considered mystifying, were made easily understandable. The 1911 Triangle Shirtwaist Factory fire led to regulations on sprinkler systems and emergency exits. And the preventable 1912 sinking of the Titanic resulted in new regulations on lifeboats, safety audits, and on-ship radios.

Perhaps the best analogy is the evolution of the Federal Aviation Administration. Fatalities in the first decades of aviation forced regulation, which required new developments in both law and technology. Starting with the Air Commerce Act of 1926, Congress recognized that the integration of aerospace tech into people’s lives and our economy demanded the highest scrutiny. Today, every airline crash is closely examined, motivating new technologies and procedures.

Any regulation of industrial robots stems from existing industrial regulation, which has been evolving for many decades. The Occupational Safety and Health Act of 1970 established safety standards for machinery, and the Robotic Industries Association, now merged into the Association for Advancing Automation, has been instrumental in developing and updating specific robot-safety standards since its founding in 1974. Those standards, with obscure names such as R15.06 and ISO 10218, emphasize inherent safe design, protective measures, and rigorous risk assessments for industrial robots.

But as technology continues to change, the government needs to more clearly regulate how and when robots can be used in society. Laws need to clarify who is responsible, and what the legal consequences are, when a robot’s actions result in harm. Yes, accidents happen. But the lessons of aviation and workplace safety demonstrate that accidents are preventable when they are openly discussed and subjected to proper expert scrutiny.

AI and robotics companies don’t want this to happen. OpenAI, for example, has reportedly fought to “water down” safety regulations and reduce AI-quality requirements. According to an article in Time, it lobbied European Union officials against classifying models like ChatGPT as “high risk” which would have brought “stringent legal requirements including transparency, traceability, and human oversight.” The reasoning was supposedly that OpenAI did not intend to put its products to high-risk use—a logical twist akin to the Titanic owners lobbying that the ship should not be inspected for lifeboats on the principle that it was a “general purpose” vessel that also could sail in warm waters where there were no icebergs and people could float for days. (OpenAI did not comment when asked about its stance on regulation; previously, it has said that “achieving our mission requires that we work to mitigate both current and longer-term risks,” and that it is working toward that goal by “collaborating with policymakers, researchers and users.”)

Large corporations have a tendency to develop computer technologies to self-servingly shift the burdens of their own shortcomings onto society at large, or to claim that safety regulations protecting society impose an unjust cost on corporations themselves, or that security baselines stifle innovation. We’ve heard it all before, and we should be extremely skeptical of such claims. Today’s AI-related robot deaths are no different from the robot accidents of the past. Those industrial robots malfunctioned, and human operators trying to assist were killed in unexpected ways. Since the first-known death resulting from the feature in January 2016, Tesla’s Autopilot has been implicated in more than 40 deaths according to official report estimates. Malfunctioning Teslas on Autopilot have deviated from their advertised capabilities by misreading road markings, suddenly veering into other cars or trees, crashing into well-marked service vehicles, or ignoring red lights, stop signs, and crosswalks. We’re concerned that AI-controlled robots already are moving beyond accidental killing in the name of efficiency and “deciding” to kill someone in order to achieve opaque and remotely controlled objectives.

As we move into a future where robots are becoming integral to our lives, we can’t forget that safety is a crucial part of innovation. True technological progress comes from applying comprehensive safety standards across technologies, even in the realm of the most futuristic and captivating robotic visions. By learning lessons from past fatalities, we can enhance safety protocols, rectify design flaws, and prevent further unnecessary loss of life.

For example, the UK government already sets out statements that safety matters. Lawmakers must reach further back in history to become more future-focused on what we must demand right now: modeling threats, calculating potential scenarios, enabling technical blueprints, and ensuring responsible engineering for building within parameters that protect society at large. Decades of experience have given us the empirical evidence to guide our actions toward a safer future with robots. Now we need the political will to regulate.

This essay was written with Bruce Schneier, and previously appeared on Atlantic.com.

Quoted in “Enterprises will test the limits of LLMs”

A new interesting report by Jon Reed has a lot of thought about use of LLMs in the Enterprise. He included the following comment from me, calling it “spicier” than others:

As per my assertion that ChatGPT won’t adapt well to the enterprise, Ottenheimer responded:

I agree with this line of thinking. Your belief is well-founded in what probably should be described as post-enlightenment thinking. The post-Newton – or perhaps post-Hume – scientific rigor of discarding things that are probabilistically false helps Enterprises navigate towards a progressive upside of profit without causing harm. The very nature of an Enterprise business model is that it operates using efficiencies, better known as regulations, to avoid costly known bad behaviors. Academic approaches to data lean towards overly “wide spectrum” or “both sides” doctrines, dragging everything and anything into view yet eschewing accountability for errors. ChatGPT has had some very problematic missteps and hasn’t yet proven it won’t blindly drive an Enterprise into some wasteful, entirely avoidable accidents and real harm.

MIT Study: RoboTaxis Ruin Cities

From the outset, it was evident to anyone with basic knowledge of urban dynamics that Uber would have negative effects on cities. It’s not a matter of complex calculations; it’s a fundamental principle of urban transportation. Increasing the number of cars on the road leads to more problems.

The same common sense reasoning suggests that RoboTaxis will likely exacerbate these issues.

Even MIT has acknowledged its mistake in endorsing Uber, as they were so juiced on Utopia they failed to recognize the glaring warning signs.

This utopian vision was not only compelling but within reach. After publishing our results, we started the first collaboration between MIT and Uber to research a then-new product: Uber Pool (now rebranded UberX Share), a service that allows riders to share cars when heading to similar destinations for a lower cost.

One thing that really bothers me is when people discuss Utopia like this as if they are working on a tangible reality. The term itself, derived from Latin, essentially means to go nowhere. The term was coined by Sir Thomas More in 1516 in his book of the same name. It is a combination of two Greek words, “ou” (meaning “not”) and “topos” (meaning “place”), which roughly translates to “no place” or “nowhere.” MIT’s “utopian vision” of cars was going nowhere, by definition!

Considering this, MIT might want to consider reimbursing students’ tuition fees if they’re teaching that achieving utopian vision is feasible, and the end is neigh. Honestly, one might as well attend church for free instead of paying stupid money to MIT, if it’s just study of compelling unattainable beliefs.

MIT’s fixation on the dogma of revolution through detached-STEM thinking (pun intended) totally neglected the intricate, real-world dynamics of human behavior within complex systems. This classic mistake is exactly what prompted formation of a Fabian Society in London in 1884, an anti-upheaval (gradualism of social reform) organization that successfully has handled such issues for a significant stretch of time.

In an article brimming with STEM remorse about falling for the false profits of Uber (pun intended), MIT emphatically cautions now against getting involved with RoboTaxis.

Our research was technically right, but we had not taken into account changes in human behavior. Cars are more convenient and comfortable than walking, buses and subways — and that is why they are so popular. Make them even cheaper through ride-sharing and people are coaxed away from those other forms of transit. This dynamic became clear in the data a few years later: On average, ride-hailing trips generated far more traffic and 69% more carbon dioxide than the trips they displaced. We were proud of our contribution to ride-sharing but dismayed to see the results of a 2018 study that found that Uber Pool was so cheap it increased overall city travel: For every mile of personal driving it removed, it added 2.6 miles of people who otherwise would have taken another mode of transportation. As robotaxis are on the cusp of proliferating across the world, we are about to repeat the same mistake, but at a far greater scale.

Ah, the sweet taste of pride. Often followed by a not-so-graceful tumble, right? But seriously, why was MIT “proud of our contribution to ride-sharing”?

The correlation between an increase in subsidized cars and heightened traffic congestion seems like an obvious observation. Individuals from places as distant as Davis (a three-hour journey) were opting to “commute” to San Francisco, not for conventional employment but to essentially circle the city, nap in their vehicles, and contribute to the issue of public defecation while serving as “drivers” for those unwilling to walk. The city’s functionality came to a standstill because Uber failed to consider the overall capacity of people on the streets, which was significantly dwindling.

The issue at hand? Once more, it’s the age-old equation: more cars equals more headaches.

This is wisdom older than your grandma’s secret apple pie recipe. It’s not brain surgery; it’s not even “hey, we’re on the brink of utopia” snake-oil salesmanship as ancient as steam engines.

Taxis, or cars in general, really shine when it comes to serving folks who can’t sprint or comfortably hop onto a bus. But when you attempt to cram everyone into a car, you’re basically signing up for a masterclass in waste and inefficiency.

Hold onto your cabriolet hats, folks! The Uber saga was a spectacular display of wastefulness and inefficiency, a saga foreshadowed by centuries of shared transportation history.

  • Hacker? England 1625 from Hackney (although some claim French haquenee)
  • Cab (Cabriolet)? England 1820 from French cabrioler
  • Taxi? Germany 1895 from French taxe

This is not new stuff. Yet in the 2010s instead of taking a leisurely stroll for a few blocks, people decided to gather on bustling street corners, waiting to congest the roads in colossal, empty, self-centered metal boxes, leaving behind a trail of pollution and traffic snarls that could rival a spaghetti monster convention.

I think it’s pretty amazing that these people without scientific backgrounds — or really any education at all — think they have the right to decide the [transit systems]. And it blows my mind that they are getting away with it.

Picture this: an entire city street, maybe even a whole neighborhood, held hostage by the one guy who’s treating his latte frappuccino like it’s a cauldron of magic potion. Meanwhile, his Uber, just a stone’s throw away, has decided to throw a temper tantrum in the middle of the road, blocking all traffic while screaming at the top of its metallic lungs, “CHAD! Anyone, anywhere, please tell me if you’ve seen a CHAD! I’m here, waiting for CHAD!”

It’s like a scene from a sitcom where the latte-sipping wizard and the rogue Uber car team up to create a traffic jam worthy of a Shakespearean tragedy. “To honk or not to honk, that is the question!” No five-star ratings allowed by the thousands of people inconvenienced by the stupid “ride-hailing app” performances they have to witness.

Source: Twitter, 1 Sep 2022 “Пишут «Яндекс.Такси» хакнули и заставили многих водил ехать на один и тот же адрес в Москве – как итог образовалась огромнейшая пробка на Кутузовском проспекте”

Ah, the history lesson of the ages! People were grappling with this conundrum for centuries, especially when those old-timey hackney cabs were turning city streets into chaos. Then, like a dazzling revelation from the 1800s, we discovered that buses, streetcars, and subways were the real MVPs. We all have places to be, so you hop on board, take a seat, and zip your lip. It works wonders because we’re all in this together, not stuck in that backward world of cutthroat competition like Uber.

For god’s sake they even named the stupid company Uber. How could the disaster of callous capitalist nihilism be any less obvious?

The whole ride-share “explosion” felt like a group of overconfident, privileged guys who missed the memo on history, thinking they could throw billions at reinventing the wheel and somehow make it less, well, wheel-like. I was there, both inside and outside those companies, shaking my head in disbelief.

Around 2012, it seemed like I was a lone tech warrior resisting the awful siren call of Uber, amidst meetings with dozens of big-shot executives running billion-dollar empires. They were all in, betting our fortunes on Uber, while I stood my engineering ground armed with the simple wisdom of human behavior studies. I’d show up with train tickets and bus receipts, each sporting single-digit price tags, and they laughed at my “lesser” status while they casually tossed around hundreds and more for an Uber ride.

At one point I found myself in an exclusive invitation to Uber’s HQ, where they rolled out the red carpet and asked me to help lead their security team. I decided to go out of courtesy and a dash of curiosity. But as soon as they started pitching their grand plan — “We want to be as indispensable as water, so people can’t survive without paying us…” — I felt like I was in a bad sci-fi Dystopia (lesser Utopia) and made a hasty exit, practically diving for the door.

“Let them ride in cake,” did you say?!

Approximately ten years later, while strolling through the streets of San Francisco, one of the original developers of Uber made a startling confession to me. They admitted that their early product design was shockingly naive. In their detached-STEM enthusiasm, they believed that constantly harvesting sensitive personal data through various sensors (such as geolocation, motion, video, and microphones) during every car ride, essentially engaging in covert surveillance of customers without their knowledge, would somehow benefit society. However, they acknowledged that they eventually had to leave the project when they came to the stark realization that this approach was a grave ethical error: creating an app pretending to be essential for survival while relentlessly spying on users for profit.

So here we are, MIT advising us to study human behavior to predict, well… known human behavior. Makes perfect sense to me. What took them so long? I guess they had to stop believing.

Now, if you’re dealing with a disability, a robotaxi might make as much sense as a luxurious, oversized wheelchair or, heck, even an elevator (vertical subway car) instead of stairs. But for the rest of us, it’s time to put on those walking shoes, head to the train, and stop being the roadblock to progress!

A train to a remote office in Connecticut was no joke, for example, a logistical maze in America’s shameless suburban planning system. I arrived early using a $18 train from NYC and finding a local bus that cost me $3 for the last mile, walking a sidewalk from the stop to the door. The other executive pulled up behind, sweaty, late and barking “no problem, it was only $200 for my Uber but we got some bad traffic”. He should have been fired on the spot.

Despite decades of working in Silicon Valley, working inside the largest and most successful tech companies, to this day I’ve taken only one Uber ride. As soon I as I stepped in it, I knew it was wrong. The Fabians probably would have just said: “no shit Sherlock”.