Tesla Safety Fail: “Full Self Driving” Can’t Read Even Basic Traffic Signals

I just got off the phone with a technology reporter for a major newspaper, where I was asked about the risks of facial recognition.

That’s a deep topic, since crime using faces (deepfakes, impersonation) can apply to almost anything (fraud/theft, racism, classicism). However, I ended the call by warning about even broader risks in recognition linked with crimes: automobiles as human exoskeletons misidentifying any aspects of the world around them and causing death.

Tesla, to me, seems to present the most dangerous and negligent engineering examples in the world. They exhibit a recurring inability to read path obstacles or even basic traffic signals, ultimately treating unqualified drivers as their unwitting crash test dummies.

Overall Tesla is reported to have fewer screw-ups than Google in terms of sheer quantity, yet every time Tesla fails it may be another avoidable fatality.

Source: Quick visual I put together from a database query of incidents

To set the stage properly now, it was April of 2020 when Tesla announced it was giving its customers an update to detect street lights.

After testing on public roads, Tesla is rolling out a new feature of its partially automated driving system designed to spot stop signs and traffic signals.

The update of the electric car company’s cruise control and auto-steer systems is a step toward CEO Elon Musk’s pledge to convert cars to fully self-driving vehicles later this year.

But it also runs contrary to recommendations from the U.S. National Transportation Safety Board that include limiting where Tesla’s Autopilot driving system can operate because it has failed to spot and react to hazards in at least three fatal crashes.

Ok, three very important points to make based on this old press release and what we know today:

  1. Tesla has been testing on public roads by releasing unsafe, unproven engineering and then waiting to see how bad it is. They exhibit willful disregard for safety and complete lack of ethics.
  2. “Later this year” would have been 2020, and the CEO’s pledge was (like all his pledges) proven to be deceptive if not entirely false. It’s been over a year and their software still fails the most basic tests.
  3. The US safety regulator is only making recommendations, whereas it seems like they should be figuring out how to ban Tesla from operating on public roads

Perhaps it was summarized better by two safety experts in that same article, who describe Tesla engineering as a kind of scam.

Jason Levine, executive director of the Center for Auto Safety, a nonprofit watchdog group, said Tesla is using the feature to sell cars and get media attention, even though it might not work. “Unfortunately, we’ll find out the hard way,” he said.

Whenever one of its vehicles using Autopilot is involved in a crash, Tesla points to “legalese” warning drivers that they have to pay attention, Levine said. But he said Tesla drivers have a history over-relying on the company’s electronics.

Missy Cummings, a robotics and human factors professor at Duke University, fears that a Tesla will fail to stop for a traffic light and a driver won’t be paying attention. She also said Tesla is using its customers for “free testing” of new software.

Now a new video shows exactly what experts predicted, a new “Full Self Driving” product fails to register a strip of three green stop lights turning yellow.

To be clear about the failure, we can see very distinct green lights to start with.

Then watch what happens when they shift to yellow.

The human in the car observes the following sequence:

  • 0.92 seconds (~100ft): computer recognizes light change
  • 1.88 seconds (~200ft): computer applies brakes
  • 2.00 seconds: computer disables itself at 50 mph

I can confirm 100% that AutoPilot started applying brakes at +1.88s from the light change, the emergency alarm sounded at +2.0s (47mph), and I applied additional brakes myself at +3.75s (35mph) to ensure I didn’t enter the intersection.

Perhaps most notable in terms of criminal behavior, and per my comment above about unqualified drivers, a YouTube account commenting on this incident proudly admits trying to pump his accelerator and run red lights.

Did you catch that account name? The notorious racist slogan: “All Lives Matter”.

“All Lives Matter” wants everyone to know that if a Tesla says it sees a red light he has not been able to force it to drive through anyway.

Keep in mind that “All Lives Matter” is a slogan of violent social media terror campaigns that have been trying to convince American drivers to drive through crowds, run over people to kill them and silence speech.

Here we see not only Tesla safety engineering failing, but that a YouTube discussion of failures is being linked to a domestic terror campaign that violates traffic laws, specifically ignoring orders to stop.

Consider a new story about a man who just used this exact “pump” accelerator method to kill people:

Knajdek was using her car to block the intersection to protect the protesters as they demonstrated. Protester Ty Henderson says Knajdek was becoming a leader in the movement for justice. He says she was leading the game of “Red Light, Green Light” with the crowd when he witnessed the unthinkable. “The only reason I saw it was because I heard the tires screech and the engine rev up. Like, I heard it and I’m like, ‘What is that?’ And I look up and all I see is headlights, and all I could think is, like, it’s going faster,” Henderson said.

“Engine rev up” is an outsider perspective. The attacker was playing a different game called “Red light, Pedal Pump”.

In related news:

Canadian Prime Minister Justin Trudeau has denounced an attack involving a driver accused of plowing a pickup truck into an immigrant family of five, killing four of them.

When Tesla can’t see green lights switch to yellow properly, is there any real expectation it would see a human child properly?

And isn’t that exactly what some Tesla buyers are attracted towards, a sense of irresponsibility and privilege? The CEO allegedly is enticing people to give him money so they can cause harm with impunity, a sick form of social entry that seems based in apartheid.

Safety is not a joke yet Tesla’s CEO seems obsessed with spreading negligence — he tells regulators important safety words like flamethrower and driverless can be whatever he wants them to mean regardless of fact.

More than 1,000 [Tesla CEO] flamethrower purchasers abroad have had their devices confiscated by customs officers or local police, with many facing fines and weapons charges. In the U.S., the flamethrowers have been implicated in at least one local and one federal criminal investigation. There have also been at least three occasions in which the Boring Company devices have been featured in weapons hauls seized from suspected drug dealers.

[…]

Tesla, the electric automaker led by Musk, has been criticized for naming its advanced driver assistant system Autopilot and for calling the $10,000 add-on option Full Self-Driving (FSD), even though the driver must remain engaged at all times and is legally liable. A German court has banned the company from using the terms “Autopilot” or “full potential for autonomous driving” on its website or in other marketing materials.

Indeed, should Tesla be banned? German courts today know a thing or two about stopping mass harms before it’s too late.

Update January 2023: A Tesla with brake failure and acceleration ignoring a red light caused catastrophic asset damage.

Repairs to the Greater Columbus Convention Center from damage caused by a car that crashed into the building at a high rate of speed last year have turned out to cost substantially more than the original estimate of $250,000. The repair bill now stands at just under $663,000, which is 165% higher than the estimate. […] The damage was caused May 4, 2022 when a 2020 Model S Tesla estimated to be traveling 70 mph ran a red light and flew through a glass wall fronting High Street around lunchtime. The Dispatch used the Ohio Public Records Act to uncover security-camera video of the crash. It showed the car going airborne and flying through an exterior wall, smashing into a steel column that prevented it from continuing through the center of the crowded center — and stunning pedestrians on the sidewalk. The Tesla was an operating taxicab owned by Columbus Yellow Cab, and the driver told police that the vehicle wouldn’t slow down despite his attempts to brake, causing him to ultimately crash.

Why Russia Favors Attacking American Information Systems

Senator Mark Warner (D-VA) in an old interview with Wired explained the basic economics of Russian information systems warfare:

If you add up all Russia spent in the Brexit vote, the French presidential elections, and the 2016 American elections, it’s less than the cost of one new F-35 airplane.

Less cost, more damage should be the byline of Facebook as I wrote here back in 2011.

Or as a cartoonist wisely frames it (aside from the fact that nuclear missiles were managed by Russians, while Facebook is managed by Americans being funded by Russians… Twitter funded by Saudis, etc).

Today in the latest round of Geneva talks, it might seem that America is attempting a new form of peace with Russia.

It will mark the third time that Geneva has hosted U.S. and Russian leaders’ talks: The first was a multilateral meeting involving U.S. President Dwight D. Eisenhower and Soviet leader Nikita Khrushchev in 1955. The second came 30 years later, when President Ronald Reagan met Mikhail Gorbachev — an important icebreaker that some say paved the way toward the end of the Soviet Union.

However, both prior cases had a Russian leader with initiative, actively working on their own to reduce the instability and risk of attacking America. I know that’s hard to believe for many Americans, yet Khrushchev and Gorbachev both approached the table with some sense of humanity.

There seem to be no indications anywhere Putin has any real objectives other than spreading toxic white nationalism to destabilize the world for his self-enrichment. Instability seems to serve him in the same way that it serves American tech executives, his toxic takeover of Crimea like Facebook acquiring WhatsApp.

Russia in fact lacks any clear identity or mission other than Putin taking all and giving nothing, like he has an office on Sand Hill Road in Silicon Valley into which the economics of information warfare unfortunately play a huge part.

Stopping Russia today should be little different than blocking a shameless bully who always looks for inexpensive routes to do wrong without responsibility.

It is quite a different situation than the prior two Geneva talks where incentives seemed far more aligned around leaders on both sides being thoughtful about what would be right and moral.

Bailing Sand

the truck transmission whined in protest, the computer gave up. then, bailing away soft flowing sand from our door sills, shovel burning my hands even under a cool moonless starry night… something was truly exhilarating about digging out.

this machine would never understand. sat quietly and waited for rescue by a tool thousands of years old.

in a way, hacking machines is like driving off-road so far that you’ll maybe never make it out again. and that’s why to do it. humans are driven by curiosity, machines are driven by humans.

Landmines are Banned. Should Drones Aloft be Classified as Airmines?

The United Nations has this introduction to the subject of mines and disarmament:

Landmines come in two varieties: anti-personnel and anti-vehicle mines. Both have caused great suffering in the past decades. Anti-personnel landmines are prohibited under the Convention on the Prohibition of the Use, Stockpiling, Production and Transfer of Anti-Personnel Mines and on Their Destruction (or Mine Ban Convention), adopted in 1997. More than 150 countries have joined this treaty. Its positive impact includes a marked reduction of casualties, an increased number of mine-free States, destroyed stockpiles and improved assistance to victims.

The question becomes if land-mines are prohibited, aren’t drones aloft (loitering munitions of WWII that explode on some trigger mechanism) thus simply air-mines and prohibited already?

Israel’s recent use of drones is being described in Defense Update as an intentional move away from human oversight in anti-personnel explosives.

First, Israel is calling out safety and human oversight as too slow to engage in combat:

In recent years drones have proved essential for all military operations, providing critical intelligence and pursuing time-sensitive targets. As they loiter over the battlespace, drones can spot enemy activities on the ground, but transferring this insight into action may take hours as the call for fire is processed through the echelons until the order to fire is approved.

Empowering the company commanders with the means and authority to order and approve an attack by their organic weapons, supporting artillery, naval, or air support enables the IDF to engage targets having a short lifespan. These targets are often exposed by exploiting the friction created through the movement of manned or unmanned combat units in enemy territory.

That seems like a simple enough problem to solve, like giving orders to authorize a soldier holding a gun to use discretion when firing and not call for approval, unless a call is needed.

A Navy SEAL told me a story about this where he had given his men orders to immediately shoot anyone they saw pointing weapons at the American President — without delay and without need for approval.

Soon after a call came through for approval to fire at someone pointing a gun at the President. Confused by the request, he asked questions. The answer became a foreign soldier was pointing a large sniper rifle towards the President (ostensibly to help guard him by looking through its scope for targets). Obviously the SEAL leader said don’t fire.

Second, although Israel is emphasizing a chain of command and authorized discretion by a company, it is not clear that will continue to hold true.

This video makes the drone look very much more like a mine designed to be remotely staged, almost like planting to explode later on a simple wireless trigger.

Will swarms be converted by “efficiency” pressure (like how artillery shells became IED) into becoming airborne mines?

Here’s the conclusion of the Defense Update:

On May 6, 2021, as the fighting in the south erupted, the new S&D unit moved quickly to become the first military unit to operate drone swarms in combat. Within few hours, they deployed and fielded this brand-new system, seeking and destroying dozens of hidden enemy targets in complex terrain in rural and urban areas.

Within a few days, the new unit brought stunning results. A single company empowered by drone swarms, precision weapons, and comprehensive C4I delivered over 30 missions, destroying dozens of enemy targets several kilometers beyond the border. They were able to locate the enemy in complex urban and densely vegetated rural areas, designate targets, assess those targets at the company CP, strike the targets selected for engagement and perform battle damage assessment (BDA), all that done within minutes by drone swarms. Following the success of these S&D companies, the GFC recommended converting all combat support companies in regular force to S&D companies over the next year.

I read that as the only ethics/oversight gate remaining would be to “assess targets at the company CP and select for engagement”, unless you count BDA, while it’s all done within minutes”.

With that in mind, a problem of mines in the area is a story dating back to at least WWII.

The Arab Republic of Egypt is contaminated with mines and explosive remnants of war in the Western Desert, which date from World War II, and in the Sinai Peninsula and Eastern Desert, which are a legacy of wars with Israel between 1956 and 1973. […] The government has stated that some 17 million landmines were left in the Western Desert and another 5.5 million in Sinai and the Eastern Desert.

Some important lessons there, surely. What if drone swarms are left behind? What if they are commandeered or corrupted? Imagine a swarm being tricked into drifting backwards towards launchers and targeting owners, or just being left behind and then recycled into some future terrorism campaign.

The next question perhaps is blow-back, as old mines left in the desert during long-ago wars have been dug out and repurposed into modern IED by terrorists.

…these munitions have become part of a new and worrisome trend. As the Islamic State and other jihadi groups have grown throughout the region, sometimes roaming unchecked across long, porous borders, a few have realized the potential power of this massive cache of explosives, much of it buried here by the Nazis. Military and civilian officials in Cairo say ISIS and other groups have already MacGyvered these decades-old mines, using their components for bombs, improvised explosive devices (IED) and other instruments of death. “We’ve had at least 10 reports from the military of terrorists using old mines, says Fathy el-Shazly, a former ambassador to Saudi Arabia who until recently served as Egypt’s land mine clearance czar. “Even now, these things trouble us in different ways.”

A Nazi is either laying or clearing mines. Can you tell from this photo? If you were authorized to shoot, what would you do? Trick question: it’s a Nazi.