Toyota, WSJ, and Computers
I heard a segment on NPR this evening about the Toyota sudden uncontrolled acceleration problem (I’ll just call it SUAP). They were following the lead of the Wall Street Journal, who said:
The U.S. Department of Transportation has analyzed dozens of data recorders from Toyota Motor Corp. vehicles involved in accidents blamed on sudden acceleration and found that at the time of the crashes, throttles were wide open and the brakes were not engaged, people familiar with the findings said.
The results suggest that some drivers who said their Toyota and Lexus vehicles surged out of control were mistakenly flooring the accelerator when they intended to jam on the brakes. But the findings don’t exonerate Toyota from two known issues blamed for sudden acceleration in its vehicles: sticky accelerator pedals and floor mats that can trap accelerator pedals to the floor.
What the WSJ reported, though, doesn’t exonerate Toyota of anything.
NPR had a commentator on who said something to the effect that 100% of the cases examined showed the same thing, and that one would be hard pressed to argue that the computers got it wrong every time. Not at all, Mr. Non-programmer dude on the radio; all it shows is that the fault is upstream of the black-box recorder and not downstream of it. And it isn’t just the driver who is upstream; there is a lot of Toyota software and hardware there, too. If the Toyotas have an intermittent fault that causes the brake to be recognized as if it were the accelerator, it would explain the data far better than the “all those drivers forgot which pedal is the brake pedal, some of them for minutes at a time” conjecture. That’s just one way in which the problem might occur. In any case, it appears that the data recorders do tell us what the computer controlling the car operated upon, which is full-throttle acceleration and no attention to brakes whatsoever, which corresponds neatly with the survivors’ reports of what happened to them.
I’m thinking when all is said and done, this is going to be discovered to be a software fault in Toyota’s control program. I’m hoping the commentator on NPR gets 30 seconds of airtime to make an abject apology to the survivors when that happens.
Update: I found the NPR All Things Considered transcript, and the fellow whose name I didn’t recall is Mike Ramsey of the Wall Street Journal.
NORRIS: How many data recorders were analyzed? And of those, how many of these accidents were found to have been caused by driver error?
Mr. RAMSEY: Well, we have been saying several dozen, all of them that were -fit the criteria, were found to have the brake not depressed and the accelerator wide open. So 100 percent of the incidents where it fit that criteria, that’s what was found.
NORRIS: One hundred percent?
Mr. RAMSEY: Yes.
NORRIS: It sounds like, upon hearing that, that the government might be on its way toward exonerating Toyota.
Mr. RAMSEY: Well, when it comes to incidents where people are claiming electronic throttle control, the government has already said they have no evidence of it. This set of data, what it does is it completes the other side of it, which is if it’s not that, then what is it, right? It’s probably driver error. So the government has been hesitant to say that so far.
[…]
I totally understand the position of these people. And if you hear many of these anecdotes, it’s incredibly compelling to hear them and all of their evidence. That said, when you have dozens of incidents that are similar where people say they were stepping on the brake and the car accelerated anyway and hit and that all of these incidents show virtually the same findings, that’s difficult to believe that the computer was wrong and, you know, they had a special instance.
(Emphasis added.)
Mike, the data recorder can say what it says and the survivors still be right. Try doing some embedded programming sometime. You haven’t come up with anything that in the least puts their accounts in a bad light, at least not to those who know something about computer control systems.
And be scripting your apology.
Update 2: I’ve marked in bold a particularly interesting piece of information from Ramsey. We have dozens of incidents that show exactly the same thing: no depression of brakes ever, and full depression of the accelerator throughout. This pattern is not what one would expect of humans behaving either in panic, where accidental touching of the brake would be likely, or in Mr. Ramsey’s alternative of confusion of pedals. Pumping the brake is common, so if people were confusing the accelerator with the brake, we’d expect to see some fraction of those incidents showing variation in the accelerator control, and according to Mr. Ramsey, we never see that. That’s pretty damning for Toyota, I think. Having absolutely the same data pattern across dozens of drivers when some of those incidents went on for a significant amount of time doesn’t speak to mass confusion of drivers; it says “computer screw-up” to me.
Update 3: After a few years, the dust settles. Toyota’s firmware was analyzed and found wanting. The logs were a case of garbage-in, garbage-out. Toyota paid $1.1 billion (with a B!) to settle a class action lawsuit on this matter. The NHTSA has egg on its face because they exonerated Toyota and a more careful look later found definite problems. Mike Ramsey, so far as I know, still owes an apology to all those drivers. Life goes on, for those who managed to survive. Embedded programming is tough and requires discipline to do well, especially for mission-critical things like controlling an automobile.
I was thinking the same thing; when I listened to the commentator I wondered how there could be a return of 100% of the same result unless there really was a problem with the data collection and that would be common to computers all with the same programming.
I test drove a Toyota recently and asked The Question. I was told that it was aftermarket floormats, not genuine Toyota floormats, that had caused the problem. Hence the problem did not arise elsewhere in the world, because only in the US were aftermarket floormats used in new cars, and they slipped from a lack of rubber backing.
It does make sense, if there are only cases in the US. Were different programs used in the US than in, say, Australia, Japan or Europe? If so, why? Some Whewellian reasoning is needed here.
Millions of Toyotas on the roads of the USA, a handful of cases of SUAP going on. What’s the expectation for the numbers of vehicles in Australia, Japan, or Europe? Is the premise even true? You are relying on a car salesman for that piece of data. I don’t know what Whewell would think of that.
Bayes, though, would likely think that the expectation that in a few dozen cases the panicked drivers attempting to stop runaway vehicles would never, even by accident, touch the actual brake pedal to be beyond reasonable.
John,
A commenter on NPR pointed out,
That seems a fair question, and another point for the Bayesian approach.
On on the NPR thread, a commenter derided the notion that software could be at fault:
There’s a lot of ways that the standard v. automatic transmission difference could be causative. Simple example: let’s say the sensor table is constructed using a “union”, where the two variants are “standard transmission case” and “automatic transmission case”, the sensor data for the transmissions differs in length, and the brake and steering sensors are defined later in the union. Then, if the computer rarely spuriously recognized the transmission as “standard” when it was “automatic”, but not the reverse, then what sensors are treated as which would shift. Maybe “radio on” becomes the new “accelerator” input, and “trunk latch open” the new “brake” input, in which case braking wouldn’t happen unless the driver happened to pop the trunk latch. That’s not likely to be the actual case, but only because so many different similar possibilities exist, not because a software fault specific to the automatic transmission case is difficult to conceptualize. That sort of programming would be pretty egregious, but more subtle flaws could still put the driver at risk.
Why might the “automatic” case sometimes not stick? Let’s posit that the standard transmission is treated as the default case, and that for the car to recognize the automatic transmission, it has to get a positive signal from some unit in the automatic transmission. Unit overheats or glitches and stops going positive, and suddenly the car shifts to the standard transmission case in reading its sensor tables. For the standard transmission, there’s never any risk that the line would go positive with the “automatic” signal, and thus no faults leading to SUAP. That’s entirely speculative, but it is at least a plausible account of why the two cases might differ in risk.
Of course, I’m assuming here for the sake of argument that the automatic v. standard transmission assertion is true.
There are factors that lean different ways in this.
One effect of the early Toyota suppression of inquiry is that, at least for those months until the media picked up the story, one can’t simply dismiss the cases as people piling on Toyota to excuse their own bad driving. Humans being humans, I’m sure that later on we had people doing just that, and the normal rate of people getting confused over what pedal was which. But that doesn’t mean that there is no fault in the Toyota system, just that it may be more difficult to recognize the cases where hardware and software might be the cause rather than the driver.
I don’t have the numbers for Toyota vehicles in different countries such that I can derive expectations for SUAP incidents. If SUAP is very rare, though, in other countries there could be a small number of cases of fatal incidents written off to “driver error” already based on the same misinterpretation of the data recorder logs. If we are biased against detecting mechanical fault, we are less likely to even think an incident is suspicious. This is just basic signal detection theory. But if the numbers argue that more effect should be seen elsewhere, this would be an argument that the proportion of false reports and driver error is higher.
What I keep coming back to is that human behavior is variable and what has been reported for the complete set of cases examined is not variable. The data recorder logs support the account of the drivers: the car did not recognize the brake as being on, and kept the accelerator on full throttle.
I hope the NHTSA does not succumb to the temptation to stop putting effort into figuring out what’s going on.
Unless the brakes are applied by wire (and therefore were not applied despite stepping on the pedal), there’s no way the car’s engine could overcome them.
Yes, that’s the issue… the Toyota vehicles are fly-by-wire devices, so only if the computer is properly recognizing the brake input will brakes be applied on a brake pedal press. The lack of variation in the data recorders logs sounds less like human reactions and more like a computer glitch.
After a pretty comprehensive servicing, my Infiniti accelerator pedal stuck wide open when I passed a car in upstate New York. I can tell you that my very first reaction was to tromp down on the brake pedal, which slowed the car down immediately. (Thank Zeus the brakes worked – I was going 100 mph in about 1.5 seconds after I took my foot off the accelerator.)
It was, in fact, a problem of the accelerator pedal being trapped by an aftermarket rubber snow and ice mat. Needless to say, my first action after I get in the car these days is to pull that mat all the way toward the rear of the car.
The idea that none of the drivers involved with the Toyota problems didn’t hit their brakes is ludicrous. Hitting the brakes is a reflex action – it occurs even faster than you can shout out an expletive.
Pingback: Toyota, WSJ, and Computers: An Update | The Austringer