Software bugs may compromise breath test machine

As previously noted here, courts in Minnesota and New Jersey have ordered the state to provide the source code that runs their breath test machines to defense counsel for independent testing. The defense bar has long hypothesized that these machines are unreliable at their core due to problems in the computer programs that "run" the machines.

In the New Jersey litigation, John J. Wisniewski of Base One Technologies analyzed the source code in Draeger's Alcotest 7110 Mk III C and his conclusions are eye-opening, to say the least.

Testing the Alcotest 7110Mk III Source Code uncovered 24 major defects. For the
purposes of this overview,we have identified 9 defects with thegreatest impact on the instrument test results, and the validity of those tests.

The study concluded that the Alcotest software would not pass industry standards in the U.S. for development and testing because Draeger refused to make the code available for audit. I find it very disconcerting that the companies that manufacture these breath test machines that are used to brand our fellow citizens as criminals, claim they cannot release the source code because it is a trade secret.

He also stated that the Alcotest software had yet to be tested completely. The program contains more than 45,000 lines of code of which 3,200 lines are used to make decisions. The report states that the lack of use of industry coding standards prevented testing of critical paths in the software and prevented errors from being removed.

The source code also prevents the machine from detecting catastrophic problems and shutting itself down. As a result, should the machine encounter these conditions, the results of breath tests become unpredictable.

The Alcotest code also lacks the ability to determine whether motors or valves controlled by the software are functioning properly. As a result, the machine assumes its internal components are working correctly.

The Alcotest conducts its diagnostic routine on the Analog/Digital converter during the data measurement cycle -- not before the measurement is taken. In the event of a diagnostic failure, the machine will substitute default values.

The machine assumes there is no airflow through the machine at the beginning of a test. The machine takes a measurement and then uses that measurement as a baseline. The software is incapable of determining if the amount of air flow detected at that point indicates a problem. The machine also substitutes a default value in the event the measurement fails. So, if the machine was off at inception, it will continue to be off everytime is has to revert to a default value.

While the Alcotest will detect measurement errors, it will only report them if they have occurred a certain number of times consecutively.

The software doesn't guard against the entry of incorrect global values in other memory locations. As these global values are very important in the Alcotest machine, any errors in these global values can cause great variations in breath test scores.

There are also timing problems as the code is written in C and not assembly language. As a result, there may be delays in operation of the machine during a test.

Base One's testing also revealed:

the software has to beconsidered unreliable and untested, and in several cases it does not meet stated requirements. The source code supplied has creation dates and modification dates from 1993 to 1997, but the coding architecture, style, organization, and modification documentation (audit trail) more closely resemble the software principles used in the 1970’s and 1980’s.

The testing also found instances in which, instead of deleting unnecessary code, programmers merely disabled it by means of comments.

The study concluded that:

As a matter of public safety, the Alcotest should be suspended from use until the softwarehas been reviewed against an acceptable set of software development standards, and recoded and tested if necessary. An incorrect breath test could lead to accidents and possible loss of life, because the device might not detect a person who is under the influence, and that person would be allowed to drive. The possibility also exists that a person not under the influence could be wrongly accused and/or convicted.

No one has yet subjected the source code of CMI's Intoxilyzer 5000 to the scrutiny faced by the Draeger Alcotest, but I suspect we would see some of the same problems in the code running Texas' breath test machines as well. Is this really what we want determining whether our fellow citizens should be branded criminals for life?

See also:

• Buggy breathalyzer code reflects importance of source review (Arstechnica.com, May 15, 2009)