by Crime Lab Report
January 21, 2009

Michael J. Saks of Arizona State University and David L. Faigman of the University of California at San Francisco have become increasingly effective advocates for what they argue is a better brand of forensic science. From their perspective, the pattern identification disciplines such as those involving the identification of latent prints, firearms & toolmarks, and shoe impressions do not meet the standards to which they argue real science must conform.

A blistering critique titled Failed Forensics: How Forensic Science Lost its Way and How it Might Yet Find It, was published by Saks and Faigman in July 2008 in the Annual Review of Law and Social Science.

[http://arjournals.annualreviews.org/doi/abs/10.1146/annurev.lawsocsci.4.110707.172303]

In their words, pattern identification disciplines "have little or no basis in actual science. They neither borrow from established science nor systematically test their hypotheses."

As a courtesy to our readers, here is a quick peek at some of the claims that appeared in the aforementioned article:

- "The nonscience forensic sciences....are scientific failures in the sense that science...played little more than a rhetorical part in the development of these fields."

- "Absent any testing that can be replicated by other researchers and independently verified by courts, forensic identification science is not really a science at all."

- "Although knowledge from organic chemistry can be brought to bear in identifying what drugs, poisons, or medications might be discovered in a corpse found at a crime scene, what knowledge from conventional sciences like biology or chemistry or physics support the notion of individualization."

- "Most of the forensic identification sciences, however, missed the school bus. They never joined the university system.....They became an instrument of law enforcement, largely controlled by police technicians and their superiors."

- "If forensic individualization science had emerged from normal science, its approach and its techniques probably would resemble DNA typing, with its measurement of attributes, sampling of variation in populations, and statistical bases."

In the better world that Saks and Faigman envision, "the underlying assumptions of forensic identification would be subjected to intense questioning and empirical testing. In the course of this work, researchers would report what they tested, how they tested it, and what they found. Their results—whether good, bad, or indifferent—would be reported with equal candor."

Crime Lab Report is troubled by Professors Saks' and Faigman's failure to do due diligence in their review of the available literature. If they are unhappy with the fact that they could not find the evidence of research and scholarly review that they would expect, we would politely argue that they should have looked a bit harder. The evidence they seek cannot be found in the New York Times, legal journals, or papers written by misguided academicians who have joined the chorus of forensic science critics hoping to bring attention to themselves and their universities.

In the better world that we envision, Saks and Faigman would have practiced what they preach. They would have referenced any one of the numerous papers published over the last several decades in the Journal of Forensic Identification or the Journal of the Association of Firearm & Toolmark Examiners. Perhaps we would have seen a mention of groundbreaking studies in firearm identification such as the one conducted by David Brundage and recently repeated by Dr. James Hamby, which will show an impressively low rate of error among several hundred firearm examiners tested from multiple countries worldwide.

If Saks and Faigman had done their due diligence, they might have mentioned the extensive fingerprint and biometric research conducted by Lockheed Martin. Maybe they would have educated their readers about the high-tech security systems that are based on the biometric reading of friction-ridge patterns – not to mention the underlying research and development that was necessary to bring these systems to market. These technologies are based on the same principles that undergird the work of reputable fingerprint examiners around the world.

Finally, even the slightest bit of effort would have allowed Saks and Faigman to find and summarize the groundbreaking work sponsored by the Midwest Forensic Resource Center (MFRC) at the Ames Laboratory, which is operated for the U.S. Department of Energy by Iowa State University. Research investigators including Christophe Champod at the University of Lausanne in Switzerland are systematically invalidating claims by critics, including Michael Saks, that contextual bias in latent print verifications tends to corrupt the results of forensic examiners.

In fact, it appears that the research may show the exact opposite to be true. According to a research and development program summary published by the MFRC in October 2008:

“…fingerprint experts under the biasing conditions provided significantly fewer definitive and erroneous conclusions than the control group. They tended to provide opinions that were inconclusive.

“Novice participants were more influenced by the bias conditions and did tend to make incorrect judgments, especially when prompted towards an incorrect response by the bias prompt. This was not the case with the fingerprint experts.”

The results of this research have been presented at national and international conferences. Therefore, Crime Lab Report can only speculate why Saks and Faigman either missed it entirely or simply did not follow their own advice when they argued that good research must "maximize the contribution of the phenomenon under scrutiny and minimize the contribution of expectations and biases."

In our view, Saks and Faigman are simply guilty of using their resources and academic affiliations to promote social changes despite overwhelming factual evidence that invalidates their core arguments.

This kind of behavior is not research nor science. It is activism.

The public suffers immeasurably when activism is fraudulently packaged and presented as scholarly research. But the trend will continue if more qualified professionals and professional organizations aren't willing to take the time or risk to vigorously challenge it.

Crime Lab Report's managing editors are aware that they too have been labeled as activists by some who disagree with their point of view. But there is a difference. Crime Lab Report does not prioritize the promotion of specific changes. It promotes the dissemination of accurate information about forensic science and is willing to accept whatever changes come as a result.

That being said, we have a responsibility to acknowledge that Saks and Faigman offer a perspective that has some intellectual and scientific value; we will address this in a moment.

In the meantime, we would like to offer Saks, Faigman, and other commentators a few valuable lessons that we hope will shape their thinking about disciplines practiced in forensic testing laboratories:

1. If DNA analysts could observe and compare DNA with their own eyes they would do it. Examiners of latent prints, firearm evidence, and toolmarks are fortunate because they can actually observe the evidence in question. They can see the ridge detail of a fingerprint and they can see the striae and impressions on fired bullets and cartridge cases. They don't need instrumental data to tell them what they are looking at. They can see it. They can report it. They can even photograph it. Even better, the analytical processes in these disciplines rarely require them to consume evidence, which makes it available for others to review at a later time.

2. There is no such thing as a purely objective science. All science requires that imperfect human beings draw conclusions about what they see and measure. DNA and chemistry are no exceptions. Even in mathematics, commonly described as the only pure science, calculations must be applied to real world problems using careful interpretation and professional judgment. When all subjectivity is eliminated from an endeavor, science is no longer needed because there are no interpretations to govern. Without interpretation, there is no science.

3. Universities do not have a monopoly on science. Science does not require the control and oversight of universities as Saks and Faigman repeatedly emphasized. While universities are highly regarded for their expertise and resources, their focus is more often drawn toward projects that bring notoriety and/or funding.

Science is, and should be, a very inclusive institution that avails itself to any number of people seeking to solve any number of problems. Its fundamental tenets demand that knowledge be gathered in a controlled and systematic way. Then, when it comes time to apply this knowledge, practitioners must exercise professionalism, caution, and self-restraint so that they do not stray beyond what the accumulated knowledge can justify. The forensic laboratory disciplines are very young when compared to other fields. But as long as they continue to advance and provide knowledge that improves the human condition, they are traveling successfully on the path we call science.

4. Until recently, there was no economic justification for universities to invest in forensic research on a large programmatic scale. It wasn't until the early 1960s that epidemic increases in crime necessitated the creation of the large network of crime laboratories that we know today. Furthermore, many of the legal decisions issued by the United States Supreme Court under Chief Justice Earl Warren during the 1950s and 1960s placed an increased emphasis on scientific evidence and a decreased emphasis on information gathered exclusively from police interrogations and suspect confessions. Eventually, with the dawn of the 21st century and demand for services at all-time highs, forensic science funding became more widespread and therefore attracted the attention of many universities (and critics) that had little prior interest in the field.

5. The history of pattern identification and uniqueness is rooted solidly in academics. The most famous pioneers of the pattern identification disciplines had strong academic backgrounds. For example, Calvin Goddard, regarded as the father of firearm identification, was a medical doctor and professor at Northwestern University. He was also the military editor for the Encyclopedia Britannica.

Another pioneer in the forensic identification of firearms, Dr. J. Howard Matthews, worked for nearly forty years in the field. Matthews obtained his Master's and Ph.D. from Harvard and served as a professor of chemistry at the University of Wisconsin for over thirty years. He was a fellow of the American Association for the Advancement of Science and one of the founders of the professional chemical fraternity Alpha Chi Sigma. His classic three-volume treatise, Firearms Identification, was the largest single source of information on firearms identification ever assembled.

Finally, Sir Francis Galton, a brilliant statistician who demonstrated the uniqueness and permanence of fingerprints, conducted research at Trinity College and the

University of Cambridge. He was instrumental in developing methods for studying variations in the human population, which ultimately fueled the growth of latent print identification as we know it.

6. The innocent are protected by the pattern identification disciplines. If allowed to go unchecked, Saks’ and Faigman’s zeal for activism would actually harm the innocent. Firearm examiners, for example, frequently identify firearms that were not used in the commission of crimes. Similarly, latent print examiners are more likely to exclude individuals as viable suspects than to include them. Therefore, the tendency of activists to portray forensic science as being reserved for the demonstration of guilt ignores the value of pattern evidence in preventing the wrongful arrest and/or conviction of innocent persons.

So here are the facts. Disciplines such as latent print identification and firearm & toolmark identification are reliable, valid, and useful sciences. They are bodies of knowledge and applied methods that have developed over a long period of time during which many competent researchers attempted to falsify their underlying hypotheses and failed.

Certainly, continuing research and improvement must be a constant force in the evolution of all forensic disciplines. But critics who have committed themselves to lowering public confidence in our criminal justice system are choosing to ignore compelling evidence at the expense of public safety. For this alone their tactics and rhetoric should be repudiated.

Saks and Faigman complain that there is insufficient research to support the conclusions rendered by pattern identification experts. Then why can’t they present research (preferably that which meets their stated standards of validity) that demonstrates such conclusions to be unreliable?

Because it doesn't exist.

When the proper methods are used and the appropriate quality-assurance checks are employed, the subsequent results in the pattern identification disciplines can be reported with a degree of confidence that makes them useful to the criminal justice system.

This leads us into what we believe is the real issue that Saks and Faigman are trying to address but are too distracted by their own biases and expectations to take notice of.

Crime Lab Report believes that the single most serious technical problem in forensic testing laboratories today is not invalid methods nor lack of research. It is poorly and ambiguously worded conclusions that leave laypersons with an incomplete or confused understanding of what the results actually mean.

This problem has nothing to do with the validity or admissibility of a science and can often be mitigated with some simple questioning. But it does represent a sort of malpractice that should not be tolerated by the forensic laboratory community nor its stakeholders.

In the forensic testing sciences, a particularly heavy burden is placed on practitioners to report clear and complete conclusions that are unlikely to be misconstrued. If the wrong words are used or if poor writing skills preclude the reader from understanding the meaning of a testing report, even the most reliable science can be made to look suspicious or even inferior.

Too often, laboratory directors and quality-control managers struggling with overflowing backlogs and shoestring budgets don't provide solid training to scientists in the areas of courtroom testimony and technical writing. Other labs are simply hesitant to change the status quo and prefer to stick with the language to which they have become accustomed. As a result, they inappropriately prioritize the preservation of tradition at the expense of scientific clarity.

Perhaps evidenced by the criticisms of Saks and Faigman, these weaknesses seem to bear heavily on practitioners in the pattern identification disciplines which necessarily rely upon direct observation instead of instrumental analysis. Pattern identification experts would be wise to carefully review and standardize the wording of their conclusions to eliminate ambiguity and maximize scientific value. We believe the right changes would drastically empower these embattled disciplines and better serve their stakeholders.

The forensic science community has worked tirelessly and successfully to improve its administrative and technical practices through accreditation, certification, and more comprehensive methods for managing quality. We know this progress will continue. Crime Lab Report believes, however, that better and more consistent wording of conclusions is a new frontier that forensic practitioners will embark on in the next several years. By doing so, they will make it harder for critics like Saks and Faigman to confuse weak communication skills for scientific invalidity.

We would like to believe that Michael Saks and David Faigman are intelligent men who want our criminal justice system to be accurate and fair. But their publications reveal a systemic ignorance and carelessness that will only inhibit their ability to make a positive and lasting impact on the criminal justice system.

It's up to them to decide if they want to be activists or genuine truth-seekers.

We advise the latter.

****************************

This Crime Lab Report is made available to Detail readers and subscribers free of charge and not for profit.  The Copyright is maintained and enforced by Crime Lab Report and represented here only for Fair Use. FAIR USE NOTICE: This newsletter contains copyrighted material from the public domain, the use of which has not been specifically authorized by the copyright owner.  www.clpex.com makes such material available in an effort to advance scientific understanding in the field of latent prints, thus constituting a 'fair use' of copyrighted material as provided for in section 107 of the US Copyright Law.  In accordance with Title 17 U.S.C. Section 107, the material on this site is displayed without profit to those who wish to view this information for research and/or educational purposes.  If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the original copyright owner.