e-Discovery Team Training

Welcome to Module 2-N.

Criminal Case Raises Interesting e-Discovery Search Issues

Judge John Facciola has authored many intriguing opinions on e-discovery, including one in a criminal case. United States v. O’Keefe, 2008 U.S. Dist. LEXIS 12220 (D.D.C. Feb. 18, 2008).  The decision begins with criminal law specific issues, such as what discovery rules to apply, but quickly transcends them to address universal e-discovery questions. The most interesting issue was raised by the accused defendants who challenged the adequacy of the government’s keyword search. Judge Facciola held that this issue was too complicated for lawyers and judges to address without the aid of expert testimony. He implies that to do otherwise would be foolish, since he concluded that keyword search analysis is an area of e-discovery “where angels fear to tread.” This is another important class for students wanting greater expertise on search and the role of experts.

Obviously I am no angel, nor are thousands of other lawyers who must routinely deal with keyword search issues. Perhaps we are indeed foolish to run these searches without more help. As Alexander Pope said: “A little learning is a dangerous thing.” Here is the full quote from Judge Facciola’s opinion:

As noted above, defendants protest the search terms the government used. Fn6. Whether search terms or “keywords” will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics. See George L. Paul & Jason R. Baron, Information Inflation: the Legal System Adapt?, 13 RICH. J.L. & TECH. 10 (2007). Indeed, a special project of the Working Group on Electronic Discovery of the Sedona Conference is studying that subject and their work indicates how difficult this question is. See The Sedona Conference, Best Practices Commentary on the Use of Search and Information Retrieval, 8 THE SEDONA CONF. J. 189 (2007) Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread. This topic is clearly beyond the ken of a layman and requires that any such conclusion be based on evidence that, for example, meets the criteria of Rule 702 of the Federal Rules of Evidence. Accordingly, if defendants are going to contend that the search terms used by the government were insufficient, they will have to specifically so contend in a motion to compel and their contention must be based on evidence that meets the requirements of Rule 702 of the Federal Rules of Evidence.

By the way, for those who have not read Rule 702 in a while, it says:

Rule 702 Testimony by Experts
If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.

To my knowledge, O’Keefe is the first opinion to suggest that judicial review of alleged search deficiencies requires expert testimony. This may well portend a new type of expert in the future who opines concerning the search methods employed by parties to litigation. A whole new line of employment could open up for information scientists, e-discovery lawyers and others who specialize in the field of search.

The other issues in this case are interesting too, and shed some needed background on the search ruling. The defendant O’Keefe is a government official accused of taking bribes from co-defendant Agrawal for expediting visas to the U.S. for Agrawal’s employees. In a prior order, District Court Judge Friedman:

required the government to conduct a thorough and complete search of both its hard copy and electronic files in “a good faith effort to uncover all responsive information in its ‘possession custody or control.’” United States v. O’Keefe, No. 06-CR-0249, 2007 WL 1239204, at *3 (D.D.C. April 27, 2007) (quoting Fed. R. Crim. P. 16(a)(1)(E)).

The defendants received the submission from the government and responded with a motion to compel, complaining that the search was neither thorough, complete, nor a good faith effort as the court had required. The defendants sought to compel the government to provide them with detailed information on their search for both paper and electronic records, and to redo the searches.

Judge Facciola begins his analysis by noting that there is no rule governing criminal procedure as to appropriate format for production of documents. He finds it reasonable in this circumstance to apply Rule 34 of the Federal Rules of Civil Procedure. In his words: “it is far better to use these rules than to reinvent the wheel when the production of documents in criminal and civil cases raises the same problems.”

I am going to skip most of the paper search issues, although they are interesting, even from a civil perspective. Suffice it to say, the government just dumped boxes of paper documents with no folders or organization. That was not sufficient, and Judge Facciola ordered the government to meet with defendants and explain each document produced, the author, author’s title, recipient, date of creation and location of the document (where it came from).

The government opposed defendants’ motion to compel with an affidavit by a Visa Unit Chief which describes the search efforts that she and her five-member staff performed. There is no mention of attorneys participating in this search effort. All personal hard drives and active servers were searched, and the backup tapes going back two weeks were searched (they are erased after two weeks). Conspicuously absent is a description of other possible sources of ESI being searched, such as portable storage devices. Defendants do not, however, object to the limited scope of the search.

The only ESI search effort described is an automated keyword search. The Visa Unit Chief, whose qualifications to perform computer searches are not described, states that she used the following search terms: “early or expedite* or appointment or early & interview or expedite* & interview.” She also states that she had “[t]he Information Management Staff conduct[ ] the search of personal and hard drives because they have access to all drives from the network server, not just shared drives.”

Defendants argued that this ESI search was deficient, and did not comply with Judge Friedman’s order, for three reasons. First, the government did not interview the employees whose computers were searched. In other words, defendants accuse the AUSAs (Assistant United States Attorneys) who represent the government in this case of not discharging a key responsibility of the Zubulake duty, the duty to interview key players. These widely accepted duties include the directive of Judge Scheindlin in Zubulake V to communicate with the “key players” in the litigation, in order to understand how they stored information. Zubulake v. UBS, 229 F.R.D. 422 (S.D.N.Y. 2004). I would add that such direct talks also allow you to understand the language the key players used so that you can more intelligently pick the words to use in a search.

Judge Facciola does not discuss this point directly, except in footnote six where he says that “if the search terms used actually captured everything there was to capture, such interviews would be unnecessary.” With respect, I think this misses the point, since interviews are a good way to develop effective search terms to begin with. When and if the search experts testify in this case, they will probably opine that it is impossible to develop search terms that will “actually capture everything there is to capture.” One hundred percent capture is not possible, not even close. Further, only by interviews and learning the linguistics involved can you hope to obtain the right keywords to improve the capture rate.

The second deficiency alleged was the government’s alleged failure to have any of the government employees search their own computers. Instead, the only search performed was a keyword search by the government’s search team. The employees know their own data the best. Why not ask them to search for it? You could also do a keyword search. How can a search be “thorough” and “complete” unless the custodians are involved in the search? Is that a “good faith effort to uncover all responsive information” as Judge Friedman ordered? Judge Facciola’s order does not discuss this issue, and I suspect that if the defendants renew their motion to compel, and this time support it with expert testimony, that the experts’ opinions will focus on this point.

The third deficiency is the government’s failure to disclose what software it used to conduct the search, and how it ascertained what search terms it would use. Defendants explained that they needed to know the type of software used to be assured that the metadata was properly preserved, and the search and collection were done according to standard forensic procedures. Now you can better see why Judge Facciola deferred ruling pending expert input, and otherwise ordered the parties to meet and confer, and attempt to resolve these issues through discussion with his active participation.

The defendants also alleged that the government failed to preserve ESI; but they did not offer any proof for that allegation, aside from paucity of production, which prompted Judge Facciola to say:

Defendants protest that there are inexplicable deficiencies in the government’s production of electronically stored information, but, as I have indicated in another case, vague notions that there should have been more than what was produced are speculative and are an insufficient premise for judicial action. See Hubbard v. Potter, No. 03-CV-1062, 2008 WL 43867, at *4 (D.D.C. Jan. 3, 2008). Accusations that the government purposefully destroyed what they were obliged to produce or knowingly failed to produce what a court ordered are serious. I must therefore remind the defendants of the wise advice given the revolutionary: “If you strike at a king, kill him.” If the defendants intend to charge the government with destroying information that they were obliged to preserve and produce pursuant to Judge Friedman’s order or the due process clause itself, they must make that claim directly and support it with an evidentiary basis—not merely surmise that they should have gotten more than they did. If they do not do so within 21 business days of this opinion, I will deem any such claim to have been waived.

Of course, the best way to obtain such evidence is through more discovery, a series of depositions actually. I wonder whether that is even possible in criminal cases. Still, you should not jump the gun and allege spoliation before you have proof. You should instead point to the sparsity of production as a justification for discovery on this issue.

Overall, in this opinion, Judge Facciola demonstrates a tolerance for the mistakes made by both sides, and a willingness to move on and look for practical solutions to the problem. Again, Alexander Pope said it well almost 300 years ago: “To err is human, to forgive divine.” Also recall Pope’s lesser known quote: “A man should never be ashamed to own that he is wrong, which is but saying in other words that he is wiser today than he was yesterday.”

That is an everyday reality in the fast changing world of e-discovery. It is especially true in the area of search, where newly developing forms of concept searching may soon make keyword searches obsolete. Judge Facciola mentioned this possibility in Disability Rights Council of Greater Wash. v. Wash. Metro. Area Transit Auth., 2007 WL 1585452 (D.D.C. June 1, 2007). For more information into this area, take a look at the West Legalworks CLE webinar I did with Jason R. Baron ; Doug Oard, Ph.D. – Associate Dean for Research, College of Information Studies, University of Maryland; and Michael S. Simon. The 1.5 hour audio CLE is entitled The e-Discovery Search Quagmire: New Approaches to the Problem of Finding Relevant Needles in the Electronic Haystack and can be downloaded on demand from West. As previously discussed, Doug Oard and Jason Baron and colleagues of theirs have been doing very interesting work in this area, including serious research concerning how keyword and Boolean searches measure against other types of search methods at the scholarly TREC Legal Track. Finally, you should also check out the excellent work being done on search by Anne Kershaw and her non-profit group e-Discovery Institute.

The enlightened attitude usually found in Judge Facciola’s e-discovery opinions is a product of his many years of experience, and his high level of expertise in this area. But be warned, very few judges today have this kind of deep understanding in e-discovery. Most have little or no judicial experience with e-discovery. To make matters worse, very few had any experience in this area before they came on the bench. In the sub-specialty of search, their experience is limited to Google, West or Lexis, which is completely different from e-discovery search in lawsuits and tends to give a distorted perspective.

With no personal background in e-discovery, it is hard for many judges to appreciate the complexity and magnitude of the problems faced today by practitioners. For these reasons, and others, it is foolish to rush into court and expect the same kind of learned results from your judge. It is always better to try to work things out with opposing counsel, and only go to the judge for relief as a last resort.

SUPPLEMENTAL READING: Review the links and cases cited that you have not seen before, then review one of the many articles that have been written about these cases.

Additional Non-Mandatory Supplemental Reading: Study the work and publications of the e-Discovery Institute (EDI). See if you can find out about the search “contest” sponsored by the EDI and Oracle. Who supposedly won? Let Ralph know what you think and why, as he is interested in the public perception of this event.

EXERCISE: Think about what might be problematic about the importation of civil rules of procedure on e-discovery into criminal case and again write up a one paragraph synopsis of your analysis.

Students are invited to leave a public comment below. Insights that might help other students are especially welcome. Let’s collaborate!

Copyright Ralph Losey 2015