Fighting over discovery search methodology makes me think of President Richard Nixon’s resignation speech: “Always remember, there are those who hate you. And the only way to keep them from winning is to hate them right back. And then you destroy yourself.”
Attacking a party who used predictive coding to reduce discovery review time to save money and time, only to result in a larger collection of ESI produced upon the requesting party, means the requesting party now has more to review. This is the end result of Progressive Cas. Ins. Co. v. Delaney.
I have never been a fan of agreeing to the use of predictive coding in ESI Protocols. I do not believe such agreements are required at all by the Federal Rules of Civil Procedure. The issue is whether or not a production is adequate. That requires the requesting party offering evidence that the production is somehow delinquent with facts. That discussion does not happen without first having a production.
Parties do not have a veto power over which review application is used by a reviewing party and what features they may or may not use. The requesting party should not attack a review methodology, unless there is a production to challenge. The entire discussion is premature and the issue is not ripe for the Court.
In the case at bar, the original dataset was narrowed by search terms to 565,000 “hits” from the original 1.8 million dataset. This search term methodology had been agreed to in an ESI Protocol by the parties. After one month of document review, the producing party realized that it could take 6 to 8 months to manually review the narrowed dataset for responsiveness or privilege. The party unilaterally decided to use predictive coding instead. Moreover, after telling the other side about their change in technology, motion practice followed. Progressive Cas. Ins. Co. v. Delaney, 2014 U.S. Dist. LEXIS 69166.
The Requesting Party wanted the Producing Party to 1) produce the 565,000 culled from the 1.8 million data set using the parties’ agreed-upon search terms subject to a clawback for privileged documents, or 2) the Producing Party apply the predictive coding methodology to the entire 1.8 million record dataset. Progressive, at *12 and *15.
The Producing Party did not want to do either approach, specifically concerned that searching the entire dataset would result in a larger privilege review.
The Court noted what has been judge-made law on using technology assisted review: Courts require the producing party to provide the requesting party with “full disclosure about the technology used, the process, and the methodology, including the documents used to “train” the computer.” Progressive, at *27-28, citing Da Silva Moore 2012 U.S. Dist. LEXIS 23550 (S.D.N.Y. Feb. 24, 2012).
The Court ordered the producing party to follow the original agreed to protocol and produce the “hit” documents to the Requesting Party within fourteen days without further review. Progressive, at *30.
The Court stated that following the Requesting Party’s protocol, the cost of review would be shifted to them. Progressive, at *31. Moreover, the Requesting Party believed they had the manpower to complete the review within one month. Id.
The Requesting Party could apply privilege filters before production and produce a privilege log. Id.
Bow Tie Thoughts
I do not encourage clients to have ESI Protocols that limit their ability to review discovery efficiently. I also strongly argue against the idea you need approval from the opposing party on what search methodology you can use to identify responsive discovery. It is the producing party’s right to use keywords, concept search, visual analytics, email threading, clustering, find similar, or any other form of technology-assisted review. That includes predictive coding. The requesting party does not get a veto power over what technology the producing party can use. The requesting party has “the burden of proving that a discovery response is inadequate.” Abt v. Jewell, 2014 U.S. Dist. LEXIS 50766, 12-14 (D.D.C. Apr. 11, 2014), citing Barnes v. D.C., 289 F.R.D. 1, 6 (D.D.C. 2012) and Equal Rights Ctr. v. Post Props., Inc., 246 F.R.D. 29, 32 (D.D.C. 2007).
You cannot prove a discovery response is inadequate if you do not have a discovery response. The entire idea of attacking the use of predictive coding is premature if there is no production to be reviewed.
The ugly wrinkle in this case is the search and production methodology mandated in the ESI Protocol agreed to by the parties. I strongly encourage parties to not agree to actual technological steps to be used, because it limits the ability to conduct discovery in a cost effective manner. These agreements are often enacted without the advice of an eDiscovery Consultant who knows how the actual technology works.
What parties should discuss and codify in an ESI Protocol is the subject matter of the lawsuit. Who are the key players? What are the relevant date ranges? What are the terms of art used by the parties? What is the data actually making up the case? Those, and many others, are the topics parties need to agree to, not whether you can use visual analytics to identify date ranges or data clustering to determine what was relevant.
The use of predictive coding has been twisted because lawyers think they need permission to use it. They do not. The burden is on the opposing side to prove a production was inadequate, not the fact the producing party used one form of technology-assisted review over another.
Joshua,
Thank you for bringing the topic up for discussion.
I agree parties should not have to get approval for the specific discovery process they choose to use. However, I believe that this issue belies the underlying problem facing legal discovery procedures in an environment that is shifting from a traditional, fairly irrational attorney expert-based system to one that acknowledges that decisions, assertions and certifications should be supported by available data. In an era of “big data”, current procedures increasingly create a facade of fair resolution of disputes while actually providing a system that does less to assure it in practice.
Therefore, I’d argue that the truly progressive view is an evidence-based one that requires disclosure and approval of what data about the production will be provided with it. It would require the producing party in every instance, no matter what methodology employed, to indicate the metrics to be generated that will be provided with the production to assure that attorney certifications are supported by data. (A less than ideal example of the generation of metrics can be seen in the Biomet Defendant’s motion papers http://bit.ly/1pmd3r4) Parties certainly have the right to choose the discovery process that they prefer, but the process would be fundamentally more rational if parties had to back up their certification signatures with data supporting them. The size and diversity of potentially relevant information now present in many disputes is such that exclusive reliance upon the unilateral judgment of an attorney about the adequacy of discovery efforts on behalf of his/her client has increasingly little logical to support it.
In line with this view, logic and efficiency dictate that requesting parties have the right and do need to request upfront not the disclosure of specific “machinery” or processes used but rather the type of metrics that the producing party will provide about its production.
I realize that this is not current practice. However, I would argue that as the legal process slowly adopts some of the progressive data-driven approaches now becoming embedded elsewhere, it is inevitable that procedural rules will evolve to recognize the need for robust objective data to validate discovery efforts.
Thanks,
Gerry
I haven’t looked at the case yet; so, I am commenting on the basis of your post alone. This sounds like one of those “damned if you do/don’t” scenarios. Once the producing party revealed that they’d used keywords to cull the dataset before predictive coding, the requesting party was on notice that the production effort would largely be a costly charade going forward. So, if they sat back and waited for a deeply flawed production, they risked the court asking, “Why didn’t you speak up before they wasted all the time and money?” Plus, everyone now must wait for the do-over. yet, if the requesting party challenges the process before the money and time is wasted, you say they are acting prematurely. Respectfully disagree,
What is the lesser of the two evils here? Why wait for the smell of burned flesh to in the air before saying, “Hey, that stove’s HOT!” A process that’s flawed from the start isn’t going to miraculously heal itself because we wait to see that Garbage In actually DOES produce Garbage Out.
Personally, I think that the goals of FRCP Rule 1 are better realized when parties raise issues while processes can be corrected and savings can still be realized. Yes, it may be the producing parties’ right to use any method that reasonably suffices to produce responsive, non-privileged information while culling that which is not responsive. But, it is *not* the parties right to engender delay and burden an opponent with a flawed production. Put another way, a producing party can unilaterally employ almost any method that gets the job done correctly; but getting the job done correctly is not optional. They must act timely and reasonably to make it happen. When we see the engines are on the same track and heading toward each other fast, we shouldn’t have to wait for the train wreck.
I greatly appreciate your observation that the Federal Rules (or any other rules of civil procedure of which I am aware) do not require parties to agree on the review methodology beforehand. I think that this is an element thrust into the common lore by a few judicial opinions, and I think that it is a generally bad idea. As you state, the issue is whether or not I have produced the information requested, not how I came to find that information. Given the many articles and posts recently about how it is impossible to compare different predictive coding applications and methodologies case-to-case, achieving agreement on an application or methodology in most cases seems like a virtually impossible task. I agree wholeheartedly that our focus should return to the substance of what has been produced, and not on how it was identified – at least in the first instance. If there is some indication that information is missing or has been withheld, then that is a separate issue, but one that should come later in the process.