I remember visiting the La Brea Tar Pits as a child. Opinions that focus on whether or not parties should use Technology Assist Review always makes me think of mastodons drowning in tar.
Asking a court for permission to use predictive coding, or trying to force a party to use predictive coding, is nothing but asking a judge to grant an impermissible advisory opinion. No party should attack another side’s production until there is actually a production to review.
We return again to the predictive coding tar pits in Hyles v. New York City. Magistrate Judge Andrew Peck knows how to not get trapped in the issue of ordering an opposing party to use predictive coding.
In Hyles, the plaintiff sought an order for the responding party to use predictive coding instead of keywords. Hyles v. New York City (S.D.N.Y. Aug. 1, 2016) 2016 U.S. Dist. LEXIS 100390. Judge Peck explained why the answer to this motion was no:
- The Responding party is best situated to decide how to search for and produce ESI responsive” to the discovery requests; and
- Plaintiff counsel admitted they had no authority to support the argument to compel the responding party to use predictive coding.
Hyles, at *7-8, citing Sedona Principle 6.
Judge Peck drove home the point from Dynamo Holdings, which stated:
[T]he Court is not normally in the business of dictating to parties the process that they should use when responding to discovery. If our focus were on paper discovery, we would not (for example) be dictating to a party the manner in which it should review documents for responsiveness or privilege, such as whether that review should be done by a paralegal, a junior attorney, or a senior attorney. Yet that is, in essence, what the parties are asking the Court to consider — whether document review should be done by humans or with the assistance of computers.
Hyles, at *8-9, citing Dynamo Holdings Ltd. P’ship v. Comm’r of Internal Revenue 143 T.C. 9, 2014 WL 4636526 at *3 (2014).
Judge Peck explained that the requesting party could not force the responding party to use TAR. Hyles, at *9. Furthermore, the standard for productions is “not perfection, or using the ‘best’ tool, but whether the search results are reasonable and proportional.” Id., citing Cf. Fed. R. Civ. P. 26(g)(1)(B).
The Court concluded its order with the following:
To be clear, the Court believes that for most cases today, TAR is the best and most efficient search tool. That is particularly so, according to research studies (cited in Rio Tinto), where the TAR methodology uses continuous active learning (“CAL”), which eliminates issues about the seed set and stabilizing the TAR tool. The Court would have liked the City to use TAR in this case. But the Court cannot, and will not, force the City to do so. There may come a time when TAR is so widely used that it might be unreasonable for a party to decline to use TAR. We are not there yet. Thus, despite what the Court might want a responding party to do, Sedona Principle 6 controls. Hyles’ application to force the City to use TAR is DENIED.
Hyles, at *9-10.
Bow Tie Thoughts
I think it is wrong to 1) think court-approval is required to use TAR; 2) challenge the other side for using TAR like it is some form of witchcraft; and 3) force an opposing party to use the review technology the requesting party prefers.
Asking court approval for TAR is right up there with asking courts to conduct reading comprehension tests for attorneys conducting the review. It is inappropriate. A producing party is in control of their document review strategy. A requesting party should not challenge a production before it is even made. What is the dispute before the court? The answer is nothing, because there is nothing for the court to adjudicate.
Few opinions on TAR address what TAR is being used. No one seeks permission for email threading. No one petitions the court for near-de-duplication. No one does that, because it is “crazy talk.”
Opinions on predictive coding rarely discuss what predictive coding is being used or how it works. For example, I help with document review using Everlaw. Their predictive coding system is a regression-based learning algorithm. The “how it works” is explained in detail on their help page. The short explanation is a model is built from identified documents in a training set (or “it learns from reviewers”). An algorithm then applies the model to all documents in a case to generate predictions through a linear classification system. All of this works in the background as attorneys issue code to the issues in their case or production requests.
In theory, if the document review in this case were hosted in Everlaw, the documents identified by keywords and issue coded as “relevant” could be run in the prediction engine in the background. Lawyers conducting review with keywords alone would still have the option of checking what the predicted results would be from their keyword review. The choice on whether to use keywords or TAR is not mutually exclusive.
A project manager could also take the issue-coding plan that determines relevance for different issues, and create different prediction models in Everlaw based on that coding. The results from this prediction model should give attorneys focused results on possibly responsive ESI based on their keyword review.
There are many types of TAR. Attorneys should specify what type of TAR that is at issue in a dispute, how it operates, and what the attorney workflow would be. This would do wonders to demystify Technology Assisted Review, so attorneys do not approach TAR like a mastodon about to be trapped in a tar pit.