Last week, Magistrate Judge Andrew Peck (U.S. District Court of the Southern District of New York) issued an opinion and order which, while perfectly serviceable as a standalone decision, even better serves as an intriguing bookend to his landmark February 2012 decision, Da Silva Moore (Da Silva Moore v. Publicis Groupe & MSL Grp., 287 F.R.D. 182 (S.D.N.Y. 2012))—the case that threw the door wide open to the use of computer-assisted review.
The new case, Rio Tinto Plc v. Vale S.A., 2015 U.S. Dist. LEXIS 24996, 8 (S.D.N.Y. Mar. 2, 2015), in some ways is not a ruling at all, but rather an endorsement of the parties’ cooperation in creating mutually agreed-upon protocols. All that was required was judicial signoff. However, “because of the interest within the e-discovery community about [technology-assisted review (TAR)] cases and protocols,” Judge Peck decided to weigh in and tie up some loose ends.
It’s also interesting to note that both parties decided to use computer-assisted review of their own accord. Plaintiff Rio Tinto opted to use our own Relativity Assisted Review via Precision Discovery, while the defendant, Vale S.A., is working with Deloitte's Dynamic Review.
In this ruling, Judge Peck doesn’t only approve of these technologies, but he also revisits, expands upon, and clarifies some of his earlier comments.
Takeaway #1: A court should honor a producing party's request to use technology-assisted review.
Before Da Silva Moore, getting permission to use computer-assisted review from either the judge or the opposition could’ve presented difficulties. While it’s still true that attempting to force the other side to use a specific technology is typically unsuccessful, a producing party can now comfortably consider computer-assisted review as a reasonable option:
In the three years since Da Silva Moore, the case law has developed to the point that it is now black letter law that where the producing party wants to utilize TAR for documents review, courts will permit it.
Takeaway #2: Transparency and cooperation are essential, but sharing seed documents may not be ...
Some of Judge Peck’s most interesting comments concern transparency and cooperation, especially when viewed through the lens of Da Silva Moore. That case, while hugely influential for those that followed, demonstrated a failure to cooperate on how to apply computer-assisted review that likely negated any possible savings which might have resulted from its use.
Since then, nearly every judge we’ve seen opine on this topic has very pointedly directed the parties to cooperate, often pre-emptively so. Rio Tinto may be viewed as the welcome culmination of this trend.
That said, we’ve seen parties in other cases misinterpret Da Silva Moore, taking some of its suggestions to extreme and perhaps unintended places. A prime example involves the sharing of seed documents with opposition as part of the computer-assisted review protocol. Even though the parties in Rio Tinto had already separately agreed on this issue, Judge Peck takes a step back to clarify his position, while acknowledging that other judges have yet to reach a consensus (emphasis mine):
In any event, while I generally believe in cooperation, requesting parties can insure that training and review was done appropriately by other means, such as statistical estimation of recall at the conclusion of the review as well as by whether there are gaps in the production, and quality control review of samples from the documents categorized as non-responsive.
As significant as this point is, however, it’s what he states almost immediately after which will resonate in the courtrooms of the future:
One point must be stressed—it is inappropriate to hold TAR to a higher standard than keywords or manual review. Doing so discourages parties from using TAR for fear of spending more in motion practice than the savings from using TAR for review.
Takeaway #3: … because no one protocol can dictate every computer-assisted review project.
Another past misinterpretation of Da Silva Moore involves efforts to adhere a bit too closely to its prescribed workflow. We often receive questions on whether a specific workflow is court-approved, and case teams sometimes feel compelled to precisely mimic Da Silva Moore’s very specific—and in some ways idiosyncratic—protocol.
In Rio Tinto, Judge Peck reminds us via a quotation from Da Silva Moore that these measures are misguided and unnecessary (emphasis mine):
And as in Da Silva Moore, the Court's approval "does not mean ... that the exact ESI protocol approved here will be appropriate in all future cases that utilize computer-assisted review. Nor does this Opinion endorse any vendor ... nor any particular computer-assisted review tool.”
In reading this, it’s clearer than ever that the workflows behind computer-assisted review projects may be as flexible and customizable as simple review has always been.
A final comment in the opinion references a theory regarding different classifications of computer-assisted review that have been put forth in the last year or so:
If the TAR methodology uses "continuous active learning" (CAL) (as opposed to simple passive learning (SPL) or simple active learning (SAL)), the contents of the seed set is much less significant. (See generally Gordon V. Cormack & Maura R. Grossman, Evaluation of Machine Learning Protocols for Technology-Assisted Review in Electronic Discovery, in Proceedings of the 37th Int'l ACM SIGIR Conf. on Research & Dev. in Info. Retrieval (SIGIR '14), at 153-62 (ACM New York, N.Y. 2014)).
This cursory statement is offered at the end of the discussion on transparency and the disclosure of seed documents. It follows from a theory that, as machine learning occurs, CAL systems may shift away from their seed sets more expediently than other systems. As a result, the initial seed set may be less crucial, because the initial documents may not have as lasting an impact on the project.
In such a situation, a more influential seed set only serves to render the parties’ cooperation all the more impressive. After all, offering a near-useless seed set from early on could be interpreted as an empty gesture. This is one of many reasons that not all protocols—in workflow or transparency—are applicable to every project.
In conclusion, the true winners in Rio Tinto are the parties themselves. Too often, we witness litigants engage in protracted, futile, and unnecessary squabbling. In Rio Tinto, however, a shared interest, some common sense, and a learned judge seem to have won the day.