Your single source for new lessons on legal technology, e-discovery, and the people innovating behind the scenes.

3 Transparency Considerations to Address before Computer-assisted Review

Constantine Pappas

Please note that this post discusses topics from the Sedona Conference Institute’s annual program in late March. The conference was attended by kCura team member Constantine Pappas, licensed attorney and computer-assisted review expert. Panels at the Sedona Conference are subject to a “no attribution rule,” which prohibits attendees from quoting a panelist by name.

During the Sedona Conference Institute’s 7th Annual e-Discovery Program in San Diego, computer-assisted review was a hot topic on several panels. We were inspired to take some of what we heard there and compile a post that takes a deeper look at this issue from the attorney’s perspective.

Computer-assisted review is still a relatively new tool in the legal community, and it’s often held to higher levels of scrutiny than traditional forms of search and review. This has led to discussions of how computer-assisted review will change the strategy and transparency of the e-discovery process. Based on what we heard at the Sedona Conference, we want to highlight three transparency concerns that might be most relevant for our users.

1. Do I need to disclose that I’m using computer-assisted review to meet my discovery obligations?

Some attorneys will argue that this disclosure shouldn’t be compulsory. They are not required to make similar disclosures of other review plans and strategies. They don’t declare openly that they will, for example, break review into two stages: first and second passes, where second pass reviewers will only quality-check the responsive documents identified during the first pass. Isn’t this decision, too, within the attorney’s discretion, only to be questioned when a production is alleged to be incomplete or in bad faith?

The counterargument here is that the use of new technology is a departure from the status quo. Manual review is considered to be the accepted way of reviewing documents, and following this thread, any departures from the norm should be accompanied by some sort of notification. Overall, there doesn’t seem to be any set procedure yet, but that may shift as the issue becomes better understood.

2. Do I need to disclose details of my entire computer-assisted review process?

Suppose that the other side in your case knows you’re using computer-assisted review. Do they have the right to know which software you’re using? What about the vendor, engine, confidence level, margin of error, or other details of your process?

Since keywords are routinely negotiated between disputing parties, one might argue that this is an extension of that courtesy and cooperation. Absent a clear obligation, some case teams may decide to concede these details to help move the process along.

While some might find that argument reasonable, others express concern about which details should be disclosed and which should not. For example, if an attorney decides the project has stabilized after eight rounds, disclosing that fact to opposing counsel could prompt the other side to demand additional rounds. Additionally, other interference might arise with regards to searches used for judgmental sampling, or filters and other index settings.

When considering these issues, it’s important to remember that budgets may play a role in these decisions. Attorneys and their clients often decide to invoke proportionality. Consider a situation where the current overturn rate might be determined sufficient, and lowering it would require another $100,000 of review time. Is the other side really entitled to know about that decision and question the intersection between proportional cost and reasonable effort?

3. Do I need to provide the opposition with my non-responsive example documents?

This last question is a thorny one. Arguments on both sides are compelling and reasonable, but they are diametrically opposed to one another. As a growing trend since Da Silva Moore, the use of computer-assisted review is sometimes predicated on an agreement to supply the other side with all of your seed documents. Because the responsive seeds are going to be produced anyway, there is no issue with that portion of this disclosure. Similarly, privileged documents are going to be withheld regardless, so there isn’t a concern there, either. As a result, the debate in this kind of disclosure stems from the non-responsive example documents.

A defense attorney at the Sedona Conference said she gladly provides her projects’ non-responsive seed documents. Her reasoning was twofold: she doesn’t mind sharing junk documents, and she wants to expedite the process. In fact, she mentioned that many attorneys are willing to provide non-responsive documents to be seen as cooperative, or to avoid being branded as uncooperative.

This form of disclosure may be acceptable to some, but it isn’t always the best fit. For instance, in a government investigation the producing party is typically unconcerned with the investigating agency seeing documents with no bearing on the case. During other types of litigation, however, the opposition may be a competitor—and handing over non-responsive sample documents may result in relinquishing free competitive intelligence. Should that be a required risk for securing the right to use computer-assisted review? If so, attorneys may be dissuaded from using computer-assisted review altogether, despite its inherent benefits.

While these considerations are being ironed out, it’s a good idea for attorneys starting a new computer-assisted review project to be mindful of these transparency concerns. Based on what we heard at the Sedona Conference, it’s important to give them some attention.


Constantine Pappas is a licensed attorney with more than 15 years of legal experience. He has served as in-house counsel and managed both paper and electronic discovery for large-scale lawsuits and government investigations. As a member of Relativity’s customer success team, Constantine helps Relativity users with workflows for text analytics and computer-assisted review.

The latest insights, trends, and spotlights — directly to your inbox.

The Relativity Blog covers the latest in legal tech and compliance, professional development topics, and spotlights on the many bright minds in our space. Subscribe today to learn something new, stay ahead of emerging tech, and up-level your career.

Interested in being one of our authors? Learn more about how to contribute to The Relativity Blog.