I could go on and on about the savvy, creativity, selflessness, and fun of the folks out there making the most of Relativity every day. But I don’t have to, because their work speaks for itself.
In addition to solving their team’s everyday e-discovery challenges, many of these professionals are leading the charge on evangelizing best practices among the entire legal community.
Take Kate Bauer, for instance. She recently wrote an extensive report entitled “Leveling the Playing Field: Playing Technology-Assisted Review by the [Federal] Rules.” The piece is a well-researched, deep look at the history of technology-assisted review (TAR), why it’s made waves in the legal world, and its influence on the modern practice of law.
Kate shared the paper with our team during Relativity Fest this year, and we found it to be a phenomenal resource for attorneys looking to better understand TAR—from beginning to end—and the impact it can have on their practices (as well as their bottom lines).
We encourage you to read the paper for yourself (and share it with any of the skeptics in your life), but in the meantime, here are three key takeaways we pulled from Kate’s work.
1. The Federal Rules have evolved, and legal practice has evolved with them.
“Long before there was TAR, there was gamesmanship. Gamesmanship at trial, particularly its impediments to accurate adjudication, was a significant factor motivating the adoption of the Federal Rules of Civil Procedure in 1938,” Kate explains in the first section of the paper.
“These Rules shifted litigation from limited to liberal pretrial discovery. It was hoped that the exchange of information facilitated by Rules 26 through 37—rather than surprise attacks and tricks at trial—would help courts adjudicate claims on the merits,” she goes on. “In particular, the new Rules were intended ‘to secure the just, speedy, and inexpensive determination of every action and proceeding.’”
Kate outlines a history of rule changes, new tactics, and more rule changes since that first set of rules hit the stage in 1938. Where we’ve landed is a world with heavy discovery requirements on the front-end of litigation juxtaposed against an obligation to stay proportional and “inexpensive.” Rule makers ended up here in the best interests of clients and their attorneys, but it can be a difficult balance to strike in a world of proliferating data volumes. Still, Kate argues, the rules are well adapted to today’s landscape—especially with a little help from the right technology.
2. Technology has been cast as a foe for legal teams, but it can (and should) be a friend.
It’s technology that got us into this position, sure. Digital data proliferation has had enormous impacts on discovery and litigation. But, as Kate argues, it’s technology that can get us out of this predicament of proportionality, too.
“In the Digital Age, discovery expenses soared yet higher as the advent of computers facilitated document creation and storage, yielding more documents in need of review. This increase, however, was a miniscule compared to the avalanche of electronically stored information the Information Age brought,” she said.
The numbers are staggering:
The review of digital documents, commonly referred to as e-discovery, brought discovery expense to a crisis point. Computers made it easy to accumulate and store large amounts of data. In 1990, a typical gigabyte of storage cost about $20,000; as of 2013, two-terabyte drives readily sell for less than $70, or 3.5¢ per gigabyte, with even lesser rates charged for hosting gigabytes in the “cloud.” Although a gigabyte of information costs little to store, it can easily cost $30,000 to review. Large American companies back up terabytes of information daily, with each terabyte being equal to approximately a billion pages.
In a section delightfully entitled “Technology: The Cause of (and Solution to) Document Discovery Volume,” Kate discusses why more technology (in this case, TAR) is the solution to more technology.
Whereas traditional, manual review methods and keyword searching have yielded less-than-ideal results and unimpressive accuracy rates, “In contrast to the mixed bag of keyword search results, TAR allows attorneys to quickly and accurately separate the document collection into relevant and irrelevant predictions,” Kate explains. “Attorneys can then focus their attention on manually reviewing the relevant predictions. Rather than evaluating irrelevant predictions individually, documents predicted to be irrelevant can instead be statistically sampled to see if relevant documents have been missed.”
3. Traditional review methods are becoming less palatable as data volumes grow.
Kate’s assessment of traditional review methods in the face of extreme data volumes is unforgiving: “Though many attorneys still rely on human review of each document and/or keyword searches, empirical studies have revealed that, in addition to being formidably expensive, both of these methods miss many relevant documents while requiring review of many irrelevant documents.”
She cites several sources to support this criticism.
In an oft-cited study by David Blair and M. E. Maron, researchers asked a team of paralegals to use iterative keyword searches to locate relevant documents in a large subway crash case. The paralegals were allowed to adjust their searches as they worked, and were asked to inform the researchers when they believed they had located 75% of the relevant documents. When the paralegals declared the project finished, the researchers found that the keyword searches had only identified 20% of the relevant documents.
…
Problems with keywords are just the tip of the iceberg. Research shows that manual human review is itself fraught with inconsistency. For example, in a study of information retrieval professionals, assessors of relevance disagree about whether a document was relevant at least as often as they agreed. A recent study revealed attorneys fare even worse, agreeing on relevance, at best, a mere 28.1% of the time and, at worst, 15.1% of the time.
In the end, when millions of documents are at stake, exhaustive, manual review is often too expensive to support a proportional strategy, too inconsistent to find the proverbial needles in the haystack, or both.
Next Step: Cooperation Considerations
“Leveling the Playing Field: Playing Technology-Assisted Review by the [Federal] Rules” walks through a wealth of information about the efficacy of TAR, its benefits to modern litigation, and the ways the current Federal Rules enable its use. The paper does an excellent job dispelling doubts and addressing critics of the technology.
Past that, if the paper delivers a call to action, it’s this: teams who want to forward the use of TAR need to learn how to strike a balance on cooperation while employing it. Working well with clients, courts, and even opposing counsel will help ensure the technology and its proof points are taken seriously and applied effectively.
But that doesn’t mean elevated disclosure requirements—such as instructions for a party using TAR to disclose non-responsive samples to opposing counsel—are the answer. Rather, flexible, data-driven, and transparent behavior by both parties can help make e-discovery efforts more effective while preserving client expectations of privacy.
Take a look at the paper for yourself, and let us know your thoughts in the comments or on Twitter.
