by Constantine Pappas on December 03, 2013
As we mentioned in our recent post about proportionality, the costs of litigation and e-discovery are being scrutinized more than ever, and any way to leverage technology must be considered. Relativity Assisted Review has tremendous potential for expediting a review, but success requires some preparation and thought. Before you jump into an Assisted Review project, be sure to address the following considerations to get the most out of your workflow.
Verify the quality of your text.
The most important data consideration is the text in your data set. Assisted Review uses the text of the documents to determine conceptual relationships and make decisions. Poor or minimal text—which you might find in a data set dominated by drawings or numerical spreadsheets—means Assisted Review probably isn’t the right fit.
Use a minimum of 50,000 records.
While it is possible to perform computer-assisted review on data sets of smaller than 50,000 records, the ROI simply isn’t as strong. It may be necessary to review 15,000 documents to complete a computer-assisted review project. If your total population is 1 million documents, the value in that small review is obvious. If the total is 20,000, however, it is more difficult to justify the workflow.
Focus on concept-rich files.
File types have a significant impact on computer-assisted review. Videos, for instance, don't have text and are not useful for training the engine. Similarly, calendar items typically don’t include much text. While a video or event might be important to the outcome of your case, the little text they provide isn’t rich enough for the engine to accurately identify the concepts at the heart of your review.
Use caution with previously reviewed data.
New users often test computer-assisted review by attempting to use previous projects to verify quality. Additionally, case teams may leverage already reviewed data when beginning a project. In both workflows, it’s important to remember that the engine only considers individual documents—not family groups. Family groups can be reconstituted at the end of the process for production purposes. Before then, for the system to accurately categorize, each document needs to have a value based on its textual content alone. This is defined by the four corner rule. When using previously coded data to train or test Assisted Review, it is essential to keep this in mind.
Be mindful of your number of reviewers.
Human reviewers can look at the same document multiple times and come away from each encounter with different ideas. However, consistency is crucial to computer-assisted review because one incorrect coding decision can negatively impact an entire project. Assisted Review helps identify inconsistencies, but it’s best to prevent them in the first place. More reviewers can mean more room for inconsistency, so a very small team is best.
Identify an expected timeline and workflow.
Be sure your review team understands that computer-assisted review is not a magic button. They will need to review documents just like they always have—just on a smaller scale. It’s a safe bet that the project will take a few weeks to complete. Setting expectations is important for preventing frustration and ensuring better results.
Consider your level of reviewers.
Reviewers can make or break a successful review. Though their job titles may vary, reviewers must be very familiar with the details and nuances of your case and know the review specifications. A small, cooperative team of expert reviewers will yield the best results.
Set a target for your level of accuracy.
Assisted Review allows users to statistically measure the quality of a project’s results. It's important to discuss the acceptable confidence level and margin of error—or discuss the recall and precision, depending on your workflow—required to consider your case defensible and complete. Preparing the team for the different options will help keep the project on track.
Assisted Review is a powerful and effective workflow for tackling large data sets. As with any review workflow, it’s key to identify the needs of your case and create a plan of attack before beginning a project. This roll-up of considerations comes from common experiences we’ve seen with clients, so we hope it’s helpful for your next case.
As always, if you have any questions about this list—or if other questions come up along the way—please let us know. We’ll be happy to help.