Recommended Practices for Document Review Workflow

The process of litigating is simple: Know the law. Get the facts. Apply the law to the facts. Tell a compelling story.

In the paper world, “getting to the facts” was routine and predictable, with generally understood scope, costs and risks. With the explosion of electronic data and the evolution of ever more complex systems, “getting to the facts” is no longer routine and predictable. Data volumes and complexity have driven costs to unsustainable levels, and the risks (omission, spoliation of data) can now adversely impact the merits of a case and the reputations of both client and counsel. The concept of workflow development can help reign in costs and mitigate risks for document review projects.

Building workflow involves:

1. Breaking projects down into common subparts;
2. Analyzing each step to identify goals and options;
3. Building standard approaches for each element that lends itself to repetition;
4. Accommodating flexibility for elements that are fact specific; and
5. Using actual experience to improve the process.

The following recommended workflow and overview of important factors for each phase can help reestablish predictability and transparency for document review projects.

Early Case Assessment ('ECA')

ECA can mean different things to different people. For review workflow, it involves:

1. Knowledge about preservation and collection activity, best recorded in a log or system;
2. Access to information about the nature of the data (volumes, metadata); and
3. Access to substantive content for initial liability and damages assessment.

All this is necessary for estimating project scope, preparing for early meetings of counsel and discovery conferences, and preparing initial risk evaluations for the client. Tools today offer varying functionality. The following features are particularly useful:

• Discussion Threading. Groups related e-mails and identifies characteristics such as common phrases;
• Topic Clustering. Groups related documents based on text content;
• Participant Analytics. Shows lines of communication and document volumes;
• Term Analytics. Shows stem and wildcard variations and document volumes, useful for search term analysis;
• File Analytics. Links files with identical content found in different contexts;
• Filtering. Shows the distribution of the collection or search results across different metadata fields; and
• Near-Dupe Analytics. Permits finding like records or text, useful for batching and quality control.

Users leverage such features to begin identifying groups of data to move through the review workflow. ECA may be done on samples or data sets known to be implicated by the matter, such as e-mail for key players. A system of folders and tags is ideal for managing and documenting the process. Use folders as containers of convenience that can carry case-specific labels, and build tag trees for marking folder contents ' for example, “move to culling” or “hold for further scope negotiations.”

Culling

The goal of culling is to systematically remove populations known or likely non-responsive or low-value data from potential review. The benefits can be profound, as the cost to produce a document through “eyes-on” review can run as much as $3.00-$5.00, and there can be as many as 7,500 documents in one gigabyte of data. Further, industry standards show that significant amounts of processed and/or reviewed ESI is never produced (sometimes as much as 80%-90%).

The process of culling involves the same folder/tag approach outlined above: Identify a potential set of data, copy it to a folder for tracking, analyze the data using ECA features and tag the records appropriately. For example, one might use domain filtering to mark large sets of e-mail with tags such as “likely responsive,” “privileged” or “non-responsive,” based on sender domain or name. The goal is a “reasonable” process, with an eye on proportionality to what is in dispute. The marking serves as documentation of the decision-making process and can be reported, supplemented or changed, if necessary, further bolstering a reasonable process and result. Other metadata useful for culling via filters include date, recipient domain, sender/recipient group/name, document type, custodian and language. For tools without filtering, the process can be done via search, but will take longer and is prone to user error.

Review Set Creation

Building review sets involves executing searches on ECA and culling result sets based on tag values. Unfortunately, many practitioners jump right to search, missing the intelligence that ECA provides and the data volume reduction that culling can yield. Again, use folders as containers and document the contents. A key benefit of foldering is that subsets of data known to need review can begin moving through the process while other data is still being worked in ECA or culling.

A common approach for review set creation is application of search terms. When search lists are agreed upon, it is a simple matter of foldering results, accessing ECA features for possible bulk coding, and moving the remainder through batching and review. Without agreement, search-term development is best done using ECA term analytics and preparation of “hit” reports for gauging the appropriateness of hit volumes and content, and providing transparency given the greater level of judicial scrutiny in this area (see, Victor Stanley Inc. v. Creative Pipe Inc., 250 F.R.D. 251 (2008) (poorly conceived or cursory privilege review risks waiver when privileged documents are inadvertently produced). It is best to start with a small list of terms, test variations, analyze results and further limit scope, when necessary, via proximity and qualifiers.

Batching and Coding

Batching is the creation of review subsets for eyes-on review and application of any required coding. Batching criteria can vary by case, but typically involves prioritizing and routing particular content to particular reviewers. Record all batching logic as part of subfolder labeling to help document and track the batch folder. A batching process such as this allows subsets of documents to move through eyes-on coding while other data may still be in the ECA or culling stages.

Reuse tested and stored tag trees for review status (responsiveness, privilege, confidentiality, redaction), importance (key/important/neutral/irrelevant) and issue codes on each matter through templates and save training time while minimizing coding errors. Fact-specific values such as unique privilege or issue codes can be accommodated through sub-tag values, thereby offering customization within a standard process.

Quality Control

Quality control (“QC”) is necessary at each phase of a review project. Tools that allow filtering of folder contents and tag values can significantly reduce QC time and effort, as values and record quantities will display dynamically, obviating the need for creating reports. QC of review calls is particularly important for accuracy and consistency, and is best handled through sampling. Many tools will generate random samples of specified percentages from search result sets, and the same folder/tag combination can be used to build and mark QC sets. The QC folders/tags also serve as built-in documentation, if necessary, to defend process reasonableness and privilege screening measures, which can help avoid waiver in the event of inadvertent disclosure.

Production

Production involves identifying responsive, non-privileged material, and creating native file and/or image sets for delivery to the other party in the litigation. This may involve branding redactions and/or confidentiality legends, and should always incorporate some form of numbering for page and file identification. A standard workflow (i.e., pulling from the same QC folders only those records with specific tag values) will help ensure that only documents intended to be produced actually are produced.

Conclusion

The benefits of a structured, repeatable review workflow are many and include:

• Reasonableness, through standard and repeatable processes;
• Lower document review project costs;
• Fewer fire drills and delays;
• Faster and better results; and
• Greater defensibility of the
  e-discovery process.

The process of litigating is simple: Know the law. Get the facts. Apply the law to the facts. Tell a compelling story.

In the paper world, “getting to the facts” was routine and predictable, with generally understood scope, costs and risks. With the explosion of electronic data and the evolution of ever more complex systems, “getting to the facts” is no longer routine and predictable. Data volumes and complexity have driven costs to unsustainable levels, and the risks (omission, spoliation of data) can now adversely impact the merits of a case and the reputations of both client and counsel. The concept of workflow development can help reign in costs and mitigate risks for document review projects.

Building workflow involves:

1. Breaking projects down into common subparts;
2. Analyzing each step to identify goals and options;
3. Building standard approaches for each element that lends itself to repetition;
4. Accommodating flexibility for elements that are fact specific; and
5. Using actual experience to improve the process.

The following recommended workflow and overview of important factors for each phase can help reestablish predictability and transparency for document review projects.

Early Case Assessment ('ECA')

ECA can mean different things to different people. For review workflow, it involves:

1. Knowledge about preservation and collection activity, best recorded in a log or system;
2. Access to information about the nature of the data (volumes, metadata); and
3. Access to substantive content for initial liability and damages assessment.

All this is necessary for estimating project scope, preparing for early meetings of counsel and discovery conferences, and preparing initial risk evaluations for the client. Tools today offer varying functionality. The following features are particularly useful:

• Discussion Threading. Groups related e-mails and identifies characteristics such as common phrases;
• Topic Clustering. Groups related documents based on text content;
• Participant Analytics. Shows lines of communication and document volumes;
• Term Analytics. Shows stem and wildcard variations and document volumes, useful for search term analysis;
• File Analytics. Links files with identical content found in different contexts;
• Filtering. Shows the distribution of the collection or search results across different metadata fields; and
• Near-Dupe Analytics. Permits finding like records or text, useful for batching and quality control.

Users leverage such features to begin identifying groups of data to move through the review workflow. ECA may be done on samples or data sets known to be implicated by the matter, such as e-mail for key players. A system of folders and tags is ideal for managing and documenting the process. Use folders as containers of convenience that can carry case-specific labels, and build tag trees for marking folder contents ' for example, “move to culling” or “hold for further scope negotiations.”

Culling

The goal of culling is to systematically remove populations known or likely non-responsive or low-value data from potential review. The benefits can be profound, as the cost to produce a document through “eyes-on” review can run as much as $3.00-$5.00, and there can be as many as 7,500 documents in one gigabyte of data. Further, industry standards show that significant amounts of processed and/or reviewed ESI is never produced (sometimes as much as 80%-90%).

The process of culling involves the same folder/tag approach outlined above: Identify a potential set of data, copy it to a folder for tracking, analyze the data using ECA features and tag the records appropriately. For example, one might use domain filtering to mark large sets of e-mail with tags such as “likely responsive,” “privileged” or “non-responsive,” based on sender domain or name. The goal is a “reasonable” process, with an eye on proportionality to what is in dispute. The marking serves as documentation of the decision-making process and can be reported, supplemented or changed, if necessary, further bolstering a reasonable process and result. Other metadata useful for culling via filters include date, recipient domain, sender/recipient group/name, document type, custodian and language. For tools without filtering, the process can be done via search, but will take longer and is prone to user error.

Review Set Creation

Building review sets involves executing searches on ECA and culling result sets based on tag values. Unfortunately, many practitioners jump right to search, missing the intelligence that ECA provides and the data volume reduction that culling can yield. Again, use folders as containers and document the contents. A key benefit of foldering is that subsets of data known to need review can begin moving through the process while other data is still being worked in ECA or culling.

A common approach for review set creation is application of search terms. When search lists are agreed upon, it is a simple matter of foldering results, accessing ECA features for possible bulk coding, and moving the remainder through batching and review. Without agreement, search-term development is best done using ECA term analytics and preparation of “hit” reports for gauging the appropriateness of hit volumes and content, and providing transparency given the greater level of judicial scrutiny in this area ( see , Victor Stanley Inc. v. Creative Pipe Inc. , 250 F.R.D. 251 (2008) (poorly conceived or cursory privilege review risks waiver when privileged documents are inadvertently produced). It is best to start with a small list of terms, test variations, analyze results and further limit scope, when necessary, via proximity and qualifiers.

Batching and Coding

Batching is the creation of review subsets for eyes-on review and application of any required coding. Batching criteria can vary by case, but typically involves prioritizing and routing particular content to particular reviewers. Record all batching logic as part of subfolder labeling to help document and track the batch folder. A batching process such as this allows subsets of documents to move through eyes-on coding while other data may still be in the ECA or culling stages.

Reuse tested and stored tag trees for review status (responsiveness, privilege, confidentiality, redaction), importance (key/important/neutral/irrelevant) and issue codes on each matter through templates and save training time while minimizing coding errors. Fact-specific values such as unique privilege or issue codes can be accommodated through sub-tag values, thereby offering customization within a standard process.

Quality Control

Quality control (“QC”) is necessary at each phase of a review project. Tools that allow filtering of folder contents and tag values can significantly reduce QC time and effort, as values and record quantities will display dynamically, obviating the need for creating reports. QC of review calls is particularly important for accuracy and consistency, and is best handled through sampling. Many tools will generate random samples of specified percentages from search result sets, and the same folder/tag combination can be used to build and mark QC sets. The QC folders/tags also serve as built-in documentation, if necessary, to defend process reasonableness and privilege screening measures, which can help avoid waiver in the event of inadvertent disclosure.

Production

Production involves identifying responsive, non-privileged material, and creating native file and/or image sets for delivery to the other party in the litigation. This may involve branding redactions and/or confidentiality legends, and should always incorporate some form of numbering for page and file identification. A standard workflow (i.e., pulling from the same QC folders only those records with specific tag values) will help ensure that only documents intended to be produced actually are produced.

Conclusion

The benefits of a structured, repeatable review workflow are many and include:

• Reasonableness, through standard and repeatable processes;
• Lower document review project costs;
• Fewer fire drills and delays;
• Faster and better results; and
• Greater defensibility of the
  e-discovery process.