There is Only One Way to Rock Search Terms

In Re National Association of Music Merchants, Musical Instruments and Equipment Antitrust Litigation involved a dispute over additional searches of electronically stored information using abbreviations and acronyms for previously agreed-upon search terms.

Instead of turning the discovery dial up to 11, the Court pulled the plug on the additional search terms.  In re Nat’l Ass’n of Music Merchs., Musical Instruments & Equip. Antitrust Litig., 2011 U.S. Dist. LEXIS 145804, 21-24 (S.D. Cal. Dec. 19, 2011).

Search Terms: I Still Haven’t Found What I’m Looking For

The Plaintiffs contented that the Defendants needed to run additional searches of their ESI that included abbreviations and acronyms for the previously agreed-upon search terms. In re Nat’l Ass’n of Music Merchs., at *17.

The Defendants were not cool with the Plaintiffs’ new song request.

If producing discovery was like recording an album, here is the digital booklet overview of the facts:

Before responding to the Plaintiff’s discovery requests, the Defendants asked the Plaintiffs to provide them with search terms.

Plaintiffs argued they could not provide search terms, because the Defendants had not provided them “relevant information.”

After reviewing the Defendants search strings, the Plaintiffs agreed with the Defendants to modify the Defendants search strings to include names to capture “Defendant-to-Defendant” communications.

In re Nat’l Ass’n of Music Merchs., at *17-18.

Once the Plaintiffs began their document review after the Defendants produced their ESI, the Plaintiffs learned abbreviations or acronyms were used in the internal communications.  In re Nat’l Ass’n of Music Merchs., at *18.

The Plaintiffs argued the Defendants’ search did not capture the “agreed-upon universe” of ESI because of the failure to include the internal abbreviations or acronyms.  In re Nat’l Ass’n of Music Merchs., at *18-19.

The Plaintiffs requested each Defendant to

(i) Identify abbreviations or acronyms commonly used within the Defendant corporation for any of the agreed-upon search terms, or affirmatively state that no abbreviations or acronyms are commonly used;

(ii) State whether the Defendant included all identified abbreviations and acronyms in its search for responsive documents; and

(iii) If abbreviations and acronyms were identified but not included in Defendant’s search, re-run its search and produce new responsive documents.

In re Nat’l Ass’n of Music Merchs., at *19.

Search Terms: Don’t Bring Me Down

The Defendants argued the Plaintiffs ignored a basic fact: the terms already searched were selected and agreed upon because they were the terms most likely to produce responsive documents.   In re Nat’l Ass’n of Music Merchs., at *19.

Moreover, the Defendants had already spent a substantial amount of time and money reviewing the prior production in what was supposed to be a limited discovery period.  In re Nat’l Ass’n of Music Merchs., at *19.

Additionally, the new search protocol would require the review of tens of thousands of additional responsive documents. In re Nat’l Ass’n of Music Merchs., at *20.

Further, the Defendants argued the Plaintiffs had “ample opportunity” to add abbreviations to the search methodology, but did not do so.  In re Nat’l Ass’n of Music Merchs., at *20.

Determining Keywords: With or Without You

The Defendants requested the Plaintiffs to meet and confer over search terms before the searches were run, which the Plaintiffs declined.  In re Nat’l Ass’n of Music Merchs., at *20.  Additionally, the Plaintiffs did not raise any objections or concerns over the Defendants proposed search terms they shared by letter with the Plaintiffs. In re Nat’l Ass’n of Music Merchs., at *20.

After the Defendants had begun their document review based on their searches, the Plaintiffs contacted each Defendant to expand their search terms, reaching an expanded search agreement with each Defendant. Id.

Judgment Day

The Court cited the search term truism that “effective search terms cannot be identified in an information vacuum and that a party’s ‘human knowledge’ should be utilized to craft keyword searches that are responsive to the “inherent malleability and ambiguity of spoken and written English.” In re Nat’l Ass’n of Music Merchs., at *21, citing The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189, 201, 209 (Fall 2007).

Moreover, “[W]hile key word searching is a recognized method to winnow relevant documents from large repositories, use of this technique must be a cooperative and informed process.” In re Nat’l Ass’n of Music Merchs., at *21-22, citing In re Seroquel Products Liability Litigation, 244 F.R.D. 650, 662 (M.D. Fla. 2007).

The Court stated that the “preferable” method to reduce challenges to search terms is a “full and transparent discussion among counsel of the search terminology.”  In re Nat’l Ass’n of Music Merchs., at *22, citing The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189, 204 (Fall 2007).

Finally, a Court may limit discovery if a requesting party has had “ample opportunity to obtain the information by discovery in the action,” or “if the burden or expense outweighs its likely benefit.” In re Nat’l Ass’n of Music Merchs., at *22, citing Fed. R. Civ. P. 26(b)(2)(ii), (iii).

The One That Got Away

The Court held the Plaintiff had “ample opportunity to obtain discovery regarding abbreviations and acronyms” and that the burden and expense of searching the abbreviations/ acronyms outweighed its likely benefit.  In re Nat’l Ass’n of Music Merchs., at *22-23, citing Fed. R. Civ. P. 26(b)(2)(ii), (iii).

The Court noted the Plaintiffs had two opportunities to suggest the Defendants include abbreviations/ acronyms in their searches and did not do so.  In re Nat’l Ass’n of Music Merchs., at *23.  As the Court explained:

In the spirit of the conclusions made at the Sedona Conference, and in light of the transparent discussion among counsel of the search terminology and subsequent agreement on the search method, the Court finds it unreasonable for Defendant to re-search documents they have already searched and produced.

In re Nat’l Ass’n of Music Merchs., at *23, citing The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189, 204 (Fall 2007).

The Court further found ordering a re-search of the Defendants’ ESI was overly burdensome, because the Defendants had met & conferred with the Plaintiff and relied on their agreement over search terms in conducting their search and review for responsive ESI.  In re Nat’l Ass’n of Music Merchs., at *23-24.

Based on the above, the Court denied the additional searching of the Defendants ESI.

Bow Tie Thoughts

The search for electronically stored information is constantly a hotly debated topic.  Search generally may be conducted at the collection stage for strategic collections, during processing for data reduction and in document review.  Each stage has a slightly different methodology, but all have the same intent: find what is responsive.

The best time to discuss search terms between the parties is during a Rule 26(f) conference.  There is no shame in holding more than Rule 26(f) conference specifically dedicated to search terms between the parties search professionals, in order to ensure the search is as productive as possible.

Search technology is always improving to save time and costs by efficiently finding responsive electronically stored information.  One upcoming product leveraging new search technology is XERA by iConect.  In full disclosure, I recently did a webinar highlighting XERA and search technology.  iConect has also contracted with me for several continuing legal educational events on search technology.

XERA utilizes faceted search, which is a technique for accessing information organized according to a faceted classification system, allowing users to explore a collection of information by applying multiple filters. Wikipedia, Faceted search, http://en.wikipedia.org/wiki/Faceted_search.

Anyone who has done online shopping or booked airline travel has used faceted search technology.  XERA will use faceted search to improve the search and review of electronically stored information.

For more on XERA, please visit http://www.iconect.com/xera/


  1. Technology has advanced to a point where determining a corpus of likely search terms is feasible, albeit non-trivial. There are open source search log analyzers such as Avi Rappoport’s Perl script highlighted in Louis Rosenfeld’s search analytics book which can quickly digest site search logs and provide a list of the most often searched terms. Utilizing Solr, an open source search engine which powers, among others, iTunes, Netflix, and Zappos enables a party to dump the entire corpus of documents into one search index to enable, out of the box, faceting, more like this, and many other search features. Carrot2 is a clustering suite which finds relevant terms. OpenCalais features entity extraction, allowing a user to pull key terms and concepts out of a corpus of documents.

    Potentially the easiest way to get a sufficient body of keywords is to use site search analytics, whereby the parties can examine what the site’s own users are searching for. This is internally analogous to looking at search terms from Google, Yahoo, or Bing to see what search terms drive traffic to a site. Using site search analytics, and the aforementioned log analyzers tells the parties what search terms and events happen once a user is actually on the site. Merging external search terms from Google et al and internal site search analytics from log files should provide a sufficient list of keywords to get agreement at a Rule 26(f) conference, but if it does not, then using deeper search analysis through the other technologies described above should suffice.

  2. From the perspective of a litigator with a pretty good balance of federal and state cases, I think the “search term” dance is ripe for game-playing – – especially if one side of the “v.” in a case is not particularly savvy with e-discovery issues. Using search terms to help reduce the burden of e-discovery can be very useful, but search terms also present opportunities for overzealous advocates / less than ethical practitioners to use them as a mechanism to hide the ball in litigation. I’ve found that demanding sample sets from a few key custodians from particularly relevant time periods in order to validate search term selection is a good way to short-circuit some of the silliness.