ILTA Day 4 – e-Discovery in the Country Music Capital

Wednesday night at ILTA was set aside as a “free” night for attendees, giving vendors an opportunity to host their corporate parties. Being in Nashville, the locations ranged from Tennessee bourbon bars to the Country Music Hall of Fame. The latter included a tour of the museum, displaying both the instruments and the people who played them, and showing how a guitar in the right hands could produce wonderful music.

One of the most interesting sessions at ILTA was presented by Patrick Oot of the eDiscovery Institute. He discussed the preliminary findings of a very comprehensive study on the accuracy of machine learning technology vs. human review. The study, involving about 1.7 million records from a real litigation, was funded by Oracle and the volunteered resources of the participants. Although the human review results have not yet been published, the machine learning phase is complete, and some interesting information has come to light.

The costs ranged from $45,000 to over $800,000. The results, measured in recall and precision, were similarly all over the chart. The best results were provided by a vendor whose overall cost was in the bottom third, but was not the lowest. The results from the most expensive vendor were rather poor.

​The key piece of information from this study is that the technology is just a tool. Just as with country music, the human process wrapped around the tool is what determined success.