The emergence of Large Language Models (LLMs) as a potential game-changer in the legal field has sparked both excitement and skepticism. LLMs, powered by artificial intelligence, have been hailed as a versatile tool capable of revolutionizing various legal tasks, including document summarization, translation, research, and contract drafting. However, their use in eDiscovery, particularly in replacing traditional search methods and technology-assisted review (TAR), raises important questions about their effectiveness and reliability.
LLMs, such as transformer models, are computer programs that learn from vast amounts of online text to recognize human language patterns. While they excel at generating text and reproducing general knowledge, their ability to handle case-specific information and nuanced legal issues remains unproven. The format of eDiscovery tasks, which require case-specific knowledge and understanding, may not align with the capabilities of LLMs.
TAR, a well-established method in eDiscovery, relies on supervised machine learning to distinguish responsive documents from non-responsive ones. It has been recognized by the courts as an effective tool, but only when accompanied by appropriate protocols and quality control testing. The validation of TAR methods involves independent blind reviews of a representative sample of documents, ensuring unbiased results.
In contrast, LLMs are largely unsupervised machine-learning methods, trained on unlabeled data. While they can be fine-tuned and improved through additional data and human feedback, their effectiveness in eDiscovery tasks remains uncertain. Claims of superior performance by LLMs in eDiscovery lack empirical evidence and rigorous peer-reviewed studies.
The use of LLMs in eDiscovery has been primarily associated with peripheral tasks, such as summarization, translation, and case-law search. These tasks, while useful, do not address the core challenge of identifying substantially all responsive or material documents. The effectiveness of LLMs in answering specific eDiscovery questions, such as document responsiveness, is yet to be demonstrated and compared to existing methods like TAR.
To establish the credibility of LLMs in eDiscovery, empirical studies similar to those conducted for TAR are necessary. These studies should evaluate the effectiveness of LLMs on a diverse range of requests for production (RFPs) and datasets. Additionally, a statistically sound and well-accepted validation protocol should be implemented to ensure the reliability of LLM-based eDiscovery efforts.
