Feb 26, 2025 | AI
Beauty Is in the AI of the Beholder
A byline by Raghu Ramanathan in Above the Law looks at how speed and accuracy benchmarks misrepresent the real value of legal AI
鈥淲elcome to the era of the AI superlative. While the first two years of generative artificial intelligence (GenAI) development were an all-out sprint to create new models, establish proof-of-concept solutions, and define optimal use cases, the next phase to deliver increased efficiency and better work product to clients in the AI lifecycle will be dominated by marketing as well.鈥�
Raghu Ramanathan, president of Legal Professionals at 成人VR视频, opened with these statements and shared his view on industry benchmarks in an article on Above the Law titled .
He noted as more companies develop AI solutions and start-ups seek capital investment, customers will look for benchmarks to evaluate these tools. 成人VR视频 does see value in benchmarking, however, Ramanathan added benchmarks must measure products the way they鈥檙e designed to be used and should focus on results customers care about.
鈥淭he challenge is that one-dimensional metrics do not offer a reliable representation of the real value of GenAI in the legal research process,鈥� stated Ramanathan. 鈥淣o LLM-based legal research products in the market today provide answers with 100% accuracy, so users must engage in a two-step process of 1) getting the answer and 2) checking the answer for accuracy.鈥�
Chief Legal Operations Officer Meredith Williams-Range from Gibson, Dunn & Crutcher LLP discussed how they are using and seeing results from AI-enabled resources. 鈥淭here is a widespread misperception around how law firms are using AI and how we conduct legal research. We are not bringing in AI and saying: 鈥楪o do all the research and write a brief,鈥� and then replacing all of our junior associates with automated results. We鈥檙e using AI-enabled tools that are integrated directly into the research and drafting tools we were using already, and, as a result, we鈥檙e getting deeper, more nuanced, and more comprehensive insights faster. We have highly trained professionals doing sophisticated information analysis and reporting, augmented by technology.鈥�
Read the full article on , and as Ramanathan concludes, 鈥渢he value of legal AI 鈥� of any technological innovation for that matter 鈥� is in how it gets used in the real world and how well all the different components come together to help lawyers do their jobs more effectively.鈥�
