Lawson Lundell, British Columbia’s largest law firm, with over 230 lawyers across four offices in Vancouver, Calgary, Kelowna, and Yellowknife, specializes in business law with deep expertise in corporate matters, real estate, labor and employment, and business litigation. As the firm continued to grow and attract top talent, leadership recognized that staying competitive meant embracing AI not just as an experiment, but as a strategic imperative.

“It’s about attracting and retaining the best and brightest lawyers, especially the next generation, and making sure we have the best tools to equip them from day one.鈥� 听said Clifford Proudfoot, KC, Managing Partner, Lawson Lundell. 鈥淏ut ultimately, it’s about our clients.”

“We’re already seeing expectations for AI use in many of our client agreements” said Derrick Li, COO and CFO, Lawson Lundell.

After piloting multiple AI solutions, Lawson Lundell selected 成人VR视频 for its unique combination of Westlaw鈥檚 authoritative legal content, enterprise-grade security, and seamless integration of the content and AI workflow capabilities. The decision was also influenced by the fact that 80% of the Am Law 100 choose 成人VR视频, as well as their strong Canadian presence and consistent relationship management.

“There’s significant resources required from an R&D perspective to stay ahead in AI,” Li explained. “In the race to dominate Legal AI, I would bet my money on 成人VR视频.”

CoCounsel’s ability to support research, drafting, and document analysis within a unified, intuitive platform allows Lawson Lundell to reduce tool sprawl while maintaining the high standards of accuracy and reliability the firm’s clients expect. With guided onboarding, best-practice templates, and enterprise governance built in, the firm is well-positioned for rapid, responsible adoption across all practice areas.

“I almost feel like a kid at a toy store,” Li said. “I’m most excited about equipping every single articling student, paralegal, and lawyer with AI solutions. This will be the first group that will have AI from day one, and it will be fascinating to see how that changes how lawyers approach client solutions.”

Lawson Lundell plans to roll out CoCounsel firmwide alongside its own internally developed AI tools, creating a comprehensive AI ecosystem designed to serve both legal professionals and support staff. The firm has established an AI committee and pilot groups to ensure thoughtful implementation and maximize value realization.

Version fran莽aise

Lawson Lundell choisi CoCounsel de 成人VR视频 pour offrir une valeur client am茅lior茅e et renforcer la nouvelle g茅n茅ration de professionnels juridiques

Alors que l’intelligence artificielle (IA) transforme le paysage juridique, Lawson Lundell LLP se positionne 脿 l’avant-garde de l’innovation en int茅grant 听CoCounsel de 成人VR视频 脿 l’茅chelle de l’entreprise. Cette d茅cision refl猫te l’engagement du cabinet 脿 茅quiper ses professionnels du droit avec des outils de premier ordre tout en r茅pondant aux attentes 茅volu茅es des clients dans une 猫re ax茅e sur l’IA.

Lawson Lundell, le plus grand cabinet d’avocats de Colombie-Britannique, avec plus de 230 avocats r茅partis dans quatre bureaux 脿 Vancouver, Calgary, Kelowna et Yellowknife, se sp茅cialise dans le droit des affaires avec une expertise approfondie en mati猫re d’entreprise, d’immobilier, de droit du travail,听 et de litiges commerciaux. Alors que le cabinet continue de cro卯tre et d’attirer les meilleurs professionnels, la direction a reconnu que rester comp茅titif signifiait adopter l’IA non seulement comme une exp茅rience, mais comme une n茅cessit茅 strat茅gique.

芦 Il s’agit d’attirer et de retenir les meilleurs et les plus brillants avocats, en particulier la prochaine g茅n茅ration, et de s’assurer que nous avons les meilleurs outils pour les 茅quiper d猫s le premier jour 禄, a d茅clar茅 Clifford Proudfoot, KC, associ茅 directeur de Lawson Lundell. 芦 Mais en fin de compte, il s’agit de nos clients. 禄

芦 Nous voyons d茅j脿 des attentes pour l’utilisation de l’IA dans bon nombre de nos accords clients 禄, a d茅clar茅 Derrick Li, COO et CFO de Lawson Lundell.

Apr猫s avoir test茅 plusieurs solutions d’IA, Lawson Lundell a choisi 成人VR视频 pour sa combinaison unique de contenu juridique autoritaire de Westlaw, de s茅curit茅 de niveau entreprise et d’int茅gration transparente des capacit茅s de contenu et de flux de travail d’IA. La d茅cision a 茅galement 茅t茅 influenc茅e par le fait que 80 % des Am Law 100 choisissent 成人VR视频, ainsi que par leur forte pr茅sence canadienne et leur gestion coh茅rente des relations apr猫s vente.

芦 Il faut des ressources importantes du point de vue de la R&D pour rester en t锚te dans l’IA 禄, a expliqu茅 Li. 芦 Dans la course pour dominer l’IA juridique, je parierais sur 成人VR视频. 禄

La capacit茅 de CoCounsel听 脿 soutenir la recherche, la r茅daction et l’analyse de documents au sein d’une plateforme unifi茅e et intuitive permet 脿 Lawson Lundell de r茅duire la prolif茅ration des outils tout en maintenant les normes 茅lev茅es de pr茅cision et de fiabilit茅 attendues par les clients du cabinet. Avec une int茅gration guid茅e, des mod猫les de meilleures pratiques et une gouvernance d’entreprise int茅gr茅e, le cabinet est bien positionn茅 pour une adoption rapide et responsable dans tous les domaines de pratique.

芦 Je me sens presque comme un enfant dans un magasin de jouets 禄, a d茅clar茅 Li. 听听听听芦 Je suis le plus enthousiaste 脿 l’id茅e d’茅quiper chaque 茅tudiant stagiaire, parajuriste et avocat avec des solutions d’IA. Ce sera le premier groupe 脿 avoir l’IA d猫s le premier jour, et il sera fascinant de voir comment cela change la fa莽on dont les avocats abordent les solutions pour les clients. 禄

Lawson Lundell pr茅voit d茅ployer CoCounsel 脿 l’茅chelle de l’entreprise aux c么t茅s de ses propres outils d’IA d茅velopp茅s 脿 l鈥檌nterne, cr茅ant un 茅cosyst猫me d’IA complet con莽u pour servir 脿 la fois les professionnels du droit et le personnel de soutien. Le cabinet a 茅tabli un comit茅 d’IA et des groupes pilotes pour garantir une mise en 艙uvre r茅fl茅chie et maximiser leur investissement.

Double-Digit Profit Growth

Law firms ended 2024 on a high note, with the average firm experiencing double-digit profit growth for the year. In the fourth quarter alone, profits grew by an impressive 11.5%. This strong financial performance underscores the industry’s ability to adapt and thrive.

Transactional Practices Lead the Way

Transactional practices were a significant driver of growth in Q4, with corporates seeing a 4.0% increase and real estate growing by 3.0%. Counter-cyclical practices slowed after two years of record growth, counterbalancing the increase in transactional work. The report suggests that demand is expected to be more static in the first half of 2025 compared to 2024.

Investment in Technology and Knowledge Management

Law firms have been investing heavily in technology and knowledge management, leading to a 6.9% increase in overhead expenses in Q4. These investments are crucial for maintaining competitive advantage and improving client service. Direct expenses also rose by 6.2% as firms paid out sizeable bonuses to their associates and staff.

Embracing AI and Upskilling

Many law firms are strategically using AI and that is expected to drive sustainable productivity growth and position firms for continued success in 2025.

鈥淟aw firms continue to successfully navigate the dynamic landscape and demonstrate their commitment to serve their clients at the highest level by investing significantly in technology, including generative AI, and the upskilling of their people,鈥� said Raghu Ramanathan, president of Legal Professionals, 成人VR视频. 鈥淟aw firms enter 2025 from a position of strength and appear ready to drive profitability. As the AI market matures, law firms can enhance job satisfaction, well-being, and work-life balance by using AI for routine tasks, freeing up time to focus on their clients鈥� complex needs and driving sustainable productivity growth.鈥�

Download the full report听for additional insights on the factors shaping the future of law firms.

The Emergence of LLMs in Automatic Evaluation

Traditional evaluation metrics in NLG primarily rely on comparing the generated text to reference texts using word overlap measures. These metrics, while useful, often fall short of capturing the nuances of language quality, coherence, and relevance. For example, suppose we have a reference summary 鈥淭he cat is on the mat.鈥� and a generated summary from a model 鈥淎 feline is resting on a rug.鈥�. If we use ROUGE as a metric for the evaluation, ROUGE only considers lexical overlap and cannot capture the semantic similarity between words or phrases. These summaries have the same meaning but would score poorly on ROUGE due to low word overlap.

LLMs have demonstrated an exceptional understanding of language, context, and semantics, making them attractive candidates for evaluating generated text. They can assess factors like fluency, coherence, and even factual accuracy, which are crucial for more sophisticated and context-aware evaluations. For instance, LLMs can be fine-tuned to understand the specific jargon and style of a particular domain, such as medical or legal texts, making them great potential evaluators.

The Promise of LLMs in Automated Evaluation

The recent technological advancements of LLMs have encouraged the development of LLM-based evaluation methods in various tasks and systems. LLMs can offer several advantages in the evaluation of NLG tasks:

• Context-Aware Evaluation: Unlike traditional metrics, LLMs can comprehend the context and generate evaluations that account for the subtleties and intricacies of human language.
• Scalability: LLMs can evaluate large volumes of text quickly and consistently, offering scalability that human evaluators cannot match.
• Reduced Subjectivity: Automated evaluation can minimize the subjective bias that human evaluators might introduce, leading to more consistent and objective assessments.

How to use LLM as an Evaluator

LLM-based evaluators are conceptually much simpler than traditional automatic evaluators for evaluation. While traditional evaluation methods rely on predefined metrics and comparisons to reference datasets, LLM-based evaluators work by directly assessing the generated text.

Figure 1 shows an overview of the LLM-based evaluation frameworks. To evaluate the quality of the text, you embed it into a prompt template that contains the evaluation criteria, then provide this prompt to an LLM. The LLM then analyzes the text based on the given criteria and provides feedback on its quality. This approach bypasses the need for extensive preprocessing and reference comparisons, making the evaluation process more straightforward and versatile.

Figure 2 illustrates the step-by-step workflow of evaluating summaries using clarity as a metric. A document and its corresponding summary are inputs. The summary is then embedded into a pre-formulated prompt template that includes detailed evaluation criteria, such as clarity, defined on a 1-to-5 scale. This prompt is then inputted to an LLM for automated analysis and evaluation. Based on the specified criteria, the LLM evaluator reviews the summary, assigns a score, and provides justification for the rating.

The Limitations of LLMs: Why Humans Are Still Indispensable

Despite the promising capabilities of LLMs, there are significant limitations that necessitate the continued involvement of human experts in the evaluation process, especially for domain-specific tasks:

• Lacking Specialized Domain Knowledge: Domain-specific tasks often involve complex knowledge. LLMs are typically trained as general-purpose assistants, and they still lack specialised domain knowledge. In contrast, subject matter experts bring in- depth domain knowledge obtained by years of dedicated training and education.
• Evolving Knowledge: Especially in fast-evolving fields like medicine and technology, staying up-to-date with the latest information is challenging for static models. Human experts, however, continuously learn and adapt to new knowledge and standards.
• Handling Ambiguities: In specialized domains, the language can be highly ambiguous and complex, and the ability to disambiguate based on deep contextual knowledge is something LLMs still struggle with.
• Ethical and Bias Concerns: LLMs can inadvertently reinforce biases present in their training data. Human oversight is crucial to identify and mitigate these biases, ensuring fair and ethical evaluations.

The Human-in-the-Loop Model: Best of Both Worlds

To harness the strengths of LLMs while addressing their limitations, a human-in-the-loop approach is essential. This combines the efficiency and scalability of LLMs with the expertise and judgment of human evaluators:

• Initial Screening: LLMs can perform initial screenings and provide preliminary evaluations, identifying clear cases of high or low quality.
• Expert Review: Human experts then review and refine these evaluations, focusing on cases that require nuanced understanding or where the LLM鈥檚 assessment is inadequate.
• Continuous Feedback Loop: Feedback from human evaluators can be used to fine-tune and improve LLMs, creating a continuous improvement cycle.

Conclusion

The integration of LLMs into the automatic evaluation of NLG tasks marks a significant step forward in the field of NLP. However, for domain-specific evaluations, the complexity and nuance of human language still necessitate human experts. By adopting a human-in- the-loop approach, we can leverage the best of both worlds: the speed and scalability of LLMs and the depth and discernment of human evaluators. This constructive interaction ensures that we maintain high standards of accuracy, fairness, and relevance in evaluating natural language generation tasks, ultimately driving the field towards more sophisticated and reliable applications.

This post was written by Grace Lee, lead applied scientist at 成人VR视频 Labs (TR Labs).

Note. This work has been done as part of the internship of Hossein A. (Seed) Rahmani at 成人VR视频 Labs (TR Labs).

Foundation of enterprise-wide innovation听

Maria Apazoglou, head of AI, BI & Data Platforms, Technology, recalled early efforts to create a GenAI platform within 成人VR视频, upon which the company鈥檚 GenAI-powered solutions 鈥� including 鈥� would be built. 听

鈥淥ur GenAI platform is designed to democratize access to AI, making it easier for non-technical users to utilize AI effectively,” Apazoglou explained. “In the creation of the platform, we focused on two critical aspects: developing a multi-cloud platform to support various AI models and LLMs, and creating an accessible interface to allow users to self-serve, enabling early experimentation and viability assessment.鈥�听

This approach facilitated rigorous testing by developers and evaluation by subject matter experts (SMEs), ensuring optimal model performance. Today, the platform has 10,000 internal users monthly 鈥� equal to 40% of the population of 成人VR视频 25,000 colleagues.听

Bridging content and customer needs听听

When it comes to responsible AI, human-in-the-loop is critical to 成人VR视频 strategy, with considerations at every stage of design, development and deployment.听听

The company has a team of hundreds of tax and accounting experts 鈥� many of whom are attorneys and CPAs themselves. These SMEs are critical to enhancing the quality of the responses produced by the GenAI models and to creating a system that meets customers鈥� needs. Evaluating results and grading answers is an ongoing and constant process. 听

Catherine Murray, director of Editorial, US Tax & Accounting Commentary, coordinated the SME grading efforts, working with 成人VR视频 developers to interpret the results. She channeled her expertise in tax law, along with her understanding of customers鈥� needs, into ensuring that the models provided valuable and accessible responses.听听

鈥淲orking closely with 成人VR视频 Labs team to conduct experiments and optimize the solution for large language models was an exciting part of the work,鈥� she said. 鈥淭hinking about how customers are going to use the solution to surface our high-quality content was fascinating to be involved in, particularly as GenAI tech is evolving all the time.鈥�听

Melissa Oaks, director of Editorial, Current Awareness, noted the excitement from customers during the alpha and beta testing phases. 鈥淐ustomers were eager to keep the beta version and asked, 鈥榃hen can I buy it?鈥� It鈥檚 thrilling to work on something that has such a significant impact on their daily lives,鈥� Oaks remarked. 听

Oaks noted that other solutions on the market rely on “just the tax code or the primary sources, and that’s really not enough to do tax research.” She said: 鈥淲hat I love about this solution is we鈥檙e providing professional insights and step-by-step guidance on how to actually go do what we鈥檙e saying is the recommended course of action, not just the primary sources. This comprehensive approach sets us apart in the market.鈥�听

Transformative time-savings听

Nancy Hawkins, vice president of Product Management, Research, underscored the product鈥檚 potential to alleviate industry pressures: 鈥淲ith current tax staffing shortages, our solution offers crucial time-savings and work augmentation, especially for non-attorney professionals conducting tax research. It鈥檚 helping customers gain more capacity, whether for work-life balance or business growth.”听

鈥淭he exacting nature of the tax profession means mistakes can have significant consequences, and our customers are hungry for solutions that can bring them time-savings and augment their work,鈥� Hawkins said. 听

Delivering accuracy, efficiency, and simplicity听

Erica Butcher, vice president of Sales & Retention – Strategic Firms, noted the industry’s cautious optimism towards GenAI.听 She said tax professionals are 鈥渂oth cautious and hopeful鈥� about GenAI, and they are looking to the tech to deliver accuracy, efficiency, and simplicity.听

鈥淥verall, we are mainly hearing excitement from customers about the potential for GenAI to support top priorities of improving efficiency and addressing the current talent shortage we鈥檙e seeing across tax and accounting,鈥� Butcher said. “We鈥檝e really listened to our customers, and we built this solution with our customers鈥� feedback and needs in mind to ensure Checkpoint Edge with CoCounsel can deliver meaningful, measurable results.鈥�听

For more on Checkpoint Edge with CoCounsel, visit鈥� or check out鈥�.

鈥淲hen I think about efficiencies in the tax industry, especially given the staffing shortages and the uptick in retirements, efficiencies are so important and a huge driver for the development of our tax tools,鈥� Hawkins said. 鈥淲e know from our research that customers are prioritizing these efficiency gains and looking to technology to really help them unlock them.鈥�

Wong cited findings from the latest 成人VR视频 Future of Professionals report that demonstrated a shift in the tax and accounting industry鈥檚 adoption of AI tools.

鈥淚n contrast to a year ago, where we had the most excitement and the most interest from the legal industry, this year in 2024, we’ve had a much clearer signal from the tax, accounting and audit industry that they are ready and eager to apply some of the AI technology to their work to seek efficiencies,鈥� Wong said.

He added that this shift can help the tax and accounting industry address a labor crunch.

鈥淭here aren鈥檛 enough CPAs or enough people who are willing to jump into the tax and accounting profession, and technology is seen as this sort of savior for this ever-growing pile of work within the industry,鈥� Wong said. 鈥淎I is one additional set of tools to help the industry get to a new frontier of automation.鈥�

Wong and Hawkins noted that AI tools must work in conjunction with the other workflow tools tax and accounting professionals already depend on.

鈥淭he exacting nature of the tax profession is extremely unforgiving when it comes to inaccuracies or mistakes,鈥� Hawkins said. 鈥淎s we put our AI-Assisted Research together, we really had that in mind 鈥� to help with making research more efficient for the new entrants into the industry.鈥�

鈥淗aving an efficient way to research is critical,鈥� Hawkins added. 鈥淭he AI-Assisted Research that we have in Checkpoint Edge really allows someone to come in and simply ask a question, like you would to a trusted colleague down the hall and get an answer back that is easily digestible.鈥�

Hawkins said she hears customers talk about how senior members of firms want to mentor junior members but don鈥檛 have the time. She said firm leaders are excited about how generative AI tools can help 鈥渦pskill more junior members of a firm quickly to have the mature and robust conversations around tax topics.鈥�

Watch the听of the TechConnect series, which brings diverse and dynamic perspectives from all corners of the technology world with thought-provoking questions and conversation.

Jake Heller, head of Product for CoCounsel, 成人VR视频, and Jake Porway, co-founder of DataKind and an NCSC AI consultant, will host the webinar as part of a new AI and the Courts series of monthly discussions. The first session will offer participants a foundational understanding of AI and its potential to enhance the efficiency and effectiveness of the judicial process, highlighting current applications of AI in the court system and the ethical implications of its use.

Launched in June, the consortium is a joint initiative designed to educate the judiciary about the opportunities and challenges of evolving AI and generative AI solutions, enabling judges and legal and court professionals to make informed decisions about adoption and use.

鈥淚鈥檓 thrilled to help bring together the legal industry鈥檚 top AI experts from the courts, law firms, academia, and technology organizations,鈥� said Heller. 鈥淭he pace of innovation in the legal industry is fast and furious, and 成人VR视频 has a long tradition of customer collaboration and leadership in applying cutting-edge technologies to legal research and legal workflows. The AI Policy Consortium for Law and Courts will be a tremendous resource for the judiciary and legal professionals seeking to keep up with the quickly evolving AI and generative AI tools available to augment the practice of law.鈥�

鈥淗aving more than 1,000 participants registered for the webinar speaks to the judiciary and legal profession鈥檚 eagerness to better understand AI solutions and their impact on how legal professionals work,鈥� Porway said. 鈥淥ur consortium is filling a gap in the industry as courts and legal professionals navigate how to best evaluate, adopt, and sanction the uses of generative AI. I鈥檓 excited to introduce the AI and the Courts series with an exploration of the current and future ways AI is used in the court system as well as its challenges and implications.鈥�

The consortium鈥檚 four workstreams 鈥� AI governance and ethics, workforce readiness for AI adoption, rules and practices pertaining to AI, and AI鈥檚 impact on access to justice 鈥� examine the opportunities and risks of AI and generative AI. The second webinar, Ethics of Generative AI: A Guide for Judges and Legal Professionals, will be held on Sept. 18. Future topics will include AI鈥檚 impact on the judicial and legal workforce, how AI can enhance court efficiency, and more.

Register for the webinar . To learn more about on the consortium, read the press release.

What truly stood out to me at ILTACON was the energy and enthusiasm of the event. I was struck by how the collaborative atmosphere among attendees 鈥� ranging from seasoned legal professionals to tech enthusiasts 鈥� created an enriching environment for knowledge exchange and networking.

The conference center was brimming with excited representatives from law firms and vendors showcasing innovative solutions. Everyone was grappling with how 鈥� and when 鈥� generative AI could transform the practice of law.

The insightful sessions on cybersecurity, data privacy, and the future of legal tech trends provided valuable takeaways that are likely to shape the industry in the coming years. Overall, the conference underscored the rapid advancements and the pivotal role of technology in transforming the legal landscape.

CIO insights

I moderated a among G100 and G200 CIOs. The CIOs discussed the evolving role of the document management system, the opportunity for generative AI to drive efficiency improvements now and in the future, the evolving expectations of corporate law departments, and data security practices.

Audience attendees seemed especially interested in the tenor of the conversation around transformative AI solutions. One of the CIOs said that his firm was using CoCounsel as its AI assistant and that lawyers at the firm were eager adopters. He mentioned wanting to buy more seats. That was a fun moment for me!

CoCounsel 2.0

We hosted a number of extremely well-attended events, including customer dinners, an appreciation event, and other celebrations. The opportunity to share perspectives and learn from law firms was invaluable.

I was thrilled to see that the announcement of CoCounsel 2.0 seemed to generate the most excitement among product news. Jake Heller, head of Product for CoCounsel, 成人VR视频, showcased a side-by-side view on the speed of the application, highlighting how CoCounsel 2.0 moves more than 3 times faster than the current version CoCounsel Core. After customers saw that, they were clamoring for access.

On a personal level, I enjoyed seeing so many former colleagues and peers. The legal community is a tight-knit group. It was amazing to walk the halls of the conference centers and catch up with so many great people.

Check out other 成人VR视频听ILTACON recaps, including the legal industry鈥檚 reaction to the听launch of CoCounsel 2.0. and highlights from the 听鈥淗ow Today鈥檚 Lawyers Are Enhancing Their Practice with AI鈥� session.

鈥淭here was a significant focus on the practical application and optimization of generative AI,鈥� said Valerie McConnell, senior director, CoCounsel, 成人VR视频. 鈥淎ALL attendees wanted to learn how to use AI tools in the most effective ways possible. They were looking for strategies to integrate generative AI seamlessly into their workflows. This shift in dialogue reflects a growing maturity in the adoption of AI, highlighting a readiness to leverage these technologies to their fullest potential.鈥�

Rachel Beithon, senior product manager, Westlaw AI & Analytics, 成人VR视频, also noticed attendees鈥� heightened interest in AI tools.

“I was a little concerned that after the last 18 months of near-constant AI discussion in the legal industry that there might be some fatigue setting in,鈥� Beithon said. 鈥淏ut people were so eager to talk more about generative AI solutions.鈥�

Claims Explorer, the new generative AI skill in that launched during AALL, was among the solutions sparking conversations with attendees.

鈥淭hey were absolutely thrilled with Claims Explorer and its use cases,鈥� Beithon said. 鈥淚t was obvious how much time this would save when doing the often-arduous task of exploring potential claims.鈥�

CoCounsel was also generating buzz.

鈥淥ne of the most frequent topics of discussion during our CoCounsel sessions concerned our integrations with Microsoft products and document management systems,鈥� McConnell said. 鈥淭here seems to be a strong interest in using generative AI seamlessly across multiple platforms and systems. Attendees recognized the potential efficiency and productivity gains from having CoCounsel work with the tools they already use daily.鈥�

Generative AI enabling small-scale innovation

The law librarian community鈥檚 readiness to implement generative AI stood out to Zach Warren, manager, Enterprise Content, 成人VR视频 Institute.

鈥淚鈥檓 intrigued by how much courts and government legal departments will play into this tech-enabled future,鈥� Warren said. 鈥淚 saw one presentation given by Judge Keven O鈥橤rady of Kansas where he explained how his court built a 鈥榯riage鈥� tool, helping categorize his family law docket into different pathways based on how much attention different litigants need. It鈥檚 this sort of small investment that can pay huge dividends, and particularly with GenAI allowing for more small-scale innovation, I鈥檓 excited to see what courts and government come up with to do more with less in the coming years.鈥�

Empowering the next generation of legal professionals

Beithon and McConnell were pleased with the feedback on their Day 2 session, 鈥淪ummoning the Magic Words: Crafting Prompts to be an AI Wizard.鈥�

鈥淲e both heard over and over throughout the conference how valuable that session was and how much more excited attendees were to try out prompting techniques in the Westlaw and CoCounsel Core AI skills, and to teach those to other users at their organization,鈥� Beithon said.

鈥淎ttendees were not just interested in refining their own skills but were also focused on how to empower the next generation of legal professionals,鈥� McConnell added. 鈥淭here was a strong emphasis on teaching law students, associates, and new lawyers the art of prompting. This proactive approach to education and training recognizes the importance of ensuring that everyone in the legal field can harness the full capabilities of generative AI.鈥�

Warren agreed, noting AALL鈥檚 optimistic tone.

鈥淭his year鈥檚 conference was shaped with optimism about the role law librarians can play as shepherds of this new paradigm rather than simply impacted by it,鈥� Warren said. 鈥淭his change comes as no surprise to me, as the 成人VR视频 Institute has noted that librarian and research positions are beginning to increase at law firms, reversing a long-standing trend. The crucial role law librarians play has been noted, both anecdotally and in dollars and cents, and law librarians know that now is the time to make their influence felt.鈥�

They offered advice on evaluating a vendor鈥檚 technology expertise, support services, and transparency. Also, they shared how firms and organizations can mitigate the risks posed by acquiring a vendor鈥檚 AI services while maximizing their investment in AI. Below are highlights from their conversation.

Keys to choosing AI vendors

To start the process of assessing potential AI vendors, Yaghoubi emphasized the importance of reviewing their experience and expertise 鈥渢o allow your business to make informed decisions about whether to engage the vendor.鈥�

Wong said that in addition to performance and cost, firms should consider safety and trust factors.

Ramanathan noted it鈥檚 important to consider whether you want a consumer- grade model 鈥� that鈥檚 cheaper 鈥� or a more reliable professional-grade model. He emphasized three criteria to focus on when choosing an AI vendor:

1. 鈥淲hat鈥檚 your philosophy and principles around how AI should be used?鈥� He said asking a vendor this question allows you to see if your firm鈥檚 vision and long-term strategy and roadmap are aligned with the vendor鈥檚 approach.
2. Request a vendor鈥檚 references and testimonials. Ramanathan explained that firms and organizations should ask vendors how many customers are already using their solutions. 鈥淎I is still a game of scale,鈥� he said. 鈥淵ou don鈥檛 want to be the first customer training a model.鈥�
3. Clarify the level of support and training a vendor provides. Ramanathan said this is key to ensuring that all levels of staff are trained and can use the AI solutions constructively.

Wong added that the questions he receives from potential clients focus on data, technology, and talent. He warned that some companies simply repackaged existing large language models (LLMs) for legal use cases without adding much.

鈥淐lients that are working with companies that are building AI have a say,鈥� Wong said. 鈥淭hey can contribute, iterate, and build the products.鈥�

Also, he stressed the importance of working with a vendor that knows how to customize solutions and integrate customer feedback into product development.

How vendors use data

Kapoor asked what customers should consider regarding how vendors use their data. Wong said that understanding the data flow and how the data is processed are key, as well as understanding licensing and data rights, including intellectual property usage rights, cyber risk, and data leakage.

Ramanathan noted encryption standards as well as access control are critical as is demanding transparency from vendors: 鈥淵ou have the right to ask how the data your inputting is used.鈥�

Ramanathan added, 鈥淕ood vendors should have governance systems that answer鈥� details such as where data is stored and who has access to it.

鈥淟ook for transparency鈥� on data output

Wong advised firms to 鈥渓ook for transparency鈥� from vendors, making sure they provide qualitative and quantitative information about the quality of their outputs. He said vendors should be guided by a set of AI principles and should follow a data governance and AI model governance process to mitigate hallucinations and potential risks.

Ramanathan noted that good vendors conduct regular model validation on a periodic basis. He also flagged that professional-grade AI solutions 鈥� unlike consumer-grade AI solutions 鈥� give a sense for the reliability of the answer.

Data output considerations also include encryption standards as well as vendors鈥� privacy and security policies. Ramanathan said a baseline is compliance with standards such as GDPR and CCPA.

鈥淭he privacy and security measures a vendor takes are a result of their philosophy about AI and how to use AI,鈥� Ramanathan said. 鈥淚t gives you a clue as to what you can expect downstream in terms of execution.鈥�

Ramanathan added that vendors should share their risk management framework and enterprise risk framework as well as disclose how frequently they conduct audits and what mitigating actions they put in place.

Wong added that most firms and organizations have 鈥渢ried and tested approaches for technology procurement鈥� that they should apply to assessing AI vendors too.

Lack of AI-Specific SLAs

When exploring initial and ongoing training and documentation, Shokoh asked if AI service-level agreements (SLAs) are similar to those offered for SaaS-type platforms.

Ramanathan said there are elements of SLAs similar to cloud software 鈥渢hat you can and should expect,鈥� such as uptime and maintenance. He noted the hard part is the lack of industry standards for AI-specific SLAs to address issues like response time and accuracy.

In the absence of industry standards, Ramanathan recommended asking questions around issues like product reliability controls and internal testing programs.

Going above the legal requirements

Part of assessing an AI vendor involves anticipating it will adapt to new and changing AI regulations, given the lack of a comprehensive federal law in United States and various states implementing their own guidance.

鈥淭here鈥檚 wide range and little consistency across the market,鈥� Wong said. 鈥淲hat 成人VR视频 has done is look at AI standards in all the markets that we operate in and identify the most restrictive standards. We use a combination of the NIST standards and the EU AI directive as the basis for much of our governance framework.鈥�

Wong added that 成人VR视频 applies this viewpoint to its risk management framework and to its data and AI model governance framework.

鈥淲e projected what the regulation would be rather than look at where the regulation is today,鈥� Ramanathan explained. 鈥淲e proactively defined what we call our Data and AI Ethics principles, which are very hard-coded guidelines that go into engineering our products as well.鈥�

To watch a recording of the webinar, .