成人VR视频

Raghu Ramanathan, president, Legal Professionals, 成人VR视频, and David Wong, chief product officer, 成人VR视频, shared their insights on evaluating AI vendors during a with Morgan Lewis partner Rahul Kapoor and associate Shokoh Yaghoubi.

They offered advice on evaluating a vendor鈥檚 technology expertise, support services, and transparency. Also, they shared how firms and organizations can mitigate the risks posed by acquiring a vendor鈥檚 AI services while maximizing their investment in AI. Below are highlights from their conversation.

Keys to choosing AI vendors

To start the process of assessing potential AI vendors, Yaghoubi emphasized the importance of reviewing their experience and expertise 鈥渢o allow your business to make informed decisions about whether to engage the vendor.鈥�

Wong said that in addition to performance and cost, firms should consider safety and trust factors.

Ramanathan noted it鈥檚 important to consider whether you want a consumer- grade model 鈥� that鈥檚 cheaper 鈥� or a more reliable professional-grade model. He emphasized three criteria to focus on when choosing an AI vendor:

1. 鈥淲hat鈥檚 your philosophy and principles around how AI should be used?鈥� He said asking a vendor this question allows you to see if your firm鈥檚 vision and long-term strategy and roadmap are aligned with the vendor鈥檚 approach.
2. Request a vendor鈥檚 references and testimonials. Ramanathan explained that firms and organizations should ask vendors how many customers are already using their solutions. 鈥淎I is still a game of scale,鈥� he said. 鈥淵ou don鈥檛 want to be the first customer training a model.鈥�
3. Clarify the level of support and training a vendor provides. Ramanathan said this is key to ensuring that all levels of staff are trained and can use the AI solutions constructively.

Wong added that the questions he receives from potential clients focus on data, technology, and talent. He warned that some companies simply repackaged existing large language models (LLMs) for legal use cases without adding much.

鈥淐lients that are working with companies that are building AI have a say,鈥� Wong said. 鈥淭hey can contribute, iterate, and build the products.鈥�

Also, he stressed the importance of working with a vendor that knows how to customize solutions and integrate customer feedback into product development.

How vendors use data

Kapoor asked what customers should consider regarding how vendors use their data. Wong said that understanding the data flow and how the data is processed are key, as well as understanding licensing and data rights, including intellectual property usage rights, cyber risk, and data leakage.

Ramanathan noted encryption standards as well as access control are critical as is demanding transparency from vendors: 鈥淵ou have the right to ask how the data your inputting is used.鈥�

Ramanathan added, 鈥淕ood vendors should have governance systems that answer鈥� details such as where data is stored and who has access to it.

鈥淟ook for transparency鈥� on data output

Wong advised firms to 鈥渓ook for transparency鈥� from vendors, making sure they provide qualitative and quantitative information about the quality of their outputs. He said vendors should be guided by a set of AI principles and should follow a data governance and AI model governance process to mitigate hallucinations and potential risks.

Ramanathan noted that good vendors conduct regular model validation on a periodic basis. He also flagged that professional-grade AI solutions 鈥� unlike consumer-grade AI solutions 鈥� give a sense for the reliability of the answer.

Data output considerations also include encryption standards as well as vendors鈥� privacy and security policies. Ramanathan said a baseline is compliance with standards such as GDPR and CCPA.

鈥淭he privacy and security measures a vendor takes are a result of their philosophy about AI and how to use AI,鈥� Ramanathan said. 鈥淚t gives you a clue as to what you can expect downstream in terms of execution.鈥�

Ramanathan added that vendors should share their risk management framework and enterprise risk framework as well as disclose how frequently they conduct audits and what mitigating actions they put in place.

Wong added that most firms and organizations have 鈥渢ried and tested approaches for technology procurement鈥� that they should apply to assessing AI vendors too.

Lack of AI-Specific SLAs

When exploring initial and ongoing training and documentation, Shokoh asked if AI service-level agreements (SLAs) are similar to those offered for SaaS-type platforms.

Ramanathan said there are elements of SLAs similar to cloud software 鈥渢hat you can and should expect,鈥� such as uptime and maintenance. He noted the hard part is the lack of industry standards for AI-specific SLAs to address issues like response time and accuracy.

In the absence of industry standards, Ramanathan recommended asking questions around issues like product reliability controls and internal testing programs.

Going above the legal requirements

Part of assessing an AI vendor involves anticipating it will adapt to new and changing AI regulations, given the lack of a comprehensive federal law in United States and various states implementing their own guidance.

鈥淭here鈥檚 wide range and little consistency across the market,鈥� Wong said. 鈥淲hat 成人VR视频 has done is look at AI standards in all the markets that we operate in and identify the most restrictive standards. We use a combination of the NIST standards and the EU AI directive as the basis for much of our governance framework.鈥�

Wong added that 成人VR视频 applies this viewpoint to its risk management framework and to its data and AI model governance framework.

鈥淲e projected what the regulation would be rather than look at where the regulation is today,鈥� Ramanathan explained. 鈥淲e proactively defined what we call our Data and AI Ethics principles, which are very hard-coded guidelines that go into engineering our products as well.鈥�

To watch a recording of the webinar, .