Quick Hits
- AI tools can streamline employee lifecycle events, such as recruiting, hiring, evaluations, and productivity monitoring, but they can also create privacy, effectiveness, and discrimination risks.
- Employers must assess applicable local, state, federal, and international laws, conduct appropriate bias and privacy reviews, provide required notices and accommodations, and maintain meaningful human oversight.
- Companies that use AI across jurisdictions may want to establish ongoing compliance, monitoring, vendor management, and data protection practices as AI-related employment law continues to evolve.
There is an ever-increasing number of exciting AI tools that are available to companies to perform employment-related tasks, such as recruiting, hiring, employee evaluations, evaluating employee productivity and safety programs, and even offboarding employees. For example, AI tools can screen applications, parse resumes, rank candidates, create employee evaluations, evaluate pay equity, monitor drivers for fatigue, and spot fake candidates.
There are numerous advantages to using these technologies, but there is an important analysis that must be conducted when the AI is replacing human decision-making. If the technology will rank candidates and the professional who oversees reviewing resumes is unable to review all resumes, then the AI has effectively chosen the individuals who are eligible for the job. This would mean that the human decision-maker has been replaced by AI. Notably, however, many laws are not limited to tools that fully replace a human decision-maker; some also reach tools that substantially assist, materially influence, or facilitate human decisions.
The main concerns associated with these technologies center around:
- data privacy concerns if a third-party service provider has access to the data,
- effectiveness of the technology (i.e., is it doing what it claims it should do), and
- bias (whether the AI potentially creates a result that could bias a specific group).
For instance, if the AI notes that historically individuals who live in a particular neighborhood have more leadership roles and it begins choosing candidates based on neighborhood, and neighborhood correlates with a protected characteristic such as race or national origin, the tool could have an unintended discriminatory effect on individuals in that protected group. If a technology is aimed at evaluating productiveness, it might not consider that an employee has accommodations in place associated with a disability. For technologies that have an impact on employees, such as evaluations that lead to raises or being chosen for a promotion, the potential consequences, including exposure to discrimination claims, can be significant.
Federal, State, and Local Laws
Although AI use is relatively new, many laws are worth considering when implementing any AI tool, such as federal and state laws that address discrimination (e.g., Title VII of the Civil Rights Act of 1964, the Americans with Disabilities Act (ADA), and the Age Discrimination in Employment Act), accommodations, and conducting background checks (e.g., the Fair Credit Reporting Act).
In addition to these existing laws that may apply to any new AI tool, there are specific state and local laws and regulations that could require notice of use, bias audits, and risk assessments. For example, New York City’s Local Law 144 requires employers that use automated employment decision tools for hiring or promotion to obtain an independent bias audit, publish a summary of the results, and notify candidates and employees. California’s civil rights regulations allow evidence of anti-bias testing, or its absence, to be considered in discrimination claims, and its privacy regulations will require risk assessments, notices, and opt-out and access rights for certain automated employment decisions beginning in 2027. Illinois prohibits using AI that has a discriminatory effect or using zip codes as a proxy for protected classes and requires notice of AI use, and Colorado, Connecticut, and Texas have also enacted AI-related requirements.
Other jurisdictions require, or will soon require, companies to provide individuals with an explanation as to how their AI tool works, and in particular what factors and criteria were used in making the determination. Individuals may also have the right to request that a human review the results in certain jurisdictions.
It is reasonable to anticipate that AI use will be subject to additional laws and regulations moving forward, and litigation is already active, including pending cases testing whether AI vendors can be liable alongside employers for discriminatory screening and whether AI-generated applicant scores trigger federal consumer reporting laws. Because bias testing results may become evidence in such disputes, companies may want to conduct bias audits at the direction of counsel to help preserve attorney-client privilege, although privilege may not cover underlying data or results that must be published, such as New York City’s bias audit summaries.
Bias testing also is not a one-time exercise. Tools can change as they are updated or as the data they process shifts, and New York City, for example, requires a bias audit conducted no more than one year before a tool is used. Companies adopting AI for employment decisions may therefore want to plan for ongoing bias monitoring as a recurring cost of using these tools.
Questions to Consider Before Purchasing an AI Tool
In considering how a company will use AI, understanding which obligations may attach to the particular AI tool and the proposed use can help inform the decision. Questions companies may want to consider before purchasing an AI tool can be grouped into two categories: “Regulatory scope, effectiveness, and fairness” and “Data privacy and security.”
Regulatory scope, effectiveness, and fairness
- Is the proposed use subject to regulation? If so, what are the regulatory obligations? Can the company comply with the regulations? Does the proposed AI use warrant the effort and expense to comply with the regulatory obligations? Is the company prepared to budget for ongoing bias monitoring?
- How does the tool work? Is it possible to explain the results to individuals or to have the results reviewed by a human?
- Is the tool effective, i.e., is it doing what it is intended to do?
- Does the tool discriminate, or indicate bias against individuals in a particular group? Will the company test for bias with its own data, is there a plan for ongoing monitoring, and is the testing structured to help preserve privilege?
- Can the company provide reasonable accommodations, such as an alternative assessment or process, for applicants and employees with disabilities who may be disadvantaged by the tool?
- If the tool is provided by a vendor, do the contract terms address bias risks, cooperation with audits, and allocation of liability?
Data privacy and security
- Does the tool comply with applicable data privacy laws?
- Has a data protection impact assessment or similar privacy risk assessment been conducted, as required under applicable law, to evaluate the risks the tool poses to individuals whose data is processed?
- Does the tool involve automated decision-making or profiling that produces legal or similarly significant effects on individuals? If so, can the company provide meaningful information about the logic involved and ensure human oversight?
- Does the company have adequate notice and consent mechanisms in place to inform employees and applicants about the use of the AI tool, the categories of data collected, and the purposes of processing?
- Does the tool adhere to data collection and retention principles, i.e., does it collect and retain only the personal data that is necessary for the stated purpose, and does the company have a retention schedule that complies with applicable law?
- If the tool is provided by a third-party vendor, has the company entered into a data processing agreement or equivalent contractual arrangement that addresses the vendor’s obligations regarding data security, sub-processing, breach notification, and data return or deletion?
- Does the tool transfer personal data across borders? If so, has the company ensured that adequate safeguards are in place to comply with cross-border transfer restrictions under applicable laws?
- Does the tool have appropriate technical and organizational security measures in place to protect personal data against unauthorized access, loss, or breach, consistent with applicable cybersecurity requirements and industry standards?
- Does the tool process any categories of sensitive personal data (e.g., health information, biometric data, or data revealing racial or ethnic origin)? If so, are enhanced protections and retention required under the Health Insurance Portability and Accountability Act (HIPAA), the Illinois Biometric Information Privacy Act, and similar laws being satisfied?
- Can the company honor individual rights requests in connection with the tool—including rights of access, correction, deletion, portability, and the right to opt out of automated decision-making as required under applicable privacy laws? If a vendor processes the data, is there a data processing agreement addressing security, sub-processing, breach notification, and data return or deletion?
When implementing any AI tool, companies with a global workforce may be subject to the laws of multiple jurisdictions. The European Union’s AI Act, for example, establishes a comprehensive regulatory framework that classifies AI systems by risk level and treats many AI tools used in recruitment and workforce management as high-risk. Employers that purchase such tools from vendors may be “deployers,” with obligations that include human oversight, monitoring, and informing workers’ representatives and affected workers before use, scheduled for effect in 2027.
Beyond the EU, many countries have their own patchwork of laws that may be triggered by AI use in the employment context. For instance, in Ontario, Canada, some employers are required to indicate the use of AI in job postings, and in Quebec employers are required to have transparency around the use of AI and employees have the right to request human review; each Canadian province has its own discrimination and accommodation laws that can be implicated by the deployment of AI tools. For companies operating across borders, obligations can vary significantly by jurisdiction, making a jurisdiction-by-jurisdiction view of the applicable local, national, and supranational requirements governing AI in the workplace particularly important.
As companies embrace this expanding and dynamic technology, seeing the whole picture, including the regulatory landscape, is critical in determining what AI tool to purchase and how to use it.
Ogletree Deakins’ Artificial Intelligence and Innovation Practice Group, Cybersecurity and Privacy Practice Group, and Workforce Analytics and Compliance Practice Group will continue to monitor developments and will post updates on the Artificial Intelligence and Innovation, Cross-Border, Cybersecurity and Privacy, State Developments, and Workforce Analytics and Compliance blogs as additional information becomes available.
Follow and Subscribe
LinkedIn | Instagram | Webinars | Podcasts