After enacting a measure in 2021 restricting the use of artificial intelligence in employment-related decisions, New York City has released proposed regulations in advance of the new law’s effective date of January 1, 2023.
The law regulates the use of “automated employment decision tools” (AEDT) on candidates and employees residing in NYC.
The law defines such tools as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.”
An employer is prohibited from using an AI tool in making employment decisions unless the tool has been subject to a “bias audit” by an “independent auditor” within the prior year and a summary of the audit results and distribution data for the tool have been made publicly available on the employer’s or employment agency’s website.
Individuals have the ability to request an accommodation from being subject to an AI tool and can request information regarding the data that was collected about them.
To provide clarity for employers, the Department of Consumer and Workplace Protection (DCWP) released proposed rules implementing the new law, scheduled a public hearing for discussion and will accept public comment on the proposal until October 24.
The proposed rules provide new and expanded definitions for key terms, including those found in the law’s definition of “automated employment decision tool,” such as “machine learning, statistical modeling, data analytics, or artificial intelligence,” “simplified output,” and “to substantially assist or replace discretionary decision making.”
For example, the DCWP said “to substantially assist or replace discretionary decision making” means to rely solely on a simplified output (e.g., score, tag, classification, ranking) with no other factors considered, to use a simplified output as one of a set of criteria where the output is weighted more than any other criterion in the set, or to use a simplified output to overrule or modify conclusions derived from other factors including human decision making. According to the proposed rule, a “simplified output” may “take the form of a score (e.g., rating a candidate’s estimated technical skills), tag or categorization (e.g., categorizing a candidate’s resume based on keywords, assigning a skill or trait to a candidate), recommendation (e.g., whether a candidate should be given an interview), or ranking (e.g., arranging a list of candidates based on how well their cover letters match the job description).”
Addressing the requirements for bias audits, the proposed rules define an independent auditor as “a person or group that is not involved in using or developing an AEDT that is responsible for conducting a bias audit of such AEDT.”
Depending on the type of AEDT being used, there are two types of audits authorized by the proposed rules. When an AEDT selects individuals to move forward in the hiring process or classifies individuals into groups, the bias audit must calculate the selection rate and impact ratio for each EEO-1 demographic category; and when an AEDT classifies individuals into groups, the calculations must be performed for each such classification.
Alternatively, when an AEDT scores applicants or candidates, the bias audit must calculate the average score for each individual in each category and the impact ratio for each category.
The proposed rules also detail what information must be made publicly available before an employer may use an AEDT (such as a summary of the most recent bias audit for the AEDT being used).
As for the notice requirements under the law, at least ten business days before an AEDT is used, employers must notify all candidates and employees residing within New York City that an AEDT will be used to assess their candidacy and provide the job qualifications and characteristics the tool will be assessing.
The rules provide multiple options for notification, including a notice on a website in a clear and conspicuous manner, listing the notice in a job posting, or providing it by U.S. mail or email. The notice must also provide instructions for how to request an alternative selection process or accommodation, according to the proposal.
To read the proposed rules, click here.
Why it matters: Employers in New York City should keep a close eye on the proposed rules as the effective date of the new law nears, and employers elsewhere in the country should brace themselves for similar regulations in their own states or even federal oversight on the issue of AI. Not long after the city released its proposed rules, the White House published a blueprint for an AI Bill of Rights, tracking recent federal guidance from the Department of Justice and Equal Employment Opportunity Commission on how employers can avoid running afoul of the Americans with Disabilities Act when using AI for employment-related decisions.