Who Can Sign Your Local Law 144 Bias Audit — and Who Disqualifies It
Independence under Local Law 144 is a structural test, not a sentence in the cover letter. Three relationships disqualify an auditor, and the most common arrangement in HR tech — the vendor's own analytics team producing the numbers — fails every one of them.
The independence test is about relationships, not intentions
Local Law 144 requires an independent auditor to conduct the bias audit of an Automated Employment Decision Tool before that tool screens candidates for a New York City role. The Department of Consumer and Worker Protection defined independence negatively: rather than listing qualifications, the rules list the relationships that disqualify a person or firm.
That framing matters, because it means competence is not the question DCWP asks first. A brilliant, scrupulous internal team produces a document that is not a bias audit within the meaning of the law. An outside firm with a modest methodology, no financial stake, and no hand in building the tool produces one that is.
No involvement in using, developing, or distributing the tool
STRUCTURALThe auditor cannot have been part of building the AEDT, selling it, deploying it, or operating it. This is the disqualifier that catches vendors auditing their own models and catches the consultancy that implemented your ATS integration last year.
It also reaches the reseller. If a staffing partner configured the scoring for you and now offers to audit it, they distributed the tool within the meaning of the rule.
No employment relationship during the audit period
PAST AND PRESENTThe auditor cannot be an employee or officer of the employer, the employment agency, or the AEDT vendor — and the disqualification reaches back over the period being audited, not just the date of signature.
Hiring the vendor's former head of data science as an independent contractor to audit the model they trained does not cure the conflict; it packages it.
No direct financial interest in the tool or its parties
FOLLOW THE MONEYAn auditor holding equity in the vendor, taking referral fees for placing the tool, or carrying a contingent fee tied to the audit outcome has a direct financial interest and is disqualified.
Watch the incentive structure in your own engagement letter. A fee that changes depending on whether the impact ratios clear 0.80 is a disqualifying interest by construction.
The vendor audit you were handed probably does not cover you
Most employers in scope never commission an audit. They receive a PDF from the vendor, link it from a careers page, and consider the requirement met. Sometimes that works. Frequently it does not, for a reason that has nothing to do with independence: scope.
A vendor audit describes the tool as the vendor ships it, scored across the vendor's pooled customer data. The tool you deployed may differ in ways that change the selection rates materially:
- Threshold configuration — you set the advance cutoff at a different score than the vendor's default, which changes the selection rate for every subgroup.
- Module selection — you use the assessment but not the resume ranker, so the audited composite is not the composite that screened your applicants.
- Knockout questions — your own configuration adds eliminations that run before the model ever scores anyone.
- Applicant pool — the vendor's pooled data may be dominated by industries and geographies that look nothing like your funnel.
Before relying on a vendor audit, read the scope and data-source sections and ask one question in writing: does this audit cover our configuration of the tool as deployed? Keep the answer. It is the document you will want if DCWP asks why you believed the summary applied to you.
What to put in the engagement letter
An affirmative independence representation
The auditor states, in writing, that it has no employment relationship, no involvement in developing or distributing the AEDT, and no direct financial interest in any party — covering the full period under audit, not just the signing date.
Named scope of the tool and configuration
Identify the model version, the modules in use, the score thresholds, and the requisitions covered. An audit of 'the platform' is not an audit of the thing that screened your candidates.
Data lineage and the right to publish it
The public summary must state the data source and the number of applicants scored. Agree up front that those facts are publishable, so a confidentiality clause does not collide with the disclosure duty later.
Flat or milestone fees, never outcome-contingent
Any fee that varies with the impact ratios creates a direct financial interest in the result and undermines the independence representation you just paid for.
A no-remediation covenant
The auditing firm agrees not to take remediation work on the same tool for the audit period, so the next audit cannot be an audit of its own recommendations.
Delivery of the summary in publishable form
You need a document you can post, with the audit date, the distribution date of the tool, and the ratios — not a slide deck marked 'draft, do not distribute.'
The rolling one-year clock is the failure most teams actually hit
The audit must be no more than one year old at the time of use. Teams treat that as an annual to-do and let it slide a quarter. But every screening decision made after the anniversary is use of an unaudited AEDT, and the penalty structure is per day of continued use, which means a lapse is not one violation — it accrues while nobody is looking.
Two practices keep this from becoming a discovery exhibit. First, book the next audit engagement at least ninety days before the anniversary, because data extraction is the slow part. Second, treat a material model change — a retrain, a new feature, a moved threshold — as a new tool, and re-audit before it screens anyone in NYC, whatever the date on the current summary says.
Check what your careers page is actually publishing
The bias audit summary and the AEDT notice both live on public web pages — pages that have to be findable and readable to do their job. RatedWithAI scans your public pages free and shows you what a visitor, a candidate, or an investigator would encounter.
Scan Your Careers Page for Free →Frequently Asked Questions
Can our own data science team run the Local Law 144 bias audit?
No. DCWP's rules disqualify anyone who is or was an employee or officer of the employer, employment agency, or vendor during the audit period. An internal team fails the first disqualifier outright, no matter how methodologically rigorous the work is. The same applies to a contractor who sits inside your engineering org and reports to your CTO.
Does the auditor have to be a lawyer, a statistician, or hold a certification?
None of those are required. The law does not license bias auditors and there is no accreditation body. Independence and methodology are what the rules constrain. In practice you want someone who can compute selection rates and impact ratios correctly and will put their name on a public summary — that is usually an employment-testing statistician, an I/O psychology consultancy, or an audit firm's algorithmic assurance practice.
Can the auditor also help us fix the model afterward?
Remediation work creates the exact conflict the independence rule targets. If the same firm tunes thresholds and then audits the tuned model, it is auditing its own work. Separate the engagements: one firm audits, a different firm (or your internal team) remediates, and the remediated model gets a fresh audit before it screens NYC candidates again.
Our vendor gave us their bias audit. Are we covered?
Usually not fully. A vendor audit typically covers the vendor's own aggregated data, not your configuration, your thresholds, or your applicant pool. If you tuned scoring, changed the knockout questions, or use only one module of the product, the vendor's audit may not describe the tool you actually deployed. Read the scope section before relying on it, and confirm the audit is less than one year old.
What does a Local Law 144 bias audit cost?
Quotes for a single AEDT commonly land in the low five figures, driven by how many selection decisions the tool makes, whether demographic data has to be reconstructed, and how many intersectional cells need to be computed. Cost climbs when historical data is scattered across an ATS and a vendor system and someone has to join them before any statistics can run.
How long is an audit valid?
The audit must be no more than one year old at the moment the AEDT is used. That is a rolling deadline, not an annual calendar event — if your audit is dated March 3, every screening decision after March 3 of the following year is unaudited use. Material model changes reset the clock in substance even if the paper date is still current.