The Model Closed the Account. Nobody Can Say Why.
A risk score crosses a threshold that was set to match staffing capacity, a letter goes out citing "internal policy", and a small business loses its payment rails on a Tuesday. Every part of that sequence is a decision someone made — and none of it is written down.
ECOA prohibits discrimination in any aspect of a credit transaction, and Regulation B requires a creditor taking adverse action to give specific principal reasons for it. Neither contains a carve-out for decisions produced by a model, and supervisory guidance has been consistent that an institution unable to explain a model's individual outcome should not be using that model for the decision. Separately, a facially neutral practice that falls disproportionately on a protected class is exposed to disparate-impact analysis unless it is justified by business necessity with no less discriminatory alternative available.
Six Decisions, One Question
"Account closure" describes a dozen different legal postures depending on what was closed and what drove it. The useful exercise is to inventory each automated decision the institution makes about continuing a relationship, and to name the regime that reaches it — because the operations teams running these queues generally cannot.
Where the Disparity Actually Gets Created
Institutions look for bias in the model. It is more often produced by the operating decisions wrapped around the model — the threshold chosen to fit a review team's headcount, the triage rule that exits low-balance accounts without analysis, the appeal queue nobody staffed. None of those are modelling choices, and none of them are usually recorded as policy.
A vendor score is bought on accuracy metrics. Nobody asks for reason codes, subgroup performance or the right to test on the institution's own population.
A cutoff is chosen to hit an alert-volume target the operations team can staff. This business constraint, not risk, decides who gets closed.
Volume exceeds review capacity. Queues get triaged by score, and low-value accounts are exited without individual analysis.
Notice is generic, funds may be held pending investigation, and the customer has no channel that reaches a human with authority.
Appeals are routed to the same model or to staff with no ability to override. A reconsideration process that cannot reverse anything is evidence, not a defence.
Nobody computes closure rates by geography or segment, so a pattern only becomes visible when an examiner or plaintiff computes it first.
Proxies Do the Work the Prohibited Inputs Were Removed From
A fraud or AML model is trained on the institution's own history of alerts and investigations. If that history reflects where scrutiny was historically applied, the model learns to apply scrutiny in the same places — and it does so through features nobody flagged as sensitive. Remittance corridor, cash intensity, merchant category, transliterated-name matching against watchlists, prepaid device signals and account tenure all carry demographic signal without a single protected attribute in the schema.
Fuzzy name matching deserves specific attention because its error rate is not evenly distributed. Naming conventions with fewer distinct surnames, multiple transliterations, or patronymic structures generate systematically more false positives against sanctions and PEP lists. Those false positives become alerts, alerts become closures, and the resulting pattern is national-origin disparity produced entirely by a string-matching library that nobody in compliance has ever tested for subgroup accuracy.
"We Can't Tell You" Is a Statement About the Vendor Contract
When an institution cannot give a specific reason, the cause is almost always a procurement decision: a third-party score was bought without reason-code output, without subgroup performance data, and without the right to validate on the institution's own population. Those terms are negotiable at signature and unobtainable afterwards. Third-party risk management guidance already places responsibility for vendor model behaviour on the institution deploying it, so the practical effect of accepting an opaque score is to assume liability for outputs you cannot inspect.
Six Controls Worth Having
A model that cannot emit the specific principal reasons for an individual outcome cannot lawfully drive credit decisions. Make reason-code output and its accuracy a contractual deliverable before purchase, not an integration wishlist item afterwards.
Store the model version, feature values, score, threshold in force and reviewer identity with each closure. Thresholds move weekly; without a pinned record the institution cannot reconstruct its own decision six months later.
Run closure, hold and rejection rates by geography, merchant category and corridor on a schedule, with results going to a committee that has authority to change thresholds. Excluding protected characteristics from inputs tells you nothing about outputs.
Where a disparity appears, business necessity is only half the test. Document the alternatives evaluated — different thresholds, additional verification steps, narrower features — and why each was or was not adopted.
Define what a reviewer must examine and record before an exit, and audit whether it happens. Review that consists of confirming a score is not review, and it will be characterised that way by whoever reads the queue timestamps.
A reconsideration path must reach someone with authority to overturn, on a stated timeline, with the outcome logged. Reversal statistics are also the cheapest ongoing measure of whether the model is actually right.
Related Reading
- AI lending discrimination under ECOA — the same adverse-action machinery at the origination end of the relationship.
- AI credit scoring and FCRA — when a risk score becomes a consumer report and triggers a separate notice regime.
- CCPA and AI fraud detection — the privacy-side obligations attached to the same signals.
Check What You Promise Before You Check the Model
Public pages routinely promise "fair, automated decisions", "instant approvals" and "human review of every case". Those sentences are the first thing an examiner or plaintiff compares against the actual queue.
See every claim your site makes in one pass. Run a free scan and reconcile each one against your real review and appeal process.
This article is general information and not legal advice. Obligations differ substantially depending on whether a product is credit or deposit, whether a consumer report is used, charter and licensing status, and applicable state law. Consult qualified counsel before designing or changing an automated account-action programme.