How Can Hiring Teams Choose AI Recruiting Software Without Rejecting Good Candidates?

Can AI recruiting software reject qualified candidates by mistake?

Yes. It can reject qualified candidates by mistake when a rule, score, or ranking treats a weak proxy as proof that someone cannot do the job.

The clearest warning is not theoretical. In its 2023 annual performance report, the U.S. Equal Employment Opportunity Commission described allegations that iTutorGroup programmed application software to automatically reject female applicants older than 55 and male applicants older than 60. The EEOC said more than 200 qualified U.S. tutor applicants age 55 or older were rejected over several weeks in 2020.

That is the buyer's guide in one sentence: automation can move fast, including in the wrong direction. Treat a rejection rule as a high-stakes employment decision, not a convenience feature.

Rigid screening creates similar exposure even without an explicit age rule. Exact-keyword matching can miss an applicant who describes equivalent work differently. A career break, a portfolio-based career, or freelance work can be informative evidence, not a defect. For example, a hiring team assessing creative experience should be able to recognize portfolio work and project pricing, like the work described in How to Price a Graphic-Design Project Without Scope Surprises, rather than requiring one narrow job-title sequence.

Speech tools need the same scrutiny. A Journal of Applied Psychology study comparing human transcription with four commercial automated speech-recognition services found significant bias across nearly all demographic comparisons, with the magnitude differing by service. Its tested samples did not show meaningful measurement bias in the downstream machine-learning interview scores, but that does not make transcription errors harmless in every procurement or deployment. Test the entire chain: recording, transcription, score, recruiter view, and decision.

How can I test whether an AI screener is filtering out good applicants?

Test it before launch with matched, qualified candidate cases, then inspect every difference in the tool's output. Do not ask only whether the system produces a score. Ask whether the score is job-related, explainable, and stable when irrelevant wording or presentation changes.

Use a small pre-launch drill. Build test applications that meet the role's actual requirements while varying resume phrasing, job titles, career paths, language variety, and any speech-based input the product will process. Keep the qualification level constant where possible. The point is not to manufacture a passing grade; it is to reveal whether the software confuses a proxy with capability.

The EEOC's guidance on employment tests and selection procedures says employers should validate procedures for the positions and purposes for which they are used. It also states that vendor documentation can help, but the employer remains responsible for validity. That means “our vendor says it works” is a weak answer. Replace it with: “Show how it works for this role, this purpose, and these applicants.”

Ask for audit logs that preserve the input, tool version, score or recommendation, reasons shown to the recruiter, human override, and final decision. Without that record, a team cannot investigate an apparent error or improve the workflow. Also ask who can override a score and whether that person can see the evidence, not just a red or green label.

New York City's approach provides a concrete procurement benchmark. The New York City Department of Consumer and Worker Protection states that Local Law 144 prohibits use of an automated employment decision tool unless it has had a bias audit within one year, audit information is publicly available, and required notices are provided. For New York City residents, notice includes the qualifications or characteristics assessed and accommodation-request instructions at least 10 business days before use. Even where that rule does not govern a hiring process, those are practical questions to put in a contract.

What hiring decisions should never be fully automated?

No qualified applicant should be finally rejected, denied an accommodation path, or selected for hire solely because an AI system said so. Software can organize information and reduce recruiter administration; people must remain accountable for decisions that determine access to work.

Use the tool for bounded tasks: routing applications, identifying missing information for a recruiter to check, preparing consistent candidate communications, and organizing evidence against a defined job requirement. Keep a trained person in charge of interpreting unusual experience, reviewing negative recommendations, responding to accommodation requests, and making final hiring choices.

Disability is where a hands-off workflow fails fast. The EEOC's disability-discrimination guidance identifies risks including failure to provide reasonable accommodations and screening out people who could do a job with or without an accommodation. Its worker guidance gives the example of a qualified employee who stutters and receives a poor result on a speech-pattern test. Build the affirmative replacement: “This result needs human review. Would an accommodation or an alternative process allow us to assess the relevant qualification fairly?”

A vendor should be able to answer six direct questions: “What does the tool assess?” “What does it not assess?” “What evidence supports each recommendation?” “What candidate data does it retain and for how long?” “How does a candidate request an accommodation or alternative process?” and “How can our recruiter reverse or appeal a result?” If the answers are vague, the product is not ready to make high-consequence recommendations.

Run this week's drill: take one planned automated screen-out rule, name the actual job requirement behind it, and identify one qualified applicant the rule could wrongly exclude. Then write the human-review trigger before you turn the feature on.