Responsible GenAI in HR: Data Minimization and Bias Testing

HR systems handle sensitive data and influence people's livelihoods. Beyond general security, they need controls for privacy, fairness, and accountability. This is engineering guidance, not legal advice; requirements differ by jurisdiction, so involve counsel.

Keep the AI assistive

Use GenAI to summarize, draft, and answer policy questions. Keep selection, ranking, and rejection decisions with people, with documented criteria.

Minimize data

Strip attributes the task does not need before sending text to a model, such as names, photos, age indicators, and addresses, particularly for any screening support.

DROP_FIELDS = {"name", "photo_url", "date_of_birth", "address", "gender", "nationality"}

def minimize(candidate: dict) -> dict:
    return {k: v for k, v in candidate.items() if k not in DROP_FIELDS}

Free-text fields can still reveal protected traits. Treat redaction as risk reduction, not a guarantee.

Test outcomes across groups

If a tool influences any shortlist, test whether outcomes differ across groups, using the data and methods your legal team approves. A common first check compares selection rates between groups.

def selection_rates(rows, group_key):
    totals, selected = {}, {}
    for r in rows:
        g = r[group_key]
        totals[g] = totals.get(g, 0) + 1
        selected[g] = selected.get(g, 0) + (1 if r["shortlisted"] else 0)
    rates = {g: selected[g] / totals[g] for g in totals}
    best = max(rates.values())
    return {g: {"rate": rate, "ratio_to_highest": rate / best if best else None} for g, rate in rates.items()}

Ratios far below 1.0 for a group are a signal to investigate, not a verdict. Which thresholds apply is a legal question.

Document and review

• Record intended use, data used, evaluation results, and known limits.
• Re-test after any model, prompt, or data change.
• Give candidates and employees a route to contest and ask for human review.
• Retain and delete data according to policy.

Scope and permitted use

Write down what the system is for and what it must not do, and make the boundaries visible to users. For example: it may draft job descriptions, summarize an application for a recruiter, and answer policy questions; it may not rank candidates, recommend rejection, or infer personal characteristics.

Enforcing boundaries in code

BLOCKED_REQUEST_PATTERNS = [
    r"\b(rank|score|rate)\b.*\b(candidates?|applicants?)\b",
    r"\bshould we (hire|reject)\b",
    r"\b(age|gender|religion|ethnicity|pregnan\w*)\b.*\b(candidate|applicant)\b",
]

def allowed(request_text):
    t = request_text.lower()
    return not any(re.search(p, t) for p in BLOCKED_REQUEST_PATTERNS)

Patterns are a coarse first layer. Combine them with instructions in the system prompt and with review of logged requests.

Testing language for bias

For generated job descriptions and feedback summaries, run paired tests: create inputs that are identical except for a name or pronoun, and compare outputs for differences in tone, length, or content. Examine differences with a reviewer, and adjust prompts and filters.

def paired_test(generate, template, variants):
    outputs = {v: generate(template.format(name=v)) for v in variants}
    return {v: {"words": len(o.split()), "sample": o[:160]} for v, o in outputs.items()}

Employee-facing assistants

• Answer policy questions from the current handbook with citations.
• Route anything about individual cases, grievances, or health to a human.
• Keep conversations confidential, restrict access to logs, and set short retention.
• Make clear to employees what is stored and who can see it.

Governance and records

Keep a register of HR AI uses with their purpose, data, risks, owners, and test results. Review it with HR, legal, and employee representatives where appropriate. Because employment law and AI regulation differ by location and are changing, ask counsel to confirm the requirements for each deployment.

How we can help

We build HR assistants that stay assistive and implement the testing and documentation your counsel needs. Contact us to scope a low-risk first use case.