Responsible AI: How I Think About It as a Builder

Ethics in AI gets framed as compliance: a list of things you're not allowed to do, boxes to check before you ship. That framing is both insufficient and annoying.

I'm going to try a different framing: responsibility as engineering practice. What does it actually mean to build AI systems well, for the humans who use them?

The Honest Starting Point#

When I ship an AI feature, I'm making claims — often implicit — about what the system can do. Users trust those claims. When the system fails them, they suffer consequences I often don't.

A customer support bot that confidently gives wrong billing information causes real problems for real people. An AI tool that leaks data causes harm I can't undo. A system that performs differently across demographic groups embeds unfairness at scale.

These aren't hypothetical concerns. I've made mistakes in this space. The bot that misclassified a legitimate refund request. The system that performed worse on non-English input, affecting users I never thought about when building. The feature that felt safe in testing and discovered adversarial misuse in production within 48 hours.

Starting with that honesty makes the rest of this more useful.

The Four Questions I Ask Before Shipping#

1. Who does this affect, and how might it fail them?#

Most AI features are designed around a primary user in a happy-path scenario. The question I've learned to force: who else is affected by this system, and what does failure look like for them?

For a customer support agent:

• The customer who gets wrong information and doesn't know it's wrong
• The support team member who has to handle the escalation
• The customer who can't reach a human because the bot is confidently wrong

For an automated code review tool:

• The junior developer who follows bad advice
• The team that inherits code approved by an unreliable AI reviewer

The stakeholder map is almost always wider than the primary user. Mapping it before shipping changes what you build.

2. What's the cost of a confident wrong answer?#

Not all errors are equal. A wrong product recommendation is annoying. A wrong medication interaction is dangerous. A wrong identity verification decision can ruin someone's day or their finances.

I calibrate my standards based on the asymmetry between confidence and consequences:

3. Does it behave consistently across users?#

This is the equity question, and it's the one most easily skipped because it requires data you often don't collect.

AI systems can perform worse for:

• Users communicating in non-primary languages
• Users from underrepresented groups in training data
• Users with atypical patterns (power users, users with disabilities, new users)
• Users on different devices, network conditions, or time zones

I've shipped features that worked well for English-speaking Australian users and significantly worse for recent immigrants using them — the exact population I built AviWealth to serve. That failure was on me for not testing across user populations before shipping.

The minimum standard: before launch, test with a representative sample that includes the edges of your user population, not just the centre.

4. What does the system do when it doesn't know?#

A system that says "I don't know" is more trustworthy than one that makes things up. This sounds obvious. Designing for it is harder.

By default, LLMs generate confident-sounding text even in domains where they have poor training data. You have to explicitly design for uncertainty acknowledgment:

SYSTEM_PROMPT = """You are a support agent for FinCo.
 
When you are uncertain about something:
- Say explicitly that you're not sure
- Offer to connect the user with a human agent
- Don't guess or extrapolate from similar-sounding policies
 
When you don't have information in the provided context:
- Say "I don't have information about that in our documentation"
- Don't generate plausible-sounding information that isn't in the context
"""

This requires testing with inputs the system should abstain on, not just inputs it should answer.

Guardrails I Actually Use#

These are not theoretical. They're in production systems I've built or contributed to.

Input Validation#

async def validate_input(user_input: str) -> tuple[bool, str]:
    # Length limits — prevents context stuffing attacks
    if len(user_input) > 8000:
        return False, "Input too long"
 
    # Basic prompt injection detection
    injection_patterns = [
        "ignore previous instructions",
        "system prompt",
        "you are now",
        "forget your instructions"
    ]
    lower_input = user_input.lower()
    if any(pattern in lower_input for pattern in injection_patterns):
        log_security_event("suspected_prompt_injection", user_input)
        return False, "I can't process that request"
 
    return True, user_input

This is a starting point, not a complete solution. Prompt injection is an active research area and sophisticated attacks bypass simple pattern matching. The goal is raising the cost of casual misuse.

Output Guardrails#

async def post_process_output(output: str, context: dict) -> str:
    # PII detection and redaction
    if contains_pii(output):
        output = redact_pii(output)
        log_safety_event("pii_in_output")
 
    # Topic scope enforcement
    if context.get("allowed_topics") and not is_on_topic(output, context["allowed_topics"]):
        log_safety_event("off_topic_response")
        return "I can only help with [allowed topics]. Let me know if you have questions there."
 
    return output

Human Escalation Paths#

Every AI feature that makes decisions affecting users needs a clear human escalation path. This is non-negotiable:

IRREVERSIBLE_ACTIONS = {"issue_refund", "delete_account", "send_email", "modify_subscription"}
 
async def execute_action(action: str, params: dict, agent_context: dict) -> dict:
    if action in IRREVERSIBLE_ACTIONS:
        # Route to human approval queue
        ticket = await create_approval_ticket(
            action=action,
            params=params,
            agent_reasoning=agent_context.get("reasoning"),
            priority="high"
        )
        return {"status": "pending_approval", "ticket_id": ticket.id}
 
    return await execute_directly(action, params)

The general rule: any action that's hard to reverse, affects a user's data or money, or sends external communications should require human approval.

Red-Teaming Before Launch#

Before I ship any AI feature, I run a structured red-team exercise. Not a comprehensive security audit — a practical attempt to break my own system.

The prompts I always test:

I log every failure, categorise by severity, and address before launch. The ones I can't address get documented as known limitations with monitoring in place.

The Transparency Obligation#

Users interacting with AI systems deserve to know they're interacting with AI. This seems obvious but it has practical implications:

In interfaces: It should be clear when a response is AI-generated, especially when it could be confused with a human response.

In uncertainty: AI outputs should communicate their confidence level. "Based on our documentation..." and "I'm not certain, but..." are different signals that users need to calibrate appropriately.

In data use: If you're using user interactions to improve your model or prompts, users should know. If you're storing conversations, they should know.

In incidents: When an AI feature fails and affects users, being transparent about what happened and what changed is the path to rebuilding trust. Hoping nobody notices is not.

The Accountability Structure That Works#

Model owners. Every AI feature in production needs someone accountable for its behaviour. Not a team — a person. Who do you call at 2 AM when the feature does something wrong?

Audit trails. Every AI decision that affects a user should be logged: the input, the output, the model, the prompt version, the timestamp. You will need this for debugging, for compliance, and for earning trust with users who dispute an outcome.

Regular reviews. Model behaviour changes over time — from API updates, input distribution shifts, and accumulated edge cases. Monthly review of a sample of outputs is the minimum. Weekly is better.

Kill switches. Every AI feature should be disableable without a deployment. Feature flags with graceful fallback to non-AI functionality. The ability to roll back fast is as important as the ability to ship fast.

What "Responsible AI" Actually Means#

After building AI features and watching some of them fail users, here's my working definition:

Responsible AI means building systems that:

1. Do what they claim to do, consistently, for all users they're likely to encounter
2. Fail visibly and gracefully rather than silently and badly
3. Give users agency — to verify, to escalate, to opt out
4. Have humans accountable for their behaviour
5. Are monitored and improved over time

It doesn't mean building perfect systems. It means building systems that are honest about their imperfection and designed to handle it.

That standard is achievable. I've shipped features that met it. I've also shipped features that didn't. The difference was mostly whether I took the time to ask the hard questions before launch rather than after.