AI Security, Ethics and Governance

Introduction

Powerful AI brings new risks, for the people who use it and for those it affects. Building responsibly means thinking about security, fairness, privacy and accountability from the start, not after an incident.

Security risks of AI applications

The OWASP project lists the top risks for LLM applications. Key themes:

• Prompt injection: malicious instructions hidden in user input or in content the system reads, such as a web page or document, try to hijack the model. Defence: treat all external text as untrusted, separate instructions from data, limit what tools the model can use, and require approval for sensitive actions. No defence is perfect, so design so that a hijacked model can do little harm.
• Sensitive data leakage: the model reveals private data from its context, from other users, or from documents it should not expose. Defence: give it only the data needed, enforce user permissions on retrieval, and do not put secrets in prompts.
• Insecure output handling: treating model output as safe. If it is inserted into a web page or run as code or a database command, it can cause cross-site scripting or injection. Defence: validate and escape output like any untrusted input.
• Excessive agency: an agent with too much power. Defence: least privilege, limits and human approval.
• Supply chain and data poisoning: untrusted models, plug-ins or training data. Defence: use trusted sources and verify them.
• Denial of wallet: abuse that runs up your bill. Defence: rate limits and budgets.

Fairness and bias

Models learn from human data and can reproduce its unfairness. Responsible teams:

• Test across groups and languages, not only overall averages
• Avoid fully automated decisions about people's jobs, credit, health or safety, and keep human review and a way to appeal
• Document known limitations honestly

Privacy

• Collect and send only the data you need.
• Know where data goes, how long it is kept and whether it is used for training.
• Follow privacy laws that apply to your users, and get legal advice for sensitive uses.
• Let people know when they are interacting with AI and how their data is used.

Transparency and accountability

• Tell users when content is AI-generated, and where it matters, label it.
• Be able to explain what the system does, and what it cannot do.
• Keep logs and records so decisions can be reviewed.
• Make clear who is responsible when something goes wrong. A person or team must own the outcome, not "the AI".

Governance frameworks and regulation

• The NIST AI Risk Management Framework organises the work into four functions: Govern, Map, Measure and Manage. It is voluntary guidance used widely.
• The EU AI Act is a law that sorts AI uses by risk level, with stricter duties for high-risk uses and bans on some practices. Other countries are creating their own rules, and rules keep changing, so check the current requirements where you operate.
• Many organisations set up AI policies, review boards, approved tools lists and training.

A practical responsible-AI checklist

1. What is the system for, and who could it harm?
2. What data does it use, and do we have the right to use it?
3. Have we tested it for accuracy, bias, security and misuse?
4. Is there human oversight for important decisions?
5. Can users understand, question and report problems?
6. Who owns it, and what is the plan for monitoring and incidents?

Try it yourself

Pick an AI feature, such as an assistant that screens job applications. Write the biggest risk in each of security, fairness and privacy. Decide what you would change, and whether this use should be automated at all.

Summary

• Key AI security risks are prompt injection, data leakage, unsafe output handling and excessive agency.
• Fairness, privacy and transparency need testing, limits on data, human oversight and clear ownership.
• Frameworks such as NIST AI RMF and laws such as the EU AI Act guide responsible AI, and they keep changing.