Kishor.Academy
Log in
← All courses

Online course · advanced

AI Safety & Alignment for Builders

Threat-model LLM applications, defend against direct and indirect prompt injection with controls enforced in code, handle personal data carefully, and build a red-team suite. Offline labs; no paid API.

With Admin

advanced9 lessons · 3 sections8–12 hours of reading and lab work, plus 4–6 hours for the project; estimates exclude video.

THE TAKEAWAYS

What you’ll learn

  • Build a risk map covering reliability, privacy, attack, and misuse with owners and tests.
  • Explain how models are aligned and why jailbreaks and adversarial inputs still succeed.
  • Draw a threat model that marks trust boundaries and the most dangerous data path.
  • Distinguish direct and indirect prompt injection and explain why text filters cannot be the boundary.
  • Enforce tool allowlists, argument validation, egress checks, and approval in code.
  • Minimise, redact, and set retention for personal data across logs and datasets.
  • Design a red-team suite that separates false allows from false blocks.
  • Plan human oversight, incident response, and a responsible-use policy.
  • Deliver a safety case with enforced controls, test results, and open risks.

Is this for you?

Developers who build LLM features with retrieval or tools and need practical safety, security, and privacy controls.

Before you start

  • Complete LLM Engineering, or demonstrate retrieval, evaluation, and access-control fundamentals. Agentic AI is recommended.
  • Python 3.10 or newer; familiarity with dictionaries, regular expressions, and URLs.
  • No paid API, real credentials, or external accounts required. Attack only systems you own or are authorised to test.

PUT IT TO WORK

What you’ll build

Safety case for an LLM application. Client scenario: a course assistant reads uploaded documents, searches notes, fetches documentation, and can email learners. Deliverable: a threat model, enforced controls, red-team suite, privacy rules, and incident runbook.

Map the risks of an LLM application, learn what model-level alignment does and where it fails, and draw a threat model with trust boundaries. Test a tool-call policy gate against hand-written injected documents and see a keyword filter miss a paraphrased attack. Study exfiltration, least privilege, privacy-aware logging, red-teaming, and incident response. Finish with a safety case that reports what was tested and what remains open.

Course curriculum

SECTION 1Risks, alignment, and threat models3 lessons
  1. What can go wrong: a builder's risk mapReading
  2. How models are aligned, and the limitsReading
  3. Threat modelling an LLM applicationReading
SECTION 2Prompt injection, least privilege, and privacy3 lessons
  1. Prompt injection, direct and indirectReading
  2. Defence in depth for tools and dataReading
  3. Privacy and data handlingReading
SECTION 3Red-teaming, oversight, and a safety case3 lessons
  1. Red-teaming and safety evaluationReading
  2. Human oversight, incidents, and policyReading
  3. Capstone: a threat model and safety test suiteReading