Kishor.Academy
Log in
← All courses

Artificial Intelligence · intermediate

LLM Engineering: RAG, Evals & Multimodal Apps

Build a transparent retrieval baseline, validate citations and access, design multimodal workflows, and evaluate an evidence-based tutor prototype.

With Admin

intermediate9 lessons · 3 sections10–15 hours of reading and lab work, plus 4–6 hours for the project; estimates exclude video and opt

THE TAKEAWAYS

What you’ll learn

  • Define input, output, source, and failure contracts for a lesson tutor.
  • Distinguish lexical retrieval from embeddings and design a fair comparison.
  • Validate structured outputs and tool arguments independently of model instructions.
  • Chunk notes with metadata and enforce access before ranking.
  • Check citation scope and claim support; abstain when evidence is absent.
  • Trace OCR or transcription errors through a multimodal application.
  • Build an evaluation set covering ordinary, missing-evidence, adversarial, and access cases.
  • Measure operational limitations and distinguish an offline baseline from actual model generation.
  • Deliver a ten-case evaluated tutor prototype and a documented improvement experiment.

Is this for you?

Python developers ready to move from prompts to source-grounded applications with explicit evaluation and access boundaries.

Before you start

  • Python & Data and LLM Foundations, or equivalent Python, JSON, and prompt-evaluation skills.
  • Python 3.10 or newer for the required retrieval lab; no required paid API or GPU.
  • Optional generation needs an already installed Ollama server and a suitable licensed local model; hardware requirements vary.

PUT IT TO WORK

What you’ll build

Evidence-based tutor prototype. Client scenario: a learner asks questions about approved course notes. Deliverable: a command-line retriever, ten-case evaluation file/report, source-supported responses, and a limitations document.

Design an application contract, chunk source notes with metadata, and retrieve only authorized evidence. Compare lexical and semantic retrieval conceptually while implementing an inspectable lexical baseline. Validate JSON and citation membership without confusing those checks with factual support. Plan OCR and voice workflows, evaluate no-answer and adversarial cases, and document latency and operating costs. Optional local inference is clearly separated from the offline baseline.

Course curriculum

SECTION 1Application contracts and retrieval foundations3 lessons
  1. Tokens, context, and prompt contractsReading
  2. Embeddings and semantic retrievalReading
  3. Structured outputs and function callingReading
SECTION 2Evidence pipelines and multimodal inputs3 lessons
  1. Chunking, metadata, and retrieval qualityReading
  2. Grounded answers, citations, and refusalReading
  3. Multimodal documents and voice interfacesReading
SECTION 3Evaluation, operations, and a tutor prototype3 lessons
  1. Create an evaluation dataset before launchReading
  2. Measure quality, safety, latency, and costReading
  3. Capstone: a cited knowledge assistantReading