All articles

LLM Engineering: RAG, Evals & Multimodal Apps

A course tutor should be able to show its evidence

Learn why retrieval, permissions, citations, and unanswered questions matter when building an AI learning assistant.

2 min read

A learner asks an AI tutor what a project requires. The tutor gives a clear answer. The next question should be easy: where in the course does that requirement appear?

An evidence-based tutor connects the answer to an approved lesson or resource. Retrieval finds relevant material; the response explains it; a citation lets the learner inspect it. For a recorded video, that may eventually mean a verified timestamp. Before the video exists, a lesson or section reference is more honest than an invented time.

Retrieval must also respect access. A student should not receive restricted instructor notes because those notes happen to match the query well. In a web application, enrollment and source permissions belong in server-side code. A student's prompt should not be able to grant access to another course.

Even a valid citation is not enough. A source may exist without supporting the claim. If the note lists project components but contains no deadline, attaching its ID does not justify an invented Friday deadline. Test both citation scope and actual claim support.

Start with a small baseline. The academy's example uses word overlap to retrieve approved text and return an excerpt. This makes the behavior inspectable, including its weakness with paraphrases. It is not an embedding model or a conversational LLM. Adding real generation later creates another component to evaluate.

Useful tests include a straightforward question, a paraphrase, a missing fact, an unauthorized-source attempt, and malicious instructions embedded inside retrieved material. Measure what the system does in each situation rather than giving the entire application one reassuring score.

In LLM Engineering, learners build a text tutor prototype and document its limits. A good tutor does more than produce an answer. It helps a learner locate evidence, recognize uncertainty, and decide what to study next.