Audience
- Developers building document-aware assistants or internal knowledge tools.
- Data and knowledge-management practitioners with basic coding experience.
- Technical consultants evaluating AI access to organisational information.
AI & technology
AI Agents & RAG
Connect AI assistants to approved knowledge and build more grounded responses.
Learning with a practical purpose.
Connect an AI assistant to a defined collection of knowledge and evaluate whether it can answer questions with useful supporting evidence. This module follows the full retrieval-augmented generation workflow: understanding the question, finding relevant material, supplying context and assessing the resulting answer. It treats retrieval and answer generation as separate stages that can fail in different ways.
Your practical project is a knowledge assistant for a small approved document collection. You will compare search methods, inspect retrieved passages and create questions that reveal missing, outdated or contradictory information. The emphasis is a maintainable knowledge service with visible sources and appropriate refusal or escalation when the available evidence is insufficient.
Live onlineTeaching through Kent Business College, UK
2 hoursOne live session each teaching week
14 sessionsA semester for each module
28 live hoursPer module, plus independent study
The outline below is a 14-session online module, with one two-hour live session each teaching week. Reading weeks, breaks and additional activities follow the agreed cohort calendar. The cohort syllabus, full route, dates, fees, independent study and assessment requirements are confirmed before enrolment.
Who this module is designed for.
Starting knowledge and preparation
- Basic Python or JavaScript and familiarity with API calls.
- Confidence handling text files and structured records.
- Introductory understanding of language models and embeddings is helpful.
What you will work towards.
- Define a knowledge-assistant scope and prepare a traceable source collection.
- Build and compare keyword, semantic and combined retrieval approaches.
- Assemble answer context with relevant source references and controlled boundaries.
- Evaluate retrieval quality separately from answer correctness and groundedness.
- Design document updates, access restrictions and unsupported-question handling for a prototype.
Explore the 14-session learning plan.
Explore the focus and practical output of each teaching session. Open individual sessions below, or expand the complete plan.
Session 01 Knowledge questions and system scope
Define the users, source collection and kinds of questions that the assistant should and should not answer.
Practical output: A knowledge-assistant scope and question set.
Session 02 Retrieval rather than model training
Trace the retrieval-and-generation workflow and compare it with directly prompting a model without external evidence.
Practical output: A pipeline diagram and baseline comparison.
Session 03 Preparing trustworthy source material
Inventory documents, capture provenance and identify outdated, duplicate or contradictory information before indexing.
Practical output: A source register and preparation plan.
Session 04 Parsing, chunking and metadata
Split documents into useful units while preserving headings, references and metadata needed to interpret each passage.
Practical output: A documented ingestion and chunking routine.
Session 05 Keyword retrieval as a baseline
Build a transparent search method and inspect which questions are answered poorly despite containing familiar terms.
Practical output: A keyword-search baseline and error log.
Session 06 Embeddings and semantic retrieval
Create a small semantic index and compare retrieved passages with the baseline on the same questions.
Practical output: A semantic-retrieval experiment.
Session 07 Filtering and combined search
Use metadata constraints and complementary search signals to reduce irrelevant or inappropriate results.
Practical output: A filtered retrieval strategy.
Session 08 Ranking and assembling context
Select passages for the answer, manage context limits and preserve their relationship to original sources.
Practical output: A context-assembly procedure.
Session 09 Generating source-aware answers
Ask the model to answer from the selected evidence and show references that a reviewer can inspect.
Practical output: An answer-generation prompt and interface.
Session 10 Measuring retrieval quality
Create relevance judgements and examine whether the correct evidence appears among the retrieved results.
Practical output: A retrieval evaluation report.
Session 11 Measuring answer quality
Score factual support, completeness and relevance while distinguishing retrieval failures from generation errors.
Practical output: An answer evaluation and error taxonomy.
Session 12 Permissions, missing evidence and updates
Test restricted sources, unsupported questions and replaced documents, then define update and escalation behaviour.
Practical output: An access-and-update test suite.
Session 13 Knowledge-assistant integration
Combine ingestion, retrieval, source-aware answers and evaluation into a small, maintainable application.
Practical output: A working knowledge-assistant prototype.
Session 14 Demonstration and maintenance handover
Show successful and unsuccessful questions, explain evaluation results and propose ownership of the knowledge lifecycle.
Practical output: A final review and maintenance guide.
Apply the learning
An approved-knowledge assistant
Build an assistant for a bounded collection such as product guidance, training notes or internal procedures. Show the evidence behind each answer and provide useful behaviour when the sources cannot support a response. Your project must include questions with known relevant passages, not only questions chosen because the prototype already answers them well.
Your project evidence can include
- A source register, ingestion routine and retrieval configuration.
- A working assistant with inspectable references.
- Separate retrieval and answer evaluations plus a maintenance plan.
Use anonymised, public or suitable synthetic material where workplace information is confidential. The issued assessment brief confirms the required evidence.
Assessment, feedback and certification.
- Exercises assess source preparation, search comparison and context design.
- The project is evaluated against a documented question set containing both answerable and unsupported requests.
- A review discussion explores errors, permissions and how changed documents will be incorporated.
Teaching and the relevant course certificate are provided through Kent Business College in the UK. Completion requirements are set out in the module information; attendance alone should not be assumed to meet every assessment requirement.
Questions worth asking.
Does RAG eliminate incorrect answers?
No. Retrieval may return the wrong material and the model may still misinterpret it. You will evaluate both stages and design review or abstention behaviour.
Must every source go into a vector database?
No. Retrieval can use search indexes, databases or existing services. The module compares approaches against the source type and question.
Can I use an internal knowledge base?
Yes, with permission and suitable handling of confidential information. A public or synthetic collection is available as an alternative project basis.