To build an AI knowledge base is to create a maintained collection of approved questions and answers and put a retrieval and writing layer on top that returns every answer with its source. The build takes six steps: define who asks which questions; allow and limit sources; review conversation knowledge legally and factually; build small knowledge units, one question, one answer, one source each; return answers with evidence and say so when the evidence is missing; feed the responses back into the next maintenance round. In a small business, much of the valuable knowledge is not in handbooks. It lives in sales, support and advisory calls. The wrong starting point is, “Upload everything to AI.” The better starting point is: Which questions should the system answer, who may see which sources, and who approves the answer?
In my business the knowledge base therefore starts with the conversation, not the document. The reviewed intermediate product is the pattern register for call analysis: recurring questions, each with my clearest answer so far and its source location. This article shows how that register becomes an answer system a team can use. Which role handles which task in my business is under AI employees.
Separate the knowledge base from the AI answer layer
The knowledge base is the maintained collection. It contains approved questions, answers, sources, versions and review dates. AI is the search and writing layer above it.
A robust answer flow has four simplified steps:
- A person asks a question.
- The system retrieves relevant, approved knowledge entries.
- AI formulates an answer from those results, and only those results.
- The answer shows its evidence or says that the available basis is not sufficient.
The pattern behind this is called retrieval augmented generation, or RAG. Microsoft describes it as a pattern that extends a language model by grounding its responses in your own content. Grounding is the point: the model answers from the retrieved entries, not from what it saw in training. This reduces unsupported answers but does not eliminate them. Sources can be outdated, contradictory or poorly matched, and a model can stretch a result further than it goes. Maintaining the collection therefore remains the most important part, and every answer shows its source: whoever sees the evidence can check the answer in thirty seconds.
Which call content belongs in the system
A raw transcript is not a knowledge base. It contains filler, personal data, one-off situations and contradictions. You need a reviewed intermediate product first; in my case that is the register. An approved entry should contain at least:
The question in customer language is the most important field: retrieval finds “What does something like this cost, roughly?” only if the entry is called that and not “Pricing, initial consultation.”
Build an AI knowledge base: the process in six steps
1. Define users and questions
An internal support specialist needs different answers from a website visitor. Write down ten real questions and, next to each, who asks it: a customer before buying, a customer after buying, a new colleague in her first week, you yourself in a sales conversation. Only then choose sources and technology. The ten questions later become your test set.
2. Allow and limit sources
Useful sources include approved FAQs, process descriptions, product information and reviewed call patterns. Uncontrolled archives, private notes and recordings without a clear legal basis do not belong in the system. Write down the list of allowed sources, with an owner for each.
3. Review conversation knowledge
Consent, purpose, access and retention come before analysis. Replacing names with roles and companies with industries is pseudonymisation, not automatic anonymisation; the entries remain personal data while attribution is possible. I am a developer, not a lawyer, and this is not legal advice; when in doubt, have your specific case checked. Transcription software compared explains how local tools can shorten the data path.
4. Build small knowledge units
One entry answers one question. Split long call summaries while preserving context and source. Microsoft explicitly recommends chunking large documents for RAG so that individual portions can be matched independently. Retrieval can then find the relevant paragraph rather than searching an entire 45-minute transcript. The register already has this shape: one entry, one question, one answer, one source location.
5. Return answers with evidence
The interface should show source, freshness and ownership. If the evidence is not sufficient, “There is no approved answer yet” is better than a convincing invention. That answer is not a failure of the system; it is a work order: the question comes up, and the collection has no entry for it yet.
6. Feed feedback back into maintenance
Track unanswered questions, weak results and answers that users frequently correct. They become the next maintenance queue. Knowledge management is not a one-time import; in my business, maintenance runs quarterly, like the register.
A worked example with demo data
The following example is constructed; every call, number and statement is fictional. A trade business with twelve staff analyses ten demo first conversations. Afterwards the register holds fourteen entries, three of them about setting up a new workflow. The most frequent entry:
Now a new colleague asks the system: “Customer says he has no time for the setup, what do I answer?” Retrieval finds the entry through the customer phrasing, the model formulates the short answer and attaches the three source locations and the review date. The colleague sees that the answer was approved by the owner and was given this way in three conversations.
Second question, a week later: “What do I answer when the customer asks whether we also do the setup at weekends?” No entry matches. The system answers, “There is no approved answer yet,” and the question lands on the list for the next maintenance round. That is the real yield: a list of the questions your collection does not answer, instead of ten improvised answers that contradict each other.
Permissions and separate knowledge spaces
Not every source belongs in every channel. Separate at least:
- public: approved website and FAQ content;
- internal: processes, training and internal explanations;
- confidential: personal data or contractually protected information.
Access control must apply before the answer is generated. If someone may not see a source, the system must not reveal its content through a summary either. Microsoft lists this as a challenge of its own in its RAG overview: users and agents must only retrieve authorised content. Technically this means the permission filter sits at retrieval, at step 2, and is applied to every single query. A language model that has already read the source cannot unread it. For a small business, the “visibility” column in the register and a system that searches only the entries matching the channel is often enough at the start.
Search, RAG or a knowledge graph?
These approaches can work together. A knowledge graph from calls can provide relationships, text retrieval can find passages and a language model can produce a readable answer. For a small business, a maintained register is often the right first version.
Typical mistakes when building one
- Uploading everything. A collection without selection is a collection without an owner. Every answer from it is only as good as the worst source inside it.
- No validity date. An answer from two years ago sounds just as confident as one from yesterday. Without a review date nobody notices.
- No “I don't know.” A system that always answers will eventually invent. The behaviour when evidence is missing is a decision you make before launch.
- Permissions after the answer. Filtering after the model has read the source is filtering too late.
Three direct outputs
A good knowledge base is not an archive. It feeds workflows. Common customer questions become FAQs from conversations. Recurring sales barriers become objection cards. Knowledge with owners and review dates becomes AI knowledge management.
How Gustav helps
Gustav handles the preparation in my setup. Whisper transcribes locally, followed by the fixed pseudonymisation pass. Gustav distils questions, answers, objections and source locations in batches of ten to twenty calls, with a sample review after every batch. The result is a register whose entries already have the shape a knowledge base needs: a question in customer language, my reviewed answer, the source location. An internal knowledge base is one of the outputs he is built for from the start, alongside a playbook, FAQ and objection library. Expert approval stays with the owner, and you define the permission model; both are decisions of your business, and that is exactly why they are not built into a role.
Frequently asked questions
What is an AI knowledge base?
A maintained collection of approved questions, answers and sources, with an AI handling retrieval and wording on top. The collection is the product; the AI is the interface. A good AI knowledge base shows the evidence for every answer and says when it has no approved entry for a question. That is what separates it from a chatbot that answers everything.
How do I create a knowledge base with AI?
In six steps: define users and ten real questions, allow and limit sources, review conversation knowledge legally and factually, build small units, return answers with evidence, feed responses back into maintenance. The order is the point: technology comes last.
What is the difference between a knowledge base and a knowledge graph?
The knowledge base answers a question with a source; the knowledge graph shows what is connected to what. In the knowledge base you search for an answer and get it with evidence. In the graph you ask which entries depend on an answer you are about to change. Both can grow out of the same register. For most small businesses the knowledge base is the first step; the graph comes when dependencies start to matter.
Can an AI knowledge base run locally?
Transcription, yes: in my setup Whisper runs locally on the machine itself. Whether retrieval and the language model run locally or in the cloud depends on the setup; in my case only the pseudonymised extracts go into cloud processing, and the raw transcript never leaves the machine. Which level fits you is decided by the visibility of your entries: for confidential knowledge, the data path is the first question, not the model.
How to carry on
Write down the ten questions from step 1 today, with the person who asks them next to each. Then find, for every question, the answer you last gave and the place where you gave it. If you have an answer with a source for seven out of ten, you have the core of your knowledge base; the other three are your first maintenance round.
The register before it is kept by Gustav in my business, and his package is in my community. The source list, visibility and the behaviour when evidence is missing are yours to define for your business, and you are not alone in that: in the calls we work through such decisions together, and whoever shows their source list in a post usually gets a question back that saves a later correction. Everything about it is under Community.
