Copilot Studio Knowledge Sources: Why PDF Grounding Fails on SOPs

Someone points a Copilot Studio agent at the SharePoint library holding standard operating procedures, work instructions and quality manuals, asks three questions in the maker portal, and gets three clean answers with citations. Everyone is impressed.

On Monday a shift supervisor asks what the hold time is before releasing a batch after a filter change. The agent gives an answer that reads perfectly — numbered, confident, in the house style — and it is wrong.

In a business running an ISO 9001 or ISO 13485 quality system, or under a regulator that expects controlled procedures, that is not a poor user experience. It is a deviation. The reflex is to blame the model. It is almost never the model.

Retrieval-augmented grounding does not let an agent read your documents. It converts each document into text, cuts that text into fragments called chunks, converts each chunk into a numeric vector, stores them in an index, and at question time retrieves the handful of chunks numerically closest to the question — then asks the model to answer from those.

• Extraction — the PDF is turned into plain text
• Chunking — the text is cut into pieces small enough for an embedding model
• Embedding — each chunk becomes a vector representing its meaning
• Retrieval — the question is embedded the same way; the closest chunks come back
• Generation — the model is handed those chunks and asked to answer from them

Note what the last step implies. If the chunk containing step 4 of a procedure arrives without the heading that names the procedure, and without the warning that sits above step 4, the model cannot know they belong together. It will still answer — fluently.

Be precise, because the marketing and the mechanics differ. Copilot Studio supports several knowledge-source types: public websites via Bing, files uploaded into Dataverse, SharePoint sites via Graph search, Dataverse tables, enterprise connectors indexed by Microsoft Search, and Azure AI Search. For uploaded and synchronised files, Microsoft's pipeline is plain: files are stored in Dataverse, automatically divided into smaller pieces, indexed as vector embeddings, and matched semantically. "Automatic" is the operative word — you do not control the chunking.

Two of these bite quality teams immediately: the 7 MB threshold without a Microsoft 365 Copilot licence excludes most scanned quality manuals, and the 4–6 hour sync with no manual refresh means a released revision is not instantly live.

Generic RAG guidance assumes prose. An SOP is not prose, and six things go wrong at once: numbered steps split across a chunk boundary (a fixed-size splitter does not care that it landed between step 6 and step 7); warnings orphaned from the step they qualify ("CAUTION: isolate and lock off" sits in a shaded box above step 9); parameter and tolerance tables flattened into unreadable strings; revision headers and page footers polluting every chunk; multi-column and scanned PDFs read as interleaved nonsense; and meaning locked inside images — P&ID extracts, HMI screenshots, panel photos with callouts.

Underneath all six sits the one no chunker fixes: superseded revisions sitting in the same library as current ones. Retrieval has no concept of "controlled" — it returns the closest chunk, whether that chunk is from Rev 5 or the Rev 2 someone forgot to archive.

If your documents are clean, born-digital Word files with real headings, Copilot Studio's built-in knowledge source is often good enough. Try it first. When it is not — and for procedural documents it usually is not — build the index yourself in Azure AI Search and connect Copilot Studio to it. That moves five decisions inside your control:

• Extract with layout, not plain text — Azure AI Document Intelligence's layout model performs OCR, extracts tables with row/column spans, and assigns logical roles (title, sectionHeading, pageHeader, pageFooter)
• Split on structure, then on size — the Document Layout skill produces chunks aligned to document structure, with heading depth controlled and headings surfaced as separate fields
• Keep tables whole — a parameter table is one chunk, with its caption and the heading above it
• Enrich every chunk with metadata — document number, title, revision, effective date, status, site, discipline, equipment class, owner
• Filter superseded revisions out at query time, not by hope — the step that turns a clever assistant into something a quality manager will sign off

None of the above survives a disorderly library. Three things are non-negotiable before an index is built. One controlled source — if the current revision exists on SharePoint, in a QMS, on a shared drive and in a folder called "SOPs FINAL v2", the agent will find all four. Superseded revisions out of the indexed set — not deleted (retention usually forbids that), but moved to a library the agent is not pointed at, or tagged with a status the retrieval filter excludes. And a named owner — someone who signs off which documents are in scope, and whose approval workflow updates the index metadata when a revision is released.

Three questions in the maker portal is not a test. Collect 40–60 real questions from supervisors, technicians and quality staff — the ones they actually ask, in the words they use, including the ambiguous ones. For each, record the correct answer and the specific document and revision it comes from.

The number that matters is the confidently-wrong rate. An agent that is right 90% of the time and confidently wrong 10% of the time, with no way for the user to tell which is which, is more dangerous than no agent. So make verification cheap: keep in-text citations visible, and know your channel limits — Teams shows at most 20 citations, titles truncated near 80 characters.

An agent must not be the system of record for a controlled procedure — it is a retrieval aid that points to the controlled document. An answer without a revision-specific citation is unusable in a regulated environment: if the citation says "Filtration SOP" rather than "SOP-PRD-014 Rev 5, effective 12-Mar-2026", a supervisor cannot verify it and an auditor will not accept it. No chunking strategy fixes contradictory source documents — better retrieval surfaces the contradiction faster, which is useful, and is a document-control problem, not an AI one.

Two more: sensitivity labels and password protection silently defeat indexing — encrypted documents show as ready and return nothing. And a custom Azure AI Search pipeline is a system you now own — indexers, skillsets, embedding costs, Document Intelligence page charges, region constraints, and a re-index whenever the chunking strategy changes. Retrieval quality also decays as the library grows; an index that answers well over 200 procedures will not necessarily answer well over 2,000.

Before anyone builds anything, answer four questions this week:

• How many copies of your current SOP set exist across SharePoint, the QMS and shared drives — and which one is controlled?
• What proportion of your procedures are scanned images rather than born-digital PDFs, and how many carry sensitivity labels that apply encryption?
• Can you produce, today, a machine-readable list of every document number with its current revision and effective date?
• Which 40 questions would you use to prove the agent is safe — and who signs off that it passed?

If the third question has no answer, start there. A revision register is the single artefact that makes the retrieval filter possible, and it is useful whether or not the agent ever gets built. We build these on Copilot Studio, SharePoint, Azure AI Search and Azure AI Document Intelligence, with Power Automate handling the approval and re-index loop — with an evaluation set before a go-live date.