Copilot Studio Topic Design: Writing Instructions That Reduce Agent Hallucination on ERP Data

The appeal of a Copilot Studio agent over ERP data is obvious: a plant manager asks "what is the stock of part 4471 at Jebel Ali" in plain English and gets an answer, without learning the ERP's transaction codes. The risk is equally obvious the first time the agent invents a number.

A confidently wrong stock figure is worse than no agent, because someone acts on it. And the failure is rarely the language model being "bad" — it is the agent answering from its general training instead of from the ERP, or the orchestrator routing the question to the wrong topic because it could not tell them apart.

Both of those are controlled at design time, in how the topics, tools and their descriptions are written. The model is the last thing to tune, not the first.

Copilot Studio agents respond in one of two ways. With generative orchestration — on by default — the agent chooses the most appropriate combination of topics, tools and knowledge for each query, using their descriptions to decide. With classic orchestration, each topic has a set of trigger phrases and the agent matches the user's message against them using natural-language understanding.

Either way, the agent is making a routing decision before it produces an answer: which topic or tool handles this question. If that decision is wrong — the "check stock" question routes to a general knowledge answer instead of the stock-lookup tool — the response will be wrong no matter how good the underlying data is.

This is why topic design is upstream of everything. The quality of the routing decision is set by how clearly each topic and tool declares its purpose.

Under generative orchestration, the description you write for each topic and tool is what the agent reads to decide when to use it. A vague description — "handles inventory" — invites the orchestrator to guess. A precise one — "use this tool to look up current on-hand stock quantity for a specific part number at a specific plant from the ERP; do not use it for stock valuation or historical movements" — tells the orchestrator exactly when to route here and when not to.

The discipline is to write each description as an instruction to the router: what this does, what inputs it needs, and explicitly what it is not for. The "not for" clause is the one teams skip, and it is the one that stops a stock-quantity question being answered by a valuation topic.

Under classic orchestration the equivalent lever is trigger phrases — Microsoft's guidance is five to ten per topic, covering the real ways users phrase the request. Either way, you are teaching the agent to tell your topics apart. The clearer that boundary, the fewer wrong routes, and wrong routes are where a lot of "hallucination" actually originates.

The second source of hallucination is answering a factual ERP question from the model rather than from the ERP. The rule for an ERP agent is simple: any answer that contains a number, a status or a record must come from a tool that read it — a connector, an agent flow, or a REST call to the ERP — not from generated text.

In practice this means the "what is the stock of part 4471" path always runs the stock-lookup tool and returns its result, and the agent is instructed to present that result rather than compose a plausible-sounding figure. Generated language is fine for the wrapper — "The current stock at Jebel Ali is…" — but the value inside it has to be the tool's output.

This is also where write-back needs care: a tool that changes ERP data should be gated so the agent confirms with the user before running it, which Copilot Studio supports as an explicit per-tool setting. Read questions ground in a lookup; write questions ground in a lookup and a confirmation.

The most under-used instruction in agent design is permission to decline. An agent under pressure to be helpful will fill a gap with a guess unless it has been explicitly told that "I do not have that information" is an acceptable, preferred response when no grounded tool can answer.

For an ERP agent this is the single highest-value guardrail. If the stock-lookup tool returns nothing for a part number, the correct behaviour is to say so — not to estimate. Building that into the topic design, and testing it with deliberately unanswerable questions, is what separates an agent operations will trust from one they quietly stop using after it invents a figure.

Test the failure paths as hard as the happy paths. The question that has no answer is the one that reveals whether your design controls hallucination or merely hides it.

A reliable Copilot Studio ERP agent is a design achievement before it is an AI one. Write topic and tool descriptions precise enough that the orchestrator routes correctly, ground every factual answer in a tool that reads the ERP, gate write-back behind confirmation, and explicitly permit the agent to decline when it cannot ground an answer.

Do that and generative orchestration becomes an asset — it flexibly routes real questions to the right grounded tool. Skip it and the same orchestration confidently routes questions to whatever topic looked vaguely relevant, and the agent's fluency becomes a liability.

The model is not the bottleneck. The topic design layered on top is. That is the same lesson that shows up everywhere in operational data work, and it holds here: get the foundation right, then let the AI do its job.