Designing Topics and Instructions in Copilot Studio for Reliable ERP Query Agents

An ERP query agent that works in a demo and fails in production usually fails the same way: it answers a question it was never designed for, fluently and wrongly. A manager asks something slightly outside what the agent was built to handle, and instead of saying "I cannot answer that", it produces a confident, plausible, incorrect number. In operations, that is worse than no agent at all.

The property you need is reliability — the same question returns the same correct answer every time, and anything outside scope is refused rather than guessed. That property does not come from the language model being clever. It comes from how you design the agent’s topics and instructions to bound what it will attempt and ground what it answers.

Copilot Studio gives you both controls. Used deliberately, they turn a plausible chatbot into a dependable ERP query agent; left loose, they produce exactly the confident-wrong behaviour that kills trust on the first bad answer.

A topic in Copilot Studio defines a category of question the agent is built to handle and how it responds — including which data it reads and which action it takes. The discipline is to scope topics to the questions users genuinely ask, discovered from real demand, not to every question you imagine. "What is the stock position for this SKU at this warehouse?" is a topic; a vague "tell me about inventory" is not a design, it is an invitation to improvise.

Well-scoped topics are what make the agent predictable. Each one maps a recognised question to a defined path — the right query against the right governed source, returning the answer in a consistent shape. Because the path is designed, the answer is repeatable, which is the whole point.

The questions that do not match any topic are where reliability is won or lost. An agent with tightly-scoped topics and no catch-all improvisation knows what it does not handle, which is exactly the knowledge that lets it refuse gracefully instead of inventing. Scope narrow and deliberate; expand topics as real demand appears, not pre-emptively.

Every answer the agent gives about ERP data must come from a governed source, not from the language model’s own training. The model’s job is to understand the question and shape the response; the number must come from your data. In a Microsoft estate that governed source is typically a Fabric semantic model or a Fabric Data Agent sitting over the gold layer, so the agent answers from the same definitions the dashboards use.

This grounding is what makes the answers correct and consistent with the rest of the business. Ask the agent for last month’s revenue and ask Power BI the same question, and they should agree, because they read the same governed model. An agent that answers from the model’s general knowledge instead will be fluent and untethered — the fastest route to a number that contradicts the board pack.

Grounding also inherits the governed source’s security. When the agent queries through a governed model that honours row-level security, a user only gets answers from data they are entitled to see — so opening an ERP query agent to more people does not open the data to more people.

Instructions set the agent’s behaviour across topics: the tone, the format of an answer, the units and definitions to use, and — most importantly — what it must not do. The instruction that matters most on an ERP agent is the one that tells it to refuse or hand off anything outside its defined topics rather than attempt it. "If you cannot answer from the governed data within a defined topic, say so and offer to connect the user to a person" is the single line that prevents most confident-wrong incidents.

Good instructions also set guardrails on action. An agent that can read is low-risk; one that can change ERP data needs explicit instruction on what it may and may not do, and a human approval step on anything consequential. Instructions are where you encode "answer these, refuse those, never do this without approval" in language the agent follows every time.

Keep instructions specific and testable. Vague instructions ("be helpful and accurate") do nothing; concrete ones ("always state the period and source of a figure; never estimate a number that is not in the data; refuse questions outside your topics") produce behaviour you can actually verify.

Before an ERP query agent goes to users, it has to pass the same kind of test a semantic model does: a set of known-answer questions whose correct responses you can verify independently. Run them against the agent and confirm it returns the right answer in the right shape — and, just as important, that it refuses the out-of-scope questions you deliberately include in the test set.

That second half is what most teams skip. Testing only the questions the agent should answer tells you it works on the happy path; testing the questions it should refuse tells you it will not improvise in production. Both belong in the validation set, and both should pass before release.

So the pattern is: scope topics to real questions, ground every answer in a governed model that honours security, write instructions that refuse out-of-scope asks and gate any action, and validate against known-answer and should-refuse questions before release. Reliability is designed in through those four, not hoped for from the model. We build these on the Microsoft Fabric and Copilot Studio stack, grounded in the governed data foundation first — because an agent over ungoverned data is a fast way to be confidently wrong.