Keep your AI assistant’s business information current
Track source changes, effective dates and answer checks so your customer AI assistant reflects the business information your team has approved.
By Noe Guerrero III
When your hours, services or offers change, include the customer AI assistant in the update. Editing the website doesn't prove that the assistant is using the new information. Give the change an owner, identify the source to replace, confirm the refresh, and check the answers customers will actually receive.
A small update log makes this manageable. It connects the business decision to the material the assistant uses and the evidence that the change worked.
Record when the information applies
Imagine an equipment rental shop in McAllen moving its Saturday closing time from noon to 2 p.m. This is a hypothetical example, not an actual business policy. The new hours begin next month. A staff member updates a document today, but customers still need the current hours until the change takes effect.
That calls for separate effective and review dates. “Reviewed today” means someone checked the information. It doesn't mean the new policy starts today or stays valid forever.
Use a short record for each meaningful change:
Business fact: Saturday closing time, with the approved current and upcoming answers.
Applies from and until: The exact effective date and any expiration or replacement date. Specify the business's local time when timing matters.
Approved source: The page or document location, version or revision reference, and person who can approve the fact.
Last reviewed: Who checked it, when they checked it, and the next review or known change to watch for.
Replacement work: The old source to retire, the updated source to activate, the responsible person and the observed completion status.
Answer check: Questions asked, expected facts, observed answers and the date tested.
This can be a shared sheet. It doesn't need to become another large software project.
If the website, an old uploaded PDF and a staff note disagree, settle the business policy first. A newer file timestamp isn't enough to establish which instruction is authoritative. Mark an unresolved change as pending rather than quietly treating an unapproved answer as current.
Confirm the replacement reached the assistant
Ask the person maintaining the assistant to show which sources are active and how changes reach it. Does it read a connected page, search an indexed copy or use files someone uploaded? What confirms that the update completed?
Microsoft's Azure AI Search indexer guidance illustrates why this matters: processing source changes depends on change detection and indexer execution. Don't assume a saved document has already refreshed the assistant's searchable content. Your particular platform may use a different process.
Track the retirement of old material too. Microsoft's document deletion guidance describes cases where removed source documents leave indexed copies behind. Ask how your setup handles replacements and removals, and verify that superseded material is no longer an active answer source. Keep historical records separately if the business needs them.
A date written inside a document also isn't proof that the assistant will apply it correctly. Have the maintainer demonstrate how the current and upcoming policies are selected. For a change that must happen at a specific time, agree on the activation step and the person who will verify it then.
Check the customer-facing answers
After the refresh, start a fresh conversation and ask the affected question. Try another wording and a question that should remain unchanged. For the hypothetical rental shop, check:
“What time do you close this Saturday?” The answer should match the hours applicable to that date.
“Will you be open until 2 next month?” The answer should preserve the approved effective date and avoid applying the new hours early.
An unrelated service question. The update shouldn't quietly introduce a different service promise.
If the assistant supports English and Spanish, check both. Record the facts you expect, rather than requiring one exact sentence. Save the observed answer and source reference where the system makes that available.
Microsoft's guide to evaluating AI answers against retrieved sources separates finding relevant material, staying faithful to it, and covering the expected facts. My practical takeaway: compare the answer with the approved business fact as well as the source it used. An answer can repeat an outdated document faithfully and still be wrong for today's customer.
When a check fails, keep the change open and use the agreed staff fallback for unresolved information. Our customer AI assistant handoff guide explains that workflow.
Close the record when the source replacement and answer checks are complete. Keep it available for the next person who changes the hours. A successful check covers the cases you tried; it doesn't guarantee every future response. The log gives your team a practical place to start when an answer needs correcting.