How do I stop an AI agent from taking an action it should not take in a senior housing & care operator?
You do not manage that risk by monitoring agent behaviour. You remove the authority. An agent should be able to read what the requesting person may already read, propose an action, and nothing more; the write, the submission and the clinical or financial ruling stay with a named human who can be held responsible. An action that is architecturally unavailable cannot be triggered by a bad prompt, a compromised extension, or an unpredictable model.
The risk moved from accuracy to authority
The first generation of AI review asked whether a model was accurate. The current question is different: what can an autonomous agent do on its own authority, at machine speed, inside a system of record, before anyone notices. Accuracy is a quality question. Authority is a control question, and only the second one is architectural.
This matters more in senior housing & care than in most sectors, because the record touches medication administration, regulatory submission and payroll simultaneously. An action taken wrongly is not a data-quality event; it is a clinical or compliance event.
Five controls a security committee should require in writing
Each of these is verifiable during diligence, and each fails safe rather than relying on detection.
Closed autonomous-action class. No pathway exists for a model to write to the record, submit to a payor or regulator, or change a care plan without a human accepting it — closed, not configurable.
Suggestion-only output. Model output is written as a suggestion attributed to a named model version, and a human with authority to be wrong must accept it before it becomes part of the record.
Retrieval bounded by identity. Model calls are constructed only from data the requesting identity may already read under fail-closed row-level security, so a prompt cannot widen a query or cross a tenant boundary.
No sensitive data leaving to a model provider, and no training on operator data. Where no provider agreement exists to cover protected data, the pathway is closed rather than permitted with controls.
Reconstructable accountability. Prompt inputs, model version, accepting or rejecting user and disposition are written to an append-only audit log so an AI-assisted entry can be reconstructed months later.
Why withholding authority is a governance decision, not a limitation
Vendors are under commercial pressure to describe agents as autonomous, because autonomy sounds like leverage. In an operating environment where a human has to attest to a regulatory submission and a licensed clinician has to sign a care decision, autonomy at the point of action removes the person the regulator expects to find.
DATA → TRUTH → DECISION → EXECUTION. Data arrives from the systems that already hold it; truth is what the operator has ruled the business will accept; decision is the action taken on that ruling; execution is what actually happened, recorded against it.
The agent can participate in DATA and it can propose within DECISION. TRUTH and EXECUTION belong to the operator. When systems disagree, the operator governs — an agent does not.
What to ask any vendor, including us
The questions that separate a structural answer from a reassuring one are short, and the answers should be in writing.
Which of your AI surfaces can write to the record without a human accepting the output?
Is the restriction architectural, or a configuration that an administrator can switch on?
Can a prompt cause a query to return data the requesting user could not otherwise read?
Is our data used to train or fine-tune anything you sell to another operator?
Can we reconstruct, months later, which model version produced a suggestion and who accepted it?
Can we disable an AI surface per facility, and is that disablement enforced at the server or only hidden in the interface?
What this answer does not assert
This answer describes design constraints and governance rules. It is not a certification, an audit result, or a security guarantee.
No third-party AI audit, bias audit, or penetration-test conclusion is cited here, and no threat-intelligence claim is made.
Nothing here characterises any other company’s product, agent framework, or incident history.
Per-surface maturity states, including controls that are still being hardened, are published on the AI governance page rather than summarised as complete here.
https://seniorcre.com/answers/govern-ai-agents-senior-living