No system simply is safe. Safety is a property we argue for: a claim, structured reasoning, and evidence that supports the reasoning. A safety case — and, more broadly, an assurance case — is the organised form of that argument. This note explains what a good one contains, and the ways they usually fail.
What a safety case is
A safety case is a structured argument, supported by evidence, that a system is acceptably safe for use in a defined context. The top-level claim is decomposed into sub-claims, each backed by evidence: analysis, test results, review records, operational experience. Goal Structuring Notation is a common way to draw the argument so that its structure can be read and challenged.
The word that matters most is argument. A folder of test reports is not a safety case. The case is the reasoning that says what the evidence demonstrates, and why that is enough for the claim being made.
How assurance cases fail
Evidence without argument. The most common failure: a large volume of evidence with no visible reasoning connecting it to the claim. Nobody can tell what the tests demonstrate, what they assume, or what remains unaddressed.
Claims too broad to be true. “The system is safe” is not a claim that evidence can support. Credible claims are bounded: this system, in this operating context, under these assumptions, to this defined level of confidence.
Stale cases. A safety case describes a system. When the system changes and the case does not, the case comes to describe something that no longer exists — often worse than no case at all, because it inspires confidence it no longer earns.
Untraceable claims. A claim whose supporting evidence cannot be traced and re-checked is a statement of faith. Every claim in the argument should lead, in a few steps, to evidence someone else can find and examine.
Assumptions nobody owns. Arguments rest on assumptions — about the environment, operators, failure rates, interfaces. If each assumption has no owner and no review, the argument’s foundation is unexamined.
What this means for autonomous systems
Autonomy makes assurance harder, because system behaviour depends on software responding to conditions that were not fully enumerated in advance. The practical response seen in current research and practice is to work with the grain of that difficulty: bound the claims — what the system may do, under what conditions, with what human oversight — monitor operation against those bounds, and treat the evidence as ongoing rather than one-off.
That last point generalises. An assurance case is not a document produced at the end of a programme; it is a living argument that has to keep step with the system it describes.
Where this connects to our work
At Fallon Apus Orbital Systems, assurance is one of three current areas of focus. We are investigating software architectures that keep engineering claims, evidence and system change connected — so that an argument about a system can be maintained, challenged and updated, not merely filed.
Status: research and prototyping. Fallon Apus makes no claims of certification, approval or deployment.
If your organisation is thinking hard about assurance for software-intensive or autonomous systems, we would value the conversation — write to Paul at paulorbitalsystems@proton.me. Weekly notes on these topics go to newsletter subscribers first; you can join below.