> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chargerdojo.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Read your report
> The five verdicts a run can return, why a warning is never a failure, and why a red result can be trusted to name the exact rule that broke.
This is the chapter that decides whether you believe anything else here.
A conformance report is only worth reading if you know two things about it: what its
verdict actually licenses you to do, and whether a green tick could have gone red. Most
tools answer neither. Here are both.
## Fetch the report
```bash theme={null}
curl -fsS "https://chargerdojo.com/api/v1/testing/runs/$REPORT_ID" \
-H "Authorization: Bearer $DOJO_KEY"
```
Returns the full report: the per-check verdicts, the expected versus actual outcome, the
validation messages, and the recorded request and response exchange behind every check.
The summary counts live in `data.summary`. Reports are owner scoped and retained for your
plan's retention window, which `GET /api/v1/usage` reports.
## The five verdicts
A run counts four things: checks that **passed**, checks that **failed**, checks that **warned**,
and checks that were **unobservable**, meaning they graded nothing because the response carried
nothing to grade. The verdict is derived from those counts and from what the run was allowed to
cover.
| Verdict | What happened | What it licenses you to do |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| **Fully compliant** | Nothing failed. Nothing was declined. | Ship. Every rule the spec requires is met, and every recommendation was taken. |
| **Compliant** | Nothing failed. Some recommendations were declined. | **Ship.** This is a pass. See below, because this row is the one people misread. |
| **Not compliant** | At least one required rule is broken. | Fix the failures. Nothing outranks a failure: a run with a hundred passes and one failure is not compliant. |
| **Partial coverage** | Nothing failed, but something was not graded: a module your plan does not include, or checks that came back unobservable. | Read it as a scoped pass. What ran met the rules; what was skipped is named in the report, and proves nothing either way. |
| **No checks ran** | The run finished without running a single check. | Nothing. An empty run proves nothing, so it is never a pass. Run a suite against the endpoint to get a verdict. |
These five words are worked out from the counts when a report is displayed. The API sends the
counts, not the word: `data.summary` carries `total`, `passed`, `failed`, `warned` and
`unobservable`, and
`data.status` says whether the run reached a verdict at all. So gate CI on `failed`, and read
the word off the screen.
## Why a warning is not a failure
The OCPI specification says two different kinds of thing. It says what you **must** do
(`SHALL`, `MUST`), and it says what you **should** do (`SHOULD`, "it is advised"). These
are not the same, and the difference is not a matter of emphasis. It is the difference
between breaking the protocol and declining a suggestion.
**A partner who ignores a `SHOULD` has broken nothing.**
So we will never fail you for one. We will never call you "partially compliant" either,
because that phrase would send you to go and fix code that already works. If your report
says **Compliant** with three warnings, you are compliant. The warnings are there because
you might want to know, not because you owe anyone anything.
This is the single most important sentence in this documentation, so it is worth being
blunt about the alternative. A tester that fails you for declining a recommendation is
not being strict, it is being **wrong**: it fails a compliant partner and sends a
working integration back for repair. That is the worst thing a conformance tool can do,
worse than missing a real bug, and it is the failure this product is built to avoid.
The verdict word carries the pass. Only the count carries the amber. In CI, that means
gate on `data.summary.failed`, never on `warned`.
## Why a red result can be trusted
Here is the question nobody asks a test suite, and everybody should:
> If your check were broken, would it still show green?
A check that can never fail certifies everybody. It looks exactly like a check that
works, it costs nothing to write, and it will happily approve a partner who is completely
broken. A test suite full of those is not a weak test suite. It is a **decorative** one.
So we test our own tests. Alongside the tester we run a set of **mutant peers**: copies
of a correct OCPI server, each deliberately broken in exactly one way. One sends no
pagination headers. One returns an HTTP error where the spec requires a 2xxx rejection to
ride HTTP 200. One accepts any credentials token you give it. One emits timestamps in a
format the spec forbids.
Each mutant breaks one rule, and the check that guards that rule **must** turn red. If it
does not, we have found a check that cannot fail, and we either fix it or delete it.
**Every conformance check on 2.1.1, 2.2, 2.2.1 and 2.3.0 has been proven capable of
failing**, and that is enforced in our build rather than asserted in a blog post. A check
that stops being falsifiable breaks our own test suite before it ever reaches you.
You can see it on the check in front of you, not just in the aggregate. Expand a failed
conformance check in your report and it names the mutant that proved that check can go red,
along with the rule that mutant breaks.
Silence there means one of two things, and neither is a check we skipped. A few conformance
checks name no mutant at all: where a check grades only that a partner accepted something
the spec also lets them refuse, a mutant aimed at it would mean failing partners who broke
nothing, so we write down why no mutant can exist instead of inventing one. A failed step in
a scenario names none either, because the scenario sweep described below proves its checks
separately and that proof is not carried onto the step yet.
### The limits of that claim, stated plainly
**"Conformance check" is narrower than it sounds, and the difference matters.** A
conformance check asserts one rule against one exchange. A **scenario** walks a story end
to end, chaining its own output from step to step. They run the same way and appear in the same
suite list, so if you are weighing this claim, the count above is the modules whose names do not
begin with `scenario_`.
Scenarios have their own sweep, and it is stricter. A conformance mutant has to redden
*something*; a scenario mutant has to redden the check we **named** in advance, and every check
in the flow has to be proven, not just covered. So the claim holds for scenarios too. It is
stated separately because the two sweeps prove different things and collapsing them would
overstate what the conformance one does.
Terms in this section that are ours rather than the specification's (scenario, unobservable,
warning) are defined in the [glossary](/guide/glossary).
### What our own sandbox does not do
A verdict is only as good as the peer behind it, so these are the places our sandbox is not a
complete OCPI implementation. Nothing here affects a run against **your** endpoint; it affects
what a run against the sandbox proves.
* **We are not the EVRoaming Foundation.** Internal conformance coverage is not an external OCPI
certification. Internal parity coverage is not an external OCPI certification.
* **ChargingProfiles serves the Receiver interface only** on 2.2.1. The Sender interface is not
implemented, so a green 2.2.1 run has not touched it.
* **The sandbox reserves a few Commands ids.** `UNKNOWN-LOCATION` and `UNKNOWN-EVSE` always
answer `REJECTED`, and `UNKNOWN-SESSION` always answers `UNKNOWN_SESSION`, so you can exercise
the rejection paths on purpose.
* **Other roles register without endpoint discovery.** On 2.3.0, endpoint discovery at
registration completes for CPO and EMSP peers.
* **The Sender and Receiver interfaces are kept apart from 2.2 onward.** The sandbox's Sender
endpoints serve only the objects it owns, so a Session you push to
`/sessions/{country_code}/{party_id}/{session_id}` does not come back on `GET /sessions`: those
two URLs belong to two different parties. On 2.1.1 the sandbox still merges them, so there your
push does come back on the Sender list. One exception, on 2.3.0 CDRs: a CDR can never be changed
once sent, so a CDR pushed to the sandbox before this split shipped has no way to pick up the
marker that would hide it, and those older CDRs are still served on `GET /cdrs`. Anything you
push from now on is split correctly.
## The exchange behind every check
Every check that was exercised carries the HTTP request that was sent and the response that came
back, the rule it was testing, and whether that rule is one the specification requires
or only recommends (`severity`, which is `shall` unless it says `should`).
A check can also carry the clause behind it: the specification file, the section, and the anchor
that survives the document being re-vendored. Where someone has read that clause and written it
down, you get its words too, and whether it binds as SHALL or only SHOULD.
Not every check carries one yet. A citation with no strength is an address we recorded but have
not read back to you; one with a quote has been checked against the specification text on every
build. Charging journey assertions carry a reference on every assertion.
This is deliberate, and it is the feature that ends arguments. When you forward a report
to a partner, they do not have to trust our verdict. They can read the exchange and see
for themselves. A conformance report that says "failed" without showing you what was sent
is asking for faith, and no engineer under deadline pressure is going to give it.
## What to do next
Send the evidence to your partner.
Common failures and what they usually mean.