Skip to main content
This is the chapter that decides whether you believe anything else here. A conformance report is only worth reading if you know two things about it: what its verdict actually licenses you to do, and whether a green tick could have gone red. Most tools answer neither. Here are both.

​
Fetch the report

Returns the full report: the per-check verdicts, the expected versus actual outcome, the validation messages, and the recorded request and response exchange behind every check. The summary counts live in data.summary. Reports are owner scoped and retained for your plan’s retention window, which GET /api/v1/usage reports.

​
The five verdicts

A run counts four things: checks that passed, checks that failed, checks that warned, and checks that were unobservable, meaning they graded nothing because the response carried nothing to grade. The verdict is derived from those counts and from what the run was allowed to cover.
These five words are worked out from the counts when a report is displayed. The API sends the counts, not the word: data.summary carries total, passed, failed, warned and unobservable, and data.status says whether the run reached a verdict at all. So gate CI on failed, and read the word off the screen.

​
Why a warning is not a failure

The OCPI specification says two different kinds of thing. It says what you must do (SHALL, MUST), and it says what you should do (SHOULD, “it is advised”). These are not the same, and the difference is not a matter of emphasis. It is the difference between breaking the protocol and declining a suggestion. A partner who ignores a SHOULD has broken nothing. So we will never fail you for one. We will never call you “partially compliant” either, because that phrase would send you to go and fix code that already works. If your report says Compliant with three warnings, you are compliant. The warnings are there because you might want to know, not because you owe anyone anything.
This is the single most important sentence in this documentation, so it is worth being blunt about the alternative. A tester that fails you for declining a recommendation is not being strict, it is being wrong: it fails a compliant partner and sends a working integration back for repair. That is the worst thing a conformance tool can do, worse than missing a real bug, and it is the failure this product is built to avoid.
The verdict word carries the pass. Only the count carries the amber. In CI, that means gate on data.summary.failed, never on warned.

​
Why a red result can be trusted

Here is the question nobody asks a test suite, and everybody should:
If your check were broken, would it still show green?
A check that can never fail certifies everybody. It looks exactly like a check that works, it costs nothing to write, and it will happily approve a partner who is completely broken. A test suite full of those is not a weak test suite. It is a decorative one. So we test our own tests. Alongside the tester we run a set of mutant peers: copies of a correct OCPI server, each deliberately broken in exactly one way. One sends no pagination headers. One returns an HTTP error where the spec requires a 2xxx rejection to ride HTTP 200. One accepts any credentials token you give it. One emits timestamps in a format the spec forbids. Each mutant breaks one rule, and the check that guards that rule must turn red. If it does not, we have found a check that cannot fail, and we either fix it or delete it. Every conformance check on 2.1.1, 2.2, 2.2.1 and 2.3.0 has been proven capable of failing, and that is enforced in our build rather than asserted in a blog post. A check that stops being falsifiable breaks our own test suite before it ever reaches you. You can see it on the check in front of you, not just in the aggregate. Expand a failed conformance check in your report and it names the mutant that proved that check can go red, along with the rule that mutant breaks. Silence there means one of two things, and neither is a check we skipped. A few conformance checks name no mutant at all: where a check grades only that a partner accepted something the spec also lets them refuse, a mutant aimed at it would mean failing partners who broke nothing, so we write down why no mutant can exist instead of inventing one. A failed step in a scenario names none either, because the scenario sweep described below proves its checks separately and that proof is not carried onto the step yet.

​
The limits of that claim, stated plainly

“Conformance check” is narrower than it sounds, and the difference matters. A conformance check asserts one rule against one exchange. A scenario walks a story end to end, chaining its own output from step to step. They run the same way and appear in the same suite list, so if you are weighing this claim, the count above is the modules whose names do not begin with scenario_. Scenarios have their own sweep, and it is stricter. A conformance mutant has to redden something; a scenario mutant has to redden the check we named in advance, and every check in the flow has to be proven, not just covered. So the claim holds for scenarios too. It is stated separately because the two sweeps prove different things and collapsing them would overstate what the conformance one does. Terms in this section that are ours rather than the specification’s (scenario, unobservable, warning) are defined in the glossary.

​
What our own sandbox does not do

A verdict is only as good as the peer behind it, so these are the places our sandbox is not a complete OCPI implementation. Nothing here affects a run against your endpoint; it affects what a run against the sandbox proves.
  • We are not the EVRoaming Foundation. Internal conformance coverage is not an external OCPI certification. Internal parity coverage is not an external OCPI certification.
  • ChargingProfiles serves the Receiver interface only on 2.2.1. The Sender interface is not implemented, so a green 2.2.1 run has not touched it.
  • The sandbox reserves a few Commands ids. UNKNOWN-LOCATION and UNKNOWN-EVSE always answer REJECTED, and UNKNOWN-SESSION always answers UNKNOWN_SESSION, so you can exercise the rejection paths on purpose.
  • Other roles register without endpoint discovery. On 2.3.0, endpoint discovery at registration completes for CPO and EMSP peers.
  • The Sender and Receiver interfaces are kept apart from 2.2 onward. The sandbox’s Sender endpoints serve only the objects it owns, so a Session you push to /sessions/{country_code}/{party_id}/{session_id} does not come back on GET /sessions: those two URLs belong to two different parties. On 2.1.1 the sandbox still merges them, so there your push does come back on the Sender list. One exception, on 2.3.0 CDRs: a CDR can never be changed once sent, so a CDR pushed to the sandbox before this split shipped has no way to pick up the marker that would hide it, and those older CDRs are still served on GET /cdrs. Anything you push from now on is split correctly.

​
The exchange behind every check

Every check that was exercised carries the HTTP request that was sent and the response that came back, the rule it was testing, and whether that rule is one the specification requires or only recommends (severity, which is shall unless it says should).
A check can also carry the clause behind it: the specification file, the section, and the anchor that survives the document being re-vendored. Where someone has read that clause and written it down, you get its words too, and whether it binds as SHALL or only SHOULD.Not every check carries one yet. A citation with no strength is an address we recorded but have not read back to you; one with a quote has been checked against the specification text on every build. Charging journey assertions carry a reference on every assertion.
This is deliberate, and it is the feature that ends arguments. When you forward a report to a partner, they do not have to trust our verdict. They can read the exchange and see for themselves. A conformance report that says “failed” without showing you what was sent is asking for faith, and no engineer under deadline pressure is going to give it.

​
What to do next

Share a report

Send the evidence to your partner.

Troubleshooting

Common failures and what they usually mean.