The quality of the generated code
archec gen writes code that should build clean. A reviewer should be able to
trace it back to the model, and the model's own test plans should test it. This
page shows how to check each of these, from the model.
A first check#
A part that counts ticks, a requirement, and a test plan that verifies it:
message TICK "A tick" { n: uint32 }
component Counter "Counts the ticks" {
in tick: data receives TICK {
on-receive ```arche
count += 1
```
}
var count: uint32 = 0
satisfies REQ-COUNT
}
system Demo {
part counter: Counter
}
requirement REQ-COUNT "Count each tick" {
text "The counter shall add one for each TICK it receives."
}
test-plan TP-COUNT "Counting" {
case TC-COUNT-01 "Two ticks count two" {
verifies REQ-COUNT
at 0ms do send(counter.tick, TICK(1))
at 1ms do send(counter.tick, TICK(2))
within 5ms expect counter.count == 2
}
}Build and lint the C:
archec gen main.arche --target c -o generated --build --lint
── cc -std=c99 -Wall -Wextra -Werror -pedantic -O2
note: cppcheck: the MISRA C:2012 check (its misra addon) is not run — install cppcheck 2.x
note: clang-tidy: not run — install it (LLVM) for its checks
✓ built clean, 0 finding(s) from the linters
Run the test plan on the generated C:
archec gen main.arche --target c-tests -o generated --run
── make -C generated/c-tests test
PASS TP-COUNT/TC-COUNT-01 Two ticks count two
arche-tests: 1 passed, 0 failed, 0 skipped
results: generated/c-tests/results.json
Then write the traceability matrix:
archec gen main.arche --target trace-matrix -o generated
generated/trace-matrix/matrix.md has a row per requirement:
| Requirement | Model element | Generated code | Test case | Last result |
|---|---|---|---|---|
| REQ-COUNT Count each tick | component Counter (main.arche:3) | c/counter.c:9 counter_init |
TP-COUNT/TC-COUNT-01 | pass (C) |
The sections below say more about each check:
- Build and lint:
archec gen --build --lint. Each finding is shown at the model element it comes from. - C unit tests from the test plans:
archec gen --target c-tests,make test, and the coverage of states and transitions. - Defensive code:
--defensive. Ranges, stale inputs and end-to-end checks (CRC, counter) on receive. - The traceability matrix:
archec gen --target trace-matrix. Requirement → element → generated function → test case → result.
archec runs other tools: cppcheck, clang-tidy, cargo clippy, gcov. When one is
not installed, the command says so, and what it would check is left out. It is
never pretended. test/codegen-quality.test.js runs the tools that are
installed, for real.
Build and lint#
archec gen main.arche --target c -o generated --build # build it
archec gen main.arche --target c -o generated --build --lint # and lint it
archec gen main.arche --target embassy -o generated --lint --sarif gen.sarif
--build builds what was just written. --lint also runs the linters:
| Target | Build | --lint adds |
|---|---|---|
c |
each binary's Makefile (make -B). With no deployment, every module on its own (-fsyntax-only). The flags are -std=c99 -Wall -Wextra -Werror -pedantic. |
cppcheck with its MISRA C:2012 addon (--addon=misra). clang-tidy, with bugprone, cert, clang-analyzer and performance; when the output folder has a .clang-tidy, yours is used instead. |
embassy |
cargo build in each crate |
cargo clippy instead of the build |
The generated C is meant to build clean. So with -Werror, a warning is a
build error. The command exits 1 when the build fails, and 0 otherwise. A
linter's finding alone does not fail it: read the findings, or gate on the
SARIF in CI.
Each finding is shown at the model element its generated line comes from. Then comes where it is in the generated code:
main.arche:57 error [cc-unused-variable] unused variable 'x' — c/controller_ports.c:6 in controller_send_cmd (port cmd of Controller)
components.arche:172 warning [clippy-blocks_in_conditions] … — embassy/brake_fw/src/app.rs:26 in main (fn main of brakes)
components.arche:118 warning [misra-10.4] … — c/adcu.c:61 in adcu_fire_with (transition Manual -> Ready on self_test_ok of Adcu)
The rule in brackets is:
cc-<warning>for the compiler;misra-<rule>for a MISRA C:2012 rule;cppcheck-<id>for cppcheck's other checks;- clang-tidy's check name;
clippy-<lint>for clippy.
The element is found from the names the generators give and the comments they write:
| Generated | Model element |
|---|---|
<part>_init, any other line of c/<part>.c |
the component |
<part>_<fn> |
its function |
<part>_on_receive_<port>, _on_deadline_, _command_, _output_, _send_ |
the port |
<part>_fire_with, _timeout, below /* transition A -> B on M */ |
that transition |
<part>_enter, _exit, below case <PART>_<STATE>: |
that state |
<msg>_encode, _decode in c/arche_types.c |
the message |
thread_<id>, a send function in c/<binary>/wiring.c |
the thread, the port |
fn x in embassy/<binary>/src/app.rs, after /// <part path> |
the function (or port) of that part |
A line that maps to nothing, such as a runtime file or main.c, is shown at the
generated file and line only.
--sarif <file> writes the findings as SARIF 2.1.0, as archec check --sarif
does:
- each finding is at the model element's file and line, so code scanning shows
it on the
.archefile; - its message gives the place in the generated code;
- there is a rule per code.
C unit tests from the test plans#
archec gen main.arche --target c-tests -o generated # write them
make -C generated/c-tests test # build and run them
archec gen main.arche --target c-tests -o generated --run # both
archec gen main.arche --target c-tests -o generated --coverage # and gcov's coverage
--target c-tests writes, under c-tests/:
- the C of each component, as
--target cwrites it; arche_tests.c;- a
Makefile.
arche_tests.c is the harness: the program that archec difftest drives
the generated C with. It runs the system's parts on a virtual clock. Each
message goes along its links, with their latency. Each part does one thing at a
time, in the order the specification gives. The harness
also has a function per test case.
There is no test framework: the harness is the test runner. make test exits 1
when a case fails.
make test CASES="TC-1 TP-2"runs some cases, by id or by plan.ARCHE_TRACE=1prints what the parts do, asarchec difftesttraces it.
A case runs as Arche's test runner runs it. It starts fresh. Each step runs at
its at: its do, its wait, then its expectations. An expectation is one of:
- an expression, checked once the step has settled;
within 40ms:, true no later than 40 ms after the step began;at d:;eventually:, within the case'stimeout(100 ms);never:andalways:, watched after each event until the case ends. A case ends 10 ms after its last step.
What a case says in Python is translated where C can say it:
| In the case | In C |
|---|---|
send(ctrl.engage, ENGAGE(True)) — a port that receives |
the part handles it now, as a message from outside |
send(hmi.bus, ENGAGE_REQ(True)) — a port that sends |
sent along the port's links |
MSG(points=1000, nearest=30.0), MSG(1000, 30.0) |
the message's fields by name or in order |
adcu.speed = 20.0, +=, -= |
the part's variable |
brakes.state == "Braking" |
the part is in Braking (or in a state inside it) |
sent("BRAKE_CMD") >= 1 |
that message's sends since the case began |
numbers, constants, True/False, and/or/not, comparisons, arithmetic |
the same |
A case with anything else is generated as SKIP, with what C could not say.
Examples are print, received(…), a fault, a keyframe, or a function test.
Nothing is guessed. The output has the form archec test reads from Arche:
PASS TP-BRK/TC-BRK-01 Brakes when fast
FAIL TP-BRK/TC-BRK-03 A wrong expectation
ctrl.state == "Braking" — false at 1.000 ms (main.arche:111)
SKIP TP-BRK/TC-BRK-04 Python the C tests cannot say — do print("hello") (main.arche:114)
arche-tests: 2 passed, 1 failed, 1 skipped
--run builds and runs them, and prints that. It also writes
c-tests/results.json, as archec test writes .archec/test-results.json. It
holds the verdicts by PLAN/CASE, and what they say of the requirements the
cases verify. The traceability matrix reads it.
A body that waits (await wait(…)) runs in the harness, but its wait takes no
time. A body that loops forever on its waits (while True: await wait(WATCHDOG)) runs past 100000 events. Its case is then an ERROR, and the
next case still runs. The autonomous car's watchdog loop does this today. In C
it needs a thread, and the harness does not give a body a thread.
Coverage#
With gcov (--coverage, or make coverage), each module is built with
--coverage. Then the lines that ran are read back as parts of the model:
| gcov counts | covered in the model |
|---|---|
the line after /* transition A -> B on M */ |
that transition was taken |
case <PART>_<STATE>: in <part>_enter ran, the initial state at init, or a taken transition's target |
that state was entered |
a line of <part>_<fn> |
that function ran |
The result is reported as archec test --coverage reports Arche's coverage. It
is also written to c-tests/coverage.json:
Coverage (the generated C (gcov)):
states 5/5 100 %
transitions 3/4 75 %
not covered: brakes: Braking -> Released on after 100ms
functions 0/1 0 %
not covered: sensor.scan
A part with one instance is named by its path (brakes). A module that several
instances share is named by its component (Brakes).
Defensive code#
archec gen main.arche --target c --defensive -o generated
--defensive adds checks to the C. They check what the model says a message may
be, where the message comes in:
| The model says | The C does on receive |
|---|---|
a field's range (or its type's), or a value message's |
<msg>_in_range() in arche_types.c. A message with a value out of range is rejected, not clamped. It is counted in arche_errors.range. |
a port's stale-after, else the message's period |
An input that has not come for that long is flagged stale. For a period, that is the period × 1.5, the tolerance archec replay gives it. <binary>_input_stale(<BINARY>_IN_k) says whether it is stale now. arche_errors.stale counts each time it goes stale. |
a wire format with a checksum |
A frame whose checksum is wrong does not decode. It is counted in arche_errors.crc. |
a wire format with a counter |
In the style of AUTOSAR E2E. A counter that did not move means a duplicate: it is rejected (arche_errors.repeat). A counter that jumped means messages were lost: the message is kept, and the loss is counted (arche_errors.jump) and logged. |
Why reject and not clamp? A value out of its range is a fault upstream: a sensor, a bus, a sender's bug. A clamped value hides that fault behind a plausible value. A rejected message never reaches the part, which keeps acting on the last good one. The stale flag says when that has gone on too long.
With a deployment, each binary's wiring.c:
- checks every message it dispatches to a part (
arche_accept_<link>_<msg>).wiring.hlists its inputs; - checks for stale inputs at each period of a periodic thread, and at each message;
- logs each rejection once, until the input is accepted again. So a stuck sensor does not flood the log.
Without a deployment, <msg>_in_range() and arche_errors are there for your
receive code.
A thread that waits for messages and has no period sees an input go stale only
at its next message, or when asked (<binary>_input_stale).
The checks are opt-in, not on by default, for two reasons:
- They change what the parts see: a rejected message never reaches them. So
archec difftestwould no longer compare like with like against the simulator, which checks none of this. - Each checked input costs a few bytes of RAM and a few compares. A small target should choose to pay that.
The traceability matrix#
archec test main.arche # Arche's simulator: arche-out/.archec/test-results.json
archec gen main.arche --target c-tests -o generated --run # the C: generated/c-tests/results.json
archec gen main.arche --target trace-matrix -o generated # generated/trace-matrix/matrix.md, .csv
For each requirement, the matrix gives:
- the model elements that satisfy it:
satisfies(and the old forms, read until 0.13:satisfied-by,trace satisfies); - the C generated for each:
--target c's file, line and function, read back as in build and lint; - the test cases that verify it: a plan's or a case's
verifies, and the old formsverified-by,trace verifies,verify { tests … }; - what the last run of each case said.
| Requirement | Model element | Generated code | Test case | Last result |
|---|---|---|---|---|
| REQ-BRAKE Brake when fast | component Controller (main.arche:54) | c/controller.c:9 controller_init |
TP-BRK/TC-BRK-01 TP-BRK/TC-BRK-03 |
pass (C), pass (sim) fail (C) |
| REQ-SCAN Report the speed | fn scan (main.arche:46) | c/sensor.c:11 sensor_scan |
— | — |
How each element is shown in the "Generated code" column:
- a component, by where its module starts (its
init); - a function, a state, a transition or a port traced to a requirement, by its own generated function;
- a part, by its own and its component's.
The results come from --results <file>: each a results JSON, and the option
can be repeated. Without it, the results found are used:
arche-out/.archec/test-results.json (sim) and <out>/c-tests/results.json
(C). A case in neither is not run.
Below the table come the gaps. A gap is a requirement:
- that no element satisfies;
- whose elements have no generated C;
- that no case verifies;
- with a case that failed;
- whose cases have not run.
matrix.csv has a row per requirement, element and case
(requirement,title,element,element_at,generated,test_case,result). It is for a
spreadsheet or a certification tool.
Limits#
- A linter or gcov that is not installed is not run. What it would check is left out, and the command says so.
- A test case with anything the C cannot say (
print,received(…), a fault, a keyframe, a function test) is aSKIP, with the reason. - In the C tests, a wait takes no time. A body that loops forever on its waits
is an
ERRORafter 100000 events: in C it needs a thread, which the harness does not give a body. - The defensive checks are in C only: the Embassy crates do not have them yet.