Integration targets
A firmware lives among other tools: protocol tools, build systems, flashers and
test benches. archec gen --target <t> can write what those tools read, from
the same model as the code. Each target writes one kind of file, and its notes
say what the model did not give it.
A first target#
examples/can-bench has three boards on one CAN bus. The bench target writes
the configuration of the test rack for its deployment:
cd examples/can-bench
archec gen main.arche --target bench -o gen
note: platform BenchHw: no power link — the rack's supplies are not in the model (link a supply's `out v: power` to each board's `in pwr`)
gen/bench/OnBench.yaml
gen/bench/setup_can_bus.sh
gen/bench/bench_can_bus.py
The start of gen/bench/OnBench.yaml:
deployment: OnBench
buses:
- id: can_bus
name: "CAN bus"
type: can
bitrate: 500000
interface: can0 # the rack's adapter on it
nodes:
- part: pi
port: can0
binaries: [gw]
…
frames:
- id: 0x100
message: FAN_CMD
length: 1
sender: gw
receiver: fan_fw
- id: 0x200
message: ROOM_TEMP
length: 2
sender: gw
receiver: ui_fw
period_ms: 500
…
The note says what the model lacks: here, how the boards are powered. The other targets work the same way.
How the targets work#
Each target is one generator, like the others that archec gen --list shows.
It is a pure function of the model and the deployment: the same model always
gives the same files. The deployment is the one --deployment names, or else
the active one.
Each target writes only what the model says. Some protocols need a value the
language has no word for, such as a J1939 SPN, a MAVLink id or a CANopen vendor
id. The target reads that value from an attr on the declaration it belongs
to. When the attr is missing, the notes say so. A value is never made up
silently.
| Target | Writes | From |
|---|---|---|
canopen |
canopen/<bus>/<binary>.eds: a CiA 306 EDS per binary on a CAN bus |
the routes over the bus, the messages' layout |
j1939 |
j1939/<bus>.json, .md: the parameter groups and SPNs |
the routes with a 29-bit id |
mavlink |
mavlink/<project>.xml: a MAVLink 2 dialect |
the message catalog |
someip |
someip/<deployment>.json: SOME/IP services, methods, events, instances |
interfaces and Ethernet ports |
openapi |
openapi/<Component>.json: OpenAPI 3.1 |
the ports' HTTP handlers |
asyncapi |
asyncapi/<system>.json: AsyncAPI 3.0 |
the links that carry messages, their topics |
cmake |
cmake/: a CMake project per C binary, a toolchain per microcontroller |
the binaries, their processors' cores |
west |
west/<binary>/: a Zephyr application with its west manifest |
the binaries on Zephyr boards |
platformio |
platformio/<binary>/platformio.ini |
the binaries on boards PlatformIO knows |
ci |
ci/.github/workflows/archec.yml, ci/.gitlab-ci.yml |
the targets the deployment uses |
flash |
flash/<binary>/flash.sh, monitor.sh |
each microcontroller binary's chip and board |
bench |
bench/<deployment>.yaml, setup_<bus>.sh, bench_<bus>.py |
the platform: buses, boards, power and IO links |
There is one target per format, rather than one target with sub-kinds. So each
format has its own --check, its own notes and its own row in
archec gen --list. A CI job regenerates only the formats it uses.
Each section below ends with how the output is checked in archec's own tests
(test/integration.test.js).
Protocol descriptions#
CANopen (--target canopen)#
It writes an electronic data sheet (EDS, CiA 306, version 4.0) for each binary
that sends or receives on a CAN bus of the deployment:
canopen/<bus>/<binary>.eds. The EDS has:
- the mandatory objects: device type (0x1000), error register (0x1001) and identity (0x1018);
- a transmit PDO for each frame the binary sends on the bus (0x1800 + n, with its mapping at 0x1A00 + n);
- a receive PDO for each frame it takes (0x1400 + n, and 0x1600 + n);
- an object for each message, in the manufacturer area (from 0x2000). The index
is the same in every node's EDS. Each signal is a sub-index, named as in the
model (
TEMPERATURE.value), with its CANopen type.
For each PDO:
- The COB-ID is the route's
id, with bit 29 set for a 29-bit id. - A message with a
periodis sent on the event timer: transmission type 0xFE, with the period in ms. - The other messages are sent when the application sends them.
The signals follow the layout that the DBC and the generated C and Rust use, so the PDO carries the same bytes. A message with a wire format maps its fields at their bits, with dummy entries over the gaps.
What a classic PDO cannot carry is left out, and the notes say so:
- a frame over 8 bytes (CANopen FD, CiA 1301, is not written);
- a big-endian field;
- a field whose bit count no CANopen type has, such as a 4-bit field (there is no UNSIGNED4).
The model has no CANopen identity. Give it on the binary, or on its processor:
binary pump_fw "pump-fw" on pumpb {
attr canopen_node_id = 5 // [DeviceComissioning] NodeID
attr vendor_id = 0x1234 // 0x1018 sub 1, VendorNumber
attr product_code = 42 // 0x1018 sub 2
attr revision = 1 // 0x1018 sub 3
…
}
Without them the EDS says 0, and the notes name what is missing. For example,
on examples/can-bench:
note: can_bus/gw: no vendor_id, product_code, canopen_node_id in the model — the EDS says 0; write `attr vendor_id = …` on binary gw
A bit rate that is not one of CANopen's is noted too. CANopen's rates are 10, 20, 50, 125, 250, 500, 800 and 1000 kbit/s.
Checked: each file is read with the canopen Python package, both its object
dictionary and the PDO maps it builds from it.
J1939 (--target j1939)#
For each CAN bus that carries frames with a 29-bit id, it writes
j1939/<bus>.json, and the same as Markdown tables in j1939/<bus>.md.
In J1939-21, the identifier holds everything about a parameter group. So the
target reads it from the route's id:
- the priority;
- the PGN, with the data page and the extended data page;
- the PDU format: PDU 1 with its destination address, or PDU 2 with its group extension;
- the source address.
Then it adds the group's length, its rate (the message's period), the binary
that sends it and the one that receives it.
Each signal is an SPN, with the columns of J1939-71:
- its start position, as
byte.bit, both counted from 1; - its length in bits;
- its resolution and offset (a wire format's
scaleandoffset); - its unit and range.
The SPN number is not in the identifier, so you write it on the field:
message EEC1 "Electronic engine controller 1" {
attr pgn = 61444 // checked against the route's id
period 10ms
size 8
torque_mode: uint8 bits 4 range 0..15 { attr spn = 899 }
speed: float32 unit rpm range 0..8031 at 24 bits 16 scale 0.125 { attr spn = 190 }
}
…
route pump.engine -> gateway.engine over can_bus { id 0x0CF00405 }
A signal without attr spn is listed with "spn": null, and the notes name
it. The notes also name:
- a frame over 8 bytes (it goes by the transport protocol, TP.BAM or TP.CMDT);
- a big-endian field;
- a binary that sends from more than one source address, or from another
address than its
attr j1939_address; - a PGN that two binaries send;
- the frames with 11-bit ids on the bus, which are left out.
Checked against cantools.j1939: it reads the same priority, PGN and source
address from the same id.
MAVLink (--target mavlink)#
It writes the message catalog as a MAVLink message-definition file, the format
mavgen reads: mavlink/<project>.xml. The file is a dialect of its own. It
includes nothing, so the model's HEARTBEAT does not clash with the one in
common.xml.
- Each message with a fixed layout is a
<message>. Each signal of that layout is a<field>:- a struct is flattened (
header_seq); - an array of scalars is one array field (
float[4]); - an enum is a
uint8_twithenum="MODE".
- a struct is flattened (
- Each enum the messages use is an
<enum>, with entries such asMODE_RUN. - A field's unit is written when MAVLink's schema knows it:
m/s,degC,rad,%. The model's°,°/sandm/s²becomedeg,deg/sandm/s/s. Another unit goes into the field's description. - The message's description holds its name or description, and its
period.
The MAVLink id is not in the model. Give it with attr mavlink_id = 30 on the
message. Messages without one are numbered from 65536, in the catalog's order.
That is above every range the MAVLink registry gives to dialects, so they clash
with none. The notes list them: give them their real ids before two systems
rely on them.
A payload over 255 bytes, or with more than 64 fields, is left out, and the notes say so.
Checked with mavgen --strict-units, which checks the schema and the units.
Its C headers are compiled and a message is packed. Its Python round-trips a
message.
SOME/IP (--target someip)#
It writes a service description as JSON: someip/<deployment>.json. It holds
what FIBEX and ARXML hold: the services, their methods, events and event
groups, and where each instance runs. It also holds the data types the payloads
use.
- A service per interface that a port
provides:- Each operation is a method. It is request/response when it returns something, and fire-and-forget when it does not.
- The interface's messages that the provider sends are events, in one
event group. An event is cyclic at the message's
period, or else sent on change. - The messages the provider only receives are fire-and-forget methods.
- A service per
outport on an Ethernet link that sends messages and takes part in no interface. Its messages are events. - Instances, for each service: the part that provides it, the binary and processor it runs on, and the parts at the other end of its links.
dataTypes: the structs, enums and unions the payloads use. Scalars are named as SOME/IP names them (sint16,float32,boolean).
The ids are not in the model. Give them with:
attr |
on |
|---|---|
someip_service_id |
the interface (or the port) |
someip_instance_id |
the providing port |
someip_method_id |
an operation |
someip_event_id |
a message |
someip_major_version, someip_minor_version |
the interface |
Without them, services are numbered from 0x1000, methods from 0x0001 and events from 0x8001, in the model's order. The notes list each one. An interface provided on a port that is not on IP (a CAN port) is noted.
SOME/IP serializes big-endian. The codecs archec generates for CAN are little-endian, so they are not this wire format.
OpenAPI (--target openapi)#
It writes an OpenAPI 3.1 document for each component definition whose ports
have HTTP handlers, such as handler GET "/api/v1/readings/{node}" -> reading.
The file is openapi/<Component>.json.
- Each handler is an operation:
- its
operationIdis the function's name; - its summary is the function's name in quotes;
- its description is the handler's, or else the function's;
- it is tagged with the port.
- its
{node}in the path is a path parameter, typed by the function's parameter of that name.- The other parameters are the query (GET, DELETE, HEAD), or a JSON body (POST,
PUT, PATCH). They are required unless
optional. - The function's return type is the 200 answer, as JSON. With no return type, the answer is 204.
serverslists each part of that definition withprop hostname = "…"(orattr url = "…").
The schemas are JSON Schema, shared with AsyncAPI:
- A struct is an object with its fields required (an
optionalone is not), and no other fields. - An enum is its literal names.
x-valuesgives their numbers. - A fixed array has its size as both min and max items.
- An integer has its type's bounds, or its
range. A float has itsrange. - A unit is
x-unit. - An alias type (
type Speed = float32 unit m/s range 0..70) is its scalar, with its unit and range. - A union is a
oneOfof{case, value}.
Checked with openapi-spec-validator.
AsyncAPI (--target asyncapi)#
It writes an AsyncAPI 3.0 document for each system, or for the deployment's
system when there is one: asyncapi/<system>.json.
- A channel per port that sends over a link. Its address is the link's or
the port's
topic. Without one, it is<part>/<port>, as the ROS 2 nodes name it. Its messages are what the links carry. - A send operation for the part that publishes, and a receive operation for each part at the other end.
- A server per bus a route takes, when there is a deployment. Its protocol
comes from the port type (
can,serial,udp,mqtt…), with its bit rate in the description. The channel is on that server. - The messages are in
components.messages, and their payloads in the shared schemas.
Checked against the official AsyncAPI 3.0.0 JSON schema (@asyncapi/specs with
ajv), when ARCHE_NODE_TOOLS names a folder where they are installed.
Build systems#
The build targets wrap the C that --target c writes, where it is. They refer
to ../../c from their own folder, so write both into the same output folder:
archec gen main.arche --target c -o gen
archec gen main.arche --target cmake -o gen
cmake -S gen/cmake -B build && cmake --build build # every C binary, on this host
A binary in Rust (--target embassy) or Go (--target go) has its own build,
with cargo or go. These targets note it and leave it out.
CMake (--target cmake)#
cmake/<binary>/CMakeLists.txtbuilds two things:- the binary's model, as a static library
<binary>_model: the types, its parts, its wiring and its drivers; - the executable, with its
main.cand its runtime. The runtime is POSIX on a host, and FreeRTOS when cross-compiled.ARCHE_RTchooses.
- the binary's model, as a static library
cmake/<binary>/toolchain.cmake, for a binary on a microcontroller:- GCC, by its prefix (
arm-none-eabi-,riscv32-esp-elf-,xtensa-esp32-elf-); - the core's flags, from its
core-type. For exampleCortex-M4Fgives-mcpu=cortex-m4 -mthumb -mfpu=fpv4-sp-d16 -mfloat-abi=hard.
- GCC, by its prefix (
cmake/<binary>/port.cmakeis written once, and then it is yours. It names:- the FreeRTOS kernel (
FREERTOS_KERNEL_PATH), and its port for the core (GCC_ARM_CM4F…); - the chip's startup code and linker script.
Without a kernel, only the model library is built, with a warning.
- the FreeRTOS kernel (
cmake/CMakeLists.txtbuilds every C binary for this host. The microcontrollers' binaries are included too: their logic runs on the POSIX runtime, as their Makefile builds it. This is what CI builds and runs.
Some cores have no upstream FreeRTOS port, such as Xtensa (the ESP32's). This is
noted. ESP-IDF brings its own port, which --target platformio uses.
Checked:
- the rig's three C binaries are configured, built and run with CMake on the host;
- the sensor node's model library is cross-compiled for its STM32G4 with the
toolchain file, when an
arm-none-eabi-gccwith newlib is installed.
Zephyr (--target west)#
For each C binary on a board Zephyr knows, it writes a Zephyr application in
west/<binary>/. The boards are the store's Nucleo, nRF52840 DK, Pico and ESP32
boards.
CMakeLists.txtadds the binary's C to Zephyr'sapp, on Zephyr's POSIX API (arche_rt_posix.c).prj.confis written once, and then it is yours. It turns onCONFIG_POSIX_APIand the drivers that the overlay's nodes need (CONFIG_CAN,CONFIG_SERIAL…).boards/<board>.overlayis what--target devicetreewrites for that board: the linked CAN, UART, I2C and SPI on their pins, and an alias per bus that the C opens.west.ymlis a manifest. It pins Zephyr (v4.1.0), and allows only the modules the board's HAL needs (cmsis,hal_stm32…).
west init -l gen/west/node_fw && west update && west build -b nucleo_g474re gen/west/node_fw
Checked: west's own Python package validates and loads the manifest.
PlatformIO (--target platformio)#
For each C binary on a board PlatformIO knows, it writes
platformio/<binary>/platformio.ini. The file gives:
- the platform and board:
ststm32/nucleo_g474re,espressif32/esp32dev,nordicnrf52/nrf52840_dk,raspberrypi/pico; - a framework with threads under the runtime:
- Zephyr on STM32 and nRF52, with
zephyr/CMakeLists.txt,prj.confandapp.overlay; - ESP-IDF's FreeRTOS on ESP32;
- the FreeRTOS of the Earle Philhower Arduino core on the RP2040;
- Zephyr on STM32 and nRF52, with
- the generated C as its sources:
src_dir = ../../c, and abuild_src_filterof the binary's files.
The notes say what is left to do by hand. For example, ESP-IDF starts
app_main, not main.
Checked: pio project config reads each file. The files are not built, because
that downloads each platform's toolchain.
CI (--target ci)#
It writes a pipeline for GitHub Actions (ci/.github/workflows/archec.yml), and
the same for GitLab CI (ci/.gitlab-ci.yml). Copy ci/ to the root of the
repository. The jobs:
model:archec check, thenarchec ciwith its JUnit report. The report is uploaded on GitHub, and is a test report on GitLab.generated:archec gen --checkfor each target the deployment uses, in a matrix. That iscandcmakefor C binaries,embassyfor Rust ones,go, anddbcfor its CAN buses. It shows that the generated code committed is what the model gives.build-c: the CMake project of--target cmake, built on the host. Each binary runs for a second.build-<binary>, for each Rust binary:cargo build --releasefor its chip's target (thumbv7em-none-eabihf…, or the Espressif toolchain for an Xtensa ESP32).
Three variables at the top set where things are. Edit them when the project is not at the root of the repository:
ARCHE_MAIN: the model's main file, from the repository root;ARCHE_GEN: the folder the generated code is committed in (generated);- the deployment.
Checked: both files are valid YAML, and valid against SchemaStore's schemas for
GitHub workflows and GitLab CI (check-jsonschema).
The lab#
Flash and monitor (--target flash)#
For each binary on a microcontroller, it writes two executable scripts in
flash/<binary>/.
flash.sh [image] flashes the binary. It uses the first of its chip's tools
that is installed, or the one TOOL= names:
| Chip | Tools, in order |
|---|---|
| STM32 | probe-rs (--chip STM32G474RETx), OpenOCD over the ST-LINK (target/stm32g4x.cfg) |
| nRF52 | probe-rs (nRF52840_xxAA), nrfjprog (the ELF made a HEX), OpenOCD over a J-Link |
| RP2040 | picotool (load -x), probe-rs, OpenOCD over a CMSIS-DAP probe |
| ESP32 | espflash, esptool (the ELF made an image, written at 0x10000) |
The image is $1. Without it, the script takes the image where the build puts
it:
- for Rust, cargo's
embassy/<binary>/target/<triple>/release/<crate>; - for C,
build-<binary>/<binary>.elf: the cross build of--target cmake, from the folder archec gen wrote into.
monitor.sh reads the binary's trace for ARCHE_DURATION seconds (10 by
default) into ARCHE_TRACE (<binary>.trace). Then it runs archec timing
and archec live --replay on the trace, with the deployment.
- A C binary's trace comes from the board's UART: its virtual COM port when it
has one.
ARCHE_PORTnames it (/dev/ttyACM0by default). It is read with pyserial, at the bitrate of the link on it (or else 115200). - A Rust binary's trace is defmt over RTT, through
probe-rs attach. - The model is
ARCHE_MAIN: the main file's name, from where the script runs.
The trace is what archec gen --trace makes the firmware write:
- the C, built with
-DARCHE_TRACE_LIVE, with itsc/arche_trace_port.cwriting to that UART; - the Rust, built with
--features trace.
archec timing skips whatever comes before arche-trace on a line, so a log
prefix does no harm.
A binary on Linux is not flashed, and this is noted: copy it there.
Checked:
- each script parses (
sh -n); flash.sh, run against a stand-in probe-rs, gets the chip and the image;- OpenOCD reads the interface, transport and target of each OpenOCD line;
monitor.shreads arche-trace lines written to a pseudo-terminal, andarchec timingandarchec livemeasure the thread from them.
It has not been run on a board here.
The bench (--target bench)#
It writes the configuration of the deployment's hardware test rack, from its
platform. A first target shows it on examples/can-bench.
bench/<deployment>.yaml has:
buses: each bus, with:- its type (
can,can-fd,uart,ethernet); - its bitrate, and its
data_bitratefromattr data_bitrateon a CAN FD bus (the language has no word for the data phase); - the rack's interface on it:
can0,can1… in order, or/dev/ttyUSB0… for a serial bus; - the nodes on it, with their binaries;
- for a CAN bus, every frame it carries: id, message, length, sender, receiver and period.
- its type (
duts(devices under test): each binary on a microcontroller, with its board, its chip, its flash and monitor scripts (--target flash) and its console UART.hosts: the binaries on Linux.power: each supply (anout … : powerport) and what it feeds.io: the GPIO, PWM and analog links.
What the model cannot say is null, such as the channels of the supplies and
the IO on your rack. The notes say what else is missing, such as a bitrate or a
power link.
bench/setup_<bus>.sh, for each CAN bus, sets up its SocketCAN interface at the
bus's bitrate (restart-ms 100, and dbitrate … fd on for CAN FD). When the
interface given is named vcan…, it makes a virtual bus instead.
bench/bench_<bus>.py, for each CAN bus, uses python-can:
- It lists the frames that go by, named after the model.
- It fails (exit 1) when a periodic frame is missing, or off its period by more than half.
--as <binary>stands in for a binary that is not on the rack: it sends that binary's frames at their period.--send <MESSAGE>sends one message.--interface virtualruns it all in one process, without hardware.
sh gen/bench/setup_can_bus.sh can0
python3 gen/bench/bench_can_bus.py --channel can0 --duration 10 --as gw_app
Checked:
- the YAML loads, and the setup script parses;
- the python-can script on its virtual bus sees the sensor's and the pump's frames at their periods when it stands in for them;
- it fails with the pump's frame missing when it does not.
Limits#
- Each target writes only what the model says. A value the language has no word for (a J1939 SPN, a MAVLink id, a CANopen identity, SOME/IP ids) comes from an
attr. Without it the target writes 0,nullor a numbered stand-in, and the notes say so. - CANopen: no CANopen FD (CiA 1301). A frame over 8 bytes, a big-endian field, or a field whose bit count no CANopen type has, is left out of the PDOs.
- J1939: frames with 11-bit ids are left out. A frame over 8 bytes needs the transport protocol, which is noted but not written.
- MAVLink: a payload over 255 bytes or with more than 64 fields is left out.
- SOME/IP: the CAN codecs archec generates are little-endian, so they are not SOME/IP's wire format.
- The build targets build C only. A Rust or Go binary is left out, with a note. A core with no upstream FreeRTOS port (Xtensa) is noted.
- PlatformIO files are read by
pio project configin the tests, not built. ESP-IDF startsapp_main, notmain: that is left to do by hand. - The flash and monitor scripts have not been run on a board in archec's tests. A binary on Linux is not flashed.
- The bench file cannot say the channels of your rack's supplies and IO: they are
null.