Part two of three. How sensor data gets from an Altruist to the chain in 5.0: the signed envelope, why a 219-byte Meshtastic frame reshaped the message format, the connectivity-cloud pipeline, and the single-binary edge gateway. What is already running, and what is not yet. EnsRationis [Robonomics Software Architect]
Part one was about the protocol: the runtime, the repositories, the architecture of 5.0. This part is about the road data takes to get there — from a sensor on someone’s balcony to a record on the chain that anyone can check.
It is the least visible layer of the system and the one that decides whether the rest is worth anything. A chain that faithfully stores whatever a middleman hands it proves only that the middleman said so.
The sensors.social map currently runs on sensors-connectivity, a Python service that has been doing this job for years. An Altruist sends a JSON measurement over HTTP, signed with its Ed25519 key. The service checks the signature and the device’s subscription, republishes the reading to an IPFS pubsub topic for the live map, collects readings into batches, pins each batch to IPFS, and writes the batch hash to the parachain as a datalog.
It works, and it is still being maintained — version 1.7.2 shipped last week with support for a radiation sensor. But look at what ends up on the chain: a pointer to a file, written by the service’s account. The chain can vouch that the service published that hash, and nothing about the readings inside the file. To trust the data, you trust the operator of that service and the pinning gateways that keep the file alive.
5.0 changes the unit of trust. The thing that travels, and the thing that ends up on the chain, is what the sensor itself signed.
The new format lives in its own repository, connectivity-protocol, started in August, published on buf.build and licensed Apache-2.0. One schema is shared by the Altruist firmware, the connectivity services and the sensors.social front end.
The outer layer is four fields:
message SignedEnvelope {
bytes sensor_id = 1; // the sensor's Ed25519 public key, 32 bytes
bytes nonce = 2; // random, unique per message
bytes message = 3; // the telemetry, as raw protobuf bytes
bytes signature = 4; // Ed25519 over sensor_id + nonce + message, 64 bytes
}
The rule that matters is the one about bytes. Protobuf serialization is not deterministic: decode a message and encode it again, and you may get different bytes that mean the same thing. Different bytes break the signature. So the device serializes once and signs; every layer after it verifies and forwards the envelope untouched; the backend may decode it for indexing but stores the original bytes. Nobody re-encodes. That is how a reading can pass through three services and a mesh radio and still be checkable against the key of the device that produced it.
Inside the envelope, a message carries a small header and the measurements:
message Meta {
uint64 node_id = 1; // the device's node in the Robonomics CPS tree
uint64 timestamp = 2; // UTC, milliseconds
}
node_id is where this layer meets part one. A sensor is no longer identified by an owner field typed into firmware; it is a node in the CPS tree, the same topology the 5.0 runtime is built around. Measurements are typed per board — Urban for outdoor units, Insight for indoor ones — and each can be sent in the clear or as ciphertext addressed to someone specific, so a city can receive precise location while the public map gets an approximate one. The protocol carries ciphertext as opaque bytes and deliberately does not fix the cipher.
The envelope was first designed for HTTP, where a few extra bytes cost nothing. Then we tried to send it over a LoRa mesh.
Altruists sit where there is often no Wi-Fi: rooftops, parks, construction sites. Meshtastic is an open firmware that turns cheap LoRa boards into a self-organising radio mesh, and it was the obvious candidate for carrying readings out of such places. Its constraint is hard: a LoRa packet is 255 bytes, full stop. The transport spec we wrote for it in September does the arithmetic:
Now subtract the envelope itself. A 32-byte key, a 64-byte signature, a 16-byte nonce and their field tags take about 120 bytes. That leaves under a hundred bytes for the timestamp, the node id and every measurement — if the reading is to fit in one packet and avoid fragmentation over a radio link that may drop any of them.
The first schema did not fit. Every measurement was a double: eight bytes plus a tag, for a temperature sensor whose real resolution is a hundredth of a degree. So in September the schema was reworked around the radio:
Scaled integers instead of doubles. Temperature in 0.01 °C as sint32, humidity in 0.01 %RH, pressure in 0.1 Pa, PM2.5 and PM10 in 0.1 µg/m³, CO₂ and noise in whole units. Protobuf varints make small integers small on the wire.
One timestamp per message, moved into Meta, instead of one per measurement.
The CPS node_id instead of an owner address — a number of a few bytes instead of a 32-byte key.
None of this was needed for HTTP. All of it makes the HTTP path cheaper too.
The Meshtastic transport itself is specified to be boring: encrypted unicast to a known gateway with acknowledgement on; refuse to send rather than silently fall back to weaker channel encryption; fragment only when a payload genuinely does not fit; leave retries to the Meshtastic firmware instead of running a second retry loop on top of it. The test bench is an Altruist Urban talking over UART to a Heltec V4 board at 868 MHz, and a Raspberry Pi 4 with a LoRa hat as the gateway; the wiring and setup are in the hardware repository.
On the receiving side, connectivity-cloud replaces the single Python process with a set of small TypeScript services around a Kafka log:
The endpoint rejects a message whose timestamp is more than five minutes off, whose nonce it has already seen, or whose signature does not verify, and it answers the sensor only once the event is safely in Kafka. Everything after that reads from the log independently, so a slow chain does not slow the live map.
Two decisions this autumn shaped the result.
The data goes on the chain, not a pointer to it. Until September the pipeline followed the old pattern: batch, pin to IPFS, anchor the CID. In late September the IPFS step was removed. The batcher now compresses each batch of signed envelopes with zstd, splits it to fit 8 KB, and the anchor service writes those bytes into the CPS pallet with cps.setPayload. The chain holds the readings themselves, still wrapped in the sensors’ own signatures, and nobody has to keep a file pinned for the record to stay readable.
Who may send is answered by the CPS tree. At the end of October the static allowlist gave way to a check against the chain: a sensor is accepted if its CPS node sits under the sensors.social root node. Adding a sensor to the network becomes an on-chain operation in the same tree that 5.0 uses for every other device, and the connectivity service only has to read it.
In early November the services were hardened for Google Cloud Run: authenticated TLS to Redis and Kafka, atomic nonce storage shared between instances, and a refusal to start in production with any of that switched off.
Not every deployment wants Kafka and a cloud account. For a single board computer on a rooftop, the robins repository has edge: one binary that accepts the same signed envelopes over HTTP, runs the same checks — signature, CPS authorization, replay protection — and republishes accepted messages to the same GossipSub topic, with health and Prometheus metrics built in. It also has a Meshtastic ingress, so the gateway on the receiving end of the radio link and the gateway on the network can be the same small device. The same binary doubles as a toolbox: generate a sensor key, compose a message from the command line, sign it, inspect it.
Part three: the devices and the people — Altruist Urban and Insight, firmware channels, the tester, sensors.social and cyberpunks.shop.
Protocol: github.com/airalab/connectivity-protocol · Pipeline: github.com/airalab/connectivity-cloud
— EnsRationis, Robonomics Software Architect · @EnsRationis