Apache Avro: the contract between modules

The commerce team adds a discount percentage to the order. It is a small change, agreed in a meeting, and the module that creates orders starts sending it the same afternoon. The finance module, which has read orders for two years, does not know the field and stops accepting the message. Nobody intended a contract change. There was a contract, it was just never written anywhere except in two pieces of code that used to agree.

In a system built from modules that exchange events, every event is an agreement. Apache Avro is one of the formats that writes that agreement into the data itself, and the reason it sits in the design of Hurukan’s Data layer, which is in build, is less about bytes than about what happens when the agreement has to change.

The schema travels with the data

An Avro record is a compact binary row. On its own it cannot be read: the field names, types and order are not in the bytes. They are in the schema, written in JSON, and the specification defines a small set of types to write it with: null, boolean, int, long, float, double, bytes, string, and the complex record, enum, array, map, union and fixed. A field can carry a default value, and that default is what later makes change survivable. The current release is 1.12.2 (project blog, checked 21 October 2026).

The schema reaches the reader in one of two ways. In a file, the Object Container File format puts the schema in the header, followed by blocks of records separated by a sync marker, so a file found years later still explains itself. On a message bus, where sending the schema with every event would be waste, the specification’s single-object encoding puts a two-byte marker and an eight-byte fingerprint of the schema in front of each record; a registry keeps the schema under that fingerprint. The widely used Confluent registry does the same with its own framing, a magic byte and a four-byte schema identifier before the Avro bytes (Confluent documentation).

Either way, the reader never guesses. It has the writer’s schema and its own, and the rest is rules.

Resolution: how a reader meets a writer it has never seen

The specification sets out what happens when the writer’s schema and the reader’s schema differ. Fields are matched by name. A field the writer sent and the reader does not have is ignored. A field the reader expects and the writer did not send is filled from the reader’s default, and if there is no default, the read fails with a clear reason. Some types may be promoted on the way: an int may be read as a long, a long as a float or double, a float as a double, and string and bytes may be read as each other. A union is matched to the first compatible branch on the reader’s side.

Written out, those rules are the whole discipline of the discount field. Add it with a default and the finance module keeps reading, ignoring the field until it is taught to use it. Add it without a default and remove an old field in the same change, and the oldest reader in the company stops. The rules are not advice; they decide which version of the morning you get.

A schema change as a proposal

In Hurukan the design is that an event schema is an object of the ontology like any other: it has an owner, versions and a place in the record. That turns the discount field from an afternoon’s code change into a short, recorded path.

An agent proposes. The commerce module needs a new field; an agent drafts the schema change, with the default value, the type and the reason, and names the modules that read this event.

The registry checks. The proposed version is tested against the readers’ schemas under the resolution rules above. A change that would stop an existing reader is rejected before anyone sees it, with the reader named.

The owner approves. A compatible change is still a change to an agreement between modules. The owner of the event, a person, approves it, and only then do producers start writing the new version.

The record keeps it. The approval, the version and the time are written into the record, so a year later the question “when did discounts appear on orders, and who allowed it” has an answer. That is the same path every change in the system takes: an agent proposes, a person decides, the system records. The wider idea is in the live state and the record.

What Avro is not for

Avro carries events well and reads whole records well. It is a poor layout for the question superintelligence most often asks, a few fields across a million rows, because every field is in the way of every other. That is why, in the design, events in Avro are batched into columns once they have been accepted. Which shape belongs where is set out in Parquet and Avro: the event and the record.

Avro also appears in a place few people look: the table format Apache Iceberg writes its manifests, the lists of data files that make up a table, as Avro files (Iceberg specification). The row format that carries events between modules also carries the bookkeeping of the record.

What stays with people

An agent can draft a schema, run the compatibility check and name the readers. It cannot decide that a field the finance module relied on may go, that a default of zero is a true statement about old orders, or that a reader may be allowed to fail. Those are decisions about the meaning of the business, and in Hurukan they are taken by the owner of the event and written down.

A test you can run today

Pick one message that moves between two of your systems. Find where its fields are defined. If the answer is “in the sending code” and “in the receiving code”, you have two definitions and no contract. Then ask who would approve adding a field to it, by name. If there is no name, the next field will be added by whoever needs it first, and discovered by whoever it breaks. The Data layer page describes where that contract is meant to live in Hurukan; the overview shows the modules around it.