--- title: "Parquet and Avro: the event between modules and the record superintelligence reads" url: "https://hurukan.com/blog/parquet-and-avro-the-event-between-modules-and-the-record-superintelligence-reads" description: "An event travels between modules as a row with a contract and rests in the record as columns. Why Avro carries the first and Parquet the second." --- # Parquet and Avro: the event between modules and the record superintelligence reads September 11, 2026·5 min read·Editorial team A logistics office seen from behind an operator at a standing desk: on the left monitor a single abstract blue-and-grey form, on the right a wide abstract table of thin columns, a thin blue LED strip along the desk edge, morning light from a tall window A goods receipt is confirmed at the dock. Within the same second the commerce module needs to know the backorder can ship, finance needs the invoice match to start, and the inventory module needs the stock level to move. Each of them wants the whole receipt: supplier, item, quantity, batch, who confirmed it. Three months later the finance agent asks a different question: how often did receipts from this supplier arrive short in the last two years. It wants two columns out of a million rows, and it wants them in seconds. Those are two different shapes of the same fact. The first is a row that travels. The second is a column that rests. A company operating system has to store both, and it is a mistake to store them the same way. In Hurukan’s [Data layer](https://hurukan.com/system/data), which is in build, the design is that an event between modules is carried by Apache Avro, and the record that superintelligence reads is laid out in Apache Parquet. This text explains why the split is there and what each side pays for. ## Two questions, two layouts A file format decides how one file arranges bytes: whether the records lie one after another or whether all the values of one field lie together, how they are compressed, and where the description of their types lives. **Row layout.** Each record is written whole: every field of this receipt, then every field of the next. Writing one record is cheap, reading one record is cheap, and reading only the quantity across a million receipts means reading all the other fields too, because they are in the way. **Column layout.** All the quantities lie together, all the supplier identifiers lie together. A question about two fields touches two narrow stripes of the file and skips the rest. Writing a single record is awkward, because a file has to be closed before anyone can read it, so columns are written in batches. Avro is the row layout with a schema that travels with the data. Parquet is the column layout with statistics that let a reader skip what it does not need. Neither knows about the other, and neither knows whether it belongs to a table; that is the work of a table format above them, described in [the live state and the record](https://hurukan.com/blog/the-live-state-and-the-record-two-memories-of-a-company-operating-system). ## The event: a row with a contract When the receipt leaves the inventory module it is a message. Every module that reads it has to agree on what the fields mean, and that agreement must survive the day somebody adds a field. [Avro](https://avro.apache.org/docs/1.12.0/specification/) carries that agreement with the message: the schema is JSON, the record is a compact binary row, and the specification sets out how a reader with one version of the schema resolves a record written with another. A field the reader does not know is skipped. A field the reader expects and the writer did not send takes the default from the reader’s schema, or the read fails and says so. The current release is 1.12.2 ([project blog](https://avro.apache.org/blog/), checked 21 October 2026). In a system where agents extend modules, that resolution rule is the difference between a change that is safe and one that stops a module at three in the morning. The way a schema change moves through proposal, compatibility check and approval is the subject of [Apache Avro: the contract between modules](https://hurukan.com/blog/apache-avro-the-contract-between-modules). The event does not stay an event. Once it is accepted by the modules that need it, the same receipt is written into the record, and from there it will be read in a very different way. ## The record: columns that superintelligence reads Superintelligence does not ask for one receipt. It asks for a pattern across all receipts from one supplier, all orders in one quarter, every non-conformance on one item revision. These are questions about a few fields across many rows, and that is what [Parquet](https://parquet.apache.org/docs/file-format/) is built for: a file is divided into row groups, each row group into one chunk per column, each chunk into pages, with the footer describing where everything is. A reader asks for two columns and receives two columns. Each row group carries minimum and maximum values, so whole stretches of the file are skipped when the question is about one supplier. The format specification is at version 2.14.0 ([project blog](https://parquet.apache.org/blog/), checked 21 October 2026). The point for a sovereign system is cost. The record lives on the infrastructure the company chooses, and every question superintelligence asks is paid for in bytes read and processor time on that infrastructure. A record laid out in columns, compressed by dictionary and sorted by the fields most questions filter on, is a record the company can afford to let its agents read as often as they need. How the layout inside one file turns into that cost is covered in [Apache Parquet: how the record is laid out](https://hurukan.com/blog/apache-parquet-how-the-record-is-laid-out-so-superintelligence-can-read-the-whole-company). ## Where the split is decided The fact is Shape Format in the Data layer Travelling between modules or in from a connector one whole record, now Avro, under a registered schema Resting in the record, read by agents and people many records, a few fields Parquet, inside a table format Arriving one at a time but read in bulk later both, in sequence Avro in, batched into Parquet The third row is where most of the engineering is. Events arrive one by one; Parquet wants batches. Something has to collect a minute or an hour of events, write them as one file and tell the table which file is new. Do that too often and the record becomes a heap of tiny files that is slow to read. Do it too rarely and the agent that needs today’s receipts is reading yesterday’s record. The interval is a decision about which questions need to be fresh, and it is a decision for a person, not a default. ## What an agent may change here, and what it may not Agents extend Hurukan, and the Data layer is no exception. An agent can propose a new field on the receipt event, draft the batching job that lands events into the record, or suggest that a table be sorted by supplier because the last month of questions filtered on it. Each of those is a proposal with its reasoning attached, and each is recorded. Two things stay with people. The contract itself: a change to an event schema is a change to the agreement between modules, and the owner of the receiving module approves it. And the shape of the record: how long events are kept, which fields leave the record under a deletion request, and how often the batches close. Those decide cost and obligation, and an agent does not sign for either. ## A test you can run today Take one fact that moves between two systems in your company today, an order confirmation or a goods receipt. Write down three things: where its field list is defined, who would be told if a field were added, and where the same fact is later read in bulk for a report. If the field list is defined in two places, the contract does not exist yet. If nobody would be told, the next added field will break something quietly. If the bulk reading happens against the same store that receives the events, the two shapes are sharing one layout, and one of them is paying for it. The [Data layer](https://hurukan.com/system/data) page describes where each shape is meant to live; the [Hurukan overview](https://hurukan.com/) shows the layers around it. - Apache Parquet - Apache Avro - event contracts - the record - data layer ## Keep reading - [One identity for people and agents: who may do what](https://hurukan.com/blog/one-identity-for-people-and-agents) - [The live state and the record: two memories of a company operating system](https://hurukan.com/blog/the-live-state-and-the-record-two-memories-of-a-company-operating-system)