Skip to main content
Disk sinks write data to local files. Two types are available: disk_binary for high-throughput binary storage and disk_plaintext for human-readable logs.

​
Plaintext

Human-readable log files, split by data type. Easy to grep, tail, and debug.

​
Output format

Events and logs are decoded from FlatBuffer format and written as structured text:

​
Directory structure

Files are separated by type within each workspace and date:

​
Binary

High-performance binary storage with 24-byte metadata headers. Use this for archival when you need maximum throughput and plan to read with tell read.

​
File format

Each message is stored as:
Metadata contains the batch timestamp (8 bytes, Unix ms) and source IP (16 bytes, IPv6 format). Read binary files with tell read:

​
Atomic rotation

Both disk sinks use atomic rotation for zero data loss during file switches. The hot write path is lock-free and non-blocking. File rotation runs on a separate task and checks every 60 seconds. If the write queue fills up (sink can’t keep up), batches are dropped for that sink and logged as backpressure events.