> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tell.rs/llms.txt
> Use this file to discover all available pages before exploring further.

# Arrow IPC Sink

> Fast columnar storage for hot data with zero-copy reads for Polars, DuckDB, and DataFusion.

The Arrow IPC sink writes data to Arrow IPC files (also known as Feather) for fast I/O and zero-copy reads. Use it for hot data that needs frequent access — real-time dashboards, local development, and inter-process communication.

## Configuration

```toml theme={null}
[sinks.hot]
type = "arrow_ipc"
path = "/data/arrow"
rotation = "daily"
```

| Field            | Default   | Notes                       |
| ---------------- | --------- | --------------------------- |
| `path`           | —         | Output directory (required) |
| `rotation`       | `"daily"` | `"hourly"` or `"daily"`     |
| `buffer_size`    | 10,000    | Rows buffered before flush  |
| `flush_interval` | `"60s"`   | Time-based flush interval   |

## Tables

The Arrow IPC sink writes the full Tell v1.1 schema — the same 7 tables as the ClickHouse sink:

| File                 | Populated by                       |
| -------------------- | ---------------------------------- |
| `events_v1.arrow`    | TRACK events                       |
| `logs_v1.arrow`      | Log entries                        |
| `snapshots_v1.arrow` | Integration snapshots              |
| `context_v1.arrow`   | CONTEXT events (device/location)   |
| `users_v1.arrow`     | IDENTIFY events (core identity)    |
| `user_devices.arrow` | IDENTIFY events (device links)     |
| `user_traits.arrow`  | IDENTIFY events (key-value traits) |

Like the ClickHouse sink, IDENTIFY events fan out to `users_v1`, `user_devices`, and `user_traits`. CONTEXT events extract device and location fields from the payload.

## File organization

```
arrow_ipc/
└── {workspace_id}/
    └── {date}/
        ├── events_v1.arrow
        ├── logs_v1.arrow
        ├── context_v1.arrow
        ├── users_v1.arrow
        ├── user_devices.arrow
        └── user_traits.arrow
```

## Reading Arrow IPC files

Arrow IPC is supported natively by all major data tools:

```python theme={null}
# Polars (recommended — zero-copy read)
import polars as pl
df = pl.read_ipc("arrow_ipc/1/2025-01-15/events_v1.arrow")

# DuckDB
SELECT * FROM 'arrow_ipc/1/2025-01-15/events_v1.arrow';

# PyArrow
import pyarrow.ipc as ipc
reader = ipc.open_file("arrow_ipc/1/2025-01-15/events_v1.arrow")
```

Also supported by DataFusion (native) and any Arrow-compatible tool.

## Local query backend

Tell's local query engine (Polars backend) reads from Arrow IPC files directly. Configure this in `[query]`:

```toml theme={null}
[query]
backend = "polars"
data_dir = "/data/arrow"
```

This is useful for development and small deployments where ClickHouse isn't needed.