> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tell.rs/llms.txt
> Use this file to discover all available pages before exploring further.

# Parquet Sink

> Columnar storage for data warehousing with Snappy, Zstd, or LZ4 compression.

The Parquet sink writes data to Apache Parquet files — a columnar format optimized for analytical queries with excellent compression ratios. Use it for cold storage and data warehousing.

## Configuration

```toml theme={null}
[sinks.archive]
type = "parquet"
path = "/data/parquet"
compression = "snappy"
rotation = "hourly"
```

| Field            | Default    | Notes                                              |
| ---------------- | ---------- | -------------------------------------------------- |
| `path`           | —          | Output directory (required)                        |
| `compression`    | `"snappy"` | `"snappy"`, `"zstd"`, `"lz4"`, or `"uncompressed"` |
| `rotation`       | `"hourly"` | `"hourly"` or `"daily"`                            |
| `buffer_size`    | 10,000     | Rows buffered before flush                         |
| `row_group_size` | 100,000    | Max rows per row group                             |
| `flush_interval` | `"60s"`    | Time-based flush interval                          |

## Schemas

Data is separated into different Parquet files by type, each with an optimized schema. Fields are ordered for predicate pushdown — frequently filtered columns come first, large payloads last.

**Events** (9 fields): `timestamp`, `batch_timestamp`, `workspace_id`, `event_type`, `event_name`, `device_id`, `session_id`, `source_ip`, `payload`

**Logs** (10 fields): `timestamp`, `batch_timestamp`, `workspace_id`, `level`, `event_type`, `source`, `service`, `session_id`, `source_ip`, `payload`

**Snapshots** (7 fields): `timestamp`, `batch_timestamp`, `workspace_id`, `source`, `entity`, `source_ip`, `payload`

## File organization

```
parquet/
└── {workspace_id}/
    └── {date}/
        └── {hour}/
            ├── events.parquet
            ├── logs.parquet
            └── snapshots.parquet
```

## Reading Parquet files

Parquet is a standard format readable by most analytics tools:

```python theme={null}
# DuckDB
SELECT * FROM 'parquet/1/2025-01-15/10/*.parquet';

# Polars
import polars as pl
df = pl.read_parquet("parquet/1/2025-01-15/10/events.parquet")

# Pandas
import pandas as pd
df = pd.read_parquet("parquet/1/2025-01-15/10/events.parquet")
```

Also readable by Apache Spark, ClickHouse (`file()` table function), and PyArrow.

## When to use Parquet vs Arrow IPC

Use **Parquet** for cold data — archival, compliance, data warehousing. Compression ratios are excellent (especially with Zstd) and the format is widely supported.

Use **[Arrow IPC](/pipeline/sinks/arrow-ipc)** for hot data — real-time dashboards, frequent reads, inter-process communication. Arrow IPC is \~10x faster to read but files are larger.