Option
Cases
- ARROW_WRITE_BATCH_SIZE
- When using the Arrow engine, controls how many rows are buffered in PHP before sending them to the extension's writeBatch() in a single call.
- BROTLI_COMPRESSION_LEVEL
- Compression level for Brotli codec. This option is going to be passed to gzcompress function when Compression is set to Brotli.
- COLUMN_INDEX_TRUNCATE_LENGTH
- Column index bounds of BYTE_ARRAY and FIXED_LEN_BYTE_ARRAY columns are truncated to this many bytes, as STATISTICS_TRUNCATE_LENGTH truncates column chunk statistics (arrow-rs set_column_index_truncate_length).
- COLUMNS_BLOOM_FILTERS
- Per-column bloom filter configuration using flat path notation.
- COLUMNS_COMPRESSIONS
- Per-column compression configuration using flat path notation.
- COLUMNS_ENCODINGS
- Per-column encoding configuration using flat path notation.
- DICTIONARY_ENABLED
- Columns without a COLUMNS_ENCODINGS entry are dictionary-encoded (RLE_DICTIONARY), except BOOLEAN and, under writer version 1, FIXED_LEN_BYTE_ARRAY, as arrow-rs and parquet-java write them. False writes the writer version's fallback encoding instead. A COLUMNS_ENCODINGS entry always wins.
- DICTIONARY_PAGE_SIZE
- Once the column chunk's dictionary reaches this size, the rest of the chunk falls back to the writer version's fallback encoding (PLAIN under v1).
- GZIP_COMPRESSION_LEVEL
- Compression level for GZIP codec. This option is going to be passed to gzcompress function when Compression is set to GZIP.
- INT_96_AS_DATETIME
- When this option is set to true, reader will try to convert INT96 logical type to DateTimeImmutable object.
- LZ4_COMPRESSION_LEVEL
- Compression level for LZ4 codec. This option is going to be passed to lz4_compress function when Compression is set to LZ4.
- PAGE_HEADER_STATISTICS
- Data page headers carry the page's statistics when the statistics level is PAGE, as arrow-rs set_write_page_header_statistics. Readers prune pages through the column index instead.
- PAGE_MAXIMUM_ROWS_COUNT
- The rows a data page holds at most, beside PAGE_SIZE_BYTES (arrow-rs data_page_row_count_limit). The PHP writer checks it every PAGE_SIZE_CHECK_INTERVAL rows, the arrow writer every write batch. Smaller pages let the page index skip more of a filtered read, at the cost of more page headers.
- PAGE_SIZE_BYTES
- PageBuilder is going to use this value to determine how many rows should be stored in one page.
- PAGE_SIZE_CHECK_INTERVAL
- ColumnChunksBuilder is going to use this value to determine how often it should check if ColumnChunk Page size is not exceeded.
- ROW_GROUP_MAXIMUM_ROWS_COUNT
- The rows a row group holds at most, beside ROW_GROUP_SIZE_BYTES (arrow-rs max_row_group_row_count): a row group is closed when either limit is reached. Readers decode a row group at a time, so this bounds their memory too.
- ROW_GROUP_SIZE_BYTES
- RowGroupBuilder is going to use this value to determine for how long it should keep adding rows to the buffer before flushing it on disk.
- ROW_GROUP_SIZE_CHECK_INTERVAL
- RowGroupBuilder is going to use this value to determine how often it should check if RowGroup size is not exceeded.
- STATISTICS_LEVEL
- A Writer\StatisticsLevel: NONE writes no statistics, CHUNK writes column chunk statistics, PAGE also writes a column index (per page min, max, null and NaN counts). The offset index is written at every level.
- STATISTICS_TRUNCATE_LENGTH
- Column chunk and page statistics of BYTE_ARRAY and FIXED_LEN_BYTE_ARRAY columns are truncated to this many bytes, as arrow-rs does: the min is cut, the max is cut and incremented, and a truncated bound is marked inexact.
- VALIDATE_DATA
- This option is going to tell the writer to validate data against schema.
- WRITER_VERSION
- There are only two available versions of Parquet format: V1 and V2.
- ZSTD_COMPRESSION_LEVEL
- Compression level for ZSTD codec. This option is going to be passed to zstd_compress function when Compression is set to ZSTD.
Cases
ARROW_WRITE_BATCH_SIZE
When using the Arrow engine, controls how many rows are buffered in PHP before sending them to the extension's writeBatch() in a single call.
Larger values reduce call overhead but increase PHP memory usage.
Only affects the Arrow engine. Ignored by the PHP engine.
Default: 1000
BROTLI_COMPRESSION_LEVEL
Compression level for Brotli codec. This option is going to be passed to gzcompress function when Compression is set to Brotli.
The higher the quality, the slower the compression.
Default value is 11 (BROTLI_COMPRESS_LEVEL_DEFAULT)
COLUMNS_COMPRESSIONS
Per-column compression configuration using flat path notation.
Accepts array<string, Compressions> where key is column flat path and value is compression enum.
Flat path examples:
- Simple columns: 'column_name'
- Nested structs: 'user.address.street'
- List elements: 'items.list.element'
- Map keys/values: 'metadata.key_value.key', 'metadata.key_value.value'
Values must be Compressions enum instances.
Default: null (use global compression setting)
COLUMNS_BLOOM_FILTERS
Per-column bloom filter configuration using flat path notation.
Accepts array<string, BloomFilterOptions> where key is column flat path and value is the filter's false positive probability and expected distinct values.
Flat path examples:
- Simple columns: 'column_name'
- Nested structs: 'user.address.street'
- List elements: 'items.list.element'
- Map keys/values: 'metadata.key_value.key', 'metadata.key_value.value'
Values must be BloomFilterOptions instances. Readers use the filters for equality and in predicates only.
Default: null (no bloom filters)
COLUMNS_ENCODINGS
Per-column encoding configuration using flat path notation.
Accepts array<string, Encodings> where key is column flat path and value is encoding enum.
Flat path examples:
- Simple columns: 'column_name'
- Nested structs: 'user.address.street'
- List elements: 'items.list.element'
- Map keys/values: 'metadata.key_value.key', 'metadata.key_value.value'
Values must be Encodings enum instances.
Default: null (use automatic encoding selection)
COLUMN_INDEX_TRUNCATE_LENGTH
Column index bounds of BYTE_ARRAY and FIXED_LEN_BYTE_ARRAY columns are truncated to this many bytes, as STATISTICS_TRUNCATE_LENGTH truncates column chunk statistics (arrow-rs set_column_index_truncate_length).
Default value is 64
DICTIONARY_ENABLED
Columns without a COLUMNS_ENCODINGS entry are dictionary-encoded (RLE_DICTIONARY), except BOOLEAN and, under writer version 1, FIXED_LEN_BYTE_ARRAY, as arrow-rs and parquet-java write them. False writes the writer version's fallback encoding instead. A COLUMNS_ENCODINGS entry always wins.
Default value is true
DICTIONARY_PAGE_SIZE
Once the column chunk's dictionary reaches this size, the rest of the chunk falls back to the writer version's fallback encoding (PLAIN under v1).
Default value is 1Mb
GZIP_COMPRESSION_LEVEL
Compression level for GZIP codec. This option is going to be passed to gzcompress function when Compression is set to GZIP.
Lower level means faster compression, but bigger file size.
Default value is 9
INT_96_AS_DATETIME
When this option is set to true, reader will try to convert INT96 logical type to DateTimeImmutable object.
Some parquet writers due to historical reasons might still use INT96 to store timestamps with nanoseconds precision instead of using TIMESTAMP logical type. Since PHP does not support nanoseconds precision for DateTime objects, when this options is set to true, reader will round nanoseconds to microseconds.
INT96 in general is not supported anymore, this option should be set to true by default, otherwise it will return array of bytes (12) that represents INT96.
Default value is true
LZ4_COMPRESSION_LEVEL
Compression level for LZ4 codec. This option is going to be passed to lz4_compress function when Compression is set to LZ4.
The level of compression (1-12, Recommended values are between 4 and 9).
Default value is 4
PAGE_HEADER_STATISTICS
Data page headers carry the page's statistics when the statistics level is PAGE, as arrow-rs set_write_page_header_statistics. Readers prune pages through the column index instead.
Default value is false
PAGE_MAXIMUM_ROWS_COUNT
The rows a data page holds at most, beside PAGE_SIZE_BYTES (arrow-rs data_page_row_count_limit). The PHP writer checks it every PAGE_SIZE_CHECK_INTERVAL rows, the arrow writer every write batch. Smaller pages let the page index skip more of a filtered read, at the cost of more page headers.
Default value is 1000 rows
PAGE_SIZE_BYTES
PageBuilder is going to use this value to determine how many rows should be stored in one page.
PageBuilder is not going to make it precisely equal to this value, but it will try to make it as close as possible. This should be considered as a threshold rather than a strict value.
Default value is 128Kb
https://parquet.apache.org/docs/file-format/configurations/#data-page--size
PAGE_SIZE_CHECK_INTERVAL
ColumnChunksBuilder is going to use this value to determine how often it should check if ColumnChunk Page size is not exceeded.
This is a performance optimization, since checking Page size is a costly operation. If the value is set to 100, ColumnChunksBuilder is going to check the size only after adding 100 rows to the buffer.
Default value is 100
ROW_GROUP_MAXIMUM_ROWS_COUNT
The rows a row group holds at most, beside ROW_GROUP_SIZE_BYTES (arrow-rs max_row_group_row_count): a row group is closed when either limit is reached. Readers decode a row group at a time, so this bounds their memory too.
Default value is 1048576 rows
ROW_GROUP_SIZE_BYTES
RowGroupBuilder is going to use this value to determine for how long it should keep adding rows to the buffer before flushing it on disk.
Default value is 32Mb
Recommended: 128-512 MB. https://parquet.apache.org/docs/file-format/configurations/#row-group-size
ROW_GROUP_SIZE_CHECK_INTERVAL
RowGroupBuilder is going to use this value to determine how often it should check if RowGroup size is not exceeded.
This is a performance optimization, since checking RowGroup size is a costly operation. If the value is set to 1000, RowGroupBuilder is going to check the size only after adding 1000 rows to the buffer.
Default value is 1000
STATISTICS_LEVEL
A Writer\StatisticsLevel: NONE writes no statistics, CHUNK writes column chunk statistics, PAGE also writes a column index (per page min, max, null and NaN counts). The offset index is written at every level.
Default value is StatisticsLevel::PAGE
STATISTICS_TRUNCATE_LENGTH
Column chunk and page statistics of BYTE_ARRAY and FIXED_LEN_BYTE_ARRAY columns are truncated to this many bytes, as arrow-rs does: the min is cut, the max is cut and incremented, and a truncated bound is marked inexact.
Decimals and FLOAT16 are never truncated.
Default value is 64
VALIDATE_DATA
This option is going to tell the writer to validate data against schema.
In most cases that should be enabled, however if performance is critical, it can be disabled.
WRITER_VERSION
There are only two available versions of Parquet format: V1 and V2.
This option is going to tell the writer which version should be used to create DataPages.
- 1 will use DataPage
- 2 will use DataPageV2.
Default 1
ZSTD_COMPRESSION_LEVEL
Compression level for ZSTD codec. This option is going to be passed to zstd_compress function when Compression is set to ZSTD.
A value smaller than 0 means a faster compression level. (Zstandard library 1.3.4 or later).
Default value is 3