Browse documentation

#Building Blocks

The Schema describes the columns of the Data Frame. It owns their names, order, types and nullability. The Rows class represents a batch of records: one column for each schema definition. Each Rows carries the one Schema that describes it.

Look at this example:

<?php

declare(strict_types=1);

use function Flow\ETL\DSL\array_to_rows;
use function Flow\ETL\DSL\bool_schema;
use function Flow\ETL\DSL\int_schema;
use function Flow\ETL\DSL\schema;
use function Flow\ETL\DSL\str_schema;

$rows = array_to_rows(
    [
        ['id' => 1, 'name' => 'user_01', 'active' => true],
        ['id' => 2, 'name' => 'user_02', 'active' => false],
        ['id' => 3, 'name' => 'user_03', 'active' => true],
        ['id' => 4, 'name' => 'user_04', 'active' => false],
    ],
    schema(int_schema('id'), str_schema('name'), bool_schema('active')),
);

Extractors yield Rows, transformers return new Rows, and loaders write them. To read a record back, use $rows->values($i) (one row as an array), $rows->toArray() (all rows) or $rows->column('name')->value($i).

#Column Types

A Definition describes each column. The matching *_schema() DSL function builds it. A definition owns the column name, its Flow Type, nullability and metadata.

A datetime column has one time zone, UTC by default: datetime_schema('at', zone: 'Europe/Warsaw'). At the admission of the row, a value in a different zone is converted to it. The instant does not change.

A column holds exactly one type. null|T writes a nullable column, and a nullable list element, map value or structure element. All other unions are refused. Declare the widest common type with str_schema(). For a shape that is really dynamic, use json_schema().

The schema is declared, never guessed. array_to_rows() takes it as its second argument. The column backend builds Rows against it, and casts each value to the type of its column. A value that the declared type refuses stops the batch with a SchemaMismatchException. The exception names the column and its row.

Schema inference belongs to the readers. They sample a source and derive a schema from it before the first row flows. For more information, read Schema.

Found a typo or an outdated section? Edit this page on GitHub ↗