FloeExtractor implements BatchableExtractor, Extractor, FileExtractor, LimitPushDown, MetadataColumnsExtractor, RewindableExtractor uses Batches, PushesLimit, FileReading
FinalYes
Interfaces
- BatchableExtractor
- A source that decides how many rows it puts in one Rows. withBatchSize() bounds what the source BUILDS; batches() re-slices what a source already emitted and cannot lower its peak.
- Extractor
- FileExtractor
- LimitPushDown
- A source that can read fewer rows when the plan has a LIMIT above it. This is an optimization hint, never a guarantee: the limit operator stays in the plan and enforces the exact count, so an implementation that yields more rows than asked - or ignores the hint - is still correct.
- MetadataColumnsExtractor
- RewindableExtractor
- A rewindable extractor can be read more than once and yield the same rows each time, so a build-time pass may scan it before extraction without consuming it.
Methods
- __construct() : mixed
- batchSize() : int<1, max>
- extract() : Generator<Signal|null, void>
- Yields rows in batches. When the source is a BatchableExtractor a batch holds between 0 and that extractor's own batchSize() rows; the size is chosen by the extractor and MAY vary between batches. Consumers must not assume a minimum size, a constant size, or a non-empty batch.
- isRepeatable() : bool
- pushedLimit() : int|null
- pushLimit() : void
- schema() : Schema
- Footer-only source schema (two ranged reads per file, no row scan). One file unless unionByName() asks for the fold, and memoised, so repeated calls cost nothing.
- source() : Path
- unionByName() : self
- Reconcile every listed file's footer instead of trusting the first one.
- withBatchSize() : static
- withOffset() : self
- withSchema() : static
- Declares the shape every yielded batch must carry. A source that describes itself uses this instead of its own description, and values are cast to fit.
Methods
__construct()
public
__construct(Path $path[, Codec $codec = new NoopCodec() ][, int $chunkSize = 65536 ][, FloeEngine $engine = FloeEngine::adaptive ][, Filesystem $filesystem = new NativeLocalFilesystem() ]) : mixed
Parameters
- $path : Path
- $codec : Codec = new NoopCodec()
- $chunkSize : int = 65536
- $engine : FloeEngine = FloeEngine::adaptive
- $filesystem : Filesystem = new NativeLocalFilesystem()
batchSize()
public
batchSize() : int<1, max>
Return values
int<1, max>extract()
Yields rows in batches. When the source is a BatchableExtractor a batch holds between 0 and that extractor's own batchSize() rows; the size is chosen by the extractor and MAY vary between batches. Consumers must not assume a minimum size, a constant size, or a non-empty batch.
public
extract(FlowContext $context) : Generator<Signal|null, void>
Parameters
- $context : FlowContext
Return values
Generator<Signal|null, void>isRepeatable()
public
isRepeatable() : bool
Return values
boolpushedLimit()
public
pushedLimit() : int|null
Return values
int|nullpushLimit()
public
pushLimit(int $limit) : void
Parameters
- $limit : int
schema()
Footer-only source schema (two ranged reads per file, no row scan). One file unless unionByName() asks for the fold, and memoised, so repeated calls cost nothing.
public
schema() : Schema
Return values
Schemasource()
public
source() : Path
Return values
PathunionByName()
Reconcile every listed file's footer instead of trusting the first one.
public
unionByName([bool $union = true ]) : self
Parameters
- $union : bool = true
Return values
selfwithBatchSize()
public
withBatchSize(int $batchSize) : static
Parameters
- $batchSize : int
Return values
staticwithOffset()
public
withOffset(int $offset) : self
Parameters
- $offset : int
Return values
selfwithSchema()
Declares the shape every yielded batch must carry. A source that describes itself uses this instead of its own description, and values are cast to fit.
public
withSchema(Schema $schema) : static
Parameters
- $schema : Schema