Processor
Interfaces
- Distributable
- Unlike a Transformer, receives the whole upstream generator, not one batch.
Classes
- BatchingByProcessor
- Assumes input is pre-sorted by the batching column.
- BatchingProcessor
- Re-batches rows into fixed-size batches.
- BoundExpansion
- CachingProcessor
- An existing cache id passes data through unchanged.
- CollectingProcessor
- ConstrainedProcessor
- Validates constraints on each row.
- CountingProcessor
- DistinctProcessor
- Distribution
- ExpandingProcessor
- GroupByAggregationProcessor
- Past the memory limit, the rest is bucketed by key hash and aggregated per bucket.
- MemorySortProcessor
- MergeSortProcessor
- OffsetProcessor
- Skips the first N rows.
- PivotProcessor
- RepartitionProcessor
- One batch per key; past the memory limit, input is bucketed and grouped per bucket.
- RequiredColumns
- TopNProcessor
- Holds at most 2×$limit rows plus one batch; ties may order unlike ExternalSort.
- VoidProcessor
- Discards all rows and yields an empty batch.
- WindowProcessor
- Applies window functions over partitioned and ordered data.