JsonExtractor implements BatchableExtractor, Extractor, FileExtractor, InfersSchema, MetadataColumnsExtractor, CorruptedRecordsRouter, SplittableExtractor
FinalYes
Interfaces
- BatchableExtractor
- Extractor
- FileExtractor
- InfersSchema
- MetadataColumnsExtractor
- CorruptedRecordsRouter
- SplittableExtractor
Methods
- __construct() : mixed
- batchSize() : int<1, max>
- corruptedRecordsLoader() : Loader|null
- extract() : Generator<Signal|null, void>
- inferSchema() : static
- isRepeatable() : bool
- partitionSchema() : Schema
- partitionTypes() : static
- prunes() : bool
- reader() : FileSplitReader
- refresh() : void
- schema() : Schema
- source() : Path
- splits() : array<int, FileSplit>
- JSON lines are cut into byte windows, both ends resynced to a line start by the reader; a JSON document is a tree with no offset a second reader could start at, so it is one split per admitted file.
- statistics() : Statistics
- Rows come from what the sample schema inference already read; without one they are unknown, a sample is never read just for them. A document has no bytes-per-row unit, so its rows are known only when that sample read every element of every file; JSON lines estimate them from the sampled bytes per row.
- withBatchSize() : static
- withCorruptedRecords() : static
- withMetadataColumns() : static
- withPointer() : self
- withSchema() : static
Methods
__construct()
public
__construct(Path $path[, Filesystem $filesystem = new NativeLocalFilesystem() ][, JsonFraming $format = JsonFraming::Array ]) : mixed
Parameters
- $path : Path
- $filesystem : Filesystem = new NativeLocalFilesystem()
- $format : JsonFraming = JsonFraming::Array
batchSize()
public
batchSize() : int<1, max>
Return values
int<1, max>corruptedRecordsLoader()
public
corruptedRecordsLoader() : Loader|null
Return values
Loader|nullextract()
public
extract(Backend $backend, Criteria $criteria) : Generator<Signal|null, void>
Parameters
- $backend : Backend
- $criteria : Criteria
Return values
Generator<Signal|null, void>inferSchema()
public
inferSchema(SchemaInferenceBuilder $builder) : static
Parameters
- $builder : SchemaInferenceBuilder
Return values
staticisRepeatable()
public
isRepeatable() : bool
Return values
boolpartitionSchema()
public
partitionSchema() : Schema
Return values
SchemapartitionTypes()
public
partitionTypes(PartitionTypes $types) : static
Parameters
- $types : PartitionTypes
Return values
staticprunes()
public
prunes(ScalarFunction $conjunct) : bool
Parameters
- $conjunct : ScalarFunction
Return values
boolreader()
public
reader() : FileSplitReader
Return values
FileSplitReaderrefresh()
public
refresh() : void
schema()
public
schema() : Schema
Return values
Schemasource()
public
source() : Path
Return values
Pathsplits()
JSON lines are cut into byte windows, both ends resynced to a line start by the reader; a JSON document is a tree with no offset a second reader could start at, so it is one split per admitted file.
public
splits(Criteria $criteria, int $targetSplitBytes) : array<int, FileSplit>
Parameters
- $criteria : Criteria
- $targetSplitBytes : int
Return values
array<int, FileSplit>statistics()
Rows come from what the sample schema inference already read; without one they are unknown, a sample is never read just for them. A document has no bytes-per-row unit, so its rows are known only when that sample read every element of every file; JSON lines estimate them from the sampled bytes per row.
public
statistics() : Statistics
Return values
StatisticswithBatchSize()
public
withBatchSize(int $batchSize) : static
Parameters
- $batchSize : int
Return values
staticwithCorruptedRecords()
public
withCorruptedRecords(Loader|null $loader) : static
Parameters
- $loader : Loader|null
Return values
staticwithMetadataColumns()
public
withMetadataColumns(bool $addMetadataColumns) : static
Parameters
- $addMetadataColumns : bool
Return values
staticwithPointer()
public
withPointer(string $pointer[, bool $pointerToColumnName = false ]) : self
Parameters
- $pointer : string
- $pointerToColumnName : bool = false
-
- when true pointer will be used as column name for extracted data
Return values
selfwithSchema()
public
withSchema(Schema $schema) : static
Parameters
- $schema : Schema