You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Context: an implementer of TableProvider must implement the method fn schema(&self) -> SchemaRef, besides implementing the scan method that returns an ExecutionPlan implementation. And that ExecutionPlan implementation can return a wrapped RecordBatchStream implementation from its execute method, but that in turn must define a fn schema(&self) -> SchemaRef method as well, beyond implmenting Stream<Item = Result<RecordBatch, DataFusionError>> trait.
There are, though, conceivable circumstances where the only real way to know the schema is run through enough of the logic to get the first RecordBatch, or just shy of that some IPC header information, and then repeat the same in work in the eventual RecordBatchStream. Is there any strategy to avoid that double effort?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Context: an implementer of
TableProvider
must implement the methodfn schema(&self) -> SchemaRef
, besides implementing thescan
method that returns anExecutionPlan
implementation. And thatExecutionPlan
implementation can return a wrappedRecordBatchStream
implementation from itsexecute
method, but that in turn must define afn schema(&self) -> SchemaRef
method as well, beyond implmentingStream<Item = Result<RecordBatch, DataFusionError>>
trait.There are, though, conceivable circumstances where the only real way to know the schema is run through enough of the logic to get the first
RecordBatch
, or just shy of that some IPC header information, and then repeat the same in work in the eventualRecordBatchStream
. Is there any strategy to avoid that double effort?Beta Was this translation helpful? Give feedback.
All reactions