Skip to main content
Version: 0.5.4 (latest)

common.data_writers.writers

FileWriterSpec Objects

class FileWriterSpec(NamedTuple)

[view_source]

file_format

format of the output file

data_item_format

format of the input data

supports_schema_changes

File format supports changes of schema: True - at any moment, Buffer - in memory buffer before opening file, False - not at all

DataWriter Objects

class DataWriter(abc.ABC)

[view_source]

item_format_from_file_extension

@classmethod
def item_format_from_file_extension(cls, extension: str) -> TDataItemFormat

[view_source]

Simple heuristic to get data item format from file extension

ImportFileWriter Objects

class ImportFileWriter(DataWriter)

[view_source]

May only import files, fails on any open/write operations

ArrowToObjectAdapter Objects

class ArrowToObjectAdapter()

[view_source]

A mixin that will convert object writer into arrow writer.

is_native_writer

def is_native_writer(writer_type: Type[DataWriter]) -> bool

[view_source]

Checks if writer has adapter mixin. Writers with adapters are not native and typically decrease the performance.

resolve_best_writer_spec

def resolve_best_writer_spec(
item_format: TDataItemFormat,
possible_file_formats: Sequence[TLoaderFileFormat],
preferred_format: TLoaderFileFormat = None) -> FileWriterSpec

[view_source]

Finds best writer for item_format out of possible_file_formats. Tries preferred_format first. Best possible writer is a native writer for item_format writing files in preferred_format. If not found, any native writer for possible_file_formats is picked. Native writer supports item_format directly without a need to convert to other item formats.

get_best_writer_spec

def get_best_writer_spec(item_format: TDataItemFormat,
file_format: TLoaderFileFormat) -> FileWriterSpec

[view_source]

Gets writer for item_format writing files in {file_format}. Looks for native writer first

create_import_spec

def create_import_spec(
item_file_format: TLoaderFileFormat,
possible_file_formats: Sequence[TLoaderFileFormat]) -> FileWriterSpec

[view_source]

Creates writer spec that may be used only to import files

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.