otp.Source.write_parquet#

Source.write_parquet(output_path, compression_type='snappy', num_tick_per_row_group=1000, partitioning_keys='', propagate_input_ticks=False, inplace=False)#

Writes the input tick series to parquet data file.

Input must not have field ‘time’ as that field will also be added by the EP in the resulting file(s)

Parameters
  • output_path (str) – Path for saving ticks to Parquet file. Partitioned: Path to the root directory of the parquet files. Non-partitioned: Path to the parquet file.

  • compression_type (str) – Compression type for parquet files. Should be one of these: gzip, lz4, none, snappy (default), zstd.

  • num_tick_per_row_group (int) – Number of rows per row group.

  • partitioning_keys (list, str) –

    List of fields (list or comma-separated string) to be used as keys for partitioning.

    Setting this parameter will switch this EP to partitioned mode.

    In non-partitioned mode, if the path points to a file that already exists, it will be overridden. When partitioning is active:

    • The target directory must be empty

    • Key fields and their string values will be automatically URL-encoded to avoid conflicts with

      filesystem naming rules.

    Pseudo-fields ‘_SYMBOL_NAME’ and ‘_TICK_TYPE’ may be used as partitioning_keys and will be added to the schema automatically.

  • propagate_input_ticks (bool) – Switches propagation of the ticks. If set to True, ticks will be propagated.

  • inplace (bool) – A flag controls whether operation should be applied inplace. If inplace=True, then it returns nothing. Otherwise method returns a new modified object.

Examples

Simple usage

>>> data = otp.Ticks(A=[1, 2, 3])
>>> data = data.write_parquet("/path/to/parquet/file")  
>>> df = otp.run(data)  

See also

WRITE_TO_PARQUET OneTick event processor