# Corrections and Time Travel This section contains 5 examples for Corrections and Time Travel using the `onetick-py`.
\\\\ Each example is a self-contained script that can be run against the OneTick Cloud sample databases. ```default # onetick-py WebAPI configuration for OneTick Cloud import os os.environ['OTP_WEBAPI'] = '1' os.environ['OTP_HTTP_ADDRESS'] = 'https://rest.cloud.onetick.com' os.environ['OTP_ACCESS_TOKEN_URL'] = 'https://cloud-auth.parent.onetick.com/realms/OMD/protocol/openid-connect/token' os.environ['OTP_CLIENT_ID'] = '__FILL_IN__' os.environ['OTP_CLIENT_SECRET'] = '__FILL_IN__' ``` ## Corrected Trade Retrieval Standard Trade Retrieval returns data adjusted for Trade Corrections.
\\\\ Deleted Trades will not be visible. ```ipython3 import onetick.py as otp data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD') data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']] data = data.limit(10) result = otp.run(data, start=otp.dt(2024, 1, 4, 11, 4, 0), end=otp.dt(2024, 1, 6), timezone='UTC', symbols='VOD') result ``` ```myst-ansi Time TRADE_ID PRICE SIZE TRADE_TYPE \ 0 2024-01-04 11:04:29.404 912346159533907 69.7400 4121 AT 1 2024-01-04 11:04:29.428 912346159533908 69.7200 344 AT 2 2024-01-04 11:04:29.431 912346159533909 69.7200 4715 AT 3 2024-01-04 11:04:29.431 912346159533910 69.7200 2480 AT 4 2024-01-04 11:04:57.354 730098958764564592 69.7200 555 OB 5 2024-01-04 11:05:21.798 912346159533939 69.7300 340 AT 6 2024-01-04 11:05:22.238 912346159533940 69.7500 2843 AT 7 2024-01-04 11:05:22.238 912346159533941 69.7400 2647 AT 8 2024-01-04 11:05:24.294 582380190048149616 69.7252 6485 OB 9 2024-01-04 11:05:36.145 912346159533960 69.7300 4715 AT TRADE_VENUE TICK_STATUS DELETED_TIME 0 XLON 0 1970-01-01 1 XLON 0 1970-01-01 2 XLON 0 1970-01-01 3 XLON 0 1970-01-01 4 SINT 0 1970-01-01 5 XLON 0 1970-01-01 6 XLON 0 1970-01-01 7 XLON 0 1970-01-01 8 XLON 0 1970-01-01 9 XLON 0 1970-01-01 ``` ## Hidden Records Including Corrections All Records including trade corrections can be retrieved by using [`show_hidden_ticks()`](https://docs.pip.distribution.sol.onetick.com/api/source/show_hidden_ticks.html.md#onetick.py.Source.show_hidden_ticks).
\\\\ This propagates all ticks, even those with a `TICK_STATUS` not equal to 0, which are normally hidden.
\\\\ Corrected Trades can be identified by their `DELETED_TIME` and `TICK_STATUS` fields.
\\\\ `DELETED_TIME` corresponds to the time the record was corrected, which may be days after the original record.
\\\\ `TICK_STATUS` refers to the type of change: * 0 - Default * 1 - Deleted record * 2 - Updated record * 3 - Insert Corrected * 4 - Record that has been Canceled * 5 - Record that has been Corrected * 6 - New Correction record * 7 - New Cancellation record ```ipython3 import onetick.py as otp data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD') data = data.show_hidden_ticks() data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']] data = data.limit(10) result = otp.run(data, start=otp.dt(2024, 1, 4, 11, 4, 0), end=otp.dt(2024, 1, 6), timezone='UTC', symbols='VOD') result ``` ```myst-ansi Time TRADE_ID PRICE SIZE TRADE_TYPE \ 0 2024-01-04 11:04:04.096 577551547295817840 69.6300 344 OB 1 2024-01-04 11:04:29.404 912346159533907 69.7400 4121 AT 2 2024-01-04 11:04:29.428 912346159533908 69.7200 344 AT 3 2024-01-04 11:04:29.431 912346159533909 69.7200 4715 AT 4 2024-01-04 11:04:29.431 912346159533910 69.7200 2480 AT 5 2024-01-04 11:04:57.354 730098958764564592 69.7200 555 OB 6 2024-01-04 11:05:21.798 912346159533939 69.7300 340 AT 7 2024-01-04 11:05:22.238 912346159533940 69.7500 2843 AT 8 2024-01-04 11:05:22.238 912346159533941 69.7400 2647 AT 9 2024-01-04 11:05:24.294 582380190048149616 69.7252 6485 OB TRADE_VENUE TICK_STATUS DELETED_TIME 0 XLON 1 2024-01-05 10:15:22.417 1 XLON 0 1970-01-01 00:00:00.000 2 XLON 0 1970-01-01 00:00:00.000 3 XLON 0 1970-01-01 00:00:00.000 4 XLON 0 1970-01-01 00:00:00.000 5 SINT 0 1970-01-01 00:00:00.000 6 XLON 0 1970-01-01 00:00:00.000 7 XLON 0 1970-01-01 00:00:00.000 8 XLON 0 1970-01-01 00:00:00.000 9 XLON 0 1970-01-01 00:00:00.000 ``` ## Trade Corrections Trade corrections can be retrieved by using [`show_corrected_ticks()`](https://docs.pip.distribution.sol.onetick.com/api/source/show_corrected_ticks.html.md#onetick.py.Source.show_corrected_ticks).
\\\\ Only corrected and correction ticks will be propagated.
\\\\ Corrected Trades can be identified by their `DELETED_TIME` and `TICK_STATUS` fields.
\\\\ `DELETED_TIME` corresponds to the time the record was corrected, which may be days after the original record.
\\\\ `TICK_STATUS` refers to the type of change, in the example below: * 4 - Record that has been Canceled * 7 - New Cancellation record ```ipython3 import onetick.py as otp data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD') data = data.show_corrected_ticks() data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']] data = data.limit(10) result = otp.run(data, start=otp.dt(2024, 1, 4, 11, 4, 0), end=otp.dt(2024, 1, 6), timezone='UTC', symbols='VOD') result ``` ```myst-ansi Time TRADE_ID PRICE SIZE TRADE_TYPE \ 0 2024-01-05 10:15:22.417 577551547295817840 69.63 344 OB 1 2024-01-05 10:15:22.417 577551547295817840 69.63 344 OB 2 2024-01-05 13:01:28.511 432561984411230320 70.00 7750 OB 3 2024-01-05 13:01:28.511 432561984411230320 70.00 7750 OB 4 2024-01-05 15:49:25.706 592575148552052848 70.08 5000 OB 5 2024-01-05 15:49:25.706 592575148552052848 70.08 5000 OB TRADE_VENUE TICK_STATUS DELETED_TIME 0 XLON 4 2024-01-04 11:04:04.096 1 XLON 7 1970-01-01 00:00:00.000 2 XLON 4 2024-01-05 12:59:35.020 3 XLON 7 1970-01-01 00:00:00.000 4 XLON 4 2024-01-05 13:01:32.811 5 XLON 7 1970-01-01 00:00:00.000 ``` ## Trades Before Correction Trade data can be retrieved as it was at a specific point in time using [`correct_tick_filter()`](https://docs.pip.distribution.sol.onetick.com/api/source/correct_tick_filter.html.md#onetick.py.Source.correct_tick_filter). * If the specified `as_of_time` is set before the trade corrections, uncorrected data is returned. * If the specified `as_of_time` is set after the trade corrections, corrected data is returned. This provides a Time Travel capability, returning data before and after changes to the data occur.
\\\\ Here the `as_of_time` is set to a date before the trade corrections, so uncorrected data is returned. ```ipython3 import onetick.py as otp data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD') data = data.correct_tick_filter(discard_on_match=False, as_of_time=otp.dt(2024, 1, 4)) data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']] data = data.limit(10) result = otp.run(data, start=otp.dt(2024, 1, 4, 11, 4, 0), end=otp.dt(2024, 1, 6), timezone='UTC', symbols='VOD') result ``` ```myst-ansi Time TRADE_ID PRICE SIZE TRADE_TYPE \ 0 2024-01-04 11:04:04.096 577551547295817840 69.6300 344 OB 1 2024-01-04 11:04:29.404 912346159533907 69.7400 4121 AT 2 2024-01-04 11:04:29.428 912346159533908 69.7200 344 AT 3 2024-01-04 11:04:29.431 912346159533909 69.7200 4715 AT 4 2024-01-04 11:04:29.431 912346159533910 69.7200 2480 AT 5 2024-01-04 11:04:57.354 730098958764564592 69.7200 555 OB 6 2024-01-04 11:05:21.798 912346159533939 69.7300 340 AT 7 2024-01-04 11:05:22.238 912346159533940 69.7500 2843 AT 8 2024-01-04 11:05:22.238 912346159533941 69.7400 2647 AT 9 2024-01-04 11:05:24.294 582380190048149616 69.7252 6485 OB TRADE_VENUE TICK_STATUS DELETED_TIME 0 XLON 1 2024-01-05 10:15:22.417 1 XLON 0 1970-01-01 00:00:00.000 2 XLON 0 1970-01-01 00:00:00.000 3 XLON 0 1970-01-01 00:00:00.000 4 XLON 0 1970-01-01 00:00:00.000 5 SINT 0 1970-01-01 00:00:00.000 6 XLON 0 1970-01-01 00:00:00.000 7 XLON 0 1970-01-01 00:00:00.000 8 XLON 0 1970-01-01 00:00:00.000 9 XLON 0 1970-01-01 00:00:00.000 ``` ## Trades After Correction Trade data can be retrieved as it was at a specific point in time using [`correct_tick_filter()`](https://docs.pip.distribution.sol.onetick.com/api/source/correct_tick_filter.html.md#onetick.py.Source.correct_tick_filter). * If the specified `as_of_time` is set before the trade corrections, uncorrected data is returned. * If the specified `as_of_time` is set after the trade corrections, corrected data is returned. This provides a Time Travel capability, returning data before and after changes to the data occur.
\\\\ Here the `as_of_time` is set to a date after the trade corrections, so corrected data is returned. ```ipython3 import onetick.py as otp data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD') data = data.correct_tick_filter(discard_on_match=False, as_of_time=otp.dt(2024, 1, 6)) data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']] data = data.limit(10) result = otp.run(data, start=otp.dt(2024, 1, 4, 11, 4, 0), end=otp.dt(2024, 1, 6), timezone='UTC', symbols='VOD') result ``` ```myst-ansi Time TRADE_ID PRICE SIZE TRADE_TYPE \ 0 2024-01-04 11:04:29.404 912346159533907 69.7400 4121 AT 1 2024-01-04 11:04:29.428 912346159533908 69.7200 344 AT 2 2024-01-04 11:04:29.431 912346159533909 69.7200 4715 AT 3 2024-01-04 11:04:29.431 912346159533910 69.7200 2480 AT 4 2024-01-04 11:04:57.354 730098958764564592 69.7200 555 OB 5 2024-01-04 11:05:21.798 912346159533939 69.7300 340 AT 6 2024-01-04 11:05:22.238 912346159533940 69.7500 2843 AT 7 2024-01-04 11:05:22.238 912346159533941 69.7400 2647 AT 8 2024-01-04 11:05:24.294 582380190048149616 69.7252 6485 OB 9 2024-01-04 11:05:36.145 912346159533960 69.7300 4715 AT TRADE_VENUE TICK_STATUS DELETED_TIME 0 XLON 0 1970-01-01 1 XLON 0 1970-01-01 2 XLON 0 1970-01-01 3 XLON 0 1970-01-01 4 SINT 0 1970-01-01 5 XLON 0 1970-01-01 6 XLON 0 1970-01-01 7 XLON 0 1970-01-01 8 XLON 0 1970-01-01 9 XLON 0 1970-01-01 ```