Corrections and Time Travel#

This section contains 5 examples for Corrections and Time Travel using the onetick-py.
Each example is a self-contained script that can be run against the OneTick Cloud sample databases.

# onetick-py WebAPI configuration for OneTick Cloud
import os
os.environ['OTP_WEBAPI'] = '1'
os.environ['OTP_HTTP_ADDRESS'] = 'https://rest.cloud.onetick.com'
os.environ['OTP_ACCESS_TOKEN_URL'] = 'https://cloud-auth.parent.onetick.com/realms/OMD/protocol/openid-connect/token'
os.environ['OTP_CLIENT_ID'] = '__FILL_IN__'
os.environ['OTP_CLIENT_SECRET'] = '__FILL_IN__'

Corrected Trade Retrieval#

Standard Trade Retrieval returns data adjusted for Trade Corrections.
Deleted Trades will not be visible.

import onetick.py as otp

data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD')
data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']]
data = data.limit(10)
result = otp.run(data,
                 start=otp.dt(2024, 1, 4, 11, 4, 0),
                 end=otp.dt(2024, 1, 6),
                 timezone='UTC',
                 symbols='VOD')
result

	Time	TRADE_ID	PRICE	SIZE	TRADE_TYPE	TRADE_VENUE	DELETED_TIME
0	2024-01-04 11:04:29.404	912346159533907	69.7400	4121	AT	XLON	1970-01-01
1	2024-01-04 11:04:29.428	912346159533908	69.7200	344	AT	XLON	1970-01-01
2	2024-01-04 11:04:29.431	912346159533909	69.7200	4715	AT	XLON	1970-01-01
3	2024-01-04 11:04:29.431	912346159533910	69.7200	2480	AT	XLON	1970-01-01
4	2024-01-04 11:04:57.354	730098958764564592	69.7200	555	OB	SINT	1970-01-01
5	2024-01-04 11:05:21.798	912346159533939	69.7300	340	AT	XLON	1970-01-01
6	2024-01-04 11:05:22.238	912346159533940	69.7500	2843	AT	XLON	1970-01-01
7	2024-01-04 11:05:22.238	912346159533941	69.7400	2647	AT	XLON	1970-01-01
8	2024-01-04 11:05:24.294	582380190048149616	69.7252	6485	OB	XLON	1970-01-01
9	2024-01-04 11:05:36.145	912346159533960	69.7300	4715	AT	XLON	1970-01-01

Hidden Records Including Corrections#

All Records including trade corrections can be retrieved by using show_hidden_ticks().
This propagates all ticks, even those with a TICK_STATUS not equal to 0, which are normally hidden.
Corrected Trades can be identified by their DELETED_TIME and TICK_STATUS fields.
DELETED_TIME corresponds to the time the record was corrected, which may be days after the original record.
TICK_STATUS refers to the type of change:

0 - Default
1 - Deleted record
2 - Updated record
3 - Insert Corrected
4 - Record that has been Canceled
5 - Record that has been Corrected
6 - New Correction record
7 - New Cancellation record

import onetick.py as otp

data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD')
data = data.show_hidden_ticks()
data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']]
data = data.limit(10)
result = otp.run(data,
                 start=otp.dt(2024, 1, 4, 11, 4, 0),
                 end=otp.dt(2024, 1, 6),
                 timezone='UTC',
                 symbols='VOD')
result

	Time	TRADE_ID	PRICE	SIZE	TRADE_TYPE	TRADE_VENUE	TICK_STATUS	DELETED_TIME
0	2024-01-04 11:04:04.096	577551547295817840	69.6300	344	OB	XLON	1	2024-01-05 10:15:22.417
1	2024-01-04 11:04:29.404	912346159533907	69.7400	4121	AT	XLON	0	1970-01-01 00:00:00.000
2	2024-01-04 11:04:29.428	912346159533908	69.7200	344	AT	XLON	0	1970-01-01 00:00:00.000
3	2024-01-04 11:04:29.431	912346159533909	69.7200	4715	AT	XLON	0	1970-01-01 00:00:00.000
4	2024-01-04 11:04:29.431	912346159533910	69.7200	2480	AT	XLON	0	1970-01-01 00:00:00.000
5	2024-01-04 11:04:57.354	730098958764564592	69.7200	555	OB	SINT	0	1970-01-01 00:00:00.000
6	2024-01-04 11:05:21.798	912346159533939	69.7300	340	AT	XLON	0	1970-01-01 00:00:00.000
7	2024-01-04 11:05:22.238	912346159533940	69.7500	2843	AT	XLON	0	1970-01-01 00:00:00.000
8	2024-01-04 11:05:22.238	912346159533941	69.7400	2647	AT	XLON	0	1970-01-01 00:00:00.000
9	2024-01-04 11:05:24.294	582380190048149616	69.7252	6485	OB	XLON	0	1970-01-01 00:00:00.000

Trade Corrections#

Trade corrections can be retrieved by using show_corrected_ticks().
Only corrected and correction ticks will be propagated.
Corrected Trades can be identified by their DELETED_TIME and TICK_STATUS fields.
DELETED_TIME corresponds to the time the record was corrected, which may be days after the original record.
TICK_STATUS refers to the type of change, in the example below:

4 - Record that has been Canceled
7 - New Cancellation record

import onetick.py as otp

data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD')
data = data.show_corrected_ticks()
data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']]
data = data.limit(10)
result = otp.run(data,
                 start=otp.dt(2024, 1, 4, 11, 4, 0),
                 end=otp.dt(2024, 1, 6),
                 timezone='UTC',
                 symbols='VOD')
result

	Time	TRADE_ID	PRICE	SIZE	TRADE_TYPE	TRADE_VENUE	TICK_STATUS	DELETED_TIME
0	2024-01-05 10:15:22.417	577551547295817840	69.63	344	OB	XLON	4	2024-01-04 11:04:04.096
1	2024-01-05 10:15:22.417	577551547295817840	69.63	344	OB	XLON	7	1970-01-01 00:00:00.000
2	2024-01-05 13:01:28.511	432561984411230320	70.00	7750	OB	XLON	4	2024-01-05 12:59:35.020
3	2024-01-05 13:01:28.511	432561984411230320	70.00	7750	OB	XLON	7	1970-01-01 00:00:00.000
4	2024-01-05 15:49:25.706	592575148552052848	70.08	5000	OB	XLON	4	2024-01-05 13:01:32.811
5	2024-01-05 15:49:25.706	592575148552052848	70.08	5000	OB	XLON	7	1970-01-01 00:00:00.000

Trades Before Correction#

Trade data can be retrieved as it was at a specific point in time using correct_tick_filter().

If the specified as_of_time is set before the trade corrections, uncorrected data is returned.
If the specified as_of_time is set after the trade corrections, corrected data is returned.

This provides a Time Travel capability, returning data before and after changes to the data occur.
Here the as_of_time is set to a date before the trade corrections, so uncorrected data is returned.

import onetick.py as otp

data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD')
data = data.correct_tick_filter(discard_on_match=False, as_of_time=otp.dt(2024, 1, 4))
data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']]
data = data.limit(10)
result = otp.run(data,
                 start=otp.dt(2024, 1, 4, 11, 4, 0),
                 end=otp.dt(2024, 1, 6),
                 timezone='UTC',
                 symbols='VOD')
result

	Time	TRADE_ID	PRICE	SIZE	TRADE_TYPE	TRADE_VENUE	TICK_STATUS	DELETED_TIME
0	2024-01-04 11:04:04.096	577551547295817840	69.6300	344	OB	XLON	1	2024-01-05 10:15:22.417
1	2024-01-04 11:04:29.404	912346159533907	69.7400	4121	AT	XLON	0	1970-01-01 00:00:00.000
2	2024-01-04 11:04:29.428	912346159533908	69.7200	344	AT	XLON	0	1970-01-01 00:00:00.000
3	2024-01-04 11:04:29.431	912346159533909	69.7200	4715	AT	XLON	0	1970-01-01 00:00:00.000
4	2024-01-04 11:04:29.431	912346159533910	69.7200	2480	AT	XLON	0	1970-01-01 00:00:00.000
5	2024-01-04 11:04:57.354	730098958764564592	69.7200	555	OB	SINT	0	1970-01-01 00:00:00.000
6	2024-01-04 11:05:21.798	912346159533939	69.7300	340	AT	XLON	0	1970-01-01 00:00:00.000
7	2024-01-04 11:05:22.238	912346159533940	69.7500	2843	AT	XLON	0	1970-01-01 00:00:00.000
8	2024-01-04 11:05:22.238	912346159533941	69.7400	2647	AT	XLON	0	1970-01-01 00:00:00.000
9	2024-01-04 11:05:24.294	582380190048149616	69.7252	6485	OB	XLON	0	1970-01-01 00:00:00.000

Trades After Correction#

Trade data can be retrieved as it was at a specific point in time using correct_tick_filter().

If the specified as_of_time is set before the trade corrections, uncorrected data is returned.
If the specified as_of_time is set after the trade corrections, corrected data is returned.

This provides a Time Travel capability, returning data before and after changes to the data occur.
Here the as_of_time is set to a date after the trade corrections, so corrected data is returned.

import onetick.py as otp

data = otp.DataSource(db='LSE_SAMPLE', tick_type='TRD')
data = data.correct_tick_filter(discard_on_match=False, as_of_time=otp.dt(2024, 1, 6))
data = data[['TRADE_ID', 'PRICE', 'SIZE', 'TRADE_TYPE', 'TRADE_VENUE', 'TICK_STATUS', 'DELETED_TIME']]
data = data.limit(10)
result = otp.run(data,
                 start=otp.dt(2024, 1, 4, 11, 4, 0),
                 end=otp.dt(2024, 1, 6),
                 timezone='UTC',
                 symbols='VOD')
result

	Time	TRADE_ID	PRICE	SIZE	TRADE_TYPE	TRADE_VENUE	DELETED_TIME
0	2024-01-04 11:04:29.404	912346159533907	69.7400	4121	AT	XLON	1970-01-01
1	2024-01-04 11:04:29.428	912346159533908	69.7200	344	AT	XLON	1970-01-01
2	2024-01-04 11:04:29.431	912346159533909	69.7200	4715	AT	XLON	1970-01-01
3	2024-01-04 11:04:29.431	912346159533910	69.7200	2480	AT	XLON	1970-01-01
4	2024-01-04 11:04:57.354	730098958764564592	69.7200	555	OB	SINT	1970-01-01
5	2024-01-04 11:05:21.798	912346159533939	69.7300	340	AT	XLON	1970-01-01
6	2024-01-04 11:05:22.238	912346159533940	69.7500	2843	AT	XLON	1970-01-01
7	2024-01-04 11:05:22.238	912346159533941	69.7400	2647	AT	XLON	1970-01-01
8	2024-01-04 11:05:24.294	582380190048149616	69.7252	6485	OB	XLON	1970-01-01
9	2024-01-04 11:05:36.145	912346159533960	69.7300	4715	AT	XLON	1970-01-01