otp.Source.modify_query_times#

Source.modify_query_times(start=None, end=None, output_timestamp=None, propagate_heartbeats=True, inplace=False)#

Modify start and end time of the query.

query times are changed for all operations only before this method up to the source of the graph.
all ticks’ timestamps produced by this method must fall into original time range of the query.

It is possible to change ticks’ timestamps with parameter output_timestamp, so they will stay inside the original time range.

Parameters

start (onetick.py.datetime or MetaFields or Operation) – Expression to replace query start time. By default, start time is not changed. Note that expression in this parameter can’t depend on ticks, thus only MetaFields and constants can be used.
end (onetick.py.datetime or MetaFields or Operation) – Expression to replace query end time. By default, end time is not changed. Note that expression in this parameter can’t depend on ticks, thus only MetaFields and constants can be used.
output_timestamp (onetick.py.Operation) – Expression that produces timestamp for each tick. By default, the following expression is used: orig_start + orig_timestamp - start This expression covers cases when start time of the query is changed and keeps timestamp inside original time range. Note that it doesn’t cover cases, for example, if end time was increased, you have to handle such cases yourself.
propagate_heartbeats (bool) – Controls heartbeat propagation.
inplace (bool) – The flag controls whether operation should be applied inplace or not. If inplace=True, then it returns nothing. Otherwise method returns a new modified object.

Return type

Source or None

Note

Due to how OneTick works internally, tick generators otp.Tick and otp.Ticks are not affected by this method.

Examples

>>> start = otp.dt(2022, 3, 2)
>>> end = otp.dt(2022, 3, 2) + otp.Milli(3)
>>> data = otp.DataSource('NYSE_TAQ', symbols='AAPL', tick_type='TRD')

By default, method does nothing:

>>> t = data.modify_query_times()
>>> otp.run(t, start=start, end=end)
                     Time  PRICE  SIZE
0 2022-03-02 00:00:00.000    1.0   100
1 2022-03-02 00:00:00.001    1.1   101
2 2022-03-02 00:00:00.002    1.2   102

See how _START_TIME and _END_TIME meta fields are changed. They are changed before modify_query_times:

>>> t = data.copy()
>>> t['S_BEFORE'] = t['_START_TIME']
>>> t['E_BEFORE'] = t['_END_TIME']
>>> t = t.modify_query_times(start=t['_START_TIME'] + otp.Milli(1),
...                          end=t['_END_TIME'] - otp.Milli(1))
>>> t['S_AFTER'] = t['_START_TIME']
>>> t['E_AFTER'] = t['_END_TIME']
>>> otp.run(t, start=start, end=end)
        Time  PRICE  SIZE                S_BEFORE                E_BEFORE    S_AFTER                 E_AFTER
0 2022-03-02    1.1   101 2022-03-02 00:00:00.001 2022-03-02 00:00:00.002 2022-03-02 2022-03-02 00:00:00.003

You can decrease time interval without problems:

>>> t = data.modify_query_times(start=data['_START_TIME'] + otp.Milli(1),
...                             end=data['_END_TIME'] - otp.Milli(1))
>>> otp.run(t, start=start, end=end)
        Time  PRICE  SIZE
0 2022-03-02    1.1   101

Note that the timestamp of the tick was changed with default expression. In this case we can output original timestamps, because they fall into original time range:

>>> t = data.modify_query_times(start=data['_START_TIME'] + otp.Milli(1),
...                             end=data['_END_TIME'] - otp.Milli(1),
...                             output_timestamp=data['TIMESTAMP'])
>>> otp.run(t, start=start, end=end)
                     Time  PRICE  SIZE
0 2022-03-02 00:00:00.001    1.1   101

But it will not work if new time range is wider than original:

>>> t = data.modify_query_times(start=data['_START_TIME'] - otp.Milli(1),
...                             output_timestamp=data['TIMESTAMP'])
>>> otp.run(t, start=start + otp.Milli(1), end=end + otp.Milli(1)) 
Traceback (most recent call last):
Exception...timestamp is falling out of initial start/end time range...

In this case default output_timestamp expression would work just fine:

>>> t = data.modify_query_times(start=data['_START_TIME'] - otp.Milli(1))
>>> otp.run(t, start=start + otp.Milli(1), end=end + otp.Milli(1))
                     Time  PRICE  SIZE
0 2022-03-02 00:00:00.001    1.0   100
1 2022-03-02 00:00:00.002    1.1   101
2 2022-03-02 00:00:00.003    1.2   102

But it doesn’t work, for example, if end time has crossed the borders of original time range. In this case other output_timestamp expression must be specified:

>>> t = data.modify_query_times(
...     start=data['_START_TIME'] - otp.Milli(2),
...     output_timestamp=otp.math.min(data['TIMESTAMP'] + otp.Milli(2), data['_END_TIME'])
... )
>>> otp.run(t, start=start + otp.Milli(2), end=end)
                     Time  PRICE  SIZE
0 2022-03-02 00:00:00.002    1.0   100
1 2022-03-02 00:00:00.003    1.1   101
2 2022-03-02 00:00:00.003    1.2   102

Remember that start and end parameters can’t depend on ticks:

>>> t = data.copy()
>>> t['X'] = 12345
>>> t = t.modify_query_times(start=t['_START_TIME'] + t['X'] - t['X'],
...                          end=t['_END_TIME'] - otp.Milli(1))
>>> otp.run(t, start=start, end=end) 
Traceback (most recent call last):
Exception...parameter must not depend on ticks...

Constant datetime values can be used as parameters too:

>>> t = data.modify_query_times(start=start + otp.Milli(1),
...                             end=end - otp.Milli(1))
>>> otp.run(t, start=start, end=end)
        Time  PRICE  SIZE
0 2022-03-02    1.1   101

Note that some graph patterns are not allowed when using this method. For example, modifying query times for a branch that will be merged later:

>>> t1, t2 = data[data['PRICE'] > 1.3]
>>> t2 = t2.modify_query_times(start=start + otp.Milli(1))
>>> t = otp.merge([t1, t2])
>>> otp.run(t, start=start, end=end) 
Traceback (most recent call last):
Exception...Invalid graph...time bound to a node...an intermediate node in one of the cycles in graph...