Keep the two clocks distinct
When a feed supplies an event timestamp, retain it separately from the time our collector receives the event. The event time helps reconstruct what happened at the source. Receipt time helps us reason about what the model could have observed at a given moment. Review delays and out-of-order events before defining features.
Identify repeated events
Use a source event identifier or sequence number where the feed provides one. Observations can share a price and timestamp, so deduplication needs to follow the feed’s documented event semantics. Keep a record of what was excluded from the research dataset.
Investigate gaps in the stream
Review discontinuities against market schedules, feed status and collector logs where available. Quiet trading and a disconnected feed can both produce gaps. Mark intervals with unresolved gaps so we can assess their effect on an experiment.
Record the decisions
For each reviewed dataset, record the source, covered time range, observed issues and exclusions. Keep that record with the dataset version used for training and evaluation so the experiment can be traced back to its inputs.