Skip to main content

How to handle order book flat files can go back in time?

It's expected behavior by design, we sometimes need to go back and failover to stream from another geographical site if we detect the issues with the feed. This mechanism will still exist in the future, however, we have assigned work to remove this backward jump in this or next quarter; it's not our priority at the moment as the impact is too low and it does not affect the quality of the data.

You should always process all items in the file in the order in which they are recorded in the file and never change this order.

When processing items according to the file order you will have the book which you can use for further processing. The current timestamp of the book is determined by the last processed line in the flat file (timestamps on this last line). 

In the event when the time going backward you should still normally process the file, but you can restrict supplying the book outside the flat-file processing algorithm until flat-file book time will be greater again than maximum seen timestamp; using this algorithm you will process the backward snapshot and corresponding updates to it transparently.

Take a look at the example source code that describes the algorithm above: https://github.com/coinapi/coinapi-sdk/blob/master/csharp-flatfiles-example/Program.cs