Jump to content
Sign in to follow this  
SirNewton

Streaming Data and Historical Data

Recommended Posts

Hello @Physicsman

While processing Historical data, I saw something unusual that I need your help with.

We know that the Steaming Data along with the market closes on FRIDAY 9:59PM GMT

And, we know that the Steaming Data along with the market opens on SUNDAY 10:00PM GMT

And, we know that Each "day" historical data covers from midnight (00:00) to 23:59 GMT

But, I noticed that in the Historical Data that was outside that range. See Blow:

AUD/JPY:
ASK: {"BarDate": "2021-02-20 00:00:00", "Open": 83.062, "High": 83.062, "Low": 83.062, "Close": 83.062}
BID: {"BarDate": "2021-02-20 00:00:00", "Open": 82.912, "High": 82.912, "Low": 82.912, "Close": 82.912}

USD/CHF:
ASK: {"BarDate": "2021-02-20 00:00:00", "Open": 0.89672, "High": 0.89672, "Low": 0.89672, "Close": 0.89672}
BID: {"BarDate": "2021-02-20 00:00:00", "Open": 0.89607, "High": 0.89607, "Low": 0.89607, "Close": 0.89607}

USD/JPY:
ASK: {"BarDate": "2021-02-20 00:00:00", "Open": 105.514, "High": 105.514, "Low": 105.514, "Close": 105.514}
BID: {"BarDate": "2021-02-20 00:00:00", "Open": 105.392, "High": 105.392, "Low": 105.392, "Close": 105.392}

2021-02-20 is a Saturday. The other pairs, I am processing had a BarDate of "2021-02-19 00:00:00", which is what I expected.

Can you please explain the discrepancy of the BarDate of pairs AUD/JPY, USD/CHF, and USD/JPY?

Thank you.

Share this post


Link to post

Hi SirNewton,

According to the Pricing team, an erroneous price tick was sent after market close on Friday resulting in the Saturday price bar. The team are working on removing this from the chart database.

Kind Regards, PM 

Share this post


Link to post

Thank you. I looked through the full history and found similar out of bound entries. Will provide sqlscript used to find out of bound entries.

Note: I use Postgres, but you should be able to do similar queries in other databases. In the DB datetime is "BarDate".

Will give you Saturday and Sunday

select * forex_history where extract(isodow from datetime)in(0,6,7) 

Or to see just Saturday

select * forex_history where extract(isodow from datetime)=6

Share this post


Link to post

"According to the Pricing team, an erroneous price tick was sent after market close on Friday resulting in the Saturday price bar"

By the way, if those price ticks are actual and not generated by error. I rather you keep it in the history. I can write code to combine those with Friday historical.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  
×