SirNewton 3 Report post Posted February 22 Hello @Physicsman While processing Historical data, I saw something unusual that I need your help with. We know that the Steaming Data along with the market closes on FRIDAY 9:59PM GMT And, we know that the Steaming Data along with the market opens on SUNDAY 10:00PM GMT And, we know that Each "day" historical data covers from midnight (00:00) to 23:59 GMT But, I noticed that in the Historical Data that was outside that range. See Blow: AUD/JPY: ASK: {"BarDate": "2021-02-20 00:00:00", "Open": 83.062, "High": 83.062, "Low": 83.062, "Close": 83.062} BID: {"BarDate": "2021-02-20 00:00:00", "Open": 82.912, "High": 82.912, "Low": 82.912, "Close": 82.912} USD/CHF: ASK: {"BarDate": "2021-02-20 00:00:00", "Open": 0.89672, "High": 0.89672, "Low": 0.89672, "Close": 0.89672} BID: {"BarDate": "2021-02-20 00:00:00", "Open": 0.89607, "High": 0.89607, "Low": 0.89607, "Close": 0.89607} USD/JPY: ASK: {"BarDate": "2021-02-20 00:00:00", "Open": 105.514, "High": 105.514, "Low": 105.514, "Close": 105.514} BID: {"BarDate": "2021-02-20 00:00:00", "Open": 105.392, "High": 105.392, "Low": 105.392, "Close": 105.392} 2021-02-20 is a Saturday. The other pairs, I am processing had a BarDate of "2021-02-19 00:00:00", which is what I expected. Can you please explain the discrepancy of the BarDate of pairs AUD/JPY, USD/CHF, and USD/JPY? Thank you. Share this post Link to post
Physicsman 25 Report post Posted February 22 Hi SirNewton, I'll look into this for you and reply back with our findings. Kind Regards, PM Share this post Link to post
Physicsman 25 Report post Posted February 23 Hi SirNewton, According to the Pricing team, an erroneous price tick was sent after market close on Friday resulting in the Saturday price bar. The team are working on removing this from the chart database. Kind Regards, PM Share this post Link to post
SirNewton 3 Report post Posted February 23 Thank you. I looked through the full history and found similar out of bound entries. Will provide sqlscript used to find out of bound entries. Note: I use Postgres, but you should be able to do similar queries in other databases. In the DB datetime is "BarDate". Will give you Saturday and Sunday select * forex_history where extract(isodow from datetime)in(0,6,7) Or to see just Saturday select * forex_history where extract(isodow from datetime)=6 Share this post Link to post
SirNewton 3 Report post Posted February 23 "According to the Pricing team, an erroneous price tick was sent after market close on Friday resulting in the Saturday price bar" By the way, if those price ticks are actual and not generated by error. I rather you keep it in the history. I can write code to combine those with Friday historical. Share this post Link to post
Physicsman 25 Report post Posted February 24 Hi Sir Newton, If those are indeed actual price ticks the team will be able to correct it appropriately without losing the data. Kind Regards, PM Share this post Link to post