Jump to content
jflaggs

Aggregating Historical Trades Via ApiTradeHistoryDTO

Recommended Posts

Hello, 

 

I am trying to aggregate my historical trades to calculate my average position price and size. To do this I need to know which trades are buy/sell to open (buy/sell to close can be calculated by tracking when size becomes zero). The only other alternative I can think of would be to request ALL history since the account was created and funded to ensure starting from a flat position. Of course, this is highly impractical when we only want to analyze the last month of historical trades. Below is an example simplification of a ApiTradeHistoryDTO[ ] array filtered by a single symbol.

As an example:

Sell 1 at 99 (closing a previous position) 

Buy 1 at 100 (opening new position, we need a bit to know to start here rather than the prior order)

Buy 1 at 101

Buy 1 at 102

Sell 1 at 103

Sell 1 at 104

Sell 1 at 105 (net position closed)

 

The average position is Long 3 at 101. The only way to calculate this is by knowing that the second order in the array was a buy to open order. Otherwise, we would mistake the first order for an opening order when it is actually a closing order. What is the recommended way to get this result with my historical trades? 

Share this post


Link to post

Hello Jflaggs,

Determing which trades are opening can be done by inference - check whether the RealisedPnl field is null. Closing trades all have a value for RealisedPnl, whereas an opening trade has null. 

Kind Regards, PM

Share this post


Link to post

The problem with that is all "scale in trades" have a realized pnl of null, and all "scale out" trades have a non-null realized pnl value. So you cannot distinguish which trade opened a position and which trades were subsequent scale in trades to that position. This is necessary to do things like, get the average price for a historical net position or do an MAE/MFE analysis on a historical net position. 

Perhaps a more concise way to ask this question is: How to know the transitions from Flat to Long/Short and vice versa using these historical trade objects?

Share this post


Link to post

Hi Jflaggs,

In answer to your question: 

Quote

How to know the transitions from Flat to Long/Short and vice versa using these historical trade objects?

 

Unfortunately, the answer is you cannot. The historical trade information does not contain anything on the account open position state prior to each trade. No information is given as to whether you were flat in the market or already long 2 for example.

The historical trade information is mainly: Buy/Sell placed in market X with quantity Y at price Z along with the time/date stamp for the transaction.    

Kind Regards, PM

Share this post


Link to post

Thank you for confirming this for me. Are you able to comment on why it is this way? It would only require a single bit "BuyToOpen = true/false" to achieve this. Could this be a feature request? 

 

Also, I am not able to get my entire trade history from this api call. I have actively traded this account for about 2-3 years, but my api call only returns trade history going back to May 20th of this year. Is this by design or is this an error on my part? If so, why -- and how can I retrieve my full trade history?

 

Here is the request string following the api reference here:

I have also tried the other method using the ClientAccountID and TradingAccountId:

Share this post


Link to post

Hi,

Not having buy/sell to open flag is not in the API because it was never needed before, or at least was never asked for by anyone until now. :)  I'll pass on this request to the API team whether they could add this in for a future update.

Your trade history request should look similar to what is shown below. Note the username and session authentication informatoin is supplied in the header. I've selected a From date 2 years ago of June 2018. 

https://ciapi.cityindex.com/TradingApi/order/tradehistory?MaxResults=10000&from=1529038800

image.png

Share this post


Link to post

That would be excellent. How can I track the status of that feature request and/or get an update when it's implemented?

 

I tried using your GET request string along with my session and username in headers via postman but the result is still the same. I get exactly 200 trades including my latest trade, but it will not return trades before May 2020. Any ideas for troubleshooting this? 

 

Request string https://ciapi.cityindex.com/TradingApi/order/tradehistory?MaxResults=10000&from=1529038800 + Session=*** & UserName=*** headers still only returns 200 results going back to may 2020. 

Share this post


Link to post

Hi,

Since the development backlog is an internal system only, clients won't be able to track what is on there. 

I should also note that the request still has to be evaluated by the backlog owner as to whether it is something the API team will implement - the possibility exists that it could be rejected. The other thing to note is that even if accepted, it will be prioritised against all the other items the team have scheduled currently. Given what they are currently working on, I'd estimate that if the request is accepted, it would be in the order of months rather than weeks.

On your account history retrieval, lets check that the account is storing the trade history information and something hasn't happened to it. 

  • Please login to our Web Trader platform using your account and open the Reports component from the Workspace menu.
  • Select the More History tab.
  • Enter a "Start" date say a year agao or longer, and leave the "end" date as today.
  • Click Search.

Does this procedure return all the trades/orders history for your account from the selected period?

image.png

Kind Regards, PM  

Share this post


Link to post

I gave this a try and discovered that my Web Trader report only yields 400 lines of trades.

 

I requested a year of trades up to today, and it returns 400 lines of trades in the date range of 26/05/2020 - 16/07/2020. So again, it cuts off in May but a few days ago it went back to May 14th or so -- which tells me it's only returning a fixed number of trades.

Then I requested last years trades (Jan 1 2019 to Jan 1 2020) and the Web Trader report yields 400 lines again spanning from 29/08/2019 - 24/10/2019. This is consistent with the results I got via Postman and the GCAPI, so it looks like the API "MaxResults" parameter is not working on the server side. 

Share this post


Link to post

Hi,

The API team want to check the interaction of your account with the API. Please contact Client Services who are able to take your account information and can pass that onto the team for investigation.

Thank you for your assistance. PM

Share this post


Link to post

I sent an email to api support two weeks ago per your instructions (and also sent a link to this discussion for reference). I just received a reply today:

"Please try to use monthly reports to view older information."

I'm not sure how to interpret this response to apply it to an api call. I believe they are referring to the web trader platform. Can you offer any interpretation on this reply?

It seems a bit impractical to have a 10 day lag in communication. Is there another channel to go through to get a more in depth response for this bug?

Share this post


Link to post

Hi,

Apologies for the long delay - it should not have taken that long to receive a response. It looks as though the responding agent did not understand the query correctly, and is indeed asking you to use the Web platform to view the full account information. :(

I'm sending you a private message with some contact information to progress this query further.

Kind Regards. PM

Share this post


Link to post

I am seeing the same behaviour from tradehistory.

When I make a call like this in C#:

string requestString = serverBaseURL + string.Format("/order/tradehistory?TradingAccountId={0}&maxResults=500&from={1}", tradingAccountId, unixTimestamp);

Where unixTimestamp = 1584955458 (equivalent to approx 23/3/2020)

I always get the last 200 trades. It appears that the "from" parameter is ignored. The trading account ID I'm using is correct as I got that from a call to /useraccount/ClientAndTradingAccount. And this code used to work as I previously used it to back-populate some trade history into an Excel sheet.

Kind Regards
Chris.

Share this post


Link to post

Hi chriss,

I have received a response from the API team.

The Order Gateway that is used by the API to retrieve trade/order history has a technical limitation in its implementation that restricts the maximum number of results you can get through it. That is why you were not able to retrieve all your history since you have more trades/orders than can be returned through this gateway. The first time you ran the query when you did receive your entire history, is likely to have been when your trades/orders were less than the maximum number that could be returned through the gateway.

The team are working on a new Reporting Gateway that will NOT have this limitation, but that is some way off at this time. 

Currently, the only way to see a full trade/order history of your account is to request a statement through client services. Statements use a different (non-API) service that directly queries the database.

It it not possible to receive a full history through the API at this time.

Kind Regards, PM

Share this post


Link to post

Hi PM

Many thanks for looking into this.

I may have not been clear in my bug report, for which I apologise. It's returning the last 200 trades as of the date when the query is made. It's ignoring the from date.

I can understand the limitation of 200 results and I could easily work around that. But I would have thought this was for each query not a limit on the entire trade history. If I ask for trades from 23/3/2020, I would expect the 200 trades from that date. The query appears to just be returning the last 200 trades back from today and ignoring the specified date. Surely this is a bug rather than a limitation?

Kind Regards

Chris.

Share this post


Link to post

Update: the team have created an internal ticket to track this investigation. Since I have the ticket number I can keep an eye on its progress and update here when I see a status change. 

Share this post


Link to post

Hi @chriss,

A quick question from the team. In your ListTradeHistory call, did you supply your TradingAccountId in the call parameters? If no account ID is supplied in the parameters the call defaults to using ClientAccountId. 

The From date is supported with the TradingAaccountID but not the ClientAaccountID version of the call. That might be why it is nor working for you, although we'll still invetigate whether there is a bug as well.

Kind Regards, PM

Share this post


Link to post

Hi PM

I did supply the TradingAccountID parameter. The C# code is:

string requestString = serverBaseURL + string.Format("/order/tradehistory?TradingAccountId={0}&maxResults=500&from={1}", tradingAccountId, unixTimestamp);

Kind Regards

Chris.

Share this post


Link to post
On 02/09/2020 at 10:57 AM, Physicsman said:

Update: the team have created an internal ticket to track this investigation. Since I have the ticket number I can keep an eye on its progress and update here when I see a status change. 

PM
Just wondering if there had been any progress on this issue?

Kind Regards

Chris.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×