Receiving 408 HTTP code using the transactiondistributions endpoint

@Erik Leaver I've made several hundred requests to the endpoint already, slowing working through the retrieving the recordset. I've also just tried a direct request with postman, the timeout there and in my tool is infinite, but after about a minute it timed out. With what I've tried so far I wonder if the server itself it taking too long to respond, since that can also contribute to a 408 error. I've had a routine running for at least 10 hours overnight and maybe 10 times it worked and should have made hundreds of requests overnight.

@Daniel Maxwell I've tried the “same” request reducing the limit from 5,000 to 1,000 and it didn't work and then reduced to 100 and it didn't work via the “Try It” tool in the API Docs

@Erik Leaver I just asked for access. However, to be clear this was working. It also works when I do not put an offset in. When I provide an offset of say 4980000 or in the range of that it won't work.

@Erik Leaver Thank you. It almost seems like either the indexing isn't being used so response time is long because of table-scanning type techniques. Or perhaps with the “fixed indexing" there is another table at play that isn't playing nice. Without knowing the backend systems, and applying the knowledge I have of SQL databases, indexing and general programming, I'd guess that the server side query or such is just taking too long and my perception is that the 408 error is because the API server side is closing the connection due to perhaps a timeout on that side vs my side.

Thanks for keeping an eye on it. I really have to get this fixed as this is hampering us.

@Erik Leaver I actually did not use their suggestion. I continued to use the Transaction Distribution (List) API call and incorporated the from_date and to_date parameters and went after the data in yearly segments. For this set of tasks my goal was to replicate the data as it exists via the API. Next, I need to get an incremental tool in place for ongoing transactions.

@Alex Wong
I ended up having to use other criteria in my requests. I basically couldn't seem to do a sequential set of requests of our request to get a complete copy, starting from 1 to present. I hit somewhere around 4,000,000 records and then that was as far as I could get. The Team suggested that I use some calls to deal with things in batches, but I had alternated just before that to using the dates to limit my requests for a year of data at a time. Ultimately, I was able to get the historical data out of the API and stored locally and now I'm retrieving much smaller numbers of records.