Find the answer to your question
Advanced Search
How can I retrieve all the items if there are more than 10k since pagination only allows a max of 10k in the search results?
Summary
Since the Finding API only allows a maximum of 10,000 items to be returned in the search (100 pages and 100 entries per page), you have to use filters to get back more items
Detailed Description
All of the Finding API's have the itemFilter. When searching for items you may run across a case where there are more than 10,000 items being returned. You will know this by looking at the total number on entries,
<totalEntries>29147</totalEntries>
If this has more than 10,000 entries, you will have to use the item filter to narrow down your results. As you can see, http://developer.ebay.com/DevZone/finding/CallRef/findItemsByCategory.html#Request.itemFilter, there are many filters you can use. I will show 1 example here using findItemsByCategory and the ModTimeFrom filter.
1. Basic query to pick up all the items. You can make a call similar to this and loop through the page numbers to pick up all the items. As you will see, there are more than 10.000 items returned. At the time I made this request, there were 29147 items returned. So you will have to use a filter to get all the items, please refer to #2 below.
http://svcs.ebay.com/services/search/FindingService/v1?OPERATION-NAME=findItemsByCategory&SERVICE-VERSION=1.11.0&SECURITY-APPNAME={YOUR APPID}&GLOBAL-ID=EBAY-GB&RESPONSE-DATA-FORMAT=XML&EST-PAYLOAD&categoryId=39642&paginationInput.entriesPerPage=100&paginationInput.pageNumber=1
2. Basic query to pick up all the items with a mod time. This just gives you a basic idea, but if you use some sort of time range (Mod or End times), and make the call consistently, you will pick up any items that have been modified. By default the way it works is it takes the ModTimeFrom you enter until the current time (all times are UTC).
http://svcs.ebay.com/services/search/FindingService/v1?OPERATION-NAME=findItemsByCategory&SERVICE-VERSION=1.11.0&SECURITY-APPNAME={YOUR APPID}&GLOBAL-ID=EBAY-GB&RESPONSE-DATA-FORMAT=XML&EST-PAYLOAD&categoryId=39642&paginationInput.entriesPerPage=100&paginationInput.pageNumber=1&itemFilter(0).name=ModTimeFrom&itemFilter(0).value=2012-08-01T00:00:00.000Z
Now #2 will also give you back more results that you can handle, so in order to get all the items initially, you will have to work through a few times ranges and check for duplicates items. For example items can't be listed for longer than 30 days. So you can go back 30 days to pick up the oldest items. From there move your time range forward always going through all 10,000 items until you get all items.
Additional Resources
- findItemsByCategory: http://developer.ebay.com/DevZone/finding/CallRef/findItemsByCategory.html