I’m working on an application that’s dependent on getting data from a REST API, but the source (AirTable) limits cURL/http request to 100 rows, then returns an additional query parameter called ‘offset’ to help with pagination. I will then be synching that data to a local table. Last week’s Kscope21 presentations by Carsten Czarski and Moritz Klein both discussed API Pagination in the context of the API provided by www.themoviedb.org (TMDb).
Both used Carsten’s fixed-page-size plug-in, which can be found on Oracle’s APEX GitHub page (https://github.com/oracle/apex/blob/main/plugins/rest-source/fixed-page-size/plugin-plsql-code.sql),,) as the foundation for their presentations. The way pagination is handled by the TMDb API is a bit different from the source I am using, though. TMDb handles it by producing values for page number, total pages, and total available results, the JSON of which looks like this:
{
"page": 1,
"results":
[ARRAY OF RESULTS],
"total_pages": 23,
"total_results": 446
}
After the initial call, to fetch the next page of results, the plugin simply adds ‘&page=2’ to the end of the HTTP/cURL endpoint and makes the call again until the value for “page” is equal to the value of “total_pages”. Pretty slick.
The AirTable API handles pagination by producing a single value called ‘offset’, which looks like this:
{
"records":
[ARRAY OF RESULTS],
"offset": "stringofnonsense"
}
That offset value would similarly be placed at the end of the HTTP/URL endpoint (i.e. ‘&offset=stringofnonsense’) for the call to be made again until no offset value is returned. In essence, I have a theory of how to accomplish what I need done:
run the request
if an offset value is returned...
save the returned rows to a local table...
place offset string at the end of the URL and...
run request again
else, if an offset value is not returned, end.
My issue is that Carsten’s plug-in is 300 lines and uses a bunch of methods and procedures that I don’t really understand, and most likely don’t need. It sounds simple, but I was hoping some generous person out there might help me with some guidance. Thank you all. I’m a big fan.