Live article: https://www.storyblok.com/faq/how-to-load-more-than-100-stories-with-the-js-client
- FAQ
- How to load more than 100 stories with the JS client?
If you need to retrieve a list of stories or, in general, a list of elements managed by Storyblok (links, assets, etc.) you need to consider managing the pagination in case you will retrieve a huge number of elements.
The suggestion is to use the Open Source Storyblok JS client that provides you with some helpers and methods to manage the integration with the Storyblok APIs (Management API and Content Delivery API).
You can use the getAll()
method to retrieve all the stories.
The getAll()
method is very convenient because it automatically manages the pagination mechanism and allows you to use all the parameters for filtering the items.
A practical example of how to use the getAll()
method is in the article Generating an RSS Feed.
If you want to manage the pagination or understand how it works under the hood, you can use the get()
method.
Calling the get()
method, you can set the parameters per_page
and page
. With the per_page
parameter, you can define the page size in terms of the number of items (in this case, the number of stories) for each page. With the page
parameter, you can set the number of the page.
When you use the pagination with these two parameters, you will receive one important information in the HTTP headers: the total
header. The total
value represents the total number of items (across all the pages). With the total
parameter, you can calculate how many pages you need to retrieve, so how many API calls you need to call.
So, the strategy is that you can perform the first API call for the first page for retrieving the first set of elements and the total
information.
With the total
header, you can calculate if, within the first response, you have all the stories or if you need to perform other API calls incrementing the page
parameter.
At the end, you need to merge all the responses you collected across all the paginated API calls
The first step is to be sure you are using the Storyblok JS Client. You can install manually, or if you are using one of the Storyblok Frontend SDKs for example, for React, Nuxt, Vue, SvelteKit the Storyblok JS Client is a dependency of the SDKs.
// 001 Import the Storyblok JS Client
import StoryblokClient from "storyblok-js-client";
// 002 init the Stroyblok client with access token
const Storyblok = new StoryblokClient({
accessToken: "YOUR_PREVIEW_TOKEN",
cache: {
clear: "auto",
type: "memory",
},
});
// 003 setting some parameters, 25 as per_page parameter
const perPage = 25;
let params = {
version: "draft",
per_page: perPage,
page: 1,
};
// 004 retrieving the first page
let firstResponse = await Storyblok.get("cdn/stories", params);
// 005 calculating the needed pages via total information
let lastPage = firstResponse.total
? Math.ceil(firstResponse.total / perPage)
: 1;
// 006 retrieving the other pages (accessing to the data.stories)
let otherStories = [];
for (let currentPage = 2; currentPage <= lastPage; currentPage++) {
params.page = currentPage;
otherStories.push((await Storyblok.get("cdn/stories", params)).data.stories);
}
// 007 merging all the arrays
let allStories = [firstResponse.data.stories, ...otherStories].flat();
console.log(allStories);
console.log(allStories.length);
console.log(perPage);
console.log(firstResponse.total);
The code snippet above highlights the steps needed to retrieve multiple pages of data. Certainly, the javascript code can be optimized, but let's follow the basic steps needed:
001
importing the Storyblok JS ClientStoryblokClient
;002
initializing the Stroyblok client with the access token;003
setting parameters, thepage
and theper_page
parameters for the pagination;004
retrieving the first page to retrieve the first chunk of data and thetotal
information in the HTTP headers;005
calculating how many pages are needed via total information;006
retrieving the other pages (accessing thedata.stories
JSON attribute);007
merging all the arrays, obtaining one array with all the stories.
Some practical scenarios
Managing the rate limits
According to the limits defined here https://www.storyblok.com/docs/api/content-delivery/v2#topics/rate-limit you can perform some fine-tuning within the rateLimits
parameter. For example, if you want to increase the rate limits, in the StoryblokClient
initialization, you can set the rateLimit
parameter as shown in the following snippet.
const Storyblok = new StoryblokClient({
accessToken: "YOUR_PREVIEW_TOKEN",
cache: {
clear: "auto",
type: "memory",
},
rateLimit: 20,
});
Retrieving the stories published in the current month
This example allows you to understand how to combine the getAll()
method and the filtering on the published date. You can customize the code according to your needs:
const date = new Date();
let month = date.getMonth() + 1; // retrieving the current month (months are 0 index)
let year = date.getFullYear();
let stories = await Storyblok.getAll("cdn/stories", {
version: "published",
per_page: 25,
cv: "undefined",
published_at_gt: `${year}-${month}-20 00:00`,
});
As you can see, we are using the published_at_gt
parameter that acts on the published_at
attribute with the gt
operator (greater than).
You can see all the query parameters you can use, like
starts_with
,search_term
,content_type
, andpublished_at_lt
in the Content Delivery API documentation. You can also filter for the content with thefilter_query
parameter here.
Recap
Retrieving multiple stories or all the stories can be useful in multiple contexts when you need to retrieve multiple stories like:
- building your frontend with the
SSG
approach, and you need to access all the stories (maybe filtered via the published date); - building your RSS feed, you can read specific suggestions in the How to generate RSS feed article;
- building your sitemap and the information retrieved (or the filter you want to apply) from the
links
endpoint is not enough for your case. The article for building the sitemap using thelinks
endpoint is here; - integrating with an external system, and you need to retrieve the content from Storyblok.