Paginating a REST API as a data source

Paginate the results of a REST API request to limit the number of results.

Sometimes a backend returns a lot more information than your end-users need. Paginating the results enables you to limit the number of results returned. You can then return additional results in subsequent requests.

GraphQL specifies cursor-based pagination as the best practice for paginating through data to traverse relationships between sets of objects in a GraphQL API. A "cursor" is a type of pointer to the last item in the set of data, which is sent to the client so that the server can return the results after it. Cursor pagination returns a specified number of results per request, relative to a cursor in the result set managed by the backend API.

GraphQL uses nodes and edges to implement cursors. A node is a group of data. An edge represents the connection between two nodes, and consists of the edge object, the underlying node object, and the cursor.

Configuring pagination

API Connect Essentials allows you to use three different methods of pagination, based on what's supported by your Backend API.

Enable and configure pagination in API Connect for GraphQL by adding pagination to your @rest directive and configuring its two properties:

  • type: Specifies the type of pagination to use:
    • PAGE_NUMBER: Returns a "page" of results based on the number of results-per-page that you specify.
    • OFFSET: Returns results from a zero-based index into the result set. In each request, you can specify the number of results to return.
    • NEXT_CURSOR: Returns a set of results from a "cursor" location managed by the backend API. In each request you can specify the number of results to return.
  • setters: Specifies the total number of results or pages based on the pagination type. Specify the setter using the format shown in Table 1.
    Table 1. The setters format for each pagination type
    Type Setters
    PAGE_NUMBER [{field:"total" path: "meta.total_pages"}]
    OFFSET [{field:"total" path: "meta.total_count"}]
    NEXT_CURSOR [{field:"nextCursor" path: "meta.next"}]

You then add arguments to the endpoint to specify the starting page/result and number of results to return. These arguments vary depending on the pagination type as shown in Table 2:

Table 2. Starting page/result and number of results formats for each type of pagination
Type Starting Page/Result Number of Results to Return
PAGE_NUMBER page=$after per_page=$first
OFFSET offset=$after limit=$first
NEXT_CURSOR offset=$after limit=$first

The following example shows a pagination object within an @rest directive:

type User {
    id: ID!
    email: String!
    ...
  }

type UserEdge {
  node: User
  cursor: String
}

type UserConnection {
  pageInfo: PageInfo!
  edges: [UserEdge]
}

  ...

type Query {
  user(id: ID!): User
    @rest(endpoint: "https://reqres.in/api/users/$id", resultroot: "data")
  users(first: Int! = 6, after: String! = ""): UserConnection
    @rest(
      endpoint: "https://reqres.in/api/users?page=$after&per_page=$first"
      resultroot: "data[]"
      pagination: {
        type: PAGE_NUMBER
        setters: [{ field: "total", path: "total_pages" }]
      }
    )
}

Implementing pagination

Whether functionality is cursor-based, offset, or by page number, it is implemented using the following types.

type Customer {
  activities: [Activity]
  addresses: [Address]
  contacts: Contacts
  description: String
  designation: String
}

type CustomerEdge {
  node: Customer
  cursor: String
}

type CustomerConnection {
  pageInfo: PageInfo!
  edges: [CustomerEdge]
}

The type CustomerEdge takes the initial type Customer as its node. Then, type CustomerConnection takes type CustomerEdge as its edge field value. You might wonder what role the pageInfo field is playing here. The PageInfo value provided to it is returned by the server with information to assist with pagination.

That means that a query like the following will return data to help the user paginate the API.

query MyQuery {
  customers {
    pageInfo {
      endCursor
      hasNextPage
      hasPreviousPage
      startCursor
    }
  }
}

In this example, the data returned is under the customers object.

{
  "data": {
    "customers": {
      "pageInfo": {
        "endCursor": "eyJjIjoiTzpRdWVyeTpwYXJrcyIsIm8iOjE5fQ==",
        "hasNextPage": true,
        "hasPreviousPage": false,
        "startCursor": ""
      }
    }
  }
}
Page number pagination
Page number pagination returns one page of results for each request. The same number of results are returned each time (unless the number of results left to return is less than the number requested).

The following example shows how to use page number pagination:

customers(first:Int! =20 after:String! =""): CustomerConnection
  @rest(
    endpoint:"https://api.example.com/customers?page=$after&per_page=$first"
    resultroot:"data[]"
    pagination: {
        type: PAGE_NUMBER
        setters: [{field:"total" path: "meta.total_pages"}]
      }
    )

The following key aspects are shown in the example:

  • after is set to an empty string or null for the first request. This indicates that it's the first request for results.
  • The initial value for first is set to 20 to indicate that 20 edges (results) are to be returned per page. first must be set to the same value in subsequent requests.

The following properties must be set in subsequent requests:

  • The value for first must be the same from the initial request (20 in the preceding example).
  • after must be set to the next page of results to return (for example, 2 for page two). To get the next page of results, set the value to connection.pageInfo.endCursor from the previous request.
  • The virtual field total must be set to the value of pagination.setters from the previous response. This specifies the total number of groups in the result set.

The opaque cursor after argument is unpacked to contain the backend API service's page number integer value when used in the context of @rest (that is, as $after in endpoint). The page number is one-based, so the first edge in the paged set will be from page 1.

Offset pagination
Offset pagination returns a specified number of results per request, relative to a zero-based index from the start of the result set.

The following example shows how to use offset pagination:

customers(first:Int! =20 after:String! =""): CustomerConnection
  @rest(
    endpoint:"https://api.example.com/customers?limit=$first&offset=$after"
    resultroot:"data[]"
    pagination: {
        type: OFFSET
        setters: [{field:"total" path: "meta.total_count"}]
      }
    )

The following key aspects are shown in the example:

  • after is set to an empty string or null for the first request. This indicates that it's the first request for results.

The following must be set in subsequent requests:

  • The value for first can be any number of results to return.
  • The value for after is set to the opaque cursor value of connection.pageInfo.endCursor of the previous request to get the next group of results.
  • The virtual field total must be set to the value of pagination.setters from the previous response. This specifies the total number of groups in the result set.

The opaque cursor after argument is unpacked to contain the backend API service's page number integer value when used in the context of @rest (e.g. as $after in endpoint). The offset is zero-based, so the first edge in the paged set has offset zero.

Cursor pagination

Here is an example of a GraphQL query that uses an edge.

query MyQuery {
  customers(first: 3, after: "eyJjIjoiTzpRdWVyeTpwYXJrcyIsIm8iOjl9") {
    edges {
      node {
        id
        customerCode
      }
    }
  }
}

The first argument specifies that only the first three API responses should be returned. The after argument specifies a starting cursor position. The preceding GraphQL query returns the first 3 responses after the cursor location "eyJjIjoiTzpRdWVyeTpwYXJrcyIsIm8iOjl9".