LDAP paged results: LDAP Control Extension for Simple Paged Results Manipulation
An LDAP control that lets a client fetch a large search result set one page at a time instead of all at once.
Weider, C., Herron, A., Anantha, A., and T. Howes, "LDAP Control Extension for Simple Paged Results Manipulation", RFC 2696, DOI 10.17487/RFC2696, September 1999, <https://www.rfc-editor.org/info/rfc2696>.
@misc{rfc2696,
series = {Request for Comments},
number = 2696,
howpublished = {RFC 2696},
publisher = {RFC Editor},
doi = {10.17487/RFC2696},
url = {https://www.rfc-editor.org/info/rfc2696},
author = {C. Weider and A. Herron and A. Anantha and T. Howes},
title = {{LDAP Control Extension for Simple Paged Results Manipulation}},
pagetotal = 7,
year = 1999,
month = sep,
} RFC text
Contents
Network Working Group C. Weider
Request for Comments: 2696 A. Herron
Category: Informational A. Anantha
Microsoft
T. Howes
Netscape
September 1999
LDAP Control Extension for Simple Paged Results Manipulation
Status of this Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (1999). All Rights Reserved.
1. Abstract
This document describes an LDAPv3 control extension for simple paging
of search results. This control extension allows a client to control
the rate at which an LDAP server returns the results of an LDAP
search operation. This control may be useful when the LDAP client has
limited resources and may not be able to process the entire result
set from a given LDAP query, or when the LDAP client is connected
over a low-bandwidth connection. Other operations on the result set
are not defined in this extension. This extension is not designed to
provide more sophisticated result set management.
The key words "MUST", "SHOULD", and "MAY" used in this document are
to be interpreted as described in [bradner97].
2. The Control
This control is included in the searchRequest and searchResultDone
messages as part of the controls field of the LDAPMessage, as defined
in Section 4.1.12 of [LDAPv3]. The structure of this control is as
follows:
pagedResultsControl ::= SEQUENCE {
controlType 1.2.840.113556.1.4.319,
criticality BOOLEAN DEFAULT FALSE,
controlValue searchControlValue
}
The searchControlValue is an OCTET STRING wrapping the BER-encoded
version of the following SEQUENCE:
realSearchControlValue ::= SEQUENCE {
size INTEGER (0..maxInt),
-- requested page size from client
-- result set size estimate from server
cookie OCTET STRING
}
3. Client-Server Interaction
An LDAP client application that needs to control the rate at which
results are returned MAY specify on the searchRequest a
pagedResultsControl with size set to the desired page size and cookie
set to the zero-length string. The page size specified MAY be greater
than zero and less than the sizeLimit value specified in the
searchRequest.
If the page size is greater than or equal to the sizeLimit value, the
server should ignore the control as the request can be satisfied in a
single page. If the server does not support this control, the server
MUST return an error of unsupportedCriticalExtension if the client
requested it as critical, otherwise the server SHOULD ignore the
control. The remainder of this section assumes the server does not
ignore the client's pagedResultsControl.
Each time the server returns a set of results to the client when
processing a search request containing the pagedResultsControl, the
server includes the pagedResultsControl control in the
searchResultDone message. In the control returned to the client, the
size MAY be set to the server's estimate of the total number of
entries in the entire result set. Servers that cannot provide such an
estimate MAY set this size to zero (0). The cookie MUST be set to an
empty value if there are no more entries to return (i.e., the page of
search results returned was the last), or, if there are more entries
to return, to an octet string of the server's choosing,used to resume
the search.
The client MUST consider the cookie to be an opaque structure and
make no assumptions about its internal organization or value. When
the client wants to retrieve more entries for the result set, it MUST
send to the server a searchRequest with all values identical to the
initial request with the exception of the messageID, the cookie, and
optionally a modified pageSize. The cookie MUST be the octet string
on the last searchResultDone response returned by the server.
Returning cookies from previous searchResultDone responses besides
the last one is undefined, as the server implementation may restrict
cookies from being reused.
The server will then return the next set of results from the whole
result set. This interaction will continue until the client has
retrieved all the results, in which case the cookie in the
searchResultDone field will be empty, or until the client abandons
the search sequence as described below. Once the paged search
sequence has been completed, the cookie is no longer valid and MUST
NOT be used.
A sequence of paged search requests is abandoned by the client
sending a search request containing a pagedResultsControl with the
size set to zero (0) and the cookie set to the last cookie returned
by the server. A client MAY use the LDAP Abandon operation to
abandon one paged search request in progress, but this is discouraged
as it MAY invalidate the client's cookie.
If, for any reason, the server cannot resume a paged search operation
for a client, then it SHOULD return the appropriate error in a
searchResultDone entry. If this occurs, both client and server should
assume the paged result set is closed and no longer resumable.
A client may have any number of outstanding search requests pending,
any of which may have used the pagedResultsControl. A server
implementation which requires a limit on the number of outstanding
paged search requests from a given client MAY either return
unwillingToPerform when the client attempts to create a new paged
search request, or age out an older result set. If the server
implementation ages out an older paged search request, it SHOULD
return "unwilling to perform" if the client attempts to resume the
paged search that was aged out.
A client may safely assume that all entries that satisfy a given
search query are returned once and only once during the set of paged
search requests/responses necessary to enumerate the entire result
set, unless the result set for that query has changed since the
searchRequest starting the request/response sequence was processed.
In that case, the client may receive a given entry multiple times
and/or may not receive all entries matching the given search
criteria.
4. Example
The following example illustrates the client-server interaction
between a client doing a search requesting a page size limit of 3.
The entire result set returned by the server contains 5 entries.
Lines beginning with "C:" indicate requests sent from client to
server. Lines beginning with "S:" indicate responses sent from server
to client. Lines beginning with "--" are comments to help explain the
example.
-- Client sends a search request asking for paged results
-- with a page size of 3.
C: SearchRequest + pagedResultsControl(3,"")
-- Server responds with three entries plus an indication
-- of 5 total entries in the search result and an opaque
-- cooking to be used by the client when retrieving subsequent
-- pages.
S: SearchResultEntry
S: SearchResultEntry
S: SearchResultEntry
S: SearchResultDone + pagedResultsControl(5, "opaque")
-- Client sends an identical search request (except for
-- message id), returning the opaque cooking, asking for
-- the next page.
C: SearchRequest + PagedResultsControl(3, "opaque")
-- Server responds with two entries plus an indication
-- that there are no more entries (null cookie).
S: SearchResultEntry
S: SearchResultEntry
S: SearchResultDone + pagedResultsControl(5,"")
5. Relationship to X.500
For LDAP servers providing a front end to X.500 (93) directories, the
paged results control defined in this document may be mapped directly
onto the X.500 (93) PagedResultsRequest defined in X.511 [x500]. The
size parameter may be mapped onto pageSize. The cookie parameter may
be mapped onto queryReference. The sortKeys and reverse fields in
the X.500 PagedResultsRequest are excluded.
6. Security Considerations
Server implementors should consider the resources used when clients
send searches with the simple paged control, to ensure that a
client's misuse of this control does not lock out other legitimate
operations.
Servers implementations may enforce an overriding sizelimit, to
prevent the retrieval of large portions of a publically-accessible
directory.
Clients can, using this control, determine how many entries match a
particular filter, before the entries are returned to the client.
This may require special processing in servers which perform access
control checks on entries to determine whether the existence of the
entry can be disclosed to the client.
7. References
[LDAPv3] Wahl, M., Howes, T. and S. Kille, "Lightweight Directory
Access Protocol (v3)", RFC 2251, December 1997.
[Bradner97] Bradner, S., "Key Words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
8. Authors' Addresses
Chris Weider
Microsoft Corp.
1 Microsoft Way
Redmond, WA 98052
USA
Phone: +1 425 882-8080
EMail: cweider@microsoft.com
Andy Herron
Microsoft Corp.
1 Microsoft Way
Redmond, WA 98052
USA
Phone: +1 425 882-8080
EMail: andyhe@microsoft.com
Anoop Anantha
Microsoft Corp.
1 Microsoft Way
Redmond, WA 98052
USA
Phone: +1 425 882-8080
EMail: anoopa@microsoft.com
Tim Howes
Netscape Communications Corp.
501 E. Middlefield Road
Mountain View, CA 94043
USA
Phone: +1 415 937-2600
EMail: howes@netscape.com
9. Full Copyright Statement
Copyright (C) The Internet Society (1999). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.
ISO Plain Language
On this page
§1The Control
RFC 2696 describes an LDAPv3 control extension for simple paging of search results. The control lets a client control the rate at which an LDAP server returns the results of a search operation. It may be useful when the LDAP client has limited resources and cannot process the entire result set from a given query, or when the client is connected over a low-bandwidth connection. Other operations on the result set are not defined, and the extension is not designed to provide more sophisticated result set management.
The control is included in the searchRequest and searchResultDone messages, as part of the controls field of the LDAPMessage. Its structure is a SEQUENCE of three parts: a control type, whose value is the object identifier 1.2.840.113556.1.4.319; a criticality flag, a BOOLEAN that defaults to false; and a control value.
The control value is an octet string wrapping the BER-encoded version of a SEQUENCE with two fields. The size field is an integer with the range 0 to maxInt: on the request it carries the page size asked for by the client, and on the response it carries the server's estimate of the result set size. The cookie field is an octet string, empty on the first request and carrying the server's token on later ones.
The control is a small structure added to two of LDAP's messages. It holds a number, which is the page size the client wants, and a token, which is empty on the first request. The server fills the number in with its own estimate of how many entries match, and fills the token in with something the client must send back to continue.
pagedResultsControl ::= SEQUENCE {
controlType 1.2.840.113556.1.4.319,
criticality BOOLEAN DEFAULT FALSE,
controlValue searchControlValue
}
realSearchControlValue ::= SEQUENCE {
size INTEGER (0..maxInt),
-- requested page size from client
-- result set size estimate from server
cookie OCTET STRING
}§3Client-Server Interaction
A client that needs to control the rate at which results are returned MAY specify on the searchRequest a paged results control with the size set to the desired page size and the cookie set to the zero-length string. The page size MAY be greater than zero and less than the size limit value specified in the request.
If the page size is greater than or equal to the size limit, the server should ignore the control, because the request can then be satisfied in a single page. If the server does not support the control, it MUST return an error of unsupportedCriticalExtension if the client requested it as critical, and otherwise the server SHOULD ignore the control.
Each time the server returns a set of results, it includes the control in the searchResultDone message. The size MAY be set to the server's estimate of the total number of entries in the entire result set, and a server that cannot provide such an estimate MAY set it to zero. The cookie MUST be set to an empty value if there are no more entries to return, that is, if the page just returned was the last, and otherwise it is an octet string of the server's choosing, used to resume the search.
The client MUST consider the cookie to be an opaque structure and make no assumptions about its internal organization or value. When it wants more entries, it MUST send a searchRequest with all values identical to the initial request, except for the messageID, the cookie, and optionally a modified page size. The cookie MUST be the octet string from the last searchResultDone response. Returning cookies from earlier responses is undefined, because the server implementation may restrict cookies from being reused.
The exchange continues until the client has retrieved all the results, at which point the cookie in the searchResultDone field is empty, or until the client abandons the search sequence. Once the paged search sequence has been completed, the cookie is no longer valid and MUST NOT be used. A sequence of paged search requests is abandoned by sending a search request with the size set to zero and the cookie set to the last cookie returned by the server. A client MAY use the LDAP Abandon operation to abandon one paged search request in progress, but this is discouraged because it MAY invalidate the client's cookie.
If the server cannot resume a paged search operation for a client, it SHOULD return the appropriate error in a searchResultDone entry, and both client and server should then assume the paged result set is closed and no longer resumable. A client may have any number of outstanding search requests pending, any of which may have used the control. A server that requires a limit on the number of outstanding paged search requests from a given client MAY either return unwillingToPerform when the client attempts to create a new one, or age out an older result set, in which case it SHOULD return unwilling to perform if the client tries to resume the aged-out search.
A client may safely assume that all entries satisfying a given search query are returned once and only once during the set of requests and responses needed to enumerate the entire result set, unless the result set for that query has changed since the request that started the sequence was processed. In that case the client may receive a given entry multiple times, and may not receive all the entries matching the criteria.
The client sends the search with a page size. The server sends back that many entries plus a token, and the client sends the token back with an otherwise identical search to get the next page. The server chooses the token and the client must treat it as an opaque string it cannot inspect or modify. When the last page has been sent the token is empty, and the client must not send an empty or old token again because the server may have forgotten it.
- searchRequest with pagedResultsControl(3, "")
- searchResultEntry
- searchResultDone with pagedResultsControl(5, "opaque")
- searchRequest with pagedResultsControl(3, "opaque")
- searchResultDone with pagedResultsControl(5, "")An empty cookie means there are no more entries.
§4The Example Exchange
The memo's example shows a client doing a search with a page size limit of 3, against a result set of 5 entries. Lines beginning with C are requests sent from the client to the server, lines beginning with S are responses sent from the server to the client, and lines beginning with two dashes are explanatory comments.
The client first sends a search request with a paged results control of (3, an empty cookie). The server responds with three search result entries, then a searchResultDone carrying a paged results control of (5, an opaque token). The 5 is the server's indication of the total number of entries in the search result, and the opaque token is what the client must send back when it wants the next page.
The client then sends an identical search request, except for the message id, carrying the opaque token and asking for the next page. The server responds with two more entries and a searchResultDone with a paged results control of (5, an empty cookie), which says there are no more entries to return.
In the example the client asks for three entries at a time from a result set of five. The first reply brings three entries and a token. The client sends the token back and gets the last two entries, this time with an empty token, which means the search is over.
§5Relationship to X.500
For LDAP servers providing a front end to X.500 directories, the paged results control defined in this document may be mapped directly onto the X.500 PagedResultsRequest. The size parameter may be mapped onto pageSize, and the cookie parameter may be mapped onto queryReference. The sortKeys and reverse fields of the X.500 PagedResultsRequest are excluded.
For a directory server that also speaks the older X.500 protocol, the two mechanisms line up field for field: the page size maps to pageSize and the token maps to queryReference. Only the sorting options of the X.500 version have no equivalent here.
§6Security Considerations
Server implementors should consider the resources used when clients send searches with the simple paged control, to ensure that a client's misuse of the control does not lock out other legitimate operations. Server implementations may enforce an overriding size limit, to prevent the retrieval of large portions of a publicly accessible directory.
The memo also notes a side effect of the design. Clients can, using this control, determine how many entries match a particular filter before the entries are returned to the client. This may require special processing in servers which perform access control checks on entries, to determine whether the existence of the entry can be disclosed to the client.
Two points matter for a server operator. A client can use paging to make the server work hard, so the server should keep a limit on how much work one client can ask for. And the size the server reports back is a count of matching entries, which can reveal that an entry exists even when the client is not allowed to read it, so a server that filters entries by access rights has to think about that before it reports a count.