Moving DIGEST-MD5 to Historic
The case against DIGEST-MD5, the SASL mechanism that this document moves to Historic.
Melnikov, A., "Moving DIGEST-MD5 to Historic", RFC 6331, DOI 10.17487/RFC6331, July 2011, <https://www.rfc-editor.org/info/rfc6331>.
@misc{rfc6331,
series = {Request for Comments},
number = 6331,
howpublished = {RFC 6331},
publisher = {RFC Editor},
doi = {10.17487/RFC6331},
url = {https://www.rfc-editor.org/info/rfc6331},
author = {A. Melnikov},
title = {{Moving DIGEST-MD5 to Historic}},
pagetotal = 6,
year = 2011,
month = jul,
} RFC text
Contents
Internet Engineering Task Force (IETF) A. Melnikov
Request for Comments: 6331 Isode Limited
Obsoletes: 2831 July 2011
Category: Informational
ISSN: 2070-1721
Moving DIGEST-MD5 to Historic
Abstract
This memo describes problems with the DIGEST-MD5 Simple
Authentication and Security Layer (SASL) mechanism as specified in
RFC 2831. It marks DIGEST-MD5 as OBSOLETE in the IANA Registry of
SASL mechanisms and moves RFC 2831 to Historic status.
Status of This Memo
This document is not an Internet Standards Track specification; it is
published for informational purposes.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are a candidate for any level of Internet
Standard; see Section 2 of RFC 5741.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc6331.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other
than English.
Table of Contents
1. Introduction and Overview . . . . . . . . . . . . . . . . . . 2
2. Security Considerations . . . . . . . . . . . . . . . . . . . 5
3. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 5
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 5
5. References . . . . . . . . . . . . . . . . . . . . . . . . . 5
5.1. Normative References . . . . . . . . . . . . . . . . . . . . 5
5.2. Informative References . . . . . . . . . . . . . . . . . . . 5
1. Introduction and Overview
[RFC2831] defines how HTTP Digest Authentication [RFC2617] can be
used as a Simple Authentication and Security Layer (SASL) [RFC4422]
mechanism for any protocol that has a SASL profile. It was intended
both as an improvement over CRAM-MD5 [RFC2195] and as a convenient
way to support a single authentication mechanism for web, email, the
Lightweight Directory Access Protocol (LDAP), and other protocols.
While it can be argued that it is an improvement over CRAM-MD5, many
implementors commented that the additional complexity of DIGEST-MD5
makes it difficult to implement fully and securely.
Below is an incomplete list of problems with the DIGEST-MD5 mechanism
as specified in [RFC2831]:
1. The mechanism has too many options and modes. Some of them are
not well described and are not widely implemented. For example,
DIGEST-MD5 allows the "qop" directive to contain multiple values,
but it also allows for multiple qop directives to be specified.
The handling of multiple options is not specified, which results
in minor interoperability problems. Some implementations
amalgamate multiple qop values into one, while others treat
multiple qops as an error. Another example is the use of an
empty authorization identity. In SASL, an empty authorization
identity means that the client is willing to authorize as the
authentication identity. The document is not clear on whether
the authzid must be omitted or if it can be specified with an
empty value to convey this. The requirement for backward
compatibility with HTTP Digest means that the situation is even
worse. For example, DIGEST-MD5 requires all usernames/passwords
that can be entirely represented in the ISO-8859-1 charset to be
down converted from UTF-8 [RFC3629] to ISO-8859-1 [ISO-8859-1].
Another example is the use of quoted strings. Handling of
characters that need escaping is not properly described, and the
DIGEST-MD5 document has no examples to demonstrate correct
behavior.
2. The DIGEST-MD5 document uses ABNF from RFC 822 [RFC0822], which
allows an extra construct and allows for "implied folding
whitespace" to be inserted in many places. The difference from a
more common ABNF defined in [RFC5234] is confusing for some
implementors. As a result, many implementations do not accept
folding whitespace in many places where it is allowed.
3. The DIGEST-MD5 document uses the concept of a "realm" to define a
collection of accounts. A DIGEST-MD5 server can support one or
more realms. The DIGEST-MD5 document does not provide any
guidance on how realms should be named and, more importantly, how
they can be entered in User Interfaces (UIs). As a result, many
DIGEST-MD5 clients have confusing UIs, do not allow users to
enter a realm, and/or do not allow users to pick one of the
server-supported realms.
4. Use of username in the inner hash is problematic. The inner hash
of DIGEST-MD5 is an MD5 hash of colon-separated username, realm,
and password. Implementations may choose to store inner hashes
instead of clear text passwords. This has some useful
properties, such as protection from compromise of authentication
databases containing the same username and password on other
servers if a server with the username and password is
compromised; however, this is rarely done in practice. First,
the inner hash is not compatible with widely deployed Unix
password databases, and second, changing the username would
invalidate the inner hash.
5. Description of DES/3DES [DES] and RC4 security layers are
inadequate to produce independently developed interoperable
implementations. In the DES/3DES case, this is partly a problem
with existing DES APIs.
6. DIGEST-MD5 outer hash (the value of the "response" directive)
does not protect the whole authentication exchange, which makes
the mechanism vulnerable to "man-in-the-middle" (MITM) attacks,
such as modification of the list of supported qops or ciphers.
7. The following features are missing from DIGEST-MD5, making it
insecure or unsuitable for use in protocols:
A. Channel bindings [RFC5056].
B. Hash agility (i.e., no easy way to replace the MD5 hash
function with another one).
C. Support for SASLPrep [RFC4013] or any other type of Unicode
character normalization of usernames and passwords. The
original DIGEST-MD5 document predates SASLPrep and does not
recommend any Unicode character normalization.
8. The cryptographic primitives in DIGEST-MD5 are not up to today's
standards, in particular:
A. The MD5 hash is sufficiently weak to make a brute force
attack on DIGEST-MD5 easy with common hardware [RFC6151].
B. The RC4 algorithm is prone to attack when used as the
security layer without discarding the initial key stream
output [RFC6229].
C. The DES cipher for the security layer is considered insecure
due to its small key space [RFC3766].
Note that most of the problems listed above are already present in
the HTTP Digest authentication mechanism.
Because DIGEST-MD5 is defined as an extensible mechanism, it is
possible to fix most of the problems listed above. However, this
would increase implementation complexity of an already complex
mechanism even further, so the effort is not worth the cost. In
addition, an implementation of a "fixed" DIGEST-MD5 specification
would likely either not interoperate with any existing implementation
of [RFC2831] or would be vulnerable to various downgrade attacks.
Note that despite DIGEST-MD5 seeing some deployment on the Internet,
this specification recommends obsoleting DIGEST-MD5 because DIGEST-
MD5, as implemented, is not a reasonable candidate for further
standardization and should be deprecated in favor of one or more new
password-based mechanisms currently being designed.
The Salted Challenge Response Authentication Mechanism (SCRAM) family
of SASL mechanisms [RFC5802] has been developed to provide similar
features as DIGEST-MD5 but with a better design.
2. Security Considerations
Security issues are discussed throughout this document.
3. IANA Considerations
IANA has changed the "Intended usage" of the DIGEST-MD5 mechanism
registration in the SASL mechanism registry to OBSOLETE. The SASL
mechanism registry is specified in [RFC4422] and is currently
available at:
http://www.iana.org/assignments/sasl-mechanisms
4. Acknowledgements
The author gratefully acknowledges the feedback provided by Chris
Newman, Simon Josefsson, Kurt Zeilenga, Sean Turner, and Abhijit
Menon-Sen. Various text was copied from other RFCs, in particular,
from [RFC2831].
5. References
5.1. Normative References
[RFC2617] Franks, J., Hallam-Baker, P., Hostetler, J., Lawrence,
S., Leach, P., Luotonen, A., and L. Stewart, "HTTP
Authentication: Basic and Digest Access
Authentication", RFC 2617, June 1999.
[RFC2831] Leach, P. and C. Newman, "Using Digest Authentication
as a SASL Mechanism", RFC 2831, May 2000.
5.2. Informative References
[DES] National Institute of Standards and Technology, "Data
Encryption Standard (DES)", FIPS PUB 46-3,
October 1999.
[ISO-8859-1] International Organization for Standardization,
"Information technology - 8-bit single-byte coded
graphic character sets - Part 1: Latin alphabet No. 1",
ISO/IEC 8859-1, 1998.
[RFC0822] Crocker, D., "Standard for the format of ARPA Internet
text messages", STD 11, RFC 822, August 1982.
[RFC2195] Klensin, J., Catoe, R., and P. Krumviede, "IMAP/POP
AUTHorize Extension for Simple Challenge/Response",
RFC 2195, September 1997.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO
10646", STD 63, RFC 3629, November 2003.
[RFC3766] Orman, H. and P. Hoffman, "Determining Strengths For
Public Keys Used For Exchanging Symmetric Keys",
BCP 86, RFC 3766, April 2004.
[RFC4013] Zeilenga, K., "SASLprep: Stringprep Profile for User
Names and Passwords", RFC 4013, February 2005.
[RFC4422] Melnikov, A. and K. Zeilenga, "Simple Authentication
and Security Layer (SASL)", RFC 4422, June 2006.
[RFC5056] Williams, N., "On the Use of Channel Bindings to Secure
Channels", RFC 5056, November 2007.
[RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", STD 68, RFC 5234, January 2008.
[RFC5802] Newman, C., Menon-Sen, A., Melnikov, A., and N.
Williams, "Salted Challenge Response Authentication
Mechanism (SCRAM) SASL and GSS-API Mechanisms",
RFC 5802, July 2010.
[RFC6151] Turner, S. and L. Chen, "Updated Security
Considerations for the MD5 Message-Digest and the HMAC-
MD5 Algorithms", RFC 6151, March 2011.
[RFC6229] Strombergson, J. and S. Josefsson, "Test Vectors for
the Stream Cipher RC4", RFC 6229, May 2011.
Author's Address
Alexey Melnikov
Isode Limited
5 Castle Business Village
36 Station Road
Hampton, Middlesex TW12 2BX
UK
EMail: Alexey.Melnikov@isode.com
URI: http://www.melnikov.ca/
ISO Plain Language
On this page
§1Why DIGEST-MD5 Was Retired
RFC 2831 defines how HTTP Digest Authentication can be used as a Simple Authentication and Security Layer mechanism for any protocol that has a SASL profile. It was intended both as an improvement over CRAM-MD5 and as a convenient way to support a single authentication mechanism for web, email, the Lightweight Directory Access Protocol and other protocols.
It can be argued that it is an improvement over CRAM-MD5, but many implementors commented that the additional complexity of DIGEST-MD5 makes it difficult to implement fully and securely. The RFC then lists an incomplete set of problems with the mechanism as specified.
Because DIGEST-MD5 is defined as an extensible mechanism, it would be possible to fix most of the problems. The RFC argues that doing so would increase the implementation complexity of an already complex mechanism even further, so the effort is not worth the cost. It adds that an implementation of a fixed DIGEST-MD5 specification would likely either fail to interoperate with any existing implementation of RFC 2831, or would be vulnerable to various downgrade attacks.
Despite DIGEST-MD5 having seen some deployment on the Internet, the RFC recommends obsoleting it, because as implemented it is not a reasonable candidate for further standardization and should be deprecated in favour of one or more new password-based mechanisms. The Salted Challenge Response Authentication Mechanism family of SASL mechanisms has been developed to provide similar features with a better design.
The mechanism could have been patched, since it was designed to be extended. The RFC judges that repairing it would make an already complicated protocol even harder to build correctly, and that a fixed version would probably not talk to the versions already in use. So it was retired rather than repaired, and SCRAM was named as the mechanism to use instead.
- 1997CRAM-MD5, the mechanism DIGEST-MD5 set out to improve onRFC 2195
- 1999HTTP Digest Authentication, the design that DIGEST-MD5 reusesRFC 2617
- 2000DIGEST-MD5 specified as a SASL mechanismRFC 2831
- 2010SCRAM, developed to offer similar features with a better designRFC 5802
- 2011This document moves DIGEST-MD5 to HistoricRFC 6331
§1Too Many Options and Confusing Syntax
The first problem listed is that the mechanism has too many options and modes, some of them not well described and not widely implemented. DIGEST-MD5 allows the qop directive to contain multiple values, but it also allows multiple qop directives to be specified, and the handling of multiple options is not specified. The result is minor interoperability problems: some implementations amalgamate multiple qop values into one, while others treat multiple qop values as an error. Another example is an empty authorization identity. In SASL, an empty authorization identity means that the client is willing to authorize as the authentication identity, but the document is not clear on whether the field must be omitted or can be given with an empty value to convey this. The requirement for backward compatibility with HTTP Digest makes the situation worse. For example, DIGEST-MD5 requires all user names and passwords that can be entirely represented in the ISO-8859-1 charset to be converted down from UTF-8. The use of quoted strings is another example: handling of characters that need escaping is not properly described, and the document has no examples to demonstrate correct behaviour.
The second problem is the syntax. The DIGEST-MD5 document uses ABNF from RFC 822, which allows an extra construct and allows implied folding whitespace to be inserted in many places. The difference from the more common ABNF defined in RFC 5234 is confusing for some implementors, and as a result many implementations do not accept folding whitespace in many of the places where it is allowed.
The rules left room for two different readings, so different implementations behaved differently and stopped talking to each other. The grammar used was also an older one that allows spaces to be folded across lines, which many implementations did not accept even though the document allowed it.
§1Realms, Stored Hashes and Security Layers
The third problem is the realm. The mechanism uses a realm to define a collection of accounts, and a server can support one or more of them, but the document gives no guidance on how realms should be named, or, more importantly, how they can be entered in user interfaces. As a result, many clients have confusing user interfaces, do not allow users to enter a realm, and or do not let users pick one of the realms the server supports.
The fourth problem is the use of the user name in the inner hash. The inner hash of DIGEST-MD5 is an MD5 hash of the colon-separated user name, realm and password. Implementations may choose to store inner hashes instead of clear text passwords, which has some useful properties, such as protection from a compromise of authentication databases that hold the same user name and password on other servers. This is rarely done in practice, because the inner hash is not compatible with widely deployed Unix password databases, and because changing the user name would invalidate the inner hash.
The fifth problem is that the descriptions of the DES and 3DES and RC4 security layers are inadequate to produce independently developed interoperable implementations. In the DES and 3DES case this is partly a problem with existing DES APIs.
The account area, called a realm, had no naming rules and no way for a user to pick one, so client programs ended up with confusing prompts. Storing a hashed password instead of the password itself was possible but fitted badly with the password files that Unix systems already had, and it broke as soon as a user changed their user name. The optional encryption layers were described too loosely for two independent teams to build compatible software.
§2Attack Surface and Missing Features
The sixth problem is that the outer hash, the value of the response directive, does not protect the whole authentication exchange. That makes the mechanism vulnerable to man-in-the-middle attacks, such as modification of the list of supported qop values or ciphers.
The seventh problem is a list of features that are missing, which makes the mechanism insecure or unsuitable for use in protocols. There are no channel bindings. There is no hash agility, meaning no easy way to replace the MD5 hash function with another one. And there is no support for SASLPrep or any other kind of Unicode character normalization of user names and passwords: the original document predates SASLPrep and recommends no normalization at all.
The eighth problem is that the cryptographic primitives are not up to today's standards. The MD5 hash is sufficiently weak to make a brute force attack on DIGEST-MD5 easy with common hardware. The RC4 algorithm is prone to attack when used as the security layer without discarding the initial key stream output. And the DES cipher for the security layer is considered insecure because of its small key space. The RFC notes that most of these problems are already present in the HTTP Digest authentication mechanism. Its Security Considerations section adds only one sentence: security issues are discussed throughout the document.
Someone in the middle could change the list of offered options and the exchange would still go through, because only part of it was protected. The mechanism also lacked features that later protocols expect, such as tying the login to the encrypted channel it runs over. And the underlying cryptography, MD5, RC4 and DES, had all fallen below an acceptable standard.
- Too many optionsToo many options and modes, some of them not well described and not widely implemented, which causes minor interoperability problems.
- Confusing grammarThe ABNF of RFC 822 is used, with its extra construct and implied folding whitespace, and many implementations do not accept the whitespace the document allows.
- Realm handlingNo guidance on naming realms or letting a user pick one in a user interface, so many clients have confusing interfaces.
- User name in the inner hashThe stored inner hash is not compatible with widely deployed Unix password databases, and changing the user name invalidates it.
- Weak security layersThe DES, 3DES and RC4 descriptions are inadequate to produce independently developed interoperable implementations.
- Missing featuresNo channel bindings, no hash agility and no Unicode normalization of user names and passwords.
- Unprotected exchangeThe outer hash does not protect the whole exchange, so the mechanism is vulnerable to man-in-the-middle attacks.
- Outdated cryptographyMD5 is weak enough for an easy brute force attack, and RC4 and DES are both considered insecure.
§3The IANA Registry Change
IANA changed the intended usage of the DIGEST-MD5 mechanism registration in the SASL mechanism registry to OBSOLETE. The SASL mechanism registry is specified by RFC 4422.
The change is administrative rather than technical: the mechanism itself is untouched, and this document does not describe any protocol behaviour. What it does is record that the mechanism is no longer one that anyone should build into new software, and that the document which defined it, RFC 2831, has moved to Historic status.
The practical effect is a line in a central registry that says do not use this mechanism in new software. Nothing about the protocol itself changed; what changed is the official record of whether it is worth building.