Tor directory protocol, version 3

This directory protocol is used by Tor version 0.2.0.x-alpha and later. See dir-spec-v1.txt for information on the protocol used up to the 0.1.0.x series, and dir-spec-v2.txt for information on the protocol used by the 0.1.1.x and 0.1.2.x series.

This document merges and supersedes the following proposals:

• 101 Voting on the Tor Directory System
• 103 Splitting identity key from regularly used signing key
• 104 Long and Short Router Descriptors

XXX timeline XXX fill in XXXXs

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

History

The earliest versions of Onion Routing shipped with a list of known routers and their keys. When the set of routers changed, users needed to fetch a new list.

The Version 1 Directory protocol

Early versions of Tor (0.0.2) introduced "Directory authorities": servers that served signed "directory" documents containing a list of signed "server descriptors", along with short summary of the status of each router. Thus, clients could get up-to-date information on the state of the network automatically, and be certain that the list they were getting was attested by a trusted directory authority.

Later versions (0.0.8) added directory caches, which download directories from the authorities and serve them to clients. Non-caches fetch from the caches in preference to fetching from the authorities, thus distributing bandwidth requirements.

Also added during the version 1 directory protocol were "router status" documents: short documents that listed only the up/down status of the routers on the network, rather than a complete list of all the descriptors. Clients and caches would fetch these documents far more frequently than they would fetch full directories.

The Version 2 Directory Protocol

During the Tor 0.1.1.x series, Tor revised its handling of directory documents in order to address two major problems:

      * Directories had grown quite large (over 1MB), and most directory
        downloads consisted mainly of server descriptors that clients
        already had.

      * Every directory authority was a trust bottleneck: if a single
        directory authority lied, it could make clients believe for a time
        an arbitrarily distorted view of the Tor network.  (Clients
        trusted the most recent signed document they downloaded.) Thus,
        adding more authorities would make the system less secure, not
        more.

To address these, we extended the directory protocol so that authorities now published signed "network status" documents. Each network status listed, for every router in the network: a hash of its identity key, a hash of its most recent descriptor, and a summary of what the authority believed about its status. Clients would download the authorities' network status documents in turn, and believe statements about routers iff they were attested to by more than half of the authorities.

Instead of downloading all server descriptors at once, clients downloaded only the descriptors that they did not have. Descriptors were indexed by their digests, in order to prevent malicious caches from giving different versions of a server descriptor to different clients.

Routers began working harder to upload new descriptors only when their contents were substantially changed.

Goals of the version 3 protocol

Version 3 of the Tor directory protocol tries to solve the following issues:

      * A great deal of bandwidth used to transmit server descriptors was
        used by two fields that are not actually used by Tor routers
        (namely read-history and write-history).  We save about 60% by
        moving them into a separate document that most clients do not
        fetch or use.

      * It was possible under certain perverse circumstances for clients
        to download an unusual set of network status documents, thus
        partitioning themselves from clients who have a more recent and/or
        typical set of documents.  Even under the best of circumstances,
        clients were sensitive to the ages of the network status documents
        they downloaded.  Therefore, instead of having the clients
        correlate multiple network status documents, we have the
        authorities collectively vote on a single consensus network status
        document.

      * The most sensitive data in the entire network (the identity keys
        of the directory authorities) needed to be stored unencrypted so
        that the authorities can sign network-status documents on the fly.
        Now, the authorities' identity keys are stored offline, and used
        to certify medium-term signing keys that can be rotated.