diff options
Diffstat (limited to 'attic/text_formats/dir-spec.txt')
| -rw-r--r-- | attic/text_formats/dir-spec.txt | 4299 |
1 files changed, 4299 insertions, 0 deletions
diff --git a/attic/text_formats/dir-spec.txt b/attic/text_formats/dir-spec.txt new file mode 100644 index 0000000..f133c39 --- /dev/null +++ b/attic/text_formats/dir-spec.txt @@ -0,0 +1,4299 @@ + + Tor directory protocol, version 3 + +Table of Contents + + 0. Scope and preliminaries + 0.1. History + 0.2. Goals of the version 3 protoc + 0.3. Some Remaining questions + 1. Outline + 1.1. What's different from version 2? + 1.2. Document meta-format + 1.3. Signing documents + 1.4. Voting timeline + 2. Router operation and formats + 2.1. Uploading server descriptors and extra-info documents + 2.1.1. Server descriptor format + 2.1.2. Extra-info document format + 2.1.3. Nonterminals in server descriptors + 3. Directory authority operation and formats + 3.1. Creating key certificates + 3.2. Accepting server descriptor and extra-info document uploads + 3.3. Computing microdescriptors + 3.4. Exchanging votes + 3.4.1. Vote and consensus status document formats + 3.4.2. Assigning flags in a vote + 3.4.3. Serving bandwidth list files + 3.5. Downloading missing certificates from other directory authorities + 3.6. Downloading server descriptors from other directory authorities + 3.7. Downloading extra-info documents from other directory authorities + 3.8. Computing a consensus from a set of votes + 3.8.0.1. Deciding which Ids to include. + 3.8.0.2. Deciding which descriptors to include + 3.8.1. Forward compatibility + 3.8.2. Encoding port lists + 3.8.3. Computing Bandwidth Weights + 3.9. Computing consensus flavors + 3.9.1. ns consensus + 3.9.2. Microdescriptor consensus + 3.10. Exchanging detached signatures + 3.11. Publishing the signed consensus + 4. Directory cache operation + 4.1. Downloading consensus status documents from directory authorities + 4.2. Downloading server descriptors from directory authorities + 4.3. Downloading microdescriptors from directory authorities + 4.4. Downloading extra-info documents from directory authorities + 4.5. Consensus diffs + 4.5.1. Consensus diff format + 4.5.2. Serving and requesting diff + 4.6 Retrying failed downloads + 5. Client operation + 5.1. Downloading network-status documents + 5.2. Downloading server descriptors or microdescriptors + 5.3. Downloading extra-info documents + 5.4. Using directory information + 5.4.1. Choosing routers for circuits. + 5.4.2. Managing naming + 5.4.3. Software versions + 5.4.4. Warning about a router's status. + 5.5. Retrying failed downloads + 6. Standards compliance + 6.1. HTTP headers + 6.2. HTTP status codes + A. Consensus-negotiation timeline. + B. General-use HTTP URLs + C. Converting a curve25519 public key to an ed25519 public key + D. Inferring missing proto lines. + E. Limited ed diff format + +0. Scope and preliminaries + + This directory protocol is used by Tor version 0.2.0.x-alpha and later. + See dir-spec-v1.txt for information on the protocol used up to the + 0.1.0.x series, and dir-spec-v2.txt for information on the protocol + used by the 0.1.1.x and 0.1.2.x series. + + This document merges and supersedes the following proposals: + + 101 Voting on the Tor Directory System + 103 Splitting identity key from regularly used signing key + 104 Long and Short Router Descriptors + + XXX timeline + XXX fill in XXXXs + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL + NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and + "OPTIONAL" in this document are to be interpreted as described in + RFC 2119. + +0.1. History + + The earliest versions of Onion Routing shipped with a list of known + routers and their keys. When the set of routers changed, users needed to + fetch a new list. + + The Version 1 Directory protocol + -------------------------------- + + Early versions of Tor (0.0.2) introduced "Directory authorities": servers + that served signed "directory" documents containing a list of signed + "server descriptors", along with short summary of the status of each + router. Thus, clients could get up-to-date information on the state of + the network automatically, and be certain that the list they were getting + was attested by a trusted directory authority. + + Later versions (0.0.8) added directory caches, which download + directories from the authorities and serve them to clients. Non-caches + fetch from the caches in preference to fetching from the authorities, thus + distributing bandwidth requirements. + + Also added during the version 1 directory protocol were "router status" + documents: short documents that listed only the up/down status of the + routers on the network, rather than a complete list of all the + descriptors. Clients and caches would fetch these documents far more + frequently than they would fetch full directories. + + The Version 2 Directory Protocol + -------------------------------- + + During the Tor 0.1.1.x series, Tor revised its handling of directory + documents in order to address two major problems: + + * Directories had grown quite large (over 1MB), and most directory + downloads consisted mainly of server descriptors that clients + already had. + + * Every directory authority was a trust bottleneck: if a single + directory authority lied, it could make clients believe for a time + an arbitrarily distorted view of the Tor network. (Clients + trusted the most recent signed document they downloaded.) Thus, + adding more authorities would make the system less secure, not + more. + + To address these, we extended the directory protocol so that + authorities now published signed "network status" documents. Each + network status listed, for every router in the network: a hash of its + identity key, a hash of its most recent descriptor, and a summary of + what the authority believed about its status. Clients would download + the authorities' network status documents in turn, and believe + statements about routers iff they were attested to by more than half of + the authorities. + + Instead of downloading all server descriptors at once, clients + downloaded only the descriptors that they did not have. Descriptors + were indexed by their digests, in order to prevent malicious caches + from giving different versions of a server descriptor to different + clients. + + Routers began working harder to upload new descriptors only when their + contents were substantially changed. + + +0.2. Goals of the version 3 protocol + + Version 3 of the Tor directory protocol tries to solve the following + issues: + + * A great deal of bandwidth used to transmit server descriptors was + used by two fields that are not actually used by Tor routers + (namely read-history and write-history). We save about 60% by + moving them into a separate document that most clients do not + fetch or use. + + * It was possible under certain perverse circumstances for clients + to download an unusual set of network status documents, thus + partitioning themselves from clients who have a more recent and/or + typical set of documents. Even under the best of circumstances, + clients were sensitive to the ages of the network status documents + they downloaded. Therefore, instead of having the clients + correlate multiple network status documents, we have the + authorities collectively vote on a single consensus network status + document. + + * The most sensitive data in the entire network (the identity keys + of the directory authorities) needed to be stored unencrypted so + that the authorities can sign network-status documents on the fly. + Now, the authorities' identity keys are stored offline, and used + to certify medium-term signing keys that can be rotated. + +0.3. Some Remaining questions + + Things we could solve on a v3 timeframe: + + The SHA-1 hash is showing its age. We should do something about our + dependency on it. We could probably future-proof ourselves here in + this revision, at least so far as documents from the authorities are + concerned. + + Too many things about the authorities are hardcoded by IP. + + Perhaps we should start accepting longer identity keys for routers + too. + + Things to solve eventually: + + Requiring every client to know about every router won't scale forever. + + Requiring every directory cache to know every router won't scale + forever. + + +1. Outline + + There is a small set (say, around 5-10) of semi-trusted directory + authorities. A default list of authorities is shipped with the Tor + software. Users can change this list, but are encouraged not to do so, + in order to avoid partitioning attacks. + + Every authority has a very-secret, long-term "Authority Identity Key". + This is stored encrypted and/or offline, and is used to sign "key + certificate" documents. Every key certificate contains a medium-term + (3-12 months) "authority signing key", that is used by the authority to + sign other directory information. (Note that the authority identity + key is distinct from the router identity key that the authority uses + in its role as an ordinary router.) + + Routers periodically upload signed "routers descriptors" to the + directory authorities describing their keys, capabilities, and other + information. Routers may also upload signed "extra-info documents" + containing information that is not required for the Tor protocol. + Directory authorities serve server descriptors indexed by router + identity, or by hash of the descriptor. + + Routers may act as directory caches to reduce load on the directory + authorities. They announce this in their descriptors. + + Periodically, each directory authority generates a view of + the current descriptors and status for known routers. They send a + signed summary of this view (a "status vote") to the other + authorities. The authorities compute the result of this vote, and sign + a "consensus status" document containing the result of the vote. + + Directory caches download, cache, and re-serve consensus documents. + + Clients, directory caches, and directory authorities all use consensus + documents to find out when their list of routers is out-of-date. + (Directory authorities also use vote statuses.) If it is, they download + any missing server descriptors. Clients download missing descriptors + from caches; caches and authorities download from authorities. + Descriptors are downloaded by the hash of the descriptor, not by the + relay's identity key: this prevents directory servers from attacking + clients by giving them descriptors nobody else uses. + + All directory information is uploaded and downloaded with HTTP. + +1.1. What's different from version 2? + + Clients used to download multiple network status documents, + corresponding roughly to "status votes" above. They would compute the + result of the vote on the client side. + + Authorities used to sign documents using the same private keys they used + for their roles as routers. This forced them to keep these extremely + sensitive keys in memory unencrypted. + + All of the information in extra-info documents used to be kept in the + main descriptors. + +1.2. Document meta-format + + Server descriptors, directories, and running-routers documents all obey the + following lightweight extensible information format. + + The highest level object is a Document, which consists of one or more + Items. Every Item begins with a KeywordLine, followed by zero or more + Objects. A KeywordLine begins with a Keyword, optionally followed by + whitespace and more non-newline characters, and ends with a newline. A + Keyword is a sequence of one or more characters in the set [A-Za-z0-9-], + but may not start with -. + An Object is a block of encoded data in pseudo-Privacy-Enhanced-Mail (PEM) + style format: that is, lines of encoded data MAY be wrapped by inserting + an ascii linefeed ("LF", also called newline, or "NL" here) character + (cf. RFC 4648 §3.1). When line wrapping, implementations MUST wrap lines + at 64 characters. Upon decoding, implementations MUST ignore and discard + all linefeed characters. + + More formally: + + NL = The ascii LF character (hex value 0x0a). + Document ::= (Item | NL)+ + Item ::= KeywordLine Object? + KeywordLine ::= Keyword (WS Argument)* NL + Keyword = KeywordStart KeywordChar* + KeywordStart ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' + KeywordChar ::= KeywordStart | '-' + Argument := ArgumentChar+ + ArgumentChar ::= any graphical printing ASCII character. + WS = (SP | TAB)+ + Object ::= BeginLine Base64-encoded-data EndLine + BeginLine ::= "-----BEGIN " Keyword (" " Keyword)* "-----" NL + EndLine ::= "-----END " Keyword (" " Keyword)* "-----" NL + + A Keyword may not be "-----BEGIN". + + The BeginLine and EndLine of an Object must use the same keyword. + + When interpreting a Document, software MUST ignore any KeywordLine that + starts with a keyword it doesn't recognize; future implementations MUST NOT + require current clients to understand any KeywordLine not currently + described. + + Other implementations that want to extend Tor's directory format MAY + introduce their own items. The keywords for extension items SHOULD start + with the characters "x-" or "X-", to guarantee that they will not conflict + with keywords used by future versions of Tor. + + In our document descriptions below, we tag Items with a multiplicity in + brackets. Possible tags are: + + "At start, exactly once": These items MUST occur in every instance of + the document type, and MUST appear exactly once, and MUST be the + first item in their documents. + + "Exactly once": These items MUST occur exactly one time in every + instance of the document type. + + "At end, exactly once": These items MUST occur in every instance of + the document type, and MUST appear exactly once, and MUST be the + last item in their documents. + + "At most once": These items MAY occur zero or one times in any + instance of the document type, but MUST NOT occur more than once. + + "Any number": These items MAY occur zero, one, or more times in any + instance of the document type. + + "Once or more": These items MUST occur at least once in any instance + of the document type, and MAY occur more. + + For forward compatibility, each item MUST allow extra arguments at the + end of the line unless otherwise noted. So if an item's description below + is given as: + + "thing" int int int NL + + then implementations SHOULD accept this string as well: + + "thing 5 9 11 13 16 12" NL + + but not this string: + + "thing 5" NL + + and not this string: + + "thing 5 10 thing" NL + . + + Whenever an item DOES NOT allow extra arguments, we will tag it with + "no extra arguments". + +1.3. Signing documents + + Every signable document below is signed in a similar manner, using a + given "Initial Item", a final "Signature Item", a digest algorithm, and + a signing key. + + The Initial Item must be the first item in the document. + + The Signature Item has the following format: + + <signature item keyword> [arguments] NL SIGNATURE NL + + The "SIGNATURE" Object contains a signature (using the signing key) of + the PKCS#1 1.5 padded digest of the entire document, taken from the + beginning of the Initial item, through the newline after the Signature + Item's keyword and its arguments. + + The signature does not include the algorithmIdentifier specified in PKCS #1. + + Unless specified otherwise, the digest algorithm is SHA-1. + + All documents are invalid unless signed with the correct signing key. + + The "Digest" of a document, unless stated otherwise, is its digest *as + signed by this signature scheme*. + +1.4. Voting timeline + + Every consensus document has a "valid-after" (VA) time, a "fresh-until" + (FU) time and a "valid-until" (VU) time. VA MUST precede FU, which MUST + in turn precede VU. Times are chosen so that every consensus will be + "fresh" until the next consensus becomes valid, and "valid" for a while + after. At least 3 consensuses should be valid at any given time. + + The timeline for a given consensus is as follows: + + VA-DistSeconds-VoteSeconds: The authorities exchange votes. Each authority + uploads their vote to all other authorities. + + VA-DistSeconds-VoteSeconds/2: The authorities try to download any + votes they don't have. + + Authorities SHOULD also reject any votes that other authorities try to + upload after this time. (0.4.4.1-alpha was the first version to reject votes + in this way.) + + Note: Refusing late uploaded votes minimizes the chance of a consensus + split, particular when authorities are under bandwidth pressure. If an + authority is struggling to upload its vote, and finally uploads to a + fraction of authorities after this period, they will compute a consensus + different from the others. By refusing uploaded votes after this time, + we increase the likelihood that most authorities will use the same vote + set. + + Rejecting late uploaded votes does not fix the problem entirely. If + some authorities are able to download a specific vote, but others fail + to do so, then there may still be a consensus split. However, this + change does remove one common cause of consensus splits. + + VA-DistSeconds: The authorities calculate the consensus and exchange + signatures. (This is the earliest point at which anybody can + possibly get a given consensus if they ask for it.) + + VA-DistSeconds/2: The authorities try to download any signatures + they don't have. + + VA: All authorities have a multiply signed consensus. + + VA ... FU: Caches download the consensus. (Note that since caches have + no way of telling what VA and FU are until they have downloaded + the consensus, they assume that the present consensus's VA is + equal to the previous one's FU, and that its FU is one interval after + that.) + + FU: The consensus is no longer the freshest consensus. + + FU ... (the current consensus's VU): Clients download the consensus. + (See note above: clients guess that the next consensus's FU will be + two intervals after the current VA.) + + VU: The consensus is no longer valid; clients should continue to try to + download a new consensus if they have not done so already. + + VU + 24 hours: Clients will no longer use the consensus at all. + + VoteSeconds and DistSeconds MUST each be at least 20 seconds; FU-VA and + VU-FU MUST each be at least 5 minutes. + +2. Router operation and formats + +2.1. Uploading server descriptors and extra-info documents + + ORs SHOULD generate a new server descriptor and a new extra-info + document whenever any of the following events have occurred: + + - A period of time (18 hrs by default) has passed since the last + time a descriptor was generated. + + - A descriptor field other than bandwidth or uptime has changed. + + - Its uptime is less than 24h and bandwidth has changed by a factor of 2 + from the last time a descriptor was generated, and at least a given + interval of time (3 hours by default) has passed since then. + + - Its uptime has been reset (by restarting). + + - It receives a networkstatus consensus in which it is not listed. + + - It receives a networkstatus consensus in which it is listed + with the StaleDesc flag. + + [XXX this list is incomplete; see router_differences_are_cosmetic() + in routerlist.c for others] + + ORs SHOULD NOT publish a new server descriptor or extra-info document + if none of the above events have occurred and not much time has passed + (12 hours by default). + + Tor versions older than 0.3.5.1-alpha ignore uptime when checking for + bandwidth changes. + + After generating a descriptor, ORs upload them to every directory + authority they know, by posting them (in order) to the URL + + http://<hostname:port>/tor/ + + Server descriptors may not exceed 20,000 bytes in length; extra-info + documents may not exceed 50,000 bytes in length. If they do, the + authorities SHOULD reject them. + +2.1.1. Server descriptor format + + Server descriptors consist of the following items. + + In lines that take multiple arguments, extra arguments SHOULD be + accepted and ignored. Many of the nonterminals below are defined in + section 2.1.3. + + Note that many versions of Tor will generate an extra newline at the + end of their descriptors. Implementations MUST tolerate one or + more blank lines at the end of a single descriptor or a list of + concatenated descriptors. New implementations SHOULD NOT generate + such blank lines. + + "router" nickname address ORPort SOCKSPort DirPort NL + + [At start, exactly once.] + + Indicates the beginning of a server descriptor. "nickname" must be a + valid router nickname as specified in section 2.1.3. "address" must + be an IPv4 + address in dotted-quad format. The last three numbers indicate the + TCP ports at which this OR exposes functionality. ORPort is a port at + which this OR accepts TLS connections for the main OR protocol; + SOCKSPort is deprecated and should always be 0; and DirPort is the + port at which this OR accepts directory-related HTTP connections. If + any port is not supported, the value 0 is given instead of a port + number. (At least one of DirPort and ORPort SHOULD be set; + authorities MAY reject any descriptor with both DirPort and ORPort of + 0.) + + "identity-ed25519" NL "-----BEGIN ED25519 CERT-----" NL certificate + "-----END ED25519 CERT-----" NL + + [Exactly once, in second position in document.] + [No extra arguments] + + The certificate is a base64-encoded Ed25519 certificate (see + cert-spec.txt) with terminating =s removed. When this element + is present, it MUST appear as the first or second element in + the router descriptor. + + The certificate has CERT_TYPE of [04]. It must include a + signed-with-ed25519-key extension (see cert-spec.txt, + section 2.2.1), so that we can extract the master identity key. + + [Before Tor 0.4.5.1-alpha, this field was optional.] + + "master-key-ed25519" SP MasterKey NL + + [Exactly once] + + Contains the base-64 encoded ed25519 master key as a single + argument. If it is present, it MUST match the identity key + in the identity-ed25519 entry. + + [Before Tor 0.4.5.1-alpha, this field was optional.] + + "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed NL + + [Exactly once] + + Estimated bandwidth for this router, in bytes per second. The + "average" bandwidth is the volume per second that the OR is willing to + sustain over long periods; the "burst" bandwidth is the volume that + the OR is willing to sustain in very short intervals. The "observed" + value is an estimate of the capacity this relay can handle. The + relay remembers the max bandwidth sustained output over any ten + second period in the past 5 days, and another sustained input. The + "observed" value is the lesser of these two numbers. + + Tor versions released before 2018 only kept bandwidth-observed for one + day. These versions are no longer supported or recommended. + + "platform" string NL + + [At most once] + + A human-readable string describing the system on which this OR is + running. This MAY include the operating system, and SHOULD include + the name and version of the software implementing the Tor protocol. + + "published" YYYY-MM-DD HH:MM:SS NL + + [Exactly once] + + The time, in UTC, when this descriptor (and its corresponding + extra-info document if any) was generated. + + "fingerprint" fingerprint NL + + [At most once] + + A fingerprint (a HASH_LEN-byte of asn1 encoded public key, encoded in + hex, with a single space after every 4 characters) for this router's + identity key. A descriptor is considered invalid (and MUST be + rejected) if the fingerprint line does not match the public key. + + [We didn't start parsing this line until Tor 0.1.0.6-rc; it should + be marked with "opt" until earlier versions of Tor are obsolete.] + + "hibernating" bool NL + + [At most once] + + If the value is 1, then the Tor relay was hibernating when the + descriptor was published, and shouldn't be used to build circuits. + + [We didn't start parsing this line until Tor 0.1.0.6-rc; it should be + marked with "opt" until earlier versions of Tor are obsolete.] + + "uptime" number NL + + [At most once] + + The number of seconds that this OR process has been running. + + "onion-key" NL a public key in PEM format + + [Exactly once] + [No extra arguments] + + This key is used to encrypt CREATE cells for this OR. The key MUST be + accepted for at least 1 week after any new key is published in a + subsequent descriptor. It MUST be 1024 bits. + + The key encoding is the encoding of the key as a PKCS#1 RSAPublicKey + structure, encoded in base64, and wrapped in "-----BEGIN RSA PUBLIC + KEY-----" and "-----END RSA PUBLIC KEY-----". + + "onion-key-crosscert" NL a RSA signature in PEM format. + + [Exactly once] + [No extra arguments] + + This element contains an RSA signature, generated using the + onion-key, of the following: + + A SHA1 hash of the RSA identity key, + i.e. RSA key from "signing-key" (see below) [20 bytes] + The Ed25519 identity key, + i.e. Ed25519 key from "master-key-ed25519" [32 bytes] + + If there is no Ed25519 identity key, or if in some future version + there is no RSA identity key, the corresponding field must be + zero-filled. + + Parties verifying this signature MUST allow additional data + beyond the 52 bytes listed above. + + This signature proves that the party creating the descriptor + had control over the secret key corresponding to the + onion-key. + + [Before Tor 0.4.5.1-alpha, this field was optional whenever + identity-ed25519 was absent.] + + "ntor-onion-key" base-64-encoded-key + + [Exactly once] + + A curve25519 public key used for the ntor circuit extended + handshake. It's the standard encoding of the OR's curve25519 + public key, encoded in base 64. The trailing '=' sign MAY be + omitted from the base64 encoding. The key MUST be accepted + for at least 1 week after any new key is published in a + subsequent descriptor. + + [Before Tor 0.4.5.1-alpha, this field was optional.] + + "ntor-onion-key-crosscert" SP Bit NL + "-----BEGIN ED25519 CERT-----" NL certificate + "-----END ED25519 CERT-----" NL + + [Exactly once] + [No extra arguments] + + A signature created with the ntor-onion-key, using the + certificate format documented in cert-spec.txt, with type + [0a]. The signed key here is the master identity key. + + Bit must be "0" or "1". It indicates the sign of the ed25519 + public key corresponding to the ntor onion key. If Bit is "0", + then implementations MUST guarantee that the x-coordinate of + the resulting ed25519 public key is positive. Otherwise, if + Bit is "1", then the sign of the x-coordinate MUST be negative. + + To compute the ed25519 public key corresponding to a curve25519 + key, and for further explanation on key formats, see appendix C. + + This signature proves that the party creating the descriptor + had control over the secret key corresponding to the + ntor-onion-key. + + [Before Tor 0.4.5.1-alpha, this field was optional whenever + identity-ed25519 was absent.] + + "signing-key" NL a public key in PEM format + + [Exactly once] + [No extra arguments] + + The OR's long-term RSA identity key. It MUST be 1024 bits. + + The encoding is as for "onion-key" above. + + "accept" exitpattern NL + "reject" exitpattern NL + + [Any number] + + These lines describe an "exit policy": the rules that an OR follows + when deciding whether to allow a new stream to a given address. The + 'exitpattern' syntax is described below. There MUST be at least one + such entry. The rules are considered in order; if no rule matches, + the address will be accepted. For clarity, the last such entry SHOULD + be accept *:* or reject *:*. + + "ipv6-policy" SP ("accept" / "reject") SP PortList NL + + [At most once.] + + An exit-policy summary as specified in sections 3.4.1 and 3.8.2, + summarizing + the router's rules for connecting to IPv6 addresses. A missing + "ipv6-policy" line is equivalent to "ipv6-policy reject + 1-65535". + + "overload-general" SP version SP YYYY-MM-DD HH:MM:SS NL + + [At most once.] + + Indicates that a relay has reached an "overloaded state" which can be + one or many of the following load metrics: + + - Any OOM invocation due to memory pressure + - Any ntor onionskins are dropped + - TCP port exhaustion + + The timestamp is when at least one metrics was detected. It should always + be at the hour and thus, as an example, "2020-01-10 13:00:00" is an + expected timestamp. Because this is a binary state, if the line is + present, we consider that it was hit at the very least once somewhere + between the provided timestamp and the "published" timestamp of the + document which is when the document was generated. + + The overload-general line should remain in place for 72 hours since last + triggered. If the limits are reached again in this period, the timestamp + is updated, and this 72 hour period restarts. + + The 'version' field is set to '1' for now. + + (Introduced in tor-0.4.6.1-alpha, but moved from extra-info to general + descriptor in tor-0.4.6.2-alpha) + + "router-sig-ed25519" SP Signature NL + + [Exactly once.] + + It MUST be the next-to-last element in the descriptor, appearing + immediately before the RSA signature. It MUST contain an Ed25519 + signature of a SHA256 digest of the entire document. This digest is + taken from the first character up to and including the first space + after the "router-sig-ed25519" string. Before computing the digest, + the string "Tor router descriptor signature v1" is prefixed to the + document. + + The signature is encoded in Base64, with terminating =s removed. + + The signing key in the identity-ed25519 certificate MUST + be the one used to sign the document. + + [Before Tor 0.4.5.1-alpha, this field was optional whenever + identity-ed25519 was absent.] + + "router-signature" NL Signature NL + + [At end, exactly once] + [No extra arguments] + + The "SIGNATURE" object contains a signature of the PKCS1-padded + hash of the entire server descriptor, taken from the beginning of the + "router" line, through the newline after the "router-signature" line. + The server descriptor is invalid unless the signature is performed + with the router's identity key. + + "contact" info NL + + [At most once] + + Describes a way to contact the relay's administrator, preferably + including an email address and a PGP key fingerprint. + + "bridge-distribution-request" SP Method NL + + [At most once, bridges only.] + + The "Method" describes how a Bridge address is distributed by + BridgeDB. Recognized methods are: "none", "any", "https", "email", + "moat". If set to "none", BridgeDB will avoid distributing your bridge + address. If set to "any", BridgeDB will choose how to distribute your + bridge address. Choosing any of the other methods will tell BridgeDB to + distribute your bridge via a specific method: + + - "https" specifies distribution via the web interface at + https://bridges.torproject.org; + - "email" specifies distribution via the email autoresponder at + bridges@torproject.org; + - "moat" specifies distribution via an interactive menu inside Tor + Browser; and + + Potential future "Method" specifiers must be as follows: + Method = (KeywordChar | "_") + + + All bridges SHOULD include this line. Non-bridges MUST NOT include + it. + + BridgeDB SHOULD treat unrecognized Method values as if they were + "none". + + (Default: "any") + + [This line was introduced in 0.3.2.3-alpha, with a minimal backport + to 0.2.5.16, 0.2.8.17, 0.2.9.14, 0.3.0.13, 0.3.1.9, and later.] + + "family" names NL + + [At most once] + + 'Names' is a space-separated list of relay nicknames or + hexdigests. If two ORs list one another in their "family" entries, + then OPs should treat them as a single OR for the purpose of path + selection. + + For example, if node A's descriptor contains "family B", and node B's + descriptor contains "family A", then node A and node B should never + be used on the same circuit. + + "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL + [At most once] + "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL + [At most once] + + (These fields once appeared in router descriptors, but have + appeared in extra-info descriptors since 0.2.0.x.) + + "eventdns" bool NL + + [At most once] + + Declare whether this version of Tor is using the newer enhanced + dns logic. Versions of Tor with this field set to false SHOULD NOT + be used for reverse hostname lookups. + + [This option is obsolete. All Tor current relays should be presumed + to have the evdns backend.] + + "caches-extra-info" NL + + [At most once.] + [No extra arguments] + + Present only if this router is a directory cache that provides + extra-info documents. + + [Versions before 0.2.0.1-alpha don't recognize this] + + "extra-info-digest" SP sha1-digest [SP sha256-digest] NL + + [At most once] + + "sha1-digest" is a hex-encoded SHA1 digest (using upper-case characters) + of the router's extra-info document, as signed in the router's + extra-info (that is, not including the signature). (If this field is + absent, the router is not uploading a corresponding extra-info + document.) + + "sha256-digest" is a base64-encoded SHA256 digest of the extra-info + document. Unlike the "sha1-digest", this digest is calculated over the + entire document, including the signature. This difference is due to + a long-lived bug in the tor implementation that it would be difficult + to roll out an incremental fix for, not a design choice. Future digest + algorithms specified should not include the signature in the data used + to compute the digest. + + [Versions before 0.2.7.2-alpha did not include a SHA256 digest.] + [Versions before 0.2.0.1-alpha don't recognize this field at all.] + + "hidden-service-dir" NL + + [At most once.] + + Present only if this router stores and serves hidden service + descriptors. This router supports the descriptor versions declared + in the HSDir "proto" entry. If there is no "proto" entry, this + router supports version 2 descriptors. + + "protocols" SP "Link" SP LINK-VERSION-LIST SP "Circuit" SP + CIRCUIT-VERSION-LIST NL + + [At most once.] + + An obsolete list of protocol versions, superseded by the "proto" + entry. This list was never parsed, and has not been emitted + since Tor 0.2.9.4-alpha. New code should neither generate nor + parse this line. + + "allow-single-hop-exits" NL + + [At most once.] + [No extra arguments] + + Present only if the router allows single-hop circuits to make exit + connections. Most Tor relays do not support this: this is + included for specialized controllers designed to support perspective + access and such. This is obsolete in tor version >= 0.3.1.0-alpha. + + "or-address" SP ADDRESS ":" PORT NL + + [Any number] + + ADDRESS = IP6ADDR | IP4ADDR + IPV6ADDR = an ipv6 address, surrounded by square brackets. + IPV4ADDR = an ipv4 address, represented as a dotted quad. + PORT = a number between 1 and 65535 inclusive. + + An alternative for the address and ORPort of the "router" line, but with + two added capabilities: + + * or-address can be either an IPv4 or IPv6 address + * or-address allows for multiple ORPorts and addresses + + A descriptor SHOULD NOT include an or-address line that does nothing but + duplicate the address:port pair from its "router" line. + + The ordering of or-address lines and their PORT entries matter because + Tor MAY accept a limited number of address/port pairs. As of + Tor 0.2.3.x only the first address/port pair is advertised and used. + + "tunnelled-dir-server" NL + + [At most once.] + [No extra arguments] + + Present if the router accepts "tunneled" directory requests using a + BEGIN_DIR cell over the router's OR port. + (Added in 0.2.8.1-alpha. Before this, Tor relays accepted + tunneled directory requests only if they had a DirPort open, + or if they were bridges.) + + "proto" SP Entries NL + + [Exactly once.] + + Entries = + Entries = Entry + Entries = Entry SP Entries + + Entry = Keyword "=" Values + + Values = + Values = Value + Values = Value "," Values + + Value = Int + Value = Int "-" Int + + Int = NON_ZERO_DIGIT + Int = Int DIGIT + + Each 'Entry' in the "proto" line indicates that the Tor relay supports + one or more versions of the protocol in question. Entries should be + sorted by keyword. Values should be numerically ascending within each + entry. (This implies that there should be no overlapping ranges.) + Ranges should be represented as compactly as possible. Ints must be no + larger than 63. + + This field was first added in Tor 0.2.9.x. + + [Before Tor 0.4.5.1-alpha, this field was optional.] + + +2.1.2. Extra-info document format + + Extra-info documents consist of the following items: + + "extra-info" Nickname Fingerprint NL + [At start, exactly once.] + + Identifies what router this is an extra-info descriptor for. + Fingerprint is encoded in hex (using upper-case letters), with + no spaces. + + "identity-ed25519" + [As in router descriptors] + + "published" YYYY-MM-DD HH:MM:SS NL + + [Exactly once.] + + The time, in UTC, when this document (and its corresponding router + descriptor if any) was generated. It MUST match the published time + in the corresponding server descriptor. + + "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL + [At most once.] + "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL + [At most once.] + + Declare how much bandwidth the OR has used recently. Usage is divided + into intervals of NSEC seconds. The YYYY-MM-DD HH:MM:SS field + defines the end of the most recent interval. The numbers are the + number of bytes used in the most recent intervals, ordered from + oldest to newest. + + These fields include both IPv4 and IPv6 traffic. + + "ipv6-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL + [At most once] + "ipv6-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL + [At most once] + + Declare how much bandwidth the OR has used recently, on IPv6 + connections. See "read-history" and "write-history" for full details. + + "geoip-db-digest" Digest NL + [At most once.] + + SHA1 digest of the IPv4 GeoIP database file that is used to + resolve IPv4 addresses to country codes. + + "geoip6-db-digest" Digest NL + [At most once.] + + SHA1 digest of the IPv6 GeoIP database file that is used to + resolve IPv6 addresses to country codes. + + ("geoip-start-time" YYYY-MM-DD HH:MM:SS NL) + ("geoip-client-origins" CC=NUM,CC=NUM,... NL) + + Only generated by bridge routers (see blocking.pdf), and only + when they have been configured with a geoip database. + Non-bridges SHOULD NOT generate these fields. Contains a list + of mappings from two-letter country codes (CC) to the number + of clients that have connected to that bridge from that + country (approximate, and rounded up to the nearest multiple of 8 + in order to hamper traffic analysis). A country is included + only if it has at least one address. The time in + "geoip-start-time" is the time at which we began collecting geoip + statistics. + + "geoip-start-time" and "geoip-client-origins" have been replaced by + "bridge-stats-end" and "bridge-ips" in 0.2.2.4-alpha. The + reason is that the measurement interval with "geoip-stats" as + determined by subtracting "geoip-start-time" from "published" could + have had a variable length, whereas the measurement interval in + 0.2.2.4-alpha and later is set to be exactly 24 hours long. In + order to clearly distinguish the new measurement intervals from + the old ones, the new keywords have been introduced. + + "bridge-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + A "bridge-stats-end" line, as well as any other "bridge-*" line, + is only added when the relay has been running as a bridge for at + least 24 hours. + + "bridge-ips" CC=NUM,CC=NUM,... NL + [At most once.] + + List of mappings from two-letter country codes to the number of + unique IP addresses that have connected from that country to the + bridge and which are no known relays, rounded up to the nearest + multiple of 8. + + "bridge-ip-versions" FAM=NUM,FAM=NUM,... NL + [At most once.] + + List of unique IP addresses that have connected to the bridge + per protocol family. + + "bridge-ip-transports" PT=NUM,PT=NUM,... NL + [At most once.] + + List of mappings from pluggable transport names to the number + of unique IP addresses that have connected using that + pluggable transport. Unobfuscated connections are counted + using the reserved pluggable transport name "<OR>" (without + quotes). If we received a connection from a transport proxy + but we couldn't figure out the name of the pluggable + transport, we use the reserved pluggable transport name + "<??>". + + ("<OR>" and "<??>" are reserved because normal pluggable + transport names MUST match the following regular expression: + "[a-zA-Z_][a-zA-Z0-9_]*" ) + + The pluggable transport name list is sorted into lexically + ascending order. + + If no clients have connected to the bridge yet, we only write + "bridge-ip-transports" to the stats file. + + "dirreq-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + A "dirreq-stats-end" line, as well as any other "dirreq-*" line, + is only added when the relay has opened its Dir port and after 24 + hours of measuring directory requests. + + "dirreq-v2-ips" CC=NUM,CC=NUM,... NL + [At most once.] + "dirreq-v3-ips" CC=NUM,CC=NUM,... NL + [At most once.] + + List of mappings from two-letter country codes to the number of + unique IP addresses that have connected from that country to + request a v2/v3 network status, rounded up to the nearest multiple + of 8. Only those IP addresses are counted that the directory can + answer with a 200 OK status code. (Note here and below: current Tor + versions, as of 0.2.5.2-alpha, no longer cache or serve v2 + networkstatus documents.) + + "dirreq-v2-reqs" CC=NUM,CC=NUM,... NL + [At most once.] + "dirreq-v3-reqs" CC=NUM,CC=NUM,... NL + [At most once.] + + List of mappings from two-letter country codes to the number of + requests for v2/v3 network statuses from that country, rounded up + to the nearest multiple of 8. Only those requests are counted that + the directory can answer with a 200 OK status code. + + "dirreq-v2-share" NUM% NL + [At most once.] + "dirreq-v3-share" NUM% NL + [At most once.] + + The share of v2/v3 network status requests that the directory + expects to receive from clients based on its advertised bandwidth + compared to the overall network bandwidth capacity. Shares are + formatted in percent with two decimal places. Shares are + calculated as means over the whole 24-hour interval. + + "dirreq-v2-resp" status=NUM,... NL + [At most once.] + "dirreq-v3-resp" status=NUM,... NL + [At most once.] + + List of mappings from response statuses to the number of requests + for v2/v3 network statuses that were answered with that response + status, rounded up to the nearest multiple of 4. Only response + statuses with at least 1 response are reported. New response + statuses can be added at any time. The current list of response + statuses is as follows: + + "ok": a network status request is answered; this number + corresponds to the sum of all requests as reported in + "dirreq-v2-reqs" or "dirreq-v3-reqs", respectively, before + rounding up. + "not-enough-sigs: a version 3 network status is not signed by a + sufficient number of requested authorities. + "unavailable": a requested network status object is unavailable. + "not-found": a requested network status is not found. + "not-modified": a network status has not been modified since the + If-Modified-Since time that is included in the request. + "busy": the directory is busy. + + "dirreq-v2-direct-dl" key=NUM,... NL + [At most once.] + "dirreq-v3-direct-dl" key=NUM,... NL + [At most once.] + "dirreq-v2-tunneled-dl" key=NUM,... NL + [At most once.] + "dirreq-v3-tunneled-dl" key=NUM,... NL + [At most once.] + + List of statistics about possible failures in the download process + of v2/v3 network statuses. Requests are either "direct" + HTTP-encoded requests over the relay's directory port, or + "tunneled" requests using a BEGIN_DIR cell over the relay's OR + port. The list of possible statistics can change, and statistics + can be left out from reporting. The current list of statistics is + as follows: + + Successful downloads and failures: + + "complete": a client has finished the download successfully. + "timeout": a download did not finish within 10 minutes after + starting to send the response. + "running": a download is still running at the end of the + measurement period for less than 10 minutes after starting to + send the response. + + Download times: + + "min", "max": smallest and largest measured bandwidth in B/s. + "d[1-4,6-9]": 1st to 4th and 6th to 9th decile of measured + bandwidth in B/s. For a given decile i, i/10 of all downloads + had a smaller bandwidth than di, and (10-i)/10 of all downloads + had a larger bandwidth than di. + "q[1,3]": 1st and 3rd quartile of measured bandwidth in B/s. One + fourth of all downloads had a smaller bandwidth than q1, one + fourth of all downloads had a larger bandwidth than q3, and the + remaining half of all downloads had a bandwidth between q1 and + q3. + "md": median of measured bandwidth in B/s. Half of the downloads + had a smaller bandwidth than md, the other half had a larger + bandwidth than md. + + "dirreq-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL + [At most once] + "dirreq-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL + [At most once] + + Declare how much bandwidth the OR has spent on answering directory + requests. Usage is divided into intervals of NSEC seconds. The + YYYY-MM-DD HH:MM:SS field defines the end of the most recent + interval. The numbers are the number of bytes used in the most + recent intervals, ordered from oldest to newest. + + "entry-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + An "entry-stats-end" line, as well as any other "entry-*" + line, is first added after the relay has been running for at least + 24 hours. + + "entry-ips" CC=NUM,CC=NUM,... NL + [At most once.] + + List of mappings from two-letter country codes to the number of + unique IP addresses that have connected from that country to the + relay and which are no known other relays, rounded up to the + nearest multiple of 8. + + "cell-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + A "cell-stats-end" line, as well as any other "cell-*" line, + is first added after the relay has been running for at least 24 + hours. + + "cell-processed-cells" NUM,...,NUM NL + [At most once.] + + Mean number of processed cells per circuit, subdivided into + deciles of circuits by the number of cells they have processed in + descending order from loudest to quietest circuits. + + "cell-queued-cells" NUM,...,NUM NL + [At most once.] + + Mean number of cells contained in queues by circuit decile. These + means are calculated by 1) determining the mean number of cells in + a single circuit between its creation and its termination and 2) + calculating the mean for all circuits in a given decile as + determined in "cell-processed-cells". Numbers have a precision of + two decimal places. + + Note that this statistic can be inaccurate for circuits that had + queued cells at the start or end of the measurement interval. + + "cell-time-in-queue" NUM,...,NUM NL + [At most once.] + + Mean time cells spend in circuit queues in milliseconds. Times are + calculated by 1) determining the mean time cells spend in the + queue of a single circuit and 2) calculating the mean for all + circuits in a given decile as determined in + "cell-processed-cells". + + Note that this statistic can be inaccurate for circuits that had + queued cells at the start or end of the measurement interval. + + "cell-circuits-per-decile" NUM NL + [At most once.] + + Mean number of circuits that are included in any of the deciles, + rounded up to the next integer. + + "conn-bi-direct" YYYY-MM-DD HH:MM:SS (NSEC s) BELOW,READ,WRITE,BOTH NL + [At most once] + + Number of connections, split into 10-second intervals, that are + used uni-directionally or bi-directionally as observed in the NSEC + seconds (usually 86400 seconds) before YYYY-MM-DD HH:MM:SS. Every + 10 seconds, we determine for every connection whether we read and + wrote less than a threshold of 20 KiB (BELOW), read at least 10 + times more than we wrote (READ), wrote at least 10 times more than + we read (WRITE), or read and wrote more than the threshold, but + not 10 times more in either direction (BOTH). After classifying a + connection, read and write counters are reset for the next + 10-second interval. + + This measurement includes both IPv4 and IPv6 connections. + + "ipv6-conn-bi-direct" YYYY-MM-DD HH:MM:SS (NSEC s) BELOW,READ,WRITE,BOTH NL + [At most once] + + Number of IPv6 connections that are used uni-directionally or + bi-directionally. See "conn-bi-direct" for more details. + + "exit-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + An "exit-stats-end" line, as well as any other "exit-*" line, is + first added after the relay has been running for at least 24 hours + and only if the relay permits exiting (where exiting to a single + port and IP address is sufficient). + + "exit-kibibytes-written" port=N,port=N,... NL + [At most once.] + "exit-kibibytes-read" port=N,port=N,... NL + [At most once.] + + List of mappings from ports to the number of kibibytes that the + relay has written to or read from exit connections to that port, + rounded up to the next full kibibyte. Relays may limit the + number of listed ports and subsume any remaining kibibytes under + port "other". + + "exit-streams-opened" port=N,port=N,... NL + [At most once.] + + List of mappings from ports to the number of opened exit streams + to that port, rounded up to the nearest multiple of 4. Relays may + limit the number of listed ports and subsume any remaining opened + streams under port "other". + + "hidserv-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + "hidserv-v3-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). + + A "hidserv-stats-end" line, as well as any other "hidserv-*" line, + is first added after the relay has been running for at least 24 + hours. + + (Introduced in tor-0.4.6.1-alpha) + + "hidserv-rend-relayed-cells" SP NUM SP key=val SP key=val ... NL + [At most once.] + "hidserv-rend-v3-relayed-cells" SP NUM SP key=val SP key=val ... NL + [At most once.] + + Approximate number of RELAY cells seen in either direction on a + circuit after receiving and successfully processing a RENDEZVOUS1 + cell. + + The original measurement value is obfuscated in several steps: + first, it is rounded up to the nearest multiple of 'bin_size' + which is reported in the key=val part of this line; second, a + (possibly negative) noise value is added to the result of the + first step by randomly sampling from a Laplace distribution with + mu = 0 and b = (delta_f / epsilon) with 'delta_f' and 'epsilon' + being reported in the key=val part, too; third, the result of the + previous obfuscation steps is truncated to the next smaller + integer and included as 'NUM'. Note that the overall reported + value can be negative. + + (Introduced in tor-0.4.6.1-alpha) + + "hidserv-dir-onions-seen" SP NUM SP key=val SP key=val ... NL + [At most once.] + "hidserv-dir-v3-onions-seen" SP NUM SP key=val SP key=val ... NL + [At most once.] + + Approximate number of unique hidden-service identities seen in + descriptors published to and accepted by this hidden-service + directory. + + The original measurement value is obfuscated in the same way as + the 'NUM' value reported in "hidserv-rend-relayed-cells", but + possibly with different parameters as reported in the key=val part + of this line. Note that the overall reported value can be + negative. + + (Introduced in tor-0.4.6.1-alpha) + + "transport" transportname address:port [arglist] NL + [Any number.] + + Signals that the router supports the 'transportname' pluggable + transport in IP address 'address' and TCP port 'port'. A single + descriptor MUST not have more than one transport line with the + same 'transportname'. + + Pluggable transports are only relevant to bridges, but these entries + can appear in non-bridge relays as well. + + "padding-counts" YYYY-MM-DD HH:MM:SS (NSEC s) key=NUM key=NUM ... NL + [At most once.] + + YYYY-MM-DD HH:MM:SS defines the end of the included measurement + interval of length NSEC seconds (86400 seconds by default). Counts + are reset to 0 at the end of this interval. + + The keyword list is currently as follows: + + bin-size + - The current rounding value for cell count fields (10000 by + default) + write-drop + - The number of RELAY_DROP cells this relay sent + write-pad + - The number of CELL_PADDING cells this relay sent + write-total + - The total number of cells this relay cent + read-drop + - The number of RELAY_DROP cells this relay received + read-pad + - The number of CELL_PADDING cells this relay received + read-total + - The total number of cells this relay received + enabled-read-pad + - The number of CELL_PADDING cells this relay received on + connections that support padding + enabled-read-total + - The total number of cells this relay received on connections + that support padding + enabled-write-pad + - The total number of cells this relay received on connections + that support padding + enabled-write-total + - The total number of cells sent by this relay on connections + that support padding + max-chanpad-timers + - The maximum number of timers that this relay scheduled for + padding in the previous NSEC interval + + "overload-ratelimits" SP version SP YYYY-MM-DD SP HH:MM:SS + SP rate-limit SP burst-limit + SP read-overload-count SP write-overload-count NL + [At most once.] + + Indicates that a bandwidth limit was exhausted for this relay. + + The "rate-limit" and "burst-limit" are the raw values from the + BandwidthRate and BandwidthBurst found in the torrc configuration file. + + The "{read|write}-overload-count" are the counts of how many times the + reported limits of burst/rate were exhausted and thus the maximum + between the read and write count occurrences. To make the counter more + meaningful and to avoid multiple connections saturating the counter + when a relay is overloaded, we only increment it once a minute. + + The 'version' field is set to '1' for now. + + (Introduced in tor-0.4.6.1-alpha) + + "overload-fd-exhausted" SP version YYYY-MM-DD HH:MM:SS NL + [At most once.] + + Indicates that a file descriptor exhaustion was experienced by this + relay. + + The timestamp indicates that the maximum was reached between the + timestamp and the "published" timestamp of the document. + + This overload field should remain in place for 72 hours since last + triggered. If the limits are reached again in this period, the + timestamp is updated, and this 72 hour period restarts. + + The 'version' field is set to '1' for the initial implementation which + detects fd exhaustion only when a socket open fails. + + (Introduced in tor-0.4.6.1-alpha) + + "router-sig-ed25519" + [As in router descriptors] + + "router-signature" NL Signature NL + [At end, exactly once.] + [No extra arguments] + + A document signature as documented in section 1.3, using the + initial item "extra-info" and the final item "router-signature", + signed with the router's identity key. + +2.1.3. Nonterminals in server descriptors + + nickname ::= between 1 and 19 alphanumeric characters ([A-Za-z0-9]), + case-insensitive. + hexdigest ::= a '$', followed by 40 hexadecimal characters + ([A-Fa-f0-9]). [Represents a relay by the digest of its identity + key.] + + exitpattern ::= addrspec ":" portspec + portspec ::= "*" | port | port "-" port + port ::= an integer between 1 and 65535, inclusive. + + [Some implementations incorrectly generate ports with value 0. + Implementations SHOULD accept this, and SHOULD NOT generate it. + Connections to port 0 are never permitted.] + + addrspec ::= "*" | ip4spec | ip6spec + ipv4spec ::= ip4 | ip4 "/" num_ip4_bits | ip4 "/" ip4mask + ip4 ::= an IPv4 address in dotted-quad format + ip4mask ::= an IPv4 mask in dotted-quad format + num_ip4_bits ::= an integer between 0 and 32 + ip6spec ::= ip6 | ip6 "/" num_ip6_bits + ip6 ::= an IPv6 address, surrounded by square brackets. + num_ip6_bits ::= an integer between 0 and 128 + + bool ::= "0" | "1" + +3. Directory authority operation and formats + + Every authority has two keys used in this protocol: a signing key, and + an authority identity key. (Authorities also have a router identity + key used in their role as a router and by earlier versions of the + directory protocol.) The identity key is used from time to time to + sign new key certificates using new signing keys; it is very sensitive. + The signing key is used to sign key certificates and status documents. + +3.1. Creating key certificates + + Key certificates consist of the following items: + + "dir-key-certificate-version" version NL + + [At start, exactly once.] + + Determines the version of the key certificate. MUST be "3" for + the protocol described in this document. Implementations MUST + reject formats they don't understand. + + "dir-address" IPPort NL + [At most once] + + An IP:Port for this authority's directory port. + + "fingerprint" fingerprint NL + + [Exactly once.] + + Hexadecimal encoding without spaces based on the authority's + identity key. + + "dir-identity-key" NL a public key in PEM format + + [Exactly once.] + [No extra arguments] + + The long-term authority identity key for this authority. This key + SHOULD be at least 2048 bits long; it MUST NOT be shorter than + 1024 bits. + + "dir-key-published" YYYY-MM-DD HH:MM:SS NL + + [Exactly once.] + + The time (in UTC) when this document and corresponding key were + last generated. + + Implementations SHOULD reject certificates that are published + too far in the future, though they MAY tolerate some clock skew. + + "dir-key-expires" YYYY-MM-DD HH:MM:SS NL + + [Exactly once.] + + A time (in UTC) after which this key is no longer valid. + + Implementations SHOULD reject expired certificates, though they + MAY tolerate some clock skew. + + "dir-signing-key" NL a key in PEM format + + [Exactly once.] + [No extra arguments] + + The directory server's public signing key. This key MUST be at + least 1024 bits, and MAY be longer. + + "dir-key-crosscert" NL CrossSignature NL + + [Exactly once.] + [No extra arguments] + + CrossSignature is a signature, made using the certificate's signing + key, of the digest of the PKCS1-padded hash of the certificate's + identity key. For backward compatibility with broken versions of the + parser, we wrap the base64-encoded signature in -----BEGIN ID + SIGNATURE---- and -----END ID SIGNATURE----- tags. Implementations + MUST allow the "ID " portion to be omitted, however. + + Implementations MUST verify that the signature is a correct signature + of the hash of the identity key using the signing key. + + "dir-key-certification" NL Signature NL + + [At end, exactly once.] + [No extra arguments] + + A document signature as documented in section 1.3, using the + initial item "dir-key-certificate-version" and the final item + "dir-key-certification", signed with the authority identity key. + + Authorities MUST generate a new signing key and corresponding + certificate before the key expires. + +3.2. Accepting server descriptor and extra-info document uploads + + When a router posts a signed descriptor to a directory authority, the + authority first checks whether it is well-formed and correctly + self-signed. If it is, the authority next verifies that the nickname + in question is not already assigned to a router with a different + public key. + Finally, the authority MAY check that the router is not blacklisted + because of its key, IP, or another reason. + + An authority also keeps a record of all the Ed25519/RSA1024 + identity key pairs that it has seen before. It rejects any + descriptor that has a known Ed/RSA identity key that it has + already seen accompanied by a different RSA/Ed identity key + in an older descriptor. + + At a future date, authorities will begin rejecting all + descriptors whose RSA key was previously accompanied by an + Ed25519 key, if the descriptor does not list an Ed25519 key. + + At a future date, authorities will begin rejecting all descriptors + that do not list an Ed25519 key. + + If the descriptor passes these tests, and the authority does not already + have a descriptor for a router with this public key, it accepts the + descriptor and remembers it. + + If the authority _does_ have a descriptor with the same public key, the + newly uploaded descriptor is remembered if its publication time is more + recent than the most recent old descriptor for that router, and either: + + - There are non-cosmetic differences between the old descriptor and the + new one. + - Enough time has passed between the descriptors' publication times. + (Currently, 2 hours.) + + Differences between server descriptors are "non-cosmetic" if they would be + sufficient to force an upload as described in section 2.1 above. + + Note that the "cosmetic difference" test only applies to uploaded + descriptors, not to descriptors that the authority downloads from other + authorities. + + When a router posts a signed extra-info document to a directory authority, + the authority again checks it for well-formedness and correct signature, + and checks that its matches the extra-info-digest in some router + descriptor that it believes is currently useful. If so, it accepts it and + stores it and serves it as requested. If not, it drops it. + + +3.3. Computing microdescriptors + + Microdescriptors are a stripped-down version of server descriptors + generated by the directory authorities which may additionally contain + authority-generated information. Microdescriptors contain only the + most relevant parts that clients care about. Microdescriptors are + expected to be relatively static and only change about once per week. + Microdescriptors do not contain any information that clients need to + use to decide which servers to fetch information about, or which + servers to fetch information from. + + Microdescriptors are a straight transform from the server descriptor + and the consensus method. Microdescriptors have no header or footer. + Microdescriptors are identified by the hash of its concatenated + elements without a signature by the router. Microdescriptors do not + contain any version information, because their version is determined + by the consensus method. + + Starting with consensus method 8, microdescriptors contain the + following elements taken from or based on the server descriptor. Order + matters here, because different directory authorities must be able to + transform a given server descriptor and consensus method into the exact + same microdescriptor. + + "onion-key" NL a public key in PEM format + + [Exactly once, at start] + [No extra arguments] + + The "onion-key" element as specified in section 2.1.1. + + When generating microdescriptors for consensus method 30 or later, + the trailing = sign must be absent. For consensus method 29 or + earlier, the trailing = sign must be present. + + "ntor-onion-key" SP base-64-encoded-key NL + + [Exactly once] + + The "ntor-onion-key" element as specified in section 2.1.1. + + (Only included when generating microdescriptors for + consensus-method 16 or later.) + + [Before Tor 0.4.5.1-alpha, this field was optional.] + + "a" SP address ":" port NL + + [Any number] + + Additional advertised addresses for the OR. + + Present currently only if the OR advertises at least one IPv6 + address; currently, the first address is included and all others are + omitted. Any other IPv4 or IPv6 addresses should be ignored. + + Address and port are as for "or-address" as specified in + section 2.1.1. + + (Only included when generating microdescriptors for + consensus-methods 14 to 27.) + + "family" names NL + + [At most once] + + The "family" element as specified in section 2.1.1. + + When generating microdescriptors for consensus method 29 or later, + the following canonicalization algorithm is applied to improve + compression: + + For all entries of the form $hexid=name or $hexid~name, + remove the =name or ~name portion. + + Remove all entries of the form $hexid, where hexid is not + 40 hexadecimal characters long. + + If an entry is a valid nickname, put it into lower case. + + If an entry is a valid $hexid, put it into upper case. + + If there are any entries, add a single $hexid entry for + the relay in question, so that it is a member of its own + family. + + Sort all entries in lexical order. + + Remove duplicate entries. + + (Note that if an entry is not of the form "nickname", "$hexid", + "$hexid=nickname" or "$hexid~nickname", then it will be unchanged: + this is what makes the algorithm forward-compatible.) + + "p" SP ("accept" / "reject") SP PortList NL + + [Exactly once.] + + The exit-policy summary as specified in sections 3.4.1 and 3.8.2. + + [With microdescriptors, clients don't learn exact exit policies: + clients can only guess whether a relay accepts their request, try the + BEGIN request, and might get end-reason-exit-policy if they guessed + wrong, in which case they'll have to try elsewhere.] + + [In consensus methods before 5, this line was omitted.] + + "p6" SP ("accept" / "reject") SP PortList NL + + [At most once] + + The IPv6 exit policy summary as specified in sections 3.4.1 and + 3.8.2. A missing "p6" line is equivalent to "p6 reject 1-65535". + + (Only included when generating microdescriptors for + consensus-method 15 or later.) + + "id" SP "rsa1024" SP base64-encoded-identity-digest NL + + [At most once] + + The node identity digest (as described in tor-spec.txt), base64 + encoded, without trailing =s. This line is included to prevent + collisions between microdescriptors. + + Implementations SHOULD ignore these lines: they are + added to microdescriptors only to prevent collisions. + + (Only included when generating microdescriptors for + consensus-method 18 or later.) + + "id" SP "ed25519" SP base64-encoded-ed25519-identity NL + + [At most once] + + The node's master Ed25519 identity key, base64 encoded, + without trailing =s. + + All implementations MUST ignore this key for any microdescriptor + whose corresponding entry in the consensus includes the + 'NoEdConsensus' flag. + + (Only included when generating microdescriptors for + consensus-method 21 or later.) + + "id" SP keytype ... NL + + [At most once per distinct keytype.] + + Implementations MUST ignore "id" lines with unrecognized + key-types in place of "rsa1024" or "ed25519" + + "pr" SP Entries NL + + [Exactly once.] + + The "proto" element as specified in section 2.1.1. + + [Before Tor 0.4.5.1-alpha, this field was optional.] + + (Note that with microdescriptors, clients do not learn the RSA identity of + their routers: they only learn a hash of the RSA identity key. This is + all they need to confirm the actual identity key when doing a TLS + handshake, and all they need to put the identity key digest in their + CREATE cells.) + +3.4. Exchanging votes + + Authorities divide time into Intervals. Authority administrators SHOULD + try to all pick the same interval length, and SHOULD pick intervals that + are commonly used divisions of time (e.g., 5 minutes, 15 minutes, 30 + minutes, 60 minutes, 90 minutes). Voting intervals SHOULD be chosen to + divide evenly into a 24-hour day. + + Authorities SHOULD act according to interval and delays in the + latest consensus. Lacking a latest consensus, they SHOULD default to a + 30-minute Interval, a 5 minute VotingDelay, and a 5 minute DistDelay. + + Authorities MUST take pains to ensure that their clocks remain accurate + within a few seconds. (Running NTP is usually sufficient.) + + The first voting period of each day begins at 00:00 (midnight) UTC. If + the last period of the day would be truncated by one-half or more, it is + merged with the second-to-last period. + + An authority SHOULD publish its vote immediately at the start of each voting + period (minus VoteSeconds+DistSeconds). It does this by making it + available at + + http://<hostname>/tor/status-vote/next/authority.z + + and sending it in an HTTP POST request to each other authority at the URL + + http://<hostname>/tor/post/vote + + If, at the start of the voting period, minus DistSeconds, an authority + does not have a current statement from another authority, the first + authority downloads the other's statement. + + Once an authority has a vote from another authority, it makes it available + at + + http://<hostname>/tor/status-vote/next/<fp>.z + + where <fp> is the fingerprint of the other authority's identity key. + And at + + http://<hostname>/tor/status-vote/next/d/<d>.z + + where <d> is the digest of the vote document. + + Also, once an authority receives a vote from another authority, it + examines it for new descriptors and fetches them from that authority. + This may be the only way for an authority to hear about relays that didn't + publish their descriptor to all authorities, and, while it's too late + for the authority to include relays in its current vote, it can include + them in its next vote. See section 3.6 below for details. + +3.4.1. Vote and consensus status document formats + + Votes and consensuses are more strictly formatted than other documents + in this specification, since different authorities must be able to + generate exactly the same consensus given the same set of votes. + + The procedure for deciding when to generate vote and consensus status + documents are described in section 1.4 on the voting timeline. + + Status documents contain a preamble, an authority section, a list of + router status entries, and one or more footer signature, in that order. + + Unlike other formats described above, a SP in these documents must be a + single space character (hex 20). + + Some items appear only in votes, and some items appear only in + consensuses. Unless specified, items occur in both. + + The preamble contains the following items. They SHOULD occur in the + order given here: + + "network-status-version" SP version NL + + [At start, exactly once.] + + A document format version. For this specification, the version is + "3". + + "vote-status" SP type NL + + [Exactly once.] + + The status MUST be "vote" or "consensus", depending on the type of + the document. + + "consensus-methods" SP IntegerList NL + + [At most once for votes; does not occur in consensuses.] + + A space-separated list of supported methods for generating + consensuses from votes. See section 3.8.1 for details. Absence of + the line means that only method "1" is supported. + + "consensus-method" SP Integer NL + + [At most once for consensuses; does not occur in votes.] + [No extra arguments] + + See section 3.8.1 for details. + + (Only included when the vote is generated with consensus-method 2 or + later.) + + "published" SP YYYY-MM-DD SP HH:MM:SS NL + + [Exactly once for votes; does not occur in consensuses.] + + The publication time for this status document (if a vote). + + "valid-after" SP YYYY-MM-DD SP HH:MM:SS NL + + [Exactly once.] + + The start of the Interval for this vote. Before this time, the + consensus document produced from this vote is not officially in + use. + + (Note that because of propagation delays, clients and relays + may see consensus documents that are up to `DistSeconds` + earlier than this time, and should not warn about them.) + + See section 1.4 for voting timeline information. + + "fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL + + [Exactly once.] + + The time at which the next consensus should be produced; before this + time, there is no point in downloading another consensus, since there + won't be a new one. See section 1.4 for voting timeline information. + + "valid-until" SP YYYY-MM-DD SP HH:MM:SS NL + + [Exactly once.] + + The end of the Interval for this vote. After this time, all + clients should try to find a more recent consensus. See section 1.4 + for voting timeline information. + + In practice, clients continue to use the consensus for up to 24 hours + after it is no longer valid, if no more recent consensus can be + downloaded. + + "voting-delay" SP VoteSeconds SP DistSeconds NL + + [Exactly once.] + + VoteSeconds is the number of seconds that we will allow to collect + votes from all authorities; DistSeconds is the number of seconds + we'll allow to collect signatures from all authorities. See + section 1.4 for voting timeline information. + + "client-versions" SP VersionList NL + + [At most once.] + + A comma-separated list of recommended Tor versions for client + usage, in ascending order. The versions are given as defined by + version-spec.txt. If absent, no opinion is held about client + versions. + + "server-versions" SP VersionList NL + + [At most once.] + + A comma-separated list of recommended Tor versions for relay + usage, in ascending order. The versions are given as defined by + version-spec.txt. If absent, no opinion is held about server + versions. + + "package" SP PackageName SP Version SP URL SP DIGESTS NL + + [Any number of times.] + + For this element: + + PACKAGENAME = NONSPACE + VERSION = NONSPACE + URL = NONSPACE + DIGESTS = DIGEST | DIGESTS SP DIGEST + DIGEST = DIGESTTYPE "=" DIGESTVAL + NONSPACE = one or more non-space printing characters + DIGESTVAL = DIGESTTYPE = one or more non-space printing characters + other than "=". + + Indicates that a package called "package" of version VERSION may be + found at URL, and its digest as computed with DIGESTTYPE is equal to + DIGESTVAL. In consensuses, these lines are sorted lexically by + "PACKAGENAME VERSION" pairs, and DIGESTTYPES must appear in ascending + order. A consensus must not contain the same "PACKAGENAME VERSION" + more than once. If a vote contains the same "PACKAGENAME VERSION" + more than once, all but the last is ignored. + + Included in consensuses only for methods 19-33. Earlier methods + did not include this; method 34 removed it. + + "known-flags" SP FlagList NL + + [Exactly once.] + + A space-separated list of all of the flags that this document + might contain. A flag is "known" either because the authority + knows about them and might set them (if in a vote), or because + enough votes were counted for the consensus for an authoritative + opinion to have been formed about their status. + + "flag-thresholds" SP Thresholds NL + + [At most once for votes; does not occur in consensuses.] + + A space-separated list of the internal performance thresholds + that the directory authority had at the moment it was forming + a vote. + + The metaformat is: + Thresholds = Threshold | Threshold SP Thresholds + Threshold = ThresholdKey '=' ThresholdVal + ThresholdKey = (KeywordChar | "_") + + ThresholdVal = [0-9]+("."[0-9]+)? "%"? + + Commonly used Thresholds at this point include: + + "stable-uptime" -- Uptime (in seconds) required for a relay + to be marked as stable. + + "stable-mtbf" -- MTBF (in seconds) required for a relay to be + marked as stable. + + "enough-mtbf" -- Whether we have measured enough MTBF to look + at stable-mtbf instead of stable-uptime. + + "fast-speed" -- Bandwidth (in bytes per second) required for + a relay to be marked as fast. + + "guard-wfu" -- WFU (in seconds) required for a relay to be + marked as guard. + + "guard-tk" -- Weighted Time Known (in seconds) required for a + relay to be marked as guard. + + "guard-bw-inc-exits" -- If exits can be guards, then all guards + must have a bandwidth this high. + + "guard-bw-exc-exits" -- If exits can't be guards, then all guards + must have a bandwidth this high. + + "ignoring-advertised-bws" -- 1 if we have enough measured bandwidths + that we'll ignore the advertised bandwidth + claims of routers without measured bandwidth. + + "recommended-client-protocols" SP Entries NL + "recommended-relay-protocols" SP Entries NL + "required-client-protocols" SP Entries NL + "required-relay-protocols" SP Entries NL + + [At most once for each.] + + The "proto" element as specified in section 2.1.1. + + To vote on these entries, a protocol/version combination is included + only if it is listed by a majority of the voters. + + These lines should be voted on. A majority of votes is sufficient to + make a protocol un-supported. A supermajority of authorities (2/3) + are needed to make a protocol required. The required protocols + should not be torrc-configurable, but rather should be hardwired in + the Tor code. + + The tor-spec.txt section 9 details how a relay and a client should + behave when they encounter these lines in the consensus. + + "params" SP [Parameters] NL + + [At most once] + + Parameter ::= Keyword '=' Int32 + Int32 ::= A decimal integer between -2147483648 and 2147483647. + Parameters ::= Parameter | Parameters SP Parameter + + The parameters list, if present, contains a space-separated list of + case-sensitive key-value pairs, sorted in lexical order by their + keyword (as ASCII byte strings). Each parameter has its own meaning. + + (Only included when the vote is generated with consensus-method 7 or + later.) + + See param-spec.txt for a list of parameters and their meanings. + + "shared-rand-previous-value" SP NumReveals SP Value NL + + [At most once] + + NumReveals ::= An integer greater or equal to 0. + Value ::= Base64-encoded-data + + The shared_random_value that was generated during the second-to-last + shared randomness protocol run. For example, if this document was + created on the 5th of November, this field carries the shared random + value generated during the protocol run of the 3rd of November. + + See section [SRCALC] of srv-spec.txt for instructions on how to compute + this value, and see section [CONS] for why we include old shared random + values in votes and consensus. + + Value is the actual shared random value encoded in base64. It will + be exactly 256 bits long. NumReveals is the number of commits used + to generate this SRV. + + "shared-rand-current-value" SP NumReveals SP Value NL + + [At most once] + + NumReveals ::= An integer greater or equal to 0. + Value ::= Base64-encoded-data + + The shared_random_value that was generated during the latest shared + randomness protocol run. For example, if this document was created on + the 5th of November, this field carries the shared random value + generated during the protocol run of the 4th of November + + See section [SRCALC] of srv-spec.txt for instructions on how to compute + this value given the active commits. + + Value is the actual shared random value encoded in base64. It will + be exactly 256 bits long. NumReveals is the number of commits used to + generate this SRV. + + "bandwidth-file-headers" SP KeyValues NL + + [At most once for votes; does not occur in consensuses.] + + KeyValues ::= "" | KeyValue | KeyValues SP KeyValue + KeyValue ::= Keyword '=' Value + Value ::= ArgumentCharValue+ + ArgumentCharValue ::= any printing ASCII character except NL and SP. + + The headers from the bandwidth file used to generate this vote. + The bandwidth file headers are described in bandwidth-file-spec.txt. + + If an authority is not configured with a V3BandwidthsFile, this line + SHOULD NOT appear in its vote. + + If an authority is configured with a V3BandwidthsFile, but parsing + fails, this line SHOULD appear in its vote, but without any headers. + + First-appeared: Tor 0.3.5.1-alpha. + + "bandwidth-file-digest" 1*(SP algorithm "=" digest) NL + + [At most once for votes; does not occur in consensuses.] + + A digest of the bandwidth file used to generate this vote. + "algorithm" is the name of the hash algorithm producing "digest", + which can be "sha256" or another algorithm. "digest" is the + base64 encoding of the hash of the bandwidth file, with trailing =s + omitted. + + If an authority is not configured with a V3BandwidthsFile, this line + SHOULD NOT appear in its vote. + + If an authority is configured with a V3BandwidthsFile, but parsing + fails, this line SHOULD appear in its vote, with the digest(s) of the + unparseable file. + + First-appeared: Tor 0.4.0.4-alpha + + The authority section of a vote contains the following items, followed + in turn by the authority's current key certificate: + + "dir-source" SP nickname SP identity SP address SP IP SP dirport SP + orport NL + + [Exactly once, at start] + + Describes this authority. The nickname is a convenient identifier + for the authority. The identity is an uppercase hex fingerprint of + the authority's current (v3 authority) identity key. The address is + the server's hostname. The IP is the server's current IP address, + and dirport is its current directory port. The orport is the + port at that address where the authority listens for OR + connections. + + "contact" SP string NL + + [Exactly once] + + An arbitrary string describing how to contact the directory + server's administrator. Administrators should include at least an + email address and a PGP fingerprint. + + "legacy-dir-key" SP FINGERPRINT NL + + [At most once] + + Lists a fingerprint for an obsolete _identity_ key still used + by this authority to keep older clients working. This option + is used to keep key around for a little while in case the + authorities need to migrate many identity keys at once. + (Generally, this would only happen because of a security + vulnerability that affected multiple authorities, like the + Debian OpenSSL RNG bug of May 2008.) + + "shared-rand-participate" NL + + [At most once] + + Denotes that the directory authority supports and can participate in the + shared random protocol. + + "shared-rand-commit" SP Version SP AlgName SP Identity SP Commit [SP Reveal] NL + + [Any number of times] + + Version ::= An integer greater or equal to 0. + AlgName ::= 1*(ALPHA / DIGIT / "_" / "-") + Identity ::= 40 * HEXDIG + Commit ::= Base64-encoded-data + Reveal ::= Base64-encoded-data + + Denotes a directory authority commit for the shared randomness + protocol, containing the commitment value and potentially also the + reveal value. See sections [COMMITREVEAL] and [VALIDATEVALUES] of + srv-spec.txt on how to generate and validate these values. + + Version is the current shared randomness protocol version. AlgName is + the hash algorithm that is used (e.g. "sha3-256") and Identity is the + authority's SHA1 v3 identity fingerprint. Commit is the encoded + commitment value in base64. Reveal is optional and if it's set, it + contains the reveal value in base64. + + If a vote contains multiple commits from the same authority, the + receiver MUST only consider the first commit listed. + + "shared-rand-previous-value" SP NumReveals SP Value NL + + [At most once] + + See shared-rand-previous-value description above. + + "shared-rand-current-value" SP NumReveals SP Value NL + + [At most once] + + See shared-rand-current-value description above. + + The authority section of a consensus contains groups of the following items, + in the order given, with one group for each authority that contributed to + the consensus, with groups sorted by authority identity digest: + + "dir-source" SP nickname SP identity SP address SP IP SP dirport SP + orport NL + + [Exactly once, at start] + + As in the authority section of a vote. + + "contact" SP string NL + + [Exactly once.] + + As in the authority section of a vote. + + "vote-digest" SP digest NL + + [Exactly once.] + + A digest of the vote from the authority that contributed to this + consensus, as signed (that is, not including the signature). + (Hex, upper-case.) + + For each "legacy-dir-key" in the vote, there is an additional "dir-source" + line containing that legacy key's fingerprint, the authority's nickname + with "-legacy" appended, and all other fields as in the main "dir-source" + line for that authority. These "dir-source" lines do not have + corresponding "contact" or "vote-digest" entries. + + Each router status entry contains the following items. Router status + entries are sorted in ascending order by identity digest. + + "r" SP nickname SP identity SP digest SP publication SP IP SP ORPort + SP DirPort NL + + [At start, exactly once.] + + "Nickname" is the OR's nickname. "Identity" is a hash of its + identity key, encoded in base64, with trailing equals sign(s) + removed. "Digest" is a hash of its most recent descriptor as + signed (that is, not including the signature) by the RSA identity + key (see section 1.3.), encoded in base64. + + "Publication" was once the publication time of the router's most + recent descriptor, in the form YYYY-MM-DD HH:MM:SS, in UTC. Now + it is only used in votes, and may be set to a fixed value in + consensus documents. Implementations SHOULD ignore this value + in non-vote documents. + + "IP" is its current IP address; ORPort is its current OR port, + "DirPort" is its current directory port, or "0" for "none". + + "a" SP address ":" port NL + + [Any number] + + The first advertised IPv6 address for the OR, if it is reachable. + + Present only if the OR advertises at least one IPv6 address, and the + authority believes that the first advertised address is reachable. + Any other IPv4 or IPv6 addresses should be ignored. + + Address and port are as for "or-address" as specified in + section 2.1.1. + + (Only included when the vote or consensus is generated with + consensus-method 14 or later.) + + "s" SP Flags NL + + [Exactly once.] + + A series of space-separated status flags, in lexical order (as ASCII + byte strings). Currently documented flags are: + + "Authority" if the router is a directory authority. + "BadExit" if the router is believed to be useless as an exit node + (because its ISP censors it, because it is behind a restrictive + proxy, or for some similar reason). + "Exit" if the router is more useful for building + general-purpose exit circuits than for relay circuits. The + path building algorithm uses this flag; see path-spec.txt. + "Fast" if the router is suitable for high-bandwidth circuits. + "Guard" if the router is suitable for use as an entry guard. + "HSDir" if the router is considered a v2 hidden service directory. + "MiddleOnly" if the router is considered unsuitable for + usage other than as a middle relay. Clients do not need + to handle this option, since when it is present, the authorities + will automatically vote against flags that would make the router + usable in other positions. (Since 0.4.7.2-alpha.) + "NoEdConsensus" if any Ed25519 key in the router's descriptor or + microdescriptor does not reflect authority consensus. + "Stable" if the router is suitable for long-lived circuits. + "StaleDesc" if the router should upload a new descriptor because + the old one is too old. + "Running" if the router is currently usable over all its published + ORPorts. (Authorities ignore IPv6 ORPorts unless configured to + check IPv6 reachability.) Relays without this flag are omitted + from the consensus, and current clients (since 0.2.9.4-alpha) + assume that every listed relay has this flag. + "Valid" if the router has been 'validated'. Clients before + 0.2.9.4-alpha would not use routers without this flag by + default. Currently, relays without this flag are omitted + from the consensus, and current (post-0.2.9.4-alpha) clients + assume that every listed relay has this flag. + "V2Dir" if the router implements the v2 directory protocol or + higher. + + "v" SP version NL + + [At most once.] + + The version of the Tor protocol that this relay is running. If + the value begins with "Tor" SP, the rest of the string is a Tor + version number, and the protocol is "The Tor protocol as supported + by the given version of Tor." Otherwise, if the value begins with + some other string, Tor has upgraded to a more sophisticated + protocol versioning system, and the protocol is "a version of the + Tor protocol more recent than any we recognize." + + Directory authorities SHOULD omit version strings they receive from + descriptors if they would cause "v" lines to be over 128 characters + long. + + "pr" SP Entries NL + + [At most once.] + + The "proto" family element as specified in section 2.1.1. + + During voting, authorities copy these lines immediately below the "v" + lines. When a descriptor does not contain a "proto" entry, the + authorities should reconstruct it using the approach described below + in section D. They are included in the consensus using the same rules + as currently used for "v" lines, if a sufficiently late consensus + method is in use. + + "w" SP "Bandwidth=" INT [SP "Measured=" INT] [SP "Unmeasured=1"] NL + + [At most once.] + + An estimate of the bandwidth of this relay, in an arbitrary + unit (currently kilobytes per second). Used to weight router + selection. See section 3.4.2 for details on how the value of + Bandwidth is determined in a consensus. + + Additionally, the Measured= keyword is present in votes by + participating bandwidth measurement authorities to indicate + a measured bandwidth currently produced by measuring stream + capacities. It does not occur in consensuses. + + 'Bandwidth=' and 'Measured=' values must be between 0 and + 2^32 - 1 inclusive. + + The "Unmeasured=1" value is included in consensuses generated + with method 17 or later when the 'Bandwidth=' value is not + based on a threshold of 3 or more measurements for this relay. + + Other weighting keywords may be added later. + Clients MUST ignore keywords they do not recognize. + + "p" SP ("accept" / "reject") SP PortList NL + + [At most once.] + + PortList = PortOrRange + PortList = PortList "," PortOrRange + PortOrRange = INT "-" INT / INT + + A list of those ports that this router supports (if 'accept') + or does not support (if 'reject') for exit to "most + addresses". + + "m" SP methods 1*(SP algorithm "=" digest) NL + + [Any number, only in votes.] + + Microdescriptor hashes for all consensus methods that an authority + supports and that use the same microdescriptor format. "methods" + is a comma-separated list of the consensus methods that the + authority believes will produce "digest". "algorithm" is the name + of the hash algorithm producing "digest", which can be "sha256" or + something else, depending on the consensus "methods" supporting + this algorithm. "digest" is the base64 encoding of the hash of + the router's microdescriptor with trailing =s omitted. + + "id" SP "ed25519" SP ed25519-identity NL + "id" SP "ed25519" SP "none" NL + [vote only, at most once] + + "stats" SP [KeyValues] NL + + [At most once. Vote only] + + KeyValue ::= Keyword '=' Number + Number ::= [0-9]+("."[0-9]+)? + KeyValues ::= KeyValue | KeyValues SP KeyValue + + Line containing various statistics that an authority has computed for + this relay. Each stats is represented as a key + value. Reported keys + are: + + "wfu" - Weighted Fractional Uptime + "tk" - Weighted Time Known + "mtbf" - Mean Time Between Failure (stability) + + (As of tor-0.4.6.1-alpha) + + The footer section is delineated in all votes and consensuses supporting + consensus method 9 and above with the following: + + "directory-footer" NL + [No extra arguments] + + It contains two subsections, a bandwidths-weights line and a + directory-signature. (Prior to consensus method 9, footers only contained + directory-signatures without a 'directory-footer' line or + bandwidth-weights.) + + The bandwidths-weights line appears At Most Once for a consensus. It does + not appear in votes. + + "bandwidth-weights" [SP Weights] NL + + Weight ::= Keyword '=' Int32 + Int32 ::= A decimal integer between -2147483648 and 2147483647. + Weights ::= Weight | Weights SP Weight + + List of optional weights to apply to router bandwidths during path + selection. They are sorted in lexical order (as ASCII byte strings) and + values are divided by the consensus' "bwweightscale" param. Definition + of our known entries are... + + Wgg - Weight for Guard-flagged nodes in the guard position + Wgm - Weight for non-flagged nodes in the guard Position + Wgd - Weight for Guard+Exit-flagged nodes in the guard Position + + Wmg - Weight for Guard-flagged nodes in the middle Position + Wmm - Weight for non-flagged nodes in the middle Position + Wme - Weight for Exit-flagged nodes in the middle Position + Wmd - Weight for Guard+Exit flagged nodes in the middle Position + + Weg - Weight for Guard flagged nodes in the exit Position + Wem - Weight for non-flagged nodes in the exit Position + Wee - Weight for Exit-flagged nodes in the exit Position + Wed - Weight for Guard+Exit-flagged nodes in the exit Position + + Wgb - Weight for BEGIN_DIR-supporting Guard-flagged nodes + Wmb - Weight for BEGIN_DIR-supporting non-flagged nodes + Web - Weight for BEGIN_DIR-supporting Exit-flagged nodes + Wdb - Weight for BEGIN_DIR-supporting Guard+Exit-flagged nodes + + Wbg - Weight for Guard flagged nodes for BEGIN_DIR requests + Wbm - Weight for non-flagged nodes for BEGIN_DIR requests + Wbe - Weight for Exit-flagged nodes for BEGIN_DIR requests + Wbd - Weight for Guard+Exit-flagged nodes for BEGIN_DIR requests + + These values are calculated as specified in section 3.8.3. + + The signature contains the following item, which appears Exactly Once + for a vote, and At Least Once for a consensus. + + "directory-signature" [SP Algorithm] SP identity SP signing-key-digest + NL Signature + + This is a signature of the status document, with the initial item + "network-status-version", and the signature item + "directory-signature", using the signing key. (In this case, we take + the hash through the _space_ after directory-signature, not the + newline: this ensures that all authorities sign the same thing.) + "identity" is the hex-encoded digest of the authority identity key of + the signing authority, and "signing-key-digest" is the hex-encoded + digest of the current authority signing key of the signing authority. + + The Algorithm is one of "sha1" or "sha256" if it is present; + implementations MUST ignore directory-signature entries with an + unrecognized Algorithm. "sha1" is the default, if no Algorithm is + given. The algorithm describes how to compute the hash of the + document before signing it. + + "ns"-flavored consensus documents must contain only sha1 signatures. + Votes and microdescriptor documents may contain other signature + types. Note that only one signature from each authority should be + "counted" as meaning that the authority has signed the consensus. + + (Tor clients before 0.2.3.x did not understand the 'algorithm' + field.) + +3.4.2. Assigning flags in a vote + + (This section describes how directory authorities choose which status + flags to apply to routers. Later directory authorities MAY do things + differently, so long as clients keep working well. Clients MUST NOT + depend on the exact behaviors in this section.) + + In the below definitions, a router is considered "active" if it is + running, valid, and not hibernating. + + When we speak of a router's bandwidth in this section, we mean either + its measured bandwidth, or its advertised bandwidth. If a sufficient + threshold (configurable with MinMeasuredBWsForAuthToIgnoreAdvertised, + 500 by default) of routers have measured bandwidth values, then the + authority bases flags on _measured_ bandwidths, and treats nodes with + non-measured bandwidths as if their bandwidths were zero. Otherwise, + it uses measured bandwidths for nodes that have them, and advertised + bandwidths for other nodes. + + When computing thresholds based on percentiles of nodes, an authority + only considers nodes that are active, that have not been + omitted as a sybil (see below), and whose bandwidth is at least + 4 KB. Nodes that don't meet these criteria do not influence any + threshold calculations (including calculation of stability and uptime + and bandwidth thresholds) and also do not have their Exit status + change. + + "Valid" -- a router is 'Valid' if it is running a version of Tor not + known to be broken, and the directory authority has not blacklisted + it as suspicious. + + "Named" -- + "Unnamed" -- Directory authorities no longer assign these flags. + They were once used to determine whether a relay's nickname was + canonically linked to its public key. + + "Running" -- A router is 'Running' if the authority managed to connect to + it successfully within the last 45 minutes on all its published ORPorts. + Authorities check reachability on: + + * the IPv4 ORPort in the "r" line, and + * the IPv6 ORPort considered for the "a" line, if: + * the router advertises at least one IPv6 ORPort, and + * AuthDirHasIPv6Connectivity 1 is set on the authority. + + A minority of voting authorities that set AuthDirHasIPv6Connectivity will + drop unreachable IPv6 ORPorts from the full consensus. Consensus method 27 + in 0.3.3.x puts IPv6 ORPorts in the microdesc consensus, so that + authorities can drop unreachable IPv6 ORPorts from all consensus flavors. + Consensus method 28 removes IPv6 ORPorts from microdescriptors. + + "Stable" -- A router is 'Stable' if it is active, and either its Weighted + MTBF is at least the median for known active routers or its Weighted MTBF + corresponds to at least 7 days. Routers are never called Stable if they are + running a version of Tor known to drop circuits stupidly. (0.1.1.10-alpha + through 0.1.1.16-rc are stupid this way.) + + To calculate weighted MTBF, compute the weighted mean of the lengths + of all intervals when the router was observed to be up, weighting + intervals by $\alpha^n$, where $n$ is the amount of time that has + passed since the interval ended, and $\alpha$ is chosen so that + measurements over approximately one month old no longer influence the + weighted MTBF much. + + [XXXX what happens when we have less than 4 days of MTBF info.] + + "Exit" -- A router is called an 'Exit' iff it allows exits to at + least one /8 address space on each of ports 80 and 443. (Up until + Tor version 0.3.2, the flag was assigned if relays exit to at least + two of the ports 80, 443, and 6667.) + + "Fast" -- A router is 'Fast' if it is active, and its bandwidth is either in + the top 7/8ths for known active routers or at least 100KB/s. + + "Guard" -- A router is a possible Guard if all of the following apply: + + - It is Fast, + - It is Stable, + - Its Weighted Fractional Uptime is at least the median for "familiar" + active routers, + - It is "familiar", + - Its bandwidth is at least AuthDirGuardBWGuarantee (if set, 2 MB by + default), OR its bandwidth is among the 25% fastest relays, + - It qualifies for the V2Dir flag as described below (this + constraint was added in 0.3.3.x, because in 0.3.0.x clients + started avoiding guards that didn't also have the V2Dir flag). + + To calculate weighted fractional uptime, compute the fraction + of time that the router is up in any given day, weighting so that + downtime and uptime in the past counts less. + + A node is 'familiar' if 1/8 of all active nodes have appeared more + recently than it, OR it has been around for a few weeks. + + "Authority" -- A router is called an 'Authority' if the authority + generating the network-status document believes it is an authority. + + "V2Dir" -- A router supports the v2 directory protocol or higher if it has + an open directory port OR a tunnelled-dir-server line in its router + descriptor, and it is running a version of the directory + protocol that supports the functionality clients need. (Currently, every + supported version of Tor supports the functionality that clients need, + but some relays might set "DirCache 0" or set really low rate limiting, + making them unqualified to be a directory mirror, i.e. they will omit + the tunnelled-dir-server line from their descriptor.) + + "HSDir" -- A router is a v2 hidden service directory if it stores and + serves v2 hidden service descriptors, has the Stable and Fast flag, and the + authority believes that it's been up for at least 96 hours (or the current + value of MinUptimeHidServDirectoryV2). + + "MiddleOnly" -- An authority should vote for this flag if it believes + that a relay is unsuitable for use except as a middle relay. When + voting for this flag, the authority should also vote against "Exit", + "Guard", "HsDir", and "V2Dir". When voting for this flag, if the + authority votes on the "BadExit" flag, the authority should vote in + favor of "BadExit". (This flag was added in 0.4.7.2-alpha.) + + "NoEdConsensus" -- authorities should not vote on this flag; it is + produced as part of the consensus for consensus method 22 or later. + + "StaleDesc" -- authorities should vote to assign this flag if the + published time on the descriptor is over 18 hours in the past. (This flag + was added in 0.4.0.1-alpha.) + + "Sybil" -- authorities SHOULD NOT accept more than 2 relays on a single IP. + If this happens, the authority *should* vote for the excess relays, but + should omit the Running or Valid flags and instead should assign the "Sybil" + flag. When there are more than 2 (or AuthDirMaxServersPerAddr) relays to + choose from, authorities should first prefer authorities to non-authorities, + then prefer Running to non-Running, and then prefer high-bandwidth to + low-bandwidth relays. In this comparison, measured bandwidth is used unless + it is not present for a router, in which case advertised bandwidth is used. + + Thus, the network-status vote includes all non-blacklisted, + non-expired, non-superseded descriptors. + + The bandwidth in a "w" line should be taken as the best estimate + of the router's actual capacity that the authority has. For now, + this should be the lesser of the observed bandwidth and bandwidth + rate limit from the server descriptor. It is given in kilobytes + per second, and capped at some arbitrary value (currently 10 MB/s). + + The Measured= keyword on a "w" line vote is currently computed + by multiplying the previous published consensus bandwidth by the + ratio of the measured average node stream capacity to the network + average. If 3 or more authorities provide a Measured= keyword for + a router, the authorities produce a consensus containing a "w" + Bandwidth= keyword equal to the median of the Measured= votes. + + As a special case, if the "w" line in a vote is about a relay with the + Authority flag, it should not include a Measured= keyword. The goal is + to leave such relays marked as Unmeasured, so they can reserve their + attention for authority-specific activities. "w" lines for votes about + authorities may include the bandwidth authority's measurement using + a different keyword, e.g. MeasuredButAuthority=, so it can still be + reported and recorded for posterity. + + The ports listed in a "p" line should be taken as those ports for + which the router's exit policy permits 'most' addresses, ignoring any + accept not for all addresses, ignoring all rejects for private + netblocks. "Most" addresses are permitted if no more than 2^25 + IPv4 addresses (two /8 networks) were blocked. The list is encoded + as described in section 3.8.2. + +3.4.3. Serving bandwidth list files + + If an authority has used a bandwidth list file to generate a vote + document it SHOULD make it available at + + http://<hostname>/tor/status-vote/next/bandwidth.z + + at the start of each voting period. + + It MUST NOT attempt to send its bandwidth list file in a HTTP POST to + other authorities and it SHOULD NOT make bandwidth list files from other + authorities available. + + If an authority makes this file available, it MUST be the bandwidth file + used to create the vote document available at + + http://<hostname>/tor/status-vote/next/authority.z + + To avoid inconsistent reads, authorities SHOULD only read the bandwidth + file once per voting period. Further processing and serving SHOULD use a + cached copy. + + The bandwidth list format is described in bandwidth-file-spec.txt. + + The standard URLs for bandwidth list files first-appeared in + Tor 0.4.0.4-alpha. + +3.5. Downloading missing certificates from other directory authorities + + XXX when to download certificates. + +3.6. Downloading server descriptors from other directory authorities + + Periodically (currently, every 10 seconds), directory authorities check + whether there are any specific descriptors that they do not have and that + they are not currently trying to download. + Authorities identify them by hash in vote (if publication date is more + recent than the descriptor we currently have). + + [XXXX need a way to fetch descriptors ahead of the vote? v2 status docs can + do that for now.] + + If so, the directory authority launches requests to the authorities for these + descriptors, such that each authority is only asked for descriptors listed + in its most recent vote. If more + than one authority lists the descriptor, we choose which to ask at random. + + If one of these downloads fails, we do not try to download that descriptor + from the authority that failed to serve it again unless we receive a newer + network-status (consensus or vote) from that authority that lists the same + descriptor. + + Directory authorities must potentially cache multiple descriptors for each + router. Authorities must not discard any descriptor listed by any recent + consensus. If there is enough space to store additional descriptors, + authorities SHOULD try to hold those which clients are likely to download the + most. (Currently, this is judged based on the interval for which each + descriptor seemed newest.) +[XXXX define recent] + + Authorities SHOULD NOT download descriptors for routers that they would + immediately reject for reasons listed in section 3.2. + +3.7. Downloading extra-info documents from other directory authorities + + Periodically, an authority checks whether it is missing any extra-info + documents: in other words, if it has any server descriptors with an + extra-info-digest field that does not match any of the extra-info + documents currently held. If so, it downloads whatever extra-info + documents are missing. We follow the same splitting and back-off rules + as in section 3.6. + +3.8. Computing a consensus from a set of votes + + Given a set of votes, authorities compute the contents of the consensus. + + The consensus status, along with as many signatures as the server + currently knows (see section 3.10 below), should be available at + + http://<hostname>/tor/status-vote/next/consensus.z + + The contents of the consensus document are as follows: + + The "valid-after", "valid-until", and "fresh-until" times are taken as + the median of the respective values from all the votes. + + The times in the "voting-delay" line are taken as the median of the + VoteSeconds and DistSeconds times in the votes. + + Known-flags is the union of all flags known by any voter. + + Entries are given on the "params" line for every keyword on which a + majority of authorities (total authorities, not just those + participating in this vote) voted on, or if at least three + authorities voted for that parameter. The values given are the + low-median of all votes on that keyword. + + (In consensus methods 7 to 11 inclusive, entries were given on + the "params" line for every keyword on which *any* authority voted, + the value given being the low-median of all votes on that keyword.) + + "client-versions" and "server-versions" are sorted in ascending + order; A version is recommended in the consensus if it is recommended + by more than half of the voting authorities that included a + client-versions or server-versions lines in their votes. + + With consensus methods 19 through 33, a package line is generated for a + given PACKAGENAME/VERSION pair if at least three authorities list such a + package in their votes. (Call these lines the "input" lines for + PACKAGENAME.) The consensus will contain every "package" line that is + listed verbatim by more than half of the authorities listing a line for + the PACKAGENAME/VERSION pair, and no others. + + The authority item groups (dir-source, contact, fingerprint, + vote-digest) are taken from the votes of the voting + authorities. These groups are sorted by the digests of the + authorities identity keys, in ascending order. If the consensus + method is 3 or later, a dir-source line must be included for + every vote with legacy-key entry, using the legacy-key's + fingerprint, the voter's ordinary nickname with the string + "-legacy" appended, and all other fields as from the original + vote's dir-source line. + + A router status entry: + * is included in the result if some router status entry with the same + identity is included by more than half of the authorities (total + authorities, not just those whose votes we have). + (Consensus method earlier than 21) + + * is included according to the rules in section 3.8.0.1 and + 3.8.0.2 below. (Consensus method 22 or later) + + * For any given RSA identity digest, we include at most + one router status entry. + + * For any given Ed25519 identity, we include at most one router + status entry. + + * A router entry has a flag set if that is included by more than half + of the authorities who care about that flag. + + * Two router entries are "the same" if they have the same + <descriptor digest, published time, nickname, IP, ports> tuple. + We choose the tuple for a given router as whichever tuple appears + for that router in the most votes. We break ties first in favor of + the more recently published, then in favor of smaller server + descriptor digest. + + [ + * The Named flag appears if it is included for this routerstatus by + _any_ authority, and if all authorities that list it list the same + nickname. However, if consensus-method 2 or later is in use, and + any authority calls this identity/nickname pair Unnamed, then + this routerstatus does not get the Named flag. + + * If consensus-method 2 or later is in use, the Unnamed flag is + set for a routerstatus if any authorities have voted for a different + identities to be Named with that nickname, or if any authority + lists that nickname/ID pair as Unnamed. + + (With consensus-method 1, Unnamed is set like any other flag.) + + [But note that authorities no longer vote for the Named flag, + and the above two bulletpoints are now irrelevant.] + ] + + * The version is given as whichever version is listed by the most + voters, with ties decided in favor of more recent versions. + + * If consensus-method 4 or later is in use, then routers that + do not have the Running flag are not listed at all. + + * If consensus-method 5 or later is in use, then the "w" line + is generated using a low-median of the bandwidth values from + the votes that included "w" lines for this router. + + * If consensus-method 5 or later is in use, then the "p" line + is taken from the votes that have the same policy summary + for the descriptor we are listing. (They should all be the + same. If they are not, we pick the most commonly listed + one, breaking ties in favor of the lexicographically larger + vote.) The port list is encoded as specified in section 3.8.2. + + * If consensus-method 6 or later is in use and if 3 or more + authorities provide a Measured= keyword in their votes for + a router, the authorities produce a consensus containing a + Bandwidth= keyword equal to the median of the Measured= votes. + + * If consensus-method 7 or later is in use, the params line is + included in the output. + + * If the consensus method is under 11, bad exits are considered as + possible exits when computing bandwidth weights. Otherwise, if + method 11 or later is in use, any router that is determined to get + the BadExit flag doesn't count when we're calculating weights. + + * If consensus method 12 or later is used, only consensus + parameters that more than half of the total number of + authorities voted for are included in the consensus. + + [ As of 0.2.6.1-alpha, authorities no longer advertise or negotiate + any consensus methods lower than 13. ] + + * If consensus method 13 or later is used, microdesc consensuses + omit any router for which no microdesc was agreed upon. + + * If consensus method 14 or later is used, the ns consensus and + microdescriptors may include an "a" line for each router, listing + an IPv6 OR port. + + * If consensus method 15 or later is used, microdescriptors + include "p6" lines including IPv6 exit policies. + + * If consensus method 16 or later is used, ntor-onion-key + are included in microdescriptors + + * If consensus method 17 or later is used, authorities impose a + maximum on the Bandwidth= values that they'll put on a 'w' + line for any router that doesn't have at least 3 measured + bandwidth values in votes. They also add an "Unmeasured=1" + flag to such 'w' lines. + + * If consensus method 18 or later is used, authorities include + "id" lines in microdescriptors. This method adds RSA ids. + + * If consensus method 19 or later is used, authorities may include + "package" lines in consensuses. + + * If consensus method 20 or later is used, authorities may include + GuardFraction information in microdescriptors. + + * If consensus method 21 or later is used, authorities may include + an "id" line for ed25519 identities in microdescriptors. + + [ As of 0.2.8.2-alpha, authorities no longer advertise or negotiate + consensus method 21, because it contains bugs. ] + + * If consensus method 22 or later is used, and the votes do not + produce a majority consensus about a relay's Ed25519 key (see + 3.8.0.1 below), the consensus must include a NoEdConsensus flag on + the "s" line for every relay whose listed Ed key does not reflect + consensus. + + * If consensus method 23 or later is used, authorities include + shared randomness protocol data on their votes and consensus. + + * If consensus-method 24 or later is in use, then routers that + do not have the Valid flag are not listed at all. + + [ As of 0.3.4.1-alpha, authorities no longer advertise or negotiate + any consensus methods lower than 25. ] + + * If consensus-method 25 or later is in use, then we vote + on recommended-protocols and required-protocols lines in the + consensus. We also include protocols lines in routerstatus + entries. + + * If consensus-method 26 or later is in use, then we initialize + bandwidth weights to 1 in our calculations, to avoid + division-by-zero errors on unusual networks. + + * If consensus method 27 or later is used, the microdesc consensus + may include an "a" line for each router, listing an IPv6 OR port. + + [ As of 0.4.3.1-alpha, authorities no longer advertise or negotiate + any consensus methods lower than 28. ] + + * If consensus method 28 or later is used, microdescriptors no longer + include "a" lines. + + * If consensus method 29 or later is used, microdescriptor "family" + lines are canonicalized to improve compression. + + * If consensus method 30 or later is used, the base64 encoded + ntor-onion-key does not include the trailing = sign. + + * If consensus method 31 or later is used, authorities parse the + "bwweightscale" and "maxunmeasuredbw" parameters correctly when + computing votes. + + * If consensus method 32 or later is used, authorities handle the + "MiddleOnly" flag specially when computing a consensus. When the + voters agree to include "MiddleOnly" in a routerstatus, they + automatically remove "Exit", "Guard", "V2Dir", and "HSDir". If + the BadExit flag is included in the consensus, they automatically + add it to the routerstatus. + + * If consensus method 33 or later is used, and the consensus + flavor is "microdesc", then the "Publication" field in the "r" + line is set to "2038-01-01 00:00:00". + + * If consensus method 34 or later is used, the consensus + does not include any "package" lines. + + The signatures at the end of a consensus document are sorted in + ascending order by identity digest. + + All ties in computing medians are broken in favor of the smaller or + earlier item. + +3.8.0.1. Deciding which Ids to include. + + This sorting algorithm is used for consensus-method 22 and later. + + First, consider each listing by tuple of <Ed,Rsa> identities, where 'Ed' + may be "None" if the voter included "id ed25519 none" to indicate that + the authority knows what ed25519 identities are, and thinks that the RSA + key doesn't have one. + + For each such <Ed, RSA> tuple that is listed by more than half of the + total authorities (not just total votes), include it. (It is not + possible for any other <id-Ed, id-RSA'> to have as many votes.) If more + than half of the authorities list a single <Ed,Rsa> pair of this type, we + consider that Ed key to be "consensus"; see description of the + NoEdConsensus flag. + + Log any other id-RSA values corresponding to an id-Ed we included, and any + other id-Ed values corresponding to an id-RSA we included. + + For each <id-RSA> that is not yet included, if it is listed by more than + half of the total authorities, and we do not already have it listed with + some <id-Ed>, include it, but do not consider its Ed identity canonical. + +3.8.0.2. Deciding which descriptors to include + + Deciding which descriptors to include. + + A tuple belongs to an <id-RSA, id-Ed> identity if it is a new tuple that + matches both ID parts, or if it is an old tuple (one with no Ed opinion) + that matches the RSA part. A tuple belongs to an <id-RSA> identity if its + RSA identity matches. + + A tuple matches another tuple if all the fields that are present in both + tuples are the same. + + For every included identity, consider the tuples belonging to that + identity. Group them into sets of matching tuples. Include the tuple + that matches the largest set, breaking ties in favor of the most recently + published, and then in favor of the smaller server descriptor digest. + +3.8.1. Forward compatibility + + Future versions of Tor will need to include new information in the + consensus documents, but it is important that all authorities (or at least + half) generate and sign the same signed consensus. + + To achieve this, authorities list in their votes their supported methods + for generating consensuses from votes. Later methods will be assigned + higher numbers. Currently specified methods: + + "1" -- The first implemented version. + "2" -- Added support for the Unnamed flag. + "3" -- Added legacy ID key support to aid in authority ID key rollovers + "4" -- No longer list routers that are not running in the consensus + "5" -- adds support for "w" and "p" lines. + "6" -- Prefers measured bandwidth values rather than advertised + "7" -- Provides keyword=integer pairs of consensus parameters + "8" -- Provides microdescriptor summaries + "9" -- Provides weights for selecting flagged routers in paths + "10" -- Fixes edge case bugs in router flag selection weights + "11" -- Don't consider BadExits when calculating bandwidth weights + "12" -- Params are only included if enough auths voted for them + "13" -- Omit router entries with missing microdescriptors. + "14" -- Adds support for "a" lines in ns consensuses and microdescriptors. + "15" -- Adds support for "p6" lines. + "16" -- Adds ntor keys to microdescriptors + "17" -- Adds "Unmeasured=1" flags to "w" lines + "18" -- Adds 'id' to microdescriptors. + "19" -- Adds "package" lines to consensuses + "20" -- Adds GuardFraction information to microdescriptors. + "21" -- Adds Ed25519 keys to microdescriptors. + "22" -- Instantiates Ed25519 voting algorithm correctly. + "23" -- Adds shared randomness protocol data. + "24" -- No longer lists routers that are not Valid in the consensus. + "25" -- Vote on recommended-protocols and required-protocols. + "26" -- Initialize bandwidth weights to 1 to avoid division-by-zero. + "27" -- Adds support for "a" lines in microdescriptor consensuses. + "28" -- Removes "a" lines from microdescriptors. + "29" -- Canonicalizes families in microdescriptors. + "30" -- Removes padding from ntor-onion-key. + "31" -- Uses correct parsing for bwweightscale and maxunmeasuredbw + when computing weights + "32" -- Adds special handling for MiddleOnly flag. + "33" -- Sets "publication" field in microdesc consensus "r" lines + to a meaningless value. + "34" -- Removes "package" lines from consensus. + + Before generating a consensus, an authority must decide which consensus + method to use. To do this, it looks for the highest version number + supported by more than 2/3 of the authorities voting. If it supports this + method, then it uses it. Otherwise, it falls back to the newest consensus + method that it supports (which will probably not result in a sufficiently + signed consensus). + + All authorities MUST support method 25; authorities SHOULD support + more recent methods as well. Authorities SHOULD NOT support or + advertise support for any method before 25. Clients MAY assume that + they will never see a current valid signed consensus for any method + before method 25. + + (The consensuses generated by new methods must be parsable by + implementations that only understand the old methods, and must not cause + those implementations to compromise their anonymity. This is a means for + making changes in the contents of consensus; not for making + backward-incompatible changes in their format.) + + The following methods have incorrect implementations; authorities SHOULD + NOT advertise support for them: + + "21" -- Did not correctly enable support for ed25519 key collation. + +3.8.2. Encoding port lists + + Whether the summary shows the list of accepted ports or the list of + rejected ports depends on which list is shorter (has a shorter string + representation). In case of ties we choose the list of accepted + ports. As an exception to this rule an allow-all policy is + represented as "accept 1-65535" instead of "reject " and a reject-all + policy is similarly given as "reject 1-65535". + + Summary items are compressed, that is instead of "80-88,89-100" there + only is a single item of "80-100", similarly instead of "20,21" a + summary will say "20-21". + + Port lists are sorted in ascending order. + + The maximum allowed length of a policy summary (including the "accept " + or "reject ") is 1000 characters. If a summary exceeds that length we + use an accept-style summary and list as much of the port list as is + possible within these 1000 bytes. [XXXX be more specific.] + +3.8.3. Computing Bandwidth Weights + + Let weight_scale = 10000, or the value of the "bwweightscale" parameter. + (Before consensus method 31 there was a bug in parsing bwweightscale, so + that if there were any consensus parameters after it alphabetically, it + would always be treated as 10000. A similar bug existed for + "maxunmeasuredbw".) + + Starting with consensus method 26, G, M, E, and D are initialized to 1 and + T to 4. Prior consensus methods initialize them all to 0. With this change, + test tor networks that are small or new are much more likely to produce + bandwidth-weights in their consensus. The extra bandwidth has a negligible + impact on the bandwidth weights in the public tor network. + + Let G be the total bandwidth for Guard-flagged nodes. + Let M be the total bandwidth for non-flagged nodes. + Let E be the total bandwidth for Exit-flagged nodes. + Let D be the total bandwidth for Guard+Exit-flagged nodes. + Let T = G+M+E+D + + Let Wgd be the weight for choosing a Guard+Exit for the guard position. + Let Wmd be the weight for choosing a Guard+Exit for the middle position. + Let Wed be the weight for choosing a Guard+Exit for the exit position. + + Let Wme be the weight for choosing an Exit for the middle position. + Let Wmg be the weight for choosing a Guard for the middle position. + + Let Wgg be the weight for choosing a Guard for the guard position. + Let Wee be the weight for choosing an Exit for the exit position. + + Balanced network conditions then arise from solutions to the following + system of equations: + + Wgg*G + Wgd*D == M + Wmd*D + Wme*E + Wmg*G (guard bw = middle bw) + Wgg*G + Wgd*D == Wee*E + Wed*D (guard bw = exit bw) + Wed*D + Wmd*D + Wgd*D == D (aka: Wed+Wmd+Wdg = weight_scale) + Wmg*G + Wgg*G == G (aka: Wgg = weight_scale-Wmg) + Wme*E + Wee*E == E (aka: Wee = weight_scale-Wme) + + We are short 2 constraints with the above set. The remaining constraints + come from examining different cases of network load. The following + constraints are used in consensus method 10 and above. There are another + incorrect and obsolete set of constraints used for these same cases in + consensus method 9. For those, see dir-spec.txt in Tor 0.2.2.10-alpha + to 0.2.2.16-alpha. + + Case 1: E >= T/3 && G >= T/3 (Neither Exit nor Guard Scarce) + + In this case, the additional two constraints are: Wmg == Wmd, + Wed == 1/3. + + This leads to the solution: + Wgd = weight_scale/3 + Wed = weight_scale/3 + Wmd = weight_scale/3 + Wee = (weight_scale*(E+G+M))/(3*E) + Wme = weight_scale - Wee + Wmg = (weight_scale*(2*G-E-M))/(3*G) + Wgg = weight_scale - Wmg + + Case 2: E < T/3 && G < T/3 (Both are scarce) + + Let R denote the more scarce class (Rare) between Guard vs Exit. + Let S denote the less scarce class. + + Subcase a: R+D < S + + In this subcase, we simply devote all of D bandwidth to the + scarce class. + + Wgg = Wee = weight_scale + Wmg = Wme = Wmd = 0; + if E < G: + Wed = weight_scale + Wgd = 0 + else: + Wed = 0 + Wgd = weight_scale + + Subcase b: R+D >= S + + In this case, if M <= T/3, we have enough bandwidth to try to achieve + a balancing condition. + + Add constraints Wgg = weight_scale, Wmd == Wgd to maximize bandwidth in + the guard position while still allowing exits to be used as middle nodes: + + Wee = (weight_scale*(E - G + M))/E + Wed = (weight_scale*(D - 2*E + 4*G - 2*M))/(3*D) + Wme = (weight_scale*(G-M))/E + Wmg = 0 + Wgg = weight_scale + Wmd = (weight_scale - Wed)/2 + Wgd = (weight_scale - Wed)/2 + + If this system ends up with any values out of range (ie negative, or + above weight_scale), use the constraints Wgg == weight_scale and Wee == + weight_scale, since both those positions are scarce: + + Wgg = weight_scale + Wee = weight_scale + Wed = (weight_scale*(D - 2*E + G + M))/(3*D) + Wmd = (weight_Scale*(D - 2*M + G + E))/(3*D) + Wme = 0 + Wmg = 0 + Wgd = weight_scale - Wed - Wmd + + If M > T/3, then the Wmd weight above will become negative. Set it to 0 + in this case: + Wmd = 0 + Wgd = weight_scale - Wed + + Case 3: One of E < T/3 or G < T/3 + + Let S be the scarce class (of E or G). + + Subcase a: (S+D) < T/3: + if G=S: + Wgg = Wgd = weight_scale; + Wmd = Wed = Wmg = 0; + // Minor subcase, if E is more scarce than M, + // keep its bandwidth in place. + if (E < M) Wme = 0; + else Wme = (weight_scale*(E-M))/(2*E); + Wee = weight_scale-Wme; + if E=S: + Wee = Wed = weight_scale; + Wmd = Wgd = Wme = 0; + // Minor subcase, if G is more scarce than M, + // keep its bandwidth in place. + if (G < M) Wmg = 0; + else Wmg = (weight_scale*(G-M))/(2*G); + Wgg = weight_scale-Wmg; + + Subcase b: (S+D) >= T/3 + if G=S: + Add constraints Wgg = weight_scale, Wmd == Wed to maximize bandwidth + in the guard position, while still allowing exits to be + used as middle nodes: + Wgg = weight_scale + Wgd = (weight_scale*(D - 2*G + E + M))/(3*D) + Wmg = 0 + Wee = (weight_scale*(E+M))/(2*E) + Wme = weight_scale - Wee + Wmd = (weight_scale - Wgd)/2 + Wed = (weight_scale - Wgd)/2 + if E=S: + Add constraints Wee == weight_scale, Wmd == Wgd to maximize bandwidth + in the exit position: + Wee = weight_scale; + Wed = (weight_scale*(D - 2*E + G + M))/(3*D); + Wme = 0; + Wgg = (weight_scale*(G+M))/(2*G); + Wmg = weight_scale - Wgg; + Wmd = (weight_scale - Wed)/2; + Wgd = (weight_scale - Wed)/2; + + To ensure consensus, all calculations are performed using integer math + with a fixed precision determined by the bwweightscale consensus + parameter (defaults at 10000, Min: 1, Max: INT32_MAX). (See note above + about parsing bug in bwweightscale before consensus method 31.) + + For future balancing improvements, Tor clients support 11 additional weights + for directory requests and middle weighting. These weights are currently + set at weight_scale, with the exception of the following groups of + assignments: + + Directory requests use middle weights: + + Wbd=Wmd, Wbg=Wmg, Wbe=Wme, Wbm=Wmm + + Handle bridges and strange exit policies: + + Wgm=Wgg, Wem=Wee, Weg=Wed + +3.9. Computing consensus flavors + + Consensus flavors are variants of the consensus that clients can choose + to download and use instead of the unflavored consensus. The purpose + of a consensus flavor is to remove or replace information in the + unflavored consensus without forcing clients to download information + they would not use anyway. + + Directory authorities can produce and serve an arbitrary number of + flavors of the same consensus. A downside of creating too many new + flavors is that clients will be distinguishable based on which flavor + they download. A new flavor should not be created when adding a field + instead wouldn't be too onerous. + + Examples for consensus flavors include: + + - Publishing hashes of microdescriptors instead of hashes of + full descriptors (see section 3.9.2). + - Including different digests of descriptors, instead of the + perhaps-soon-to-be-totally-broken SHA1. + + Consensus flavors are derived from the unflavored consensus once the + voting process is complete. This is to avoid consensus synchronization + problems. + + Every consensus flavor has a name consisting of a sequence of one + or more alphanumeric characters and dashes. For compatibility, + the original (unflavored) consensus type is called "ns". + + The supported consensus flavors are defined as part of the + authorities' consensus method. + + All consensus flavors have in common that their first line is + "network-status-version" where version is 3 or higher, and the flavor + is a string consisting of alphanumeric characters and dashes: + + "network-status-version" SP version [SP flavor] NL + +3.9.1. ns consensus + + The ns consensus flavor is equivalent to the unflavored consensus. + When the flavor is omitted from the "network-status-version" line, + it should be assumed to be "ns". Some implementations may explicitly + state that the flavor is "ns" when generating consensuses, but should + accept consensuses where the flavor is omitted. + +3.9.2. Microdescriptor consensus + + The microdescriptor consensus is a consensus flavor that contains + microdescriptor hashes instead of descriptor hashes and that omits + exit-policy summaries which are contained in microdescriptors. The + microdescriptor consensus was designed to contain elements that are + small and frequently changing. Clients use the information in the + microdescriptor consensus to decide which servers to fetch information + about and which servers to fetch information from. + + The microdescriptor consensus is based on the unflavored consensus with + the exceptions as follows: + + "network-status-version" SP version SP "microdesc" NL + + [At start, exactly once.] + + The flavor name of a microdescriptor consensus is "microdesc". + + Changes to router status entries are as follows: + + "r" SP nickname SP identity SP publication SP IP SP ORPort + SP DirPort NL + + [At start, exactly once.] + + Similar to "r" lines in section 3.4.1, but without the digest element. + + "a" SP address ":" port NL + + [Any number] + + Identical to the "r" lines in section 3.4.1. + + (Only included when the vote is generated with consensus-method 14 + or later, and the consensus is generated with consensus-method 27 or + later.) + + "p" ... NL + + [At most once] + + Not currently generated. + + Exit policy summaries are contained in microdescriptors and + therefore omitted in the microdescriptor consensus. + + "m" SP digest NL + + [Exactly once.*] + + "digest" is the base64 of the SHA256 hash of the router's + microdescriptor with trailing =s omitted. For a given router + descriptor digest and consensus method there should only be a + single microdescriptor digest in the "m" lines of all votes. + If different votes have different microdescriptor digests for + the same descriptor digest and consensus method, at least one + of the authorities is broken. If this happens, the microdesc + consensus should contain whichever microdescriptor digest is + most common. If there is no winner, we break ties in the favor + of the lexically earliest. + + [*Before consensus method 13, this field was sometimes erroneously + omitted.] + + Additionally, a microdescriptor consensus SHOULD use the sha256 digest + algorithm for its signatures. + +3.10. Exchanging detached signatures + + Once an authority has computed and signed a consensus network status, it + should send its detached signature to each other authority in an HTTP POST + request to the URL: + + http://<hostname>/tor/post/consensus-signature + + [XXX Note why we support push-and-then-pull.] + + All of the detached signatures it knows for consensus status should be + available at: + + http://<hostname>/tor/status-vote/next/consensus-signatures.z + + Assuming full connectivity, every authority should compute and sign the + same consensus including any flavors in each period. Therefore, it + isn't necessary to download the consensus or any flavors of it computed + by each authority; instead, the authorities only push/fetch each + others' signatures. A "detached signature" document contains items as + follows: + + "consensus-digest" SP Digest NL + + [At start, at most once.] + + The digest of the consensus being signed. + + "valid-after" SP YYYY-MM-DD SP HH:MM:SS NL + "fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL + "valid-until" SP YYYY-MM-DD SP HH:MM:SS NL + + [As in the consensus] + + "additional-digest" SP flavor SP algname SP digest NL + + [Any number.] + + For each supported consensus flavor, every directory authority + adds one or more "additional-digest" lines. "flavor" is the name + of the consensus flavor, "algname" is the name of the hash + algorithm that is used to generate the digest, and "digest" is the + hex-encoded digest. + + The hash algorithm for the microdescriptor consensus flavor is + defined as SHA256 with algname "sha256". + + "additional-signature" SP flavor SP algname SP identity SP + signing-key-digest NL signature. + + [Any number.] + + For each supported consensus flavor and defined digest algorithm, + every directory authority adds an "additional-signature" line. + "flavor" is the name of the consensus flavor. "algname" is the + name of the algorithm that was used to hash the identity and + signing keys, and to compute the signature. "identity" is the + hex-encoded digest of the authority identity key of the signing + authority, and "signing-key-digest" is the hex-encoded digest of + the current authority signing key of the signing authority. + + The "sha256" signature format is defined as the RSA signature of + the OAEP+-padded SHA256 digest of the item to be signed. When + checking signatures, the signature MUST be treated as valid if the + signature material begins with SHA256(document), so that other + data can get added later. + [To be honest, I didn't fully understand the previous paragraph + and only copied it from the proposals. Review carefully. -KL] + + "directory-signature" + + [As in the consensus; the signature object is the same as in the + consensus document.] + +3.11. Publishing the signed consensus + + The voting period ends at the valid-after time. If the consensus has + been signed by a majority of authorities, these documents are made + available at + + http://<hostname>/tor/status-vote/current/consensus.z + + and + + http://<hostname>/tor/status-vote/current/consensus-signatures.z + + [XXX current/consensus-signatures is not currently implemented, as it + is not used in the voting protocol.] + + [XXX possible future features include support for downloading old + consensuses.] + + The other vote documents are analogously made available under + + http://<hostname>/tor/status-vote/current/authority.z + http://<hostname>/tor/status-vote/current/<fp>.z + http://<hostname>/tor/status-vote/current/d/<d>.z + http://<hostname>/tor/status-vote/current/bandwidth.z + + once the voting period ends, regardless of the number of signatures. + + The authorities serve another consensus of each flavor "F" from the + locations + + /tor/status-vote/(current|next)/consensus-F.z. and + /tor/status-vote/(current|next)/consensus-F/<FP1>+....z. + + The standard URLs for bandwidth list files first-appeared in Tor 0.3.5. + +4. Directory cache operation + + All directory caches implement this section, except as noted. + +4.1. Downloading consensus status documents from directory authorities + + All directory caches try to keep a recent + network-status consensus document to serve to clients. A cache ALWAYS + downloads a network-status consensus if any of the following are true: + + - The cache has no consensus document. + - The cache's consensus document is no longer valid. + + Otherwise, the cache downloads a new consensus document at a randomly + chosen time in the first half-interval after its current consensus + stops being fresh. (This time is chosen at random to avoid swarming + the authorities at the start of each period. The interval size is + inferred from the difference between the valid-after time and the + fresh-until time on the consensus.) + + [For example, if a cache has a consensus that became valid at 1:00, + and is fresh until 2:00, that cache will fetch a new consensus at + a random time between 2:00 and 2:30.] + + Directory caches also fetch consensus flavors from the authorities. + Caches check the correctness of consensus flavors, but do not check + anything about an unrecognized consensus document beyond its digest and + length. Caches serve all consensus flavors from the same locations as + the directory authorities. + +4.2. Downloading server descriptors from directory authorities + + Periodically (currently, every 10 seconds), directory caches check + whether there are any specific descriptors that they do not have and that + they are not currently trying to download. Caches identify these + descriptors by hash in the recent network-status consensus documents. + + If so, the directory cache launches requests to the authorities for these + descriptors. + + If one of these downloads fails, we do not try to download that descriptor + from the authority that failed to serve it again unless we receive a newer + network-status consensus that lists the same descriptor. + + Directory caches must potentially cache multiple descriptors for each + router. Caches must not discard any descriptor listed by any recent + consensus. If there is enough space to store additional descriptors, + caches SHOULD try to hold those which clients are likely to download the + most. (Currently, this is judged based on the interval for which each + descriptor seemed newest.) + + [XXXX define recent] + +4.3. Downloading microdescriptors from directory authorities + + Directory mirrors should fetch, cache, and serve each microdescriptor + from the authorities. + + The microdescriptors with base64 hashes <D1>,<D2>,<D3> are available + at: + + http://<hostname>/tor/micro/d/<D1>-<D2>-<D3>[.z] + + <Dn> are base64 encoded with trailing =s omitted for size and for + consistency with the microdescriptor consensus format. -s are used + instead of +s to separate items, since the + character is used in + base64 encoding. + + Directory mirrors should check to make sure that the microdescriptors + they're about to serve match the right hashes (either the hashes from + the fetch URL or the hashes from the consensus, respectively). + + (NOTE: Due to squid proxy url limitations at most 92 microdescriptor hashes + can be retrieved in a single request.) + +4.4. Downloading extra-info documents from directory authorities + + Any cache that chooses to cache extra-info documents should implement this + section. + + Periodically, the Tor instance checks whether it is missing any extra-info + documents: in other words, if it has any server descriptors with an + extra-info-digest field that does not match any of the extra-info + documents currently held. If so, it downloads whatever extra-info + documents are missing. Caches download from authorities. We follow the + same splitting and back-off rules as in section 4.2. + +4.5. Consensus diffs + + Instead of downloading an entire consensus, clients may download + a "diff" document containing an ed-style diff from a previous + consensus document. Caches (and authorities) make these diffs as + they learn about new consensuses. To do so, they must store a + record of older consensuses. + + (Support for consensus diffs was added in 0.3.1.1-alpha, and is + advertised with the DirCache protocol version "2" or later.) + +4.5.1. Consensus diff format + + Consensus diffs are formatted as follows: + + The first line is "network-status-diff-version 1" NL + + The second line is + + "hash" SP FromDigest SP ToDigest NL + + where FromDigest is the hex-encoded SHA3-256 digest of the _signed + part_ of the consensus that the diff should be applied to, and + ToDigest is the hex-encoded SHA3-256 digest of the _entire_ + consensus resulting from applying the diff. (See 3.4.1 for + information on that part of a consensus is signed.) + + The third and subsequent lines encode the diff from FromDigest to + ToDigest in a limited subset of the ed diff format, as specified + in appendix E. + +4.5.2. Serving and requesting diffs. + + When downloading the current consensus, a client may include an + HTTP header of the form + + X-Or-Diff-From-Consensus: HASH1, HASH2, ... + + where the HASH values are hex-encoded SHA3-256 digests of the + _signed part_ of one or more consensuses that the client knows + about. + + If a cache knows a consensus diff from one of those consensuses + to the most recent consensus of the requested flavor, it may + send that diff instead of the specified consensus. + + Caches also serve diffs from the URIs: + + /tor/status-vote/current/consensus/diff/<HASH>/<FPRLIST>.z + /tor/status-vote/current/consensus-<FLAVOR>/diff/<HASH>/<FPRLIST>.z + + where FLAVOR is the consensus flavor, defaulting to "ns", and + FPRLIST is +-separated list of recognized authority identity + fingerprints as in appendix B. + +4.6 Retrying failed downloads + + See section 5.5 below; it applies to caches as well as clients. + +5. Client operation + + Every Tor that is not a directory server (that is, those that do + not have a DirPort set) implements this section. + +5.1. Downloading network-status documents + + Each client maintains a list of directory authorities. Insofar as + possible, clients SHOULD all use the same list. + + [Newer versions of Tor (0.2.8.1-alpha and later): + Each client also maintains a list of default fallback directory mirrors + (fallbacks). Each released version of Tor MAY have a different list, + depending on the mirrors that satisfy the fallback directory criteria at + release time.] + + Clients try to have a live consensus network-status document at all times. + A network-status document is "live" if the time in its valid-after field + has passed, and the time in its valid-until field has not passed. + + When a client has no consensus network-status document, it downloads it + from a randomly chosen fallback directory mirror or authority. Clients + prefer fallbacks to authorities, trying them earlier and more frequently. + In all other cases, the client downloads from caches randomly chosen from + among those believed to be V3 directory servers. (This information comes + from the network-status documents.) + + After receiving any response client MUST discard any network-status + documents that it did not request. + + On failure, the client waits briefly, then tries that network-status + document again from another cache. The client does not build circuits + until it has a live network-status consensus document, and it has + descriptors for a significant proportion of the routers that it believes + are running (this is configurable using torrc options and consensus + parameters). + + [Newer versions of Tor (0.2.6.2-alpha and later): + If the consensus contains Exits (the typical case), Tor will build both + exit and internal circuits. When bootstrap completes, Tor will be ready + to handle an application requesting an exit circuit to services like the + World Wide Web. + + If the consensus does not contain Exits, Tor will only build internal + circuits. In this case, earlier statuses will have included "internal" + as indicated above. When bootstrap completes, Tor will be ready to handle + an application requesting an internal circuit to hidden services at + ".onion" addresses. + + If a future consensus contains Exits, exit circuits may become available.] + + (Note: clients can and should pick caches based on the network-status + information they have: once they have first fetched network-status info + from an authority or fallback, they should not need to go to the authority + directly again, and should only choose the fallback at random, based on its + consensus weight in the current consensus.) + + To avoid swarming the caches whenever a consensus expires, the + clients download new consensuses at a randomly chosen time after the + caches are expected to have a fresh consensus, but before their + consensus will expire. (This time is chosen uniformly at random from + the interval between the time 3/4 into the first interval after the + consensus is no longer fresh, and 7/8 of the time remaining after + that before the consensus is invalid.) + + [For example, if a client has a consensus that became valid at 1:00, + and is fresh until 2:00, and expires at 4:00, that client will fetch + a new consensus at a random time between 2:45 and 3:50, since 3/4 + of the one-hour interval is 45 minutes, and 7/8 of the remaining 75 + minutes is 65 minutes.] + + Clients may choose to download the microdescriptor consensus instead + of the general network status consensus. In that case they should use + the same update strategy as for the normal consensus. They should not + download more than one consensus flavor. + + When a client does not have a live consensus, it will generally use the + most recent consensus it has if that consensus is "reasonably live". A + "reasonably live" consensus is one that expired less than 24 hours ago. + +5.2. Downloading server descriptors or microdescriptors + + Clients try to have the best descriptor for each router. A descriptor is + "best" if: + + * It is listed in the consensus network-status document. + + Periodically (currently every 10 seconds) clients check whether there are + any "downloadable" descriptors. A descriptor is downloadable if: + + - It is the "best" descriptor for some router. + - The descriptor was published at least 10 minutes in the past. + (This prevents clients from trying to fetch descriptors that the + mirrors have probably not yet retrieved and cached.) + - The client does not currently have it. + - The client is not currently trying to download it. + - The client would not discard it immediately upon receiving it. + - The client thinks it is running and valid (see section 5.4.1 below). + + If at least 16 known routers have downloadable descriptors, or if + enough time (currently 10 minutes) has passed since the last time the + client tried to download descriptors, it launches requests for all + downloadable descriptors. + + When downloading multiple server descriptors, the client chooses multiple + mirrors so that: + + - At least 3 different mirrors are used, except when this would result + in more than one request for under 4 descriptors. + - No more than 128 descriptors are requested from a single mirror. + - Otherwise, as few mirrors as possible are used. + After choosing mirrors, the client divides the descriptors among them + randomly. + + After receiving any response the client MUST discard any descriptors that + it did not request. + + When a descriptor download fails, the client notes it, and does not + consider the descriptor downloadable again until a certain amount of time + has passed. (Currently 0 seconds for the first failure, 60 seconds for the + second, 5 minutes for the third, 10 minutes for the fourth, and 1 day + thereafter.) Periodically (currently once an hour) clients reset the + failure count. + + Clients retain the most recent descriptor they have downloaded for each + router so long as it is listed in the consensus. If it is not listed, + they keep it so long as it is not too old (currently, ROUTER_MAX_AGE=48 + hours) and no better router descriptor has been downloaded for the same + relay. Caches retain descriptors until they are at least + OLD_ROUTER_DESC_MAX_AGE=5 days old. + + Clients which chose to download the microdescriptor consensus instead + of the general consensus must download the referenced microdescriptors + instead of server descriptors. Clients fetch and cache + microdescriptors preemptively from dir mirrors when starting up, like + they currently fetch descriptors. After bootstrapping, clients only + need to fetch the microdescriptors that have changed. + + When a client gets a new microdescriptor consensus, it looks to see if + there are any microdescriptors it needs to learn, and launches a request + for them. + + Clients maintain a cache of microdescriptors along with metadata like + when it was last referenced by a consensus, and which identity key + it corresponds to. They keep a microdescriptor until it hasn't been + mentioned in any consensus for a week. Future clients might cache them + for longer or shorter times. + +5.3. Downloading extra-info documents + + Any client that uses extra-info documents should implement this + section. + + Note that generally, clients don't need extra-info documents. + + Periodically, the Tor instance checks whether it is missing any extra-info + documents: in other words, if it has any server descriptors with an + extra-info-digest field that does not match any of the extra-info + documents currently held. If so, it downloads whatever extra-info + documents are missing. Clients try to download from caches. + We follow the same splitting and back-off rules as in section 5.2. + +5.4. Using directory information + + [XXX This subsection really belongs in path-spec.txt, not here. -KL] + + Everyone besides directory authorities uses the approaches in this section + to decide which relays to use and what their keys are likely to be. + (Directory authorities just believe their own opinions, as in section 3.4.2 + above.) + +5.4.1. Choosing routers for circuits. + + Circuits SHOULD NOT be built until the client has enough directory + information: a live consensus network status [XXXX fallback?] and + descriptors for at least 1/4 of the relays believed to be running. + + A relay is "listed" if it is included by the consensus network-status + document. Clients SHOULD NOT use unlisted relays. + + These flags are used as follows: + + - Clients SHOULD NOT use non-'Valid' or non-'Running' routers unless + requested to do so. + + - Clients SHOULD NOT use non-'Fast' routers for any purpose other than + very-low-bandwidth circuits (such as introduction circuits). + + - Clients SHOULD NOT use non-'Stable' routers for circuits that are + likely to need to be open for a very long time (such as those used for + IRC or SSH connections). + + - Clients SHOULD NOT choose non-'Guard' nodes when picking entry guard + nodes. + + See the "path-spec.txt" document for more details. + +5.4.2. Managing naming + + (This section is removed; authorities no longer assign the 'Named' flag.) + +5.4.3. Software versions + + An implementation of Tor SHOULD warn when it has fetched a consensus + network-status, and it is running a software version not listed. + +5.4.4. Warning about a router's status. + + (This section is removed; authorities no longer assign the 'Named' flag.) + +5.5. Retrying failed downloads + + This section applies to caches as well as to clients. + + When a client fails to download a resource (a consensus, a router + descriptor, a microdescriptor, etc) it waits for a certain amount of + time before retrying the download. To determine the amount of time + to wait, clients use a randomized exponential backoff algorithm. + (Specifically, they use a variation of the "decorrelated jitter" + algorithm from + https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ .) + + The specific formula used to compute the 'i+1'th delay is: + + Delay_{i+1} = MIN(cap, random_between(lower_bound, upper_bound))) + where upper_bound = MAX(lower_bound+1, Delay_i * 3) + lower_bound = MAX(1, base_delay). + + The value of 'cap' is set to INT_MAX; the value of 'base_delay' + depends on what is being downloaded, whether the client is fully + bootstrapped, how the client is configured, and where it is + downloading from. Current base_delay values are: + + Consensus objects, as a non-bridge cache: + 0 (TestingServerConsensusDownloadInitialDelay) + + Consensus objects, as a client or bridge that has bootstrapped: + 0 (TestingClientConsensusDownloadInitialDelay) + + Consensus objects, as a client or bridge that is bootstrapping, + when connecting to an authority because no "fallback" caches are + known: + 0 (ClientBootstrapConsensusAuthorityOnlyDownloadInitialDelay) + + Consensus objects, as a client or bridge that is bootstrapping, + when "fallback" caches are known but connecting to an authority + anyway: + 6 (ClientBootstrapConsensusAuthorityDownloadInitialDelay) + + Consensus objects, as a client or bridge that is bootstrapping, + when downloading from a "fallback" cache. + 0 (ClientBootstrapConsensusFallbackDownloadInitialDelay) + + Bridge descriptors, as a bridge-using client when at least one bridge + is usable: + 10800 (TestingBridgeDownloadInitialDelay) + + Bridge descriptors, otherwise: + 0 (TestingBridgeBootstrapDownloadInitialDelay) + + Other objects, as cache or authority: + 0 (TestingServerDownloadInitialDelay) + + Other objects, as client: + 0 (TestingClientDownloadInitialDelay) + + +6. Standards compliance + + All clients and servers MUST support HTTP 1.0. Clients and servers MAY + support later versions of HTTP as well. + +6.1. HTTP headers + + Servers SHOULD set Content-Encoding to the algorithm used to compress the + document(s) being served. Recognized algorithms are: + + - "identity" -- RFC2616 section 3.5 + - "deflate" -- RFC2616 section 3.5 + - "gzip" -- RFC2616 section 3.5 + - "x-zstd" -- The zstandard compression algorithm (www.zstd.net) + - "x-tor-lzma" -- The lzma compression algorithm, with a "preset" + value no higher than 6. + + Clients SHOULD use Accept-Encoding on most directory requests to indicate + which of the above compression algorithms they support. If they omit it + (as Tor clients did before 0.3.1.1-alpha), then the server should serve + only "deflate" or "identity" encoded documents, based on the presence or + absence of the ".z" suffix on the requested URL. + + Note that for anonymous directory requests (that is, requests made over + multi-hop circuits, like those for onion service lookups) implementations + SHOULD NOT advertise any Accept-Encoding values other than deflate. To do + so would be to create a fingerprinting opportunity. + + When receiving multiple documents, clients MUST accept compressed + concatenated documents and concatenated compressed documents as + equivalent. + + Servers MAY set the Content-Length: header. When they do, it should + match the number of compressed bytes that they are sending. + + Servers MAY include an X-Your-Address-Is: header, whose value is the + apparent IP address of the client connecting to them (as a dotted quad). + For directory connections tunneled over a BEGIN_DIR stream, servers SHOULD + report the IP from which the circuit carrying the BEGIN_DIR stream reached + them. + + Servers SHOULD disable caching of multiple network statuses or multiple + server descriptors. Servers MAY enable caching of single descriptors, + single network statuses, the list of all server descriptors, a v1 + directory, or a v1 running routers document. XXX mention times. + +6.2. HTTP status codes + + Tor delivers the following status codes. Some were chosen without much + thought; other code SHOULD NOT rely on specific status codes yet. + + 200 -- the operation completed successfully + -- the user requested statuses or serverdescs, and none of the ones we + requested were found (0.2.0.4-alpha and earlier). + + 304 -- the client specified an if-modified-since time, and none of the + requested resources have changed since that time. + + 400 -- the request is malformed, or + -- the URL is for a malformed variation of one of the URLs we support, + or + -- the client tried to post to a non-authority, or + -- the authority rejected a malformed posted document, or + + 404 -- the requested document was not found. + -- the user requested statuses or serverdescs, and none of the ones + requested were found (0.2.0.5-alpha and later). + + 503 -- we are declining the request in order to save bandwidth + -- user requested some items that we ordinarily generate or store, + but we do not have any available. + +A. Consensus-negotiation timeline. + + Period begins: this is the Published time. + Everybody sends votes + Reconciliation: everybody tries to fetch missing votes. + consensus may exist at this point. + End of voting period: + everyone swaps signatures. + Now it's okay for caches to download + Now it's okay for clients to download. + + Valid-after/valid-until switchover + +B. General-use HTTP URLs + + "Fingerprints" in these URLs are base16-encoded SHA1 hashes. + + The most recent v3 consensus should be available at: + + http://<hostname>/tor/status-vote/current/consensus.z + + Similarly, the v3 microdescriptor consensus should be available at: + + http://<hostname>/tor/status-vote/current/consensus-microdesc.z + + Starting with Tor version 0.2.1.1-alpha is also available at: + + http://<hostname>/tor/status-vote/current/consensus/<F1>+<F2>+<F3>.z + + (NOTE: Due to squid proxy url limitations at most 96 fingerprints can be + retrieved in a single request.) + + Where F1, F2, etc. are authority identity fingerprints the client trusts. + Servers will only return a consensus if more than half of the requested + authorities have signed the document, otherwise a 404 error will be sent + back. The fingerprints can be shortened to a length of any multiple of + two, using only the leftmost part of the encoded fingerprint. Tor uses + 3 bytes (6 hex characters) of the fingerprint. + + Clients SHOULD sort the fingerprints in ascending order. Server MUST + accept any order. + + Clients SHOULD use this format when requesting consensus documents from + directory authority servers and from caches running a version of Tor + that is known to support this URL format. + + A concatenated set of all the current key certificates should be available + at: + + http://<hostname>/tor/keys/all.z + + The key certificate for this server should be available at: + + http://<hostname>/tor/keys/authority.z + + The key certificate for an authority whose authority identity fingerprint + is <F> should be available at: + + http://<hostname>/tor/keys/fp/<F>.z + + The key certificate whose signing key fingerprint is <F> should be + available at: + + http://<hostname>/tor/keys/sk/<F>.z + + The key certificate whose identity key fingerprint is <F> and whose signing + key fingerprint is <S> should be available at: + + http://<hostname>/tor/keys/fp-sk/<F>-<S>.z + + (As usual, clients may request multiple certificates using: + + http://<hostname>/tor/keys/fp-sk/<F1>-<S1>+<F2>-<S2>.z ) + + [The above fp-sk format was not supported before Tor 0.2.1.9-alpha.] + + The most recent descriptor for a server whose identity key has a + fingerprint of <F> should be available at: + + http://<hostname>/tor/server/fp/<F>.z + + The most recent descriptors for servers with identity fingerprints + <F1>,<F2>,<F3> should be available at: + + http://<hostname>/tor/server/fp/<F1>+<F2>+<F3>.z + + (NOTE: Due to squid proxy url limitations at most 96 fingerprints can be + retrieved in a single request. + + Implementations SHOULD NOT download descriptors by identity key + fingerprint. This allows a corrupted server (in collusion with a cache) to + provide a unique descriptor to a client, and thereby partition that client + from the rest of the network.) + + The server descriptor with (descriptor) digest <D> (in hex) should be + available at: + + http://<hostname>/tor/server/d/<D>.z + + The most recent descriptors with digests <D1>,<D2>,<D3> should be + available at: + + http://<hostname>/tor/server/d/<D1>+<D2>+<D3>.z + + The most recent descriptor for this server should be at: + + http://<hostname>/tor/server/authority.z + + This is used for authorities, and also if a server is configured + as a bridge. The official Tor implementations (starting at + 0.1.1.x) use this resource to test whether a server's own DirPort + is reachable. It is also useful for debugging purposes. + + A concatenated set of the most recent descriptors for all known servers + should be available at: + + http://<hostname>/tor/server/all.z + + Extra-info documents are available at the URLS + + http://<hostname>/tor/extra/d/... + http://<hostname>/tor/extra/fp/... + http://<hostname>/tor/extra/all[.z] + http://<hostname>/tor/extra/authority[.z] + (As for /tor/server/ URLs: supports fetching extra-info + documents by their digest, by the fingerprint of their servers, + or all at once. When serving by fingerprint, we serve the + extra-info that corresponds to the descriptor we would serve by + that fingerprint. Only directory authorities of version + 0.2.0.1-alpha or later are guaranteed to support the first + three classes of URLs. Caches may support them, and MUST + support them if they have advertised "caches-extra-info".) + + For debugging, directories SHOULD expose non-compressed objects at + URLs like the above, but without the final ".z". If the client uses + Accept-Encodings header, it should override the presence or absence + of the ".z" (see section 6.1). + + Clients SHOULD use upper case letters (A-F) when base16-encoding + fingerprints. Servers MUST accept both upper and lower case fingerprints + in requests. + +C. Converting a curve25519 public key to an ed25519 public key + + Given an X25519 key, that is, an affine point (u,v) on the + Montgomery curve defined by + + bv^2 = u(u^2 + au +1) + + where + + a = 486662 + b = 1 + + and comprised of the compressed form (i.e. consisting of only the + u-coordinate), we can retrieve the y-coordinate of the affine point + (x,y) on the twisted Edwards form of the curve defined by + + -x^2 + y^2 = 1 + d x^2 y^2 + + where + + d = - 121665/121666 + + by computing + + y = (u-1)/(u+1). + + and then we can apply the usual curve25519 twisted Edwards point + decompression algorithm to find _an_ x-coordinate of an affine + twisted Edwards point to check signatures with. Signing keys for + ed25519 are compressed curve points in twisted Edwards form (so a + y-coordinate and the sign of the x-coordinate), and X25519 keys are + compressed curve points in Montgomery form (i.e. a u-coordinate). + + However, note that compressed point in Montgomery form neglects to + encode what the sign of the corresponding twisted Edwards + x-coordinate would be. Thus, we need the sign of the x-coordinate + to do this operation; otherwise, we'll have two possible + x-coordinates that might have correspond to the ed25519 public key. + + To get the sign, the easiest way is to take the corresponding + private key, feed it to the ed25519 public key generation + algorithm, and see what the sign is. + + [Recomputing the sign bit from the private key every time sounds + rather strange and inefficient to me… —isis] + + Note that in addition to its coordinates, an expanded Ed25519 private key + also has a 32-byte random value, "prefix", used to compute internal `r` + values in the signature. For security, this prefix value should be + derived deterministically from the curve25519 key. The Tor + implementation derives it as SHA512(private_key | STR)[0..32], where + STR is the nul-terminated string: + + "Derive high part of ed25519 key from curve25519 key\0" + + + On the client side, where there is no access to the curve25519 private + keys, one may use the curve25519 public key's Montgomery u-coordinate to + recover the Montgomery v-coordinate by computing the right-hand side of + the Montgomery curve equation: + + bv^2 = u(u^2 + au +1) + + where + + a = 486662 + b = 1 + + Then, knowing the intended sign of the Edwards x-coordinate, one + may recover said x-coordinate by computing: + + x = (u/v) * sqrt(-a - 2) + +D. Inferring missing proto lines. + + The directory authorities no longer allow versions of Tor before + 0.2.4.18-rc. But right now, there is no version of Tor in the consensus + before 0.2.4.19. Therefore, we should disallow versions of Tor earlier + than 0.2.4.19, so that we can have the protocol list for all current Tor + versions include: + + Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 Link=1-4 + LinkAuth=1 Microdesc=1-2 Relay=1-2 + + For Desc, Microdesc and Cons, Tor versions before 0.2.7.stable should be + taken to only support version 1. + +E. Limited ed diff format + + We support the following format for consensus diffs. It's a + subset of the ed diff format, but clients MUST NOT accept other + ed commands. + + We support the following ed commands, each on a line by itself: + + - "<n1>d" Delete line n1 + - "<n1>,<n2>d" Delete lines n1 through n2, inclusive + - "<n1>,$d" Delete line n1 through the end of the file, inclusive. + - "<n1>c" Replace line n1 with the following block + - "<n1>,<n2>c" Replace lines n1 through n2, inclusive, with the + following block. + - "<n1>a" Append the following block after line n1. + + Note that line numbers always apply to the file after all previous + commands have already been applied. Note also that line numbers + are 1-indexed. + + The commands MUST apply to the file from back to front, such that + lines are only ever referred to by their position in the original + file. + + If there are any directory signatures on the original document, the + first command MUST be a "<n1>,$d" form to remove all of the directory + signatures. Using this format ensures that the client will + successfully apply the diff even if they have an unusual encoding for + the signatures. + + The replace and append command take blocks. These blocks are simply + appended to the diff after the line with the command. A line with + just a period (".") ends the block (and is not part of the lines + to add). Note that it is impossible to insert a line with just + a single dot. |