From e4e0d93d56ee8c1aec4c2efaa7046b651f0fe55c Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Thu, 12 Oct 2023 12:27:58 -0400 Subject: Move all text-only specifications into the OLD_TXT directory. --- dir-spec.txt | 4299 ---------------------------------------------------------- 1 file changed, 4299 deletions(-) delete mode 100644 dir-spec.txt (limited to 'dir-spec.txt') diff --git a/dir-spec.txt b/dir-spec.txt deleted file mode 100644 index f133c39..0000000 --- a/dir-spec.txt +++ /dev/null @@ -1,4299 +0,0 @@ - - Tor directory protocol, version 3 - -Table of Contents - - 0. Scope and preliminaries - 0.1. History - 0.2. Goals of the version 3 protoc - 0.3. Some Remaining questions - 1. Outline - 1.1. What's different from version 2? - 1.2. Document meta-format - 1.3. Signing documents - 1.4. Voting timeline - 2. Router operation and formats - 2.1. Uploading server descriptors and extra-info documents - 2.1.1. Server descriptor format - 2.1.2. Extra-info document format - 2.1.3. Nonterminals in server descriptors - 3. Directory authority operation and formats - 3.1. Creating key certificates - 3.2. Accepting server descriptor and extra-info document uploads - 3.3. Computing microdescriptors - 3.4. Exchanging votes - 3.4.1. Vote and consensus status document formats - 3.4.2. Assigning flags in a vote - 3.4.3. Serving bandwidth list files - 3.5. Downloading missing certificates from other directory authorities - 3.6. Downloading server descriptors from other directory authorities - 3.7. Downloading extra-info documents from other directory authorities - 3.8. Computing a consensus from a set of votes - 3.8.0.1. Deciding which Ids to include. - 3.8.0.2. Deciding which descriptors to include - 3.8.1. Forward compatibility - 3.8.2. Encoding port lists - 3.8.3. Computing Bandwidth Weights - 3.9. Computing consensus flavors - 3.9.1. ns consensus - 3.9.2. Microdescriptor consensus - 3.10. Exchanging detached signatures - 3.11. Publishing the signed consensus - 4. Directory cache operation - 4.1. Downloading consensus status documents from directory authorities - 4.2. Downloading server descriptors from directory authorities - 4.3. Downloading microdescriptors from directory authorities - 4.4. Downloading extra-info documents from directory authorities - 4.5. Consensus diffs - 4.5.1. Consensus diff format - 4.5.2. Serving and requesting diff - 4.6 Retrying failed downloads - 5. Client operation - 5.1. Downloading network-status documents - 5.2. Downloading server descriptors or microdescriptors - 5.3. Downloading extra-info documents - 5.4. Using directory information - 5.4.1. Choosing routers for circuits. - 5.4.2. Managing naming - 5.4.3. Software versions - 5.4.4. Warning about a router's status. - 5.5. Retrying failed downloads - 6. Standards compliance - 6.1. HTTP headers - 6.2. HTTP status codes - A. Consensus-negotiation timeline. - B. General-use HTTP URLs - C. Converting a curve25519 public key to an ed25519 public key - D. Inferring missing proto lines. - E. Limited ed diff format - -0. Scope and preliminaries - - This directory protocol is used by Tor version 0.2.0.x-alpha and later. - See dir-spec-v1.txt for information on the protocol used up to the - 0.1.0.x series, and dir-spec-v2.txt for information on the protocol - used by the 0.1.1.x and 0.1.2.x series. - - This document merges and supersedes the following proposals: - - 101 Voting on the Tor Directory System - 103 Splitting identity key from regularly used signing key - 104 Long and Short Router Descriptors - - XXX timeline - XXX fill in XXXXs - - The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL - NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and - "OPTIONAL" in this document are to be interpreted as described in - RFC 2119. - -0.1. History - - The earliest versions of Onion Routing shipped with a list of known - routers and their keys. When the set of routers changed, users needed to - fetch a new list. - - The Version 1 Directory protocol - -------------------------------- - - Early versions of Tor (0.0.2) introduced "Directory authorities": servers - that served signed "directory" documents containing a list of signed - "server descriptors", along with short summary of the status of each - router. Thus, clients could get up-to-date information on the state of - the network automatically, and be certain that the list they were getting - was attested by a trusted directory authority. - - Later versions (0.0.8) added directory caches, which download - directories from the authorities and serve them to clients. Non-caches - fetch from the caches in preference to fetching from the authorities, thus - distributing bandwidth requirements. - - Also added during the version 1 directory protocol were "router status" - documents: short documents that listed only the up/down status of the - routers on the network, rather than a complete list of all the - descriptors. Clients and caches would fetch these documents far more - frequently than they would fetch full directories. - - The Version 2 Directory Protocol - -------------------------------- - - During the Tor 0.1.1.x series, Tor revised its handling of directory - documents in order to address two major problems: - - * Directories had grown quite large (over 1MB), and most directory - downloads consisted mainly of server descriptors that clients - already had. - - * Every directory authority was a trust bottleneck: if a single - directory authority lied, it could make clients believe for a time - an arbitrarily distorted view of the Tor network. (Clients - trusted the most recent signed document they downloaded.) Thus, - adding more authorities would make the system less secure, not - more. - - To address these, we extended the directory protocol so that - authorities now published signed "network status" documents. Each - network status listed, for every router in the network: a hash of its - identity key, a hash of its most recent descriptor, and a summary of - what the authority believed about its status. Clients would download - the authorities' network status documents in turn, and believe - statements about routers iff they were attested to by more than half of - the authorities. - - Instead of downloading all server descriptors at once, clients - downloaded only the descriptors that they did not have. Descriptors - were indexed by their digests, in order to prevent malicious caches - from giving different versions of a server descriptor to different - clients. - - Routers began working harder to upload new descriptors only when their - contents were substantially changed. - - -0.2. Goals of the version 3 protocol - - Version 3 of the Tor directory protocol tries to solve the following - issues: - - * A great deal of bandwidth used to transmit server descriptors was - used by two fields that are not actually used by Tor routers - (namely read-history and write-history). We save about 60% by - moving them into a separate document that most clients do not - fetch or use. - - * It was possible under certain perverse circumstances for clients - to download an unusual set of network status documents, thus - partitioning themselves from clients who have a more recent and/or - typical set of documents. Even under the best of circumstances, - clients were sensitive to the ages of the network status documents - they downloaded. Therefore, instead of having the clients - correlate multiple network status documents, we have the - authorities collectively vote on a single consensus network status - document. - - * The most sensitive data in the entire network (the identity keys - of the directory authorities) needed to be stored unencrypted so - that the authorities can sign network-status documents on the fly. - Now, the authorities' identity keys are stored offline, and used - to certify medium-term signing keys that can be rotated. - -0.3. Some Remaining questions - - Things we could solve on a v3 timeframe: - - The SHA-1 hash is showing its age. We should do something about our - dependency on it. We could probably future-proof ourselves here in - this revision, at least so far as documents from the authorities are - concerned. - - Too many things about the authorities are hardcoded by IP. - - Perhaps we should start accepting longer identity keys for routers - too. - - Things to solve eventually: - - Requiring every client to know about every router won't scale forever. - - Requiring every directory cache to know every router won't scale - forever. - - -1. Outline - - There is a small set (say, around 5-10) of semi-trusted directory - authorities. A default list of authorities is shipped with the Tor - software. Users can change this list, but are encouraged not to do so, - in order to avoid partitioning attacks. - - Every authority has a very-secret, long-term "Authority Identity Key". - This is stored encrypted and/or offline, and is used to sign "key - certificate" documents. Every key certificate contains a medium-term - (3-12 months) "authority signing key", that is used by the authority to - sign other directory information. (Note that the authority identity - key is distinct from the router identity key that the authority uses - in its role as an ordinary router.) - - Routers periodically upload signed "routers descriptors" to the - directory authorities describing their keys, capabilities, and other - information. Routers may also upload signed "extra-info documents" - containing information that is not required for the Tor protocol. - Directory authorities serve server descriptors indexed by router - identity, or by hash of the descriptor. - - Routers may act as directory caches to reduce load on the directory - authorities. They announce this in their descriptors. - - Periodically, each directory authority generates a view of - the current descriptors and status for known routers. They send a - signed summary of this view (a "status vote") to the other - authorities. The authorities compute the result of this vote, and sign - a "consensus status" document containing the result of the vote. - - Directory caches download, cache, and re-serve consensus documents. - - Clients, directory caches, and directory authorities all use consensus - documents to find out when their list of routers is out-of-date. - (Directory authorities also use vote statuses.) If it is, they download - any missing server descriptors. Clients download missing descriptors - from caches; caches and authorities download from authorities. - Descriptors are downloaded by the hash of the descriptor, not by the - relay's identity key: this prevents directory servers from attacking - clients by giving them descriptors nobody else uses. - - All directory information is uploaded and downloaded with HTTP. - -1.1. What's different from version 2? - - Clients used to download multiple network status documents, - corresponding roughly to "status votes" above. They would compute the - result of the vote on the client side. - - Authorities used to sign documents using the same private keys they used - for their roles as routers. This forced them to keep these extremely - sensitive keys in memory unencrypted. - - All of the information in extra-info documents used to be kept in the - main descriptors. - -1.2. Document meta-format - - Server descriptors, directories, and running-routers documents all obey the - following lightweight extensible information format. - - The highest level object is a Document, which consists of one or more - Items. Every Item begins with a KeywordLine, followed by zero or more - Objects. A KeywordLine begins with a Keyword, optionally followed by - whitespace and more non-newline characters, and ends with a newline. A - Keyword is a sequence of one or more characters in the set [A-Za-z0-9-], - but may not start with -. - An Object is a block of encoded data in pseudo-Privacy-Enhanced-Mail (PEM) - style format: that is, lines of encoded data MAY be wrapped by inserting - an ascii linefeed ("LF", also called newline, or "NL" here) character - (cf. RFC 4648 §3.1). When line wrapping, implementations MUST wrap lines - at 64 characters. Upon decoding, implementations MUST ignore and discard - all linefeed characters. - - More formally: - - NL = The ascii LF character (hex value 0x0a). - Document ::= (Item | NL)+ - Item ::= KeywordLine Object? - KeywordLine ::= Keyword (WS Argument)* NL - Keyword = KeywordStart KeywordChar* - KeywordStart ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' - KeywordChar ::= KeywordStart | '-' - Argument := ArgumentChar+ - ArgumentChar ::= any graphical printing ASCII character. - WS = (SP | TAB)+ - Object ::= BeginLine Base64-encoded-data EndLine - BeginLine ::= "-----BEGIN " Keyword (" " Keyword)* "-----" NL - EndLine ::= "-----END " Keyword (" " Keyword)* "-----" NL - - A Keyword may not be "-----BEGIN". - - The BeginLine and EndLine of an Object must use the same keyword. - - When interpreting a Document, software MUST ignore any KeywordLine that - starts with a keyword it doesn't recognize; future implementations MUST NOT - require current clients to understand any KeywordLine not currently - described. - - Other implementations that want to extend Tor's directory format MAY - introduce their own items. The keywords for extension items SHOULD start - with the characters "x-" or "X-", to guarantee that they will not conflict - with keywords used by future versions of Tor. - - In our document descriptions below, we tag Items with a multiplicity in - brackets. Possible tags are: - - "At start, exactly once": These items MUST occur in every instance of - the document type, and MUST appear exactly once, and MUST be the - first item in their documents. - - "Exactly once": These items MUST occur exactly one time in every - instance of the document type. - - "At end, exactly once": These items MUST occur in every instance of - the document type, and MUST appear exactly once, and MUST be the - last item in their documents. - - "At most once": These items MAY occur zero or one times in any - instance of the document type, but MUST NOT occur more than once. - - "Any number": These items MAY occur zero, one, or more times in any - instance of the document type. - - "Once or more": These items MUST occur at least once in any instance - of the document type, and MAY occur more. - - For forward compatibility, each item MUST allow extra arguments at the - end of the line unless otherwise noted. So if an item's description below - is given as: - - "thing" int int int NL - - then implementations SHOULD accept this string as well: - - "thing 5 9 11 13 16 12" NL - - but not this string: - - "thing 5" NL - - and not this string: - - "thing 5 10 thing" NL - . - - Whenever an item DOES NOT allow extra arguments, we will tag it with - "no extra arguments". - -1.3. Signing documents - - Every signable document below is signed in a similar manner, using a - given "Initial Item", a final "Signature Item", a digest algorithm, and - a signing key. - - The Initial Item must be the first item in the document. - - The Signature Item has the following format: - - [arguments] NL SIGNATURE NL - - The "SIGNATURE" Object contains a signature (using the signing key) of - the PKCS#1 1.5 padded digest of the entire document, taken from the - beginning of the Initial item, through the newline after the Signature - Item's keyword and its arguments. - - The signature does not include the algorithmIdentifier specified in PKCS #1. - - Unless specified otherwise, the digest algorithm is SHA-1. - - All documents are invalid unless signed with the correct signing key. - - The "Digest" of a document, unless stated otherwise, is its digest *as - signed by this signature scheme*. - -1.4. Voting timeline - - Every consensus document has a "valid-after" (VA) time, a "fresh-until" - (FU) time and a "valid-until" (VU) time. VA MUST precede FU, which MUST - in turn precede VU. Times are chosen so that every consensus will be - "fresh" until the next consensus becomes valid, and "valid" for a while - after. At least 3 consensuses should be valid at any given time. - - The timeline for a given consensus is as follows: - - VA-DistSeconds-VoteSeconds: The authorities exchange votes. Each authority - uploads their vote to all other authorities. - - VA-DistSeconds-VoteSeconds/2: The authorities try to download any - votes they don't have. - - Authorities SHOULD also reject any votes that other authorities try to - upload after this time. (0.4.4.1-alpha was the first version to reject votes - in this way.) - - Note: Refusing late uploaded votes minimizes the chance of a consensus - split, particular when authorities are under bandwidth pressure. If an - authority is struggling to upload its vote, and finally uploads to a - fraction of authorities after this period, they will compute a consensus - different from the others. By refusing uploaded votes after this time, - we increase the likelihood that most authorities will use the same vote - set. - - Rejecting late uploaded votes does not fix the problem entirely. If - some authorities are able to download a specific vote, but others fail - to do so, then there may still be a consensus split. However, this - change does remove one common cause of consensus splits. - - VA-DistSeconds: The authorities calculate the consensus and exchange - signatures. (This is the earliest point at which anybody can - possibly get a given consensus if they ask for it.) - - VA-DistSeconds/2: The authorities try to download any signatures - they don't have. - - VA: All authorities have a multiply signed consensus. - - VA ... FU: Caches download the consensus. (Note that since caches have - no way of telling what VA and FU are until they have downloaded - the consensus, they assume that the present consensus's VA is - equal to the previous one's FU, and that its FU is one interval after - that.) - - FU: The consensus is no longer the freshest consensus. - - FU ... (the current consensus's VU): Clients download the consensus. - (See note above: clients guess that the next consensus's FU will be - two intervals after the current VA.) - - VU: The consensus is no longer valid; clients should continue to try to - download a new consensus if they have not done so already. - - VU + 24 hours: Clients will no longer use the consensus at all. - - VoteSeconds and DistSeconds MUST each be at least 20 seconds; FU-VA and - VU-FU MUST each be at least 5 minutes. - -2. Router operation and formats - -2.1. Uploading server descriptors and extra-info documents - - ORs SHOULD generate a new server descriptor and a new extra-info - document whenever any of the following events have occurred: - - - A period of time (18 hrs by default) has passed since the last - time a descriptor was generated. - - - A descriptor field other than bandwidth or uptime has changed. - - - Its uptime is less than 24h and bandwidth has changed by a factor of 2 - from the last time a descriptor was generated, and at least a given - interval of time (3 hours by default) has passed since then. - - - Its uptime has been reset (by restarting). - - - It receives a networkstatus consensus in which it is not listed. - - - It receives a networkstatus consensus in which it is listed - with the StaleDesc flag. - - [XXX this list is incomplete; see router_differences_are_cosmetic() - in routerlist.c for others] - - ORs SHOULD NOT publish a new server descriptor or extra-info document - if none of the above events have occurred and not much time has passed - (12 hours by default). - - Tor versions older than 0.3.5.1-alpha ignore uptime when checking for - bandwidth changes. - - After generating a descriptor, ORs upload them to every directory - authority they know, by posting them (in order) to the URL - - http:///tor/ - - Server descriptors may not exceed 20,000 bytes in length; extra-info - documents may not exceed 50,000 bytes in length. If they do, the - authorities SHOULD reject them. - -2.1.1. Server descriptor format - - Server descriptors consist of the following items. - - In lines that take multiple arguments, extra arguments SHOULD be - accepted and ignored. Many of the nonterminals below are defined in - section 2.1.3. - - Note that many versions of Tor will generate an extra newline at the - end of their descriptors. Implementations MUST tolerate one or - more blank lines at the end of a single descriptor or a list of - concatenated descriptors. New implementations SHOULD NOT generate - such blank lines. - - "router" nickname address ORPort SOCKSPort DirPort NL - - [At start, exactly once.] - - Indicates the beginning of a server descriptor. "nickname" must be a - valid router nickname as specified in section 2.1.3. "address" must - be an IPv4 - address in dotted-quad format. The last three numbers indicate the - TCP ports at which this OR exposes functionality. ORPort is a port at - which this OR accepts TLS connections for the main OR protocol; - SOCKSPort is deprecated and should always be 0; and DirPort is the - port at which this OR accepts directory-related HTTP connections. If - any port is not supported, the value 0 is given instead of a port - number. (At least one of DirPort and ORPort SHOULD be set; - authorities MAY reject any descriptor with both DirPort and ORPort of - 0.) - - "identity-ed25519" NL "-----BEGIN ED25519 CERT-----" NL certificate - "-----END ED25519 CERT-----" NL - - [Exactly once, in second position in document.] - [No extra arguments] - - The certificate is a base64-encoded Ed25519 certificate (see - cert-spec.txt) with terminating =s removed. When this element - is present, it MUST appear as the first or second element in - the router descriptor. - - The certificate has CERT_TYPE of [04]. It must include a - signed-with-ed25519-key extension (see cert-spec.txt, - section 2.2.1), so that we can extract the master identity key. - - [Before Tor 0.4.5.1-alpha, this field was optional.] - - "master-key-ed25519" SP MasterKey NL - - [Exactly once] - - Contains the base-64 encoded ed25519 master key as a single - argument. If it is present, it MUST match the identity key - in the identity-ed25519 entry. - - [Before Tor 0.4.5.1-alpha, this field was optional.] - - "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed NL - - [Exactly once] - - Estimated bandwidth for this router, in bytes per second. The - "average" bandwidth is the volume per second that the OR is willing to - sustain over long periods; the "burst" bandwidth is the volume that - the OR is willing to sustain in very short intervals. The "observed" - value is an estimate of the capacity this relay can handle. The - relay remembers the max bandwidth sustained output over any ten - second period in the past 5 days, and another sustained input. The - "observed" value is the lesser of these two numbers. - - Tor versions released before 2018 only kept bandwidth-observed for one - day. These versions are no longer supported or recommended. - - "platform" string NL - - [At most once] - - A human-readable string describing the system on which this OR is - running. This MAY include the operating system, and SHOULD include - the name and version of the software implementing the Tor protocol. - - "published" YYYY-MM-DD HH:MM:SS NL - - [Exactly once] - - The time, in UTC, when this descriptor (and its corresponding - extra-info document if any) was generated. - - "fingerprint" fingerprint NL - - [At most once] - - A fingerprint (a HASH_LEN-byte of asn1 encoded public key, encoded in - hex, with a single space after every 4 characters) for this router's - identity key. A descriptor is considered invalid (and MUST be - rejected) if the fingerprint line does not match the public key. - - [We didn't start parsing this line until Tor 0.1.0.6-rc; it should - be marked with "opt" until earlier versions of Tor are obsolete.] - - "hibernating" bool NL - - [At most once] - - If the value is 1, then the Tor relay was hibernating when the - descriptor was published, and shouldn't be used to build circuits. - - [We didn't start parsing this line until Tor 0.1.0.6-rc; it should be - marked with "opt" until earlier versions of Tor are obsolete.] - - "uptime" number NL - - [At most once] - - The number of seconds that this OR process has been running. - - "onion-key" NL a public key in PEM format - - [Exactly once] - [No extra arguments] - - This key is used to encrypt CREATE cells for this OR. The key MUST be - accepted for at least 1 week after any new key is published in a - subsequent descriptor. It MUST be 1024 bits. - - The key encoding is the encoding of the key as a PKCS#1 RSAPublicKey - structure, encoded in base64, and wrapped in "-----BEGIN RSA PUBLIC - KEY-----" and "-----END RSA PUBLIC KEY-----". - - "onion-key-crosscert" NL a RSA signature in PEM format. - - [Exactly once] - [No extra arguments] - - This element contains an RSA signature, generated using the - onion-key, of the following: - - A SHA1 hash of the RSA identity key, - i.e. RSA key from "signing-key" (see below) [20 bytes] - The Ed25519 identity key, - i.e. Ed25519 key from "master-key-ed25519" [32 bytes] - - If there is no Ed25519 identity key, or if in some future version - there is no RSA identity key, the corresponding field must be - zero-filled. - - Parties verifying this signature MUST allow additional data - beyond the 52 bytes listed above. - - This signature proves that the party creating the descriptor - had control over the secret key corresponding to the - onion-key. - - [Before Tor 0.4.5.1-alpha, this field was optional whenever - identity-ed25519 was absent.] - - "ntor-onion-key" base-64-encoded-key - - [Exactly once] - - A curve25519 public key used for the ntor circuit extended - handshake. It's the standard encoding of the OR's curve25519 - public key, encoded in base 64. The trailing '=' sign MAY be - omitted from the base64 encoding. The key MUST be accepted - for at least 1 week after any new key is published in a - subsequent descriptor. - - [Before Tor 0.4.5.1-alpha, this field was optional.] - - "ntor-onion-key-crosscert" SP Bit NL - "-----BEGIN ED25519 CERT-----" NL certificate - "-----END ED25519 CERT-----" NL - - [Exactly once] - [No extra arguments] - - A signature created with the ntor-onion-key, using the - certificate format documented in cert-spec.txt, with type - [0a]. The signed key here is the master identity key. - - Bit must be "0" or "1". It indicates the sign of the ed25519 - public key corresponding to the ntor onion key. If Bit is "0", - then implementations MUST guarantee that the x-coordinate of - the resulting ed25519 public key is positive. Otherwise, if - Bit is "1", then the sign of the x-coordinate MUST be negative. - - To compute the ed25519 public key corresponding to a curve25519 - key, and for further explanation on key formats, see appendix C. - - This signature proves that the party creating the descriptor - had control over the secret key corresponding to the - ntor-onion-key. - - [Before Tor 0.4.5.1-alpha, this field was optional whenever - identity-ed25519 was absent.] - - "signing-key" NL a public key in PEM format - - [Exactly once] - [No extra arguments] - - The OR's long-term RSA identity key. It MUST be 1024 bits. - - The encoding is as for "onion-key" above. - - "accept" exitpattern NL - "reject" exitpattern NL - - [Any number] - - These lines describe an "exit policy": the rules that an OR follows - when deciding whether to allow a new stream to a given address. The - 'exitpattern' syntax is described below. There MUST be at least one - such entry. The rules are considered in order; if no rule matches, - the address will be accepted. For clarity, the last such entry SHOULD - be accept *:* or reject *:*. - - "ipv6-policy" SP ("accept" / "reject") SP PortList NL - - [At most once.] - - An exit-policy summary as specified in sections 3.4.1 and 3.8.2, - summarizing - the router's rules for connecting to IPv6 addresses. A missing - "ipv6-policy" line is equivalent to "ipv6-policy reject - 1-65535". - - "overload-general" SP version SP YYYY-MM-DD HH:MM:SS NL - - [At most once.] - - Indicates that a relay has reached an "overloaded state" which can be - one or many of the following load metrics: - - - Any OOM invocation due to memory pressure - - Any ntor onionskins are dropped - - TCP port exhaustion - - The timestamp is when at least one metrics was detected. It should always - be at the hour and thus, as an example, "2020-01-10 13:00:00" is an - expected timestamp. Because this is a binary state, if the line is - present, we consider that it was hit at the very least once somewhere - between the provided timestamp and the "published" timestamp of the - document which is when the document was generated. - - The overload-general line should remain in place for 72 hours since last - triggered. If the limits are reached again in this period, the timestamp - is updated, and this 72 hour period restarts. - - The 'version' field is set to '1' for now. - - (Introduced in tor-0.4.6.1-alpha, but moved from extra-info to general - descriptor in tor-0.4.6.2-alpha) - - "router-sig-ed25519" SP Signature NL - - [Exactly once.] - - It MUST be the next-to-last element in the descriptor, appearing - immediately before the RSA signature. It MUST contain an Ed25519 - signature of a SHA256 digest of the entire document. This digest is - taken from the first character up to and including the first space - after the "router-sig-ed25519" string. Before computing the digest, - the string "Tor router descriptor signature v1" is prefixed to the - document. - - The signature is encoded in Base64, with terminating =s removed. - - The signing key in the identity-ed25519 certificate MUST - be the one used to sign the document. - - [Before Tor 0.4.5.1-alpha, this field was optional whenever - identity-ed25519 was absent.] - - "router-signature" NL Signature NL - - [At end, exactly once] - [No extra arguments] - - The "SIGNATURE" object contains a signature of the PKCS1-padded - hash of the entire server descriptor, taken from the beginning of the - "router" line, through the newline after the "router-signature" line. - The server descriptor is invalid unless the signature is performed - with the router's identity key. - - "contact" info NL - - [At most once] - - Describes a way to contact the relay's administrator, preferably - including an email address and a PGP key fingerprint. - - "bridge-distribution-request" SP Method NL - - [At most once, bridges only.] - - The "Method" describes how a Bridge address is distributed by - BridgeDB. Recognized methods are: "none", "any", "https", "email", - "moat". If set to "none", BridgeDB will avoid distributing your bridge - address. If set to "any", BridgeDB will choose how to distribute your - bridge address. Choosing any of the other methods will tell BridgeDB to - distribute your bridge via a specific method: - - - "https" specifies distribution via the web interface at - https://bridges.torproject.org; - - "email" specifies distribution via the email autoresponder at - bridges@torproject.org; - - "moat" specifies distribution via an interactive menu inside Tor - Browser; and - - Potential future "Method" specifiers must be as follows: - Method = (KeywordChar | "_") + - - All bridges SHOULD include this line. Non-bridges MUST NOT include - it. - - BridgeDB SHOULD treat unrecognized Method values as if they were - "none". - - (Default: "any") - - [This line was introduced in 0.3.2.3-alpha, with a minimal backport - to 0.2.5.16, 0.2.8.17, 0.2.9.14, 0.3.0.13, 0.3.1.9, and later.] - - "family" names NL - - [At most once] - - 'Names' is a space-separated list of relay nicknames or - hexdigests. If two ORs list one another in their "family" entries, - then OPs should treat them as a single OR for the purpose of path - selection. - - For example, if node A's descriptor contains "family B", and node B's - descriptor contains "family A", then node A and node B should never - be used on the same circuit. - - "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - [At most once] - "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - [At most once] - - (These fields once appeared in router descriptors, but have - appeared in extra-info descriptors since 0.2.0.x.) - - "eventdns" bool NL - - [At most once] - - Declare whether this version of Tor is using the newer enhanced - dns logic. Versions of Tor with this field set to false SHOULD NOT - be used for reverse hostname lookups. - - [This option is obsolete. All Tor current relays should be presumed - to have the evdns backend.] - - "caches-extra-info" NL - - [At most once.] - [No extra arguments] - - Present only if this router is a directory cache that provides - extra-info documents. - - [Versions before 0.2.0.1-alpha don't recognize this] - - "extra-info-digest" SP sha1-digest [SP sha256-digest] NL - - [At most once] - - "sha1-digest" is a hex-encoded SHA1 digest (using upper-case characters) - of the router's extra-info document, as signed in the router's - extra-info (that is, not including the signature). (If this field is - absent, the router is not uploading a corresponding extra-info - document.) - - "sha256-digest" is a base64-encoded SHA256 digest of the extra-info - document. Unlike the "sha1-digest", this digest is calculated over the - entire document, including the signature. This difference is due to - a long-lived bug in the tor implementation that it would be difficult - to roll out an incremental fix for, not a design choice. Future digest - algorithms specified should not include the signature in the data used - to compute the digest. - - [Versions before 0.2.7.2-alpha did not include a SHA256 digest.] - [Versions before 0.2.0.1-alpha don't recognize this field at all.] - - "hidden-service-dir" NL - - [At most once.] - - Present only if this router stores and serves hidden service - descriptors. This router supports the descriptor versions declared - in the HSDir "proto" entry. If there is no "proto" entry, this - router supports version 2 descriptors. - - "protocols" SP "Link" SP LINK-VERSION-LIST SP "Circuit" SP - CIRCUIT-VERSION-LIST NL - - [At most once.] - - An obsolete list of protocol versions, superseded by the "proto" - entry. This list was never parsed, and has not been emitted - since Tor 0.2.9.4-alpha. New code should neither generate nor - parse this line. - - "allow-single-hop-exits" NL - - [At most once.] - [No extra arguments] - - Present only if the router allows single-hop circuits to make exit - connections. Most Tor relays do not support this: this is - included for specialized controllers designed to support perspective - access and such. This is obsolete in tor version >= 0.3.1.0-alpha. - - "or-address" SP ADDRESS ":" PORT NL - - [Any number] - - ADDRESS = IP6ADDR | IP4ADDR - IPV6ADDR = an ipv6 address, surrounded by square brackets. - IPV4ADDR = an ipv4 address, represented as a dotted quad. - PORT = a number between 1 and 65535 inclusive. - - An alternative for the address and ORPort of the "router" line, but with - two added capabilities: - - * or-address can be either an IPv4 or IPv6 address - * or-address allows for multiple ORPorts and addresses - - A descriptor SHOULD NOT include an or-address line that does nothing but - duplicate the address:port pair from its "router" line. - - The ordering of or-address lines and their PORT entries matter because - Tor MAY accept a limited number of address/port pairs. As of - Tor 0.2.3.x only the first address/port pair is advertised and used. - - "tunnelled-dir-server" NL - - [At most once.] - [No extra arguments] - - Present if the router accepts "tunneled" directory requests using a - BEGIN_DIR cell over the router's OR port. - (Added in 0.2.8.1-alpha. Before this, Tor relays accepted - tunneled directory requests only if they had a DirPort open, - or if they were bridges.) - - "proto" SP Entries NL - - [Exactly once.] - - Entries = - Entries = Entry - Entries = Entry SP Entries - - Entry = Keyword "=" Values - - Values = - Values = Value - Values = Value "," Values - - Value = Int - Value = Int "-" Int - - Int = NON_ZERO_DIGIT - Int = Int DIGIT - - Each 'Entry' in the "proto" line indicates that the Tor relay supports - one or more versions of the protocol in question. Entries should be - sorted by keyword. Values should be numerically ascending within each - entry. (This implies that there should be no overlapping ranges.) - Ranges should be represented as compactly as possible. Ints must be no - larger than 63. - - This field was first added in Tor 0.2.9.x. - - [Before Tor 0.4.5.1-alpha, this field was optional.] - - -2.1.2. Extra-info document format - - Extra-info documents consist of the following items: - - "extra-info" Nickname Fingerprint NL - [At start, exactly once.] - - Identifies what router this is an extra-info descriptor for. - Fingerprint is encoded in hex (using upper-case letters), with - no spaces. - - "identity-ed25519" - [As in router descriptors] - - "published" YYYY-MM-DD HH:MM:SS NL - - [Exactly once.] - - The time, in UTC, when this document (and its corresponding router - descriptor if any) was generated. It MUST match the published time - in the corresponding server descriptor. - - "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - [At most once.] - "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - [At most once.] - - Declare how much bandwidth the OR has used recently. Usage is divided - into intervals of NSEC seconds. The YYYY-MM-DD HH:MM:SS field - defines the end of the most recent interval. The numbers are the - number of bytes used in the most recent intervals, ordered from - oldest to newest. - - These fields include both IPv4 and IPv6 traffic. - - "ipv6-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL - [At most once] - "ipv6-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL - [At most once] - - Declare how much bandwidth the OR has used recently, on IPv6 - connections. See "read-history" and "write-history" for full details. - - "geoip-db-digest" Digest NL - [At most once.] - - SHA1 digest of the IPv4 GeoIP database file that is used to - resolve IPv4 addresses to country codes. - - "geoip6-db-digest" Digest NL - [At most once.] - - SHA1 digest of the IPv6 GeoIP database file that is used to - resolve IPv6 addresses to country codes. - - ("geoip-start-time" YYYY-MM-DD HH:MM:SS NL) - ("geoip-client-origins" CC=NUM,CC=NUM,... NL) - - Only generated by bridge routers (see blocking.pdf), and only - when they have been configured with a geoip database. - Non-bridges SHOULD NOT generate these fields. Contains a list - of mappings from two-letter country codes (CC) to the number - of clients that have connected to that bridge from that - country (approximate, and rounded up to the nearest multiple of 8 - in order to hamper traffic analysis). A country is included - only if it has at least one address. The time in - "geoip-start-time" is the time at which we began collecting geoip - statistics. - - "geoip-start-time" and "geoip-client-origins" have been replaced by - "bridge-stats-end" and "bridge-ips" in 0.2.2.4-alpha. The - reason is that the measurement interval with "geoip-stats" as - determined by subtracting "geoip-start-time" from "published" could - have had a variable length, whereas the measurement interval in - 0.2.2.4-alpha and later is set to be exactly 24 hours long. In - order to clearly distinguish the new measurement intervals from - the old ones, the new keywords have been introduced. - - "bridge-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL - [At most once.] - - YYYY-MM-DD HH:MM:SS defines the end of the included measurement - interval of length NSEC seconds (86400 seconds by default). - - A "bridge-stats-end" line, as well as any other "bridge-*" line, - is only added when the relay has been running as a bridge for at - least 24 hours. - - "bridge-ips" CC=NUM,CC=NUM,... NL - [At most once.] - - List of mappings from two-letter country codes to the number of - unique IP addresses that have connected from that country to the - bridge and which are no known relays, rounded up to the nearest - multiple of 8. - - "bridge-ip-versions" FAM=NUM,FAM=NUM,... NL - [At most once.] - - List of unique IP addresses that have connected to the bridge - per protocol family. - - "bridge-ip-transports" PT=NUM,PT=NUM,... NL - [At most once.] - - List of mappings from pluggable transport names to the number - of unique IP addresses that have connected using that - pluggable transport. Unobfuscated connections are counted - using the reserved pluggable transport name "" (without - quotes). If we received a connection from a transport proxy - but we couldn't figure out the name of the pluggable - transport, we use the reserved pluggable transport name - "". - - ("" and "" are reserved because normal pluggable - transport names MUST match the following regular expression: - "[a-zA-Z_][a-zA-Z0-9_]*" ) - - The pluggable transport name list is sorted into lexically - ascending order. - - If no clients have connected to the bridge yet, we only write - "bridge-ip-transports" to the stats file. - - "dirreq-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL - [At most once.] - - YYYY-MM-DD HH:MM:SS defines the end of the included measurement - interval of length NSEC seconds (86400 seconds by default). - - A "dirreq-stats-end" line, as well as any other "dirreq-*" line, - is only added when the relay has opened its Dir port and after 24 - hours of measuring directory requests. - - "dirreq-v2-ips" CC=NUM,CC=NUM,... NL - [At most once.] - "dirreq-v3-ips" CC=NUM,CC=NUM,... NL - [At most once.] - - List of mappings from two-letter country codes to the number of - unique IP addresses that have connected from that country to - request a v2/v3 network status, rounded up to the nearest multiple - of 8. Only those IP addresses are counted that the directory can - answer with a 200 OK status code. (Note here and below: current Tor - versions, as of 0.2.5.2-alpha, no longer cache or serve v2 - networkstatus documents.) - - "dirreq-v2-reqs" CC=NUM,CC=NUM,... NL - [At most once.] - "dirreq-v3-reqs" CC=NUM,CC=NUM,... NL - [At most once.] - - List of mappings from two-letter country codes to the number of - requests for v2/v3 network statuses from that country, rounded up - to the nearest multiple of 8. Only those requests are counted that - the directory can answer with a 200 OK status code. - - "dirreq-v2-share" NUM% NL - [At most once.] - "dirreq-v3-share" NUM% NL - [At most once.] - - The share of v2/v3 network status requests that the directory - expects to receive from clients based on its advertised bandwidth - compared to the overall network bandwidth capacity. Shares are - formatted in percent with two decimal places. Shares are - calculated as means over the whole 24-hour interval. - - "dirreq-v2-resp" status=NUM,... NL - [At most once.] - "dirreq-v3-resp" status=NUM,... NL - [At most once.] - - List of mappings from response statuses to the number of requests - for v2/v3 network statuses that were answered with that response - status, rounded up to the nearest multiple of 4. Only response - statuses with at least 1 response are reported. New response - statuses can be added at any time. The current list of response - statuses is as follows: - - "ok": a network status request is answered; this number - corresponds to the sum of all requests as reported in - "dirreq-v2-reqs" or "dirreq-v3-reqs", respectively, before - rounding up. - "not-enough-sigs: a version 3 network status is not signed by a - sufficient number of requested authorities. - "unavailable": a requested network status object is unavailable. - "not-found": a requested network status is not found. - "not-modified": a network status has not been modified since the - If-Modified-Since time that is included in the request. - "busy": the directory is busy. - - "dirreq-v2-direct-dl" key=NUM,... NL - [At most once.] - "dirreq-v3-direct-dl" key=NUM,... NL - [At most once.] - "dirreq-v2-tunneled-dl" key=NUM,... NL - [At most once.] - "dirreq-v3-tunneled-dl" key=NUM,... NL - [At most once.] - - List of statistics about possible failures in the download process - of v2/v3 network statuses. Requests are either "direct" - HTTP-encoded requests over the relay's directory port, or - "tunneled" requests using a BEGIN_DIR cell over the relay's OR - port. The list of possible statistics can change, and statistics - can be left out from reporting. The current list of statistics is - as follows: - - Successful downloads and failures: - - "complete": a client has finished the download successfully. - "timeout": a download did not finish within 10 minutes after - starting to send the response. - "running": a download is still running at the end of the - measurement period for less than 10 minutes after starting to - send the response. - - Download times: - - "min", "max": smallest and largest measured bandwidth in B/s. - "d[1-4,6-9]": 1st to 4th and 6th to 9th decile of measured - bandwidth in B/s. For a given decile i, i/10 of all downloads - had a smaller bandwidth than di, and (10-i)/10 of all downloads - had a larger bandwidth than di. - "q[1,3]": 1st and 3rd quartile of measured bandwidth in B/s. One - fourth of all downloads had a smaller bandwidth than q1, one - fourth of all downloads had a larger bandwidth than q3, and the - remaining half of all downloads had a bandwidth between q1 and - q3. - "md": median of measured bandwidth in B/s. Half of the downloads - had a smaller bandwidth than md, the other half had a larger - bandwidth than md. - - "dirreq-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL - [At most once] - "dirreq-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL - [At most once] - - Declare how much bandwidth the OR has spent on answering directory - requests. Usage is divided into intervals of NSEC seconds. The - YYYY-MM-DD HH:MM:SS field defines the end of the most recent - interval. The numbers are the number of bytes used in the most - recent intervals, ordered from oldest to newest. - - "entry-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL - [At most once.] - - YYYY-MM-DD HH:MM:SS defines the end of the included measurement - interval of length NSEC seconds (86400 seconds by default). - - An "entry-stats-end" line, as well as any other "entry-*" - line, is first added after the relay has been running for at least - 24 hours. - - "entry-ips" CC=NUM,CC=NUM,... NL - [At most once.] - - List of mappings from two-letter country codes to the number of - unique IP addresses that have connected from that country to the - relay and which are no known other relays, rounded up to the - nearest multiple of 8. - - "cell-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL - [At most once.] - - YYYY-MM-DD HH:MM:SS defines the end of the included measurement - interval of length NSEC seconds (86400 seconds by default). - - A "cell-stats-end" line, as well as any other "cell-*" line, - is first added after the relay has been running for at least 24 - hours. - - "cell-processed-cells" NUM,...,NUM NL - [At most once.] - - Mean number of processed cells per circuit, subdivided into - deciles of circuits by the number of cells they have processed in - descending order from loudest to quietest circuits. - - "cell-queued-cells" NUM,...,NUM NL - [At most once.] - - Mean number of cells contained in queues by circuit decile. These - means are calculated by 1) determining the mean number of cells in - a single circuit between its creation and its termination and 2) - calculating the mean for all circuits in a given decile as - determined in "cell-processed-cells". Numbers have a precision of - two decimal places. - - Note that this statistic can be inaccurate for circuits that had - queued cells at the start or end of the measurement interval. - - "cell-time-in-queue" NUM,...,NUM NL - [At most once.] - - Mean time cells spend in circuit queues in milliseconds. Times are - calculated by 1) determining the mean time cells spend in the - queue of a single circuit and 2) calculating the mean for all - circuits in a given decile as determined in - "cell-processed-cells". - - Note that this statistic can be inaccurate for circuits that had - queued cells at the start or end of the measurement interval. - - "cell-circuits-per-decile" NUM NL - [At most once.] - - Mean number of circuits that are included in any of the deciles, - rounded up to the next integer. - - "conn-bi-direct" YYYY-MM-DD HH:MM:SS (NSEC s) BELOW,READ,WRITE,BOTH NL - [At most once] - - Number of connections, split into 10-second intervals, that are - used uni-directionally or bi-directionally as observed in the NSEC - seconds (usually 86400 seconds) before YYYY-MM-DD HH:MM:SS. Every - 10 seconds, we determine for every connection whether we read and - wrote less than a threshold of 20 KiB (BELOW), read at least 10 - times more than we wrote (READ), wrote at least 10 times more than - we read (WRITE), or read and wrote more than the threshold, but - not 10 times more in either direction (BOTH). After classifying a - connection, read and write counters are reset for the next - 10-second interval. - - This measurement includes both IPv4 and IPv6 connections. - - "ipv6-conn-bi-direct" YYYY-MM-DD HH:MM:SS (NSEC s) BELOW,READ,WRITE,BOTH NL - [At most once] - - Number of IPv6 connections that are used uni-directionally or - bi-directionally. See "conn-bi-direct" for more details. - - "exit-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL - [At most once.] - - YYYY-MM-DD HH:MM:SS defines the end of the included measurement - interval of length NSEC seconds (86400 seconds by default). - - An "exit-stats-end" line, as well as any other "exit-*" line, is - first added after the relay has been running for at least 24 hours - and only if the relay permits exiting (where exiting to a single - port and IP address is sufficient). - - "exit-kibibytes-written" port=N,port=N,... NL - [At most once.] - "exit-kibibytes-read" port=N,port=N,... NL - [At most once.] - - List of mappings from ports to the number of kibibytes that the - relay has written to or read from exit connections to that port, - rounded up to the next full kibibyte. Relays may limit the - number of listed ports and subsume any remaining kibibytes under - port "other". - - "exit-streams-opened" port=N,port=N,... NL - [At most once.] - - List of mappings from ports to the number of opened exit streams - to that port, rounded up to the nearest multiple of 4. Relays may - limit the number of listed ports and subsume any remaining opened - streams under port "other". - - "hidserv-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL - [At most once.] - "hidserv-v3-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL - [At most once.] - - YYYY-MM-DD HH:MM:SS defines the end of the included measurement - interval of length NSEC seconds (86400 seconds by default). - - A "hidserv-stats-end" line, as well as any other "hidserv-*" line, - is first added after the relay has been running for at least 24 - hours. - - (Introduced in tor-0.4.6.1-alpha) - - "hidserv-rend-relayed-cells" SP NUM SP key=val SP key=val ... NL - [At most once.] - "hidserv-rend-v3-relayed-cells" SP NUM SP key=val SP key=val ... NL - [At most once.] - - Approximate number of RELAY cells seen in either direction on a - circuit after receiving and successfully processing a RENDEZVOUS1 - cell. - - The original measurement value is obfuscated in several steps: - first, it is rounded up to the nearest multiple of 'bin_size' - which is reported in the key=val part of this line; second, a - (possibly negative) noise value is added to the result of the - first step by randomly sampling from a Laplace distribution with - mu = 0 and b = (delta_f / epsilon) with 'delta_f' and 'epsilon' - being reported in the key=val part, too; third, the result of the - previous obfuscation steps is truncated to the next smaller - integer and included as 'NUM'. Note that the overall reported - value can be negative. - - (Introduced in tor-0.4.6.1-alpha) - - "hidserv-dir-onions-seen" SP NUM SP key=val SP key=val ... NL - [At most once.] - "hidserv-dir-v3-onions-seen" SP NUM SP key=val SP key=val ... NL - [At most once.] - - Approximate number of unique hidden-service identities seen in - descriptors published to and accepted by this hidden-service - directory. - - The original measurement value is obfuscated in the same way as - the 'NUM' value reported in "hidserv-rend-relayed-cells", but - possibly with different parameters as reported in the key=val part - of this line. Note that the overall reported value can be - negative. - - (Introduced in tor-0.4.6.1-alpha) - - "transport" transportname address:port [arglist] NL - [Any number.] - - Signals that the router supports the 'transportname' pluggable - transport in IP address 'address' and TCP port 'port'. A single - descriptor MUST not have more than one transport line with the - same 'transportname'. - - Pluggable transports are only relevant to bridges, but these entries - can appear in non-bridge relays as well. - - "padding-counts" YYYY-MM-DD HH:MM:SS (NSEC s) key=NUM key=NUM ... NL - [At most once.] - - YYYY-MM-DD HH:MM:SS defines the end of the included measurement - interval of length NSEC seconds (86400 seconds by default). Counts - are reset to 0 at the end of this interval. - - The keyword list is currently as follows: - - bin-size - - The current rounding value for cell count fields (10000 by - default) - write-drop - - The number of RELAY_DROP cells this relay sent - write-pad - - The number of CELL_PADDING cells this relay sent - write-total - - The total number of cells this relay cent - read-drop - - The number of RELAY_DROP cells this relay received - read-pad - - The number of CELL_PADDING cells this relay received - read-total - - The total number of cells this relay received - enabled-read-pad - - The number of CELL_PADDING cells this relay received on - connections that support padding - enabled-read-total - - The total number of cells this relay received on connections - that support padding - enabled-write-pad - - The total number of cells this relay received on connections - that support padding - enabled-write-total - - The total number of cells sent by this relay on connections - that support padding - max-chanpad-timers - - The maximum number of timers that this relay scheduled for - padding in the previous NSEC interval - - "overload-ratelimits" SP version SP YYYY-MM-DD SP HH:MM:SS - SP rate-limit SP burst-limit - SP read-overload-count SP write-overload-count NL - [At most once.] - - Indicates that a bandwidth limit was exhausted for this relay. - - The "rate-limit" and "burst-limit" are the raw values from the - BandwidthRate and BandwidthBurst found in the torrc configuration file. - - The "{read|write}-overload-count" are the counts of how many times the - reported limits of burst/rate were exhausted and thus the maximum - between the read and write count occurrences. To make the counter more - meaningful and to avoid multiple connections saturating the counter - when a relay is overloaded, we only increment it once a minute. - - The 'version' field is set to '1' for now. - - (Introduced in tor-0.4.6.1-alpha) - - "overload-fd-exhausted" SP version YYYY-MM-DD HH:MM:SS NL - [At most once.] - - Indicates that a file descriptor exhaustion was experienced by this - relay. - - The timestamp indicates that the maximum was reached between the - timestamp and the "published" timestamp of the document. - - This overload field should remain in place for 72 hours since last - triggered. If the limits are reached again in this period, the - timestamp is updated, and this 72 hour period restarts. - - The 'version' field is set to '1' for the initial implementation which - detects fd exhaustion only when a socket open fails. - - (Introduced in tor-0.4.6.1-alpha) - - "router-sig-ed25519" - [As in router descriptors] - - "router-signature" NL Signature NL - [At end, exactly once.] - [No extra arguments] - - A document signature as documented in section 1.3, using the - initial item "extra-info" and the final item "router-signature", - signed with the router's identity key. - -2.1.3. Nonterminals in server descriptors - - nickname ::= between 1 and 19 alphanumeric characters ([A-Za-z0-9]), - case-insensitive. - hexdigest ::= a '$', followed by 40 hexadecimal characters - ([A-Fa-f0-9]). [Represents a relay by the digest of its identity - key.] - - exitpattern ::= addrspec ":" portspec - portspec ::= "*" | port | port "-" port - port ::= an integer between 1 and 65535, inclusive. - - [Some implementations incorrectly generate ports with value 0. - Implementations SHOULD accept this, and SHOULD NOT generate it. - Connections to port 0 are never permitted.] - - addrspec ::= "*" | ip4spec | ip6spec - ipv4spec ::= ip4 | ip4 "/" num_ip4_bits | ip4 "/" ip4mask - ip4 ::= an IPv4 address in dotted-quad format - ip4mask ::= an IPv4 mask in dotted-quad format - num_ip4_bits ::= an integer between 0 and 32 - ip6spec ::= ip6 | ip6 "/" num_ip6_bits - ip6 ::= an IPv6 address, surrounded by square brackets. - num_ip6_bits ::= an integer between 0 and 128 - - bool ::= "0" | "1" - -3. Directory authority operation and formats - - Every authority has two keys used in this protocol: a signing key, and - an authority identity key. (Authorities also have a router identity - key used in their role as a router and by earlier versions of the - directory protocol.) The identity key is used from time to time to - sign new key certificates using new signing keys; it is very sensitive. - The signing key is used to sign key certificates and status documents. - -3.1. Creating key certificates - - Key certificates consist of the following items: - - "dir-key-certificate-version" version NL - - [At start, exactly once.] - - Determines the version of the key certificate. MUST be "3" for - the protocol described in this document. Implementations MUST - reject formats they don't understand. - - "dir-address" IPPort NL - [At most once] - - An IP:Port for this authority's directory port. - - "fingerprint" fingerprint NL - - [Exactly once.] - - Hexadecimal encoding without spaces based on the authority's - identity key. - - "dir-identity-key" NL a public key in PEM format - - [Exactly once.] - [No extra arguments] - - The long-term authority identity key for this authority. This key - SHOULD be at least 2048 bits long; it MUST NOT be shorter than - 1024 bits. - - "dir-key-published" YYYY-MM-DD HH:MM:SS NL - - [Exactly once.] - - The time (in UTC) when this document and corresponding key were - last generated. - - Implementations SHOULD reject certificates that are published - too far in the future, though they MAY tolerate some clock skew. - - "dir-key-expires" YYYY-MM-DD HH:MM:SS NL - - [Exactly once.] - - A time (in UTC) after which this key is no longer valid. - - Implementations SHOULD reject expired certificates, though they - MAY tolerate some clock skew. - - "dir-signing-key" NL a key in PEM format - - [Exactly once.] - [No extra arguments] - - The directory server's public signing key. This key MUST be at - least 1024 bits, and MAY be longer. - - "dir-key-crosscert" NL CrossSignature NL - - [Exactly once.] - [No extra arguments] - - CrossSignature is a signature, made using the certificate's signing - key, of the digest of the PKCS1-padded hash of the certificate's - identity key. For backward compatibility with broken versions of the - parser, we wrap the base64-encoded signature in -----BEGIN ID - SIGNATURE---- and -----END ID SIGNATURE----- tags. Implementations - MUST allow the "ID " portion to be omitted, however. - - Implementations MUST verify that the signature is a correct signature - of the hash of the identity key using the signing key. - - "dir-key-certification" NL Signature NL - - [At end, exactly once.] - [No extra arguments] - - A document signature as documented in section 1.3, using the - initial item "dir-key-certificate-version" and the final item - "dir-key-certification", signed with the authority identity key. - - Authorities MUST generate a new signing key and corresponding - certificate before the key expires. - -3.2. Accepting server descriptor and extra-info document uploads - - When a router posts a signed descriptor to a directory authority, the - authority first checks whether it is well-formed and correctly - self-signed. If it is, the authority next verifies that the nickname - in question is not already assigned to a router with a different - public key. - Finally, the authority MAY check that the router is not blacklisted - because of its key, IP, or another reason. - - An authority also keeps a record of all the Ed25519/RSA1024 - identity key pairs that it has seen before. It rejects any - descriptor that has a known Ed/RSA identity key that it has - already seen accompanied by a different RSA/Ed identity key - in an older descriptor. - - At a future date, authorities will begin rejecting all - descriptors whose RSA key was previously accompanied by an - Ed25519 key, if the descriptor does not list an Ed25519 key. - - At a future date, authorities will begin rejecting all descriptors - that do not list an Ed25519 key. - - If the descriptor passes these tests, and the authority does not already - have a descriptor for a router with this public key, it accepts the - descriptor and remembers it. - - If the authority _does_ have a descriptor with the same public key, the - newly uploaded descriptor is remembered if its publication time is more - recent than the most recent old descriptor for that router, and either: - - - There are non-cosmetic differences between the old descriptor and the - new one. - - Enough time has passed between the descriptors' publication times. - (Currently, 2 hours.) - - Differences between server descriptors are "non-cosmetic" if they would be - sufficient to force an upload as described in section 2.1 above. - - Note that the "cosmetic difference" test only applies to uploaded - descriptors, not to descriptors that the authority downloads from other - authorities. - - When a router posts a signed extra-info document to a directory authority, - the authority again checks it for well-formedness and correct signature, - and checks that its matches the extra-info-digest in some router - descriptor that it believes is currently useful. If so, it accepts it and - stores it and serves it as requested. If not, it drops it. - - -3.3. Computing microdescriptors - - Microdescriptors are a stripped-down version of server descriptors - generated by the directory authorities which may additionally contain - authority-generated information. Microdescriptors contain only the - most relevant parts that clients care about. Microdescriptors are - expected to be relatively static and only change about once per week. - Microdescriptors do not contain any information that clients need to - use to decide which servers to fetch information about, or which - servers to fetch information from. - - Microdescriptors are a straight transform from the server descriptor - and the consensus method. Microdescriptors have no header or footer. - Microdescriptors are identified by the hash of its concatenated - elements without a signature by the router. Microdescriptors do not - contain any version information, because their version is determined - by the consensus method. - - Starting with consensus method 8, microdescriptors contain the - following elements taken from or based on the server descriptor. Order - matters here, because different directory authorities must be able to - transform a given server descriptor and consensus method into the exact - same microdescriptor. - - "onion-key" NL a public key in PEM format - - [Exactly once, at start] - [No extra arguments] - - The "onion-key" element as specified in section 2.1.1. - - When generating microdescriptors for consensus method 30 or later, - the trailing = sign must be absent. For consensus method 29 or - earlier, the trailing = sign must be present. - - "ntor-onion-key" SP base-64-encoded-key NL - - [Exactly once] - - The "ntor-onion-key" element as specified in section 2.1.1. - - (Only included when generating microdescriptors for - consensus-method 16 or later.) - - [Before Tor 0.4.5.1-alpha, this field was optional.] - - "a" SP address ":" port NL - - [Any number] - - Additional advertised addresses for the OR. - - Present currently only if the OR advertises at least one IPv6 - address; currently, the first address is included and all others are - omitted. Any other IPv4 or IPv6 addresses should be ignored. - - Address and port are as for "or-address" as specified in - section 2.1.1. - - (Only included when generating microdescriptors for - consensus-methods 14 to 27.) - - "family" names NL - - [At most once] - - The "family" element as specified in section 2.1.1. - - When generating microdescriptors for consensus method 29 or later, - the following canonicalization algorithm is applied to improve - compression: - - For all entries of the form $hexid=name or $hexid~name, - remove the =name or ~name portion. - - Remove all entries of the form $hexid, where hexid is not - 40 hexadecimal characters long. - - If an entry is a valid nickname, put it into lower case. - - If an entry is a valid $hexid, put it into upper case. - - If there are any entries, add a single $hexid entry for - the relay in question, so that it is a member of its own - family. - - Sort all entries in lexical order. - - Remove duplicate entries. - - (Note that if an entry is not of the form "nickname", "$hexid", - "$hexid=nickname" or "$hexid~nickname", then it will be unchanged: - this is what makes the algorithm forward-compatible.) - - "p" SP ("accept" / "reject") SP PortList NL - - [Exactly once.] - - The exit-policy summary as specified in sections 3.4.1 and 3.8.2. - - [With microdescriptors, clients don't learn exact exit policies: - clients can only guess whether a relay accepts their request, try the - BEGIN request, and might get end-reason-exit-policy if they guessed - wrong, in which case they'll have to try elsewhere.] - - [In consensus methods before 5, this line was omitted.] - - "p6" SP ("accept" / "reject") SP PortList NL - - [At most once] - - The IPv6 exit policy summary as specified in sections 3.4.1 and - 3.8.2. A missing "p6" line is equivalent to "p6 reject 1-65535". - - (Only included when generating microdescriptors for - consensus-method 15 or later.) - - "id" SP "rsa1024" SP base64-encoded-identity-digest NL - - [At most once] - - The node identity digest (as described in tor-spec.txt), base64 - encoded, without trailing =s. This line is included to prevent - collisions between microdescriptors. - - Implementations SHOULD ignore these lines: they are - added to microdescriptors only to prevent collisions. - - (Only included when generating microdescriptors for - consensus-method 18 or later.) - - "id" SP "ed25519" SP base64-encoded-ed25519-identity NL - - [At most once] - - The node's master Ed25519 identity key, base64 encoded, - without trailing =s. - - All implementations MUST ignore this key for any microdescriptor - whose corresponding entry in the consensus includes the - 'NoEdConsensus' flag. - - (Only included when generating microdescriptors for - consensus-method 21 or later.) - - "id" SP keytype ... NL - - [At most once per distinct keytype.] - - Implementations MUST ignore "id" lines with unrecognized - key-types in place of "rsa1024" or "ed25519" - - "pr" SP Entries NL - - [Exactly once.] - - The "proto" element as specified in section 2.1.1. - - [Before Tor 0.4.5.1-alpha, this field was optional.] - - (Note that with microdescriptors, clients do not learn the RSA identity of - their routers: they only learn a hash of the RSA identity key. This is - all they need to confirm the actual identity key when doing a TLS - handshake, and all they need to put the identity key digest in their - CREATE cells.) - -3.4. Exchanging votes - - Authorities divide time into Intervals. Authority administrators SHOULD - try to all pick the same interval length, and SHOULD pick intervals that - are commonly used divisions of time (e.g., 5 minutes, 15 minutes, 30 - minutes, 60 minutes, 90 minutes). Voting intervals SHOULD be chosen to - divide evenly into a 24-hour day. - - Authorities SHOULD act according to interval and delays in the - latest consensus. Lacking a latest consensus, they SHOULD default to a - 30-minute Interval, a 5 minute VotingDelay, and a 5 minute DistDelay. - - Authorities MUST take pains to ensure that their clocks remain accurate - within a few seconds. (Running NTP is usually sufficient.) - - The first voting period of each day begins at 00:00 (midnight) UTC. If - the last period of the day would be truncated by one-half or more, it is - merged with the second-to-last period. - - An authority SHOULD publish its vote immediately at the start of each voting - period (minus VoteSeconds+DistSeconds). It does this by making it - available at - - http:///tor/status-vote/next/authority.z - - and sending it in an HTTP POST request to each other authority at the URL - - http:///tor/post/vote - - If, at the start of the voting period, minus DistSeconds, an authority - does not have a current statement from another authority, the first - authority downloads the other's statement. - - Once an authority has a vote from another authority, it makes it available - at - - http:///tor/status-vote/next/.z - - where is the fingerprint of the other authority's identity key. - And at - - http:///tor/status-vote/next/d/.z - - where is the digest of the vote document. - - Also, once an authority receives a vote from another authority, it - examines it for new descriptors and fetches them from that authority. - This may be the only way for an authority to hear about relays that didn't - publish their descriptor to all authorities, and, while it's too late - for the authority to include relays in its current vote, it can include - them in its next vote. See section 3.6 below for details. - -3.4.1. Vote and consensus status document formats - - Votes and consensuses are more strictly formatted than other documents - in this specification, since different authorities must be able to - generate exactly the same consensus given the same set of votes. - - The procedure for deciding when to generate vote and consensus status - documents are described in section 1.4 on the voting timeline. - - Status documents contain a preamble, an authority section, a list of - router status entries, and one or more footer signature, in that order. - - Unlike other formats described above, a SP in these documents must be a - single space character (hex 20). - - Some items appear only in votes, and some items appear only in - consensuses. Unless specified, items occur in both. - - The preamble contains the following items. They SHOULD occur in the - order given here: - - "network-status-version" SP version NL - - [At start, exactly once.] - - A document format version. For this specification, the version is - "3". - - "vote-status" SP type NL - - [Exactly once.] - - The status MUST be "vote" or "consensus", depending on the type of - the document. - - "consensus-methods" SP IntegerList NL - - [At most once for votes; does not occur in consensuses.] - - A space-separated list of supported methods for generating - consensuses from votes. See section 3.8.1 for details. Absence of - the line means that only method "1" is supported. - - "consensus-method" SP Integer NL - - [At most once for consensuses; does not occur in votes.] - [No extra arguments] - - See section 3.8.1 for details. - - (Only included when the vote is generated with consensus-method 2 or - later.) - - "published" SP YYYY-MM-DD SP HH:MM:SS NL - - [Exactly once for votes; does not occur in consensuses.] - - The publication time for this status document (if a vote). - - "valid-after" SP YYYY-MM-DD SP HH:MM:SS NL - - [Exactly once.] - - The start of the Interval for this vote. Before this time, the - consensus document produced from this vote is not officially in - use. - - (Note that because of propagation delays, clients and relays - may see consensus documents that are up to `DistSeconds` - earlier than this time, and should not warn about them.) - - See section 1.4 for voting timeline information. - - "fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL - - [Exactly once.] - - The time at which the next consensus should be produced; before this - time, there is no point in downloading another consensus, since there - won't be a new one. See section 1.4 for voting timeline information. - - "valid-until" SP YYYY-MM-DD SP HH:MM:SS NL - - [Exactly once.] - - The end of the Interval for this vote. After this time, all - clients should try to find a more recent consensus. See section 1.4 - for voting timeline information. - - In practice, clients continue to use the consensus for up to 24 hours - after it is no longer valid, if no more recent consensus can be - downloaded. - - "voting-delay" SP VoteSeconds SP DistSeconds NL - - [Exactly once.] - - VoteSeconds is the number of seconds that we will allow to collect - votes from all authorities; DistSeconds is the number of seconds - we'll allow to collect signatures from all authorities. See - section 1.4 for voting timeline information. - - "client-versions" SP VersionList NL - - [At most once.] - - A comma-separated list of recommended Tor versions for client - usage, in ascending order. The versions are given as defined by - version-spec.txt. If absent, no opinion is held about client - versions. - - "server-versions" SP VersionList NL - - [At most once.] - - A comma-separated list of recommended Tor versions for relay - usage, in ascending order. The versions are given as defined by - version-spec.txt. If absent, no opinion is held about server - versions. - - "package" SP PackageName SP Version SP URL SP DIGESTS NL - - [Any number of times.] - - For this element: - - PACKAGENAME = NONSPACE - VERSION = NONSPACE - URL = NONSPACE - DIGESTS = DIGEST | DIGESTS SP DIGEST - DIGEST = DIGESTTYPE "=" DIGESTVAL - NONSPACE = one or more non-space printing characters - DIGESTVAL = DIGESTTYPE = one or more non-space printing characters - other than "=". - - Indicates that a package called "package" of version VERSION may be - found at URL, and its digest as computed with DIGESTTYPE is equal to - DIGESTVAL. In consensuses, these lines are sorted lexically by - "PACKAGENAME VERSION" pairs, and DIGESTTYPES must appear in ascending - order. A consensus must not contain the same "PACKAGENAME VERSION" - more than once. If a vote contains the same "PACKAGENAME VERSION" - more than once, all but the last is ignored. - - Included in consensuses only for methods 19-33. Earlier methods - did not include this; method 34 removed it. - - "known-flags" SP FlagList NL - - [Exactly once.] - - A space-separated list of all of the flags that this document - might contain. A flag is "known" either because the authority - knows about them and might set them (if in a vote), or because - enough votes were counted for the consensus for an authoritative - opinion to have been formed about their status. - - "flag-thresholds" SP Thresholds NL - - [At most once for votes; does not occur in consensuses.] - - A space-separated list of the internal performance thresholds - that the directory authority had at the moment it was forming - a vote. - - The metaformat is: - Thresholds = Threshold | Threshold SP Thresholds - Threshold = ThresholdKey '=' ThresholdVal - ThresholdKey = (KeywordChar | "_") + - ThresholdVal = [0-9]+("."[0-9]+)? "%"? - - Commonly used Thresholds at this point include: - - "stable-uptime" -- Uptime (in seconds) required for a relay - to be marked as stable. - - "stable-mtbf" -- MTBF (in seconds) required for a relay to be - marked as stable. - - "enough-mtbf" -- Whether we have measured enough MTBF to look - at stable-mtbf instead of stable-uptime. - - "fast-speed" -- Bandwidth (in bytes per second) required for - a relay to be marked as fast. - - "guard-wfu" -- WFU (in seconds) required for a relay to be - marked as guard. - - "guard-tk" -- Weighted Time Known (in seconds) required for a - relay to be marked as guard. - - "guard-bw-inc-exits" -- If exits can be guards, then all guards - must have a bandwidth this high. - - "guard-bw-exc-exits" -- If exits can't be guards, then all guards - must have a bandwidth this high. - - "ignoring-advertised-bws" -- 1 if we have enough measured bandwidths - that we'll ignore the advertised bandwidth - claims of routers without measured bandwidth. - - "recommended-client-protocols" SP Entries NL - "recommended-relay-protocols" SP Entries NL - "required-client-protocols" SP Entries NL - "required-relay-protocols" SP Entries NL - - [At most once for each.] - - The "proto" element as specified in section 2.1.1. - - To vote on these entries, a protocol/version combination is included - only if it is listed by a majority of the voters. - - These lines should be voted on. A majority of votes is sufficient to - make a protocol un-supported. A supermajority of authorities (2/3) - are needed to make a protocol required. The required protocols - should not be torrc-configurable, but rather should be hardwired in - the Tor code. - - The tor-spec.txt section 9 details how a relay and a client should - behave when they encounter these lines in the consensus. - - "params" SP [Parameters] NL - - [At most once] - - Parameter ::= Keyword '=' Int32 - Int32 ::= A decimal integer between -2147483648 and 2147483647. - Parameters ::= Parameter | Parameters SP Parameter - - The parameters list, if present, contains a space-separated list of - case-sensitive key-value pairs, sorted in lexical order by their - keyword (as ASCII byte strings). Each parameter has its own meaning. - - (Only included when the vote is generated with consensus-method 7 or - later.) - - See param-spec.txt for a list of parameters and their meanings. - - "shared-rand-previous-value" SP NumReveals SP Value NL - - [At most once] - - NumReveals ::= An integer greater or equal to 0. - Value ::= Base64-encoded-data - - The shared_random_value that was generated during the second-to-last - shared randomness protocol run. For example, if this document was - created on the 5th of November, this field carries the shared random - value generated during the protocol run of the 3rd of November. - - See section [SRCALC] of srv-spec.txt for instructions on how to compute - this value, and see section [CONS] for why we include old shared random - values in votes and consensus. - - Value is the actual shared random value encoded in base64. It will - be exactly 256 bits long. NumReveals is the number of commits used - to generate this SRV. - - "shared-rand-current-value" SP NumReveals SP Value NL - - [At most once] - - NumReveals ::= An integer greater or equal to 0. - Value ::= Base64-encoded-data - - The shared_random_value that was generated during the latest shared - randomness protocol run. For example, if this document was created on - the 5th of November, this field carries the shared random value - generated during the protocol run of the 4th of November - - See section [SRCALC] of srv-spec.txt for instructions on how to compute - this value given the active commits. - - Value is the actual shared random value encoded in base64. It will - be exactly 256 bits long. NumReveals is the number of commits used to - generate this SRV. - - "bandwidth-file-headers" SP KeyValues NL - - [At most once for votes; does not occur in consensuses.] - - KeyValues ::= "" | KeyValue | KeyValues SP KeyValue - KeyValue ::= Keyword '=' Value - Value ::= ArgumentCharValue+ - ArgumentCharValue ::= any printing ASCII character except NL and SP. - - The headers from the bandwidth file used to generate this vote. - The bandwidth file headers are described in bandwidth-file-spec.txt. - - If an authority is not configured with a V3BandwidthsFile, this line - SHOULD NOT appear in its vote. - - If an authority is configured with a V3BandwidthsFile, but parsing - fails, this line SHOULD appear in its vote, but without any headers. - - First-appeared: Tor 0.3.5.1-alpha. - - "bandwidth-file-digest" 1*(SP algorithm "=" digest) NL - - [At most once for votes; does not occur in consensuses.] - - A digest of the bandwidth file used to generate this vote. - "algorithm" is the name of the hash algorithm producing "digest", - which can be "sha256" or another algorithm. "digest" is the - base64 encoding of the hash of the bandwidth file, with trailing =s - omitted. - - If an authority is not configured with a V3BandwidthsFile, this line - SHOULD NOT appear in its vote. - - If an authority is configured with a V3BandwidthsFile, but parsing - fails, this line SHOULD appear in its vote, with the digest(s) of the - unparseable file. - - First-appeared: Tor 0.4.0.4-alpha - - The authority section of a vote contains the following items, followed - in turn by the authority's current key certificate: - - "dir-source" SP nickname SP identity SP address SP IP SP dirport SP - orport NL - - [Exactly once, at start] - - Describes this authority. The nickname is a convenient identifier - for the authority. The identity is an uppercase hex fingerprint of - the authority's current (v3 authority) identity key. The address is - the server's hostname. The IP is the server's current IP address, - and dirport is its current directory port. The orport is the - port at that address where the authority listens for OR - connections. - - "contact" SP string NL - - [Exactly once] - - An arbitrary string describing how to contact the directory - server's administrator. Administrators should include at least an - email address and a PGP fingerprint. - - "legacy-dir-key" SP FINGERPRINT NL - - [At most once] - - Lists a fingerprint for an obsolete _identity_ key still used - by this authority to keep older clients working. This option - is used to keep key around for a little while in case the - authorities need to migrate many identity keys at once. - (Generally, this would only happen because of a security - vulnerability that affected multiple authorities, like the - Debian OpenSSL RNG bug of May 2008.) - - "shared-rand-participate" NL - - [At most once] - - Denotes that the directory authority supports and can participate in the - shared random protocol. - - "shared-rand-commit" SP Version SP AlgName SP Identity SP Commit [SP Reveal] NL - - [Any number of times] - - Version ::= An integer greater or equal to 0. - AlgName ::= 1*(ALPHA / DIGIT / "_" / "-") - Identity ::= 40 * HEXDIG - Commit ::= Base64-encoded-data - Reveal ::= Base64-encoded-data - - Denotes a directory authority commit for the shared randomness - protocol, containing the commitment value and potentially also the - reveal value. See sections [COMMITREVEAL] and [VALIDATEVALUES] of - srv-spec.txt on how to generate and validate these values. - - Version is the current shared randomness protocol version. AlgName is - the hash algorithm that is used (e.g. "sha3-256") and Identity is the - authority's SHA1 v3 identity fingerprint. Commit is the encoded - commitment value in base64. Reveal is optional and if it's set, it - contains the reveal value in base64. - - If a vote contains multiple commits from the same authority, the - receiver MUST only consider the first commit listed. - - "shared-rand-previous-value" SP NumReveals SP Value NL - - [At most once] - - See shared-rand-previous-value description above. - - "shared-rand-current-value" SP NumReveals SP Value NL - - [At most once] - - See shared-rand-current-value description above. - - The authority section of a consensus contains groups of the following items, - in the order given, with one group for each authority that contributed to - the consensus, with groups sorted by authority identity digest: - - "dir-source" SP nickname SP identity SP address SP IP SP dirport SP - orport NL - - [Exactly once, at start] - - As in the authority section of a vote. - - "contact" SP string NL - - [Exactly once.] - - As in the authority section of a vote. - - "vote-digest" SP digest NL - - [Exactly once.] - - A digest of the vote from the authority that contributed to this - consensus, as signed (that is, not including the signature). - (Hex, upper-case.) - - For each "legacy-dir-key" in the vote, there is an additional "dir-source" - line containing that legacy key's fingerprint, the authority's nickname - with "-legacy" appended, and all other fields as in the main "dir-source" - line for that authority. These "dir-source" lines do not have - corresponding "contact" or "vote-digest" entries. - - Each router status entry contains the following items. Router status - entries are sorted in ascending order by identity digest. - - "r" SP nickname SP identity SP digest SP publication SP IP SP ORPort - SP DirPort NL - - [At start, exactly once.] - - "Nickname" is the OR's nickname. "Identity" is a hash of its - identity key, encoded in base64, with trailing equals sign(s) - removed. "Digest" is a hash of its most recent descriptor as - signed (that is, not including the signature) by the RSA identity - key (see section 1.3.), encoded in base64. - - "Publication" was once the publication time of the router's most - recent descriptor, in the form YYYY-MM-DD HH:MM:SS, in UTC. Now - it is only used in votes, and may be set to a fixed value in - consensus documents. Implementations SHOULD ignore this value - in non-vote documents. - - "IP" is its current IP address; ORPort is its current OR port, - "DirPort" is its current directory port, or "0" for "none". - - "a" SP address ":" port NL - - [Any number] - - The first advertised IPv6 address for the OR, if it is reachable. - - Present only if the OR advertises at least one IPv6 address, and the - authority believes that the first advertised address is reachable. - Any other IPv4 or IPv6 addresses should be ignored. - - Address and port are as for "or-address" as specified in - section 2.1.1. - - (Only included when the vote or consensus is generated with - consensus-method 14 or later.) - - "s" SP Flags NL - - [Exactly once.] - - A series of space-separated status flags, in lexical order (as ASCII - byte strings). Currently documented flags are: - - "Authority" if the router is a directory authority. - "BadExit" if the router is believed to be useless as an exit node - (because its ISP censors it, because it is behind a restrictive - proxy, or for some similar reason). - "Exit" if the router is more useful for building - general-purpose exit circuits than for relay circuits. The - path building algorithm uses this flag; see path-spec.txt. - "Fast" if the router is suitable for high-bandwidth circuits. - "Guard" if the router is suitable for use as an entry guard. - "HSDir" if the router is considered a v2 hidden service directory. - "MiddleOnly" if the router is considered unsuitable for - usage other than as a middle relay. Clients do not need - to handle this option, since when it is present, the authorities - will automatically vote against flags that would make the router - usable in other positions. (Since 0.4.7.2-alpha.) - "NoEdConsensus" if any Ed25519 key in the router's descriptor or - microdescriptor does not reflect authority consensus. - "Stable" if the router is suitable for long-lived circuits. - "StaleDesc" if the router should upload a new descriptor because - the old one is too old. - "Running" if the router is currently usable over all its published - ORPorts. (Authorities ignore IPv6 ORPorts unless configured to - check IPv6 reachability.) Relays without this flag are omitted - from the consensus, and current clients (since 0.2.9.4-alpha) - assume that every listed relay has this flag. - "Valid" if the router has been 'validated'. Clients before - 0.2.9.4-alpha would not use routers without this flag by - default. Currently, relays without this flag are omitted - from the consensus, and current (post-0.2.9.4-alpha) clients - assume that every listed relay has this flag. - "V2Dir" if the router implements the v2 directory protocol or - higher. - - "v" SP version NL - - [At most once.] - - The version of the Tor protocol that this relay is running. If - the value begins with "Tor" SP, the rest of the string is a Tor - version number, and the protocol is "The Tor protocol as supported - by the given version of Tor." Otherwise, if the value begins with - some other string, Tor has upgraded to a more sophisticated - protocol versioning system, and the protocol is "a version of the - Tor protocol more recent than any we recognize." - - Directory authorities SHOULD omit version strings they receive from - descriptors if they would cause "v" lines to be over 128 characters - long. - - "pr" SP Entries NL - - [At most once.] - - The "proto" family element as specified in section 2.1.1. - - During voting, authorities copy these lines immediately below the "v" - lines. When a descriptor does not contain a "proto" entry, the - authorities should reconstruct it using the approach described below - in section D. They are included in the consensus using the same rules - as currently used for "v" lines, if a sufficiently late consensus - method is in use. - - "w" SP "Bandwidth=" INT [SP "Measured=" INT] [SP "Unmeasured=1"] NL - - [At most once.] - - An estimate of the bandwidth of this relay, in an arbitrary - unit (currently kilobytes per second). Used to weight router - selection. See section 3.4.2 for details on how the value of - Bandwidth is determined in a consensus. - - Additionally, the Measured= keyword is present in votes by - participating bandwidth measurement authorities to indicate - a measured bandwidth currently produced by measuring stream - capacities. It does not occur in consensuses. - - 'Bandwidth=' and 'Measured=' values must be between 0 and - 2^32 - 1 inclusive. - - The "Unmeasured=1" value is included in consensuses generated - with method 17 or later when the 'Bandwidth=' value is not - based on a threshold of 3 or more measurements for this relay. - - Other weighting keywords may be added later. - Clients MUST ignore keywords they do not recognize. - - "p" SP ("accept" / "reject") SP PortList NL - - [At most once.] - - PortList = PortOrRange - PortList = PortList "," PortOrRange - PortOrRange = INT "-" INT / INT - - A list of those ports that this router supports (if 'accept') - or does not support (if 'reject') for exit to "most - addresses". - - "m" SP methods 1*(SP algorithm "=" digest) NL - - [Any number, only in votes.] - - Microdescriptor hashes for all consensus methods that an authority - supports and that use the same microdescriptor format. "methods" - is a comma-separated list of the consensus methods that the - authority believes will produce "digest". "algorithm" is the name - of the hash algorithm producing "digest", which can be "sha256" or - something else, depending on the consensus "methods" supporting - this algorithm. "digest" is the base64 encoding of the hash of - the router's microdescriptor with trailing =s omitted. - - "id" SP "ed25519" SP ed25519-identity NL - "id" SP "ed25519" SP "none" NL - [vote only, at most once] - - "stats" SP [KeyValues] NL - - [At most once. Vote only] - - KeyValue ::= Keyword '=' Number - Number ::= [0-9]+("."[0-9]+)? - KeyValues ::= KeyValue | KeyValues SP KeyValue - - Line containing various statistics that an authority has computed for - this relay. Each stats is represented as a key + value. Reported keys - are: - - "wfu" - Weighted Fractional Uptime - "tk" - Weighted Time Known - "mtbf" - Mean Time Between Failure (stability) - - (As of tor-0.4.6.1-alpha) - - The footer section is delineated in all votes and consensuses supporting - consensus method 9 and above with the following: - - "directory-footer" NL - [No extra arguments] - - It contains two subsections, a bandwidths-weights line and a - directory-signature. (Prior to consensus method 9, footers only contained - directory-signatures without a 'directory-footer' line or - bandwidth-weights.) - - The bandwidths-weights line appears At Most Once for a consensus. It does - not appear in votes. - - "bandwidth-weights" [SP Weights] NL - - Weight ::= Keyword '=' Int32 - Int32 ::= A decimal integer between -2147483648 and 2147483647. - Weights ::= Weight | Weights SP Weight - - List of optional weights to apply to router bandwidths during path - selection. They are sorted in lexical order (as ASCII byte strings) and - values are divided by the consensus' "bwweightscale" param. Definition - of our known entries are... - - Wgg - Weight for Guard-flagged nodes in the guard position - Wgm - Weight for non-flagged nodes in the guard Position - Wgd - Weight for Guard+Exit-flagged nodes in the guard Position - - Wmg - Weight for Guard-flagged nodes in the middle Position - Wmm - Weight for non-flagged nodes in the middle Position - Wme - Weight for Exit-flagged nodes in the middle Position - Wmd - Weight for Guard+Exit flagged nodes in the middle Position - - Weg - Weight for Guard flagged nodes in the exit Position - Wem - Weight for non-flagged nodes in the exit Position - Wee - Weight for Exit-flagged nodes in the exit Position - Wed - Weight for Guard+Exit-flagged nodes in the exit Position - - Wgb - Weight for BEGIN_DIR-supporting Guard-flagged nodes - Wmb - Weight for BEGIN_DIR-supporting non-flagged nodes - Web - Weight for BEGIN_DIR-supporting Exit-flagged nodes - Wdb - Weight for BEGIN_DIR-supporting Guard+Exit-flagged nodes - - Wbg - Weight for Guard flagged nodes for BEGIN_DIR requests - Wbm - Weight for non-flagged nodes for BEGIN_DIR requests - Wbe - Weight for Exit-flagged nodes for BEGIN_DIR requests - Wbd - Weight for Guard+Exit-flagged nodes for BEGIN_DIR requests - - These values are calculated as specified in section 3.8.3. - - The signature contains the following item, which appears Exactly Once - for a vote, and At Least Once for a consensus. - - "directory-signature" [SP Algorithm] SP identity SP signing-key-digest - NL Signature - - This is a signature of the status document, with the initial item - "network-status-version", and the signature item - "directory-signature", using the signing key. (In this case, we take - the hash through the _space_ after directory-signature, not the - newline: this ensures that all authorities sign the same thing.) - "identity" is the hex-encoded digest of the authority identity key of - the signing authority, and "signing-key-digest" is the hex-encoded - digest of the current authority signing key of the signing authority. - - The Algorithm is one of "sha1" or "sha256" if it is present; - implementations MUST ignore directory-signature entries with an - unrecognized Algorithm. "sha1" is the default, if no Algorithm is - given. The algorithm describes how to compute the hash of the - document before signing it. - - "ns"-flavored consensus documents must contain only sha1 signatures. - Votes and microdescriptor documents may contain other signature - types. Note that only one signature from each authority should be - "counted" as meaning that the authority has signed the consensus. - - (Tor clients before 0.2.3.x did not understand the 'algorithm' - field.) - -3.4.2. Assigning flags in a vote - - (This section describes how directory authorities choose which status - flags to apply to routers. Later directory authorities MAY do things - differently, so long as clients keep working well. Clients MUST NOT - depend on the exact behaviors in this section.) - - In the below definitions, a router is considered "active" if it is - running, valid, and not hibernating. - - When we speak of a router's bandwidth in this section, we mean either - its measured bandwidth, or its advertised bandwidth. If a sufficient - threshold (configurable with MinMeasuredBWsForAuthToIgnoreAdvertised, - 500 by default) of routers have measured bandwidth values, then the - authority bases flags on _measured_ bandwidths, and treats nodes with - non-measured bandwidths as if their bandwidths were zero. Otherwise, - it uses measured bandwidths for nodes that have them, and advertised - bandwidths for other nodes. - - When computing thresholds based on percentiles of nodes, an authority - only considers nodes that are active, that have not been - omitted as a sybil (see below), and whose bandwidth is at least - 4 KB. Nodes that don't meet these criteria do not influence any - threshold calculations (including calculation of stability and uptime - and bandwidth thresholds) and also do not have their Exit status - change. - - "Valid" -- a router is 'Valid' if it is running a version of Tor not - known to be broken, and the directory authority has not blacklisted - it as suspicious. - - "Named" -- - "Unnamed" -- Directory authorities no longer assign these flags. - They were once used to determine whether a relay's nickname was - canonically linked to its public key. - - "Running" -- A router is 'Running' if the authority managed to connect to - it successfully within the last 45 minutes on all its published ORPorts. - Authorities check reachability on: - - * the IPv4 ORPort in the "r" line, and - * the IPv6 ORPort considered for the "a" line, if: - * the router advertises at least one IPv6 ORPort, and - * AuthDirHasIPv6Connectivity 1 is set on the authority. - - A minority of voting authorities that set AuthDirHasIPv6Connectivity will - drop unreachable IPv6 ORPorts from the full consensus. Consensus method 27 - in 0.3.3.x puts IPv6 ORPorts in the microdesc consensus, so that - authorities can drop unreachable IPv6 ORPorts from all consensus flavors. - Consensus method 28 removes IPv6 ORPorts from microdescriptors. - - "Stable" -- A router is 'Stable' if it is active, and either its Weighted - MTBF is at least the median for known active routers or its Weighted MTBF - corresponds to at least 7 days. Routers are never called Stable if they are - running a version of Tor known to drop circuits stupidly. (0.1.1.10-alpha - through 0.1.1.16-rc are stupid this way.) - - To calculate weighted MTBF, compute the weighted mean of the lengths - of all intervals when the router was observed to be up, weighting - intervals by $\alpha^n$, where $n$ is the amount of time that has - passed since the interval ended, and $\alpha$ is chosen so that - measurements over approximately one month old no longer influence the - weighted MTBF much. - - [XXXX what happens when we have less than 4 days of MTBF info.] - - "Exit" -- A router is called an 'Exit' iff it allows exits to at - least one /8 address space on each of ports 80 and 443. (Up until - Tor version 0.3.2, the flag was assigned if relays exit to at least - two of the ports 80, 443, and 6667.) - - "Fast" -- A router is 'Fast' if it is active, and its bandwidth is either in - the top 7/8ths for known active routers or at least 100KB/s. - - "Guard" -- A router is a possible Guard if all of the following apply: - - - It is Fast, - - It is Stable, - - Its Weighted Fractional Uptime is at least the median for "familiar" - active routers, - - It is "familiar", - - Its bandwidth is at least AuthDirGuardBWGuarantee (if set, 2 MB by - default), OR its bandwidth is among the 25% fastest relays, - - It qualifies for the V2Dir flag as described below (this - constraint was added in 0.3.3.x, because in 0.3.0.x clients - started avoiding guards that didn't also have the V2Dir flag). - - To calculate weighted fractional uptime, compute the fraction - of time that the router is up in any given day, weighting so that - downtime and uptime in the past counts less. - - A node is 'familiar' if 1/8 of all active nodes have appeared more - recently than it, OR it has been around for a few weeks. - - "Authority" -- A router is called an 'Authority' if the authority - generating the network-status document believes it is an authority. - - "V2Dir" -- A router supports the v2 directory protocol or higher if it has - an open directory port OR a tunnelled-dir-server line in its router - descriptor, and it is running a version of the directory - protocol that supports the functionality clients need. (Currently, every - supported version of Tor supports the functionality that clients need, - but some relays might set "DirCache 0" or set really low rate limiting, - making them unqualified to be a directory mirror, i.e. they will omit - the tunnelled-dir-server line from their descriptor.) - - "HSDir" -- A router is a v2 hidden service directory if it stores and - serves v2 hidden service descriptors, has the Stable and Fast flag, and the - authority believes that it's been up for at least 96 hours (or the current - value of MinUptimeHidServDirectoryV2). - - "MiddleOnly" -- An authority should vote for this flag if it believes - that a relay is unsuitable for use except as a middle relay. When - voting for this flag, the authority should also vote against "Exit", - "Guard", "HsDir", and "V2Dir". When voting for this flag, if the - authority votes on the "BadExit" flag, the authority should vote in - favor of "BadExit". (This flag was added in 0.4.7.2-alpha.) - - "NoEdConsensus" -- authorities should not vote on this flag; it is - produced as part of the consensus for consensus method 22 or later. - - "StaleDesc" -- authorities should vote to assign this flag if the - published time on the descriptor is over 18 hours in the past. (This flag - was added in 0.4.0.1-alpha.) - - "Sybil" -- authorities SHOULD NOT accept more than 2 relays on a single IP. - If this happens, the authority *should* vote for the excess relays, but - should omit the Running or Valid flags and instead should assign the "Sybil" - flag. When there are more than 2 (or AuthDirMaxServersPerAddr) relays to - choose from, authorities should first prefer authorities to non-authorities, - then prefer Running to non-Running, and then prefer high-bandwidth to - low-bandwidth relays. In this comparison, measured bandwidth is used unless - it is not present for a router, in which case advertised bandwidth is used. - - Thus, the network-status vote includes all non-blacklisted, - non-expired, non-superseded descriptors. - - The bandwidth in a "w" line should be taken as the best estimate - of the router's actual capacity that the authority has. For now, - this should be the lesser of the observed bandwidth and bandwidth - rate limit from the server descriptor. It is given in kilobytes - per second, and capped at some arbitrary value (currently 10 MB/s). - - The Measured= keyword on a "w" line vote is currently computed - by multiplying the previous published consensus bandwidth by the - ratio of the measured average node stream capacity to the network - average. If 3 or more authorities provide a Measured= keyword for - a router, the authorities produce a consensus containing a "w" - Bandwidth= keyword equal to the median of the Measured= votes. - - As a special case, if the "w" line in a vote is about a relay with the - Authority flag, it should not include a Measured= keyword. The goal is - to leave such relays marked as Unmeasured, so they can reserve their - attention for authority-specific activities. "w" lines for votes about - authorities may include the bandwidth authority's measurement using - a different keyword, e.g. MeasuredButAuthority=, so it can still be - reported and recorded for posterity. - - The ports listed in a "p" line should be taken as those ports for - which the router's exit policy permits 'most' addresses, ignoring any - accept not for all addresses, ignoring all rejects for private - netblocks. "Most" addresses are permitted if no more than 2^25 - IPv4 addresses (two /8 networks) were blocked. The list is encoded - as described in section 3.8.2. - -3.4.3. Serving bandwidth list files - - If an authority has used a bandwidth list file to generate a vote - document it SHOULD make it available at - - http:///tor/status-vote/next/bandwidth.z - - at the start of each voting period. - - It MUST NOT attempt to send its bandwidth list file in a HTTP POST to - other authorities and it SHOULD NOT make bandwidth list files from other - authorities available. - - If an authority makes this file available, it MUST be the bandwidth file - used to create the vote document available at - - http:///tor/status-vote/next/authority.z - - To avoid inconsistent reads, authorities SHOULD only read the bandwidth - file once per voting period. Further processing and serving SHOULD use a - cached copy. - - The bandwidth list format is described in bandwidth-file-spec.txt. - - The standard URLs for bandwidth list files first-appeared in - Tor 0.4.0.4-alpha. - -3.5. Downloading missing certificates from other directory authorities - - XXX when to download certificates. - -3.6. Downloading server descriptors from other directory authorities - - Periodically (currently, every 10 seconds), directory authorities check - whether there are any specific descriptors that they do not have and that - they are not currently trying to download. - Authorities identify them by hash in vote (if publication date is more - recent than the descriptor we currently have). - - [XXXX need a way to fetch descriptors ahead of the vote? v2 status docs can - do that for now.] - - If so, the directory authority launches requests to the authorities for these - descriptors, such that each authority is only asked for descriptors listed - in its most recent vote. If more - than one authority lists the descriptor, we choose which to ask at random. - - If one of these downloads fails, we do not try to download that descriptor - from the authority that failed to serve it again unless we receive a newer - network-status (consensus or vote) from that authority that lists the same - descriptor. - - Directory authorities must potentially cache multiple descriptors for each - router. Authorities must not discard any descriptor listed by any recent - consensus. If there is enough space to store additional descriptors, - authorities SHOULD try to hold those which clients are likely to download the - most. (Currently, this is judged based on the interval for which each - descriptor seemed newest.) -[XXXX define recent] - - Authorities SHOULD NOT download descriptors for routers that they would - immediately reject for reasons listed in section 3.2. - -3.7. Downloading extra-info documents from other directory authorities - - Periodically, an authority checks whether it is missing any extra-info - documents: in other words, if it has any server descriptors with an - extra-info-digest field that does not match any of the extra-info - documents currently held. If so, it downloads whatever extra-info - documents are missing. We follow the same splitting and back-off rules - as in section 3.6. - -3.8. Computing a consensus from a set of votes - - Given a set of votes, authorities compute the contents of the consensus. - - The consensus status, along with as many signatures as the server - currently knows (see section 3.10 below), should be available at - - http:///tor/status-vote/next/consensus.z - - The contents of the consensus document are as follows: - - The "valid-after", "valid-until", and "fresh-until" times are taken as - the median of the respective values from all the votes. - - The times in the "voting-delay" line are taken as the median of the - VoteSeconds and DistSeconds times in the votes. - - Known-flags is the union of all flags known by any voter. - - Entries are given on the "params" line for every keyword on which a - majority of authorities (total authorities, not just those - participating in this vote) voted on, or if at least three - authorities voted for that parameter. The values given are the - low-median of all votes on that keyword. - - (In consensus methods 7 to 11 inclusive, entries were given on - the "params" line for every keyword on which *any* authority voted, - the value given being the low-median of all votes on that keyword.) - - "client-versions" and "server-versions" are sorted in ascending - order; A version is recommended in the consensus if it is recommended - by more than half of the voting authorities that included a - client-versions or server-versions lines in their votes. - - With consensus methods 19 through 33, a package line is generated for a - given PACKAGENAME/VERSION pair if at least three authorities list such a - package in their votes. (Call these lines the "input" lines for - PACKAGENAME.) The consensus will contain every "package" line that is - listed verbatim by more than half of the authorities listing a line for - the PACKAGENAME/VERSION pair, and no others. - - The authority item groups (dir-source, contact, fingerprint, - vote-digest) are taken from the votes of the voting - authorities. These groups are sorted by the digests of the - authorities identity keys, in ascending order. If the consensus - method is 3 or later, a dir-source line must be included for - every vote with legacy-key entry, using the legacy-key's - fingerprint, the voter's ordinary nickname with the string - "-legacy" appended, and all other fields as from the original - vote's dir-source line. - - A router status entry: - * is included in the result if some router status entry with the same - identity is included by more than half of the authorities (total - authorities, not just those whose votes we have). - (Consensus method earlier than 21) - - * is included according to the rules in section 3.8.0.1 and - 3.8.0.2 below. (Consensus method 22 or later) - - * For any given RSA identity digest, we include at most - one router status entry. - - * For any given Ed25519 identity, we include at most one router - status entry. - - * A router entry has a flag set if that is included by more than half - of the authorities who care about that flag. - - * Two router entries are "the same" if they have the same - tuple. - We choose the tuple for a given router as whichever tuple appears - for that router in the most votes. We break ties first in favor of - the more recently published, then in favor of smaller server - descriptor digest. - - [ - * The Named flag appears if it is included for this routerstatus by - _any_ authority, and if all authorities that list it list the same - nickname. However, if consensus-method 2 or later is in use, and - any authority calls this identity/nickname pair Unnamed, then - this routerstatus does not get the Named flag. - - * If consensus-method 2 or later is in use, the Unnamed flag is - set for a routerstatus if any authorities have voted for a different - identities to be Named with that nickname, or if any authority - lists that nickname/ID pair as Unnamed. - - (With consensus-method 1, Unnamed is set like any other flag.) - - [But note that authorities no longer vote for the Named flag, - and the above two bulletpoints are now irrelevant.] - ] - - * The version is given as whichever version is listed by the most - voters, with ties decided in favor of more recent versions. - - * If consensus-method 4 or later is in use, then routers that - do not have the Running flag are not listed at all. - - * If consensus-method 5 or later is in use, then the "w" line - is generated using a low-median of the bandwidth values from - the votes that included "w" lines for this router. - - * If consensus-method 5 or later is in use, then the "p" line - is taken from the votes that have the same policy summary - for the descriptor we are listing. (They should all be the - same. If they are not, we pick the most commonly listed - one, breaking ties in favor of the lexicographically larger - vote.) The port list is encoded as specified in section 3.8.2. - - * If consensus-method 6 or later is in use and if 3 or more - authorities provide a Measured= keyword in their votes for - a router, the authorities produce a consensus containing a - Bandwidth= keyword equal to the median of the Measured= votes. - - * If consensus-method 7 or later is in use, the params line is - included in the output. - - * If the consensus method is under 11, bad exits are considered as - possible exits when computing bandwidth weights. Otherwise, if - method 11 or later is in use, any router that is determined to get - the BadExit flag doesn't count when we're calculating weights. - - * If consensus method 12 or later is used, only consensus - parameters that more than half of the total number of - authorities voted for are included in the consensus. - - [ As of 0.2.6.1-alpha, authorities no longer advertise or negotiate - any consensus methods lower than 13. ] - - * If consensus method 13 or later is used, microdesc consensuses - omit any router for which no microdesc was agreed upon. - - * If consensus method 14 or later is used, the ns consensus and - microdescriptors may include an "a" line for each router, listing - an IPv6 OR port. - - * If consensus method 15 or later is used, microdescriptors - include "p6" lines including IPv6 exit policies. - - * If consensus method 16 or later is used, ntor-onion-key - are included in microdescriptors - - * If consensus method 17 or later is used, authorities impose a - maximum on the Bandwidth= values that they'll put on a 'w' - line for any router that doesn't have at least 3 measured - bandwidth values in votes. They also add an "Unmeasured=1" - flag to such 'w' lines. - - * If consensus method 18 or later is used, authorities include - "id" lines in microdescriptors. This method adds RSA ids. - - * If consensus method 19 or later is used, authorities may include - "package" lines in consensuses. - - * If consensus method 20 or later is used, authorities may include - GuardFraction information in microdescriptors. - - * If consensus method 21 or later is used, authorities may include - an "id" line for ed25519 identities in microdescriptors. - - [ As of 0.2.8.2-alpha, authorities no longer advertise or negotiate - consensus method 21, because it contains bugs. ] - - * If consensus method 22 or later is used, and the votes do not - produce a majority consensus about a relay's Ed25519 key (see - 3.8.0.1 below), the consensus must include a NoEdConsensus flag on - the "s" line for every relay whose listed Ed key does not reflect - consensus. - - * If consensus method 23 or later is used, authorities include - shared randomness protocol data on their votes and consensus. - - * If consensus-method 24 or later is in use, then routers that - do not have the Valid flag are not listed at all. - - [ As of 0.3.4.1-alpha, authorities no longer advertise or negotiate - any consensus methods lower than 25. ] - - * If consensus-method 25 or later is in use, then we vote - on recommended-protocols and required-protocols lines in the - consensus. We also include protocols lines in routerstatus - entries. - - * If consensus-method 26 or later is in use, then we initialize - bandwidth weights to 1 in our calculations, to avoid - division-by-zero errors on unusual networks. - - * If consensus method 27 or later is used, the microdesc consensus - may include an "a" line for each router, listing an IPv6 OR port. - - [ As of 0.4.3.1-alpha, authorities no longer advertise or negotiate - any consensus methods lower than 28. ] - - * If consensus method 28 or later is used, microdescriptors no longer - include "a" lines. - - * If consensus method 29 or later is used, microdescriptor "family" - lines are canonicalized to improve compression. - - * If consensus method 30 or later is used, the base64 encoded - ntor-onion-key does not include the trailing = sign. - - * If consensus method 31 or later is used, authorities parse the - "bwweightscale" and "maxunmeasuredbw" parameters correctly when - computing votes. - - * If consensus method 32 or later is used, authorities handle the - "MiddleOnly" flag specially when computing a consensus. When the - voters agree to include "MiddleOnly" in a routerstatus, they - automatically remove "Exit", "Guard", "V2Dir", and "HSDir". If - the BadExit flag is included in the consensus, they automatically - add it to the routerstatus. - - * If consensus method 33 or later is used, and the consensus - flavor is "microdesc", then the "Publication" field in the "r" - line is set to "2038-01-01 00:00:00". - - * If consensus method 34 or later is used, the consensus - does not include any "package" lines. - - The signatures at the end of a consensus document are sorted in - ascending order by identity digest. - - All ties in computing medians are broken in favor of the smaller or - earlier item. - -3.8.0.1. Deciding which Ids to include. - - This sorting algorithm is used for consensus-method 22 and later. - - First, consider each listing by tuple of identities, where 'Ed' - may be "None" if the voter included "id ed25519 none" to indicate that - the authority knows what ed25519 identities are, and thinks that the RSA - key doesn't have one. - - For each such tuple that is listed by more than half of the - total authorities (not just total votes), include it. (It is not - possible for any other to have as many votes.) If more - than half of the authorities list a single pair of this type, we - consider that Ed key to be "consensus"; see description of the - NoEdConsensus flag. - - Log any other id-RSA values corresponding to an id-Ed we included, and any - other id-Ed values corresponding to an id-RSA we included. - - For each that is not yet included, if it is listed by more than - half of the total authorities, and we do not already have it listed with - some , include it, but do not consider its Ed identity canonical. - -3.8.0.2. Deciding which descriptors to include - - Deciding which descriptors to include. - - A tuple belongs to an identity if it is a new tuple that - matches both ID parts, or if it is an old tuple (one with no Ed opinion) - that matches the RSA part. A tuple belongs to an identity if its - RSA identity matches. - - A tuple matches another tuple if all the fields that are present in both - tuples are the same. - - For every included identity, consider the tuples belonging to that - identity. Group them into sets of matching tuples. Include the tuple - that matches the largest set, breaking ties in favor of the most recently - published, and then in favor of the smaller server descriptor digest. - -3.8.1. Forward compatibility - - Future versions of Tor will need to include new information in the - consensus documents, but it is important that all authorities (or at least - half) generate and sign the same signed consensus. - - To achieve this, authorities list in their votes their supported methods - for generating consensuses from votes. Later methods will be assigned - higher numbers. Currently specified methods: - - "1" -- The first implemented version. - "2" -- Added support for the Unnamed flag. - "3" -- Added legacy ID key support to aid in authority ID key rollovers - "4" -- No longer list routers that are not running in the consensus - "5" -- adds support for "w" and "p" lines. - "6" -- Prefers measured bandwidth values rather than advertised - "7" -- Provides keyword=integer pairs of consensus parameters - "8" -- Provides microdescriptor summaries - "9" -- Provides weights for selecting flagged routers in paths - "10" -- Fixes edge case bugs in router flag selection weights - "11" -- Don't consider BadExits when calculating bandwidth weights - "12" -- Params are only included if enough auths voted for them - "13" -- Omit router entries with missing microdescriptors. - "14" -- Adds support for "a" lines in ns consensuses and microdescriptors. - "15" -- Adds support for "p6" lines. - "16" -- Adds ntor keys to microdescriptors - "17" -- Adds "Unmeasured=1" flags to "w" lines - "18" -- Adds 'id' to microdescriptors. - "19" -- Adds "package" lines to consensuses - "20" -- Adds GuardFraction information to microdescriptors. - "21" -- Adds Ed25519 keys to microdescriptors. - "22" -- Instantiates Ed25519 voting algorithm correctly. - "23" -- Adds shared randomness protocol data. - "24" -- No longer lists routers that are not Valid in the consensus. - "25" -- Vote on recommended-protocols and required-protocols. - "26" -- Initialize bandwidth weights to 1 to avoid division-by-zero. - "27" -- Adds support for "a" lines in microdescriptor consensuses. - "28" -- Removes "a" lines from microdescriptors. - "29" -- Canonicalizes families in microdescriptors. - "30" -- Removes padding from ntor-onion-key. - "31" -- Uses correct parsing for bwweightscale and maxunmeasuredbw - when computing weights - "32" -- Adds special handling for MiddleOnly flag. - "33" -- Sets "publication" field in microdesc consensus "r" lines - to a meaningless value. - "34" -- Removes "package" lines from consensus. - - Before generating a consensus, an authority must decide which consensus - method to use. To do this, it looks for the highest version number - supported by more than 2/3 of the authorities voting. If it supports this - method, then it uses it. Otherwise, it falls back to the newest consensus - method that it supports (which will probably not result in a sufficiently - signed consensus). - - All authorities MUST support method 25; authorities SHOULD support - more recent methods as well. Authorities SHOULD NOT support or - advertise support for any method before 25. Clients MAY assume that - they will never see a current valid signed consensus for any method - before method 25. - - (The consensuses generated by new methods must be parsable by - implementations that only understand the old methods, and must not cause - those implementations to compromise their anonymity. This is a means for - making changes in the contents of consensus; not for making - backward-incompatible changes in their format.) - - The following methods have incorrect implementations; authorities SHOULD - NOT advertise support for them: - - "21" -- Did not correctly enable support for ed25519 key collation. - -3.8.2. Encoding port lists - - Whether the summary shows the list of accepted ports or the list of - rejected ports depends on which list is shorter (has a shorter string - representation). In case of ties we choose the list of accepted - ports. As an exception to this rule an allow-all policy is - represented as "accept 1-65535" instead of "reject " and a reject-all - policy is similarly given as "reject 1-65535". - - Summary items are compressed, that is instead of "80-88,89-100" there - only is a single item of "80-100", similarly instead of "20,21" a - summary will say "20-21". - - Port lists are sorted in ascending order. - - The maximum allowed length of a policy summary (including the "accept " - or "reject ") is 1000 characters. If a summary exceeds that length we - use an accept-style summary and list as much of the port list as is - possible within these 1000 bytes. [XXXX be more specific.] - -3.8.3. Computing Bandwidth Weights - - Let weight_scale = 10000, or the value of the "bwweightscale" parameter. - (Before consensus method 31 there was a bug in parsing bwweightscale, so - that if there were any consensus parameters after it alphabetically, it - would always be treated as 10000. A similar bug existed for - "maxunmeasuredbw".) - - Starting with consensus method 26, G, M, E, and D are initialized to 1 and - T to 4. Prior consensus methods initialize them all to 0. With this change, - test tor networks that are small or new are much more likely to produce - bandwidth-weights in their consensus. The extra bandwidth has a negligible - impact on the bandwidth weights in the public tor network. - - Let G be the total bandwidth for Guard-flagged nodes. - Let M be the total bandwidth for non-flagged nodes. - Let E be the total bandwidth for Exit-flagged nodes. - Let D be the total bandwidth for Guard+Exit-flagged nodes. - Let T = G+M+E+D - - Let Wgd be the weight for choosing a Guard+Exit for the guard position. - Let Wmd be the weight for choosing a Guard+Exit for the middle position. - Let Wed be the weight for choosing a Guard+Exit for the exit position. - - Let Wme be the weight for choosing an Exit for the middle position. - Let Wmg be the weight for choosing a Guard for the middle position. - - Let Wgg be the weight for choosing a Guard for the guard position. - Let Wee be the weight for choosing an Exit for the exit position. - - Balanced network conditions then arise from solutions to the following - system of equations: - - Wgg*G + Wgd*D == M + Wmd*D + Wme*E + Wmg*G (guard bw = middle bw) - Wgg*G + Wgd*D == Wee*E + Wed*D (guard bw = exit bw) - Wed*D + Wmd*D + Wgd*D == D (aka: Wed+Wmd+Wdg = weight_scale) - Wmg*G + Wgg*G == G (aka: Wgg = weight_scale-Wmg) - Wme*E + Wee*E == E (aka: Wee = weight_scale-Wme) - - We are short 2 constraints with the above set. The remaining constraints - come from examining different cases of network load. The following - constraints are used in consensus method 10 and above. There are another - incorrect and obsolete set of constraints used for these same cases in - consensus method 9. For those, see dir-spec.txt in Tor 0.2.2.10-alpha - to 0.2.2.16-alpha. - - Case 1: E >= T/3 && G >= T/3 (Neither Exit nor Guard Scarce) - - In this case, the additional two constraints are: Wmg == Wmd, - Wed == 1/3. - - This leads to the solution: - Wgd = weight_scale/3 - Wed = weight_scale/3 - Wmd = weight_scale/3 - Wee = (weight_scale*(E+G+M))/(3*E) - Wme = weight_scale - Wee - Wmg = (weight_scale*(2*G-E-M))/(3*G) - Wgg = weight_scale - Wmg - - Case 2: E < T/3 && G < T/3 (Both are scarce) - - Let R denote the more scarce class (Rare) between Guard vs Exit. - Let S denote the less scarce class. - - Subcase a: R+D < S - - In this subcase, we simply devote all of D bandwidth to the - scarce class. - - Wgg = Wee = weight_scale - Wmg = Wme = Wmd = 0; - if E < G: - Wed = weight_scale - Wgd = 0 - else: - Wed = 0 - Wgd = weight_scale - - Subcase b: R+D >= S - - In this case, if M <= T/3, we have enough bandwidth to try to achieve - a balancing condition. - - Add constraints Wgg = weight_scale, Wmd == Wgd to maximize bandwidth in - the guard position while still allowing exits to be used as middle nodes: - - Wee = (weight_scale*(E - G + M))/E - Wed = (weight_scale*(D - 2*E + 4*G - 2*M))/(3*D) - Wme = (weight_scale*(G-M))/E - Wmg = 0 - Wgg = weight_scale - Wmd = (weight_scale - Wed)/2 - Wgd = (weight_scale - Wed)/2 - - If this system ends up with any values out of range (ie negative, or - above weight_scale), use the constraints Wgg == weight_scale and Wee == - weight_scale, since both those positions are scarce: - - Wgg = weight_scale - Wee = weight_scale - Wed = (weight_scale*(D - 2*E + G + M))/(3*D) - Wmd = (weight_Scale*(D - 2*M + G + E))/(3*D) - Wme = 0 - Wmg = 0 - Wgd = weight_scale - Wed - Wmd - - If M > T/3, then the Wmd weight above will become negative. Set it to 0 - in this case: - Wmd = 0 - Wgd = weight_scale - Wed - - Case 3: One of E < T/3 or G < T/3 - - Let S be the scarce class (of E or G). - - Subcase a: (S+D) < T/3: - if G=S: - Wgg = Wgd = weight_scale; - Wmd = Wed = Wmg = 0; - // Minor subcase, if E is more scarce than M, - // keep its bandwidth in place. - if (E < M) Wme = 0; - else Wme = (weight_scale*(E-M))/(2*E); - Wee = weight_scale-Wme; - if E=S: - Wee = Wed = weight_scale; - Wmd = Wgd = Wme = 0; - // Minor subcase, if G is more scarce than M, - // keep its bandwidth in place. - if (G < M) Wmg = 0; - else Wmg = (weight_scale*(G-M))/(2*G); - Wgg = weight_scale-Wmg; - - Subcase b: (S+D) >= T/3 - if G=S: - Add constraints Wgg = weight_scale, Wmd == Wed to maximize bandwidth - in the guard position, while still allowing exits to be - used as middle nodes: - Wgg = weight_scale - Wgd = (weight_scale*(D - 2*G + E + M))/(3*D) - Wmg = 0 - Wee = (weight_scale*(E+M))/(2*E) - Wme = weight_scale - Wee - Wmd = (weight_scale - Wgd)/2 - Wed = (weight_scale - Wgd)/2 - if E=S: - Add constraints Wee == weight_scale, Wmd == Wgd to maximize bandwidth - in the exit position: - Wee = weight_scale; - Wed = (weight_scale*(D - 2*E + G + M))/(3*D); - Wme = 0; - Wgg = (weight_scale*(G+M))/(2*G); - Wmg = weight_scale - Wgg; - Wmd = (weight_scale - Wed)/2; - Wgd = (weight_scale - Wed)/2; - - To ensure consensus, all calculations are performed using integer math - with a fixed precision determined by the bwweightscale consensus - parameter (defaults at 10000, Min: 1, Max: INT32_MAX). (See note above - about parsing bug in bwweightscale before consensus method 31.) - - For future balancing improvements, Tor clients support 11 additional weights - for directory requests and middle weighting. These weights are currently - set at weight_scale, with the exception of the following groups of - assignments: - - Directory requests use middle weights: - - Wbd=Wmd, Wbg=Wmg, Wbe=Wme, Wbm=Wmm - - Handle bridges and strange exit policies: - - Wgm=Wgg, Wem=Wee, Weg=Wed - -3.9. Computing consensus flavors - - Consensus flavors are variants of the consensus that clients can choose - to download and use instead of the unflavored consensus. The purpose - of a consensus flavor is to remove or replace information in the - unflavored consensus without forcing clients to download information - they would not use anyway. - - Directory authorities can produce and serve an arbitrary number of - flavors of the same consensus. A downside of creating too many new - flavors is that clients will be distinguishable based on which flavor - they download. A new flavor should not be created when adding a field - instead wouldn't be too onerous. - - Examples for consensus flavors include: - - - Publishing hashes of microdescriptors instead of hashes of - full descriptors (see section 3.9.2). - - Including different digests of descriptors, instead of the - perhaps-soon-to-be-totally-broken SHA1. - - Consensus flavors are derived from the unflavored consensus once the - voting process is complete. This is to avoid consensus synchronization - problems. - - Every consensus flavor has a name consisting of a sequence of one - or more alphanumeric characters and dashes. For compatibility, - the original (unflavored) consensus type is called "ns". - - The supported consensus flavors are defined as part of the - authorities' consensus method. - - All consensus flavors have in common that their first line is - "network-status-version" where version is 3 or higher, and the flavor - is a string consisting of alphanumeric characters and dashes: - - "network-status-version" SP version [SP flavor] NL - -3.9.1. ns consensus - - The ns consensus flavor is equivalent to the unflavored consensus. - When the flavor is omitted from the "network-status-version" line, - it should be assumed to be "ns". Some implementations may explicitly - state that the flavor is "ns" when generating consensuses, but should - accept consensuses where the flavor is omitted. - -3.9.2. Microdescriptor consensus - - The microdescriptor consensus is a consensus flavor that contains - microdescriptor hashes instead of descriptor hashes and that omits - exit-policy summaries which are contained in microdescriptors. The - microdescriptor consensus was designed to contain elements that are - small and frequently changing. Clients use the information in the - microdescriptor consensus to decide which servers to fetch information - about and which servers to fetch information from. - - The microdescriptor consensus is based on the unflavored consensus with - the exceptions as follows: - - "network-status-version" SP version SP "microdesc" NL - - [At start, exactly once.] - - The flavor name of a microdescriptor consensus is "microdesc". - - Changes to router status entries are as follows: - - "r" SP nickname SP identity SP publication SP IP SP ORPort - SP DirPort NL - - [At start, exactly once.] - - Similar to "r" lines in section 3.4.1, but without the digest element. - - "a" SP address ":" port NL - - [Any number] - - Identical to the "r" lines in section 3.4.1. - - (Only included when the vote is generated with consensus-method 14 - or later, and the consensus is generated with consensus-method 27 or - later.) - - "p" ... NL - - [At most once] - - Not currently generated. - - Exit policy summaries are contained in microdescriptors and - therefore omitted in the microdescriptor consensus. - - "m" SP digest NL - - [Exactly once.*] - - "digest" is the base64 of the SHA256 hash of the router's - microdescriptor with trailing =s omitted. For a given router - descriptor digest and consensus method there should only be a - single microdescriptor digest in the "m" lines of all votes. - If different votes have different microdescriptor digests for - the same descriptor digest and consensus method, at least one - of the authorities is broken. If this happens, the microdesc - consensus should contain whichever microdescriptor digest is - most common. If there is no winner, we break ties in the favor - of the lexically earliest. - - [*Before consensus method 13, this field was sometimes erroneously - omitted.] - - Additionally, a microdescriptor consensus SHOULD use the sha256 digest - algorithm for its signatures. - -3.10. Exchanging detached signatures - - Once an authority has computed and signed a consensus network status, it - should send its detached signature to each other authority in an HTTP POST - request to the URL: - - http:///tor/post/consensus-signature - - [XXX Note why we support push-and-then-pull.] - - All of the detached signatures it knows for consensus status should be - available at: - - http:///tor/status-vote/next/consensus-signatures.z - - Assuming full connectivity, every authority should compute and sign the - same consensus including any flavors in each period. Therefore, it - isn't necessary to download the consensus or any flavors of it computed - by each authority; instead, the authorities only push/fetch each - others' signatures. A "detached signature" document contains items as - follows: - - "consensus-digest" SP Digest NL - - [At start, at most once.] - - The digest of the consensus being signed. - - "valid-after" SP YYYY-MM-DD SP HH:MM:SS NL - "fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL - "valid-until" SP YYYY-MM-DD SP HH:MM:SS NL - - [As in the consensus] - - "additional-digest" SP flavor SP algname SP digest NL - - [Any number.] - - For each supported consensus flavor, every directory authority - adds one or more "additional-digest" lines. "flavor" is the name - of the consensus flavor, "algname" is the name of the hash - algorithm that is used to generate the digest, and "digest" is the - hex-encoded digest. - - The hash algorithm for the microdescriptor consensus flavor is - defined as SHA256 with algname "sha256". - - "additional-signature" SP flavor SP algname SP identity SP - signing-key-digest NL signature. - - [Any number.] - - For each supported consensus flavor and defined digest algorithm, - every directory authority adds an "additional-signature" line. - "flavor" is the name of the consensus flavor. "algname" is the - name of the algorithm that was used to hash the identity and - signing keys, and to compute the signature. "identity" is the - hex-encoded digest of the authority identity key of the signing - authority, and "signing-key-digest" is the hex-encoded digest of - the current authority signing key of the signing authority. - - The "sha256" signature format is defined as the RSA signature of - the OAEP+-padded SHA256 digest of the item to be signed. When - checking signatures, the signature MUST be treated as valid if the - signature material begins with SHA256(document), so that other - data can get added later. - [To be honest, I didn't fully understand the previous paragraph - and only copied it from the proposals. Review carefully. -KL] - - "directory-signature" - - [As in the consensus; the signature object is the same as in the - consensus document.] - -3.11. Publishing the signed consensus - - The voting period ends at the valid-after time. If the consensus has - been signed by a majority of authorities, these documents are made - available at - - http:///tor/status-vote/current/consensus.z - - and - - http:///tor/status-vote/current/consensus-signatures.z - - [XXX current/consensus-signatures is not currently implemented, as it - is not used in the voting protocol.] - - [XXX possible future features include support for downloading old - consensuses.] - - The other vote documents are analogously made available under - - http:///tor/status-vote/current/authority.z - http:///tor/status-vote/current/.z - http:///tor/status-vote/current/d/.z - http:///tor/status-vote/current/bandwidth.z - - once the voting period ends, regardless of the number of signatures. - - The authorities serve another consensus of each flavor "F" from the - locations - - /tor/status-vote/(current|next)/consensus-F.z. and - /tor/status-vote/(current|next)/consensus-F/+....z. - - The standard URLs for bandwidth list files first-appeared in Tor 0.3.5. - -4. Directory cache operation - - All directory caches implement this section, except as noted. - -4.1. Downloading consensus status documents from directory authorities - - All directory caches try to keep a recent - network-status consensus document to serve to clients. A cache ALWAYS - downloads a network-status consensus if any of the following are true: - - - The cache has no consensus document. - - The cache's consensus document is no longer valid. - - Otherwise, the cache downloads a new consensus document at a randomly - chosen time in the first half-interval after its current consensus - stops being fresh. (This time is chosen at random to avoid swarming - the authorities at the start of each period. The interval size is - inferred from the difference between the valid-after time and the - fresh-until time on the consensus.) - - [For example, if a cache has a consensus that became valid at 1:00, - and is fresh until 2:00, that cache will fetch a new consensus at - a random time between 2:00 and 2:30.] - - Directory caches also fetch consensus flavors from the authorities. - Caches check the correctness of consensus flavors, but do not check - anything about an unrecognized consensus document beyond its digest and - length. Caches serve all consensus flavors from the same locations as - the directory authorities. - -4.2. Downloading server descriptors from directory authorities - - Periodically (currently, every 10 seconds), directory caches check - whether there are any specific descriptors that they do not have and that - they are not currently trying to download. Caches identify these - descriptors by hash in the recent network-status consensus documents. - - If so, the directory cache launches requests to the authorities for these - descriptors. - - If one of these downloads fails, we do not try to download that descriptor - from the authority that failed to serve it again unless we receive a newer - network-status consensus that lists the same descriptor. - - Directory caches must potentially cache multiple descriptors for each - router. Caches must not discard any descriptor listed by any recent - consensus. If there is enough space to store additional descriptors, - caches SHOULD try to hold those which clients are likely to download the - most. (Currently, this is judged based on the interval for which each - descriptor seemed newest.) - - [XXXX define recent] - -4.3. Downloading microdescriptors from directory authorities - - Directory mirrors should fetch, cache, and serve each microdescriptor - from the authorities. - - The microdescriptors with base64 hashes ,, are available - at: - - http:///tor/micro/d/--[.z] - - are base64 encoded with trailing =s omitted for size and for - consistency with the microdescriptor consensus format. -s are used - instead of +s to separate items, since the + character is used in - base64 encoding. - - Directory mirrors should check to make sure that the microdescriptors - they're about to serve match the right hashes (either the hashes from - the fetch URL or the hashes from the consensus, respectively). - - (NOTE: Due to squid proxy url limitations at most 92 microdescriptor hashes - can be retrieved in a single request.) - -4.4. Downloading extra-info documents from directory authorities - - Any cache that chooses to cache extra-info documents should implement this - section. - - Periodically, the Tor instance checks whether it is missing any extra-info - documents: in other words, if it has any server descriptors with an - extra-info-digest field that does not match any of the extra-info - documents currently held. If so, it downloads whatever extra-info - documents are missing. Caches download from authorities. We follow the - same splitting and back-off rules as in section 4.2. - -4.5. Consensus diffs - - Instead of downloading an entire consensus, clients may download - a "diff" document containing an ed-style diff from a previous - consensus document. Caches (and authorities) make these diffs as - they learn about new consensuses. To do so, they must store a - record of older consensuses. - - (Support for consensus diffs was added in 0.3.1.1-alpha, and is - advertised with the DirCache protocol version "2" or later.) - -4.5.1. Consensus diff format - - Consensus diffs are formatted as follows: - - The first line is "network-status-diff-version 1" NL - - The second line is - - "hash" SP FromDigest SP ToDigest NL - - where FromDigest is the hex-encoded SHA3-256 digest of the _signed - part_ of the consensus that the diff should be applied to, and - ToDigest is the hex-encoded SHA3-256 digest of the _entire_ - consensus resulting from applying the diff. (See 3.4.1 for - information on that part of a consensus is signed.) - - The third and subsequent lines encode the diff from FromDigest to - ToDigest in a limited subset of the ed diff format, as specified - in appendix E. - -4.5.2. Serving and requesting diffs. - - When downloading the current consensus, a client may include an - HTTP header of the form - - X-Or-Diff-From-Consensus: HASH1, HASH2, ... - - where the HASH values are hex-encoded SHA3-256 digests of the - _signed part_ of one or more consensuses that the client knows - about. - - If a cache knows a consensus diff from one of those consensuses - to the most recent consensus of the requested flavor, it may - send that diff instead of the specified consensus. - - Caches also serve diffs from the URIs: - - /tor/status-vote/current/consensus/diff//.z - /tor/status-vote/current/consensus-/diff//.z - - where FLAVOR is the consensus flavor, defaulting to "ns", and - FPRLIST is +-separated list of recognized authority identity - fingerprints as in appendix B. - -4.6 Retrying failed downloads - - See section 5.5 below; it applies to caches as well as clients. - -5. Client operation - - Every Tor that is not a directory server (that is, those that do - not have a DirPort set) implements this section. - -5.1. Downloading network-status documents - - Each client maintains a list of directory authorities. Insofar as - possible, clients SHOULD all use the same list. - - [Newer versions of Tor (0.2.8.1-alpha and later): - Each client also maintains a list of default fallback directory mirrors - (fallbacks). Each released version of Tor MAY have a different list, - depending on the mirrors that satisfy the fallback directory criteria at - release time.] - - Clients try to have a live consensus network-status document at all times. - A network-status document is "live" if the time in its valid-after field - has passed, and the time in its valid-until field has not passed. - - When a client has no consensus network-status document, it downloads it - from a randomly chosen fallback directory mirror or authority. Clients - prefer fallbacks to authorities, trying them earlier and more frequently. - In all other cases, the client downloads from caches randomly chosen from - among those believed to be V3 directory servers. (This information comes - from the network-status documents.) - - After receiving any response client MUST discard any network-status - documents that it did not request. - - On failure, the client waits briefly, then tries that network-status - document again from another cache. The client does not build circuits - until it has a live network-status consensus document, and it has - descriptors for a significant proportion of the routers that it believes - are running (this is configurable using torrc options and consensus - parameters). - - [Newer versions of Tor (0.2.6.2-alpha and later): - If the consensus contains Exits (the typical case), Tor will build both - exit and internal circuits. When bootstrap completes, Tor will be ready - to handle an application requesting an exit circuit to services like the - World Wide Web. - - If the consensus does not contain Exits, Tor will only build internal - circuits. In this case, earlier statuses will have included "internal" - as indicated above. When bootstrap completes, Tor will be ready to handle - an application requesting an internal circuit to hidden services at - ".onion" addresses. - - If a future consensus contains Exits, exit circuits may become available.] - - (Note: clients can and should pick caches based on the network-status - information they have: once they have first fetched network-status info - from an authority or fallback, they should not need to go to the authority - directly again, and should only choose the fallback at random, based on its - consensus weight in the current consensus.) - - To avoid swarming the caches whenever a consensus expires, the - clients download new consensuses at a randomly chosen time after the - caches are expected to have a fresh consensus, but before their - consensus will expire. (This time is chosen uniformly at random from - the interval between the time 3/4 into the first interval after the - consensus is no longer fresh, and 7/8 of the time remaining after - that before the consensus is invalid.) - - [For example, if a client has a consensus that became valid at 1:00, - and is fresh until 2:00, and expires at 4:00, that client will fetch - a new consensus at a random time between 2:45 and 3:50, since 3/4 - of the one-hour interval is 45 minutes, and 7/8 of the remaining 75 - minutes is 65 minutes.] - - Clients may choose to download the microdescriptor consensus instead - of the general network status consensus. In that case they should use - the same update strategy as for the normal consensus. They should not - download more than one consensus flavor. - - When a client does not have a live consensus, it will generally use the - most recent consensus it has if that consensus is "reasonably live". A - "reasonably live" consensus is one that expired less than 24 hours ago. - -5.2. Downloading server descriptors or microdescriptors - - Clients try to have the best descriptor for each router. A descriptor is - "best" if: - - * It is listed in the consensus network-status document. - - Periodically (currently every 10 seconds) clients check whether there are - any "downloadable" descriptors. A descriptor is downloadable if: - - - It is the "best" descriptor for some router. - - The descriptor was published at least 10 minutes in the past. - (This prevents clients from trying to fetch descriptors that the - mirrors have probably not yet retrieved and cached.) - - The client does not currently have it. - - The client is not currently trying to download it. - - The client would not discard it immediately upon receiving it. - - The client thinks it is running and valid (see section 5.4.1 below). - - If at least 16 known routers have downloadable descriptors, or if - enough time (currently 10 minutes) has passed since the last time the - client tried to download descriptors, it launches requests for all - downloadable descriptors. - - When downloading multiple server descriptors, the client chooses multiple - mirrors so that: - - - At least 3 different mirrors are used, except when this would result - in more than one request for under 4 descriptors. - - No more than 128 descriptors are requested from a single mirror. - - Otherwise, as few mirrors as possible are used. - After choosing mirrors, the client divides the descriptors among them - randomly. - - After receiving any response the client MUST discard any descriptors that - it did not request. - - When a descriptor download fails, the client notes it, and does not - consider the descriptor downloadable again until a certain amount of time - has passed. (Currently 0 seconds for the first failure, 60 seconds for the - second, 5 minutes for the third, 10 minutes for the fourth, and 1 day - thereafter.) Periodically (currently once an hour) clients reset the - failure count. - - Clients retain the most recent descriptor they have downloaded for each - router so long as it is listed in the consensus. If it is not listed, - they keep it so long as it is not too old (currently, ROUTER_MAX_AGE=48 - hours) and no better router descriptor has been downloaded for the same - relay. Caches retain descriptors until they are at least - OLD_ROUTER_DESC_MAX_AGE=5 days old. - - Clients which chose to download the microdescriptor consensus instead - of the general consensus must download the referenced microdescriptors - instead of server descriptors. Clients fetch and cache - microdescriptors preemptively from dir mirrors when starting up, like - they currently fetch descriptors. After bootstrapping, clients only - need to fetch the microdescriptors that have changed. - - When a client gets a new microdescriptor consensus, it looks to see if - there are any microdescriptors it needs to learn, and launches a request - for them. - - Clients maintain a cache of microdescriptors along with metadata like - when it was last referenced by a consensus, and which identity key - it corresponds to. They keep a microdescriptor until it hasn't been - mentioned in any consensus for a week. Future clients might cache them - for longer or shorter times. - -5.3. Downloading extra-info documents - - Any client that uses extra-info documents should implement this - section. - - Note that generally, clients don't need extra-info documents. - - Periodically, the Tor instance checks whether it is missing any extra-info - documents: in other words, if it has any server descriptors with an - extra-info-digest field that does not match any of the extra-info - documents currently held. If so, it downloads whatever extra-info - documents are missing. Clients try to download from caches. - We follow the same splitting and back-off rules as in section 5.2. - -5.4. Using directory information - - [XXX This subsection really belongs in path-spec.txt, not here. -KL] - - Everyone besides directory authorities uses the approaches in this section - to decide which relays to use and what their keys are likely to be. - (Directory authorities just believe their own opinions, as in section 3.4.2 - above.) - -5.4.1. Choosing routers for circuits. - - Circuits SHOULD NOT be built until the client has enough directory - information: a live consensus network status [XXXX fallback?] and - descriptors for at least 1/4 of the relays believed to be running. - - A relay is "listed" if it is included by the consensus network-status - document. Clients SHOULD NOT use unlisted relays. - - These flags are used as follows: - - - Clients SHOULD NOT use non-'Valid' or non-'Running' routers unless - requested to do so. - - - Clients SHOULD NOT use non-'Fast' routers for any purpose other than - very-low-bandwidth circuits (such as introduction circuits). - - - Clients SHOULD NOT use non-'Stable' routers for circuits that are - likely to need to be open for a very long time (such as those used for - IRC or SSH connections). - - - Clients SHOULD NOT choose non-'Guard' nodes when picking entry guard - nodes. - - See the "path-spec.txt" document for more details. - -5.4.2. Managing naming - - (This section is removed; authorities no longer assign the 'Named' flag.) - -5.4.3. Software versions - - An implementation of Tor SHOULD warn when it has fetched a consensus - network-status, and it is running a software version not listed. - -5.4.4. Warning about a router's status. - - (This section is removed; authorities no longer assign the 'Named' flag.) - -5.5. Retrying failed downloads - - This section applies to caches as well as to clients. - - When a client fails to download a resource (a consensus, a router - descriptor, a microdescriptor, etc) it waits for a certain amount of - time before retrying the download. To determine the amount of time - to wait, clients use a randomized exponential backoff algorithm. - (Specifically, they use a variation of the "decorrelated jitter" - algorithm from - https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ .) - - The specific formula used to compute the 'i+1'th delay is: - - Delay_{i+1} = MIN(cap, random_between(lower_bound, upper_bound))) - where upper_bound = MAX(lower_bound+1, Delay_i * 3) - lower_bound = MAX(1, base_delay). - - The value of 'cap' is set to INT_MAX; the value of 'base_delay' - depends on what is being downloaded, whether the client is fully - bootstrapped, how the client is configured, and where it is - downloading from. Current base_delay values are: - - Consensus objects, as a non-bridge cache: - 0 (TestingServerConsensusDownloadInitialDelay) - - Consensus objects, as a client or bridge that has bootstrapped: - 0 (TestingClientConsensusDownloadInitialDelay) - - Consensus objects, as a client or bridge that is bootstrapping, - when connecting to an authority because no "fallback" caches are - known: - 0 (ClientBootstrapConsensusAuthorityOnlyDownloadInitialDelay) - - Consensus objects, as a client or bridge that is bootstrapping, - when "fallback" caches are known but connecting to an authority - anyway: - 6 (ClientBootstrapConsensusAuthorityDownloadInitialDelay) - - Consensus objects, as a client or bridge that is bootstrapping, - when downloading from a "fallback" cache. - 0 (ClientBootstrapConsensusFallbackDownloadInitialDelay) - - Bridge descriptors, as a bridge-using client when at least one bridge - is usable: - 10800 (TestingBridgeDownloadInitialDelay) - - Bridge descriptors, otherwise: - 0 (TestingBridgeBootstrapDownloadInitialDelay) - - Other objects, as cache or authority: - 0 (TestingServerDownloadInitialDelay) - - Other objects, as client: - 0 (TestingClientDownloadInitialDelay) - - -6. Standards compliance - - All clients and servers MUST support HTTP 1.0. Clients and servers MAY - support later versions of HTTP as well. - -6.1. HTTP headers - - Servers SHOULD set Content-Encoding to the algorithm used to compress the - document(s) being served. Recognized algorithms are: - - - "identity" -- RFC2616 section 3.5 - - "deflate" -- RFC2616 section 3.5 - - "gzip" -- RFC2616 section 3.5 - - "x-zstd" -- The zstandard compression algorithm (www.zstd.net) - - "x-tor-lzma" -- The lzma compression algorithm, with a "preset" - value no higher than 6. - - Clients SHOULD use Accept-Encoding on most directory requests to indicate - which of the above compression algorithms they support. If they omit it - (as Tor clients did before 0.3.1.1-alpha), then the server should serve - only "deflate" or "identity" encoded documents, based on the presence or - absence of the ".z" suffix on the requested URL. - - Note that for anonymous directory requests (that is, requests made over - multi-hop circuits, like those for onion service lookups) implementations - SHOULD NOT advertise any Accept-Encoding values other than deflate. To do - so would be to create a fingerprinting opportunity. - - When receiving multiple documents, clients MUST accept compressed - concatenated documents and concatenated compressed documents as - equivalent. - - Servers MAY set the Content-Length: header. When they do, it should - match the number of compressed bytes that they are sending. - - Servers MAY include an X-Your-Address-Is: header, whose value is the - apparent IP address of the client connecting to them (as a dotted quad). - For directory connections tunneled over a BEGIN_DIR stream, servers SHOULD - report the IP from which the circuit carrying the BEGIN_DIR stream reached - them. - - Servers SHOULD disable caching of multiple network statuses or multiple - server descriptors. Servers MAY enable caching of single descriptors, - single network statuses, the list of all server descriptors, a v1 - directory, or a v1 running routers document. XXX mention times. - -6.2. HTTP status codes - - Tor delivers the following status codes. Some were chosen without much - thought; other code SHOULD NOT rely on specific status codes yet. - - 200 -- the operation completed successfully - -- the user requested statuses or serverdescs, and none of the ones we - requested were found (0.2.0.4-alpha and earlier). - - 304 -- the client specified an if-modified-since time, and none of the - requested resources have changed since that time. - - 400 -- the request is malformed, or - -- the URL is for a malformed variation of one of the URLs we support, - or - -- the client tried to post to a non-authority, or - -- the authority rejected a malformed posted document, or - - 404 -- the requested document was not found. - -- the user requested statuses or serverdescs, and none of the ones - requested were found (0.2.0.5-alpha and later). - - 503 -- we are declining the request in order to save bandwidth - -- user requested some items that we ordinarily generate or store, - but we do not have any available. - -A. Consensus-negotiation timeline. - - Period begins: this is the Published time. - Everybody sends votes - Reconciliation: everybody tries to fetch missing votes. - consensus may exist at this point. - End of voting period: - everyone swaps signatures. - Now it's okay for caches to download - Now it's okay for clients to download. - - Valid-after/valid-until switchover - -B. General-use HTTP URLs - - "Fingerprints" in these URLs are base16-encoded SHA1 hashes. - - The most recent v3 consensus should be available at: - - http:///tor/status-vote/current/consensus.z - - Similarly, the v3 microdescriptor consensus should be available at: - - http:///tor/status-vote/current/consensus-microdesc.z - - Starting with Tor version 0.2.1.1-alpha is also available at: - - http:///tor/status-vote/current/consensus/++.z - - (NOTE: Due to squid proxy url limitations at most 96 fingerprints can be - retrieved in a single request.) - - Where F1, F2, etc. are authority identity fingerprints the client trusts. - Servers will only return a consensus if more than half of the requested - authorities have signed the document, otherwise a 404 error will be sent - back. The fingerprints can be shortened to a length of any multiple of - two, using only the leftmost part of the encoded fingerprint. Tor uses - 3 bytes (6 hex characters) of the fingerprint. - - Clients SHOULD sort the fingerprints in ascending order. Server MUST - accept any order. - - Clients SHOULD use this format when requesting consensus documents from - directory authority servers and from caches running a version of Tor - that is known to support this URL format. - - A concatenated set of all the current key certificates should be available - at: - - http:///tor/keys/all.z - - The key certificate for this server should be available at: - - http:///tor/keys/authority.z - - The key certificate for an authority whose authority identity fingerprint - is should be available at: - - http:///tor/keys/fp/.z - - The key certificate whose signing key fingerprint is should be - available at: - - http:///tor/keys/sk/.z - - The key certificate whose identity key fingerprint is and whose signing - key fingerprint is should be available at: - - http:///tor/keys/fp-sk/-.z - - (As usual, clients may request multiple certificates using: - - http:///tor/keys/fp-sk/-+-.z ) - - [The above fp-sk format was not supported before Tor 0.2.1.9-alpha.] - - The most recent descriptor for a server whose identity key has a - fingerprint of should be available at: - - http:///tor/server/fp/.z - - The most recent descriptors for servers with identity fingerprints - ,, should be available at: - - http:///tor/server/fp/++.z - - (NOTE: Due to squid proxy url limitations at most 96 fingerprints can be - retrieved in a single request. - - Implementations SHOULD NOT download descriptors by identity key - fingerprint. This allows a corrupted server (in collusion with a cache) to - provide a unique descriptor to a client, and thereby partition that client - from the rest of the network.) - - The server descriptor with (descriptor) digest (in hex) should be - available at: - - http:///tor/server/d/.z - - The most recent descriptors with digests ,, should be - available at: - - http:///tor/server/d/++.z - - The most recent descriptor for this server should be at: - - http:///tor/server/authority.z - - This is used for authorities, and also if a server is configured - as a bridge. The official Tor implementations (starting at - 0.1.1.x) use this resource to test whether a server's own DirPort - is reachable. It is also useful for debugging purposes. - - A concatenated set of the most recent descriptors for all known servers - should be available at: - - http:///tor/server/all.z - - Extra-info documents are available at the URLS - - http:///tor/extra/d/... - http:///tor/extra/fp/... - http:///tor/extra/all[.z] - http:///tor/extra/authority[.z] - (As for /tor/server/ URLs: supports fetching extra-info - documents by their digest, by the fingerprint of their servers, - or all at once. When serving by fingerprint, we serve the - extra-info that corresponds to the descriptor we would serve by - that fingerprint. Only directory authorities of version - 0.2.0.1-alpha or later are guaranteed to support the first - three classes of URLs. Caches may support them, and MUST - support them if they have advertised "caches-extra-info".) - - For debugging, directories SHOULD expose non-compressed objects at - URLs like the above, but without the final ".z". If the client uses - Accept-Encodings header, it should override the presence or absence - of the ".z" (see section 6.1). - - Clients SHOULD use upper case letters (A-F) when base16-encoding - fingerprints. Servers MUST accept both upper and lower case fingerprints - in requests. - -C. Converting a curve25519 public key to an ed25519 public key - - Given an X25519 key, that is, an affine point (u,v) on the - Montgomery curve defined by - - bv^2 = u(u^2 + au +1) - - where - - a = 486662 - b = 1 - - and comprised of the compressed form (i.e. consisting of only the - u-coordinate), we can retrieve the y-coordinate of the affine point - (x,y) on the twisted Edwards form of the curve defined by - - -x^2 + y^2 = 1 + d x^2 y^2 - - where - - d = - 121665/121666 - - by computing - - y = (u-1)/(u+1). - - and then we can apply the usual curve25519 twisted Edwards point - decompression algorithm to find _an_ x-coordinate of an affine - twisted Edwards point to check signatures with. Signing keys for - ed25519 are compressed curve points in twisted Edwards form (so a - y-coordinate and the sign of the x-coordinate), and X25519 keys are - compressed curve points in Montgomery form (i.e. a u-coordinate). - - However, note that compressed point in Montgomery form neglects to - encode what the sign of the corresponding twisted Edwards - x-coordinate would be. Thus, we need the sign of the x-coordinate - to do this operation; otherwise, we'll have two possible - x-coordinates that might have correspond to the ed25519 public key. - - To get the sign, the easiest way is to take the corresponding - private key, feed it to the ed25519 public key generation - algorithm, and see what the sign is. - - [Recomputing the sign bit from the private key every time sounds - rather strange and inefficient to me… —isis] - - Note that in addition to its coordinates, an expanded Ed25519 private key - also has a 32-byte random value, "prefix", used to compute internal `r` - values in the signature. For security, this prefix value should be - derived deterministically from the curve25519 key. The Tor - implementation derives it as SHA512(private_key | STR)[0..32], where - STR is the nul-terminated string: - - "Derive high part of ed25519 key from curve25519 key\0" - - - On the client side, where there is no access to the curve25519 private - keys, one may use the curve25519 public key's Montgomery u-coordinate to - recover the Montgomery v-coordinate by computing the right-hand side of - the Montgomery curve equation: - - bv^2 = u(u^2 + au +1) - - where - - a = 486662 - b = 1 - - Then, knowing the intended sign of the Edwards x-coordinate, one - may recover said x-coordinate by computing: - - x = (u/v) * sqrt(-a - 2) - -D. Inferring missing proto lines. - - The directory authorities no longer allow versions of Tor before - 0.2.4.18-rc. But right now, there is no version of Tor in the consensus - before 0.2.4.19. Therefore, we should disallow versions of Tor earlier - than 0.2.4.19, so that we can have the protocol list for all current Tor - versions include: - - Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 Link=1-4 - LinkAuth=1 Microdesc=1-2 Relay=1-2 - - For Desc, Microdesc and Cons, Tor versions before 0.2.7.stable should be - taken to only support version 1. - -E. Limited ed diff format - - We support the following format for consensus diffs. It's a - subset of the ed diff format, but clients MUST NOT accept other - ed commands. - - We support the following ed commands, each on a line by itself: - - - "d" Delete line n1 - - ",d" Delete lines n1 through n2, inclusive - - ",$d" Delete line n1 through the end of the file, inclusive. - - "c" Replace line n1 with the following block - - ",c" Replace lines n1 through n2, inclusive, with the - following block. - - "a" Append the following block after line n1. - - Note that line numbers always apply to the file after all previous - commands have already been applied. Note also that line numbers - are 1-indexed. - - The commands MUST apply to the file from back to front, such that - lines are only ever referred to by their position in the original - file. - - If there are any directory signatures on the original document, the - first command MUST be a ",$d" form to remove all of the directory - signatures. Using this format ensures that the client will - successfully apply the diff even if they have an unusual encoding for - the signatures. - - The replace and append command take blocks. These blocks are simply - appended to the diff after the line with the command. A line with - just a period (".") ends the block (and is not part of the lines - to add). Note that it is impossible to insert a line with just - a single dot. -- cgit v1.2.3-54-g00ecf