diff options
Diffstat (limited to 'spec/dir-spec/consensus-formats.md')
| -rw-r--r-- | spec/dir-spec/consensus-formats.md | 706 |
1 files changed, 706 insertions, 0 deletions
diff --git a/spec/dir-spec/consensus-formats.md b/spec/dir-spec/consensus-formats.md new file mode 100644 index 0000000..7a6cd46 --- /dev/null +++ b/spec/dir-spec/consensus-formats.md @@ -0,0 +1,706 @@ +<a id="dir-spec.txt-3.4.1"></a> + +# Vote and consensus status document formats + +Votes and consensuses are more strictly formatted than other documents +in this specification, since different authorities must be able to +generate exactly the same consensus given the same set of votes. + +The procedure for deciding when to generate vote and consensus status +documents are described in section 1.4 on the voting timeline. + +Status documents contain a preamble, an authority section, a list of +router status entries, and one or more footer signature, in that order. + +Unlike other formats described above, a SP in these documents must be a +single space character (hex 20). + +Some items appear only in votes, and some items appear only in +consensuses. Unless specified, items occur in both. + +The preamble contains the following items. They SHOULD occur in the +order given here: + +"network-status-version" SP version NL + +\[At start, exactly once.\] + +A document format version. For this specification, the version is +"3". + +"vote-status" SP type NL + +\[Exactly once.\] + +The status MUST be "vote" or "consensus", depending on the type of +the document. + +"consensus-methods" SP IntegerList NL + +\[At most once for votes; does not occur in consensuses.\] + +A space-separated list of supported methods for generating +consensuses from votes. See section 3.8.1 for details. Absence of +the line means that only method "1" is supported. + +"consensus-method" SP Integer NL + +\[At most once for consensuses; does not occur in votes.\] +\[No extra arguments\] + +See section 3.8.1 for details. + +(Only included when the vote is generated with consensus-method 2 or +later.) + +"published" SP YYYY-MM-DD SP HH:MM:SS NL + +\[Exactly once for votes; does not occur in consensuses.\] + +The publication time for this status document (if a vote). + +"valid-after" SP YYYY-MM-DD SP HH:MM:SS NL + +\[Exactly once.\] + +The start of the Interval for this vote. Before this time, the +consensus document produced from this vote is not officially in +use. + +(Note that because of propagation delays, clients and relays +may see consensus documents that are up to `DistSeconds` +earlier than this time, and should not warn about them.) + +See section 1.4 for voting timeline information. + +"fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL + +\[Exactly once.\] + +The time at which the next consensus should be produced; before this +time, there is no point in downloading another consensus, since there +won't be a new one. See section 1.4 for voting timeline information. + +"valid-until" SP YYYY-MM-DD SP HH:MM:SS NL + +\[Exactly once.\] + +The end of the Interval for this vote. After this time, all +clients should try to find a more recent consensus. See section 1.4 +for voting timeline information. + +In practice, clients continue to use the consensus for up to 24 hours +after it is no longer valid, if no more recent consensus can be +downloaded. + +"voting-delay" SP VoteSeconds SP DistSeconds NL + +\[Exactly once.\] + +VoteSeconds is the number of seconds that we will allow to collect +votes from all authorities; DistSeconds is the number of seconds +we'll allow to collect signatures from all authorities. See +section 1.4 for voting timeline information. + +"client-versions" SP VersionList NL + +\[At most once.\] + +A comma-separated list of recommended Tor versions for client +usage, in ascending order. The versions are given as defined by +version-spec.txt. If absent, no opinion is held about client +versions. + +"server-versions" SP VersionList NL + +\[At most once.\] + +A comma-separated list of recommended Tor versions for relay +usage, in ascending order. The versions are given as defined by +version-spec.txt. If absent, no opinion is held about server +versions. + +"package" SP PackageName SP Version SP URL SP DIGESTS NL + +\[Any number of times.\] + +For this element: + +```text + PACKAGENAME = NONSPACE + VERSION = NONSPACE + URL = NONSPACE + DIGESTS = DIGEST | DIGESTS SP DIGEST + DIGEST = DIGESTTYPE "=" DIGESTVAL + NONSPACE = one or more non-space printing characters + DIGESTVAL = DIGESTTYPE = one or more non-space printing characters + other than "=". +``` + +Indicates that a package called "package" of version VERSION may be +found at URL, and its digest as computed with DIGESTTYPE is equal to +DIGESTVAL. In consensuses, these lines are sorted lexically by +"PACKAGENAME VERSION" pairs, and DIGESTTYPES must appear in ascending +order. A consensus must not contain the same "PACKAGENAME VERSION" +more than once. If a vote contains the same "PACKAGENAME VERSION" +more than once, all but the last is ignored. + +Included in consensuses only for methods 19-33. Earlier methods +did not include this; method 34 removed it. + +"known-flags" SP FlagList NL + +\[Exactly once.\] + +A space-separated list of all of the flags that this document +might contain. A flag is "known" either because the authority +knows about them and might set them (if in a vote), or because +enough votes were counted for the consensus for an authoritative +opinion to have been formed about their status. + +"flag-thresholds" SP Thresholds NL + +\[At most once for votes; does not occur in consensuses.\] + +```text + A space-separated list of the internal performance thresholds + that the directory authority had at the moment it was forming + a vote. + + The metaformat is: + Thresholds = Threshold | Threshold SP Thresholds + Threshold = ThresholdKey '=' ThresholdVal + ThresholdKey = (KeywordChar | "_") + + ThresholdVal = [0-9]+("."[0-9]+)? "%"? + + Commonly used Thresholds at this point include: + + "stable-uptime" -- Uptime (in seconds) required for a relay + to be marked as stable. + + "stable-mtbf" -- MTBF (in seconds) required for a relay to be + marked as stable. + + "enough-mtbf" -- Whether we have measured enough MTBF to look + at stable-mtbf instead of stable-uptime. + + "fast-speed" -- Bandwidth (in bytes per second) required for + a relay to be marked as fast. + + "guard-wfu" -- WFU (in seconds) required for a relay to be + marked as guard. + + "guard-tk" -- Weighted Time Known (in seconds) required for a + relay to be marked as guard. + + "guard-bw-inc-exits" -- If exits can be guards, then all guards + must have a bandwidth this high. + + "guard-bw-exc-exits" -- If exits can't be guards, then all guards + must have a bandwidth this high. + + "ignoring-advertised-bws" -- 1 if we have enough measured bandwidths + that we'll ignore the advertised bandwidth + claims of routers without measured bandwidth. +``` + +"recommended-client-protocols" SP Entries NL +"recommended-relay-protocols" SP Entries NL +"required-client-protocols" SP Entries NL +"required-relay-protocols" SP Entries NL + +\[At most once for each.\] + +The "proto" element as specified in section 2.1.1. + +To vote on these entries, a protocol/version combination is included +only if it is listed by a majority of the voters. + +These lines should be voted on. A majority of votes is sufficient to +make a protocol un-supported. A supermajority of authorities (2/3) +are needed to make a protocol required. The required protocols +should not be torrc-configurable, but rather should be hardwired in +the Tor code. + +The tor-spec.txt section 9 details how a relay and a client should +behave when they encounter these lines in the consensus. + +"params" SP \[Parameters\] NL + +\[At most once\] + +Parameter ::= Keyword '=' Int32 +Int32 ::= A decimal integer between -2147483648 and 2147483647. +Parameters ::= Parameter | Parameters SP Parameter + +The parameters list, if present, contains a space-separated list of +case-sensitive key-value pairs, sorted in lexical order by their +keyword (as ASCII byte strings). Each parameter has its own meaning. + +(Only included when the vote is generated with consensus-method 7 or +later.) + +See param-spec.txt for a list of parameters and their meanings. + +"shared-rand-previous-value" SP NumReveals SP Value NL + +\[At most once\] + +NumReveals ::= An integer greater or equal to 0. +Value ::= Base64-encoded-data + +The shared_random_value that was generated during the second-to-last +shared randomness protocol run. For example, if this document was +created on the 5th of November, this field carries the shared random +value generated during the protocol run of the 3rd of November. + +See section \[SRCALC\] of srv-spec.txt for instructions on how to compute +this value, and see section \[CONS\] for why we include old shared random +values in votes and consensus. + +Value is the actual shared random value encoded in base64. It will +be exactly 256 bits long. NumReveals is the number of commits used +to generate this SRV. + +"shared-rand-current-value" SP NumReveals SP Value NL + +\[At most once\] + +NumReveals ::= An integer greater or equal to 0. +Value ::= Base64-encoded-data + +The shared_random_value that was generated during the latest shared +randomness protocol run. For example, if this document was created on +the 5th of November, this field carries the shared random value +generated during the protocol run of the 4th of November + +See section \[SRCALC\] of srv-spec.txt for instructions on how to compute +this value given the active commits. + +Value is the actual shared random value encoded in base64. It will +be exactly 256 bits long. NumReveals is the number of commits used to +generate this SRV. + +"bandwidth-file-headers" SP KeyValues NL + +\[At most once for votes; does not occur in consensuses.\] + +KeyValues ::= "" | KeyValue | KeyValues SP KeyValue +KeyValue ::= Keyword '=' Value +Value ::= ArgumentCharValue+ +ArgumentCharValue ::= any printing ASCII character except NL and SP. + +The headers from the bandwidth file used to generate this vote. +The bandwidth file headers are described in bandwidth-file-spec.txt. + +If an authority is not configured with a V3BandwidthsFile, this line +SHOULD NOT appear in its vote. + +If an authority is configured with a V3BandwidthsFile, but parsing +fails, this line SHOULD appear in its vote, but without any headers. + +First-appeared: Tor 0.3.5.1-alpha. + +"bandwidth-file-digest" 1\*(SP algorithm "=" digest) NL + +\[At most once for votes; does not occur in consensuses.\] + +A digest of the bandwidth file used to generate this vote. +"algorithm" is the name of the hash algorithm producing "digest", +which can be "sha256" or another algorithm. "digest" is the +base64 encoding of the hash of the bandwidth file, with trailing =s +omitted. + +If an authority is not configured with a V3BandwidthsFile, this line +SHOULD NOT appear in its vote. + +If an authority is configured with a V3BandwidthsFile, but parsing +fails, this line SHOULD appear in its vote, with the digest(s) of the +unparseable file. + +First-appeared: Tor 0.4.0.4-alpha + +The authority section of a vote contains the following items, followed +in turn by the authority's current key certificate: + +```text + "dir-source" SP nickname SP identity SP address SP IP SP dirport SP + orport NL + + [Exactly once, at start] +``` + +Describes this authority. The nickname is a convenient identifier +for the authority. The identity is an uppercase hex fingerprint of +the authority's current (v3 authority) identity key. The address is +the server's hostname. The IP is the server's current IP address, +and dirport is its current directory port. The orport is the +port at that address where the authority listens for OR +connections. + +"contact" SP string NL + +\[Exactly once\] + +An arbitrary string describing how to contact the directory +server's administrator. Administrators should include at least an +email address and a PGP fingerprint. + +"legacy-dir-key" SP FINGERPRINT NL + +\[At most once\] + +Lists a fingerprint for an obsolete _identity_ key still used +by this authority to keep older clients working. This option +is used to keep key around for a little while in case the +authorities need to migrate many identity keys at once. +(Generally, this would only happen because of a security +vulnerability that affected multiple authorities, like the +Debian OpenSSL RNG bug of May 2008.) + +"shared-rand-participate" NL + +\[At most once\] + +Denotes that the directory authority supports and can participate in the +shared random protocol. + +"shared-rand-commit" SP Version SP AlgName SP Identity SP Commit \[SP Reveal\] NL + +\[Any number of times\] + +Version ::= An integer greater or equal to 0. +AlgName ::= 1\*(ALPHA / DIGIT / "\_" / "-") +Identity ::= 40\* HEXDIG +Commit ::= Base64-encoded-data +Reveal ::= Base64-encoded-data + +Denotes a directory authority commit for the shared randomness +protocol, containing the commitment value and potentially also the +reveal value. See sections \[COMMITREVEAL\] and \[VALIDATEVALUES\] of +srv-spec.txt on how to generate and validate these values. + +Version is the current shared randomness protocol version. AlgName is +the hash algorithm that is used (e.g. "sha3-256") and Identity is the +authority's SHA1 v3 identity fingerprint. Commit is the encoded +commitment value in base64. Reveal is optional and if it's set, it +contains the reveal value in base64. + +If a vote contains multiple commits from the same authority, the +receiver MUST only consider the first commit listed. + +"shared-rand-previous-value" SP NumReveals SP Value NL + +\[At most once\] + +See shared-rand-previous-value description above. + +"shared-rand-current-value" SP NumReveals SP Value NL + +\[At most once\] + +See shared-rand-current-value description above. + +The authority section of a consensus contains groups of the following items, +in the order given, with one group for each authority that contributed to +the consensus, with groups sorted by authority identity digest: + +```text + "dir-source" SP nickname SP identity SP address SP IP SP dirport SP + orport NL + + [Exactly once, at start] + + As in the authority section of a vote. + + "contact" SP string NL + + [Exactly once.] + + As in the authority section of a vote. + + "vote-digest" SP digest NL + + [Exactly once.] +``` + +A digest of the vote from the authority that contributed to this +consensus, as signed (that is, not including the signature). +(Hex, upper-case.) + +For each "legacy-dir-key" in the vote, there is an additional "dir-source" +line containing that legacy key's fingerprint, the authority's nickname +with "-legacy" appended, and all other fields as in the main "dir-source" +line for that authority. These "dir-source" lines do not have +corresponding "contact" or "vote-digest" entries. + +Each router status entry contains the following items. Router status +entries are sorted in ascending order by identity digest. + +```text + "r" SP nickname SP identity SP digest SP publication SP IP SP ORPort + SP DirPort NL + + [At start, exactly once.] +``` + +"Nickname" is the OR's nickname. "Identity" is a hash of its +identity key, encoded in base64, with trailing equals sign(s) +removed. "Digest" is a hash of its most recent descriptor as +signed (that is, not including the signature) by the RSA identity +key (see section 1.3.), encoded in base64. + +"Publication" was once the publication time of the router's most +recent descriptor, in the form YYYY-MM-DD HH:MM:SS, in UTC. Now +it is only used in votes, and may be set to a fixed value in +consensus documents. Implementations SHOULD ignore this value +in non-vote documents. + +"IP" is its current IP address; ORPort is its current OR port, +"DirPort" is its current directory port, or "0" for "none". + +"a" SP address ":" port NL + +\[Any number\] + +The first advertised IPv6 address for the OR, if it is reachable. + +Present only if the OR advertises at least one IPv6 address, and the +authority believes that the first advertised address is reachable. +Any other IPv4 or IPv6 addresses should be ignored. + +Address and port are as for "or-address" as specified in +section 2.1.1. + +(Only included when the vote or consensus is generated with +consensus-method 14 or later.) + +"s" SP Flags NL + +\[Exactly once.\] + +A series of space-separated status flags, in lexical order (as ASCII +byte strings). Currently documented flags are: + +```text + "Authority" if the router is a directory authority. + "BadExit" if the router is believed to be useless as an exit node + (because its ISP censors it, because it is behind a restrictive + proxy, or for some similar reason). + "Exit" if the router is more useful for building + general-purpose exit circuits than for relay circuits. The + path building algorithm uses this flag; see path-spec.txt. + "Fast" if the router is suitable for high-bandwidth circuits. + "Guard" if the router is suitable for use as an entry guard. + "HSDir" if the router is considered a v2 hidden service directory. + "MiddleOnly" if the router is considered unsuitable for + usage other than as a middle relay. Clients do not need + to handle this option, since when it is present, the authorities + will automatically vote against flags that would make the router + usable in other positions. (Since 0.4.7.2-alpha.) + "NoEdConsensus" if any Ed25519 key in the router's descriptor or + microdescriptor does not reflect authority consensus. + "Stable" if the router is suitable for long-lived circuits. + "StaleDesc" if the router should upload a new descriptor because + the old one is too old. + "Running" if the router is currently usable over all its published + ORPorts. (Authorities ignore IPv6 ORPorts unless configured to + check IPv6 reachability.) Relays without this flag are omitted + from the consensus, and current clients (since 0.2.9.4-alpha) + assume that every listed relay has this flag. + "Valid" if the router has been 'validated'. Clients before + 0.2.9.4-alpha would not use routers without this flag by + default. Currently, relays without this flag are omitted + from the consensus, and current (post-0.2.9.4-alpha) clients + assume that every listed relay has this flag. + "V2Dir" if the router implements the v2 directory protocol or + higher. + + "v" SP version NL + + [At most once.] +``` + +The version of the Tor protocol that this relay is running. If +the value begins with "Tor" SP, the rest of the string is a Tor +version number, and the protocol is "The Tor protocol as supported +by the given version of Tor." Otherwise, if the value begins with +some other string, Tor has upgraded to a more sophisticated +protocol versioning system, and the protocol is "a version of the +Tor protocol more recent than any we recognize." + +Directory authorities SHOULD omit version strings they receive from +descriptors if they would cause "v" lines to be over 128 characters +long. + +"pr" SP Entries NL + +\[At most once.\] + +The "proto" family element as specified in section 2.1.1. + +During voting, authorities copy these lines immediately below the "v" +lines. When a descriptor does not contain a "proto" entry, the +authorities should reconstruct it using the approach described below +in section D. They are included in the consensus using the same rules +as currently used for "v" lines, if a sufficiently late consensus +method is in use. + +"w" SP "Bandwidth=" INT \[SP "Measured=" INT\] \[SP "Unmeasured=1"\] NL + +\[At most once.\] + +An estimate of the bandwidth of this relay, in an arbitrary +unit (currently kilobytes per second). Used to weight router +selection. See section 3.4.2 for details on how the value of +Bandwidth is determined in a consensus. + +Additionally, the Measured= keyword is present in votes by +participating bandwidth measurement authorities to indicate +a measured bandwidth currently produced by measuring stream +capacities. It does not occur in consensuses. + +'Bandwidth=' and 'Measured=' values must be between 0 and +2^32 - 1 inclusive. + +The "Unmeasured=1" value is included in consensuses generated +with method 17 or later when the 'Bandwidth=' value is not +based on a threshold of 3 or more measurements for this relay. + +Other weighting keywords may be added later. +Clients MUST ignore keywords they do not recognize. + +"p" SP ("accept" / "reject") SP PortList NL + +\[At most once.\] + +PortList = PortOrRange +PortList = PortList "," PortOrRange +PortOrRange = INT "-" INT / INT + +A list of those ports that this router supports (if 'accept') +or does not support (if 'reject') for exit to "most +addresses". + +"m" SP methods 1\*(SP algorithm "=" digest) NL + +\[Any number, only in votes.\] + +Microdescriptor hashes for all consensus methods that an authority +supports and that use the same microdescriptor format. "methods" +is a comma-separated list of the consensus methods that the +authority believes will produce "digest". "algorithm" is the name +of the hash algorithm producing "digest", which can be "sha256" or +something else, depending on the consensus "methods" supporting +this algorithm. "digest" is the base64 encoding of the hash of +the router's microdescriptor with trailing =s omitted. + +```text + "id" SP "ed25519" SP ed25519-identity NL + "id" SP "ed25519" SP "none" NL + [vote only, at most once] + + "stats" SP [KeyValues] NL + + [At most once. Vote only] +``` + +KeyValue ::= Keyword '=' Number +Number ::= \[0-9\]+("."\[0-9\]+)? +KeyValues ::= KeyValue | KeyValues SP KeyValue + +Line containing various statistics that an authority has computed for +this relay. Each stats is represented as a key + value. Reported keys +are: + +```text + "wfu" - Weighted Fractional Uptime + "tk" - Weighted Time Known + "mtbf" - Mean Time Between Failure (stability) + + (As of tor-0.4.6.1-alpha) +``` + +The footer section is delineated in all votes and consensuses supporting +consensus method 9 and above with the following: + +"directory-footer" NL +\[No extra arguments\] + +It contains two subsections, a bandwidths-weights line and a +directory-signature. (Prior to consensus method 9, footers only contained +directory-signatures without a 'directory-footer' line or +bandwidth-weights.) + +The bandwidths-weights line appears At Most Once for a consensus. It does +not appear in votes. + +"bandwidth-weights" \[SP Weights\] NL + +Weight ::= Keyword '=' Int32 +Int32 ::= A decimal integer between -2147483648 and 2147483647. +Weights ::= Weight | Weights SP Weight + +List of optional weights to apply to router bandwidths during path +selection. They are sorted in lexical order (as ASCII byte strings) and +values are divided by the consensus' "bwweightscale" param. Definition +of our known entries are... + +```text + Wgg - Weight for Guard-flagged nodes in the guard position + Wgm - Weight for non-flagged nodes in the guard Position + Wgd - Weight for Guard+Exit-flagged nodes in the guard Position + + Wmg - Weight for Guard-flagged nodes in the middle Position + Wmm - Weight for non-flagged nodes in the middle Position + Wme - Weight for Exit-flagged nodes in the middle Position + Wmd - Weight for Guard+Exit flagged nodes in the middle Position + + Weg - Weight for Guard flagged nodes in the exit Position + Wem - Weight for non-flagged nodes in the exit Position + Wee - Weight for Exit-flagged nodes in the exit Position + Wed - Weight for Guard+Exit-flagged nodes in the exit Position + + Wgb - Weight for BEGIN_DIR-supporting Guard-flagged nodes + Wmb - Weight for BEGIN_DIR-supporting non-flagged nodes + Web - Weight for BEGIN_DIR-supporting Exit-flagged nodes + Wdb - Weight for BEGIN_DIR-supporting Guard+Exit-flagged nodes + + Wbg - Weight for Guard flagged nodes for BEGIN_DIR requests + Wbm - Weight for non-flagged nodes for BEGIN_DIR requests + Wbe - Weight for Exit-flagged nodes for BEGIN_DIR requests + Wbd - Weight for Guard+Exit-flagged nodes for BEGIN_DIR requests + + These values are calculated as specified in section 3.8.3. +``` + +The signature contains the following item, which appears Exactly Once +for a vote, and At Least Once for a consensus. + +```text + "directory-signature" [SP Algorithm] SP identity SP signing-key-digest + NL Signature +``` + +This is a signature of the status document, with the initial item +"network-status-version", and the signature item +"directory-signature", using the signing key. (In this case, we take +the hash through the _space_ after directory-signature, not the +newline: this ensures that all authorities sign the same thing.) +"identity" is the hex-encoded digest of the authority identity key of +the signing authority, and "signing-key-digest" is the hex-encoded +digest of the current authority signing key of the signing authority. + +The Algorithm is one of "sha1" or "sha256" if it is present; +implementations MUST ignore directory-signature entries with an +unrecognized Algorithm. "sha1" is the default, if no Algorithm is +given. The algorithm describes how to compute the hash of the +document before signing it. + +"ns"-flavored consensus documents must contain only sha1 signatures. +Votes and microdescriptor documents may contain other signature +types. Note that only one signature from each authority should be +"counted" as meaning that the authority has signed the consensus. + +(Tor clients before 0.2.3.x did not understand the 'algorithm' +field.) |