diff options
Diffstat (limited to 'dir-spec.txt')
| -rw-r--r-- | dir-spec.txt | 407 |
1 files changed, 132 insertions, 275 deletions
diff --git a/dir-spec.txt b/dir-spec.txt index 27d380f..148163d 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -200,7 +200,8 @@ Items. Every Item begins with a KeywordLine, followed by zero or more Objects. A KeywordLine begins with a Keyword, optionally followed by whitespace and more non-newline characters, and ends with a newline. A - Keyword is a sequence of one or more characters in the set [A-Za-z0-9-]. + Keyword is a sequence of one or more characters in the set [A-Za-z0-9-], + but may not start with -. An Object is a block of encoded data in pseudo-Privacy-Enhanced-Mail (PEM) style format: that is, lines of encoded data MAY be wrapped by inserting an ascii linefeed ("LF", also called newline, or "NL" here) character @@ -214,13 +215,14 @@ Document ::= (Item | NL)+ Item ::= KeywordLine Object* KeywordLine ::= Keyword NL | Keyword WS ArgumentChar+ NL - Keyword = KeywordChar+ - KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-' + Keyword = KeywordStart KeywordChar* + KeywordStart ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' + KeywordChar ::= KeywordStart | '-' ArgumentChar ::= any printing ASCII character except NL. WS = (SP | TAB)+ Object ::= BeginLine Base64-encoded-data EndLine - BeginLine ::= "-----BEGIN " Keyword "-----" NL - EndLine ::= "-----END " Keyword "-----" NL + BeginLine ::= "-----BEGIN " Keyword (" " Keyword)* "-----" NL + EndLine ::= "-----END " Keyword (" " Keyword)* "-----" NL A Keyword may not be "-----BEGIN". @@ -361,7 +363,10 @@ (See note above: clients guess that the next consensus's FU will be two intervals after the current VA.) - VU: The consensus is no longer valid. + VU: The consensus is no longer valid; clients should continue to try to + download a new consensus if they have not done so already. + + VU + 24 hours: Clients will no longer use the consensus at all. VoteSeconds and DistSeconds MUST each be at least 20 seconds; FU-VA and VU-FU MUST each be at least 5 minutes. @@ -436,7 +441,7 @@ "identity-ed25519" NL "-----BEGIN ED25519 CERT-----" NL certificate "-----END ED25519 CERT-----" NL - [At most once, in second position in document.] + [Exactly once, in second position in document.] [No extra arguments] The certificate is a base64-encoded Ed25519 certificate (see @@ -448,14 +453,18 @@ signed-with-ed25519-key extension (see cert-spec.txt, section 2.2.1), so that we can extract the master identity key. + [Before Tor 0.4.5.1-alpha, this field was optional.] + "master-key-ed25519" SP MasterKey NL - [At most once] + [Exactly once] Contains the base-64 encoded ed25519 master key as a single argument. If it is present, it MUST match the identity key in the identity-ed25519 entry. + [Before Tor 0.4.5.1-alpha, this field was optional.] + "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed NL [Exactly once] @@ -530,10 +539,9 @@ "onion-key-crosscert" NL a RSA signature in PEM format. - [At most once, required when identity-ed25519 is present] + [Exactly once] [No extra arguments] - This element contains an RSA signature, generated using the onion-key, of the following: @@ -553,9 +561,12 @@ had control over the secret key corresponding to the onion-key. + [Before Tor 0.4.5.1-alpha, this field was optional whenever + identity-ed25519 was absent.] + "ntor-onion-key" base-64-encoded-key - [At most once] + [Exactly once] A curve25519 public key used for the ntor circuit extended handshake. It's the standard encoding of the OR's curve25519 @@ -564,11 +575,13 @@ for at least 1 week after any new key is published in a subsequent descriptor. + [Before Tor 0.4.5.1-alpha, this field was optional.] + "ntor-onion-key-crosscert" SP Bit NL "-----BEGIN ED25519 CERT-----" NL certificate "-----END ED25519 CERT-----" NL - [At most once, required when identity-ed25519 is present] + [Exactly once] [No extra arguments] A signature created with the ntor-onion-key, using the @@ -588,6 +601,9 @@ had control over the secret key corresponding to the ntor-onion-key. + [Before Tor 0.4.5.1-alpha, this field was optional whenever + identity-ed25519 was absent.] + "signing-key" NL a public key in PEM format [Exactly once] @@ -621,21 +637,24 @@ "router-sig-ed25519" SP Signature NL - [At most once.] - [Required when identity-ed25519 is present; forbidden otherwise.] + [Exactly once.] It MUST be the next-to-last element in the descriptor, appearing immediately before the RSA signature. It MUST contain an Ed25519 - signature of a SHA256 digest of the entire document, from the - first character up to and including the first space after the - "router-sig-ed25519" string, prefixed with the string "Tor - router descriptor signature v1". Its format is: + signature of a SHA256 digest of the entire document. This digest is + taken from the first character up to and including the first space + after the "router-sig-ed25519" string. Before computing the digest, + the string "Tor router descriptor signature v1" is prefixed to the + document. - The signature is encoded in Base64 with terminating =s removed. + The signature is encoded in Base64, with terminating =s removed. The signing key in the identity-ed25519 certificate MUST be the one used to sign the document. + [Before Tor 0.4.5.1-alpha, this field was optional whenever + identity-ed25519 was absent.] + "router-signature" NL Signature NL [At end, exactly once] @@ -813,8 +832,7 @@ "proto" SP Entries NL - [At most one.] - [Will become mandatory at some point after May 2018.] + [Exactly once.] Entries = Entries = Entry @@ -837,10 +855,12 @@ sorted by keyword. Values should be numerically ascending within each entry. (This implies that there should be no overlapping ranges.) Ranges should be represented as compactly as possible. Ints must be no - more than 2^32 - 2. (Yes, 2.) + larger than 63. + + This field was first added in Tor 0.2.9.x. + + [Before Tor 0.4.5.1-alpha, this field was optional.] - This field was first added in Tor 0.2.9.x. Some time after all earlier - Tor relay versions are obsolete, it will become mandatory. 2.1.2. Extra-info document format @@ -1207,6 +1227,8 @@ "hidserv-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL [At most once.] + "hidserv-v3-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL + [At most once.] YYYY-MM-DD HH:MM:SS defines the end of the included measurement interval of length NSEC seconds (86400 seconds by default). @@ -1217,6 +1239,8 @@ "hidserv-rend-relayed-cells" SP NUM SP key=val SP key=val ... NL [At most once.] + "hidserv-rend-v3-relayed-cells" SP NUM SP key=val SP key=val ... NL + [At most once.] Approximate number of RELAY cells seen in either direction on a circuit after receiving and successfully processing a RENDEZVOUS1 @@ -1235,6 +1259,8 @@ "hidserv-dir-onions-seen" SP NUM SP key=val SP key=val ... NL [At most once.] + "hidserv-dir-v3-onions-seen" SP NUM SP key=val SP key=val ... NL + [At most once.] Approximate number of unique hidden-service identities seen in descriptors published to and accepted by this hidden-service @@ -1512,13 +1538,15 @@ "ntor-onion-key" SP base-64-encoded-key NL - [At most once] + [Exactly once] The "ntor-onion-key" element as specified in section 2.1.1. (Only included when generating microdescriptors for consensus-method 16 or later.) + [Before Tor 0.4.5.1-alpha, this field was optional.] + "a" SP address ":" port NL [Any number] @@ -1627,10 +1655,12 @@ "pr" SP Entries NL - [At most once.] + [Exactly once.] The "proto" element as specified in section 2.1.1. + [Before Tor 0.4.5.1-alpha, this field was optional.] + (Note that with microdescriptors, clients do not learn the RSA identity of their routers: they only learn a hash of the RSA identity key. This is all they need to confirm the actual identity key when doing a TLS @@ -1758,8 +1788,7 @@ (Note that because of propagation delays, clients and relays may see consensus documents that are up to `DistSeconds` - earlier than this this time, and should not warn about - them.) + earlier than this time, and should not warn about them.) See section 1.4 for voting timeline information. @@ -1775,10 +1804,14 @@ [Exactly once.] - The end of the Interval for this vote. After this time, the - consensus produced by this vote should not be used. See section 1.4 + The end of the Interval for this vote. After this time, all + clients should try to find a more recent consensus. See section 1.4 for voting timeline information. + In practice, clients continue to use the consensus for up to 24 hours + after it is no longer valid, if no more recent consensus can be + downloaded. + "voting-delay" SP VoteSeconds SP DistSeconds NL [Exactly once.] @@ -1921,238 +1954,7 @@ (Only included when the vote is generated with consensus-method 7 or later.) - Commonly used "param" arguments at this point include: - - "circwindow" -- the default package window that circuits should - be established with. It started out at 1000 cells, but some - research indicates that a lower value would mean fewer cells in - transit in the network at any given time. - Min: 100, Max: 1000 - First-appeared: Tor 0.2.1.20 - - "CircuitPriorityHalflifeMsec" -- the halflife parameter used when - weighting which circuit will send the next cell. Obeyed by Tor - 0.2.2.10-alpha and later. (Versions of Tor between 0.2.2.7-alpha - and 0.2.2.10-alpha recognized a "CircPriorityHalflifeMsec" parameter, - but mishandled it badly.) - Min: -1, Max: 2147483647 (INT32_MAX) - First-appeared: Tor 0.2.2.11-alpha - - "perconnbwrate" and "perconnbwburst" -- if set, each relay sets - up a separate token bucket for every client OR connection, - and rate limits that connection indepedently. Typically left - unset, except when used for performance experiments around trac - entry 1750. Only honored by relays running Tor 0.2.2.16-alpha - and later. (Note that relays running 0.2.2.7-alpha through - 0.2.2.14-alpha looked for bwconnrate and bwconnburst, but then - did the wrong thing with them; see bug 1830 for details.) - Min: 1, Max: 2147483647 (INT32_MAX) - First-appeared: 0.2.2.7-alpha - Removed-in: 0.2.2.16-alpha - - "refuseunknownexits" -- if set to one, exit relays look at - the previous hop of circuits that ask to open an exit stream, - and refuse to exit if they don't recognize it as a relay. The - goal is to make it harder for people to use them as one-hop - proxies. See trac entry 1751 for details. - Min: 0, Max: 1 - First-appeared: 0.2.2.17-alpha - - "bwweightscale" -- Value that bandwidth-weights are divided by. If not - present then this defaults to 10000. - Min: 1 - First-appeared: 0.2.2.10-alpha - - "cbtdisabled", "cbtnummodes", "cbtrecentcount", "cbtmaxtimeouts", - "cbtmincircs", "cbtquantile", "cbtclosequantile", "cbttestfreq", - "cbtmintimeout", "cbtlearntimeout", "cbtmaxopencircs", and - "cbtinitialtimeout" -- see "2.4.5. Consensus parameters governing - behavior" in path-spec.txt for a series of circuit build time related - consensus params. - - "UseOptimisticData" -- If set to zero, clients by default - shouldn't try to send optimistic data to servers until they have - received a RELAY_CONNECTED cell. - Min: 0, Max: 1, Default: 1 - First-appeared: 0.2.3.3-alpha - Default was 0 before: 0.2.9.1-alpha - - "maxunmeasuredbw" -- Used by authorities during voting with - method 17 or later. The maximum value to give for any Bandwidth= - entry for a router that isn't based on at least three - measurements. - First-appeared: 0.2.4.11-alpha - - "Support022HiddenServices" -- Used to implement a mass switch-over - from sending timestamps to hidden services by default to sending - no timestamps at all. If this option is absent, or is set to 1, - clients with the default configuration send timestamps; otherwise, - they do not. - Min: 0, Max: 1. Default: 1. - First-appeared: 0.2.4.18-rc - - "usecreatefast" -- Used to control whether clients use the - CREATE_FAST handshake on the first hop of their circuits. - Min: 0, Max: 1. Default: 1. - First-appeared: 0.2.4.23, 0.2.5.2-alpha - - "pb_mincircs", "pb_noticepct", "pb_warnpct", "pb_extremepct", - "pb_dropguards", "pb_scalecircs", "pb_scalefactor", - "pb_multfactor", "pb_minuse", "pb_noticeusepct", - "pb_extremeusepct", "pb_scaleuse" -- DOCDOC - - "UseNTorHandshake" -- If true, then versions of Tor that support - NTor will prefer to use it by default. - Min: 0, Max: 1. Default: 1. - First-appeared: 0.2.4.8-alpha - - "FastFlagMinThreshold", "FastFlagMaxThreshold" -- lowest and - highest allowable values for the cutoff for routers that should get - the Fast flag. This is used during voting to prevent the threshold - for getting the Fast flag from being too low or too high. - FastFlagMinThreshold: Min: 4. Max: INT32_MAX: Default: 4. - FastFlagMaxThreshold: Min: -. Max: INT32_MAX: Default: INT32_MAX - First-appeared: 0.2.3.11-alpha - - "NumDirectoryGuards", "NumEntryGuards" -- Number of guard nodes - clients should use by default. If NumDirectoryGuards is 0, - we default to NumEntryGuards. - NumDirectoryGuards: Min: 0. Max: 10. Default: 0 - NumEntryGuards: Min: 1. Max: 10. Default: 3 - First-appeared: 0.2.4.23, 0.2.5.6-alpha - - "GuardLifetime" -- Duration for which clients should choose guard - nodes, in seconds. - Min: 30 days. Max: 1826 days. Default: 60 days. - First-appeared: 0.2.4.12-alpha - - "min_paths_for_circs_pct" -- DOCDOC - - "NumNTorsPerTAP" -- When balancing ntor and TAP cells at relays, - how many ntor handshakes should we perform for each TAP handshake? - Min: 1. Max: 100000. Default: 10. - First-appeared: 0.2.4.17-rc - - "AllowNonearlyExtend" -- If true, permit EXTEND cells that are not - inside RELAY_EARLY cells. - Min: 0. Max: 1. Default: 0. - First-appeared: 0.2.3.11-alpha - - "AuthDirNumSRVAgreements" -- Minimum number of agreeing directory - authority votes required for a fresh shared random value to be written - in the consensus (this rule only applies on the first commit round of - the shared randomness protocol). - Min: 1. Max: INT32_MAX. Default: 2/3 of the total number of - dirauth. - - "max-consensuses-age-to-cache-for-diff" -- Determines how - much consensus history (in hours) relays should try to cache - in order to serve diffs. (min 0, max 8192, default 72) - - "try-diff-for-consensus-newer-than" -- This parameter - determines how old a consensus can be (in hours) before a - client should no longer try to find a diff for it. (min 0, - max 8192, default 72) - - onion key lifetime parameters: - "onion-key-rotation-days" -- (min 1, max 90, default 28) - "onion-key-grace-period-days" -- (min 1, max - onion-key-rotation-days, default 7) - Every relay should list each onion key it generates for - onion-key-rotation-days days after generating it, and then - replace it. Relays should continue to accept their most recent - previous onion key for an additional onion-key-grace-period-days - days after it is replaced. (Introduced in 0.3.1.1-alpha; - prior versions of tor hardcoded both of these values to 7 days.) - - Hidden service v3 parameters: - "hs_intro_min_introduce2" - "hs_intro_max_introduce2" -- Minimum/maximum amount of INTRODUCE2 cells - allowed per circuits before rotation (actual - amount picked at random between these two values). - "hs_intro_min_lifetime" - "hs_intro_max_lifetime" -- Minimum/maximum lifetime in seconds that a service - should keep an intro point for (actual lifetime picked at - random between these two values). - "hs_intro_num_extra" -- Number of extra intro points a service is allowed to open. - This concept comes from proposal #155. - "hsdir_interval" -- The length of a time period. See rend-spec-v3.txt - section [TIME-PERIODS]. - "hsdir_n_replicas" -- Number of HS descriptor replicas. - "hsdir_spread_fetch" -- Total number of HSDirs per replica a tor client - should select to try to fetch a descriptor. - "hsdir_spread_store" -- Total number of HSDirs per replica a service - will upload its descriptor to. - "HSV3MaxDescriptorSize" -- Maximum descriptor size (in bytes). - - "hs_service_max_rdv_failures" -- This parameter determines the maximum - number of rendezvous attempt an HS service can make per introduction. - Min 1. Max 10. Default 2. - First-appeared: 0.3.3.0-alpha. - - Denial of Service mitigation parameters. Introduced in 0.3.3.2-alpha: - - "DoSCircuitCreationEnabled" -- Enable the circuit creation DoS - mitigation. - - "DoSCircuitCreationMinConnections" -- Minimum threshold of concurrent - connections before a client address can be flagged as executing a - circuit creation DoS - - "DoSCircuitCreationRate" -- Allowed circuit creation rate per second - per client IP address once the minimum concurrent connection - threshold is reached. - - "DoSCircuitCreationBurst" -- The allowed circuit creation burst per - client IP address once the minimum concurrent connection threshold is - reached. - - "DoSCircuitCreationDefenseType" -- Defense type applied to a detected - client address for the circuit creation mitigation. - - 1: No defense. - 2: Refuse circuit creation for the - DoSCircuitCreationDefenseTimePeriod period. - - "DoSCircuitCreationDefenseTimePeriod" -- The base time period that - the DoS defense is activated for. - - "DoSConnectionEnabled" -- Enable the connection DoS mitigation. - - "DoSConnectionMaxConcurrentCount" -- The maximum threshold of - concurrent connection from a client IP address. - - "DoSConnectionDefenseType" -- Defense type applied to a detected - client address for the connection mitigation. Possible values are: - - 1: No defense. - 2: Immediately close new connections. - - "DoSRefuseSingleHopClientRendezvous" -- Refuse establishment of - rendezvous points for single hop clients. - - "circ_max_cell_queue_size" -- This parameter determines the maximum - number of cells allowed per circuit queue. - Min 1000. Max 4294967295. Default 50000. - First-appeared: 0.3.3.6-rc. - - "circpad_max_circ_queued_cells" -- The circuitpadding module will - stop sending more padding cells if more than this many cells are in - the circuit queue a given circuit. Min: 0. Max: 50000. Default 1000. - First appeared: 0.4.0.3-alpha. - - "sendme_emit_min_version" -- Minimum SENDME version that can be sent. - Min: 0. Max: 255. Default 0. First - appeared: 0.4.1.1-alpha. - - "sendme_accept_min_version" -- Minimum SENDME version that is accepted. - Min: 0. Max: 255. Default 0. First - appeared: 0.4.1.1-alpha. - - "HiddenServiceEnableIntroDoSDefense" -- This parameter makes tor start - using this new proposed extension if available by the introduction - point (for protover HSIntro=5). Min: 0. Max: 1. Default: 0. First - appeared: 0.4.2.1-alpha. + See param-spec.txt for a list of parameters and their meanings. "shared-rand-previous-value" SP NumReveals SP Value NL @@ -3669,6 +3471,10 @@ FPRLIST is +-separated list of recognized authority identity fingerprints as in appendix B. +4.6 Retrying failed downloads + + See section 5.5 below; it applies to caches as well as clients. + 5. Client operation Every Tor that is not a directory server (that is, those that do @@ -3794,12 +3600,11 @@ failure count. Clients retain the most recent descriptor they have downloaded for each - router so long as it is not too old (currently, 48 hours), OR so long as - no better descriptor has been downloaded for the same router. - - [Versions of Tor before 0.1.2.3-alpha would discard descriptors simply for - being published too far in the past.] [The code seems to discard - descriptors in all cases after they're 5 days old. True? -RD] + router so long as it is listed in the consensus. If it is not listed, + they keep it so long as it is not too old (currently, ROUTER_MAX_AGE=48 + hours) and no better router descriptor has been downloaded for the same + relay. Caches retain descriptors until they are at least + OLD_ROUTER_DESC_MAX_AGE=5 days old. Clients which chose to download the microdescriptor consensus instead of the general consensus must download the referenced microdescriptors @@ -3809,13 +3614,8 @@ need to fetch the microdescriptors that have changed. When a client gets a new microdescriptor consensus, it looks to see if - there are any microdescriptors it needs to learn. If it needs to learn - more than half of the microdescriptors, it requests 'all', else it - requests only the missing ones. Clients MAY try to determine whether - the upload bandwidth for listing the microdescriptors they want is more - or less than the download bandwidth for the microdescriptors they do - not want. - [XXX The 'all' URL is not implemented yet. -KL] + there are any microdescriptors it needs to learn, and launches a request + for them. Clients maintain a cache of microdescriptors along with metadata like when it was last referenced by a consensus, and which identity key @@ -3885,6 +3685,63 @@ (This section is removed; authorities no longer assign the 'Named' flag.) +5.5. Retrying failed downloads + + This section applies to caches as well as to clients. + + When a client fails to download a resource (a consensus, a router + descriptor, a microdescriptor, etc) it waits for a certain amount of + time before retrying the download. To determine the amount of time + to wait, clients use a randomized exponential backoff algorithm. + (Specifically, they use a variation of the "decorrelated jitter" + algorithm from + https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ .) + + The specific formula used to compute the 'i+1'th delay is: + + Delay_{i+1} = MIN(cap, random_between(lower_bound, upper_bound))) + where upper_bound = MAX(lower_bound+1, Delay_i * 3) + lower_bound = MAX(1, base_delay). + + The value of 'cap' is set to INT_MAX; the value of 'base_delay' + depends on what is being downloaded, whether the client is fully + bootstrapped, how the client is configured, and where it is + downloading from. Current base_delay values are: + + Consensus objects, as a non-bridge cache: + 0 (TestingServerConsensusDownloadInitialDelay) + + Consensus objects, as a client or bridge that has bootstrapped: + 0 (TestingClientConsensusDownloadInitialDelay) + + Consensus objects, as a client or bridge that is bootstrapping, + when connecting to an authority because no "fallback" caches are + known: + 0 (ClientBootstrapConsensusAuthorityOnlyDownloadInitialDelay) + + Consensus objects, as a client or bridge that is bootstrapping, + when "fallback" caches are known but connecting to an authority + anyway: + 6 (ClientBootstrapConsensusAuthorityDownloadInitialDelay) + + Consensus objects, as a client or bridge that is bootstrapping, + when downloading from a "fallback" cache. + 0 (ClientBootstrapConsensusFallbackDownloadInitialDelay) + + Bridge descriptors, as a bridge-using client when at least one bridge + is usable: + 10800 (TestingBridgeDownloadInitialDelay) + + Bridge descriptors, otherwise: + 0 (TestingBridgeBootstrapDownloadInitialDelay) + + Other objects, as cache or authority: + 0 (TestingServerDownloadInitialDelay) + + Other objects, as client: + 0 (TestingClientDownloadInitialDelay) + + 6. Standards compliance All clients and servers MUST support HTTP 1.0. Clients and servers MAY |