1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
|
<a id="dir-spec.txt-5"></a>
# Client operation
Every Tor that is not a directory server (that is, those that do
not have a DirPort set) implements this section.
<a id="dir-spec.txt-5.1"></a>
## Downloading network-status documents { #download-ns }
Each client maintains a list of directory authorities. Insofar as
possible, clients SHOULD all use the same list.
```text
[Newer versions of Tor (0.2.8.1-alpha and later):
Each client also maintains a list of default fallback directory mirrors
(fallbacks). Each released version of Tor MAY have a different list,
depending on the mirrors that satisfy the fallback directory criteria at
release time.]
```
Clients try to have a live consensus network-status document at all times.
A network-status document is "live" if the time in its valid-after field
has passed, and the time in its valid-until field has not passed.
When a client has no consensus network-status document, it downloads it
from a randomly chosen fallback directory mirror or authority. Clients
prefer fallbacks to authorities, trying them earlier and more frequently.
In all other cases, the client downloads from caches randomly chosen from
among those believed to be V3 directory servers. (This information comes
from the network-status documents.)
After receiving any response client MUST discard any network-status
documents that it did not request.
On failure, the client waits briefly, then tries that network-status
document again from another cache. The client does not build circuits
until it has a live network-status consensus document, and it has
descriptors for a significant proportion of the routers that it believes
are running (this is configurable using torrc options and consensus
parameters).
```text
[Newer versions of Tor (0.2.6.2-alpha and later):
If the consensus contains Exits (the typical case), Tor will build both
exit and internal circuits. When bootstrap completes, Tor will be ready
to handle an application requesting an exit circuit to services like the
World Wide Web.
```
If the consensus does not contain Exits, Tor will only build internal
circuits. In this case, earlier statuses will have included "internal"
as indicated above. When bootstrap completes, Tor will be ready to handle
an application requesting an internal circuit to hidden services at
".onion" addresses.
If a future consensus contains Exits, exit circuits may become available.\]
(Note: clients can and should pick caches based on the network-status
information they have: once they have first fetched network-status info
from an authority or fallback, they should not need to go to the authority
directly again, and should only choose the fallback at random, based on its
consensus weight in the current consensus.)
To avoid swarming the caches whenever a consensus expires, the
clients download new consensuses at a randomly chosen time after the
caches are expected to have a fresh consensus, but before their
consensus will expire. (This time is chosen uniformly at random from
the interval between the time 3/4 into the first interval after the
consensus is no longer fresh, and 7/8 of the time remaining after
that before the consensus is invalid.)
```text
[For example, if a client has a consensus that became valid at 1:00,
and is fresh until 2:00, and expires at 4:00, that client will fetch
a new consensus at a random time between 2:45 and 3:50, since 3/4
of the one-hour interval is 45 minutes, and 7/8 of the remaining 75
minutes is 65 minutes.]
```
Clients may choose to download the microdescriptor consensus instead
of the general network status consensus. In that case they should use
the same update strategy as for the normal consensus. They should not
download more than one consensus flavor.
When a client does not have a live consensus, it will generally use the
most recent consensus it has if that consensus is "reasonably live". A
"reasonably live" consensus is one that expired less than 24 hours ago.
<a id="dir-spec.txt-5.2"></a>
## Downloading server descriptors or microdescriptors { #download-desc }
Clients try to have the best descriptor for each router. A descriptor is
"best" if:
- It is listed in the consensus network-status document.
Periodically (currently every 10 seconds) clients check whether there are
any "downloadable" descriptors. A descriptor is downloadable if:
```text
- It is the "best" descriptor for some router.
- The descriptor was published at least 10 minutes in the past.
(This prevents clients from trying to fetch descriptors that the
mirrors have probably not yet retrieved and cached.)
- The client does not currently have it.
- The client is not currently trying to download it.
- The client would not discard it immediately upon receiving it.
- The client thinks it is running and valid (see section 5.4.1 below).
```
If at least 16 known routers have downloadable descriptors, or if
enough time (currently 10 minutes) has passed since the last time the
client tried to download descriptors, it launches requests for all
downloadable descriptors.
When downloading multiple server descriptors, the client chooses multiple
mirrors so that:
```text
- At least 3 different mirrors are used, except when this would result
in more than one request for under 4 descriptors.
- No more than 128 descriptors are requested from a single mirror.
- Otherwise, as few mirrors as possible are used.
After choosing mirrors, the client divides the descriptors among them
randomly.
```
After receiving any response the client MUST discard any descriptors that
it did not request.
When a descriptor download fails, the client notes it, and does not
consider the descriptor downloadable again until a certain amount of time
has passed. (Currently 0 seconds for the first failure, 60 seconds for the
second, 5 minutes for the third, 10 minutes for the fourth, and 1 day
thereafter.) Periodically (currently once an hour) clients reset the
failure count.
Clients retain the most recent descriptor they have downloaded for each
router so long as it is listed in the consensus. If it is not listed,
they keep it so long as it is not too old (currently, ROUTER_MAX_AGE=48
hours) and no better router descriptor has been downloaded for the same
relay. Caches retain descriptors until they are at least
OLD_ROUTER_DESC_MAX_AGE=5 days old.
Clients which chose to download the microdescriptor consensus instead
of the general consensus must download the referenced microdescriptors
instead of server descriptors. Clients fetch and cache
microdescriptors preemptively from dir mirrors when starting up, like
they currently fetch descriptors. After bootstrapping, clients only
need to fetch the microdescriptors that have changed.
When a client gets a new microdescriptor consensus, it looks to see if
there are any microdescriptors it needs to learn, and launches a request
for them.
Clients maintain a cache of microdescriptors along with metadata like
when it was last referenced by a consensus, and which identity key
it corresponds to. They keep a microdescriptor until it hasn't been
mentioned in any consensus for a week. Future clients might cache them
for longer or shorter times.
<a id="dir-spec.txt-5.3"></a>
## Downloading extra-info documents { #download-extra }
Any client that uses extra-info documents should implement this
section.
Note that generally, clients don't need extra-info documents.
Periodically, the Tor instance checks whether it is missing any extra-info
documents: in other words, if it has any server descriptors with an
extra-info-digest field that does not match any of the extra-info
documents currently held. If so, it downloads whatever extra-info
documents are missing. Clients try to download from caches.
We follow the same splitting and back-off rules as in section 5.2.
<a id="dir-spec.txt-5.5"></a>
## Retrying failed downloads
This section applies to caches as well as to clients.
When a client fails to download a resource (a consensus, a router
descriptor, a microdescriptor, etc) it waits for a certain amount of
time before retrying the download. To determine the amount of time
to wait, clients use a randomized exponential backoff algorithm.
(Specifically, they use a variation of the "decorrelated jitter"
algorithm from
<https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/> .)
The specific formula used to compute the 'i+1'th delay is:
```text
Delay_{i+1} = MIN(cap, random_between(lower_bound, upper_bound)))
where upper_bound = MAX(lower_bound+1, Delay_i * 3)
lower_bound = MAX(1, base_delay).
```
The value of 'cap' is set to INT_MAX; the value of 'base_delay'
depends on what is being downloaded, whether the client is fully
bootstrapped, how the client is configured, and where it is
downloading from. Current base_delay values are:
```text
Consensus objects, as a non-bridge cache:
0 (TestingServerConsensusDownloadInitialDelay)
Consensus objects, as a client or bridge that has bootstrapped:
0 (TestingClientConsensusDownloadInitialDelay)
Consensus objects, as a client or bridge that is bootstrapping,
when connecting to an authority because no "fallback" caches are
known:
0 (ClientBootstrapConsensusAuthorityOnlyDownloadInitialDelay)
Consensus objects, as a client or bridge that is bootstrapping,
when "fallback" caches are known but connecting to an authority
anyway:
6 (ClientBootstrapConsensusAuthorityDownloadInitialDelay)
Consensus objects, as a client or bridge that is bootstrapping,
when downloading from a "fallback" cache.
0 (ClientBootstrapConsensusFallbackDownloadInitialDelay)
Bridge descriptors, as a bridge-using client when at least one bridge
is usable:
10800 (TestingBridgeDownloadInitialDelay)
Bridge descriptors, otherwise:
0 (TestingBridgeBootstrapDownloadInitialDelay)
Other objects, as cache or authority:
0 (TestingServerDownloadInitialDelay)
Other objects, as client:
0 (TestingClientDownloadInitialDelay)
```
|
