1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
|
<a id="dir-spec.txt-4"></a>
# Directory cache operation
All directory caches implement this section, except as noted.
<a id="dir-spec.txt-4.1"></a>
## Downloading consensus status documents from directory authorities { #download-ns-from-auth }
All directory caches try to keep a recent
network-status consensus document to serve to clients. A cache ALWAYS
downloads a network-status consensus if any of the following are true:
- The cache has no consensus document.
- The cache's consensus document is no longer valid.
Otherwise, the cache downloads a new consensus document at a randomly
chosen time in the first half-interval after its current consensus
stops being fresh. (This time is chosen at random to avoid swarming
the authorities at the start of each period. The interval size is
inferred from the difference between the valid-after time and the
fresh-until time on the consensus.)
```text
[For example, if a cache has a consensus that became valid at 1:00,
and is fresh until 2:00, that cache will fetch a new consensus at
a random time between 2:00 and 2:30.]
```
Directory caches also fetch consensus flavors from the authorities.
Caches check the correctness of consensus flavors, but do not check
anything about an unrecognized consensus document beyond its digest and
length. Caches serve all consensus flavors from the same locations as
the directory authorities.
<a id="dir-spec.txt-4.2"></a>
## Downloading server descriptors from directory authorities { #download-desc-from-auth }
Periodically (currently, every 10 seconds), directory caches check
whether there are any specific descriptors that they do not have and that
they are not currently trying to download. Caches identify these
descriptors by hash in the recent network-status consensus documents.
If so, the directory cache launches requests to the authorities for these
descriptors.
If one of these downloads fails, we do not try to download that descriptor
from the authority that failed to serve it again unless we receive a newer
network-status consensus that lists the same descriptor.
Directory caches must potentially cache multiple descriptors for each
router. Caches must not discard any descriptor listed by any recent
consensus. If there is enough space to store additional descriptors,
caches SHOULD try to hold those which clients are likely to download the
most. (Currently, this is judged based on the interval for which each
descriptor seemed newest.)
\[XXXX define recent\]
<a id="dir-spec.txt-4.3"></a>
## Downloading microdescriptors from directory authorities { #download-md-from-auth }
Directory mirrors should fetch, cache, and serve each microdescriptor
from the authorities.
The microdescriptors with base64 hashes `<D1>`, `<D2>`, `<D3>` are available
at:
`http://<hostname>/tor/micro/d/<D1>-<D2>-<D3>[.z]`
`<Dn>` are base64 encoded with trailing =s omitted for size and for
consistency with the microdescriptor consensus format. -s are used
instead of +s to separate items, since the + character is used in
base64 encoding.
Directory mirrors should check to make sure that the microdescriptors
they're about to serve match the right hashes (either the hashes from
the fetch URL or the hashes from the consensus, respectively).
(NOTE: Due to squid proxy url limitations at most 92 microdescriptor hashes
can be retrieved in a single request.)
<a id="dir-spec.txt-4.4"></a>
## Downloading extra-info documents from directory authorities { #download-ei-from-auth }
Any cache that chooses to cache extra-info documents should implement this
section.
Periodically, the Tor instance checks whether it is missing any extra-info
documents: in other words, if it has any server descriptors with an
extra-info-digest field that does not match any of the extra-info
documents currently held. If so, it downloads whatever extra-info
documents are missing. Caches download from authorities. We follow the
same splitting and back-off rules as in section 4.2.
<a id="dir-spec.txt-4.5"></a>
## Consensus diffs { #diffs }
Instead of downloading an entire consensus, clients may download
a "diff" document containing an ed-style diff from a previous
consensus document. Caches (and authorities) make these diffs as
they learn about new consensuses. To do so, they must store a
record of older consensuses.
(Support for consensus diffs was added in 0.3.1.1-alpha, and is
advertised with the DirCache protocol version "2" or later.)
<a id="dir-spec.txt-4.5.1"></a>
### Consensus diff format { #diff-format }
Consensus diffs are formatted as follows:
The first line is "network-status-diff-version 1" NL
The second line is
"hash" SP FromDigest SP ToDigest NL
where FromDigest is the hex-encoded SHA3-256 digest of the _signed
part_ of the consensus that the diff should be applied to, and
ToDigest is the hex-encoded SHA3-256 digest of the _entire_
consensus resulting from applying the diff. (See 3.4.1 for
information on that part of a consensus is signed.)
The third and subsequent lines encode the diff from FromDigest to
ToDigest in a limited subset of the ed diff format, as specified
in appendix E.
<a id="dir-spec.txt-4.5.2"></a>
### Serving and requesting diffs { #diff-requests }
When downloading the current consensus, a client may include an
HTTP header of the form
X-Or-Diff-From-Consensus: HASH1, HASH2, ...
where the HASH values are hex-encoded SHA3-256 digests of the
_signed part_ of one or more consensuses that the client knows
about.
If a cache knows a consensus diff from one of those consensuses
to the most recent consensus of the requested flavor, it may
send that diff instead of the specified consensus.
Caches also serve diffs from the URIs:
```text
/tor/status-vote/current/consensus/diff/<HASH>/<FPRLIST>.z
/tor/status-vote/current/consensus-<FLAVOR>/diff/<HASH>/<FPRLIST>.z
```
where FLAVOR is the consensus flavor, defaulting to "ns", and
FPRLIST is +-separated list of recognized authority identity
fingerprints as in appendix B.
<a id="dir-spec.txt-4.6"></a>
## Retrying failed downloads { #retry-as-cache }
See section 5.5 below; it applies to caches as well as clients.
|
