From 0b4e613f1bb48f8e5976de479426e76f08a69ebc Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Wed, 21 Mar 2007 03:20:01 +0000 Subject: r12257@catbus: nickm | 2007-03-20 23:17:52 -0400 Add specification to proposal 104; settle on "extra info" approach rather than "long descriptor" approach. svn:r9890 --- proposals/104-short-descriptors.txt | 186 ++++++++++++++++++++++-------------- 1 file changed, 114 insertions(+), 72 deletions(-) (limited to 'proposals/104-short-descriptors.txt') diff --git a/proposals/104-short-descriptors.txt b/proposals/104-short-descriptors.txt index f1d4a9a..04df6c5 100644 --- a/proposals/104-short-descriptors.txt +++ b/proposals/104-short-descriptors.txt @@ -9,9 +9,7 @@ Status: Open Overview: This document proposes moving unused-by-clients information from regular - router descriptors into a special "long form" router descriptor. - - It presents options; it is not yet a complete proposal. + router descriptors into a new "extra info" router descriptor. Proposal: @@ -20,23 +18,21 @@ Proposal: "write-history" fields are used only by the authorities for monitoring the status of the network. If we took them out, the size of a compressed list of all the routers would fall by about 60%. (No other disposable field - would save more than 2%.) - - One possible solution here is that routers should generate and upload a - short-form and long-form descriptor. Only the short-form descriptor should - ever be used by anybody for routing. The long-form descriptor should be - used only for analytics and other tools. (If we allowed people to route - with long descriptors, we'd have to ensure that they stayed in sync with - the short ones somehow. So let's not do that.) We can ensure that the - short descriptors are used by only recommending those in the network - statuses. - - Another possible solution would be to drop these fields from descriptors, - and have them uploaded as a part of a separate "bandwidth report" to the - authorities. This could help prevent the mistake of using long descriptors - in the place of short ones. It could also be generalized later to be an - overall status report, to include sanitized GeoIP information and whatever - else comes up. + would save much more than 2%.) + + We propose to remove these fields from descriptors, and and have them + uploaded as a part of a separate signed "extra info" to the authorities. + This document will be signed. A hash of this document will be included in + the regular descriptors. + + (We considered another design, where routers would generate and upload a + short-form and a long-form descriptor. Only the short-form descriptor would + ever be used by anybody for routing. The long-form descriptor would be + used only for analytics and other tools. We decided against this because + well-behaved tools would need to download short-form descriptors too (as + these would be the only ones indexed), and hence get redundant info. Badly + behaved tools would download only long-form descriptors, and expose + themselves to partitioning attacks and that like.) Other disposable fields: @@ -55,63 +51,109 @@ Other disposable fields: have much disk, or have really slow disk (e.g. USB). And we don't store caches compressed right now. -RD] -Issues: - - Indexing long descriptor or bandwidth reports presents an issue: right now - the way to make sure you have the same copy of a descriptor as everyone - else is to request the descriptor by its digest, and to make sure that - the digest you request is the one that the authorities like. - - Authorities should presumably list the digests of short descriptors, since - that's what most everybody will be using. Including a second digest for - long descriptors/bandwidth reports in the networkstatus would only bloat it - with information nobody wants. - - Possible solutions are: - 1) Drop the property that you can be sure of having the same long - descriptor as others. This seems unoptimal, but if nobody caches - long descriptors so you have to go to the authority to get them, - maybe it's not so bad. - 2) Have a separate extra-information-status that also gets generated by the - authorities; use it to tell which long descriptors others have. Also a - pain. - 3) Have short descriptors include a hash of the corresponding long - descriptor/extra-info. This would keep the same order of magnitude - performance increase (~59.2% savings as opposed to 61% savings.) - This would require longdesc/extra-info downloaders to fetch - router data before they could know which longdescs/extra info to fetch. - 4) Have each authority make a signed concatenated "extra info" document, - and hope we never need to reconcile them. - 5) ???? +Specification: + + 1. Extra Info Format. + + A "extra info" descriptor contains the following fields: + + "extra-info" Nickname IP FINGERPRINT + Identifies what router this is an extra info descriptor for. + FINGERPRINT is encoded in hex, with no spaces. + + "published" + As currently documented in dir-spec.txt + + "read-history" + "write-history" + As currently documented in dir-spec.txt + + "router-signature" NL Signature NL + + A signature of the PKCS1-padded hash of the entire extra info + document, taken from the beginning of the "extra-info" line, through + the newline after the "router-signature" line. An extra info + document is not valid unless the signature is performed with the + identity key whose digest matches FINGERPRINT. + + The "extra-info" field is required and MUST appear first. The + router-signature field is required and MUST appear last. All others are + optional. As for other documents, unrecognized fields must be ignored. + + 2. Existing formats + + Implementations that use "read-history" and "write-history" SHOULD + accept router descriptors that contain them. (Prior to 0.2.0.x, this + information was encoded in ordinary router descriptors.) + + Add these field to router descriptors: + "extra-info-digest" DIGEST + DIGEST is a hex-encoded digest of the router's extra-info document, + as signed in the router's extra-info. (If this field is absent, + no extra-info-digest exists.) + + "caches-extra-info" + Present if this router is a directory cache that provides + extra-info documents. + + 3. New communications rules + + Clients SHOULD generate and upload an extra-info document after each + descriptor they generate and upload; no more, no less. Clients MUST + upload the new descriptor before they upload the new extra-info. + + Authorities receiving an extra-info documents SHOULD verify all of the + following: + * They have a router descriptor for some server with a matching + nickname, IP, and identity fingerprint. + * That server's identity key has been used to sign the extra-info + document. + * The extra-info-digest field in the router descriptor matches + the digest of the extra-info document. + + Authorities SHOULD try to fetch extra-info documents from one another if + they do not have one matching the digest declared in a router + descriptor. + + Caches that are running locally with a tool that needs to use extra-info + documents MAY download and store extra-info documents. They should do + so when they notice that the recommended descriptor has an + extra-info-digest not matching any extra-info document they currently + have. (Caches not running on a host that needs to use extra-info + documents SHOULD NOT download or cache them.) + + 4. New URLs + + http:///tor/extra/d/... + http:///tor/extra/fp/... + http:///tor/extra/all.z + (As for /tor/server/ URLs: supports fetching extra-info documents + by their digest, by the fingerprint of their servers, or all at + once. Only directory authorities are guaranteed to support these + URLs.) + + http:///tor/extra/authority.z + (The extra-info document for this router.) + + Extra-info documents are uploaded to the same URLs as regular + router descriptors. + Migration: - For long/short descriptor approach: + For extra info approach: * First: - * Authorities should accept both, now, and silently drop short - descriptors. - * Routers should upload both once authorities accept them. - * There should be a "long descriptor" url named - /tor/server/fp-detailed/ and the current "normal" URL. - Authorities should serve long descriptors from both URLs. - There's no such thing as asking for a long descriptor by - its digest. - * Once tools that want long descriptors support fetching them from the - "long descriptor" URL: - * Have authorities remember short descriptors, and serve them from the - 'normal' URL. - These tools include: - lefkada's exit.py script. - tor26's noreply script and general directory cache. - https://nighteffect.us/tns/ for its graphs - and check with or-talk for the rest, once it's time. - - For bandwidth info approach: - * First: - * Rename it; it won't be just bandwidth forever. - * Authorities should accept bandwidth info + * Authorities should accept extra info, and support downloading it. * Routers should upload bandwidth info once authorities accept it. - * There should be a way to download bandwidth info + * Caches should support an option to download and cache it, once + authorities serve it. + * Tools should be updated to use locally cached information. + These tools include: + lefkada's exit.py script. + tor26's noreply script and general directory cache. + https://nighteffect.us/tns/ for its graphs + and check with or-talk for the rest, once it's time. + * Once tools that want bandwidth info support fetching it: * Have routers stop including bandwidth info in their router descriptors. -- cgit v1.2.3-54-g00ecf