From 2f357f50a0775cc684169e83d21e8e87c97bfc90 Mon Sep 17 00:00:00 2001 From: Nick Mathewson Date: Sat, 14 Oct 2023 14:36:12 -0400 Subject: Rename all long files. --- spec/SUMMARY.md | 72 +-- spec/dir-spec/accepting-relay-documents.md | 52 ++ ...epting-server-descriptor-extra-info-document.md | 52 -- spec/dir-spec/consensus-formats.md | 706 +++++++++++++++++++++ ...ting-curve25519-public-key-to-ed25519-public.md | 77 --- spec/dir-spec/converting-to-ed25519.md | 77 +++ spec/dir-spec/uploading-relay-documents.md | 43 ++ ...ding-server-descriptors-extra-info-documents.md | 43 -- .../vote-consensus-status-document-formats.md | 706 --------------------- ...uit-creation-entry-guard-selection-1000-foot.md | 72 --- spec/guard-spec/guard-selection-intro.md | 72 +++ .../still-non-addressed-issues-sectiontodo.md | 29 - spec/guard-spec/todo.md | 29 + ...cting-route-manipulation-by-guard-nodes-path.md | 198 ------ spec/path-spec/detecting-route-manipulation.md | 198 ++++++ spec/path-spec/learning-timeouts.md | 300 +++++++++ ...n-to-give-up-timeout-on-circuit-construction.md | 300 --------- spec/pt-spec/configuration-environment.md | 258 ++++++++ .../example-client-pluggable-transport-session.md | 19 - spec/pt-spec/example-client-session.md | 19 + .../example-server-pluggable-transport-session.md | 18 - spec/pt-spec/example-server-session.md | 18 + spec/pt-spec/ipc.md | 303 +++++++++ spec/pt-spec/naming.md | 13 + spec/pt-spec/per-connection-args.md | 38 ++ ...le-transport-client-per-connection-arguments.md | 38 -- ...luggable-transport-configuration-environment.md | 258 -------- spec/pt-spec/pluggable-transport-naming.md | 13 - spec/pt-spec/pluggable-transport-shutdown.md | 32 - ...le-transport-to-parent-process-communication.md | 303 --------- spec/pt-spec/shutdown.md | 32 + spec/rend-spec-v3/client-authorization.md | 105 +++ ...deriving-blinded-keys-subcredentials-subcred.md | 418 ------------ spec/rend-spec-v3/deriving-keys.md | 418 ++++++++++++ .../encoding-onion-addresses-onionaddress.md | 27 - spec/rend-spec-v3/encoding-onion-addresses.md | 27 + .../encrypting-data-between-client-host.md | 12 - spec/rend-spec-v3/encrypting-user-data.md | 12 + spec/rend-spec-v3/fs-contents.md | 30 + ...rating-publishing-hidden-service-descriptors.md | 8 - ...ce-descriptors-encryption-format-hs-desc-enc.md | 456 ------------- ...service-descriptors-outer-wrapper-desc-outer.md | 74 --- ...n-service-directory-format-hidservdir-format.md | 30 - .../hidden-services-overview-preliminaries.md | 315 --------- spec/rend-spec-v3/hsdesc-encrypt.md | 456 +++++++++++++ spec/rend-spec-v3/hsdesc-outer.md | 74 +++ spec/rend-spec-v3/hsdesc.md | 8 + .../introduction-protocol-intro-protocol.md | 632 ------------------ spec/rend-spec-v3/introduction-protocol.md | 632 ++++++++++++++++++ spec/rend-spec-v3/keyblinding-scheme.md | 104 +++ ...ging-authorized-client-data-client-auth-mgmt.md | 105 --- .../numeric-values-reserved-this-document.md | 5 - spec/rend-spec-v3/overview.md | 315 +++++++++ ...ishing-shared-random-values-pub-sharedrandom.md | 49 -- ...ommendations-for-searching-for-vanity-onions.md | 45 -- spec/rend-spec-v3/reserved-numbers-empty.md | 5 + spec/rend-spec-v3/revision-counter-mgt.md | 81 +++ spec/rend-spec-v3/shared-random.md | 49 ++ .../signature-scheme-with-key-blinding-keyblind.md | 104 --- .../two-methods-for-managing-revision-counters.md | 81 --- spec/rend-spec-v3/vanity-onions.md | 45 ++ .../application-connections-stream-management.md | 3 - spec/tor-spec/channels.md | 231 +++++++ spec/tor-spec/connections.md | 231 ------- spec/tor-spec/handling-relay_early-cells.md | 20 - spec/tor-spec/handling-resource-exhaustion.md | 33 - spec/tor-spec/negotiating-channels.md | 410 ++++++++++++ .../negotiating-initializing-connections.md | 410 ------------ spec/tor-spec/opening-streams-transferring-data.md | 119 ---- spec/tor-spec/opening-streams.md | 119 ++++ spec/tor-spec/relay-early.md | 20 + spec/tor-spec/resource-exhaustion.md | 33 + spec/tor-spec/streams.md | 3 + 73 files changed, 5371 insertions(+), 5371 deletions(-) create mode 100644 spec/dir-spec/accepting-relay-documents.md delete mode 100644 spec/dir-spec/accepting-server-descriptor-extra-info-document.md create mode 100644 spec/dir-spec/consensus-formats.md delete mode 100644 spec/dir-spec/converting-curve25519-public-key-to-ed25519-public.md create mode 100644 spec/dir-spec/converting-to-ed25519.md create mode 100644 spec/dir-spec/uploading-relay-documents.md delete mode 100644 spec/dir-spec/uploading-server-descriptors-extra-info-documents.md delete mode 100644 spec/dir-spec/vote-consensus-status-document-formats.md delete mode 100644 spec/guard-spec/circuit-creation-entry-guard-selection-1000-foot.md create mode 100644 spec/guard-spec/guard-selection-intro.md delete mode 100644 spec/guard-spec/still-non-addressed-issues-sectiontodo.md create mode 100644 spec/guard-spec/todo.md delete mode 100644 spec/path-spec/detecting-route-manipulation-by-guard-nodes-path.md create mode 100644 spec/path-spec/detecting-route-manipulation.md create mode 100644 spec/path-spec/learning-timeouts.md delete mode 100644 spec/path-spec/learning-when-to-give-up-timeout-on-circuit-construction.md create mode 100644 spec/pt-spec/configuration-environment.md delete mode 100644 spec/pt-spec/example-client-pluggable-transport-session.md create mode 100644 spec/pt-spec/example-client-session.md delete mode 100644 spec/pt-spec/example-server-pluggable-transport-session.md create mode 100644 spec/pt-spec/example-server-session.md create mode 100644 spec/pt-spec/ipc.md create mode 100644 spec/pt-spec/naming.md create mode 100644 spec/pt-spec/per-connection-args.md delete mode 100644 spec/pt-spec/pluggable-transport-client-per-connection-arguments.md delete mode 100644 spec/pt-spec/pluggable-transport-configuration-environment.md delete mode 100644 spec/pt-spec/pluggable-transport-naming.md delete mode 100644 spec/pt-spec/pluggable-transport-shutdown.md delete mode 100644 spec/pt-spec/pluggable-transport-to-parent-process-communication.md create mode 100644 spec/pt-spec/shutdown.md create mode 100644 spec/rend-spec-v3/client-authorization.md delete mode 100644 spec/rend-spec-v3/deriving-blinded-keys-subcredentials-subcred.md create mode 100644 spec/rend-spec-v3/deriving-keys.md delete mode 100644 spec/rend-spec-v3/encoding-onion-addresses-onionaddress.md create mode 100644 spec/rend-spec-v3/encoding-onion-addresses.md delete mode 100644 spec/rend-spec-v3/encrypting-data-between-client-host.md create mode 100644 spec/rend-spec-v3/encrypting-user-data.md create mode 100644 spec/rend-spec-v3/fs-contents.md delete mode 100644 spec/rend-spec-v3/generating-publishing-hidden-service-descriptors.md delete mode 100644 spec/rend-spec-v3/hidden-service-descriptors-encryption-format-hs-desc-enc.md delete mode 100644 spec/rend-spec-v3/hidden-service-descriptors-outer-wrapper-desc-outer.md delete mode 100644 spec/rend-spec-v3/hidden-service-directory-format-hidservdir-format.md delete mode 100644 spec/rend-spec-v3/hidden-services-overview-preliminaries.md create mode 100644 spec/rend-spec-v3/hsdesc-encrypt.md create mode 100644 spec/rend-spec-v3/hsdesc-outer.md create mode 100644 spec/rend-spec-v3/hsdesc.md delete mode 100644 spec/rend-spec-v3/introduction-protocol-intro-protocol.md create mode 100644 spec/rend-spec-v3/introduction-protocol.md create mode 100644 spec/rend-spec-v3/keyblinding-scheme.md delete mode 100644 spec/rend-spec-v3/managing-authorized-client-data-client-auth-mgmt.md delete mode 100644 spec/rend-spec-v3/numeric-values-reserved-this-document.md create mode 100644 spec/rend-spec-v3/overview.md delete mode 100644 spec/rend-spec-v3/publishing-shared-random-values-pub-sharedrandom.md delete mode 100644 spec/rend-spec-v3/recommendations-for-searching-for-vanity-onions.md create mode 100644 spec/rend-spec-v3/reserved-numbers-empty.md create mode 100644 spec/rend-spec-v3/revision-counter-mgt.md create mode 100644 spec/rend-spec-v3/shared-random.md delete mode 100644 spec/rend-spec-v3/signature-scheme-with-key-blinding-keyblind.md delete mode 100644 spec/rend-spec-v3/two-methods-for-managing-revision-counters.md create mode 100644 spec/rend-spec-v3/vanity-onions.md delete mode 100644 spec/tor-spec/application-connections-stream-management.md create mode 100644 spec/tor-spec/channels.md delete mode 100644 spec/tor-spec/connections.md delete mode 100644 spec/tor-spec/handling-relay_early-cells.md delete mode 100644 spec/tor-spec/handling-resource-exhaustion.md create mode 100644 spec/tor-spec/negotiating-channels.md delete mode 100644 spec/tor-spec/negotiating-initializing-connections.md delete mode 100644 spec/tor-spec/opening-streams-transferring-data.md create mode 100644 spec/tor-spec/opening-streams.md create mode 100644 spec/tor-spec/relay-early.md create mode 100644 spec/tor-spec/resource-exhaustion.md create mode 100644 spec/tor-spec/streams.md diff --git a/spec/SUMMARY.md b/spec/SUMMARY.md index 851f9a2..08ae2c3 100644 --- a/spec/SUMMARY.md +++ b/spec/SUMMARY.md @@ -7,39 +7,39 @@ - [`Tor Protocol Specification`](./tor-spec-intro.md) - [Preliminaries](./tor-spec/preliminaries.md) - [System overview](./tor-spec/system-overview.md) - - [Connections](./tor-spec/connections.md) + - [Connections](./tor-spec/channels.md) - [Cell Packet format](./tor-spec/cell-packet-format.md) - - [Negotiating and initializing connections](./tor-spec/negotiating-initializing-connections.md) + - [Negotiating and initializing connections](./tor-spec/negotiating-channels.md) - [Circuit management](./tor-spec/circuit-management.md) - [CREATE and CREATED cells](./tor-spec/create-created-cells.md) - [Setting circuit keys](./tor-spec/setting-circuit-keys.md) - [Creating circuits](./tor-spec/creating-circuits.md) - [Tearing down circuits](./tor-spec/tearing-down-circuits.md) - [Routing relay cells](./tor-spec/routing-relay-cells.md) - - [Handling relay_early cells](./tor-spec/handling-relay_early-cells.md) - - [Application connections and stream management](./tor-spec/application-connections-stream-management.md) + - [Handling relay_early cells](./tor-spec/relay-early.md) + - [Application connections and stream management](./tor-spec/streams.md) - [Relay cells](./tor-spec/relay-cells.md) - - [Opening streams and transferring data](./tor-spec/opening-streams-transferring-data.md) + - [Opening streams and transferring data](./tor-spec/opening-streams.md) - [Closing streams](./tor-spec/closing-streams.md) - [Remote hostname lookup](./tor-spec/remote-hostname-lookup.md) - [Flow control](./tor-spec/flow-control.md) - - [Handling resource exhaustion](./tor-spec/handling-resource-exhaustion.md) + - [Handling resource exhaustion](./tor-spec/resource-exhaustion.md) - [Subprotocol versioning](./tor-spec/subprotocol-versioning.md) - [`Ed25519 certificates in Tor`](./cert-spec.md) - [`Tor directory protocol, version 3`](./dir-spec-intro.md) - [Scope and preliminaries](./dir-spec/scope-preliminaries.md) - [Outline](./dir-spec/outline.md) - [Router operation and formats](./dir-spec/router-operation-formats.md) - - [Uploading server descriptors and extra-info documents](./dir-spec/uploading-server-descriptors-extra-info-documents.md) + - [Uploading server descriptors and extra-info documents](./dir-spec/uploading-relay-documents.md) - [Server descriptor format](./dir-spec/server-descriptor-format.md) - [Extra-info document format](./dir-spec/extra-info-document-format.md) - [Nonterminals in server descriptors](./dir-spec/nonterminals-server-descriptors.md) - [Directory authority operation and formats](./dir-spec/directory-authority-operation-formats.md) - [Creating key certificates](./dir-spec/creating-key-certificates.md) - - [Accepting server descriptor and extra-info document uploads](./dir-spec/accepting-server-descriptor-extra-info-document.md) + - [Accepting server descriptor and extra-info document uploads](./dir-spec/accepting-relay-documents.md) - [Computing microdescriptors](./dir-spec/computing-microdescriptors.md) - [Exchanging votes](./dir-spec/exchanging-votes.md) - - [Vote and consensus status document formats](./dir-spec/vote-consensus-status-document-formats.md) + - [Vote and consensus status document formats](./dir-spec/consensus-formats.md) - [Assigning flags in a vote](./dir-spec/assigning-flags-vote.md) - [Serving bandwidth list files](./dir-spec/serving-bandwidth-list-files.md) - [Directory cache operation](./dir-spec/directory-cache-operation.md) @@ -47,7 +47,7 @@ - [Standards compliance](./dir-spec/standards-compliance.md) - [Consensus-negotiation timeline.](./dir-spec/consensus-negotiation-timeline.md) - [General-use HTTP URLs](./dir-spec/general-use-http-urls.md) - - [Converting a curve25519 public key to an ed25519 public key](./dir-spec/converting-curve25519-public-key-to-ed25519-public.md) + - [Converting a curve25519 public key to an ed25519 public key](./dir-spec/converting-to-ed25519.md) - [Inferring missing proto lines.](./dir-spec/inferring-missing-proto-lines.md) - [Limited ed diff format](./dir-spec/limited-ed-diff-format.md) - [`Tor Shared Random Subsystem Specification`](./srv-spec-intro.md) @@ -64,21 +64,21 @@ - [When we build](./path-spec/when-we-build.md) - [Path selection and constraints](./path-spec/path-selection-constraints.md) - [Cannibalizing circuits](./path-spec/cannibalizing-circuits.md) - - [Learning when to give up ("timeout") on circuit construction](./path-spec/learning-when-to-give-up-timeout-on-circuit-construction.md) + - [Learning when to give up ("timeout") on circuit construction](./path-spec/learning-timeouts.md) - [Handling failure](./path-spec/handling-failure.md) - [Attaching streams to circuits](./path-spec/attaching-streams-to-circuits.md) - [Hidden-service related circuits](./path-spec/hidden-service-related-circuits.md) - [Guard nodes](./path-spec/guard-nodes.md) - [Server descriptor purposes](./path-spec/server-descriptor-purposes.md) - - [Detecting route manipulation by Guard nodes (Path Bias)](./path-spec/detecting-route-manipulation-by-guard-nodes-path.md) + - [Detecting route manipulation by Guard nodes (Path Bias)](./path-spec/detecting-route-manipulation.md) - [Old notes](./path-spec/old-notes.md) - [`Tor Guard Specification`](./guard-spec-intro.md) - [Introduction and motivation](./guard-spec/introduction-motivation.md) - [State instances](./guard-spec/state-instances.md) - - [Circuit Creation, Entry Guard Selection (1000 foot view)](./guard-spec/circuit-creation-entry-guard-selection-1000-foot.md) + - [Circuit Creation, Entry Guard Selection (1000 foot view)](./guard-spec/guard-selection-intro.md) - [The algorithm.](./guard-spec/algorithm.md) - [Appendices](./guard-spec/appendices.md) - - [Still non-addressed issues \[Section:TODO\]](./guard-spec/still-non-addressed-issues-sectiontodo.md) + - [Still non-addressed issues \[Section:TODO\]](./guard-spec/todo.md) - [`Tor Padding Specification`](./padding-spec-intro.md) - [Overview](./padding-spec/overview.md) - [Connection-level padding](./padding-spec/connection-level-padding.md) @@ -94,26 +94,26 @@ # Onion services - [`Tor Rendezvous Specification - Version 3`](./rend-spec-v3-intro.md) - - [Hidden services: overview and preliminaries.](./rend-spec-v3/hidden-services-overview-preliminaries.md) + - [Hidden services: overview and preliminaries.](./rend-spec-v3/overview.md) - [Protocol overview](./rend-spec-v3/protocol-overview.md) - - [Generating and publishing hidden service descriptors \[HSDIR\]](./rend-spec-v3/generating-publishing-hidden-service-descriptors.md) - - [Deriving blinded keys and subcredentials \[SUBCRED\]](./rend-spec-v3/deriving-blinded-keys-subcredentials-subcred.md) - - [Publishing shared random values \[PUB-SHAREDRANDOM\]](./rend-spec-v3/publishing-shared-random-values-pub-sharedrandom.md) - - [Hidden service descriptors: outer wrapper \[DESC-OUTER\]](./rend-spec-v3/hidden-service-descriptors-outer-wrapper-desc-outer.md) - - [Hidden service descriptors: encryption format \[HS-DESC-ENC\]](./rend-spec-v3/hidden-service-descriptors-encryption-format-hs-desc-enc.md) - - [The introduction protocol \[INTRO-PROTOCOL\]](./rend-spec-v3/introduction-protocol-intro-protocol.md) + - [Generating and publishing hidden service descriptors \[HSDIR\]](./rend-spec-v3/hsdesc.md) + - [Deriving blinded keys and subcredentials \[SUBCRED\]](./rend-spec-v3/deriving-keys.md) + - [Publishing shared random values \[PUB-SHAREDRANDOM\]](./rend-spec-v3/shared-random.md) + - [Hidden service descriptors: outer wrapper \[DESC-OUTER\]](./rend-spec-v3/hsdesc-outer.md) + - [Hidden service descriptors: encryption format \[HS-DESC-ENC\]](./rend-spec-v3/hsdesc-encrypt.md) + - [The introduction protocol \[INTRO-PROTOCOL\]](./rend-spec-v3/introduction-protocol.md) - [The rendezvous protocol](./rend-spec-v3/rendezvous-protocol.md) - - [Encrypting data between client and host](./rend-spec-v3/encrypting-data-between-client-host.md) - - [Encoding onion addresses \[ONIONADDRESS\]](./rend-spec-v3/encoding-onion-addresses-onionaddress.md) + - [Encrypting data between client and host](./rend-spec-v3/encrypting-user-data.md) + - [Encoding onion addresses \[ONIONADDRESS\]](./rend-spec-v3/encoding-onion-addresses.md) - [Open Questions:](./rend-spec-v3/open-questions.md) - - [Appendix A: Signature scheme with key blinding \[KEYBLIND\]](./rend-spec-v3/signature-scheme-with-key-blinding-keyblind.md) + - [Appendix A: Signature scheme with key blinding \[KEYBLIND\]](./rend-spec-v3/keyblinding-scheme.md) - [Appendix B: Selecting nodes \[PICKNODES\]](./rend-spec-v3/selecting-nodes-picknodes.md) - - [Appendix C: Recommendations for searching for vanity .onions \[VANITY\]](./rend-spec-v3/recommendations-for-searching-for-vanity-onions.md) - - [Appendix D: Numeric values reserved in this document](./rend-spec-v3/numeric-values-reserved-this-document.md) + - [Appendix C: Recommendations for searching for vanity .onions \[VANITY\]](./rend-spec-v3/vanity-onions.md) + - [Appendix D: Numeric values reserved in this document](./rend-spec-v3/reserved-numbers-empty.md) - [Appendix E: Reserved numbers](./rend-spec-v3/reserved-numbers.md) - - [Appendix F: Hidden service directory format \[HIDSERVDIR-FORMAT\]](./rend-spec-v3/hidden-service-directory-format-hidservdir-format.md) - - [Appendix G: Managing authorized client data \[CLIENT-AUTH-MGMT\]](./rend-spec-v3/managing-authorized-client-data-client-auth-mgmt.md) - - [Appendix F: Two methods for managing revision counters.](./rend-spec-v3/two-methods-for-managing-revision-counters.md) + - [Appendix F: Hidden service directory format \[HIDSERVDIR-FORMAT\]](./rend-spec-v3/fs-contents.md) + - [Appendix G: Managing authorized client data \[CLIENT-AUTH-MGMT\]](./rend-spec-v3/client-authorization.md) + - [Appendix F: Two methods for managing revision counters.](./rend-spec-v3/revision-counter-mgt.md) - [Appendix G: Text vectors](./rend-spec-v3/text-vectors.md) # Anticensorship tools and protocols @@ -124,16 +124,16 @@ - [Introduction](./pt-spec/introduction.md) - [Architecture Overview](./pt-spec/architecture-overview.md) - [Specification](./pt-spec/specification.md) - - [Pluggable Transport Naming](./pt-spec/pluggable-transport-naming.md) - - [Pluggable Transport Configuration Environment Variables](./pt-spec/pluggable-transport-configuration-environment.md) - - [Pluggable Transport To Parent Process Communication](./pt-spec/pluggable-transport-to-parent-process-communication.md) - - [Pluggable Transport Shutdown](./pt-spec/pluggable-transport-shutdown.md) - - [Pluggable Transport Client Per-Connection Arguments](./pt-spec/pluggable-transport-client-per-connection-arguments.md) + - [Pluggable Transport Naming](./pt-spec/naming.md) + - [Pluggable Transport Configuration Environment Variables](./pt-spec/configuration-environment.md) + - [Pluggable Transport To Parent Process Communication](./pt-spec/ipc.md) + - [Pluggable Transport Shutdown](./pt-spec/shutdown.md) + - [Pluggable Transport Client Per-Connection Arguments](./pt-spec/per-connection-args.md) - [Anonymity Considerations](./pt-spec/anonymity-considerations.md) - [References](./pt-spec/references.md) - [Acknowledgments](./pt-spec/acknowledgments.md) - - [Appendix A: Example Client Pluggable Transport Session](./pt-spec/example-client-pluggable-transport-session.md) - - [Appendix B: Example Server Pluggable Transport Session](./pt-spec/example-server-pluggable-transport-session.md) + - [Appendix A: Example Client Pluggable Transport Session](./pt-spec/example-client-session.md) + - [Appendix B: Example Server Pluggable Transport Session](./pt-spec/example-server-session.md) - [`GetTor specification`](./gettor-spec.md) # For C Tor only diff --git a/spec/dir-spec/accepting-relay-documents.md b/spec/dir-spec/accepting-relay-documents.md new file mode 100644 index 0000000..b507b9c --- /dev/null +++ b/spec/dir-spec/accepting-relay-documents.md @@ -0,0 +1,52 @@ + + +## Accepting server descriptor and extra-info document uploads + +When a router posts a signed descriptor to a directory authority, the +authority first checks whether it is well-formed and correctly +self-signed. If it is, the authority next verifies that the nickname +in question is not already assigned to a router with a different +public key. +Finally, the authority MAY check that the router is not blacklisted +because of its key, IP, or another reason. + +An authority also keeps a record of all the Ed25519/RSA1024 +identity key pairs that it has seen before. It rejects any +descriptor that has a known Ed/RSA identity key that it has +already seen accompanied by a different RSA/Ed identity key +in an older descriptor. + +At a future date, authorities will begin rejecting all +descriptors whose RSA key was previously accompanied by an +Ed25519 key, if the descriptor does not list an Ed25519 key. + +At a future date, authorities will begin rejecting all descriptors +that do not list an Ed25519 key. + +If the descriptor passes these tests, and the authority does not already +have a descriptor for a router with this public key, it accepts the +descriptor and remembers it. + +If the authority _does_ have a descriptor with the same public key, the +newly uploaded descriptor is remembered if its publication time is more +recent than the most recent old descriptor for that router, and either: + +```text + - There are non-cosmetic differences between the old descriptor and the + new one. + - Enough time has passed between the descriptors' publication times. + (Currently, 2 hours.) +``` + +Differences between server descriptors are "non-cosmetic" if they would be +sufficient to force an upload as described in section 2.1 above. + +Note that the "cosmetic difference" test only applies to uploaded +descriptors, not to descriptors that the authority downloads from other +authorities. + +When a router posts a signed extra-info document to a directory authority, +the authority again checks it for well-formedness and correct signature, +and checks that its matches the extra-info-digest in some router +descriptor that it believes is currently useful. If so, it accepts it and +stores it and serves it as requested. If not, it drops it. diff --git a/spec/dir-spec/accepting-server-descriptor-extra-info-document.md b/spec/dir-spec/accepting-server-descriptor-extra-info-document.md deleted file mode 100644 index b507b9c..0000000 --- a/spec/dir-spec/accepting-server-descriptor-extra-info-document.md +++ /dev/null @@ -1,52 +0,0 @@ - - -## Accepting server descriptor and extra-info document uploads - -When a router posts a signed descriptor to a directory authority, the -authority first checks whether it is well-formed and correctly -self-signed. If it is, the authority next verifies that the nickname -in question is not already assigned to a router with a different -public key. -Finally, the authority MAY check that the router is not blacklisted -because of its key, IP, or another reason. - -An authority also keeps a record of all the Ed25519/RSA1024 -identity key pairs that it has seen before. It rejects any -descriptor that has a known Ed/RSA identity key that it has -already seen accompanied by a different RSA/Ed identity key -in an older descriptor. - -At a future date, authorities will begin rejecting all -descriptors whose RSA key was previously accompanied by an -Ed25519 key, if the descriptor does not list an Ed25519 key. - -At a future date, authorities will begin rejecting all descriptors -that do not list an Ed25519 key. - -If the descriptor passes these tests, and the authority does not already -have a descriptor for a router with this public key, it accepts the -descriptor and remembers it. - -If the authority _does_ have a descriptor with the same public key, the -newly uploaded descriptor is remembered if its publication time is more -recent than the most recent old descriptor for that router, and either: - -```text - - There are non-cosmetic differences between the old descriptor and the - new one. - - Enough time has passed between the descriptors' publication times. - (Currently, 2 hours.) -``` - -Differences between server descriptors are "non-cosmetic" if they would be -sufficient to force an upload as described in section 2.1 above. - -Note that the "cosmetic difference" test only applies to uploaded -descriptors, not to descriptors that the authority downloads from other -authorities. - -When a router posts a signed extra-info document to a directory authority, -the authority again checks it for well-formedness and correct signature, -and checks that its matches the extra-info-digest in some router -descriptor that it believes is currently useful. If so, it accepts it and -stores it and serves it as requested. If not, it drops it. diff --git a/spec/dir-spec/consensus-formats.md b/spec/dir-spec/consensus-formats.md new file mode 100644 index 0000000..6fe5637 --- /dev/null +++ b/spec/dir-spec/consensus-formats.md @@ -0,0 +1,706 @@ + + +### Vote and consensus status document formats + +Votes and consensuses are more strictly formatted than other documents +in this specification, since different authorities must be able to +generate exactly the same consensus given the same set of votes. + +The procedure for deciding when to generate vote and consensus status +documents are described in section 1.4 on the voting timeline. + +Status documents contain a preamble, an authority section, a list of +router status entries, and one or more footer signature, in that order. + +Unlike other formats described above, a SP in these documents must be a +single space character (hex 20). + +Some items appear only in votes, and some items appear only in +consensuses. Unless specified, items occur in both. + +The preamble contains the following items. They SHOULD occur in the +order given here: + +"network-status-version" SP version NL + +\[At start, exactly once.\] + +A document format version. For this specification, the version is +"3". + +"vote-status" SP type NL + +\[Exactly once.\] + +The status MUST be "vote" or "consensus", depending on the type of +the document. + +"consensus-methods" SP IntegerList NL + +\[At most once for votes; does not occur in consensuses.\] + +A space-separated list of supported methods for generating +consensuses from votes. See section 3.8.1 for details. Absence of +the line means that only method "1" is supported. + +"consensus-method" SP Integer NL + +\[At most once for consensuses; does not occur in votes.\] +\[No extra arguments\] + +See section 3.8.1 for details. + +(Only included when the vote is generated with consensus-method 2 or +later.) + +"published" SP YYYY-MM-DD SP HH:MM:SS NL + +\[Exactly once for votes; does not occur in consensuses.\] + +The publication time for this status document (if a vote). + +"valid-after" SP YYYY-MM-DD SP HH:MM:SS NL + +\[Exactly once.\] + +The start of the Interval for this vote. Before this time, the +consensus document produced from this vote is not officially in +use. + +(Note that because of propagation delays, clients and relays +may see consensus documents that are up to `DistSeconds` +earlier than this time, and should not warn about them.) + +See section 1.4 for voting timeline information. + +"fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL + +\[Exactly once.\] + +The time at which the next consensus should be produced; before this +time, there is no point in downloading another consensus, since there +won't be a new one. See section 1.4 for voting timeline information. + +"valid-until" SP YYYY-MM-DD SP HH:MM:SS NL + +\[Exactly once.\] + +The end of the Interval for this vote. After this time, all +clients should try to find a more recent consensus. See section 1.4 +for voting timeline information. + +In practice, clients continue to use the consensus for up to 24 hours +after it is no longer valid, if no more recent consensus can be +downloaded. + +"voting-delay" SP VoteSeconds SP DistSeconds NL + +\[Exactly once.\] + +VoteSeconds is the number of seconds that we will allow to collect +votes from all authorities; DistSeconds is the number of seconds +we'll allow to collect signatures from all authorities. See +section 1.4 for voting timeline information. + +"client-versions" SP VersionList NL + +\[At most once.\] + +A comma-separated list of recommended Tor versions for client +usage, in ascending order. The versions are given as defined by +version-spec.txt. If absent, no opinion is held about client +versions. + +"server-versions" SP VersionList NL + +\[At most once.\] + +A comma-separated list of recommended Tor versions for relay +usage, in ascending order. The versions are given as defined by +version-spec.txt. If absent, no opinion is held about server +versions. + +"package" SP PackageName SP Version SP URL SP DIGESTS NL + +\[Any number of times.\] + +For this element: + +```text + PACKAGENAME = NONSPACE + VERSION = NONSPACE + URL = NONSPACE + DIGESTS = DIGEST | DIGESTS SP DIGEST + DIGEST = DIGESTTYPE "=" DIGESTVAL + NONSPACE = one or more non-space printing characters + DIGESTVAL = DIGESTTYPE = one or more non-space printing characters + other than "=". +``` + +Indicates that a package called "package" of version VERSION may be +found at URL, and its digest as computed with DIGESTTYPE is equal to +DIGESTVAL. In consensuses, these lines are sorted lexically by +"PACKAGENAME VERSION" pairs, and DIGESTTYPES must appear in ascending +order. A consensus must not contain the same "PACKAGENAME VERSION" +more than once. If a vote contains the same "PACKAGENAME VERSION" +more than once, all but the last is ignored. + +Included in consensuses only for methods 19-33. Earlier methods +did not include this; method 34 removed it. + +"known-flags" SP FlagList NL + +\[Exactly once.\] + +A space-separated list of all of the flags that this document +might contain. A flag is "known" either because the authority +knows about them and might set them (if in a vote), or because +enough votes were counted for the consensus for an authoritative +opinion to have been formed about their status. + +"flag-thresholds" SP Thresholds NL + +\[At most once for votes; does not occur in consensuses.\] + +```text + A space-separated list of the internal performance thresholds + that the directory authority had at the moment it was forming + a vote. + + The metaformat is: + Thresholds = Threshold | Threshold SP Thresholds + Threshold = ThresholdKey '=' ThresholdVal + ThresholdKey = (KeywordChar | "_") + + ThresholdVal = [0-9]+("."[0-9]+)? "%"? + + Commonly used Thresholds at this point include: + + "stable-uptime" -- Uptime (in seconds) required for a relay + to be marked as stable. + + "stable-mtbf" -- MTBF (in seconds) required for a relay to be + marked as stable. + + "enough-mtbf" -- Whether we have measured enough MTBF to look + at stable-mtbf instead of stable-uptime. + + "fast-speed" -- Bandwidth (in bytes per second) required for + a relay to be marked as fast. + + "guard-wfu" -- WFU (in seconds) required for a relay to be + marked as guard. + + "guard-tk" -- Weighted Time Known (in seconds) required for a + relay to be marked as guard. + + "guard-bw-inc-exits" -- If exits can be guards, then all guards + must have a bandwidth this high. + + "guard-bw-exc-exits" -- If exits can't be guards, then all guards + must have a bandwidth this high. + + "ignoring-advertised-bws" -- 1 if we have enough measured bandwidths + that we'll ignore the advertised bandwidth + claims of routers without measured bandwidth. +``` + +"recommended-client-protocols" SP Entries NL +"recommended-relay-protocols" SP Entries NL +"required-client-protocols" SP Entries NL +"required-relay-protocols" SP Entries NL + +\[At most once for each.\] + +The "proto" element as specified in section 2.1.1. + +To vote on these entries, a protocol/version combination is included +only if it is listed by a majority of the voters. + +These lines should be voted on. A majority of votes is sufficient to +make a protocol un-supported. A supermajority of authorities (2/3) +are needed to make a protocol required. The required protocols +should not be torrc-configurable, but rather should be hardwired in +the Tor code. + +The tor-spec.txt section 9 details how a relay and a client should +behave when they encounter these lines in the consensus. + +"params" SP \[Parameters\] NL + +\[At most once\] + +Parameter ::= Keyword '=' Int32 +Int32 ::= A decimal integer between -2147483648 and 2147483647. +Parameters ::= Parameter | Parameters SP Parameter + +The parameters list, if present, contains a space-separated list of +case-sensitive key-value pairs, sorted in lexical order by their +keyword (as ASCII byte strings). Each parameter has its own meaning. + +(Only included when the vote is generated with consensus-method 7 or +later.) + +See param-spec.txt for a list of parameters and their meanings. + +"shared-rand-previous-value" SP NumReveals SP Value NL + +\[At most once\] + +NumReveals ::= An integer greater or equal to 0. +Value ::= Base64-encoded-data + +The shared_random_value that was generated during the second-to-last +shared randomness protocol run. For example, if this document was +created on the 5th of November, this field carries the shared random +value generated during the protocol run of the 3rd of November. + +See section \[SRCALC\] of srv-spec.txt for instructions on how to compute +this value, and see section \[CONS\] for why we include old shared random +values in votes and consensus. + +Value is the actual shared random value encoded in base64. It will +be exactly 256 bits long. NumReveals is the number of commits used +to generate this SRV. + +"shared-rand-current-value" SP NumReveals SP Value NL + +\[At most once\] + +NumReveals ::= An integer greater or equal to 0. +Value ::= Base64-encoded-data + +The shared_random_value that was generated during the latest shared +randomness protocol run. For example, if this document was created on +the 5th of November, this field carries the shared random value +generated during the protocol run of the 4th of November + +See section \[SRCALC\] of srv-spec.txt for instructions on how to compute +this value given the active commits. + +Value is the actual shared random value encoded in base64. It will +be exactly 256 bits long. NumReveals is the number of commits used to +generate this SRV. + +"bandwidth-file-headers" SP KeyValues NL + +\[At most once for votes; does not occur in consensuses.\] + +KeyValues ::= "" | KeyValue | KeyValues SP KeyValue +KeyValue ::= Keyword '=' Value +Value ::= ArgumentCharValue+ +ArgumentCharValue ::= any printing ASCII character except NL and SP. + +The headers from the bandwidth file used to generate this vote. +The bandwidth file headers are described in bandwidth-file-spec.txt. + +If an authority is not configured with a V3BandwidthsFile, this line +SHOULD NOT appear in its vote. + +If an authority is configured with a V3BandwidthsFile, but parsing +fails, this line SHOULD appear in its vote, but without any headers. + +First-appeared: Tor 0.3.5.1-alpha. + +"bandwidth-file-digest" 1\*(SP algorithm "=" digest) NL + +\[At most once for votes; does not occur in consensuses.\] + +A digest of the bandwidth file used to generate this vote. +"algorithm" is the name of the hash algorithm producing "digest", +which can be "sha256" or another algorithm. "digest" is the +base64 encoding of the hash of the bandwidth file, with trailing =s +omitted. + +If an authority is not configured with a V3BandwidthsFile, this line +SHOULD NOT appear in its vote. + +If an authority is configured with a V3BandwidthsFile, but parsing +fails, this line SHOULD appear in its vote, with the digest(s) of the +unparseable file. + +First-appeared: Tor 0.4.0.4-alpha + +The authority section of a vote contains the following items, followed +in turn by the authority's current key certificate: + +```text + "dir-source" SP nickname SP identity SP address SP IP SP dirport SP + orport NL + + [Exactly once, at start] +``` + +Describes this authority. The nickname is a convenient identifier +for the authority. The identity is an uppercase hex fingerprint of +the authority's current (v3 authority) identity key. The address is +the server's hostname. The IP is the server's current IP address, +and dirport is its current directory port. The orport is the +port at that address where the authority listens for OR +connections. + +"contact" SP string NL + +\[Exactly once\] + +An arbitrary string describing how to contact the directory +server's administrator. Administrators should include at least an +email address and a PGP fingerprint. + +"legacy-dir-key" SP FINGERPRINT NL + +\[At most once\] + +Lists a fingerprint for an obsolete _identity_ key still used +by this authority to keep older clients working. This option +is used to keep key around for a little while in case the +authorities need to migrate many identity keys at once. +(Generally, this would only happen because of a security +vulnerability that affected multiple authorities, like the +Debian OpenSSL RNG bug of May 2008.) + +"shared-rand-participate" NL + +\[At most once\] + +Denotes that the directory authority supports and can participate in the +shared random protocol. + +"shared-rand-commit" SP Version SP AlgName SP Identity SP Commit \[SP Reveal\] NL + +\[Any number of times\] + +Version ::= An integer greater or equal to 0. +AlgName ::= 1\*(ALPHA / DIGIT / "\_" / "-") +Identity ::= 40\* HEXDIG +Commit ::= Base64-encoded-data +Reveal ::= Base64-encoded-data + +Denotes a directory authority commit for the shared randomness +protocol, containing the commitment value and potentially also the +reveal value. See sections \[COMMITREVEAL\] and \[VALIDATEVALUES\] of +srv-spec.txt on how to generate and validate these values. + +Version is the current shared randomness protocol version. AlgName is +the hash algorithm that is used (e.g. "sha3-256") and Identity is the +authority's SHA1 v3 identity fingerprint. Commit is the encoded +commitment value in base64. Reveal is optional and if it's set, it +contains the reveal value in base64. + +If a vote contains multiple commits from the same authority, the +receiver MUST only consider the first commit listed. + +"shared-rand-previous-value" SP NumReveals SP Value NL + +\[At most once\] + +See shared-rand-previous-value description above. + +"shared-rand-current-value" SP NumReveals SP Value NL + +\[At most once\] + +See shared-rand-current-value description above. + +The authority section of a consensus contains groups of the following items, +in the order given, with one group for each authority that contributed to +the consensus, with groups sorted by authority identity digest: + +```text + "dir-source" SP nickname SP identity SP address SP IP SP dirport SP + orport NL + + [Exactly once, at start] + + As in the authority section of a vote. + + "contact" SP string NL + + [Exactly once.] + + As in the authority section of a vote. + + "vote-digest" SP digest NL + + [Exactly once.] +``` + +A digest of the vote from the authority that contributed to this +consensus, as signed (that is, not including the signature). +(Hex, upper-case.) + +For each "legacy-dir-key" in the vote, there is an additional "dir-source" +line containing that legacy key's fingerprint, the authority's nickname +with "-legacy" appended, and all other fields as in the main "dir-source" +line for that authority. These "dir-source" lines do not have +corresponding "contact" or "vote-digest" entries. + +Each router status entry contains the following items. Router status +entries are sorted in ascending order by identity digest. + +```text + "r" SP nickname SP identity SP digest SP publication SP IP SP ORPort + SP DirPort NL + + [At start, exactly once.] +``` + +"Nickname" is the OR's nickname. "Identity" is a hash of its +identity key, encoded in base64, with trailing equals sign(s) +removed. "Digest" is a hash of its most recent descriptor as +signed (that is, not including the signature) by the RSA identity +key (see section 1.3.), encoded in base64. + +"Publication" was once the publication time of the router's most +recent descriptor, in the form YYYY-MM-DD HH:MM:SS, in UTC. Now +it is only used in votes, and may be set to a fixed value in +consensus documents. Implementations SHOULD ignore this value +in non-vote documents. + +"IP" is its current IP address; ORPort is its current OR port, +"DirPort" is its current directory port, or "0" for "none". + +"a" SP address ":" port NL + +\[Any number\] + +The first advertised IPv6 address for the OR, if it is reachable. + +Present only if the OR advertises at least one IPv6 address, and the +authority believes that the first advertised address is reachable. +Any other IPv4 or IPv6 addresses should be ignored. + +Address and port are as for "or-address" as specified in +section 2.1.1. + +(Only included when the vote or consensus is generated with +consensus-method 14 or later.) + +"s" SP Flags NL + +\[Exactly once.\] + +A series of space-separated status flags, in lexical order (as ASCII +byte strings). Currently documented flags are: + +```text + "Authority" if the router is a directory authority. + "BadExit" if the router is believed to be useless as an exit node + (because its ISP censors it, because it is behind a restrictive + proxy, or for some similar reason). + "Exit" if the router is more useful for building + general-purpose exit circuits than for relay circuits. The + path building algorithm uses this flag; see path-spec.txt. + "Fast" if the router is suitable for high-bandwidth circuits. + "Guard" if the router is suitable for use as an entry guard. + "HSDir" if the router is considered a v2 hidden service directory. + "MiddleOnly" if the router is considered unsuitable for + usage other than as a middle relay. Clients do not need + to handle this option, since when it is present, the authorities + will automatically vote against flags that would make the router + usable in other positions. (Since 0.4.7.2-alpha.) + "NoEdConsensus" if any Ed25519 key in the router's descriptor or + microdescriptor does not reflect authority consensus. + "Stable" if the router is suitable for long-lived circuits. + "StaleDesc" if the router should upload a new descriptor because + the old one is too old. + "Running" if the router is currently usable over all its published + ORPorts. (Authorities ignore IPv6 ORPorts unless configured to + check IPv6 reachability.) Relays without this flag are omitted + from the consensus, and current clients (since 0.2.9.4-alpha) + assume that every listed relay has this flag. + "Valid" if the router has been 'validated'. Clients before + 0.2.9.4-alpha would not use routers without this flag by + default. Currently, relays without this flag are omitted + from the consensus, and current (post-0.2.9.4-alpha) clients + assume that every listed relay has this flag. + "V2Dir" if the router implements the v2 directory protocol or + higher. + + "v" SP version NL + + [At most once.] +``` + +The version of the Tor protocol that this relay is running. If +the value begins with "Tor" SP, the rest of the string is a Tor +version number, and the protocol is "The Tor protocol as supported +by the given version of Tor." Otherwise, if the value begins with +some other string, Tor has upgraded to a more sophisticated +protocol versioning system, and the protocol is "a version of the +Tor protocol more recent than any we recognize." + +Directory authorities SHOULD omit version strings they receive from +descriptors if they would cause "v" lines to be over 128 characters +long. + +"pr" SP Entries NL + +\[At most once.\] + +The "proto" family element as specified in section 2.1.1. + +During voting, authorities copy these lines immediately below the "v" +lines. When a descriptor does not contain a "proto" entry, the +authorities should reconstruct it using the approach described below +in section D. They are included in the consensus using the same rules +as currently used for "v" lines, if a sufficiently late consensus +method is in use. + +"w" SP "Bandwidth=" INT \[SP "Measured=" INT\] \[SP "Unmeasured=1"\] NL + +\[At most once.\] + +An estimate of the bandwidth of this relay, in an arbitrary +unit (currently kilobytes per second). Used to weight router +selection. See section 3.4.2 for details on how the value of +Bandwidth is determined in a consensus. + +Additionally, the Measured= keyword is present in votes by +participating bandwidth measurement authorities to indicate +a measured bandwidth currently produced by measuring stream +capacities. It does not occur in consensuses. + +'Bandwidth=' and 'Measured=' values must be between 0 and +2^32 - 1 inclusive. + +The "Unmeasured=1" value is included in consensuses generated +with method 17 or later when the 'Bandwidth=' value is not +based on a threshold of 3 or more measurements for this relay. + +Other weighting keywords may be added later. +Clients MUST ignore keywords they do not recognize. + +"p" SP ("accept" / "reject") SP PortList NL + +\[At most once.\] + +PortList = PortOrRange +PortList = PortList "," PortOrRange +PortOrRange = INT "-" INT / INT + +A list of those ports that this router supports (if 'accept') +or does not support (if 'reject') for exit to "most +addresses". + +"m" SP methods 1\*(SP algorithm "=" digest) NL + +\[Any number, only in votes.\] + +Microdescriptor hashes for all consensus methods that an authority +supports and that use the same microdescriptor format. "methods" +is a comma-separated list of the consensus methods that the +authority believes will produce "digest". "algorithm" is the name +of the hash algorithm producing "digest", which can be "sha256" or +something else, depending on the consensus "methods" supporting +this algorithm. "digest" is the base64 encoding of the hash of +the router's microdescriptor with trailing =s omitted. + +```text + "id" SP "ed25519" SP ed25519-identity NL + "id" SP "ed25519" SP "none" NL + [vote only, at most once] + + "stats" SP [KeyValues] NL + + [At most once. Vote only] +``` + +KeyValue ::= Keyword '=' Number +Number ::= \[0-9\]+("."\[0-9\]+)? +KeyValues ::= KeyValue | KeyValues SP KeyValue + +Line containing various statistics that an authority has computed for +this relay. Each stats is represented as a key + value. Reported keys +are: + +```text + "wfu" - Weighted Fractional Uptime + "tk" - Weighted Time Known + "mtbf" - Mean Time Between Failure (stability) + + (As of tor-0.4.6.1-alpha) +``` + +The footer section is delineated in all votes and consensuses supporting +consensus method 9 and above with the following: + +"directory-footer" NL +\[No extra arguments\] + +It contains two subsections, a bandwidths-weights line and a +directory-signature. (Prior to consensus method 9, footers only contained +directory-signatures without a 'directory-footer' line or +bandwidth-weights.) + +The bandwidths-weights line appears At Most Once for a consensus. It does +not appear in votes. + +"bandwidth-weights" \[SP Weights\] NL + +Weight ::= Keyword '=' Int32 +Int32 ::= A decimal integer between -2147483648 and 2147483647. +Weights ::= Weight | Weights SP Weight + +List of optional weights to apply to router bandwidths during path +selection. They are sorted in lexical order (as ASCII byte strings) and +values are divided by the consensus' "bwweightscale" param. Definition +of our known entries are... + +```text + Wgg - Weight for Guard-flagged nodes in the guard position + Wgm - Weight for non-flagged nodes in the guard Position + Wgd - Weight for Guard+Exit-flagged nodes in the guard Position + + Wmg - Weight for Guard-flagged nodes in the middle Position + Wmm - Weight for non-flagged nodes in the middle Position + Wme - Weight for Exit-flagged nodes in the middle Position + Wmd - Weight for Guard+Exit flagged nodes in the middle Position + + Weg - Weight for Guard flagged nodes in the exit Position + Wem - Weight for non-flagged nodes in the exit Position + Wee - Weight for Exit-flagged nodes in the exit Position + Wed - Weight for Guard+Exit-flagged nodes in the exit Position + + Wgb - Weight for BEGIN_DIR-supporting Guard-flagged nodes + Wmb - Weight for BEGIN_DIR-supporting non-flagged nodes + Web - Weight for BEGIN_DIR-supporting Exit-flagged nodes + Wdb - Weight for BEGIN_DIR-supporting Guard+Exit-flagged nodes + + Wbg - Weight for Guard flagged nodes for BEGIN_DIR requests + Wbm - Weight for non-flagged nodes for BEGIN_DIR requests + Wbe - Weight for Exit-flagged nodes for BEGIN_DIR requests + Wbd - Weight for Guard+Exit-flagged nodes for BEGIN_DIR requests + + These values are calculated as specified in section 3.8.3. +``` + +The signature contains the following item, which appears Exactly Once +for a vote, and At Least Once for a consensus. + +```text + "directory-signature" [SP Algorithm] SP identity SP signing-key-digest + NL Signature +``` + +This is a signature of the status document, with the initial item +"network-status-version", and the signature item +"directory-signature", using the signing key. (In this case, we take +the hash through the _space_ after directory-signature, not the +newline: this ensures that all authorities sign the same thing.) +"identity" is the hex-encoded digest of the authority identity key of +the signing authority, and "signing-key-digest" is the hex-encoded +digest of the current authority signing key of the signing authority. + +The Algorithm is one of "sha1" or "sha256" if it is present; +implementations MUST ignore directory-signature entries with an +unrecognized Algorithm. "sha1" is the default, if no Algorithm is +given. The algorithm describes how to compute the hash of the +document before signing it. + +"ns"-flavored consensus documents must contain only sha1 signatures. +Votes and microdescriptor documents may contain other signature +types. Note that only one signature from each authority should be +"counted" as meaning that the authority has signed the consensus. + +(Tor clients before 0.2.3.x did not understand the 'algorithm' +field.) diff --git a/spec/dir-spec/converting-curve25519-public-key-to-ed25519-public.md b/spec/dir-spec/converting-curve25519-public-key-to-ed25519-public.md deleted file mode 100644 index 57f9d1d..0000000 --- a/spec/dir-spec/converting-curve25519-public-key-to-ed25519-public.md +++ /dev/null @@ -1,77 +0,0 @@ - - -# Converting a curve25519 public key to an ed25519 public key - -Given an X25519 key, that is, an affine point (u,v) on the -Montgomery curve defined by - -bv^2 = u(u^2 + au +1) - -where - -```text - a = 486662 - b = 1 -``` - -and comprised of the compressed form (i.e. consisting of only the -u-coordinate), we can retrieve the y-coordinate of the affine point -(x,y) on the twisted Edwards form of the curve defined by - --x^2 + y^2 = 1 + d x^2 y^2 - -where - -d = - 121665/121666 - -by computing - -y = (u-1)/(u+1). - -and then we can apply the usual curve25519 twisted Edwards point -decompression algorithm to find _an_ x-coordinate of an affine -twisted Edwards point to check signatures with. Signing keys for -ed25519 are compressed curve points in twisted Edwards form (so a -y-coordinate and the sign of the x-coordinate), and X25519 keys are -compressed curve points in Montgomery form (i.e. a u-coordinate). - -However, note that compressed point in Montgomery form neglects to -encode what the sign of the corresponding twisted Edwards -x-coordinate would be. Thus, we need the sign of the x-coordinate -to do this operation; otherwise, we'll have two possible -x-coordinates that might have correspond to the ed25519 public key. - -To get the sign, the easiest way is to take the corresponding -private key, feed it to the ed25519 public key generation -algorithm, and see what the sign is. - -\[Recomputing the sign bit from the private key every time sounds -rather strange and inefficient to me… —isis\] - -Note that in addition to its coordinates, an expanded Ed25519 private key -also has a 32-byte random value, "prefix", used to compute internal `r` -values in the signature. For security, this prefix value should be -derived deterministically from the curve25519 key. The Tor -implementation derives it as `SHA512(private_key | STR)[0..32]`, where -STR is the nul-terminated string: - -"Derive high part of ed25519 key from curve25519 key\\0" - -On the client side, where there is no access to the curve25519 private -keys, one may use the curve25519 public key's Montgomery u-coordinate to -recover the Montgomery v-coordinate by computing the right-hand side of -the Montgomery curve equation: - -bv^2 = u(u^2 + au +1) - -where - -```text - a = 486662 - b = 1 -``` - -Then, knowing the intended sign of the Edwards x-coordinate, one -may recover said x-coordinate by computing: - -x = (u/v) * sqrt(-a - 2) diff --git a/spec/dir-spec/converting-to-ed25519.md b/spec/dir-spec/converting-to-ed25519.md new file mode 100644 index 0000000..57f9d1d --- /dev/null +++ b/spec/dir-spec/converting-to-ed25519.md @@ -0,0 +1,77 @@ + + +# Converting a curve25519 public key to an ed25519 public key + +Given an X25519 key, that is, an affine point (u,v) on the +Montgomery curve defined by + +bv^2 = u(u^2 + au +1) + +where + +```text + a = 486662 + b = 1 +``` + +and comprised of the compressed form (i.e. consisting of only the +u-coordinate), we can retrieve the y-coordinate of the affine point +(x,y) on the twisted Edwards form of the curve defined by + +-x^2 + y^2 = 1 + d x^2 y^2 + +where + +d = - 121665/121666 + +by computing + +y = (u-1)/(u+1). + +and then we can apply the usual curve25519 twisted Edwards point +decompression algorithm to find _an_ x-coordinate of an affine +twisted Edwards point to check signatures with. Signing keys for +ed25519 are compressed curve points in twisted Edwards form (so a +y-coordinate and the sign of the x-coordinate), and X25519 keys are +compressed curve points in Montgomery form (i.e. a u-coordinate). + +However, note that compressed point in Montgomery form neglects to +encode what the sign of the corresponding twisted Edwards +x-coordinate would be. Thus, we need the sign of the x-coordinate +to do this operation; otherwise, we'll have two possible +x-coordinates that might have correspond to the ed25519 public key. + +To get the sign, the easiest way is to take the corresponding +private key, feed it to the ed25519 public key generation +algorithm, and see what the sign is. + +\[Recomputing the sign bit from the private key every time sounds +rather strange and inefficient to me… —isis\] + +Note that in addition to its coordinates, an expanded Ed25519 private key +also has a 32-byte random value, "prefix", used to compute internal `r` +values in the signature. For security, this prefix value should be +derived deterministically from the curve25519 key. The Tor +implementation derives it as `SHA512(private_key | STR)[0..32]`, where +STR is the nul-terminated string: + +"Derive high part of ed25519 key from curve25519 key\\0" + +On the client side, where there is no access to the curve25519 private +keys, one may use the curve25519 public key's Montgomery u-coordinate to +recover the Montgomery v-coordinate by computing the right-hand side of +the Montgomery curve equation: + +bv^2 = u(u^2 + au +1) + +where + +```text + a = 486662 + b = 1 +``` + +Then, knowing the intended sign of the Edwards x-coordinate, one +may recover said x-coordinate by computing: + +x = (u/v) * sqrt(-a - 2) diff --git a/spec/dir-spec/uploading-relay-documents.md b/spec/dir-spec/uploading-relay-documents.md new file mode 100644 index 0000000..d354068 --- /dev/null +++ b/spec/dir-spec/uploading-relay-documents.md @@ -0,0 +1,43 @@ + + +## Uploading server descriptors and extra-info documents + +ORs SHOULD generate a new server descriptor and a new extra-info +document whenever any of the following events have occurred: + +```text + - A period of time (18 hrs by default) has passed since the last + time a descriptor was generated. + + - A descriptor field other than bandwidth or uptime has changed. + + - Its uptime is less than 24h and bandwidth has changed by a factor of 2 + from the last time a descriptor was generated, and at least a given + interval of time (3 hours by default) has passed since then. + + - Its uptime has been reset (by restarting). + + - It receives a networkstatus consensus in which it is not listed. + + - It receives a networkstatus consensus in which it is listed + with the StaleDesc flag. + + [XXX this list is incomplete; see router_differences_are_cosmetic() + in routerlist.c for others] +``` + +ORs SHOULD NOT publish a new server descriptor or extra-info document +if none of the above events have occurred and not much time has passed +(12 hours by default). + +Tor versions older than 0.3.5.1-alpha ignore uptime when checking for +bandwidth changes. + +After generating a descriptor, ORs upload them to every directory +authority they know, by posting them (in order) to the URL + +http:///tor/ + +Server descriptors may not exceed 20,000 bytes in length; extra-info +documents may not exceed 50,000 bytes in length. If they do, the +authorities SHOULD reject them. diff --git a/spec/dir-spec/uploading-server-descriptors-extra-info-documents.md b/spec/dir-spec/uploading-server-descriptors-extra-info-documents.md deleted file mode 100644 index d354068..0000000 --- a/spec/dir-spec/uploading-server-descriptors-extra-info-documents.md +++ /dev/null @@ -1,43 +0,0 @@ - - -## Uploading server descriptors and extra-info documents - -ORs SHOULD generate a new server descriptor and a new extra-info -document whenever any of the following events have occurred: - -```text - - A period of time (18 hrs by default) has passed since the last - time a descriptor was generated. - - - A descriptor field other than bandwidth or uptime has changed. - - - Its uptime is less than 24h and bandwidth has changed by a factor of 2 - from the last time a descriptor was generated, and at least a given - interval of time (3 hours by default) has passed since then. - - - Its uptime has been reset (by restarting). - - - It receives a networkstatus consensus in which it is not listed. - - - It receives a networkstatus consensus in which it is listed - with the StaleDesc flag. - - [XXX this list is incomplete; see router_differences_are_cosmetic() - in routerlist.c for others] -``` - -ORs SHOULD NOT publish a new server descriptor or extra-info document -if none of the above events have occurred and not much time has passed -(12 hours by default). - -Tor versions older than 0.3.5.1-alpha ignore uptime when checking for -bandwidth changes. - -After generating a descriptor, ORs upload them to every directory -authority they know, by posting them (in order) to the URL - -http:///tor/ - -Server descriptors may not exceed 20,000 bytes in length; extra-info -documents may not exceed 50,000 bytes in length. If they do, the -authorities SHOULD reject them. diff --git a/spec/dir-spec/vote-consensus-status-document-formats.md b/spec/dir-spec/vote-consensus-status-document-formats.md deleted file mode 100644 index 6fe5637..0000000 --- a/spec/dir-spec/vote-consensus-status-document-formats.md +++ /dev/null @@ -1,706 +0,0 @@ - - -### Vote and consensus status document formats - -Votes and consensuses are more strictly formatted than other documents -in this specification, since different authorities must be able to -generate exactly the same consensus given the same set of votes. - -The procedure for deciding when to generate vote and consensus status -documents are described in section 1.4 on the voting timeline. - -Status documents contain a preamble, an authority section, a list of -router status entries, and one or more footer signature, in that order. - -Unlike other formats described above, a SP in these documents must be a -single space character (hex 20). - -Some items appear only in votes, and some items appear only in -consensuses. Unless specified, items occur in both. - -The preamble contains the following items. They SHOULD occur in the -order given here: - -"network-status-version" SP version NL - -\[At start, exactly once.\] - -A document format version. For this specification, the version is -"3". - -"vote-status" SP type NL - -\[Exactly once.\] - -The status MUST be "vote" or "consensus", depending on the type of -the document. - -"consensus-methods" SP IntegerList NL - -\[At most once for votes; does not occur in consensuses.\] - -A space-separated list of supported methods for generating -consensuses from votes. See section 3.8.1 for details. Absence of -the line means that only method "1" is supported. - -"consensus-method" SP Integer NL - -\[At most once for consensuses; does not occur in votes.\] -\[No extra arguments\] - -See section 3.8.1 for details. - -(Only included when the vote is generated with consensus-method 2 or -later.) - -"published" SP YYYY-MM-DD SP HH:MM:SS NL - -\[Exactly once for votes; does not occur in consensuses.\] - -The publication time for this status document (if a vote). - -"valid-after" SP YYYY-MM-DD SP HH:MM:SS NL - -\[Exactly once.\] - -The start of the Interval for this vote. Before this time, the -consensus document produced from this vote is not officially in -use. - -(Note that because of propagation delays, clients and relays -may see consensus documents that are up to `DistSeconds` -earlier than this time, and should not warn about them.) - -See section 1.4 for voting timeline information. - -"fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL - -\[Exactly once.\] - -The time at which the next consensus should be produced; before this -time, there is no point in downloading another consensus, since there -won't be a new one. See section 1.4 for voting timeline information. - -"valid-until" SP YYYY-MM-DD SP HH:MM:SS NL - -\[Exactly once.\] - -The end of the Interval for this vote. After this time, all -clients should try to find a more recent consensus. See section 1.4 -for voting timeline information. - -In practice, clients continue to use the consensus for up to 24 hours -after it is no longer valid, if no more recent consensus can be -downloaded. - -"voting-delay" SP VoteSeconds SP DistSeconds NL - -\[Exactly once.\] - -VoteSeconds is the number of seconds that we will allow to collect -votes from all authorities; DistSeconds is the number of seconds -we'll allow to collect signatures from all authorities. See -section 1.4 for voting timeline information. - -"client-versions" SP VersionList NL - -\[At most once.\] - -A comma-separated list of recommended Tor versions for client -usage, in ascending order. The versions are given as defined by -version-spec.txt. If absent, no opinion is held about client -versions. - -"server-versions" SP VersionList NL - -\[At most once.\] - -A comma-separated list of recommended Tor versions for relay -usage, in ascending order. The versions are given as defined by -version-spec.txt. If absent, no opinion is held about server -versions. - -"package" SP PackageName SP Version SP URL SP DIGESTS NL - -\[Any number of times.\] - -For this element: - -```text - PACKAGENAME = NONSPACE - VERSION = NONSPACE - URL = NONSPACE - DIGESTS = DIGEST | DIGESTS SP DIGEST - DIGEST = DIGESTTYPE "=" DIGESTVAL - NONSPACE = one or more non-space printing characters - DIGESTVAL = DIGESTTYPE = one or more non-space printing characters - other than "=". -``` - -Indicates that a package called "package" of version VERSION may be -found at URL, and its digest as computed with DIGESTTYPE is equal to -DIGESTVAL. In consensuses, these lines are sorted lexically by -"PACKAGENAME VERSION" pairs, and DIGESTTYPES must appear in ascending -order. A consensus must not contain the same "PACKAGENAME VERSION" -more than once. If a vote contains the same "PACKAGENAME VERSION" -more than once, all but the last is ignored. - -Included in consensuses only for methods 19-33. Earlier methods -did not include this; method 34 removed it. - -"known-flags" SP FlagList NL - -\[Exactly once.\] - -A space-separated list of all of the flags that this document -might contain. A flag is "known" either because the authority -knows about them and might set them (if in a vote), or because -enough votes were counted for the consensus for an authoritative -opinion to have been formed about their status. - -"flag-thresholds" SP Thresholds NL - -\[At most once for votes; does not occur in consensuses.\] - -```text - A space-separated list of the internal performance thresholds - that the directory authority had at the moment it was forming - a vote. - - The metaformat is: - Thresholds = Threshold | Threshold SP Thresholds - Threshold = ThresholdKey '=' ThresholdVal - ThresholdKey = (KeywordChar | "_") + - ThresholdVal = [0-9]+("."[0-9]+)? "%"? - - Commonly used Thresholds at this point include: - - "stable-uptime" -- Uptime (in seconds) required for a relay - to be marked as stable. - - "stable-mtbf" -- MTBF (in seconds) required for a relay to be - marked as stable. - - "enough-mtbf" -- Whether we have measured enough MTBF to look - at stable-mtbf instead of stable-uptime. - - "fast-speed" -- Bandwidth (in bytes per second) required for - a relay to be marked as fast. - - "guard-wfu" -- WFU (in seconds) required for a relay to be - marked as guard. - - "guard-tk" -- Weighted Time Known (in seconds) required for a - relay to be marked as guard. - - "guard-bw-inc-exits" -- If exits can be guards, then all guards - must have a bandwidth this high. - - "guard-bw-exc-exits" -- If exits can't be guards, then all guards - must have a bandwidth this high. - - "ignoring-advertised-bws" -- 1 if we have enough measured bandwidths - that we'll ignore the advertised bandwidth - claims of routers without measured bandwidth. -``` - -"recommended-client-protocols" SP Entries NL -"recommended-relay-protocols" SP Entries NL -"required-client-protocols" SP Entries NL -"required-relay-protocols" SP Entries NL - -\[At most once for each.\] - -The "proto" element as specified in section 2.1.1. - -To vote on these entries, a protocol/version combination is included -only if it is listed by a majority of the voters. - -These lines should be voted on. A majority of votes is sufficient to -make a protocol un-supported. A supermajority of authorities (2/3) -are needed to make a protocol required. The required protocols -should not be torrc-configurable, but rather should be hardwired in -the Tor code. - -The tor-spec.txt section 9 details how a relay and a client should -behave when they encounter these lines in the consensus. - -"params" SP \[Parameters\] NL - -\[At most once\] - -Parameter ::= Keyword '=' Int32 -Int32 ::= A decimal integer between -2147483648 and 2147483647. -Parameters ::= Parameter | Parameters SP Parameter - -The parameters list, if present, contains a space-separated list of -case-sensitive key-value pairs, sorted in lexical order by their -keyword (as ASCII byte strings). Each parameter has its own meaning. - -(Only included when the vote is generated with consensus-method 7 or -later.) - -See param-spec.txt for a list of parameters and their meanings. - -"shared-rand-previous-value" SP NumReveals SP Value NL - -\[At most once\] - -NumReveals ::= An integer greater or equal to 0. -Value ::= Base64-encoded-data - -The shared_random_value that was generated during the second-to-last -shared randomness protocol run. For example, if this document was -created on the 5th of November, this field carries the shared random -value generated during the protocol run of the 3rd of November. - -See section \[SRCALC\] of srv-spec.txt for instructions on how to compute -this value, and see section \[CONS\] for why we include old shared random -values in votes and consensus. - -Value is the actual shared random value encoded in base64. It will -be exactly 256 bits long. NumReveals is the number of commits used -to generate this SRV. - -"shared-rand-current-value" SP NumReveals SP Value NL - -\[At most once\] - -NumReveals ::= An integer greater or equal to 0. -Value ::= Base64-encoded-data - -The shared_random_value that was generated during the latest shared -randomness protocol run. For example, if this document was created on -the 5th of November, this field carries the shared random value -generated during the protocol run of the 4th of November - -See section \[SRCALC\] of srv-spec.txt for instructions on how to compute -this value given the active commits. - -Value is the actual shared random value encoded in base64. It will -be exactly 256 bits long. NumReveals is the number of commits used to -generate this SRV. - -"bandwidth-file-headers" SP KeyValues NL - -\[At most once for votes; does not occur in consensuses.\] - -KeyValues ::= "" | KeyValue | KeyValues SP KeyValue -KeyValue ::= Keyword '=' Value -Value ::= ArgumentCharValue+ -ArgumentCharValue ::= any printing ASCII character except NL and SP. - -The headers from the bandwidth file used to generate this vote. -The bandwidth file headers are described in bandwidth-file-spec.txt. - -If an authority is not configured with a V3BandwidthsFile, this line -SHOULD NOT appear in its vote. - -If an authority is configured with a V3BandwidthsFile, but parsing -fails, this line SHOULD appear in its vote, but without any headers. - -First-appeared: Tor 0.3.5.1-alpha. - -"bandwidth-file-digest" 1\*(SP algorithm "=" digest) NL - -\[At most once for votes; does not occur in consensuses.\] - -A digest of the bandwidth file used to generate this vote. -"algorithm" is the name of the hash algorithm producing "digest", -which can be "sha256" or another algorithm. "digest" is the -base64 encoding of the hash of the bandwidth file, with trailing =s -omitted. - -If an authority is not configured with a V3BandwidthsFile, this line -SHOULD NOT appear in its vote. - -If an authority is configured with a V3BandwidthsFile, but parsing -fails, this line SHOULD appear in its vote, with the digest(s) of the -unparseable file. - -First-appeared: Tor 0.4.0.4-alpha - -The authority section of a vote contains the following items, followed -in turn by the authority's current key certificate: - -```text - "dir-source" SP nickname SP identity SP address SP IP SP dirport SP - orport NL - - [Exactly once, at start] -``` - -Describes this authority. The nickname is a convenient identifier -for the authority. The identity is an uppercase hex fingerprint of -the authority's current (v3 authority) identity key. The address is -the server's hostname. The IP is the server's current IP address, -and dirport is its current directory port. The orport is the -port at that address where the authority listens for OR -connections. - -"contact" SP string NL - -\[Exactly once\] - -An arbitrary string describing how to contact the directory -server's administrator. Administrators should include at least an -email address and a PGP fingerprint. - -"legacy-dir-key" SP FINGERPRINT NL - -\[At most once\] - -Lists a fingerprint for an obsolete _identity_ key still used -by this authority to keep older clients working. This option -is used to keep key around for a little while in case the -authorities need to migrate many identity keys at once. -(Generally, this would only happen because of a security -vulnerability that affected multiple authorities, like the -Debian OpenSSL RNG bug of May 2008.) - -"shared-rand-participate" NL - -\[At most once\] - -Denotes that the directory authority supports and can participate in the -shared random protocol. - -"shared-rand-commit" SP Version SP AlgName SP Identity SP Commit \[SP Reveal\] NL - -\[Any number of times\] - -Version ::= An integer greater or equal to 0. -AlgName ::= 1\*(ALPHA / DIGIT / "\_" / "-") -Identity ::= 40\* HEXDIG -Commit ::= Base64-encoded-data -Reveal ::= Base64-encoded-data - -Denotes a directory authority commit for the shared randomness -protocol, containing the commitment value and potentially also the -reveal value. See sections \[COMMITREVEAL\] and \[VALIDATEVALUES\] of -srv-spec.txt on how to generate and validate these values. - -Version is the current shared randomness protocol version. AlgName is -the hash algorithm that is used (e.g. "sha3-256") and Identity is the -authority's SHA1 v3 identity fingerprint. Commit is the encoded -commitment value in base64. Reveal is optional and if it's set, it -contains the reveal value in base64. - -If a vote contains multiple commits from the same authority, the -receiver MUST only consider the first commit listed. - -"shared-rand-previous-value" SP NumReveals SP Value NL - -\[At most once\] - -See shared-rand-previous-value description above. - -"shared-rand-current-value" SP NumReveals SP Value NL - -\[At most once\] - -See shared-rand-current-value description above. - -The authority section of a consensus contains groups of the following items, -in the order given, with one group for each authority that contributed to -the consensus, with groups sorted by authority identity digest: - -```text - "dir-source" SP nickname SP identity SP address SP IP SP dirport SP - orport NL - - [Exactly once, at start] - - As in the authority section of a vote. - - "contact" SP string NL - - [Exactly once.] - - As in the authority section of a vote. - - "vote-digest" SP digest NL - - [Exactly once.] -``` - -A digest of the vote from the authority that contributed to this -consensus, as signed (that is, not including the signature). -(Hex, upper-case.) - -For each "legacy-dir-key" in the vote, there is an additional "dir-source" -line containing that legacy key's fingerprint, the authority's nickname -with "-legacy" appended, and all other fields as in the main "dir-source" -line for that authority. These "dir-source" lines do not have -corresponding "contact" or "vote-digest" entries. - -Each router status entry contains the following items. Router status -entries are sorted in ascending order by identity digest. - -```text - "r" SP nickname SP identity SP digest SP publication SP IP SP ORPort - SP DirPort NL - - [At start, exactly once.] -``` - -"Nickname" is the OR's nickname. "Identity" is a hash of its -identity key, encoded in base64, with trailing equals sign(s) -removed. "Digest" is a hash of its most recent descriptor as -signed (that is, not including the signature) by the RSA identity -key (see section 1.3.), encoded in base64. - -"Publication" was once the publication time of the router's most -recent descriptor, in the form YYYY-MM-DD HH:MM:SS, in UTC. Now -it is only used in votes, and may be set to a fixed value in -consensus documents. Implementations SHOULD ignore this value -in non-vote documents. - -"IP" is its current IP address; ORPort is its current OR port, -"DirPort" is its current directory port, or "0" for "none". - -"a" SP address ":" port NL - -\[Any number\] - -The first advertised IPv6 address for the OR, if it is reachable. - -Present only if the OR advertises at least one IPv6 address, and the -authority believes that the first advertised address is reachable. -Any other IPv4 or IPv6 addresses should be ignored. - -Address and port are as for "or-address" as specified in -section 2.1.1. - -(Only included when the vote or consensus is generated with -consensus-method 14 or later.) - -"s" SP Flags NL - -\[Exactly once.\] - -A series of space-separated status flags, in lexical order (as ASCII -byte strings). Currently documented flags are: - -```text - "Authority" if the router is a directory authority. - "BadExit" if the router is believed to be useless as an exit node - (because its ISP censors it, because it is behind a restrictive - proxy, or for some similar reason). - "Exit" if the router is more useful for building - general-purpose exit circuits than for relay circuits. The - path building algorithm uses this flag; see path-spec.txt. - "Fast" if the router is suitable for high-bandwidth circuits. - "Guard" if the router is suitable for use as an entry guard. - "HSDir" if the router is considered a v2 hidden service directory. - "MiddleOnly" if the router is considered unsuitable for - usage other than as a middle relay. Clients do not need - to handle this option, since when it is present, the authorities - will automatically vote against flags that would make the router - usable in other positions. (Since 0.4.7.2-alpha.) - "NoEdConsensus" if any Ed25519 key in the router's descriptor or - microdescriptor does not reflect authority consensus. - "Stable" if the router is suitable for long-lived circuits. - "StaleDesc" if the router should upload a new descriptor because - the old one is too old. - "Running" if the router is currently usable over all its published - ORPorts. (Authorities ignore IPv6 ORPorts unless configured to - check IPv6 reachability.) Relays without this flag are omitted - from the consensus, and current clients (since 0.2.9.4-alpha) - assume that every listed relay has this flag. - "Valid" if the router has been 'validated'. Clients before - 0.2.9.4-alpha would not use routers without this flag by - default. Currently, relays without this flag are omitted - from the consensus, and current (post-0.2.9.4-alpha) clients - assume that every listed relay has this flag. - "V2Dir" if the router implements the v2 directory protocol or - higher. - - "v" SP version NL - - [At most once.] -``` - -The version of the Tor protocol that this relay is running. If -the value begins with "Tor" SP, the rest of the string is a Tor -version number, and the protocol is "The Tor protocol as supported -by the given version of Tor." Otherwise, if the value begins with -some other string, Tor has upgraded to a more sophisticated -protocol versioning system, and the protocol is "a version of the -Tor protocol more recent than any we recognize." - -Directory authorities SHOULD omit version strings they receive from -descriptors if they would cause "v" lines to be over 128 characters -long. - -"pr" SP Entries NL - -\[At most once.\] - -The "proto" family element as specified in section 2.1.1. - -During voting, authorities copy these lines immediately below the "v" -lines. When a descriptor does not contain a "proto" entry, the -authorities should reconstruct it using the approach described below -in section D. They are included in the consensus using the same rules -as currently used for "v" lines, if a sufficiently late consensus -method is in use. - -"w" SP "Bandwidth=" INT \[SP "Measured=" INT\] \[SP "Unmeasured=1"\] NL - -\[At most once.\] - -An estimate of the bandwidth of this relay, in an arbitrary -unit (currently kilobytes per second). Used to weight router -selection. See section 3.4.2 for details on how the value of -Bandwidth is determined in a consensus. - -Additionally, the Measured= keyword is present in votes by -participating bandwidth measurement authorities to indicate -a measured bandwidth currently produced by measuring stream -capacities. It does not occur in consensuses. - -'Bandwidth=' and 'Measured=' values must be between 0 and -2^32 - 1 inclusive. - -The "Unmeasured=1" value is included in consensuses generated -with method 17 or later when the 'Bandwidth=' value is not -based on a threshold of 3 or more measurements for this relay. - -Other weighting keywords may be added later. -Clients MUST ignore keywords they do not recognize. - -"p" SP ("accept" / "reject") SP PortList NL - -\[At most once.\] - -PortList = PortOrRange -PortList = PortList "," PortOrRange -PortOrRange = INT "-" INT / INT - -A list of those ports that this router supports (if 'accept') -or does not support (if 'reject') for exit to "most -addresses". - -"m" SP methods 1\*(SP algorithm "=" digest) NL - -\[Any number, only in votes.\] - -Microdescriptor hashes for all consensus methods that an authority -supports and that use the same microdescriptor format. "methods" -is a comma-separated list of the consensus methods that the -authority believes will produce "digest". "algorithm" is the name -of the hash algorithm producing "digest", which can be "sha256" or -something else, depending on the consensus "methods" supporting -this algorithm. "digest" is the base64 encoding of the hash of -the router's microdescriptor with trailing =s omitted. - -```text - "id" SP "ed25519" SP ed25519-identity NL - "id" SP "ed25519" SP "none" NL - [vote only, at most once] - - "stats" SP [KeyValues] NL - - [At most once. Vote only] -``` - -KeyValue ::= Keyword '=' Number -Number ::= \[0-9\]+("."\[0-9\]+)? -KeyValues ::= KeyValue | KeyValues SP KeyValue - -Line containing various statistics that an authority has computed for -this relay. Each stats is represented as a key + value. Reported keys -are: - -```text - "wfu" - Weighted Fractional Uptime - "tk" - Weighted Time Known - "mtbf" - Mean Time Between Failure (stability) - - (As of tor-0.4.6.1-alpha) -``` - -The footer section is delineated in all votes and consensuses supporting -consensus method 9 and above with the following: - -"directory-footer" NL -\[No extra arguments\] - -It contains two subsections, a bandwidths-weights line and a -directory-signature. (Prior to consensus method 9, footers only contained -directory-signatures without a 'directory-footer' line or -bandwidth-weights.) - -The bandwidths-weights line appears At Most Once for a consensus. It does -not appear in votes. - -"bandwidth-weights" \[SP Weights\] NL - -Weight ::= Keyword '=' Int32 -Int32 ::= A decimal integer between -2147483648 and 2147483647. -Weights ::= Weight | Weights SP Weight - -List of optional weights to apply to router bandwidths during path -selection. They are sorted in lexical order (as ASCII byte strings) and -values are divided by the consensus' "bwweightscale" param. Definition -of our known entries are... - -```text - Wgg - Weight for Guard-flagged nodes in the guard position - Wgm - Weight for non-flagged nodes in the guard Position - Wgd - Weight for Guard+Exit-flagged nodes in the guard Position - - Wmg - Weight for Guard-flagged nodes in the middle Position - Wmm - Weight for non-flagged nodes in the middle Position - Wme - Weight for Exit-flagged nodes in the middle Position - Wmd - Weight for Guard+Exit flagged nodes in the middle Position - - Weg - Weight for Guard flagged nodes in the exit Position - Wem - Weight for non-flagged nodes in the exit Position - Wee - Weight for Exit-flagged nodes in the exit Position - Wed - Weight for Guard+Exit-flagged nodes in the exit Position - - Wgb - Weight for BEGIN_DIR-supporting Guard-flagged nodes - Wmb - Weight for BEGIN_DIR-supporting non-flagged nodes - Web - Weight for BEGIN_DIR-supporting Exit-flagged nodes - Wdb - Weight for BEGIN_DIR-supporting Guard+Exit-flagged nodes - - Wbg - Weight for Guard flagged nodes for BEGIN_DIR requests - Wbm - Weight for non-flagged nodes for BEGIN_DIR requests - Wbe - Weight for Exit-flagged nodes for BEGIN_DIR requests - Wbd - Weight for Guard+Exit-flagged nodes for BEGIN_DIR requests - - These values are calculated as specified in section 3.8.3. -``` - -The signature contains the following item, which appears Exactly Once -for a vote, and At Least Once for a consensus. - -```text - "directory-signature" [SP Algorithm] SP identity SP signing-key-digest - NL Signature -``` - -This is a signature of the status document, with the initial item -"network-status-version", and the signature item -"directory-signature", using the signing key. (In this case, we take -the hash through the _space_ after directory-signature, not the -newline: this ensures that all authorities sign the same thing.) -"identity" is the hex-encoded digest of the authority identity key of -the signing authority, and "signing-key-digest" is the hex-encoded -digest of the current authority signing key of the signing authority. - -The Algorithm is one of "sha1" or "sha256" if it is present; -implementations MUST ignore directory-signature entries with an -unrecognized Algorithm. "sha1" is the default, if no Algorithm is -given. The algorithm describes how to compute the hash of the -document before signing it. - -"ns"-flavored consensus documents must contain only sha1 signatures. -Votes and microdescriptor documents may contain other signature -types. Note that only one signature from each authority should be -"counted" as meaning that the authority has signed the consensus. - -(Tor clients before 0.2.3.x did not understand the 'algorithm' -field.) diff --git a/spec/guard-spec/circuit-creation-entry-guard-selection-1000-foot.md b/spec/guard-spec/circuit-creation-entry-guard-selection-1000-foot.md deleted file mode 100644 index b31609f..0000000 --- a/spec/guard-spec/circuit-creation-entry-guard-selection-1000-foot.md +++ /dev/null @@ -1,72 +0,0 @@ - - -# Circuit Creation, Entry Guard Selection (1000 foot view) - -A circuit in Tor is a path through the network connecting a client to -its destination. At a high-level, a three-hop exit circuit will look -like this: - -Client \<-> Entry Guard \<-> Middle Node \<-> Exit Node \<-> Destination - -Entry guards are the only nodes which a client will connect to -directly. Exit relays are the nodes by which traffic exits the -Tor network in order to connect to an external destination. - -3.1 Path selection - -For any multi-hop circuit, at least one entry guard and middle node(s) are -required. An exit node is required if traffic will exit the Tor -network. Depending on its configuration, a relay listed in a -consensus could be used for any of these roles. However, this -specification defines how entry guards specifically should be selected and -managed, as opposed to middle or exit nodes. - -3.1.1 Managing entry guards - -At a high level, a relay listed in a consensus will move through the -following states in the process from initial selection to eventual -usage as an entry guard: - -```text - relays listed in consensus - | - sampled - | | - confirmed filtered - | | | - primary usable_filtered -``` - -Relays listed in the latest consensus can be sampled for guard usage -if they have the "Guard" flag. Sampling is random but weighted by -a measured bandwidth multiplied by bandwidth-weights (Wgg if guard only, -Wgd if guard+exit flagged). - -Once a path is built and a circuit established using this guard, it -is marked as confirmed. Until this point, guards are first sampled -and then filtered based on information such as our current -configuration (see SAMPLED and FILTERED sections) and later marked as -usable_filtered if the guard is not primary but can be reached. - -It is always preferable to use a primary guard when building a new -circuit in order to reduce guard churn; only on failure to connect to -existing primary guards will new guards be used. - -3.1.2 Middle and exit node selection - -Middle nodes are selected at random from relays listed in the latest -consensus, weighted by bandwidth and bandwidth-weights. Exit nodes are -chosen similarly but restricted to relays with a sufficiently permissive -exit policy. - -3.2 Circuit Building - -Once a path is chosen, Tor will use this path to build a new circuit. - -If the circuit is built successfully, Tor will either use it -immediately, or Tor will wait for a circuit with a more preferred -guard if there's a good chance that it will be able to make one. - -If the circuit fails in a way that makes us conclude that a guard -is not reachable, the guard is marked as unreachable, the circuit is -closed, and waiting circuits are updated. diff --git a/spec/guard-spec/guard-selection-intro.md b/spec/guard-spec/guard-selection-intro.md new file mode 100644 index 0000000..b31609f --- /dev/null +++ b/spec/guard-spec/guard-selection-intro.md @@ -0,0 +1,72 @@ + + +# Circuit Creation, Entry Guard Selection (1000 foot view) + +A circuit in Tor is a path through the network connecting a client to +its destination. At a high-level, a three-hop exit circuit will look +like this: + +Client \<-> Entry Guard \<-> Middle Node \<-> Exit Node \<-> Destination + +Entry guards are the only nodes which a client will connect to +directly. Exit relays are the nodes by which traffic exits the +Tor network in order to connect to an external destination. + +3.1 Path selection + +For any multi-hop circuit, at least one entry guard and middle node(s) are +required. An exit node is required if traffic will exit the Tor +network. Depending on its configuration, a relay listed in a +consensus could be used for any of these roles. However, this +specification defines how entry guards specifically should be selected and +managed, as opposed to middle or exit nodes. + +3.1.1 Managing entry guards + +At a high level, a relay listed in a consensus will move through the +following states in the process from initial selection to eventual +usage as an entry guard: + +```text + relays listed in consensus + | + sampled + | | + confirmed filtered + | | | + primary usable_filtered +``` + +Relays listed in the latest consensus can be sampled for guard usage +if they have the "Guard" flag. Sampling is random but weighted by +a measured bandwidth multiplied by bandwidth-weights (Wgg if guard only, +Wgd if guard+exit flagged). + +Once a path is built and a circuit established using this guard, it +is marked as confirmed. Until this point, guards are first sampled +and then filtered based on information such as our current +configuration (see SAMPLED and FILTERED sections) and later marked as +usable_filtered if the guard is not primary but can be reached. + +It is always preferable to use a primary guard when building a new +circuit in order to reduce guard churn; only on failure to connect to +existing primary guards will new guards be used. + +3.1.2 Middle and exit node selection + +Middle nodes are selected at random from relays listed in the latest +consensus, weighted by bandwidth and bandwidth-weights. Exit nodes are +chosen similarly but restricted to relays with a sufficiently permissive +exit policy. + +3.2 Circuit Building + +Once a path is chosen, Tor will use this path to build a new circuit. + +If the circuit is built successfully, Tor will either use it +immediately, or Tor will wait for a circuit with a more preferred +guard if there's a good chance that it will be able to make one. + +If the circuit fails in a way that makes us conclude that a guard +is not reachable, the guard is marked as unreachable, the circuit is +closed, and waiting circuits are updated. diff --git a/spec/guard-spec/still-non-addressed-issues-sectiontodo.md b/spec/guard-spec/still-non-addressed-issues-sectiontodo.md deleted file mode 100644 index 38b0be4..0000000 --- a/spec/guard-spec/still-non-addressed-issues-sectiontodo.md +++ /dev/null @@ -1,29 +0,0 @@ - - -# Still non-addressed issues \[Section:TODO\] - -Simulate to answer: Will this work in a dystopic world? - -Simulate actual behavior. - -For all lifetimes: instead of storing the "this began at" time, -store the "remove this at" time, slightly randomized. - -Clarify that when you get a `` circuit, you might need to -relaunch circuits through that same guard immediately, if they -are circuits that have to be independent. - -Fix all items marked XX or TODO. - -"Directory guards" -- do they matter? - -```text - Suggestion: require that all guards support downloads via BEGINDIR. - We don't need to worry about directory guards for relays, since we - aren't trying to prevent relay enumeration. - - IP version preferences via ClientPreferIPv6ORPort - - Suggestion: Treat it as a preference when adding to - {CONFIRMED_GUARDS}, but not otherwise. -``` diff --git a/spec/guard-spec/todo.md b/spec/guard-spec/todo.md new file mode 100644 index 0000000..38b0be4 --- /dev/null +++ b/spec/guard-spec/todo.md @@ -0,0 +1,29 @@ + + +# Still non-addressed issues \[Section:TODO\] + +Simulate to answer: Will this work in a dystopic world? + +Simulate actual behavior. + +For all lifetimes: instead of storing the "this began at" time, +store the "remove this at" time, slightly randomized. + +Clarify that when you get a `` circuit, you might need to +relaunch circuits through that same guard immediately, if they +are circuits that have to be independent. + +Fix all items marked XX or TODO. + +"Directory guards" -- do they matter? + +```text + Suggestion: require that all guards support downloads via BEGINDIR. + We don't need to worry about directory guards for relays, since we + aren't trying to prevent relay enumeration. + + IP version preferences via ClientPreferIPv6ORPort + + Suggestion: Treat it as a preference when adding to + {CONFIRMED_GUARDS}, but not otherwise. +``` diff --git a/spec/path-spec/detecting-route-manipulation-by-guard-nodes-path.md b/spec/path-spec/detecting-route-manipulation-by-guard-nodes-path.md deleted file mode 100644 index c7ff50c..0000000 --- a/spec/path-spec/detecting-route-manipulation-by-guard-nodes-path.md +++ /dev/null @@ -1,198 +0,0 @@ - - -# Detecting route manipulation by Guard nodes (Path Bias) - -The Path Bias defense is designed to defend against a type of route -capture where malicious Guard nodes deliberately fail or choke circuits -that extend to non-colluding Exit nodes to maximize their network -utilization in favor of carrying only compromised traffic. - -In the extreme, the attack allows an adversary that carries c/n -of the network capacity to deanonymize c/n of the network -connections, breaking the O((c/n)^2) property of Tor's original -threat model. It also allows targeted attacks aimed at monitoring -the activity of specific users, bridges, or Guard nodes. - -There are two points where path selection can be manipulated: -during construction, and during usage. Circuit construction -can be manipulated by inducing circuit failures during circuit -extend steps, which causes the Tor client to transparently retry -the circuit construction with a new path. Circuit usage can be -manipulated by abusing the stream retry features of Tor (for -example by withholding stream attempt responses from the client -until the stream timeout has expired), at which point the tor client -will also transparently retry the stream on a new path. - -The defense as deployed therefore makes two independent sets of -measurements of successful path use: one during circuit construction, -and one during circuit usage. - -The intended behavior is for clients to ultimately disable the use -of Guards responsible for excessive circuit failure of either type -(see section 7.4); however known issues with the Tor network currently -restrict the defense to being informational only at this stage (see -section 7.5). - - - -## Measuring path construction success rates - -Clients maintain two counts for each of their guards: a count of the -number of times a circuit was extended to at least two hops through that -guard, and a count of the number of circuits that successfully complete -through that guard. The ratio of these two numbers is used to determine -a circuit success rate for that Guard. - -Circuit build timeouts are counted as construction failures if the -circuit fails to complete before the 95% "right-censored" timeout -interval, not the 80% timeout condition (see section 2.4). - -If a circuit closes prematurely after construction but before being -requested to close by the client, this is counted as a failure. - - - -## Measuring path usage success rates - -Clients maintain two usage counts for each of their guards: a count -of the number of usage attempts, and a count of the number of -successful usages. - -A usage attempt means any attempt to attach a stream to a circuit. - -Usage success status is temporarily recorded by state flags on circuits. -Guard usage success counts are not incremented until circuit close. A -circuit is marked as successfully used if we receive a properly -recognized RELAY cell on that circuit that was expected for the current -circuit purpose. - -If subsequent stream attachments fail or time out, the successfully used -state of the circuit is cleared, causing it once again to be regarded -as a usage attempt only. - -Upon close by the client, all circuits that are still marked as usage -attempts are probed using a RELAY_BEGIN cell constructed with a -destination of the form 0.a.b.c:25, where a.b.c is a 24 bit random -nonce. If we get a RELAY_COMMAND_END in response matching our nonce, -the circuit is counted as successfully used. - -If any unrecognized RELAY cells arrive after the probe has been sent, -the circuit is counted as a usage failure. - -If the stream failure reason codes DESTROY, TORPROTOCOL, or INTERNAL -are received in response to any stream attempt, such circuits are not -probed and are declared usage failures. - -Prematurely closed circuits are not probed, and are counted as usage -failures. - - - -## Scaling success counts - -To provide a moving average of recent Guard activity while -still preserving the ability to verify correctness, we periodically -"scale" the success counts by multiplying them by a scale factor -between 0 and 1.0. - -Scaling is performed when either usage or construction attempt counts -exceed a parametrized value. - -To avoid error due to scaling during circuit construction and use, -currently open circuits are subtracted from the usage counts before -scaling, and added back after scaling. - - - -## Parametrization - -The following consensus parameters tune various aspects of the -defense. - -```text - pb_mincircs - Default: 150 - Min: 5 - Effect: This is the minimum number of circuits that must complete - at least 2 hops before we begin evaluating construction rates. - - pb_noticepct - Default: 70 - Min: 0 - Max: 100 - Effect: If the circuit success rate falls below this percentage, - we emit a notice log message. - - pb_warnpct - Default: 50 - Min: 0 - Max: 100 - Effect: If the circuit success rate falls below this percentage, - we emit a warn log message. - - pb_extremepct - Default: 30 - Min: 0 - Max: 100 - Effect: If the circuit success rate falls below this percentage, - we emit a more alarmist warning log message. If - pb_dropguard is set to 1, we also disable the use of the - guard. - - pb_dropguards - Default: 0 - Min: 0 - Max: 1 - Effect: If the circuit success rate falls below pb_extremepct, - when pb_dropguard is set to 1, we disable use of that - guard. - - pb_scalecircs - Default: 300 - Min: 10 - Effect: After this many circuits have completed at least two hops, - Tor performs the scaling described in Section 7.3. - - pb_multfactor and pb_scalefactor - Default: 1/2 - Min: 0.0 - Max: 1.0 - Effect: The double-precision result obtained from - pb_multfactor/pb_scalefactor is multiplied by our current - counts to scale them. - - pb_minuse - Default: 20 - Min: 3 - Effect: This is the minimum number of circuits that we must attempt to - use before we begin evaluating construction rates. - - pb_noticeusepct - Default: 80 - Min: 3 - Effect: If the circuit usage success rate falls below this percentage, - we emit a notice log message. - - pb_extremeusepct - Default: 60 - Min: 3 - Effect: If the circuit usage success rate falls below this percentage, - we emit a warning log message. We also disable the use of the - guard if pb_dropguards is set. - - pb_scaleuse - Default: 100 - Min: 10 - Effect: After we have attempted to use this many circuits, - Tor performs the scaling described in Section 7.3. -``` - - - -## Known barriers to enforcement - -Due to intermittent CPU overload at relays, the normal rate of -successful circuit completion is highly variable. The Guard-dropping -version of the defense is unlikely to be deployed until the ntor -circuit handshake is enabled, or the nature of CPU overload induced -failure is better understood. diff --git a/spec/path-spec/detecting-route-manipulation.md b/spec/path-spec/detecting-route-manipulation.md new file mode 100644 index 0000000..c7ff50c --- /dev/null +++ b/spec/path-spec/detecting-route-manipulation.md @@ -0,0 +1,198 @@ + + +# Detecting route manipulation by Guard nodes (Path Bias) + +The Path Bias defense is designed to defend against a type of route +capture where malicious Guard nodes deliberately fail or choke circuits +that extend to non-colluding Exit nodes to maximize their network +utilization in favor of carrying only compromised traffic. + +In the extreme, the attack allows an adversary that carries c/n +of the network capacity to deanonymize c/n of the network +connections, breaking the O((c/n)^2) property of Tor's original +threat model. It also allows targeted attacks aimed at monitoring +the activity of specific users, bridges, or Guard nodes. + +There are two points where path selection can be manipulated: +during construction, and during usage. Circuit construction +can be manipulated by inducing circuit failures during circuit +extend steps, which causes the Tor client to transparently retry +the circuit construction with a new path. Circuit usage can be +manipulated by abusing the stream retry features of Tor (for +example by withholding stream attempt responses from the client +until the stream timeout has expired), at which point the tor client +will also transparently retry the stream on a new path. + +The defense as deployed therefore makes two independent sets of +measurements of successful path use: one during circuit construction, +and one during circuit usage. + +The intended behavior is for clients to ultimately disable the use +of Guards responsible for excessive circuit failure of either type +(see section 7.4); however known issues with the Tor network currently +restrict the defense to being informational only at this stage (see +section 7.5). + + + +## Measuring path construction success rates + +Clients maintain two counts for each of their guards: a count of the +number of times a circuit was extended to at least two hops through that +guard, and a count of the number of circuits that successfully complete +through that guard. The ratio of these two numbers is used to determine +a circuit success rate for that Guard. + +Circuit build timeouts are counted as construction failures if the +circuit fails to complete before the 95% "right-censored" timeout +interval, not the 80% timeout condition (see section 2.4). + +If a circuit closes prematurely after construction but before being +requested to close by the client, this is counted as a failure. + + + +## Measuring path usage success rates + +Clients maintain two usage counts for each of their guards: a count +of the number of usage attempts, and a count of the number of +successful usages. + +A usage attempt means any attempt to attach a stream to a circuit. + +Usage success status is temporarily recorded by state flags on circuits. +Guard usage success counts are not incremented until circuit close. A +circuit is marked as successfully used if we receive a properly +recognized RELAY cell on that circuit that was expected for the current +circuit purpose. + +If subsequent stream attachments fail or time out, the successfully used +state of the circuit is cleared, causing it once again to be regarded +as a usage attempt only. + +Upon close by the client, all circuits that are still marked as usage +attempts are probed using a RELAY_BEGIN cell constructed with a +destination of the form 0.a.b.c:25, where a.b.c is a 24 bit random +nonce. If we get a RELAY_COMMAND_END in response matching our nonce, +the circuit is counted as successfully used. + +If any unrecognized RELAY cells arrive after the probe has been sent, +the circuit is counted as a usage failure. + +If the stream failure reason codes DESTROY, TORPROTOCOL, or INTERNAL +are received in response to any stream attempt, such circuits are not +probed and are declared usage failures. + +Prematurely closed circuits are not probed, and are counted as usage +failures. + + + +## Scaling success counts + +To provide a moving average of recent Guard activity while +still preserving the ability to verify correctness, we periodically +"scale" the success counts by multiplying them by a scale factor +between 0 and 1.0. + +Scaling is performed when either usage or construction attempt counts +exceed a parametrized value. + +To avoid error due to scaling during circuit construction and use, +currently open circuits are subtracted from the usage counts before +scaling, and added back after scaling. + + + +## Parametrization + +The following consensus parameters tune various aspects of the +defense. + +```text + pb_mincircs + Default: 150 + Min: 5 + Effect: This is the minimum number of circuits that must complete + at least 2 hops before we begin evaluating construction rates. + + pb_noticepct + Default: 70 + Min: 0 + Max: 100 + Effect: If the circuit success rate falls below this percentage, + we emit a notice log message. + + pb_warnpct + Default: 50 + Min: 0 + Max: 100 + Effect: If the circuit success rate falls below this percentage, + we emit a warn log message. + + pb_extremepct + Default: 30 + Min: 0 + Max: 100 + Effect: If the circuit success rate falls below this percentage, + we emit a more alarmist warning log message. If + pb_dropguard is set to 1, we also disable the use of the + guard. + + pb_dropguards + Default: 0 + Min: 0 + Max: 1 + Effect: If the circuit success rate falls below pb_extremepct, + when pb_dropguard is set to 1, we disable use of that + guard. + + pb_scalecircs + Default: 300 + Min: 10 + Effect: After this many circuits have completed at least two hops, + Tor performs the scaling described in Section 7.3. + + pb_multfactor and pb_scalefactor + Default: 1/2 + Min: 0.0 + Max: 1.0 + Effect: The double-precision result obtained from + pb_multfactor/pb_scalefactor is multiplied by our current + counts to scale them. + + pb_minuse + Default: 20 + Min: 3 + Effect: This is the minimum number of circuits that we must attempt to + use before we begin evaluating construction rates. + + pb_noticeusepct + Default: 80 + Min: 3 + Effect: If the circuit usage success rate falls below this percentage, + we emit a notice log message. + + pb_extremeusepct + Default: 60 + Min: 3 + Effect: If the circuit usage success rate falls below this percentage, + we emit a warning log message. We also disable the use of the + guard if pb_dropguards is set. + + pb_scaleuse + Default: 100 + Min: 10 + Effect: After we have attempted to use this many circuits, + Tor performs the scaling described in Section 7.3. +``` + + + +## Known barriers to enforcement + +Due to intermittent CPU overload at relays, the normal rate of +successful circuit completion is highly variable. The Guard-dropping +version of the defense is unlikely to be deployed until the ntor +circuit handshake is enabled, or the nature of CPU overload induced +failure is better understood. diff --git a/spec/path-spec/learning-timeouts.md b/spec/path-spec/learning-timeouts.md new file mode 100644 index 0000000..939f3a2 --- /dev/null +++ b/spec/path-spec/learning-timeouts.md @@ -0,0 +1,300 @@ + + +## Learning when to give up ("timeout") on circuit construction + +Since version 0.2.2.8-alpha, Tor clients attempt to learn when to give +up on circuits based on network conditions. + + + +### Distribution choice + +Based on studies of build times, we found that the distribution of +circuit build times appears to be a Frechet distribution (and a multi-modal +Frechet distribution, if more than one guard or bridge is used). However, +estimators and quantile functions of the Frechet distribution are difficult +to work with and slow to converge. So instead, since we are only interested +in the accuracy of the tail, clients approximate the tail of the multi-modal +distribution with a single Pareto curve. + + + +### How much data to record + +From our observations, the minimum number of circuit build times for a +reasonable fit appears to be on the order of 100. However, to keep a +good fit over the long term, clients store 1000 most recent circuit build +times in a circular array. + +These build times only include the times required to build three-hop +circuits, and the times required to build the first three hops of circuits +with more than three hops. Circuits of fewer than three hops are not +recorded, and hops past the third are not recorded. + +The Tor client should build test circuits at a rate of one every 'cbttestfreq' +(10 seconds) until 'cbtmincircs' (100 circuits) are built, with a maximum of +'cbtmaxopencircs' (default: 10) circuits open at once. This allows a fresh +Tor to have a CircuitBuildTimeout estimated within 30 minutes after install +or network change (see section 2.4.5 below). + +Timeouts are stored on disk in a histogram of 10ms bin width, the same +width used to calculate the Xm value above. The timeouts recorded in the +histogram must be shuffled after being read from disk, to preserve a +proper expiration of old values after restart. + +Thus, some build time resolution is lost during restart. Implementations may +choose a different persistence mechanism than this histogram, but be aware +that build time binning is still needed for parameter estimation. + + + +### Parameter estimation + +Once 'cbtmincircs' build times are recorded, Tor clients update the +distribution parameters and recompute the timeout every circuit completion +(though see section 2.4.5 for when to pause and reset timeout due to +too many circuits timing out). + +Tor clients calculate the parameters for a Pareto distribution fitting the +data using the maximum likelihood estimator. For derivation, see: + + +Because build times are not a true Pareto distribution, we alter how Xm is +computed. In a max likelihood estimator, the mode of the distribution is +used directly as Xm. + +Instead of using the mode of discrete build times directly, Tor clients +compute the Xm parameter using the weighted average of the midpoints +of the 'cbtnummodes' (10) most frequently occurring 10ms histogram bins. +Ties are broken in favor of earlier bins (that is, in favor of bins +corresponding to shorter build times). + +(The use of 10 modes was found to minimize error from the selected +cbtquantile, with 10ms bins for quantiles 60-80, compared to many other +heuristics). + +To avoid ln(1.0+epsilon) precision issues, use log laws to rewrite the +estimator for 'alpha' as the sum of logs followed by subtraction, rather +than multiplication and division: + +alpha = n/(Sum_n{ln(MAX(Xm, x_i))} - n\*ln(Xm)) + +In this, n is the total number of build times that have completed, x_i is +the ith recorded build time, and Xm is the modes of x_i as above. + +All times below Xm are counted as having the Xm value via the MAX(), +because in Pareto estimators, Xm is supposed to be the lowest value. +However, since clients use mode averaging to estimate Xm, there can be +values below our Xm. Effectively, the Pareto estimator then treats that +everything smaller than Xm happened at Xm. One can also see that if +clients did not do this, alpha could underflow to become negative, which +results in an exponential curve, not a Pareto probability distribution. + +The timeout itself is calculated by using the Pareto Quantile function (the +inverted CDF) to give us the value on the CDF such that 80% of the mass +of the distribution is below the timeout value (parameter 'cbtquantile'). + +The Pareto Quantile Function (inverse CDF) is: + +F(q) = Xm/((1.0-q)^(1.0/alpha)) + +Thus, clients obtain the circuit build timeout for 3-hop circuits by +computing: + +timeout_ms = F(0.8) # 'cbtquantile' == 0.8 + +With this, we expect that the Tor client will accept the fastest 80% of the +total number of paths on the network. + +Clients obtain the circuit close time to completely abandon circuits as: + +close_ms = F(0.99) # 'cbtclosequantile' == 0.99 + +To avoid waiting an unreasonably long period of time for circuits that +simply have relays that are down, Tor clients cap timeout_ms at the max +build time actually observed so far, and cap close_ms at twice this max, +but at least 60 seconds: + +```text + timeout_ms = MIN(timeout_ms, max_observed_timeout) + close_ms = MAX(MIN(close_ms, 2*max_observed_timeout), 'cbtinitialtimeout') +``` + + + +### Calculating timeouts thresholds for circuits of different lengths + +The timeout_ms and close_ms estimates above are good only for 3-hop +circuits, since only 3-hop circuits are recorded in the list of build +times. + +To calculate the appropriate timeouts and close timeouts for circuits of +other lengths, the client multiples the timeout_ms and close_ms values +by a scaling factor determined by the number of communication hops +needed to build their circuits: + +timeout_ms\[hops=n\] = timeout_ms * Actions(N) / Actions(3) + +close_ms\[hops=n\] = close_ms * Actions(N) / Actions(3) + +where Actions(N) = N * (N + 1) / 2. + +To calculate timeouts for operations other than circuit building, +the client should add X to Actions(N) for every round-trip communication +required with the Xth hop. + + + +### How to record timeouts + +Pareto estimators begin to lose their accuracy if the tail is omitted. +Hence, Tor clients actually calculate two timeouts: a usage timeout, and a +close timeout. + +Circuits that pass the usage timeout are marked as measurement circuits, +and are allowed to continue to build until the close timeout corresponding +to the point 'cbtclosequantile' (default 99) on the Pareto curve, or 60 +seconds, whichever is greater. + +The actual completion times for these measurement circuits should be +recorded. + +Implementations should completely abandon a circuit and ignore the circuit +if the total build time exceeds the close threshold. Such closed circuits +should be ignored, as this typically means one of the relays in the path is +offline. + + + +### Detecting Changing Network Conditions + +Tor clients attempt to detect both network connectivity loss and drastic +changes in the timeout characteristics. + +To detect changing network conditions, clients keep a history of +the timeout or non-timeout status of the past 'cbtrecentcount' circuits +(20 circuits) that successfully completed at least one hop. If more than +90% of these circuits timeout, the client discards all buildtimes history, +resets the timeout to 'cbtinitialtimeout' (60 seconds), and then begins +recomputing the timeout. + +If the timeout was already at least `cbtinitialtimeout`, +the client doubles the timeout. + +The records here (of how many circuits succeeded or failed among the most +recent 'cbrrecentcount') are not stored as persistent state. On reload, +we start with a new, empty state. + + + +### Consensus parameters governing behavior + +Clients that implement circuit build timeout learning should obey the +following consensus parameters that govern behavior, in order to allow +us to handle bugs or other emergent behaviors due to client circuit +construction. If these parameters are not present in the consensus, +the listed default values should be used instead. + +```text + cbtdisabled + Default: 0 + Min: 0 + Max: 1 + Effect: If 1, all CircuitBuildTime learning code should be + disabled and history should be discarded. For use in + emergency situations only. + + cbtnummodes + Default: 10 + Min: 1 + Max: 20 + Effect: This value governs how many modes to use in the weighted + average calculation of Pareto parameter Xm. Selecting Xm as the + average of multiple modes improves accuracy of the Pareto tail + for quantile cutoffs from 60-80% (see cbtquantile). + + cbtrecentcount + Default: 20 + Min: 3 + Max: 1000 + Effect: This is the number of circuit build outcomes (success vs + timeout) to keep track of for the following option. + + cbtmaxtimeouts + Default: 18 + Min: 3 + Max: 10000 + Effect: When this many timeouts happen in the last 'cbtrecentcount' + circuit attempts, the client should discard all of its + history and begin learning a fresh timeout value. + + Note that if this parameter's value is greater than the value + of 'cbtrecentcount', then the history will never be + discarded because of this feature. + + cbtmincircs + Default: 100 + Min: 1 + Max: 10000 + Effect: This is the minimum number of circuits to build before + computing a timeout. + + Note that if this parameter's value is higher than 1000 (the + number of time observations that a client keeps in its + circular buffer), circuit build timeout calculation is + effectively disabled, and the default timeouts are used + indefinitely. + + cbtquantile + Default: 80 + Min: 10 + Max: 99 + Effect: This is the position on the quantile curve to use to set the + timeout value. It is a percent (10-99). + + cbtclosequantile + Default: 99 + Min: Value of cbtquantile parameter + Max: 99 + Effect: This is the position on the quantile curve to use to set the + timeout value to use to actually close circuits. It is a + percent (0-99). + + cbttestfreq + Default: 10 + Min: 1 + Max: 2147483647 (INT32_MAX) + Effect: Describes how often in seconds to build a test circuit to + gather timeout values. Only applies if less than 'cbtmincircs' + have been recorded. + + cbtmintimeout + Default: 10 + Min: 10 + Max: 2147483647 (INT32_MAX) + Effect: This is the minimum allowed timeout value in milliseconds. + + cbtinitialtimeout + Default: 60000 + Min: Value of cbtmintimeout + Max: 2147483647 (INT32_MAX) + Effect: This is the timeout value to use before we have enough data + to compute a timeout, in milliseconds. If we do not have + enough data to compute a timeout estimate (see cbtmincircs), + then we use this interval both for the close timeout and the + abandon timeout. + + cbtlearntimeout + Default: 180 + Min: 10 + Max: 60000 + Effect: This is how long idle circuits will be kept open while cbt is + learning a new timeout value. + + cbtmaxopencircs + Default: 10 + Min: 0 + Max: 14 + Effect: This is the maximum number of circuits that can be open at + at the same time during the circuit build time learning phase. +``` diff --git a/spec/path-spec/learning-when-to-give-up-timeout-on-circuit-construction.md b/spec/path-spec/learning-when-to-give-up-timeout-on-circuit-construction.md deleted file mode 100644 index 939f3a2..0000000 --- a/spec/path-spec/learning-when-to-give-up-timeout-on-circuit-construction.md +++ /dev/null @@ -1,300 +0,0 @@ - - -## Learning when to give up ("timeout") on circuit construction - -Since version 0.2.2.8-alpha, Tor clients attempt to learn when to give -up on circuits based on network conditions. - - - -### Distribution choice - -Based on studies of build times, we found that the distribution of -circuit build times appears to be a Frechet distribution (and a multi-modal -Frechet distribution, if more than one guard or bridge is used). However, -estimators and quantile functions of the Frechet distribution are difficult -to work with and slow to converge. So instead, since we are only interested -in the accuracy of the tail, clients approximate the tail of the multi-modal -distribution with a single Pareto curve. - - - -### How much data to record - -From our observations, the minimum number of circuit build times for a -reasonable fit appears to be on the order of 100. However, to keep a -good fit over the long term, clients store 1000 most recent circuit build -times in a circular array. - -These build times only include the times required to build three-hop -circuits, and the times required to build the first three hops of circuits -with more than three hops. Circuits of fewer than three hops are not -recorded, and hops past the third are not recorded. - -The Tor client should build test circuits at a rate of one every 'cbttestfreq' -(10 seconds) until 'cbtmincircs' (100 circuits) are built, with a maximum of -'cbtmaxopencircs' (default: 10) circuits open at once. This allows a fresh -Tor to have a CircuitBuildTimeout estimated within 30 minutes after install -or network change (see section 2.4.5 below). - -Timeouts are stored on disk in a histogram of 10ms bin width, the same -width used to calculate the Xm value above. The timeouts recorded in the -histogram must be shuffled after being read from disk, to preserve a -proper expiration of old values after restart. - -Thus, some build time resolution is lost during restart. Implementations may -choose a different persistence mechanism than this histogram, but be aware -that build time binning is still needed for parameter estimation. - - - -### Parameter estimation - -Once 'cbtmincircs' build times are recorded, Tor clients update the -distribution parameters and recompute the timeout every circuit completion -(though see section 2.4.5 for when to pause and reset timeout due to -too many circuits timing out). - -Tor clients calculate the parameters for a Pareto distribution fitting the -data using the maximum likelihood estimator. For derivation, see: - - -Because build times are not a true Pareto distribution, we alter how Xm is -computed. In a max likelihood estimator, the mode of the distribution is -used directly as Xm. - -Instead of using the mode of discrete build times directly, Tor clients -compute the Xm parameter using the weighted average of the midpoints -of the 'cbtnummodes' (10) most frequently occurring 10ms histogram bins. -Ties are broken in favor of earlier bins (that is, in favor of bins -corresponding to shorter build times). - -(The use of 10 modes was found to minimize error from the selected -cbtquantile, with 10ms bins for quantiles 60-80, compared to many other -heuristics). - -To avoid ln(1.0+epsilon) precision issues, use log laws to rewrite the -estimator for 'alpha' as the sum of logs followed by subtraction, rather -than multiplication and division: - -alpha = n/(Sum_n{ln(MAX(Xm, x_i))} - n\*ln(Xm)) - -In this, n is the total number of build times that have completed, x_i is -the ith recorded build time, and Xm is the modes of x_i as above. - -All times below Xm are counted as having the Xm value via the MAX(), -because in Pareto estimators, Xm is supposed to be the lowest value. -However, since clients use mode averaging to estimate Xm, there can be -values below our Xm. Effectively, the Pareto estimator then treats that -everything smaller than Xm happened at Xm. One can also see that if -clients did not do this, alpha could underflow to become negative, which -results in an exponential curve, not a Pareto probability distribution. - -The timeout itself is calculated by using the Pareto Quantile function (the -inverted CDF) to give us the value on the CDF such that 80% of the mass -of the distribution is below the timeout value (parameter 'cbtquantile'). - -The Pareto Quantile Function (inverse CDF) is: - -F(q) = Xm/((1.0-q)^(1.0/alpha)) - -Thus, clients obtain the circuit build timeout for 3-hop circuits by -computing: - -timeout_ms = F(0.8) # 'cbtquantile' == 0.8 - -With this, we expect that the Tor client will accept the fastest 80% of the -total number of paths on the network. - -Clients obtain the circuit close time to completely abandon circuits as: - -close_ms = F(0.99) # 'cbtclosequantile' == 0.99 - -To avoid waiting an unreasonably long period of time for circuits that -simply have relays that are down, Tor clients cap timeout_ms at the max -build time actually observed so far, and cap close_ms at twice this max, -but at least 60 seconds: - -```text - timeout_ms = MIN(timeout_ms, max_observed_timeout) - close_ms = MAX(MIN(close_ms, 2*max_observed_timeout), 'cbtinitialtimeout') -``` - - - -### Calculating timeouts thresholds for circuits of different lengths - -The timeout_ms and close_ms estimates above are good only for 3-hop -circuits, since only 3-hop circuits are recorded in the list of build -times. - -To calculate the appropriate timeouts and close timeouts for circuits of -other lengths, the client multiples the timeout_ms and close_ms values -by a scaling factor determined by the number of communication hops -needed to build their circuits: - -timeout_ms\[hops=n\] = timeout_ms * Actions(N) / Actions(3) - -close_ms\[hops=n\] = close_ms * Actions(N) / Actions(3) - -where Actions(N) = N * (N + 1) / 2. - -To calculate timeouts for operations other than circuit building, -the client should add X to Actions(N) for every round-trip communication -required with the Xth hop. - - - -### How to record timeouts - -Pareto estimators begin to lose their accuracy if the tail is omitted. -Hence, Tor clients actually calculate two timeouts: a usage timeout, and a -close timeout. - -Circuits that pass the usage timeout are marked as measurement circuits, -and are allowed to continue to build until the close timeout corresponding -to the point 'cbtclosequantile' (default 99) on the Pareto curve, or 60 -seconds, whichever is greater. - -The actual completion times for these measurement circuits should be -recorded. - -Implementations should completely abandon a circuit and ignore the circuit -if the total build time exceeds the close threshold. Such closed circuits -should be ignored, as this typically means one of the relays in the path is -offline. - - - -### Detecting Changing Network Conditions - -Tor clients attempt to detect both network connectivity loss and drastic -changes in the timeout characteristics. - -To detect changing network conditions, clients keep a history of -the timeout or non-timeout status of the past 'cbtrecentcount' circuits -(20 circuits) that successfully completed at least one hop. If more than -90% of these circuits timeout, the client discards all buildtimes history, -resets the timeout to 'cbtinitialtimeout' (60 seconds), and then begins -recomputing the timeout. - -If the timeout was already at least `cbtinitialtimeout`, -the client doubles the timeout. - -The records here (of how many circuits succeeded or failed among the most -recent 'cbrrecentcount') are not stored as persistent state. On reload, -we start with a new, empty state. - - - -### Consensus parameters governing behavior - -Clients that implement circuit build timeout learning should obey the -following consensus parameters that govern behavior, in order to allow -us to handle bugs or other emergent behaviors due to client circuit -construction. If these parameters are not present in the consensus, -the listed default values should be used instead. - -```text - cbtdisabled - Default: 0 - Min: 0 - Max: 1 - Effect: If 1, all CircuitBuildTime learning code should be - disabled and history should be discarded. For use in - emergency situations only. - - cbtnummodes - Default: 10 - Min: 1 - Max: 20 - Effect: This value governs how many modes to use in the weighted - average calculation of Pareto parameter Xm. Selecting Xm as the - average of multiple modes improves accuracy of the Pareto tail - for quantile cutoffs from 60-80% (see cbtquantile). - - cbtrecentcount - Default: 20 - Min: 3 - Max: 1000 - Effect: This is the number of circuit build outcomes (success vs - timeout) to keep track of for the following option. - - cbtmaxtimeouts - Default: 18 - Min: 3 - Max: 10000 - Effect: When this many timeouts happen in the last 'cbtrecentcount' - circuit attempts, the client should discard all of its - history and begin learning a fresh timeout value. - - Note that if this parameter's value is greater than the value - of 'cbtrecentcount', then the history will never be - discarded because of this feature. - - cbtmincircs - Default: 100 - Min: 1 - Max: 10000 - Effect: This is the minimum number of circuits to build before - computing a timeout. - - Note that if this parameter's value is higher than 1000 (the - number of time observations that a client keeps in its - circular buffer), circuit build timeout calculation is - effectively disabled, and the default timeouts are used - indefinitely. - - cbtquantile - Default: 80 - Min: 10 - Max: 99 - Effect: This is the position on the quantile curve to use to set the - timeout value. It is a percent (10-99). - - cbtclosequantile - Default: 99 - Min: Value of cbtquantile parameter - Max: 99 - Effect: This is the position on the quantile curve to use to set the - timeout value to use to actually close circuits. It is a - percent (0-99). - - cbttestfreq - Default: 10 - Min: 1 - Max: 2147483647 (INT32_MAX) - Effect: Describes how often in seconds to build a test circuit to - gather timeout values. Only applies if less than 'cbtmincircs' - have been recorded. - - cbtmintimeout - Default: 10 - Min: 10 - Max: 2147483647 (INT32_MAX) - Effect: This is the minimum allowed timeout value in milliseconds. - - cbtinitialtimeout - Default: 60000 - Min: Value of cbtmintimeout - Max: 2147483647 (INT32_MAX) - Effect: This is the timeout value to use before we have enough data - to compute a timeout, in milliseconds. If we do not have - enough data to compute a timeout estimate (see cbtmincircs), - then we use this interval both for the close timeout and the - abandon timeout. - - cbtlearntimeout - Default: 180 - Min: 10 - Max: 60000 - Effect: This is how long idle circuits will be kept open while cbt is - learning a new timeout value. - - cbtmaxopencircs - Default: 10 - Min: 0 - Max: 14 - Effect: This is the maximum number of circuits that can be open at - at the same time during the circuit build time learning phase. -``` diff --git a/spec/pt-spec/configuration-environment.md b/spec/pt-spec/configuration-environment.md new file mode 100644 index 0000000..79e9f67 --- /dev/null +++ b/spec/pt-spec/configuration-environment.md @@ -0,0 +1,258 @@ + + +## Pluggable Transport Configuration Environment Variables + +All Pluggable Transport proxy instances are configured by their +parent process at launch time via a set of well defined +environment variables. + +The "TOR_PT\_" prefix is used for namespacing reasons and does not +indicate any relations to Tor, except for the origins of this +specification. + + + +### Common Environment Variables + +When launching either a client or server Pluggable Transport proxy, +the following common environment variables MUST be set. + +"TOR_PT_MANAGED_TRANSPORT_VER" + +Specifies the versions of the Pluggable Transport specification +the parent process supports, delimited by commas. All PTs MUST +accept any well-formed list, as long as a compatible version is +present. + +Valid versions MUST consist entirely of non-whitespace, +non-comma printable ASCII characters. + +The version of the Pluggable Transport specification as of this +document is "1". + +Example: + +TOR_PT_MANAGED_TRANSPORT_VER=1,1a,2b,this_is_a_valid_ver + +"TOR_PT_STATE_LOCATION" + +Specifies an absolute path to a directory where the PT is +allowed to store state that will be persisted across +invocations. The directory is not required to exist when +the PT is launched, however PT implementations SHOULD be +able to create it as required. + +PTs MUST only store files in the path provided, and MUST NOT +create or modify files elsewhere on the system. + +Example: + +TOR_PT_STATE_LOCATION=/var/lib/tor/pt_state/ + +"TOR_PT_EXIT_ON_STDIN_CLOSE" + +Specifies that the parent process will close the PT proxy's +standard input (stdin) stream to indicate that the PT proxy +should gracefully exit. + +PTs MUST NOT treat a closed stdin as a signal to terminate +unless this environment variable is set to "1". + +PTs SHOULD treat stdin being closed as a signal to gracefully +terminate if this environment variable is set to "1". + +Example: + +TOR_PT_EXIT_ON_STDIN_CLOSE=1 + +"TOR_PT_OUTBOUND_BIND_ADDRESS_V4" + +Specifies an IPv4 IP address that the PT proxy SHOULD use as source address for +outgoing IPv4 IP packets. This feature allows people with multiple network +interfaces to specify explicitly which interface they prefer the PT proxy to +use. + +If this value is unset or empty, the PT proxy MUST use the default source +address for outgoing connections. + +This setting MUST be ignored for connections to +loopback addresses (127.0.0.0/8). + +Example: + +TOR_PT_OUTBOUND_BIND_ADDRESS_V4=203.0.113.4 + +"TOR_PT_OUTBOUND_BIND_ADDRESS_V6" + +Specifies an IPv6 IP address that the PT proxy SHOULD use as source address for +outgoing IPv6 IP packets. This feature allows people with multiple network +interfaces to specify explicitly which interface they prefer the PT proxy to +use. + +If this value is unset or empty, the PT proxy MUST use the default source +address for outgoing connections. + +This setting MUST be ignored for connections to the loopback address (\[::1\]). + +IPv6 addresses MUST always be wrapped in square brackets. + +Example:: + +TOR_PT_OUTBOUND_BIND_ADDRESS_V6=\[2001:db8::4\] + + + +### Pluggable Transport Client Environment Variables + +Client-side Pluggable Transport forward proxies are configured +via the following environment variables. + +"TOR_PT_CLIENT_TRANSPORTS" + +Specifies the PT protocols the client proxy should initialize, +as a comma separated list of PT names. + +PTs SHOULD ignore PT names that it does not recognize. + +Parent processes MUST set this environment variable when +launching a client-side PT proxy instance. + +Example: + +TOR_PT_CLIENT_TRANSPORTS=obfs2,obfs3,obfs4 + +"TOR_PT_PROXY" + +Specifies an upstream proxy that the PT MUST use when making +outgoing network connections. It is a URI \[RFC3986\] of the +format: + +`://[[:][@]:`. + +The "TOR_PT_PROXY" environment variable is OPTIONAL and +MUST be omitted if there is no need to connect via an +upstream proxy. + +Examples: + +```text + TOR_PT_PROXY=socks5://tor:test1234@198.51.100.1:8000 + TOR_PT_PROXY=socks4a://198.51.100.2:8001 + TOR_PT_PROXY=http://198.51.100.3:443 +``` + + + +### Pluggable Transport Server Environment Variables + +Server-side Pluggable Transport reverse proxies are configured +via the following environment variables. + +"TOR_PT_SERVER_TRANSPORTS" + +Specifies the PT protocols the server proxy should initialize, +as a comma separated list of PT names. + +PTs SHOULD ignore PT names that it does not recognize. + +Parent processes MUST set this environment variable when +launching a server-side PT reverse proxy instance. + +Example: + +TOR_PT_SERVER_TRANSPORTS=obfs3,scramblesuit + +"TOR_PT_SERVER_TRANSPORT_OPTIONS" + +Specifies per-PT protocol configuration directives, as a +semicolon-separated list of `:` pairs, where `` +is a PT name and `` is a k=v string value with options +that are to be passed to the transport. + +Colons, semicolons, and backslashes MUST be +escaped with a backslash. + +If there are no arguments that need to be passed to any of +PT transport protocols, "TOR_PT_SERVER_TRANSPORT_OPTIONS" +MAY be omitted. + +Example: + +TOR_PT_SERVER_TRANSPORT_OPTIONS=scramblesuit:key=banana;automata:rule=110;automata:depth=3 + +```text + Will pass to 'scramblesuit' the parameter 'key=banana' and to + 'automata' the arguments 'rule=110' and 'depth=3'. + + "TOR_PT_SERVER_BINDADDR" +``` + +A comma separated list of `-` pairs, where `` is +a PT name and `` is the `
:` on which it +should listen for incoming client connections. + +The keys holding transport names MUST be in the same order as +they appear in "TOR_PT_SERVER_TRANSPORTS". + +The `
` MAY be a locally scoped address as long as port +forwarding is done externally. + +The `
:` combination MUST be an IP address +supported by `bind()`, and MUST NOT be a host name. + +Applications MUST NOT set more than one `
:` pair +per PT name. + +If there is no specific `
:` combination to be +configured for any transports, "TOR_PT_SERVER_BINDADDR" MAY +be omitted. + +Example: + +TOR_PT_SERVER_BINDADDR=obfs3-198.51.100.1:1984,scramblesuit-127.0.0.1:4891 + +"TOR_PT_ORPORT" + +Specifies the destination that the PT reverse proxy should forward +traffic to after transforming it as appropriate, as an +`
:`. + +Connections to the destination specified via "TOR_PT_ORPORT" +MUST only contain application payload. If the parent process +requires the actual source IP address of client connections +(or other metadata), it should set "TOR_PT_EXTENDED_SERVER_PORT" +instead. + +Example: + +TOR_PT_ORPORT=127.0.0.1:9001 + +"TOR_PT_EXTENDED_SERVER_PORT" + +Specifies the destination that the PT reverse proxy should +forward traffic to, via the Extended ORPort protocol \[EXTORPORT\] +as an `
:`. + +The Extended ORPort protocol allows the PT reverse proxy to +communicate per-connection metadata such as the PT name and +client IP address/port to the parent process. + +If the parent process does not support the ExtORPort protocol, +it MUST set "TOR_PT_EXTENDED_SERVER_PORT" to an empty string. + +Example: + +TOR_PT_EXTENDED_SERVER_PORT=127.0.0.1:4200 + +"TOR_PT_AUTH_COOKIE_FILE" + +Specifies an absolute filesystem path to the Extended ORPort +authentication cookie, required to communicate with the +Extended ORPort specified via "TOR_PT_EXTENDED_SERVER_PORT". + +If the parent process is not using the ExtORPort protocol for +incoming traffic, "TOR_PT_AUTH_COOKIE_FILE" MUST be omitted. + +Example: + +TOR_PT_AUTH_COOKIE_FILE=/var/lib/tor/extended_orport_auth_cookie diff --git a/spec/pt-spec/example-client-pluggable-transport-session.md b/spec/pt-spec/example-client-pluggable-transport-session.md deleted file mode 100644 index 2f20f5d..0000000 --- a/spec/pt-spec/example-client-pluggable-transport-session.md +++ /dev/null @@ -1,19 +0,0 @@ - - -# Appendix A: Example Client Pluggable Transport Session - -Environment variables: - -TOR_PT_MANAGED_TRANSPORT_VER=1 -TOR_PT_STATE_LOCATION=/var/lib/tor/pt_state/ -TOR_PT_EXIT_ON_STDIN_CLOSE=1 -TOR_PT_PROXY=socks5://127.0.0.1:8001 -TOR_PT_CLIENT_TRANSPORTS=obfs3,obfs4 - -Messages the PT Proxy writes to stdin: - -VERSION 1 -PROXY DONE -CMETHOD obfs3 socks5 127.0.0.1:32525 -CMETHOD obfs4 socks5 127.0.0.1:37347 -CMETHODS DONE diff --git a/spec/pt-spec/example-client-session.md b/spec/pt-spec/example-client-session.md new file mode 100644 index 0000000..2f20f5d --- /dev/null +++ b/spec/pt-spec/example-client-session.md @@ -0,0 +1,19 @@ + + +# Appendix A: Example Client Pluggable Transport Session + +Environment variables: + +TOR_PT_MANAGED_TRANSPORT_VER=1 +TOR_PT_STATE_LOCATION=/var/lib/tor/pt_state/ +TOR_PT_EXIT_ON_STDIN_CLOSE=1 +TOR_PT_PROXY=socks5://127.0.0.1:8001 +TOR_PT_CLIENT_TRANSPORTS=obfs3,obfs4 + +Messages the PT Proxy writes to stdin: + +VERSION 1 +PROXY DONE +CMETHOD obfs3 socks5 127.0.0.1:32525 +CMETHOD obfs4 socks5 127.0.0.1:37347 +CMETHODS DONE diff --git a/spec/pt-spec/example-server-pluggable-transport-session.md b/spec/pt-spec/example-server-pluggable-transport-session.md deleted file mode 100644 index 3fc3962..0000000 --- a/spec/pt-spec/example-server-pluggable-transport-session.md +++ /dev/null @@ -1,18 +0,0 @@ - - -# Appendix B: Example Server Pluggable Transport Session - -Environment variables: - -TOR_PT_MANAGED_TRANSPORT_VER=1 -TOR_PT_STATE_LOCATION=/var/lib/tor/pt_state -TOR_PT_EXIT_ON_STDIN_CLOSE=1 -TOR_PT_SERVER_TRANSPORTS=obfs3,obfs4 -TOR_PT_SERVER_BINDADDR=obfs3-198.51.100.1:1984 - -Messages the PT Proxy writes to stdin: - -VERSION 1 -SMETHOD obfs3 198.51.100.1:1984 -SMETHOD obfs4 198.51.100.1:43734 ARGS:cert=HszPy3vWfjsESCEOo9ZBkRv6zQ/1mGHzc8arF0y2SpwFr3WhsMu8rK0zyaoyERfbz3ddFw,iat-mode=0 -SMETHODS DONE diff --git a/spec/pt-spec/example-server-session.md b/spec/pt-spec/example-server-session.md new file mode 100644 index 0000000..3fc3962 --- /dev/null +++ b/spec/pt-spec/example-server-session.md @@ -0,0 +1,18 @@ + + +# Appendix B: Example Server Pluggable Transport Session + +Environment variables: + +TOR_PT_MANAGED_TRANSPORT_VER=1 +TOR_PT_STATE_LOCATION=/var/lib/tor/pt_state +TOR_PT_EXIT_ON_STDIN_CLOSE=1 +TOR_PT_SERVER_TRANSPORTS=obfs3,obfs4 +TOR_PT_SERVER_BINDADDR=obfs3-198.51.100.1:1984 + +Messages the PT Proxy writes to stdin: + +VERSION 1 +SMETHOD obfs3 198.51.100.1:1984 +SMETHOD obfs4 198.51.100.1:43734 ARGS:cert=HszPy3vWfjsESCEOo9ZBkRv6zQ/1mGHzc8arF0y2SpwFr3WhsMu8rK0zyaoyERfbz3ddFw,iat-mode=0 +SMETHODS DONE diff --git a/spec/pt-spec/ipc.md b/spec/pt-spec/ipc.md new file mode 100644 index 0000000..21fd18b --- /dev/null +++ b/spec/pt-spec/ipc.md @@ -0,0 +1,303 @@ + + +## Pluggable Transport To Parent Process Communication + +All Pluggable Transport Proxies communicate to the parent process +via writing NL-terminated lines to stdout. The line metaformat is: + +```text + ::= + ::= | + ::= + ::= * + ::= | + ::= + ::= + ::= +``` + +The parent process MUST ignore lines received from PT proxies with +unknown keywords. + + + +### Common Messages + +When a PT proxy first starts up, it must determine which version +of the Pluggable Transports Specification to use to configure +itself. + +It does this via the "TOR_PT_MANAGED_TRANSPORT_VER" (3.2.1) +environment variable which contains all of the versions supported +by the application. + +Upon determining the version to use, or lack thereof, the PT +proxy responds with one of two messages. + +`VERSION-ERROR ` + +The "VERSION-ERROR" message is used to signal that there was +no compatible Pluggable Transport Specification version +present in the "TOR_PT_MANAGED_TRANSPORT_VER" list. + +The `` SHOULD be set to "no-version" for +historical reasons but MAY be set to a useful error message +instead. + +PT proxies MUST terminate after outputting a "VERSION-ERROR" +message. + +Example: + +`VERSION-ERROR no-version` + +`VERSION ` + +The "VERSION" message is used to signal the Pluggable Transport +Specification version (as in "TOR_PT_MANAGED_TRANSPORT_VER") +that the PT proxy will use to configure its transports and +communicate with the parent process. + +The version for the environment values and reply messages +specified by this document is "1". + +PT proxies MUST either report an error and terminate, or output +a "VERSION" message before moving on to client/server proxy +initialization and configuration. + +Example: + +VERSION 1 + +After version negotiation has been completed the PT proxy must +then validate that all of the required environment variables are +provided, and that all of the configuration values supplied are +well formed. + +At any point, if there is an error encountered related to +configuration supplied via the environment variables, it MAY +respond with an error message and terminate. + +`ENV-ERROR ` + +The "ENV-ERROR" message is used to signal the PT proxy's +failure to parse the configuration environment variables (3.2). + +The `` SHOULD consist of a useful error message +that can be used to diagnose and correct the root cause of +the failure. + +PT proxies MUST terminate after outputting a "ENV-ERROR" +message. + +Example: + +ENV-ERROR No TOR_PT_AUTH_COOKIE_FILE when TOR_PT_EXTENDED_SERVER_PORT set + + + +### Pluggable Transport Client Messages + +After negotiating the Pluggable Transport Specification version, +PT client proxies MUST first validate "TOR_PT_PROXY" (3.2.2) if +it is set, before initializing any transports. + +Assuming that an upstream proxy is provided, PT client proxies +MUST respond with a message indicating that the proxy is valid, +supported, and will be used OR a failure message. + +PROXY DONE + +The "PROXY DONE" message is used to signal the PT proxy's +acceptance of the upstream proxy specified by "TOR_PT_PROXY". + +`PROXY-ERROR ` + +The "PROXY-ERROR" message is used to signal that the upstream +proxy is malformed/unsupported or otherwise unusable. + +PT proxies MUST terminate immediately after outputting a +"PROXY-ERROR" message. + +Example: + +PROXY-ERROR SOCKS 4 upstream proxies unsupported. + +After the upstream proxy (if any) is configured, PT clients then +iterate over the requested transports in "TOR_PT_CLIENT_TRANSPORTS" +and initialize the listeners. + +For each transport initialized, the PT proxy reports the listener +status back to the parent via messages to stdout. + +`CMETHOD <'socks4','socks5'> ` + +The "CMETHOD" message is used to signal that a requested +PT transport has been launched, the protocol which the parent +should use to make outgoing connections, and the IP address +and port that the PT transport's forward proxy is listening on. + +Example: + +`CMETHOD trebuchet socks5 127.0.0.1:19999` + +`CMETHOD-ERROR ` + +The "CMETHOD-ERROR" message is used to signal that +requested PT transport was unable to be launched. + +Example: + +CMETHOD-ERROR trebuchet no rocks available + +Once all PT transports have been initialized (or have failed), the +PT proxy MUST send a final message indicating that it has finished +initializing. + +CMETHODS DONE + +The "CMETHODS DONE" message signals that the PT proxy has +finished initializing all of the transports that it is capable +of handling. + +Upon sending the "CMETHODS DONE" message, the PT proxy +initialization is complete. + +Notes: + +```text + - Unknown transports in "TOR_PT_CLIENT_TRANSPORTS" are ignored + entirely, and MUST NOT result in a "CMETHOD-ERROR" message. + Thus it is entirely possible for a given PT proxy to + immediately output "CMETHODS DONE". + + - Parent processes MUST handle "CMETHOD"/"CMETHOD-ERROR" + messages in any order, regardless of ordering in + "TOR_PT_CLIENT_TRANSPORTS". +``` + + + +### Pluggable Transport Server Messages + +PT server reverse proxies iterate over the requested transports +in "TOR_PT_CLIENT_TRANSPORTS" and initialize the listeners. + +For each transport initialized, the PT proxy reports the listener +status back to the parent via messages to stdout. + +`SMETHOD [options]` + +The "SMETHOD" message is used to signal that a requested +PT transport has been launched, the protocol which will be +used to handle incoming connections, and the IP address and +port that clients should use to reach the reverse-proxy. + +If there is a specific provided for a given +PT transport via "TOR_PT_SERVER_BINDADDR", the transport +MUST be initialized using that as the server address. + +The OPTIONAL 'options' field is used to pass additional +per-transport information back to the parent process. + +The currently recognized 'options' are: + +`ARGS:[=,]+[=]` + +```text + The "ARGS" option is used to pass additional key/value + formatted information that clients will require to use + the reverse proxy. + + Equal signs and commas MUST be escaped with a backslash. + + Tor: The ARGS are included in the transport line of the + Bridge's extra-info document. + + Examples: + + SMETHOD trebuchet 198.51.100.1:19999 + SMETHOD rot_by_N 198.51.100.1:2323 ARGS:N=13 + + SMETHOD-ERROR +``` + +The "SMETHOD-ERROR" message is used to signal that +requested PT transport reverse proxy was unable to be +launched. + +Example: + +SMETHOD-ERROR trebuchet no cows available + +Once all PT transports have been initialized (or have failed), the +PT proxy MUST send a final message indicating that it has finished +initializing. + +SMETHODS DONE + +The "SMETHODS DONE" message signals that the PT proxy has +finished initializing all of the transports that it is capable +of handling. + +Upon sending the "SMETHODS DONE" message, the PT proxy +initialization is complete. + + + +### Pluggable Transport Log Message + +This message is for a client or server PT to be able to signal back to the +parent process via stdout or stderr any log messages. + +A log message can be any kind of messages (human readable) that the PT +sends back so the parent process can gather information about what is going +on in the child process. It is not intended for the parent process to parse +and act accordingly but rather a message used for plain logging. + +For example, the tor daemon logs those messages at the Severity level and +sends them onto the control port using the PT_LOG (see control-spec.txt) +event so any third party can pick them up for debugging. + +The format of the message: + +`LOG SEVERITY=Severity MESSAGE=Message` + +The SEVERITY value indicate at which logging level the message applies. +The accepted values for `` are: error, warning, notice, info, debug + +The MESSAGE value is a human readable string formatted by the PT. The +`` contains the log message which can be a String or CString (see +section 2 in control-spec.txt). + +Example: + +`LOG SEVERITY=debug MESSAGE="Connected to bridge A"` + + + +### Pluggable Transport Status Message + +This message is for a client or server PT to be able to signal back to the +parent process via stdout or stderr any status messages. + +The format of the message: + +`STATUS TRANSPORT=Transport = [= ...]` + +The TRANSPORT value indicates a hint on what the PT is such has the name or +the protocol used for instance. As an example, obfs4proxy would use +"obfs4". Thus, the Transport value can be anything the PT itself defines +and it can be a String or CString (see section 2 in control-spec.txt). + +The `=` values are specific to the PT and there has to be at least +one. They are messages that reflects the status that the PT wants to +report. `` can be a String or CString. + +Examples (fictional): + +```text +STATUS TRANSPORT=obfs4 ADDRESS=198.51.100.123:1234 CONNECT=Success +STATUS TRANSPORT=obfs4 ADDRESS=198.51.100.222:2222 CONNECT=Failed FINGERPRINT= ERRSTR="Connection refused" +STATUS TRANSPORT=trebuchet ADDRESS=198.51.100.15:443 PERCENT=42 +``` diff --git a/spec/pt-spec/naming.md b/spec/pt-spec/naming.md new file mode 100644 index 0000000..394b420 --- /dev/null +++ b/spec/pt-spec/naming.md @@ -0,0 +1,13 @@ + + +## Pluggable Transport Naming + +Pluggable Transport names serve as unique identifiers, and every +PT MUST have a unique name. + +PT names MUST be valid C identifiers. PT names MUST begin with +a letter or underscore, and the remaining characters MUST be +ASCII letters, numbers or underscores. No length limit is +imposted. + +PT names MUST satisfy the regular expression "`[a-zA-Z_][a-zA-Z0-9_]*`". diff --git a/spec/pt-spec/per-connection-args.md b/spec/pt-spec/per-connection-args.md new file mode 100644 index 0000000..90becab --- /dev/null +++ b/spec/pt-spec/per-connection-args.md @@ -0,0 +1,38 @@ + + +## Pluggable Transport Client Per-Connection Arguments + +Certain PT transport protocols require that the client provides +per-connection arguments when making outgoing connections. On +the server side, this is handled by the "ARGS" optional argument +as part of the "SMETHOD" message. + +On the client side, arguments are passed via the authentication +fields that are part of the SOCKS protocol. + +First the "`=`" formatted arguments MUST be escaped, +such that all backslash, equal sign, and semicolon characters +are escaped with a backslash. + +Second, all of the escaped are concatenated together. + +Example: + +shared-secret=rahasia;secrets-file=/tmp/blob + +Lastly the arguments are transmitted when making the outgoing +connection using the authentication mechanism specific to the +SOCKS protocol version. + +```text + - In the case of SOCKS 4, the concatenated argument list is + transmitted in the "USERID" field of the "CONNECT" request. + + - In the case of SOCKS 5, the parent process must negotiate + "Username/Password" authentication [RFC1929], and transmit + the arguments encoded in the "UNAME" and "PASSWD" fields. +``` + +If the encoded argument list is less than 255 bytes in +length, the "PLEN" field must be set to "1" and the "PASSWD" +field must contain a single NUL character. diff --git a/spec/pt-spec/pluggable-transport-client-per-connection-arguments.md b/spec/pt-spec/pluggable-transport-client-per-connection-arguments.md deleted file mode 100644 index 90becab..0000000 --- a/spec/pt-spec/pluggable-transport-client-per-connection-arguments.md +++ /dev/null @@ -1,38 +0,0 @@ - - -## Pluggable Transport Client Per-Connection Arguments - -Certain PT transport protocols require that the client provides -per-connection arguments when making outgoing connections. On -the server side, this is handled by the "ARGS" optional argument -as part of the "SMETHOD" message. - -On the client side, arguments are passed via the authentication -fields that are part of the SOCKS protocol. - -First the "`=`" formatted arguments MUST be escaped, -such that all backslash, equal sign, and semicolon characters -are escaped with a backslash. - -Second, all of the escaped are concatenated together. - -Example: - -shared-secret=rahasia;secrets-file=/tmp/blob - -Lastly the arguments are transmitted when making the outgoing -connection using the authentication mechanism specific to the -SOCKS protocol version. - -```text - - In the case of SOCKS 4, the concatenated argument list is - transmitted in the "USERID" field of the "CONNECT" request. - - - In the case of SOCKS 5, the parent process must negotiate - "Username/Password" authentication [RFC1929], and transmit - the arguments encoded in the "UNAME" and "PASSWD" fields. -``` - -If the encoded argument list is less than 255 bytes in -length, the "PLEN" field must be set to "1" and the "PASSWD" -field must contain a single NUL character. diff --git a/spec/pt-spec/pluggable-transport-configuration-environment.md b/spec/pt-spec/pluggable-transport-configuration-environment.md deleted file mode 100644 index 79e9f67..0000000 --- a/spec/pt-spec/pluggable-transport-configuration-environment.md +++ /dev/null @@ -1,258 +0,0 @@ - - -## Pluggable Transport Configuration Environment Variables - -All Pluggable Transport proxy instances are configured by their -parent process at launch time via a set of well defined -environment variables. - -The "TOR_PT\_" prefix is used for namespacing reasons and does not -indicate any relations to Tor, except for the origins of this -specification. - - - -### Common Environment Variables - -When launching either a client or server Pluggable Transport proxy, -the following common environment variables MUST be set. - -"TOR_PT_MANAGED_TRANSPORT_VER" - -Specifies the versions of the Pluggable Transport specification -the parent process supports, delimited by commas. All PTs MUST -accept any well-formed list, as long as a compatible version is -present. - -Valid versions MUST consist entirely of non-whitespace, -non-comma printable ASCII characters. - -The version of the Pluggable Transport specification as of this -document is "1". - -Example: - -TOR_PT_MANAGED_TRANSPORT_VER=1,1a,2b,this_is_a_valid_ver - -"TOR_PT_STATE_LOCATION" - -Specifies an absolute path to a directory where the PT is -allowed to store state that will be persisted across -invocations. The directory is not required to exist when -the PT is launched, however PT implementations SHOULD be -able to create it as required. - -PTs MUST only store files in the path provided, and MUST NOT -create or modify files elsewhere on the system. - -Example: - -TOR_PT_STATE_LOCATION=/var/lib/tor/pt_state/ - -"TOR_PT_EXIT_ON_STDIN_CLOSE" - -Specifies that the parent process will close the PT proxy's -standard input (stdin) stream to indicate that the PT proxy -should gracefully exit. - -PTs MUST NOT treat a closed stdin as a signal to terminate -unless this environment variable is set to "1". - -PTs SHOULD treat stdin being closed as a signal to gracefully -terminate if this environment variable is set to "1". - -Example: - -TOR_PT_EXIT_ON_STDIN_CLOSE=1 - -"TOR_PT_OUTBOUND_BIND_ADDRESS_V4" - -Specifies an IPv4 IP address that the PT proxy SHOULD use as source address for -outgoing IPv4 IP packets. This feature allows people with multiple network -interfaces to specify explicitly which interface they prefer the PT proxy to -use. - -If this value is unset or empty, the PT proxy MUST use the default source -address for outgoing connections. - -This setting MUST be ignored for connections to -loopback addresses (127.0.0.0/8). - -Example: - -TOR_PT_OUTBOUND_BIND_ADDRESS_V4=203.0.113.4 - -"TOR_PT_OUTBOUND_BIND_ADDRESS_V6" - -Specifies an IPv6 IP address that the PT proxy SHOULD use as source address for -outgoing IPv6 IP packets. This feature allows people with multiple network -interfaces to specify explicitly which interface they prefer the PT proxy to -use. - -If this value is unset or empty, the PT proxy MUST use the default source -address for outgoing connections. - -This setting MUST be ignored for connections to the loopback address (\[::1\]). - -IPv6 addresses MUST always be wrapped in square brackets. - -Example:: - -TOR_PT_OUTBOUND_BIND_ADDRESS_V6=\[2001:db8::4\] - - - -### Pluggable Transport Client Environment Variables - -Client-side Pluggable Transport forward proxies are configured -via the following environment variables. - -"TOR_PT_CLIENT_TRANSPORTS" - -Specifies the PT protocols the client proxy should initialize, -as a comma separated list of PT names. - -PTs SHOULD ignore PT names that it does not recognize. - -Parent processes MUST set this environment variable when -launching a client-side PT proxy instance. - -Example: - -TOR_PT_CLIENT_TRANSPORTS=obfs2,obfs3,obfs4 - -"TOR_PT_PROXY" - -Specifies an upstream proxy that the PT MUST use when making -outgoing network connections. It is a URI \[RFC3986\] of the -format: - -`://[[:][@]:`. - -The "TOR_PT_PROXY" environment variable is OPTIONAL and -MUST be omitted if there is no need to connect via an -upstream proxy. - -Examples: - -```text - TOR_PT_PROXY=socks5://tor:test1234@198.51.100.1:8000 - TOR_PT_PROXY=socks4a://198.51.100.2:8001 - TOR_PT_PROXY=http://198.51.100.3:443 -``` - - - -### Pluggable Transport Server Environment Variables - -Server-side Pluggable Transport reverse proxies are configured -via the following environment variables. - -"TOR_PT_SERVER_TRANSPORTS" - -Specifies the PT protocols the server proxy should initialize, -as a comma separated list of PT names. - -PTs SHOULD ignore PT names that it does not recognize. - -Parent processes MUST set this environment variable when -launching a server-side PT reverse proxy instance. - -Example: - -TOR_PT_SERVER_TRANSPORTS=obfs3,scramblesuit - -"TOR_PT_SERVER_TRANSPORT_OPTIONS" - -Specifies per-PT protocol configuration directives, as a -semicolon-separated list of `:` pairs, where `` -is a PT name and `` is a k=v string value with options -that are to be passed to the transport. - -Colons, semicolons, and backslashes MUST be -escaped with a backslash. - -If there are no arguments that need to be passed to any of -PT transport protocols, "TOR_PT_SERVER_TRANSPORT_OPTIONS" -MAY be omitted. - -Example: - -TOR_PT_SERVER_TRANSPORT_OPTIONS=scramblesuit:key=banana;automata:rule=110;automata:depth=3 - -```text - Will pass to 'scramblesuit' the parameter 'key=banana' and to - 'automata' the arguments 'rule=110' and 'depth=3'. - - "TOR_PT_SERVER_BINDADDR" -``` - -A comma separated list of `-` pairs, where `` is -a PT name and `` is the `
:` on which it -should listen for incoming client connections. - -The keys holding transport names MUST be in the same order as -they appear in "TOR_PT_SERVER_TRANSPORTS". - -The `
` MAY be a locally scoped address as long as port -forwarding is done externally. - -The `
:` combination MUST be an IP address -supported by `bind()`, and MUST NOT be a host name. - -Applications MUST NOT set more than one `
:` pair -per PT name. - -If there is no specific `
:` combination to be -configured for any transports, "TOR_PT_SERVER_BINDADDR" MAY -be omitted. - -Example: - -TOR_PT_SERVER_BINDADDR=obfs3-198.51.100.1:1984,scramblesuit-127.0.0.1:4891 - -"TOR_PT_ORPORT" - -Specifies the destination that the PT reverse proxy should forward -traffic to after transforming it as appropriate, as an -`
:`. - -Connections to the destination specified via "TOR_PT_ORPORT" -MUST only contain application payload. If the parent process -requires the actual source IP address of client connections -(or other metadata), it should set "TOR_PT_EXTENDED_SERVER_PORT" -instead. - -Example: - -TOR_PT_ORPORT=127.0.0.1:9001 - -"TOR_PT_EXTENDED_SERVER_PORT" - -Specifies the destination that the PT reverse proxy should -forward traffic to, via the Extended ORPort protocol \[EXTORPORT\] -as an `
:`. - -The Extended ORPort protocol allows the PT reverse proxy to -communicate per-connection metadata such as the PT name and -client IP address/port to the parent process. - -If the parent process does not support the ExtORPort protocol, -it MUST set "TOR_PT_EXTENDED_SERVER_PORT" to an empty string. - -Example: - -TOR_PT_EXTENDED_SERVER_PORT=127.0.0.1:4200 - -"TOR_PT_AUTH_COOKIE_FILE" - -Specifies an absolute filesystem path to the Extended ORPort -authentication cookie, required to communicate with the -Extended ORPort specified via "TOR_PT_EXTENDED_SERVER_PORT". - -If the parent process is not using the ExtORPort protocol for -incoming traffic, "TOR_PT_AUTH_COOKIE_FILE" MUST be omitted. - -Example: - -TOR_PT_AUTH_COOKIE_FILE=/var/lib/tor/extended_orport_auth_cookie diff --git a/spec/pt-spec/pluggable-transport-naming.md b/spec/pt-spec/pluggable-transport-naming.md deleted file mode 100644 index 394b420..0000000 --- a/spec/pt-spec/pluggable-transport-naming.md +++ /dev/null @@ -1,13 +0,0 @@ - - -## Pluggable Transport Naming - -Pluggable Transport names serve as unique identifiers, and every -PT MUST have a unique name. - -PT names MUST be valid C identifiers. PT names MUST begin with -a letter or underscore, and the remaining characters MUST be -ASCII letters, numbers or underscores. No length limit is -imposted. - -PT names MUST satisfy the regular expression "`[a-zA-Z_][a-zA-Z0-9_]*`". diff --git a/spec/pt-spec/pluggable-transport-shutdown.md b/spec/pt-spec/pluggable-transport-shutdown.md deleted file mode 100644 index cd826d9..0000000 --- a/spec/pt-spec/pluggable-transport-shutdown.md +++ /dev/null @@ -1,32 +0,0 @@ - - -## Pluggable Transport Shutdown - -The recommended way for Pluggable Transport using applications and -Pluggable Transports to handle graceful shutdown is as follows. - -```text - - (Parent) Set "TOR_PT_EXIT_ON_STDIN_CLOSE" (3.2.1) when - launching the PT proxy, to indicate that stdin will be used - for graceful shutdown notification. - - - (Parent) When the time comes to terminate the PT proxy: - - 1. Close the PT proxy's stdin. - 2. Wait for a "reasonable" amount of time for the PT to exit. - 3. Attempt to use OS specific mechanisms to cause graceful - PT shutdown (eg: 'SIGTERM') - 4. Use OS specific mechanisms to force terminate the PT - (eg: 'SIGKILL', 'ProccessTerminate()'). - - - PT proxies SHOULD monitor stdin, and exit gracefully when - it is closed, if the parent supports that behavior. - - - PT proxies SHOULD handle OS specific mechanisms to gracefully - terminate (eg: Install a signal handler on 'SIGTERM' that - causes cleanup and a graceful shutdown if able). - - - PT proxies SHOULD attempt to detect when the parent has - terminated (eg: via detecting that its parent process ID has - changed on U*IX systems), and gracefully terminate. -``` diff --git a/spec/pt-spec/pluggable-transport-to-parent-process-communication.md b/spec/pt-spec/pluggable-transport-to-parent-process-communication.md deleted file mode 100644 index 21fd18b..0000000 --- a/spec/pt-spec/pluggable-transport-to-parent-process-communication.md +++ /dev/null @@ -1,303 +0,0 @@ - - -## Pluggable Transport To Parent Process Communication - -All Pluggable Transport Proxies communicate to the parent process -via writing NL-terminated lines to stdout. The line metaformat is: - -```text - ::= - ::= | - ::= - ::= * - ::= | - ::= - ::= - ::= -``` - -The parent process MUST ignore lines received from PT proxies with -unknown keywords. - - - -### Common Messages - -When a PT proxy first starts up, it must determine which version -of the Pluggable Transports Specification to use to configure -itself. - -It does this via the "TOR_PT_MANAGED_TRANSPORT_VER" (3.2.1) -environment variable which contains all of the versions supported -by the application. - -Upon determining the version to use, or lack thereof, the PT -proxy responds with one of two messages. - -`VERSION-ERROR ` - -The "VERSION-ERROR" message is used to signal that there was -no compatible Pluggable Transport Specification version -present in the "TOR_PT_MANAGED_TRANSPORT_VER" list. - -The `` SHOULD be set to "no-version" for -historical reasons but MAY be set to a useful error message -instead. - -PT proxies MUST terminate after outputting a "VERSION-ERROR" -message. - -Example: - -`VERSION-ERROR no-version` - -`VERSION ` - -The "VERSION" message is used to signal the Pluggable Transport -Specification version (as in "TOR_PT_MANAGED_TRANSPORT_VER") -that the PT proxy will use to configure its transports and -communicate with the parent process. - -The version for the environment values and reply messages -specified by this document is "1". - -PT proxies MUST either report an error and terminate, or output -a "VERSION" message before moving on to client/server proxy -initialization and configuration. - -Example: - -VERSION 1 - -After version negotiation has been completed the PT proxy must -then validate that all of the required environment variables are -provided, and that all of the configuration values supplied are -well formed. - -At any point, if there is an error encountered related to -configuration supplied via the environment variables, it MAY -respond with an error message and terminate. - -`ENV-ERROR ` - -The "ENV-ERROR" message is used to signal the PT proxy's -failure to parse the configuration environment variables (3.2). - -The `` SHOULD consist of a useful error message -that can be used to diagnose and correct the root cause of -the failure. - -PT proxies MUST terminate after outputting a "ENV-ERROR" -message. - -Example: - -ENV-ERROR No TOR_PT_AUTH_COOKIE_FILE when TOR_PT_EXTENDED_SERVER_PORT set - - - -### Pluggable Transport Client Messages - -After negotiating the Pluggable Transport Specification version, -PT client proxies MUST first validate "TOR_PT_PROXY" (3.2.2) if -it is set, before initializing any transports. - -Assuming that an upstream proxy is provided, PT client proxies -MUST respond with a message indicating that the proxy is valid, -supported, and will be used OR a failure message. - -PROXY DONE - -The "PROXY DONE" message is used to signal the PT proxy's -acceptance of the upstream proxy specified by "TOR_PT_PROXY". - -`PROXY-ERROR ` - -The "PROXY-ERROR" message is used to signal that the upstream -proxy is malformed/unsupported or otherwise unusable. - -PT proxies MUST terminate immediately after outputting a -"PROXY-ERROR" message. - -Example: - -PROXY-ERROR SOCKS 4 upstream proxies unsupported. - -After the upstream proxy (if any) is configured, PT clients then -iterate over the requested transports in "TOR_PT_CLIENT_TRANSPORTS" -and initialize the listeners. - -For each transport initialized, the PT proxy reports the listener -status back to the parent via messages to stdout. - -`CMETHOD <'socks4','socks5'> ` - -The "CMETHOD" message is used to signal that a requested -PT transport has been launched, the protocol which the parent -should use to make outgoing connections, and the IP address -and port that the PT transport's forward proxy is listening on. - -Example: - -`CMETHOD trebuchet socks5 127.0.0.1:19999` - -`CMETHOD-ERROR ` - -The "CMETHOD-ERROR" message is used to signal that -requested PT transport was unable to be launched. - -Example: - -CMETHOD-ERROR trebuchet no rocks available - -Once all PT transports have been initialized (or have failed), the -PT proxy MUST send a final message indicating that it has finished -initializing. - -CMETHODS DONE - -The "CMETHODS DONE" message signals that the PT proxy has -finished initializing all of the transports that it is capable -of handling. - -Upon sending the "CMETHODS DONE" message, the PT proxy -initialization is complete. - -Notes: - -```text - - Unknown transports in "TOR_PT_CLIENT_TRANSPORTS" are ignored - entirely, and MUST NOT result in a "CMETHOD-ERROR" message. - Thus it is entirely possible for a given PT proxy to - immediately output "CMETHODS DONE". - - - Parent processes MUST handle "CMETHOD"/"CMETHOD-ERROR" - messages in any order, regardless of ordering in - "TOR_PT_CLIENT_TRANSPORTS". -``` - - - -### Pluggable Transport Server Messages - -PT server reverse proxies iterate over the requested transports -in "TOR_PT_CLIENT_TRANSPORTS" and initialize the listeners. - -For each transport initialized, the PT proxy reports the listener -status back to the parent via messages to stdout. - -`SMETHOD [options]` - -The "SMETHOD" message is used to signal that a requested -PT transport has been launched, the protocol which will be -used to handle incoming connections, and the IP address and -port that clients should use to reach the reverse-proxy. - -If there is a specific provided for a given -PT transport via "TOR_PT_SERVER_BINDADDR", the transport -MUST be initialized using that as the server address. - -The OPTIONAL 'options' field is used to pass additional -per-transport information back to the parent process. - -The currently recognized 'options' are: - -`ARGS:[=,]+[=]` - -```text - The "ARGS" option is used to pass additional key/value - formatted information that clients will require to use - the reverse proxy. - - Equal signs and commas MUST be escaped with a backslash. - - Tor: The ARGS are included in the transport line of the - Bridge's extra-info document. - - Examples: - - SMETHOD trebuchet 198.51.100.1:19999 - SMETHOD rot_by_N 198.51.100.1:2323 ARGS:N=13 - - SMETHOD-ERROR -``` - -The "SMETHOD-ERROR" message is used to signal that -requested PT transport reverse proxy was unable to be -launched. - -Example: - -SMETHOD-ERROR trebuchet no cows available - -Once all PT transports have been initialized (or have failed), the -PT proxy MUST send a final message indicating that it has finished -initializing. - -SMETHODS DONE - -The "SMETHODS DONE" message signals that the PT proxy has -finished initializing all of the transports that it is capable -of handling. - -Upon sending the "SMETHODS DONE" message, the PT proxy -initialization is complete. - - - -### Pluggable Transport Log Message - -This message is for a client or server PT to be able to signal back to the -parent process via stdout or stderr any log messages. - -A log message can be any kind of messages (human readable) that the PT -sends back so the parent process can gather information about what is going -on in the child process. It is not intended for the parent process to parse -and act accordingly but rather a message used for plain logging. - -For example, the tor daemon logs those messages at the Severity level and -sends them onto the control port using the PT_LOG (see control-spec.txt) -event so any third party can pick them up for debugging. - -The format of the message: - -`LOG SEVERITY=Severity MESSAGE=Message` - -The SEVERITY value indicate at which logging level the message applies. -The accepted values for `` are: error, warning, notice, info, debug - -The MESSAGE value is a human readable string formatted by the PT. The -`` contains the log message which can be a String or CString (see -section 2 in control-spec.txt). - -Example: - -`LOG SEVERITY=debug MESSAGE="Connected to bridge A"` - - - -### Pluggable Transport Status Message - -This message is for a client or server PT to be able to signal back to the -parent process via stdout or stderr any status messages. - -The format of the message: - -`STATUS TRANSPORT=Transport = [= ...]` - -The TRANSPORT value indicates a hint on what the PT is such has the name or -the protocol used for instance. As an example, obfs4proxy would use -"obfs4". Thus, the Transport value can be anything the PT itself defines -and it can be a String or CString (see section 2 in control-spec.txt). - -The `=` values are specific to the PT and there has to be at least -one. They are messages that reflects the status that the PT wants to -report. `` can be a String or CString. - -Examples (fictional): - -```text -STATUS TRANSPORT=obfs4 ADDRESS=198.51.100.123:1234 CONNECT=Success -STATUS TRANSPORT=obfs4 ADDRESS=198.51.100.222:2222 CONNECT=Failed FINGERPRINT= ERRSTR="Connection refused" -STATUS TRANSPORT=trebuchet ADDRESS=198.51.100.15:443 PERCENT=42 -``` diff --git a/spec/pt-spec/shutdown.md b/spec/pt-spec/shutdown.md new file mode 100644 index 0000000..cd826d9 --- /dev/null +++ b/spec/pt-spec/shutdown.md @@ -0,0 +1,32 @@ + + +## Pluggable Transport Shutdown + +The recommended way for Pluggable Transport using applications and +Pluggable Transports to handle graceful shutdown is as follows. + +```text + - (Parent) Set "TOR_PT_EXIT_ON_STDIN_CLOSE" (3.2.1) when + launching the PT proxy, to indicate that stdin will be used + for graceful shutdown notification. + + - (Parent) When the time comes to terminate the PT proxy: + + 1. Close the PT proxy's stdin. + 2. Wait for a "reasonable" amount of time for the PT to exit. + 3. Attempt to use OS specific mechanisms to cause graceful + PT shutdown (eg: 'SIGTERM') + 4. Use OS specific mechanisms to force terminate the PT + (eg: 'SIGKILL', 'ProccessTerminate()'). + + - PT proxies SHOULD monitor stdin, and exit gracefully when + it is closed, if the parent supports that behavior. + + - PT proxies SHOULD handle OS specific mechanisms to gracefully + terminate (eg: Install a signal handler on 'SIGTERM' that + causes cleanup and a graceful shutdown if able). + + - PT proxies SHOULD attempt to detect when the parent has + terminated (eg: via detecting that its parent process ID has + changed on U*IX systems), and gracefully terminate. +``` diff --git a/spec/rend-spec-v3/client-authorization.md b/spec/rend-spec-v3/client-authorization.md new file mode 100644 index 0000000..1cffce4 --- /dev/null +++ b/spec/rend-spec-v3/client-authorization.md @@ -0,0 +1,105 @@ + + +# Appendix G: Managing authorized client data \[CLIENT-AUTH-MGMT\] + +Hidden services and clients can configure their authorized client data either +using the torrc, or using the control port. This section presents a suggested +scheme for configuring client authorization. Please see appendix +\[HIDSERVDIR-FORMAT\] for more information about relevant hidden service files. + +(NOTE: client authorization is implemented as of 0.3.5.1-alpha.) + +G.1. Configuring client authorization using torrc + +G.1.1. Hidden Service side configuration + +```text + A hidden service that wants to enable client authorization, needs to + populate the "authorized_clients/" directory of its HiddenServiceDir + directory with the ".auth" files of its authorized clients. + + When Tor starts up with a configured onion service, Tor checks its + /authorized_clients/ directory for ".auth" files, and if + any recognized and parseable such files are found, then client + authorization becomes activated for that service. + + G.1.2. Service-side bookkeeping + + This section contains more details on how onion services should be keeping + track of their client ".auth" files. + + For the "descriptor" authentication type, the ".auth" file MUST contain + the x25519 public key of that client. Here is a suggested file format: + + :: + + Here is an an example: + + descriptor:x25519:OM7TGIVRYMY6PFX6GAC6ATRTA5U6WW6U7A4ZNHQDI6OVL52XVV2Q + + Tor SHOULD ignore lines it does not recognize. + Tor SHOULD ignore files that don't use the ".auth" suffix. + + G.1.3. Client side configuration + + A client who wants to register client authorization data for onion + services needs to add the following line to their torrc to indicate the + directory which hosts ".auth_private" files containing client-side + credentials for onion services: + + ClientOnionAuthDir + + The contains a file with the suffix ".auth_private" for each onion + service the client is authorized with. Tor should scan the directory for + ".auth_private" files to find which onion services require client + authorization from this client. + + For the "descriptor" auth-type, a ".auth_private" file contains the + private x25519 key: + + :descriptor:x25519: + + The keypair used for client authorization is created by a third party tool + for which the public key needs to be transferred to the service operator + in a secure out-of-band way. The third party tool SHOULD add appropriate + headers to the private key file to ensure that users won't accidentally + give out their private key. + + G.2. Configuring client authorization using the control port + + G.2.1. Service side + + A hidden service also has the option to configure authorized clients + using the control port. The idea is that hidden service operators can use + controller utilities that manage their access control instead of using + the filesystem to register client keys. + + Specifically, we require a new control port command ADD_ONION_CLIENT_AUTH + which is able to register x25519/ed25519 public keys tied to a specific + authorized client. + [XXX figure out control port command format] + + Hidden services who use the control port interface for client auth need + to perform their own key management. + + G.2.2. Client side + + There should also be a control port interface for clients to register + authorization data for hidden services without having to use the + torrc. It should allow both generation of client authorization private + keys, and also to import client authorization data provided by a hidden + service + + This way, Tor Browser can present "Generate client auth keys" and "Import + client auth keys" dialogs to users when they try to visit a hidden service + that is protected by client authorization. + + Specifically, we require two new control port commands: + IMPORT_ONION_CLIENT_AUTH_DATA + GENERATE_ONION_CLIENT_AUTH_DATA + which import and generate client authorization data respectively. + + [XXX how does key management work here?] + [XXX what happens when people use both the control port interface and the + filesystem interface?] +``` diff --git a/spec/rend-spec-v3/deriving-blinded-keys-subcredentials-subcred.md b/spec/rend-spec-v3/deriving-blinded-keys-subcredentials-subcred.md deleted file mode 100644 index 201fc92..0000000 --- a/spec/rend-spec-v3/deriving-blinded-keys-subcredentials-subcred.md +++ /dev/null @@ -1,418 +0,0 @@ - - -## Deriving blinded keys and subcredentials \[SUBCRED\] - -In each time period (see \[TIME-PERIODS\] for a definition of time -periods), a hidden service host uses a different blinded private key -to sign its directory information, and clients use a different -blinded public key as the index for fetching that information. - -For a candidate for a key derivation method, see Appendix \[KEYBLIND\]. - -Additionally, clients and hosts derive a subcredential for each -period. Knowledge of the subcredential is needed to decrypt hidden -service descriptors for each period and to authenticate with the -hidden service host in the introduction process. Unlike the -credential, it changes each period. Knowing the subcredential, even -in combination with the blinded private key, does not enable the -hidden service host to derive the main credential--therefore, it is -safe to put the subcredential on the hidden service host while -leaving the hidden service's private key offline. - -The subcredential for a period is derived as: - -N_hs_subcred = H("subcredential" | N_hs_cred | blinded-public-key). - -In the above formula, credential corresponds to: - -N_hs_cred = H("credential" | public-identity-key) - -where public-identity-key is the public identity master key of the hidden -service. - -```text -2.2. Locating, uploading, and downloading hidden service descriptors - [HASHRING] -``` - -To avoid attacks where a hidden service's descriptor is easily -targeted for censorship, we store them at different directories over -time, and use shared random values to prevent those directories from -being predictable far in advance. - -Which Tor servers hosts a hidden service depends on: - -```text - * the current time period, - * the daily subcredential, - * the hidden service directories' public keys, - * a shared random value that changes in each time period, - shared_random_value. - * a set of network-wide networkstatus consensus parameters. - (Consensus parameters are integer values voted on by authorities - and published in the consensus documents, described in - dir-spec.txt, section 3.3.) - - Below we explain in more detail. -``` - - - -### Dividing time into periods \[TIME-PERIODS\] - -To prevent a single set of hidden service directory from becoming a -target by adversaries looking to permanently censor a hidden service, -hidden service descriptors are uploaded to different locations that -change over time. - -The length of a "time period" is controlled by the consensus -parameter 'hsdir-interval', and is a number of minutes between 30 and -14400 (10 days). The default time period length is 1440 (one day). - -Time periods start at the Unix epoch (Jan 1, 1970), and are computed by -taking the number of minutes since the epoch and dividing by the time -period. However, we want our time periods to start at a regular offset -from the SRV voting schedule, so we subtract a "rotation time offset" -of 12 voting periods from the number of minutes since the epoch, before -dividing by the time period (effectively making "our" epoch start at Jan -1, 1970 12:00UTC when the voting period is 1 hour.) - -Example: If the current time is 2016-04-13 11:15:01 UTC, making the seconds -since the epoch 1460546101, and the number of minutes since the epoch -24342435\. We then subtract the "rotation time offset" of 12\*60 minutes from -the minutes since the epoch, to get 24341715. If the current time period -length is 1440 minutes, by doing the division we see that we are currently -in time period number 16903. - -Specifically, time period #16903 began 16903*1440*60 + (12*60*60) seconds -after the epoch, at 2016-04-12 12:00 UTC, and ended at 16904*1440*60 + -(12*60*60) seconds after the epoch, at 2016-04-13 12:00 UTC. - - - -### When to publish a hidden service descriptor \[WHEN-HSDESC\] - -Hidden services periodically publish their descriptor to the responsible -HSDirs. The set of responsible HSDirs is determined as specified in -\[WHERE-HSDESC\]. - -Specifically, every time a hidden service publishes its descriptor, it also -sets up a timer for a random time between 60 minutes and 120 minutes in the -future. When the timer triggers, the hidden service needs to publish its -descriptor again to the responsible HSDirs for that time period. -\[TODO: Control republish period using a consensus parameter?\] - - - -#### Overlapping descriptors - -Hidden services need to upload multiple descriptors so that they can be -reachable to clients with older or newer consensuses than them. Services -need to upload their descriptors to the HSDirs *before* the beginning of -each upcoming time period, so that they are readily available for clients to -fetch them. Furthermore, services should keep uploading their old descriptor -even after the end of a time period, so that they can be reachable by -clients that still have consensuses from the previous time period. - -Hence, services maintain two active descriptors at every point. Clients on -the other hand, don't have a notion of overlapping descriptors, and instead -always download the descriptor for the current time period and shared random -value. It's the job of the service to ensure that descriptors will be -available for all clients. See section \[FETCHUPLOADDESC\] for how this is -achieved. - -\[TODO: What to do when we run multiple hidden services in a single host?\] - - - -### Where to publish a hidden service descriptor \[WHERE-HSDESC\] - -This section specifies how the HSDir hash ring is formed at any given -time. Whenever a time value is needed (e.g. to get the current time period -number), we assume that clients and services use the valid-after time from -their latest live consensus. - -The following consensus parameters control where a hidden service -descriptor is stored; - -```text - hsdir_n_replicas = an integer in range [1,16] with default value 2. - hsdir_spread_fetch = an integer in range [1,128] with default value 3. - hsdir_spread_store = an integer in range [1,128] with default value 4. - (Until 0.3.2.8-rc, the default was 3.) -``` - -To determine where a given hidden service descriptor will be stored -in a given period, after the blinded public key for that period is -derived, the uploading or downloading party calculates: - -```text - for replicanum in 1...hsdir_n_replicas: - hs_service_index(replicanum) = H("store-at-idx" | - blinded_public_key | - INT_8(replicanum) | - INT_8(period_length) | - INT_8(period_num) ) -``` - -where blinded_public_key is specified in section \[KEYBLIND\], period_length -is the length of the time period in minutes, and period_num is calculated -using the current consensus "valid-after" as specified in section -\[TIME-PERIODS\]. - -Then, for each node listed in the current consensus with the HSDir flag, -we compute a directory index for that node as: - -```text - hs_relay_index(node) = H("node-idx" | node_identity | - shared_random_value | - INT_8(period_num) | - INT_8(period_length) ) -``` - -where shared_random_value is the shared value generated by the authorities -in section \[PUB-SHAREDRANDOM\], and node_identity is the ed25519 identity -key of the node. - -Finally, for replicanum in 1...hsdir_n_replicas, the hidden service -host uploads descriptors to the first hsdir_spread_store nodes whose -indices immediately follow hs_service_index(replicanum). If any of those -nodes have already been selected for a lower-numbered replica of the -service, any nodes already chosen are disregarded (i.e. skipped over) -when choosing a replica's hsdir_spread_store nodes. - -When choosing an HSDir to download from, clients choose randomly from -among the first hsdir_spread_fetch nodes after the indices. (Note -that, in order to make the system better tolerate disappearing -HSDirs, hsdir_spread_fetch may be less than hsdir_spread_store.) -Again, nodes from lower-numbered replicas are disregarded when -choosing the spread for a replica. - - - -### Using time periods and SRVs to fetch/upload HS descriptors \[FETCHUPLOADDESC\] - -Hidden services and clients need to make correct use of time periods (TP) -and shared random values (SRVs) to successfully fetch and upload -descriptors. Furthermore, to avoid problems with skewed clocks, both clients -and services use the 'valid-after' time of a live consensus as a way to take -decisions with regards to uploading and fetching descriptors. By using the -consensus times as the ground truth here, we minimize the desynchronization -of clients and services due to system clock. Whenever time-based decisions -are taken in this section, assume that they are consensus times and not -system times. - -As \[PUB-SHAREDRANDOM\] specifies, consensuses contain two shared random -values (the current one and the previous one). Hidden services and clients -are asked to match these shared random values with descriptor time periods -and use the right SRV when fetching/uploading descriptors. This section -attempts to precisely specify how this works. - -Let's start with an illustration of the system: - -```text - +------------------------------------------------------------------+ - | | - | 00:00 12:00 00:00 12:00 00:00 12:00 | - | SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | - | | - | $==========|-----------$===========|-----------$===========| | - | | - | | - +------------------------------------------------------------------+ - - Legend: [TP#1 = Time Period #1] - [SRV#1 = Shared Random Value #1] - ["$" = descriptor rotation moment] -``` - - - -#### Client behavior for fetching descriptors \[CLIENTFETCH\] - -And here is how clients use TPs and SRVs to fetch descriptors: - -Clients always aim to synchronize their TP with SRV, so they always want to -use TP#N with SRV#N: To achieve this wrt time periods, clients always use -the current time period when fetching descriptors. Now wrt SRVs, if a client -is in the time segment between a new time period and a new SRV (i.e. the -segments drawn with "-") it uses the current SRV, else if the client is in a -time segment between a new SRV and a new time period (i.e. the segments -drawn with "="), it uses the previous SRV. - -Example: - -+------------------------------------------------------------------+ -| | -| 00:00 12:00 00:00 12:00 00:00 12:00 | -| SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | -| | -| $==========|-----------$===========|-----------$===========| | -| ^ ^ | -| C1 C2 | -+------------------------------------------------------------------+ - -If a client (C1) is at 13:00 right after TP#1, then it will use TP#1 and -SRV#1 for fetching descriptors. Also, if a client (C2) is at 01:00 right -after SRV#2, it will still use TP#1 and SRV#1. - - - -#### Service behavior for uploading descriptors \[SERVICEUPLOAD\] - -As discussed above, services maintain two active descriptors at any time. We -call these the "first" and "second" service descriptors. Services rotate -their descriptor every time they receive a consensus with a valid_after time -past the next SRV calculation time. They rotate their descriptors by -discarding their first descriptor, pushing the second descriptor to the -first, and rebuilding their second descriptor with the latest data. - -Services like clients also employ a different logic for picking SRV and TP -values based on their position in the graph above. Here is the logic: - - - -##### First descriptor upload logic \[FIRSTDESCUPLOAD\] - -Here is the service logic for uploading its first descriptor: - -When a service is in the time segment between a new time period a new SRV -(i.e. the segments drawn with "-"), it uses the previous time period and -previous SRV for uploading its first descriptor: that's meant to cover -for clients that have a consensus that is still in the previous time period. - -Example: Consider in the above illustration that the service is at 13:00 -right after TP#1. It will upload its first descriptor using TP#0 and SRV#0. -So if a client still has a 11:00 consensus it will be able to access it -based on the client logic above. - -Now if a service is in the time segment between a new SRV and a new time -period (i.e. the segments drawn with "=") it uses the current time period -and the previous SRV for its first descriptor: that's meant to cover clients -with an up-to-date consensus in the same time period as the service. - -Example: - -+------------------------------------------------------------------+ -| | -| 00:00 12:00 00:00 12:00 00:00 12:00 | -| SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | -| | -| $==========|-----------$===========|-----------$===========| | -| ^ | -| S | -+------------------------------------------------------------------+ - -Consider that the service is at 01:00 right after SRV#2: it will upload its -first descriptor using TP#1 and SRV#1. - - - -##### Second descriptor upload logic \[SECONDDESCUPLOAD\] - -Here is the service logic for uploading its second descriptor: - -When a service is in the time segment between a new time period a new SRV -(i.e. the segments drawn with "-"), it uses the current time period and -current SRV for uploading its second descriptor: that's meant to cover for -clients that have an up-to-date consensus on the same TP as the service. - -Example: Consider in the above illustration that the service is at 13:00 -right after TP#1: it will upload its second descriptor using TP#1 and SRV#1. - -Now if a service is in the time segment between a new SRV and a new time -period (i.e. the segments drawn with "=") it uses the next time period and -the current SRV for its second descriptor: that's meant to cover clients -with a newer consensus than the service (in the next time period). - -Example: - -+------------------------------------------------------------------+ -| | -| 00:00 12:00 00:00 12:00 00:00 12:00 | -| SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | -| | -| $==========|-----------$===========|-----------$===========| | -| ^ | -| S | -+------------------------------------------------------------------+ - -Consider that the service is at 01:00 right after SRV#2: it will upload its -second descriptor using TP#2 and SRV#2. - - - -#### Directory behavior for handling descriptor uploads \[DIRUPLOAD\] - -Upon receiving a hidden service descriptor publish request, directories MUST -check the following: - -```text - * The outer wrapper of the descriptor can be parsed according to - [DESC-OUTER] - * The version-number of the descriptor is "3" - * If the directory has already cached a descriptor for this hidden service, - the revision-counter of the uploaded descriptor must be greater than the - revision-counter of the cached one - * The descriptor signature is valid -``` - -If any of these basic validity checks fails, the directory MUST reject the -descriptor upload. - -NOTE: Even if the descriptor passes the checks above, its first and second -layers could still be invalid: directories cannot validate the encrypted -layers of the descriptor, as they do not have access to the public key of the -service (required for decrypting the first layer of encryption), or the -necessary client credentials (for decrypting the second layer). - - - -### Expiring hidden service descriptors \[EXPIRE-DESC\] - -Hidden services set their descriptor's "descriptor-lifetime" field to 180 -minutes (3 hours). Hidden services ensure that their descriptor will remain -valid in the HSDir caches, by republishing their descriptors periodically as -specified in \[WHEN-HSDESC\]. - -Hidden services MUST also keep their introduction circuits alive for as long -as descriptors including those intro points are valid (even if that's after -the time period has changed). - - - -### URLs for anonymous uploading and downloading - -Hidden service descriptors conforming to this specification are uploaded -with an HTTP POST request to the URL `/tor/hs//publish` relative to -the hidden service directory's root, and downloaded with an HTTP GET -request for the URL `/tor/hs//` where `` is a base64 encoding of -the hidden service's blinded public key and `` is the protocol -version which is "3" in this case. - -These requests must be made anonymously, on circuits not used for -anything else. - - - -### Client-side validation of onion addresses - -When a Tor client receives a prop224 onion address from the user, it -MUST first validate the onion address before attempting to connect or -fetch its descriptor. If the validation fails, the client MUST -refuse to connect. - -As part of the address validation, Tor clients should check that the -underlying ed25519 key does not have a torsion component. If Tor accepted -ed25519 keys with torsion components, attackers could create multiple -equivalent onion addresses for a single ed25519 key, which would map to the -same service. We want to avoid that because it could lead to phishing -attacks and surprising behaviors (e.g. imagine a browser plugin that blocks -onion addresses, but could be bypassed using an equivalent onion address -with a torsion component). - -The right way for clients to detect such fraudulent addresses (which should -only occur malevolently and never naturally) is to extract the ed25519 -public key from the onion address and multiply it by the ed25519 group order -and ensure that the result is the ed25519 identity element. For more -details, please see \[TORSION-REFS\]. diff --git a/spec/rend-spec-v3/deriving-keys.md b/spec/rend-spec-v3/deriving-keys.md new file mode 100644 index 0000000..201fc92 --- /dev/null +++ b/spec/rend-spec-v3/deriving-keys.md @@ -0,0 +1,418 @@ + + +## Deriving blinded keys and subcredentials \[SUBCRED\] + +In each time period (see \[TIME-PERIODS\] for a definition of time +periods), a hidden service host uses a different blinded private key +to sign its directory information, and clients use a different +blinded public key as the index for fetching that information. + +For a candidate for a key derivation method, see Appendix \[KEYBLIND\]. + +Additionally, clients and hosts derive a subcredential for each +period. Knowledge of the subcredential is needed to decrypt hidden +service descriptors for each period and to authenticate with the +hidden service host in the introduction process. Unlike the +credential, it changes each period. Knowing the subcredential, even +in combination with the blinded private key, does not enable the +hidden service host to derive the main credential--therefore, it is +safe to put the subcredential on the hidden service host while +leaving the hidden service's private key offline. + +The subcredential for a period is derived as: + +N_hs_subcred = H("subcredential" | N_hs_cred | blinded-public-key). + +In the above formula, credential corresponds to: + +N_hs_cred = H("credential" | public-identity-key) + +where public-identity-key is the public identity master key of the hidden +service. + +```text +2.2. Locating, uploading, and downloading hidden service descriptors + [HASHRING] +``` + +To avoid attacks where a hidden service's descriptor is easily +targeted for censorship, we store them at different directories over +time, and use shared random values to prevent those directories from +being predictable far in advance. + +Which Tor servers hosts a hidden service depends on: + +```text + * the current time period, + * the daily subcredential, + * the hidden service directories' public keys, + * a shared random value that changes in each time period, + shared_random_value. + * a set of network-wide networkstatus consensus parameters. + (Consensus parameters are integer values voted on by authorities + and published in the consensus documents, described in + dir-spec.txt, section 3.3.) + + Below we explain in more detail. +``` + + + +### Dividing time into periods \[TIME-PERIODS\] + +To prevent a single set of hidden service directory from becoming a +target by adversaries looking to permanently censor a hidden service, +hidden service descriptors are uploaded to different locations that +change over time. + +The length of a "time period" is controlled by the consensus +parameter 'hsdir-interval', and is a number of minutes between 30 and +14400 (10 days). The default time period length is 1440 (one day). + +Time periods start at the Unix epoch (Jan 1, 1970), and are computed by +taking the number of minutes since the epoch and dividing by the time +period. However, we want our time periods to start at a regular offset +from the SRV voting schedule, so we subtract a "rotation time offset" +of 12 voting periods from the number of minutes since the epoch, before +dividing by the time period (effectively making "our" epoch start at Jan +1, 1970 12:00UTC when the voting period is 1 hour.) + +Example: If the current time is 2016-04-13 11:15:01 UTC, making the seconds +since the epoch 1460546101, and the number of minutes since the epoch +24342435\. We then subtract the "rotation time offset" of 12\*60 minutes from +the minutes since the epoch, to get 24341715. If the current time period +length is 1440 minutes, by doing the division we see that we are currently +in time period number 16903. + +Specifically, time period #16903 began 16903*1440*60 + (12*60*60) seconds +after the epoch, at 2016-04-12 12:00 UTC, and ended at 16904*1440*60 + +(12*60*60) seconds after the epoch, at 2016-04-13 12:00 UTC. + + + +### When to publish a hidden service descriptor \[WHEN-HSDESC\] + +Hidden services periodically publish their descriptor to the responsible +HSDirs. The set of responsible HSDirs is determined as specified in +\[WHERE-HSDESC\]. + +Specifically, every time a hidden service publishes its descriptor, it also +sets up a timer for a random time between 60 minutes and 120 minutes in the +future. When the timer triggers, the hidden service needs to publish its +descriptor again to the responsible HSDirs for that time period. +\[TODO: Control republish period using a consensus parameter?\] + + + +#### Overlapping descriptors + +Hidden services need to upload multiple descriptors so that they can be +reachable to clients with older or newer consensuses than them. Services +need to upload their descriptors to the HSDirs *before* the beginning of +each upcoming time period, so that they are readily available for clients to +fetch them. Furthermore, services should keep uploading their old descriptor +even after the end of a time period, so that they can be reachable by +clients that still have consensuses from the previous time period. + +Hence, services maintain two active descriptors at every point. Clients on +the other hand, don't have a notion of overlapping descriptors, and instead +always download the descriptor for the current time period and shared random +value. It's the job of the service to ensure that descriptors will be +available for all clients. See section \[FETCHUPLOADDESC\] for how this is +achieved. + +\[TODO: What to do when we run multiple hidden services in a single host?\] + + + +### Where to publish a hidden service descriptor \[WHERE-HSDESC\] + +This section specifies how the HSDir hash ring is formed at any given +time. Whenever a time value is needed (e.g. to get the current time period +number), we assume that clients and services use the valid-after time from +their latest live consensus. + +The following consensus parameters control where a hidden service +descriptor is stored; + +```text + hsdir_n_replicas = an integer in range [1,16] with default value 2. + hsdir_spread_fetch = an integer in range [1,128] with default value 3. + hsdir_spread_store = an integer in range [1,128] with default value 4. + (Until 0.3.2.8-rc, the default was 3.) +``` + +To determine where a given hidden service descriptor will be stored +in a given period, after the blinded public key for that period is +derived, the uploading or downloading party calculates: + +```text + for replicanum in 1...hsdir_n_replicas: + hs_service_index(replicanum) = H("store-at-idx" | + blinded_public_key | + INT_8(replicanum) | + INT_8(period_length) | + INT_8(period_num) ) +``` + +where blinded_public_key is specified in section \[KEYBLIND\], period_length +is the length of the time period in minutes, and period_num is calculated +using the current consensus "valid-after" as specified in section +\[TIME-PERIODS\]. + +Then, for each node listed in the current consensus with the HSDir flag, +we compute a directory index for that node as: + +```text + hs_relay_index(node) = H("node-idx" | node_identity | + shared_random_value | + INT_8(period_num) | + INT_8(period_length) ) +``` + +where shared_random_value is the shared value generated by the authorities +in section \[PUB-SHAREDRANDOM\], and node_identity is the ed25519 identity +key of the node. + +Finally, for replicanum in 1...hsdir_n_replicas, the hidden service +host uploads descriptors to the first hsdir_spread_store nodes whose +indices immediately follow hs_service_index(replicanum). If any of those +nodes have already been selected for a lower-numbered replica of the +service, any nodes already chosen are disregarded (i.e. skipped over) +when choosing a replica's hsdir_spread_store nodes. + +When choosing an HSDir to download from, clients choose randomly from +among the first hsdir_spread_fetch nodes after the indices. (Note +that, in order to make the system better tolerate disappearing +HSDirs, hsdir_spread_fetch may be less than hsdir_spread_store.) +Again, nodes from lower-numbered replicas are disregarded when +choosing the spread for a replica. + + + +### Using time periods and SRVs to fetch/upload HS descriptors \[FETCHUPLOADDESC\] + +Hidden services and clients need to make correct use of time periods (TP) +and shared random values (SRVs) to successfully fetch and upload +descriptors. Furthermore, to avoid problems with skewed clocks, both clients +and services use the 'valid-after' time of a live consensus as a way to take +decisions with regards to uploading and fetching descriptors. By using the +consensus times as the ground truth here, we minimize the desynchronization +of clients and services due to system clock. Whenever time-based decisions +are taken in this section, assume that they are consensus times and not +system times. + +As \[PUB-SHAREDRANDOM\] specifies, consensuses contain two shared random +values (the current one and the previous one). Hidden services and clients +are asked to match these shared random values with descriptor time periods +and use the right SRV when fetching/uploading descriptors. This section +attempts to precisely specify how this works. + +Let's start with an illustration of the system: + +```text + +------------------------------------------------------------------+ + | | + | 00:00 12:00 00:00 12:00 00:00 12:00 | + | SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | + | | + | $==========|-----------$===========|-----------$===========| | + | | + | | + +------------------------------------------------------------------+ + + Legend: [TP#1 = Time Period #1] + [SRV#1 = Shared Random Value #1] + ["$" = descriptor rotation moment] +``` + + + +#### Client behavior for fetching descriptors \[CLIENTFETCH\] + +And here is how clients use TPs and SRVs to fetch descriptors: + +Clients always aim to synchronize their TP with SRV, so they always want to +use TP#N with SRV#N: To achieve this wrt time periods, clients always use +the current time period when fetching descriptors. Now wrt SRVs, if a client +is in the time segment between a new time period and a new SRV (i.e. the +segments drawn with "-") it uses the current SRV, else if the client is in a +time segment between a new SRV and a new time period (i.e. the segments +drawn with "="), it uses the previous SRV. + +Example: + ++------------------------------------------------------------------+ +| | +| 00:00 12:00 00:00 12:00 00:00 12:00 | +| SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | +| | +| $==========|-----------$===========|-----------$===========| | +| ^ ^ | +| C1 C2 | ++------------------------------------------------------------------+ + +If a client (C1) is at 13:00 right after TP#1, then it will use TP#1 and +SRV#1 for fetching descriptors. Also, if a client (C2) is at 01:00 right +after SRV#2, it will still use TP#1 and SRV#1. + + + +#### Service behavior for uploading descriptors \[SERVICEUPLOAD\] + +As discussed above, services maintain two active descriptors at any time. We +call these the "first" and "second" service descriptors. Services rotate +their descriptor every time they receive a consensus with a valid_after time +past the next SRV calculation time. They rotate their descriptors by +discarding their first descriptor, pushing the second descriptor to the +first, and rebuilding their second descriptor with the latest data. + +Services like clients also employ a different logic for picking SRV and TP +values based on their position in the graph above. Here is the logic: + + + +##### First descriptor upload logic \[FIRSTDESCUPLOAD\] + +Here is the service logic for uploading its first descriptor: + +When a service is in the time segment between a new time period a new SRV +(i.e. the segments drawn with "-"), it uses the previous time period and +previous SRV for uploading its first descriptor: that's meant to cover +for clients that have a consensus that is still in the previous time period. + +Example: Consider in the above illustration that the service is at 13:00 +right after TP#1. It will upload its first descriptor using TP#0 and SRV#0. +So if a client still has a 11:00 consensus it will be able to access it +based on the client logic above. + +Now if a service is in the time segment between a new SRV and a new time +period (i.e. the segments drawn with "=") it uses the current time period +and the previous SRV for its first descriptor: that's meant to cover clients +with an up-to-date consensus in the same time period as the service. + +Example: + ++------------------------------------------------------------------+ +| | +| 00:00 12:00 00:00 12:00 00:00 12:00 | +| SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | +| | +| $==========|-----------$===========|-----------$===========| | +| ^ | +| S | ++------------------------------------------------------------------+ + +Consider that the service is at 01:00 right after SRV#2: it will upload its +first descriptor using TP#1 and SRV#1. + + + +##### Second descriptor upload logic \[SECONDDESCUPLOAD\] + +Here is the service logic for uploading its second descriptor: + +When a service is in the time segment between a new time period a new SRV +(i.e. the segments drawn with "-"), it uses the current time period and +current SRV for uploading its second descriptor: that's meant to cover for +clients that have an up-to-date consensus on the same TP as the service. + +Example: Consider in the above illustration that the service is at 13:00 +right after TP#1: it will upload its second descriptor using TP#1 and SRV#1. + +Now if a service is in the time segment between a new SRV and a new time +period (i.e. the segments drawn with "=") it uses the next time period and +the current SRV for its second descriptor: that's meant to cover clients +with a newer consensus than the service (in the next time period). + +Example: + ++------------------------------------------------------------------+ +| | +| 00:00 12:00 00:00 12:00 00:00 12:00 | +| SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | +| | +| $==========|-----------$===========|-----------$===========| | +| ^ | +| S | ++------------------------------------------------------------------+ + +Consider that the service is at 01:00 right after SRV#2: it will upload its +second descriptor using TP#2 and SRV#2. + + + +#### Directory behavior for handling descriptor uploads \[DIRUPLOAD\] + +Upon receiving a hidden service descriptor publish request, directories MUST +check the following: + +```text + * The outer wrapper of the descriptor can be parsed according to + [DESC-OUTER] + * The version-number of the descriptor is "3" + * If the directory has already cached a descriptor for this hidden service, + the revision-counter of the uploaded descriptor must be greater than the + revision-counter of the cached one + * The descriptor signature is valid +``` + +If any of these basic validity checks fails, the directory MUST reject the +descriptor upload. + +NOTE: Even if the descriptor passes the checks above, its first and second +layers could still be invalid: directories cannot validate the encrypted +layers of the descriptor, as they do not have access to the public key of the +service (required for decrypting the first layer of encryption), or the +necessary client credentials (for decrypting the second layer). + + + +### Expiring hidden service descriptors \[EXPIRE-DESC\] + +Hidden services set their descriptor's "descriptor-lifetime" field to 180 +minutes (3 hours). Hidden services ensure that their descriptor will remain +valid in the HSDir caches, by republishing their descriptors periodically as +specified in \[WHEN-HSDESC\]. + +Hidden services MUST also keep their introduction circuits alive for as long +as descriptors including those intro points are valid (even if that's after +the time period has changed). + + + +### URLs for anonymous uploading and downloading + +Hidden service descriptors conforming to this specification are uploaded +with an HTTP POST request to the URL `/tor/hs//publish` relative to +the hidden service directory's root, and downloaded with an HTTP GET +request for the URL `/tor/hs//` where `` is a base64 encoding of +the hidden service's blinded public key and `` is the protocol +version which is "3" in this case. + +These requests must be made anonymously, on circuits not used for +anything else. + + + +### Client-side validation of onion addresses + +When a Tor client receives a prop224 onion address from the user, it +MUST first validate the onion address before attempting to connect or +fetch its descriptor. If the validation fails, the client MUST +refuse to connect. + +As part of the address validation, Tor clients should check that the +underlying ed25519 key does not have a torsion component. If Tor accepted +ed25519 keys with torsion components, attackers could create multiple +equivalent onion addresses for a single ed25519 key, which would map to the +same service. We want to avoid that because it could lead to phishing +attacks and surprising behaviors (e.g. imagine a browser plugin that blocks +onion addresses, but could be bypassed using an equivalent onion address +with a torsion component). + +The right way for clients to detect such fraudulent addresses (which should +only occur malevolently and never naturally) is to extract the ed25519 +public key from the onion address and multiply it by the ed25519 group order +and ensure that the result is the ed25519 identity element. For more +details, please see \[TORSION-REFS\]. diff --git a/spec/rend-spec-v3/encoding-onion-addresses-onionaddress.md b/spec/rend-spec-v3/encoding-onion-addresses-onionaddress.md deleted file mode 100644 index 14b3c44..0000000 --- a/spec/rend-spec-v3/encoding-onion-addresses-onionaddress.md +++ /dev/null @@ -1,27 +0,0 @@ - - -# Encoding onion addresses \[ONIONADDRESS\] - -The onion address of a hidden service includes its identity public key, a -version field and a basic checksum. All this information is then base32 -encoded as shown below: - -```text - onion_address = base32(PUBKEY | CHECKSUM | VERSION) + ".onion" - CHECKSUM = H(".onion checksum" | PUBKEY | VERSION)[:2] - - where: - - PUBKEY is the 32 bytes ed25519 master pubkey of the hidden service. - - VERSION is a one byte version field (default value '\x03') - - ".onion checksum" is a constant string - - CHECKSUM is truncated to two bytes before inserting it in onion_address - - Here are a few example addresses: - - pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd.onion - sp3k262uwy4r2k3ycr5awluarykdpag6a7y33jxop4cs2lu5uz5sseqd.onion - xa4r2iadxm55fbnqgwwi5mymqdcofiu3w6rpbtqn7b2dyn7mgwj64jyd.onion -``` - -For more information about this encoding, please see our discussion thread -at \[ONIONADDRESS-REFS\]. diff --git a/spec/rend-spec-v3/encoding-onion-addresses.md b/spec/rend-spec-v3/encoding-onion-addresses.md new file mode 100644 index 0000000..14b3c44 --- /dev/null +++ b/spec/rend-spec-v3/encoding-onion-addresses.md @@ -0,0 +1,27 @@ + + +# Encoding onion addresses \[ONIONADDRESS\] + +The onion address of a hidden service includes its identity public key, a +version field and a basic checksum. All this information is then base32 +encoded as shown below: + +```text + onion_address = base32(PUBKEY | CHECKSUM | VERSION) + ".onion" + CHECKSUM = H(".onion checksum" | PUBKEY | VERSION)[:2] + + where: + - PUBKEY is the 32 bytes ed25519 master pubkey of the hidden service. + - VERSION is a one byte version field (default value '\x03') + - ".onion checksum" is a constant string + - CHECKSUM is truncated to two bytes before inserting it in onion_address + + Here are a few example addresses: + + pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd.onion + sp3k262uwy4r2k3ycr5awluarykdpag6a7y33jxop4cs2lu5uz5sseqd.onion + xa4r2iadxm55fbnqgwwi5mymqdcofiu3w6rpbtqn7b2dyn7mgwj64jyd.onion +``` + +For more information about this encoding, please see our discussion thread +at \[ONIONADDRESS-REFS\]. diff --git a/spec/rend-spec-v3/encrypting-data-between-client-host.md b/spec/rend-spec-v3/encrypting-data-between-client-host.md deleted file mode 100644 index f3ce6f7..0000000 --- a/spec/rend-spec-v3/encrypting-data-between-client-host.md +++ /dev/null @@ -1,12 +0,0 @@ - - -# Encrypting data between client and host - -A successfully completed handshake, as embedded in the -INTRODUCE/RENDEZVOUS cells, gives the client and hidden service host -a shared set of keys Kf, Kb, Df, Db, which they use for sending -end-to-end traffic encryption and authentication as in the regular -Tor relay encryption protocol, applying encryption with these keys -before other encryption, and decrypting with these keys before other -decryption. The client encrypts with Kf and decrypts with Kb; the -service host does the opposite. diff --git a/spec/rend-spec-v3/encrypting-user-data.md b/spec/rend-spec-v3/encrypting-user-data.md new file mode 100644 index 0000000..f3ce6f7 --- /dev/null +++ b/spec/rend-spec-v3/encrypting-user-data.md @@ -0,0 +1,12 @@ + + +# Encrypting data between client and host + +A successfully completed handshake, as embedded in the +INTRODUCE/RENDEZVOUS cells, gives the client and hidden service host +a shared set of keys Kf, Kb, Df, Db, which they use for sending +end-to-end traffic encryption and authentication as in the regular +Tor relay encryption protocol, applying encryption with these keys +before other encryption, and decrypting with these keys before other +decryption. The client encrypts with Kf and decrypts with Kb; the +service host does the opposite. diff --git a/spec/rend-spec-v3/fs-contents.md b/spec/rend-spec-v3/fs-contents.md new file mode 100644 index 0000000..f5c06f4 --- /dev/null +++ b/spec/rend-spec-v3/fs-contents.md @@ -0,0 +1,30 @@ + + +# Appendix F: Hidden service directory format \[HIDSERVDIR-FORMAT\] + +This appendix section specifies the contents of the HiddenServiceDir directory: + +- "hostname" \[FILE\] + +This file contains the onion address of the onion service. + +- "private_key_ed25519" \[FILE\] + +This file contains the private master ed25519 key of the onion service. +\[TODO: Offline keys\] + +```text + - "./authorized_clients/" [DIRECTORY] + "./authorized_clients/alice.auth" [FILE] + "./authorized_clients/bob.auth" [FILE] + "./authorized_clients/charlie.auth" [FILE] +``` + +If client authorization is enabled, this directory MUST contain a ".auth" +file for each authorized client. Each such file contains the public key of +the respective client. The files are transmitted to the service operator by +the client. + +See section \[CLIENT-AUTH-MGMT\] for more details and the format of the client file. + +(NOTE: client authorization is implemented as of 0.3.5.1-alpha.) diff --git a/spec/rend-spec-v3/generating-publishing-hidden-service-descriptors.md b/spec/rend-spec-v3/generating-publishing-hidden-service-descriptors.md deleted file mode 100644 index 8b1d75a..0000000 --- a/spec/rend-spec-v3/generating-publishing-hidden-service-descriptors.md +++ /dev/null @@ -1,8 +0,0 @@ - - -# Generating and publishing hidden service descriptors \[HSDIR\] - -Hidden service descriptors follow the same metaformat as other Tor -directory objects. They are published anonymously to Tor servers with the -HSDir flag, HSDir=2 protocol version and tor version >= 0.3.0.8 (because a -bug was fixed in this version). diff --git a/spec/rend-spec-v3/hidden-service-descriptors-encryption-format-hs-desc-enc.md b/spec/rend-spec-v3/hidden-service-descriptors-encryption-format-hs-desc-enc.md deleted file mode 100644 index 834aceb..0000000 --- a/spec/rend-spec-v3/hidden-service-descriptors-encryption-format-hs-desc-enc.md +++ /dev/null @@ -1,456 +0,0 @@ - - -## Hidden service descriptors: encryption format \[HS-DESC-ENC\] - -Hidden service descriptors are protected by two layers of encryption. -Clients need to decrypt both layers to connect to the hidden service. - -The first layer of encryption provides confidentiality against entities who -don't know the public key of the hidden service (e.g. HSDirs), while the -second layer of encryption is only useful when client authorization is enabled -and protects against entities that do not possess valid client credentials. - - - -### First layer of encryption \[HS-DESC-FIRST-LAYER\] - -The first layer of HS descriptor encryption is designed to protect -descriptor confidentiality against entities who don't know the public -identity key of the hidden service. - - - -#### First layer encryption logic - -The encryption keys and format for the first layer of encryption are -generated as specified in \[HS-DESC-ENCRYPTION-KEYS\] with customization -parameters: - -```text - SECRET_DATA = blinded-public-key - STRING_CONSTANT = "hsdir-superencrypted-data" -``` - -The encryption scheme in \[HS-DESC-ENCRYPTION-KEYS\] uses the service -credential which is derived from the public identity key (see \[SUBCRED\]) to -ensure that only entities who know the public identity key can decrypt the -first descriptor layer. - -The ciphertext is placed on the "superencrypted" field of the descriptor. - -Before encryption the plaintext is padded with NUL bytes to the nearest -multiple of 10k bytes. - - - -#### First layer plaintext format - -After clients decrypt the first layer of encryption, they need to parse the -plaintext to get to the second layer ciphertext which is contained in the -"encrypted" field. - -If client auth is enabled, the hidden service generates a fresh -descriptor_cookie key (`N_hs_desc_enc`, 32 random bytes) and encrypts -it using each authorized client's identity x25519 key. Authorized -clients can use the descriptor cookie (`N_hs_desc_enc`) to decrypt -the second (inner) layer of encryption. Our encryption scheme -requires the hidden service to also generate an ephemeral x25519 -keypair for each new descriptor. - -If client auth is disabled, fake data is placed in each of the fields below -to obfuscate whether client authorization is enabled. - -Here are all the supported fields: - -"desc-auth-type" SP type NL - -\[Exactly once\] - -```text - This field contains the type of authorization used to protect the - descriptor. The only recognized type is "x25519" and specifies the - encryption scheme described in this section. - - If client authorization is disabled, the value here should be "x25519". - - "desc-auth-ephemeral-key" SP KP_hs_desc_ephem NL - - [Exactly once] - - This field contains `KP_hss_desc_enc`, an ephemeral x25519 public - key generated by the hidden service and encoded in base64. The key - is used by the encryption scheme below. - - If client authorization is disabled, the value here should be a fresh - x25519 pubkey that will remain unused. - - "auth-client" SP client-id SP iv SP encrypted-cookie - - [At least once] - - When client authorization is enabled, the hidden service inserts an - "auth-client" line for each of its authorized clients. If client - authorization is disabled, the fields here can be populated with random - data of the right size (that's 8 bytes for 'client-id', 16 bytes for 'iv' - and 16 bytes for 'encrypted-cookie' all encoded with base64). - - When client authorization is enabled, each "auth-client" line - contains the descriptor cookie `N_hs_desc_enc` encrypted to each - individual client. We assume that each authorized client possesses - a pre-shared x25519 keypair (`KP_hsc_desc_enc`) which is used to - decrypt the descriptor cookie. - - We now describe the descriptor cookie encryption scheme. Here is what - the hidden service computes: - - SECRET_SEED = x25519(KS_hs_desc_ephem, KP_hsc_desc_enc) - KEYS = KDF(N_hs_subcred | SECRET_SEED, 40) - CLIENT-ID = fist 8 bytes of KEYS - COOKIE-KEY = last 32 bytes of KEYS - - Here is a description of the fields in the "auth-client" line: - - - The "client-id" field is CLIENT-ID from above encoded in base64. - - - The "iv" field is 16 random bytes encoded in base64. - - - The "encrypted-cookie" field contains the descriptor cookie ciphertext - as follows and is encoded in base64: - encrypted-cookie = STREAM(iv, COOKIE-KEY) XOR N_hs_desc_enc. - - See section [FIRST-LAYER-CLIENT-BEHAVIOR] for the client-side logic of - how to decrypt the descriptor cookie. - - "encrypted" NL encrypted-string - - [Exactly once] - - An encrypted blob containing the second layer ciphertext, whose format is - discussed in [HS-DESC-SECOND-LAYER] below. The blob is base64 encoded - and enclosed in -----BEGIN MESSAGE---- and ----END MESSAGE---- wrappers. - - Compatibility note: The C Tor implementation does not include a final - newline when generating this first-layer-plaintext section; other - implementations MUST accept this section even if it is missing its final - newline. Other implementations MAY generate this section without a final - newline themselves, to avoid being distinguishable from C tor. -``` - - - -#### Client behavior \[FIRST-LAYER-CLIENT-BEHAVIOR\] - -```text - The goal of clients at this stage is to decrypt the "encrypted" field as - described in [HS-DESC-SECOND-LAYER]. - - If client authorization is enabled, authorized clients need to extract the - descriptor cookie to proceed with decryption of the second layer as - follows: - - An authorized client parsing the first layer of an encrypted descriptor, - extracts the ephemeral key from "desc-auth-ephemeral-key" and calculates - CLIENT-ID and COOKIE-KEY as described in the section above using their - x25519 private key. The client then uses CLIENT-ID to find the right - "auth-client" field which contains the ciphertext of the descriptor - cookie. The client then uses COOKIE-KEY and the iv to decrypt the - descriptor_cookie, which is used to decrypt the second layer of descriptor - encryption as described in [HS-DESC-SECOND-LAYER]. -``` - - - -#### Hiding client authorization data - -```text - Hidden services should avoid leaking whether client authorization is - enabled or how many authorized clients there are. - - Hence even when client authorization is disabled, the hidden service adds - fake "desc-auth-type", "desc-auth-ephemeral-key" and "auth-client" lines to - the descriptor, as described in [HS-DESC-FIRST-LAYER]. - - The hidden service also avoids leaking the number of authorized clients by - adding fake "auth-client" entries to its descriptor. Specifically, - descriptors always contain a number of authorized clients that is a - multiple of 16 by adding fake "auth-client" entries if needed. - [XXX consider randomization of the value 16] - - Clients MUST accept descriptors with any number of "auth-client" lines as - long as the total descriptor size is within the max limit of 50k (also - controlled with a consensus parameter). -``` - - - -### Second layer of encryption \[HS-DESC-SECOND-LAYER\] - -The second layer of descriptor encryption is designed to protect descriptor -confidentiality against unauthorized clients. If client authorization is -enabled, it's encrypted using the descriptor_cookie, and contains needed -information for connecting to the hidden service, like the list of its -introduction points. - -If client authorization is disabled, then the second layer of HS encryption -does not offer any additional security, but is still used. - - - -#### Second layer encryption keys - -The encryption keys and format for the second layer of encryption are -generated as specified in \[HS-DESC-ENCRYPTION-KEYS\] with customization -parameters as follows: - -```text - SECRET_DATA = blinded-public-key | descriptor_cookie - STRING_CONSTANT = "hsdir-encrypted-data" - - If client authorization is disabled the 'descriptor_cookie' field is left blank. - - The ciphertext is placed on the "encrypted" field of the descriptor. -``` - - - -#### Second layer plaintext format - -After decrypting the second layer ciphertext, clients can finally learn the -list of intro points etc. The plaintext has the following format: - -"create2-formats" SP formats NL - -\[Exactly once\] - -```text - A space-separated list of integers denoting CREATE2 cell HTYPEs - (handshake types) that the server recognizes. Must include at least - ntor as described in tor-spec.txt. See tor-spec section 5.1 for a list - of recognized handshake types. - - "intro-auth-required" SP types NL - - [At most once] - - A space-separated list of introduction-layer authentication types; see - section [INTRO-AUTH] for more info. A client that does not support at - least one of these authentication types will not be able to contact the - host. Recognized types are: 'ed25519'. - - "single-onion-service" - - [None or at most once] - - If present, this line indicates that the service is a Single Onion - Service (see prop260 for more details about that type of service). This - field has been introduced in 0.3.0 meaning 0.2.9 service don't include - this. - - Followed by zero or more introduction points as follows (see section - [NUM_INTRO_POINT] below for accepted values): - - "introduction-point" SP link-specifiers NL - - [Exactly once per introduction point at start of introduction - point section] - - The link-specifiers is a base64 encoding of a link specifier - block in the format described in [BUILDING-BLOCKS] above. - - As of 0.4.1.1-alpha, services include both IPv4 and IPv6 link - specifiers in descriptors. All available addresses SHOULD be - included in the descriptor, regardless of the address that the - onion service actually used to connect/extend to the intro - point. - - The client SHOULD NOT reject any LSTYPE fields which it doesn't - recognize; instead, it should use them verbatim in its EXTEND - request to the introduction point. - - The client SHOULD perform the basic validity checks on the link - specifiers in the descriptor, described in `tor-spec.txt` - section 5.1.2. These checks SHOULD NOT leak - detailed information about the client's version, configuration, - or consensus. (See 3.3 for service link specifier handling.) - - When connecting to the introduction point, the client SHOULD send - this list of link specifiers verbatim, in the same order as given - here. - - The client MAY reject the list of link specifiers if it is - inconsistent with relay information from the directory, but SHOULD - NOT modify it. - - "onion-key" SP "ntor" SP key NL - - [Exactly once per introduction point] - - The key is a base64 encoded curve25519 public key which is the onion - key of the introduction point Tor node used for the ntor handshake - when a client extends to it. - - "onion-key" SP KeyType SP key.. NL - - [Any number of times] - - Implementations should accept other types of onion keys using this - syntax (where "KeyType" is some string other than "ntor"); - unrecognized key types should be ignored. - - "auth-key" NL certificate NL - - [Exactly once per introduction point] - - The certificate is a proposal 220 certificate wrapped in - "-----BEGIN ED25519 CERT-----". It contains the introduction - point authentication key (`KP_hs_ipt_sid`), signed by - the descriptor signing key (`KP_hs_desc_sign`). The - certificate type must be [09], and the signing key extension - is mandatory. - - NOTE: This certificate was originally intended to be - constructed the other way around: the signing and signed keys - are meant to be reversed. However, C tor implemented it - backwards, and other implementations now need to do the same - in order to conform. (Since this section is inside the - descriptor, which is _already_ signed by `KP_hs_desc_sign`, - the verification aspect of this certificate serves no point in - its current form.) - - "enc-key" SP "ntor" SP key NL - - [Exactly once per introduction point] - - The key is a base64 encoded curve25519 public key used to encrypt - the introduction request to service. (`KP_hss_ntor`) - - "enc-key" SP KeyType SP key.. NL - - [Any number of times] - - Implementations should accept other types of onion keys using this - syntax (where "KeyType" is some string other than "ntor"); - unrecognized key types should be ignored. - - "enc-key-cert" NL certificate NL - - [Exactly once per introduction point] - - Cross-certification of the encryption key using the descriptor - signing key. - - For "ntor" keys, certificate is a proposal 220 certificate - wrapped in "-----BEGIN ED25519 CERT-----" armor. The subject - key is the the ed25519 equivalent of a curve25519 public - encryption key (`KP_hss_ntor`), with the ed25519 key - derived using the process in proposal 228 appendix A. The - signing key is the descriptor signing key (`KP_hs_desc_sign`). - The certificate type must be [0B], and the signing-key - extension is mandatory. - - NOTE: As with "auth-key", this certificate was intended to be - constructed the other way around. However, for compatibility - with C tor, implementations need to construct it this way. It - serves even less point than "auth-key", however, since the - encryption key `KP_hss_ntor` is already available from - the `enc-key` entry. - - "legacy-key" NL key NL - - [None or at most once per introduction point] - [This field is obsolete and should never be generated; it - is included for historical reasons only.] - - The key is an ASN.1 encoded RSA public key in PEM format used for a - legacy introduction point as described in [LEGACY_EST_INTRO]. - - This field is only present if the introduction point only supports - legacy protocol (v2) that is <= 0.2.9 or the protocol version value - "HSIntro 3". - - "legacy-key-cert" NL certificate NL - - [None or at most once per introduction point] - [This field is obsolete and should never be generated; it - is included for historical reasons only.] - - MUST be present if "legacy-key" is present. - - The certificate is a proposal 220 RSA->Ed cross-certificate wrapped - in "-----BEGIN CROSSCERT-----" armor, cross-certifying the RSA - public key found in "legacy-key" using the descriptor signing key. -``` - -To remain compatible with future revisions to the descriptor format, -clients should ignore unrecognized lines in the descriptor. -Other encryption and authentication key formats are allowed; clients -should ignore ones they do not recognize. - -Clients who manage to extract the introduction points of the hidden service -can proceed with the introduction protocol as specified in \[INTRO-PROTOCOL\]. - -Compatibility note: At least some versions of OnionBalance do not include -a final newline when generating this inner plaintext section; other -implementations MUST accept this section even if it is missing its final -newline. - - - -### Deriving hidden service descriptor encryption keys \[HS-DESC-ENCRYPTION-KEYS\] - -In this section we present the generic encryption format for hidden service -descriptors. We use the same encryption format in both encryption layers, -hence we introduce two customization parameters SECRET_DATA and -STRING_CONSTANT which vary between the layers. - -The SECRET_DATA parameter specifies the secret data that are used during -encryption key generation, while STRING_CONSTANT is merely a string constant -that is used as part of the KDF. - -Here is the key generation logic: - -```text - SALT = 16 bytes from H(random), changes each time we rebuild the - descriptor even if the content of the descriptor hasn't changed. - (So that we don't leak whether the intro point list etc. changed) - - secret_input = SECRET_DATA | N_hs_subcred | INT_8(revision_counter) - - keys = KDF(secret_input | salt | STRING_CONSTANT, S_KEY_LEN + S_IV_LEN + MAC_KEY_LEN) - - SECRET_KEY = first S_KEY_LEN bytes of keys - SECRET_IV = next S_IV_LEN bytes of keys - MAC_KEY = last MAC_KEY_LEN bytes of keys - - The encrypted data has the format: - - SALT hashed random bytes from above [16 bytes] - ENCRYPTED The ciphertext [variable] - MAC D_MAC of both above fields [32 bytes] - - The final encryption format is ENCRYPTED = STREAM(SECRET_IV,SECRET_KEY) XOR Plaintext . - - Where D_MAC = H(mac_key_len | MAC_KEY | salt_len | SALT | ENCRYPTED) - and - mac_key_len = htonll(len(MAC_KEY)) - and - salt_len = htonll(len(SALT)). -``` - - - -### Number of introduction points \[NUM_INTRO_POINT\] - -This section defines how many introduction points an hidden service -descriptor can have at minimum, by default and the maximum: - -Minimum: 0 - Default: 3 - Maximum: 20 - -A value of 0 would means that the service is still alive but doesn't want -to be reached by any client at the moment. Note that the descriptor size -increases considerably as more introduction points are added. - -The reason for a maximum value of 20 is to give enough scalability to tools -like OnionBalance to be able to load balance up to 120 servers (20 x 6 -HSDirs) but also in order for the descriptor size to not overwhelmed hidden -service directories with user defined values that could be gigantic. diff --git a/spec/rend-spec-v3/hidden-service-descriptors-outer-wrapper-desc-outer.md b/spec/rend-spec-v3/hidden-service-descriptors-outer-wrapper-desc-outer.md deleted file mode 100644 index 5f4b5de..0000000 --- a/spec/rend-spec-v3/hidden-service-descriptors-outer-wrapper-desc-outer.md +++ /dev/null @@ -1,74 +0,0 @@ - - -## Hidden service descriptors: outer wrapper \[DESC-OUTER\] - -The format for a hidden service descriptor is as follows, using the -meta-format from dir-spec.txt. - -"hs-descriptor" SP version-number NL - -\[At start, exactly once.\] - -```text - The version-number is a 32 bit unsigned integer indicating the version - of the descriptor. Current version is "3". - - "descriptor-lifetime" SP LifetimeMinutes NL - - [Exactly once] - - The lifetime of a descriptor in minutes. An HSDir SHOULD expire the - hidden service descriptor at least LifetimeMinutes after it was - uploaded. - - The LifetimeMinutes field can take values between 30 and 720 (12 - hours). - - "descriptor-signing-key-cert" NL certificate NL - - [Exactly once.] - - The 'certificate' field contains a certificate in the format from - proposal 220, wrapped with "-----BEGIN ED25519 CERT-----". The - certificate cross-certifies the short-term descriptor signing key with - the blinded public key. The certificate type must be [08], and the - blinded public key must be present as the signing-key extension. - - "revision-counter" SP Integer NL - - [Exactly once.] - - The revision number of the descriptor. If an HSDir receives a - second descriptor for a key that it already has a descriptor for, - it should retain and serve the descriptor with the higher - revision-counter. - - (Checking for monotonically increasing revision-counter values - prevents an attacker from replacing a newer descriptor signed by - a given key with a copy of an older version.) - - Implementations MUST be able to parse 64-bit values for these - counters. - - "superencrypted" NL encrypted-string - - [Exactly once.] - - An encrypted blob, whose format is discussed in [HS-DESC-ENC] below. The - blob is base64 encoded and enclosed in -----BEGIN MESSAGE---- and - ----END MESSAGE---- wrappers. (The resulting document does not end with - a newline character.) - - "signature" SP signature NL - - [exactly once, at end.] - - A signature of all previous fields, using the signing key in the - descriptor-signing-key-cert line, prefixed by the string "Tor onion - service descriptor sig v3". We use a separate key for signing, so that - the hidden service host does not need to have its private blinded key - online. -``` - -HSDirs accept hidden service descriptors of up to 50k bytes (a consensus -parameter should also be introduced to control this value). diff --git a/spec/rend-spec-v3/hidden-service-directory-format-hidservdir-format.md b/spec/rend-spec-v3/hidden-service-directory-format-hidservdir-format.md deleted file mode 100644 index f5c06f4..0000000 --- a/spec/rend-spec-v3/hidden-service-directory-format-hidservdir-format.md +++ /dev/null @@ -1,30 +0,0 @@ - - -# Appendix F: Hidden service directory format \[HIDSERVDIR-FORMAT\] - -This appendix section specifies the contents of the HiddenServiceDir directory: - -- "hostname" \[FILE\] - -This file contains the onion address of the onion service. - -- "private_key_ed25519" \[FILE\] - -This file contains the private master ed25519 key of the onion service. -\[TODO: Offline keys\] - -```text - - "./authorized_clients/" [DIRECTORY] - "./authorized_clients/alice.auth" [FILE] - "./authorized_clients/bob.auth" [FILE] - "./authorized_clients/charlie.auth" [FILE] -``` - -If client authorization is enabled, this directory MUST contain a ".auth" -file for each authorized client. Each such file contains the public key of -the respective client. The files are transmitted to the service operator by -the client. - -See section \[CLIENT-AUTH-MGMT\] for more details and the format of the client file. - -(NOTE: client authorization is implemented as of 0.3.5.1-alpha.) diff --git a/spec/rend-spec-v3/hidden-services-overview-preliminaries.md b/spec/rend-spec-v3/hidden-services-overview-preliminaries.md deleted file mode 100644 index cf16ff4..0000000 --- a/spec/rend-spec-v3/hidden-services-overview-preliminaries.md +++ /dev/null @@ -1,315 +0,0 @@ - - -# Hidden services: overview and preliminaries - -Hidden services aim to provide responder anonymity for bidirectional -stream-based communication on the Tor network. Unlike regular Tor -connections, where the connection initiator receives anonymity but -the responder does not, hidden services attempt to provide -bidirectional anonymity. - -Participants: - -Operator -- A person running a hidden service - -```text - Host, "Server" -- The Tor software run by the operator to provide - a hidden service. - - User -- A person contacting a hidden service. - - Client -- The Tor software running on the User's computer - - Hidden Service Directory (HSDir) -- A Tor node that hosts signed - statements from hidden service hosts so that users can make - contact with them. - - Introduction Point -- A Tor node that accepts connection requests - for hidden services and anonymously relays those requests to the - hidden service. - - Rendezvous Point -- A Tor node to which clients and servers - connect and which relays traffic between them. -``` - - - -## Improvements over previous versions - -Here is a list of improvements of this proposal over the legacy hidden -services: - -a) Better crypto (replaced SHA1/DH/RSA1024 with SHA3/ed25519/curve25519) -b) Improved directory protocol leaking less to directory servers. -c) Improved directory protocol with smaller surface for targeted attacks. -d) Better onion address security against impersonation. -e) More extensible introduction/rendezvous protocol. -f) Offline keys for onion services -g) Advanced client authorization - - - -## Notation and vocabulary - -Unless specified otherwise, all multi-octet integers are big-endian. - -We write sequences of bytes in two ways: - -```text - 1. A sequence of two-digit hexadecimal values in square brackets, - as in [AB AD 1D EA]. - - 2. A string of characters enclosed in quotes, as in "Hello". The - characters in these strings are encoded in their ascii - representations; strings are NOT nul-terminated unless - explicitly described as NUL terminated. - - We use the words "byte" and "octet" interchangeably. - - We use the vertical bar | to denote concatenation. -``` - -We use INT_N(val) to denote the network (big-endian) encoding of the -unsigned integer "val" in N bytes. For example, INT_4(1337) is \[00 00 -05 39\]. Values are truncated like so: val % (2 ^ (N * 8)). For example, -INT_4(42) is 42 % 4294967296 (32 bit). - - - -## Cryptographic building blocks - -This specification uses the following cryptographic building blocks: - -```text - * A pseudorandom number generator backed by a strong entropy source. - The output of the PRNG should always be hashed before being posted on - the network to avoid leaking raw PRNG bytes to the network - (see [PRNG-REFS]). - - * A stream cipher STREAM(iv, k) where iv is a nonce of length - S_IV_LEN bytes and k is a key of length S_KEY_LEN bytes. - - * A public key signature system SIGN_KEYGEN()->seckey, pubkey; - SIGN_SIGN(seckey,msg)->sig; and SIGN_CHECK(pubkey, sig, msg) -> - { "OK", "BAD" }; where secret keys are of length SIGN_SECKEY_LEN - bytes, public keys are of length SIGN_PUBKEY_LEN bytes, and - signatures are of length SIGN_SIG_LEN bytes. - - This signature system must also support key blinding operations - as discussed in appendix [KEYBLIND] and in section [SUBCRED]: - SIGN_BLIND_SECKEY(seckey, blind)->seckey2 and - SIGN_BLIND_PUBKEY(pubkey, blind)->pubkey2 . - - * A public key agreement system "PK", providing - PK_KEYGEN()->seckey, pubkey; PK_VALID(pubkey) -> {"OK", "BAD"}; - and PK_HANDSHAKE(seckey, pubkey)->output; where secret keys are - of length PK_SECKEY_LEN bytes, public keys are of length - PK_PUBKEY_LEN bytes, and the handshake produces outputs of - length PK_OUTPUT_LEN bytes. - - * A cryptographic hash function H(d), which should be preimage and - collision resistant. It produces hashes of length HASH_LEN - bytes. - - * A cryptographic message authentication code MAC(key,msg) that - produces outputs of length MAC_LEN bytes. - - * A key derivation function KDF(message, n) that outputs n bytes. - - As a first pass, I suggest: - - * Instantiate STREAM with AES256-CTR. - - * Instantiate SIGN with Ed25519 and the blinding protocol in - [KEYBLIND]. - - * Instantiate PK with Curve25519. - - * Instantiate H with SHA3-256. - - * Instantiate KDF with SHAKE-256. - - * Instantiate MAC(key=k, message=m) with H(k_len | k | m), - where k_len is htonll(len(k)). -``` - -When we need a particular MAC key length below, we choose -MAC_KEY_LEN=32 (256 bits). - -For legacy purposes, we specify compatibility with older versions of -the Tor introduction point and rendezvous point protocols. These used -RSA1024, DH1024, AES128, and SHA1, as discussed in -rend-spec.txt. - -As in \[proposal 220\], all signatures are generated not over strings -themselves, but over those strings prefixed with a distinguishing -value. - - - -## Protocol building blocks \[BUILDING-BLOCKS\] - -In sections below, we need to transmit the locations and identities -of Tor nodes. We do so in the link identification format used by -EXTEND2 cells in the Tor protocol. - -```text - NSPEC (Number of link specifiers) [1 byte] - NSPEC times: - LSTYPE (Link specifier type) [1 byte] - LSLEN (Link specifier length) [1 byte] - LSPEC (Link specifier) [LSLEN bytes] -``` - -Link specifier types are as described in tor-spec.txt. Every set of -link specifiers SHOULD include at minimum specifiers of type \[00\] -(TLS-over-TCP, IPv4), \[02\] (legacy node identity) and \[03\] (ed25519 -identity key). Sets of link specifiers without these three types -SHOULD be rejected. - -As of 0.4.1.1-alpha, Tor includes both IPv4 and IPv6 link specifiers -in v3 onion service protocol link specifier lists. All available -addresses SHOULD be included as link specifiers, regardless of the -address that Tor actually used to connect/extend to the remote relay. - -We also incorporate Tor's circuit extension handshakes, as used in -the CREATE2 and CREATED2 cells described in tor-spec.txt. In these -handshakes, a client who knows a public key for a server sends a -message and receives a message from that server. Once the exchange is -done, the two parties have a shared set of forward-secure key -material, and the client knows that nobody else shares that key -material unless they control the secret key corresponding to the -server's public key. - - - -## Assigned relay cell types - -These relay cell types are reserved for use in the hidden service -protocol. - -32 -- RELAY_COMMAND_ESTABLISH_INTRO - -```text - Sent from hidden service host to introduction point; - establishes introduction point. Discussed in - [REG_INTRO_POINT]. - - 33 -- RELAY_COMMAND_ESTABLISH_RENDEZVOUS - - Sent from client to rendezvous point; creates rendezvous - point. Discussed in [EST_REND_POINT]. - - 34 -- RELAY_COMMAND_INTRODUCE1 - - Sent from client to introduction point; requests - introduction. Discussed in [SEND_INTRO1] - - 35 -- RELAY_COMMAND_INTRODUCE2 - - Sent from introduction point to hidden service host; requests - introduction. Same format as INTRODUCE1. Discussed in - [FMT_INTRO1] and [PROCESS_INTRO2] - - 36 -- RELAY_COMMAND_RENDEZVOUS1 - - Sent from hidden service host to rendezvous point; - attempts to join host's circuit to - client's circuit. Discussed in [JOIN_REND] - - 37 -- RELAY_COMMAND_RENDEZVOUS2 - - Sent from rendezvous point to client; - reports join of host's circuit to - client's circuit. Discussed in [JOIN_REND] - - 38 -- RELAY_COMMAND_INTRO_ESTABLISHED - - Sent from introduction point to hidden service host; - reports status of attempt to establish introduction - point. Discussed in [INTRO_ESTABLISHED] - - 39 -- RELAY_COMMAND_RENDEZVOUS_ESTABLISHED - - Sent from rendezvous point to client; acknowledges - receipt of ESTABLISH_RENDEZVOUS cell. Discussed in - [EST_REND_POINT] - - 40 -- RELAY_COMMAND_INTRODUCE_ACK - - Sent from introduction point to client; acknowledges - receipt of INTRODUCE1 cell and reports success/failure. - Discussed in [INTRO_ACK] -``` - - - -## Acknowledgments - -This design includes ideas from many people, including - -```text - Christopher Baines, - Daniel J. Bernstein, - Matthew Finkel, - Ian Goldberg, - George Kadianakis, - Aniket Kate, - Tanja Lange, - Robert Ransom, - Roger Dingledine, - Aaron Johnson, - Tim Wilson-Brown ("teor"), - special (John Brooks), - s7r -``` - -It's based on Tor's original hidden service design by Roger -Dingledine, Nick Mathewson, and Paul Syverson, and on improvements to -that design over the years by people including - -```text - Tobias Kamm, - Thomas Lauterbach, - Karsten Loesing, - Alessandro Preite Martinez, - Robert Ransom, - Ferdinand Rieger, - Christoph Weingarten, - Christian Wilms, -``` - -We wouldn't be able to do any of this work without good attack -designs from researchers including - -```text - Alex Biryukov, - Lasse Øverlier, - Ivan Pustogarov, - Paul Syverson, - Ralf-Philipp Weinmann, - - See [ATTACK-REFS] for their papers. - - Several of these ideas have come from conversations with - - Christian Grothoff, - Brian Warner, - Zooko Wilcox-O'Hearn, -``` - -And if this document makes any sense at all, it's thanks to -editing help from - -```text - Matthew Finkel, - George Kadianakis, - Peter Palfrader, - Tim Wilson-Brown ("teor"), -``` - -\[XXX Acknowledge the huge bunch of people working on 8106.\] -\[XXX Acknowledge the huge bunch of people working on 8244.\] - -Please forgive me if I've missed you; please forgive me if I've -misunderstood your best ideas here too. diff --git a/spec/rend-spec-v3/hsdesc-encrypt.md b/spec/rend-spec-v3/hsdesc-encrypt.md new file mode 100644 index 0000000..834aceb --- /dev/null +++ b/spec/rend-spec-v3/hsdesc-encrypt.md @@ -0,0 +1,456 @@ + + +## Hidden service descriptors: encryption format \[HS-DESC-ENC\] + +Hidden service descriptors are protected by two layers of encryption. +Clients need to decrypt both layers to connect to the hidden service. + +The first layer of encryption provides confidentiality against entities who +don't know the public key of the hidden service (e.g. HSDirs), while the +second layer of encryption is only useful when client authorization is enabled +and protects against entities that do not possess valid client credentials. + + + +### First layer of encryption \[HS-DESC-FIRST-LAYER\] + +The first layer of HS descriptor encryption is designed to protect +descriptor confidentiality against entities who don't know the public +identity key of the hidden service. + + + +#### First layer encryption logic + +The encryption keys and format for the first layer of encryption are +generated as specified in \[HS-DESC-ENCRYPTION-KEYS\] with customization +parameters: + +```text + SECRET_DATA = blinded-public-key + STRING_CONSTANT = "hsdir-superencrypted-data" +``` + +The encryption scheme in \[HS-DESC-ENCRYPTION-KEYS\] uses the service +credential which is derived from the public identity key (see \[SUBCRED\]) to +ensure that only entities who know the public identity key can decrypt the +first descriptor layer. + +The ciphertext is placed on the "superencrypted" field of the descriptor. + +Before encryption the plaintext is padded with NUL bytes to the nearest +multiple of 10k bytes. + + + +#### First layer plaintext format + +After clients decrypt the first layer of encryption, they need to parse the +plaintext to get to the second layer ciphertext which is contained in the +"encrypted" field. + +If client auth is enabled, the hidden service generates a fresh +descriptor_cookie key (`N_hs_desc_enc`, 32 random bytes) and encrypts +it using each authorized client's identity x25519 key. Authorized +clients can use the descriptor cookie (`N_hs_desc_enc`) to decrypt +the second (inner) layer of encryption. Our encryption scheme +requires the hidden service to also generate an ephemeral x25519 +keypair for each new descriptor. + +If client auth is disabled, fake data is placed in each of the fields below +to obfuscate whether client authorization is enabled. + +Here are all the supported fields: + +"desc-auth-type" SP type NL + +\[Exactly once\] + +```text + This field contains the type of authorization used to protect the + descriptor. The only recognized type is "x25519" and specifies the + encryption scheme described in this section. + + If client authorization is disabled, the value here should be "x25519". + + "desc-auth-ephemeral-key" SP KP_hs_desc_ephem NL + + [Exactly once] + + This field contains `KP_hss_desc_enc`, an ephemeral x25519 public + key generated by the hidden service and encoded in base64. The key + is used by the encryption scheme below. + + If client authorization is disabled, the value here should be a fresh + x25519 pubkey that will remain unused. + + "auth-client" SP client-id SP iv SP encrypted-cookie + + [At least once] + + When client authorization is enabled, the hidden service inserts an + "auth-client" line for each of its authorized clients. If client + authorization is disabled, the fields here can be populated with random + data of the right size (that's 8 bytes for 'client-id', 16 bytes for 'iv' + and 16 bytes for 'encrypted-cookie' all encoded with base64). + + When client authorization is enabled, each "auth-client" line + contains the descriptor cookie `N_hs_desc_enc` encrypted to each + individual client. We assume that each authorized client possesses + a pre-shared x25519 keypair (`KP_hsc_desc_enc`) which is used to + decrypt the descriptor cookie. + + We now describe the descriptor cookie encryption scheme. Here is what + the hidden service computes: + + SECRET_SEED = x25519(KS_hs_desc_ephem, KP_hsc_desc_enc) + KEYS = KDF(N_hs_subcred | SECRET_SEED, 40) + CLIENT-ID = fist 8 bytes of KEYS + COOKIE-KEY = last 32 bytes of KEYS + + Here is a description of the fields in the "auth-client" line: + + - The "client-id" field is CLIENT-ID from above encoded in base64. + + - The "iv" field is 16 random bytes encoded in base64. + + - The "encrypted-cookie" field contains the descriptor cookie ciphertext + as follows and is encoded in base64: + encrypted-cookie = STREAM(iv, COOKIE-KEY) XOR N_hs_desc_enc. + + See section [FIRST-LAYER-CLIENT-BEHAVIOR] for the client-side logic of + how to decrypt the descriptor cookie. + + "encrypted" NL encrypted-string + + [Exactly once] + + An encrypted blob containing the second layer ciphertext, whose format is + discussed in [HS-DESC-SECOND-LAYER] below. The blob is base64 encoded + and enclosed in -----BEGIN MESSAGE---- and ----END MESSAGE---- wrappers. + + Compatibility note: The C Tor implementation does not include a final + newline when generating this first-layer-plaintext section; other + implementations MUST accept this section even if it is missing its final + newline. Other implementations MAY generate this section without a final + newline themselves, to avoid being distinguishable from C tor. +``` + + + +#### Client behavior \[FIRST-LAYER-CLIENT-BEHAVIOR\] + +```text + The goal of clients at this stage is to decrypt the "encrypted" field as + described in [HS-DESC-SECOND-LAYER]. + + If client authorization is enabled, authorized clients need to extract the + descriptor cookie to proceed with decryption of the second layer as + follows: + + An authorized client parsing the first layer of an encrypted descriptor, + extracts the ephemeral key from "desc-auth-ephemeral-key" and calculates + CLIENT-ID and COOKIE-KEY as described in the section above using their + x25519 private key. The client then uses CLIENT-ID to find the right + "auth-client" field which contains the ciphertext of the descriptor + cookie. The client then uses COOKIE-KEY and the iv to decrypt the + descriptor_cookie, which is used to decrypt the second layer of descriptor + encryption as described in [HS-DESC-SECOND-LAYER]. +``` + + + +#### Hiding client authorization data + +```text + Hidden services should avoid leaking whether client authorization is + enabled or how many authorized clients there are. + + Hence even when client authorization is disabled, the hidden service adds + fake "desc-auth-type", "desc-auth-ephemeral-key" and "auth-client" lines to + the descriptor, as described in [HS-DESC-FIRST-LAYER]. + + The hidden service also avoids leaking the number of authorized clients by + adding fake "auth-client" entries to its descriptor. Specifically, + descriptors always contain a number of authorized clients that is a + multiple of 16 by adding fake "auth-client" entries if needed. + [XXX consider randomization of the value 16] + + Clients MUST accept descriptors with any number of "auth-client" lines as + long as the total descriptor size is within the max limit of 50k (also + controlled with a consensus parameter). +``` + + + +### Second layer of encryption \[HS-DESC-SECOND-LAYER\] + +The second layer of descriptor encryption is designed to protect descriptor +confidentiality against unauthorized clients. If client authorization is +enabled, it's encrypted using the descriptor_cookie, and contains needed +information for connecting to the hidden service, like the list of its +introduction points. + +If client authorization is disabled, then the second layer of HS encryption +does not offer any additional security, but is still used. + + + +#### Second layer encryption keys + +The encryption keys and format for the second layer of encryption are +generated as specified in \[HS-DESC-ENCRYPTION-KEYS\] with customization +parameters as follows: + +```text + SECRET_DATA = blinded-public-key | descriptor_cookie + STRING_CONSTANT = "hsdir-encrypted-data" + + If client authorization is disabled the 'descriptor_cookie' field is left blank. + + The ciphertext is placed on the "encrypted" field of the descriptor. +``` + + + +#### Second layer plaintext format + +After decrypting the second layer ciphertext, clients can finally learn the +list of intro points etc. The plaintext has the following format: + +"create2-formats" SP formats NL + +\[Exactly once\] + +```text + A space-separated list of integers denoting CREATE2 cell HTYPEs + (handshake types) that the server recognizes. Must include at least + ntor as described in tor-spec.txt. See tor-spec section 5.1 for a list + of recognized handshake types. + + "intro-auth-required" SP types NL + + [At most once] + + A space-separated list of introduction-layer authentication types; see + section [INTRO-AUTH] for more info. A client that does not support at + least one of these authentication types will not be able to contact the + host. Recognized types are: 'ed25519'. + + "single-onion-service" + + [None or at most once] + + If present, this line indicates that the service is a Single Onion + Service (see prop260 for more details about that type of service). This + field has been introduced in 0.3.0 meaning 0.2.9 service don't include + this. + + Followed by zero or more introduction points as follows (see section + [NUM_INTRO_POINT] below for accepted values): + + "introduction-point" SP link-specifiers NL + + [Exactly once per introduction point at start of introduction + point section] + + The link-specifiers is a base64 encoding of a link specifier + block in the format described in [BUILDING-BLOCKS] above. + + As of 0.4.1.1-alpha, services include both IPv4 and IPv6 link + specifiers in descriptors. All available addresses SHOULD be + included in the descriptor, regardless of the address that the + onion service actually used to connect/extend to the intro + point. + + The client SHOULD NOT reject any LSTYPE fields which it doesn't + recognize; instead, it should use them verbatim in its EXTEND + request to the introduction point. + + The client SHOULD perform the basic validity checks on the link + specifiers in the descriptor, described in `tor-spec.txt` + section 5.1.2. These checks SHOULD NOT leak + detailed information about the client's version, configuration, + or consensus. (See 3.3 for service link specifier handling.) + + When connecting to the introduction point, the client SHOULD send + this list of link specifiers verbatim, in the same order as given + here. + + The client MAY reject the list of link specifiers if it is + inconsistent with relay information from the directory, but SHOULD + NOT modify it. + + "onion-key" SP "ntor" SP key NL + + [Exactly once per introduction point] + + The key is a base64 encoded curve25519 public key which is the onion + key of the introduction point Tor node used for the ntor handshake + when a client extends to it. + + "onion-key" SP KeyType SP key.. NL + + [Any number of times] + + Implementations should accept other types of onion keys using this + syntax (where "KeyType" is some string other than "ntor"); + unrecognized key types should be ignored. + + "auth-key" NL certificate NL + + [Exactly once per introduction point] + + The certificate is a proposal 220 certificate wrapped in + "-----BEGIN ED25519 CERT-----". It contains the introduction + point authentication key (`KP_hs_ipt_sid`), signed by + the descriptor signing key (`KP_hs_desc_sign`). The + certificate type must be [09], and the signing key extension + is mandatory. + + NOTE: This certificate was originally intended to be + constructed the other way around: the signing and signed keys + are meant to be reversed. However, C tor implemented it + backwards, and other implementations now need to do the same + in order to conform. (Since this section is inside the + descriptor, which is _already_ signed by `KP_hs_desc_sign`, + the verification aspect of this certificate serves no point in + its current form.) + + "enc-key" SP "ntor" SP key NL + + [Exactly once per introduction point] + + The key is a base64 encoded curve25519 public key used to encrypt + the introduction request to service. (`KP_hss_ntor`) + + "enc-key" SP KeyType SP key.. NL + + [Any number of times] + + Implementations should accept other types of onion keys using this + syntax (where "KeyType" is some string other than "ntor"); + unrecognized key types should be ignored. + + "enc-key-cert" NL certificate NL + + [Exactly once per introduction point] + + Cross-certification of the encryption key using the descriptor + signing key. + + For "ntor" keys, certificate is a proposal 220 certificate + wrapped in "-----BEGIN ED25519 CERT-----" armor. The subject + key is the the ed25519 equivalent of a curve25519 public + encryption key (`KP_hss_ntor`), with the ed25519 key + derived using the process in proposal 228 appendix A. The + signing key is the descriptor signing key (`KP_hs_desc_sign`). + The certificate type must be [0B], and the signing-key + extension is mandatory. + + NOTE: As with "auth-key", this certificate was intended to be + constructed the other way around. However, for compatibility + with C tor, implementations need to construct it this way. It + serves even less point than "auth-key", however, since the + encryption key `KP_hss_ntor` is already available from + the `enc-key` entry. + + "legacy-key" NL key NL + + [None or at most once per introduction point] + [This field is obsolete and should never be generated; it + is included for historical reasons only.] + + The key is an ASN.1 encoded RSA public key in PEM format used for a + legacy introduction point as described in [LEGACY_EST_INTRO]. + + This field is only present if the introduction point only supports + legacy protocol (v2) that is <= 0.2.9 or the protocol version value + "HSIntro 3". + + "legacy-key-cert" NL certificate NL + + [None or at most once per introduction point] + [This field is obsolete and should never be generated; it + is included for historical reasons only.] + + MUST be present if "legacy-key" is present. + + The certificate is a proposal 220 RSA->Ed cross-certificate wrapped + in "-----BEGIN CROSSCERT-----" armor, cross-certifying the RSA + public key found in "legacy-key" using the descriptor signing key. +``` + +To remain compatible with future revisions to the descriptor format, +clients should ignore unrecognized lines in the descriptor. +Other encryption and authentication key formats are allowed; clients +should ignore ones they do not recognize. + +Clients who manage to extract the introduction points of the hidden service +can proceed with the introduction protocol as specified in \[INTRO-PROTOCOL\]. + +Compatibility note: At least some versions of OnionBalance do not include +a final newline when generating this inner plaintext section; other +implementations MUST accept this section even if it is missing its final +newline. + + + +### Deriving hidden service descriptor encryption keys \[HS-DESC-ENCRYPTION-KEYS\] + +In this section we present the generic encryption format for hidden service +descriptors. We use the same encryption format in both encryption layers, +hence we introduce two customization parameters SECRET_DATA and +STRING_CONSTANT which vary between the layers. + +The SECRET_DATA parameter specifies the secret data that are used during +encryption key generation, while STRING_CONSTANT is merely a string constant +that is used as part of the KDF. + +Here is the key generation logic: + +```text + SALT = 16 bytes from H(random), changes each time we rebuild the + descriptor even if the content of the descriptor hasn't changed. + (So that we don't leak whether the intro point list etc. changed) + + secret_input = SECRET_DATA | N_hs_subcred | INT_8(revision_counter) + + keys = KDF(secret_input | salt | STRING_CONSTANT, S_KEY_LEN + S_IV_LEN + MAC_KEY_LEN) + + SECRET_KEY = first S_KEY_LEN bytes of keys + SECRET_IV = next S_IV_LEN bytes of keys + MAC_KEY = last MAC_KEY_LEN bytes of keys + + The encrypted data has the format: + + SALT hashed random bytes from above [16 bytes] + ENCRYPTED The ciphertext [variable] + MAC D_MAC of both above fields [32 bytes] + + The final encryption format is ENCRYPTED = STREAM(SECRET_IV,SECRET_KEY) XOR Plaintext . + + Where D_MAC = H(mac_key_len | MAC_KEY | salt_len | SALT | ENCRYPTED) + and + mac_key_len = htonll(len(MAC_KEY)) + and + salt_len = htonll(len(SALT)). +``` + + + +### Number of introduction points \[NUM_INTRO_POINT\] + +This section defines how many introduction points an hidden service +descriptor can have at minimum, by default and the maximum: + +Minimum: 0 - Default: 3 - Maximum: 20 + +A value of 0 would means that the service is still alive but doesn't want +to be reached by any client at the moment. Note that the descriptor size +increases considerably as more introduction points are added. + +The reason for a maximum value of 20 is to give enough scalability to tools +like OnionBalance to be able to load balance up to 120 servers (20 x 6 +HSDirs) but also in order for the descriptor size to not overwhelmed hidden +service directories with user defined values that could be gigantic. diff --git a/spec/rend-spec-v3/hsdesc-outer.md b/spec/rend-spec-v3/hsdesc-outer.md new file mode 100644 index 0000000..5f4b5de --- /dev/null +++ b/spec/rend-spec-v3/hsdesc-outer.md @@ -0,0 +1,74 @@ + + +## Hidden service descriptors: outer wrapper \[DESC-OUTER\] + +The format for a hidden service descriptor is as follows, using the +meta-format from dir-spec.txt. + +"hs-descriptor" SP version-number NL + +\[At start, exactly once.\] + +```text + The version-number is a 32 bit unsigned integer indicating the version + of the descriptor. Current version is "3". + + "descriptor-lifetime" SP LifetimeMinutes NL + + [Exactly once] + + The lifetime of a descriptor in minutes. An HSDir SHOULD expire the + hidden service descriptor at least LifetimeMinutes after it was + uploaded. + + The LifetimeMinutes field can take values between 30 and 720 (12 + hours). + + "descriptor-signing-key-cert" NL certificate NL + + [Exactly once.] + + The 'certificate' field contains a certificate in the format from + proposal 220, wrapped with "-----BEGIN ED25519 CERT-----". The + certificate cross-certifies the short-term descriptor signing key with + the blinded public key. The certificate type must be [08], and the + blinded public key must be present as the signing-key extension. + + "revision-counter" SP Integer NL + + [Exactly once.] + + The revision number of the descriptor. If an HSDir receives a + second descriptor for a key that it already has a descriptor for, + it should retain and serve the descriptor with the higher + revision-counter. + + (Checking for monotonically increasing revision-counter values + prevents an attacker from replacing a newer descriptor signed by + a given key with a copy of an older version.) + + Implementations MUST be able to parse 64-bit values for these + counters. + + "superencrypted" NL encrypted-string + + [Exactly once.] + + An encrypted blob, whose format is discussed in [HS-DESC-ENC] below. The + blob is base64 encoded and enclosed in -----BEGIN MESSAGE---- and + ----END MESSAGE---- wrappers. (The resulting document does not end with + a newline character.) + + "signature" SP signature NL + + [exactly once, at end.] + + A signature of all previous fields, using the signing key in the + descriptor-signing-key-cert line, prefixed by the string "Tor onion + service descriptor sig v3". We use a separate key for signing, so that + the hidden service host does not need to have its private blinded key + online. +``` + +HSDirs accept hidden service descriptors of up to 50k bytes (a consensus +parameter should also be introduced to control this value). diff --git a/spec/rend-spec-v3/hsdesc.md b/spec/rend-spec-v3/hsdesc.md new file mode 100644 index 0000000..8b1d75a --- /dev/null +++ b/spec/rend-spec-v3/hsdesc.md @@ -0,0 +1,8 @@ + + +# Generating and publishing hidden service descriptors \[HSDIR\] + +Hidden service descriptors follow the same metaformat as other Tor +directory objects. They are published anonymously to Tor servers with the +HSDir flag, HSDir=2 protocol version and tor version >= 0.3.0.8 (because a +bug was fixed in this version). diff --git a/spec/rend-spec-v3/introduction-protocol-intro-protocol.md b/spec/rend-spec-v3/introduction-protocol-intro-protocol.md deleted file mode 100644 index f4e6358..0000000 --- a/spec/rend-spec-v3/introduction-protocol-intro-protocol.md +++ /dev/null @@ -1,632 +0,0 @@ - - -# The introduction protocol \[INTRO-PROTOCOL\] - -The introduction protocol proceeds in three steps. - -First, a hidden service host builds an anonymous circuit to a Tor -node and registers that circuit as an introduction point. - -Single Onion Services attempt to build a non-anonymous single-hop circuit, -but use an anonymous 3-hop circuit if: - -```text - * the intro point is on an address that is configured as unreachable via - a direct connection, or - * the initial attempt to connect to the intro point over a single-hop - circuit fails, and they are retrying the intro point connection. - - [After 'First' and before 'Second', the hidden service publishes its - introduction points and associated keys, and the client fetches - them as described in section [HSDIR] above.] -``` - -Second, a client builds an anonymous circuit to the introduction -point, and sends an introduction request. - -Third, the introduction point relays the introduction request along -the introduction circuit to the hidden service host, and acknowledges -the introduction request to the client. - - - -## Registering an introduction point \[REG_INTRO_POINT\] - - - -### Extensible ESTABLISH_INTRO protocol. \[EST_INTRO\] - -When a hidden service is establishing a new introduction point, it -sends an ESTABLISH_INTRO cell with the following contents: - -```text - AUTH_KEY_TYPE [1 byte] - AUTH_KEY_LEN [2 bytes] - AUTH_KEY [AUTH_KEY_LEN bytes] - N_EXTENSIONS [1 byte] - N_EXTENSIONS times: - EXT_FIELD_TYPE [1 byte] - EXT_FIELD_LEN [1 byte] - EXT_FIELD [EXT_FIELD_LEN bytes] - HANDSHAKE_AUTH [MAC_LEN bytes] - SIG_LEN [2 bytes] - SIG [SIG_LEN bytes] -``` - -The AUTH_KEY_TYPE field indicates the type of the introduction point -authentication key and the type of the MAC to use in -HANDSHAKE_AUTH. Recognized types are: - -```text - [00, 01] -- Reserved for legacy introduction cells; see - [LEGACY_EST_INTRO below] - [02] -- Ed25519; SHA3-256. -``` - -The AUTH_KEY_LEN field determines the length of the AUTH_KEY -field. The AUTH_KEY field contains the public introduction point -authentication key, KP_hs_ipt_sid. - -The EXT_FIELD_TYPE, EXT_FIELD_LEN, EXT_FIELD entries are reserved for -extensions to the introduction protocol. Extensions with -unrecognized EXT_FIELD_TYPE values must be ignored. -(`EXT_FIELD_LEN` may be zero, in which case EXT_FIELD is absent.) - -```text - Unless otherwise specified in the documentation for an extension type: - * Each extension type SHOULD be sent only once in a message. - * Parties MUST ignore any occurrences all occurrences of an extension - with a given type after the first such occurrence. - * Extensions SHOULD be sent in numerically ascending order by type. - (The above extension sorting and multiplicity rules are only defaults; - they may be overridden in the descriptions of individual extensions.) -``` - -The HANDSHAKE_AUTH field contains the MAC of all earlier fields in -the cell using as its key the shared per-circuit material ("KH") -generated during the circuit extension protocol; see tor-spec.txt -section 5.2, "Setting circuit keys". It prevents replays of -ESTABLISH_INTRO cells. - -SIG_LEN is the length of the signature. - -SIG is a signature, using AUTH_KEY, of all contents of the cell, up -to but not including SIG_LEN and SIG. These contents are prefixed -with the string "Tor establish-intro cell v1". - -Upon receiving an ESTABLISH_INTRO cell, a Tor node first decodes the -key and the signature, and checks the signature. The node must reject -the ESTABLISH_INTRO cell and destroy the circuit in these cases: - -```text - * If the key type is unrecognized - * If the key is ill-formatted - * If the signature is incorrect - * If the HANDSHAKE_AUTH value is incorrect - - * If the circuit is already a rendezvous circuit. - * If the circuit is already an introduction circuit. - [TODO: some scalability designs fail there.] - * If the key is already in use by another circuit. -``` - -Otherwise, the node must associate the key with the circuit, for use -later in INTRODUCE1 cells. - - - -#### Denial-of-Service Defense Extension. \[EST_INTRO_DOS_EXT\] - -This extension can be used to send Denial-of-Service (DoS) parameters to -the introduction point in order for it to apply them for the introduction -circuit. - -If used, it needs to be encoded within the N_EXTENSIONS field of the -ESTABLISH_INTRO cell defined in the previous section. The content is -defined as follow: - -EXT_FIELD_TYPE: - -\[01\] -- Denial-of-Service Parameters. - -```text - If this flag is set, the extension should be used by the introduction - point to learn what values the denial of service subsystem should be - using. - - EXT_FIELD content format is: - - N_PARAMS [1 byte] - N_PARAMS times: - PARAM_TYPE [1 byte] - PARAM_VALUE [8 byte] - - The PARAM_TYPE possible values are: - - [01] -- DOS_INTRODUCE2_RATE_PER_SEC - The rate per second of INTRODUCE2 cell relayed to the - service. - - [02] -- DOS_INTRODUCE2_BURST_PER_SEC - The burst per second of INTRODUCE2 cell relayed to the - service. - - The PARAM_VALUE size is 8 bytes in order to accommodate 64bit values. - It MUST match the specified limit for the following PARAM_TYPE: - - [01] -- Min: 0, Max: 2147483647 - [02] -- Min: 0, Max: 2147483647 - - A value of 0 means the defense is disabled. If the rate per second is - set to 0 (param 0x01) then the burst value should be ignored. And - vice-versa, if the burst value is 0 (param 0x02), then the rate value - should be ignored. In other words, setting one single parameter to 0 - disables the defense. - - The burst can NOT be smaller than the rate. If so, the parameters - should be ignored by the introduction point. - - Any valid value does have precedence over the network wide consensus - parameter. -``` - -Using this extension extends the payload of the ESTABLISH_INTRO cell by 19 -bytes bringing it from 134 bytes to 155 bytes. - -This extension can only be used with relays supporting the protocol version -"HSIntro=5". - -Introduced in tor-0.4.2.1-alpha. - -```text -3.1.2. Registering an introduction point on a legacy Tor node - [LEGACY_EST_INTRO] - - [This section is obsolete and refers to a workaround for now-obsolete Tor - relay versions. It is included for historical reasons.] -``` - -Tor nodes should also support an older version of the ESTABLISH_INTRO -cell, first documented in rend-spec.txt. New hidden service hosts -must use this format when establishing introduction points at older -Tor nodes that do not support the format above in \[EST_INTRO\]. - -In this older protocol, an ESTABLISH_INTRO cell contains: - -```text - KEY_LEN [2 bytes] - KEY [KEY_LEN bytes] - HANDSHAKE_AUTH [20 bytes] - SIG [variable, up to end of relay payload] - - The KEY_LEN variable determines the length of the KEY field. -``` - -The KEY field is the ASN1-encoded legacy RSA public key that was also -included in the hidden service descriptor. - -The HANDSHAKE_AUTH field contains the SHA1 digest of (KH | "INTRODUCE"). - -The SIG field contains an RSA signature, using PKCS1 padding, of all -earlier fields. - -Older versions of Tor always use a 1024-bit RSA key for these introduction -authentication keys. - - - -### Acknowledging establishment of introduction point \[INTRO_ESTABLISHED\] - -After setting up an introduction circuit, the introduction point reports its -status back to the hidden service host with an INTRO_ESTABLISHED cell. - -The INTRO_ESTABLISHED cell has the following contents: - -```text - N_EXTENSIONS [1 byte] - N_EXTENSIONS times: - EXT_FIELD_TYPE [1 byte] - EXT_FIELD_LEN [1 byte] - EXT_FIELD [EXT_FIELD_LEN bytes] -``` - -Older versions of Tor send back an empty INTRO_ESTABLISHED cell instead. -Services must accept an empty INTRO_ESTABLISHED cell from a legacy relay. -\[The above paragraph is obsolete and refers to a workaround for -now-obsolete Tor relay versions. It is included for historical reasons.\] - -The same rules for multiplicity, ordering, and handling unknown types -apply to the extension fields here as described \[EST_INTRO\] above. - - - -## Sending an INTRODUCE1 cell to the introduction point. \[SEND_INTRO1\] - -In order to participate in the introduction protocol, a client must -know the following: - -```text - * An introduction point for a service. - * The introduction authentication key for that introduction point. - * The introduction encryption key for that introduction point. -``` - -The client sends an INTRODUCE1 cell to the introduction point, -containing an identifier for the service, an identifier for the -encryption key that the client intends to use, and an opaque blob to -be relayed to the hidden service host. - -In reply, the introduction point sends an INTRODUCE_ACK cell back to -the client, either informing it that its request has been delivered, -or that its request will not succeed. - -```text - [TODO: specify what tor should do when receiving a malformed cell. Drop it? - Kill circuit? This goes for all possible cells.] -``` - - - -### INTRODUCE1 cell format \[FMT_INTRO1\] - -When a client is connecting to an introduction point, INTRODUCE1 cells -should be of the form: - -```text - LEGACY_KEY_ID [20 bytes] - AUTH_KEY_TYPE [1 byte] - AUTH_KEY_LEN [2 bytes] - AUTH_KEY [AUTH_KEY_LEN bytes] - N_EXTENSIONS [1 byte] - N_EXTENSIONS times: - EXT_FIELD_TYPE [1 byte] - EXT_FIELD_LEN [1 byte] - EXT_FIELD [EXT_FIELD_LEN bytes] - ENCRYPTED [Up to end of relay payload] -``` - -AUTH_KEY_TYPE is defined as in \[EST_INTRO\]. Currently, the only value of -AUTH_KEY_TYPE for this cell is an Ed25519 public key \[02\]. - -The LEGACY_KEY_ID field is used to distinguish between legacy and new style -INTRODUCE1 cells. In new style INTRODUCE1 cells, LEGACY_KEY_ID is 20 zero -bytes. Upon receiving an INTRODUCE1 cell, the introduction point checks the -LEGACY_KEY_ID field. If LEGACY_KEY_ID is non-zero, the INTRODUCE1 cell -should be handled as a legacy INTRODUCE1 cell by the intro point. - -Upon receiving a INTRODUCE1 cell, the introduction point checks -whether AUTH_KEY matches the introduction point authentication key for an -active introduction circuit. If so, the introduction point sends an -INTRODUCE2 cell with exactly the same contents to the service, and sends an -INTRODUCE_ACK response to the client. - -(Note that the introduction point does not "clean up" the -INTRODUCE1 cells that it retransmits. Specifically, it does not -change the order or multiplicity of the extensions sent by the -client.) - -The same rules for multiplicity, ordering, and handling unknown types -apply to the extension fields here as described \[EST_INTRO\] above. - - - -### INTRODUCE_ACK cell format. \[INTRO_ACK\] - -An INTRODUCE_ACK cell has the following fields: - -```text - STATUS [2 bytes] - N_EXTENSIONS [1 bytes] - N_EXTENSIONS times: - EXT_FIELD_TYPE [1 byte] - EXT_FIELD_LEN [1 byte] - EXT_FIELD [EXT_FIELD_LEN bytes] - - Recognized status values are: - - [00 00] -- Success: cell relayed to hidden service host. - [00 01] -- Failure: service ID not recognized - [00 02] -- Bad message format - [00 03] -- Can't relay cell to service -``` - -The same rules for multiplicity, ordering, and handling unknown types -apply to the extension fields here as described \[EST_INTRO\] above. - - - -## Processing an INTRODUCE2 cell at the hidden service. \[PROCESS_INTRO2\] - -Upon receiving an INTRODUCE2 cell, the hidden service host checks whether -the AUTH_KEY or LEGACY_KEY_ID field matches the keys for this -introduction circuit. - -The service host then checks whether it has received a cell with these -contents or rendezvous cookie before. If it has, it silently drops it as a -replay. (It must maintain a replay cache for as long as it accepts cells -with the same encryption key. Note that the encryption format below should -be non-malleable.) - -If the cell is not a replay, it decrypts the ENCRYPTED field, -establishes a shared key with the client, and authenticates the whole -contents of the cell as having been unmodified since they left the -client. There may be multiple ways of decrypting the ENCRYPTED field, -depending on the chosen type of the encryption key. Requirements for -an introduction handshake protocol are described in -\[INTRO-HANDSHAKE-REQS\]. We specify one below in section -\[NTOR-WITH-EXTRA-DATA\]. - -The decrypted plaintext must have the form: - -```text - RENDEZVOUS_COOKIE [20 bytes] - N_EXTENSIONS [1 byte] - N_EXTENSIONS times: - EXT_FIELD_TYPE [1 byte] - EXT_FIELD_LEN [1 byte] - EXT_FIELD [EXT_FIELD_LEN bytes] - ONION_KEY_TYPE [1 bytes] - ONION_KEY_LEN [2 bytes] - ONION_KEY [ONION_KEY_LEN bytes] - NSPEC (Number of link specifiers) [1 byte] - NSPEC times: - LSTYPE (Link specifier type) [1 byte] - LSLEN (Link specifier length) [1 byte] - LSPEC (Link specifier) [LSLEN bytes] - PAD (optional padding) [up to end of plaintext] -``` - -Upon processing this plaintext, the hidden service makes sure that -any required authentication is present in the extension fields, and -then extends a rendezvous circuit to the node described in the LSPEC -fields, using the ONION_KEY to complete the extension. As mentioned -in \[BUILDING-BLOCKS\], the "TLS-over-TCP, IPv4" and "Legacy node -identity" specifiers must be present. - -As of 0.4.1.1-alpha, clients include both IPv4 and IPv6 link specifiers -in INTRODUCE1 cells. All available addresses SHOULD be included in the -cell, regardless of the address that the client actually used to extend -to the rendezvous point. - -The hidden service should handle invalid or unrecognised link specifiers -the same way as clients do in section 2.5.2.2. In particular, services -SHOULD perform basic validity checks on link specifiers, and SHOULD NOT -reject unrecognised link specifiers, to avoid information leaks. -The list of link specifiers received here SHOULD either be rejected, or -sent verbatim when extending to the rendezvous point, in the same order -received. - -The service MAY reject the list of link specifiers if it is -inconsistent with relay information from the directory, but SHOULD -NOT modify it. - -The ONION_KEY_TYPE field is: - -\[01\] NTOR: ONION_KEY is 32 bytes long. - -The ONION_KEY field describes the onion key that must be used when -extending to the rendezvous point. It must be of a type listed as -supported in the hidden service descriptor. - -The PAD field should be filled with zeros; its size should be chosen -so that the INTRODUCE2 message occupies a fixed maximum size, in -order to hide the length of the encrypted data. (This maximum size is -490, since we assume that a future Tor implementations will implement -proposal 340 and thus lower the number of bytes that can be contained -in a single relay message.) Note also that current versions of Tor -only pad the INTRODUCE2 message up to 246 bytes. - -Upon receiving a well-formed INTRODUCE2 cell, the hidden service host -will have: - -```text - * The information needed to connect to the client's chosen - rendezvous point. - * The second half of a handshake to authenticate and establish a - shared key with the hidden service client. - * A set of shared keys to use for end-to-end encryption. -``` - -The same rules for multiplicity, ordering, and handling unknown types -apply to the extension fields here as described \[EST_INTRO\] above. - - - -### Introduction handshake encryption requirements \[INTRO-HANDSHAKE-REQS\] - -When decoding the encrypted information in an INTRODUCE2 cell, a -hidden service host must be able to: - -```text - * Decrypt additional information included in the INTRODUCE2 cell, - to include the rendezvous token and the information needed to - extend to the rendezvous point. - - * Establish a set of shared keys for use with the client. - - * Authenticate that the cell has not been modified since the client - generated it. -``` - -Note that the old TAP-derived protocol of the previous hidden service -design achieved the first two requirements, but not the third. - -```text -3.3.2. Example encryption handshake: ntor with extra data - [NTOR-WITH-EXTRA-DATA] - - [TODO: relocate this] -``` - -This is a variant of the ntor handshake (see tor-spec.txt, section -5.1.4; see proposal 216; and see "Anonymity and one-way -authentication in key-exchange protocols" by Goldberg, Stebila, and -Ustaoglu). - -It behaves the same as the ntor handshake, except that, in addition -to negotiating forward secure keys, it also provides a means for -encrypting non-forward-secure data to the server (in this case, to -the hidden service host) as part of the handshake. - -Notation here is as in section 5.1.4 of tor-spec.txt, which defines -the ntor handshake. - -The PROTOID for this variant is "tor-hs-ntor-curve25519-sha3-256-1". -We also use the following tweak values: - -```text - t_hsenc = PROTOID | ":hs_key_extract" - t_hsverify = PROTOID | ":hs_verify" - t_hsmac = PROTOID | ":hs_mac" - m_hsexpand = PROTOID | ":hs_key_expand" -``` - -To make an INTRODUCE1 cell, the client must know a public encryption -key B for the hidden service on this introduction circuit. The client -generates a single-use keypair: - -x,X = KEYGEN() - -and computes: - -```text - intro_secret_hs_input = EXP(B,x) | AUTH_KEY | X | B | PROTOID - info = m_hsexpand | N_hs_subcred - hs_keys = KDF(intro_secret_hs_input | t_hsenc | info, S_KEY_LEN+MAC_LEN) - ENC_KEY = hs_keys[0:S_KEY_LEN] - MAC_KEY = hs_keys[S_KEY_LEN:S_KEY_LEN+MAC_KEY_LEN] - - and sends, as the ENCRYPTED part of the INTRODUCE1 cell: - - CLIENT_PK [PK_PUBKEY_LEN bytes] - ENCRYPTED_DATA [Padded to length of plaintext] - MAC [MAC_LEN bytes] -``` - -Substituting those fields into the INTRODUCE1 cell body format -described in \[FMT_INTRO1\] above, we have - -```text - LEGACY_KEY_ID [20 bytes] - AUTH_KEY_TYPE [1 byte] - AUTH_KEY_LEN [2 bytes] - AUTH_KEY [AUTH_KEY_LEN bytes] - N_EXTENSIONS [1 bytes] - N_EXTENSIONS times: - EXT_FIELD_TYPE [1 byte] - EXT_FIELD_LEN [1 byte] - EXT_FIELD [EXT_FIELD_LEN bytes] - ENCRYPTED: - CLIENT_PK [PK_PUBKEY_LEN bytes] - ENCRYPTED_DATA [Padded to length of plaintext] - MAC [MAC_LEN bytes] -``` - -(This format is as documented in \[FMT_INTRO1\] above, except that here -we describe how to build the ENCRYPTED portion.) - -Here, the encryption key plays the role of B in the regular ntor -handshake, and the AUTH_KEY field plays the role of the node ID. -The CLIENT_PK field is the public key X. The ENCRYPTED_DATA field is -the message plaintext, encrypted with the symmetric key ENC_KEY. The -MAC field is a MAC of all of the cell from the AUTH_KEY through the -end of ENCRYPTED_DATA, using the MAC_KEY value as its key. - -To process this format, the hidden service checks PK_VALID(CLIENT_PK) -as necessary, and then computes ENC_KEY and MAC_KEY as the client did -above, except using EXP(CLIENT_PK,b) in the calculation of -intro_secret_hs_input. The service host then checks whether the MAC is -correct. If it is invalid, it drops the cell. Otherwise, it computes -the plaintext by decrypting ENCRYPTED_DATA. - -The hidden service host now completes the service side of the -extended ntor handshake, as described in tor-spec.txt section 5.1.4, -with the modified PROTOID as given above. To be explicit, the hidden -service host generates a keypair of y,Y = KEYGEN(), and uses its -introduction point encryption key 'b' to compute: - -```text - intro_secret_hs_input = EXP(X,b) | AUTH_KEY | X | B | PROTOID - info = m_hsexpand | N_hs_subcred - hs_keys = KDF(intro_secret_hs_input | t_hsenc | info, S_KEY_LEN+MAC_LEN) - HS_DEC_KEY = hs_keys[0:S_KEY_LEN] - HS_MAC_KEY = hs_keys[S_KEY_LEN:S_KEY_LEN+MAC_KEY_LEN] - - (The above are used to check the MAC and then decrypt the - encrypted data.) - - rend_secret_hs_input = EXP(X,y) | EXP(X,b) | AUTH_KEY | B | X | Y | PROTOID - NTOR_KEY_SEED = MAC(rend_secret_hs_input, t_hsenc) - verify = MAC(rend_secret_hs_input, t_hsverify) - auth_input = verify | AUTH_KEY | B | Y | X | PROTOID | "Server" - AUTH_INPUT_MAC = MAC(auth_input, t_hsmac) - - (The above are used to finish the ntor handshake.) - - The server's handshake reply is: - - SERVER_PK Y [PK_PUBKEY_LEN bytes] - AUTH AUTH_INPUT_MAC [MAC_LEN bytes] -``` - -These fields will be sent to the client in a RENDEZVOUS1 cell using the -HANDSHAKE_INFO element (see \[JOIN_REND\]). - -The hidden service host now also knows the keys generated by the -handshake, which it will use to encrypt and authenticate data -end-to-end between the client and the server. These keys are as -computed in tor-spec.txt section 5.1.4, except that instead of using -AES-128 and SHA1 for this hop, we use AES-256 and SHA3-256. - - - -## Authentication during the introduction phase. \[INTRO-AUTH\] - -Hidden services may restrict access only to authorized users. -One mechanism to do so is the credential mechanism, where only users who -know the credential for a hidden service may connect at all. - -There is one defined authentication type: `ed25519`. - - - -### Ed25519-based authentication `ed25519` - -(NOTE: This section is not implemented by Tor. It is likely -that we would want to change its design substantially before -deploying any implementation. At the very least, we would -want to bind these extensions to a single onion service, to -prevent replays. We might also want to look for ways to limit -the number of keys a user needs to have.) - -To authenticate with an Ed25519 private key, the user must include an -extension field in the encrypted part of the INTRODUCE1 cell with an -EXT_FIELD_TYPE type of \[02\] and the contents: - -```text - Nonce [16 bytes] - Pubkey [32 bytes] - Signature [64 bytes] -``` - -Nonce is a random value. Pubkey is the public key that will be used -to authenticate. \[TODO: should this be an identifier for the public -key instead?\] Signature is the signature, using Ed25519, of: - -```text - "hidserv-userauth-ed25519" - Nonce (same as above) - Pubkey (same as above) - AUTH_KEY (As in the INTRODUCE1 cell) -``` - -The hidden service host checks this by seeing whether it recognizes -and would accept a signature from the provided public key. If it -would, then it checks whether the signature is correct. If it is, -then the correct user has authenticated. - -Replay prevention on the whole cell is sufficient to prevent replays -on the authentication. - -Users SHOULD NOT use the same public key with multiple hidden -services. diff --git a/spec/rend-spec-v3/introduction-protocol.md b/spec/rend-spec-v3/introduction-protocol.md new file mode 100644 index 0000000..f4e6358 --- /dev/null +++ b/spec/rend-spec-v3/introduction-protocol.md @@ -0,0 +1,632 @@ + + +# The introduction protocol \[INTRO-PROTOCOL\] + +The introduction protocol proceeds in three steps. + +First, a hidden service host builds an anonymous circuit to a Tor +node and registers that circuit as an introduction point. + +Single Onion Services attempt to build a non-anonymous single-hop circuit, +but use an anonymous 3-hop circuit if: + +```text + * the intro point is on an address that is configured as unreachable via + a direct connection, or + * the initial attempt to connect to the intro point over a single-hop + circuit fails, and they are retrying the intro point connection. + + [After 'First' and before 'Second', the hidden service publishes its + introduction points and associated keys, and the client fetches + them as described in section [HSDIR] above.] +``` + +Second, a client builds an anonymous circuit to the introduction +point, and sends an introduction request. + +Third, the introduction point relays the introduction request along +the introduction circuit to the hidden service host, and acknowledges +the introduction request to the client. + + + +## Registering an introduction point \[REG_INTRO_POINT\] + + + +### Extensible ESTABLISH_INTRO protocol. \[EST_INTRO\] + +When a hidden service is establishing a new introduction point, it +sends an ESTABLISH_INTRO cell with the following contents: + +```text + AUTH_KEY_TYPE [1 byte] + AUTH_KEY_LEN [2 bytes] + AUTH_KEY [AUTH_KEY_LEN bytes] + N_EXTENSIONS [1 byte] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + HANDSHAKE_AUTH [MAC_LEN bytes] + SIG_LEN [2 bytes] + SIG [SIG_LEN bytes] +``` + +The AUTH_KEY_TYPE field indicates the type of the introduction point +authentication key and the type of the MAC to use in +HANDSHAKE_AUTH. Recognized types are: + +```text + [00, 01] -- Reserved for legacy introduction cells; see + [LEGACY_EST_INTRO below] + [02] -- Ed25519; SHA3-256. +``` + +The AUTH_KEY_LEN field determines the length of the AUTH_KEY +field. The AUTH_KEY field contains the public introduction point +authentication key, KP_hs_ipt_sid. + +The EXT_FIELD_TYPE, EXT_FIELD_LEN, EXT_FIELD entries are reserved for +extensions to the introduction protocol. Extensions with +unrecognized EXT_FIELD_TYPE values must be ignored. +(`EXT_FIELD_LEN` may be zero, in which case EXT_FIELD is absent.) + +```text + Unless otherwise specified in the documentation for an extension type: + * Each extension type SHOULD be sent only once in a message. + * Parties MUST ignore any occurrences all occurrences of an extension + with a given type after the first such occurrence. + * Extensions SHOULD be sent in numerically ascending order by type. + (The above extension sorting and multiplicity rules are only defaults; + they may be overridden in the descriptions of individual extensions.) +``` + +The HANDSHAKE_AUTH field contains the MAC of all earlier fields in +the cell using as its key the shared per-circuit material ("KH") +generated during the circuit extension protocol; see tor-spec.txt +section 5.2, "Setting circuit keys". It prevents replays of +ESTABLISH_INTRO cells. + +SIG_LEN is the length of the signature. + +SIG is a signature, using AUTH_KEY, of all contents of the cell, up +to but not including SIG_LEN and SIG. These contents are prefixed +with the string "Tor establish-intro cell v1". + +Upon receiving an ESTABLISH_INTRO cell, a Tor node first decodes the +key and the signature, and checks the signature. The node must reject +the ESTABLISH_INTRO cell and destroy the circuit in these cases: + +```text + * If the key type is unrecognized + * If the key is ill-formatted + * If the signature is incorrect + * If the HANDSHAKE_AUTH value is incorrect + + * If the circuit is already a rendezvous circuit. + * If the circuit is already an introduction circuit. + [TODO: some scalability designs fail there.] + * If the key is already in use by another circuit. +``` + +Otherwise, the node must associate the key with the circuit, for use +later in INTRODUCE1 cells. + + + +#### Denial-of-Service Defense Extension. \[EST_INTRO_DOS_EXT\] + +This extension can be used to send Denial-of-Service (DoS) parameters to +the introduction point in order for it to apply them for the introduction +circuit. + +If used, it needs to be encoded within the N_EXTENSIONS field of the +ESTABLISH_INTRO cell defined in the previous section. The content is +defined as follow: + +EXT_FIELD_TYPE: + +\[01\] -- Denial-of-Service Parameters. + +```text + If this flag is set, the extension should be used by the introduction + point to learn what values the denial of service subsystem should be + using. + + EXT_FIELD content format is: + + N_PARAMS [1 byte] + N_PARAMS times: + PARAM_TYPE [1 byte] + PARAM_VALUE [8 byte] + + The PARAM_TYPE possible values are: + + [01] -- DOS_INTRODUCE2_RATE_PER_SEC + The rate per second of INTRODUCE2 cell relayed to the + service. + + [02] -- DOS_INTRODUCE2_BURST_PER_SEC + The burst per second of INTRODUCE2 cell relayed to the + service. + + The PARAM_VALUE size is 8 bytes in order to accommodate 64bit values. + It MUST match the specified limit for the following PARAM_TYPE: + + [01] -- Min: 0, Max: 2147483647 + [02] -- Min: 0, Max: 2147483647 + + A value of 0 means the defense is disabled. If the rate per second is + set to 0 (param 0x01) then the burst value should be ignored. And + vice-versa, if the burst value is 0 (param 0x02), then the rate value + should be ignored. In other words, setting one single parameter to 0 + disables the defense. + + The burst can NOT be smaller than the rate. If so, the parameters + should be ignored by the introduction point. + + Any valid value does have precedence over the network wide consensus + parameter. +``` + +Using this extension extends the payload of the ESTABLISH_INTRO cell by 19 +bytes bringing it from 134 bytes to 155 bytes. + +This extension can only be used with relays supporting the protocol version +"HSIntro=5". + +Introduced in tor-0.4.2.1-alpha. + +```text +3.1.2. Registering an introduction point on a legacy Tor node + [LEGACY_EST_INTRO] + + [This section is obsolete and refers to a workaround for now-obsolete Tor + relay versions. It is included for historical reasons.] +``` + +Tor nodes should also support an older version of the ESTABLISH_INTRO +cell, first documented in rend-spec.txt. New hidden service hosts +must use this format when establishing introduction points at older +Tor nodes that do not support the format above in \[EST_INTRO\]. + +In this older protocol, an ESTABLISH_INTRO cell contains: + +```text + KEY_LEN [2 bytes] + KEY [KEY_LEN bytes] + HANDSHAKE_AUTH [20 bytes] + SIG [variable, up to end of relay payload] + + The KEY_LEN variable determines the length of the KEY field. +``` + +The KEY field is the ASN1-encoded legacy RSA public key that was also +included in the hidden service descriptor. + +The HANDSHAKE_AUTH field contains the SHA1 digest of (KH | "INTRODUCE"). + +The SIG field contains an RSA signature, using PKCS1 padding, of all +earlier fields. + +Older versions of Tor always use a 1024-bit RSA key for these introduction +authentication keys. + + + +### Acknowledging establishment of introduction point \[INTRO_ESTABLISHED\] + +After setting up an introduction circuit, the introduction point reports its +status back to the hidden service host with an INTRO_ESTABLISHED cell. + +The INTRO_ESTABLISHED cell has the following contents: + +```text + N_EXTENSIONS [1 byte] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] +``` + +Older versions of Tor send back an empty INTRO_ESTABLISHED cell instead. +Services must accept an empty INTRO_ESTABLISHED cell from a legacy relay. +\[The above paragraph is obsolete and refers to a workaround for +now-obsolete Tor relay versions. It is included for historical reasons.\] + +The same rules for multiplicity, ordering, and handling unknown types +apply to the extension fields here as described \[EST_INTRO\] above. + + + +## Sending an INTRODUCE1 cell to the introduction point. \[SEND_INTRO1\] + +In order to participate in the introduction protocol, a client must +know the following: + +```text + * An introduction point for a service. + * The introduction authentication key for that introduction point. + * The introduction encryption key for that introduction point. +``` + +The client sends an INTRODUCE1 cell to the introduction point, +containing an identifier for the service, an identifier for the +encryption key that the client intends to use, and an opaque blob to +be relayed to the hidden service host. + +In reply, the introduction point sends an INTRODUCE_ACK cell back to +the client, either informing it that its request has been delivered, +or that its request will not succeed. + +```text + [TODO: specify what tor should do when receiving a malformed cell. Drop it? + Kill circuit? This goes for all possible cells.] +``` + + + +### INTRODUCE1 cell format \[FMT_INTRO1\] + +When a client is connecting to an introduction point, INTRODUCE1 cells +should be of the form: + +```text + LEGACY_KEY_ID [20 bytes] + AUTH_KEY_TYPE [1 byte] + AUTH_KEY_LEN [2 bytes] + AUTH_KEY [AUTH_KEY_LEN bytes] + N_EXTENSIONS [1 byte] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + ENCRYPTED [Up to end of relay payload] +``` + +AUTH_KEY_TYPE is defined as in \[EST_INTRO\]. Currently, the only value of +AUTH_KEY_TYPE for this cell is an Ed25519 public key \[02\]. + +The LEGACY_KEY_ID field is used to distinguish between legacy and new style +INTRODUCE1 cells. In new style INTRODUCE1 cells, LEGACY_KEY_ID is 20 zero +bytes. Upon receiving an INTRODUCE1 cell, the introduction point checks the +LEGACY_KEY_ID field. If LEGACY_KEY_ID is non-zero, the INTRODUCE1 cell +should be handled as a legacy INTRODUCE1 cell by the intro point. + +Upon receiving a INTRODUCE1 cell, the introduction point checks +whether AUTH_KEY matches the introduction point authentication key for an +active introduction circuit. If so, the introduction point sends an +INTRODUCE2 cell with exactly the same contents to the service, and sends an +INTRODUCE_ACK response to the client. + +(Note that the introduction point does not "clean up" the +INTRODUCE1 cells that it retransmits. Specifically, it does not +change the order or multiplicity of the extensions sent by the +client.) + +The same rules for multiplicity, ordering, and handling unknown types +apply to the extension fields here as described \[EST_INTRO\] above. + + + +### INTRODUCE_ACK cell format. \[INTRO_ACK\] + +An INTRODUCE_ACK cell has the following fields: + +```text + STATUS [2 bytes] + N_EXTENSIONS [1 bytes] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + + Recognized status values are: + + [00 00] -- Success: cell relayed to hidden service host. + [00 01] -- Failure: service ID not recognized + [00 02] -- Bad message format + [00 03] -- Can't relay cell to service +``` + +The same rules for multiplicity, ordering, and handling unknown types +apply to the extension fields here as described \[EST_INTRO\] above. + + + +## Processing an INTRODUCE2 cell at the hidden service. \[PROCESS_INTRO2\] + +Upon receiving an INTRODUCE2 cell, the hidden service host checks whether +the AUTH_KEY or LEGACY_KEY_ID field matches the keys for this +introduction circuit. + +The service host then checks whether it has received a cell with these +contents or rendezvous cookie before. If it has, it silently drops it as a +replay. (It must maintain a replay cache for as long as it accepts cells +with the same encryption key. Note that the encryption format below should +be non-malleable.) + +If the cell is not a replay, it decrypts the ENCRYPTED field, +establishes a shared key with the client, and authenticates the whole +contents of the cell as having been unmodified since they left the +client. There may be multiple ways of decrypting the ENCRYPTED field, +depending on the chosen type of the encryption key. Requirements for +an introduction handshake protocol are described in +\[INTRO-HANDSHAKE-REQS\]. We specify one below in section +\[NTOR-WITH-EXTRA-DATA\]. + +The decrypted plaintext must have the form: + +```text + RENDEZVOUS_COOKIE [20 bytes] + N_EXTENSIONS [1 byte] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + ONION_KEY_TYPE [1 bytes] + ONION_KEY_LEN [2 bytes] + ONION_KEY [ONION_KEY_LEN bytes] + NSPEC (Number of link specifiers) [1 byte] + NSPEC times: + LSTYPE (Link specifier type) [1 byte] + LSLEN (Link specifier length) [1 byte] + LSPEC (Link specifier) [LSLEN bytes] + PAD (optional padding) [up to end of plaintext] +``` + +Upon processing this plaintext, the hidden service makes sure that +any required authentication is present in the extension fields, and +then extends a rendezvous circuit to the node described in the LSPEC +fields, using the ONION_KEY to complete the extension. As mentioned +in \[BUILDING-BLOCKS\], the "TLS-over-TCP, IPv4" and "Legacy node +identity" specifiers must be present. + +As of 0.4.1.1-alpha, clients include both IPv4 and IPv6 link specifiers +in INTRODUCE1 cells. All available addresses SHOULD be included in the +cell, regardless of the address that the client actually used to extend +to the rendezvous point. + +The hidden service should handle invalid or unrecognised link specifiers +the same way as clients do in section 2.5.2.2. In particular, services +SHOULD perform basic validity checks on link specifiers, and SHOULD NOT +reject unrecognised link specifiers, to avoid information leaks. +The list of link specifiers received here SHOULD either be rejected, or +sent verbatim when extending to the rendezvous point, in the same order +received. + +The service MAY reject the list of link specifiers if it is +inconsistent with relay information from the directory, but SHOULD +NOT modify it. + +The ONION_KEY_TYPE field is: + +\[01\] NTOR: ONION_KEY is 32 bytes long. + +The ONION_KEY field describes the onion key that must be used when +extending to the rendezvous point. It must be of a type listed as +supported in the hidden service descriptor. + +The PAD field should be filled with zeros; its size should be chosen +so that the INTRODUCE2 message occupies a fixed maximum size, in +order to hide the length of the encrypted data. (This maximum size is +490, since we assume that a future Tor implementations will implement +proposal 340 and thus lower the number of bytes that can be contained +in a single relay message.) Note also that current versions of Tor +only pad the INTRODUCE2 message up to 246 bytes. + +Upon receiving a well-formed INTRODUCE2 cell, the hidden service host +will have: + +```text + * The information needed to connect to the client's chosen + rendezvous point. + * The second half of a handshake to authenticate and establish a + shared key with the hidden service client. + * A set of shared keys to use for end-to-end encryption. +``` + +The same rules for multiplicity, ordering, and handling unknown types +apply to the extension fields here as described \[EST_INTRO\] above. + + + +### Introduction handshake encryption requirements \[INTRO-HANDSHAKE-REQS\] + +When decoding the encrypted information in an INTRODUCE2 cell, a +hidden service host must be able to: + +```text + * Decrypt additional information included in the INTRODUCE2 cell, + to include the rendezvous token and the information needed to + extend to the rendezvous point. + + * Establish a set of shared keys for use with the client. + + * Authenticate that the cell has not been modified since the client + generated it. +``` + +Note that the old TAP-derived protocol of the previous hidden service +design achieved the first two requirements, but not the third. + +```text +3.3.2. Example encryption handshake: ntor with extra data + [NTOR-WITH-EXTRA-DATA] + + [TODO: relocate this] +``` + +This is a variant of the ntor handshake (see tor-spec.txt, section +5.1.4; see proposal 216; and see "Anonymity and one-way +authentication in key-exchange protocols" by Goldberg, Stebila, and +Ustaoglu). + +It behaves the same as the ntor handshake, except that, in addition +to negotiating forward secure keys, it also provides a means for +encrypting non-forward-secure data to the server (in this case, to +the hidden service host) as part of the handshake. + +Notation here is as in section 5.1.4 of tor-spec.txt, which defines +the ntor handshake. + +The PROTOID for this variant is "tor-hs-ntor-curve25519-sha3-256-1". +We also use the following tweak values: + +```text + t_hsenc = PROTOID | ":hs_key_extract" + t_hsverify = PROTOID | ":hs_verify" + t_hsmac = PROTOID | ":hs_mac" + m_hsexpand = PROTOID | ":hs_key_expand" +``` + +To make an INTRODUCE1 cell, the client must know a public encryption +key B for the hidden service on this introduction circuit. The client +generates a single-use keypair: + +x,X = KEYGEN() + +and computes: + +```text + intro_secret_hs_input = EXP(B,x) | AUTH_KEY | X | B | PROTOID + info = m_hsexpand | N_hs_subcred + hs_keys = KDF(intro_secret_hs_input | t_hsenc | info, S_KEY_LEN+MAC_LEN) + ENC_KEY = hs_keys[0:S_KEY_LEN] + MAC_KEY = hs_keys[S_KEY_LEN:S_KEY_LEN+MAC_KEY_LEN] + + and sends, as the ENCRYPTED part of the INTRODUCE1 cell: + + CLIENT_PK [PK_PUBKEY_LEN bytes] + ENCRYPTED_DATA [Padded to length of plaintext] + MAC [MAC_LEN bytes] +``` + +Substituting those fields into the INTRODUCE1 cell body format +described in \[FMT_INTRO1\] above, we have + +```text + LEGACY_KEY_ID [20 bytes] + AUTH_KEY_TYPE [1 byte] + AUTH_KEY_LEN [2 bytes] + AUTH_KEY [AUTH_KEY_LEN bytes] + N_EXTENSIONS [1 bytes] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + ENCRYPTED: + CLIENT_PK [PK_PUBKEY_LEN bytes] + ENCRYPTED_DATA [Padded to length of plaintext] + MAC [MAC_LEN bytes] +``` + +(This format is as documented in \[FMT_INTRO1\] above, except that here +we describe how to build the ENCRYPTED portion.) + +Here, the encryption key plays the role of B in the regular ntor +handshake, and the AUTH_KEY field plays the role of the node ID. +The CLIENT_PK field is the public key X. The ENCRYPTED_DATA field is +the message plaintext, encrypted with the symmetric key ENC_KEY. The +MAC field is a MAC of all of the cell from the AUTH_KEY through the +end of ENCRYPTED_DATA, using the MAC_KEY value as its key. + +To process this format, the hidden service checks PK_VALID(CLIENT_PK) +as necessary, and then computes ENC_KEY and MAC_KEY as the client did +above, except using EXP(CLIENT_PK,b) in the calculation of +intro_secret_hs_input. The service host then checks whether the MAC is +correct. If it is invalid, it drops the cell. Otherwise, it computes +the plaintext by decrypting ENCRYPTED_DATA. + +The hidden service host now completes the service side of the +extended ntor handshake, as described in tor-spec.txt section 5.1.4, +with the modified PROTOID as given above. To be explicit, the hidden +service host generates a keypair of y,Y = KEYGEN(), and uses its +introduction point encryption key 'b' to compute: + +```text + intro_secret_hs_input = EXP(X,b) | AUTH_KEY | X | B | PROTOID + info = m_hsexpand | N_hs_subcred + hs_keys = KDF(intro_secret_hs_input | t_hsenc | info, S_KEY_LEN+MAC_LEN) + HS_DEC_KEY = hs_keys[0:S_KEY_LEN] + HS_MAC_KEY = hs_keys[S_KEY_LEN:S_KEY_LEN+MAC_KEY_LEN] + + (The above are used to check the MAC and then decrypt the + encrypted data.) + + rend_secret_hs_input = EXP(X,y) | EXP(X,b) | AUTH_KEY | B | X | Y | PROTOID + NTOR_KEY_SEED = MAC(rend_secret_hs_input, t_hsenc) + verify = MAC(rend_secret_hs_input, t_hsverify) + auth_input = verify | AUTH_KEY | B | Y | X | PROTOID | "Server" + AUTH_INPUT_MAC = MAC(auth_input, t_hsmac) + + (The above are used to finish the ntor handshake.) + + The server's handshake reply is: + + SERVER_PK Y [PK_PUBKEY_LEN bytes] + AUTH AUTH_INPUT_MAC [MAC_LEN bytes] +``` + +These fields will be sent to the client in a RENDEZVOUS1 cell using the +HANDSHAKE_INFO element (see \[JOIN_REND\]). + +The hidden service host now also knows the keys generated by the +handshake, which it will use to encrypt and authenticate data +end-to-end between the client and the server. These keys are as +computed in tor-spec.txt section 5.1.4, except that instead of using +AES-128 and SHA1 for this hop, we use AES-256 and SHA3-256. + + + +## Authentication during the introduction phase. \[INTRO-AUTH\] + +Hidden services may restrict access only to authorized users. +One mechanism to do so is the credential mechanism, where only users who +know the credential for a hidden service may connect at all. + +There is one defined authentication type: `ed25519`. + + + +### Ed25519-based authentication `ed25519` + +(NOTE: This section is not implemented by Tor. It is likely +that we would want to change its design substantially before +deploying any implementation. At the very least, we would +want to bind these extensions to a single onion service, to +prevent replays. We might also want to look for ways to limit +the number of keys a user needs to have.) + +To authenticate with an Ed25519 private key, the user must include an +extension field in the encrypted part of the INTRODUCE1 cell with an +EXT_FIELD_TYPE type of \[02\] and the contents: + +```text + Nonce [16 bytes] + Pubkey [32 bytes] + Signature [64 bytes] +``` + +Nonce is a random value. Pubkey is the public key that will be used +to authenticate. \[TODO: should this be an identifier for the public +key instead?\] Signature is the signature, using Ed25519, of: + +```text + "hidserv-userauth-ed25519" + Nonce (same as above) + Pubkey (same as above) + AUTH_KEY (As in the INTRODUCE1 cell) +``` + +The hidden service host checks this by seeing whether it recognizes +and would accept a signature from the provided public key. If it +would, then it checks whether the signature is correct. If it is, +then the correct user has authenticated. + +Replay prevention on the whole cell is sufficient to prevent replays +on the authentication. + +Users SHOULD NOT use the same public key with multiple hidden +services. diff --git a/spec/rend-spec-v3/keyblinding-scheme.md b/spec/rend-spec-v3/keyblinding-scheme.md new file mode 100644 index 0000000..8a7b1a2 --- /dev/null +++ b/spec/rend-spec-v3/keyblinding-scheme.md @@ -0,0 +1,104 @@ + + +# Appendix A: Signature scheme with key blinding \[KEYBLIND\] + + + +## Key derivation overview + +As described in \[IMD:DIST\] and \[SUBCRED\] above, we require a "key +blinding" system that works (roughly) as follows: + +There is a master keypair (sk, pk). + +```text + Given the keypair and a nonce n, there is a derivation function + that gives a new blinded keypair (sk_n, pk_n). This keypair can + be used for signing. + + Given only the public key and the nonce, there is a function + that gives pk_n. + + Without knowing pk, it is not possible to derive pk_n; without + knowing sk, it is not possible to derive sk_n. + + It's possible to check that a signature was made with sk_n while + knowing only pk_n. + + Someone who sees a large number of blinded public keys and + signatures made using those public keys can't tell which + signatures and which blinded keys were derived from the same + master keypair. + + You can't forge signatures. + + [TODO: Insert a more rigorous definition and better references.] +``` + + + +## Tor's key derivation scheme + +We propose the following scheme for key blinding, based on Ed25519. + +(This is an ECC group, so remember that scalar multiplication is the +trapdoor function, and it's defined in terms of iterated point +addition. See the Ed25519 paper \[Reference ED25519-REFS\] for a fairly +clear writeup.) + +Let B be the ed25519 basepoint as found in section 5 of \[ED25519-B-REF\]: + +```text + B = (15112221349535400772501151409588531511454012693041857206046113283949847762202, + 46316835694926478169428394003475163141307993866256225615783033603165251855960) +``` + +Assume B has prime order l, so lB=0. Let a master keypair be written as +(a,A), where a is the private key and A is the public key (A=aB). + +To derive the key for a nonce N and an optional secret s, compute the +blinding factor like this: + +```text + h = H(BLIND_STRING | A | s | B | N) + BLIND_STRING = "Derive temporary signing key" | INT_1(0) + N = "key-blind" | INT_8(period-number) | INT_8(period_length) + B = "(1511[...]2202, 4631[...]5960)" + + then clamp the blinding factor 'h' according to the ed25519 spec: + + h[0] &= 248; + h[31] &= 63; + h[31] |= 64; + + and do the key derivation as follows: + + private key for the period: + + a' = h a mod l + RH' = SHA-512(RH_BLIND_STRING | RH)[:32] + RH_BLIND_STRING = "Derive temporary signing key hash input" + + public key for the period: + + A' = h A = (ha)B +``` + +Generating a signature of M: given a deterministic random-looking r +(see EdDSA paper), take R=rB, S=r+hash(R,A',M)ah mod l. Send signature +(R,S) and public key A'. + +Verifying the signature: Check whether SB = R+hash(R,A',M)A'. + +```text + (If the signature is valid, + SB = (r + hash(R,A',M)ah)B + = rB + (hash(R,A',M)ah)B + = R + hash(R,A',M)A' ) + + This boils down to regular Ed25519 with key pair (a', A'). +``` + +See \[KEYBLIND-REFS\] for an extensive discussion on this scheme and +possible alternatives. Also, see \[KEYBLIND-PROOF\] for a security +proof of this scheme. diff --git a/spec/rend-spec-v3/managing-authorized-client-data-client-auth-mgmt.md b/spec/rend-spec-v3/managing-authorized-client-data-client-auth-mgmt.md deleted file mode 100644 index 1cffce4..0000000 --- a/spec/rend-spec-v3/managing-authorized-client-data-client-auth-mgmt.md +++ /dev/null @@ -1,105 +0,0 @@ - - -# Appendix G: Managing authorized client data \[CLIENT-AUTH-MGMT\] - -Hidden services and clients can configure their authorized client data either -using the torrc, or using the control port. This section presents a suggested -scheme for configuring client authorization. Please see appendix -\[HIDSERVDIR-FORMAT\] for more information about relevant hidden service files. - -(NOTE: client authorization is implemented as of 0.3.5.1-alpha.) - -G.1. Configuring client authorization using torrc - -G.1.1. Hidden Service side configuration - -```text - A hidden service that wants to enable client authorization, needs to - populate the "authorized_clients/" directory of its HiddenServiceDir - directory with the ".auth" files of its authorized clients. - - When Tor starts up with a configured onion service, Tor checks its - /authorized_clients/ directory for ".auth" files, and if - any recognized and parseable such files are found, then client - authorization becomes activated for that service. - - G.1.2. Service-side bookkeeping - - This section contains more details on how onion services should be keeping - track of their client ".auth" files. - - For the "descriptor" authentication type, the ".auth" file MUST contain - the x25519 public key of that client. Here is a suggested file format: - - :: - - Here is an an example: - - descriptor:x25519:OM7TGIVRYMY6PFX6GAC6ATRTA5U6WW6U7A4ZNHQDI6OVL52XVV2Q - - Tor SHOULD ignore lines it does not recognize. - Tor SHOULD ignore files that don't use the ".auth" suffix. - - G.1.3. Client side configuration - - A client who wants to register client authorization data for onion - services needs to add the following line to their torrc to indicate the - directory which hosts ".auth_private" files containing client-side - credentials for onion services: - - ClientOnionAuthDir - - The contains a file with the suffix ".auth_private" for each onion - service the client is authorized with. Tor should scan the directory for - ".auth_private" files to find which onion services require client - authorization from this client. - - For the "descriptor" auth-type, a ".auth_private" file contains the - private x25519 key: - - :descriptor:x25519: - - The keypair used for client authorization is created by a third party tool - for which the public key needs to be transferred to the service operator - in a secure out-of-band way. The third party tool SHOULD add appropriate - headers to the private key file to ensure that users won't accidentally - give out their private key. - - G.2. Configuring client authorization using the control port - - G.2.1. Service side - - A hidden service also has the option to configure authorized clients - using the control port. The idea is that hidden service operators can use - controller utilities that manage their access control instead of using - the filesystem to register client keys. - - Specifically, we require a new control port command ADD_ONION_CLIENT_AUTH - which is able to register x25519/ed25519 public keys tied to a specific - authorized client. - [XXX figure out control port command format] - - Hidden services who use the control port interface for client auth need - to perform their own key management. - - G.2.2. Client side - - There should also be a control port interface for clients to register - authorization data for hidden services without having to use the - torrc. It should allow both generation of client authorization private - keys, and also to import client authorization data provided by a hidden - service - - This way, Tor Browser can present "Generate client auth keys" and "Import - client auth keys" dialogs to users when they try to visit a hidden service - that is protected by client authorization. - - Specifically, we require two new control port commands: - IMPORT_ONION_CLIENT_AUTH_DATA - GENERATE_ONION_CLIENT_AUTH_DATA - which import and generate client authorization data respectively. - - [XXX how does key management work here?] - [XXX what happens when people use both the control port interface and the - filesystem interface?] -``` diff --git a/spec/rend-spec-v3/numeric-values-reserved-this-document.md b/spec/rend-spec-v3/numeric-values-reserved-this-document.md deleted file mode 100644 index 7a2f01f..0000000 --- a/spec/rend-spec-v3/numeric-values-reserved-this-document.md +++ /dev/null @@ -1,5 +0,0 @@ - - -# Appendix D: Numeric values reserved in this document - -\[TODO: collect all the lists of commands and values mentioned above\] diff --git a/spec/rend-spec-v3/overview.md b/spec/rend-spec-v3/overview.md new file mode 100644 index 0000000..cf16ff4 --- /dev/null +++ b/spec/rend-spec-v3/overview.md @@ -0,0 +1,315 @@ + + +# Hidden services: overview and preliminaries + +Hidden services aim to provide responder anonymity for bidirectional +stream-based communication on the Tor network. Unlike regular Tor +connections, where the connection initiator receives anonymity but +the responder does not, hidden services attempt to provide +bidirectional anonymity. + +Participants: + +Operator -- A person running a hidden service + +```text + Host, "Server" -- The Tor software run by the operator to provide + a hidden service. + + User -- A person contacting a hidden service. + + Client -- The Tor software running on the User's computer + + Hidden Service Directory (HSDir) -- A Tor node that hosts signed + statements from hidden service hosts so that users can make + contact with them. + + Introduction Point -- A Tor node that accepts connection requests + for hidden services and anonymously relays those requests to the + hidden service. + + Rendezvous Point -- A Tor node to which clients and servers + connect and which relays traffic between them. +``` + + + +## Improvements over previous versions + +Here is a list of improvements of this proposal over the legacy hidden +services: + +a) Better crypto (replaced SHA1/DH/RSA1024 with SHA3/ed25519/curve25519) +b) Improved directory protocol leaking less to directory servers. +c) Improved directory protocol with smaller surface for targeted attacks. +d) Better onion address security against impersonation. +e) More extensible introduction/rendezvous protocol. +f) Offline keys for onion services +g) Advanced client authorization + + + +## Notation and vocabulary + +Unless specified otherwise, all multi-octet integers are big-endian. + +We write sequences of bytes in two ways: + +```text + 1. A sequence of two-digit hexadecimal values in square brackets, + as in [AB AD 1D EA]. + + 2. A string of characters enclosed in quotes, as in "Hello". The + characters in these strings are encoded in their ascii + representations; strings are NOT nul-terminated unless + explicitly described as NUL terminated. + + We use the words "byte" and "octet" interchangeably. + + We use the vertical bar | to denote concatenation. +``` + +We use INT_N(val) to denote the network (big-endian) encoding of the +unsigned integer "val" in N bytes. For example, INT_4(1337) is \[00 00 +05 39\]. Values are truncated like so: val % (2 ^ (N * 8)). For example, +INT_4(42) is 42 % 4294967296 (32 bit). + + + +## Cryptographic building blocks + +This specification uses the following cryptographic building blocks: + +```text + * A pseudorandom number generator backed by a strong entropy source. + The output of the PRNG should always be hashed before being posted on + the network to avoid leaking raw PRNG bytes to the network + (see [PRNG-REFS]). + + * A stream cipher STREAM(iv, k) where iv is a nonce of length + S_IV_LEN bytes and k is a key of length S_KEY_LEN bytes. + + * A public key signature system SIGN_KEYGEN()->seckey, pubkey; + SIGN_SIGN(seckey,msg)->sig; and SIGN_CHECK(pubkey, sig, msg) -> + { "OK", "BAD" }; where secret keys are of length SIGN_SECKEY_LEN + bytes, public keys are of length SIGN_PUBKEY_LEN bytes, and + signatures are of length SIGN_SIG_LEN bytes. + + This signature system must also support key blinding operations + as discussed in appendix [KEYBLIND] and in section [SUBCRED]: + SIGN_BLIND_SECKEY(seckey, blind)->seckey2 and + SIGN_BLIND_PUBKEY(pubkey, blind)->pubkey2 . + + * A public key agreement system "PK", providing + PK_KEYGEN()->seckey, pubkey; PK_VALID(pubkey) -> {"OK", "BAD"}; + and PK_HANDSHAKE(seckey, pubkey)->output; where secret keys are + of length PK_SECKEY_LEN bytes, public keys are of length + PK_PUBKEY_LEN bytes, and the handshake produces outputs of + length PK_OUTPUT_LEN bytes. + + * A cryptographic hash function H(d), which should be preimage and + collision resistant. It produces hashes of length HASH_LEN + bytes. + + * A cryptographic message authentication code MAC(key,msg) that + produces outputs of length MAC_LEN bytes. + + * A key derivation function KDF(message, n) that outputs n bytes. + + As a first pass, I suggest: + + * Instantiate STREAM with AES256-CTR. + + * Instantiate SIGN with Ed25519 and the blinding protocol in + [KEYBLIND]. + + * Instantiate PK with Curve25519. + + * Instantiate H with SHA3-256. + + * Instantiate KDF with SHAKE-256. + + * Instantiate MAC(key=k, message=m) with H(k_len | k | m), + where k_len is htonll(len(k)). +``` + +When we need a particular MAC key length below, we choose +MAC_KEY_LEN=32 (256 bits). + +For legacy purposes, we specify compatibility with older versions of +the Tor introduction point and rendezvous point protocols. These used +RSA1024, DH1024, AES128, and SHA1, as discussed in +rend-spec.txt. + +As in \[proposal 220\], all signatures are generated not over strings +themselves, but over those strings prefixed with a distinguishing +value. + + + +## Protocol building blocks \[BUILDING-BLOCKS\] + +In sections below, we need to transmit the locations and identities +of Tor nodes. We do so in the link identification format used by +EXTEND2 cells in the Tor protocol. + +```text + NSPEC (Number of link specifiers) [1 byte] + NSPEC times: + LSTYPE (Link specifier type) [1 byte] + LSLEN (Link specifier length) [1 byte] + LSPEC (Link specifier) [LSLEN bytes] +``` + +Link specifier types are as described in tor-spec.txt. Every set of +link specifiers SHOULD include at minimum specifiers of type \[00\] +(TLS-over-TCP, IPv4), \[02\] (legacy node identity) and \[03\] (ed25519 +identity key). Sets of link specifiers without these three types +SHOULD be rejected. + +As of 0.4.1.1-alpha, Tor includes both IPv4 and IPv6 link specifiers +in v3 onion service protocol link specifier lists. All available +addresses SHOULD be included as link specifiers, regardless of the +address that Tor actually used to connect/extend to the remote relay. + +We also incorporate Tor's circuit extension handshakes, as used in +the CREATE2 and CREATED2 cells described in tor-spec.txt. In these +handshakes, a client who knows a public key for a server sends a +message and receives a message from that server. Once the exchange is +done, the two parties have a shared set of forward-secure key +material, and the client knows that nobody else shares that key +material unless they control the secret key corresponding to the +server's public key. + + + +## Assigned relay cell types + +These relay cell types are reserved for use in the hidden service +protocol. + +32 -- RELAY_COMMAND_ESTABLISH_INTRO + +```text + Sent from hidden service host to introduction point; + establishes introduction point. Discussed in + [REG_INTRO_POINT]. + + 33 -- RELAY_COMMAND_ESTABLISH_RENDEZVOUS + + Sent from client to rendezvous point; creates rendezvous + point. Discussed in [EST_REND_POINT]. + + 34 -- RELAY_COMMAND_INTRODUCE1 + + Sent from client to introduction point; requests + introduction. Discussed in [SEND_INTRO1] + + 35 -- RELAY_COMMAND_INTRODUCE2 + + Sent from introduction point to hidden service host; requests + introduction. Same format as INTRODUCE1. Discussed in + [FMT_INTRO1] and [PROCESS_INTRO2] + + 36 -- RELAY_COMMAND_RENDEZVOUS1 + + Sent from hidden service host to rendezvous point; + attempts to join host's circuit to + client's circuit. Discussed in [JOIN_REND] + + 37 -- RELAY_COMMAND_RENDEZVOUS2 + + Sent from rendezvous point to client; + reports join of host's circuit to + client's circuit. Discussed in [JOIN_REND] + + 38 -- RELAY_COMMAND_INTRO_ESTABLISHED + + Sent from introduction point to hidden service host; + reports status of attempt to establish introduction + point. Discussed in [INTRO_ESTABLISHED] + + 39 -- RELAY_COMMAND_RENDEZVOUS_ESTABLISHED + + Sent from rendezvous point to client; acknowledges + receipt of ESTABLISH_RENDEZVOUS cell. Discussed in + [EST_REND_POINT] + + 40 -- RELAY_COMMAND_INTRODUCE_ACK + + Sent from introduction point to client; acknowledges + receipt of INTRODUCE1 cell and reports success/failure. + Discussed in [INTRO_ACK] +``` + + + +## Acknowledgments + +This design includes ideas from many people, including + +```text + Christopher Baines, + Daniel J. Bernstein, + Matthew Finkel, + Ian Goldberg, + George Kadianakis, + Aniket Kate, + Tanja Lange, + Robert Ransom, + Roger Dingledine, + Aaron Johnson, + Tim Wilson-Brown ("teor"), + special (John Brooks), + s7r +``` + +It's based on Tor's original hidden service design by Roger +Dingledine, Nick Mathewson, and Paul Syverson, and on improvements to +that design over the years by people including + +```text + Tobias Kamm, + Thomas Lauterbach, + Karsten Loesing, + Alessandro Preite Martinez, + Robert Ransom, + Ferdinand Rieger, + Christoph Weingarten, + Christian Wilms, +``` + +We wouldn't be able to do any of this work without good attack +designs from researchers including + +```text + Alex Biryukov, + Lasse Øverlier, + Ivan Pustogarov, + Paul Syverson, + Ralf-Philipp Weinmann, + + See [ATTACK-REFS] for their papers. + + Several of these ideas have come from conversations with + + Christian Grothoff, + Brian Warner, + Zooko Wilcox-O'Hearn, +``` + +And if this document makes any sense at all, it's thanks to +editing help from + +```text + Matthew Finkel, + George Kadianakis, + Peter Palfrader, + Tim Wilson-Brown ("teor"), +``` + +\[XXX Acknowledge the huge bunch of people working on 8106.\] +\[XXX Acknowledge the huge bunch of people working on 8244.\] + +Please forgive me if I've missed you; please forgive me if I've +misunderstood your best ideas here too. diff --git a/spec/rend-spec-v3/publishing-shared-random-values-pub-sharedrandom.md b/spec/rend-spec-v3/publishing-shared-random-values-pub-sharedrandom.md deleted file mode 100644 index 707c7da..0000000 --- a/spec/rend-spec-v3/publishing-shared-random-values-pub-sharedrandom.md +++ /dev/null @@ -1,49 +0,0 @@ - - -## Publishing shared random values \[PUB-SHAREDRANDOM\] - -Our design for limiting the predictability of HSDir upload locations -relies on a shared random value (SRV) that isn't predictable in advance or -too influenceable by an attacker. The authorities must run a protocol -to generate such a value at least once per hsdir period. Here we -describe how they publish these values; the procedure they use to -generate them can change independently of the rest of this -specification. For more information see \[SHAREDRANDOM-REFS\]. - -According to proposal 250, we add two new lines in consensuses: - -```text - "shared-rand-previous-value" SP NUM_REVEALS SP VALUE NL - "shared-rand-current-value" SP NUM_REVEALS SP VALUE NL -``` - - - -### Client behavior in the absence of shared random values - -If the previous or current shared random value cannot be found in a -consensus, then Tor clients and services need to generate their own random -value for use when choosing HSDirs. - -To do so, Tor clients and services use: - -SRV = H("shared-random-disaster" | INT_8(period_length) | INT_8(period_num)) - -where period_length is the length of a time period in minutes, -rounded down; period_num is calculated as specified in -\[TIME-PERIODS\] for the wanted shared random value that could not be -found originally. - - - -### Hidden services and changing shared random values - -It's theoretically possible that the consensus shared random values will -change or disappear in the middle of a time period because of directory -authorities dropping offline or misbehaving. - -To avoid client reachability issues in this rare event, hidden services -should use the new shared random values to find the new responsible HSDirs -and upload their descriptors there. - -XXX How long should they upload descriptors there for? diff --git a/spec/rend-spec-v3/recommendations-for-searching-for-vanity-onions.md b/spec/rend-spec-v3/recommendations-for-searching-for-vanity-onions.md deleted file mode 100644 index f07d108..0000000 --- a/spec/rend-spec-v3/recommendations-for-searching-for-vanity-onions.md +++ /dev/null @@ -1,45 +0,0 @@ - - -# Appendix C: Recommendations for searching for vanity .onions \[VANITY\] - -EDITORIAL NOTE: The author thinks that it's silly to brute-force the -keyspace for a key that, when base-32 encoded, spells out the name of -your website. It also feels a bit dangerous to me. If you train your -users to connect to - -llamanymityx4fi3l6x2gyzmtmgxjyqyorj9qsb5r543izcwymle.onion - -I worry that you're making it easier for somebody to trick them into -connecting to - -llamanymityb4sqi0ta0tsw6uovyhwlezkcrmczeuzdvfauuemle.onion - -Nevertheless, people are probably going to try to do this, so here's a -decent algorithm to use. - -To search for a public key with some criterion X: - -Generate a random (sk,pk) pair. - -While pk does not satisfy X: - -```text - Add the number 8 to sk - Add the point 8*B to pk - - Return sk, pk. -``` - -We add 8 and 8\*B, rather than 1 and B, so that sk is always a valid -Curve25519 private key, with the lowest 3 bits equal to 0. - -This algorithm is safe \[source: djb, personal communication\] \[TODO: -Make sure I understood correctly!\] so long as only the final (sk,pk) -pair is used, and all previous values are discarded. - -To parallelize this algorithm, start with an independent (sk,pk) pair -generated for each independent thread, and let each search proceed -independently. - -See \[VANITY-REFS\] for a reference implementation of this vanity .onion -search scheme. diff --git a/spec/rend-spec-v3/reserved-numbers-empty.md b/spec/rend-spec-v3/reserved-numbers-empty.md new file mode 100644 index 0000000..7a2f01f --- /dev/null +++ b/spec/rend-spec-v3/reserved-numbers-empty.md @@ -0,0 +1,5 @@ + + +# Appendix D: Numeric values reserved in this document + +\[TODO: collect all the lists of commands and values mentioned above\] diff --git a/spec/rend-spec-v3/revision-counter-mgt.md b/spec/rend-spec-v3/revision-counter-mgt.md new file mode 100644 index 0000000..9088cf7 --- /dev/null +++ b/spec/rend-spec-v3/revision-counter-mgt.md @@ -0,0 +1,81 @@ + + +# Appendix F: Two methods for managing revision counters + +Implementations MAY generate revision counters in any way they please, +so long as they are monotonically increasing over the lifetime of each +blinded public key. But to avoid fingerprinting, implementors SHOULD +choose a strategy also used by other Tor implementations. Here we +describe two, and additionally list some strategies that implementors +should NOT use. + +F.1. Increment-on-generation + +```text + This is the simplest strategy, and the one used by Tor through at + least version 0.3.4.0-alpha. + + Whenever using a new blinded key, the service records the + highest revision counter it has used with that key. When generating + a descriptor, the service uses the smallest non-negative number + higher than any number it has already used. + + In other words, the revision counters under this system start fresh + with each blinded key as 0, 1, 2, 3, and so on. + + F.2. Encrypted time in period + + This scheme is what we recommend for situations when multiple + service instances need to coordinate their revision counters, + without an actual coordination mechanism. + + Let T be the number of seconds that have elapsed since the descriptor + became valid, plus 1. (T must be at least 1.) Implementations can use the + number of seconds since the start time of the shared random protocol run + that corresponds to this descriptor. + + Let S be a secret that all the service providers share. For + example, it could be the private signing key corresponding to the + current blinded key. + + Let K be an AES-256 key, generated as + K = H("rev-counter-generation" | S) + + Use K, and AES in counter mode with IV=0, to generate a stream of T + * 2 bytes. Consider these bytes as a sequence of T 16-bit + little-endian words. Add these words. + + Let the sum of these words be the revision counter. + + Cryptowiki attributes roughly this scheme to G. Bebek in: + + G. Bebek. Anti-tamper database research: Inference control + techniques. Technical Report EECS 433 Final Report, Case + Western Reserve University, November 2002. + + Although we believe it is suitable for use in this application, it + is not a perfect order-preserving encryption algorithm (and all + order-preserving encryption has weaknesses). Please think twice + before using it for anything else. + + (This scheme can be optimized pretty easily by caching the encryption of + X*1, X*2, X*3, etc for some well chosen X.) + + For a slow reference implementation, see src/test/ope_ref.py in the + Tor source repository. [XXXX for now, see the same file in Nick's + "ope_hax" branch -- it isn't merged yet.] + + This scheme is not currently implemented in Tor. + + F.X. Some revision-counter strategies to avoid + + Though it might be tempting, implementations SHOULD NOT use the + current time or the current time within the period directly as their + revision counter -- doing so leaks their view of the current time, + which can be used to link the onion service to other services run on + the same host. + + Similarly, implementations SHOULD NOT let the revision counter + increase forever without resetting it -- doing so links the service + across changes in the blinded public key. +``` diff --git a/spec/rend-spec-v3/shared-random.md b/spec/rend-spec-v3/shared-random.md new file mode 100644 index 0000000..707c7da --- /dev/null +++ b/spec/rend-spec-v3/shared-random.md @@ -0,0 +1,49 @@ + + +## Publishing shared random values \[PUB-SHAREDRANDOM\] + +Our design for limiting the predictability of HSDir upload locations +relies on a shared random value (SRV) that isn't predictable in advance or +too influenceable by an attacker. The authorities must run a protocol +to generate such a value at least once per hsdir period. Here we +describe how they publish these values; the procedure they use to +generate them can change independently of the rest of this +specification. For more information see \[SHAREDRANDOM-REFS\]. + +According to proposal 250, we add two new lines in consensuses: + +```text + "shared-rand-previous-value" SP NUM_REVEALS SP VALUE NL + "shared-rand-current-value" SP NUM_REVEALS SP VALUE NL +``` + + + +### Client behavior in the absence of shared random values + +If the previous or current shared random value cannot be found in a +consensus, then Tor clients and services need to generate their own random +value for use when choosing HSDirs. + +To do so, Tor clients and services use: + +SRV = H("shared-random-disaster" | INT_8(period_length) | INT_8(period_num)) + +where period_length is the length of a time period in minutes, +rounded down; period_num is calculated as specified in +\[TIME-PERIODS\] for the wanted shared random value that could not be +found originally. + + + +### Hidden services and changing shared random values + +It's theoretically possible that the consensus shared random values will +change or disappear in the middle of a time period because of directory +authorities dropping offline or misbehaving. + +To avoid client reachability issues in this rare event, hidden services +should use the new shared random values to find the new responsible HSDirs +and upload their descriptors there. + +XXX How long should they upload descriptors there for? diff --git a/spec/rend-spec-v3/signature-scheme-with-key-blinding-keyblind.md b/spec/rend-spec-v3/signature-scheme-with-key-blinding-keyblind.md deleted file mode 100644 index 8a7b1a2..0000000 --- a/spec/rend-spec-v3/signature-scheme-with-key-blinding-keyblind.md +++ /dev/null @@ -1,104 +0,0 @@ - - -# Appendix A: Signature scheme with key blinding \[KEYBLIND\] - - - -## Key derivation overview - -As described in \[IMD:DIST\] and \[SUBCRED\] above, we require a "key -blinding" system that works (roughly) as follows: - -There is a master keypair (sk, pk). - -```text - Given the keypair and a nonce n, there is a derivation function - that gives a new blinded keypair (sk_n, pk_n). This keypair can - be used for signing. - - Given only the public key and the nonce, there is a function - that gives pk_n. - - Without knowing pk, it is not possible to derive pk_n; without - knowing sk, it is not possible to derive sk_n. - - It's possible to check that a signature was made with sk_n while - knowing only pk_n. - - Someone who sees a large number of blinded public keys and - signatures made using those public keys can't tell which - signatures and which blinded keys were derived from the same - master keypair. - - You can't forge signatures. - - [TODO: Insert a more rigorous definition and better references.] -``` - - - -## Tor's key derivation scheme - -We propose the following scheme for key blinding, based on Ed25519. - -(This is an ECC group, so remember that scalar multiplication is the -trapdoor function, and it's defined in terms of iterated point -addition. See the Ed25519 paper \[Reference ED25519-REFS\] for a fairly -clear writeup.) - -Let B be the ed25519 basepoint as found in section 5 of \[ED25519-B-REF\]: - -```text - B = (15112221349535400772501151409588531511454012693041857206046113283949847762202, - 46316835694926478169428394003475163141307993866256225615783033603165251855960) -``` - -Assume B has prime order l, so lB=0. Let a master keypair be written as -(a,A), where a is the private key and A is the public key (A=aB). - -To derive the key for a nonce N and an optional secret s, compute the -blinding factor like this: - -```text - h = H(BLIND_STRING | A | s | B | N) - BLIND_STRING = "Derive temporary signing key" | INT_1(0) - N = "key-blind" | INT_8(period-number) | INT_8(period_length) - B = "(1511[...]2202, 4631[...]5960)" - - then clamp the blinding factor 'h' according to the ed25519 spec: - - h[0] &= 248; - h[31] &= 63; - h[31] |= 64; - - and do the key derivation as follows: - - private key for the period: - - a' = h a mod l - RH' = SHA-512(RH_BLIND_STRING | RH)[:32] - RH_BLIND_STRING = "Derive temporary signing key hash input" - - public key for the period: - - A' = h A = (ha)B -``` - -Generating a signature of M: given a deterministic random-looking r -(see EdDSA paper), take R=rB, S=r+hash(R,A',M)ah mod l. Send signature -(R,S) and public key A'. - -Verifying the signature: Check whether SB = R+hash(R,A',M)A'. - -```text - (If the signature is valid, - SB = (r + hash(R,A',M)ah)B - = rB + (hash(R,A',M)ah)B - = R + hash(R,A',M)A' ) - - This boils down to regular Ed25519 with key pair (a', A'). -``` - -See \[KEYBLIND-REFS\] for an extensive discussion on this scheme and -possible alternatives. Also, see \[KEYBLIND-PROOF\] for a security -proof of this scheme. diff --git a/spec/rend-spec-v3/two-methods-for-managing-revision-counters.md b/spec/rend-spec-v3/two-methods-for-managing-revision-counters.md deleted file mode 100644 index 9088cf7..0000000 --- a/spec/rend-spec-v3/two-methods-for-managing-revision-counters.md +++ /dev/null @@ -1,81 +0,0 @@ - - -# Appendix F: Two methods for managing revision counters - -Implementations MAY generate revision counters in any way they please, -so long as they are monotonically increasing over the lifetime of each -blinded public key. But to avoid fingerprinting, implementors SHOULD -choose a strategy also used by other Tor implementations. Here we -describe two, and additionally list some strategies that implementors -should NOT use. - -F.1. Increment-on-generation - -```text - This is the simplest strategy, and the one used by Tor through at - least version 0.3.4.0-alpha. - - Whenever using a new blinded key, the service records the - highest revision counter it has used with that key. When generating - a descriptor, the service uses the smallest non-negative number - higher than any number it has already used. - - In other words, the revision counters under this system start fresh - with each blinded key as 0, 1, 2, 3, and so on. - - F.2. Encrypted time in period - - This scheme is what we recommend for situations when multiple - service instances need to coordinate their revision counters, - without an actual coordination mechanism. - - Let T be the number of seconds that have elapsed since the descriptor - became valid, plus 1. (T must be at least 1.) Implementations can use the - number of seconds since the start time of the shared random protocol run - that corresponds to this descriptor. - - Let S be a secret that all the service providers share. For - example, it could be the private signing key corresponding to the - current blinded key. - - Let K be an AES-256 key, generated as - K = H("rev-counter-generation" | S) - - Use K, and AES in counter mode with IV=0, to generate a stream of T - * 2 bytes. Consider these bytes as a sequence of T 16-bit - little-endian words. Add these words. - - Let the sum of these words be the revision counter. - - Cryptowiki attributes roughly this scheme to G. Bebek in: - - G. Bebek. Anti-tamper database research: Inference control - techniques. Technical Report EECS 433 Final Report, Case - Western Reserve University, November 2002. - - Although we believe it is suitable for use in this application, it - is not a perfect order-preserving encryption algorithm (and all - order-preserving encryption has weaknesses). Please think twice - before using it for anything else. - - (This scheme can be optimized pretty easily by caching the encryption of - X*1, X*2, X*3, etc for some well chosen X.) - - For a slow reference implementation, see src/test/ope_ref.py in the - Tor source repository. [XXXX for now, see the same file in Nick's - "ope_hax" branch -- it isn't merged yet.] - - This scheme is not currently implemented in Tor. - - F.X. Some revision-counter strategies to avoid - - Though it might be tempting, implementations SHOULD NOT use the - current time or the current time within the period directly as their - revision counter -- doing so leaks their view of the current time, - which can be used to link the onion service to other services run on - the same host. - - Similarly, implementations SHOULD NOT let the revision counter - increase forever without resetting it -- doing so links the service - across changes in the blinded public key. -``` diff --git a/spec/rend-spec-v3/vanity-onions.md b/spec/rend-spec-v3/vanity-onions.md new file mode 100644 index 0000000..f07d108 --- /dev/null +++ b/spec/rend-spec-v3/vanity-onions.md @@ -0,0 +1,45 @@ + + +# Appendix C: Recommendations for searching for vanity .onions \[VANITY\] + +EDITORIAL NOTE: The author thinks that it's silly to brute-force the +keyspace for a key that, when base-32 encoded, spells out the name of +your website. It also feels a bit dangerous to me. If you train your +users to connect to + +llamanymityx4fi3l6x2gyzmtmgxjyqyorj9qsb5r543izcwymle.onion + +I worry that you're making it easier for somebody to trick them into +connecting to + +llamanymityb4sqi0ta0tsw6uovyhwlezkcrmczeuzdvfauuemle.onion + +Nevertheless, people are probably going to try to do this, so here's a +decent algorithm to use. + +To search for a public key with some criterion X: + +Generate a random (sk,pk) pair. + +While pk does not satisfy X: + +```text + Add the number 8 to sk + Add the point 8*B to pk + + Return sk, pk. +``` + +We add 8 and 8\*B, rather than 1 and B, so that sk is always a valid +Curve25519 private key, with the lowest 3 bits equal to 0. + +This algorithm is safe \[source: djb, personal communication\] \[TODO: +Make sure I understood correctly!\] so long as only the final (sk,pk) +pair is used, and all previous values are discarded. + +To parallelize this algorithm, start with an independent (sk,pk) pair +generated for each independent thread, and let each search proceed +independently. + +See \[VANITY-REFS\] for a reference implementation of this vanity .onion +search scheme. diff --git a/spec/tor-spec/application-connections-stream-management.md b/spec/tor-spec/application-connections-stream-management.md deleted file mode 100644 index e30ce8b..0000000 --- a/spec/tor-spec/application-connections-stream-management.md +++ /dev/null @@ -1,3 +0,0 @@ - - -# Application connections and stream management diff --git a/spec/tor-spec/channels.md b/spec/tor-spec/channels.md new file mode 100644 index 0000000..9495fe4 --- /dev/null +++ b/spec/tor-spec/channels.md @@ -0,0 +1,231 @@ + + +# Connections + +Connections between two Tor relays, or between a client and a relay, +use TLS/SSLv3 for link authentication and encryption. All +implementations MUST support the SSLv3 ciphersuite +"TLS_DHE_RSA_WITH_AES_128_CBC_SHA" if it is available. They SHOULD +support better ciphersuites if available. + +There are three ways to perform TLS handshakes with a Tor server. In +the first way, "certificates-up-front", both the initiator and +responder send a two-certificate chain as part of their initial +handshake. (This is supported in all Tor versions.) In the second +way, "renegotiation", the responder provides a single certificate, +and the initiator immediately performs a TLS renegotiation. (This is +supported in Tor 0.2.0.21 and later.) And in the third way, +"in-protocol", the initial TLS negotiation completes, and the +parties bootstrap themselves to mutual authentication via use of the +Tor protocol without further TLS handshaking. (This is supported in +0.2.3.6-alpha and later.) + +Each of these options provides a way for the parties to learn it is +available: a client does not need to know the version of the Tor +server in order to connect to it properly. + +In "certificates up-front" (a.k.a "the v1 handshake"), +the connection initiator always sends a +two-certificate chain, consisting of an X.509 certificate using a +short-term connection public key and a second, self-signed X.509 +certificate containing its identity key. The other party sends a similar +certificate chain. The initiator's ClientHello MUST NOT include any +ciphersuites other than: + +```text + TLS_DHE_RSA_WITH_AES_256_CBC_SHA + TLS_DHE_RSA_WITH_AES_128_CBC_SHA + SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA +``` + +In "renegotiation" (a.k.a. "the v2 handshake"), +the connection initiator sends no certificates, and +the responder sends a single connection certificate. Once the TLS +handshake is complete, the initiator renegotiates the handshake, with each +party sending a two-certificate chain as in "certificates up-front". +The initiator's ClientHello MUST include at least one ciphersuite not in +the list above -- that's how the initiator indicates that it can +handle this handshake. For other considerations on the initiator's +ClientHello, see section 2.1 below. + +In "in-protocol" (a.k.a. "the v3 handshake"), the initiator sends no +certificates, and the +responder sends a single connection certificate. The choice of +ciphersuites must be as in a "renegotiation" handshake. There are +additionally a set of constraints on the connection certificate, +which the initiator can use to learn that the in-protocol handshake +is in use. Specifically, at least one of these properties must be +true of the certificate: + +```text + * The certificate is self-signed + * Some component other than "commonName" is set in the subject or + issuer DN of the certificate. + * The commonName of the subject or issuer of the certificate ends + with a suffix other than ".net". + * The certificate's public key modulus is longer than 1024 bits. +``` + +The initiator then sends a VERSIONS cell to the responder, which then +replies with a VERSIONS cell; they have then negotiated a Tor +protocol version. Assuming that the version they negotiate is 3 or higher +(the only ones specified for use with this handshake right now), the +responder sends a CERTS cell, an AUTH_CHALLENGE cell, and a NETINFO +cell to the initiator, which may send either CERTS, AUTHENTICATE, +NETINFO if it wants to authenticate, or just NETINFO if it does not. + +For backward compatibility between later handshakes and "certificates +up-front", the ClientHello of an initiator that supports a later +handshake MUST include at least one ciphersuite other than those listed +above. The connection responder examines the initiator's ciphersuite list +to see whether it includes any ciphers other than those included in the +list above. If extra ciphers are included, the responder proceeds as in +"renegotiation" and "in-protocol": it sends a single certificate and +does not request +client certificates. Otherwise (in the case that no extra ciphersuites +are included in the ClientHello) the responder proceeds as in +"certificates up-front": it requests client certificates, and sends a +two-certificate chain. In either case, once the responder has sent its +certificate or certificates, the initiator counts them. If two +certificates have been sent, it proceeds as in "certificates up-front"; +otherwise, it proceeds as in "renegotiation" or "in-protocol". + +To decide whether to do "renegotiation" or "in-protocol", the +initiator checks whether the responder's initial certificate matches +the criteria listed above. + +All new relay implementations of the Tor protocol MUST support +backwards-compatible renegotiation; clients SHOULD do this too. If +this is not possible, new client implementations MUST support both +"renegotiation" and "in-protocol" and use the router's +published link protocols list (see dir-spec.txt on the "protocols" entry) +to decide which to use. + +In all of the above handshake variants, certificates sent in the clear +SHOULD NOT include any strings to identify the host as a Tor relay. In +the "renegotiation" and "backwards-compatible renegotiation" steps, the +initiator SHOULD choose a list of ciphersuites and TLS extensions +to mimic one used by a popular web browser. + +Even though the connection protocol is identical, we will think of the +initiator as either an onion router (OR) if it is willing to relay +traffic for other Tor users, or an onion proxy (OP) if it only handles +local requests. Onion proxies SHOULD NOT provide long-term-trackable +identifiers in their handshakes. + +In all handshake variants, once all certificates are exchanged, all +parties receiving certificates must confirm that the identity key is as +expected. If the key is not as expected, the party must close the +connection. + +(When initiating a connection, if a reasonably live consensus is +available, then the expected identity key is taken from that +consensus. But when initiating a connection otherwise, the expected +identity key is the one given in the hard-coded authority or +fallback list. Finally, when creating a connection because of an +EXTEND/EXTEND2 cell, the expected identity key is the one given in +the cell.) + +When connecting to an OR, all parties SHOULD reject the connection if that +OR has a malformed or missing certificate. When accepting an incoming +connection, an OR SHOULD NOT reject incoming connections from parties with +malformed or missing certificates. (However, an OR should not believe +that an incoming connection is from another OR unless the certificates +are present and well-formed.) + +\[Before version 0.1.2.8-rc, ORs rejected incoming connections from ORs and +OPs alike if their certificates were missing or malformed.\] + +Once a TLS connection is established, the two sides send cells +(specified below) to one another. Cells are sent serially. Standard +cells are CELL_LEN(link_proto) bytes long, but variable-length cells +also exist; see Section 3. Cells may be sent embedded in TLS records +of any size or divided across TLS records, but the framing of TLS +records MUST NOT leak information about the type or contents of the +cells. + +TLS connections are not permanent. Either side MAY close a connection +if there are no circuits running over it and an amount of time +(KeepalivePeriod, defaults to 5 minutes) has passed since the last time +any traffic was transmitted over the TLS connection. Clients SHOULD +also hold a TLS connection with no circuits open, if it is likely that a +circuit will be built soon using that connection. + +Client-only Tor instances are encouraged to avoid using handshake +variants that include certificates, if those certificates provide +any persistent tags to the relays they contact. If clients do use +certificates, they SHOULD NOT keep using the same certificates when +their IP address changes. Clients MAY send certificates using any +of the above handshake variants. + + + +## Picking TLS ciphersuites + +Clients SHOULD send a ciphersuite list chosen to emulate some popular +web browser or other program common on the internet. Clients may send +the "Fixed Cipheruite List" below. If they do not, they MUST NOT +advertise any ciphersuite that they cannot actually support, unless that +cipher is one not supported by OpenSSL 1.0.1. + +The fixed ciphersuite list is: + +```text + TLS1_ECDHE_ECDSA_WITH_AES_256_CBC_SHA + TLS1_ECDHE_RSA_WITH_AES_256_CBC_SHA + TLS1_DHE_RSA_WITH_AES_256_SHA + TLS1_DHE_DSS_WITH_AES_256_SHA + TLS1_ECDH_RSA_WITH_AES_256_CBC_SHA + TLS1_ECDH_ECDSA_WITH_AES_256_CBC_SHA + TLS1_RSA_WITH_AES_256_SHA + TLS1_ECDHE_ECDSA_WITH_RC4_128_SHA + TLS1_ECDHE_ECDSA_WITH_AES_128_CBC_SHA + TLS1_ECDHE_RSA_WITH_RC4_128_SHA + TLS1_ECDHE_RSA_WITH_AES_128_CBC_SHA + TLS1_DHE_RSA_WITH_AES_128_SHA + TLS1_DHE_DSS_WITH_AES_128_SHA + TLS1_ECDH_RSA_WITH_RC4_128_SHA + TLS1_ECDH_RSA_WITH_AES_128_CBC_SHA + TLS1_ECDH_ECDSA_WITH_RC4_128_SHA + TLS1_ECDH_ECDSA_WITH_AES_128_CBC_SHA + SSL3_RSA_RC4_128_MD5 + SSL3_RSA_RC4_128_SHA + TLS1_RSA_WITH_AES_128_SHA + TLS1_ECDHE_ECDSA_WITH_DES_192_CBC3_SHA + TLS1_ECDHE_RSA_WITH_DES_192_CBC3_SHA + SSL3_EDH_RSA_DES_192_CBC3_SHA + SSL3_EDH_DSS_DES_192_CBC3_SHA + TLS1_ECDH_RSA_WITH_DES_192_CBC3_SHA + TLS1_ECDH_ECDSA_WITH_DES_192_CBC3_SHA + SSL3_RSA_FIPS_WITH_3DES_EDE_CBC_SHA + SSL3_RSA_DES_192_CBC3_SHA + [*] The "extended renegotiation is supported" ciphersuite, 0x00ff, is + not counted when checking the list of ciphersuites. +``` + +If the client sends the Fixed Ciphersuite List, the responder MUST NOT +select any ciphersuite besides TLS_DHE_RSA_WITH_AES_256_CBC_SHA, +TLS_DHE_RSA_WITH_AES_128_CBC_SHA, and SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA: +such ciphers might not actually be supported by the client. + +If the client sends a v2+ ClientHello with a list of ciphers other then +the Fixed Ciphersuite List, the responder can trust that the client +supports every cipher advertised in that list, so long as that ciphersuite +is also supported by OpenSSL 1.0.1. + +Responders MUST NOT select any TLS ciphersuite that lacks ephemeral keys, +or whose symmetric keys are less then KEY_LEN bits, or whose digests are +less than HASH_LEN bits. Responders SHOULD NOT select any SSLv3 +ciphersuite other than the DHE+3DES suites listed above. + + + +## TLS security considerations + +Implementations MUST NOT allow TLS session resumption -- it can +exacerbate some attacks (e.g. the "Triple Handshake" attack from +Feb 2013), and it plays havoc with forward secrecy guarantees. + +Implementations SHOULD NOT allow TLS compression -- although we don't +know a way to apply a CRIME-style attack to current Tor directly, +it's a waste of resources. diff --git a/spec/tor-spec/connections.md b/spec/tor-spec/connections.md deleted file mode 100644 index 9495fe4..0000000 --- a/spec/tor-spec/connections.md +++ /dev/null @@ -1,231 +0,0 @@ - - -# Connections - -Connections between two Tor relays, or between a client and a relay, -use TLS/SSLv3 for link authentication and encryption. All -implementations MUST support the SSLv3 ciphersuite -"TLS_DHE_RSA_WITH_AES_128_CBC_SHA" if it is available. They SHOULD -support better ciphersuites if available. - -There are three ways to perform TLS handshakes with a Tor server. In -the first way, "certificates-up-front", both the initiator and -responder send a two-certificate chain as part of their initial -handshake. (This is supported in all Tor versions.) In the second -way, "renegotiation", the responder provides a single certificate, -and the initiator immediately performs a TLS renegotiation. (This is -supported in Tor 0.2.0.21 and later.) And in the third way, -"in-protocol", the initial TLS negotiation completes, and the -parties bootstrap themselves to mutual authentication via use of the -Tor protocol without further TLS handshaking. (This is supported in -0.2.3.6-alpha and later.) - -Each of these options provides a way for the parties to learn it is -available: a client does not need to know the version of the Tor -server in order to connect to it properly. - -In "certificates up-front" (a.k.a "the v1 handshake"), -the connection initiator always sends a -two-certificate chain, consisting of an X.509 certificate using a -short-term connection public key and a second, self-signed X.509 -certificate containing its identity key. The other party sends a similar -certificate chain. The initiator's ClientHello MUST NOT include any -ciphersuites other than: - -```text - TLS_DHE_RSA_WITH_AES_256_CBC_SHA - TLS_DHE_RSA_WITH_AES_128_CBC_SHA - SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA -``` - -In "renegotiation" (a.k.a. "the v2 handshake"), -the connection initiator sends no certificates, and -the responder sends a single connection certificate. Once the TLS -handshake is complete, the initiator renegotiates the handshake, with each -party sending a two-certificate chain as in "certificates up-front". -The initiator's ClientHello MUST include at least one ciphersuite not in -the list above -- that's how the initiator indicates that it can -handle this handshake. For other considerations on the initiator's -ClientHello, see section 2.1 below. - -In "in-protocol" (a.k.a. "the v3 handshake"), the initiator sends no -certificates, and the -responder sends a single connection certificate. The choice of -ciphersuites must be as in a "renegotiation" handshake. There are -additionally a set of constraints on the connection certificate, -which the initiator can use to learn that the in-protocol handshake -is in use. Specifically, at least one of these properties must be -true of the certificate: - -```text - * The certificate is self-signed - * Some component other than "commonName" is set in the subject or - issuer DN of the certificate. - * The commonName of the subject or issuer of the certificate ends - with a suffix other than ".net". - * The certificate's public key modulus is longer than 1024 bits. -``` - -The initiator then sends a VERSIONS cell to the responder, which then -replies with a VERSIONS cell; they have then negotiated a Tor -protocol version. Assuming that the version they negotiate is 3 or higher -(the only ones specified for use with this handshake right now), the -responder sends a CERTS cell, an AUTH_CHALLENGE cell, and a NETINFO -cell to the initiator, which may send either CERTS, AUTHENTICATE, -NETINFO if it wants to authenticate, or just NETINFO if it does not. - -For backward compatibility between later handshakes and "certificates -up-front", the ClientHello of an initiator that supports a later -handshake MUST include at least one ciphersuite other than those listed -above. The connection responder examines the initiator's ciphersuite list -to see whether it includes any ciphers other than those included in the -list above. If extra ciphers are included, the responder proceeds as in -"renegotiation" and "in-protocol": it sends a single certificate and -does not request -client certificates. Otherwise (in the case that no extra ciphersuites -are included in the ClientHello) the responder proceeds as in -"certificates up-front": it requests client certificates, and sends a -two-certificate chain. In either case, once the responder has sent its -certificate or certificates, the initiator counts them. If two -certificates have been sent, it proceeds as in "certificates up-front"; -otherwise, it proceeds as in "renegotiation" or "in-protocol". - -To decide whether to do "renegotiation" or "in-protocol", the -initiator checks whether the responder's initial certificate matches -the criteria listed above. - -All new relay implementations of the Tor protocol MUST support -backwards-compatible renegotiation; clients SHOULD do this too. If -this is not possible, new client implementations MUST support both -"renegotiation" and "in-protocol" and use the router's -published link protocols list (see dir-spec.txt on the "protocols" entry) -to decide which to use. - -In all of the above handshake variants, certificates sent in the clear -SHOULD NOT include any strings to identify the host as a Tor relay. In -the "renegotiation" and "backwards-compatible renegotiation" steps, the -initiator SHOULD choose a list of ciphersuites and TLS extensions -to mimic one used by a popular web browser. - -Even though the connection protocol is identical, we will think of the -initiator as either an onion router (OR) if it is willing to relay -traffic for other Tor users, or an onion proxy (OP) if it only handles -local requests. Onion proxies SHOULD NOT provide long-term-trackable -identifiers in their handshakes. - -In all handshake variants, once all certificates are exchanged, all -parties receiving certificates must confirm that the identity key is as -expected. If the key is not as expected, the party must close the -connection. - -(When initiating a connection, if a reasonably live consensus is -available, then the expected identity key is taken from that -consensus. But when initiating a connection otherwise, the expected -identity key is the one given in the hard-coded authority or -fallback list. Finally, when creating a connection because of an -EXTEND/EXTEND2 cell, the expected identity key is the one given in -the cell.) - -When connecting to an OR, all parties SHOULD reject the connection if that -OR has a malformed or missing certificate. When accepting an incoming -connection, an OR SHOULD NOT reject incoming connections from parties with -malformed or missing certificates. (However, an OR should not believe -that an incoming connection is from another OR unless the certificates -are present and well-formed.) - -\[Before version 0.1.2.8-rc, ORs rejected incoming connections from ORs and -OPs alike if their certificates were missing or malformed.\] - -Once a TLS connection is established, the two sides send cells -(specified below) to one another. Cells are sent serially. Standard -cells are CELL_LEN(link_proto) bytes long, but variable-length cells -also exist; see Section 3. Cells may be sent embedded in TLS records -of any size or divided across TLS records, but the framing of TLS -records MUST NOT leak information about the type or contents of the -cells. - -TLS connections are not permanent. Either side MAY close a connection -if there are no circuits running over it and an amount of time -(KeepalivePeriod, defaults to 5 minutes) has passed since the last time -any traffic was transmitted over the TLS connection. Clients SHOULD -also hold a TLS connection with no circuits open, if it is likely that a -circuit will be built soon using that connection. - -Client-only Tor instances are encouraged to avoid using handshake -variants that include certificates, if those certificates provide -any persistent tags to the relays they contact. If clients do use -certificates, they SHOULD NOT keep using the same certificates when -their IP address changes. Clients MAY send certificates using any -of the above handshake variants. - - - -## Picking TLS ciphersuites - -Clients SHOULD send a ciphersuite list chosen to emulate some popular -web browser or other program common on the internet. Clients may send -the "Fixed Cipheruite List" below. If they do not, they MUST NOT -advertise any ciphersuite that they cannot actually support, unless that -cipher is one not supported by OpenSSL 1.0.1. - -The fixed ciphersuite list is: - -```text - TLS1_ECDHE_ECDSA_WITH_AES_256_CBC_SHA - TLS1_ECDHE_RSA_WITH_AES_256_CBC_SHA - TLS1_DHE_RSA_WITH_AES_256_SHA - TLS1_DHE_DSS_WITH_AES_256_SHA - TLS1_ECDH_RSA_WITH_AES_256_CBC_SHA - TLS1_ECDH_ECDSA_WITH_AES_256_CBC_SHA - TLS1_RSA_WITH_AES_256_SHA - TLS1_ECDHE_ECDSA_WITH_RC4_128_SHA - TLS1_ECDHE_ECDSA_WITH_AES_128_CBC_SHA - TLS1_ECDHE_RSA_WITH_RC4_128_SHA - TLS1_ECDHE_RSA_WITH_AES_128_CBC_SHA - TLS1_DHE_RSA_WITH_AES_128_SHA - TLS1_DHE_DSS_WITH_AES_128_SHA - TLS1_ECDH_RSA_WITH_RC4_128_SHA - TLS1_ECDH_RSA_WITH_AES_128_CBC_SHA - TLS1_ECDH_ECDSA_WITH_RC4_128_SHA - TLS1_ECDH_ECDSA_WITH_AES_128_CBC_SHA - SSL3_RSA_RC4_128_MD5 - SSL3_RSA_RC4_128_SHA - TLS1_RSA_WITH_AES_128_SHA - TLS1_ECDHE_ECDSA_WITH_DES_192_CBC3_SHA - TLS1_ECDHE_RSA_WITH_DES_192_CBC3_SHA - SSL3_EDH_RSA_DES_192_CBC3_SHA - SSL3_EDH_DSS_DES_192_CBC3_SHA - TLS1_ECDH_RSA_WITH_DES_192_CBC3_SHA - TLS1_ECDH_ECDSA_WITH_DES_192_CBC3_SHA - SSL3_RSA_FIPS_WITH_3DES_EDE_CBC_SHA - SSL3_RSA_DES_192_CBC3_SHA - [*] The "extended renegotiation is supported" ciphersuite, 0x00ff, is - not counted when checking the list of ciphersuites. -``` - -If the client sends the Fixed Ciphersuite List, the responder MUST NOT -select any ciphersuite besides TLS_DHE_RSA_WITH_AES_256_CBC_SHA, -TLS_DHE_RSA_WITH_AES_128_CBC_SHA, and SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA: -such ciphers might not actually be supported by the client. - -If the client sends a v2+ ClientHello with a list of ciphers other then -the Fixed Ciphersuite List, the responder can trust that the client -supports every cipher advertised in that list, so long as that ciphersuite -is also supported by OpenSSL 1.0.1. - -Responders MUST NOT select any TLS ciphersuite that lacks ephemeral keys, -or whose symmetric keys are less then KEY_LEN bits, or whose digests are -less than HASH_LEN bits. Responders SHOULD NOT select any SSLv3 -ciphersuite other than the DHE+3DES suites listed above. - - - -## TLS security considerations - -Implementations MUST NOT allow TLS session resumption -- it can -exacerbate some attacks (e.g. the "Triple Handshake" attack from -Feb 2013), and it plays havoc with forward secrecy guarantees. - -Implementations SHOULD NOT allow TLS compression -- although we don't -know a way to apply a CRIME-style attack to current Tor directly, -it's a waste of resources. diff --git a/spec/tor-spec/handling-relay_early-cells.md b/spec/tor-spec/handling-relay_early-cells.md deleted file mode 100644 index 2517dcc..0000000 --- a/spec/tor-spec/handling-relay_early-cells.md +++ /dev/null @@ -1,20 +0,0 @@ - - -## Handling relay_early cells - -A RELAY_EARLY cell is designed to limit the length any circuit can reach. -When an OR receives a RELAY_EARLY cell, and the next node in the circuit -is speaking v2 of the link protocol or later, the OR relays the cell as a -RELAY_EARLY cell. Otherwise, older Tors will relay it as a RELAY cell. - -If a node ever receives more than 8 RELAY_EARLY cells on a given -outbound circuit, it SHOULD close the circuit. If it receives any -inbound RELAY_EARLY cells, it MUST close the circuit immediately. - -When speaking v2 of the link protocol or later, clients MUST only send -EXTEND/EXTEND2 cells inside RELAY_EARLY cells. Clients SHOULD send the first -~8 RELAY cells that are not targeted at the first hop of any circuit as -RELAY_EARLY cells too, in order to partially conceal the circuit length. - -\[Starting with Tor 0.2.3.11-alpha, relays should reject any -EXTEND/EXTEND2 cell not received in a RELAY_EARLY cell.\] diff --git a/spec/tor-spec/handling-resource-exhaustion.md b/spec/tor-spec/handling-resource-exhaustion.md deleted file mode 100644 index 7179066..0000000 --- a/spec/tor-spec/handling-resource-exhaustion.md +++ /dev/null @@ -1,33 +0,0 @@ - - -# Handling resource exhaustion - - - -## Memory exhaustion - -(See also dos-spec.md.) - -If RAM becomes low, an OR should begin destroying circuits until -more memory is free again. We recommend the following algorithm: - -- Set a threshold amount of RAM to recover at 10% of the total RAM. - -```text - - Sort the circuits by their 'staleness', defined as the age of the - oldest data queued on the circuit. This data can be: - - * Bytes that are waiting to flush to or from a stream on that - circuit. - - * Bytes that are waiting to flush from a connection created with - BEGIN_DIR. - - * Cells that are waiting to flush or be processed. - - - While we have not yet recovered enough RAM: - - * Free all memory held by the most stale circuit, and send DESTROY - cells in both directions on that circuit. Count the amount of - memory we recovered towards the total. -``` diff --git a/spec/tor-spec/negotiating-channels.md b/spec/tor-spec/negotiating-channels.md new file mode 100644 index 0000000..2b8ebd6 --- /dev/null +++ b/spec/tor-spec/negotiating-channels.md @@ -0,0 +1,410 @@ + + +# Negotiating and initializing connections + +After Tor instances negotiate handshake with either the "renegotiation" or +"in-protocol" handshakes, they must exchange a set of cells to set up +the Tor connection and make it "open" and usable for circuits. + +When the renegotiation handshake is used, both parties immediately +send a VERSIONS cell (4.1 below), and after negotiating a link +protocol version (which will be 2), each send a NETINFO cell (4.5 +below) to confirm their addresses and timestamps. No other intervening +cell types are allowed. + +When the in-protocol handshake is used, the initiator sends a +VERSIONS cell to indicate that it will not be renegotiating. The +responder sends a VERSIONS cell, a CERTS cell (4.2 below) to give the +initiator the certificates it needs to learn the responder's +identity, an AUTH_CHALLENGE cell (4.3) that the initiator must include +as part of its answer if it chooses to authenticate, and a NETINFO +cell (4.5). As soon as it gets the CERTS cell, the initiator knows +whether the responder is correctly authenticated. At this point the +initiator behaves differently depending on whether it wants to +authenticate or not. If it does not want to authenticate, it MUST +send a NETINFO cell. If it does want to authenticate, it MUST send a +CERTS cell, an AUTHENTICATE cell (4.4), and a NETINFO. When this +handshake is in use, the first cell must be VERSIONS, VPADDING, or +AUTHORIZE, and no other cell type is allowed to intervene besides +those specified, except for VPADDING cells. + +The AUTHORIZE cell type is reserved for future use by scanning-resistance +designs. + +\[Tor versions before 0.2.3.11-alpha did not recognize the AUTHORIZE cell, +and did not permit any command other than VERSIONS as the first cell of +the in-protocol handshake.\] + + + +## Negotiating versions with VERSIONS cells + +There are multiple instances of the Tor link connection protocol. Any +connection negotiated using the "certificates up front" handshake (see +section 2 above) is "version 1". In any connection where both parties +have behaved as in the "renegotiation" handshake, the link protocol +version must be 2. In any connection where both parties have behaved +as in the "in-protocol" handshake, the link protocol must be 3 or higher. + +To determine the version, in any connection where the "renegotiation" +or "in-protocol" handshake was used (that is, where the responder +sent only one certificate at first and where the initiator did not +send any certificates in the first negotiation), both parties MUST +send a VERSIONS cell. In "renegotiation", they send a VERSIONS cell +right after the renegotiation is finished, before any other cells are +sent. In "in-protocol", the initiator sends a VERSIONS cell +immediately after the initial TLS handshake, and the responder +replies immediately with a VERSIONS cell. (As an exception to this rule, +if both sides support the "in-protocol" handshake, either side may send +VPADDING cells at any time.) + +The payload in a VERSIONS cell is a series of big-endian two-byte +integers. Both parties MUST select as the link protocol version the +highest number contained both in the VERSIONS cell they sent and in the +versions cell they received. If they have no such version in common, +they cannot communicate and MUST close the connection. Either party MUST +close the connection if the versions cell is not well-formed (for example, +if the payload contains an odd number of bytes). + +Any VERSIONS cells sent after the first VERSIONS cell MUST be ignored. +(To be interpreted correctly, later VERSIONS cells MUST have a CIRCID_LEN +matching the version negotiated with the first VERSIONS cell.) + +Since the version 1 link protocol does not use the "renegotiation" +handshake, implementations MUST NOT list version 1 in their VERSIONS +cell. When the "renegotiation" handshake is used, implementations +MUST list only the version 2. When the "in-protocol" handshake is +used, implementations MUST NOT list any version before 3, and SHOULD +list at least version 3. + +Link protocols differences are: + +```text + 1 -- The "certs up front" handshake. + 2 -- Uses the renegotiation-based handshake. Introduces + variable-length cells. + 3 -- Uses the in-protocol handshake. + 4 -- Increases circuit ID width to 4 bytes. + 5 -- Adds support for link padding and negotiation (padding-spec.txt). +``` + + + +## CERTS cells + +The CERTS cell describes the keys that a Tor instance is claiming +to have. It is a variable-length cell. Its payload format is: + +```text + N: Number of certs in cell [1 octet] + N times: + CertType [1 octet] + CLEN [2 octets] + Certificate [CLEN octets] + + Any extra octets at the end of a CERTS cell MUST be ignored. + + Relevant certType values are: + 1: Link key certificate certified by RSA1024 identity + 2: RSA1024 Identity certificate, self-signed. + 3: RSA1024 AUTHENTICATE cell link certificate, signed with RSA1024 key. + 4: Ed25519 signing key, signed with identity key. + 5: TLS link certificate, signed with ed25519 signing key. + 6: Ed25519 AUTHENTICATE cell key, signed with ed25519 signing key. + 7: Ed25519 identity, signed with RSA identity. +``` + +The certificate format for certificate types 1-3 is DER encoded +X509. For others, the format is as documented in cert-spec.txt. +Note that type 7 uses a different format from types 4-6. + +A CERTS cell may have no more than one certificate of each CertType. + +To authenticate the responder as having a given Ed25519,RSA identity key +combination, the initiator MUST check the following. + +```text + * The CERTS cell contains exactly one CertType 2 "ID" certificate. + * The CERTS cell contains exactly one CertType 4 Ed25519 + "Id->Signing" cert. + * The CERTS cell contains exactly one CertType 5 Ed25519 + "Signing->link" certificate. + * The CERTS cell contains exactly one CertType 7 "RSA->Ed25519" + cross-certificate. + * All X.509 certificates above have validAfter and validUntil dates; + no X.509 or Ed25519 certificates are expired. + * All certificates are correctly signed. + * The certified key in the Signing->Link certificate matches the + SHA256 digest of the certificate that was used to + authenticate the TLS connection. + * The identity key listed in the ID->Signing cert was used to + sign the ID->Signing Cert. + * The Signing->Link cert was signed with the Signing key listed + in the ID->Signing cert. + * The RSA->Ed25519 cross-certificate certifies the Ed25519 + identity, and is signed with the RSA identity listed in the + "ID" certificate. + * The certified key in the ID certificate is a 1024-bit RSA key. + * The RSA ID certificate is correctly self-signed. +``` + +To authenticate the responder as having a given RSA identity only, +the initiator MUST check the following: + +```text + * The CERTS cell contains exactly one CertType 1 "Link" certificate. + * The CERTS cell contains exactly one CertType 2 "ID" certificate. + * Both certificates have validAfter and validUntil dates that + are not expired. + * The certified key in the Link certificate matches the + link key that was used to negotiate the TLS connection. + * The certified key in the ID certificate is a 1024-bit RSA key. + * The certified key in the ID certificate was used to sign both + certificates. + * The link certificate is correctly signed with the key in the + ID certificate + * The ID certificate is correctly self-signed. +``` + +In both cases above, checking these conditions is sufficient to +authenticate that the initiator is talking to the Tor node with the +expected identity, as certified in the ID certificate(s). + +To authenticate the initiator as having a given Ed25519,RSA +identity key combination, the responder MUST check the following: + +```text + * The CERTS cell contains exactly one CertType 2 "ID" certificate. + * The CERTS cell contains exactly one CertType 4 Ed25519 + "Id->Signing" certificate. + * The CERTS cell contains exactly one CertType 6 Ed25519 + "Signing->auth" certificate. + * The CERTS cell contains exactly one CertType 7 "RSA->Ed25519" + cross-certificate. + * All X.509 certificates above have validAfter and validUntil dates; + no X.509 or Ed25519 certificates are expired. + * All certificates are correctly signed. + * The identity key listed in the ID->Signing cert was used to + sign the ID->Signing Cert. + * The Signing->AUTH cert was signed with the Signing key listed + in the ID->Signing cert. + * The RSA->Ed25519 cross-certificate certifies the Ed25519 + identity, and is signed with the RSA identity listed in the + "ID" certificate. + * The certified key in the ID certificate is a 1024-bit RSA key. + * The RSA ID certificate is correctly self-signed. +``` + +To authenticate the initiator as having an RSA identity key only, +the responder MUST check the following: + +```text + * The CERTS cell contains exactly one CertType 3 "AUTH" certificate. + * The CERTS cell contains exactly one CertType 2 "ID" certificate. + * Both certificates have validAfter and validUntil dates that + are not expired. + * The certified key in the AUTH certificate is a 1024-bit RSA key. + * The certified key in the ID certificate is a 1024-bit RSA key. + * The certified key in the ID certificate was used to sign both + certificates. + * The auth certificate is correctly signed with the key in the + ID certificate. + * The ID certificate is correctly self-signed. +``` + +Checking these conditions is NOT sufficient to authenticate that the +initiator has the ID it claims; to do so, the cells in 4.3 and 4.4 +below must be exchanged. + + + +## AUTH_CHALLENGE cells + +An AUTH_CHALLENGE cell is a variable-length cell with the following +fields: + +```text + Challenge [32 octets] + N_Methods [2 octets] + Methods [2 * N_Methods octets] +``` + +It is sent from the responder to the initiator. Initiators MUST +ignore unexpected bytes at the end of the cell. Responders MUST +generate every challenge independently using a strong RNG or PRNG. + +The Challenge field is a randomly generated string that the +initiator must sign (a hash of) as part of authenticating. The +methods are the authentication methods that the responder will +accept. Only two authentication methods are defined right now: +see 4.4.1 and 4.4.2 below. + + + +## AUTHENTICATE cells + +If an initiator wants to authenticate, it responds to the +AUTH_CHALLENGE cell with a CERTS cell and an AUTHENTICATE cell. +The CERTS cell is as a server would send, except that instead of +sending a CertType 1 (and possibly CertType 5) certs for arbitrary link +certificates, the initiator sends a CertType 3 (and possibly +CertType 6) cert for an RSA/Ed25519 AUTHENTICATE key. + +This difference is because we allow any link key type on a TLS +link, but the protocol described here will only work for specific key +types as described in 4.4.1 and 4.4.2 below. + +An AUTHENTICATE cell contains the following: + +```text + AuthType [2 octets] + AuthLen [2 octets] + Authentication [AuthLen octets] +``` + +Responders MUST ignore extra bytes at the end of an AUTHENTICATE +cell. Recognized AuthTypes are 1 and 3, described in the next +two sections. + +Initiators MUST NOT send an AUTHENTICATE cell before they have +verified the certificates presented in the responder's CERTS +cell, and authenticated the responder. + + + +### Link authentication type 1: RSA-SHA256-TLSSecret + +If AuthType is 1 (meaning "RSA-SHA256-TLSSecret"), then the +Authentication field of the AUTHENTICATE cell contains the following: + +```text + TYPE: The characters "AUTH0001" [8 octets] + CID: A SHA256 hash of the initiator's RSA1024 identity key [32 octets] + SID: A SHA256 hash of the responder's RSA1024 identity key [32 octets] + SLOG: A SHA256 hash of all bytes sent from the responder to the + initiator as part of the negotiation up to and including the + AUTH_CHALLENGE cell; that is, the VERSIONS cell, the CERTS cell, + the AUTH_CHALLENGE cell, and any padding cells. [32 octets] + CLOG: A SHA256 hash of all bytes sent from the initiator to the + responder as part of the negotiation so far; that is, the + VERSIONS cell and the CERTS cell and any padding cells. [32 + octets] + SCERT: A SHA256 hash of the responder's TLS link certificate. [32 + octets] + TLSSECRETS: A SHA256 HMAC, using the TLS master secret as the + secret key, of the following: + - client_random, as sent in the TLS Client Hello + - server_random, as sent in the TLS Server Hello + - the NUL terminated ASCII string: + "Tor V3 handshake TLS cross-certification" + [32 octets] + RAND: A 24 byte value, randomly chosen by the initiator. (In an + imitation of SSL3's gmt_unix_time field, older versions of Tor + sent an 8-byte timestamp as the first 8 bytes of this field; + new implementations should not do that.) [24 octets] + SIG: A signature of a SHA256 hash of all the previous fields + using the initiator's "Authenticate" key as presented. (As + always in Tor, we use OAEP-MGF1 padding; see tor-spec.txt + section 0.3.) + [variable length] +``` + +To check the AUTHENTICATE cell, a responder checks that all fields +from TYPE through TLSSECRETS contain their unique +correct values as described above, and then verifies the signature. +The server MUST ignore any extra bytes in the signed data after +the RAND field. + +Responders MUST NOT accept this AuthType if the initiator has +claimed to have an Ed25519 identity. + +(There is no AuthType 2: It was reserved but never implemented.) + + + +### Link authentication type 3: Ed25519-SHA256-RFC5705 + +If AuthType is 3, meaning "Ed25519-SHA256-RFC5705", the +Authentication field of the AuthType cell is as below: + +Modified values and new fields below are marked with asterisks. + +```text + TYPE: The characters "AUTH0003" [8 octets] + CID: A SHA256 hash of the initiator's RSA1024 identity key [32 octets] + SID: A SHA256 hash of the responder's RSA1024 identity key [32 octets] + CID_ED: The initiator's Ed25519 identity key [32 octets] + SID_ED: The responder's Ed25519 identity key, or all-zero. [32 octets] + SLOG: A SHA256 hash of all bytes sent from the responder to the + initiator as part of the negotiation up to and including the + AUTH_CHALLENGE cell; that is, the VERSIONS cell, the CERTS cell, + the AUTH_CHALLENGE cell, and any padding cells. [32 octets] + CLOG: A SHA256 hash of all bytes sent from the initiator to the + responder as part of the negotiation so far; that is, the + VERSIONS cell and the CERTS cell and any padding cells. [32 + octets] + SCERT: A SHA256 hash of the responder's TLS link certificate. [32 + octets] + TLSSECRETS: The output of an RFC5705 Exporter function on the + TLS session, using as its inputs: + - The label string "EXPORTER FOR TOR TLS CLIENT BINDING AUTH0003" + - The context value equal to the initiator's Ed25519 identity key. + - The length 32. + [32 octets] + RAND: A 24 byte value, randomly chosen by the initiator. [24 octets] + SIG: A signature of all previous fields using the initiator's + Ed25519 authentication key (as in the cert with CertType 6). + [variable length] +``` + +To check the AUTHENTICATE cell, a responder checks that all fields +from TYPE through TLSSECRETS contain their unique +correct values as described above, and then verifies the signature. +The server MUST ignore any extra bytes in the signed data after +the RAND field. + + + +## NETINFO cells + +If version 2 or higher is negotiated, each party sends the other a +NETINFO cell. The cell's payload is: + +```text + TIME (Timestamp) [4 bytes] + OTHERADDR (Other OR's address) [variable] + ATYPE (Address type) [1 byte] + ALEN (Address length) [1 byte] + AVAL (Address value in NBO) [ALEN bytes] + NMYADDR (Number of this OR's addresses) [1 byte] + NMYADDR times: + ATYPE (Address type) [1 byte] + ALEN (Address length) [1 byte] + AVAL (Address value in NBO)) [ALEN bytes] + + Recognized address types (ATYPE) are: + + [04] IPv4. + [06] IPv6. +``` + +ALEN MUST be 4 when ATYPE is 0x04 (IPv4) and 16 when ATYPE is 0x06 +(IPv6). If the ALEN value is wrong for the given ATYPE value, then +the provided address should be ignored. + +The timestamp is a big-endian unsigned integer number of seconds +since the Unix epoch. Implementations MUST ignore unexpected bytes +at the end of the cell. Clients SHOULD send "0" as their timestamp, to +avoid fingerprinting. + +Implementations MAY use the timestamp value to help decide if their +clocks are skewed. Initiators MAY use "other OR's address" to help +learn which address their connections may be originating from, if they do +not know it; and to learn whether the peer will treat the current +connection as canonical. Implementations SHOULD NOT trust these +values unconditionally, especially when they come from non-authorities, +since the other party can lie about the time or IP addresses it sees. + +Initiators SHOULD use "this OR's address" to make sure +that they have connected to another OR at its canonical address. +(See 5.3.1 below.) diff --git a/spec/tor-spec/negotiating-initializing-connections.md b/spec/tor-spec/negotiating-initializing-connections.md deleted file mode 100644 index 2b8ebd6..0000000 --- a/spec/tor-spec/negotiating-initializing-connections.md +++ /dev/null @@ -1,410 +0,0 @@ - - -# Negotiating and initializing connections - -After Tor instances negotiate handshake with either the "renegotiation" or -"in-protocol" handshakes, they must exchange a set of cells to set up -the Tor connection and make it "open" and usable for circuits. - -When the renegotiation handshake is used, both parties immediately -send a VERSIONS cell (4.1 below), and after negotiating a link -protocol version (which will be 2), each send a NETINFO cell (4.5 -below) to confirm their addresses and timestamps. No other intervening -cell types are allowed. - -When the in-protocol handshake is used, the initiator sends a -VERSIONS cell to indicate that it will not be renegotiating. The -responder sends a VERSIONS cell, a CERTS cell (4.2 below) to give the -initiator the certificates it needs to learn the responder's -identity, an AUTH_CHALLENGE cell (4.3) that the initiator must include -as part of its answer if it chooses to authenticate, and a NETINFO -cell (4.5). As soon as it gets the CERTS cell, the initiator knows -whether the responder is correctly authenticated. At this point the -initiator behaves differently depending on whether it wants to -authenticate or not. If it does not want to authenticate, it MUST -send a NETINFO cell. If it does want to authenticate, it MUST send a -CERTS cell, an AUTHENTICATE cell (4.4), and a NETINFO. When this -handshake is in use, the first cell must be VERSIONS, VPADDING, or -AUTHORIZE, and no other cell type is allowed to intervene besides -those specified, except for VPADDING cells. - -The AUTHORIZE cell type is reserved for future use by scanning-resistance -designs. - -\[Tor versions before 0.2.3.11-alpha did not recognize the AUTHORIZE cell, -and did not permit any command other than VERSIONS as the first cell of -the in-protocol handshake.\] - - - -## Negotiating versions with VERSIONS cells - -There are multiple instances of the Tor link connection protocol. Any -connection negotiated using the "certificates up front" handshake (see -section 2 above) is "version 1". In any connection where both parties -have behaved as in the "renegotiation" handshake, the link protocol -version must be 2. In any connection where both parties have behaved -as in the "in-protocol" handshake, the link protocol must be 3 or higher. - -To determine the version, in any connection where the "renegotiation" -or "in-protocol" handshake was used (that is, where the responder -sent only one certificate at first and where the initiator did not -send any certificates in the first negotiation), both parties MUST -send a VERSIONS cell. In "renegotiation", they send a VERSIONS cell -right after the renegotiation is finished, before any other cells are -sent. In "in-protocol", the initiator sends a VERSIONS cell -immediately after the initial TLS handshake, and the responder -replies immediately with a VERSIONS cell. (As an exception to this rule, -if both sides support the "in-protocol" handshake, either side may send -VPADDING cells at any time.) - -The payload in a VERSIONS cell is a series of big-endian two-byte -integers. Both parties MUST select as the link protocol version the -highest number contained both in the VERSIONS cell they sent and in the -versions cell they received. If they have no such version in common, -they cannot communicate and MUST close the connection. Either party MUST -close the connection if the versions cell is not well-formed (for example, -if the payload contains an odd number of bytes). - -Any VERSIONS cells sent after the first VERSIONS cell MUST be ignored. -(To be interpreted correctly, later VERSIONS cells MUST have a CIRCID_LEN -matching the version negotiated with the first VERSIONS cell.) - -Since the version 1 link protocol does not use the "renegotiation" -handshake, implementations MUST NOT list version 1 in their VERSIONS -cell. When the "renegotiation" handshake is used, implementations -MUST list only the version 2. When the "in-protocol" handshake is -used, implementations MUST NOT list any version before 3, and SHOULD -list at least version 3. - -Link protocols differences are: - -```text - 1 -- The "certs up front" handshake. - 2 -- Uses the renegotiation-based handshake. Introduces - variable-length cells. - 3 -- Uses the in-protocol handshake. - 4 -- Increases circuit ID width to 4 bytes. - 5 -- Adds support for link padding and negotiation (padding-spec.txt). -``` - - - -## CERTS cells - -The CERTS cell describes the keys that a Tor instance is claiming -to have. It is a variable-length cell. Its payload format is: - -```text - N: Number of certs in cell [1 octet] - N times: - CertType [1 octet] - CLEN [2 octets] - Certificate [CLEN octets] - - Any extra octets at the end of a CERTS cell MUST be ignored. - - Relevant certType values are: - 1: Link key certificate certified by RSA1024 identity - 2: RSA1024 Identity certificate, self-signed. - 3: RSA1024 AUTHENTICATE cell link certificate, signed with RSA1024 key. - 4: Ed25519 signing key, signed with identity key. - 5: TLS link certificate, signed with ed25519 signing key. - 6: Ed25519 AUTHENTICATE cell key, signed with ed25519 signing key. - 7: Ed25519 identity, signed with RSA identity. -``` - -The certificate format for certificate types 1-3 is DER encoded -X509. For others, the format is as documented in cert-spec.txt. -Note that type 7 uses a different format from types 4-6. - -A CERTS cell may have no more than one certificate of each CertType. - -To authenticate the responder as having a given Ed25519,RSA identity key -combination, the initiator MUST check the following. - -```text - * The CERTS cell contains exactly one CertType 2 "ID" certificate. - * The CERTS cell contains exactly one CertType 4 Ed25519 - "Id->Signing" cert. - * The CERTS cell contains exactly one CertType 5 Ed25519 - "Signing->link" certificate. - * The CERTS cell contains exactly one CertType 7 "RSA->Ed25519" - cross-certificate. - * All X.509 certificates above have validAfter and validUntil dates; - no X.509 or Ed25519 certificates are expired. - * All certificates are correctly signed. - * The certified key in the Signing->Link certificate matches the - SHA256 digest of the certificate that was used to - authenticate the TLS connection. - * The identity key listed in the ID->Signing cert was used to - sign the ID->Signing Cert. - * The Signing->Link cert was signed with the Signing key listed - in the ID->Signing cert. - * The RSA->Ed25519 cross-certificate certifies the Ed25519 - identity, and is signed with the RSA identity listed in the - "ID" certificate. - * The certified key in the ID certificate is a 1024-bit RSA key. - * The RSA ID certificate is correctly self-signed. -``` - -To authenticate the responder as having a given RSA identity only, -the initiator MUST check the following: - -```text - * The CERTS cell contains exactly one CertType 1 "Link" certificate. - * The CERTS cell contains exactly one CertType 2 "ID" certificate. - * Both certificates have validAfter and validUntil dates that - are not expired. - * The certified key in the Link certificate matches the - link key that was used to negotiate the TLS connection. - * The certified key in the ID certificate is a 1024-bit RSA key. - * The certified key in the ID certificate was used to sign both - certificates. - * The link certificate is correctly signed with the key in the - ID certificate - * The ID certificate is correctly self-signed. -``` - -In both cases above, checking these conditions is sufficient to -authenticate that the initiator is talking to the Tor node with the -expected identity, as certified in the ID certificate(s). - -To authenticate the initiator as having a given Ed25519,RSA -identity key combination, the responder MUST check the following: - -```text - * The CERTS cell contains exactly one CertType 2 "ID" certificate. - * The CERTS cell contains exactly one CertType 4 Ed25519 - "Id->Signing" certificate. - * The CERTS cell contains exactly one CertType 6 Ed25519 - "Signing->auth" certificate. - * The CERTS cell contains exactly one CertType 7 "RSA->Ed25519" - cross-certificate. - * All X.509 certificates above have validAfter and validUntil dates; - no X.509 or Ed25519 certificates are expired. - * All certificates are correctly signed. - * The identity key listed in the ID->Signing cert was used to - sign the ID->Signing Cert. - * The Signing->AUTH cert was signed with the Signing key listed - in the ID->Signing cert. - * The RSA->Ed25519 cross-certificate certifies the Ed25519 - identity, and is signed with the RSA identity listed in the - "ID" certificate. - * The certified key in the ID certificate is a 1024-bit RSA key. - * The RSA ID certificate is correctly self-signed. -``` - -To authenticate the initiator as having an RSA identity key only, -the responder MUST check the following: - -```text - * The CERTS cell contains exactly one CertType 3 "AUTH" certificate. - * The CERTS cell contains exactly one CertType 2 "ID" certificate. - * Both certificates have validAfter and validUntil dates that - are not expired. - * The certified key in the AUTH certificate is a 1024-bit RSA key. - * The certified key in the ID certificate is a 1024-bit RSA key. - * The certified key in the ID certificate was used to sign both - certificates. - * The auth certificate is correctly signed with the key in the - ID certificate. - * The ID certificate is correctly self-signed. -``` - -Checking these conditions is NOT sufficient to authenticate that the -initiator has the ID it claims; to do so, the cells in 4.3 and 4.4 -below must be exchanged. - - - -## AUTH_CHALLENGE cells - -An AUTH_CHALLENGE cell is a variable-length cell with the following -fields: - -```text - Challenge [32 octets] - N_Methods [2 octets] - Methods [2 * N_Methods octets] -``` - -It is sent from the responder to the initiator. Initiators MUST -ignore unexpected bytes at the end of the cell. Responders MUST -generate every challenge independently using a strong RNG or PRNG. - -The Challenge field is a randomly generated string that the -initiator must sign (a hash of) as part of authenticating. The -methods are the authentication methods that the responder will -accept. Only two authentication methods are defined right now: -see 4.4.1 and 4.4.2 below. - - - -## AUTHENTICATE cells - -If an initiator wants to authenticate, it responds to the -AUTH_CHALLENGE cell with a CERTS cell and an AUTHENTICATE cell. -The CERTS cell is as a server would send, except that instead of -sending a CertType 1 (and possibly CertType 5) certs for arbitrary link -certificates, the initiator sends a CertType 3 (and possibly -CertType 6) cert for an RSA/Ed25519 AUTHENTICATE key. - -This difference is because we allow any link key type on a TLS -link, but the protocol described here will only work for specific key -types as described in 4.4.1 and 4.4.2 below. - -An AUTHENTICATE cell contains the following: - -```text - AuthType [2 octets] - AuthLen [2 octets] - Authentication [AuthLen octets] -``` - -Responders MUST ignore extra bytes at the end of an AUTHENTICATE -cell. Recognized AuthTypes are 1 and 3, described in the next -two sections. - -Initiators MUST NOT send an AUTHENTICATE cell before they have -verified the certificates presented in the responder's CERTS -cell, and authenticated the responder. - - - -### Link authentication type 1: RSA-SHA256-TLSSecret - -If AuthType is 1 (meaning "RSA-SHA256-TLSSecret"), then the -Authentication field of the AUTHENTICATE cell contains the following: - -```text - TYPE: The characters "AUTH0001" [8 octets] - CID: A SHA256 hash of the initiator's RSA1024 identity key [32 octets] - SID: A SHA256 hash of the responder's RSA1024 identity key [32 octets] - SLOG: A SHA256 hash of all bytes sent from the responder to the - initiator as part of the negotiation up to and including the - AUTH_CHALLENGE cell; that is, the VERSIONS cell, the CERTS cell, - the AUTH_CHALLENGE cell, and any padding cells. [32 octets] - CLOG: A SHA256 hash of all bytes sent from the initiator to the - responder as part of the negotiation so far; that is, the - VERSIONS cell and the CERTS cell and any padding cells. [32 - octets] - SCERT: A SHA256 hash of the responder's TLS link certificate. [32 - octets] - TLSSECRETS: A SHA256 HMAC, using the TLS master secret as the - secret key, of the following: - - client_random, as sent in the TLS Client Hello - - server_random, as sent in the TLS Server Hello - - the NUL terminated ASCII string: - "Tor V3 handshake TLS cross-certification" - [32 octets] - RAND: A 24 byte value, randomly chosen by the initiator. (In an - imitation of SSL3's gmt_unix_time field, older versions of Tor - sent an 8-byte timestamp as the first 8 bytes of this field; - new implementations should not do that.) [24 octets] - SIG: A signature of a SHA256 hash of all the previous fields - using the initiator's "Authenticate" key as presented. (As - always in Tor, we use OAEP-MGF1 padding; see tor-spec.txt - section 0.3.) - [variable length] -``` - -To check the AUTHENTICATE cell, a responder checks that all fields -from TYPE through TLSSECRETS contain their unique -correct values as described above, and then verifies the signature. -The server MUST ignore any extra bytes in the signed data after -the RAND field. - -Responders MUST NOT accept this AuthType if the initiator has -claimed to have an Ed25519 identity. - -(There is no AuthType 2: It was reserved but never implemented.) - - - -### Link authentication type 3: Ed25519-SHA256-RFC5705 - -If AuthType is 3, meaning "Ed25519-SHA256-RFC5705", the -Authentication field of the AuthType cell is as below: - -Modified values and new fields below are marked with asterisks. - -```text - TYPE: The characters "AUTH0003" [8 octets] - CID: A SHA256 hash of the initiator's RSA1024 identity key [32 octets] - SID: A SHA256 hash of the responder's RSA1024 identity key [32 octets] - CID_ED: The initiator's Ed25519 identity key [32 octets] - SID_ED: The responder's Ed25519 identity key, or all-zero. [32 octets] - SLOG: A SHA256 hash of all bytes sent from the responder to the - initiator as part of the negotiation up to and including the - AUTH_CHALLENGE cell; that is, the VERSIONS cell, the CERTS cell, - the AUTH_CHALLENGE cell, and any padding cells. [32 octets] - CLOG: A SHA256 hash of all bytes sent from the initiator to the - responder as part of the negotiation so far; that is, the - VERSIONS cell and the CERTS cell and any padding cells. [32 - octets] - SCERT: A SHA256 hash of the responder's TLS link certificate. [32 - octets] - TLSSECRETS: The output of an RFC5705 Exporter function on the - TLS session, using as its inputs: - - The label string "EXPORTER FOR TOR TLS CLIENT BINDING AUTH0003" - - The context value equal to the initiator's Ed25519 identity key. - - The length 32. - [32 octets] - RAND: A 24 byte value, randomly chosen by the initiator. [24 octets] - SIG: A signature of all previous fields using the initiator's - Ed25519 authentication key (as in the cert with CertType 6). - [variable length] -``` - -To check the AUTHENTICATE cell, a responder checks that all fields -from TYPE through TLSSECRETS contain their unique -correct values as described above, and then verifies the signature. -The server MUST ignore any extra bytes in the signed data after -the RAND field. - - - -## NETINFO cells - -If version 2 or higher is negotiated, each party sends the other a -NETINFO cell. The cell's payload is: - -```text - TIME (Timestamp) [4 bytes] - OTHERADDR (Other OR's address) [variable] - ATYPE (Address type) [1 byte] - ALEN (Address length) [1 byte] - AVAL (Address value in NBO) [ALEN bytes] - NMYADDR (Number of this OR's addresses) [1 byte] - NMYADDR times: - ATYPE (Address type) [1 byte] - ALEN (Address length) [1 byte] - AVAL (Address value in NBO)) [ALEN bytes] - - Recognized address types (ATYPE) are: - - [04] IPv4. - [06] IPv6. -``` - -ALEN MUST be 4 when ATYPE is 0x04 (IPv4) and 16 when ATYPE is 0x06 -(IPv6). If the ALEN value is wrong for the given ATYPE value, then -the provided address should be ignored. - -The timestamp is a big-endian unsigned integer number of seconds -since the Unix epoch. Implementations MUST ignore unexpected bytes -at the end of the cell. Clients SHOULD send "0" as their timestamp, to -avoid fingerprinting. - -Implementations MAY use the timestamp value to help decide if their -clocks are skewed. Initiators MAY use "other OR's address" to help -learn which address their connections may be originating from, if they do -not know it; and to learn whether the peer will treat the current -connection as canonical. Implementations SHOULD NOT trust these -values unconditionally, especially when they come from non-authorities, -since the other party can lie about the time or IP addresses it sees. - -Initiators SHOULD use "this OR's address" to make sure -that they have connected to another OR at its canonical address. -(See 5.3.1 below.) diff --git a/spec/tor-spec/opening-streams-transferring-data.md b/spec/tor-spec/opening-streams-transferring-data.md deleted file mode 100644 index 956e2c8..0000000 --- a/spec/tor-spec/opening-streams-transferring-data.md +++ /dev/null @@ -1,119 +0,0 @@ - - -## Opening streams and transferring data - -To open a new anonymized TCP connection, the OP chooses an open -circuit to an exit that may be able to connect to the destination -address, selects an arbitrary StreamID not yet used on that circuit, -and constructs a RELAY_BEGIN cell with a payload encoding the address -and port of the destination host. The payload format is: - -```text - ADDRPORT [nul-terminated string] - FLAGS [4 bytes] - - ADDRPORT is made of ADDRESS | ':' | PORT | [00] -``` - -where ADDRESS can be a DNS hostname, or an IPv4 address in -dotted-quad format, or an IPv6 address surrounded by square brackets; -and where PORT is a decimal integer between 1 and 65535, inclusive. - -The ADDRPORT string SHOULD be sent in lower case, to avoid -fingerprinting. Implementations MUST accept strings in any case. - -The FLAGS value has one or more of the following bits set, where -"bit 1" is the LSB of the 32-bit value, and "bit 32" is the MSB. -(Remember that all values in Tor are big-endian (see 0.1.1 above), so -the MSB of a 4-byte value is the MSB of the first byte, and the LSB -of a 4-byte value is the LSB of its last byte.) - -```text - bit meaning - 1 -- IPv6 okay. We support learning about IPv6 addresses and - connecting to IPv6 addresses. - 2 -- IPv4 not okay. We don't want to learn about IPv4 addresses - or connect to them. - 3 -- IPv6 preferred. If there are both IPv4 and IPv6 addresses, - we want to connect to the IPv6 one. (By default, we connect - to the IPv4 address.) - 4..32 -- Reserved. Current clients MUST NOT set these. Servers - MUST ignore them. -``` - -Upon receiving this cell, the exit node resolves the address as -necessary, and opens a new TCP connection to the target port. If the -address cannot be resolved, or a connection can't be established, the -exit node replies with a RELAY_END cell. (See 6.3 below.) -Otherwise, the exit node replies with a RELAY_CONNECTED cell, whose -payload is in one of the following formats: - -```text - The IPv4 address to which the connection was made [4 octets] - A number of seconds (TTL) for which the address may be cached [4 octets] - - or - - Four zero-valued octets [4 octets] - An address type (6) [1 octet] - The IPv6 address to which the connection was made [16 octets] - A number of seconds (TTL) for which the address may be cached [4 octets] -``` - -\[Tor exit nodes before 0.1.2.0 set the TTL field to a fixed value. Later -versions set the TTL to the last value seen from a DNS server, and expire -their own cached entries after a fixed interval. This prevents certain -attacks.\] - -Once a connection has been established, the OP and exit node -package stream data in RELAY_DATA cells, and upon receiving such -cells, echo their contents to the corresponding TCP stream. - -If the exit node does not support optimistic data (i.e. its -version number is before 0.2.3.1-alpha), then the OP MUST wait -for a RELAY_CONNECTED cell before sending any data. If the exit -node supports optimistic data (i.e. its version number is -0.2.3.1-alpha or later), then the OP MAY send RELAY_DATA cells -immediately after sending the RELAY_BEGIN cell (and before -receiving either a RELAY_CONNECTED or RELAY_END cell). - -RELAY_DATA cells sent to unrecognized streams are dropped. If -the exit node supports optimistic data, then RELAY_DATA cells it -receives on streams which have seen RELAY_BEGIN but have not yet -been replied to with a RELAY_CONNECTED or RELAY_END are queued. -If the stream creation succeeds with a RELAY_CONNECTED, the queue -is processed immediately afterwards; if the stream creation fails -with a RELAY_END, the contents of the queue are deleted. - -Relay RELAY_DROP cells are long-range dummies; upon receiving such -a cell, the OR or OP must drop it. - - - -### Opening a directory stream - -If a Tor relay is a directory server, it should respond to a -RELAY_BEGIN_DIR cell as if it had received a BEGIN cell requesting a -connection to its directory port. RELAY_BEGIN_DIR cells ignore exit -policy, since the stream is local to the Tor process. - -Directory servers may be: - -- authoritative directories (RELAY_BEGIN_DIR, usually non-anonymous), -- bridge authoritative directories (RELAY_BEGIN_DIR, anonymous), -- directory mirrors (RELAY_BEGIN_DIR, usually non-anonymous), -- onion service directories (RELAY_BEGIN_DIR, anonymous). - -If the Tor relay is not running a directory service, it should respond -with a REASON_NOTDIRECTORY RELAY_END cell. - -Clients MUST generate an all-zero payload for RELAY_BEGIN_DIR cells, -and relays MUST ignore the payload. - -In response to a RELAY_BEGIN_DIR cell, relays respond either with a -RELAY_CONNECTED cell on success, or a RELAY_END cell on failure. They -MUST send a RELAY_CONNECTED cell all-zero payload, and clients MUST ignore -the payload. - -\[RELAY_BEGIN_DIR was not supported before Tor 0.1.2.2-alpha; clients -SHOULD NOT send it to routers running earlier versions of Tor.\] diff --git a/spec/tor-spec/opening-streams.md b/spec/tor-spec/opening-streams.md new file mode 100644 index 0000000..956e2c8 --- /dev/null +++ b/spec/tor-spec/opening-streams.md @@ -0,0 +1,119 @@ + + +## Opening streams and transferring data + +To open a new anonymized TCP connection, the OP chooses an open +circuit to an exit that may be able to connect to the destination +address, selects an arbitrary StreamID not yet used on that circuit, +and constructs a RELAY_BEGIN cell with a payload encoding the address +and port of the destination host. The payload format is: + +```text + ADDRPORT [nul-terminated string] + FLAGS [4 bytes] + + ADDRPORT is made of ADDRESS | ':' | PORT | [00] +``` + +where ADDRESS can be a DNS hostname, or an IPv4 address in +dotted-quad format, or an IPv6 address surrounded by square brackets; +and where PORT is a decimal integer between 1 and 65535, inclusive. + +The ADDRPORT string SHOULD be sent in lower case, to avoid +fingerprinting. Implementations MUST accept strings in any case. + +The FLAGS value has one or more of the following bits set, where +"bit 1" is the LSB of the 32-bit value, and "bit 32" is the MSB. +(Remember that all values in Tor are big-endian (see 0.1.1 above), so +the MSB of a 4-byte value is the MSB of the first byte, and the LSB +of a 4-byte value is the LSB of its last byte.) + +```text + bit meaning + 1 -- IPv6 okay. We support learning about IPv6 addresses and + connecting to IPv6 addresses. + 2 -- IPv4 not okay. We don't want to learn about IPv4 addresses + or connect to them. + 3 -- IPv6 preferred. If there are both IPv4 and IPv6 addresses, + we want to connect to the IPv6 one. (By default, we connect + to the IPv4 address.) + 4..32 -- Reserved. Current clients MUST NOT set these. Servers + MUST ignore them. +``` + +Upon receiving this cell, the exit node resolves the address as +necessary, and opens a new TCP connection to the target port. If the +address cannot be resolved, or a connection can't be established, the +exit node replies with a RELAY_END cell. (See 6.3 below.) +Otherwise, the exit node replies with a RELAY_CONNECTED cell, whose +payload is in one of the following formats: + +```text + The IPv4 address to which the connection was made [4 octets] + A number of seconds (TTL) for which the address may be cached [4 octets] + + or + + Four zero-valued octets [4 octets] + An address type (6) [1 octet] + The IPv6 address to which the connection was made [16 octets] + A number of seconds (TTL) for which the address may be cached [4 octets] +``` + +\[Tor exit nodes before 0.1.2.0 set the TTL field to a fixed value. Later +versions set the TTL to the last value seen from a DNS server, and expire +their own cached entries after a fixed interval. This prevents certain +attacks.\] + +Once a connection has been established, the OP and exit node +package stream data in RELAY_DATA cells, and upon receiving such +cells, echo their contents to the corresponding TCP stream. + +If the exit node does not support optimistic data (i.e. its +version number is before 0.2.3.1-alpha), then the OP MUST wait +for a RELAY_CONNECTED cell before sending any data. If the exit +node supports optimistic data (i.e. its version number is +0.2.3.1-alpha or later), then the OP MAY send RELAY_DATA cells +immediately after sending the RELAY_BEGIN cell (and before +receiving either a RELAY_CONNECTED or RELAY_END cell). + +RELAY_DATA cells sent to unrecognized streams are dropped. If +the exit node supports optimistic data, then RELAY_DATA cells it +receives on streams which have seen RELAY_BEGIN but have not yet +been replied to with a RELAY_CONNECTED or RELAY_END are queued. +If the stream creation succeeds with a RELAY_CONNECTED, the queue +is processed immediately afterwards; if the stream creation fails +with a RELAY_END, the contents of the queue are deleted. + +Relay RELAY_DROP cells are long-range dummies; upon receiving such +a cell, the OR or OP must drop it. + + + +### Opening a directory stream + +If a Tor relay is a directory server, it should respond to a +RELAY_BEGIN_DIR cell as if it had received a BEGIN cell requesting a +connection to its directory port. RELAY_BEGIN_DIR cells ignore exit +policy, since the stream is local to the Tor process. + +Directory servers may be: + +- authoritative directories (RELAY_BEGIN_DIR, usually non-anonymous), +- bridge authoritative directories (RELAY_BEGIN_DIR, anonymous), +- directory mirrors (RELAY_BEGIN_DIR, usually non-anonymous), +- onion service directories (RELAY_BEGIN_DIR, anonymous). + +If the Tor relay is not running a directory service, it should respond +with a REASON_NOTDIRECTORY RELAY_END cell. + +Clients MUST generate an all-zero payload for RELAY_BEGIN_DIR cells, +and relays MUST ignore the payload. + +In response to a RELAY_BEGIN_DIR cell, relays respond either with a +RELAY_CONNECTED cell on success, or a RELAY_END cell on failure. They +MUST send a RELAY_CONNECTED cell all-zero payload, and clients MUST ignore +the payload. + +\[RELAY_BEGIN_DIR was not supported before Tor 0.1.2.2-alpha; clients +SHOULD NOT send it to routers running earlier versions of Tor.\] diff --git a/spec/tor-spec/relay-early.md b/spec/tor-spec/relay-early.md new file mode 100644 index 0000000..2517dcc --- /dev/null +++ b/spec/tor-spec/relay-early.md @@ -0,0 +1,20 @@ + + +## Handling relay_early cells + +A RELAY_EARLY cell is designed to limit the length any circuit can reach. +When an OR receives a RELAY_EARLY cell, and the next node in the circuit +is speaking v2 of the link protocol or later, the OR relays the cell as a +RELAY_EARLY cell. Otherwise, older Tors will relay it as a RELAY cell. + +If a node ever receives more than 8 RELAY_EARLY cells on a given +outbound circuit, it SHOULD close the circuit. If it receives any +inbound RELAY_EARLY cells, it MUST close the circuit immediately. + +When speaking v2 of the link protocol or later, clients MUST only send +EXTEND/EXTEND2 cells inside RELAY_EARLY cells. Clients SHOULD send the first +~8 RELAY cells that are not targeted at the first hop of any circuit as +RELAY_EARLY cells too, in order to partially conceal the circuit length. + +\[Starting with Tor 0.2.3.11-alpha, relays should reject any +EXTEND/EXTEND2 cell not received in a RELAY_EARLY cell.\] diff --git a/spec/tor-spec/resource-exhaustion.md b/spec/tor-spec/resource-exhaustion.md new file mode 100644 index 0000000..7179066 --- /dev/null +++ b/spec/tor-spec/resource-exhaustion.md @@ -0,0 +1,33 @@ + + +# Handling resource exhaustion + + + +## Memory exhaustion + +(See also dos-spec.md.) + +If RAM becomes low, an OR should begin destroying circuits until +more memory is free again. We recommend the following algorithm: + +- Set a threshold amount of RAM to recover at 10% of the total RAM. + +```text + - Sort the circuits by their 'staleness', defined as the age of the + oldest data queued on the circuit. This data can be: + + * Bytes that are waiting to flush to or from a stream on that + circuit. + + * Bytes that are waiting to flush from a connection created with + BEGIN_DIR. + + * Cells that are waiting to flush or be processed. + + - While we have not yet recovered enough RAM: + + * Free all memory held by the most stale circuit, and send DESTROY + cells in both directions on that circuit. Count the amount of + memory we recovered towards the total. +``` diff --git a/spec/tor-spec/streams.md b/spec/tor-spec/streams.md new file mode 100644 index 0000000..e30ce8b --- /dev/null +++ b/spec/tor-spec/streams.md @@ -0,0 +1,3 @@ + + +# Application connections and stream management -- cgit v1.2.3-54-g00ecf