Move all MD-only specifications into spec/.

2 files changed, 166 insertions, 0 deletions

diff --git a/spec/README.md b/spec/README.md
new file mode 100644
index 0000000..944f029
--- /dev/null
+++ b/spec/README.md
@@ -0,0 +1,69 @@
+# Tor specifications
+
+This repository holds the specifications that describe how Tor works.
+They try to present Tor's protocols in sufficient detail to allow
+the reader to implement a compatible implementation of Tor without ever
+having to read the Tor source code.
+
+The [proposals](/proposals) directory holds our design proposals.  These
+include historical documents that have now been merged into the main
+specifications.  For more information on the proposal process, including an
+explanation of how to make new proposals, see
+[001-process.txt](/proposals/001-process.txt).
+
+## What you can find here
+
+Tor's specification is pretty big, and we've broken it into a bunch of
+files.
+
+* General interest
+  * [tor-spec.txt](tor-spec.txt)
+    contains the specification for the core Tor protocol
+    itself; this is a good place to start reading.
+  * [cert-spec.txt](cert-spec.txt) describes a certificate format used
+    in the other parts of the protocol.
+  * [dir-spec.txt](dir-spec.txt) specifies the operations and formats used to
+    maintain a view of the network directory.
+  * [padding-spec.txt](padding-spec.txt) describes a set of padding mechanisms
+    used to impede traffic analysis.
+  * [version-spec.txt](version-spec.txt) explains how to parse Tor
+    version numbers.
+  * [glossary.txt](glossary.txt) is a glossary of terms used
+    in the other specifications.
+* Client operations
+  * [address-spec.txt](address-spec.txt) lists a set of special
+    addresses that Tor handles differently from the regular DNS system.
+  * [guard-spec.txt](guard-spec.txt) explains the "guard node" algorithm
+    that Tor clients use to avoid sampling attacks.
+  * [path-spec.txt](path-spec.txt) explains how clients choose their paths
+    through the Tor network.
+  * [socks-extensions](socks-extensions.txt) specifies Tor-specific
+    extensions to the SOCKS protocol.
+* Onion services
+  * [rend-spec-v2.txt](rend-spec-v2.txt) is the old, deprecated version
+    of the onion service protocol.
+  * [rend-spec-v3.txt](rend-spec-v3.txt) is the current version of the
+    onion service protocol.
+* Censorship resistance
+  * [bridgedb-spec.txt](bridgedb-spec.txt) explains how the `bridgedb`
+    server gives out bridges to censored clients.
+  * [gettor-spec.txt](gettor-spec.txt) describes the `gettor` tool,
+    which is used to download Tor in censored areas.
+  * [pt-spec.txt](pt-spec.txt) describes the protocol that Tor clients
+    and relays  use to communicate with pluggable transports used for
+    traffic obfuscation.
+* Directory authorities
+  * [bandwidth-file-spec.txt](bandwidth-file-spec.txt) specifies the
+    file format used by bandwidth-measuring tools to report their
+    observations to directory authorities.
+  * [srv-spec.txt](srv-spec.txt) specifies the protocol that
+    directory authorities use to securely compute shared random values
+    for the network.
+* Controller protocol
+  * [control-spec.txt](control-spec.txt) explains the protocol used by
+    controllers to communicate with a running Tor process.
+* Miscellaneous
+  * [dir-list-spec.txt](dir-list-spec.txt) explains the format used by
+    tools like the fallback directory scripts to output a list of
+    Tor directories for inclusion in the Tor source code.
+  * The [attic](attic) directory has obsolete or historical documents.
diff --git a/spec/dos-spec.md b/spec/dos-spec.md
new file mode 100644
index 0000000..04470b2
--- /dev/null
+++ b/spec/dos-spec.md
@@ -0,0 +1,97 @@
+# Denial-of-service prevention mechanisms in Tor
+
+This document is incomplete; it describes some mechanisms that Tor
+uses to avoid different kinds of denial-of-service attacks.
+
+## Handling low-memory conditions
+
+(See also `tor-spec.txt`, section 8.1.)
+
+The Tor protocol requires clients, onion services, relays, and
+authorities to store various kind of information in buffers and
+caches.  But an attacker can use these buffers and queues to queues
+to exhaust the memory of the a targeted Tor process, and force the
+operating system to kill that process.
+
+Worse still, the ability to kill targeted Tor instances can be used
+to facilitate traffic analysis. (For example, see
+[the "Sniper Attack" paper](https://www.freehaven.net/anonbib/#sniper14)
+by Jansen, Tschorsch, Johnson, and Scheuermann.
+
+With this in mind, any Tor implementation—especially one that
+runs as a relay or onion service—must take steps to prevent
+memory-based denial-of-service attacks.
+
+### Detecting low memory
+
+The easiest way to notice you're out of memory would, in theory, be
+getting an error when you try to allocate more.  Unfortunately, some
+systems (e.g. Linux) won't actually give you an "out of memory"
+error when you're low on memory.  Instead, they overcommit and
+promise you memory that they can't actually provide… and then later on,
+they might kill processes that actually try to use more memory than
+they wish they'd given out.
+
+So in practice, the mainline Tor implementation uses a different
+strategy.  It uses a self-imposed "MaxMemInQueues" value as an
+upper bound for how much memory it's willing to allocate to certain
+kinds of queued usages.  This value can either be set by the user,
+or derived from a fraction of the total amount of system RAM.
+
+As of Tor 0.4.7.x, the MaxMemInQueues mechanism tracks the following
+kinds of allocation:
+  * Cells queued on circuits.
+  * Per-connection read or write buffers.
+  * On-the-fly compression or decompression state.
+  * Half-open stream records.
+  * Cached onion service descriptors (hsdir only).
+  * Cached DNS resolves (relay only).
+  * GEOIP-based usage activity statistics.
+
+Note that directory caches aren't counted, since those are stored on
+disk and accessed via mmap.
+
+### Responding to low memory
+
+If our allocations exceed MaxMemInQueues, then we take the following
+steps to reduce our memory allocation.
+
+*Freeing from caches*: For each of our onion service descriptor
+cache, our DNS cache, and our GEOIP statistics cache, we check
+whether they account for greater than 20% of our total allocation.
+If they do, we free memory from the offending cache until the total
+remaining is no more than 10% of our total allocation.
+
+When freeing entries from a cache, we aim to free (approximately)
+the oldest entries first.
+
+*Freeing from buffers*: After freeing data from caches, we see
+whether allocations are still above 90% of MaxMemInQueues. If they
+are, we try to close circuits and connections until we are below 90%
+of MaxMemInQueues.
+
+When deciding to what circuits to free, we sort them based on the
+age of the oldest data in their queues, and free the ones with the
+oldest data.  (For example, a circuit on which a single cell has
+been queued for 5 minutes would be freed before a circuit where 100
+cells have been queued for 5 seconds.)  "Data queued on a circuit"
+includes all data that we could drop if the circuit were destroyed:
+not only the cells on the circuit's cell queue, but also any bytes
+queued in buffers associated with streams or half-stream records
+attached to the circuit.
+
+We free non-tunneled directory connections according to a similar
+rule, according to the age of their oldest queued data.
+
+Upon freeing a circuit, a "DESTROY cell" must be sent in both
+directions.
+
+### Reporting low memory.
+
+We define a "low threshold" equal to 3/4 of MaxMemInQueues.  Every
+time our memory usage is above the low threshold, we record
+ourselves as being "under memory pressure".
+
+(This is not currently reported.)
+
+