7 files changed, 178 insertions, 10 deletions
diff --git a/doc/HACKING/CodeStructure.md b/doc/HACKING/CodeStructure.md
new file mode 100644
index 0000000000..736d6cd484
--- /dev/null
+++ b/doc/HACKING/CodeStructure.md
@@ -0,0 +1,129 @@
+
+TODO: revise this to talk about how things are, rather than how things
+have changed.
+
+TODO: Make this into good markdown.
+
+
+
+For quite a while now, the program "tor" has been built from source
+code in just two directories: src/common and src/or.
+
+This has become more-or-less untenable, for a few reasons -- most
+notably of which is that it has led our code to become more
+spaghetti-ish than I can endorse with a clean conscience.
+
+So to fix that, we've gone and done a huge code movement in our git
+master branch, which will land in a release once Tor 0.3.5.1-alpha is
+out.
+
+Here's what we did:
+
+  * src/common has been turned into a set of static libraries.  These
+all live in the "src/lib/*" directories.  The dependencies between
+these libraries should have no cycles.  The libraries are:
+
+    arch -- Headers to handle architectural differences
+    cc -- headers to handle differences among compilers
+    compress -- wraps zlib, zstd, lzma
+    container -- high-level container types
+    crypt_ops -- Cryptographic operations. Planning to split this into
+a higher and lower level library
+    ctime -- Operations that need to run in constant-time. (Properly,
+data-invariant time)
+    defs -- miscelaneous definitions needed throughout Tor.
+    encoding -- transforming one data type into another, and various
+data types into strings.
+    err -- lowest-level error handling, in cases where we can't use
+the logs because something that the logging system needs has broken.
+    evloop -- Generic event-loop handling logic
+    fdio -- Low-level IO wrapper functions for file descriptors.
+    fs -- Operations on the filesystem
+    intmath -- low-level integer math and misc bit-twiddling hacks
+    lock -- low-level locking code
+    log -- Tor's logging module.  This library sits roughly halfway up
+the library dependency diagram, since everything it depends on has to
+be carefully crafted to *not* log.
+    malloc -- Low-level wrappers for the platform memory allocation functions.
+    math -- Higher-level mathematical functions, and floating-point math
+    memarea -- An arena allocator
+    meminfo -- Functions for querying the current process's memory
+status and resources
+    net -- Networking compatibility and convenience code
+    osinfo -- Querying information about the operating system
+    process -- Launching and querying the status of other processes
+    sandbox -- Backend for the linux seccomp2 sandbox
+    smartlist_core -- The lowest-level of the smartlist_t data type.
+Separated from the rest of the containers library because the logging
+subsystem depends on it.
+    string -- Compatibility and convenience functions for manipulating
+C strings.
+    term -- Terminal-related functions (currently limited to a getpass
+function).
+    testsupport -- Macros for mocking, unit tests, etc.
+    thread -- Higher-level thread compatibility code
+    time -- Higher-level time management code, including format
+conversions and monotonic time
+    tls -- Our wrapper around our TLS library
+    trace -- Formerly src/trace -- a generic event tracing API
+    wallclock -- Low-level time code, used by the log module.
+
+  * To ensure that the dependency graph in src/common remains under
+control, there is a tool that you can run called "make
+check-includes".  It verifies that each module in Tor only includes
+the headers that it is permitted to include, using a per-directory
+".may_include" file.
+
+  * The src/or/or.h header has been split into numerous smaller
+headers.  Notably, many important structures are now declared in a
+header called foo_st.h, where "foo" is the name of the structure.
+
+  * The src/or directory, which had most of Tor's code, had been split
+up into several directories.  This is still a work in progress:  This
+code has not itself been refactored, and its dependency graph is still
+a tangled web.  I hope we'll be working on that over the coming
+releases, but it will take a while to do.
+
+    The new top-level source directories are:
+
+     src/core -- Code necessary to actually perform or use onion routing.
+     src/feature -- Code used only by some onion routing
+configurations, or only for a special purpose.
+     src/app -- Top-level code to run, invoke, and configure the
+lower-level code
+
+   The new second-level source directories are:
+     src/core/crypto -- High-level cryptographic protocols used in Tor
+     src/core/mainloop -- Tor's event loop, connection-handling, and
+traffic-routing code.
+     src/core/or -- Parts related to handling onion routing itself
+     src/core/proto -- support for encoding and decoding different
+wire protocols
+
+     src/feature/api -- Support for making Tor embeddable
+     src/feature/client -- Functionality which only Tor clients need
+     src/feature/control -- Controller implementation
+     src/feature/dirauth -- Directory authority
+     src/feature/dircache -- Directory cache
+     src/feature/dirclient -- Directory client
+     src/feature/dircommon -- Shared code between the other directory modules
+     src/feature/hibernate -- Hibernating when Tor is out of bandwidth
+or shutting down
+     src/feature/hs -- v3 onion service implementation
+     src/feature/hs_common -- shared code between both onion service
+implementations
+     src/feature/nodelist -- storing and accessing the list of relays on
+the network.
+     src/feature/relay -- code that only relay servers and exit servers need.
+     src/feature/rend -- v2 onion service implementation
+     src/feature/stats -- statistics and history
+
+     src/app/config -- configuration and state for Tor
+     src/app/main -- Top-level functions to invoke the rest or Tor.
+
+  * The "tor" executable is now built in src/app/tor rather than src/or/tor.
+
+  * There are more static libraries than before that you need to build
+into your application if you want to embed Tor.  Rather than
+maintaining this list yourself, I recommend that you run "make
+show-libs" to have Tor emit a list of what you need to link.
diff --git a/doc/HACKING/CodingStandards.md b/doc/HACKING/CodingStandards.md
index b830ecea93..4f229348e4 100644
--- a/doc/HACKING/CodingStandards.md
+++ b/doc/HACKING/CodingStandards.md
@@ -200,8 +200,8 @@ We have some wrapper functions like `tor_malloc`, `tor_free`, `tor_strdup`, and
 always succeed or exit.)
 
 You can get a full list of the compatibility functions that Tor provides by
-looking through `src/common/util*.h` and `src/common/compat*.h`.  You can see the
-available containers in `src/common/containers*.h`.  You should probably
+looking through `src/lib/*/*.h`.  You can see the
+available containers in `src/lib/containers/*.h`.  You should probably
 familiarize yourself with these modules before you write too much code, or
 else you'll wind up reinventing the wheel.
 
@@ -214,6 +214,24 @@ We don't call `memcmp()` directly.  Use `fast_memeq()`, `fast_memneq()`,
 Also see a longer list of functions to avoid in:
 https://people.torproject.org/~nickm/tor-auto/internal/this-not-that.html
 
+What code can use what other code?
+----------------------------------
+
+We're trying to simplify Tor's structure over time.  In the long run, we want
+Tor to be structured as a set of modules with *no circular dependencies*.
+
+This property is currently provided by the modules in src/lib, but not
+throughout the rest of Tor.  In general, higher-level libraries may use
+lower-level libraries, but never the reverse.
+
+To prevent new circular dependencies from landing, we have a tool that
+you can invoke with `make check-includes`, and which is run
+automatically as part of `make check`.  This tool will verify that, for
+every source directory with a `.may_include` file, no local headers are
+included except those specifically permitted by the `.may_include` file.
+When editing one of these files, please make sure that you are not
+introducing any cycles into Tor's dependency graph.
+
 Floating point math is hard
 ---------------------------
 
diff --git a/doc/HACKING/HelpfulTools.md b/doc/HACKING/HelpfulTools.md
index 13d1c4b0d7..d499238526 100644
--- a/doc/HACKING/HelpfulTools.md
+++ b/doc/HACKING/HelpfulTools.md
@@ -4,9 +4,16 @@ Useful tools
 These aren't strictly necessary for hacking on Tor, but they can help track
 down bugs.
 
-Travis CI
----------
-It's CI. Looks like this: https://travis-ci.org/torproject/tor.
+Travis/Appveyor CI
+------------------
+It's CI.
+
+Looks like this:
+* https://travis-ci.org/torproject/tor
+* https://ci.appveyor.com/project/torproject/tor
+
+Travis builds and runs tests on Linux, and eventually macOS (#24629).
+Appveyor builds and runs tests on Windows (using Windows Services for Linux).
 
 Runs automatically on Pull Requests sent to torproject/tor. You can set it up
 for your fork to build commits outside of PRs too:
@@ -16,6 +23,8 @@ for your fork to build commits outside of PRs too:
    https://help.github.com/articles/fork-a-repo/
 3. follow https://docs.travis-ci.com/user/getting-started/#To-get-started-with-Travis-CI.
    skip steps involving `.travis.yml` (we already have one).
+4. go to https://ci.appveyor.com/login , log in with github, and select
+   "NEW PROJECT"
 
 Builds should show up on the web at travis-ci.com and on IRC at #tor-ci on
 OFTC. If they don't, ask #tor-dev (also on OFTC).
@@ -23,7 +32,16 @@ OFTC. If they don't, ask #tor-dev (also on OFTC).
 Jenkins
 -------
 
-    https://jenkins.torproject.org
+It's CI/builders. Looks like this: https://jenkins.torproject.org
+
+Runs automatically on commits merged to git.torproject.org. We CI the
+master branch and all supported tor versions. We also build nightly debian
+packages from master.
+
+Builds Linux and Windows cross-compilation. Runs Linux tests.
+
+Builds should show up on the web at jenkins.torproject.org and on IRC at
+#tor-bots on OFTC. If they don't, ask #tor-dev (also on OFTC).
 
 Valgrind
 --------
diff --git a/doc/HACKING/Module.md b/doc/HACKING/Module.md
index 6684e258df..9cf36090b4 100644
--- a/doc/HACKING/Module.md
+++ b/doc/HACKING/Module.md
@@ -96,8 +96,8 @@ There are couples of "rules" you want to follow:
   filename as the one in the module. For example, this is a bad idea and
   should never be done:
 
-    - `src/or/shared_random.c`
-    - `src/or/dirauth/shared_random.c`
+    - `src/feature/dirclient/shared_random.c`
+    - `src/feature/dirauth/shared_random.c`
 
 * When you include headers from the module, **always** use the full module
   path in your statement. Example:
diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md
index e70416c354..55a40fc89b 100644
--- a/doc/HACKING/ReleasingTor.md
+++ b/doc/HACKING/ReleasingTor.md
@@ -7,7 +7,7 @@ new Tor release:
 
 === 0. Preliminaries
 
-1. Get at least three of weasel/arma/Sebastian/Sina to put the new
+1. Get at least two of weasel/arma/Sebastian to put the new
    version number in their approved versions list.  Give them a few
    days to do this if you can.
 
diff --git a/doc/include.am b/doc/include.am
index 3e3b8e612c..0a123aae11 100644
--- a/doc/include.am
+++ b/doc/include.am
@@ -36,6 +36,7 @@ EXTRA_DIST+= doc/asciidoc-helper.sh			\
 	     doc/HACKING/README.1st.md				\
 	     doc/HACKING/CodingStandards.md 			\
 	     doc/HACKING/CodingStandardsRust.md			\
+	     doc/HACKING/CodeStructure.md			\
 	     doc/HACKING/Fuzzing.md				\
 	     doc/HACKING/GettingStarted.md 			\
 	     doc/HACKING/GettingStartedRust.md 			\
diff --git a/doc/tor.1.txt b/doc/tor.1.txt
index f42ad0dd3c..1db8cabf86 100644
--- a/doc/tor.1.txt
+++ b/doc/tor.1.txt
@@ -2747,7 +2747,9 @@ on the public Tor network.
 [[V3BandwidthsFile]] **V3BandwidthsFile** __FILENAME__::
     V3 authoritative directories only. Configures the location of the
     bandwidth-authority generated file storing information on relays' measured
-    bandwidth capacities. (Default: unset)
+    bandwidth capacities. To avoid inconsistent reads, bandwidth data should
+    be written to temporary file, then renamed to the configured filename.
+    (Default: unset)
 
 [[V3AuthUseLegacyKey]] **V3AuthUseLegacyKey** **0**|**1**::
     If set, the directory authority will sign consensuses not only with its