diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/HACKING | 85 | ||||
| -rw-r--r-- | doc/WritingTests.txt | 273 | ||||
| -rw-r--r-- | doc/include.am | 15 | ||||
| -rw-r--r-- | doc/tor-fw-helper.1.txt | 60 | ||||
| -rw-r--r-- | doc/tor.1.txt | 248 | ||||
| -rw-r--r-- | doc/torrc_format.txt | 205 |
6 files changed, 754 insertions, 132 deletions
diff --git a/doc/HACKING b/doc/HACKING index 5c71b74bd1..e92d675a43 100644 --- a/doc/HACKING +++ b/doc/HACKING @@ -61,9 +61,10 @@ it's a bugfix, mention what bug it fixes and when the bug was introduced. To find out which Git tag the change was introduced in, you can use "git describe --contains <sha1 of commit>". -If at all possible, try to create this file in the same commit where -you are making the change. Please give it a distinctive name that no -other branch will use for the lifetime of your change. +If at all possible, try to create this file in the same commit where you are +making the change. Please give it a distinctive name that no other branch will +use for the lifetime of your change. To verify the format of the changes file, +you can use "make check-changes". When we go to make a release, we will concatenate all the entries in changes to make a draft changelog, and clear the directory. We'll @@ -114,6 +115,32 @@ valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor pass --undef-value-errors=no to valgrind, or rebuild your openssl with -DPURIFY.) +Coverity +~~~~~~~~ + +Nick regularly runs the coverity static analyzer on the Tor codebase. + +The preprocessor define __COVERITY__ is used to work around instances +where coverity picks up behavior that we wish to permit. + +clang Static Analyzer +~~~~~~~~~~~~~~~~~~~~~ + +The clang static analyzer can be run on the Tor codebase using Xcode (WIP) +or a command-line build. + +The preprocessor define __clang_analyzer__ is used to work around instances +where clang picks up behavior that we wish to permit. + +clang Runtime Sanitizers +~~~~~~~~~~~~~~~~ + +To build the Tor codebase with the clang Address and Undefined Behavior +sanitizers, see the file contrib/clang/sanitize_blacklist.txt. + +Preprocessor workarounds for instances where clang picks up behavior that +we wish to permit are also documented in the blacklist file. + Running lcov for unit test coverage ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -137,12 +164,12 @@ investigated (as of July 2014). Running the unit tests ~~~~~~~~~~~~~~~~~~~~~~ -To quickly run all tests: +To quickly run all the tests distributed with Tor: ----- make check ----- -To run unit tests only: +To run the fast unit tests only: ----- make test ----- @@ -155,6 +182,17 @@ arbitrarily): ./src/test/test :<name_of_excluded_test> [:<name_of_excluded_test2]... ----- +To run all tests, including those based on Stem or Chutney: +----- + make test-full +----- + +To run all tests, including those basedd on Stem or Chutney that require a +working connection to the internet: +----- + make test-full-online +----- + Running gcov for unit test coverage ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -162,6 +200,7 @@ Running gcov for unit test coverage ./configure --enable-coverage make make check + # or--- make test-full ? make test-full-online? mkdir coverage-output ./scripts/test/coverage coverage-output ----- @@ -197,6 +236,9 @@ We have the beginnings of a set of scripts to run integration tests using Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and run "make test-network". +We also have scripts to run integration tests using Stem. To try them, set +STEM_SOURCE_DIR to your Stem source directory, and run "test-stem". + Profiling Tor with oprofile ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -223,6 +265,23 @@ Here are some basic instructions * "opreport -l that_dir/*" - Profit +Generating and analyzing a callgraph +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +1. Run ./scripts/maint/generate_callgraph.sh . This will generate a + bunch of files in a new ./callgraph directory. + +2. Run ./scripts/maint/analyze_callgraph.py callgraph/src/*/* . This + will do a lot of graph operations and then dump out a new + "callgraph.pkl" file, containing data in Python's "pickle" format. + +3. Run ./scripts/maint/display_callgraph.py . It will display: + - the number of functions reachable from each function. + - all strongly-connnected components in the Tor callgraph + - the largest bottlenecks in the largest SCC in the Tor callgraph. + +Note that currently the callgraph generator can't detect calls that pass +through function pointers. Coding conventions ------------------ @@ -373,7 +432,7 @@ do your own profiling to determine otherwise. Log conventions ~~~~~~~~~~~~~~~ -https://trac.torproject.org/projects/tor/wiki/doc/TorFAQ#loglevel +https://www.torproject.org/docs/faq#LogLevel No error or warning messages should be expected during normal OR or OP operation. @@ -541,7 +600,13 @@ a stable release, add it to the ReleaseNotes file too. If we're adding to a release-0.2.x branch, manually commit the changelogs to the later git branches too. -4) Bump the version number in configure.ac and rebuild. +4) In maint-0.2.x, bump the version number in configure.ac and run + scripts/maint/updateVersions.pl to update version numbers in other + places, and commit. Then merge maint-0.2.x into release-0.2.x. + + (NOTE: TO bump the version number, edit configure.ac, and then run + either make, or 'perl scripts/maint/updateVersions.pl', depending on + your version.) 5) Make dist, put the tarball up somewhere, and tell #tor about it. Wait a while to see if anybody has problems building it. Try to get Sebastian @@ -563,6 +628,12 @@ on dist-master. 8b) Edit "include/versions.wmi" and "Makefile" to note the new version. 9) Email the packagers (cc'ing tor-assistants) that a new tarball is up. + The current list of packagers is: + {weasel,gk,mikeperry} at torproject dot org + {blueness} at gentoo dot org + {paul} at invizbox dot io + {ondrej.mikle} at gmail dot com + {lfleischer} at archlinux dot org 10) Add the version number to Trac. To do this, go to Trac, log in, select "Admin" near the top of the screen, then select "Versions" from diff --git a/doc/WritingTests.txt b/doc/WritingTests.txt new file mode 100644 index 0000000000..62a17e3709 --- /dev/null +++ b/doc/WritingTests.txt @@ -0,0 +1,273 @@ + +Writing tests for Tor: an incomplete guide +========================================== + +Tor uses a variety of testing frameworks and methodologies to try to +keep from introducing bugs. The major ones are: + + 1. Unit tests written in C and shipped with the Tor distribution. + + 2. Integration tests written in Python and shipped with the Tor + distribution. + + 3. Integration tests written in Python and shipped with the Stem + library. Some of these use the Tor controller protocol. + + 4. System tests written in Python and SH, and shipped with the + Chutney package. These work by running many instances of Tor + locally, and sending traffic through them. + + 5. The Shadow network simulator. + +How to run these tests +---------------------- + +=== The easy version + +To run all the tests that come bundled with Tor, run "make check" + +To run the Stem tests as well, fetch stem from the git repository, +set STEM_SOURCE_DIR to the checkout, and run "make test-stem". + +To run the Chutney tests as well, fetch chutney from the git repository, +set CHUTNEY_PATH to the checkout, and run "make test-network". + +To run all of the above, run "make test-full". + +To run all of the above, plus tests that require a working connection to the +internet, run "make test-full-online". + +=== Running particular subtests + +The Tor unit tests are divided into separate programs and a couple of +bundled unit test programs. + +Separate programs are easy. For example, to run the memwipe tests in +isolation, you just run ./src/test/test-memwipe . + +To run tests within the unit test programs, you can specify the name +of the test. The string ".." can be used as a wildcard at the end of the +test name. For example, to run all the cell format tests, enter +"./src/test/test cellfmt/..". To run + +Many tests that need to mess with global state run in forked subprocesses in +order to keep from contaminating one another. But when debugging a failing test, +you might want to run it without forking a subprocess. To do so, use the +"--no-fork" option with a single test. (If you specify it along with +multiple tests, they might interfere.) + +You can turn on logging in the unit tests by passing one of "--debug", +"--info", "--notice", or "--warn". By default only errors are displayed. + +Unit tests are divided into "./src/test/test" and "./src/test/test-slow". +The former are those that should finish in a few seconds; the latter tend to +take more time, and may include CPU-intensive operations, deliberate delays, +and stuff like that. + +=== Finding test coverage + +When you configure Tor with the --enable-coverage option, it should +build with support for coverage in the unit tests, and in a special +"tor-cov" binary. + +Then, run the tests you'd like to see coverage from. If you have old +coverage output, you may need to run "reset-gcov" first. + +Now you've got a bunch of files scattered around your build directories +called "*.gcda". In order to extract the coverage output from them, make a +temporary directory for them and run "./scripts/test/coverage ${TMPDIR}", +where ${TMPDIR} is the temporary directory you made. This will create a +".gcov" file for each source file under tests, containing that file's source +annotated with the number of times the tests hit each line. (You'll need to +have gcov installed.) + +You can get a summary of the test coverage for each file by running +"./scripts/test/cov-display ${TMPDIR}/*" . Each line lists the file's name, +the number of uncovered lines, the number of uncovered lines, and the +coverage percentage. + +For a summary of the test coverage for each _function_, run +"./scripts/test/cov-display -f ${TMPDIR}/*" . + +=== Comparing test coverage + +Sometimes it's useful to compare test coverage for a branch you're writing to +coverage from another branch (such as git master, for example). But you +can't run "diff" on the two coverage outputs directly, since the actual +number of times each line is executed aren't so important, and aren't wholly +deterministic. + +Instead, follow the instructions above for each branch, creating a separate +temporary directory for each. Then, run "./scripts/test/cov-diff ${D1} +${D2}", where D1 and D2 are the directories you want to compare. This will +produce a diff of the two directories, with all lines normalized to be either +covered or uncovered. + +To count new or modified uncovered lines in D2, you can run: + + "./scripts/test/cov-diff ${D1} ${D2}" | grep '^+ *\#' |wc -l + + +What kinds of test should I write? +---------------------------------- + +Integration testing and unit testing are complementary: it's probably a +good idea to make sure that your code is hit by both if you can. + +If your code is very-low level, and its behavior is easily described in +terms of a relation between inputs and outputs, or a set of state +transitions, then it's a natural fit for unit tests. (If not, please +consider refactoring it until most of it _is_ a good fit for unit +tests!) + +If your code adds new externally visible functionality to Tor, it would +be great to have a test for that functionality. That's where +integration tests more usually come in. + +Unit and regression tests: Does this function do what it's supposed to? +----------------------------------------------------------------------- + +Most of Tor's unit tests are made using the "tinytest" testing framework. +You can see a guide to using it in the tinytest manual at + + https://github.com/nmathewson/tinytest/blob/master/tinytest-manual.md + +To add a new test of this kind, either edit an existing C file in src/test/, +or create a new C file there. Each test is a single function that must +be indexed in the table at the end of the file. We use the label "done:" as +a cleanup point for all test functions. + +(Make sure you read tinytest-manual.md before proceeding.) + +I use the term "unit test" and "regression tests" very sloppily here. + +=== A simple example + +Here's an example of a test function for a simple function in util.c: + + static void + test_util_writepid(void *arg) + { + (void) arg; + + char *contents = NULL; + const char *fname = get_fname("tmp_pid"); + unsigned long pid; + char c; + + write_pidfile(fname); + + contents = read_file_to_str(fname, 0, NULL); + tt_assert(contents); + + int n = sscanf(contents, "%lu\n%c", &pid, &c); + tt_int_op(n, OP_EQ, 1); + tt_int_op(pid, OP_EQ, getpid()); + + done: + tor_free(contents); + } + +This should look pretty familiar to you if you've read the tinytest +manual. One thing to note here is that we use the testing-specific +function "get_fname" to generate a file with respect to a temporary +directory that the tests use. You don't need to delete the file; +it will get removed when the tests are done. + +Also note our use of OP_EQ instead of == in the tt_int_op() calls. +We define OP_* macros to use instead of the binary comparison +operators so that analysis tools can more easily parse our code. +(Coccinelle really hates to see == used as a macro argument.) + +Finally, remember that by convention, all *_free() functions that +Tor defines are defined to accept NULL harmlessly. Thus, you don't +need to say "if (contents)" in the cleanup block. + +=== Exposing static functions for testing + +Sometimes you need to test a function, but you don't want to expose +it outside its usual module. + +To support this, Tor's build system compiles a testing version of +each module, with extra identifiers exposed. If you want to +declare a function as static but available for testing, use the +macro "STATIC" instead of "static". Then, make sure there's a +macro-protected declaration of the function in the module's header. + +For example, crypto_curve25519.h contains: + +#ifdef CRYPTO_CURVE25519_PRIVATE +STATIC int curve25519_impl(uint8_t *output, const uint8_t *secret, + const uint8_t *basepoint); +#endif + +The crypto_curve25519.c file and the test_crypto.c file both define +CRYPTO_CURVE25519_PRIVATE, so they can see this declaration. + +=== Mock functions for testing in isolation + +Often we want to test that a function works right, but the function to +be tested depends on other functions whose behavior is hard to observe, +or which require a working Tor network, or something like that. + +To write tests for this case, you can replace the underlying functions +with testing stubs while your unit test is running. You need to declare +the underlying function as 'mockable', as follows: + + MOCK_DECL(returntype, functionname, (argument list)); + +and then later implement it as: + + MOCK_IMPL(returntype, functionname, (argument list)) + { + /* implementation here */ + } + +For example, if you had a 'connect to remote server' function, you could +declare it as: + + + MOCK_DECL(int, connect_to_remote, (const char *name, status_t *status)); + +When you declare a function this way, it will be declared as normal in +regular builds, but when the module is built for testing, it is declared +as a function pointer initialized to the actual implementation. + +In your tests, if you want to override the function with a temporary +replacement, you say: + + MOCK(functionname, replacement_function_name); + +And later, you can restore the original function with: + + UNMOCK(functionname); + +For more information, see the definitions of this mocking logic in +testsupport.h. + + +=== Advanced techniques: Namespaces + +XXXX write this. danah boyd made us some really awesome stuff here. + + +Integration tests: Calling Tor from the outside +----------------------------------------------- + +XXXX WRITEME + +Writing integration tests with Stem +----------------------------------- + +XXXX WRITEME + +System testing with Chutney +--------------------------- + +XXXX WRITEME + +Who knows what evil lurks in the timings of networks? The Shadow knows! +----------------------------------------------------------------------- + +XXXX WRITEME + diff --git a/doc/include.am b/doc/include.am index 30d3e20d83..41d3d2a0c7 100644 --- a/doc/include.am +++ b/doc/include.am @@ -13,7 +13,7 @@ # just use the .1 and .html files. base_mans = doc/tor doc/tor-gencert doc/tor-resolve doc/torify -all_mans = $(base_mans) doc/tor-fw-helper +all_mans = $(base_mans) if USE_FW_HELPER install_mans = $(all_mans) else @@ -36,7 +36,10 @@ endif EXTRA_DIST+= doc/HACKING doc/asciidoc-helper.sh \ $(html_in) $(man_in) $(txt_in) \ - doc/state-contents.txt + doc/state-contents.txt \ + doc/torrc_format.txt \ + doc/TUNING \ + doc/WritingTests.txt docdir = @docdir@ @@ -56,34 +59,30 @@ doc/tor.1.in: doc/tor.1.txt doc/torify.1.in: doc/torify.1.txt doc/tor-gencert.1.in: doc/tor-gencert.1.txt doc/tor-resolve.1.in: doc/tor-resolve.1.txt -doc/tor-fw-helper.1.in: doc/tor-fw-helper.1.txt doc/tor.html.in: doc/tor.1.txt doc/torify.html.in: doc/torify.1.txt doc/tor-gencert.html.in: doc/tor-gencert.1.txt doc/tor-resolve.html.in: doc/tor-resolve.1.txt -doc/tor-fw-helper.html.in: doc/tor-fw-helper.1.txt -# use ../config.status to swap all machine-specific magic strings +# use config.status to swap all machine-specific magic strings # in the asciidoc with their replacements. $(asciidoc_product) : $(AM_V_GEN)$(MKDIR_P) $(@D) $(AM_V_at)if test -e $(top_srcdir)/$@.in && ! test -e $@.in ; then \ cp $(top_srcdir)/$@.in $@; \ fi - $(AM_V_at)./config.status -q --file=$@; + $(AM_V_at)$(top_builddir)/config.status -q --file=$@; doc/tor.html: doc/tor.html.in doc/tor-gencert.html: doc/tor-gencert.html.in doc/tor-resolve.html: doc/tor-resolve.html.in doc/torify.html: doc/torify.html.in -doc/tor-fw-helper.html: doc/tor-fw-helper.html.in doc/tor.1: doc/tor.1.in doc/tor-gencert.1: doc/tor-gencert.1.in doc/tor-resolve.1: doc/tor-resolve.1.in doc/torify.1: doc/torify.1.in -doc/tor-fw-helper.1: doc/tor-fw-helper.1.in CLEANFILES+= $(asciidoc_product) config.log DISTCLEANFILES+= $(html_in) $(man_in) diff --git a/doc/tor-fw-helper.1.txt b/doc/tor-fw-helper.1.txt deleted file mode 100644 index 1c103d9250..0000000000 --- a/doc/tor-fw-helper.1.txt +++ /dev/null @@ -1,60 +0,0 @@ -// Copyright (c) The Tor Project, Inc. -// See LICENSE for licensing information -// This is an asciidoc file used to generate the manpage/html reference. -// Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html -:man source: Tor -:man manual: Tor Manual -tor-fw-helper(1) -================ -Jacob Appelbaum - -NAME ----- -tor-fw-helper - Manage upstream firewall/NAT devices - -SYNOPSIS --------- -**tor-fw-helper** [-h|--help] [-T|--test-commandline] [-v|--verbose] [-g|--fetch-public-ip] - [-p __external port__:__internal_port__] - -DESCRIPTION ------------ -**tor-fw-helper** currently supports Apple's NAT-PMP protocol and the UPnP -standard for TCP port mapping. It is written as the reference implementation of -tor-fw-helper-spec.txt and conforms to that loose plugin API. If your network -supports either NAT-PMP or UPnP, tor-fw-helper will attempt to automatically -map the required TCP ports for Tor's Or and Dir ports. + - -OPTIONS -------- -**-h** or **--help**:: - Display help text and exit. - -**-v** or **--verbose**:: - Display verbose output. - -**-T** or **--test-commandline**:: - Display test information and print the test information in - tor-fw-helper.log - -**-g** or **--fetch-public-ip**:: - Fetch the the public ip address for each supported NAT helper method. - -**-p** or **--port** __external_port__:__internal_port__:: - Forward external_port to internal_port. This option can appear - more than once. - -BUGS ----- -This probably doesn't run on Windows. That's not a big issue, since we don't -really want to deal with Windows before October 2010 anyway. - -SEE ALSO --------- -**tor**(1) + - -See also the "tor-fw-helper-spec.txt" file, distributed with Tor. - -AUTHORS -------- - Jacob Appelbaum <jacob@torproject.org>, Steven J. Murdoch <Steven.Murdoch@cl.cam.ac.uk> diff --git a/doc/tor.1.txt b/doc/tor.1.txt index e136bd0f7e..5ea5623952 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -274,7 +274,7 @@ GENERAL OPTIONS all sockets will be set to this limit. Must be a value between 2048 and 262144, in 1024 byte increments. Default of 8192 is recommended. -[[ControlPort]] **ControlPort** __PORT__|**unix:**__path__|**auto**:: +[[ControlPort]] **ControlPort** __PORT__|**unix:**__path__|**auto** [__flags__]:: If set, Tor will accept connections on this port and allow those connections to control the Tor process using the Tor Control Protocol (described in control-spec.txt). Note: unless you also specify one or @@ -284,6 +284,14 @@ GENERAL OPTIONS method is sufficient to authenticate to Tor.) This option is required for many Tor controllers; most use the value of 9051. Set it to "auto" to have Tor pick a port for you. (Default: 0) + + + Recognized flags are:: + **GroupWritable**;; + Unix domain sockets only: makes the socket get created as + group-writable. + **WorldWritable**;; + Unix domain sockets only: makes the socket get created as + world-writable. [[ControlListenAddress]] **ControlListenAddress** __IP__[:__PORT__]:: Bind the controller listener to this address. If you specify a port, bind @@ -371,12 +379,6 @@ GENERAL OPTIONS chosen with their regular weights, multiplied by this number, which should be 1.0 or less. (Default: 1.0) -[[DynamicDHGroups]] **DynamicDHGroups** **0**|**1**:: - If this option is set to 1, when running as a server, generate our - own Diffie-Hellman group instead of using the one from Apache's mod_ssl. - This option may help circumvent censorship based on static - Diffie-Hellman parameters. (Default: 0) - [[AlternateDirAuthority]] **AlternateDirAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__ + [[AlternateBridgeAuthority]] **AlternateBridgeAuthority** [__nickname__] [**flags**] __address__:__port__ __ fingerprint__:: @@ -932,7 +934,9 @@ The following options are useful only for clients (that is, if Feel free to reuse a circuit that was first used at most NUM seconds ago, but never attach a new stream to a circuit that is too old. For hidden services, this applies to the __last__ time a circuit was used, not the - first. (Default: 10 minutes) + first. Circuits with streams constructed with SOCKS authentication via + SocksPorts that have **KeepAliveIsolateSOCKSAuth** ignore this value. + (Default: 10 minutes) [[MaxClientCircuitsPending]] **MaxClientCircuitsPending** __NUM__:: Do not allow more than NUM circuits to be pending at a time for handling @@ -961,6 +965,13 @@ The following options are useful only for clients (that is, if you. This directive can be specified multiple times to bind to multiple addresses/ports. (Default: 9050) + + + NOTE: Although this option allows you to specify an IP address + other than localhost, you should do so only with extreme caution. + The SOCKS protocol is unencrypted and (as we use it) + unauthenticated, so exposing it in this way could leak your + information to anybody watching your network, and allow anybody + to use your computer as an open proxy. + + + The _isolation flags_ arguments give Tor rules for which streams received on this SOCKSPort are allowed to share circuits with one another. Recognized isolation flags are: @@ -982,6 +993,9 @@ The following options are useful only for clients (that is, if **IsolateDestAddr**;; Don't share circuits with streams targeting a different destination address. + **KeepAliveIsolateSOCKSAuth**;; + If **IsolateSOCKSAuth** is enabled, keep alive circuits that have + streams with SOCKS authentication set indefinitely. **SessionGroup=**__INT__;; If no other isolation rules would prevent it, allow streams on this port to share circuits with streams from every other @@ -1001,20 +1015,18 @@ The following options are useful only for clients (that is, if **PreferIPv6**;; Tells exits that, if a host has both an IPv4 and an IPv6 address, we would prefer to connect to it via IPv6. (IPv4 is the default.) + - + - NOTE: Although this option allows you to specify an IP address - other than localhost, you should do so only with extreme caution. - The SOCKS protocol is unencrypted and (as we use it) - unauthenticated, so exposing it in this way could leak your - information to anybody watching your network, and allow anybody - to use your computer as an open proxy. + - + **CacheIPv4DNS**;; Tells the client to remember IPv4 DNS answers we receive from exit nodes via this connection. (On by default.) **CacheIPv6DNS**;; Tells the client to remember IPv6 DNS answers we receive from exit nodes via this connection. + **GroupWritable**;; + Unix domain sockets only: makes the socket get created as + group-writable. + **WorldWritable**;; + Unix domain sockets only: makes the socket get created as + world-writable. **CacheDNS**;; Tells the client to remember all DNS answers we receive from exit nodes via this connection. @@ -1311,7 +1323,7 @@ The following options are useful only for clients (that is, if [[DownloadExtraInfo]] **DownloadExtraInfo** **0**|**1**:: If true, Tor downloads and caches "extra-info" documents. These documents contain information about servers other than the information in their - regular router descriptors. Tor does not use this information for anything + regular server descriptors. Tor does not use this information for anything itself; to save bandwidth, leave this option turned off. (Default: 0) [[WarnPlaintextPorts]] **WarnPlaintextPorts** __port__,__port__,__...__:: @@ -1490,8 +1502,8 @@ is non-zero): [[BridgeRelay]] **BridgeRelay** **0**|**1**:: Sets the relay to act as a "bridge" with respect to relaying connections from bridge users to the Tor network. It mainly causes Tor to publish a - server descriptor to the bridge database, rather than publishing a relay - descriptor to the public directory authorities. + server descriptor to the bridge database, rather than + to the public directory authorities. [[ContactInfo]] **ContactInfo** __email_address__:: Administrative contact information for this relay or bridge. This line @@ -1517,22 +1529,41 @@ is non-zero): [[ExitPolicy]] **ExitPolicy** __policy__,__policy__,__...__:: Set an exit policy for this server. Each policy is of the form - "**accept**|**reject** __ADDR__[/__MASK__][:__PORT__]". If /__MASK__ is + "**accept[6]**|**reject[6]** __ADDR__[/__MASK__][:__PORT__]". If /__MASK__ is omitted then this policy just applies to the host given. Instead of giving - a host or network you can also use "\*" to denote the universe (0.0.0.0/0). + a host or network you can also use "\*" to denote the universe (0.0.0.0/0 + and ::/128), or \*4 to denote all IPv4 addresses, and \*6 to denote all + IPv6 addresses. __PORT__ can be a single port number, an interval of ports "__FROM_PORT__-__TO_PORT__", or "\*". If __PORT__ is omitted, that means "\*". + + For example, "accept 18.7.22.69:\*,reject 18.0.0.0/8:\*,accept \*:\*" would - reject any traffic destined for MIT except for web.mit.edu, and accept - anything else. + + reject any IPv4 traffic destined for MIT except for web.mit.edu, and accept + any other IPv4 or IPv6 traffic. + + + + Tor also allows IPv6 exit policy entries. For instance, "reject6 [FC00::]/7:\*" + rejects all destinations that share 7 most significant bit prefix with + address FC00::. Respectively, "accept6 [C000::]/3:\*" accepts all destinations + that share 3 most significant bit prefix with address C000::. + + + + accept6 and reject6 only produce IPv6 exit policy entries. Using an IPv4 + address with accept6 or reject6 is ignored and generates a warning. + accept/reject allows either IPv4 or IPv6 addresses. Use \*4 as an IPv4 + wildcard address, and \*6 as an IPv6 wildcard address. accept/reject * + expands to matching IPv4 and IPv6 wildcard address rules. + + + + To specify all IPv4 and IPv6 internal and link-local networks (including + 0.0.0.0/8, 169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, + 172.16.0.0/12, [::]/8, [FC00::]/7, [FE80::]/10, [FEC0::]/10, [FF00::]/8, + and [::]/127), you can use the "private" alias instead of an address. + ("private" always produces rules for IPv4 and IPv6 addresses, even when + used with accept6/reject6.) + + - To specify all internal and link-local networks (including 0.0.0.0/8, - 169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, and - 172.16.0.0/12), you can use the "private" alias instead of an address. - These addresses are rejected by default (at the beginning of your exit - policy), along with your public IP address, unless you set the + Private addresses are rejected by default (at the beginning of your exit + policy), along with the configured primary public IPv4 and IPv6 addresses, + and any public IPv4 and IPv6 addresses on any interface on the relay. + These private addresses are rejected unless you set the ExitPolicyRejectPrivate config option to 0. For example, once you've done that, you could allow HTTP to 127.0.0.1 and block all other connections to internal networks with "accept 127.0.0.1:80,reject private:\*", though that @@ -1540,18 +1571,17 @@ is non-zero): public (external) IP address. See RFC 1918 and RFC 3330 for more details about internal and reserved IP address space. + + - Tor also allow IPv6 exit policy entries. For instance, "reject6 [FC00::]/7:*" - rejects all destinations that share 7 most significant bit prefix with - address FC00::. Respectively, "accept6 [C000::]/3:*" accepts all destinations - that share 3 most significant bit prefix with address C000::. + - + This directive can be specified multiple times so you don't have to put it all on one line. + + Policies are considered first to last, and the first match wins. If you - want to \_replace_ the default exit policy, end your exit policy with - either a reject \*:* or an accept \*:*. Otherwise, you're \_augmenting_ - (prepending to) the default exit policy. The default exit policy is: + + want to allow the same ports on IPv4 and IPv6, write your rules using + accept/reject \*. If you want to allow different ports on IPv4 and IPv6, + write your IPv6 rules using accept6/reject6 \*6, and your IPv4 rules using + accept/reject \*4. If you want to \_replace_ the default exit policy, end + your exit policy with either a reject \*:* or an accept \*:*. Otherwise, + you're \_augmenting_ (prepending to) the default exit policy. The default + exit policy is: + reject *:25 reject *:119 @@ -1565,9 +1595,15 @@ is non-zero): reject *:6881-6999 accept *:* + Since the default exit policy uses accept/reject *, it applies to both + IPv4 and IPv6 addresses. + [[ExitPolicyRejectPrivate]] **ExitPolicyRejectPrivate** **0**|**1**:: - Reject all private (local) networks, along with your own public IP address, - at the beginning of your exit policy. See above entry on ExitPolicy. + Reject all private (local) networks, along with your own configured public + IPv4 and IPv6 addresses, at the beginning of your exit policy. Also reject + any public IPv4 and IPv6 addresses on any interface on the relay. (If + IPv6Exit is not set, all IPv6 addresses will be rejected anyway.) + See above entry on ExitPolicy. (Default: 1) [[IPv6Exit]] **IPv6Exit** **0**|**1**:: @@ -1790,32 +1826,58 @@ is non-zero): (Default: P256) [[CellStatistics]] **CellStatistics** **0**|**1**:: - When this option is enabled, Tor writes statistics on the mean time that - cells spend in circuit queues to disk every 24 hours. (Default: 0) + Relays only. + When this option is enabled, Tor collects statistics about cell + processing (i.e. mean time a cell is spending in a queue, mean + number of cells in a queue and mean number of processed cells per + circuit) and writes them into disk every 24 hours. Onion router + operators may use the statistics for performance monitoring. + If ExtraInfoStatistics is enabled, it will published as part of + extra-info document. (Default: 0) [[DirReqStatistics]] **DirReqStatistics** **0**|**1**:: + Relays and bridges only. When this option is enabled, a Tor directory writes statistics on the number and response time of network status requests to disk every 24 - hours. (Default: 1) + hours. Enables relay and bridge operators to monitor how much their + server is being used by clients to learn about Tor network. + If ExtraInfoStatistics is enabled, it will published as part of + extra-info document. (Default: 1) [[EntryStatistics]] **EntryStatistics** **0**|**1**:: + Relays only. When this option is enabled, Tor writes statistics on the number of - directly connecting clients to disk every 24 hours. (Default: 0) + directly connecting clients to disk every 24 hours. Enables relay + operators to monitor how much inbound traffic that originates from + Tor clients passes through their server to go further down the + Tor network. If ExtraInfoStatistics is enabled, it will be published + as part of extra-info document. (Default: 0) [[ExitPortStatistics]] **ExitPortStatistics** **0**|**1**:: - When this option is enabled, Tor writes statistics on the number of relayed - bytes and opened stream per exit port to disk every 24 hours. (Default: 0) + Exit relays only. + When this option is enabled, Tor writes statistics on the number of + relayed bytes and opened stream per exit port to disk every 24 hours. + Enables exit relay operators to measure and monitor amounts of traffic + that leaves Tor network through their exit node. If ExtraInfoStatistics + is enabled, it will be published as part of extra-info document. + (Default: 0) [[ConnDirectionStatistics]] **ConnDirectionStatistics** **0**|**1**:: - When this option is enabled, Tor writes statistics on the bidirectional use - of connections to disk every 24 hours. (Default: 0) + Relays only. + When this option is enabled, Tor writes statistics on the amounts of + traffic it passes between itself and other relays to disk every 24 + hours. Enables relay operators to monitor how much their relay is + being used as middle node in the circuit. If ExtraInfoStatistics is + enabled, it will be published as part of extra-info document. + (Default: 0) [[HiddenServiceStatistics]] **HiddenServiceStatistics** **0**|**1**:: + Relays only. When this option is enabled, a Tor relay writes obfuscated statistics on its role as hidden-service directory, introduction point, or rendezvous point to disk every 24 hours. If ExtraInfoStatistics is also enabled, these statistics are further - published to the directory authorities. (Default: 0) + published to the directory authorities. (Default: 1) [[ExtraInfoStatistics]] **ExtraInfoStatistics** **0**|**1**:: When this option is enabled, Tor includes previously gathered statistics in @@ -1837,6 +1899,18 @@ is non-zero): this. If this option is set to 0, Tor will try to pick a reasonable default based on your system's physical memory. (Default: 0) +[[SigningKeyLifetime]] **SigningKeyLifetime** __N__ **days**|**weeks**|**months**:: + For how long should each Ed25519 signing key be valid? Tor uses a + permanent master identity key that can be kept offline, and periodically + generates new "signing" keys that it uses online. This option + configures their lifetime. + (Default: 30 days) + +[[OfflineMasterKey]] **OfflineMasterKey** **0**|**1**:: + If non-zero, the Tor relay will never generate or load its master secret + key. Instead, you'll have to use "tor --keygen" to manage the master + secret key. (Default: 0) + DIRECTORY SERVER OPTIONS ------------------------ @@ -1929,7 +2003,7 @@ on the public Tor network. [[BridgeAuthoritativeDir]] **BridgeAuthoritativeDir** **0**|**1**:: When this option is set in addition to **AuthoritativeDirectory**, Tor - accepts and serves router descriptors, but it caches and serves the main + accepts and serves server descriptors, but it caches and serves the main networkstatus documents rather than generating its own. (Default: 0) [[MinUptimeHidServDirectoryV2]] **MinUptimeHidServDirectoryV2** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: @@ -1948,9 +2022,9 @@ on the public Tor network. in the "params" line of its networkstatus vote. [[DirAllowPrivateAddresses]] **DirAllowPrivateAddresses** **0**|**1**:: - If set to 1, Tor will accept router descriptors with arbitrary "Address" + If set to 1, Tor will accept server descriptors with arbitrary "Address" elements. Otherwise, if the address is not an IP address or is a private IP - address, it will reject the router descriptor. (Default: 0) + address, it will reject the server descriptor. (Default: 0) [[AuthDirBadExit]] **AuthDirBadExit** __AddressPattern...__:: Authoritative directories only. A set of address patterns for servers that @@ -2007,6 +2081,13 @@ on the public Tor network. or more is always sufficient to satisfy the bandwidth requirement for the Guard flag. (Default: 250 KBytes) +[[AuthDirPinKeys]] **AuthDirPinKeys** **0**|**1**:: + Authoritative directories only. If non-zero, do not allow any relay to + publish a descriptor if any other relay has reserved its <Ed25519,RSA> + identity keypair. In all cases, Tor records every keypair it accepts + in a journal if it is new, or if it differs from the most recently + accepted pinning for one of the keys it contains. (Default: 0) + [[BridgePassword]] **BridgePassword** __Password__:: If set, contains an HTTP authenticator that tells a bridge authority to serve all requested bridge information. Used by the (only partially @@ -2095,8 +2176,8 @@ The following options are used to configure a hidden service. option multiple times; each time applies to the service using the most recent HiddenServiceDir. By default, this option maps the virtual port to the same port on 127.0.0.1 over TCP. You may override the target port, - address, or both by specifying a target of addr, port, or addr:port. - (You can specify an IPv6 target as [addr]:port.) + address, or both by specifying a target of addr, port, addr:port, or + **unix:**__path__. (You can specify an IPv6 target as [addr]:port.) You may also have multiple lines with the same VIRTPORT: when a user connects to that VIRTPORT, one of the TARGETs from those lines will be chosen at random. @@ -2129,6 +2210,16 @@ The following options are used to configure a hidden service. not an authorization mechanism; it is instead meant to be a mild inconvenience to port-scanners.) (Default: 0) +[[HiddenServiceMaxStreams]] **HiddenServiceMaxStreams** __N__:: + The maximum number of simultaneous streams (connections) per rendezvous + circuit. (Setting this to 0 will allow an unlimited number of simultanous + streams.) (Default: 0) + +[[HiddenServiceMaxStreamsCloseCircuit]] **HiddenServiceMaxStreamsCloseCircuit** **0**|**1**:: + If set to 1, then exceeding **HiddenServiceMaxStreams** will cause the + offending rendezvous circuit to be torn down, as opposed to stream creation + requests that exceed the limit being silently ignored. (Default: 0) + [[RendPostPeriod]] **RendPostPeriod** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: Every time the specified period elapses, Tor uploads any rendezvous service descriptors to the directory servers. This information is also @@ -2140,6 +2231,10 @@ The following options are used to configure a hidden service. only owner is able to read the hidden service directory. (Default: 0) Has no effect on Windows. +[[HiddenServiceNumIntroductionPoints]] **HiddenServiceNumIntroductionPoints** __NUM__:: + Number of introduction points the hidden service will have. You can't + have more than 10. (Default: 3) + TESTING NETWORK OPTIONS ----------------------- @@ -2212,7 +2307,7 @@ The following options are used for running a testing Tor network. that **TestingTorNetwork** is set. (Default: 30 minutes) [[TestingEstimatedDescriptorPropagationTime]] **TestingEstimatedDescriptorPropagationTime** __N__ **minutes**|**hours**:: - Clients try downloading router descriptors from directory caches after this + Clients try downloading server descriptors from directory caches after this time. Changing this requires that **TestingTorNetwork** is set. (Default: 10 minutes) @@ -2260,7 +2355,7 @@ The following options are used for running a testing Tor network. this requires that **TestingTorNetwork** is set. (Default: 8) [[TestingDescriptorMaxDownloadTries]] **TestingDescriptorMaxDownloadTries** __NUM__:: - Try this often to download a router descriptor before giving up. + Try this often to download a server descriptor before giving up. Changing this requires that **TestingTorNetwork** is set. (Default: 8) [[TestingMicrodescMaxDownloadTries]] **TestingMicrodescMaxDownloadTries** __NUM__:: @@ -2281,6 +2376,14 @@ The following options are used for running a testing Tor network. has to be set. See the **ExcludeNodes** option for more information on how to specify nodes. +[[TestingDirAuthVoteExitIsStrict]] **TestingDirAuthVoteExitIsStrict** **0**|**1** :: + If True (1), a node will never receive the Exit flag unless it is specified + in the **TestingDirAuthVoteExit** list, regardless of its uptime, bandwidth, + or exit policy. + + + In order for this option to have any effect, **TestingTorNetwork** + has to be set. + [[TestingDirAuthVoteGuard]] **TestingDirAuthVoteGuard** __node__,__node__,__...__:: A list of identity fingerprints and country codes and address patterns of nodes to vote Guard for regardless of their @@ -2290,15 +2393,29 @@ The following options are used for running a testing Tor network. In order for this option to have any effect, **TestingTorNetwork** has to be set. +[[TestingDirAuthVoteGuardIsStrict]] **TestingDirAuthVoteGuardIsStrict** **0**|**1** :: + If True (1), a node will never receive the Guard flag unless it is specified + in the **TestingDirAuthVoteGuard** list, regardless of its uptime and bandwidth. + + + In order for this option to have any effect, **TestingTorNetwork** + has to be set. + [[TestingDirAuthVoteHSDir]] **TestingDirAuthVoteHSDir** __node__,__node__,__...__:: A list of identity fingerprints and country codes and address patterns of nodes to vote HSDir for regardless of their - uptime and ORPort connectivity. See the **ExcludeNodes** option for more + uptime and DirPort. See the **ExcludeNodes** option for more information on how to specify nodes. + In order for this option to have any effect, **TestingTorNetwork** and **VoteOnHidServDirectoriesV2** both have to be set. +[[TestingDirAuthVoteHSDirIsStrict]] **TestingDirAuthVoteHSDirIsStrict** **0**|**1** :: + If True (1), a node will never receive the HSDir flag unless it is specified + in the **TestingDirAuthVoteHSDir** list, regardless of its uptime and DirPort. + + + In order for this option to have any effect, **TestingTorNetwork** + has to be set. + [[TestingEnableConnBwEvent]] **TestingEnableConnBwEvent** **0**|**1**:: If this option is set, then Tor controllers may register for CONN_BW events. Changing this requires that **TestingTorNetwork** is set. @@ -2319,6 +2436,23 @@ The following options are used for running a testing Tor network. authority on a testing network. Overrides the usual default lower bound of 4 KB. (Default: 0) +[[TestingLinkCertLifetime]] **TestingLinkCertifetime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**|**months**:: + Overrides the default lifetime for the certificates used to authenticate + our X509 link cert with our ed25519 signing key. + (Default: 2 days) + +[[TestingAuthKeyLifetime]] **TestingAuthKeyLifetime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**|**months**:: + Overrides the default lifetime for a signing Ed25519 TLS Link authentication + key. + (Default: 2 days) + +[[TestingLinkKeySlop]] **TestingLinkKeySlop** __N__ **seconds**|**minutes**|**hours**:: +[[TestingAuthKeySlop]] **TestingAuthKeySlop** __N__ **seconds**|**minutes**|**hours**:: +[[TestingSigningKeySlop]] **TestingSigningKeySlop** __N__ **seconds**|**minutes**|**hours**:: + How early before the official expiration of a an Ed25519 signing key do + we replace it and issue a new key? + (Default: 3 hours for link and auth; 1 day for signing.) + SIGNALS ------- @@ -2402,7 +2536,7 @@ __DataDirectory__**/state**:: below). - When the file was last written - What version of Tor generated the state file - - A short history of bandwidth usage, as produced in the router + - A short history of bandwidth usage, as produced in the server descriptors. __DataDirectory__**/bw_accounting**:: @@ -2447,7 +2581,7 @@ __DataDirectory__**/unverified-microdesc-consensus**:: to check yet. __DataDirectory__**/unparseable-desc**:: - Onion router descriptors that Tor was unable to parse are dumped to this + Onion server descriptors that Tor was unable to parse are dumped to this file. Only used for debugging. __DataDirectory__**/router-stability**:: diff --git a/doc/torrc_format.txt b/doc/torrc_format.txt new file mode 100644 index 0000000000..7a809901d9 --- /dev/null +++ b/doc/torrc_format.txt @@ -0,0 +1,205 @@ + +This document specifies the current format and semantics of the torrc +file, as of July 2015. Note that we make no guarantee about the +stability of this format. If you write something designed for strict +compatibility with this document, please expect us to break it sooner or +later. + +Yes, some of this is quite stupid. My goal here is to explain what it +does, not what it should do. + + - Nick + + + +1. File Syntax + + ; The syntax here is defined an Augmented Backus-Naur form, as + ; specified in RFC5234. + + ; A file is interpreted as every Entry in the file, in order. + TorrcFile = *Line + + Line = BlankLine / Entry + + BlankLine = *WSP OptComment LF + BlankLine =/ *WSP LF + + OptComment = [ Comment ] + + Comment = "#" *NonLF + + ; Each Entry is interpreted as an optional "Magic" flag, a key, and a + ; value. + Entry = *WSP [ Magic ] Key 1*(1*WSP / "\" NL *WSP) Val LF + Entry =/ *WSP [ Magic ] Key *( *WSP / "\" NL *WSP) LF + + Magic = "+" / "/" + + ; Keys are always specified verbatim. They are case insensitive. It + ; is an error to specify a key that Tor does not recognize. + Key = 1*KC + + ; Sadly, every kind of value is decoded differently... + Val = QuotedVal / ContinuedVal / PlainVal + + ; The text of a PlainVal is the text of its PVBody portion, + ; plus the optional trailing backslash. + PlainVal = PVBody [ "\" ] *WSP OptComment + + ; Note that a PVBody is copied verbatim. Slashes are included + ; verbatim. No changes are made. Note that a body may be empty. + PVBody = * (VC / "\" NonLF ) + + ; The text of a ContinuedVal is the text of each of its PVBody + ; sub-elements, in order, concatenated. + ContinuedVal = CVal1 *CVal2 CVal3 + + CVal1 = PVBody "\" LF + CVal2 = PVBody ( "\" LF / Comment LF ) + CVal3 = PVBody + + ; The text of a QuotedVal is decoded as if it were a C string. + QuotedVal = DQ QVBody DQ *WSP Comment + + QVBody = QC + QVBody =/ "\" ( "n" / "r" / "t" / "\" / "'" / DQUOTE ) + QVBOdy =/ "\" ( "x" 2HEXDIG / 1*3OCTDIG ) + + ; Anything besides NUL and LF + NonLF = %x01-%x09 / %x0b - %xff + + OCTDIG = '0' - '7' + + KC = Any character except an isspace() character or '#' or NUL + VC = Any character except '\\', '\n', '#', or NUL + QC = Any character except '\n', '\\', '\"', or NUL + +2. Mid-level Semantics + + + There are four configuration "domains", from lowest to highest priority: + + * Built-in defaults + * The "torrc_defaults" file, if any + * The "torrc" file, if any + * Arguments provided on the command line, if any. + + Normally, values from high-priority domains override low-priority + domains, but see 'magic' below. + + Configuration keys fall into three categories: singletons, lists, and + groups. + + A singleton key may appear at most once in any domain. Its + corresponding value is equal to its value in the highest-priority + domain in which it occurs. + + A list key may appear any number of times in a domain. By default, + its corresponding value is equal to all of the values specified for + it in the highest-priority domain in which it appears. (See 'magic' + below). + + A group key may appear any number of times in a domain. It is + associated with a number of other keys in the same group. The + relative positions of entries with the keys in a single group + matters, but entries with keys not in the group may be freely + interspersed. By default, the group has a value equal to all keys + and values it contains, from the highest-priority domain in which any + of its keys occurs. + + Magic: + + If the '/' flag is specified for an entry, it sets the value for + that entry to an empty list. (This will cause a higher-priority + domain to clear a list from a lower-priority domain, without + actually adding any entries.) + + If the '+' flag is specified for the first entry in a list or a + group that appears in a given domain, that list or group is + appended to the list or group from the next-lowest-priority + domain, rather than replacing it. + +3. High-level semantics + + There are further constraints on the values that each entry can take. + These constraints are out-of-scope for this document. + +4. Examples + + (Indentation is removed in this section, to avoid confusion.) + +4.1. Syntax examples + +# Here is a simple configuration entry. The key is "Foo"; the value is +# "Bar" + +Foo Bar + +# A configuration entry can have spaces in its value, as below. Here the +# key is "Foo" and the value is "Bar Baz" +Foo Bar Baz + +# This configuration entry has space at the end of the line, but those +# spaces don't count, so the key and value are still "Foo" and "Bar Baz" +Foo Bar Baz + +# There can be an escaped newline between the value and the key. This +# is another way to say key="Hello", value="World" +Hello\ +World + +# In regular entries of this kind, you can have a comment at the end of +# the line, either with a space before it or not. Each of these is a +# different spelling of key="Hello", value="World" + +Hello World #today +Hello World#tomorrow + +# One way to encode a complex entry is as a C string. This is the same +# as key="Hello", value="World!" +Hello "World!" + +# The string can contain the usual set of C escapes. This entry has +# key="Hello", and value="\"World\"\nand\nuniverse" +Hello "\"World\"\nand\nuniverse" + +# And now we get to the more-or-less awful part. +# +# Multi-line entries ending with a backslash on each line aren't so +# bad. The backslash is removed, and everything else is included +# verbatim. So this entry has key="Hello" and value="Worldandfriends" +Hello\ +World\ +and\ +friends + +# Backslashes in the middle of a line are included as-is. The key of +# this one is "Too" and the value is "Many\\Backsl\ashes here" (with +# backslashes in that last string as-is) +Too \ +Many\\\ +Backsl\ashes \\ +here + +# And here's the really yucky part. If a comment appears in a multi-line +# entry, the entry is still able to continue on the next line, as in the +# following, where the key is "This" and the value is +# "entry and some are silly" +This entry \ + # has comments \ + and some \ + are # generally \ + silly + +# But you can also write that without the backslashes at the end of the +# comment lines. That is to say, this entry is exactly the same as the +# one above! +This entry \ + # has comments + and some \ + are # generally + silly + + + |