diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/HACKING | 545 | ||||
| -rw-r--r-- | doc/HACKING/CodingStandards.md | 245 | ||||
| -rw-r--r-- | doc/HACKING/GettingStarted.md | 187 | ||||
| -rw-r--r-- | doc/HACKING/HelpfulTools.md | 293 | ||||
| -rw-r--r-- | doc/HACKING/HowToReview.md | 85 | ||||
| -rw-r--r-- | doc/HACKING/README.1st.md | 62 | ||||
| -rw-r--r-- | doc/HACKING/ReleasingTor.md | 143 | ||||
| -rw-r--r-- | doc/HACKING/WritingTests.md | 445 | ||||
| -rw-r--r-- | doc/TUNING | 86 | ||||
| -rwxr-xr-x | doc/asciidoc-helper.sh | 2 | ||||
| -rw-r--r-- | doc/building-tor-msvc.txt | 122 | ||||
| -rw-r--r-- | doc/contrib/tor-rpm-creation.txt | 2 | ||||
| -rw-r--r-- | doc/include.am | 25 | ||||
| -rw-r--r-- | doc/tor-fw-helper.1.txt | 60 | ||||
| -rw-r--r-- | doc/tor-resolve.1.txt | 5 | ||||
| -rw-r--r-- | doc/tor.1.txt | 837 | ||||
| -rw-r--r-- | doc/torrc_format.txt | 205 |
17 files changed, 2516 insertions, 833 deletions
diff --git a/doc/HACKING b/doc/HACKING deleted file mode 100644 index c69b2a6fee..0000000000 --- a/doc/HACKING +++ /dev/null @@ -1,545 +0,0 @@ -Hacking Tor: An Incomplete Guide -================================ - -Getting started ---------------- - -For full information on how Tor is supposed to work, look at the files in -https://gitweb.torproject.org/torspec.git/tree - -For an explanation of how to change Tor's design to work differently, look at -https://gitweb.torproject.org/torspec.git/blob_plain/HEAD:/proposals/001-process.txt - -For the latest version of the code, get a copy of git, and - - git clone https://git.torproject.org/git/tor - -We talk about Tor on the tor-talk mailing list. Design proposals and -discussion belong on the tor-dev mailing list. We hang around on -irc.oftc.net, with general discussion happening on #tor and development -happening on #tor-dev. - -How we use Git branches ------------------------ - -Each main development series (like 0.2.1, 0.2.2, etc) has its main work -applied to a single branch. At most one series can be the development series -at a time; all other series are maintenance series that get bug-fixes only. -The development series is built in a git branch called "master"; the -maintenance series are built in branches called "maint-0.2.0", "maint-0.2.1", -and so on. We regularly merge the active maint branches forward. - -For all series except the development series, we also have a "release" branch -(as in "release-0.2.1"). The release series is based on the corresponding -maintenance series, except that it deliberately lags the maint series for -most of its patches, so that bugfix patches are not typically included in a -maintenance release until they've been tested for a while in a development -release. Occasionally, we'll merge an urgent bugfix into the release branch -before it gets merged into maint, but that's rare. - -If you're working on a bugfix for a bug that occurs in a particular version, -base your bugfix branch on the "maint" branch for the first supported series -that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) If -you're working on a new feature, base it on the master branch. - - -How we log changes ------------------- - -When you do a commit that needs a ChangeLog entry, add a new file to -the "changes" toplevel subdirectory. It should have the format of a -one-entry changelog section from the current ChangeLog file, as in - - o Major bugfixes: - - Fix a potential buffer overflow. Fixes bug 99999; bugfix on - 0.3.1.4-beta. - -To write a changes file, first categorize the change. Some common categories -are: Minor bugfixes, Major bugfixes, Minor features, Major features, Code -simplifications and refactoring. Then say what the change does. If -it's a bugfix, mention what bug it fixes and when the bug was -introduced. To find out which Git tag the change was introduced in, -you can use "git describe --contains <sha1 of commit>". - -If at all possible, try to create this file in the same commit where -you are making the change. Please give it a distinctive name that no -other branch will use for the lifetime of your change. - -When we go to make a release, we will concatenate all the entries -in changes to make a draft changelog, and clear the directory. We'll -then edit the draft changelog into a nice readable format. - -What needs a changes file?:: - A not-exhaustive list: Anything that might change user-visible - behavior. Anything that changes internals, documentation, or the build - system enough that somebody could notice. Big or interesting code - rewrites. Anything about which somebody might plausibly wonder "when - did that happen, and/or why did we do that" 6 months down the line. - -Why use changes files instead of Git commit messages?:: - Git commit messages are written for developers, not users, and they - are nigh-impossible to revise after the fact. - -Why use changes files instead of entries in the ChangeLog?:: - Having every single commit touch the ChangeLog file tended to create - zillions of merge conflicts. - -Useful tools ------------- - -These aren't strictly necessary for hacking on Tor, but they can help track -down bugs. - -Jenkins -~~~~~~~ - -https://jenkins.torproject.org - -Dmalloc -~~~~~~~ - -The dmalloc library will keep track of memory allocation, so you can find out -if we're leaking memory, doing any double-frees, or so on. - - dmalloc -l ~/dmalloc.log - (run the commands it tells you) - ./configure --with-dmalloc - -Valgrind -~~~~~~~~ - -valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor - -(Note that if you get a zillion openssl warnings, you will also need to -pass --undef-value-errors=no to valgrind, or rebuild your openssl -with -DPURIFY.) - -Running gcov for unit test coverage -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ------ - ./configure --enable-coverage - make - make check - mkdir coverage-output - ./scripts/test/coverage coverage-output ------ - -(On OSX, you'll need to start with "--enable-coverage CC=clang".) - -Then, look at the .gcov files in coverage-output. '-' before a line means -that the compiler generated no code for that line. '######' means that the -line was never reached. Lines with numbers were called that number of times. - -If that doesn't work: - * Try configuring Tor with --disable-gcc-hardening - * You might need to run 'make clean' after you run './configure'. - -If you make changes to Tor and want to get another set of coverage results, -you can run "make reset-gcov" to clear the intermediary gcov output. - -If you have two different "coverage-output" directories, and you want to see -a meaningful diff between them, you can run: - ------ - ./scripts/test/cov-diff coverage-output1 coverage-output2 | less ------ - -In this diff, any lines that were visited at least once will have coverage -"1". This lets you inspect what you (probably) really want to know: which -untested lines were changed? Are there any new untested lines? - -Running integration tests -~~~~~~~~~~~~~~~~~~~~~~~~~ - -We have the beginnings of a set of scripts to run integration tests using -Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and -run "make test-network". - -Profiling Tor with oprofile -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The oprofile tool runs (on Linux only!) to tell you what functions Tor is -spending its CPU time in, so we can identify berformance pottlenecks. - -Here are some basic instructions - - - Build tor with debugging symbols (you probably already have, unless - you messed with CFLAGS during the build process). - - Build all the libraries you care about with debugging symbols - (probably you only care about libssl, maybe zlib and Libevent). - - Copy this tor to a new directory - - Copy all the libraries it uses to that dir too (ldd ./tor will - tell you) - - Set LD_LIBRARY_PATH to include that dir. ldd ./tor should now - show you it's using the libs in that dir - - Run that tor - - Reset oprofiles counters/start it - * "opcontrol --reset; opcontrol --start", if Nick remembers right. - - After a while, have it dump the stats on tor and all the libs - in that dir you created. - * "opcontrol --dump;" - * "opreport -l that_dir/*" - - Profit - - -Coding conventions ------------------- - -Patch checklist -~~~~~~~~~~~~~~~ - -If possible, send your patch as one of these (in descending order of -preference) - - - A git branch we can pull from - - Patches generated by git format-patch - - A unified diff - -Did you remember... - - - To build your code while configured with --enable-gcc-warnings? - - To run "make check-spaces" on your code? - - To run "make check-docs" to see whether all new options are on - the manpage? - - To write unit tests, as possible? - - To base your code on the appropriate branch? - - To include a file in the "changes" directory as appropriate? - -Whitespace and C conformance -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Invoke "make check-spaces" from time to time, so it can tell you about -deviations from our C whitespace style. Generally, we use: - - - Unix-style line endings - - K&R-style indentation - - No space before newlines - - A blank line at the end of each file - - Never more than one blank line in a row - - Always spaces, never tabs - - No more than 79-columns per line. - - Two spaces per indent. - - A space between control keywords and their corresponding paren - "if (x)", "while (x)", and "switch (x)", never "if(x)", "while(x)", or - "switch(x)". - - A space between anything and an open brace. - - No space between a function name and an opening paren. "puts(x)", not - "puts (x)". - - Function declarations at the start of the line. - -We try hard to build without warnings everywhere. In particular, if you're -using gcc, you should invoke the configure script with the option -"--enable-gcc-warnings". This will give a bunch of extra warning flags to -the compiler, and help us find divergences from our preferred C style. - -Getting emacs to edit Tor source properly -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Nick likes to put the following snippet in his .emacs file: - ------ - (add-hook 'c-mode-hook - (lambda () - (font-lock-mode 1) - (set-variable 'show-trailing-whitespace t) - - (let ((fname (expand-file-name (buffer-file-name)))) - (cond - ((string-match "^/home/nickm/src/libevent" fname) - (set-variable 'indent-tabs-mode t) - (set-variable 'c-basic-offset 4) - (set-variable 'tab-width 4)) - ((string-match "^/home/nickm/src/tor" fname) - (set-variable 'indent-tabs-mode nil) - (set-variable 'c-basic-offset 2)) - ((string-match "^/home/nickm/src/openssl" fname) - (set-variable 'indent-tabs-mode t) - (set-variable 'c-basic-offset 8) - (set-variable 'tab-width 8)) - )))) ------ - -You'll note that it defaults to showing all trailing whitespace. The "cond" -test detects whether the file is one of a few C free software projects that I -often edit, and sets up the indentation level and tab preferences to match -what they want. - -If you want to try this out, you'll need to change the filename regex -patterns to match where you keep your Tor files. - -If you use emacs for editing Tor and nothing else, you could always just say: - ------ - (add-hook 'c-mode-hook - (lambda () - (font-lock-mode 1) - (set-variable 'show-trailing-whitespace t) - (set-variable 'indent-tabs-mode nil) - (set-variable 'c-basic-offset 2))) ------ - -There is probably a better way to do this. No, we are probably not going -to clutter the files with emacs stuff. - - -Functions to use -~~~~~~~~~~~~~~~~ - -We have some wrapper functions like tor_malloc, tor_free, tor_strdup, and -tor_gettimeofday; use them instead of their generic equivalents. (They -always succeed or exit.) - -You can get a full list of the compatibility functions that Tor provides by -looking through src/common/util.h and src/common/compat.h. You can see the -available containers in src/common/containers.h. You should probably -familiarize yourself with these modules before you write too much code, or -else you'll wind up reinventing the wheel. - -Use 'INLINE' instead of 'inline', so that we work properly on Windows. - -Calling and naming conventions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Whenever possible, functions should return -1 on error and 0 on success. - -For multi-word identifiers, use lowercase words combined with -underscores. (e.g., "multi_word_identifier"). Use ALL_CAPS for macros and -constants. - -Typenames should end with "_t". - -Function names should be prefixed with a module name or object name. (In -general, code to manipulate an object should be a module with the same name -as the object, so it's hard to tell which convention is used.) - -Functions that do things should have imperative-verb names -(e.g. buffer_clear, buffer_resize); functions that return booleans should -have predicate names (e.g. buffer_is_empty, buffer_needs_resizing). - -If you find that you have four or more possible return code values, it's -probably time to create an enum. If you find that you are passing three or -more flags to a function, it's probably time to create a flags argument that -takes a bitfield. - -What To Optimize -~~~~~~~~~~~~~~~~ - -Don't optimize anything if it's not in the critical path. Right now, the -critical path seems to be AES, logging, and the network itself. Feel free to -do your own profiling to determine otherwise. - -Log conventions -~~~~~~~~~~~~~~~ - -https://trac.torproject.org/projects/tor/wiki/doc/TorFAQ#loglevel - -No error or warning messages should be expected during normal OR or OP -operation. - -If a library function is currently called such that failure always means ERR, -then the library function should log WARN and let the caller log ERR. - -Every message of severity INFO or higher should either (A) be intelligible -to end-users who don't know the Tor source; or (B) somehow inform the -end-users that they aren't expected to understand the message (perhaps -with a string like "internal error"). Option (A) is to be preferred to -option (B). - -Doxygen -~~~~~~~~ - -We use the 'doxygen' utility to generate documentation from our -source code. Here's how to use it: - - 1. Begin every file that should be documented with - /** - * \file filename.c - * \brief Short description of the file. - **/ - - (Doxygen will recognize any comment beginning with /** as special.) - - 2. Before any function, structure, #define, or variable you want to - document, add a comment of the form: - - /** Describe the function's actions in imperative sentences. - * - * Use blank lines for paragraph breaks - * - and - * - hyphens - * - for - * - lists. - * - * Write <b>argument_names</b> in boldface. - * - * \code - * place_example_code(); - * between_code_and_endcode_commands(); - * \endcode - */ - - 3. Make sure to escape the characters "<", ">", "\", "%" and "#" as "\<", - "\>", "\\", "\%", and "\#". - - 4. To document structure members, you can use two forms: - - struct foo { - /** You can put the comment before an element; */ - int a; - int b; /**< Or use the less-than symbol to put the comment - * after the element. */ - }; - - 5. To generate documentation from the Tor source code, type: - - $ doxygen -g - - To generate a file called 'Doxyfile'. Edit that file and run - 'doxygen' to generate the API documentation. - - 6. See the Doxygen manual for more information; this summary just - scratches the surface. - -Doxygen comment conventions -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Say what functions do as a series of one or more imperative sentences, as -though you were telling somebody how to be the function. In other words, DO -NOT say: - - /** The strtol function parses a number. - * - * nptr -- the string to parse. It can include whitespace. - * endptr -- a string pointer to hold the first thing that is not part - * of the number, if present. - * base -- the numeric base. - * returns: the resulting number. - */ - long strtol(const char *nptr, char **nptr, int base); - -Instead, please DO say: - - /** Parse a number in radix <b>base</b> from the string <b>nptr</b>, - * and return the result. Skip all leading whitespace. If - * <b>endptr</b> is not NULL, set *<b>endptr</b> to the first character - * after the number parsed. - **/ - long strtol(const char *nptr, char **nptr, int base); - -Doxygen comments are the contract in our abstraction-by-contract world: if -the functions that call your function rely on it doing something, then your -function should mention that it does that something in the documentation. If -you rely on a function doing something beyond what is in its documentation, -then you should watch out, or it might do something else later. - -Putting out a new release -------------------------- - -Here are the steps Roger takes when putting out a new Tor release: - -1) Use it for a while, as a client, as a relay, as a hidden service, -and as a directory authority. See if it has any obvious bugs, and -resolve those. - -1.5) As applicable, merge the maint-X branch into the release-X branch. - -2) Gather the changes/* files into a changelog entry, rewriting many -of them and reordering to focus on what users and funders would find -interesting and understandable. - - 2.1) Make sure that everything that wants a bug number has one. - Make sure that everything which is a bugfix says what version - it was a bugfix on. - 2.2) Concatenate them. - 2.3) Sort them by section. Within each section, sort by "version it's - a bugfix on", else by numerical ticket order. - - 2.4) Clean them up: - - Standard idioms: - "Fixes bug 9999; bugfix on 0.3.3.3-alpha." - - One period after a space. - - Make stuff very terse - - Make sure each section name ends with a colon - - Describe the user-visible problem right away - - Mention relevant config options by name. If they're rare or unusual, - remind people what they're for - - Avoid starting lines with open-paren - - Present and imperative tense: not past. - - 'Relays', not 'servers' or 'nodes' or 'Tor relays'. - - "Stop FOOing", not "Fix a bug where we would FOO". - - Try not to let any given section be longer than about a page. Break up - long sections into subsections by some sort of common subtopic. This - guideline is especially important when organizing Release Notes for - new stable releases. - - If a given changes stanza showed up in a different release (e.g. - maint-0.2.1), be sure to make the stanzas identical (so people can - distinguish if these are the same change). - - 2.5) Merge them in. - - 2.6) Clean everything one last time. - - 2.7) Run it through fmt to make it pretty. - -3) Compose a short release blurb to highlight the user-facing -changes. Insert said release blurb into the ChangeLog stanza. If it's -a stable release, add it to the ReleaseNotes file too. If we're adding -to a release-0.2.x branch, manually commit the changelogs to the later -git branches too. - -4) Bump the version number in configure.ac and rebuild. - -5) Make dist, put the tarball up somewhere, and tell #tor about it. Wait -a while to see if anybody has problems building it. Try to get Sebastian -or somebody to try building it on Windows. - -6) Get at least two of weasel/arma/sebastian to put the new version number -in their approved versions list. - -7) Sign the tarball, then sign and push the git tag: - gpg -ba <the_tarball> - git tag -u <keyid> tor-0.2.x.y-status - git push origin tag tor-0.2.x.y-status - -8) scp the tarball and its sig to the website in the dist/ directory -(i.e. /srv/www-master.torproject.org/htdocs/dist/ on vescum). Edit -"include/versions.wmi" and "Makefile" to note the new version. From your -website checkout, run ./publish to build and publish the website. - -9) Email the packagers (cc'ing tor-assistants) that a new tarball is up. - -10) Add the version number to Trac. To do this, go to Trac, log in, -select "Admin" near the top of the screen, then select "Versions" from -the menu on the left. At the right, there will be an "Add version" -box. By convention, we enter the version in the form "Tor: -0.2.2.23-alpha" (or whatever the version is), and we select the date as -the date in the ChangeLog. - -11) Forward-port the ChangeLog. - -12) Wait up to a day or two (for a development release), or until most -packages are up (for a stable release), and mail the release blurb and -changelog to tor-talk or tor-announce. - - (We might be moving to faster announcements, but don't announce until - the website is at least updated.) - -13) If it's a stable release, bump the version number in the maint-x.y.z - branch to "newversion-dev", and do a "merge -s ours" merge to avoid - taking that change into master. Do a similar 'merge -s theirs' - merge to get the change (and only that change) into release. (Some - of the build scripts require that maint merge cleanly into release.) - diff --git a/doc/HACKING/CodingStandards.md b/doc/HACKING/CodingStandards.md new file mode 100644 index 0000000000..4aafa5ddd4 --- /dev/null +++ b/doc/HACKING/CodingStandards.md @@ -0,0 +1,245 @@ +Coding conventions for Tor +========================== + +tl;dr: + + - Run configure with `--enable-gcc-warnings` + - Run `make check-spaces` to catch whitespace errors + - Document your functions + - Write unit tests + - Add a file in `changes` for your branch. + +Patch checklist +--------------- + +If possible, send your patch as one of these (in descending order of +preference) + + - A git branch we can pull from + - Patches generated by git format-patch + - A unified diff + +Did you remember... + + - To build your code while configured with `--enable-gcc-warnings`? + - To run `make check-spaces` on your code? + - To run `make check-docs` to see whether all new options are on + the manpage? + - To write unit tests, as possible? + - To base your code on the appropriate branch? + - To include a file in the `changes` directory as appropriate? + +How we use Git branches +======================= + +Each main development series (like 0.2.1, 0.2.2, etc) has its main work +applied to a single branch. At most one series can be the development series +at a time; all other series are maintenance series that get bug-fixes only. +The development series is built in a git branch called "master"; the +maintenance series are built in branches called "maint-0.2.0", "maint-0.2.1", +and so on. We regularly merge the active maint branches forward. + +For all series except the development series, we also have a "release" branch +(as in "release-0.2.1"). The release series is based on the corresponding +maintenance series, except that it deliberately lags the maint series for +most of its patches, so that bugfix patches are not typically included in a +maintenance release until they've been tested for a while in a development +release. Occasionally, we'll merge an urgent bugfix into the release branch +before it gets merged into maint, but that's rare. + +If you're working on a bugfix for a bug that occurs in a particular version, +base your bugfix branch on the "maint" branch for the first supported series +that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) If +you're working on a new feature, base it on the master branch. + + +How we log changes +================== + +When you do a commit that needs a ChangeLog entry, add a new file to +the `changes` toplevel subdirectory. It should have the format of a +one-entry changelog section from the current ChangeLog file, as in + +- Major bugfixes: + - Fix a potential buffer overflow. Fixes bug 99999; bugfix on + 0.3.1.4-beta. + +To write a changes file, first categorize the change. Some common categories +are: Minor bugfixes, Major bugfixes, Minor features, Major features, Code +simplifications and refactoring. Then say what the change does. If +it's a bugfix, mention what bug it fixes and when the bug was +introduced. To find out which Git tag the change was introduced in, +you can use `git describe --contains <sha1 of commit>`. + +If at all possible, try to create this file in the same commit where you are +making the change. Please give it a distinctive name that no other branch will +use for the lifetime of your change. To verify the format of the changes file, +you can use `make check-changes`. + +When we go to make a release, we will concatenate all the entries +in changes to make a draft changelog, and clear the directory. We'll +then edit the draft changelog into a nice readable format. + +To make sure that stuff is in the right format, we use +scripts/maint/lintChanges.py to check the changes files for +(superficial) validity. You can run this script on your own changes +files! + +What needs a changes file? + + * A not-exhaustive list: Anything that might change user-visible + behavior. Anything that changes internals, documentation, or the build + system enough that somebody could notice. Big or interesting code + rewrites. Anything about which somebody might plausibly wonder "when + did that happen, and/or why did we do that" 6 months down the line. + +Why use changes files instead of Git commit messages? + + * Git commit messages are written for developers, not users, and they + are nigh-impossible to revise after the fact. + +Why use changes files instead of entries in the ChangeLog? + + * Having every single commit touch the ChangeLog file tended to create + zillions of merge conflicts. + +Whitespace and C conformance +---------------------------- + +Invoke `make check-spaces` from time to time, so it can tell you about +deviations from our C whitespace style. Generally, we use: + + - Unix-style line endings + - K&R-style indentation + - No space before newlines + - A blank line at the end of each file + - Never more than one blank line in a row + - Always spaces, never tabs + - No more than 79-columns per line. + - Two spaces per indent. + - A space between control keywords and their corresponding paren + `if (x)`, `while (x)`, and `switch (x)`, never `if(x)`, `while(x)`, or + `switch(x)`. + - A space between anything and an open brace. + - No space between a function name and an opening paren. `puts(x)`, not + `puts (x)`. + - Function declarations at the start of the line. + +We try hard to build without warnings everywhere. In particular, if you're +using gcc, you should invoke the configure script with the option +`--enable-gcc-warnings`. This will give a bunch of extra warning flags to +the compiler, and help us find divergences from our preferred C style. + +Functions to use; functions not to use +-------------------------------------- + +We have some wrapper functions like `tor_malloc`, `tor_free`, `tor_strdup`, and +`tor_gettimeofday;` use them instead of their generic equivalents. (They +always succeed or exit.) + +You can get a full list of the compatibility functions that Tor provides by +looking through `src/common/util*.h` and `src/common/compat*.h`. You can see the +available containers in `src/common/containers*.h`. You should probably +familiarize yourself with these modules before you write too much code, or +else you'll wind up reinventing the wheel. + +We don't use `strcat` or `strcpy` or `sprintf` of any of those notoriously broken +old C functions. Use `strlcat`, `strlcpy`, or `tor_snprintf/tor_asprintf` instead. + +We don't call `memcmp()` directly. Use `fast_memeq()`, `fast_memneq()`, +`tor_memeq()`, or `tor_memneq()` for most purposes. + +Functions not to write +---------------------- + +Try to never hand-write new code to parse or generate binary +formats. Instead, use trunnel if at all possible. See + + https://gitweb.torproject.org/trunnel.git/tree + +for more information about trunnel. + +For information on adding new trunnel code to Tor, see src/trunnel/README + + +Calling and naming conventions +------------------------------ + +Whenever possible, functions should return -1 on error and 0 on success. + +For multi-word identifiers, use lowercase words combined with +underscores. (e.g., `multi_word_identifier`). Use ALL_CAPS for macros and +constants. + +Typenames should end with `_t`. + +Function names should be prefixed with a module name or object name. (In +general, code to manipulate an object should be a module with the same name +as the object, so it's hard to tell which convention is used.) + +Functions that do things should have imperative-verb names +(e.g. `buffer_clear`, `buffer_resize`); functions that return booleans should +have predicate names (e.g. `buffer_is_empty`, `buffer_needs_resizing`). + +If you find that you have four or more possible return code values, it's +probably time to create an enum. If you find that you are passing three or +more flags to a function, it's probably time to create a flags argument that +takes a bitfield. + +What To Optimize +---------------- + +Don't optimize anything if it's not in the critical path. Right now, the +critical path seems to be AES, logging, and the network itself. Feel free to +do your own profiling to determine otherwise. + +Log conventions +--------------- + +`https://www.torproject.org/docs/faq#LogLevel` + +No error or warning messages should be expected during normal OR or OP +operation. + +If a library function is currently called such that failure always means ERR, +then the library function should log WARN and let the caller log ERR. + +Every message of severity INFO or higher should either (A) be intelligible +to end-users who don't know the Tor source; or (B) somehow inform the +end-users that they aren't expected to understand the message (perhaps +with a string like "internal error"). Option (A) is to be preferred to +option (B). + + + +Doxygen comment conventions +--------------------------- + +Say what functions do as a series of one or more imperative sentences, as +though you were telling somebody how to be the function. In other words, DO +NOT say: + + /** The strtol function parses a number. + * + * nptr -- the string to parse. It can include whitespace. + * endptr -- a string pointer to hold the first thing that is not part + * of the number, if present. + * base -- the numeric base. + * returns: the resulting number. + */ + long strtol(const char *nptr, char **nptr, int base); + +Instead, please DO say: + + /** Parse a number in radix <b>base</b> from the string <b>nptr</b>, + * and return the result. Skip all leading whitespace. If + * <b>endptr</b> is not NULL, set *<b>endptr</b> to the first character + * after the number parsed. + **/ + long strtol(const char *nptr, char **nptr, int base); + +Doxygen comments are the contract in our abstraction-by-contract world: if +the functions that call your function rely on it doing something, then your +function should mention that it does that something in the documentation. If +you rely on a function doing something beyond what is in its documentation, +then you should watch out, or it might do something else later. diff --git a/doc/HACKING/GettingStarted.md b/doc/HACKING/GettingStarted.md new file mode 100644 index 0000000000..0295adc1ff --- /dev/null +++ b/doc/HACKING/GettingStarted.md @@ -0,0 +1,187 @@ + +Getting started in Tor development +================================== + +Congratulations! You've found this file, and you're reading it! This +means that you might be interested in getting started in developing Tor. + +(This guide is just about Tor itself--the small network program at the +heart of the Tor network--and not about all the other programs in the +whole Tor ecosystem.) + + +If you are looking for a more bare-bones, less user-friendly information +dump of important information, you might like reading doc/HACKING +instead. You should probably read it before you write your first patch. + + +Required background +------------------- + +First, I'm going to assume that you can build Tor from source, and that +you know enough of the C language to read and write it. (See the README +file that comes with the Tor source for more information on building it, +and any high-quality guide to C for information on programming.) + +I'm also going to assume that you know a little bit about how to use +Git, or that you're able to follow one of the several excellent guides +at http://git-scm.org to learn. + +Most Tor developers develop using some Unix-based system, such as Linux, +BSD, or OSX. It's okay to develop on Windows if you want, but you're +going to have a more difficult time. + + +Getting your first patch into Tor +--------------------------------- + +Once you've reached this point, here's what you need to know. + + 1. Get the source. + + We keep our source under version control in Git. To get the latest + version, run + + git clone https://git.torproject.org/git/tor + + This will give you a checkout of the master branch. If you're + going to fix a bug that appears in a stable version, check out the + appropriate "maint" branch, as in: + + git checkout maint-0.2.7 + + 2. Find your way around the source + + Our overall code structure is explained in the "torguts" documents, + currently at + + git clone https://git.torproject.org/user/nickm/torguts.git + + Find a part of the code that looks interesting to you, and start + looking around it to see how it fits together! + + We do some unusual things in our codebase. Our testing-related + practices and kludges are explained in doc/WritingTests.txt. + + If you see something that doesn't make sense, we love to get + questions! + + 3. Find something cool to hack on. + + You may already have a good idea of what you'd like to work on, or + you might be looking for a way to contribute. + + Many people have gotten started by looking for an area where they + personally felt Tor was underperforming, and investigating ways to + fix it. If you're looking for ideas, you can head to our bug + tracker at trac.torproject.org and look for tickets that have + received the "easy" tag: these are ones that developers think would + be pretty simple for a new person to work on. For a bigger + challenge, you might want to look for tickets with the "lorax" + keyword: these are tickets that the developers think might be a + good idea to build, but which we have no time to work on any time + soon. + + Or you might find another open ticket that piques your + interest. It's all fine! + + For your first patch, it is probably NOT a good idea to make + something huge or invasive. In particular, you should probably + avoid: + + * Major changes spread across many parts of the codebase. + * Major changes to programming practice or coding style. + * Huge new features or protocol changes. + + 4. Meet the developers! + + We discuss stuff on the tor-dev mailing list and on the #tor-dev + IRC channel on OFTC. We're generally friendly and approachable, + and we like to talk about how Tor fits together. If we have ideas + about how something should be implemented, we'll be happy to share + them. + + We currently have a patch workshop at least once a week, where + people share patches they've made and discuss how to make them + better. The time might change in the future, but generally, + there's no bad time to talk, and ask us about patch ideas. + + 5. Do you need to write a design proposal? + + If your idea is very large, or it will require a change to Tor's + protocols, there needs to be a written design proposal before it + can be merged. (We use this process to manage changes in the + protocols.) To write one, see the instructions at + https://gitweb.torproject.org/torspec.git/tree/proposals/001-process.txt + . If you'd like help writing a proposal, just ask! We're happy to + help out with good ideas. + + You might also like to look around the rest of that directory, to + see more about open and past proposed changes to Tor's behavior. + + 6. Writing your patch + + As you write your code, you'll probably want it to fit in with the + standards of the rest of the Tor codebase so it will be easy for us + to review and merge. You can learn our coding standards in + doc/HACKING. + + If your patch is large and/or is divided into multiple logical + components, remember to divide it into a series of Git commits. A + series of small changes is much easier to review than one big lump. + + 7. Testing your patch + + We prefer that all new or modified code have unit tests for it to + ensure that it runs correctly. Also, all code should actually be + _run_ by somebody, to make sure it works. + + See doc/WritingTests.txt for more information on how we test things + in Tor. If you'd like any help writing tests, just ask! We're + glad to help out. + + 8. Submitting your patch + + We review patches through tickets on our bugtracker at + trac.torproject.org. You can either upload your patches there, or + put them at a public git repository somewhere we can fetch them + (like github or bitbucket) and then paste a link on the appropriate + trac ticket. + + Once your patches are available, write a short explanation of what + you've done on trac, and then change the status of the ticket to + needs_review. + + 9. Review, Revision, and Merge + + With any luck, somebody will review your patch soon! If not, you + can ask on the IRC channel; sometimes we get really busy and take + longer than we should. But don't let us slow you down: you're the + one who's offering help here, and we should respect your time and + contributions. + + When your patch is reviewed, one of these things will happen: + + * The reviewer will say "looks good to me" and your + patch will get merged right into Tor. [Assuming we're not + in the middle of a code-freeze window. If the codebase is + frozen, your patch will go into the next release series.] + + * OR the reviewer will say "looks good, just needs some small + changes!" And then the reviewer will make those changes, + and merge the modified patch into Tor. + + * OR the reviewer will say "Here are some questions and + comments," followed by a bunch of stuff that the reviewer + thinks should change in your code, or questions that the + reviewer has. + + At this point, you might want to make the requested changes + yourself, and comment on the trac ticket once you have done + so. Or if you disagree with any of the comments, you should + say so! And if you won't have time to make some of the + changes, you should say that too, so that other developers + will be able to pick up the unfinished portion. + + Congratulations! You have now written your first patch, and gotten + it integrated into mainline Tor. diff --git a/doc/HACKING/HelpfulTools.md b/doc/HACKING/HelpfulTools.md new file mode 100644 index 0000000000..a7f36e6c7e --- /dev/null +++ b/doc/HACKING/HelpfulTools.md @@ -0,0 +1,293 @@ +Useful tools +============ + +These aren't strictly necessary for hacking on Tor, but they can help track +down bugs. + +Jenkins +------- + + https://jenkins.torproject.org + +Dmalloc +------- + +The dmalloc library will keep track of memory allocation, so you can find out +if we're leaking memory, doing any double-frees, or so on. + + dmalloc -l -/dmalloc.log + (run the commands it tells you) + ./configure --with-dmalloc + +Valgrind +-------- + + valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor + +(Note that if you get a zillion openssl warnings, you will also need to +pass `--undef-value-errors=no` to valgrind, or rebuild your openssl +with `-DPURIFY`.) + +Coverity +-------- + +Nick regularly runs the coverity static analyzer on the Tor codebase. + +The preprocessor define `__COVERITY__` is used to work around instances +where coverity picks up behavior that we wish to permit. + +clang Static Analyzer +--------------------- + +The clang static analyzer can be run on the Tor codebase using Xcode (WIP) +or a command-line build. + +The preprocessor define `__clang_analyzer__` is used to work around instances +where clang picks up behavior that we wish to permit. + +clang Runtime Sanitizers +------------------------ + +To build the Tor codebase with the clang Address and Undefined Behavior +sanitizers, see the file `contrib/clang/sanitize_blacklist.txt`. + +Preprocessor workarounds for instances where clang picks up behavior that +we wish to permit are also documented in the blacklist file. + +Running lcov for unit test coverage +----------------------------------- + +Lcov is a utility that generates pretty HTML reports of test code coverage. +To generate such a report: + + ./configure --enable-coverage + make + make coverage-html + $BROWSER ./coverage_html/index.html + +This will run the tor unit test suite `./src/test/test` and generate the HTML +coverage code report under the directory `./coverage_html/`. To change the +output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`. + +Coverage diffs using lcov are not currently implemented, but are being +investigated (as of July 2014). + +Running the unit tests +---------------------- + +To quickly run all the tests distributed with Tor: + + make check + +To run the fast unit tests only: + + make test + +To selectively run just some tests (the following can be combined +arbitrarily): + + ./src/test/test <name_of_test> [<name of test 2>] ... + ./src/test/test <prefix_of_name_of_test>.. [<prefix_of_name_of_test2>..] ... + ./src/test/test :<name_of_excluded_test> [:<name_of_excluded_test2]... + +To run all tests, including those based on Stem or Chutney: + + make test-full + +To run all tests, including those based on Stem or Chutney that require a +working connection to the internet: + + make test-full-online + +Running gcov for unit test coverage +----------------------------------- + + ./configure --enable-coverage + make + make check + # or--- make test-full ? make test-full-online? + mkdir coverage-output + ./scripts/test/coverage coverage-output + +(On OSX, you'll need to start with `--enable-coverage CC=clang`.) + +Then, look at the .gcov files in `coverage-output`. '-' before a line means +that the compiler generated no code for that line. '######' means that the +line was never reached. Lines with numbers were called that number of times. + +If that doesn't work: + + * Try configuring Tor with `--disable-gcc-hardening` + * You might need to run `make clean` after you run `./configure`. + +If you make changes to Tor and want to get another set of coverage results, +you can run `make reset-gcov` to clear the intermediary gcov output. + +If you have two different `coverage-output` directories, and you want to see +a meaningful diff between them, you can run: + + ./scripts/test/cov-diff coverage-output1 coverage-output2 | less + +In this diff, any lines that were visited at least once will have coverage +"1". This lets you inspect what you (probably) really want to know: which +untested lines were changed? Are there any new untested lines? + +Running integration tests +------------------------- + +We have the beginnings of a set of scripts to run integration tests using +Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and +run `make test-network`. + +We also have scripts to run integration tests using Stem. To try them, set +`STEM_SOURCE_DIR` to your Stem source directory, and run `test-stem`. + +Profiling Tor with oprofile +--------------------------- + +The oprofile tool runs (on Linux only!) to tell you what functions Tor is +spending its CPU time in, so we can identify performance bottlenecks. + +Here are some basic instructions + + - Build tor with debugging symbols (you probably already have, unless + you messed with CFLAGS during the build process). + - Build all the libraries you care about with debugging symbols + (probably you only care about libssl, maybe zlib and Libevent). + - Copy this tor to a new directory + - Copy all the libraries it uses to that dir too (`ldd ./tor` will + tell you) + - Set LD_LIBRARY_PATH to include that dir. `ldd ./tor` should now + show you it's using the libs in that dir + - Run that tor + - Reset oprofiles counters/start it + * `opcontrol --reset; opcontrol --start`, if Nick remembers right. + - After a while, have it dump the stats on tor and all the libs + in that dir you created. + * `opcontrol --dump;` + * `opreport -l that_dir/*` + - Profit + +Generating and analyzing a callgraph +------------------------------------ + +1. Run `./scripts/maint/generate_callgraph.sh`. This will generate a + bunch of files in a new ./callgraph directory. + +2. Run `./scripts/maint/analyze_callgraph.py callgraph/src/*/*`. This + will do a lot of graph operations and then dump out a new + `callgraph.pkl` file, containing data in Python's 'pickle' format. + +3. Run `./scripts/maint/display_callgraph.py`. It will display: + - the number of functions reachable from each function. + - all strongly-connnected components in the Tor callgraph + - the largest bottlenecks in the largest SCC in the Tor callgraph. + +Note that currently the callgraph generator can't detect calls that pass +through function pointers. + +Getting emacs to edit Tor source properly +----------------------------------------- + +Nick likes to put the following snippet in his .emacs file: + + + (add-hook 'c-mode-hook + (lambda () + (font-lock-mode 1) + (set-variable 'show-trailing-whitespace t) + + (let ((fname (expand-file-name (buffer-file-name)))) + (cond + ((string-match "^/home/nickm/src/libevent" fname) + (set-variable 'indent-tabs-mode t) + (set-variable 'c-basic-offset 4) + (set-variable 'tab-width 4)) + ((string-match "^/home/nickm/src/tor" fname) + (set-variable 'indent-tabs-mode nil) + (set-variable 'c-basic-offset 2)) + ((string-match "^/home/nickm/src/openssl" fname) + (set-variable 'indent-tabs-mode t) + (set-variable 'c-basic-offset 8) + (set-variable 'tab-width 8)) + )))) + + +You'll note that it defaults to showing all trailing whitespace. The `cond` +test detects whether the file is one of a few C free software projects that I +often edit, and sets up the indentation level and tab preferences to match +what they want. + +If you want to try this out, you'll need to change the filename regex +patterns to match where you keep your Tor files. + +If you use emacs for editing Tor and nothing else, you could always just say: + + + (add-hook 'c-mode-hook + (lambda () + (font-lock-mode 1) + (set-variable 'show-trailing-whitespace t) + (set-variable 'indent-tabs-mode nil) + (set-variable 'c-basic-offset 2))) + + +There is probably a better way to do this. No, we are probably not going +to clutter the files with emacs stuff. + + +Doxygen +------- + +We use the 'doxygen' utility to generate documentation from our +source code. Here's how to use it: + + 1. Begin every file that should be documented with + + /** + * \file filename.c + * \brief Short description of the file. + */ + + (Doxygen will recognize any comment beginning with /** as special.) + + 2. Before any function, structure, #define, or variable you want to + document, add a comment of the form: + + /** Describe the function's actions in imperative sentences. + * + * Use blank lines for paragraph breaks + * - and + * - hyphens + * - for + * - lists. + * + * Write <b>argument_names</b> in boldface. + * + * \code + * place_example_code(); + * between_code_and_endcode_commands(); + * \endcode + */ + + 3. Make sure to escape the characters `<`, `>`, `\`, `%` and `#` as `\<`, + `\>`, `\\`, `\%` and `\#`. + + 4. To document structure members, you can use two forms: + + struct foo { + /** You can put the comment before an element; */ + int a; + int b; /**< Or use the less-than symbol to put the comment + * after the element. */ + }; + + 5. To generate documentation from the Tor source code, type: + + $ doxygen -g + + to generate a file called `Doxyfile`. Edit that file and run + `doxygen` to generate the API documentation. + + 6. See the Doxygen manual for more information; this summary just + scratches the surface. diff --git a/doc/HACKING/HowToReview.md b/doc/HACKING/HowToReview.md new file mode 100644 index 0000000000..de7891c923 --- /dev/null +++ b/doc/HACKING/HowToReview.md @@ -0,0 +1,85 @@ +How to review a patch +===================== + +Some folks have said that they'd like to review patches more often, but they +don't know how. + +So, here are a bunch of things to check for when reviewing a patch! + +Note that if you can't do every one of these, that doesn't mean you can't do +a good review! Just make it clear what you checked for and what you didn't. + + +Top-level smell-checks +---------------------- + +(Difficulty: easy) + +- Does it compile with `--enable-gcc-warnings`? + +- Does `make check-spaces` pass? + +- Does it have a reasonable amount of tests? Do they pass? Do they leak + memory? + +- Do all the new functions, global variables, types, and structure members have + documentation? + +- Do all the functions, global variables, types, and structure members with + modified behavior have modified documentation? + +- Do all the new torrc options have documentation? + +- If this changes Tor's behavior on the wire, is there a design proposal? + + + +Let's look at the code! +----------------------- + +- Does the code conform to CodingStandards.txt? + +- Does the code leak memory? + +- If two or more pointers ever point to the same object, is it clear which + pointer "owns" the object? + +- Are all allocated resources freed? + +- Are all pointers that should be const, const? + +- Are `#defines` used for 'magic' numbers? + +- Can you understand what the code is trying to do? + +- Can you convince yourself that the code really does that? + +- Is there duplicated code that could be turned into a function? + + +Let's look at the documentation! +-------------------------------- + +- Does the documentation confirm to CodingStandards.txt? + +- Does it make sense? + +- Can you predict what the function will do from its documentation? + + +Let's think about security! +--------------------------- + +- If there are any arrays, buffers, are you 100% sure that they cannot + overflow? + +- If there is any integer math, can it overflow or underflow? + +- If there are any allocations, are you sure there are corresponding + deallocations? + +- Is there a safer pattern that could be used in any case? + +- Have they used one of the Forbidden Functions? + +(Also see your favorite secure C programming guides.) diff --git a/doc/HACKING/README.1st.md b/doc/HACKING/README.1st.md new file mode 100644 index 0000000000..8299fe634a --- /dev/null +++ b/doc/HACKING/README.1st.md @@ -0,0 +1,62 @@ + +In this directory +----------------- + +This directory has helpful information about what you need to know to +hack on Tor! + +First, read `GettingStarted.md` to learn how to get a start in Tor +development. + +If you've decided to write a patch, `CodingStandards.txt` will give +you a bunch of information about how we structure our code. + +It's important to get code right! Reading `WritingTests.md` will +tell you how to write and run tests in the Tor codebase. + +There are a bunch of other programs we use to help maintain and +develop the codebase: `HelpfulTools.md` can tell you how to use them +with Tor. + +If it's your job to put out Tor releases, see `ReleasingTor.md` so +that you don't miss any steps! + + +----------------------- + +For full information on how Tor is supposed to work, look at the files in +`https://gitweb.torproject.org/torspec.git/tree`. + +For an explanation of how to change Tor's design to work differently, look at +`https://gitweb.torproject.org/torspec.git/blob_plain/HEAD:/proposals/001-process.txt`. + +For the latest version of the code, get a copy of git, and + + git clone https://git.torproject.org/git/tor + +We talk about Tor on the `tor-talk` mailing list. Design proposals and +discussion belong on the `tor-dev` mailing list. We hang around on +irc.oftc.net, with general discussion happening on #tor and development +happening on `#tor-dev`. + +The other files in this `HACKING` directory may also be useful as you +get started working with Tor. + +Happy hacking! + + +----------------------- + +XXXXX also describe + +doc/HACKING/WritingTests.md + +torguts.git + +torspec.git + +The design paper + +freehaven.net/anonbib + +XXXX describe these and add links. diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md new file mode 100644 index 0000000000..2378aef568 --- /dev/null +++ b/doc/HACKING/ReleasingTor.md @@ -0,0 +1,143 @@ + +Putting out a new release +------------------------- + +Here are the steps Roger takes when putting out a new Tor release: + +1. Use it for a while, as a client, as a relay, as a hidden service, + and as a directory authority. See if it has any obvious bugs, and + resolve those. + + As applicable, merge the `maint-X` branch into the `release-X` branch. + +2. Gather the `changes/*` files into a changelog entry, rewriting many + of them and reordering to focus on what users and funders would find + interesting and understandable. + + 1. Make sure that everything that wants a bug number has one. + Make sure that everything which is a bugfix says what version + it was a bugfix on. + + 2. Concatenate them. + + 3. Sort them by section. Within each section, sort by "version it's + a bugfix on", else by numerical ticket order. + + 4. Clean them up: + + Standard idioms: + `Fixes bug 9999; bugfix on 0.3.3.3-alpha.` + + One space after a period. + + Make stuff very terse + + Make sure each section name ends with a colon + + Describe the user-visible problem right away + + Mention relevant config options by name. If they're rare or unusual, + remind people what they're for + + Avoid starting lines with open-paren + + Present and imperative tense: not past. + + 'Relays', not 'servers' or 'nodes' or 'Tor relays'. + + "Stop FOOing", not "Fix a bug where we would FOO". + + Try not to let any given section be longer than about a page. Break up + long sections into subsections by some sort of common subtopic. This + guideline is especially important when organizing Release Notes for + new stable releases. + + If a given changes stanza showed up in a different release (e.g. + maint-0.2.1), be sure to make the stanzas identical (so people can + distinguish if these are the same change). + + 5. Merge them in. + + 6. Clean everything one last time. + + 7. Run `./scripts/maint/format_changelog.py` to make it prettier. + +3. Compose a short release blurb to highlight the user-facing + changes. Insert said release blurb into the ChangeLog stanza. If it's + a stable release, add it to the ReleaseNotes file too. If we're adding + to a release-0.2.x branch, manually commit the changelogs to the later + git branches too. + + If you're doing the first stable release in a series, you need to + create a ReleaseNotes for the series as a whole. To get started + there, copy all of the Changelog entries from the series into a new + file, and run `./scripts/maint/sortChanges.py` on it. That will + group them by category. Then kill every bugfix entry for fixing + bugs that were introduced within that release series; those aren't + relevant changes since the last series. At that point, it's time + to start sorting and condensing entries. (Generally, we don't edit the + text of existing entries, though.) + +4. In `maint-0.2.x`, bump the version number in `configure.ac` and run + `scripts/maint/updateVersions.pl` to update version numbers in other + places, and commit. Then merge `maint-0.2.x` into `release-0.2.x`. + + (NOTE: To bump the version number, edit `configure.ac`, and then run + either `make`, or `perl scripts/maint/updateVersions.pl`, depending on + your version.) + +5. Make distcheck, put the tarball up somewhere, and tell `#tor` about + it. Wait a while to see if anybody has problems building it. Try to + get Sebastian or somebody to try building it on Windows. + +6. Get at least two of weasel/arma/Sebastian to put the new version number + in their approved versions list. + +7. Sign the tarball, then sign and push the git tag: + + gpg -ba <the_tarball> + git tag -u <keyid> tor-0.2.x.y-status + git push origin tag tor-0.2.x.y-status + +8. scp the tarball and its sig to the dist website, i.e. + `/srv/dist-master.torproject.org/htdocs/` on dist-master. When you want + it to go live, you run "static-update-component dist.torproject.org" + on dist-master. + + Edit `include/versions.wmi` and `Makefile` to note the new version. + + (NOTE: Due to #17805, there can only be one stable version listed at + once. Nonetheless, do not call your version "alpha" if it is stable, + or people will get confused.) + +9. Email the packagers (cc'ing tor-assistants) that a new tarball is up. + The current list of packagers is: + + - {weasel,gk,mikeperry} at torproject dot org + - {blueness} at gentoo dot org + - {paul} at invizbox dot io + - {ondrej.mikle} at gmail dot com + - {lfleischer} at archlinux dot org + - {tails-dev} at boum dot org + +10. Add the version number to Trac. To do this, go to Trac, log in, + select "Admin" near the top of the screen, then select "Versions" from + the menu on the left. At the right, there will be an "Add version" + box. By convention, we enter the version in the form "Tor: + 0.2.2.23-alpha" (or whatever the version is), and we select the date as + the date in the ChangeLog. + +11. Forward-port the ChangeLog (and ReleaseNotes if appropriate). + +12. Wait up to a day or two (for a development release), or until most + packages are up (for a stable release), and mail the release blurb and + changelog to tor-talk or tor-announce. + + (We might be moving to faster announcements, but don't announce until + the website is at least updated.) + +13. If it's a stable release, bump the version number in the `maint-x.y.z` + branch to "newversion-dev", and do a `merge -s ours` merge to avoid + taking that change into master. Do a similar `merge -s theirs` + merge to get the change (and only that change) into release. (Some + of the build scripts require that maint merge cleanly into release.) diff --git a/doc/HACKING/WritingTests.md b/doc/HACKING/WritingTests.md new file mode 100644 index 0000000000..4e98d3d645 --- /dev/null +++ b/doc/HACKING/WritingTests.md @@ -0,0 +1,445 @@ + +Writing tests for Tor: an incomplete guide +========================================== + +Tor uses a variety of testing frameworks and methodologies to try to +keep from introducing bugs. The major ones are: + + 1. Unit tests written in C and shipped with the Tor distribution. + + 2. Integration tests written in Python and shipped with the Tor + distribution. + + 3. Integration tests written in Python and shipped with the Stem + library. Some of these use the Tor controller protocol. + + 4. System tests written in Python and SH, and shipped with the + Chutney package. These work by running many instances of Tor + locally, and sending traffic through them. + + 5. The Shadow network simulator. + +How to run these tests +---------------------- + +### The easy version + +To run all the tests that come bundled with Tor, run `make check`. + +To run the Stem tests as well, fetch stem from the git repository, +set `STEM_SOURCE_DIR` to the checkout, and run `make test-stem`. + +To run the Chutney tests as well, fetch chutney from the git repository, +set `CHUTNEY_PATH` to the checkout, and run `make test-network`. + +To run all of the above, run `make test-full`. + +To run all of the above, plus tests that require a working connection to the +internet, run `make test-full-online`. + +### Running particular subtests + +The Tor unit tests are divided into separate programs and a couple of +bundled unit test programs. + +Separate programs are easy. For example, to run the memwipe tests in +isolation, you just run `./src/test/test-memwipe`. + +To run tests within the unit test programs, you can specify the name +of the test. The string ".." can be used as a wildcard at the end of the +test name. For example, to run all the cell format tests, enter +`./src/test/test cellfmt/..`. To run + +Many tests that need to mess with global state run in forked subprocesses in +order to keep from contaminating one another. But when debugging a failing test, +you might want to run it without forking a subprocess. To do so, use the +`--no-fork` option with a single test. (If you specify it along with +multiple tests, they might interfere.) + +You can turn on logging in the unit tests by passing one of `--debug`, +`--info`, `--notice`, or `--warn`. By default only errors are displayed. + +Unit tests are divided into `./src/test/test` and `./src/test/test-slow`. +The former are those that should finish in a few seconds; the latter tend to +take more time, and may include CPU-intensive operations, deliberate delays, +and stuff like that. + +### Finding test coverage + +Test coverage is a measurement of which lines your tests actually visit. + +When you configure Tor with the `--enable-coverage` option, it should +build with support for coverage in the unit tests, and in a special +`tor-cov` binary. + +Then, run the tests you'd like to see coverage from. If you have old +coverage output, you may need to run `reset-gcov` first. + +Now you've got a bunch of files scattered around your build directories +called `*.gcda`. In order to extract the coverage output from them, make a +temporary directory for them and run `./scripts/test/coverage ${TMPDIR}`, +where `${TMPDIR}` is the temporary directory you made. This will create a +`.gcov` file for each source file under tests, containing that file's source +annotated with the number of times the tests hit each line. (You'll need to +have gcov installed.) + +You can get a summary of the test coverage for each file by running +`./scripts/test/cov-display ${TMPDIR}/*` . Each line lists the file's name, +the number of uncovered lines, the number of uncovered lines, and the +coverage percentage. + +For a summary of the test coverage for each _function_, run +`./scripts/test/cov-display -f ${TMPDIR}/*`. + +### Comparing test coverage + +Sometimes it's useful to compare test coverage for a branch you're writing to +coverage from another branch (such as git master, for example). But you +can't run `diff` on the two coverage outputs directly, since the actual +number of times each line is executed aren't so important, and aren't wholly +deterministic. + +Instead, follow the instructions above for each branch, creating a separate +temporary directory for each. Then, run `./scripts/test/cov-diff ${D1} +${D2}`, where D1 and D2 are the directories you want to compare. This will +produce a diff of the two directories, with all lines normalized to be either +covered or uncovered. + +To count new or modified uncovered lines in D2, you can run: + + ./scripts/test/cov-diff ${D1} ${D2}" | grep '^+ *\#' | wc -l + + +What kinds of test should I write? +---------------------------------- + +Integration testing and unit testing are complementary: it's probably a +good idea to make sure that your code is hit by both if you can. + +If your code is very-low level, and its behavior is easily described in +terms of a relation between inputs and outputs, or a set of state +transitions, then it's a natural fit for unit tests. (If not, please +consider refactoring it until most of it _is_ a good fit for unit +tests!) + +If your code adds new externally visible functionality to Tor, it would +be great to have a test for that functionality. That's where +integration tests more usually come in. + +Unit and regression tests: Does this function do what it's supposed to? +----------------------------------------------------------------------- + +Most of Tor's unit tests are made using the "tinytest" testing framework. +You can see a guide to using it in the tinytest manual at + + https://github.com/nmathewson/tinytest/blob/master/tinytest-manual.md + +To add a new test of this kind, either edit an existing C file in `src/test/`, +or create a new C file there. Each test is a single function that must +be indexed in the table at the end of the file. We use the label "done:" as +a cleanup point for all test functions. + +(Make sure you read `tinytest-manual.md` before proceeding.) + +I use the term "unit test" and "regression tests" very sloppily here. + +### A simple example + +Here's an example of a test function for a simple function in util.c: + + static void + test_util_writepid(void *arg) + { + (void) arg; + + char *contents = NULL; + const char *fname = get_fname("tmp_pid"); + unsigned long pid; + char c; + + write_pidfile(fname); + + contents = read_file_to_str(fname, 0, NULL); + tt_assert(contents); + + int n = sscanf(contents, "%lu\n%c", &pid, &c); + tt_int_op(n, OP_EQ, 1); + tt_int_op(pid, OP_EQ, getpid()); + + done: + tor_free(contents); + } + +This should look pretty familiar to you if you've read the tinytest +manual. One thing to note here is that we use the testing-specific +function `get_fname` to generate a file with respect to a temporary +directory that the tests use. You don't need to delete the file; +it will get removed when the tests are done. + +Also note our use of `OP_EQ` instead of `==` in the `tt_int_op()` calls. +We define `OP_*` macros to use instead of the binary comparison +operators so that analysis tools can more easily parse our code. +(Coccinelle really hates to see `==` used as a macro argument.) + +Finally, remember that by convention, all `*_free()` functions that +Tor defines are defined to accept NULL harmlessly. Thus, you don't +need to say `if (contents)` in the cleanup block. + +### Exposing static functions for testing + +Sometimes you need to test a function, but you don't want to expose +it outside its usual module. + +To support this, Tor's build system compiles a testing version of +each module, with extra identifiers exposed. If you want to +declare a function as static but available for testing, use the +macro `STATIC` instead of `static`. Then, make sure there's a +macro-protected declaration of the function in the module's header. + +For example, `crypto_curve25519.h` contains: + + #ifdef CRYPTO_CURVE25519_PRIVATE + STATIC int curve25519_impl(uint8_t *output, const uint8_t *secret, + const uint8_t *basepoint); + #endif + +The `crypto_curve25519.c` file and the `test_crypto.c` file both define +`CRYPTO_CURVE25519_PRIVATE`, so they can see this declaration. + +### STOP! Does this test really test? + +When writing tests, it's not enough to just generate coverage on all the +lines of the code that you're testing: It's important to make sure that +the test _really tests_ the code. + +For example, here is a _bad_ test for the unlink() function (which is +supposed to remove a file). + + static void + test_unlink_badly(void *arg) + { + (void) arg; + int r; + + const char *fname = get_fname("tmpfile"); + + /* If the file isn't there, unlink returns -1 and sets ENOENT */ + r = unlink(fname); + tt_int_op(n, OP_EQ, -1); + tt_int_op(errno, OP_EQ, ENOENT); + + /* If the file DOES exist, unlink returns 0. */ + write_str_to_file(fname, "hello world", 0); + r = unlink(fnme); + tt_int_op(r, OP_EQ, 0); + + done: + tor_free(contents); + } + + +This test might get very high coverage on unlink(). So why is it a +bad test? Because it doesn't check that unlink() *actually removes the +named file*! + +Remember, the purpose of a test is to succeed if the code does what +it's supposed to do, and fail otherwise. Try to design your tests so +that they check for the code's intended and documented functionality +as much as possible. + + +### Mock functions for testing in isolation + +Often we want to test that a function works right, but the function to +be tested depends on other functions whose behavior is hard to observe, +or which require a working Tor network, or something like that. + +To write tests for this case, you can replace the underlying functions +with testing stubs while your unit test is running. You need to declare +the underlying function as 'mockable', as follows: + + MOCK_DECL(returntype, functionname, (argument list)); + +and then later implement it as: + + MOCK_IMPL(returntype, functionname, (argument list)) + { + /* implementation here */ + } + +For example, if you had a 'connect to remote server' function, you could +declare it as: + + + MOCK_DECL(int, connect_to_remote, (const char *name, status_t *status)); + +When you declare a function this way, it will be declared as normal in +regular builds, but when the module is built for testing, it is declared +as a function pointer initialized to the actual implementation. + +In your tests, if you want to override the function with a temporary +replacement, you say: + + MOCK(functionname, replacement_function_name); + +And later, you can restore the original function with: + + UNMOCK(functionname); + +For more information, see the definitions of this mocking logic in +`testsupport.h`. + +### Okay but what should my tests actually do? + +We talk above about "test coverage" -- making sure that your tests visit +every line of code, or every branch of code. But visiting the code isn't +enough: we want to verify that it's correct. + +So when writing tests, try to make tests that should pass with any correct +implementation of the code, and that should fail if the code doesn't do what +it's supposed to do. + +You can write "black-box" tests or "glass-box" tests. A black-box test is +one that you write without looking at the structure of the function. A +glass-box one is one you implement while looking at how the function is +implemented. + +In either case, make sure to consider common cases *and* edge cases; success +cases and failure csaes. + +For example, consider testing this function: + + /** Remove all elements E from sl such that E==element. Preserve + * the order of any elements before E, but elements after E can be + * rearranged. + */ + void smartlist_remove(smartlist_t *sl, const void *element); + +In order to test it well, you should write tests for at least all of the +following cases. (These would be black-box tests, since we're only looking +at the declared behavior for the function: + + * Remove an element that is in the smartlist. + * Remove an element that is not in the smartlist. + * Remove an element that appears in the smartlist more than once. + +And your tests should verify that it behaves correct. At minimum, you should +test: + + * That other elements before E are in the same order after you call the + functions. + * That the target element is really removed. + * That _only_ the target element is removed. + +When you consider edge cases, you might try: + + * Remove an element from an empty list. + * Remove an element from a singleton list containing that element. + * Remove an element for a list containing several instances of that + element, and nothing else. + +Now let's look at the implementation: + + void + smartlist_remove(smartlist_t *sl, const void *element) + { + int i; + if (element == NULL) + return; + for (i=0; i < sl->num_used; i++) + if (sl->list[i] == element) { + sl->list[i] = sl->list[--sl->num_used]; /* swap with the end */ + i--; /* so we process the new i'th element */ + sl->list[sl->num_used] = NULL; + } + } + +Based on the implementation, we now see three more edge cases to test: + + * Removing NULL from the list. + * Removing an element from the end of the list + * Removing an element from a position other than the end of the list. + + +### What should my tests NOT do? + +Tests shouldn't require a network connection. + +Whenever possible, tests shouldn't take more than a second. Put the test +into test/slow if it genuinely needs to be run. + +Tests should not alter global state unless they run with `TT_FORK`: Tests +should not require other tests to be run before or after them. + +Tests should not leak memory or other resources. To find out if your tests +are leaking memory, run them under valgrind (see HelpfulTools.txt for more +information on how to do that). + +When possible, tests should not be over-fit to the implementation. That is, +the test should verify that the documented behavior is implemented, but +should not break if other permissible behavior is later implemented. + + +### Advanced techniques: Namespaces + +Sometimes, when you're doing a lot of mocking at once, it's convenient to +isolate your identifiers within a single namespace. If this were C++, we'd +already have namespaces, but for C, we do the best we can with macros and +token-pasting. + +We have some macros defined for this purpose in `src/test/test.h`. To use +them, you define `NS_MODULE` to a prefix to be used for your identifiers, and +then use other macros in place of identifier names. See `src/test/test.h` for +more documentation. + + +Integration tests: Calling Tor from the outside +----------------------------------------------- + +Some tests need to invoke Tor from the outside, and shouldn't run from the +same process as the Tor test program. Reasons for doing this might include: + + * Testing the actual behavior of Tor when run from the command line + * Testing that a crash-handler correctly logs a stack trace + * Verifying that violating a sandbox or capability requirement will + actually crash the program. + * Needing to run as root in order to test capability inheritance or + user switching. + +To add one of these, you generally want a new C program in `src/test`. Add it +to `TESTS` and `noinst_PROGRAMS` if it can run on its own and return success or +failure. If it needs to be invoked multiple times, or it needs to be +wrapped, add a new shell script to `TESTS`, and the new program to +`noinst_PROGRAMS`. If you need access to any environment variable from the +makefile (eg `${PYTHON}` for a python interpreter), then make sure that the +makefile exports them. + +Writing integration tests with Stem +----------------------------------- + +The 'stem' library includes extensive unit tests for the Tor controller +protocol. + +For more information on writing new tests for stem, have a look around +the `test/*` directory in stem, and find a good example to emulate. You +might want to start with +`https://gitweb.torproject.org/stem.git/tree/test/integ/control/controller.py` +to improve Tor's test coverage. + +You can run stem tests from tor with `make test-stem`, or see +`https://stem.torproject.org/faq.html#how-do-i-run-the-tests`. + +System testing with Chutney +--------------------------- + +The 'chutney' program configures and launches a set of Tor relays, +authorities, and clients on your local host. It has a `test network` +functionality to send traffic through them and verify that the traffic +arrives correctly. + +You can write new test networks by adding them to `networks`. To add +them to Tor's tests, add them to the `test-network` or `test-network-all` +targets in `Makefile.am`. + +(Adding new kinds of program to chutney will still require hacking the +code.) diff --git a/doc/TUNING b/doc/TUNING new file mode 100644 index 0000000000..24552a38cb --- /dev/null +++ b/doc/TUNING @@ -0,0 +1,86 @@ +Most operating systems limit an amount of TCP sockets that can be used +simultaneously. It is possible for a busy Tor relay to run into these +limits, thus being unable to fully utilize the bandwidth resources it +has at its disposal. Following system-specific tips might be helpful +to alleviate the aforementioned problem. + +Linux +----- + +Use 'ulimit -n' to raise an allowed number of file descriptors to be +opened on your host at the same time. + +FreeBSD +------- + +Tune the followind sysctl(8) variables: + * kern.maxfiles - maximum allowed file descriptors (for entire system) + * kern.maxfilesperproc - maximum file descriptors one process is allowed + to use + * kern.ipc.maxsockets - overall maximum numbers of sockets for entire + system + * kern.ipc.somaxconn - size of listen queue for incoming TCP connections + for entire system + +See also: + * https://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html + * https://wiki.freebsd.org/NetworkPerformanceTuning + +Mac OS X +-------- + +Since Mac OS X is BSD-based system, most of the above hold for OS X as well. +However, launchd(8) is known to modify kern.maxfiles and kern.maxfilesperproc +when it launches tor service (see launchd.plist(5) manpage). Also, +kern.ipc.maxsockets is determined dynamically by the system and thus is +read-only on OS X. + +OpenBSD +------- + +Because OpenBSD is primarily focused on security and stability, it uses default +resource limits stricter than those of more popular Unix-like operating systems. + +OpenBSD stores a kernel-level file descriptor limit in the sysctl variable +kern.maxfiles. It defaults to 7,030. To change it to, for example, 16,000 while +the system is running, use the command 'sudo sysctl kern.maxfiles=16000'. +kern.maxfiles will reset to the default value upon system reboot unless you also +add 'kern.maxfiles=16000' to the file /etc/sysctl.conf. + +There are stricter resource limits set on user classes, which are stored in +/etc/login.conf. This config file also allows limit sets for daemons started +with scripts in the /etc/rc.d directory, which presumably includes Tor. + +To increase the file descriptor limit from its default of 1,024, add the +following to /etc/login.conf: + +tor:\ + :openfiles-max=13500:\ + :tc=daemon: + +Upon restarting Tor, it will be able to open up to 13,500 file descriptors. + +This will work *only* if you are starting Tor with the script /etc/rc.d/tor. If +you're using a custom build instead of the package, you can easily copy the rc.d +script from the Tor port directory. Alternatively, you can ensure that the Tor's +daemon user has its own user class and make a /etc/login.conf entry for it. + +High-bandwidth relays sometimes give the syslog warning: + +/bsd: WARNING: mclpools limit reached; increase kern.maxclusters + +In this case, increase kern.maxclusters with the sysctl command and in the file +/etc/sysctl.conf, as described with kern.maxfiles above. Use 'sysctl +kern.maxclusters' to query the current value. Increasing by about 15% per day +until the error no longer appears is a good guideline. + +Disclaimer +---------- + +Do note that this document is a draft and above information may be +technically incorrect and/or incomplete. If so, please open a ticket +on https://trac.torproject.org or post to tor-relays mailing list. + +Are you running a busy Tor relay? Let us know how you are solving +the out-of-sockets problem on your system. + diff --git a/doc/asciidoc-helper.sh b/doc/asciidoc-helper.sh index c06b57026b..a3ef53f884 100755 --- a/doc/asciidoc-helper.sh +++ b/doc/asciidoc-helper.sh @@ -19,7 +19,7 @@ if [ "$1" = "html" ]; then base=${output%%.html.in} if [ "$2" != none ]; then - "$2" -d manpage -o $output $input; + TZ=UTC "$2" -d manpage -o $output $input; else echo "=================================="; echo; diff --git a/doc/building-tor-msvc.txt b/doc/building-tor-msvc.txt new file mode 100644 index 0000000000..3d3eced8af --- /dev/null +++ b/doc/building-tor-msvc.txt @@ -0,0 +1,122 @@ +Building Tor with MSVC.
+=======================
+
+NOTE: This is not the preferred method for building Tor on windows: we use
+mingw for our packages.
+
+Last updated 9 September 2014.
+
+
+Requirements:
+-------------
+
+ * Visual Studio 2010
+ http://go.microsoft.com/fwlink/?LinkId=323467
+ * CMake 2.8.12.2
+ http://www.cmake.org/download/
+ * Perl 5.16
+ http://www.activestate.com/activeperl/downloads
+ * Latest stable OpenSSL tarball
+ https://www.openssl.org/source/
+ * Latest stable zlib tarball
+ http://zlib.net/
+ * Latest stable libevent Libevent tarball
+ https://github.com/libevent/libevent/releases
+
+Make sure you check signatures for all these packages.
+
+Steps:
+------
+
+Building OpenSSL from source as a shared library:
+
+ cd <openssl source dir>
+ perl Configure VC-WIN32
+ perl util\mkfiles.pl >MINFO
+ perl util\mk1mf.pl no-asm dll VC-WIN32 >32dll.mak
+ perl util\mkdef.pl 32 libeay > ms\libeay32.def
+ perl util\mkdef.pl 32 ssleay > ms\ssleay32.def
+ nmake -f 32dll.mak
+
+Making OpenSSL final package:
+
+ Create <openssl final package dir>, I'd recommend using a name like <openssl
+ source dir>-vc10.
+
+ Copy the following directories and files to their respective locations
+ <openssl source dir>\inc32\openssl => <openssl final package dir>\include\openssl
+ <openssl source dir>\out32dll\libeay32.lib => <openssl final package dir>\lib\libeay32.lib
+ <openssl source dir>\out32dll\ssleay32.lib => <openssl final package dir>\lib\ssleay32.lib
+ <openssl source dir>\out32dll\libeay32.dll => <openssl final package dir>\bin\libeay32.dll
+ <openssl source dir>\out32dll\openssl.exe => <openssl final package dir>\bin\openssl.exe
+ <openssl source dir>\out32dll\ssleay32.dll => <openssl final package dir>\bin\ssleay32.dll
+
+Building Zlib from source:
+
+ cd <zlib source dir>
+ nmake -f win32/Makefile.msc
+
+Building libevent:
+
+ cd <libevent source dir>
+ mkdir build && cd build
+ SET OPENSSL_ROOT_DIR=<openssl final package dir>
+ cmake -G "NMake Makefiles" .. -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo -DCMAKE_C_FLAGS_RELWITHDEBINFO:STRING="/MT /Zi /O2 /Ob1 /D NDEBUG" -DZLIB_LIBRARY:FILEPATH="<zlib source dir>\zdll.lib" -DZLIB_INCLUDE_DIR:PATH="<zlib source dir>"
+ nmake event
+
+Building Tor:
+
+ Create a dir above tor source dir named build-alpha and two subdirs include
+ and lib.
+
+ Your build tree should now be similar to this one:
+ * build-alpha
+ - include
+ - lib
+ * <libevent source dir>
+ - build
+ - cmake
+ - ...
+ * <openssl source dir>
+ - ...
+ - ms
+ - util
+ - ...
+ * <openssl final package dir>
+ - bin
+ - include
+ - lib
+ * <tor source dir>
+ - ...
+ - src
+ - ...
+ * <zlib source dir>
+ - ...
+ - win32
+ - ...
+
+ Copy the following dirs and files to the following locations:
+ <openssl final package dir>\include\openssl => build-alpha\include\openssl
+ <libevent source dir>\include => build-alpha\include
+ <libevent source dir>\WIN32-Code\nmake\event2 => build-alpha\include\event2
+ <zlib source dir>\z*.h => build-alpha\include\z*.h
+
+ Now copy the following files to the following locations and rename them
+ according new names:
+
+ <libevent source dir>\build\lib\event.lib => build-alpha\lib\libevent.lib
+ <openssl final package dir>\lib\libeay32.lib => build-alpha\lib\libcrypto.lib
+ <openssl final package dir>\lib\ssleay32.lib => build-alpha\lib\libssl.lib
+ <zlib source dir>\zdll.lib => build-alpha\lib\libz.lib
+
+ And we are now ready for the build process:
+
+ cd <tor source dir>
+ nmake -f Makefile.nmake
+
+ After the above process is completed there should be a tor.exe in <tor
+ source dir>\src\or
+
+ Copy tor.exe to desired location and also copy zlib1.dll, libeay32.dll and
+ ssleay32.dll from built zlib and openssl packages
+
diff --git a/doc/contrib/tor-rpm-creation.txt b/doc/contrib/tor-rpm-creation.txt index a03891e2b9..9c4e05764e 100644 --- a/doc/contrib/tor-rpm-creation.txt +++ b/doc/contrib/tor-rpm-creation.txt @@ -42,7 +42,7 @@ Here's a workaround: Before even building the source RPM, install fedora-packager and instruct the build system to use rpmbuild-md5 like this: -yum install fedora-packager +dnf install fedora-packager export RPMBUILD=rpmbuild-md5 Then proceed as usual to create the source RPM and binary RPMs: diff --git a/doc/include.am b/doc/include.am index 30d3e20d83..7164a4b2a0 100644 --- a/doc/include.am +++ b/doc/include.am @@ -13,7 +13,7 @@ # just use the .1 and .html files. base_mans = doc/tor doc/tor-gencert doc/tor-resolve doc/torify -all_mans = $(base_mans) doc/tor-fw-helper +all_mans = $(base_mans) if USE_FW_HELPER install_mans = $(all_mans) else @@ -34,9 +34,18 @@ nodist_man1_MANS = doc_DATA = endif -EXTRA_DIST+= doc/HACKING doc/asciidoc-helper.sh \ +EXTRA_DIST+= doc/asciidoc-helper.sh \ $(html_in) $(man_in) $(txt_in) \ - doc/state-contents.txt + doc/state-contents.txt \ + doc/torrc_format.txt \ + doc/TUNING \ + doc/HACKING/README.1st.md \ + doc/HACKING/CodingStandards.md \ + doc/HACKING/GettingStarted.md \ + doc/HACKING/HelpfulTools.md \ + doc/HACKING/HowToReview.md \ + doc/HACKING/ReleasingTor.md \ + doc/HACKING/WritingTests.md docdir = @docdir@ @@ -56,34 +65,30 @@ doc/tor.1.in: doc/tor.1.txt doc/torify.1.in: doc/torify.1.txt doc/tor-gencert.1.in: doc/tor-gencert.1.txt doc/tor-resolve.1.in: doc/tor-resolve.1.txt -doc/tor-fw-helper.1.in: doc/tor-fw-helper.1.txt doc/tor.html.in: doc/tor.1.txt doc/torify.html.in: doc/torify.1.txt doc/tor-gencert.html.in: doc/tor-gencert.1.txt doc/tor-resolve.html.in: doc/tor-resolve.1.txt -doc/tor-fw-helper.html.in: doc/tor-fw-helper.1.txt -# use ../config.status to swap all machine-specific magic strings +# use config.status to swap all machine-specific magic strings # in the asciidoc with their replacements. $(asciidoc_product) : $(AM_V_GEN)$(MKDIR_P) $(@D) $(AM_V_at)if test -e $(top_srcdir)/$@.in && ! test -e $@.in ; then \ cp $(top_srcdir)/$@.in $@; \ fi - $(AM_V_at)./config.status -q --file=$@; + $(AM_V_at)$(top_builddir)/config.status -q --file=$@; doc/tor.html: doc/tor.html.in doc/tor-gencert.html: doc/tor-gencert.html.in doc/tor-resolve.html: doc/tor-resolve.html.in doc/torify.html: doc/torify.html.in -doc/tor-fw-helper.html: doc/tor-fw-helper.html.in doc/tor.1: doc/tor.1.in doc/tor-gencert.1: doc/tor-gencert.1.in doc/tor-resolve.1: doc/tor-resolve.1.in doc/torify.1: doc/torify.1.in -doc/tor-fw-helper.1: doc/tor-fw-helper.1.in -CLEANFILES+= $(asciidoc_product) config.log +CLEANFILES+= $(asciidoc_product) DISTCLEANFILES+= $(html_in) $(man_in) diff --git a/doc/tor-fw-helper.1.txt b/doc/tor-fw-helper.1.txt deleted file mode 100644 index 1c103d9250..0000000000 --- a/doc/tor-fw-helper.1.txt +++ /dev/null @@ -1,60 +0,0 @@ -// Copyright (c) The Tor Project, Inc. -// See LICENSE for licensing information -// This is an asciidoc file used to generate the manpage/html reference. -// Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html -:man source: Tor -:man manual: Tor Manual -tor-fw-helper(1) -================ -Jacob Appelbaum - -NAME ----- -tor-fw-helper - Manage upstream firewall/NAT devices - -SYNOPSIS --------- -**tor-fw-helper** [-h|--help] [-T|--test-commandline] [-v|--verbose] [-g|--fetch-public-ip] - [-p __external port__:__internal_port__] - -DESCRIPTION ------------ -**tor-fw-helper** currently supports Apple's NAT-PMP protocol and the UPnP -standard for TCP port mapping. It is written as the reference implementation of -tor-fw-helper-spec.txt and conforms to that loose plugin API. If your network -supports either NAT-PMP or UPnP, tor-fw-helper will attempt to automatically -map the required TCP ports for Tor's Or and Dir ports. + - -OPTIONS -------- -**-h** or **--help**:: - Display help text and exit. - -**-v** or **--verbose**:: - Display verbose output. - -**-T** or **--test-commandline**:: - Display test information and print the test information in - tor-fw-helper.log - -**-g** or **--fetch-public-ip**:: - Fetch the the public ip address for each supported NAT helper method. - -**-p** or **--port** __external_port__:__internal_port__:: - Forward external_port to internal_port. This option can appear - more than once. - -BUGS ----- -This probably doesn't run on Windows. That's not a big issue, since we don't -really want to deal with Windows before October 2010 anyway. - -SEE ALSO --------- -**tor**(1) + - -See also the "tor-fw-helper-spec.txt" file, distributed with Tor. - -AUTHORS -------- - Jacob Appelbaum <jacob@torproject.org>, Steven J. Murdoch <Steven.Murdoch@cl.cam.ac.uk> diff --git a/doc/tor-resolve.1.txt b/doc/tor-resolve.1.txt index 341d302244..30e16d5daa 100644 --- a/doc/tor-resolve.1.txt +++ b/doc/tor-resolve.1.txt @@ -14,7 +14,7 @@ tor-resolve - resolve a hostname to an IP address via tor SYNOPSIS -------- -**tor-resolve** [-4|-5] [-v] [-x] __hostname__ [__sockshost__[:__socksport__]] +**tor-resolve** [-4|-5] [-v] [-x] [-p __socksport__] __hostname__ [__sockshost__[:__socksport__]] DESCRIPTION ----------- @@ -40,6 +40,9 @@ OPTIONS Use the SOCKS4a protocol rather than the default SOCKS5 protocol. Doesn't support reverse DNS. +**-p** __socksport__:: + Override the default SOCKS port without setting the hostname. + SEE ALSO -------- **tor**(1), **torify**(1). + diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 8d51f6e3c2..74915b7119 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -30,7 +30,7 @@ Users bounce their TCP streams -- web traffic, ftp, ssh, etc. -- around the network, and recipients, observers, and even the relays themselves have difficulty tracking the source of the stream. -By default, **tor** will only act as a client only. To help the network +By default, **tor** will act as a client only. To help the network by providing bandwidth as a relay, change the **ORPort** configuration option -- see below. Please also consult the documentation on the Tor Project's website. @@ -42,7 +42,8 @@ COMMAND-LINE OPTIONS [[opt-f]] **-f** __FILE__:: Specify a new configuration file to contain further Tor configuration - options. (Default: @CONFDIR@/torrc, or $HOME/.torrc if that file is not + options OR pass *-* to make Tor read its configuration from standard + input. (Default: @CONFDIR@/torrc, or $HOME/.torrc if that file is not found) [[opt-allow-missing-torrc]] **--allow-missing-torrc**:: @@ -72,7 +73,7 @@ COMMAND-LINE OPTIONS [[opt-serviceinstall]] **--service install** [**--options** __command-line options__]:: Install an instance of Tor as a Windows service, with the provided command-line options. Current instructions can be found at - https://trac.torproject.org/projects/tor/wiki/doc/TorFAQ#HowdoIrunmyTorrelayasanNTservice + https://www.torproject.org/docs/faq#NTService [[opt-service]] **--service** **remove**|**start**|**stop**:: Remove, start, or stop a configured Tor Windows service. @@ -94,11 +95,34 @@ COMMAND-LINE OPTIONS which tells Tor to only send warnings and errors to the console, or with the **--quiet** option, which tells Tor not to log to the console at all. +[[opt-keygen]] **--keygen** [**--newpass**]:: + Running "tor --keygen" creates a new ed25519 master identity key for a + relay, or only a fresh temporary signing key and certificate, if you + already have a master key. Optionally you can encrypt the master identity + key with a passphrase: Tor will ask you for one. If you don't want to + encrypt the master key, just don't enter any passphrase when asked. + + + + The **--newpass** option should be used with --keygen only when you need + to add, change, or remove a passphrase on an existing ed25519 master + identity key. You will be prompted for the old passphase (if any), + and the new passphrase (if any). + + + + When generating a master key, you will probably want to use + **--DataDirectory** to control where the keys + and certificates will be stored, and **--SigningKeyLifetime** to + control their lifetimes. Their behavior is as documented in the + server options section below. (You must have write access to the specified + DataDirectory.) + + + + To use the generated files, you must copy them to the DataDirectory/keys + directory of your Tor daemon, and make sure that they are owned by the + user actually running the Tor daemon on your system. + Other options can be specified on the command-line in the format "--option value", in the format "option value", or in a configuration file. For instance, you can tell Tor to start listening for SOCKS connections on port -9999 by passing --SOCKSPort 9999 or SOCKSPort 9999 to it on the command line, -or by putting "SOCKSPort 9999" in the configuration file. You will need to +9999 by passing --SocksPort 9999 or SocksPort 9999 to it on the command line, +or by putting "SocksPort 9999" in the configuration file. You will need to quote options with spaces in them: if you want Tor to log all debugging messages to debug.log, you will probably need to say --Log 'debug file debug.log'. @@ -124,26 +148,31 @@ the defaults file. This rule is simple for options that take a single value, but it can become complicated for options that are allowed to occur more than once: if you -specify four SOCKSPorts in your configuration file, and one more SOCKSPort on +specify four SocksPorts in your configuration file, and one more SocksPort on the command line, the option on the command line will replace __all__ of the -SOCKSPorts in the configuration file. If this isn't what you want, prefix -the option name with a plus sign, and it will be appended to the previous set -of options instead. +SocksPorts in the configuration file. If this isn't what you want, prefix +the option name with a plus sign (+), and it will be appended to the previous +set of options instead. For example, setting SocksPort 9100 will use only +port 9100, but setting +SocksPort 9100 will use ports 9100 and 9050 (because +this is the default). Alternatively, you might want to remove every instance of an option in the configuration file, and not replace it at all: you might want to say on the -command line that you want no SOCKSPorts at all. To do that, prefix the -option name with a forward slash. +command line that you want no SocksPorts at all. To do that, prefix the +option name with a forward slash (/). You can use the plus sign (+) and the +forward slash (/) in the configuration file and on the command line. GENERAL OPTIONS --------------- [[BandwidthRate]] **BandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: - A token bucket limits the average incoming bandwidth usage on this node to - the specified number of bytes per second, and the average outgoing + A token bucket limits the average incoming bandwidth usage on this node + to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. If you want to run a relay in the - public network, this needs to be _at the very least_ 30 KBytes (that is, - 30720 bytes). (Default: 1 GByte) + + public network, this needs to be _at the very least_ 75 KBytes for a + relay (that is, 600 kbits) or 50 KBytes for a bridge (400 kbits) -- but of + course, more is better; we recommend at least 250 KBytes (2 mbits) if + possible. (Default: 1 GByte) + + With this option, and in other options that take arguments in bytes, KBytes, and so on, other formats are also supported. Notably, "KBytes" can @@ -215,7 +244,7 @@ GENERAL OPTIONS any pluggable transport proxy that tries to launch __transport__. + (Example: ServerTransportOptions obfs45 shared-secret=bridgepasswd cache=/var/lib/tor/cache) -[[ExtORPort]] **ExtORPort** \['address':]__port__|**auto** +[[ExtORPort]] **ExtORPort** \['address':]__port__|**auto**:: Open this port to listen for Extended ORPort connections from your pluggable transports. @@ -273,16 +302,28 @@ GENERAL OPTIONS all sockets will be set to this limit. Must be a value between 2048 and 262144, in 1024 byte increments. Default of 8192 is recommended. -[[ControlPort]] **ControlPort** __PORT__|**auto**:: +[[ControlPort]] **ControlPort** __PORT__|**unix:**__path__|**auto** [__flags__]:: If set, Tor will accept connections on this port and allow those connections to control the Tor process using the Tor Control Protocol - (described in control-spec.txt). Note: unless you also specify one or - more of **HashedControlPassword** or **CookieAuthentication**, - setting this option will cause Tor to allow any process on the local - host to control it. (Setting both authentication methods means either - method is sufficient to authenticate to Tor.) This + (described in control-spec.txt in + https://spec.torproject.org[torspec]). Note: unless you also + specify one or more of **HashedControlPassword** or + **CookieAuthentication**, setting this option will cause Tor to allow + any process on the local host to control it. (Setting both authentication + methods means eithermethod is sufficient to authenticate to Tor.) This option is required for many Tor controllers; most use the value of 9051. - Set it to "auto" to have Tor pick a port for you. (Default: 0) + Set it to "auto" to have Tor pick a port for you. (Default: 0) + + + + Recognized flags are... + **GroupWritable**;; + Unix domain sockets only: makes the socket get created as + group-writable. + **WorldWritable**;; + Unix domain sockets only: makes the socket get created as + world-writable. + **RelaxDirModeCheck**;; + Unix domain sockets only: Do not insist that the directory + that holds the socket be read-restricted. [[ControlListenAddress]] **ControlListenAddress** __IP__[:__PORT__]:: Bind the controller listener to this address. If you specify a port, bind @@ -294,7 +335,7 @@ GENERAL OPTIONS [[ControlSocket]] **ControlSocket** __Path__:: Like ControlPort, but listens on a Unix domain socket, rather than a TCP - socket. (Unix and Unix-like systems only.) + socket. '0' disables ControlSocket (Unix and Unix-like systems only.) [[ControlSocketsGroupWritable]] **ControlSocketsGroupWritable** **0**|**1**:: If this option is set to 0, don't allow the filesystem group to read and @@ -338,10 +379,26 @@ GENERAL OPTIONS [[DataDirectory]] **DataDirectory** __DIR__:: Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor) -[[FallbackDir]] **FallbackDir** __address__:__port__ orport=__port__ id=__fingerprint__ [weight=__num__]:: +[[DataDirectoryGroupReadable]] **DataDirectoryGroupReadable** **0**|**1**:: + If this option is set to 0, don't allow the filesystem group to read the + DataDirectory. If the option is set to 1, make the DataDirectory readable + by the default GID. (Default: 0) + +[[FallbackDir]] **FallbackDir** __address__:__port__ orport=__port__ id=__fingerprint__ [weight=__num__] [ipv6=__address__:__orport__]:: When we're unable to connect to any directory cache for directory info - (usually because we don't know about any yet) we try a FallbackDir. - By default, the directory authorities are also FallbackDirs. + (usually because we don't know about any yet) we try a directory authority. + Clients also simultaneously try a FallbackDir, to avoid hangs on client + startup if a directory authority is down. Clients retry FallbackDirs more + often than directory authorities, to reduce the load on the directory + authorities. + By default, the directory authorities are also FallbackDirs. Specifying a + FallbackDir replaces Tor's default hard-coded FallbackDirs (if any). + (See the **DirAuthority** entry for an explanation of each flag.) + +[[UseDefaultFallbackDirs]] **UseDefaultFallbackDirs** **0**|**1**:: + Use Tor's default hard-coded FallbackDirs (if any). (When a + FallbackDir line is present, it replaces the hard-coded FallbackDirs, + regardless of the value of UseDefaultFallbackDirs.) (Default: 1) [[DirAuthority]] **DirAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__:: Use a nonstandard authoritative directory server at the provided address @@ -354,9 +411,16 @@ GENERAL OPTIONS "bridge" flag is set. If a flag "orport=**port**" is given, Tor will use the given port when opening encrypted tunnels to the dirserver. If a flag "weight=**num**" is given, then the directory server is chosen randomly - with probability proportional to that weight (default 1.0). Lastly, if a + with probability proportional to that weight (default 1.0). If a flag "v3ident=**fp**" is given, the dirserver is a v3 directory authority - whose v3 long-term signing key has the fingerprint **fp**. + + whose v3 long-term signing key has the fingerprint **fp**. Lastly, + if an "ipv6=__address__:__orport__" flag is present, then the directory + authority is listening for IPv6 connections on the indicated IPv6 address + and OR Port. + + + + Tor will contact the authority at __address__:__port__ (the DirPort) to + download directory documents. If an IPv6 address is supplied, Tor will + also download directory documents at the IPv6 address on the DirPort. + + If no **DirAuthority** line is given, Tor will use the default directory authorities. NOTE: this option is intended for setting up a private Tor @@ -370,12 +434,6 @@ GENERAL OPTIONS chosen with their regular weights, multiplied by this number, which should be 1.0 or less. (Default: 1.0) -[[DynamicDHGroups]] **DynamicDHGroups** **0**|**1**:: - If this option is set to 1, when running as a server, generate our - own Diffie-Hellman group instead of using the one from Apache's mod_ssl. - This option may help circumvent censorship based on static - Diffie-Hellman parameters. (Default: 0) - [[AlternateDirAuthority]] **AlternateDirAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__ + [[AlternateBridgeAuthority]] **AlternateBridgeAuthority** [__nickname__] [**flags**] __address__:__port__ __ fingerprint__:: @@ -483,6 +541,11 @@ GENERAL OPTIONS in accordance to RFC 1929. Both username and password must be between 1 and 255 characters. +[[SocksSocketsGroupWritable]] **SocksSocketsGroupWritable** **0**|**1**:: + If this option is set to 0, don't allow the filesystem group to read and + write unix sockets (e.g. SocksSocket). If the option is set to 1, make + the SocksSocket socket readable and writable by the default GID. (Default: 0) + [[KeepalivePeriod]] **KeepalivePeriod** __NUM__:: To keep firewalls from expiring connections, send a padding keepalive cell every NUM seconds on open connections that are in use. If the connection @@ -550,7 +613,7 @@ GENERAL OPTIONS \'info'. (Default: 0) [[PredictedPortsRelevanceTime]] **PredictedPortsRelevanceTime** __NUM__:: - Set how long, after the client has mad an anonymized connection to a + Set how long, after the client has made an anonymized connection to a given port, we will try to make sure that we build circuits to exits that support that port. The maximum value for this option is 1 hour. (Default: 1 hour) @@ -568,6 +631,14 @@ GENERAL OPTIONS messages to affect times logged by a controller, times attached to syslog messages, or the mtime fields on log files. (Default: 1 second) +[[TruncateLogFile]] **TruncateLogFile** **0**|**1**:: + If 1, Tor will overwrite logs at startup and in response to a HUP signal, + instead of appending to them. (Default: 0) + +[[SyslogIdentityTag]] **SyslogIdentityTag** __tag__:: + When logging to syslog, adds a tag to the syslog identity such that + log entries are marked with "Tor-__tag__". (Default: none) + [[SafeLogging]] **SafeLogging** **0**|**1**|**relay**:: Tor can scrub potentially sensitive strings from log messages (e.g. addresses) by replacing them with the string [scrubbed]. This way logs can @@ -582,6 +653,14 @@ GENERAL OPTIONS [[User]] **User** __UID__:: On startup, setuid to this user and setgid to their primary group. +[[KeepBindCapabilities]] **KeepBindCapabilities** **0**|**1**|**auto**:: + On Linux, when we are started as root and we switch our identity using + the **User** option, the **KeepBindCapabilities** option tells us whether to + try to retain our ability to bind to low ports. If this value is 1, we + try to keep the capability; if it is 0 we do not; and if it is **auto**, + we keep the capability only if we are configured to listen on a low port. + (Default: auto.) + [[HardwareAccel]] **HardwareAccel** **0**|**1**:: If non-zero, try to use built-in (static) crypto hardware acceleration when available. (Default: 0) @@ -668,9 +747,12 @@ The following options are useful only for clients (that is, if fingerprint to look up the bridge descriptor at the bridge authority, if it's provided and if UpdateBridgesFromAuthority is set too. + + - If "transport" is provided, and matches to a ClientTransportPlugin - line, we use that pluggable transports proxy to transfer data to - the bridge. + If "transport" is provided, it must match a ClientTransportPlugin line. We + then use that pluggable transport's proxy to transfer data to the bridge, + rather than connecting to the bridge directly. Some transports use a + transport-specific method to work out the remote address to connect to. + These transports typically ignore the "IP:ORPort" specified in the bridge + line. [[LearnCircuitBuildTimeout]] **LearnCircuitBuildTimeout** **0**|**1**:: If 0, CircuitBuildTimeout adaptive learning is disabled. (Default: 1) @@ -707,10 +789,12 @@ The following options are useful only for clients (that is, if unless ORPort, ExtORPort, or DirPort are configured.) (Default: 0) [[ExcludeNodes]] **ExcludeNodes** __node__,__node__,__...__:: - A list of identity fingerprints, nicknames, country codes and address - patterns of nodes to avoid when building a circuit. + A list of identity fingerprints, country codes, and address + patterns of nodes to avoid when building a circuit. Country codes are + 2-letter ISO3166 codes, and must + be wrapped in braces; fingerprints may be preceded by a dollar sign. (Example: - ExcludeNodes SlowServer, ABCD1234CDEF5678ABCD1234CDEF5678ABCD1234, \{cc}, 255.254.0.0/8) + + ExcludeNodes ABCD1234CDEF5678ABCD1234CDEF5678ABCD1234, \{cc}, 255.254.0.0/8) + + By default, this option is treated as a preference that Tor is allowed to override in order to keep working. @@ -730,11 +814,13 @@ The following options are useful only for clients (that is, if [[ExcludeExitNodes]] **ExcludeExitNodes** __node__,__node__,__...__:: - A list of identity fingerprints, nicknames, country codes and address + A list of identity fingerprints, country codes, and address patterns of nodes to never use when picking an exit node---that is, a node that delivers traffic for you outside the Tor network. Note that any node listed in ExcludeNodes is automatically considered to be part of this - list too. See also the caveats on the "ExitNodes" option below. + list too. See + the **ExcludeNodes** option for more information on how to specify + nodes. See also the caveats on the "ExitNodes" option below. [[GeoIPExcludeUnknown]] **GeoIPExcludeUnknown** **0**|**1**|**auto**:: If this option is set to 'auto', then whenever any country code is set in @@ -745,9 +831,10 @@ The following options are useful only for clients (that is, if configured or can't be found. (Default: auto) [[ExitNodes]] **ExitNodes** __node__,__node__,__...__:: - A list of identity fingerprints, nicknames, country codes and address + A list of identity fingerprints, country codes, and address patterns of nodes to use as exit node---that is, a - node that delivers traffic for you outside the Tor network. + + node that delivers traffic for you outside the Tor network. See + the **ExcludeNodes** option for more information on how to specify nodes. + + Note that if you list too few nodes here, or if you exclude too many exit nodes with ExcludeExitNodes, you can degrade functionality. For example, @@ -768,7 +855,7 @@ The following options are useful only for clients (that is, if this option. [[EntryNodes]] **EntryNodes** __node__,__node__,__...__:: - A list of identity fingerprints, nicknames, and country codes of nodes + A list of identity fingerprints and country codes of nodes to use for the first hop in your normal circuits. Normal circuits include all circuits except for direct connections to directory servers. The Bridge @@ -776,7 +863,8 @@ The following options are useful only for clients (that is, if UseBridges is 1, the Bridges are used as your entry nodes. + + The ExcludeNodes option overrides this option: any node listed in both - EntryNodes and ExcludeNodes is treated as excluded. + EntryNodes and ExcludeNodes is treated as excluded. See + the **ExcludeNodes** option for more information on how to specify nodes. [[StrictNodes]] **StrictNodes** **0**|**1**:: If StrictNodes is set to 1, Tor will treat the ExcludeNodes option as a @@ -873,12 +961,12 @@ The following options are useful only for clients (that is, if When a request for address arrives to Tor, it will transform to newaddress before processing it. For example, if you always want connections to www.example.com to exit via __torserver__ (where __torserver__ is the - nickname of the server), use "MapAddress www.example.com + fingerprint of the server), use "MapAddress www.example.com www.example.com.torserver.exit". If the value is prefixed with a "\*.", matches an entire domain. For example, if you always want connections to example.com and any if its subdomains to exit via - __torserver__ (where __torserver__ is the nickname of the server), use + __torserver__ (where __torserver__ is the fingerprint of the server), use "MapAddress \*.example.com \*.example.com.torserver.exit". (Note the leading "*." in each part of the directive.) You can also redirect all subdomains of a domain to a single address. For example, "MapAddress @@ -917,7 +1005,9 @@ The following options are useful only for clients (that is, if Feel free to reuse a circuit that was first used at most NUM seconds ago, but never attach a new stream to a circuit that is too old. For hidden services, this applies to the __last__ time a circuit was used, not the - first. (Default: 10 minutes) + first. Circuits with streams constructed with SOCKS authentication via + SocksPorts that have **KeepAliveIsolateSOCKSAuth** ignore this value. + (Default: 10 minutes) [[MaxClientCircuitsPending]] **MaxClientCircuitsPending** __NUM__:: Do not allow more than NUM circuits to be pending at a time for handling @@ -925,27 +1015,36 @@ The following options are useful only for clients (that is, if but it has not yet been completely constructed. (Default: 32) [[NodeFamily]] **NodeFamily** __node__,__node__,__...__:: - The Tor servers, defined by their identity fingerprints or nicknames, + The Tor servers, defined by their identity fingerprints, constitute a "family" of similar or co-administered servers, so never use any two of them in the same circuit. Defining a NodeFamily is only needed when a server doesn't list the family itself (with MyFamily). This option - can be used multiple times. In addition to nodes, you can also list - IP address and ranges and country codes in {curly braces}. + can be used multiple times; each instance defines a separate family. In + addition to nodes, you can also list IP address and ranges and country + codes in {curly braces}. See the **ExcludeNodes** option for more + information on how to specify nodes. [[EnforceDistinctSubnets]] **EnforceDistinctSubnets** **0**|**1**:: If 1, Tor will not put two servers whose IP addresses are "too close" on the same circuit. Currently, two addresses are "too close" if they lie in the same /16 range. (Default: 1) -[[SOCKSPort]] **SOCKSPort** \['address':]__port__|**auto** [_flags_] [_isolation flags_]:: +[[SocksPort]] **SocksPort** \['address':]__port__|**unix:**__path__|**auto** [_flags_] [_isolation flags_]:: Open this port to listen for connections from SOCKS-speaking applications. Set this to 0 if you don't want to allow application connections via SOCKS. Set it to "auto" to have Tor pick a port for you. This directive can be specified multiple times to bind to multiple addresses/ports. (Default: 9050) + + + NOTE: Although this option allows you to specify an IP address + other than localhost, you should do so only with extreme caution. + The SOCKS protocol is unencrypted and (as we use it) + unauthenticated, so exposing it in this way could leak your + information to anybody watching your network, and allow anybody + to use your computer as an open proxy. + + + The _isolation flags_ arguments give Tor rules for which streams - received on this SOCKSPort are allowed to share circuits with one + received on this SocksPort are allowed to share circuits with one another. Recognized isolation flags are: **IsolateClientAddr**;; Don't share circuits with streams from a different @@ -960,20 +1059,23 @@ The following options are useful only for clients (that is, if (SOCKS 4, SOCKS 5, TransPort connections, NATDPort connections, and DNSPort requests are all considered to be different protocols.) **IsolateDestPort**;; - Don't share circuits with streams targetting a different + Don't share circuits with streams targeting a different destination port. **IsolateDestAddr**;; - Don't share circuits with streams targetting a different + Don't share circuits with streams targeting a different destination address. + **KeepAliveIsolateSOCKSAuth**;; + If **IsolateSOCKSAuth** is enabled, keep alive circuits that have + streams with SOCKS authentication set indefinitely. **SessionGroup=**__INT__;; If no other isolation rules would prevent it, allow streams on this port to share circuits with streams from every other port with the same session group. (By default, streams received - on different SOCKSPorts, TransPorts, etc are always isolated from one + on different SocksPorts, TransPorts, etc are always isolated from one another. This option overrides that behavior.) -[[OtherSOCKSPortFlags]]:: - Other recognized __flags__ for a SOCKSPort are: +[[OtherSocksPortFlags]]:: + Other recognized __flags__ for a SocksPort are: **NoIPv4Traffic**;; Tell exits to not connect to IPv4 addresses in response to SOCKS requests on this connection. @@ -984,20 +1086,18 @@ The following options are useful only for clients (that is, if **PreferIPv6**;; Tells exits that, if a host has both an IPv4 and an IPv6 address, we would prefer to connect to it via IPv6. (IPv4 is the default.) + - + - NOTE: Although this option allows you to specify an IP address - other than localhost, you should do so only with extreme caution. - The SOCKS protocol is unencrypted and (as we use it) - unauthenticated, so exposing it in this way could leak your - information to anybody watching your network, and allow anybody - to use your computer as an open proxy. + - + **CacheIPv4DNS**;; Tells the client to remember IPv4 DNS answers we receive from exit nodes via this connection. (On by default.) **CacheIPv6DNS**;; Tells the client to remember IPv6 DNS answers we receive from exit nodes via this connection. + **GroupWritable**;; + Unix domain sockets only: makes the socket get created as + group-writable. + **WorldWritable**;; + Unix domain sockets only: makes the socket get created as + world-writable. **CacheDNS**;; Tells the client to remember all DNS answers we receive from exit nodes via this connection. @@ -1014,7 +1114,7 @@ The following options are useful only for clients (that is, if requests via this connection. **PreferIPv6Automap**;; When serving a hostname lookup request on this port that - should get automapped (according to AutomapHostsOnResove), + should get automapped (according to AutomapHostsOnResolve), if we could return either an IPv4 or an IPv6 answer, prefer an IPv6 answer. (On by default.) **PreferSOCKSNoAuth**;; @@ -1027,14 +1127,14 @@ The following options are useful only for clients (that is, if authentication" when IsolateSOCKSAuth is disabled, or when this option is set. -[[SOCKSListenAddress]] **SOCKSListenAddress** __IP__[:__PORT__]:: +[[SocksListenAddress]] **SocksListenAddress** __IP__[:__PORT__]:: Bind to this address to listen for connections from Socks-speaking applications. (Default: 127.0.0.1) You can also specify a port (e.g. 192.168.0.1:9100). This directive can be specified multiple times to bind to multiple addresses/ports. (DEPRECATED: As of 0.2.3.x-alpha, you can - now use multiple SOCKSPort entries, and provide addresses for SOCKSPort - entries, so SOCKSListenAddress no longer has a purpose. For backward - compatibility, SOCKSListenAddress is only allowed when SOCKSPort is just + now use multiple SocksPort entries, and provide addresses for SocksPort + entries, so SocksListenAddress no longer has a purpose. For backward + compatibility, SocksListenAddress is only allowed when SocksPort is just a port number.) [[SocksPolicy]] **SocksPolicy** __policy__,__policy__,__...__:: @@ -1097,6 +1197,17 @@ The following options are useful only for clients (that is, if download any non-default directory material. It doesn't currently do anything when we lack a live consensus. (Default: 1) +[[GuardfractionFile]] **GuardfractionFile** __FILENAME__:: + V3 authoritative directories only. Configures the location of the + guardfraction file which contains information about how long relays + have been guards. (Default: unset) + +[[UseGuardFraction]] **UseGuardFraction** **0**|**1**|**auto**:: + This torrc option specifies whether clients should use the + guardfraction information found in the consensus during path + selection. If it's set to 'auto', clients will do what the + UseGuardFraction consensus parameter tells them to do. (Default: auto) + [[NumEntryGuards]] **NumEntryGuards** __NUM__:: If UseEntryGuards is set to 1, we will try to pick a total of NUM routers as long-term entries for our circuits. If NUM is 0, we try to learn @@ -1230,7 +1341,7 @@ The following options are useful only for clients (that is, if Use 0 if you don't want to allow NATD connections. Set the port to "auto" to have Tor pick a port for you. This directive can be specified multiple times to bind to multiple addresses/ports. See - SOCKSPort for an explanation of isolation flags. + + SocksPort for an explanation of isolation flags. + + This option is only for people who cannot use TransPort. (Default: 0) @@ -1258,7 +1369,7 @@ The following options are useful only for clients (that is, if doesn't handle arbitrary DNS request types. Set the port to "auto" to have Tor pick a port for you. This directive can be specified multiple times to bind to multiple - addresses/ports. See SOCKSPort for an explanation of isolation + addresses/ports. See SocksPort for an explanation of isolation flags. (Default: 0) [[DNSListenAddress]] **DNSListenAddress** __IP__[:__PORT__]:: @@ -1283,7 +1394,7 @@ The following options are useful only for clients (that is, if [[DownloadExtraInfo]] **DownloadExtraInfo** **0**|**1**:: If true, Tor downloads and caches "extra-info" documents. These documents contain information about servers other than the information in their - regular router descriptors. Tor does not use this information for anything + regular server descriptors. Tor does not use this information for anything itself; to save bandwidth, leave this option turned off. (Default: 0) [[WarnPlaintextPorts]] **WarnPlaintextPorts** __port__,__port__,__...__:: @@ -1318,6 +1429,22 @@ The following options are useful only for clients (that is, if To enable this option the compile time flag --enable-tor2webmode must be specified. (Default: 0) +[[Tor2webRendezvousPoints]] **Tor2webRendezvousPoints** __node__,__node__,__...__:: + A list of identity fingerprints, nicknames, country codes and + address patterns of nodes that are allowed to be used as RPs + in HS circuits; any other nodes will not be used as RPs. + (Example: + Tor2webRendezvousPoints Fastyfasty, ABCD1234CDEF5678ABCD1234CDEF5678ABCD1234, \{cc}, 255.254.0.0/8) + + + + This feature can only be used if Tor2webMode is also enabled. + + + ExcludeNodes have higher priority than Tor2webRendezvousPoints, + which means that nodes specified in ExcludeNodes will not be + picked as RPs. + + + If no nodes in Tor2webRendezvousPoints are currently available for + use, Tor will choose a random node when building HS circuits. + [[UseMicrodescriptors]] **UseMicrodescriptors** **0**|**1**|**auto**:: Microdescriptors are a smaller version of the information that Tor needs in order to build its circuits. Using microdescriptors makes Tor clients @@ -1391,17 +1518,33 @@ The following options are useful only for clients (that is, if If no defaults are available there, these options default to 20, .80, .60, and 100, respectively. +[[ClientUseIPv4]] **ClientUseIPv4** **0**|**1**:: + If this option is set to 0, Tor will avoid connecting to directory servers + and entry nodes over IPv4. Note that clients with an IPv4 + address in a **Bridge**, proxy, or pluggable transport line will try + connecting over IPv4 even if **ClientUseIPv4** is set to 0. (Default: 1) + [[ClientUseIPv6]] **ClientUseIPv6** **0**|**1**:: - If this option is set to 1, Tor might connect to entry nodes over - IPv6. Note that clients configured with an IPv6 address in a - **Bridge** line will try connecting over IPv6 even if - **ClientUseIPv6** is set to 0. (Default: 0) + If this option is set to 1, Tor might connect to directory servers or + entry nodes over IPv6. Note that clients configured with an IPv6 address + in a **Bridge**, proxy, or pluggable transport line will try connecting + over IPv6 even if **ClientUseIPv6** is set to 0. (Default: 0) + +[[ClientPreferIPv6DirPort]] **ClientPreferIPv6DirPort** **0**|**1**|**auto**:: + If this option is set to 1, Tor prefers a directory port with an IPv6 + address over one with IPv4, for direct connections, if a given directory + server has both. (Tor also prefers an IPv6 DirPort if IPv4Client is set to + 0.) If this option is set to auto, clients prefer IPv4. Other things may + influence the choice. This option breaks a tie to the favor of IPv6. + (Default: auto) -[[ClientPreferIPv6ORPort]] **ClientPreferIPv6ORPort** **0**|**1**:: +[[ClientPreferIPv6ORPort]] **ClientPreferIPv6ORPort** **0**|**1**|**auto**:: If this option is set to 1, Tor prefers an OR port with an IPv6 - address over one with IPv4 if a given entry node has both. Other - things may influence the choice. This option breaks a tie to the - favor of IPv6. (Default: 0) + address over one with IPv4 if a given entry node has both. (Tor also + prefers an IPv6 ORPort if IPv4Client is set to 0.) If this option is set + to auto, Tor bridge clients prefer the configured bridge address, and + other clients prefer IPv4. Other things may influence the choice. This + option breaks a tie to the favor of IPv6. (Default: auto) [[PathsNeededToBuildCircuits]] **PathsNeededToBuildCircuits** __NUM__:: Tor clients don't build circuits for user traffic until they know @@ -1415,15 +1558,44 @@ The following options are useful only for clients (that is, if Tor will use a default value chosen by the directory authorities. (Default: -1.) -[[Support022HiddenServices]] **Support022HiddenServices** **0**|**1**|**auto**:: - Tor hidden services running versions before 0.2.3.x required clients to - send timestamps, which can potentially be used to distinguish clients - whose view of the current time is skewed. If this option is set to 0, we - do not send this timestamp, and hidden services on obsolete Tor versions - will not work. If this option is set to 1, we send the timestamp. If - this optoin is "auto", we take a recommendation from the latest consensus - document. (Default: auto) - +[[ClientBootstrapConsensusAuthorityDownloadSchedule]] **ClientBootstrapConsensusAuthorityDownloadSchedule** __N__,__N__,__...__:: + Schedule for when clients should download consensuses from authorities + if they are bootstrapping (that is, they don't have a usable, reasonably + live consensus). Only used by clients fetching from a list of fallback + directory mirrors. This schedule is advanced by (potentially concurrent) + connection attempts, unlike other schedules, which are advanced by + connection failures. (Default: 10, 11, 3600, 10800, 25200, 54000, + 111600, 262800) + +[[ClientBootstrapConsensusFallbackDownloadSchedule]] **ClientBootstrapConsensusFallbackDownloadSchedule** __N__,__N__,__...__:: + Schedule for when clients should download consensuses from fallback + directory mirrors if they are bootstrapping (that is, they don't have a + usable, reasonably live consensus). Only used by clients fetching from a + list of fallback directory mirrors. This schedule is advanced by + (potentially concurrent) connection attempts, unlike other schedules, + which are advanced by connection failures. (Default: 0, 1, 4, 11, 3600, + 10800, 25200, 54000, 111600, 262800) + +[[ClientBootstrapConsensusAuthorityOnlyDownloadSchedule]] **ClientBootstrapConsensusAuthorityOnlyDownloadSchedule** __N__,__N__,__...__:: + Schedule for when clients should download consensuses from authorities + if they are bootstrapping (that is, they don't have a usable, reasonably + live consensus). Only used by clients which don't have or won't fetch + from a list of fallback directory mirrors. This schedule is advanced by + (potentially concurrent) connection attempts, unlike other schedules, + which are advanced by connection failures. (Default: 0, 3, 7, 3600, + 10800, 25200, 54000, 111600, 262800) + +[[ClientBootstrapConsensusMaxDownloadTries]] **ClientBootstrapConsensusMaxDownloadTries** __NUM__:: + Try this many times to download a consensus while bootstrapping using + fallback directory mirrors before giving up. (Default: 7) + +[[ClientBootstrapConsensusAuthorityOnlyMaxDownloadTries]] **ClientBootstrapConsensusAuthorityOnlyMaxDownloadTries** __NUM__:: + Try this many times to download a consensus while bootstrapping using + authorities before giving up. (Default: 4) + +[[ClientBootstrapConsensusMaxInProgressTries]] **ClientBootstrapConsensusMaxInProgressTries** __NUM__:: + Try this many simultaneous connections to download a consensus before + waiting for one to complete, timeout, or error out. (Default: 4) SERVER OPTIONS -------------- @@ -1456,8 +1628,8 @@ is non-zero): [[BridgeRelay]] **BridgeRelay** **0**|**1**:: Sets the relay to act as a "bridge" with respect to relaying connections from bridge users to the Tor network. It mainly causes Tor to publish a - server descriptor to the bridge database, rather than publishing a relay - descriptor to the public directory authorities. + server descriptor to the bridge database, rather than + to the public directory authorities. [[ContactInfo]] **ContactInfo** __email_address__:: Administrative contact information for this relay or bridge. This line @@ -1468,24 +1640,56 @@ is non-zero): that it's an email address and/or generate a new address for this purpose. +[[ExitRelay]] **ExitRelay** **0**|**1**|**auto**:: + Tells Tor whether to run as an exit relay. If Tor is running as a + non-bridge server, and ExitRelay is set to 1, then Tor allows traffic to + exit according to the ExitPolicy option (or the default ExitPolicy if + none is specified). + + + If ExitRelay is set to 0, no traffic is allowed to + exit, and the ExitPolicy option is ignored. + + + + If ExitRelay is set to "auto", then Tor behaves as if it were set to 1, but + warns the user if this would cause traffic to exit. In a future version, + the default value will be 0. (Default: auto) + [[ExitPolicy]] **ExitPolicy** __policy__,__policy__,__...__:: Set an exit policy for this server. Each policy is of the form - "**accept**|**reject** __ADDR__[/__MASK__][:__PORT__]". If /__MASK__ is + "**accept[6]**|**reject[6]** __ADDR__[/__MASK__][:__PORT__]". If /__MASK__ is omitted then this policy just applies to the host given. Instead of giving - a host or network you can also use "\*" to denote the universe (0.0.0.0/0). + a host or network you can also use "\*" to denote the universe (0.0.0.0/0 + and ::/128), or \*4 to denote all IPv4 addresses, and \*6 to denote all + IPv6 addresses. __PORT__ can be a single port number, an interval of ports "__FROM_PORT__-__TO_PORT__", or "\*". If __PORT__ is omitted, that means "\*". + + For example, "accept 18.7.22.69:\*,reject 18.0.0.0/8:\*,accept \*:\*" would - reject any traffic destined for MIT except for web.mit.edu, and accept - anything else. + - + - To specify all internal and link-local networks (including 0.0.0.0/8, - 169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, and - 172.16.0.0/12), you can use the "private" alias instead of an address. - These addresses are rejected by default (at the beginning of your exit - policy), along with your public IP address, unless you set the + reject any IPv4 traffic destined for MIT except for web.mit.edu, and accept + any other IPv4 or IPv6 traffic. + + + + Tor also allows IPv6 exit policy entries. For instance, "reject6 [FC00::]/7:\*" + rejects all destinations that share 7 most significant bit prefix with + address FC00::. Respectively, "accept6 [C000::]/3:\*" accepts all destinations + that share 3 most significant bit prefix with address C000::. + + + + accept6 and reject6 only produce IPv6 exit policy entries. Using an IPv4 + address with accept6 or reject6 is ignored and generates a warning. + accept/reject allows either IPv4 or IPv6 addresses. Use \*4 as an IPv4 + wildcard address, and \*6 as an IPv6 wildcard address. accept/reject * + expands to matching IPv4 and IPv6 wildcard address rules. + + + + To specify all IPv4 and IPv6 internal and link-local networks (including + 0.0.0.0/8, 169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, + 172.16.0.0/12, [::]/8, [FC00::]/7, [FE80::]/10, [FEC0::]/10, [FF00::]/8, + and [::]/127), you can use the "private" alias instead of an address. + ("private" always produces rules for IPv4 and IPv6 addresses, even when + used with accept6/reject6.) + + + + Private addresses are rejected by default (at the beginning of your exit + policy), along with any configured primary public IPv4 and IPv6 addresses, + and any public IPv4 and IPv6 addresses on any interface on the relay. + These private addresses are rejected unless you set the ExitPolicyRejectPrivate config option to 0. For example, once you've done that, you could allow HTTP to 127.0.0.1 and block all other connections to internal networks with "accept 127.0.0.1:80,reject private:\*", though that @@ -1493,18 +1697,17 @@ is non-zero): public (external) IP address. See RFC 1918 and RFC 3330 for more details about internal and reserved IP address space. + + - Tor also allow IPv6 exit policy entries. For instance, "reject6 [FC00::]/7:*" - rejects all destinations that share 7 most significant bit prefix with - address FC00::. Respectively, "accept6 [C000::]/3:*" accepts all destinations - that share 3 most significant bit prefix with address C000::. + - + This directive can be specified multiple times so you don't have to put it all on one line. + + Policies are considered first to last, and the first match wins. If you - want to \_replace_ the default exit policy, end your exit policy with - either a reject \*:* or an accept \*:*. Otherwise, you're \_augmenting_ - (prepending to) the default exit policy. The default exit policy is: + + want to allow the same ports on IPv4 and IPv6, write your rules using + accept/reject \*. If you want to allow different ports on IPv4 and IPv6, + write your IPv6 rules using accept6/reject6 \*6, and your IPv4 rules using + accept/reject \*4. If you want to \_replace_ the default exit policy, end + your exit policy with either a reject \*:* or an accept \*:*. Otherwise, + you're \_augmenting_ (prepending to) the default exit policy. The default + exit policy is: + reject *:25 reject *:119 @@ -1518,9 +1721,18 @@ is non-zero): reject *:6881-6999 accept *:* + Since the default exit policy uses accept/reject *, it applies to both + IPv4 and IPv6 addresses. + [[ExitPolicyRejectPrivate]] **ExitPolicyRejectPrivate** **0**|**1**:: - Reject all private (local) networks, along with your own public IP address, - at the beginning of your exit policy. See above entry on ExitPolicy. + Reject all private (local) networks, along with any configured public + IPv4 and IPv6 addresses, at the beginning of your exit policy. (This + includes the IPv4 and IPv6 addresses advertised by the relay, any + OutboundBindAddress, and the bind addresses of any port options, such as + ORPort and DirPort.) This also rejects any public IPv4 and IPv6 addresses + on any interface on the relay. (If IPv6Exit is not set, all IPv6 addresses + will be rejected anyway.) + See above entry on ExitPolicy. (Default: 1) [[IPv6Exit]] **IPv6Exit** **0**|**1**:: @@ -1534,7 +1746,7 @@ is non-zero): [[MyFamily]] **MyFamily** __node__,__node__,__...__:: Declare that this Tor server is controlled or administered by a group or organization identical or similar to that of the other servers, defined by - their identity fingerprints or nicknames. When two servers both declare + their identity fingerprints. When two servers both declare that they are in the same \'family', Tor clients will not use them in the same circuit. (Each server only needs to list the other servers in its family; it doesn't need to list itself, but it won't hurt.) Do not list @@ -1628,22 +1840,37 @@ is non-zero): Log a heartbeat message every **HeartbeatPeriod** seconds. This is a log level __notice__ message, designed to let you know your Tor server is still alive and doing useful things. Settings this - to 0 will disable the heartbeat. (Default: 6 hours) + to 0 will disable the heartbeat. Otherwise, it must be at least 30 + minutes. (Default: 6 hours) [[AccountingMax]] **AccountingMax** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**|**TBytes**:: - Never send more than the specified number of bytes in a given accounting - period, or receive more than that number in the period. For example, with - AccountingMax set to 1 GByte, a server could send 900 MBytes and - receive 800 MBytes and continue running. It will only hibernate once - one of the two reaches 1 GByte. When the number of bytes gets low, - Tor will stop accepting new connections and circuits. When the - number of bytes is exhausted, Tor will hibernate until some - time in the next accounting period. To prevent all servers from waking at - the same time, Tor will also wait until a random point in each period - before waking up. If you have bandwidth cost issues, enabling hibernation - is preferable to setting a low bandwidth, since it provides users with a - collection of fast servers that are up some of the time, which is more - useful than a set of slow servers that are always "available". + Limits the max number of bytes sent and received within a set time period + using a given calculation rule (see: AccountingStart, AccountingRule). + Useful if you need to stay under a specific bandwidth. By default, the + number used for calculation is the max of either the bytes sent or + received. For example, with AccountingMax set to 1 GByte, a server + could send 900 MBytes and receive 800 MBytes and continue running. + It will only hibernate once one of the two reaches 1 GByte. This can + be changed to use the sum of the both bytes received and sent by setting + the AccountingRule option to "sum" (total bandwidth in/out). When the + number of bytes remaining gets low, Tor will stop accepting new connections + and circuits. When the number of bytes is exhausted, Tor will hibernate + until some time in the next accounting period. To prevent all servers + from waking at the same time, Tor will also wait until a random point + in each period before waking up. If you have bandwidth cost issues, + enabling hibernation is preferable to setting a low bandwidth, since + it provides users with a collection of fast servers that are up some + of the time, which is more useful than a set of slow servers that are + always "available". + +[[AccountingRule]] **AccountingRule** **sum**|**max**|**in**|**out**:: + How we determine when our AccountingMax has been reached (when we + should hibernate) during a time interval. Set to "max" to calculate + using the higher of either the sent or received bytes (this is the + default functionality). Set to "sum" to calculate using the sent + plus received bytes. Set to "in" to calculate using only the + received bytes. Set to "out" to calculate using only the sent bytes. + (Default: max) [[AccountingStart]] **AccountingStart** **day**|**week**|**month** [__day__] __HH:MM__:: Specify how long accounting periods last. If **month** is given, each @@ -1693,7 +1920,7 @@ is non-zero): [[ServerDNSTestAddresses]] **ServerDNSTestAddresses** __address__,__address__,__...__:: When we're detecting DNS hijacking, make sure that these __valid__ addresses aren't getting redirected. If they are, then our DNS is completely useless, - and we'll reset our exit policy to "reject *:*". This option only affects + and we'll reset our exit policy to "reject \*:*". This option only affects name lookups that your server does on behalf of clients. (Default: "www.google.com, www.mit.edu, www.yahoo.com, www.slashdot.org") @@ -1731,25 +1958,58 @@ is non-zero): (Default: P256) [[CellStatistics]] **CellStatistics** **0**|**1**:: - When this option is enabled, Tor writes statistics on the mean time that - cells spend in circuit queues to disk every 24 hours. (Default: 0) + Relays only. + When this option is enabled, Tor collects statistics about cell + processing (i.e. mean time a cell is spending in a queue, mean + number of cells in a queue and mean number of processed cells per + circuit) and writes them into disk every 24 hours. Onion router + operators may use the statistics for performance monitoring. + If ExtraInfoStatistics is enabled, it will published as part of + extra-info document. (Default: 0) [[DirReqStatistics]] **DirReqStatistics** **0**|**1**:: + Relays and bridges only. When this option is enabled, a Tor directory writes statistics on the number and response time of network status requests to disk every 24 - hours. (Default: 1) + hours. Enables relay and bridge operators to monitor how much their + server is being used by clients to learn about Tor network. + If ExtraInfoStatistics is enabled, it will published as part of + extra-info document. (Default: 1) [[EntryStatistics]] **EntryStatistics** **0**|**1**:: + Relays only. When this option is enabled, Tor writes statistics on the number of - directly connecting clients to disk every 24 hours. (Default: 0) + directly connecting clients to disk every 24 hours. Enables relay + operators to monitor how much inbound traffic that originates from + Tor clients passes through their server to go further down the + Tor network. If ExtraInfoStatistics is enabled, it will be published + as part of extra-info document. (Default: 0) [[ExitPortStatistics]] **ExitPortStatistics** **0**|**1**:: - When this option is enabled, Tor writes statistics on the number of relayed - bytes and opened stream per exit port to disk every 24 hours. (Default: 0) + Exit relays only. + When this option is enabled, Tor writes statistics on the number of + relayed bytes and opened stream per exit port to disk every 24 hours. + Enables exit relay operators to measure and monitor amounts of traffic + that leaves Tor network through their exit node. If ExtraInfoStatistics + is enabled, it will be published as part of extra-info document. + (Default: 0) [[ConnDirectionStatistics]] **ConnDirectionStatistics** **0**|**1**:: - When this option is enabled, Tor writes statistics on the bidirectional use - of connections to disk every 24 hours. (Default: 0) + Relays only. + When this option is enabled, Tor writes statistics on the amounts of + traffic it passes between itself and other relays to disk every 24 + hours. Enables relay operators to monitor how much their relay is + being used as middle node in the circuit. If ExtraInfoStatistics is + enabled, it will be published as part of extra-info document. + (Default: 0) + +[[HiddenServiceStatistics]] **HiddenServiceStatistics** **0**|**1**:: + Relays only. + When this option is enabled, a Tor relay writes obfuscated + statistics on its role as hidden-service directory, introduction + point, or rendezvous point to disk every 24 hours. If + ExtraInfoStatistics is also enabled, these statistics are further + published to the directory authorities. (Default: 1) [[ExtraInfoStatistics]] **ExtraInfoStatistics** **0**|**1**:: When this option is enabled, Tor includes previously gathered statistics in @@ -1757,9 +2017,13 @@ is non-zero): (Default: 1) [[ExtendAllowPrivateAddresses]] **ExtendAllowPrivateAddresses** **0**|**1**:: - When this option is enabled, Tor routers allow EXTEND request to - localhost, RFC1918 addresses, and so on. This can create security issues; - you should probably leave it off. (Default: 0) + When this option is enabled, Tor will connect to relays on localhost, + RFC1918 addresses, and so on. In particular, Tor will make direct OR + connections, and Tor routers allow EXTEND requests, to these private + addresses. (Tor will always allow connections to bridges, proxies, and + pluggable transports configured on private addresses.) Enabling this + option can create security issues; you should probably leave it off. + (Default: 0) [[MaxMemInQueues]] **MaxMemInQueues** __N__ **bytes**|**KB**|**MB**|**GB**:: This option configures a threshold above which Tor will assume that it @@ -1771,6 +2035,19 @@ is non-zero): this. If this option is set to 0, Tor will try to pick a reasonable default based on your system's physical memory. (Default: 0) +[[SigningKeyLifetime]] **SigningKeyLifetime** __N__ **days**|**weeks**|**months**:: + For how long should each Ed25519 signing key be valid? Tor uses a + permanent master identity key that can be kept offline, and periodically + generates new "signing" keys that it uses online. This option + configures their lifetime. + (Default: 30 days) + +[[OfflineMasterKey]] **OfflineMasterKey** **0**|**1**:: + If non-zero, the Tor relay will never generate or load its master secret + key. Instead, you'll have to use "tor --keygen" to manage the permanent + ed25519 master identity key, as well as the corresponding temporary + signing keys and certificates. (Default: 0) + DIRECTORY SERVER OPTIONS ------------------------ @@ -1783,11 +2060,6 @@ if DirPort is non-zero): to set up a separate webserver. There's a sample disclaimer in contrib/operator-tools/tor-exit-notice.html. -[[HidServDirectoryV2]] **HidServDirectoryV2** **0**|**1**:: - When this option is set, Tor accepts and serves v2 hidden service - descriptors. Setting DirPort is not required for this, because clients - connect via the ORPort by default. (Default: 1) - [[DirPort]] **DirPort** \['address':]__PORT__|**auto** [_flags_]:: If this option is nonzero, advertise the directory service on this port. Set it to "auto" to have Tor pick a port for you. This option can occur @@ -1811,6 +2083,12 @@ if DirPort is non-zero): except that port specifiers are ignored. Any address not matched by some entry in the policy is accepted. +[[DirCache]] **DirCache** **0**|**1**:: + When this option is set, Tor caches all current directory documents and + accepts client requests for them. Setting DirPort is not required for this, + because clients connect via the ORPort by default. Setting either DirPort + or BridgeRelay and setting DirCache to 0 is not supported. (Default: 1) + DIRECTORY AUTHORITY SERVER OPTIONS ---------------------------------- @@ -1831,8 +2109,8 @@ on the public Tor network. [[V3AuthoritativeDirectory]] **V3AuthoritativeDirectory** **0**|**1**:: When this option is set in addition to **AuthoritativeDirectory**, Tor generates version 3 network statuses and serves descriptors, etc as - described in doc/spec/dir-spec.txt (for Tor clients and servers running at - least 0.2.0.x). + described in dir-spec.txt file of https://spec.torproject.org/[torspec] + (for Tor clients and servers running atleast 0.2.0.x). [[VersioningAuthoritativeDirectory]] **VersioningAuthoritativeDirectory** **0**|**1**:: When this option is set to 1, Tor adds information on which versions of @@ -1841,15 +2119,6 @@ on the public Tor network. authorities provide this service optionally. See **RecommendedVersions**, **RecommendedClientVersions**, and **RecommendedServerVersions**. -[[NamingAuthoritativeDirectory]] **NamingAuthoritativeDirectory** **0**|**1**:: - When this option is set to 1, then the server advertises that it has - opinions about nickname-to-fingerprint bindings. It will include these - opinions in its published network-status pages, by listing servers with - the flag "Named" if a correct binding between that nickname and fingerprint - has been registered with the dirserver. Naming dirservers will refuse to - accept or publish descriptors that contradict a registered binding. See - **approved-routers** in the **FILES** section below. - [[RecommendedVersions]] **RecommendedVersions** __STRING__:: STRING is a comma-separated list of Tor versions currently believed to be safe. The list is included in each directory, and nodes which pull down the @@ -1857,6 +2126,12 @@ on the public Tor network. multiple times: the values from multiple lines are spliced together. When this is set then **VersioningAuthoritativeDirectory** should be set too. +[[RecommendedPackages]] **RecommendedPackages** __PACKAGENAME__ __VERSION__ __URL__ __DIGESTTYPE__**=**__DIGEST__ :: + Adds "package" line to the directory authority's vote. This information + is used to vote on the correct URL and digest for the released versions + of different Tor-related packages, so that the consensus can certify + them. This line may appear any number of times. + [[RecommendedClientVersions]] **RecommendedClientVersions** __STRING__:: STRING is a comma-separated list of Tor versions currently believed to be safe for clients to use. This information is included in version 2 @@ -1866,7 +2141,7 @@ on the public Tor network. [[BridgeAuthoritativeDir]] **BridgeAuthoritativeDir** **0**|**1**:: When this option is set in addition to **AuthoritativeDirectory**, Tor - accepts and serves router descriptors, but it caches and serves the main + accepts and serves server descriptors, but it caches and serves the main networkstatus documents rather than generating its own. (Default: 0) [[MinUptimeHidServDirectoryV2]] **MinUptimeHidServDirectoryV2** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: @@ -1885,24 +2160,19 @@ on the public Tor network. in the "params" line of its networkstatus vote. [[DirAllowPrivateAddresses]] **DirAllowPrivateAddresses** **0**|**1**:: - If set to 1, Tor will accept router descriptors with arbitrary "Address" + If set to 1, Tor will accept server descriptors with arbitrary "Address" elements. Otherwise, if the address is not an IP address or is a private IP - address, it will reject the router descriptor. (Default: 0) + address, it will reject the server descriptor. (Default: 0) -[[AuthDirBadDir]] **AuthDirBadDir** __AddressPattern...__:: +[[AuthDirBadExit]] **AuthDirBadExit** __AddressPattern...__:: Authoritative directories only. A set of address patterns for servers that - will be listed as bad directories in any network status document this - authority publishes, if **AuthDirListBadDirs** is set. + + will be listed as bad exits in any network status document this authority + publishes, if **AuthDirListBadExits** is set. + (The address pattern syntax here and in the options below is the same as for exit policies, except that you don't need to say "accept" or "reject", and ports are not needed.) -[[AuthDirBadExit]] **AuthDirBadExit** __AddressPattern...__:: - Authoritative directories only. A set of address patterns for servers that - will be listed as bad exits in any network status document this authority - publishes, if **AuthDirListBadExits** is set. - [[AuthDirInvalid]] **AuthDirInvalid** __AddressPattern...__:: Authoritative directories only. A set of address patterns for servers that will never be listed as "valid" in any network status document that this @@ -1914,8 +2184,6 @@ on the public Tor network. authority publishes, or accepted as an OR address in any descriptor submitted for publication by this authority. -[[AuthDirBadDirCCs]] **AuthDirBadDirCCs** __CC__,... + - [[AuthDirBadExitCCs]] **AuthDirBadExitCCs** __CC__,... + [[AuthDirInvalidCCs]] **AuthDirInvalidCCs** __CC__,... + @@ -1923,28 +2191,15 @@ on the public Tor network. [[AuthDirRejectCCs]] **AuthDirRejectCCs** __CC__,...:: Authoritative directories only. These options contain a comma-separated list of country codes such that any server in one of those country codes - will be marked as a bad directory/bad exit/invalid for use, or rejected + will be marked as a bad exit/invalid for use, or rejected entirely. -[[AuthDirListBadDirs]] **AuthDirListBadDirs** **0**|**1**:: - Authoritative directories only. If set to 1, this directory has some - opinion about which nodes are unsuitable as directory caches. (Do not set - this to 1 unless you plan to list non-functioning directories as bad; - otherwise, you are effectively voting in favor of every declared - directory.) - [[AuthDirListBadExits]] **AuthDirListBadExits** **0**|**1**:: Authoritative directories only. If set to 1, this directory has some opinion about which nodes are unsuitable as exit nodes. (Do not set this to 1 unless you plan to list non-functioning exits as bad; otherwise, you are effectively voting in favor of every declared exit as an exit.) -[[AuthDirRejectUnlisted]] **AuthDirRejectUnlisted** **0**|**1**:: - Authoritative directories only. If set to 1, the directory server rejects - all uploaded server descriptors that aren't explicitly listed in the - fingerprints file. This acts as a "panic button" if we get hit with a Sybil - attack. (Default: 0) - [[AuthDirMaxServersPerAddr]] **AuthDirMaxServersPerAddr** __NUM__:: Authoritative directories only. The maximum number of servers that we will list as acceptable on a single IP address. Set this to "0" for "no limit". @@ -1964,6 +2219,13 @@ on the public Tor network. or more is always sufficient to satisfy the bandwidth requirement for the Guard flag. (Default: 250 KBytes) +[[AuthDirPinKeys]] **AuthDirPinKeys** **0**|**1**:: + Authoritative directories only. If non-zero, do not allow any relay to + publish a descriptor if any other relay has reserved its <Ed25519,RSA> + identity keypair. In all cases, Tor records every keypair it accepts + in a journal if it is new, or if it differs from the most recently + accepted pinning for one of the keys it contains. (Default: 0) + [[BridgePassword]] **BridgePassword** __Password__:: If set, contains an HTTP authenticator that tells a bridge authority to serve all requested bridge information. Used by the (only partially @@ -2015,11 +2277,6 @@ on the public Tor network. that fine-grained information about nodes can be discarded when it hasn't changed for a given amount of time. (Default: 24 hours) -[[VoteOnHidServDirectoriesV2]] **VoteOnHidServDirectoriesV2** **0**|**1**:: - When this option is set in addition to **AuthoritativeDirectory**, Tor - votes on whether to accept relays as hidden service directories. - (Default: 1) - [[AuthDirHasIPv6Connectivity]] **AuthDirHasIPv6Connectivity** **0**|**1**:: Authoritative directories only. When set to 0, OR ports with an IPv6 address are being accepted without reachability testing. @@ -2041,13 +2298,19 @@ The following options are used to configure a hidden service. Store data files for a hidden service in DIRECTORY. Every hidden service must have a separate directory. You may use this option multiple times to specify multiple services. DIRECTORY must be an existing directory. + (Note: in current versions of Tor, if DIRECTORY is a relative path, + it will be relative to current + working directory of Tor instance, not to its DataDirectory. Do not + rely on this behavior; it is not guaranteed to remain the same in future + versions.) [[HiddenServicePort]] **HiddenServicePort** __VIRTPORT__ [__TARGET__]:: Configure a virtual port VIRTPORT for a hidden service. You may use this option multiple times; each time applies to the service using the most - recent hiddenservicedir. By default, this option maps the virtual port to + recent HiddenServiceDir. By default, this option maps the virtual port to the same port on 127.0.0.1 over TCP. You may override the target port, - address, or both by specifying a target of addr, port, or addr:port. + address, or both by specifying a target of addr, port, addr:port, or + **unix:**__path__. (You can specify an IPv6 target as [addr]:port.) You may also have multiple lines with the same VIRTPORT: when a user connects to that VIRTPORT, one of the TARGETs from those lines will be chosen at random. @@ -2074,11 +2337,37 @@ The following options are used to configure a hidden service. found in the hostname file. Clients need to put this authorization data in their configuration file using **HidServAuth**. +[[HiddenServiceAllowUnknownPorts]] **HiddenServiceAllowUnknownPorts** **0**|**1**:: + If set to 1, then connections to unrecognized ports do not cause the + current hidden service to close rendezvous circuits. (Setting this to 0 is + not an authorization mechanism; it is instead meant to be a mild + inconvenience to port-scanners.) (Default: 0) + +[[HiddenServiceMaxStreams]] **HiddenServiceMaxStreams** __N__:: + The maximum number of simultaneous streams (connections) per rendezvous + circuit. (Setting this to 0 will allow an unlimited number of simultanous + streams.) (Default: 0) + +[[HiddenServiceMaxStreamsCloseCircuit]] **HiddenServiceMaxStreamsCloseCircuit** **0**|**1**:: + If set to 1, then exceeding **HiddenServiceMaxStreams** will cause the + offending rendezvous circuit to be torn down, as opposed to stream creation + requests that exceed the limit being silently ignored. (Default: 0) + [[RendPostPeriod]] **RendPostPeriod** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: Every time the specified period elapses, Tor uploads any rendezvous service descriptors to the directory servers. This information is also uploaded whenever it changes. (Default: 1 hour) +[[HiddenServiceDirGroupReadable]] **HiddenServiceDirGroupReadable** **0**|**1**:: + If this option is set to 1, allow the filesystem group to read the + hidden service directory and hostname file. If the option is set to 0, + only owner is able to read the hidden service directory. (Default: 0) + Has no effect on Windows. + +[[HiddenServiceNumIntroductionPoints]] **HiddenServiceNumIntroductionPoints** __NUM__:: + Number of introduction points the hidden service will have. You can't + have more than 10. (Default: 3) + TESTING NETWORK OPTIONS ----------------------- @@ -2097,6 +2386,14 @@ The following options are used for running a testing Tor network. AssumeReachable 1 AuthDirMaxServersPerAddr 0 AuthDirMaxServersPerAuthAddr 0 + ClientBootstrapConsensusAuthorityDownloadSchedule 0, 2, + 4 (for 40 seconds), 8, 16, 32, 60 + ClientBootstrapConsensusFallbackDownloadSchedule 0, 1, + 4 (for 40 seconds), 8, 16, 32, 60 + ClientBootstrapConsensusAuthorityOnlyDownloadSchedule 0, 1, + 4 (for 40 seconds), 8, 16, 32, 60 + ClientBootstrapConsensusMaxDownloadTries 80 + ClientBootstrapConsensusAuthorityOnlyMaxDownloadTries 80 ClientDNSRejectInternalAddresses 0 ClientRejectInternalAddresses 0 CountPrivateBandwidth 1 @@ -2151,7 +2448,7 @@ The following options are used for running a testing Tor network. that **TestingTorNetwork** is set. (Default: 30 minutes) [[TestingEstimatedDescriptorPropagationTime]] **TestingEstimatedDescriptorPropagationTime** __N__ **minutes**|**hours**:: - Clients try downloading router descriptors from directory caches after this + Clients try downloading server descriptors from directory caches after this time. Changing this requires that **TestingTorNetwork** is set. (Default: 10 minutes) @@ -2195,11 +2492,11 @@ The following options are used for running a testing Tor network. 5 minutes) [[TestingConsensusMaxDownloadTries]] **TestingConsensusMaxDownloadTries** __NUM__:: - Try this often to download a consensus before giving up. Changing + Try this many times to download a consensus before giving up. Changing this requires that **TestingTorNetwork** is set. (Default: 8) [[TestingDescriptorMaxDownloadTries]] **TestingDescriptorMaxDownloadTries** __NUM__:: - Try this often to download a router descriptor before giving up. + Try this often to download a server descriptor before giving up. Changing this requires that **TestingTorNetwork** is set. (Default: 8) [[TestingMicrodescMaxDownloadTries]] **TestingMicrodescMaxDownloadTries** __NUM__:: @@ -2210,8 +2507,26 @@ The following options are used for running a testing Tor network. Try this often to download a v3 authority certificate before giving up. Changing this requires that **TestingTorNetwork** is set. (Default: 8) +[[TestingDirAuthVoteExit]] **TestingDirAuthVoteExit** __node__,__node__,__...__:: + A list of identity fingerprints, country codes, and + address patterns of nodes to vote Exit for regardless of their + uptime, bandwidth, or exit policy. See the **ExcludeNodes** + option for more information on how to specify nodes. + + + In order for this option to have any effect, **TestingTorNetwork** + has to be set. See the **ExcludeNodes** option for more + information on how to specify nodes. + +[[TestingDirAuthVoteExitIsStrict]] **TestingDirAuthVoteExitIsStrict** **0**|**1** :: + If True (1), a node will never receive the Exit flag unless it is specified + in the **TestingDirAuthVoteExit** list, regardless of its uptime, bandwidth, + or exit policy. + + + In order for this option to have any effect, **TestingTorNetwork** + has to be set. + [[TestingDirAuthVoteGuard]] **TestingDirAuthVoteGuard** __node__,__node__,__...__:: - A list of identity fingerprints, nicknames, country codes and + A list of identity fingerprints and country codes and address patterns of nodes to vote Guard for regardless of their uptime and bandwidth. See the **ExcludeNodes** option for more information on how to specify nodes. @@ -2219,6 +2534,29 @@ The following options are used for running a testing Tor network. In order for this option to have any effect, **TestingTorNetwork** has to be set. +[[TestingDirAuthVoteGuardIsStrict]] **TestingDirAuthVoteGuardIsStrict** **0**|**1** :: + If True (1), a node will never receive the Guard flag unless it is specified + in the **TestingDirAuthVoteGuard** list, regardless of its uptime and bandwidth. + + + In order for this option to have any effect, **TestingTorNetwork** + has to be set. + +[[TestingDirAuthVoteHSDir]] **TestingDirAuthVoteHSDir** __node__,__node__,__...__:: + A list of identity fingerprints and country codes and + address patterns of nodes to vote HSDir for regardless of their + uptime and DirPort. See the **ExcludeNodes** option for more + information on how to specify nodes. + + + In order for this option to have any effect, **TestingTorNetwork** + must be set. + +[[TestingDirAuthVoteHSDirIsStrict]] **TestingDirAuthVoteHSDirIsStrict** **0**|**1** :: + If True (1), a node will never receive the HSDir flag unless it is specified + in the **TestingDirAuthVoteHSDir** list, regardless of its uptime and DirPort. + + + In order for this option to have any effect, **TestingTorNetwork** + has to be set. + [[TestingEnableConnBwEvent]] **TestingEnableConnBwEvent** **0**|**1**:: If this option is set, then Tor controllers may register for CONN_BW events. Changing this requires that **TestingTorNetwork** is set. @@ -2239,6 +2577,25 @@ The following options are used for running a testing Tor network. authority on a testing network. Overrides the usual default lower bound of 4 KB. (Default: 0) +[[TestingLinkCertLifetime]] **TestingLinkCertLifetime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**|**months**:: + Overrides the default lifetime for the certificates used to authenticate + our X509 link cert with our ed25519 signing key. + (Default: 2 days) + +[[TestingAuthKeyLifetime]] **TestingAuthKeyLifetime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**|**months**:: + Overrides the default lifetime for a signing Ed25519 TLS Link authentication + key. + (Default: 2 days) + +[[TestingLinkKeySlop]] **TestingLinkKeySlop** __N__ **seconds**|**minutes**|**hours** + + +[[TestingAuthKeySlop]] **TestingAuthKeySlop** __N__ **seconds**|**minutes**|**hours** + + +[[TestingSigningKeySlop]] **TestingSigningKeySlop** __N__ **seconds**|**minutes**|**hours**:: + How early before the official expiration of a an Ed25519 signing key do + we replace it and issue a new key? + (Default: 3 hours for link and auth; 1 day for signing.) + SIGNALS ------- @@ -2322,7 +2679,7 @@ __DataDirectory__**/state**:: below). - When the file was last written - What version of Tor generated the state file - - A short history of bandwidth usage, as produced in the router + - A short history of bandwidth usage, as produced in the server descriptors. __DataDirectory__**/bw_accounting**:: @@ -2334,8 +2691,8 @@ __DataDirectory__**/bw_accounting**:: __DataDirectory__**/control_auth_cookie**:: Used for cookie authentication with the controller. Location can be overridden by the CookieAuthFile config option. Regenerated on startup. See - control-spec.txt for details. Only used when cookie authentication is - enabled. + control-spec.txt in https://spec.torproject.org/[torspec] for details. + Only used when cookie authentication is enabled. __DataDirectory__**/lock**:: This file is used to prevent two Tor instances from using same data @@ -2345,6 +2702,61 @@ __DataDirectory__**/lock**:: __DataDirectory__**/keys/***:: Only used by servers. Holds identity keys and onion keys. +__DataDirectory__**/keys/authority_identity_key**:: + A v3 directory authority's master identity key, used to authenticate its + signing key. Tor doesn't use this while it's running. The tor-gencert + program uses this. If you're running an authority, you should keep this + key offline, and not actually put it here. + +__DataDirectory__**/keys/authority_certificate**:: + A v3 directory authority's certificate, which authenticates the authority's + current vote- and consensus-signing key using its master identity key. + Only directory authorities use this file. + +__DataDirectory__**/keys/authority_signing_key**:: + A v3 directory authority's signing key, used to sign votes and consensuses. + Only directory authorities use this file. Corresponds to the + **authority_certificate** cert. + +__DataDirectory__**/keys/legacy_certificate**:: + As authority_certificate: used only when V3AuthUseLegacyKey is set. + See documentation for V3AuthUseLegacyKey. + +__DataDirectory__**/keys/legacy_signing_key**:: + As authority_signing_key: used only when V3AuthUseLegacyKey is set. + See documentation for V3AuthUseLegacyKey. + +__DataDirectory__**/keys/secret_id_key**:: + A relay's RSA1024 permanent identity key, including private and public + components. Used to sign router descriptors, and to sign other keys. + +__DataDirectory__**/keys/ed25519_master_id_public_key**:: + The public part of a relay's Ed25519 permanent identity key. + +__DataDirectory__**/keys/ed25519_master_id_secret_key**:: + The private part of a relay's Ed25519 permanent identity key. This key + is used to sign the medium-term ed25519 signing key. This file can be + kept offline, or kept encrypted. If so, Tor will not be able to generate + new signing keys itself; you'll need to use tor --keygen yourself to do + so. + +__DataDirectory__**/keys/ed25519_signing_secret_key**:: + The private and public components of a relay's medium-term Ed25519 signing + key. This key is authenticated by the Ed25519 master key, in turn + authenticates other keys (and router descriptors). + +__DataDirectory__**/keys/ed25519_signing_cert**:: + The certificate which authenticates "ed25519_signing_secret_key" as + having been signed by the Ed25519 master key. + +__DataDirectory__**/keys/secret_onion_key**:: + A relay's RSA1024 short-term onion key. Used to decrypt old-style ("TAP") + circuit extension requests. + +__DataDirectory__**/keys/secret_onion_key_ntor**:: + A relay's Curve25519 short-term onion key. Used to handle modern ("ntor") + circuit extension requests. + __DataDirectory__**/fingerprint**:: Only used by servers. Holds the fingerprint of the server's identity key. @@ -2352,20 +2764,9 @@ __DataDirectory__**/hashed-fingerprint**:: Only used by bridges. Holds the hashed fingerprint of the bridge's identity key. (That is, the hash of the hash of the identity key.) -__DataDirectory__**/approved-routers**:: - Only for naming authoritative directory servers (see - **NamingAuthoritativeDirectory**). This file lists nickname to identity - bindings. Each line lists a nickname and a fingerprint separated by - whitespace. See your **fingerprint** file in the __DataDirectory__ for an - example line. If the nickname is **!reject** then descriptors from the - given identity (fingerprint) are rejected by this server. If it is - **!invalid** then descriptors are accepted but marked in the directory as - not valid, that is, not recommended. - __DataDirectory__**/v3-status-votes**:: - Only for authoritative directory servers. This file contains status votes - from all the authoritative directory servers and is used to generate the - network consensus document. + Only for v3 authoritative directory servers. This file contains + status votes from all the authoritative directory servers. __DataDirectory__**/unverified-consensus**:: This file contains a network consensus document that has been downloaded, @@ -2377,7 +2778,7 @@ __DataDirectory__**/unverified-microdesc-consensus**:: to check yet. __DataDirectory__**/unparseable-desc**:: - Onion router descriptors that Tor was unable to parse are dumped to this + Onion server descriptors that Tor was unable to parse are dumped to this file. Only used for debugging. __DataDirectory__**/router-stability**:: @@ -2409,6 +2810,11 @@ __DataDirectory__**/stats/conn-stats**:: Only used by servers. This file is used to collect approximate connection history (number of active connections over time). +__DataDirectory__**/networkstatus-bridges**:: + Only used by authoritative bridge directories. Contains information + about bridges that have self-reported themselves to the bridge + authority. + __HiddenServiceDirectory__**/hostname**:: The <base32-encoded-fingerprint>.onion domain name for this hidden service. If the hidden service is restricted to authorized clients only, this file @@ -2427,11 +2833,12 @@ SEE ALSO **https://www.torproject.org/** +**torspec: https://spec.torproject.org ** BUGS ---- -Plenty, probably. Tor is still in development. Please report them. +Plenty, probably. Tor is still in development. Please report them at https://trac.torproject.org/. AUTHORS ------- diff --git a/doc/torrc_format.txt b/doc/torrc_format.txt new file mode 100644 index 0000000000..7a809901d9 --- /dev/null +++ b/doc/torrc_format.txt @@ -0,0 +1,205 @@ + +This document specifies the current format and semantics of the torrc +file, as of July 2015. Note that we make no guarantee about the +stability of this format. If you write something designed for strict +compatibility with this document, please expect us to break it sooner or +later. + +Yes, some of this is quite stupid. My goal here is to explain what it +does, not what it should do. + + - Nick + + + +1. File Syntax + + ; The syntax here is defined an Augmented Backus-Naur form, as + ; specified in RFC5234. + + ; A file is interpreted as every Entry in the file, in order. + TorrcFile = *Line + + Line = BlankLine / Entry + + BlankLine = *WSP OptComment LF + BlankLine =/ *WSP LF + + OptComment = [ Comment ] + + Comment = "#" *NonLF + + ; Each Entry is interpreted as an optional "Magic" flag, a key, and a + ; value. + Entry = *WSP [ Magic ] Key 1*(1*WSP / "\" NL *WSP) Val LF + Entry =/ *WSP [ Magic ] Key *( *WSP / "\" NL *WSP) LF + + Magic = "+" / "/" + + ; Keys are always specified verbatim. They are case insensitive. It + ; is an error to specify a key that Tor does not recognize. + Key = 1*KC + + ; Sadly, every kind of value is decoded differently... + Val = QuotedVal / ContinuedVal / PlainVal + + ; The text of a PlainVal is the text of its PVBody portion, + ; plus the optional trailing backslash. + PlainVal = PVBody [ "\" ] *WSP OptComment + + ; Note that a PVBody is copied verbatim. Slashes are included + ; verbatim. No changes are made. Note that a body may be empty. + PVBody = * (VC / "\" NonLF ) + + ; The text of a ContinuedVal is the text of each of its PVBody + ; sub-elements, in order, concatenated. + ContinuedVal = CVal1 *CVal2 CVal3 + + CVal1 = PVBody "\" LF + CVal2 = PVBody ( "\" LF / Comment LF ) + CVal3 = PVBody + + ; The text of a QuotedVal is decoded as if it were a C string. + QuotedVal = DQ QVBody DQ *WSP Comment + + QVBody = QC + QVBody =/ "\" ( "n" / "r" / "t" / "\" / "'" / DQUOTE ) + QVBOdy =/ "\" ( "x" 2HEXDIG / 1*3OCTDIG ) + + ; Anything besides NUL and LF + NonLF = %x01-%x09 / %x0b - %xff + + OCTDIG = '0' - '7' + + KC = Any character except an isspace() character or '#' or NUL + VC = Any character except '\\', '\n', '#', or NUL + QC = Any character except '\n', '\\', '\"', or NUL + +2. Mid-level Semantics + + + There are four configuration "domains", from lowest to highest priority: + + * Built-in defaults + * The "torrc_defaults" file, if any + * The "torrc" file, if any + * Arguments provided on the command line, if any. + + Normally, values from high-priority domains override low-priority + domains, but see 'magic' below. + + Configuration keys fall into three categories: singletons, lists, and + groups. + + A singleton key may appear at most once in any domain. Its + corresponding value is equal to its value in the highest-priority + domain in which it occurs. + + A list key may appear any number of times in a domain. By default, + its corresponding value is equal to all of the values specified for + it in the highest-priority domain in which it appears. (See 'magic' + below). + + A group key may appear any number of times in a domain. It is + associated with a number of other keys in the same group. The + relative positions of entries with the keys in a single group + matters, but entries with keys not in the group may be freely + interspersed. By default, the group has a value equal to all keys + and values it contains, from the highest-priority domain in which any + of its keys occurs. + + Magic: + + If the '/' flag is specified for an entry, it sets the value for + that entry to an empty list. (This will cause a higher-priority + domain to clear a list from a lower-priority domain, without + actually adding any entries.) + + If the '+' flag is specified for the first entry in a list or a + group that appears in a given domain, that list or group is + appended to the list or group from the next-lowest-priority + domain, rather than replacing it. + +3. High-level semantics + + There are further constraints on the values that each entry can take. + These constraints are out-of-scope for this document. + +4. Examples + + (Indentation is removed in this section, to avoid confusion.) + +4.1. Syntax examples + +# Here is a simple configuration entry. The key is "Foo"; the value is +# "Bar" + +Foo Bar + +# A configuration entry can have spaces in its value, as below. Here the +# key is "Foo" and the value is "Bar Baz" +Foo Bar Baz + +# This configuration entry has space at the end of the line, but those +# spaces don't count, so the key and value are still "Foo" and "Bar Baz" +Foo Bar Baz + +# There can be an escaped newline between the value and the key. This +# is another way to say key="Hello", value="World" +Hello\ +World + +# In regular entries of this kind, you can have a comment at the end of +# the line, either with a space before it or not. Each of these is a +# different spelling of key="Hello", value="World" + +Hello World #today +Hello World#tomorrow + +# One way to encode a complex entry is as a C string. This is the same +# as key="Hello", value="World!" +Hello "World!" + +# The string can contain the usual set of C escapes. This entry has +# key="Hello", and value="\"World\"\nand\nuniverse" +Hello "\"World\"\nand\nuniverse" + +# And now we get to the more-or-less awful part. +# +# Multi-line entries ending with a backslash on each line aren't so +# bad. The backslash is removed, and everything else is included +# verbatim. So this entry has key="Hello" and value="Worldandfriends" +Hello\ +World\ +and\ +friends + +# Backslashes in the middle of a line are included as-is. The key of +# this one is "Too" and the value is "Many\\Backsl\ashes here" (with +# backslashes in that last string as-is) +Too \ +Many\\\ +Backsl\ashes \\ +here + +# And here's the really yucky part. If a comment appears in a multi-line +# entry, the entry is still able to continue on the next line, as in the +# following, where the key is "This" and the value is +# "entry and some are silly" +This entry \ + # has comments \ + and some \ + are # generally \ + silly + +# But you can also write that without the backslashes at the end of the +# comment lines. That is to say, this entry is exactly the same as the +# one above! +This entry \ + # has comments + and some \ + are # generally + silly + + + |