diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/HACKING/CodeStructure.md | 129 | ||||
| -rw-r--r-- | doc/HACKING/CodingStandards.md | 237 | ||||
| -rw-r--r-- | doc/HACKING/CodingStandardsRust.md | 523 | ||||
| -rw-r--r-- | doc/HACKING/Fuzzing.md | 123 | ||||
| -rw-r--r-- | doc/HACKING/GettingStarted.md | 5 | ||||
| -rw-r--r-- | doc/HACKING/GettingStartedRust.md | 183 | ||||
| -rw-r--r-- | doc/HACKING/HelpfulTools.md | 132 | ||||
| -rw-r--r-- | doc/HACKING/HowToReview.md | 3 | ||||
| -rw-r--r-- | doc/HACKING/Module.md | 111 | ||||
| -rw-r--r-- | doc/HACKING/ReleasingTor.md | 135 | ||||
| -rw-r--r-- | doc/HACKING/Tracing.md | 91 | ||||
| -rw-r--r-- | doc/HACKING/WritingTests.md | 12 | ||||
| -rw-r--r-- | doc/HACKING/android/Simpleperf.md | 98 | ||||
| -rw-r--r-- | doc/include.am | 28 | ||||
| -rw-r--r-- | doc/tor-print-ed-signing-cert.1.txt | 32 | ||||
| -rw-r--r-- | doc/tor-resolve.1.txt | 2 | ||||
| -rw-r--r-- | doc/tor.1.txt | 1706 | ||||
| -rw-r--r-- | doc/torify.1.txt | 20 | ||||
| -rw-r--r-- | doc/torrc_format.txt | 15 |
19 files changed, 2843 insertions, 742 deletions
diff --git a/doc/HACKING/CodeStructure.md b/doc/HACKING/CodeStructure.md new file mode 100644 index 0000000000..736d6cd484 --- /dev/null +++ b/doc/HACKING/CodeStructure.md @@ -0,0 +1,129 @@ + +TODO: revise this to talk about how things are, rather than how things +have changed. + +TODO: Make this into good markdown. + + + +For quite a while now, the program "tor" has been built from source +code in just two directories: src/common and src/or. + +This has become more-or-less untenable, for a few reasons -- most +notably of which is that it has led our code to become more +spaghetti-ish than I can endorse with a clean conscience. + +So to fix that, we've gone and done a huge code movement in our git +master branch, which will land in a release once Tor 0.3.5.1-alpha is +out. + +Here's what we did: + + * src/common has been turned into a set of static libraries. These +all live in the "src/lib/*" directories. The dependencies between +these libraries should have no cycles. The libraries are: + + arch -- Headers to handle architectural differences + cc -- headers to handle differences among compilers + compress -- wraps zlib, zstd, lzma + container -- high-level container types + crypt_ops -- Cryptographic operations. Planning to split this into +a higher and lower level library + ctime -- Operations that need to run in constant-time. (Properly, +data-invariant time) + defs -- miscelaneous definitions needed throughout Tor. + encoding -- transforming one data type into another, and various +data types into strings. + err -- lowest-level error handling, in cases where we can't use +the logs because something that the logging system needs has broken. + evloop -- Generic event-loop handling logic + fdio -- Low-level IO wrapper functions for file descriptors. + fs -- Operations on the filesystem + intmath -- low-level integer math and misc bit-twiddling hacks + lock -- low-level locking code + log -- Tor's logging module. This library sits roughly halfway up +the library dependency diagram, since everything it depends on has to +be carefully crafted to *not* log. + malloc -- Low-level wrappers for the platform memory allocation functions. + math -- Higher-level mathematical functions, and floating-point math + memarea -- An arena allocator + meminfo -- Functions for querying the current process's memory +status and resources + net -- Networking compatibility and convenience code + osinfo -- Querying information about the operating system + process -- Launching and querying the status of other processes + sandbox -- Backend for the linux seccomp2 sandbox + smartlist_core -- The lowest-level of the smartlist_t data type. +Separated from the rest of the containers library because the logging +subsystem depends on it. + string -- Compatibility and convenience functions for manipulating +C strings. + term -- Terminal-related functions (currently limited to a getpass +function). + testsupport -- Macros for mocking, unit tests, etc. + thread -- Higher-level thread compatibility code + time -- Higher-level time management code, including format +conversions and monotonic time + tls -- Our wrapper around our TLS library + trace -- Formerly src/trace -- a generic event tracing API + wallclock -- Low-level time code, used by the log module. + + * To ensure that the dependency graph in src/common remains under +control, there is a tool that you can run called "make +check-includes". It verifies that each module in Tor only includes +the headers that it is permitted to include, using a per-directory +".may_include" file. + + * The src/or/or.h header has been split into numerous smaller +headers. Notably, many important structures are now declared in a +header called foo_st.h, where "foo" is the name of the structure. + + * The src/or directory, which had most of Tor's code, had been split +up into several directories. This is still a work in progress: This +code has not itself been refactored, and its dependency graph is still +a tangled web. I hope we'll be working on that over the coming +releases, but it will take a while to do. + + The new top-level source directories are: + + src/core -- Code necessary to actually perform or use onion routing. + src/feature -- Code used only by some onion routing +configurations, or only for a special purpose. + src/app -- Top-level code to run, invoke, and configure the +lower-level code + + The new second-level source directories are: + src/core/crypto -- High-level cryptographic protocols used in Tor + src/core/mainloop -- Tor's event loop, connection-handling, and +traffic-routing code. + src/core/or -- Parts related to handling onion routing itself + src/core/proto -- support for encoding and decoding different +wire protocols + + src/feature/api -- Support for making Tor embeddable + src/feature/client -- Functionality which only Tor clients need + src/feature/control -- Controller implementation + src/feature/dirauth -- Directory authority + src/feature/dircache -- Directory cache + src/feature/dirclient -- Directory client + src/feature/dircommon -- Shared code between the other directory modules + src/feature/hibernate -- Hibernating when Tor is out of bandwidth +or shutting down + src/feature/hs -- v3 onion service implementation + src/feature/hs_common -- shared code between both onion service +implementations + src/feature/nodelist -- storing and accessing the list of relays on +the network. + src/feature/relay -- code that only relay servers and exit servers need. + src/feature/rend -- v2 onion service implementation + src/feature/stats -- statistics and history + + src/app/config -- configuration and state for Tor + src/app/main -- Top-level functions to invoke the rest or Tor. + + * The "tor" executable is now built in src/app/tor rather than src/or/tor. + + * There are more static libraries than before that you need to build +into your application if you want to embed Tor. Rather than +maintaining this list yourself, I recommend that you run "make +show-libs" to have Tor emit a list of what you need to link. diff --git a/doc/HACKING/CodingStandards.md b/doc/HACKING/CodingStandards.md index f1c65850a4..4f229348e4 100644 --- a/doc/HACKING/CodingStandards.md +++ b/doc/HACKING/CodingStandards.md @@ -4,9 +4,10 @@ Coding conventions for Tor tl;dr: - Run configure with `--enable-fatal-warnings` - - Run `make check-spaces` to catch whitespace errors - Document your functions - Write unit tests + - Run `make check` before submitting a patch + - Run `make distcheck` if you have made changes to build system components - Add a file in `changes` for your branch. Patch checklist @@ -22,13 +23,42 @@ preference) Did you remember... - To build your code while configured with `--enable-fatal-warnings`? - - To run `make check-spaces` on your code? - To run `make check-docs` to see whether all new options are on the manpage? - To write unit tests, as possible? + - To run `make test-full` to test against all unit and integration tests (or + `make test-full-online` if you have a working connection to the internet)? + - To test that the distribution will actually work via `make distcheck`? - To base your code on the appropriate branch? - To include a file in the `changes` directory as appropriate? +If you are submitting a major patch or new feature, or want to in the future... + + - Set up Chutney and Stem, see HACKING/WritingTests.md + - Run `make test-full` to test against all unit and integration tests. + +If you have changed build system components: + - Please run `make distcheck` + - For example, if you have changed Makefiles, autoconf files, or anything + else that affects the build system. + +License issues +============== + +Tor is distributed under the license terms in the LICENSE -- in +brief, the "3-clause BSD license". If you send us code to +distribute with Tor, it needs to be code that we can distribute +under those terms. Please don't send us patches unless you agree +to allow this. + +Some compatible licenses include: + + - 3-clause BSD + - 2-clause BSD + - CC0 Public Domain Dedication + + + How we use Git branches ======================= @@ -49,8 +79,17 @@ before it gets merged into maint, but that's rare. If you're working on a bugfix for a bug that occurs in a particular version, base your bugfix branch on the "maint" branch for the first supported series -that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) If -you're working on a new feature, base it on the master branch. +that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) + +If you're working on a new feature, base it on the master branch. If you're +working on a new feature and it will take a while to implement and/or you'd +like to avoid the possibility of unrelated bugs in Tor while you're +implementing your feature, consider branching off of the latest maint- branch. +_Never_ branch off a relase- branch. Don't branch off a tag either: they come +from release branches. Doing so will likely produce a nightmare of merge +conflicts in the ChangeLog when it comes time to merge your branch into Tor. +Best advice: don't try to keep an independent branch forked for more than 6 +months and expect it to merge cleanly. Try to merge pieces early and often. How we log changes @@ -74,17 +113,34 @@ you can use `git describe --contains <sha1 of commit>`. If at all possible, try to create this file in the same commit where you are making the change. Please give it a distinctive name that no other branch will use for the lifetime of your change. To verify the format of the changes file, -you can use `make check-changes`. +you can use `make check-changes`. This is run automatically as part of +`make check` -- if it fails, we must fix it before we release. These +checks are implemented in `scripts/maint/lintChanges.py`. + +Changes file style guide: + * Changes files begin with " o Header (subheading):". The header + should usually be "Minor/Major bugfixes/features". The subheading is a + particular area within Tor. See the ChangeLog for examples. + + * Make everything terse. + + * Write from the user's point of view: describe the user-visible changes + right away. + + * Mention configuration options by name. If they're rare or unusual, + remind people what they're for. + + * Describe changes in the present tense and in the imperative: not past. + + * Every bugfix should have a sentence of the form "Fixes bug 1234; bugfix + on 0.1.2.3-alpha", describing what bug was fixed and where it came from. + + * "Relays", not "servers", "nodes", or "Tor relays". When we go to make a release, we will concatenate all the entries in changes to make a draft changelog, and clear the directory. We'll then edit the draft changelog into a nice readable format. -To make sure that stuff is in the right format, we use -scripts/maint/lintChanges.py to check the changes files for -(superficial) validity. You can run this script on your own changes -files! - What needs a changes file? * A not-exhaustive list: Anything that might change user-visible @@ -93,6 +149,10 @@ What needs a changes file? rewrites. Anything about which somebody might plausibly wonder "when did that happen, and/or why did we do that" 6 months down the line. +What does not need a changes file? + + * Bugfixes for code that hasn't shipped in any released version of Tor + Why use changes files instead of Git commit messages? * Git commit messages are written for developers, not users, and they @@ -112,7 +172,6 @@ deviations from our C whitespace style. Generally, we use: - Unix-style line endings - K&R-style indentation - No space before newlines - - A blank line at the end of each file - Never more than one blank line in a row - Always spaces, never tabs - No more than 79-columns per line. @@ -125,6 +184,9 @@ deviations from our C whitespace style. Generally, we use: `puts (x)`. - Function declarations at the start of the line. +If you use an editor that has plugins for editorconfig.org, the file +`.editorconfig` will help you to conform this coding style. + We try hard to build without warnings everywhere. In particular, if you're using gcc, you should invoke the configure script with the option `--enable-fatal-warnings`. This will tell the compiler @@ -138,8 +200,8 @@ We have some wrapper functions like `tor_malloc`, `tor_free`, `tor_strdup`, and always succeed or exit.) You can get a full list of the compatibility functions that Tor provides by -looking through `src/common/util*.h` and `src/common/compat*.h`. You can see the -available containers in `src/common/containers*.h`. You should probably +looking through `src/lib/*/*.h`. You can see the +available containers in `src/lib/containers/*.h`. You should probably familiarize yourself with these modules before you write too much code, or else you'll wind up reinventing the wheel. @@ -149,6 +211,97 @@ old C functions. Use `strlcat`, `strlcpy`, or `tor_snprintf/tor_asprintf` inste We don't call `memcmp()` directly. Use `fast_memeq()`, `fast_memneq()`, `tor_memeq()`, or `tor_memneq()` for most purposes. +Also see a longer list of functions to avoid in: +https://people.torproject.org/~nickm/tor-auto/internal/this-not-that.html + +What code can use what other code? +---------------------------------- + +We're trying to simplify Tor's structure over time. In the long run, we want +Tor to be structured as a set of modules with *no circular dependencies*. + +This property is currently provided by the modules in src/lib, but not +throughout the rest of Tor. In general, higher-level libraries may use +lower-level libraries, but never the reverse. + +To prevent new circular dependencies from landing, we have a tool that +you can invoke with `make check-includes`, and which is run +automatically as part of `make check`. This tool will verify that, for +every source directory with a `.may_include` file, no local headers are +included except those specifically permitted by the `.may_include` file. +When editing one of these files, please make sure that you are not +introducing any cycles into Tor's dependency graph. + +Floating point math is hard +--------------------------- + +Floating point arithmetic as typically implemented by computers is +very counterintuitive. Failure to adequately analyze floating point +usage can result in surprising behavior and even security +vulnerabilities! + +General advice: + + - Don't use floating point. + - If you must use floating point, document how the limits of + floating point precision and calculation accuracy affect function + outputs. + - Try to do as much as possible of your calculations using integers + (possibly acting as fixed-point numbers) and convert to floating + point for display. + - If you must send floating point numbers on the wire, serialize + them in a platform-independent way. Tor avoids exchanging + floating-point values, but when it does, it uses ASCII numerals, + with a decimal point ("."). + - Binary fractions behave very differently from decimal fractions. + Make sure you understand how these differences affect your + calculations. + - Every floating point arithmetic operation is an opportunity to + lose precision, overflow, underflow, or otherwise produce + undesired results. Addition and subtraction tend to be worse + than multiplication and division (due to things like catastrophic + cancellation). Try to arrange your calculations to minimize such + effects. + - Changing the order of operations changes the results of many + floating-point calculations. Be careful when you simplify + calculations! If the order is significant, document it using a + code comment. + - Comparing most floating point values for equality is unreliable. + Avoid using `==`, instead, use `>=` or `<=`. If you use an + epsilon value, make sure it's appropriate for the ranges in + question. + - Different environments (including compiler flags and per-thread + state on a single platform!) can get different results from the + same floating point calculations. This means you can't use + floats in anything that needs to be deterministic, like consensus + generation. This also makes reliable unit tests of + floating-point outputs hard to write. + +For additional useful advice (and a little bit of background), see +[What Every Programmer Should Know About Floating-Point +Arithmetic](http://floating-point-gui.de/). + +A list of notable (and surprising) facts about floating point +arithmetic is at [Floating-point +complexities](https://randomascii.wordpress.com/2012/04/05/floating-point-complexities/). +Most of that [series of posts on floating +point](https://randomascii.wordpress.com/category/floating-point/) is +helpful. + +For more detailed (and math-intensive) background, see [What Every +Computer Scientist Should Know About Floating-Point +Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html). + +Other C conventions +------------------- + +The `a ? b : c` trinary operator only goes inside other expressions; +don't use it as a replacement for if. (You can ignore this inside macro +definitions when necessary.) + +Assignment operators shouldn't nest inside other expressions. (You can +ignore this inside macro definitions when necessary.) + Functions not to write ---------------------- @@ -210,6 +363,64 @@ end-users that they aren't expected to understand the message (perhaps with a string like "internal error"). Option (A) is to be preferred to option (B). +Assertions In Tor +----------------- + +Assertions should be used for bug-detection only. Don't use assertions to +detect bad user inputs, network errors, resource exhaustion, or similar +issues. + +Tor is always built with assertions enabled, so try to only use +`tor_assert()` for cases where you are absolutely sure that crashing is the +least bad option. Many bugs have been caused by use of `tor_assert()` when +another kind of check would have been safer. + +If you're writing an assertion to test for a bug that you _can_ recover from, +use `tor_assert_nonfatal()` in place of `tor_assert()`. If you'd like to +write a conditional that incorporates a nonfatal assertion, use the `BUG()` +macro, as in: + + if (BUG(ptr == NULL)) + return -1; + +Allocator conventions +--------------------- + +By convention, any tor type with a name like `abc_t` should be allocated +by a function named `abc_new()`. This function should never return +NULL. + +Also, a type named `abc_t` should be freed by a function named `abc_free_()`. +Don't call this `abc_free_()` function directly -- instead, wrap it in a +macro called `abc_free()`, using the `FREE_AND_NULL` macro: + + void abc_free_(abc_t *obj); + #define abc_free(obj) FREE_AND_NULL(abc_t, abc_free_, (obj)) + +This macro will free the underlying `abc_t` object, and will also set +the object pointer to NULL. + +You should define all `abc_free_()` functions to accept NULL inputs: + + void + abc_free_(abc_t *obj) + { + if (!obj) + return; + tor_free(obj->name); + thing_free(obj->thing); + tor_free(obj); + } + +If you need a free function that takes a `void *` argument (for example, +to use it as a function callback), define it with a name like +`abc_free_void()`: + + static void + abc_free_void_(void *obj) + { + abc_free_(obj); + } Doxygen comment conventions diff --git a/doc/HACKING/CodingStandardsRust.md b/doc/HACKING/CodingStandardsRust.md new file mode 100644 index 0000000000..fc562816db --- /dev/null +++ b/doc/HACKING/CodingStandardsRust.md @@ -0,0 +1,523 @@ + + Rust Coding Standards +======================= + +You MUST follow the standards laid out in `.../doc/HACKING/CodingStandards.md`, +where applicable. + + Module/Crate Declarations +--------------------------- + +Each Tor C module which is being rewritten MUST be in its own crate. +See the structure of `.../src/rust` for examples. + +In your crate, you MUST use `lib.rs` ONLY for pulling in external +crates (e.g. `extern crate libc;`) and exporting public objects from +other Rust modules (e.g. `pub use mymodule::foo;`). For example, if +you create a crate in `.../src/rust/yourcrate`, your Rust code should +live in `.../src/rust/yourcrate/yourcode.rs` and the public interface +to it should be exported in `.../src/rust/yourcrate/lib.rs`. + +If your code is to be called from Tor C code, you MUST define a safe +`ffi.rs`. See the "Safety" section further down for more details. + +For example, in a hypothetical `tor_addition` Rust module: + +In `.../src/rust/tor_addition/addition.rs`: + + pub fn get_sum(a: i32, b: i32) -> i32 { + a + b + } + +In `.../src/rust/tor_addition/lib.rs`: + + pub use addition::*; + +In `.../src/rust/tor_addition/ffi.rs`: + + #[no_mangle] + pub extern "C" fn tor_get_sum(a: c_int, b: c_int) -> c_int { + get_sum(a, b) + } + +If your Rust code must call out to parts of Tor's C code, you must +declare the functions you are calling in the `external` crate, located +at `.../src/rust/external`. + +<!-- XXX get better examples of how to declare these externs, when/how they --> +<!-- XXX are unsafe, what they are expected to do —isis --> + +Modules should strive to be below 500 lines (tests excluded). Single +responsibility and limited dependencies should be a guiding standard. + +If you have any external modules as dependencies (e.g. `extern crate +libc;`), you MUST declare them in your crate's `lib.rs` and NOT in any +other module. + + Dependencies and versions +--------------------------- + +In general, we use modules from only the Rust standard library +whenever possible. We will review including external crates on a +case-by-case basis. + +If a crate only contains traits meant for compatibility between Rust +crates, such as [the digest crate](https://crates.io/crates/digest) or +[the failure crate](https://crates.io/crates/failure), it is very likely +permissible to add it as a dependency. However, a brief review should +be conducted as to the usefulness of implementing external traits +(i.e. how widespread is the usage, how many other crates either +implement the traits or have trait bounds based upon them), as well as +the stability of the traits (i.e. if the trait is going to change, we'll +potentially have to re-do all our implementations of it). + +For large external libraries, especially which implement features which +would be labour-intensive to reproduce/maintain ourselves, such as +cryptographic or mathematical/statistics libraries, only crates which +have stabilised to 1.0.0 should be considered, however, again, we may +make exceptions on a case-by-case basis. + +Currently, Tor requires that you use the latest stable Rust version. At +some point in the future, we will freeze on a given stable Rust version, +to ensure backward compatibility with stable distributions that ship it. + + Updating/Adding Dependencies +------------------------------ + +To add/remove/update dependencies, first add your dependencies, +exactly specifying their versions, into the appropriate *crate-level* +`Cargo.toml` in `src/rust/` (i.e. *not* `/src/rust/Cargo.toml`, but +instead the one for your crate). Also, investigate whether your +dependency has any optional dependencies which are unnecessary but are +enabled by default. If so, you'll likely be able to enable/disable +them via some feature, e.g.: + +```toml +[dependencies] +foo = { version = "1.0.0", default-features = false } +``` + +Next, run `/scripts/maint/updateRustDependencies.sh`. Then, go into +`src/ext/rust` and commit the changes to the `tor-rust-dependencies` +repo. + + Documentation +--------------- + +You MUST include `#![deny(missing_docs)]` in your crate. + +For function/method comments, you SHOULD include a one-sentence, "first person" +description of function behaviour (see requirements for documentation as +described in `.../src/HACKING/CodingStandards.md`), then an `# Inputs` section +for inputs or initialisation values, a `# Returns` section for return +values/types, a `# Warning` section containing warnings for unsafe behaviours or +panics that could happen. For publicly accessible +types/constants/objects/functions/methods, you SHOULD also include an +`# Examples` section with runnable doctests. + +You MUST document your module with _module docstring_ comments, +i.e. `//!` at the beginning of each line. + + Style +------- + +You SHOULD consider breaking up large literal numbers with `_` when it makes it +more human readable to do so, e.g. `let x: u64 = 100_000_000_000`. + + Testing +--------- + +All code MUST be unittested and integration tested. + +Public functions/objects exported from a crate SHOULD include doctests +describing how the function/object is expected to be used. + +Integration tests SHOULD go into a `tests/` directory inside your +crate. Unittests SHOULD go into their own module inside the module +they are testing, e.g. in `.../src/rust/tor_addition/addition.rs` you +should put: + + #[cfg(test)] + mod test { + use super::*; + + #[test] + fn addition_with_zero() { + let sum: i32 = get_sum(5i32, 0i32); + assert_eq!(sum, 5); + } + } + + Benchmarking +-------------- + +The external `test` crate can be used for most benchmarking. However, using +this crate requires nightly Rust. Since we may want to switch to a more +stable Rust compiler eventually, we shouldn't do things which will automatically +break builds for stable compilers. Therefore, you MUST feature-gate your +benchmarks in the following manner. + +If you wish to benchmark some of your Rust code, you MUST put the +following in the `[features]` section of your crate's `Cargo.toml`: + + [features] + bench = [] + +Next, in your crate's `lib.rs` you MUST put: + + #[cfg(all(test, feature = "bench"))] + extern crate test; + +This ensures that the external crate `test`, which contains utilities +for basic benchmarks, is only used when running benchmarks via `cargo +bench --features bench`. + +Finally, to write your benchmark code, in +`.../src/rust/tor_addition/addition.rs` you SHOULD put: + + #[cfg(all(test, features = "bench"))] + mod bench { + use test::Bencher; + use super::*; + + #[bench] + fn addition_small_integers(b: &mut Bencher) { + b.iter(| | get_sum(5i32, 0i32)); + } + } + + Fuzzing +--------- + +If you wish to fuzz parts of your code, please see the +[`cargo fuzz`](https://github.com/rust-fuzz/cargo-fuzz) crate, which uses +[libfuzzer-sys](https://github.com/rust-fuzz/libfuzzer-sys). + + Whitespace & Formatting +------------------------- + +You MUST run `rustfmt` (https://github.com/rust-lang-nursery/rustfmt) +on your code before your code will be merged. You can install rustfmt +by doing `cargo install rustfmt-nightly` and then run it with `cargo +fmt`. + + Safety +-------- + +You SHOULD read [the nomicon](https://doc.rust-lang.org/nomicon/) before writing +Rust FFI code. It is *highly advised* that you read and write normal Rust code +before attempting to write FFI or any other unsafe code. + +Here are some additional bits of advice and rules: + +0. Any behaviours which Rust considers to be undefined are forbidden + + From https://doc.rust-lang.org/reference/behavior-considered-undefined.html: + + > Behavior considered undefined + > + > The following is a list of behavior which is forbidden in all Rust code, + > including within unsafe blocks and unsafe functions. Type checking provides the + > guarantee that these issues are never caused by safe code. + > + > * Data races + > * Dereferencing a null/dangling raw pointer + > * Reads of [undef](http://llvm.org/docs/LangRef.html#undefined-values) + > (uninitialized) memory + > * Breaking the + > [pointer aliasing rules](http://llvm.org/docs/LangRef.html#pointer-aliasing-rules) + > with raw pointers (a subset of the rules used by C) + > * `&mut T` and `&T` follow LLVM’s scoped noalias model, except if the `&T` + > contains an `UnsafeCell<U>`. Unsafe code must not violate these aliasing + > guarantees. + > * Mutating non-mutable data (that is, data reached through a shared + > reference or data owned by a `let` binding), unless that data is + > contained within an `UnsafeCell<U>`. + > * Invoking undefined behavior via compiler intrinsics: + > - Indexing outside of the bounds of an object with + > `std::ptr::offset` (`offset` intrinsic), with the exception of + > one byte past the end which is permitted. + > - Using `std::ptr::copy_nonoverlapping_memory` (`memcpy32`/`memcpy64` + > intrinsics) on overlapping buffers + > * Invalid values in primitive types, even in private fields/locals: + > - Dangling/null references or boxes + > - A value other than `false` (0) or `true` (1) in a `bool` + > - A discriminant in an `enum` not included in the type definition + > - A value in a `char` which is a surrogate or above `char::MAX` + > - Non-UTF-8 byte sequences in a `str` + > * Unwinding into Rust from foreign code or unwinding from Rust into foreign + > code. Rust's failure system is not compatible with exception handling in other + > languages. Unwinding must be caught and handled at FFI boundaries. + +1. `unwrap()` + + If you call `unwrap()`, anywhere, even in a test, you MUST include + an inline comment stating how the unwrap will either 1) never fail, + or 2) should fail (i.e. in a unittest). + + You SHOULD NOT use `unwrap()` anywhere in which it is possible to handle the + potential error with either `expect()` or the eel operator, `?`. + For example, consider a function which parses a string into an integer: + + fn parse_port_number(config_string: &str) -> u16 { + u16::from_str_radix(config_string, 10).unwrap() + } + + There are numerous ways this can fail, and the `unwrap()` will cause the + whole program to byte the dust! Instead, either you SHOULD use `expect()` + (or another equivalent function which will return an `Option` or a `Result`) + and change the return type to be compatible: + + fn parse_port_number(config_string: &str) -> Option<u16> { + u16::from_str_radix(config_string, 10).expect("Couldn't parse port into a u16") + } + + or you SHOULD use `or()` (or another similar method): + + fn parse_port_number(config_string: &str) -> Option<u16> { + u16::from_str_radix(config_string, 10).or(Err("Couldn't parse port into a u16") + } + + Using methods like `or()` can be particularly handy when you must do + something afterwards with the data, for example, if we wanted to guarantee + that the port is high. Combining these methods with the eel operator (`?`) + makes this even easier: + + fn parse_port_number(config_string: &str) -> Result<u16, Err> { + let port = u16::from_str_radix(config_string, 10).or(Err("Couldn't parse port into a u16"))?; + + if port > 1024 { + return Ok(port); + } else { + return Err("Low ports not allowed"); + } + } + +2. `unsafe` + + If you use `unsafe`, you MUST describe a contract in your + documentation which describes how and when the unsafe code may + fail, and what expectations are made w.r.t. the interfaces to + unsafe code. This is also REQUIRED for major pieces of FFI between + C and Rust. + + When creating an FFI in Rust for C code to call, it is NOT REQUIRED + to declare the entire function `unsafe`. For example, rather than doing: + + #[no_mangle] + pub unsafe extern "C" fn increment_and_combine_numbers(mut numbers: [u8; 4]) -> u32 { + for number in &mut numbers { + *number += 1; + } + std::mem::transmute::<[u8; 4], u32>(numbers) + } + + You SHOULD instead do: + + #[no_mangle] + pub extern "C" fn increment_and_combine_numbers(mut numbers: [u8; 4]) -> u32 { + for index in 0..numbers.len() { + numbers[index] += 1; + } + unsafe { + std::mem::transmute::<[u8; 4], u32>(numbers) + } + } + +3. Pass only C-compatible primitive types and bytes over the boundary + + Rust's C-compatible primitive types are integers and floats. + These types are declared in the [libc crate](https://doc.rust-lang.org/libc/x86_64-unknown-linux-gnu/libc/index.html#types). + Most Rust objects have different [representations](https://doc.rust-lang.org/libc/x86_64-unknown-linux-gnu/libc/index.html#types) + in C and Rust, so they can't be passed using FFI. + + Tor currently uses the following Rust primitive types from libc for FFI: + * defined-size integers: `uint32_t` + * native-sized integers: `c_int` + * native-sized floats: `c_double` + * native-sized raw pointers: `* c_void`, `* c_char`, `** c_char` + + TODO: C smartlist to Stringlist conversion using FFI + + The only non-primitive type which may cross the FFI boundary is + bytes, e.g. `&[u8]`. This SHOULD be done on the Rust side by + passing a pointer (`*mut libc::c_char`). The length can be passed + explicitly (`libc::size_t`), or the string can be NUL-byte terminated + C string. + + One might be tempted to do this via doing + `CString::new("blah").unwrap().into_raw()`. This has several problems: + + a) If you do `CString::new("bl\x00ah")` then the unwrap() will fail + due to the additional NULL terminator, causing a dangling + pointer to be returned (as well as a potential use-after-free). + + b) Returning the raw pointer will cause the CString to run its deallocator, + which causes any C code which tries to access the contents to dereference a + NULL pointer. + + c) If we were to do `as_raw()` this would result in a potential double-free + since the Rust deallocator would run and possibly Tor's deallocator. + + d) Calling `into_raw()` without later using the same pointer in Rust to call + `from_raw()` and then deallocate in Rust can result in a + [memory leak](https://doc.rust-lang.org/std/ffi/struct.CString.html#method.into_raw). + + [It was determined](https://github.com/rust-lang/rust/pull/41074) that this + is safe to do if you use the same allocator in C and Rust and also specify + the memory alignment for CString (except that there is no way to specify + the alignment for CString). It is believed that the alignment is always 1, + which would mean it's safe to dealloc the resulting `*mut c_char` in Tor's + C code. However, the Rust developers are not willing to guarantee the + stability of, or a contract for, this behaviour, citing concerns that this + is potentially extremely and subtly unsafe. + +4. Perform an allocation on the other side of the boundary + + After crossing the boundary, the other side MUST perform an + allocation to copy the data and is therefore responsible for + freeing that memory later. + +5. No touching other language's enums + + Rust enums should never be touched from C (nor can they be safely + `#[repr(C)]`) nor vice versa: + + > "The chosen size is the default enum size for the target platform's C + > ABI. Note that enum representation in C is implementation defined, so this is + > really a "best guess". In particular, this may be incorrect when the C code + > of interest is compiled with certain flags." + + (from https://gankro.github.io/nomicon/other-reprs.html) + +6. Type safety + + Wherever possible and sensical, you SHOULD create new types in a + manner which prevents type confusion or misuse. For example, + rather than using an untyped mapping between strings and integers + like so: + + use std::collections::HashMap; + + pub fn get_elements_with_over_9000_points(map: &HashMap<String, usize>) -> Vec<String> { + ... + } + + It would be safer to define a new type, such that some other usage + of `HashMap<String, usize>` cannot be confused for this type: + + pub struct DragonBallZPowers(pub HashMap<String, usize>); + + impl DragonBallZPowers { + pub fn over_nine_thousand<'a>(&'a self) -> Vec<&'a String> { + let mut powerful_enough: Vec<&'a String> = Vec::with_capacity(5); + + for (character, power) in &self.0 { + if *power > 9000 { + powerful_enough.push(character); + } + } + powerful_enough + } + } + + Note the following code, which uses Rust's type aliasing, is valid + but it does NOT meet the desired type safety goals: + + pub type Power = usize; + + pub fn over_nine_thousand(power: &Power) -> bool { + if *power > 9000 { + return true; + } + false + } + + // We can still do the following: + let his_power: usize = 9001; + over_nine_thousand(&his_power); + +7. Unsafe mucking around with lifetimes + + Because lifetimes are technically, in type theory terms, a kind, i.e. a + family of types, individual lifetimes can be treated as types. For example, + one can arbitrarily extend and shorten lifetime using `std::mem::transmute`: + + struct R<'a>(&'a i32); + + unsafe fn extend_lifetime<'b>(r: R<'b>) -> R<'static> { + std::mem::transmute::<R<'b>, R<'static>>(r) + } + + unsafe fn shorten_invariant_lifetime<'b, 'c>(r: &'b mut R<'static>) -> &'b mut R<'c> { + std::mem::transmute::<&'b mut R<'static>, &'b mut R<'c>>(r) + } + + Calling `extend_lifetime()` would cause an `R` passed into it to live forever + for the life of the program (the `'static` lifetime). Similarly, + `shorten_invariant_lifetime()` could be used to take something meant to live + forever, and cause it to disappear! This is incredibly unsafe. If you're + going to be mucking around with lifetimes like this, first, you better have + an extremely good reason, and second, you may as be honest and explicit about + it, and for ferris' sake just use a raw pointer. + + In short, just because lifetimes can be treated like types doesn't mean you + should do it. + +8. Doing excessively unsafe things when there's a safer alternative + + Similarly to #7, often there are excessively unsafe ways to do a task and a + simpler, safer way. You MUST choose the safer option where possible. + + For example, `std::mem::transmute` can be abused in ways where casting with + `as` would be both simpler and safer: + + // Don't do this + let ptr = &0; + let ptr_num_transmute = unsafe { std::mem::transmute::<&i32, usize>(ptr)}; + + // Use an `as` cast instead + let ptr_num_cast = ptr as *const i32 as usize; + + In fact, using `std::mem::transmute` for *any* reason is a code smell and as + such SHOULD be avoided. + +9. Casting integers with `as` + + This is generally fine to do, but it has some behaviours which you should be + aware of. Casting down chops off the high bits, e.g.: + + let x: u32 = 4294967295; + println!("{}", x as u16); // prints 65535 + + Some cases which you MUST NOT do include: + + * Casting an `u128` down to an `f32` or vice versa (e.g. + `u128::MAX as f32` but this isn't only a problem with overflowing + as it is also undefined behaviour for `42.0f32 as u128`), + + * Casting between integers and floats when the thing being cast + cannot fit into the type it is being casted into, e.g.: + + println!("{}", 42949.0f32 as u8); // prints 197 in debug mode and 0 in release + println!("{}", 1.04E+17 as u8); // prints 0 in both modes + println!("{}", (0.0/0.0) as i64); // prints whatever the heck LLVM wants + + Because this behaviour is undefined, it can even produce segfaults in + safe Rust code. For example, the following program built in release + mode segfaults: + + #[inline(never)] + pub fn trigger_ub(sl: &[u8; 666]) -> &[u8] { + // Note that the float is out of the range of `usize`, invoking UB when casting. + let idx = 1e99999f64 as usize; + &sl[idx..] // The bound check is elided due to `idx` being of an undefined value. + } + + fn main() { + println!("{}", trigger_ub(&[1; 666])[999999]); // ~ out of bound + } + + And in debug mode panics with: + + thread 'main' panicked at 'slice index starts at 140721821254240 but ends at 666', /checkout/src/libcore/slice/mod.rs:754:4 diff --git a/doc/HACKING/Fuzzing.md b/doc/HACKING/Fuzzing.md new file mode 100644 index 0000000000..2039d6a4c0 --- /dev/null +++ b/doc/HACKING/Fuzzing.md @@ -0,0 +1,123 @@ += Fuzzing Tor + +== The simple version (no fuzzing, only tests) + +Check out fuzzing-corpora, and set TOR_FUZZ_CORPORA to point to the place +where you checked it out. + +To run the fuzzing test cases in a deterministic fashion, use: + make test-fuzz-corpora + +This won't actually fuzz Tor! It will just run all the fuzz binaries +on our existing set of testcases for the fuzzer. + + +== Different kinds of fuzzing + +Right now we support three different kinds of fuzzer. + +First, there's American Fuzzy Lop (AFL), a fuzzer that works by forking +a target binary and passing it lots of different inputs on stdin. It's the +trickiest one to set up, so I'll be describing it more below. + +Second, there's libFuzzer, a llvm-based fuzzer that you link in as a library, +and it runs a target function over and over. To use this one, you'll need to +have a reasonably recent clang and libfuzzer installed. At that point, you +just build with --enable-expensive-hardening and --enable-libfuzzer. That +will produce a set of binaries in src/test/fuzz/lf-fuzz-* . These programs +take as input a series of directories full of fuzzing examples. For more +information on libfuzzer, see http://llvm.org/docs/LibFuzzer.html + +Third, there's Google's OSS-Fuzz infrastructure, which expects to get all of +its. For more on this, see https://github.com/google/oss-fuzz and the +projects/tor subdirectory. You'll need to mess around with Docker a bit to +test this one out; it's meant to run on Google's infrastructure. + +In all cases, you'll need some starting examples to give the fuzzer when it +starts out. There's a set in the "fuzzing-corpora" git repository. Try +setting TOR_FUZZ_CORPORA to point to a checkout of that repository + +== Writing Tor fuzzers + +A tor fuzzing harness should have: +* a fuzz_init() function to set up any necessary global state. +* a fuzz_main() function to receive input and pass it to a parser. +* a fuzz_cleanup() function to clear global state. + +Most fuzzing frameworks will produce many invalid inputs - a tor fuzzing +harness should rejecting invalid inputs without crashing or behaving badly. + +But the fuzzing harness should crash if tor fails an assertion, triggers a +bug, or accesses memory it shouldn't. This helps fuzzing frameworks detect +"interesting" cases. + + +== Guided Fuzzing with AFL + +There is no HTTPS, hash, or signature for American Fuzzy Lop's source code, so +its integrity can't be verified. That said, you really shouldn't fuzz on a +machine you care about, anyway. + +To Build: + Get AFL from http://lcamtuf.coredump.cx/afl/ and unpack it + cd afl + make + cd ../tor + PATH=$PATH:../afl/ CC="../afl/afl-gcc" ./configure --enable-expensive-hardening + AFL_HARDEN=1 make clean fuzzers + +To Find The ASAN Memory Limit: (64-bit only) + +On 64-bit platforms, afl needs to know how much memory ASAN uses, +because ASAN tends to allocate a ridiculous amount of virtual memory, +and then not actually use it. + +Read afl/docs/notes_for_asan.txt for more details. + + Download recidivm from http://jwilk.net/software/recidivm + Download the signature + Check the signature + tar xvzf recidivm*.tar.gz + cd recidivm* + make + /path/to/recidivm -v src/test/fuzz/fuzz-http + Use the final "ok" figure as the input to -m when calling afl-fuzz + (Normally, recidivm would output a figure automatically, but in some cases, + the fuzzing harness will hang when the memory limit is too small.) + +You could also just say "none" instead of the memory limit below, if you +don't care about memory limits. + + +To Run: + mkdir -p src/test/fuzz/fuzz_http_findings + ../afl/afl-fuzz -i ${TOR_FUZZ_CORPORA}/http -o src/test/fuzz/fuzz_http_findings -m <asan-memory-limit> -- src/test/fuzz/fuzz-http + + +AFL has a multi-core mode, check the documentation for details. +You might find the included fuzz-multi.sh script useful for this. + +macOS (OS X) requires slightly more preparation, including: +* using afl-clang (or afl-clang-fast from the llvm directory) +* disabling external crash reporting (AFL will guide you through this step) + +== Triaging Issues + +Crashes are usually interesting, particularly if using AFL_HARDEN=1 and --enable-expensive-hardening. Sometimes crashes are due to bugs in the harness code. + +Hangs might be interesting, but they might also be spurious machine slowdowns. +Check if a hang is reproducible before reporting it. Sometimes, processing +valid inputs may take a second or so, particularly with the fuzzer and +sanitizers enabled. + +To see what fuzz-http is doing with a test case, call it like this: + src/test/fuzz/fuzz-http --debug < /path/to/test.case + +(Logging is disabled while fuzzing to increase fuzzing speed.) + +== Reporting Issues + +Please report any issues discovered using the process in Tor's security issue +policy: + +https://trac.torproject.org/projects/tor/wiki/org/meetings/2016SummerDevMeeting/Notes/SecurityIssuePolicy diff --git a/doc/HACKING/GettingStarted.md b/doc/HACKING/GettingStarted.md index 0295adc1ff..0c42404634 100644 --- a/doc/HACKING/GettingStarted.md +++ b/doc/HACKING/GettingStarted.md @@ -11,8 +11,9 @@ whole Tor ecosystem.) If you are looking for a more bare-bones, less user-friendly information -dump of important information, you might like reading doc/HACKING -instead. You should probably read it before you write your first patch. +dump of important information, you might like reading the "torguts" +documents linked to below. You should probably read it before you write +your first patch. Required background diff --git a/doc/HACKING/GettingStartedRust.md b/doc/HACKING/GettingStartedRust.md new file mode 100644 index 0000000000..aa29c097da --- /dev/null +++ b/doc/HACKING/GettingStartedRust.md @@ -0,0 +1,183 @@ + + Hacking on Rust in Tor +======================== + + Getting Started +----------------- + +Please read or review our documentation on Rust coding standards +(`.../doc/HACKING/CodingStandardsRust.md`) before doing anything. + +Please also read +[the Rust Code of Conduct](https://www.rust-lang.org/en-US/conduct.html). We +aim to follow the good example set by the Rust community and be +excellent to one another. Let's be careful with each other, so we can +be memory-safe together! + +Next, please contact us before rewriting anything! Rust in Tor is still +an experiment. It is an experiment that we very much want to see +succeed, so we're going slowly and carefully. For the moment, it's also +a completely volunteer-driven effort: while many, if not most, of us are +paid to work on Tor, we are not yet funded to write Rust code for Tor. +Please be patient with the other people who are working on getting more +Rust code into Tor, because they are graciously donating their free time +to contribute to this effort. + + Resources for learning Rust +----------------------------- + +**Beginning resources** + +The primary resource for learning Rust is +[The Book](https://doc.rust-lang.org/book/). If you'd like to start writing +Rust immediately, without waiting for anything to install, there is +[an interactive browser-based playground](https://play.rust-lang.org/). + +**Advanced resources** + +If you're interested in playing with various Rust compilers and viewing +a very nicely displayed output of the generated assembly, there is +[the Godbolt compiler explorer](https://rust.godbolt.org/) + +For learning how to write unsafe Rust, read +[The Rustonomicon](https://doc.rust-lang.org/nomicon/). + +For learning everything you ever wanted to know about Rust macros, there +is +[The Little Book of Rust Macros](https://danielkeep.github.io/tlborm/book/index.html). + +For learning more about FFI and Rust, see Jake Goulding's +[Rust FFI Omnibus](http://jakegoulding.com/rust-ffi-omnibus/). + + Compiling Tor with Rust enabled +--------------------------------- + +You will need to run the `configure` script with the `--enable-rust` +flag to explicitly build with Rust. Additionally, you will need to +specify where to fetch Rust dependencies, as we allow for either +fetching dependencies from Cargo or specifying a local directory. + +**Fetch dependencies from Cargo** + + ./configure --enable-rust --enable-cargo-online-mode + +**Using a local dependency cache** + +You'll need the following Rust dependencies (as of this writing): + + libc==0.2.39 + +We vendor our Rust dependencies in a separate repo using +[cargo-vendor](https://github.com/alexcrichton/cargo-vendor). To use +them, do: + + git submodule init + git submodule update + +To specify the local directory containing the dependencies, (assuming +you are in the top level of the repository) configure tor with: + + TOR_RUST_DEPENDENCIES='path_to_dependencies_directory' ./configure --enable-rust + +(Note that TOR_RUST_DEPENDENCIES must be the full path to the directory; it +cannot be relative.) + +Assuming you used the above `git submodule` commands and you're in the +topmost directory of the repository, this would be: + + TOR_RUST_DEPENDENCIES=`pwd`/src/ext/rust/crates ./configure --enable-rust + + + Identifying which modules to rewrite +====================================== + +The places in the Tor codebase that are good candidates for porting to +Rust are: + +1. loosely coupled to other Tor submodules, +2. have high test coverage, and +3. would benefit from being implemented in a memory safe language. + +Help in either identifying places such as this, or working to improve +existing areas of the C codebase by adding regression tests and +simplifying dependencies, would be really helpful. + +Furthermore, as submodules in C are implemented in Rust, this is a good +opportunity to refactor, add more tests, and split modules into smaller +areas of responsibility. + +A good first step is to build a module-level callgraph to understand how +interconnected your target module is. + + git clone https://git.torproject.org/user/nickm/calltool.git + cd tor + CFLAGS=0 ./configure + ../calltool/src/main.py module_callgraph + +The output will tell you each module name, along with a set of every module that +the module calls. Modules which call fewer other modules are better targets. + + Writing your Rust module +========================== + +Strive to change the C API as little as possible. + +We are currently targetting Rust stable. (See CodingStandardsRust.md for more +details.) + +It is on our TODO list to try to cultivate good +standing with various distro maintainers of `rustc` and `cargo`, in +order to ensure that whatever version we solidify on is readily +available. + +If parts of your Rust code needs to stay in sync with C code (such as +handling enums across the FFI boundary), annonotate these places in a +comment structured as follows: + + /// C_RUST_COUPLED: <path_to_file> `<name_of_c_object>` + +Where <name_of_c_object> can be an enum, struct, constant, etc. Then, +do the same in the C code, to note that rust will need to be changed +when the C does. + + + Adding your Rust module to Tor's build system +----------------------------------------------- + +0. Your translation of the C module should live in its own crate(s) + in the `.../tor/src/rust/` directory. +1. Add your crate to `.../tor/src/rust/Cargo.toml`, in the + `[workspace.members]` section. +2. Add your crate's files to src/rust/include.am + +If your crate should be available to C (rather than just being included as a +dependency of other Rust modules): +0. Declare the crate as a dependency of tor_rust in + `src/rust/tor_util/Cargo.toml` and include it in + `src/rust/tor_rust/lib.rs` + + How to test your Rust code +---------------------------- + +Everything should be tested full stop. Even non-public functionality. + +Be sure to edit `.../tor/src/test/test_rust.sh` to add the name of your +crate to the `crates` variable! This will ensure that `cargo test` is +run on your crate. + +Configure Tor's build system to build with Rust enabled: + + ./configure --enable-fatal-warnings --enable-rust --enable-cargo-online-mode + +Tor's test should be run by doing: + + make check + +Tor's integration tests should also pass: + + make test-stem + + Submitting a patch +===================== + +Please follow the instructions in `.../doc/HACKING/GettingStarted.md`. diff --git a/doc/HACKING/HelpfulTools.md b/doc/HACKING/HelpfulTools.md index a7f36e6c7e..d499238526 100644 --- a/doc/HACKING/HelpfulTools.md +++ b/doc/HACKING/HelpfulTools.md @@ -4,25 +4,49 @@ Useful tools These aren't strictly necessary for hacking on Tor, but they can help track down bugs. +Travis/Appveyor CI +------------------ +It's CI. + +Looks like this: +* https://travis-ci.org/torproject/tor +* https://ci.appveyor.com/project/torproject/tor + +Travis builds and runs tests on Linux, and eventually macOS (#24629). +Appveyor builds and runs tests on Windows (using Windows Services for Linux). + +Runs automatically on Pull Requests sent to torproject/tor. You can set it up +for your fork to build commits outside of PRs too: + +1. sign up for GitHub: https://github.com/join +2. fork https://github.com/torproject/tor: + https://help.github.com/articles/fork-a-repo/ +3. follow https://docs.travis-ci.com/user/getting-started/#To-get-started-with-Travis-CI. + skip steps involving `.travis.yml` (we already have one). +4. go to https://ci.appveyor.com/login , log in with github, and select + "NEW PROJECT" + +Builds should show up on the web at travis-ci.com and on IRC at #tor-ci on +OFTC. If they don't, ask #tor-dev (also on OFTC). + Jenkins ------- - https://jenkins.torproject.org +It's CI/builders. Looks like this: https://jenkins.torproject.org -Dmalloc -------- +Runs automatically on commits merged to git.torproject.org. We CI the +master branch and all supported tor versions. We also build nightly debian +packages from master. -The dmalloc library will keep track of memory allocation, so you can find out -if we're leaking memory, doing any double-frees, or so on. +Builds Linux and Windows cross-compilation. Runs Linux tests. - dmalloc -l -/dmalloc.log - (run the commands it tells you) - ./configure --with-dmalloc +Builds should show up on the web at jenkins.torproject.org and on IRC at +#tor-bots on OFTC. If they don't, ask #tor-dev (also on OFTC). Valgrind -------- - valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor + valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/app/tor (Note that if you get a zillion openssl warnings, you will also need to pass `--undef-value-errors=no` to valgrind, or rebuild your openssl @@ -111,15 +135,19 @@ Running gcov for unit test coverage (On OSX, you'll need to start with `--enable-coverage CC=clang`.) -Then, look at the .gcov files in `coverage-output`. '-' before a line means -that the compiler generated no code for that line. '######' means that the -line was never reached. Lines with numbers were called that number of times. - If that doesn't work: * Try configuring Tor with `--disable-gcc-hardening` * You might need to run `make clean` after you run `./configure`. +Then, look at the .gcov files in `coverage-output`. '-' before a line means +that the compiler generated no code for that line. '######' means that the +line was never reached. Lines with numbers were called that number of times. + +For more details about how to read gcov output, see the [Invoking +gcov](https://gcc.gnu.org/onlinedocs/gcc/Invoking-Gcov.html) chapter +of the GCC manual. + If you make changes to Tor and want to get another set of coverage results, you can run `make reset-gcov` to clear the intermediary gcov output. @@ -128,9 +156,13 @@ a meaningful diff between them, you can run: ./scripts/test/cov-diff coverage-output1 coverage-output2 | less -In this diff, any lines that were visited at least once will have coverage -"1". This lets you inspect what you (probably) really want to know: which -untested lines were changed? Are there any new untested lines? +In this diff, any lines that were visited at least once will have coverage "1", +and line numbers are deleted. This lets you inspect what you (probably) really +want to know: which untested lines were changed? Are there any new untested +lines? + +If you run ./scripts/test/cov-exclude, it marks excluded unreached +lines with 'x', and excluded reached lines with '!!!'. Running integration tests ------------------------- @@ -142,6 +174,12 @@ run `make test-network`. We also have scripts to run integration tests using Stem. To try them, set `STEM_SOURCE_DIR` to your Stem source directory, and run `test-stem`. +Profiling Tor +------------- + +Ongoing notes about Tor profiling can be found at +https://pad.riseup.net/p/profiling-tor + Profiling Tor with oprofile --------------------------- @@ -168,20 +206,62 @@ Here are some basic instructions * `opreport -l that_dir/*` - Profit +Profiling Tor with perf +----------------------- + +This works with a running Tor, and requires root. + +1. Decide how long you want to profile for. Start with (say) 30 seconds. If that + works, try again with longer times. + +2. Find the PID of your running tor process. + +3. Run `perf record --call-graph dwarf -p <PID> sleep <SECONDS>` + + (You may need to do this as root.) + + You might need to add `-e cpu-clock` as an option to the perf record line + above, if you are on an older CPU without access to hardware profiling + events, or in a VM, or something. + +4. Now you have a perf.data file. Have a look at it with `perf report + --no-children --sort symbol,dso` or `perf report --no-children --sort + symbol,dso --stdio --header`. How does it look? + +5a. Once you have a nice big perf.data file, you can compress it, encrypt it, + and send it to your favorite Tor developers. + +5b. Or maybe you'd rather not send a nice big perf.data file. Who knows what's + in that!? It's kinda scary. To generate a less scary file, you can use `perf + report -g > <FILENAME>.out`. Then you can compress that and put it somewhere + public. + +Profiling Tor with gperftools aka Google-performance-tools +---------------------------------------------------------- + +This should work on nearly any unixy system. It doesn't seem to be compatible +with RunAsDaemon though. + +Beforehand, install google-perftools. + +1. You need to rebuild Tor, hack the linking steps to add `-lprofiler` to the + libs. You can do this by adding `LIBS=-lprofiler` when you call `./configure`. + +Now you can run Tor with profiling enabled, and use the pprof utility to look at +performance! See the gperftools manual for more info, but basically: + +2. Run `env CPUPROFILE=/tmp/profile src/app/tor -f <path/torrc>`. The profile file + is not written to until Tor finishes execuction. + +3. Run `pprof src/app/tor /tm/profile` to start the REPL. + Generating and analyzing a callgraph ------------------------------------ -1. Run `./scripts/maint/generate_callgraph.sh`. This will generate a - bunch of files in a new ./callgraph directory. - -2. Run `./scripts/maint/analyze_callgraph.py callgraph/src/*/*`. This - will do a lot of graph operations and then dump out a new - `callgraph.pkl` file, containing data in Python's 'pickle' format. +0. Build Tor on linux or mac, ideally with -O0 or -fno-inline. -3. Run `./scripts/maint/display_callgraph.py`. It will display: - - the number of functions reachable from each function. - - all strongly-connnected components in the Tor callgraph - - the largest bottlenecks in the largest SCC in the Tor callgraph. +1. Clone 'https://gitweb.torproject.org/user/nickm/calltool.git/' . + Follow the README in that repository. Note that currently the callgraph generator can't detect calls that pass through function pointers. diff --git a/doc/HACKING/HowToReview.md b/doc/HACKING/HowToReview.md index d53318942f..2d1f3d1c9e 100644 --- a/doc/HACKING/HowToReview.md +++ b/doc/HACKING/HowToReview.md @@ -19,6 +19,8 @@ Top-level smell-checks - Does `make check-spaces` pass? +- Does `make check-changes` pass? + - Does it have a reasonable amount of tests? Do they pass? Do they leak memory? @@ -32,6 +34,7 @@ Top-level smell-checks - If this changes Tor's behavior on the wire, is there a design proposal? +- If this changes anything in the code, is there a "changes" file? Let's look at the code! diff --git a/doc/HACKING/Module.md b/doc/HACKING/Module.md new file mode 100644 index 0000000000..9cf36090b4 --- /dev/null +++ b/doc/HACKING/Module.md @@ -0,0 +1,111 @@ +# Modules in Tor # + +This document describes the build system and coding standards when writing a +module in Tor. + +## What is a module? ## + +In the context of the tor code base, a module is a subsystem that we can +selectively enable or disable, at `configure` time. + +Currently, there is only one module: + + - Directory Authority subsystem (dirauth) + +It is located in its own directory in `src/feature/dirauth/`. To disable it, +one need to pass `--disable-module-dirauth` at configure time. All modules +are currently enabled by default. + +## Build System ## + +The changes to the build system are pretty straightforward. + +1. Locate in the `configure.ac` file this define: `m4_define(MODULES`. It + contains a list (white-space separated) of the module in tor. Add yours to + the list. + +2. Use the `AC_ARG_ENABLE([module-dirauth]` template for your new module. We + use the "disable module" approach instead of enabling them one by one. So, + by default, tor will build all the modules. + + This will define the `HAVE_MODULE_<name>` statement which can be used in + the C code to conditionally compile things for your module. And the + `BUILD_MODULE_<name>` is also defined for automake files (e.g: include.am). + +3. In the `src/core/include.am` file, locate the `MODULE_DIRAUTH_SOURCES` + value. You need to create your own `_SOURCES` variable for your module + and then conditionally add the it to `LIBTOR_A_SOURCES` if you should + build the module. + + It is then **very** important to add your SOURCES variable to + `src_or_libtor_testing_a_SOURCES` so the tests can build it. + +4. Do the same for header files, locate `ORHEADERS +=` which always add all + headers of all modules so the symbol can be found for the module entry + points. + +Finally, your module will automatically be included in the +`TOR_MODULES_ALL_ENABLED` variable which is used to build the unit tests. They +always build everything in order to tests everything. + +## Coding ## + +As mentioned above, a module must be isolated in its own directory (name of +the module) in `src/feature/`. + +There are couples of "rules" you want to follow: + +* Minimize as much as you can the number of entry points into your module. + Less is always better but of course that doesn't work out for every use + case. However, it is a good thing to always keep that in mind. + +* Do **not** use the `HAVE_MODULE_<name>` define outside of the module code + base. Every entry point should have a second definition if the module is + disabled. For instance: + + ``` + #ifdef HAVE_MODULE_DIRAUTH + + int sr_init(int save_to_disk); + + #else /* HAVE_MODULE_DIRAUTH */ + + static inline int + sr_init(int save_to_disk) + { + (void) save_to_disk; + return 0; + } + + #endif /* HAVE_MODULE_DIRAUTH */ + + ``` + + The main reason for this approach is to avoid having conditional code + everywhere in the code base. It should be centralized as much as possible + which helps maintainability but also avoids conditional spaghetti code + making the code much more difficult to follow/understand. + +* It is possible that you end up with code that needs to be used by the rest + of the code base but is still part of your module. As a good example, if + you look at `src/feature/shared_random_client.c`: it contains code needed + by the hidden service subsystem but mainly related to the shared random + subsystem very specific to the dirauth module. + + This is fine but try to keep it as lean as possible and never use the same + filename as the one in the module. For example, this is a bad idea and + should never be done: + + - `src/feature/dirclient/shared_random.c` + - `src/feature/dirauth/shared_random.c` + +* When you include headers from the module, **always** use the full module + path in your statement. Example: + + `#include "feature/dirauth/dirvote.h"` + + The main reason is that we do **not** add the module include path by default + so it needs to be specified. But also, it helps our human brain understand + which part comes from a module or not. + + Even **in** the module itself, use the full include path like above. diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md index 7595398241..55a40fc89b 100644 --- a/doc/HACKING/ReleasingTor.md +++ b/doc/HACKING/ReleasingTor.md @@ -7,9 +7,16 @@ new Tor release: === 0. Preliminaries -1. Get at least three of weasel/arma/Sebastian/Sina to put the new - version number in their approved versions list. +1. Get at least two of weasel/arma/Sebastian to put the new + version number in their approved versions list. Give them a few + days to do this if you can. +2. If this is going to be an important security release, give the packagers + some advance warning: See this list of packagers in IV.3 below. + +3. Given the release date for Tor, ask the TB team about the likely release + date of a TB that contains it. See note below in "commit, upload, + announce". === I. Make sure it works @@ -18,6 +25,7 @@ new Tor release: resolve those. As applicable, merge the `maint-X` branch into the `release-X` branch. + But you've been doing that all along, right? 2. Are all of the jenkins builders happy? See jenkins.torproject.org. @@ -26,39 +34,37 @@ new Tor release: What about Coverity Scan? - Is make check-spaces happy? + What about clang scan-build? - Does 'make distcheck' compain? + Does 'make distcheck' complain? - How about 'make test-stem' and 'make test-network'? + How about 'make test-stem' and 'make test-network' and + `make test-network-full`? - Are all those tests still happy with --enable-expensive-hardening ? Any memory leaks? -=== II. Write a changelog. +=== II. Write a changelog + +1a. (Alpha release variant) -1. Gather the `changes/*` files into a changelog entry, rewriting many + Gather the `changes/*` files into a changelog entry, rewriting many of them and reordering to focus on what users and funders would find interesting and understandable. - 1. Make sure that everything that wants a bug number has one. - Make sure that everything which is a bugfix says what version - it was a bugfix on. - - 2. Concatenate them. - - 3. Sort them by section. Within each section, sort by "version it's - a bugfix on", else by numerical ticket order. + To do this, first run `./scripts/maint/lintChanges.py changes/*` and + fix as many warnings as you can. Then run `./scripts/maint/sortChanges.py + changes/* > changelog.in` to combine headings and sort the entries. + After that, it's time to hand-edit and fix the issues that lintChanges + can't find: - 4. Clean them up: + 1. Within each section, sort by "version it's a bugfix on", else by + numerical ticket order. - Standard idioms: - `Fixes bug 9999; bugfix on 0.3.3.3-alpha.` - - One space after a period. + 2. Clean them up: Make stuff very terse @@ -86,19 +92,32 @@ new Tor release: maint-0.2.1), be sure to make the stanzas identical (so people can distinguish if these are the same change). - 5. Merge them in. + 3. Clean everything one last time. + + 4. Run `./scripts/maint/format_changelog.py --inplace` to make it prettier - 6. Clean everything one last time. +1b. (old-stable release variant) - 7. Run `./scripts/maint/format_changelog.py` to make it prettier. + For stable releases that backport things from later, we try to compose + their releases, we try to make sure that we keep the changelog entries + identical to their original versions, with a 'backport from 0.x.y.z' + note added to each section. So in this case, once you have the items + from the changes files copied together, don't use them to build a new + changelog: instead, look up the corrected versions that were merged + into ChangeLog in the master branch, and use those. 2. Compose a short release blurb to highlight the user-facing changes. Insert said release blurb into the ChangeLog stanza. If it's a stable release, add it to the ReleaseNotes file too. If we're adding - to a release-0.2.x branch, manually commit the changelogs to the later + to a release-* branch, manually commit the changelogs to the later git branches too. -3. If you're doing the first stable release in a series, you need to +3. If there are changes that require or suggest operator intervention + before or during the update, mail operators (either dirauth or relays + list) with a headline that indicates that an action is required or + appreciated. + +4. If you're doing the first stable release in a series, you need to create a ReleaseNotes for the series as a whole. To get started there, copy all of the Changelog entries from the series into a new file, and run `./scripts/maint/sortChanges.py` on it. That will @@ -111,51 +130,61 @@ new Tor release: === III. Making the source release. -1. In `maint-0.2.x`, bump the version number in `configure.ac` and run - `scripts/maint/updateVersions.pl` to update version numbers in other - places, and commit. Then merge `maint-0.2.x` into `release-0.2.x`. +1. In `maint-0.?.x`, bump the version number in `configure.ac` and run + `perl scripts/maint/updateVersions.pl` to update version numbers in other + places, and commit. Then merge `maint-0.?.x` into `release-0.?.x`. (NOTE: To bump the version number, edit `configure.ac`, and then run either `make`, or `perl scripts/maint/updateVersions.pl`, depending on your version.) -2. Make distcheck, put the tarball up somewhere, and tell `#tor` about - it. Wait a while to see if anybody has problems building it. Try to - get Sebastian or somebody to try building it on Windows. + When you merge the maint branch forward to the next maint branch, or into + master, merge it with "-s ours" to avoid a needless version bump. + +2. Make distcheck, put the tarball up in somewhere (how about your + homedir on your homedir on people.torproject.org?) , and tell `#tor` + about it. Wait a while to see if anybody has problems building it. + (Though jenkins is usually pretty good about catching these things.) === IV. Commit, upload, announce 1. Sign the tarball, then sign and push the git tag: gpg -ba <the_tarball> - git tag -u <keyid> tor-0.2.x.y-status - git push origin tag tor-0.2.x.y-status + git tag -u <keyid> tor-0.3.x.y-status + git push origin tag tor-0.3.x.y-status + + (You must do this before you update the website: it relies on finding + the version by tag.) 2. scp the tarball and its sig to the dist website, i.e. `/srv/dist-master.torproject.org/htdocs/` on dist-master. When you want it to go live, you run "static-update-component dist.torproject.org" on dist-master. - Edit `include/versions.wmi` and `Makefile` to note the new version. + In the webwml.git repository, `include/versions.wmi` and `Makefile` + to note the new version. (NOTE: Due to #17805, there can only be one stable version listed at once. Nonetheless, do not call your version "alpha" if it is stable, or people will get confused.) -3. Email the packagers (cc'ing tor-assistants) that a new tarball is up. +3. Email the packagers (cc'ing tor-team) that a new tarball is up. The current list of packagers is: - {weasel,gk,mikeperry} at torproject dot org - {blueness} at gentoo dot org - {paul} at invizbox dot io + - {vincent} at invizbox dot com - {lfleischer} at archlinux dot org - {Nathan} at freitas dot net - {mike} at tig dot as - - {tails-rm} at boum dot org (for pre-release announcments) - - - - {tails-dev} at boum dot org (for at-release announcements) + - {tails-rm} at boum dot org + - {simon} at sdeziel.info + - {yuri} at freebsd.org + - {mh+tor} at scrit.ch + Also, email tor-packagers@lists.torproject.org. 4. Add the version number to Trac. To do this, go to Trac, log in, select "Admin" near the top of the screen, then select "Versions" from @@ -164,22 +193,30 @@ new Tor release: 0.2.2.23-alpha" (or whatever the version is), and we select the date as the date in the ChangeLog. -5. Wait up to a day or two (for a development release), or until most - packages are up (for a stable release), and mail the release blurb and - changelog to tor-talk or tor-announce. +5. Double-check: did the version get recommended in the consensus yet? Is + the website updated? If not, don't announce until they have the + up-to-date versions, or people will get confused. + +6. Mail the release blurb and ChangeLog to tor-talk (development release) or + tor-announce (stable). + + Post the changelog on the blog as well. You can generate a + blog-formatted version of the changelog with the -B option to + format-changelog. - (We might be moving to faster announcements, but don't announce until - the website is at least updated.) + When you post, include an estimate of when the next TorBrowser + releases will come out that include this Tor release. This will + usually track https://wiki.mozilla.org/RapidRelease/Calendar , but it + can vary. === V. Aftermath and cleanup -1. If it's a stable release, bump the version number in the `maint-x.y.z` - branch to "newversion-dev", and do a `merge -s ours` merge to avoid - taking that change into master. Do a similar `merge -s theirs` - merge to get the change (and only that change) into release. (Some - of the build scripts require that maint merge cleanly into release.) +1. If it's a stable release, bump the version number in the + `maint-x.y.z` branch to "newversion-dev", and do a `merge -s ours` + merge to avoid taking that change into master. 2. Forward-port the ChangeLog (and ReleaseNotes if appropriate). +3. Keep an eye on the blog post, to moderate comments and answer questions. diff --git a/doc/HACKING/Tracing.md b/doc/HACKING/Tracing.md new file mode 100644 index 0000000000..24fa761310 --- /dev/null +++ b/doc/HACKING/Tracing.md @@ -0,0 +1,91 @@ +# Tracing # + +This document describes how the event tracing subsystem works in tor so +developers can add events to the code base but also hook them to an event +tracing framework. + +## Basics ### + +Event tracing is separated in two concepts, trace events and a tracer. The +tracing subsystem can be found in `src/trace`. The `events.h` header file is +the main file that maps the different tracers to trace events. + +### Events ### + +A trace event is basically a function from which we can pass any data that +we want to collect. In addition, we specify a context for the event such as +a subsystem and an event name. + +A trace event in tor has the following standard format: + + tor_trace(subsystem, event\_name, args...) + +The `subsystem` parameter is the name of the subsytem the trace event is in. +For example that could be "scheduler" or "vote" or "hs". The idea is to add +some context to the event so when we collect them we know where it's coming +from. The `event_name` is the name of the event which helps a lot with +adding some semantic to the event. Finally, `args` is any number of +arguments we want to collect. + +Here is an example of a possible tracepoint in main(): + + tor_trace(main, init_phase, argc) + +The above is a tracepoint in the `main` subsystem with `init_phase` as the +event name and the `int argc` is passed to the event as well. + +How `argc` is collected or used has nothing to do with the instrumentation +(adding trace events to the code). It is the work of the tracer so this is why +the trace events and collection framework (tracer) are decoupled. You _can_ +have trace events without a tracer. + +### Tracer ### + +In `src/trace/events.h`, we map the `tor_trace()` function to the right +tracer. A tracer support is only enabled at compile time. For instance, the +file `src/trace/debug.h` contains the mapping of the generic tracing function +`tor_trace()` to the `log_debug()` function. More specialized function can be +mapped depending on the tracepoint. + +## Build System ## + +This section describes how it is integrated into the build system of tor. + +By default, every tracing events are disabled in tor that is `tor_trace()` +is a NOP. + +To enable a tracer, there is a configure option on the form of: + + --enable-tracing-<tracer> + +We have an option that will send every trace events to a `log_debug()` (as +mentionned above) which will print you the subsystem and name of the event but +not the arguments for technical reasons. This is useful if you want to quickly +see if your trace event is being hit or well written. To do so, use this +configure option: + + --enable-tracing-debug + +## Instrument Tor ## + +This is pretty easy. Let's say you want to add a trace event in +`src/feature/rend/rendcache.c`, you only have to add this include statement: + + #include "trace/events.h" + +Once done, you can add as many as you want `tor_trace()` that you need. +Please use the right subsystem (here it would be `hs`) and a unique name that +tells what the event is for. For example: + + tor_trace(hs, store_desc_as_client, desc, desc_id); + +If you look in `src/trace/events.h`, you'll see that if tracing is enabled it +will be mapped to a function called: + + tor_trace_hs_store_desc_as_client(desc, desc_id) + +And the point of all this is for that function to be defined in a new file +that you might want to add named `src/trace/hs.{c|h}` which would defined how +to collect the data for the `tor_trace_hs_store_desc_as_client()` function +like for instance sending it to a `log_debug()` or do more complex operations +or use a userspace tracer like LTTng (https://lttng.org). diff --git a/doc/HACKING/WritingTests.md b/doc/HACKING/WritingTests.md index de80bbdef2..05de8e0be8 100644 --- a/doc/HACKING/WritingTests.md +++ b/doc/HACKING/WritingTests.md @@ -7,8 +7,8 @@ keep from introducing bugs. The major ones are: 1. Unit tests written in C and shipped with the Tor distribution. - 2. Integration tests written in Python and shipped with the Tor - distribution. + 2. Integration tests written in Python 2 (>= 2.7) or Python 3 + (>= 3.1) and shipped with the Tor distribution. 3. Integration tests written in Python and shipped with the Stem library. Some of these use the Tor controller protocol. @@ -48,7 +48,7 @@ isolation, you just run `./src/test/test-memwipe`. To run tests within the unit test programs, you can specify the name of the test. The string ".." can be used as a wildcard at the end of the test name. For example, to run all the cell format tests, enter -`./src/test/test cellfmt/..`. To run +`./src/test/test cellfmt/..`. Many tests that need to mess with global state run in forked subprocesses in order to keep from contaminating one another. But when debugging a failing test, @@ -91,6 +91,9 @@ coverage percentage. For a summary of the test coverage for each _function_, run `./scripts/test/cov-display -f ${TMPDIR}/*`. +For more details on using gcov, including the helper scripts in +scripts/test, see HelpfulTools.md. + ### Comparing test coverage Sometimes it's useful to compare test coverage for a branch you're writing to @@ -117,7 +120,8 @@ with LCOV_EXCL_START... LCOV_EXCL_STOP. Note that older versions of lcov don't understand these lines. You can post-process .gcov files to make these lines 'unreached' by -running ./scripts/test/cov-exclude on them. +running ./scripts/test/cov-exclude on them. It marks excluded +unreached lines with 'x', and excluded reached lines with '!!!'. Note: you should never do this unless the line is meant to 100% unreachable by actual code. diff --git a/doc/HACKING/android/Simpleperf.md b/doc/HACKING/android/Simpleperf.md new file mode 100644 index 0000000000..25f39a3d23 --- /dev/null +++ b/doc/HACKING/android/Simpleperf.md @@ -0,0 +1,98 @@ +# Using `simpleperf` to collect CPU profiling on Android + +This document describes how you can use Android's `simpleperf` +command-line tool to get CPU profiling information from Tor via the +Orbot application. The tool is particularly useful for Tor development +because it is able to profile native applications on the platform +whereas a lot of the normal tooling for the Android platform is only +able to collect information from Java-based applications. + +## Prerequisites + +Before using `simpleperf` there is a couple of steps that must be +followed. You should make sure you have both a recent installation of +the Android Software Development Kit (SDK) and Native Development Kit +(NDK) installed. These can be found on the Android Developers website. + +1. Follow the build instructions from the `BUILD` file in the Orbot + repository and build an Orbot APK (Android Package) file with + debugging enabled. Make sure that when you build the native content of + the Orbot application that you run the `make -C external` command with + an additional `DEBUG=1` as parameter to ensure that the Orbot build + process does not strip the debug symbols from the Tor binary. + +2. (Optional) Uninstall and clean-up your old Orbot installation that + is most likely downloaded from Google's Play Store or via fdroid: + + $ adb shell pm clear org.torproject.android + $ adb uninstall org.torproject.android + +3. Install the Android Package you generated in step 1: + + $ adb install /path/to/your/app-fullperm-debug.apk + +4. Check on your device that the newly installed Orbot actually works + and behaves in the way you expect it to. + +## Profiling using `simpleperf` + +The `simpleperf` tool can be found in the `simpleperf/` directory in +the directory where you installed the Android NDK to. In this +directory there is a set of Python files that will help you deploy the +tool to a device and collect the measurement data such that you can +analyze the results on your computer rather than on your phone. + +1. Change directory to the location of the `simpleperf` directory. +2. Open the `app_profiler.config` file and change + `app_package_name` to `org.torproject.android`, `apk_file_path` to + the path of your Orbot Android Package (APK file). +3. Optionally change the duration parameter in the `record_options` + variable in `app_profiler.config` to the duration which you would like + to collect samples in. The value is specified in seconds. +4. Run the app profiler using `python app_profiler.py`. This helper + script will push the `simpleperf` tool to your device, start the + profiler, and once it has completed copy the generated `perf.data` + file over to your computer with the results. + +### Analyzing the results + +You can inspect your resulting `perf.data` file via a simple GUI +program `python report.py` or via the command-line tool `simpleperf +report`. I've found the GUI tool to be easier to navigate around with +than the command-line tool. + +The `-g` option can be passed to the command line `simpleperf report` +tool allows you to see the call graph of functions and how much time +was spend on the call. + +## Tips & Tricks + +- When you have installed Orbot the first time, you will notice that + if you get a shell on the Android device that there is no Tor binary + available. This is because Orbot unpacks the Tor binary first time it + is executed and places it under the `app_bin/` directory on the + device. + + To access binaries, `torrc` files, and other useful information on + the device do the following: + + $ adb shell + (device):/ $ run-as org.torproject.android + (device):/data/data/org.torproject.android $ ls + app_bin app_data cache databases files lib shared_prefs + + Descriptors, control authentication cookie, state, and other files can be + found in the `app_data` directory. The `torrc` can be found in the `app_bin/` + directory. + +- You can enable logging in Tor via the syslog (or android) log + mechanism with: + + $ adb shell + (device):/ $ run-as org.torproject.android + (device):/data/data/org.torproject.android $ echo -e "\nLog info syslog" >> app_bin/torrc + + Start Tor the normal way via Orbot and collect the logs from your computer using + + $ adb logcat + diff --git a/doc/include.am b/doc/include.am index 7164a4b2a0..0a123aae11 100644 --- a/doc/include.am +++ b/doc/include.am @@ -12,17 +12,11 @@ # part of the source distribution, so that people without asciidoc can # just use the .1 and .html files. -base_mans = doc/tor doc/tor-gencert doc/tor-resolve doc/torify -all_mans = $(base_mans) -if USE_FW_HELPER -install_mans = $(all_mans) -else -install_mans = $(base_mans) -endif +all_mans = doc/tor doc/tor-gencert doc/tor-resolve doc/torify doc/tor-print-ed-signing-cert if USE_ASCIIDOC -nodist_man1_MANS = $(install_mans:=.1) -doc_DATA = $(install_mans:=.html) +nodist_man1_MANS = $(all_mans:=.1) +doc_DATA = $(all_mans:=.html) html_in = $(all_mans:=.html.in) man_in = $(all_mans:=.1.in) txt_in = $(all_mans:=.1.txt) @@ -35,16 +29,22 @@ doc_DATA = endif EXTRA_DIST+= doc/asciidoc-helper.sh \ - $(html_in) $(man_in) $(txt_in) \ - doc/state-contents.txt \ - doc/torrc_format.txt \ + $(html_in) $(man_in) $(txt_in) \ + doc/state-contents.txt \ + doc/torrc_format.txt \ doc/TUNING \ doc/HACKING/README.1st.md \ doc/HACKING/CodingStandards.md \ + doc/HACKING/CodingStandardsRust.md \ + doc/HACKING/CodeStructure.md \ + doc/HACKING/Fuzzing.md \ doc/HACKING/GettingStarted.md \ + doc/HACKING/GettingStartedRust.md \ doc/HACKING/HelpfulTools.md \ doc/HACKING/HowToReview.md \ + doc/HACKING/Module.md \ doc/HACKING/ReleasingTor.md \ + doc/HACKING/Tracing.md \ doc/HACKING/WritingTests.md docdir = @docdir@ @@ -65,11 +65,13 @@ doc/tor.1.in: doc/tor.1.txt doc/torify.1.in: doc/torify.1.txt doc/tor-gencert.1.in: doc/tor-gencert.1.txt doc/tor-resolve.1.in: doc/tor-resolve.1.txt +doc/tor-print-ed-signing-cert.1.in: doc/tor-print-ed-signing-cert.1.txt doc/tor.html.in: doc/tor.1.txt doc/torify.html.in: doc/torify.1.txt doc/tor-gencert.html.in: doc/tor-gencert.1.txt doc/tor-resolve.html.in: doc/tor-resolve.1.txt +doc/tor-print-ed-signing-cert.html.in: doc/tor-print-ed-signing-cert.1.txt # use config.status to swap all machine-specific magic strings # in the asciidoc with their replacements. @@ -83,11 +85,13 @@ $(asciidoc_product) : doc/tor.html: doc/tor.html.in doc/tor-gencert.html: doc/tor-gencert.html.in doc/tor-resolve.html: doc/tor-resolve.html.in +doc/tor-print-ed-signing-cert.html: doc/tor-print-ed-signing-cert.html.in doc/torify.html: doc/torify.html.in doc/tor.1: doc/tor.1.in doc/tor-gencert.1: doc/tor-gencert.1.in doc/tor-resolve.1: doc/tor-resolve.1.in +doc/tor-print-ed-signing-cert.1: doc/tor-print-ed-signing-cert.1.in doc/torify.1: doc/torify.1.in CLEANFILES+= $(asciidoc_product) diff --git a/doc/tor-print-ed-signing-cert.1.txt b/doc/tor-print-ed-signing-cert.1.txt new file mode 100644 index 0000000000..1a3109df95 --- /dev/null +++ b/doc/tor-print-ed-signing-cert.1.txt @@ -0,0 +1,32 @@ +// Copyright (c) The Tor Project, Inc. +// See LICENSE for licensing information +// This is an asciidoc file used to generate the manpage/html reference. +// Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +:man source: Tor +:man manual: Tor Manual +tor-print-ed-signing-cert(1) +============================ +Tor Project, Inc. + +NAME +---- +tor-print-ed-signing-cert - print expiration date of ed25519 signing certificate + +SYNOPSIS +-------- +**tor-print-ed-signing-cert** __<path to ed25519_signing_cert file>__ + +DESCRIPTION +----------- +**tor-print-ed-signing-cert** is utility program for Tor relay operators to +check expiration date of ed25519 signing certificate. + +SEE ALSO +-------- +**tor**(1) + + +https://spec.torproject.org/cert-spec + +AUTHORS +------- +Roger Dingledine <arma@mit.edu>, Nick Mathewson <nickm@alum.mit.edu>. diff --git a/doc/tor-resolve.1.txt b/doc/tor-resolve.1.txt index 30e16d5daa..f1f8f77a42 100644 --- a/doc/tor-resolve.1.txt +++ b/doc/tor-resolve.1.txt @@ -47,7 +47,7 @@ SEE ALSO -------- **tor**(1), **torify**(1). + -See doc/socks-extensions.txt in the Tor package for protocol details. +For protocol details, see: https://spec.torproject.org/socks-extensions AUTHORS ------- diff --git a/doc/tor.1.txt b/doc/tor.1.txt index a7ee7d11ca..21b482802e 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -88,6 +88,10 @@ COMMAND-LINE OPTIONS List all valid options that are scheduled to become obsolete in a future version. (This is a warning, not a promise.) +[[opt-list-modules]] **--list-modules**:: + For each optional module, list whether or not it has been compiled + into Tor. (Any module not listed is not optional in this version of Tor.) + [[opt-version]] **--version**:: Display Tor version and exit. @@ -128,6 +132,16 @@ COMMAND-LINE OPTIONS the passphrase, including any trailing newlines. Default: read from the terminal. +[[opt-key-expiration]] **--key-expiration** [**purpose**]:: + The **purpose** specifies which type of key certificate to determine + the expiration of. The only currently recognised **purpose** is + "sign". + + + + Running "tor --key-expiration sign" will attempt to find your signing + key certificate and will output, both in the logs as well as to stdout, + the signing key certificate's expiration time in ISO-8601 format. + For example, the output sent to stdout will be of the form: + "signing-cert-expiry: 2017-07-25 08:30:15 UTC" Other options can be specified on the command-line in the format "--option value", in the format "option value", or in a configuration file. For @@ -153,6 +167,13 @@ values. To split one configuration entry into multiple lines, use a single backslash character (\) before the end of the line. Comments can be used in such multiline entries, but they must start at the beginning of a line. +Configuration options can be imported from files or folders using the %include +option with the value being a path. If the path is a file, the options from the +file will be parsed as if they were written where the %include option is. If +the path is a folder, all files on that folder will be parsed following lexical +order. Files starting with a dot are ignored. Files on subfolders are ignored. +The %include option can be used recursively. + By default, an option on the command line overrides an option found in the configuration file, and an option in a configuration file overrides one in the defaults file. @@ -176,7 +197,7 @@ forward slash (/) in the configuration file and on the command line. GENERAL OPTIONS --------------- -[[BandwidthRate]] **BandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[BandwidthRate]] **BandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: A token bucket limits the average incoming bandwidth usage on this node to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. If you want to run a relay in the @@ -185,6 +206,9 @@ GENERAL OPTIONS course, more is better; we recommend at least 250 KBytes (2 mbits) if possible. (Default: 1 GByte) + + + Note that this option, and other bandwidth-limiting options, apply to TCP + data only: They do not count TCP headers or DNS traffic. + + + With this option, and in other options that take arguments in bytes, KBytes, and so on, other formats are also supported. Notably, "KBytes" can also be written as "kilobytes" or "kb"; "MBytes" can be written as @@ -195,43 +219,48 @@ GENERAL OPTIONS To avoid confusion, we recommend writing "bytes" or "bits" explicitly, since it's easy to forget that "B" means bytes, not bits. -[[BandwidthBurst]] **BandwidthBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[BandwidthBurst]] **BandwidthBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Limit the maximum token bucket size (also known as the burst) to the given number of bytes in each direction. (Default: 1 GByte) -[[MaxAdvertisedBandwidth]] **MaxAdvertisedBandwidth** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[MaxAdvertisedBandwidth]] **MaxAdvertisedBandwidth** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: If set, we will not advertise more than this amount of bandwidth for our BandwidthRate. Server operators who want to reduce the number of clients who ask to build circuits through them (since this is proportional to advertised bandwidth rate) can thus reduce the CPU demands on their server without impacting network performance. -[[RelayBandwidthRate]] **RelayBandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[RelayBandwidthRate]] **RelayBandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: If not 0, a separate token bucket limits the average incoming bandwidth usage for \_relayed traffic_ on this node to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. Relayed traffic currently is calculated to include answers to directory - requests, but that may change in future versions. (Default: 0) + requests, but that may change in future versions. They do not include directory + fetches by the relay (from authority or other relays), because that is considered + "client" activity. (Default: 0) -[[RelayBandwidthBurst]] **RelayBandwidthBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[RelayBandwidthBurst]] **RelayBandwidthBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: If not 0, limit the maximum token bucket size (also known as the burst) for \_relayed traffic_ to the given number of bytes in each direction. - (Default: 0) + They do not include directory fetches by the relay (from authority + or other relays), because that is considered "client" activity. (Default: 0) -[[PerConnBWRate]] **PerConnBWRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: - If set, do separate rate limiting for each connection from a non-relay. - You should never need to change this value, since a network-wide value is - published in the consensus and your relay will use that value. (Default: 0) +[[PerConnBWRate]] **PerConnBWRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: + If this option is set manually, or via the "perconnbwrate" consensus + field, Tor will use it for separate rate limiting for each connection + from a non-relay. (Default: 0) -[[PerConnBWBurst]] **PerConnBWBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: - If set, do separate rate limiting for each connection from a non-relay. - You should never need to change this value, since a network-wide value is - published in the consensus and your relay will use that value. (Default: 0) +[[PerConnBWBurst]] **PerConnBWBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: + If this option is set manually, or via the "perconnbwburst" consensus + field, Tor will use it for separate rate limiting for each connection + from a non-relay. (Default: 0) [[ClientTransportPlugin]] **ClientTransportPlugin** __transport__ socks4|socks5 __IP__:__PORT__:: **ClientTransportPlugin** __transport__ exec __path-to-binary__ [options]:: In its first form, when set along with a corresponding Bridge line, the Tor - client forwards its traffic to a SOCKS-speaking proxy on "IP:PORT". It's the + client forwards its traffic to a SOCKS-speaking proxy on "IP:PORT". + (IPv4 addresses should written as-is; IPv6 addresses should be wrapped in + square brackets.) It's the duty of that proxy to properly forward the traffic to the bridge. + + In its second form, when set along with a corresponding Bridge line, the Tor @@ -248,7 +277,8 @@ GENERAL OPTIONS [[ServerTransportListenAddr]] **ServerTransportListenAddr** __transport__ __IP__:__PORT__:: When this option is set, Tor will suggest __IP__:__PORT__ as the listening address of any pluggable transport proxy that tries to - launch __transport__. + launch __transport__. (IPv4 addresses should written as-is; IPv6 + addresses should be wrapped in square brackets.) [[ServerTransportOptions]] **ServerTransportOptions** __transport__ __k=v__ __k=v__ ...:: When this option is set, Tor will pass the __k=v__ parameters to @@ -277,15 +307,24 @@ GENERAL OPTIONS descriptors as the OS will allow (you can find this by "ulimit -H -n"). If this number is less than ConnLimit, then Tor will refuse to start. + + - You probably don't need to adjust this. It has no effect on Windows - since that platform lacks getrlimit(). (Default: 1000) + Tor relays need thousands of sockets, to connect to every other relay. + If you are running a private bridge, you can reduce the number of sockets + that Tor uses. For example, to limit Tor to 500 sockets, run + "ulimit -n 500" in a shell. Then start tor in the same shell, with + **ConnLimit 500**. You may also need to set **DisableOOSCheck 0**. + + + + Unless you have severely limited sockets, you probably don't need to + adjust **ConnLimit** itself. It has no effect on Windows, since that + platform lacks getrlimit(). (Default: 1000) [[DisableNetwork]] **DisableNetwork** **0**|**1**:: When this option is set, we don't listen for or accept any connections other than controller connections, and we close (and don't reattempt) any outbound connections. Controllers sometimes use this option to avoid using - the network until Tor is fully configured. (Default: 0) + the network until Tor is fully configured. Tor will make still certain + network-related calls (like DNS lookups) as a part of its configuration + process, even if DisableNetwork is set. (Default: 0) [[ConstrainedSockets]] **ConstrainedSockets** **0**|**1**:: If set, Tor will tell the kernel to attempt to shrink the buffers for all @@ -338,17 +377,10 @@ GENERAL OPTIONS Unix domain sockets only: Do not insist that the directory that holds the socket be read-restricted. -[[ControlListenAddress]] **ControlListenAddress** __IP__[:__PORT__]:: - Bind the controller listener to this address. If you specify a port, bind - to this port rather than the one specified in ControlPort. We strongly - recommend that you leave this alone unless you know what you're doing, - since giving attackers access to your control listener is really - dangerous. This directive can be specified multiple - times to bind to multiple addresses/ports. (Default: 127.0.0.1) - [[ControlSocket]] **ControlSocket** __Path__:: Like ControlPort, but listens on a Unix domain socket, rather than a TCP - socket. '0' disables ControlSocket (Unix and Unix-like systems only.) + socket. '0' disables ControlSocket. (Unix and Unix-like systems only.) + (Default: 0) [[ControlSocketsGroupWritable]] **ControlSocketsGroupWritable** **0**|**1**:: If this option is set to 0, don't allow the filesystem group to read and @@ -390,14 +422,29 @@ GENERAL OPTIONS file readable by the default GID. (Default: 0) [[DataDirectory]] **DataDirectory** __DIR__:: - Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor) + Store working data in DIR. Can not be changed while tor is running. + (Default: ~/.tor if your home directory is not /; otherwise, + @LOCALSTATEDIR@/lib/tor. On Windows, the default is + your ApplicationData folder.) [[DataDirectoryGroupReadable]] **DataDirectoryGroupReadable** **0**|**1**:: If this option is set to 0, don't allow the filesystem group to read the DataDirectory. If the option is set to 1, make the DataDirectory readable by the default GID. (Default: 0) -[[FallbackDir]] **FallbackDir** __address__:__port__ orport=__port__ id=__fingerprint__ [weight=__num__] [ipv6=__address__:__orport__]:: +[[CacheDirectory]] **CacheDirectory** __DIR__:: + Store cached directory data in DIR. Can not be changed while tor is + running. + (Default: uses the value of DataDirectory.) + +[[CacheDirectoryGroupReadable]] **CacheDirectoryGroupReadable** **0**|**1**|**auto**:: + If this option is set to 0, don't allow the filesystem group to read the + CacheDirectory. If the option is set to 1, make the CacheDirectory readable + by the default GID. If the option is "auto", then we use the + setting for DataDirectoryGroupReadable when the CacheDirectory is the + same as the DataDirectory, and 0 otherwise. (Default: auto) + +[[FallbackDir]] **FallbackDir** __ipv4address__:__port__ orport=__port__ id=__fingerprint__ [weight=__num__] [ipv6=**[**__ipv6address__**]**:__orport__]:: When we're unable to connect to any directory cache for directory info (usually because we don't know about any yet) we try a directory authority. Clients also simultaneously try a FallbackDir, to avoid hangs on client @@ -413,7 +460,7 @@ GENERAL OPTIONS FallbackDir line is present, it replaces the hard-coded FallbackDirs, regardless of the value of UseDefaultFallbackDirs.) (Default: 1) -[[DirAuthority]] **DirAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__:: +[[DirAuthority]] **DirAuthority** [__nickname__] [**flags**] __ipv4address__:__port__ __fingerprint__:: Use a nonstandard authoritative directory server at the provided address and port, with the specified key fingerprint. This option can be repeated many times, for multiple authoritative directory servers. Flags are @@ -427,13 +474,16 @@ GENERAL OPTIONS with probability proportional to that weight (default 1.0). If a flag "v3ident=**fp**" is given, the dirserver is a v3 directory authority whose v3 long-term signing key has the fingerprint **fp**. Lastly, - if an "ipv6=__address__:__orport__" flag is present, then the directory + if an "ipv6=**[**__ipv6address__**]**:__orport__" flag is present, then + the directory authority is listening for IPv6 connections on the indicated IPv6 address and OR Port. + + - Tor will contact the authority at __address__:__port__ (the DirPort) to - download directory documents. If an IPv6 address is supplied, Tor will - also download directory documents at the IPv6 address on the DirPort. + + Tor will contact the authority at __ipv4address__ to + download directory documents. The provided __port__ value is a dirport; + clients ignore this in favor of the specified "orport=" value. If an + IPv6 ORPort is supplied, Tor will + also download directory documents at the IPv6 ORPort. + + If no **DirAuthority** line is given, Tor will use the default directory authorities. NOTE: this option is intended for setting up a private Tor @@ -448,9 +498,9 @@ GENERAL OPTIONS should be 1.0 or less. The default is less than 1, to reduce load on authorities. (Default: 0.1) -[[AlternateDirAuthority]] **AlternateDirAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__ + +[[AlternateDirAuthority]] **AlternateDirAuthority** [__nickname__] [**flags**] __ipv4address__:__port__ __fingerprint__ + -[[AlternateBridgeAuthority]] **AlternateBridgeAuthority** [__nickname__] [**flags**] __address__:__port__ __ fingerprint__:: +[[AlternateBridgeAuthority]] **AlternateBridgeAuthority** [__nickname__] [**flags**] __ipv4address__:__port__ __ fingerprint__:: These options behave as DirAuthority, but they replace fewer of the default directory authorities. Using AlternateDirAuthority replaces the default Tor directory authorities, but @@ -465,7 +515,8 @@ GENERAL OPTIONS not supported. We believe that this feature works on modern Gnu/Linux distributions, and that it should work on *BSD systems (untested). This option requires that you start your Tor as root, and you should use the - **User** option to properly reduce Tor's privileges. (Default: 0) + **User** option to properly reduce Tor's privileges. + Can not be changed while tor is running. (Default: 0) [[DisableDebuggerAttachment]] **DisableDebuggerAttachment** **0**|**1**:: If set to 1, Tor will attempt to prevent basic debugging attachment attempts @@ -505,22 +556,33 @@ GENERAL OPTIONS (Default: 1) [[FetchUselessDescriptors]] **FetchUselessDescriptors** **0**|**1**:: - If set to 1, Tor will fetch every non-obsolete descriptor from the - authorities that it hears about. Otherwise, it will avoid fetching useless - descriptors, for example for routers that are not running. This option is - useful if you're using the contributed "exitlist" script to enumerate Tor - nodes that exit to certain addresses. (Default: 0) + If set to 1, Tor will fetch every consensus flavor, and all server + descriptors and authority certificates referenced by those consensuses, + except for extra info descriptors. When this option is 1, Tor will also + keep fetching descriptors, even when idle. + If set to 0, Tor will avoid fetching useless descriptors: flavors that it + is not using to build circuits, and authority certificates it does not + trust. When Tor hasn't built any application circuits, it will go idle, + and stop fetching descriptors. This option is useful if you're using a + tor client with an external parser that uses a full consensus. + This option fetches all documents except extrainfo descriptors, + **DirCache** fetches and serves all documents except extrainfo + descriptors, **DownloadExtraInfo*** fetches extrainfo documents, and serves + them if **DirCache** is on, and **UseMicrodescriptors** changes the + flavour of consensues and descriptors that is fetched and used for + building circuits. (Default: 0) [[HTTPProxy]] **HTTPProxy** __host__[:__port__]:: Tor will make all its directory requests through this host:port (or host:80 if port is not specified), rather than connecting directly to any directory - servers. + servers. (DEPRECATED: As of 0.3.1.0-alpha you should use HTTPSProxy.) [[HTTPProxyAuthenticator]] **HTTPProxyAuthenticator** __username:password__:: If defined, Tor will use this username:password for Basic HTTP proxy authentication, as in RFC 2617. This is currently the only form of HTTP proxy authentication that Tor supports; feel free to submit a patch if you - want it to support others. + want it to support others. (DEPRECATED: As of 0.3.1.0-alpha you should use + HTTPSProxyAuthenticator.) [[HTTPSProxy]] **HTTPSProxy** __host__[:__port__]:: Tor will make all its OR (SSL) connections through this host:port (or @@ -538,7 +600,23 @@ GENERAL OPTIONS [[Sandbox]] **Sandbox** **0**|**1**:: If set to 1, Tor will run securely through the use of a syscall sandbox. Otherwise the sandbox will be disabled. The option is currently an - experimental feature. (Default: 0) + experimental feature. It only works on Linux-based operating systems, + and only when Tor has been built with the libseccomp library. This option + can not be changed while tor is running. + + + When the Sandbox is 1, the following options can not be changed when tor + is running: + Address + ConnLimit + CookieAuthFile + DirPortFrontPage + ExtORPortCookieAuthFile + Logs + ServerDNSResolvConfFile + Tor must remain in client or server mode (some changes to ClientOnly and + ORPort are not allowed). + ClientOnionAuthDir and any files in it won't reload on HUP signal. + (Default: 0) [[Socks4Proxy]] **Socks4Proxy** __host__[:__port__]:: Tor will make all OR connections through the SOCKS 4 proxy at host:port @@ -555,16 +633,14 @@ GENERAL OPTIONS in accordance to RFC 1929. Both username and password must be between 1 and 255 characters. -[[SocksSocketsGroupWritable]] **SocksSocketsGroupWritable** **0**|**1**:: +[[UnixSocksGroupWritable]] **UnixSocksGroupWritable** **0**|**1**:: If this option is set to 0, don't allow the filesystem group to read and - write unix sockets (e.g. SocksSocket). If the option is set to 1, make - the SocksSocket socket readable and writable by the default GID. (Default: 0) + write unix sockets (e.g. SocksPort unix:). If the option is set to 1, make + the Unix socket readable and writable by the default GID. (Default: 0) [[KeepalivePeriod]] **KeepalivePeriod** __NUM__:: To keep firewalls from expiring connections, send a padding keepalive cell - every NUM seconds on open connections that are in use. If the connection - has no open circuits, it will instead be closed after NUM seconds of - idleness. (Default: 5 minutes) + every NUM seconds on open connections that are in use. (Default: 5 minutes) [[Log]] **Log** __minSeverity__[-__maxSeverity__] **stderr**|**stdout**|**syslog**:: Send all messages between __minSeverity__ and __maxSeverity__ to the standard @@ -597,7 +673,8 @@ GENERAL OPTIONS + The currently recognized domains are: general, crypto, net, config, fs, protocol, mm, http, app, control, circ, rend, bug, dir, dirserv, or, edge, - acct, hist, and handshake. Domain names are case-insensitive. + + acct, hist, handshake, heartbeat, channel, sched, guard, consdiff, and dos. + Domain names are case-insensitive. + + For example, "`Log [handshake]debug [~net,~mm]info notice stdout`" sends to stdout: all handshake messages of any severity, all info-and-higher @@ -609,7 +686,7 @@ GENERAL OPTIONS message currently has at least one domain; most currently have exactly one. This doesn't affect controller log messages. (Default: 0) -[[MaxUnparseableDescSizeToLog]] **MaxUnparseableDescSizeToLog** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**:: +[[MaxUnparseableDescSizeToLog]] **MaxUnparseableDescSizeToLog** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**:: Unparseable descriptors (e.g. for votes, consensuses, routers) are logged in separate files by hash, up to the specified size in total. Note that only files logged during the lifetime of this Tor process count toward the @@ -621,27 +698,41 @@ GENERAL OPTIONS is only useful when you have multiple network interfaces, and you want all of Tor's outgoing connections to use a single one. This option may be used twice, once with an IPv4 address and once with an IPv6 address. + IPv6 addresses should be wrapped in square brackets. This setting will be ignored for connections to the loopback addresses - (127.0.0.0/8 and ::1). + (127.0.0.0/8 and ::1), and is not used for DNS requests as well. + +[[OutboundBindAddressOR]] **OutboundBindAddressOR** __IP__:: + Make all outbound non-exit (relay and other) connections + originate from the IP address specified. This option overrides + **OutboundBindAddress** for the same IP version. This option may + be used twice, once with an IPv4 address and once with an IPv6 + address. IPv6 addresses should be wrapped in square brackets. + This setting will be ignored for connections to the loopback + addresses (127.0.0.0/8 and ::1). + +[[OutboundBindAddressExit]] **OutboundBindAddressExit** __IP__:: + Make all outbound exit connections originate from the IP address + specified. This option overrides **OutboundBindAddress** for the + same IP version. This option may be used twice, once with an IPv4 + address and once with an IPv6 address. + IPv6 addresses should be wrapped in square brackets. + This setting will be ignored + for connections to the loopback addresses (127.0.0.0/8 and ::1). [[PidFile]] **PidFile** __FILE__:: On startup, write our PID to FILE. On clean shutdown, remove - FILE. + FILE. Can not be changed while tor is running. [[ProtocolWarnings]] **ProtocolWarnings** **0**|**1**:: If 1, Tor will log with severity \'warn' various cases of other parties not following the Tor specification. Otherwise, they are logged with severity \'info'. (Default: 0) -[[PredictedPortsRelevanceTime]] **PredictedPortsRelevanceTime** __NUM__:: - Set how long, after the client has made an anonymized connection to a - given port, we will try to make sure that we build circuits to - exits that support that port. The maximum value for this option is 1 - hour. (Default: 1 hour) - [[RunAsDaemon]] **RunAsDaemon** **0**|**1**:: If 1, Tor forks and daemonizes to the background. This option has no effect on Windows; instead you should use the --service command-line option. + Can not be changed while tor is running. (Default: 0) [[LogTimeGranularity]] **LogTimeGranularity** __NUM__:: @@ -658,7 +749,13 @@ GENERAL OPTIONS [[SyslogIdentityTag]] **SyslogIdentityTag** __tag__:: When logging to syslog, adds a tag to the syslog identity such that - log entries are marked with "Tor-__tag__". (Default: none) + log entries are marked with "Tor-__tag__". Can not be changed while tor is + running. (Default: none) + +[[AndroidIdentityTag]] **AndroidIdentityTag** __tag__:: + When logging to Android's logging subsystem, adds a tag to the log identity + such that log entries are marked with "Tor-__tag__". Can not be changed while + tor is running. (Default: none) [[SafeLogging]] **SafeLogging** **0**|**1**|**relay**:: Tor can scrub potentially sensitive strings from log messages (e.g. @@ -669,10 +766,13 @@ GENERAL OPTIONS If this option is set to 0, Tor will not perform any scrubbing, if it is set to 1, all potentially sensitive strings are replaced. If it is set to relay, all log messages generated when acting as a relay are sanitized, but - all messages generated when acting as a client are not. (Default: 1) + all messages generated when acting as a client are not. + Note: Tor may not heed this option when logging at log levels below Notice. + (Default: 1) [[User]] **User** __Username__:: On startup, setuid to this user and setgid to their primary group. + Can not be changed while tor is running. [[KeepBindCapabilities]] **KeepBindCapabilities** **0**|**1**|**auto**:: On Linux, when we are started as root and we switch our identity using @@ -680,37 +780,38 @@ GENERAL OPTIONS try to retain our ability to bind to low ports. If this value is 1, we try to keep the capability; if it is 0 we do not; and if it is **auto**, we keep the capability only if we are configured to listen on a low port. + Can not be changed while tor is running. (Default: auto.) [[HardwareAccel]] **HardwareAccel** **0**|**1**:: If non-zero, try to use built-in (static) crypto hardware acceleration when - available. (Default: 0) + available. Can not be changed while tor is running. (Default: 0) [[AccelName]] **AccelName** __NAME__:: When using OpenSSL hardware crypto acceleration attempt to load the dynamic engine of this name. This must be used for any dynamic hardware engine. - Names can be verified with the openssl engine command. + Names can be verified with the openssl engine command. Can not be changed + while tor is running. [[AccelDir]] **AccelDir** __DIR__:: Specify this option if using dynamic hardware acceleration and the engine implementation library resides somewhere other than the OpenSSL default. + Can not be changed while tor is running. [[AvoidDiskWrites]] **AvoidDiskWrites** **0**|**1**:: If non-zero, try to write to disk less frequently than we would otherwise. This is useful when running on flash memory or other media that support only a limited number of writes. (Default: 0) -[[CircuitPriorityHalflife]] **CircuitPriorityHalflife** __NUM1__:: +[[CircuitPriorityHalflife]] **CircuitPriorityHalflife** __NUM__:: If this value is set, we override the default algorithm for choosing which - circuit's cell to deliver or relay next. When the value is 0, we - round-robin between the active circuits on a connection, delivering one - cell from each in turn. When the value is positive, we prefer delivering - cells from whichever connection has the lowest weighted cell count, where - cells are weighted exponentially according to the supplied - CircuitPriorityHalflife value (in seconds). If this option is not set at - all, we use the behavior recommended in the current consensus - networkstatus. This is an advanced option; you generally shouldn't have - to mess with it. (Default: not set) + circuit's cell to deliver or relay next. It is delivered first to the + circuit that has the lowest weighted cell count, where cells are weighted + exponentially according to this value (in seconds). If the value is -1, it + is taken from the consensus if possible else it will fallback to the + default value of 30. Minimum: 1, Maximum: 2147483647. This can be defined + as a float value. This is an advanced option; you generally shouldn't have + to mess with it. (Default: -1) [[CountPrivateBandwidth]] **CountPrivateBandwidth** **0**|**1**:: If this option is set, then Tor's rate-limiting applies not only to @@ -718,27 +819,63 @@ GENERAL OPTIONS 127.0.0.1 or 10.0.0.1. This is mostly useful for debugging rate-limiting. (Default: 0) +[[ExtendByEd25519ID]] **ExtendByEd25519ID** **0**|**1**|**auto**:: + If this option is set to 1, we always try to include a relay's Ed25519 ID + when telling the proceeding relay in a circuit to extend to it. + If this option is set to 0, we never include Ed25519 IDs when extending + circuits. If the option is set to "default", we obey a + parameter in the consensus document. (Default: auto) + +[[NoExec]] **NoExec** **0**|**1**:: + If this option is set to 1, then Tor will never launch another + executable, regardless of the settings of ClientTransportPlugin + or ServerTransportPlugin. Once this option has been set to 1, + it cannot be set back to 0 without restarting Tor. (Default: 0) + +[[Schedulers]] **Schedulers** **KIST**|**KISTLite**|**Vanilla**:: + Specify the scheduler type that tor should use. The scheduler is + responsible for moving data around within a Tor process. This is an ordered + list by priority which means that the first value will be tried first and if + unavailable, the second one is tried and so on. It is possible to change + these values at runtime. This option mostly effects relays, and most + operators should leave it set to its default value. + (Default: KIST,KISTLite,Vanilla) + + + The possible scheduler types are: + + + **KIST**: Kernel-Informed Socket Transport. Tor will use TCP information + from the kernel to make informed decisions regarding how much data to send + and when to send it. KIST also handles traffic in batches (see + KISTSchedRunInterval) in order to improve traffic prioritization decisions. + As implemented, KIST will only work on Linux kernel version 2.6.39 or + higher. + + + **KISTLite**: Same as KIST but without kernel support. Tor will use all + the same mechanics as with KIST, including the batching, but its decisions + regarding how much data to send will not be as good. KISTLite will work on + all kernels and operating systems, and the majority of the benefits of KIST + are still realized with KISTLite. + + + **Vanilla**: The scheduler that Tor used before KIST was implemented. It + sends as much data as possible, as soon as possible. Vanilla will work on + all kernels and operating systems. + +[[KISTSchedRunInterval]] **KISTSchedRunInterval** __NUM__ **msec**:: + If KIST or KISTLite is used in the Schedulers option, this controls at which + interval the scheduler tick is. If the value is 0 msec, the value is taken + from the consensus if possible else it will fallback to the default 10 + msec. Maximum possible value is 100 msec. (Default: 0 msec) + +[[KISTSockBufSizeFactor]] **KISTSockBufSizeFactor** __NUM__:: + If KIST is used in Schedulers, this is a multiplier of the per-socket + limit calculation of the KIST algorithm. (Default: 1.0) + CLIENT OPTIONS -------------- The following options are useful only for clients (that is, if -**SocksPort**, **TransPort**, **DNSPort**, or **NATDPort** is non-zero): - -[[AllowInvalidNodes]] **AllowInvalidNodes** **entry**|**exit**|**middle**|**introduction**|**rendezvous**|**...**:: - If some Tor servers are obviously not working right, the directory - authorities can manually mark them as invalid, meaning that it's not - recommended you use them for entry or exit positions in your circuits. You - can opt to use them in some circuit positions, though. The default is - "middle,rendezvous", and other choices are not advised. - -[[ExcludeSingleHopRelays]] **ExcludeSingleHopRelays** **0**|**1**:: - This option controls whether circuits built by Tor will include relays with - the AllowSingleHopExits flag set to true. If ExcludeSingleHopRelays is set - to 0, these relays will be included. Note that these relays might be at - higher risk of being seized or observed, so they are not normally - included. Also note that relatively few clients turn off this option, - so using these relays might make your client stand out. - (Default: 1) +**SocksPort**, **HTTPTunnelPort**, **TransPort**, **DNSPort**, or +**NATDPort** is non-zero): [[Bridge]] **Bridge** [__transport__] __IP__:__ORPort__ [__fingerprint__]:: When set along with UseBridges, instructs Tor to use the relay at @@ -753,7 +890,12 @@ The following options are useful only for clients (that is, if rather than connecting to the bridge directly. Some transports use a transport-specific method to work out the remote address to connect to. These transports typically ignore the "IP:ORPort" specified in the bridge - line. + line. + + + + Tor passes any "key=val" settings to the pluggable transport proxy as + per-connection arguments when connecting to the bridge. Consult + the documentation of the pluggable transport for details of what + arguments it supports. [[LearnCircuitBuildTimeout]] **LearnCircuitBuildTimeout** **0**|**1**:: If 0, CircuitBuildTimeout adaptive learning is disabled. (Default: 1) @@ -766,13 +908,15 @@ The following options are useful only for clients (that is, if LearnCircuitBuildTimeout is 0, this value is the only value used. (Default: 60 seconds) -[[CircuitIdleTimeout]] **CircuitIdleTimeout** __NUM__:: - If we have kept a clean (never used) circuit around for NUM seconds, then - close it. This way when the Tor client is entirely idle, it can expire all - of its circuits, and then expire its TLS connections. Also, if we end up - making a circuit that is not useful for exiting any of the requests we're - receiving, it won't forever take up a slot in the circuit list. (Default: 1 - hour) +[[CircuitsAvailableTimeout]] **CircuitsAvailableTimeout** __NUM__:: + Tor will attempt to keep at least one open, unused circuit available for + this amount of time. This option governs how long idle circuits are kept + open, as well as the amount of time Tor will keep a circuit open to each + of the recently used ports. This way when the Tor client is entirely + idle, it can expire all of its circuits, and then expire its TLS + connections. Note that the actual timeout value is uniformly randomized + from the specified value to twice that amount. (Default: 30 minutes; + Max: 24 hours) [[CircuitStreamTimeout]] **CircuitStreamTimeout** __NUM__:: If non-zero, this option overrides our internal timeout schedule for how @@ -789,6 +933,22 @@ The following options are useful only for clients (that is, if and fast enough. The current behavior is simply that Tor is a client unless ORPort, ExtORPort, or DirPort are configured.) (Default: 0) +[[ConnectionPadding]] **ConnectionPadding** **0**|**1**|**auto**:: + This option governs Tor's use of padding to defend against some forms of + traffic analysis. If it is set to 'auto', Tor will send padding only + if both the client and the relay support it. If it is set to 0, Tor will + not send any padding cells. If it is set to 1, Tor will still send padding + for client connections regardless of relay support. Only clients may set + this option. This option should be offered via the UI to mobile users + for use where bandwidth may be expensive. + (Default: auto) + +[[ReducedConnectionPadding]] **ReducedConnectionPadding** **0**|**1**:: + If set to 1, Tor will not not hold OR connections open for very long, + and will send less padding on these connections. Only clients may set + this option. This option should be offered via the UI to mobile users + for use where bandwidth may be expensive. (Default: 0) + [[ExcludeNodes]] **ExcludeNodes** __node__,__node__,__...__:: A list of identity fingerprints, country codes, and address patterns of nodes to avoid when building a circuit. Country codes are @@ -817,7 +977,7 @@ The following options are useful only for clients (that is, if [[ExcludeExitNodes]] **ExcludeExitNodes** __node__,__node__,__...__:: A list of identity fingerprints, country codes, and address patterns of nodes to never use when picking an exit node---that is, a - node that delivers traffic for you outside the Tor network. Note that any + node that delivers traffic for you *outside* the Tor network. Note that any node listed in ExcludeNodes is automatically considered to be part of this list too. See the **ExcludeNodes** option for more information on how to specify @@ -834,7 +994,7 @@ The following options are useful only for clients (that is, if [[ExitNodes]] **ExitNodes** __node__,__node__,__...__:: A list of identity fingerprints, country codes, and address patterns of nodes to use as exit node---that is, a - node that delivers traffic for you outside the Tor network. See + node that delivers traffic for you *outside* the Tor network. See the **ExcludeNodes** option for more information on how to specify nodes. + + Note that if you list too few nodes here, or if you exclude too many exit @@ -842,7 +1002,7 @@ The following options are useful only for clients (that is, if if none of the exits you list allows traffic on port 80 or 443, you won't be able to browse the web. + + - Note also that not every circuit is used to deliver traffic outside of + Note also that not every circuit is used to deliver traffic *outside* of the Tor network. It is normal to see non-exit circuits (such as those used to connect to hidden services, those that do directory fetches, those used for relay reachability self-tests, and so on) that end @@ -852,7 +1012,7 @@ The following options are useful only for clients (that is, if The ExcludeNodes option overrides this option: any node listed in both ExitNodes and ExcludeNodes is treated as excluded. + + - The .exit address notation, if enabled via AllowDotExit, overrides + The .exit address notation, if enabled via MapAddress, overrides this option. [[EntryNodes]] **EntryNodes** __node__,__node__,__...__:: @@ -868,16 +1028,16 @@ The following options are useful only for clients (that is, if the **ExcludeNodes** option for more information on how to specify nodes. [[StrictNodes]] **StrictNodes** **0**|**1**:: - If StrictNodes is set to 1, Tor will treat the ExcludeNodes option as a - requirement to follow for all the circuits you generate, even if doing so - will break functionality for you. If StrictNodes is set to 0, Tor will + If StrictNodes is set to 1, Tor will treat solely the ExcludeNodes option + as a requirement to follow for all the circuits you generate, even if + doing so will break functionality for you (StrictNodes applies to neither + ExcludeExitNodes nor to ExitNodes). If StrictNodes is set to 0, Tor will still try to avoid nodes in the ExcludeNodes list, but it will err on the - side of avoiding unexpected errors. Specifically, StrictNodes 0 tells - Tor that it is okay to use an excluded node when it is *necessary* to - perform relay reachability self-tests, connect to - a hidden service, provide a hidden service to a client, fulfill a .exit - request, upload directory information, or download directory information. - (Default: 0) + side of avoiding unexpected errors. Specifically, StrictNodes 0 tells Tor + that it is okay to use an excluded node when it is *necessary* to perform + relay reachability self-tests, connect to a hidden service, provide a + hidden service to a client, fulfill a .exit request, upload directory + information, or download directory information. (Default: 0) [[FascistFirewall]] **FascistFirewall** **0**|**1**:: If 1, Tor will only create outgoing connections to ORs running on ports @@ -892,7 +1052,7 @@ The following options are useful only for clients (that is, if **FascistFirewall** is set. This option is deprecated; use ReachableAddresses instead. (Default: 80, 443) -[[ReachableAddresses]] **ReachableAddresses** __ADDR__[/__MASK__][:__PORT__]...:: +[[ReachableAddresses]] **ReachableAddresses** __IP__[/__MASK__][:__PORT__]...:: A comma-separated list of IP addresses and ports that your firewall allows you to connect to. The format is as for the addresses in ExitPolicy, except that "accept" is understood unless "reject" is explicitly provided. For @@ -901,14 +1061,15 @@ The following options are useful only for clients (that is, if 99, rejects port 80 connections to net 18, and accepts connections to port 80 otherwise. (Default: \'accept \*:*'.) -[[ReachableDirAddresses]] **ReachableDirAddresses** __ADDR__[/__MASK__][:__PORT__]...:: +[[ReachableDirAddresses]] **ReachableDirAddresses** __IP__[/__MASK__][:__PORT__]...:: Like **ReachableAddresses**, a list of addresses and ports. Tor will obey these restrictions when fetching directory information, using standard HTTP GET requests. If not set explicitly then the value of **ReachableAddresses** is used. If **HTTPProxy** is set then these - connections will go through that proxy. + connections will go through that proxy. (DEPRECATED: This option has + had no effect for some time.) -[[ReachableORAddresses]] **ReachableORAddresses** __ADDR__[/__MASK__][:__PORT__]...:: +[[ReachableORAddresses]] **ReachableORAddresses** __IP__[/__MASK__][:__PORT__]...:: Like **ReachableAddresses**, a list of addresses and ports. Tor will obey these restrictions when connecting to Onion Routers, using TLS/SSL. If not set explicitly then the value of **ReachableAddresses** is used. If @@ -931,23 +1092,17 @@ The following options are useful only for clients (that is, if services can be configured to require authorization using the **HiddenServiceAuthorizeClient** option. -[[CloseHSClientCircuitsImmediatelyOnTimeout]] **CloseHSClientCircuitsImmediatelyOnTimeout** **0**|**1**:: - If 1, Tor will close unfinished hidden service client circuits - which have not moved closer to connecting to their destination - hidden service when their internal state has not changed for the - duration of the current circuit-build timeout. Otherwise, such - circuits will be left open, in the hope that they will finish - connecting to their destination hidden services. In either case, - another set of introduction and rendezvous circuits for the same - destination hidden service will be launched. (Default: 0) - -[[CloseHSServiceRendCircuitsImmediatelyOnTimeout]] **CloseHSServiceRendCircuitsImmediatelyOnTimeout** **0**|**1**:: - If 1, Tor will close unfinished hidden-service-side rendezvous - circuits after the current circuit-build timeout. Otherwise, such - circuits will be left open, in the hope that they will finish - connecting to their destinations. In either case, another - rendezvous circuit for the same destination client will be - launched. (Default: 0) +[[ClientOnionAuthDir]] **ClientOnionAuthDir** __path__:: + Path to the directory containing v3 hidden service authorization files. + Each file is for a single onion address, and the files MUST have the suffix + ".auth_private" (i.e. "bob_onion.auth_private"). The content format MUST be: + + + <onion-address>:descriptor:x25519:<base32-encoded-privkey> + + + The <onion-address> MUST NOT have the ".onion" suffix. The + <base32-encoded-privkey> is the base32 representation of the raw key bytes + only (32 bytes for x25519). See Appendix G in the rend-spec-v3.txt file of + https://spec.torproject.org/[torspec] for more information. [[LongLivedPorts]] **LongLivedPorts** __PORTS__:: A list of ports for services that tend to have long-running connections @@ -1007,7 +1162,8 @@ The following options are useful only for clients (that is, if but never attach a new stream to a circuit that is too old. For hidden services, this applies to the __last__ time a circuit was used, not the first. Circuits with streams constructed with SOCKS authentication via - SocksPorts that have **KeepAliveIsolateSOCKSAuth** ignore this value. + SocksPorts that have **KeepAliveIsolateSOCKSAuth** also remain alive + for MaxCircuitDirtiness seconds after carrying the last such stream. (Default: 10 minutes) [[MaxClientCircuitsPending]] **MaxClientCircuitsPending** __NUM__:: @@ -1056,7 +1212,9 @@ The following options are useful only for clients (that is, if Unsupported and force-disabled when using Unix domain sockets.) **IsolateSOCKSAuth**;; Don't share circuits with streams for which different - SOCKS authentication was provided. (On by default; + SOCKS authentication was provided. (For HTTPTunnelPort + connections, this option looks at the Proxy-Authorization and + X-Tor-Stream-Isolation headers. On by default; you can disable it with **NoIsolateSOCKSAuth**.) **IsolateClientProtocol**;; Don't share circuits with streams using a different protocol. @@ -1069,8 +1227,9 @@ The following options are useful only for clients (that is, if Don't share circuits with streams targeting a different destination address. **KeepAliveIsolateSOCKSAuth**;; - If **IsolateSOCKSAuth** is enabled, keep alive circuits that have - streams with SOCKS authentication set indefinitely. + If **IsolateSOCKSAuth** is enabled, keep alive circuits while they have + at least one stream with SOCKS authentication active. After such a circuit + is idle for more than MaxCircuitDirtiness seconds, it can be closed. **SessionGroup=**__INT__;; If no other isolation rules would prevent it, allow streams on this port to share circuits with streams from every other @@ -1078,6 +1237,7 @@ The following options are useful only for clients (that is, if on different SocksPorts, TransPorts, etc are always isolated from one another. This option overrides that behavior.) +// Anchor only for formatting, not visible in the man page. [[OtherSocksPortFlags]]:: Other recognized __flags__ for a SocksPort are: **NoIPv4Traffic**;; @@ -1103,7 +1263,7 @@ The following options are useful only for clients (that is, if flag is not supported. **CacheIPv4DNS**;; Tells the client to remember IPv4 DNS answers we receive from exit - nodes via this connection. (On by default.) + nodes via this connection. **CacheIPv6DNS**;; Tells the client to remember IPv6 DNS answers we receive from exit nodes via this connection. @@ -1118,8 +1278,8 @@ The following options are useful only for clients (that is, if nodes via this connection. **UseIPv4Cache**;; Tells the client to use any cached IPv4 DNS answers we have when making - requests via this connection. (NOTE: This option, along UseIPv6Cache - and UseDNSCache, can harm your anonymity, and probably + requests via this connection. (NOTE: This option, or UseIPv6Cache + or UseDNSCache, can harm your anonymity, and probably won't help performance as much as you might expect. Use with care!) **UseIPv6Cache**;; Tells the client to use any cached IPv6 DNS answers we have when making @@ -1142,20 +1302,12 @@ The following options are useful only for clients (that is, if authentication" when IsolateSOCKSAuth is disabled, or when this option is set. +// Anchor only for formatting, not visible in the man page. +[[SocksPortFlagsMisc]]:: Flags are processed left to right. If flags conflict, the last flag on the line is used, and all earlier flags are ignored. No error is issued for conflicting flags. -[[SocksListenAddress]] **SocksListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for connections from Socks-speaking - applications. (Default: 127.0.0.1) You can also specify a port (e.g. - 192.168.0.1:9100). This directive can be specified multiple times to bind - to multiple addresses/ports. (DEPRECATED: As of 0.2.3.x-alpha, you can - now use multiple SocksPort entries, and provide addresses for SocksPort - entries, so SocksListenAddress no longer has a purpose. For backward - compatibility, SocksListenAddress is only allowed when SocksPort is just - a port number.) - [[SocksPolicy]] **SocksPolicy** __policy__,__policy__,__...__:: Set an entrance policy for this server, to limit who can connect to the SocksPort and DNSPort ports. The policies have the same form as exit @@ -1168,11 +1320,14 @@ The following options are useful only for clients (that is, if 2 minutes) [[TokenBucketRefillInterval]] **TokenBucketRefillInterval** __NUM__ [**msec**|**second**]:: - Set the refill interval of Tor's token bucket to NUM milliseconds. - NUM must be between 1 and 1000, inclusive. Note that the configured - bandwidth limits are still expressed in bytes per second: this + Set the refill delay interval of Tor's token bucket to NUM milliseconds. + NUM must be between 1 and 1000, inclusive. When Tor is out of bandwidth, + on a connection or globally, it will wait up to this long before it tries + to use that connection again. + Note that bandwidth limits are still expressed in bytes per second: this option only affects the frequency with which Tor checks to see whether - previously exhausted connections may read again. (Default: 100 msec) + previously exhausted connections may read again. + Can not be changed while tor is running. (Default: 100 msec) [[TrackHostExits]] **TrackHostExits** __host__,__.domain__,__...__:: For each value in the comma separated list, Tor will track recent @@ -1206,17 +1361,8 @@ The following options are useful only for clients (that is, if to stick with them. This is desirable because constantly changing servers increases the odds that an adversary who owns some servers will observe a fraction of your paths. Entry Guards can not be used by Directory - Authorities, Single Onion Services, and Tor2web clients. In these cases, - the this option is ignored. (Default: 1) - -[[UseEntryGuardsAsDirGuards]] **UseEntryGuardsAsDirGuards** **0**|**1**:: - If this option is set to 1, and UseEntryGuards is also set to 1, - we try to use our entry guards as directory - guards, and failing that, pick more nodes to act as our directory guards. - This helps prevent an adversary from enumerating clients. It's only - available for clients (non-relay, non-bridge) that aren't configured to - download any non-default directory material. It doesn't currently - do anything when we lack a live consensus. (Default: 1) + Authorities or Single Onion Services. In these cases, + this option is ignored. (Default: 1) [[GuardfractionFile]] **GuardfractionFile** __FILENAME__:: V3 authoritative directories only. Configures the location of the @@ -1224,23 +1370,29 @@ The following options are useful only for clients (that is, if have been guards. (Default: unset) [[UseGuardFraction]] **UseGuardFraction** **0**|**1**|**auto**:: - This torrc option specifies whether clients should use the + This option specifies whether clients should use the guardfraction information found in the consensus during path selection. If it's set to 'auto', clients will do what the UseGuardFraction consensus parameter tells them to do. (Default: auto) [[NumEntryGuards]] **NumEntryGuards** __NUM__:: If UseEntryGuards is set to 1, we will try to pick a total of NUM routers - as long-term entries for our circuits. If NUM is 0, we try to learn - the number from the NumEntryGuards consensus parameter, and default - to 3 if the consensus parameter isn't set. (Default: 0) + as long-term entries for our circuits. If NUM is 0, we try to learn the + number from the guard-n-primary-guards-to-use consensus parameter, and + default to 1 if the consensus parameter isn't set. (Default: 0) + +[[NumPrimaryGuards]] **NumPrimaryGuards** __NUM__:: + If UseEntryGuards is set to 1, we will try to pick NUM routers for our + primary guard list, which is the set of routers we strongly prefer when + connecting to the Tor network. If NUM is 0, we try to learn the number from + the guard-n-primary-guards consensus parameter, and default to 3 if the + consensus parameter isn't set. (Default: 0) [[NumDirectoryGuards]] **NumDirectoryGuards** __NUM__:: - If UseEntryGuardsAsDirectoryGuards is enabled, we try to make sure we - have at least NUM routers to use as directory guards. If this option - is set to 0, use the value from the NumDirectoryGuards consensus - parameter, falling back to the value from NumEntryGuards if the - consensus parameter is 0 or isn't set. (Default: 0) + If UseEntryGuards is set to 1, we try to make sure we have at least NUM + routers to use as directory guards. If this option is set to 0, use the + value from the guard-n-primary-dir-guards-to-use consensus parameter, and + default to 3 if the consensus parameter isn't set. (Default: 0) [[GuardLifetime]] **GuardLifetime** __N__ **days**|**weeks**|**months**:: If nonzero, and UseEntryGuards is set, minimum time to keep a guard before @@ -1262,15 +1414,9 @@ The following options are useful only for clients (that is, if helps to determine whether an application using Tor is possibly leaking DNS requests. (Default: 0) -[[WarnUnsafeSocks]] **WarnUnsafeSocks** **0**|**1**:: - When this option is enabled, Tor will warn whenever a request is - received that only contains an IP address instead of a hostname. Allowing - applications to do DNS resolves themselves is usually a bad idea and - can leak your location to attackers. (Default: 1) +[[VirtualAddrNetworkIPv4]] **VirtualAddrNetworkIPv4** __IPv4Address__/__bits__ + -[[VirtualAddrNetworkIPv4]] **VirtualAddrNetworkIPv4** __Address__/__bits__ + - -[[VirtualAddrNetworkIPv6]] **VirtualAddrNetworkIPv6** [__Address__]/__bits__:: +[[VirtualAddrNetworkIPv6]] **VirtualAddrNetworkIPv6** [__IPv6Address__]/__bits__:: When Tor needs to assign a virtual (unused) address because of a MAPADDRESS command from the controller or the AutomapHostsOnResolve feature, Tor picks an unassigned address from this range. (Defaults: @@ -1293,23 +1439,13 @@ The following options are useful only for clients (that is, if resolved. This helps trap accidental attempts to resolve URLs and so on. (Default: 0) -[[AllowDotExit]] **AllowDotExit** **0**|**1**:: - If enabled, we convert "www.google.com.foo.exit" addresses on the - SocksPort/TransPort/NATDPort into "www.google.com" addresses that exit from - the node "foo". Disabled by default since attacking websites and exit - relays can use it to manipulate your path selection. (Default: 0) - -[[FastFirstHopPK]] **FastFirstHopPK** **0**|**1**|**auto**:: - When this option is disabled, Tor uses the public key step for the first - hop of creating circuits. Skipping it is generally safe since we have - already used TLS to authenticate the relay and to establish forward-secure - keys. Turning this option off makes circuit building a little - slower. Setting this option to "auto" takes advice from the authorities - in the latest consensus about whether to use this feature. + - + - Note that Tor will always use the public key step for the first hop if it's - operating as a relay, and it will never use the public key step if it - doesn't yet know the onion key of the first hop. (Default: auto) +[[HTTPTunnelPort]] **HTTPTunnelPort** \['address':]__port__|**auto** [_isolation flags_]:: + Open this port to listen for proxy connections using the "HTTP CONNECT" + protocol instead of SOCKS. Set this to + 0 if you don't want to allow "HTTP CONNECT" connections. Set the port + to "auto" to have Tor pick a port for you. This directive can be + specified multiple times to bind to multiple addresses/ports. See + SOCKSPort for an explanation of isolation flags. (Default: 0) [[TransPort]] **TransPort** \['address':]__port__|**auto** [_isolation flags_]:: Open this port to listen for transparent proxy connections. Set this to @@ -1321,43 +1457,31 @@ The following options are useful only for clients (that is, if TransPort requires OS support for transparent proxies, such as BSDs' pf or Linux's IPTables. If you're planning to use Tor as a transparent proxy for a network, you'll want to examine and change VirtualAddrNetwork from the - default setting. You'll also want to set the TransListenAddress option for - the network you'd like to proxy. (Default: 0) - -[[TransListenAddress]] **TransListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for transparent proxy connections. (Default: - 127.0.0.1). This is useful for exporting a transparent proxy server to an - entire network. (DEPRECATED: As of 0.2.3.x-alpha, you can - now use multiple TransPort entries, and provide addresses for TransPort - entries, so TransListenAddress no longer has a purpose. For backward - compatibility, TransListenAddress is only allowed when TransPort is just - a port number.) + default setting. (Default: 0) [[TransProxyType]] **TransProxyType** **default**|**TPROXY**|**ipfw**|**pf-divert**:: TransProxyType may only be enabled when there is transparent proxy listener - enabled. + enabled. + + Set this to "TPROXY" if you wish to be able to use the TPROXY Linux module to transparently proxy connections that are configured using the TransPort - option. This setting lets the listener on the TransPort accept connections - for all addresses, even when the TransListenAddress is configured for an - internal address. Detailed information on how to configure the TPROXY + option. Detailed information on how to configure the TPROXY feature can be found in the Linux kernel source tree in the file - Documentation/networking/tproxy.txt. + Documentation/networking/tproxy.txt. + + - Set this option to "ipfw" to use the FreeBSD ipfw interface. + Set this option to "ipfw" to use the FreeBSD ipfw interface. + + On *BSD operating systems when using pf, set this to "pf-divert" to take advantage of +divert-to+ rules, which do not modify the packets like +rdr-to+ rules do. Detailed information on how to configure pf to use +divert-to+ rules can be found in the pf.conf(5) manual page. On OpenBSD, +divert-to+ is available to use on versions greater than or equal to - OpenBSD 4.4. + OpenBSD 4.4. + + Set this to "default", or leave it unconfigured, to use regular IPTables - on Linux, or to use pf +rdr-to+ rules on *BSD systems. + on Linux, or to use pf +rdr-to+ rules on *BSD systems. + + - (Default: "default".) + (Default: "default") [[NATDPort]] **NATDPort** \['address':]__port__|**auto** [_isolation flags_]:: Open this port to listen for connections from old versions of ipfw (as @@ -1369,13 +1493,6 @@ The following options are useful only for clients (that is, if + This option is only for people who cannot use TransPort. (Default: 0) -[[NATDListenAddress]] **NATDListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for NATD connections. (DEPRECATED: As of - 0.2.3.x-alpha, you can now use multiple NATDPort entries, and provide - addresses for NATDPort entries, so NATDListenAddress no longer has a - purpose. For backward compatibility, NATDListenAddress is only allowed - when NATDPort is just a port number.) - [[AutomapHostsOnResolve]] **AutomapHostsOnResolve** **0**|**1**:: When this option is enabled, and we get a request to resolve an address that ends with one of the suffixes in **AutomapHostsSuffixes**, we map an @@ -1396,24 +1513,18 @@ The following options are useful only for clients (that is, if addresses/ports. See SocksPort for an explanation of isolation flags. (Default: 0) -[[DNSListenAddress]] **DNSListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for DNS connections. (DEPRECATED: As of - 0.2.3.x-alpha, you can now use multiple DNSPort entries, and provide - addresses for DNSPort entries, so DNSListenAddress no longer has a - purpose. For backward compatibility, DNSListenAddress is only allowed - when DNSPort is just a port number.) - [[ClientDNSRejectInternalAddresses]] **ClientDNSRejectInternalAddresses** **0**|**1**:: If true, Tor does not believe any anonymously retrieved DNS answer that tells it that an address resolves to an internal address (like 127.0.0.1 or - 192.168.0.1). This option prevents certain browser-based attacks; don't - turn it off unless you know what you're doing. (Default: 1) + 192.168.0.1). This option prevents certain browser-based attacks; it + is not allowed to be set on the default network. (Default: 1) [[ClientRejectInternalAddresses]] **ClientRejectInternalAddresses** **0**|**1**:: If true, Tor does not try to fulfill requests to connect to an internal - address (like 127.0.0.1 or 192.168.0.1) __unless a exit node is + address (like 127.0.0.1 or 192.168.0.1) __unless an exit node is specifically requested__ (for example, via a .exit hostname, or a - controller request). (Default: 1) + controller request). If true, multicast DNS hostnames for machines on the + local network (of the form *.local) are also rejected. (Default: 1) [[DownloadExtraInfo]] **DownloadExtraInfo** **0**|**1**:: If true, Tor downloads and caches "extra-info" documents. These documents @@ -1431,11 +1542,6 @@ The following options are useful only for clients (that is, if Like WarnPlaintextPorts, but instead of warning about risky port uses, Tor will instead refuse to make the connection. (Default: None) -[[AllowSingleHopCircuits]] **AllowSingleHopCircuits** **0**|**1**:: - When this option is set, the attached Tor controller can use relays - that have the **AllowSingleHopExits** option turned on to build - one-hop Tor connections. (Default: 0) - [[OptimisticData]] **OptimisticData** **0**|**1**|**auto**:: When this option is set, and Tor is using an exit node that supports the feature, it will try optimistically to send data to the exit node @@ -1445,40 +1551,123 @@ The following options are useful only for clients (that is, if Tor will look at the UseOptimisticData parameter in the networkstatus. (Default: auto) -[[Tor2webMode]] **Tor2webMode** **0**|**1**:: - When this option is set, Tor connects to hidden services - **non-anonymously**. This option also disables client connections to - non-hidden-service hostnames through Tor. It **must only** be used when - running a tor2web Hidden Service web proxy. - To enable this option the compile time flag --enable-tor2web-mode must be - specified. Since Tor2webMode is non-anonymous, you can not run an - anonymous Hidden Service on a tor version compiled with Tor2webMode. - (Default: 0) - -[[Tor2webRendezvousPoints]] **Tor2webRendezvousPoints** __node__,__node__,__...__:: - A list of identity fingerprints, nicknames, country codes and - address patterns of nodes that are allowed to be used as RPs - in HS circuits; any other nodes will not be used as RPs. - (Example: - Tor2webRendezvousPoints Fastyfasty, ABCD1234CDEF5678ABCD1234CDEF5678ABCD1234, \{cc}, 255.254.0.0/8) + - + - This feature can only be used if Tor2webMode is also enabled. +[[HSLayer2Nodes]] **HSLayer2Nodes** __node__,__node__,__...__:: + A list of identity fingerprints, nicknames, country codes, and + address patterns of nodes that are allowed to be used as the + second hop in all client or service-side Onion Service circuits. + This option mitigates attacks where the adversary runs middle nodes + and induces your client or service to create many circuits, in order + to discover your primary guard node. + (Default: Any node in the network may be used in the second hop.) + - ExcludeNodes have higher priority than Tor2webRendezvousPoints, + (Example: + HSLayer2Nodes ABCD1234CDEF5678ABCD1234CDEF5678ABCD1234, \{cc}, 255.254.0.0/8) + + + + When this is set, the resulting hidden service paths will + look like: + + + C - G - L2 - M - Rend + + C - G - L2 - M - HSDir + + C - G - L2 - M - Intro + + S - G - L2 - M - Rend + + S - G - L2 - M - HSDir + + S - G - L2 - M - Intro + + + + where C is this client, S is the service, G is the Guard node, + L2 is a node from this option, and M is a random middle node. + Rend, HSDir, and Intro point selection is not affected by this + option. + + + This option may be combined with HSLayer3Nodes to create + paths of the form: + + + C - G - L2 - L3 - Rend + + C - G - L2 - L3 - M - HSDir + + C - G - L2 - L3 - M - Intro + + S - G - L2 - L3 - M - Rend + + S - G - L2 - L3 - HSDir + + S - G - L2 - L3 - Intro + + + + ExcludeNodes have higher priority than HSLayer2Nodes, which means that nodes specified in ExcludeNodes will not be - picked as RPs. + picked. + + + When either this option or HSLayer3Nodes are set, the /16 subnet + and node family restrictions are removed for hidden service + circuits. Additionally, we allow the guard node to be present + as the Rend, HSDir, and IP node, and as the hop before it. This + is done to prevent the adversary from inferring information + about our guard, layer2, and layer3 node choices at later points + in the path. + + + This option is meant to be managed by a Tor controller such as + https://github.com/mikeperry-tor/vanguards that selects and + updates this set of nodes for you. Hence it does not do load + balancing if fewer than 20 nodes are selected, and if no nodes in + HSLayer2Nodes are currently available for use, Tor will not work. + Please use extreme care if you are setting this option manually. + +[[HSLayer3Nodes]] **HSLayer3Nodes** __node__,__node__,__...__:: + A list of identity fingerprints, nicknames, country codes, and + address patterns of nodes that are allowed to be used as the + third hop in all client and service-side Onion Service circuits. + This option mitigates attacks where the adversary runs middle nodes + and induces your client or service to create many circuits, in order + to discover your primary or Layer2 guard nodes. + (Default: Any node in the network may be used in the third hop.) + - If no nodes in Tor2webRendezvousPoints are currently available for - use, Tor will choose a random node when building HS circuits. + (Example: + HSLayer3Nodes ABCD1234CDEF5678ABCD1234CDEF5678ABCD1234, \{cc}, 255.254.0.0/8) + + + + When this is set by itself, the resulting hidden service paths + will look like: + + C - G - M - L3 - Rend + + C - G - M - L3 - M - HSDir + + C - G - M - L3 - M - Intro + + S - G - M - L3 - M - Rend + + S - G - M - L3 - HSDir + + S - G - M - L3 - Intro + + where C is this client, S is the service, G is the Guard node, + L2 is a node from this option, and M is a random middle node. + Rend, HSDir, and Intro point selection is not affected by this + option. + + + While it is possible to use this option by itself, it should be + combined with HSLayer2Nodes to create paths of the form: + + + C - G - L2 - L3 - Rend + + C - G - L2 - L3 - M - HSDir + + C - G - L2 - L3 - M - Intro + + S - G - L2 - L3 - M - Rend + + S - G - L2 - L3 - HSDir + + S - G - L2 - L3 - Intro + + + + ExcludeNodes have higher priority than HSLayer3Nodes, + which means that nodes specified in ExcludeNodes will not be + picked. + + + When either this option or HSLayer2Nodes are set, the /16 subnet + and node family restrictions are removed for hidden service + circuits. Additionally, we allow the guard node to be present + as the Rend, HSDir, and IP node, and as the hop before it. This + is done to prevent the adversary from inferring information + about our guard, layer2, and layer3 node choices at later points + in the path. + + + This option is meant to be managed by a Tor controller such as + https://github.com/mikeperry-tor/vanguards that selects and + updates this set of nodes for you. Hence it does not do load + balancing if fewer than 20 nodes are selected, and if no nodes in + HSLayer3Nodes are currently available for use, Tor will not work. + Please use extreme care if you are setting this option manually. [[UseMicrodescriptors]] **UseMicrodescriptors** **0**|**1**|**auto**:: Microdescriptors are a smaller version of the information that Tor needs in order to build its circuits. Using microdescriptors makes Tor clients download less directory information, thus saving bandwidth. Directory caches need to fetch regular descriptors and microdescriptors, so this - option doesn't save any bandwidth for them. If this option is set to - "auto" (recommended) then it is on for all clients that do not set - FetchUselessDescriptors. (Default: auto) + option doesn't save any bandwidth for them. For legacy reasons, auto is + accepted, but it has the same effect as 1. (Default: auto) [[PathBiasCircThreshold]] **PathBiasCircThreshold** __NUM__ + @@ -1494,7 +1683,7 @@ The following options are useful only for clients (that is, if These options override the default behavior of Tor's (**currently experimental**) path bias detection algorithm. To try to find broken or misbehaving guard nodes, Tor looks for nodes where more than a certain - fraction of circuits through that guard fail to get built. + fraction of circuits through that guard fail to get built. + + The PathBiasCircThreshold option controls how many circuits we need to build through a guard before we make these checks. The PathBiasNoticeRate, @@ -1520,14 +1709,14 @@ The following options are useful only for clients (that is, if [[PathBiasScaleUseThreshold]] **PathBiasScaleUseThreshold** __NUM__:: Similar to the above options, these options override the default behavior - of Tor's (**currently experimental**) path use bias detection algorithm. + of Tor's (**currently experimental**) path use bias detection algorithm. + + Where as the path bias parameters govern thresholds for successfully building circuits, these four path use bias parameters govern thresholds only for circuit usage. Circuits which receive no stream usage are not counted by this detection algorithm. A used circuit is considered successful if it is capable of carrying streams or otherwise receiving - well-formed responses to RELAY cells. + well-formed responses to RELAY cells. + + By default, or if a negative value is provided for one of these options, Tor uses reasonable defaults from the networkstatus consensus document. @@ -1542,9 +1731,10 @@ The following options are useful only for clients (that is, if [[ClientUseIPv6]] **ClientUseIPv6** **0**|**1**:: If this option is set to 1, Tor might connect to directory servers or - entry nodes over IPv6. Note that clients configured with an IPv6 address - in a **Bridge**, proxy, or pluggable transport line will try connecting - over IPv6 even if **ClientUseIPv6** is set to 0. (Default: 0) + entry nodes over IPv6. For IPv6 only hosts, you need to also set + **ClientUseIPv4** to 0 to disable IPv4. Note that clients configured with + an IPv6 address in a **Bridge**, proxy, or pluggable transportline will + try connecting over IPv6 even if **ClientUseIPv6** is set to 0. (Default: 0) [[ClientPreferIPv6DirPort]] **ClientPreferIPv6DirPort** **0**|**1**|**auto**:: If this option is set to 1, Tor prefers a directory port with an IPv6 @@ -1552,7 +1742,8 @@ The following options are useful only for clients (that is, if server has both. (Tor also prefers an IPv6 DirPort if IPv4Client is set to 0.) If this option is set to auto, clients prefer IPv4. Other things may influence the choice. This option breaks a tie to the favor of IPv6. - (Default: auto) + (Default: auto) (DEPRECATED: This option has had no effect for some + time.) [[ClientPreferIPv6ORPort]] **ClientPreferIPv6ORPort** **0**|**1**|**auto**:: If this option is set to 1, Tor prefers an OR port with an IPv6 @@ -1573,46 +1764,35 @@ The following options are useful only for clients (that is, if prevent your Tor client from bootstrapping. If this option is negative, Tor will use a default value chosen by the directory authorities. If the directory authorities do not choose a value, Tor will default to 0.6. - (Default: -1.) + (Default: -1) -[[ClientBootstrapConsensusAuthorityDownloadSchedule]] **ClientBootstrapConsensusAuthorityDownloadSchedule** __N__,__N__,__...__:: - Schedule for when clients should download consensuses from authorities +[[ClientBootstrapConsensusAuthorityDownloadInitialDelay]] **ClientBootstrapConsensusAuthorityDownloadInitialDelay** __N__:: + Initial delay in seconds for when clients should download consensuses from authorities if they are bootstrapping (that is, they don't have a usable, reasonably live consensus). Only used by clients fetching from a list of fallback directory mirrors. This schedule is advanced by (potentially concurrent) connection attempts, unlike other schedules, which are advanced by - connection failures. (Default: 10, 11, 3600, 10800, 25200, 54000, - 111600, 262800) + connection failures. (Default: 6) -[[ClientBootstrapConsensusFallbackDownloadSchedule]] **ClientBootstrapConsensusFallbackDownloadSchedule** __N__,__N__,__...__:: - Schedule for when clients should download consensuses from fallback +[[ClientBootstrapConsensusFallbackDownloadInitialDelay]] **ClientBootstrapConsensusFallbackDownloadInitialDelay** __N__:: + Initial delay in seconds for when clients should download consensuses from fallback directory mirrors if they are bootstrapping (that is, they don't have a usable, reasonably live consensus). Only used by clients fetching from a list of fallback directory mirrors. This schedule is advanced by (potentially concurrent) connection attempts, unlike other schedules, - which are advanced by connection failures. (Default: 0, 1, 4, 11, 3600, - 10800, 25200, 54000, 111600, 262800) + which are advanced by connection failures. (Default: 0) -[[ClientBootstrapConsensusAuthorityOnlyDownloadSchedule]] **ClientBootstrapConsensusAuthorityOnlyDownloadSchedule** __N__,__N__,__...__:: - Schedule for when clients should download consensuses from authorities +[[ClientBootstrapConsensusAuthorityOnlyDownloadInitialDelay]] **ClientBootstrapConsensusAuthorityOnlyDownloadInitialDelay** __N__:: + Initial delay in seconds for when clients should download consensuses from authorities if they are bootstrapping (that is, they don't have a usable, reasonably live consensus). Only used by clients which don't have or won't fetch from a list of fallback directory mirrors. This schedule is advanced by (potentially concurrent) connection attempts, unlike other schedules, - which are advanced by connection failures. (Default: 0, 3, 7, 3600, - 10800, 25200, 54000, 111600, 262800) - -[[ClientBootstrapConsensusMaxDownloadTries]] **ClientBootstrapConsensusMaxDownloadTries** __NUM__:: - Try this many times to download a consensus while bootstrapping using - fallback directory mirrors before giving up. (Default: 7) - -[[ClientBootstrapConsensusAuthorityOnlyMaxDownloadTries]] **ClientBootstrapConsensusAuthorityOnlyMaxDownloadTries** __NUM__:: - Try this many times to download a consensus while bootstrapping using - authorities before giving up. (Default: 4) + which are advanced by connection failures. (Default: 0) [[ClientBootstrapConsensusMaxInProgressTries]] **ClientBootstrapConsensusMaxInProgressTries** __NUM__:: Try this many simultaneous connections to download a consensus before - waiting for one to complete, timeout, or error out. (Default: 4) + waiting for one to complete, timeout, or error out. (Default: 3) SERVER OPTIONS -------------- @@ -1621,19 +1801,13 @@ The following options are useful only for servers (that is, if ORPort is non-zero): [[Address]] **Address** __address__:: - The IP address or fully qualified domain name of this server (e.g. - moria.mit.edu). You can leave this unset, and Tor will guess your IP - address. This IP address is the one used to tell clients and other - servers where to find your Tor server; it doesn't affect the IP that your - Tor client binds to. To bind to a different address, use the - *ListenAddress and OutboundBindAddress options. - -[[AllowSingleHopExits]] **AllowSingleHopExits** **0**|**1**:: - This option controls whether clients can use this server as a single hop - proxy. If set to 1, clients can use this server as an exit even if it is - the only hop in the circuit. Note that most clients will refuse to use - servers that set this option, since most clients have - ExcludeSingleHopRelays set. (Default: 0) + The IPv4 address of this server, or a fully qualified domain name of + this server that resolves to an IPv4 address. You can leave this + unset, and Tor will try to guess your IPv4 address. This IPv4 + address is the one used to tell clients and other servers where to + find your Tor server; it doesn't affect the address that your server + binds to. To bind to a different address, use the ORPort and + OutboundBindAddress options. [[AssumeReachable]] **AssumeReachable** **0**|**1**:: This option is used when bootstrapping a new Tor network. If set to 1, @@ -1646,7 +1820,21 @@ is non-zero): Sets the relay to act as a "bridge" with respect to relaying connections from bridge users to the Tor network. It mainly causes Tor to publish a server descriptor to the bridge database, rather than - to the public directory authorities. + to the public directory authorities. + + + + Note: make sure that no MyFamily lines are present in your torrc when + relay is configured in bridge mode. + +[[BridgeDistribution]] **BridgeDistribution** __string__:: + If set along with BridgeRelay, Tor will include a new line in its + bridge descriptor which indicates to the BridgeDB service how it + would like its bridge address to be given out. Set it to "none" if + you want BridgeDB to avoid distributing your bridge address, or "any" to + let BridgeDB decide. (Default: any) + + + Note: as of Oct 2017, the BridgeDB part of this option is not yet + implemented. Until BridgeDB is updated to obey this option, your + bridge will make this request, but it will not (yet) be obeyed. [[ContactInfo]] **ContactInfo** __email_address__:: Administrative contact information for this relay or bridge. This line @@ -1655,28 +1843,34 @@ is non-zero): descriptors containing these lines and that Google indexes them, so spammers might also collect them. You may want to obscure the fact that it's an email address and/or generate a new address for this - purpose. + purpose. + + + + ContactInfo **must** be set to a working address if you run more than one + relay or bridge. (Really, everybody running a relay or bridge should set + it.) + [[ExitRelay]] **ExitRelay** **0**|**1**|**auto**:: Tells Tor whether to run as an exit relay. If Tor is running as a non-bridge server, and ExitRelay is set to 1, then Tor allows traffic to - exit according to the ExitPolicy option (or the default ExitPolicy if - none is specified). + exit according to the ExitPolicy option, the ReducedExitPolicy option, + or the default ExitPolicy (if no other exit policy option is specified). + + If ExitRelay is set to 0, no traffic is allowed to - exit, and the ExitPolicy option is ignored. + + exit, and the ExitPolicy and ReducedExitPolicy options are ignored. + + - If ExitRelay is set to "auto", then Tor behaves as if it were set to 1, but - warns the user if this would cause traffic to exit. In a future version, - the default value will be 0. (Default: auto) + If ExitRelay is set to "auto", then Tor checks the ExitPolicy and + ReducedExitPolicy options. If either is set, Tor behaves as if ExitRelay + were set to 1. If neither exit policy option is set, Tor behaves as if + ExitRelay were set to 0. (Default: auto) [[ExitPolicy]] **ExitPolicy** __policy__,__policy__,__...__:: Set an exit policy for this server. Each policy is of the form "**accept[6]**|**reject[6]** __ADDR__[/__MASK__][:__PORT__]". If /__MASK__ is omitted then this policy just applies to the host given. Instead of giving a host or network you can also use "\*" to denote the universe (0.0.0.0/0 - and ::/128), or \*4 to denote all IPv4 addresses, and \*6 to denote all - IPv6 addresses. + and ::/0), or \*4 to denote all IPv4 addresses, and \*6 to denote all IPv6 + addresses. __PORT__ can be a single port number, an interval of ports "__FROM_PORT__-__TO_PORT__", or "\*". If __PORT__ is omitted, that means "\*". + @@ -1724,8 +1918,15 @@ is non-zero): write your IPv6 rules using accept6/reject6 \*6, and your IPv4 rules using accept/reject \*4. If you want to \_replace_ the default exit policy, end your exit policy with either a reject \*:* or an accept \*:*. Otherwise, - you're \_augmenting_ (prepending to) the default exit policy. The default - exit policy is: + + you're \_augmenting_ (prepending to) the default exit policy. + + + + If you want to use a reduced exit policy rather than the default exit + policy, set "ReducedExitPolicy 1". If you want to _replace_ the default + exit policy with your custom exit policy, end your exit policy with either + a reject *:* or an accept *:*. Otherwise, you're _augmenting_ (prepending + to) the default or reduced exit policy. + + + + The default exit policy is: reject *:25 reject *:119 @@ -1739,6 +1940,8 @@ is non-zero): reject *:6881-6999 accept *:* +// Anchor only for formatting, not visible in the man page. +[[ExitPolicyDefault]]:: Since the default exit policy uses accept/reject *, it applies to both IPv4 and IPv6 addresses. @@ -1760,6 +1963,99 @@ is non-zero): to disclose. (Default: 0) +[[ReducedExitPolicy]] **ReducedExitPolicy** **0**|**1**:: + If set, use a reduced exit policy rather than the default one. + + + + The reduced exit policy is an alternative to the default exit policy. It + allows as many Internet services as possible while still blocking the + majority of TCP ports. Currently, the policy allows approximately 65 ports. + This reduces the odds that your node will be used for peer-to-peer + applications. + + + + The reduced exit policy is: + + accept *:20-21 + accept *:22 + accept *:23 + accept *:43 + accept *:53 + accept *:79 + accept *:80-81 + accept *:88 + accept *:110 + accept *:143 + accept *:194 + accept *:220 + accept *:389 + accept *:443 + accept *:464 + accept *:465 + accept *:531 + accept *:543-544 + accept *:554 + accept *:563 + accept *:587 + accept *:636 + accept *:706 + accept *:749 + accept *:873 + accept *:902-904 + accept *:981 + accept *:989-990 + accept *:991 + accept *:992 + accept *:993 + accept *:994 + accept *:995 + accept *:1194 + accept *:1220 + accept *:1293 + accept *:1500 + accept *:1533 + accept *:1677 + accept *:1723 + accept *:1755 + accept *:1863 + accept *:2082 + accept *:2083 + accept *:2086-2087 + accept *:2095-2096 + accept *:2102-2104 + accept *:3128 + accept *:3389 + accept *:3690 + accept *:4321 + accept *:4643 + accept *:5050 + accept *:5190 + accept *:5222-5223 + accept *:5228 + accept *:5900 + accept *:6660-6669 + accept *:6679 + accept *:6697 + accept *:8000 + accept *:8008 + accept *:8074 + accept *:8080 + accept *:8082 + accept *:8087-8088 + accept *:8232-8233 + accept *:8332-8333 + accept *:8443 + accept *:8888 + accept *:9418 + accept *:9999 + accept *:10000 + accept *:11371 + accept *:19294 + accept *:19638 + accept *:50002 + accept *:64738 + reject *:* + + (Default: 0) + [[IPv6Exit]] **IPv6Exit** **0**|**1**:: If set, and we are an exit node, allow clients to use us for IPv6 traffic. (Default: 0) @@ -1768,21 +2064,33 @@ is non-zero): If we have more onionskins queued for processing than we can process in this amount of time, reject new ones. (Default: 1750 msec) -[[MyFamily]] **MyFamily** __node__,__node__,__...__:: - Declare that this Tor server is controlled or administered by a group or - organization identical or similar to that of the other servers, defined by - their identity fingerprints. When two servers both declare - that they are in the same \'family', Tor clients will not use them in the - same circuit. (Each server only needs to list the other servers in its - family; it doesn't need to list itself, but it won't hurt.) Do not list - any bridge relay as it would compromise its concealment. +[[MyFamily]] **MyFamily** __fingerprint__,__fingerprint__,...:: + Declare that this Tor relay is controlled or administered by a group or + organization identical or similar to that of the other relays, defined by + their (possibly $-prefixed) identity fingerprints. + This option can be repeated many times, for + convenience in defining large families: all fingerprints in all MyFamily + lines are merged into one list. + When two relays both declare that they are in the + same \'family', Tor clients will not use them in the same circuit. (Each + relay only needs to list the other servers in its family; it doesn't need to + list itself, but it won't hurt if it does.) Do not list any bridge relay as it would + compromise its concealment. + + When listing a node, it's better to list it by fingerprint than by - nickname: fingerprints are more reliable. + nickname: fingerprints are more reliable. + + + + If you run more than one relay, the MyFamily option on each relay + **must** list all other relays, as described above. + + + + Note: do not use MyFamily when configuring your Tor instance as a + brigde. [[Nickname]] **Nickname** __name__:: Set the server's nickname to \'name'. Nicknames must be between 1 and 19 characters inclusive, and must contain only the characters [a-zA-Z0-9]. + If not set, **Unnamed** will be used. Relays can always be uniquely identified + by their identity fingerprints. [[NumCPUs]] **NumCPUs** __num__:: How many processes to use at once for decrypting onionskins and other @@ -1793,62 +2101,45 @@ is non-zero): Advertise this port to listen for connections from Tor clients and servers. This option is required to be a Tor server. Set it to "auto" to have Tor pick a port for you. Set it to 0 to not - run an ORPort at all. This option can occur more than once. (Default: 0) -+ + run an ORPort at all. This option can occur more than once. (Default: 0) + + + Tor recognizes these flags on each ORPort: - **NoAdvertise**:: + **NoAdvertise**;; By default, we bind to a port and tell our users about it. If NoAdvertise is specified, we don't advertise, but listen anyway. This can be useful if the port everybody will be connecting to (for example, one that's opened on our firewall) is somewhere else. - **NoListen**:: + **NoListen**;; By default, we bind to a port and tell our users about it. If NoListen is specified, we don't bind, but advertise anyway. This can be useful if something else (for example, a firewall's port forwarding configuration) is causing connections to reach us. - **IPv4Only**:: + **IPv4Only**;; If the address is absent, or resolves to both an IPv4 and an IPv6 address, only listen to the IPv4 address. - **IPv6Only**:: + **IPv6Only**;; If the address is absent, or resolves to both an IPv4 and an IPv6 address, only listen to the IPv6 address. -+ + +// Anchor only for formatting, not visible in the man page. +[[ORPortFlagsExclusive]]:: For obvious reasons, NoAdvertise and NoListen are mutually exclusive, and IPv4Only and IPv6Only are mutually exclusive. -[[ORListenAddress]] **ORListenAddress** __IP__[:__PORT__]:: - Bind to this IP address to listen for connections from Tor clients and - servers. If you specify a port, bind to this port rather than the one - specified in ORPort. (Default: 0.0.0.0) This directive can be specified - multiple times to bind to multiple addresses/ports. -+ - This option is deprecated; you can get the same behavior with ORPort now - that it supports NoAdvertise and explicit addresses. - -[[PortForwarding]] **PortForwarding** **0**|**1**:: - Attempt to automatically forward the DirPort and ORPort on a NAT router - connecting this Tor server to the Internet. If set, Tor will try both - NAT-PMP (common on Apple routers) and UPnP (common on routers from other - manufacturers). (Default: 0) - -[[PortForwardingHelper]] **PortForwardingHelper** __filename__|__pathname__:: - If PortForwarding is set, use this executable to configure the forwarding. - If set to a filename, the system path will be searched for the executable. - If set to a path, only the specified path will be executed. - (Default: tor-fw-helper) - [[PublishServerDescriptor]] **PublishServerDescriptor** **0**|**1**|**v3**|**bridge**,**...**:: This option specifies which descriptors Tor will publish when acting as a relay. You can - choose multiple arguments, separated by commas. + choose multiple arguments, separated by commas. + + If this option is set to 0, Tor will not publish its descriptors to any directories. (This is useful if you're testing - out your server, or if you're using a Tor controller that handles directory - publishing for you.) Otherwise, Tor will publish its descriptors of all - type(s) specified. The default is "1", - which means "if running as a server, publish the - appropriate descriptors to the authorities". + out your server, or if you're using a Tor controller that handles + directory publishing for you.) Otherwise, Tor will publish its + descriptors of all type(s) specified. The default is "1", which + means "if running as a relay or bridge, publish descriptors to the + appropriate authorities". Other possibilities are "v3", meaning + "publish as if you're a relay", and "bridge", meaning "publish as + if you're a bridge". [[ShutdownWaitLength]] **ShutdownWaitLength** __NUM__:: When we get a SIGINT and we're a server, we begin shutting down: @@ -1868,7 +2159,12 @@ is non-zero): to 0 will disable the heartbeat. Otherwise, it must be at least 30 minutes. (Default: 6 hours) -[[AccountingMax]] **AccountingMax** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**|**TBytes**:: +[[MainloopStats]] **MainloopStats** **0**|**1**:: + Log main loop statistics every **HeartbeatPeriod** seconds. This is a log + level __notice__ message designed to help developers instrumenting Tor's + main event loop. (Default: 0) + +[[AccountingMax]] **AccountingMax** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Limits the max number of bytes sent and received within a set time period using a given calculation rule (see: AccountingStart, AccountingRule). Useful if you need to stay under a specific bandwidth. By default, the @@ -1898,15 +2194,16 @@ is non-zero): (Default: max) [[AccountingStart]] **AccountingStart** **day**|**week**|**month** [__day__] __HH:MM__:: - Specify how long accounting periods last. If **month** is given, each - accounting period runs from the time __HH:MM__ on the __dayth__ day of one - month to the same day and time of the next. (The day must be between 1 and - 28.) If **week** is given, each accounting period runs from the time __HH:MM__ - of the __dayth__ day of one week to the same day and time of the next week, - with Monday as day 1 and Sunday as day 7. If **day** is given, each - accounting period runs from the time __HH:MM__ each day to the same time on - the next day. All times are local, and given in 24-hour time. (Default: - "month 1 0:00") + Specify how long accounting periods last. If **month** is given, + each accounting period runs from the time __HH:MM__ on the __dayth__ day of one + month to the same day and time of the next. The relay will go at full speed, + use all the quota you specify, then hibernate for the rest of the period. (The + day must be between 1 and 28.) If **week** is given, each accounting period + runs from the time __HH:MM__ of the __dayth__ day of one week to the same day + and time of the next week, with Monday as day 1 and Sunday as day 7. If **day** + is given, each accounting period runs from the time __HH:MM__ each day to the + same time on the next day. All times are local, and given in 24-hour time. + (Default: "month 1 0:00") [[RefuseUnknownExits]] **RefuseUnknownExits** **0**|**1**|**auto**:: Prevent nodes that don't appear in the consensus from exiting using this @@ -1942,7 +2239,7 @@ is non-zero): correct this. This option only affects name lookups that your server does on behalf of clients. (Default: 1) -[[ServerDNSTestAddresses]] **ServerDNSTestAddresses** __address__,__address__,__...__:: +[[ServerDNSTestAddresses]] **ServerDNSTestAddresses** __hostname__,__hostname__,__...__:: When we're detecting DNS hijacking, make sure that these __valid__ addresses aren't getting redirected. If they are, then our DNS is completely useless, and we'll reset our exit policy to "reject \*:*". This option only affects @@ -1976,12 +2273,6 @@ is non-zero): [[GeoIPv6File]] **GeoIPv6File** __filename__:: A filename containing IPv6 GeoIP data, for use with by-country statistics. -[[TLSECGroup]] **TLSECGroup** **P224**|**P256**:: - What EC group should we try to use for incoming TLS connections? - P224 is faster, but makes us stand out more. Has no effect if - we're a client, or if our OpenSSL version lacks support for ECDHE. - (Default: P256) - [[CellStatistics]] **CellStatistics** **0**|**1**:: Relays only. When this option is enabled, Tor collects statistics about cell @@ -1992,6 +2283,15 @@ is non-zero): If ExtraInfoStatistics is enabled, it will published as part of extra-info document. (Default: 0) +[[PaddingStatistics]] **PaddingStatistics** **0**|**1**:: + Relays and bridges only. + When this option is enabled, Tor collects statistics for padding cells + sent and received by this relay, in addition to total cell counts. + These statistics are rounded, and omitted if traffic is low. This + information is important for load balancing decisions related to padding. + If ExtraInfoStatistics is enabled, it will be published + as a part of extra-info document. (Default: 1) + [[DirReqStatistics]] **DirReqStatistics** **0**|**1**:: Relays and bridges only. When this option is enabled, a Tor directory writes statistics on the @@ -2079,11 +2379,28 @@ is non-zero): ed25519 master identity key, as well as the corresponding temporary signing keys and certificates. (Default: 0) +[[KeyDirectory]] **KeyDirectory** __DIR__:: + Store secret keys in DIR. Can not be changed while tor is + running. + (Default: the "keys" subdirectory of DataDirectory.) + +[[KeyDirectoryGroupReadable]] **KeyDirectoryGroupReadable** **0**|**1**:: + If this option is set to 0, don't allow the filesystem group to read the + KeywDirectory. If the option is set to 1, make the KeyDirectory readable + by the default GID. (Default: 0) + +[[RephistTrackTime]] **RephistTrackTime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: + Tells an authority, or other node tracking node reliability and history, + that fine-grained information about nodes can be discarded when it hasn't + changed for a given amount of time. (Default: 24 hours) + + DIRECTORY SERVER OPTIONS ------------------------ -The following options are useful only for directory servers (that is, -if DirPort is non-zero): +The following options are useful only for directory servers. (Relays with +enough bandwidth automatically become directory servers; see DirCache for +details.) [[DirPortFrontPage]] **DirPortFrontPage** __FILENAME__:: When this option is set, it takes an HTML file and publishes it as "/" on @@ -2095,19 +2412,10 @@ if DirPort is non-zero): If this option is nonzero, advertise the directory service on this port. Set it to "auto" to have Tor pick a port for you. This option can occur more than once, but only one advertised DirPort is supported: all - but one DirPort must have the **NoAdvertise** flag set. (Default: 0) -+ + but one DirPort must have the **NoAdvertise** flag set. (Default: 0) + + + The same flags are supported here as are supported by ORPort. -[[DirListenAddress]] **DirListenAddress** __IP__[:__PORT__]:: - Bind the directory service to this address. If you specify a port, bind to - this port rather than the one specified in DirPort. (Default: 0.0.0.0) - This directive can be specified multiple times to bind to multiple - addresses/ports. -+ - This option is deprecated; you can get the same behavior with DirPort now - that it supports NoAdvertise and explicit addresses. - [[DirPolicy]] **DirPolicy** __policy__,__policy__,__...__:: Set an entrance policy for this server, to limit who can connect to the directory ports. The policies have the same form as exit policies above, @@ -2115,10 +2423,152 @@ if DirPort is non-zero): some entry in the policy is accepted. [[DirCache]] **DirCache** **0**|**1**:: - When this option is set, Tor caches all current directory documents and - accepts client requests for them. Setting DirPort is not required for this, - because clients connect via the ORPort by default. Setting either DirPort - or BridgeRelay and setting DirCache to 0 is not supported. (Default: 1) + When this option is set, Tor caches all current directory documents except + extra info documents, and accepts client requests for them. If + **DownloadExtraInfo** is set, cached extra info documents are also cached. + Setting **DirPort** is not required for **DirCache**, because clients + connect via the ORPort by default. Setting either DirPort or BridgeRelay + and setting DirCache to 0 is not supported. (Default: 1) + +[[MaxConsensusAgeForDiffs]] **MaxConsensusAgeForDiffs** __N__ **minutes**|**hours**|**days**|**weeks**:: + When this option is nonzero, Tor caches will not try to generate + consensus diffs for any consensus older than this amount of time. + If this option is set to zero, Tor will pick a reasonable default from + the current networkstatus document. You should not set this + option unless your cache is severely low on disk space or CPU. + If you need to set it, keeping it above 3 or 4 hours will help clients + much more than setting it to zero. + (Default: 0) + + +DENIAL OF SERVICE MITIGATION OPTIONS +------------------------------------ + +Tor has three built-in mitigation options that can be individually +enabled/disabled and fine-tuned, but by default Tor directory authorities will +define reasonable values for relays and no explicit configuration is required +to make use of these protections. The mitigations take place at relays, +and are as follows: + + 1. If a single client address makes too many concurrent connections (this is + configurable via DoSConnectionMaxConcurrentCount), hang up on further + connections. + + + 2. If a single client IP address (v4 or v6) makes circuits too quickly + (default values are more than 3 per second, with an allowed burst of 90, + see DoSCircuitCreationRate and DoSCircuitCreationBurst) while also having + too many connections open (default is 3, see + DoSCircuitCreationMinConnections), tor will refuse any new circuit (CREATE + cells) for the next while (random value between 1 and 2 hours). + + + 3. If a client asks to establish a rendezvous point to you directly (ex: + Tor2Web client), ignore the request. + +These defenses can be manually controlled by torrc options, but relays will +also take guidance from consensus parameters using these same names, so there's +no need to configure anything manually. In doubt, do not change those values. + +The values set by the consensus, if any, can be found here: +https://consensus-health.torproject.org/#consensusparams + +If any of the DoS mitigations are enabled, a heartbeat message will appear in +your log at NOTICE level which looks like: + + DoS mitigation since startup: 429042 circuits rejected, 17 marked addresses. + 2238 connections closed. 8052 single hop clients refused. + +The following options are useful only for a public relay. They control the +Denial of Service mitigation subsystem described above. + +[[DoSCircuitCreationEnabled]] **DoSCircuitCreationEnabled** **0**|**1**|**auto**:: + + Enable circuit creation DoS mitigation. If set to 1 (enabled), tor will + cache client IPs along with statistics in order to detect circuit DoS + attacks. If an address is positively identified, tor will activate + defenses against the address. See the DoSCircuitCreationDefenseType option + for more details. This is a client to relay detection only. "auto" means + use the consensus parameter. If not defined in the consensus, the value is 0. + (Default: auto) + +[[DoSCircuitCreationMinConnections]] **DoSCircuitCreationMinConnections** __NUM__:: + + Minimum threshold of concurrent connections before a client address can be + flagged as executing a circuit creation DoS. In other words, once a client + address reaches the circuit rate and has a minimum of NUM concurrent + connections, a detection is positive. "0" means use the consensus + parameter. If not defined in the consensus, the value is 3. + (Default: 0) + +[[DoSCircuitCreationRate]] **DoSCircuitCreationRate** __NUM__:: + + The allowed circuit creation rate per second applied per client IP + address. If this option is 0, it obeys a consensus parameter. If not + defined in the consensus, the value is 3. + (Default: 0) + +[[DoSCircuitCreationBurst]] **DoSCircuitCreationBurst** __NUM__:: + + The allowed circuit creation burst per client IP address. If the circuit + rate and the burst are reached, a client is marked as executing a circuit + creation DoS. "0" means use the consensus parameter. If not defined in the + consensus, the value is 90. + (Default: 0) + +[[DoSCircuitCreationDefenseType]] **DoSCircuitCreationDefenseType** __NUM__:: + + This is the type of defense applied to a detected client address. The + possible values are: + + + 1: No defense. + + + 2: Refuse circuit creation for the DoSCircuitCreationDefenseTimePeriod period of time. + + + "0" means use the consensus parameter. If not defined in the consensus, the value is 2. + (Default: 0) + +[[DoSCircuitCreationDefenseTimePeriod]] **DoSCircuitCreationDefenseTimePeriod** __N__ **seconds**|**minutes**|**hours**:: + + The base time period in seconds that the DoS defense is activated for. The + actual value is selected randomly for each activation from N+1 to 3/2 * N. + "0" means use the consensus parameter. If not defined in the consensus, + the value is 3600 seconds (1 hour). + (Default: 0) + +[[DoSConnectionEnabled]] **DoSConnectionEnabled** **0**|**1**|**auto**:: + + Enable the connection DoS mitigation. If set to 1 (enabled), for client + address only, this allows tor to mitigate against large number of + concurrent connections made by a single IP address. "auto" means use the + consensus parameter. If not defined in the consensus, the value is 0. + (Default: auto) + +[[DoSConnectionMaxConcurrentCount]] **DoSConnectionMaxConcurrentCount** __NUM__:: + + The maximum threshold of concurrent connection from a client IP address. + Above this limit, a defense selected by DoSConnectionDefenseType is + applied. "0" means use the consensus parameter. If not defined in the + consensus, the value is 100. + (Default: 0) + +[[DoSConnectionDefenseType]] **DoSConnectionDefenseType** __NUM__:: + + This is the type of defense applied to a detected client address for the + connection mitigation. The possible values are: + + + 1: No defense. + + + 2: Immediately close new connections. + + + "0" means use the consensus parameter. If not defined in the consensus, the value is 2. + (Default: 0) + +[[DoSRefuseSingleHopClientRendezvous]] **DoSRefuseSingleHopClientRendezvous** **0**|**1**|**auto**:: + + Refuse establishment of rendezvous points for single hop clients. In other + words, if a client directly connects to the relay and sends an + ESTABLISH_RENDEZVOUS cell, it is silently dropped. "auto" means use the + consensus parameter. If not defined in the consensus, the value is 0. + (Default: auto) DIRECTORY AUTHORITY SERVER OPTIONS @@ -2199,7 +2649,7 @@ on the public Tor network. [[AuthDirBadExit]] **AuthDirBadExit** __AddressPattern...__:: Authoritative directories only. A set of address patterns for servers that will be listed as bad exits in any network status document this authority - publishes, if **AuthDirListBadExits** is set. + publishes, if **AuthDirListBadExits** is set. + + (The address pattern syntax here and in the options below is the same as for exit policies, except that you don't need to say @@ -2237,26 +2687,22 @@ on the public Tor network. list as acceptable on a single IP address. Set this to "0" for "no limit". (Default: 2) -[[AuthDirMaxServersPerAuthAddr]] **AuthDirMaxServersPerAuthAddr** __NUM__:: - Authoritative directories only. Like AuthDirMaxServersPerAddr, but applies - to addresses shared with directory authorities. (Default: 5) - -[[AuthDirFastGuarantee]] **AuthDirFastGuarantee** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[AuthDirFastGuarantee]] **AuthDirFastGuarantee** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Authoritative directories only. If non-zero, always vote the Fast flag for any relay advertising this amount of capacity or more. (Default: 100 KBytes) -[[AuthDirGuardBWGuarantee]] **AuthDirGuardBWGuarantee** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[AuthDirGuardBWGuarantee]] **AuthDirGuardBWGuarantee** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Authoritative directories only. If non-zero, this advertised capacity or more is always sufficient to satisfy the bandwidth requirement - for the Guard flag. (Default: 250 KBytes) + for the Guard flag. (Default: 2 MBytes) [[AuthDirPinKeys]] **AuthDirPinKeys** **0**|**1**:: Authoritative directories only. If non-zero, do not allow any relay to publish a descriptor if any other relay has reserved its <Ed25519,RSA> identity keypair. In all cases, Tor records every keypair it accepts in a journal if it is new, or if it differs from the most recently - accepted pinning for one of the keys it contains. (Default: 0) + accepted pinning for one of the keys it contains. (Default: 1) [[AuthDirSharedRandomness]] **AuthDirSharedRandomness** **0**|**1**:: Authoritative directories only. Switch for the shared random protocol. @@ -2264,6 +2710,13 @@ on the public Tor network. (default), the flag "shared-rand-participate" is added to the authority vote indicating participation in the protocol. (Default: 1) +[[AuthDirTestEd25519LinkKeys]] **AuthDirTestEd25519LinkKeys** **0**|**1**:: + Authoritative directories only. If this option is set to 0, then we treat + relays as "Running" if their RSA key is correct when we probe them, + regardless of their Ed25519 key. We should only ever set this option to 0 + if there is some major bug in Ed25519 link authentication that causes us + to label all the relays as not Running. (Default: 1) + [[BridgePassword]] **BridgePassword** __Password__:: If set, contains an HTTP authenticator that tells a bridge authority to serve all requested bridge information. Used by the (only partially @@ -2302,7 +2755,9 @@ on the public Tor network. [[V3BandwidthsFile]] **V3BandwidthsFile** __FILENAME__:: V3 authoritative directories only. Configures the location of the bandwidth-authority generated file storing information on relays' measured - bandwidth capacities. (Default: unset) + bandwidth capacities. To avoid inconsistent reads, bandwidth data should + be written to temporary file, then renamed to the configured filename. + (Default: unset) [[V3AuthUseLegacyKey]] **V3AuthUseLegacyKey** **0**|**1**:: If set, the directory authority will sign consensuses not only with its @@ -2310,16 +2765,31 @@ on the public Tor network. different identity. This feature is used to migrate directory authority keys in the event of a compromise. (Default: 0) -[[RephistTrackTime]] **RephistTrackTime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: - Tells an authority, or other node tracking node reliability and history, - that fine-grained information about nodes can be discarded when it hasn't - changed for a given amount of time. (Default: 24 hours) - [[AuthDirHasIPv6Connectivity]] **AuthDirHasIPv6Connectivity** **0**|**1**:: Authoritative directories only. When set to 0, OR ports with an - IPv6 address are being accepted without reachability testing. - When set to 1, IPv6 OR ports are being tested just like IPv4 OR - ports. (Default: 0) + IPv6 address are not included in the authority's votes. When set to 1, + IPv6 OR ports are tested for reachability like IPv4 OR ports. If the + reachability test succeeds, the authority votes for the IPv6 ORPort, and + votes Running for the relay. If the reachability test fails, the authority + does not vote for the IPv6 ORPort, and does not vote Running (Default: 0) + ++ + The content of the consensus depends on the number of voting authorities + that set AuthDirHasIPv6Connectivity: + + If no authorities set AuthDirHasIPv6Connectivity 1, there will be no + IPv6 ORPorts in the consensus. + + If a minority of authorities set AuthDirHasIPv6Connectivity 1, + unreachable IPv6 ORPorts will be removed from the consensus. But the + majority of IPv4-only authorities will still vote the relay as Running. + Reachable IPv6 ORPort lines will be included in the consensus + + If a majority of voting authorities set AuthDirHasIPv6Connectivity 1, + relays with unreachable IPv6 ORPorts will not be listed as Running. + Reachable IPv6 ORPort lines will be included in the consensus + (To ensure that any valid majority will vote relays with unreachable + IPv6 ORPorts not Running, 75% of authorities must set + AuthDirHasIPv6Connectivity 1.) [[MinMeasuredBWsForAuthToIgnoreAdvertised]] **MinMeasuredBWsForAuthToIgnoreAdvertised** __N__:: A total value, in abstract bandwidth units, describing how much @@ -2335,9 +2805,9 @@ The following options are used to configure a hidden service. [[HiddenServiceDir]] **HiddenServiceDir** __DIRECTORY__:: Store data files for a hidden service in DIRECTORY. Every hidden service must have a separate directory. You may use this option multiple times to - specify multiple services. DIRECTORY must be an existing directory. + specify multiple services. If DIRECTORY does not exist, Tor will create it. (Note: in current versions of Tor, if DIRECTORY is a relative path, - it will be relative to current + it will be relative to the current working directory of Tor instance, not to its DataDirectory. Do not rely on this behavior; it is not guaranteed to remain the same in future versions.) @@ -2352,7 +2822,7 @@ The following options are used to configure a hidden service. paths may be quoted, and may use standard C escapes.) You may also have multiple lines with the same VIRTPORT: when a user connects to that VIRTPORT, one of the TARGETs from those lines will be - chosen at random. + chosen at random. Note that address-port pairs have to be comma-separated. [[PublishHidServDescriptors]] **PublishHidServDescriptors** **0**|**1**:: If set to 0, Tor will run any hidden services you configure, but it won't @@ -2360,9 +2830,9 @@ The following options are used to configure a hidden service. you're using a Tor controller that handles hidserv publishing for you. (Default: 1) -[[HiddenServiceVersion]] **HiddenServiceVersion** __version__,__version__,__...__:: +[[HiddenServiceVersion]] **HiddenServiceVersion** **2**|**3**:: A list of rendezvous service descriptor versions to publish for the hidden - service. Currently, only version 2 is supported. (Default: 2) + service. Currently, versions 2 and 3 are supported. (Default: 3) [[HiddenServiceAuthorizeClient]] **HiddenServiceAuthorizeClient** __auth-type__ __client-name__,__client-name__,__...__:: If configured, the hidden service is accessible for authorized clients @@ -2374,7 +2844,9 @@ The following options are used to configure a hidden service. spaces). If this option is set, the hidden service is not accessible for clients without authorization any more. Generated authorization data can be found in the hostname file. Clients need to put this authorization data in - their configuration file using **HidServAuth**. + their configuration file using **HidServAuth**. This option is only for v2 + services; v3 services configure client authentication in a subdirectory of + HiddenServiceDir instead (see the **Client Authorization** section). [[HiddenServiceAllowUnknownPorts]] **HiddenServiceAllowUnknownPorts** **0**|**1**:: If set to 1, then connections to unrecognized ports do not cause the @@ -2382,10 +2854,37 @@ The following options are used to configure a hidden service. not an authorization mechanism; it is instead meant to be a mild inconvenience to port-scanners.) (Default: 0) +[[HiddenServiceExportCircuitID]] **HiddenServiceExportCircuitID** __protocol__:: + The onion service will use the given protocol to expose the global circuit + identifier of each inbound client circuit via the selected protocol. The only + protocol supported right now \'haproxy'. This option is only for v3 + services. (Default: none) + + + + The haproxy option works in the following way: when the feature is + enabled, the Tor process will write a header line when a client is connecting + to the onion service. The header will look like this: + + + + "PROXY TCP6 fc00:dead:beef:4dad::ffff:ffff ::1 65535 42\r\n" + + + + We encode the "global circuit identifier" as the last 32-bits of the first + IPv6 address. All other values in the header can safely be ignored. You can + compute the global circuit identifier using the following formula given the + IPv6 address "fc00:dead:beef:4dad::AABB:CCDD": + + + + global_circuit_id = (0xAA << 24) + (0xBB << 16) + (0xCC << 8) + 0xDD; + + + + In the case above, where the last 32-bit is 0xffffffff, the global circuit + identifier would be 4294967295. You can use this value together with Tor's + control port where it is possible to terminate a circuit given the global + circuit identifier. For more information about this see controls-spec.txt. + + + + The HAProxy version 1 proxy protocol is described in detail at + https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt + [[HiddenServiceMaxStreams]] **HiddenServiceMaxStreams** __N__:: The maximum number of simultaneous streams (connections) per rendezvous - circuit. (Setting this to 0 will allow an unlimited number of simultanous - streams.) (Default: 0) + circuit. The maximum value allowed is 65535. (Setting this to 0 will allow + an unlimited number of simultaneous streams.) (Default: 0) [[HiddenServiceMaxStreamsCloseCircuit]] **HiddenServiceMaxStreamsCloseCircuit** **0**|**1**:: If set to 1, then exceeding **HiddenServiceMaxStreams** will cause the @@ -2394,8 +2893,10 @@ The following options are used to configure a hidden service. [[RendPostPeriod]] **RendPostPeriod** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: Every time the specified period elapses, Tor uploads any rendezvous - service descriptors to the directory servers. This information is also - uploaded whenever it changes. (Default: 1 hour) + service descriptors to the directory servers. This information is also + uploaded whenever it changes. Minimum value allowed is 10 minutes and + maximum is 3.5 days. This option is only for v2 services. + (Default: 1 hour) [[HiddenServiceDirGroupReadable]] **HiddenServiceDirGroupReadable** **0**|**1**:: If this option is set to 1, allow the filesystem group to read the @@ -2405,7 +2906,7 @@ The following options are used to configure a hidden service. [[HiddenServiceNumIntroductionPoints]] **HiddenServiceNumIntroductionPoints** __NUM__:: Number of introduction points the hidden service will have. You can't - have more than 10. (Default: 3) + have more than 10 for v2 service and 20 for v3. (Default: 3) [[HiddenServiceSingleHopMode]] **HiddenServiceSingleHopMode** **0**|**1**:: **Experimental - Non Anonymous** Hidden Services on a tor instance in @@ -2417,20 +2918,20 @@ The following options are used to configure a hidden service. Single Onion Service. One-hop circuits make Single Onion servers easily locatable, but clients remain location-anonymous. However, the fact that a client is accessing a Single Onion rather than a Hidden Service may be - statistically distinguishable. - + statistically distinguishable. + + + **WARNING:** Once a hidden service directory has been used by a tor instance in HiddenServiceSingleHopMode, it can **NEVER** be used again for a hidden service. It is best practice to create a new hidden service directory, key, and address for each new Single Onion Service and Hidden Service. It is not possible to run Single Onion Services and Hidden Services from the same tor instance: they should be run on different - servers with different IP addresses. - + servers with different IP addresses. + + + HiddenServiceSingleHopMode requires HiddenServiceNonAnonymousMode to be set to 1. Since a Single Onion service is non-anonymous, you can not configure a SOCKSPort on a tor instance that is running in - **HiddenServiceSingleHopMode**. + **HiddenServiceSingleHopMode**. Can not be changed while tor is running. (Default: 0) [[HiddenServiceNonAnonymousMode]] **HiddenServiceNonAnonymousMode** **0**|**1**:: @@ -2438,103 +2939,39 @@ The following options are used to configure a hidden service. non-anonymous HiddenServiceSingleHopMode. Enables direct connections in the server-side hidden service protocol. If you are using this option, you need to disable all client-side services on your Tor instance, - including setting SOCKSPort to "0". - (Default: 0) + including setting SOCKSPort to "0". Can not be changed while tor is + running. (Default: 0) -DENIAL OF SERVICE MITIGATION OPTIONS ------------------------------------- - -The following options are useful only for a public relay. They control the -Denial of Service mitigation subsystem. - -[[DoSCircuitCreationEnabled]] **DoSCircuitCreationEnabled** **0**|**1**|**auto**:: - - Enable circuit creation DoS mitigation. If enabled, tor will cache client - IPs along with statistics in order to detect circuit DoS attacks. If an - address is positively identified, tor will activate defenses against the - address. See the DoSCircuitCreationDefenseType option for more details. - This is a client to relay detection only. "auto" means use the consensus - parameter. If not defined in the consensus, the value is 0. - (Default: auto) - -[[DoSCircuitCreationMinConnections]] **DoSCircuitCreationMinConnections** __NUM__:: - - Minimum threshold of concurrent connections before a client address can be - flagged as executing a circuit creation DoS. In other words, once a client - address reaches the circuit rate and has a minimum of NUM concurrent - connections, a detection is positive. "0" means use the consensus - parameter. If not defined in the consensus, the value is 3. - (Default: 0) - -[[DoSCircuitCreationRate]] **DoSCircuitCreationRate** __NUM__:: - - The allowed circuit creation rate per second applied per client IP - address. If this option is 0, it obeys a consensus parameter. If not - defined in the consensus, the value is 3. - (Default: 0) - -[[DoSCircuitCreationBurst]] **DoSCircuitCreationBurst** __NUM__:: - - The allowed circuit creation burst per client IP address. If the circuit - rate and the burst are reached, a client is marked as executing a circuit - creation DoS. "0" means use the consensus parameter. If not defined in the - consensus, the value is 90. - (Default: 0) - -[[DoSCircuitCreationDefenseType]] **DoSCircuitCreationDefenseType** __NUM__:: - - This is the type of defense applied to a detected client address. The - possible values are: - - 1: No defense. - 2: Refuse circuit creation for the DoSCircuitCreationDefenseTimePeriod period of time. -+ - "0" means use the consensus parameter. If not defined in the consensus, - the value is 2. - (Default: 0) - -[[DoSCircuitCreationDefenseTimePeriod]] **DoSCircuitCreationDefenseTimePeriod** __N__ **seconds**|**minutes**|**hours**:: - - The base time period in seconds that the DoS defense is activated for. The - actual value is selected randomly for each activation from N+1 to 3/2 * N. - "0" means use the consensus parameter. If not defined in the consensus, - the value is 3600 seconds (1 hour). (Default: 0) - -[[DoSConnectionEnabled]] **DoSConnectionEnabled** **0**|**1**|**auto**:: +Client Authorization +-------------------- - Enable the connection DoS mitigation. For client address only, this allows - tor to mitigate against large number of concurrent connections made by a - single IP address. "auto" means use the consensus parameter. If not - defined in the consensus, the value is 0. - (Default: auto) +(Version 3 only) -[[DoSConnectionMaxConcurrentCount]] **DoSConnectionMaxConcurrentCount** __NUM__:: +To configure client authorization on the service side, the +"<HiddenServiceDir>/authorized_clients/" directory needs to exist. Each file +in that directory should be suffixed with ".auth" (i.e. "alice.auth"; the +file name is irrelevant) and its content format MUST be: - The maximum threshold of concurrent connection from a client IP address. - Above this limit, a defense selected by DoSConnectionDefenseType is - applied. "0" means use the consensus parameter. If not defined in the - consensus, the value is 100. - (Default: 0) + <auth-type>:<key-type>:<base32-encoded-public-key> -[[DoSConnectionDefenseType]] **DoSConnectionDefenseType** __NUM__:: +The supported <auth-type> are: "descriptor". The supported <key-type> are: +"x25519". The <base32-encoded-public-key> is the base32 representation of +the raw key bytes only (32 bytes for x25519). - This is the type of defense applied to a detected client address for the - connection mitigation. The possible values are: +Each file MUST contain one line only. Any malformed file will be +ignored. Client authorization will only be enabled for the service if tor +successfully loads at least one authorization file. - 1: No defense. - 2: Immediately close new connections. -+ - "0" means use the consensus parameter. If not defined in the consensus, - the value is 2. - (Default: 0) +Note that once you've configured client authorization, anyone else with the +address won't be able to access it from this point on. If no authorization is +configured, the service will be accessible to anyone with the onion address. -[[DoSRefuseSingleHopClientRendezvous]] **DoSRefuseSingleHopClientRendezvous** **0**|**1**|**auto**:: +Revoking a client can be done by removing their ".auth" file, however the +revocation will be in effect only after the tor process gets restarted even if +a SIGHUP takes place. - Refuse establishment of rendezvous points for single hop clients. In other - words, if a client directly connects to the relay and sends an - ESTABLISH_RENDEZVOUS cell, it is silently dropped. "auto" means use the - consensus parameter. If not defined in the consensus, the value is 0. - (Default: auto) +See the Appendix G in the rend-spec-v3.txt file of +https://spec.torproject.org/[torspec] for more information. TESTING NETWORK OPTIONS ----------------------- @@ -2554,14 +2991,9 @@ The following options are used for running a testing Tor network. AssumeReachable 1 AuthDirMaxServersPerAddr 0 AuthDirMaxServersPerAuthAddr 0 - ClientBootstrapConsensusAuthorityDownloadSchedule 0, 2, - 4 (for 40 seconds), 8, 16, 32, 60 - ClientBootstrapConsensusFallbackDownloadSchedule 0, 1, - 4 (for 40 seconds), 8, 16, 32, 60 - ClientBootstrapConsensusAuthorityOnlyDownloadSchedule 0, 1, - 4 (for 40 seconds), 8, 16, 32, 60 - ClientBootstrapConsensusMaxDownloadTries 80 - ClientBootstrapConsensusAuthorityOnlyMaxDownloadTries 80 + ClientBootstrapConsensusAuthorityDownloadInitialDelay 0 + ClientBootstrapConsensusFallbackDownloadInitialDelay 0 + ClientBootstrapConsensusAuthorityOnlyDownloadInitialDelay 0 ClientDNSRejectInternalAddresses 0 ClientRejectInternalAddresses 0 CountPrivateBandwidth 1 @@ -2576,20 +3008,16 @@ The following options are used for running a testing Tor network. TestingV3AuthInitialDistDelay 20 seconds TestingAuthDirTimeToLearnReachability 0 minutes TestingEstimatedDescriptorPropagationTime 0 minutes - TestingServerDownloadSchedule 0, 0, 0, 5, 10, 15, 20, 30, 60 - TestingClientDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60 - TestingServerConsensusDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60 - TestingClientConsensusDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60 - TestingBridgeDownloadSchedule 60, 30, 30, 60 + TestingServerDownloadInitialDelay 0 + TestingClientDownloadInitialDelay 0 + TestingServerConsensusDownloadInitialDelay 0 + TestingClientConsensusDownloadInitialDelay 0 + TestingBridgeDownloadInitialDelay 10 + TestingBridgeBootstrapDownloadInitialDelay 0 TestingClientMaxIntervalWithoutRequest 5 seconds TestingDirConnectionMaxStall 30 seconds - TestingConsensusMaxDownloadTries 80 - TestingDescriptorMaxDownloadTries 80 - TestingMicrodescMaxDownloadTries 80 - TestingCertMaxDownloadTries 80 TestingEnableConnBwEvent 1 TestingEnableCellStatsEvent 1 - TestingEnableTbEmptyEvent 1 [[TestingV3AuthInitialVotingInterval]] **TestingV3AuthInitialVotingInterval** __N__ **minutes**|**hours**:: Like V3AuthVotingInterval, but for initial voting interval before the first @@ -2620,33 +3048,35 @@ The following options are used for running a testing Tor network. time. Changing this requires that **TestingTorNetwork** is set. (Default: 10 minutes) -[[TestingMinFastFlagThreshold]] **TestingMinFastFlagThreshold** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[TestingMinFastFlagThreshold]] **TestingMinFastFlagThreshold** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Minimum value for the Fast flag. Overrides the ordinary minimum taken from the consensus when TestingTorNetwork is set. (Default: 0.) -[[TestingServerDownloadSchedule]] **TestingServerDownloadSchedule** __N__,__N__,__...__:: - Schedule for when servers should download things in general. Changing this - requires that **TestingTorNetwork** is set. (Default: 0, 0, 0, 60, 60, 120, - 300, 900, 2147483647) +[[TestingServerDownloadInitialDelay]] **TestingServerDownloadInitialDelay** __N__:: + Initial delay in seconds for when servers should download things in general. Changing this + requires that **TestingTorNetwork** is set. (Default: 0) -[[TestingClientDownloadSchedule]] **TestingClientDownloadSchedule** __N__,__N__,__...__:: - Schedule for when clients should download things in general. Changing this - requires that **TestingTorNetwork** is set. (Default: 0, 0, 60, 300, 600, - 2147483647) +[[TestingClientDownloadInitialDelay]] **TestingClientDownloadInitialDelay** __N__:: + Initial delay in seconds for when clients should download things in general. Changing this + requires that **TestingTorNetwork** is set. (Default: 0) -[[TestingServerConsensusDownloadSchedule]] **TestingServerConsensusDownloadSchedule** __N__,__N__,__...__:: - Schedule for when servers should download consensuses. Changing this - requires that **TestingTorNetwork** is set. (Default: 0, 0, 60, 300, 600, - 1800, 1800, 1800, 1800, 1800, 3600, 7200) +[[TestingServerConsensusDownloadInitialDelay]] **TestingServerConsensusDownloadInitialDelay** __N__:: + Initial delay in seconds for when servers should download consensuses. Changing this + requires that **TestingTorNetwork** is set. (Default: 0) -[[TestingClientConsensusDownloadSchedule]] **TestingClientConsensusDownloadSchedule** __N__,__N__,__...__:: - Schedule for when clients should download consensuses. Changing this - requires that **TestingTorNetwork** is set. (Default: 0, 0, 60, 300, 600, - 1800, 3600, 3600, 3600, 10800, 21600, 43200) +[[TestingClientConsensusDownloadInitialDelay]] **TestingClientConsensusDownloadInitialDelay** __N__:: + Initial delay in seconds for when clients should download consensuses. Changing this + requires that **TestingTorNetwork** is set. (Default: 0) -[[TestingBridgeDownloadSchedule]] **TestingBridgeDownloadSchedule** __N__,__N__,__...__:: - Schedule for when clients should download bridge descriptors. Changing this - requires that **TestingTorNetwork** is set. (Default: 3600, 900, 900, 3600) +[[TestingBridgeDownloadInitialDelay]] **TestingBridgeDownloadInitialDelay** __N__:: + Initial delay in seconds for when clients should download each bridge descriptor when they + know that one or more of their configured bridges are running. Changing + this requires that **TestingTorNetwork** is set. (Default: 10800) + +[[TestingBridgeBootstrapDownloadInitialDelay]] **TestingBridgeBootstrapDownloadInitialDelay** __N__:: + Initial delay in seconds for when clients should download each bridge descriptor when they + have just started, or when they can not contact any of their bridges. + Changing this requires that **TestingTorNetwork** is set. (Default: 0) [[TestingClientMaxIntervalWithoutRequest]] **TestingClientMaxIntervalWithoutRequest** __N__ **seconds**|**minutes**:: When directory clients have only a few descriptors to request, they batch @@ -2659,27 +3089,11 @@ The following options are used for running a testing Tor network. Changing this requires that **TestingTorNetwork** is set. (Default: 5 minutes) -[[TestingConsensusMaxDownloadTries]] **TestingConsensusMaxDownloadTries** __NUM__:: - Try this many times to download a consensus before giving up. Changing - this requires that **TestingTorNetwork** is set. (Default: 8) - -[[TestingDescriptorMaxDownloadTries]] **TestingDescriptorMaxDownloadTries** __NUM__:: - Try this often to download a server descriptor before giving up. - Changing this requires that **TestingTorNetwork** is set. (Default: 8) - -[[TestingMicrodescMaxDownloadTries]] **TestingMicrodescMaxDownloadTries** __NUM__:: - Try this often to download a microdesc descriptor before giving up. - Changing this requires that **TestingTorNetwork** is set. (Default: 8) - -[[TestingCertMaxDownloadTries]] **TestingCertMaxDownloadTries** __NUM__:: - Try this often to download a v3 authority certificate before giving up. - Changing this requires that **TestingTorNetwork** is set. (Default: 8) - [[TestingDirAuthVoteExit]] **TestingDirAuthVoteExit** __node__,__node__,__...__:: A list of identity fingerprints, country codes, and address patterns of nodes to vote Exit for regardless of their uptime, bandwidth, or exit policy. See the **ExcludeNodes** - option for more information on how to specify nodes. + option for more information on how to specify nodes. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. See the **ExcludeNodes** option for more @@ -2688,7 +3102,7 @@ The following options are used for running a testing Tor network. [[TestingDirAuthVoteExitIsStrict]] **TestingDirAuthVoteExitIsStrict** **0**|**1** :: If True (1), a node will never receive the Exit flag unless it is specified in the **TestingDirAuthVoteExit** list, regardless of its uptime, bandwidth, - or exit policy. + or exit policy. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. @@ -2697,14 +3111,14 @@ The following options are used for running a testing Tor network. A list of identity fingerprints and country codes and address patterns of nodes to vote Guard for regardless of their uptime and bandwidth. See the **ExcludeNodes** option for more - information on how to specify nodes. + information on how to specify nodes. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. [[TestingDirAuthVoteGuardIsStrict]] **TestingDirAuthVoteGuardIsStrict** **0**|**1** :: If True (1), a node will never receive the Guard flag unless it is specified - in the **TestingDirAuthVoteGuard** list, regardless of its uptime and bandwidth. + in the **TestingDirAuthVoteGuard** list, regardless of its uptime and bandwidth. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. @@ -2713,14 +3127,14 @@ The following options are used for running a testing Tor network. A list of identity fingerprints and country codes and address patterns of nodes to vote HSDir for regardless of their uptime and DirPort. See the **ExcludeNodes** option for more - information on how to specify nodes. + information on how to specify nodes. + + In order for this option to have any effect, **TestingTorNetwork** must be set. [[TestingDirAuthVoteHSDirIsStrict]] **TestingDirAuthVoteHSDirIsStrict** **0**|**1** :: If True (1), a node will never receive the HSDir flag unless it is specified - in the **TestingDirAuthVoteHSDir** list, regardless of its uptime and DirPort. + in the **TestingDirAuthVoteHSDir** list, regardless of its uptime and DirPort. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. @@ -2735,12 +3149,7 @@ The following options are used for running a testing Tor network. events. Changing this requires that **TestingTorNetwork** is set. (Default: 0) -[[TestingEnableTbEmptyEvent]] **TestingEnableTbEmptyEvent** **0**|**1**:: - If this option is set, then Tor controllers may register for TB_EMPTY - events. Changing this requires that **TestingTorNetwork** is set. - (Default: 0) - -[[TestingMinExitFlagThreshold]] **TestingMinExitFlagThreshold** __N__ **KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[TestingMinExitFlagThreshold]] **TestingMinExitFlagThreshold** __N__ **KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Sets a lower-bound for assigning an exit flag when running as an authority on a testing network. Overrides the usual default lower bound of 4 KB. (Default: 0) @@ -2764,6 +3173,19 @@ The following options are used for running a testing Tor network. we replace it and issue a new key? (Default: 3 hours for link and auth; 1 day for signing.) +NON-PERSISTENT OPTIONS +---------------------- + +These options are not saved to the torrc file by the "SAVECONF" controller +command. Other options of this type are documented in control-spec.txt, +section 5.4. End-users should mostly ignore them. + +[[UnderscorePorts]] **\_\_ControlPort**, **\_\_DirPort**, **\_\_DNSPort**, **\_\_ExtORPort**, **\_\_NATDPort**, **\_\_ORPort**, **\_\_SocksPort**, **\_\_TransPort**:: + These underscore-prefixed options are variants of the regular Port + options. They behave the same, except they are not saved to the + torrc file by the controller's SAVECONF command. + + SIGNALS ------- @@ -2810,32 +3232,35 @@ FILES **@LOCALSTATEDIR@/lib/tor/**:: The tor process stores keys and other data here. -__DataDirectory__**/cached-status/**:: - The most recently downloaded network status document for each authority. - Each file holds one such document; the filenames are the hexadecimal - identity key fingerprints of the directory authorities. Mostly obsolete. -__DataDirectory__**/cached-certs**:: +__CacheDirectory__**/cached-certs**:: This file holds downloaded directory key certificates that are used to verify authenticity of documents generated by Tor directory authorities. -__DataDirectory__**/cached-consensus** and/or **cached-microdesc-consensus**:: +__CacheDirectory__**/cached-consensus** and/or **cached-microdesc-consensus**:: The most recent consensus network status document we've downloaded. -__DataDirectory__**/cached-descriptors** and **cached-descriptors.new**:: +__CacheDirectory__**/cached-descriptors** and **cached-descriptors.new**:: These files hold downloaded router statuses. Some routers may appear more than once; if so, the most recently published descriptor is used. Lines beginning with @-signs are annotations that contain more information about a given router. The ".new" file is an append-only journal; when it gets too large, all entries are merged into a new cached-descriptors file. -__DataDirectory__**/cached-microdescs** and **cached-microdescs.new**:: +__CacheDirectory__**/cached-extrainfo** and **cached-extrainfo.new**:: + As "cached-descriptors", but holds optionally-downloaded "extra-info" + documents. Relays use these documents to send inessential information + about statistics, bandwidth history, and network health to the + authorities. They aren't fetched by default; see the DownloadExtraInfo + option for more info. + +__CacheDirectory__**/cached-microdescs** and **cached-microdescs.new**:: These files hold downloaded microdescriptors. Lines beginning with @-signs are annotations that contain more information about a given router. The ".new" file is an append-only journal; when it gets too large, all entries are merged into a new cached-microdescs file. -__DataDirectory__**/cached-routers** and **cached-routers.new**:: +__CacheDirectory__**/cached-routers** and **cached-routers.new**:: Obsolete versions of cached-descriptors and cached-descriptors.new. When Tor can't find the newer files, it looks here instead. @@ -2843,18 +3268,27 @@ __DataDirectory__**/state**:: A set of persistent key-value mappings. These are documented in the file. These include: - The current entry guards and their status. - - The current bandwidth accounting values (unused so far; see - below). + - The current bandwidth accounting values. - When the file was last written - What version of Tor generated the state file - A short history of bandwidth usage, as produced in the server descriptors. +__DataDirectory__**/sr-state**:: + Authority only. State file used to record information about the current + status of the shared-random-value voting state. + +__CacheDirectory__**/diff-cache**:: + Directory cache only. Holds older consensuses, and diffs from older + consensuses to the most recent consensus of each type, compressed + in various ways. Each file contains a set of key-value arguments + describing its contents, followed by a single NUL byte, followed by the + main file contents. + __DataDirectory__**/bw_accounting**:: Used to track bandwidth accounting values (when the current period starts and ends; how much has been read and written so far this period). This file - is obsolete, and the data is now stored in the \'state' file as well. Only - used when bandwidth accounting is enabled. + is obsolete, and the data is now stored in the \'state' file instead. __DataDirectory__**/control_auth_cookie**:: Used for cookie authentication with the controller. Location can be @@ -2867,63 +3301,71 @@ __DataDirectory__**/lock**:: directory. If access to this file is locked, data directory is already in use by Tor. -__DataDirectory__**/keys/***:: - Only used by servers. Holds identity keys and onion keys. +__DataDirectory__**/key-pinning-journal**:: + Used by authorities. A line-based file that records mappings between + RSA1024 identity keys and Ed25519 identity keys. Authorities enforce + these mappings, so that once a relay has picked an Ed25519 key, stealing + or factoring the RSA1024 key will no longer let an attacker impersonate + the relay. -__DataDirectory__**/keys/authority_identity_key**:: +__KeyDirectory__**/authority_identity_key**:: A v3 directory authority's master identity key, used to authenticate its signing key. Tor doesn't use this while it's running. The tor-gencert program uses this. If you're running an authority, you should keep this key offline, and not actually put it here. -__DataDirectory__**/keys/authority_certificate**:: +__KeyDirectory__**/authority_certificate**:: A v3 directory authority's certificate, which authenticates the authority's current vote- and consensus-signing key using its master identity key. Only directory authorities use this file. -__DataDirectory__**/keys/authority_signing_key**:: +__KeyDirectory__**/authority_signing_key**:: A v3 directory authority's signing key, used to sign votes and consensuses. Only directory authorities use this file. Corresponds to the **authority_certificate** cert. -__DataDirectory__**/keys/legacy_certificate**:: +__KeyDirectory__**/legacy_certificate**:: As authority_certificate: used only when V3AuthUseLegacyKey is set. See documentation for V3AuthUseLegacyKey. -__DataDirectory__**/keys/legacy_signing_key**:: +__KeyDirectory__**/legacy_signing_key**:: As authority_signing_key: used only when V3AuthUseLegacyKey is set. See documentation for V3AuthUseLegacyKey. -__DataDirectory__**/keys/secret_id_key**:: +__KeyDirectory__**/secret_id_key**:: A relay's RSA1024 permanent identity key, including private and public components. Used to sign router descriptors, and to sign other keys. -__DataDirectory__**/keys/ed25519_master_id_public_key**:: +__KeyDirectory__**/ed25519_master_id_public_key**:: The public part of a relay's Ed25519 permanent identity key. -__DataDirectory__**/keys/ed25519_master_id_secret_key**:: +__KeyDirectory__**/ed25519_master_id_secret_key**:: The private part of a relay's Ed25519 permanent identity key. This key is used to sign the medium-term ed25519 signing key. This file can be kept offline, or kept encrypted. If so, Tor will not be able to generate new signing keys itself; you'll need to use tor --keygen yourself to do so. -__DataDirectory__**/keys/ed25519_signing_secret_key**:: +__KeyDirectory__**/ed25519_signing_secret_key**:: The private and public components of a relay's medium-term Ed25519 signing key. This key is authenticated by the Ed25519 master key, in turn authenticates other keys (and router descriptors). -__DataDirectory__**/keys/ed25519_signing_cert**:: +__KeyDirectory__**/ed25519_signing_cert**:: The certificate which authenticates "ed25519_signing_secret_key" as having been signed by the Ed25519 master key. -__DataDirectory__**/keys/secret_onion_key**:: +__KeyDirectory__**/secret_onion_key** and **secret_onion_key.old**:: A relay's RSA1024 short-term onion key. Used to decrypt old-style ("TAP") - circuit extension requests. + circuit extension requests. The ".old" file holds the previously + generated key, which the relay uses to handle any requests that were + made by clients that didn't have the new one. -__DataDirectory__**/keys/secret_onion_key_ntor**:: +__KeyDirectory__**/secret_onion_key_ntor** and **secret_onion_key_ntor.old**:: A relay's Curve25519 short-term onion key. Used to handle modern ("ntor") - circuit extension requests. + circuit extension requests. The ".old" file holds the previously + generated key, which the relay uses to handle any requests that were + made by clients that didn't have the new one. __DataDirectory__**/fingerprint**:: Only used by servers. Holds the fingerprint of the server's identity key. @@ -2932,15 +3374,25 @@ __DataDirectory__**/hashed-fingerprint**:: Only used by bridges. Holds the hashed fingerprint of the bridge's identity key. (That is, the hash of the hash of the identity key.) +__DataDirectory__**/approved-routers**:: + Only used by authoritative directory servers. This file lists + the status of routers by their identity fingerprint. + Each line lists a status and a fingerprint separated by + whitespace. See your **fingerprint** file in the __DataDirectory__ for an + example line. If the status is **!reject** then descriptors from the + given identity (fingerprint) are rejected by this server. If it is + **!invalid** then descriptors are accepted but marked in the directory as + not valid, that is, not recommended. + __DataDirectory__**/v3-status-votes**:: Only for v3 authoritative directory servers. This file contains status votes from all the authoritative directory servers. -__DataDirectory__**/unverified-consensus**:: +__CacheDirectory__**/unverified-consensus**:: This file contains a network consensus document that has been downloaded, but which we didn't have the right certificates to check yet. -__DataDirectory__**/unverified-microdesc-consensus**:: +__CacheDirectory__**/unverified-microdesc-consensus**:: This file contains a microdescriptor-flavored network consensus document that has been downloaded, but which we didn't have the right certificates to check yet. @@ -2978,15 +3430,29 @@ __DataDirectory__**/stats/conn-stats**:: Only used by servers. This file is used to collect approximate connection history (number of active connections over time). +__DataDirectory__**/stats/hidserv-stats**:: + Only used by servers. This file is used to collect approximate counts + of what fraction of the traffic is hidden service rendezvous traffic, and + approximately how many hidden services the relay has seen. + __DataDirectory__**/networkstatus-bridges**:: Only used by authoritative bridge directories. Contains information about bridges that have self-reported themselves to the bridge authority. +__DataDirectory__**/approved-routers**:: + Authorities only. This file is used to configure which relays are + known to be valid, invalid, and so forth. + __HiddenServiceDirectory__**/hostname**:: The <base32-encoded-fingerprint>.onion domain name for this hidden service. If the hidden service is restricted to authorized clients only, this file also contains authorization data for all clients. + + + Note that clients will ignore any extra subdomains prepended to a hidden + service hostname. So if you have "xyz.onion" as your hostname, you + can tell clients to connect to "www.xyz.onion" or "irc.xyz.onion" + for virtual-hosting purposes. __HiddenServiceDirectory__**/private_key**:: The private key for this hidden service. diff --git a/doc/torify.1.txt b/doc/torify.1.txt index 8dca901317..7e49081cfc 100644 --- a/doc/torify.1.txt +++ b/doc/torify.1.txt @@ -17,25 +17,23 @@ SYNOPSIS DESCRIPTION ----------- -**torify** is a simple wrapper that attempts to find the best underlying Tor -wrapper available on a system. It calls torsocks with a tor specific -configuration file. + +**torify** is a simple wrapper that calls torsocks with a tor-specific +configuration file. -torsocks is an improved wrapper that explicitly rejects UDP, safely resolves DNS -lookups and properly socksifies your TCP connections. + - -Please note that since both method use LD_PRELOAD, torify cannot be applied to -suid binaries. +It is provided for backward compatibility; instead you should use torsocks. WARNING ------- -When used with torsocks, torify should not leak DNS requests or UDP data. + +When used with torsocks, torify should not leak DNS requests or UDP data. + +torify can leak ICMP data. -Both will leak ICMP data. +torify will not ensure that different requests are processed on +different circuits. SEE ALSO -------- -**tor**(1), **tor-resolve**(1), **torsocks**(1) +**tor**(1), **torsocks**(1) AUTHORS ------- diff --git a/doc/torrc_format.txt b/doc/torrc_format.txt index 7a809901d9..7a4a92a663 100644 --- a/doc/torrc_format.txt +++ b/doc/torrc_format.txt @@ -18,9 +18,10 @@ does, not what it should do. ; specified in RFC5234. ; A file is interpreted as every Entry in the file, in order. - TorrcFile = *Line + TorrcFile = *Line [ UnterminatedLine ] - Line = BlankLine / Entry + Line = BlankLine LF / Entry LF + UnterminatedLine = BlankLine / Entry BlankLine = *WSP OptComment LF BlankLine =/ *WSP LF @@ -69,6 +70,12 @@ does, not what it should do. ; Anything besides NUL and LF NonLF = %x01-%x09 / %x0b - %xff + ; Note that on windows, we open our configuration files in "text" mode, + ; which causes CRLF pairs to be interpreted as LF. So, on windows: + ; LF = [ %x0d ] %x0a + ; but everywhere else, + LF = %0x0a + OCTDIG = '0' - '7' KC = Any character except an isspace() character or '#' or NUL @@ -175,7 +182,7 @@ and\ friends # Backslashes in the middle of a line are included as-is. The key of -# this one is "Too" and the value is "Many\\Backsl\ashes here" (with +# this one is "Too" and the value is "Many\\Backsl\ashes \here" (with # backslashes in that last string as-is) Too \ Many\\\ @@ -185,7 +192,7 @@ here # And here's the really yucky part. If a comment appears in a multi-line # entry, the entry is still able to continue on the next line, as in the # following, where the key is "This" and the value is -# "entry and some are silly" +# "entry and some are silly" This entry \ # has comments \ and some \ |