diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/HACKING/CodingStandards.md | 154 | ||||
| -rw-r--r-- | doc/HACKING/CodingStandardsRust.md | 469 | ||||
| -rw-r--r-- | doc/HACKING/Fuzzing.md | 123 | ||||
| -rw-r--r-- | doc/HACKING/GettingStarted.md | 5 | ||||
| -rw-r--r-- | doc/HACKING/GettingStartedRust.md | 162 | ||||
| -rw-r--r-- | doc/HACKING/HelpfulTools.md | 90 | ||||
| -rw-r--r-- | doc/HACKING/HowToReview.md | 3 | ||||
| -rw-r--r-- | doc/HACKING/ReleasingTor.md | 118 | ||||
| -rw-r--r-- | doc/HACKING/Tracing.md | 91 | ||||
| -rw-r--r-- | doc/HACKING/WritingTests.md | 8 | ||||
| -rw-r--r-- | doc/include.am | 12 | ||||
| -rw-r--r-- | doc/tor.1.txt | 799 | ||||
| -rw-r--r-- | doc/torify.1.txt | 20 | ||||
| -rw-r--r-- | doc/torrc_format.txt | 15 |
14 files changed, 1634 insertions, 435 deletions
diff --git a/doc/HACKING/CodingStandards.md b/doc/HACKING/CodingStandards.md index f1c65850a4..dd21d6fd2c 100644 --- a/doc/HACKING/CodingStandards.md +++ b/doc/HACKING/CodingStandards.md @@ -4,9 +4,10 @@ Coding conventions for Tor tl;dr: - Run configure with `--enable-fatal-warnings` - - Run `make check-spaces` to catch whitespace errors - Document your functions - Write unit tests + - Run `make check` before submitting a patch + - Run `make distcheck` if you have made changes to build system components - Add a file in `changes` for your branch. Patch checklist @@ -22,13 +23,25 @@ preference) Did you remember... - To build your code while configured with `--enable-fatal-warnings`? - - To run `make check-spaces` on your code? - To run `make check-docs` to see whether all new options are on the manpage? - To write unit tests, as possible? + - To run `make test-full` to test against all unit and integration tests (or + `make test-full-online` if you have a working connection to the internet)? + - To test that the distribution will actually work via `make distcheck`? - To base your code on the appropriate branch? - To include a file in the `changes` directory as appropriate? +If you are submitting a major patch or new feature, or want to in the future... + + - Set up Chutney and Stem, see HACKING/WritingTests.md + - Run `make test-full` to test against all unit and integration tests. + +If you have changed build system components: + - Please run `make distcheck` + - For example, if you have changed Makefiles, autoconf files, or anything + else that affects the build system. + How we use Git branches ======================= @@ -49,8 +62,17 @@ before it gets merged into maint, but that's rare. If you're working on a bugfix for a bug that occurs in a particular version, base your bugfix branch on the "maint" branch for the first supported series -that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) If -you're working on a new feature, base it on the master branch. +that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) + +If you're working on a new feature, base it on the master branch. If you're +working on a new feature and it will take a while to implement and/or you'd +like to avoid the possibility of unrelated bugs in Tor while you're +implementing your feature, consider branching off of the latest maint- branch. +_Never_ branch off a relase- branch. Don't branch off a tag either: they come +from release branches. Doing so will likely produce a nightmare of merge +conflicts in the ChangeLog when it comes time to merge your branch into Tor. +Best advice: don't try to keep an independent branch forked for more than 6 +months and expect it to merge cleanly. Try to merge pieces early and often. How we log changes @@ -74,17 +96,34 @@ you can use `git describe --contains <sha1 of commit>`. If at all possible, try to create this file in the same commit where you are making the change. Please give it a distinctive name that no other branch will use for the lifetime of your change. To verify the format of the changes file, -you can use `make check-changes`. +you can use `make check-changes`. This is run automatically as part of +`make check` -- if it fails, we must fix it before we release. These +checks are implemented in `scripts/maint/lintChanges.py`. + +Changes file style guide: + * Changes files begin with " o Header (subheading):". The header + should usually be "Minor/Major bugfixes/features". The subheading is a + particular area within Tor. See the ChangeLog for examples. + + * Make everything terse. + + * Write from the user's point of view: describe the user-visible changes + right away. + + * Mention configuration options by name. If they're rare or unusual, + remind people what they're for. + + * Describe changes in the present tense and in the imperative: not past. + + * Every bugfix should have a sentence of the form "Fixes bug 1234; bugfix + on 0.1.2.3-alpha", describing what bug was fixed and where it came from. + + * "Relays", not "servers", "nodes", or "Tor relays". When we go to make a release, we will concatenate all the entries in changes to make a draft changelog, and clear the directory. We'll then edit the draft changelog into a nice readable format. -To make sure that stuff is in the right format, we use -scripts/maint/lintChanges.py to check the changes files for -(superficial) validity. You can run this script on your own changes -files! - What needs a changes file? * A not-exhaustive list: Anything that might change user-visible @@ -93,6 +132,10 @@ What needs a changes file? rewrites. Anything about which somebody might plausibly wonder "when did that happen, and/or why did we do that" 6 months down the line. +What does not need a changes file? + + * Bugfixes for code that hasn't shipped in any released version of Tor + Why use changes files instead of Git commit messages? * Git commit messages are written for developers, not users, and they @@ -149,6 +192,79 @@ old C functions. Use `strlcat`, `strlcpy`, or `tor_snprintf/tor_asprintf` inste We don't call `memcmp()` directly. Use `fast_memeq()`, `fast_memneq()`, `tor_memeq()`, or `tor_memneq()` for most purposes. +Also see a longer list of functions to avoid in: +https://people.torproject.org/~nickm/tor-auto/internal/this-not-that.html + +Floating point math is hard +--------------------------- + +Floating point arithmetic as typically implemented by computers is +very counterintuitive. Failure to adequately analyze floating point +usage can result in surprising behavior and even security +vulnerabilities! + +General advice: + + - Don't use floating point. + - If you must use floating point, document how the limits of + floating point precision and calculation accuracy affect function + outputs. + - Try to do as much as possible of your calculations using integers + (possibly acting as fixed-point numbers) and convert to floating + point for display. + - If you must send floating point numbers on the wire, serialize + them in a platform-independent way. Tor avoids exchanging + floating-point values, but when it does, it uses ASCII numerals, + with a decimal point ("."). + - Binary fractions behave very differently from decimal fractions. + Make sure you understand how these differences affect your + calculations. + - Every floating point arithmetic operation is an opportunity to + lose precision, overflow, underflow, or otherwise produce + undesired results. Addition and subtraction tend to be worse + than multiplication and division (due to things like catastrophic + cancellation). Try to arrange your calculations to minimize such + effects. + - Changing the order of operations changes the results of many + floating-point calculations. Be careful when you simplify + calculations! If the order is significant, document it using a + code comment. + - Comparing most floating point values for equality is unreliable. + Avoid using `==`, instead, use `>=` or `<=`. If you use an + epsilon value, make sure it's appropriate for the ranges in + question. + - Different environments (including compiler flags and per-thread + state on a single platform!) can get different results from the + same floating point calculations. This means you can't use + floats in anything that needs to be deterministic, like consensus + generation. This also makes reliable unit tests of + floating-point outputs hard to write. + +For additional useful advice (and a little bit of background), see +[What Every Programmer Should Know About Floating-Point +Arithmetic](http://floating-point-gui.de/). + +A list of notable (and surprising) facts about floating point +arithmetic is at [Floating-point +complexities](https://randomascii.wordpress.com/2012/04/05/floating-point-complexities/). +Most of that [series of posts on floating +point](https://randomascii.wordpress.com/category/floating-point/) is +helpful. + +For more detailed (and math-intensive) background, see [What Every +Computer Scientist Should Know About Floating-Point +Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html). + +Other C conventions +------------------- + +The `a ? b : c` trinary operator only goes inside other expressions; +don't use it as a replacement for if. (You can ignore this inside macro +definitions when necessary.) + +Assignment operators shouldn't nest inside other expressions. (You can +ignore this inside macro definitions when necessary.) + Functions not to write ---------------------- @@ -210,7 +326,25 @@ end-users that they aren't expected to understand the message (perhaps with a string like "internal error"). Option (A) is to be preferred to option (B). +Assertions In Tor +----------------- + +Assertions should be used for bug-detection only. Don't use assertions to +detect bad user inputs, network errors, resource exhaustion, or similar +issues. + +Tor is always built with assertions enabled, so try to only use +`tor_assert()` for cases where you are absolutely sure that crashing is the +least bad option. Many bugs have been caused by use of `tor_assert()` when +another kind of check would have been safer. + +If you're writing an assertion to test for a bug that you _can_ recover from, +use `tor_assert_nonfatal()` in place of `tor_assert()`. If you'd like to +write a conditional that incorporates a nonfatal assertion, use the `BUG()` +macro, as in: + if (BUG(ptr == NULL)) + return -1; Doxygen comment conventions --------------------------- diff --git a/doc/HACKING/CodingStandardsRust.md b/doc/HACKING/CodingStandardsRust.md new file mode 100644 index 0000000000..d0b17c1604 --- /dev/null +++ b/doc/HACKING/CodingStandardsRust.md @@ -0,0 +1,469 @@ + + Rust Coding Standards +======================= + +You MUST follow the standards laid out in `.../doc/HACKING/CodingStandards.md`, +where applicable. + + Module/Crate Declarations +--------------------------- + +Each Tor C module which is being rewritten MUST be in its own crate. +See the structure of `.../src/rust` for examples. + +In your crate, you MUST use `lib.rs` ONLY for pulling in external +crates (e.g. `extern crate libc;`) and exporting public objects from +other Rust modules (e.g. `pub use mymodule::foo;`). For example, if +you create a crate in `.../src/rust/yourcrate`, your Rust code should +live in `.../src/rust/yourcrate/yourcode.rs` and the public interface +to it should be exported in `.../src/rust/yourcrate/lib.rs`. + +If your code is to be called from Tor C code, you MUST define a safe +`ffi.rs`. See the "Safety" section further down for more details. + +For example, in a hypothetical `tor_addition` Rust module: + +In `.../src/rust/tor_addition/addition.rs`: + + pub fn get_sum(a: i32, b: i32) -> i32 { + a + b + } + +In `.../src/rust/tor_addition/lib.rs`: + + pub use addition::*; + +In `.../src/rust/tor_addition/ffi.rs`: + + #[no_mangle] + pub extern "C" fn tor_get_sum(a: c_int, b: c_int) -> c_int { + get_sum(a, b) + } + +If your Rust code must call out to parts of Tor's C code, you must +declare the functions you are calling in the `external` crate, located +at `.../src/rust/external`. + +<!-- XXX get better examples of how to declare these externs, when/how they --> +<!-- XXX are unsafe, what they are expected to do —isis --> + +Modules should strive to be below 500 lines (tests excluded). Single +responsibility and limited dependencies should be a guiding standard. + +If you have any external modules as dependencies (e.g. `extern crate +libc;`), you MUST declare them in your crate's `lib.rs` and NOT in any +other module. + + Dependencies +-------------- + +In general, we use modules from only the Rust standard library +whenever possible. We will review including external crates on a +case-by-case basis. + + Documentation +--------------- + +You MUST include `#[deny(missing_docs)]` in your crate. + +For function/method comments, you SHOULD include a one-sentence, "first person" +description of function behaviour (see requirements for documentation as +described in `.../src/HACKING/CodingStandards.md`), then an `# Inputs` section +for inputs or initialisation values, a `# Returns` section for return +values/types, a `# Warning` section containing warnings for unsafe behaviours or +panics that could happen. For publicly accessible +types/constants/objects/functions/methods, you SHOULD also include an +`# Examples` section with runnable doctests. + +You MUST document your module with _module docstring_ comments, +i.e. `//!` at the beginning of each line. + + Style +------- + +You SHOULD consider breaking up large literal numbers with `_` when it makes it +more human readable to do so, e.g. `let x: u64 = 100_000_000_000`. + + Testing +--------- + +All code MUST be unittested and integration tested. + +Public functions/objects exported from a crate SHOULD include doctests +describing how the function/object is expected to be used. + +Integration tests SHOULD go into a `tests/` directory inside your +crate. Unittests SHOULD go into their own module inside the module +they are testing, e.g. in `.../src/rust/tor_addition/addition.rs` you +should put: + + #[cfg(test)] + mod test { + use super::*; + + #[test] + fn addition_with_zero() { + let sum: i32 = get_sum(5i32, 0i32); + assert_eq!(sum, 5); + } + } + + Benchmarking +-------------- + +The external `test` crate can be used for most benchmarking. However, using +this crate requires nightly Rust. Since we may want to switch to a more +stable Rust compiler eventually, we shouldn't do things which will automatically +break builds for stable compilers. Therefore, you MUST feature-gate your +benchmarks in the following manner. + +If you wish to benchmark some of your Rust code, you MUST put the +following in the `[features]` section of your crate's `Cargo.toml`: + + [features] + bench = [] + +Next, in your crate's `lib.rs` you MUST put: + + #[cfg(all(test, feature = "bench"))] + extern crate test; + +This ensures that the external crate `test`, which contains utilities +for basic benchmarks, is only used when running benchmarks via `cargo +bench --features bench`. + +Finally, to write your benchmark code, in +`.../src/rust/tor_addition/addition.rs` you SHOULD put: + + #[cfg(all(test, features = "bench"))] + mod bench { + use test::Bencher; + use super::*; + + #[bench] + fn addition_small_integers(b: &mut Bencher) { + b.iter(| | get_sum(5i32, 0i32)); + } + } + + Fuzzing +--------- + +If you wish to fuzz parts of your code, please see the +[`cargo fuzz`](https://github.com/rust-fuzz/cargo-fuzz) crate, which uses +[libfuzzer-sys](https://github.com/rust-fuzz/libfuzzer-sys). + + Whitespace & Formatting +------------------------- + +You MUST run `rustfmt` (https://github.com/rust-lang-nursery/rustfmt) +on your code before your code will be merged. You can install rustfmt +by doing `cargo install rustfmt-nightly` and then run it with `cargo +fmt`. + + Safety +-------- + +You SHOULD read [the nomicon](https://doc.rust-lang.org/nomicon/) before writing +Rust FFI code. It is *highly advised* that you read and write normal Rust code +before attempting to write FFI or any other unsafe code. + +Here are some additional bits of advice and rules: + +0. Any behaviours which Rust considers to be undefined are forbidden + + From https://doc.rust-lang.org/reference/behavior-considered-undefined.html: + + > Behavior considered undefined + > + > The following is a list of behavior which is forbidden in all Rust code, + > including within unsafe blocks and unsafe functions. Type checking provides the + > guarantee that these issues are never caused by safe code. + > + > * Data races + > * Dereferencing a null/dangling raw pointer + > * Reads of [undef](http://llvm.org/docs/LangRef.html#undefined-values) + > (uninitialized) memory + > * Breaking the + > [pointer aliasing rules](http://llvm.org/docs/LangRef.html#pointer-aliasing-rules) + > with raw pointers (a subset of the rules used by C) + > * `&mut T` and `&T` follow LLVM’s scoped noalias model, except if the `&T` + > contains an `UnsafeCell<U>`. Unsafe code must not violate these aliasing + > guarantees. + > * Mutating non-mutable data (that is, data reached through a shared + > reference or data owned by a `let` binding), unless that data is + > contained within an `UnsafeCell<U>`. + > * Invoking undefined behavior via compiler intrinsics: + > - Indexing outside of the bounds of an object with + > `std::ptr::offset` (`offset` intrinsic), with the exception of + > one byte past the end which is permitted. + > - Using `std::ptr::copy_nonoverlapping_memory` (`memcpy32`/`memcpy64` + > intrinsics) on overlapping buffers + > * Invalid values in primitive types, even in private fields/locals: + > - Dangling/null references or boxes + > - A value other than `false` (0) or `true` (1) in a `bool` + > - A discriminant in an `enum` not included in the type definition + > - A value in a `char` which is a surrogate or above `char::MAX` + > - Non-UTF-8 byte sequences in a `str` + > * Unwinding into Rust from foreign code or unwinding from Rust into foreign + > code. Rust's failure system is not compatible with exception handling in other + > languages. Unwinding must be caught and handled at FFI boundaries. + +1. `unwrap()` + + If you call `unwrap()`, anywhere, even in a test, you MUST include + an inline comment stating how the unwrap will either 1) never fail, + or 2) should fail (i.e. in a unittest). + + You SHOULD NOT use `unwrap()` anywhere in which it is possible to handle the + potential error with either `expect()` or the eel operator, `?`. + For example, consider a function which parses a string into an integer: + + fn parse_port_number(config_string: &str) -> u16 { + u16::from_str_radix(config_string, 10).unwrap() + } + + There are numerous ways this can fail, and the `unwrap()` will cause the + whole program to byte the dust! Instead, either you SHOULD use `expect()` + (or another equivalent function which will return an `Option` or a `Result`) + and change the return type to be compatible: + + fn parse_port_number(config_string: &str) -> Option<u16> { + u16::from_str_radix(config_string, 10).expect("Couldn't parse port into a u16") + } + + or you SHOULD use `or()` (or another similar method): + + fn parse_port_number(config_string: &str) -> Option<u16> { + u16::from_str_radix(config_string, 10).or(Err("Couldn't parse port into a u16") + } + + Using methods like `or()` can be particularly handy when you must do + something afterwards with the data, for example, if we wanted to guarantee + that the port is high. Combining these methods with the eel operator (`?`) + makes this even easier: + + fn parse_port_number(config_string: &str) -> Result<u16, Err> { + let port = u16::from_str_radix(config_string, 10).or(Err("Couldn't parse port into a u16"))?; + + if port > 1024 { + return Ok(port); + } else { + return Err("Low ports not allowed"); + } + } + +2. `unsafe` + + If you use `unsafe`, you MUST describe a contract in your + documentation which describes how and when the unsafe code may + fail, and what expectations are made w.r.t. the interfaces to + unsafe code. This is also REQUIRED for major pieces of FFI between + C and Rust. + + When creating an FFI in Rust for C code to call, it is NOT REQUIRED + to declare the entire function `unsafe`. For example, rather than doing: + + #[no_mangle] + pub unsafe extern "C" fn increment_and_combine_numbers(mut numbers: [u8; 4]) -> u32 { + for number in &mut numbers { + *number += 1; + } + std::mem::transmute::<[u8; 4], u32>(numbers) + } + + You SHOULD instead do: + + #[no_mangle] + pub extern "C" fn increment_and_combine_numbers(mut numbers: [u8; 4]) -> u32 { + for index in 0..numbers.len() { + numbers[index] += 1; + } + unsafe { + std::mem::transmute::<[u8; 4], u32>(numbers) + } + } + +3. Pass only integer types and bytes over the boundary + + The only non-integer type which may cross the FFI boundary is + bytes, e.g. `&[u8]`. This SHOULD be done on the Rust side by + passing a pointer (`*mut libc::c_char`) and a length + (`libc::size_t`). + + One might be tempted to do this via doing + `CString::new("blah").unwrap().into_raw()`. This has several problems: + + a) If you do `CString::new("bl\x00ah")` then the unwrap() will fail + due to the additional NULL terminator, causing a dangling + pointer to be returned (as well as a potential use-after-free). + + b) Returning the raw pointer will cause the CString to run its deallocator, + which causes any C code which tries to access the contents to dereference a + NULL pointer. + + c) If we were to do `as_raw()` this would result in a potential double-free + since the Rust deallocator would run and possibly Tor's deallocator. + + d) Calling `into_raw()` without later using the same pointer in Rust to call + `from_raw()` and then deallocate in Rust can result in a + [memory leak](https://doc.rust-lang.org/std/ffi/struct.CString.html#method.into_raw). + + [It was determined](https://github.com/rust-lang/rust/pull/41074) that this + is safe to do if you use the same allocator in C and Rust and also specify + the memory alignment for CString (except that there is no way to specify + the alignment for CString). It is believed that the alignment is always 1, + which would mean it's safe to dealloc the resulting `*mut c_char` in Tor's + C code. However, the Rust developers are not willing to guarantee the + stability of, or a contract for, this behaviour, citing concerns that this + is potentially extremely and subtly unsafe. + +4. Perform an allocation on the other side of the boundary + + After crossing the boundary, the other side MUST perform an + allocation to copy the data and is therefore responsible for + freeing that memory later. + +5. No touching other language's enums + + Rust enums should never be touched from C (nor can they be safely + `#[repr(C)]`) nor vice versa: + + > "The chosen size is the default enum size for the target platform's C + > ABI. Note that enum representation in C is implementation defined, so this is + > really a "best guess". In particular, this may be incorrect when the C code + > of interest is compiled with certain flags." + + (from https://gankro.github.io/nomicon/other-reprs.html) + +6. Type safety + + Wherever possible and sensical, you SHOULD create new types in a + manner which prevents type confusion or misuse. For example, + rather than using an untyped mapping between strings and integers + like so: + + use std::collections::HashMap; + + pub fn get_elements_with_over_9000_points(map: &HashMap<String, usize>) -> Vec<String> { + ... + } + + It would be safer to define a new type, such that some other usage + of `HashMap<String, usize>` cannot be confused for this type: + + pub struct DragonBallZPowers(pub HashMap<String, usize>); + + impl DragonBallZPowers { + pub fn over_nine_thousand<'a>(&'a self) -> Vec<&'a String> { + let mut powerful_enough: Vec<&'a String> = Vec::with_capacity(5); + + for (character, power) in &self.0 { + if *power > 9000 { + powerful_enough.push(character); + } + } + powerful_enough + } + } + + Note the following code, which uses Rust's type aliasing, is valid + but it does NOT meet the desired type safety goals: + + pub type Power = usize; + + pub fn over_nine_thousand(power: &Power) -> bool { + if *power > 9000 { + return true; + } + false + } + + // We can still do the following: + let his_power: usize = 9001; + over_nine_thousand(&his_power); + +7. Unsafe mucking around with lifetimes + + Because lifetimes are technically, in type theory terms, a kind, i.e. a + family of types, individual lifetimes can be treated as types. For example, + one can arbitrarily extend and shorten lifetime using `std::mem::transmute`: + + struct R<'a>(&'a i32); + + unsafe fn extend_lifetime<'b>(r: R<'b>) -> R<'static> { + std::mem::transmute::<R<'b>, R<'static>>(r) + } + + unsafe fn shorten_invariant_lifetime<'b, 'c>(r: &'b mut R<'static>) -> &'b mut R<'c> { + std::mem::transmute::<&'b mut R<'static>, &'b mut R<'c>>(r) + } + + Calling `extend_lifetime()` would cause an `R` passed into it to live forever + for the life of the program (the `'static` lifetime). Similarly, + `shorten_invariant_lifetime()` could be used to take something meant to live + forever, and cause it to disappear! This is incredibly unsafe. If you're + going to be mucking around with lifetimes like this, first, you better have + an extremely good reason, and second, you may as be honest and explicit about + it, and for ferris' sake just use a raw pointer. + + In short, just because lifetimes can be treated like types doesn't mean you + should do it. + +8. Doing excessively unsafe things when there's a safer alternative + + Similarly to #7, often there are excessively unsafe ways to do a task and a + simpler, safer way. You MUST choose the safer option where possible. + + For example, `std::mem::transmute` can be abused in ways where casting with + `as` would be both simpler and safer: + + // Don't do this + let ptr = &0; + let ptr_num_transmute = unsafe { std::mem::transmute::<&i32, usize>(ptr)}; + + // Use an `as` cast instead + let ptr_num_cast = ptr as *const i32 as usize; + + In fact, using `std::mem::transmute` for *any* reason is a code smell and as + such SHOULD be avoided. + +9. Casting integers with `as` + + This is generally fine to do, but it has some behaviours which you should be + aware of. Casting down chops off the high bits, e.g.: + + let x: u32 = 4294967295; + println!("{}", x as u16); // prints 65535 + + Some cases which you MUST NOT do include: + + * Casting an `u128` down to an `f32` or vice versa (e.g. + `u128::MAX as f32` but this isn't only a problem with overflowing + as it is also undefined behaviour for `42.0f32 as u128`), + + * Casting between integers and floats when the thing being cast + cannot fit into the type it is being casted into, e.g.: + + println!("{}", 42949.0f32 as u8); // prints 197 in debug mode and 0 in release + println!("{}", 1.04E+17 as u8); // prints 0 in both modes + println!("{}", (0.0/0.0) as i64); // prints whatever the heck LLVM wants + + Because this behaviour is undefined, it can even produce segfaults in + safe Rust code. For example, the following program built in release + mode segfaults: + + #[inline(never)] + pub fn trigger_ub(sl: &[u8; 666]) -> &[u8] { + // Note that the float is out of the range of `usize`, invoking UB when casting. + let idx = 1e99999f64 as usize; + &sl[idx..] // The bound check is elided due to `idx` being of an undefined value. + } + + fn main() { + println!("{}", trigger_ub(&[1; 666])[999999]); // ~ out of bound + } + + And in debug mode panics with: + + thread 'main' panicked at 'slice index starts at 140721821254240 but ends at 666', /checkout/src/libcore/slice/mod.rs:754:4 diff --git a/doc/HACKING/Fuzzing.md b/doc/HACKING/Fuzzing.md new file mode 100644 index 0000000000..2039d6a4c0 --- /dev/null +++ b/doc/HACKING/Fuzzing.md @@ -0,0 +1,123 @@ += Fuzzing Tor + +== The simple version (no fuzzing, only tests) + +Check out fuzzing-corpora, and set TOR_FUZZ_CORPORA to point to the place +where you checked it out. + +To run the fuzzing test cases in a deterministic fashion, use: + make test-fuzz-corpora + +This won't actually fuzz Tor! It will just run all the fuzz binaries +on our existing set of testcases for the fuzzer. + + +== Different kinds of fuzzing + +Right now we support three different kinds of fuzzer. + +First, there's American Fuzzy Lop (AFL), a fuzzer that works by forking +a target binary and passing it lots of different inputs on stdin. It's the +trickiest one to set up, so I'll be describing it more below. + +Second, there's libFuzzer, a llvm-based fuzzer that you link in as a library, +and it runs a target function over and over. To use this one, you'll need to +have a reasonably recent clang and libfuzzer installed. At that point, you +just build with --enable-expensive-hardening and --enable-libfuzzer. That +will produce a set of binaries in src/test/fuzz/lf-fuzz-* . These programs +take as input a series of directories full of fuzzing examples. For more +information on libfuzzer, see http://llvm.org/docs/LibFuzzer.html + +Third, there's Google's OSS-Fuzz infrastructure, which expects to get all of +its. For more on this, see https://github.com/google/oss-fuzz and the +projects/tor subdirectory. You'll need to mess around with Docker a bit to +test this one out; it's meant to run on Google's infrastructure. + +In all cases, you'll need some starting examples to give the fuzzer when it +starts out. There's a set in the "fuzzing-corpora" git repository. Try +setting TOR_FUZZ_CORPORA to point to a checkout of that repository + +== Writing Tor fuzzers + +A tor fuzzing harness should have: +* a fuzz_init() function to set up any necessary global state. +* a fuzz_main() function to receive input and pass it to a parser. +* a fuzz_cleanup() function to clear global state. + +Most fuzzing frameworks will produce many invalid inputs - a tor fuzzing +harness should rejecting invalid inputs without crashing or behaving badly. + +But the fuzzing harness should crash if tor fails an assertion, triggers a +bug, or accesses memory it shouldn't. This helps fuzzing frameworks detect +"interesting" cases. + + +== Guided Fuzzing with AFL + +There is no HTTPS, hash, or signature for American Fuzzy Lop's source code, so +its integrity can't be verified. That said, you really shouldn't fuzz on a +machine you care about, anyway. + +To Build: + Get AFL from http://lcamtuf.coredump.cx/afl/ and unpack it + cd afl + make + cd ../tor + PATH=$PATH:../afl/ CC="../afl/afl-gcc" ./configure --enable-expensive-hardening + AFL_HARDEN=1 make clean fuzzers + +To Find The ASAN Memory Limit: (64-bit only) + +On 64-bit platforms, afl needs to know how much memory ASAN uses, +because ASAN tends to allocate a ridiculous amount of virtual memory, +and then not actually use it. + +Read afl/docs/notes_for_asan.txt for more details. + + Download recidivm from http://jwilk.net/software/recidivm + Download the signature + Check the signature + tar xvzf recidivm*.tar.gz + cd recidivm* + make + /path/to/recidivm -v src/test/fuzz/fuzz-http + Use the final "ok" figure as the input to -m when calling afl-fuzz + (Normally, recidivm would output a figure automatically, but in some cases, + the fuzzing harness will hang when the memory limit is too small.) + +You could also just say "none" instead of the memory limit below, if you +don't care about memory limits. + + +To Run: + mkdir -p src/test/fuzz/fuzz_http_findings + ../afl/afl-fuzz -i ${TOR_FUZZ_CORPORA}/http -o src/test/fuzz/fuzz_http_findings -m <asan-memory-limit> -- src/test/fuzz/fuzz-http + + +AFL has a multi-core mode, check the documentation for details. +You might find the included fuzz-multi.sh script useful for this. + +macOS (OS X) requires slightly more preparation, including: +* using afl-clang (or afl-clang-fast from the llvm directory) +* disabling external crash reporting (AFL will guide you through this step) + +== Triaging Issues + +Crashes are usually interesting, particularly if using AFL_HARDEN=1 and --enable-expensive-hardening. Sometimes crashes are due to bugs in the harness code. + +Hangs might be interesting, but they might also be spurious machine slowdowns. +Check if a hang is reproducible before reporting it. Sometimes, processing +valid inputs may take a second or so, particularly with the fuzzer and +sanitizers enabled. + +To see what fuzz-http is doing with a test case, call it like this: + src/test/fuzz/fuzz-http --debug < /path/to/test.case + +(Logging is disabled while fuzzing to increase fuzzing speed.) + +== Reporting Issues + +Please report any issues discovered using the process in Tor's security issue +policy: + +https://trac.torproject.org/projects/tor/wiki/org/meetings/2016SummerDevMeeting/Notes/SecurityIssuePolicy diff --git a/doc/HACKING/GettingStarted.md b/doc/HACKING/GettingStarted.md index 0295adc1ff..0c42404634 100644 --- a/doc/HACKING/GettingStarted.md +++ b/doc/HACKING/GettingStarted.md @@ -11,8 +11,9 @@ whole Tor ecosystem.) If you are looking for a more bare-bones, less user-friendly information -dump of important information, you might like reading doc/HACKING -instead. You should probably read it before you write your first patch. +dump of important information, you might like reading the "torguts" +documents linked to below. You should probably read it before you write +your first patch. Required background diff --git a/doc/HACKING/GettingStartedRust.md b/doc/HACKING/GettingStartedRust.md new file mode 100644 index 0000000000..920596cadf --- /dev/null +++ b/doc/HACKING/GettingStartedRust.md @@ -0,0 +1,162 @@ + + Hacking on Rust in Tor +======================== + + Getting Started +----------------- + +Please read or review our documentation on Rust coding standards +(`.../doc/HACKING/CodingStandardsRust.md`) before doing anything. + +Please also read +[the Rust Code of Conduct](https://www.rust-lang.org/en-US/conduct.html). We aim +to follow the good example set by the Rust community and be excellent to one +another. Let's be careful with each other, so we can be memory-safe together! + +Next, please contact us before rewriting anything! Rust in Tor is still an +experiment. It is an experiment that we very much want to see succeed, so we're +going slowly and carefully. For the moment, it's also a completely +volunteer-driven effort: while many, if not most, of us are paid to work on Tor, +we are not yet funded to write Rust code for Tor. Please be patient with the +other people who are working on getting more Rust code into Tor, because they +are graciously donating their free time to contribute to this effort. + + Resources for learning Rust +----------------------------- + +**Beginning resources** + +The primary resource for learning Rust is +[The Book](https://doc.rust-lang.org/book/). If you'd like to start writing +Rust immediately, without waiting for anything to install, there is +[an interactive browser-based playground](https://play.rust-lang.org/). + +**Advanced resources** + +If you're interested in playing with various Rust compilers and viewing a very +nicely displayed output of the generated assembly, there is +[the Godbolt compiler explorer](https://rust.godbolt.org/) + +For learning how to write unsafe Rust, read +[The Rustonomicon](https://doc.rust-lang.org/nomicon/). + +For learning everything you ever wanted to know about Rust macros, there is +[The Little Book of Rust Macros](https://danielkeep.github.io/tlborm/book/index.html). + +For learning more about FFI and Rust, see Jake Goulding's +[Rust FFI Omnibus](http://jakegoulding.com/rust-ffi-omnibus/). + + Compiling Tor with Rust enabled +--------------------------------- + +You will need to run the `configure` script with the `--enable-rust` flag to +explicitly build with Rust. Additionally, you will need to specify where to +fetch Rust dependencies, as we allow for either fetching dependencies from Cargo +or specifying a local directory. + +**Fetch dependencies from Cargo** + + ./configure --enable-rust --enable-cargo-online-mode + +**Using a local dependency cache** + +**NOTE**: local dependency caches which were not *originally* created via + `--enable-cargo-online-mode` are broken. See https://bugs.torproject.org/22907 + +To specify a local directory: + + RUST_DEPENDENCIES='path_to_dependencies_directory' ./configure --enable-rust + +(Note that RUST_DEPENDENCIES must be the full path to the directory; it cannot +be relative.) + +You'll need the following Rust dependencies (as of this writing): + + libc==0.2.22 + +To get them, do: + + mkdir path_to_dependencies_directory + cd path_to_dependencies_directory + git clone https://github.com/rust-lang/libc + cd libc + git checkout 0.2.22 + cargo package + cd .. + ln -s libc/target/package/libc-0.2.22 libc-0.2.22 + + + Identifying which modules to rewrite +====================================== + +The places in the Tor codebase that are good candidates for porting to Rust are: + +1. loosely coupled to other Tor submodules, +2. have high test coverage, and +3. would benefit from being implemented in a memory safe language. + +Help in either identifying places such as this, or working to improve existing +areas of the C codebase by adding regression tests and simplifying dependencies, +would be really helpful. + +Furthermore, as submodules in C are implemented in Rust, this is a good +opportunity to refactor, add more tests, and split modules into smaller areas of +responsibility. + +A good first step is to build a module-level callgraph to understand how +interconnected your target module is. + + git clone https://git.torproject.org/user/nickm/calltool.git + cd tor + CFLAGS=0 ./configure + ../calltool/src/main.py module_callgraph + +The output will tell you each module name, along with a set of every module that +the module calls. Modules which call fewer other modules are better targets. + + Writing your Rust module +========================== + +Strive to change the C API as little as possible. + +We are currently targetting Rust stable. (See CodingStandardsRust.md for more +details.) + +It is on our TODO list to try to cultivate good standing with various distro +maintainers of `rustc` and `cargo`, in order to ensure that whatever version we +solidify on is readily available. + + Adding your Rust module to Tor's build system +----------------------------------------------- + +0. Your translation of the C module should live in its own crate(s) + in the `.../tor/src/rust/` directory. +1. Add your crate to `.../tor/src/rust/Cargo.toml`, in the + `[workspace.members]` section. +2. Append your crate's static library to the `rust_ldadd` definition + (underneath `if USE_RUST`) in `.../tor/Makefile.am`. + + How to test your Rust code +---------------------------- + +Everything should be tested full stop. Even non-public functionality. + +Be sure to edit `.../tor/src/test/test_rust.sh` to add the name of your crate to +the `crates` variable! This will ensure that `cargo test` is run on your crate. + +Configure Tor's build system to build with Rust enabled: + + ./configure --enable-fatal-warnings --enable-rust --enable-cargo-online-mode + +Tor's test should be run by doing: + + make check + +Tor's integration tests should also pass: + + make test-stem + + Submitting a patch +===================== + +Please follow the instructions in `.../doc/HACKING/GettingStarted.md`. diff --git a/doc/HACKING/HelpfulTools.md b/doc/HACKING/HelpfulTools.md index a7f36e6c7e..f919d08ec1 100644 --- a/doc/HACKING/HelpfulTools.md +++ b/doc/HACKING/HelpfulTools.md @@ -111,15 +111,19 @@ Running gcov for unit test coverage (On OSX, you'll need to start with `--enable-coverage CC=clang`.) -Then, look at the .gcov files in `coverage-output`. '-' before a line means -that the compiler generated no code for that line. '######' means that the -line was never reached. Lines with numbers were called that number of times. - If that doesn't work: * Try configuring Tor with `--disable-gcc-hardening` * You might need to run `make clean` after you run `./configure`. +Then, look at the .gcov files in `coverage-output`. '-' before a line means +that the compiler generated no code for that line. '######' means that the +line was never reached. Lines with numbers were called that number of times. + +For more details about how to read gcov output, see the [Invoking +gcov](https://gcc.gnu.org/onlinedocs/gcc/Invoking-Gcov.html) chapter +of the GCC manual. + If you make changes to Tor and want to get another set of coverage results, you can run `make reset-gcov` to clear the intermediary gcov output. @@ -128,9 +132,13 @@ a meaningful diff between them, you can run: ./scripts/test/cov-diff coverage-output1 coverage-output2 | less -In this diff, any lines that were visited at least once will have coverage -"1". This lets you inspect what you (probably) really want to know: which -untested lines were changed? Are there any new untested lines? +In this diff, any lines that were visited at least once will have coverage "1", +and line numbers are deleted. This lets you inspect what you (probably) really +want to know: which untested lines were changed? Are there any new untested +lines? + +If you run ./scripts/test/cov-exclude, it marks excluded unreached +lines with 'x', and excluded reached lines with '!!!'. Running integration tests ------------------------- @@ -142,6 +150,12 @@ run `make test-network`. We also have scripts to run integration tests using Stem. To try them, set `STEM_SOURCE_DIR` to your Stem source directory, and run `test-stem`. +Profiling Tor +------------- + +Ongoing notes about Tor profiling can be found at +https://pad.riseup.net/p/profiling-tor + Profiling Tor with oprofile --------------------------- @@ -168,20 +182,62 @@ Here are some basic instructions * `opreport -l that_dir/*` - Profit +Profiling Tor with perf +----------------------- + +This works with a running Tor, and requires root. + +1. Decide how long you want to profile for. Start with (say) 30 seconds. If that + works, try again with longer times. + +2. Find the PID of your running tor process. + +3. Run `perf record --call-graph dwarf -p <PID> sleep <SECONDS>` + + (You may need to do this as root.) + + You might need to add `-e cpu-clock` as an option to the perf record line + above, if you are on an older CPU without access to hardware profiling + events, or in a VM, or something. + +4. Now you have a perf.data file. Have a look at it with `perf report + --no-children --sort symbol,dso` or `perf report --no-children --sort + symbol,dso --stdio --header`. How does it look? + +5a. Once you have a nice big perf.data file, you can compress it, encrypt it, + and send it to your favorite Tor developers. + +5b. Or maybe you'd rather not send a nice big perf.data file. Who knows what's + in that!? It's kinda scary. To generate a less scary file, you can use `perf + report -g > <FILENAME>.out`. Then you can compress that and put it somewhere + public. + +Profiling Tor with gperftools aka Google-performance-tools +---------------------------------------------------------- + +This should work on nearly any unixy system. It doesn't seem to be compatible +with RunAsDaemon though. + +Beforehand, install google-perftools. + +1. You need to rebuild Tor, hack the linking steps to add `-lprofiler` to the + libs. You can do this by adding `LIBS=-lprofiler` when you call `./configure`. + +Now you can run Tor with profiling enabled, and use the pprof utility to look at +performance! See the gperftools manual for more info, but basically: + +2. Run `env CPUPROFILE=/tmp/profile src/or/tor -f <path/torrc>`. The profile file + is not written to until Tor finishes execuction. + +3. Run `pprof src/or/tor /tm/profile` to start the REPL. + Generating and analyzing a callgraph ------------------------------------ -1. Run `./scripts/maint/generate_callgraph.sh`. This will generate a - bunch of files in a new ./callgraph directory. - -2. Run `./scripts/maint/analyze_callgraph.py callgraph/src/*/*`. This - will do a lot of graph operations and then dump out a new - `callgraph.pkl` file, containing data in Python's 'pickle' format. +0. Build Tor on linux or mac, ideally with -O0 or -fno-inline. -3. Run `./scripts/maint/display_callgraph.py`. It will display: - - the number of functions reachable from each function. - - all strongly-connnected components in the Tor callgraph - - the largest bottlenecks in the largest SCC in the Tor callgraph. +1. Clone 'https://gitweb.torproject.org/user/nickm/calltool.git/' . + Follow the README in that repository. Note that currently the callgraph generator can't detect calls that pass through function pointers. diff --git a/doc/HACKING/HowToReview.md b/doc/HACKING/HowToReview.md index d53318942f..2d1f3d1c9e 100644 --- a/doc/HACKING/HowToReview.md +++ b/doc/HACKING/HowToReview.md @@ -19,6 +19,8 @@ Top-level smell-checks - Does `make check-spaces` pass? +- Does `make check-changes` pass? + - Does it have a reasonable amount of tests? Do they pass? Do they leak memory? @@ -32,6 +34,7 @@ Top-level smell-checks - If this changes Tor's behavior on the wire, is there a design proposal? +- If this changes anything in the code, is there a "changes" file? Let's look at the code! diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md index 7595398241..62029b44f0 100644 --- a/doc/HACKING/ReleasingTor.md +++ b/doc/HACKING/ReleasingTor.md @@ -8,7 +8,11 @@ new Tor release: === 0. Preliminaries 1. Get at least three of weasel/arma/Sebastian/Sina to put the new - version number in their approved versions list. + version number in their approved versions list. Give them a few + days to do this if you can. + +2. If this is going to be an important security release, give the packagers + some advance warning: See this list of packagers in IV.3 below. === I. Make sure it works @@ -26,39 +30,37 @@ new Tor release: What about Coverity Scan? - Is make check-spaces happy? + What about clan scan-build? - Does 'make distcheck' compain? + Does 'make distcheck' complain? - How about 'make test-stem' and 'make test-network'? + How about 'make test-stem' and 'make test-network' and + `make test-network-full`? - Are all those tests still happy with --enable-expensive-hardening ? Any memory leaks? -=== II. Write a changelog. +=== II. Write a changelog + +1a. (Alpha release variant) -1. Gather the `changes/*` files into a changelog entry, rewriting many + Gather the `changes/*` files into a changelog entry, rewriting many of them and reordering to focus on what users and funders would find interesting and understandable. - 1. Make sure that everything that wants a bug number has one. - Make sure that everything which is a bugfix says what version - it was a bugfix on. - - 2. Concatenate them. + To do this, first run `./scripts/maint/lintChanges.py changes/*` and + fix as many warnings as you can. Then run `./scripts/maint/sortChanges.py + changes/* > changelog.in` to combine headings and sort the entries. + After that, it's time to hand-edit and fix the issues that lintChanges + can't find: - 3. Sort them by section. Within each section, sort by "version it's - a bugfix on", else by numerical ticket order. + 1. Within each section, sort by "version it's a bugfix on", else by + numerical ticket order. - 4. Clean them up: - - Standard idioms: - `Fixes bug 9999; bugfix on 0.3.3.3-alpha.` - - One space after a period. + 2. Clean them up: Make stuff very terse @@ -86,19 +88,32 @@ new Tor release: maint-0.2.1), be sure to make the stanzas identical (so people can distinguish if these are the same change). - 5. Merge them in. + 3. Clean everything one last time. + + 4. Run `./scripts/maint/format_changelog.py --inplace` to make it prettier - 6. Clean everything one last time. +1b. (old-stable release variant) - 7. Run `./scripts/maint/format_changelog.py` to make it prettier. + For stable releases that backport things from later, we try to compose + their releases, we try to make sure that we keep the changelog entries + identical to their original versions, with a 'backport from 0.x.y.z' + note added to each section. So in this case, once you have the items + from the changes files copied together, don't use them to build a new + changelog: instead, look up the corrected versions that were merged + into ChangeLog in the master branch, and use those. 2. Compose a short release blurb to highlight the user-facing changes. Insert said release blurb into the ChangeLog stanza. If it's a stable release, add it to the ReleaseNotes file too. If we're adding - to a release-0.2.x branch, manually commit the changelogs to the later + to a release-* branch, manually commit the changelogs to the later git branches too. -3. If you're doing the first stable release in a series, you need to +3. If there are changes that require or suggest operator intervention + before or during the update, mail operators (either dirauth or relays + list) with a headline that indicates that an action is required or + appreciated. + +4. If you're doing the first stable release in a series, you need to create a ReleaseNotes for the series as a whole. To get started there, copy all of the Changelog entries from the series into a new file, and run `./scripts/maint/sortChanges.py` on it. That will @@ -111,51 +126,52 @@ new Tor release: === III. Making the source release. -1. In `maint-0.2.x`, bump the version number in `configure.ac` and run - `scripts/maint/updateVersions.pl` to update version numbers in other - places, and commit. Then merge `maint-0.2.x` into `release-0.2.x`. +1. In `maint-0.?.x`, bump the version number in `configure.ac` and run + `perl scripts/maint/updateVersions.pl` to update version numbers in other + places, and commit. Then merge `maint-0.?.x` into `release-0.?.x`. (NOTE: To bump the version number, edit `configure.ac`, and then run either `make`, or `perl scripts/maint/updateVersions.pl`, depending on your version.) -2. Make distcheck, put the tarball up somewhere, and tell `#tor` about - it. Wait a while to see if anybody has problems building it. Try to - get Sebastian or somebody to try building it on Windows. +2. Make distcheck, put the tarball up in somewhere (how about your + homedir on your homedir on people.torproject.org?) , and tell `#tor` + about it. Wait a while to see if anybody has problems building it. + (Though jenkins is usually pretty good about catching these things.) === IV. Commit, upload, announce 1. Sign the tarball, then sign and push the git tag: gpg -ba <the_tarball> - git tag -u <keyid> tor-0.2.x.y-status - git push origin tag tor-0.2.x.y-status + git tag -u <keyid> tor-0.3.x.y-status + git push origin tag tor-0.3.x.y-status 2. scp the tarball and its sig to the dist website, i.e. `/srv/dist-master.torproject.org/htdocs/` on dist-master. When you want it to go live, you run "static-update-component dist.torproject.org" on dist-master. - Edit `include/versions.wmi` and `Makefile` to note the new version. + In the webwml.git repository, `include/versions.wmi` and `Makefile` + to note the new version. (NOTE: Due to #17805, there can only be one stable version listed at once. Nonetheless, do not call your version "alpha" if it is stable, or people will get confused.) -3. Email the packagers (cc'ing tor-assistants) that a new tarball is up. +3. Email the packagers (cc'ing tor-team) that a new tarball is up. The current list of packagers is: - {weasel,gk,mikeperry} at torproject dot org - {blueness} at gentoo dot org - {paul} at invizbox dot io + - {vincent} at invizbox dot com - {lfleischer} at archlinux dot org - {Nathan} at freitas dot net - {mike} at tig dot as - - {tails-rm} at boum dot org (for pre-release announcments) - - - - {tails-dev} at boum dot org (for at-release announcements) - + - {tails-rm} at boum dot org + - {simon} at sdeziel.info + - {yuri} at rawbw.com 4. Add the version number to Trac. To do this, go to Trac, log in, select "Admin" near the top of the screen, then select "Versions" from @@ -164,22 +180,26 @@ new Tor release: 0.2.2.23-alpha" (or whatever the version is), and we select the date as the date in the ChangeLog. -5. Wait up to a day or two (for a development release), or until most - packages are up (for a stable release), and mail the release blurb and - changelog to tor-talk or tor-announce. +5. Mail the release blurb and ChangeLog to tor-talk (development release) or + tor-announce (stable). + + Post the changelog on the blog as well. You can generate a + blog-formatted version of the changelog with the -B option to + format-changelog. - (We might be moving to faster announcements, but don't announce until - the website is at least updated.) + When you post, include an estimate of when the next TorBrowser + releases will come out that include this Tor release. This will + usually track https://wiki.mozilla.org/RapidRelease/Calendar , but it + can vary. === V. Aftermath and cleanup -1. If it's a stable release, bump the version number in the `maint-x.y.z` - branch to "newversion-dev", and do a `merge -s ours` merge to avoid - taking that change into master. Do a similar `merge -s theirs` - merge to get the change (and only that change) into release. (Some - of the build scripts require that maint merge cleanly into release.) +1. If it's a stable release, bump the version number in the + `maint-x.y.z` branch to "newversion-dev", and do a `merge -s ours` + merge to avoid taking that change into master. 2. Forward-port the ChangeLog (and ReleaseNotes if appropriate). +3. Keep an eye on the blog post, to moderate comments and answer questions. diff --git a/doc/HACKING/Tracing.md b/doc/HACKING/Tracing.md new file mode 100644 index 0000000000..a5fb5165e2 --- /dev/null +++ b/doc/HACKING/Tracing.md @@ -0,0 +1,91 @@ +# Tracing # + +This document describes how the event tracing subsystem works in tor so +developers can add events to the code base but also hook them to an event +tracing framework. + +## Basics ### + +Event tracing is seperated in two concepts, trace events and a tracer. The +tracing subsystem can be found in `src/trace`. The `events.h` header file is +the main file that maps the different tracers to trace events. + +### Events ### + +A trace event is basically a function from which we can pass any data that +we want to collect. In addition, we specify a context for the event such as +a subsystem and an event name. + +A trace event in tor has the following standard format: + + tor_trace(subsystem, event\_name, args...) + +The `subsystem` parameter is the name of the subsytem the trace event is in. +For example that could be "scheduler" or "vote" or "hs". The idea is to add +some context to the event so when we collect them we know where it's coming +from. The `event_name` is the name of the event which helps a lot with +adding some semantic to the event. Finally, `args` is any number of +arguments we want to collect. + +Here is an example of a possible tracepoint in main(): + + tor_trace(main, init_phase, argc) + +The above is a tracepoint in the `main` subsystem with `init_phase` as the +event name and the `int argc` is passed to the event as well. + +How `argc` is collected or used has nothing to do with the instrumentation +(adding trace events to the code). It is the work of the tracer so this is why +the trace events and collection framework (tracer) are decoupled. You _can_ +have trace events without a tracer. + +### Tracer ### + +In `src/trace/events.h`, we map the `tor_trace()` function to the right +tracer. A tracer support is only enabled at compile time. For instance, the +file `src/trace/debug.h` contains the mapping of the generic tracing function +`tor_trace()` to the `log_debug()` function. More specialized function can be +mapped depending on the tracepoint. + +## Build System ## + +This section describes how it is integrated into the build system of tor. + +By default, every tracing events are disabled in tor that is `tor_trace()` +is a NOP. + +To enable a tracer, there is a configure option on the form of: + + --enable-tracing-<tracer> + +We have an option that will send every trace events to a `log_debug()` (as +mentionned above) which will print you the subsystem and name of the event but +not the arguments for technical reasons. This is useful if you want to quickly +see if your trace event is being hit or well written. To do so, use this +configure option: + + --enable-tracing-debug + +## Instrument Tor ## + +This is pretty easy. Let's say you want to add a trace event in +`src/or/rendcache.c`, you only have to add this include statement: + + #include "trace/events.h" + +Once done, you can add as many as you want `tor_trace()` that you need. +Please use the right subsystem (here it would be `hs`) and a unique name that +tells what the event is for. For example: + + tor_trace(hs, store_desc_as_client, desc, desc_id); + +If you look in `src/trace/events.h`, you'll see that if tracing is enabled it +will be mapped to a function called: + + tor_trace_hs_store_desc_as_client(desc, desc_id) + +And the point of all this is for that function to be defined in a new file +that you might want to add named `src/trace/hs.{c|h}` which would defined how +to collect the data for the `tor_trace_hs_store_desc_as_client()` function +like for instance sending it to a `log_debug()` or do more complex operations +or use a userspace tracer like LTTng (https://lttng.org). diff --git a/doc/HACKING/WritingTests.md b/doc/HACKING/WritingTests.md index de80bbdef2..cc393494ec 100644 --- a/doc/HACKING/WritingTests.md +++ b/doc/HACKING/WritingTests.md @@ -48,7 +48,7 @@ isolation, you just run `./src/test/test-memwipe`. To run tests within the unit test programs, you can specify the name of the test. The string ".." can be used as a wildcard at the end of the test name. For example, to run all the cell format tests, enter -`./src/test/test cellfmt/..`. To run +`./src/test/test cellfmt/..`. Many tests that need to mess with global state run in forked subprocesses in order to keep from contaminating one another. But when debugging a failing test, @@ -91,6 +91,9 @@ coverage percentage. For a summary of the test coverage for each _function_, run `./scripts/test/cov-display -f ${TMPDIR}/*`. +For more details on using gcov, including the helper scripts in +scripts/test, see HelpfulTools.md. + ### Comparing test coverage Sometimes it's useful to compare test coverage for a branch you're writing to @@ -117,7 +120,8 @@ with LCOV_EXCL_START... LCOV_EXCL_STOP. Note that older versions of lcov don't understand these lines. You can post-process .gcov files to make these lines 'unreached' by -running ./scripts/test/cov-exclude on them. +running ./scripts/test/cov-exclude on them. It marks excluded +unreached lines with 'x', and excluded reached lines with '!!!'. Note: you should never do this unless the line is meant to 100% unreachable by actual code. diff --git a/doc/include.am b/doc/include.am index 7164a4b2a0..0e8de231e1 100644 --- a/doc/include.am +++ b/doc/include.am @@ -12,17 +12,11 @@ # part of the source distribution, so that people without asciidoc can # just use the .1 and .html files. -base_mans = doc/tor doc/tor-gencert doc/tor-resolve doc/torify -all_mans = $(base_mans) -if USE_FW_HELPER -install_mans = $(all_mans) -else -install_mans = $(base_mans) -endif +all_mans = doc/tor doc/tor-gencert doc/tor-resolve doc/torify if USE_ASCIIDOC -nodist_man1_MANS = $(install_mans:=.1) -doc_DATA = $(install_mans:=.html) +nodist_man1_MANS = $(all_mans:=.1) +doc_DATA = $(all_mans:=.html) html_in = $(all_mans:=.html.in) man_in = $(all_mans:=.1.in) txt_in = $(all_mans:=.1.txt) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index a7ee7d11ca..d7884968e6 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -128,6 +128,16 @@ COMMAND-LINE OPTIONS the passphrase, including any trailing newlines. Default: read from the terminal. +[[opt-key-expiration]] **--key-expiration** [**purpose**]:: + The **purpose** specifies which type of key certificate to determine + the expiration of. The only currently recognised **purpose** is + "sign". + + + + Running "tor --key-expiration sign" will attempt to find your signing + key certificate and will output, both in the logs as well as to stdout, + the signing key certificate's expiration time in ISO-8601 format. + For example, the output sent to stdout will be of the form: + "signing-cert-expiry: 2017-07-25 08:30:15 UTC" Other options can be specified on the command-line in the format "--option value", in the format "option value", or in a configuration file. For @@ -153,6 +163,13 @@ values. To split one configuration entry into multiple lines, use a single backslash character (\) before the end of the line. Comments can be used in such multiline entries, but they must start at the beginning of a line. +Configuration options can be imported from files or folders using the %include +option with the value being a path. If the path is a file, the options from the +file will be parsed as if they were written where the %include option is. If +the path is a folder, all files on that folder will be parsed following lexical +order. Files starting with a dot are ignored. Files on subfolders are ignored. +The %include option can be used recursively. + By default, an option on the command line overrides an option found in the configuration file, and an option in a configuration file overrides one in the defaults file. @@ -176,7 +193,7 @@ forward slash (/) in the configuration file and on the command line. GENERAL OPTIONS --------------- -[[BandwidthRate]] **BandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[BandwidthRate]] **BandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: A token bucket limits the average incoming bandwidth usage on this node to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. If you want to run a relay in the @@ -185,6 +202,9 @@ GENERAL OPTIONS course, more is better; we recommend at least 250 KBytes (2 mbits) if possible. (Default: 1 GByte) + + + Note that this option, and other bandwidth-limiting options, apply to TCP + data only: They do not count TCP headers or DNS traffic. + + + With this option, and in other options that take arguments in bytes, KBytes, and so on, other formats are also supported. Notably, "KBytes" can also be written as "kilobytes" or "kb"; "MBytes" can be written as @@ -195,43 +215,45 @@ GENERAL OPTIONS To avoid confusion, we recommend writing "bytes" or "bits" explicitly, since it's easy to forget that "B" means bytes, not bits. -[[BandwidthBurst]] **BandwidthBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[BandwidthBurst]] **BandwidthBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Limit the maximum token bucket size (also known as the burst) to the given number of bytes in each direction. (Default: 1 GByte) -[[MaxAdvertisedBandwidth]] **MaxAdvertisedBandwidth** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[MaxAdvertisedBandwidth]] **MaxAdvertisedBandwidth** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: If set, we will not advertise more than this amount of bandwidth for our BandwidthRate. Server operators who want to reduce the number of clients who ask to build circuits through them (since this is proportional to advertised bandwidth rate) can thus reduce the CPU demands on their server without impacting network performance. -[[RelayBandwidthRate]] **RelayBandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[RelayBandwidthRate]] **RelayBandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: If not 0, a separate token bucket limits the average incoming bandwidth usage for \_relayed traffic_ on this node to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. Relayed traffic currently is calculated to include answers to directory requests, but that may change in future versions. (Default: 0) -[[RelayBandwidthBurst]] **RelayBandwidthBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[RelayBandwidthBurst]] **RelayBandwidthBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: If not 0, limit the maximum token bucket size (also known as the burst) for \_relayed traffic_ to the given number of bytes in each direction. (Default: 0) -[[PerConnBWRate]] **PerConnBWRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: - If set, do separate rate limiting for each connection from a non-relay. - You should never need to change this value, since a network-wide value is - published in the consensus and your relay will use that value. (Default: 0) +[[PerConnBWRate]] **PerConnBWRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: + If this option is set manually, or via the "perconnbwrate" consensus + field, Tor will use it for separate rate limiting for each connection + from a non-relay. (Default: 0) -[[PerConnBWBurst]] **PerConnBWBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: - If set, do separate rate limiting for each connection from a non-relay. - You should never need to change this value, since a network-wide value is - published in the consensus and your relay will use that value. (Default: 0) +[[PerConnBWBurst]] **PerConnBWBurst** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: + If this option is set manually, or via the "perconnbwburst" consensus + field, Tor will use it for separate rate limiting for each connection + from a non-relay. (Default: 0) [[ClientTransportPlugin]] **ClientTransportPlugin** __transport__ socks4|socks5 __IP__:__PORT__:: **ClientTransportPlugin** __transport__ exec __path-to-binary__ [options]:: In its first form, when set along with a corresponding Bridge line, the Tor - client forwards its traffic to a SOCKS-speaking proxy on "IP:PORT". It's the + client forwards its traffic to a SOCKS-speaking proxy on "IP:PORT". + (IPv4 addresses should written as-is; IPv6 addresses should be wrapped in + square brackets.) It's the duty of that proxy to properly forward the traffic to the bridge. + + In its second form, when set along with a corresponding Bridge line, the Tor @@ -248,7 +270,8 @@ GENERAL OPTIONS [[ServerTransportListenAddr]] **ServerTransportListenAddr** __transport__ __IP__:__PORT__:: When this option is set, Tor will suggest __IP__:__PORT__ as the listening address of any pluggable transport proxy that tries to - launch __transport__. + launch __transport__. (IPv4 addresses should written as-is; IPv6 + addresses should be wrapped in square brackets.) [[ServerTransportOptions]] **ServerTransportOptions** __transport__ __k=v__ __k=v__ ...:: When this option is set, Tor will pass the __k=v__ parameters to @@ -338,14 +361,6 @@ GENERAL OPTIONS Unix domain sockets only: Do not insist that the directory that holds the socket be read-restricted. -[[ControlListenAddress]] **ControlListenAddress** __IP__[:__PORT__]:: - Bind the controller listener to this address. If you specify a port, bind - to this port rather than the one specified in ControlPort. We strongly - recommend that you leave this alone unless you know what you're doing, - since giving attackers access to your control listener is really - dangerous. This directive can be specified multiple - times to bind to multiple addresses/ports. (Default: 127.0.0.1) - [[ControlSocket]] **ControlSocket** __Path__:: Like ControlPort, but listens on a Unix domain socket, rather than a TCP socket. '0' disables ControlSocket (Unix and Unix-like systems only.) @@ -390,14 +405,17 @@ GENERAL OPTIONS file readable by the default GID. (Default: 0) [[DataDirectory]] **DataDirectory** __DIR__:: - Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor) + Store working data in DIR. Can not be changed while tor is running. + (Default: ~/.tor if your home directory is not /; otherwise, + @LOCALSTATEDIR@/lib/tor. On Windows, the default is + your ApplicationData folder.) [[DataDirectoryGroupReadable]] **DataDirectoryGroupReadable** **0**|**1**:: If this option is set to 0, don't allow the filesystem group to read the DataDirectory. If the option is set to 1, make the DataDirectory readable by the default GID. (Default: 0) -[[FallbackDir]] **FallbackDir** __address__:__port__ orport=__port__ id=__fingerprint__ [weight=__num__] [ipv6=__address__:__orport__]:: +[[FallbackDir]] **FallbackDir** __ipv4address__:__port__ orport=__port__ id=__fingerprint__ [weight=__num__] [ipv6=**[**__ipv6address__**]**:__orport__]:: When we're unable to connect to any directory cache for directory info (usually because we don't know about any yet) we try a directory authority. Clients also simultaneously try a FallbackDir, to avoid hangs on client @@ -413,7 +431,7 @@ GENERAL OPTIONS FallbackDir line is present, it replaces the hard-coded FallbackDirs, regardless of the value of UseDefaultFallbackDirs.) (Default: 1) -[[DirAuthority]] **DirAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__:: +[[DirAuthority]] **DirAuthority** [__nickname__] [**flags**] __ipv4address__:__port__ __fingerprint__:: Use a nonstandard authoritative directory server at the provided address and port, with the specified key fingerprint. This option can be repeated many times, for multiple authoritative directory servers. Flags are @@ -427,13 +445,16 @@ GENERAL OPTIONS with probability proportional to that weight (default 1.0). If a flag "v3ident=**fp**" is given, the dirserver is a v3 directory authority whose v3 long-term signing key has the fingerprint **fp**. Lastly, - if an "ipv6=__address__:__orport__" flag is present, then the directory + if an "ipv6=**[**__ipv6address__**]**:__orport__" flag is present, then + the directory authority is listening for IPv6 connections on the indicated IPv6 address and OR Port. + + - Tor will contact the authority at __address__:__port__ (the DirPort) to - download directory documents. If an IPv6 address is supplied, Tor will - also download directory documents at the IPv6 address on the DirPort. + + Tor will contact the authority at __ipv4address__ to + download directory documents. The provided __port__ value is a dirport; + clients ignore this in favor of the specified "orport=" value. If an + IPv6 ORPort is supplied, Tor will + also download directory documents at the IPv6 ORPort. + + If no **DirAuthority** line is given, Tor will use the default directory authorities. NOTE: this option is intended for setting up a private Tor @@ -448,9 +469,9 @@ GENERAL OPTIONS should be 1.0 or less. The default is less than 1, to reduce load on authorities. (Default: 0.1) -[[AlternateDirAuthority]] **AlternateDirAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__ + +[[AlternateDirAuthority]] **AlternateDirAuthority** [__nickname__] [**flags**] __ipv4address__:__port__ __fingerprint__ + -[[AlternateBridgeAuthority]] **AlternateBridgeAuthority** [__nickname__] [**flags**] __address__:__port__ __ fingerprint__:: +[[AlternateBridgeAuthority]] **AlternateBridgeAuthority** [__nickname__] [**flags**] __ipv4address__:__port__ __ fingerprint__:: These options behave as DirAuthority, but they replace fewer of the default directory authorities. Using AlternateDirAuthority replaces the default Tor directory authorities, but @@ -465,7 +486,8 @@ GENERAL OPTIONS not supported. We believe that this feature works on modern Gnu/Linux distributions, and that it should work on *BSD systems (untested). This option requires that you start your Tor as root, and you should use the - **User** option to properly reduce Tor's privileges. (Default: 0) + **User** option to properly reduce Tor's privileges. + Can not be changed while tor is running. (Default: 0) [[DisableDebuggerAttachment]] **DisableDebuggerAttachment** **0**|**1**:: If set to 1, Tor will attempt to prevent basic debugging attachment attempts @@ -505,22 +527,25 @@ GENERAL OPTIONS (Default: 1) [[FetchUselessDescriptors]] **FetchUselessDescriptors** **0**|**1**:: - If set to 1, Tor will fetch every non-obsolete descriptor from the - authorities that it hears about. Otherwise, it will avoid fetching useless - descriptors, for example for routers that are not running. This option is - useful if you're using the contributed "exitlist" script to enumerate Tor - nodes that exit to certain addresses. (Default: 0) + If set to 1, Tor will fetch every consensus flavor, descriptor, and + certificate that it hears about. Otherwise, it will avoid fetching useless + descriptors: flavors that it is not using to build circuits, and authority + certificates it does not trust. This option is useful if you're using a + tor client with an external parser that uses a full consensus. + This option fetches all documents, **DirCache** fetches and serves + all documents. (Default: 0) [[HTTPProxy]] **HTTPProxy** __host__[:__port__]:: Tor will make all its directory requests through this host:port (or host:80 if port is not specified), rather than connecting directly to any directory - servers. + servers. (DEPRECATED: As of 0.3.1.0-alpha you should use HTTPSProxy.) [[HTTPProxyAuthenticator]] **HTTPProxyAuthenticator** __username:password__:: If defined, Tor will use this username:password for Basic HTTP proxy authentication, as in RFC 2617. This is currently the only form of HTTP proxy authentication that Tor supports; feel free to submit a patch if you - want it to support others. + want it to support others. (DEPRECATED: As of 0.3.1.0-alpha you should use + HTTPSProxyAuthenticator.) [[HTTPSProxy]] **HTTPSProxy** __host__[:__port__]:: Tor will make all its OR (SSL) connections through this host:port (or @@ -538,7 +563,22 @@ GENERAL OPTIONS [[Sandbox]] **Sandbox** **0**|**1**:: If set to 1, Tor will run securely through the use of a syscall sandbox. Otherwise the sandbox will be disabled. The option is currently an - experimental feature. (Default: 0) + experimental feature. It only works on Linux-based operating systems, + and only when Tor has been built with the libseccomp library. This option + can not be changed while tor is running. + + + When the Sandbox is 1, the following options can not be changed when tor + is running: + Address + ConnLimit + CookieAuthFile + DirPortFrontPage + ExtORPortCookieAuthFile + Logs + ServerDNSResolvConfFile + Tor must remain in client or server mode (some changes to ClientOnly and + ORPort are not allowed). + (Default: 0) [[Socks4Proxy]] **Socks4Proxy** __host__[:__port__]:: Tor will make all OR connections through the SOCKS 4 proxy at host:port @@ -609,7 +649,7 @@ GENERAL OPTIONS message currently has at least one domain; most currently have exactly one. This doesn't affect controller log messages. (Default: 0) -[[MaxUnparseableDescSizeToLog]] **MaxUnparseableDescSizeToLog** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**:: +[[MaxUnparseableDescSizeToLog]] **MaxUnparseableDescSizeToLog** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**:: Unparseable descriptors (e.g. for votes, consensuses, routers) are logged in separate files by hash, up to the specified size in total. Note that only files logged during the lifetime of this Tor process count toward the @@ -621,27 +661,41 @@ GENERAL OPTIONS is only useful when you have multiple network interfaces, and you want all of Tor's outgoing connections to use a single one. This option may be used twice, once with an IPv4 address and once with an IPv6 address. + IPv6 addresses should be wrapped in square brackets. This setting will be ignored for connections to the loopback addresses (127.0.0.0/8 and ::1). +[[OutboundBindAddressOR]] **OutboundBindAddressOR** __IP__:: + Make all outbound non-exit (relay and other) connections + originate from the IP address specified. This option overrides + **OutboundBindAddress** for the same IP version. This option may + be used twice, once with an IPv4 address and once with an IPv6 + address. IPv6 addresses should be wrapped in square brackets. + This setting will be ignored for connections to the loopback + addresses (127.0.0.0/8 and ::1). + +[[OutboundBindAddressExit]] **OutboundBindAddressExit** __IP__:: + Make all outbound exit connections originate from the IP address + specified. This option overrides **OutboundBindAddress** for the + same IP version. This option may be used twice, once with an IPv4 + address and once with an IPv6 address. + IPv6 addresses should be wrapped in square brackets. + This setting will be ignored + for connections to the loopback addresses (127.0.0.0/8 and ::1). + [[PidFile]] **PidFile** __FILE__:: On startup, write our PID to FILE. On clean shutdown, remove - FILE. + FILE. Can not be changed while tor is running. [[ProtocolWarnings]] **ProtocolWarnings** **0**|**1**:: If 1, Tor will log with severity \'warn' various cases of other parties not following the Tor specification. Otherwise, they are logged with severity \'info'. (Default: 0) -[[PredictedPortsRelevanceTime]] **PredictedPortsRelevanceTime** __NUM__:: - Set how long, after the client has made an anonymized connection to a - given port, we will try to make sure that we build circuits to - exits that support that port. The maximum value for this option is 1 - hour. (Default: 1 hour) - [[RunAsDaemon]] **RunAsDaemon** **0**|**1**:: If 1, Tor forks and daemonizes to the background. This option has no effect on Windows; instead you should use the --service command-line option. + Can not be changed while tor is running. (Default: 0) [[LogTimeGranularity]] **LogTimeGranularity** __NUM__:: @@ -658,7 +712,8 @@ GENERAL OPTIONS [[SyslogIdentityTag]] **SyslogIdentityTag** __tag__:: When logging to syslog, adds a tag to the syslog identity such that - log entries are marked with "Tor-__tag__". (Default: none) + log entries are marked with "Tor-__tag__". Can not be changed while tor is + running. (Default: none) [[SafeLogging]] **SafeLogging** **0**|**1**|**relay**:: Tor can scrub potentially sensitive strings from log messages (e.g. @@ -673,6 +728,7 @@ GENERAL OPTIONS [[User]] **User** __Username__:: On startup, setuid to this user and setgid to their primary group. + Can not be changed while tor is running. [[KeepBindCapabilities]] **KeepBindCapabilities** **0**|**1**|**auto**:: On Linux, when we are started as root and we switch our identity using @@ -680,20 +736,23 @@ GENERAL OPTIONS try to retain our ability to bind to low ports. If this value is 1, we try to keep the capability; if it is 0 we do not; and if it is **auto**, we keep the capability only if we are configured to listen on a low port. + Can not be changed while tor is running. (Default: auto.) [[HardwareAccel]] **HardwareAccel** **0**|**1**:: If non-zero, try to use built-in (static) crypto hardware acceleration when - available. (Default: 0) + available. Can not be changed while tor is running. (Default: 0) [[AccelName]] **AccelName** __NAME__:: When using OpenSSL hardware crypto acceleration attempt to load the dynamic engine of this name. This must be used for any dynamic hardware engine. - Names can be verified with the openssl engine command. + Names can be verified with the openssl engine command. Can not be changed + while tor is running. [[AccelDir]] **AccelDir** __DIR__:: Specify this option if using dynamic hardware acceleration and the engine implementation library resides somewhere other than the OpenSSL default. + Can not be changed while tor is running. [[AvoidDiskWrites]] **AvoidDiskWrites** **0**|**1**:: If non-zero, try to write to disk less frequently than we would otherwise. @@ -718,27 +777,64 @@ GENERAL OPTIONS 127.0.0.1 or 10.0.0.1. This is mostly useful for debugging rate-limiting. (Default: 0) +[[ExtendByEd25519ID]] **ExtendByEd25519ID** **0**|**1**|**auto**:: + If this option is set to 1, we always try to include a relay's Ed25519 ID + when telling the proceeding relay in a circuit to extend to it. + If this option is set to 0, we never include Ed25519 IDs when extending + circuits. If the option is set to "default", we obey a + parameter in the consensus document. (Default: auto) + +[[NoExec]] **NoExec** **0**|**1**:: + If this option is set to 1, then Tor will never launch another + executable, regardless of the settings of PortForwardingHelper, + ClientTransportPlugin, or ServerTransportPlugin. Once this + option has been set to 1, it cannot be set back to 0 without + restarting Tor. (Default: 0) + +[[Schedulers]] **Schedulers** **KIST**|**KISTLite**|**Vanilla**:: + Specify the scheduler type that tor should use. The scheduler is + responsible for moving data around within a Tor process. This is an ordered + list by priority which means that the first value will be tried first and if + unavailable, the second one is tried and so on. It is possible to change + these values at runtime. This option mostly effects relays, and most + operators should leave it set to its default value. + (Default: KIST,KISTLite,Vanilla) + + + The possible scheduler types are: + + + **KIST**: Kernel-Informed Socket Transport. Tor will use TCP information + from the kernel to make informed decisions regarding how much data to send + and when to send it. KIST also handles traffic in batches (see + KISTSchedRunInterval) in order to improve traffic prioritization decisions. + As implemented, KIST will only work on Linux kernel version 2.6.39 or + higher. + + + **KISTLite**: Same as KIST but without kernel support. Tor will use all + the same mechanics as with KIST, including the batching, but its decisions + regarding how much data to send will not be as good. KISTLite will work on + all kernels and operating systems, and the majority of the benefits of KIST + are still realized with KISTLite. + + + **Vanilla**: The scheduler that Tor used before KIST was implemented. It + sends as much data as possible, as soon as possible. Vanilla will work on + all kernels and operating systems. + +[[KISTSchedRunInterval]] **KISTSchedRunInterval** __NUM__ **msec**:: + If KIST or KISTLite is used in the Schedulers option, this controls at which + interval the scheduler tick is. If the value is 0 msec, the value is taken + from the consensus if possible else it will fallback to the default 10 + msec. Maximum possible value is 100 msec. (Default: 0 msec) + +[[KISTSockBufSizeFactor]] **KISTSockBufSizeFactor** __NUM__:: + If KIST is used in Schedulers, this is a multiplier of the per-socket + limit calculation of the KIST algorithm. (Default: 1.0) + CLIENT OPTIONS -------------- The following options are useful only for clients (that is, if -**SocksPort**, **TransPort**, **DNSPort**, or **NATDPort** is non-zero): - -[[AllowInvalidNodes]] **AllowInvalidNodes** **entry**|**exit**|**middle**|**introduction**|**rendezvous**|**...**:: - If some Tor servers are obviously not working right, the directory - authorities can manually mark them as invalid, meaning that it's not - recommended you use them for entry or exit positions in your circuits. You - can opt to use them in some circuit positions, though. The default is - "middle,rendezvous", and other choices are not advised. - -[[ExcludeSingleHopRelays]] **ExcludeSingleHopRelays** **0**|**1**:: - This option controls whether circuits built by Tor will include relays with - the AllowSingleHopExits flag set to true. If ExcludeSingleHopRelays is set - to 0, these relays will be included. Note that these relays might be at - higher risk of being seized or observed, so they are not normally - included. Also note that relatively few clients turn off this option, - so using these relays might make your client stand out. - (Default: 1) +**SocksPort**, **HTTPTunnelPort**, **TransPort**, **DNSPort**, or +**NATDPort** is non-zero): [[Bridge]] **Bridge** [__transport__] __IP__:__ORPort__ [__fingerprint__]:: When set along with UseBridges, instructs Tor to use the relay at @@ -753,7 +849,12 @@ The following options are useful only for clients (that is, if rather than connecting to the bridge directly. Some transports use a transport-specific method to work out the remote address to connect to. These transports typically ignore the "IP:ORPort" specified in the bridge - line. + line. + + + + Tor passes any "key=val" settings to the pluggable transport proxy as + per-connection arguments when connecting to the bridge. Consult + the documentation of the pluggable transport for details of what + arguments it supports. [[LearnCircuitBuildTimeout]] **LearnCircuitBuildTimeout** **0**|**1**:: If 0, CircuitBuildTimeout adaptive learning is disabled. (Default: 1) @@ -766,13 +867,15 @@ The following options are useful only for clients (that is, if LearnCircuitBuildTimeout is 0, this value is the only value used. (Default: 60 seconds) -[[CircuitIdleTimeout]] **CircuitIdleTimeout** __NUM__:: - If we have kept a clean (never used) circuit around for NUM seconds, then - close it. This way when the Tor client is entirely idle, it can expire all - of its circuits, and then expire its TLS connections. Also, if we end up - making a circuit that is not useful for exiting any of the requests we're - receiving, it won't forever take up a slot in the circuit list. (Default: 1 - hour) +[[CircuitsAvailableTimeout]] **CircuitsAvailableTimeout** __NUM__:: + Tor will attempt to keep at least one open, unused circuit available for + this amount of time. This option governs how long idle circuits are kept + open, as well as the amount of time Tor will keep a circuit open to each + of the recently used ports. This way when the Tor client is entirely + idle, it can expire all of its circuits, and then expire its TLS + connections. Note that the actual timeout value is uniformly randomized + from the specified value to twice that amount. (Default: 30 minutes; + Max: 24 hours) [[CircuitStreamTimeout]] **CircuitStreamTimeout** __NUM__:: If non-zero, this option overrides our internal timeout schedule for how @@ -789,6 +892,22 @@ The following options are useful only for clients (that is, if and fast enough. The current behavior is simply that Tor is a client unless ORPort, ExtORPort, or DirPort are configured.) (Default: 0) +[[ConnectionPadding]] **ConnectionPadding** **0**|**1**|**auto**:: + This option governs Tor's use of padding to defend against some forms of + traffic analysis. If it is set to 'auto', Tor will send padding only + if both the client and the relay support it. If it is set to 0, Tor will + not send any padding cells. If it is set to 1, Tor will still send padding + for client connections regardless of relay support. Only clients may set + this option. This option should be offered via the UI to mobile users + for use where bandwidth may be expensive. + (Default: auto) + +[[ReducedConnectionPadding]] **ReducedConnectionPadding** **0**|**1**:: + If set to 1, Tor will not not hold OR connections open for very long, + and will send less padding on these connections. Only clients may set + this option. This option should be offered via the UI to mobile users + for use where bandwidth may be expensive. (Default: 0) + [[ExcludeNodes]] **ExcludeNodes** __node__,__node__,__...__:: A list of identity fingerprints, country codes, and address patterns of nodes to avoid when building a circuit. Country codes are @@ -817,7 +936,7 @@ The following options are useful only for clients (that is, if [[ExcludeExitNodes]] **ExcludeExitNodes** __node__,__node__,__...__:: A list of identity fingerprints, country codes, and address patterns of nodes to never use when picking an exit node---that is, a - node that delivers traffic for you outside the Tor network. Note that any + node that delivers traffic for you *outside* the Tor network. Note that any node listed in ExcludeNodes is automatically considered to be part of this list too. See the **ExcludeNodes** option for more information on how to specify @@ -834,7 +953,7 @@ The following options are useful only for clients (that is, if [[ExitNodes]] **ExitNodes** __node__,__node__,__...__:: A list of identity fingerprints, country codes, and address patterns of nodes to use as exit node---that is, a - node that delivers traffic for you outside the Tor network. See + node that delivers traffic for you *outside* the Tor network. See the **ExcludeNodes** option for more information on how to specify nodes. + + Note that if you list too few nodes here, or if you exclude too many exit @@ -842,7 +961,7 @@ The following options are useful only for clients (that is, if if none of the exits you list allows traffic on port 80 or 443, you won't be able to browse the web. + + - Note also that not every circuit is used to deliver traffic outside of + Note also that not every circuit is used to deliver traffic *outside* of the Tor network. It is normal to see non-exit circuits (such as those used to connect to hidden services, those that do directory fetches, those used for relay reachability self-tests, and so on) that end @@ -852,7 +971,7 @@ The following options are useful only for clients (that is, if The ExcludeNodes option overrides this option: any node listed in both ExitNodes and ExcludeNodes is treated as excluded. + + - The .exit address notation, if enabled via AllowDotExit, overrides + The .exit address notation, if enabled via MapAddress, overrides this option. [[EntryNodes]] **EntryNodes** __node__,__node__,__...__:: @@ -868,16 +987,16 @@ The following options are useful only for clients (that is, if the **ExcludeNodes** option for more information on how to specify nodes. [[StrictNodes]] **StrictNodes** **0**|**1**:: - If StrictNodes is set to 1, Tor will treat the ExcludeNodes option as a - requirement to follow for all the circuits you generate, even if doing so - will break functionality for you. If StrictNodes is set to 0, Tor will + If StrictNodes is set to 1, Tor will treat solely the ExcludeNodes option + as a requirement to follow for all the circuits you generate, even if + doing so will break functionality for you (StrictNodes applies to neither + ExcludeExitNodes nor to ExitNodes). If StrictNodes is set to 0, Tor will still try to avoid nodes in the ExcludeNodes list, but it will err on the - side of avoiding unexpected errors. Specifically, StrictNodes 0 tells - Tor that it is okay to use an excluded node when it is *necessary* to - perform relay reachability self-tests, connect to - a hidden service, provide a hidden service to a client, fulfill a .exit - request, upload directory information, or download directory information. - (Default: 0) + side of avoiding unexpected errors. Specifically, StrictNodes 0 tells Tor + that it is okay to use an excluded node when it is *necessary* to perform + relay reachability self-tests, connect to a hidden service, provide a + hidden service to a client, fulfill a .exit request, upload directory + information, or download directory information. (Default: 0) [[FascistFirewall]] **FascistFirewall** **0**|**1**:: If 1, Tor will only create outgoing connections to ORs running on ports @@ -892,7 +1011,7 @@ The following options are useful only for clients (that is, if **FascistFirewall** is set. This option is deprecated; use ReachableAddresses instead. (Default: 80, 443) -[[ReachableAddresses]] **ReachableAddresses** __ADDR__[/__MASK__][:__PORT__]...:: +[[ReachableAddresses]] **ReachableAddresses** __IP__[/__MASK__][:__PORT__]...:: A comma-separated list of IP addresses and ports that your firewall allows you to connect to. The format is as for the addresses in ExitPolicy, except that "accept" is understood unless "reject" is explicitly provided. For @@ -901,14 +1020,15 @@ The following options are useful only for clients (that is, if 99, rejects port 80 connections to net 18, and accepts connections to port 80 otherwise. (Default: \'accept \*:*'.) -[[ReachableDirAddresses]] **ReachableDirAddresses** __ADDR__[/__MASK__][:__PORT__]...:: +[[ReachableDirAddresses]] **ReachableDirAddresses** __IP__[/__MASK__][:__PORT__]...:: Like **ReachableAddresses**, a list of addresses and ports. Tor will obey these restrictions when fetching directory information, using standard HTTP GET requests. If not set explicitly then the value of **ReachableAddresses** is used. If **HTTPProxy** is set then these - connections will go through that proxy. + connections will go through that proxy. (DEPRECATED: This option has + had no effect for some time.) -[[ReachableORAddresses]] **ReachableORAddresses** __ADDR__[/__MASK__][:__PORT__]...:: +[[ReachableORAddresses]] **ReachableORAddresses** __IP__[/__MASK__][:__PORT__]...:: Like **ReachableAddresses**, a list of addresses and ports. Tor will obey these restrictions when connecting to Onion Routers, using TLS/SSL. If not set explicitly then the value of **ReachableAddresses** is used. If @@ -931,24 +1051,6 @@ The following options are useful only for clients (that is, if services can be configured to require authorization using the **HiddenServiceAuthorizeClient** option. -[[CloseHSClientCircuitsImmediatelyOnTimeout]] **CloseHSClientCircuitsImmediatelyOnTimeout** **0**|**1**:: - If 1, Tor will close unfinished hidden service client circuits - which have not moved closer to connecting to their destination - hidden service when their internal state has not changed for the - duration of the current circuit-build timeout. Otherwise, such - circuits will be left open, in the hope that they will finish - connecting to their destination hidden services. In either case, - another set of introduction and rendezvous circuits for the same - destination hidden service will be launched. (Default: 0) - -[[CloseHSServiceRendCircuitsImmediatelyOnTimeout]] **CloseHSServiceRendCircuitsImmediatelyOnTimeout** **0**|**1**:: - If 1, Tor will close unfinished hidden-service-side rendezvous - circuits after the current circuit-build timeout. Otherwise, such - circuits will be left open, in the hope that they will finish - connecting to their destinations. In either case, another - rendezvous circuit for the same destination client will be - launched. (Default: 0) - [[LongLivedPorts]] **LongLivedPorts** __PORTS__:: A list of ports for services that tend to have long-running connections (e.g. chat and interactive shells). Circuits for streams that use these @@ -1007,7 +1109,8 @@ The following options are useful only for clients (that is, if but never attach a new stream to a circuit that is too old. For hidden services, this applies to the __last__ time a circuit was used, not the first. Circuits with streams constructed with SOCKS authentication via - SocksPorts that have **KeepAliveIsolateSOCKSAuth** ignore this value. + SocksPorts that have **KeepAliveIsolateSOCKSAuth** also remain alive + for MaxCircuitDirtiness seconds after carrying the last such stream. (Default: 10 minutes) [[MaxClientCircuitsPending]] **MaxClientCircuitsPending** __NUM__:: @@ -1056,7 +1159,9 @@ The following options are useful only for clients (that is, if Unsupported and force-disabled when using Unix domain sockets.) **IsolateSOCKSAuth**;; Don't share circuits with streams for which different - SOCKS authentication was provided. (On by default; + SOCKS authentication was provided. (For HTTPTunnelPort + connections, this option looks at the Proxy-Authorization and + X-Tor-Stream-Isolation headers. On by default; you can disable it with **NoIsolateSOCKSAuth**.) **IsolateClientProtocol**;; Don't share circuits with streams using a different protocol. @@ -1069,8 +1174,9 @@ The following options are useful only for clients (that is, if Don't share circuits with streams targeting a different destination address. **KeepAliveIsolateSOCKSAuth**;; - If **IsolateSOCKSAuth** is enabled, keep alive circuits that have - streams with SOCKS authentication set indefinitely. + If **IsolateSOCKSAuth** is enabled, keep alive circuits while they have + at least one stream with SOCKS authentication active. After such a circuit + is idle for more than MaxCircuitDirtiness seconds, it can be closed. **SessionGroup=**__INT__;; If no other isolation rules would prevent it, allow streams on this port to share circuits with streams from every other @@ -1142,20 +1248,11 @@ The following options are useful only for clients (that is, if authentication" when IsolateSOCKSAuth is disabled, or when this option is set. +[[SocksPortFlagsMisc]]:: Flags are processed left to right. If flags conflict, the last flag on the line is used, and all earlier flags are ignored. No error is issued for conflicting flags. -[[SocksListenAddress]] **SocksListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for connections from Socks-speaking - applications. (Default: 127.0.0.1) You can also specify a port (e.g. - 192.168.0.1:9100). This directive can be specified multiple times to bind - to multiple addresses/ports. (DEPRECATED: As of 0.2.3.x-alpha, you can - now use multiple SocksPort entries, and provide addresses for SocksPort - entries, so SocksListenAddress no longer has a purpose. For backward - compatibility, SocksListenAddress is only allowed when SocksPort is just - a port number.) - [[SocksPolicy]] **SocksPolicy** __policy__,__policy__,__...__:: Set an entrance policy for this server, to limit who can connect to the SocksPort and DNSPort ports. The policies have the same form as exit @@ -1172,7 +1269,8 @@ The following options are useful only for clients (that is, if NUM must be between 1 and 1000, inclusive. Note that the configured bandwidth limits are still expressed in bytes per second: this option only affects the frequency with which Tor checks to see whether - previously exhausted connections may read again. (Default: 100 msec) + previously exhausted connections may read again. + Can not be changed while tor is running. (Default: 100 msec) [[TrackHostExits]] **TrackHostExits** __host__,__.domain__,__...__:: For each value in the comma separated list, Tor will track recent @@ -1209,15 +1307,6 @@ The following options are useful only for clients (that is, if Authorities, Single Onion Services, and Tor2web clients. In these cases, the this option is ignored. (Default: 1) -[[UseEntryGuardsAsDirGuards]] **UseEntryGuardsAsDirGuards** **0**|**1**:: - If this option is set to 1, and UseEntryGuards is also set to 1, - we try to use our entry guards as directory - guards, and failing that, pick more nodes to act as our directory guards. - This helps prevent an adversary from enumerating clients. It's only - available for clients (non-relay, non-bridge) that aren't configured to - download any non-default directory material. It doesn't currently - do anything when we lack a live consensus. (Default: 1) - [[GuardfractionFile]] **GuardfractionFile** __FILENAME__:: V3 authoritative directories only. Configures the location of the guardfraction file which contains information about how long relays @@ -1231,16 +1320,15 @@ The following options are useful only for clients (that is, if [[NumEntryGuards]] **NumEntryGuards** __NUM__:: If UseEntryGuards is set to 1, we will try to pick a total of NUM routers - as long-term entries for our circuits. If NUM is 0, we try to learn - the number from the NumEntryGuards consensus parameter, and default - to 3 if the consensus parameter isn't set. (Default: 0) + as long-term entries for our circuits. If NUM is 0, we try to learn the + number from the guard-n-primary-guards-to-use consensus parameter, and + default to 1 if the consensus parameter isn't set. (Default: 0) [[NumDirectoryGuards]] **NumDirectoryGuards** __NUM__:: - If UseEntryGuardsAsDirectoryGuards is enabled, we try to make sure we - have at least NUM routers to use as directory guards. If this option - is set to 0, use the value from the NumDirectoryGuards consensus - parameter, falling back to the value from NumEntryGuards if the - consensus parameter is 0 or isn't set. (Default: 0) + If UseEntryGuards is set to 1, we try to make sure we have at least NUM + routers to use as directory guards. If this option is set to 0, use the + value from the guard-n-primary-dir-guards-to-use consensus parameter, and + default to 3 if the consensus parameter isn't set. (Default: 0) [[GuardLifetime]] **GuardLifetime** __N__ **days**|**weeks**|**months**:: If nonzero, and UseEntryGuards is set, minimum time to keep a guard before @@ -1262,15 +1350,9 @@ The following options are useful only for clients (that is, if helps to determine whether an application using Tor is possibly leaking DNS requests. (Default: 0) -[[WarnUnsafeSocks]] **WarnUnsafeSocks** **0**|**1**:: - When this option is enabled, Tor will warn whenever a request is - received that only contains an IP address instead of a hostname. Allowing - applications to do DNS resolves themselves is usually a bad idea and - can leak your location to attackers. (Default: 1) +[[VirtualAddrNetworkIPv4]] **VirtualAddrNetworkIPv4** __IPv4Address__/__bits__ + -[[VirtualAddrNetworkIPv4]] **VirtualAddrNetworkIPv4** __Address__/__bits__ + - -[[VirtualAddrNetworkIPv6]] **VirtualAddrNetworkIPv6** [__Address__]/__bits__:: +[[VirtualAddrNetworkIPv6]] **VirtualAddrNetworkIPv6** [__IPv6Address__]/__bits__:: When Tor needs to assign a virtual (unused) address because of a MAPADDRESS command from the controller or the AutomapHostsOnResolve feature, Tor picks an unassigned address from this range. (Defaults: @@ -1293,23 +1375,13 @@ The following options are useful only for clients (that is, if resolved. This helps trap accidental attempts to resolve URLs and so on. (Default: 0) -[[AllowDotExit]] **AllowDotExit** **0**|**1**:: - If enabled, we convert "www.google.com.foo.exit" addresses on the - SocksPort/TransPort/NATDPort into "www.google.com" addresses that exit from - the node "foo". Disabled by default since attacking websites and exit - relays can use it to manipulate your path selection. (Default: 0) - -[[FastFirstHopPK]] **FastFirstHopPK** **0**|**1**|**auto**:: - When this option is disabled, Tor uses the public key step for the first - hop of creating circuits. Skipping it is generally safe since we have - already used TLS to authenticate the relay and to establish forward-secure - keys. Turning this option off makes circuit building a little - slower. Setting this option to "auto" takes advice from the authorities - in the latest consensus about whether to use this feature. + - + - Note that Tor will always use the public key step for the first hop if it's - operating as a relay, and it will never use the public key step if it - doesn't yet know the onion key of the first hop. (Default: auto) +[[HTTPTunnelPort]] **HTTPTunnelPort** \['address':]__port__|**auto** [_isolation flags_]:: + Open this port to listen for proxy connections using the "HTTP CONNECT" + protocol instead of SOCKS. Set this to 0 + 0 if you don't want to allow "HTTP CONNECT" connections. Set the port + to "auto" to have Tor pick a port for you. This directive can be + specified multiple times to bind to multiple addresses/ports. See + SOCKSPort for an explanation of isolation flags. (Default: 0) [[TransPort]] **TransPort** \['address':]__port__|**auto** [_isolation flags_]:: Open this port to listen for transparent proxy connections. Set this to @@ -1321,41 +1393,29 @@ The following options are useful only for clients (that is, if TransPort requires OS support for transparent proxies, such as BSDs' pf or Linux's IPTables. If you're planning to use Tor as a transparent proxy for a network, you'll want to examine and change VirtualAddrNetwork from the - default setting. You'll also want to set the TransListenAddress option for - the network you'd like to proxy. (Default: 0) - -[[TransListenAddress]] **TransListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for transparent proxy connections. (Default: - 127.0.0.1). This is useful for exporting a transparent proxy server to an - entire network. (DEPRECATED: As of 0.2.3.x-alpha, you can - now use multiple TransPort entries, and provide addresses for TransPort - entries, so TransListenAddress no longer has a purpose. For backward - compatibility, TransListenAddress is only allowed when TransPort is just - a port number.) + default setting. (Default: 0) [[TransProxyType]] **TransProxyType** **default**|**TPROXY**|**ipfw**|**pf-divert**:: TransProxyType may only be enabled when there is transparent proxy listener - enabled. + enabled. + + Set this to "TPROXY" if you wish to be able to use the TPROXY Linux module to transparently proxy connections that are configured using the TransPort - option. This setting lets the listener on the TransPort accept connections - for all addresses, even when the TransListenAddress is configured for an - internal address. Detailed information on how to configure the TPROXY + option. Detailed information on how to configure the TPROXY feature can be found in the Linux kernel source tree in the file - Documentation/networking/tproxy.txt. + Documentation/networking/tproxy.txt. + + - Set this option to "ipfw" to use the FreeBSD ipfw interface. + Set this option to "ipfw" to use the FreeBSD ipfw interface. + + On *BSD operating systems when using pf, set this to "pf-divert" to take advantage of +divert-to+ rules, which do not modify the packets like +rdr-to+ rules do. Detailed information on how to configure pf to use +divert-to+ rules can be found in the pf.conf(5) manual page. On OpenBSD, +divert-to+ is available to use on versions greater than or equal to - OpenBSD 4.4. + OpenBSD 4.4. + + Set this to "default", or leave it unconfigured, to use regular IPTables - on Linux, or to use pf +rdr-to+ rules on *BSD systems. + on Linux, or to use pf +rdr-to+ rules on *BSD systems. + + (Default: "default".) @@ -1369,13 +1429,6 @@ The following options are useful only for clients (that is, if + This option is only for people who cannot use TransPort. (Default: 0) -[[NATDListenAddress]] **NATDListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for NATD connections. (DEPRECATED: As of - 0.2.3.x-alpha, you can now use multiple NATDPort entries, and provide - addresses for NATDPort entries, so NATDListenAddress no longer has a - purpose. For backward compatibility, NATDListenAddress is only allowed - when NATDPort is just a port number.) - [[AutomapHostsOnResolve]] **AutomapHostsOnResolve** **0**|**1**:: When this option is enabled, and we get a request to resolve an address that ends with one of the suffixes in **AutomapHostsSuffixes**, we map an @@ -1396,24 +1449,18 @@ The following options are useful only for clients (that is, if addresses/ports. See SocksPort for an explanation of isolation flags. (Default: 0) -[[DNSListenAddress]] **DNSListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for DNS connections. (DEPRECATED: As of - 0.2.3.x-alpha, you can now use multiple DNSPort entries, and provide - addresses for DNSPort entries, so DNSListenAddress no longer has a - purpose. For backward compatibility, DNSListenAddress is only allowed - when DNSPort is just a port number.) - [[ClientDNSRejectInternalAddresses]] **ClientDNSRejectInternalAddresses** **0**|**1**:: If true, Tor does not believe any anonymously retrieved DNS answer that tells it that an address resolves to an internal address (like 127.0.0.1 or - 192.168.0.1). This option prevents certain browser-based attacks; don't - turn it off unless you know what you're doing. (Default: 1) + 192.168.0.1). This option prevents certain browser-based attacks; it + is not allowed to be set on the default network. (Default: 1) [[ClientRejectInternalAddresses]] **ClientRejectInternalAddresses** **0**|**1**:: If true, Tor does not try to fulfill requests to connect to an internal - address (like 127.0.0.1 or 192.168.0.1) __unless a exit node is + address (like 127.0.0.1 or 192.168.0.1) __unless an exit node is specifically requested__ (for example, via a .exit hostname, or a - controller request). (Default: 1) + controller request). If true, multicast DNS hostnames for machines on the + local network (of the form *.local) are also rejected. (Default: 1) [[DownloadExtraInfo]] **DownloadExtraInfo** **0**|**1**:: If true, Tor downloads and caches "extra-info" documents. These documents @@ -1431,11 +1478,6 @@ The following options are useful only for clients (that is, if Like WarnPlaintextPorts, but instead of warning about risky port uses, Tor will instead refuse to make the connection. (Default: None) -[[AllowSingleHopCircuits]] **AllowSingleHopCircuits** **0**|**1**:: - When this option is set, the attached Tor controller can use relays - that have the **AllowSingleHopExits** option turned on to build - one-hop Tor connections. (Default: 0) - [[OptimisticData]] **OptimisticData** **0**|**1**|**auto**:: When this option is set, and Tor is using an exit node that supports the feature, it will try optimistically to send data to the exit node @@ -1462,11 +1504,11 @@ The following options are useful only for clients (that is, if (Example: Tor2webRendezvousPoints Fastyfasty, ABCD1234CDEF5678ABCD1234CDEF5678ABCD1234, \{cc}, 255.254.0.0/8) + + - This feature can only be used if Tor2webMode is also enabled. + This feature can only be used if Tor2webMode is also enabled. + + ExcludeNodes have higher priority than Tor2webRendezvousPoints, which means that nodes specified in ExcludeNodes will not be - picked as RPs. + picked as RPs. + + If no nodes in Tor2webRendezvousPoints are currently available for use, Tor will choose a random node when building HS circuits. @@ -1494,7 +1536,7 @@ The following options are useful only for clients (that is, if These options override the default behavior of Tor's (**currently experimental**) path bias detection algorithm. To try to find broken or misbehaving guard nodes, Tor looks for nodes where more than a certain - fraction of circuits through that guard fail to get built. + fraction of circuits through that guard fail to get built. + + The PathBiasCircThreshold option controls how many circuits we need to build through a guard before we make these checks. The PathBiasNoticeRate, @@ -1520,14 +1562,14 @@ The following options are useful only for clients (that is, if [[PathBiasScaleUseThreshold]] **PathBiasScaleUseThreshold** __NUM__:: Similar to the above options, these options override the default behavior - of Tor's (**currently experimental**) path use bias detection algorithm. + of Tor's (**currently experimental**) path use bias detection algorithm. + + Where as the path bias parameters govern thresholds for successfully building circuits, these four path use bias parameters govern thresholds only for circuit usage. Circuits which receive no stream usage are not counted by this detection algorithm. A used circuit is considered successful if it is capable of carrying streams or otherwise receiving - well-formed responses to RELAY cells. + well-formed responses to RELAY cells. + + By default, or if a negative value is provided for one of these options, Tor uses reasonable defaults from the networkstatus consensus document. @@ -1552,7 +1594,8 @@ The following options are useful only for clients (that is, if server has both. (Tor also prefers an IPv6 DirPort if IPv4Client is set to 0.) If this option is set to auto, clients prefer IPv4. Other things may influence the choice. This option breaks a tie to the favor of IPv6. - (Default: auto) + (Default: auto) (DEPRECATED: This option has had no effect for some + time.) [[ClientPreferIPv6ORPort]] **ClientPreferIPv6ORPort** **0**|**1**|**auto**:: If this option is set to 1, Tor prefers an OR port with an IPv6 @@ -1581,8 +1624,8 @@ The following options are useful only for clients (that is, if live consensus). Only used by clients fetching from a list of fallback directory mirrors. This schedule is advanced by (potentially concurrent) connection attempts, unlike other schedules, which are advanced by - connection failures. (Default: 10, 11, 3600, 10800, 25200, 54000, - 111600, 262800) + connection failures. (Default: 6, 11, 3600, 10800, 25200, 54000, 111600, + 262800) [[ClientBootstrapConsensusFallbackDownloadSchedule]] **ClientBootstrapConsensusFallbackDownloadSchedule** __N__,__N__,__...__:: Schedule for when clients should download consensuses from fallback @@ -1612,7 +1655,7 @@ The following options are useful only for clients (that is, if [[ClientBootstrapConsensusMaxInProgressTries]] **ClientBootstrapConsensusMaxInProgressTries** __NUM__:: Try this many simultaneous connections to download a consensus before - waiting for one to complete, timeout, or error out. (Default: 4) + waiting for one to complete, timeout, or error out. (Default: 3) SERVER OPTIONS -------------- @@ -1621,19 +1664,13 @@ The following options are useful only for servers (that is, if ORPort is non-zero): [[Address]] **Address** __address__:: - The IP address or fully qualified domain name of this server (e.g. - moria.mit.edu). You can leave this unset, and Tor will guess your IP - address. This IP address is the one used to tell clients and other - servers where to find your Tor server; it doesn't affect the IP that your - Tor client binds to. To bind to a different address, use the - *ListenAddress and OutboundBindAddress options. - -[[AllowSingleHopExits]] **AllowSingleHopExits** **0**|**1**:: - This option controls whether clients can use this server as a single hop - proxy. If set to 1, clients can use this server as an exit even if it is - the only hop in the circuit. Note that most clients will refuse to use - servers that set this option, since most clients have - ExcludeSingleHopRelays set. (Default: 0) + The IPv4 address of this server, or a fully qualified domain name of + this server that resolves to an IPv4 address. You can leave this + unset, and Tor will try to guess your IPv4 address. This IPv4 + address is the one used to tell clients and other servers where to + find your Tor server; it doesn't affect the address that your server + binds to. To bind to a different address, use the ORPort and + OutboundBindAddress options. [[AssumeReachable]] **AssumeReachable** **0**|**1**:: This option is used when bootstrapping a new Tor network. If set to 1, @@ -1648,6 +1685,17 @@ is non-zero): server descriptor to the bridge database, rather than to the public directory authorities. +[[BridgeDistribution]] **BridgeDistribution** __string__:: + If set along with BridgeRelay, Tor will include a new line in its + bridge descriptor which indicates to the BridgeDB service how it + would like its bridge address to be given out. Set it to "none" if + you want BridgeDB to avoid distributing your bridge address, or "any" to + let BridgeDB decide. (Default: any) + + + Note: as of Oct 2017, the BridgeDB part of this option is not yet + implemented. Until BridgeDB is updated to obey this option, your + bridge will make this request, but it will not (yet) be obeyed. + [[ContactInfo]] **ContactInfo** __email_address__:: Administrative contact information for this relay or bridge. This line can be used to contact you if your relay or bridge is misconfigured or @@ -1655,13 +1703,18 @@ is non-zero): descriptors containing these lines and that Google indexes them, so spammers might also collect them. You may want to obscure the fact that it's an email address and/or generate a new address for this - purpose. + purpose. + + + + ContactInfo **must** be set to a working address if you run more than one + relay or bridge. (Really, everybody running a relay or bridge should set + it.) + [[ExitRelay]] **ExitRelay** **0**|**1**|**auto**:: Tells Tor whether to run as an exit relay. If Tor is running as a non-bridge server, and ExitRelay is set to 1, then Tor allows traffic to exit according to the ExitPolicy option (or the default ExitPolicy if - none is specified). + none is specified). + + If ExitRelay is set to 0, no traffic is allowed to exit, and the ExitPolicy option is ignored. + @@ -1739,6 +1792,7 @@ is non-zero): reject *:6881-6999 accept *:* +[[ExitPolicyDefault]]:: Since the default exit policy uses accept/reject *, it applies to both IPv4 and IPv6 addresses. @@ -1768,17 +1822,24 @@ is non-zero): If we have more onionskins queued for processing than we can process in this amount of time, reject new ones. (Default: 1750 msec) -[[MyFamily]] **MyFamily** __node__,__node__,__...__:: - Declare that this Tor server is controlled or administered by a group or - organization identical or similar to that of the other servers, defined by - their identity fingerprints. When two servers both declare - that they are in the same \'family', Tor clients will not use them in the - same circuit. (Each server only needs to list the other servers in its - family; it doesn't need to list itself, but it won't hurt.) Do not list - any bridge relay as it would compromise its concealment. +[[MyFamily]] **MyFamily** __fingerprint__,__fingerprint__,...:: + Declare that this Tor relay is controlled or administered by a group or + organization identical or similar to that of the other relays, defined by + their (possibly $-prefixed) identity fingerprints. + This option can be repeated many times, for + convenience in defining large families: all fingerprints in all MyFamily + lines are merged into one list. + When two relays both declare that they are in the + same \'family', Tor clients will not use them in the same circuit. (Each + relay only needs to list the other servers in its family; it doesn't need to + list itself, but it won't hurt if it does.) Do not list any bridge relay as it would + compromise its concealment. + + When listing a node, it's better to list it by fingerprint than by - nickname: fingerprints are more reliable. + nickname: fingerprints are more reliable. + + + + If you run more than one relay, the MyFamily option on each relay + **must** list all other relays, as described above. [[Nickname]] **Nickname** __name__:: Set the server's nickname to \'name'. Nicknames must be between 1 and 19 @@ -1793,38 +1854,30 @@ is non-zero): Advertise this port to listen for connections from Tor clients and servers. This option is required to be a Tor server. Set it to "auto" to have Tor pick a port for you. Set it to 0 to not - run an ORPort at all. This option can occur more than once. (Default: 0) -+ + run an ORPort at all. This option can occur more than once. (Default: 0) + + + Tor recognizes these flags on each ORPort: - **NoAdvertise**:: + **NoAdvertise**;; By default, we bind to a port and tell our users about it. If NoAdvertise is specified, we don't advertise, but listen anyway. This can be useful if the port everybody will be connecting to (for example, one that's opened on our firewall) is somewhere else. - **NoListen**:: + **NoListen**;; By default, we bind to a port and tell our users about it. If NoListen is specified, we don't bind, but advertise anyway. This can be useful if something else (for example, a firewall's port forwarding configuration) is causing connections to reach us. - **IPv4Only**:: + **IPv4Only**;; If the address is absent, or resolves to both an IPv4 and an IPv6 address, only listen to the IPv4 address. - **IPv6Only**:: + **IPv6Only**;; If the address is absent, or resolves to both an IPv4 and an IPv6 address, only listen to the IPv6 address. -+ + +[[ORPortFlagsExclusive]]:: For obvious reasons, NoAdvertise and NoListen are mutually exclusive, and IPv4Only and IPv6Only are mutually exclusive. -[[ORListenAddress]] **ORListenAddress** __IP__[:__PORT__]:: - Bind to this IP address to listen for connections from Tor clients and - servers. If you specify a port, bind to this port rather than the one - specified in ORPort. (Default: 0.0.0.0) This directive can be specified - multiple times to bind to multiple addresses/ports. -+ - This option is deprecated; you can get the same behavior with ORPort now - that it supports NoAdvertise and explicit addresses. - [[PortForwarding]] **PortForwarding** **0**|**1**:: Attempt to automatically forward the DirPort and ORPort on a NAT router connecting this Tor server to the Internet. If set, Tor will try both @@ -1840,15 +1893,17 @@ is non-zero): [[PublishServerDescriptor]] **PublishServerDescriptor** **0**|**1**|**v3**|**bridge**,**...**:: This option specifies which descriptors Tor will publish when acting as a relay. You can - choose multiple arguments, separated by commas. + choose multiple arguments, separated by commas. + + If this option is set to 0, Tor will not publish its descriptors to any directories. (This is useful if you're testing - out your server, or if you're using a Tor controller that handles directory - publishing for you.) Otherwise, Tor will publish its descriptors of all - type(s) specified. The default is "1", - which means "if running as a server, publish the - appropriate descriptors to the authorities". + out your server, or if you're using a Tor controller that handles + directory publishing for you.) Otherwise, Tor will publish its + descriptors of all type(s) specified. The default is "1", which + means "if running as a relay or bridge, publish descriptors to the + appropriate authorities". Other possibilities are "v3", meaning + "publish as if you're a relay", and "bridge", meaning "publish as + if you're a bridge". [[ShutdownWaitLength]] **ShutdownWaitLength** __NUM__:: When we get a SIGINT and we're a server, we begin shutting down: @@ -1868,7 +1923,7 @@ is non-zero): to 0 will disable the heartbeat. Otherwise, it must be at least 30 minutes. (Default: 6 hours) -[[AccountingMax]] **AccountingMax** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**|**TBytes**:: +[[AccountingMax]] **AccountingMax** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Limits the max number of bytes sent and received within a set time period using a given calculation rule (see: AccountingStart, AccountingRule). Useful if you need to stay under a specific bandwidth. By default, the @@ -1942,7 +1997,7 @@ is non-zero): correct this. This option only affects name lookups that your server does on behalf of clients. (Default: 1) -[[ServerDNSTestAddresses]] **ServerDNSTestAddresses** __address__,__address__,__...__:: +[[ServerDNSTestAddresses]] **ServerDNSTestAddresses** __hostname__,__hostname__,__...__:: When we're detecting DNS hijacking, make sure that these __valid__ addresses aren't getting redirected. If they are, then our DNS is completely useless, and we'll reset our exit policy to "reject \*:*". This option only affects @@ -1976,12 +2031,6 @@ is non-zero): [[GeoIPv6File]] **GeoIPv6File** __filename__:: A filename containing IPv6 GeoIP data, for use with by-country statistics. -[[TLSECGroup]] **TLSECGroup** **P224**|**P256**:: - What EC group should we try to use for incoming TLS connections? - P224 is faster, but makes us stand out more. Has no effect if - we're a client, or if our OpenSSL version lacks support for ECDHE. - (Default: P256) - [[CellStatistics]] **CellStatistics** **0**|**1**:: Relays only. When this option is enabled, Tor collects statistics about cell @@ -1992,6 +2041,14 @@ is non-zero): If ExtraInfoStatistics is enabled, it will published as part of extra-info document. (Default: 0) +[[PaddingStatistics]] **PaddingStatistics** **0**|**1**:: + Relays only. + When this option is enabled, Tor collects statistics for padding cells + sent and received by this relay, in addition to total cell counts. + These statistics are rounded, and omitted if traffic is low. This + information is important for load balancing decisions related to padding. + (Default: 1) + [[DirReqStatistics]] **DirReqStatistics** **0**|**1**:: Relays and bridges only. When this option is enabled, a Tor directory writes statistics on the @@ -2082,8 +2139,9 @@ is non-zero): DIRECTORY SERVER OPTIONS ------------------------ -The following options are useful only for directory servers (that is, -if DirPort is non-zero): +The following options are useful only for directory servers. (Relays with +enough bandwidth automatically become directory servers; see DirCache for +details.) [[DirPortFrontPage]] **DirPortFrontPage** __FILENAME__:: When this option is set, it takes an HTML file and publishes it as "/" on @@ -2095,19 +2153,10 @@ if DirPort is non-zero): If this option is nonzero, advertise the directory service on this port. Set it to "auto" to have Tor pick a port for you. This option can occur more than once, but only one advertised DirPort is supported: all - but one DirPort must have the **NoAdvertise** flag set. (Default: 0) -+ + but one DirPort must have the **NoAdvertise** flag set. (Default: 0) + + + The same flags are supported here as are supported by ORPort. -[[DirListenAddress]] **DirListenAddress** __IP__[:__PORT__]:: - Bind the directory service to this address. If you specify a port, bind to - this port rather than the one specified in DirPort. (Default: 0.0.0.0) - This directive can be specified multiple times to bind to multiple - addresses/ports. -+ - This option is deprecated; you can get the same behavior with DirPort now - that it supports NoAdvertise and explicit addresses. - [[DirPolicy]] **DirPolicy** __policy__,__policy__,__...__:: Set an entrance policy for this server, to limit who can connect to the directory ports. The policies have the same form as exit policies above, @@ -2120,6 +2169,16 @@ if DirPort is non-zero): because clients connect via the ORPort by default. Setting either DirPort or BridgeRelay and setting DirCache to 0 is not supported. (Default: 1) +[[MaxConsensusAgeForDiffs]] **MaxConsensusAgeForDiffs** __N__ **minutes**|**hours**|**days**|**weeks**:: + When this option is nonzero, Tor caches will not try to generate + consensus diffs for any consensus older than this amount of time. + If this option is set to zero, Tor will pick a reasonable default from + the current networkstatus document. You should not set this + option unless your cache is severely low on disk space or CPU. + If you need to set it, keeping it above 3 or 4 hours will help clients + much more than setting it to zero. + (Default: 0) + DIRECTORY AUTHORITY SERVER OPTIONS ---------------------------------- @@ -2199,7 +2258,7 @@ on the public Tor network. [[AuthDirBadExit]] **AuthDirBadExit** __AddressPattern...__:: Authoritative directories only. A set of address patterns for servers that will be listed as bad exits in any network status document this authority - publishes, if **AuthDirListBadExits** is set. + publishes, if **AuthDirListBadExits** is set. + + (The address pattern syntax here and in the options below is the same as for exit policies, except that you don't need to say @@ -2237,26 +2296,22 @@ on the public Tor network. list as acceptable on a single IP address. Set this to "0" for "no limit". (Default: 2) -[[AuthDirMaxServersPerAuthAddr]] **AuthDirMaxServersPerAuthAddr** __NUM__:: - Authoritative directories only. Like AuthDirMaxServersPerAddr, but applies - to addresses shared with directory authorities. (Default: 5) - -[[AuthDirFastGuarantee]] **AuthDirFastGuarantee** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[AuthDirFastGuarantee]] **AuthDirFastGuarantee** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Authoritative directories only. If non-zero, always vote the Fast flag for any relay advertising this amount of capacity or more. (Default: 100 KBytes) -[[AuthDirGuardBWGuarantee]] **AuthDirGuardBWGuarantee** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[AuthDirGuardBWGuarantee]] **AuthDirGuardBWGuarantee** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Authoritative directories only. If non-zero, this advertised capacity or more is always sufficient to satisfy the bandwidth requirement - for the Guard flag. (Default: 250 KBytes) + for the Guard flag. (Default: 2 MBytes) [[AuthDirPinKeys]] **AuthDirPinKeys** **0**|**1**:: Authoritative directories only. If non-zero, do not allow any relay to publish a descriptor if any other relay has reserved its <Ed25519,RSA> identity keypair. In all cases, Tor records every keypair it accepts in a journal if it is new, or if it differs from the most recently - accepted pinning for one of the keys it contains. (Default: 0) + accepted pinning for one of the keys it contains. (Default: 1) [[AuthDirSharedRandomness]] **AuthDirSharedRandomness** **0**|**1**:: Authoritative directories only. Switch for the shared random protocol. @@ -2264,6 +2319,13 @@ on the public Tor network. (default), the flag "shared-rand-participate" is added to the authority vote indicating participation in the protocol. (Default: 1) +[[AuthDirTestEd25519LinkKeys]] **AuthDirTestEd25519LinkKeys** **0**|**1**:: + Authoritative directories only. If this option is set to 0, then we treat + relays as "Running" if their RSA key is correct when we probe them, + regardless of their Ed25519 key. We should only ever set this option to 0 + if there is some major bug in Ed25519 link authentication that causes us + to label all the relays as not Running. (Default: 1) + [[BridgePassword]] **BridgePassword** __Password__:: If set, contains an HTTP authenticator that tells a bridge authority to serve all requested bridge information. Used by the (only partially @@ -2335,9 +2397,9 @@ The following options are used to configure a hidden service. [[HiddenServiceDir]] **HiddenServiceDir** __DIRECTORY__:: Store data files for a hidden service in DIRECTORY. Every hidden service must have a separate directory. You may use this option multiple times to - specify multiple services. DIRECTORY must be an existing directory. + specify multiple services. If DIRECTORY does not exist, Tor will create it. (Note: in current versions of Tor, if DIRECTORY is a relative path, - it will be relative to current + it will be relative to the current working directory of Tor instance, not to its DataDirectory. Do not rely on this behavior; it is not guaranteed to remain the same in future versions.) @@ -2362,7 +2424,7 @@ The following options are used to configure a hidden service. [[HiddenServiceVersion]] **HiddenServiceVersion** __version__,__version__,__...__:: A list of rendezvous service descriptor versions to publish for the hidden - service. Currently, only version 2 is supported. (Default: 2) + service. Currently, versions 2 and 3 are supported. (Default: 2) [[HiddenServiceAuthorizeClient]] **HiddenServiceAuthorizeClient** __auth-type__ __client-name__,__client-name__,__...__:: If configured, the hidden service is accessible for authorized clients @@ -2384,8 +2446,8 @@ The following options are used to configure a hidden service. [[HiddenServiceMaxStreams]] **HiddenServiceMaxStreams** __N__:: The maximum number of simultaneous streams (connections) per rendezvous - circuit. (Setting this to 0 will allow an unlimited number of simultanous - streams.) (Default: 0) + circuit. The maximum value allowed is 65535. (Setting this to 0 will allow + an unlimited number of simultanous streams.) (Default: 0) [[HiddenServiceMaxStreamsCloseCircuit]] **HiddenServiceMaxStreamsCloseCircuit** **0**|**1**:: If set to 1, then exceeding **HiddenServiceMaxStreams** will cause the @@ -2394,8 +2456,9 @@ The following options are used to configure a hidden service. [[RendPostPeriod]] **RendPostPeriod** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: Every time the specified period elapses, Tor uploads any rendezvous - service descriptors to the directory servers. This information is also - uploaded whenever it changes. (Default: 1 hour) + service descriptors to the directory servers. This information is also + uploaded whenever it changes. Minimum value allowed is 10 minutes and + maximum is 3.5 days. (Default: 1 hour) [[HiddenServiceDirGroupReadable]] **HiddenServiceDirGroupReadable** **0**|**1**:: If this option is set to 1, allow the filesystem group to read the @@ -2405,7 +2468,7 @@ The following options are used to configure a hidden service. [[HiddenServiceNumIntroductionPoints]] **HiddenServiceNumIntroductionPoints** __NUM__:: Number of introduction points the hidden service will have. You can't - have more than 10. (Default: 3) + have more than 10 for v2 service and 20 for v3. (Default: 3) [[HiddenServiceSingleHopMode]] **HiddenServiceSingleHopMode** **0**|**1**:: **Experimental - Non Anonymous** Hidden Services on a tor instance in @@ -2417,20 +2480,20 @@ The following options are used to configure a hidden service. Single Onion Service. One-hop circuits make Single Onion servers easily locatable, but clients remain location-anonymous. However, the fact that a client is accessing a Single Onion rather than a Hidden Service may be - statistically distinguishable. - + statistically distinguishable. + + + **WARNING:** Once a hidden service directory has been used by a tor instance in HiddenServiceSingleHopMode, it can **NEVER** be used again for a hidden service. It is best practice to create a new hidden service directory, key, and address for each new Single Onion Service and Hidden Service. It is not possible to run Single Onion Services and Hidden Services from the same tor instance: they should be run on different - servers with different IP addresses. - + servers with different IP addresses. + + + HiddenServiceSingleHopMode requires HiddenServiceNonAnonymousMode to be set to 1. Since a Single Onion service is non-anonymous, you can not configure a SOCKSPort on a tor instance that is running in - **HiddenServiceSingleHopMode**. + **HiddenServiceSingleHopMode**. Can not be changed while tor is running. (Default: 0) [[HiddenServiceNonAnonymousMode]] **HiddenServiceNonAnonymousMode** **0**|**1**:: @@ -2438,8 +2501,8 @@ The following options are used to configure a hidden service. non-anonymous HiddenServiceSingleHopMode. Enables direct connections in the server-side hidden service protocol. If you are using this option, you need to disable all client-side services on your Tor instance, - including setting SOCKSPort to "0". - (Default: 0) + including setting SOCKSPort to "0". Can not be changed while tor is + running. (Default: 0) DENIAL OF SERVICE MITIGATION OPTIONS ------------------------------------ @@ -2580,7 +2643,8 @@ The following options are used for running a testing Tor network. TestingClientDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60 TestingServerConsensusDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60 TestingClientConsensusDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60 - TestingBridgeDownloadSchedule 60, 30, 30, 60 + TestingBridgeDownloadSchedule 10, 30, 60 + TestingBridgeBootstrapDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60 TestingClientMaxIntervalWithoutRequest 5 seconds TestingDirConnectionMaxStall 30 seconds TestingConsensusMaxDownloadTries 80 @@ -2620,7 +2684,7 @@ The following options are used for running a testing Tor network. time. Changing this requires that **TestingTorNetwork** is set. (Default: 10 minutes) -[[TestingMinFastFlagThreshold]] **TestingMinFastFlagThreshold** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[TestingMinFastFlagThreshold]] **TestingMinFastFlagThreshold** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Minimum value for the Fast flag. Overrides the ordinary minimum taken from the consensus when TestingTorNetwork is set. (Default: 0.) @@ -2645,8 +2709,16 @@ The following options are used for running a testing Tor network. 1800, 3600, 3600, 3600, 10800, 21600, 43200) [[TestingBridgeDownloadSchedule]] **TestingBridgeDownloadSchedule** __N__,__N__,__...__:: - Schedule for when clients should download bridge descriptors. Changing this - requires that **TestingTorNetwork** is set. (Default: 3600, 900, 900, 3600) + Schedule for when clients should download each bridge descriptor when they + know that one or more of their configured bridges are running. Changing + this requires that **TestingTorNetwork** is set. (Default: 10800, 25200, + 54000, 111600, 262800) + +[[TestingBridgeBootstrapDownloadSchedule]] **TestingBridgeBootstrapDownloadSchedule** __N__,__N__,__...__:: + Schedule for when clients should download each bridge descriptor when they + have just started, or when they can not contact any of their bridges. + Changing this requires that **TestingTorNetwork** is set. (Default: 0, 30, + 90, 600, 3600, 10800, 25200, 54000, 111600, 262800) [[TestingClientMaxIntervalWithoutRequest]] **TestingClientMaxIntervalWithoutRequest** __N__ **seconds**|**minutes**:: When directory clients have only a few descriptors to request, they batch @@ -2679,7 +2751,7 @@ The following options are used for running a testing Tor network. A list of identity fingerprints, country codes, and address patterns of nodes to vote Exit for regardless of their uptime, bandwidth, or exit policy. See the **ExcludeNodes** - option for more information on how to specify nodes. + option for more information on how to specify nodes. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. See the **ExcludeNodes** option for more @@ -2688,7 +2760,7 @@ The following options are used for running a testing Tor network. [[TestingDirAuthVoteExitIsStrict]] **TestingDirAuthVoteExitIsStrict** **0**|**1** :: If True (1), a node will never receive the Exit flag unless it is specified in the **TestingDirAuthVoteExit** list, regardless of its uptime, bandwidth, - or exit policy. + or exit policy. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. @@ -2697,14 +2769,14 @@ The following options are used for running a testing Tor network. A list of identity fingerprints and country codes and address patterns of nodes to vote Guard for regardless of their uptime and bandwidth. See the **ExcludeNodes** option for more - information on how to specify nodes. + information on how to specify nodes. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. [[TestingDirAuthVoteGuardIsStrict]] **TestingDirAuthVoteGuardIsStrict** **0**|**1** :: If True (1), a node will never receive the Guard flag unless it is specified - in the **TestingDirAuthVoteGuard** list, regardless of its uptime and bandwidth. + in the **TestingDirAuthVoteGuard** list, regardless of its uptime and bandwidth. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. @@ -2713,14 +2785,14 @@ The following options are used for running a testing Tor network. A list of identity fingerprints and country codes and address patterns of nodes to vote HSDir for regardless of their uptime and DirPort. See the **ExcludeNodes** option for more - information on how to specify nodes. + information on how to specify nodes. + + In order for this option to have any effect, **TestingTorNetwork** must be set. [[TestingDirAuthVoteHSDirIsStrict]] **TestingDirAuthVoteHSDirIsStrict** **0**|**1** :: If True (1), a node will never receive the HSDir flag unless it is specified - in the **TestingDirAuthVoteHSDir** list, regardless of its uptime and DirPort. + in the **TestingDirAuthVoteHSDir** list, regardless of its uptime and DirPort. + + In order for this option to have any effect, **TestingTorNetwork** has to be set. @@ -2740,7 +2812,7 @@ The following options are used for running a testing Tor network. events. Changing this requires that **TestingTorNetwork** is set. (Default: 0) -[[TestingMinExitFlagThreshold]] **TestingMinExitFlagThreshold** __N__ **KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: +[[TestingMinExitFlagThreshold]] **TestingMinExitFlagThreshold** __N__ **KBytes**|**MBytes**|**GBytes**|**TBytes**|**KBits**|**MBits**|**GBits**|**TBits**:: Sets a lower-bound for assigning an exit flag when running as an authority on a testing network. Overrides the usual default lower bound of 4 KB. (Default: 0) @@ -2764,6 +2836,19 @@ The following options are used for running a testing Tor network. we replace it and issue a new key? (Default: 3 hours for link and auth; 1 day for signing.) +NON-PERSISTENT OPTIONS +---------------------- + +These options are not saved to the torrc file by the "SAVECONF" controller +command. Other options of this type are documented in control-spec.txt, +section 5.4. End-users should mostly ignore them. + +[[UnderscorePorts]] **\_\_ControlPort**, **\_\_DirPort**, **\_\_DNSPort**, **\_\_ExtORPort**, **\_\_NATDPort**, **\_\_ORPort**, **\_\_SocksPort**, **\_\_TransPort**:: + These underscore-prefixed options are variants of the regular Port + options. They behave the same, except they are not saved to the + torrc file by the controller's SAVECONF command. + + SIGNALS ------- @@ -2813,7 +2898,8 @@ FILES __DataDirectory__**/cached-status/**:: The most recently downloaded network status document for each authority. Each file holds one such document; the filenames are the hexadecimal - identity key fingerprints of the directory authorities. Mostly obsolete. + identity key fingerprints of the directory authorities. Obsolete; + no longer in use. __DataDirectory__**/cached-certs**:: This file holds downloaded directory key certificates that are used to @@ -2829,6 +2915,13 @@ __DataDirectory__**/cached-descriptors** and **cached-descriptors.new**:: a given router. The ".new" file is an append-only journal; when it gets too large, all entries are merged into a new cached-descriptors file. +__DataDirectory__**/cached-extrainfo** and **cached-extrainfo.new**:: + As "cached-descriptors", but holds optionally-downloaded "extra-info" + documents. Relays use these documents to send inessential information + about statistics, bandwidth history, and network health to the + authorities. They aren't fetched by default; see the DownloadExtraInfo + option for more info. + __DataDirectory__**/cached-microdescs** and **cached-microdescs.new**:: These files hold downloaded microdescriptors. Lines beginning with @-signs are annotations that contain more information about a given @@ -2843,18 +2936,27 @@ __DataDirectory__**/state**:: A set of persistent key-value mappings. These are documented in the file. These include: - The current entry guards and their status. - - The current bandwidth accounting values (unused so far; see - below). + - The current bandwidth accounting values. - When the file was last written - What version of Tor generated the state file - A short history of bandwidth usage, as produced in the server descriptors. +__DataDirectory__**/sr-state**:: + Authority only. State file used to record information about the current + status of the shared-random-value voting state. + +__DataDirectory__**/diff-cache**:: + Directory cache only. Holds older consensuses, and diffs from older + consensuses to the most recent consensus of each type, compressed + in various ways. Each file contains a set of key-value arguments + decribing its contents, followed by a single NUL byte, followed by the + main file contents. + __DataDirectory__**/bw_accounting**:: Used to track bandwidth accounting values (when the current period starts and ends; how much has been read and written so far this period). This file - is obsolete, and the data is now stored in the \'state' file as well. Only - used when bandwidth accounting is enabled. + is obsolete, and the data is now stored in the \'state' file instead. __DataDirectory__**/control_auth_cookie**:: Used for cookie authentication with the controller. Location can be @@ -2867,6 +2969,13 @@ __DataDirectory__**/lock**:: directory. If access to this file is locked, data directory is already in use by Tor. +__DataDirectory__**/key-pinning-journal**:: + Used by authorities. A line-based file that records mappings between + RSA1024 identity keys and Ed25519 identity keys. Authorities enforce + these mappings, so that once a relay has picked an Ed25519 key, stealing + or factoring the RSA1024 key will no longer let an attacker impersonate + the relay. + __DataDirectory__**/keys/***:: Only used by servers. Holds identity keys and onion keys. @@ -2917,13 +3026,17 @@ __DataDirectory__**/keys/ed25519_signing_cert**:: The certificate which authenticates "ed25519_signing_secret_key" as having been signed by the Ed25519 master key. -__DataDirectory__**/keys/secret_onion_key**:: +__DataDirectory__**/keys/secret_onion_key** and **secret_onion_key.old**:: A relay's RSA1024 short-term onion key. Used to decrypt old-style ("TAP") - circuit extension requests. + circuit extension requests. The ".old" file holds the previously + generated key, which the relay uses to handle any requests that were + made by clients that didn't have the new one. -__DataDirectory__**/keys/secret_onion_key_ntor**:: +__DataDirectory__**/keys/secret_onion_key_ntor** and **secret_onion_key_ntor.old**:: A relay's Curve25519 short-term onion key. Used to handle modern ("ntor") - circuit extension requests. + circuit extension requests. The ".old" file holds the previously + generated key, which the relay uses to handle any requests that were + made by clients that didn't have the new one. __DataDirectory__**/fingerprint**:: Only used by servers. Holds the fingerprint of the server's identity key. @@ -2932,6 +3045,16 @@ __DataDirectory__**/hashed-fingerprint**:: Only used by bridges. Holds the hashed fingerprint of the bridge's identity key. (That is, the hash of the hash of the identity key.) +__DataDirectory__**/approved-routers**:: + Only used by authoritative directory servers. This file lists + the status of routers by their identity fingerprint. + Each line lists a status and a fingerprint separated by + whitespace. See your **fingerprint** file in the __DataDirectory__ for an + example line. If the status is **!reject** then descriptors from the + given identity (fingerprint) are rejected by this server. If it is + **!invalid** then descriptors are accepted but marked in the directory as + not valid, that is, not recommended. + __DataDirectory__**/v3-status-votes**:: Only for v3 authoritative directory servers. This file contains status votes from all the authoritative directory servers. @@ -2978,15 +3101,29 @@ __DataDirectory__**/stats/conn-stats**:: Only used by servers. This file is used to collect approximate connection history (number of active connections over time). +__DataDirectory__**/stats/hidserv-stats**:: + Only used by servers. This file is used to collect approximate counts + of what fraction of the traffic is hidden service rendezvous traffic, and + approximately how many hidden services the relay has seen. + __DataDirectory__**/networkstatus-bridges**:: Only used by authoritative bridge directories. Contains information about bridges that have self-reported themselves to the bridge authority. +__DataDirectory__**/approved-routers**:: + Authorities only. This file is used to configure which relays are + known to be valid, invalid, and so forth. + __HiddenServiceDirectory__**/hostname**:: The <base32-encoded-fingerprint>.onion domain name for this hidden service. If the hidden service is restricted to authorized clients only, this file also contains authorization data for all clients. + + + Note that clients will ignore any extra subdomains prepended to a hidden + service hostname. So if you have "xyz.onion" as your hostname, you + can tell clients to connect to "www.xyz.onion" or "irc.xyz.onion" + for virtual-hosting purposes. __HiddenServiceDirectory__**/private_key**:: The private key for this hidden service. diff --git a/doc/torify.1.txt b/doc/torify.1.txt index 8dca901317..7e49081cfc 100644 --- a/doc/torify.1.txt +++ b/doc/torify.1.txt @@ -17,25 +17,23 @@ SYNOPSIS DESCRIPTION ----------- -**torify** is a simple wrapper that attempts to find the best underlying Tor -wrapper available on a system. It calls torsocks with a tor specific -configuration file. + +**torify** is a simple wrapper that calls torsocks with a tor-specific +configuration file. -torsocks is an improved wrapper that explicitly rejects UDP, safely resolves DNS -lookups and properly socksifies your TCP connections. + - -Please note that since both method use LD_PRELOAD, torify cannot be applied to -suid binaries. +It is provided for backward compatibility; instead you should use torsocks. WARNING ------- -When used with torsocks, torify should not leak DNS requests or UDP data. + +When used with torsocks, torify should not leak DNS requests or UDP data. + +torify can leak ICMP data. -Both will leak ICMP data. +torify will not ensure that different requests are processed on +different circuits. SEE ALSO -------- -**tor**(1), **tor-resolve**(1), **torsocks**(1) +**tor**(1), **torsocks**(1) AUTHORS ------- diff --git a/doc/torrc_format.txt b/doc/torrc_format.txt index 7a809901d9..7a4a92a663 100644 --- a/doc/torrc_format.txt +++ b/doc/torrc_format.txt @@ -18,9 +18,10 @@ does, not what it should do. ; specified in RFC5234. ; A file is interpreted as every Entry in the file, in order. - TorrcFile = *Line + TorrcFile = *Line [ UnterminatedLine ] - Line = BlankLine / Entry + Line = BlankLine LF / Entry LF + UnterminatedLine = BlankLine / Entry BlankLine = *WSP OptComment LF BlankLine =/ *WSP LF @@ -69,6 +70,12 @@ does, not what it should do. ; Anything besides NUL and LF NonLF = %x01-%x09 / %x0b - %xff + ; Note that on windows, we open our configuration files in "text" mode, + ; which causes CRLF pairs to be interpreted as LF. So, on windows: + ; LF = [ %x0d ] %x0a + ; but everywhere else, + LF = %0x0a + OCTDIG = '0' - '7' KC = Any character except an isspace() character or '#' or NUL @@ -175,7 +182,7 @@ and\ friends # Backslashes in the middle of a line are included as-is. The key of -# this one is "Too" and the value is "Many\\Backsl\ashes here" (with +# this one is "Too" and the value is "Many\\Backsl\ashes \here" (with # backslashes in that last string as-is) Too \ Many\\\ @@ -185,7 +192,7 @@ here # And here's the really yucky part. If a comment appears in a multi-line # entry, the entry is still able to continue on the next line, as in the # following, where the key is "This" and the value is -# "entry and some are silly" +# "entry and some are silly" This entry \ # has comments \ and some \ |