diff options
54 files changed, 847 insertions, 171 deletions
@@ -1,3 +1,154 @@ +Changes in version 0.3.3.2-alpha - 2018-02-10 + Tor 0.3.3.2-alpha is the second alpha in the 0.3.3.x series. It + introduces a mechanism to handle the high loads that many relay + operators have been reporting recently. It also fixes several bugs in + older releases. If this new code proves reliable, we plan to backport + it to older supported release series. + + o Major features (denial-of-service mitigation): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Major bugfixes (netflow padding): + - Stop adding unneeded channel padding right after we finish + flushing to a connection that has been trying to flush for many + seconds. Instead, treat all partial or complete flushes as + activity on the channel, which will defer the time until we need + to add padding. This fix should resolve confusing and scary log + messages like "Channel padding timeout scheduled 221453ms in the + past." Fixes bug 22212; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (protocol versions): + - Add Link protocol version 5 to the supported protocols list. Fixes + bug 25070; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (scheduler, consensus): + - The scheduler subsystem was failing to promptly notice changes in + consensus parameters, making it harder to switch schedulers + network-wide. Fixes bug 24975; bugfix on 0.3.2.1-alpha. + + o Minor features (denial-of-service avoidance): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor features (directory authority): + - When directory authorities are unable to add signatures to a + pending consensus, log the reason why. Closes ticket 24849. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor features (logging, diagnostic): + - When logging a failure to create an onion service's descriptor, + also log what the problem with the descriptor was. Diagnostic for + ticket 24972. + + o Minor bugfix (channel connection): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfix (directory authority): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (all versions of Tor): + - Use the "misspell" tool to detect and fix typos throughout the + source code. Fixes bug 23650; bugfix on various versions of Tor. + Patch from Deepesh Pathak. + + o Minor bugfixes (circuit, cannibalization): + - Don't cannibalize preemptively-built circuits if we no longer + recognize their first hop. This situation can happen if our Guard + relay went off the consensus after the circuit was created. Fixes + bug 24469; bugfix on 0.0.6. + + o Minor bugfixes (correctness): + - Remove a nonworking, unnecessary check to see whether a circuit + hop's identity digest was set when the circuit failed. Fixes bug + 24927; bugfix on 0.2.4.4-alpha. + + o Minor bugfixes (logging): + - Don't treat inability to store a cached consensus object as a bug: + it can happen normally when we are out of disk space. Fixes bug + 24859; bugfix on 0.3.1.1-alpha. + - Fix a (mostly harmless) race condition when invoking + LOG_PROTOCOL_WARN message from a subthread while the torrc options + are changing. Fixes bug 23954; bugfix on 0.1.1.9-alpha. + + o Minor bugfixes (onion services): + - Remove a BUG() statement when a client fetches an onion descriptor + that has a lower revision counter than the one in its cache. This + can happen in normal circumstances due to HSDir desync. Fixes bug + 24976; bugfix on 0.3.2.1-alpha. + - If we are configured to offer a single onion service, don't log + long-term established one hop rendezvous points in the heartbeat. + Fixes bug 25116; bugfix on 0.2.9.6-rc. + + o Minor bugfixes (performance): + - Avoid calling protocol_list_supports_protocol() from inside tight + loops when running with cached routerinfo_t objects. Instead, + summarize the relevant protocols as flags in the routerinfo_t, as + we do for routerstatus_t objects. This change simplifies our code + a little, and saves a large amount of short-term memory allocation + operations. Fixes bug 25008; bugfix on 0.2.9.4-alpha. + + o Minor bugfixes (Rust FFI): + - Fix a minor memory leak which would happen whenever the C code + would call the Rust implementation of + protover_get_supported_protocols(). This was due to the C version + returning a static string, whereas the Rust version newly allocated + a CString to pass accross the FFI boundary. Consequently, the C + code was not expecting to need to free() what it was given. Fixes + bug 25127; bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (scheduler, KIST): + - Avoid adding the same channel twice in the KIST scheduler pending + list, which would waste CPU cycles. Fixes bug 24700; bugfix + on 0.3.2.1-alpha. + + o Minor bugfixes (unit test, monotonic time): + - Increase a constant (1msec to 10msec) in the monotonic time test + that makes sure the nsec/usec/msec times read are synchronized. + This change was needed to accommodate slow systems like armel or + when the clock_gettime() is not a VDSO on the running kernel. + Fixes bug 25113; bugfix on 0.2.9.1. + + o Minor bugfixes (v3 onion services): + - Look at the "HSRend" protocol version, not the "HSDir" protocol + version, when deciding whether a consensus entry can support the + v3 onion service protocol as a rendezvous point. Fixes bug 25105; + bugfix on 0.3.2.1-alpha. + + o Code simplification and refactoring: + - Remove the unused nodelist_recompute_all_hsdir_indices(). Closes + ticket 25108. + - Remove a series of counters used to track circuit extend attempts + and connection status but that in reality we aren't using for + anything other than stats logged by a SIGUSR1 signal. Closes + ticket 25163. + + o Documentation (man page): + - The HiddenServiceVersion torrc option accepts only one number: + either version 2 or 3. Closes ticket 25026; bugfix + on 0.3.2.2-alpha. + + Changes in version 0.3.3.1-alpha - 2018-01-25 Tor 0.3.3.1-alpha is the first release in the 0.3.3.x series. It adds several new features to Tor, including several improvements to diff --git a/changes/bug22212-forreal b/changes/bug22212-forreal deleted file mode 100644 index 159d0990ee..0000000000 --- a/changes/bug22212-forreal +++ /dev/null @@ -1,8 +0,0 @@ - o Major bugfixes (netflow padding): - - Stop adding unneeded channel padding right after we finish flushing - to a connection that has been trying to flush for many seconds. - Instead, treat all partial or complete flushes as activity on the - channel, which will defer the time until we need to add padding. - This fix should resolve confusing and scary log messages like - "Channel padding timeout scheduled 221453ms in the past." Fixes - bug 22212; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug23954 b/changes/bug23954 deleted file mode 100644 index 185814f12e..0000000000 --- a/changes/bug23954 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (logging, race conditions): - - Fix a (mostly harmless) race condition when invoking - LOG_PROTOCOL_WARN message from a subthread while the options are - changing. Fixes bug 23954; bugfix on 0.1.1.9-alpha. diff --git a/changes/bug24469 b/changes/bug24469 deleted file mode 100644 index 2e137b49b8..0000000000 --- a/changes/bug24469 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (circuit, cannibalization): - - Don't cannibalize circuits for which we don't know the first hop which - can happen if our Guard relay went off the consensus after the circuit - was created preemptively. Fixes bug 24469; bugfix on 0.0.6. diff --git a/changes/bug24700 b/changes/bug24700 deleted file mode 100644 index 74dc581a0b..0000000000 --- a/changes/bug24700 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (scheduler, KIST): - - Avoid adding the same channel twice in the KIST scheduler pending list - wasting CPU cycles at handling the same channel twice. Fixes bug 24700; - bugfix on 0.3.2.1-alpha. diff --git a/changes/bug24859 b/changes/bug24859 deleted file mode 100644 index 122109d650..0000000000 --- a/changes/bug24859 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (logging): - - Don't treat inability to store a cached consensus object as a - bug: it can happen normally when we are out of disk space. - Fixes bug 24859; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug24927 b/changes/bug24927 deleted file mode 100644 index 6997306956..0000000000 --- a/changes/bug24927 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (correctness): - - Remove nonworking, unnecessary check to see whether a circuit hop's - identity was set when the circuit failed. Fixes bug 24927; bugfix on - 0.2.4.4-alpha. diff --git a/changes/bug24952 b/changes/bug24952 deleted file mode 100644 index 93174c04f5..0000000000 --- a/changes/bug24952 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfix (channel connection): - - The accurate address of a connection is real_addr, not the addr member. - TLS Channel remote address is now real_addr content instead of addr - member. Fixes bug 24952; bugfix on 707c1e2e26 in 0.2.4.11-alpha. - Patch by "ffmancera". diff --git a/changes/bug24972 b/changes/bug24972 deleted file mode 100644 index 5adf970abf..0000000000 --- a/changes/bug24972 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (logging, diagnostic): - - When logging a failure to check a hidden service's certificate, - also log what the problem with the certificate was. Diagnostic - for ticket 24972. diff --git a/changes/bug24975 b/changes/bug24975 deleted file mode 100644 index 32a5dfc929..0000000000 --- a/changes/bug24975 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes (scheduler, consensus): - - A logic in the code was preventing the scheduler subystem to properly - make a decision based on the latest consensus when it arrives. This lead - to the scheduler failing to notice any consensus parameters that might - have changed between consensuses. Fixes bug 24975; bugfix on - 0.3.2.1-alpha. diff --git a/changes/bug24976 b/changes/bug24976 deleted file mode 100644 index 9c3be86eab..0000000000 --- a/changes/bug24976 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (hidden service v3 client): - - Remove a BUG() statement which can be triggered in normal circumstances - where a client fetches a descriptor that has a lower revision counter - than the one in its cache. This can happen due to HSDir desync. Fixes - bug 24976; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug24978 b/changes/bug24978 new file mode 100644 index 0000000000..5dc45c7442 --- /dev/null +++ b/changes/bug24978 @@ -0,0 +1,7 @@ + o Minor features (compatibility, OpenSSL): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL + 1.1.1, since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + diff --git a/changes/bug25008 b/changes/bug25008 deleted file mode 100644 index 5ddc062982..0000000000 --- a/changes/bug25008 +++ /dev/null @@ -1,9 +0,0 @@ - o Minor bugfixes (performance): - - Avoid calling protocol_list_supports_protocol() from inside tight loops - when running with cached routerinfo_t objects. Instead, - summarize the relevant protocols as flags in the routerinfo_t, as we do - for routerstatus_t objects. This change simplifies our code a little, - and saves a large amount of short-term memory allocation operations. - Fixes bug 25008; bugfix on 0.2.9.4-alpha. - - diff --git a/changes/bug25026 b/changes/bug25026 deleted file mode 100644 index 24a64a562b..0000000000 --- a/changes/bug25026 +++ /dev/null @@ -1,4 +0,0 @@ - o Documentation: - - HiddenServiceVersion only accepts one version to be specified. This - version can either be version 2 or 3. Closes ticket 25026; bugfix on - 0.3.2.2-alpha. diff --git a/changes/bug25070 b/changes/bug25070 deleted file mode 100644 index c2f4e58c45..0000000000 --- a/changes/bug25070 +++ /dev/null @@ -1,3 +0,0 @@ - o Major bugfixes (protocol versions): - - Add Link protocol version 5 to the supported protocols list. - Fixes bug 25070; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug25105 b/changes/bug25105 deleted file mode 100644 index 36d1a5f16f..0000000000 --- a/changes/bug25105 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (v3 onion services): - - Look at the "HSRend" protocol version, not the "HSDir" protocol - version, when deciding whether a consensus entry can support - the v3 onion service protocol as a rendezvous point. - Fixes bug 25105; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug25113 b/changes/bug25113 deleted file mode 100644 index 4a020b784d..0000000000 --- a/changes/bug25113 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (unit test, monotonic time): - - Bump a gap of 1msec to 10msec used in the monotonic time test that makes - sure the nsec/usec/msec time read are synchronized. This change was - needed to accommodate slow system like armel or when the clock_gettime() - is not a VDSO on the running kernel. Fixes bug 25113; bugfix on 0.2.9.1. diff --git a/changes/bug25116 b/changes/bug25116 deleted file mode 100644 index b3e73feeaa..0000000000 --- a/changes/bug25116 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (hidden service, heartbeat): - - Don't log in the heartbeat any long term established one hop rendezvous - points if tor is a single onion service. Fixes bug 25116; bugfix on - 0.2.9.6-rc; diff --git a/changes/bug25120 b/changes/bug25120 new file mode 100644 index 0000000000..7215756ef3 --- /dev/null +++ b/changes/bug25120 @@ -0,0 +1,4 @@ + o Minor features (logging): + - Clarify the log messages produced when getrandom() or a related + entropy-generation mechanism gives an error. Closes ticket + 25120. diff --git a/changes/bug25127 b/changes/bug25127 deleted file mode 100644 index 3438ed1256..0000000000 --- a/changes/bug25127 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (Rust FFI): - - Fix a minor memory leak which would happen whenever the C code would call - the Rust implementation of protover_get_supported_protocols(). This was - due to the C version returning a static string, whereas the Rust version - newly allocated a CString to pass accross the FFI boundary. Consequently, - the C code was not expecting to need to free() what it was given. Fixes - bug 25127; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug25223 b/changes/bug25223 new file mode 100644 index 0000000000..fdd5563500 --- /dev/null +++ b/changes/bug25223 @@ -0,0 +1,4 @@ + o Minor bugfixes (DoS mitigation): + - Make sure we don't modify consensus parameters if we aren't a public + relay when a new consensus arrives. Fixes bug 25223; bugfix on + 0.3.3.2-alpha. diff --git a/changes/geoip-2018-02-07 b/changes/geoip-2018-02-07 deleted file mode 100644 index f45228fd76..0000000000 --- a/changes/geoip-2018-02-07 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 - Country database. - diff --git a/changes/ticket23650 b/changes/ticket23650 deleted file mode 100644 index cf5eb39d50..0000000000 --- a/changes/ticket23650 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (all versions of Tor): - - Use the "misspell" tool to detect and fix typos throughout the source - code. Fixes bug 23650; bugfix on various versions of Tor. Patch - from Deepesh Pathak. diff --git a/changes/ticket24849 b/changes/ticket24849 deleted file mode 100644 index fd9492acb9..0000000000 --- a/changes/ticket24849 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features (directory authority): - - When unable to add signatures to a pending consensus, log the reason - why. Closes ticket 24849. diff --git a/changes/ticket24902 b/changes/ticket24902 deleted file mode 100644 index 1a2ef95cc9..0000000000 --- a/changes/ticket24902 +++ /dev/null @@ -1,13 +0,0 @@ - o Major features (denial of service mitigation): - - Give relays some defenses against the recent network overload. We start - with three defenses (default parameters in parentheses). First: if a - single client address makes too many concurrent connections (>100), hang - up on further connections. Second: if a single client address makes - circuits too quickly (more than 3 per second, with an allowed burst of - 90) while also having too many connections open (3), refuse new create - cells for the next while (1-2 hours). Third: if a client asks to - establish a rendezvous point to you directly, ignore the request. These - defenses can be manually controlled by new torrc options, but relays - will also take guidance from consensus parameters, so there's no need to - configure anything manually. Implements ticket 24902. - diff --git a/changes/ticket25108 b/changes/ticket25108 deleted file mode 100644 index 6aefac16db..0000000000 --- a/changes/ticket25108 +++ /dev/null @@ -1,3 +0,0 @@ - o Code simplification and refactoring: - - Remove the unused nodelist_recompute_all_hsdir_indices(). Closes ticket - 25108. diff --git a/changes/ticket25122 b/changes/ticket25122 deleted file mode 100644 index 2921811b22..0000000000 --- a/changes/ticket25122 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor feature (geoip cache): - - Make our OOM handler aware of the geoip client history cache so it - doesn't fill up the memory which is especially important for IPv6 and - our DoS mitigation subsystem. Closes ticket 25122. diff --git a/changes/ticket25163 b/changes/ticket25163 deleted file mode 100644 index 6d237db75e..0000000000 --- a/changes/ticket25163 +++ /dev/null @@ -1,4 +0,0 @@ - o Code simplification and refactoring (rephist): - - Remove a series of counters used to track circuit extend attemps and - connection status but that in reality we aren't using for anything other - than stats logged by a SIGUSR1 signal. Closes ticket 25163. diff --git a/changes/ticket25170 b/changes/ticket25170 deleted file mode 100644 index 0652139400..0000000000 --- a/changes/ticket25170 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfix (directory authority, documentation): - - When a fingerprint or network address is marked as rejected, the - returned message by the authority now explicitly mention to set a valid - ContactInfo address and contact the bad-relays@ mailing list. Fixes bug - 25170; bugfix on 0.2.9.1. diff --git a/changes/ticket25202 b/changes/ticket25202 new file mode 100644 index 0000000000..ba64abad7b --- /dev/null +++ b/changes/ticket25202 @@ -0,0 +1,4 @@ + o Minor bugfixes (DoS mitigation): + - Add extra safety checks when refilling the circuit creation bucket to + ensure we never set a value that is above the allowed burst. Fixes + bug 25202; bugfix on 0.3.3.2-alpha. diff --git a/configure.ac b/configure.ac index 7eb189a0e9..1e27dfa563 100644 --- a/configure.ac +++ b/configure.ac @@ -4,7 +4,7 @@ dnl Copyright (c) 2007-2017, The Tor Project, Inc. dnl See LICENSE for licensing information AC_PREREQ([2.63]) -AC_INIT([tor],[0.3.3.1-alpha-dev]) +AC_INIT([tor],[0.3.3.2-alpha-dev]) AC_CONFIG_SRCDIR([src/or/main.c]) AC_CONFIG_MACRO_DIR([m4]) diff --git a/contrib/win32build/tor-mingw.nsi.in b/contrib/win32build/tor-mingw.nsi.in index f4d18a77db..77e75c4502 100644 --- a/contrib/win32build/tor-mingw.nsi.in +++ b/contrib/win32build/tor-mingw.nsi.in @@ -8,7 +8,7 @@ !include "LogicLib.nsh" !include "FileFunc.nsh" !insertmacro GetParameters -!define VERSION "0.3.3.1-alpha-dev" +!define VERSION "0.3.3.2-alpha-dev" !define INSTALLER "tor-${VERSION}-win32.exe" !define WEBSITE "https://www.torproject.org/" !define LICENSE "LICENSE" diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 9a4e70c769..2c8135ff05 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -2765,7 +2765,7 @@ Denial of Service mitigation subsystem. address is positively identified, tor will activate defenses against the address. See the DoSCircuitCreationDefenseType option for more details. This is a client to relay detection only. "auto" means use the consensus - parameter. + parameter. If not defined in the consensus, the value is 0. (Default: auto) [[DoSCircuitCreationMinConnections]] **DoSCircuitCreationMinConnections** __NUM__:: @@ -2774,19 +2774,22 @@ Denial of Service mitigation subsystem. flagged as executing a circuit creation DoS. In other words, once a client address reaches the circuit rate and has a minimum of NUM concurrent connections, a detection is positive. "0" means use the consensus - parameter. + parameter. If not defined in the consensus, the value is 3. (Default: 0) [[DoSCircuitCreationRate]] **DoSCircuitCreationRate** __NUM__:: The allowed circuit creation rate per second applied per client IP - address. If this option is 0, it obeys a consensus parameter. (Default: 0) + address. If this option is 0, it obeys a consensus parameter. If not + defined in the consensus, the value is 3. + (Default: 0) [[DoSCircuitCreationBurst]] **DoSCircuitCreationBurst** __NUM__:: The allowed circuit creation burst per client IP address. If the circuit rate and the burst are reached, a client is marked as executing a circuit - creation DoS. "0" means use the consensus parameter. + creation DoS. "0" means use the consensus parameter. If not defined in the + consensus, the value is 90. (Default: 0) [[DoSCircuitCreationDefenseType]] **DoSCircuitCreationDefenseType** __NUM__:: @@ -2797,28 +2800,31 @@ Denial of Service mitigation subsystem. 1: No defense. 2: Refuse circuit creation for the DoSCircuitCreationDefenseTimePeriod period of time. + - "0" means use the consensus parameter. + "0" means use the consensus parameter. If not defined in the consensus, + the value is 2. (Default: 0) -[[DoSCircuitCreationDefenseTimePeriod]] **DoSCircuitCreationDefenseTimePeriod** __NUM__:: +[[DoSCircuitCreationDefenseTimePeriod]] **DoSCircuitCreationDefenseTimePeriod** __N__ **seconds**|**minutes**|**hours**:: - The base time period that the DoS defense is activated for. The actual - value is selected randomly for each activation from NUM+1 to 3/2 * NUM. - "0" means use the consensus parameter. - (Default: 0) + The base time period in seconds that the DoS defense is activated for. The + actual value is selected randomly for each activation from N+1 to 3/2 * N. + "0" means use the consensus parameter. If not defined in the consensus, + the value is 3600 seconds (1 hour). (Default: 0) [[DoSConnectionEnabled]] **DoSConnectionEnabled** **0**|**1**|**auto**:: Enable the connection DoS mitigation. For client address only, this allows tor to mitigate against large number of concurrent connections made by a - single IP address. "auto" means use the consensus parameter. + single IP address. "auto" means use the consensus parameter. If not + defined in the consensus, the value is 0. (Default: auto) [[DoSConnectionMaxConcurrentCount]] **DoSConnectionMaxConcurrentCount** __NUM__:: The maximum threshold of concurrent connection from a client IP address. Above this limit, a defense selected by DoSConnectionDefenseType is - applied. "0" means use the consensus parameter. + applied. "0" means use the consensus parameter. If not defined in the + consensus, the value is 100. (Default: 0) [[DoSConnectionDefenseType]] **DoSConnectionDefenseType** __NUM__:: @@ -2829,7 +2835,8 @@ Denial of Service mitigation subsystem. 1: No defense. 2: Immediately close new connections. + - "0" means use the consensus parameter. + "0" means use the consensus parameter. If not defined in the consensus, + the value is 2. (Default: 0) [[DoSRefuseSingleHopClientRendezvous]] **DoSRefuseSingleHopClientRendezvous** **0**|**1**|**auto**:: @@ -2837,7 +2844,7 @@ Denial of Service mitigation subsystem. Refuse establishment of rendezvous points for single hop clients. In other words, if a client directly connects to the relay and sends an ESTABLISH_RENDEZVOUS cell, it is silently dropped. "auto" means use the - consensus parameter. + consensus parameter. If not defined in the consensus, the value is 0. (Default: auto) TESTING NETWORK OPTIONS diff --git a/src/common/address.c b/src/common/address.c index 7ff81206e5..a2f4c93b91 100644 --- a/src/common/address.c +++ b/src/common/address.c @@ -1185,6 +1185,9 @@ tor_addr_compare_masked(const tor_addr_t *addr1, const tor_addr_t *addr2, } } +/** Input for siphash, to produce some output for an unspec value. */ +static const uint32_t unspec_hash_input[] = { 0x4e4df09f, 0x92985342 }; + /** Return a hash code based on the address addr. DOCDOC extra */ uint64_t tor_addr_hash(const tor_addr_t *addr) @@ -1193,7 +1196,7 @@ tor_addr_hash(const tor_addr_t *addr) case AF_INET: return siphash24g(&addr->addr.in_addr.s_addr, 4); case AF_UNSPEC: - return 0x4e4d5342; + return siphash24g(unspec_hash_input, sizeof(unspec_hash_input)); case AF_INET6: return siphash24g(&addr->addr.in6_addr.s6_addr, 16); /* LCOV_EXCL_START */ @@ -1204,6 +1207,28 @@ tor_addr_hash(const tor_addr_t *addr) } } +/** As tor_addr_hash, but use a particular siphash key. */ +uint64_t +tor_addr_keyed_hash(const struct sipkey *key, const tor_addr_t *addr) +{ + /* This is duplicate code with tor_addr_hash, since this function needs to + * be backportable all the way to 0.2.9. */ + + switch (tor_addr_family(addr)) { + case AF_INET: + return siphash24(&addr->addr.in_addr.s_addr, 4, key); + case AF_UNSPEC: + return siphash24(unspec_hash_input, sizeof(unspec_hash_input), key); + case AF_INET6: + return siphash24(&addr->addr.in6_addr.s6_addr, 16, key); + default: + /* LCOV_EXCL_START */ + tor_fragile_assert(); + return 0; + /* LCOV_EXCL_END */ + } +} + /** Return a newly allocated string with a representation of <b>addr</b>. */ char * tor_addr_to_str_dup(const tor_addr_t *addr) diff --git a/src/common/address.h b/src/common/address.h index 7607c76bae..c9d9543dee 100644 --- a/src/common/address.h +++ b/src/common/address.h @@ -231,6 +231,8 @@ int tor_addr_compare_masked(const tor_addr_t *addr1, const tor_addr_t *addr2, #define tor_addr_eq(a,b) (0==tor_addr_compare((a),(b),CMP_EXACT)) uint64_t tor_addr_hash(const tor_addr_t *addr); +struct sipkey; +uint64_t tor_addr_keyed_hash(const struct sipkey *key, const tor_addr_t *addr); int tor_addr_is_v4(const tor_addr_t *addr); int tor_addr_is_internal_(const tor_addr_t *ip, int for_listening, const char *filename, int lineno); diff --git a/src/common/address_set.c b/src/common/address_set.c new file mode 100644 index 0000000000..f61fa294e0 --- /dev/null +++ b/src/common/address_set.c @@ -0,0 +1,129 @@ +/* Copyright (c) 2018, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file address_set.c + * \brief Implementation for a set of addresses. + * + * This module was first written on a semi-emergency basis to improve the + * robustness of the anti-DoS module. As such, it's written in a pretty + * conservative way, and should be susceptible to improvement later on. + **/ + +#include "orconfig.h" +#include "address_set.h" +#include "address.h" +#include "compat.h" +#include "container.h" +#include "crypto.h" +#include "util.h" +#include "siphash.h" + +/** How many 64-bit siphash values to extract per address */ +#define N_HASHES 2 +/** How many bloom-filter bits we set per address. This is twice the N_HASHES + * value, since we split the siphash output into two 32-bit values. */ +#define N_BITS_PER_ITEM (N_HASHES * 2) + +/* XXXX This code is largely duplicated with digestset_t. We should merge + * them together into a common bloom-filter implementation. I'm keeping + * them separate for now, though, since this module needs to be backported + * all the way to 0.2.9. + * + * The main difference between digestset_t and this code is that we use + * independent siphashes rather than messing around with bit-shifts. The + * approach here is probably more sound, and we should prefer it if&when we + * unify the implementations. + */ + +struct address_set_t { + /** siphash keys to make N_HASHES independent hashes for each address. */ + struct sipkey key[N_HASHES]; + int mask; /**< One less than the number of bits in <b>ba</b>; always one less + * than a power of two. */ + bitarray_t *ba; /**< A bit array to implement the Bloom filter. */ +}; + +/** + * Allocate and return an address_set, suitable for holding up to + * <b>max_address_guess</b> distinct values. + */ +address_set_t * +address_set_new(int max_addresses_guess) +{ + /* See digestset_new() for rationale on this equation. */ + int n_bits = 1u << (tor_log2(max_addresses_guess)+5); + + address_set_t *set = tor_malloc_zero(sizeof(address_set_t)); + set->mask = n_bits - 1; + set->ba = bitarray_init_zero(n_bits); + crypto_rand((char*) set->key, sizeof(set->key)); + + return set; +} + +/** + * Release all storage associated with <b>set</b>. + */ +void +address_set_free(address_set_t *set) +{ + if (! set) + return; + + bitarray_free(set->ba); + tor_free(set); +} + +/** Yield the bit index corresponding to 'val' for set. */ +#define BIT(set, val) ((val) & (set)->mask) + +/** + * Add <b>addr</b> to <b>set</b>. + * + * All future queries for <b>addr</b> in set will return true. Removing + * items is not possible. + */ +void +address_set_add(address_set_t *set, const struct tor_addr_t *addr) +{ + int i; + for (i = 0; i < N_HASHES; ++i) { + uint64_t h = tor_addr_keyed_hash(&set->key[i], addr); + uint32_t high_bits = (uint32_t)(h >> 32); + uint32_t low_bits = (uint32_t)(h); + bitarray_set(set->ba, BIT(set, high_bits)); + bitarray_set(set->ba, BIT(set, low_bits)); + } +} + +/** As address_set_add(), but take an ipv4 address in host order. */ +void +address_set_add_ipv4h(address_set_t *set, uint32_t addr) +{ + tor_addr_t a; + tor_addr_from_ipv4h(&a, addr); + address_set_add(set, &a); +} + +/** + * Return true if <b>addr</b> is a member of <b>set</b>. (And probably, + * return false if <b>addr</b> is not a member of set.) + */ +int +address_set_probably_contains(address_set_t *set, + const struct tor_addr_t *addr) +{ + int i, matches = 0; + for (i = 0; i < N_HASHES; ++i) { + uint64_t h = tor_addr_keyed_hash(&set->key[i], addr); + uint32_t high_bits = (uint32_t)(h >> 32); + uint32_t low_bits = (uint32_t)(h); + // Note that !! is necessary here, since bitarray_is_set does not + // necessarily return 1 on true. + matches += !! bitarray_is_set(set->ba, BIT(set, high_bits)); + matches += !! bitarray_is_set(set->ba, BIT(set, low_bits)); + } + return matches == N_BITS_PER_ITEM; +} + diff --git a/src/common/address_set.h b/src/common/address_set.h new file mode 100644 index 0000000000..28d29f3fdf --- /dev/null +++ b/src/common/address_set.h @@ -0,0 +1,35 @@ +/* Copyright (c) 2018, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file address_set.h + * \brief Types to handle sets of addresses. + * + * This module was first written on a semi-emergency basis to improve the + * robustness of the anti-DoS module. As such, it's written in a pretty + * conservative way, and should be susceptible to improvement later on. + **/ + +#ifndef TOR_ADDRESS_SET_H +#define TOR_ADDRESS_SET_H + +#include "orconfig.h" +#include "torint.h" + +/** + * An address_set_t represents a set of tor_addr_t values. The implementation + * is probabilistic: false negatives cannot occur but false positives are + * possible. + */ +typedef struct address_set_t address_set_t; +struct tor_addr_t; + +address_set_t *address_set_new(int max_addresses_guess); +void address_set_free(address_set_t *set); +void address_set_add(address_set_t *set, const struct tor_addr_t *addr); +void address_set_add_ipv4h(address_set_t *set, uint32_t addr); +int address_set_probably_contains(address_set_t *set, + const struct tor_addr_t *addr); + +#endif + diff --git a/src/common/ciphers.inc b/src/common/ciphers.inc index a336e2dfa0..0084b3e325 100644 --- a/src/common/ciphers.inc +++ b/src/common/ciphers.inc @@ -2,8 +2,27 @@ * advertise. Before including it, you should define the CIPHER and XCIPHER * macros. * - * This file was automatically generated by get_mozilla_ciphers.py. + * This file was automatically generated by get_mozilla_ciphers.py; + * TLSv1.3 ciphers were added manually. */ + +/* Here are the TLS1.3 ciphers. Note that we don't have XCIPHER instances + * here, since we don't want to ever fake them. + */ +#ifdef TLS1_3_TXT_AES_128_GCM_SHA256 + CIPHER(0x1301, TLS1_3_TXT_AES_128_GCM_SHA256) +#endif +#ifdef TLS1_3_TXT_AES_256_GCM_SHA384 + CIPHER(0x1302, TLS1_3_TXT_AES_256_GCM_SHA384) +#endif +#ifdef TLS1_3_TXT_CHACHA20_POLY1305_SHA256 + CIPHER(0x1303, TLS1_3_TXT_CHACHA20_POLY1305_SHA256) +#endif +#ifdef TLS1_3_TXT_AES_128_CCM_SHA256 + CIPHER(0x1304, TLS1_3_TXT_AES_128_CCM_SHA256) +#endif + +/* Here's the machine-generated list. */ #ifdef TLS1_TXT_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 CIPHER(0xc02b, TLS1_TXT_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256) #else diff --git a/src/common/crypto.c b/src/common/crypto.c index affcda40fa..d85aca4004 100644 --- a/src/common/crypto.c +++ b/src/common/crypto.c @@ -1891,6 +1891,12 @@ crypto_strongest_rand_syscall(uint8_t *out, size_t out_len) { tor_assert(out_len <= MAX_STRONGEST_RAND_SIZE); + /* We only log at notice-level here because in the case that this function + * fails the crypto_strongest_rand_raw() caller will log with a warning-level + * message and let crypto_strongest_rand() error out and finally terminating + * Tor with an assertion error. + */ + #ifdef TOR_UNIT_TESTS if (break_strongest_rng_syscall) return -1; @@ -1903,13 +1909,13 @@ crypto_strongest_rand_syscall(uint8_t *out, size_t out_len) if (!provider_set) { if (!CryptAcquireContext(&provider, NULL, NULL, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT)) { - log_warn(LD_CRYPTO, "Can't get CryptoAPI provider [1]"); + log_notice(LD_CRYPTO, "Unable to set Windows CryptoAPI provider [1]."); return -1; } provider_set = 1; } if (!CryptGenRandom(provider, out_len, out)) { - log_warn(LD_CRYPTO, "Can't get entropy from CryptoAPI."); + log_notice(LD_CRYPTO, "Unable get entropy from the Windows CryptoAPI."); return -1; } @@ -1951,13 +1957,15 @@ crypto_strongest_rand_syscall(uint8_t *out, size_t out_len) /* Useful log message for errno. */ if (errno == ENOSYS) { - log_warn(LD_CRYPTO, "Can't get entropy from getrandom()." - " You are running a version of Tor built to support" - " getrandom(), but the kernel doesn't implement this" - " function--probably because it is too old?"); + log_notice(LD_CRYPTO, "Can't get entropy from getrandom()." + " You are running a version of Tor built to support" + " getrandom(), but the kernel doesn't implement this" + " function--probably because it is too old?" + " Trying fallback method instead."); } else { - log_warn(LD_CRYPTO, "Can't get entropy from getrandom(): %s.", - strerror(errno)); + log_notice(LD_CRYPTO, "Can't get entropy from getrandom(): %s." + " Trying fallback method instead.", + strerror(errno)); } getrandom_works = 0; /* Don't bother trying again. */ @@ -2009,7 +2017,7 @@ crypto_strongest_rand_fallback(uint8_t *out, size_t out_len) size_t n; for (i = 0; filenames[i]; ++i) { - log_debug(LD_FS, "Considering %s for entropy", filenames[i]); + log_debug(LD_FS, "Considering %s as entropy source", filenames[i]); fd = open(sandbox_intern_string(filenames[i]), O_RDONLY, 0); if (fd<0) continue; log_info(LD_CRYPTO, "Reading entropy from \"%s\"", filenames[i]); @@ -2018,9 +2026,10 @@ crypto_strongest_rand_fallback(uint8_t *out, size_t out_len) if (n != out_len) { /* LCOV_EXCL_START * We can't make /dev/foorandom actually fail. */ - log_warn(LD_CRYPTO, - "Error reading from entropy source (read only %lu bytes).", - (unsigned long)n); + log_notice(LD_CRYPTO, + "Error reading from entropy source %s (read only %lu bytes).", + filenames[i], + (unsigned long)n); return -1; /* LCOV_EXCL_STOP */ } diff --git a/src/common/include.am b/src/common/include.am index 1777f33ad9..6945285108 100644 --- a/src/common/include.am +++ b/src/common/include.am @@ -80,6 +80,7 @@ src_common_libor_ctime_testing_a_CFLAGS = @CFLAGS_CONSTTIME@ $(TEST_CFLAGS) LIBOR_A_SRC = \ src/common/address.c \ + src/common/address_set.c \ src/common/backtrace.c \ src/common/buffers.c \ src/common/compat.c \ @@ -145,6 +146,7 @@ src_common_libor_event_testing_a_CFLAGS = $(AM_CFLAGS) $(TEST_CFLAGS) COMMONHEADERS = \ src/common/address.h \ + src/common/address_set.h \ src/common/backtrace.h \ src/common/buffers.h \ src/common/buffers_tls.h \ diff --git a/src/common/tortls.c b/src/common/tortls.c index bff59833ff..50609b8ac7 100644 --- a/src/common/tortls.c +++ b/src/common/tortls.c @@ -570,13 +570,35 @@ tor_tls_create_certificate,(crypto_pk_t *rsa, /** List of ciphers that servers should select from when the client might be * claiming extra unsupported ciphers in order to avoid fingerprinting. */ -#define SERVER_CIPHER_LIST \ - (TLS1_TXT_DHE_RSA_WITH_AES_256_SHA ":" \ - TLS1_TXT_DHE_RSA_WITH_AES_128_SHA) +static const char SERVER_CIPHER_LIST[] = +#ifdef TLS1_3_TXT_AES_128_GCM_SHA256 + /* This one can never actually get selected, since if the client lists it, + * we will assume that the client is honest, and not use this list. + * Nonetheless we list it if it's available, so that the server doesn't + * conclude that it has no valid ciphers if it's running with TLS1.3. + */ + TLS1_3_TXT_AES_128_GCM_SHA256 ":" +#endif + TLS1_TXT_DHE_RSA_WITH_AES_256_SHA ":" + TLS1_TXT_DHE_RSA_WITH_AES_128_SHA; /** List of ciphers that servers should select from when we actually have * our choice of what cipher to use. */ static const char UNRESTRICTED_SERVER_CIPHER_LIST[] = + /* Here are the TLS 1.3 ciphers we like, in the order we prefer. */ +#ifdef TLS1_3_TXT_AES_256_GCM_SHA384 + TLS1_3_TXT_AES_256_GCM_SHA384 ":" +#endif +#ifdef TLS1_3_TXT_CHACHA20_POLY1305_SHA256 + TLS1_3_TXT_CHACHA20_POLY1305_SHA256 ":" +#endif +#ifdef TLS1_3_TXT_AES_128_GCM_SHA256 + TLS1_3_TXT_AES_128_GCM_SHA256 ":" +#endif +#ifdef TLS1_3_TXT_AES_128_CCM_SHA256 + TLS1_3_TXT_AES_128_CCM_SHA256 ":" +#endif + /* This list is autogenerated with the gen_server_ciphers.py script; * don't hand-edit it. */ #ifdef TLS1_TXT_ECDHE_RSA_WITH_AES_256_GCM_SHA384 diff --git a/src/or/dos.c b/src/or/dos.c index 88f1351a3f..4d1797eece 100644 --- a/src/or/dos.c +++ b/src/or/dos.c @@ -14,6 +14,7 @@ #include "geoip.h" #include "main.h" #include "networkstatus.h" +#include "nodelist.h" #include "router.h" #include "dos.h" @@ -308,6 +309,16 @@ cc_stats_refill_bucket(cc_client_stats_t *stats, const tor_addr_t *addr) new_circuit_bucket_count = MIN(stats->circuit_bucket + (uint32_t)num_token, dos_cc_circuit_burst); } + + /* This function is not allowed to make the bucket count larger than the + * burst value */ + tor_assert_nonfatal(new_circuit_bucket_count <= dos_cc_circuit_burst); + /* This function is not allowed to make the bucket count smaller, unless it + * is decreasing it to a newly configured, lower burst value. We allow the + * bucket to stay the same size, in case the circuit rate is zero. */ + tor_assert_nonfatal(new_circuit_bucket_count >= stats->circuit_bucket || + new_circuit_bucket_count == dos_cc_circuit_burst); + log_debug(LD_DOS, "DoS address %s has its circuit bucket value: %" PRIu32 ". Filling it to %" PRIu32 ". Circuit rate is %" PRIu64 ". Elapsed time is %" PRIi64, @@ -664,6 +675,14 @@ dos_new_client_conn(or_connection_t *or_conn) goto end; } + /* We ignore any known address meaning an address of a known relay. The + * reason to do so is because network reentry is possible where a client + * connection comes from an Exit node. Even when we'll fix reentry, this is + * a robust defense to keep in place. */ + if (nodelist_probably_contains_address(&or_conn->real_addr)) { + goto end; + } + /* We are only interested in client connection from the geoip cache. */ entry = geoip_lookup_client(&or_conn->real_addr, NULL, GEOIP_CLIENT_CONNECT); @@ -729,6 +748,14 @@ dos_close_client_conn(const or_connection_t *or_conn) void dos_consensus_has_changed(const networkstatus_t *ns) { + /* There are two ways to configure this subsystem, one at startup through + * dos_init() which is called when the options are parsed. And this one + * through the consensus. We don't want to enable any DoS mitigation if we + * aren't a public relay. */ + if (!public_server_mode(get_options())) { + return; + } + cc_consensus_has_changed(ns); conn_consensus_has_changed(ns); diff --git a/src/or/hs_circuit.c b/src/or/hs_circuit.c index faa4b5d450..3a674f6223 100644 --- a/src/or/hs_circuit.c +++ b/src/or/hs_circuit.c @@ -722,7 +722,7 @@ hs_circ_retry_service_rendezvous_point(origin_circuit_t *circ) * circuit to the same rendezvous point at the same time. */ circ->hs_service_side_rend_circ_has_been_relaunched = 1; - /* Legacy service don't have a hidden service ident. */ + /* Legacy services don't have a hidden service ident. */ if (circ->hs_ident) { retry_service_rendezvous_point(circ); } else { diff --git a/src/or/nodelist.c b/src/or/nodelist.c index 17a50ca862..391b31d683 100644 --- a/src/or/nodelist.c +++ b/src/or/nodelist.c @@ -42,6 +42,7 @@ #include "or.h" #include "address.h" +#include "address_set.h" #include "config.h" #include "control.h" #include "dirserv.h" @@ -87,6 +88,7 @@ static void count_usable_descriptors(int *num_present, static void update_router_have_minimum_dir_info(void); static double get_frac_paths_needed_for_circs(const or_options_t *options, const networkstatus_t *ns); +static void node_add_to_address_set(const node_t *node); /** A nodelist_t holds a node_t object for every router we're "willing to use * for something". Specifically, it should hold a node_t for every node that @@ -105,6 +107,9 @@ typedef struct nodelist_t { * you should add it to this map with node_add_to_ed25519_map(). */ HT_HEAD(nodelist_ed_map, node_t) nodes_by_ed_id; + + /* Set of addresses that belong to nodes we believe in. */ + address_set_t *node_addrs; } nodelist_t; static inline unsigned int @@ -381,6 +386,50 @@ node_addrs_changed(node_t *node) node->country = -1; } +/** Add all address information about <b>node</b> to the current address + * set (if there is one). + */ +static void +node_add_to_address_set(const node_t *node) +{ + if (!the_nodelist || !the_nodelist->node_addrs) + return; + + /* These various address sources can be redundant, but it's likely faster + * to add them all than to compare them all for equality. */ + + if (node->rs) { + if (node->rs->addr) + address_set_add_ipv4h(the_nodelist->node_addrs, node->rs->addr); + if (!tor_addr_is_null(&node->rs->ipv6_addr)) + address_set_add(the_nodelist->node_addrs, &node->rs->ipv6_addr); + } + if (node->ri) { + if (node->ri->addr) + address_set_add_ipv4h(the_nodelist->node_addrs, node->ri->addr); + if (!tor_addr_is_null(&node->ri->ipv6_addr)) + address_set_add(the_nodelist->node_addrs, &node->ri->ipv6_addr); + } + if (node->md) { + if (!tor_addr_is_null(&node->md->ipv6_addr)) + address_set_add(the_nodelist->node_addrs, &node->md->ipv6_addr); + } +} + +/** Return true if <b>addr</b> is the address of some node in the nodelist. + * If not, probably return false. */ +int +nodelist_probably_contains_address(const tor_addr_t *addr) +{ + if (BUG(!addr)) + return 0; + + if (!the_nodelist || !the_nodelist->node_addrs) + return 0; + + return address_set_probably_contains(the_nodelist->node_addrs, addr); +} + /** Add <b>ri</b> to an appropriate node in the nodelist. If we replace an * old routerinfo, and <b>ri_old_out</b> is not NULL, set *<b>ri_old_out</b> * to the previous routerinfo. @@ -431,6 +480,8 @@ nodelist_set_routerinfo(routerinfo_t *ri, routerinfo_t **ri_old_out) networkstatus_get_latest_consensus()); } + node_add_to_address_set(node); + return node; } @@ -472,9 +523,22 @@ nodelist_add_microdesc(microdesc_t *md) node_add_to_ed25519_map(node); } + node_add_to_address_set(node); + return node; } +/* Default value. */ +#define ESTIMATED_ADDRESS_PER_NODE 2 + +/* Return the estimated number of address per node_t. This is used for the + * size of the bloom filter in the nodelist (node_addrs). */ +MOCK_IMPL(int, +get_estimated_address_per_node, (void)) +{ + return ESTIMATED_ADDRESS_PER_NODE; +} + /** Tell the nodelist that the current usable consensus is <b>ns</b>. * This makes the nodelist change all of the routerstatus entries for * the nodes, drop nodes that no longer have enough info to get used, @@ -493,6 +557,12 @@ nodelist_set_consensus(networkstatus_t *ns) SMARTLIST_FOREACH(the_nodelist->nodes, node_t *, node, node->rs = NULL); + /* Conservatively estimate that every node will have 2 addresses. */ + const int estimated_addresses = smartlist_len(ns->routerstatus_list) * + get_estimated_address_per_node(); + address_set_free(the_nodelist->node_addrs); + the_nodelist->node_addrs = address_set_new(estimated_addresses); + SMARTLIST_FOREACH_BEGIN(ns->routerstatus_list, routerstatus_t *, rs) { node_t *node = node_get_or_create(rs->identity_digest); node->rs = rs; @@ -536,6 +606,11 @@ nodelist_set_consensus(networkstatus_t *ns) nodelist_purge(); + /* Now add all the nodes we have to the address set. */ + SMARTLIST_FOREACH_BEGIN(the_nodelist->nodes, node_t *, node) { + node_add_to_address_set(node); + } SMARTLIST_FOREACH_END(node); + if (! authdir) { SMARTLIST_FOREACH_BEGIN(the_nodelist->nodes, node_t *, node) { /* We have no routerstatus for this router. Clear flags so we can skip @@ -694,6 +769,9 @@ nodelist_free_all(void) smartlist_free(the_nodelist->nodes); + address_set_free(the_nodelist->node_addrs); + the_nodelist->node_addrs = NULL; + tor_free(the_nodelist); } diff --git a/src/or/nodelist.h b/src/or/nodelist.h index 0abdcd6045..dc20eaf0a5 100644 --- a/src/or/nodelist.h +++ b/src/or/nodelist.h @@ -29,6 +29,7 @@ const node_t *node_get_by_hex_id(const char *identity_digest, node_t *nodelist_set_routerinfo(routerinfo_t *ri, routerinfo_t **ri_old_out); node_t *nodelist_add_microdesc(microdesc_t *md); void nodelist_set_consensus(networkstatus_t *ns); +int nodelist_probably_contains_address(const tor_addr_t *addr); void nodelist_remove_microdesc(const char *identity_digest, microdesc_t *md); void nodelist_remove_routerinfo(routerinfo_t *ri); @@ -154,5 +155,7 @@ node_set_hsdir_index(node_t *node, const networkstatus_t *ns); #endif /* defined(NODELIST_PRIVATE) */ +MOCK_DECL(int, get_estimated_address_per_node, (void)); + #endif /* !defined(TOR_NODELIST_H) */ diff --git a/src/or/replaycache.c b/src/or/replaycache.c index 4a56bfd7d4..a9a6709937 100644 --- a/src/or/replaycache.c +++ b/src/or/replaycache.c @@ -26,7 +26,6 @@ /** Free the replaycache r and all of its entries. */ - void replaycache_free_(replaycache_t *r) { @@ -44,7 +43,6 @@ replaycache_free_(replaycache_t *r) * for entries to age out and interval is the time after which the cache * should be scrubbed for old entries. */ - replaycache_t * replaycache_new(time_t horizon, time_t interval) { @@ -72,9 +70,8 @@ replaycache_new(time_t horizon, time_t interval) return r; } -/** See documentation for replaycache_add_and_test() +/** See documentation for replaycache_add_and_test(). */ - STATIC int replaycache_add_and_test_internal( time_t present, replaycache_t *r, const void *data, size_t len, @@ -136,9 +133,8 @@ replaycache_add_and_test_internal( return rv; } -/** See documentation for replaycache_scrub_if_needed() +/** See documentation for replaycache_scrub_if_needed(). */ - STATIC void replaycache_scrub_if_needed_internal(time_t present, replaycache_t *r) { @@ -186,7 +182,6 @@ replaycache_scrub_if_needed_internal(time_t present, replaycache_t *r) * and the function will return 1 if it was already seen within the cache's * horizon, or 0 otherwise. */ - int replaycache_add_and_test(replaycache_t *r, const void *data, size_t len) { @@ -196,7 +191,6 @@ replaycache_add_and_test(replaycache_t *r, const void *data, size_t len) /** Like replaycache_add_and_test(), but if it's a hit also return the time * elapsed since this digest was last seen. */ - int replaycache_add_test_and_elapsed( replaycache_t *r, const void *data, size_t len, time_t *elapsed) @@ -207,7 +201,6 @@ replaycache_add_test_and_elapsed( /** Scrub aged entries out of r if sufficiently long has elapsed since r was * last scrubbed. */ - void replaycache_scrub_if_needed(replaycache_t *r) { diff --git a/src/rust/protover/ffi.rs b/src/rust/protover/ffi.rs index d724c102d3..2ee0286ecf 100644 --- a/src/rust/protover/ffi.rs +++ b/src/rust/protover/ffi.rs @@ -232,7 +232,7 @@ pub extern "C" fn protover_compute_for_old_tor(version: *const c_char) -> *const // we can see that the bytes we're passing into it 1) are valid UTF-8, // 2) have no intermediate NUL bytes, and 3) are terminated with a NUL // byte. - supported = CStr::from_bytes_with_nul(elder_protocols).unwrap(); + supported = CStr::from_bytes_with_nul(elder_protocols).unwrap_or(empty); supported.as_ptr() } diff --git a/src/rust/protover/protover.rs b/src/rust/protover/protover.rs index 826f1b73f1..25f776aed4 100644 --- a/src/rust/protover/protover.rs +++ b/src/rust/protover/protover.rs @@ -110,7 +110,7 @@ pub fn get_supported_protocols() -> &'static str { // The `unwrap` is safe becauase we SUPPORTED_PROTOCOLS is under // our control. str::from_utf8(&SUPPORTED_PROTOCOLS[..SUPPORTED_PROTOCOLS.len() - 1]) - .unwrap() + .unwrap_or("") } pub struct SupportedProtocols(HashMap<Proto, Versions>); diff --git a/src/test/include.am b/src/test/include.am index fd70cc345d..1a49367c6b 100644 --- a/src/test/include.am +++ b/src/test/include.am @@ -87,6 +87,7 @@ src_test_test_SOURCES = \ src/test/test_accounting.c \ src/test/test_addr.c \ src/test/test_address.c \ + src/test/test_address_set.c \ src/test/test_buffers.c \ src/test/test_cell_formats.c \ src/test/test_cell_queue.c \ diff --git a/src/test/test.c b/src/test/test.c index 810cbafb11..78dd7bbb04 100644 --- a/src/test/test.c +++ b/src/test/test.c @@ -1185,6 +1185,7 @@ struct testgroup_t testgroups[] = { { "accounting/", accounting_tests }, { "addr/", addr_tests }, { "address/", address_tests }, + { "address_set/", address_set_tests }, { "buffer/", buffer_tests }, { "cellfmt/", cell_format_tests }, { "cellqueue/", cell_queue_tests }, diff --git a/src/test/test.h b/src/test/test.h index b41f0e54bb..26139fc5fe 100644 --- a/src/test/test.h +++ b/src/test/test.h @@ -177,6 +177,7 @@ extern const struct testcase_setup_t ed25519_test_setup; extern struct testcase_t accounting_tests[]; extern struct testcase_t addr_tests[]; extern struct testcase_t address_tests[]; +extern struct testcase_t address_set_tests[]; extern struct testcase_t buffer_tests[]; extern struct testcase_t cell_format_tests[]; extern struct testcase_t cell_queue_tests[]; diff --git a/src/test/test_address_set.c b/src/test/test_address_set.c new file mode 100644 index 0000000000..df022f539a --- /dev/null +++ b/src/test/test_address_set.c @@ -0,0 +1,174 @@ +/* Copyright (c) 2017, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "address_set.h" +#include "microdesc.h" +#include "networkstatus.h" +#include "nodelist.h" +#include "routerlist.h" +#include "torcert.h" + +#include "test.h" + +static networkstatus_t *dummy_ns = NULL; +static networkstatus_t * +mock_networkstatus_get_latest_consensus(void) +{ + return dummy_ns; +} + +static networkstatus_t * +mock_networkstatus_get_latest_consensus_by_flavor(consensus_flavor_t f) +{ + tor_assert(f == FLAV_MICRODESC); + return dummy_ns; +} + +/* Number of address a single node_t can have. Default to the production + * value. This is to control the size of the bloom filter. */ +static int addr_per_node = 2; +static int +mock_get_estimated_address_per_node(void) +{ + return addr_per_node; +} + +static void +test_contains(void *arg) +{ + int ret; + address_set_t *set = NULL; + + (void) arg; + + /* Setup an IPv4 and IPv6 addresses. */ + tor_addr_t addr_v6; + tor_addr_parse(&addr_v6, "1:2:3:4::"); + tor_addr_t addr_v4; + tor_addr_parse(&addr_v4, "42.42.42.42"); + uint32_t ipv4h = tor_addr_to_ipv4h(&addr_v4); + + /* Make it very big so the chance of failing the contain test will be + * extremely rare. */ + set = address_set_new(1024); + tt_assert(set); + + /* Add and lookup IPv6. */ + address_set_add(set, &addr_v6); + ret = address_set_probably_contains(set, &addr_v6); + tt_int_op(ret, OP_EQ, 1); + + /* Add and lookup IPv4. */ + address_set_add_ipv4h(set, ipv4h); + ret = address_set_probably_contains(set, &addr_v4); + tt_int_op(ret, OP_EQ, 1); + + /* Try a lookup of rubbish. */ + tor_addr_t dummy_addr; + memset(&dummy_addr, 'A', sizeof(dummy_addr)); + dummy_addr.family = AF_INET; + ret = address_set_probably_contains(set, &dummy_addr); + tt_int_op(ret, OP_EQ, 0); + dummy_addr.family = AF_INET6; + ret = address_set_probably_contains(set, &dummy_addr); + tt_int_op(ret, OP_EQ, 0); + + done: + address_set_free(set); +} + +static void +test_nodelist(void *arg) +{ + int ret; + routerstatus_t *rs = NULL; microdesc_t *md = NULL; routerinfo_t *ri = NULL; + + (void) arg; + + MOCK(networkstatus_get_latest_consensus, + mock_networkstatus_get_latest_consensus); + MOCK(networkstatus_get_latest_consensus_by_flavor, + mock_networkstatus_get_latest_consensus_by_flavor); + MOCK(get_estimated_address_per_node, + mock_get_estimated_address_per_node); + + dummy_ns = tor_malloc_zero(sizeof(*dummy_ns)); + dummy_ns->flavor = FLAV_MICRODESC; + dummy_ns->routerstatus_list = smartlist_new(); + + tor_addr_t addr_v4, addr_v6, dummy_addr; + tor_addr_parse(&addr_v4, "42.42.42.42"); + uint32_t ipv4h = tor_addr_to_ipv4h(&addr_v4); + tor_addr_parse(&addr_v6, "1:2:3:4::"); + memset(&dummy_addr, 'A', sizeof(dummy_addr)); + + /* This will make the nodelist bloom filter very large + * (the_nodelist->node_addrs) so we will fail the contain test rarely. */ + addr_per_node = 1024; + + /* No node no nothing. The lookups should be empty. */ + nodelist_set_consensus(dummy_ns); + + /* The address set should be empty. */ + ret = nodelist_probably_contains_address(&addr_v4); + tt_int_op(ret, OP_EQ, 0); + ret = nodelist_probably_contains_address(&addr_v6); + tt_int_op(ret, OP_EQ, 0); + dummy_addr.family = AF_INET; + ret = nodelist_probably_contains_address(&dummy_addr); + tt_int_op(ret, OP_EQ, 0); + dummy_addr.family = AF_INET6; + ret = nodelist_probably_contains_address(&dummy_addr); + tt_int_op(ret, OP_EQ, 0); + + md = tor_malloc_zero(sizeof(*md)); + ri = tor_malloc_zero(sizeof(*ri)); + rs = tor_malloc_zero(sizeof(*rs)); + crypto_rand(rs->identity_digest, sizeof(rs->identity_digest)); + crypto_rand(md->digest, sizeof(md->digest)); + memcpy(rs->descriptor_digest, md->digest, DIGEST256_LEN); + + /* Setup the rs, ri and md addresses. */ + rs->addr = ipv4h; + tor_addr_parse(&rs->ipv6_addr, "1:2:3:4::"); + ri->addr = ipv4h; + tor_addr_parse(&ri->ipv6_addr, "1:2:3:4::"); + tor_addr_parse(&md->ipv6_addr, "1:2:3:4::"); + + /* Add the rs to the consensus becoming a node_t. */ + smartlist_add(dummy_ns->routerstatus_list, rs); + nodelist_set_consensus(dummy_ns); + + /* At this point, the address set should be initialized in the nodelist and + * we should be able to lookup. */ + ret = nodelist_probably_contains_address(&addr_v4); + tt_int_op(ret, OP_EQ, 1); + ret = nodelist_probably_contains_address(&addr_v6); + tt_int_op(ret, OP_EQ, 1); + /* Lookup unknown address. */ + dummy_addr.family = AF_INET; + ret = nodelist_probably_contains_address(&dummy_addr); + tt_int_op(ret, OP_EQ, 0); + dummy_addr.family = AF_INET6; + ret = nodelist_probably_contains_address(&dummy_addr); + tt_int_op(ret, OP_EQ, 0); + + done: + routerstatus_free(rs); routerinfo_free(ri); microdesc_free(md); + smartlist_clear(dummy_ns->routerstatus_list); + networkstatus_vote_free(dummy_ns); + UNMOCK(networkstatus_get_latest_consensus); + UNMOCK(networkstatus_get_latest_consensus_by_flavor); + UNMOCK(get_estimated_address_per_node); +} + +struct testcase_t address_set_tests[] = { + { "contains", test_contains, TT_FORK, + NULL, NULL }, + { "nodelist", test_nodelist, TT_FORK, + NULL, NULL }, + + END_OF_TESTCASES +}; + diff --git a/src/test/test_dos.c b/src/test/test_dos.c index 6db98b9ed3..cb9d9e559c 100644 --- a/src/test/test_dos.c +++ b/src/test/test_dos.c @@ -10,9 +10,36 @@ #include "circuitlist.h" #include "geoip.h" #include "channel.h" +#include "microdesc.h" +#include "networkstatus.h" +#include "nodelist.h" +#include "routerlist.h" #include "test.h" #include "log_test_helpers.h" +static networkstatus_t *dummy_ns = NULL; +static networkstatus_t * +mock_networkstatus_get_latest_consensus(void) +{ + return dummy_ns; +} + +static networkstatus_t * +mock_networkstatus_get_latest_consensus_by_flavor(consensus_flavor_t f) +{ + tor_assert(f == FLAV_MICRODESC); + return dummy_ns; +} + +/* Number of address a single node_t can have. Default to the production + * value. This is to control the size of the bloom filter. */ +static int addr_per_node = 2; +static int +mock_get_estimated_address_per_node(void) +{ + return addr_per_node; +} + static unsigned int mock_enable_dos_protection(const networkstatus_t *ns) { @@ -385,10 +412,86 @@ test_dos_bucket_refill(void *arg) dos_free_all(); } +/* Test if we avoid counting a known relay. */ +static void +test_known_relay(void *arg) +{ + clientmap_entry_t *entry = NULL; + routerstatus_t *rs = NULL; microdesc_t *md = NULL; routerinfo_t *ri = NULL; + + (void) arg; + + MOCK(networkstatus_get_latest_consensus, + mock_networkstatus_get_latest_consensus); + MOCK(networkstatus_get_latest_consensus_by_flavor, + mock_networkstatus_get_latest_consensus_by_flavor); + MOCK(get_estimated_address_per_node, + mock_get_estimated_address_per_node); + MOCK(get_param_cc_enabled, mock_enable_dos_protection); + + dos_init(); + + dummy_ns = tor_malloc_zero(sizeof(*dummy_ns)); + dummy_ns->flavor = FLAV_MICRODESC; + dummy_ns->routerstatus_list = smartlist_new(); + + /* Setup an OR conn so we can pass it to the DoS subsystem. */ + or_connection_t or_conn; + tor_addr_parse(&or_conn.real_addr, "42.42.42.42"); + + rs = tor_malloc_zero(sizeof(*rs)); + rs->addr = tor_addr_to_ipv4h(&or_conn.real_addr); + crypto_rand(rs->identity_digest, sizeof(rs->identity_digest)); + smartlist_add(dummy_ns->routerstatus_list, rs); + + /* This will make the nodelist bloom filter very large + * (the_nodelist->node_addrs) so we will fail the contain test rarely. */ + addr_per_node = 1024; + nodelist_set_consensus(dummy_ns); + + /* We have now a node in our list so we'll make sure we don't count it as a + * client connection. */ + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &or_conn.real_addr, NULL, 0); + /* Suppose we have 5 connections in rapid succession, the counter should + * always be 0 because we should ignore this. */ + dos_new_client_conn(&or_conn); + dos_new_client_conn(&or_conn); + dos_new_client_conn(&or_conn); + dos_new_client_conn(&or_conn); + dos_new_client_conn(&or_conn); + entry = geoip_lookup_client(&or_conn.real_addr, NULL, GEOIP_CLIENT_CONNECT); + tt_assert(entry); + /* We should have a count of 0. */ + tt_uint_op(entry->dos_stats.concurrent_count, OP_EQ, 0); + + /* To make sure that his is working properly, make a unknown client + * connection and see if we do get it. */ + tor_addr_parse(&or_conn.real_addr, "42.42.42.43"); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &or_conn.real_addr, NULL, 0); + dos_new_client_conn(&or_conn); + dos_new_client_conn(&or_conn); + entry = geoip_lookup_client(&or_conn.real_addr, NULL, GEOIP_CLIENT_CONNECT); + tt_assert(entry); + /* We should have a count of 2. */ + tt_uint_op(entry->dos_stats.concurrent_count, OP_EQ, 2); + + done: + routerstatus_free(rs); routerinfo_free(ri); microdesc_free(md); + smartlist_clear(dummy_ns->routerstatus_list); + networkstatus_vote_free(dummy_ns); + dos_free_all(); + UNMOCK(networkstatus_get_latest_consensus); + UNMOCK(networkstatus_get_latest_consensus_by_flavor); + UNMOCK(get_estimated_address_per_node); + UNMOCK(get_param_cc_enabled); +} + struct testcase_t dos_tests[] = { { "conn_creation", test_dos_conn_creation, TT_FORK, NULL, NULL }, { "circuit_creation", test_dos_circuit_creation, TT_FORK, NULL, NULL }, { "bucket_refill", test_dos_bucket_refill, TT_FORK, NULL, NULL }, + { "known_relay" , test_known_relay, TT_FORK, + NULL, NULL }, END_OF_TESTCASES }; diff --git a/src/win32/orconfig.h b/src/win32/orconfig.h index a4767e54c0..218cc36ec9 100644 --- a/src/win32/orconfig.h +++ b/src/win32/orconfig.h @@ -218,7 +218,7 @@ #define USING_TWOS_COMPLEMENT /* Version number of package */ -#define VERSION "0.3.3.1-alpha-dev" +#define VERSION "0.3.3.2-alpha-dev" |