diff options
203 files changed, 5849 insertions, 3903 deletions
@@ -1,3 +1,765 @@ +Changes in version 0.3.3.4-alpha - 2018-03-29 + Tor 0.3.3.4-alpha includes various bugfixes for issues found during + the alpha testing of earlier releases in its series. We are + approaching a stable 0.3.3 release: more testing is welcome! + + o New system requirements: + - When built with Rust, Tor now depends on version 0.2.39 of the + libc crate. Closes tickets 25310 and 25664. + + o Major bugfixes (relay, connection): + - If we have failed to connect to a relay and received a connection + refused, timeout, or similar error (at the TCP level), do not try + that same address/port again for 60 seconds after the failure has + occurred. Fixes bug 24767; bugfix on 0.0.6. + + o Minor features (geoip): + - Update geoip and geoip6 to the March 8 2018 Maxmind GeoLite2 + Country database. Closes ticket 25469. + + o Minor features (log messages): + - Improve log message in the out-of-memory handler to include + information about memory usage from the different compression + backends. Closes ticket 25372. + + o Minor features (sandbox): + - Explicitly permit the poll() system call when the Linux + seccomp2-based sandbox is enabled: apparently, some versions of + libc use poll() when calling getpwnam(). Closes ticket 25313. + + o Minor bugfixes (C correctness): + - Fix a very unlikely (impossible, we believe) null pointer + dereference. Fixes bug 25629; bugfix on 0.2.9.15. Found by + Coverity; this is CID 1430932. + + o Minor bugfixes (channel, client): + - Better identify client connection when reporting to the geoip + client cache. Fixes bug 24904; bugfix on 0.3.1.7. + + o Minor bugfixes (compilation): + - Fix a C99 compliance issue in our configuration script that caused + compilation issues when compiling Tor with certain versions of + xtools. Fixes bug 25474; bugfix on 0.3.2.5-alpha. + + o Minor bugfixes (controller, reliability): + - Avoid a (nonfatal) assertion failure when extending a one-hop + circuit from the controller to become a multihop circuit. Fixes + bug 24903; bugfix on 0.2.5.2-alpha. + + o Minor bugfixes (networking): + - Tor will no longer reject IPv6 address strings from TorBrowser + when they are passed as hostnames in SOCKS5 requests. Fixes bug + 25036, bugfix on Tor 0.3.1.2. + - string_is_valid_hostname() will not consider IP strings to be + valid hostnames. Fixes bug 25055; bugfix on Tor 0.2.5.5. + + o Minor bugfixes (onion service v3): + - Avoid an assertion failure when the next the next onion service + descriptor rotation type is out of sync with the consensus's + valid-after time. Instead, log a warning message with extra + information, so we can better hunt down the cause of this + assertion. Fixes bug 25306; bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (testing): + - Avoid intermittent test failures due to a test that had relied on + onion service introduction point creation finishing within 5 + seconds of real clock time. Fixes bug 25450; bugfix + on 0.3.1.3-alpha. + - Rust crates are now automatically detected and tested. Previously, + some crates were not tested by `make test-rust` due to a static + string in the `src/test/test_rust.sh` script specifying which + crates to test. Fixes bug 25560; bugfix on 0.3.3.3-alpha. + + o Minor bugfixes (testing, benchmarks): + - Fix a crash when running benchmark tests on win32 systems. The + crash was due to a mutex that wasn't initialized before logging + and options were initialized. Fixes bug 25479; bugfix + on 0.3.3.3-alpha. + + o Minor bugfixes (warnings, ipv6): + - Avoid a bug warning that could occur when trying to connect to a + relay over IPv6. This warning would occur on a Tor instance that + downloads router descriptors, but prefers to use microdescriptors. + Fixes bug 25213; bugfix on 0.3.3.1-alpha. + + o Code simplification and refactoring: + - Remove the old (deterministic) directory retry logic entirely: + We've used exponential backoff exclusively for some time. Closes + ticket 23814. + + o Documentation: + - Improved the documentation of AccountingStart parameter. Closes + ticket 23635. + - Update the documentation for "Log" to include the current list of + logging domains. Closes ticket 25378. + + +Changes in version 0.3.3.3-alpha - 2018-03-03 + Tor 0.3.3.3-alpha is the third alpha release for the 0.3.3.x series. + It includes an important security fix for a remote crash attack + against directory authorities tracked as TROVE-2018-001. + + Additionally, with this release, we are upgrading the severity of a + bug fixed in 0.3.3.2-alpha. Bug 24700, which was fixed in + 0.3.3.2-alpha, can be remotely triggered in order to crash relays with + a use-after-free pattern. As such, we are now tracking that bug as + TROVE-2018-002 and CVE-2018-0491. This bug affected versions + 0.3.2.1-alpha through 0.3.2.9, as well as 0.3.3.1-alpha. + + This release also fixes several minor bugs and annoyances from + earlier releases. + + Relays running 0.3.2.x should upgrade to one of the versions released + today, for the fix to TROVE-2018-002. Directory authorities should + also upgrade. (Relays on earlier versions might want to update too for + the DoS mitigations.) + + o Major bugfixes (denial-of-service, directory authority): + - Fix a protocol-list handling bug that could be used to remotely crash + directory authorities with a null-pointer exception. Fixes bug 25074; + bugfix on 0.2.9.4-alpha. Also tracked as TROVE-2018-001 and + CVE-2018-0490. + + o Minor features (compatibility, OpenSSL): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (logging): + - Clarify the log messages produced when getrandom() or a related + entropy-generation mechanism gives an error. Closes ticket 25120. + + o Minor features (testing): + - Add a "make test-rust" target to run the rust tests only. Closes + ticket 25071. + + o Minor bugfixes (denial-of-service): + - Fix a possible crash on malformed consensus. If a consensus had + contained an unparseable protocol line, it could have made clients + and relays crash with a null-pointer exception. To exploit this + issue, however, an attacker would need to be able to subvert the + directory authority system. Fixes bug 25251; bugfix on + 0.2.9.4-alpha. Also tracked as TROVE-2018-004. + + o Minor bugfixes (DoS mitigation): + - Add extra safety checks when refilling the circuit creation bucket + to ensure we never set a value above the allowed maximum burst. + Fixes bug 25202; bugfix on 0.3.3.2-alpha. + - When a new consensus arrives, don't update our DoS-mitigation + parameters if we aren't a public relay. Fixes bug 25223; bugfix + on 0.3.3.2-alpha. + + o Minor bugfixes (man page, SocksPort): + - Remove dead code from the old "SocksSocket" option, and rename + SocksSocketsGroupWritable to UnixSocksGroupWritable. The old option + still works, but is deprecated. Fixes bug 24343; bugfix on 0.2.6.3. + + o Minor bugfixes (performance): + - Reduce the number of circuits that will be opened at once during + the circuit build timeout phase. This is done by increasing the + idle timeout to 3 minutes, and lowering the maximum number of + concurrent learning circuits to 10. Fixes bug 24769; bugfix + on 0.3.1.1-alpha. + + o Minor bugfixes (spec conformance): + - Forbid "-0" as a protocol version. Fixes part of bug 25249; bugfix on + 0.2.9.4-alpha. + - Forbid UINT32_MAX as a protocol version. Fixes part of bug 25249; + bugfix on 0.2.9.4-alpha. + + o Minor bugfixes (spec conformance, rust): + - Resolve a denial-of-service issue caused by an infinite loop in + the rust protover code. Fixes bug 25250, bugfix on 0.3.3.1-alpha. + Also tracked as TROVE-2018-003. + + o Code simplification and refactoring: + - Update the "rust dependencies" submodule to be a project-level + repository, rather than a user repository. Closes ticket 25323. + + +Changes in version 0.3.2.10 - 2018-03-03 + Tor 0.3.2.10 is the second stable release in the 0.3.2 series. It + backports a number of bugfixes, including important fixes for security + issues. + + It includes an important security fix for a remote crash attack + against directory authorities, tracked as TROVE-2018-001. + + Additionally, it backports a fix for a bug whose severity we have + upgraded: Bug 24700, which was fixed in 0.3.3.2-alpha, can be remotely + triggered in order to crash relays with a use-after-free pattern. As + such, we are now tracking that bug as TROVE-2018-002 and + CVE-2018-0491, and backporting it to earlier releases. This bug + affected versions 0.3.2.1-alpha through 0.3.2.9, as well as version + 0.3.3.1-alpha. + + This release also backports our new system for improved resistance to + denial-of-service attacks against relays. + + This release also fixes several minor bugs and annoyances from + earlier releases. + + Relays running 0.3.2.x SHOULD upgrade to one of the versions released + today, for the fix to TROVE-2018-002. Directory authorities should + also upgrade. (Relays on earlier versions might want to update too for + the DoS mitigations.) + + o Major bugfixes (denial-of-service, directory authority, backport from 0.3.3.3-alpha): + - Fix a protocol-list handling bug that could be used to remotely crash + directory authorities with a null-pointer exception. Fixes bug 25074; + bugfix on 0.2.9.4-alpha. Also tracked as TROVE-2018-001 and + CVE-2018-0490. + + o Major bugfixes (scheduler, KIST, denial-of-service, backport from 0.3.3.2-alpha): + - Avoid adding the same channel twice in the KIST scheduler pending + list, which could lead to remote denial-of-service use-after-free + attacks against relays. Fixes bug 24700; bugfix on 0.3.2.1-alpha. + + o Major features (denial-of-service mitigation, backport from 0.3.3.2-alpha): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Major bugfixes (onion services, retry behavior, backport from 0.3.3.1-alpha): + - Fix an "off by 2" error in counting rendezvous failures on the + onion service side. While we thought we would stop the rendezvous + attempt after one failed circuit, we were actually making three + circuit attempts before giving up. Now switch to a default of 2, + and allow the consensus parameter "hs_service_max_rdv_failures" to + override. Fixes bug 24895; bugfix on 0.0.6. + - New-style (v3) onion services now obey the "max rendezvous circuit + attempts" logic. Previously they would make as many rendezvous + circuit attempts as they could fit in the MAX_REND_TIMEOUT second + window before giving up. Fixes bug 24894; bugfix on 0.3.2.1-alpha. + + o Major bugfixes (protocol versions, backport from 0.3.3.2-alpha): + - Add Link protocol version 5 to the supported protocols list. Fixes + bug 25070; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (relay, backport from 0.3.3.1-alpha): + - Fix a set of false positives where relays would consider + connections to other relays as being client-only connections (and + thus e.g. deserving different link padding schemes) if those + relays fell out of the consensus briefly. Now we look only at the + initial handshake and whether the connection authenticated as a + relay. Fixes bug 24898; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (scheduler, consensus, backport from 0.3.3.2-alpha): + - The scheduler subsystem was failing to promptly notice changes in + consensus parameters, making it harder to switch schedulers + network-wide. Fixes bug 24975; bugfix on 0.3.2.1-alpha. + + o Minor features (denial-of-service avoidance, backport from 0.3.3.2-alpha): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor features (compatibility, OpenSSL, backport from 0.3.3.3-alpha): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor features (logging, diagnostic, backport from 0.3.3.2-alpha): + - When logging a failure to check a hidden service's certificate, + also log what the problem with the certificate was. Diagnostic + for ticket 24972. + + o Minor bugfix (channel connection, backport from 0.3.3.2-alpha): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfixes (denial-of-service, backport from 0.3.3.3-alpha): + - Fix a possible crash on malformed consensus. If a consensus had + contained an unparseable protocol line, it could have made clients + and relays crash with a null-pointer exception. To exploit this + issue, however, an attacker would need to be able to subvert the + directory authority system. Fixes bug 25251; bugfix on + 0.2.9.4-alpha. Also tracked as TROVE-2018-004. + + o Minor bugfix (directory authority, backport from 0.3.3.2-alpha): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (build, rust, backport from 0.3.3.1-alpha): + - When building with Rust on OSX, link against libresolv, to work + around the issue at https://github.com/rust-lang/rust/issues/46797. + Fixes bug 24652; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (onion services, backport from 0.3.3.2-alpha): + - Remove a BUG() statement when a client fetches an onion descriptor + that has a lower revision counter than the one in its cache. This + can happen in normal circumstances due to HSDir desync. Fixes bug + 24976; bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (logging, backport from 0.3.3.2-alpha): + - Don't treat inability to store a cached consensus object as a bug: + it can happen normally when we are out of disk space. Fixes bug + 24859; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (performance, fragile-hardening, backport from 0.3.3.1-alpha): + - Improve the performance of our consensus-diff application code + when Tor is built with the --enable-fragile-hardening option set. + Fixes bug 24826; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (OSX, backport from 0.3.3.1-alpha): + - Don't exit the Tor process if setrlimit() fails to change the file + limit (which can happen sometimes on some versions of OSX). Fixes + bug 21074; bugfix on 0.0.9pre5. + + o Minor bugfixes (spec conformance, backport from 0.3.3.3-alpha): + - Forbid "-0" as a protocol version. Fixes part of bug 25249; bugfix on + 0.2.9.4-alpha. + - Forbid UINT32_MAX as a protocol version. Fixes part of bug 25249; + bugfix on 0.2.9.4-alpha. + + o Minor bugfixes (testing, backport from 0.3.3.1-alpha): + - Fix a memory leak in the scheduler/loop_kist unit test. Fixes bug + 25005; bugfix on 0.3.2.7-rc. + + o Minor bugfixes (v3 onion services, backport from 0.3.3.2-alpha): + - Look at the "HSRend" protocol version, not the "HSDir" protocol + version, when deciding whether a consensus entry can support the + v3 onion service protocol as a rendezvous point. Fixes bug 25105; + bugfix on 0.3.2.1-alpha. + + o Code simplification and refactoring (backport from 0.3.3.3-alpha): + - Update the "rust dependencies" submodule to be a project-level + repository, rather than a user repository. Closes ticket 25323. + + o Documentation (backport from 0.3.3.1-alpha) + - Document that operators who run more than one relay or bridge are + expected to set MyFamily and ContactInfo correctly. Closes + ticket 24526. + + +Changes in version 0.3.1.10 - 2018-03-03 + Tor 0.3.1.10 backports a number of bugfixes, including important fixes for + security issues. + + It includes an important security fix for a remote crash attack + against directory authorities, tracked as TROVE-2018-001. + + This release also backports our new system for improved resistance to + denial-of-service attacks against relays. + + This release also fixes several minor bugs and annoyances from + earlier releases. + + All directory authorities should upgrade to one of the versions + released today. Relays running 0.3.1.x may wish to update to one of + the versions released today, for the DoS mitigations. + + Please note: according to our release calendar, Tor 0.3.1 will no + longer be supported after 1 July 2018. If you will be running Tor + after that date, you should make sure to plan to upgrade to the latest + stable version, or downgrade to 0.2.9 (which will receive long-term + support). + + o Major bugfixes (denial-of-service, directory authority, backport from 0.3.3.3-alpha): + - Fix a protocol-list handling bug that could be used to remotely crash + directory authorities with a null-pointer exception. Fixes bug 25074; + bugfix on 0.2.9.4-alpha. Also tracked as TROVE-2018-001 and + CVE-2018-0490. + + o Major features (denial-of-service mitigation, backport from 0.3.3.2-alpha): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Minor features (linux seccomp2 sandbox, backport from 0.3.2.5-alpha): + - Update the sandbox rules so that they should now work correctly + with Glibc 2.26. Closes ticket 24315. + + o Major bugfixes (onion services, retry behavior, backport from 0.3.3.1-alpha): + - Fix an "off by 2" error in counting rendezvous failures on the + onion service side. While we thought we would stop the rendezvous + attempt after one failed circuit, we were actually making three + circuit attempts before giving up. Now switch to a default of 2, + and allow the consensus parameter "hs_service_max_rdv_failures" to + override. Fixes bug 24895; bugfix on 0.0.6. + + o Major bugfixes (protocol versions, backport from 0.3.3.2-alpha): + - Add Link protocol version 5 to the supported protocols list. Fixes + bug 25070; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (relay, backport from 0.3.3.1-alpha): + - Fix a set of false positives where relays would consider + connections to other relays as being client-only connections (and + thus e.g. deserving different link padding schemes) if those + relays fell out of the consensus briefly. Now we look only at the + initial handshake and whether the connection authenticated as a + relay. Fixes bug 24898; bugfix on 0.3.1.1-alpha. + + o Minor features (denial-of-service avoidance, backport from 0.3.3.2-alpha): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor feature (relay statistics, backport from 0.3.2.6-alpha): + - Change relay bandwidth reporting stats interval from 4 hours to 24 + hours in order to reduce the efficiency of guard discovery + attacks. Fixes ticket 23856. + + o Minor features (compatibility, OpenSSL, backport from 0.3.3.3-alpha): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (fallback directory mirrors, backport from 0.3.2.9): + - The fallback directory list has been re-generated based on the + current status of the network. Tor uses fallback directories to + bootstrap when it doesn't yet have up-to-date directory + information. Closes ticket 24801. + - Make the default DirAuthorityFallbackRate 0.1, so that clients + prefer to bootstrap from fallback directory mirrors. This is a + follow-up to 24679, which removed weights from the default + fallbacks. Implements ticket 24681. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor bugfix (channel connection, backport from 0.3.3.2-alpha): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfix (directory authority, backport from 0.3.3.2-alpha): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (address selection, backport from 0.3.2.9): + - When the fascist_firewall_choose_address_ functions don't find a + reachable address, set the returned address to the null address + and port. This is a precautionary measure, because some callers do + not check the return value. Fixes bug 24736; bugfix + on 0.2.8.2-alpha. + + o Major bugfixes (bootstrapping, backport from 0.3.2.5-alpha): + - Fetch descriptors aggressively whenever we lack enough to build + circuits, regardless of how many descriptors we are missing. + Previously, we would delay launching the fetch when we had fewer + than 15 missing descriptors, even if some of those descriptors + were blocking circuits from building. Fixes bug 23985; bugfix on + 0.1.1.11-alpha. The effects of this bug became worse in + 0.3.0.3-alpha, when we began treating missing descriptors from our + primary guards as a reason to delay circuits. + - Don't try fetching microdescriptors from relays that have failed + to deliver them in the past. Fixes bug 23817; bugfix + on 0.3.0.1-alpha. + + o Minor bugfixes (compilation, backport from 0.3.2.7-rc): + - Fix a signed/unsigned comparison warning introduced by our fix to + TROVE-2017-009. Fixes bug 24480; bugfix on 0.2.5.16. + + o Minor bugfixes (control port, linux seccomp2 sandbox, backport from 0.3.2.5-alpha): + - Avoid a crash when attempting to use the seccomp2 sandbox together + with the OwningControllerProcess feature. Fixes bug 24198; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (denial-of-service, backport from 0.3.3.3-alpha): + - Fix a possible crash on malformed consensus. If a consensus had + contained an unparseable protocol line, it could have made clients + and relays crash with a null-pointer exception. To exploit this + issue, however, an attacker would need to be able to subvert the + directory authority system. Fixes bug 25251; bugfix on + 0.2.9.4-alpha. Also tracked as TROVE-2018-004. + + o Minor bugfixes (directory cache, backport from 0.3.2.5-alpha): + - Recover better from empty or corrupt files in the consensus cache + directory. Fixes bug 24099; bugfix on 0.3.1.1-alpha. + - When a consensus diff calculation is only partially successful, + only record the successful parts as having succeeded. Partial + success can happen if (for example) one compression method fails + but the others succeed. Previously we misrecorded all the + calculations as having succeeded, which would later cause a + nonfatal assertion failure. Fixes bug 24086; bugfix + on 0.3.1.1-alpha. + + o Minor bugfixes (entry guards, backport from 0.3.2.3-alpha): + - Tor now updates its guard state when it reads a consensus + regardless of whether it's missing descriptors. That makes tor use + its primary guards to fetch descriptors in some edge cases where + it would previously have used fallback directories. Fixes bug + 23862; bugfix on 0.3.0.1-alpha. + + o Minor bugfixes (logging, backport from 0.3.3.2-alpha): + - Don't treat inability to store a cached consensus object as a bug: + it can happen normally when we are out of disk space. Fixes bug + 24859; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (memory usage, backport from 0.3.2.8-rc): + - When queuing DESTROY cells on a channel, only queue the circuit-id + and reason fields: not the entire 514-byte cell. This fix should + help mitigate any bugs or attacks that fill up these queues, and + free more RAM for other uses. Fixes bug 24666; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (network layer, backport from 0.3.2.5-alpha): + - When closing a connection via close_connection_immediately(), we + mark it as "not blocked on bandwidth", to prevent later calls from + trying to unblock it, and give it permission to read. This fixes a + backtrace warning that can happen on relays under various + circumstances. Fixes bug 24167; bugfix on 0.1.0.1-rc. + + o Minor bugfixes (path selection, backport from 0.3.2.4-alpha): + - When selecting relays by bandwidth, avoid a rounding error that + could sometimes cause load to be imbalanced incorrectly. + Previously, we would always round upwards; now, we round towards + the nearest integer. This had the biggest effect when a relay's + weight adjustments should have given it weight 0, but it got + weight 1 instead. Fixes bug 23318; bugfix on 0.2.4.3-alpha. + - When calculating the fraction of nodes that have descriptors, and + all nodes in the network have zero bandwidths, count the number of + nodes instead. Fixes bug 23318; bugfix on 0.2.4.10-alpha. + - Actually log the total bandwidth in compute_weighted_bandwidths(). + Fixes bug 24170; bugfix on 0.2.4.3-alpha. + + o Minor bugfixes (performance, fragile-hardening, backport from 0.3.3.1-alpha): + - Improve the performance of our consensus-diff application code + when Tor is built with the --enable-fragile-hardening option set. + Fixes bug 24826; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (OSX, backport from 0.3.3.1-alpha): + - Don't exit the Tor process if setrlimit() fails to change the file + limit (which can happen sometimes on some versions of OSX). Fixes + bug 21074; bugfix on 0.0.9pre5. + + o Minor bugfixes (portability, msvc, backport from 0.3.2.9): + - Fix a bug in the bit-counting parts of our timing-wheel code on + MSVC. (Note that MSVC is still not a supported build platform, due + to cyptographic timing channel risks.) Fixes bug 24633; bugfix + on 0.2.9.1-alpha. + + o Minor bugfixes (relay, partial backport): + - Make the internal channel_is_client() function look at what sort + of connection handshake the other side used, rather than whether + the other side ever sent a create_fast cell to us. Backports part + of the fixes from bugs 22805 and 24898. + + o Minor bugfixes (spec conformance, backport from 0.3.3.3-alpha): + - Forbid "-0" as a protocol version. Fixes part of bug 25249; bugfix on + 0.2.9.4-alpha. + - Forbid UINT32_MAX as a protocol version. Fixes part of bug 25249; + bugfix on 0.2.9.4-alpha. + + o Code simplification and refactoring (backport from 0.3.3.3-alpha): + - Update the "rust dependencies" submodule to be a project-level + repository, rather than a user repository. Closes ticket 25323. + + +Changes in version 0.2.9.15 - 2018-03-03 + Tor 0.2.9.15 backports important security and stability bugfixes from + later Tor releases. + + It includes an important security fix for a remote crash attack + against directory authorities, tracked as TROVE-2018-001. + + This release also backports our new system for improved resistance to + denial-of-service attacks against relays. + + This release also fixes several minor bugs and annoyances from + earlier releases. + + All directory authorities should upgrade to one of the versions + released today. Relays running 0.2.9.x may wish to update to one of + the versions released today, for the DoS mitigations. + + o Major bugfixes (denial-of-service, directory authority, backport from 0.3.3.3-alpha): + - Fix a protocol-list handling bug that could be used to remotely crash + directory authorities with a null-pointer exception. Fixes bug 25074; + bugfix on 0.2.9.4-alpha. Also tracked as TROVE-2018-001 and + CVE-2018-0490. + + o Major features (denial-of-service mitigation): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Major bugfixes (bootstrapping): + - Fetch descriptors aggressively whenever we lack enough to build + circuits, regardless of how many descriptors we are missing. + Previously, we would delay launching the fetch when we had fewer + than 15 missing descriptors, even if some of those descriptors + were blocking circuits from building. Fixes bug 23985; bugfix on + 0.1.1.11-alpha. The effects of this bug became worse in + 0.3.0.3-alpha, when we began treating missing descriptors from our + primary guards as a reason to delay circuits. + + o Major bugfixes (onion services, retry behavior): + - Fix an "off by 2" error in counting rendezvous failures on the + onion service side. While we thought we would stop the rendezvous + attempt after one failed circuit, we were actually making three + circuit attempts before giving up. Now switch to a default of 2, + and allow the consensus parameter "hs_service_max_rdv_failures" to + override. Fixes bug 24895; bugfix on 0.0.6. + + o Minor feature (relay statistics): + - Change relay bandwidth reporting stats interval from 4 hours to 24 + hours in order to reduce the efficiency of guard discovery + attacks. Fixes ticket 23856. + + o Minor features (compatibility, OpenSSL): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (denial-of-service avoidance): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor features (fallback directory mirrors): + - The fallback directory list has been re-generated based on the + current status of the network. Tor uses fallback directories to + bootstrap when it doesn't yet have up-to-date directory + information. Closes ticket 24801. + - Make the default DirAuthorityFallbackRate 0.1, so that clients + prefer to bootstrap from fallback directory mirrors. This is a + follow-up to 24679, which removed weights from the default + fallbacks. Implements ticket 24681. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor features (linux seccomp2 sandbox): + - Update the sandbox rules so that they should now work correctly + with Glibc 2.26. Closes ticket 24315. + + o Minor bugfix (channel connection): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfix (directory authority): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (address selection): + - When the fascist_firewall_choose_address_ functions don't find a + reachable address, set the returned address to the null address + and port. This is a precautionary measure, because some callers do + not check the return value. Fixes bug 24736; bugfix + on 0.2.8.2-alpha. + + o Minor bugfixes (compilation): + - Fix a signed/unsigned comparison warning introduced by our fix to + TROVE-2017-009. Fixes bug 24480; bugfix on 0.2.5.16. + + o Minor bugfixes (control port, linux seccomp2 sandbox): + - Avoid a crash when attempting to use the seccomp2 sandbox together + with the OwningControllerProcess feature. Fixes bug 24198; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (denial-of-service, backport from 0.3.3.3-alpha): + - Fix a possible crash on malformed consensus. If a consensus had + contained an unparseable protocol line, it could have made clients + and relays crash with a null-pointer exception. To exploit this + issue, however, an attacker would need to be able to subvert the + directory authority system. Fixes bug 25251; bugfix on + 0.2.9.4-alpha. Also tracked as TROVE-2018-004. + + o Minor bugfixes (memory usage): + - When queuing DESTROY cells on a channel, only queue the circuit-id + and reason fields: not the entire 514-byte cell. This fix should + help mitigate any bugs or attacks that fill up these queues, and + free more RAM for other uses. Fixes bug 24666; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (network layer): + - When closing a connection via close_connection_immediately(), we + mark it as "not blocked on bandwidth", to prevent later calls from + trying to unblock it, and give it permission to read. This fixes a + backtrace warning that can happen on relays under various + circumstances. Fixes bug 24167; bugfix on 0.1.0.1-rc. + + o Minor bugfixes (OSX): + - Don't exit the Tor process if setrlimit() fails to change the file + limit (which can happen sometimes on some versions of OSX). Fixes + bug 21074; bugfix on 0.0.9pre5. + + o Minor bugfixes (path selection): + - When selecting relays by bandwidth, avoid a rounding error that + could sometimes cause load to be imbalanced incorrectly. + Previously, we would always round upwards; now, we round towards + the nearest integer. This had the biggest effect when a relay's + weight adjustments should have given it weight 0, but it got + weight 1 instead. Fixes bug 23318; bugfix on 0.2.4.3-alpha. + - When calculating the fraction of nodes that have descriptors, and + all nodes in the network have zero bandwidths, count the number of + nodes instead. Fixes bug 23318; bugfix on 0.2.4.10-alpha. + - Actually log the total bandwidth in compute_weighted_bandwidths(). + Fixes bug 24170; bugfix on 0.2.4.3-alpha. + + o Minor bugfixes (portability, msvc): + - Fix a bug in the bit-counting parts of our timing-wheel code on + MSVC. (Note that MSVC is still not a supported build platform, due + to cryptographic timing channel risks.) Fixes bug 24633; bugfix + on 0.2.9.1-alpha. + + o Minor bugfixes (relay): + - Make the internal channel_is_client() function look at what sort + of connection handshake the other side used, rather than whether + the other side ever sent a create_fast cell to us. Backports part + of the fixes from bugs 22805 and 24898. + + o Minor bugfixes (spec conformance, backport from 0.3.3.3-alpha): + - Forbid "-0" as a protocol version. Fixes part of bug 25249; bugfix on + 0.2.9.4-alpha. + - Forbid UINT32_MAX as a protocol version. Fixes part of bug 25249; + bugfix on 0.2.9.4-alpha. + + Changes in version 0.3.3.2-alpha - 2018-02-10 Tor 0.3.3.2-alpha is the second alpha in the 0.3.3.x series. It introduces a mechanism to handle the high loads that many relay @@ -3143,7 +3905,7 @@ Changes in version 0.3.0.8 - 2017-06-08 o Major bugfixes (hidden service v3, backport from 0.3.1.1-alpha): - Stop rejecting v3 hidden service descriptors because their size did not match an old padding rule. Fixes bug 22447; bugfix on - tor-0.3.0.1-alpha. + 0.3.0.1-alpha. o Minor features (fallback directory list, backport from 0.3.1.3-alpha): - Replace the 177 fallbacks originally introduced in Tor 0.2.9.8 in diff --git a/ReleaseNotes b/ReleaseNotes index 6c31d8ab0c..93ff6d6ff6 100644 --- a/ReleaseNotes +++ b/ReleaseNotes @@ -2,6 +2,586 @@ This document summarizes new features and bugfixes in each stable release of Tor. If you want to see more detailed descriptions of the changes in each development snapshot, see the ChangeLog file. +Changes in version 0.3.2.10 - 2018-03-03 + Tor 0.3.2.10 is the second stable release in the 0.3.2 series. It + backports a number of bugfixes, including important fixes for security + issues. + + It includes an important security fix for a remote crash attack + against directory authorities, tracked as TROVE-2018-001. + + Additionally, it backports a fix for a bug whose severity we have + upgraded: Bug 24700, which was fixed in 0.3.3.2-alpha, can be remotely + triggered in order to crash relays with a use-after-free pattern. As + such, we are now tracking that bug as TROVE-2018-002 and + CVE-2018-0491, and backporting it to earlier releases. This bug + affected versions 0.3.2.1-alpha through 0.3.2.9, as well as version + 0.3.3.1-alpha. + + This release also backports our new system for improved resistance to + denial-of-service attacks against relays. + + This release also fixes several minor bugs and annoyances from + earlier releases. + + Relays running 0.3.2.x SHOULD upgrade to one of the versions released + today, for the fix to TROVE-2018-002. Directory authorities should + also upgrade. (Relays on earlier versions might want to update too for + the DoS mitigations.) + + o Major bugfixes (denial-of-service, directory authority, backport from 0.3.3.3-alpha): + - Fix a protocol-list handling bug that could be used to remotely crash + directory authorities with a null-pointer exception. Fixes bug 25074; + bugfix on 0.2.9.4-alpha. Also tracked as TROVE-2018-001 and + CVE-2018-0490. + + o Major bugfixes (scheduler, KIST, denial-of-service, backport from 0.3.3.2-alpha): + - Avoid adding the same channel twice in the KIST scheduler pending + list, which could lead to remote denial-of-service use-after-free + attacks against relays. Fixes bug 24700; bugfix on 0.3.2.1-alpha. + + o Major features (denial-of-service mitigation, backport from 0.3.3.2-alpha): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Major bugfixes (onion services, retry behavior, backport from 0.3.3.1-alpha): + - Fix an "off by 2" error in counting rendezvous failures on the + onion service side. While we thought we would stop the rendezvous + attempt after one failed circuit, we were actually making three + circuit attempts before giving up. Now switch to a default of 2, + and allow the consensus parameter "hs_service_max_rdv_failures" to + override. Fixes bug 24895; bugfix on 0.0.6. + - New-style (v3) onion services now obey the "max rendezvous circuit + attempts" logic. Previously they would make as many rendezvous + circuit attempts as they could fit in the MAX_REND_TIMEOUT second + window before giving up. Fixes bug 24894; bugfix on 0.3.2.1-alpha. + + o Major bugfixes (protocol versions, backport from 0.3.3.2-alpha): + - Add Link protocol version 5 to the supported protocols list. Fixes + bug 25070; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (relay, backport from 0.3.3.1-alpha): + - Fix a set of false positives where relays would consider + connections to other relays as being client-only connections (and + thus e.g. deserving different link padding schemes) if those + relays fell out of the consensus briefly. Now we look only at the + initial handshake and whether the connection authenticated as a + relay. Fixes bug 24898; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (scheduler, consensus, backport from 0.3.3.2-alpha): + - The scheduler subsystem was failing to promptly notice changes in + consensus parameters, making it harder to switch schedulers + network-wide. Fixes bug 24975; bugfix on 0.3.2.1-alpha. + + o Minor features (denial-of-service avoidance, backport from 0.3.3.2-alpha): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor features (compatibility, OpenSSL, backport from 0.3.3.3-alpha): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor features (logging, diagnostic, backport from 0.3.3.2-alpha): + - When logging a failure to check a hidden service's certificate, + also log what the problem with the certificate was. Diagnostic + for ticket 24972. + + o Minor bugfix (channel connection, backport from 0.3.3.2-alpha): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfixes (denial-of-service, backport from 0.3.3.3-alpha): + - Fix a possible crash on malformed consensus. If a consensus had + contained an unparseable protocol line, it could have made clients + and relays crash with a null-pointer exception. To exploit this + issue, however, an attacker would need to be able to subvert the + directory authority system. Fixes bug 25251; bugfix on + 0.2.9.4-alpha. Also tracked as TROVE-2018-004. + + o Minor bugfix (directory authority, backport from 0.3.3.2-alpha): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (build, rust, backport from 0.3.3.1-alpha): + - When building with Rust on OSX, link against libresolv, to work + around the issue at https://github.com/rust-lang/rust/issues/46797. + Fixes bug 24652; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (onion services, backport from 0.3.3.2-alpha): + - Remove a BUG() statement when a client fetches an onion descriptor + that has a lower revision counter than the one in its cache. This + can happen in normal circumstances due to HSDir desync. Fixes bug + 24976; bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (logging, backport from 0.3.3.2-alpha): + - Don't treat inability to store a cached consensus object as a bug: + it can happen normally when we are out of disk space. Fixes bug + 24859; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (performance, fragile-hardening, backport from 0.3.3.1-alpha): + - Improve the performance of our consensus-diff application code + when Tor is built with the --enable-fragile-hardening option set. + Fixes bug 24826; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (OSX, backport from 0.3.3.1-alpha): + - Don't exit the Tor process if setrlimit() fails to change the file + limit (which can happen sometimes on some versions of OSX). Fixes + bug 21074; bugfix on 0.0.9pre5. + + o Minor bugfixes (spec conformance, backport from 0.3.3.3-alpha): + - Forbid "-0" as a protocol version. Fixes part of bug 25249; bugfix on + 0.2.9.4-alpha. + - Forbid UINT32_MAX as a protocol version. Fixes part of bug 25249; + bugfix on 0.2.9.4-alpha. + + o Minor bugfixes (testing, backport from 0.3.3.1-alpha): + - Fix a memory leak in the scheduler/loop_kist unit test. Fixes bug + 25005; bugfix on 0.3.2.7-rc. + + o Minor bugfixes (v3 onion services, backport from 0.3.3.2-alpha): + - Look at the "HSRend" protocol version, not the "HSDir" protocol + version, when deciding whether a consensus entry can support the + v3 onion service protocol as a rendezvous point. Fixes bug 25105; + bugfix on 0.3.2.1-alpha. + + o Code simplification and refactoring (backport from 0.3.3.3-alpha): + - Update the "rust dependencies" submodule to be a project-level + repository, rather than a user repository. Closes ticket 25323. + + o Documentation (backport from 0.3.3.1-alpha) + - Document that operators who run more than one relay or bridge are + expected to set MyFamily and ContactInfo correctly. Closes + ticket 24526. + + +Changes in version 0.3.1.10 - 2018-03-03 + Tor 0.3.1.10 backports a number of bugfixes, including important fixes for + security issues. + + It includes an important security fix for a remote crash attack + against directory authorities, tracked as TROVE-2018-001. + + This release also backports our new system for improved resistance to + denial-of-service attacks against relays. + + This release also fixes several minor bugs and annoyances from + earlier releases. + + All directory authorities should upgrade to one of the versions + released today. Relays running 0.3.1.x may wish to update to one of + the versions released today, for the DoS mitigations. + + Please note: according to our release calendar, Tor 0.3.1 will no + longer be supported after 1 July 2018. If you will be running Tor + after that date, you should make sure to plan to upgrade to the latest + stable version, or downgrade to 0.2.9 (which will receive long-term + support). + + o Major bugfixes (denial-of-service, directory authority, backport from 0.3.3.3-alpha): + - Fix a protocol-list handling bug that could be used to remotely crash + directory authorities with a null-pointer exception. Fixes bug 25074; + bugfix on 0.2.9.4-alpha. Also tracked as TROVE-2018-001 and + CVE-2018-0490. + + o Major features (denial-of-service mitigation, backport from 0.3.3.2-alpha): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Minor features (linux seccomp2 sandbox, backport from 0.3.2.5-alpha): + - Update the sandbox rules so that they should now work correctly + with Glibc 2.26. Closes ticket 24315. + + o Major bugfixes (onion services, retry behavior, backport from 0.3.3.1-alpha): + - Fix an "off by 2" error in counting rendezvous failures on the + onion service side. While we thought we would stop the rendezvous + attempt after one failed circuit, we were actually making three + circuit attempts before giving up. Now switch to a default of 2, + and allow the consensus parameter "hs_service_max_rdv_failures" to + override. Fixes bug 24895; bugfix on 0.0.6. + + o Major bugfixes (protocol versions, backport from 0.3.3.2-alpha): + - Add Link protocol version 5 to the supported protocols list. Fixes + bug 25070; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (relay, backport from 0.3.3.1-alpha): + - Fix a set of false positives where relays would consider + connections to other relays as being client-only connections (and + thus e.g. deserving different link padding schemes) if those + relays fell out of the consensus briefly. Now we look only at the + initial handshake and whether the connection authenticated as a + relay. Fixes bug 24898; bugfix on 0.3.1.1-alpha. + + o Minor features (denial-of-service avoidance, backport from 0.3.3.2-alpha): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor feature (relay statistics, backport from 0.3.2.6-alpha): + - Change relay bandwidth reporting stats interval from 4 hours to 24 + hours in order to reduce the efficiency of guard discovery + attacks. Fixes ticket 23856. + + o Minor features (compatibility, OpenSSL, backport from 0.3.3.3-alpha): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (fallback directory mirrors, backport from 0.3.2.9): + - The fallback directory list has been re-generated based on the + current status of the network. Tor uses fallback directories to + bootstrap when it doesn't yet have up-to-date directory + information. Closes ticket 24801. + - Make the default DirAuthorityFallbackRate 0.1, so that clients + prefer to bootstrap from fallback directory mirrors. This is a + follow-up to 24679, which removed weights from the default + fallbacks. Implements ticket 24681. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor bugfix (channel connection, backport from 0.3.3.2-alpha): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfix (directory authority, backport from 0.3.3.2-alpha): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (address selection, backport from 0.3.2.9): + - When the fascist_firewall_choose_address_ functions don't find a + reachable address, set the returned address to the null address + and port. This is a precautionary measure, because some callers do + not check the return value. Fixes bug 24736; bugfix + on 0.2.8.2-alpha. + + o Major bugfixes (bootstrapping, backport from 0.3.2.5-alpha): + - Fetch descriptors aggressively whenever we lack enough to build + circuits, regardless of how many descriptors we are missing. + Previously, we would delay launching the fetch when we had fewer + than 15 missing descriptors, even if some of those descriptors + were blocking circuits from building. Fixes bug 23985; bugfix on + 0.1.1.11-alpha. The effects of this bug became worse in + 0.3.0.3-alpha, when we began treating missing descriptors from our + primary guards as a reason to delay circuits. + - Don't try fetching microdescriptors from relays that have failed + to deliver them in the past. Fixes bug 23817; bugfix + on 0.3.0.1-alpha. + + o Minor bugfixes (compilation, backport from 0.3.2.7-rc): + - Fix a signed/unsigned comparison warning introduced by our fix to + TROVE-2017-009. Fixes bug 24480; bugfix on 0.2.5.16. + + o Minor bugfixes (control port, linux seccomp2 sandbox, backport from 0.3.2.5-alpha): + - Avoid a crash when attempting to use the seccomp2 sandbox together + with the OwningControllerProcess feature. Fixes bug 24198; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (denial-of-service, backport from 0.3.3.3-alpha): + - Fix a possible crash on malformed consensus. If a consensus had + contained an unparseable protocol line, it could have made clients + and relays crash with a null-pointer exception. To exploit this + issue, however, an attacker would need to be able to subvert the + directory authority system. Fixes bug 25251; bugfix on + 0.2.9.4-alpha. Also tracked as TROVE-2018-004. + + o Minor bugfixes (directory cache, backport from 0.3.2.5-alpha): + - Recover better from empty or corrupt files in the consensus cache + directory. Fixes bug 24099; bugfix on 0.3.1.1-alpha. + - When a consensus diff calculation is only partially successful, + only record the successful parts as having succeeded. Partial + success can happen if (for example) one compression method fails + but the others succeed. Previously we misrecorded all the + calculations as having succeeded, which would later cause a + nonfatal assertion failure. Fixes bug 24086; bugfix + on 0.3.1.1-alpha. + + o Minor bugfixes (entry guards, backport from 0.3.2.3-alpha): + - Tor now updates its guard state when it reads a consensus + regardless of whether it's missing descriptors. That makes tor use + its primary guards to fetch descriptors in some edge cases where + it would previously have used fallback directories. Fixes bug + 23862; bugfix on 0.3.0.1-alpha. + + o Minor bugfixes (logging, backport from 0.3.3.2-alpha): + - Don't treat inability to store a cached consensus object as a bug: + it can happen normally when we are out of disk space. Fixes bug + 24859; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (memory usage, backport from 0.3.2.8-rc): + - When queuing DESTROY cells on a channel, only queue the circuit-id + and reason fields: not the entire 514-byte cell. This fix should + help mitigate any bugs or attacks that fill up these queues, and + free more RAM for other uses. Fixes bug 24666; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (network layer, backport from 0.3.2.5-alpha): + - When closing a connection via close_connection_immediately(), we + mark it as "not blocked on bandwidth", to prevent later calls from + trying to unblock it, and give it permission to read. This fixes a + backtrace warning that can happen on relays under various + circumstances. Fixes bug 24167; bugfix on 0.1.0.1-rc. + + o Minor bugfixes (path selection, backport from 0.3.2.4-alpha): + - When selecting relays by bandwidth, avoid a rounding error that + could sometimes cause load to be imbalanced incorrectly. + Previously, we would always round upwards; now, we round towards + the nearest integer. This had the biggest effect when a relay's + weight adjustments should have given it weight 0, but it got + weight 1 instead. Fixes bug 23318; bugfix on 0.2.4.3-alpha. + - When calculating the fraction of nodes that have descriptors, and + all nodes in the network have zero bandwidths, count the number of + nodes instead. Fixes bug 23318; bugfix on 0.2.4.10-alpha. + - Actually log the total bandwidth in compute_weighted_bandwidths(). + Fixes bug 24170; bugfix on 0.2.4.3-alpha. + + o Minor bugfixes (performance, fragile-hardening, backport from 0.3.3.1-alpha): + - Improve the performance of our consensus-diff application code + when Tor is built with the --enable-fragile-hardening option set. + Fixes bug 24826; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (OSX, backport from 0.3.3.1-alpha): + - Don't exit the Tor process if setrlimit() fails to change the file + limit (which can happen sometimes on some versions of OSX). Fixes + bug 21074; bugfix on 0.0.9pre5. + + o Minor bugfixes (portability, msvc, backport from 0.3.2.9): + - Fix a bug in the bit-counting parts of our timing-wheel code on + MSVC. (Note that MSVC is still not a supported build platform, due + to cyptographic timing channel risks.) Fixes bug 24633; bugfix + on 0.2.9.1-alpha. + + o Minor bugfixes (relay, partial backport): + - Make the internal channel_is_client() function look at what sort + of connection handshake the other side used, rather than whether + the other side ever sent a create_fast cell to us. Backports part + of the fixes from bugs 22805 and 24898. + + o Minor bugfixes (spec conformance, backport from 0.3.3.3-alpha): + - Forbid "-0" as a protocol version. Fixes part of bug 25249; bugfix on + 0.2.9.4-alpha. + - Forbid UINT32_MAX as a protocol version. Fixes part of bug 25249; + bugfix on 0.2.9.4-alpha. + + o Code simplification and refactoring (backport from 0.3.3.3-alpha): + - Update the "rust dependencies" submodule to be a project-level + repository, rather than a user repository. Closes ticket 25323. + +Changes in version 0.2.9.15 - 2018-03-03 + Tor 0.2.9.15 backports important security and stability bugfixes from + later Tor releases. + + It includes an important security fix for a remote crash attack + against directory authorities, tracked as TROVE-2018-001. + + This release also backports our new system for improved resistance to + denial-of-service attacks against relays. + + This release also fixes several minor bugs and annoyances from + earlier releases. + + All directory authorities should upgrade to one of the versions + released today. Relays running 0.2.9.x may wish to update to one of + the versions released today, for the DoS mitigations. + + o Major bugfixes (denial-of-service, directory authority, backport from 0.3.3.3-alpha): + - Fix a protocol-list handling bug that could be used to remotely crash + directory authorities with a null-pointer exception. Fixes bug 25074; + bugfix on 0.2.9.4-alpha. Also tracked as TROVE-2018-001 and + CVE-2018-0490. + + o Major features (denial-of-service mitigation): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Major bugfixes (bootstrapping): + - Fetch descriptors aggressively whenever we lack enough to build + circuits, regardless of how many descriptors we are missing. + Previously, we would delay launching the fetch when we had fewer + than 15 missing descriptors, even if some of those descriptors + were blocking circuits from building. Fixes bug 23985; bugfix on + 0.1.1.11-alpha. The effects of this bug became worse in + 0.3.0.3-alpha, when we began treating missing descriptors from our + primary guards as a reason to delay circuits. + + o Major bugfixes (onion services, retry behavior): + - Fix an "off by 2" error in counting rendezvous failures on the + onion service side. While we thought we would stop the rendezvous + attempt after one failed circuit, we were actually making three + circuit attempts before giving up. Now switch to a default of 2, + and allow the consensus parameter "hs_service_max_rdv_failures" to + override. Fixes bug 24895; bugfix on 0.0.6. + + o Minor feature (relay statistics): + - Change relay bandwidth reporting stats interval from 4 hours to 24 + hours in order to reduce the efficiency of guard discovery + attacks. Fixes ticket 23856. + + o Minor features (compatibility, OpenSSL): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (denial-of-service avoidance): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor features (fallback directory mirrors): + - The fallback directory list has been re-generated based on the + current status of the network. Tor uses fallback directories to + bootstrap when it doesn't yet have up-to-date directory + information. Closes ticket 24801. + - Make the default DirAuthorityFallbackRate 0.1, so that clients + prefer to bootstrap from fallback directory mirrors. This is a + follow-up to 24679, which removed weights from the default + fallbacks. Implements ticket 24681. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor features (linux seccomp2 sandbox): + - Update the sandbox rules so that they should now work correctly + with Glibc 2.26. Closes ticket 24315. + + o Minor bugfix (channel connection): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfix (directory authority): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (address selection): + - When the fascist_firewall_choose_address_ functions don't find a + reachable address, set the returned address to the null address + and port. This is a precautionary measure, because some callers do + not check the return value. Fixes bug 24736; bugfix + on 0.2.8.2-alpha. + + o Minor bugfixes (compilation): + - Fix a signed/unsigned comparison warning introduced by our fix to + TROVE-2017-009. Fixes bug 24480; bugfix on 0.2.5.16. + + o Minor bugfixes (control port, linux seccomp2 sandbox): + - Avoid a crash when attempting to use the seccomp2 sandbox together + with the OwningControllerProcess feature. Fixes bug 24198; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (denial-of-service, backport from 0.3.3.3-alpha): + - Fix a possible crash on malformed consensus. If a consensus had + contained an unparseable protocol line, it could have made clients + and relays crash with a null-pointer exception. To exploit this + issue, however, an attacker would need to be able to subvert the + directory authority system. Fixes bug 25251; bugfix on + 0.2.9.4-alpha. Also tracked as TROVE-2018-004. + + o Minor bugfixes (memory usage): + - When queuing DESTROY cells on a channel, only queue the circuit-id + and reason fields: not the entire 514-byte cell. This fix should + help mitigate any bugs or attacks that fill up these queues, and + free more RAM for other uses. Fixes bug 24666; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (network layer): + - When closing a connection via close_connection_immediately(), we + mark it as "not blocked on bandwidth", to prevent later calls from + trying to unblock it, and give it permission to read. This fixes a + backtrace warning that can happen on relays under various + circumstances. Fixes bug 24167; bugfix on 0.1.0.1-rc. + + o Minor bugfixes (OSX): + - Don't exit the Tor process if setrlimit() fails to change the file + limit (which can happen sometimes on some versions of OSX). Fixes + bug 21074; bugfix on 0.0.9pre5. + + o Minor bugfixes (path selection): + - When selecting relays by bandwidth, avoid a rounding error that + could sometimes cause load to be imbalanced incorrectly. + Previously, we would always round upwards; now, we round towards + the nearest integer. This had the biggest effect when a relay's + weight adjustments should have given it weight 0, but it got + weight 1 instead. Fixes bug 23318; bugfix on 0.2.4.3-alpha. + - When calculating the fraction of nodes that have descriptors, and + all nodes in the network have zero bandwidths, count the number of + nodes instead. Fixes bug 23318; bugfix on 0.2.4.10-alpha. + - Actually log the total bandwidth in compute_weighted_bandwidths(). + Fixes bug 24170; bugfix on 0.2.4.3-alpha. + + o Minor bugfixes (portability, msvc): + - Fix a bug in the bit-counting parts of our timing-wheel code on + MSVC. (Note that MSVC is still not a supported build platform, due + to cryptographic timing channel risks.) Fixes bug 24633; bugfix + on 0.2.9.1-alpha. + + o Minor bugfixes (relay): + - Make the internal channel_is_client() function look at what sort + of connection handshake the other side used, rather than whether + the other side ever sent a create_fast cell to us. Backports part + of the fixes from bugs 22805 and 24898. + + o Minor bugfixes (spec conformance, backport from 0.3.3.3-alpha): + - Forbid "-0" as a protocol version. Fixes part of bug 25249; bugfix on + 0.2.9.4-alpha. + - Forbid UINT32_MAX as a protocol version. Fixes part of bug 25249; + bugfix on 0.2.9.4-alpha. + + Changes in version 0.3.2.9 - 2018-01-09 Tor 0.3.2.9 is the first stable release in the 0.3.2 series. diff --git a/changes/18105 b/changes/18105 new file mode 100644 index 0000000000..87e6e61d2f --- /dev/null +++ b/changes/18105 @@ -0,0 +1,4 @@ + o Code simplification and refactoring: + - Introduce a function to call getsockname() and return + tor_addr_t, to save a little complexity throughout the codebase. + Closes ticket 18105. diff --git a/changes/bug18918 b/changes/bug18918 new file mode 100644 index 0000000000..c939168f49 --- /dev/null +++ b/changes/bug18918 @@ -0,0 +1,6 @@ + o Code simplification and refactoring: + - In order to make the OR and dir checking function in router.c less + confusing we renamed some functions and consider_testing_reachability() + has been splitted into router_should_check_reachability() and + router_do_reachability_checks(). Also we improved the documentation in + some functions. Closes ticket 18918. diff --git a/changes/bug20887 b/changes/bug20887 new file mode 100644 index 0000000000..5d4e4ed233 --- /dev/null +++ b/changes/bug20887 @@ -0,0 +1,4 @@ + o Minor bugfixes (freebsd): + - In have_enough_mem_for_dircache(), the variable DIRCACHE_MIN_MEM_MB + does not stringify on FreeBSD, so we switch to tor_asprintf(). Fixes + bug 20887; bugfix on 0.2.8.1-alpha. Patch by Neel Chauhan. diff --git a/changes/bug23909 b/changes/bug23909 new file mode 100644 index 0000000000..6b84e711e7 --- /dev/null +++ b/changes/bug23909 @@ -0,0 +1,4 @@ + o Minor features (directory authority): + - Directory authorities now open their key-pinning files as O_SYNC, + to prevent themselves from accidentally writing partial lines. + Closes ticket 23909. diff --git a/changes/bug24484 b/changes/bug24484 new file mode 100644 index 0000000000..35a2044923 --- /dev/null +++ b/changes/bug24484 @@ -0,0 +1,4 @@ + o Code simplification and refactoring: + - Since Tor requires C99, remove our old workaround code for libc + implementations where free(NULL) doesn't work. Closes ticket 24484. + diff --git a/changes/bug24767 b/changes/bug24767 deleted file mode 100644 index 56fbe51a98..0000000000 --- a/changes/bug24767 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes (relay, connection): - - Refuse to connect again to a relay from which we failed previously with - a connection refused, timeout or error (at the TCP level). The relay - won't be retried for 60 seconds after the failure occured. Fixes bug - 24767; bugfix on 0.0.6. diff --git a/changes/bug24769 b/changes/bug24769 deleted file mode 100644 index 2893e0ff07..0000000000 --- a/changes/bug24769 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (performance): - - Reduce the number of circuits that can be opened at once during the - circuit build timeout phase. This is done by increasing the idle timeout - to 3 minutes, and lowering the maximum number of concurrent learning - circuits to 10. Fixes bug 24769; bugfix on 0.3.1.1-alpha. - - diff --git a/changes/bug24903 b/changes/bug24903 deleted file mode 100644 index 01c9b53f23..0000000000 --- a/changes/bug24903 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (controller, reliability): - - Avoid a (nonfatal) assertion failure when extending a one-hop circuit - from the controller to become a multihop circuit. Fixes bug 24903; - bugfix on 0.2.5.2-alpha. - diff --git a/changes/bug24904 b/changes/bug24904 deleted file mode 100644 index 648d9a5834..0000000000 --- a/changes/bug24904 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfix (channel, client): - - Better identify client connection when reporting to the geoip client - cache. Fixes bug 24904; bugfix on 0.3.1.7. - diff --git a/changes/bug24910 b/changes/bug24910 new file mode 100644 index 0000000000..58574c0069 --- /dev/null +++ b/changes/bug24910 @@ -0,0 +1,7 @@ + o Minor bugfixes (relay statistics): + - When a relay is collecting internal statistics about how many + create cell requests it has seen of each type, accurately count the + requests from relays that temporarily fall out of the consensus. (To + be extra conservative, we were already ignoring requests from + clients in our counts, and we continue ignoring them here.) Fixes + bug 24910; bugfix on 0.2.4.17-rc. diff --git a/changes/bug24914 b/changes/bug24914 new file mode 100644 index 0000000000..ea441fd38c --- /dev/null +++ b/changes/bug24914 @@ -0,0 +1,3 @@ + o Minor features (performance): + - Avoid a needless call to malloc() when processing an incoming + relay cell. Closes ticket 24914. diff --git a/changes/bug24978 b/changes/bug24978 deleted file mode 100644 index 5dc45c7442..0000000000 --- a/changes/bug24978 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor features (compatibility, OpenSSL): - - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. - Previous versions of Tor would not have worked with OpenSSL - 1.1.1, since they neither disabled TLS 1.3 nor enabled any of the - ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. - Closes ticket 24978. - diff --git a/changes/bug25120 b/changes/bug25120 deleted file mode 100644 index 7215756ef3..0000000000 --- a/changes/bug25120 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (logging): - - Clarify the log messages produced when getrandom() or a related - entropy-generation mechanism gives an error. Closes ticket - 25120. diff --git a/changes/bug25213 b/changes/bug25213 deleted file mode 100644 index bb196ca724..0000000000 --- a/changes/bug25213 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (warnings, ipv6): - - Avoid a bug warning that could occur when trying to connect to - a relay over IPv6 on a Tor instance that downloads router descriptors, - but prefers to use microdescriptors. Fixes bug 25213; bugfix on - 0.3.3.1-alpha. diff --git a/changes/bug25223 b/changes/bug25223 deleted file mode 100644 index fdd5563500..0000000000 --- a/changes/bug25223 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (DoS mitigation): - - Make sure we don't modify consensus parameters if we aren't a public - relay when a new consensus arrives. Fixes bug 25223; bugfix on - 0.3.3.2-alpha. diff --git a/changes/bug25249 b/changes/bug25249 deleted file mode 100644 index b4153eeaef..0000000000 --- a/changes/bug25249 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (spec conformance): - - Forbid "-0" as a protocol version. Fixes part of bug 25249; bugfix on - 0.2.9.4-alpha. diff --git a/changes/bug25249.2 b/changes/bug25249.2 deleted file mode 100644 index 9058c11071..0000000000 --- a/changes/bug25249.2 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (spec conformance): - - Forbid UINT32_MAX as a protocol version. Fixes part of bug 25249; - bugfix on 0.2.9.4-alpha. diff --git a/changes/bug25306 b/changes/bug25306 deleted file mode 100644 index a2e6306f42..0000000000 --- a/changes/bug25306 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (hidden service v3): - - Avoid asserting when building descriptors in the next rotation time is - out of sync with the consensus valid after time. Instead, log a bug - warning with extra information to hunt down the cause of this assert. - Fixes bug 25306; bugfix on 0.3.2.1-alpha. - diff --git a/changes/bug25372 b/changes/bug25372 deleted file mode 100644 index 4cceab3a45..0000000000 --- a/changes/bug25372 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features (log messages): - - Improve log message in the out of memory handler to include information - about memory usage from the different compression backends. Closes ticket 25372. diff --git a/changes/bug25378 b/changes/bug25378 deleted file mode 100644 index 4648a0a313..0000000000 --- a/changes/bug25378 +++ /dev/null @@ -1,4 +0,0 @@ - o Documentation: - - Update the documentation for "Log" to include the current list - of logging domains. Closes ticket 25378. - diff --git a/changes/bug25398 b/changes/bug25398 new file mode 100644 index 0000000000..227a95d21a --- /dev/null +++ b/changes/bug25398 @@ -0,0 +1,5 @@ + o New system requirements: + - Tor no longer tries to support systems without mmap() or some local + equivalent. Apparently, compilation on such systems has been broken for + some time, without anybody noticing or complaining. Closes ticket + 25398. diff --git a/changes/bug25399 b/changes/bug25399 new file mode 100644 index 0000000000..4554574939 --- /dev/null +++ b/changes/bug25399 @@ -0,0 +1,5 @@ + o Minor bugfixes (portability): + - Do not align mmap length, as it is not required by POSIX, and the + getpagesize function is deprecated. Fixes bug 25399; bugfix on + 0.1.1.23. + diff --git a/changes/bug25409 b/changes/bug25409 new file mode 100644 index 0000000000..093d8f58bd --- /dev/null +++ b/changes/bug25409 @@ -0,0 +1,12 @@ + o Removed features: + - The PortForwarding and PortForwardingHelper features have been + removed. The reasoning is, given that implementations of NAT traversal + protocols within common consumer grade routers are frequently buggy, and + that the target audience for a NAT punching feature is a perhaps + less-technically-inclined relay operator, when the helper fails to setup + traversal the problems are usually deep, ugly, and very router specific, + making them horrendously impossible for technical support to reliable + assist with, and thus resulting in frustration all around. Unfortunately, + relay operators who would like to run relays behind NATs will need to + become more familiar with the port forwarding configurations on their + local router. Closes 25409. diff --git a/changes/bug25415 b/changes/bug25415 new file mode 100644 index 0000000000..ec851aee8d --- /dev/null +++ b/changes/bug25415 @@ -0,0 +1,4 @@ + o Major bugfixes (directory authority): + - Avoid a crash when testing router reachability on a router that could + have an ed25519 ID, but which does not. Fixes bug 25415; bugfix on + 0.3.3.2-alpha. diff --git a/changes/bug25425 b/changes/bug25425 new file mode 100644 index 0000000000..41f1a47b15 --- /dev/null +++ b/changes/bug25425 @@ -0,0 +1,4 @@ + o Minor features (testing): + - A new unittests module specifically for testing the functions in the + (new-ish) bridges.c module has been created with new unittests, raising + the code coverage percentages. Closes 25425. diff --git a/changes/bug25450 b/changes/bug25450 deleted file mode 100644 index ae14135a72..0000000000 --- a/changes/bug25450 +++ /dev/null @@ -1,8 +0,0 @@ - o Minor bugfixes (testing): - - Avoid intermittent test failures due to relying on hidden service - introductory point creation within 5 seconds of real clock time. The - time limit for the test has been increased to 500 seconds, which may - still result in intermittent failures (e.g. if the system doing the - testing enters sleep/hibernation or experiences some other clock jump). - However, this should elliminate test failures currently happening on - developer and CI systems. Fixes bug 25450; bugfix on 0.3.1.3-alpha. diff --git a/changes/bug25474 b/changes/bug25474 deleted file mode 100644 index 7d3bd1c5f5..0000000000 --- a/changes/bug25474 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (compilation): - - Fix a c99 compliance issue in our configuration script that was - causing compilation issues when compiling Tor with certain - versions of xtools. Fixes bug 25474; bugfix on 0.3.2.5-alpha. - diff --git a/changes/bug25479 b/changes/bug25479 deleted file mode 100644 index e693b8ec7a..0000000000 --- a/changes/bug25479 +++ /dev/null @@ -1,4 +0,0 @@ - o Major bugfixes (benchmarks): - - Fix a crash when running benchmark tests on win32 systems which - was due to an uninitialised mutex before logging and options - were initialised. Fixes bug 25479; bugfix on 0.3.3.3-alpha. diff --git a/changes/bug25512 b/changes/bug25512 new file mode 100644 index 0000000000..4b6491867e --- /dev/null +++ b/changes/bug25512 @@ -0,0 +1,5 @@ + o Minor bugfixes (restart-in-process): + - When shutting down, Tor now clears all the flags in the control.c + module. This should prevent a bug where authentication cookies + are not generated on restart. Fixes bug 25512; bugfix on 0.3.3.1-alpha. + diff --git a/changes/bug25560 b/changes/bug25560 deleted file mode 100644 index 5947fa03a3..0000000000 --- a/changes/bug25560 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (testing, rust): - - Some of our Rust crates were not having their tests run with `make - test-rust` due to a static string in the `src/test/test_rust.sh` script - specifying which crates to test. Rust crates are not automatically - detected and tested. Fixes bug 25560; bugfix on 0.3.3.3-alpha. diff --git a/changes/bug25629 b/changes/bug25629 deleted file mode 100644 index 190928a941..0000000000 --- a/changes/bug25629 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (C correctness): - - Fix a very unlikely null pointer dereference. Fixes bug 25629; - bugfix on 0.2.9.15. Found by Coverity; this is CID 1430932. diff --git a/changes/bug25664 b/changes/bug25664 deleted file mode 100644 index c8b3ca6187..0000000000 --- a/changes/bug25664 +++ /dev/null @@ -1,3 +0,0 @@ - o New system requirements: - - When built with Rust, Tor now depends on version 0.2.39 of the libc - crate. Closes tickets 25310 and 25664. diff --git a/changes/bug25675 b/changes/bug25675 new file mode 100644 index 0000000000..d0f0287b59 --- /dev/null +++ b/changes/bug25675 @@ -0,0 +1,4 @@ + o Minor bugfixes (C correctness): + - Add a missing lock acquisition in the shutdown code of the + control subsystem. Fixes bug 25675; bugfix on 0.2.7.3-rc. Found + by Coverity; this is CID 1433643. diff --git a/changes/bugs_25036_25055 b/changes/bugs_25036_25055 deleted file mode 100644 index daa46321c0..0000000000 --- a/changes/bugs_25036_25055 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (networking): - - Tor will not reject IPv6 address strings from TorBrowser when they - are passed as hostnames in SOCKS5 requests. Fixes bug 25036, - bugfix on Tor 0.3.1.2. - - string_is_valid_hostname() will not consider IP strings to be valid - hostnames. Fixes bug 25055; bugfix on Tor 0.2.5.5. - diff --git a/changes/feature25150 b/changes/feature25150 new file mode 100644 index 0000000000..eb65327a8d --- /dev/null +++ b/changes/feature25150 @@ -0,0 +1,4 @@ + o Minor features (performance, allocation): + - Avoid a needless malloc()/free() pair every time we handle an ntor + handshake. Closes ticket 25150. + diff --git a/changes/feature25313 b/changes/feature25313 deleted file mode 100644 index 90f421169f..0000000000 --- a/changes/feature25313 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (sandbox): - - Explicitly permit the poll() system call when the Linux seccomp2-based - sandbox is enabled: apparently, some versions of libc use poll() when - calling getpwnam(). Closes ticket 25313. diff --git a/changes/geoip-2018-03-08 b/changes/geoip-2018-03-08 deleted file mode 100644 index d9696aab53..0000000000 --- a/changes/geoip-2018-03-08 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the March 8 2018 Maxmind GeoLite2 - Country database. Closes ticket 25469. - diff --git a/changes/isolate_libevent b/changes/isolate_libevent new file mode 100644 index 0000000000..852e533fd3 --- /dev/null +++ b/changes/isolate_libevent @@ -0,0 +1,5 @@ + o Code simplification and refactoring: + - Initial work to isolate Libevent usage to a handful of modules in our + codebase, to simplify our call structure, and so that we can more + easily change event loops in the future if needed. Closes ticket + 23750. diff --git a/changes/refactor23814 b/changes/refactor23814 deleted file mode 100644 index a67b6989f8..0000000000 --- a/changes/refactor23814 +++ /dev/null @@ -1,4 +0,0 @@ - o Code simplification and refactoring: - - Remove the old (deterministic) directory retry logic entirely: - We've used exponential backoff exclusively for some time. - Closes ticket 23814. diff --git a/changes/ticket23635 b/changes/ticket23635 deleted file mode 100644 index 54d303e4b8..0000000000 --- a/changes/ticket23635 +++ /dev/null @@ -1,3 +0,0 @@ - o Documentation: - - Improved the documentation of AccountingStart paremeter. - Closes ticket 23635. diff --git a/changes/ticket23873 b/changes/ticket23873 new file mode 100644 index 0000000000..ffe65e5ed4 --- /dev/null +++ b/changes/ticket23873 @@ -0,0 +1,6 @@ + o Code simplification and refactoring: + - We remove the return value of node_get_prim_orport() and + node_get_prim_dirport(), and introduce node_get_prim_orport() + in node_ipv6_or_preferred() and node_ipv6_dir_preferred() in + order to check for a null address. Closes ticket 23873. Patch + by Neel Chauhan. diff --git a/changes/ticket24343 b/changes/ticket24343 deleted file mode 100644 index e62d65eb54..0000000000 --- a/changes/ticket24343 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (man page, SocksPort): - - Remove dead code about the old "SocksSockets" option. To do so, the - SocksSocketsGroupWritable option has been renamed to - UnixSockssGroupWritable which does the same exact thing. The old option - is still usable but will warn that it is deprecated. Fixes bug 24343; - bugfix on 0.2.6.3. diff --git a/changes/ticket24714 b/changes/ticket24714 new file mode 100644 index 0000000000..60353abec9 --- /dev/null +++ b/changes/ticket24714 @@ -0,0 +1,6 @@ + o Code simplification and refactoring: + - Rename two fields of connection_t struct. + timestamp_lastwritten is renamed to timestamp_last_write_allowed and + timestamp_lastread is renamed to timestamp_last_read_allowed. + Closes ticket 24714, patch by "valentecaio". + diff --git a/changes/ticket24740 b/changes/ticket24740 new file mode 100644 index 0000000000..253cdb65e2 --- /dev/null +++ b/changes/ticket24740 @@ -0,0 +1,5 @@ + o Minor bugfixes (directory server cert fetch): + - Fixed launching a certificate fetch always during the scheduled + periodic consensus fetch by fetching only in those cases when + consensus are waiting for certs. + Fixes bug 24740; bugfix on 0.2.9.1-alpha. diff --git a/changes/ticket25071 b/changes/ticket25071 deleted file mode 100644 index 5e2917e10b..0000000000 --- a/changes/ticket25071 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (testing): - - Add a "make test-rust" target to run the rust tests only. - Closes ticket 25071. - diff --git a/changes/ticket25081 b/changes/ticket25081 new file mode 100644 index 0000000000..b6e6c2a5a6 --- /dev/null +++ b/changes/ticket25081 @@ -0,0 +1,5 @@ + o Code simplification and refactoring: + - Remove extern declaration of stats_n_seconds_working variable from main, + protecting its accesses with get_uptime() and reset_uptime() functions. + Closes ticket 25081, patch by “valentecaio”. + diff --git a/changes/ticket25162 b/changes/ticket25162 new file mode 100644 index 0000000000..37019a7f8f --- /dev/null +++ b/changes/ticket25162 @@ -0,0 +1,6 @@ + o Minor features (compression, zstd): + - When running with zstd, Tor now considers using advanced functions that + the zstd maintainers have labeled as potentially unstable. To + prevent breakage, Tor will only use this functionality when + the runtime version of the zstd library matches the version + with which it were compiled. Closes ticket 25162. diff --git a/changes/ticket25202 b/changes/ticket25202 deleted file mode 100644 index ba64abad7b..0000000000 --- a/changes/ticket25202 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (DoS mitigation): - - Add extra safety checks when refilling the circuit creation bucket to - ensure we never set a value that is above the allowed burst. Fixes - bug 25202; bugfix on 0.3.3.2-alpha. diff --git a/changes/ticket25261 b/changes/ticket25261 new file mode 100644 index 0000000000..604a09d975 --- /dev/null +++ b/changes/ticket25261 @@ -0,0 +1,3 @@ + o Minor bugfix (Multiple includes): + - Fixed multiple includes of trasports.h in src/or/connection.c + Fixes bug 25261; bugfix on 0.2.5.1-alpha. diff --git a/changes/ticket25268 b/changes/ticket25268 new file mode 100644 index 0000000000..e444984dc4 --- /dev/null +++ b/changes/ticket25268 @@ -0,0 +1,7 @@ + o Removed features: + - The old "round-robin" circuit multiplexer (circuitmux) + implementation has been removed, along with a fairly large set of + code that existed to support it. It has not been the default + circuitmux since we introduced the "EWMA" circuitmux in 0.2.4.x, + but it still required an unreasonable amount of memory and CPU. + Closes ticket 25268. diff --git a/changes/ticket25290 b/changes/ticket25290 new file mode 100644 index 0000000000..5e44f5e05c --- /dev/null +++ b/changes/ticket25290 @@ -0,0 +1,5 @@ + o Code simplification and refactoring: + - We switch to should_record_bridge_info() in geoip_note_client_seen() and + options_need_geoip_info() instead of accessing the configuration values + directly. Fixes bug 25290; bugfix on 0.2.1.6-alpha. Patch by Neel + Chauhan. diff --git a/changes/ticket25323 b/changes/ticket25323 deleted file mode 100644 index 836825de5d..0000000000 --- a/changes/ticket25323 +++ /dev/null @@ -1,4 +0,0 @@ - o Code simplification and refactoring: - - Update the "rust dependencies" submodule to be an project-level - repository, rather than a user repository. Closes ticket 25323. - diff --git a/changes/ticket25409 b/changes/ticket25409 new file mode 100644 index 0000000000..f0006fbc5d --- /dev/null +++ b/changes/ticket25409 @@ -0,0 +1,6 @@ + o Code simplification and refactoring: + We remove the PortForwsrding and PortForwardingHelper options, related + functions, and the port_forwarding tests. These options were used by + the now-deprecated Vidalia to help ordinary users become Tor relays or + bridges. Closes ticket 25409. Patch by Neel Chauhan. + diff --git a/changes/ticket25432 b/changes/ticket25432 new file mode 100644 index 0000000000..21ca201343 --- /dev/null +++ b/changes/ticket25432 @@ -0,0 +1,6 @@ + o Code simplification and refactoring: + - Merge functions used for describing nodes and suppress the functions + that do not allocate memory for the output buffer string. + NODE_DESC_BUF_LEN constant and format_node_description() function + cannot be used externally from router.c module anymore. + Closes ticket 25432. Patch by valentecaio. diff --git a/changes/ticket25645 b/changes/ticket25645 new file mode 100644 index 0000000000..ab20a5ca4c --- /dev/null +++ b/changes/ticket25645 @@ -0,0 +1,4 @@ + o Minor bugfixes (channel_get_for_extend()): + - Remove the unused variable n_possible from the function + Fixes bug 25645; bugfix on 0.2.4.4-alpha + diff --git a/changes/ticket4187 b/changes/ticket4187 new file mode 100644 index 0000000000..c5f795b1a6 --- /dev/null +++ b/changes/ticket4187 @@ -0,0 +1,3 @@ + o Minor bugfixes (directory client): + - When unverified-consensus is verified, rename it to cached-consenus. + Fixes bug 4187; bugfix on 0.2.0.3-alpha. diff --git a/changes/ticket6236 b/changes/ticket6236 new file mode 100644 index 0000000000..9dea07e696 --- /dev/null +++ b/changes/ticket6236 @@ -0,0 +1,4 @@ + o Minor bugfixes (Duplicate code): + - Remove duplicate code in parse_{c,s}method_line and bootstrap + their functionalities into a single function. Fixes + bug 6236; bugfix on 0.2.3.6-alpha. diff --git a/changes/trove-2018-001.1 b/changes/trove-2018-001.1 deleted file mode 100644 index f0ee92f409..0000000000 --- a/changes/trove-2018-001.1 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes (denial-of-service, directory authority): - - Fix a protocol-list handling bug that could be used to remotely crash - directory authorities with a null-pointer exception. Fixes bug 25074; - bugfix on 0.2.9.4-alpha. Also tracked as TROVE-2018-001. - - diff --git a/changes/trove-2018-004 b/changes/trove-2018-004 deleted file mode 100644 index 37e0a89b0d..0000000000 --- a/changes/trove-2018-004 +++ /dev/null @@ -1,8 +0,0 @@ - o Minor bugfixes (denial-of-service): - - Fix a possible crash on malformed consensus. If a consensus had - contained an unparseable protocol line, it could have made clients - and relays crash with a null-pointer exception. To exploit this - issue, however, an attacker would need to be able to subvert the - directory-authority system. Fixes bug 25251; bugfix on - 0.2.9.4-alpha. Also tracked as TROVE-2018-004. - diff --git a/configure.ac b/configure.ac index 143044956f..878f5a88bb 100644 --- a/configure.ac +++ b/configure.ac @@ -4,7 +4,7 @@ dnl Copyright (c) 2007-2017, The Tor Project, Inc. dnl See LICENSE for licensing information AC_PREREQ([2.63]) -AC_INIT([tor],[0.3.3.4-alpha-dev]) +AC_INIT([tor],[0.3.4.0-alpha-dev]) AC_CONFIG_SRCDIR([src/or/main.c]) AC_CONFIG_MACRO_DIR([m4]) @@ -61,6 +61,8 @@ AC_ARG_ENABLE(cargo-online-mode, AS_HELP_STRING(--enable-cargo-online-mode, [Allow cargo to make network requests to fetch crates. For builds with rust only.])) AC_ARG_ENABLE(restart-debugging, AS_HELP_STRING(--enable-restart-debugging, [Build Tor with support for debugging in-process restart. Developers only.])) +AC_ARG_ENABLE(zstd-advanced-apis, + AS_HELP_STRING(--disable-zstd-advanced-apis, [Build without support for zstd's "static-only" APIs.])) if test "x$enable_coverage" != "xyes" -a "x$enable_asserts_in_tests" = "xno" ; then AC_MSG_ERROR([Can't disable assertions outside of coverage build]) @@ -114,6 +116,10 @@ if test "$enable_restart_debugging" = "yes"; then [Defined if we're building with support for in-process restart debugging.]) fi +if test "$enable_zstd_advanced_apis" != "no"; then + AC_DEFINE(ENABLE_ZSTD_ADVANCED_APIS, 1, + [Defined if we're going to try to use zstd's "static-only" APIs.]) +fi # systemd support if test "x$enable_systemd" = "xno"; then @@ -547,6 +553,7 @@ AC_CHECK_FUNCS( mach_approximate_time \ memmem \ memset_s \ + mmap \ pipe \ pipe2 \ prctl \ @@ -615,6 +622,21 @@ fi AM_CONDITIONAL(BUILD_READPASSPHRASE_C, test "x$ac_cv_func_readpassphrase" = "xno" && test "$bwin32" = "false") +AC_MSG_CHECKING([whether free(NULL) works]) +AC_RUN_IFELSE([AC_LANG_PROGRAM([ + #include <stdlib.h> +], [ +char *p = NULL; +free(p); +])], +[free_null_ok=true; AC_MSG_RESULT(yes)], +[free_null_ok=false; AC_MSG_RESULT(no)], +[free_null_ok=cross; AC_MSG_RESULT(cross)]) + +if test "$free_null_ok" = "false"; then + AC_MSG_ERROR([Your libc implementation doesn't allow free(NULL), as required by C99.]) +fi + dnl ------------------------------------------------------ dnl Where do you live, libevent? And how do we call you? @@ -835,6 +857,10 @@ AC_CHECK_MEMBERS([SSL.state], , , [#include <openssl/ssl.h> ]) +AC_CHECK_SIZEOF(SHA_CTX, , [AC_INCLUDES_DEFAULT() +#include <openssl/sha.h> +]) + dnl Define the set of checks for KIST scheduler support. AC_DEFUN([CHECK_KIST_SUPPORT],[ dnl KIST needs struct tcp_info and for certain members to exist. @@ -953,6 +979,16 @@ if test "x$have_zstd" = "xyes"; then AC_DEFINE(HAVE_ZSTD,1,[Have Zstd]) TOR_ZSTD_CFLAGS="${ZSTD_CFLAGS}" TOR_ZSTD_LIBS="${ZSTD_LIBS}" + + dnl now check for zstd functions + save_LIBS="$LIBS" + save_CFLAGS="$CFLAGS" + LIBS="$LIBS $ZSTD_LIBS" + CFLAGS="$CFLAGS $ZSTD_CFLAGS" + AC_CHECK_FUNCS(ZSTD_estimateCStreamSize \ + ZSTD_estimateDCtxSize) + LIBS="$save_LIBS" + CFLAGS="$save_CFLAGS" fi AC_SUBST(TOR_ZSTD_CFLAGS) AC_SUBST(TOR_ZSTD_LIBS) @@ -1705,14 +1741,6 @@ AC_CHECK_DECLS([mlockall], , , [ #include <sys/mman.h> #endif]) -# Some MinGW environments don't have getpagesize in unistd.h. We don't use -# AC_CHECK_FUNCS(getpagesize), because other environments rename getpagesize -# using macros -AC_CHECK_DECLS([getpagesize], , , [ -#ifdef HAVE_UNISTD_H -#include <unistd.h> -#endif]) - # Allow user to specify an alternate syslog facility AC_ARG_WITH(syslog-facility, AS_HELP_STRING(--with-syslog-facility=LOG, [syslog facility to use (default=LOG_DAEMON)]), @@ -1850,6 +1878,12 @@ AC_SUBST(BUILDDIR) AH_TEMPLATE([BUILDDIR],[tor's build directory]) AC_DEFINE_UNQUOTED(BUILDDIR,"$BUILDDIR") +if test "x$SRCDIR" = "x"; then + SRCDIR=$(cd "$srcdir"; pwd) +fi +AH_TEMPLATE([SRCDIR],[tor's sourcedir directory]) +AC_DEFINE_UNQUOTED(SRCDIR,"$SRCDIR") + if test "x$CONFDIR" = "x"; then CONFDIR=`eval echo $sysconfdir/tor` fi diff --git a/contrib/win32build/tor-mingw.nsi.in b/contrib/win32build/tor-mingw.nsi.in index 979273d8e7..dfcb0644dc 100644 --- a/contrib/win32build/tor-mingw.nsi.in +++ b/contrib/win32build/tor-mingw.nsi.in @@ -8,7 +8,7 @@ !include "LogicLib.nsh" !include "FileFunc.nsh" !insertmacro GetParameters -!define VERSION "0.3.3.4-alpha-dev" +!define VERSION "0.3.4.0-alpha-dev" !define INSTALLER "tor-${VERSION}-win32.exe" !define WEBSITE "https://www.torproject.org/" !define LICENSE "LICENSE" diff --git a/doc/HACKING/CodingStandardsRust.md b/doc/HACKING/CodingStandardsRust.md index 7c6405e624..d9496c08f7 100644 --- a/doc/HACKING/CodingStandardsRust.md +++ b/doc/HACKING/CodingStandardsRust.md @@ -324,12 +324,26 @@ Here are some additional bits of advice and rules: } } -3. Pass only integer types and bytes over the boundary +3. Pass only C-compatible primitive types and bytes over the boundary - The only non-integer type which may cross the FFI boundary is + Rust's C-compatible primitive types are integers and floats. + These types are declared in the [libc crate](https://doc.rust-lang.org/libc/x86_64-unknown-linux-gnu/libc/index.html#types). + Most Rust objects have different [representations](https://doc.rust-lang.org/libc/x86_64-unknown-linux-gnu/libc/index.html#types) + in C and Rust, so they can't be passed using FFI. + + Tor currently uses the following Rust primitive types from libc for FFI: + * defined-size integers: `uint32_t` + * native-sized integers: `c_int` + * native-sized floats: `c_double` + * native-sized raw pointers: `* c_void`, `* c_char`, `** c_char` + + TODO: C smartlist to Stringlist conversion using FFI + + The only non-primitive type which may cross the FFI boundary is bytes, e.g. `&[u8]`. This SHOULD be done on the Rust side by - passing a pointer (`*mut libc::c_char`) and a length - (`libc::size_t`). + passing a pointer (`*mut libc::c_char`). The length can be passed + explicitly (`libc::size_t`), or the string can be NUL-byte terminated + C string. One might be tempted to do this via doing `CString::new("blah").unwrap().into_raw()`. This has several problems: diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md index 6c8fa1331f..e70416c354 100644 --- a/doc/HACKING/ReleasingTor.md +++ b/doc/HACKING/ReleasingTor.md @@ -34,7 +34,7 @@ new Tor release: What about Coverity Scan? - What about clan scan-build? + What about clang scan-build? Does 'make distcheck' complain? diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 9aad3af98d..c3493f7181 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -311,7 +311,9 @@ GENERAL OPTIONS other than controller connections, and we close (and don't reattempt) any outbound connections. Controllers sometimes use this option to avoid using - the network until Tor is fully configured. (Default: 0) + the network until Tor is fully configured. Tor will make still certain + network-related calls (like DNS lookups) as a part of its configuration + process, even if DisableNetwork is set. (Default: 0) [[ConstrainedSockets]] **ConstrainedSockets** **0**|**1**:: If set, Tor will tell the kernel to attempt to shrink the buffers for all @@ -778,17 +780,15 @@ GENERAL OPTIONS This is useful when running on flash memory or other media that support only a limited number of writes. (Default: 0) -[[CircuitPriorityHalflife]] **CircuitPriorityHalflife** __NUM1__:: +[[CircuitPriorityHalflife]] **CircuitPriorityHalflife** __NUM__:: If this value is set, we override the default algorithm for choosing which - circuit's cell to deliver or relay next. When the value is 0, we - round-robin between the active circuits on a connection, delivering one - cell from each in turn. When the value is positive, we prefer delivering - cells from whichever connection has the lowest weighted cell count, where - cells are weighted exponentially according to the supplied - CircuitPriorityHalflife value (in seconds). If this option is not set at - all, we use the behavior recommended in the current consensus - networkstatus. This is an advanced option; you generally shouldn't have - to mess with it. (Default: not set) + circuit's cell to deliver or relay next. It is delivered first to the + circuit that has the lowest weighted cell count, where cells are weighted + exponentially according to this value (in seconds). If the value is -1, it + is taken from the consensus if possible else it will fallback to the + default value of 30. Minimum: 1, Maximum: 2147483647. This can be defined + as a float value. This is an advanced option; you generally shouldn't have + to mess with it. (Default: -1) [[CountPrivateBandwidth]] **CountPrivateBandwidth** **0**|**1**:: If this option is set, then Tor's rate-limiting applies not only to @@ -805,10 +805,9 @@ GENERAL OPTIONS [[NoExec]] **NoExec** **0**|**1**:: If this option is set to 1, then Tor will never launch another - executable, regardless of the settings of PortForwardingHelper, - ClientTransportPlugin, or ServerTransportPlugin. Once this - option has been set to 1, it cannot be set back to 0 without - restarting Tor. (Default: 0) + executable, regardless of the settings of ClientTransportPlugin + or ServerTransportPlugin. Once this option has been set to 1, + it cannot be set back to 0 without restarting Tor. (Default: 0) [[Schedulers]] **Schedulers** **KIST**|**KISTLite**|**Vanilla**:: Specify the scheduler type that tor should use. The scheduler is @@ -1398,7 +1397,7 @@ The following options are useful only for clients (that is, if [[HTTPTunnelPort]] **HTTPTunnelPort** \['address':]__port__|**auto** [_isolation flags_]:: Open this port to listen for proxy connections using the "HTTP CONNECT" - protocol instead of SOCKS. Set this to 0 + protocol instead of SOCKS. Set this to 0 if you don't want to allow "HTTP CONNECT" connections. Set the port to "auto" to have Tor pick a port for you. This directive can be specified multiple times to bind to multiple addresses/ports. See @@ -1438,7 +1437,7 @@ The following options are useful only for clients (that is, if Set this to "default", or leave it unconfigured, to use regular IPTables on Linux, or to use pf +rdr-to+ rules on *BSD systems. + + - (Default: "default".) + (Default: "default") [[NATDPort]] **NATDPort** \['address':]__port__|**auto** [_isolation flags_]:: Open this port to listen for connections from old versions of ipfw (as @@ -1731,7 +1730,7 @@ The following options are useful only for clients (that is, if prevent your Tor client from bootstrapping. If this option is negative, Tor will use a default value chosen by the directory authorities. If the directory authorities do not choose a value, Tor will default to 0.6. - (Default: -1.) + (Default: -1) [[ClientBootstrapConsensusAuthorityDownloadSchedule]] **ClientBootstrapConsensusAuthorityDownloadSchedule** __N__,__N__,__...__:: Schedule for when clients should download consensuses from authorities @@ -1889,7 +1888,7 @@ is non-zero): If you want to use a reduced exit policy rather than the default exit policy, set "ReducedExitPolicy 1". If you want to _replace_ the default exit policy with your custom exit policy, end your exit policy with either - a reject *:* or an accept *:*. Otherwise, you’re _augmenting_ (prepending + a reject *:* or an accept *:*. Otherwise, you're _augmenting_ (prepending to) the default or reduced exit policy. + + The default exit policy is: @@ -2087,18 +2086,6 @@ is non-zero): For obvious reasons, NoAdvertise and NoListen are mutually exclusive, and IPv4Only and IPv6Only are mutually exclusive. -[[PortForwarding]] **PortForwarding** **0**|**1**:: - Attempt to automatically forward the DirPort and ORPort on a NAT router - connecting this Tor server to the Internet. If set, Tor will try both - NAT-PMP (common on Apple routers) and UPnP (common on routers from other - manufacturers). (Default: 0) - -[[PortForwardingHelper]] **PortForwardingHelper** __filename__|__pathname__:: - If PortForwarding is set, use this executable to configure the forwarding. - If set to a filename, the system path will be searched for the executable. - If set to a path, only the specified path will be executed. - (Default: tor-fw-helper) - [[PublishServerDescriptor]] **PublishServerDescriptor** **0**|**1**|**v3**|**bridge**,**...**:: This option specifies which descriptors Tor will publish when acting as a relay. You can @@ -2262,7 +2249,8 @@ is non-zero): sent and received by this relay, in addition to total cell counts. These statistics are rounded, and omitted if traffic is low. This information is important for load balancing decisions related to padding. - (Default: 1) + If ExtraInfoStatistics is enabled, it will be published + as a part of extra-info document. (Default: 1) [[DirReqStatistics]] **DirReqStatistics** **0**|**1**:: Relays and bridges only. @@ -2361,6 +2349,11 @@ is non-zero): KeywDirectory. If the option is set to 1, make the KeyDirectory readable by the default GID. (Default: 0) +[[RephistTrackTime]] **RephistTrackTime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: + Tells an authority, or other node tracking node reliability and history, + that fine-grained information about nodes can be discarded when it hasn't + changed for a given amount of time. (Default: 24 hours) + DIRECTORY SERVER OPTIONS ------------------------ @@ -2598,11 +2591,6 @@ on the public Tor network. different identity. This feature is used to migrate directory authority keys in the event of a compromise. (Default: 0) -[[RephistTrackTime]] **RephistTrackTime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: - Tells an authority, or other node tracking node reliability and history, - that fine-grained information about nodes can be discarded when it hasn't - changed for a given amount of time. (Default: 24 hours) - [[AuthDirHasIPv6Connectivity]] **AuthDirHasIPv6Connectivity** **0**|**1**:: Authoritative directories only. When set to 0, OR ports with an IPv6 address are not included in the authority's votes. When set to 1, diff --git a/src/common/address.c b/src/common/address.c index a2f4c93b91..ca263425f0 100644 --- a/src/common/address.c +++ b/src/common/address.c @@ -1697,7 +1697,7 @@ get_interface_address6_via_udp_socket_hack,(int severity, sa_family_t family, tor_addr_t *addr)) { - struct sockaddr_storage my_addr, target_addr; + struct sockaddr_storage target_addr; int sock=-1, r=-1; socklen_t addr_len; @@ -1740,21 +1740,19 @@ get_interface_address6_via_udp_socket_hack,(int severity, goto err; } - if (tor_getsockname(sock,(struct sockaddr*)&my_addr, &addr_len)) { + if (tor_addr_from_getsockname(addr, sock) < 0) { int e = tor_socket_errno(sock); log_fn(severity, LD_NET, "getsockname() to determine interface failed: %s", tor_socket_strerror(e)); goto err; } - if (tor_addr_from_sockaddr(addr, (struct sockaddr*)&my_addr, NULL) == 0) { - if (tor_addr_is_loopback(addr) || tor_addr_is_multicast(addr)) { - log_fn(severity, LD_NET, "Address that we determined via UDP socket" - " magic is unsuitable for public comms."); - } else { - r=0; - } - } + if (tor_addr_is_loopback(addr) || tor_addr_is_multicast(addr)) { + log_fn(severity, LD_NET, "Address that we determined via UDP socket" + " magic is unsuitable for public comms."); + } else { + r=0; + } err: if (sock >= 0) diff --git a/src/common/buffers.h b/src/common/buffers.h index 22a5f7bfa3..4275152de2 100644 --- a/src/common/buffers.h +++ b/src/common/buffers.h @@ -13,7 +13,6 @@ #define TOR_BUFFERS_H #include "compat.h" -#include "compat.h" #include "torint.h" #include "testsupport.h" diff --git a/src/common/compat.c b/src/common/compat.c index 7d9add50b2..6fdd6ecf00 100644 --- a/src/common/compat.c +++ b/src/common/compat.c @@ -100,7 +100,6 @@ SecureZeroMemory(PVOID ptr, SIZE_T cnt) /* Only use the linux prctl; the IRIX prctl is totally different */ #include <sys/prctl.h> #elif defined(__APPLE__) -#include <sys/types.h> #include <sys/ptrace.h> #endif /* defined(HAVE_SYS_PRCTL_H) && defined(__linux__) || ... */ @@ -116,7 +115,7 @@ SecureZeroMemory(PVOID ptr, SIZE_T cnt) #ifdef HAVE_SIGNAL_H #include <signal.h> #endif -#ifdef HAVE_SYS_MMAN_H +#ifdef HAVE_MMAP #include <sys/mman.h> #endif #ifdef HAVE_SYS_SYSLIMITS_H @@ -204,25 +203,17 @@ tor_rename(const char *path_old, const char *path_new) sandbox_intern_string(path_new)); } -/* Some MinGW builds have sys/mman.h, but not the corresponding symbols. - * Other configs rename the symbols using macros (including getpagesize). - * So check for sys/mman.h and unistd.h, and a getpagesize declaration. */ -#if (defined(HAVE_SYS_MMAN_H) && defined(HAVE_UNISTD_H) && \ - defined(HAVE_DECL_GETPAGESIZE)) -#define COMPAT_HAS_MMAN_AND_PAGESIZE -#endif - -#if defined(COMPAT_HAS_MMAN_AND_PAGESIZE) || \ - defined(RUNNING_DOXYGEN) +#if defined(HAVE_MMAP) || defined(RUNNING_DOXYGEN) /** Try to create a memory mapping for <b>filename</b> and return it. On - * failure, return NULL. Sets errno properly, using ERANGE to mean - * "empty file". */ + * failure, return NULL. Sets errno properly, using ERANGE to mean + * "empty file". Must only be called on trusted Tor-owned files, as changing + * the underlying file's size causes unspecified behavior. */ tor_mmap_t * tor_mmap_file(const char *filename) { int fd; /* router file */ char *string; - int page_size, result; + int result; tor_mmap_t *res; size_t size, filesize; struct stat st; @@ -251,13 +242,6 @@ tor_mmap_file(const char *filename) return NULL; } size = filesize = (size_t)(st.st_size); - /* - * Should we check for weird crap like mmapping a named pipe here, - * or just wait for if (!size) below to fail? - */ - /* ensure page alignment */ - page_size = getpagesize(); - size += (size%page_size) ? page_size-(size%page_size) : 0; if (st.st_size > SSIZE_T_CEILING || (off_t)size < st.st_size) { log_warn(LD_FS, "File \"%s\" is too large. Ignoring.",filename); @@ -418,40 +402,8 @@ tor_munmap_file(tor_mmap_t *handle) return 0; } #else -tor_mmap_t * -tor_mmap_file(const char *filename) -{ - struct stat st; - char *res = read_file_to_str(filename, RFTS_BIN|RFTS_IGNORE_MISSING, &st); - tor_mmap_t *handle; - if (! res) - return NULL; - handle = tor_malloc_zero(sizeof(tor_mmap_t)); - handle->data = res; - handle->size = st.st_size; - return handle; -} - -/** Unmap the file mapped with tor_mmap_file(), and return 0 for success - * or -1 for failure. - */ - -int -tor_munmap_file(tor_mmap_t *handle) -{ - char *d = NULL; - if (handle == NULL) - return 0; - - d = (char*)handle->data; - tor_free(d); - memwipe(handle, 0, sizeof(tor_mmap_t)); - tor_free(handle); - - /* Can't fail in this mmap()/munmap()-free case */ - return 0; -} -#endif /* defined(COMPAT_HAS_MMAN_AND_PAGESIZE) || ... || ... */ +#error "cannot implement tor_mmap_file" +#endif /* defined(HAVE_MMAP) || ... || ... */ /** Replacement for snprintf. Differs from platform snprintf in two * ways: First, always NUL-terminates its output. Second, always @@ -1392,6 +1344,24 @@ tor_getsockname,(tor_socket_t sock, struct sockaddr *address, return getsockname(sock, address, address_len); } +/** + * Find the local address associated with the socket <b>sock</b>, and + * place it in *<b>addr_out</b>. Return 0 on success, -1 on failure. + * + * (As tor_getsockname, but instead places the result in a tor_addr_t.) */ +int +tor_addr_from_getsockname(tor_addr_t *addr_out, tor_socket_t sock) +{ + struct sockaddr_storage ss; + socklen_t ss_len = sizeof(ss); + memset(&ss, 0, sizeof(ss)); + + if (tor_getsockname(sock, (struct sockaddr *) &ss, &ss_len) < 0) + return -1; + + return tor_addr_from_sockaddr(addr_out, (struct sockaddr *)&ss, NULL); +} + /** Turn <b>socket</b> into a nonblocking socket. Return 0 on success, -1 * on failure. */ diff --git a/src/common/compat.h b/src/common/compat.h index 3088e68355..c7e7f8d9ef 100644 --- a/src/common/compat.h +++ b/src/common/compat.h @@ -318,12 +318,12 @@ typedef struct tor_mmap_t { size_t size; /**< Size of the file. */ /* None of the fields below should be accessed from outside compat.c */ -#ifdef HAVE_SYS_MMAN_H +#ifdef HAVE_MMAP size_t mapping_size; /**< Size of the actual mapping. (This is this file * size, rounded up to the nearest page.) */ #elif defined _WIN32 HANDLE mmap_handle; -#endif /* defined(HAVE_SYS_MMAN_H) || ... */ +#endif /* defined(HAVE_MMAP) || ... */ } tor_mmap_t; @@ -510,6 +510,8 @@ int get_n_open_sockets(void); MOCK_DECL(int, tor_getsockname,(tor_socket_t socket, struct sockaddr *address, socklen_t *address_len)); +struct tor_addr_t; +int tor_addr_from_getsockname(struct tor_addr_t *addr_out, tor_socket_t sock); #define tor_socket_send(s, buf, len, flags) send(s, buf, len, flags) #define tor_socket_recv(s, buf, len, flags) recv(s, buf, len, flags) diff --git a/src/common/compat_libevent.c b/src/common/compat_libevent.c index 735385557c..0d0e4337bb 100644 --- a/src/common/compat_libevent.c +++ b/src/common/compat_libevent.c @@ -221,6 +221,121 @@ periodic_timer_free_(periodic_timer_t *timer) tor_free(timer); } +/** + * Type used to represent events that run directly from the main loop, + * either because they are activated from elsewhere in the code, or + * because they have a simple timeout. + * + * We use this type to avoid exposing Libevent's API throughout the rest + * of the codebase. + * + * This type can't be used for all events: it doesn't handle events that + * are triggered by signals or by sockets. + */ +struct mainloop_event_t { + struct event *ev; + void (*cb)(mainloop_event_t *, void *); + void *userdata; +}; + +/** + * Internal: Implements mainloop event using a libevent event. + */ +static void +mainloop_event_cb(evutil_socket_t fd, short what, void *arg) +{ + (void)fd; + (void)what; + mainloop_event_t *mev = arg; + mev->cb(mev, mev->userdata); +} + +/** + * Create and return a new mainloop_event_t to run the function <b>cb</b>. + * + * When run, the callback function will be passed the mainloop_event_t + * and <b>userdata</b> as its arguments. The <b>userdata</b> pointer + * must remain valid for as long as the mainloop_event_t event exists: + * it is your responsibility to free it. + * + * The event is not scheduled by default: Use mainloop_event_activate() + * or mainloop_event_schedule() to make it run. + */ +mainloop_event_t * +mainloop_event_new(void (*cb)(mainloop_event_t *, void *), + void *userdata) +{ + tor_assert(cb); + + struct event_base *base = tor_libevent_get_base(); + mainloop_event_t *mev = tor_malloc_zero(sizeof(mainloop_event_t)); + mev->ev = tor_event_new(base, -1, 0, mainloop_event_cb, mev); + tor_assert(mev->ev); + mev->cb = cb; + mev->userdata = userdata; + return mev; +} + +/** + * Schedule <b>event</b> to run in the main loop, immediately. If it is + * not scheduled, it will run anyway. If it is already scheduled to run + * later, it will run now instead. This function will have no effect if + * the event is already scheduled to run. + * + * This function may only be called from the main thread. + */ +void +mainloop_event_activate(mainloop_event_t *event) +{ + tor_assert(event); + event_active(event->ev, EV_READ, 1); +} + +/** Schedule <b>event</b> to run in the main loop, after a delay of <b>tv</b>. + * + * If the event is scheduled for a different time, cancel it and run + * after this delay instead. If the event is currently pending to run + * <em>now</b>, has no effect. + * + * Do not call this function with <b>tv</b> == NULL -- use + * mainloop_event_activate() instead. + * + * This function may only be called from the main thread. + */ +int +mainloop_event_schedule(mainloop_event_t *event, const struct timeval *tv) +{ + tor_assert(event); + if (BUG(tv == NULL)) { + // LCOV_EXCL_START + mainloop_event_activate(event); + return 0; + // LCOV_EXCL_STOP + } + return event_add(event->ev, tv); +} + +/** Cancel <b>event</b> if it is currently active or pending. (Do nothing if + * the event is not currently active or pending.) */ +void +mainloop_event_cancel(mainloop_event_t *event) +{ + if (!event) + return; + (void) event_del(event->ev); +} + +/** Cancel <b>event</b> and release all storage associated with it. */ +void +mainloop_event_free_(mainloop_event_t *event) +{ + if (!event) + return; + tor_event_free(event->ev); + memset(event, 0xb8, sizeof(*event)); + tor_free(event); +} + int tor_init_libevent_rng(void) { @@ -248,6 +363,38 @@ tor_libevent_free_all(void) the_event_base = NULL; } +/** + * Run the event loop for the provided event_base, handling events until + * something stops it. If <b>once</b> is set, then just poll-and-run + * once, then exit. Return 0 on success, -1 if an error occurred, or 1 + * if we exited because no events were pending or active. + * + * This isn't reentrant or multithreaded. + */ +int +tor_libevent_run_event_loop(struct event_base *base, int once) +{ + const int flags = once ? EVLOOP_ONCE : 0; + return event_base_loop(base, flags); +} + +/** Tell the event loop to exit after <b>delay</b>. If <b>delay</b> is NULL, + * instead exit after we're done running the currently active events. */ +void +tor_libevent_exit_loop_after_delay(struct event_base *base, + const struct timeval *delay) +{ + event_base_loopexit(base, delay); +} + +/** Tell the event loop to exit after running whichever callback is currently + * active. */ +void +tor_libevent_exit_loop_after_callback(struct event_base *base) +{ + event_base_loopbreak(base); +} + #if defined(LIBEVENT_VERSION_NUMBER) && \ LIBEVENT_VERSION_NUMBER >= V(2,1,1) && \ !defined(TOR_UNIT_TESTS) diff --git a/src/common/compat_libevent.h b/src/common/compat_libevent.h index 1853e50917..c8b0371564 100644 --- a/src/common/compat_libevent.h +++ b/src/common/compat_libevent.h @@ -7,8 +7,6 @@ #include "orconfig.h" #include "testsupport.h" -#include <event2/event.h> - void configure_libevent_logging(void); void suppress_libevent_log_msg(const char *msg); @@ -19,6 +17,9 @@ void suppress_libevent_log_msg(const char *msg); evdns_add_server_port_with_base(tor_libevent_get_base(), \ (sock),(tcp),(cb),(data)); +struct event; +struct event_base; + void tor_event_free_(struct event *ev); #define tor_event_free(ev) \ FREE_AND_NULL(struct event, tor_event_free_, (ev)) @@ -33,8 +34,16 @@ void periodic_timer_free_(periodic_timer_t *); #define periodic_timer_free(t) \ FREE_AND_NULL(periodic_timer_t, periodic_timer_free_, (t)) -#define tor_event_base_loopexit event_base_loopexit -#define tor_event_base_loopbreak event_base_loopbreak +typedef struct mainloop_event_t mainloop_event_t; +mainloop_event_t *mainloop_event_new(void (*cb)(mainloop_event_t *, void *), + void *userdata); +void mainloop_event_activate(mainloop_event_t *event); +int mainloop_event_schedule(mainloop_event_t *event, + const struct timeval *delay); +void mainloop_event_cancel(mainloop_event_t *event); +void mainloop_event_free_(mainloop_event_t *event); +#define mainloop_event_free(event) \ + FREE_AND_NULL(mainloop_event_t, mainloop_event_free_, (event)) /** Defines a configuration for using libevent with Tor: passed as an argument * to tor_libevent_initialize() to describe how we want to set up. */ @@ -63,6 +72,11 @@ void tor_gettimeofday_cache_set(const struct timeval *tv); void tor_libevent_postfork(void); #endif +int tor_libevent_run_event_loop(struct event_base *base, int once); +void tor_libevent_exit_loop_after_delay(struct event_base *base, + const struct timeval *delay); +void tor_libevent_exit_loop_after_callback(struct event_base *base); + #ifdef COMPAT_LIBEVENT_PRIVATE /** Macro: returns the number of a Libevent version as a 4-byte number, diff --git a/src/common/compat_winthreads.c b/src/common/compat_winthreads.c index 5f7ec94c23..7021344f6e 100644 --- a/src/common/compat_winthreads.c +++ b/src/common/compat_winthreads.c @@ -18,7 +18,6 @@ #include "util.h" #include "container.h" #include "torlog.h" -#include <process.h> /* This value is more or less total cargo-cult */ #define SPIN_COUNT 2000 diff --git a/src/common/compress.c b/src/common/compress.c index 47c93cf6a9..cb1549f1aa 100644 --- a/src/common/compress.c +++ b/src/common/compress.c @@ -663,3 +663,13 @@ tor_compress_init(void) tor_zstd_init(); } +/** Warn if we had any problems while setting up our compression libraries. + * + * (This isn't part of tor_compress_init, since the logs aren't set up yet.) + */ +void +tor_compress_log_init_warnings(void) +{ + tor_zstd_warn_if_version_mismatched(); +} + diff --git a/src/common/compress.h b/src/common/compress.h index 952102bf97..65d63a4386 100644 --- a/src/common/compress.h +++ b/src/common/compress.h @@ -87,6 +87,7 @@ void tor_compress_free_(tor_compress_state_t *state); size_t tor_compress_state_size(const tor_compress_state_t *state); void tor_compress_init(void); +void tor_compress_log_init_warnings(void); #endif /* !defined(TOR_COMPRESS_H) */ diff --git a/src/common/compress_zstd.c b/src/common/compress_zstd.c index 0db87d61b7..4024f5594d 100644 --- a/src/common/compress_zstd.c +++ b/src/common/compress_zstd.c @@ -18,6 +18,13 @@ #include "compress.h" #include "compress_zstd.h" +#ifdef ENABLE_ZSTD_ADVANCED_APIS +/* This is a lie, but we make sure it doesn't get us in trouble by wrapping + * all invocations of zstd's static-only functions in a check to make sure + * that the compile-time version matches the run-time version. */ +#define ZSTD_STATIC_LINKING_ONLY +#endif + #ifdef HAVE_ZSTD #include <zstd.h> #endif @@ -51,21 +58,31 @@ tor_zstd_method_supported(void) #endif } +#ifdef HAVE_ZSTD +/** Format a zstd version number as a string in <b>buf</b>. */ +static void +tor_zstd_format_version(char *buf, size_t buflen, unsigned version_number) +{ + tor_snprintf(buf, buflen, + "%u.%u.%u", + version_number / 10000 % 100, + version_number / 100 % 100, + version_number % 100); +} +#endif + +#define VERSION_STR_MAX_LEN 16 /* more than enough space for 99.99.99 */ + /** Return a string representation of the version of the currently running * version of libzstd. Returns NULL if Zstandard is unsupported. */ const char * tor_zstd_get_version_str(void) { #ifdef HAVE_ZSTD - static char version_str[16]; - size_t version_number; + static char version_str[VERSION_STR_MAX_LEN]; - version_number = ZSTD_versionNumber(); - tor_snprintf(version_str, sizeof(version_str), - "%d.%d.%d", - (int) version_number / 10000 % 100, - (int) version_number / 100 % 100, - (int) version_number % 100); + tor_zstd_format_version(version_str, sizeof(version_str), + ZSTD_versionNumber()); return version_str; #else /* !(defined(HAVE_ZSTD)) */ @@ -85,6 +102,26 @@ tor_zstd_get_header_version_str(void) #endif } +#ifdef TOR_UNIT_TESTS +static int static_apis_disable_for_testing = 0; +#endif + +/** Return true iff we can use the "static-only" APIs. */ +int +tor_zstd_can_use_static_apis(void) +{ +#if defined(ZSTD_STATIC_LINKING_ONLY) && defined(HAVE_ZSTD) +#ifdef TOR_UNIT_TESTS + if (static_apis_disable_for_testing) { + return 0; + } +#endif + return (ZSTD_VERSION_NUMBER == ZSTD_versionNumber()); +#else + return 0; +#endif +} + /** Internal Zstandard state for incremental compression/decompression. * The body of this struct is not exposed. */ struct tor_zstd_compress_state_t { @@ -112,9 +149,11 @@ struct tor_zstd_compress_state_t { #ifdef HAVE_ZSTD /** Return an approximate number of bytes stored in memory to hold the - * Zstandard compression/decompression state. */ + * Zstandard compression/decompression state. This is a fake estimate + * based on inspecting the zstd source: tor_zstd_state_size_precalc() is + * more accurate when it's allowed to use "static-only" functions */ static size_t -tor_zstd_state_size_precalc(int compress, int preset) +tor_zstd_state_size_precalc_fake(int compress, int preset) { tor_assert(preset > 0); @@ -171,6 +210,28 @@ tor_zstd_state_size_precalc(int compress, int preset) return memory_usage; } + +/** Return an approximate number of bytes stored in memory to hold the + * Zstandard compression/decompression state. */ +static size_t +tor_zstd_state_size_precalc(int compress, int preset) +{ +#ifdef ZSTD_STATIC_LINKING_ONLY + if (tor_zstd_can_use_static_apis()) { + if (compress) { +#ifdef HAVE_ZSTD_ESTIMATECSTREAMSIZE + return ZSTD_estimateCStreamSize(preset); +#endif + } else { +#ifdef HAVE_ZSTD_ESTIMATEDCTXSIZE + /* Could use DStream, but that takes a windowSize. */ + return ZSTD_estimateDCtxSize(); +#endif + } + } +#endif + return tor_zstd_state_size_precalc_fake(compress, preset); +} #endif /* defined(HAVE_ZSTD) */ /** Construct and return a tor_zstd_compress_state_t object using @@ -440,3 +501,34 @@ tor_zstd_init(void) atomic_counter_init(&total_zstd_allocation); } +/** Warn if the header and library versions don't match. */ +void +tor_zstd_warn_if_version_mismatched(void) +{ +#if defined(HAVE_ZSTD) && defined(ENABLE_ZSTD_ADVANCED_APIS) + if (! tor_zstd_can_use_static_apis()) { + char header_version[VERSION_STR_MAX_LEN]; + char runtime_version[VERSION_STR_MAX_LEN]; + tor_zstd_format_version(header_version, sizeof(header_version), + ZSTD_VERSION_NUMBER); + tor_zstd_format_version(runtime_version, sizeof(runtime_version), + ZSTD_versionNumber()); + + log_warn(LD_GENERAL, + "Tor was compiled with zstd %s, but is running with zstd %s. " + "For safety, we'll avoid using advanced zstd functionality.", + header_version, runtime_version); + } +#endif +} + +#ifdef TOR_UNIT_TESTS +/** Testing only: disable usage of static-only APIs, so we can make sure that + * we still work without them. */ +void +tor_zstd_set_static_apis_disabled_for_testing(int disabled) +{ + static_apis_disable_for_testing = disabled; +} +#endif + diff --git a/src/common/compress_zstd.h b/src/common/compress_zstd.h index 9bca24ded7..bd42cf65ce 100644 --- a/src/common/compress_zstd.h +++ b/src/common/compress_zstd.h @@ -17,6 +17,8 @@ const char *tor_zstd_get_version_str(void); const char *tor_zstd_get_header_version_str(void); +int tor_zstd_can_use_static_apis(void); + /** Internal state for an incremental Zstandard compression/decompression. */ typedef struct tor_zstd_compress_state_t tor_zstd_compress_state_t; @@ -41,6 +43,11 @@ size_t tor_zstd_compress_state_size(const tor_zstd_compress_state_t *state); size_t tor_zstd_get_total_allocation(void); void tor_zstd_init(void); +void tor_zstd_warn_if_version_mismatched(void); + +#ifdef TOR_UNIT_TESTS +void tor_zstd_set_static_apis_disabled_for_testing(int disabled); +#endif #endif /* !defined(TOR_COMPRESS_ZSTD_H) */ diff --git a/src/common/container.c b/src/common/container.c index 54b0b2028f..5386e6458b 100644 --- a/src/common/container.c +++ b/src/common/container.c @@ -15,7 +15,7 @@ #include "util.h" #include "torlog.h" #include "container.h" -#include "crypto.h" +#include "crypto_digest.h" #include <stdlib.h> #include <string.h> diff --git a/src/common/crypto.c b/src/common/crypto.c index d85aca4004..9fcd17742c 100644 --- a/src/common/crypto.c +++ b/src/common/crypto.c @@ -29,6 +29,7 @@ #include "crypto_ed25519.h" #include "crypto_format.h" #include "crypto_rsa.h" +#include "crypto_digest.h" DISABLE_GCC_WARNING(redundant-decls) @@ -397,266 +398,6 @@ crypto_cipher_free_(crypto_cipher_t *env) aes_cipher_free(env); } -/* public key crypto */ - -/** Check a siglen-byte long signature at <b>sig</b> against - * <b>datalen</b> bytes of data at <b>data</b>, using the public key - * in <b>env</b>. Return 0 if <b>sig</b> is a correct signature for - * SHA1(data). Else return -1. - */ -MOCK_IMPL(int, -crypto_pk_public_checksig_digest,(crypto_pk_t *env, const char *data, - size_t datalen, const char *sig, - size_t siglen)) -{ - char digest[DIGEST_LEN]; - char *buf; - size_t buflen; - int r; - - tor_assert(env); - tor_assert(data); - tor_assert(sig); - tor_assert(datalen < SIZE_T_CEILING); - tor_assert(siglen < SIZE_T_CEILING); - - if (crypto_digest(digest,data,datalen)<0) { - log_warn(LD_BUG, "couldn't compute digest"); - return -1; - } - buflen = crypto_pk_keysize(env); - buf = tor_malloc(buflen); - r = crypto_pk_public_checksig(env,buf,buflen,sig,siglen); - if (r != DIGEST_LEN) { - log_warn(LD_CRYPTO, "Invalid signature"); - tor_free(buf); - return -1; - } - if (tor_memneq(buf, digest, DIGEST_LEN)) { - log_warn(LD_CRYPTO, "Signature mismatched with digest."); - tor_free(buf); - return -1; - } - tor_free(buf); - - return 0; -} - -/** Compute a SHA1 digest of <b>fromlen</b> bytes of data stored at - * <b>from</b>; sign the data with the private key in <b>env</b>, and - * store it in <b>to</b>. Return the number of bytes written on - * success, and -1 on failure. - * - * <b>tolen</b> is the number of writable bytes in <b>to</b>, and must be - * at least the length of the modulus of <b>env</b>. - */ -int -crypto_pk_private_sign_digest(crypto_pk_t *env, char *to, size_t tolen, - const char *from, size_t fromlen) -{ - int r; - char digest[DIGEST_LEN]; - if (crypto_digest(digest,from,fromlen)<0) - return -1; - r = crypto_pk_private_sign(env,to,tolen,digest,DIGEST_LEN); - memwipe(digest, 0, sizeof(digest)); - return r; -} - -/** Perform a hybrid (public/secret) encryption on <b>fromlen</b> - * bytes of data from <b>from</b>, with padding type 'padding', - * storing the results on <b>to</b>. - * - * Returns the number of bytes written on success, -1 on failure. - * - * The encrypted data consists of: - * - The source data, padded and encrypted with the public key, if the - * padded source data is no longer than the public key, and <b>force</b> - * is false, OR - * - The beginning of the source data prefixed with a 16-byte symmetric key, - * padded and encrypted with the public key; followed by the rest of - * the source data encrypted in AES-CTR mode with the symmetric key. - * - * NOTE that this format does not authenticate the symmetrically encrypted - * part of the data, and SHOULD NOT BE USED for new protocols. - */ -int -crypto_pk_obsolete_public_hybrid_encrypt(crypto_pk_t *env, - char *to, size_t tolen, - const char *from, - size_t fromlen, - int padding, int force) -{ - int overhead, outlen, r; - size_t pkeylen, symlen; - crypto_cipher_t *cipher = NULL; - char *buf = NULL; - - tor_assert(env); - tor_assert(from); - tor_assert(to); - tor_assert(fromlen < SIZE_T_CEILING); - - overhead = crypto_get_rsa_padding_overhead(crypto_get_rsa_padding(padding)); - pkeylen = crypto_pk_keysize(env); - - if (!force && fromlen+overhead <= pkeylen) { - /* It all fits in a single encrypt. */ - return crypto_pk_public_encrypt(env,to, - tolen, - from,fromlen,padding); - } - tor_assert(tolen >= fromlen + overhead + CIPHER_KEY_LEN); - tor_assert(tolen >= pkeylen); - - char key[CIPHER_KEY_LEN]; - crypto_rand(key, sizeof(key)); /* generate a new key. */ - cipher = crypto_cipher_new(key); - - buf = tor_malloc(pkeylen+1); - memcpy(buf, key, CIPHER_KEY_LEN); - memcpy(buf+CIPHER_KEY_LEN, from, pkeylen-overhead-CIPHER_KEY_LEN); - - /* Length of symmetrically encrypted data. */ - symlen = fromlen-(pkeylen-overhead-CIPHER_KEY_LEN); - - outlen = crypto_pk_public_encrypt(env,to,tolen,buf,pkeylen-overhead,padding); - if (outlen!=(int)pkeylen) { - goto err; - } - r = crypto_cipher_encrypt(cipher, to+outlen, - from+pkeylen-overhead-CIPHER_KEY_LEN, symlen); - - if (r<0) goto err; - memwipe(buf, 0, pkeylen); - memwipe(key, 0, sizeof(key)); - tor_free(buf); - crypto_cipher_free(cipher); - tor_assert(outlen+symlen < INT_MAX); - return (int)(outlen + symlen); - err: - - memwipe(buf, 0, pkeylen); - memwipe(key, 0, sizeof(key)); - tor_free(buf); - crypto_cipher_free(cipher); - return -1; -} - -/** Invert crypto_pk_obsolete_public_hybrid_encrypt. Returns the number of - * bytes written on success, -1 on failure. - * - * NOTE that this format does not authenticate the symmetrically encrypted - * part of the data, and SHOULD NOT BE USED for new protocols. - */ -int -crypto_pk_obsolete_private_hybrid_decrypt(crypto_pk_t *env, - char *to, - size_t tolen, - const char *from, - size_t fromlen, - int padding, int warnOnFailure) -{ - int outlen, r; - size_t pkeylen; - crypto_cipher_t *cipher = NULL; - char *buf = NULL; - - tor_assert(fromlen < SIZE_T_CEILING); - pkeylen = crypto_pk_keysize(env); - - if (fromlen <= pkeylen) { - return crypto_pk_private_decrypt(env,to,tolen,from,fromlen,padding, - warnOnFailure); - } - - buf = tor_malloc(pkeylen); - outlen = crypto_pk_private_decrypt(env,buf,pkeylen,from,pkeylen,padding, - warnOnFailure); - if (outlen<0) { - log_fn(warnOnFailure?LOG_WARN:LOG_DEBUG, LD_CRYPTO, - "Error decrypting public-key data"); - goto err; - } - if (outlen < CIPHER_KEY_LEN) { - log_fn(warnOnFailure?LOG_WARN:LOG_INFO, LD_CRYPTO, - "No room for a symmetric key"); - goto err; - } - cipher = crypto_cipher_new(buf); - if (!cipher) { - goto err; - } - memcpy(to,buf+CIPHER_KEY_LEN,outlen-CIPHER_KEY_LEN); - outlen -= CIPHER_KEY_LEN; - tor_assert(tolen - outlen >= fromlen - pkeylen); - r = crypto_cipher_decrypt(cipher, to+outlen, from+pkeylen, fromlen-pkeylen); - if (r<0) - goto err; - memwipe(buf,0,pkeylen); - tor_free(buf); - crypto_cipher_free(cipher); - tor_assert(outlen + fromlen < INT_MAX); - return (int)(outlen + (fromlen-pkeylen)); - err: - memwipe(buf,0,pkeylen); - tor_free(buf); - crypto_cipher_free(cipher); - return -1; -} - -/** Given a private or public key <b>pk</b>, put a SHA1 hash of the - * public key into <b>digest_out</b> (must have DIGEST_LEN bytes of space). - * Return 0 on success, -1 on failure. - */ -int -crypto_pk_get_digest(const crypto_pk_t *pk, char *digest_out) -{ - char *buf; - size_t buflen; - int len; - int rv = -1; - - buflen = crypto_pk_keysize(pk)*2; - buf = tor_malloc(buflen); - len = crypto_pk_asn1_encode(pk, buf, buflen); - if (len < 0) - goto done; - - if (crypto_digest(digest_out, buf, len) < 0) - goto done; - - rv = 0; - done: - tor_free(buf); - return rv; -} - -/** Compute all digests of the DER encoding of <b>pk</b>, and store them - * in <b>digests_out</b>. Return 0 on success, -1 on failure. */ -int -crypto_pk_get_common_digests(crypto_pk_t *pk, common_digests_t *digests_out) -{ - char *buf; - size_t buflen; - int len; - int rv = -1; - - buflen = crypto_pk_keysize(pk)*2; - buf = tor_malloc(buflen); - len = crypto_pk_asn1_encode(pk, buf, buflen); - if (len < 0) - goto done; - - if (crypto_common_digests(digests_out, (char*)buf, len) < 0) - goto done; - - rv = 0; - done: - tor_free(buf); - return rv; -} - /** Copy <b>in</b> to the <b>outlen</b>-byte buffer <b>out</b>, adding spaces * every four characters. */ void @@ -788,524 +529,6 @@ crypto_cipher_decrypt_with_iv(const char *key, return (int)(fromlen - CIPHER_IV_LEN); } -/* SHA-1 */ - -/** Compute the SHA1 digest of the <b>len</b> bytes on data stored in - * <b>m</b>. Write the DIGEST_LEN byte result into <b>digest</b>. - * Return 0 on success, -1 on failure. - */ -int -crypto_digest(char *digest, const char *m, size_t len) -{ - tor_assert(m); - tor_assert(digest); - if (SHA1((const unsigned char*)m,len,(unsigned char*)digest) == NULL) - return -1; - return 0; -} - -/** Compute a 256-bit digest of <b>len</b> bytes in data stored in <b>m</b>, - * using the algorithm <b>algorithm</b>. Write the DIGEST_LEN256-byte result - * into <b>digest</b>. Return 0 on success, -1 on failure. */ -int -crypto_digest256(char *digest, const char *m, size_t len, - digest_algorithm_t algorithm) -{ - tor_assert(m); - tor_assert(digest); - tor_assert(algorithm == DIGEST_SHA256 || algorithm == DIGEST_SHA3_256); - - int ret = 0; - if (algorithm == DIGEST_SHA256) - ret = (SHA256((const uint8_t*)m,len,(uint8_t*)digest) != NULL); - else - ret = (sha3_256((uint8_t *)digest, DIGEST256_LEN,(const uint8_t *)m, len) - > -1); - - if (!ret) - return -1; - return 0; -} - -/** Compute a 512-bit digest of <b>len</b> bytes in data stored in <b>m</b>, - * using the algorithm <b>algorithm</b>. Write the DIGEST_LEN512-byte result - * into <b>digest</b>. Return 0 on success, -1 on failure. */ -int -crypto_digest512(char *digest, const char *m, size_t len, - digest_algorithm_t algorithm) -{ - tor_assert(m); - tor_assert(digest); - tor_assert(algorithm == DIGEST_SHA512 || algorithm == DIGEST_SHA3_512); - - int ret = 0; - if (algorithm == DIGEST_SHA512) - ret = (SHA512((const unsigned char*)m,len,(unsigned char*)digest) - != NULL); - else - ret = (sha3_512((uint8_t*)digest, DIGEST512_LEN, (const uint8_t*)m, len) - > -1); - - if (!ret) - return -1; - return 0; -} - -/** Set the common_digests_t in <b>ds_out</b> to contain every digest on the - * <b>len</b> bytes in <b>m</b> that we know how to compute. Return 0 on - * success, -1 on failure. */ -int -crypto_common_digests(common_digests_t *ds_out, const char *m, size_t len) -{ - tor_assert(ds_out); - memset(ds_out, 0, sizeof(*ds_out)); - if (crypto_digest(ds_out->d[DIGEST_SHA1], m, len) < 0) - return -1; - if (crypto_digest256(ds_out->d[DIGEST_SHA256], m, len, DIGEST_SHA256) < 0) - return -1; - - return 0; -} - -/** Return the name of an algorithm, as used in directory documents. */ -const char * -crypto_digest_algorithm_get_name(digest_algorithm_t alg) -{ - switch (alg) { - case DIGEST_SHA1: - return "sha1"; - case DIGEST_SHA256: - return "sha256"; - case DIGEST_SHA512: - return "sha512"; - case DIGEST_SHA3_256: - return "sha3-256"; - case DIGEST_SHA3_512: - return "sha3-512"; - // LCOV_EXCL_START - default: - tor_fragile_assert(); - return "??unknown_digest??"; - // LCOV_EXCL_STOP - } -} - -/** Given the name of a digest algorithm, return its integer value, or -1 if - * the name is not recognized. */ -int -crypto_digest_algorithm_parse_name(const char *name) -{ - if (!strcmp(name, "sha1")) - return DIGEST_SHA1; - else if (!strcmp(name, "sha256")) - return DIGEST_SHA256; - else if (!strcmp(name, "sha512")) - return DIGEST_SHA512; - else if (!strcmp(name, "sha3-256")) - return DIGEST_SHA3_256; - else if (!strcmp(name, "sha3-512")) - return DIGEST_SHA3_512; - else - return -1; -} - -/** Given an algorithm, return the digest length in bytes. */ -size_t -crypto_digest_algorithm_get_length(digest_algorithm_t alg) -{ - switch (alg) { - case DIGEST_SHA1: - return DIGEST_LEN; - case DIGEST_SHA256: - return DIGEST256_LEN; - case DIGEST_SHA512: - return DIGEST512_LEN; - case DIGEST_SHA3_256: - return DIGEST256_LEN; - case DIGEST_SHA3_512: - return DIGEST512_LEN; - default: - tor_assert(0); // LCOV_EXCL_LINE - return 0; /* Unreachable */ // LCOV_EXCL_LINE - } -} - -/** Intermediate information about the digest of a stream of data. */ -struct crypto_digest_t { - digest_algorithm_t algorithm; /**< Which algorithm is in use? */ - /** State for the digest we're using. Only one member of the - * union is usable, depending on the value of <b>algorithm</b>. Note also - * that space for other members might not even be allocated! - */ - union { - SHA_CTX sha1; /**< state for SHA1 */ - SHA256_CTX sha2; /**< state for SHA256 */ - SHA512_CTX sha512; /**< state for SHA512 */ - keccak_state sha3; /**< state for SHA3-[256,512] */ - } d; -}; - -#ifdef TOR_UNIT_TESTS - -digest_algorithm_t -crypto_digest_get_algorithm(crypto_digest_t *digest) -{ - tor_assert(digest); - - return digest->algorithm; -} - -#endif /* defined(TOR_UNIT_TESTS) */ - -/** - * Return the number of bytes we need to malloc in order to get a - * crypto_digest_t for <b>alg</b>, or the number of bytes we need to wipe - * when we free one. - */ -static size_t -crypto_digest_alloc_bytes(digest_algorithm_t alg) -{ - /* Helper: returns the number of bytes in the 'f' field of 'st' */ -#define STRUCT_FIELD_SIZE(st, f) (sizeof( ((st*)0)->f )) - /* Gives the length of crypto_digest_t through the end of the field 'd' */ -#define END_OF_FIELD(f) (offsetof(crypto_digest_t, f) + \ - STRUCT_FIELD_SIZE(crypto_digest_t, f)) - switch (alg) { - case DIGEST_SHA1: - return END_OF_FIELD(d.sha1); - case DIGEST_SHA256: - return END_OF_FIELD(d.sha2); - case DIGEST_SHA512: - return END_OF_FIELD(d.sha512); - case DIGEST_SHA3_256: - case DIGEST_SHA3_512: - return END_OF_FIELD(d.sha3); - default: - tor_assert(0); // LCOV_EXCL_LINE - return 0; // LCOV_EXCL_LINE - } -#undef END_OF_FIELD -#undef STRUCT_FIELD_SIZE -} - -/** - * Internal function: create and return a new digest object for 'algorithm'. - * Does not typecheck the algorithm. - */ -static crypto_digest_t * -crypto_digest_new_internal(digest_algorithm_t algorithm) -{ - crypto_digest_t *r = tor_malloc(crypto_digest_alloc_bytes(algorithm)); - r->algorithm = algorithm; - - switch (algorithm) - { - case DIGEST_SHA1: - SHA1_Init(&r->d.sha1); - break; - case DIGEST_SHA256: - SHA256_Init(&r->d.sha2); - break; - case DIGEST_SHA512: - SHA512_Init(&r->d.sha512); - break; - case DIGEST_SHA3_256: - keccak_digest_init(&r->d.sha3, 256); - break; - case DIGEST_SHA3_512: - keccak_digest_init(&r->d.sha3, 512); - break; - default: - tor_assert_unreached(); - } - - return r; -} - -/** Allocate and return a new digest object to compute SHA1 digests. - */ -crypto_digest_t * -crypto_digest_new(void) -{ - return crypto_digest_new_internal(DIGEST_SHA1); -} - -/** Allocate and return a new digest object to compute 256-bit digests - * using <b>algorithm</b>. */ -crypto_digest_t * -crypto_digest256_new(digest_algorithm_t algorithm) -{ - tor_assert(algorithm == DIGEST_SHA256 || algorithm == DIGEST_SHA3_256); - return crypto_digest_new_internal(algorithm); -} - -/** Allocate and return a new digest object to compute 512-bit digests - * using <b>algorithm</b>. */ -crypto_digest_t * -crypto_digest512_new(digest_algorithm_t algorithm) -{ - tor_assert(algorithm == DIGEST_SHA512 || algorithm == DIGEST_SHA3_512); - return crypto_digest_new_internal(algorithm); -} - -/** Deallocate a digest object. - */ -void -crypto_digest_free_(crypto_digest_t *digest) -{ - if (!digest) - return; - size_t bytes = crypto_digest_alloc_bytes(digest->algorithm); - memwipe(digest, 0, bytes); - tor_free(digest); -} - -/** Add <b>len</b> bytes from <b>data</b> to the digest object. - */ -void -crypto_digest_add_bytes(crypto_digest_t *digest, const char *data, - size_t len) -{ - tor_assert(digest); - tor_assert(data); - /* Using the SHA*_*() calls directly means we don't support doing - * SHA in hardware. But so far the delay of getting the question - * to the hardware, and hearing the answer, is likely higher than - * just doing it ourselves. Hashes are fast. - */ - switch (digest->algorithm) { - case DIGEST_SHA1: - SHA1_Update(&digest->d.sha1, (void*)data, len); - break; - case DIGEST_SHA256: - SHA256_Update(&digest->d.sha2, (void*)data, len); - break; - case DIGEST_SHA512: - SHA512_Update(&digest->d.sha512, (void*)data, len); - break; - case DIGEST_SHA3_256: /* FALLSTHROUGH */ - case DIGEST_SHA3_512: - keccak_digest_update(&digest->d.sha3, (const uint8_t *)data, len); - break; - default: - /* LCOV_EXCL_START */ - tor_fragile_assert(); - break; - /* LCOV_EXCL_STOP */ - } -} - -/** Compute the hash of the data that has been passed to the digest - * object; write the first out_len bytes of the result to <b>out</b>. - * <b>out_len</b> must be \<= DIGEST512_LEN. - */ -void -crypto_digest_get_digest(crypto_digest_t *digest, - char *out, size_t out_len) -{ - unsigned char r[DIGEST512_LEN]; - crypto_digest_t tmpenv; - tor_assert(digest); - tor_assert(out); - tor_assert(out_len <= crypto_digest_algorithm_get_length(digest->algorithm)); - - /* The SHA-3 code handles copying into a temporary ctx, and also can handle - * short output buffers by truncating appropriately. */ - if (digest->algorithm == DIGEST_SHA3_256 || - digest->algorithm == DIGEST_SHA3_512) { - keccak_digest_sum(&digest->d.sha3, (uint8_t *)out, out_len); - return; - } - - const size_t alloc_bytes = crypto_digest_alloc_bytes(digest->algorithm); - /* memcpy into a temporary ctx, since SHA*_Final clears the context */ - memcpy(&tmpenv, digest, alloc_bytes); - switch (digest->algorithm) { - case DIGEST_SHA1: - SHA1_Final(r, &tmpenv.d.sha1); - break; - case DIGEST_SHA256: - SHA256_Final(r, &tmpenv.d.sha2); - break; - case DIGEST_SHA512: - SHA512_Final(r, &tmpenv.d.sha512); - break; -//LCOV_EXCL_START - case DIGEST_SHA3_256: /* FALLSTHROUGH */ - case DIGEST_SHA3_512: - default: - log_warn(LD_BUG, "Handling unexpected algorithm %d", digest->algorithm); - /* This is fatal, because it should never happen. */ - tor_assert_unreached(); - break; -//LCOV_EXCL_STOP - } - memcpy(out, r, out_len); - memwipe(r, 0, sizeof(r)); -} - -/** Allocate and return a new digest object with the same state as - * <b>digest</b> - */ -crypto_digest_t * -crypto_digest_dup(const crypto_digest_t *digest) -{ - tor_assert(digest); - const size_t alloc_bytes = crypto_digest_alloc_bytes(digest->algorithm); - return tor_memdup(digest, alloc_bytes); -} - -/** Replace the state of the digest object <b>into</b> with the state - * of the digest object <b>from</b>. Requires that 'into' and 'from' - * have the same digest type. - */ -void -crypto_digest_assign(crypto_digest_t *into, - const crypto_digest_t *from) -{ - tor_assert(into); - tor_assert(from); - tor_assert(into->algorithm == from->algorithm); - const size_t alloc_bytes = crypto_digest_alloc_bytes(from->algorithm); - memcpy(into,from,alloc_bytes); -} - -/** Given a list of strings in <b>lst</b>, set the <b>len_out</b>-byte digest - * at <b>digest_out</b> to the hash of the concatenation of those strings, - * plus the optional string <b>append</b>, computed with the algorithm - * <b>alg</b>. - * <b>out_len</b> must be \<= DIGEST512_LEN. */ -void -crypto_digest_smartlist(char *digest_out, size_t len_out, - const smartlist_t *lst, - const char *append, - digest_algorithm_t alg) -{ - crypto_digest_smartlist_prefix(digest_out, len_out, NULL, lst, append, alg); -} - -/** Given a list of strings in <b>lst</b>, set the <b>len_out</b>-byte digest - * at <b>digest_out</b> to the hash of the concatenation of: the - * optional string <b>prepend</b>, those strings, - * and the optional string <b>append</b>, computed with the algorithm - * <b>alg</b>. - * <b>len_out</b> must be \<= DIGEST512_LEN. */ -void -crypto_digest_smartlist_prefix(char *digest_out, size_t len_out, - const char *prepend, - const smartlist_t *lst, - const char *append, - digest_algorithm_t alg) -{ - crypto_digest_t *d = crypto_digest_new_internal(alg); - if (prepend) - crypto_digest_add_bytes(d, prepend, strlen(prepend)); - SMARTLIST_FOREACH(lst, const char *, cp, - crypto_digest_add_bytes(d, cp, strlen(cp))); - if (append) - crypto_digest_add_bytes(d, append, strlen(append)); - crypto_digest_get_digest(d, digest_out, len_out); - crypto_digest_free(d); -} - -/** Compute the HMAC-SHA-256 of the <b>msg_len</b> bytes in <b>msg</b>, using - * the <b>key</b> of length <b>key_len</b>. Store the DIGEST256_LEN-byte - * result in <b>hmac_out</b>. Asserts on failure. - */ -void -crypto_hmac_sha256(char *hmac_out, - const char *key, size_t key_len, - const char *msg, size_t msg_len) -{ - unsigned char *rv = NULL; - /* If we've got OpenSSL >=0.9.8 we can use its hmac implementation. */ - tor_assert(key_len < INT_MAX); - tor_assert(msg_len < INT_MAX); - tor_assert(hmac_out); - rv = HMAC(EVP_sha256(), key, (int)key_len, (unsigned char*)msg, (int)msg_len, - (unsigned char*)hmac_out, NULL); - tor_assert(rv); -} - -/** Compute a MAC using SHA3-256 of <b>msg_len</b> bytes in <b>msg</b> using a - * <b>key</b> of length <b>key_len</b> and a <b>salt</b> of length - * <b>salt_len</b>. Store the result of <b>len_out</b> bytes in in - * <b>mac_out</b>. This function can't fail. */ -void -crypto_mac_sha3_256(uint8_t *mac_out, size_t len_out, - const uint8_t *key, size_t key_len, - const uint8_t *msg, size_t msg_len) -{ - crypto_digest_t *digest; - - const uint64_t key_len_netorder = tor_htonll(key_len); - - tor_assert(mac_out); - tor_assert(key); - tor_assert(msg); - - digest = crypto_digest256_new(DIGEST_SHA3_256); - - /* Order matters here that is any subsystem using this function should - * expect this very precise ordering in the MAC construction. */ - crypto_digest_add_bytes(digest, (const char *) &key_len_netorder, - sizeof(key_len_netorder)); - crypto_digest_add_bytes(digest, (const char *) key, key_len); - crypto_digest_add_bytes(digest, (const char *) msg, msg_len); - crypto_digest_get_digest(digest, (char *) mac_out, len_out); - crypto_digest_free(digest); -} - -/** Internal state for a eXtendable-Output Function (XOF). */ -struct crypto_xof_t { - keccak_state s; -}; - -/** Allocate a new XOF object backed by SHAKE-256. The security level - * provided is a function of the length of the output used. Read and - * understand FIPS-202 A.2 "Additional Consideration for Extendable-Output - * Functions" before using this construct. - */ -crypto_xof_t * -crypto_xof_new(void) -{ - crypto_xof_t *xof; - xof = tor_malloc(sizeof(crypto_xof_t)); - keccak_xof_init(&xof->s, 256); - return xof; -} - -/** Absorb bytes into a XOF object. Must not be called after a call to - * crypto_xof_squeeze_bytes() for the same instance, and will assert - * if attempted. - */ -void -crypto_xof_add_bytes(crypto_xof_t *xof, const uint8_t *data, size_t len) -{ - int i = keccak_xof_absorb(&xof->s, data, len); - tor_assert(i == 0); -} - -/** Squeeze bytes out of a XOF object. Calling this routine will render - * the XOF instance ineligible to absorb further data. - */ -void -crypto_xof_squeeze_bytes(crypto_xof_t *xof, uint8_t *out, size_t len) -{ - int i = keccak_xof_squeeze(&xof->s, out, len); - tor_assert(i == 0); -} - -/** Cleanse and deallocate a XOF object. */ -void -crypto_xof_free_(crypto_xof_t *xof) -{ - if (!xof) - return; - memwipe(xof, 0, sizeof(crypto_xof_t)); - tor_free(xof); -} - /* DH */ /** Our DH 'g' parameter */ diff --git a/src/common/crypto.h b/src/common/crypto.h index a9c8837b9e..b586790329 100644 --- a/src/common/crypto.h +++ b/src/common/crypto.h @@ -24,13 +24,6 @@ #include "keccak-tiny/keccak-tiny.h" -/** Length of the output of our message digest. */ -#define DIGEST_LEN 20 -/** Length of the output of our second (improved) message digests. (For now - * this is just sha256, but it could be any other 256-bit digest.) */ -#define DIGEST256_LEN 32 -/** Length of the output of our 64-bit optimized message digests (SHA512). */ -#define DIGEST512_LEN 64 /** Length of our symmetric cipher's keys of 128-bit. */ #define CIPHER_KEY_LEN 16 /** Length of our symmetric cipher's IV of 128-bit. */ @@ -40,54 +33,11 @@ /** Length of our DH keys. */ #define DH_BYTES (1024/8) -/** Length of a sha1 message digest when encoded in base32 with trailing = - * signs removed. */ -#define BASE32_DIGEST_LEN 32 -/** Length of a sha1 message digest when encoded in base64 with trailing = - * signs removed. */ -#define BASE64_DIGEST_LEN 27 -/** Length of a sha256 message digest when encoded in base64 with trailing = - * signs removed. */ -#define BASE64_DIGEST256_LEN 43 -/** Length of a sha512 message digest when encoded in base64 with trailing = - * signs removed. */ -#define BASE64_DIGEST512_LEN 86 - /** Length of encoded public key fingerprints, including space; but not * including terminating NUL. */ #define FINGERPRINT_LEN 49 -/** Length of hex encoding of SHA1 digest, not including final NUL. */ -#define HEX_DIGEST_LEN 40 -/** Length of hex encoding of SHA256 digest, not including final NUL. */ -#define HEX_DIGEST256_LEN 64 -/** Length of hex encoding of SHA512 digest, not including final NUL. */ -#define HEX_DIGEST512_LEN 128 - -typedef enum { - DIGEST_SHA1 = 0, - DIGEST_SHA256 = 1, - DIGEST_SHA512 = 2, - DIGEST_SHA3_256 = 3, - DIGEST_SHA3_512 = 4, -} digest_algorithm_t; -#define N_DIGEST_ALGORITHMS (DIGEST_SHA3_512+1) -#define N_COMMON_DIGEST_ALGORITHMS (DIGEST_SHA256+1) - -/** A set of all the digests we commonly compute, taken on a single - * string. Any digests that are shorter than 512 bits are right-padded - * with 0 bits. - * - * Note that this representation wastes 44 bytes for the SHA1 case, so - * don't use it for anything where we need to allocate a whole bunch at - * once. - **/ -typedef struct { - char d[N_COMMON_DIGEST_ALGORITHMS][DIGEST256_LEN]; -} common_digests_t; typedef struct aes_cnt_cipher crypto_cipher_t; -typedef struct crypto_digest_t crypto_digest_t; -typedef struct crypto_xof_t crypto_xof_t; typedef struct crypto_dh_t crypto_dh_t; /* global state */ @@ -114,24 +64,6 @@ void crypto_cipher_free_(crypto_cipher_t *env); #define crypto_cipher_free(c) \ FREE_AND_NULL(crypto_cipher_t, crypto_cipher_free_, (c)) -/* public key crypto */ -MOCK_DECL(int, crypto_pk_public_checksig_digest,(crypto_pk_t *env, - const char *data, size_t datalen, - const char *sig, size_t siglen)); -int crypto_pk_private_sign_digest(crypto_pk_t *env, char *to, size_t tolen, - const char *from, size_t fromlen); -int crypto_pk_obsolete_public_hybrid_encrypt(crypto_pk_t *env, char *to, - size_t tolen, - const char *from, size_t fromlen, - int padding, int force); -int crypto_pk_obsolete_private_hybrid_decrypt(crypto_pk_t *env, char *to, - size_t tolen, - const char *from, size_t fromlen, - int padding, int warnOnFailure); -int crypto_pk_get_digest(const crypto_pk_t *pk, char *digest_out); -int crypto_pk_get_common_digests(crypto_pk_t *pk, - common_digests_t *digests_out); - /* symmetric crypto */ const char *crypto_cipher_get_key(crypto_cipher_t *env); @@ -148,52 +80,6 @@ int crypto_cipher_decrypt_with_iv(const char *key, char *to, size_t tolen, const char *from, size_t fromlen); -/* SHA-1 and other digests. */ -int crypto_digest(char *digest, const char *m, size_t len); -int crypto_digest256(char *digest, const char *m, size_t len, - digest_algorithm_t algorithm); -int crypto_digest512(char *digest, const char *m, size_t len, - digest_algorithm_t algorithm); -int crypto_common_digests(common_digests_t *ds_out, const char *m, size_t len); -struct smartlist_t; -void crypto_digest_smartlist_prefix(char *digest_out, size_t len_out, - const char *prepend, - const struct smartlist_t *lst, - const char *append, - digest_algorithm_t alg); -void crypto_digest_smartlist(char *digest_out, size_t len_out, - const struct smartlist_t *lst, const char *append, - digest_algorithm_t alg); -const char *crypto_digest_algorithm_get_name(digest_algorithm_t alg); -size_t crypto_digest_algorithm_get_length(digest_algorithm_t alg); -int crypto_digest_algorithm_parse_name(const char *name); -crypto_digest_t *crypto_digest_new(void); -crypto_digest_t *crypto_digest256_new(digest_algorithm_t algorithm); -crypto_digest_t *crypto_digest512_new(digest_algorithm_t algorithm); -void crypto_digest_free_(crypto_digest_t *digest); -#define crypto_digest_free(d) \ - FREE_AND_NULL(crypto_digest_t, crypto_digest_free_, (d)) -void crypto_digest_add_bytes(crypto_digest_t *digest, const char *data, - size_t len); -void crypto_digest_get_digest(crypto_digest_t *digest, - char *out, size_t out_len); -crypto_digest_t *crypto_digest_dup(const crypto_digest_t *digest); -void crypto_digest_assign(crypto_digest_t *into, - const crypto_digest_t *from); -void crypto_hmac_sha256(char *hmac_out, - const char *key, size_t key_len, - const char *msg, size_t msg_len); -void crypto_mac_sha3_256(uint8_t *mac_out, size_t len_out, - const uint8_t *key, size_t key_len, - const uint8_t *msg, size_t msg_len); - -crypto_xof_t *crypto_xof_new(void); -void crypto_xof_add_bytes(crypto_xof_t *xof, const uint8_t *data, size_t len); -void crypto_xof_squeeze_bytes(crypto_xof_t *xof, uint8_t *out, size_t len); -void crypto_xof_free_(crypto_xof_t *xof); -#define crypto_xof_free(xof) \ - FREE_AND_NULL(crypto_xof_t, crypto_xof_free_, (xof)) - /* Key negotiation */ #define DH_TYPE_CIRCUIT 1 #define DH_TYPE_REND 2 @@ -262,9 +148,5 @@ extern int break_strongest_rng_fallback; #endif #endif /* defined(CRYPTO_PRIVATE) */ -#ifdef TOR_UNIT_TESTS -digest_algorithm_t crypto_digest_get_algorithm(crypto_digest_t *digest); -#endif - #endif /* !defined(TOR_CRYPTO_H) */ diff --git a/src/common/crypto_curve25519.c b/src/common/crypto_curve25519.c index 8793fa6274..ccf12d00f9 100644 --- a/src/common/crypto_curve25519.c +++ b/src/common/crypto_curve25519.c @@ -24,6 +24,7 @@ #include "crypto.h" #include "crypto_curve25519.h" #include "crypto_format.h" +#include "crypto_digest.h" #include "util.h" #include "torlog.h" diff --git a/src/common/crypto_curve25519.h b/src/common/crypto_curve25519.h index 11f7423b07..4834fa0836 100644 --- a/src/common/crypto_curve25519.h +++ b/src/common/crypto_curve25519.h @@ -6,6 +6,7 @@ #include "testsupport.h" #include "torint.h" +#include "crypto_digest.h" #include "crypto_openssl_mgt.h" /** Length of a curve25519 public key when encoded. */ diff --git a/src/common/crypto_digest.c b/src/common/crypto_digest.c new file mode 100644 index 0000000000..cdcc1828c8 --- /dev/null +++ b/src/common/crypto_digest.c @@ -0,0 +1,569 @@ +/* Copyright (c) 2001, Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2017, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file crypto_digest.c + * \brief Block of functions related with digest and xof utilities and + * operations. + **/ + +#include "crypto_digest.h" + +#include "crypto.h" /* common functions */ +#include "crypto_openssl_mgt.h" + +DISABLE_GCC_WARNING(redundant-decls) + +#include <openssl/hmac.h> +#include <openssl/sha.h> + +ENABLE_GCC_WARNING(redundant-decls) + +#include "container.h" + +/* Crypto digest functions */ + +/** Compute the SHA1 digest of the <b>len</b> bytes on data stored in + * <b>m</b>. Write the DIGEST_LEN byte result into <b>digest</b>. + * Return 0 on success, -1 on failure. + */ +int +crypto_digest(char *digest, const char *m, size_t len) +{ + tor_assert(m); + tor_assert(digest); + if (SHA1((const unsigned char*)m,len,(unsigned char*)digest) == NULL) + return -1; + return 0; +} + +/** Compute a 256-bit digest of <b>len</b> bytes in data stored in <b>m</b>, + * using the algorithm <b>algorithm</b>. Write the DIGEST_LEN256-byte result + * into <b>digest</b>. Return 0 on success, -1 on failure. */ +int +crypto_digest256(char *digest, const char *m, size_t len, + digest_algorithm_t algorithm) +{ + tor_assert(m); + tor_assert(digest); + tor_assert(algorithm == DIGEST_SHA256 || algorithm == DIGEST_SHA3_256); + + int ret = 0; + if (algorithm == DIGEST_SHA256) + ret = (SHA256((const uint8_t*)m,len,(uint8_t*)digest) != NULL); + else + ret = (sha3_256((uint8_t *)digest, DIGEST256_LEN,(const uint8_t *)m, len) + > -1); + + if (!ret) + return -1; + return 0; +} + +/** Compute a 512-bit digest of <b>len</b> bytes in data stored in <b>m</b>, + * using the algorithm <b>algorithm</b>. Write the DIGEST_LEN512-byte result + * into <b>digest</b>. Return 0 on success, -1 on failure. */ +int +crypto_digest512(char *digest, const char *m, size_t len, + digest_algorithm_t algorithm) +{ + tor_assert(m); + tor_assert(digest); + tor_assert(algorithm == DIGEST_SHA512 || algorithm == DIGEST_SHA3_512); + + int ret = 0; + if (algorithm == DIGEST_SHA512) + ret = (SHA512((const unsigned char*)m,len,(unsigned char*)digest) + != NULL); + else + ret = (sha3_512((uint8_t*)digest, DIGEST512_LEN, (const uint8_t*)m, len) + > -1); + + if (!ret) + return -1; + return 0; +} + +/** Set the common_digests_t in <b>ds_out</b> to contain every digest on the + * <b>len</b> bytes in <b>m</b> that we know how to compute. Return 0 on + * success, -1 on failure. */ +int +crypto_common_digests(common_digests_t *ds_out, const char *m, size_t len) +{ + tor_assert(ds_out); + memset(ds_out, 0, sizeof(*ds_out)); + if (crypto_digest(ds_out->d[DIGEST_SHA1], m, len) < 0) + return -1; + if (crypto_digest256(ds_out->d[DIGEST_SHA256], m, len, DIGEST_SHA256) < 0) + return -1; + + return 0; +} + +/** Return the name of an algorithm, as used in directory documents. */ +const char * +crypto_digest_algorithm_get_name(digest_algorithm_t alg) +{ + switch (alg) { + case DIGEST_SHA1: + return "sha1"; + case DIGEST_SHA256: + return "sha256"; + case DIGEST_SHA512: + return "sha512"; + case DIGEST_SHA3_256: + return "sha3-256"; + case DIGEST_SHA3_512: + return "sha3-512"; + // LCOV_EXCL_START + default: + tor_fragile_assert(); + return "??unknown_digest??"; + // LCOV_EXCL_STOP + } +} + +/** Given the name of a digest algorithm, return its integer value, or -1 if + * the name is not recognized. */ +int +crypto_digest_algorithm_parse_name(const char *name) +{ + if (!strcmp(name, "sha1")) + return DIGEST_SHA1; + else if (!strcmp(name, "sha256")) + return DIGEST_SHA256; + else if (!strcmp(name, "sha512")) + return DIGEST_SHA512; + else if (!strcmp(name, "sha3-256")) + return DIGEST_SHA3_256; + else if (!strcmp(name, "sha3-512")) + return DIGEST_SHA3_512; + else + return -1; +} + +/** Given an algorithm, return the digest length in bytes. */ +size_t +crypto_digest_algorithm_get_length(digest_algorithm_t alg) +{ + switch (alg) { + case DIGEST_SHA1: + return DIGEST_LEN; + case DIGEST_SHA256: + return DIGEST256_LEN; + case DIGEST_SHA512: + return DIGEST512_LEN; + case DIGEST_SHA3_256: + return DIGEST256_LEN; + case DIGEST_SHA3_512: + return DIGEST512_LEN; + default: + tor_assert(0); // LCOV_EXCL_LINE + return 0; /* Unreachable */ // LCOV_EXCL_LINE + } +} + +/** Intermediate information about the digest of a stream of data. */ +struct crypto_digest_t { + digest_algorithm_t algorithm; /**< Which algorithm is in use? */ + /** State for the digest we're using. Only one member of the + * union is usable, depending on the value of <b>algorithm</b>. Note also + * that space for other members might not even be allocated! + */ + union { + SHA_CTX sha1; /**< state for SHA1 */ + SHA256_CTX sha2; /**< state for SHA256 */ + SHA512_CTX sha512; /**< state for SHA512 */ + keccak_state sha3; /**< state for SHA3-[256,512] */ + } d; +}; + +#ifdef TOR_UNIT_TESTS + +digest_algorithm_t +crypto_digest_get_algorithm(crypto_digest_t *digest) +{ + tor_assert(digest); + + return digest->algorithm; +} + +#endif /* defined(TOR_UNIT_TESTS) */ + +/** + * Return the number of bytes we need to malloc in order to get a + * crypto_digest_t for <b>alg</b>, or the number of bytes we need to wipe + * when we free one. + */ +static size_t +crypto_digest_alloc_bytes(digest_algorithm_t alg) +{ + /* Helper: returns the number of bytes in the 'f' field of 'st' */ +#define STRUCT_FIELD_SIZE(st, f) (sizeof( ((st*)0)->f )) + /* Gives the length of crypto_digest_t through the end of the field 'd' */ +#define END_OF_FIELD(f) (offsetof(crypto_digest_t, f) + \ + STRUCT_FIELD_SIZE(crypto_digest_t, f)) + switch (alg) { + case DIGEST_SHA1: + return END_OF_FIELD(d.sha1); + case DIGEST_SHA256: + return END_OF_FIELD(d.sha2); + case DIGEST_SHA512: + return END_OF_FIELD(d.sha512); + case DIGEST_SHA3_256: + case DIGEST_SHA3_512: + return END_OF_FIELD(d.sha3); + default: + tor_assert(0); // LCOV_EXCL_LINE + return 0; // LCOV_EXCL_LINE + } +#undef END_OF_FIELD +#undef STRUCT_FIELD_SIZE +} + +/** + * Internal function: create and return a new digest object for 'algorithm'. + * Does not typecheck the algorithm. + */ +static crypto_digest_t * +crypto_digest_new_internal(digest_algorithm_t algorithm) +{ + crypto_digest_t *r = tor_malloc(crypto_digest_alloc_bytes(algorithm)); + r->algorithm = algorithm; + + switch (algorithm) + { + case DIGEST_SHA1: + SHA1_Init(&r->d.sha1); + break; + case DIGEST_SHA256: + SHA256_Init(&r->d.sha2); + break; + case DIGEST_SHA512: + SHA512_Init(&r->d.sha512); + break; + case DIGEST_SHA3_256: + keccak_digest_init(&r->d.sha3, 256); + break; + case DIGEST_SHA3_512: + keccak_digest_init(&r->d.sha3, 512); + break; + default: + tor_assert_unreached(); + } + + return r; +} + +/** Allocate and return a new digest object to compute SHA1 digests. + */ +crypto_digest_t * +crypto_digest_new(void) +{ + return crypto_digest_new_internal(DIGEST_SHA1); +} + +/** Allocate and return a new digest object to compute 256-bit digests + * using <b>algorithm</b>. */ +crypto_digest_t * +crypto_digest256_new(digest_algorithm_t algorithm) +{ + tor_assert(algorithm == DIGEST_SHA256 || algorithm == DIGEST_SHA3_256); + return crypto_digest_new_internal(algorithm); +} + +/** Allocate and return a new digest object to compute 512-bit digests + * using <b>algorithm</b>. */ +crypto_digest_t * +crypto_digest512_new(digest_algorithm_t algorithm) +{ + tor_assert(algorithm == DIGEST_SHA512 || algorithm == DIGEST_SHA3_512); + return crypto_digest_new_internal(algorithm); +} + +/** Deallocate a digest object. + */ +void +crypto_digest_free_(crypto_digest_t *digest) +{ + if (!digest) + return; + size_t bytes = crypto_digest_alloc_bytes(digest->algorithm); + memwipe(digest, 0, bytes); + tor_free(digest); +} + +/** Add <b>len</b> bytes from <b>data</b> to the digest object. + */ +void +crypto_digest_add_bytes(crypto_digest_t *digest, const char *data, + size_t len) +{ + tor_assert(digest); + tor_assert(data); + /* Using the SHA*_*() calls directly means we don't support doing + * SHA in hardware. But so far the delay of getting the question + * to the hardware, and hearing the answer, is likely higher than + * just doing it ourselves. Hashes are fast. + */ + switch (digest->algorithm) { + case DIGEST_SHA1: + SHA1_Update(&digest->d.sha1, (void*)data, len); + break; + case DIGEST_SHA256: + SHA256_Update(&digest->d.sha2, (void*)data, len); + break; + case DIGEST_SHA512: + SHA512_Update(&digest->d.sha512, (void*)data, len); + break; + case DIGEST_SHA3_256: /* FALLSTHROUGH */ + case DIGEST_SHA3_512: + keccak_digest_update(&digest->d.sha3, (const uint8_t *)data, len); + break; + default: + /* LCOV_EXCL_START */ + tor_fragile_assert(); + break; + /* LCOV_EXCL_STOP */ + } +} + +/** Compute the hash of the data that has been passed to the digest + * object; write the first out_len bytes of the result to <b>out</b>. + * <b>out_len</b> must be \<= DIGEST512_LEN. + */ +void +crypto_digest_get_digest(crypto_digest_t *digest, + char *out, size_t out_len) +{ + unsigned char r[DIGEST512_LEN]; + crypto_digest_t tmpenv; + tor_assert(digest); + tor_assert(out); + tor_assert(out_len <= crypto_digest_algorithm_get_length(digest->algorithm)); + + /* The SHA-3 code handles copying into a temporary ctx, and also can handle + * short output buffers by truncating appropriately. */ + if (digest->algorithm == DIGEST_SHA3_256 || + digest->algorithm == DIGEST_SHA3_512) { + keccak_digest_sum(&digest->d.sha3, (uint8_t *)out, out_len); + return; + } + + const size_t alloc_bytes = crypto_digest_alloc_bytes(digest->algorithm); + /* memcpy into a temporary ctx, since SHA*_Final clears the context */ + memcpy(&tmpenv, digest, alloc_bytes); + switch (digest->algorithm) { + case DIGEST_SHA1: + SHA1_Final(r, &tmpenv.d.sha1); + break; + case DIGEST_SHA256: + SHA256_Final(r, &tmpenv.d.sha2); + break; + case DIGEST_SHA512: + SHA512_Final(r, &tmpenv.d.sha512); + break; +//LCOV_EXCL_START + case DIGEST_SHA3_256: /* FALLSTHROUGH */ + case DIGEST_SHA3_512: + default: + log_warn(LD_BUG, "Handling unexpected algorithm %d", digest->algorithm); + /* This is fatal, because it should never happen. */ + tor_assert_unreached(); + break; +//LCOV_EXCL_STOP + } + memcpy(out, r, out_len); + memwipe(r, 0, sizeof(r)); +} + +/** Allocate and return a new digest object with the same state as + * <b>digest</b> + */ +crypto_digest_t * +crypto_digest_dup(const crypto_digest_t *digest) +{ + tor_assert(digest); + const size_t alloc_bytes = crypto_digest_alloc_bytes(digest->algorithm); + return tor_memdup(digest, alloc_bytes); +} + +/** Temporarily save the state of <b>digest</b> in <b>checkpoint</b>. + * Asserts that <b>digest</b> is a SHA1 digest object. + */ +void +crypto_digest_checkpoint(crypto_digest_checkpoint_t *checkpoint, + const crypto_digest_t *digest) +{ + const size_t bytes = crypto_digest_alloc_bytes(digest->algorithm); + tor_assert(bytes <= sizeof(checkpoint->mem)); + memcpy(checkpoint->mem, digest, bytes); +} + +/** Restore the state of <b>digest</b> from <b>checkpoint</b>. + * Asserts that <b>digest</b> is a SHA1 digest object. Requires that the + * state was previously stored with crypto_digest_checkpoint() */ +void +crypto_digest_restore(crypto_digest_t *digest, + const crypto_digest_checkpoint_t *checkpoint) +{ + const size_t bytes = crypto_digest_alloc_bytes(digest->algorithm); + memcpy(digest, checkpoint->mem, bytes); +} + +/** Replace the state of the digest object <b>into</b> with the state + * of the digest object <b>from</b>. Requires that 'into' and 'from' + * have the same digest type. + */ +void +crypto_digest_assign(crypto_digest_t *into, + const crypto_digest_t *from) +{ + tor_assert(into); + tor_assert(from); + tor_assert(into->algorithm == from->algorithm); + const size_t alloc_bytes = crypto_digest_alloc_bytes(from->algorithm); + memcpy(into,from,alloc_bytes); +} + +/** Given a list of strings in <b>lst</b>, set the <b>len_out</b>-byte digest + * at <b>digest_out</b> to the hash of the concatenation of those strings, + * plus the optional string <b>append</b>, computed with the algorithm + * <b>alg</b>. + * <b>out_len</b> must be \<= DIGEST512_LEN. */ +void +crypto_digest_smartlist(char *digest_out, size_t len_out, + const smartlist_t *lst, + const char *append, + digest_algorithm_t alg) +{ + crypto_digest_smartlist_prefix(digest_out, len_out, NULL, lst, append, alg); +} + +/** Given a list of strings in <b>lst</b>, set the <b>len_out</b>-byte digest + * at <b>digest_out</b> to the hash of the concatenation of: the + * optional string <b>prepend</b>, those strings, + * and the optional string <b>append</b>, computed with the algorithm + * <b>alg</b>. + * <b>len_out</b> must be \<= DIGEST512_LEN. */ +void +crypto_digest_smartlist_prefix(char *digest_out, size_t len_out, + const char *prepend, + const smartlist_t *lst, + const char *append, + digest_algorithm_t alg) +{ + crypto_digest_t *d = crypto_digest_new_internal(alg); + if (prepend) + crypto_digest_add_bytes(d, prepend, strlen(prepend)); + SMARTLIST_FOREACH(lst, const char *, cp, + crypto_digest_add_bytes(d, cp, strlen(cp))); + if (append) + crypto_digest_add_bytes(d, append, strlen(append)); + crypto_digest_get_digest(d, digest_out, len_out); + crypto_digest_free(d); +} + +/** Compute the HMAC-SHA-256 of the <b>msg_len</b> bytes in <b>msg</b>, using + * the <b>key</b> of length <b>key_len</b>. Store the DIGEST256_LEN-byte + * result in <b>hmac_out</b>. Asserts on failure. + */ +void +crypto_hmac_sha256(char *hmac_out, + const char *key, size_t key_len, + const char *msg, size_t msg_len) +{ + unsigned char *rv = NULL; + /* If we've got OpenSSL >=0.9.8 we can use its hmac implementation. */ + tor_assert(key_len < INT_MAX); + tor_assert(msg_len < INT_MAX); + tor_assert(hmac_out); + rv = HMAC(EVP_sha256(), key, (int)key_len, (unsigned char*)msg, (int)msg_len, + (unsigned char*)hmac_out, NULL); + tor_assert(rv); +} + +/** Compute a MAC using SHA3-256 of <b>msg_len</b> bytes in <b>msg</b> using a + * <b>key</b> of length <b>key_len</b> and a <b>salt</b> of length + * <b>salt_len</b>. Store the result of <b>len_out</b> bytes in in + * <b>mac_out</b>. This function can't fail. */ +void +crypto_mac_sha3_256(uint8_t *mac_out, size_t len_out, + const uint8_t *key, size_t key_len, + const uint8_t *msg, size_t msg_len) +{ + crypto_digest_t *digest; + + const uint64_t key_len_netorder = tor_htonll(key_len); + + tor_assert(mac_out); + tor_assert(key); + tor_assert(msg); + + digest = crypto_digest256_new(DIGEST_SHA3_256); + + /* Order matters here that is any subsystem using this function should + * expect this very precise ordering in the MAC construction. */ + crypto_digest_add_bytes(digest, (const char *) &key_len_netorder, + sizeof(key_len_netorder)); + crypto_digest_add_bytes(digest, (const char *) key, key_len); + crypto_digest_add_bytes(digest, (const char *) msg, msg_len); + crypto_digest_get_digest(digest, (char *) mac_out, len_out); + crypto_digest_free(digest); +} + +/* xof functions */ + +/** Internal state for a eXtendable-Output Function (XOF). */ +struct crypto_xof_t { + keccak_state s; +}; + +/** Allocate a new XOF object backed by SHAKE-256. The security level + * provided is a function of the length of the output used. Read and + * understand FIPS-202 A.2 "Additional Consideration for Extendable-Output + * Functions" before using this construct. + */ +crypto_xof_t * +crypto_xof_new(void) +{ + crypto_xof_t *xof; + xof = tor_malloc(sizeof(crypto_xof_t)); + keccak_xof_init(&xof->s, 256); + return xof; +} + +/** Absorb bytes into a XOF object. Must not be called after a call to + * crypto_xof_squeeze_bytes() for the same instance, and will assert + * if attempted. + */ +void +crypto_xof_add_bytes(crypto_xof_t *xof, const uint8_t *data, size_t len) +{ + int i = keccak_xof_absorb(&xof->s, data, len); + tor_assert(i == 0); +} + +/** Squeeze bytes out of a XOF object. Calling this routine will render + * the XOF instance ineligible to absorb further data. + */ +void +crypto_xof_squeeze_bytes(crypto_xof_t *xof, uint8_t *out, size_t len) +{ + int i = keccak_xof_squeeze(&xof->s, out, len); + tor_assert(i == 0); +} + +/** Cleanse and deallocate a XOF object. */ +void +crypto_xof_free_(crypto_xof_t *xof) +{ + if (!xof) + return; + memwipe(xof, 0, sizeof(crypto_xof_t)); + tor_free(xof); +} + diff --git a/src/common/crypto_digest.h b/src/common/crypto_digest.h new file mode 100644 index 0000000000..3bd74acdfa --- /dev/null +++ b/src/common/crypto_digest.h @@ -0,0 +1,136 @@ +/* Copyright (c) 2001, Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2017, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file crypto_digest.h + * + * \brief Headers for crypto_digest.c + **/ + +#ifndef TOR_CRYPTO_DIGEST_H +#define TOR_CRYPTO_DIGEST_H + +#include <stdio.h> + +#include "container.h" +#include "torint.h" + +/** Length of the output of our message digest. */ +#define DIGEST_LEN 20 +/** Length of the output of our second (improved) message digests. (For now + * this is just sha256, but it could be any other 256-bit digest.) */ +#define DIGEST256_LEN 32 +/** Length of the output of our 64-bit optimized message digests (SHA512). */ +#define DIGEST512_LEN 64 + +/** Length of a sha1 message digest when encoded in base32 with trailing = + * signs removed. */ +#define BASE32_DIGEST_LEN 32 +/** Length of a sha1 message digest when encoded in base64 with trailing = + * signs removed. */ +#define BASE64_DIGEST_LEN 27 +/** Length of a sha256 message digest when encoded in base64 with trailing = + * signs removed. */ +#define BASE64_DIGEST256_LEN 43 +/** Length of a sha512 message digest when encoded in base64 with trailing = + * signs removed. */ +#define BASE64_DIGEST512_LEN 86 + +/** Length of hex encoding of SHA1 digest, not including final NUL. */ +#define HEX_DIGEST_LEN 40 +/** Length of hex encoding of SHA256 digest, not including final NUL. */ +#define HEX_DIGEST256_LEN 64 +/** Length of hex encoding of SHA512 digest, not including final NUL. */ +#define HEX_DIGEST512_LEN 128 + +typedef enum { + DIGEST_SHA1 = 0, + DIGEST_SHA256 = 1, + DIGEST_SHA512 = 2, + DIGEST_SHA3_256 = 3, + DIGEST_SHA3_512 = 4, +} digest_algorithm_t; +#define N_DIGEST_ALGORITHMS (DIGEST_SHA3_512+1) +#define N_COMMON_DIGEST_ALGORITHMS (DIGEST_SHA256+1) + +#define DIGEST_CHECKPOINT_BYTES (SIZEOF_VOID_P + 512) +/** Structure used to temporarily save the a digest object. Only implemented + * for SHA1 digest for now. */ +typedef struct crypto_digest_checkpoint_t { + uint8_t mem[DIGEST_CHECKPOINT_BYTES]; +} crypto_digest_checkpoint_t; + +/** A set of all the digests we commonly compute, taken on a single + * string. Any digests that are shorter than 512 bits are right-padded + * with 0 bits. + * + * Note that this representation wastes 44 bytes for the SHA1 case, so + * don't use it for anything where we need to allocate a whole bunch at + * once. + **/ +typedef struct { + char d[N_COMMON_DIGEST_ALGORITHMS][DIGEST256_LEN]; +} common_digests_t; + +typedef struct crypto_digest_t crypto_digest_t; +typedef struct crypto_xof_t crypto_xof_t; + +/* SHA-1 and other digests */ +int crypto_digest(char *digest, const char *m, size_t len); +int crypto_digest256(char *digest, const char *m, size_t len, + digest_algorithm_t algorithm); +int crypto_digest512(char *digest, const char *m, size_t len, + digest_algorithm_t algorithm); +int crypto_common_digests(common_digests_t *ds_out, const char *m, size_t len); +void crypto_digest_smartlist_prefix(char *digest_out, size_t len_out, + const char *prepend, + const struct smartlist_t *lst, + const char *append, + digest_algorithm_t alg); +void crypto_digest_smartlist(char *digest_out, size_t len_out, + const struct smartlist_t *lst, const char *append, + digest_algorithm_t alg); +const char *crypto_digest_algorithm_get_name(digest_algorithm_t alg); +size_t crypto_digest_algorithm_get_length(digest_algorithm_t alg); +int crypto_digest_algorithm_parse_name(const char *name); +crypto_digest_t *crypto_digest_new(void); +crypto_digest_t *crypto_digest256_new(digest_algorithm_t algorithm); +crypto_digest_t *crypto_digest512_new(digest_algorithm_t algorithm); +void crypto_digest_free_(crypto_digest_t *digest); +#define crypto_digest_free(d) \ + FREE_AND_NULL(crypto_digest_t, crypto_digest_free_, (d)) +void crypto_digest_add_bytes(crypto_digest_t *digest, const char *data, + size_t len); +void crypto_digest_get_digest(crypto_digest_t *digest, + char *out, size_t out_len); +crypto_digest_t *crypto_digest_dup(const crypto_digest_t *digest); +void crypto_digest_checkpoint(crypto_digest_checkpoint_t *checkpoint, + const crypto_digest_t *digest); +void crypto_digest_restore(crypto_digest_t *digest, + const crypto_digest_checkpoint_t *checkpoint); +void crypto_digest_assign(crypto_digest_t *into, + const crypto_digest_t *from); +void crypto_hmac_sha256(char *hmac_out, + const char *key, size_t key_len, + const char *msg, size_t msg_len); +void crypto_mac_sha3_256(uint8_t *mac_out, size_t len_out, + const uint8_t *key, size_t key_len, + const uint8_t *msg, size_t msg_len); + +/* xof functions*/ +crypto_xof_t *crypto_xof_new(void); +void crypto_xof_add_bytes(crypto_xof_t *xof, const uint8_t *data, size_t len); +void crypto_xof_squeeze_bytes(crypto_xof_t *xof, uint8_t *out, size_t len); +void crypto_xof_free_(crypto_xof_t *xof); +#define crypto_xof_free(xof) \ + FREE_AND_NULL(crypto_xof_t, crypto_xof_free_, (xof)) + +#ifdef TOR_UNIT_TESTS +digest_algorithm_t crypto_digest_get_algorithm(crypto_digest_t *digest); +#endif + +#endif /* !defined(TOR_CRYPTO_DIGEST_H) */ + diff --git a/src/common/crypto_ed25519.c b/src/common/crypto_ed25519.c index b962a59de1..f1cc0cb188 100644 --- a/src/common/crypto_ed25519.c +++ b/src/common/crypto_ed25519.c @@ -23,6 +23,7 @@ #include "crypto.h" +#include "crypto_digest.h" #include "crypto_curve25519.h" #include "crypto_ed25519.h" #include "crypto_format.h" diff --git a/src/common/crypto_format.c b/src/common/crypto_format.c index 1d090a8770..3f6fb9f54c 100644 --- a/src/common/crypto_format.c +++ b/src/common/crypto_format.c @@ -19,6 +19,7 @@ #include "crypto_curve25519.h" #include "crypto_ed25519.h" #include "crypto_format.h" +#include "crypto_digest.h" #include "util.h" #include "util_format.h" #include "torlog.h" diff --git a/src/common/crypto_pwbox.c b/src/common/crypto_pwbox.c index 12acc9331c..604fc68e97 100644 --- a/src/common/crypto_pwbox.c +++ b/src/common/crypto_pwbox.c @@ -11,6 +11,7 @@ #include "crypto.h" #include "crypto_s2k.h" #include "crypto_pwbox.h" +#include "crypto_digest.h" #include "di_ops.h" #include "util.h" #include "pwbox.h" diff --git a/src/common/crypto_rsa.c b/src/common/crypto_rsa.c index fa572580a4..986ccb0ee2 100644 --- a/src/common/crypto_rsa.c +++ b/src/common/crypto_rsa.c @@ -13,8 +13,8 @@ #include "crypto.h" #include "compat_openssl.h" #include "crypto_curve25519.h" -#include "crypto_ed25519.h" #include "crypto_format.h" +#include "crypto_digest.h" DISABLE_GCC_WARNING(redundant-decls) @@ -627,6 +627,148 @@ crypto_pk_copy_full(crypto_pk_t *env) return crypto_new_pk_from_rsa_(new_key); } +/** Perform a hybrid (public/secret) encryption on <b>fromlen</b> + * bytes of data from <b>from</b>, with padding type 'padding', + * storing the results on <b>to</b>. + * + * Returns the number of bytes written on success, -1 on failure. + * + * The encrypted data consists of: + * - The source data, padded and encrypted with the public key, if the + * padded source data is no longer than the public key, and <b>force</b> + * is false, OR + * - The beginning of the source data prefixed with a 16-byte symmetric key, + * padded and encrypted with the public key; followed by the rest of + * the source data encrypted in AES-CTR mode with the symmetric key. + * + * NOTE that this format does not authenticate the symmetrically encrypted + * part of the data, and SHOULD NOT BE USED for new protocols. + */ +int +crypto_pk_obsolete_public_hybrid_encrypt(crypto_pk_t *env, + char *to, size_t tolen, + const char *from, + size_t fromlen, + int padding, int force) +{ + int overhead, outlen, r; + size_t pkeylen, symlen; + crypto_cipher_t *cipher = NULL; + char *buf = NULL; + + tor_assert(env); + tor_assert(from); + tor_assert(to); + tor_assert(fromlen < SIZE_T_CEILING); + + overhead = crypto_get_rsa_padding_overhead(crypto_get_rsa_padding(padding)); + pkeylen = crypto_pk_keysize(env); + + if (!force && fromlen+overhead <= pkeylen) { + /* It all fits in a single encrypt. */ + return crypto_pk_public_encrypt(env,to, + tolen, + from,fromlen,padding); + } + tor_assert(tolen >= fromlen + overhead + CIPHER_KEY_LEN); + tor_assert(tolen >= pkeylen); + + char key[CIPHER_KEY_LEN]; + crypto_rand(key, sizeof(key)); /* generate a new key. */ + cipher = crypto_cipher_new(key); + + buf = tor_malloc(pkeylen+1); + memcpy(buf, key, CIPHER_KEY_LEN); + memcpy(buf+CIPHER_KEY_LEN, from, pkeylen-overhead-CIPHER_KEY_LEN); + + /* Length of symmetrically encrypted data. */ + symlen = fromlen-(pkeylen-overhead-CIPHER_KEY_LEN); + + outlen = crypto_pk_public_encrypt(env,to,tolen,buf,pkeylen-overhead,padding); + if (outlen!=(int)pkeylen) { + goto err; + } + r = crypto_cipher_encrypt(cipher, to+outlen, + from+pkeylen-overhead-CIPHER_KEY_LEN, symlen); + + if (r<0) goto err; + memwipe(buf, 0, pkeylen); + memwipe(key, 0, sizeof(key)); + tor_free(buf); + crypto_cipher_free(cipher); + tor_assert(outlen+symlen < INT_MAX); + return (int)(outlen + symlen); + err: + + memwipe(buf, 0, pkeylen); + memwipe(key, 0, sizeof(key)); + tor_free(buf); + crypto_cipher_free(cipher); + return -1; +} + +/** Invert crypto_pk_obsolete_public_hybrid_encrypt. Returns the number of + * bytes written on success, -1 on failure. + * + * NOTE that this format does not authenticate the symmetrically encrypted + * part of the data, and SHOULD NOT BE USED for new protocols. + */ +int +crypto_pk_obsolete_private_hybrid_decrypt(crypto_pk_t *env, + char *to, + size_t tolen, + const char *from, + size_t fromlen, + int padding, int warnOnFailure) +{ + int outlen, r; + size_t pkeylen; + crypto_cipher_t *cipher = NULL; + char *buf = NULL; + + tor_assert(fromlen < SIZE_T_CEILING); + pkeylen = crypto_pk_keysize(env); + + if (fromlen <= pkeylen) { + return crypto_pk_private_decrypt(env,to,tolen,from,fromlen,padding, + warnOnFailure); + } + + buf = tor_malloc(pkeylen); + outlen = crypto_pk_private_decrypt(env,buf,pkeylen,from,pkeylen,padding, + warnOnFailure); + if (outlen<0) { + log_fn(warnOnFailure?LOG_WARN:LOG_DEBUG, LD_CRYPTO, + "Error decrypting public-key data"); + goto err; + } + if (outlen < CIPHER_KEY_LEN) { + log_fn(warnOnFailure?LOG_WARN:LOG_INFO, LD_CRYPTO, + "No room for a symmetric key"); + goto err; + } + cipher = crypto_cipher_new(buf); + if (!cipher) { + goto err; + } + memcpy(to,buf+CIPHER_KEY_LEN,outlen-CIPHER_KEY_LEN); + outlen -= CIPHER_KEY_LEN; + tor_assert(tolen - outlen >= fromlen - pkeylen); + r = crypto_cipher_decrypt(cipher, to+outlen, from+pkeylen, fromlen-pkeylen); + if (r<0) + goto err; + memwipe(buf,0,pkeylen); + tor_free(buf); + crypto_cipher_free(cipher); + tor_assert(outlen + fromlen < INT_MAX); + return (int)(outlen + (fromlen-pkeylen)); + err: + memwipe(buf,0,pkeylen); + tor_free(buf); + crypto_cipher_free(cipher); + return -1; +} + /** Encrypt <b>fromlen</b> bytes from <b>from</b> with the public key * in <b>env</b>, using the padding method <b>padding</b>. On success, * write the result to <b>to</b>, and return the number of bytes @@ -849,6 +991,122 @@ crypto_pk_get_hashed_fingerprint(crypto_pk_t *pk, char *fp_out) return 0; } +/** Check a siglen-byte long signature at <b>sig</b> against + * <b>datalen</b> bytes of data at <b>data</b>, using the public key + * in <b>env</b>. Return 0 if <b>sig</b> is a correct signature for + * SHA1(data). Else return -1. + */ +MOCK_IMPL(int, +crypto_pk_public_checksig_digest,(crypto_pk_t *env, const char *data, + size_t datalen, const char *sig, + size_t siglen)) +{ + char digest[DIGEST_LEN]; + char *buf; + size_t buflen; + int r; + + tor_assert(env); + tor_assert(data); + tor_assert(sig); + tor_assert(datalen < SIZE_T_CEILING); + tor_assert(siglen < SIZE_T_CEILING); + + if (crypto_digest(digest,data,datalen)<0) { + log_warn(LD_BUG, "couldn't compute digest"); + return -1; + } + buflen = crypto_pk_keysize(env); + buf = tor_malloc(buflen); + r = crypto_pk_public_checksig(env,buf,buflen,sig,siglen); + if (r != DIGEST_LEN) { + log_warn(LD_CRYPTO, "Invalid signature"); + tor_free(buf); + return -1; + } + if (tor_memneq(buf, digest, DIGEST_LEN)) { + log_warn(LD_CRYPTO, "Signature mismatched with digest."); + tor_free(buf); + return -1; + } + tor_free(buf); + + return 0; +} + +/** Compute a SHA1 digest of <b>fromlen</b> bytes of data stored at + * <b>from</b>; sign the data with the private key in <b>env</b>, and + * store it in <b>to</b>. Return the number of bytes written on + * success, and -1 on failure. + * + * <b>tolen</b> is the number of writable bytes in <b>to</b>, and must be + * at least the length of the modulus of <b>env</b>. + */ +int +crypto_pk_private_sign_digest(crypto_pk_t *env, char *to, size_t tolen, + const char *from, size_t fromlen) +{ + int r; + char digest[DIGEST_LEN]; + if (crypto_digest(digest,from,fromlen)<0) + return -1; + r = crypto_pk_private_sign(env,to,tolen,digest,DIGEST_LEN); + memwipe(digest, 0, sizeof(digest)); + return r; +} + +/** Given a private or public key <b>pk</b>, put a SHA1 hash of the + * public key into <b>digest_out</b> (must have DIGEST_LEN bytes of space). + * Return 0 on success, -1 on failure. + */ +int +crypto_pk_get_digest(const crypto_pk_t *pk, char *digest_out) +{ + char *buf; + size_t buflen; + int len; + int rv = -1; + + buflen = crypto_pk_keysize(pk)*2; + buf = tor_malloc(buflen); + len = crypto_pk_asn1_encode(pk, buf, buflen); + if (len < 0) + goto done; + + if (crypto_digest(digest_out, buf, len) < 0) + goto done; + + rv = 0; + done: + tor_free(buf); + return rv; +} + +/** Compute all digests of the DER encoding of <b>pk</b>, and store them + * in <b>digests_out</b>. Return 0 on success, -1 on failure. */ +int +crypto_pk_get_common_digests(crypto_pk_t *pk, common_digests_t *digests_out) +{ + char *buf; + size_t buflen; + int len; + int rv = -1; + + buflen = crypto_pk_keysize(pk)*2; + buf = tor_malloc(buflen); + len = crypto_pk_asn1_encode(pk, buf, buflen); + if (len < 0) + goto done; + + if (crypto_common_digests(digests_out, (char*)buf, len) < 0) + goto done; + + rv = 0; + done: + tor_free(buf); + return rv; +} + /** Given a crypto_pk_t <b>pk</b>, allocate a new buffer containing the * Base64 encoding of the DER representation of the private key as a NUL * terminated string, and return it via <b>priv_out</b>. Return 0 on diff --git a/src/common/crypto_rsa.h b/src/common/crypto_rsa.h index 5b9025c629..2f5442a5d2 100644 --- a/src/common/crypto_rsa.h +++ b/src/common/crypto_rsa.h @@ -15,13 +15,13 @@ #include "orconfig.h" +#include "crypto_digest.h" #include <stdio.h> #include "torint.h" #include "testsupport.h" #include "compat.h" #include "util.h" #include "torlog.h" -#include "crypto_curve25519.h" /** Length of our public keys. */ #define PK_BYTES (1024/8) @@ -69,6 +69,14 @@ crypto_pk_t *crypto_pk_dup_key(crypto_pk_t *orig); crypto_pk_t *crypto_pk_copy_full(crypto_pk_t *orig); int crypto_pk_key_is_private(const crypto_pk_t *key); int crypto_pk_public_exponent_ok(crypto_pk_t *env); +int crypto_pk_obsolete_public_hybrid_encrypt(crypto_pk_t *env, char *to, + size_t tolen, + const char *from, size_t fromlen, + int padding, int force); +int crypto_pk_obsolete_private_hybrid_decrypt(crypto_pk_t *env, char *to, + size_t tolen, + const char *from, size_t fromlen, + int padding, int warnOnFailure); int crypto_pk_public_encrypt(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen, int padding); int crypto_pk_private_decrypt(crypto_pk_t *env, char *to, size_t tolen, @@ -84,6 +92,13 @@ crypto_pk_t *crypto_pk_asn1_decode(const char *str, size_t len); int crypto_pk_get_fingerprint(crypto_pk_t *pk, char *fp_out,int add_space); int crypto_pk_get_hashed_fingerprint(crypto_pk_t *pk, char *fp_out); +MOCK_DECL(int, crypto_pk_public_checksig_digest,(crypto_pk_t *env, + const char *data, size_t datalen, const char *sig, size_t siglen)); +int crypto_pk_private_sign_digest(crypto_pk_t *env, char *to, size_t tolen, + const char *from, size_t fromlen); +int crypto_pk_get_digest(const crypto_pk_t *pk, char *digest_out); +int crypto_pk_get_common_digests(crypto_pk_t *pk, + common_digests_t *digests_out); int crypto_pk_base64_encode(const crypto_pk_t *pk, char **priv_out); crypto_pk_t *crypto_pk_base64_decode(const char *str, size_t len); diff --git a/src/common/crypto_s2k.c b/src/common/crypto_s2k.c index b2fcca54c4..316445e40f 100644 --- a/src/common/crypto_s2k.c +++ b/src/common/crypto_s2k.c @@ -16,6 +16,7 @@ #include "util.h" #include "compat.h" #include "crypto_s2k.h" +#include "crypto_digest.h" #include <openssl/evp.h> diff --git a/src/common/include.am b/src/common/include.am index 6945285108..73c51ff0b2 100644 --- a/src/common/include.am +++ b/src/common/include.am @@ -114,6 +114,7 @@ LIBOR_CRYPTO_A_SRC = \ src/common/compress_zlib.c \ src/common/compress_zstd.c \ src/common/crypto.c \ + src/common/crypto_digest.c \ src/common/crypto_rsa.c \ src/common/crypto_openssl_mgt.c \ src/common/crypto_pwbox.c \ @@ -165,6 +166,7 @@ COMMONHEADERS = \ src/common/confline.h \ src/common/container.h \ src/common/crypto.h \ + src/common/crypto_digest.h \ src/common/crypto_curve25519.h \ src/common/crypto_ed25519.h \ src/common/crypto_format.h \ diff --git a/src/common/log.c b/src/common/log.c index 9f4a8b2bc2..922e9dd38f 100644 --- a/src/common/log.c +++ b/src/common/log.c @@ -52,6 +52,13 @@ #define raw_assert(x) assert(x) // assert OK +/** Defining compile-time constants for Tor log levels (used by the Rust + * log wrapper at src/rust/tor_log) */ +const int LOG_WARN_ = LOG_WARN; +const int LOG_NOTICE_ = LOG_NOTICE; +const log_domain_mask_t LD_GENERAL_ = LD_GENERAL; +const log_domain_mask_t LD_NET_ = LD_NET; + /** Information for a single logfile; only used in log.c */ typedef struct logfile_t { struct logfile_t *next; /**< Next logfile_t in the linked list. */ @@ -225,6 +232,30 @@ log_set_application_name(const char *name) appname = name ? tor_strdup(name) : NULL; } +/** Return true if some of the running logs might be interested in a log + * message of the given severity in the given domains. If this function + * returns true, the log message might be ignored anyway, but if it returns + * false, it is definitely_ safe not to log the message. */ +int +log_message_is_interesting(int severity, log_domain_mask_t domain) +{ + (void) domain; + return (severity <= log_global_min_severity_); +} + +/** + * As tor_log, but takes an optional function name, and does not treat its + * <b>string</b> as a printf format. + * + * For use by Rust integration. + */ +void +tor_log_string(int severity, log_domain_mask_t domain, + const char *function, const char *string) +{ + log_fn_(severity, domain, function, "%s", string); +} + /** Log time granularity in milliseconds. */ static int log_time_granularity = 1; diff --git a/src/common/procmon.c b/src/common/procmon.c index abcbbeaa21..73c14cd584 100644 --- a/src/common/procmon.c +++ b/src/common/procmon.c @@ -10,8 +10,6 @@ #include "util.h" -#include <event2/event.h> - #ifdef HAVE_SIGNAL_H #include <signal.h> #endif @@ -44,7 +42,7 @@ typedef int pid_t; /* Currently we need to poll in some way on all systems. */ #ifdef PROCMON_POLLS -static void tor_process_monitor_poll_cb(evutil_socket_t unused1, short unused2, +static void tor_process_monitor_poll_cb(periodic_timer_t *ev, void *procmon_); #endif @@ -136,7 +134,7 @@ struct tor_process_monitor_t { /** A Libevent event structure, to either poll for the process's * existence or receive a notification when the process ends. */ - struct event *e; + periodic_timer_t *e; /** A callback to be called when the process ends. */ tor_procmon_callback_t cb; @@ -159,9 +157,6 @@ tor_validate_process_specifier(const char *process_spec, return parse_process_specifier(process_spec, &ppspec, msg); } -/* XXXX we should use periodic_timer_new() for this stuff */ -#define PERIODIC_TIMER_FLAGS EV_PERSIST - /* DOCDOC poll_interval_tv */ static const struct timeval poll_interval_tv = {15, 0}; @@ -225,13 +220,9 @@ tor_process_monitor_new(struct event_base *base, procmon->cb_arg = cb_arg; #ifdef PROCMON_POLLS - procmon->e = tor_event_new(base, -1 /* no FD */, PERIODIC_TIMER_FLAGS, - tor_process_monitor_poll_cb, procmon); - /* Note: If you port this file to plain Libevent 2, check that - * procmon->e is non-NULL. We don't need to here because - * tor_evtimer_new never returns NULL. */ - - evtimer_add(procmon->e, &poll_interval_tv); + procmon->e = periodic_timer_new(base, + &poll_interval_tv, + tor_process_monitor_poll_cb, procmon); #else /* !(defined(PROCMON_POLLS)) */ #error OOPS? #endif /* defined(PROCMON_POLLS) */ @@ -246,14 +237,12 @@ tor_process_monitor_new(struct event_base *base, /** Libevent callback to poll for the existence of the process * monitored by <b>procmon_</b>. */ static void -tor_process_monitor_poll_cb(evutil_socket_t unused1, short unused2, - void *procmon_) +tor_process_monitor_poll_cb(periodic_timer_t *event, void *procmon_) { + (void)event; tor_process_monitor_t *procmon = (tor_process_monitor_t *)(procmon_); int its_dead_jim; - (void)unused1; (void)unused2; - tor_assert(procmon != NULL); #ifdef _WIN32 @@ -336,7 +325,7 @@ tor_process_monitor_free_(tor_process_monitor_t *procmon) #endif if (procmon->e != NULL) - tor_event_free(procmon->e); + periodic_timer_free(procmon->e); tor_free(procmon); } diff --git a/src/common/timers.c b/src/common/timers.c index 552080b11e..a90817da1c 100644 --- a/src/common/timers.c +++ b/src/common/timers.c @@ -37,8 +37,6 @@ #include "torlog.h" #include "util.h" -#include <event2/event.h> - struct timeout_cb { timer_cb_fn_t cb; void *arg; @@ -69,7 +67,7 @@ struct timeout_cb { #include "src/ext/timeouts/timeout.c" static struct timeouts *global_timeouts = NULL; -static struct event *global_timer_event = NULL; +static struct mainloop_event_t *global_timer_event = NULL; static monotime_t start_of_time; @@ -147,7 +145,7 @@ libevent_timer_reschedule(void) if (delay > MIN_CHECK_TICKS) delay = MIN_CHECK_TICKS; timeout_to_tv(delay, &d); - event_add(global_timer_event, &d); + mainloop_event_schedule(global_timer_event, &d); } /** Run the callback of every timer that has expired, based on the current @@ -170,10 +168,9 @@ timers_run_pending(void) * have fired, activate their callbacks, and reschedule the libevent timer. */ static void -libevent_timer_callback(evutil_socket_t fd, short what, void *arg) +libevent_timer_callback(mainloop_event_t *ev, void *arg) { - (void)fd; - (void)what; + (void)ev; (void)arg; timers_run_pending(); @@ -203,9 +200,8 @@ timers_initialize(void) monotime_init(); monotime_get(&start_of_time); - struct event *timer_event; - timer_event = tor_event_new(tor_libevent_get_base(), - -1, 0, libevent_timer_callback, NULL); + mainloop_event_t *timer_event; + timer_event = mainloop_event_new(libevent_timer_callback, NULL); tor_assert(timer_event); global_timer_event = timer_event; @@ -219,7 +215,7 @@ void timers_shutdown(void) { if (global_timer_event) { - tor_event_free(global_timer_event); + mainloop_event_free(global_timer_event); global_timer_event = NULL; } if (global_timeouts) { diff --git a/src/common/torlog.h b/src/common/torlog.h index cadfe3b879..ac632ff521 100644 --- a/src/common/torlog.h +++ b/src/common/torlog.h @@ -191,6 +191,10 @@ void log_fn_ratelim_(struct ratelim_t *ratelim, int severity, const char *format, ...) CHECK_PRINTF(5,6); +int log_message_is_interesting(int severity, log_domain_mask_t domain); +void tor_log_string(int severity, log_domain_mask_t domain, + const char *function, const char *string); + #if defined(__GNUC__) && __GNUC__ <= 3 /* These are the GCC varidaic macros, so that older versions of GCC don't @@ -248,6 +252,16 @@ void log_fn_ratelim_(struct ratelim_t *ratelim, int severity, args, ##__VA_ARGS__) #endif /* defined(__GNUC__) && __GNUC__ <= 3 */ +/** This defines log levels that are linked in the Rust log module, rather + * than re-defining these in both Rust and C. + * + * C_RUST_COUPLED src/rust/tor_log LogSeverity, LogDomain + */ +extern const int LOG_WARN_; +extern const int LOG_NOTICE_; +extern const log_domain_mask_t LD_NET_; +extern const log_domain_mask_t LD_GENERAL_; + #ifdef LOG_PRIVATE MOCK_DECL(STATIC void, logv, (int severity, log_domain_mask_t domain, const char *funcname, const char *suffix, const char *format, diff --git a/src/common/tortls.c b/src/common/tortls.c index 50609b8ac7..05e29e22ff 100644 --- a/src/common/tortls.c +++ b/src/common/tortls.c @@ -25,6 +25,7 @@ #include <ws2tcpip.h> #endif +#include "crypto.h" #include "compat.h" /* Some versions of OpenSSL declare SSL_get_selected_srtp_profile twice in @@ -32,7 +33,6 @@ DISABLE_GCC_WARNING(redundant-decls) #include <openssl/opensslv.h> -#include "crypto.h" #ifdef OPENSSL_NO_EC #error "We require OpenSSL with ECC support" diff --git a/src/common/tortls.h b/src/common/tortls.h index 1dbf0b332f..7c867bfff2 100644 --- a/src/common/tortls.h +++ b/src/common/tortls.h @@ -11,7 +11,7 @@ * \brief Headers for tortls.c **/ -#include "crypto.h" +#include "crypto_rsa.h" #include "compat_openssl.h" #include "compat.h" #include "testsupport.h" diff --git a/src/common/util.c b/src/common/util.c index a68fd30d09..065b3245fe 100644 --- a/src/common/util.c +++ b/src/common/util.c @@ -16,7 +16,7 @@ #define UTIL_PRIVATE #include "util.h" #include "torlog.h" -#include "crypto.h" +#include "crypto_digest.h" #include "torint.h" #include "container.h" #include "address.h" @@ -5111,30 +5111,6 @@ stream_status_to_string(enum stream_status stream_status) } } -/* DOCDOC */ -static void -log_portfw_spawn_error_message(const char *buf, - const char *executable, int *child_status) -{ - /* Parse error message */ - int retval, child_state, saved_errno; - retval = tor_sscanf(buf, SPAWN_ERROR_MESSAGE "%x/%x", - &child_state, &saved_errno); - if (retval == 2) { - log_warn(LD_GENERAL, - "Failed to start child process \"%s\" in state %d: %s", - executable, child_state, strerror(saved_errno)); - if (child_status) - *child_status = 1; - } else { - /* Failed to parse message from child process, log it as a - warning */ - log_warn(LD_GENERAL, - "Unexpected message from port forwarding helper \"%s\": %s", - executable, buf); - } -} - #ifdef _WIN32 /** Return a smartlist containing lines outputted from @@ -5254,42 +5230,6 @@ tor_get_lines_from_handle, (int fd, enum stream_status *stream_status_out)) return lines; } -/** Read from fd, and send lines to log at the specified log level. - * Returns 1 if stream is closed normally, -1 if there is a error reading, and - * 0 otherwise. Handles lines from tor-fw-helper and - * tor_spawn_background() specially. - */ -static int -log_from_pipe(int fd, int severity, const char *executable, - int *child_status) -{ - char buf[256]; - enum stream_status r; - - for (;;) { - r = get_string_from_pipe(fd, buf, sizeof(buf) - 1); - - if (r == IO_STREAM_CLOSED) { - return 1; - } else if (r == IO_STREAM_EAGAIN) { - return 0; - } else if (r == IO_STREAM_TERM) { - return -1; - } - - tor_assert(r == IO_STREAM_OKAY); - - /* Check if buf starts with SPAWN_ERROR_MESSAGE */ - if (strcmpstart(buf, SPAWN_ERROR_MESSAGE) == 0) { - log_portfw_spawn_error_message(buf, executable, child_status); - } else { - log_fn(severity, LD_GENERAL, "Port forwarding helper says: %s", buf); - } - } - - /* We should never get here */ - return -1; -} #endif /* defined(_WIN32) */ /** Reads from <b>fd</b> and stores input in <b>buf_out</b> making @@ -5332,294 +5272,6 @@ get_string_from_pipe(int fd, char *buf_out, size_t count) return IO_STREAM_OKAY; } -/** Parse a <b>line</b> from tor-fw-helper and issue an appropriate - * log message to our user. */ -static void -handle_fw_helper_line(const char *executable, const char *line) -{ - smartlist_t *tokens = smartlist_new(); - char *message = NULL; - char *message_for_log = NULL; - const char *external_port = NULL; - const char *internal_port = NULL; - const char *result = NULL; - int port = 0; - int success = 0; - - if (strcmpstart(line, SPAWN_ERROR_MESSAGE) == 0) { - /* We need to check for SPAWN_ERROR_MESSAGE again here, since it's - * possible that it got sent after we tried to read it in log_from_pipe. - * - * XXX Ideally, we should be using one of stdout/stderr for the real - * output, and one for the output of the startup code. We used to do that - * before cd05f35d2c. - */ - int child_status; - log_portfw_spawn_error_message(line, executable, &child_status); - goto done; - } - - smartlist_split_string(tokens, line, NULL, - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); - - if (smartlist_len(tokens) < 5) - goto err; - - if (strcmp(smartlist_get(tokens, 0), "tor-fw-helper") || - strcmp(smartlist_get(tokens, 1), "tcp-forward")) - goto err; - - external_port = smartlist_get(tokens, 2); - internal_port = smartlist_get(tokens, 3); - result = smartlist_get(tokens, 4); - - if (smartlist_len(tokens) > 5) { - /* If there are more than 5 tokens, they are part of [<message>]. - Let's use a second smartlist to form the whole message; - strncat loops suck. */ - int i; - int message_words_n = smartlist_len(tokens) - 5; - smartlist_t *message_sl = smartlist_new(); - for (i = 0; i < message_words_n; i++) - smartlist_add(message_sl, smartlist_get(tokens, 5+i)); - - tor_assert(smartlist_len(message_sl) > 0); - message = smartlist_join_strings(message_sl, " ", 0, NULL); - - /* wrap the message in log-friendly wrapping */ - tor_asprintf(&message_for_log, " ('%s')", message); - - smartlist_free(message_sl); - } - - port = atoi(external_port); - if (port < 1 || port > 65535) - goto err; - - port = atoi(internal_port); - if (port < 1 || port > 65535) - goto err; - - if (!strcmp(result, "SUCCESS")) - success = 1; - else if (!strcmp(result, "FAIL")) - success = 0; - else - goto err; - - if (!success) { - log_warn(LD_GENERAL, "Tor was unable to forward TCP port '%s' to '%s'%s. " - "Please make sure that your router supports port " - "forwarding protocols (like NAT-PMP). Note that if '%s' is " - "your ORPort, your relay will be unable to receive inbound " - "traffic.", external_port, internal_port, - message_for_log ? message_for_log : "", - internal_port); - } else { - log_info(LD_GENERAL, - "Tor successfully forwarded TCP port '%s' to '%s'%s.", - external_port, internal_port, - message_for_log ? message_for_log : ""); - } - - goto done; - - err: - log_warn(LD_GENERAL, "tor-fw-helper sent us a string we could not " - "parse (%s).", line); - - done: - SMARTLIST_FOREACH(tokens, char *, cp, tor_free(cp)); - smartlist_free(tokens); - tor_free(message); - tor_free(message_for_log); -} - -/** Read what tor-fw-helper has to say in its stdout and handle it - * appropriately */ -static int -handle_fw_helper_output(const char *executable, - process_handle_t *process_handle) -{ - smartlist_t *fw_helper_output = NULL; - enum stream_status stream_status = 0; - - fw_helper_output = - tor_get_lines_from_handle(tor_process_get_stdout_pipe(process_handle), - &stream_status); - if (!fw_helper_output) { /* didn't get any output from tor-fw-helper */ - /* if EAGAIN we should retry in the future */ - return (stream_status == IO_STREAM_EAGAIN) ? 0 : -1; - } - - /* Handle the lines we got: */ - SMARTLIST_FOREACH_BEGIN(fw_helper_output, char *, line) { - handle_fw_helper_line(executable, line); - tor_free(line); - } SMARTLIST_FOREACH_END(line); - - smartlist_free(fw_helper_output); - - return 0; -} - -/** Spawn tor-fw-helper and ask it to forward the ports in - * <b>ports_to_forward</b>. <b>ports_to_forward</b> contains strings - * of the form "<external port>:<internal port>", which is the format - * that tor-fw-helper expects. */ -void -tor_check_port_forwarding(const char *filename, - smartlist_t *ports_to_forward, - time_t now) -{ -/* When fw-helper succeeds, how long do we wait until running it again */ -#define TIME_TO_EXEC_FWHELPER_SUCCESS 300 -/* When fw-helper failed to start, how long do we wait until running it again - */ -#define TIME_TO_EXEC_FWHELPER_FAIL 60 - - /* Static variables are initialized to zero, so child_handle.status=0 - * which corresponds to it not running on startup */ - static process_handle_t *child_handle=NULL; - - static time_t time_to_run_helper = 0; - int stderr_status, retval; - int stdout_status = 0; - - tor_assert(filename); - - /* Start the child, if it is not already running */ - if ((!child_handle || child_handle->status != PROCESS_STATUS_RUNNING) && - time_to_run_helper < now) { - /*tor-fw-helper cli looks like this: tor_fw_helper -p :5555 -p 4555:1111 */ - const char **argv; /* cli arguments */ - int args_n, status; - int argv_index = 0; /* index inside 'argv' */ - - tor_assert(smartlist_len(ports_to_forward) > 0); - - /* check for overflow during 'argv' allocation: - (len(ports_to_forward)*2 + 2)*sizeof(char*) > SIZE_MAX == - len(ports_to_forward) > (((SIZE_MAX/sizeof(char*)) - 2)/2) */ - if ((size_t) smartlist_len(ports_to_forward) > - (((SIZE_MAX/sizeof(char*)) - 2)/2)) { - log_warn(LD_GENERAL, - "Overflow during argv allocation. This shouldn't happen."); - return; - } - /* check for overflow during 'argv_index' increase: - ((len(ports_to_forward)*2 + 2) > INT_MAX) == - len(ports_to_forward) > (INT_MAX - 2)/2 */ - if (smartlist_len(ports_to_forward) > (INT_MAX - 2)/2) { - log_warn(LD_GENERAL, - "Overflow during argv_index increase. This shouldn't happen."); - return; - } - - /* Calculate number of cli arguments: one for the filename, two - for each smartlist element (one for "-p" and one for the - ports), and one for the final NULL. */ - args_n = 1 + 2*smartlist_len(ports_to_forward) + 1; - argv = tor_calloc(args_n, sizeof(char *)); - - argv[argv_index++] = filename; - SMARTLIST_FOREACH_BEGIN(ports_to_forward, const char *, port) { - argv[argv_index++] = "-p"; - argv[argv_index++] = port; - } SMARTLIST_FOREACH_END(port); - argv[argv_index] = NULL; - - /* Assume tor-fw-helper will succeed, start it later*/ - time_to_run_helper = now + TIME_TO_EXEC_FWHELPER_SUCCESS; - - if (child_handle) { - tor_process_handle_destroy(child_handle, 1); - child_handle = NULL; - } - -#ifdef _WIN32 - /* Passing NULL as lpApplicationName makes Windows search for the .exe */ - status = tor_spawn_background(NULL, argv, NULL, &child_handle); -#else - status = tor_spawn_background(filename, argv, NULL, &child_handle); -#endif /* defined(_WIN32) */ - - tor_free_((void*)argv); - argv=NULL; - - if (PROCESS_STATUS_ERROR == status) { - log_warn(LD_GENERAL, "Failed to start port forwarding helper %s", - filename); - time_to_run_helper = now + TIME_TO_EXEC_FWHELPER_FAIL; - return; - } - - log_info(LD_GENERAL, - "Started port forwarding helper (%s) with pid '%d'", - filename, tor_process_get_pid(child_handle)); - } - - /* If child is running, read from its stdout and stderr) */ - if (child_handle && PROCESS_STATUS_RUNNING == child_handle->status) { - /* Read from stdout/stderr and log result */ - retval = 0; -#ifdef _WIN32 - stderr_status = log_from_handle(child_handle->stderr_pipe, LOG_INFO); -#else - stderr_status = log_from_pipe(child_handle->stderr_pipe, - LOG_INFO, filename, &retval); -#endif /* defined(_WIN32) */ - if (handle_fw_helper_output(filename, child_handle) < 0) { - log_warn(LD_GENERAL, "Failed to handle fw helper output."); - stdout_status = -1; - retval = -1; - } - - if (retval) { - /* There was a problem in the child process */ - time_to_run_helper = now + TIME_TO_EXEC_FWHELPER_FAIL; - } - - /* Combine the two statuses in order of severity */ - if (-1 == stdout_status || -1 == stderr_status) - /* There was a failure */ - retval = -1; -#ifdef _WIN32 - else if (!child_handle || tor_get_exit_code(child_handle, 0, NULL) != - PROCESS_EXIT_RUNNING) { - /* process has exited or there was an error */ - /* TODO: Do something with the process return value */ - /* TODO: What if the process output something since - * between log_from_handle and tor_get_exit_code? */ - retval = 1; - } -#else /* !(defined(_WIN32)) */ - else if (1 == stdout_status || 1 == stderr_status) - /* stdout or stderr was closed, the process probably - * exited. It will be reaped by waitpid() in main.c */ - /* TODO: Do something with the process return value */ - retval = 1; -#endif /* defined(_WIN32) */ - else - /* Both are fine */ - retval = 0; - - /* If either pipe indicates a failure, act on it */ - if (0 != retval) { - if (1 == retval) { - log_info(LD_GENERAL, "Port forwarding helper terminated"); - child_handle->status = PROCESS_STATUS_NOTRUNNING; - } else { - log_warn(LD_GENERAL, "Failed to read from port forwarding helper"); - child_handle->status = PROCESS_STATUS_ERROR; - } - - /* TODO: The child might not actually be finished (maybe it failed or - closed stdout/stderr), so maybe we shouldn't start another? */ - } - } -} - /** Initialize the insecure RNG <b>rng</b> from a seed value <b>seed</b>. */ void tor_init_weak_random(tor_weak_rng_t *rng, unsigned seed) diff --git a/src/common/util.h b/src/common/util.h index 9380789128..ae27e5f016 100644 --- a/src/common/util.h +++ b/src/common/util.h @@ -73,9 +73,9 @@ extern int dmalloc_free(const char *file, const int line, void *pnt, } \ STMT_END #else /* !(defined(USE_DMALLOC)) */ -/** Release memory allocated by tor_malloc, tor_realloc, tor_strdup, etc. - * Unlike the free() function, tor_free() will still work on NULL pointers, - * and it sets the pointer value to NULL after freeing it. +/** Release memory allocated by tor_malloc, tor_realloc, tor_strdup, + * etc. Unlike the free() function, the tor_free() macro sets the + * pointer value to NULL after freeing it. * * This is a macro. If you need a function pointer to release memory from * tor_malloc(), use tor_free_(). @@ -88,17 +88,13 @@ extern int dmalloc_free(const char *file, const int line, void *pnt, #ifdef __GNUC__ #define tor_free(p) STMT_BEGIN \ typeof(&(p)) tor_free__tmpvar = &(p); \ - if (PREDICT_LIKELY((*tor_free__tmpvar)!=NULL)) { \ - raw_free(*tor_free__tmpvar); \ - *tor_free__tmpvar=NULL; \ - } \ + raw_free(*tor_free__tmpvar); \ + *tor_free__tmpvar=NULL; \ STMT_END #else #define tor_free(p) STMT_BEGIN \ - if (PREDICT_LIKELY((p)!=NULL)) { \ - raw_free(p); \ - (p)=NULL; \ - } \ + raw_free(p); \ + (p)=NULL; \ STMT_END #endif #endif /* defined(USE_DMALLOC) */ @@ -418,11 +414,6 @@ void start_daemon(void); void finish_daemon(const char *desired_cwd); int write_pidfile(const char *filename); -/* Port forwarding */ -void tor_check_port_forwarding(const char *filename, - struct smartlist_t *ports_to_forward, - time_t now); - void tor_disable_spawning_background_processes(void); typedef struct process_handle_t process_handle_t; @@ -461,9 +452,7 @@ void set_environment_variable_in_smartlist(struct smartlist_t *env_vars, void (*free_old)(void*), int free_p); -/* Values of process_handle_t.status. PROCESS_STATUS_NOTRUNNING must be - * 0 because tor_check_port_forwarding depends on this being the initial - * statue of the static instance of process_handle_t */ +/* Values of process_handle_t.status. */ #define PROCESS_STATUS_NOTRUNNING 0 #define PROCESS_STATUS_RUNNING 1 #define PROCESS_STATUS_ERROR -1 diff --git a/src/common/workqueue.c b/src/common/workqueue.c index ec96959b7d..12e31414e7 100644 --- a/src/common/workqueue.c +++ b/src/common/workqueue.c @@ -1,3 +1,4 @@ + /* copyright (c) 2013-2015, The Tor Project, Inc. */ /* See LICENSE for licensing information */ @@ -24,6 +25,7 @@ #include "orconfig.h" #include "compat.h" +#include "compat_libevent.h" #include "compat_threads.h" #include "crypto.h" #include "util.h" @@ -31,6 +33,8 @@ #include "tor_queue.h" #include "torlog.h" +#include <event2/event.h> + #define WORKQUEUE_PRIORITY_FIRST WQ_PRI_HIGH #define WORKQUEUE_PRIORITY_LAST WQ_PRI_LOW #define WORKQUEUE_N_PRIORITIES (((int) WORKQUEUE_PRIORITY_LAST)+1) @@ -63,6 +67,9 @@ struct threadpool_s { void (*free_update_arg_fn)(void *); /** Array of n_threads update arguments. */ void **update_args; + /** Event to notice when another thread has sent a reply. */ + struct event *reply_event; + void (*reply_cb)(threadpool_t *); /** Number of elements in threads. */ int n_threads; @@ -597,15 +604,41 @@ replyqueue_new(uint32_t alertsocks_flags) return rq; } -/** - * Return the "read socket" for a given reply queue. The main thread should - * listen for read events on this socket, and call replyqueue_process() every - * time it triggers. +/** Internal: Run from the libevent mainloop when there is work to handle in + * the reply queue handler. */ +static void +reply_event_cb(evutil_socket_t sock, short events, void *arg) +{ + threadpool_t *tp = arg; + (void) sock; + (void) events; + replyqueue_process(tp->reply_queue); + if (tp->reply_cb) + tp->reply_cb(tp); +} + +/** Register the threadpool <b>tp</b>'s reply queue with the libevent + * mainloop of <b>base</b>. If <b>tp</b> is provided, it is run after + * each time there is work to process from the reply queue. Return 0 on + * success, -1 on failure. */ -tor_socket_t -replyqueue_get_socket(replyqueue_t *rq) +int +threadpool_register_reply_event(threadpool_t *tp, + void (*cb)(threadpool_t *tp)) { - return rq->alert.read_fd; + struct event_base *base = tor_libevent_get_base(); + + if (tp->reply_event) { + tor_event_free(tp->reply_event); + } + tp->reply_event = tor_event_new(base, + tp->reply_queue->alert.read_fd, + EV_READ|EV_PERSIST, + reply_event_cb, + tp); + tor_assert(tp->reply_event); + tp->reply_cb = cb; + return event_add(tp->reply_event, NULL); } /** diff --git a/src/common/workqueue.h b/src/common/workqueue.h index eb885e680d..e1fe612e2b 100644 --- a/src/common/workqueue.h +++ b/src/common/workqueue.h @@ -56,8 +56,11 @@ threadpool_t *threadpool_new(int n_threads, replyqueue_t *threadpool_get_replyqueue(threadpool_t *tp); replyqueue_t *replyqueue_new(uint32_t alertsocks_flags); -tor_socket_t replyqueue_get_socket(replyqueue_t *rq); void replyqueue_process(replyqueue_t *queue); +struct event_base; +int threadpool_register_reply_event(threadpool_t *tp, + void (*cb)(threadpool_t *tp)); + #endif /* !defined(TOR_WORKQUEUE_H) */ diff --git a/src/ext/ed25519/donna/ed25519-hash-custom.h b/src/ext/ed25519/donna/ed25519-hash-custom.h index 609451abd5..cdeab3e45b 100644 --- a/src/ext/ed25519/donna/ed25519-hash-custom.h +++ b/src/ext/ed25519/donna/ed25519-hash-custom.h @@ -9,7 +9,7 @@ void ed25519_hash(uint8_t *hash, const uint8_t *in, size_t inlen); */ -#include "crypto.h" +#include "crypto_digest.h" typedef struct ed25519_hash_context { crypto_digest_t *ctx; diff --git a/src/ext/ed25519/ref10/crypto_hash_sha512.h b/src/ext/ed25519/ref10/crypto_hash_sha512.h index 5dad935c79..7faddb1597 100644 --- a/src/ext/ed25519/ref10/crypto_hash_sha512.h +++ b/src/ext/ed25519/ref10/crypto_hash_sha512.h @@ -1,5 +1,5 @@ /* Added for Tor. */ -#include "crypto.h" +#include "crypto_digest.h" /* Set 'out' to the 512-bit SHA512 hash of the 'len'-byte string in 'inp' */ #define crypto_hash_sha512(out, inp, len) \ diff --git a/src/or/bridges.c b/src/or/bridges.c index 29d00f37ba..699e030e6c 100644 --- a/src/or/bridges.c +++ b/src/or/bridges.c @@ -11,6 +11,8 @@ * Bridges are fixed entry nodes, used for censorship circumvention. **/ +#define TOR_BRIDGES_PRIVATE + #include "or.h" #include "bridges.h" #include "circuitbuild.h" @@ -93,7 +95,7 @@ sweep_bridge_list(void) } /** Initialize the bridge list to empty, creating it if needed. */ -static void +STATIC void clear_bridge_list(void) { if (!bridge_list) @@ -156,7 +158,7 @@ bridge_get_addr_port(const bridge_info_t *bridge) * bridge with no known digest whose address matches any of the * tor_addr_port_t's in <b>orports</b>, return that bridge. Else return * NULL. */ -static bridge_info_t * +STATIC bridge_info_t * get_configured_bridge_by_orports_digest(const char *digest, const smartlist_t *orports) { @@ -350,7 +352,7 @@ bridge_has_digest(const bridge_info_t *bridge, const char *digest) * existing bridge with the same address and port, and warn the user as * appropriate. */ -static void +STATIC void bridge_resolve_conflicts(const tor_addr_t *addr, uint16_t port, const char *digest, const char *transport_name) { @@ -471,7 +473,7 @@ bridge_add_from_config(bridge_line_t *bridge_line) } /** If <b>digest</b> is one of our known bridges, return it. */ -bridge_info_t * +STATIC bridge_info_t * find_bridge_by_digest(const char *digest) { if (! bridge_list) diff --git a/src/or/bridges.h b/src/or/bridges.h index 54a6250259..3108eb555d 100644 --- a/src/or/bridges.h +++ b/src/or/bridges.h @@ -20,7 +20,6 @@ typedef struct bridge_info_t bridge_info_t; void mark_bridge_list(void); void sweep_bridge_list(void); const smartlist_t *bridge_list_get(void); -bridge_info_t *find_bridge_by_digest(const char *digest); const uint8_t *bridge_get_rsa_id_digest(const bridge_info_t *bridge); const tor_addr_port_t * bridge_get_addr_port(const bridge_info_t *bridge); bridge_info_t *get_configured_bridge_by_addr_port_digest( @@ -65,5 +64,17 @@ MOCK_DECL(download_status_t *, get_bridge_dl_status_by_id, void bridges_free_all(void); +#ifdef TOR_BRIDGES_PRIVATE +STATIC void clear_bridge_list(void); +STATIC bridge_info_t *find_bridge_by_digest(const char *digest); +STATIC bridge_info_t *get_configured_bridge_by_orports_digest( + const char *digest, + const smartlist_t *orports); +STATIC void bridge_resolve_conflicts(const tor_addr_t *addr, + uint16_t port, + const char *digest, + const char *transport_name); +#endif /* defined(TOR_BRIDGES_PRIVATE) */ + #endif /* !defined(TOR_BRIDGES_H) */ diff --git a/src/or/channel.c b/src/or/channel.c index a4740dd752..68245db497 100644 --- a/src/or/channel.c +++ b/src/or/channel.c @@ -2109,21 +2109,6 @@ channel_listener_dumpstats(int severity) } /** - * Set the cmux policy on all active channels. - */ -void -channel_set_cmux_policy_everywhere(circuitmux_policy_t *pol) -{ - if (!active_channels) return; - - SMARTLIST_FOREACH_BEGIN(active_channels, channel_t *, curr) { - if (curr->cmux) { - circuitmux_set_policy(curr->cmux, pol); - } - } SMARTLIST_FOREACH_END(curr); -} - -/** * Clean up channels. * * This gets called periodically from run_scheduled_events() in main.c; @@ -2402,7 +2387,7 @@ channel_get_for_extend(const char *rsa_id_digest, { channel_t *chan, *best = NULL; int n_inprogress_goodaddr = 0, n_old = 0; - int n_noncanonical = 0, n_possible = 0; + int n_noncanonical = 0; tor_assert(msg_out); tor_assert(launch_out); @@ -2465,8 +2450,6 @@ channel_get_for_extend(const char *rsa_id_digest, continue; } - ++n_possible; - if (!best) { best = chan; /* If we have no 'best' so far, this one is good enough. */ continue; diff --git a/src/or/channel.h b/src/or/channel.h index 0af5aed414..6cf8cd7f72 100644 --- a/src/or/channel.h +++ b/src/or/channel.h @@ -422,9 +422,6 @@ void channel_free_all(void); void channel_dumpstats(int severity); void channel_listener_dumpstats(int severity); -/* Set the cmux policy on all active channels */ -void channel_set_cmux_policy_everywhere(circuitmux_policy_t *pol); - #ifdef TOR_CHANNEL_INTERNAL_ #ifdef CHANNEL_PRIVATE_ diff --git a/src/or/channelpadding.c b/src/or/channelpadding.c index 5da3009e67..33b1cba355 100644 --- a/src/or/channelpadding.c +++ b/src/or/channelpadding.c @@ -20,7 +20,6 @@ #include "rephist.h" #include "router.h" #include "compat_time.h" -#include <event2/event.h> #include "rendservice.h" STATIC int32_t channelpadding_get_netflow_inactive_timeout_ms( diff --git a/src/or/channeltls.c b/src/or/channeltls.c index 9000703b01..54d94f6109 100644 --- a/src/or/channeltls.c +++ b/src/or/channeltls.c @@ -160,9 +160,8 @@ channel_tls_common_init(channel_tls_t *tlschan) chan->write_var_cell = channel_tls_write_var_cell_method; chan->cmux = circuitmux_alloc(); - if (cell_ewma_enabled()) { - circuitmux_set_policy(chan->cmux, &ewma_policy); - } + /* We only have one policy for now so always set it to EWMA. */ + circuitmux_set_policy(chan->cmux, &ewma_policy); } /** diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index 01921bac15..488f6a2148 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -56,6 +56,7 @@ #include "onion_fast.h" #include "policies.h" #include "relay.h" +#include "relay_crypto.h" #include "rendcommon.h" #include "rephist.h" #include "router.h" @@ -1055,7 +1056,7 @@ circuit_build_no_more_hops(origin_circuit_t *circ) clear_broken_connection_map(1); if (server_mode(options) && !check_whether_orport_reachable(options)) { inform_testing_reachability(); - consider_testing_reachability(1, 1); + router_do_reachability_checks(1, 1); } } @@ -1336,69 +1337,10 @@ circuit_init_cpath_crypto(crypt_path_t *cpath, const char *key_data, size_t key_data_len, int reverse, int is_hs_v3) { - crypto_digest_t *tmp_digest; - crypto_cipher_t *tmp_crypto; - size_t digest_len = 0; - size_t cipher_key_len = 0; tor_assert(cpath); - tor_assert(key_data); - tor_assert(!(cpath->f_crypto || cpath->b_crypto || - cpath->f_digest || cpath->b_digest)); - - /* Basic key size validation */ - if (is_hs_v3 && BUG(key_data_len != HS_NTOR_KEY_EXPANSION_KDF_OUT_LEN)) { - return -1; - } else if (!is_hs_v3 && BUG(key_data_len != CPATH_KEY_MATERIAL_LEN)) { - return -1; - } - - /* If we are using this cpath for next gen onion services use SHA3-256, - otherwise use good ol' SHA1 */ - if (is_hs_v3) { - digest_len = DIGEST256_LEN; - cipher_key_len = CIPHER256_KEY_LEN; - cpath->f_digest = crypto_digest256_new(DIGEST_SHA3_256); - cpath->b_digest = crypto_digest256_new(DIGEST_SHA3_256); - } else { - digest_len = DIGEST_LEN; - cipher_key_len = CIPHER_KEY_LEN; - cpath->f_digest = crypto_digest_new(); - cpath->b_digest = crypto_digest_new(); - } - - tor_assert(digest_len != 0); - tor_assert(cipher_key_len != 0); - const int cipher_key_bits = (int) cipher_key_len * 8; - - crypto_digest_add_bytes(cpath->f_digest, key_data, digest_len); - crypto_digest_add_bytes(cpath->b_digest, key_data+digest_len, digest_len); - - cpath->f_crypto = crypto_cipher_new_with_bits(key_data+(2*digest_len), - cipher_key_bits); - if (!cpath->f_crypto) { - log_warn(LD_BUG,"Forward cipher initialization failed."); - return -1; - } - - cpath->b_crypto = crypto_cipher_new_with_bits( - key_data+(2*digest_len)+cipher_key_len, - cipher_key_bits); - if (!cpath->b_crypto) { - log_warn(LD_BUG,"Backward cipher initialization failed."); - return -1; - } - - if (reverse) { - tmp_digest = cpath->f_digest; - cpath->f_digest = cpath->b_digest; - cpath->b_digest = tmp_digest; - tmp_crypto = cpath->f_crypto; - cpath->f_crypto = cpath->b_crypto; - cpath->b_crypto = tmp_crypto; - } - - return 0; + return relay_crypto_init(&cpath->crypto, key_data, key_data_len, reverse, + is_hs_v3); } /** A "created" cell <b>reply</b> came back to us on circuit <b>circ</b>. @@ -1521,7 +1463,6 @@ onionskin_answer(or_circuit_t *circ, const uint8_t *rend_circ_nonce) { cell_t cell; - crypt_path_t *tmp_cpath; tor_assert(keys_len == CPATH_KEY_MATERIAL_LEN); @@ -1532,25 +1473,15 @@ onionskin_answer(or_circuit_t *circ, } cell.circ_id = circ->p_circ_id; - tmp_cpath = tor_malloc_zero(sizeof(crypt_path_t)); - tmp_cpath->magic = CRYPT_PATH_MAGIC; - circuit_set_state(TO_CIRCUIT(circ), CIRCUIT_STATE_OPEN); log_debug(LD_CIRC,"init digest forward 0x%.8x, backward 0x%.8x.", (unsigned int)get_uint32(keys), (unsigned int)get_uint32(keys+20)); - if (circuit_init_cpath_crypto(tmp_cpath, keys, keys_len, 0, 0)<0) { + if (relay_crypto_init(&circ->crypto, keys, keys_len, 0, 0)<0) { log_warn(LD_BUG,"Circuit initialization failed"); - tor_free(tmp_cpath); return -1; } - circ->n_digest = tmp_cpath->f_digest; - circ->n_crypto = tmp_cpath->f_crypto; - circ->p_digest = tmp_cpath->b_digest; - circ->p_crypto = tmp_cpath->b_crypto; - tmp_cpath->magic = 0; - tor_free(tmp_cpath); memcpy(circ->rend_circ_nonce, rend_circ_nonce, DIGEST_LEN); @@ -1611,7 +1542,7 @@ onionskin_answer(or_circuit_t *circ, * rend_service_launch_establish_intro()) * * - We are a router testing its own reachabiity - * (CIRCUIT_PURPOSE_TESTING, via consider_testing_reachability()) + * (CIRCUIT_PURPOSE_TESTING, via router_do_reachability_checks()) * * onion_pick_cpath_exit() bypasses us (by not calling * new_route_len()) in the one-hop tunnel case, so we don't need to @@ -2876,8 +2807,10 @@ extend_info_from_node(const node_t *node, int for_direct_connect) valid_addr = fascist_firewall_choose_address_node(node, FIREWALL_OR_CONNECTION, 0, &ap); - else - valid_addr = !node_get_prim_orport(node, &ap); + else { + node_get_prim_orport(node, &ap); + valid_addr = tor_addr_port_is_valid_ap(&ap, 0); + } if (valid_addr) log_debug(LD_CIRC, "using %s for %s", diff --git a/src/or/circuitlist.c b/src/or/circuitlist.c index 7bdef0b878..9a82713cbe 100644 --- a/src/or/circuitlist.c +++ b/src/or/circuitlist.c @@ -76,6 +76,7 @@ #include "onion_fast.h" #include "policies.h" #include "relay.h" +#include "relay_crypto.h" #include "rendclient.h" #include "rendcommon.h" #include "rephist.h" @@ -406,9 +407,6 @@ circuit_set_p_circid_chan(or_circuit_t *or_circ, circid_t id, circuit_set_circid_chan_helper(circ, CELL_DIRECTION_IN, id, chan); if (chan) { - tor_assert(bool_eq(or_circ->p_chan_cells.n, - or_circ->next_active_on_p_chan)); - chan->timestamp_last_had_circuits = approx_time(); } @@ -431,8 +429,6 @@ circuit_set_n_circid_chan(circuit_t *circ, circid_t id, circuit_set_circid_chan_helper(circ, CELL_DIRECTION_OUT, id, chan); if (chan) { - tor_assert(bool_eq(circ->n_chan_cells.n, circ->next_active_on_n_chan)); - chan->timestamp_last_had_circuits = approx_time(); } @@ -1087,10 +1083,7 @@ circuit_free_(circuit_t *circ) should_free = (ocirc->workqueue_entry == NULL); - crypto_cipher_free(ocirc->p_crypto); - crypto_digest_free(ocirc->p_digest); - crypto_cipher_free(ocirc->n_crypto); - crypto_digest_free(ocirc->n_digest); + relay_crypto_clear(ô->crypto); if (ocirc->rend_splice) { or_circuit_t *other = ocirc->rend_splice; @@ -1230,10 +1223,7 @@ circuit_free_cpath_node(crypt_path_t *victim) if (!victim) return; - crypto_cipher_free(victim->f_crypto); - crypto_cipher_free(victim->b_crypto); - crypto_digest_free(victim->f_digest); - crypto_digest_free(victim->b_digest); + relay_crypto_clear(&victim->crypto); onion_handshake_state_release(&victim->handshake_state); crypto_dh_free(victim->rend_dh_handshake_state); extend_info_free(victim->extend_info); @@ -2596,8 +2586,7 @@ assert_cpath_layer_ok(const crypt_path_t *cp) switch (cp->state) { case CPATH_STATE_OPEN: - tor_assert(cp->f_crypto); - tor_assert(cp->b_crypto); + relay_crypto_assert_ok(&cp->crypto); /* fall through */ case CPATH_STATE_CLOSED: /*XXXX Assert that there's no handshake_state either. */ @@ -2687,10 +2676,7 @@ assert_circuit_ok,(const circuit_t *c)) c->state == CIRCUIT_STATE_GUARD_WAIT) { tor_assert(!c->n_chan_create_cell); if (or_circ) { - tor_assert(or_circ->n_crypto); - tor_assert(or_circ->p_crypto); - tor_assert(or_circ->n_digest); - tor_assert(or_circ->p_digest); + relay_crypto_assert_ok(&or_circ->crypto); } } if (c->state == CIRCUIT_STATE_CHAN_WAIT && !c->marked_for_close) { diff --git a/src/or/circuitmux.c b/src/or/circuitmux.c index fe3d8f1332..f9f5faa057 100644 --- a/src/or/circuitmux.c +++ b/src/or/circuitmux.c @@ -114,13 +114,6 @@ struct circuitmux_s { */ chanid_circid_muxinfo_map_t *chanid_circid_map; - /* - * Double-linked ring of circuits with queued cells waiting for room to - * free up on this connection's outbuf. Every time we pull cells from - * a circuit, we advance this pointer to the next circuit in the ring. - */ - struct circuit_t *active_circuits_head, *active_circuits_tail; - /** List of queued destroy cells */ destroy_cell_queue_t destroy_cell_queue; /** Boolean: True iff the last cell to circuitmux_get_first_active_circuit @@ -177,17 +170,6 @@ struct chanid_circid_muxinfo_t { }; /* - * Internal-use #defines - */ - -#ifdef CMUX_PARANOIA -#define circuitmux_assert_okay_paranoid(cmux) \ - circuitmux_assert_okay(cmux) -#else -#define circuitmux_assert_okay_paranoid(cmux) -#endif /* defined(CMUX_PARANOIA) */ - -/* * Static function declarations */ @@ -199,21 +181,9 @@ chanid_circid_entry_hash(chanid_circid_muxinfo_t *a); static chanid_circid_muxinfo_t * circuitmux_find_map_entry(circuitmux_t *cmux, circuit_t *circ); static void -circuitmux_make_circuit_active(circuitmux_t *cmux, circuit_t *circ, - cell_direction_t direction); +circuitmux_make_circuit_active(circuitmux_t *cmux, circuit_t *circ); static void -circuitmux_make_circuit_inactive(circuitmux_t *cmux, circuit_t *circ, - cell_direction_t direction); -static inline void -circuitmux_move_active_circ_to_tail(circuitmux_t *cmux, circuit_t *circ, - cell_direction_t direction); -static inline circuit_t ** -circuitmux_next_active_circ_p(circuitmux_t *cmux, circuit_t *circ); -static inline circuit_t ** -circuitmux_prev_active_circ_p(circuitmux_t *cmux, circuit_t *circ); -static void circuitmux_assert_okay_pass_one(circuitmux_t *cmux); -static void circuitmux_assert_okay_pass_two(circuitmux_t *cmux); -static void circuitmux_assert_okay_pass_three(circuitmux_t *cmux); +circuitmux_make_circuit_inactive(circuitmux_t *cmux, circuit_t *circ); /* Static global variables */ @@ -223,119 +193,6 @@ static int64_t global_destroy_ctr = 0; /* Function definitions */ /** - * Linked list helpers - */ - -/** - * Move an active circuit to the tail of the cmux's active circuits list; - * used by circuitmux_notify_xmit_cells(). - */ - -static inline void -circuitmux_move_active_circ_to_tail(circuitmux_t *cmux, circuit_t *circ, - cell_direction_t direction) -{ - circuit_t **next_p = NULL, **prev_p = NULL; - circuit_t **next_prev = NULL, **prev_next = NULL; - circuit_t **tail_next = NULL; - or_circuit_t *or_circ = NULL; - - tor_assert(cmux); - tor_assert(circ); - - circuitmux_assert_okay_paranoid(cmux); - - /* Figure out our next_p and prev_p for this cmux/direction */ - if (direction) { - if (direction == CELL_DIRECTION_OUT) { - tor_assert(circ->n_mux == cmux); - next_p = &(circ->next_active_on_n_chan); - prev_p = &(circ->prev_active_on_n_chan); - } else { - or_circ = TO_OR_CIRCUIT(circ); - tor_assert(or_circ->p_mux == cmux); - next_p = &(or_circ->next_active_on_p_chan); - prev_p = &(or_circ->prev_active_on_p_chan); - } - } else { - if (circ->n_mux == cmux) { - next_p = &(circ->next_active_on_n_chan); - prev_p = &(circ->prev_active_on_n_chan); - } else { - or_circ = TO_OR_CIRCUIT(circ); - tor_assert(or_circ->p_mux == cmux); - next_p = &(or_circ->next_active_on_p_chan); - prev_p = &(or_circ->prev_active_on_p_chan); - } - } - tor_assert(next_p); - tor_assert(prev_p); - - /* Check if this really is an active circuit */ - if ((*next_p == NULL && *prev_p == NULL) && - !(circ == cmux->active_circuits_head || - circ == cmux->active_circuits_tail)) { - /* Not active, no-op */ - return; - } - - /* Check if this is already the tail */ - if (circ == cmux->active_circuits_tail) return; - - /* Okay, we have to move it; figure out next_prev and prev_next */ - if (*next_p) next_prev = circuitmux_prev_active_circ_p(cmux, *next_p); - if (*prev_p) prev_next = circuitmux_next_active_circ_p(cmux, *prev_p); - /* Adjust the previous node's next pointer, if any */ - if (prev_next) *prev_next = *next_p; - /* Otherwise, we were the head */ - else cmux->active_circuits_head = *next_p; - /* Adjust the next node's previous pointer, if any */ - if (next_prev) *next_prev = *prev_p; - /* We're out of the list; now re-attach at the tail */ - /* Adjust our next and prev pointers */ - *next_p = NULL; - *prev_p = cmux->active_circuits_tail; - /* Set the next pointer of the tail, or the head if none */ - if (cmux->active_circuits_tail) { - tail_next = circuitmux_next_active_circ_p(cmux, - cmux->active_circuits_tail); - *tail_next = circ; - } else { - cmux->active_circuits_head = circ; - } - /* Set the tail to this circuit */ - cmux->active_circuits_tail = circ; - - circuitmux_assert_okay_paranoid(cmux); -} - -static inline circuit_t ** -circuitmux_next_active_circ_p(circuitmux_t *cmux, circuit_t *circ) -{ - tor_assert(cmux); - tor_assert(circ); - - if (circ->n_mux == cmux) return &(circ->next_active_on_n_chan); - else { - tor_assert(TO_OR_CIRCUIT(circ)->p_mux == cmux); - return &(TO_OR_CIRCUIT(circ)->next_active_on_p_chan); - } -} - -static inline circuit_t ** -circuitmux_prev_active_circ_p(circuitmux_t *cmux, circuit_t *circ) -{ - tor_assert(cmux); - tor_assert(circ); - - if (circ->n_mux == cmux) return &(circ->prev_active_on_n_chan); - else { - tor_assert(TO_OR_CIRCUIT(circ)->p_mux == cmux); - return &(TO_OR_CIRCUIT(circ)->prev_active_on_p_chan); - } -} - -/** * Helper for chanid_circid_cell_count_map_t hash table: compare the channel * ID and circuit ID for a and b, and return less than, equal to, or greater * than zero appropriately. @@ -406,11 +263,6 @@ circuitmux_detach_all_circuits(circuitmux_t *cmux, smartlist_t *detached_out) circuit_t *circ = NULL; tor_assert(cmux); - /* - * Don't circuitmux_assert_okay_paranoid() here; this gets called when - * channels are being freed and have already been unregistered, so - * the channel ID lookups it does will fail. - */ i = HT_START(chanid_circid_muxinfo_map, cmux->chanid_circid_map); while (i) { @@ -435,7 +287,7 @@ circuitmux_detach_all_circuits(circuitmux_t *cmux, smartlist_t *detached_out) */ if (to_remove->muxinfo.cell_count > 0) { - circuitmux_make_circuit_inactive(cmux, circ, CELL_DIRECTION_OUT); + circuitmux_make_circuit_inactive(cmux, circ); } /* Clear n_mux */ @@ -450,7 +302,7 @@ circuitmux_detach_all_circuits(circuitmux_t *cmux, smartlist_t *detached_out) */ if (to_remove->muxinfo.cell_count > 0) { - circuitmux_make_circuit_inactive(cmux, circ, CELL_DIRECTION_IN); + circuitmux_make_circuit_inactive(cmux, circ); } /* @@ -606,9 +458,7 @@ circuitmux_clear_policy(circuitmux_t *cmux) tor_assert(cmux); /* Internally, this is just setting policy to NULL */ - if (cmux->policy) { - circuitmux_set_policy(cmux, NULL); - } + circuitmux_set_policy(cmux, NULL); } /** @@ -944,7 +794,6 @@ circuitmux_attach_circuit,(circuitmux_t *cmux, circuit_t *circ, tor_assert(circ); tor_assert(direction == CELL_DIRECTION_IN || direction == CELL_DIRECTION_OUT); - circuitmux_assert_okay_paranoid(cmux); /* * Figure out which channel we're using, and get the circuit's current @@ -1002,10 +851,10 @@ circuitmux_attach_circuit,(circuitmux_t *cmux, circuit_t *circ, */ if (hashent->muxinfo.cell_count > 0 && cell_count == 0) { --(cmux->n_active_circuits); - circuitmux_make_circuit_inactive(cmux, circ, direction); + circuitmux_make_circuit_inactive(cmux, circ); } else if (hashent->muxinfo.cell_count == 0 && cell_count > 0) { ++(cmux->n_active_circuits); - circuitmux_make_circuit_active(cmux, circ, direction); + circuitmux_make_circuit_active(cmux, circ); } cmux->n_cells -= hashent->muxinfo.cell_count; cmux->n_cells += cell_count; @@ -1033,7 +882,7 @@ circuitmux_attach_circuit,(circuitmux_t *cmux, circuit_t *circ, hashent->muxinfo.cell_count = cell_count; hashent->muxinfo.direction = direction; /* Allocate policy specific circuit data if we need it */ - if (cmux->policy && cmux->policy->alloc_circ_data) { + if (cmux->policy->alloc_circ_data) { /* Assert that we have the means to free policy-specific data */ tor_assert(cmux->policy->free_circ_data); /* Allocate it */ @@ -1053,25 +902,14 @@ circuitmux_attach_circuit,(circuitmux_t *cmux, circuit_t *circ, if (direction == CELL_DIRECTION_OUT) circ->n_mux = cmux; else TO_OR_CIRCUIT(circ)->p_mux = cmux; - /* Make sure the next/prev pointers are NULL */ - if (direction == CELL_DIRECTION_OUT) { - circ->next_active_on_n_chan = NULL; - circ->prev_active_on_n_chan = NULL; - } else { - TO_OR_CIRCUIT(circ)->next_active_on_p_chan = NULL; - TO_OR_CIRCUIT(circ)->prev_active_on_p_chan = NULL; - } - /* Update counters */ ++(cmux->n_circuits); if (cell_count > 0) { ++(cmux->n_active_circuits); - circuitmux_make_circuit_active(cmux, circ, direction); + circuitmux_make_circuit_active(cmux, circ); } cmux->n_cells += cell_count; } - - circuitmux_assert_okay_paranoid(cmux); } /** @@ -1095,7 +933,6 @@ circuitmux_detach_circuit,(circuitmux_t *cmux, circuit_t *circ)) tor_assert(cmux); tor_assert(cmux->chanid_circid_map); tor_assert(circ); - circuitmux_assert_okay_paranoid(cmux); /* See if we have it for n_chan/n_circ_id */ if (circ->n_chan) { @@ -1133,7 +970,7 @@ circuitmux_detach_circuit,(circuitmux_t *cmux, circuit_t *circ)) if (hashent->muxinfo.cell_count > 0) { --(cmux->n_active_circuits); /* This does policy notifies, so comes before freeing policy data */ - circuitmux_make_circuit_inactive(cmux, circ, last_searched_direction); + circuitmux_make_circuit_inactive(cmux, circ); } cmux->n_cells -= hashent->muxinfo.cell_count; @@ -1162,8 +999,6 @@ circuitmux_detach_circuit,(circuitmux_t *cmux, circuit_t *circ)) /* Free the hash entry */ tor_free(hashent); } - - circuitmux_assert_okay_paranoid(cmux); } /** @@ -1172,94 +1007,22 @@ circuitmux_detach_circuit,(circuitmux_t *cmux, circuit_t *circ)) */ static void -circuitmux_make_circuit_active(circuitmux_t *cmux, circuit_t *circ, - cell_direction_t direction) +circuitmux_make_circuit_active(circuitmux_t *cmux, circuit_t *circ) { - circuit_t **next_active = NULL, **prev_active = NULL, **next_prev = NULL; - circuitmux_t *circuit_cmux = NULL; - chanid_circid_muxinfo_t *hashent = NULL; - channel_t *chan = NULL; - circid_t circ_id; - int already_active; - tor_assert(cmux); + tor_assert(cmux->policy); tor_assert(circ); - tor_assert(direction == CELL_DIRECTION_OUT || - direction == CELL_DIRECTION_IN); - /* - * Don't circuitmux_assert_okay_paranoid(cmux) here because the cell count - * already got changed and we have to update the list for it to be consistent - * again. - */ - - /* Get the right set of active list links for this direction */ - if (direction == CELL_DIRECTION_OUT) { - next_active = &(circ->next_active_on_n_chan); - prev_active = &(circ->prev_active_on_n_chan); - circuit_cmux = circ->n_mux; - chan = circ->n_chan; - circ_id = circ->n_circ_id; - } else { - next_active = &(TO_OR_CIRCUIT(circ)->next_active_on_p_chan); - prev_active = &(TO_OR_CIRCUIT(circ)->prev_active_on_p_chan); - circuit_cmux = TO_OR_CIRCUIT(circ)->p_mux; - chan = TO_OR_CIRCUIT(circ)->p_chan; - circ_id = TO_OR_CIRCUIT(circ)->p_circ_id; - } - - /* Assert that it is attached to this mux and a channel */ - tor_assert(cmux == circuit_cmux); - tor_assert(chan != NULL); - - /* - * Check if the circuit really was inactive; if it's active, at least one - * of the next_active and prev_active pointers will not be NULL, or this - * circuit will be either the head or tail of the list for this cmux. - */ - already_active = (*prev_active != NULL || *next_active != NULL || - cmux->active_circuits_head == circ || - cmux->active_circuits_tail == circ); - - /* If we're already active, log a warning and finish */ - if (already_active) { - log_warn(LD_CIRC, - "Circuit %u on channel " U64_FORMAT " was already active", - (unsigned)circ_id, U64_PRINTF_ARG(chan->global_identifier)); - return; - } - - /* - * This is going at the head of the list; if the old head is not NULL, - * then its prev pointer should point to this. - */ - *next_active = cmux->active_circuits_head; /* Next is old head */ - *prev_active = NULL; /* Prev is NULL (this will be the head) */ - if (cmux->active_circuits_head) { - /* The list had an old head; update its prev pointer */ - next_prev = - circuitmux_prev_active_circ_p(cmux, cmux->active_circuits_head); - tor_assert(next_prev); - *next_prev = circ; - } else { - /* The list was empty; this becomes the tail as well */ - cmux->active_circuits_tail = circ; - } - /* This becomes the new head of the list */ - cmux->active_circuits_head = circ; /* Policy-specific notification */ - if (cmux->policy && - cmux->policy->notify_circ_active) { + if (cmux->policy->notify_circ_active) { /* Okay, we need to check the circuit for policy data now */ - hashent = circuitmux_find_map_entry(cmux, circ); + chanid_circid_muxinfo_t *hashent = circuitmux_find_map_entry(cmux, circ); /* We should have found something */ tor_assert(hashent); /* Notify */ cmux->policy->notify_circ_active(cmux, cmux->policy_data, circ, hashent->muxinfo.policy_data); } - - circuitmux_assert_okay_paranoid(cmux); } /** @@ -1268,112 +1031,22 @@ circuitmux_make_circuit_active(circuitmux_t *cmux, circuit_t *circ, */ static void -circuitmux_make_circuit_inactive(circuitmux_t *cmux, circuit_t *circ, - cell_direction_t direction) +circuitmux_make_circuit_inactive(circuitmux_t *cmux, circuit_t *circ) { - circuit_t **next_active = NULL, **prev_active = NULL; - circuit_t **next_prev = NULL, **prev_next = NULL; - circuitmux_t *circuit_cmux = NULL; - chanid_circid_muxinfo_t *hashent = NULL; - channel_t *chan = NULL; - circid_t circ_id; - int already_inactive; - tor_assert(cmux); + tor_assert(cmux->policy); tor_assert(circ); - tor_assert(direction == CELL_DIRECTION_OUT || - direction == CELL_DIRECTION_IN); - /* - * Don't circuitmux_assert_okay_paranoid(cmux) here because the cell count - * already got changed and we have to update the list for it to be consistent - * again. - */ - - /* Get the right set of active list links for this direction */ - if (direction == CELL_DIRECTION_OUT) { - next_active = &(circ->next_active_on_n_chan); - prev_active = &(circ->prev_active_on_n_chan); - circuit_cmux = circ->n_mux; - chan = circ->n_chan; - circ_id = circ->n_circ_id; - } else { - next_active = &(TO_OR_CIRCUIT(circ)->next_active_on_p_chan); - prev_active = &(TO_OR_CIRCUIT(circ)->prev_active_on_p_chan); - circuit_cmux = TO_OR_CIRCUIT(circ)->p_mux; - chan = TO_OR_CIRCUIT(circ)->p_chan; - circ_id = TO_OR_CIRCUIT(circ)->p_circ_id; - } - - /* Assert that it is attached to this mux and a channel */ - tor_assert(cmux == circuit_cmux); - tor_assert(chan != NULL); - - /* - * Check if the circuit really was active; if it's inactive, the - * next_active and prev_active pointers will be NULL and this circuit - * will not be the head or tail of the list for this cmux. - */ - already_inactive = (*prev_active == NULL && *next_active == NULL && - cmux->active_circuits_head != circ && - cmux->active_circuits_tail != circ); - - /* If we're already inactive, log a warning and finish */ - if (already_inactive) { - log_warn(LD_CIRC, - "Circuit %d on channel " U64_FORMAT " was already inactive", - (unsigned)circ_id, U64_PRINTF_ARG(chan->global_identifier)); - return; - } - - /* Remove from the list; first get next_prev and prev_next */ - if (*next_active) { - /* - * If there's a next circuit, its previous circuit becomes this - * circuit's previous circuit. - */ - next_prev = circuitmux_prev_active_circ_p(cmux, *next_active); - } else { - /* Else, the tail becomes this circuit's previous circuit */ - next_prev = &(cmux->active_circuits_tail); - } - - /* Got next_prev, now prev_next */ - if (*prev_active) { - /* - * If there's a previous circuit, its next circuit becomes this circuit's - * next circuit. - */ - prev_next = circuitmux_next_active_circ_p(cmux, *prev_active); - } else { - /* Else, the head becomes this circuit's next circuit */ - prev_next = &(cmux->active_circuits_head); - } - - /* Assert that we got sensible values for the next/prev pointers */ - tor_assert(next_prev != NULL); - tor_assert(prev_next != NULL); - - /* Update the next/prev pointers - this removes circ from the list */ - *next_prev = *prev_active; - *prev_next = *next_active; - - /* Now null out prev_active/next_active */ - *prev_active = NULL; - *next_active = NULL; /* Policy-specific notification */ - if (cmux->policy && - cmux->policy->notify_circ_inactive) { + if (cmux->policy->notify_circ_inactive) { /* Okay, we need to check the circuit for policy data now */ - hashent = circuitmux_find_map_entry(cmux, circ); + chanid_circid_muxinfo_t *hashent = circuitmux_find_map_entry(cmux, circ); /* We should have found something */ tor_assert(hashent); /* Notify */ cmux->policy->notify_circ_inactive(cmux, cmux->policy_data, circ, hashent->muxinfo.policy_data); } - - circuitmux_assert_okay_paranoid(cmux); } /** @@ -1400,8 +1073,6 @@ circuitmux_set_num_cells(circuitmux_t *cmux, circuit_t *circ, tor_assert(cmux); tor_assert(circ); - circuitmux_assert_okay_paranoid(cmux); - /* Search for this circuit's entry */ hashent = circuitmux_find_map_entry(cmux, circ); /* Assert that we found one */ @@ -1412,7 +1083,7 @@ circuitmux_set_num_cells(circuitmux_t *cmux, circuit_t *circ, cmux->n_cells += n_cells; /* Do we need to notify a cmux policy? */ - if (cmux->policy && cmux->policy->notify_set_n_cells) { + if (cmux->policy->notify_set_n_cells) { /* Call notify_set_n_cells */ cmux->policy->notify_set_n_cells(cmux, cmux->policy_data, @@ -1428,21 +1099,15 @@ circuitmux_set_num_cells(circuitmux_t *cmux, circuit_t *circ, if (hashent->muxinfo.cell_count > 0 && n_cells == 0) { --(cmux->n_active_circuits); hashent->muxinfo.cell_count = n_cells; - circuitmux_make_circuit_inactive(cmux, circ, hashent->muxinfo.direction); + circuitmux_make_circuit_inactive(cmux, circ); /* Is the old cell count == 0 and the new cell count > 0 ? */ } else if (hashent->muxinfo.cell_count == 0 && n_cells > 0) { ++(cmux->n_active_circuits); hashent->muxinfo.cell_count = n_cells; - circuitmux_make_circuit_active(cmux, circ, hashent->muxinfo.direction); + circuitmux_make_circuit_active(cmux, circ); } else { - /* - * Update the entry cell count like this so we can put a - * circuitmux_assert_okay_paranoid inside make_circuit_(in)active() too. - */ hashent->muxinfo.cell_count = n_cells; } - - circuitmux_assert_okay_paranoid(cmux); } /* @@ -1468,6 +1133,9 @@ circuitmux_get_first_active_circuit(circuitmux_t *cmux, circuit_t *circ = NULL; tor_assert(cmux); + tor_assert(cmux->policy); + /* This callback is mandatory. */ + tor_assert(cmux->policy->pick_active_circuit); tor_assert(destroy_queue_out); *destroy_queue_out = NULL; @@ -1486,14 +1154,7 @@ circuitmux_get_first_active_circuit(circuitmux_t *cmux, /* We also must have a cell available for this to be the case */ tor_assert(cmux->n_cells > 0); /* Do we have a policy-provided circuit selector? */ - if (cmux->policy && cmux->policy->pick_active_circuit) { - circ = cmux->policy->pick_active_circuit(cmux, cmux->policy_data); - } - /* Fall back on the head of the active circuits list */ - if (!circ) { - tor_assert(cmux->active_circuits_head); - circ = cmux->active_circuits_head; - } + circ = cmux->policy->pick_active_circuit(cmux, cmux->policy_data); cmux->last_cell_was_destroy = 0; } else { tor_assert(cmux->n_cells == 0); @@ -1517,7 +1178,6 @@ circuitmux_notify_xmit_cells(circuitmux_t *cmux, circuit_t *circ, tor_assert(cmux); tor_assert(circ); - circuitmux_assert_okay_paranoid(cmux); if (n_cells == 0) return; @@ -1544,17 +1204,11 @@ circuitmux_notify_xmit_cells(circuitmux_t *cmux, circuit_t *circ, /* Adjust the mux cell counter */ cmux->n_cells -= n_cells; - /* If we aren't making it inactive later, move it to the tail of the list */ - if (!becomes_inactive) { - circuitmux_move_active_circ_to_tail(cmux, circ, - hashent->muxinfo.direction); - } - /* * We call notify_xmit_cells() before making the circuit inactive if needed, * so the policy can always count on this coming in on an active circuit. */ - if (cmux->policy && cmux->policy->notify_xmit_cells) { + if (cmux->policy->notify_xmit_cells) { cmux->policy->notify_xmit_cells(cmux, cmux->policy_data, circ, hashent->muxinfo.policy_data, n_cells); @@ -1566,10 +1220,8 @@ circuitmux_notify_xmit_cells(circuitmux_t *cmux, circuit_t *circ, */ if (becomes_inactive) { --(cmux->n_active_circuits); - circuitmux_make_circuit_inactive(cmux, circ, hashent->muxinfo.direction); + circuitmux_make_circuit_inactive(cmux, circ); } - - circuitmux_assert_okay_paranoid(cmux); } /** @@ -1592,282 +1244,6 @@ circuitmux_notify_xmit_destroy(circuitmux_t *cmux) I64_PRINTF_ARG(global_destroy_ctr)); } -/* - * Circuitmux consistency checking assertions - */ - -/** - * Check that circuitmux data structures are consistent and fail with an - * assert if not. - */ - -void -circuitmux_assert_okay(circuitmux_t *cmux) -{ - tor_assert(cmux); - - /* - * Pass 1: iterate the hash table; for each entry: - * a) Check that the circuit has this cmux for n_mux or p_mux - * b) If the cell_count is > 0, set the mark bit; otherwise clear it - * c) Also check activeness (cell_count > 0 should be active) - * d) Count the number of circuits, active circuits and queued cells - * and at the end check that they match the counters in the cmux. - * - * Pass 2: iterate the active circuits list; for each entry, - * make sure the circuit is attached to this mux and appears - * in the hash table. Make sure the mark bit is 1, and clear - * it in the hash table entry. Consistency-check the linked - * list pointers. - * - * Pass 3: iterate the hash table again; assert if any active circuits - * (mark bit set to 1) are discovered that weren't cleared in pass 2 - * (don't appear in the linked list). - */ - - circuitmux_assert_okay_pass_one(cmux); - circuitmux_assert_okay_pass_two(cmux); - circuitmux_assert_okay_pass_three(cmux); -} - -/** - * Do the first pass of circuitmux_assert_okay(); see the comment in that - * function. - */ - -static void -circuitmux_assert_okay_pass_one(circuitmux_t *cmux) -{ - chanid_circid_muxinfo_t **i = NULL; - uint64_t chan_id; - channel_t *chan; - circid_t circ_id; - circuit_t *circ; - or_circuit_t *or_circ; - circuit_t **next_p, **prev_p; - unsigned int n_circuits, n_active_circuits, n_cells; - - tor_assert(cmux); - tor_assert(cmux->chanid_circid_map); - - /* Reset the counters */ - n_circuits = n_active_circuits = n_cells = 0; - /* Start iterating the hash table */ - i = HT_START(chanid_circid_muxinfo_map, cmux->chanid_circid_map); - while (i) { - /* Assert that the hash table entry isn't null */ - tor_assert(*i); - - /* Get the channel and circuit id */ - chan_id = (*i)->chan_id; - circ_id = (*i)->circ_id; - - /* Find the channel and circuit, assert that they exist */ - chan = channel_find_by_global_id(chan_id); - tor_assert(chan); - circ = circuit_get_by_circid_channel_even_if_marked(circ_id, chan); - tor_assert(circ); - - /* Assert that we know which direction this is going */ - tor_assert((*i)->muxinfo.direction == CELL_DIRECTION_OUT || - (*i)->muxinfo.direction == CELL_DIRECTION_IN); - - if ((*i)->muxinfo.direction == CELL_DIRECTION_OUT) { - /* We should be n_mux on this circuit */ - tor_assert(cmux == circ->n_mux); - tor_assert(chan == circ->n_chan); - /* Get next and prev for next test */ - next_p = &(circ->next_active_on_n_chan); - prev_p = &(circ->prev_active_on_n_chan); - } else { - /* This should be an or_circuit_t and we should be p_mux */ - or_circ = TO_OR_CIRCUIT(circ); - tor_assert(cmux == or_circ->p_mux); - tor_assert(chan == or_circ->p_chan); - /* Get next and prev for next test */ - next_p = &(or_circ->next_active_on_p_chan); - prev_p = &(or_circ->prev_active_on_p_chan); - } - - /* - * Should this circuit be active? I.e., does the mux know about > 0 - * cells on it? - */ - const int circ_is_active = ((*i)->muxinfo.cell_count > 0); - - /* It should be in the linked list iff it's active */ - if (circ_is_active) { - /* Either we have a next link or we are the tail */ - tor_assert(*next_p || (circ == cmux->active_circuits_tail)); - /* Either we have a prev link or we are the head */ - tor_assert(*prev_p || (circ == cmux->active_circuits_head)); - /* Increment the active circuits counter */ - ++n_active_circuits; - } else { - /* Shouldn't be in list, so no next or prev link */ - tor_assert(!(*next_p)); - tor_assert(!(*prev_p)); - /* And can't be head or tail */ - tor_assert(circ != cmux->active_circuits_head); - tor_assert(circ != cmux->active_circuits_tail); - } - - /* Increment the circuits counter */ - ++n_circuits; - /* Adjust the cell counter */ - n_cells += (*i)->muxinfo.cell_count; - - /* Set the mark bit to circ_is_active */ - (*i)->muxinfo.mark = circ_is_active; - - /* Advance to the next entry */ - i = HT_NEXT(chanid_circid_muxinfo_map, cmux->chanid_circid_map, i); - } - - /* Now check the counters */ - tor_assert(n_cells == cmux->n_cells); - tor_assert(n_circuits == cmux->n_circuits); - tor_assert(n_active_circuits == cmux->n_active_circuits); -} - -/** - * Do the second pass of circuitmux_assert_okay(); see the comment in that - * function. - */ - -static void -circuitmux_assert_okay_pass_two(circuitmux_t *cmux) -{ - circuit_t *curr_circ, *prev_circ = NULL, *next_circ; - or_circuit_t *curr_or_circ; - uint64_t curr_chan_id; - circid_t curr_circ_id; - circuit_t **next_p, **prev_p; - channel_t *chan; - unsigned int n_active_circuits = 0; - chanid_circid_muxinfo_t search, *hashent = NULL; - - tor_assert(cmux); - tor_assert(cmux->chanid_circid_map); - - /* - * Walk the linked list of active circuits in cmux; keep track of the - * previous circuit seen for consistency checking purposes. Count them - * to make sure the number in the linked list matches - * cmux->n_active_circuits. - */ - curr_circ = cmux->active_circuits_head; - while (curr_circ) { - /* Reset some things */ - chan = NULL; - curr_or_circ = NULL; - next_circ = NULL; - next_p = prev_p = NULL; - cell_direction_t direction; - - /* Figure out if this is n_mux or p_mux */ - if (cmux == curr_circ->n_mux) { - /* Get next_p and prev_p */ - next_p = &(curr_circ->next_active_on_n_chan); - prev_p = &(curr_circ->prev_active_on_n_chan); - /* Get the channel */ - chan = curr_circ->n_chan; - /* Get the circuit id */ - curr_circ_id = curr_circ->n_circ_id; - /* Remember the direction */ - direction = CELL_DIRECTION_OUT; - } else { - /* We must be p_mux and this must be an or_circuit_t */ - curr_or_circ = TO_OR_CIRCUIT(curr_circ); - tor_assert(cmux == curr_or_circ->p_mux); - /* Get next_p and prev_p */ - next_p = &(curr_or_circ->next_active_on_p_chan); - prev_p = &(curr_or_circ->prev_active_on_p_chan); - /* Get the channel */ - chan = curr_or_circ->p_chan; - /* Get the circuit id */ - curr_circ_id = curr_or_circ->p_circ_id; - /* Remember the direction */ - direction = CELL_DIRECTION_IN; - } - - /* Assert that we got a channel and get the channel ID */ - tor_assert(chan); - curr_chan_id = chan->global_identifier; - - /* Assert that prev_p points to last circuit we saw */ - tor_assert(*prev_p == prev_circ); - /* If that's NULL, assert that we are the head */ - if (!(*prev_p)) tor_assert(curr_circ == cmux->active_circuits_head); - - /* Get the next circuit */ - next_circ = *next_p; - /* If it's NULL, assert that we are the tail */ - if (!(*next_p)) tor_assert(curr_circ == cmux->active_circuits_tail); - - /* Now find the hash table entry for this circuit */ - search.chan_id = curr_chan_id; - search.circ_id = curr_circ_id; - hashent = HT_FIND(chanid_circid_muxinfo_map, cmux->chanid_circid_map, - &search); - - /* Assert that we have one */ - tor_assert(hashent); - - /* Assert that the direction matches */ - tor_assert(direction == hashent->muxinfo.direction); - - /* Assert that the hash entry got marked in pass one */ - tor_assert(hashent->muxinfo.mark); - - /* Clear the mark */ - hashent->muxinfo.mark = 0; - - /* Increment the counter */ - ++n_active_circuits; - - /* Advance to the next active circuit and update prev_circ */ - prev_circ = curr_circ; - curr_circ = next_circ; - } - - /* Assert that the counter matches the cmux */ - tor_assert(n_active_circuits == cmux->n_active_circuits); -} - -/** - * Do the third pass of circuitmux_assert_okay(); see the comment in that - * function. - */ - -static void -circuitmux_assert_okay_pass_three(circuitmux_t *cmux) -{ - chanid_circid_muxinfo_t **i = NULL; - - tor_assert(cmux); - tor_assert(cmux->chanid_circid_map); - - /* Start iterating the hash table */ - i = HT_START(chanid_circid_muxinfo_map, cmux->chanid_circid_map); - - /* Advance through each entry */ - while (i) { - /* Assert that it isn't null */ - tor_assert(*i); - - /* - * Assert that this entry is not marked - i.e., that either we didn't - * think it should be active in pass one or we saw it in the active - * circuits linked list. - */ - tor_assert(!((*i)->muxinfo.mark)); - - /* Advance to the next entry */ - i = HT_NEXT(chanid_circid_muxinfo_map, cmux->chanid_circid_map, i); - } -} - /*DOCDOC */ void circuitmux_append_destroy_cell(channel_t *chan, diff --git a/src/or/circuitmux_ewma.c b/src/or/circuitmux_ewma.c index fde2d22a89..b2ace8a9fa 100644 --- a/src/or/circuitmux_ewma.c +++ b/src/or/circuitmux_ewma.c @@ -223,8 +223,6 @@ ewma_cmp_cmux(circuitmux_t *cmux_1, circuitmux_policy_data_t *pol_data_1, * has value ewma_scale_factor ** N.) */ static double ewma_scale_factor = 0.1; -/* DOCDOC ewma_enabled */ -static int ewma_enabled = 0; /*** EWMA circuitmux_policy_t method table ***/ @@ -243,6 +241,13 @@ circuitmux_policy_t ewma_policy = { /*** EWMA method implementations using the below EWMA helper functions ***/ +/** Compute and return the current cell_ewma tick. */ +static inline unsigned int +cell_ewma_get_tick(void) +{ + return ((unsigned)approx_time() / EWMA_TICK_LEN); +} + /** * Allocate an ewma_policy_data_t and upcast it to a circuitmux_policy_data_t; * this is called when setting the policy on a circuitmux_t to ewma_policy. @@ -612,59 +617,79 @@ cell_ewma_tick_from_timeval(const struct timeval *now, return res; } -/** Tell the caller whether ewma_enabled is set */ -int -cell_ewma_enabled(void) +/* Default value for the CircuitPriorityHalflifeMsec consensus parameter in + * msec. */ +#define CMUX_PRIORITY_HALFLIFE_MSEC_DEFAULT 30000 +/* Minimum and maximum value for the CircuitPriorityHalflifeMsec consensus + * parameter. */ +#define CMUX_PRIORITY_HALFLIFE_MSEC_MIN 1 +#define CMUX_PRIORITY_HALFLIFE_MSEC_MAX INT32_MAX + +/* Return the value of the circuit priority halflife from the options if + * available or else from the consensus (in that order). If none can be found, + * a default value is returned. + * + * The source_msg points to a string describing from where the value was + * picked so it can be used for logging. */ +static double +get_circuit_priority_halflife(const or_options_t *options, + const networkstatus_t *consensus, + const char **source_msg) { - return ewma_enabled; -} + int32_t halflife_ms; + double halflife; + /* Compute the default value now. We might need it. */ + double halflife_default = + ((double) CMUX_PRIORITY_HALFLIFE_MSEC_DEFAULT) / 1000.0; -/** Compute and return the current cell_ewma tick. */ -unsigned int -cell_ewma_get_tick(void) -{ - return ((unsigned)approx_time() / EWMA_TICK_LEN); + /* Try to get it from configuration file first. */ + if (options && options->CircuitPriorityHalflife < EPSILON) { + halflife = options->CircuitPriorityHalflife; + *source_msg = "CircuitPriorityHalflife in configuration"; + goto end; + } + + /* Try to get the msec value from the consensus. */ + halflife_ms = networkstatus_get_param(consensus, + "CircuitPriorityHalflifeMsec", + CMUX_PRIORITY_HALFLIFE_MSEC_DEFAULT, + CMUX_PRIORITY_HALFLIFE_MSEC_MIN, + CMUX_PRIORITY_HALFLIFE_MSEC_MAX); + halflife = ((double) halflife_ms) / 1000.0; + *source_msg = "CircuitPriorityHalflifeMsec in consensus"; + + end: + /* We should never go below the EPSILON else we would consider it disabled + * and we can't have that. */ + if (halflife < EPSILON) { + log_warn(LD_CONFIG, "CircuitPriorityHalflife is too small (%f). " + "Adjusting to the smallest value allowed: %f.", + halflife, halflife_default); + halflife = halflife_default; + } + return halflife; } /** Adjust the global cell scale factor based on <b>options</b> */ void -cell_ewma_set_scale_factor(const or_options_t *options, - const networkstatus_t *consensus) +cmux_ewma_set_options(const or_options_t *options, + const networkstatus_t *consensus) { - int32_t halflife_ms; double halflife; const char *source; - if (options && options->CircuitPriorityHalflife >= -EPSILON) { - halflife = options->CircuitPriorityHalflife; - source = "CircuitPriorityHalflife in configuration"; - } else if (consensus && (halflife_ms = networkstatus_get_param( - consensus, "CircuitPriorityHalflifeMsec", - -1, -1, INT32_MAX)) >= 0) { - halflife = ((double)halflife_ms)/1000.0; - source = "CircuitPriorityHalflifeMsec in consensus"; - } else { - halflife = EWMA_DEFAULT_HALFLIFE; - source = "Default value"; - } - if (halflife <= EPSILON) { - /* The cell EWMA algorithm is disabled. */ - ewma_scale_factor = 0.1; - ewma_enabled = 0; - log_info(LD_OR, - "Disabled cell_ewma algorithm because of value in %s", - source); - } else { - /* convert halflife into halflife-per-tick. */ - halflife /= EWMA_TICK_LEN; - /* compute per-tick scale factor. */ - ewma_scale_factor = exp( LOG_ONEHALF / halflife ); - ewma_enabled = 1; - log_info(LD_OR, - "Enabled cell_ewma algorithm because of value in %s; " - "scale factor is %f per %d seconds", - source, ewma_scale_factor, EWMA_TICK_LEN); - } + /* Both options and consensus can be NULL. This assures us to either get a + * valid configured value or the default one. */ + halflife = get_circuit_priority_halflife(options, consensus, &source); + + /* convert halflife into halflife-per-tick. */ + halflife /= EWMA_TICK_LEN; + /* compute per-tick scale factor. */ + ewma_scale_factor = exp( LOG_ONEHALF / halflife ); + log_info(LD_OR, + "Enabled cell_ewma algorithm because of value in %s; " + "scale factor is %f per %d seconds", + source, ewma_scale_factor, EWMA_TICK_LEN); } /** Return the multiplier necessary to convert the value of a cell sent in diff --git a/src/or/circuitmux_ewma.h b/src/or/circuitmux_ewma.h index 8f4e57865e..2ef8c2586d 100644 --- a/src/or/circuitmux_ewma.h +++ b/src/or/circuitmux_ewma.h @@ -12,13 +12,12 @@ #include "or.h" #include "circuitmux.h" +/* The public EWMA policy callbacks object. */ extern circuitmux_policy_t ewma_policy; /* Externally visible EWMA functions */ -int cell_ewma_enabled(void); -unsigned int cell_ewma_get_tick(void); -void cell_ewma_set_scale_factor(const or_options_t *options, - const networkstatus_t *consensus); +void cmux_ewma_set_options(const or_options_t *options, + const networkstatus_t *consensus); #endif /* !defined(TOR_CIRCUITMUX_EWMA_H) */ diff --git a/src/or/circuituse.c b/src/or/circuituse.c index 3125fff650..fe28dd9a1a 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -55,7 +55,6 @@ #include "rephist.h" #include "router.h" #include "routerlist.h" -#include "config.h" static void circuit_expire_old_circuits_clientside(void); static void circuit_increment_failure_count(void); @@ -1632,7 +1631,7 @@ circuit_testing_opened(origin_circuit_t *circ) router_perform_bandwidth_test(NUM_PARALLEL_TESTING_CIRCS, time(NULL)); have_performed_bandwidth_test = 1; } else - consider_testing_reachability(1, 0); + router_do_reachability_checks(1, 0); } /** A testing circuit has failed to build. Take whatever stats we want. */ @@ -2584,7 +2583,7 @@ link_apconn_to_circ(entry_connection_t *apconn, origin_circuit_t *circ, log_debug(LD_APP|LD_CIRC, "attaching new conn to circ. n_circ_id %u.", (unsigned)circ->base_.n_circ_id); /* reset it, so we can measure circ timeouts */ - ENTRY_TO_CONN(apconn)->timestamp_lastread = time(NULL); + ENTRY_TO_CONN(apconn)->timestamp_last_read_allowed = time(NULL); ENTRY_TO_EDGE_CONN(apconn)->next_stream = circ->p_streams; ENTRY_TO_EDGE_CONN(apconn)->on_circuit = TO_CIRCUIT(circ); /* assert_connection_ok(conn, time(NULL)); */ diff --git a/src/or/command.c b/src/or/command.c index 7280be1396..4f99462f38 100644 --- a/src/or/command.c +++ b/src/or/command.c @@ -339,7 +339,9 @@ command_process_create_cell(cell_t *cell, channel_t *chan) return; } - if (connection_or_digest_is_known_relay(chan->identity_digest)) { + if (!channel_is_client(chan)) { + /* remember create types we've seen, but don't remember them from + * clients, to be extra conservative about client statistics. */ rep_hist_note_circuit_handshake_requested(create_cell->handshake_type); } diff --git a/src/or/config.c b/src/or/config.c index 6b8885521a..1eabf49f7b 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -267,7 +267,7 @@ static config_var_t option_vars_[] = { OBSOLETE("CircuitIdleTimeout"), V(CircuitsAvailableTimeout, INTERVAL, "0"), V(CircuitStreamTimeout, INTERVAL, "0"), - V(CircuitPriorityHalflife, DOUBLE, "-100.0"), /*negative:'Use default'*/ + V(CircuitPriorityHalflife, DOUBLE, "-1.0"), /*negative:'Use default'*/ V(ClientDNSRejectInternalAddresses, BOOL,"1"), V(ClientOnly, BOOL, "0"), V(ClientPreferIPv6ORPort, AUTOBOOL, "auto"), @@ -495,8 +495,8 @@ static config_var_t option_vars_[] = { V(TestingSigningKeySlop, INTERVAL, "1 day"), V(OptimisticData, AUTOBOOL, "auto"), - V(PortForwarding, BOOL, "0"), - V(PortForwardingHelper, FILENAME, "tor-fw-helper"), + OBSOLETE("PortForwarding"), + OBSOLETE("PortForwardingHelper"), OBSOLETE("PreferTunneledDirConns"), V(ProtocolWarnings, BOOL, "0"), V(PublishServerDescriptor, CSV, "1"), @@ -1651,8 +1651,7 @@ options_act_reversible(const or_options_t *old_options, char **msg) int options_need_geoip_info(const or_options_t *options, const char **reason_out) { - int bridge_usage = - options->BridgeRelay && options->BridgeRecordUsageByCountry; + int bridge_usage = should_record_bridge_info(options); int routerset_usage = routerset_needs_geoip(options->EntryNodes) || routerset_needs_geoip(options->ExitNodes) || @@ -1760,7 +1759,6 @@ options_act(const or_options_t *old_options) char *msg=NULL; const int transition_affects_workers = old_options && options_transition_affects_workers(old_options, options); - int old_ewma_enabled; const int transition_affects_guards = old_options && options_transition_affects_guards(old_options, options); @@ -2034,16 +2032,8 @@ options_act(const or_options_t *old_options) if (accounting_is_enabled(options)) configure_accounting(time(NULL)); - old_ewma_enabled = cell_ewma_enabled(); /* Change the cell EWMA settings */ - cell_ewma_set_scale_factor(options, networkstatus_get_latest_consensus()); - /* If we just enabled ewma, set the cmux policy on all active channels */ - if (cell_ewma_enabled() && !old_ewma_enabled) { - channel_set_cmux_policy_everywhere(&ewma_policy); - } else if (!cell_ewma_enabled() && old_ewma_enabled) { - /* Turn it off everywhere */ - channel_set_cmux_policy_everywhere(NULL); - } + cmux_ewma_set_options(options, networkstatus_get_latest_consensus()); /* Update the BridgePassword's hashed version as needed. We store this as a * digest so that we can do side-channel-proof comparisons on it. @@ -2248,6 +2238,11 @@ options_act(const or_options_t *old_options) } if ((!old_options || !old_options->EntryStatistics) && options->EntryStatistics && !should_record_bridge_info(options)) { + /* If we get here, we've started recording bridge info when we didn't + * do so before. Note that "should_record_bridge_info()" will + * always be false at this point, because of the earlier block + * that cleared EntryStatistics when public_server_mode() was false. + * We're leaving it in as defensive programming. */ if (geoip_is_loaded(AF_INET) || geoip_is_loaded(AF_INET6)) { geoip_entry_stats_init(now); print_notice = 1; @@ -3885,15 +3880,6 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options->KeepalivePeriod < 1) REJECT("KeepalivePeriod option must be positive."); - if (options->PortForwarding && options->Sandbox) { - REJECT("PortForwarding is not compatible with Sandbox; at most one can " - "be set"); - } - if (options->PortForwarding && options->NoExec) { - COMPLAIN("Both PortForwarding and NoExec are set; PortForwarding will " - "be ignored."); - } - if (ensure_bandwidth_cap(&options->BandwidthRate, "BandwidthRate", msg) < 0) return -1; @@ -4614,15 +4600,14 @@ have_enough_mem_for_dircache(const or_options_t *options, size_t total_mem, if (options->DirCache) { if (total_mem < DIRCACHE_MIN_MEM_BYTES) { if (options->BridgeRelay) { - *msg = tor_strdup("Running a Bridge with less than " - STRINGIFY(DIRCACHE_MIN_MEM_MB) " MB of memory is not " - "recommended."); + tor_asprintf(msg, "Running a Bridge with less than %d MB of memory " + "is not recommended.", DIRCACHE_MIN_MEM_MB); } else { - *msg = tor_strdup("Being a directory cache (default) with less than " - STRINGIFY(DIRCACHE_MIN_MEM_MB) " MB of memory is not " - "recommended and may consume most of the available " - "resources, consider disabling this functionality by " - "setting the DirCache option to 0."); + tor_asprintf(msg, "Being a directory cache (default) with less than " + "%d MB of memory is not recommended and may consume " + "most of the available resources. Consider disabling " + "this functionality by setting the DirCache option " + "to 0.", DIRCACHE_MIN_MEM_MB); } } } else { diff --git a/src/or/connection.c b/src/or/connection.c index 2a6b10763e..5532551cfe 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -101,7 +101,6 @@ #include "transports.h" #include "routerparse.h" #include "sandbox.h" -#include "transports.h" #ifdef HAVE_PWD_H #include <pwd.h> @@ -460,8 +459,8 @@ connection_init(time_t now, connection_t *conn, int type, int socket_family) } conn->timestamp_created = now; - conn->timestamp_lastread = now; - conn->timestamp_lastwritten = now; + conn->timestamp_last_read_allowed = now; + conn->timestamp_last_write_allowed = now; } /** Create a link between <b>conn_a</b> and <b>conn_b</b>. */ @@ -859,7 +858,7 @@ connection_mark_for_close_internal_, (connection_t *conn, /* in case we're going to be held-open-til-flushed, reset * the number of seconds since last successful write, so * we get our whole 15 seconds */ - conn->timestamp_lastwritten = time(NULL); + conn->timestamp_last_write_allowed = time(NULL); } /** Find each connection that has hold_open_until_flushed set to @@ -881,7 +880,7 @@ connection_expire_held_open(void) */ if (conn->hold_open_until_flushed) { tor_assert(conn->marked_for_close); - if (now - conn->timestamp_lastwritten >= 15) { + if (now - conn->timestamp_last_write_allowed >= 15) { int severity; if (conn->type == CONN_TYPE_EXIT || (conn->type == CONN_TYPE_DIR && @@ -1259,15 +1258,12 @@ connection_listener_new(const struct sockaddr *listensockaddr, gotPort = usePort; } else { tor_addr_t addr2; - struct sockaddr_storage ss; - socklen_t ss_len=sizeof(ss); - if (getsockname(s, (struct sockaddr*)&ss, &ss_len)<0) { + if (tor_addr_from_getsockname(&addr2, s)<0) { log_warn(LD_NET, "getsockname() couldn't learn address for %s: %s", conn_type_to_string(type), tor_socket_strerror(tor_socket_errno(s))); gotPort = 0; } - tor_addr_from_sockaddr(&addr2, (struct sockaddr*)&ss, &gotPort); } #ifdef HAVE_SYS_UN_H /* @@ -3418,7 +3414,7 @@ connection_handle_read_impl(connection_t *conn) if (conn->marked_for_close) return 0; /* do nothing */ - conn->timestamp_lastread = approx_time(); + conn->timestamp_last_read_allowed = approx_time(); switch (conn->type) { case CONN_TYPE_OR_LISTENER: @@ -3819,7 +3815,7 @@ update_send_buffer_size(tor_socket_t sock) * when libevent tells us that conn wants to write, or below * from connection_buf_add() when an entire TLS record is ready. * - * Update <b>conn</b>-\>timestamp_lastwritten to now, and call flush_buf + * Update <b>conn</b>-\>timestamp_last_write_allowed to now, and call flush_buf * or flush_buf_tls appropriately. If it succeeds and there are no more * more bytes on <b>conn</b>-\>outbuf, then call connection_finished_flushing * on it too. @@ -3852,7 +3848,7 @@ connection_handle_write_impl(connection_t *conn, int force) return 0; } - conn->timestamp_lastwritten = now; + conn->timestamp_last_write_allowed = now; /* Sometimes, "writable" means "connected". */ if (connection_state_is_connecting(conn)) { @@ -4528,8 +4524,6 @@ alloc_http_authenticator(const char *authenticator) static void client_check_address_changed(tor_socket_t sock) { - struct sockaddr_storage out_sockaddr; - socklen_t out_addr_len = (socklen_t) sizeof(out_sockaddr); tor_addr_t out_addr, iface_addr; tor_addr_t **last_interface_ip_ptr; sa_family_t family; @@ -4537,13 +4531,12 @@ client_check_address_changed(tor_socket_t sock) if (!outgoing_addrs) outgoing_addrs = smartlist_new(); - if (getsockname(sock, (struct sockaddr*)&out_sockaddr, &out_addr_len)<0) { + if (tor_addr_from_getsockname(&out_addr, sock) < 0) { int e = tor_socket_errno(sock); log_warn(LD_NET, "getsockname() to check for address change failed: %s", tor_socket_strerror(e)); return; } - tor_addr_from_sockaddr(&out_addr, (struct sockaddr*)&out_sockaddr, NULL); family = tor_addr_family(&out_addr); if (family == AF_INET) diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index a47f044e08..948c8722bf 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -739,7 +739,7 @@ connection_ap_expire_beginning(void) /* if it's an internal linked connection, don't yell its status. */ severity = (tor_addr_is_null(&base_conn->addr) && !base_conn->port) ? LOG_INFO : LOG_NOTICE; - seconds_idle = (int)( now - base_conn->timestamp_lastread ); + seconds_idle = (int)( now - base_conn->timestamp_last_read_allowed ); seconds_since_born = (int)( now - base_conn->timestamp_created ); if (base_conn->state == AP_CONN_STATE_OPEN) @@ -825,7 +825,7 @@ connection_ap_expire_beginning(void) mark_circuit_unusable_for_new_conns(TO_ORIGIN_CIRCUIT(circ)); /* give our stream another 'cutoff' seconds to try */ - conn->base_.timestamp_lastread += cutoff; + conn->base_.timestamp_last_read_allowed += cutoff; if (entry_conn->num_socks_retries < 250) /* avoid overflow */ entry_conn->num_socks_retries++; /* move it back into 'pending' state, and try to attach. */ @@ -1135,7 +1135,7 @@ connection_ap_detach_retriable(entry_connection_t *conn, int reason) { control_event_stream_status(conn, STREAM_EVENT_FAILED_RETRIABLE, reason); - ENTRY_TO_CONN(conn)->timestamp_lastread = time(NULL); + ENTRY_TO_CONN(conn)->timestamp_last_read_allowed = time(NULL); /* Roll back path bias use state so that we probe the circuit * if nothing else succeeds on it */ diff --git a/src/or/control.c b/src/or/control.c index fa62e9dbde..5a2fae64e7 100644 --- a/src/or/control.c +++ b/src/or/control.c @@ -83,8 +83,6 @@ #include <sys/resource.h> #endif -#include <event2/event.h> - #include "crypto_s2k.h" #include "procmon.h" @@ -216,7 +214,7 @@ static void orconn_target_get_name(char *buf, size_t len, static int get_cached_network_liveness(void); static void set_cached_network_liveness(int liveness); -static void flush_queued_events_cb(evutil_socket_t fd, short what, void *arg); +static void flush_queued_events_cb(mainloop_event_t *event, void *arg); static char * download_status_to_string(const download_status_t *dl); @@ -691,7 +689,7 @@ static tor_mutex_t *queued_control_events_lock = NULL; /** An event that should fire in order to flush the contents of * queued_control_events. */ -static struct event *flush_queued_events_event = NULL; +static mainloop_event_t *flush_queued_events_event = NULL; void control_initialize_event_queue(void) @@ -703,9 +701,8 @@ control_initialize_event_queue(void) if (flush_queued_events_event == NULL) { struct event_base *b = tor_libevent_get_base(); if (b) { - flush_queued_events_event = tor_event_new(b, - -1, 0, flush_queued_events_cb, - NULL); + flush_queued_events_event = + mainloop_event_new(flush_queued_events_cb, NULL); tor_assert(flush_queued_events_event); } } @@ -781,7 +778,7 @@ queue_control_event_string,(uint16_t event, char *msg)) */ if (activate_event) { tor_assert(flush_queued_events_event); - event_active(flush_queued_events_event, EV_READ, 1); + mainloop_event_activate(flush_queued_events_event); } } @@ -863,10 +860,9 @@ queued_events_flush_all(int force) /** Libevent callback: Flushes pending events to controllers that are * interested in them. */ static void -flush_queued_events_cb(evutil_socket_t fd, short what, void *arg) +flush_queued_events_cb(mainloop_event_t *event, void *arg) { - (void) fd; - (void) what; + (void) event; (void) arg; queued_events_flush_all(0); } @@ -7586,22 +7582,38 @@ control_event_hs_descriptor_upload_failed(const char *id_digest, void control_free_all(void) { + smartlist_t *queued_events = NULL; + if (authentication_cookie) /* Free the auth cookie */ tor_free(authentication_cookie); if (detached_onion_services) { /* Free the detached onion services */ SMARTLIST_FOREACH(detached_onion_services, char *, cp, tor_free(cp)); smartlist_free(detached_onion_services); } - if (queued_control_events) { - SMARTLIST_FOREACH(queued_control_events, queued_event_t *, ev, - queued_event_free(ev)); - smartlist_free(queued_control_events); + + if (queued_control_events_lock) { + tor_mutex_acquire(queued_control_events_lock); + flush_queued_event_pending = 0; + queued_events = queued_control_events; queued_control_events = NULL; + tor_mutex_release(queued_control_events_lock); + } + if (queued_events) { + SMARTLIST_FOREACH(queued_events, queued_event_t *, ev, + queued_event_free(ev)); + smartlist_free(queued_events); } if (flush_queued_events_event) { - tor_event_free(flush_queued_events_event); + mainloop_event_free(flush_queued_events_event); flush_queued_events_event = NULL; } + bootstrap_percent = BOOTSTRAP_STATUS_UNDEF; + notice_bootstrap_percent = 0; + bootstrap_problems = 0; + authentication_cookie_is_set = 0; + global_event_mask = 0; + disable_log_messages = 0; + memset(last_sent_bootstrap_message, 0, sizeof(last_sent_bootstrap_message)); } #ifdef TOR_UNIT_TESTS diff --git a/src/or/cpuworker.c b/src/or/cpuworker.c index 50761dd4d3..083691c4f6 100644 --- a/src/or/cpuworker.c +++ b/src/or/cpuworker.c @@ -30,8 +30,6 @@ #include "router.h" #include "workqueue.h" -#include <event2/event.h> - static void queue_pending_tasks(void); typedef struct worker_state_s { @@ -69,22 +67,12 @@ worker_state_free_void(void *arg) static replyqueue_t *replyqueue = NULL; static threadpool_t *threadpool = NULL; -static struct event *reply_event = NULL; static tor_weak_rng_t request_sample_rng = TOR_WEAK_RNG_INIT; static int total_pending_tasks = 0; static int max_pending_tasks = 128; -static void -replyqueue_process_cb(evutil_socket_t sock, short events, void *arg) -{ - replyqueue_t *rq = arg; - (void) sock; - (void) events; - replyqueue_process(rq); -} - /** Initialize the cpuworker subsystem. It is OK to call this more than once * during Tor's lifetime. */ @@ -94,14 +82,6 @@ cpu_init(void) if (!replyqueue) { replyqueue = replyqueue_new(0); } - if (!reply_event) { - reply_event = tor_event_new(tor_libevent_get_base(), - replyqueue_get_socket(replyqueue), - EV_READ|EV_PERSIST, - replyqueue_process_cb, - replyqueue); - event_add(reply_event, NULL); - } if (!threadpool) { /* In our threadpool implementation, half the threads are permissive and @@ -115,7 +95,12 @@ cpu_init(void) worker_state_new, worker_state_free_void, NULL); + + int r = threadpool_register_reply_event(threadpool, NULL); + + tor_assert(r == 0); } + /* Total voodoo. Can we make this more sensible? */ max_pending_tasks = get_num_cpus(get_options()) * 64; crypto_seed_weak_rng(&request_sample_rng); @@ -547,7 +532,7 @@ assign_onionskin_to_cpuworker(or_circuit_t *circ, return 0; } - if (connection_or_digest_is_known_relay(circ->p_chan->identity_digest)) + if (!channel_is_client(circ->p_chan)) rep_hist_note_circuit_handshake_assigned(onionskin->handshake_type); should_time = should_time_request(onionskin->handshake_type); diff --git a/src/or/directory.c b/src/or/directory.c index 29d091af38..c419b61d02 100644 --- a/src/or/directory.c +++ b/src/or/directory.c @@ -2437,7 +2437,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) * and the date header. (We used to check now-date_header, but that's * inaccurate if we spend a lot of time downloading.) */ - apparent_skew = conn->base_.timestamp_lastwritten - date_header; + apparent_skew = conn->base_.timestamp_last_write_allowed - date_header; if (labs(apparent_skew)>ALLOW_DIRECTORY_TIME_SKEW) { int trusted = router_digest_is_trusted_dir(conn->identity_digest); clock_skew_warning(TO_CONN(conn), apparent_skew, trusted, LD_HTTP, diff --git a/src/or/dirserv.c b/src/or/dirserv.c index f0333e288f..68727f0718 100644 --- a/src/or/dirserv.c +++ b/src/or/dirserv.c @@ -858,13 +858,13 @@ directory_remove_invalid(void) SMARTLIST_FOREACH_BEGIN(nodes, node_t *, node) { const char *msg = NULL; + const char *description; routerinfo_t *ent = node->ri; - char description[NODE_DESC_BUF_LEN]; uint32_t r; if (!ent) continue; r = dirserv_router_get_status(ent, &msg, LOG_INFO); - router_get_description(description, ent); + description = router_describe(ent); if (r & FP_REJECT) { log_info(LD_DIRSERV, "Router %s is now rejected: %s", description, msg?msg:""); @@ -1423,7 +1423,7 @@ dirserv_thinks_router_is_hs_dir(const routerinfo_t *router, * tests aren't instant. If we haven't been running long enough, * trust the relay. */ - if (stats_n_seconds_working > + if (get_uptime() > get_options()->MinUptimeHidServDirectoryV2 * 1.1) uptime = MIN(rep_hist_get_uptime(router->cache_info.identity_digest, now), real_uptime(router, now)); @@ -3393,7 +3393,8 @@ dirserv_single_reachability_test(time_t now, routerinfo_t *router) tor_assert(node); if (options->AuthDirTestEd25519LinkKeys && - node_supports_ed25519_link_authentication(node, 1)) { + node_supports_ed25519_link_authentication(node, 1) && + router->cache_info.signing_key_cert) { ed_id_key = &router->cache_info.signing_key_cert->signing_key; } else { ed_id_key = NULL; diff --git a/src/or/entrynodes.c b/src/or/entrynodes.c index 2b6ff38c9c..88d1b94deb 100644 --- a/src/or/entrynodes.c +++ b/src/or/entrynodes.c @@ -2334,7 +2334,7 @@ entry_guard_cancel(circuit_guard_state_t **guard_state_p) } /** - * Called by the circuit building module when a circuit has succeeded: + * Called by the circuit building module when a circuit has failed: * informs the guards code that the guard in *<b>guard_state_p</b> is * not working, and advances the state of the guard module. */ diff --git a/src/or/geoip.c b/src/or/geoip.c index 0ff1c6ce0d..2c917c564d 100644 --- a/src/or/geoip.c +++ b/src/or/geoip.c @@ -628,8 +628,7 @@ geoip_note_client_seen(geoip_client_action_t action, /* Only remember statistics if the DoS mitigation subsystem is enabled. If * not, only if as entry guard or as bridge. */ if (!dos_enabled()) { - if (!options->EntryStatistics && - (!(options->BridgeRelay && options->BridgeRecordUsageByCountry))) { + if (!options->EntryStatistics && !should_record_bridge_info(options)) { return; } } @@ -1881,5 +1880,8 @@ geoip_free_all(void) clear_geoip_db(); tor_free(bridge_stats_extrainfo); + + memset(geoip_digest, 0, sizeof(geoip_digest)); + memset(geoip6_digest, 0, sizeof(geoip6_digest)); } diff --git a/src/or/hibernate.c b/src/or/hibernate.c index 4dc35f68d0..7261cf8002 100644 --- a/src/or/hibernate.c +++ b/src/or/hibernate.c @@ -866,7 +866,7 @@ hibernate_end(hibernate_state_t new_state) hibernate_state = new_state; hibernate_end_time = 0; /* no longer hibernating */ - stats_n_seconds_working = 0; /* reset published uptime */ + reset_uptime(); /* reset published uptime */ } /** A wrapper around hibernate_begin, for when we get SIGINT. */ diff --git a/src/or/hs_cell.c b/src/or/hs_cell.c index 5244cfa3dd..ad92521d34 100644 --- a/src/or/hs_cell.c +++ b/src/or/hs_cell.c @@ -369,7 +369,7 @@ introduce1_encrypt_and_encode(trn_cell_introduce1_t *cell, crypto_cipher_free(cipher); offset += encoded_enc_cell_len; /* Compute MAC from the above and put it in the buffer. This function will - * make the adjustment to the encryptled_len to ommit the MAC length. */ + * make the adjustment to the encrypted_len to omit the MAC length. */ compute_introduce_mac(encoded_cell, encoded_cell_len, encrypted, encrypted_len, keys.mac_key, sizeof(keys.mac_key), diff --git a/src/or/hs_client.c b/src/or/hs_client.c index d3978f22f0..20963cd453 100644 --- a/src/or/hs_client.c +++ b/src/or/hs_client.c @@ -17,7 +17,6 @@ #include "hs_descriptor.h" #include "hs_cache.h" #include "hs_cell.h" -#include "hs_ident.h" #include "config.h" #include "directory.h" #include "hs_client.h" @@ -29,7 +28,6 @@ #include "connection.h" #include "nodelist.h" #include "circpathbias.h" -#include "connection.h" #include "hs_ntor.h" #include "circuitbuild.h" #include "networkstatus.h" @@ -1439,8 +1437,8 @@ hs_client_desc_has_arrived(const hs_ident_dir_conn_t *ident) * connection is considered "fresh" and can continue without being closed * too early. */ base_conn->timestamp_created = now; - base_conn->timestamp_lastread = now; - base_conn->timestamp_lastwritten = now; + base_conn->timestamp_last_read_allowed = now; + base_conn->timestamp_last_write_allowed = now; /* Change connection's state into waiting for a circuit. */ base_conn->state = AP_CONN_STATE_CIRCUIT_WAIT; diff --git a/src/or/hs_common.c b/src/or/hs_common.c index 6d97c8775c..aa34b0e8fb 100644 --- a/src/or/hs_common.c +++ b/src/or/hs_common.c @@ -28,7 +28,6 @@ #include "rendservice.h" #include "routerset.h" #include "router.h" -#include "routerset.h" #include "shared_random.h" #include "shared_random_state.h" diff --git a/src/or/hs_intropoint.c b/src/or/hs_intropoint.c index 8c6453e6fd..3274e8e9c0 100644 --- a/src/or/hs_intropoint.c +++ b/src/or/hs_intropoint.c @@ -12,7 +12,6 @@ #include "config.h" #include "circuitlist.h" #include "circuituse.h" -#include "config.h" #include "relay.h" #include "rendmid.h" #include "rephist.h" diff --git a/src/or/hs_service.c b/src/or/hs_service.c index 6fa9ec6b16..ba8abc4237 100644 --- a/src/or/hs_service.c +++ b/src/or/hs_service.c @@ -31,7 +31,6 @@ #include "hs_common.h" #include "hs_config.h" #include "hs_control.h" -#include "hs_circuit.h" #include "hs_descriptor.h" #include "hs_ident.h" #include "hs_intropoint.h" diff --git a/src/or/include.am b/src/or/include.am index c1e23dd3d9..9a68df5c3e 100644 --- a/src/or/include.am +++ b/src/or/include.am @@ -91,6 +91,7 @@ LIBTOR_A_SOURCES = \ src/or/policies.c \ src/or/reasons.c \ src/or/relay.c \ + src/or/relay_crypto.c \ src/or/rendcache.c \ src/or/rendclient.c \ src/or/rendcommon.c \ @@ -238,6 +239,7 @@ ORHEADERS = \ src/or/proto_socks.h \ src/or/reasons.h \ src/or/relay.h \ + src/or/relay_crypto.h \ src/or/rendcache.h \ src/or/rendclient.h \ src/or/rendcommon.h \ diff --git a/src/or/keypin.c b/src/or/keypin.c index 1698dc184f..97e16c1f78 100644 --- a/src/or/keypin.c +++ b/src/or/keypin.c @@ -12,7 +12,7 @@ #include "orconfig.h" #include "compat.h" -#include "crypto.h" +#include "crypto_digest.h" #include "crypto_format.h" #include "di_ops.h" #include "ht.h" @@ -289,8 +289,10 @@ static int keypin_journal_fd = -1; int keypin_open_journal(const char *fname) { - /* O_SYNC ??*/ - int fd = tor_open_cloexec(fname, O_WRONLY|O_CREAT|O_BINARY, 0600); +#ifndef O_SYNC +#define O_SYNC 0 +#endif + int fd = tor_open_cloexec(fname, O_WRONLY|O_CREAT|O_BINARY|O_SYNC, 0600); if (fd < 0) goto err; @@ -417,10 +419,11 @@ keypin_load_journal_impl(const char *data, size_t size) ++n_entries; } - int severity = (n_corrupt_lines || n_duplicates) ? LOG_WARN : LOG_INFO; + int severity = (n_corrupt_lines || n_duplicates) ? LOG_NOTICE : LOG_INFO; tor_log(severity, LD_DIRSERV, "Loaded %d entries from keypin journal. " - "Found %d corrupt lines, %d duplicates, and %d conflicts.", + "Found %d corrupt lines (ignored), %d duplicates (harmless), " + "and %d conflicts (resolved in favor or more recent entry).", n_entries, n_corrupt_lines, n_duplicates, n_conflicts); return 0; diff --git a/src/or/main.c b/src/or/main.c index d1f0044095..4cc51de07b 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -133,7 +133,7 @@ void evdns_shutdown(int); #ifdef HAVE_RUST // helper function defined in Rust to output a log message indicating if tor is // running with Rust enabled. See src/rust/tor_util -char *rust_welcome_string(void); +void rust_log_welcome_string(void); #endif /********* PROTOTYPES **********/ @@ -179,7 +179,7 @@ static uint64_t stats_n_bytes_written = 0; /** What time did this process start up? */ time_t time_of_process_start = 0; /** How many seconds have we been running? */ -long stats_n_seconds_working = 0; +static long stats_n_seconds_working = 0; /** How many times have we returned from the main loop successfully? */ static uint64_t stats_n_main_loop_successes = 0; /** How many times have we received an error from the main loop? */ @@ -719,7 +719,7 @@ tell_event_loop_to_run_external_code(void) { if (!called_loop_once) { struct timeval tv = { 0, 0 }; - tor_event_base_loopexit(tor_libevent_get_base(), &tv); + tor_libevent_exit_loop_after_delay(tor_libevent_get_base(), &tv); called_loop_once = 1; /* hack to avoid adding more exit events */ } } @@ -779,8 +779,9 @@ tor_shutdown_event_loop_and_exit(int exitcode) shutdown_did_not_work_callback, NULL); event_add(shutdown_did_not_work_event, &ten_seconds); - /* Unlike loopexit, loopbreak prevents other callbacks from running. */ - tor_event_base_loopbreak(tor_libevent_get_base()); + /* Unlike exit_loop_after_delay(), exit_loop_after_callback + * prevents other callbacks from running. */ + tor_libevent_exit_loop_after_callback(tor_libevent_get_base()); } /** Return true iff tor_shutdown_event_loop_and_exit() has been called. */ @@ -1008,7 +1009,8 @@ conn_close_if_marked(int i) LOG_FN_CONN(conn, (LOG_INFO,LD_NET, "Holding conn (fd %d) open for more flushing.", (int)conn->s)); - conn->timestamp_lastwritten = now; /* reset so we can flush more */ + conn->timestamp_last_write_allowed = now; /* reset so we can flush + * more */ } else if (sz == 0) { /* Also, retval==0. If we get here, we didn't want to write anything * (because of rate-limiting) and we didn't. */ @@ -1059,9 +1061,8 @@ conn_close_if_marked(int i) * reason. */ static void -directory_all_unreachable_cb(evutil_socket_t fd, short event, void *arg) +directory_all_unreachable_cb(mainloop_event_t *event, void *arg) { - (void)fd; (void)event; (void)arg; @@ -1081,7 +1082,7 @@ directory_all_unreachable_cb(evutil_socket_t fd, short event, void *arg) control_event_general_error("DIR_ALL_UNREACHABLE"); } -static struct event *directory_all_unreachable_cb_event = NULL; +static mainloop_event_t *directory_all_unreachable_cb_event = NULL; /** We've just tried every dirserver we know about, and none of * them were reachable. Assume the network is down. Change state @@ -1094,16 +1095,15 @@ directory_all_unreachable(time_t now) { (void)now; - stats_n_seconds_working=0; /* reset it */ + reset_uptime(); /* reset it */ if (!directory_all_unreachable_cb_event) { directory_all_unreachable_cb_event = - tor_event_new(tor_libevent_get_base(), - -1, EV_READ, directory_all_unreachable_cb, NULL); + mainloop_event_new(directory_all_unreachable_cb, NULL); tor_assert(directory_all_unreachable_cb_event); } - event_active(directory_all_unreachable_cb_event, EV_READ, 1); + mainloop_event_activate(directory_all_unreachable_cb_event); } /** This function is called whenever we successfully pull down some new @@ -1143,7 +1143,7 @@ directory_info_has_arrived(time_t now, int from_cache, int suppress_logs) if (server_mode(options) && !net_is_disabled() && !from_cache && (have_completed_a_circuit() || !any_predicted_circuits(now))) - consider_testing_reachability(1, 1); + router_do_reachability_checks(1, 1); } /** Perform regular maintenance tasks for a single connection. This @@ -1159,7 +1159,7 @@ run_connection_housekeeping(int i, time_t now) channel_t *chan = NULL; int have_any_circuits; int past_keepalive = - now >= conn->timestamp_lastwritten + options->KeepalivePeriod; + now >= conn->timestamp_last_write_allowed + options->KeepalivePeriod; if (conn->outbuf && !connection_get_outbuf_len(conn) && conn->type == CONN_TYPE_OR) @@ -1174,10 +1174,10 @@ run_connection_housekeeping(int i, time_t now) * if a server or received if a client) for 5 min */ if (conn->type == CONN_TYPE_DIR && ((DIR_CONN_IS_SERVER(conn) && - conn->timestamp_lastwritten + conn->timestamp_last_write_allowed + options->TestingDirConnectionMaxStall < now) || (!DIR_CONN_IS_SERVER(conn) && - conn->timestamp_lastread + conn->timestamp_last_read_allowed + options->TestingDirConnectionMaxStall < now))) { log_info(LD_DIR,"Expiring wedged directory conn (fd %d, purpose %d)", (int)conn->s, conn->purpose); @@ -1253,13 +1253,14 @@ run_connection_housekeeping(int i, time_t now) connection_or_close_normally(TO_OR_CONN(conn), 0); } else if ( now >= or_conn->timestamp_lastempty + options->KeepalivePeriod*10 && - now >= conn->timestamp_lastwritten + options->KeepalivePeriod*10) { + now >= + conn->timestamp_last_write_allowed + options->KeepalivePeriod*10) { log_fn(LOG_PROTOCOL_WARN,LD_PROTOCOL, "Expiring stuck OR connection to fd %d (%s:%d). (%d bytes to " "flush; %d seconds since last write)", (int)conn->s, conn->address, conn->port, (int)connection_get_outbuf_len(conn), - (int)(now-conn->timestamp_lastwritten)); + (int)(now-conn->timestamp_last_write_allowed)); connection_or_close_normally(TO_OR_CONN(conn), 0); } else if (past_keepalive && !connection_get_outbuf_len(conn)) { /* send a padding cell */ @@ -1334,7 +1335,6 @@ CALLBACK(retry_listeners); CALLBACK(expire_old_ciruits_serverside); CALLBACK(check_dns_honesty); CALLBACK(write_bridge_ns); -CALLBACK(check_fw_helper_app); CALLBACK(heartbeat); CALLBACK(clean_consdiffmgr); CALLBACK(reset_padding_counts); @@ -1370,7 +1370,6 @@ static periodic_event_item_t periodic_events[] = { CALLBACK(expire_old_ciruits_serverside), CALLBACK(check_dns_honesty), CALLBACK(write_bridge_ns), - CALLBACK(check_fw_helper_app), CALLBACK(heartbeat), CALLBACK(clean_consdiffmgr), CALLBACK(reset_padding_counts), @@ -1473,6 +1472,7 @@ teardown_periodic_events(void) for (i = 0; periodic_events[i].name; ++i) { periodic_event_destroy(&periodic_events[i]); } + periodic_events_initialized = 0; } /** @@ -1940,14 +1940,14 @@ reset_padding_counts_callback(time_t now, const or_options_t *options) return REPHIST_CELL_PADDING_COUNTS_INTERVAL; } +static int should_init_bridge_stats = 1; + /** * Periodic callback: Write bridge statistics to disk if appropriate. */ static int record_bridge_stats_callback(time_t now, const or_options_t *options) { - static int should_init_bridge_stats = 1; - /* 1h. Check whether we should write bridge statistics to disk. */ if (should_record_bridge_info(options)) { @@ -2062,8 +2062,8 @@ check_for_reachability_bw_callback(time_t now, const or_options_t *options) if (server_mode(options) && (have_completed_a_circuit() || !any_predicted_circuits(now)) && !net_is_disabled()) { - if (stats_n_seconds_working < TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { - consider_testing_reachability(1, dirport_reachability_count==0); + if (get_uptime() < TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { + router_do_reachability_checks(1, dirport_reachability_count==0); if (++dirport_reachability_count > 5) dirport_reachability_count = 0; return 1; @@ -2140,6 +2140,8 @@ expire_old_ciruits_serverside_callback(time_t now, const or_options_t *options) return 11; } +static int dns_honesty_first_time = 1; + /** * Periodic event: if we're an exit, see if our DNS server is telling us * obvious lies. @@ -2155,10 +2157,9 @@ check_dns_honesty_callback(time_t now, const or_options_t *options) router_my_exit_policy_is_reject_star()) return PERIODIC_EVENT_NO_UPDATE; - static int first_time = 1; - if (first_time) { + if (dns_honesty_first_time) { /* Don't launch right when we start */ - first_time = 0; + dns_honesty_first_time = 0; return crypto_rand_int_range(60, 180); } @@ -2182,32 +2183,7 @@ write_bridge_ns_callback(time_t now, const or_options_t *options) return PERIODIC_EVENT_NO_UPDATE; } -/** - * Periodic callback: poke the tor-fw-helper app if we're using one. - */ -static int -check_fw_helper_app_callback(time_t now, const or_options_t *options) -{ - if (net_is_disabled() || - ! server_mode(options) || - ! options->PortForwarding || - options->NoExec) { - return PERIODIC_EVENT_NO_UPDATE; - } - /* 11. check the port forwarding app */ - -#define PORT_FORWARDING_CHECK_INTERVAL 5 - smartlist_t *ports_to_forward = get_list_of_ports_to_forward(); - if (ports_to_forward) { - tor_check_port_forwarding(options->PortForwardingHelper, - ports_to_forward, - now); - - SMARTLIST_FOREACH(ports_to_forward, char *, cp, tor_free(cp)); - smartlist_free(ports_to_forward); - } - return PORT_FORWARDING_CHECK_INTERVAL; -} +static int heartbeat_callback_first_time = 1; /** * Periodic callback: write the heartbeat message in the logs. @@ -2218,16 +2194,14 @@ check_fw_helper_app_callback(time_t now, const or_options_t *options) static int heartbeat_callback(time_t now, const or_options_t *options) { - static int first = 1; - /* Check if heartbeat is disabled */ if (!options->HeartbeatPeriod) { return PERIODIC_EVENT_NO_UPDATE; } /* Skip the first one. */ - if (first) { - first = 0; + if (heartbeat_callback_first_time) { + heartbeat_callback_first_time = 0; return options->HeartbeatPeriod; } @@ -2279,6 +2253,8 @@ hs_service_callback(time_t now, const or_options_t *options) static periodic_timer_t *second_timer = NULL; /** Number of libevent errors in the last second: we die if we get too many. */ static int n_libevent_errors = 0; +/** Last time that second_elapsed_callback was called. */ +static time_t current_second = 0; /** Libevent callback: invoked once every second. */ static void @@ -2287,7 +2263,6 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) /* XXXX This could be sensibly refactored into multiple callbacks, and we * could use Libevent's timers for this rather than checking the current * time against a bunch of timeouts every second. */ - static time_t current_second = 0; time_t now; size_t bytes_written; size_t bytes_read; @@ -2319,8 +2294,8 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) !net_is_disabled() && seconds_elapsed > 0 && have_completed_a_circuit() && - stats_n_seconds_working / TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT != - (stats_n_seconds_working+seconds_elapsed) / + get_uptime() / TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT != + (get_uptime()+seconds_elapsed) / TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { /* every 20 minutes, check and complain if necessary */ const routerinfo_t *me = router_get_my_routerinfo(); @@ -2382,12 +2357,14 @@ systemd_watchdog_callback(periodic_timer_t *timer, void *arg) /** Timer: used to invoke refill_callback(). */ static periodic_timer_t *refill_timer = NULL; +/** Millisecond when refall_callback was last invoked. */ +static struct timeval refill_timer_current_millisecond; + /** Libevent callback: invoked periodically to refill token buckets * and count r/w bytes. */ static void refill_callback(periodic_timer_t *timer, void *arg) { - static struct timeval current_millisecond; struct timeval now; size_t bytes_written; @@ -2403,12 +2380,13 @@ refill_callback(periodic_timer_t *timer, void *arg) tor_gettimeofday(&now); /* If this is our first time, no time has passed. */ - if (current_millisecond.tv_sec) { - long mdiff = tv_mdiff(¤t_millisecond, &now); + if (refill_timer_current_millisecond.tv_sec) { + long mdiff = tv_mdiff(&refill_timer_current_millisecond, &now); if (mdiff > INT_MAX) mdiff = INT_MAX; milliseconds_elapsed = (int)mdiff; - seconds_rolled_over = (int)(now.tv_sec - current_millisecond.tv_sec); + seconds_rolled_over = (int)(now.tv_sec - + refill_timer_current_millisecond.tv_sec); } bytes_written = stats_prev_global_write_bucket - global_write_bucket; @@ -2425,7 +2403,8 @@ refill_callback(periodic_timer_t *timer, void *arg) stats_prev_global_read_bucket = global_read_bucket; stats_prev_global_write_bucket = global_write_bucket; - current_millisecond = now; /* remember what time it is, for next time */ + /* remember what time it is, for next time */ + refill_timer_current_millisecond = now; } #ifndef _WIN32 @@ -2464,9 +2443,9 @@ ip_address_changed(int at_interface) } } else { if (server) { - if (stats_n_seconds_working > UPTIME_CUTOFF_FOR_NEW_BANDWIDTH_TEST) + if (get_uptime() > UPTIME_CUTOFF_FOR_NEW_BANDWIDTH_TEST) reset_bandwidth_test(); - stats_n_seconds_working = 0; + reset_uptime(); router_reset_reachability(); } } @@ -2824,8 +2803,8 @@ run_main_loop_once(void) * an event, or the second ends, or until we have some active linked * connections to trigger events for. Libevent will wait till one * of these happens, then run all the appropriate callbacks. */ - loop_result = event_base_loop(tor_libevent_get_base(), - called_loop_once ? EVLOOP_ONCE : 0); + loop_result = tor_libevent_run_event_loop(tor_libevent_get_base(), + called_loop_once); if (get_options()->MainloopStats) { /* Update our main loop counters. */ @@ -3004,6 +2983,13 @@ get_uptime,(void)) return stats_n_seconds_working; } +/** Reset Tor's uptime. */ +MOCK_IMPL(void, +reset_uptime,(void)) +{ + stats_n_seconds_working = 0; +} + /** * Write current memory usage information to the log. */ @@ -3047,13 +3033,13 @@ dumpstats(int severity) i, (int)connection_get_inbuf_len(conn), (int)buf_allocation(conn->inbuf), - (int)(now - conn->timestamp_lastread)); + (int)(now - conn->timestamp_last_read_allowed)); tor_log(severity,LD_GENERAL, "Conn %d: %d bytes waiting on outbuf " "(len %d, last written %d secs ago)",i, (int)connection_get_outbuf_len(conn), (int)buf_allocation(conn->outbuf), - (int)(now - conn->timestamp_lastwritten)); + (int)(now - conn->timestamp_last_write_allowed)); if (conn->type == CONN_TYPE_OR) { or_connection_t *or_conn = TO_OR_CONN(conn); if (or_conn->tls) { @@ -3323,14 +3309,12 @@ tor_init(int argc, char *argv[]) if (strstr(version, "alpha") || strstr(version, "beta")) log_notice(LD_GENERAL, "This version is not a stable Tor release. " "Expect more bugs than usual."); + + tor_compress_log_init_warnings(); } #ifdef HAVE_RUST - char *rust_str = rust_welcome_string(); - if (rust_str != NULL && strlen(rust_str) > 0) { - log_notice(LD_GENERAL, "%s", rust_str); - } - tor_free(rust_str); + rust_log_welcome_string(); #endif /* defined(HAVE_RUST) */ if (network_init()<0) { @@ -3511,6 +3495,32 @@ tor_free_all(int postfork) periodic_timer_free(refill_timer); tor_event_free(shutdown_did_not_work_event); tor_event_free(initialize_periodic_events_event); + mainloop_event_free(directory_all_unreachable_cb_event); + +#ifdef HAVE_SYSTEMD_209 + periodic_timer_free(systemd_watchdog_timer); +#endif + + global_read_bucket = global_write_bucket = 0; + global_relayed_read_bucket = global_relayed_write_bucket = 0; + stats_prev_global_read_bucket = stats_prev_global_write_bucket = 0; + stats_prev_n_read = stats_prev_n_written = 0; + stats_n_bytes_read = stats_n_bytes_written = 0; + time_of_process_start = 0; + time_of_last_signewnym = 0; + signewnym_is_pending = 0; + newnym_epoch = 0; + called_loop_once = 0; + main_loop_should_exit = 0; + main_loop_exit_value = 0; + can_complete_circuits = 0; + quiet_level = 0; + should_init_bridge_stats = 1; + dns_honesty_first_time = 1; + heartbeat_callback_first_time = 1; + n_libevent_errors = 0; + current_second = 0; + memset(&refill_timer_current_millisecond, 0, sizeof(struct timeval)); if (!postfork) { release_lockfile(); diff --git a/src/or/main.h b/src/or/main.h index c49d216f4e..f01506fcea 100644 --- a/src/or/main.h +++ b/src/or/main.h @@ -63,6 +63,7 @@ void reschedule_descriptor_update_check(void); void reschedule_directory_downloads(void); MOCK_DECL(long,get_uptime,(void)); +MOCK_DECL(void,reset_uptime,(void)); unsigned get_signewnym_epoch(void); @@ -87,7 +88,6 @@ uint64_t get_main_loop_error_count(void); uint64_t get_main_loop_idle_count(void); extern time_t time_of_process_start; -extern long stats_n_seconds_working; extern int quiet_level; extern int global_read_bucket; extern int global_write_bucket; diff --git a/src/or/networkstatus.c b/src/or/networkstatus.c index 7777473f11..235b95b704 100644 --- a/src/or/networkstatus.c +++ b/src/or/networkstatus.c @@ -237,7 +237,7 @@ router_reload_consensus_networkstatus(void) s = networkstatus_read_cached_consensus_impl(flav, flavor, 1); if (s) { if (networkstatus_set_current_consensus(s, flavor, - flags|NSSET_WAS_WAITING_FOR_CERTS, + flags | NSSET_WAS_WAITING_FOR_CERTS, NULL)) { log_info(LD_FS, "Couldn't load unverified consensus %s networkstatus " "from cache", flavor); @@ -951,9 +951,12 @@ update_consensus_networkstatus_downloads(time_t now) continue; } - /* Check if we're waiting for certificates to download */ - if (check_consensus_waiting_for_certs(i, now, &consensus_dl_status[i])) + /** Check if we're waiting for certificates to download. If we are, + * launch download for missing directory authority certificates. */ + if (check_consensus_waiting_for_certs(i, now, &consensus_dl_status[i])) { + update_certificate_downloads(now); continue; + } /* Try the requested attempt */ log_info(LD_DIR, "Launching %s standard networkstatus consensus " @@ -1230,16 +1233,20 @@ should_delay_dir_fetches(const or_options_t *options, const char **msg_out) return 0; } -/** Launch requests for networkstatus documents and authority certificates as - * appropriate. */ +/** Launch requests for networkstatus documents as appropriate. This is called + * when we retry all the connections on a SIGHUP and periodically by a Periodic + * event which checks whether we want to download any networkstatus documents. + */ void update_networkstatus_downloads(time_t now) { const or_options_t *options = get_options(); if (should_delay_dir_fetches(options, NULL)) return; + /** Launch a consensus download request, we will wait for the consensus to + * download and when it completes we will launch a certificate download + * request. */ update_consensus_networkstatus_downloads(now); - update_certificate_downloads(now); } /** Launch requests as appropriate for missing directory authority @@ -1768,7 +1775,6 @@ networkstatus_set_current_consensus(const char *consensus, consensus_waiting_for_certs_t *waiting = NULL; time_t current_valid_after = 0; int free_consensus = 1; /* Free 'c' at the end of the function */ - int old_ewma_enabled; int checked_protocols_already = 0; if (flav < 0) { @@ -1917,6 +1923,15 @@ networkstatus_set_current_consensus(const char *consensus, } } + /* Signatures from the consensus are verified */ + if (from_cache && was_waiting_for_certs) { + /* We check if the consensus is loaded from disk cache and that it + * it is an unverified consensus. If it is unverified, rename it to + * cached-*-consensus since it has been verified. */ + log_info(LD_DIR, "Unverified consensus signatures verified."); + tor_rename(unverified_fname, consensus_fname); + } + if (!from_cache && flav == usable_consensus_flavor()) control_event_client_status(LOG_NOTICE, "CONSENSUS_ARRIVED"); @@ -1992,17 +2007,8 @@ networkstatus_set_current_consensus(const char *consensus, /* XXXXNM Microdescs: needs a non-ns variant. ???? NM*/ update_consensus_networkstatus_fetch_time(now); - /* Update ewma and adjust policy if needed; first cache the old value */ - old_ewma_enabled = cell_ewma_enabled(); /* Change the cell EWMA settings */ - cell_ewma_set_scale_factor(options, c); - /* If we just enabled ewma, set the cmux policy on all active channels */ - if (cell_ewma_enabled() && !old_ewma_enabled) { - channel_set_cmux_policy_everywhere(&ewma_policy); - } else if (!cell_ewma_enabled() && old_ewma_enabled) { - /* Turn it off everywhere */ - channel_set_cmux_policy_everywhere(NULL); - } + cmux_ewma_set_options(options, c); /* XXXX this call might be unnecessary here: can changing the * current consensus really alter our view of any OR's rate limits? */ diff --git a/src/or/nodelist.c b/src/or/nodelist.c index 3a26aee611..71eee3fa21 100644 --- a/src/or/nodelist.c +++ b/src/or/nodelist.c @@ -1456,9 +1456,11 @@ node_ipv6_or_preferred(const node_t *node) /* XX/teor - node->ipv6_preferred is set from * fascist_firewall_prefer_ipv6_orport() each time the consensus is loaded. */ + node_get_prim_orport(node, &ipv4_addr); if (!fascist_firewall_use_ipv6(options)) { return 0; - } else if (node->ipv6_preferred || node_get_prim_orport(node, &ipv4_addr)) { + } else if (node->ipv6_preferred || + !tor_addr_port_is_valid_ap(&ipv4_addr, 0)) { return node_has_ipv6_orport(node); } return 0; @@ -1469,14 +1471,12 @@ node_ipv6_or_preferred(const node_t *node) if (r && tor_addr_port_is_valid_ipv4h((r)->addr, (r)->port_field, 0)) { \ tor_addr_from_ipv4h(&(ap_out)->addr, (r)->addr); \ (ap_out)->port = (r)->port_field; \ - return 0; \ } \ STMT_END -/** Copy the primary (IPv4) OR port (IP address and TCP port) for - * <b>node</b> into *<b>ap_out</b>. Return 0 if a valid address and - * port was copied, else return non-zero.*/ -int +/** Copy the primary (IPv4) OR port (IP address and TCP port) for <b>node</b> + * into *<b>ap_out</b>. */ +void node_get_prim_orport(const node_t *node, tor_addr_port_t *ap_out) { node_assert_ok(node); @@ -1493,8 +1493,6 @@ node_get_prim_orport(const node_t *node, tor_addr_port_t *ap_out) RETURN_IPV4_AP(node->ri, or_port, ap_out); RETURN_IPV4_AP(node->rs, or_port, ap_out); /* Microdescriptors only have an IPv6 address */ - - return -1; } /** Copy the preferred OR port (IP address and TCP port) for @@ -1566,19 +1564,19 @@ node_ipv6_dir_preferred(const node_t *node) * so we can't use it to determine DirPort IPv6 preference. * This means that bridge clients will use IPv4 DirPorts by default. */ + node_get_prim_dirport(node, &ipv4_addr); if (!fascist_firewall_use_ipv6(options)) { return 0; - } else if (node_get_prim_dirport(node, &ipv4_addr) + } else if (!tor_addr_port_is_valid_ap(&ipv4_addr, 0) || fascist_firewall_prefer_ipv6_dirport(get_options())) { return node_has_ipv6_dirport(node); } return 0; } -/** Copy the primary (IPv4) Dir port (IP address and TCP port) for - * <b>node</b> into *<b>ap_out</b>. Return 0 if a valid address and - * port was copied, else return non-zero.*/ -int +/** Copy the primary (IPv4) Dir port (IP address and TCP port) for <b>node</b> + * into *<b>ap_out</b>. */ +void node_get_prim_dirport(const node_t *node, tor_addr_port_t *ap_out) { node_assert_ok(node); @@ -1590,8 +1588,6 @@ node_get_prim_dirport(const node_t *node, tor_addr_port_t *ap_out) RETURN_IPV4_AP(node->ri, dir_port, ap_out); RETURN_IPV4_AP(node->rs, dir_port, ap_out); /* Microdescriptors only have an IPv6 address */ - - return -1; } #undef RETURN_IPV4_AP diff --git a/src/or/nodelist.h b/src/or/nodelist.h index 043d7b3414..53b18ab48a 100644 --- a/src/or/nodelist.h +++ b/src/or/nodelist.h @@ -77,11 +77,11 @@ int node_has_ipv6_dirport(const node_t *node); /* Deprecated - use node_ipv6_or_preferred or node_ipv6_dir_preferred */ #define node_ipv6_preferred(node) node_ipv6_or_preferred(node) int node_ipv6_or_preferred(const node_t *node); -int node_get_prim_orport(const node_t *node, tor_addr_port_t *ap_out); +void node_get_prim_orport(const node_t *node, tor_addr_port_t *ap_out); void node_get_pref_orport(const node_t *node, tor_addr_port_t *ap_out); void node_get_pref_ipv6_orport(const node_t *node, tor_addr_port_t *ap_out); int node_ipv6_dir_preferred(const node_t *node); -int node_get_prim_dirport(const node_t *node, tor_addr_port_t *ap_out); +void node_get_prim_dirport(const node_t *node, tor_addr_port_t *ap_out); void node_get_pref_dirport(const node_t *node, tor_addr_port_t *ap_out); void node_get_pref_ipv6_dirport(const node_t *node, tor_addr_port_t *ap_out); int node_has_curve25519_onion_key(const node_t *node); diff --git a/src/or/ntmain.c b/src/or/ntmain.c index ebbe0018bd..e9a299807a 100644 --- a/src/or/ntmain.c +++ b/src/or/ntmain.c @@ -24,8 +24,6 @@ #include "main.h" #include "ntmain.h" -#include <event2/event.h> - #include <windows.h> #define GENSRV_SERVICENAME "tor" #define GENSRV_DISPLAYNAME "Tor Win32 Service" @@ -245,7 +243,8 @@ nt_service_control(DWORD request) log_notice(LD_GENERAL, "Got stop/shutdown request; shutting down cleanly."); service_status.dwCurrentState = SERVICE_STOP_PENDING; - event_base_loopexit(tor_libevent_get_base(), &exit_now); + tor_libevent_exit_loop_after_delay(tor_libevent_get_base(), + &exit_now); return; } service_fns.SetServiceStatus_fn(hStatus, &service_status); diff --git a/src/or/onion.c b/src/or/onion.c index bd80c2f503..0c88c4d7ee 100644 --- a/src/or/onion.c +++ b/src/or/onion.c @@ -521,6 +521,11 @@ onion_skin_create(int type, return r; } +/* This is the maximum value for keys_out_len passed to + * onion_skin_server_handshake, plus 16. We can make it bigger if needed: + * It just defines how many bytes to stack-allocate. */ +#define MAX_KEYS_TMP_LEN 128 + /** Perform the second (server-side) step of a circuit-creation handshake of * type <b>type</b>, responding to the client request in <b>onion_skin</b> * using the keys in <b>keys</b>. On success, write our response into @@ -563,20 +568,21 @@ onion_skin_server_handshake(int type, return -1; { size_t keys_tmp_len = keys_out_len + DIGEST_LEN; - uint8_t *keys_tmp = tor_malloc(keys_out_len + DIGEST_LEN); + tor_assert(keys_tmp_len <= MAX_KEYS_TMP_LEN); + uint8_t keys_tmp[MAX_KEYS_TMP_LEN]; if (onion_skin_ntor_server_handshake( onion_skin, keys->curve25519_key_map, keys->junk_keypair, keys->my_identity, reply_out, keys_tmp, keys_tmp_len)<0) { - tor_free(keys_tmp); + /* no need to memwipe here, since the output will never be used */ return -1; } + memcpy(keys_out, keys_tmp, keys_out_len); memcpy(rend_nonce_out, keys_tmp+keys_out_len, DIGEST_LEN); - memwipe(keys_tmp, 0, keys_tmp_len); - tor_free(keys_tmp); + memwipe(keys_tmp, 0, sizeof(keys_tmp)); r = NTOR_REPLY_LEN; } break; diff --git a/src/or/onion_ntor.c b/src/or/onion_ntor.c index b167cb61fb..8ad876a587 100644 --- a/src/or/onion_ntor.c +++ b/src/or/onion_ntor.c @@ -22,6 +22,7 @@ #define ONION_NTOR_PRIVATE #include "crypto.h" +#include "crypto_digest.h" #include "onion_ntor.h" #include "torlog.h" #include "util.h" diff --git a/src/or/or.h b/src/or/or.h index e54d3806ef..b845443947 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -1369,10 +1369,10 @@ typedef struct connection_t { * connection. */ size_t outbuf_flushlen; /**< How much data should we try to flush from the * outbuf? */ - time_t timestamp_lastread; /**< When was the last time libevent said we could - * read? */ - time_t timestamp_lastwritten; /**< When was the last time libevent said we - * could write? */ + time_t timestamp_last_read_allowed; /**< When was the last time libevent said + * we could read? */ + time_t timestamp_last_write_allowed; /**< When was the last time libevent + * said we could write? */ time_t timestamp_created; /**< When was this connection_t created? */ @@ -2351,10 +2351,10 @@ typedef struct routerstatus_t { * If it's a descriptor, we only use the first DIGEST_LEN bytes. */ char descriptor_digest[DIGEST256_LEN]; uint32_t addr; /**< IPv4 address for this router, in host order. */ - uint16_t or_port; /**< OR port for this router. */ + uint16_t or_port; /**< IPv4 OR port for this router. */ uint16_t dir_port; /**< Directory port for this router. */ tor_addr_t ipv6_addr; /**< IPv6 address for this router. */ - uint16_t ipv6_orport; /**<IPV6 OR port for this router. */ + uint16_t ipv6_orport; /**< IPv6 OR port for this router. */ unsigned int is_authority:1; /**< True iff this router is an authority. */ unsigned int is_exit:1; /**< True iff this router is a good exit. */ unsigned int is_stable:1; /**< True iff this router stays up a long time. */ @@ -2907,11 +2907,7 @@ typedef struct { } u; } onion_handshake_state_t; -/** Holds accounting information for a single step in the layered encryption - * performed by a circuit. Used only at the client edge of a circuit. */ -typedef struct crypt_path_t { - uint32_t magic; - +typedef struct relay_crypto_t { /* crypto environments */ /** Encryption key and counter for cells heading towards the OR at this * step. */ @@ -2925,6 +2921,17 @@ typedef struct crypt_path_t { /** Digest state for cells heading away from the OR at this step. */ crypto_digest_t *b_digest; +} relay_crypto_t; + +/** Holds accounting information for a single step in the layered encryption + * performed by a circuit. Used only at the client edge of a circuit. */ +typedef struct crypt_path_t { + uint32_t magic; + + /** Cryptographic state used for encrypting and authenticating relay + * cells to and from this hop. */ + relay_crypto_t crypto; + /** Current state of the handshake as performed with the OR at this * step. */ onion_handshake_state_t handshake_state; @@ -3170,15 +3177,6 @@ typedef struct circuit_t { /** Index in smartlist of all circuits (global_circuitlist). */ int global_circuitlist_idx; - /** Next circuit in the doubly-linked ring of circuits waiting to add - * cells to n_conn. NULL if we have no cells pending, or if we're not - * linked to an OR connection. */ - struct circuit_t *next_active_on_n_chan; - /** Previous circuit in the doubly-linked ring of circuits waiting to add - * cells to n_conn. NULL if we have no cells pending, or if we're not - * linked to an OR connection. */ - struct circuit_t *prev_active_on_n_chan; - /** Various statistics about cells being added to or removed from this * circuit's queues; used only if CELL_STATS events are enabled and * cleared after being sent to control port. */ @@ -3458,14 +3456,6 @@ struct onion_queue_t; typedef struct or_circuit_t { circuit_t base_; - /** Next circuit in the doubly-linked ring of circuits waiting to add - * cells to p_chan. NULL if we have no cells pending, or if we're not - * linked to an OR connection. */ - struct circuit_t *next_active_on_p_chan; - /** Previous circuit in the doubly-linked ring of circuits waiting to add - * cells to p_chan. NULL if we have no cells pending, or if we're not - * linked to an OR connection. */ - struct circuit_t *prev_active_on_p_chan; /** Pointer to an entry on the onion queue, if this circuit is waiting for a * chance to give an onionskin to a cpuworker. Used only in onion.c */ struct onion_queue_t *onionqueue_entry; @@ -3490,21 +3480,10 @@ typedef struct or_circuit_t { /** Linked list of Exit streams associated with this circuit that are * still being resolved. */ edge_connection_t *resolving_streams; - /** The cipher used by intermediate hops for cells heading toward the - * OP. */ - crypto_cipher_t *p_crypto; - /** The cipher used by intermediate hops for cells heading away from - * the OP. */ - crypto_cipher_t *n_crypto; - - /** The integrity-checking digest used by intermediate hops, for - * cells packaged here and heading towards the OP. - */ - crypto_digest_t *p_digest; - /** The integrity-checking digest used by intermediate hops, for - * cells packaged at the OP and arriving here. - */ - crypto_digest_t *n_digest; + + /** Cryptographic state used for encrypting and authenticating relay + * cells to and from this hop. */ + relay_crypto_t crypto; /** Points to spliced circuit if purpose is REND_ESTABLISHED, and circuit * is not marked for close. */ @@ -4229,10 +4208,6 @@ typedef struct { * testing our DNS server. */ int EnforceDistinctSubnets; /**< If true, don't allow multiple routers in the * same network zone in the same circuit. */ - int PortForwarding; /**< If true, use NAT-PMP or UPnP to automatically - * forward the DirPort and ORPort on the NAT device */ - char *PortForwardingHelper; /** < Filename or full path of the port - forwarding helper executable */ int AllowNonRFC953Hostnames; /**< If true, we allow connections to hostnames * with weird characters. */ /** If true, we try resolving hostnames with weird characters. */ diff --git a/src/or/periodic.c b/src/or/periodic.c index 6896b41c86..fa40965de1 100644 --- a/src/or/periodic.c +++ b/src/or/periodic.c @@ -16,8 +16,6 @@ #include "config.h" #include "periodic.h" -#include <event2/event.h> - /** We disable any interval greater than this number of seconds, on the * grounds that it is probably an absolute time mistakenly passed in as a * relative time. @@ -34,17 +32,16 @@ periodic_event_set_interval(periodic_event_item_t *event, struct timeval tv; tv.tv_sec = next_interval; tv.tv_usec = 0; - event_add(event->ev, &tv); + mainloop_event_schedule(event->ev, &tv); } /** Wraps dispatches for periodic events, <b>data</b> will be a pointer to the * event that needs to be called */ static void -periodic_event_dispatch(evutil_socket_t fd, short what, void *data) +periodic_event_dispatch(mainloop_event_t *ev, void *data) { - (void)fd; - (void)what; periodic_event_item_t *event = data; + tor_assert(ev == event->ev); time_t now = time(NULL); const or_options_t *options = get_options(); @@ -74,7 +71,7 @@ periodic_event_dispatch(evutil_socket_t fd, short what, void *data) // log_debug(LD_GENERAL, "Scheduling %s for %d seconds", event->name, // next_interval); struct timeval tv = { next_interval , 0 }; - event_add(event->ev, &tv); + mainloop_event_schedule(ev, &tv); } /** Schedules <b>event</b> to run as soon as possible from now. */ @@ -93,10 +90,8 @@ periodic_event_setup(periodic_event_item_t *event) tor_assert(0); } - event->ev = tor_event_new(tor_libevent_get_base(), - -1, 0, - periodic_event_dispatch, - event); + event->ev = mainloop_event_new(periodic_event_dispatch, + event); tor_assert(event->ev); } @@ -111,7 +106,7 @@ periodic_event_launch(periodic_event_item_t *event) } // Initial dispatch - periodic_event_dispatch(-1, EV_TIMEOUT, event); + periodic_event_dispatch(event->ev, event); } /** Release all storage associated with <b>event</b> */ @@ -120,7 +115,7 @@ periodic_event_destroy(periodic_event_item_t *event) { if (!event) return; - tor_event_free(event->ev); + mainloop_event_free(event->ev); event->last_action_time = 0; } diff --git a/src/or/periodic.h b/src/or/periodic.h index 8baf3994eb..285400b8bb 100644 --- a/src/or/periodic.h +++ b/src/or/periodic.h @@ -14,13 +14,14 @@ typedef int (*periodic_event_helper_t)(time_t now, const or_options_t *options); -struct event; +struct mainloop_event_t; /** A single item for the periodic-events-function table. */ typedef struct periodic_event_item_t { periodic_event_helper_t fn; /**< The function to run the event */ time_t last_action_time; /**< The last time the function did something */ - struct event *ev; /**< Libevent callback we're using to implement this */ + struct mainloop_event_t *ev; /**< Libevent callback we're using to implement + * this */ const char *name; /**< Name of the function -- for debug */ } periodic_event_item_t; diff --git a/src/or/relay.c b/src/or/relay.c index 506b7eccb3..c6f030945b 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -70,6 +70,7 @@ #include "policies.h" #include "reasons.h" #include "relay.h" +#include "relay_crypto.h" #include "rendcache.h" #include "rendcommon.h" #include "router.h" @@ -122,77 +123,6 @@ uint64_t stats_n_relay_cells_delivered = 0; /** Used to tell which stream to read from first on a circuit. */ static tor_weak_rng_t stream_choice_rng = TOR_WEAK_RNG_INIT; -/** Update digest from the payload of cell. Assign integrity part to - * cell. - */ -static void -relay_set_digest(crypto_digest_t *digest, cell_t *cell) -{ - char integrity[4]; - relay_header_t rh; - - crypto_digest_add_bytes(digest, (char*)cell->payload, CELL_PAYLOAD_SIZE); - crypto_digest_get_digest(digest, integrity, 4); -// log_fn(LOG_DEBUG,"Putting digest of %u %u %u %u into relay cell.", -// integrity[0], integrity[1], integrity[2], integrity[3]); - relay_header_unpack(&rh, cell->payload); - memcpy(rh.integrity, integrity, 4); - relay_header_pack(cell->payload, &rh); -} - -/** Does the digest for this circuit indicate that this cell is for us? - * - * Update digest from the payload of cell (with the integrity part set - * to 0). If the integrity part is valid, return 1, else restore digest - * and cell to their original state and return 0. - */ -static int -relay_digest_matches(crypto_digest_t *digest, cell_t *cell) -{ - uint32_t received_integrity, calculated_integrity; - relay_header_t rh; - crypto_digest_t *backup_digest=NULL; - - backup_digest = crypto_digest_dup(digest); - - relay_header_unpack(&rh, cell->payload); - memcpy(&received_integrity, rh.integrity, 4); - memset(rh.integrity, 0, 4); - relay_header_pack(cell->payload, &rh); - -// log_fn(LOG_DEBUG,"Reading digest of %u %u %u %u from relay cell.", -// received_integrity[0], received_integrity[1], -// received_integrity[2], received_integrity[3]); - - crypto_digest_add_bytes(digest, (char*) cell->payload, CELL_PAYLOAD_SIZE); - crypto_digest_get_digest(digest, (char*) &calculated_integrity, 4); - - if (calculated_integrity != received_integrity) { -// log_fn(LOG_INFO,"Recognized=0 but bad digest. Not recognizing."); -// (%d vs %d).", received_integrity, calculated_integrity); - /* restore digest to its old form */ - crypto_digest_assign(digest, backup_digest); - /* restore the relay header */ - memcpy(rh.integrity, &received_integrity, 4); - relay_header_pack(cell->payload, &rh); - crypto_digest_free(backup_digest); - return 0; - } - crypto_digest_free(backup_digest); - return 1; -} - -/** Apply <b>cipher</b> to CELL_PAYLOAD_SIZE bytes of <b>in</b> - * (in place). - * - * Note that we use the same operation for encrypting and for decrypting. - */ -static void -relay_crypt_one_payload(crypto_cipher_t *cipher, uint8_t *in) -{ - crypto_cipher_crypt_inplace(cipher, (char*) in, CELL_PAYLOAD_SIZE); -} - /** * Update channel usage state based on the type of relay cell and * circuit properties. @@ -297,7 +227,8 @@ circuit_receive_relay_cell(cell_t *cell, circuit_t *circ, if (circ->marked_for_close) return 0; - if (relay_crypt(circ, cell, cell_direction, &layer_hint, &recognized) < 0) { + if (relay_decrypt_cell(circ, cell, cell_direction, &layer_hint, &recognized) + < 0) { log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, "relay crypt failed. Dropping connection."); return -END_CIRC_REASON_INTERNAL; @@ -402,87 +333,6 @@ circuit_receive_relay_cell(cell_t *cell, circuit_t *circ, return 0; } -/** Do the appropriate en/decryptions for <b>cell</b> arriving on - * <b>circ</b> in direction <b>cell_direction</b>. - * - * If cell_direction == CELL_DIRECTION_IN: - * - If we're at the origin (we're the OP), for hops 1..N, - * decrypt cell. If recognized, stop. - * - Else (we're not the OP), encrypt one hop. Cell is not recognized. - * - * If cell_direction == CELL_DIRECTION_OUT: - * - decrypt one hop. Check if recognized. - * - * If cell is recognized, set *recognized to 1, and set - * *layer_hint to the hop that recognized it. - * - * Return -1 to indicate that we should mark the circuit for close, - * else return 0. - */ -int -relay_crypt(circuit_t *circ, cell_t *cell, cell_direction_t cell_direction, - crypt_path_t **layer_hint, char *recognized) -{ - relay_header_t rh; - - tor_assert(circ); - tor_assert(cell); - tor_assert(recognized); - tor_assert(cell_direction == CELL_DIRECTION_IN || - cell_direction == CELL_DIRECTION_OUT); - - if (cell_direction == CELL_DIRECTION_IN) { - if (CIRCUIT_IS_ORIGIN(circ)) { /* We're at the beginning of the circuit. - * We'll want to do layered decrypts. */ - crypt_path_t *thishop, *cpath = TO_ORIGIN_CIRCUIT(circ)->cpath; - thishop = cpath; - if (thishop->state != CPATH_STATE_OPEN) { - log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, - "Relay cell before first created cell? Closing."); - return -1; - } - do { /* Remember: cpath is in forward order, that is, first hop first. */ - tor_assert(thishop); - - /* decrypt one layer */ - relay_crypt_one_payload(thishop->b_crypto, cell->payload); - - relay_header_unpack(&rh, cell->payload); - if (rh.recognized == 0) { - /* it's possibly recognized. have to check digest to be sure. */ - if (relay_digest_matches(thishop->b_digest, cell)) { - *recognized = 1; - *layer_hint = thishop; - return 0; - } - } - - thishop = thishop->next; - } while (thishop != cpath && thishop->state == CPATH_STATE_OPEN); - log_fn(LOG_PROTOCOL_WARN, LD_OR, - "Incoming cell at client not recognized. Closing."); - return -1; - } else { - /* We're in the middle. Encrypt one layer. */ - relay_crypt_one_payload(TO_OR_CIRCUIT(circ)->p_crypto, cell->payload); - } - } else /* cell_direction == CELL_DIRECTION_OUT */ { - /* We're in the middle. Decrypt one layer. */ - - relay_crypt_one_payload(TO_OR_CIRCUIT(circ)->n_crypto, cell->payload); - - relay_header_unpack(&rh, cell->payload); - if (rh.recognized == 0) { - /* it's possibly recognized. have to check digest to be sure. */ - if (relay_digest_matches(TO_OR_CIRCUIT(circ)->n_digest, cell)) { - *recognized = 1; - return 0; - } - } - } - return 0; -} - /** Package a relay cell from an edge: * - Encrypt it to the right layer * - Append it to the appropriate cell_queue on <b>circ</b>. @@ -501,7 +351,6 @@ circuit_package_relay_cell(cell_t *cell, circuit_t *circ, } if (cell_direction == CELL_DIRECTION_OUT) { - crypt_path_t *thishop; /* counter for repeated crypts */ chan = circ->n_chan; if (!chan) { log_warn(LD_BUG,"outgoing relay cell sent from %s:%d has n_chan==NULL." @@ -524,20 +373,8 @@ circuit_package_relay_cell(cell_t *cell, circuit_t *circ, return 0; /* just drop it */ } - relay_set_digest(layer_hint->f_digest, cell); - - thishop = layer_hint; - /* moving from farthest to nearest hop */ - do { - tor_assert(thishop); - log_debug(LD_OR,"encrypting a layer of the relay cell."); - relay_crypt_one_payload(thishop->f_crypto, cell->payload); - - thishop = thishop->prev; - } while (thishop != TO_ORIGIN_CIRCUIT(circ)->cpath->prev); - + relay_encrypt_cell_outbound(cell, TO_ORIGIN_CIRCUIT(circ), layer_hint); } else { /* incoming cell */ - or_circuit_t *or_circ; if (CIRCUIT_IS_ORIGIN(circ)) { /* We should never package an _incoming_ cell from the circuit * origin; that means we messed up somewhere. */ @@ -545,11 +382,9 @@ circuit_package_relay_cell(cell_t *cell, circuit_t *circ, assert_circuit_ok(circ); return 0; /* just drop it */ } - or_circ = TO_OR_CIRCUIT(circ); + or_circuit_t *or_circ = TO_OR_CIRCUIT(circ); + relay_encrypt_cell_inbound(cell, or_circ); chan = or_circ->p_chan; - relay_set_digest(or_circ->p_digest, cell); - /* encrypt one layer */ - relay_crypt_one_payload(or_circ->p_crypto, cell->payload); } ++stats_n_relay_cells_relayed; @@ -1449,7 +1284,7 @@ connection_edge_process_relay_cell_not_open( "after %d seconds.", (unsigned)circ->n_circ_id, rh->stream_id, - (int)(time(NULL) - conn->base_.timestamp_lastread)); + (int)(time(NULL) - conn->base_.timestamp_last_read_allowed)); if (connected_cell_parse(rh, cell, &addr, &ttl) < 0) { log_fn(LOG_PROTOCOL_WARN, LD_APP, "Got a badly formatted connected cell. Closing."); @@ -2397,13 +2232,6 @@ circuit_consider_sending_sendme(circuit_t *circ, crypt_path_t *layer_hint) } } -#ifdef ACTIVE_CIRCUITS_PARANOIA -#define assert_cmux_ok_paranoid(chan) \ - assert_circuit_mux_okay(chan) -#else -#define assert_cmux_ok_paranoid(chan) -#endif /* defined(ACTIVE_CIRCUITS_PARANOIA) */ - /** The total number of cells we have allocated. */ static size_t total_cells_allocated = 0; @@ -2691,16 +2519,12 @@ update_circuit_on_cmux_(circuit_t *circ, cell_direction_t direction, } tor_assert(circuitmux_attached_circuit_direction(cmux, circ) == direction); - assert_cmux_ok_paranoid(chan); - /* Update the number of cells we have for the circuit mux */ if (direction == CELL_DIRECTION_OUT) { circuitmux_set_num_cells(cmux, circ, circ->n_chan_cells.n); } else { circuitmux_set_num_cells(cmux, circ, or_circ->p_chan_cells.n); } - - assert_cmux_ok_paranoid(chan); } /** Remove all circuits from the cmux on <b>chan</b>. @@ -2845,7 +2669,6 @@ channel_flush_from_first_active_circuit, (channel_t *chan, int max)) } /* If it returns NULL, no cells left to send */ if (!circ) break; - assert_cmux_ok_paranoid(chan); if (circ->n_chan == chan) { queue = &circ->n_chan_cells; @@ -2949,15 +2772,13 @@ channel_flush_from_first_active_circuit, (channel_t *chan, int max)) } /* Okay, we're done sending now */ - assert_cmux_ok_paranoid(chan); - return n_flushed; } /** Add <b>cell</b> to the queue of <b>circ</b> writing to <b>chan</b> * transmitting in <b>direction</b>. * - * The given <b>cell</b> is copied over the circuit queue so the caller must + * The given <b>cell</b> is copied onto the circuit queue so the caller must * cleanup the memory. * * This function is part of the fast path. */ @@ -3101,17 +2922,6 @@ circuit_clear_cell_queue(circuit_t *circ, channel_t *chan) update_circuit_on_cmux(circ, direction); } -/** Fail with an assert if the circuit mux on chan is corrupt - */ -void -assert_circuit_mux_okay(channel_t *chan) -{ - tor_assert(chan); - tor_assert(chan->cmux); - - circuitmux_assert_okay(chan->cmux); -} - /** Return 1 if we shouldn't restart reading on this circuit, even if * we get a SENDME. Else return 0. */ diff --git a/src/or/relay.h b/src/or/relay.h index f0fa7e9870..cc78758883 100644 --- a/src/or/relay.h +++ b/src/or/relay.h @@ -76,7 +76,6 @@ void destroy_cell_queue_append(destroy_cell_queue_t *queue, void channel_unlink_all_circuits(channel_t *chan, smartlist_t *detached_out); MOCK_DECL(int, channel_flush_from_first_active_circuit, (channel_t *chan, int max)); -void assert_circuit_mux_okay(channel_t *chan); void update_circuit_on_cmux_(circuit_t *circ, cell_direction_t direction, const char *file, int lineno); #define update_circuit_on_cmux(circ, direction) \ @@ -90,9 +89,6 @@ void circuit_clear_cell_queue(circuit_t *circ, channel_t *chan); void stream_choice_seed_weak_rng(void); -int relay_crypt(circuit_t *circ, cell_t *cell, cell_direction_t cell_direction, - crypt_path_t **layer_hint, char *recognized); - circid_t packed_cell_get_circid(const packed_cell_t *cell, int wide_circ_ids); #ifdef RELAY_PRIVATE diff --git a/src/or/relay_crypto.c b/src/or/relay_crypto.c new file mode 100644 index 0000000000..c42a4f9cca --- /dev/null +++ b/src/or/relay_crypto.c @@ -0,0 +1,326 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2018, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "config.h" +#include "hs_ntor.h" // for HS_NTOR_KEY_EXPANSION_KDF_OUT_LEN +#include "relay_crypto.h" +#include "relay.h" + +/** Update digest from the payload of cell. Assign integrity part to + * cell. + */ +static void +relay_set_digest(crypto_digest_t *digest, cell_t *cell) +{ + char integrity[4]; + relay_header_t rh; + + crypto_digest_add_bytes(digest, (char*)cell->payload, CELL_PAYLOAD_SIZE); + crypto_digest_get_digest(digest, integrity, 4); +// log_fn(LOG_DEBUG,"Putting digest of %u %u %u %u into relay cell.", +// integrity[0], integrity[1], integrity[2], integrity[3]); + relay_header_unpack(&rh, cell->payload); + memcpy(rh.integrity, integrity, 4); + relay_header_pack(cell->payload, &rh); +} + +/** Does the digest for this circuit indicate that this cell is for us? + * + * Update digest from the payload of cell (with the integrity part set + * to 0). If the integrity part is valid, return 1, else restore digest + * and cell to their original state and return 0. + */ +static int +relay_digest_matches(crypto_digest_t *digest, cell_t *cell) +{ + uint32_t received_integrity, calculated_integrity; + relay_header_t rh; + crypto_digest_checkpoint_t backup_digest; + + crypto_digest_checkpoint(&backup_digest, digest); + + relay_header_unpack(&rh, cell->payload); + memcpy(&received_integrity, rh.integrity, 4); + memset(rh.integrity, 0, 4); + relay_header_pack(cell->payload, &rh); + +// log_fn(LOG_DEBUG,"Reading digest of %u %u %u %u from relay cell.", +// received_integrity[0], received_integrity[1], +// received_integrity[2], received_integrity[3]); + + crypto_digest_add_bytes(digest, (char*) cell->payload, CELL_PAYLOAD_SIZE); + crypto_digest_get_digest(digest, (char*) &calculated_integrity, 4); + + int rv = 1; + + if (calculated_integrity != received_integrity) { +// log_fn(LOG_INFO,"Recognized=0 but bad digest. Not recognizing."); +// (%d vs %d).", received_integrity, calculated_integrity); + /* restore digest to its old form */ + crypto_digest_restore(digest, &backup_digest); + /* restore the relay header */ + memcpy(rh.integrity, &received_integrity, 4); + relay_header_pack(cell->payload, &rh); + rv = 0; + } + + memwipe(&backup_digest, 0, sizeof(backup_digest)); + return rv; +} + +/** Apply <b>cipher</b> to CELL_PAYLOAD_SIZE bytes of <b>in</b> + * (in place). + * + * Note that we use the same operation for encrypting and for decrypting. + */ +static void +relay_crypt_one_payload(crypto_cipher_t *cipher, uint8_t *in) +{ + crypto_cipher_crypt_inplace(cipher, (char*) in, CELL_PAYLOAD_SIZE); +} + +/** Do the appropriate en/decryptions for <b>cell</b> arriving on + * <b>circ</b> in direction <b>cell_direction</b>. + * + * If cell_direction == CELL_DIRECTION_IN: + * - If we're at the origin (we're the OP), for hops 1..N, + * decrypt cell. If recognized, stop. + * - Else (we're not the OP), encrypt one hop. Cell is not recognized. + * + * If cell_direction == CELL_DIRECTION_OUT: + * - decrypt one hop. Check if recognized. + * + * If cell is recognized, set *recognized to 1, and set + * *layer_hint to the hop that recognized it. + * + * Return -1 to indicate that we should mark the circuit for close, + * else return 0. + */ +int +relay_decrypt_cell(circuit_t *circ, cell_t *cell, + cell_direction_t cell_direction, + crypt_path_t **layer_hint, char *recognized) +{ + relay_header_t rh; + + tor_assert(circ); + tor_assert(cell); + tor_assert(recognized); + tor_assert(cell_direction == CELL_DIRECTION_IN || + cell_direction == CELL_DIRECTION_OUT); + + if (cell_direction == CELL_DIRECTION_IN) { + if (CIRCUIT_IS_ORIGIN(circ)) { /* We're at the beginning of the circuit. + * We'll want to do layered decrypts. */ + crypt_path_t *thishop, *cpath = TO_ORIGIN_CIRCUIT(circ)->cpath; + thishop = cpath; + if (thishop->state != CPATH_STATE_OPEN) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Relay cell before first created cell? Closing."); + return -1; + } + do { /* Remember: cpath is in forward order, that is, first hop first. */ + tor_assert(thishop); + + /* decrypt one layer */ + relay_crypt_one_payload(thishop->crypto.b_crypto, cell->payload); + + relay_header_unpack(&rh, cell->payload); + if (rh.recognized == 0) { + /* it's possibly recognized. have to check digest to be sure. */ + if (relay_digest_matches(thishop->crypto.b_digest, cell)) { + *recognized = 1; + *layer_hint = thishop; + return 0; + } + } + + thishop = thishop->next; + } while (thishop != cpath && thishop->state == CPATH_STATE_OPEN); + log_fn(LOG_PROTOCOL_WARN, LD_OR, + "Incoming cell at client not recognized. Closing."); + return -1; + } else { + relay_crypto_t *crypto = &TO_OR_CIRCUIT(circ)->crypto; + /* We're in the middle. Encrypt one layer. */ + relay_crypt_one_payload(crypto->b_crypto, cell->payload); + } + } else /* cell_direction == CELL_DIRECTION_OUT */ { + /* We're in the middle. Decrypt one layer. */ + relay_crypto_t *crypto = &TO_OR_CIRCUIT(circ)->crypto; + + relay_crypt_one_payload(crypto->f_crypto, cell->payload); + + relay_header_unpack(&rh, cell->payload); + if (rh.recognized == 0) { + /* it's possibly recognized. have to check digest to be sure. */ + if (relay_digest_matches(crypto->f_digest, cell)) { + *recognized = 1; + return 0; + } + } + } + return 0; +} + +/** + * Encrypt a cell <b>cell</b> that we are creating, and sending outbound on + * <b>circ</b> until the hop corresponding to <b>layer_hint</b>. + * + * The integrity field and recognized field of <b>cell</b>'s relay headers + * must be set to zero. + */ +void +relay_encrypt_cell_outbound(cell_t *cell, + origin_circuit_t *circ, + crypt_path_t *layer_hint) +{ + crypt_path_t *thishop; /* counter for repeated crypts */ + relay_set_digest(layer_hint->crypto.f_digest, cell); + + thishop = layer_hint; + /* moving from farthest to nearest hop */ + do { + tor_assert(thishop); + log_debug(LD_OR,"encrypting a layer of the relay cell."); + relay_crypt_one_payload(thishop->crypto.f_crypto, cell->payload); + + thishop = thishop->prev; + } while (thishop != circ->cpath->prev); +} + +/** + * Encrypt a cell <b>cell</b> that we are creating, and sending on + * <b>circuit</b> to the origin. + * + * The integrity field and recognized field of <b>cell</b>'s relay headers + * must be set to zero. + */ +void +relay_encrypt_cell_inbound(cell_t *cell, + or_circuit_t *or_circ) +{ + relay_set_digest(or_circ->crypto.b_digest, cell); + /* encrypt one layer */ + relay_crypt_one_payload(or_circ->crypto.b_crypto, cell->payload); +} + +/** + * Release all storage held inside <b>crypto</b>, but do not free + * <b>crypto</b> itself: it lives inside another object. + */ +void +relay_crypto_clear(relay_crypto_t *crypto) +{ + if (BUG(!crypto)) + return; + crypto_cipher_free(crypto->f_crypto); + crypto_cipher_free(crypto->b_crypto); + crypto_digest_free(crypto->f_digest); + crypto_digest_free(crypto->b_digest); +} + +/** Initialize <b>crypto</b> from the key material in key_data. + * + * If <b>is_hs_v3</b> is set, this cpath will be used for next gen hidden + * service circuits and <b>key_data</b> must be at least + * HS_NTOR_KEY_EXPANSION_KDF_OUT_LEN bytes in length. + * + * If <b>is_hs_v3</b> is not set, key_data must contain CPATH_KEY_MATERIAL_LEN + * bytes, which are used as follows: + * - 20 to initialize f_digest + * - 20 to initialize b_digest + * - 16 to key f_crypto + * - 16 to key b_crypto + * + * (If 'reverse' is true, then f_XX and b_XX are swapped.) + * + * Return 0 if init was successful, else -1 if it failed. + */ +int +relay_crypto_init(relay_crypto_t *crypto, + const char *key_data, size_t key_data_len, + int reverse, int is_hs_v3) +{ + crypto_digest_t *tmp_digest; + crypto_cipher_t *tmp_crypto; + size_t digest_len = 0; + size_t cipher_key_len = 0; + + tor_assert(crypto); + tor_assert(key_data); + tor_assert(!(crypto->f_crypto || crypto->b_crypto || + crypto->f_digest || crypto->b_digest)); + + /* Basic key size validation */ + if (is_hs_v3 && BUG(key_data_len != HS_NTOR_KEY_EXPANSION_KDF_OUT_LEN)) { + goto err; + } else if (!is_hs_v3 && BUG(key_data_len != CPATH_KEY_MATERIAL_LEN)) { + goto err; + } + + /* If we are using this crypto for next gen onion services use SHA3-256, + otherwise use good ol' SHA1 */ + if (is_hs_v3) { + digest_len = DIGEST256_LEN; + cipher_key_len = CIPHER256_KEY_LEN; + crypto->f_digest = crypto_digest256_new(DIGEST_SHA3_256); + crypto->b_digest = crypto_digest256_new(DIGEST_SHA3_256); + } else { + digest_len = DIGEST_LEN; + cipher_key_len = CIPHER_KEY_LEN; + crypto->f_digest = crypto_digest_new(); + crypto->b_digest = crypto_digest_new(); + } + + tor_assert(digest_len != 0); + tor_assert(cipher_key_len != 0); + const int cipher_key_bits = (int) cipher_key_len * 8; + + crypto_digest_add_bytes(crypto->f_digest, key_data, digest_len); + crypto_digest_add_bytes(crypto->b_digest, key_data+digest_len, digest_len); + + crypto->f_crypto = crypto_cipher_new_with_bits(key_data+(2*digest_len), + cipher_key_bits); + if (!crypto->f_crypto) { + log_warn(LD_BUG,"Forward cipher initialization failed."); + goto err; + } + + crypto->b_crypto = crypto_cipher_new_with_bits( + key_data+(2*digest_len)+cipher_key_len, + cipher_key_bits); + if (!crypto->b_crypto) { + log_warn(LD_BUG,"Backward cipher initialization failed."); + goto err; + } + + if (reverse) { + tmp_digest = crypto->f_digest; + crypto->f_digest = crypto->b_digest; + crypto->b_digest = tmp_digest; + tmp_crypto = crypto->f_crypto; + crypto->f_crypto = crypto->b_crypto; + crypto->b_crypto = tmp_crypto; + } + + return 0; + err: + relay_crypto_clear(crypto); + return -1; +} + +/** Assert that <b>crypto</b> is valid and set. */ +void +relay_crypto_assert_ok(const relay_crypto_t *crypto) +{ + tor_assert(crypto->f_crypto); + tor_assert(crypto->b_crypto); + tor_assert(crypto->f_digest); + tor_assert(crypto->b_digest); +} + diff --git a/src/or/relay_crypto.h b/src/or/relay_crypto.h new file mode 100644 index 0000000000..66ae02cee9 --- /dev/null +++ b/src/or/relay_crypto.h @@ -0,0 +1,31 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2017, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file relay.h + * \brief Header file for relay.c. + **/ + +#ifndef TOR_RELAY_CRYPTO_H +#define TOR_RELAY_CRYPTO_H + +int relay_crypto_init(relay_crypto_t *crypto, + const char *key_data, size_t key_data_len, + int reverse, int is_hs_v3); + +int relay_decrypt_cell(circuit_t *circ, cell_t *cell, + cell_direction_t cell_direction, + crypt_path_t **layer_hint, char *recognized); +void relay_encrypt_cell_outbound(cell_t *cell, origin_circuit_t *or_circ, + crypt_path_t *layer_hint); +void relay_encrypt_cell_inbound(cell_t *cell, or_circuit_t *or_circ); + +void relay_crypto_clear(relay_crypto_t *crypto); + +void relay_crypto_assert_ok(const relay_crypto_t *crypto); + +#endif /* !defined(TOR_RELAY_CRYPTO_H) */ + diff --git a/src/or/rendclient.c b/src/or/rendclient.c index 57815815b9..9a1b97c6d6 100644 --- a/src/or/rendclient.c +++ b/src/or/rendclient.c @@ -915,8 +915,8 @@ rend_client_desc_trynow(const char *query) /* restart their timeout values, so they get a fair shake at * connecting to the hidden service. */ base_conn->timestamp_created = now; - base_conn->timestamp_lastread = now; - base_conn->timestamp_lastwritten = now; + base_conn->timestamp_last_read_allowed = now; + base_conn->timestamp_last_write_allowed = now; connection_ap_mark_as_pending_circuit(conn); } else { /* 404, or fetch didn't get that far */ diff --git a/src/or/rephist.c b/src/or/rephist.c index 43494692cb..ac3e9f502e 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -86,7 +86,6 @@ #include "ht.h" #include "channelpadding.h" -#include "channelpadding.h" #include "connection_or.h" static void bw_arrays_init(void); diff --git a/src/or/router.c b/src/or/router.c index 9c053cad46..e5996f665e 100644 --- a/src/or/router.c +++ b/src/or/router.c @@ -103,6 +103,13 @@ static authority_cert_t *legacy_key_certificate = NULL; * used by tor-gencert to sign new signing keys and make new key * certificates. */ +const char *format_node_description(char *buf, + const char *id_digest, + int is_named, + const char *nickname, + const tor_addr_t *addr, + uint32_t addr32h); + /** Replace the current onion key with <b>k</b>. Does not affect * lastonionkey; to update lastonionkey correctly, call rotate_onion_key(). */ @@ -1227,7 +1234,8 @@ check_whether_dirport_reachable(const or_options_t *options) /* XXX Should this be increased? */ #define MIN_BW_TO_ADVERTISE_DIRSERVER 51200 -/** Return true iff we have enough configured bandwidth to cache directory +/** Return true iff we have enough configured bandwidth to advertise or + * automatically provide directory services from cache directory * information. */ static int router_has_bandwidth_to_be_dirserver(const or_options_t *options) @@ -1250,7 +1258,7 @@ router_has_bandwidth_to_be_dirserver(const or_options_t *options) * MIN_BW_TO_ADVERTISE_DIRSERVER, don't bother trying to serve requests. */ static int -router_should_be_directory_server(const or_options_t *options, int dir_port) +router_should_be_dirserver(const or_options_t *options, int dir_port) { static int advertising=1; /* start out assuming we will advertise */ int new_choice=1; @@ -1355,7 +1363,7 @@ decide_to_advertise_dir_impl(const or_options_t *options, /* Part two: consider config options that could make us choose to * publish or not publish that the user might find surprising. */ - return router_should_be_directory_server(options, dir_port); + return router_should_be_dirserver(options, dir_port); } /** Front-end to decide_to_advertise_dir_impl(): return 0 if we don't want to @@ -1363,7 +1371,7 @@ decide_to_advertise_dir_impl(const or_options_t *options, * DirPort we want to advertise. */ static int -decide_to_advertise_dirport(const or_options_t *options, uint16_t dir_port) +router_should_advertise_dirport(const or_options_t *options, uint16_t dir_port) { /* supports_tunnelled_dir_requests is not relevant, pass 0 */ return decide_to_advertise_dir_impl(options, dir_port, 0) ? dir_port : 0; @@ -1373,7 +1381,7 @@ decide_to_advertise_dirport(const or_options_t *options, uint16_t dir_port) * advertise the fact that we support begindir requests, else return 1. */ static int -decide_to_advertise_begindir(const or_options_t *options, +router_should_advertise_begindir(const or_options_t *options, int supports_tunnelled_dir_requests) { /* dir_port is not relevant, pass 0 */ @@ -1406,26 +1414,17 @@ extend_info_from_router(const routerinfo_t *r) &ap.addr, ap.port); } -/** Some time has passed, or we just got new directory information. - * See if we currently believe our ORPort or DirPort to be - * unreachable. If so, launch a new test for it. - * - * For ORPort, we simply try making a circuit that ends at ourselves. - * Success is noticed in onionskin_answer(). - * - * For DirPort, we make a connection via Tor to our DirPort and ask - * for our own server descriptor. - * Success is noticed in connection_dir_client_reached_eof(). +/**See if we currently believe our ORPort or DirPort to be + * unreachable. If so, return 1 else return 0. */ -void -consider_testing_reachability(int test_or, int test_dir) +static int +router_should_check_reachability(int test_or, int test_dir) { const routerinfo_t *me = router_get_my_routerinfo(); const or_options_t *options = get_options(); - int orport_reachable = check_whether_orport_reachable(options); - tor_addr_t addr; + if (!me) - return; + return 0; if (routerset_contains_router(options->ExcludeNodes, me, -1) && options->StrictNodes) { @@ -1440,43 +1439,66 @@ consider_testing_reachability(int test_or, int test_dir) "We cannot learn whether we are usable, and will not " "be able to advertise ourself."); } - return; + return 0; } + return 1; +} + +/** Some time has passed, or we just got new directory information. + * See if we currently believe our ORPort or DirPort to be + * unreachable. If so, launch a new test for it. + * + * For ORPort, we simply try making a circuit that ends at ourselves. + * Success is noticed in onionskin_answer(). + * + * For DirPort, we make a connection via Tor to our DirPort and ask + * for our own server descriptor. + * Success is noticed in connection_dir_client_reached_eof(). + */ +void +router_do_reachability_checks(int test_or, int test_dir) +{ + const routerinfo_t *me = router_get_my_routerinfo(); + const or_options_t *options = get_options(); + int orport_reachable = check_whether_orport_reachable(options); + tor_addr_t addr; + + if (router_should_check_reachability(test_or, test_dir)) { + if (test_or && (!orport_reachable || !circuit_enough_testing_circs())) { + extend_info_t *ei = extend_info_from_router(me); + /* XXX IPv6 self testing */ + log_info(LD_CIRC, "Testing %s of my ORPort: %s:%d.", + !orport_reachable ? "reachability" : "bandwidth", + fmt_addr32(me->addr), me->or_port); + circuit_launch_by_extend_info(CIRCUIT_PURPOSE_TESTING, ei, + CIRCLAUNCH_NEED_CAPACITY|CIRCLAUNCH_IS_INTERNAL); + extend_info_free(ei); + } - if (test_or && (!orport_reachable || !circuit_enough_testing_circs())) { - extend_info_t *ei = extend_info_from_router(me); /* XXX IPv6 self testing */ - log_info(LD_CIRC, "Testing %s of my ORPort: %s:%d.", - !orport_reachable ? "reachability" : "bandwidth", - fmt_addr32(me->addr), me->or_port); - circuit_launch_by_extend_info(CIRCUIT_PURPOSE_TESTING, ei, - CIRCLAUNCH_NEED_CAPACITY|CIRCLAUNCH_IS_INTERNAL); - extend_info_free(ei); - } - - /* XXX IPv6 self testing */ - tor_addr_from_ipv4h(&addr, me->addr); - if (test_dir && !check_whether_dirport_reachable(options) && - !connection_get_by_type_addr_port_purpose( - CONN_TYPE_DIR, &addr, me->dir_port, - DIR_PURPOSE_FETCH_SERVERDESC)) { - tor_addr_port_t my_orport, my_dirport; - memcpy(&my_orport.addr, &addr, sizeof(addr)); - memcpy(&my_dirport.addr, &addr, sizeof(addr)); - my_orport.port = me->or_port; - my_dirport.port = me->dir_port; - /* ask myself, via tor, for my server descriptor. */ - directory_request_t *req = - directory_request_new(DIR_PURPOSE_FETCH_SERVERDESC); - directory_request_set_or_addr_port(req, &my_orport); - directory_request_set_dir_addr_port(req, &my_dirport); - directory_request_set_directory_id_digest(req, + tor_addr_from_ipv4h(&addr, me->addr); + if (test_dir && !check_whether_dirport_reachable(options) && + !connection_get_by_type_addr_port_purpose( + CONN_TYPE_DIR, &addr, me->dir_port, + DIR_PURPOSE_FETCH_SERVERDESC)) { + tor_addr_port_t my_orport, my_dirport; + memcpy(&my_orport.addr, &addr, sizeof(addr)); + memcpy(&my_dirport.addr, &addr, sizeof(addr)); + my_orport.port = me->or_port; + my_dirport.port = me->dir_port; + /* ask myself, via tor, for my server descriptor. */ + directory_request_t *req = + directory_request_new(DIR_PURPOSE_FETCH_SERVERDESC); + directory_request_set_or_addr_port(req, &my_orport); + directory_request_set_dir_addr_port(req, &my_dirport); + directory_request_set_directory_id_digest(req, me->cache_info.identity_digest); - // ask via an anon circuit, connecting to our dirport. - directory_request_set_indirection(req, DIRIND_ANON_DIRPORT); - directory_request_set_resource(req, "authority.z"); - directory_initiate_request(req); - directory_request_free(req); + // ask via an anon circuit, connecting to our dirport. + directory_request_set_indirection(req, DIRIND_ANON_DIRPORT); + directory_request_set_resource(req, "authority.z"); + directory_initiate_request(req); + directory_request_free(req); + } } } @@ -1521,7 +1543,7 @@ router_dirport_found_reachable(void) && check_whether_orport_reachable(options) ? " Publishing server descriptor." : ""); can_reach_dir_port = 1; - if (decide_to_advertise_dirport(options, me->dir_port)) { + if (router_should_advertise_dirport(options, me->dir_port)) { mark_my_descriptor_dirty("DirPort found reachable"); /* This is a significant enough change to upload immediately, * at least in a test network */ @@ -2915,14 +2937,14 @@ router_dump_router_to_string(routerinfo_t *router, router->nickname, address, router->or_port, - decide_to_advertise_dirport(options, router->dir_port), + router_should_advertise_dirport(options, router->dir_port), ed_cert_line ? ed_cert_line : "", extra_or_address ? extra_or_address : "", router->platform, proto_line, published, fingerprint, - stats_n_seconds_working, + get_uptime(), (int) router->bandwidthrate, (int) router->bandwidthburst, (int) router->bandwidthcapacity, @@ -2989,7 +3011,7 @@ router_dump_router_to_string(routerinfo_t *router, tor_free(p6); } - if (decide_to_advertise_begindir(options, + if (router_should_advertise_begindir(options, router->supports_tunnelled_dir_requests)) { smartlist_add_strdup(chunks, "tunnelled-dir-server\n"); } @@ -3438,6 +3460,15 @@ is_legal_hexdigest(const char *s) strspn(s,HEX_CHARACTERS)==HEX_DIGEST_LEN); } +/** + * Longest allowed output of format_node_description, plus 1 character for + * NUL. This allows space for: + * "$FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF~xxxxxxxxxxxxxxxxxxx at" + * " [ffff:ffff:ffff:ffff:ffff:ffff:255.255.255.255]" + * plus a terminating NUL. + */ +#define NODE_DESC_BUF_LEN (MAX_VERBOSE_NICKNAME_LEN+4+TOR_ADDR_BUF_LEN) + /** Use <b>buf</b> (which must be at least NODE_DESC_BUF_LEN bytes long) to * hold a human-readable description of a node with identity digest * <b>id_digest</b>, named-status <b>is_named</b>, nickname <b>nickname</b>, @@ -3483,15 +3514,16 @@ format_node_description(char *buf, return buf; } -/** Use <b>buf</b> (which must be at least NODE_DESC_BUF_LEN bytes long) to - * hold a human-readable description of <b>ri</b>. - * +/** Return a human-readable description of the routerinfo_t <b>ri</b>. * - * Return a pointer to the front of <b>buf</b>. + * This function is not thread-safe. Each call to this function invalidates + * previous values returned by this function. */ const char * -router_get_description(char *buf, const routerinfo_t *ri) +router_describe(const routerinfo_t *ri) { + static char buf[NODE_DESC_BUF_LEN]; + if (!ri) return "<null>"; return format_node_description(buf, @@ -3502,14 +3534,15 @@ router_get_description(char *buf, const routerinfo_t *ri) ri->addr); } -/** Use <b>buf</b> (which must be at least NODE_DESC_BUF_LEN bytes long) to - * hold a human-readable description of <b>node</b>. +/** Return a human-readable description of the node_t <b>node</b>. * - * Return a pointer to the front of <b>buf</b>. + * This function is not thread-safe. Each call to this function invalidates + * previous values returned by this function. */ const char * -node_get_description(char *buf, const node_t *node) +node_describe(const node_t *node) { + static char buf[NODE_DESC_BUF_LEN]; const char *nickname = NULL; uint32_t addr32h = 0; int is_named = 0; @@ -3534,14 +3567,16 @@ node_get_description(char *buf, const node_t *node) addr32h); } -/** Use <b>buf</b> (which must be at least NODE_DESC_BUF_LEN bytes long) to - * hold a human-readable description of <b>rs</b>. +/** Return a human-readable description of the routerstatus_t <b>rs</b>. * - * Return a pointer to the front of <b>buf</b>. + * This function is not thread-safe. Each call to this function invalidates + * previous values returned by this function. */ const char * -routerstatus_get_description(char *buf, const routerstatus_t *rs) +routerstatus_describe(const routerstatus_t *rs) { + static char buf[NODE_DESC_BUF_LEN]; + if (!rs) return "<null>"; return format_node_description(buf, @@ -3552,14 +3587,16 @@ routerstatus_get_description(char *buf, const routerstatus_t *rs) rs->addr); } -/** Use <b>buf</b> (which must be at least NODE_DESC_BUF_LEN bytes long) to - * hold a human-readable description of <b>ei</b>. +/** Return a human-readable description of the extend_info_t <b>ei</b>. * - * Return a pointer to the front of <b>buf</b>. + * This function is not thread-safe. Each call to this function invalidates + * previous values returned by this function. */ const char * -extend_info_get_description(char *buf, const extend_info_t *ei) +extend_info_describe(const extend_info_t *ei) { + static char buf[NODE_DESC_BUF_LEN]; + if (!ei) return "<null>"; return format_node_description(buf, @@ -3570,54 +3607,6 @@ extend_info_get_description(char *buf, const extend_info_t *ei) 0); } -/** Return a human-readable description of the routerinfo_t <b>ri</b>. - * - * This function is not thread-safe. Each call to this function invalidates - * previous values returned by this function. - */ -const char * -router_describe(const routerinfo_t *ri) -{ - static char buf[NODE_DESC_BUF_LEN]; - return router_get_description(buf, ri); -} - -/** Return a human-readable description of the node_t <b>node</b>. - * - * This function is not thread-safe. Each call to this function invalidates - * previous values returned by this function. - */ -const char * -node_describe(const node_t *node) -{ - static char buf[NODE_DESC_BUF_LEN]; - return node_get_description(buf, node); -} - -/** Return a human-readable description of the routerstatus_t <b>rs</b>. - * - * This function is not thread-safe. Each call to this function invalidates - * previous values returned by this function. - */ -const char * -routerstatus_describe(const routerstatus_t *rs) -{ - static char buf[NODE_DESC_BUF_LEN]; - return routerstatus_get_description(buf, rs); -} - -/** Return a human-readable description of the extend_info_t <b>ei</b>. - * - * This function is not thread-safe. Each call to this function invalidates - * previous values returned by this function. - */ -const char * -extend_info_describe(const extend_info_t *ei) -{ - static char buf[NODE_DESC_BUF_LEN]; - return extend_info_get_description(buf, ei); -} - /** Set <b>buf</b> (which must have MAX_VERBOSE_NICKNAME_LEN+1 bytes) to the * verbose representation of the identity of <b>router</b>. The format is: * A dollar sign. diff --git a/src/or/router.h b/src/or/router.h index 696e983662..e5efe577e3 100644 --- a/src/or/router.h +++ b/src/or/router.h @@ -47,7 +47,7 @@ int init_keys_client(void); int check_whether_orport_reachable(const or_options_t *options); int check_whether_dirport_reachable(const or_options_t *options); int dir_server_mode(const or_options_t *options); -void consider_testing_reachability(int test_or, int test_dir); +void router_do_reachability_checks(int test_or, int test_dir); void router_orport_found_reachable(void); void router_dirport_found_reachable(void); void router_perform_bandwidth_test(int num_circs, time_t now); @@ -123,24 +123,6 @@ int is_legal_nickname(const char *s); int is_legal_nickname_or_hexdigest(const char *s); int is_legal_hexdigest(const char *s); -/** - * Longest allowed output of format_node_description, plus 1 character for - * NUL. This allows space for: - * "$FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF~xxxxxxxxxxxxxxxxxxx at" - * " [ffff:ffff:ffff:ffff:ffff:ffff:255.255.255.255]" - * plus a terminating NUL. - */ -#define NODE_DESC_BUF_LEN (MAX_VERBOSE_NICKNAME_LEN+4+TOR_ADDR_BUF_LEN) -const char *format_node_description(char *buf, - const char *id_digest, - int is_named, - const char *nickname, - const tor_addr_t *addr, - uint32_t addr32h); -const char *router_get_description(char *buf, const routerinfo_t *ri); -const char *node_get_description(char *buf, const node_t *node); -const char *routerstatus_get_description(char *buf, const routerstatus_t *rs); -const char *extend_info_get_description(char *buf, const extend_info_t *ei); const char *router_describe(const routerinfo_t *ri); const char *node_describe(const node_t *node); const char *routerstatus_describe(const routerstatus_t *ri); diff --git a/src/or/scheduler.c b/src/or/scheduler.c index 382b3e3ca9..da894294bf 100644 --- a/src/or/scheduler.c +++ b/src/or/scheduler.c @@ -13,8 +13,6 @@ #define TOR_CHANNEL_INTERNAL_ #include "channeltls.h" -#include <event2/event.h> - /** * \file scheduler.c * \brief Channel scheduling system: decides which channels should send and @@ -169,7 +167,7 @@ STATIC smartlist_t *channels_pending = NULL; * This event runs the scheduler from its callback, and is manually * activated whenever a channel enters open for writes/cells to send. */ -STATIC struct event *run_sched_ev = NULL; +STATIC struct mainloop_event_t *run_sched_ev = NULL; static int have_logged_kist_suddenly_disabled = 0; @@ -203,10 +201,9 @@ get_scheduler_type_string(scheduler_types_t type) * if any scheduling work was created during the event loop. */ static void -scheduler_evt_callback(evutil_socket_t fd, short events, void *arg) +scheduler_evt_callback(mainloop_event_t *event, void *arg) { - (void) fd; - (void) events; + (void) event; (void) arg; log_debug(LD_SCHED, "Scheduler event callback called"); @@ -487,10 +484,7 @@ scheduler_free_all(void) log_debug(LD_SCHED, "Shutting down scheduler"); if (run_sched_ev) { - if (event_del(run_sched_ev) < 0) { - log_warn(LD_BUG, "Problem deleting run_sched_ev"); - } - tor_event_free(run_sched_ev); + mainloop_event_free(run_sched_ev); run_sched_ev = NULL; } @@ -589,7 +583,7 @@ scheduler_ev_add(const struct timeval *next_run) { tor_assert(run_sched_ev); tor_assert(next_run); - if (BUG(event_add(run_sched_ev, next_run) < 0)) { + if (BUG(mainloop_event_schedule(run_sched_ev, next_run) < 0)) { log_warn(LD_SCHED, "Adding to libevent failed. Next run time was set to: " "%ld.%06ld", next_run->tv_sec, (long)next_run->tv_usec); return; @@ -598,10 +592,10 @@ scheduler_ev_add(const struct timeval *next_run) /** Make the scheduler event active with the given flags. */ void -scheduler_ev_active(int flags) +scheduler_ev_active(void) { tor_assert(run_sched_ev); - event_active(run_sched_ev, flags, 1); + mainloop_event_activate(run_sched_ev); } /* @@ -618,11 +612,10 @@ scheduler_init(void) IF_BUG_ONCE(!!run_sched_ev) { log_warn(LD_SCHED, "We should not already have a libevent scheduler event." "I'll clean the old one up, but this is odd."); - tor_event_free(run_sched_ev); + mainloop_event_free(run_sched_ev); run_sched_ev = NULL; } - run_sched_ev = tor_event_new(tor_libevent_get_base(), -1, - 0, scheduler_evt_callback, NULL); + run_sched_ev = mainloop_event_new(scheduler_evt_callback, NULL); channels_pending = smartlist_new(); set_scheduler(); diff --git a/src/or/scheduler.h b/src/or/scheduler.h index aeba9e2b75..08b02e286f 100644 --- a/src/or/scheduler.h +++ b/src/or/scheduler.h @@ -155,12 +155,12 @@ void scheduler_bug_occurred(const channel_t *chan); smartlist_t *get_channels_pending(void); MOCK_DECL(int, scheduler_compare_channels, (const void *c1_v, const void *c2_v)); -void scheduler_ev_active(int flags); +void scheduler_ev_active(void); void scheduler_ev_add(const struct timeval *next_run); #ifdef TOR_UNIT_TESTS extern smartlist_t *channels_pending; -extern struct event *run_sched_ev; +extern struct mainloop_event_t *run_sched_ev; extern const scheduler_t *the_scheduler; void scheduler_touch_channel(channel_t *chan); #endif /* defined(TOR_UNIT_TESTS) */ diff --git a/src/or/scheduler_kist.c b/src/or/scheduler_kist.c index 6d6490077d..c6e9b72c48 100644 --- a/src/or/scheduler_kist.c +++ b/src/or/scheduler_kist.c @@ -3,8 +3,6 @@ #define SCHEDULER_KIST_PRIVATE -#include <event2/event.h> - #include "or.h" #include "buffers.h" #include "config.h" @@ -553,7 +551,7 @@ kist_scheduler_schedule(void) /* Re-adding an event reschedules it. It does not duplicate it. */ scheduler_ev_add(&next_run); } else { - scheduler_ev_active(EV_TIMEOUT); + scheduler_ev_active(); } } diff --git a/src/or/scheduler_vanilla.c b/src/or/scheduler_vanilla.c index 7a83b9da18..b674d8256c 100644 --- a/src/or/scheduler_vanilla.c +++ b/src/or/scheduler_vanilla.c @@ -1,8 +1,6 @@ /* Copyright (c) 2017, The Tor Project, Inc. */ /* See LICENSE for licensing information */ -#include <event2/event.h> - #include "or.h" #include "config.h" #define TOR_CHANNEL_INTERNAL_ @@ -42,7 +40,7 @@ vanilla_scheduler_schedule(void) } /* Activate our event so it can process channels. */ - scheduler_ev_active(EV_TIMEOUT); + scheduler_ev_active(); } static void diff --git a/src/or/status.c b/src/or/status.c index 4f7be164b1..4c497739e8 100644 --- a/src/or/status.c +++ b/src/or/status.c @@ -25,7 +25,6 @@ #include "main.h" #include "rephist.h" #include "hibernate.h" -#include "rephist.h" #include "statefile.h" #include "hs_stats.h" #include "hs_service.h" diff --git a/src/or/transports.c b/src/or/transports.c index b08dcd1613..614fc81da8 100644 --- a/src/or/transports.c +++ b/src/or/transports.c @@ -135,7 +135,7 @@ static smartlist_t *transport_list = NULL; /** Returns a transport_t struct for a transport proxy supporting the protocol <b>name</b> listening at <b>addr</b>:<b>port</b> using SOCKS version <b>socks_ver</b>. */ -static transport_t * +STATIC transport_t * transport_new(const tor_addr_t *addr, uint16_t port, const char *name, int socks_ver, const char *extra_info_args) @@ -222,8 +222,8 @@ transport_copy(const transport_t *transport) /** Returns the transport in our transport list that has the name <b>name</b>. * Else returns NULL. */ -transport_t * -transport_get_by_name(const char *name) +MOCK_IMPL(transport_t *, +transport_get_by_name,(const char *name)) { tor_assert(name); @@ -1025,48 +1025,71 @@ parse_method_error(const char *line, int is_server) line+strlen(error)+1); } -/** Parses an SMETHOD <b>line</b> and if well-formed it registers the - * new transport in <b>mp</b>. */ -STATIC int -parse_smethod_line(const char *line, managed_proxy_t *mp) +/** A helper for parse_{c,s}method_line(), bootstraps its + * functionalities. If <b>is_smethod</b> is true then the + * the line to parse is a SMETHOD line otherwise it is a + * CMETHOD line*/ +static int +parse_method_line_helper(const char *line, + managed_proxy_t *mp, + int is_smethod) { + int item_index = 0; int r; - smartlist_t *items = NULL; - char *method_name=NULL; + char *transport_name=NULL; char *args_string=NULL; char *addrport=NULL; - tor_addr_t tor_addr; + int socks_ver=PROXY_NONE; char *address=NULL; uint16_t port = 0; + const char *method_str = is_smethod ? PROTO_SMETHOD : PROTO_CMETHOD; + const int min_args_count = is_smethod ? 3 : 4; + + tor_addr_t tor_addr; transport_t *transport=NULL; + smartlist_t *items= smartlist_new(); - items = smartlist_new(); smartlist_split_string(items, line, NULL, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); - if (smartlist_len(items) < 3) { - log_warn(LD_CONFIG, "Server managed proxy sent us a SMETHOD line " - "with too few arguments."); + if (smartlist_len(items) < min_args_count) { + log_warn(LD_CONFIG, "Managed proxy sent us a %s line " + "with too few arguments.", method_str); goto err; } - /* Example of legit SMETHOD line: - SMETHOD obfs2 0.0.0.0:25612 ARGS:secret=supersekrit,key=superkey */ - - tor_assert(!strcmp(smartlist_get(items,0),PROTO_SMETHOD)); + tor_assert(!strcmp(smartlist_get(items, item_index),method_str)); + ++item_index; - method_name = smartlist_get(items,1); - if (!string_is_C_identifier(method_name)) { + transport_name = smartlist_get(items,item_index); + ++item_index; + if (!string_is_C_identifier(transport_name)) { log_warn(LD_CONFIG, "Transport name is not a C identifier (%s).", - method_name); + transport_name); goto err; } - addrport = smartlist_get(items, 2); + /** Check for the proxy method sent to us in CMETHOD line. */ + if (!is_smethod) { + const char *socks_ver_str = smartlist_get(items,item_index); + ++item_index; + + if (!strcmp(socks_ver_str,"socks4")) { + socks_ver = PROXY_SOCKS4; + } else if (!strcmp(socks_ver_str,"socks5")) { + socks_ver = PROXY_SOCKS5; + } else { + log_warn(LD_CONFIG, "Client managed proxy sent us a proxy protocol " + "we don't recognize. (%s)", socks_ver_str); + goto err; + } + } + + addrport = smartlist_get(items, item_index); + ++item_index; if (tor_addr_port_split(LOG_WARN, addrport, &address, &port)<0) { - log_warn(LD_CONFIG, "Error parsing transport " - "address '%s'", addrport); + log_warn(LD_CONFIG, "Error parsing transport address '%s'", addrport); goto err; } @@ -1081,10 +1104,11 @@ parse_smethod_line(const char *line, managed_proxy_t *mp) goto err; } - if (smartlist_len(items) > 3) { + /** Check for options in the SMETHOD line. */ + if (is_smethod && smartlist_len(items) > min_args_count) { /* Seems like there are also some [options] in the SMETHOD line. Let's see if we can parse them. */ - char *options_string = smartlist_get(items, 3); + char *options_string = smartlist_get(items, item_index); log_debug(LD_CONFIG, "Got options_string: %s", options_string); if (!strcmpstart(options_string, "ARGS:")) { args_string = options_string+strlen("ARGS:"); @@ -1092,15 +1116,20 @@ parse_smethod_line(const char *line, managed_proxy_t *mp) } } - transport = transport_new(&tor_addr, port, method_name, - PROXY_NONE, args_string); + transport = transport_new(&tor_addr, port, transport_name, + socks_ver, args_string); smartlist_add(mp->transports, transport); - /* For now, notify the user so that they know where the server - transport is listening. */ - log_info(LD_CONFIG, "Server transport %s at %s:%d.", - method_name, address, (int)port); + /** Logs info about line parsing success for client or server */ + if (is_smethod) { + log_info(LD_CONFIG, "Server transport %s at %s:%d.", + transport_name, address, (int)port); + } else { + log_info(LD_CONFIG, "Transport %s at %s:%d with SOCKS %d. " + "Attached to managed proxy.", + transport_name, address, (int)port, socks_ver); + } r=0; goto done; @@ -1115,93 +1144,24 @@ parse_smethod_line(const char *line, managed_proxy_t *mp) return r; } +/** Parses an SMETHOD <b>line</b> and if well-formed it registers the + * new transport in <b>mp</b>. */ +STATIC int +parse_smethod_line(const char *line, managed_proxy_t *mp) +{ + /* Example of legit SMETHOD line: + SMETHOD obfs2 0.0.0.0:25612 ARGS:secret=supersekrit,key=superkey */ + return parse_method_line_helper(line, mp, 1); +} + /** Parses a CMETHOD <b>line</b>, and if well-formed it registers * the new transport in <b>mp</b>. */ STATIC int parse_cmethod_line(const char *line, managed_proxy_t *mp) { - int r; - smartlist_t *items = NULL; - - char *method_name=NULL; - - char *socks_ver_str=NULL; - int socks_ver=PROXY_NONE; - - char *addrport=NULL; - tor_addr_t tor_addr; - char *address=NULL; - uint16_t port = 0; - - transport_t *transport=NULL; - - items = smartlist_new(); - smartlist_split_string(items, line, NULL, - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); - if (smartlist_len(items) < 4) { - log_warn(LD_CONFIG, "Client managed proxy sent us a CMETHOD line " - "with too few arguments."); - goto err; - } - - tor_assert(!strcmp(smartlist_get(items,0),PROTO_CMETHOD)); - - method_name = smartlist_get(items,1); - if (!string_is_C_identifier(method_name)) { - log_warn(LD_CONFIG, "Transport name is not a C identifier (%s).", - method_name); - goto err; - } - - socks_ver_str = smartlist_get(items,2); - - if (!strcmp(socks_ver_str,"socks4")) { - socks_ver = PROXY_SOCKS4; - } else if (!strcmp(socks_ver_str,"socks5")) { - socks_ver = PROXY_SOCKS5; - } else { - log_warn(LD_CONFIG, "Client managed proxy sent us a proxy protocol " - "we don't recognize. (%s)", socks_ver_str); - goto err; - } - - addrport = smartlist_get(items, 3); - if (tor_addr_port_split(LOG_WARN, addrport, &address, &port)<0) { - log_warn(LD_CONFIG, "Error parsing transport " - "address '%s'", addrport); - goto err; - } - - if (!port) { - log_warn(LD_CONFIG, - "Transport address '%s' has no port.", addrport); - goto err; - } - - if (tor_addr_parse(&tor_addr, address) < 0) { - log_warn(LD_CONFIG, "Error parsing transport address '%s'", address); - goto err; - } - - transport = transport_new(&tor_addr, port, method_name, socks_ver, NULL); - - smartlist_add(mp->transports, transport); - - log_info(LD_CONFIG, "Transport %s at %s:%d with SOCKS %d. " - "Attached to managed proxy.", - method_name, address, (int)port, socks_ver); - - r=0; - goto done; - - err: - r = -1; - - done: - SMARTLIST_FOREACH(items, char*, s, tor_free(s)); - smartlist_free(items); - tor_free(address); - return r; + /* Example of legit CMETHOD line: + CMETHOD obfs2 socks5 127.0.0.1:35713 */ + return parse_method_line_helper(line, mp, 0); } /** Parses an PROXY-ERROR <b>line</b> and warns the user accordingly. */ diff --git a/src/or/transports.h b/src/or/transports.h index 1b2786472c..022b926a03 100644 --- a/src/or/transports.h +++ b/src/or/transports.h @@ -38,7 +38,7 @@ MOCK_DECL(int, transport_add_from_config, void transport_free_(transport_t *transport); #define transport_free(tr) FREE_AND_NULL(transport_t, transport_free_, (tr)) -transport_t *transport_get_by_name(const char *name); +MOCK_DECL(transport_t*, transport_get_by_name, (const char *name)); MOCK_DECL(void, pt_kickstart_proxy, (const smartlist_t *transport_list, char **proxy_argv, @@ -113,6 +113,9 @@ typedef struct { smartlist_t *transports; } managed_proxy_t; +STATIC transport_t *transport_new(const tor_addr_t *addr, uint16_t port, + const char *name, int socks_ver, + const char *extra_info_args); STATIC int parse_cmethod_line(const char *line, managed_proxy_t *mp); STATIC int parse_smethod_line(const char *line, managed_proxy_t *mp); diff --git a/src/rust/Cargo.lock b/src/rust/Cargo.lock index 4f918c0221..91c0502c60 100644 --- a/src/rust/Cargo.lock +++ b/src/rust/Cargo.lock @@ -18,6 +18,7 @@ dependencies = [ "libc 0.2.39 (registry+https://github.com/rust-lang/crates.io-index)", "smartlist 0.0.1", "tor_allocate 0.0.1", + "tor_log 0.1.0", "tor_util 0.0.1", ] @@ -36,6 +37,14 @@ dependencies = [ ] [[package]] +name = "tor_log" +version = "0.1.0" +dependencies = [ + "libc 0.2.39 (registry+https://github.com/rust-lang/crates.io-index)", + "tor_allocate 0.0.1", +] + +[[package]] name = "tor_rust" version = "0.1.0" dependencies = [ @@ -49,6 +58,7 @@ version = "0.0.1" dependencies = [ "libc 0.2.39 (registry+https://github.com/rust-lang/crates.io-index)", "tor_allocate 0.0.1", + "tor_log 0.1.0", ] [metadata] diff --git a/src/rust/Cargo.toml b/src/rust/Cargo.toml index 953c9b96b7..4ae8033eb3 100644 --- a/src/rust/Cargo.toml +++ b/src/rust/Cargo.toml @@ -1,5 +1,6 @@ [workspace] -members = ["tor_util", "protover", "smartlist", "external", "tor_allocate", "tor_rust"] +members = ["tor_util", "protover", "smartlist", "external", "tor_allocate", +"tor_rust", "tor_log"] [profile.release] debug = true diff --git a/src/rust/include.am b/src/rust/include.am index 7a0181e373..f1aa0bd5ac 100644 --- a/src/rust/include.am +++ b/src/rust/include.am @@ -20,6 +20,9 @@ EXTRA_DIST +=\ src/rust/tor_allocate/Cargo.toml \ src/rust/tor_allocate/lib.rs \ src/rust/tor_allocate/tor_allocate.rs \ + src/rust/tor_log/Cargo.toml \ + src/rust/tor_log/lib.rs \ + src/rust/tor_log/tor_log.rs \ src/rust/tor_rust/Cargo.toml \ src/rust/tor_rust/include.am \ src/rust/tor_rust/lib.rs \ diff --git a/src/rust/protover/Cargo.toml b/src/rust/protover/Cargo.toml index 86301b8787..af1089c914 100644 --- a/src/rust/protover/Cargo.toml +++ b/src/rust/protover/Cargo.toml @@ -3,6 +3,9 @@ authors = ["The Tor Project"] version = "0.0.1" name = "protover" +[features] +testing = ["tor_log/testing"] + [dependencies] libc = "=0.2.39" @@ -18,6 +21,9 @@ path = "../tor_util" [dependencies.tor_allocate] path = "../tor_allocate" +[dependencies.tor_log] +path = "../tor_log" + [lib] name = "protover" path = "lib.rs" diff --git a/src/rust/protover/ffi.rs b/src/rust/protover/ffi.rs index a40353eb13..2dfeda87b2 100644 --- a/src/rust/protover/ffi.rs +++ b/src/rust/protover/ffi.rs @@ -11,9 +11,6 @@ use std::ffi::CString; use smartlist::*; use tor_allocate::allocate_and_copy_string; -use tor_util::strings::byte_slice_is_c_like; -use tor_util::strings::empty_static_cstr; - use errors::ProtoverError; use protover::*; @@ -158,18 +155,7 @@ pub extern "C" fn protocol_list_supports_protocol_or_later( pub extern "C" fn protover_get_supported_protocols() -> *const c_char { let supported: &'static CStr; - // If we're going to pass it to C, there cannot be any intermediate NUL - // bytes. An assert is okay here, since changing the const byte slice - // in protover.rs to contain a NUL byte somewhere in the middle would be a - // programming error. - assert!(byte_slice_is_c_like(SUPPORTED_PROTOCOLS)); - - // It's okay to unwrap the result of this function because - // we can see that the bytes we're passing into it 1) are valid UTF-8, - // 2) have no intermediate NUL bytes, and 3) are terminated with a NUL - // byte. - supported = CStr::from_bytes_with_nul(SUPPORTED_PROTOCOLS).unwrap(); - + supported = get_supported_protocols_cstr(); supported.as_ptr() } @@ -228,10 +214,9 @@ pub extern "C" fn protover_is_supported_here( #[no_mangle] pub extern "C" fn protover_compute_for_old_tor(version: *const c_char) -> *const c_char { let supported: &'static CStr; - let elder_protocols: &'static [u8]; let empty: &'static CStr; - empty = empty_static_cstr(); + empty = cstr!(""); if version.is_null() { return empty.as_ptr(); @@ -246,19 +231,6 @@ pub extern "C" fn protover_compute_for_old_tor(version: *const c_char) -> *const Err(_) => return empty.as_ptr(), }; - elder_protocols = compute_for_old_tor_cstr(&version); - - // If we're going to pass it to C, there cannot be any intermediate NUL - // bytes. An assert is okay here, since changing the const byte slice - // in protover.rs to contain a NUL byte somewhere in the middle would be a - // programming error. - assert!(byte_slice_is_c_like(elder_protocols)); - - // It's okay to unwrap the result of this function because - // we can see that the bytes we're passing into it 1) are valid UTF-8, - // 2) have no intermediate NUL bytes, and 3) are terminated with a NUL - // byte. - supported = CStr::from_bytes_with_nul(elder_protocols).unwrap_or(empty); - + supported = compute_for_old_tor_cstr(&version); supported.as_ptr() } diff --git a/src/rust/protover/lib.rs b/src/rust/protover/lib.rs index 483260bca8..ce964196fd 100644 --- a/src/rust/protover/lib.rs +++ b/src/rust/protover/lib.rs @@ -28,6 +28,7 @@ extern crate libc; extern crate smartlist; extern crate external; extern crate tor_allocate; +#[macro_use] extern crate tor_util; pub mod errors; diff --git a/src/rust/protover/protover.rs b/src/rust/protover/protover.rs index 5e5a31cd33..514aeffc58 100644 --- a/src/rust/protover/protover.rs +++ b/src/rust/protover/protover.rs @@ -3,12 +3,12 @@ use std::collections::HashMap; use std::collections::hash_map; +use std::ffi::CStr; use std::fmt; use std::str; use std::str::FromStr; use std::string::String; -use tor_util::strings::NUL_BYTE; use external::c_tor_version_as_new_as; use errors::ProtoverError; @@ -28,30 +28,6 @@ const FIRST_TOR_VERSION_TO_ADVERTISE_PROTOCOLS: &'static str = "0.2.9.3-alpha"; /// C_RUST_COUPLED: src/or/protover.c `MAX_PROTOCOLS_TO_EXPAND` const MAX_PROTOCOLS_TO_EXPAND: usize = (1<<16); -/// Currently supported protocols and their versions, as a byte-slice. -/// -/// # Warning -/// -/// This byte-slice ends in a NUL byte. This is so that we can directly convert -/// it to an `&'static CStr` in the FFI code, in order to hand the static string -/// to C in a way that is compatible with C static strings. -/// -/// Rust code which wishes to accesses this string should use -/// `protover::get_supported_protocols()` instead. -/// -/// C_RUST_COUPLED: src/or/protover.c `protover_get_supported_protocols` -pub(crate) const SUPPORTED_PROTOCOLS: &'static [u8] = - b"Cons=1-2 \ - Desc=1-2 \ - DirCache=1-2 \ - HSDir=1-2 \ - HSIntro=3-4 \ - HSRend=1-2 \ - Link=1-5 \ - LinkAuth=1,3 \ - Microdesc=1-2 \ - Relay=1-2\0"; - /// Known subprotocols in Tor. Indicates which subprotocol a relay supports. /// /// C_RUST_COUPLED: src/or/protover.h `protocol_type_t` @@ -124,21 +100,33 @@ impl From<Protocol> for UnknownProtocol { } } -/// Get the string representation of current supported protocols +/// Get a CStr representation of current supported protocols, for +/// passing to C, or for converting to a `&str` for Rust. /// /// # Returns /// -/// A `String` whose value is the existing protocols supported by tor. +/// An `&'static CStr` whose value is the existing protocols supported by tor. /// Returned data is in the format as follows: /// /// "HSDir=1-1 LinkAuth=1" /// -pub fn get_supported_protocols() -> &'static str { - // The `len() - 1` is to remove the NUL byte. - // The `unwrap` is safe becauase we SUPPORTED_PROTOCOLS is under - // our control. - str::from_utf8(&SUPPORTED_PROTOCOLS[..SUPPORTED_PROTOCOLS.len() - 1]) - .unwrap_or("") +/// # Note +/// +/// Rust code can use the `&'static CStr` as a normal `&'a str` by +/// calling `protover::get_supported_protocols`. +/// +// C_RUST_COUPLED: src/or/protover.c `protover_get_supported_protocols` +pub(crate) fn get_supported_protocols_cstr() -> &'static CStr { + cstr!("Cons=1-2 \ + Desc=1-2 \ + DirCache=1-2 \ + HSDir=1-2 \ + HSIntro=3-4 \ + HSRend=1-2 \ + Link=1-5 \ + LinkAuth=1,3 \ + Microdesc=1-2 \ + Relay=1-2") } /// A map of protocol names to the versions of them which are supported. @@ -161,7 +149,8 @@ impl ProtoEntry { /// ProtoEntry, which is useful when looking up a specific /// subprotocol. pub fn supported() -> Result<Self, ProtoverError> { - let supported: &'static str = get_supported_protocols(); + let supported_cstr: &'static CStr = get_supported_protocols_cstr(); + let supported: &str = supported_cstr.to_str().unwrap_or(""); supported.parse() } @@ -627,7 +616,7 @@ pub fn is_supported_here(proto: &Protocol, vers: &Version) -> bool { /// /// # Returns /// -/// A `&'static [u8]` encoding a list of protocol names and supported +/// A `&'static CStr` encoding a list of protocol names and supported /// versions. The string takes the following format: /// /// "HSDir=1-1 LinkAuth=1" @@ -643,24 +632,25 @@ pub fn is_supported_here(proto: &Protocol, vers: &Version) -> bool { /// like to use this code in Rust, please see `compute_for_old_tor()`. // // C_RUST_COUPLED: src/rust/protover.c `compute_for_old_tor` -pub(crate) fn compute_for_old_tor_cstr(version: &str) -> &'static [u8] { +pub(crate) fn compute_for_old_tor_cstr(version: &str) -> &'static CStr { + let empty: &'static CStr = cstr!(""); + if c_tor_version_as_new_as(version, FIRST_TOR_VERSION_TO_ADVERTISE_PROTOCOLS) { - return NUL_BYTE; + return empty; } if c_tor_version_as_new_as(version, "0.2.9.1-alpha") { - return b"Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 \ - Link=1-4 LinkAuth=1 Microdesc=1-2 Relay=1-2\0"; + return cstr!("Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 \ + Link=1-4 LinkAuth=1 Microdesc=1-2 Relay=1-2"); } if c_tor_version_as_new_as(version, "0.2.7.5") { - return b"Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1 \ - Link=1-4 LinkAuth=1 Microdesc=1-2 Relay=1-2\0"; + return cstr!("Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1 \ + Link=1-4 LinkAuth=1 Microdesc=1-2 Relay=1-2"); } if c_tor_version_as_new_as(version, "0.2.4.19") { - return b"Cons=1 Desc=1 DirCache=1 HSDir=1 HSIntro=3 HSRend=1 \ - Link=1-4 LinkAuth=1 Microdesc=1 Relay=1-2\0"; + return cstr!("Cons=1 Desc=1 DirCache=1 HSDir=1 HSIntro=3 HSRend=1 \ + Link=1-4 LinkAuth=1 Microdesc=1 Relay=1-2"); } - - NUL_BYTE + empty } /// Since older versions of Tor cannot infer their own subprotocols, @@ -691,14 +681,9 @@ pub(crate) fn compute_for_old_tor_cstr(version: &str) -> &'static [u8] { // // C_RUST_COUPLED: src/rust/protover.c `compute_for_old_tor` pub fn compute_for_old_tor(version: &str) -> Result<&'static str, ProtoverError> { - let mut computed: &'static [u8] = compute_for_old_tor_cstr(version); - - // Remove the NULL byte at the end. - computed = &computed[..computed.len() - 1]; - - // .from_utf8() fails with a Utf8Error if it couldn't validate the + // .to_str() fails with a Utf8Error if it couldn't validate the // utf-8, so convert that here into an Unparseable ProtoverError. - str::from_utf8(computed).or(Err(ProtoverError::Unparseable)) + compute_for_old_tor_cstr(version).to_str().or(Err(ProtoverError::Unparseable)) } #[cfg(test)] diff --git a/src/rust/tor_allocate/tor_allocate.rs b/src/rust/tor_allocate/tor_allocate.rs index 359df1cd7a..3c0037f139 100644 --- a/src/rust/tor_allocate/tor_allocate.rs +++ b/src/rust/tor_allocate/tor_allocate.rs @@ -1,12 +1,17 @@ // Copyright (c) 2016-2017, The Tor Project, Inc. */ // See LICENSE for licensing information */ +// No-op defined purely for testing at the module level +use libc::c_char; -use libc::{c_char, c_void}; +#[cfg(not(feature = "testing"))] use std::{ptr, slice, mem}; +use libc::c_void; -#[cfg(not(test))] -extern "C" { - fn tor_malloc_(size: usize) -> *mut c_void; +// Define a no-op implementation for testing Rust modules without linking to C +#[cfg(feature = "testing")] +pub fn allocate_and_copy_string(s: &String) -> *mut c_char { + use std::ffi::CString; + CString::new(s.as_str()).unwrap().into_raw() } // Defined only for tests, used for testing purposes, so that we don't need @@ -17,6 +22,11 @@ unsafe extern "C" fn tor_malloc_(size: usize) -> *mut c_void { malloc(size) } +#[cfg(all(not(test), not(feature = "testing")))] +extern "C" { + fn tor_malloc_(size: usize) -> *mut c_void; +} + /// Allocate memory using tor_malloc_ and copy an existing string into the /// allocated buffer, returning a pointer that can later be called in C. /// @@ -28,6 +38,7 @@ unsafe extern "C" fn tor_malloc_(size: usize) -> *mut c_void { /// /// A `*mut c_char` that should be freed by tor_free in C /// +#[cfg(not(feature = "testing"))] pub fn allocate_and_copy_string(src: &String) -> *mut c_char { let bytes: &[u8] = src.as_bytes(); diff --git a/src/rust/tor_log/Cargo.toml b/src/rust/tor_log/Cargo.toml new file mode 100644 index 0000000000..971cd658b1 --- /dev/null +++ b/src/rust/tor_log/Cargo.toml @@ -0,0 +1,18 @@ +[package] +name = "tor_log" +version = "0.1.0" +authors = ["The Tor Project"] + +[lib] +name = "tor_log" +path = "lib.rs" +crate_type = ["rlib", "staticlib"] + +[features] +testing = [] + +[dependencies] +libc = "0.2.39" + +[dependencies.tor_allocate] +path = "../tor_allocate" diff --git a/src/rust/tor_log/lib.rs b/src/rust/tor_log/lib.rs new file mode 100644 index 0000000000..72f9e38339 --- /dev/null +++ b/src/rust/tor_log/lib.rs @@ -0,0 +1,16 @@ +//! Copyright (c) 2016-2017, The Tor Project, Inc. */ +//! See LICENSE for licensing information */ + +//! Logging wrapper for Rust to utilize Tor's logger, found at +//! src/common/log.c and src/common/torlog.h +//! +//! Exposes different interfaces depending on whether we are running in test +//! or non-test mode. When testing, we use a no-op implementation, +//! otherwise we link directly to C. + +extern crate libc; +extern crate tor_allocate; + +mod tor_log; + +pub use tor_log::*; diff --git a/src/rust/tor_log/tor_log.rs b/src/rust/tor_log/tor_log.rs new file mode 100644 index 0000000000..1fdc0026bf --- /dev/null +++ b/src/rust/tor_log/tor_log.rs @@ -0,0 +1,270 @@ +// Copyright (c) 2016-2017, The Tor Project, Inc. */ +// See LICENSE for licensing information */ + +// Note that these functions are untested due to the fact that there are no +// return variables to test and they are calling into a C API. + +/// The related domain which the logging message is relevant. For example, +/// log messages relevant to networking would use LogDomain::LdNet, whereas +/// general messages can use LdGeneral. +#[derive(Eq, PartialEq)] +pub enum LogDomain { + Net, + General, +} + +/// The severity level at which to log messages. +#[derive(Eq, PartialEq)] +pub enum LogSeverity { + Notice, + Warn, +} + +/// Main entry point for Rust modules to log messages. +/// +/// # Inputs +/// +/// * A `severity` of type LogSeverity, which defines the level of severity the +/// message will be logged. +/// * A `domain` of type LogDomain, which defines the domain the log message +/// will be associated with. +/// * A `function` of type &str, which defines the name of the function where +/// the message is being logged. There is a current RFC for a macro that +/// defines function names. When it is, we should use it. See +/// https://github.com/rust-lang/rfcs/pull/1719 +/// * A `message` of type &str, which is the log message itself. +#[macro_export] +macro_rules! tor_log_msg { + ($severity: path, + $domain: path, + $function: expr, + $($message:tt)*) => + { + { + let msg = format!($($message)*); + $crate::tor_log_msg_impl($severity, $domain, $function, msg) + } + }; +} + +#[inline] +pub fn tor_log_msg_impl( + severity: LogSeverity, + domain: LogDomain, + function: &str, + message: String, +) { + use std::ffi::CString; + + /// Default function name to log in case of errors when converting + /// a function name to a CString + const ERR_LOG_FUNCTION: &str = "tor_log_msg"; + + /// Default message to log in case of errors when converting a log + /// message to a CString + const ERR_LOG_MSG: &str = "Unable to log message from Rust \ + module due to error when converting to CString"; + + let func = match CString::new(function) { + Ok(n) => n, + Err(_) => CString::new(ERR_LOG_FUNCTION).unwrap(), + }; + + let msg = match CString::new(message) { + Ok(n) => n, + Err(_) => CString::new(ERR_LOG_MSG).unwrap(), + }; + + // Bind to a local variable to preserve ownership. This is essential so + // that ownership is guaranteed until these local variables go out of scope + let func_ptr = func.as_ptr(); + let msg_ptr = msg.as_ptr(); + + let c_severity = unsafe { log::translate_severity(severity) }; + let c_domain = unsafe { log::translate_domain(domain) }; + + unsafe { log::tor_log_string(c_severity, c_domain, func_ptr, msg_ptr) } +} + +/// This implementation is used when compiling for actual use, as opposed to +/// testing. +#[cfg(all(not(test), not(feature = "testing")))] +pub mod log { + use libc::{c_char, c_int}; + use super::LogDomain; + use super::LogSeverity; + + /// Severity log types. These mirror definitions in /src/common/torlog.h + /// C_RUST_COUPLED: src/common/log.c, log domain types + extern "C" { + static LOG_WARN_: c_int; + static LOG_NOTICE_: c_int; + } + + /// Domain log types. These mirror definitions in /src/common/torlog.h + /// C_RUST_COUPLED: src/common/log.c, log severity types + extern "C" { + static LD_NET_: u32; + static LD_GENERAL_: u32; + } + + /// Translate Rust defintions of log domain levels to C. This exposes a 1:1 + /// mapping between types. + #[inline] + pub unsafe fn translate_domain(domain: LogDomain) -> u32 { + match domain { + LogDomain::Net => LD_NET_, + LogDomain::General => LD_GENERAL_, + } + } + + /// Translate Rust defintions of log severity levels to C. This exposes a + /// 1:1 mapping between types. + #[inline] + pub unsafe fn translate_severity(severity: LogSeverity) -> c_int { + match severity { + LogSeverity::Warn => LOG_WARN_, + LogSeverity::Notice => LOG_NOTICE_, + } + } + + /// The main entry point into Tor's logger. When in non-test mode, this + /// will link directly with `tor_log_string` in /src/or/log.c + extern "C" { + pub fn tor_log_string( + severity: c_int, + domain: u32, + function: *const c_char, + string: *const c_char, + ); + } +} + +/// This module exposes no-op functionality for testing other Rust modules +/// without linking to C. +#[cfg(any(test, feature = "testing"))] +pub mod log { + use libc::{c_char, c_int}; + use super::LogDomain; + use super::LogSeverity; + + pub static mut LAST_LOGGED_FUNCTION: *mut String = 0 as *mut String; + pub static mut LAST_LOGGED_MESSAGE: *mut String = 0 as *mut String; + + pub unsafe fn tor_log_string( + _severity: c_int, + _domain: u32, + function: *const c_char, + message: *const c_char, + ) { + use std::ffi::CStr; + + let f = CStr::from_ptr(function); + let fct = match f.to_str() { + Ok(n) => n, + Err(_) => "", + }; + LAST_LOGGED_FUNCTION = Box::into_raw(Box::new(String::from(fct))); + + let m = CStr::from_ptr(message); + let msg = match m.to_str() { + Ok(n) => n, + Err(_) => "", + }; + LAST_LOGGED_MESSAGE = Box::into_raw(Box::new(String::from(msg))); + } + + pub unsafe fn translate_domain(_domain: LogDomain) -> u32 { + 1 + } + + pub unsafe fn translate_severity(_severity: LogSeverity) -> c_int { + 1 + } +} + +#[cfg(test)] +mod test { + use tor_log::*; + use tor_log::log::{LAST_LOGGED_FUNCTION, LAST_LOGGED_MESSAGE}; + + #[test] + fn test_get_log_message() { + { + fn test_macro() { + tor_log_msg!( + LogSeverity::Warn, + LogDomain::Net, + "test_macro", + "test log message {}", + "a", + ); + } + + test_macro(); + + let function = unsafe { Box::from_raw(LAST_LOGGED_FUNCTION) }; + assert_eq!("test_macro", *function); + + let message = unsafe { Box::from_raw(LAST_LOGGED_MESSAGE) }; + assert_eq!("test log message a", *message); + } + + // test multiple inputs into the log message + { + fn test_macro() { + tor_log_msg!( + LogSeverity::Warn, + LogDomain::Net, + "next_test_macro", + "test log message {} {} {} {} {}", + 1, + 2, + 3, + 4, + 5 + ); + } + + test_macro(); + + let function = unsafe { Box::from_raw(LAST_LOGGED_FUNCTION) }; + assert_eq!("next_test_macro", *function); + + let message = unsafe { Box::from_raw(LAST_LOGGED_MESSAGE) }; + assert_eq!("test log message 1 2 3 4 5", *message); + } + + // test how a long log message will be formatted + { + fn test_macro() { + tor_log_msg!( + LogSeverity::Warn, + LogDomain::Net, + "test_macro", + "{}", + "All the world's a stage, and all the men and women \ + merely players: they have their exits and their \ + entrances; and one man in his time plays many parts, his \ + acts being seven ages." + ); + } + + test_macro(); + + let expected_string = "All the world's a \ + stage, and all the men \ + and women merely players: \ + they have their exits and \ + their entrances; and one man \ + in his time plays many parts, \ + his acts being seven ages."; + + let function = unsafe { Box::from_raw(LAST_LOGGED_FUNCTION) }; + assert_eq!("test_macro", *function); + + let message = unsafe { Box::from_raw(LAST_LOGGED_MESSAGE) }; + assert_eq!(expected_string, *message); + } + } +} diff --git a/src/rust/tor_util/Cargo.toml b/src/rust/tor_util/Cargo.toml index b540d8c847..a606a280b2 100644 --- a/src/rust/tor_util/Cargo.toml +++ b/src/rust/tor_util/Cargo.toml @@ -11,6 +11,9 @@ crate_type = ["rlib", "staticlib"] [dependencies.tor_allocate] path = "../tor_allocate" +[dependencies.tor_log] +path = "../tor_log" + [dependencies] libc = "=0.2.39" diff --git a/src/rust/tor_util/ffi.rs b/src/rust/tor_util/ffi.rs index 5c3cdba4be..32779ed476 100644 --- a/src/rust/tor_util/ffi.rs +++ b/src/rust/tor_util/ffi.rs @@ -5,8 +5,7 @@ //! called from C. //! -use libc::c_char; -use tor_allocate::allocate_and_copy_string; +use tor_log::{LogSeverity, LogDomain}; /// Returns a short string to announce Rust support during startup. /// @@ -17,10 +16,12 @@ use tor_allocate::allocate_and_copy_string; /// tor_free(rust_str); /// ``` #[no_mangle] -pub extern "C" fn rust_welcome_string() -> *mut c_char { - let rust_welcome = String::from( +pub extern "C" fn rust_log_welcome_string() { + tor_log_msg!( + LogSeverity::Notice, + LogDomain::General, + "rust_log_welcome_string", "Tor is running with Rust integration. Please report \ - any bugs you encounter.", + any bugs you encounter." ); - allocate_and_copy_string(&rust_welcome) } diff --git a/src/rust/tor_util/lib.rs b/src/rust/tor_util/lib.rs index 12cb3896b6..94697b6069 100644 --- a/src/rust/tor_util/lib.rs +++ b/src/rust/tor_util/lib.rs @@ -7,5 +7,8 @@ extern crate libc; extern crate tor_allocate; +#[macro_use] +extern crate tor_log; + pub mod ffi; pub mod strings; diff --git a/src/rust/tor_util/strings.rs b/src/rust/tor_util/strings.rs index 9321ce4f85..505191d913 100644 --- a/src/rust/tor_util/strings.rs +++ b/src/rust/tor_util/strings.rs @@ -3,80 +3,138 @@ //! Utilities for working with static strings. -use std::ffi::CStr; - -/// A byte-array containing a single NUL byte (`b"\0"`). -pub const NUL_BYTE: &'static [u8] = b"\0"; - -/// Determine if a byte slice is a C-like string. -/// -/// These checks guarantee that: -/// -/// 1. there are no intermediate NUL bytes -/// 2. the last byte *is* a NUL byte +/// Create a `CStr` from a literal byte slice, appending a NUL byte to it first. /// /// # Warning /// -/// This function does _not_ guarantee that the bytes represent any valid -/// encoding such as ASCII or UTF-8. +/// The literal byte slice which is taken as an argument *MUST NOT* have any NUL +/// bytes (`b"\0"`) in it, anywhere, or else an empty string will be returned +/// (`CStr::from_bytes_with_nul_unchecked(b"\0")`) so as to avoid `panic!()`ing. /// /// # Examples /// /// ``` -/// # use tor_util::strings::byte_slice_is_c_like; -/// # -/// let bytes: &[u8] = b"foo bar baz"; +/// #[macro_use] +/// extern crate tor_util; /// -/// assert!(byte_slice_is_c_like(&bytes) == false); +/// use std::ffi::CStr; /// -/// let bytes: &[u8] = b"foo\0bar baz"; +/// # fn do_test() -> Result<&'static CStr, &'static str> { +/// let message: &'static str = "This is a test of the tsunami warning system."; +/// let tuesday: &'static CStr; +/// let original: &str; /// -/// assert!(byte_slice_is_c_like(&bytes) == false); +/// tuesday = cstr!("This is a test of the tsunami warning system."); +/// original = tuesday.to_str().or(Err("Couldn't unwrap CStr!"))?; /// -/// let bytes: &[u8] = b"foo bar baz\0"; +/// assert!(original == message); +/// # +/// # Ok(tuesday) +/// # } +/// # fn main() { +/// # do_test(); // so that we can use the ? operator in the test +/// # } +/// ``` +/// It is also possible to pass several string literals to this macro. They +/// will be concatenated together in the order of the arguments, unmodified, +/// before finally being suffixed with a NUL byte: /// -/// assert!(byte_slice_is_c_like(&bytes) == true); /// ``` -pub fn byte_slice_is_c_like(bytes: &[u8]) -> bool { - if !bytes[..bytes.len() - 1].contains(&0x00) && bytes[bytes.len() - 1] == 0x00 { - return true; - } - false -} - -/// Get a static `CStr` containing a single `NUL_BYTE`. +/// #[macro_use] +/// extern crate tor_util; +/// # +/// # use std::ffi::CStr; +/// # +/// # fn do_test() -> Result<&'static CStr, &'static str> { /// -/// # Examples +/// let quux: &'static CStr = cstr!("foo", "bar", "baz"); +/// let orig: &'static str = quux.to_str().or(Err("Couldn't unwrap CStr!"))?; /// -/// When used as follows in a Rust FFI function, which could be called -/// from C: +/// assert!(orig == "foobarbaz"); +/// # Ok(quux) +/// # } +/// # fn main() { +/// # do_test(); // so that we can use the ? operator in the test +/// # } +/// ``` +/// This is useful for passing static strings to C from Rust FFI code. To do so +/// so, use the `.as_ptr()` method on the resulting `&'static CStr` to convert +/// it to the Rust equivalent of a C `const char*`: /// /// ``` -/// # extern crate libc; -/// # extern crate tor_util; -/// # -/// # use tor_util::strings::empty_static_cstr; -/// use libc::c_char; +/// #[macro_use] +/// extern crate tor_util; +/// /// use std::ffi::CStr; +/// use std::os::raw::c_char; /// -/// pub extern "C" fn give_c_code_an_empty_static_string() -> *const c_char { -/// let empty: &'static CStr = empty_static_cstr(); +/// pub extern "C" fn give_static_borrowed_string_to_c() -> *const c_char { +/// let hello: &'static CStr = cstr!("Hello, language my parents wrote."); /// -/// empty.as_ptr() +/// hello.as_ptr() /// } -/// /// # fn main() { -/// # give_c_code_an_empty_static_string(); +/// # let greetings = give_static_borrowed_string_to_c(); /// # } /// ``` +/// Note that the C code this static borrowed string is passed to *MUST NOT* +/// attempt to free the memory for the string. +/// +/// # Note +/// +/// An unfortunate limitation of the rustc compiler (as of 1.25.0-nightly), is +/// that the first example above compiles, but if we were to change the +/// assignment of `tuesday` as follows, it will fail to compile, because Rust +/// macros are expanded at parse time, and at parse time there is no symbol +/// table available. /// -/// This equates to an "empty" `const char*` static string in C. -pub fn empty_static_cstr() -> &'static CStr { - let empty: &'static CStr; +/// ```ignore +/// tuesday = cstr!(message); +/// ``` +/// with the error message `error: expected a literal`. +/// +/// # Returns +/// +/// If the string literals passed as arguments contain no NUL bytes anywhere, +/// then an `&'static CStr` containing the (concatenated) bytes of the string +/// literal(s) passed as arguments, with a NUL byte appended, is returned. +/// Otherwise, an `&'static CStr` containing a single NUL byte is returned (an +/// "empty" string in C). +#[macro_export] +macro_rules! cstr { + ($($bytes:expr),*) => ( + ::std::ffi::CStr::from_bytes_with_nul( + concat!($($bytes),*, "\0").as_bytes() + ).unwrap_or( + unsafe{ + ::std::ffi::CStr::from_bytes_with_nul_unchecked(b"\0") + } + ) + ) +} + +#[cfg(test)] +mod test { + use std::ffi::CStr; + + #[test] + fn cstr_macro() { + let _: &'static CStr = cstr!("boo"); + } + + #[test] + fn cstr_macro_multi_input() { + let quux: &'static CStr = cstr!("foo", "bar", "baz"); - unsafe { - empty = CStr::from_bytes_with_nul_unchecked(NUL_BYTE); + assert!(quux.to_str().unwrap() == "foobarbaz"); } - empty + #[test] + fn cstr_macro_bad_input() { + let waving: &'static CStr = cstr!("waving not drowning o/"); + let drowning: &'static CStr = cstr!("\0 drowning not waving"); + + assert!(waving.to_str().unwrap() == "waving not drowning o/"); + assert!(drowning.to_str().unwrap() == "") + } } diff --git a/src/test/bench.c b/src/test/bench.c index 92d7a244f7..be04c5209a 100644 --- a/src/test/bench.c +++ b/src/test/bench.c @@ -12,7 +12,7 @@ #include "or.h" #include "onion_tap.h" -#include "relay.h" +#include "relay_crypto.h" #include <openssl/opensslv.h> #include <openssl/evp.h> #include <openssl/ec.h> @@ -505,10 +505,10 @@ bench_cell_ops(void) char key1[CIPHER_KEY_LEN], key2[CIPHER_KEY_LEN]; crypto_rand(key1, sizeof(key1)); crypto_rand(key2, sizeof(key2)); - or_circ->p_crypto = crypto_cipher_new(key1); - or_circ->n_crypto = crypto_cipher_new(key2); - or_circ->p_digest = crypto_digest_new(); - or_circ->n_digest = crypto_digest_new(); + or_circ->crypto.f_crypto = crypto_cipher_new(key1); + or_circ->crypto.b_crypto = crypto_cipher_new(key2); + or_circ->crypto.f_digest = crypto_digest_new(); + or_circ->crypto.b_digest = crypto_digest_new(); reset_perftime(); @@ -518,7 +518,8 @@ bench_cell_ops(void) for (i = 0; i < iters; ++i) { char recognized = 0; crypt_path_t *layer_hint = NULL; - relay_crypt(TO_CIRCUIT(or_circ), cell, d, &layer_hint, &recognized); + relay_decrypt_cell(TO_CIRCUIT(or_circ), cell, d, + &layer_hint, &recognized); } end = perftime(); printf("%sbound cells: %.2f ns per cell. (%.2f ns per byte of payload)\n", @@ -527,10 +528,7 @@ bench_cell_ops(void) NANOCOUNT(start,end,iters*CELL_PAYLOAD_SIZE)); } - crypto_digest_free(or_circ->p_digest); - crypto_digest_free(or_circ->n_digest); - crypto_cipher_free(or_circ->p_crypto); - crypto_cipher_free(or_circ->n_crypto); + relay_crypto_clear(&or_circ->crypto); tor_free(or_circ); tor_free(cell); } diff --git a/src/test/include.am b/src/test/include.am index b768f74475..a663fa5524 100644 --- a/src/test/include.am +++ b/src/test/include.am @@ -88,6 +88,7 @@ src_test_test_SOURCES = \ src/test/test_addr.c \ src/test/test_address.c \ src/test/test_address_set.c \ + src/test/test_bridges.c \ src/test/test_buffers.c \ src/test/test_cell_formats.c \ src/test/test_cell_queue.c \ @@ -118,6 +119,7 @@ src_test_test_SOURCES = \ src/test/test_dos.c \ src/test/test_entryconn.c \ src/test/test_entrynodes.c \ + src/test/test_geoip.c \ src/test/test_guardfraction.c \ src/test/test_extorport.c \ src/test/test_hs.c \ @@ -150,6 +152,7 @@ src_test_test_SOURCES = \ src/test/test_pubsub.c \ src/test/test_relay.c \ src/test/test_relaycell.c \ + src/test/test_relaycrypt.c \ src/test/test_rendcache.c \ src/test/test_replay.c \ src/test/test_router.c \ diff --git a/src/test/test-timers.c b/src/test/test-timers.c index a0b5b535c2..5efd99cacf 100644 --- a/src/test/test-timers.c +++ b/src/test/test-timers.c @@ -7,8 +7,6 @@ #include <stdio.h> #include <string.h> -#include <event2/event.h> - #include "compat.h" #include "compat_libevent.h" #include "crypto.h" @@ -50,7 +48,7 @@ timer_cb(tor_timer_t *t, void *arg, const monotime_t *now_mono) // printf("%d / %d\n",n_fired, N_TIMERS); if (n_fired == n_active_timers) { - event_base_loopbreak(tor_libevent_get_base()); + tor_libevent_exit_loop_after_callback(tor_libevent_get_base()); } } @@ -90,7 +88,7 @@ main(int argc, char **argv) --n_active_timers; } - event_base_loop(tor_libevent_get_base(), 0); + tor_libevent_run_event_loop(tor_libevent_get_base(), 0); int64_t total_difference = 0; uint64_t total_square_difference = 0; diff --git a/src/test/test.c b/src/test/test.c index 2712de4ed1..f90669b5dd 100644 --- a/src/test/test.c +++ b/src/test/test.c @@ -24,7 +24,6 @@ /* These macros pull in declarations for some functions and structures that * are typically file-private. */ -#define GEOIP_PRIVATE #define ROUTER_PRIVATE #define CIRCUITSTATS_PRIVATE #define CIRCUITLIST_PRIVATE @@ -47,7 +46,6 @@ double fabs(double x); #include "compress.h" #include "config.h" #include "connection_edge.h" -#include "geoip.h" #include "rendcommon.h" #include "rendcache.h" #include "test.h" @@ -629,376 +627,6 @@ test_rend_fns(void *arg) tor_free(intro_points_encrypted); } - /* Record odd numbered fake-IPs using ipv6, even numbered fake-IPs - * using ipv4. Since our fake geoip database is the same between - * ipv4 and ipv6, we should get the same result no matter which - * address family we pick for each IP. */ -#define SET_TEST_ADDRESS(i) do { \ - if ((i) & 1) { \ - SET_TEST_IPV6(i); \ - tor_addr_from_in6(&addr, &in6); \ - } else { \ - tor_addr_from_ipv4h(&addr, (uint32_t) i); \ - } \ - } while (0) - - /* Make sure that country ID actually works. */ -#define SET_TEST_IPV6(i) \ - do { \ - set_uint32(in6.s6_addr + 12, htonl((uint32_t) (i))); \ - } while (0) -#define CHECK_COUNTRY(country, val) do { \ - /* test ipv4 country lookup */ \ - tt_str_op(country, OP_EQ, \ - geoip_get_country_name(geoip_get_country_by_ipv4(val))); \ - /* test ipv6 country lookup */ \ - SET_TEST_IPV6(val); \ - tt_str_op(country, OP_EQ, \ - geoip_get_country_name(geoip_get_country_by_ipv6(&in6))); \ - } while (0) - -/** Run unit tests for GeoIP code. */ -static void -test_geoip(void *arg) -{ - int i, j; - time_t now = 1281533250; /* 2010-08-11 13:27:30 UTC */ - char *s = NULL, *v = NULL; - const char *bridge_stats_1 = - "bridge-stats-end 2010-08-12 13:27:30 (86400 s)\n" - "bridge-ips zz=24,xy=8\n" - "bridge-ip-versions v4=16,v6=16\n" - "bridge-ip-transports <OR>=24\n", - *dirreq_stats_1 = - "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" - "dirreq-v3-ips ab=8\n" - "dirreq-v3-reqs ab=8\n" - "dirreq-v3-resp ok=0,not-enough-sigs=0,unavailable=0,not-found=0," - "not-modified=0,busy=0\n" - "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" - "dirreq-v3-tunneled-dl complete=0,timeout=0,running=0\n", - *dirreq_stats_2 = - "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" - "dirreq-v3-ips \n" - "dirreq-v3-reqs \n" - "dirreq-v3-resp ok=0,not-enough-sigs=0,unavailable=0,not-found=0," - "not-modified=0,busy=0\n" - "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" - "dirreq-v3-tunneled-dl complete=0,timeout=0,running=0\n", - *dirreq_stats_3 = - "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" - "dirreq-v3-ips \n" - "dirreq-v3-reqs \n" - "dirreq-v3-resp ok=8,not-enough-sigs=0,unavailable=0,not-found=0," - "not-modified=0,busy=0\n" - "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" - "dirreq-v3-tunneled-dl complete=0,timeout=0,running=0\n", - *dirreq_stats_4 = - "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" - "dirreq-v3-ips \n" - "dirreq-v3-reqs \n" - "dirreq-v3-resp ok=8,not-enough-sigs=0,unavailable=0,not-found=0," - "not-modified=0,busy=0\n" - "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" - "dirreq-v3-tunneled-dl complete=0,timeout=0,running=4\n", - *entry_stats_1 = - "entry-stats-end 2010-08-12 13:27:30 (86400 s)\n" - "entry-ips ab=8\n", - *entry_stats_2 = - "entry-stats-end 2010-08-12 13:27:30 (86400 s)\n" - "entry-ips \n"; - tor_addr_t addr; - struct in6_addr in6; - - /* Populate the DB a bit. Add these in order, since we can't do the final - * 'sort' step. These aren't very good IP addresses, but they're perfectly - * fine uint32_t values. */ - (void)arg; - tt_int_op(0,OP_EQ, geoip_parse_entry("10,50,AB", AF_INET)); - tt_int_op(0,OP_EQ, geoip_parse_entry("52,90,XY", AF_INET)); - tt_int_op(0,OP_EQ, geoip_parse_entry("95,100,AB", AF_INET)); - tt_int_op(0,OP_EQ, geoip_parse_entry("\"105\",\"140\",\"ZZ\"", AF_INET)); - tt_int_op(0,OP_EQ, geoip_parse_entry("\"150\",\"190\",\"XY\"", AF_INET)); - tt_int_op(0,OP_EQ, geoip_parse_entry("\"200\",\"250\",\"AB\"", AF_INET)); - - /* Populate the IPv6 DB equivalently with fake IPs in the same range */ - tt_int_op(0,OP_EQ, geoip_parse_entry("::a,::32,AB", AF_INET6)); - tt_int_op(0,OP_EQ, geoip_parse_entry("::34,::5a,XY", AF_INET6)); - tt_int_op(0,OP_EQ, geoip_parse_entry("::5f,::64,AB", AF_INET6)); - tt_int_op(0,OP_EQ, geoip_parse_entry("::69,::8c,ZZ", AF_INET6)); - tt_int_op(0,OP_EQ, geoip_parse_entry("::96,::be,XY", AF_INET6)); - tt_int_op(0,OP_EQ, geoip_parse_entry("::c8,::fa,AB", AF_INET6)); - - /* We should have 4 countries: ??, ab, xy, zz. */ - tt_int_op(4,OP_EQ, geoip_get_n_countries()); - memset(&in6, 0, sizeof(in6)); - - CHECK_COUNTRY("??", 3); - CHECK_COUNTRY("ab", 32); - CHECK_COUNTRY("??", 5); - CHECK_COUNTRY("??", 51); - CHECK_COUNTRY("xy", 150); - CHECK_COUNTRY("xy", 190); - CHECK_COUNTRY("??", 2000); - - tt_int_op(0,OP_EQ, geoip_get_country_by_ipv4(3)); - SET_TEST_IPV6(3); - tt_int_op(0,OP_EQ, geoip_get_country_by_ipv6(&in6)); - - get_options_mutable()->BridgeRelay = 1; - get_options_mutable()->BridgeRecordUsageByCountry = 1; - /* Put 9 observations in AB... */ - for (i=32; i < 40; ++i) { - SET_TEST_ADDRESS(i); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now-7200); - } - SET_TEST_ADDRESS(225); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now-7200); - /* and 3 observations in XY, several times. */ - for (j=0; j < 10; ++j) - for (i=52; i < 55; ++i) { - SET_TEST_ADDRESS(i); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now-3600); - } - /* and 17 observations in ZZ... */ - for (i=110; i < 127; ++i) { - SET_TEST_ADDRESS(i); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); - } - geoip_get_client_history(GEOIP_CLIENT_CONNECT, &s, &v); - tt_assert(s); - tt_assert(v); - tt_str_op("zz=24,ab=16,xy=8",OP_EQ, s); - tt_str_op("v4=16,v6=16",OP_EQ, v); - tor_free(s); - tor_free(v); - - /* Now clear out all the AB observations. */ - geoip_remove_old_clients(now-6000); - geoip_get_client_history(GEOIP_CLIENT_CONNECT, &s, &v); - tt_assert(s); - tt_assert(v); - tt_str_op("zz=24,xy=8",OP_EQ, s); - tt_str_op("v4=16,v6=16",OP_EQ, v); - tor_free(s); - tor_free(v); - - /* Start testing bridge statistics by making sure that we don't output - * bridge stats without initializing them. */ - s = geoip_format_bridge_stats(now + 86400); - tt_ptr_op(s, OP_EQ, NULL); - - /* Initialize stats and generate the bridge-stats history string out of - * the connecting clients added above. */ - geoip_bridge_stats_init(now); - s = geoip_format_bridge_stats(now + 86400); - tt_assert(s); - tt_str_op(bridge_stats_1,OP_EQ, s); - tor_free(s); - - /* Stop collecting bridge stats and make sure we don't write a history - * string anymore. */ - geoip_bridge_stats_term(); - s = geoip_format_bridge_stats(now + 86400); - tt_ptr_op(s, OP_EQ, NULL); - - /* Stop being a bridge and start being a directory mirror that gathers - * directory request statistics. */ - geoip_bridge_stats_term(); - get_options_mutable()->BridgeRelay = 0; - get_options_mutable()->BridgeRecordUsageByCountry = 0; - get_options_mutable()->DirReqStatistics = 1; - - /* Start testing dirreq statistics by making sure that we don't collect - * dirreq stats without initializing them. */ - SET_TEST_ADDRESS(100); - geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, NULL, now); - s = geoip_format_dirreq_stats(now + 86400); - tt_ptr_op(s, OP_EQ, NULL); - - /* Initialize stats, note one connecting client, and generate the - * dirreq-stats history string. */ - geoip_dirreq_stats_init(now); - SET_TEST_ADDRESS(100); - geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, NULL, now); - s = geoip_format_dirreq_stats(now + 86400); - tt_str_op(dirreq_stats_1,OP_EQ, s); - tor_free(s); - - /* Stop collecting stats, add another connecting client, and ensure we - * don't generate a history string. */ - geoip_dirreq_stats_term(); - SET_TEST_ADDRESS(101); - geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, NULL, now); - s = geoip_format_dirreq_stats(now + 86400); - tt_ptr_op(s, OP_EQ, NULL); - - /* Re-start stats, add a connecting client, reset stats, and make sure - * that we get an all empty history string. */ - geoip_dirreq_stats_init(now); - SET_TEST_ADDRESS(100); - geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, NULL, now); - geoip_reset_dirreq_stats(now); - s = geoip_format_dirreq_stats(now + 86400); - tt_str_op(dirreq_stats_2,OP_EQ, s); - tor_free(s); - - /* Note a successful network status response and make sure that it - * appears in the history string. */ - geoip_note_ns_response(GEOIP_SUCCESS); - s = geoip_format_dirreq_stats(now + 86400); - tt_str_op(dirreq_stats_3,OP_EQ, s); - tor_free(s); - - /* Start a tunneled directory request. */ - geoip_start_dirreq((uint64_t) 1, 1024, DIRREQ_TUNNELED); - s = geoip_format_dirreq_stats(now + 86400); - tt_str_op(dirreq_stats_4,OP_EQ, s); - tor_free(s); - - /* Stop collecting directory request statistics and start gathering - * entry stats. */ - geoip_dirreq_stats_term(); - get_options_mutable()->DirReqStatistics = 0; - get_options_mutable()->EntryStatistics = 1; - - /* Start testing entry statistics by making sure that we don't collect - * anything without initializing entry stats. */ - SET_TEST_ADDRESS(100); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); - s = geoip_format_entry_stats(now + 86400); - tt_ptr_op(s, OP_EQ, NULL); - - /* Initialize stats, note one connecting client, and generate the - * entry-stats history string. */ - geoip_entry_stats_init(now); - SET_TEST_ADDRESS(100); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); - s = geoip_format_entry_stats(now + 86400); - tt_str_op(entry_stats_1,OP_EQ, s); - tor_free(s); - - /* Stop collecting stats, add another connecting client, and ensure we - * don't generate a history string. */ - geoip_entry_stats_term(); - SET_TEST_ADDRESS(101); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); - s = geoip_format_entry_stats(now + 86400); - tt_ptr_op(s, OP_EQ, NULL); - - /* Re-start stats, add a connecting client, reset stats, and make sure - * that we get an all empty history string. */ - geoip_entry_stats_init(now); - SET_TEST_ADDRESS(100); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); - geoip_reset_entry_stats(now); - s = geoip_format_entry_stats(now + 86400); - tt_str_op(entry_stats_2,OP_EQ, s); - tor_free(s); - - /* Test the OOM handler. Add a client, run the OOM. */ - geoip_entry_stats_init(now); - SET_TEST_ADDRESS(100); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, - now - (12 * 60 * 60)); - /* We've seen this 12 hours ago. Run the OOM, it should clean the entry - * because it is above the minimum cutoff of 4 hours. */ - size_t bytes_removed = geoip_client_cache_handle_oom(now, 1000); - tt_size_op(bytes_removed, OP_GT, 0); - - /* Do it again but this time with an entry with a lower cutoff. */ - geoip_entry_stats_init(now); - SET_TEST_ADDRESS(100); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, - now - (3 * 60 * 60)); - bytes_removed = geoip_client_cache_handle_oom(now, 1000); - tt_size_op(bytes_removed, OP_EQ, 0); - - /* Stop collecting entry statistics. */ - geoip_entry_stats_term(); - get_options_mutable()->EntryStatistics = 0; - - done: - tor_free(s); - tor_free(v); -} - -static void -test_geoip_with_pt(void *arg) -{ - time_t now = 1281533250; /* 2010-08-11 13:27:30 UTC */ - char *s = NULL; - int i; - tor_addr_t addr; - struct in6_addr in6; - - (void)arg; - get_options_mutable()->BridgeRelay = 1; - get_options_mutable()->BridgeRecordUsageByCountry = 1; - - memset(&in6, 0, sizeof(in6)); - - /* No clients seen yet. */ - s = geoip_get_transport_history(); - tor_assert(!s); - - /* 4 connections without a pluggable transport */ - for (i=0; i < 4; ++i) { - SET_TEST_ADDRESS(i); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now-7200); - } - - /* 9 connections with "alpha" */ - for (i=4; i < 13; ++i) { - SET_TEST_ADDRESS(i); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "alpha", now-7200); - } - - /* one connection with "beta" */ - SET_TEST_ADDRESS(13); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "beta", now-7200); - - /* 14 connections with "charlie" */ - for (i=14; i < 28; ++i) { - SET_TEST_ADDRESS(i); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "charlie", now-7200); - } - - /* 131 connections with "ddr" */ - for (i=28; i < 159; ++i) { - SET_TEST_ADDRESS(i); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "ddr", now-7200); - } - - /* 8 connections with "entropy" */ - for (i=159; i < 167; ++i) { - SET_TEST_ADDRESS(i); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "entropy", now-7200); - } - - /* 2 connections from the same IP with two different transports. */ - SET_TEST_ADDRESS(++i); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "fire", now-7200); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "google", now-7200); - - /* Test the transport history string. */ - s = geoip_get_transport_history(); - tor_assert(s); - tt_str_op(s,OP_EQ, "<OR>=8,alpha=16,beta=8,charlie=16,ddr=136," - "entropy=8,fire=8,google=8"); - - /* Stop collecting entry statistics. */ - geoip_entry_stats_term(); - get_options_mutable()->EntryStatistics = 0; - - done: - tor_free(s); -} - -#undef SET_TEST_ADDRESS -#undef SET_TEST_IPV6 -#undef CHECK_COUNTRY - /** Run unit tests for stats code. */ static void test_stats(void *arg) @@ -1172,8 +800,6 @@ static struct testcase_t test_array[] = { { "fast_handshake", test_fast_handshake, 0, NULL, NULL }, FORK(circuit_timeout), FORK(rend_fns), - ENT(geoip), - FORK(geoip_with_pt), FORK(stats), END_OF_TESTCASES @@ -1185,6 +811,7 @@ struct testgroup_t testgroups[] = { { "addr/", addr_tests }, { "address/", address_tests }, { "address_set/", address_set_tests }, + { "bridges/", bridges_tests }, { "buffer/", buffer_tests }, { "cellfmt/", cell_format_tests }, { "cellqueue/", cell_queue_tests }, @@ -1216,6 +843,7 @@ struct testgroup_t testgroups[] = { { "entrynodes/", entrynodes_tests }, { "guardfraction/", guardfraction_tests }, { "extorport/", extorport_tests }, + { "geoip/", geoip_tests }, { "legacy_hs/", hs_tests }, { "hs_cache/", hs_cache }, { "hs_cell/", hs_cell_tests }, @@ -1242,6 +870,7 @@ struct testgroup_t testgroups[] = { { "pt/", pt_tests }, { "relay/" , relay_tests }, { "relaycell/", relaycell_tests }, + { "relaycrypt/", relaycrypt_tests }, { "rend_cache/", rend_cache_tests }, { "replaycache/", replaycache_tests }, { "router/", router_tests }, diff --git a/src/test/test.h b/src/test/test.h index 26139fc5fe..34c6e46427 100644 --- a/src/test/test.h +++ b/src/test/test.h @@ -72,6 +72,14 @@ I64_PRINTF_TYPE, I64_FORMAT, \ {print_ = (I64_PRINTF_TYPE) value_;}, {}, TT_EXIT_TEST_FUNCTION) +/** + * Declare that the test is done, even though no tt___op() calls were made. + * + * For use when you only want to test calling something, but not check + * any values/pointers/etc afterwards. + */ +#define tt_finished() TT_EXIT_TEST_FUNCTION + const char *get_fname(const char *name); const char *get_fname_rnd(const char *name); struct crypto_pk_t *pk_generate(int idx); @@ -178,6 +186,7 @@ extern struct testcase_t accounting_tests[]; extern struct testcase_t addr_tests[]; extern struct testcase_t address_tests[]; extern struct testcase_t address_set_tests[]; +extern struct testcase_t bridges_tests[]; extern struct testcase_t buffer_tests[]; extern struct testcase_t cell_format_tests[]; extern struct testcase_t cell_queue_tests[]; @@ -208,6 +217,7 @@ extern struct testcase_t entryconn_tests[]; extern struct testcase_t entrynodes_tests[]; extern struct testcase_t guardfraction_tests[]; extern struct testcase_t extorport_tests[]; +extern struct testcase_t geoip_tests[]; extern struct testcase_t hs_tests[]; extern struct testcase_t hs_cache[]; extern struct testcase_t hs_cell_tests[]; @@ -237,6 +247,7 @@ extern struct testcase_t pubsub_tests[]; extern struct testcase_t pt_tests[]; extern struct testcase_t relay_tests[]; extern struct testcase_t relaycell_tests[]; +extern struct testcase_t relaycrypt_tests[]; extern struct testcase_t rend_cache_tests[]; extern struct testcase_t replaycache_tests[]; extern struct testcase_t router_tests[]; diff --git a/src/test/test_bridges.c b/src/test/test_bridges.c new file mode 100644 index 0000000000..b5bd27e098 --- /dev/null +++ b/src/test/test_bridges.c @@ -0,0 +1,618 @@ +/* Copyright (c) 2018, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file test_bridges.c + * \brief Unittests for code in src/or/bridges.c + **/ + +#define TOR_BRIDGES_PRIVATE +#define PT_PRIVATE /* Only needed for the mock_* items below */ + +#include <stdbool.h> + +#include "or.h" +#include "address.h" +#include "bridges.h" +#include "config.h" +#include "container.h" +#include "transports.h" +#include "util.h" + +/* Test suite stuff */ +#include "test.h" + +/** + * A mocked transport_t, constructed via mock_transport_get_by_name(). + */ +static transport_t *mock_transport = NULL; + +/** + * Mock transport_get_by_name() to simply return a transport_t for the + * transport name that was input to it. + */ +static transport_t * +mock_transport_get_by_name(const char *name) +{ + tor_addr_t *addr = tor_malloc(sizeof(tor_addr_t)); + uint16_t port = 9999; + int socksv = 9; + char *args = tor_strdup("foo=bar"); + + if (!mock_transport) { + tor_addr_parse(addr, "99.99.99.99"); + mock_transport = transport_new(addr, port, name, socksv, args); + } + + tor_free(addr); + tor_free(args); + + return mock_transport; +} + +#undef PT_PRIVATE /* defined(PT_PRIVATE) */ + +/** + * Test helper: Add a variety of bridges to our global bridgelist. + */ +static void +helper_add_bridges_to_bridgelist(void *arg) +{ + /* Note: the two bridges which do not have specified fingerprints will be + * internally stored as both having the same fingerprint of all-zero bytes. + */ + + (void)arg; + char *bridge0 = tor_strdup("6.6.6.6:6666"); + char *bridge1 = tor_strdup("6.6.6.7:6667 " + "A10C4F666D27364036B562823E5830BC448E046A"); + char *bridge2 = tor_strdup("obfs4 198.245.60.51:443 " + "752CF7825B3B9EA6A98C83AC41F7099D67007EA5 " + "cert=xpmQtKUqQ/6v5X7ijgYE/f03+l2/EuQ1dexjyUhh16wQlu/" + "cpXUGalmhDIlhuiQPNEKmKw iat-mode=0"); + char *bridge3 = tor_strdup("banana 5.5.5.5:5555 " + "9D6AE1BD4FDF39721CE908966E79E16F9BFCCF2F"); + char *bridge4 = tor_strdup("obfs4 1.2.3.4:1234 " + "foo=abcdefghijklmnopqrstuvwxyz"); + char *bridge5 = tor_strdup("apple 4.4.4.4:4444 " + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA " + "foo=abcdefghijklmnopqrstuvwxyz"); + + mark_bridge_list(); + +#define ADD_BRIDGE(bridge) \ + bridge_line_t *bridge_line_ ##bridge = parse_bridge_line(bridge); \ + if (!bridge_line_ ##bridge) { \ + printf("Unparseable bridge line: '%s'", #bridge); \ + } else { \ + bridge_add_from_config(bridge_line_ ##bridge); \ + } \ + tor_free(bridge); + + ADD_BRIDGE(bridge0); + ADD_BRIDGE(bridge1); + ADD_BRIDGE(bridge2); + ADD_BRIDGE(bridge3); + ADD_BRIDGE(bridge4); + ADD_BRIDGE(bridge5); +#undef ADD_BRIDGES + + sweep_bridge_list(); +} + +/** + * Make sure our test helper works too. + */ +static void +test_bridges_helper_func_add_bridges_to_bridgelist(void *arg) +{ + helper_add_bridges_to_bridgelist(arg); + tt_finished(); + + done: + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling bridge_list_get() should create a new bridgelist if we + * didn't have one before. + */ +static void +test_bridges_bridge_list_get_creates_new_bridgelist(void *arg) +{ + const smartlist_t *bridgelist = bridge_list_get(); + + (void)arg; + + tt_ptr_op(bridgelist, OP_NE, NULL); + + done: + return; +} + +/** + * Calling clear_bridge_list() should remove all bridges from the bridgelist. + */ +static void +test_bridges_clear_bridge_list(void *arg) +{ + const smartlist_t *bridgelist; + const smartlist_t *bridgelist_after; + const bridge_info_t *bridge; + const bridge_info_t *bridge_after; + + helper_add_bridges_to_bridgelist(arg); + bridgelist = bridge_list_get(); + tt_ptr_op(bridgelist, OP_NE, NULL); + + bridge = smartlist_get(bridgelist, 0); + tt_ptr_op(bridge, OP_NE, NULL); + + clear_bridge_list(); + bridgelist_after = bridge_list_get(); + tt_ptr_op(bridgelist_after, OP_NE, NULL); + + bridge_after = smartlist_get(bridgelist, 0); + // There now shouldn't be a first bridge + tt_ptr_op(bridge_after, OP_EQ, NULL); + + done: + return; +} + +/** + * Calling bridge_get_addrport() should give me the address and port + * of the bridge. In this case, we sort the smartlist of bridges on + * fingerprints and choose the first one. + */ +static void +test_bridges_bridge_get_addrport(void *arg) +{ + smartlist_t *bridgelist; + const bridge_info_t *bridge; + const tor_addr_port_t *addrport; + + helper_add_bridges_to_bridgelist(arg); + bridgelist = (smartlist_t*)bridge_list_get(); + tt_ptr_op(bridgelist, OP_NE, NULL); + + // This should be the bridge at 6.6.6.6:6666 with fingerprint + // 0000000000000000000000000000000000000000 + bridge = smartlist_get(bridgelist, 0); + tt_ptr_op(bridge, OP_NE, NULL); + + addrport = bridge_get_addr_port(bridge); + tt_int_op(addrport->port, OP_EQ, 6666); + + done: + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling get_configured_bridge_by_orports_digest() with two + * configured bridge orports and an invalid digest should return the + * bridge of the first addrport in the list. + */ +static void +test_bridges_get_configured_bridge_by_orports_digest(void *arg) +{ + smartlist_t *orports = NULL; + const smartlist_t *bridgelist; + const bridge_info_t *bridge1; + const bridge_info_t *bridge2; + const bridge_info_t *ret; + tor_addr_port_t *addrport1; + tor_addr_port_t *addrport2; + const char *digest; + + helper_add_bridges_to_bridgelist(arg); + bridgelist = bridge_list_get(); + tt_ptr_op(bridgelist, OP_NE, NULL); + + // This should be the bridge at 6.6.6.6:6666 with fingerprint + // 0000000000000000000000000000000000000000 + bridge1 = smartlist_get(bridgelist, 0); + tt_ptr_op(bridge1, OP_NE, NULL); + // This should be the bridge at 6.6.6.7:6667 with fingerprint + // A10C4F666D27364036B562823E5830BC448E046A + bridge2 = smartlist_get(bridgelist, 1); + tt_ptr_op(bridge2, OP_NE, NULL); + + addrport1 = (tor_addr_port_t*)bridge_get_addr_port(bridge1); + tt_int_op(addrport1->port, OP_EQ, 6666); + addrport2 = (tor_addr_port_t*)bridge_get_addr_port(bridge2); + tt_int_op(addrport2->port, OP_EQ, 6667); + + orports = smartlist_new(); + smartlist_add(orports, addrport1); + smartlist_add(orports, addrport2); + + digest = "zzzzzzzzzzzzzzzz"; + + ret = get_configured_bridge_by_orports_digest(digest, orports); + tt_ptr_op(ret, OP_NE, NULL); + + tt_mem_op(addrport1, OP_EQ, bridge_get_addr_port(ret), sizeof(ret)); + + done: + smartlist_free(orports); + + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling get_configured_bridge_by_addr_port_digest() with a digest that we do + * have and an addr:port pair we don't should return the bridge for that + * digest. + */ +static void +test_bridges_get_configured_bridge_by_addr_port_digest_digest_only(void *arg) +{ + char digest[DIGEST_LEN]; + bridge_info_t *bridge; + const char fingerprint[HEX_DIGEST_LEN] = + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + tor_addr_t *addr = tor_malloc(sizeof(tor_addr_t)); + char ret_addr[16]; + uint16_t port = 11111; + int ret; + + helper_add_bridges_to_bridgelist(arg); + + // We don't actually have a bridge with this addr:port pair + base16_decode(digest, DIGEST_LEN, fingerprint, HEX_DIGEST_LEN); + ret = tor_addr_parse(addr, "111.111.111.111"); + tt_int_op(ret, OP_EQ, 2); // it returns the address family on success + + bridge = get_configured_bridge_by_addr_port_digest(addr, port, digest); + tt_ptr_op(bridge, OP_NE, NULL); + + tor_addr_to_str(ret_addr, &bridge_get_addr_port(bridge)->addr, 16, 0); + tt_str_op("4.4.4.4", OP_EQ, ret_addr); + + done: + tor_free(addr); + + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling get_configured_bridge_by_addr_port_digest() with only an + * addr:port (i.e. digest set to NULL) should return the bridge for + * that digest when there is such a bridge. + */ +static void +test_bridges_get_configured_bridge_by_addr_port_digest_address_only(void *arg) +{ + bridge_info_t *bridge; + tor_addr_t *addr = tor_malloc(sizeof(tor_addr_t)); + char ret_addr[16]; + uint16_t port = 6666; + int ret; + + helper_add_bridges_to_bridgelist(arg); + + ret = tor_addr_parse(addr, "6.6.6.6"); + tt_int_op(ret, OP_EQ, 2); // it returns the address family on success + + bridge = get_configured_bridge_by_addr_port_digest(addr, port, NULL); + tt_ptr_op(bridge, OP_NE, NULL); + + tor_addr_to_str(ret_addr, &bridge_get_addr_port(bridge)->addr, 16, 0); + tt_str_op("6.6.6.6", OP_EQ, ret_addr); + + done: + tor_free(addr); + + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling get_configured_bridge_by_exact_addr_port_digest() with a digest that + * we do have, and an addr:port pair we don't have, should return NULL. + */ +static void +test_bridges_get_configured_bridge_by_exact_addr_port_digest_donly(void *arg) +{ + char digest[DIGEST_LEN]; + bridge_info_t *bridge; + const char fingerprint[HEX_DIGEST_LEN] = + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + tor_addr_t *addr = tor_malloc(sizeof(tor_addr_t)); + uint16_t port = 11111; + int ret; + + helper_add_bridges_to_bridgelist(arg); + + // We don't actually have a bridge with this addr:port pair + base16_decode(digest, DIGEST_LEN, fingerprint, HEX_DIGEST_LEN); + ret = tor_addr_parse(addr, "111.111.111.111"); + tt_int_op(ret, OP_EQ, 2); // it returns the address family on success + + bridge = get_configured_bridge_by_exact_addr_port_digest(addr, port, digest); + tt_ptr_op(bridge, OP_EQ, NULL); + + done: + tor_free(addr); + + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling get_configured_bridge_by_exact_addr_port_digest() with a digest that + * we do have, and an addr:port pair we do have, should return the bridge. + */ +static void +test_bridges_get_configured_bridge_by_exact_addr_port_digest_both(void *arg) +{ + char digest[DIGEST_LEN]; + bridge_info_t *bridge; + const char fingerprint[HEX_DIGEST_LEN] = + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + tor_addr_t *addr = tor_malloc(sizeof(tor_addr_t)); + uint16_t port = 4444; + char ret_addr[16]; + int ret; + + helper_add_bridges_to_bridgelist(arg); + + base16_decode(digest, DIGEST_LEN, fingerprint, HEX_DIGEST_LEN); + ret = tor_addr_parse(addr, "4.4.4.4"); + tt_int_op(ret, OP_EQ, 2); // it returns the address family on success + + bridge = get_configured_bridge_by_exact_addr_port_digest(addr, port, digest); + tt_ptr_op(bridge, OP_NE, NULL); + + tor_addr_to_str(ret_addr, &bridge_get_addr_port(bridge)->addr, 16, 0); + tt_str_op("4.4.4.4", OP_EQ, ret_addr); + + done: + tor_free(addr); + + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling get_configured_bridge_by_exact_addr_port_digest() with no digest, + * and an addr:port pair we do have, should return the bridge. + */ +static void +test_bridges_get_configured_bridge_by_exact_addr_port_digest_aonly(void *arg) +{ + bridge_info_t *bridge; + tor_addr_t *addr = tor_malloc(sizeof(tor_addr_t)); + uint16_t port = 4444; + char ret_addr[16]; + int ret; + + helper_add_bridges_to_bridgelist(arg); + + ret = tor_addr_parse(addr, "4.4.4.4"); + tt_int_op(ret, OP_EQ, 2); // it returns the address family on success + + bridge = get_configured_bridge_by_exact_addr_port_digest(addr, port, NULL); + tt_ptr_op(bridge, OP_NE, NULL); + + tor_addr_to_str(ret_addr, &bridge_get_addr_port(bridge)->addr, 16, 0); + tt_str_op("4.4.4.4", OP_EQ, ret_addr); + + done: + tor_free(addr); + + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling find_bridge_by_digest() when we have a bridge with a known + * identity digest should return the bridge's information. + */ +static void +test_bridges_find_bridge_by_digest_known(void *arg) +{ + char digest1[DIGEST_LEN]; + bridge_info_t *bridge; + const char fingerprint[HEX_DIGEST_LEN] = + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + + helper_add_bridges_to_bridgelist(arg); + + base16_decode(digest1, DIGEST_LEN, fingerprint, HEX_DIGEST_LEN); + bridge = find_bridge_by_digest(digest1); + + tt_ptr_op(bridge, OP_NE, NULL); + + /* We have to call bridge_get_rsa_id_digest() here because the bridge_info_t + * struct is opaquely defined in bridges.h. */ + const uint8_t *digest2 = bridge_get_rsa_id_digest(bridge); + + tt_mem_op((char*)digest2, OP_EQ, digest1, DIGEST_LEN); + + done: + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling find_bridge_by_digest() when we do NOT have a bridge with that + * identity digest should return NULL. + */ +static void +test_bridges_find_bridge_by_digest_unknown(void *arg) +{ + const char *fingerprint = "cccccccccccccccccccccccccccccccccccccccc"; + bridge_info_t *bridge; + + helper_add_bridges_to_bridgelist(arg); + + bridge = find_bridge_by_digest(fingerprint); + + tt_ptr_op(bridge, OP_EQ, NULL); + + done: + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling bridge_resolve_conflicts() with an identical bridge to one we've + * already configure should mark the pre-configured bridge for removal. + */ +static void +test_bridges_bridge_resolve_conflicts(void *arg) +{ + tor_addr_t *addr = tor_malloc(sizeof(tor_addr_t)); + uint16_t port = 4444; + const char *digest = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"; + const char *transport = "apple"; + int ret; + + helper_add_bridges_to_bridgelist(arg); + + ret = tor_addr_parse(addr, "4.4.4.4"); + tt_int_op(ret, OP_EQ, 2); // it returns the address family on success + + bridge_resolve_conflicts((const tor_addr_t*)addr, port, digest, transport); + + /* The bridge should now be marked for removal, and removed when we sweep the + * bridge_list */ + sweep_bridge_list(); + ret = addr_is_a_configured_bridge((const tor_addr_t*)addr, port, digest); + tt_int_op(ret, OP_EQ, 0); + + done: + tor_free(addr); + + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling transport_is_needed() with a transport we do need ("obfs4") and a + * bogus transport that we don't need should return 1 and 0, respectively. + */ +static void +test_bridges_transport_is_needed(void *arg) +{ + int ret; + + helper_add_bridges_to_bridgelist(arg); + + ret = transport_is_needed("obfs4"); + tt_int_op(ret, OP_EQ, 1); + + ret = transport_is_needed("apowefjaoewpaief"); + tt_int_op(ret, OP_EQ, 0); + + done: + mark_bridge_list(); + sweep_bridge_list(); +} + +/** + * Calling get_transport_by_bridge_addrport() with the address and port of a + * configured bridge which uses a pluggable transport when there is no global + * transport_list should return -1 and the transport_t should be NULL. + */ +static void +test_bridges_get_transport_by_bridge_addrport_no_ptlist(void *arg) +{ + transport_t *transport = NULL; + tor_addr_t *addr = tor_malloc(sizeof(tor_addr_t)); + uint16_t port = 1234; + int ret; + + helper_add_bridges_to_bridgelist(arg); + + ret = tor_addr_parse(addr, "1.2.3.4"); + tt_int_op(ret, OP_EQ, 2); // it returns the address family on success? + + /* This will fail because the global transport_list has nothing in it, and so + * transport_get_by_name() has nothing to return, even the the bridge *did* + * say it had an obfs4 transport. + */ + ret = get_transport_by_bridge_addrport((const tor_addr_t*)addr, port, + (const transport_t**)&transport); + tt_int_op(ret, OP_EQ, -1); // returns -1 on failure + tt_ptr_op(transport, OP_EQ, NULL); + + done: + tor_free(addr); + + mark_bridge_list(); + sweep_bridge_list(); +} + +#define PT_PRIVATE + +/** + * Calling get_transport_by_bridge_addrport() with the address and port of a + * configured bridge which uses a pluggable transport should return 0 and set + * appropriate transport_t. + */ +static void +test_bridges_get_transport_by_bridge_addrport(void *arg) +{ + transport_t *transport = NULL; + tor_addr_t *addr = tor_malloc(sizeof(tor_addr_t)); + uint16_t port = 1234; + int ret; + + helper_add_bridges_to_bridgelist(arg); + mark_transport_list(); // Also initialise our transport_list + + ret = tor_addr_parse(addr, "1.2.3.4"); + tt_int_op(ret, OP_EQ, 2); // it returns the address family on success? + + /* After we mock transport_get_by_name() to return a bogus transport_t with + * the name it was asked for, the call should succeed. + */ + MOCK(transport_get_by_name, mock_transport_get_by_name); + ret = get_transport_by_bridge_addrport((const tor_addr_t*)addr, port, + (const transport_t**)&transport); + tt_int_op(ret, OP_EQ, 0); // returns 0 on success + tt_ptr_op(transport, OP_NE, NULL); + tt_str_op(transport->name, OP_EQ, "obfs4"); + + done: + UNMOCK(transport_get_by_name); + + tor_free(addr); + transport_free(transport); + + mark_bridge_list(); + sweep_bridge_list(); +} + +#undef PT_PRIVATE /* defined(PT_PRIVATE) */ + +#define B_TEST(name, flags) \ + { #name, test_bridges_ ##name, (flags), NULL, NULL } + +struct testcase_t bridges_tests[] = { + B_TEST(helper_func_add_bridges_to_bridgelist, 0), + B_TEST(bridge_list_get_creates_new_bridgelist, 0), + B_TEST(clear_bridge_list, 0), + B_TEST(bridge_get_addrport, 0), + B_TEST(get_configured_bridge_by_orports_digest, 0), + B_TEST(get_configured_bridge_by_addr_port_digest_digest_only, 0), + B_TEST(get_configured_bridge_by_addr_port_digest_address_only, 0), + B_TEST(get_configured_bridge_by_exact_addr_port_digest_donly, 0), + B_TEST(get_configured_bridge_by_exact_addr_port_digest_both, 0), + B_TEST(get_configured_bridge_by_exact_addr_port_digest_aonly, 0), + B_TEST(find_bridge_by_digest_known, 0), + B_TEST(find_bridge_by_digest_unknown, 0), + B_TEST(bridge_resolve_conflicts, 0), + B_TEST(get_transport_by_bridge_addrport_no_ptlist, 0), + B_TEST(get_transport_by_bridge_addrport, 0), + B_TEST(transport_is_needed, 0), + END_OF_TESTCASES +}; + diff --git a/src/test/test_channel.c b/src/test/test_channel.c index bdc9d32f78..812ec6c1ac 100644 --- a/src/test/test_channel.c +++ b/src/test/test_channel.c @@ -281,6 +281,7 @@ new_fake_channel(void) chan->state = CHANNEL_STATE_OPEN; chan->cmux = circuitmux_alloc(); + circuitmux_set_policy(chan->cmux, &ewma_policy); return chan; } @@ -575,15 +576,13 @@ test_channel_outbound_cell(void *arg) channel_register(chan); tt_int_op(chan->registered, OP_EQ, 1); /* Set EWMA policy so we can pick it when flushing. */ - channel_set_cmux_policy_everywhere(&ewma_policy); + circuitmux_set_policy(chan->cmux, &ewma_policy); tt_ptr_op(circuitmux_get_policy(chan->cmux), OP_EQ, &ewma_policy); /* Register circuit to the channel circid map which will attach the circuit * to the channel's cmux as well. */ circuit_set_n_circid_chan(TO_CIRCUIT(circ), 42, chan); tt_int_op(channel_num_circuits(chan), OP_EQ, 1); - tt_assert(!TO_CIRCUIT(circ)->next_active_on_n_chan); - tt_assert(!TO_CIRCUIT(circ)->prev_active_on_n_chan); /* Test the cmux state. */ tt_ptr_op(TO_CIRCUIT(circ)->n_mux, OP_EQ, chan->cmux); tt_int_op(circuitmux_is_circuit_attached(chan->cmux, TO_CIRCUIT(circ)), diff --git a/src/test/test_channelpadding.c b/src/test/test_channelpadding.c index 9e570b81a7..2c803c3443 100644 --- a/src/test/test_channelpadding.c +++ b/src/test/test_channelpadding.c @@ -15,7 +15,6 @@ #include "channelpadding.h" #include "compat_libevent.h" #include "config.h" -#include <event2/event.h> #include "compat_time.h" #include "main.h" #include "networkstatus.h" @@ -65,7 +64,7 @@ mock_channel_write_cell_relay2(channel_t *chan, cell_t *cell) (void)chan; tried_to_write_cell++; channel_tls_handle_cell(cell, ((channel_tls_t*)relay1_relay2)->conn); - event_base_loopbreak(tor_libevent_get_base()); + tor_libevent_exit_loop_after_callback(tor_libevent_get_base()); return 0; } @@ -75,7 +74,7 @@ mock_channel_write_cell_relay1(channel_t *chan, cell_t *cell) (void)chan; tried_to_write_cell++; channel_tls_handle_cell(cell, ((channel_tls_t*)relay2_relay1)->conn); - event_base_loopbreak(tor_libevent_get_base()); + tor_libevent_exit_loop_after_callback(tor_libevent_get_base()); return 0; } @@ -85,7 +84,7 @@ mock_channel_write_cell_relay3(channel_t *chan, cell_t *cell) (void)chan; tried_to_write_cell++; channel_tls_handle_cell(cell, ((channel_tls_t*)client_relay3)->conn); - event_base_loopbreak(tor_libevent_get_base()); + tor_libevent_exit_loop_after_callback(tor_libevent_get_base()); return 0; } @@ -95,7 +94,7 @@ mock_channel_write_cell_client(channel_t *chan, cell_t *cell) (void)chan; tried_to_write_cell++; channel_tls_handle_cell(cell, ((channel_tls_t*)relay3_client)->conn); - event_base_loopbreak(tor_libevent_get_base()); + tor_libevent_exit_loop_after_callback(tor_libevent_get_base()); return 0; } @@ -105,7 +104,7 @@ mock_channel_write_cell(channel_t *chan, cell_t *cell) tried_to_write_cell++; channel_tls_handle_cell(cell, ((channel_tls_t*)chan)->conn); if (!dont_stop_libevent) - event_base_loopbreak(tor_libevent_get_base()); + tor_libevent_exit_loop_after_callback(tor_libevent_get_base()); return 0; } @@ -246,7 +245,7 @@ static void dummy_timer_cb(tor_timer_t *t, void *arg, const monotime_t *now_mono) { (void)t; (void)arg; (void)now_mono; - event_base_loopbreak(tor_libevent_get_base()); + tor_libevent_exit_loop_after_callback(tor_libevent_get_base()); return; } @@ -264,7 +263,8 @@ dummy_nop_timer(void) timer_schedule(dummy_timer, &timeout); - event_base_loop(tor_libevent_get_base(), 0); + tor_libevent_run_event_loop(tor_libevent_get_base(), 0); + timer_free(dummy_timer); } diff --git a/src/test/test_circuitlist.c b/src/test/test_circuitlist.c index d170009a9c..3794ffc2c6 100644 --- a/src/test/test_circuitlist.c +++ b/src/test/test_circuitlist.c @@ -9,6 +9,7 @@ #include "channel.h" #include "circuitbuild.h" #include "circuitlist.h" +#include "circuitmux_ewma.h" #include "hs_circuitmap.h" #include "test.h" #include "log_test_helpers.h" diff --git a/src/test/test_circuitmux.c b/src/test/test_circuitmux.c index 854f725054..75b7a0ea47 100644 --- a/src/test/test_circuitmux.c +++ b/src/test/test_circuitmux.c @@ -7,6 +7,7 @@ #include "or.h" #include "channel.h" #include "circuitmux.h" +#include "circuitmux_ewma.h" #include "relay.h" #include "scheduler.h" #include "test.h" @@ -45,6 +46,7 @@ test_cmux_destroy_cell_queue(void *arg) cmux = circuitmux_alloc(); tt_assert(cmux); ch = new_fake_channel(); + circuitmux_set_policy(cmux, &ewma_policy); ch->has_queued_writes = has_queued_writes; ch->wide_circ_ids = 1; diff --git a/src/test/test_compat_libevent.c b/src/test/test_compat_libevent.c index 7dd8e65194..376524c2d1 100644 --- a/src/test/test_compat_libevent.c +++ b/src/test/test_compat_libevent.c @@ -10,7 +10,6 @@ #include "compat_libevent.h" #include <event2/event.h> -#include <event2/thread.h> #include "log_test_helpers.h" diff --git a/src/test/test_config.c b/src/test/test_config.c index d82f6d21da..1756f9040d 100644 --- a/src/test/test_config.c +++ b/src/test/test_config.c @@ -20,7 +20,6 @@ #include "connection_edge.h" #include "test.h" #include "util.h" -#include "address.h" #include "connection_or.h" #include "control.h" #include "cpuworker.h" @@ -42,9 +41,6 @@ #include "routerlist.h" #include "routerset.h" #include "statefile.h" -#include "test.h" -#include "transports.h" -#include "util.h" #include "test_helpers.h" diff --git a/src/test/test_dir_handle_get.c b/src/test/test_dir_handle_get.c index ca64dce5fe..71faf70af2 100644 --- a/src/test/test_dir_handle_get.c +++ b/src/test/test_dir_handle_get.c @@ -16,7 +16,6 @@ #include "directory.h" #include "test.h" #include "compress.h" -#include "connection.h" #include "rendcommon.h" #include "rendcache.h" #include "router.h" diff --git a/src/test/test_geoip.c b/src/test/test_geoip.c new file mode 100644 index 0000000000..be1376a3b1 --- /dev/null +++ b/src/test/test_geoip.c @@ -0,0 +1,459 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2018, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" + +/* These macros pull in declarations for some functions and structures that + * are typically file-private. */ +#define GEOIP_PRIVATE +#include "or.h" +#include "config.h" +#include "geoip.h" +#include "test.h" + + /* Record odd numbered fake-IPs using ipv6, even numbered fake-IPs + * using ipv4. Since our fake geoip database is the same between + * ipv4 and ipv6, we should get the same result no matter which + * address family we pick for each IP. */ +#define SET_TEST_ADDRESS(i) do { \ + if ((i) & 1) { \ + SET_TEST_IPV6(i); \ + tor_addr_from_in6(&addr, &in6); \ + } else { \ + tor_addr_from_ipv4h(&addr, (uint32_t) i); \ + } \ + } while (0) + + /* Make sure that country ID actually works. */ +#define SET_TEST_IPV6(i) \ + do { \ + set_uint32(in6.s6_addr + 12, htonl((uint32_t) (i))); \ + } while (0) +#define CHECK_COUNTRY(country, val) do { \ + /* test ipv4 country lookup */ \ + tt_str_op(country, OP_EQ, \ + geoip_get_country_name(geoip_get_country_by_ipv4(val))); \ + /* test ipv6 country lookup */ \ + SET_TEST_IPV6(val); \ + tt_str_op(country, OP_EQ, \ + geoip_get_country_name(geoip_get_country_by_ipv6(&in6))); \ + } while (0) + +/** Run unit tests for GeoIP code. */ +static void +test_geoip(void *arg) +{ + int i, j; + time_t now = 1281533250; /* 2010-08-11 13:27:30 UTC */ + char *s = NULL, *v = NULL; + const char *bridge_stats_1 = + "bridge-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "bridge-ips zz=24,xy=8\n" + "bridge-ip-versions v4=16,v6=16\n" + "bridge-ip-transports <OR>=24\n", + *dirreq_stats_1 = + "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "dirreq-v3-ips ab=8\n" + "dirreq-v3-reqs ab=8\n" + "dirreq-v3-resp ok=0,not-enough-sigs=0,unavailable=0,not-found=0," + "not-modified=0,busy=0\n" + "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v3-tunneled-dl complete=0,timeout=0,running=0\n", + *dirreq_stats_2 = + "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "dirreq-v3-ips \n" + "dirreq-v3-reqs \n" + "dirreq-v3-resp ok=0,not-enough-sigs=0,unavailable=0,not-found=0," + "not-modified=0,busy=0\n" + "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v3-tunneled-dl complete=0,timeout=0,running=0\n", + *dirreq_stats_3 = + "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "dirreq-v3-ips \n" + "dirreq-v3-reqs \n" + "dirreq-v3-resp ok=8,not-enough-sigs=0,unavailable=0,not-found=0," + "not-modified=0,busy=0\n" + "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v3-tunneled-dl complete=0,timeout=0,running=0\n", + *dirreq_stats_4 = + "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "dirreq-v3-ips \n" + "dirreq-v3-reqs \n" + "dirreq-v3-resp ok=8,not-enough-sigs=0,unavailable=0,not-found=0," + "not-modified=0,busy=0\n" + "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v3-tunneled-dl complete=0,timeout=0,running=4\n", + *entry_stats_1 = + "entry-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "entry-ips ab=8\n", + *entry_stats_2 = + "entry-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "entry-ips \n"; + tor_addr_t addr; + struct in6_addr in6; + + /* Populate the DB a bit. Add these in order, since we can't do the final + * 'sort' step. These aren't very good IP addresses, but they're perfectly + * fine uint32_t values. */ + (void)arg; + tt_int_op(0,OP_EQ, geoip_parse_entry("10,50,AB", AF_INET)); + tt_int_op(0,OP_EQ, geoip_parse_entry("52,90,XY", AF_INET)); + tt_int_op(0,OP_EQ, geoip_parse_entry("95,100,AB", AF_INET)); + tt_int_op(0,OP_EQ, geoip_parse_entry("\"105\",\"140\",\"ZZ\"", AF_INET)); + tt_int_op(0,OP_EQ, geoip_parse_entry("\"150\",\"190\",\"XY\"", AF_INET)); + tt_int_op(0,OP_EQ, geoip_parse_entry("\"200\",\"250\",\"AB\"", AF_INET)); + + /* Populate the IPv6 DB equivalently with fake IPs in the same range */ + tt_int_op(0,OP_EQ, geoip_parse_entry("::a,::32,AB", AF_INET6)); + tt_int_op(0,OP_EQ, geoip_parse_entry("::34,::5a,XY", AF_INET6)); + tt_int_op(0,OP_EQ, geoip_parse_entry("::5f,::64,AB", AF_INET6)); + tt_int_op(0,OP_EQ, geoip_parse_entry("::69,::8c,ZZ", AF_INET6)); + tt_int_op(0,OP_EQ, geoip_parse_entry("::96,::be,XY", AF_INET6)); + tt_int_op(0,OP_EQ, geoip_parse_entry("::c8,::fa,AB", AF_INET6)); + + /* We should have 4 countries: ??, ab, xy, zz. */ + tt_int_op(4,OP_EQ, geoip_get_n_countries()); + memset(&in6, 0, sizeof(in6)); + + CHECK_COUNTRY("??", 3); + CHECK_COUNTRY("ab", 32); + CHECK_COUNTRY("??", 5); + CHECK_COUNTRY("??", 51); + CHECK_COUNTRY("xy", 150); + CHECK_COUNTRY("xy", 190); + CHECK_COUNTRY("??", 2000); + + tt_int_op(0,OP_EQ, geoip_get_country_by_ipv4(3)); + SET_TEST_IPV6(3); + tt_int_op(0,OP_EQ, geoip_get_country_by_ipv6(&in6)); + + get_options_mutable()->BridgeRelay = 1; + get_options_mutable()->BridgeRecordUsageByCountry = 1; + /* Put 9 observations in AB... */ + for (i=32; i < 40; ++i) { + SET_TEST_ADDRESS(i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now-7200); + } + SET_TEST_ADDRESS(225); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now-7200); + /* and 3 observations in XY, several times. */ + for (j=0; j < 10; ++j) + for (i=52; i < 55; ++i) { + SET_TEST_ADDRESS(i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now-3600); + } + /* and 17 observations in ZZ... */ + for (i=110; i < 127; ++i) { + SET_TEST_ADDRESS(i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); + } + geoip_get_client_history(GEOIP_CLIENT_CONNECT, &s, &v); + tt_assert(s); + tt_assert(v); + tt_str_op("zz=24,ab=16,xy=8",OP_EQ, s); + tt_str_op("v4=16,v6=16",OP_EQ, v); + tor_free(s); + tor_free(v); + + /* Now clear out all the AB observations. */ + geoip_remove_old_clients(now-6000); + geoip_get_client_history(GEOIP_CLIENT_CONNECT, &s, &v); + tt_assert(s); + tt_assert(v); + tt_str_op("zz=24,xy=8",OP_EQ, s); + tt_str_op("v4=16,v6=16",OP_EQ, v); + tor_free(s); + tor_free(v); + + /* Start testing bridge statistics by making sure that we don't output + * bridge stats without initializing them. */ + s = geoip_format_bridge_stats(now + 86400); + tt_ptr_op(s, OP_EQ, NULL); + + /* Initialize stats and generate the bridge-stats history string out of + * the connecting clients added above. */ + geoip_bridge_stats_init(now); + s = geoip_format_bridge_stats(now + 86400); + tt_assert(s); + tt_str_op(bridge_stats_1,OP_EQ, s); + tor_free(s); + + /* Stop collecting bridge stats and make sure we don't write a history + * string anymore. */ + geoip_bridge_stats_term(); + s = geoip_format_bridge_stats(now + 86400); + tt_ptr_op(s, OP_EQ, NULL); + + /* Stop being a bridge and start being a directory mirror that gathers + * directory request statistics. */ + geoip_bridge_stats_term(); + get_options_mutable()->BridgeRelay = 0; + get_options_mutable()->BridgeRecordUsageByCountry = 0; + get_options_mutable()->DirReqStatistics = 1; + + /* Start testing dirreq statistics by making sure that we don't collect + * dirreq stats without initializing them. */ + SET_TEST_ADDRESS(100); + geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, NULL, now); + s = geoip_format_dirreq_stats(now + 86400); + tt_ptr_op(s, OP_EQ, NULL); + + /* Initialize stats, note one connecting client, and generate the + * dirreq-stats history string. */ + geoip_dirreq_stats_init(now); + SET_TEST_ADDRESS(100); + geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, NULL, now); + s = geoip_format_dirreq_stats(now + 86400); + tt_str_op(dirreq_stats_1,OP_EQ, s); + tor_free(s); + + /* Stop collecting stats, add another connecting client, and ensure we + * don't generate a history string. */ + geoip_dirreq_stats_term(); + SET_TEST_ADDRESS(101); + geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, NULL, now); + s = geoip_format_dirreq_stats(now + 86400); + tt_ptr_op(s, OP_EQ, NULL); + + /* Re-start stats, add a connecting client, reset stats, and make sure + * that we get an all empty history string. */ + geoip_dirreq_stats_init(now); + SET_TEST_ADDRESS(100); + geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, NULL, now); + geoip_reset_dirreq_stats(now); + s = geoip_format_dirreq_stats(now + 86400); + tt_str_op(dirreq_stats_2,OP_EQ, s); + tor_free(s); + + /* Note a successful network status response and make sure that it + * appears in the history string. */ + geoip_note_ns_response(GEOIP_SUCCESS); + s = geoip_format_dirreq_stats(now + 86400); + tt_str_op(dirreq_stats_3,OP_EQ, s); + tor_free(s); + + /* Start a tunneled directory request. */ + geoip_start_dirreq((uint64_t) 1, 1024, DIRREQ_TUNNELED); + s = geoip_format_dirreq_stats(now + 86400); + tt_str_op(dirreq_stats_4,OP_EQ, s); + tor_free(s); + + /* Stop collecting directory request statistics and start gathering + * entry stats. */ + geoip_dirreq_stats_term(); + get_options_mutable()->DirReqStatistics = 0; + get_options_mutable()->EntryStatistics = 1; + + /* Start testing entry statistics by making sure that we don't collect + * anything without initializing entry stats. */ + SET_TEST_ADDRESS(100); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); + s = geoip_format_entry_stats(now + 86400); + tt_ptr_op(s, OP_EQ, NULL); + + /* Initialize stats, note one connecting client, and generate the + * entry-stats history string. */ + geoip_entry_stats_init(now); + SET_TEST_ADDRESS(100); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); + s = geoip_format_entry_stats(now + 86400); + tt_str_op(entry_stats_1,OP_EQ, s); + tor_free(s); + + /* Stop collecting stats, add another connecting client, and ensure we + * don't generate a history string. */ + geoip_entry_stats_term(); + SET_TEST_ADDRESS(101); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); + s = geoip_format_entry_stats(now + 86400); + tt_ptr_op(s, OP_EQ, NULL); + + /* Re-start stats, add a connecting client, reset stats, and make sure + * that we get an all empty history string. */ + geoip_entry_stats_init(now); + SET_TEST_ADDRESS(100); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now); + geoip_reset_entry_stats(now); + s = geoip_format_entry_stats(now + 86400); + tt_str_op(entry_stats_2,OP_EQ, s); + tor_free(s); + + /* Test the OOM handler. Add a client, run the OOM. */ + geoip_entry_stats_init(now); + SET_TEST_ADDRESS(100); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, + now - (12 * 60 * 60)); + /* We've seen this 12 hours ago. Run the OOM, it should clean the entry + * because it is above the minimum cutoff of 4 hours. */ + size_t bytes_removed = geoip_client_cache_handle_oom(now, 1000); + tt_size_op(bytes_removed, OP_GT, 0); + + /* Do it again but this time with an entry with a lower cutoff. */ + geoip_entry_stats_init(now); + SET_TEST_ADDRESS(100); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, + now - (3 * 60 * 60)); + bytes_removed = geoip_client_cache_handle_oom(now, 1000); + tt_size_op(bytes_removed, OP_EQ, 0); + + /* Stop collecting entry statistics. */ + geoip_entry_stats_term(); + get_options_mutable()->EntryStatistics = 0; + + done: + tor_free(s); + tor_free(v); +} + +static void +test_geoip_with_pt(void *arg) +{ + time_t now = 1281533250; /* 2010-08-11 13:27:30 UTC */ + char *s = NULL; + int i; + tor_addr_t addr; + struct in6_addr in6; + + (void)arg; + get_options_mutable()->BridgeRelay = 1; + get_options_mutable()->BridgeRecordUsageByCountry = 1; + + memset(&in6, 0, sizeof(in6)); + + /* No clients seen yet. */ + s = geoip_get_transport_history(); + tor_assert(!s); + + /* 4 connections without a pluggable transport */ + for (i=0; i < 4; ++i) { + SET_TEST_ADDRESS(i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, NULL, now-7200); + } + + /* 9 connections with "alpha" */ + for (i=4; i < 13; ++i) { + SET_TEST_ADDRESS(i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "alpha", now-7200); + } + + /* one connection with "beta" */ + SET_TEST_ADDRESS(13); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "beta", now-7200); + + /* 14 connections with "charlie" */ + for (i=14; i < 28; ++i) { + SET_TEST_ADDRESS(i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "charlie", now-7200); + } + + /* 131 connections with "ddr" */ + for (i=28; i < 159; ++i) { + SET_TEST_ADDRESS(i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "ddr", now-7200); + } + + /* 8 connections with "entropy" */ + for (i=159; i < 167; ++i) { + SET_TEST_ADDRESS(i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "entropy", now-7200); + } + + /* 2 connections from the same IP with two different transports. */ + SET_TEST_ADDRESS(++i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "fire", now-7200); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, "google", now-7200); + + /* Test the transport history string. */ + s = geoip_get_transport_history(); + tor_assert(s); + tt_str_op(s,OP_EQ, "<OR>=8,alpha=16,beta=8,charlie=16,ddr=136," + "entropy=8,fire=8,google=8"); + + /* Stop collecting entry statistics. */ + geoip_entry_stats_term(); + get_options_mutable()->EntryStatistics = 0; + + done: + tor_free(s); +} + +#undef SET_TEST_ADDRESS +#undef SET_TEST_IPV6 +#undef CHECK_COUNTRY + +static void +test_geoip_load_file(void *arg) +{ + (void)arg; + char *contents = NULL; + char *dhex = NULL; + + /* A nonexistant filename should fail. */ + tt_int_op(-1, OP_EQ, + geoip_load_file(AF_INET, "/you/did/not/put/a/file/here/I/hope")); + + /* We start out with only "Ningunpartia" in the database. */ + tt_int_op(1, OP_EQ, geoip_get_n_countries()); + tt_str_op("??", OP_EQ, geoip_get_country_name(0)); + /* Any lookup attempt should say "-1" because we have no info */ + tt_int_op(-1, OP_EQ, geoip_get_country_by_ipv4(0x01020304)); + /* There should be no 'digest' for a nonexistant file */ + tt_str_op("0000000000000000000000000000000000000000", OP_EQ, + geoip_db_digest(AF_INET)); + + const char FNAME[] = SRCDIR "/src/config/geoip"; + int rv = geoip_load_file(AF_INET, FNAME); + if (rv != 0) { + TT_GRIPE(("Unable to load geoip from %s", escaped(FNAME))); + } + tt_int_op(0, OP_EQ, rv); + + /* Check that we loaded some countries; this will fail if there are ever + * fewer than 50 countries in the world. */ + tt_int_op(geoip_get_n_countries(), OP_GE, 50); + + /* Let's see where 8.8.8.8 is. */ + int country = geoip_get_country_by_ipv4(0x08080808); + tt_int_op(country, OP_GE, 1); /* It shouldn't be 'unknown' or 'nowhere' */ + const char *cc = geoip_get_country_name(country); + tt_int_op(strlen(cc), OP_EQ, 2); + + /* The digest should be set.... */ + tt_str_op("0000000000000000000000000000000000000000", OP_NE, + geoip_db_digest(AF_INET)); + + /* And it should be set correctly */ + contents = read_file_to_str(FNAME, RFTS_BIN, NULL); + uint8_t d[DIGEST_LEN]; + crypto_digest((char*)d, contents, strlen(contents)); + dhex = tor_strdup(hex_str((char*)d, DIGEST_LEN)); + tt_str_op(dhex, OP_EQ, geoip_db_digest(AF_INET)); + + /* Make sure geoip_free_all() works. */ + geoip_free_all(); + tt_int_op(1, OP_EQ, geoip_get_n_countries()); + tt_str_op("??", OP_EQ, geoip_get_country_name(0)); + tt_int_op(-1, OP_EQ, geoip_get_country_by_ipv4(0x01020304)); + tt_str_op("0000000000000000000000000000000000000000", OP_EQ, + geoip_db_digest(AF_INET)); // <--- nick bets this will fail. + + done: + tor_free(contents); + tor_free(dhex); +} + +#define ENT(name) \ + { #name, test_ ## name , 0, NULL, NULL } +#define FORK(name) \ + { #name, test_ ## name , TT_FORK, NULL, NULL } + +struct testcase_t geoip_tests[] = { + { "geoip", test_geoip, TT_FORK, NULL, NULL }, + { "geoip_with_pt", test_geoip_with_pt, TT_FORK, NULL, NULL }, + { "load_file", test_geoip_load_file, TT_FORK, NULL, NULL }, + + END_OF_TESTCASES +}; + diff --git a/src/test/test_helpers.c b/src/test/test_helpers.c index 0da9cf64d0..6ada64dcff 100644 --- a/src/test/test_helpers.c +++ b/src/test/test_helpers.c @@ -33,7 +33,6 @@ DISABLE_GCC_WARNING(overlength-strings) * at large. */ #endif #include "test_descriptors.inc" -#include "or.h" #include "circuitlist.h" #ifdef HAVE_CFLAG_WOVERLENGTH_STRINGS ENABLE_GCC_WARNING(overlength-strings) @@ -156,7 +155,7 @@ mock_tor_addr_lookup__fail_on_bad_addrs(const char *name, /* Helper for test_conn_get_connection() */ static int -fake_close_socket(evutil_socket_t sock) +fake_close_socket(tor_socket_t sock) { (void)sock; return 0; diff --git a/src/test/test_hs_client.c b/src/test/test_hs_client.c index 7ee7210bc9..58e12abca0 100644 --- a/src/test/test_hs_client.c +++ b/src/test/test_hs_client.c @@ -213,12 +213,12 @@ test_e2e_rend_circuit_setup_legacy(void *arg) tt_int_op(retval, OP_EQ, 1); /* Check the digest algo */ - tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->f_digest), + tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->crypto.f_digest), OP_EQ, DIGEST_SHA1); - tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->b_digest), + tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->crypto.b_digest), OP_EQ, DIGEST_SHA1); - tt_assert(or_circ->cpath->f_crypto); - tt_assert(or_circ->cpath->b_crypto); + tt_assert(or_circ->cpath->crypto.f_crypto); + tt_assert(or_circ->cpath->crypto.b_crypto); /* Ensure that circ purpose was changed */ tt_int_op(or_circ->base_.purpose, OP_EQ, CIRCUIT_PURPOSE_C_REND_JOINED); @@ -283,12 +283,12 @@ test_e2e_rend_circuit_setup(void *arg) tt_int_op(retval, OP_EQ, 1); /* Check that the crypt path has prop224 algorithm parameters */ - tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->f_digest), + tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->crypto.f_digest), OP_EQ, DIGEST_SHA3_256); - tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->b_digest), + tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->crypto.b_digest), OP_EQ, DIGEST_SHA3_256); - tt_assert(or_circ->cpath->f_crypto); - tt_assert(or_circ->cpath->b_crypto); + tt_assert(or_circ->cpath->crypto.f_crypto); + tt_assert(or_circ->cpath->crypto.b_crypto); /* Ensure that circ purpose was changed */ tt_int_op(or_circ->base_.purpose, OP_EQ, CIRCUIT_PURPOSE_C_REND_JOINED); diff --git a/src/test/test_hs_descriptor.c b/src/test/test_hs_descriptor.c index 8e9d461c40..388b85f975 100644 --- a/src/test/test_hs_descriptor.c +++ b/src/test/test_hs_descriptor.c @@ -9,6 +9,7 @@ #define HS_DESCRIPTOR_PRIVATE #include "crypto_ed25519.h" +#include "crypto_digest.h" #include "ed25519_cert.h" #include "or.h" #include "hs_descriptor.h" diff --git a/src/test/test_hs_intropoint.c b/src/test/test_hs_intropoint.c index 55dfafbeac..fe3236c331 100644 --- a/src/test/test_hs_intropoint.c +++ b/src/test/test_hs_intropoint.c @@ -14,7 +14,6 @@ #include "test.h" #include "log_test_helpers.h" #include "crypto.h" -#include "log_test_helpers.h" #include "or.h" #include "circuitlist.h" diff --git a/src/test/test_hs_service.c b/src/test/test_hs_service.c index 2e5280610f..481521520c 100644 --- a/src/test/test_hs_service.c +++ b/src/test/test_hs_service.c @@ -173,12 +173,12 @@ test_e2e_rend_circuit_setup(void *arg) tt_int_op(retval, OP_EQ, 1); /* Check the digest algo */ - tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->f_digest), + tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->crypto.f_digest), OP_EQ, DIGEST_SHA3_256); - tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->b_digest), + tt_int_op(crypto_digest_get_algorithm(or_circ->cpath->crypto.b_digest), OP_EQ, DIGEST_SHA3_256); - tt_assert(or_circ->cpath->f_crypto); - tt_assert(or_circ->cpath->b_crypto); + tt_assert(or_circ->cpath->crypto.f_crypto); + tt_assert(or_circ->cpath->crypto.b_crypto); /* Ensure that circ purpose was changed */ tt_int_op(or_circ->base_.purpose, OP_EQ, CIRCUIT_PURPOSE_S_REND_JOINED); @@ -1237,7 +1237,7 @@ test_build_update_descriptors(void *arg) node->is_running = node->is_valid = node->is_fast = node->is_stable = 1; } - /* We have to set thise, or the lack of microdescriptors for these + /* We have to set this, or the lack of microdescriptors for these * nodes will make them unusable. */ get_options_mutable()->UseMicrodescriptors = 0; diff --git a/src/test/test_options.c b/src/test/test_options.c index eaf5034397..af349ed015 100644 --- a/src/test/test_options.c +++ b/src/test/test_options.c @@ -2422,37 +2422,6 @@ test_options_validate__circuits(void *ignored) } static void -test_options_validate__port_forwarding(void *ignored) -{ - (void)ignored; - int ret; - char *msg; - options_test_data_t *tdata = NULL; - - free_options_test_data(tdata); - tdata = get_options_test_data(TEST_OPTIONS_DEFAULT_VALUES - "PortForwarding 1\nSandbox 1\n"); - ret = options_validate(tdata->old_opt, tdata->opt, tdata->def_opt, 0, &msg); - tt_int_op(ret, OP_EQ, -1); - tt_str_op(msg, OP_EQ, "PortForwarding is not compatible with Sandbox;" - " at most one can be set"); - tor_free(msg); - - free_options_test_data(tdata); - tdata = get_options_test_data(TEST_OPTIONS_DEFAULT_VALUES - "PortForwarding 1\nSandbox 0\n"); - ret = options_validate(tdata->old_opt, tdata->opt, tdata->def_opt, 0, &msg); - tt_int_op(ret, OP_EQ, 0); - tt_assert(!msg); - tor_free(msg); - - done: - free_options_test_data(tdata); - policies_free_all(); - tor_free(msg); -} - -static void test_options_validate__tor2web(void *ignored) { (void)ignored; @@ -4261,7 +4230,6 @@ struct testcase_t options_tests[] = { LOCAL_VALIDATE_TEST(path_bias), LOCAL_VALIDATE_TEST(bandwidth), LOCAL_VALIDATE_TEST(circuits), - LOCAL_VALIDATE_TEST(port_forwarding), LOCAL_VALIDATE_TEST(tor2web), LOCAL_VALIDATE_TEST(rend), LOCAL_VALIDATE_TEST(single_onion), diff --git a/src/test/test_relaycrypt.c b/src/test/test_relaycrypt.c new file mode 100644 index 0000000000..eba9897184 --- /dev/null +++ b/src/test/test_relaycrypt.c @@ -0,0 +1,184 @@ +/* Copyright 2001-2004 Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2018, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "circuitbuild.h" +#define CIRCUITLIST_PRIVATE +#include "circuitlist.h" +#include "relay.h" +#include "relay_crypto.h" +#include "test.h" + +static const char KEY_MATERIAL[3][CPATH_KEY_MATERIAL_LEN] = { + " 'My public key is in this signed x509 object', said Tom assertively.", + "'Let's chart the pedal phlanges in the tomb', said Tom cryptographically", + " 'Segmentation fault bugs don't _just happen_', said Tom seethingly.", +}; + +typedef struct testing_circuitset_t { + or_circuit_t *or_circ[3]; + origin_circuit_t *origin_circ; +} testing_circuitset_t; + +static int testing_circuitset_teardown(const struct testcase_t *testcase, + void *ptr); + +static void * +testing_circuitset_setup(const struct testcase_t *testcase) +{ + testing_circuitset_t *cs = tor_malloc_zero(sizeof(testing_circuitset_t)); + int i; + + for (i=0; i<3; ++i) { + cs->or_circ[i] = or_circuit_new(0, NULL); + tt_int_op(0, OP_EQ, + relay_crypto_init(&cs->or_circ[i]->crypto, + KEY_MATERIAL[i], sizeof(KEY_MATERIAL[i]), + 0, 0)); + } + + cs->origin_circ = origin_circuit_new(); + cs->origin_circ->base_.purpose = CIRCUIT_PURPOSE_C_GENERAL; + for (i=0; i<3; ++i) { + crypt_path_t *hop = tor_malloc_zero(sizeof(*hop)); + relay_crypto_init(&hop->crypto, KEY_MATERIAL[i], sizeof(KEY_MATERIAL[i]), + 0, 0); + hop->state = CPATH_STATE_OPEN; + onion_append_to_cpath(&cs->origin_circ->cpath, hop); + tt_ptr_op(hop, OP_EQ, cs->origin_circ->cpath->prev); + } + + return cs; + done: + testing_circuitset_teardown(testcase, cs); + return NULL; +} + +static int +testing_circuitset_teardown(const struct testcase_t *testcase, void *ptr) +{ + (void)testcase; + testing_circuitset_t *cs = ptr; + int i; + for (i=0; i<3; ++i) { + circuit_free_(TO_CIRCUIT(cs->or_circ[i])); + } + circuit_free_(TO_CIRCUIT(cs->origin_circ)); + tor_free(cs); + return 1; +} + +static const struct testcase_setup_t relaycrypt_setup = { + testing_circuitset_setup, testing_circuitset_teardown +}; + +/* Test encrypting a cell to the final hop on a circuit, decrypting it + * at each hop, and recognizing it at the other end. Then do it again + * and again as the state evolves. */ +static void +test_relaycrypt_outbound(void *arg) +{ + testing_circuitset_t *cs = arg; + tt_assert(cs); + + relay_header_t rh; + cell_t orig; + cell_t encrypted; + int i, j; + + for (i = 0; i < 50; ++i) { + crypto_rand((char *)&orig, sizeof(orig)); + + relay_header_unpack(&rh, orig.payload); + rh.recognized = 0; + memset(rh.integrity, 0, sizeof(rh.integrity)); + relay_header_pack(orig.payload, &rh); + + memcpy(&encrypted, &orig, sizeof(orig)); + + /* Encrypt the cell to the last hop */ + relay_encrypt_cell_outbound(&encrypted, cs->origin_circ, + cs->origin_circ->cpath->prev); + + for (j = 0; j < 3; ++j) { + crypt_path_t *layer_hint = NULL; + char recognized = 0; + int r = relay_decrypt_cell(TO_CIRCUIT(cs->or_circ[j]), + &encrypted, + CELL_DIRECTION_OUT, + &layer_hint, &recognized); + tt_int_op(r, OP_EQ, 0); + tt_ptr_op(layer_hint, OP_EQ, NULL); + tt_int_op(recognized != 0, OP_EQ, j == 2); + } + + tt_mem_op(orig.payload, OP_EQ, encrypted.payload, CELL_PAYLOAD_SIZE); + } + + done: + ; +} + +/* As above, but simulate inbound cells from the last hop. */ +static void +test_relaycrypt_inbound(void *arg) +{ + testing_circuitset_t *cs = arg; + tt_assert(cs); + + relay_header_t rh; + cell_t orig; + cell_t encrypted; + int i, j; + + for (i = 0; i < 50; ++i) { + crypto_rand((char *)&orig, sizeof(orig)); + + relay_header_unpack(&rh, orig.payload); + rh.recognized = 0; + memset(rh.integrity, 0, sizeof(rh.integrity)); + relay_header_pack(orig.payload, &rh); + + memcpy(&encrypted, &orig, sizeof(orig)); + + /* Encrypt the cell to the last hop */ + relay_encrypt_cell_inbound(&encrypted, cs->or_circ[2]); + + crypt_path_t *layer_hint = NULL; + char recognized = 0; + int r; + for (j = 1; j >= 0; --j) { + r = relay_decrypt_cell(TO_CIRCUIT(cs->or_circ[j]), + &encrypted, + CELL_DIRECTION_IN, + &layer_hint, &recognized); + tt_int_op(r, OP_EQ, 0); + tt_ptr_op(layer_hint, OP_EQ, NULL); + tt_int_op(recognized, OP_EQ, 0); + } + + relay_decrypt_cell(TO_CIRCUIT(cs->origin_circ), + &encrypted, + CELL_DIRECTION_IN, + &layer_hint, &recognized); + tt_int_op(r, OP_EQ, 0); + tt_int_op(recognized, OP_EQ, 1); + tt_ptr_op(layer_hint, OP_EQ, cs->origin_circ->cpath->prev); + + tt_mem_op(orig.payload, OP_EQ, encrypted.payload, CELL_PAYLOAD_SIZE); + } + done: + ; +} + +#define TEST(name) \ + { # name, test_relaycrypt_ ## name, 0, &relaycrypt_setup, NULL } + +struct testcase_t relaycrypt_tests[] = { + TEST(outbound), + TEST(inbound), + END_OF_TESTCASES +}; + diff --git a/src/test/test_rust.sh b/src/test/test_rust.sh index d87336e700..c35c57456f 100755 --- a/src/test/test_rust.sh +++ b/src/test/test_rust.sh @@ -12,4 +12,3 @@ CARGO_TARGET_DIR="${abs_top_builddir:-../../..}/src/rust/target" \ --manifest-path '{}' \; exit $? - diff --git a/src/test/test_scheduler.c b/src/test/test_scheduler.c index ebba71266c..841fc69456 100644 --- a/src/test/test_scheduler.c +++ b/src/test/test_scheduler.c @@ -4,7 +4,6 @@ #include "orconfig.h" #include <math.h> -#include <event2/event.h> #define SCHEDULER_KIST_PRIVATE #define TOR_CHANNEL_INTERNAL_ @@ -101,62 +100,6 @@ mock_kist_networkstatus_get_param( return 12; } -/* Event base for scheduelr tests */ -static struct event_base *mock_event_base = NULL; -/* Setup for mock event stuff */ -static void mock_event_free_all(void); -static void mock_event_init(void); -static void -mock_event_free_all(void) -{ - tt_ptr_op(mock_event_base, OP_NE, NULL); - - if (mock_event_base) { - event_base_free(mock_event_base); - mock_event_base = NULL; - } - - tt_ptr_op(mock_event_base, OP_EQ, NULL); - - done: - return; -} - -static void -mock_event_init(void) -{ - struct event_config *cfg = NULL; - - tt_ptr_op(mock_event_base, OP_EQ, NULL); - - /* - * Really cut down from tor_libevent_initialize of - * src/common/compat_libevent.c to kill config dependencies - */ - - if (!mock_event_base) { - cfg = event_config_new(); -#if LIBEVENT_VERSION_NUMBER >= V(2,0,9) - /* We can enable changelist support with epoll, since we don't give - * Libevent any dup'd fds. This lets us avoid some syscalls. */ - event_config_set_flag(cfg, EVENT_BASE_FLAG_EPOLL_USE_CHANGELIST); -#endif - mock_event_base = event_base_new_with_config(cfg); - event_config_free(cfg); - } - - tt_ptr_op(mock_event_base, OP_NE, NULL); - - done: - return; -} - -static struct event_base * -tor_libevent_get_base_mock(void) -{ - return mock_event_base; -} - static int scheduler_compare_channels_mock(const void *c1_v, const void *c2_v) @@ -417,9 +360,7 @@ perform_channel_state_tests(int KISTSchedRunInterval, int sched_type) mocked_options.KISTSchedRunInterval = KISTSchedRunInterval; set_scheduler_options(sched_type); - /* Set up libevent and scheduler */ - mock_event_init(); - MOCK(tor_libevent_get_base, tor_libevent_get_base_mock); + /* Set up scheduler */ scheduler_init(); /* * Install the compare channels mock so we can test @@ -523,14 +464,12 @@ perform_channel_state_tests(int KISTSchedRunInterval, int sched_type) channel_free_all(); scheduler_free_all(); - mock_event_free_all(); done: tor_free(ch1); tor_free(ch2); UNMOCK(scheduler_compare_channels); - UNMOCK(tor_libevent_get_base); UNMOCK(get_options); cleanup_scheduler_options(); @@ -635,10 +574,7 @@ test_scheduler_loop_vanilla(void *arg) set_scheduler_options(SCHEDULER_VANILLA); mocked_options.KISTSchedRunInterval = 0; - /* Set up libevent and scheduler */ - - mock_event_init(); - MOCK(tor_libevent_get_base, tor_libevent_get_base_mock); + /* Set up scheduler */ scheduler_init(); /* * Install the compare channels mock so we can test @@ -786,7 +722,6 @@ test_scheduler_loop_vanilla(void *arg) channel_flush_some_cells_mock_free_all(); channel_free_all(); scheduler_free_all(); - mock_event_free_all(); done: tor_free(ch1); @@ -795,7 +730,6 @@ test_scheduler_loop_vanilla(void *arg) UNMOCK(channel_flush_some_cells); UNMOCK(scheduler_compare_channels); - UNMOCK(tor_libevent_get_base); UNMOCK(get_options); } @@ -917,8 +851,6 @@ test_scheduler_initfree(void *arg) tt_ptr_op(channels_pending, ==, NULL); tt_ptr_op(run_sched_ev, ==, NULL); - mock_event_init(); - MOCK(tor_libevent_get_base, tor_libevent_get_base_mock); MOCK(get_options, mock_get_options); set_scheduler_options(SCHEDULER_KIST); set_scheduler_options(SCHEDULER_KIST_LITE); @@ -935,9 +867,6 @@ test_scheduler_initfree(void *arg) scheduler_free_all(); - UNMOCK(tor_libevent_get_base); - mock_event_free_all(); - tt_ptr_op(channels_pending, ==, NULL); tt_ptr_op(run_sched_ev, ==, NULL); diff --git a/src/test/test_util.c b/src/test/test_util.c index 036f739b89..ce8567d9af 100644 --- a/src/test/test_util.c +++ b/src/test/test_util.c @@ -16,6 +16,7 @@ #include "memarea.h" #include "util_process.h" #include "log_test_helpers.h" +#include "compress_zstd.h" #ifdef HAVE_PWD_H #include <pwd.h> @@ -2396,6 +2397,37 @@ test_util_compress_stream_impl(compress_method_t method, tor_free(buf3); } +/** Setup function for compression tests: handles x-zstd:nostatic + */ +static void * +compression_test_setup(const struct testcase_t *testcase) +{ + tor_assert(testcase->setup_data); + tor_assert(testcase->setup_data != (void*)TT_SKIP); + const char *methodname = testcase->setup_data; + + if (!strcmp(methodname, "x-zstd:nostatic")) { + methodname = "x-zstd"; + tor_zstd_set_static_apis_disabled_for_testing(1); + } + + return (void *)methodname; +} + +/** Cleanup for compression tests: disables nostatic */ +static int +compression_test_cleanup(const struct testcase_t *testcase, void *ptr) +{ + (void)testcase; + (void)ptr; + tor_zstd_set_static_apis_disabled_for_testing(0); + return 1; +} + +static const struct testcase_setup_t compress_setup = { + compression_test_setup, compression_test_cleanup +}; + /** Run unit tests for compression functions */ static void test_util_compress(void *arg) @@ -6122,22 +6154,22 @@ test_util_get_unquoted_path(void *arg) { #name, test_util_ ## name, flags, NULL, NULL } #define COMPRESS(name, identifier) \ - { "compress/" #name, test_util_compress, 0, &passthrough_setup, \ + { "compress/" #name, test_util_compress, 0, &compress_setup, \ (char*)(identifier) } #define COMPRESS_CONCAT(name, identifier) \ { "compress_concat/" #name, test_util_decompress_concatenated, 0, \ - &passthrough_setup, \ + &compress_setup, \ (char*)(identifier) } #define COMPRESS_JUNK(name, identifier) \ { "compress_junk/" #name, test_util_decompress_junk, 0, \ - &passthrough_setup, \ + &compress_setup, \ (char*)(identifier) } #define COMPRESS_DOS(name, identifier) \ { "compress_dos/" #name, test_util_decompress_dos, 0, \ - &passthrough_setup, \ + &compress_setup, \ (char*)(identifier) } #ifdef _WIN32 @@ -6168,11 +6200,13 @@ struct testcase_t util_tests[] = { COMPRESS(gzip, "gzip"), COMPRESS(lzma, "x-tor-lzma"), COMPRESS(zstd, "x-zstd"), + COMPRESS(zstd_nostatic, "x-zstd:nostatic"), COMPRESS(none, "identity"), COMPRESS_CONCAT(zlib, "deflate"), COMPRESS_CONCAT(gzip, "gzip"), COMPRESS_CONCAT(lzma, "x-tor-lzma"), COMPRESS_CONCAT(zstd, "x-zstd"), + COMPRESS_CONCAT(zstd_nostatic, "x-zstd:nostatic"), COMPRESS_CONCAT(none, "identity"), COMPRESS_JUNK(zlib, "deflate"), COMPRESS_JUNK(gzip, "gzip"), @@ -6181,6 +6215,7 @@ struct testcase_t util_tests[] = { COMPRESS_DOS(gzip, "gzip"), COMPRESS_DOS(lzma, "x-tor-lzma"), COMPRESS_DOS(zstd, "x-zstd"), + COMPRESS_DOS(zstd_nostatic, "x-zstd:nostatic"), UTIL_TEST(gzip_compression_bomb, TT_FORK), UTIL_LEGACY(datadir), UTIL_LEGACY(memarea), diff --git a/src/test/test_workqueue.c b/src/test/test_workqueue.c index 2b03173717..940973cda5 100644 --- a/src/test/test_workqueue.c +++ b/src/test/test_workqueue.c @@ -12,7 +12,6 @@ #include "compat_libevent.h" #include <stdio.h> -#include <event2/event.h> #define MAX_INFLIGHT (1<<16) @@ -159,6 +158,7 @@ static tor_weak_rng_t weak_rng; static int n_sent = 0; static int rsa_sent = 0; static int ecdh_sent = 0; +static int n_received_previously = 0; static int n_received = 0; static int no_shutdown = 0; @@ -230,7 +230,7 @@ add_n_work_items(threadpool_t *tp, int n) ent = add_work(tp); if (! ent) { puts("Z"); - tor_event_base_loopexit(tor_libevent_get_base(), NULL); + tor_libevent_exit_loop_after_delay(tor_libevent_get_base(), NULL); return -1; } if (n_try_cancel < opt_n_cancel && @@ -256,19 +256,13 @@ add_n_work_items(threadpool_t *tp, int n) static int shutting_down = 0; static void -replysock_readable_cb(tor_socket_t sock, short what, void *arg) +replysock_readable_cb(threadpool_t *tp) { - threadpool_t *tp = arg; - replyqueue_t *rq = threadpool_get_replyqueue(tp); - - int old_r = n_received; - (void) sock; - (void) what; - - replyqueue_process(rq); - if (old_r == n_received) + if (n_received_previously == n_received) return; + n_received_previously = n_received; + if (opt_verbose) { printf("%d / %d", n_received, n_sent); if (opt_n_cancel) @@ -308,7 +302,7 @@ replysock_readable_cb(tor_socket_t sock, short what, void *arg) handle_reply_shutdown, NULL); { struct timeval limit = { 2, 0 }; - tor_event_base_loopexit(tor_libevent_get_base(), &limit); + tor_libevent_exit_loop_after_delay(tor_libevent_get_base(), &limit); } } } @@ -337,7 +331,6 @@ main(int argc, char **argv) threadpool_t *tp; int i; tor_libevent_cfg evcfg; - struct event *ev; uint32_t as_flags = 0; for (i = 1; i < argc; ++i) { @@ -411,11 +404,11 @@ main(int argc, char **argv) memset(&evcfg, 0, sizeof(evcfg)); tor_libevent_initialize(&evcfg); - ev = tor_event_new(tor_libevent_get_base(), - replyqueue_get_socket(rq), EV_READ|EV_PERSIST, - replysock_readable_cb, tp); - - event_add(ev, NULL); + { + int r = threadpool_register_reply_event(tp, + replysock_readable_cb); + tor_assert(r == 0); + } #ifdef TRACK_RESPONSES handled = bitarray_init_zero(opt_n_items); @@ -433,10 +426,10 @@ main(int argc, char **argv) { struct timeval limit = { 180, 0 }; - tor_event_base_loopexit(tor_libevent_get_base(), &limit); + tor_libevent_exit_loop_after_delay(tor_libevent_get_base(), &limit); } - event_base_loop(tor_libevent_get_base(), 0); + tor_libevent_run_event_loop(tor_libevent_get_base(), 0); if (n_sent != opt_n_items || n_received+n_successful_cancel != n_sent) { printf("%d vs %d\n", n_sent, opt_n_items); diff --git a/src/test/testing_common.c b/src/test/testing_common.c index 52729147b2..b9b36d96d0 100644 --- a/src/test/testing_common.c +++ b/src/test/testing_common.c @@ -29,8 +29,6 @@ #include <dirent.h> #endif /* defined(_WIN32) */ -#include "or.h" - #ifdef USE_DMALLOC #include <dmalloc.h> #include "main.h" diff --git a/src/tools/tor-fw-helper/README b/src/tools/tor-fw-helper/README deleted file mode 100644 index 6a1ecaa1e4..0000000000 --- a/src/tools/tor-fw-helper/README +++ /dev/null @@ -1,10 +0,0 @@ - -We no longer recommend the use of this tool. Instead, please use the -pure-Go version of tor-fw-helper available at - https://gitweb.torproject.org/tor-fw-helper.git - -Why? - -The C code here was fine, but frankly: we don't trust the underlying -libraries. They don't seem to have been written with network security -in mind, and we have very little faith in their safety. diff --git a/src/tools/tor-gencert.c b/src/tools/tor-gencert.c index fb7465c0eb..639a6fbc23 100644 --- a/src/tools/tor-gencert.c +++ b/src/tools/tor-gencert.c @@ -36,10 +36,10 @@ ENABLE_GCC_WARNING(redundant-decls) #include <assert.h> #endif -#include "compat.h" #include "util.h" #include "torlog.h" #include "crypto.h" +#include "crypto_digest.h" #include "address.h" #include "util_format.h" diff --git a/src/win32/orconfig.h b/src/win32/orconfig.h index a8c6f9af3c..238e0f6962 100644 --- a/src/win32/orconfig.h +++ b/src/win32/orconfig.h @@ -218,7 +218,7 @@ #define USING_TWOS_COMPLEMENT /* Version number of package */ -#define VERSION "0.3.3.4-alpha-dev" +#define VERSION "0.3.4.0-alpha-dev" |