diff options
215 files changed, 15098 insertions, 2888 deletions
diff --git a/.gitignore b/.gitignore index 135df7b204..c1dff8bfec 100644 --- a/.gitignore +++ b/.gitignore @@ -27,6 +27,7 @@ cscope.* # OSX junk *.dSYM +.DS_Store # / /Makefile @@ -1,3 +1,135 @@ +Changes in version 0.2.7.6 - 2015-12-10 + Tor version 0.2.7.6 fixes a major bug in entry guard selection, as + well as a minor bug in hidden service reliability. + + o Major bugfixes (guard selection): + - Actually look at the Guard flag when selecting a new directory + guard. When we implemented the directory guard design, we + accidentally started treating all relays as if they have the Guard + flag during guard selection, leading to weaker anonymity and worse + performance. Fixes bug 17772; bugfix on 0.2.4.8-alpha. Discovered + by Mohsen Imani. + + o Minor features (geoip): + - Update geoip and geoip6 to the December 1 2015 Maxmind GeoLite2 + Country database. + + o Minor bugfixes (compilation): + - When checking for net/pfvar.h, include netinet/in.h if possible. + This fixes transparent proxy detection on OpenBSD. Fixes bug + 17551; bugfix on 0.1.2.1-alpha. Patch from "rubiate". + - Fix a compilation warning with Clang 3.6: Do not check the + presence of an address which can never be NULL. Fixes bug 17781. + + o Minor bugfixes (correctness): + - When displaying an IPv6 exit policy, include the mask bits + correctly even when the number is greater than 31. Fixes bug + 16056; bugfix on 0.2.4.7-alpha. Patch from "gturner". + - The wrong list was used when looking up expired intro points in a + rend service object, causing what we think could be reachability + issues for hidden services, and triggering a BUG log. Fixes bug + 16702; bugfix on 0.2.7.2-alpha. + - Fix undefined behavior in the tor_cert_checksig function. Fixes + bug 17722; bugfix on 0.2.7.2-alpha. + + +Changes in version 0.2.7.5 - 2015-11-20 + The Tor 0.2.7 release series is dedicated to the memory of Tor user + and privacy advocate Caspar Bowden (1961-2015). Caspar worked + tirelessly to advocate human rights regardless of national borders, + and oppose the encroachments of mass surveillance. He opposed national + exceptionalism, he brought clarity to legal and policy debates, he + understood and predicted the impact of mass surveillance on the world, + and he laid the groundwork for resisting it. While serving on the Tor + Project's board of directors, he brought us his uncompromising focus + on technical excellence in the service of humankind. Caspar was an + inimitable force for good and a wonderful friend. He was kind, + humorous, generous, gallant, and believed we should protect one + another without exception. We honor him here for his ideals, his + efforts, and his accomplishments. Please honor his memory with works + that would make him proud. + + Tor 0.2.7.5 is the first stable release in the Tor 0.2.7 series. + + The 0.2.7 series adds a more secure identity key type for relays, + improves cryptography performance, resolves several longstanding + hidden-service performance issues, improves controller support for + hidden services, and includes small bugfixes and performance + improvements throughout the program. This release series also includes + more tests than before, and significant simplifications to which parts + of Tor invoke which others. + + (This release contains no code changes since 0.2.7.4-rc.) + + +Changes in version 0.2.7.4-rc - 2015-10-21 + Tor 0.2.7.4-rc is the second release candidate in the 0.2.7 series. It + fixes some important memory leaks, and a scary-looking (but mostly + harmless in practice) invalid-read bug. It also has a few small + bugfixes, notably fixes for compilation and portability on different + platforms. If no further significant bounds are found, the next + release will the the official stable release. + + o Major bugfixes (security, correctness): + - Fix an error that could cause us to read 4 bytes before the + beginning of an openssl string. This bug could be used to cause + Tor to crash on systems with unusual malloc implementations, or + systems with unusual hardening installed. Fixes bug 17404; bugfix + on 0.2.3.6-alpha. + + o Major bugfixes (correctness): + - Fix a use-after-free bug in validate_intro_point_failure(). Fixes + bug 17401; bugfix on 0.2.7.3-rc. + + o Major bugfixes (memory leaks): + - Fix a memory leak in ed25519 batch signature checking. Fixes bug + 17398; bugfix on 0.2.6.1-alpha. + - Fix a memory leak in rend_cache_failure_entry_free(). Fixes bug + 17402; bugfix on 0.2.7.3-rc. + - Fix a memory leak when reading an expired signing key from disk. + Fixes bug 17403; bugfix on 0.2.7.2-rc. + + o Minor features (geoIP): + - Update geoip and geoip6 to the October 9 2015 Maxmind GeoLite2 + Country database. + + o Minor bugfixes (compilation): + - Repair compilation with the most recent (unreleased, alpha) + vesions of OpenSSL 1.1. Fixes part of ticket 17237. + - Fix an integer overflow warning in test_crypto_slow.c. Fixes bug + 17251; bugfix on 0.2.7.2-alpha. + - Fix compilation of sandbox.c with musl-libc. Fixes bug 17347; + bugfix on 0.2.5.1-alpha. Patch from 'jamestk'. + + o Minor bugfixes (portability): + - Use libexecinfo on FreeBSD to enable backtrace support. Fixes + part of bug 17151; bugfix on 0.2.5.2-alpha. Patch from + Marcin Cieślak. + + o Minor bugfixes (sandbox): + - Add the "hidserv-stats" filename to our sandbox filter for the + HiddenServiceStatistics option to work properly. Fixes bug 17354; + bugfix on tor-0.2.6.2-alpha. Patch from David Goulet. + + o Minor bugfixes (testing): + - Add unit tests for get_interface_address* failure cases. Fixes bug + 17173; bugfix on 0.2.7.3-rc. Patch by fk/teor. + - Fix breakage when running 'make check' with BSD make. Fixes bug + 17154; bugfix on 0.2.7.3-rc. Patch by Marcin Cieślak. + - Make the get_ifaddrs_* unit tests more tolerant of different + network configurations. (Don't assume every test box has an IPv4 + address, and don't assume every test box has a non-localhost + address.) Fixes bug 17255; bugfix on 0.2.7.3-rc. Patch by "teor". + - Skip backtrace tests when backtrace support is not compiled in. + Fixes part of bug 17151; bugfix on 0.2.7.1-alpha. Patch from + Marcin Cieślak. + + o Documentation: + - Fix capitalization of SOCKS in sample torrc. Closes ticket 15609. + - Note that HiddenServicePorts can take a unix domain socket. Closes + ticket 17364. + + Changes in version 0.2.7.3-rc - 2015-09-25 Tor 0.2.7.3-rc is the first release candidate in the 0.2.7 series. It contains numerous usability fixes for Ed25519 keys, safeguards against @@ -26,4 +26,4 @@ Frequently Asked Questions: To get started working on Tor development: - See the doc/HACKING file. + See the doc/HACKING directory. diff --git a/ReleaseNotes b/ReleaseNotes index 44cda49b2d..1e9f7f2e8a 100644 --- a/ReleaseNotes +++ b/ReleaseNotes @@ -1,7 +1,715 @@ - This document summarizes new features and bugfixes in each stable release of Tor. If you want to see more detailed descriptions of the changes in each development snapshot, see the ChangeLog file. + +Changes in version 0.2.7.6 - 2015-12-10 + Tor version 0.2.7.6 fixes a major bug in entry guard selection, as + well as a minor bug in hidden service reliability. + + o Major bugfixes (guard selection): + - Actually look at the Guard flag when selecting a new directory + guard. When we implemented the directory guard design, we + accidentally started treating all relays as if they have the Guard + flag during guard selection, leading to weaker anonymity and worse + performance. Fixes bug 17772; bugfix on 0.2.4.8-alpha. Discovered + by Mohsen Imani. + + o Minor features (geoip): + - Update geoip and geoip6 to the December 1 2015 Maxmind GeoLite2 + Country database. + + o Minor bugfixes (compilation): + - When checking for net/pfvar.h, include netinet/in.h if possible. + This fixes transparent proxy detection on OpenBSD. Fixes bug + 17551; bugfix on 0.1.2.1-alpha. Patch from "rubiate". + - Fix a compilation warning with Clang 3.6: Do not check the + presence of an address which can never be NULL. Fixes bug 17781. + + o Minor bugfixes (correctness): + - When displaying an IPv6 exit policy, include the mask bits + correctly even when the number is greater than 31. Fixes bug + 16056; bugfix on 0.2.4.7-alpha. Patch from "gturner". + - The wrong list was used when looking up expired intro points in a + rend service object, causing what we think could be reachability + issues for hidden services, and triggering a BUG log. Fixes bug + 16702; bugfix on 0.2.7.2-alpha. + - Fix undefined behavior in the tor_cert_checksig function. Fixes + bug 17722; bugfix on 0.2.7.2-alpha. + + +Changes in version 0.2.7.5 - 2015-11-20 + The Tor 0.2.7 release series is dedicated to the memory of Tor user + and privacy advocate Caspar Bowden (1961-2015). Caspar worked + tirelessly to advocate human rights regardless of national borders, + and oppose the encroachments of mass surveillance. He opposed national + exceptionalism, he brought clarity to legal and policy debates, he + understood and predicted the impact of mass surveillance on the world, + and he laid the groundwork for resisting it. While serving on the Tor + Project's board of directors, he brought us his uncompromising focus + on technical excellence in the service of humankind. Caspar was an + inimitable force for good and a wonderful friend. He was kind, + humorous, generous, gallant, and believed we should protect one + another without exception. We honor him here for his ideals, his + efforts, and his accomplishments. Please honor his memory with works + that would make him proud. + + Tor 0.2.7.5 is the first stable release in the Tor 0.2.7 series. + + The 0.2.7 series adds a more secure identity key type for relays, + improves cryptography performance, resolves several longstanding + hidden-service performance issues, improves controller support for + hidden services, and includes small bugfixes and performance + improvements throughout the program. This release series also includes + more tests than before, and significant simplifications to which parts + of Tor invoke which others. For a full list of changes, see below. + + o New system requirements: + - Tor no longer includes workarounds to support Libevent versions + before 1.3e. Libevent 2.0 or later is recommended. Closes + ticket 15248. + - Tor no longer supports copies of OpenSSL that are missing support + for Elliptic Curve Cryptography. (We began using ECC when + available in 0.2.4.8-alpha, for more safe and efficient key + negotiation.) In particular, support for at least one of P256 or + P224 is now required, with manual configuration needed if only + P224 is available. Resolves ticket 16140. + - Tor no longer supports versions of OpenSSL before 1.0. (If you are + on an operating system that has not upgraded to OpenSSL 1.0 or + later, and you compile Tor from source, you will need to install a + more recent OpenSSL to link Tor against.) These versions of + OpenSSL are still supported by the OpenSSL, but the numerous + cryptographic improvements in later OpenSSL releases makes them a + clear choice. Resolves ticket 16034. + + o Major features (controller): + - Add the ADD_ONION and DEL_ONION commands that allow the creation + and management of hidden services via the controller. Closes + ticket 6411. + - New "GETINFO onions/current" and "GETINFO onions/detached" + commands to get information about hidden services created via the + controller. Part of ticket 6411. + - New HSFETCH command to launch a request for a hidden service + descriptor. Closes ticket 14847. + - New HSPOST command to upload a hidden service descriptor. Closes + ticket 3523. Patch by "DonnchaC". + + o Major features (Ed25519 identity keys, Proposal 220): + - Add support for offline encrypted Ed25519 master keys. To use this + feature on your tor relay, run "tor --keygen" to make a new master + key (or to make a new signing key if you already have a master + key). Closes ticket 13642. + - All relays now maintain a stronger identity key, using the Ed25519 + elliptic curve signature format. This master key is designed so + that it can be kept offline. Relays also generate an online + signing key, and a set of other Ed25519 keys and certificates. + These are all automatically regenerated and rotated as needed. + Implements part of ticket 12498. + - Directory authorities now vote on Ed25519 identity keys along with + RSA1024 keys. Implements part of ticket 12498. + - Directory authorities track which Ed25519 identity keys have been + used with which RSA1024 identity keys, and do not allow them to + vary freely. Implements part of ticket 12498. + - Microdescriptors now include Ed25519 identity keys. Implements + part of ticket 12498. + - Add a --newpass option to allow changing or removing the + passphrase of an encrypted key with tor --keygen. Implements part + of ticket 16769. + - Add a new OfflineMasterKey option to tell Tor never to try loading + or generating a secret Ed25519 identity key. You can use this in + combination with tor --keygen to manage offline and/or encrypted + Ed25519 keys. Implements ticket 16944. + - On receiving a HUP signal, check to see whether the Ed25519 + signing key has changed, and reload it if so. Closes ticket 16790. + - Significant usability improvements for Ed25519 key management. Log + messages are better, and the code can recover from far more + failure conditions. Thanks to "s7r" for reporting and diagnosing + so many of these! + + o Major features (ECC performance): + - Improve the runtime speed of Ed25519 signature verification by + using Ed25519-donna's batch verification support. Implements + ticket 16533. + - Improve the speed of Ed25519 operations and Curve25519 keypair + generation when built targeting 32 bit x86 platforms with SSE2 + available. Implements ticket 16535. + - Improve the runtime speed of Ed25519 operations by using the + public-domain Ed25519-donna by Andrew M. ("floodyberry"). + Implements ticket 16467. + - Improve the runtime speed of the ntor handshake by using an + optimized curve25519 basepoint scalarmult implementation from the + public-domain Ed25519-donna by Andrew M. ("floodyberry"), based on + ideas by Adam Langley. Implements ticket 9663. + + o Major features (Hidden services): + - Hidden services, if using the EntryNodes option, are required to + use more than one EntryNode, in order to avoid a guard discovery + attack. (This would only affect people who had configured hidden + services and manually specified the EntryNodes option with a + single entry-node. The impact was that it would be easy to + remotely identify the guard node used by such a hidden service. + See ticket for more information.) Fixes ticket 14917. + - Add the torrc option HiddenServiceNumIntroductionPoints, to + specify a fixed number of introduction points. Its maximum value + is 10 and default is 3. Using this option can increase a hidden + service's reliability under load, at the cost of making it more + visible that the hidden service is facing extra load. Closes + ticket 4862. + - Remove the adaptive algorithm for choosing the number of + introduction points, which used to change the number of + introduction points (poorly) depending on the number of + connections the HS sees. Closes ticket 4862. + + o Major features (onion key cross-certification): + - Relay descriptors now include signatures of their own identity + keys, made using the TAP and ntor onion keys. These signatures + allow relays to prove ownership of their own onion keys. Because + of this change, microdescriptors will no longer need to include + RSA identity keys. Implements proposal 228; closes ticket 12499. + + o Major bugfixes (client-side privacy, also in 0.2.6.9): + - Properly separate out each SOCKSPort when applying stream + isolation. The error occurred because each port's session group + was being overwritten by a default value when the listener + connection was initialized. Fixes bug 16247; bugfix on + 0.2.6.3-alpha. Patch by "jojelino". + + o Major bugfixes (hidden service clients, stability, also in 0.2.6.10): + - Stop refusing to store updated hidden service descriptors on a + client. This reverts commit 9407040c59218 (which indeed fixed bug + 14219, but introduced a major hidden service reachability + regression detailed in bug 16381). This is a temporary fix since + we can live with the minor issue in bug 14219 (it just results in + some load on the network) but the regression of 16381 is too much + of a setback. First-round fix for bug 16381; bugfix + on 0.2.6.3-alpha. + + o Major bugfixes (hidden services): + - Revert commit that made directory authorities assign the HSDir + flag to relay without a DirPort; this was bad because such relays + can't handle BEGIN_DIR cells. Fixes bug 15850; bugfix + on tor-0.2.6.3-alpha. + - When cannibalizing a circuit for an introduction point, always + extend to the chosen exit node (creating a 4 hop circuit). + Previously Tor would use the current circuit exit node, which + changed the original choice of introduction point, and could cause + the hidden service to skip excluded introduction points or + reconnect to a skipped introduction point. Fixes bug 16260; bugfix + on 0.1.0.1-rc. + + o Major bugfixes (memory leaks): + - Fix a memory leak in ed25519 batch signature checking. Fixes bug + 17398; bugfix on 0.2.6.1-alpha. + + o Major bugfixes (open file limit): + - The open file limit wasn't checked before calling + tor_accept_socket_nonblocking(), which would make Tor exceed the + limit. Now, before opening a new socket, Tor validates the open + file limit just before, and if the max has been reached, return an + error. Fixes bug 16288; bugfix on 0.1.1.1-alpha. + + o Major bugfixes (security, correctness): + - Fix an error that could cause us to read 4 bytes before the + beginning of an openssl string. This bug could be used to cause + Tor to crash on systems with unusual malloc implementations, or + systems with unusual hardening installed. Fixes bug 17404; bugfix + on 0.2.3.6-alpha. + + o Major bugfixes (stability, also in 0.2.6.10): + - Stop crashing with an assertion failure when parsing certain kinds + of malformed or truncated microdescriptors. Fixes bug 16400; + bugfix on 0.2.6.1-alpha. Found by "torkeln"; fix based on a patch + by "cypherpunks_backup". + - Stop random client-side assertion failures that could occur when + connecting to a busy hidden service, or connecting to a hidden + service while a NEWNYM is in progress. Fixes bug 16013; bugfix + on 0.1.0.1-rc. + + o Minor features (client, SOCKS): + - Add GroupWritable and WorldWritable options to unix-socket based + SocksPort and ControlPort options. These options apply to a single + socket, and override {Control,Socks}SocketsGroupWritable. Closes + ticket 15220. + - Relax the validation done to hostnames in SOCKS5 requests, and + allow a single trailing '.' to cope with clients that pass FQDNs + using that syntax to explicitly indicate that the domain name is + fully-qualified. Fixes bug 16674; bugfix on 0.2.6.2-alpha. + - Relax the validation of hostnames in SOCKS5 requests, allowing the + character '_' to appear, in order to cope with domains observed in + the wild that are serving non-RFC compliant records. Resolves + ticket 16430. + + o Minor features (client-side privacy): + - New KeepAliveIsolateSOCKSAuth option to indefinitely extend circuit + lifespan when IsolateSOCKSAuth and streams with SOCKS + authentication are attached to the circuit. This allows + applications like TorBrowser to manage circuit lifetime on their + own. Implements feature 15482. + - When logging malformed hostnames from SOCKS5 requests, respect + SafeLogging configuration. Fixes bug 16891; bugfix on 0.1.1.16-rc. + + o Minor features (clock-jump tolerance): + - Recover better when our clock jumps back many hours, like might + happen for Tails or Whonix users who start with a very wrong + hardware clock, use Tor to discover a more accurate time, and then + fix their clock. Resolves part of ticket 8766. + + o Minor features (command-line interface): + - Make --hash-password imply --hush to prevent unnecessary noise. + Closes ticket 15542. Patch from "cypherpunks". + - Print a warning whenever we find a relative file path being used + as torrc option. Resolves issue 14018. + + o Minor features (compilation): + - Give a warning as early as possible when trying to build with an + unsupported OpenSSL version. Closes ticket 16901. + - Use C99 variadic macros when the compiler is not GCC. This avoids + failing compilations on MSVC, and fixes a log-file-based race + condition in our old workarounds. Original patch from Gisle Vanem. + + o Minor features (control protocol): + - Support network-liveness GETINFO key and NETWORK_LIVENESS event in + the control protocol. Resolves ticket 15358. + + o Minor features (controller): + - Add DirAuthority lines for default directory authorities to the + output of the "GETINFO config/defaults" command if not already + present. Implements ticket 14840. + - Controllers can now use "GETINFO hs/client/desc/id/..." to + retrieve items from the client's hidden service descriptor cache. + Closes ticket 14845. + - Implement a new controller command "GETINFO status/fresh-relay- + descs" to fetch a descriptor/extrainfo pair that was generated on + demand just for the controller's use. Implements ticket 14784. + + o Minor features (directory authorities): + - Directory authorities no longer vote against the "Fast", "Stable", + and "HSDir" flags just because they were going to vote against + "Running": if the consensus turns out to be that the router was + running, then the authority's vote should count. Patch from Peter + Retzlaff; closes issue 8712. + + o Minor features (directory authorities, security, also in 0.2.6.9): + - The HSDir flag given by authorities now requires the Stable flag. + For the current network, this results in going from 2887 to 2806 + HSDirs. Also, it makes it harder for an attacker to launch a sybil + attack by raising the effort for a relay to become Stable to + require at the very least 7 days, while maintaining the 96 hours + uptime requirement for HSDir. Implements ticket 8243. + + o Minor features (DoS-resistance): + - Make it harder for attackers to overload hidden services with + introductions, by blocking multiple introduction requests on the + same circuit. Resolves ticket 15515. + + o Minor features (geoip): + - Update geoip and geoip6 to the October 9 2015 Maxmind GeoLite2 + Country database. + + o Minor features (hidden services): + - Add the new options "HiddenServiceMaxStreams" and + "HiddenServiceMaxStreamsCloseCircuit" to allow hidden services to + limit the maximum number of simultaneous streams per circuit, and + optionally tear down the circuit when the limit is exceeded. Part + of ticket 16052. + - Client now uses an introduction point failure cache to know when + to fetch or keep a descriptor in their cache. Previously, failures + were recorded implicitly, but not explicitly remembered. Closes + ticket 16389. + - Relays need to have the Fast flag to get the HSDir flag. As this + is being written, we'll go from 2745 HSDirs down to 2342, a ~14% + drop. This change should make some attacks against the hidden + service directory system harder. Fixes ticket 15963. + - Turn on hidden service statistics collection by setting the torrc + option HiddenServiceStatistics to "1" by default. (This keeps + track only of the fraction of traffic used by hidden services, and + the total number of hidden services in existence.) Closes + ticket 15254. + - To avoid leaking HS popularity, don't cycle the introduction point + when we've handled a fixed number of INTRODUCE2 cells but instead + cycle it when a random number of introductions is reached, thus + making it more difficult for an attacker to find out the amount of + clients that have used the introduction point for a specific HS. + Closes ticket 15745. + + o Minor features (logging): + - Include the Tor version in all LD_BUG log messages, since people + tend to cut and paste those into the bugtracker. Implements + ticket 15026. + + o Minor features (pluggable transports): + - When launching managed pluggable transports on Linux systems, + attempt to have the kernel deliver a SIGTERM on tor exit if the + pluggable transport process is still running. Resolves + ticket 15471. + - When launching managed pluggable transports, setup a valid open + stdin in the child process that can be used to detect if tor has + terminated. The "TOR_PT_EXIT_ON_STDIN_CLOSE" environment variable + can be used by implementations to detect this new behavior. + Resolves ticket 15435. + + o Minor bugfixes (torrc exit policies): + - In each instance above, usage advice is provided to avoid the + message. Resolves ticket 16069. Patch by "teor". Fixes part of bug + 16069; bugfix on 0.2.4.7-alpha. + - In torrc, "accept6 *" and "reject6 *" ExitPolicy lines now only + produce IPv6 wildcard addresses. Previously they would produce + both IPv4 and IPv6 wildcard addresses. Patch by "teor". Fixes part + of bug 16069; bugfix on 0.2.4.7-alpha. + - When parsing torrc ExitPolicies, we now issue an info-level + message when expanding an "accept/reject *" line to include both + IPv4 and IPv6 wildcard addresses. Related to ticket 16069. + - When parsing torrc ExitPolicies, we now warn for a number of cases + where the user's intent is likely to differ from Tor's actual + behavior. These include: using an IPv4 address with an accept6 or + reject6 line; using "private" on an accept6 or reject6 line; and + including any ExitPolicy lines after accept *:* or reject *:*. + Related to ticket 16069. + + o Minor bugfixes (command-line interface): + - When "--quiet" is provided along with "--validate-config", do not + write anything to stdout on success. Fixes bug 14994; bugfix + on 0.2.3.3-alpha. + - When complaining about bad arguments to "--dump-config", use + stderr, not stdout. + - Print usage information for --dump-config when it is used without + an argument. Also, fix the error message to use different wording + and add newline at the end. Fixes bug 15541; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (compilation): + - Fix compilation of sandbox.c with musl-libc. Fixes bug 17347; + bugfix on 0.2.5.1-alpha. Patch from 'jamestk'. + - Repair compilation with the most recent (unreleased, alpha) + vesions of OpenSSL 1.1. Fixes part of ticket 17237. + + o Minor bugfixes (compilation, also in 0.2.6.9): + - Build with --enable-systemd correctly when libsystemd is + installed, but systemd is not. Fixes bug 16164; bugfix on + 0.2.6.3-alpha. Patch from Peter Palfrader. + + o Minor bugfixes (configuration, unit tests): + - Only add the default fallback directories when the DirAuthorities, + AlternateDirAuthority, and FallbackDir directory config options + are set to their defaults. The default fallback directory list is + currently empty, this fix will only change tor's behavior when it + has default fallback directories. Includes unit tests for + consider_adding_dir_servers(). Fixes bug 15642; bugfix on + 90f6071d8dc0 in 0.2.4.7-alpha. Patch by "teor". + + o Minor bugfixes (controller): + - Add the descriptor ID in each HS_DESC control event. It was + missing, but specified in control-spec.txt. Fixes bug 15881; + bugfix on 0.2.5.2-alpha. + + o Minor bugfixes (correctness): + - For correctness, avoid modifying a constant string in + handle_control_postdescriptor. Fixes bug 15546; bugfix + on 0.1.1.16-rc. + - Remove side-effects from tor_assert() calls. This was harmless, + because we never disable assertions, but it is bad style and + unnecessary. Fixes bug 15211; bugfix on 0.2.5.5, 0.2.2.36, + and 0.2.0.10. + - When calling channel_free_list(), avoid calling smartlist_remove() + while inside a FOREACH loop. This partially reverts commit + 17356fe7fd96af where the correct SMARTLIST_DEL_CURRENT was + incorrectly removed. Fixes bug 16924; bugfix on 0.2.4.4-alpha. + + o Minor bugfixes (crypto error-handling, also in 0.2.6.10): + - Check for failures from crypto_early_init, and refuse to continue. + A previous typo meant that we could keep going with an + uninitialized crypto library, and would have OpenSSL initialize + its own PRNG. Fixes bug 16360; bugfix on 0.2.5.2-alpha, introduced + when implementing ticket 4900. Patch by "teor". + + o Minor bugfixes (hidden service): + - Fix an out-of-bounds read when parsing invalid INTRODUCE2 cells on + a client authorized hidden service. Fixes bug 15823; bugfix + on 0.2.1.6-alpha. + - Remove an extraneous newline character from the end of hidden + service descriptors. Fixes bug 15296; bugfix on 0.2.0.10-alpha. + + o Minor bugfixes (Linux seccomp2 sandbox): + - Use the sandbox in tor_open_cloexec whether or not O_CLOEXEC is + defined. Patch by "teor". Fixes bug 16515; bugfix on 0.2.3.1-alpha. + - Allow bridge authorities to run correctly under the seccomp2 + sandbox. Fixes bug 16964; bugfix on 0.2.5.1-alpha. + - Add the "hidserv-stats" filename to our sandbox filter for the + HiddenServiceStatistics option to work properly. Fixes bug 17354; + bugfix on tor-0.2.6.2-alpha. Patch from David Goulet. + + o Minor bugfixes (Linux seccomp2 sandbox, also in 0.2.6.10): + - Allow pipe() and pipe2() syscalls in the seccomp2 sandbox: we need + these when eventfd2() support is missing. Fixes bug 16363; bugfix + on 0.2.6.3-alpha. Patch from "teor". + + o Minor bugfixes (Linux seccomp2 sandbox, also in 0.2.6.9): + - Allow systemd connections to work with the Linux seccomp2 sandbox + code. Fixes bug 16212; bugfix on 0.2.6.2-alpha. Patch by + Peter Palfrader. + - Fix sandboxing to work when running as a relay, by allowing the + renaming of secret_id_key, and allowing the eventfd2 and futex + syscalls. Fixes bug 16244; bugfix on 0.2.6.1-alpha. Patch by + Peter Palfrader. + + o Minor bugfixes (logging): + - When building Tor under Clang, do not include an extra set of + parentheses in log messages that include function names. Fixes bug + 15269; bugfix on every released version of Tor when compiled with + recent enough Clang. + + o Minor bugfixes (network): + - When attempting to use fallback technique for network interface + lookup, disregard loopback and multicast addresses since they are + unsuitable for public communications. + + o Minor bugfixes (open file limit): + - Fix set_max_file_descriptors() to set by default the max open file + limit to the current limit when setrlimit() fails. Fixes bug + 16274; bugfix on tor- 0.2.0.10-alpha. Patch by dgoulet. + + o Minor bugfixes (portability): + - Check correctly for Windows socket errors in the workqueue + backend. Fixes bug 16741; bugfix on 0.2.6.3-alpha. + - Try harder to normalize the exit status of the Tor process to the + standard-provided range. Fixes bug 16975; bugfix on every version + of Tor ever. + - Use libexecinfo on FreeBSD to enable backtrace support. Fixes part + of bug 17151; bugfix on 0.2.5.2-alpha. Patch from Marcin Cieślak. + + o Minor bugfixes (relay): + - Ensure that worker threads actually exit when a fatal error or + shutdown is indicated. This fix doesn't currently affect the + behavior of Tor, because Tor workers never indicates fatal error + or shutdown except in the unit tests. Fixes bug 16868; bugfix + on 0.2.6.3-alpha. + - Fix a rarely-encountered memory leak when failing to initialize + the thread pool. Fixes bug 16631; bugfix on 0.2.6.3-alpha. Patch + from "cypherpunks". + - Unblock threads before releasing the work queue mutex to ensure + predictable scheduling behavior. Fixes bug 16644; bugfix + on 0.2.6.3-alpha. + + o Minor bugfixes (security, exit policies): + - ExitPolicyRejectPrivate now also rejects the relay's published + IPv6 address (if any), and any publicly routable IPv4 or IPv6 + addresses on any local interfaces. ticket 17027. Patch by "teor". + Fixes bug 17027; bugfix on 0.2.0.11-alpha. + + o Minor bugfixes (statistics): + - Disregard the ConnDirectionStatistics torrc options when Tor is + not a relay since in that mode of operation no sensible data is + being collected and because Tor might run into measurement hiccups + when running as a client for some time, then becoming a relay. + Fixes bug 15604; bugfix on 0.2.2.35. + + o Minor bugfixes (systemd): + - Tor's systemd unit file no longer contains extraneous spaces. + These spaces would sometimes confuse tools like deb-systemd- + helper. Fixes bug 16162; bugfix on 0.2.5.5-alpha. + + o Minor bugfixes (test networks): + - When self-testing reachability, use ExtendAllowPrivateAddresses to + determine if local/private addresses imply reachability. The + previous fix used TestingTorNetwork, which implies + ExtendAllowPrivateAddresses, but this excluded rare configurations + where ExtendAllowPrivateAddresses is set but TestingTorNetwork is + not. Fixes bug 15771; bugfix on 0.2.6.1-alpha. Patch by "teor", + issue discovered by CJ Ess. + + o Minor bugfixes (tests, also in 0.2.6.9): + - Fix a crash in the unit tests when built with MSVC2013. Fixes bug + 16030; bugfix on 0.2.6.2-alpha. Patch from "NewEraCracker". + + o Code simplification and refactoring: + - Change the function that's called when we need to retry all + downloads so that it only reschedules the downloads to happen + immediately, rather than launching them all at once itself. This + further simplifies Tor's callgraph. + - Define WINVER and _WIN32_WINNT centrally, in orconfig.h, in order + to ensure they remain consistent and visible everywhere. + - Move some format-parsing functions out of crypto.c and + crypto_curve25519.c into crypto_format.c and/or util_format.c. + - Move the client-only parts of init_keys() into a separate + function. Closes ticket 16763. + - Move the hacky fallback code out of get_interface_address6() into + separate function and get it covered with unit-tests. Resolves + ticket 14710. + - Refactor hidden service client-side cache lookup to intelligently + report its various failure cases, and disentangle failure cases + involving a lack of introduction points. Closes ticket 14391. + - Remove some vestigial workarounds for the MSVC6 compiler. We + haven't supported that in ages. + - Remove the unused "nulterminate" argument from buf_pullup(). + - Simplify the microdesc_free() implementation so that it no longer + appears (to code analysis tools) to potentially invoke a huge + suite of other microdesc functions. + - Simply the control graph further by deferring the inner body of + directory_all_unreachable() into a callback. Closes ticket 16762. + - The link authentication code has been refactored for better + testability and reliability. It now uses code generated with the + "trunnel" binary encoding generator, to reduce the risk of bugs + due to programmer error. Done as part of ticket 12498. + - Treat the loss of an owning controller as equivalent to a SIGTERM + signal. This removes a tiny amount of duplicated code, and + simplifies our callgraph. Closes ticket 16788. + - Use our own Base64 encoder instead of OpenSSL's, to allow more + control over the output. Part of ticket 15652. + - When generating an event to send to the controller, we no longer + put the event over the network immediately. Instead, we queue + these events, and use a Libevent callback to deliver them. This + change simplifies Tor's callgraph by reducing the number of + functions from which all other Tor functions are reachable. Closes + ticket 16695. + - Wrap Windows-only C files inside '#ifdef _WIN32' so that tools + that try to scan or compile every file on Unix won't decide that + they are broken. + + o Documentation: + - Fix capitalization of SOCKS in sample torrc. Closes ticket 15609. + - Improve the descriptions of statistics-related torrc options in + the manpage to describe rationale and possible uses cases. Fixes + issue 15550. + - Improve the layout and formatting of ./configure --help messages. + Closes ticket 15024. Patch from "cypherpunks". + - Include a specific and (hopefully) accurate documentation of the + torrc file's meta-format in doc/torrc_format.txt. This is mainly + of interest to people writing programs to parse or generate torrc + files. This document is not a commitment to long-term + compatibility; some aspects of the current format are a bit + ridiculous. Closes ticket 2325. + - Include the TUNING document in our source tarball. It is referred + to in the ChangeLog and an error message. Fixes bug 16929; bugfix + on 0.2.6.1-alpha. + - Note that HiddenServicePorts can take a unix domain socket. Closes + ticket 17364. + - Recommend a 40 GB example AccountingMax in torrc.sample rather + than a 4 GB max. Closes ticket 16742. + - Standardize on the term "server descriptor" in the manual page. + Previously, we had used "router descriptor", "server descriptor", + and "relay descriptor" interchangeably. Part of ticket 14987. + - Advise users on how to configure separate IPv4 and IPv6 exit + policies in the manpage and sample torrcs. Related to ticket 16069. + - Fix an error in the manual page and comments for + TestingDirAuthVoteHSDir[IsStrict], which suggested that a HSDir + required "ORPort connectivity". While this is true, it is in no + way unique to the HSDir flag. Of all the flags, only HSDirs need a + DirPort configured in order for the authorities to assign that + particular flag. Patch by "teor". Fixed as part of 14882; bugfix + on 0.2.6.3-alpha. + - Fix the usage message of tor-resolve(1) so that it no longer lists + the removed -F option. Fixes bug 16913; bugfix on 0.2.2.28-beta. + + o Removed code: + - Remove `USE_OPENSSL_BASE64` and the corresponding fallback code + and always use the internal Base64 decoder. The internal decoder + has been part of tor since tor-0.2.0.10-alpha, and no one should + be using the OpenSSL one. Part of ticket 15652. + - Remove the 'tor_strclear()' function; use memwipe() instead. + Closes ticket 14922. + - Remove the code that would try to aggressively flush controller + connections while writing to them. This code was introduced in + 0.1.2.7-alpha, in order to keep output buffers from exceeding + their limits. But there is no longer a maximum output buffer size, + and flushing data in this way caused some undesirable recursions + in our call graph. Closes ticket 16480. + - The internal pure-C tor-fw-helper tool is now removed from the Tor + distribution, in favor of the pure-Go clone available from + https://gitweb.torproject.org/tor-fw-helper.git/ . The libraries + used by the C tor-fw-helper are not, in our opinion, very + confidence- inspiring in their secure-programming techniques. + Closes ticket 13338. + + o Removed features: + - Remove the (seldom-used) DynamicDHGroups feature. For anti- + fingerprinting we now recommend pluggable transports; for forward- + secrecy in TLS, we now use the P-256 group. Closes ticket 13736. + - Remove the HidServDirectoryV2 option. Now all relays offer to + store hidden service descriptors. Related to 16543. + - Remove the VoteOnHidServDirectoriesV2 option, since all + authorities have long set it to 1. Closes ticket 16543. + - Remove the undocumented "--digests" command-line option. It + complicated our build process, caused subtle build issues on + multiple platforms, and is now redundant since we started + including git version identifiers. Closes ticket 14742. + - Tor no longer contains checks for ancient directory cache versions + that didn't know about microdescriptors. + - Tor no longer contains workarounds for stat files generated by + super-old versions of Tor that didn't choose guards sensibly. + + o Testing: + - The test-network.sh script now supports performance testing. + Requires corresponding chutney performance testing changes. Patch + by "teor". Closes ticket 14175. + - Add a new set of callgraph analysis scripts that use clang to + produce a list of which Tor functions are reachable from which + other Tor functions. We're planning to use these to help simplify + our code structure by identifying illogical dependencies. + - Add new 'test-full' and 'test-full-online' targets to run all + tests, including integration tests with stem and chutney. + - Autodetect CHUTNEY_PATH if the chutney and Tor sources are side- + by-side in the same parent directory. Closes ticket 16903. Patch + by "teor". + - Document use of coverity, clang static analyzer, and clang dynamic + undefined behavior and address sanitizers in doc/HACKING. Include + detailed usage instructions in the blacklist. Patch by "teor". + Closes ticket 15817. + - Make "bridges+hs" the default test network. This tests almost all + tor functionality during make test-network, while allowing tests + to succeed on non-IPv6 systems. Requires chutney commit 396da92 in + test-network-bridges-hs. Closes tickets 16945 (tor) and 16946 + (chutney). Patches by "teor". + - Make the test-workqueue test work on Windows by initializing the + network before we begin. + - New make target (make test-network-all) to run multiple applicable + chutney test cases. Patch from Teor; closes 16953. + - Now that OpenSSL has its own scrypt implementation, add an unit + test that checks for interoperability between libscrypt_scrypt() + and OpenSSL's EVP_PBE_scrypt() so that we could not use libscrypt + and rely on EVP_PBE_scrypt() whenever possible. Resolves + ticket 16189. + - The link authentication protocol code now has extensive tests. + - The relay descriptor signature testing code now has + extensive tests. + - The test_workqueue program now runs faster, and is enabled by + default as a part of "make check". + - Unit test dns_resolve(), dns_clip_ttl() and dns_get_expiry_ttl() + functions in dns.c. Implements a portion of ticket 16831. + - Use environment variables rather than autoconf substitutions to + send variables from the build system to the test scripts. This + change should be easier to maintain, and cause 'make distcheck' to + work better than before. Fixes bug 17148. + - When building Tor with testing coverage enabled, run Chutney tests + (if any) using the 'tor-cov' coverage binary. + - When running test-network or test-stem, check for the absence of + stem/chutney before doing any build operations. + - Add a test to verify that the compiler does not eliminate our + memwipe() implementation. Closes ticket 15377. + - Add make rule `check-changes` to verify the format of changes + files. Closes ticket 15180. + - Add unit tests for control_event_is_interesting(). Add a compile- + time check that the number of events doesn't exceed the capacity + of control_event_t.event_mask. Closes ticket 15431, checks for + bugs similar to 13085. Patch by "teor". + - Command-line argument tests moved to Stem. Resolves ticket 14806. + - Integrate the ntor, backtrace, and zero-length keys tests into the + automake test suite. Closes ticket 15344. + - Remove assertions during builds to determine Tor's test coverage. + We don't want to trigger these even in assertions, so including + them artificially makes our branch coverage look worse than it is. + This patch provides the new test-stem-full and coverage-html-full + configure options. Implements ticket 15400. + - New TestingDirAuthVote{Exit,Guard,HSDir}IsStrict flags to + explicitly manage consensus flags in testing networks. Patch by + "robgjansen", modified by "teor". Implements part of ticket 14882. + - Check for matching value in server response in ntor_ref.py. Fixes + bug 15591; bugfix on 0.2.4.8-alpha. Reported and fixed + by "joelanders". + - Set the severity correctly when testing + get_interface_addresses_ifaddrs() and + get_interface_addresses_win32(), so that the tests fail gracefully + instead of triggering an assertion. Fixes bug 15759; bugfix on + 0.2.6.3-alpha. Reported by Nicolas Derive. + Changes in version 0.2.6.10 - 2015-07-12 Tor version 0.2.6.10 fixes some significant stability and hidden service client bugs, bulletproofs the cryptography init process, and diff --git a/changes/11150 b/changes/11150 new file mode 100644 index 0000000000..b4d40ed07c --- /dev/null +++ b/changes/11150 @@ -0,0 +1,6 @@ + o Removed features: + - Remove client-side support for connecting to Tor servers running + versions of Tor before 0.2.3.6-alpha. These servers didn't + support the v3 TLS handshake protocol, and are no longer allowed + on the Tor network. Implements the client side of ticket + 11150. Based on patches by Tom van der Woerdt. diff --git a/changes/17004 b/changes/17004 new file mode 100644 index 0000000000..1dc9a237d4 --- /dev/null +++ b/changes/17004 @@ -0,0 +1,3 @@ + o Testing: + - Unit tests for directory_handle_command_get. Closes ticket 17004. + Patch from Reinaldo de Souza Jr. diff --git a/changes/17075 b/changes/17075 new file mode 100644 index 0000000000..a91ac673e6 --- /dev/null +++ b/changes/17075 @@ -0,0 +1,3 @@ + o Testing: + - More unit tests for compat_libevent.c. Closes ticket 17075. + Patch from Ola Bini. diff --git a/changes/17078 b/changes/17078 new file mode 100644 index 0000000000..af02877898 --- /dev/null +++ b/changes/17078 @@ -0,0 +1,3 @@ + o Testing: + - More unit tests for procmon.c. Closes ticket 17078. + Patch from Ola Bini. diff --git a/changes/17082 b/changes/17082 new file mode 100644 index 0000000000..30ed01473e --- /dev/null +++ b/changes/17082 @@ -0,0 +1,3 @@ + o Testing: + - More unit tests for tortls.c. Closes ticket 17082. + Patch from Ola Bini. diff --git a/changes/17084 b/changes/17084 new file mode 100644 index 0000000000..361e26f264 --- /dev/null +++ b/changes/17084 @@ -0,0 +1,3 @@ + o Testing: + - More unit tests for util_format.c. Closes ticket 17084. + Patch from Ola Bini. diff --git a/changes/bug15609 b/changes/bug15609 deleted file mode 100644 index efaccdeaae..0000000000 --- a/changes/bug15609 +++ /dev/null @@ -1,2 +0,0 @@ - o Documentation: - - Fix capitalization of SOCKS in sample torrc. Closes ticket 15609. diff --git a/changes/bug16056 b/changes/bug16056 deleted file mode 100644 index e3311c0f93..0000000000 --- a/changes/bug16056 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (relay, IPv6): - - When displaying an IPv6 exit policy, include the mask bits correctly - even when the number is greater than 31. Fixes bug 16056; bugfix on - 0.2.4.7-alpha. Patch from "gturner".
\ No newline at end of file diff --git a/changes/bug16382 b/changes/bug16382 new file mode 100644 index 0000000000..8faee98ad8 --- /dev/null +++ b/changes/bug16382 @@ -0,0 +1,3 @@ + o Documentation: + - Explain actual minima for BandwidthRate. Closes ticket 16382. + diff --git a/changes/bug16563 b/changes/bug16563 new file mode 100644 index 0000000000..19e59b3821 --- /dev/null +++ b/changes/bug16563 @@ -0,0 +1,6 @@ + o Minor bugfixes (logging): + - In log messages that include a function name, use __FUNCTION__ instead + of __PRETTY_FUNCTION__. In GCC, these are synonymous, but with clang + __PRETTY_FUNCTION__ has extra information we don't need. + Fixes bug 16563; bugfix on 0.0.2pre8. Fix by Tom van der Woerdt. +
\ No newline at end of file diff --git a/changes/bug16651 b/changes/bug16651 new file mode 100644 index 0000000000..096daeaf70 --- /dev/null +++ b/changes/bug16651 @@ -0,0 +1,5 @@ + o Minor bugfixes (compilation): + + - Fix search for libevent libraries on OpenBSD (and similar systems + which install libevent 1 and libevent 2 in parallel). Resolves + ticket 16651. Patch from "rubiate". diff --git a/changes/bug16702 b/changes/bug16702 deleted file mode 100644 index 5de36cd351..0000000000 --- a/changes/bug16702 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (hidden service) - - The wrong list was used when looking up expired intro points in a rend - service object causing what we think could be reachability issues and - triggering a BUG log. Fixes 16702; bugfix on tor-0.2.7.2-alpha. diff --git a/changes/bug17026 b/changes/bug17026 new file mode 100644 index 0000000000..8b1ce3c61b --- /dev/null +++ b/changes/bug17026 @@ -0,0 +1,5 @@ + o Minor features: + - Set unused entires in a smartlist to NULL. This helped catch a + (harmless) bug, and shouldn't affect performance too much. + Implements ticket 17026. + diff --git a/changes/bug17027-reject-private-bind-port b/changes/bug17027-reject-private-bind-port new file mode 100644 index 0000000000..abc1431c9a --- /dev/null +++ b/changes/bug17027-reject-private-bind-port @@ -0,0 +1,7 @@ + o Minor bug fixes (security, exit policies): + - ExitPolicyRejectPrivate rejects more private addresses by default: + * the relay's outbound bind addresses (if configured), and + * the relay's configured port addresses (such as ORPort and DirPort). + Resolves ticket 17027. Patch by "teor". + Patch on 42b8fb5a1523 (11 Nov 2007), released in 0.2.0.11-alpha, + and on 0.2.7.3-rc. diff --git a/changes/bug17151 b/changes/bug17151 deleted file mode 100644 index 0993b90eac..0000000000 --- a/changes/bug17151 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (portability): - - Use libexecinfo on FreeBSD, to enable backtrace support. Fixes part of - bug 17151; bugfix on 0.2.5.2-alpha. Patch from Marcin Cieślak. - - o Minor bugfixes (testing): - - Skip backtrace tests when backtrace support is not compiled in. Fixes - part of bug 17151; bugfix on 0.2.7.1-alpha. Patch from Marcin Cieślak. diff --git a/changes/bug17154 b/changes/bug17154 deleted file mode 100644 index 6ad7b74468..0000000000 --- a/changes/bug17154 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (testing): - - Fix breakage when running 'make check' with BSD make. Fixes bug - 17154; bugfix on 0.2.7.3-rc. Patch by Marcin Cieślak. diff --git a/changes/bug17173-socket-hack-rv b/changes/bug17173-socket-hack-rv deleted file mode 100644 index d5132114b4..0000000000 --- a/changes/bug17173-socket-hack-rv +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bug fixes (addresses, testing): - - Add unit tests for get_interface_address* failure cases. - Fixes bug 17173; bugfix on 0.2.7.3-rc. Patch by fk/teor. diff --git a/changes/bug17194 b/changes/bug17194 new file mode 100644 index 0000000000..26549b307f --- /dev/null +++ b/changes/bug17194 @@ -0,0 +1,7 @@ + o Minor feature: + - When logging to syslog, allow a tag to be added to the syslog + identity ("Tor"), i.e. the string prepended to every log message. + The tag can be configured by setting SyslogIdentityTag and defaults + to none. Setting it to "foo" will cause logs to be tagged as + "Tor-foo". + diff --git a/changes/bug17237_027 b/changes/bug17237_027 deleted file mode 100644 index e5978d0ec8..0000000000 --- a/changes/bug17237_027 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features (compilation): - - Repair compilation with the most recent (unreleased, alpha) - vesions of OpenSSL 1.1. Fixes part of ticket 17237. diff --git a/changes/bug17251 b/changes/bug17251 deleted file mode 100644 index edd7739d2f..0000000000 --- a/changes/bug17251 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (compilation): - - Fix an integer overflow warning in test_crypto_slow.c. - Fixes bug 17251; bugfix on 0.2.7.2-alpha. diff --git a/changes/bug17354 b/changes/bug17354 deleted file mode 100644 index 53da007fbb..0000000000 --- a/changes/bug17354 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (sandbox): - - Add the "hidserv-stats" filename to our sandbox filter for the - HiddenServiceStatistics option to work properly. Fixes bug 17354; - bugfix on tor-0.2.6.2-alpha~54^2~1. Patch from David Goulet. diff --git a/changes/bug17364 b/changes/bug17364 deleted file mode 100644 index dd9ff12784..0000000000 --- a/changes/bug17364 +++ /dev/null @@ -1,3 +0,0 @@ - o Documentation: - - Note that HiddenServicePorts can take a unix domain socket. - Closes ticket 17364. diff --git a/changes/bug17398 b/changes/bug17398 deleted file mode 100644 index 66e27a6966..0000000000 --- a/changes/bug17398 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (memory leaks): - - Fix a memory leak in ed25519 batch signature checking. - Fixes bug 17398; bugfix on 0.2.6.1-alpha. diff --git a/changes/bug17401 b/changes/bug17401 deleted file mode 100644 index a22f79c431..0000000000 --- a/changes/bug17401 +++ /dev/null @@ -1,3 +0,0 @@ - o Major bugfixes (correctness): - - Fix a use-after-free bug in validate_intro_point_failure(). - Fixes bug 17401; bugfix on 0.2.7.3-rc. diff --git a/changes/bug17402 b/changes/bug17402 deleted file mode 100644 index 4760e00b04..0000000000 --- a/changes/bug17402 +++ /dev/null @@ -1,3 +0,0 @@ - o Major bugfixes (memory leak): - - Fix a memory leak in rend_cache_failure_entry_free(). - Fixes bug 17402; bugfix on 0.2.7.3-rc. diff --git a/changes/bug17403 b/changes/bug17403 deleted file mode 100644 index e83a4a247b..0000000000 --- a/changes/bug17403 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (memory leaks): - - Fix a memory leak when reading an expired signing key from disk. - Fixes bug 17403; bugfix on 0.2.7.2-rc. diff --git a/changes/bug17404 b/changes/bug17404 deleted file mode 100644 index d524f6662d..0000000000 --- a/changes/bug17404 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes (security, correctness): - - Fix a programming error that could cause us to read 4 bytes before - the beginning of an openssl string. This could be used to provoke - a crash on systems with an unusual malloc implementation, or - systems with unsual hardening installed. Fixes bug 17404; bugfix - on 0.2.3.6-alpha. diff --git a/changes/bug17544 b/changes/bug17544 new file mode 100644 index 0000000000..4316d0709c --- /dev/null +++ b/changes/bug17544 @@ -0,0 +1,4 @@ + o Minor bugfix (SipHash-2-4 performance): + - Improve performance when hashing non-multiple of 8 sized buffers, + based on Andrew Moon's Public Domain SipHash-2-4 implementation. + Fixes bug 17544; bugfix on 0.2.5.3-alpha. diff --git a/changes/bug17549 b/changes/bug17549 new file mode 100644 index 0000000000..3650608141 --- /dev/null +++ b/changes/bug17549 @@ -0,0 +1,3 @@ + o Minor bugfixes (compilation): + - Repair compilation with the most recent (unreleased, alpha) + vesions of OpenSSL 1.1. Fixes bug 17549. diff --git a/changes/bug17551 b/changes/bug17551 deleted file mode 100644 index 27e467979e..0000000000 --- a/changes/bug17551 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (compilation): - - When checking for net/pfvar.h, include netinet/in.h if possible. - This fixes transparent proxy detection on OpenBSD. Fixes bug - 17551; bugfix on 0.1.2.1-alpha. Patch from "rubiate". diff --git a/changes/bug17562-DataDirectoryGroupReadable b/changes/bug17562-DataDirectoryGroupReadable new file mode 100644 index 0000000000..524e5ef28a --- /dev/null +++ b/changes/bug17562-DataDirectoryGroupReadable @@ -0,0 +1,3 @@ + o Minor bug fixes: + - Introduce DataDirectoryGroupReadable boolean. If set to 1, the + DataDirectory will be made readable by the default GID. diff --git a/changes/bug17562-allow-root-group-read b/changes/bug17562-allow-root-group-read new file mode 100644 index 0000000000..7a0903c662 --- /dev/null +++ b/changes/bug17562-allow-root-group-read @@ -0,0 +1,6 @@ + o Minor bug fixes: + - If any directory created by Tor is marked as group readable, the + filesystem group is allowed to be either the default GID or the root + user. Allowing root to read the DataDirectory prevents the need for + CAP_READ_SEARCH when using systemd's CapabilityBoundingSet, or + dac_read_search when using SELinux. diff --git a/changes/bug17562-defer-unix-socket-creation b/changes/bug17562-defer-unix-socket-creation new file mode 100644 index 0000000000..f1896c044a --- /dev/null +++ b/changes/bug17562-defer-unix-socket-creation @@ -0,0 +1,4 @@ + o Minor bug fixes: + - Defer creation of Unix sockets until after setuid. This avoids needing + CAP_CHOWN and CAP_FOWNER when using systemd's CapabilityBoundingSet, or + chown and fowner when using SELinux. diff --git a/changes/bug17572-fallback-by-digest b/changes/bug17572-fallback-by-digest new file mode 100644 index 0000000000..3fba123360 --- /dev/null +++ b/changes/bug17572-fallback-by-digest @@ -0,0 +1,5 @@ + o Minor bugfix (fallback directories): + - Mark fallbacks as "too busy" when they return a 503 response, + rather than just marking authorities. + Fixes bug 17572; bugfix on 5c51b3f1f0d4 released in 0.2.4.7-alpha. + Patch by "teor". diff --git a/changes/bug17589 b/changes/bug17589 new file mode 100644 index 0000000000..91103276df --- /dev/null +++ b/changes/bug17589 @@ -0,0 +1,7 @@ + o Code simplificiation and refactoring: + - When a direct directory request fails immediately on launch, + instead of relaunching that request from inside the code that + launches it, instead mark the connection for teardown. This + change simplifies Tor's callback and prevents the directory- + request launching code from invoking itself recursively. + Closes ticket 17589.
\ No newline at end of file diff --git a/changes/bug17632-no-ipv4-no-localhost b/changes/bug17632-no-ipv4-no-localhost new file mode 100644 index 0000000000..04622079d3 --- /dev/null +++ b/changes/bug17632-no-ipv4-no-localhost @@ -0,0 +1,7 @@ + o Minor bugfix (unit tests): + - Make unit tests pass on IPv6-only systems, and systems without + localhost addresses (like some FreeBSD jails). + Fixes bug #17632; bugfix on unit tests in 0.2.7.3-rc. + c464a367728d was a partial fix for this issue in #17255; + it was released in unit tests in 0.2.7.4-rc. + Patch by "teor". diff --git a/changes/bug17638-ipv6-ersatz-socketpair b/changes/bug17638-ipv6-ersatz-socketpair new file mode 100644 index 0000000000..6193065ff3 --- /dev/null +++ b/changes/bug17638-ipv6-ersatz-socketpair @@ -0,0 +1,5 @@ + o Minor bugfix (IPv6 compatibility, unit tests): + - Make tor_ersatz_socketpair work on IPv6-only systems. + Fixes bug #17638; bugfix on a very early tor version, + earlier than 22dba27d8dd5 (23 Nov 2004) / svn:r2943. + Patch by "teor". diff --git a/changes/bug17683 b/changes/bug17683 new file mode 100644 index 0000000000..e9d47513ab --- /dev/null +++ b/changes/bug17683 @@ -0,0 +1,3 @@ + o Minor bugfixes (TLS context): + - Assert when the TLS contexts fail to initialize. Fixes bug 17683; + bugfix on 0.0.6. diff --git a/changes/bug17686 b/changes/bug17686 new file mode 100644 index 0000000000..8fa16c794b --- /dev/null +++ b/changes/bug17686 @@ -0,0 +1,4 @@ + o Minor features: + - Adjust Tor's use of OpenSSL's RNG APIs so that they absolutely, + positively are not allowed to fail. Previously we depended on + internals about OpenSSL behavior. Closes ticket 17686. diff --git a/changes/bug17694_strongest b/changes/bug17694_strongest new file mode 100644 index 0000000000..0a8954a25e --- /dev/null +++ b/changes/bug17694_strongest @@ -0,0 +1,6 @@ + o Minor features (security): + - Never use the system entropy output directly for anything besides + seeding the PRNG. When we want to generate important keys, instead + of using system entropy directly, hash it with the PRNG stream. + This may help resist certain attacks based on broken OS entropy + implementations. Closes part of ticket 17694.
\ No newline at end of file diff --git a/changes/bug17722 b/changes/bug17722 deleted file mode 100644 index 1b18d4af2b..0000000000 --- a/changes/bug17722 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (code correctness) - - Fix undefined behavior in the tor_cert_checksig function. Fixes bug - 17722; bugfix on tor-0.2.7.2-alpha. diff --git a/changes/bug17724 b/changes/bug17724 new file mode 100644 index 0000000000..7ace99eece --- /dev/null +++ b/changes/bug17724 @@ -0,0 +1,4 @@ + o Minor bug fixes (unit tests, hidden services): + - Avoid relying on malloc internals in test_rend_cache_purge. + Closes ticket 17724. Bug fix on ade5005853c1 and 5e9f2384cf0f, + not in any released version of Tor. Patch by "teor". diff --git a/changes/bug17753 b/changes/bug17753 new file mode 100644 index 0000000000..7d227d856c --- /dev/null +++ b/changes/bug17753 @@ -0,0 +1,4 @@ + o Minor bugfixes (code correctness) + - Assert that allocated memory held by the reputation code is freed + according to its internal counters. Fixes bug 17753; bugfix on + tor-0.1.1.1-alpha. diff --git a/changes/bug17763 b/changes/bug17763 new file mode 100644 index 0000000000..d565d13a7d --- /dev/null +++ b/changes/bug17763 @@ -0,0 +1,5 @@ + o Minor bug fixes (exit policies): + - Consistently ignore multicast addresses when automatically + generating reject private exit policies. + Closes ticket 17763. Bug fix on 10a6390deb3c9, + not in any released version of Tor. Patch by "teor". diff --git a/changes/bug17772 b/changes/bug17772 deleted file mode 100644 index 54d457c601..0000000000 --- a/changes/bug17772 +++ /dev/null @@ -1,7 +0,0 @@ - o Major bugfixes (guard selection): - - Actually look at the Guard flag when selecting a new directory - guard. When we implemented the directory guard design, we - accidentally started treating all relays as if they have the Guard - flag during guard selection, leading to weaker anonymity and worse - performance. Fixes bug 17222; bugfix on 0.2.4.8-alpha. Discovered - by Mohsen Imani. diff --git a/changes/bug17776 b/changes/bug17776 new file mode 100644 index 0000000000..a949625baa --- /dev/null +++ b/changes/bug17776 @@ -0,0 +1,6 @@ + o Minor bugfixes (tests): + - Fix buffer over-reads in the directory tests. Fixes bug 17776; not in any + released version of Tor. + - Fix buffer over-reads in the rendcache tests. Fixes bug 17776; not in any + released version of Tor. + diff --git a/changes/bug17778 b/changes/bug17778 new file mode 100644 index 0000000000..9844969a3b --- /dev/null +++ b/changes/bug17778 @@ -0,0 +1,3 @@ + o Minor bugfixes (tests): + - Fix a memory leak in the ntor test. Fixes bug 17778; bugfix on + 0.2.4.8-alpha. diff --git a/changes/bug17781 b/changes/bug17781 deleted file mode 100644 index 01ed231b0a..0000000000 --- a/changes/bug17781 +++ /dev/null @@ -1,3 +0,0 @@ - o Compilation fixes: - - Fix a compilation warning with Clang 3.6: Do not check the - presence of an address which can never be NULL. Fixes bug 17781. diff --git a/changes/bug17791 b/changes/bug17791 new file mode 100644 index 0000000000..f191012cd4 --- /dev/null +++ b/changes/bug17791 @@ -0,0 +1,4 @@ + o Documentation: + - Fix a minor formatting typo in the manpage. Closes ticket + 17791. + diff --git a/changes/bug17804 b/changes/bug17804 new file mode 100644 index 0000000000..bd2a3cbdff --- /dev/null +++ b/changes/bug17804 @@ -0,0 +1,3 @@ + o Minor bugfixes (compilation): + - Replace usage of 'INLINE' with 'inline'. Fixes bug 17804; bugfix + on tor-0.0.2pre8. diff --git a/changes/check-crypto-errors b/changes/check-crypto-errors new file mode 100644 index 0000000000..e41862ca13 --- /dev/null +++ b/changes/check-crypto-errors @@ -0,0 +1,5 @@ + o Minor bugfix (crypto): + - Check the return value of HMAC and assert on failure. + Fixes bug #17658; bugfix on commit in fdbb9cdf746b (11 Oct 2011) + in tor version 0.2.3.5-alpha-dev. + Patch by "teor". diff --git a/changes/cleanup_17587 b/changes/cleanup_17587 new file mode 100644 index 0000000000..05e00fd9e5 --- /dev/null +++ b/changes/cleanup_17587 @@ -0,0 +1,3 @@ + o Code simplifications and refactorings: + - Clean up a little duplicated code in crypto_expand_key_material_TAP. + Closes ticket 17587; patch from "pfrankw". diff --git a/changes/decouple_circuit_mark b/changes/decouple_circuit_mark new file mode 100644 index 0000000000..4b7ed778a2 --- /dev/null +++ b/changes/decouple_circuit_mark @@ -0,0 +1,6 @@ + o Code simplification and refactoring: + - Extract the more complicated parts of circuit_mark_for_close into + a new function run periodically before connections are freed. + This change removes more than half of the functions currently + in the "blob". + Closes ticket #17218. diff --git a/changes/decouple_conn_attach b/changes/decouple_conn_attach new file mode 100644 index 0000000000..6167b4e932 --- /dev/null +++ b/changes/decouple_conn_attach @@ -0,0 +1,6 @@ + o Code simplification and refactorings: + - Decouple the list of streams needing to be attached to circuits + from the overall connection list. This change makes it possible to + attach streams quickly while both simplifying Tor's callgraph and + avoiding O(N) scans of the entire connection list. Closes ticket + 17590. diff --git a/changes/doc17392 b/changes/doc17392 new file mode 100644 index 0000000000..3c93497b00 --- /dev/null +++ b/changes/doc17392 @@ -0,0 +1,4 @@ + o Documentation: + - Mention torspec URL in the manpage and point the reader to it + whenever we mention a document that belongs in torspce. + Fixes issue 17392. diff --git a/changes/feature13696 b/changes/feature13696 new file mode 100644 index 0000000000..21c2188d12 --- /dev/null +++ b/changes/feature13696 @@ -0,0 +1,3 @@ + o Minor features (security, cryptography): + - Use modern system calls to generate strong entropy on platforms that + provide them. Closes ticket 13696. diff --git a/changes/feature14846 b/changes/feature14846 new file mode 100644 index 0000000000..4668761f22 --- /dev/null +++ b/changes/feature14846 @@ -0,0 +1,4 @@ + o Major features (controller): + - New "GETINFO hs/service/desc/id/" command to retrieve a hidden service + descriptor from a service's local hidden service descriptor cache. + Closes ticket 14846. diff --git a/changes/feature17608 b/changes/feature17608 new file mode 100644 index 0000000000..d56bb7d4a7 --- /dev/null +++ b/changes/feature17608 @@ -0,0 +1,4 @@ + o Minor feature (refactoring): + - Move logging of redundant policy entries in + policies_parse_exit_policy_internal into its own function. + Closes ticket 17608; patch from "juce". diff --git a/changes/feature17663 b/changes/feature17663 new file mode 100644 index 0000000000..baad9436fc --- /dev/null +++ b/changes/feature17663 @@ -0,0 +1,3 @@ + o Minor feature (crypto): + - Add SHA512 support to crypto.c. Closes ticket 17663; patch from + George Tankersley. diff --git a/changes/feature8961-replaycache-sha256 b/changes/feature8961-replaycache-sha256 new file mode 100644 index 0000000000..c5b5c857db --- /dev/null +++ b/changes/feature8961-replaycache-sha256 @@ -0,0 +1,4 @@ + o Minor enhancement (replaycache): + - The replay cache now uses SHA256 instead of SHA1. + Implements feature #8961. + Patch by "teor", issue reported by "rransom". diff --git a/changes/first-hop-no-private b/changes/first-hop-no-private new file mode 100644 index 0000000000..e8d0684061 --- /dev/null +++ b/changes/first-hop-no-private @@ -0,0 +1,8 @@ + o Minor bugfix (relays, hidden services): + - Refuse connection requests to private OR addresses unless + ExtendAllowPrivateAddresses is set. Previously, tor would + connect, then refuse to send any cells to a private address. + Fixes bugs 17674 and 8976; bugfix on b7c172c9ec76 (28 Aug 2012) + Original bug 6710, released in 0.2.3.21-rc and an 0.2.2 maint + release. + Patch by "teor". diff --git a/changes/geoip-december2015 b/changes/geoip-december2015 deleted file mode 100644 index 597bcc92f8..0000000000 --- a/changes/geoip-december2015 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features: - - Update geoip and geoip6 to the December 1 2015 Maxmind GeoLite2 - Country database. - diff --git a/changes/geoip-october2015 b/changes/geoip-october2015 deleted file mode 100644 index f20febec5a..0000000000 --- a/changes/geoip-october2015 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update geoip and geoip6 to the October 9 2015 Maxmind GeoLite2 Country database. - diff --git a/changes/getinfo-private-exitpolicy b/changes/getinfo-private-exitpolicy new file mode 100644 index 0000000000..e8345167e9 --- /dev/null +++ b/changes/getinfo-private-exitpolicy @@ -0,0 +1,6 @@ + o Minor features (exit policies, controllers): + - Add controller getinfo exit-policy/reject-private/[default,relay] + for the reject rules added by ExitPolicyRejectPrivate. This makes + it easier for stem to display exit policies. + - Add unit tests for getinfo exit-policy/*. + Completes ticket #17183. Patch by "teor". diff --git a/changes/ifaddrs-tests-network-configs b/changes/ifaddrs-tests-network-configs deleted file mode 100644 index 6b5ed4d484..0000000000 --- a/changes/ifaddrs-tests-network-configs +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (testing): - - Make the get_ifaddrs_* unit tests more tolerant of different network - configurations. (Don't assume every test box has an IPv4 address, and - Don't assume every test box has a non-localhost address. - Fixes bug 17255; bugfix on 0.2.7.3-rc. Patch by "teor". diff --git a/changes/laplace-edge-cases b/changes/laplace-edge-cases new file mode 100644 index 0000000000..6c8c77b040 --- /dev/null +++ b/changes/laplace-edge-cases @@ -0,0 +1,8 @@ + o Code simplifications and unit tests: + - Handle edge cases in the laplace functions: avoid division by zero, + avoid taking the log of zero, and silence clang type conversion + warnings using round and trunc. Add unit tests for edge cases with + maximal values. + - Consistently check for overflow in round_*_to_next_multiple_of + functions, and add unit tests with additional and maximal values. + diff --git a/changes/rand-failure-modes b/changes/rand-failure-modes new file mode 100644 index 0000000000..cc6ef4744e --- /dev/null +++ b/changes/rand-failure-modes @@ -0,0 +1,5 @@ + o Minor features (unit tests, random number generation): + - Add unit tests that check for common RNG failure modes, such as + returning all zeroes, identical values, or incrementing values + (OpenSSL's rand_predictable feature). + Patch by "teor". diff --git a/changes/routerset-parse-IPv6-literals b/changes/routerset-parse-IPv6-literals new file mode 100644 index 0000000000..c80c82c229 --- /dev/null +++ b/changes/routerset-parse-IPv6-literals @@ -0,0 +1,5 @@ + o Minor bug fixes (routersets, IPv6): + - routerset_parse now accepts IPv6 literal addresses. + Fix for ticket 17060. Patch by "teor". + Patch on 3ce6e2fba290 (24 Jul 2008), and related commits, + released in 0.2.1.3-alpha. diff --git a/changes/sha-unit-tests b/changes/sha-unit-tests new file mode 100644 index 0000000000..457578d337 --- /dev/null +++ b/changes/sha-unit-tests @@ -0,0 +1,5 @@ + o Minor bugfixes (unit tests): + - Check the full results of SHA256 and SHA512 digests in the + unit tests. + Bugfix on a tor version before the refactoring in git commit + cea12251995d (23 Sep 2009). Patch by "teor". diff --git a/changes/test16831 b/changes/test16831 new file mode 100644 index 0000000000..7db2d14df5 --- /dev/null +++ b/changes/test16831 @@ -0,0 +1,3 @@ + o Testing: + - Cover dns_resolve_impl() in dns.c with unit tests. Implements a + portion of ticket 16831. diff --git a/changes/warn-when-time-goes-backwards b/changes/warn-when-time-goes-backwards new file mode 100644 index 0000000000..d7e584d9ff --- /dev/null +++ b/changes/warn-when-time-goes-backwards @@ -0,0 +1,5 @@ + o Minor features (security, clock): + - Warn when the system clock is set back in time (when the + state file was last written in the future). Tor doesn't know + that consensuses have expired if the clock is in the past. + Patch by "teor". Implements ticket #17188. diff --git a/configure.ac b/configure.ac index d37c34daae..4dbb3c4fa4 100644 --- a/configure.ac +++ b/configure.ac @@ -3,7 +3,7 @@ dnl Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson dnl Copyright (c) 2007-2015, The Tor Project, Inc. dnl See LICENSE for licensing information -AC_INIT([tor],[0.2.7.6-dev]) +AC_INIT([tor],[0.2.8.0-alpha-dev]) AC_CONFIG_SRCDIR([src/or/main.c]) AC_CONFIG_MACRO_DIR([m4]) AM_INIT_AUTOMAKE @@ -374,10 +374,6 @@ AC_SEARCH_LIBS(pthread_detach, [pthread]) AM_CONDITIONAL(THREADS_WIN32, test "$bwin32" = "true") AM_CONDITIONAL(THREADS_PTHREADS, test "$bwin32" = "false") -dnl ------------------------------------------------------------------- -dnl Check for functions before libevent, since libevent-1.2 apparently -dnl exports strlcpy without defining it in a header. - AC_CHECK_FUNCS( _NSGetEnviron \ accept4 \ @@ -388,6 +384,7 @@ AC_CHECK_FUNCS( flock \ ftime \ getaddrinfo \ + getentropy \ getifaddrs \ getpass \ getrlimit \ @@ -504,10 +501,7 @@ AC_CHECK_MEMBERS([struct event.min_heap_idx], , , AC_CHECK_HEADERS(event2/event.h event2/dns.h event2/bufferevent_ssl.h) -LIBS="$save_LIBS" -LDFLAGS="$save_LDFLAGS" -CPPFLAGS="$save_CPPFLAGS" - +LIBS="$STATIC_LIBEVENT_FLAGS $TOR_LIB_WS32 $save_LIBS" AM_CONDITIONAL(USE_EXTERNAL_EVDNS, test x$ac_cv_header_event2_dns_h = xyes) @@ -518,9 +512,25 @@ if test "$enable_static_libevent" = "yes"; then TOR_LIBEVENT_LIBS="$TOR_LIBDIR_libevent/libevent.a $STATIC_LIBEVENT_FLAGS" fi else - TOR_LIBEVENT_LIBS="-levent" + if test "x$ac_cv_header_event2_event_h" = "xyes"; then + AC_SEARCH_LIBS(event_new, [event event_core], , AC_MSG_ERROR("libevent2 is installed but linking it failed while searching for event_new")) + AC_SEARCH_LIBS(evdns_base_new, [event event_extra], , AC_MSG_ERROR("libevent2 is installed but linking it failed while searching for evdns_base_new")) + + if test "$ac_cv_search_event_new" != "none required"; then + TOR_LIBEVENT_LIBS="$ac_cv_search_event_new" + fi + if test "$ac_cv_search_evdns_base_new" != "none required"; then + TOR_LIBEVENT_LIBS="$ac_cv_search_evdns_base_new $TOR_LIBEVENT_LIBS" + fi + else + TOR_LIBEVENT_LIBS="-levent" + fi fi +LIBS="$save_LIBS" +LDFLAGS="$save_LDFLAGS" +CPPFLAGS="$save_CPPFLAGS" + dnl This isn't the best test for Libevent 2.0.3-alpha. Once it's released, dnl we can do much better. if test "$enable_bufferevents" = "yes" ; then @@ -942,6 +952,7 @@ AC_CHECK_HEADERS( sys/select.h \ sys/socket.h \ sys/statvfs.h \ + sys/syscall.h \ sys/sysctl.h \ sys/syslimits.h \ sys/time.h \ diff --git a/contrib/win32build/tor-mingw.nsi.in b/contrib/win32build/tor-mingw.nsi.in index 08cef8dfbc..2da4210499 100644 --- a/contrib/win32build/tor-mingw.nsi.in +++ b/contrib/win32build/tor-mingw.nsi.in @@ -8,7 +8,7 @@ !include "LogicLib.nsh" !include "FileFunc.nsh" !insertmacro GetParameters -!define VERSION "0.2.7.6-dev" +!define VERSION "0.2.8.0-alpha-dev" !define INSTALLER "tor-${VERSION}-win32.exe" !define WEBSITE "https://www.torproject.org/" !define LICENSE "LICENSE" diff --git a/doc/HACKING b/doc/HACKING deleted file mode 100644 index e92d675a43..0000000000 --- a/doc/HACKING +++ /dev/null @@ -1,659 +0,0 @@ -Hacking Tor: An Incomplete Guide -================================ - -Getting started ---------------- - -For full information on how Tor is supposed to work, look at the files in -https://gitweb.torproject.org/torspec.git/tree - -For an explanation of how to change Tor's design to work differently, look at -https://gitweb.torproject.org/torspec.git/blob_plain/HEAD:/proposals/001-process.txt - -For the latest version of the code, get a copy of git, and - - git clone https://git.torproject.org/git/tor - -We talk about Tor on the tor-talk mailing list. Design proposals and -discussion belong on the tor-dev mailing list. We hang around on -irc.oftc.net, with general discussion happening on #tor and development -happening on #tor-dev. - -How we use Git branches ------------------------ - -Each main development series (like 0.2.1, 0.2.2, etc) has its main work -applied to a single branch. At most one series can be the development series -at a time; all other series are maintenance series that get bug-fixes only. -The development series is built in a git branch called "master"; the -maintenance series are built in branches called "maint-0.2.0", "maint-0.2.1", -and so on. We regularly merge the active maint branches forward. - -For all series except the development series, we also have a "release" branch -(as in "release-0.2.1"). The release series is based on the corresponding -maintenance series, except that it deliberately lags the maint series for -most of its patches, so that bugfix patches are not typically included in a -maintenance release until they've been tested for a while in a development -release. Occasionally, we'll merge an urgent bugfix into the release branch -before it gets merged into maint, but that's rare. - -If you're working on a bugfix for a bug that occurs in a particular version, -base your bugfix branch on the "maint" branch for the first supported series -that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) If -you're working on a new feature, base it on the master branch. - - -How we log changes ------------------- - -When you do a commit that needs a ChangeLog entry, add a new file to -the "changes" toplevel subdirectory. It should have the format of a -one-entry changelog section from the current ChangeLog file, as in - - o Major bugfixes: - - Fix a potential buffer overflow. Fixes bug 99999; bugfix on - 0.3.1.4-beta. - -To write a changes file, first categorize the change. Some common categories -are: Minor bugfixes, Major bugfixes, Minor features, Major features, Code -simplifications and refactoring. Then say what the change does. If -it's a bugfix, mention what bug it fixes and when the bug was -introduced. To find out which Git tag the change was introduced in, -you can use "git describe --contains <sha1 of commit>". - -If at all possible, try to create this file in the same commit where you are -making the change. Please give it a distinctive name that no other branch will -use for the lifetime of your change. To verify the format of the changes file, -you can use "make check-changes". - -When we go to make a release, we will concatenate all the entries -in changes to make a draft changelog, and clear the directory. We'll -then edit the draft changelog into a nice readable format. - -What needs a changes file?:: - A not-exhaustive list: Anything that might change user-visible - behavior. Anything that changes internals, documentation, or the build - system enough that somebody could notice. Big or interesting code - rewrites. Anything about which somebody might plausibly wonder "when - did that happen, and/or why did we do that" 6 months down the line. - -Why use changes files instead of Git commit messages?:: - Git commit messages are written for developers, not users, and they - are nigh-impossible to revise after the fact. - -Why use changes files instead of entries in the ChangeLog?:: - Having every single commit touch the ChangeLog file tended to create - zillions of merge conflicts. - -Useful tools ------------- - -These aren't strictly necessary for hacking on Tor, but they can help track -down bugs. - -Jenkins -~~~~~~~ - -https://jenkins.torproject.org - -Dmalloc -~~~~~~~ - -The dmalloc library will keep track of memory allocation, so you can find out -if we're leaking memory, doing any double-frees, or so on. - - dmalloc -l ~/dmalloc.log - (run the commands it tells you) - ./configure --with-dmalloc - -Valgrind -~~~~~~~~ - -valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor - -(Note that if you get a zillion openssl warnings, you will also need to -pass --undef-value-errors=no to valgrind, or rebuild your openssl -with -DPURIFY.) - -Coverity -~~~~~~~~ - -Nick regularly runs the coverity static analyzer on the Tor codebase. - -The preprocessor define __COVERITY__ is used to work around instances -where coverity picks up behavior that we wish to permit. - -clang Static Analyzer -~~~~~~~~~~~~~~~~~~~~~ - -The clang static analyzer can be run on the Tor codebase using Xcode (WIP) -or a command-line build. - -The preprocessor define __clang_analyzer__ is used to work around instances -where clang picks up behavior that we wish to permit. - -clang Runtime Sanitizers -~~~~~~~~~~~~~~~~ - -To build the Tor codebase with the clang Address and Undefined Behavior -sanitizers, see the file contrib/clang/sanitize_blacklist.txt. - -Preprocessor workarounds for instances where clang picks up behavior that -we wish to permit are also documented in the blacklist file. - -Running lcov for unit test coverage -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Lcov is a utility that generates pretty HTML reports of test code coverage. -To generate such a report: - ------ - ./configure --enable-coverage - make - make coverage-html - $BROWSER ./coverage_html/index.html ------ - -This will run the tor unit test suite `./src/test/test` and generate the HTML -coverage code report under the directory ./coverage_html/. To change the -output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`. - -Coverage diffs using lcov are not currently implemented, but are being -investigated (as of July 2014). - -Running the unit tests -~~~~~~~~~~~~~~~~~~~~~~ - -To quickly run all the tests distributed with Tor: ------ - make check ------ - -To run the fast unit tests only: ------ - make test ------ - -To selectively run just some tests (the following can be combined -arbitrarily): ------ - ./src/test/test <name_of_test> [<name of test 2>] ... - ./src/test/test <prefix_of_name_of_test>.. [<prefix_of_name_of_test2>..] ... - ./src/test/test :<name_of_excluded_test> [:<name_of_excluded_test2]... ------ - -To run all tests, including those based on Stem or Chutney: ------ - make test-full ------ - -To run all tests, including those basedd on Stem or Chutney that require a -working connection to the internet: ------ - make test-full-online ------ - -Running gcov for unit test coverage -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ------ - ./configure --enable-coverage - make - make check - # or--- make test-full ? make test-full-online? - mkdir coverage-output - ./scripts/test/coverage coverage-output ------ - -(On OSX, you'll need to start with "--enable-coverage CC=clang".) - -Then, look at the .gcov files in coverage-output. '-' before a line means -that the compiler generated no code for that line. '######' means that the -line was never reached. Lines with numbers were called that number of times. - -If that doesn't work: - * Try configuring Tor with --disable-gcc-hardening - * You might need to run 'make clean' after you run './configure'. - -If you make changes to Tor and want to get another set of coverage results, -you can run "make reset-gcov" to clear the intermediary gcov output. - -If you have two different "coverage-output" directories, and you want to see -a meaningful diff between them, you can run: - ------ - ./scripts/test/cov-diff coverage-output1 coverage-output2 | less ------ - -In this diff, any lines that were visited at least once will have coverage -"1". This lets you inspect what you (probably) really want to know: which -untested lines were changed? Are there any new untested lines? - -Running integration tests -~~~~~~~~~~~~~~~~~~~~~~~~~ - -We have the beginnings of a set of scripts to run integration tests using -Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and -run "make test-network". - -We also have scripts to run integration tests using Stem. To try them, set -STEM_SOURCE_DIR to your Stem source directory, and run "test-stem". - -Profiling Tor with oprofile -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The oprofile tool runs (on Linux only!) to tell you what functions Tor is -spending its CPU time in, so we can identify berformance pottlenecks. - -Here are some basic instructions - - - Build tor with debugging symbols (you probably already have, unless - you messed with CFLAGS during the build process). - - Build all the libraries you care about with debugging symbols - (probably you only care about libssl, maybe zlib and Libevent). - - Copy this tor to a new directory - - Copy all the libraries it uses to that dir too (ldd ./tor will - tell you) - - Set LD_LIBRARY_PATH to include that dir. ldd ./tor should now - show you it's using the libs in that dir - - Run that tor - - Reset oprofiles counters/start it - * "opcontrol --reset; opcontrol --start", if Nick remembers right. - - After a while, have it dump the stats on tor and all the libs - in that dir you created. - * "opcontrol --dump;" - * "opreport -l that_dir/*" - - Profit - -Generating and analyzing a callgraph -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -1. Run ./scripts/maint/generate_callgraph.sh . This will generate a - bunch of files in a new ./callgraph directory. - -2. Run ./scripts/maint/analyze_callgraph.py callgraph/src/*/* . This - will do a lot of graph operations and then dump out a new - "callgraph.pkl" file, containing data in Python's "pickle" format. - -3. Run ./scripts/maint/display_callgraph.py . It will display: - - the number of functions reachable from each function. - - all strongly-connnected components in the Tor callgraph - - the largest bottlenecks in the largest SCC in the Tor callgraph. - -Note that currently the callgraph generator can't detect calls that pass -through function pointers. - -Coding conventions ------------------- - -Patch checklist -~~~~~~~~~~~~~~~ - -If possible, send your patch as one of these (in descending order of -preference) - - - A git branch we can pull from - - Patches generated by git format-patch - - A unified diff - -Did you remember... - - - To build your code while configured with --enable-gcc-warnings? - - To run "make check-spaces" on your code? - - To run "make check-docs" to see whether all new options are on - the manpage? - - To write unit tests, as possible? - - To base your code on the appropriate branch? - - To include a file in the "changes" directory as appropriate? - -Whitespace and C conformance -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Invoke "make check-spaces" from time to time, so it can tell you about -deviations from our C whitespace style. Generally, we use: - - - Unix-style line endings - - K&R-style indentation - - No space before newlines - - A blank line at the end of each file - - Never more than one blank line in a row - - Always spaces, never tabs - - No more than 79-columns per line. - - Two spaces per indent. - - A space between control keywords and their corresponding paren - "if (x)", "while (x)", and "switch (x)", never "if(x)", "while(x)", or - "switch(x)". - - A space between anything and an open brace. - - No space between a function name and an opening paren. "puts(x)", not - "puts (x)". - - Function declarations at the start of the line. - -We try hard to build without warnings everywhere. In particular, if you're -using gcc, you should invoke the configure script with the option -"--enable-gcc-warnings". This will give a bunch of extra warning flags to -the compiler, and help us find divergences from our preferred C style. - -Getting emacs to edit Tor source properly -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Nick likes to put the following snippet in his .emacs file: - ------ - (add-hook 'c-mode-hook - (lambda () - (font-lock-mode 1) - (set-variable 'show-trailing-whitespace t) - - (let ((fname (expand-file-name (buffer-file-name)))) - (cond - ((string-match "^/home/nickm/src/libevent" fname) - (set-variable 'indent-tabs-mode t) - (set-variable 'c-basic-offset 4) - (set-variable 'tab-width 4)) - ((string-match "^/home/nickm/src/tor" fname) - (set-variable 'indent-tabs-mode nil) - (set-variable 'c-basic-offset 2)) - ((string-match "^/home/nickm/src/openssl" fname) - (set-variable 'indent-tabs-mode t) - (set-variable 'c-basic-offset 8) - (set-variable 'tab-width 8)) - )))) ------ - -You'll note that it defaults to showing all trailing whitespace. The "cond" -test detects whether the file is one of a few C free software projects that I -often edit, and sets up the indentation level and tab preferences to match -what they want. - -If you want to try this out, you'll need to change the filename regex -patterns to match where you keep your Tor files. - -If you use emacs for editing Tor and nothing else, you could always just say: - ------ - (add-hook 'c-mode-hook - (lambda () - (font-lock-mode 1) - (set-variable 'show-trailing-whitespace t) - (set-variable 'indent-tabs-mode nil) - (set-variable 'c-basic-offset 2))) ------ - -There is probably a better way to do this. No, we are probably not going -to clutter the files with emacs stuff. - - -Functions to use -~~~~~~~~~~~~~~~~ - -We have some wrapper functions like tor_malloc, tor_free, tor_strdup, and -tor_gettimeofday; use them instead of their generic equivalents. (They -always succeed or exit.) - -You can get a full list of the compatibility functions that Tor provides by -looking through src/common/util.h and src/common/compat.h. You can see the -available containers in src/common/containers.h. You should probably -familiarize yourself with these modules before you write too much code, or -else you'll wind up reinventing the wheel. - -Use 'INLINE' instead of 'inline', so that we work properly on Windows. - -Calling and naming conventions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Whenever possible, functions should return -1 on error and 0 on success. - -For multi-word identifiers, use lowercase words combined with -underscores. (e.g., "multi_word_identifier"). Use ALL_CAPS for macros and -constants. - -Typenames should end with "_t". - -Function names should be prefixed with a module name or object name. (In -general, code to manipulate an object should be a module with the same name -as the object, so it's hard to tell which convention is used.) - -Functions that do things should have imperative-verb names -(e.g. buffer_clear, buffer_resize); functions that return booleans should -have predicate names (e.g. buffer_is_empty, buffer_needs_resizing). - -If you find that you have four or more possible return code values, it's -probably time to create an enum. If you find that you are passing three or -more flags to a function, it's probably time to create a flags argument that -takes a bitfield. - -What To Optimize -~~~~~~~~~~~~~~~~ - -Don't optimize anything if it's not in the critical path. Right now, the -critical path seems to be AES, logging, and the network itself. Feel free to -do your own profiling to determine otherwise. - -Log conventions -~~~~~~~~~~~~~~~ - -https://www.torproject.org/docs/faq#LogLevel - -No error or warning messages should be expected during normal OR or OP -operation. - -If a library function is currently called such that failure always means ERR, -then the library function should log WARN and let the caller log ERR. - -Every message of severity INFO or higher should either (A) be intelligible -to end-users who don't know the Tor source; or (B) somehow inform the -end-users that they aren't expected to understand the message (perhaps -with a string like "internal error"). Option (A) is to be preferred to -option (B). - -Doxygen -~~~~~~~~ - -We use the 'doxygen' utility to generate documentation from our -source code. Here's how to use it: - - 1. Begin every file that should be documented with - /** - * \file filename.c - * \brief Short description of the file. - **/ - - (Doxygen will recognize any comment beginning with /** as special.) - - 2. Before any function, structure, #define, or variable you want to - document, add a comment of the form: - - /** Describe the function's actions in imperative sentences. - * - * Use blank lines for paragraph breaks - * - and - * - hyphens - * - for - * - lists. - * - * Write <b>argument_names</b> in boldface. - * - * \code - * place_example_code(); - * between_code_and_endcode_commands(); - * \endcode - */ - - 3. Make sure to escape the characters "<", ">", "\", "%" and "#" as "\<", - "\>", "\\", "\%", and "\#". - - 4. To document structure members, you can use two forms: - - struct foo { - /** You can put the comment before an element; */ - int a; - int b; /**< Or use the less-than symbol to put the comment - * after the element. */ - }; - - 5. To generate documentation from the Tor source code, type: - - $ doxygen -g - - To generate a file called 'Doxyfile'. Edit that file and run - 'doxygen' to generate the API documentation. - - 6. See the Doxygen manual for more information; this summary just - scratches the surface. - -Doxygen comment conventions -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Say what functions do as a series of one or more imperative sentences, as -though you were telling somebody how to be the function. In other words, DO -NOT say: - - /** The strtol function parses a number. - * - * nptr -- the string to parse. It can include whitespace. - * endptr -- a string pointer to hold the first thing that is not part - * of the number, if present. - * base -- the numeric base. - * returns: the resulting number. - */ - long strtol(const char *nptr, char **nptr, int base); - -Instead, please DO say: - - /** Parse a number in radix <b>base</b> from the string <b>nptr</b>, - * and return the result. Skip all leading whitespace. If - * <b>endptr</b> is not NULL, set *<b>endptr</b> to the first character - * after the number parsed. - **/ - long strtol(const char *nptr, char **nptr, int base); - -Doxygen comments are the contract in our abstraction-by-contract world: if -the functions that call your function rely on it doing something, then your -function should mention that it does that something in the documentation. If -you rely on a function doing something beyond what is in its documentation, -then you should watch out, or it might do something else later. - -Putting out a new release -------------------------- - -Here are the steps Roger takes when putting out a new Tor release: - -1) Use it for a while, as a client, as a relay, as a hidden service, -and as a directory authority. See if it has any obvious bugs, and -resolve those. - -1.5) As applicable, merge the maint-X branch into the release-X branch. - -2) Gather the changes/* files into a changelog entry, rewriting many -of them and reordering to focus on what users and funders would find -interesting and understandable. - - 2.1) Make sure that everything that wants a bug number has one. - Make sure that everything which is a bugfix says what version - it was a bugfix on. - 2.2) Concatenate them. - 2.3) Sort them by section. Within each section, sort by "version it's - a bugfix on", else by numerical ticket order. - - 2.4) Clean them up: - - Standard idioms: - "Fixes bug 9999; bugfix on 0.3.3.3-alpha." - - One space after a period. - - Make stuff very terse - - Make sure each section name ends with a colon - - Describe the user-visible problem right away - - Mention relevant config options by name. If they're rare or unusual, - remind people what they're for - - Avoid starting lines with open-paren - - Present and imperative tense: not past. - - 'Relays', not 'servers' or 'nodes' or 'Tor relays'. - - "Stop FOOing", not "Fix a bug where we would FOO". - - Try not to let any given section be longer than about a page. Break up - long sections into subsections by some sort of common subtopic. This - guideline is especially important when organizing Release Notes for - new stable releases. - - If a given changes stanza showed up in a different release (e.g. - maint-0.2.1), be sure to make the stanzas identical (so people can - distinguish if these are the same change). - - 2.5) Merge them in. - - 2.6) Clean everything one last time. - - 2.7) Run ./scripts/maint/format_changelog.py to make it prettier. - -3) Compose a short release blurb to highlight the user-facing -changes. Insert said release blurb into the ChangeLog stanza. If it's -a stable release, add it to the ReleaseNotes file too. If we're adding -to a release-0.2.x branch, manually commit the changelogs to the later -git branches too. - -4) In maint-0.2.x, bump the version number in configure.ac and run - scripts/maint/updateVersions.pl to update version numbers in other - places, and commit. Then merge maint-0.2.x into release-0.2.x. - - (NOTE: TO bump the version number, edit configure.ac, and then run - either make, or 'perl scripts/maint/updateVersions.pl', depending on - your version.) - -5) Make dist, put the tarball up somewhere, and tell #tor about it. Wait -a while to see if anybody has problems building it. Try to get Sebastian -or somebody to try building it on Windows. - -6) Get at least two of weasel/arma/sebastian to put the new version number -in their approved versions list. - -7) Sign the tarball, then sign and push the git tag: - gpg -ba <the_tarball> - git tag -u <keyid> tor-0.2.x.y-status - git push origin tag tor-0.2.x.y-status - -8a) scp the tarball and its sig to the dist website, i.e. -/srv/dist-master.torproject.org/htdocs/ on dist-master. When you want -it to go live, you run "static-update-component dist.torproject.org" -on dist-master. - -8b) Edit "include/versions.wmi" and "Makefile" to note the new version. - -9) Email the packagers (cc'ing tor-assistants) that a new tarball is up. - The current list of packagers is: - {weasel,gk,mikeperry} at torproject dot org - {blueness} at gentoo dot org - {paul} at invizbox dot io - {ondrej.mikle} at gmail dot com - {lfleischer} at archlinux dot org - -10) Add the version number to Trac. To do this, go to Trac, log in, -select "Admin" near the top of the screen, then select "Versions" from -the menu on the left. At the right, there will be an "Add version" -box. By convention, we enter the version in the form "Tor: -0.2.2.23-alpha" (or whatever the version is), and we select the date as -the date in the ChangeLog. - -11) Forward-port the ChangeLog. - -12) Wait up to a day or two (for a development release), or until most -packages are up (for a stable release), and mail the release blurb and -changelog to tor-talk or tor-announce. - - (We might be moving to faster announcements, but don't announce until - the website is at least updated.) - -13) If it's a stable release, bump the version number in the maint-x.y.z - branch to "newversion-dev", and do a "merge -s ours" merge to avoid - taking that change into master. Do a similar 'merge -s theirs' - merge to get the change (and only that change) into release. (Some - of the build scripts require that maint merge cleanly into release.) - diff --git a/doc/HACKING/CodingStandards.md b/doc/HACKING/CodingStandards.md new file mode 100644 index 0000000000..bec076527f --- /dev/null +++ b/doc/HACKING/CodingStandards.md @@ -0,0 +1,240 @@ +Coding conventions for Tor +========================== + +tl;dr: + + - Run configure with `--enable-gcc-warnings` + - Run `make check-spaces` to catch whitespace errors + - Document your functions + - Write unit tests + - Add a file in `changes` for your branch. + +Patch checklist +--------------- + +If possible, send your patch as one of these (in descending order of +preference) + + - A git branch we can pull from + - Patches generated by git format-patch + - A unified diff + +Did you remember... + + - To build your code while configured with `--enable-gcc-warnings`? + - To run `make check-spaces` on your code? + - To run `make check-docs` to see whether all new options are on + the manpage? + - To write unit tests, as possible? + - To base your code on the appropriate branch? + - To include a file in the `changes` directory as appropriate? + +How we use Git branches +======================= + +Each main development series (like 0.2.1, 0.2.2, etc) has its main work +applied to a single branch. At most one series can be the development series +at a time; all other series are maintenance series that get bug-fixes only. +The development series is built in a git branch called "master"; the +maintenance series are built in branches called "maint-0.2.0", "maint-0.2.1", +and so on. We regularly merge the active maint branches forward. + +For all series except the development series, we also have a "release" branch +(as in "release-0.2.1"). The release series is based on the corresponding +maintenance series, except that it deliberately lags the maint series for +most of its patches, so that bugfix patches are not typically included in a +maintenance release until they've been tested for a while in a development +release. Occasionally, we'll merge an urgent bugfix into the release branch +before it gets merged into maint, but that's rare. + +If you're working on a bugfix for a bug that occurs in a particular version, +base your bugfix branch on the "maint" branch for the first supported series +that has that bug. (As of June 2013, we're supporting 0.2.3 and later.) If +you're working on a new feature, base it on the master branch. + + +How we log changes +================== + +When you do a commit that needs a ChangeLog entry, add a new file to +the `changes` toplevel subdirectory. It should have the format of a +one-entry changelog section from the current ChangeLog file, as in + +- Major bugfixes: + - Fix a potential buffer overflow. Fixes bug 99999; bugfix on + 0.3.1.4-beta. + +To write a changes file, first categorize the change. Some common categories +are: Minor bugfixes, Major bugfixes, Minor features, Major features, Code +simplifications and refactoring. Then say what the change does. If +it's a bugfix, mention what bug it fixes and when the bug was +introduced. To find out which Git tag the change was introduced in, +you can use `git describe --contains <sha1 of commit>`. + +If at all possible, try to create this file in the same commit where you are +making the change. Please give it a distinctive name that no other branch will +use for the lifetime of your change. To verify the format of the changes file, +you can use `make check-changes`. + +When we go to make a release, we will concatenate all the entries +in changes to make a draft changelog, and clear the directory. We'll +then edit the draft changelog into a nice readable format. + +What needs a changes file? + + * A not-exhaustive list: Anything that might change user-visible + behavior. Anything that changes internals, documentation, or the build + system enough that somebody could notice. Big or interesting code + rewrites. Anything about which somebody might plausibly wonder "when + did that happen, and/or why did we do that" 6 months down the line. + +Why use changes files instead of Git commit messages? + + * Git commit messages are written for developers, not users, and they + are nigh-impossible to revise after the fact. + +Why use changes files instead of entries in the ChangeLog? + + * Having every single commit touch the ChangeLog file tended to create + zillions of merge conflicts. + +Whitespace and C conformance +---------------------------- + +Invoke `make check-spaces` from time to time, so it can tell you about +deviations from our C whitespace style. Generally, we use: + + - Unix-style line endings + - K&R-style indentation + - No space before newlines + - A blank line at the end of each file + - Never more than one blank line in a row + - Always spaces, never tabs + - No more than 79-columns per line. + - Two spaces per indent. + - A space between control keywords and their corresponding paren + `if (x)`, `while (x)`, and `switch (x)`, never `if(x)`, `while(x)`, or + `switch(x)`. + - A space between anything and an open brace. + - No space between a function name and an opening paren. `puts(x)`, not + `puts (x)`. + - Function declarations at the start of the line. + +We try hard to build without warnings everywhere. In particular, if you're +using gcc, you should invoke the configure script with the option +`--enable-gcc-warnings`. This will give a bunch of extra warning flags to +the compiler, and help us find divergences from our preferred C style. + +Functions to use; functions not to use +-------------------------------------- + +We have some wrapper functions like `tor_malloc`, `tor_free`, `tor_strdup`, and +`tor_gettimeofday;` use them instead of their generic equivalents. (They +always succeed or exit.) + +You can get a full list of the compatibility functions that Tor provides by +looking through `src/common/util*.h` and `src/common/compat*.h`. You can see the +available containers in `src/common/containers*.h`. You should probably +familiarize yourself with these modules before you write too much code, or +else you'll wind up reinventing the wheel. + +We don't use `strcat` or `strcpy` or `sprintf` of any of those notoriously broken +old C functions. Use `strlcat`, `strlcpy`, or `tor_snprintf/tor_asprintf` instead. + +We don't call `memcmp()` directly. Use `fast_memeq()`, `fast_memneq()`, +`tor_memeq()`, or `tor_memneq()` for most purposes. + +Functions not to write +---------------------- + +Try to never hand-write new code to parse or generate binary +formats. Instead, use trunnel if at all possible. See + + https://gitweb.torproject.org/trunnel.git/tree + +for more information about trunnel. + +For information on adding new trunnel code to Tor, see src/trunnel/README + + +Calling and naming conventions +------------------------------ + +Whenever possible, functions should return -1 on error and 0 on success. + +For multi-word identifiers, use lowercase words combined with +underscores. (e.g., `multi_word_identifier`). Use ALL_CAPS for macros and +constants. + +Typenames should end with `_t`. + +Function names should be prefixed with a module name or object name. (In +general, code to manipulate an object should be a module with the same name +as the object, so it's hard to tell which convention is used.) + +Functions that do things should have imperative-verb names +(e.g. `buffer_clear`, `buffer_resize`); functions that return booleans should +have predicate names (e.g. `buffer_is_empty`, `buffer_needs_resizing`). + +If you find that you have four or more possible return code values, it's +probably time to create an enum. If you find that you are passing three or +more flags to a function, it's probably time to create a flags argument that +takes a bitfield. + +What To Optimize +---------------- + +Don't optimize anything if it's not in the critical path. Right now, the +critical path seems to be AES, logging, and the network itself. Feel free to +do your own profiling to determine otherwise. + +Log conventions +--------------- + +`https://www.torproject.org/docs/faq#LogLevel` + +No error or warning messages should be expected during normal OR or OP +operation. + +If a library function is currently called such that failure always means ERR, +then the library function should log WARN and let the caller log ERR. + +Every message of severity INFO or higher should either (A) be intelligible +to end-users who don't know the Tor source; or (B) somehow inform the +end-users that they aren't expected to understand the message (perhaps +with a string like "internal error"). Option (A) is to be preferred to +option (B). + + + +Doxygen comment conventions +--------------------------- + +Say what functions do as a series of one or more imperative sentences, as +though you were telling somebody how to be the function. In other words, DO +NOT say: + + /** The strtol function parses a number. + * + * nptr -- the string to parse. It can include whitespace. + * endptr -- a string pointer to hold the first thing that is not part + * of the number, if present. + * base -- the numeric base. + * returns: the resulting number. + */ + long strtol(const char *nptr, char **nptr, int base); + +Instead, please DO say: + + /** Parse a number in radix <b>base</b> from the string <b>nptr</b>, + * and return the result. Skip all leading whitespace. If + * <b>endptr</b> is not NULL, set *<b>endptr</b> to the first character + * after the number parsed. + **/ + long strtol(const char *nptr, char **nptr, int base); + +Doxygen comments are the contract in our abstraction-by-contract world: if +the functions that call your function rely on it doing something, then your +function should mention that it does that something in the documentation. If +you rely on a function doing something beyond what is in its documentation, +then you should watch out, or it might do something else later. diff --git a/doc/HACKING/GettingStarted.md b/doc/HACKING/GettingStarted.md new file mode 100644 index 0000000000..0295adc1ff --- /dev/null +++ b/doc/HACKING/GettingStarted.md @@ -0,0 +1,187 @@ + +Getting started in Tor development +================================== + +Congratulations! You've found this file, and you're reading it! This +means that you might be interested in getting started in developing Tor. + +(This guide is just about Tor itself--the small network program at the +heart of the Tor network--and not about all the other programs in the +whole Tor ecosystem.) + + +If you are looking for a more bare-bones, less user-friendly information +dump of important information, you might like reading doc/HACKING +instead. You should probably read it before you write your first patch. + + +Required background +------------------- + +First, I'm going to assume that you can build Tor from source, and that +you know enough of the C language to read and write it. (See the README +file that comes with the Tor source for more information on building it, +and any high-quality guide to C for information on programming.) + +I'm also going to assume that you know a little bit about how to use +Git, or that you're able to follow one of the several excellent guides +at http://git-scm.org to learn. + +Most Tor developers develop using some Unix-based system, such as Linux, +BSD, or OSX. It's okay to develop on Windows if you want, but you're +going to have a more difficult time. + + +Getting your first patch into Tor +--------------------------------- + +Once you've reached this point, here's what you need to know. + + 1. Get the source. + + We keep our source under version control in Git. To get the latest + version, run + + git clone https://git.torproject.org/git/tor + + This will give you a checkout of the master branch. If you're + going to fix a bug that appears in a stable version, check out the + appropriate "maint" branch, as in: + + git checkout maint-0.2.7 + + 2. Find your way around the source + + Our overall code structure is explained in the "torguts" documents, + currently at + + git clone https://git.torproject.org/user/nickm/torguts.git + + Find a part of the code that looks interesting to you, and start + looking around it to see how it fits together! + + We do some unusual things in our codebase. Our testing-related + practices and kludges are explained in doc/WritingTests.txt. + + If you see something that doesn't make sense, we love to get + questions! + + 3. Find something cool to hack on. + + You may already have a good idea of what you'd like to work on, or + you might be looking for a way to contribute. + + Many people have gotten started by looking for an area where they + personally felt Tor was underperforming, and investigating ways to + fix it. If you're looking for ideas, you can head to our bug + tracker at trac.torproject.org and look for tickets that have + received the "easy" tag: these are ones that developers think would + be pretty simple for a new person to work on. For a bigger + challenge, you might want to look for tickets with the "lorax" + keyword: these are tickets that the developers think might be a + good idea to build, but which we have no time to work on any time + soon. + + Or you might find another open ticket that piques your + interest. It's all fine! + + For your first patch, it is probably NOT a good idea to make + something huge or invasive. In particular, you should probably + avoid: + + * Major changes spread across many parts of the codebase. + * Major changes to programming practice or coding style. + * Huge new features or protocol changes. + + 4. Meet the developers! + + We discuss stuff on the tor-dev mailing list and on the #tor-dev + IRC channel on OFTC. We're generally friendly and approachable, + and we like to talk about how Tor fits together. If we have ideas + about how something should be implemented, we'll be happy to share + them. + + We currently have a patch workshop at least once a week, where + people share patches they've made and discuss how to make them + better. The time might change in the future, but generally, + there's no bad time to talk, and ask us about patch ideas. + + 5. Do you need to write a design proposal? + + If your idea is very large, or it will require a change to Tor's + protocols, there needs to be a written design proposal before it + can be merged. (We use this process to manage changes in the + protocols.) To write one, see the instructions at + https://gitweb.torproject.org/torspec.git/tree/proposals/001-process.txt + . If you'd like help writing a proposal, just ask! We're happy to + help out with good ideas. + + You might also like to look around the rest of that directory, to + see more about open and past proposed changes to Tor's behavior. + + 6. Writing your patch + + As you write your code, you'll probably want it to fit in with the + standards of the rest of the Tor codebase so it will be easy for us + to review and merge. You can learn our coding standards in + doc/HACKING. + + If your patch is large and/or is divided into multiple logical + components, remember to divide it into a series of Git commits. A + series of small changes is much easier to review than one big lump. + + 7. Testing your patch + + We prefer that all new or modified code have unit tests for it to + ensure that it runs correctly. Also, all code should actually be + _run_ by somebody, to make sure it works. + + See doc/WritingTests.txt for more information on how we test things + in Tor. If you'd like any help writing tests, just ask! We're + glad to help out. + + 8. Submitting your patch + + We review patches through tickets on our bugtracker at + trac.torproject.org. You can either upload your patches there, or + put them at a public git repository somewhere we can fetch them + (like github or bitbucket) and then paste a link on the appropriate + trac ticket. + + Once your patches are available, write a short explanation of what + you've done on trac, and then change the status of the ticket to + needs_review. + + 9. Review, Revision, and Merge + + With any luck, somebody will review your patch soon! If not, you + can ask on the IRC channel; sometimes we get really busy and take + longer than we should. But don't let us slow you down: you're the + one who's offering help here, and we should respect your time and + contributions. + + When your patch is reviewed, one of these things will happen: + + * The reviewer will say "looks good to me" and your + patch will get merged right into Tor. [Assuming we're not + in the middle of a code-freeze window. If the codebase is + frozen, your patch will go into the next release series.] + + * OR the reviewer will say "looks good, just needs some small + changes!" And then the reviewer will make those changes, + and merge the modified patch into Tor. + + * OR the reviewer will say "Here are some questions and + comments," followed by a bunch of stuff that the reviewer + thinks should change in your code, or questions that the + reviewer has. + + At this point, you might want to make the requested changes + yourself, and comment on the trac ticket once you have done + so. Or if you disagree with any of the comments, you should + say so! And if you won't have time to make some of the + changes, you should say that too, so that other developers + will be able to pick up the unfinished portion. + + Congratulations! You have now written your first patch, and gotten + it integrated into mainline Tor. diff --git a/doc/HACKING/HelpfulTools.md b/doc/HACKING/HelpfulTools.md new file mode 100644 index 0000000000..a7f36e6c7e --- /dev/null +++ b/doc/HACKING/HelpfulTools.md @@ -0,0 +1,293 @@ +Useful tools +============ + +These aren't strictly necessary for hacking on Tor, but they can help track +down bugs. + +Jenkins +------- + + https://jenkins.torproject.org + +Dmalloc +------- + +The dmalloc library will keep track of memory allocation, so you can find out +if we're leaking memory, doing any double-frees, or so on. + + dmalloc -l -/dmalloc.log + (run the commands it tells you) + ./configure --with-dmalloc + +Valgrind +-------- + + valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor + +(Note that if you get a zillion openssl warnings, you will also need to +pass `--undef-value-errors=no` to valgrind, or rebuild your openssl +with `-DPURIFY`.) + +Coverity +-------- + +Nick regularly runs the coverity static analyzer on the Tor codebase. + +The preprocessor define `__COVERITY__` is used to work around instances +where coverity picks up behavior that we wish to permit. + +clang Static Analyzer +--------------------- + +The clang static analyzer can be run on the Tor codebase using Xcode (WIP) +or a command-line build. + +The preprocessor define `__clang_analyzer__` is used to work around instances +where clang picks up behavior that we wish to permit. + +clang Runtime Sanitizers +------------------------ + +To build the Tor codebase with the clang Address and Undefined Behavior +sanitizers, see the file `contrib/clang/sanitize_blacklist.txt`. + +Preprocessor workarounds for instances where clang picks up behavior that +we wish to permit are also documented in the blacklist file. + +Running lcov for unit test coverage +----------------------------------- + +Lcov is a utility that generates pretty HTML reports of test code coverage. +To generate such a report: + + ./configure --enable-coverage + make + make coverage-html + $BROWSER ./coverage_html/index.html + +This will run the tor unit test suite `./src/test/test` and generate the HTML +coverage code report under the directory `./coverage_html/`. To change the +output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`. + +Coverage diffs using lcov are not currently implemented, but are being +investigated (as of July 2014). + +Running the unit tests +---------------------- + +To quickly run all the tests distributed with Tor: + + make check + +To run the fast unit tests only: + + make test + +To selectively run just some tests (the following can be combined +arbitrarily): + + ./src/test/test <name_of_test> [<name of test 2>] ... + ./src/test/test <prefix_of_name_of_test>.. [<prefix_of_name_of_test2>..] ... + ./src/test/test :<name_of_excluded_test> [:<name_of_excluded_test2]... + +To run all tests, including those based on Stem or Chutney: + + make test-full + +To run all tests, including those based on Stem or Chutney that require a +working connection to the internet: + + make test-full-online + +Running gcov for unit test coverage +----------------------------------- + + ./configure --enable-coverage + make + make check + # or--- make test-full ? make test-full-online? + mkdir coverage-output + ./scripts/test/coverage coverage-output + +(On OSX, you'll need to start with `--enable-coverage CC=clang`.) + +Then, look at the .gcov files in `coverage-output`. '-' before a line means +that the compiler generated no code for that line. '######' means that the +line was never reached. Lines with numbers were called that number of times. + +If that doesn't work: + + * Try configuring Tor with `--disable-gcc-hardening` + * You might need to run `make clean` after you run `./configure`. + +If you make changes to Tor and want to get another set of coverage results, +you can run `make reset-gcov` to clear the intermediary gcov output. + +If you have two different `coverage-output` directories, and you want to see +a meaningful diff between them, you can run: + + ./scripts/test/cov-diff coverage-output1 coverage-output2 | less + +In this diff, any lines that were visited at least once will have coverage +"1". This lets you inspect what you (probably) really want to know: which +untested lines were changed? Are there any new untested lines? + +Running integration tests +------------------------- + +We have the beginnings of a set of scripts to run integration tests using +Chutney. To try them, set CHUTNEY_PATH to your chutney source directory, and +run `make test-network`. + +We also have scripts to run integration tests using Stem. To try them, set +`STEM_SOURCE_DIR` to your Stem source directory, and run `test-stem`. + +Profiling Tor with oprofile +--------------------------- + +The oprofile tool runs (on Linux only!) to tell you what functions Tor is +spending its CPU time in, so we can identify performance bottlenecks. + +Here are some basic instructions + + - Build tor with debugging symbols (you probably already have, unless + you messed with CFLAGS during the build process). + - Build all the libraries you care about with debugging symbols + (probably you only care about libssl, maybe zlib and Libevent). + - Copy this tor to a new directory + - Copy all the libraries it uses to that dir too (`ldd ./tor` will + tell you) + - Set LD_LIBRARY_PATH to include that dir. `ldd ./tor` should now + show you it's using the libs in that dir + - Run that tor + - Reset oprofiles counters/start it + * `opcontrol --reset; opcontrol --start`, if Nick remembers right. + - After a while, have it dump the stats on tor and all the libs + in that dir you created. + * `opcontrol --dump;` + * `opreport -l that_dir/*` + - Profit + +Generating and analyzing a callgraph +------------------------------------ + +1. Run `./scripts/maint/generate_callgraph.sh`. This will generate a + bunch of files in a new ./callgraph directory. + +2. Run `./scripts/maint/analyze_callgraph.py callgraph/src/*/*`. This + will do a lot of graph operations and then dump out a new + `callgraph.pkl` file, containing data in Python's 'pickle' format. + +3. Run `./scripts/maint/display_callgraph.py`. It will display: + - the number of functions reachable from each function. + - all strongly-connnected components in the Tor callgraph + - the largest bottlenecks in the largest SCC in the Tor callgraph. + +Note that currently the callgraph generator can't detect calls that pass +through function pointers. + +Getting emacs to edit Tor source properly +----------------------------------------- + +Nick likes to put the following snippet in his .emacs file: + + + (add-hook 'c-mode-hook + (lambda () + (font-lock-mode 1) + (set-variable 'show-trailing-whitespace t) + + (let ((fname (expand-file-name (buffer-file-name)))) + (cond + ((string-match "^/home/nickm/src/libevent" fname) + (set-variable 'indent-tabs-mode t) + (set-variable 'c-basic-offset 4) + (set-variable 'tab-width 4)) + ((string-match "^/home/nickm/src/tor" fname) + (set-variable 'indent-tabs-mode nil) + (set-variable 'c-basic-offset 2)) + ((string-match "^/home/nickm/src/openssl" fname) + (set-variable 'indent-tabs-mode t) + (set-variable 'c-basic-offset 8) + (set-variable 'tab-width 8)) + )))) + + +You'll note that it defaults to showing all trailing whitespace. The `cond` +test detects whether the file is one of a few C free software projects that I +often edit, and sets up the indentation level and tab preferences to match +what they want. + +If you want to try this out, you'll need to change the filename regex +patterns to match where you keep your Tor files. + +If you use emacs for editing Tor and nothing else, you could always just say: + + + (add-hook 'c-mode-hook + (lambda () + (font-lock-mode 1) + (set-variable 'show-trailing-whitespace t) + (set-variable 'indent-tabs-mode nil) + (set-variable 'c-basic-offset 2))) + + +There is probably a better way to do this. No, we are probably not going +to clutter the files with emacs stuff. + + +Doxygen +------- + +We use the 'doxygen' utility to generate documentation from our +source code. Here's how to use it: + + 1. Begin every file that should be documented with + + /** + * \file filename.c + * \brief Short description of the file. + */ + + (Doxygen will recognize any comment beginning with /** as special.) + + 2. Before any function, structure, #define, or variable you want to + document, add a comment of the form: + + /** Describe the function's actions in imperative sentences. + * + * Use blank lines for paragraph breaks + * - and + * - hyphens + * - for + * - lists. + * + * Write <b>argument_names</b> in boldface. + * + * \code + * place_example_code(); + * between_code_and_endcode_commands(); + * \endcode + */ + + 3. Make sure to escape the characters `<`, `>`, `\`, `%` and `#` as `\<`, + `\>`, `\\`, `\%` and `\#`. + + 4. To document structure members, you can use two forms: + + struct foo { + /** You can put the comment before an element; */ + int a; + int b; /**< Or use the less-than symbol to put the comment + * after the element. */ + }; + + 5. To generate documentation from the Tor source code, type: + + $ doxygen -g + + to generate a file called `Doxyfile`. Edit that file and run + `doxygen` to generate the API documentation. + + 6. See the Doxygen manual for more information; this summary just + scratches the surface. diff --git a/doc/HACKING/HowToReview.md b/doc/HACKING/HowToReview.md new file mode 100644 index 0000000000..de7891c923 --- /dev/null +++ b/doc/HACKING/HowToReview.md @@ -0,0 +1,85 @@ +How to review a patch +===================== + +Some folks have said that they'd like to review patches more often, but they +don't know how. + +So, here are a bunch of things to check for when reviewing a patch! + +Note that if you can't do every one of these, that doesn't mean you can't do +a good review! Just make it clear what you checked for and what you didn't. + + +Top-level smell-checks +---------------------- + +(Difficulty: easy) + +- Does it compile with `--enable-gcc-warnings`? + +- Does `make check-spaces` pass? + +- Does it have a reasonable amount of tests? Do they pass? Do they leak + memory? + +- Do all the new functions, global variables, types, and structure members have + documentation? + +- Do all the functions, global variables, types, and structure members with + modified behavior have modified documentation? + +- Do all the new torrc options have documentation? + +- If this changes Tor's behavior on the wire, is there a design proposal? + + + +Let's look at the code! +----------------------- + +- Does the code conform to CodingStandards.txt? + +- Does the code leak memory? + +- If two or more pointers ever point to the same object, is it clear which + pointer "owns" the object? + +- Are all allocated resources freed? + +- Are all pointers that should be const, const? + +- Are `#defines` used for 'magic' numbers? + +- Can you understand what the code is trying to do? + +- Can you convince yourself that the code really does that? + +- Is there duplicated code that could be turned into a function? + + +Let's look at the documentation! +-------------------------------- + +- Does the documentation confirm to CodingStandards.txt? + +- Does it make sense? + +- Can you predict what the function will do from its documentation? + + +Let's think about security! +--------------------------- + +- If there are any arrays, buffers, are you 100% sure that they cannot + overflow? + +- If there is any integer math, can it overflow or underflow? + +- If there are any allocations, are you sure there are corresponding + deallocations? + +- Is there a safer pattern that could be used in any case? + +- Have they used one of the Forbidden Functions? + +(Also see your favorite secure C programming guides.) diff --git a/doc/HACKING/README.1st.md b/doc/HACKING/README.1st.md new file mode 100644 index 0000000000..8299fe634a --- /dev/null +++ b/doc/HACKING/README.1st.md @@ -0,0 +1,62 @@ + +In this directory +----------------- + +This directory has helpful information about what you need to know to +hack on Tor! + +First, read `GettingStarted.md` to learn how to get a start in Tor +development. + +If you've decided to write a patch, `CodingStandards.txt` will give +you a bunch of information about how we structure our code. + +It's important to get code right! Reading `WritingTests.md` will +tell you how to write and run tests in the Tor codebase. + +There are a bunch of other programs we use to help maintain and +develop the codebase: `HelpfulTools.md` can tell you how to use them +with Tor. + +If it's your job to put out Tor releases, see `ReleasingTor.md` so +that you don't miss any steps! + + +----------------------- + +For full information on how Tor is supposed to work, look at the files in +`https://gitweb.torproject.org/torspec.git/tree`. + +For an explanation of how to change Tor's design to work differently, look at +`https://gitweb.torproject.org/torspec.git/blob_plain/HEAD:/proposals/001-process.txt`. + +For the latest version of the code, get a copy of git, and + + git clone https://git.torproject.org/git/tor + +We talk about Tor on the `tor-talk` mailing list. Design proposals and +discussion belong on the `tor-dev` mailing list. We hang around on +irc.oftc.net, with general discussion happening on #tor and development +happening on `#tor-dev`. + +The other files in this `HACKING` directory may also be useful as you +get started working with Tor. + +Happy hacking! + + +----------------------- + +XXXXX also describe + +doc/HACKING/WritingTests.md + +torguts.git + +torspec.git + +The design paper + +freehaven.net/anonbib + +XXXX describe these and add links. diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md new file mode 100644 index 0000000000..8f63bad734 --- /dev/null +++ b/doc/HACKING/ReleasingTor.md @@ -0,0 +1,139 @@ + +Putting out a new release +------------------------- + +Here are the steps Roger takes when putting out a new Tor release: + +1. Use it for a while, as a client, as a relay, as a hidden service, + and as a directory authority. See if it has any obvious bugs, and + resolve those. + + As applicable, merge the `maint-X` branch into the `release-X` branch. + +2. Gather the `changes/*` files into a changelog entry, rewriting many + of them and reordering to focus on what users and funders would find + interesting and understandable. + + 1. Make sure that everything that wants a bug number has one. + Make sure that everything which is a bugfix says what version + it was a bugfix on. + + 2. Concatenate them. + + 3. Sort them by section. Within each section, sort by "version it's + a bugfix on", else by numerical ticket order. + + 4. Clean them up: + + Standard idioms: + `Fixes bug 9999; bugfix on 0.3.3.3-alpha.` + + One space after a period. + + Make stuff very terse + + Make sure each section name ends with a colon + + Describe the user-visible problem right away + + Mention relevant config options by name. If they're rare or unusual, + remind people what they're for + + Avoid starting lines with open-paren + + Present and imperative tense: not past. + + 'Relays', not 'servers' or 'nodes' or 'Tor relays'. + + "Stop FOOing", not "Fix a bug where we would FOO". + + Try not to let any given section be longer than about a page. Break up + long sections into subsections by some sort of common subtopic. This + guideline is especially important when organizing Release Notes for + new stable releases. + + If a given changes stanza showed up in a different release (e.g. + maint-0.2.1), be sure to make the stanzas identical (so people can + distinguish if these are the same change). + + 5. Merge them in. + + 6. Clean everything one last time. + + 7. Run `./scripts/maint/format_changelog.py` to make it prettier. + +3. Compose a short release blurb to highlight the user-facing + changes. Insert said release blurb into the ChangeLog stanza. If it's + a stable release, add it to the ReleaseNotes file too. If we're adding + to a release-0.2.x branch, manually commit the changelogs to the later + git branches too. + + If you're doing the first stable release in a series, you need to + create a ReleaseNotes for the series as a whole. To get started + there, copy all of the Changelog entries from the series into a new + file, and run `./scripts/maint/sortChanges.py` on it. That will + group them by category. Then kill every bugfix entry for fixing + bugs that were introduced within that release series; those aren't + relevant changes since the last series. At that point, it's time + to start sorting and condensing entries. (Generally, we don't edit the + text of existing entries, though.) + +4. In `maint-0.2.x`, bump the version number in `configure.ac` and run + `scripts/maint/updateVersions.pl` to update version numbers in other + places, and commit. Then merge `maint-0.2.x` into `release-0.2.x`. + + (NOTE: To bump the version number, edit `configure.ac`, and then run + either `make`, or `perl scripts/maint/updateVersions.pl`, depending on + your version.) + +5. Make distcheck, put the tarball up somewhere, and tell `#tor` about + it. Wait a while to see if anybody has problems building it. Try to + get Sebastian or somebody to try building it on Windows. + +6. Get at least two of weasel/arma/Sebastian to put the new version number + in their approved versions list. + +7. Sign the tarball, then sign and push the git tag: + + gpg -ba <the_tarball> + git tag -u <keyid> tor-0.2.x.y-status + git push origin tag tor-0.2.x.y-status + +8. scp the tarball and its sig to the dist website, i.e. + `/srv/dist-master.torproject.org/htdocs/` on dist-master. When you want + it to go live, you run "static-update-component dist.torproject.org" + on dist-master. + + Edit `include/versions.wmi` and `Makefile` to note the new version. + +9. Email the packagers (cc'ing tor-assistants) that a new tarball is up. + The current list of packagers is: + + - {weasel,gk,mikeperry} at torproject dot org + - {blueness} at gentoo dot org + - {paul} at invizbox dot io + - {ondrej.mikle} at gmail dot com + - {lfleischer} at archlinux dot org + - {tails-dev} at boum dot org + +10. Add the version number to Trac. To do this, go to Trac, log in, + select "Admin" near the top of the screen, then select "Versions" from + the menu on the left. At the right, there will be an "Add version" + box. By convention, we enter the version in the form "Tor: + 0.2.2.23-alpha" (or whatever the version is), and we select the date as + the date in the ChangeLog. + +11. Forward-port the ChangeLog (and ReleaseNotes if appropriate). + +12. Wait up to a day or two (for a development release), or until most + packages are up (for a stable release), and mail the release blurb and + changelog to tor-talk or tor-announce. + + (We might be moving to faster announcements, but don't announce until + the website is at least updated.) + +13. If it's a stable release, bump the version number in the `maint-x.y.z` + branch to "newversion-dev", and do a `merge -s ours` merge to avoid + taking that change into master. Do a similar `merge -s theirs` + merge to get the change (and only that change) into release. (Some + of the build scripts require that maint merge cleanly into release.) diff --git a/doc/HACKING/WritingTests.md b/doc/HACKING/WritingTests.md new file mode 100644 index 0000000000..42fba2d71a --- /dev/null +++ b/doc/HACKING/WritingTests.md @@ -0,0 +1,403 @@ + +Writing tests for Tor: an incomplete guide +========================================== + +Tor uses a variety of testing frameworks and methodologies to try to +keep from introducing bugs. The major ones are: + + 1. Unit tests written in C and shipped with the Tor distribution. + + 2. Integration tests written in Python and shipped with the Tor + distribution. + + 3. Integration tests written in Python and shipped with the Stem + library. Some of these use the Tor controller protocol. + + 4. System tests written in Python and SH, and shipped with the + Chutney package. These work by running many instances of Tor + locally, and sending traffic through them. + + 5. The Shadow network simulator. + +How to run these tests +---------------------- + +### The easy version + +To run all the tests that come bundled with Tor, run `make check`. + +To run the Stem tests as well, fetch stem from the git repository, +set `STEM_SOURCE_DIR` to the checkout, and run `make test-stem`. + +To run the Chutney tests as well, fetch chutney from the git repository, +set `CHUTNEY_PATH` to the checkout, and run `make test-network`. + +To run all of the above, run `make test-full`. + +To run all of the above, plus tests that require a working connection to the +internet, run `make test-full-online`. + +### Running particular subtests + +The Tor unit tests are divided into separate programs and a couple of +bundled unit test programs. + +Separate programs are easy. For example, to run the memwipe tests in +isolation, you just run `./src/test/test-memwipe`. + +To run tests within the unit test programs, you can specify the name +of the test. The string ".." can be used as a wildcard at the end of the +test name. For example, to run all the cell format tests, enter +`./src/test/test cellfmt/..`. To run + +Many tests that need to mess with global state run in forked subprocesses in +order to keep from contaminating one another. But when debugging a failing test, +you might want to run it without forking a subprocess. To do so, use the +`--no-fork` option with a single test. (If you specify it along with +multiple tests, they might interfere.) + +You can turn on logging in the unit tests by passing one of `--debug`, +`--info`, `--notice`, or `--warn`. By default only errors are displayed. + +Unit tests are divided into `./src/test/test` and `./src/test/test-slow`. +The former are those that should finish in a few seconds; the latter tend to +take more time, and may include CPU-intensive operations, deliberate delays, +and stuff like that. + +### Finding test coverage + +Test coverage is a measurement of which lines your tests actually visit. + +When you configure Tor with the `--enable-coverage` option, it should +build with support for coverage in the unit tests, and in a special +`tor-cov` binary. + +Then, run the tests you'd like to see coverage from. If you have old +coverage output, you may need to run `reset-gcov` first. + +Now you've got a bunch of files scattered around your build directories +called `*.gcda`. In order to extract the coverage output from them, make a +temporary directory for them and run `./scripts/test/coverage ${TMPDIR}`, +where `${TMPDIR}` is the temporary directory you made. This will create a +`.gcov` file for each source file under tests, containing that file's source +annotated with the number of times the tests hit each line. (You'll need to +have gcov installed.) + +You can get a summary of the test coverage for each file by running +`./scripts/test/cov-display ${TMPDIR}/*` . Each line lists the file's name, +the number of uncovered lines, the number of uncovered lines, and the +coverage percentage. + +For a summary of the test coverage for each _function_, run +`./scripts/test/cov-display -f ${TMPDIR}/*`. + +### Comparing test coverage + +Sometimes it's useful to compare test coverage for a branch you're writing to +coverage from another branch (such as git master, for example). But you +can't run `diff` on the two coverage outputs directly, since the actual +number of times each line is executed aren't so important, and aren't wholly +deterministic. + +Instead, follow the instructions above for each branch, creating a separate +temporary directory for each. Then, run `./scripts/test/cov-diff ${D1} +${D2}`, where D1 and D2 are the directories you want to compare. This will +produce a diff of the two directories, with all lines normalized to be either +covered or uncovered. + +To count new or modified uncovered lines in D2, you can run: + + ./scripts/test/cov-diff ${D1} ${D2}" | grep '^+ *\#' | wc -l + + +What kinds of test should I write? +---------------------------------- + +Integration testing and unit testing are complementary: it's probably a +good idea to make sure that your code is hit by both if you can. + +If your code is very-low level, and its behavior is easily described in +terms of a relation between inputs and outputs, or a set of state +transitions, then it's a natural fit for unit tests. (If not, please +consider refactoring it until most of it _is_ a good fit for unit +tests!) + +If your code adds new externally visible functionality to Tor, it would +be great to have a test for that functionality. That's where +integration tests more usually come in. + +Unit and regression tests: Does this function do what it's supposed to? +----------------------------------------------------------------------- + +Most of Tor's unit tests are made using the "tinytest" testing framework. +You can see a guide to using it in the tinytest manual at + + https://github.com/nmathewson/tinytest/blob/master/tinytest-manual.md + +To add a new test of this kind, either edit an existing C file in `src/test/`, +or create a new C file there. Each test is a single function that must +be indexed in the table at the end of the file. We use the label "done:" as +a cleanup point for all test functions. + +(Make sure you read `tinytest-manual.md` before proceeding.) + +I use the term "unit test" and "regression tests" very sloppily here. + +### A simple example + +Here's an example of a test function for a simple function in util.c: + + static void + test_util_writepid(void *arg) + { + (void) arg; + + char *contents = NULL; + const char *fname = get_fname("tmp_pid"); + unsigned long pid; + char c; + + write_pidfile(fname); + + contents = read_file_to_str(fname, 0, NULL); + tt_assert(contents); + + int n = sscanf(contents, "%lu\n%c", &pid, &c); + tt_int_op(n, OP_EQ, 1); + tt_int_op(pid, OP_EQ, getpid()); + + done: + tor_free(contents); + } + +This should look pretty familiar to you if you've read the tinytest +manual. One thing to note here is that we use the testing-specific +function `get_fname` to generate a file with respect to a temporary +directory that the tests use. You don't need to delete the file; +it will get removed when the tests are done. + +Also note our use of `OP_EQ` instead of `==` in the `tt_int_op()` calls. +We define `OP_*` macros to use instead of the binary comparison +operators so that analysis tools can more easily parse our code. +(Coccinelle really hates to see `==` used as a macro argument.) + +Finally, remember that by convention, all `*_free()` functions that +Tor defines are defined to accept NULL harmlessly. Thus, you don't +need to say `if (contents)` in the cleanup block. + +### Exposing static functions for testing + +Sometimes you need to test a function, but you don't want to expose +it outside its usual module. + +To support this, Tor's build system compiles a testing version of +each module, with extra identifiers exposed. If you want to +declare a function as static but available for testing, use the +macro `STATIC` instead of `static`. Then, make sure there's a +macro-protected declaration of the function in the module's header. + +For example, `crypto_curve25519.h` contains: + + #ifdef CRYPTO_CURVE25519_PRIVATE + STATIC int curve25519_impl(uint8_t *output, const uint8_t *secret, + const uint8_t *basepoint); + #endif + +The `crypto_curve25519.c` file and the `test_crypto.c` file both define +`CRYPTO_CURVE25519_PRIVATE`, so they can see this declaration. + +### Mock functions for testing in isolation + +Often we want to test that a function works right, but the function to +be tested depends on other functions whose behavior is hard to observe, +or which require a working Tor network, or something like that. + +To write tests for this case, you can replace the underlying functions +with testing stubs while your unit test is running. You need to declare +the underlying function as 'mockable', as follows: + + MOCK_DECL(returntype, functionname, (argument list)); + +and then later implement it as: + + MOCK_IMPL(returntype, functionname, (argument list)) + { + /* implementation here */ + } + +For example, if you had a 'connect to remote server' function, you could +declare it as: + + + MOCK_DECL(int, connect_to_remote, (const char *name, status_t *status)); + +When you declare a function this way, it will be declared as normal in +regular builds, but when the module is built for testing, it is declared +as a function pointer initialized to the actual implementation. + +In your tests, if you want to override the function with a temporary +replacement, you say: + + MOCK(functionname, replacement_function_name); + +And later, you can restore the original function with: + + UNMOCK(functionname); + +For more information, see the definitions of this mocking logic in +`testsupport.h`. + +### Okay but what should my tests actually do? + +We talk above about "test coverage" -- making sure that your tests visit +every line of code, or every branch of code. But visiting the code isn't +enough: we want to verify that it's correct. + +So when writing tests, try to make tests that should pass with any correct +implementation of the code, and that should fail if the code doesn't do what +it's supposed to do. + +You can write "black-box" tests or "glass-box" tests. A black-box test is +one that you write without looking at the structure of the function. A +glass-box one is one you implement while looking at how the function is +implemented. + +In either case, make sure to consider common cases *and* edge cases; success +cases and failure csaes. + +For example, consider testing this function: + + /** Remove all elements E from sl such that E==element. Preserve + * the order of any elements before E, but elements after E can be + * rearranged. + */ + void smartlist_remove(smartlist_t *sl, const void *element); + +In order to test it well, you should write tests for at least all of the +following cases. (These would be black-box tests, since we're only looking +at the declared behavior for the function: + + * Remove an element that is in the smartlist. + * Remove an element that is not in the smartlist. + * Remove an element that appears in the smartlist more than once. + +And your tests should verify that it behaves correct. At minimum, you should +test: + + * That other elements before E are in the same order after you call the + functions. + * That the target element is really removed. + * That _only_ the target element is removed. + +When you consider edge cases, you might try: + + * Remove an element from an empty list. + * Remove an element from a singleton list containing that element. + * Remove an element for a list containing several instances of that + element, and nothing else. + +Now let's look at the implementation: + + void + smartlist_remove(smartlist_t *sl, const void *element) + { + int i; + if (element == NULL) + return; + for (i=0; i < sl->num_used; i++) + if (sl->list[i] == element) { + sl->list[i] = sl->list[--sl->num_used]; /* swap with the end */ + i--; /* so we process the new i'th element */ + sl->list[sl->num_used] = NULL; + } + } + +Based on the implementation, we now see three more edge cases to test: + + * Removing NULL from the list. + * Removing an element from the end of the list + * Removing an element from a position other than the end of the list. + + +### What should my tests NOT do? + +Tests shouldn't require a network connection. + +Whenever possible, tests shouldn't take more than a second. Put the test +into test/slow if it genuinely needs to be run. + +Tests should not alter global state unless they run with `TT_FORK`: Tests +should not require other tests to be run before or after them. + +Tests should not leak memory or other resources. To find out if your tests +are leaking memory, run them under valgrind (see HelpfulTools.txt for more +information on how to do that). + +When possible, tests should not be over-fit to the implementation. That is, +the test should verify that the documented behavior is implemented, but +should not break if other permissible behavior is later implemented. + + +### Advanced techniques: Namespaces + +Sometimes, when you're doing a lot of mocking at once, it's convenient to +isolate your identifiers within a single namespace. If this were C++, we'd +already have namespaces, but for C, we do the best we can with macros and +token-pasting. + +We have some macros defined for this purpose in `src/test/test.h`. To use +them, you define `NS_MODULE` to a prefix to be used for your identifiers, and +then use other macros in place of identifier names. See `src/test/test.h` for +more documentation. + + +Integration tests: Calling Tor from the outside +----------------------------------------------- + +Some tests need to invoke Tor from the outside, and shouldn't run from the +same process as the Tor test program. Reasons for doing this might include: + + * Testing the actual behavior of Tor when run from the command line + * Testing that a crash-handler correctly logs a stack trace + * Verifying that violating a sandbox or capability requirement will + actually crash the program. + * Needing to run as root in order to test capability inheritance or + user switching. + +To add one of these, you generally want a new C program in `src/test`. Add it +to `TESTS` and `noinst_PROGRAMS` if it can run on its own and return success or +failure. If it needs to be invoked multiple times, or it needs to be +wrapped, add a new shell script to `TESTS`, and the new program to +`noinst_PROGRAMS`. If you need access to any environment variable from the +makefile (eg `${PYTHON}` for a python interpreter), then make sure that the +makefile exports them. + +Writing integration tests with Stem +----------------------------------- + +The 'stem' library includes extensive unit tests for the Tor controller +protocol. + +For more information on writing new tests for stem, have a look around +the `test/*` directory in stem, and find a good example to emulate. You +might want to start with +`https://gitweb.torproject.org/stem.git/tree/test/integ/control/controller.py` +to improve Tor's test coverage. + +You can run stem tests from tor with `make test-stem`, or see +`https://stem.torproject.org/faq.html#how-do-i-run-the-tests`. + +System testing with Chutney +--------------------------- + +The 'chutney' program configures and launches a set of Tor relays, +authorities, and clients on your local host. It has a `test network` +functionality to send traffic through them and verify that the traffic +arrives correctly. + +You can write new test networks by adding them to `networks`. To add +them to Tor's tests, add them to the `test-network` or `test-network-all` +targets in `Makefile.am`. + +(Adding new kinds of program to chutney will still require hacking the +code.) diff --git a/doc/WritingTests.txt b/doc/WritingTests.txt deleted file mode 100644 index 62a17e3709..0000000000 --- a/doc/WritingTests.txt +++ /dev/null @@ -1,273 +0,0 @@ - -Writing tests for Tor: an incomplete guide -========================================== - -Tor uses a variety of testing frameworks and methodologies to try to -keep from introducing bugs. The major ones are: - - 1. Unit tests written in C and shipped with the Tor distribution. - - 2. Integration tests written in Python and shipped with the Tor - distribution. - - 3. Integration tests written in Python and shipped with the Stem - library. Some of these use the Tor controller protocol. - - 4. System tests written in Python and SH, and shipped with the - Chutney package. These work by running many instances of Tor - locally, and sending traffic through them. - - 5. The Shadow network simulator. - -How to run these tests ----------------------- - -=== The easy version - -To run all the tests that come bundled with Tor, run "make check" - -To run the Stem tests as well, fetch stem from the git repository, -set STEM_SOURCE_DIR to the checkout, and run "make test-stem". - -To run the Chutney tests as well, fetch chutney from the git repository, -set CHUTNEY_PATH to the checkout, and run "make test-network". - -To run all of the above, run "make test-full". - -To run all of the above, plus tests that require a working connection to the -internet, run "make test-full-online". - -=== Running particular subtests - -The Tor unit tests are divided into separate programs and a couple of -bundled unit test programs. - -Separate programs are easy. For example, to run the memwipe tests in -isolation, you just run ./src/test/test-memwipe . - -To run tests within the unit test programs, you can specify the name -of the test. The string ".." can be used as a wildcard at the end of the -test name. For example, to run all the cell format tests, enter -"./src/test/test cellfmt/..". To run - -Many tests that need to mess with global state run in forked subprocesses in -order to keep from contaminating one another. But when debugging a failing test, -you might want to run it without forking a subprocess. To do so, use the -"--no-fork" option with a single test. (If you specify it along with -multiple tests, they might interfere.) - -You can turn on logging in the unit tests by passing one of "--debug", -"--info", "--notice", or "--warn". By default only errors are displayed. - -Unit tests are divided into "./src/test/test" and "./src/test/test-slow". -The former are those that should finish in a few seconds; the latter tend to -take more time, and may include CPU-intensive operations, deliberate delays, -and stuff like that. - -=== Finding test coverage - -When you configure Tor with the --enable-coverage option, it should -build with support for coverage in the unit tests, and in a special -"tor-cov" binary. - -Then, run the tests you'd like to see coverage from. If you have old -coverage output, you may need to run "reset-gcov" first. - -Now you've got a bunch of files scattered around your build directories -called "*.gcda". In order to extract the coverage output from them, make a -temporary directory for them and run "./scripts/test/coverage ${TMPDIR}", -where ${TMPDIR} is the temporary directory you made. This will create a -".gcov" file for each source file under tests, containing that file's source -annotated with the number of times the tests hit each line. (You'll need to -have gcov installed.) - -You can get a summary of the test coverage for each file by running -"./scripts/test/cov-display ${TMPDIR}/*" . Each line lists the file's name, -the number of uncovered lines, the number of uncovered lines, and the -coverage percentage. - -For a summary of the test coverage for each _function_, run -"./scripts/test/cov-display -f ${TMPDIR}/*" . - -=== Comparing test coverage - -Sometimes it's useful to compare test coverage for a branch you're writing to -coverage from another branch (such as git master, for example). But you -can't run "diff" on the two coverage outputs directly, since the actual -number of times each line is executed aren't so important, and aren't wholly -deterministic. - -Instead, follow the instructions above for each branch, creating a separate -temporary directory for each. Then, run "./scripts/test/cov-diff ${D1} -${D2}", where D1 and D2 are the directories you want to compare. This will -produce a diff of the two directories, with all lines normalized to be either -covered or uncovered. - -To count new or modified uncovered lines in D2, you can run: - - "./scripts/test/cov-diff ${D1} ${D2}" | grep '^+ *\#' |wc -l - - -What kinds of test should I write? ----------------------------------- - -Integration testing and unit testing are complementary: it's probably a -good idea to make sure that your code is hit by both if you can. - -If your code is very-low level, and its behavior is easily described in -terms of a relation between inputs and outputs, or a set of state -transitions, then it's a natural fit for unit tests. (If not, please -consider refactoring it until most of it _is_ a good fit for unit -tests!) - -If your code adds new externally visible functionality to Tor, it would -be great to have a test for that functionality. That's where -integration tests more usually come in. - -Unit and regression tests: Does this function do what it's supposed to? ------------------------------------------------------------------------ - -Most of Tor's unit tests are made using the "tinytest" testing framework. -You can see a guide to using it in the tinytest manual at - - https://github.com/nmathewson/tinytest/blob/master/tinytest-manual.md - -To add a new test of this kind, either edit an existing C file in src/test/, -or create a new C file there. Each test is a single function that must -be indexed in the table at the end of the file. We use the label "done:" as -a cleanup point for all test functions. - -(Make sure you read tinytest-manual.md before proceeding.) - -I use the term "unit test" and "regression tests" very sloppily here. - -=== A simple example - -Here's an example of a test function for a simple function in util.c: - - static void - test_util_writepid(void *arg) - { - (void) arg; - - char *contents = NULL; - const char *fname = get_fname("tmp_pid"); - unsigned long pid; - char c; - - write_pidfile(fname); - - contents = read_file_to_str(fname, 0, NULL); - tt_assert(contents); - - int n = sscanf(contents, "%lu\n%c", &pid, &c); - tt_int_op(n, OP_EQ, 1); - tt_int_op(pid, OP_EQ, getpid()); - - done: - tor_free(contents); - } - -This should look pretty familiar to you if you've read the tinytest -manual. One thing to note here is that we use the testing-specific -function "get_fname" to generate a file with respect to a temporary -directory that the tests use. You don't need to delete the file; -it will get removed when the tests are done. - -Also note our use of OP_EQ instead of == in the tt_int_op() calls. -We define OP_* macros to use instead of the binary comparison -operators so that analysis tools can more easily parse our code. -(Coccinelle really hates to see == used as a macro argument.) - -Finally, remember that by convention, all *_free() functions that -Tor defines are defined to accept NULL harmlessly. Thus, you don't -need to say "if (contents)" in the cleanup block. - -=== Exposing static functions for testing - -Sometimes you need to test a function, but you don't want to expose -it outside its usual module. - -To support this, Tor's build system compiles a testing version of -each module, with extra identifiers exposed. If you want to -declare a function as static but available for testing, use the -macro "STATIC" instead of "static". Then, make sure there's a -macro-protected declaration of the function in the module's header. - -For example, crypto_curve25519.h contains: - -#ifdef CRYPTO_CURVE25519_PRIVATE -STATIC int curve25519_impl(uint8_t *output, const uint8_t *secret, - const uint8_t *basepoint); -#endif - -The crypto_curve25519.c file and the test_crypto.c file both define -CRYPTO_CURVE25519_PRIVATE, so they can see this declaration. - -=== Mock functions for testing in isolation - -Often we want to test that a function works right, but the function to -be tested depends on other functions whose behavior is hard to observe, -or which require a working Tor network, or something like that. - -To write tests for this case, you can replace the underlying functions -with testing stubs while your unit test is running. You need to declare -the underlying function as 'mockable', as follows: - - MOCK_DECL(returntype, functionname, (argument list)); - -and then later implement it as: - - MOCK_IMPL(returntype, functionname, (argument list)) - { - /* implementation here */ - } - -For example, if you had a 'connect to remote server' function, you could -declare it as: - - - MOCK_DECL(int, connect_to_remote, (const char *name, status_t *status)); - -When you declare a function this way, it will be declared as normal in -regular builds, but when the module is built for testing, it is declared -as a function pointer initialized to the actual implementation. - -In your tests, if you want to override the function with a temporary -replacement, you say: - - MOCK(functionname, replacement_function_name); - -And later, you can restore the original function with: - - UNMOCK(functionname); - -For more information, see the definitions of this mocking logic in -testsupport.h. - - -=== Advanced techniques: Namespaces - -XXXX write this. danah boyd made us some really awesome stuff here. - - -Integration tests: Calling Tor from the outside ------------------------------------------------ - -XXXX WRITEME - -Writing integration tests with Stem ------------------------------------ - -XXXX WRITEME - -System testing with Chutney ---------------------------- - -XXXX WRITEME - -Who knows what evil lurks in the timings of networks? The Shadow knows! ------------------------------------------------------------------------ - -XXXX WRITEME - diff --git a/doc/include.am b/doc/include.am index 41d3d2a0c7..01a5ef2524 100644 --- a/doc/include.am +++ b/doc/include.am @@ -34,12 +34,18 @@ nodist_man1_MANS = doc_DATA = endif -EXTRA_DIST+= doc/HACKING doc/asciidoc-helper.sh \ +EXTRA_DIST+= doc/asciidoc-helper.sh \ $(html_in) $(man_in) $(txt_in) \ doc/state-contents.txt \ doc/torrc_format.txt \ doc/TUNING \ - doc/WritingTests.txt + doc/HACKING/README.1st.md \ + doc/HACKING/CodingStandards.md \ + doc/HACKING/GettingStarted.md \ + doc/HACKING/HelpfulTools.md \ + doc/HACKING/HowToReview.md \ + doc/HACKING/ReleasingTor.md \ + doc/HACKING/WritingTests.md docdir = @docdir@ diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 5ea5623952..041b000f09 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -98,8 +98,8 @@ COMMAND-LINE OPTIONS Other options can be specified on the command-line in the format "--option value", in the format "option value", or in a configuration file. For instance, you can tell Tor to start listening for SOCKS connections on port -9999 by passing --SOCKSPort 9999 or SOCKSPort 9999 to it on the command line, -or by putting "SOCKSPort 9999" in the configuration file. You will need to +9999 by passing --SocksPort 9999 or SocksPort 9999 to it on the command line, +or by putting "SocksPort 9999" in the configuration file. You will need to quote options with spaces in them: if you want Tor to log all debugging messages to debug.log, you will probably need to say --Log 'debug file debug.log'. @@ -125,26 +125,31 @@ the defaults file. This rule is simple for options that take a single value, but it can become complicated for options that are allowed to occur more than once: if you -specify four SOCKSPorts in your configuration file, and one more SOCKSPort on +specify four SocksPorts in your configuration file, and one more SocksPort on the command line, the option on the command line will replace __all__ of the -SOCKSPorts in the configuration file. If this isn't what you want, prefix -the option name with a plus sign, and it will be appended to the previous set -of options instead. +SocksPorts in the configuration file. If this isn't what you want, prefix +the option name with a plus sign (+), and it will be appended to the previous +set of options instead. For example, setting SocksPort 9100 will use only +port 9100, but setting +SocksPort 9100 will use ports 9100 and 9050 (because +this is the default). Alternatively, you might want to remove every instance of an option in the configuration file, and not replace it at all: you might want to say on the -command line that you want no SOCKSPorts at all. To do that, prefix the -option name with a forward slash. +command line that you want no SocksPorts at all. To do that, prefix the +option name with a forward slash (/). You can use the plus sign (+) and the +forward slash (/) in the configuration file and on the command line. GENERAL OPTIONS --------------- [[BandwidthRate]] **BandwidthRate** __N__ **bytes**|**KBytes**|**MBytes**|**GBytes**|**KBits**|**MBits**|**GBits**:: - A token bucket limits the average incoming bandwidth usage on this node to - the specified number of bytes per second, and the average outgoing + A token bucket limits the average incoming bandwidth usage on this node + to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. If you want to run a relay in the - public network, this needs to be _at the very least_ 30 KBytes (that is, - 30720 bytes). (Default: 1 GByte) + + public network, this needs to be _at the very least_ 75 KBytes for a + relay (that is, 600 kbits) or 50 KBytes for a bridge (400 kbits) -- but of + course, more is better; we recommend at least 250 KBytes (2 mbits) if + possible. (Default: 1 GByte) + + With this option, and in other options that take arguments in bytes, KBytes, and so on, other formats are also supported. Notably, "KBytes" can @@ -216,7 +221,7 @@ GENERAL OPTIONS any pluggable transport proxy that tries to launch __transport__. + (Example: ServerTransportOptions obfs45 shared-secret=bridgepasswd cache=/var/lib/tor/cache) -[[ExtORPort]] **ExtORPort** \['address':]__port__|**auto** +[[ExtORPort]] **ExtORPort** \['address':]__port__|**auto**:: Open this port to listen for Extended ORPort connections from your pluggable transports. @@ -277,15 +282,16 @@ GENERAL OPTIONS [[ControlPort]] **ControlPort** __PORT__|**unix:**__path__|**auto** [__flags__]:: If set, Tor will accept connections on this port and allow those connections to control the Tor process using the Tor Control Protocol - (described in control-spec.txt). Note: unless you also specify one or - more of **HashedControlPassword** or **CookieAuthentication**, - setting this option will cause Tor to allow any process on the local - host to control it. (Setting both authentication methods means either - method is sufficient to authenticate to Tor.) This + (described in control-spec.txt in + https://spec.torproject.org[torspec]). Note: unless you also + specify one or more of **HashedControlPassword** or + **CookieAuthentication**, setting this option will cause Tor to allow + any process on the local host to control it. (Setting both authentication + methods means eithermethod is sufficient to authenticate to Tor.) This option is required for many Tor controllers; most use the value of 9051. - Set it to "auto" to have Tor pick a port for you. (Default: 0) + Set it to "auto" to have Tor pick a port for you. (Default: 0) + + - Recognized flags are:: + Recognized flags are... **GroupWritable**;; Unix domain sockets only: makes the socket get created as group-writable. @@ -347,6 +353,11 @@ GENERAL OPTIONS [[DataDirectory]] **DataDirectory** __DIR__:: Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor) +[[DataDirectoryGroupReadable]] **DataDirectoryGroupReadable** **0**|**1**:: + If this option is set to 0, don't allow the filesystem group to read the + DataDirectory. If the option is set to 1, make the DataDirectory readable + by the default GID. (Default: 0) + [[FallbackDir]] **FallbackDir** __address__:__port__ orport=__port__ id=__fingerprint__ [weight=__num__]:: When we're unable to connect to any directory cache for directory info (usually because we don't know about any yet) we try a FallbackDir. @@ -580,6 +591,10 @@ GENERAL OPTIONS If 1, Tor will overwrite logs at startup and in response to a HUP signal, instead of appending to them. (Default: 0) +[[SyslogIdentityTag]] **SyslogIdentityTag** __tag__:: + When logging to syslog, adds a tag to the syslog identity such that + log entries are marked with "Tor-__tag__". (Default: none) + [[SafeLogging]] **SafeLogging** **0**|**1**|**relay**:: Tor can scrub potentially sensitive strings from log messages (e.g. addresses) by replacing them with the string [scrubbed]. This way logs can @@ -958,7 +973,7 @@ The following options are useful only for clients (that is, if the same circuit. Currently, two addresses are "too close" if they lie in the same /16 range. (Default: 1) -[[SOCKSPort]] **SOCKSPort** \['address':]__port__|**unix:**__path__|**auto** [_flags_] [_isolation flags_]:: +[[SocksPort]] **SocksPort** \['address':]__port__|**unix:**__path__|**auto** [_flags_] [_isolation flags_]:: Open this port to listen for connections from SOCKS-speaking applications. Set this to 0 if you don't want to allow application connections via SOCKS. Set it to "auto" to have Tor pick a port for @@ -973,7 +988,7 @@ The following options are useful only for clients (that is, if to use your computer as an open proxy. + + The _isolation flags_ arguments give Tor rules for which streams - received on this SOCKSPort are allowed to share circuits with one + received on this SocksPort are allowed to share circuits with one another. Recognized isolation flags are: **IsolateClientAddr**;; Don't share circuits with streams from a different @@ -1000,11 +1015,11 @@ The following options are useful only for clients (that is, if If no other isolation rules would prevent it, allow streams on this port to share circuits with streams from every other port with the same session group. (By default, streams received - on different SOCKSPorts, TransPorts, etc are always isolated from one + on different SocksPorts, TransPorts, etc are always isolated from one another. This option overrides that behavior.) -[[OtherSOCKSPortFlags]]:: - Other recognized __flags__ for a SOCKSPort are: +[[OtherSocksPortFlags]]:: + Other recognized __flags__ for a SocksPort are: **NoIPv4Traffic**;; Tell exits to not connect to IPv4 addresses in response to SOCKS requests on this connection. @@ -1056,14 +1071,14 @@ The following options are useful only for clients (that is, if authentication" when IsolateSOCKSAuth is disabled, or when this option is set. -[[SOCKSListenAddress]] **SOCKSListenAddress** __IP__[:__PORT__]:: +[[SocksListenAddress]] **SocksListenAddress** __IP__[:__PORT__]:: Bind to this address to listen for connections from Socks-speaking applications. (Default: 127.0.0.1) You can also specify a port (e.g. 192.168.0.1:9100). This directive can be specified multiple times to bind to multiple addresses/ports. (DEPRECATED: As of 0.2.3.x-alpha, you can - now use multiple SOCKSPort entries, and provide addresses for SOCKSPort - entries, so SOCKSListenAddress no longer has a purpose. For backward - compatibility, SOCKSListenAddress is only allowed when SOCKSPort is just + now use multiple SocksPort entries, and provide addresses for SocksPort + entries, so SocksListenAddress no longer has a purpose. For backward + compatibility, SocksListenAddress is only allowed when SocksPort is just a port number.) [[SocksPolicy]] **SocksPolicy** __policy__,__policy__,__...__:: @@ -1270,7 +1285,7 @@ The following options are useful only for clients (that is, if Use 0 if you don't want to allow NATD connections. Set the port to "auto" to have Tor pick a port for you. This directive can be specified multiple times to bind to multiple addresses/ports. See - SOCKSPort for an explanation of isolation flags. + + SocksPort for an explanation of isolation flags. + + This option is only for people who cannot use TransPort. (Default: 0) @@ -1298,7 +1313,7 @@ The following options are useful only for clients (that is, if doesn't handle arbitrary DNS request types. Set the port to "auto" to have Tor pick a port for you. This directive can be specified multiple times to bind to multiple - addresses/ports. See SOCKSPort for an explanation of isolation + addresses/ports. See SocksPort for an explanation of isolation flags. (Default: 0) [[DNSListenAddress]] **DNSListenAddress** __IP__[:__PORT__]:: @@ -1561,7 +1576,7 @@ is non-zero): used with accept6/reject6.) + + Private addresses are rejected by default (at the beginning of your exit - policy), along with the configured primary public IPv4 and IPv6 addresses, + policy), along with any configured primary public IPv4 and IPv6 addresses, and any public IPv4 and IPv6 addresses on any interface on the relay. These private addresses are rejected unless you set the ExitPolicyRejectPrivate config option to 0. For example, once you've done @@ -1599,10 +1614,13 @@ is non-zero): IPv4 and IPv6 addresses. [[ExitPolicyRejectPrivate]] **ExitPolicyRejectPrivate** **0**|**1**:: - Reject all private (local) networks, along with your own configured public - IPv4 and IPv6 addresses, at the beginning of your exit policy. Also reject - any public IPv4 and IPv6 addresses on any interface on the relay. (If - IPv6Exit is not set, all IPv6 addresses will be rejected anyway.) + Reject all private (local) networks, along with any configured public + IPv4 and IPv6 addresses, at the beginning of your exit policy. (This + includes the IPv4 and IPv6 addresses advertised by the relay, any + OutboundBindAddress, and the bind addresses of any port options, such as + ORPort and DirPort.) This also rejects any public IPv4 and IPv6 addresses + on any interface on the relay. (If IPv6Exit is not set, all IPv6 addresses + will be rejected anyway.) See above entry on ExitPolicy. (Default: 1) @@ -1717,7 +1735,7 @@ is non-zero): Limits the max number of bytes sent and received within a set time period using a given calculation rule (see: AccountingStart, AccountingRule). Useful if you need to stay under a specific bandwidth. By default, the - number used for calculation is the max of either the bytes sent or + number used for calculation is the max of either the bytes sent or received. For example, with AccountingMax set to 1 GByte, a server could send 900 MBytes and receive 800 MBytes and continue running. It will only hibernate once one of the two reaches 1 GByte. This can @@ -1788,7 +1806,7 @@ is non-zero): [[ServerDNSTestAddresses]] **ServerDNSTestAddresses** __address__,__address__,__...__:: When we're detecting DNS hijacking, make sure that these __valid__ addresses aren't getting redirected. If they are, then our DNS is completely useless, - and we'll reset our exit policy to "reject *:*". This option only affects + and we'll reset our exit policy to "reject \*:*". This option only affects name lookups that your server does on behalf of clients. (Default: "www.google.com, www.mit.edu, www.yahoo.com, www.slashdot.org") @@ -1885,9 +1903,11 @@ is non-zero): (Default: 1) [[ExtendAllowPrivateAddresses]] **ExtendAllowPrivateAddresses** **0**|**1**:: - When this option is enabled, Tor routers allow EXTEND request to - localhost, RFC1918 addresses, and so on. This can create security issues; - you should probably leave it off. (Default: 0) + When this option is enabled, Tor will connect to localhost, RFC1918 + addresses, and so on. In particular, Tor will make direct connections, and + Tor routers allow EXTEND requests, to these private addresses. This can + create security issues; you should probably leave it off. + (Default: 0) [[MaxMemInQueues]] **MaxMemInQueues** __N__ **bytes**|**KB**|**MB**|**GB**:: This option configures a threshold above which Tor will assume that it @@ -1923,11 +1943,6 @@ if DirPort is non-zero): to set up a separate webserver. There's a sample disclaimer in contrib/operator-tools/tor-exit-notice.html. -[[HidServDirectoryV2]] **HidServDirectoryV2** **0**|**1**:: - When this option is set, Tor accepts and serves v2 hidden service - descriptors. Setting DirPort is not required for this, because clients - connect via the ORPort by default. (Default: 1) - [[DirPort]] **DirPort** \['address':]__PORT__|**auto** [_flags_]:: If this option is nonzero, advertise the directory service on this port. Set it to "auto" to have Tor pick a port for you. This option can occur @@ -1971,8 +1986,8 @@ on the public Tor network. [[V3AuthoritativeDirectory]] **V3AuthoritativeDirectory** **0**|**1**:: When this option is set in addition to **AuthoritativeDirectory**, Tor generates version 3 network statuses and serves descriptors, etc as - described in doc/spec/dir-spec.txt (for Tor clients and servers running at - least 0.2.0.x). + described in dir-spec.txt file of https://spec.torproject.org/[torspec] + (for Tor clients and servers running atleast 0.2.0.x). [[VersioningAuthoritativeDirectory]] **VersioningAuthoritativeDirectory** **0**|**1**:: When this option is set to 1, Tor adds information on which versions of @@ -1988,7 +2003,7 @@ on the public Tor network. multiple times: the values from multiple lines are spliced together. When this is set then **VersioningAuthoritativeDirectory** should be set too. -[[RecommendedPackageVersions]] **RecommendedPackageVersions** __PACKAGENAME__ __VERSION__ __URL__ __DIGESTTYPE__**=**__DIGEST__ :: +[[RecommendedPackages]] **RecommendedPackages** __PACKAGENAME__ __VERSION__ __URL__ __DIGESTTYPE__**=**__DIGEST__ :: Adds "package" line to the directory authority's vote. This information is used to vote on the correct URL and digest for the released versions of different Tor-related packages, so that the consensus can certify @@ -2139,11 +2154,6 @@ on the public Tor network. that fine-grained information about nodes can be discarded when it hasn't changed for a given amount of time. (Default: 24 hours) -[[VoteOnHidServDirectoriesV2]] **VoteOnHidServDirectoriesV2** **0**|**1**:: - When this option is set in addition to **AuthoritativeDirectory**, Tor - votes on whether to accept relays as hidden service directories. - (Default: 1) - [[AuthDirHasIPv6Connectivity]] **AuthDirHasIPv6Connectivity** **0**|**1**:: Authoritative directories only. When set to 0, OR ports with an IPv6 address are being accepted without reachability testing. @@ -2407,7 +2417,7 @@ The following options are used for running a testing Tor network. information on how to specify nodes. + In order for this option to have any effect, **TestingTorNetwork** - and **VoteOnHidServDirectoriesV2** both have to be set. + must be set. [[TestingDirAuthVoteHSDirIsStrict]] **TestingDirAuthVoteHSDirIsStrict** **0**|**1** :: If True (1), a node will never receive the HSDir flag unless it is specified @@ -2436,7 +2446,7 @@ The following options are used for running a testing Tor network. authority on a testing network. Overrides the usual default lower bound of 4 KB. (Default: 0) -[[TestingLinkCertLifetime]] **TestingLinkCertifetime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**|**months**:: +[[TestingLinkCertLifetime]] **TestingLinkCertLifetime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**|**months**:: Overrides the default lifetime for the certificates used to authenticate our X509 link cert with our ed25519 signing key. (Default: 2 days) @@ -2446,8 +2456,10 @@ The following options are used for running a testing Tor network. key. (Default: 2 days) -[[TestingLinkKeySlop]] **TestingLinkKeySlop** __N__ **seconds**|**minutes**|**hours**:: -[[TestingAuthKeySlop]] **TestingAuthKeySlop** __N__ **seconds**|**minutes**|**hours**:: +[[TestingLinkKeySlop]] **TestingLinkKeySlop** __N__ **seconds**|**minutes**|**hours** + + +[[TestingAuthKeySlop]] **TestingAuthKeySlop** __N__ **seconds**|**minutes**|**hours** + + [[TestingSigningKeySlop]] **TestingSigningKeySlop** __N__ **seconds**|**minutes**|**hours**:: How early before the official expiration of a an Ed25519 signing key do we replace it and issue a new key? @@ -2548,8 +2560,8 @@ __DataDirectory__**/bw_accounting**:: __DataDirectory__**/control_auth_cookie**:: Used for cookie authentication with the controller. Location can be overridden by the CookieAuthFile config option. Regenerated on startup. See - control-spec.txt for details. Only used when cookie authentication is - enabled. + control-spec.txt in https://spec.torproject.org/[torspec] for details. + Only used when cookie authentication is enabled. __DataDirectory__**/lock**:: This file is used to prevent two Tor instances from using same data @@ -2636,11 +2648,12 @@ SEE ALSO **https://www.torproject.org/** +**torspec: https://spec.torproject.org ** BUGS ---- -Plenty, probably. Tor is still in development. Please report them. +Plenty, probably. Tor is still in development. Please report them at https://trac.torproject.org/. AUTHORS ------- diff --git a/scripts/codegen/get_mozilla_ciphers.py b/scripts/codegen/get_mozilla_ciphers.py index e0a662bea0..e673ec7dc6 100644 --- a/scripts/codegen/get_mozilla_ciphers.py +++ b/scripts/codegen/get_mozilla_ciphers.py @@ -29,7 +29,7 @@ def ossl(s): ##### # Read the cpp file to understand what Ciphers map to what name : # Make "ciphers" a map from name used in the javascript to a cipher macro name -fileA = open(ff('security/manager/ssl/src/nsNSSComponent.cpp'),'r') +fileA = open(ff('security/manager/ssl/nsNSSComponent.cpp'),'r') # The input format is a file containing exactly one section of the form: # static CipherPref CipherPrefs[] = { @@ -71,7 +71,7 @@ for line in cipherLines: assert not key_pending key_pending = m.group(1) continue - m = re.search(r'^\s*(\S+)(?:,\s*(true|false))?\s*}', line) + m = re.search(r'^\s*(\S+)(?:,\s*(true|false))+\s*}', line) if m: assert key_pending key = key_pending @@ -107,7 +107,7 @@ fileC.close() # Build a map enabled_ciphers from javascript name to "true" or "false", # and an (unordered!) list of the macro names for those ciphers that are # enabled. -fileB = open(ff('netwerk/base/public/security-prefs.js'), 'r') +fileB = open(ff('netwerk/base/security-prefs.js'), 'r') enabled_ciphers = {} for line in fileB: diff --git a/src/common/address.c b/src/common/address.c index cfa8fd1dca..627764153b 100644 --- a/src/common/address.c +++ b/src/common/address.c @@ -1469,7 +1469,8 @@ get_interface_addresses_ioctl(int severity) int fd; smartlist_t *result = NULL; - /* This interface, AFAICT, only supports AF_INET addresses */ + /* This interface, AFAICT, only supports AF_INET addresses, + * except on AIX. For Solaris, we could use SIOCGLIFCONF. */ fd = socket(AF_INET, SOCK_DGRAM, 0); if (fd < 0) { tor_log(severity, LD_NET, "socket failed: %s", strerror(errno)); @@ -1527,7 +1528,7 @@ get_interface_addresses_raw,(int severity)) } /** Return true iff <b>a</b> is a multicast address. */ -STATIC int +int tor_addr_is_multicast(const tor_addr_t *a) { sa_family_t family = tor_addr_family(a); diff --git a/src/common/address.h b/src/common/address.h index d2841e1c9d..684ba65c4f 100644 --- a/src/common/address.h +++ b/src/common/address.h @@ -73,13 +73,13 @@ typedef struct tor_addr_port_t #define TOR_ADDR_NULL {AF_UNSPEC, {0}} -static INLINE const struct in6_addr *tor_addr_to_in6(const tor_addr_t *a); -static INLINE uint32_t tor_addr_to_ipv4n(const tor_addr_t *a); -static INLINE uint32_t tor_addr_to_ipv4h(const tor_addr_t *a); -static INLINE uint32_t tor_addr_to_mapped_ipv4h(const tor_addr_t *a); -static INLINE sa_family_t tor_addr_family(const tor_addr_t *a); -static INLINE const struct in_addr *tor_addr_to_in(const tor_addr_t *a); -static INLINE int tor_addr_eq_ipv4h(const tor_addr_t *a, uint32_t u); +static inline const struct in6_addr *tor_addr_to_in6(const tor_addr_t *a); +static inline uint32_t tor_addr_to_ipv4n(const tor_addr_t *a); +static inline uint32_t tor_addr_to_ipv4h(const tor_addr_t *a); +static inline uint32_t tor_addr_to_mapped_ipv4h(const tor_addr_t *a); +static inline sa_family_t tor_addr_family(const tor_addr_t *a); +static inline const struct in_addr *tor_addr_to_in(const tor_addr_t *a); +static inline int tor_addr_eq_ipv4h(const tor_addr_t *a, uint32_t u); socklen_t tor_addr_to_sockaddr(const tor_addr_t *a, uint16_t port, struct sockaddr *sa_out, socklen_t len); @@ -91,7 +91,7 @@ char *tor_sockaddr_to_str(const struct sockaddr *sa); /** Return an in6_addr* equivalent to <b>a</b>, or NULL if <b>a</b> is not * an IPv6 address. */ -static INLINE const struct in6_addr * +static inline const struct in6_addr * tor_addr_to_in6(const tor_addr_t *a) { return a->family == AF_INET6 ? &a->addr.in6_addr : NULL; @@ -115,14 +115,14 @@ tor_addr_to_in6(const tor_addr_t *a) /** Return an IPv4 address in network order for <b>a</b>, or 0 if * <b>a</b> is not an IPv4 address. */ -static INLINE uint32_t +static inline uint32_t tor_addr_to_ipv4n(const tor_addr_t *a) { return a->family == AF_INET ? a->addr.in_addr.s_addr : 0; } /** Return an IPv4 address in host order for <b>a</b>, or 0 if * <b>a</b> is not an IPv4 address. */ -static INLINE uint32_t +static inline uint32_t tor_addr_to_ipv4h(const tor_addr_t *a) { return ntohl(tor_addr_to_ipv4n(a)); @@ -131,7 +131,7 @@ tor_addr_to_ipv4h(const tor_addr_t *a) * 0 if <b>a</b> is not an IPv6 address. * * (Does not check whether the address is really a mapped address */ -static INLINE uint32_t +static inline uint32_t tor_addr_to_mapped_ipv4h(const tor_addr_t *a) { if (a->family == AF_INET6) { @@ -149,21 +149,21 @@ tor_addr_to_mapped_ipv4h(const tor_addr_t *a) } /** Return the address family of <b>a</b>. Possible values are: * AF_INET6, AF_INET, AF_UNSPEC. */ -static INLINE sa_family_t +static inline sa_family_t tor_addr_family(const tor_addr_t *a) { return a->family; } /** Return an in_addr* equivalent to <b>a</b>, or NULL if <b>a</b> is not * an IPv4 address. */ -static INLINE const struct in_addr * +static inline const struct in_addr * tor_addr_to_in(const tor_addr_t *a) { return a->family == AF_INET ? &a->addr.in_addr : NULL; } /** Return true iff <b>a</b> is an IPv4 address equal to the host-ordered * address in <b>u</b>. */ -static INLINE int +static inline int tor_addr_eq_ipv4h(const tor_addr_t *a, uint32_t u) { return a->family == AF_INET ? (tor_addr_to_ipv4h(a) == u) : 0; @@ -221,6 +221,7 @@ int tor_addr_is_internal_(const tor_addr_t *ip, int for_listening, const char *filename, int lineno); #define tor_addr_is_internal(addr, for_listening) \ tor_addr_is_internal_((addr), (for_listening), SHORT_FILE__, __LINE__) +int tor_addr_is_multicast(const tor_addr_t *a); /** Longest length that can be required for a reverse lookup name. */ /* 32 nybbles, 32 dots, 8 characters of "ip6.arpa", 1 NUL: 73 characters. */ @@ -288,7 +289,7 @@ char *tor_dup_ip(uint32_t addr) ATTR_MALLOC; MOCK_DECL(int,get_interface_address,(int severity, uint32_t *addr)); /** Free a smartlist of IP addresses returned by get_interface_address_list. */ -static INLINE void +static inline void free_interface_address_list(smartlist_t *addrs) { free_interface_address6_list(addrs); @@ -301,7 +302,7 @@ free_interface_address_list(smartlist_t *addrs) * Returns NULL on failure. * Use free_interface_address_list to free the returned list. */ -static INLINE smartlist_t * +static inline smartlist_t * get_interface_address_list(int severity, int include_internal) { return get_interface_address6_list(severity, AF_INET, include_internal); @@ -311,7 +312,6 @@ tor_addr_port_t *tor_addr_port_new(const tor_addr_t *addr, uint16_t port); #ifdef ADDRESS_PRIVATE MOCK_DECL(smartlist_t *,get_interface_addresses_raw,(int severity)); -STATIC int tor_addr_is_multicast(const tor_addr_t *a); MOCK_DECL(int,get_interface_address6_via_udp_socket_hack,(int severity, sa_family_t family, tor_addr_t *addr)); diff --git a/src/common/aes.c b/src/common/aes.c index 5f2c3f2f03..7b6cc39b20 100644 --- a/src/common/aes.c +++ b/src/common/aes.c @@ -271,7 +271,7 @@ evaluate_ctr_for_aes(void) * Helper function: set <b>cipher</b>'s internal buffer to the encrypted * value of the current counter. */ -static INLINE void +static inline void aes_fill_buf_(aes_cnt_cipher_t *cipher) { /* We don't currently use OpenSSL's counter mode implementation because: diff --git a/src/common/compat.c b/src/common/compat.c index 7d72b4b7fd..a103e58163 100644 --- a/src/common/compat.c +++ b/src/common/compat.c @@ -1078,7 +1078,7 @@ static int n_sockets_open = 0; static tor_mutex_t *socket_accounting_mutex = NULL; /** Helper: acquire the socket accounting lock. */ -static INLINE void +static inline void socket_accounting_lock(void) { if (PREDICT_UNLIKELY(!socket_accounting_mutex)) @@ -1087,7 +1087,7 @@ socket_accounting_lock(void) } /** Helper: release the socket accounting lock. */ -static INLINE void +static inline void socket_accounting_unlock(void) { tor_mutex_release(socket_accounting_mutex); @@ -1163,7 +1163,7 @@ tor_close_socket(tor_socket_t s) #ifdef DEBUG_SOCKET_COUNTING /** Helper: if DEBUG_SOCKET_COUNTING is enabled, remember that <b>s</b> is * now an open socket. */ -static INLINE void +static inline void mark_socket_open(tor_socket_t s) { /* XXXX This bitarray business will NOT work on windows: sockets aren't @@ -1486,6 +1486,20 @@ tor_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) } #ifdef NEED_ERSATZ_SOCKETPAIR + +static inline socklen_t +SIZEOF_SOCKADDR(int domain) +{ + switch (domain) { + case AF_INET: + return sizeof(struct sockaddr_in); + case AF_INET6: + return sizeof(struct sockaddr_in6); + default: + return 0; + } +} + /** * Helper used to implement socketpair on systems that lack it, by * making a direct connection to localhost. @@ -1501,13 +1515,21 @@ tor_ersatz_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) tor_socket_t listener = TOR_INVALID_SOCKET; tor_socket_t connector = TOR_INVALID_SOCKET; tor_socket_t acceptor = TOR_INVALID_SOCKET; - struct sockaddr_in listen_addr; - struct sockaddr_in connect_addr; + tor_addr_t listen_tor_addr; + struct sockaddr_storage connect_addr_ss, listen_addr_ss; + struct sockaddr *listen_addr = (struct sockaddr *) &listen_addr_ss; + uint16_t listen_port = 0; + tor_addr_t connect_tor_addr; + uint16_t connect_port = 0; + struct sockaddr *connect_addr = (struct sockaddr *) &connect_addr_ss; socklen_t size; int saved_errno = -1; + int ersatz_domain = AF_INET; - memset(&connect_addr, 0, sizeof(connect_addr)); - memset(&listen_addr, 0, sizeof(listen_addr)); + memset(&connect_tor_addr, 0, sizeof(connect_tor_addr)); + memset(&connect_addr_ss, 0, sizeof(connect_addr_ss)); + memset(&listen_tor_addr, 0, sizeof(listen_tor_addr)); + memset(&listen_addr_ss, 0, sizeof(listen_addr_ss)); if (protocol #ifdef AF_UNIX @@ -1524,47 +1546,71 @@ tor_ersatz_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) return -EINVAL; } - listener = tor_open_socket(AF_INET, type, 0); - if (!SOCKET_OK(listener)) - return -tor_socket_errno(-1); - memset(&listen_addr, 0, sizeof(listen_addr)); - listen_addr.sin_family = AF_INET; - listen_addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); - listen_addr.sin_port = 0; /* kernel chooses port. */ - if (bind(listener, (struct sockaddr *) &listen_addr, sizeof (listen_addr)) - == -1) + listener = tor_open_socket(ersatz_domain, type, 0); + if (!SOCKET_OK(listener)) { + int first_errno = tor_socket_errno(-1); + if (first_errno == SOCK_ERRNO(EPROTONOSUPPORT) + && ersatz_domain == AF_INET) { + /* Assume we're on an IPv6-only system */ + ersatz_domain = AF_INET6; + listener = tor_open_socket(ersatz_domain, type, 0); + if (!SOCKET_OK(listener)) { + /* Keep the previous behaviour, which was to return the IPv4 error. + * (This may be less informative on IPv6-only systems.) + * XX/teor - is there a better way to decide which errno to return? + * (I doubt we care much either way, once there is an error.) + */ + return -first_errno; + } + } + } + /* If there is no 127.0.0.1 or ::1, this will and must fail. Otherwise, we + * risk exposing a socketpair on a routable IP address. (Some BSD jails + * use a routable address for localhost. Fortunately, they have the real + * AF_UNIX socketpair.) */ + if (ersatz_domain == AF_INET) { + tor_addr_from_ipv4h(&listen_tor_addr, INADDR_LOOPBACK); + } else { + tor_addr_parse(&listen_tor_addr, "[::1]"); + } + tor_assert(tor_addr_is_loopback(&listen_tor_addr)); + size = tor_addr_to_sockaddr(&listen_tor_addr, + 0 /* kernel chooses port. */, + listen_addr, + sizeof(listen_addr_ss)); + if (bind(listener, listen_addr, size) == -1) goto tidy_up_and_fail; if (listen(listener, 1) == -1) goto tidy_up_and_fail; - connector = tor_open_socket(AF_INET, type, 0); + connector = tor_open_socket(ersatz_domain, type, 0); if (!SOCKET_OK(connector)) goto tidy_up_and_fail; /* We want to find out the port number to connect to. */ - size = sizeof(connect_addr); - if (getsockname(listener, (struct sockaddr *) &connect_addr, &size) == -1) + size = sizeof(connect_addr_ss); + if (getsockname(listener, connect_addr, &size) == -1) goto tidy_up_and_fail; - if (size != sizeof (connect_addr)) + if (size != SIZEOF_SOCKADDR (connect_addr->sa_family)) goto abort_tidy_up_and_fail; - if (connect(connector, (struct sockaddr *) &connect_addr, - sizeof(connect_addr)) == -1) + if (connect(connector, connect_addr, size) == -1) goto tidy_up_and_fail; - size = sizeof(listen_addr); - acceptor = tor_accept_socket(listener, - (struct sockaddr *) &listen_addr, &size); + size = sizeof(listen_addr_ss); + acceptor = tor_accept_socket(listener, listen_addr, &size); if (!SOCKET_OK(acceptor)) goto tidy_up_and_fail; - if (size != sizeof(listen_addr)) + if (size != SIZEOF_SOCKADDR(listen_addr->sa_family)) goto abort_tidy_up_and_fail; /* Now check we are talking to ourself by matching port and host on the two sockets. */ - if (getsockname(connector, (struct sockaddr *) &connect_addr, &size) == -1) + if (getsockname(connector, connect_addr, &size) == -1) goto tidy_up_and_fail; - if (size != sizeof (connect_addr) - || listen_addr.sin_family != connect_addr.sin_family - || listen_addr.sin_addr.s_addr != connect_addr.sin_addr.s_addr - || listen_addr.sin_port != connect_addr.sin_port) { + /* Set *_tor_addr and *_port to the address and port that was used */ + tor_addr_from_sockaddr(&listen_tor_addr, listen_addr, &listen_port); + tor_addr_from_sockaddr(&connect_tor_addr, connect_addr, &connect_port); + if (size != SIZEOF_SOCKADDR (connect_addr->sa_family) + || tor_addr_compare(&listen_tor_addr, &connect_tor_addr, CMP_SEMANTIC) + || listen_port != connect_port) { goto abort_tidy_up_and_fail; } tor_close_socket(listener); @@ -1590,6 +1636,9 @@ tor_ersatz_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) tor_close_socket(acceptor); return -saved_errno; } + +#undef SIZEOF_SOCKADDR + #endif /* Return the maximum number of allowed sockets. */ diff --git a/src/common/compat.h b/src/common/compat.h index c7c468c754..fb177832af 100644 --- a/src/common/compat.h +++ b/src/common/compat.h @@ -75,9 +75,7 @@ /* inline is __inline on windows. */ #ifdef _WIN32 -#define INLINE __inline -#else -#define INLINE inline +#define inline __inline #endif /* Try to get a reasonable __func__ substitute in place. */ @@ -118,6 +116,7 @@ #define ATTR_CONST __attribute__((const)) #define ATTR_MALLOC __attribute__((malloc)) #define ATTR_NORETURN __attribute__((noreturn)) +#define ATTR_WUR __attribute__((warn_unused_result)) /* Alas, nonnull is not at present a good idea for us. We'd like to get * warnings when we pass NULL where we shouldn't (which nonnull does, albeit * spottily), but we don't want to tell the compiler to make optimizations @@ -153,6 +152,7 @@ #define ATTR_NORETURN #define ATTR_NONNULL(x) #define ATTR_UNUSED +#define ATTR_WUR #define PREDICT_LIKELY(exp) (exp) #define PREDICT_UNLIKELY(exp) (exp) #endif @@ -288,7 +288,7 @@ const void *tor_memmem(const void *haystack, size_t hlen, const void *needle, size_t nlen) ATTR_NONNULL((1,3)); static const void *tor_memstr(const void *haystack, size_t hlen, const char *needle) ATTR_NONNULL((1,3)); -static INLINE const void * +static inline const void * tor_memstr(const void *haystack, size_t hlen, const char *needle) { return tor_memmem(haystack, hlen, needle, strlen(needle)); @@ -299,7 +299,7 @@ tor_memstr(const void *haystack, size_t hlen, const char *needle) #define DECLARE_CTYPE_FN(name) \ static int TOR_##name(char c); \ extern const uint32_t TOR_##name##_TABLE[]; \ - static INLINE int TOR_##name(char c) { \ + static inline int TOR_##name(char c) { \ uint8_t u = c; \ return !!(TOR_##name##_TABLE[(u >> 5) & 7] & (1u << (u & 31))); \ } @@ -613,7 +613,7 @@ void set_uint64(void *cp, uint64_t v) ATTR_NONNULL((1)); /* These uint8 variants are defined to make the code more uniform. */ #define get_uint8(cp) (*(const uint8_t*)(cp)) static void set_uint8(void *cp, uint8_t v); -static INLINE void +static inline void set_uint8(void *cp, uint8_t v) { *(uint8_t*)cp = v; diff --git a/src/common/compat_libevent.c b/src/common/compat_libevent.c index a366b6c9c6..29e5c5f63c 100644 --- a/src/common/compat_libevent.c +++ b/src/common/compat_libevent.c @@ -11,6 +11,7 @@ #include "orconfig.h" #include "compat.h" +#define COMPAT_LIBEVENT_PRIVATE #include "compat_libevent.h" #include "crypto.h" @@ -28,39 +29,11 @@ #include <event.h> #endif -/** A number representing a version of Libevent. - - This is a 4-byte number, with the first three bytes representing the - major, minor, and patchlevel respectively of the library. The fourth - byte is unused. - - This is equivalent to the format of LIBEVENT_VERSION_NUMBER on Libevent - 2.0.1 or later. For versions of Libevent before 1.4.0, which followed the - format of "1.0, 1.0a, 1.0b", we define 1.0 to be equivalent to 1.0.0, 1.0a - to be equivalent to 1.0.1, and so on. -*/ -typedef uint32_t le_version_t; - -/** @{ */ -/** Macros: returns the number of a libevent version as a le_version_t */ -#define V(major, minor, patch) \ - (((major) << 24) | ((minor) << 16) | ((patch) << 8)) -#define V_OLD(major, minor, patch) \ - V((major), (minor), (patch)-'a'+1) -/** @} */ - -/** Represetns a version of libevent so old we can't figure out what version - * it is. */ -#define LE_OLD V(0,0,0) -/** Represents a version of libevent so weird we can't figure out what version - * it is. */ -#define LE_OTHER V(0,0,99) - /** A string which, if it appears in a libevent log, should be ignored. */ static const char *suppress_msg = NULL; /** Callback function passed to event_set_log() so we can intercept * log messages from libevent. */ -static void +STATIC void libevent_logging_callback(int severity, const char *msg) { char buf[1024]; @@ -291,7 +264,7 @@ tor_libevent_get_method(void) /** Return the le_version_t for the version of libevent specified in the * string <b>v</b>. If the version is very new or uses an unrecognized * version, format, return LE_OTHER. */ -static le_version_t +STATIC le_version_t tor_decode_libevent_version(const char *v) { unsigned major, minor, patchlevel; @@ -322,7 +295,7 @@ tor_decode_libevent_version(const char *v) * Two different versions with different numbers are sure not to be binary * compatible. Two different versions with the same numbers have a decent * chance of binary compatibility.*/ -static int +STATIC int le_versions_compatibility(le_version_t v) { if (v == LE_OTHER) diff --git a/src/common/compat_libevent.h b/src/common/compat_libevent.h index 39181efb7b..8ee02c0b6d 100644 --- a/src/common/compat_libevent.h +++ b/src/common/compat_libevent.h @@ -91,5 +91,42 @@ void tor_gettimeofday_cache_set(const struct timeval *tv); #endif void tor_gettimeofday_cached_monotonic(struct timeval *tv); +#ifdef COMPAT_LIBEVENT_PRIVATE +/** A number representing a version of Libevent. + + This is a 4-byte number, with the first three bytes representing the + major, minor, and patchlevel respectively of the library. The fourth + byte is unused. + + This is equivalent to the format of LIBEVENT_VERSION_NUMBER on Libevent + 2.0.1 or later. For versions of Libevent before 1.4.0, which followed the + format of "1.0, 1.0a, 1.0b", we define 1.0 to be equivalent to 1.0.0, 1.0a + to be equivalent to 1.0.1, and so on. +*/ +typedef uint32_t le_version_t; + +/** @{ */ +/** Macros: returns the number of a libevent version as a le_version_t */ +#define V(major, minor, patch) \ + (((major) << 24) | ((minor) << 16) | ((patch) << 8)) +#define V_OLD(major, minor, patch) \ + V((major), (minor), (patch)-'a'+1) +/** @} */ + +/** Represetns a version of libevent so old we can't figure out what version + * it is. */ +#define LE_OLD V(0,0,0) +/** Represents a version of libevent so weird we can't figure out what version + * it is. */ +#define LE_OTHER V(0,0,99) + +STATIC void +libevent_logging_callback(int severity, const char *msg); +STATIC le_version_t +tor_decode_libevent_version(const char *v); +STATIC int +le_versions_compatibility(le_version_t v); +#endif + #endif diff --git a/src/common/compat_openssl.h b/src/common/compat_openssl.h new file mode 100644 index 0000000000..3fcd684c0c --- /dev/null +++ b/src/common/compat_openssl.h @@ -0,0 +1,37 @@ +/* Copyright (c) 2001, Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#ifndef TOR_COMPAT_OPENSSL_H +#define TOR_COMPAT_OPENSSL_H + +#include <openssl/opensslv.h> + +/** + * \file compat_openssl.h + * + * \brief compatability definitions for working with different openssl forks + **/ + +#if OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(1,0,0) +#error "We require OpenSSL >= 1.0.0" +#endif + +#if OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(1,1,0) +#define OPENSSL_VERSION SSLEAY_VERSION +#define OpenSSL_version(v) SSLeay_version(v) +#define OpenSSL_version_num() SSLeay() +#define RAND_OpenSSL() RAND_SSLeay() +#define STATE_IS_SW_SERVER_HELLO(st) \ + (((st) == SSL3_ST_SW_SRVR_HELLO_A) || \ + ((st) == SSL3_ST_SW_SRVR_HELLO_B)) +#define OSSL_HANDSHAKE_STATE int +#else +#define STATE_IS_SW_SERVER_HELLO(st) \ + ((st) == TLS_ST_SW_SRVR_HELLO) +#endif + +#endif + diff --git a/src/common/container.c b/src/common/container.c index 636dfb6c57..9f40dfa2e0 100644 --- a/src/common/container.c +++ b/src/common/container.c @@ -55,11 +55,12 @@ smartlist_free,(smartlist_t *sl)) void smartlist_clear(smartlist_t *sl) { + memset(sl->list, 0, sizeof(void *) * sl->num_used); sl->num_used = 0; } /** Make sure that <b>sl</b> can hold at least <b>size</b> entries. */ -static INLINE void +static inline void smartlist_ensure_capacity(smartlist_t *sl, int size) { #if SIZEOF_SIZE_T > SIZEOF_INT @@ -82,9 +83,11 @@ smartlist_ensure_capacity(smartlist_t *sl, int size) while (size > higher) higher *= 2; } - sl->capacity = higher; sl->list = tor_reallocarray(sl->list, sizeof(void *), - ((size_t)sl->capacity)); + ((size_t)higher)); + memset(sl->list + sl->capacity, 0, + sizeof(void *) * (higher - sl->capacity)); + sl->capacity = higher; } #undef ASSERT_CAPACITY #undef MAX_CAPACITY @@ -123,6 +126,7 @@ smartlist_remove(smartlist_t *sl, const void *element) if (sl->list[i] == element) { sl->list[i] = sl->list[--sl->num_used]; /* swap with the end */ i--; /* so we process the new i'th element */ + sl->list[sl->num_used] = NULL; } } @@ -132,9 +136,11 @@ void * smartlist_pop_last(smartlist_t *sl) { tor_assert(sl); - if (sl->num_used) - return sl->list[--sl->num_used]; - else + if (sl->num_used) { + void *tmp = sl->list[--sl->num_used]; + sl->list[sl->num_used] = NULL; + return tmp; + } else return NULL; } @@ -165,6 +171,7 @@ smartlist_string_remove(smartlist_t *sl, const char *element) tor_free(sl->list[i]); sl->list[i] = sl->list[--sl->num_used]; /* swap with the end */ i--; /* so we process the new i'th element */ + sl->list[sl->num_used] = NULL; } } } @@ -321,6 +328,7 @@ smartlist_intersect(smartlist_t *sl1, const smartlist_t *sl2) if (!smartlist_contains(sl2, sl1->list[i])) { sl1->list[i] = sl1->list[--sl1->num_used]; /* swap with the end */ i--; /* so we process the new i'th element */ + sl1->list[sl1->num_used] = NULL; } } @@ -345,6 +353,7 @@ smartlist_del(smartlist_t *sl, int idx) tor_assert(idx>=0); tor_assert(idx < sl->num_used); sl->list[idx] = sl->list[--sl->num_used]; + sl->list[sl->num_used] = NULL; } /** Remove the <b>idx</b>th element of sl; if idx is not the last element, @@ -360,6 +369,7 @@ smartlist_del_keeporder(smartlist_t *sl, int idx) --sl->num_used; if (idx < sl->num_used) memmove(sl->list+idx, sl->list+idx+1, sizeof(void*)*(sl->num_used-idx)); + sl->list[sl->num_used] = NULL; } /** Insert the value <b>val</b> as the new <b>idx</b>th element of @@ -857,7 +867,7 @@ smartlist_sort_pointers(smartlist_t *sl) /** Helper. <b>sl</b> may have at most one violation of the heap property: * the item at <b>idx</b> may be greater than one or both of its children. * Restore the heap property. */ -static INLINE void +static inline void smartlist_heapify(smartlist_t *sl, int (*compare)(const void *a, const void *b), int idx_field_offset, @@ -937,9 +947,11 @@ smartlist_pqueue_pop(smartlist_t *sl, *IDXP(top)=-1; if (--sl->num_used) { sl->list[0] = sl->list[sl->num_used]; + sl->list[sl->num_used] = NULL; UPDATE_IDX(0); smartlist_heapify(sl, compare, idx_field_offset, 0); } + sl->list[sl->num_used] = NULL; return top; } @@ -959,9 +971,11 @@ smartlist_pqueue_remove(smartlist_t *sl, --sl->num_used; *IDXP(item) = -1; if (idx == sl->num_used) { + sl->list[sl->num_used] = NULL; return; } else { sl->list[idx] = sl->list[sl->num_used]; + sl->list[sl->num_used] = NULL; UPDATE_IDX(idx); smartlist_heapify(sl, compare, idx_field_offset, idx); } @@ -1054,35 +1068,35 @@ DEFINE_MAP_STRUCTS(digestmap_t, char key[DIGEST_LEN], digestmap_); DEFINE_MAP_STRUCTS(digest256map_t, uint8_t key[DIGEST256_LEN], digest256map_); /** Helper: compare strmap_entry_t objects by key value. */ -static INLINE int +static inline int strmap_entries_eq(const strmap_entry_t *a, const strmap_entry_t *b) { return !strcmp(a->key, b->key); } /** Helper: return a hash value for a strmap_entry_t. */ -static INLINE unsigned int +static inline unsigned int strmap_entry_hash(const strmap_entry_t *a) { return (unsigned) siphash24g(a->key, strlen(a->key)); } /** Helper: compare digestmap_entry_t objects by key value. */ -static INLINE int +static inline int digestmap_entries_eq(const digestmap_entry_t *a, const digestmap_entry_t *b) { return tor_memeq(a->key, b->key, DIGEST_LEN); } /** Helper: return a hash value for a digest_map_t. */ -static INLINE unsigned int +static inline unsigned int digestmap_entry_hash(const digestmap_entry_t *a) { return (unsigned) siphash24g(a->key, DIGEST_LEN); } /** Helper: compare digestmap_entry_t objects by key value. */ -static INLINE int +static inline int digest256map_entries_eq(const digest256map_entry_t *a, const digest256map_entry_t *b) { @@ -1090,7 +1104,7 @@ digest256map_entries_eq(const digest256map_entry_t *a, } /** Helper: return a hash value for a digest_map_t. */ -static INLINE unsigned int +static inline unsigned int digest256map_entry_hash(const digest256map_entry_t *a) { return (unsigned) siphash24g(a->key, DIGEST256_LEN); @@ -1113,49 +1127,49 @@ HT_GENERATE2(digest256map_impl, digest256map_entry_t, node, digest256map_entry_hash, digest256map_entries_eq, 0.6, tor_reallocarray_, tor_free_) -static INLINE void +static inline void strmap_entry_free(strmap_entry_t *ent) { tor_free(ent->key); tor_free(ent); } -static INLINE void +static inline void digestmap_entry_free(digestmap_entry_t *ent) { tor_free(ent); } -static INLINE void +static inline void digest256map_entry_free(digest256map_entry_t *ent) { tor_free(ent); } -static INLINE void +static inline void strmap_assign_tmp_key(strmap_entry_t *ent, const char *key) { ent->key = (char*)key; } -static INLINE void +static inline void digestmap_assign_tmp_key(digestmap_entry_t *ent, const char *key) { memcpy(ent->key, key, DIGEST_LEN); } -static INLINE void +static inline void digest256map_assign_tmp_key(digest256map_entry_t *ent, const uint8_t *key) { memcpy(ent->key, key, DIGEST256_LEN); } -static INLINE void +static inline void strmap_assign_key(strmap_entry_t *ent, const char *key) { ent->key = tor_strdup(key); } -static INLINE void +static inline void digestmap_assign_key(digestmap_entry_t *ent, const char *key) { memcpy(ent->key, key, DIGEST_LEN); } -static INLINE void +static inline void digest256map_assign_key(digest256map_entry_t *ent, const uint8_t *key) { memcpy(ent->key, key, DIGEST256_LEN); diff --git a/src/common/container.h b/src/common/container.h index bf4f04762c..af7d5c37ce 100644 --- a/src/common/container.h +++ b/src/common/container.h @@ -53,21 +53,21 @@ void smartlist_subtract(smartlist_t *sl1, const smartlist_t *sl2); #ifdef DEBUG_SMARTLIST /** Return the number of items in sl. */ -static INLINE int smartlist_len(const smartlist_t *sl); -static INLINE int smartlist_len(const smartlist_t *sl) { +static inline int smartlist_len(const smartlist_t *sl); +static inline int smartlist_len(const smartlist_t *sl) { tor_assert(sl); return (sl)->num_used; } /** Return the <b>idx</b>th element of sl. */ -static INLINE void *smartlist_get(const smartlist_t *sl, int idx); -static INLINE void *smartlist_get(const smartlist_t *sl, int idx) { +static inline void *smartlist_get(const smartlist_t *sl, int idx); +static inline void *smartlist_get(const smartlist_t *sl, int idx) { tor_assert(sl); tor_assert(idx>=0); tor_assert(sl->num_used > idx); return sl->list[idx]; } -static INLINE void smartlist_set(smartlist_t *sl, int idx, void *val) { +static inline void smartlist_set(smartlist_t *sl, int idx, void *val) { tor_assert(sl); tor_assert(idx>=0); tor_assert(sl->num_used > idx); @@ -81,7 +81,7 @@ static INLINE void smartlist_set(smartlist_t *sl, int idx, void *val) { /** Exchange the elements at indices <b>idx1</b> and <b>idx2</b> of the * smartlist <b>sl</b>. */ -static INLINE void smartlist_swap(smartlist_t *sl, int idx1, int idx2) +static inline void smartlist_swap(smartlist_t *sl, int idx1, int idx2) { if (idx1 != idx2) { void *elt = smartlist_get(sl, idx1); @@ -500,64 +500,64 @@ void* strmap_remove_lc(strmap_t *map, const char *key); #define DECLARE_TYPED_DIGESTMAP_FNS(prefix, maptype, valtype) \ typedef struct maptype maptype; \ typedef struct prefix##iter_t *prefix##iter_t; \ - ATTR_UNUSED static INLINE maptype* \ + ATTR_UNUSED static inline maptype* \ prefix##new(void) \ { \ return (maptype*)digestmap_new(); \ } \ - ATTR_UNUSED static INLINE digestmap_t* \ + ATTR_UNUSED static inline digestmap_t* \ prefix##to_digestmap(maptype *map) \ { \ return (digestmap_t*)map; \ } \ - ATTR_UNUSED static INLINE valtype* \ + ATTR_UNUSED static inline valtype* \ prefix##get(maptype *map, const char *key) \ { \ return (valtype*)digestmap_get((digestmap_t*)map, key); \ } \ - ATTR_UNUSED static INLINE valtype* \ + ATTR_UNUSED static inline valtype* \ prefix##set(maptype *map, const char *key, valtype *val) \ { \ return (valtype*)digestmap_set((digestmap_t*)map, key, val); \ } \ - ATTR_UNUSED static INLINE valtype* \ + ATTR_UNUSED static inline valtype* \ prefix##remove(maptype *map, const char *key) \ { \ return (valtype*)digestmap_remove((digestmap_t*)map, key); \ } \ - ATTR_UNUSED static INLINE void \ + ATTR_UNUSED static inline void \ prefix##free(maptype *map, void (*free_val)(void*)) \ { \ digestmap_free((digestmap_t*)map, free_val); \ } \ - ATTR_UNUSED static INLINE int \ + ATTR_UNUSED static inline int \ prefix##isempty(maptype *map) \ { \ return digestmap_isempty((digestmap_t*)map); \ } \ - ATTR_UNUSED static INLINE int \ + ATTR_UNUSED static inline int \ prefix##size(maptype *map) \ { \ return digestmap_size((digestmap_t*)map); \ } \ - ATTR_UNUSED static INLINE \ + ATTR_UNUSED static inline \ prefix##iter_t *prefix##iter_init(maptype *map) \ { \ return (prefix##iter_t*) digestmap_iter_init((digestmap_t*)map); \ } \ - ATTR_UNUSED static INLINE \ + ATTR_UNUSED static inline \ prefix##iter_t *prefix##iter_next(maptype *map, prefix##iter_t *iter) \ { \ return (prefix##iter_t*) digestmap_iter_next( \ (digestmap_t*)map, (digestmap_iter_t*)iter); \ } \ - ATTR_UNUSED static INLINE prefix##iter_t* \ + ATTR_UNUSED static inline prefix##iter_t* \ prefix##iter_next_rmv(maptype *map, prefix##iter_t *iter) \ { \ return (prefix##iter_t*) digestmap_iter_next_rmv( \ (digestmap_t*)map, (digestmap_iter_t*)iter); \ } \ - ATTR_UNUSED static INLINE void \ + ATTR_UNUSED static inline void \ prefix##iter_get(prefix##iter_t *iter, \ const char **keyp, \ valtype **valp) \ @@ -566,7 +566,7 @@ void* strmap_remove_lc(strmap_t *map, const char *key); digestmap_iter_get((digestmap_iter_t*) iter, keyp, &v); \ *valp = v; \ } \ - ATTR_UNUSED static INLINE int \ + ATTR_UNUSED static inline int \ prefix##iter_done(prefix##iter_t *iter) \ { \ return digestmap_iter_done((digestmap_iter_t*)iter); \ @@ -584,7 +584,7 @@ void* strmap_remove_lc(strmap_t *map, const char *key); /** A random-access array of one-bit-wide elements. */ typedef unsigned int bitarray_t; /** Create a new bit array that can hold <b>n_bits</b> bits. */ -static INLINE bitarray_t * +static inline bitarray_t * bitarray_init_zero(unsigned int n_bits) { /* round up to the next int. */ @@ -594,7 +594,7 @@ bitarray_init_zero(unsigned int n_bits) /** Expand <b>ba</b> from holding <b>n_bits_old</b> to <b>n_bits_new</b>, * clearing all new bits. Returns a possibly changed pointer to the * bitarray. */ -static INLINE bitarray_t * +static inline bitarray_t * bitarray_expand(bitarray_t *ba, unsigned int n_bits_old, unsigned int n_bits_new) { @@ -611,26 +611,26 @@ bitarray_expand(bitarray_t *ba, return (bitarray_t*) ptr; } /** Free the bit array <b>ba</b>. */ -static INLINE void +static inline void bitarray_free(bitarray_t *ba) { tor_free(ba); } /** Set the <b>bit</b>th bit in <b>b</b> to 1. */ -static INLINE void +static inline void bitarray_set(bitarray_t *b, int bit) { b[bit >> BITARRAY_SHIFT] |= (1u << (bit & BITARRAY_MASK)); } /** Set the <b>bit</b>th bit in <b>b</b> to 0. */ -static INLINE void +static inline void bitarray_clear(bitarray_t *b, int bit) { b[bit >> BITARRAY_SHIFT] &= ~ (1u << (bit & BITARRAY_MASK)); } /** Return true iff <b>bit</b>th bit in <b>b</b> is nonzero. NOTE: does * not necessarily return 1 on true. */ -static INLINE unsigned int +static inline unsigned int bitarray_is_set(bitarray_t *b, int bit) { return b[bit >> BITARRAY_SHIFT] & (1u << (bit & BITARRAY_MASK)); @@ -645,7 +645,7 @@ typedef struct { #define BIT(n) ((n) & set->mask) /** Add the digest <b>digest</b> to <b>set</b>. */ -static INLINE void +static inline void digestset_add(digestset_t *set, const char *digest) { const uint64_t x = siphash24g(digest, 20); @@ -661,7 +661,7 @@ digestset_add(digestset_t *set, const char *digest) /** If <b>digest</b> is in <b>set</b>, return nonzero. Otherwise, * <em>probably</em> return zero. */ -static INLINE int +static inline int digestset_contains(const digestset_t *set, const char *digest) { const uint64_t x = siphash24g(digest, 20); @@ -689,33 +689,33 @@ double find_nth_double(double *array, int n_elements, int nth); int32_t find_nth_int32(int32_t *array, int n_elements, int nth); uint32_t find_nth_uint32(uint32_t *array, int n_elements, int nth); long find_nth_long(long *array, int n_elements, int nth); -static INLINE int +static inline int median_int(int *array, int n_elements) { return find_nth_int(array, n_elements, (n_elements-1)/2); } -static INLINE time_t +static inline time_t median_time(time_t *array, int n_elements) { return find_nth_time(array, n_elements, (n_elements-1)/2); } -static INLINE double +static inline double median_double(double *array, int n_elements) { return find_nth_double(array, n_elements, (n_elements-1)/2); } -static INLINE uint32_t +static inline uint32_t median_uint32(uint32_t *array, int n_elements) { return find_nth_uint32(array, n_elements, (n_elements-1)/2); } -static INLINE int32_t +static inline int32_t median_int32(int32_t *array, int n_elements) { return find_nth_int32(array, n_elements, (n_elements-1)/2); } -static INLINE uint32_t +static inline uint32_t third_quartile_uint32(uint32_t *array, int n_elements) { return find_nth_uint32(array, n_elements, (n_elements*3)/4); diff --git a/src/common/crypto.c b/src/common/crypto.c index 815c2ec0c5..7f0f842419 100644 --- a/src/common/crypto.c +++ b/src/common/crypto.c @@ -21,18 +21,13 @@ #undef OCSP_RESPONSE #endif -#include <openssl/opensslv.h> - #define CRYPTO_PRIVATE #include "crypto.h" +#include "compat_openssl.h" #include "crypto_curve25519.h" #include "crypto_ed25519.h" #include "crypto_format.h" -#if OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(1,0,0) -#error "We require OpenSSL >= 1.0.0" -#endif - #include <openssl/err.h> #include <openssl/rsa.h> #include <openssl/pem.h> @@ -48,6 +43,7 @@ #include <ctype.h> #endif #ifdef HAVE_UNISTD_H +#define _GNU_SOURCE #include <unistd.h> #endif #ifdef HAVE_FCNTL_H @@ -56,6 +52,9 @@ #ifdef HAVE_SYS_FCNTL_H #include <sys/fcntl.h> #endif +#ifdef HAVE_SYS_SYSCALL_H +#include <sys/syscall.h> +#endif #include "torlog.h" #include "aes.h" @@ -73,6 +72,9 @@ /** Longest recognized */ #define MAX_DNS_LABEL_SIZE 63 +/** Largest strong entropy request */ +#define MAX_STRONGEST_RAND_SIZE 256 + /** Macro: is k a valid RSA public or private key? */ #define PUBLIC_KEY_OK(k) ((k) && (k)->key && (k)->key->n) /** Macro: is k a valid RSA private key? */ @@ -110,7 +112,7 @@ static int tor_check_dh_key(int severity, BIGNUM *bn); /** Return the number of bytes added by padding method <b>padding</b>. */ -static INLINE int +static inline int crypto_get_rsa_padding_overhead(int padding) { switch (padding) @@ -122,7 +124,7 @@ crypto_get_rsa_padding_overhead(int padding) /** Given a padding method <b>padding</b>, return the correct OpenSSL constant. */ -static INLINE int +static inline int crypto_get_rsa_padding(int padding) { switch (padding) @@ -227,7 +229,7 @@ const char * crypto_openssl_get_version_str(void) { if (crypto_openssl_version_str == NULL) { - const char *raw_version = SSLeay_version(SSLEAY_VERSION); + const char *raw_version = OpenSSL_version(OPENSSL_VERSION); crypto_openssl_version_str = parse_openssl_version_str(raw_version); } return crypto_openssl_version_str; @@ -251,11 +253,13 @@ crypto_openssl_get_header_version_str(void) static int crypto_force_rand_ssleay(void) { - if (RAND_get_rand_method() != RAND_SSLeay()) { + RAND_METHOD *default_method; + default_method = RAND_OpenSSL(); + if (RAND_get_rand_method() != default_method) { log_notice(LD_CRYPTO, "It appears that one of our engines has provided " "a replacement the OpenSSL RNG. Resetting it to the default " "implementation."); - RAND_set_rand_method(RAND_SSLeay()); + RAND_set_rand_method(default_method); return 1; } return 0; @@ -270,8 +274,7 @@ crypto_init_siphash_key(void) if (have_seeded_siphash) return 0; - if (crypto_rand((char*) &key, sizeof(key)) < 0) - return -1; + crypto_rand((char*) &key, sizeof(key)); siphash_set_global_key(&key); have_seeded_siphash = 1; return 0; @@ -291,16 +294,18 @@ crypto_early_init(void) setup_openssl_threading(); - if (SSLeay() == OPENSSL_VERSION_NUMBER && - !strcmp(SSLeay_version(SSLEAY_VERSION), OPENSSL_VERSION_TEXT)) { + unsigned long version_num = OpenSSL_version_num(); + const char *version_str = OpenSSL_version(OPENSSL_VERSION); + if (version_num == OPENSSL_VERSION_NUMBER && + !strcmp(version_str, OPENSSL_VERSION_TEXT)) { log_info(LD_CRYPTO, "OpenSSL version matches version from headers " - "(%lx: %s).", SSLeay(), SSLeay_version(SSLEAY_VERSION)); + "(%lx: %s).", version_num, version_str); } else { log_warn(LD_CRYPTO, "OpenSSL version from headers does not match the " "version we're running with. If you get weird crashes, that " "might be why. (Compiled with %lx: %s; running with %lx: %s).", (unsigned long)OPENSSL_VERSION_NUMBER, OPENSSL_VERSION_TEXT, - SSLeay(), SSLeay_version(SSLEAY_VERSION)); + version_num, version_str); } crypto_force_rand_ssleay(); @@ -322,7 +327,8 @@ int crypto_global_init(int useAccel, const char *accelName, const char *accelDir) { if (!crypto_global_initialized_) { - crypto_early_init(); + if (crypto_early_init() < 0) + return -1; crypto_global_initialized_ = 1; @@ -404,11 +410,7 @@ crypto_global_init(int useAccel, const char *accelName, const char *accelDir) void crypto_thread_cleanup(void) { -#if OPENSSL_VERSION_NUMBER >= OPENSSL_V_SERIES(1,1,0) ERR_remove_thread_state(NULL); -#else - ERR_remove_state(0); -#endif } /** used by tortls.c: wrap an RSA* in a crypto_pk_t. */ @@ -432,9 +434,10 @@ crypto_pk_get_rsa_(crypto_pk_t *env) } /** used by tortls.c: get an equivalent EVP_PKEY* for a crypto_pk_t. Iff - * private is set, include the private-key portion of the key. */ -EVP_PKEY * -crypto_pk_get_evp_pkey_(crypto_pk_t *env, int private) + * private is set, include the private-key portion of the key. Return a valid + * pointer on success, and NULL on failure. */ +MOCK_IMPL(EVP_PKEY *, + crypto_pk_get_evp_pkey_,(crypto_pk_t *env, int private)) { RSA *key = NULL; EVP_PKEY *pkey = NULL; @@ -470,8 +473,8 @@ crypto_dh_get_dh_(crypto_dh_t *dh) /** Allocate and return storage for a public key. The key itself will not yet * be set. */ -crypto_pk_t * -crypto_pk_new(void) +MOCK_IMPL(crypto_pk_t *, + crypto_pk_new,(void)) { RSA *rsa; @@ -553,8 +556,8 @@ crypto_cipher_free(crypto_cipher_t *env) /** Generate a <b>bits</b>-bit new public/private keypair in <b>env</b>. * Return 0 on success, -1 on failure. */ -int -crypto_pk_generate_key_with_bits(crypto_pk_t *env, int bits) +MOCK_IMPL(int, + crypto_pk_generate_key_with_bits,(crypto_pk_t *env, int bits)) { tor_assert(env); @@ -656,7 +659,8 @@ crypto_pk_read_private_key_from_filename(crypto_pk_t *env, return 0; } -/** Helper function to implement crypto_pk_write_*_key_to_string. */ +/** Helper function to implement crypto_pk_write_*_key_to_string. Return 0 on + * success, -1 on failure. */ static int crypto_pk_write_key_to_string_impl(crypto_pk_t *env, char **dest, size_t *len, int is_public) @@ -897,7 +901,8 @@ crypto_pk_dup_key(crypto_pk_t *env) return env; } -/** Make a real honest-to-goodness copy of <b>env</b>, and return it. */ +/** Make a real honest-to-goodness copy of <b>env</b>, and return it. + * Returns NULL on failure. */ crypto_pk_t * crypto_pk_copy_full(crypto_pk_t *env) { @@ -1189,7 +1194,8 @@ crypto_pk_public_hybrid_encrypt(crypto_pk_t *env, return -1; } -/** Invert crypto_pk_public_hybrid_encrypt. */ +/** Invert crypto_pk_public_hybrid_encrypt. Returns the number of bytes + * written on success, -1 on failure. */ int crypto_pk_private_hybrid_decrypt(crypto_pk_t *env, char *to, @@ -1332,7 +1338,7 @@ crypto_pk_get_all_digests(crypto_pk_t *pk, digests_t *digests_out) } /** Copy <b>in</b> to the <b>outlen</b>-byte buffer <b>out</b>, adding spaces - * every four spaces. */ + * every four characters. */ void crypto_add_spaces_to_fp(char *out, size_t outlen, const char *in) { @@ -1484,7 +1490,7 @@ crypto_cipher_get_key(crypto_cipher_t *env) /** Encrypt <b>fromlen</b> bytes from <b>from</b> using the cipher * <b>env</b>; on success, store the result to <b>to</b> and return 0. - * On failure, return -1. + * Does not check for failure. */ int crypto_cipher_encrypt(crypto_cipher_t *env, char *to, @@ -1503,7 +1509,7 @@ crypto_cipher_encrypt(crypto_cipher_t *env, char *to, /** Decrypt <b>fromlen</b> bytes from <b>from</b> using the cipher * <b>env</b>; on success, store the result to <b>to</b> and return 0. - * On failure, return -1. + * Does not check for failure. */ int crypto_cipher_decrypt(crypto_cipher_t *env, char *to, @@ -1519,7 +1525,7 @@ crypto_cipher_decrypt(crypto_cipher_t *env, char *to, } /** Encrypt <b>len</b> bytes on <b>from</b> using the cipher in <b>env</b>; - * on success, return 0. On failure, return -1. + * on success, return 0. Does not check for failure. */ int crypto_cipher_crypt_inplace(crypto_cipher_t *env, char *buf, size_t len) @@ -1591,7 +1597,7 @@ crypto_cipher_decrypt_with_iv(const char *key, /** Compute the SHA1 digest of the <b>len</b> bytes on data stored in * <b>m</b>. Write the DIGEST_LEN byte result into <b>digest</b>. - * Return 0 on success, -1 on failure. + * Return 0 on success, 1 on failure. */ int crypto_digest(char *digest, const char *m, size_t len) @@ -1603,7 +1609,7 @@ crypto_digest(char *digest, const char *m, size_t len) /** Compute a 256-bit digest of <b>len</b> bytes in data stored in <b>m</b>, * using the algorithm <b>algorithm</b>. Write the DIGEST_LEN256-byte result - * into <b>digest</b>. Return 0 on success, -1 on failure. */ + * into <b>digest</b>. Return 0 on success, 1 on failure. */ int crypto_digest256(char *digest, const char *m, size_t len, digest_algorithm_t algorithm) @@ -1614,6 +1620,19 @@ crypto_digest256(char *digest, const char *m, size_t len, return (SHA256((const unsigned char*)m,len,(unsigned char*)digest) == NULL); } +/** Compute a 512-bit digest of <b>len</b> bytes in data stored in <b>m</b>, + * using the algorithm <b>algorithm</b>. Write the DIGEST_LEN512-byte result + * into <b>digest</b>. Return 0 on success, 1 on failure. */ +int +crypto_digest512(char *digest, const char *m, size_t len, + digest_algorithm_t algorithm) +{ + tor_assert(m); + tor_assert(digest); + tor_assert(algorithm == DIGEST_SHA512); + return (SHA512((const unsigned char*)m,len,(unsigned char*)digest) == NULL); +} + /** Set the digests_t in <b>ds_out</b> to contain every digest on the * <b>len</b> bytes in <b>m</b> that we know how to compute. Return 0 on * success, -1 on failure. */ @@ -1626,8 +1645,18 @@ crypto_digest_all(digests_t *ds_out, const char *m, size_t len) if (crypto_digest(ds_out->d[DIGEST_SHA1], m, len) < 0) return -1; for (i = DIGEST_SHA256; i < N_DIGEST_ALGORITHMS; ++i) { - if (crypto_digest256(ds_out->d[i], m, len, i) < 0) - return -1; + switch (i) { + case DIGEST_SHA256: + if (crypto_digest256(ds_out->d[i], m, len, i) < 0) + return -1; + break; + case DIGEST_SHA512: + if (crypto_digest512(ds_out->d[i], m, len, i) < 0) + return -1; + break; + default: + return -1; + } } return 0; } @@ -1641,6 +1670,8 @@ crypto_digest_algorithm_get_name(digest_algorithm_t alg) return "sha1"; case DIGEST_SHA256: return "sha256"; + case DIGEST_SHA512: + return "sha512"; default: tor_fragile_assert(); return "??unknown_digest??"; @@ -1656,6 +1687,8 @@ crypto_digest_algorithm_parse_name(const char *name) return DIGEST_SHA1; else if (!strcmp(name, "sha256")) return DIGEST_SHA256; + else if (!strcmp(name, "sha512")) + return DIGEST_SHA512; else return -1; } @@ -1665,6 +1698,7 @@ struct crypto_digest_t { union { SHA_CTX sha1; /**< state for SHA1 */ SHA256_CTX sha2; /**< state for SHA256 */ + SHA512_CTX sha512; /**< state for SHA512 */ } d; /**< State for the digest we're using. Only one member of the * union is usable, depending on the value of <b>algorithm</b>. */ digest_algorithm_bitfield_t algorithm : 8; /**< Which algorithm is in use? */ @@ -1695,6 +1729,19 @@ crypto_digest256_new(digest_algorithm_t algorithm) return r; } +/** Allocate and return a new digest object to compute 512-bit digests + * using <b>algorithm</b>. */ +crypto_digest_t * +crypto_digest512_new(digest_algorithm_t algorithm) +{ + crypto_digest_t *r; + tor_assert(algorithm == DIGEST_SHA512); + r = tor_malloc(sizeof(crypto_digest_t)); + SHA512_Init(&r->d.sha512); + r->algorithm = algorithm; + return r; +} + /** Deallocate a digest object. */ void @@ -1726,6 +1773,9 @@ crypto_digest_add_bytes(crypto_digest_t *digest, const char *data, case DIGEST_SHA256: SHA256_Update(&digest->d.sha2, (void*)data, len); break; + case DIGEST_SHA512: + SHA512_Update(&digest->d.sha512, (void*)data, len); + break; default: tor_fragile_assert(); break; @@ -1734,13 +1784,13 @@ crypto_digest_add_bytes(crypto_digest_t *digest, const char *data, /** Compute the hash of the data that has been passed to the digest * object; write the first out_len bytes of the result to <b>out</b>. - * <b>out_len</b> must be \<= DIGEST256_LEN. + * <b>out_len</b> must be \<= DIGEST512_LEN. */ void crypto_digest_get_digest(crypto_digest_t *digest, char *out, size_t out_len) { - unsigned char r[DIGEST256_LEN]; + unsigned char r[DIGEST512_LEN]; crypto_digest_t tmpenv; tor_assert(digest); tor_assert(out); @@ -1755,6 +1805,10 @@ crypto_digest_get_digest(crypto_digest_t *digest, tor_assert(out_len <= DIGEST256_LEN); SHA256_Final(r, &tmpenv.d.sha2); break; + case DIGEST_SHA512: + tor_assert(out_len <= DIGEST512_LEN); + SHA512_Final(r, &tmpenv.d.sha512); + break; default: log_warn(LD_BUG, "Called with unknown algorithm %d", digest->algorithm); /* If fragile_assert is not enabled, then we should at least not @@ -1796,7 +1850,7 @@ crypto_digest_assign(crypto_digest_t *into, * at <b>digest_out</b> to the hash of the concatenation of those strings, * plus the optional string <b>append</b>, computed with the algorithm * <b>alg</b>. - * <b>out_len</b> must be \<= DIGEST256_LEN. */ + * <b>out_len</b> must be \<= DIGEST512_LEN. */ void crypto_digest_smartlist(char *digest_out, size_t len_out, const smartlist_t *lst, @@ -1811,7 +1865,7 @@ crypto_digest_smartlist(char *digest_out, size_t len_out, * optional string <b>prepend</b>, those strings, * and the optional string <b>append</b>, computed with the algorithm * <b>alg</b>. - * <b>out_len</b> must be \<= DIGEST256_LEN. */ + * <b>len_out</b> must be \<= DIGEST512_LEN. */ void crypto_digest_smartlist_prefix(char *digest_out, size_t len_out, const char *prepend, @@ -1819,11 +1873,25 @@ crypto_digest_smartlist_prefix(char *digest_out, size_t len_out, const char *append, digest_algorithm_t alg) { - crypto_digest_t *d; - if (alg == DIGEST_SHA1) - d = crypto_digest_new(); - else - d = crypto_digest256_new(alg); + crypto_digest_t *d = NULL; + switch (alg) { + case DIGEST_SHA1: + d = crypto_digest_new(); + break; + case DIGEST_SHA256: + d = crypto_digest256_new(alg); + break; + case DIGEST_SHA512: + d = crypto_digest512_new(alg); + break; + default: + log_warn(LD_BUG, "Called with unknown algorithm %d", alg); + /* If fragile_assert is not enabled, wipe output and return + * without running any calculations */ + memwipe(digest_out, 0xff, len_out); + tor_fragile_assert(); + goto free; + } if (prepend) crypto_digest_add_bytes(d, prepend, strlen(prepend)); SMARTLIST_FOREACH(lst, const char *, cp, @@ -1831,23 +1899,28 @@ crypto_digest_smartlist_prefix(char *digest_out, size_t len_out, if (append) crypto_digest_add_bytes(d, append, strlen(append)); crypto_digest_get_digest(d, digest_out, len_out); + + free: crypto_digest_free(d); } /** Compute the HMAC-SHA-256 of the <b>msg_len</b> bytes in <b>msg</b>, using * the <b>key</b> of length <b>key_len</b>. Store the DIGEST256_LEN-byte - * result in <b>hmac_out</b>. + * result in <b>hmac_out</b>. Asserts on failure. */ void crypto_hmac_sha256(char *hmac_out, const char *key, size_t key_len, const char *msg, size_t msg_len) { + unsigned char *rv = NULL; /* If we've got OpenSSL >=0.9.8 we can use its hmac implementation. */ tor_assert(key_len < INT_MAX); tor_assert(msg_len < INT_MAX); - HMAC(EVP_sha256(), key, (int)key_len, (unsigned char*)msg, (int)msg_len, - (unsigned char*)hmac_out, NULL); + tor_assert(hmac_out); + rv = HMAC(EVP_sha256(), key, (int)key_len, (unsigned char*)msg, (int)msg_len, + (unsigned char*)hmac_out, NULL); + tor_assert(rv); } /* DH */ @@ -1941,7 +2014,8 @@ init_dh_param(void) */ #define DH_PRIVATE_KEY_BITS 320 -/** Allocate and return a new DH object for a key exchange. +/** Allocate and return a new DH object for a key exchange. Returns NULL on + * failure. */ crypto_dh_t * crypto_dh_new(int dh_type) @@ -2164,7 +2238,7 @@ int crypto_expand_key_material_TAP(const uint8_t *key_in, size_t key_in_len, uint8_t *key_out, size_t key_out_len) { - int i; + int i, r = -1; uint8_t *cp, *tmp = tor_malloc(key_in_len+1); uint8_t digest[DIGEST_LEN]; @@ -2176,19 +2250,16 @@ crypto_expand_key_material_TAP(const uint8_t *key_in, size_t key_in_len, ++i, cp += DIGEST_LEN) { tmp[key_in_len] = i; if (crypto_digest((char*)digest, (const char *)tmp, key_in_len+1)) - goto err; + goto exit; memcpy(cp, digest, MIN(DIGEST_LEN, key_out_len-(cp-key_out))); } - memwipe(tmp, 0, key_in_len+1); - tor_free(tmp); - memwipe(digest, 0, sizeof(digest)); - return 0; - err: + r = 0; + exit: memwipe(tmp, 0, key_in_len+1); tor_free(tmp); memwipe(digest, 0, sizeof(digest)); - return -1; + return r; } /** Expand some secret key material according to RFC5869, using SHA256 as the @@ -2196,7 +2267,7 @@ crypto_expand_key_material_TAP(const uint8_t *key_in, size_t key_in_len, * secret key material; the <b>salt_in_len</b> bytes at <b>salt_in</b> and the * <b>info_in_len</b> bytes in <b>info_in_len</b> are the algorithm's "salt" * and "info" parameters respectively. On success, write <b>key_out_len</b> - * bytes to <b>key_out</b> and return 0. On failure, return -1. + * bytes to <b>key_out</b> and return 0. Assert on failure. */ int crypto_expand_key_material_rfc5869_sha256( @@ -2280,23 +2351,18 @@ crypto_seed_weak_rng(tor_weak_rng_t *rng) } /** Try to get <b>out_len</b> bytes of the strongest entropy we can generate, - * storing it into <b>out</b>. + * via system calls, storing it into <b>out</b>. Return 0 on success, -1 on + * failure. A maximum request size of 256 bytes is imposed. */ -int -crypto_strongest_rand(uint8_t *out, size_t out_len) +static int +crypto_strongest_rand_syscall(uint8_t *out, size_t out_len) { -#ifdef _WIN32 + tor_assert(out_len <= MAX_STRONGEST_RAND_SIZE); + +#if defined(_WIN32) static int provider_set = 0; static HCRYPTPROV provider; -#else - static const char *filenames[] = { - "/dev/srandom", "/dev/urandom", "/dev/random", NULL - }; - int fd, i; - size_t n; -#endif -#ifdef _WIN32 if (!provider_set) { if (!CryptAcquireContext(&provider, NULL, NULL, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT)) { @@ -2311,7 +2377,84 @@ crypto_strongest_rand(uint8_t *out, size_t out_len) } return 0; +#elif defined(__linux__) && defined(SYS_getrandom) + static int getrandom_works = 1; /* Be optimitic about our chances... */ + + /* getrandom() isn't as straight foward as getentropy(), and has + * no glibc wrapper. + * + * As far as I can tell from getrandom(2) and the source code, the + * requests we issue will always succeed (though it will block on the + * call if /dev/urandom isn't seeded yet), since we are NOT specifying + * GRND_NONBLOCK and the request is <= 256 bytes. + * + * The manpage is unclear on what happens if a signal interrupts the call + * while the request is blocked due to lack of entropy.... + * + * We optimistically assume that getrandom() is available and functional + * because it is the way of the future, and 2 branch mispredicts pale in + * comparision to the overheads involved with failing to open + * /dev/srandom followed by opening and reading from /dev/urandom. + */ + if (PREDICT_LIKELY(getrandom_works)) { + long ret; + /* A flag of '0' here means to read from '/dev/urandom', and to + * block if insufficient entropy is available to service the + * request. + */ + const unsigned int flags = 0; + do { + ret = syscall(SYS_getrandom, out, out_len, flags); + } while (ret == -1 && ((errno == EINTR) ||(errno == EAGAIN))); + + if (PREDICT_UNLIKELY(ret == -1)) { + tor_assert(errno != EAGAIN); + tor_assert(errno != EINTR); + + /* Probably ENOSYS. */ + log_warn(LD_CRYPTO, "Can't get entropy from getrandom()."); + getrandom_works = 0; /* Don't bother trying again. */ + return -1; + } + + tor_assert(ret == (long)out_len); + return 0; + } + + return -1; /* getrandom() previously failed unexpectedly. */ +#elif defined(HAVE_GETENTROPY) + /* getentropy() is what Linux's getrandom() wants to be when it grows up. + * the only gotcha is that requests are limited to 256 bytes. + */ + return getentropy(out, out_len); #else + (void) out; +#endif + + /* This platform doesn't have a supported syscall based random. */ + return -1; +} + +/** Try to get <b>out_len</b> bytes of the strongest entropy we can generate, + * via the per-platform fallback mechanism, storing it into <b>out</b>. + * Return 0 on success, -1 on failure. A maximum request size of 256 bytes + * is imposed. + */ +static int +crypto_strongest_rand_fallback(uint8_t *out, size_t out_len) +{ +#ifdef _WIN32 + /* Windows exclusively uses crypto_strongest_rand_syscall(). */ + (void)out; + (void)out_len; + return -1; +#else + static const char *filenames[] = { + "/dev/srandom", "/dev/urandom", "/dev/random", NULL + }; + int fd, i; + size_t n; + for (i = 0; filenames[i]; ++i) { log_debug(LD_FS, "Opening %s for entropy", filenames[i]); fd = open(sandbox_intern_string(filenames[i]), O_RDONLY, 0); @@ -2329,14 +2472,95 @@ crypto_strongest_rand(uint8_t *out, size_t out_len) return 0; } - log_warn(LD_CRYPTO, "Cannot get strong entropy: no entropy source found."); return -1; #endif } +/** Try to get <b>out_len</b> bytes of the strongest entropy we can generate, + * storing it into <b>out</b>. Return 0 on success, -1 on failure. A maximum + * request size of 256 bytes is imposed. + */ +static int +crypto_strongest_rand_raw(uint8_t *out, size_t out_len) +{ + static const size_t sanity_min_size = 16; + static const int max_attempts = 3; + tor_assert(out_len <= MAX_STRONGEST_RAND_SIZE); + + /* For buffers >= 16 bytes (128 bits), we sanity check the output by + * zero filling the buffer and ensuring that it actually was at least + * partially modified. + * + * Checking that any individual byte is non-zero seems like it would + * fail too often (p = out_len * 1/256) for comfort, but this is an + * "adjust according to taste" sort of check. + */ + memwipe(out, 0, out_len); + for (int i = 0; i < max_attempts; i++) { + /* Try to use the syscall/OS favored mechanism to get strong entropy. */ + if (crypto_strongest_rand_syscall(out, out_len) != 0) { + /* Try to use the less-favored mechanism to get strong entropy. */ + if (crypto_strongest_rand_fallback(out, out_len) != 0) { + /* Welp, we tried. Hopefully the calling code terminates the process + * since we're basically boned without good entropy. + */ + log_warn(LD_CRYPTO, + "Cannot get strong entropy: no entropy source found."); + return -1; + } + } + + if ((out_len < sanity_min_size) || !tor_mem_is_zero((char*)out, out_len)) + return 0; + } + + /* We tried max_attempts times to fill a buffer >= 128 bits long, + * and each time it returned all '0's. Either the system entropy + * source is busted, or the user should go out and buy a ticket to + * every lottery on the planet. + */ + log_warn(LD_CRYPTO, "Strong OS entropy returned all zero buffer."); + return -1; +} + +/** Try to get <b>out_len</b> bytes of the strongest entropy we can generate, + * storing it into <b>out</b>. + */ +void +crypto_strongest_rand(uint8_t *out, size_t out_len) +{ +#define DLEN SHA512_DIGEST_LENGTH + /* We're going to hash DLEN bytes from the system RNG together with some + * bytes from the openssl PRNG, in order to yield DLEN bytes. + */ + uint8_t inp[DLEN*2]; + uint8_t tmp[DLEN]; + tor_assert(out); + while (out_len) { + crypto_rand((char*) inp, DLEN); + if (crypto_strongest_rand_raw(inp+DLEN, DLEN) < 0) { + log_err(LD_CRYPTO, "Failed to load strong entropy when generating an " + "important key. Exiting."); + /* Die with an assertion so we get a stack trace. */ + tor_assert(0); + } + if (out_len >= DLEN) { + SHA512(inp, sizeof(inp), out); + out += DLEN; + out_len -= DLEN; + } else { + SHA512(inp, sizeof(inp), tmp); + memcpy(out, tmp, out_len); + break; + } + } + memwipe(tmp, 0, sizeof(tmp)); + memwipe(inp, 0, sizeof(inp)); +#undef DLEN +} + /** Seed OpenSSL's random number generator with bytes from the operating - * system. <b>startup</b> should be true iff we have just started Tor and - * have not yet allocated a bunch of fds. Return 0 on success, -1 on failure. + * system. Return 0 on success, -1 on failure. */ int crypto_seed_rng(void) @@ -2351,41 +2575,51 @@ crypto_seed_rng(void) if (rand_poll_ok == 0) log_warn(LD_CRYPTO, "RAND_poll() failed."); - load_entropy_ok = !crypto_strongest_rand(buf, sizeof(buf)); + load_entropy_ok = !crypto_strongest_rand_raw(buf, sizeof(buf)); if (load_entropy_ok) { RAND_seed(buf, sizeof(buf)); } memwipe(buf, 0, sizeof(buf)); - if (rand_poll_ok || load_entropy_ok) + if ((rand_poll_ok || load_entropy_ok) && RAND_status() == 1) return 0; else return -1; } -/** Write <b>n</b> bytes of strong random data to <b>to</b>. Return 0 on - * success, -1 on failure, with support for mocking for unit tests. +/** Write <b>n</b> bytes of strong random data to <b>to</b>. Supports mocking + * for unit tests. + * + * This function is not allowed to fail; if it would fail to generate strong + * entropy, it must terminate the process instead. */ -MOCK_IMPL(int, +MOCK_IMPL(void, crypto_rand, (char *to, size_t n)) { - return crypto_rand_unmocked(to, n); + crypto_rand_unmocked(to, n); } -/** Write <b>n</b> bytes of strong random data to <b>to</b>. Return 0 on - * success, -1 on failure. Most callers will want crypto_rand instead. +/** Write <b>n</b> bytes of strong random data to <b>to</b>. Most callers + * will want crypto_rand instead. + * + * This function is not allowed to fail; if it would fail to generate strong + * entropy, it must terminate the process instead. */ -int +void crypto_rand_unmocked(char *to, size_t n) { int r; + if (n == 0) + return; + tor_assert(n < INT_MAX); tor_assert(to); r = RAND_bytes((unsigned char*)to, (int)n); - if (r == 0) - crypto_log_errors(LOG_WARN, "generating random data"); - return (r == 1) ? 0 : -1; + /* We consider a PRNG failure non-survivable. Let's assert so that we get a + * stack trace about where it happened. + */ + tor_assert(r >= 0); } /** Return a pseudorandom integer, chosen uniformly from the values @@ -2411,8 +2645,8 @@ crypto_rand_int(unsigned int max) } } -/** Return a pseudorandom integer, chosen uniformly from the values <i>i</i> - * such that <b>min</b> <= <i>i</i> < <b>max</b>. +/** Return a pseudorandom integer, chosen uniformly from the values i such + * that min <= i < max. * * <b>min</b> MUST be in range [0, <b>max</b>). * <b>max</b> MUST be in range (min, INT_MAX]. @@ -2489,7 +2723,7 @@ crypto_rand_double(void) /** Generate and return a new random hostname starting with <b>prefix</b>, * ending with <b>suffix</b>, and containing no fewer than * <b>min_rand_len</b> and no more than <b>max_rand_len</b> random base32 - * characters between. + * characters. Does not check for failure. * * Clip <b>max_rand_len</b> to MAX_DNS_LABEL_SIZE. **/ @@ -2582,7 +2816,11 @@ memwipe(void *mem, uint8_t byte, size_t sz) * variable. It's an elaborate ruse to trick the compiler into not * optimizing out the "wipe this memory" code. Read it if you like zany * programming tricks! In later versions of Tor, we should look for better - * not-optimized-out memory wiping stuff. */ + * not-optimized-out memory wiping stuff... + * + * ...or maybe not. In practice, there are pure-asm implementations of + * OPENSSL_cleanse() on most platforms, which ought to do the job. + **/ OPENSSL_cleanse(mem, sz); /* Just in case some caller of memwipe() is relying on getting a buffer * filled with a particular value, fill the buffer. @@ -2671,7 +2909,7 @@ tor_set_openssl_thread_id(CRYPTO_THREADID *threadid) /** @{ */ /** Helper: Construct mutexes, and set callbacks to help OpenSSL handle being - * multithreaded. */ + * multithreaded. Returns 0. */ static int setup_openssl_threading(void) { @@ -2689,17 +2927,14 @@ setup_openssl_threading(void) return 0; } -/** Uninitialize the crypto library. Return 0 on success, -1 on failure. +/** Uninitialize the crypto library. Return 0 on success. Does not detect + * failure. */ int crypto_global_cleanup(void) { EVP_cleanup(); -#if OPENSSL_VERSION_NUMBER >= OPENSSL_V_SERIES(1,1,0) ERR_remove_thread_state(NULL); -#else - ERR_remove_state(0); -#endif ERR_free_strings(); if (dh_param_p) diff --git a/src/common/crypto.h b/src/common/crypto.h index 6256f7346b..0fba958f8d 100644 --- a/src/common/crypto.h +++ b/src/common/crypto.h @@ -16,6 +16,7 @@ #include <stdio.h> #include "torint.h" #include "testsupport.h" +#include "compat.h" /* Macro to create an arbitrary OpenSSL version number as used by @@ -54,6 +55,8 @@ /** Length of the output of our second (improved) message digests. (For now * this is just sha256, but it could be any other 256-bit digest.) */ #define DIGEST256_LEN 32 +/** Length of the output of our 64-bit optimized message digests (SHA512). */ +#define DIGEST512_LEN 64 /** Length of our symmetric cipher's keys. */ #define CIPHER_KEY_LEN 16 /** Length of our symmetric cipher's IV. */ @@ -69,6 +72,9 @@ /** Length of a sha256 message digest when encoded in base64 with trailing = * signs removed. */ #define BASE64_DIGEST256_LEN 43 +/** Length of a sha512 message digest when encoded in base64 with trailing = + * signs removed. */ +#define BASE64_DIGEST512_LEN 86 /** Constant used to indicate OAEP padding for public-key encryption */ #define PK_PKCS1_OAEP_PADDING 60002 @@ -83,24 +89,27 @@ #define HEX_DIGEST_LEN 40 /** Length of hex encoding of SHA256 digest, not including final NUL. */ #define HEX_DIGEST256_LEN 64 +/** Length of hex encoding of SHA512 digest, not including final NUL. */ +#define HEX_DIGEST512_LEN 128 typedef enum { DIGEST_SHA1 = 0, DIGEST_SHA256 = 1, + DIGEST_SHA512 = 2, } digest_algorithm_t; -#define N_DIGEST_ALGORITHMS (DIGEST_SHA256+1) +#define N_DIGEST_ALGORITHMS (DIGEST_SHA512+1) #define digest_algorithm_bitfield_t ENUM_BF(digest_algorithm_t) /** A set of all the digests we know how to compute, taken on a single - * string. Any digests that are shorter than 256 bits are right-padded + * string. Any digests that are shorter than 512 bits are right-padded * with 0 bits. * - * Note that this representation wastes 12 bytes for the SHA1 case, so + * Note that this representation wastes 44 bytes for the SHA1 case, so * don't use it for anything where we need to allocate a whole bunch at * once. **/ typedef struct { - char d[N_DIGEST_ALGORITHMS][DIGEST256_LEN]; + char d[N_DIGEST_ALGORITHMS][DIGEST512_LEN]; } digests_t; typedef struct crypto_pk_t crypto_pk_t; @@ -111,15 +120,15 @@ typedef struct crypto_dh_t crypto_dh_t; /* global state */ const char * crypto_openssl_get_version_str(void); const char * crypto_openssl_get_header_version_str(void); -int crypto_early_init(void); +int crypto_early_init(void) ATTR_WUR; int crypto_global_init(int hardwareAccel, const char *accelName, - const char *accelPath); + const char *accelPath) ATTR_WUR; void crypto_thread_cleanup(void); int crypto_global_cleanup(void); /* environment setup */ -crypto_pk_t *crypto_pk_new(void); +MOCK_DECL(crypto_pk_t *,crypto_pk_new,(void)); void crypto_pk_free(crypto_pk_t *env); void crypto_set_tls_dh_prime(void); @@ -128,7 +137,7 @@ crypto_cipher_t *crypto_cipher_new_with_iv(const char *key, const char *iv); void crypto_cipher_free(crypto_cipher_t *env); /* public key crypto */ -int crypto_pk_generate_key_with_bits(crypto_pk_t *env, int bits); +MOCK_DECL(int, crypto_pk_generate_key_with_bits,(crypto_pk_t *env, int bits)); #define crypto_pk_generate_key(env) \ crypto_pk_generate_key_with_bits((env), (PK_BYTES*8)) @@ -207,6 +216,8 @@ int crypto_cipher_decrypt_with_iv(const char *key, int crypto_digest(char *digest, const char *m, size_t len); int crypto_digest256(char *digest, const char *m, size_t len, digest_algorithm_t algorithm); +int crypto_digest512(char *digest, const char *m, size_t len, + digest_algorithm_t algorithm); int crypto_digest_all(digests_t *ds_out, const char *m, size_t len); struct smartlist_t; void crypto_digest_smartlist_prefix(char *digest_out, size_t len_out, @@ -221,6 +232,7 @@ const char *crypto_digest_algorithm_get_name(digest_algorithm_t alg); int crypto_digest_algorithm_parse_name(const char *name); crypto_digest_t *crypto_digest_new(void); crypto_digest_t *crypto_digest256_new(digest_algorithm_t algorithm); +crypto_digest_t *crypto_digest512_new(digest_algorithm_t algorithm); void crypto_digest_free(crypto_digest_t *digest); void crypto_digest_add_bytes(crypto_digest_t *digest, const char *data, size_t len); @@ -258,10 +270,10 @@ int crypto_expand_key_material_rfc5869_sha256( uint8_t *key_out, size_t key_out_len); /* random numbers */ -int crypto_seed_rng(void); -MOCK_DECL(int,crypto_rand,(char *to, size_t n)); -int crypto_rand_unmocked(char *to, size_t n); -int crypto_strongest_rand(uint8_t *out, size_t out_len); +int crypto_seed_rng(void) ATTR_WUR; +MOCK_DECL(void,crypto_rand,(char *to, size_t n)); +void crypto_rand_unmocked(char *to, size_t n); +void crypto_strongest_rand(uint8_t *out, size_t out_len); int crypto_rand_int(unsigned int max); int crypto_rand_int_range(unsigned int min, unsigned int max); uint64_t crypto_rand_uint64_range(uint64_t min, uint64_t max); @@ -289,8 +301,8 @@ struct evp_pkey_st; struct dh_st; struct rsa_st *crypto_pk_get_rsa_(crypto_pk_t *env); crypto_pk_t *crypto_new_pk_from_rsa_(struct rsa_st *rsa); -struct evp_pkey_st *crypto_pk_get_evp_pkey_(crypto_pk_t *env, - int private); +MOCK_DECL(struct evp_pkey_st *, crypto_pk_get_evp_pkey_,(crypto_pk_t *env, + int private)); struct dh_st *crypto_dh_get_dh_(crypto_dh_t *dh); void crypto_add_spaces_to_fp(char *out, size_t outlen, const char *in); diff --git a/src/common/crypto_curve25519.c b/src/common/crypto_curve25519.c index ac0b08a552..2002483265 100644 --- a/src/common/crypto_curve25519.c +++ b/src/common/crypto_curve25519.c @@ -111,19 +111,11 @@ curve25519_public_key_is_ok(const curve25519_public_key_t *key) int curve25519_rand_seckey_bytes(uint8_t *out, int extra_strong) { - uint8_t k_tmp[CURVE25519_SECKEY_LEN]; + if (extra_strong) + crypto_strongest_rand(out, CURVE25519_SECKEY_LEN); + else + crypto_rand((char*)out, CURVE25519_SECKEY_LEN); - if (crypto_rand((char*)out, CURVE25519_SECKEY_LEN) < 0) - return -1; - if (extra_strong && !crypto_strongest_rand(k_tmp, CURVE25519_SECKEY_LEN)) { - /* If they asked for extra-strong entropy and we have some, use it as an - * HMAC key to improve not-so-good entropy rather than using it directly, - * just in case the extra-strong entropy is less amazing than we hoped. */ - crypto_hmac_sha256((char*) out, - (const char *)k_tmp, sizeof(k_tmp), - (const char *)out, CURVE25519_SECKEY_LEN); - } - memwipe(k_tmp, 0, sizeof(k_tmp)); return 0; } diff --git a/src/common/crypto_ed25519.c b/src/common/crypto_ed25519.c index 1749efc34c..41ec486f0a 100644 --- a/src/common/crypto_ed25519.c +++ b/src/common/crypto_ed25519.c @@ -107,7 +107,9 @@ ed25519_secret_key_generate(ed25519_secret_key_t *seckey_out, { int r; uint8_t seed[32]; - if (! extra_strong || crypto_strongest_rand(seed, sizeof(seed)) < 0) + if (extra_strong) + crypto_strongest_rand(seed, sizeof(seed)); + else crypto_rand((char*)seed, sizeof(seed)); r = get_ed_impl()->seckey_expand(seckey_out->seckey, seed); diff --git a/src/common/include.am b/src/common/include.am index 7de93ba2ac..2fc92e2ceb 100644 --- a/src/common/include.am +++ b/src/common/include.am @@ -118,6 +118,7 @@ COMMONHEADERS = \ src/common/ciphers.inc \ src/common/compat.h \ src/common/compat_libevent.h \ + src/common/compat_openssl.h \ src/common/compat_threads.h \ src/common/container.h \ src/common/crypto.h \ diff --git a/src/common/log.c b/src/common/log.c index e23691b6ab..4a8a7b1165 100644 --- a/src/common/log.c +++ b/src/common/log.c @@ -64,7 +64,7 @@ typedef struct logfile_t { static void log_free(logfile_t *victim); /** Helper: map a log severity to descriptive string. */ -static INLINE const char * +static inline const char * sev_to_string(int severity) { switch (severity) { @@ -80,7 +80,7 @@ sev_to_string(int severity) } /** Helper: decide whether to include the function name in the log message. */ -static INLINE int +static inline int should_log_function_name(log_domain_mask_t domain, int severity) { switch (severity) { @@ -163,7 +163,7 @@ static void close_log(logfile_t *victim); static char *domain_to_string(log_domain_mask_t domain, char *buf, size_t buflen); -static INLINE char *format_msg(char *buf, size_t buf_len, +static inline char *format_msg(char *buf, size_t buf_len, log_domain_mask_t domain, int severity, const char *funcname, const char *suffix, const char *format, va_list ap, size_t *msg_len_out) @@ -199,7 +199,7 @@ set_log_time_granularity(int granularity_msec) /** Helper: Write the standard prefix for log lines to a * <b>buf_len</b> character buffer in <b>buf</b>. */ -static INLINE size_t +static inline size_t log_prefix_(char *buf, size_t buf_len, int severity) { time_t t; @@ -278,7 +278,7 @@ const char bug_suffix[] = " (on Tor " VERSION * than once.) Return a pointer to the first character of the message * portion of the formatted string. */ -static INLINE char * +static inline char * format_msg(char *buf, size_t buf_len, log_domain_mask_t domain, int severity, const char *funcname, const char *suffix, @@ -393,7 +393,7 @@ pending_log_message_free(pending_log_message_t *msg) /** Return true iff <b>lf</b> would like to receive a message with the * specified <b>severity</b> in the specified <b>domain</b>. */ -static INLINE int +static inline int logfile_wants_message(const logfile_t *lf, int severity, log_domain_mask_t domain) { @@ -416,7 +416,7 @@ logfile_wants_message(const logfile_t *lf, int severity, * we already deferred this message for pending callbacks and don't need to do * it again. Otherwise, if we need to do it, do it, and set * <b>callbacks_deferred</b> to 1. */ -static INLINE void +static inline void logfile_deliver(logfile_t *lf, const char *buf, size_t msg_len, const char *msg_after_prefix, log_domain_mask_t domain, int severity, int *callbacks_deferred) @@ -1097,14 +1097,25 @@ add_file_log(const log_severity_list_t *severity, const char *filename, #ifdef HAVE_SYSLOG_H /** * Add a log handler to send messages to they system log facility. + * + * If this is the first log handler, opens syslog with ident Tor or + * Tor-<syslog_identity_tag> if that is not NULL. */ int -add_syslog_log(const log_severity_list_t *severity) +add_syslog_log(const log_severity_list_t *severity, + const char* syslog_identity_tag) { logfile_t *lf; - if (syslog_count++ == 0) + if (syslog_count++ == 0) { /* This is the first syslog. */ - openlog("Tor", LOG_PID | LOG_NDELAY, LOGFACILITY); + static char buf[256]; + if (syslog_identity_tag) { + tor_snprintf(buf, sizeof(buf), "Tor-%s", syslog_identity_tag); + } else { + tor_snprintf(buf, sizeof(buf), "Tor"); + } + openlog(buf, LOG_PID | LOG_NDELAY, LOGFACILITY); + } lf = tor_malloc_zero(sizeof(logfile_t)); lf->fd = -1; diff --git a/src/common/memarea.c b/src/common/memarea.c index 6841ba54e7..a8e6d455d6 100644 --- a/src/common/memarea.c +++ b/src/common/memarea.c @@ -61,7 +61,7 @@ #endif /** Increment <b>ptr</b> until it is aligned to MEMAREA_ALIGN. */ -static INLINE void * +static inline void * realign_pointer(void *ptr) { uintptr_t x = (uintptr_t)ptr; diff --git a/src/common/procmon.c b/src/common/procmon.c index 2d0f021724..346a0c6943 100644 --- a/src/common/procmon.c +++ b/src/common/procmon.c @@ -192,7 +192,8 @@ tor_process_monitor_new(struct event_base *base, tor_procmon_callback_t cb, void *cb_arg, const char **msg) { - tor_process_monitor_t *procmon = tor_malloc(sizeof(tor_process_monitor_t)); + tor_process_monitor_t *procmon = tor_malloc_zero( + sizeof(tor_process_monitor_t)); struct parsed_process_specifier_t ppspec; tor_assert(msg != NULL); diff --git a/src/common/sandbox.c b/src/common/sandbox.c index 3a9f2a1898..6861bb3efc 100644 --- a/src/common/sandbox.c +++ b/src/common/sandbox.c @@ -199,6 +199,10 @@ static int filter_nopar_gen[] = { SCMP_SYS(stat64), #endif +#ifdef __NR_getrandom + SCMP_SYS(getrandom), +#endif + /* * These socket syscalls are not required on x86_64 and not supported with * some libseccomp versions (eg: 1.0.1) diff --git a/src/common/torgzip.c b/src/common/torgzip.c index 4f23407e23..5ba8ec4501 100644 --- a/src/common/torgzip.c +++ b/src/common/torgzip.c @@ -91,7 +91,7 @@ tor_zlib_get_header_version_str(void) } /** Return the 'bits' value to tell zlib to use <b>method</b>.*/ -static INLINE int +static inline int method_bits(compress_method_t method, zlib_compression_level_t level) { /* Bits+16 means "use gzip" in zlib >= 1.2 */ @@ -104,7 +104,7 @@ method_bits(compress_method_t method, zlib_compression_level_t level) } } -static INLINE int +static inline int get_memlevel(zlib_compression_level_t level) { switch (level) { diff --git a/src/common/torlog.h b/src/common/torlog.h index 67edf14c04..3e8667895f 100644 --- a/src/common/torlog.h +++ b/src/common/torlog.h @@ -135,7 +135,8 @@ void add_stream_log(const log_severity_list_t *severity, const char *name, int add_file_log(const log_severity_list_t *severity, const char *filename, const int truncate); #ifdef HAVE_SYSLOG_H -int add_syslog_log(const log_severity_list_t *severity); +int add_syslog_log(const log_severity_list_t *severity, + const char* syslog_identity_tag); #endif int add_callback_log(const log_severity_list_t *severity, log_callback cb); void logs_set_domain_logging(int enabled); @@ -183,25 +184,25 @@ void log_fn_ratelim_(struct ratelim_t *ratelim, int severity, /** Log a message at level <b>severity</b>, using a pretty-printed version * of the current function name. */ #define log_fn(severity, domain, args...) \ - log_fn_(severity, domain, __PRETTY_FUNCTION__, args) + log_fn_(severity, domain, __FUNCTION__, args) /** As log_fn, but use <b>ratelim</b> (an instance of ratelim_t) to control * the frequency at which messages can appear. */ #define log_fn_ratelim(ratelim, severity, domain, args...) \ - log_fn_ratelim_(ratelim, severity, domain, __PRETTY_FUNCTION__, args) + log_fn_ratelim_(ratelim, severity, domain, __FUNCTION__, args) #define log_debug(domain, args...) \ STMT_BEGIN \ if (PREDICT_UNLIKELY(log_global_min_severity_ == LOG_DEBUG)) \ - log_fn_(LOG_DEBUG, domain, __PRETTY_FUNCTION__, args); \ + log_fn_(LOG_DEBUG, domain, __FUNCTION__, args); \ STMT_END #define log_info(domain, args...) \ - log_fn_(LOG_INFO, domain, __PRETTY_FUNCTION__, args) + log_fn_(LOG_INFO, domain, __FUNCTION__, args) #define log_notice(domain, args...) \ - log_fn_(LOG_NOTICE, domain, __PRETTY_FUNCTION__, args) + log_fn_(LOG_NOTICE, domain, __FUNCTION__, args) #define log_warn(domain, args...) \ - log_fn_(LOG_WARN, domain, __PRETTY_FUNCTION__, args) + log_fn_(LOG_WARN, domain, __FUNCTION__, args) #define log_err(domain, args...) \ - log_fn_(LOG_ERR, domain, __PRETTY_FUNCTION__, args) + log_fn_(LOG_ERR, domain, __FUNCTION__, args) #else /* ! defined(__GNUC__) */ diff --git a/src/common/tortls.c b/src/common/tortls.c index 536043e558..b1d3f6f9e8 100644 --- a/src/common/tortls.c +++ b/src/common/tortls.c @@ -16,6 +16,8 @@ #include "orconfig.h" +#define TORTLS_PRIVATE + #include <assert.h> #ifdef _WIN32 /*wrkard for dtls1.h >= 0.9.8m of "#include <winsock.h>"*/ #include <winsock2.h> @@ -38,9 +40,6 @@ #include <openssl/opensslv.h> #include "crypto.h" -#if OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(1,0,0) -#error "We require OpenSSL >= 1.0.0" -#endif #ifdef OPENSSL_NO_EC #error "We require OpenSSL with ECC support" #endif @@ -69,6 +68,7 @@ #include "compat_libevent.h" #endif +#define TORTLS_PRIVATE #include "tortls.h" #include "util.h" #include "torlog.h" @@ -80,11 +80,6 @@ #define X509_get_notAfter_const(cert) \ ((const ASN1_TIME*) X509_get_notAfter((X509 *)cert)) -/* Enable the "v2" TLS handshake. - */ -#define V2_HANDSHAKE_SERVER -#define V2_HANDSHAKE_CLIENT - /* Copied from or.h */ #define LEGAL_NICKNAME_CHARACTERS \ "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" @@ -113,29 +108,6 @@ #define SSL3_FLAGS_ALLOW_UNSAFE_LEGACY_RENEGOTIATION 0x0010 #endif -/** Structure that we use for a single certificate. */ -struct tor_x509_cert_t { - X509 *cert; - uint8_t *encoded; - size_t encoded_len; - unsigned pkey_digests_set : 1; - digests_t cert_digests; - digests_t pkey_digests; -}; - -/** Holds a SSL_CTX object and related state used to configure TLS - * connections. - */ -typedef struct tor_tls_context_t { - int refcnt; - SSL_CTX *ctx; - tor_x509_cert_t *my_link_cert; - tor_x509_cert_t *my_id_cert; - tor_x509_cert_t *my_auth_cert; - crypto_pk_t *link_key; - crypto_pk_t *auth_key; -} tor_tls_context_t; - /** Return values for tor_tls_classify_client_ciphers. * * @{ @@ -154,60 +126,12 @@ typedef struct tor_tls_context_t { #define CIPHERS_UNRESTRICTED 3 /** @} */ -#define TOR_TLS_MAGIC 0x71571571 - -typedef enum { - TOR_TLS_ST_HANDSHAKE, TOR_TLS_ST_OPEN, TOR_TLS_ST_GOTCLOSE, - TOR_TLS_ST_SENTCLOSE, TOR_TLS_ST_CLOSED, TOR_TLS_ST_RENEGOTIATE, - TOR_TLS_ST_BUFFEREVENT -} tor_tls_state_t; -#define tor_tls_state_bitfield_t ENUM_BF(tor_tls_state_t) - -/** Holds a SSL object and its associated data. Members are only - * accessed from within tortls.c. - */ -struct tor_tls_t { - uint32_t magic; - tor_tls_context_t *context; /** A link to the context object for this tls. */ - SSL *ssl; /**< An OpenSSL SSL object. */ - int socket; /**< The underlying file descriptor for this TLS connection. */ - char *address; /**< An address to log when describing this connection. */ - tor_tls_state_bitfield_t state : 3; /**< The current SSL state, - * depending on which operations - * have completed successfully. */ - unsigned int isServer:1; /**< True iff this is a server-side connection */ - unsigned int wasV2Handshake:1; /**< True iff the original handshake for - * this connection used the updated version - * of the connection protocol (client sends - * different cipher list, server sends only - * one certificate). */ - /** True iff we should call negotiated_callback when we're done reading. */ - unsigned int got_renegotiate:1; - /** Return value from tor_tls_classify_client_ciphers, or 0 if we haven't - * called that function yet. */ - int8_t client_cipher_list_type; - /** Incremented every time we start the server side of a handshake. */ - uint8_t server_handshake_count; - size_t wantwrite_n; /**< 0 normally, >0 if we returned wantwrite last - * time. */ - /** Last values retrieved from BIO_number_read()/write(); see - * tor_tls_get_n_raw_bytes() for usage. - */ - unsigned long last_write_count; - unsigned long last_read_count; - /** If set, a callback to invoke whenever the client tries to renegotiate - * the handshake. */ - void (*negotiated_callback)(tor_tls_t *tls, void *arg); - /** Argument to pass to negotiated_callback. */ - void *callback_arg; -}; - /** The ex_data index in which we store a pointer to an SSL object's * corresponding tor_tls_t object. */ -static int tor_tls_object_ex_data_index = -1; +STATIC int tor_tls_object_ex_data_index = -1; /** Helper: Allocate tor_tls_object_ex_data_index. */ -static void +STATIC void tor_tls_allocate_tor_tls_object_ex_data_index(void) { if (tor_tls_object_ex_data_index == -1) { @@ -219,7 +143,7 @@ tor_tls_allocate_tor_tls_object_ex_data_index(void) /** Helper: given a SSL* pointer, return the tor_tls_t object using that * pointer. */ -static INLINE tor_tls_t * +STATIC inline tor_tls_t * tor_tls_get_by_ssl(const SSL *ssl) { tor_tls_t *result = SSL_get_ex_data(ssl, tor_tls_object_ex_data_index); @@ -230,21 +154,7 @@ tor_tls_get_by_ssl(const SSL *ssl) static void tor_tls_context_decref(tor_tls_context_t *ctx); static void tor_tls_context_incref(tor_tls_context_t *ctx); -static X509* tor_tls_create_certificate(crypto_pk_t *rsa, - crypto_pk_t *rsa_sign, - const char *cname, - const char *cname_sign, - unsigned int cert_lifetime); - -static int tor_tls_context_init_one(tor_tls_context_t **ppcontext, - crypto_pk_t *identity, - unsigned int key_lifetime, - unsigned int flags, - int is_client); -static tor_tls_context_t *tor_tls_context_new(crypto_pk_t *identity, - unsigned int key_lifetime, - unsigned int flags, - int is_client); + static int check_cert_lifetime_internal(int severity, const X509 *cert, int past_tolerance, int future_tolerance); @@ -252,8 +162,8 @@ static int check_cert_lifetime_internal(int severity, const X509 *cert, * to touch them. * * @{ */ -static tor_tls_context_t *server_tls_context = NULL; -static tor_tls_context_t *client_tls_context = NULL; +STATIC tor_tls_context_t *server_tls_context = NULL; +STATIC tor_tls_context_t *client_tls_context = NULL; /**@}*/ /** True iff tor_tls_init() has been called. */ @@ -347,7 +257,7 @@ tor_tls_log_one_error(tor_tls_t *tls, unsigned long err, /** Log all pending tls errors at level <b>severity</b> in log domain * <b>domain</b>. Use <b>doing</b> to describe our current activities. */ -static void +STATIC void tls_log_errors(tor_tls_t *tls, int severity, int domain, const char *doing) { unsigned long err; @@ -359,7 +269,7 @@ tls_log_errors(tor_tls_t *tls, int severity, int domain, const char *doing) /** Convert an errno (or a WSAerrno on windows) into a TOR_TLS_* error * code. */ -static int +STATIC int tor_errno_to_tls_error(int e) { switch (e) { @@ -410,7 +320,7 @@ tor_tls_err_to_string(int err) * If an error has occurred, log it at level <b>severity</b> and describe the * current action as <b>doing</b>. */ -static int +STATIC int tor_tls_get_error(tor_tls_t *tls, int r, int extra, const char *doing, int severity, int domain) { @@ -466,8 +376,9 @@ tor_tls_init(void) #if (SIZEOF_VOID_P >= 8 && \ OPENSSL_VERSION_NUMBER >= OPENSSL_V_SERIES(1,0,1)) - long version = SSLeay(); + long version = OpenSSL_version_num(); + /* LCOV_EXCL_START : we can't test these lines on the same machine */ if (version >= OPENSSL_V_SERIES(1,0,1)) { /* Warn if we could *almost* be running with much faster ECDH. If we're built for a 64-bit target, using OpenSSL 1.0.1, but we @@ -494,6 +405,7 @@ tor_tls_init(void) "support (using the enable-ec_nistp_64_gcc_128 option " "when configuring it) would make ECDH much faster."); } + /* LCOV_EXCL_STOP */ #endif tor_tls_allocate_tor_tls_object_ex_data_index(); @@ -524,7 +436,7 @@ tor_tls_free_all(void) * it: We always accept peer certs and complete the handshake. We * don't validate them until later. */ -static int +STATIC int always_accept_verify_cb(int preverify_ok, X509_STORE_CTX *x509_ctx) { @@ -539,16 +451,20 @@ tor_x509_name_new(const char *cname) { int nid; X509_NAME *name; + /* LCOV_EXCL_BR_START : these branches will only fail on OOM errors */ if (!(name = X509_NAME_new())) return NULL; if ((nid = OBJ_txt2nid("commonName")) == NID_undef) goto error; if (!(X509_NAME_add_entry_by_NID(name, nid, MBSTRING_ASC, (unsigned char*)cname, -1, -1, 0))) goto error; + /* LCOV_EXCL_BR_STOP */ return name; error: + /* LCOV_EXCL_START : these lines will only execute on out of memory errors*/ X509_NAME_free(name); return NULL; + /* LCOV_EXCL_STOP */ } /** Generate and sign an X509 certificate with the public key <b>rsa</b>, @@ -559,12 +475,12 @@ tor_x509_name_new(const char *cname) * * Return a certificate on success, NULL on failure. */ -static X509 * -tor_tls_create_certificate(crypto_pk_t *rsa, - crypto_pk_t *rsa_sign, - const char *cname, - const char *cname_sign, - unsigned int cert_lifetime) +MOCK_IMPL(STATIC X509 *, + tor_tls_create_certificate,(crypto_pk_t *rsa, + crypto_pk_t *rsa_sign, + const char *cname, + const char *cname_sign, + unsigned int cert_lifetime)) { /* OpenSSL generates self-signed certificates with random 64-bit serial * numbers, so let's do that too. */ @@ -601,8 +517,7 @@ tor_tls_create_certificate(crypto_pk_t *rsa, goto error; { /* our serial number is 8 random bytes. */ - if (crypto_rand((char *)serial_tmp, sizeof(serial_tmp)) < 0) - goto error; + crypto_rand((char *)serial_tmp, sizeof(serial_tmp)); if (!(serial_number = BN_bin2bn(serial_tmp, sizeof(serial_tmp), NULL))) goto error; if (!(BN_to_ASN1_INTEGER(serial_number, X509_get_serialNumber(x509)))) @@ -731,7 +646,9 @@ tor_x509_cert_free(tor_x509_cert_t *cert) X509_free(cert->cert); tor_free(cert->encoded); memwipe(cert, 0x03, sizeof(*cert)); + /* LCOV_EXCL_BR_START since cert will never be NULL here */ tor_free(cert); + /* LCOV_EXCL_BR_STOP */ } /** @@ -739,8 +656,8 @@ tor_x509_cert_free(tor_x509_cert_t *cert) * * Steals a reference to x509_cert. */ -static tor_x509_cert_t * -tor_x509_cert_new(X509 *x509_cert) +MOCK_IMPL(STATIC tor_x509_cert_t *, + tor_x509_cert_new,(X509 *x509_cert)) { tor_x509_cert_t *cert; EVP_PKEY *pkey; @@ -754,10 +671,12 @@ tor_x509_cert_new(X509 *x509_cert) length = i2d_X509(x509_cert, &buf); cert = tor_malloc_zero(sizeof(tor_x509_cert_t)); if (length <= 0 || buf == NULL) { + /* LCOV_EXCL_START for the same reason as the exclusion above */ tor_free(cert); log_err(LD_CRYPTO, "Couldn't get length of encoded x509 certificate"); X509_free(x509_cert); return NULL; + /* LCOV_EXCL_STOP */ } cert->encoded_len = (size_t) length; cert->encoded = tor_malloc(length); @@ -864,7 +783,9 @@ tor_tls_context_decref(tor_tls_context_t *ctx) tor_x509_cert_free(ctx->my_auth_cert); crypto_pk_free(ctx->link_key); crypto_pk_free(ctx->auth_key); + /* LCOV_EXCL_BR_START since ctx will never be NULL here */ tor_free(ctx); + /* LCOV_EXCL_BR_STOP */ } } @@ -960,11 +881,13 @@ tor_tls_cert_is_valid(int severity, int check_rsa_1024) { check_no_tls_errors(); - EVP_PKEY *cert_key; - EVP_PKEY *signing_key = X509_get_pubkey(signing_cert->cert); int r, key_ok = 0; + if (!signing_cert) + goto bad; + + EVP_PKEY *signing_key = X509_get_pubkey(signing_cert->cert); if (!signing_key) goto bad; r = X509_verify(cert->cert, signing_key); @@ -1085,7 +1008,7 @@ tor_tls_context_init(unsigned flags, * it generates new certificates; all new connections will use * the new SSL context. */ -static int +STATIC int tor_tls_context_init_one(tor_tls_context_t **ppcontext, crypto_pk_t *identity, unsigned int key_lifetime, @@ -1119,7 +1042,7 @@ tor_tls_context_init_one(tor_tls_context_t **ppcontext, * <b>identity</b> should be set to the identity key used to sign the * certificate. */ -static tor_tls_context_t * +STATIC tor_tls_context_t * tor_tls_context_new(crypto_pk_t *identity, unsigned int key_lifetime, unsigned flags, int is_client) { @@ -1200,23 +1123,6 @@ tor_tls_context_new(crypto_pk_t *identity, unsigned int key_lifetime, * historically been chosen for fingerprinting resistance. */ SSL_CTX_set_options(result->ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); - /* Disable TLS1.1 and TLS1.2 if they exist. We need to do this to - * workaround a bug present in all OpenSSL 1.0.1 versions (as of 1 - * June 2012), wherein renegotiating while using one of these TLS - * protocols will cause the client to send a TLS 1.0 ServerHello - * rather than a ServerHello written with the appropriate protocol - * version. Once some version of OpenSSL does TLS1.1 and TLS1.2 - * renegotiation properly, we can turn them back on when built with - * that version. */ -#if OPENSSL_VERSION_NUMBER < OPENSSL_V(1,0,1,'e') -#ifdef SSL_OP_NO_TLSv1_2 - SSL_CTX_set_options(result->ctx, SSL_OP_NO_TLSv1_2); -#endif -#ifdef SSL_OP_NO_TLSv1_1 - SSL_CTX_set_options(result->ctx, SSL_OP_NO_TLSv1_1); -#endif -#endif - /* Disable TLS tickets if they're supported. We never want to use them; * using them can make our perfect forward secrecy a little worse, *and* * create an opportunity to fingerprint us (since it's unusual to use them @@ -1343,11 +1249,13 @@ tor_tls_context_new(crypto_pk_t *identity, unsigned int key_lifetime, } /** Invoked when a TLS state changes: log the change at severity 'debug' */ -static void +STATIC void tor_tls_debug_state_callback(const SSL *ssl, int type, int val) { + /* LCOV_EXCL_START since this depends on whether debug is captured or not */ log_debug(LD_HANDSHAKE, "SSL %p is now in state %s [type=%d,val=%d].", ssl, SSL_state_string_long(ssl), type, val); + /* LCOV_EXCL_STOP */ } /* Return the name of the negotiated ciphersuite in use on <b>tls</b> */ @@ -1357,13 +1265,11 @@ tor_tls_get_ciphersuite_name(tor_tls_t *tls) return SSL_get_cipher(tls->ssl); } -#ifdef V2_HANDSHAKE_SERVER - /* Here's the old V2 cipher list we sent from 0.2.1.1-alpha up to * 0.2.3.17-beta. If a client is using this list, we can't believe the ciphers * that it claims to support. We'll prune this list to remove the ciphers * *we* don't recognize. */ -static uint16_t v2_cipher_list[] = { +STATIC uint16_t v2_cipher_list[] = { 0xc00a, /* TLS1_TXT_ECDHE_ECDSA_WITH_AES_256_CBC_SHA */ 0xc014, /* TLS1_TXT_ECDHE_RSA_WITH_AES_256_CBC_SHA */ 0x0039, /* TLS1_TXT_DHE_RSA_WITH_AES_256_SHA */ @@ -1399,7 +1305,7 @@ static int v2_cipher_list_pruned = 0; /** Return 0 if <b>m</b> does not support the cipher with ID <b>cipher</b>; * return 1 if it does support it, or if we have no way to tell. */ -static int +STATIC int find_cipher_by_id(const SSL *ssl, const SSL_METHOD *m, uint16_t cipher) { const SSL_CIPHER *c; @@ -1481,7 +1387,7 @@ prune_v2_cipher_list(const SSL *ssl) * client it is. Return one of CIPHERS_ERR, CIPHERS_V1, CIPHERS_V2, * CIPHERS_UNRESTRICTED. **/ -static int +STATIC int tor_tls_classify_client_ciphers(const SSL *ssl, STACK_OF(SSL_CIPHER) *peer_ciphers) { @@ -1563,7 +1469,7 @@ tor_tls_classify_client_ciphers(const SSL *ssl, /** Return true iff the cipher list suggested by the client for <b>ssl</b> is * a list that indicates that the client knows how to do the v2 TLS connection * handshake. */ -static int +STATIC int tor_tls_client_is_using_v2_ciphers(const SSL *ssl) { STACK_OF(SSL_CIPHER) *ciphers; @@ -1587,11 +1493,10 @@ tor_tls_client_is_using_v2_ciphers(const SSL *ssl) * do not send or request extra certificates in v2 handshakes.</li> * <li>To detect renegotiation</li></ul> */ -static void +STATIC void tor_tls_server_info_callback(const SSL *ssl, int type, int val) { tor_tls_t *tls; - int ssl_state; (void) val; tor_tls_debug_state_callback(ssl, type, val); @@ -1599,11 +1504,9 @@ tor_tls_server_info_callback(const SSL *ssl, int type, int val) if (type != SSL_CB_ACCEPT_LOOP) return; - ssl_state = SSL_state(ssl); - if ((ssl_state != SSL3_ST_SW_SRVR_HELLO_A) && - (ssl_state != SSL3_ST_SW_SRVR_HELLO_B)) + OSSL_HANDSHAKE_STATE ssl_state = SSL_get_state(ssl); + if (! STATE_IS_SW_SERVER_HELLO(ssl_state)) return; - tls = tor_tls_get_by_ssl(ssl); if (tls) { /* Check whether we're watching for renegotiates. If so, this is one! */ @@ -1633,11 +1536,12 @@ tor_tls_server_info_callback(const SSL *ssl, int type, int val) if (tls) { tls->wasV2Handshake = 1; } else { + /* LCOV_EXCL_START this line is not reachable */ log_warn(LD_BUG, "Couldn't look up the tls for an SSL*. How odd!"); + /* LCOV_EXCL_STOP */ } } } -#endif /** Callback to get invoked on a server after we've read the list of ciphers * the client supports, but before we pick our own ciphersuite. @@ -1651,7 +1555,7 @@ tor_tls_server_info_callback(const SSL *ssl, int type, int val) * authentication on the fly. But as long as we return 0, we won't actually be * setting up a shared secret, and all will be fine. */ -static int +STATIC int tor_tls_session_secret_cb(SSL *ssl, void *secret, int *secret_len, STACK_OF(SSL_CIPHER) *peer_ciphers, SSL_CIPHER **cipher, void *arg) @@ -1747,12 +1651,9 @@ tor_tls_new(int sock, int isServer) log_warn(LD_NET, "Newly created BIO has read count %lu, write count %lu", result->last_read_count, result->last_write_count); } -#ifdef V2_HANDSHAKE_SERVER if (isServer) { SSL_set_info_callback(result->ssl, tor_tls_server_info_callback); - } else -#endif - { + } else { SSL_set_info_callback(result->ssl, tor_tls_debug_state_callback); } @@ -1791,13 +1692,11 @@ tor_tls_set_renegotiate_callback(tor_tls_t *tls, tls->negotiated_callback = cb; tls->callback_arg = arg; tls->got_renegotiate = 0; -#ifdef V2_HANDSHAKE_SERVER if (cb) { SSL_set_info_callback(tls->ssl, tor_tls_server_info_callback); } else { SSL_set_info_callback(tls->ssl, tor_tls_debug_state_callback); } -#endif } /** If this version of openssl requires it, turn on renegotiation on @@ -1884,7 +1783,6 @@ tor_tls_read,(tor_tls_t *tls, char *cp, size_t len)) tor_assert(len<INT_MAX); r = SSL_read(tls->ssl, cp, (int)len); if (r > 0) { -#ifdef V2_HANDSHAKE_SERVER if (tls->got_renegotiate) { /* Renegotiation happened! */ log_info(LD_NET, "Got a TLS renegotiation from %s", ADDR(tls)); @@ -1892,7 +1790,6 @@ tor_tls_read,(tor_tls_t *tls, char *cp, size_t len)) tls->negotiated_callback(tls, tls->callback_arg); tls->got_renegotiate = 0; } -#endif return r; } err = tor_tls_get_error(tls, r, CATCH_ZERO, "reading", LOG_DEBUG, LD_NET); @@ -1909,10 +1806,10 @@ tor_tls_read,(tor_tls_t *tls, char *cp, size_t len)) /** Total number of bytes that we've used TLS to send. Used to track TLS * overhead. */ -static uint64_t total_bytes_written_over_tls = 0; +STATIC uint64_t total_bytes_written_over_tls = 0; /** Total number of bytes that TLS has put on the network for us. Used to * track TLS overhead. */ -static uint64_t total_bytes_written_by_tls = 0; +STATIC uint64_t total_bytes_written_by_tls = 0; /** Underlying function for TLS writing. Write up to <b>n</b> * characters from <b>cp</b> onto <b>tls</b>. On success, returns the @@ -1957,12 +1854,14 @@ int tor_tls_handshake(tor_tls_t *tls) { int r; - int oldstate; tor_assert(tls); tor_assert(tls->ssl); tor_assert(tls->state == TOR_TLS_ST_HANDSHAKE); + check_no_tls_errors(); - oldstate = SSL_state(tls->ssl); + + OSSL_HANDSHAKE_STATE oldstate = SSL_get_state(tls->ssl); + if (tls->isServer) { log_debug(LD_HANDSHAKE, "About to call SSL_accept on %p (%s)", tls, SSL_state_string_long(tls->ssl)); @@ -1972,7 +1871,10 @@ tor_tls_handshake(tor_tls_t *tls) SSL_state_string_long(tls->ssl)); r = SSL_connect(tls->ssl); } - if (oldstate != SSL_state(tls->ssl)) + + OSSL_HANDSHAKE_STATE newstate = SSL_get_state(tls->ssl); + + if (oldstate != newstate) log_debug(LD_HANDSHAKE, "After call, %p was in state %s", tls, SSL_state_string_long(tls->ssl)); /* We need to call this here and not earlier, since OpenSSL has a penchant @@ -2008,7 +1910,6 @@ tor_tls_finish_handshake(tor_tls_t *tls) SSL_set_info_callback(tls->ssl, NULL); SSL_set_verify(tls->ssl, SSL_VERIFY_PEER, always_accept_verify_cb); SSL_clear_mode(tls->ssl, SSL_MODE_NO_AUTO_CHAIN); -#ifdef V2_HANDSHAKE_SERVER if (tor_tls_client_is_using_v2_ciphers(tls->ssl)) { /* This check is redundant, but back when we did it in the callback, * we might have not been able to look up the tor_tls_t if the code @@ -2023,26 +1924,10 @@ tor_tls_finish_handshake(tor_tls_t *tls) } else { tls->wasV2Handshake = 0; } -#endif } else { -#ifdef V2_HANDSHAKE_CLIENT - /* If we got no ID cert, we're a v2 handshake. */ - X509 *cert = SSL_get_peer_certificate(tls->ssl); - STACK_OF(X509) *chain = SSL_get_peer_cert_chain(tls->ssl); - int n_certs = sk_X509_num(chain); - if (n_certs > 1 || (n_certs == 1 && cert != sk_X509_value(chain, 0))) { - log_debug(LD_HANDSHAKE, "Server sent back multiple certificates; it " - "looks like a v1 handshake on %p", tls); - tls->wasV2Handshake = 0; - } else { - log_debug(LD_HANDSHAKE, - "Server sent back a single certificate; looks like " - "a v2 handshake on %p.", tls); - tls->wasV2Handshake = 1; - } - if (cert) - X509_free(cert); -#endif + /* Client-side */ + tls->wasV2Handshake = 1; + /* XXXX this can move, probably? -NM */ if (SSL_set_cipher_list(tls->ssl, SERVER_CIPHER_LIST) == 0) { tls_log_errors(NULL, LOG_WARN, LD_HANDSHAKE, "re-setting ciphers"); r = TOR_TLS_ERROR_MISC; @@ -2052,52 +1937,6 @@ tor_tls_finish_handshake(tor_tls_t *tls) return r; } -#ifdef USE_BUFFEREVENTS -/** Put <b>tls</b>, which must be a client connection, into renegotiation - * mode. */ -int -tor_tls_start_renegotiating(tor_tls_t *tls) -{ - int r = SSL_renegotiate(tls->ssl); - if (r <= 0) { - return tor_tls_get_error(tls, r, 0, "renegotiating", LOG_WARN, - LD_HANDSHAKE); - } - return 0; -} -#endif - -/** Client only: Renegotiate a TLS session. When finished, returns - * TOR_TLS_DONE. On failure, returns TOR_TLS_ERROR, TOR_TLS_WANTREAD, or - * TOR_TLS_WANTWRITE. - */ -int -tor_tls_renegotiate(tor_tls_t *tls) -{ - int r; - tor_assert(tls); - /* We could do server-initiated renegotiation too, but that would be tricky. - * Instead of "SSL_renegotiate, then SSL_do_handshake until done" */ - tor_assert(!tls->isServer); - - check_no_tls_errors(); - if (tls->state != TOR_TLS_ST_RENEGOTIATE) { - int r = SSL_renegotiate(tls->ssl); - if (r <= 0) { - return tor_tls_get_error(tls, r, 0, "renegotiating", LOG_WARN, - LD_HANDSHAKE); - } - tls->state = TOR_TLS_ST_RENEGOTIATE; - } - r = SSL_do_handshake(tls->ssl); - if (r == 1) { - tls->state = TOR_TLS_ST_OPEN; - return TOR_TLS_DONE; - } else - return tor_tls_get_error(tls, r, 0, "renegotiating handshake", LOG_INFO, - LD_HANDSHAKE); -} - /** Shut down an open tls connection <b>tls</b>. When finished, returns * TOR_TLS_DONE. On failure, returns TOR_TLS_ERROR, TOR_TLS_WANTREAD, * or TOR_TLS_WANTWRITE. @@ -2251,15 +2090,14 @@ log_cert_lifetime(int severity, const X509 *cert, const char *problem) * * Note that a reference is added to cert_out, so it needs to be * freed. id_cert_out doesn't. */ -static void -try_to_extract_certs_from_tls(int severity, tor_tls_t *tls, - X509 **cert_out, X509 **id_cert_out) +MOCK_IMPL(STATIC void, +try_to_extract_certs_from_tls,(int severity, tor_tls_t *tls, + X509 **cert_out, X509 **id_cert_out)) { X509 *cert = NULL, *id_cert = NULL; STACK_OF(X509) *chain = NULL; int num_in_chain, i; *cert_out = *id_cert_out = NULL; - if (!(cert = SSL_get_peer_certificate(tls->ssl))) return; *cert_out = cert; @@ -2476,114 +2314,7 @@ check_no_tls_errors_(const char *fname, int line) int tor_tls_used_v1_handshake(tor_tls_t *tls) { -#if defined(V2_HANDSHAKE_SERVER) && defined(V2_HANDSHAKE_CLIENT) return ! tls->wasV2Handshake; -#else - if (tls->isServer) { -# ifdef V2_HANDSHAKE_SERVER - return ! tls->wasV2Handshake; -# endif - } else { -# ifdef V2_HANDSHAKE_CLIENT - return ! tls->wasV2Handshake; -# endif - } - return 1; -#endif -} - -/** Return true iff <b>name</b> is a DN of a kind that could only - * occur in a v3-handshake-indicating certificate */ -static int -dn_indicates_v3_cert(X509_NAME *name) -{ -#ifdef DISABLE_V3_LINKPROTO_CLIENTSIDE - (void)name; - return 0; -#else - X509_NAME_ENTRY *entry; - int n_entries; - ASN1_OBJECT *obj; - ASN1_STRING *str; - unsigned char *s; - int len, r; - - n_entries = X509_NAME_entry_count(name); - if (n_entries != 1) - return 1; /* More than one entry in the DN. */ - entry = X509_NAME_get_entry(name, 0); - - obj = X509_NAME_ENTRY_get_object(entry); - if (OBJ_obj2nid(obj) != OBJ_txt2nid("commonName")) - return 1; /* The entry isn't a commonName. */ - - str = X509_NAME_ENTRY_get_data(entry); - len = ASN1_STRING_to_UTF8(&s, str); - if (len < 0) - return 0; - if (len < 4) { - OPENSSL_free(s); - return 1; - } - r = fast_memneq(s + len - 4, ".net", 4); - OPENSSL_free(s); - return r; -#endif -} - -/** Return true iff the peer certificate we're received on <b>tls</b> - * indicates that this connection should use the v3 (in-protocol) - * authentication handshake. - * - * Only the connection initiator should use this, and only once the initial - * handshake is done; the responder detects a v1 handshake by cipher types, - * and a v3/v2 handshake by Versions cell vs renegotiation. - */ -int -tor_tls_received_v3_certificate(tor_tls_t *tls) -{ - check_no_tls_errors(); - - X509 *cert = SSL_get_peer_certificate(tls->ssl); - EVP_PKEY *key = NULL; - X509_NAME *issuer_name, *subject_name; - int is_v3 = 0; - - if (!cert) { - log_warn(LD_BUG, "Called on a connection with no peer certificate"); - goto done; - } - - subject_name = X509_get_subject_name(cert); - issuer_name = X509_get_issuer_name(cert); - - if (X509_name_cmp(subject_name, issuer_name) == 0) { - is_v3 = 1; /* purportedly self signed */ - goto done; - } - - if (dn_indicates_v3_cert(subject_name) || - dn_indicates_v3_cert(issuer_name)) { - is_v3 = 1; /* DN is fancy */ - goto done; - } - - key = X509_get_pubkey(cert); - if (EVP_PKEY_bits(key) != 1024 || - EVP_PKEY_type(key->type) != EVP_PKEY_RSA) { - is_v3 = 1; /* Key is fancy */ - goto done; - } - - done: - tls_log_errors(tls, LOG_WARN, LD_NET, "checking for a v3 cert"); - - if (key) - EVP_PKEY_free(key); - if (cert) - X509_free(cert); - - return is_v3; } /** Return the number of server handshakes that we've noticed doing on @@ -2629,7 +2360,7 @@ SSL_get_server_random(SSL *s, uint8_t *out, size_t len) #endif #ifndef HAVE_SSL_SESSION_GET_MASTER_KEY -static size_t +STATIC size_t SSL_SESSION_get_master_key(SSL_SESSION *s, uint8_t *out, size_t len) { tor_assert(s); @@ -2652,7 +2383,6 @@ tor_tls_get_tlssecrets,(tor_tls_t *tls, uint8_t *secrets_out)) #define TLSSECRET_MAGIC "Tor V3 handshake TLS cross-certification" uint8_t buf[128]; size_t len; - tor_assert(tls); SSL *const ssl = tls->ssl; @@ -2676,12 +2406,14 @@ tor_tls_get_tlssecrets,(tor_tls_t *tls, uint8_t *secrets_out)) size_t r = SSL_get_client_random(ssl, buf, client_random_len); tor_assert(r == client_random_len); } + { size_t r = SSL_get_server_random(ssl, buf+client_random_len, server_random_len); tor_assert(r == server_random_len); } + uint8_t *master_key = tor_malloc_zero(master_key_len); { size_t r = SSL_SESSION_get_master_key(session, master_key, master_key_len); diff --git a/src/common/tortls.h b/src/common/tortls.h index 124b77160f..6a4ef9aebe 100644 --- a/src/common/tortls.h +++ b/src/common/tortls.h @@ -12,6 +12,7 @@ **/ #include "crypto.h" +#include "compat_openssl.h" #include "compat.h" #include "testsupport.h" @@ -51,6 +52,119 @@ typedef struct tor_x509_cert_t tor_x509_cert_t; case TOR_TLS_ERROR_IO #define TOR_TLS_IS_ERROR(rv) ((rv) < TOR_TLS_CLOSE) + +#ifdef TORTLS_PRIVATE +#define TOR_TLS_MAGIC 0x71571571 + +typedef enum { + TOR_TLS_ST_HANDSHAKE, TOR_TLS_ST_OPEN, TOR_TLS_ST_GOTCLOSE, + TOR_TLS_ST_SENTCLOSE, TOR_TLS_ST_CLOSED, TOR_TLS_ST_RENEGOTIATE, + TOR_TLS_ST_BUFFEREVENT +} tor_tls_state_t; +#define tor_tls_state_bitfield_t ENUM_BF(tor_tls_state_t) + +/** Holds a SSL_CTX object and related state used to configure TLS + * connections. + */ +typedef struct tor_tls_context_t { + int refcnt; + SSL_CTX *ctx; + tor_x509_cert_t *my_link_cert; + tor_x509_cert_t *my_id_cert; + tor_x509_cert_t *my_auth_cert; + crypto_pk_t *link_key; + crypto_pk_t *auth_key; +} tor_tls_context_t; + +/** Structure that we use for a single certificate. */ +struct tor_x509_cert_t { + X509 *cert; + uint8_t *encoded; + size_t encoded_len; + unsigned pkey_digests_set : 1; + digests_t cert_digests; + digests_t pkey_digests; +}; + +/** Holds a SSL object and its associated data. Members are only + * accessed from within tortls.c. + */ +struct tor_tls_t { + uint32_t magic; + tor_tls_context_t *context; /** A link to the context object for this tls. */ + SSL *ssl; /**< An OpenSSL SSL object. */ + int socket; /**< The underlying file descriptor for this TLS connection. */ + char *address; /**< An address to log when describing this connection. */ + tor_tls_state_bitfield_t state : 3; /**< The current SSL state, + * depending on which operations + * have completed successfully. */ + unsigned int isServer:1; /**< True iff this is a server-side connection */ + unsigned int wasV2Handshake:1; /**< True iff the original handshake for + * this connection used the updated version + * of the connection protocol (client sends + * different cipher list, server sends only + * one certificate). */ + /** True iff we should call negotiated_callback when we're done reading. */ + unsigned int got_renegotiate:1; + /** Return value from tor_tls_classify_client_ciphers, or 0 if we haven't + * called that function yet. */ + int8_t client_cipher_list_type; + /** Incremented every time we start the server side of a handshake. */ + uint8_t server_handshake_count; + size_t wantwrite_n; /**< 0 normally, >0 if we returned wantwrite last + * time. */ + /** Last values retrieved from BIO_number_read()/write(); see + * tor_tls_get_n_raw_bytes() for usage. + */ + unsigned long last_write_count; + unsigned long last_read_count; + /** If set, a callback to invoke whenever the client tries to renegotiate + * the handshake. */ + void (*negotiated_callback)(tor_tls_t *tls, void *arg); + /** Argument to pass to negotiated_callback. */ + void *callback_arg; +}; + +STATIC int tor_errno_to_tls_error(int e); +STATIC int tor_tls_get_error(tor_tls_t *tls, int r, int extra, + const char *doing, int severity, int domain); +STATIC tor_tls_t *tor_tls_get_by_ssl(const SSL *ssl); +STATIC void tor_tls_allocate_tor_tls_object_ex_data_index(void); +STATIC int always_accept_verify_cb(int preverify_ok, X509_STORE_CTX *x509_ctx); +STATIC int tor_tls_classify_client_ciphers(const SSL *ssl, + STACK_OF(SSL_CIPHER) *peer_ciphers); +STATIC int tor_tls_client_is_using_v2_ciphers(const SSL *ssl); +MOCK_DECL(STATIC void, try_to_extract_certs_from_tls, + (int severity, tor_tls_t *tls, X509 **cert_out, X509 **id_cert_out)); +#ifndef HAVE_SSL_SESSION_GET_MASTER_KEY +STATIC size_t SSL_SESSION_get_master_key(SSL_SESSION *s, uint8_t *out, + size_t len); +#endif +STATIC void tor_tls_debug_state_callback(const SSL *ssl, int type, int val); +STATIC void tor_tls_server_info_callback(const SSL *ssl, int type, int val); +STATIC int tor_tls_session_secret_cb(SSL *ssl, void *secret, + int *secret_len, + STACK_OF(SSL_CIPHER) *peer_ciphers, + SSL_CIPHER **cipher, void *arg); +STATIC int find_cipher_by_id(const SSL *ssl, const SSL_METHOD *m, + uint16_t cipher); +MOCK_DECL(STATIC X509*, tor_tls_create_certificate,(crypto_pk_t *rsa, + crypto_pk_t *rsa_sign, + const char *cname, + const char *cname_sign, + unsigned int cert_lifetime)); +STATIC tor_tls_context_t *tor_tls_context_new(crypto_pk_t *identity, + unsigned int key_lifetime, unsigned flags, int is_client); +MOCK_DECL(STATIC tor_x509_cert_t *, tor_x509_cert_new,(X509 *x509_cert)); +STATIC int tor_tls_context_init_one(tor_tls_context_t **ppcontext, + crypto_pk_t *identity, + unsigned int key_lifetime, + unsigned int flags, + int is_client); +STATIC void tls_log_errors(tor_tls_t *tls, int severity, int domain, + const char *doing); +#endif + const char *tor_tls_err_to_string(int err); void tor_tls_get_state_description(tor_tls_t *tls, char *buf, size_t sz); @@ -81,7 +195,6 @@ MOCK_DECL(int, tor_tls_read, (tor_tls_t *tls, char *cp, size_t len)); int tor_tls_write(tor_tls_t *tls, const char *cp, size_t n); int tor_tls_handshake(tor_tls_t *tls); int tor_tls_finish_handshake(tor_tls_t *tls); -int tor_tls_renegotiate(tor_tls_t *tls); void tor_tls_unblock_renegotiation(tor_tls_t *tls); void tor_tls_block_renegotiation(tor_tls_t *tls); void tor_tls_assert_renegotiation_unblocked(tor_tls_t *tls); @@ -99,7 +212,6 @@ int tor_tls_get_buffer_sizes(tor_tls_t *tls, MOCK_DECL(double, tls_get_write_overhead_ratio, (void)); int tor_tls_used_v1_handshake(tor_tls_t *tls); -int tor_tls_received_v3_certificate(tor_tls_t *tls); int tor_tls_get_num_server_handshakes(tor_tls_t *tls); int tor_tls_server_got_renegotiate(tor_tls_t *tls); MOCK_DECL(int,tor_tls_get_tlssecrets,(tor_tls_t *tls, uint8_t *secrets_out)); diff --git a/src/common/util.c b/src/common/util.c index b33c80fd45..04f48a4eee 100644 --- a/src/common/util.c +++ b/src/common/util.c @@ -207,7 +207,7 @@ tor_malloc_zero_(size_t size DMALLOC_PARAMS) #define SQRT_SIZE_MAX_P1 (((size_t)1) << (sizeof(size_t)*4)) /** Return non-zero if and only if the product of the arguments is exact. */ -static INLINE int +static inline int size_mul_check(const size_t x, const size_t y) { /* This first check is equivalent to @@ -488,42 +488,58 @@ round_to_power_of_2(uint64_t u64) } /** Return the lowest x such that x is at least <b>number</b>, and x modulo - * <b>divisor</b> == 0. */ + * <b>divisor</b> == 0. If no such x can be expressed as an unsigned, return + * UINT_MAX */ unsigned round_to_next_multiple_of(unsigned number, unsigned divisor) { + tor_assert(divisor > 0); + if (UINT_MAX - divisor + 1 < number) + return UINT_MAX; number += divisor - 1; number -= number % divisor; return number; } /** Return the lowest x such that x is at least <b>number</b>, and x modulo - * <b>divisor</b> == 0. */ + * <b>divisor</b> == 0. If no such x can be expressed as a uint32_t, return + * UINT32_MAX */ uint32_t round_uint32_to_next_multiple_of(uint32_t number, uint32_t divisor) { + tor_assert(divisor > 0); + if (UINT32_MAX - divisor + 1 < number) + return UINT32_MAX; + number += divisor - 1; number -= number % divisor; return number; } /** Return the lowest x such that x is at least <b>number</b>, and x modulo - * <b>divisor</b> == 0. */ + * <b>divisor</b> == 0. If no such x can be expressed as a uint64_t, return + * UINT64_MAX */ uint64_t round_uint64_to_next_multiple_of(uint64_t number, uint64_t divisor) { + tor_assert(divisor > 0); + if (UINT64_MAX - divisor + 1 < number) + return UINT64_MAX; number += divisor - 1; number -= number % divisor; return number; } /** Return the lowest x in [INT64_MIN, INT64_MAX] such that x is at least - * <b>number</b>, and x modulo <b>divisor</b> == 0. */ + * <b>number</b>, and x modulo <b>divisor</b> == 0. If no such x can be + * expressed as an int64_t, return INT64_MAX */ int64_t round_int64_to_next_multiple_of(int64_t number, int64_t divisor) { tor_assert(divisor > 0); - if (number >= 0 && INT64_MAX - divisor + 1 >= number) + if (INT64_MAX - divisor + 1 < number) + return INT64_MAX; + if (number >= 0) number += divisor - 1; number -= number % divisor; return number; @@ -537,33 +553,44 @@ int64_t sample_laplace_distribution(double mu, double b, double p) { double result; - tor_assert(p >= 0.0 && p < 1.0); + /* This is the "inverse cumulative distribution function" from: * http://en.wikipedia.org/wiki/Laplace_distribution */ - result = mu - b * (p > 0.5 ? 1.0 : -1.0) - * tor_mathlog(1.0 - 2.0 * fabs(p - 0.5)); - - if (result >= INT64_MAX) - return INT64_MAX; - else if (result <= INT64_MIN) + if (p <= 0.0) { + /* Avoid taking log(0.0) == -INFINITY, as some processors or compiler + * options can cause the program to trap. */ return INT64_MIN; - else - return (int64_t) result; + } + + result = mu - b * (p > 0.5 ? 1.0 : -1.0) + * tor_mathlog(1.0 - 2.0 * fabs(p - 0.5)); + + return clamp_double_to_int64(result); } -/** Add random noise between INT64_MIN and INT64_MAX coming from a - * Laplace distribution with mu = 0 and b = <b>delta_f</b>/<b>epsilon</b> - * to <b>signal</b> based on the provided <b>random</b> value in - * [0.0, 1.0[. */ +/** Add random noise between INT64_MIN and INT64_MAX coming from a Laplace + * distribution with mu = 0 and b = <b>delta_f</b>/<b>epsilon</b> to + * <b>signal</b> based on the provided <b>random</b> value in [0.0, 1.0[. + * The epsilon value must be between ]0.0, 1.0]. delta_f must be greater + * than 0. */ int64_t add_laplace_noise(int64_t signal, double random, double delta_f, double epsilon) { - int64_t noise = sample_laplace_distribution( - 0.0, /* just add noise, no further signal */ - delta_f / epsilon, random); + int64_t noise; + + /* epsilon MUST be between ]0.0, 1.0] */ + tor_assert(epsilon > 0.0 && epsilon <= 1.0); + /* delta_f MUST be greater than 0. */ + tor_assert(delta_f > 0.0); + /* Just add noise, no further signal */ + noise = sample_laplace_distribution(0.0, + delta_f / epsilon, + random); + + /* Clip (signal + noise) to [INT64_MIN, INT64_MAX] */ if (noise > 0 && INT64_MAX - noise < signal) return INT64_MAX; else if (noise < 0 && INT64_MIN - noise > signal) @@ -2116,7 +2143,7 @@ check_private_dir(const char *dirname, cpd_check_t check, return -1; } if ( (check & (CPD_GROUP_OK|CPD_GROUP_READ)) - && (st.st_gid != running_gid) ) { + && (st.st_gid != running_gid) && (st.st_gid != 0)) { struct group *gr; char *process_groupname = NULL; gr = getgrgid(running_gid); @@ -4424,7 +4451,7 @@ tor_get_exit_code(process_handle_t *process_handle, /** Helper: return the number of characters in <b>s</b> preceding the first * occurrence of <b>ch</b>. If <b>ch</b> does not occur in <b>s</b>, return * the length of <b>s</b>. Should be equivalent to strspn(s, "ch"). */ -static INLINE size_t +static inline size_t str_num_before(const char *s, char ch) { const char *cp = strchr(s, ch); @@ -5385,3 +5412,38 @@ tor_weak_random_range(tor_weak_rng_t *rng, int32_t top) return result; } +/** Cast a given double value to a int64_t. Return 0 if number is NaN. + * Returns either INT64_MIN or INT64_MAX if number is outside of the int64_t + * range. */ +int64_t +clamp_double_to_int64(double number) +{ + int exp; + + /* NaN is a special case that can't be used with the logic below. */ + if (isnan(number)) { + return 0; + } + + /* Time to validate if result can overflows a int64_t value. Fun with + * float! Find that exponent exp such that + * number == x * 2^exp + * for some x with abs(x) in [0.5, 1.0). Note that this implies that the + * magnitude of number is strictly less than 2^exp. + * + * If number is infinite, the call to frexp is legal but the contents of + * exp are unspecified. */ + frexp(number, &exp); + + /* If the magnitude of number is strictly less than 2^63, the truncated + * version of number is guaranteed to be representable. The only + * representable integer for which this is not the case is INT64_MIN, but + * it is covered by the logic below. */ + if (isfinite(number) && exp <= 63) { + return number; + } + + /* Handle infinities and finite numbers with magnitude >= 2^63. */ + return signbit(number) ? INT64_MIN : INT64_MAX; +} + diff --git a/src/common/util.h b/src/common/util.h index 8bb4505e86..165bc0dcb3 100644 --- a/src/common/util.h +++ b/src/common/util.h @@ -185,6 +185,7 @@ int64_t sample_laplace_distribution(double mu, double b, double p); int64_t add_laplace_noise(int64_t signal, double random, double delta_f, double epsilon); int n_bits_set_u8(uint8_t v); +int64_t clamp_double_to_int64(double number); /* Compute the CEIL of <b>a</b> divided by <b>b</b>, for nonnegative <b>a</b> * and positive <b>b</b>. Works on integer types only. Not defined if a+b can diff --git a/src/common/util_format.c b/src/common/util_format.c index dc544a6c2e..8d99138506 100644 --- a/src/common/util_format.c +++ b/src/common/util_format.c @@ -465,7 +465,7 @@ base16_encode(char *dest, size_t destlen, const char *src, size_t srclen) } /** Helper: given a hex digit, return its value, or -1 if it isn't hex. */ -static INLINE int +static inline int hex_decode_digit_(char c) { switch (c) { diff --git a/src/common/util_process.c b/src/common/util_process.c index 849a5c0b63..1e3b02cc9a 100644 --- a/src/common/util_process.c +++ b/src/common/util_process.c @@ -45,13 +45,13 @@ struct waitpid_callback_t { unsigned running; }; -static INLINE unsigned int +static inline unsigned int process_map_entry_hash_(const waitpid_callback_t *ent) { return (unsigned) ent->pid; } -static INLINE unsigned int +static inline unsigned int process_map_entries_eq_(const waitpid_callback_t *a, const waitpid_callback_t *b) { diff --git a/src/ext/csiphash.c b/src/ext/csiphash.c index 27c5358ebe..b60f73a7ff 100644 --- a/src/ext/csiphash.c +++ b/src/ext/csiphash.c @@ -97,65 +97,48 @@ #endif uint64_t siphash24(const void *src, unsigned long src_sz, const struct sipkey *key) { + const uint8_t *m = src; uint64_t k0 = key->k0; uint64_t k1 = key->k1; - uint64_t b = (uint64_t)src_sz << 56; -#ifdef UNALIGNED_OK - const uint64_t *in = (uint64_t*)src; -#else - /* On platforms where alignment matters, if 'in' is a pointer to a - * datatype that must be aligned, the compiler is allowed to - * generate code that assumes that it is aligned as such. - */ - const uint8_t *in = (uint8_t *)src; -#endif - - uint64_t t; - uint8_t *pt, *m; + uint64_t last7 = (uint64_t)(src_sz & 0xff) << 56; + size_t i, blocks; uint64_t v0 = k0 ^ 0x736f6d6570736575ULL; uint64_t v1 = k1 ^ 0x646f72616e646f6dULL; uint64_t v2 = k0 ^ 0x6c7967656e657261ULL; uint64_t v3 = k1 ^ 0x7465646279746573ULL; - while (src_sz >= 8) { + for (i = 0, blocks = (src_sz & ~7); i < blocks; i+= 8) { #ifdef UNALIGNED_OK - uint64_t mi = _le64toh(*in); - in += 1; + uint64_t mi = _le64toh(*(m + i)); #else uint64_t mi; - memcpy(&mi, in, 8); + memcpy(&mi, m + i, 8); mi = _le64toh(mi); - in += 8; #endif - src_sz -= 8; v3 ^= mi; DOUBLE_ROUND(v0,v1,v2,v3); v0 ^= mi; } - t = 0; pt = (uint8_t*)&t; m = (uint8_t*)in; - switch (src_sz) { - case 7: pt[6] = m[6]; - case 6: pt[5] = m[5]; - case 5: pt[4] = m[4]; -#ifdef UNALIGNED_OK - case 4: *((uint32_t*)&pt[0]) = *((uint32_t*)&m[0]); break; -#else - case 4: pt[3] = m[3]; -#endif - case 3: pt[2] = m[2]; - case 2: pt[1] = m[1]; - case 1: pt[0] = m[0]; + switch (src_sz - blocks) { + case 7: last7 |= (uint64_t)m[i + 6] << 48; + case 6: last7 |= (uint64_t)m[i + 5] << 40; + case 5: last7 |= (uint64_t)m[i + 4] << 32; + case 4: last7 |= (uint64_t)m[i + 3] << 24; + case 3: last7 |= (uint64_t)m[i + 2] << 16; + case 2: last7 |= (uint64_t)m[i + 1] << 8; + case 1: last7 |= (uint64_t)m[i + 0] ; + case 0: + default:; } - b |= _le64toh(t); - - v3 ^= b; + v3 ^= last7; DOUBLE_ROUND(v0,v1,v2,v3); - v0 ^= b; v2 ^= 0xff; + v0 ^= last7; + v2 ^= 0xff; DOUBLE_ROUND(v0,v1,v2,v3); DOUBLE_ROUND(v0,v1,v2,v3); - return (v0 ^ v1) ^ (v2 ^ v3); + return v0 ^ v1 ^ v2 ^ v3; } diff --git a/src/ext/ed25519/donna/ed25519_tor.c b/src/ext/ed25519/donna/ed25519_tor.c index 12493f7d14..ac726ba045 100644 --- a/src/ext/ed25519/donna/ed25519_tor.c +++ b/src/ext/ed25519/donna/ed25519_tor.c @@ -148,8 +148,7 @@ ed25519_donna_seckey(unsigned char *sk) { ed25519_secret_key seed; - if (crypto_strongest_rand(seed, 32)) - return -1; + crypto_strongest_rand(seed, 32); ed25519_extsk(sk, seed); diff --git a/src/ext/ed25519/ref10/randombytes.h b/src/ext/ed25519/ref10/randombytes.h index fc709fcefc..8bf31631f0 100644 --- a/src/ext/ed25519/ref10/randombytes.h +++ b/src/ext/ed25519/ref10/randombytes.h @@ -1,4 +1,4 @@ /* Added for Tor. */ #include "crypto.h" #define randombytes(b, n) \ - (crypto_strongest_rand((b), (n))) + (crypto_strongest_rand((b), (n)), 0) diff --git a/src/ext/eventdns.c b/src/ext/eventdns.c index a0c7ff29fa..37d8a7a3df 100644 --- a/src/ext/eventdns.c +++ b/src/ext/eventdns.c @@ -805,7 +805,7 @@ reply_handle(struct evdns_request *const req, u16 flags, u32 ttl, struct reply * } } -static INLINE int +static inline int name_parse(u8 *packet, int length, int *idx, char *name_out, size_t name_out_len) { int name_end = -1; int j = *idx; diff --git a/src/ext/ht.h b/src/ext/ht.h index 19a67a6a41..28d1fe49d5 100644 --- a/src/ext/ht.h +++ b/src/ext/ht.h @@ -61,7 +61,7 @@ #define HT_INIT(name, head) name##_HT_INIT(head) #define HT_REP_IS_BAD_(name, head) name##_HT_REP_IS_BAD_(head) /* Helper: */ -static INLINE unsigned +static inline unsigned ht_improve_hash(unsigned h) { /* Aim to protect against poor hash functions by adding logic here @@ -75,7 +75,7 @@ ht_improve_hash(unsigned h) #if 0 /** Basic string hash function, from Java standard String.hashCode(). */ -static INLINE unsigned +static inline unsigned ht_string_hash(const char *s) { unsigned h = 0; @@ -90,7 +90,7 @@ ht_string_hash(const char *s) #if 0 /** Basic string hash function, from Python's str.__hash__() */ -static INLINE unsigned +static inline unsigned ht_string_hash(const char *s) { unsigned h; @@ -143,7 +143,7 @@ ht_string_hash(const char *s) int name##_HT_GROW(struct name *ht, unsigned min_capacity); \ void name##_HT_CLEAR(struct name *ht); \ int name##_HT_REP_IS_BAD_(const struct name *ht); \ - static INLINE void \ + static inline void \ name##_HT_INIT(struct name *head) { \ head->hth_table_length = 0; \ head->hth_table = NULL; \ @@ -153,7 +153,7 @@ ht_string_hash(const char *s) } \ /* Helper: returns a pointer to the right location in the table \ * 'head' to find or insert the element 'elm'. */ \ - static INLINE struct type ** \ + static inline struct type ** \ name##_HT_FIND_P_(struct name *head, struct type *elm) \ { \ struct type **p; \ @@ -169,7 +169,7 @@ ht_string_hash(const char *s) } \ /* Return a pointer to the element in the table 'head' matching 'elm', \ * or NULL if no such element exists */ \ - ATTR_UNUSED static INLINE struct type * \ + ATTR_UNUSED static inline struct type * \ name##_HT_FIND(const struct name *head, struct type *elm) \ { \ struct type **p; \ @@ -180,7 +180,7 @@ ht_string_hash(const char *s) } \ /* Insert the element 'elm' into the table 'head'. Do not call this \ * function if the table might already contain a matching element. */ \ - ATTR_UNUSED static INLINE void \ + ATTR_UNUSED static inline void \ name##_HT_INSERT(struct name *head, struct type *elm) \ { \ struct type **p; \ @@ -195,7 +195,7 @@ ht_string_hash(const char *s) /* Insert the element 'elm' into the table 'head'. If there already \ * a matching element in the table, replace that element and return \ * it. */ \ - ATTR_UNUSED static INLINE struct type * \ + ATTR_UNUSED static inline struct type * \ name##_HT_REPLACE(struct name *head, struct type *elm) \ { \ struct type **p, *r; \ @@ -216,7 +216,7 @@ ht_string_hash(const char *s) } \ /* Remove any element matching 'elm' from the table 'head'. If such \ * an element is found, return it; otherwise return NULL. */ \ - ATTR_UNUSED static INLINE struct type * \ + ATTR_UNUSED static inline struct type * \ name##_HT_REMOVE(struct name *head, struct type *elm) \ { \ struct type **p, *r; \ @@ -234,7 +234,7 @@ ht_string_hash(const char *s) * using 'data' as its second argument. If the function returns \ * nonzero, remove the most recently examined element before invoking \ * the function again. */ \ - ATTR_UNUSED static INLINE void \ + ATTR_UNUSED static inline void \ name##_HT_FOREACH_FN(struct name *head, \ int (*fn)(struct type *, void *), \ void *data) \ @@ -260,7 +260,7 @@ ht_string_hash(const char *s) /* Return a pointer to the first element in the table 'head', under \ * an arbitrary order. This order is stable under remove operations, \ * but not under others. If the table is empty, return NULL. */ \ - ATTR_UNUSED static INLINE struct type ** \ + ATTR_UNUSED static inline struct type ** \ name##_HT_START(struct name *head) \ { \ unsigned b = 0; \ @@ -279,7 +279,7 @@ ht_string_hash(const char *s) * NULL. If 'elm' is to be removed from the table, you must call \ * this function for the next value before you remove it. \ */ \ - ATTR_UNUSED static INLINE struct type ** \ + ATTR_UNUSED static inline struct type ** \ name##_HT_NEXT(struct name *head, struct type **elm) \ { \ if ((*elm)->field.hte_next) { \ @@ -299,7 +299,7 @@ ht_string_hash(const char *s) return NULL; \ } \ } \ - ATTR_UNUSED static INLINE struct type ** \ + ATTR_UNUSED static inline struct type ** \ name##_HT_NEXT_RMV(struct name *head, struct type **elm) \ { \ unsigned h = HT_ELT_HASH_(*elm, field, hashfn); \ diff --git a/src/or/buffers.c b/src/or/buffers.c index cc2f6f409b..4696bec8f4 100644 --- a/src/or/buffers.c +++ b/src/or/buffers.c @@ -78,7 +78,7 @@ static int parse_socks_client(const uint8_t *data, size_t datalen, /** Return the next character in <b>chunk</b> onto which data can be appended. * If the chunk is full, this might be off the end of chunk->mem. */ -static INLINE char * +static inline char * CHUNK_WRITE_PTR(chunk_t *chunk) { return chunk->data + chunk->datalen; @@ -86,7 +86,7 @@ CHUNK_WRITE_PTR(chunk_t *chunk) /** Return the number of bytes that can be written onto <b>chunk</b> without * running out of space. */ -static INLINE size_t +static inline size_t CHUNK_REMAINING_CAPACITY(const chunk_t *chunk) { return (chunk->mem + chunk->memlen) - (chunk->data + chunk->datalen); @@ -94,7 +94,7 @@ CHUNK_REMAINING_CAPACITY(const chunk_t *chunk) /** Move all bytes stored in <b>chunk</b> to the front of <b>chunk</b>->mem, * to free up space at the end. */ -static INLINE void +static inline void chunk_repack(chunk_t *chunk) { if (chunk->datalen && chunk->data != &chunk->mem[0]) { @@ -118,7 +118,7 @@ chunk_free_unchecked(chunk_t *chunk) total_bytes_allocated_in_chunks -= CHUNK_ALLOC_SIZE(chunk->memlen); tor_free(chunk); } -static INLINE chunk_t * +static inline chunk_t * chunk_new_with_alloc_size(size_t alloc) { chunk_t *ch; @@ -136,7 +136,7 @@ chunk_new_with_alloc_size(size_t alloc) /** Expand <b>chunk</b> until it can hold <b>sz</b> bytes, and return a * new pointer to <b>chunk</b>. Old pointers are no longer valid. */ -static INLINE chunk_t * +static inline chunk_t * chunk_grow(chunk_t *chunk, size_t sz) { off_t offset; @@ -165,7 +165,7 @@ chunk_grow(chunk_t *chunk, size_t sz) /** Return the allocation size we'd like to use to hold <b>target</b> * bytes. */ -static INLINE size_t +static inline size_t preferred_chunk_size(size_t target) { size_t sz = MIN_CHUNK_ALLOC; @@ -255,7 +255,7 @@ buf_get_first_chunk_data(const buf_t *buf, const char **cp, size_t *sz) #endif /** Remove the first <b>n</b> bytes from buf. */ -static INLINE void +static inline void buf_remove_from_front(buf_t *buf, size_t n) { tor_assert(buf->datalen >= n); @@ -452,7 +452,7 @@ buf_get_total_allocation(void) * <b>chunk</b> (which must be on <b>buf</b>). If we get an EOF, set * *<b>reached_eof</b> to 1. Return -1 on error, 0 on eof or blocking, * and the number of bytes read otherwise. */ -static INLINE int +static inline int read_to_chunk(buf_t *buf, chunk_t *chunk, tor_socket_t fd, size_t at_most, int *reached_eof, int *socket_error) { @@ -488,7 +488,7 @@ read_to_chunk(buf_t *buf, chunk_t *chunk, tor_socket_t fd, size_t at_most, /** As read_to_chunk(), but return (negative) error code on error, blocking, * or TLS, and the number of bytes read otherwise. */ -static INLINE int +static inline int read_to_chunk_tls(buf_t *buf, chunk_t *chunk, tor_tls_t *tls, size_t at_most) { @@ -611,7 +611,7 @@ read_to_buf_tls(tor_tls_t *tls, size_t at_most, buf_t *buf) * the bytes written from *<b>buf_flushlen</b>. Return the number of bytes * written on success, 0 on blocking, -1 on failure. */ -static INLINE int +static inline int flush_chunk(tor_socket_t s, buf_t *buf, chunk_t *chunk, size_t sz, size_t *buf_flushlen) { @@ -646,7 +646,7 @@ flush_chunk(tor_socket_t s, buf_t *buf, chunk_t *chunk, size_t sz, * bytes written from *<b>buf_flushlen</b>. Return the number of bytes * written on success, and a TOR_TLS error code on failure or blocking. */ -static INLINE int +static inline int flush_chunk_tls(tor_tls_t *tls, buf_t *buf, chunk_t *chunk, size_t sz, size_t *buf_flushlen) { @@ -797,7 +797,7 @@ write_to_buf(const char *string, size_t string_len, buf_t *buf) /** Helper: copy the first <b>string_len</b> bytes from <b>buf</b> * onto <b>string</b>. */ -static INLINE void +static inline void peek_from_buf(char *string, size_t string_len, const buf_t *buf) { chunk_t *chunk; @@ -842,7 +842,7 @@ fetch_from_buf(char *string, size_t string_len, buf_t *buf) /** True iff the cell command <b>command</b> is one that implies a * variable-length cell in Tor link protocol <b>linkproto</b>. */ -static INLINE int +static inline int cell_command_is_var_length(uint8_t command, int linkproto) { /* If linkproto is v2 (2), CELL_VERSIONS is the only variable-length cells @@ -1083,7 +1083,7 @@ buf_find_pos_of_char(char ch, buf_pos_t *out) /** Advance <b>pos</b> by a single character, if there are any more characters * in the buffer. Returns 0 on success, -1 on failure. */ -static INLINE int +static inline int buf_pos_inc(buf_pos_t *pos) { ++pos->pos; diff --git a/src/or/channel.c b/src/or/channel.c index 21522a5303..46e833854b 100644 --- a/src/or/channel.c +++ b/src/or/channel.c @@ -127,13 +127,13 @@ typedef struct channel_idmap_entry_s { TOR_LIST_HEAD(channel_list_s, channel_s) channel_list; } channel_idmap_entry_t; -static INLINE unsigned +static inline unsigned channel_idmap_hash(const channel_idmap_entry_t *ent) { return (unsigned) siphash24g(ent->digest, DIGEST_LEN); } -static INLINE int +static inline int channel_idmap_eq(const channel_idmap_entry_t *a, const channel_idmap_entry_t *b) { diff --git a/src/or/channel.h b/src/or/channel.h index 2b38ca7e19..5fa2aa8ab7 100644 --- a/src/or/channel.h +++ b/src/or/channel.h @@ -531,7 +531,7 @@ channel_t * channel_next_with_digest(channel_t *chan); CHANNEL_IS_OPEN(chan) || \ CHANNEL_IS_MAINT(chan)) -static INLINE int +static inline int channel_is_in_state(channel_t *chan, channel_state_t state) { return chan->state == state; diff --git a/src/or/channeltls.c b/src/or/channeltls.c index c90f569233..f0333e8da8 100644 --- a/src/or/channeltls.c +++ b/src/or/channeltls.c @@ -1663,30 +1663,9 @@ channel_tls_process_netinfo_cell(cell_t *cell, channel_tls_t *chan) #define NETINFO_NOTICE_SKEW 3600 if (labs(apparent_skew) > NETINFO_NOTICE_SKEW && router_get_by_id_digest(chan->conn->identity_digest)) { - char dbuf[64]; - int severity; - /*XXXX be smarter about when everybody says we are skewed. */ - if (router_digest_is_trusted_dir(chan->conn->identity_digest)) - severity = LOG_WARN; - else - severity = LOG_INFO; - format_time_interval(dbuf, sizeof(dbuf), apparent_skew); - log_fn(severity, LD_GENERAL, - "Received NETINFO cell with skewed time from " - "server at %s:%d. It seems that our clock is %s by %s, or " - "that theirs is %s. Tor requires an accurate clock to work: " - "please check your time and date settings.", - chan->conn->base_.address, - (int)(chan->conn->base_.port), - apparent_skew > 0 ? "ahead" : "behind", - dbuf, - apparent_skew > 0 ? "behind" : "ahead"); - if (severity == LOG_WARN) /* only tell the controller if an authority */ - control_event_general_status(LOG_WARN, - "CLOCK_SKEW SKEW=%ld SOURCE=OR:%s:%d", - apparent_skew, - chan->conn->base_.address, - chan->conn->base_.port); + int trusted = router_digest_is_trusted_dir(chan->conn->identity_digest); + clock_skew_warning(TO_CONN(chan->conn), apparent_skew, trusted, LD_GENERAL, + "NETINFO cell", "OR"); } /* XXX maybe act on my_apparent_addr, if the source is sufficiently diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index 0688398f6d..719d27caa9 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -498,6 +498,14 @@ circuit_handle_first_hop(origin_circuit_t *circ) tor_assert(firsthop); tor_assert(firsthop->extend_info); + /* XX/teor - does tor ever need build a circuit directly to itself? */ + if (tor_addr_is_internal(&firsthop->extend_info->addr, 0) && + !get_options()->ExtendAllowPrivateAddresses) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Client asked me to connect directly to a private address"); + return -END_CIRC_REASON_TORPROTOCOL; + } + /* now see if we're already connected to the first OR in 'route' */ log_debug(LD_CIRC,"Looking for firsthop '%s'", fmt_addrport(&firsthop->extend_info->addr, @@ -737,7 +745,7 @@ inform_testing_reachability(void) /** Return true iff we should send a create_fast cell to start building a given * circuit */ -static INLINE int +static inline int should_use_create_fast_for_circuit(origin_circuit_t *circ) { const or_options_t *options = get_options(); diff --git a/src/or/circuitlist.c b/src/or/circuitlist.c index 716024df6a..dcbeb1e2bb 100644 --- a/src/or/circuitlist.c +++ b/src/or/circuitlist.c @@ -44,11 +44,16 @@ static smartlist_t *global_circuitlist = NULL; /** A list of all the circuits in CIRCUIT_STATE_CHAN_WAIT. */ static smartlist_t *circuits_pending_chans = NULL; +/** A list of all the circuits that have been marked with + * circuit_mark_for_close and which are waiting for circuit_about_to_free. */ +static smartlist_t *circuits_pending_close = NULL; + static void circuit_free_cpath_node(crypt_path_t *victim); static void cpath_ref_decref(crypt_path_reference_t *cpath_ref); //static void circuit_set_rend_token(or_circuit_t *circ, int is_rend_circ, // const uint8_t *token); static void circuit_clear_rend_token(or_circuit_t *circ); +static void circuit_about_to_free(circuit_t *circ); /********* END VARIABLES ************/ @@ -66,7 +71,7 @@ typedef struct chan_circid_circuit_map_t { /** Helper for hash tables: compare the channel and circuit ID for a and * b, and return less than, equal to, or greater than zero appropriately. */ -static INLINE int +static inline int chan_circid_entries_eq_(chan_circid_circuit_map_t *a, chan_circid_circuit_map_t *b) { @@ -75,7 +80,7 @@ chan_circid_entries_eq_(chan_circid_circuit_map_t *a, /** Helper: return a hash based on circuit ID and the pointer value of * chan in <b>a</b>. */ -static INLINE unsigned int +static inline unsigned int chan_circid_entry_hash_(chan_circid_circuit_map_t *a) { /* Try to squeze the siphash input into 8 bytes to save any extra siphash @@ -451,16 +456,27 @@ circuit_count_pending_on_channel(channel_t *chan) void circuit_close_all_marked(void) { + if (circuits_pending_close == NULL) + return; + smartlist_t *lst = circuit_get_global_list(); - SMARTLIST_FOREACH_BEGIN(lst, circuit_t *, circ) { - /* Fix up index if SMARTLIST_DEL_CURRENT just moved this one. */ - circ->global_circuitlist_idx = circ_sl_idx; - if (circ->marked_for_close) { - circ->global_circuitlist_idx = -1; - circuit_free(circ); - SMARTLIST_DEL_CURRENT(lst, circ); + SMARTLIST_FOREACH_BEGIN(circuits_pending_close, circuit_t *, circ) { + tor_assert(circ->marked_for_close); + + /* Remove it from the circuit list. */ + int idx = circ->global_circuitlist_idx; + smartlist_del(lst, idx); + if (idx < smartlist_len(lst)) { + circuit_t *replacement = smartlist_get(lst, idx); + replacement->global_circuitlist_idx = idx; } + circ->global_circuitlist_idx = -1; + + circuit_about_to_free(circ); + circuit_free(circ); } SMARTLIST_FOREACH_END(circ); + + smartlist_clear(circuits_pending_close); } /** Return the head of the global linked list of circuits. */ @@ -895,6 +911,9 @@ circuit_free_all(void) smartlist_free(circuits_pending_chans); circuits_pending_chans = NULL; + smartlist_free(circuits_pending_close); + circuits_pending_close = NULL; + { chan_circid_circuit_map_t **elt, **next, *c; for (elt = HT_START(chan_circid_map, &chan_circid_map); @@ -1030,7 +1049,7 @@ circuit_get_by_global_id(uint32_t id) * If <b>found_entry_out</b> is provided, set it to true if we have a * placeholder entry for circid/chan, and leave it unset otherwise. */ -static INLINE circuit_t * +static inline circuit_t * circuit_get_by_circid_channel_impl(circid_t circ_id, channel_t *chan, int *found_entry_out) { @@ -1703,6 +1722,39 @@ circuit_mark_for_close_, (circuit_t *circ, int reason, int line, reason = END_CIRC_REASON_NONE; } + circ->marked_for_close = line; + circ->marked_for_close_file = file; + circ->marked_for_close_reason = reason; + circ->marked_for_close_orig_reason = orig_reason; + + if (!CIRCUIT_IS_ORIGIN(circ)) { + or_circuit_t *or_circ = TO_OR_CIRCUIT(circ); + if (or_circ->rend_splice) { + if (!or_circ->rend_splice->base_.marked_for_close) { + /* do this after marking this circuit, to avoid infinite recursion. */ + circuit_mark_for_close(TO_CIRCUIT(or_circ->rend_splice), reason); + } + or_circ->rend_splice = NULL; + } + } + + if (circuits_pending_close == NULL) + circuits_pending_close = smartlist_new(); + + smartlist_add(circuits_pending_close, circ); +} + +/** Called immediately before freeing a marked circuit <b>circ</b>. + * Disconnects the circuit from other data structures, launches events + * as appropriate, and performs other housekeeping. + */ +static void +circuit_about_to_free(circuit_t *circ) +{ + + int reason = circ->marked_for_close_reason; + int orig_reason = circ->marked_for_close_orig_reason; + if (circ->state == CIRCUIT_STATE_ONIONSKIN_PENDING) { onion_pending_remove(TO_OR_CIRCUIT(circ)); } @@ -1726,6 +1778,7 @@ circuit_mark_for_close_, (circuit_t *circ, int reason, int line, (circ->state == CIRCUIT_STATE_OPEN)?CIRC_EVENT_CLOSED:CIRC_EVENT_FAILED, orig_reason); } + if (circ->purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT) { origin_circuit_t *ocirc = TO_ORIGIN_CIRCUIT(circ); int timed_out = (reason == END_CIRC_REASON_TIMEOUT); @@ -1810,20 +1863,6 @@ circuit_mark_for_close_, (circuit_t *circ, int reason, int line, connection_edge_destroy(circ->n_circ_id, conn); ocirc->p_streams = NULL; } - - circ->marked_for_close = line; - circ->marked_for_close_file = file; - - if (!CIRCUIT_IS_ORIGIN(circ)) { - or_circuit_t *or_circ = TO_OR_CIRCUIT(circ); - if (or_circ->rend_splice) { - if (!or_circ->rend_splice->base_.marked_for_close) { - /* do this after marking this circuit, to avoid infinite recursion. */ - circuit_mark_for_close(TO_CIRCUIT(or_circ->rend_splice), reason); - } - or_circ->rend_splice = NULL; - } - } } /** Given a marked circuit <b>circ</b>, aggressively free its cell queues to diff --git a/src/or/circuitmux.c b/src/or/circuitmux.c index a77bffac90..94d1eb66e3 100644 --- a/src/or/circuitmux.c +++ b/src/or/circuitmux.c @@ -186,10 +186,10 @@ struct chanid_circid_muxinfo_t { * Static function declarations */ -static INLINE int +static inline int chanid_circid_entries_eq(chanid_circid_muxinfo_t *a, chanid_circid_muxinfo_t *b); -static INLINE unsigned int +static inline unsigned int chanid_circid_entry_hash(chanid_circid_muxinfo_t *a); static chanid_circid_muxinfo_t * circuitmux_find_map_entry(circuitmux_t *cmux, circuit_t *circ); @@ -199,12 +199,12 @@ circuitmux_make_circuit_active(circuitmux_t *cmux, circuit_t *circ, static void circuitmux_make_circuit_inactive(circuitmux_t *cmux, circuit_t *circ, cell_direction_t direction); -static INLINE void +static inline void circuitmux_move_active_circ_to_tail(circuitmux_t *cmux, circuit_t *circ, cell_direction_t direction); -static INLINE circuit_t ** +static inline circuit_t ** circuitmux_next_active_circ_p(circuitmux_t *cmux, circuit_t *circ); -static INLINE circuit_t ** +static inline circuit_t ** circuitmux_prev_active_circ_p(circuitmux_t *cmux, circuit_t *circ); static void circuitmux_assert_okay_pass_one(circuitmux_t *cmux); static void circuitmux_assert_okay_pass_two(circuitmux_t *cmux); @@ -226,7 +226,7 @@ static int64_t global_destroy_ctr = 0; * used by circuitmux_notify_xmit_cells(). */ -static INLINE void +static inline void circuitmux_move_active_circ_to_tail(circuitmux_t *cmux, circuit_t *circ, cell_direction_t direction) { @@ -306,7 +306,7 @@ circuitmux_move_active_circ_to_tail(circuitmux_t *cmux, circuit_t *circ, circuitmux_assert_okay_paranoid(cmux); } -static INLINE circuit_t ** +static inline circuit_t ** circuitmux_next_active_circ_p(circuitmux_t *cmux, circuit_t *circ) { tor_assert(cmux); @@ -319,7 +319,7 @@ circuitmux_next_active_circ_p(circuitmux_t *cmux, circuit_t *circ) } } -static INLINE circuit_t ** +static inline circuit_t ** circuitmux_prev_active_circ_p(circuitmux_t *cmux, circuit_t *circ) { tor_assert(cmux); @@ -338,7 +338,7 @@ circuitmux_prev_active_circ_p(circuitmux_t *cmux, circuit_t *circ) * than zero appropriately. */ -static INLINE int +static inline int chanid_circid_entries_eq(chanid_circid_muxinfo_t *a, chanid_circid_muxinfo_t *b) { @@ -349,7 +349,7 @@ chanid_circid_entries_eq(chanid_circid_muxinfo_t *a, * Helper: return a hash based on circuit ID and channel ID in a. */ -static INLINE unsigned int +static inline unsigned int chanid_circid_entry_hash(chanid_circid_muxinfo_t *a) { return (((unsigned int)(a->circ_id) << 8) ^ diff --git a/src/or/circuitmux_ewma.c b/src/or/circuitmux_ewma.c index 1c0318de06..0c61fb2ec4 100644 --- a/src/or/circuitmux_ewma.c +++ b/src/or/circuitmux_ewma.c @@ -115,7 +115,7 @@ TO_EWMA_POL_CIRC_DATA(circuitmux_policy_circ_data_t *); * if the cast is impossible. */ -static INLINE ewma_policy_data_t * +static inline ewma_policy_data_t * TO_EWMA_POL_DATA(circuitmux_policy_data_t *pol) { if (!pol) return NULL; @@ -130,7 +130,7 @@ TO_EWMA_POL_DATA(circuitmux_policy_data_t *pol) * and assert if the cast is impossible. */ -static INLINE ewma_policy_circ_data_t * +static inline ewma_policy_circ_data_t * TO_EWMA_POL_CIRC_DATA(circuitmux_policy_circ_data_t *pol) { if (!pol) return NULL; @@ -147,7 +147,7 @@ static int compare_cell_ewma_counts(const void *p1, const void *p2); static unsigned cell_ewma_tick_from_timeval(const struct timeval *now, double *remainder_out); static circuit_t * cell_ewma_to_circuit(cell_ewma_t *ewma); -static INLINE double get_scale_factor(unsigned from_tick, unsigned to_tick); +static inline double get_scale_factor(unsigned from_tick, unsigned to_tick); static cell_ewma_t * pop_first_cell_ewma(ewma_policy_data_t *pol); static void remove_cell_ewma(ewma_policy_data_t *pol, cell_ewma_t *ewma); static void scale_single_cell_ewma(cell_ewma_t *ewma, unsigned cur_tick); @@ -644,7 +644,7 @@ cell_ewma_set_scale_factor(const or_options_t *options, /** Return the multiplier necessary to convert the value of a cell sent in * 'from_tick' to one sent in 'to_tick'. */ -static INLINE double +static inline double get_scale_factor(unsigned from_tick, unsigned to_tick) { /* This math can wrap around, but that's okay: unsigned overflow is diff --git a/src/or/circuituse.c b/src/or/circuituse.c index 00340fd689..e742a5614f 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -1123,7 +1123,7 @@ circuit_build_needed_circs(time_t now) * don't require an exit circuit, review in #13814. * This allows HSs to function in a consensus without exits. */ if (router_have_consensus_path() != CONSENSUS_PATH_UNKNOWN) - connection_ap_attach_pending(); + connection_ap_rescan_and_attach_pending(); /* make sure any hidden services have enough intro points * HS intro point streams only require an internal circuit */ @@ -1475,7 +1475,7 @@ circuit_has_opened(origin_circuit_t *circ) case CIRCUIT_PURPOSE_C_ESTABLISH_REND: rend_client_rendcirc_has_opened(circ); /* Start building an intro circ if we don't have one yet. */ - connection_ap_attach_pending(); + connection_ap_attach_pending(1); /* This isn't a call to circuit_try_attaching_streams because a * circuit in _C_ESTABLISH_REND state isn't connected to its * hidden service yet, thus we can't attach streams to it yet, @@ -1537,14 +1537,14 @@ void circuit_try_attaching_streams(origin_circuit_t *circ) { /* Attach streams to this circuit if we can. */ - connection_ap_attach_pending(); + connection_ap_attach_pending(1); /* The call to circuit_try_clearing_isolation_state here will do * nothing and return 0 if we didn't attach any streams to circ * above. */ if (circuit_try_clearing_isolation_state(circ)) { /* Maybe *now* we can attach some streams to this circuit. */ - connection_ap_attach_pending(); + connection_ap_attach_pending(1); } } @@ -1986,6 +1986,7 @@ circuit_get_open_circ_or_launch(entry_connection_t *conn, "No intro points for '%s': re-fetching service descriptor.", safe_str_client(rend_data->onion_address)); rend_client_refetch_v2_renddesc(rend_data); + connection_ap_mark_as_non_pending_circuit(conn); ENTRY_TO_CONN(conn)->state = AP_CONN_STATE_RENDDESC_WAIT; return 0; } diff --git a/src/or/config.c b/src/or/config.c index fa860af337..7b42c9fdb3 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -212,6 +212,7 @@ static config_var_t option_vars_[] = { V(CookieAuthFile, STRING, NULL), V(CountPrivateBandwidth, BOOL, "0"), V(DataDirectory, FILENAME, NULL), + V(DataDirectoryGroupReadable, BOOL, "0"), V(DisableNetwork, BOOL, "0"), V(DirAllowPrivateAddresses, BOOL, "0"), V(TestingAuthDirTimeToLearnReachability, INTERVAL, "30 minutes"), @@ -312,6 +313,7 @@ static config_var_t option_vars_[] = { V(LogMessageDomains, BOOL, "0"), V(LogTimeGranularity, MSEC_INTERVAL, "1 second"), V(TruncateLogFile, BOOL, "0"), + V(SyslogIdentityTag, STRING, NULL), V(LongLivedPorts, CSV, "21,22,706,1863,5050,5190,5222,5223,6523,6667,6697,8300"), VAR("MapAddress", LINELIST, AddressMap, NULL), @@ -561,7 +563,6 @@ static char *get_bindaddr_from_transport_listen_line(const char *line, static int parse_dir_authority_line(const char *line, dirinfo_type_t required_type, int validate_only); -static void port_cfg_free(port_cfg_t *port); static int parse_ports(or_options_t *options, int validate_only, char **msg_out, int *n_ports_out, int *world_writable_control_socket); @@ -625,8 +626,8 @@ static char *global_dirfrontpagecontents = NULL; static smartlist_t *configured_ports = NULL; /** Return the contents of our frontpage string, or NULL if not configured. */ -const char * -get_dirportfrontpage(void) +MOCK_IMPL(const char*, +get_dirportfrontpage, (void)) { return global_dirfrontpagecontents; } @@ -1186,16 +1187,30 @@ options_act_reversible(const or_options_t *old_options, char **msg) } /* Ensure data directory is private; create if possible. */ + cpd_check_t cpd_opts = running_tor ? CPD_CREATE : CPD_CHECK; + if (options->DataDirectoryGroupReadable) + cpd_opts |= CPD_GROUP_READ; if (check_private_dir(options->DataDirectory, - running_tor ? CPD_CREATE : CPD_CHECK, + cpd_opts, options->User)<0) { tor_asprintf(msg, "Couldn't access/create private data directory \"%s\"", options->DataDirectory); + goto done; /* No need to roll back, since you can't change the value. */ } +#ifndef _WIN32 + if (options->DataDirectoryGroupReadable) { + /* Only new dirs created get new opts, also enforce group read. */ + if (chmod(options->DataDirectory, 0750)) { + log_warn(LD_FS,"Unable to make %s group-readable: %s", + options->DataDirectory, strerror(errno)); + } + } +#endif + /* Bail out at this point if we're not going to be a client or server: * we don't run Tor itself. */ if (!running_tor) @@ -3996,6 +4011,12 @@ options_transition_allowed(const or_options_t *old, return -1; } + if (!opt_streq(old->SyslogIdentityTag, new_val->SyslogIdentityTag)) { + *msg = tor_strdup("While Tor is running, changing " + "SyslogIdentityTag is not allowed."); + return -1; + } + if ((old->HardwareAccel != new_val->HardwareAccel) || !opt_streq(old->AccelName, new_val->AccelName) || !opt_streq(old->AccelDir, new_val->AccelDir)) { @@ -4937,7 +4958,7 @@ options_init_logs(const or_options_t *old_options, or_options_t *options, !strcasecmp(smartlist_get(elts,0), "syslog")) { #ifdef HAVE_SYSLOG_H if (!validate_only) { - add_syslog_log(severity); + add_syslog_log(severity, options->SyslogIdentityTag); } #else log_warn(LD_CONFIG, "Syslog is not supported on this system. Sorry."); @@ -5730,7 +5751,7 @@ parse_dir_fallback_line(const char *line, } /** Allocate and return a new port_cfg_t with reasonable defaults. */ -static port_cfg_t * +STATIC port_cfg_t * port_cfg_new(size_t namelen) { tor_assert(namelen <= SIZE_T_CEILING - sizeof(port_cfg_t) - 1); @@ -5742,7 +5763,7 @@ port_cfg_new(size_t namelen) } /** Free all storage held in <b>port</b> */ -static void +STATIC void port_cfg_free(port_cfg_t *port) { tor_free(port); @@ -5796,9 +5817,9 @@ warn_nonlocal_ext_orports(const smartlist_t *ports, const char *portname) } SMARTLIST_FOREACH_END(port); } -/** Given a list of port_cfg_t in <b>ports</b>, warn any controller port there - * is listening on any non-loopback address. If <b>forbid_nonlocal</b> is - * true, then emit a stronger warning and remove the port from the list. +/** Given a list of port_cfg_t in <b>ports</b>, warn if any controller port + * there is listening on any non-loopback address. If <b>forbid_nonlocal</b> + * is true, then emit a stronger warning and remove the port from the list. */ static void warn_nonlocal_controller_ports(smartlist_t *ports, unsigned forbid_nonlocal) @@ -6666,8 +6687,8 @@ check_server_ports(const smartlist_t *ports, /** Return a list of port_cfg_t for client ports parsed from the * options. */ -const smartlist_t * -get_configured_ports(void) +MOCK_IMPL(const smartlist_t *, +get_configured_ports,(void)) { if (!configured_ports) configured_ports = smartlist_new(); @@ -7329,8 +7350,7 @@ init_cookie_authentication(const char *fname, const char *header, /* Generate the cookie */ *cookie_out = tor_malloc(cookie_len); - if (crypto_rand((char *)*cookie_out, cookie_len) < 0) - goto done; + crypto_rand((char *)*cookie_out, cookie_len); /* Create the string that should be written on the file. */ memcpy(cookie_file_str, header, strlen(header)); diff --git a/src/or/config.h b/src/or/config.h index 0ee1e1a3c4..7e8868804e 100644 --- a/src/or/config.h +++ b/src/or/config.h @@ -14,8 +14,8 @@ #include "testsupport.h" -const char *get_dirportfrontpage(void); -MOCK_DECL(const or_options_t *,get_options,(void)); +MOCK_DECL(const char*, get_dirportfrontpage, (void)); +MOCK_DECL(const or_options_t *, get_options, (void)); or_options_t *get_options_mutable(void); int set_options(or_options_t *new_val, char **msg); void config_free_all(void); @@ -76,7 +76,7 @@ int write_to_data_subdir(const char* subdir, const char* fname, int get_num_cpus(const or_options_t *options); -const smartlist_t *get_configured_ports(void); +MOCK_DECL(const smartlist_t *,get_configured_ports,(void)); int get_first_advertised_port_by_type_af(int listener_type, int address_family); #define get_primary_or_port() \ @@ -140,6 +140,8 @@ smartlist_t *get_options_for_server_transport(const char *transport); extern struct config_format_t options_format; #endif +STATIC port_cfg_t *port_cfg_new(size_t namelen); +STATIC void port_cfg_free(port_cfg_t *port); STATIC void or_options_free(or_options_t *options); STATIC int options_validate(or_options_t *old_options, or_options_t *options, diff --git a/src/or/connection.c b/src/or/connection.c index 78176d3768..bff994d385 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -1597,6 +1597,8 @@ connection_init_accepted_conn(connection_t *conn, break; case CONN_TYPE_AP_TRANS_LISTENER: TO_ENTRY_CONN(conn)->is_transparent_ap = 1; + /* XXXX028 -- is this correct still, with the addition of + * pending_entry_connections ? */ conn->state = AP_CONN_STATE_CIRCUIT_WAIT; return connection_ap_process_transparent(TO_ENTRY_CONN(conn)); case CONN_TYPE_AP_NATD_LISTENER: @@ -1706,10 +1708,13 @@ connection_connect_sockaddr(connection_t *conn, } /** Take conn, make a nonblocking socket; try to connect to - * addr:port (they arrive in *host order*). If fail, return -1 and if + * addr:port (port arrives in *host order*). If fail, return -1 and if * applicable put your best guess about errno into *<b>socket_error</b>. * Else assign s to conn-\>s: if connected return 1, if EAGAIN return 0. * + * addr:port can be different to conn->addr:conn->port if connecting through + * a proxy. + * * address is used to make the logs useful. * * On success, add conn to the list of polled connections. @@ -2381,6 +2386,14 @@ retry_listener_ports(smartlist_t *old_conns, if (port->server_cfg.no_listen) continue; +#ifndef _WIN32 + /* We don't need to be root to create a UNIX socket, so defer until after + * setuid. */ + const or_options_t *options = get_options(); + if (port->is_unix_addr && !geteuid() && strcmp(options->User, "root")) + continue; +#endif + if (port->is_unix_addr) { listensockaddr = (struct sockaddr *) create_unix_sockaddr(port->unix_addr, @@ -4210,7 +4223,8 @@ connection_write_to_buf_impl_,(const char *string, size_t len, } /** Return a connection with given type, address, port, and purpose; - * or NULL if no such connection exists. */ + * or NULL if no such connection exists (or if all such connections are marked + * for close). */ connection_t * connection_get_by_type_addr_port_purpose(int type, const tor_addr_t *addr, uint16_t port, @@ -5012,3 +5026,34 @@ connection_free_all(void) #endif } +/** Log a warning, and possibly emit a control event, that <b>received</b> came + * at a skewed time. <b>trusted</b> indicates that the <b>source</b> was one + * that we had more faith in and therefore the warning level should have higher + * severity. + */ +void +clock_skew_warning(const connection_t *conn, long apparent_skew, int trusted, + log_domain_mask_t domain, const char *received, + const char *source) +{ + char dbuf[64]; + char *ext_source = NULL; + format_time_interval(dbuf, sizeof(dbuf), apparent_skew); + if (conn) + tor_asprintf(&ext_source, "%s:%s:%d", source, conn->address, conn->port); + else + ext_source = tor_strdup(source); + log_fn(trusted ? LOG_WARN : LOG_INFO, domain, + "Received %s with skewed time (%s): " + "It seems that our clock is %s by %s, or that theirs is %s%s. " + "Tor requires an accurate clock to work: please check your time, " + "timezone, and date settings.", received, ext_source, + apparent_skew > 0 ? "ahead" : "behind", dbuf, + apparent_skew > 0 ? "behind" : "ahead", + (!conn || trusted) ? "" : ", or they are sending us the wrong time"); + if (trusted) + control_event_general_status(LOG_WARN, "CLOCK_SKEW SKEW=%ld SOURCE=%s", + apparent_skew, ext_source); + tor_free(ext_source); +} + diff --git a/src/or/connection.h b/src/or/connection.h index b6ff3d7bd6..d416962c0a 100644 --- a/src/or/connection.h +++ b/src/or/connection.h @@ -146,12 +146,12 @@ static void connection_write_to_buf(const char *string, size_t len, /* DOCDOC connection_write_to_buf_zlib */ static void connection_write_to_buf_zlib(const char *string, size_t len, dir_connection_t *conn, int done); -static INLINE void +static inline void connection_write_to_buf(const char *string, size_t len, connection_t *conn) { connection_write_to_buf_impl_(string, len, conn, 0); } -static INLINE void +static inline void connection_write_to_buf_zlib(const char *string, size_t len, dir_connection_t *conn, int done) { @@ -163,7 +163,7 @@ static size_t connection_get_inbuf_len(connection_t *conn); /* DOCDOC connection_get_outbuf_len */ static size_t connection_get_outbuf_len(connection_t *conn); -static INLINE size_t +static inline size_t connection_get_inbuf_len(connection_t *conn) { IF_HAS_BUFFEREVENT(conn, { @@ -173,7 +173,7 @@ connection_get_inbuf_len(connection_t *conn) } } -static INLINE size_t +static inline size_t connection_get_outbuf_len(connection_t *conn) { IF_HAS_BUFFEREVENT(conn, { @@ -210,6 +210,10 @@ int connection_or_nonopen_was_started_here(or_connection_t *conn); void connection_dump_buffer_mem_stats(int severity); void remove_file_if_very_old(const char *fname, time_t now); +void clock_skew_warning(const connection_t *conn, long apparent_skew, + int trusted, log_domain_mask_t domain, + const char *received, const char *source); + #ifdef USE_BUFFEREVENTS int connection_type_uses_bufferevent(connection_t *conn); void connection_configure_bufferevent_callbacks(connection_t *conn); diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index 729ef8a4c7..30dcd13f4a 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -11,6 +11,9 @@ #define CONNECTION_EDGE_PRIVATE #include "or.h" + +#include "backtrace.h" + #include "addressmap.h" #include "buffers.h" #include "channel.h" @@ -503,6 +506,16 @@ connection_edge_finished_connecting(edge_connection_t *edge_conn) return connection_edge_process_inbuf(edge_conn, 1); } +/** A list of all the entry_connection_t * objects that are not marked + * for close, and are in AP_CONN_STATE_CIRCUIT_WAIT. + * + * (Right now, we check in several places to make sure that this list is + * correct. When it's incorrect, we'll fix it, and log a BUG message.) + */ +static smartlist_t *pending_entry_connections = NULL; + +static int untried_pending_connections = 0; + /** Common code to connection_(ap|exit)_about_to_close. */ static void connection_edge_about_to_close(edge_connection_t *edge_conn) @@ -514,6 +527,27 @@ connection_edge_about_to_close(edge_connection_t *edge_conn) conn->marked_for_close_file, conn->marked_for_close); tor_fragile_assert(); } + + if (TO_CONN(edge_conn)->type != CONN_TYPE_AP || + PREDICT_UNLIKELY(NULL == pending_entry_connections)) + return; + + entry_connection_t *entry_conn = EDGE_TO_ENTRY_CONN(edge_conn); + + if (TO_CONN(edge_conn)->state == AP_CONN_STATE_CIRCUIT_WAIT) { + smartlist_remove(pending_entry_connections, entry_conn); + } + +#if 1 + /* Check to make sure that this isn't in pending_entry_connections if it + * didn't actually belong there. */ + if (TO_CONN(edge_conn)->type == CONN_TYPE_AP && + smartlist_contains(pending_entry_connections, entry_conn)) { + log_warn(LD_BUG, "What was %p doing in pending_entry_connections???", + entry_conn); + smartlist_remove(pending_entry_connections, entry_conn); + } +#endif } /** Called when we're about to finally unlink and free an AP (client) @@ -711,26 +745,138 @@ connection_ap_expire_beginning(void) } SMARTLIST_FOREACH_END(base_conn); } -/** Tell any AP streams that are waiting for a new circuit to try again, - * either attaching to an available circ or launching a new one. +/** + * As connection_ap_attach_pending, but first scans the entire connection + * array to see if any elements are missing. */ void -connection_ap_attach_pending(void) +connection_ap_rescan_and_attach_pending(void) { entry_connection_t *entry_conn; smartlist_t *conns = get_connection_array(); + + if (PREDICT_UNLIKELY(NULL == pending_entry_connections)) + pending_entry_connections = smartlist_new(); + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { if (conn->marked_for_close || conn->type != CONN_TYPE_AP || conn->state != AP_CONN_STATE_CIRCUIT_WAIT) continue; + entry_conn = TO_ENTRY_CONN(conn); + if (! smartlist_contains(pending_entry_connections, entry_conn)) { + log_warn(LD_BUG, "Found a connection %p that was supposed to be " + "in pending_entry_connections, but wasn't. No worries; " + "adding it.", + pending_entry_connections); + untried_pending_connections = 1; + connection_ap_mark_as_pending_circuit(entry_conn); + } + + } SMARTLIST_FOREACH_END(conn); + + connection_ap_attach_pending(1); +} + +/** Tell any AP streams that are listed as waiting for a new circuit to try + * again, either attaching to an available circ or launching a new one. + * + * If <b>retry</b> is false, only check the list if it contains at least one + * streams that we have not yet tried to attach to a circuit. + */ +void +connection_ap_attach_pending(int retry) +{ + if (PREDICT_UNLIKELY(!pending_entry_connections)) { + return; + } + + if (untried_pending_connections == 0 && !retry) + return; + + SMARTLIST_FOREACH_BEGIN(pending_entry_connections, + entry_connection_t *, entry_conn) { + connection_t *conn = ENTRY_TO_CONN(entry_conn); + if (conn->marked_for_close) { + SMARTLIST_DEL_CURRENT(pending_entry_connections, entry_conn); + continue; + } + if (conn->magic != ENTRY_CONNECTION_MAGIC) { + log_warn(LD_BUG, "%p has impossible magic value %u", + entry_conn, (unsigned)conn->magic); + SMARTLIST_DEL_CURRENT(pending_entry_connections, entry_conn); + continue; + } + if (conn->state != AP_CONN_STATE_CIRCUIT_WAIT) { + log_warn(LD_BUG, "%p is no longer in circuit_wait. Its current state " + "is %s. Why is it on pending_entry_connections?", + entry_conn, + conn_state_to_string(conn->type, conn->state)); + SMARTLIST_DEL_CURRENT(pending_entry_connections, entry_conn); + continue; + } + if (connection_ap_handshake_attach_circuit(entry_conn) < 0) { if (!conn->marked_for_close) connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_CANT_ATTACH); } - } SMARTLIST_FOREACH_END(conn); + + if (conn->marked_for_close || + conn->type != CONN_TYPE_AP || + conn->state != AP_CONN_STATE_CIRCUIT_WAIT) { + SMARTLIST_DEL_CURRENT(pending_entry_connections, entry_conn); + continue; + } + + tor_assert(conn->magic == ENTRY_CONNECTION_MAGIC); + + } SMARTLIST_FOREACH_END(entry_conn); + + untried_pending_connections = 0; +} + +/** Mark <b>entry_conn</b> as needing to get attached to a circuit. + * + * And <b>entry_conn</b> must be in AP_CONN_STATE_CIRCUIT_WAIT, + * should not already be pending a circuit. The circuit will get + * launched or the connection will get attached the next time we + * call connection_ap_attach_pending(). + */ +void +connection_ap_mark_as_pending_circuit_(entry_connection_t *entry_conn, + const char *fname, int lineno) +{ + connection_t *conn = ENTRY_TO_CONN(entry_conn); + tor_assert(conn->state == AP_CONN_STATE_CIRCUIT_WAIT); + tor_assert(conn->magic == ENTRY_CONNECTION_MAGIC); + if (conn->marked_for_close) + return; + + if (PREDICT_UNLIKELY(NULL == pending_entry_connections)) + pending_entry_connections = smartlist_new(); + + if (PREDICT_UNLIKELY(smartlist_contains(pending_entry_connections, + entry_conn))) { + log_warn(LD_BUG, "What?? pending_entry_connections already contains %p! " + "(called from %s:%d)", + entry_conn, fname, lineno); + log_backtrace(LOG_WARN, LD_BUG, "To debug, this may help."); + return; + } + + untried_pending_connections = 1; + smartlist_add(pending_entry_connections, entry_conn); +} + +/** Mark <b>entry_conn</b> as no longer waiting for a circuit. */ +void +connection_ap_mark_as_non_pending_circuit(entry_connection_t *entry_conn) +{ + if (PREDICT_UNLIKELY(NULL == pending_entry_connections)) + return; + smartlist_remove(pending_entry_connections, entry_conn); } /** Tell any AP streams that are waiting for a one-hop tunnel to @@ -851,12 +997,13 @@ connection_ap_detach_retriable(entry_connection_t *conn, * a tunneled directory connection, then just attach it. */ ENTRY_TO_CONN(conn)->state = AP_CONN_STATE_CIRCUIT_WAIT; circuit_detach_stream(TO_CIRCUIT(circ),ENTRY_TO_EDGE_CONN(conn)); - return connection_ap_handshake_attach_circuit(conn); + connection_ap_mark_as_pending_circuit(conn); } else { + CONNECTION_AP_EXPECT_NONPENDING(conn); ENTRY_TO_CONN(conn)->state = AP_CONN_STATE_CONTROLLER_WAIT; circuit_detach_stream(TO_CIRCUIT(circ),ENTRY_TO_EDGE_CONN(conn)); - return 0; } + return 0; } /** Check if <b>conn</b> is using a dangerous port. Then warn and/or @@ -905,6 +1052,7 @@ connection_ap_rewrite_and_attach_if_allowed(entry_connection_t *conn, const or_options_t *options = get_options(); if (options->LeaveStreamsUnattached) { + CONNECTION_AP_EXPECT_NONPENDING(conn); ENTRY_TO_CONN(conn)->state = AP_CONN_STATE_CONTROLLER_WAIT; return 0; } @@ -1454,10 +1602,12 @@ connection_ap_handshake_rewrite_and_attach(entry_connection_t *conn, /* If we were given a circuit to attach to, try to attach. Otherwise, * try to find a good one and attach to that. */ int rv; - if (circ) - rv = connection_ap_handshake_attach_chosen_circuit(conn, circ, cpath); - else - rv = connection_ap_handshake_attach_circuit(conn); + if (circ) { + rv = connection_ap_handshake_attach_chosen_circuit(conn, circ, cpath); + } else { + connection_ap_mark_as_pending_circuit(conn); + rv = 0; + } /* If the above function returned 0 then we're waiting for a circuit. * if it returned 1, we're attached. Both are okay. But if it returned @@ -1554,6 +1704,7 @@ connection_ap_handshake_rewrite_and_attach(entry_connection_t *conn, * Also, a fetch could have been requested if the onion address was not * found in the cache previously. */ if (refetch_desc || !rend_client_any_intro_points_usable(entry)) { + connection_ap_mark_as_non_pending_circuit(conn); base_conn->state = AP_CONN_STATE_RENDDESC_WAIT; log_info(LD_REND, "Unknown descriptor %s. Fetching.", safe_str_client(rend_data->onion_address)); @@ -1564,11 +1715,7 @@ connection_ap_handshake_rewrite_and_attach(entry_connection_t *conn, /* We have the descriptor so launch a connection to the HS. */ base_conn->state = AP_CONN_STATE_CIRCUIT_WAIT; log_info(LD_REND, "Descriptor is here. Great."); - if (connection_ap_handshake_attach_circuit(conn) < 0) { - if (!base_conn->marked_for_close) - connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); - return -1; - } + connection_ap_mark_as_pending_circuit(conn); return 0; } @@ -2324,12 +2471,7 @@ connection_ap_make_link(connection_t *partner, control_event_stream_status(conn, STREAM_EVENT_NEW, 0); /* attaching to a dirty circuit is fine */ - if (connection_ap_handshake_attach_circuit(conn) < 0) { - if (!base_conn->marked_for_close) - connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); - return NULL; - } - + connection_ap_mark_as_pending_circuit(conn); log_info(LD_APP,"... application connection created and linked."); return conn; } @@ -3478,3 +3620,12 @@ circuit_clear_isolation(origin_circuit_t *circ) circ->socks_username_len = circ->socks_password_len = 0; } +/** Free all storage held in module-scoped variables for connection_edge.c */ +void +connection_edge_free_all(void) +{ + untried_pending_connections = 0; + smartlist_free(pending_entry_connections); + pending_entry_connections = NULL; +} + diff --git a/src/or/connection_edge.h b/src/or/connection_edge.h index 7c0b9c0767..6da51eb0c6 100644 --- a/src/or/connection_edge.h +++ b/src/or/connection_edge.h @@ -64,7 +64,20 @@ int connection_edge_is_rendezvous_stream(edge_connection_t *conn); int connection_ap_can_use_exit(const entry_connection_t *conn, const node_t *exit); void connection_ap_expire_beginning(void); -void connection_ap_attach_pending(void); +void connection_ap_rescan_and_attach_pending(void); +void connection_ap_attach_pending(int retry); +void connection_ap_mark_as_pending_circuit_(entry_connection_t *entry_conn, + const char *file, int line); +#define connection_ap_mark_as_pending_circuit(c) \ + connection_ap_mark_as_pending_circuit_((c), __FILE__, __LINE__) +void connection_ap_mark_as_non_pending_circuit(entry_connection_t *entry_conn); +#define CONNECTION_AP_EXPECT_NONPENDING(c) do { \ + if (ENTRY_TO_CONN(c)->state == AP_CONN_STATE_CIRCUIT_WAIT) { \ + log_warn(LD_BUG, "At %s:%d: %p was unexpectedly in circuit_wait.", \ + __FILE__, __LINE__, (c)); \ + connection_ap_mark_as_non_pending_circuit(c); \ + } \ + } while (0) void connection_ap_fail_onehop(const char *failed_digest, cpath_build_state_t *build_state); void circuit_discard_optional_exit_enclaves(extend_info_t *info); @@ -100,6 +113,8 @@ int connection_edge_update_circuit_isolation(const entry_connection_t *conn, void circuit_clear_isolation(origin_circuit_t *circ); streamid_t get_unique_stream_id_by_circ(origin_circuit_t *circ); +void connection_edge_free_all(void); + /** @name Begin-cell flags * * These flags are used in RELAY_BEGIN cells to change the default behavior diff --git a/src/or/connection_or.c b/src/or/connection_or.c index a967c93aca..73e4d19369 100644 --- a/src/or/connection_or.c +++ b/src/or/connection_or.c @@ -1450,17 +1450,12 @@ connection_tls_continue_handshake(or_connection_t *conn) { int result; check_no_tls_errors(); - again: - if (conn->base_.state == OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING) { - // log_notice(LD_OR, "Renegotiate with %p", conn->tls); - result = tor_tls_renegotiate(conn->tls); - // log_notice(LD_OR, "Result: %d", result); - } else { - tor_assert(conn->base_.state == OR_CONN_STATE_TLS_HANDSHAKING); - // log_notice(LD_OR, "Continue handshake with %p", conn->tls); - result = tor_tls_handshake(conn->tls); - // log_notice(LD_OR, "Result: %d", result); - } + + tor_assert(conn->base_.state == OR_CONN_STATE_TLS_HANDSHAKING); + // log_notice(LD_OR, "Continue handshake with %p", conn->tls); + result = tor_tls_handshake(conn->tls); + // log_notice(LD_OR, "Result: %d", result); + switch (result) { CASE_TOR_TLS_ERROR_ANY: log_info(LD_OR,"tls error [%s]. breaking connection.", @@ -1469,23 +1464,10 @@ connection_tls_continue_handshake(or_connection_t *conn) case TOR_TLS_DONE: if (! tor_tls_used_v1_handshake(conn->tls)) { if (!tor_tls_is_server(conn->tls)) { - if (conn->base_.state == OR_CONN_STATE_TLS_HANDSHAKING) { - if (tor_tls_received_v3_certificate(conn->tls)) { - log_info(LD_OR, "Client got a v3 cert! Moving on to v3 " - "handshake with ciphersuite %s", - tor_tls_get_ciphersuite_name(conn->tls)); - return connection_or_launch_v3_or_handshake(conn); - } else { - log_debug(LD_OR, "Done with initial SSL handshake (client-side)." - " Requesting renegotiation."); - connection_or_change_state(conn, - OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING); - goto again; - } - } - // log_notice(LD_OR,"Done. state was %d.", conn->base_.state); + tor_assert(conn->base_.state == OR_CONN_STATE_TLS_HANDSHAKING); + return connection_or_launch_v3_or_handshake(conn); } else { - /* v2/v3 handshake, but not a client. */ + /* v2/v3 handshake, but we are not a client. */ log_debug(LD_OR, "Done with initial SSL handshake (server-side). " "Expecting renegotiation or VERSIONS cell"); tor_tls_set_renegotiate_callback(conn->tls, @@ -1498,6 +1480,7 @@ connection_tls_continue_handshake(or_connection_t *conn) return 0; } } + tor_assert(tor_tls_is_server(conn->tls)); return connection_tls_finish_handshake(conn); case TOR_TLS_WANTWRITE: connection_start_writing(TO_CONN(conn)); @@ -1533,22 +1516,8 @@ connection_or_handle_event_cb(struct bufferevent *bufev, short event, if (! tor_tls_used_v1_handshake(conn->tls)) { if (!tor_tls_is_server(conn->tls)) { if (conn->base_.state == OR_CONN_STATE_TLS_HANDSHAKING) { - if (tor_tls_received_v3_certificate(conn->tls)) { - log_info(LD_OR, "Client got a v3 cert!"); - if (connection_or_launch_v3_or_handshake(conn) < 0) - connection_or_close_for_error(conn, 0); - return; - } else { - connection_or_change_state(conn, - OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING); - tor_tls_unblock_renegotiation(conn->tls); - if (bufferevent_ssl_renegotiate(conn->base_.bufev)<0) { - log_warn(LD_OR, "Start_renegotiating went badly."); - connection_or_close_for_error(conn, 0); - } - tor_tls_unblock_renegotiation(conn->tls); - return; /* ???? */ - } + if (connection_or_launch_v3_or_handshake(conn) < 0) + connection_or_close_for_error(conn, 0); } } else { const int handshakes = tor_tls_get_num_server_handshakes(conn->tls); @@ -1800,6 +1769,8 @@ connection_tls_finish_handshake(or_connection_t *conn) char digest_rcvd[DIGEST_LEN]; int started_here = connection_or_nonopen_was_started_here(conn); + tor_assert(!started_here); + log_debug(LD_HANDSHAKE,"%s tls handshake on %p with %s done, using " "ciphersuite %s. verifying.", started_here?"outgoing":"incoming", @@ -1815,10 +1786,8 @@ connection_tls_finish_handshake(or_connection_t *conn) if (tor_tls_used_v1_handshake(conn->tls)) { conn->link_proto = 1; - if (!started_here) { - connection_or_init_conn_from_address(conn, &conn->base_.addr, - conn->base_.port, digest_rcvd, 0); - } + connection_or_init_conn_from_address(conn, &conn->base_.addr, + conn->base_.port, digest_rcvd, 0); tor_tls_block_renegotiation(conn->tls); rep_hist_note_negotiated_link_proto(1, started_here); return connection_or_set_state_open(conn); @@ -1826,10 +1795,8 @@ connection_tls_finish_handshake(or_connection_t *conn) connection_or_change_state(conn, OR_CONN_STATE_OR_HANDSHAKING_V2); if (connection_init_or_handshake_state(conn, started_here) < 0) return -1; - if (!started_here) { - connection_or_init_conn_from_address(conn, &conn->base_.addr, - conn->base_.port, digest_rcvd, 0); - } + connection_or_init_conn_from_address(conn, &conn->base_.addr, + conn->base_.port, digest_rcvd, 0); return connection_or_send_versions(conn, 0); } } @@ -1844,7 +1811,6 @@ static int connection_or_launch_v3_or_handshake(or_connection_t *conn) { tor_assert(connection_or_nonopen_was_started_here(conn)); - tor_assert(tor_tls_received_v3_certificate(conn->tls)); circuit_build_times_network_is_live(get_circuit_build_times_mutable()); @@ -2290,8 +2256,7 @@ connection_or_send_auth_challenge_cell(or_connection_t *conn) auth_challenge_cell_t *ac = auth_challenge_cell_new(); - if (crypto_rand((char*)ac->challenge, sizeof(ac->challenge)) < 0) - goto done; + crypto_rand((char*)ac->challenge, sizeof(ac->challenge)); auth_challenge_cell_add_methods(ac, AUTHTYPE_RSA_SHA256_TLSSECRET); auth_challenge_cell_set_n_methods(ac, diff --git a/src/or/control.c b/src/or/control.c index 220e7e514f..66182fe2a4 100644 --- a/src/or/control.c +++ b/src/or/control.c @@ -192,7 +192,7 @@ static void flush_queued_events_cb(evutil_socket_t fd, short what, void *arg); /** Given a control event code for a message event, return the corresponding * log severity. */ -static INLINE int +static inline int event_to_log_severity(int event) { switch (event) { @@ -206,7 +206,7 @@ event_to_log_severity(int event) } /** Given a log severity, return the corresponding control event code. */ -static INLINE int +static inline int log_severity_to_event(int severity) { switch (severity) { @@ -325,7 +325,7 @@ control_event_is_interesting(int event) /** Append a NUL-terminated string <b>s</b> to the end of * <b>conn</b>-\>outbuf. */ -static INLINE void +static inline void connection_write_str_to_buf(const char *s, control_connection_t *conn) { size_t len = strlen(s); @@ -428,7 +428,7 @@ read_escaped_data(const char *data, size_t len, char **out) /** If the first <b>in_len_max</b> characters in <b>start</b> contain a * double-quoted string with escaped characters, return the length of that * string (as encoded, including quotes). Otherwise return -1. */ -static INLINE int +static inline int get_escaped_string_length(const char *start, size_t in_len_max, int *chars_out) { @@ -1927,6 +1927,22 @@ getinfo_helper_dir(control_connection_t *control_conn, *errmsg = "Not found in cache"; return -1; } + } else if (!strcmpstart(question, "hs/service/desc/id/")) { + rend_cache_entry_t *e = NULL; + + question += strlen("hs/service/desc/id/"); + if (strlen(question) != REND_SERVICE_ID_LEN_BASE32) { + *errmsg = "Invalid address"; + return -1; + } + + if (!rend_cache_lookup_v2_desc_as_service(question, &e)) { + /* Descriptor found in cache */ + *answer = tor_strdup(e->desc); + } else { + *errmsg = "Not found in cache"; + return -1; + } } else if (!strcmpstart(question, "md/id/")) { const node_t *node = node_get_by_hex_id(question+strlen("md/id/")); const microdesc_t *md = NULL; @@ -2481,6 +2497,8 @@ static const getinfo_item_t getinfo_items[] = { PREFIX("extra-info/digest/", dir, "Extra-info documents by digest."), PREFIX("hs/client/desc/id", dir, "Hidden Service descriptor in client's cache by onion."), + PREFIX("hs/service/desc/id/", dir, + "Hidden Service descriptor in services's cache by onion."), PREFIX("net/listeners/", listeners, "Bound addresses by type"), ITEM("ns/all", networkstatus, "Brief summary of router status (v2 directory format)"), @@ -2544,6 +2562,12 @@ static const getinfo_item_t getinfo_items[] = { "v3 Networkstatus consensus as retrieved from a DirPort."), ITEM("exit-policy/default", policies, "The default value appended to the configured exit policy."), + ITEM("exit-policy/reject-private/default", policies, + "The default rules appended to the configured exit policy by" + " ExitPolicyRejectPrivate."), + ITEM("exit-policy/reject-private/relay", policies, + "The relay-specific rules appended to the configured exit policy by" + " ExitPolicyRejectPrivate."), ITEM("exit-policy/full", policies, "The entire exit policy of onion router"), ITEM("exit-policy/ipv4", policies, "IPv4 parts of exit policy"), ITEM("exit-policy/ipv6", policies, "IPv6 parts of exit policy"), @@ -2987,6 +3011,7 @@ handle_control_attachstream(control_connection_t *conn, uint32_t len, edge_conn->end_reason = 0; if (tmpcirc) circuit_detach_stream(tmpcirc, edge_conn); + CONNECTION_AP_EXPECT_NONPENDING(ap_conn); TO_CONN(edge_conn)->state = AP_CONN_STATE_CONTROLLER_WAIT; } @@ -3418,8 +3443,7 @@ handle_control_authchallenge(control_connection_t *conn, uint32_t len, tor_free(client_nonce); return -1; } - const int fail = crypto_rand(server_nonce, SAFECOOKIE_SERVER_NONCE_LEN); - tor_assert(!fail); + crypto_rand(server_nonce, SAFECOOKIE_SERVER_NONCE_LEN); /* Now compute and send the server-to-controller response, and the * server's nonce. */ @@ -6233,6 +6257,31 @@ get_desc_id_from_query(const rend_data_t *rend_data, const char *hsdir_fp) return desc_id; } +/** send HS_DESC CREATED event when a local service generates a descriptor. + * + * <b>service_id</b> is the descriptor onion address. + * <b>desc_id_base32</b> is the descriptor ID. + * <b>replica</b> is the the descriptor replica number. + */ +void +control_event_hs_descriptor_created(const char *service_id, + const char *desc_id_base32, + int replica) +{ + if (!service_id || !desc_id_base32) { + log_warn(LD_BUG, "Called with service_digest==%p, " + "desc_id_base32==%p", service_id, desc_id_base32); + return; + } + + send_control_event(EVENT_HS_DESC, + "650 HS_DESC CREATED %s UNKNOWN UNKNOWN %s " + "REPLICA=%d\r\n", + service_id, + desc_id_base32, + replica); +} + /** send HS_DESC upload event. * * <b>service_id</b> is the descriptor onion address. diff --git a/src/or/control.h b/src/or/control.h index fdf7903cb8..1f8e2bcdc6 100644 --- a/src/or/control.h +++ b/src/or/control.h @@ -117,6 +117,9 @@ MOCK_DECL(const char *, node_describe_longname_by_id,(const char *id_digest)); void control_event_hs_descriptor_requested(const rend_data_t *rend_query, const char *desc_id_base32, const char *hs_dir); +void control_event_hs_descriptor_created(const char *service_id, + const char *desc_id_base32, + int replica); void control_event_hs_descriptor_upload(const char *service_id, const char *desc_id_base32, const char *hs_dir); diff --git a/src/or/directory.c b/src/or/directory.c index 9461606f1b..4e5644b854 100644 --- a/src/or/directory.c +++ b/src/or/directory.c @@ -991,10 +991,7 @@ directory_initiate_command_rend(const tor_addr_t *_addr, switch (connection_connect(TO_CONN(conn), conn->base_.address, &addr, dir_port, &socket_error)) { case -1: - connection_dir_request_failed(conn); /* retry if we want */ - /* XXX we only pass 'conn' above, not 'resource', 'payload', - * etc. So in many situations it can't retry! -RD */ - connection_free(TO_CONN(conn)); + connection_mark_for_close(TO_CONN(conn)); return; case 1: /* start flushing conn */ @@ -1598,7 +1595,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) size_t body_len = 0, orig_len = 0; int status_code; time_t date_header = 0; - long delta; + long apparent_skew; compress_method_t compression; int plausible; int skewed = 0; @@ -1657,28 +1654,15 @@ connection_dir_client_reached_eof(dir_connection_t *conn) * and the date header. (We used to check now-date_header, but that's * inaccurate if we spend a lot of time downloading.) */ - delta = conn->base_.timestamp_lastwritten - date_header; - if (labs(delta)>ALLOW_DIRECTORY_TIME_SKEW) { - char dbuf[64]; + apparent_skew = conn->base_.timestamp_lastwritten - date_header; + if (labs(apparent_skew)>ALLOW_DIRECTORY_TIME_SKEW) { int trusted = router_digest_is_trusted_dir(conn->identity_digest); - format_time_interval(dbuf, sizeof(dbuf), delta); - log_fn(trusted ? LOG_WARN : LOG_INFO, - LD_HTTP, - "Received directory with skewed time (server '%s:%d'): " - "It seems that our clock is %s by %s, or that theirs is %s. " - "Tor requires an accurate clock to work: please check your time, " - "timezone, and date settings.", - conn->base_.address, conn->base_.port, - delta>0 ? "ahead" : "behind", dbuf, - delta>0 ? "behind" : "ahead"); + clock_skew_warning(TO_CONN(conn), apparent_skew, trusted, LD_HTTP, + "directory", "DIRSERV"); skewed = 1; /* don't check the recommended-versions line */ - if (trusted) - control_event_general_status(LOG_WARN, - "CLOCK_SKEW SKEW=%ld SOURCE=DIRSERV:%s:%d", - delta, conn->base_.address, conn->base_.port); } else { log_debug(LD_HTTP, "Time on received directory is within tolerance; " - "we are %ld seconds skewed. (That's okay.)", delta); + "we are %ld seconds skewed. (That's okay.)", apparent_skew); } } (void) skewed; /* skewed isn't used yet. */ @@ -2614,7 +2598,7 @@ choose_compression_level(ssize_t n_bytes) * service descriptor. On finding one, write a response into * conn-\>outbuf. If the request is unrecognized, send a 400. * Always return 0. */ -static int +STATIC int directory_handle_command_get(dir_connection_t *conn, const char *headers, const char *req_body, size_t req_body_len) { @@ -2874,7 +2858,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, }); if (global_write_bucket_low(TO_CONN(conn), estimated_len, 2)) { - write_http_status_line(conn, 503, "Directory busy, try again later."); + write_http_status_line(conn, 503, "Directory busy, try again later"); goto vote_done; } write_http_response_header(conn, body_len ? body_len : -1, compressed, @@ -3071,7 +3055,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, len += c->cache_info.signed_descriptor_len); if (global_write_bucket_low(TO_CONN(conn), compressed?len/2:len, 2)) { - write_http_status_line(conn, 503, "Directory busy, try again later."); + write_http_status_line(conn, 503, "Directory busy, try again later"); goto keys_done; } @@ -3398,7 +3382,7 @@ connection_dir_finished_flushing(dir_connection_t *conn) tor_assert(conn->base_.type == CONN_TYPE_DIR); /* Note that we have finished writing the directory response. For direct - * connections this means we're done, for tunneled connections its only + * connections this means we're done; for tunneled connections it's only * an intermediate step. */ if (conn->dirreq_id) geoip_change_dirreq_state(conn->dirreq_id, DIRREQ_TUNNELED, diff --git a/src/or/directory.h b/src/or/directory.h index 4899eb5c8c..274227f412 100644 --- a/src/or/directory.h +++ b/src/or/directory.h @@ -101,7 +101,7 @@ static int download_status_is_ready(download_status_t *dls, time_t now, int max_failures); /** Return true iff, as of <b>now</b>, the resource tracked by <b>dls</b> is * ready to get its download reattempted. */ -static INLINE int +static inline int download_status_is_ready(download_status_t *dls, time_t now, int max_failures) { @@ -111,7 +111,7 @@ download_status_is_ready(download_status_t *dls, time_t now, static void download_status_mark_impossible(download_status_t *dl); /** Mark <b>dl</b> as never downloadable. */ -static INLINE void +static inline void download_status_mark_impossible(download_status_t *dl) { dl->n_download_failures = IMPOSSIBLE_TO_DOWNLOAD; @@ -127,6 +127,10 @@ STATIC int purpose_needs_anonymity(uint8_t dir_purpose, uint8_t router_purpose); STATIC dirinfo_type_t dir_fetch_type(int dir_purpose, int router_purpose, const char *resource); +STATIC int directory_handle_command_get(dir_connection_t *conn, + const char *headers, + const char *req_body, + size_t req_body_len); #endif #endif diff --git a/src/or/dirserv.c b/src/or/dirserv.c index 8d9f166556..39563c3932 100644 --- a/src/or/dirserv.c +++ b/src/or/dirserv.c @@ -797,7 +797,7 @@ list_single_server_status(const routerinfo_t *desc, int is_live) } /* DOCDOC running_long_enough_to_decide_unreachable */ -static INLINE int +static inline int running_long_enough_to_decide_unreachable(void) { return time_of_process_start @@ -1302,7 +1302,7 @@ static uint32_t guard_bandwidth_excluding_exits_kb = 0; /** Helper: estimate the uptime of a router given its stated uptime and the * amount of time since it last stated its stated uptime. */ -static INLINE long +static inline long real_uptime(const routerinfo_t *router, time_t now) { if (now < router->cache_info.published_on) diff --git a/src/or/dirvote.c b/src/or/dirvote.c index d8e6ee2229..0449e9d8d9 100644 --- a/src/or/dirvote.c +++ b/src/or/dirvote.c @@ -3373,8 +3373,8 @@ dirvote_free_all(void) * ==== */ /** Return the body of the consensus that we're currently trying to build. */ -const char * -dirvote_get_pending_consensus(consensus_flavor_t flav) +MOCK_IMPL(const char *, +dirvote_get_pending_consensus, (consensus_flavor_t flav)) { tor_assert(((int)flav) >= 0 && (int)flav < N_CONSENSUS_FLAVORS); return pending_consensuses[flav].body; @@ -3382,8 +3382,8 @@ dirvote_get_pending_consensus(consensus_flavor_t flav) /** Return the signatures that we know for the consensus that we're currently * trying to build. */ -const char * -dirvote_get_pending_detached_signatures(void) +MOCK_IMPL(const char *, +dirvote_get_pending_detached_signatures, (void)) { return pending_consensus_signatures; } diff --git a/src/or/dirvote.h b/src/or/dirvote.h index dca8540870..966d163088 100644 --- a/src/or/dirvote.h +++ b/src/or/dirvote.h @@ -136,8 +136,10 @@ int dirvote_add_signatures(const char *detached_signatures_body, const char **msg_out); /* Item access */ -const char *dirvote_get_pending_consensus(consensus_flavor_t flav); -const char *dirvote_get_pending_detached_signatures(void); +MOCK_DECL(const char*, dirvote_get_pending_consensus, + (consensus_flavor_t flav)); +MOCK_DECL(const char*, dirvote_get_pending_detached_signatures, (void)); + #define DGV_BY_ID 1 #define DGV_INCLUDE_PENDING 2 #define DGV_INCLUDE_PREVIOUS 4 diff --git a/src/or/dns.c b/src/or/dns.c index d71246d61e..3f5dfd2a8a 100644 --- a/src/or/dns.c +++ b/src/or/dns.c @@ -107,13 +107,9 @@ static void dns_found_answer(const char *address, uint8_t query_type, const tor_addr_t *addr, const char *hostname, uint32_t ttl); -static int launch_resolve(cached_resolve_t *resolve); static void add_wildcarded_test_address(const char *address); static int configure_nameservers(int force); static int answer_is_wildcarded(const char *ip); -static int set_exitconn_info_from_resolve(edge_connection_t *exitconn, - const cached_resolve_t *resolve, - char **hostname_out); static int evdns_err_is_transient(int err); static void inform_pending_connections(cached_resolve_t *resolve); static void make_pending_resolve_cached(cached_resolve_t *cached); @@ -138,7 +134,7 @@ static int dns_is_broken_for_ipv6 = 0; /** Function to compare hashed resolves on their addresses; used to * implement hash tables. */ -static INLINE int +static inline int cached_resolves_eq(cached_resolve_t *a, cached_resolve_t *b) { /* make this smarter one day? */ @@ -147,7 +143,7 @@ cached_resolves_eq(cached_resolve_t *a, cached_resolve_t *b) } /** Hash function for cached_resolve objects */ -static INLINE unsigned int +static inline unsigned int cached_resolve_hash(cached_resolve_t *a) { return (unsigned) siphash24g((const uint8_t*)a->address, strlen(a->address)); @@ -859,10 +855,10 @@ dns_resolve_impl,(edge_connection_t *exitconn, int is_resolve, * Return -2 on a transient error, -1 on a permenent error, and 1 on * a successful lookup. */ -static int -set_exitconn_info_from_resolve(edge_connection_t *exitconn, - const cached_resolve_t *resolve, - char **hostname_out) +MOCK_IMPL(STATIC int, +set_exitconn_info_from_resolve,(edge_connection_t *exitconn, + const cached_resolve_t *resolve, + char **hostname_out)) { int ipv4_ok, ipv6_ok, answer_with_ipv4, r; uint32_t begincell_flags; @@ -1130,7 +1126,7 @@ dns_cancel_pending_resolve,(const char *address)) /** Return true iff <b>address</b> is one of the addresses we use to verify * that well-known sites aren't being hijacked by our DNS servers. */ -static INLINE int +static inline int is_test_address(const char *address) { const or_options_t *options = get_options(); @@ -1664,8 +1660,8 @@ launch_one_resolve(const char *address, uint8_t query_type, /** For eventdns: start resolving as necessary to find the target for * <b>exitconn</b>. Returns -1 on error, -2 on transient error, * 0 on "resolve launched." */ -static int -launch_resolve(cached_resolve_t *resolve) +MOCK_IMPL(STATIC int, +launch_resolve,(cached_resolve_t *resolve)) { tor_addr_t a; int r; @@ -2118,5 +2114,18 @@ assert_cache_ok_(void) } }); } + #endif +cached_resolve_t +*dns_get_cache_entry(cached_resolve_t *query) +{ + return HT_FIND(cache_map, &cache_root, query); +} + +void +dns_insert_cache_entry(cached_resolve_t *new_entry) +{ + HT_INSERT(cache_map, &cache_root, new_entry); +} + diff --git a/src/or/dns.h b/src/or/dns.h index 6af7796dbb..c2778b216c 100644 --- a/src/or/dns.h +++ b/src/or/dns.h @@ -42,6 +42,18 @@ uint8_t answer_type,const cached_resolve_t *resolved)); MOCK_DECL(STATIC void,send_resolved_hostname_cell,(edge_connection_t *conn, const char *hostname)); + +cached_resolve_t *dns_get_cache_entry(cached_resolve_t *query); +void dns_insert_cache_entry(cached_resolve_t *new_entry); + +MOCK_DECL(STATIC int, +set_exitconn_info_from_resolve,(edge_connection_t *exitconn, + const cached_resolve_t *resolve, + char **hostname_out)); + +MOCK_DECL(STATIC int, +launch_resolve,(cached_resolve_t *resolve)); + #endif #endif diff --git a/src/or/dnsserv.c b/src/or/dnsserv.c index f7710908bd..ded0d8431b 100644 --- a/src/or/dnsserv.c +++ b/src/or/dnsserv.c @@ -125,6 +125,7 @@ evdns_server_callback(struct evdns_server_request *req, void *data_) /* Make a new dummy AP connection, and attach the request to it. */ entry_conn = entry_connection_new(CONN_TYPE_AP, AF_INET); conn = ENTRY_TO_EDGE_CONN(entry_conn); + CONNECTION_AP_EXPECT_NONPENDING(entry_conn); TO_CONN(conn)->state = AP_CONN_STATE_RESOLVE_WAIT; conn->is_dns_request = 1; @@ -199,6 +200,7 @@ dnsserv_launch_request(const char *name, int reverse, /* Make a new dummy AP connection, and attach the request to it. */ entry_conn = entry_connection_new(CONN_TYPE_AP, AF_INET); conn = ENTRY_TO_EDGE_CONN(entry_conn); + CONNECTION_AP_EXPECT_NONPENDING(entry_conn); conn->base_.state = AP_CONN_STATE_RESOLVE_WAIT; tor_addr_copy(&TO_CONN(conn)->addr, &control_conn->base_.addr); diff --git a/src/or/eventdns_tor.h b/src/or/eventdns_tor.h index 9d51f0960e..f41c5c0099 100644 --- a/src/or/eventdns_tor.h +++ b/src/or/eventdns_tor.h @@ -12,9 +12,6 @@ typedef unsigned int uint; #ifndef HAVE_U_CHAR typedef unsigned char u_char; #endif -#ifdef _WIN32 -#define inline __inline -#endif #include "torint.h" /* These are for debugging possible memory leaks. */ diff --git a/src/or/ext_orport.c b/src/or/ext_orport.c index e8c8aa60a4..f159f7d0a6 100644 --- a/src/or/ext_orport.c +++ b/src/or/ext_orport.c @@ -193,8 +193,7 @@ handle_client_auth_nonce(const char *client_nonce, size_t client_nonce_len, return -1; /* Get our nonce */ - if (crypto_rand(server_nonce, EXT_OR_PORT_AUTH_NONCE_LEN) < 0) - return -1; + crypto_rand(server_nonce, EXT_OR_PORT_AUTH_NONCE_LEN); { /* set up macs */ size_t hmac_s_msg_len = strlen(EXT_OR_PORT_AUTH_SERVER_TO_CLIENT_CONST) + diff --git a/src/or/fp_pair.c b/src/or/fp_pair.c index 42bebcd847..c863d4176c 100644 --- a/src/or/fp_pair.c +++ b/src/or/fp_pair.c @@ -21,7 +21,7 @@ struct fp_pair_map_s { */ /** Compare fp_pair_entry_t objects by key value. */ -static INLINE int +static inline int fp_pair_map_entries_eq(const fp_pair_map_entry_t *a, const fp_pair_map_entry_t *b) { @@ -29,7 +29,7 @@ fp_pair_map_entries_eq(const fp_pair_map_entry_t *a, } /** Return a hash value for an fp_pair_entry_t. */ -static INLINE unsigned int +static inline unsigned int fp_pair_map_entry_hash(const fp_pair_map_entry_t *a) { tor_assert(sizeof(a->key) == DIGEST_LEN*2); diff --git a/src/or/geoip.c b/src/or/geoip.c index 120ce479cc..3ef1672f52 100644 --- a/src/or/geoip.c +++ b/src/or/geoip.c @@ -18,7 +18,6 @@ #include "geoip.h" #include "routerlist.h" -static void clear_geoip_db(void); static void init_geoip_countries(void); /** An entry from the GeoIP IPv4 file: maps an IPv4 range to a country. */ @@ -483,7 +482,7 @@ static HT_HEAD(clientmap, clientmap_entry_t) client_history = HT_INITIALIZER(); /** Hashtable helper: compute a hash of a clientmap_entry_t. */ -static INLINE unsigned +static inline unsigned clientmap_entry_hash(const clientmap_entry_t *a) { unsigned h = (unsigned) tor_addr_hash(&a->addr); @@ -494,7 +493,7 @@ clientmap_entry_hash(const clientmap_entry_t *a) return h; } /** Hashtable helper: compare two clientmap_entry_t values for equality. */ -static INLINE int +static inline int clientmap_entries_eq(const clientmap_entry_t *a, const clientmap_entry_t *b) { if (strcmp_opt(a->transport_name, b->transport_name)) @@ -970,7 +969,7 @@ geoip_get_dirreq_history(dirreq_type_t type) &ent->completion_time); if (time_diff == 0) time_diff = 1; /* Avoid DIV/0; "instant" answers are impossible - * by law of nature or something, but a milisecond + * by law of nature or something, but a millisecond * is a bit greater than "instantly" */ bytes_per_second = (uint32_t)(1000 * ent->response_size / time_diff); dltimes[ent_sl_idx] = bytes_per_second; @@ -1207,9 +1206,9 @@ geoip_format_dirreq_stats(time_t now) { char t[ISO_TIME_LEN+1]; int i; - char *v3_ips_string, *v3_reqs_string, *v3_direct_dl_string, - *v3_tunneled_dl_string; - char *result; + char *v3_ips_string = NULL, *v3_reqs_string = NULL, + *v3_direct_dl_string = NULL, *v3_tunneled_dl_string = NULL; + char *result = NULL; if (!start_of_dirreq_stats_interval) return NULL; /* Not initialized. */ @@ -1666,7 +1665,7 @@ getinfo_helper_geoip(control_connection_t *control_conn, } /** Release all storage held by the GeoIP databases and country list. */ -static void +STATIC void clear_geoip_db(void) { if (geoip_countries) { diff --git a/src/or/geoip.h b/src/or/geoip.h index 8a3486c7ac..3f1bba01f8 100644 --- a/src/or/geoip.h +++ b/src/or/geoip.h @@ -18,6 +18,7 @@ STATIC int geoip_parse_entry(const char *line, sa_family_t family); STATIC int geoip_get_country_by_ipv4(uint32_t ipaddr); STATIC int geoip_get_country_by_ipv6(const struct in6_addr *addr); +STATIC void clear_geoip_db(void); #endif int should_record_bridge_info(const or_options_t *options); int geoip_load_file(sa_family_t family, const char *filename); diff --git a/src/or/hibernate.c b/src/or/hibernate.c index 356e11f6ec..5f727e27d4 100644 --- a/src/or/hibernate.c +++ b/src/or/hibernate.c @@ -490,7 +490,7 @@ reset_accounting(time_t now) } /** Return true iff we should save our bandwidth usage to disk. */ -static INLINE int +static inline int time_to_record_bandwidth_usage(time_t now) { /* Note every 600 sec */ diff --git a/src/or/include.am b/src/or/include.am index a3ac49c5d6..1180239c89 100644 --- a/src/or/include.am +++ b/src/or/include.am @@ -63,6 +63,7 @@ LIBTOR_A_SOURCES = \ src/or/onion_fast.c \ src/or/onion_tap.c \ src/or/transports.c \ + src/or/periodic.c \ src/or/policies.c \ src/or/reasons.c \ src/or/relay.c \ @@ -170,6 +171,7 @@ ORHEADERS = \ src/or/onion_tap.h \ src/or/or.h \ src/or/transports.h \ + src/or/periodic.h \ src/or/policies.h \ src/or/reasons.h \ src/or/relay.h \ diff --git a/src/or/keypin.c b/src/or/keypin.c index 047d2b069b..574a76d51e 100644 --- a/src/or/keypin.c +++ b/src/or/keypin.c @@ -57,14 +57,14 @@ static HT_HEAD(edmap, keypin_ent_st) the_ed_map = HT_INITIALIZER(); /** Hashtable helper: compare two keypin table entries and return true iff * they have the same RSA key IDs. */ -static INLINE int +static inline int keypin_ents_eq_rsa(const keypin_ent_t *a, const keypin_ent_t *b) { return tor_memeq(a->rsa_id, b->rsa_id, sizeof(a->rsa_id)); } /** Hashtable helper: hash a keypin table entries based on its RSA key ID */ -static INLINE unsigned +static inline unsigned keypin_ent_hash_rsa(const keypin_ent_t *a) { return (unsigned) siphash24g(a->rsa_id, sizeof(a->rsa_id)); @@ -72,14 +72,14 @@ return (unsigned) siphash24g(a->rsa_id, sizeof(a->rsa_id)); /** Hashtable helper: compare two keypin table entries and return true iff * they have the same ed25519 keys */ -static INLINE int +static inline int keypin_ents_eq_ed(const keypin_ent_t *a, const keypin_ent_t *b) { return tor_memeq(a->ed25519_key, b->ed25519_key, sizeof(a->ed25519_key)); } /** Hashtable helper: hash a keypin table entries based on its ed25519 key */ -static INLINE unsigned +static inline unsigned keypin_ent_hash_ed(const keypin_ent_t *a) { return (unsigned) siphash24g(a->ed25519_key, sizeof(a->ed25519_key)); diff --git a/src/or/main.c b/src/or/main.c index 9b3dbb5586..527e2b1ffe 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -44,6 +44,7 @@ #include "nodelist.h" #include "ntmain.h" #include "onion.h" +#include "periodic.h" #include "policies.h" #include "transports.h" #include "relay.h" @@ -499,8 +500,7 @@ connection_in_array(connection_t *conn) return smartlist_contains(connection_array, conn); } -/** Set <b>*array</b> to an array of all connections, and <b>*n</b> - * to the length of the array. <b>*array</b> and <b>*n</b> must not +/** Set <b>*array</b> to an array of all connections. <b>*array</b> must not * be modified. */ smartlist_t * @@ -1227,39 +1227,85 @@ get_signewnym_epoch(void) return newnym_epoch; } -typedef struct { - time_t last_rotated_x509_certificate; - time_t check_v3_certificate; - time_t check_listeners; - time_t download_networkstatus; - time_t try_getting_descriptors; - time_t reset_descriptor_failures; - time_t add_entropy; - time_t write_bridge_status_file; - time_t downrate_stability; - time_t save_stability; - time_t clean_caches; - time_t recheck_bandwidth; - time_t check_for_expired_networkstatus; - time_t write_stats_files; - time_t write_bridge_stats; - time_t check_port_forwarding; - time_t launch_reachability_tests; - time_t retry_dns_init; - time_t next_heartbeat; - time_t check_descriptor; - /** When do we next launch DNS wildcarding checks? */ - time_t check_for_correct_dns; - /** When do we next make sure our Ed25519 keys aren't about to expire? */ - time_t check_ed_keys; - -} time_to_t; - -static time_to_t time_to = { - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 +/** True iff we have initialized all the members of <b>periodic_events</b>. + * Used to prevent double-initialization. */ +static int periodic_events_initialized = 0; + +/* Declare all the timer callback functions... */ +#undef CALLBACK +#define CALLBACK(name) \ + static int name ## _callback(time_t, const or_options_t *) +CALLBACK(rotate_onion_key); +CALLBACK(check_ed_keys); +CALLBACK(launch_descriptor_fetches); +CALLBACK(reset_descriptor_failures); +CALLBACK(rotate_x509_certificate); +CALLBACK(add_entropy); +CALLBACK(launch_reachability_tests); +CALLBACK(downrate_stability); +CALLBACK(save_stability); +CALLBACK(check_authority_cert); +CALLBACK(check_expired_networkstatus); +CALLBACK(write_stats_file); +CALLBACK(record_bridge_stats); +CALLBACK(clean_caches); +CALLBACK(rend_cache_failure_clean); +CALLBACK(retry_dns); +CALLBACK(check_descriptor); +CALLBACK(check_for_reachability_bw); +CALLBACK(fetch_networkstatus); +CALLBACK(retry_listeners); +CALLBACK(expire_old_ciruits_serverside); +CALLBACK(check_dns_honesty); +CALLBACK(write_bridge_ns); +CALLBACK(check_fw_helper_app); +CALLBACK(heartbeat); + +#undef CALLBACK + +/* Now we declare an array of periodic_event_item_t for each periodic event */ +#define CALLBACK(name) PERIODIC_EVENT(name) + +static periodic_event_item_t periodic_events[] = { + CALLBACK(rotate_onion_key), + CALLBACK(check_ed_keys), + CALLBACK(launch_descriptor_fetches), + CALLBACK(reset_descriptor_failures), + CALLBACK(rotate_x509_certificate), + CALLBACK(add_entropy), + CALLBACK(launch_reachability_tests), + CALLBACK(downrate_stability), + CALLBACK(save_stability), + CALLBACK(check_authority_cert), + CALLBACK(check_expired_networkstatus), + CALLBACK(write_stats_file), + CALLBACK(record_bridge_stats), + CALLBACK(clean_caches), + CALLBACK(rend_cache_failure_clean), + CALLBACK(retry_dns), + CALLBACK(check_descriptor), + CALLBACK(check_for_reachability_bw), + CALLBACK(fetch_networkstatus), + CALLBACK(retry_listeners), + CALLBACK(expire_old_ciruits_serverside), + CALLBACK(check_dns_honesty), + CALLBACK(write_bridge_ns), + CALLBACK(check_fw_helper_app), + CALLBACK(heartbeat), + END_OF_PERIODIC_EVENTS }; - -/** Reset all the time_to's so we'll do all our actions again as if we +#undef CALLBACK + +/* These are pointers to members of periodic_events[] that are used to + * implement particular callbacks. We keep them separate here so that we + * can access them by name. We also keep them inside periodic_events[] + * so that we can implement "reset all timers" in a reasaonable way. */ +static periodic_event_item_t *check_descriptor_event=NULL; +static periodic_event_item_t *fetch_networkstatus_event=NULL; +static periodic_event_item_t *launch_descriptor_fetches_event=NULL; +static periodic_event_item_t *check_dns_honesty_event=NULL; + +/** Reset all the periodic events so we'll do all our actions again as if we * just started up. * Useful if our clock just moved back a long time from the future, * so we don't wait until that future arrives again before acting. @@ -1267,7 +1313,77 @@ static time_to_t time_to = { void reset_all_main_loop_timers(void) { - memset(&time_to, 0, sizeof(time_to)); + int i; + for (i = 0; periodic_events[i].name; ++i) { + periodic_event_reschedule(&periodic_events[i]); + } +} + +/** Return the member of periodic_events[] whose name is <b>name</b>. + * Return NULL if no such event is found. + */ +static periodic_event_item_t * +find_periodic_event(const char *name) +{ + int i; + for (i = 0; periodic_events[i].name; ++i) { + if (strcmp(name, periodic_events[i].name) == 0) + return &periodic_events[i]; + } + return NULL; +} + +/** Helper, run one second after setup: + * Initializes all members of periodic_events and starts them running. + * + * (We do this one second after setup for backward-compatibility reasons; + * it might not actually be necessary.) */ +static void +initialize_periodic_events_cb(evutil_socket_t fd, short events, void *data) +{ + (void) fd; + (void) events; + (void) data; + int i; + for (i = 0; periodic_events[i].name; ++i) { + periodic_event_launch(&periodic_events[i]); + } +} + +/** Set up all the members of periodic_events[], and configure them all to be + * launched from a callback. */ +STATIC void +initialize_periodic_events(void) +{ + tor_assert(periodic_events_initialized == 0); + periodic_events_initialized = 1; + + int i; + for (i = 0; periodic_events[i].name; ++i) { + periodic_event_setup(&periodic_events[i]); + } + +#define NAMED_CALLBACK(name) \ + STMT_BEGIN name ## _event = find_periodic_event( #name ); STMT_END + + NAMED_CALLBACK(check_descriptor); + NAMED_CALLBACK(fetch_networkstatus); + NAMED_CALLBACK(launch_descriptor_fetches); + NAMED_CALLBACK(check_dns_honesty); + + struct timeval one_second = { 1, 0 }; + event_base_once(tor_libevent_get_base(), -1, EV_TIMEOUT, + initialize_periodic_events_cb, NULL, + &one_second); +} + +STATIC void +teardown_periodic_events(void) +{ + int i; + for (i = 0; periodic_events[i].name; ++i) { + periodic_event_destroy(&periodic_events[i]); + } } /** @@ -1278,7 +1394,8 @@ reset_all_main_loop_timers(void) void reschedule_descriptor_update_check(void) { - time_to.check_descriptor = 0; + tor_assert(check_descriptor_event); + periodic_event_reschedule(check_descriptor_event); } /** @@ -1288,8 +1405,22 @@ reschedule_descriptor_update_check(void) void reschedule_directory_downloads(void) { - time_to.download_networkstatus = 0; - time_to.try_getting_descriptors = 0; + tor_assert(fetch_networkstatus_event); + tor_assert(launch_descriptor_fetches_event); + + periodic_event_reschedule(fetch_networkstatus_event); + periodic_event_reschedule(launch_descriptor_fetches_event); +} + +static inline int +safe_timer_diff(time_t now, time_t next) +{ + if (next > now) { + tor_assert(next - now <= INT_MAX); + return (int)(next - now); + } else { + return 1; + } } /** Perform regular maintenance tasks. This function gets run once per @@ -1298,13 +1429,8 @@ reschedule_directory_downloads(void) static void run_scheduled_events(time_t now) { - static int should_init_bridge_stats = 1; const or_options_t *options = get_options(); - int is_server = server_mode(options); - int i; - int have_dir_info; - /* 0. See if we've been asked to shut down and our timeout has * expired; or if our bandwidth limits are exhausted and we * should hibernate; or if it's time to wake up from hibernation. @@ -1322,12 +1448,98 @@ run_scheduled_events(time_t now) /* 0c. If we've deferred log messages for the controller, handle them now */ flush_pending_log_callbacks(); + if (options->UseBridges && !options->DisableNetwork) { + fetch_bridge_descriptors(options, now); + } + + if (accounting_is_enabled(options)) { + accounting_run_housekeeping(now); + } + + if (authdir_mode_v3(options)) { + dirvote_act(options, now); + } + + /* 3a. Every second, we examine pending circuits and prune the + * ones which have been pending for more than a few seconds. + * We do this before step 4, so it can try building more if + * it's not comfortable with the number of available circuits. + */ + /* (If our circuit build timeout can ever become lower than a second (which + * it can't, currently), we should do this more often.) */ + circuit_expire_building(); + + /* 3b. Also look at pending streams and prune the ones that 'began' + * a long time ago but haven't gotten a 'connected' yet. + * Do this before step 4, so we can put them back into pending + * state to be picked up by the new circuit. + */ + connection_ap_expire_beginning(); + + /* 3c. And expire connections that we've held open for too long. + */ + connection_expire_held_open(); + + /* 4. Every second, we try a new circuit if there are no valid + * circuits. Every NewCircuitPeriod seconds, we expire circuits + * that became dirty more than MaxCircuitDirtiness seconds ago, + * and we make a new circ if there are no clean circuits. + */ + const int have_dir_info = router_have_minimum_dir_info(); + if (have_dir_info && !net_is_disabled()) { + circuit_build_needed_circs(now); + } else { + circuit_expire_old_circs_as_needed(now); + } + + /* 5. We do housekeeping for each connection... */ + connection_or_set_bad_connections(NULL, 0); + int i; + for (i=0;i<smartlist_len(connection_array);i++) { + run_connection_housekeeping(i, now); + } + + /* 6. And remove any marked circuits... */ + circuit_close_all_marked(); + + /* 7. And upload service descriptors if necessary. */ + if (have_completed_a_circuit() && !net_is_disabled()) { + rend_consider_services_upload(now); + rend_consider_descriptor_republication(); + } + + /* 8. and blow away any connections that need to die. have to do this now, + * because if we marked a conn for close and left its socket -1, then + * we'll pass it to poll/select and bad things will happen. + */ + close_closeable_connections(); + + /* 8b. And if anything in our state is ready to get flushed to disk, we + * flush it. */ + or_state_save(now); + + /* 8c. Do channel cleanup just like for connections */ + channel_run_cleanup(); + channel_listener_run_cleanup(); + + /* 11b. check pending unconfigured managed proxies */ + if (!net_is_disabled() && pt_proxies_configuration_pending()) + pt_configure_remaining_proxies(); +} + +static int +rotate_onion_key_callback(time_t now, const or_options_t *options) +{ /* 1a. Every MIN_ONION_KEY_LIFETIME seconds, rotate the onion keys, * shut down and restart all cpuworkers, and update the directory if * necessary. */ - if (is_server && - get_onion_key_set_at()+MIN_ONION_KEY_LIFETIME < now) { + if (server_mode(options)) { + time_t rotation_time = get_onion_key_set_at()+MIN_ONION_KEY_LIFETIME; + if (rotation_time > now) { + return safe_timer_diff(now, rotation_time); + } + log_info(LD_GENERAL,"Rotating onion key."); rotate_onion_key(); cpuworkers_rotate_keyinfo(); @@ -1336,9 +1548,15 @@ run_scheduled_events(time_t now) } if (advertised_server_mode() && !options->DisableNetwork) router_upload_dir_desc_to_dirservers(0); + return MIN_ONION_KEY_LIFETIME; } + return PERIODIC_EVENT_NO_UPDATE; +} - if (is_server && time_to.check_ed_keys < now) { +static int +check_ed_keys_callback(time_t now, const or_options_t *options) +{ + if (server_mode(options)) { if (should_make_new_ed_keys(options, now)) { if (load_ed_keys(options, now) < 0 || generate_ed_link_cert(options, now)) { @@ -1347,199 +1565,255 @@ run_scheduled_events(time_t now) exit(0); } } - time_to.check_ed_keys = now + 30; + return 30; } + return PERIODIC_EVENT_NO_UPDATE; +} - if (!should_delay_dir_fetches(options, NULL) && - time_to.try_getting_descriptors < now) { - update_all_descriptor_downloads(now); - update_extrainfo_downloads(now); - if (router_have_minimum_dir_info()) - time_to.try_getting_descriptors = now + LAZY_DESCRIPTOR_RETRY_INTERVAL; - else - time_to.try_getting_descriptors = now + GREEDY_DESCRIPTOR_RETRY_INTERVAL; - } +static int +launch_descriptor_fetches_callback(time_t now, const or_options_t *options) +{ + if (should_delay_dir_fetches(options, NULL)) + return PERIODIC_EVENT_NO_UPDATE; - if (time_to.reset_descriptor_failures < now) { - router_reset_descriptor_download_failures(); - time_to.reset_descriptor_failures = - now + DESCRIPTOR_FAILURE_RESET_INTERVAL; - } + update_all_descriptor_downloads(now); + update_extrainfo_downloads(now); + if (router_have_minimum_dir_info()) + return LAZY_DESCRIPTOR_RETRY_INTERVAL; + else + return GREEDY_DESCRIPTOR_RETRY_INTERVAL; +} - if (options->UseBridges && !options->DisableNetwork) - fetch_bridge_descriptors(options, now); +static int +reset_descriptor_failures_callback(time_t now, const or_options_t *options) +{ + (void)now; + (void)options; + router_reset_descriptor_download_failures(); + return DESCRIPTOR_FAILURE_RESET_INTERVAL; +} + +static int +rotate_x509_certificate_callback(time_t now, const or_options_t *options) +{ + static int first = 1; + (void)now; + (void)options; + if (first) { + first = 0; + return MAX_SSL_KEY_LIFETIME_INTERNAL; + } /* 1b. Every MAX_SSL_KEY_LIFETIME_INTERNAL seconds, we change our * TLS context. */ - if (!time_to.last_rotated_x509_certificate) - time_to.last_rotated_x509_certificate = now; - if (time_to.last_rotated_x509_certificate + - MAX_SSL_KEY_LIFETIME_INTERNAL < now) { - log_info(LD_GENERAL,"Rotating tls context."); - if (router_initialize_tls_context() < 0) { - log_warn(LD_BUG, "Error reinitializing TLS context"); - /* XXX is it a bug here, that we just keep going? -RD */ - } - time_to.last_rotated_x509_certificate = now; - /* We also make sure to rotate the TLS connections themselves if they've - * been up for too long -- but that's done via is_bad_for_new_circs in - * connection_run_housekeeping() above. */ + log_info(LD_GENERAL,"Rotating tls context."); + if (router_initialize_tls_context() < 0) { + log_warn(LD_BUG, "Error reinitializing TLS context"); + tor_assert(0); } - if (time_to.add_entropy < now) { - if (time_to.add_entropy) { - /* We already seeded once, so don't die on failure. */ - crypto_seed_rng(); - } -/** How often do we add more entropy to OpenSSL's RNG pool? */ -#define ENTROPY_INTERVAL (60*60) - time_to.add_entropy = now + ENTROPY_INTERVAL; + /* We also make sure to rotate the TLS connections themselves if they've + * been up for too long -- but that's done via is_bad_for_new_circs in + * run_connection_housekeeping() above. */ + return MAX_SSL_KEY_LIFETIME_INTERNAL; +} + +static int +add_entropy_callback(time_t now, const or_options_t *options) +{ + (void)now; + (void)options; + /* We already seeded once, so don't die on failure. */ + if (crypto_seed_rng() < 0) { + log_warn(LD_GENERAL, "Tried to re-seed RNG, but failed. We already " + "seeded once, though, so we won't exit here."); } - /* 1c. If we have to change the accounting interval or record - * bandwidth used in this accounting interval, do so. */ - if (accounting_is_enabled(options)) - accounting_run_housekeeping(now); + /** How often do we add more entropy to OpenSSL's RNG pool? */ +#define ENTROPY_INTERVAL (60*60) + return ENTROPY_INTERVAL; +} - if (time_to.launch_reachability_tests < now && - (authdir_mode_tests_reachability(options)) && - !net_is_disabled()) { - time_to.launch_reachability_tests = now + REACHABILITY_TEST_INTERVAL; +static int +launch_reachability_tests_callback(time_t now, const or_options_t *options) +{ + if (authdir_mode_tests_reachability(options) && + !net_is_disabled()) { /* try to determine reachability of the other Tor relays */ dirserv_test_reachability(now); } + return REACHABILITY_TEST_INTERVAL; +} +static int +downrate_stability_callback(time_t now, const or_options_t *options) +{ + (void)options; /* 1d. Periodically, we discount older stability information so that new * stability info counts more, and save the stability information to disk as * appropriate. */ - if (time_to.downrate_stability < now) - time_to.downrate_stability = rep_hist_downrate_old_runs(now); + time_t next = rep_hist_downrate_old_runs(now); + return safe_timer_diff(now, next); +} + +static int +save_stability_callback(time_t now, const or_options_t *options) +{ if (authdir_mode_tests_reachability(options)) { - if (time_to.save_stability < now) { - if (time_to.save_stability && rep_hist_record_mtbf_data(now, 1)<0) { - log_warn(LD_GENERAL, "Couldn't store mtbf data."); - } -#define SAVE_STABILITY_INTERVAL (30*60) - time_to.save_stability = now + SAVE_STABILITY_INTERVAL; + if (rep_hist_record_mtbf_data(now, 1)<0) { + log_warn(LD_GENERAL, "Couldn't store mtbf data."); } } +#define SAVE_STABILITY_INTERVAL (30*60) + return SAVE_STABILITY_INTERVAL; +} +static int +check_authority_cert_callback(time_t now, const or_options_t *options) +{ + (void)now; + (void)options; /* 1e. Periodically, if we're a v3 authority, we check whether our cert is * close to expiring and warn the admin if it is. */ - if (time_to.check_v3_certificate < now) { - v3_authority_check_key_expiry(); + v3_authority_check_key_expiry(); #define CHECK_V3_CERTIFICATE_INTERVAL (5*60) - time_to.check_v3_certificate = now + CHECK_V3_CERTIFICATE_INTERVAL; - } + return CHECK_V3_CERTIFICATE_INTERVAL; +} +static int +check_expired_networkstatus_callback(time_t now, const or_options_t *options) +{ + (void)options; /* 1f. Check whether our networkstatus has expired. */ - if (time_to.check_for_expired_networkstatus < now) { - networkstatus_t *ns = networkstatus_get_latest_consensus(); - /*XXXX RD: This value needs to be the same as REASONABLY_LIVE_TIME in - * networkstatus_get_reasonably_live_consensus(), but that value is way - * way too high. Arma: is the bridge issue there resolved yet? -NM */ + networkstatus_t *ns = networkstatus_get_latest_consensus(); + /*XXXX RD: This value needs to be the same as REASONABLY_LIVE_TIME in + * networkstatus_get_reasonably_live_consensus(), but that value is way + * way too high. Arma: is the bridge issue there resolved yet? -NM */ #define NS_EXPIRY_SLOP (24*60*60) - if (ns && ns->valid_until < now+NS_EXPIRY_SLOP && - router_have_minimum_dir_info()) { - router_dir_info_changed(); - } -#define CHECK_EXPIRED_NS_INTERVAL (2*60) - time_to.check_for_expired_networkstatus = now + CHECK_EXPIRED_NS_INTERVAL; + if (ns && ns->valid_until < now+NS_EXPIRY_SLOP && + router_have_minimum_dir_info()) { + router_dir_info_changed(); } +#define CHECK_EXPIRED_NS_INTERVAL (2*60) + return CHECK_EXPIRED_NS_INTERVAL; +} +static int +write_stats_file_callback(time_t now, const or_options_t *options) +{ /* 1g. Check whether we should write statistics to disk. */ - if (time_to.write_stats_files < now) { #define CHECK_WRITE_STATS_INTERVAL (60*60) - time_t next_time_to_write_stats_files = (time_to.write_stats_files > 0 ? - time_to.write_stats_files : now) + CHECK_WRITE_STATS_INTERVAL; - if (options->CellStatistics) { - time_t next_write = - rep_hist_buffer_stats_write(time_to.write_stats_files); - if (next_write && next_write < next_time_to_write_stats_files) - next_time_to_write_stats_files = next_write; - } - if (options->DirReqStatistics) { - time_t next_write = geoip_dirreq_stats_write(time_to.write_stats_files); - if (next_write && next_write < next_time_to_write_stats_files) - next_time_to_write_stats_files = next_write; - } - if (options->EntryStatistics) { - time_t next_write = geoip_entry_stats_write(time_to.write_stats_files); - if (next_write && next_write < next_time_to_write_stats_files) - next_time_to_write_stats_files = next_write; - } - if (options->HiddenServiceStatistics) { - time_t next_write = rep_hist_hs_stats_write(time_to.write_stats_files); - if (next_write && next_write < next_time_to_write_stats_files) - next_time_to_write_stats_files = next_write; - } - if (options->ExitPortStatistics) { - time_t next_write = rep_hist_exit_stats_write(time_to.write_stats_files); - if (next_write && next_write < next_time_to_write_stats_files) - next_time_to_write_stats_files = next_write; - } - if (options->ConnDirectionStatistics) { - time_t next_write = rep_hist_conn_stats_write(time_to.write_stats_files); - if (next_write && next_write < next_time_to_write_stats_files) - next_time_to_write_stats_files = next_write; - } - if (options->BridgeAuthoritativeDir) { - time_t next_write = rep_hist_desc_stats_write(time_to.write_stats_files); - if (next_write && next_write < next_time_to_write_stats_files) - next_time_to_write_stats_files = next_write; - } - time_to.write_stats_files = next_time_to_write_stats_files; + time_t next_time_to_write_stats_files = now + CHECK_WRITE_STATS_INTERVAL; + if (options->CellStatistics) { + time_t next_write = + rep_hist_buffer_stats_write(now); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->DirReqStatistics) { + time_t next_write = geoip_dirreq_stats_write(now); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->EntryStatistics) { + time_t next_write = geoip_entry_stats_write(now); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->HiddenServiceStatistics) { + time_t next_write = rep_hist_hs_stats_write(now); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->ExitPortStatistics) { + time_t next_write = rep_hist_exit_stats_write(now); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->ConnDirectionStatistics) { + time_t next_write = rep_hist_conn_stats_write(now); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->BridgeAuthoritativeDir) { + time_t next_write = rep_hist_desc_stats_write(now); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; } + return safe_timer_diff(now, next_time_to_write_stats_files); +} + +static int +record_bridge_stats_callback(time_t now, const or_options_t *options) +{ + static int should_init_bridge_stats = 1; + /* 1h. Check whether we should write bridge statistics to disk. */ if (should_record_bridge_info(options)) { - if (time_to.write_bridge_stats < now) { - if (should_init_bridge_stats) { - /* (Re-)initialize bridge statistics. */ + if (should_init_bridge_stats) { + /* (Re-)initialize bridge statistics. */ geoip_bridge_stats_init(now); - time_to.write_bridge_stats = now + WRITE_STATS_INTERVAL; should_init_bridge_stats = 0; - } else { - /* Possibly write bridge statistics to disk and ask when to write - * them next time. */ - time_to.write_bridge_stats = geoip_bridge_stats_write( - time_to.write_bridge_stats); - } + return WRITE_STATS_INTERVAL; + } else { + /* Possibly write bridge statistics to disk and ask when to write + * them next time. */ + time_t next = geoip_bridge_stats_write(now); + return safe_timer_diff(now, next); } } else if (!should_init_bridge_stats) { /* Bridge mode was turned off. Ensure that stats are re-initialized * next time bridge mode is turned on. */ should_init_bridge_stats = 1; } + return PERIODIC_EVENT_NO_UPDATE; +} +static int +clean_caches_callback(time_t now, const or_options_t *options) +{ /* Remove old information from rephist and the rend cache. */ - if (time_to.clean_caches < now) { - rep_history_clean(now - options->RephistTrackTime); - rend_cache_clean(now); - rend_cache_clean_v2_descs_as_dir(now, 0); - microdesc_cache_rebuild(NULL, 0); + rep_history_clean(now - options->RephistTrackTime); + rend_cache_clean(now, REND_CACHE_TYPE_CLIENT); + rend_cache_clean(now, REND_CACHE_TYPE_SERVICE); + rend_cache_clean_v2_descs_as_dir(now, 0); + microdesc_cache_rebuild(NULL, 0); #define CLEAN_CACHES_INTERVAL (30*60) - time_to.clean_caches = now + CLEAN_CACHES_INTERVAL; - } + return CLEAN_CACHES_INTERVAL; +} + +static int +rend_cache_failure_clean_callback(time_t now, const or_options_t *options) +{ + (void)options; /* We don't keep entries that are more than five minutes old so we try to * clean it as soon as we can since we want to make sure the client waits * as little as possible for reachability reasons. */ rend_cache_failure_clean(now); + return 30; +} +static int +retry_dns_callback(time_t now, const or_options_t *options) +{ + (void)now; #define RETRY_DNS_INTERVAL (10*60) /* If we're a server and initializing dns failed, retry periodically. */ - if (time_to.retry_dns_init < now) { - time_to.retry_dns_init = now + RETRY_DNS_INTERVAL; - if (is_server && has_dns_init_failed()) - dns_init(); - } + if (server_mode(options) && has_dns_init_failed()) + dns_init(); + return RETRY_DNS_INTERVAL; +} /* 2. Periodically, we consider force-uploading our descriptor * (if we've passed our internal checks). */ +static int +check_descriptor_callback(time_t now, const or_options_t *options) +{ /** How often do we check whether part of our router info has changed in a * way that would require an upload? That includes checking whether our IP * address has changed. */ @@ -1547,185 +1821,167 @@ run_scheduled_events(time_t now) /* 2b. Once per minute, regenerate and upload the descriptor if the old * one is inaccurate. */ - if (time_to.check_descriptor < now && !options->DisableNetwork) { - static int dirport_reachability_count = 0; - time_to.check_descriptor = now + CHECK_DESCRIPTOR_INTERVAL; + if (!options->DisableNetwork) { check_descriptor_bandwidth_changed(now); check_descriptor_ipaddress_changed(now); mark_my_descriptor_dirty_if_too_old(now); consider_publishable_server(0); - /* also, check religiously for reachability, if it's within the first - * 20 minutes of our uptime. */ - if (is_server && - (have_completed_a_circuit() || !any_predicted_circuits(now)) && - !we_are_hibernating()) { - if (stats_n_seconds_working < TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { - consider_testing_reachability(1, dirport_reachability_count==0); - if (++dirport_reachability_count > 5) - dirport_reachability_count = 0; - } else if (time_to.recheck_bandwidth < now) { - /* If we haven't checked for 12 hours and our bandwidth estimate is - * low, do another bandwidth test. This is especially important for - * bridges, since they might go long periods without much use. */ - const routerinfo_t *me = router_get_my_routerinfo(); - if (time_to.recheck_bandwidth && me && - me->bandwidthcapacity < me->bandwidthrate && - me->bandwidthcapacity < 51200) { - reset_bandwidth_test(); - } -#define BANDWIDTH_RECHECK_INTERVAL (12*60*60) - time_to.recheck_bandwidth = now + BANDWIDTH_RECHECK_INTERVAL; - } - } - /* If any networkstatus documents are no longer recent, we need to * update all the descriptors' running status. */ /* Remove dead routers. */ + /* XXXX This doesn't belong here, but it was here in the pre- + * XXXX refactoring code. */ routerlist_remove_old_routers(); } - /* 2c. Every minute (or every second if TestingTorNetwork), check - * whether we want to download any networkstatus documents. */ + return CHECK_DESCRIPTOR_INTERVAL; +} -/* How often do we check whether we should download network status - * documents? */ -#define networkstatus_dl_check_interval(o) ((o)->TestingTorNetwork ? 1 : 60) +static int +check_for_reachability_bw_callback(time_t now, const or_options_t *options) +{ + /* XXXX This whole thing was stuck in the middle of what is now + * XXXX check_descriptor_callback. I'm not sure it's right. */ - if (!should_delay_dir_fetches(options, NULL) && - time_to.download_networkstatus < now) { - time_to.download_networkstatus = - now + networkstatus_dl_check_interval(options); - update_networkstatus_downloads(now); + static int dirport_reachability_count = 0; + /* also, check religiously for reachability, if it's within the first + * 20 minutes of our uptime. */ + if (server_mode(options) && + (have_completed_a_circuit() || !any_predicted_circuits(now)) && + !we_are_hibernating()) { + if (stats_n_seconds_working < TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { + consider_testing_reachability(1, dirport_reachability_count==0); + if (++dirport_reachability_count > 5) + dirport_reachability_count = 0; + return 1; + } else { + /* If we haven't checked for 12 hours and our bandwidth estimate is + * low, do another bandwidth test. This is especially important for + * bridges, since they might go long periods without much use. */ + const routerinfo_t *me = router_get_my_routerinfo(); + static int first_time = 1; + if (!first_time && me && + me->bandwidthcapacity < me->bandwidthrate && + me->bandwidthcapacity < 51200) { + reset_bandwidth_test(); + } + first_time = 0; +#define BANDWIDTH_RECHECK_INTERVAL (12*60*60) + return BANDWIDTH_RECHECK_INTERVAL; + } } + return CHECK_DESCRIPTOR_INTERVAL; +} - /* 2c. Let directory voting happen. */ - if (authdir_mode_v3(options)) - dirvote_act(options, now); +static int +fetch_networkstatus_callback(time_t now, const or_options_t *options) +{ + /* 2c. Every minute (or every second if TestingTorNetwork), check + * whether we want to download any networkstatus documents. */ - /* 3a. Every second, we examine pending circuits and prune the - * ones which have been pending for more than a few seconds. - * We do this before step 4, so it can try building more if - * it's not comfortable with the number of available circuits. - */ - /* (If our circuit build timeout can ever become lower than a second (which - * it can't, currently), we should do this more often.) */ - circuit_expire_building(); + /* How often do we check whether we should download network status + * documents? */ +#define networkstatus_dl_check_interval(o) ((o)->TestingTorNetwork ? 1 : 60) - /* 3b. Also look at pending streams and prune the ones that 'began' - * a long time ago but haven't gotten a 'connected' yet. - * Do this before step 4, so we can put them back into pending - * state to be picked up by the new circuit. - */ - connection_ap_expire_beginning(); + if (should_delay_dir_fetches(options, NULL)) + return PERIODIC_EVENT_NO_UPDATE; - /* 3c. And expire connections that we've held open for too long. - */ - connection_expire_held_open(); + update_networkstatus_downloads(now); + return networkstatus_dl_check_interval(options); +} +static int +retry_listeners_callback(time_t now, const or_options_t *options) +{ + (void)now; + (void)options; /* 3d. And every 60 seconds, we relaunch listeners if any died. */ - if (!net_is_disabled() && time_to.check_listeners < now) { + if (!net_is_disabled()) { retry_all_listeners(NULL, NULL, 0); - time_to.check_listeners = now+60; - } - - /* 4. Every second, we try a new circuit if there are no valid - * circuits. Every NewCircuitPeriod seconds, we expire circuits - * that became dirty more than MaxCircuitDirtiness seconds ago, - * and we make a new circ if there are no clean circuits. - */ - have_dir_info = router_have_minimum_dir_info(); - if (have_dir_info && !net_is_disabled()) { - circuit_build_needed_circs(now); - } else { - circuit_expire_old_circs_as_needed(now); + return 60; } + return PERIODIC_EVENT_NO_UPDATE; +} - /* every 10 seconds, but not at the same second as other such events */ - if (now % 10 == 5) - circuit_expire_old_circuits_serverside(now); - - /* 5. We do housekeeping for each connection... */ - connection_or_set_bad_connections(NULL, 0); - for (i=0;i<smartlist_len(connection_array);i++) { - run_connection_housekeeping(i, now); - } - - /* 6. And remove any marked circuits... */ - circuit_close_all_marked(); - - /* 7. And upload service descriptors if necessary. */ - if (have_completed_a_circuit() && !net_is_disabled()) { - rend_consider_services_upload(now); - rend_consider_descriptor_republication(); - } - - /* 8. and blow away any connections that need to die. have to do this now, - * because if we marked a conn for close and left its socket -1, then - * we'll pass it to poll/select and bad things will happen. - */ - close_closeable_connections(); - - /* 8b. And if anything in our state is ready to get flushed to disk, we - * flush it. */ - or_state_save(now); - - /* 8c. Do channel cleanup just like for connections */ - channel_run_cleanup(); - channel_listener_run_cleanup(); +static int +expire_old_ciruits_serverside_callback(time_t now, const or_options_t *options) +{ + (void)options; + /* every 11 seconds, so not usually the same second as other such events */ + circuit_expire_old_circuits_serverside(now); + return 11; +} +static int +check_dns_honesty_callback(time_t now, const or_options_t *options) +{ + (void)now; /* 9. and if we're an exit node, check whether our DNS is telling stories * to us. */ - if (!net_is_disabled() && - public_server_mode(options) && - time_to.check_for_correct_dns < now && - ! router_my_exit_policy_is_reject_star()) { - if (!time_to.check_for_correct_dns) { - time_to.check_for_correct_dns = - crypto_rand_time_range(now + 60, now + 180); - } else { - dns_launch_correctness_checks(); - time_to.check_for_correct_dns = now + 12*3600 + - crypto_rand_int(12*3600); - } + if (net_is_disabled() || + ! public_server_mode(options) || + router_my_exit_policy_is_reject_star()) + return PERIODIC_EVENT_NO_UPDATE; + + static int first_time = 1; + if (first_time) { + /* Don't launch right when we start */ + first_time = 0; + return crypto_rand_int_range(60, 180); } + dns_launch_correctness_checks(); + return 12*3600 + crypto_rand_int(12*3600); +} + +static int +write_bridge_ns_callback(time_t now, const or_options_t *options) +{ /* 10. write bridge networkstatus file to disk */ - if (options->BridgeAuthoritativeDir && - time_to.write_bridge_status_file < now) { + if (options->BridgeAuthoritativeDir) { networkstatus_dump_bridge_status_to_file(now); #define BRIDGE_STATUSFILE_INTERVAL (30*60) - time_to.write_bridge_status_file = now+BRIDGE_STATUSFILE_INTERVAL; + return BRIDGE_STATUSFILE_INTERVAL; } + return PERIODIC_EVENT_NO_UPDATE; +} +static int +check_fw_helper_app_callback(time_t now, const or_options_t *options) +{ + if (net_is_disabled() || + ! server_mode(options) || + ! options->PortForwarding) { + return PERIODIC_EVENT_NO_UPDATE; + } /* 11. check the port forwarding app */ - if (!net_is_disabled() && - time_to.check_port_forwarding < now && - options->PortForwarding && - is_server) { + #define PORT_FORWARDING_CHECK_INTERVAL 5 - smartlist_t *ports_to_forward = get_list_of_ports_to_forward(); - if (ports_to_forward) { - tor_check_port_forwarding(options->PortForwardingHelper, - ports_to_forward, - now); - - SMARTLIST_FOREACH(ports_to_forward, char *, cp, tor_free(cp)); - smartlist_free(ports_to_forward); - } - time_to.check_port_forwarding = now+PORT_FORWARDING_CHECK_INTERVAL; - } + smartlist_t *ports_to_forward = get_list_of_ports_to_forward(); + if (ports_to_forward) { + tor_check_port_forwarding(options->PortForwardingHelper, + ports_to_forward, + now); - /* 11b. check pending unconfigured managed proxies */ - if (!net_is_disabled() && pt_proxies_configuration_pending()) - pt_configure_remaining_proxies(); + SMARTLIST_FOREACH(ports_to_forward, char *, cp, tor_free(cp)); + smartlist_free(ports_to_forward); + } + return PORT_FORWARDING_CHECK_INTERVAL; +} +static int +heartbeat_callback(time_t now, const or_options_t *options) +{ + static int first = 1; /* 12. write the heartbeat message */ - if (options->HeartbeatPeriod && - time_to.next_heartbeat <= now) { - if (time_to.next_heartbeat) /* don't log the first heartbeat */ - log_heartbeat(now); - time_to.next_heartbeat = now+options->HeartbeatPeriod; + if (first) { + first = 0; /* Skip the first one. */ + } else { + log_heartbeat(now); } + /* XXXX This isn't such a good way to handle possible changes in the + * callback event */ + return options->HeartbeatPeriod; } /** Timer: used to invoke second_elapsed_callback() once per second. */ @@ -1947,7 +2203,10 @@ dns_servers_relaunch_checks(void) { if (server_mode(get_options())) { dns_reset_correctness_checks(); - time_to.check_for_correct_dns = 0; + if (periodic_events_initialized) { + tor_assert(check_dns_honesty_event); + periodic_event_reschedule(check_dns_honesty_event); + } } } @@ -2041,6 +2300,13 @@ do_main_loop(void) { time_t now; + /* initialize the periodic events first, so that code that depends on the + * events being present does not assert. + */ + if (! periodic_events_initialized) { + initialize_periodic_events(); + } + /* initialize dns resolve map, spawn workers if needed */ if (dns_init() < 0) { if (get_options()->ServerDNSAllowBrokenConfig) @@ -2251,6 +2517,11 @@ run_main_loop_once(void) } } + /* This will be pretty fast if nothing new is pending. Note that this gets + * called once per libevent loop, which will make it happen once per group + * of events that fire, or once per second. */ + connection_ap_attach_pending(0); + return 1; } @@ -2825,6 +3096,7 @@ tor_free_all(int postfork) channel_tls_free_all(); channel_free_all(); connection_free_all(); + connection_edge_free_all(); scheduler_free_all(); memarea_clear_freelist(); nodelist_free_all(); @@ -2851,6 +3123,7 @@ tor_free_all(int postfork) smartlist_free(closeable_connection_lst); smartlist_free(active_linked_connection_lst); periodic_timer_free(second_timer); + teardown_periodic_events(); #ifndef USE_BUFFEREVENTS periodic_timer_free(refill_timer); #endif diff --git a/src/or/main.h b/src/or/main.h index 447d3f4eca..37e93d79d3 100644 --- a/src/or/main.h +++ b/src/or/main.h @@ -78,6 +78,8 @@ int tor_init(int argc, char **argv); #ifdef MAIN_PRIVATE STATIC void init_connection_lists(void); STATIC void close_closeable_connections(void); +STATIC void initialize_periodic_events(void); +STATIC void teardown_periodic_events(void); #endif #endif diff --git a/src/or/microdesc.c b/src/or/microdesc.c index a9bab3ddc6..dc23bcb632 100644 --- a/src/or/microdesc.c +++ b/src/or/microdesc.c @@ -47,14 +47,14 @@ struct microdesc_cache_t { static microdesc_cache_t *get_microdesc_cache_noload(void); /** Helper: computes a hash of <b>md</b> to place it in a hash table. */ -static INLINE unsigned int +static inline unsigned int microdesc_hash_(microdesc_t *md) { return (unsigned) siphash24g(md->digest, sizeof(md->digest)); } /** Helper: compares <b>a</b> and </b> for equality for hash-table purposes. */ -static INLINE int +static inline int microdesc_eq_(microdesc_t *a, microdesc_t *b) { return tor_memeq(a->digest, b->digest, DIGEST256_LEN); diff --git a/src/or/nodelist.c b/src/or/nodelist.c index 2f272a1d56..fc27207851 100644 --- a/src/or/nodelist.c +++ b/src/or/nodelist.c @@ -57,13 +57,13 @@ typedef struct nodelist_t { } nodelist_t; -static INLINE unsigned int +static inline unsigned int node_id_hash(const node_t *node) { return (unsigned) siphash24g(node->identity, DIGEST_LEN); } -static INLINE unsigned int +static inline unsigned int node_id_eq(const node_t *node1, const node_t *node2) { return tor_memeq(node1->identity, node2->identity, DIGEST_LEN); @@ -291,7 +291,7 @@ nodelist_set_consensus(networkstatus_t *ns) } /** Helper: return true iff a node has a usable amount of information*/ -static INLINE int +static inline int node_is_usable(const node_t *node) { return (node->rs) || (node->ri); @@ -1021,7 +1021,7 @@ nodelist_refresh_countries(void) /** Return true iff router1 and router2 have similar enough network addresses * that we should treat them as being in the same family */ -static INLINE int +static inline int addrs_in_same_network_family(const tor_addr_t *a1, const tor_addr_t *a2) { @@ -1045,7 +1045,7 @@ node_nickname_matches(const node_t *node, const char *nickname) } /** Return true iff <b>node</b> is named by some nickname in <b>lst</b>. */ -static INLINE int +static inline int node_in_nickname_smartlist(const smartlist_t *lst, const node_t *node) { if (!lst) return 0; diff --git a/src/or/onion_fast.c b/src/or/onion_fast.c index 7584112570..22bef4eee0 100644 --- a/src/or/onion_fast.c +++ b/src/or/onion_fast.c @@ -30,10 +30,7 @@ fast_onionskin_create(fast_handshake_state_t **handshake_state_out, { fast_handshake_state_t *s; *handshake_state_out = s = tor_malloc(sizeof(fast_handshake_state_t)); - if (crypto_rand((char*)s->state, sizeof(s->state)) < 0) { - tor_free(s); - return -1; - } + crypto_rand((char*)s->state, sizeof(s->state)); memcpy(handshake_out, s->state, DIGEST_LEN); return 0; } @@ -56,8 +53,7 @@ fast_server_handshake(const uint8_t *key_in, /* DIGEST_LEN bytes */ size_t out_len; int r = -1; - if (crypto_rand((char*)handshake_reply_out, DIGEST_LEN)<0) - return -1; + crypto_rand((char*)handshake_reply_out, DIGEST_LEN); memcpy(tmp, key_in, DIGEST_LEN); memcpy(tmp+DIGEST_LEN, handshake_reply_out, DIGEST_LEN); diff --git a/src/or/or.h b/src/or/or.h index 4496cbcec3..218bea4828 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -915,18 +915,18 @@ typedef enum { #define VAR_CELL_MAX_HEADER_SIZE 7 static int get_cell_network_size(int wide_circ_ids); -static INLINE int get_cell_network_size(int wide_circ_ids) +static inline int get_cell_network_size(int wide_circ_ids) { return wide_circ_ids ? CELL_MAX_NETWORK_SIZE : CELL_MAX_NETWORK_SIZE - 2; } static int get_var_cell_header_size(int wide_circ_ids); -static INLINE int get_var_cell_header_size(int wide_circ_ids) +static inline int get_var_cell_header_size(int wide_circ_ids) { return wide_circ_ids ? VAR_CELL_MAX_HEADER_SIZE : VAR_CELL_MAX_HEADER_SIZE - 2; } static int get_circ_id_size(int wide_circ_ids); -static INLINE int get_circ_id_size(int wide_circ_ids) +static inline int get_circ_id_size(int wide_circ_ids) { return wide_circ_ids ? 4 : 2; } @@ -1799,38 +1799,38 @@ static control_connection_t *TO_CONTROL_CONN(connection_t *); * invalid. */ static listener_connection_t *TO_LISTENER_CONN(connection_t *); -static INLINE or_connection_t *TO_OR_CONN(connection_t *c) +static inline or_connection_t *TO_OR_CONN(connection_t *c) { tor_assert(c->magic == OR_CONNECTION_MAGIC); return DOWNCAST(or_connection_t, c); } -static INLINE dir_connection_t *TO_DIR_CONN(connection_t *c) +static inline dir_connection_t *TO_DIR_CONN(connection_t *c) { tor_assert(c->magic == DIR_CONNECTION_MAGIC); return DOWNCAST(dir_connection_t, c); } -static INLINE edge_connection_t *TO_EDGE_CONN(connection_t *c) +static inline edge_connection_t *TO_EDGE_CONN(connection_t *c) { tor_assert(c->magic == EDGE_CONNECTION_MAGIC || c->magic == ENTRY_CONNECTION_MAGIC); return DOWNCAST(edge_connection_t, c); } -static INLINE entry_connection_t *TO_ENTRY_CONN(connection_t *c) +static inline entry_connection_t *TO_ENTRY_CONN(connection_t *c) { tor_assert(c->magic == ENTRY_CONNECTION_MAGIC); return (entry_connection_t*) SUBTYPE_P(c, entry_connection_t, edge_.base_); } -static INLINE entry_connection_t *EDGE_TO_ENTRY_CONN(edge_connection_t *c) +static inline entry_connection_t *EDGE_TO_ENTRY_CONN(edge_connection_t *c) { tor_assert(c->base_.magic == ENTRY_CONNECTION_MAGIC); return (entry_connection_t*) SUBTYPE_P(c, entry_connection_t, edge_); } -static INLINE control_connection_t *TO_CONTROL_CONN(connection_t *c) +static inline control_connection_t *TO_CONTROL_CONN(connection_t *c) { tor_assert(c->magic == CONTROL_CONNECTION_MAGIC); return DOWNCAST(control_connection_t, c); } -static INLINE listener_connection_t *TO_LISTENER_CONN(connection_t *c) +static inline listener_connection_t *TO_LISTENER_CONN(connection_t *c) { tor_assert(c->magic == LISTENER_CONNECTION_MAGIC); return DOWNCAST(listener_connection_t, c); @@ -2892,6 +2892,14 @@ typedef struct circuit_t { * where this circuit was marked.) */ const char *marked_for_close_file; /**< For debugging: in which file was this * circuit marked for close? */ + /** For what reason (See END_CIRC_REASON...) is this circuit being closed? + * This field is set in circuit_mark_for_close and used later in + * circuit_about_to_free. */ + uint16_t marked_for_close_reason; + /** As marked_for_close_reason, but reflects the underlying reason for + * closing this circuit. + */ + uint16_t marked_for_close_orig_reason; /** Unique ID for measuring tunneled network status requests. */ uint64_t dirreq_id; @@ -3281,27 +3289,27 @@ static const origin_circuit_t *CONST_TO_ORIGIN_CIRCUIT(const circuit_t *); /** Return 1 iff <b>node</b> has Exit flag and no BadExit flag. * Otherwise, return 0. */ -static INLINE int node_is_good_exit(const node_t *node) +static inline int node_is_good_exit(const node_t *node) { return node->is_exit && ! node->is_bad_exit; } -static INLINE or_circuit_t *TO_OR_CIRCUIT(circuit_t *x) +static inline or_circuit_t *TO_OR_CIRCUIT(circuit_t *x) { tor_assert(x->magic == OR_CIRCUIT_MAGIC); return DOWNCAST(or_circuit_t, x); } -static INLINE const or_circuit_t *CONST_TO_OR_CIRCUIT(const circuit_t *x) +static inline const or_circuit_t *CONST_TO_OR_CIRCUIT(const circuit_t *x) { tor_assert(x->magic == OR_CIRCUIT_MAGIC); return DOWNCAST(or_circuit_t, x); } -static INLINE origin_circuit_t *TO_ORIGIN_CIRCUIT(circuit_t *x) +static inline origin_circuit_t *TO_ORIGIN_CIRCUIT(circuit_t *x) { tor_assert(x->magic == ORIGIN_CIRCUIT_MAGIC); return DOWNCAST(origin_circuit_t, x); } -static INLINE const origin_circuit_t *CONST_TO_ORIGIN_CIRCUIT( +static inline const origin_circuit_t *CONST_TO_ORIGIN_CIRCUIT( const circuit_t *x) { tor_assert(x->magic == ORIGIN_CIRCUIT_MAGIC); @@ -3424,9 +3432,11 @@ typedef struct { * each log message occurs? */ int TruncateLogFile; /**< Boolean: Should we truncate the log file before we start writing? */ + char *SyslogIdentityTag; /**< Identity tag to add for syslog logging. */ char *DebugLogFile; /**< Where to send verbose log messages. */ char *DataDirectory; /**< OR only: where to store long-term data. */ + int DataDirectoryGroupReadable; /**< Boolean: Is the DataDirectory g+r? */ char *Nickname; /**< OR only: nickname of this onion router. */ char *Address; /**< OR only: configured address for this onion router. */ char *PidFile; /**< Where to store PID of Tor process. */ @@ -3807,7 +3817,7 @@ typedef struct { * hibernate." */ /** How do we determine when our AccountingMax has been reached? * "max" for when in or out reaches AccountingMax - * "sum for when in plus out reaches AccountingMax */ + * "sum" for when in plus out reaches AccountingMax */ char *AccountingRule_option; enum { ACCT_MAX, ACCT_SUM } AccountingRule; @@ -4014,7 +4024,7 @@ typedef struct { char *ConsensusParams; /** Authority only: minimum number of measured bandwidths we must see - * before we only beliee measured bandwidths to assign flags. */ + * before we only believe measured bandwidths to assign flags. */ int MinMeasuredBWsForAuthToIgnoreAdvertised; /** The length of time that we think an initial consensus should be fresh. @@ -4388,7 +4398,7 @@ typedef struct { /** Change the next_write time of <b>state</b> to <b>when</b>, unless the * state is already scheduled to be written to disk earlier than <b>when</b>. */ -static INLINE void or_state_mark_dirty(or_state_t *state, time_t when) +static inline void or_state_mark_dirty(or_state_t *state, time_t when) { if (state->next_write > when) state->next_write = when; diff --git a/src/or/periodic.c b/src/or/periodic.c new file mode 100644 index 0000000000..109717f738 --- /dev/null +++ b/src/or/periodic.c @@ -0,0 +1,120 @@ +/* Copyright (c) 2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "compat_libevent.h" +#include "config.h" +#include "periodic.h" + +#ifdef HAVE_EVENT2_EVENT_H +#include <event2/event.h> +#else +#include <event.h> +#endif + +/** We disable any interval greater than this number of seconds, on the + * grounds that it is probably an absolute time mistakenly passed in as a + * relative time. + */ +static const int MAX_INTERVAL = 10 * 365 * 86400; + +/** Set the event <b>event</b> to run in <b>next_interval</b> seconds from + * now. */ +static void +periodic_event_set_interval(periodic_event_item_t *event, + time_t next_interval) +{ + tor_assert(next_interval < MAX_INTERVAL); + struct timeval tv; + tv.tv_sec = next_interval; + tv.tv_usec = 0; + event_add(event->ev, &tv); +} + +/** Wraps dispatches for periodic events, <b>data</b> will be a pointer to the + * event that needs to be called */ +static void +periodic_event_dispatch(evutil_socket_t fd, short what, void *data) +{ + (void)fd; + (void)what; + periodic_event_item_t *event = data; + + time_t now = time(NULL); + const or_options_t *options = get_options(); + log_debug(LD_GENERAL, "Dispatching %s", event->name); + int r = event->fn(now, options); + int next_interval = 0; + + /* update the last run time if action was taken */ + if (r==0) { + log_err(LD_BUG, "Invalid return value for periodic event from %s.", + event->name); + tor_assert(r != 0); + } else if (r > 0) { + event->last_action_time = now; + /* If the event is meant to happen after ten years, that's likely + * a bug, and somebody gave an absolute time rather than an interval. + */ + tor_assert(r < MAX_INTERVAL); + next_interval = r; + } else { + /* no action was taken, it is likely a precondition failed, + * we should reschedule for next second incase the precondition + * passes then */ + next_interval = 1; + } + + log_debug(LD_GENERAL, "Scheduling %s for %d seconds", event->name, + next_interval); + struct timeval tv = { next_interval , 0 }; + event_add(event->ev, &tv); +} + +/** Schedules <b>event</b> to run as soon as possible from now. */ +void +periodic_event_reschedule(periodic_event_item_t *event) +{ + periodic_event_set_interval(event, 1); +} + +/** Initializes the libevent backend for a periodic event. */ +void +periodic_event_setup(periodic_event_item_t *event) +{ + if (event->ev) { /* Already setup? This is a bug */ + log_err(LD_BUG, "Initial dispatch should only be done once."); + tor_assert(0); + } + + event->ev = tor_event_new(tor_libevent_get_base(), + -1, 0, + periodic_event_dispatch, + event); + tor_assert(event->ev); +} + +/** Handles initial dispatch for periodic events. It should happen 1 second + * after the events are created to mimic behaviour before #3199's refactor */ +void +periodic_event_launch(periodic_event_item_t *event) +{ + if (! event->ev) { /* Not setup? This is a bug */ + log_err(LD_BUG, "periodic_event_launch without periodic_event_setup"); + tor_assert(0); + } + + // Initial dispatch + periodic_event_dispatch(-1, EV_TIMEOUT, event); +} + +/** Release all storage associated with <b>event</b> */ +void +periodic_event_destroy(periodic_event_item_t *event) +{ + if (!event) + return; + tor_event_free(event->ev); + event->last_action_time = 0; +} + diff --git a/src/or/periodic.h b/src/or/periodic.h new file mode 100644 index 0000000000..bab0c91916 --- /dev/null +++ b/src/or/periodic.h @@ -0,0 +1,37 @@ +/* Copyright (c) 2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#ifndef TOR_PERIODIC_H +#define TOR_PERIODIC_H + +#define PERIODIC_EVENT_NO_UPDATE (-1) + +/** Callback function for a periodic event to take action. The return value +* influences the next time the function will get called. Return +* PERIODIC_EVENT_NO_UPDATE to not update <b>last_action_time</b> and be polled +* again in the next second. If a positive value is returned it will update the +* interval time. */ +typedef int (*periodic_event_helper_t)(time_t now, + const or_options_t *options); + +struct event; + +/** A single item for the periodic-events-function table. */ +typedef struct periodic_event_item_t { + periodic_event_helper_t fn; /**< The function to run the event */ + time_t last_action_time; /**< The last time the function did something */ + struct event *ev; /**< Libevent callback we're using to implement this */ + const char *name; /**< Name of the function -- for debug */ +} periodic_event_item_t; + +/** events will get their interval from first execution */ +#define PERIODIC_EVENT(fn) { fn##_callback, 0, NULL, #fn } +#define END_OF_PERIODIC_EVENTS { NULL, 0, NULL, NULL } + +void periodic_event_launch(periodic_event_item_t *event); +void periodic_event_setup(periodic_event_item_t *event); +void periodic_event_destroy(periodic_event_item_t *event); +void periodic_event_reschedule(periodic_event_item_t *event); + +#endif + diff --git a/src/or/policies.c b/src/or/policies.c index b247e6a64d..32a7ec2da4 100644 --- a/src/or/policies.c +++ b/src/or/policies.c @@ -8,6 +8,8 @@ * \brief Code to parse and use address policies and exit policies. **/ +#define POLICIES_PRIVATE + #include "or.h" #include "config.h" #include "dirserv.h" @@ -62,14 +64,15 @@ static const char *private_nets[] = { NULL }; -static int policies_parse_exit_policy_internal(config_line_t *cfg, - smartlist_t **dest, - int ipv6_exit, - int rejectprivate, - uint32_t local_address, - tor_addr_t *ipv6_local_address, - int reject_interface_addresses, - int add_default_policy); +static int policies_parse_exit_policy_internal( + config_line_t *cfg, + smartlist_t **dest, + int ipv6_exit, + int rejectprivate, + const smartlist_t *configured_addresses, + int reject_interface_addresses, + int reject_configured_port_addresses, + int add_default_policy); /** Replace all "private" entries in *<b>policy</b> with their expanded * equivalents. */ @@ -443,7 +446,7 @@ validate_addr_policies(const or_options_t *options, char **msg) smartlist_t *addr_policy=NULL; *msg = NULL; - if (policies_parse_exit_policy_from_options(options,0,NULL,0,&addr_policy)) { + if (policies_parse_exit_policy_from_options(options,0,NULL,&addr_policy)) { REJECT("Error in ExitPolicy entry."); } @@ -625,7 +628,7 @@ typedef struct policy_map_ent_t { static HT_HEAD(policy_map, policy_map_ent_t) policy_root = HT_INITIALIZER(); /** Return true iff a and b are equal. */ -static INLINE int +static inline int policy_eq(policy_map_ent_t *a, policy_map_ent_t *b) { return cmp_single_addr_policy(a->policy, b->policy) == 0; @@ -864,7 +867,7 @@ addr_policy_intersects(addr_policy_t *a, addr_policy_t *b) /** Add the exit policy described by <b>more</b> to <b>policy</b>. */ -static void +STATIC void append_exit_policy_string(smartlist_t **policy, const char *more) { config_line_t tmp; @@ -881,6 +884,9 @@ append_exit_policy_string(smartlist_t **policy, const char *more) void addr_policy_append_reject_addr(smartlist_t **dest, const tor_addr_t *addr) { + tor_assert(dest); + tor_assert(addr); + addr_policy_t p, *add; memset(&p, 0, sizeof(p)); p.policy_type = ADDR_POLICY_REJECT; @@ -893,6 +899,71 @@ addr_policy_append_reject_addr(smartlist_t **dest, const tor_addr_t *addr) if (!*dest) *dest = smartlist_new(); smartlist_add(*dest, add); + log_debug(LD_CONFIG, "Adding a reject ExitPolicy 'reject %s:*'", + fmt_addr(addr)); +} + +/* Is addr public for the purposes of rejection? */ +static int +tor_addr_is_public_for_reject(const tor_addr_t *addr) +{ + return (!tor_addr_is_null(addr) && !tor_addr_is_internal(addr, 0) + && !tor_addr_is_multicast(addr)); +} + +/* Add "reject <b>addr</b>:*" to <b>dest</b>, creating the list as needed. + * Filter the address, only adding an IPv4 reject rule if ipv4_rules + * is true, and similarly for ipv6_rules. Check each address returns true for + * tor_addr_is_public_for_reject before adding it. + */ +static void +addr_policy_append_reject_addr_filter(smartlist_t **dest, + const tor_addr_t *addr, + int ipv4_rules, + int ipv6_rules) +{ + tor_assert(dest); + tor_assert(addr); + + /* Only reject IP addresses which are public */ + if (tor_addr_is_public_for_reject(addr)) { + + /* Reject IPv4 addresses and IPv6 addresses based on the filters */ + int is_ipv4 = tor_addr_is_v4(addr); + if ((is_ipv4 && ipv4_rules) || (!is_ipv4 && ipv6_rules)) { + addr_policy_append_reject_addr(dest, addr); + } + } +} + +/** Add "reject addr:*" to <b>dest</b>, for each addr in addrs, creating the + * list as needed. */ +void +addr_policy_append_reject_addr_list(smartlist_t **dest, + const smartlist_t *addrs) +{ + tor_assert(dest); + tor_assert(addrs); + + SMARTLIST_FOREACH_BEGIN(addrs, tor_addr_t *, addr) { + addr_policy_append_reject_addr(dest, addr); + } SMARTLIST_FOREACH_END(addr); +} + +/** Add "reject addr:*" to <b>dest</b>, for each addr in addrs, creating the + * list as needed. Filter using */ +static void +addr_policy_append_reject_addr_list_filter(smartlist_t **dest, + const smartlist_t *addrs, + int ipv4_rules, + int ipv6_rules) +{ + tor_assert(dest); + tor_assert(addrs); + + SMARTLIST_FOREACH_BEGIN(addrs, tor_addr_t *, addr) { + addr_policy_append_reject_addr_filter(dest, addr, ipv4_rules, ipv6_rules); + } SMARTLIST_FOREACH_END(addr); } /** Detect and excise "dead code" from the policy *<b>dest</b>. */ @@ -979,127 +1050,90 @@ exit_policy_remove_redundancies(smartlist_t *dest) } } -#define DEFAULT_EXIT_POLICY \ - "reject *:25,reject *:119,reject *:135-139,reject *:445," \ - "reject *:563,reject *:1214,reject *:4661-4666," \ - "reject *:6346-6429,reject *:6699,reject *:6881-6999,accept *:*" - -/** Parse the exit policy <b>cfg</b> into the linked list *<b>dest</b>. - * - * If <b>ipv6_exit</b> is true, prepend "reject *6:*" to the policy. +/** Reject private helper for policies_parse_exit_policy_internal: rejects + * publicly routable addresses on this exit relay. * - * If <b>rejectprivate</b> is true: - * - prepend "reject private:*" to the policy. - * - if local_address is non-zero, treat it as a host-order IPv4 address, - * and prepend an entry that rejects it as a destination. - * - if ipv6_local_address is non-NULL, prepend an entry that rejects it as - * a destination. - * - if reject_interface_addresses is true, prepend entries that reject each + * Add reject entries to the linked list *dest: + * - if configured_addresses is non-NULL, add entries that reject each + * tor_addr_t* in the list as a destination. + * - if reject_interface_addresses is true, add entries that reject each * public IPv4 and IPv6 address of each interface on this machine. + * - if reject_configured_port_addresses is true, add entries that reject + * each IPv4 and IPv6 address configured for a port. * - * If cfg doesn't end in an absolute accept or reject and if - * <b>add_default_policy</b> is true, add the default exit - * policy afterwards. - * - * Return -1 if we can't parse cfg, else return 0. + * IPv6 entries are only added if ipv6_exit is true. (All IPv6 addresses are + * already blocked by policies_parse_exit_policy_internal if ipv6_exit is + * false.) * - * This function is used to parse the exit policy from our torrc. For - * the functions used to parse the exit policy from a router descriptor, - * see router_add_exit_policy. + * The list *dest is created as needed. */ -static int -policies_parse_exit_policy_internal(config_line_t *cfg, smartlist_t **dest, - int ipv6_exit, - int rejectprivate, - uint32_t local_address, - tor_addr_t *ipv6_local_address, - int reject_interface_addresses, - int add_default_policy) +void +policies_parse_exit_policy_reject_private( + smartlist_t **dest, + int ipv6_exit, + const smartlist_t *configured_addresses, + int reject_interface_addresses, + int reject_configured_port_addresses) { - if (!ipv6_exit) { - append_exit_policy_string(dest, "reject *6:*"); + tor_assert(dest); + + /* Reject configured addresses, if they are from public netblocks. */ + if (configured_addresses) { + addr_policy_append_reject_addr_list_filter(dest, configured_addresses, + 1, ipv6_exit); } - if (rejectprivate) { - /* Reject IPv4 and IPv6 reserved private netblocks */ - append_exit_policy_string(dest, "reject private:*"); - /* Reject our local IPv4 address */ - if (local_address) { - char buf[POLICY_BUF_LEN]; - tor_snprintf(buf, sizeof(buf), "reject %s:*", fmt_addr32(local_address)); - append_exit_policy_string(dest, buf); - log_info(LD_CONFIG, "Adding a reject ExitPolicy '%s' for our published " - "IPv4 address", buf); - } - /* Reject our local IPv6 address */ - if (ipv6_exit && ipv6_local_address != NULL) { - if (tor_addr_is_v4(ipv6_local_address)) { - log_warn(LD_CONFIG, "IPv4 address '%s' provided as our IPv6 local " - "address", fmt_addr(ipv6_local_address)); - } else { - char buf6[POLICY_BUF_LEN]; - tor_snprintf(buf6, sizeof(buf6), "reject [%s]:*", - fmt_addr(ipv6_local_address)); - append_exit_policy_string(dest, buf6); - log_info(LD_CONFIG, "Adding a reject ExitPolicy '%s' for our " - "published IPv6 address", buf6); - } - } - /* Reject local addresses from public netblocks on any interface, - * but don't reject our published addresses twice */ - if (reject_interface_addresses) { - smartlist_t *public_addresses = NULL; - char bufif[POLICY_BUF_LEN]; - - /* Reject public IPv4 addresses on any interface, - * but don't reject our published IPv4 address twice */ - public_addresses = get_interface_address6_list(LOG_INFO, AF_INET, 0); - SMARTLIST_FOREACH_BEGIN(public_addresses, tor_addr_t *, a) { - if (!tor_addr_eq_ipv4h(a, local_address)) { - tor_snprintf(bufif, sizeof(bufif), "reject %s:*", - fmt_addr(a)); - append_exit_policy_string(dest, bufif); - log_info(LD_CONFIG, "Adding a reject ExitPolicy '%s' for a local " - "interface's public IPv4 address", bufif); - } - } SMARTLIST_FOREACH_END(a); - free_interface_address6_list(public_addresses); - if (ipv6_exit) { - /* Reject public IPv6 addresses on any interface, - * but don't reject our published IPv6 address (if any) twice */ - public_addresses = get_interface_address6_list(LOG_INFO, AF_INET6, 0); - SMARTLIST_FOREACH_BEGIN(public_addresses, tor_addr_t *, a) { - /* if we don't have an IPv6 local address, we won't have rejected - * it above. This could happen if a future release does IPv6 - * autodiscovery, and we are waiting to discover our external IPv6 - * address */ - if (ipv6_local_address == NULL - || !tor_addr_eq(ipv6_local_address, a)) { - tor_snprintf(bufif, sizeof(bufif), "reject6 [%s]:*", - fmt_addr(a)); - append_exit_policy_string(dest, bufif); - log_info(LD_CONFIG, "Adding a reject ExitPolicy '%s' for a local " - "interface's public IPv6 address", bufif); - } - } SMARTLIST_FOREACH_END(a); - free_interface_address6_list(public_addresses); + /* Reject configured port addresses, if they are from public netblocks. */ + if (reject_configured_port_addresses) { + const smartlist_t *port_addrs = get_configured_ports(); + + SMARTLIST_FOREACH_BEGIN(port_addrs, port_cfg_t *, port) { + + /* Only reject port IP addresses, not port unix sockets */ + if (!port->is_unix_addr) { + addr_policy_append_reject_addr_filter(dest, &port->addr, 1, ipv6_exit); } + } SMARTLIST_FOREACH_END(port); + } + + /* Reject local addresses from public netblocks on any interface. */ + if (reject_interface_addresses) { + smartlist_t *public_addresses = NULL; + + /* Reject public IPv4 addresses on any interface */ + public_addresses = get_interface_address6_list(LOG_INFO, AF_INET, 0); + addr_policy_append_reject_addr_list_filter(dest, public_addresses, 1, 0); + free_interface_address6_list(public_addresses); + + /* Don't look for IPv6 addresses if we're configured as IPv4-only */ + if (ipv6_exit) { + /* Reject public IPv6 addresses on any interface */ + public_addresses = get_interface_address6_list(LOG_INFO, AF_INET6, 0); + addr_policy_append_reject_addr_list_filter(dest, public_addresses, 0, 1); + free_interface_address6_list(public_addresses); } } - if (parse_addr_policy(cfg, dest, -1)) - return -1; - /* Before we add the default policy and final rejects, check to see if - * there are any lines after accept *:* or reject *:*. These lines have no - * effect, and are most likely an error. */ + /* If addresses were added multiple times, remove all but one of them. */ + if (*dest) { + exit_policy_remove_redundancies(*dest); + } +} + +/** + * Iterate through <b>policy</b> looking for redundant entries. Log a + * warning message with the first redundant entry, if any is found. + */ +static void +policies_log_first_redundant_entry(const smartlist_t *policy) +{ int found_final_effective_entry = 0; int first_redundant_entry = 0; - for (int i = 0; i < smartlist_len(*dest); ++i) { + tor_assert(policy); + SMARTLIST_FOREACH_BEGIN(policy, const addr_policy_t *, p) { sa_family_t family; - addr_policy_t *p; int found_ipv4_wildcard = 0, found_ipv6_wildcard = 0; - - p = smartlist_get(*dest, i); + const int i = p_sl_idx; /* Look for accept/reject *[4|6|]:* entires */ if (p->prt_min <= 1 && p->prt_max == 65535 && p->maskbits == 0) { @@ -1122,22 +1156,23 @@ policies_parse_exit_policy_internal(config_line_t *cfg, smartlist_t **dest, if (found_ipv4_wildcard && found_ipv6_wildcard) { found_final_effective_entry = 1; /* if we're not on the final entry in the list */ - if (i < smartlist_len(*dest) - 1) { + if (i < smartlist_len(policy) - 1) { first_redundant_entry = i + 1; } break; } - } + } SMARTLIST_FOREACH_END(p); + /* Work out if there are redundant trailing entries in the policy list */ if (found_final_effective_entry && first_redundant_entry > 0) { - addr_policy_t *p; + const addr_policy_t *p; /* Longest possible policy is * "accept6 ffff:ffff:..255/128:10000-65535", * which contains a max-length IPv6 address, plus 24 characters. */ char line[TOR_ADDR_BUF_LEN + 32]; - tor_assert(first_redundant_entry < smartlist_len(*dest)); - p = smartlist_get(*dest, first_redundant_entry); + tor_assert(first_redundant_entry < smartlist_len(policy)); + p = smartlist_get(policy, first_redundant_entry); /* since we've already parsed the policy into an addr_policy_t struct, * we might not log exactly what the user typed in */ policy_write_item(line, TOR_ADDR_BUF_LEN + 32, p, 0); @@ -1147,6 +1182,62 @@ policies_parse_exit_policy_internal(config_line_t *cfg, smartlist_t **dest, "accept/reject *:* as the last entry in any exit policy.)", line); } +} + +#define DEFAULT_EXIT_POLICY \ + "reject *:25,reject *:119,reject *:135-139,reject *:445," \ + "reject *:563,reject *:1214,reject *:4661-4666," \ + "reject *:6346-6429,reject *:6699,reject *:6881-6999,accept *:*" + +/** Parse the exit policy <b>cfg</b> into the linked list *<b>dest</b>. + * + * If <b>ipv6_exit</b> is false, prepend "reject *6:*" to the policy. + * + * If <b>rejectprivate</b> is true: + * - prepend "reject private:*" to the policy. + * - prepend entries that reject publicly routable addresses on this exit + * relay by calling policies_parse_exit_policy_reject_private + * + * If cfg doesn't end in an absolute accept or reject and if + * <b>add_default_policy</b> is true, add the default exit + * policy afterwards. + * + * Return -1 if we can't parse cfg, else return 0. + * + * This function is used to parse the exit policy from our torrc. For + * the functions used to parse the exit policy from a router descriptor, + * see router_add_exit_policy. + */ +static int +policies_parse_exit_policy_internal(config_line_t *cfg, + smartlist_t **dest, + int ipv6_exit, + int rejectprivate, + const smartlist_t *configured_addresses, + int reject_interface_addresses, + int reject_configured_port_addresses, + int add_default_policy) +{ + if (!ipv6_exit) { + append_exit_policy_string(dest, "reject *6:*"); + } + if (rejectprivate) { + /* Reject IPv4 and IPv6 reserved private netblocks */ + append_exit_policy_string(dest, "reject private:*"); + /* Reject IPv4 and IPv6 publicly routable addresses on this exit relay */ + policies_parse_exit_policy_reject_private( + dest, ipv6_exit, + configured_addresses, + reject_interface_addresses, + reject_configured_port_addresses); + } + if (parse_addr_policy(cfg, dest, -1)) + return -1; + + /* Before we add the default policy and final rejects, check to see if + * there are any lines after accept *:* or reject *:*. These lines have no + * effect, and are most likely an error. */ + policies_log_first_redundant_entry(*dest); if (add_default_policy) { append_exit_policy_string(dest, DEFAULT_EXIT_POLICY); @@ -1167,12 +1258,8 @@ policies_parse_exit_policy_internal(config_line_t *cfg, smartlist_t **dest, * If <b>EXIT_POLICY_REJECT_PRIVATE</b> bit is set in <b>options</b>: * - prepend an entry that rejects all destinations in all netblocks * reserved for private use. - * - if local_address is non-zero, treat it as a host-order IPv4 address, - * and prepend an entry that rejects it as a destination. - * - if ipv6_local_address is non-NULL, prepend an entry that rejects it as - * a destination. - * - if reject_interface_addresses is true, prepend entries that reject each - * public IPv4 and IPv6 address of each interface on this machine. + * - prepend entries that reject publicly routable addresses on this exit + * relay by calling policies_parse_exit_policy_internal * * If <b>EXIT_POLICY_ADD_DEFAULT</b> bit is set in <b>options</b>, append * default exit policy entries to <b>result</b> smartlist. @@ -1180,9 +1267,7 @@ policies_parse_exit_policy_internal(config_line_t *cfg, smartlist_t **dest, int policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, exit_policy_parser_cfg_t options, - uint32_t local_address, - tor_addr_t *ipv6_local_address, - int reject_interface_addresses) + const smartlist_t *configured_addresses) { int ipv6_enabled = (options & EXIT_POLICY_IPV6_ENABLED) ? 1 : 0; int reject_private = (options & EXIT_POLICY_REJECT_PRIVATE) ? 1 : 0; @@ -1190,12 +1275,62 @@ policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, return policies_parse_exit_policy_internal(cfg,dest,ipv6_enabled, reject_private, - local_address, - ipv6_local_address, - reject_interface_addresses, + configured_addresses, + reject_private, + reject_private, add_default); } +/** Helper function that adds a copy of addr to a smartlist as long as it is + * non-NULL and not tor_addr_is_null(). + * + * The caller is responsible for freeing all the tor_addr_t* in the smartlist. + */ +static void +policies_copy_addr_to_smartlist(smartlist_t *addr_list, const tor_addr_t *addr) +{ + if (addr && !tor_addr_is_null(addr)) { + tor_addr_t *addr_copy = tor_malloc(sizeof(tor_addr_t)); + tor_addr_copy(addr_copy, addr); + smartlist_add(addr_list, addr_copy); + } +} + +/** Helper function that adds ipv4h_addr to a smartlist as a tor_addr_t *, + * as long as it is not tor_addr_is_null(), by converting it to a tor_addr_t + * and passing it to policies_add_addr_to_smartlist. + * + * The caller is responsible for freeing all the tor_addr_t* in the smartlist. + */ +static void +policies_copy_ipv4h_to_smartlist(smartlist_t *addr_list, uint32_t ipv4h_addr) +{ + if (ipv4h_addr) { + tor_addr_t ipv4_tor_addr; + tor_addr_from_ipv4h(&ipv4_tor_addr, ipv4h_addr); + policies_copy_addr_to_smartlist(addr_list, &ipv4_tor_addr); + } +} + +/** Helper function that adds copies of + * or_options->OutboundBindAddressIPv[4|6]_ to a smartlist as tor_addr_t *, as + * long as or_options is non-NULL, and the addresses are not + * tor_addr_is_null(), by passing them to policies_add_addr_to_smartlist. + * + * The caller is responsible for freeing all the tor_addr_t* in the smartlist. + */ +static void +policies_copy_outbound_addresses_to_smartlist(smartlist_t *addr_list, + const or_options_t *or_options) +{ + if (or_options) { + policies_copy_addr_to_smartlist(addr_list, + &or_options->OutboundBindAddressIPv4_); + policies_copy_addr_to_smartlist(addr_list, + &or_options->OutboundBindAddressIPv6_); + } +} + /** Parse <b>ExitPolicy</b> member of <b>or_options</b> into <b>result</b> * smartlist. * If <b>or_options->IPv6Exit</b> is false, prepend an entry that @@ -1205,11 +1340,13 @@ policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, * - prepend an entry that rejects all destinations in all netblocks reserved * for private use. * - if local_address is non-zero, treat it as a host-order IPv4 address, and - * prepend an entry that rejects it as a destination. - * - if ipv6_local_address is non-NULL, prepend an entry that rejects it as a - * destination. - * - if reject_interface_addresses is true, prepend entries that reject each - * public IPv4 and IPv6 address of each interface on this machine. + * add it to the list of configured addresses. + * - if ipv6_local_address is non-NULL, and not the null tor_addr_t, add it + * to the list of configured addresses. + * - if or_options->OutboundBindAddressIPv4_ is not the null tor_addr_t, add + * it to the list of configured addresses. + * - if or_options->OutboundBindAddressIPv6_ is not the null tor_addr_t, add + * it to the list of configured addresses. * * If <b>or_options->BridgeRelay</b> is false, append entries of default * Tor exit policy into <b>result</b> smartlist. @@ -1220,18 +1357,23 @@ policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, int policies_parse_exit_policy_from_options(const or_options_t *or_options, uint32_t local_address, - tor_addr_t *ipv6_local_address, - int reject_interface_addresses, + const tor_addr_t *ipv6_local_address, smartlist_t **result) { exit_policy_parser_cfg_t parser_cfg = 0; + smartlist_t *configured_addresses = NULL; + int rv = 0; + /* Short-circuit for non-exit relays */ if (or_options->ExitRelay == 0) { append_exit_policy_string(result, "reject *4:*"); append_exit_policy_string(result, "reject *6:*"); return 0; } + configured_addresses = smartlist_new(); + + /* Configure the parser */ if (or_options->IPv6Exit) { parser_cfg |= EXIT_POLICY_IPV6_ENABLED; } @@ -1244,10 +1386,19 @@ policies_parse_exit_policy_from_options(const or_options_t *or_options, parser_cfg |= EXIT_POLICY_ADD_DEFAULT; } - return policies_parse_exit_policy(or_options->ExitPolicy,result, - parser_cfg,local_address, - ipv6_local_address, - reject_interface_addresses); + /* Copy the configured addresses into the tor_addr_t* list */ + policies_copy_ipv4h_to_smartlist(configured_addresses, local_address); + policies_copy_addr_to_smartlist(configured_addresses, ipv6_local_address); + policies_copy_outbound_addresses_to_smartlist(configured_addresses, + or_options); + + rv = policies_parse_exit_policy(or_options->ExitPolicy, result, parser_cfg, + configured_addresses); + + SMARTLIST_FOREACH(configured_addresses, tor_addr_t *, a, tor_free(a)); + smartlist_free(configured_addresses); + + return rv; } /** Add "reject *:*" to the end of the policy in *<b>dest</b>, allocating @@ -1355,7 +1506,7 @@ policy_is_reject_star(const smartlist_t *policy, sa_family_t family) /** Write a single address policy to the buf_len byte buffer at buf. Return * the number of characters written, or -1 on failure. */ int -policy_write_item(char *buf, size_t buflen, addr_policy_t *policy, +policy_write_item(char *buf, size_t buflen, const addr_policy_t *policy, int format_for_desc) { size_t written = 0; @@ -1934,6 +2085,53 @@ compare_tor_addr_to_node_policy(const tor_addr_t *addr, uint16_t port, } } +/** + * Given <b>policy_list</b>, a list of addr_policy_t, produce a string + * representation of the list. + * If <b>include_ipv4</b> is true, include IPv4 entries. + * If <b>include_ipv6</b> is true, include IPv6 entries. + */ +char * +policy_dump_to_string(const smartlist_t *policy_list, + int include_ipv4, + int include_ipv6) +{ + smartlist_t *policy_string_list; + char *policy_string = NULL; + + policy_string_list = smartlist_new(); + + SMARTLIST_FOREACH_BEGIN(policy_list, addr_policy_t *, tmpe) { + char *pbuf; + int bytes_written_to_pbuf; + if ((tor_addr_family(&tmpe->addr) == AF_INET6) && (!include_ipv6)) { + continue; /* Don't include IPv6 parts of address policy */ + } + if ((tor_addr_family(&tmpe->addr) == AF_INET) && (!include_ipv4)) { + continue; /* Don't include IPv4 parts of address policy */ + } + + pbuf = tor_malloc(POLICY_BUF_LEN); + bytes_written_to_pbuf = policy_write_item(pbuf,POLICY_BUF_LEN, tmpe, 1); + + if (bytes_written_to_pbuf < 0) { + log_warn(LD_BUG, "policy_dump_to_string ran out of room!"); + tor_free(pbuf); + goto done; + } + + smartlist_add(policy_string_list,pbuf); + } SMARTLIST_FOREACH_END(tmpe); + + policy_string = smartlist_join_strings(policy_string_list, "\n", 0, NULL); + + done: + SMARTLIST_FOREACH(policy_string_list, char *, str, tor_free(str)); + smartlist_free(policy_string_list); + + return policy_string; +} + /** Implementation for GETINFO control command: knows the answer for questions * about "exit-policy/..." */ int @@ -1945,6 +2143,57 @@ getinfo_helper_policies(control_connection_t *conn, (void) errmsg; if (!strcmp(question, "exit-policy/default")) { *answer = tor_strdup(DEFAULT_EXIT_POLICY); + } else if (!strcmp(question, "exit-policy/reject-private/default")) { + smartlist_t *private_policy_strings; + const char **priv = private_nets; + + private_policy_strings = smartlist_new(); + + while (*priv != NULL) { + /* IPv6 addresses are in "[]" and contain ":", + * IPv4 addresses are not in "[]" and contain "." */ + smartlist_add_asprintf(private_policy_strings, "reject %s:*", *priv); + priv++; + } + + *answer = smartlist_join_strings(private_policy_strings, + ",", 0, NULL); + + SMARTLIST_FOREACH(private_policy_strings, char *, str, tor_free(str)); + smartlist_free(private_policy_strings); + } else if (!strcmp(question, "exit-policy/reject-private/relay")) { + const or_options_t *options = get_options(); + const routerinfo_t *me = router_get_my_routerinfo(); + + if (!me) { + *errmsg = "router_get_my_routerinfo returned NULL"; + return -1; + } + + if (!options->ExitPolicyRejectPrivate) { + *answer = tor_strdup(""); + return 0; + } + + smartlist_t *private_policy_list = smartlist_new(); + smartlist_t *configured_addresses = smartlist_new(); + + /* Copy the configured addresses into the tor_addr_t* list */ + policies_copy_ipv4h_to_smartlist(configured_addresses, me->addr); + policies_copy_addr_to_smartlist(configured_addresses, &me->ipv6_addr); + policies_copy_outbound_addresses_to_smartlist(configured_addresses, + options); + + policies_parse_exit_policy_reject_private( + &private_policy_list, + options->IPv6Exit, + configured_addresses, + 1, 1); + *answer = policy_dump_to_string(private_policy_list, 1, 1); + + addr_policy_list_free(private_policy_list); + SMARTLIST_FOREACH(configured_addresses, tor_addr_t *, a, tor_free(a)); + smartlist_free(configured_addresses); } else if (!strcmpstart(question, "exit-policy/")) { const routerinfo_t *me = router_get_my_routerinfo(); diff --git a/src/or/policies.h b/src/or/policies.h index f200d7babe..007f494482 100644 --- a/src/or/policies.h +++ b/src/or/policies.h @@ -44,30 +44,38 @@ addr_policy_t *addr_policy_get_canonical_entry(addr_policy_t *ent); int cmp_addr_policies(smartlist_t *a, smartlist_t *b); MOCK_DECL(addr_policy_result_t, compare_tor_addr_to_addr_policy, (const tor_addr_t *addr, uint16_t port, const smartlist_t *policy)); - addr_policy_result_t compare_tor_addr_to_node_policy(const tor_addr_t *addr, uint16_t port, const node_t *node); -int policies_parse_exit_policy_from_options(const or_options_t *or_options, - uint32_t local_address, - tor_addr_t *ipv6_local_address, - int reject_interface_addresses, - smartlist_t **result); +int policies_parse_exit_policy_from_options( + const or_options_t *or_options, + uint32_t local_address, + const tor_addr_t *ipv6_local_address, + smartlist_t **result); int policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, exit_policy_parser_cfg_t options, - uint32_t local_address, - tor_addr_t *ipv6_local_address, - int reject_interface_addresses); + const smartlist_t *configured_addresses); +void policies_parse_exit_policy_reject_private( + smartlist_t **dest, + int ipv6_exit, + const smartlist_t *configured_addresses, + int reject_interface_addresses, + int reject_configured_port_addresses); void policies_exit_policy_append_reject_star(smartlist_t **dest); void addr_policy_append_reject_addr(smartlist_t **dest, const tor_addr_t *addr); +void addr_policy_append_reject_addr_list(smartlist_t **dest, + const smartlist_t *addrs); void policies_set_node_exitpolicy_to_reject_all(node_t *exitrouter); int exit_policy_is_general_exit(smartlist_t *policy); int policy_is_reject_star(const smartlist_t *policy, sa_family_t family); +char * policy_dump_to_string(const smartlist_t *policy_list, + int include_ipv4, + int include_ipv6); int getinfo_helper_policies(control_connection_t *conn, const char *question, char **answer, const char **errmsg); -int policy_write_item(char *buf, size_t buflen, addr_policy_t *item, +int policy_write_item(char *buf, size_t buflen, const addr_policy_t *item, int format_for_desc); void addr_policy_list_free(smartlist_t *p); @@ -84,5 +92,9 @@ addr_policy_result_t compare_tor_addr_to_short_policy( const tor_addr_t *addr, uint16_t port, const short_policy_t *policy); +#ifdef POLICIES_PRIVATE +STATIC void append_exit_policy_string(smartlist_t **policy, const char *more); +#endif + #endif diff --git a/src/or/relay.c b/src/or/relay.c index eddad6a0cb..ee2f041dbd 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -1304,6 +1304,7 @@ connection_edge_process_relay_cell_not_open( "Got 'connected' while not in state connect_wait. Dropping."); return 0; } + CONNECTION_AP_EXPECT_NONPENDING(entry_conn); conn->base_.state = AP_CONN_STATE_OPEN; log_info(LD_APP,"'connected' received for circid %u streamid %d " "after %d seconds.", @@ -2255,7 +2256,7 @@ circuit_consider_sending_sendme(circuit_t *circ, crypt_path_t *layer_hint) static size_t total_cells_allocated = 0; /** Release storage held by <b>cell</b>. */ -static INLINE void +static inline void packed_cell_free_unchecked(packed_cell_t *cell) { --total_cells_allocated; @@ -2299,7 +2300,7 @@ dump_cell_pool_usage(int severity) } /** Allocate a new copy of packed <b>cell</b>. */ -static INLINE packed_cell_t * +static inline packed_cell_t * packed_cell_copy(const cell_t *cell, int wide_circ_ids) { packed_cell_t *c = packed_cell_new(); diff --git a/src/or/rendcache.c b/src/or/rendcache.c index d4bdd68698..c69671e289 100644 --- a/src/or/rendcache.c +++ b/src/or/rendcache.c @@ -3,9 +3,10 @@ /** * \file rendcache.c - * \brief Hidden service desriptor cache. + * \brief Hidden service descriptor cache. **/ +#define RENDCACHE_PRIVATE #include "rendcache.h" #include "config.h" @@ -15,11 +16,14 @@ /** Map from service id (as generated by rend_get_service_id) to * rend_cache_entry_t. */ -static strmap_t *rend_cache = NULL; +STATIC strmap_t *rend_cache = NULL; + +/** Map from service id to rend_cache_entry_t; only for hidden services. */ +static strmap_t *rend_cache_local_service = NULL; /** Map from descriptor id to rend_cache_entry_t; only for hidden service * directories. */ -static digestmap_t *rend_cache_v2_dir = NULL; +STATIC digestmap_t *rend_cache_v2_dir = NULL; /** (Client side only) Map from service id to rend_cache_failure_t. This * cache is used to track intro point(IP) failures so we know when to keep @@ -46,10 +50,10 @@ static digestmap_t *rend_cache_v2_dir = NULL; * This scheme allows us to not realy on the descriptor's timestamp (which * is rounded down to the hour) to know if we have a newer descriptor. We * only rely on the usability of intro points from an internal state. */ -static strmap_t *rend_cache_failure = NULL; +STATIC strmap_t *rend_cache_failure = NULL; /** DOCDOC */ -static size_t rend_cache_total_allocation = 0; +STATIC size_t rend_cache_total_allocation = 0; /** Initializes the service descriptor cache. */ @@ -58,11 +62,12 @@ rend_cache_init(void) { rend_cache = strmap_new(); rend_cache_v2_dir = digestmap_new(); + rend_cache_local_service = strmap_new(); rend_cache_failure = strmap_new(); } /** Return the approximate number of bytes needed to hold <b>e</b>. */ -static size_t +STATIC size_t rend_cache_entry_allocation(const rend_cache_entry_t *e) { if (!e) @@ -80,7 +85,7 @@ rend_cache_get_total_allocation(void) } /** Decrement the total bytes attributed to the rendezvous cache by n. */ -static void +STATIC void rend_cache_decrement_allocation(size_t n) { static int have_underflowed = 0; @@ -97,7 +102,7 @@ rend_cache_decrement_allocation(size_t n) } /** Increase the total bytes attributed to the rendezvous cache by n. */ -static void +STATIC void rend_cache_increment_allocation(size_t n) { static int have_overflowed = 0; @@ -113,7 +118,7 @@ rend_cache_increment_allocation(size_t n) } /** Helper: free a rend cache failure intro object. */ -static void +STATIC void rend_cache_failure_intro_entry_free(rend_cache_failure_intro_t *entry) { if (entry == NULL) { @@ -130,7 +135,7 @@ rend_cache_failure_intro_entry_free_(void *entry) /** Allocate a rend cache failure intro object and return it. <b>failure</b> * is set into the object. This function can not fail. */ -static rend_cache_failure_intro_t * +STATIC rend_cache_failure_intro_t * rend_cache_failure_intro_entry_new(rend_intro_point_failure_t failure) { rend_cache_failure_intro_t *entry = tor_malloc(sizeof(*entry)); @@ -140,7 +145,7 @@ rend_cache_failure_intro_entry_new(rend_intro_point_failure_t failure) } /** Helper: free a rend cache failure object. */ -static void +STATIC void rend_cache_failure_entry_free(rend_cache_failure_t *entry) { if (entry == NULL) { @@ -156,7 +161,7 @@ rend_cache_failure_entry_free(rend_cache_failure_t *entry) /** Helper: deallocate a rend_cache_failure_t. (Used with strmap_free(), * which requires a function pointer whose argument is void*). */ -static void +STATIC void rend_cache_failure_entry_free_(void *entry) { rend_cache_failure_entry_free(entry); @@ -164,7 +169,7 @@ rend_cache_failure_entry_free_(void *entry) /** Allocate a rend cache failure object and return it. This function can * not fail. */ -static rend_cache_failure_t * +STATIC rend_cache_failure_t * rend_cache_failure_entry_new(void) { rend_cache_failure_t *entry = tor_malloc(sizeof(*entry)); @@ -174,7 +179,7 @@ rend_cache_failure_entry_new(void) /** Remove failure cache entry for the service ID in the given descriptor * <b>desc</b>. */ -static void +STATIC void rend_cache_failure_remove(rend_service_descriptor_t *desc) { char service_id[REND_SERVICE_ID_LEN_BASE32 + 1]; @@ -194,7 +199,7 @@ rend_cache_failure_remove(rend_service_descriptor_t *desc) } /** Helper: free storage held by a single service descriptor cache entry. */ -static void +STATIC void rend_cache_entry_free(rend_cache_entry_t *e) { if (!e) @@ -222,9 +227,11 @@ rend_cache_free_all(void) { strmap_free(rend_cache, rend_cache_entry_free_); digestmap_free(rend_cache_v2_dir, rend_cache_entry_free_); + strmap_free(rend_cache_local_service, rend_cache_entry_free_); strmap_free(rend_cache_failure, rend_cache_failure_entry_free_); rend_cache = NULL; rend_cache_v2_dir = NULL; + rend_cache_local_service = NULL; rend_cache_failure = NULL; rend_cache_total_allocation = 0; } @@ -258,24 +265,33 @@ rend_cache_failure_clean(time_t now) } STRMAP_FOREACH_END; } -/** Removes all old entries from the service descriptor cache. +/** Removes all old entries from the client or service descriptor cache. */ void -rend_cache_clean(time_t now) +rend_cache_clean(time_t now, rend_cache_type_t cache_type) { strmap_iter_t *iter; const char *key; void *val; rend_cache_entry_t *ent; time_t cutoff = now - REND_CACHE_MAX_AGE - REND_CACHE_MAX_SKEW; - for (iter = strmap_iter_init(rend_cache); !strmap_iter_done(iter); ) { + strmap_t *cache = NULL; + + if (cache_type == REND_CACHE_TYPE_CLIENT) { + cache = rend_cache; + } else if (cache_type == REND_CACHE_TYPE_SERVICE) { + cache = rend_cache_local_service; + } + tor_assert(cache); + + for (iter = strmap_iter_init(cache); !strmap_iter_done(iter); ) { strmap_iter_get(iter, &key, &val); ent = (rend_cache_entry_t*)val; if (ent->parsed->timestamp < cutoff) { - iter = strmap_iter_next_rmv(rend_cache, iter); + iter = strmap_iter_next_rmv(cache, iter); rend_cache_entry_free(ent); } else { - iter = strmap_iter_next(rend_cache, iter); + iter = strmap_iter_next(cache, iter); } } } @@ -305,10 +321,10 @@ rend_cache_failure_purge(void) } /** Lookup the rend failure cache using a relay identity digest in - * <b>identity</b> and service ID <b>service_id</b>. If found, the intro - * failure is set in <b>intro_entry</b> else it stays untouched. Return 1 - * iff found else 0. */ -static int + * <b>identity</b> which has DIGEST_LEN bytes and service ID <b>service_id</b> + * which is a null-terminated string. If found, the intro failure is set in + * <b>intro_entry</b> else it stays untouched. Return 1 iff found else 0. */ +STATIC int cache_failure_intro_lookup(const uint8_t *identity, const char *service_id, rend_cache_failure_intro_t **intro_entry) { @@ -352,7 +368,7 @@ cache_failure_intro_dup(const rend_cache_failure_intro_t *entry) /** Add an intro point failure to the failure cache using the relay * <b>identity</b> and service ID <b>service_id</b>. Record the * <b>failure</b> in that object. */ -static void +STATIC void cache_failure_intro_add(const uint8_t *identity, const char *service_id, rend_intro_point_failure_t failure) { @@ -379,7 +395,7 @@ cache_failure_intro_add(const uint8_t *identity, const char *service_id, * descriptor and kept into the failure cache. Then, each intro points that * are NOT in the descriptor but in the failure cache for the given * <b>service_id</b> are removed from the failure cache. */ -static void +STATIC void validate_intro_point_failure(const rend_service_descriptor_t *desc, const char *service_id) { @@ -535,6 +551,42 @@ rend_cache_lookup_entry(const char *query, int version, rend_cache_entry_t **e) return ret; } +/* + * Lookup the v2 service descriptor with the service ID <b>query</b> in the + * local service descriptor cache. Return 0 if found and if <b>e</b> is + * non NULL, set it with the entry found. Else, a negative value is returned + * and <b>e</b> is untouched. + * -EINVAL means that <b>query</b> is not a valid service id. + * -ENOENT means that no entry in the cache was found. */ +int +rend_cache_lookup_v2_desc_as_service(const char *query, rend_cache_entry_t **e) +{ + int ret = 0; + rend_cache_entry_t *entry = NULL; + + tor_assert(rend_cache_local_service); + tor_assert(query); + + if (!rend_valid_service_id(query)) { + ret = -EINVAL; + goto end; + } + + /* Lookup descriptor and return. */ + entry = strmap_get_lc(rend_cache_local_service, query); + if (!entry) { + ret = -ENOENT; + goto end; + } + + if (e) { + *e = entry; + } + + end: + return ret; +} + /** Lookup the v2 service descriptor with base32-encoded <b>desc_id</b> and * copy the pointer to it to *<b>desc</b>. Return 1 on success, 0 on * well-formed-but-not-found, and -1 on failure. @@ -660,7 +712,6 @@ rend_cache_store_v2_desc_as_dir(const char *desc) log_info(LD_REND, "Successfully stored service descriptor with desc ID " "'%s' and len %d.", safe_str(desc_id_base32), (int)encoded_size); - /* Statistics: Note down this potentially new HS. */ if (options->HiddenServiceStatistics) { rep_hist_stored_maybe_new_hs(e->parsed->pk); @@ -687,6 +738,80 @@ rend_cache_store_v2_desc_as_dir(const char *desc) return RCS_OKAY; } +/** Parse the v2 service descriptor in <b>desc</b> and store it to the +* local service rend cache. Don't attempt to decrypt the included list of +* introduction points. +* +* If we have a newer descriptor with the same ID, ignore this one. +* If we have an older descriptor with the same ID, replace it. +* +* Return an appropriate rend_cache_store_status_t. +*/ +rend_cache_store_status_t +rend_cache_store_v2_desc_as_service(const char *desc) +{ + rend_service_descriptor_t *parsed = NULL; + char desc_id[DIGEST_LEN]; + char *intro_content = NULL; + size_t intro_size; + size_t encoded_size; + const char *next_desc; + char service_id[REND_SERVICE_ID_LEN_BASE32+1]; + rend_cache_entry_t *e; + rend_cache_store_status_t retval = RCS_BADDESC; + tor_assert(rend_cache_local_service); + tor_assert(desc); + + /* Parse the descriptor. */ + if (rend_parse_v2_service_descriptor(&parsed, desc_id, &intro_content, + &intro_size, &encoded_size, + &next_desc, desc, 0) < 0) { + log_warn(LD_REND, "Could not parse descriptor."); + goto err; + } + /* Compute service ID from public key. */ + if (rend_get_service_id(parsed->pk, service_id)<0) { + log_warn(LD_REND, "Couldn't compute service ID."); + goto err; + } + + /* Do we already have a newer descriptor? Allow new descriptors with a + rounded timestamp equal to or newer than the current descriptor */ + e = (rend_cache_entry_t*) strmap_get_lc(rend_cache_local_service, + service_id); + if (e && e->parsed->timestamp > parsed->timestamp) { + log_info(LD_REND, "We already have a newer service descriptor for " + "service ID %s.", safe_str_client(service_id)); + goto okay; + } + /* We don't care about the introduction points. */ + tor_free(intro_content); + if (!e) { + e = tor_malloc_zero(sizeof(rend_cache_entry_t)); + strmap_set_lc(rend_cache_local_service, service_id, e); + } else { + rend_cache_decrement_allocation(rend_cache_entry_allocation(e)); + rend_service_descriptor_free(e->parsed); + tor_free(e->desc); + } + e->parsed = parsed; + e->desc = tor_malloc_zero(encoded_size + 1); + strlcpy(e->desc, desc, encoded_size + 1); + e->len = encoded_size; + rend_cache_increment_allocation(rend_cache_entry_allocation(e)); + log_debug(LD_REND,"Successfully stored rend desc '%s', len %d.", + safe_str_client(service_id), (int)encoded_size); + return RCS_OKAY; + + okay: + retval = RCS_OKAY; + + err: + rend_service_descriptor_free(parsed); + tor_free(intro_content); + return retval; +} + /** Parse the v2 service descriptor in <b>desc</b>, decrypt the included list * of introduction points with <b>descriptor_cookie</b> (which may also be * <b>NULL</b> if decryption is not necessary), and store the descriptor to diff --git a/src/or/rendcache.h b/src/or/rendcache.h index 0512058054..decb040ee7 100644 --- a/src/or/rendcache.h +++ b/src/or/rendcache.h @@ -48,14 +48,21 @@ typedef struct rend_cache_failure_t { digestmap_t *intro_failures; } rend_cache_failure_t; +typedef enum { + REND_CACHE_TYPE_CLIENT = 1, + REND_CACHE_TYPE_SERVICE = 2, +} rend_cache_type_t; + void rend_cache_init(void); -void rend_cache_clean(time_t now); +void rend_cache_clean(time_t now, rend_cache_type_t cache_type); void rend_cache_failure_clean(time_t now); void rend_cache_clean_v2_descs_as_dir(time_t now, size_t min_to_remove); void rend_cache_purge(void); void rend_cache_free_all(void); int rend_cache_lookup_entry(const char *query, int version, rend_cache_entry_t **entry_out); +int rend_cache_lookup_v2_desc_as_service(const char *query, + rend_cache_entry_t **entry_out); int rend_cache_lookup_v2_desc_as_dir(const char *query, const char **desc); /** Return value from rend_cache_store_v2_desc_as_{dir,client}. */ typedef enum { @@ -65,6 +72,8 @@ typedef enum { } rend_cache_store_status_t; rend_cache_store_status_t rend_cache_store_v2_desc_as_dir(const char *desc); +rend_cache_store_status_t rend_cache_store_v2_desc_as_service( + const char *desc); rend_cache_store_status_t rend_cache_store_v2_desc_as_client(const char *desc, const char *desc_id_base32, const rend_data_t *rend_query, @@ -76,5 +85,31 @@ void rend_cache_intro_failure_note(rend_intro_point_failure_t failure, const char *service_id); void rend_cache_failure_purge(void); +#ifdef RENDCACHE_PRIVATE + +STATIC size_t rend_cache_entry_allocation(const rend_cache_entry_t *e); +STATIC void rend_cache_entry_free(rend_cache_entry_t *e); +STATIC void rend_cache_failure_intro_entry_free(rend_cache_failure_intro_t + *entry); +STATIC void rend_cache_failure_entry_free(rend_cache_failure_t *entry); +STATIC int cache_failure_intro_lookup(const uint8_t *identity, + const char *service_id, + rend_cache_failure_intro_t + **intro_entry); +STATIC void rend_cache_decrement_allocation(size_t n); +STATIC void rend_cache_increment_allocation(size_t n); +STATIC rend_cache_failure_intro_t *rend_cache_failure_intro_entry_new( + rend_intro_point_failure_t failure); +STATIC rend_cache_failure_t *rend_cache_failure_entry_new(void); +STATIC void rend_cache_failure_remove(rend_service_descriptor_t *desc); +STATIC void cache_failure_intro_add(const uint8_t *identity, + const char *service_id, + rend_intro_point_failure_t failure); +STATIC void validate_intro_point_failure(const rend_service_descriptor_t *desc, + const char *service_id); + +STATIC void rend_cache_failure_entry_free_(void *entry); +#endif + #endif /* TOR_RENDCACHE_H */ diff --git a/src/or/rendclient.c b/src/or/rendclient.c index a39e518e99..d9cea53c04 100644 --- a/src/or/rendclient.c +++ b/src/or/rendclient.c @@ -52,7 +52,7 @@ rend_client_introcirc_has_opened(origin_circuit_t *circ) tor_assert(circ->cpath); log_info(LD_REND,"introcirc is open"); - connection_ap_attach_pending(); + connection_ap_attach_pending(1); } /** Send the establish-rendezvous cell along a rendezvous circuit. if @@ -65,11 +65,7 @@ rend_client_send_establish_rendezvous(origin_circuit_t *circ) tor_assert(circ->rend_data); log_info(LD_REND, "Sending an ESTABLISH_RENDEZVOUS cell"); - if (crypto_rand(circ->rend_data->rend_cookie, REND_COOKIE_LEN) < 0) { - log_warn(LD_BUG, "Internal error: Couldn't produce random cookie."); - circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_INTERNAL); - return -1; - } + crypto_rand(circ->rend_data->rend_cookie, REND_COOKIE_LEN); /* Set timestamp_dirty, because circuit_expire_building expects it, * and the rend cookie also means we've used the circ. */ @@ -177,6 +173,7 @@ rend_client_send_introduction(origin_circuit_t *introcirc, while ((conn = connection_get_by_type_state_rendquery(CONN_TYPE_AP, AP_CONN_STATE_CIRCUIT_WAIT, introcirc->rend_data->onion_address))) { + connection_ap_mark_as_non_pending_circuit(TO_ENTRY_CONN(conn)); conn->state = AP_CONN_STATE_RENDDESC_WAIT; } } @@ -1059,9 +1056,11 @@ rend_client_report_intro_point_failure(extend_info_t *failed_intro, rend_client_refetch_v2_renddesc(rend_query); /* move all pending streams back to renddesc_wait */ + /* NOTE: We can now do this faster, if we use pending_entry_connections */ while ((conn = connection_get_by_type_state_rendquery(CONN_TYPE_AP, AP_CONN_STATE_CIRCUIT_WAIT, rend_query->onion_address))) { + connection_ap_mark_as_non_pending_circuit(TO_ENTRY_CONN(conn)); conn->state = AP_CONN_STATE_RENDDESC_WAIT; } @@ -1107,7 +1106,7 @@ rend_client_rendezvous_acked(origin_circuit_t *circ, const uint8_t *request, * than trying to attach them all. See comments bug 743. */ /* If we already have the introduction circuit built, make sure we send * the INTRODUCE cell _now_ */ - connection_ap_attach_pending(); + connection_ap_attach_pending(1); return 0; } @@ -1226,12 +1225,7 @@ rend_client_desc_trynow(const char *query) base_conn->timestamp_lastread = now; base_conn->timestamp_lastwritten = now; - if (connection_ap_handshake_attach_circuit(conn) < 0) { - /* it will never work */ - log_warn(LD_REND,"Rendezvous attempt failed. Closing."); - if (!base_conn->marked_for_close) - connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); - } + connection_ap_mark_as_pending_circuit(conn); } else { /* 404, or fetch didn't get that far */ log_notice(LD_REND,"Closing stream for '%s.onion': hidden service is " "unavailable (try again later).", diff --git a/src/or/rendcommon.c b/src/or/rendcommon.c index 22599e9830..8c02b67556 100644 --- a/src/or/rendcommon.c +++ b/src/or/rendcommon.c @@ -11,6 +11,7 @@ #include "or.h" #include "circuitbuild.h" #include "config.h" +#include "control.h" #include "rendclient.h" #include "rendcommon.h" #include "rendmid.h" @@ -268,11 +269,7 @@ rend_encrypt_v2_intro_points_basic(char **encrypted_out, tor_assert(client_cookies && smartlist_len(client_cookies) > 0); /* Generate session key. */ - if (crypto_rand(session_key, CIPHER_KEY_LEN) < 0) { - log_warn(LD_REND, "Unable to generate random session key to encrypt " - "introduction point string."); - goto done; - } + crypto_rand(session_key, CIPHER_KEY_LEN); /* Determine length of encrypted introduction points including session * keys. */ @@ -334,11 +331,7 @@ rend_encrypt_v2_intro_points_basic(char **encrypted_out, REND_BASIC_AUTH_CLIENT_MULTIPLE; i < REND_BASIC_AUTH_CLIENT_MULTIPLE - 1; i++) { client_part = tor_malloc_zero(REND_BASIC_AUTH_CLIENT_ENTRY_LEN); - if (crypto_rand(client_part, REND_BASIC_AUTH_CLIENT_ENTRY_LEN) < 0) { - log_warn(LD_REND, "Unable to generate fake client entry."); - tor_free(client_part); - goto done; - } + crypto_rand(client_part, REND_BASIC_AUTH_CLIENT_ENTRY_LEN); smartlist_add(encrypted_session_keys, client_part); } /* Sort smartlist and put elements in result in order. */ @@ -461,6 +454,7 @@ rend_encode_v2_descriptors(smartlist_t *descs_out, smartlist_t *client_cookies) { char service_id[DIGEST_LEN]; + char service_id_base32[REND_SERVICE_ID_LEN_BASE32+1]; uint32_t time_period; char *ipos_base64 = NULL, *ipos = NULL, *ipos_encrypted = NULL, *descriptor_cookie = NULL; @@ -655,6 +649,11 @@ rend_encode_v2_descriptors(smartlist_t *descs_out, goto err; } smartlist_add(descs_out, enc); + /* Add the uploaded descriptor to the local service's descriptor cache */ + rend_cache_store_v2_desc_as_service(enc->desc_str); + base32_encode(service_id_base32, sizeof(service_id_base32), + service_id, REND_SERVICE_ID_LEN); + control_event_hs_descriptor_created(service_id_base32, desc_id_base32, k); } log_info(LD_REND, "Successfully encoded a v2 descriptor and " diff --git a/src/or/rendcommon.h b/src/or/rendcommon.h index 3b2f86d614..04e34af453 100644 --- a/src/or/rendcommon.h +++ b/src/or/rendcommon.h @@ -19,7 +19,7 @@ typedef enum rend_intro_point_failure_t { } rend_intro_point_failure_t; /** Free all storage associated with <b>data</b> */ -static INLINE void +static inline void rend_data_free(rend_data_t *data) { if (!data) { diff --git a/src/or/rendservice.c b/src/or/rendservice.c index 77d8b716a2..15d98bfde5 100644 --- a/src/or/rendservice.c +++ b/src/or/rendservice.c @@ -3203,39 +3203,72 @@ upload_service_descriptor(rend_service_t *service) rendpostperiod = get_options()->RendPostPeriod; - /* Upload descriptor? */ - if (get_options()->PublishHidServDescriptors) { - networkstatus_t *c = networkstatus_get_latest_consensus(); - if (c && smartlist_len(c->routerstatus_list) > 0) { - int seconds_valid, i, j, num_descs; - smartlist_t *descs = smartlist_new(); - smartlist_t *client_cookies = smartlist_new(); - /* Either upload a single descriptor (including replicas) or one - * descriptor for each authorized client in case of authorization - * type 'stealth'. */ - num_descs = service->auth_type == REND_STEALTH_AUTH ? - smartlist_len(service->clients) : 1; - for (j = 0; j < num_descs; j++) { - crypto_pk_t *client_key = NULL; - rend_authorized_client_t *client = NULL; - smartlist_clear(client_cookies); - switch (service->auth_type) { - case REND_NO_AUTH: - /* Do nothing here. */ - break; - case REND_BASIC_AUTH: - SMARTLIST_FOREACH(service->clients, rend_authorized_client_t *, - cl, smartlist_add(client_cookies, cl->descriptor_cookie)); - break; - case REND_STEALTH_AUTH: - client = smartlist_get(service->clients, j); - client_key = client->client_key; - smartlist_add(client_cookies, client->descriptor_cookie); - break; - } - /* Encode the current descriptor. */ + networkstatus_t *c = networkstatus_get_latest_consensus(); + if (c && smartlist_len(c->routerstatus_list) > 0) { + int seconds_valid, i, j, num_descs; + smartlist_t *descs = smartlist_new(); + smartlist_t *client_cookies = smartlist_new(); + /* Either upload a single descriptor (including replicas) or one + * descriptor for each authorized client in case of authorization + * type 'stealth'. */ + num_descs = service->auth_type == REND_STEALTH_AUTH ? + smartlist_len(service->clients) : 1; + for (j = 0; j < num_descs; j++) { + crypto_pk_t *client_key = NULL; + rend_authorized_client_t *client = NULL; + smartlist_clear(client_cookies); + switch (service->auth_type) { + case REND_NO_AUTH: + /* Do nothing here. */ + break; + case REND_BASIC_AUTH: + SMARTLIST_FOREACH(service->clients, rend_authorized_client_t *, + cl, smartlist_add(client_cookies, cl->descriptor_cookie)); + break; + case REND_STEALTH_AUTH: + client = smartlist_get(service->clients, j); + client_key = client->client_key; + smartlist_add(client_cookies, client->descriptor_cookie); + break; + } + /* Encode the current descriptor. */ + seconds_valid = rend_encode_v2_descriptors(descs, service->desc, + now, 0, + service->auth_type, + client_key, + client_cookies); + if (seconds_valid < 0) { + log_warn(LD_BUG, "Internal error: couldn't encode service " + "descriptor; not uploading."); + smartlist_free(descs); + smartlist_free(client_cookies); + return; + } + rend_get_service_id(service->desc->pk, serviceid); + if (get_options()->PublishHidServDescriptors) { + /* Post the current descriptors to the hidden service directories. */ + log_info(LD_REND, "Launching upload for hidden service %s", + serviceid); + directory_post_to_hs_dir(service->desc, descs, NULL, serviceid, + seconds_valid); + } + /* Free memory for descriptors. */ + for (i = 0; i < smartlist_len(descs); i++) + rend_encoded_v2_service_descriptor_free(smartlist_get(descs, i)); + smartlist_clear(descs); + /* Update next upload time. */ + if (seconds_valid - REND_TIME_PERIOD_OVERLAPPING_V2_DESCS + > rendpostperiod) + service->next_upload_time = now + rendpostperiod; + else if (seconds_valid < REND_TIME_PERIOD_OVERLAPPING_V2_DESCS) + service->next_upload_time = now + seconds_valid + 1; + else + service->next_upload_time = now + seconds_valid - + REND_TIME_PERIOD_OVERLAPPING_V2_DESCS + 1; + /* Post also the next descriptors, if necessary. */ + if (seconds_valid < REND_TIME_PERIOD_OVERLAPPING_V2_DESCS) { seconds_valid = rend_encode_v2_descriptors(descs, service->desc, - now, 0, + now, 1, service->auth_type, client_key, client_cookies); @@ -3246,51 +3279,23 @@ upload_service_descriptor(rend_service_t *service) smartlist_free(client_cookies); return; } - /* Post the current descriptors to the hidden service directories. */ - rend_get_service_id(service->desc->pk, serviceid); - log_info(LD_REND, "Launching upload for hidden service %s", - serviceid); - directory_post_to_hs_dir(service->desc, descs, NULL, serviceid, - seconds_valid); + if (get_options()->PublishHidServDescriptors) { + directory_post_to_hs_dir(service->desc, descs, NULL, serviceid, + seconds_valid); + } /* Free memory for descriptors. */ for (i = 0; i < smartlist_len(descs); i++) rend_encoded_v2_service_descriptor_free(smartlist_get(descs, i)); smartlist_clear(descs); - /* Update next upload time. */ - if (seconds_valid - REND_TIME_PERIOD_OVERLAPPING_V2_DESCS - > rendpostperiod) - service->next_upload_time = now + rendpostperiod; - else if (seconds_valid < REND_TIME_PERIOD_OVERLAPPING_V2_DESCS) - service->next_upload_time = now + seconds_valid + 1; - else - service->next_upload_time = now + seconds_valid - - REND_TIME_PERIOD_OVERLAPPING_V2_DESCS + 1; - /* Post also the next descriptors, if necessary. */ - if (seconds_valid < REND_TIME_PERIOD_OVERLAPPING_V2_DESCS) { - seconds_valid = rend_encode_v2_descriptors(descs, service->desc, - now, 1, - service->auth_type, - client_key, - client_cookies); - if (seconds_valid < 0) { - log_warn(LD_BUG, "Internal error: couldn't encode service " - "descriptor; not uploading."); - smartlist_free(descs); - smartlist_free(client_cookies); - return; - } - directory_post_to_hs_dir(service->desc, descs, NULL, serviceid, - seconds_valid); - /* Free memory for descriptors. */ - for (i = 0; i < smartlist_len(descs); i++) - rend_encoded_v2_service_descriptor_free(smartlist_get(descs, i)); - smartlist_clear(descs); - } } - smartlist_free(descs); - smartlist_free(client_cookies); - uploaded = 1; + } + smartlist_free(descs); + smartlist_free(client_cookies); + uploaded = 1; + if (get_options()->PublishHidServDescriptors) { log_info(LD_REND, "Successfully uploaded v2 rend descriptors!"); + } else { + log_info(LD_REND, "Successfully stored created v2 rend descriptors!"); } } @@ -3635,9 +3640,6 @@ rend_consider_services_upload(time_t now) MIN_REND_INITIAL_POST_DELAY_TESTING : MIN_REND_INITIAL_POST_DELAY); - if (!get_options()->PublishHidServDescriptors) - return; - for (i=0; i < smartlist_len(rend_service_list); ++i) { service = smartlist_get(rend_service_list, i); if (!service->next_upload_time) { /* never been uploaded yet */ diff --git a/src/or/rephist.c b/src/or/rephist.c index fe0997c891..d55317947c 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -148,7 +148,7 @@ get_link_history(const char *from_id, const char *to_id) return NULL; if (tor_digest_is_zero(to_id)) return NULL; - lhist = (link_history_t*) digestmap_get(orhist->link_history_map, to_id); + lhist = digestmap_get(orhist->link_history_map, to_id); if (!lhist) { lhist = tor_malloc_zero(sizeof(link_history_t)); rephist_total_alloc += sizeof(link_history_t); @@ -920,7 +920,7 @@ parse_possibly_bad_iso_time(const char *s, time_t *time_out) * that's about as much before <b>now</b> as <b>t</b> was before * <b>stored_at</b>. */ -static INLINE time_t +static inline time_t correct_time(time_t t, time_t now, time_t stored_at, time_t started_measuring) { if (t < started_measuring - 24*60*60*365) @@ -1190,7 +1190,7 @@ commit_max(bw_array_t *b) } /** Shift the current observation time of <b>b</b> forward by one second. */ -static INLINE void +static inline void advance_obs(bw_array_t *b) { int nextidx; @@ -1216,7 +1216,7 @@ advance_obs(bw_array_t *b) /** Add <b>n</b> bytes to the number of bytes in <b>b</b> for second * <b>when</b>. */ -static INLINE void +static inline void add_obs(bw_array_t *b, time_t when, uint64_t n) { if (when < b->cur_obs_time) @@ -1250,6 +1250,18 @@ bw_array_new(void) return b; } +/** Free storage held by bandwidth array <b>b</b>. */ +static void +bw_array_free(bw_array_t *b) +{ + if (!b) { + return; + } + + rephist_total_alloc -= sizeof(bw_array_t); + tor_free(b); +} + /** Recent history of bandwidth observations for read operations. */ static bw_array_t *read_array = NULL; /** Recent history of bandwidth observations for write operations. */ @@ -1266,10 +1278,11 @@ static bw_array_t *dir_write_array = NULL; static void bw_arrays_init(void) { - tor_free(read_array); - tor_free(write_array); - tor_free(dir_read_array); - tor_free(dir_write_array); + bw_array_free(read_array); + bw_array_free(write_array); + bw_array_free(dir_read_array); + bw_array_free(dir_write_array); + read_array = bw_array_new(); write_array = bw_array_new(); dir_read_array = bw_array_new(); @@ -3026,21 +3039,21 @@ rep_hist_stored_maybe_new_hs(const crypto_pk_t *pubkey) /* The number of cells that are supposed to be hidden from the adversary * by adding noise from the Laplace distribution. This value, divided by - * EPSILON, is Laplace parameter b. */ + * EPSILON, is Laplace parameter b. It must be greather than 0. */ #define REND_CELLS_DELTA_F 2048 /* Security parameter for obfuscating number of cells with a value between - * 0 and 1. Smaller values obfuscate observations more, but at the same + * ]0.0, 1.0]. Smaller values obfuscate observations more, but at the same * time make statistics less usable. */ #define REND_CELLS_EPSILON 0.3 /* The number of cells that are supposed to be hidden from the adversary * by rounding up to the next multiple of this number. */ #define REND_CELLS_BIN_SIZE 1024 -/* The number of service identities that are supposed to be hidden from - * the adversary by adding noise from the Laplace distribution. This - * value, divided by EPSILON, is Laplace parameter b. */ +/* The number of service identities that are supposed to be hidden from the + * adversary by adding noise from the Laplace distribution. This value, + * divided by EPSILON, is Laplace parameter b. It must be greater than 0. */ #define ONIONS_SEEN_DELTA_F 8 /* Security parameter for obfuscating number of service identities with a - * value between 0 and 1. Smaller values obfuscate observations more, but + * value between ]0.0, 1.0]. Smaller values obfuscate observations more, but * at the same time make statistics less usable. */ #define ONIONS_SEEN_EPSILON 0.3 /* The number of service identities that are supposed to be hidden from @@ -3172,10 +3185,19 @@ rep_hist_free_all(void) { hs_stats_free(hs_stats); digestmap_free(history_map, free_or_history); - tor_free(read_array); - tor_free(write_array); - tor_free(dir_read_array); - tor_free(dir_write_array); + + bw_array_free(read_array); + read_array = NULL; + + bw_array_free(write_array); + write_array = NULL; + + bw_array_free(dir_read_array); + dir_read_array = NULL; + + bw_array_free(dir_write_array); + dir_write_array = NULL; + tor_free(exit_bytes_read); tor_free(exit_bytes_written); tor_free(exit_streams); @@ -3190,5 +3212,8 @@ rep_hist_free_all(void) } rep_hist_desc_stats_term(); total_descriptor_downloads = 0; + + tor_assert(rephist_total_alloc == 0); + tor_assert(rephist_total_num == 0); } diff --git a/src/or/replaycache.c b/src/or/replaycache.c index 569e0736cb..82e5c44d3d 100644 --- a/src/or/replaycache.c +++ b/src/or/replaycache.c @@ -23,7 +23,7 @@ replaycache_free(replaycache_t *r) return; } - if (r->digests_seen) digestmap_free(r->digests_seen, tor_free_); + if (r->digests_seen) digest256map_free(r->digests_seen, tor_free_); tor_free(r); } @@ -54,7 +54,7 @@ replaycache_new(time_t horizon, time_t interval) r->scrub_interval = interval; r->scrubbed = 0; r->horizon = horizon; - r->digests_seen = digestmap_new(); + r->digests_seen = digest256map_new(); err: return r; @@ -69,7 +69,7 @@ replaycache_add_and_test_internal( time_t *elapsed) { int rv = 0; - char digest[DIGEST_LEN]; + uint8_t digest[DIGEST256_LEN]; time_t *access_time; /* sanity check */ @@ -80,10 +80,10 @@ replaycache_add_and_test_internal( } /* compute digest */ - crypto_digest(digest, (const char *)data, len); + crypto_digest256((char *)digest, (const char *)data, len, DIGEST_SHA256); /* check map */ - access_time = digestmap_get(r->digests_seen, digest); + access_time = digest256map_get(r->digests_seen, digest); /* seen before? */ if (access_time != NULL) { @@ -114,7 +114,7 @@ replaycache_add_and_test_internal( /* No, so no hit and update the digest map with the current time */ access_time = tor_malloc(sizeof(*access_time)); *access_time = present; - digestmap_set(r->digests_seen, digest, access_time); + digest256map_set(r->digests_seen, digest, access_time); } /* now scrub the cache if it's time */ @@ -130,8 +130,8 @@ replaycache_add_and_test_internal( STATIC void replaycache_scrub_if_needed_internal(time_t present, replaycache_t *r) { - digestmap_iter_t *itr = NULL; - const char *digest; + digest256map_iter_t *itr = NULL; + const uint8_t *digest; void *valp; time_t *access_time; @@ -149,19 +149,19 @@ replaycache_scrub_if_needed_internal(time_t present, replaycache_t *r) if (r->horizon == 0) return; /* okay, scrub time */ - itr = digestmap_iter_init(r->digests_seen); - while (!digestmap_iter_done(itr)) { - digestmap_iter_get(itr, &digest, &valp); + itr = digest256map_iter_init(r->digests_seen); + while (!digest256map_iter_done(itr)) { + digest256map_iter_get(itr, &digest, &valp); access_time = (time_t *)valp; /* aged out yet? */ if (*access_time < present - r->horizon) { /* Advance the iterator and remove this one */ - itr = digestmap_iter_next_rmv(r->digests_seen, itr); + itr = digest256map_iter_next_rmv(r->digests_seen, itr); /* Free the value removed */ tor_free(access_time); } else { /* Just advance the iterator */ - itr = digestmap_iter_next(r->digests_seen, itr); + itr = digest256map_iter_next(r->digests_seen, itr); } } diff --git a/src/or/replaycache.h b/src/or/replaycache.h index 9b9daf3831..9c409f2fd7 100644 --- a/src/or/replaycache.h +++ b/src/or/replaycache.h @@ -26,7 +26,7 @@ struct replaycache_s { /* * Digest map: keys are digests, values are times the digest was last seen */ - digestmap_t *digests_seen; + digest256map_t *digests_seen; }; #endif /* REPLAYCACHE_PRIVATE */ diff --git a/src/or/router.c b/src/or/router.c index 8fdad9a5fa..bed9dc5e43 100644 --- a/src/or/router.c +++ b/src/or/router.c @@ -269,8 +269,8 @@ client_identity_key_is_set(void) /** Return the key certificate for this v3 (voting) authority, or NULL * if we have no such certificate. */ -authority_cert_t * -get_my_v3_authority_cert(void) +MOCK_IMPL(authority_cert_t *, +get_my_v3_authority_cert, (void)) { return authority_key_certificate; } @@ -1714,8 +1714,8 @@ router_compare_to_my_exit_policy(const tor_addr_t *addr, uint16_t port) /** Return true iff my exit policy is reject *:*. Return -1 if we don't * have a descriptor */ -int -router_my_exit_policy_is_reject_star(void) +MOCK_IMPL(int, +router_my_exit_policy_is_reject_star,(void)) { if (!router_get_my_routerinfo()) /* make sure desc_routerinfo exists */ return -1; @@ -1781,9 +1781,9 @@ router_get_my_descriptor(void) const char *body; if (!router_get_my_routerinfo()) return NULL; - /* Make sure this is nul-terminated. */ tor_assert(desc_routerinfo->cache_info.saved_location == SAVED_NOWHERE); body = signed_descriptor_get_body(&desc_routerinfo->cache_info); + /* Make sure this is nul-terminated. */ tor_assert(!body[desc_routerinfo->cache_info.signed_descriptor_len]); log_debug(LD_GENERAL,"my desc is '%s'", body); return body; @@ -1922,7 +1922,7 @@ router_build_fresh_descriptor(routerinfo_t **r, extrainfo_t **e) /* DNS is screwed up; don't claim to be an exit. */ policies_exit_policy_append_reject_star(&ri->exit_policy); } else { - policies_parse_exit_policy_from_options(options,ri->addr,&ri->ipv6_addr,1, + policies_parse_exit_policy_from_options(options,ri->addr,&ri->ipv6_addr, &ri->exit_policy); } ri->policy_is_reject_star = @@ -2728,44 +2728,13 @@ router_dump_exit_policy_to_string(const routerinfo_t *router, int include_ipv4, int include_ipv6) { - smartlist_t *exit_policy_strings; - char *policy_string = NULL; - if ((!router->exit_policy) || (router->policy_is_reject_star)) { return tor_strdup("reject *:*"); } - exit_policy_strings = smartlist_new(); - - SMARTLIST_FOREACH_BEGIN(router->exit_policy, addr_policy_t *, tmpe) { - char *pbuf; - int bytes_written_to_pbuf; - if ((tor_addr_family(&tmpe->addr) == AF_INET6) && (!include_ipv6)) { - continue; /* Don't include IPv6 parts of address policy */ - } - if ((tor_addr_family(&tmpe->addr) == AF_INET) && (!include_ipv4)) { - continue; /* Don't include IPv4 parts of address policy */ - } - - pbuf = tor_malloc(POLICY_BUF_LEN); - bytes_written_to_pbuf = policy_write_item(pbuf,POLICY_BUF_LEN, tmpe, 1); - - if (bytes_written_to_pbuf < 0) { - log_warn(LD_BUG, "router_dump_exit_policy_to_string ran out of room!"); - tor_free(pbuf); - goto done; - } - - smartlist_add(exit_policy_strings,pbuf); - } SMARTLIST_FOREACH_END(tmpe); - - policy_string = smartlist_join_strings(exit_policy_strings, "\n", 0, NULL); - - done: - SMARTLIST_FOREACH(exit_policy_strings, char *, str, tor_free(str)); - smartlist_free(exit_policy_strings); - - return policy_string; + return policy_dump_to_string(router->exit_policy, + include_ipv4, + include_ipv6); } /** Copy the primary (IPv4) OR port (IP address and TCP port) for diff --git a/src/or/router.h b/src/or/router.h index d8fcf0a9ad..85f43d804d 100644 --- a/src/or/router.h +++ b/src/or/router.h @@ -22,7 +22,7 @@ int server_identity_key_is_set(void); void set_client_identity_key(crypto_pk_t *k); crypto_pk_t *get_tlsclient_identity_key(void); int client_identity_key_is_set(void); -authority_cert_t *get_my_v3_authority_cert(void); +MOCK_DECL(authority_cert_t *, get_my_v3_authority_cert, (void)); crypto_pk_t *get_my_v3_authority_signing_key(void); authority_cert_t *get_my_v3_legacy_cert(void); crypto_pk_t *get_my_v3_legacy_signing_key(void); @@ -80,7 +80,7 @@ void check_descriptor_ipaddress_changed(time_t now); void router_new_address_suggestion(const char *suggestion, const dir_connection_t *d_conn); int router_compare_to_my_exit_policy(const tor_addr_t *addr, uint16_t port); -int router_my_exit_policy_is_reject_star(void); +MOCK_DECL(int, router_my_exit_policy_is_reject_star,(void)); MOCK_DECL(const routerinfo_t *, router_get_my_routerinfo, (void)); extrainfo_t *router_get_my_extrainfo(void); const char *router_get_my_descriptor(void); diff --git a/src/or/routerlist.c b/src/or/routerlist.c index 79a5bb3910..ef548d8df5 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -278,7 +278,7 @@ trusted_dirs_reload_certs(void) /** Helper: return true iff we already have loaded the exact cert * <b>cert</b>. */ -static INLINE int +static inline int already_have_cert(authority_cert_t *cert) { cert_list_t *cl = get_cert_list(cert->cache_info.identity_digest); @@ -985,7 +985,7 @@ router_should_rebuild_store(desc_store_t *store) /** Return the desc_store_t in <b>rl</b> that should be used to store * <b>sd</b>. */ -static INLINE desc_store_t * +static inline desc_store_t * desc_get_store(routerlist_t *rl, const signed_descriptor_t *sd) { if (sd->is_extrainfo) @@ -1363,10 +1363,10 @@ router_get_trusteddirserver_by_digest(const char *digest) dir_server_t * router_get_fallback_dirserver_by_digest(const char *digest) { - if (!trusted_dir_servers) + if (!fallback_dir_servers) return NULL; - SMARTLIST_FOREACH(trusted_dir_servers, dir_server_t *, ds, + SMARTLIST_FOREACH(fallback_dir_servers, dir_server_t *, ds, { if (tor_memeq(ds->digest, digest, DIGEST_LEN)) return ds; @@ -1897,7 +1897,7 @@ scale_array_elements_to_u64(u64_dbl_t *entries, int n_entries, #if SIZEOF_VOID_P == 8 #define gt_i64_timei(a,b) ((a) > (b)) #else -static INLINE int +static inline int gt_i64_timei(uint64_t a, uint64_t b) { int64_t diff = (int64_t) (b - a); @@ -1975,7 +1975,7 @@ bridge_get_advertised_bandwidth_bounded(routerinfo_t *router) /** Return bw*1000, unless bw*1000 would overflow, in which case return * INT32_MAX. */ -static INLINE int32_t +static inline int32_t kb_to_bytes(uint32_t bw) { return (bw > (INT32_MAX/1000)) ? INT32_MAX : bw*1000; @@ -2790,7 +2790,7 @@ dump_routerlist_mem_usage(int severity) * in <b>sl</b> at position <b>idx</b>. Otherwise, search <b>sl</b> for * <b>ri</b>. Return the index of <b>ri</b> in <b>sl</b>, or -1 if <b>ri</b> * is not in <b>sl</b>. */ -static INLINE int +static inline int routerlist_find_elt_(smartlist_t *sl, void *ri, int idx) { if (idx < 0) { @@ -4034,9 +4034,9 @@ router_exit_policy_rejects_all(const routerinfo_t *router) } /** Create an directory server at <b>address</b>:<b>port</b>, with OR identity - * key <b>digest</b>. If <b>address</b> is NULL, add ourself. If - * <b>is_authority</b>, this is a directory authority. Return the new - * directory server entry on success or NULL on failure. */ + * key <b>digest</b> which has DIGEST_LEN bytes. If <b>address</b> is NULL, + * add ourself. If <b>is_authority</b>, this is a directory authority. Return + * the new directory server entry on success or NULL on failure. */ static dir_server_t * dir_server_new(int is_authority, const char *nickname, @@ -4051,6 +4051,8 @@ dir_server_new(int is_authority, uint32_t a; char *hostname_ = NULL; + tor_assert(digest); + if (weight < 0) return NULL; @@ -5196,8 +5198,8 @@ hid_serv_acting_as_directory(void) /** Return true if this node is responsible for storing the descriptor ID * in <b>query</b> and false otherwise. */ -int -hid_serv_responsible_for_desc_id(const char *query) +MOCK_IMPL(int, hid_serv_responsible_for_desc_id, + (const char *query)) { const routerinfo_t *me; routerstatus_t *last_rs; diff --git a/src/or/routerlist.h b/src/or/routerlist.h index 200533fe91..b9bab2604d 100644 --- a/src/or/routerlist.h +++ b/src/or/routerlist.h @@ -109,7 +109,7 @@ static int WRA_NEVER_DOWNLOADABLE(was_router_added_t s); * was added. It might still be necessary to check whether the descriptor * generator should be notified. */ -static INLINE int +static inline int WRA_WAS_ADDED(was_router_added_t s) { return s == ROUTER_ADDED_SUCCESSFULLY || s == ROUTER_ADDED_NOTIFY_GENERATOR; } @@ -120,7 +120,7 @@ WRA_WAS_ADDED(was_router_added_t s) { * - it was outdated. * - its certificates were expired. */ -static INLINE int WRA_WAS_OUTDATED(was_router_added_t s) +static inline int WRA_WAS_OUTDATED(was_router_added_t s) { return (s == ROUTER_WAS_TOO_OLD || s == ROUTER_IS_ALREADY_KNOWN || @@ -130,13 +130,13 @@ static INLINE int WRA_WAS_OUTDATED(was_router_added_t s) } /** Return true iff the outcome code in <b>s</b> indicates that the descriptor * was flat-out rejected. */ -static INLINE int WRA_WAS_REJECTED(was_router_added_t s) +static inline int WRA_WAS_REJECTED(was_router_added_t s) { return (s == ROUTER_AUTHDIR_REJECTS); } /** Return true iff the outcome code in <b>s</b> indicates that the descriptor * was flat-out rejected. */ -static INLINE int WRA_NEVER_DOWNLOADABLE(was_router_added_t s) +static inline int WRA_NEVER_DOWNLOADABLE(was_router_added_t s) { return (s == ROUTER_AUTHDIR_REJECTS || s == ROUTER_BAD_EI || @@ -201,7 +201,7 @@ void refresh_all_country_info(void); int hid_serv_get_responsible_directories(smartlist_t *responsible_dirs, const char *id); int hid_serv_acting_as_directory(void); -int hid_serv_responsible_for_desc_id(const char *id); +MOCK_DECL(int, hid_serv_responsible_for_desc_id, (const char *id)); void list_pending_microdesc_downloads(digest256map_t *result); void launch_descriptor_downloads(int purpose, diff --git a/src/or/routerparse.c b/src/or/routerparse.c index f898ef8aef..f6619cb902 100644 --- a/src/or/routerparse.c +++ b/src/or/routerparse.c @@ -2061,7 +2061,7 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) * object (starting with "r " at the start of a line). If none is found, * return the start of the directory footer, or the next directory signature. * If none is found, return the end of the string. */ -static INLINE const char * +static inline const char * find_start_of_next_routerstatus(const char *s) { const char *eos, *footer, *sig; @@ -3930,7 +3930,7 @@ token_clear(directory_token_t *tok) * Return <b>tok</b> on success, or a new ERR_ token if the token didn't * conform to the syntax we wanted. **/ -static INLINE directory_token_t * +static inline directory_token_t * token_check_object(memarea_t *area, const char *kwd, directory_token_t *tok, obj_syntax o_syn) { @@ -3995,7 +3995,7 @@ token_check_object(memarea_t *area, const char *kwd, * number of parsed elements into the n_args field of <b>tok</b>. Allocate * all storage in <b>area</b>. Return the number of arguments parsed, or * return -1 if there was an insanely high number of arguments. */ -static INLINE int +static inline int get_token_arguments(memarea_t *area, directory_token_t *tok, const char *s, const char *eol) { diff --git a/src/or/routerset.c b/src/or/routerset.c index 3be55d3404..debe9ec6e1 100644 --- a/src/or/routerset.c +++ b/src/or/routerset.c @@ -107,10 +107,12 @@ routerset_parse(routerset_t *target, const char *s, const char *description) description); smartlist_add(target->country_names, countryname); added_countries = 1; - } else if ((strchr(nick,'.') || strchr(nick, '*')) && - (p = router_parse_addr_policy_item_from_string( + } else if ((strchr(nick,'.') || strchr(nick, ':') || strchr(nick, '*')) + && (p = router_parse_addr_policy_item_from_string( nick, ADDR_POLICY_REJECT, &malformed_list))) { + /* IPv4 addresses contain '.', IPv6 addresses contain ':', + * and wildcard addresses contain '*'. */ log_debug(LD_CONFIG, "Adding address %s to %s", nick, description); smartlist_add(target->policies, p); } else if (malformed_list) { diff --git a/src/or/statefile.c b/src/or/statefile.c index dd1894beb7..7fe8dc5077 100644 --- a/src/or/statefile.c +++ b/src/or/statefile.c @@ -9,6 +9,7 @@ #include "circuitstats.h" #include "config.h" #include "confparse.h" +#include "connection.h" #include "entrynodes.h" #include "hibernate.h" #include "rephist.h" @@ -372,6 +373,12 @@ or_state_load(void) new_state = or_state_new(); } else if (contents) { log_info(LD_GENERAL, "Loaded state from \"%s\"", fname); + /* Warn the user if their clock has been set backwards, + * they could be tricked into using old consensuses */ + time_t apparent_skew = new_state->LastWritten - time(NULL); + if (apparent_skew > 0) + clock_skew_warning(NULL, (long)apparent_skew, 1, LD_GENERAL, + "local state file", fname); } else { log_info(LD_GENERAL, "Initialized state"); } diff --git a/src/or/transports.c b/src/or/transports.c index ba2c784c2c..81b8db2508 100644 --- a/src/or/transports.c +++ b/src/or/transports.c @@ -105,7 +105,7 @@ static process_environment_t * create_managed_proxy_environment(const managed_proxy_t *mp); -static INLINE int proxy_configuration_finished(const managed_proxy_t *mp); +static inline int proxy_configuration_finished(const managed_proxy_t *mp); static void handle_finished_proxy(managed_proxy_t *mp); static void parse_method_error(const char *line, int is_server_method); @@ -713,7 +713,7 @@ register_client_proxy(const managed_proxy_t *mp) } /** Register the transports of managed proxy <b>mp</b>. */ -static INLINE void +static inline void register_proxy(const managed_proxy_t *mp) { if (mp->is_server) @@ -828,7 +828,7 @@ handle_finished_proxy(managed_proxy_t *mp) /** Return true if the configuration of the managed proxy <b>mp</b> is finished. */ -static INLINE int +static inline int proxy_configuration_finished(const managed_proxy_t *mp) { return (mp->conf_state == PT_PROTO_CONFIGURED || diff --git a/src/test/bench.c b/src/test/bench.c index 2a27377c80..70ec025b7b 100644 --- a/src/test/bench.c +++ b/src/test/bench.c @@ -643,7 +643,10 @@ main(int argc, const char **argv) reset_perftime(); - crypto_seed_rng(); + if (crypto_seed_rng() < 0) { + printf("Couldn't seed RNG; exiting.\n"); + return 1; + } crypto_init_siphash_key(); options = options_new(); init_logging(1); diff --git a/src/test/include.am b/src/test/include.am index 7a02a4e2f2..051f0e77d6 100644 --- a/src/test/include.am +++ b/src/test/include.am @@ -53,6 +53,8 @@ src_test_AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ # matters a lot there, and is quite hard to debug if you forget to do it. src_test_test_SOURCES = \ + src/test/log_test_helpers.c \ + src/test/rend_test_helpers.c \ src/test/test.c \ src/test/test_accounting.c \ src/test/test_addr.c \ @@ -65,6 +67,7 @@ src_test_test_SOURCES = \ src/test/test_checkdir.c \ src/test/test_circuitlist.c \ src/test/test_circuitmux.c \ + src/test/test_compat_libevent.c \ src/test/test_config.c \ src/test/test_containers.c \ src/test/test_controller.c \ @@ -72,6 +75,7 @@ src_test_test_SOURCES = \ src/test/test_crypto.c \ src/test/test_data.c \ src/test/test_dir.c \ + src/test/test_dir_handle_get.c \ src/test/test_entryconn.c \ src/test/test_entrynodes.c \ src/test/test_guardfraction.c \ @@ -86,9 +90,11 @@ src_test_test_SOURCES = \ src/test/test_oom.c \ src/test/test_options.c \ src/test/test_policy.c \ + src/test/test_procmon.c \ src/test/test_pt.c \ src/test/test_relay.c \ src/test/test_relaycell.c \ + src/test/test_rendcache.c \ src/test/test_replay.c \ src/test/test_routerkeys.c \ src/test/test_routerlist.c \ @@ -97,9 +103,12 @@ src_test_test_SOURCES = \ src/test/test_socks.c \ src/test/test_status.c \ src/test/test_threads.c \ + src/test/test_tortls.c \ src/test/test_util.c \ + src/test/test_util_format.c \ + src/test/test_util_process.c \ src/test/test_helpers.c \ - src/test/test_dns.c \ + src/test/test_dns.c \ src/test/testing_common.c \ src/ext/tinytest.c @@ -165,13 +174,16 @@ src_test_test_workqueue_LDADD = src/or/libtor-testing.a \ noinst_HEADERS+= \ src/test/fakechans.h \ + src/test/log_test_helpers.h \ + src/test/rend_test_helpers.h \ src/test/test.h \ src/test/test_helpers.h \ src/test/test_descriptors.inc \ src/test/example_extrainfo.inc \ src/test/failing_routerdescs.inc \ src/test/ed25519_vectors.inc \ - src/test/test_descriptors.inc + src/test/test_descriptors.inc \ + src/test/vote_descriptors.inc noinst_PROGRAMS+= src/test/test-ntor-cl src_test_test_ntor_cl_SOURCES = src/test/test_ntor_cl.c diff --git a/src/test/log_test_helpers.c b/src/test/log_test_helpers.c new file mode 100644 index 0000000000..51b5f9b7b1 --- /dev/null +++ b/src/test/log_test_helpers.c @@ -0,0 +1,111 @@ +/* Copyright (c) 2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ +#define LOG_PRIVATE +#include "torlog.h" +#include "log_test_helpers.h" + +static smartlist_t *saved_logs = NULL; + +int +setup_capture_of_logs(int new_level) +{ + int previous_log = log_global_min_severity_; + log_global_min_severity_ = new_level; + mock_clean_saved_logs(); + MOCK(logv, mock_saving_logv); + return previous_log; +} + +void +teardown_capture_of_logs(int prev) +{ + UNMOCK(logv); + log_global_min_severity_ = prev; + mock_clean_saved_logs(); +} + +void +mock_clean_saved_logs(void) +{ + if (!saved_logs) + return; + SMARTLIST_FOREACH(saved_logs, mock_saved_log_entry_t *, m, + { tor_free(m->generated_msg); tor_free(m); }); + smartlist_free(saved_logs); + saved_logs = NULL; +} + +static mock_saved_log_entry_t * +mock_get_log_entry(int ix) +{ + int saved_log_count = mock_saved_log_number(); + if (ix < 0) { + ix = saved_log_count + ix; + } + + if (saved_log_count <= ix) + return NULL; + + return smartlist_get(saved_logs, ix); +} + +const char * +mock_saved_log_at(int ix) +{ + mock_saved_log_entry_t *ent = mock_get_log_entry(ix); + if (ent) + return ent->generated_msg; + else + return ""; +} + +int +mock_saved_severity_at(int ix) +{ + mock_saved_log_entry_t *ent = mock_get_log_entry(ix); + if (ent) + return ent->severity; + else + return -1; +} + +int +mock_saved_log_number(void) +{ + if (!saved_logs) + return 0; + return smartlist_len(saved_logs); +} + +const smartlist_t * +mock_saved_logs(void) +{ + return saved_logs; +} + +void +mock_saving_logv(int severity, log_domain_mask_t domain, + const char *funcname, const char *suffix, + const char *format, va_list ap) +{ + (void)domain; + char *buf = tor_malloc_zero(10240); + int n; + n = tor_vsnprintf(buf,10240,format,ap); + tor_assert(n < 10240-1); + buf[n]='\n'; + buf[n+1]='\0'; + + mock_saved_log_entry_t *e = tor_malloc_zero(sizeof(mock_saved_log_entry_t)); + e->severity = severity; + e->funcname = funcname; + e->suffix = suffix; + e->format = format; + e->generated_msg = tor_strdup(buf); + tor_free(buf); + + if (!saved_logs) + saved_logs = smartlist_new(); + smartlist_add(saved_logs, e); +} + diff --git a/src/test/log_test_helpers.h b/src/test/log_test_helpers.h new file mode 100644 index 0000000000..af8e8a60e7 --- /dev/null +++ b/src/test/log_test_helpers.h @@ -0,0 +1,31 @@ +/* Copyright (c) 2014-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" + +#ifndef TOR_LOG_TEST_HELPERS_H +#define TOR_LOG_TEST_HELPERS_H + +typedef struct mock_saved_log_entry_t { + int severity; + const char *funcname; + const char *suffix; + const char *format; + char *generated_msg; + struct mock_saved_log_entry_t *next; +} mock_saved_log_entry_t; + +void mock_saving_logv(int severity, log_domain_mask_t domain, + const char *funcname, const char *suffix, + const char *format, va_list ap) + CHECK_PRINTF(5, 0); +void mock_clean_saved_logs(void); +const smartlist_t *mock_saved_logs(void); +int setup_capture_of_logs(int new_level); +void teardown_capture_of_logs(int prev); +const char *mock_saved_log_at(int ix); +int mock_saved_severity_at(int ix); +int mock_saved_log_number(void); + +#endif + diff --git a/src/test/rend_test_helpers.c b/src/test/rend_test_helpers.c new file mode 100644 index 0000000000..f16d67fa1a --- /dev/null +++ b/src/test/rend_test_helpers.c @@ -0,0 +1,73 @@ +/* Copyright (c) 2014-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "test.h" +#include "rendcommon.h" +#include "rend_test_helpers.h" + +void +generate_desc(int time_diff, rend_encoded_v2_service_descriptor_t **desc, + char **service_id, int intro_points) +{ + rend_service_descriptor_t *generated = NULL; + smartlist_t *descs = smartlist_new(); + time_t now; + + now = time(NULL) + time_diff; + create_descriptor(&generated, service_id, intro_points); + generated->timestamp = now; + + rend_encode_v2_descriptors(descs, generated, now, 0, REND_NO_AUTH, NULL, + NULL); + tor_assert(smartlist_len(descs) > 1); + *desc = smartlist_get(descs, 0); + smartlist_set(descs, 0, NULL); + + SMARTLIST_FOREACH(descs, rend_encoded_v2_service_descriptor_t *, d, + rend_encoded_v2_service_descriptor_free(d)); + smartlist_free(descs); + rend_service_descriptor_free(generated); +} + +void +create_descriptor(rend_service_descriptor_t **generated, char **service_id, + int intro_points) +{ + crypto_pk_t *pk1 = NULL; + crypto_pk_t *pk2 = NULL; + int i; + + *service_id = tor_malloc(REND_SERVICE_ID_LEN_BASE32+1); + pk1 = pk_generate(0); + pk2 = pk_generate(1); + + *generated = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + (*generated)->pk = crypto_pk_dup_key(pk1); + rend_get_service_id((*generated)->pk, *service_id); + + (*generated)->version = 2; + (*generated)->protocols = 42; + (*generated)->intro_nodes = smartlist_new(); + + for (i = 0; i < intro_points; i++) { + rend_intro_point_t *intro = tor_malloc_zero(sizeof(rend_intro_point_t)); + crypto_pk_t *okey = pk_generate(2 + i); + intro->extend_info = tor_malloc_zero(sizeof(extend_info_t)); + intro->extend_info->onion_key = okey; + crypto_pk_get_digest(intro->extend_info->onion_key, + intro->extend_info->identity_digest); + intro->extend_info->nickname[0] = '$'; + base16_encode(intro->extend_info->nickname + 1, + sizeof(intro->extend_info->nickname) - 1, + intro->extend_info->identity_digest, DIGEST_LEN); + tor_addr_from_ipv4h(&intro->extend_info->addr, crypto_rand_int(65536)); + intro->extend_info->port = 1 + crypto_rand_int(65535); + intro->intro_key = crypto_pk_dup_key(pk2); + smartlist_add((*generated)->intro_nodes, intro); + } + + crypto_pk_free(pk1); + crypto_pk_free(pk2); +} + diff --git a/src/test/rend_test_helpers.h b/src/test/rend_test_helpers.h new file mode 100644 index 0000000000..1ef03747d7 --- /dev/null +++ b/src/test/rend_test_helpers.h @@ -0,0 +1,15 @@ +/* Copyright (c) 2014-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" + +#ifndef TOR_REND_TEST_HELPERS_H +#define TOR_REND_TEST_HELPERS_H + +void generate_desc(int time_diff, rend_encoded_v2_service_descriptor_t **desc, + char **service_id, int intro_points); +void create_descriptor(rend_service_descriptor_t **generated, + char **service_id, int intro_points); + +#endif + diff --git a/src/test/test-memwipe.c b/src/test/test-memwipe.c index a39bad1540..5d4fcec664 100644 --- a/src/test/test-memwipe.c +++ b/src/test/test-memwipe.c @@ -62,7 +62,7 @@ fill_a_buffer_nothing(void) return sum; } -static INLINE int +static inline int vmemeq(volatile char *a, const char *b, size_t n) { while (n--) { diff --git a/src/test/test.c b/src/test/test.c index e10e260266..1c4c2921db 100644 --- a/src/test/test.c +++ b/src/test/test.c @@ -28,6 +28,7 @@ #define ROUTER_PRIVATE #define CIRCUITSTATS_PRIVATE #define CIRCUITLIST_PRIVATE +#define MAIN_PRIVATE #define STATEFILE_PRIVATE /* @@ -47,8 +48,10 @@ double fabs(double x); #include "connection_edge.h" #include "geoip.h" #include "rendcommon.h" +#include "rendcache.h" #include "test.h" #include "torgzip.h" +#include "main.h" #include "memarea.h" #include "onion.h" #include "onion_ntor.h" @@ -316,6 +319,13 @@ test_circuit_timeout(void *arg) int i, runs; double close_ms; (void)arg; + tor_libevent_cfg cfg; + + memset(&cfg, 0, sizeof(cfg)); + + tor_libevent_initialize(&cfg); + initialize_periodic_events(); + circuit_build_times_init(&initial); circuit_build_times_init(&estimate); circuit_build_times_init(&final); @@ -455,6 +465,7 @@ test_circuit_timeout(void *arg) circuit_build_times_free_timeouts(&estimate); circuit_build_times_free_timeouts(&final); or_state_free(state); + teardown_periodic_events(); } /** Test encoding and parsing of rendezvous service descriptors. */ @@ -494,6 +505,9 @@ test_rend_fns(void *arg) tt_str_op(address6,OP_EQ, "abcdefghijklmnop"); tt_assert(BAD_HOSTNAME == parse_extended_hostname(address7)); + /* Initialize the service cache. */ + rend_cache_init(); + pk1 = pk_generate(0); pk2 = pk_generate(1); generated = tor_malloc_zero(sizeof(rend_service_descriptor_t)); @@ -1105,8 +1119,8 @@ static struct testcase_t test_array[] = { { "bad_onion_handshake", test_bad_onion_handshake, 0, NULL, NULL }, ENT(onion_queues), { "ntor_handshake", test_ntor_handshake, 0, NULL, NULL }, - ENT(circuit_timeout), - ENT(rend_fns), + FORK(circuit_timeout), + FORK(rend_fns), ENT(geoip), FORK(geoip_with_pt), FORK(stats), @@ -1125,12 +1139,14 @@ extern struct testcase_t channeltls_tests[]; extern struct testcase_t checkdir_tests[]; extern struct testcase_t circuitlist_tests[]; extern struct testcase_t circuitmux_tests[]; +extern struct testcase_t compat_libevent_tests[]; extern struct testcase_t config_tests[]; extern struct testcase_t container_tests[]; extern struct testcase_t controller_tests[]; extern struct testcase_t controller_event_tests[]; extern struct testcase_t crypto_tests[]; extern struct testcase_t dir_tests[]; +extern struct testcase_t dir_handle_get_tests[]; extern struct testcase_t entryconn_tests[]; extern struct testcase_t entrynodes_tests[]; extern struct testcase_t guardfraction_tests[]; @@ -1145,9 +1161,11 @@ extern struct testcase_t nodelist_tests[]; extern struct testcase_t oom_tests[]; extern struct testcase_t options_tests[]; extern struct testcase_t policy_tests[]; +extern struct testcase_t procmon_tests[]; extern struct testcase_t pt_tests[]; extern struct testcase_t relay_tests[]; extern struct testcase_t relaycell_tests[]; +extern struct testcase_t rend_cache_tests[]; extern struct testcase_t replaycache_tests[]; extern struct testcase_t router_tests[]; extern struct testcase_t routerkeys_tests[]; @@ -1157,7 +1175,10 @@ extern struct testcase_t scheduler_tests[]; extern struct testcase_t socks_tests[]; extern struct testcase_t status_tests[]; extern struct testcase_t thread_tests[]; +extern struct testcase_t tortls_tests[]; extern struct testcase_t util_tests[]; +extern struct testcase_t util_format_tests[]; +extern struct testcase_t util_process_tests[]; extern struct testcase_t dns_tests[]; struct testgroup_t testgroups[] = { @@ -1173,12 +1194,14 @@ struct testgroup_t testgroups[] = { { "checkdir/", checkdir_tests }, { "circuitlist/", circuitlist_tests }, { "circuitmux/", circuitmux_tests }, + { "compat/libevent/", compat_libevent_tests }, { "config/", config_tests }, { "container/", container_tests }, { "control/", controller_tests }, { "control/event/", controller_event_tests }, { "crypto/", crypto_tests }, { "dir/", dir_tests }, + { "dir_handle_get/", dir_handle_get_tests }, { "dir/md/", microdesc_tests }, { "entryconn/", entryconn_tests }, { "entrynodes/", entrynodes_tests }, @@ -1192,9 +1215,11 @@ struct testgroup_t testgroups[] = { { "oom/", oom_tests }, { "options/", options_tests }, { "policy/" , policy_tests }, + { "procmon/", procmon_tests }, { "pt/", pt_tests }, { "relay/" , relay_tests }, { "relaycell/", relaycell_tests }, + { "rend_cache/", rend_cache_tests }, { "replaycache/", replaycache_tests }, { "routerkeys/", routerkeys_tests }, { "routerlist/", routerlist_tests }, @@ -1202,8 +1227,11 @@ struct testgroup_t testgroups[] = { { "scheduler/", scheduler_tests }, { "socks/", socks_tests }, { "status/" , status_tests }, + { "tortls/", tortls_tests }, { "util/", util_tests }, + { "util/format/", util_format_tests }, { "util/logging/", logging_tests }, + { "util/process/", util_process_tests }, { "util/thread/", thread_tests }, { "dns/", dns_tests }, END_OF_GROUPS diff --git a/src/test/test_address.c b/src/test/test_address.c index 3e73c3e27b..9f3d81c92c 100644 --- a/src/test/test_address.c +++ b/src/test/test_address.c @@ -119,6 +119,21 @@ smartlist_contains_internal_tor_addr(smartlist_t *smartlist) } /** Return 1 iff <b>smartlist</b> contains a tor_addr_t structure + * that is NULL or the null tor_addr_t. Otherwise, return 0. + */ +static int +smartlist_contains_null_tor_addr(smartlist_t *smartlist) +{ + SMARTLIST_FOREACH_BEGIN(smartlist, tor_addr_t *, tor_addr) { + if (tor_addr == NULL || tor_addr_is_null(tor_addr)) { + return 1; + } + } SMARTLIST_FOREACH_END(tor_addr); + + return 0; +} + +/** Return 1 iff <b>smartlist</b> contains a tor_addr_t structure * that is an IPv4 address. Otherwise, return 0. */ static int @@ -268,8 +283,18 @@ test_address_get_if_addrs_ifaddrs(void *arg) results = get_interface_addresses_ifaddrs(LOG_ERR); - tt_int_op(smartlist_len(results),>=,1); - tt_assert(smartlist_contains_localhost_tor_addr(results)); + tt_assert(results); + /* Some FreeBSD jails don't have localhost IP address. Instead, they only + * have the address assigned to the jail (whatever that may be). + * And a jail without a network connection might not have any addresses at + * all. */ + tt_assert(!smartlist_contains_null_tor_addr(results)); + + /* If there are addresses, they must be IPv4 or IPv6 */ + if (smartlist_len(results) > 0) { + tt_assert(smartlist_contains_ipv4_tor_addr(results) + || smartlist_contains_ipv6_tor_addr(results)); + } done: SMARTLIST_FOREACH(results, tor_addr_t *, t, tor_free(t)); @@ -293,6 +318,13 @@ test_address_get_if_addrs_win32(void *arg) tt_int_op(smartlist_len(results),>=,1); tt_assert(smartlist_contains_localhost_tor_addr(results)); + tt_assert(!smartlist_contains_null_tor_addr(results)); + + /* If there are addresses, they must be IPv4 or IPv6 */ + if (smartlist_len(results) > 0) { + tt_assert(smartlist_contains_ipv4_tor_addr(results) + || smartlist_contains_ipv6_tor_addr(results)); + } done: SMARTLIST_FOREACH(results, tor_addr_t *, t, tor_free(t)); @@ -481,10 +513,24 @@ test_address_get_if_addrs_ioctl(void *arg) result = get_interface_addresses_ioctl(LOG_ERR); + /* On an IPv6-only system, this will fail and return NULL tt_assert(result); - tt_int_op(smartlist_len(result),>=,1); + */ - tt_assert(smartlist_contains_localhost_tor_addr(result)); + /* Some FreeBSD jails don't have localhost IP address. Instead, they only + * have the address assigned to the jail (whatever that may be). + * And a jail without a network connection might not have any addresses at + * all. */ + if (result) { + tt_assert(!smartlist_contains_null_tor_addr(result)); + + /* If there are addresses, they must be IPv4 or IPv6. + * (AIX supports IPv6 from SIOCGIFCONF.) */ + if (smartlist_len(result) > 0) { + tt_assert(smartlist_contains_ipv4_tor_addr(result) + || smartlist_contains_ipv6_tor_addr(result)); + } + } done: if (result) { @@ -696,12 +742,13 @@ test_address_get_if_addrs_list_internal(void *arg) tt_assert(!smartlist_contains_localhost_tor_addr(results)); tt_assert(!smartlist_contains_multicast_tor_addr(results)); /* The list may or may not contain internal addresses */ + tt_assert(!smartlist_contains_null_tor_addr(results)); - /* Allow unit tests to pass on IPv6-only machines */ + /* if there are any addresses, they must be IPv4 */ if (smartlist_len(results) > 0) { - tt_assert(smartlist_contains_ipv4_tor_addr(results) - || smartlist_contains_ipv6_tor_addr(results)); + tt_assert(smartlist_contains_ipv4_tor_addr(results)); } + tt_assert(!smartlist_contains_ipv6_tor_addr(results)); done: free_interface_address_list(results); @@ -724,6 +771,7 @@ test_address_get_if_addrs_list_no_internal(void *arg) tt_assert(!smartlist_contains_localhost_tor_addr(results)); tt_assert(!smartlist_contains_multicast_tor_addr(results)); tt_assert(!smartlist_contains_internal_tor_addr(results)); + tt_assert(!smartlist_contains_null_tor_addr(results)); /* if there are any addresses, they must be IPv4 */ if (smartlist_len(results) > 0) { @@ -752,6 +800,7 @@ test_address_get_if_addrs6_list_internal(void *arg) tt_assert(!smartlist_contains_localhost_tor_addr(results)); tt_assert(!smartlist_contains_multicast_tor_addr(results)); /* The list may or may not contain internal addresses */ + tt_assert(!smartlist_contains_null_tor_addr(results)); /* if there are any addresses, they must be IPv6 */ tt_assert(!smartlist_contains_ipv4_tor_addr(results)); @@ -780,7 +829,9 @@ test_address_get_if_addrs6_list_no_internal(void *arg) tt_assert(!smartlist_contains_localhost_tor_addr(results)); tt_assert(!smartlist_contains_multicast_tor_addr(results)); tt_assert(!smartlist_contains_internal_tor_addr(results)); + tt_assert(!smartlist_contains_null_tor_addr(results)); + /* if there are any addresses, they must be IPv6 */ tt_assert(!smartlist_contains_ipv4_tor_addr(results)); if (smartlist_len(results) > 0) { tt_assert(smartlist_contains_ipv6_tor_addr(results)); @@ -848,7 +899,7 @@ test_address_get_if_addrs_internal_fail(void *arg) rv = get_interface_address(LOG_ERR, &ipv4h_addr); tt_assert(rv == -1); -done: + done: UNMOCK(get_interface_addresses_raw); UNMOCK(get_interface_address6_via_udp_socket_hack); free_interface_address6_list(results1); @@ -876,7 +927,7 @@ test_address_get_if_addrs_no_internal_fail(void *arg) tt_assert(results2 != NULL); tt_int_op(smartlist_len(results2),==,0); -done: + done: UNMOCK(get_interface_addresses_raw); UNMOCK(get_interface_address6_via_udp_socket_hack); free_interface_address6_list(results1); @@ -935,6 +986,118 @@ test_address_get_if_addrs6(void *arg) return; } +static void +test_address_tor_addr_to_in6(void *ignored) +{ + (void)ignored; + tor_addr_t *a = tor_malloc_zero(sizeof(tor_addr_t)); + const struct in6_addr *res; + uint8_t expected[16] = {42, 1, 2, 3, 4, 5, 6, 7, 8, 9, + 10, 11, 12, 13, 14, 15}; + + a->family = AF_INET; + res = tor_addr_to_in6(a); + tt_assert(!res); + + a->family = AF_INET6; + memcpy(a->addr.in6_addr.s6_addr, expected, 16); + res = tor_addr_to_in6(a); + tt_assert(res); + tt_mem_op(res->s6_addr, OP_EQ, expected, 16); + + done: + tor_free(a); +} + +static void +test_address_tor_addr_to_in(void *ignored) +{ + (void)ignored; + tor_addr_t *a = tor_malloc_zero(sizeof(tor_addr_t)); + const struct in_addr *res; + + a->family = AF_INET6; + res = tor_addr_to_in(a); + tt_assert(!res); + + a->family = AF_INET; + a->addr.in_addr.s_addr = 44; + res = tor_addr_to_in(a); + tt_assert(res); + tt_int_op(res->s_addr, OP_EQ, 44); + + done: + tor_free(a); +} + +static void +test_address_tor_addr_to_ipv4n(void *ignored) +{ + (void)ignored; + tor_addr_t *a = tor_malloc_zero(sizeof(tor_addr_t)); + uint32_t res; + + a->family = AF_INET6; + res = tor_addr_to_ipv4n(a); + tt_assert(!res); + + a->family = AF_INET; + a->addr.in_addr.s_addr = 43; + res = tor_addr_to_ipv4n(a); + tt_assert(res); + tt_int_op(res, OP_EQ, 43); + + done: + tor_free(a); +} + +static void +test_address_tor_addr_to_mapped_ipv4h(void *ignored) +{ + (void)ignored; + tor_addr_t *a = tor_malloc_zero(sizeof(tor_addr_t)); + uint32_t res; + uint8_t toset[16] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 0, 0, 0, 42}; + + a->family = AF_INET; + res = tor_addr_to_mapped_ipv4h(a); + tt_assert(!res); + + a->family = AF_INET6; + + memcpy(a->addr.in6_addr.s6_addr, toset, 16); + res = tor_addr_to_mapped_ipv4h(a); + tt_assert(res); + tt_int_op(res, OP_EQ, 42); + + done: + tor_free(a); +} + +static void +test_address_tor_addr_eq_ipv4h(void *ignored) +{ + (void)ignored; + tor_addr_t *a = tor_malloc_zero(sizeof(tor_addr_t)); + int res; + + a->family = AF_INET6; + res = tor_addr_eq_ipv4h(a, 42); + tt_assert(!res); + + a->family = AF_INET; + a->addr.in_addr.s_addr = 52; + res = tor_addr_eq_ipv4h(a, 42); + tt_assert(!res); + + a->addr.in_addr.s_addr = 52; + res = tor_addr_eq_ipv4h(a, ntohl(52)); + tt_assert(res); + + done: + tor_free(a); +} + #define ADDRESS_TEST(name, flags) \ { #name, test_address_ ## name, flags, NULL, NULL } @@ -961,6 +1124,11 @@ struct testcase_t address_tests[] = { ADDRESS_TEST(get_if_addrs_ioctl, TT_FORK), ADDRESS_TEST(ifreq_to_smartlist, 0), #endif + ADDRESS_TEST(tor_addr_to_in6, 0), + ADDRESS_TEST(tor_addr_to_in, 0), + ADDRESS_TEST(tor_addr_to_ipv4n, 0), + ADDRESS_TEST(tor_addr_to_mapped_ipv4h, 0), + ADDRESS_TEST(tor_addr_eq_ipv4h, 0), END_OF_TESTCASES }; diff --git a/src/test/test_channeltls.c b/src/test/test_channeltls.c index 016e504ab3..dff1dde87e 100644 --- a/src/test/test_channeltls.c +++ b/src/test/test_channeltls.c @@ -123,7 +123,7 @@ test_channeltls_num_bytes_queued(void *arg) /* * Next, we have to test ch->num_bytes_queued, which is * channel_tls_num_bytes_queued_method. We can't mock - * connection_get_outbuf_len() directly because it's static INLINE + * connection_get_outbuf_len() directly because it's static inline * in connection.h, but we can mock buf_datalen(). Note that * if bufferevents ever work, this will break with them enabled. */ diff --git a/src/test/test_compat_libevent.c b/src/test/test_compat_libevent.c new file mode 100644 index 0000000000..96502df308 --- /dev/null +++ b/src/test/test_compat_libevent.c @@ -0,0 +1,238 @@ +/* Copyright (c) 2010-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#define COMPAT_LIBEVENT_PRIVATE +#include "orconfig.h" +#include "or.h" + +#include "test.h" + +#include "compat_libevent.h" + +#ifdef HAVE_EVENT2_EVENT_H +#include <event2/event.h> +#include <event2/thread.h> +#ifdef USE_BUFFEREVENTS +#include <event2/bufferevent.h> +#endif +#else +#include <event.h> +#endif + +#include "log_test_helpers.h" + +#define NS_MODULE compat_libevent + +static void +test_compat_libevent_logging_callback(void *ignored) +{ + (void)ignored; + int previous_log = setup_capture_of_logs(LOG_DEBUG); + + libevent_logging_callback(_EVENT_LOG_DEBUG, "hello world"); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Message from libevent: hello world\n"); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_DEBUG); + + mock_clean_saved_logs(); + libevent_logging_callback(_EVENT_LOG_MSG, "hello world another time"); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Message from libevent: hello world another time\n"); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_INFO); + + mock_clean_saved_logs(); + libevent_logging_callback(_EVENT_LOG_WARN, "hello world a third time"); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Warning from libevent: hello world a third time\n"); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_WARN); + + mock_clean_saved_logs(); + libevent_logging_callback(_EVENT_LOG_ERR, "hello world a fourth time"); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Error from libevent: hello world a fourth time\n"); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_ERR); + + mock_clean_saved_logs(); + libevent_logging_callback(42, "hello world a fifth time"); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Message [42] from libevent: hello world a fifth time\n"); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_WARN); + + mock_clean_saved_logs(); + libevent_logging_callback(_EVENT_LOG_DEBUG, + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + ); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "Message from libevent: " + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789" + "012345678901234567890123456789\n"); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_DEBUG); + + mock_clean_saved_logs(); + libevent_logging_callback(42, "xxx\n"); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "Message [42] from libevent: xxx\n"); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_WARN); + + suppress_libevent_log_msg("something"); + mock_clean_saved_logs(); + libevent_logging_callback(_EVENT_LOG_MSG, "hello there"); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Message from libevent: hello there\n"); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_INFO); + + mock_clean_saved_logs(); + libevent_logging_callback(_EVENT_LOG_MSG, "hello there something else"); + tt_int_op(mock_saved_log_number(), OP_EQ, 0); + + // No way of verifying the result of this, it seems =/ + configure_libevent_logging(); + + done: + suppress_libevent_log_msg(NULL); + teardown_capture_of_logs(previous_log); +} + +static void +test_compat_libevent_le_versions_compatibility(void *ignored) +{ + (void)ignored; + int res; + + res = le_versions_compatibility(LE_OTHER); + tt_int_op(res, OP_EQ, 0); + + res = le_versions_compatibility(V_OLD(0,9,'c')); + tt_int_op(res, OP_EQ, 1); + + res = le_versions_compatibility(V(1,3,98)); + tt_int_op(res, OP_EQ, 2); + + res = le_versions_compatibility(V(1,4,98)); + tt_int_op(res, OP_EQ, 3); + + res = le_versions_compatibility(V(1,5,0)); + tt_int_op(res, OP_EQ, 4); + + res = le_versions_compatibility(V(2,0,0)); + tt_int_op(res, OP_EQ, 4); + + res = le_versions_compatibility(V(2,0,2)); + tt_int_op(res, OP_EQ, 5); + + done: + (void)0; +} + +static void +test_compat_libevent_tor_decode_libevent_version(void *ignored) +{ + (void)ignored; + le_version_t res; + + res = tor_decode_libevent_version("SOMETHING WRONG"); + tt_int_op(res, OP_EQ, LE_OTHER); + + res = tor_decode_libevent_version("1.4.11"); + tt_int_op(res, OP_EQ, V(1,4,11)); + + res = tor_decode_libevent_version("1.4.12b-stable"); + tt_int_op(res, OP_EQ, V(1,4,12)); + + res = tor_decode_libevent_version("1.4.17b_stable"); + tt_int_op(res, OP_EQ, V(1,4,17)); + + res = tor_decode_libevent_version("1.4.12!stable"); + tt_int_op(res, OP_EQ, LE_OTHER); + + res = tor_decode_libevent_version("1.4.12b!stable"); + tt_int_op(res, OP_EQ, LE_OTHER); + + res = tor_decode_libevent_version("1.4.13-"); + tt_int_op(res, OP_EQ, V(1,4,13)); + + res = tor_decode_libevent_version("1.4.14_"); + tt_int_op(res, OP_EQ, V(1,4,14)); + + res = tor_decode_libevent_version("1.4.15c-"); + tt_int_op(res, OP_EQ, V(1,4,15)); + + res = tor_decode_libevent_version("1.4.16c_"); + tt_int_op(res, OP_EQ, V(1,4,16)); + + res = tor_decode_libevent_version("1.4.17-s"); + tt_int_op(res, OP_EQ, V(1,4,17)); + + res = tor_decode_libevent_version("1.5"); + tt_int_op(res, OP_EQ, V(1,5,0)); + + res = tor_decode_libevent_version("1.2"); + tt_int_op(res, OP_EQ, V(1,2,0)); + + res = tor_decode_libevent_version("1.2-"); + tt_int_op(res, OP_EQ, LE_OTHER); + + res = tor_decode_libevent_version("1.6e"); + tt_int_op(res, OP_EQ, V_OLD(1,6,'e')); + + done: + (void)0; +} + +#if defined(LIBEVENT_VERSION) +#define HEADER_VERSION LIBEVENT_VERSION +#elif defined(_EVENT_VERSION) +#define HEADER_VERSION _EVENT_VERSION +#endif + +static void +test_compat_libevent_header_version(void *ignored) +{ + (void)ignored; + const char *res; + + res = tor_libevent_get_header_version_str(); + tt_str_op(res, OP_EQ, HEADER_VERSION); + + done: + (void)0; +} + +struct testcase_t compat_libevent_tests[] = { + { "logging_callback", test_compat_libevent_logging_callback, + TT_FORK, NULL, NULL }, + { "le_versions_compatibility", + test_compat_libevent_le_versions_compatibility, 0, NULL, NULL }, + { "tor_decode_libevent_version", + test_compat_libevent_tor_decode_libevent_version, 0, NULL, NULL }, + { "header_version", test_compat_libevent_header_version, 0, NULL, NULL }, + END_OF_TESTCASES +}; + diff --git a/src/test/test_crypto.c b/src/test/test_crypto.c index dbaec61ee9..a3bef2c6cb 100644 --- a/src/test/test_crypto.c +++ b/src/test/test_crypto.c @@ -284,10 +284,11 @@ test_crypto_sha(void *arg) { crypto_digest_t *d1 = NULL, *d2 = NULL; int i; - char key[160]; - char digest[32]; - char data[50]; - char d_out1[DIGEST_LEN], d_out2[DIGEST256_LEN]; +#define RFC_4231_MAX_KEY_SIZE 131 + char key[RFC_4231_MAX_KEY_SIZE]; + char digest[DIGEST256_LEN]; + char data[DIGEST512_LEN]; + char d_out1[DIGEST512_LEN], d_out2[DIGEST512_LEN]; char *mem_op_hex_tmp=NULL; /* Test SHA-1 with a test vector from the specification. */ @@ -302,6 +303,13 @@ test_crypto_sha(void *arg) "96177A9CB410FF61F20015AD"); tt_int_op(i, OP_EQ, 0); + /* Test SHA-512 with a test vector from the specification. */ + i = crypto_digest512(data, "abc", 3, DIGEST_SHA512); + test_memeq_hex(data, "ddaf35a193617abacc417349ae20413112e6fa4e89a97" + "ea20a9eeee64b55d39a2192992a274fc1a836ba3c23a3" + "feebbd454d4423643ce80e2a9ac94fa54ca49f"); + tt_int_op(i, OP_EQ, 0); + /* Test HMAC-SHA256 with test cases from wikipedia and RFC 4231 */ /* Case empty (wikipedia) */ @@ -378,15 +386,15 @@ test_crypto_sha(void *arg) d2 = crypto_digest_dup(d1); tt_assert(d2); crypto_digest_add_bytes(d2, "ghijkl", 6); - crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); + crypto_digest_get_digest(d2, d_out1, DIGEST_LEN); crypto_digest(d_out2, "abcdefghijkl", 12); tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST_LEN); crypto_digest_assign(d2, d1); crypto_digest_add_bytes(d2, "mno", 3); - crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); + crypto_digest_get_digest(d2, d_out1, DIGEST_LEN); crypto_digest(d_out2, "abcdefmno", 9); tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST_LEN); - crypto_digest_get_digest(d1, d_out1, sizeof(d_out1)); + crypto_digest_get_digest(d1, d_out1, DIGEST_LEN); crypto_digest(d_out2, "abcdef", 6); tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST_LEN); crypto_digest_free(d1); @@ -399,17 +407,38 @@ test_crypto_sha(void *arg) d2 = crypto_digest_dup(d1); tt_assert(d2); crypto_digest_add_bytes(d2, "ghijkl", 6); - crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); + crypto_digest_get_digest(d2, d_out1, DIGEST256_LEN); crypto_digest256(d_out2, "abcdefghijkl", 12, DIGEST_SHA256); - tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST_LEN); + tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST256_LEN); crypto_digest_assign(d2, d1); crypto_digest_add_bytes(d2, "mno", 3); - crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); + crypto_digest_get_digest(d2, d_out1, DIGEST256_LEN); crypto_digest256(d_out2, "abcdefmno", 9, DIGEST_SHA256); - tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST_LEN); - crypto_digest_get_digest(d1, d_out1, sizeof(d_out1)); + tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST256_LEN); + crypto_digest_get_digest(d1, d_out1, DIGEST256_LEN); crypto_digest256(d_out2, "abcdef", 6, DIGEST_SHA256); - tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST_LEN); + tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST256_LEN); + crypto_digest_free(d1); + crypto_digest_free(d2); + + /* Incremental digest code with sha512 */ + d1 = crypto_digest512_new(DIGEST_SHA512); + tt_assert(d1); + crypto_digest_add_bytes(d1, "abcdef", 6); + d2 = crypto_digest_dup(d1); + tt_assert(d2); + crypto_digest_add_bytes(d2, "ghijkl", 6); + crypto_digest_get_digest(d2, d_out1, DIGEST512_LEN); + crypto_digest512(d_out2, "abcdefghijkl", 12, DIGEST_SHA512); + tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST512_LEN); + crypto_digest_assign(d2, d1); + crypto_digest_add_bytes(d2, "mno", 3); + crypto_digest_get_digest(d2, d_out1, DIGEST512_LEN); + crypto_digest512(d_out2, "abcdefmno", 9, DIGEST_SHA512); + tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST512_LEN); + crypto_digest_get_digest(d1, d_out1, DIGEST512_LEN); + crypto_digest512(d_out2, "abcdef", 6, DIGEST_SHA512); + tt_mem_op(d_out1,OP_EQ, d_out2, DIGEST512_LEN); done: if (d1) @@ -1803,6 +1832,110 @@ test_crypto_siphash(void *arg) ; } +/* We want the likelihood that the random buffer exhibits any regular pattern + * to be far less than the memory bit error rate in the int return value. + * Using 2048 bits provides a failure rate of 1/(3 * 10^616), and we call + * 3 functions, leading to an overall error rate of 1/10^616. + * This is comparable with the 1/10^603 failure rate of test_crypto_rng_range. + */ +#define FAILURE_MODE_BUFFER_SIZE (2048/8) + +/** Check crypto_rand for a failure mode where it does nothing to the buffer, + * or it sets the buffer to all zeroes. Return 0 when the check passes, + * or -1 when it fails. */ +static int +crypto_rand_check_failure_mode_zero(void) +{ + char buf[FAILURE_MODE_BUFFER_SIZE]; + + memset(buf, 0, FAILURE_MODE_BUFFER_SIZE); + crypto_rand(buf, FAILURE_MODE_BUFFER_SIZE); + + for (size_t i = 0; i < FAILURE_MODE_BUFFER_SIZE; i++) { + if (buf[i] != 0) { + return 0; + } + } + + return -1; +} + +/** Check crypto_rand for a failure mode where every int64_t in the buffer is + * the same. Return 0 when the check passes, or -1 when it fails. */ +static int +crypto_rand_check_failure_mode_identical(void) +{ + /* just in case the buffer size isn't a multiple of sizeof(int64_t) */ +#define FAILURE_MODE_BUFFER_SIZE_I64 \ + (FAILURE_MODE_BUFFER_SIZE/SIZEOF_INT64_T) +#define FAILURE_MODE_BUFFER_SIZE_I64_BYTES \ + (FAILURE_MODE_BUFFER_SIZE_I64*SIZEOF_INT64_T) + +#if FAILURE_MODE_BUFFER_SIZE_I64 < 2 +#error FAILURE_MODE_BUFFER_SIZE needs to be at least 2*SIZEOF_INT64_T +#endif + + int64_t buf[FAILURE_MODE_BUFFER_SIZE_I64]; + + memset(buf, 0, FAILURE_MODE_BUFFER_SIZE_I64_BYTES); + crypto_rand((char *)buf, FAILURE_MODE_BUFFER_SIZE_I64_BYTES); + + for (size_t i = 1; i < FAILURE_MODE_BUFFER_SIZE_I64; i++) { + if (buf[i] != buf[i-1]) { + return 0; + } + } + + return -1; +} + +/** Check crypto_rand for a failure mode where it increments the "random" + * value by 1 for every byte in the buffer. (This is OpenSSL's PREDICT mode.) + * Return 0 when the check passes, or -1 when it fails. */ +static int +crypto_rand_check_failure_mode_predict(void) +{ + unsigned char buf[FAILURE_MODE_BUFFER_SIZE]; + + memset(buf, 0, FAILURE_MODE_BUFFER_SIZE); + crypto_rand((char *)buf, FAILURE_MODE_BUFFER_SIZE); + + for (size_t i = 1; i < FAILURE_MODE_BUFFER_SIZE; i++) { + /* check if the last byte was incremented by 1, including integer + * wrapping */ + if (buf[i] - buf[i-1] != 1 && buf[i-1] - buf[i] != 255) { + return 0; + } + } + + return -1; +} + +#undef FAILURE_MODE_BUFFER_SIZE + +static void +test_crypto_failure_modes(void *arg) +{ + int rv = 0; + (void)arg; + + rv = crypto_early_init(); + tt_assert(rv == 0); + + /* Check random works */ + rv = crypto_rand_check_failure_mode_zero(); + tt_assert(rv == 0); + + rv = crypto_rand_check_failure_mode_identical(); + tt_assert(rv == 0); + + rv = crypto_rand_check_failure_mode_predict(); + tt_assert(rv == 0); + + done: + ; +} + #define CRYPTO_LEGACY(name) \ { #name, test_crypto_ ## name , 0, NULL, NULL } @@ -1841,6 +1974,7 @@ struct testcase_t crypto_tests[] = { { "ed25519_fuzz_donna", test_crypto_ed25519_fuzz_donna, TT_FORK, NULL, NULL }, { "siphash", test_crypto_siphash, 0, NULL, NULL }, + { "failure_modes", test_crypto_failure_modes, TT_FORK, NULL, NULL }, END_OF_TESTCASES }; diff --git a/src/test/test_dir_handle_get.c b/src/test/test_dir_handle_get.c new file mode 100644 index 0000000000..be003df2c0 --- /dev/null +++ b/src/test/test_dir_handle_get.c @@ -0,0 +1,2556 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#define RENDCOMMON_PRIVATE +#define GEOIP_PRIVATE +#define CONNECTION_PRIVATE +#define CONFIG_PRIVATE +#define RENDCACHE_PRIVATE + +#include "or.h" +#include "config.h" +#include "connection.h" +#include "directory.h" +#include "test.h" +#include "connection.h" +#include "rendcommon.h" +#include "rendcache.h" +#include "router.h" +#include "routerlist.h" +#include "rend_test_helpers.h" +#include "microdesc.h" +#include "test_helpers.h" +#include "nodelist.h" +#include "entrynodes.h" +#include "routerparse.h" +#include "networkstatus.h" +#include "geoip.h" +#include "dirserv.h" +#include "torgzip.h" +#include "dirvote.h" + +#ifdef _WIN32 +/* For mkdir() */ +#include <direct.h> +#else +#include <dirent.h> +#endif + +#include "vote_descriptors.inc" + +#define NS_MODULE dir_handle_get + +static void +connection_write_to_buf_mock(const char *string, size_t len, + connection_t *conn, int zlib) +{ + (void) zlib; + + tor_assert(string); + tor_assert(conn); + + write_to_buf(string, len, conn->outbuf); +} + +#define GET(path) "GET " path " HTTP/1.0\r\n\r\n" +#define NOT_FOUND "HTTP/1.0 404 Not found\r\n\r\n" +#define BAD_REQUEST "HTTP/1.0 400 Bad request\r\n\r\n" +#define SERVER_BUSY "HTTP/1.0 503 Directory busy, try again later\r\n\r\n" +#define NOT_ENOUGH_CONSENSUS_SIGNATURES "HTTP/1.0 404 " \ + "Consensus not signed by sufficient number of requested authorities\r\n\r\n" + +static tor_addr_t MOCK_TOR_ADDR; + +static void +test_dir_handle_get_bad_request(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(directory_handle_command_get(conn, "", NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(header, OP_EQ, BAD_REQUEST); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_v1_command_not_found(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + // no frontpage configured + tt_ptr_op(get_dirportfrontpage(), OP_EQ, NULL); + + /* V1 path */ + tt_int_op(directory_handle_command_get(conn, GET("/tor/"), NULL, 0), + OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static const char* +mock_get_dirportfrontpage(void) +{ + return "HELLO FROM FRONTPAGE"; +} + +static void +test_dir_handle_get_v1_command(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0, body_len = 0; + const char *exp_body = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + MOCK(get_dirportfrontpage, mock_get_dirportfrontpage); + + exp_body = get_dirportfrontpage(); + body_len = strlen(exp_body); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(directory_handle_command_get(conn, GET("/tor/"), NULL, 0), + OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, body_len+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/html\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 20\r\n")); + + tt_int_op(body_used, OP_EQ, strlen(body)); + tt_str_op(body, OP_EQ, exp_body); + + done: + UNMOCK(connection_write_to_buf_impl_); + UNMOCK(get_dirportfrontpage); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); +} + +static void +test_dir_handle_get_not_found(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + /* Unrecognized path */ + tt_int_op(directory_handle_command_get(conn, GET("/anything"), NULL, 0), + OP_EQ, 0); + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_robots_txt(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + tt_int_op(directory_handle_command_get(conn, GET("/tor/robots.txt"), + NULL, 0), OP_EQ, 0); + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, 29, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 28\r\n")); + + tt_int_op(body_used, OP_EQ, strlen(body)); + tt_str_op(body, OP_EQ, "User-agent: *\r\nDisallow: /\r\n"); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); +} + +static void +test_dir_handle_get_bytes_txt(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0, body_len = 0; + char buff[30]; + char *exp_body = NULL; + (void) data; + + exp_body = directory_dump_request_log(); + body_len = strlen(exp_body); + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + tt_int_op(directory_handle_command_get(conn, GET("/tor/bytes.txt"), NULL, 0), + OP_EQ, 0); + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, body_len+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Pragma: no-cache\r\n")); + + sprintf(buff, "Content-Length: %ld\r\n", (long) body_len); + tt_assert(strstr(header, buff)); + + tt_int_op(body_used, OP_EQ, strlen(body)); + tt_str_op(body, OP_EQ, exp_body); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + tor_free(exp_body); +} + +#define RENDEZVOUS2_GET(descid) GET("/tor/rendezvous2/" descid) +static void +test_dir_handle_get_rendezvous2_not_found_if_not_encrypted(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + // connection is not encrypted + tt_assert(!connection_dir_is_encrypted(conn)) + + tt_int_op(directory_handle_command_get(conn, RENDEZVOUS2_GET(), NULL, 0), + OP_EQ, 0); + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_rendezvous2_on_encrypted_conn_with_invalid_desc_id( + void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + // connection is encrypted + TO_CONN(conn)->linked = 1; + tt_assert(connection_dir_is_encrypted(conn)); + + tt_int_op(directory_handle_command_get(conn, + RENDEZVOUS2_GET("invalid-desc-id"), NULL, 0), OP_EQ, 0); + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(header, OP_EQ, BAD_REQUEST); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_rendezvous2_on_encrypted_conn_not_well_formed(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + // connection is encrypted + TO_CONN(conn)->linked = 1; + tt_assert(connection_dir_is_encrypted(conn)); + + //TODO: this cant be reached because rend_valid_descriptor_id() prevents this + //case to happen. This test is the same as + //test_dir_handle_get_rendezvous2_on_encrypted_conn_with_invalid_desc_id + //We should refactor to remove the case from the switch. + + const char *req = RENDEZVOUS2_GET("1bababababababababababababababab"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(header, OP_EQ, BAD_REQUEST); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_rendezvous2_not_found(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + rend_cache_init(); + + // connection is encrypted + TO_CONN(conn)->linked = 1; + tt_assert(connection_dir_is_encrypted(conn)); + + const char *req = RENDEZVOUS2_GET("3xqunszqnaolrrfmtzgaki7mxelgvkje"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); + rend_cache_free_all(); +} + +NS_DECL(const routerinfo_t *, router_get_my_routerinfo, (void)); +NS_DECL(int, hid_serv_responsible_for_desc_id, (const char *id)); + +static routerinfo_t *mock_routerinfo; +static int hid_serv_responsible_for_desc_id_response; + +static const routerinfo_t * +NS(router_get_my_routerinfo)(void) +{ + if (!mock_routerinfo) { + mock_routerinfo = tor_malloc_zero(sizeof(routerinfo_t)); + } + + return mock_routerinfo; +} + +static int +NS(hid_serv_responsible_for_desc_id)(const char *id) +{ + (void)id; + return hid_serv_responsible_for_desc_id_response; +} + +static void +test_dir_handle_get_rendezvous2_on_encrypted_conn_success(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + char buff[30]; + char req[70]; + rend_encoded_v2_service_descriptor_t *desc_holder = NULL; + char *service_id = NULL; + char desc_id_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; + size_t body_len = 0; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + NS_MOCK(router_get_my_routerinfo); + NS_MOCK(hid_serv_responsible_for_desc_id); + + rend_cache_init(); + hid_serv_responsible_for_desc_id_response = 1; + + /* create a valid rend service descriptor */ + #define RECENT_TIME -10 + generate_desc(RECENT_TIME, &desc_holder, &service_id, 3); + + tt_int_op(rend_cache_store_v2_desc_as_dir(desc_holder->desc_str), + OP_EQ, RCS_OKAY); + + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + // connection is encrypted + TO_CONN(conn)->linked = 1; + tt_assert(connection_dir_is_encrypted(conn)); + + sprintf(req, RENDEZVOUS2_GET("%s"), desc_id_base32); + + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + body_len = strlen(desc_holder->desc_str); + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, body_len+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Pragma: no-cache\r\n")); + sprintf(buff, "Content-Length: %ld\r\n", (long) body_len); + tt_assert(strstr(header, buff)); + + tt_int_op(body_used, OP_EQ, strlen(body)); + tt_str_op(body, OP_EQ, desc_holder->desc_str); + + done: + UNMOCK(connection_write_to_buf_impl_); + NS_UNMOCK(router_get_my_routerinfo); + NS_UNMOCK(hid_serv_responsible_for_desc_id); + tor_free(mock_routerinfo->cache_info.signed_descriptor_body); + tor_free(mock_routerinfo); + + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_cache_free_all(); +} + +#define MICRODESC_GET(digest) GET("/tor/micro/d/" digest) +static void +test_dir_handle_get_micro_d_not_found(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + #define B64_256_1 "8/Pz8/u7vz8/Pz+7vz8/Pz+7u/Pz8/P7u/Pz8/P7u78" + #define B64_256_2 "zMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMw" + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = MICRODESC_GET(B64_256_1 "-" B64_256_2); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static or_options_t *mock_options = NULL; +static void +init_mock_options(void) +{ + mock_options = malloc(sizeof(or_options_t)); + memset(mock_options, 0, sizeof(or_options_t)); + mock_options->TestingTorNetwork = 1; +} + +static const or_options_t * +mock_get_options(void) +{ + tor_assert(mock_options); + return mock_options; +} + +static const char microdesc[] = + "onion-key\n" + "-----BEGIN RSA PUBLIC KEY-----\n" + "MIGJAoGBAMjlHH/daN43cSVRaHBwgUfnszzAhg98EvivJ9Qxfv51mvQUxPjQ07es\n" + "gV/3n8fyh3Kqr/ehi9jxkdgSRfSnmF7giaHL1SLZ29kA7KtST+pBvmTpDtHa3ykX\n" + "Xorc7hJvIyTZoc1HU+5XSynj3gsBE5IGK1ZRzrNS688LnuZMVp1tAgMBAAE=\n" + "-----END RSA PUBLIC KEY-----\n"; + +static void +test_dir_handle_get_micro_d(void *data) +{ + dir_connection_t *conn = NULL; + microdesc_cache_t *mc = NULL ; + smartlist_t *list = NULL; + char digest[DIGEST256_LEN]; + char digest_base64[128]; + char path[80]; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + (void) data; + + MOCK(get_options, mock_get_options); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* SETUP */ + init_mock_options(); + const char *fn = get_fname("dir_handle_datadir_test1"); + mock_options->DataDirectory = tor_strdup(fn); + +#ifdef _WIN32 + tt_int_op(0, OP_EQ, mkdir(mock_options->DataDirectory)); +#else + tt_int_op(0, OP_EQ, mkdir(mock_options->DataDirectory, 0700)); +#endif + + /* Add microdesc to cache */ + crypto_digest256(digest, microdesc, strlen(microdesc), DIGEST_SHA256); + base64_encode_nopad(digest_base64, sizeof(digest_base64), + (uint8_t *) digest, DIGEST256_LEN); + + mc = get_microdesc_cache(); + list = microdescs_add_to_cache(mc, microdesc, NULL, SAVED_NOWHERE, 0, + time(NULL), NULL); + tt_int_op(1, OP_EQ, smartlist_len(list)); + + /* Make the request */ + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + sprintf(path, MICRODESC_GET("%s"), digest_base64); + tt_int_op(directory_handle_command_get(conn, path, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(microdesc)+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + + tt_int_op(body_used, OP_EQ, strlen(body)); + tt_str_op(body, OP_EQ, microdesc); + + done: + UNMOCK(get_options); + UNMOCK(connection_write_to_buf_impl_); + + or_options_free(mock_options); mock_options = NULL; + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + smartlist_free(list); + microdesc_free_all(); +} + +static void +test_dir_handle_get_micro_d_server_busy(void *data) +{ + dir_connection_t *conn = NULL; + microdesc_cache_t *mc = NULL ; + smartlist_t *list = NULL; + char digest[DIGEST256_LEN]; + char digest_base64[128]; + char path[80]; + char *header = NULL; + (void) data; + + MOCK(get_options, mock_get_options); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* SETUP */ + init_mock_options(); + const char *fn = get_fname("dir_handle_datadir_test2"); + mock_options->DataDirectory = tor_strdup(fn); + +#ifdef _WIN32 + tt_int_op(0, OP_EQ, mkdir(mock_options->DataDirectory)); +#else + tt_int_op(0, OP_EQ, mkdir(mock_options->DataDirectory, 0700)); +#endif + + /* Add microdesc to cache */ + crypto_digest256(digest, microdesc, strlen(microdesc), DIGEST_SHA256); + base64_encode_nopad(digest_base64, sizeof(digest_base64), + (uint8_t *) digest, DIGEST256_LEN); + + mc = get_microdesc_cache(); + list = microdescs_add_to_cache(mc, microdesc, NULL, SAVED_NOWHERE, 0, + time(NULL), NULL); + tt_int_op(1, OP_EQ, smartlist_len(list)); + + //Make it busy + mock_options->CountPrivateBandwidth = 1; + + /* Make the request */ + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + sprintf(path, MICRODESC_GET("%s"), digest_base64); + tt_int_op(directory_handle_command_get(conn, path, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(SERVER_BUSY, OP_EQ, header); + + done: + UNMOCK(get_options); + UNMOCK(connection_write_to_buf_impl_); + + or_options_free(mock_options); mock_options = NULL; + connection_free_(TO_CONN(conn)); + tor_free(header); + smartlist_free(list); + microdesc_free_all(); +} + +#define BRIDGES_PATH "/tor/networkstatus-bridges" +static void +test_dir_handle_get_networkstatus_bridges_not_found_without_auth(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(get_options, mock_get_options); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* SETUP */ + init_mock_options(); + mock_options->BridgeAuthoritativeDir = 1; + mock_options->BridgePassword_AuthDigest_ = tor_strdup("digest"); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + TO_CONN(conn)->linked = 1; + + const char *req = GET(BRIDGES_PATH); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(get_options); + UNMOCK(connection_write_to_buf_impl_); + or_options_free(mock_options); mock_options = NULL; + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_networkstatus_bridges(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(get_options, mock_get_options); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* SETUP */ + init_mock_options(); + mock_options->BridgeAuthoritativeDir = 1; + mock_options->BridgePassword_AuthDigest_ = tor_malloc(DIGEST256_LEN); + crypto_digest256(mock_options->BridgePassword_AuthDigest_, + "abcdefghijklm12345", 18, DIGEST_SHA256); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + TO_CONN(conn)->linked = 1; + + const char *req = "GET " BRIDGES_PATH " HTTP/1.0\r\n" + "Authorization: Basic abcdefghijklm12345\r\n\r\n"; + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 0\r\n")); + + done: + UNMOCK(get_options); + UNMOCK(connection_write_to_buf_impl_); + or_options_free(mock_options); mock_options = NULL; + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_networkstatus_bridges_not_found_wrong_auth(void *data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(get_options, mock_get_options); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* SETUP */ + init_mock_options(); + mock_options->BridgeAuthoritativeDir = 1; + mock_options->BridgePassword_AuthDigest_ = tor_malloc(DIGEST256_LEN); + crypto_digest256(mock_options->BridgePassword_AuthDigest_, + "abcdefghijklm12345", 18, DIGEST_SHA256); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + TO_CONN(conn)->linked = 1; + + const char *req = "GET " BRIDGES_PATH " HTTP/1.0\r\n" + "Authorization: Basic NOTSAMEDIGEST\r\n\r\n"; + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(get_options); + UNMOCK(connection_write_to_buf_impl_); + or_options_free(mock_options); mock_options = NULL; + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +#define SERVER_DESC_GET(id) GET("/tor/server/" id) +static void +test_dir_handle_get_server_descriptors_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = SERVER_DESC_GET("invalid"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_str_op(NOT_FOUND, OP_EQ, header); + tt_int_op(conn->dir_spool_src, OP_EQ, DIR_SPOOL_SERVER_BY_FP); + + done: + UNMOCK(connection_write_to_buf_impl_); + or_options_free(mock_options); mock_options = NULL; + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_server_descriptors_all(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + (void) data; + + /* Setup fake routerlist. */ + helper_setup_fake_routerlist(); + + //TODO: change to router_get_my_extrainfo when testing "extra" path + NS_MOCK(router_get_my_routerinfo); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + // We are one of the routers + routerlist_t *our_routerlist = router_get_routerlist(); + tt_int_op(smartlist_len(our_routerlist->routers), OP_GE, 1); + mock_routerinfo = smartlist_get(our_routerlist->routers, 0); + set_server_identity_key(mock_routerinfo->identity_pkey); + + /* Treat "all" requests as if they were unencrypted */ + mock_routerinfo->cache_info.send_unencrypted = 1; + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = SERVER_DESC_GET("all"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + //TODO: Is this a BUG? + //It requires strlen(signed_descriptor_len)+1 as body_len but returns a body + //which is smaller than that by annotation_len bytes + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, + mock_routerinfo->cache_info.signed_descriptor_len+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + + //TODO: Is this a BUG? + //This is what should be expected: tt_int_op(body_used, OP_EQ, strlen(body)); + tt_int_op(body_used, OP_EQ, + mock_routerinfo->cache_info.signed_descriptor_len); + + tt_str_op(body, OP_EQ, mock_routerinfo->cache_info.signed_descriptor_body + + mock_routerinfo->cache_info.annotations_len); + tt_int_op(conn->dir_spool_src, OP_EQ, DIR_SPOOL_NONE); + + done: + NS_UNMOCK(router_get_my_routerinfo); + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + + routerlist_free_all(); + nodelist_free_all(); + entry_guards_free_all(); +} + +static char +TEST_DESCRIPTOR[] = +"@uploaded-at 2014-06-08 19:20:11\n" +"@source \"127.0.0.1\"\n" +"router test000a 127.0.0.1 5000 0 7000\n" +"platform Tor 0.2.5.3-alpha-dev on Linux\n" +"protocols Link 1 2 Circuit 1\n" +"published 2014-06-08 19:20:11\n" +"fingerprint C7E7 CCB8 179F 8CC3 7F5C 8A04 2B3A 180B 934B 14BA\n" +"uptime 0\n" +"bandwidth 1073741824 1073741824 0\n" +"extra-info-digest 67A152A4C7686FB07664F872620635F194D76D95\n" +"caches-extra-info\n" +"onion-key\n" +"-----BEGIN RSA PUBLIC KEY-----\n" +"MIGJAoGBAOuBUIEBARMkkka/TGyaQNgUEDLP0KG7sy6KNQTNOlZHUresPr/vlVjo\n" +"HPpLMfu9M2z18c51YX/muWwY9x4MyQooD56wI4+AqXQcJRwQfQlPn3Ay82uZViA9\n" +"DpBajRieLlKKkl145KjArpD7F5BVsqccvjErgFYXvhhjSrx7BVLnAgMBAAE=\n" +"-----END RSA PUBLIC KEY-----\n" +"signing-key\n" +"-----BEGIN RSA PUBLIC KEY-----\n" +"MIGJAoGBAN6NLnSxWQnFXxqZi5D3b0BMgV6y9NJLGjYQVP+eWtPZWgqyv4zeYsqv\n" +"O9y6c5lvxyUxmNHfoAbe/s8f2Vf3/YaC17asAVSln4ktrr3e9iY74a9RMWHv1Gzk\n" +"3042nMcqj3PEhRN0PoLkcOZNjjmNbaqki6qy9bWWZDNTdo+uI44dAgMBAAE=\n" +"-----END RSA PUBLIC KEY-----\n" +"hidden-service-dir\n" +"contact auth0@test.test\n" +"ntor-onion-key pK4bs08ERYN591jj7ca17Rn9Q02TIEfhnjR6hSq+fhU=\n" +"reject *:*\n" +"router-signature\n" +"-----BEGIN SIGNATURE-----\n" +"rx88DuM3Y7tODlHNDDEVzKpwh3csaG1or+T4l2Xs1oq3iHHyPEtB6QTLYrC60trG\n" +"aAPsj3DEowGfjga1b248g2dtic8Ab+0exfjMm1RHXfDam5TXXZU3A0wMyoHjqHuf\n" +"eChGPgFNUvEc+5YtD27qEDcUjcinYztTs7/dzxBT4PE=\n" +"-----END SIGNATURE-----\n"; + +static void +test_dir_handle_get_server_descriptors_authority(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + crypto_pk_t *identity_pkey = pk_generate(0); + (void) data; + + NS_MOCK(router_get_my_routerinfo); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* init mock */ + router_get_my_routerinfo(); + crypto_pk_get_digest(identity_pkey, + mock_routerinfo->cache_info.identity_digest); + + // the digest is mine (the channel is unnecrypted, so we must allow sending) + set_server_identity_key(identity_pkey); + mock_routerinfo->cache_info.send_unencrypted = 1; + + /* Setup descriptor */ + long annotation_len = strstr(TEST_DESCRIPTOR, "router ") - TEST_DESCRIPTOR; + mock_routerinfo->cache_info.signed_descriptor_body = + tor_strdup(TEST_DESCRIPTOR); + mock_routerinfo->cache_info.signed_descriptor_len = + strlen(TEST_DESCRIPTOR) - annotation_len;; + mock_routerinfo->cache_info.annotations_len = annotation_len; + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = SERVER_DESC_GET("authority"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + //TODO: Is this a BUG? + //It requires strlen(TEST_DESCRIPTOR)+1 as body_len but returns a body which + //is smaller than that by annotation_len bytes + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(TEST_DESCRIPTOR)+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + + tt_int_op(body_used, OP_EQ, strlen(body)); + + tt_str_op(body, OP_EQ, TEST_DESCRIPTOR + annotation_len); + tt_int_op(conn->dir_spool_src, OP_EQ, DIR_SPOOL_NONE); + + done: + NS_UNMOCK(router_get_my_routerinfo); + UNMOCK(connection_write_to_buf_impl_); + tor_free(mock_routerinfo->cache_info.signed_descriptor_body); + tor_free(mock_routerinfo); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + crypto_pk_free(identity_pkey); +} + +static void +test_dir_handle_get_server_descriptors_fp(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + crypto_pk_t *identity_pkey = pk_generate(0); + (void) data; + + NS_MOCK(router_get_my_routerinfo); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* init mock */ + router_get_my_routerinfo(); + crypto_pk_get_digest(identity_pkey, + mock_routerinfo->cache_info.identity_digest); + + // the digest is mine (the channel is unnecrypted, so we must allow sending) + set_server_identity_key(identity_pkey); + mock_routerinfo->cache_info.send_unencrypted = 1; + + /* Setup descriptor */ + long annotation_len = strstr(TEST_DESCRIPTOR, "router ") - TEST_DESCRIPTOR; + mock_routerinfo->cache_info.signed_descriptor_body = + tor_strdup(TEST_DESCRIPTOR); + mock_routerinfo->cache_info.signed_descriptor_len = + strlen(TEST_DESCRIPTOR) - annotation_len; + mock_routerinfo->cache_info.annotations_len = annotation_len; + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + #define HEX1 "Fe0daff89127389bc67558691231234551193EEE" + #define HEX2 "Deadbeef99999991111119999911111111f00ba4" + const char *hex_digest = hex_str(mock_routerinfo->cache_info.identity_digest, + DIGEST_LEN); + + char req[155]; + sprintf(req, SERVER_DESC_GET("fp/%s+" HEX1 "+" HEX2), hex_digest); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + //TODO: Is this a BUG? + //It requires strlen(TEST_DESCRIPTOR)+1 as body_len but returns a body which + //is smaller than that by annotation_len bytes + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(TEST_DESCRIPTOR)+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + + tt_int_op(body_used, OP_EQ, strlen(body)); + + tt_str_op(body, OP_EQ, TEST_DESCRIPTOR + annotation_len); + tt_int_op(conn->dir_spool_src, OP_EQ, DIR_SPOOL_NONE); + + done: + NS_UNMOCK(router_get_my_routerinfo); + UNMOCK(connection_write_to_buf_impl_); + tor_free(mock_routerinfo->cache_info.signed_descriptor_body); + tor_free(mock_routerinfo); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + crypto_pk_free(identity_pkey); +} + +#define HEX1 "Fe0daff89127389bc67558691231234551193EEE" +#define HEX2 "Deadbeef99999991111119999911111111f00ba4" + +static void +test_dir_handle_get_server_descriptors_d(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + crypto_pk_t *identity_pkey = pk_generate(0); + (void) data; + + /* Setup fake routerlist. */ + helper_setup_fake_routerlist(); + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* Get one router's signed_descriptor_digest */ + routerlist_t *our_routerlist = router_get_routerlist(); + tt_int_op(smartlist_len(our_routerlist->routers), OP_GE, 1); + routerinfo_t *router = smartlist_get(our_routerlist->routers, 0); + const char *hex_digest = hex_str(router->cache_info.signed_descriptor_digest, + DIGEST_LEN); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + char req_header[155]; + sprintf(req_header, SERVER_DESC_GET("d/%s+" HEX1 "+" HEX2), hex_digest); + tt_int_op(directory_handle_command_get(conn, req_header, NULL, 0), OP_EQ, 0); + + //TODO: Is this a BUG? + //It requires strlen(signed_descriptor_len)+1 as body_len but returns a body + //which is smaller than that by annotation_len bytes + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, + router->cache_info.signed_descriptor_len+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + + //TODO: Is this a BUG? + //This is what should be expected: + //tt_int_op(body_used, OP_EQ, strlen(body)); + tt_int_op(body_used, OP_EQ, router->cache_info.signed_descriptor_len); + + tt_str_op(body, OP_EQ, router->cache_info.signed_descriptor_body + + router->cache_info.annotations_len); + tt_int_op(conn->dir_spool_src, OP_EQ, DIR_SPOOL_NONE); + + done: + UNMOCK(connection_write_to_buf_impl_); + tor_free(mock_routerinfo); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + crypto_pk_free(identity_pkey); + + routerlist_free_all(); + nodelist_free_all(); + entry_guards_free_all(); +} + +static void +test_dir_handle_get_server_descriptors_busy(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + crypto_pk_t *identity_pkey = pk_generate(0); + (void) data; + + /* Setup fake routerlist. */ + helper_setup_fake_routerlist(); + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + //Make it busy + MOCK(get_options, mock_get_options); + init_mock_options(); + mock_options->CountPrivateBandwidth = 1; + + /* Get one router's signed_descriptor_digest */ + routerlist_t *our_routerlist = router_get_routerlist(); + tt_int_op(smartlist_len(our_routerlist->routers), OP_GE, 1); + routerinfo_t *router = smartlist_get(our_routerlist->routers, 0); + const char *hex_digest = hex_str(router->cache_info.signed_descriptor_digest, + DIGEST_LEN); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + #define HEX1 "Fe0daff89127389bc67558691231234551193EEE" + #define HEX2 "Deadbeef99999991111119999911111111f00ba4" + char req_header[155]; + sprintf(req_header, SERVER_DESC_GET("d/%s+" HEX1 "+" HEX2), hex_digest); + tt_int_op(directory_handle_command_get(conn, req_header, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_assert(header); + tt_str_op(SERVER_BUSY, OP_EQ, header); + + tt_int_op(conn->dir_spool_src, OP_EQ, DIR_SPOOL_NONE); + + done: + UNMOCK(get_options); + UNMOCK(connection_write_to_buf_impl_); + tor_free(mock_routerinfo); + connection_free_(TO_CONN(conn)); + tor_free(header); + crypto_pk_free(identity_pkey); + + routerlist_free_all(); + nodelist_free_all(); + entry_guards_free_all(); +} + +static void +test_dir_handle_get_server_keys_bad_req(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = GET("/tor/keys/"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_assert(header); + tt_str_op(BAD_REQUEST, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_server_keys_all_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = GET("/tor/keys/all"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +#define TEST_CERTIFICATE AUTHORITY_CERT_3 +#define TEST_SIGNING_KEY AUTHORITY_SIGNKEY_A_DIGEST +extern const char AUTHORITY_CERT_3[]; +extern const char AUTHORITY_SIGNKEY_A_DIGEST[]; + +static const char TEST_CERT_IDENT_KEY[] = + "D867ACF56A9D229B35C25F0090BC9867E906BE69"; + +static void +test_dir_handle_get_server_keys_all(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + const char digest[DIGEST_LEN] = ""; + + dir_server_t *ds = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + clear_dir_servers(); + routerlist_free_all(); + + /* create a trusted ds */ + ds = trusted_dir_server_new("ds", "127.0.0.1", 9059, 9060, digest, NULL, + V3_DIRINFO, 1.0); + tt_assert(ds); + dir_server_add(ds); + + /* ds v3_identity_digest is the certificate's identity_key */ + base16_decode(ds->v3_identity_digest, DIGEST_LEN, + TEST_CERT_IDENT_KEY, HEX_DIGEST_LEN); + tt_int_op(0, OP_EQ, trusted_dirs_load_certs_from_string(TEST_CERTIFICATE, + TRUSTED_DIRS_CERTS_SRC_DL_BY_ID_DIGEST, 1)); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = GET("/tor/keys/all"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(TEST_CERTIFICATE)+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 1883\r\n")); + + tt_str_op(TEST_CERTIFICATE, OP_EQ, body); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + + clear_dir_servers(); + routerlist_free_all(); +} + +static void +test_dir_handle_get_server_keys_authority_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = GET("/tor/keys/authority"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static authority_cert_t * mock_cert = NULL; + +static authority_cert_t * +get_my_v3_authority_cert_m(void) +{ + tor_assert(mock_cert); + return mock_cert; +} + +static void +test_dir_handle_get_server_keys_authority(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + (void) data; + + mock_cert = authority_cert_parse_from_string(TEST_CERTIFICATE, NULL); + + MOCK(get_my_v3_authority_cert, get_my_v3_authority_cert_m); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = GET("/tor/keys/authority"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(TEST_CERTIFICATE)+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 1883\r\n")); + + tt_str_op(TEST_CERTIFICATE, OP_EQ, body); + + done: + UNMOCK(get_my_v3_authority_cert); + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + authority_cert_free(mock_cert); mock_cert = NULL; +} + +static void +test_dir_handle_get_server_keys_fp_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = GET("/tor/keys/fp/somehex"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_server_keys_fp(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + dir_server_t *ds = NULL; + const char digest[DIGEST_LEN] = ""; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + clear_dir_servers(); + routerlist_free_all(); + + /* create a trusted ds */ + ds = trusted_dir_server_new("ds", "127.0.0.1", 9059, 9060, digest, NULL, + V3_DIRINFO, 1.0); + tt_assert(ds); + dir_server_add(ds); + + /* ds v3_identity_digest is the certificate's identity_key */ + base16_decode(ds->v3_identity_digest, DIGEST_LEN, + TEST_CERT_IDENT_KEY, HEX_DIGEST_LEN); + + tt_int_op(0, OP_EQ, trusted_dirs_load_certs_from_string(TEST_CERTIFICATE, + TRUSTED_DIRS_CERTS_SRC_DL_BY_ID_DIGEST, 1)); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + char req[71]; + sprintf(req, GET("/tor/keys/fp/%s"), TEST_CERT_IDENT_KEY); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(TEST_CERTIFICATE)+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 1883\r\n")); + + tt_str_op(TEST_CERTIFICATE, OP_EQ, body); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + clear_dir_servers(); + routerlist_free_all(); +} + +static void +test_dir_handle_get_server_keys_sk_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = GET("/tor/keys/sk/somehex"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_server_keys_sk(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + (void) data; + + mock_cert = authority_cert_parse_from_string(TEST_CERTIFICATE, NULL); + MOCK(get_my_v3_authority_cert, get_my_v3_authority_cert_m); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + clear_dir_servers(); + routerlist_free_all(); + + tt_int_op(0, OP_EQ, trusted_dirs_load_certs_from_string(TEST_CERTIFICATE, + TRUSTED_DIRS_CERTS_SRC_DL_BY_ID_DIGEST, 1)); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + char req[71]; + sprintf(req, GET("/tor/keys/sk/%s"), TEST_SIGNING_KEY); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(TEST_CERTIFICATE)+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 1883\r\n")); + + tt_str_op(TEST_CERTIFICATE, OP_EQ, body); + + done: + UNMOCK(get_my_v3_authority_cert); + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + authority_cert_free(mock_cert); mock_cert = NULL; + tor_free(header); + tor_free(body); +} + +static void +test_dir_handle_get_server_keys_fpsk_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + const char *req = GET("/tor/keys/fp-sk/somehex"); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_server_keys_fpsk(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + dir_server_t *ds = NULL; + const char digest[DIGEST_LEN] = ""; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + clear_dir_servers(); + routerlist_free_all(); + + /* create a trusted ds */ + ds = trusted_dir_server_new("ds", "127.0.0.1", 9059, 9060, digest, NULL, + V3_DIRINFO, 1.0); + tt_assert(ds); + + /* ds v3_identity_digest is the certificate's identity_key */ + base16_decode(ds->v3_identity_digest, DIGEST_LEN, + TEST_CERT_IDENT_KEY, HEX_DIGEST_LEN); + dir_server_add(ds); + + tt_int_op(0, OP_EQ, trusted_dirs_load_certs_from_string(TEST_CERTIFICATE, + TRUSTED_DIRS_CERTS_SRC_DL_BY_ID_DIGEST, 1)); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + char req[115]; + sprintf(req, GET("/tor/keys/fp-sk/%s-%s"), + TEST_CERT_IDENT_KEY, TEST_SIGNING_KEY); + + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(TEST_CERTIFICATE)+1, 0); + + tt_assert(header); + tt_assert(body); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 1883\r\n")); + + tt_str_op(TEST_CERTIFICATE, OP_EQ, body); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + + clear_dir_servers(); + routerlist_free_all(); +} + +static void +test_dir_handle_get_server_keys_busy(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + dir_server_t *ds = NULL; + const char digest[DIGEST_LEN] = ""; + (void) data; + + clear_dir_servers(); + routerlist_free_all(); + + /* create a trusted ds */ + ds = trusted_dir_server_new("ds", "127.0.0.1", 9059, 9060, digest, NULL, + V3_DIRINFO, 1.0); + tt_assert(ds); + + /* ds v3_identity_digest is the certificate's identity_key */ + base16_decode(ds->v3_identity_digest, DIGEST_LEN, + TEST_CERT_IDENT_KEY, HEX_DIGEST_LEN); + dir_server_add(ds); + + tt_int_op(0, OP_EQ, trusted_dirs_load_certs_from_string(TEST_CERTIFICATE, + TRUSTED_DIRS_CERTS_SRC_DL_BY_ID_DIGEST, 1)); + + MOCK(get_options, mock_get_options); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* setup busy server */ + init_mock_options(); + mock_options->CountPrivateBandwidth = 1; + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + char req[71]; + sprintf(req, GET("/tor/keys/fp/%s"), TEST_CERT_IDENT_KEY); + tt_int_op(directory_handle_command_get(conn, req, NULL, 0), OP_EQ, 0); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_assert(header); + tt_str_op(SERVER_BUSY, OP_EQ, header); + + done: + UNMOCK(get_options); + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); + or_options_free(mock_options); mock_options = NULL; + + clear_dir_servers(); + routerlist_free_all(); +} + +static networkstatus_t *mock_ns_val = NULL; +static networkstatus_t * +mock_ns_get_by_flavor(consensus_flavor_t f) +{ + (void)f; + return mock_ns_val; +} + +static void +test_dir_handle_get_status_vote_current_consensus_ns_not_enough_sigs(void* d) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *stats = NULL; + (void) d; + + /* init mock */ + mock_ns_val = tor_malloc_zero(sizeof(networkstatus_t)); + mock_ns_val->flavor = FLAV_NS; + mock_ns_val->voters = smartlist_new(); + + /* init mock */ + init_mock_options(); + + MOCK(get_options, mock_get_options); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + MOCK(networkstatus_get_latest_consensus_by_flavor, mock_ns_get_by_flavor); + + /* start gathering stats */ + mock_options->DirReqStatistics = 1; + geoip_dirreq_stats_init(time(NULL)); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/current/consensus-ns/" HEX1 "+" HEX2), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + + tt_assert(header); + tt_str_op(NOT_ENOUGH_CONSENSUS_SIGNATURES, OP_EQ, header); + + stats = geoip_format_dirreq_stats(time(NULL)); + tt_assert(stats); + tt_assert(strstr(stats, "not-enough-sigs=8")); + + done: + UNMOCK(networkstatus_get_latest_consensus_by_flavor); + UNMOCK(connection_write_to_buf_impl_); + UNMOCK(get_options); + + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(stats); + smartlist_free(mock_ns_val->voters); + tor_free(mock_ns_val); + or_options_free(mock_options); mock_options = NULL; +} + +static void +test_dir_handle_get_status_vote_current_consensus_ns_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + char *stats = NULL; + (void) data; + + init_mock_options(); + + MOCK(get_options, mock_get_options); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + /* start gathering stats */ + mock_options->DirReqStatistics = 1; + geoip_dirreq_stats_init(time(NULL)); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/current/consensus-ns"), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + stats = geoip_format_dirreq_stats(time(NULL)); + tt_assert(stats); + tt_assert(strstr(stats, "not-found=8")); + + done: + UNMOCK(connection_write_to_buf_impl_); + UNMOCK(get_options); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(stats); + or_options_free(mock_options); mock_options = NULL; +} + +NS_DECL(int, geoip_get_country_by_addr, (const tor_addr_t *addr)); + +int +NS(geoip_get_country_by_addr)(const tor_addr_t *addr) +{ + (void)addr; + CALLED(geoip_get_country_by_addr)++; + return 1; +} + +static void +status_vote_current_consensus_ns_test(char **header, char **body, + size_t *body_len) +{ + digests_t digests; + dir_connection_t *conn = NULL; + + #define NETWORK_STATUS "some network status string" + dirserv_set_cached_consensus_networkstatus(NETWORK_STATUS, "ns", &digests, + time(NULL)); + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + tt_assert(mock_options); + mock_options->DirReqStatistics = 1; + geoip_dirreq_stats_init(time(NULL)); + + /* init geoip database */ + geoip_parse_entry("10,50,AB", AF_INET); + tt_str_op("ab", OP_EQ, geoip_get_country_name(1)); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + TO_CONN(conn)->address = tor_strdup("127.0.0.1"); + + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/current/consensus-ns"), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, header, MAX_HEADERS_SIZE, + body, body_len, strlen(NETWORK_STATUS)+7, 0); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); +} + +static void +test_dir_handle_get_status_vote_current_consensus_ns(void* data) +{ + char *header = NULL; + char *body = NULL, *comp_body = NULL; + size_t body_used = 0, comp_body_used = 0; + char *stats = NULL, *hist = NULL; + (void) data; + + dirserv_free_all(); + clear_geoip_db(); + + NS_MOCK(geoip_get_country_by_addr); + MOCK(get_options, mock_get_options); + + init_mock_options(); + + status_vote_current_consensus_ns_test(&header, &comp_body, &comp_body_used); + tt_assert(header); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Pragma: no-cache\r\n")); + + compress_method_t compression = detect_compression_method(comp_body, + comp_body_used); + tt_int_op(ZLIB_METHOD, OP_EQ, compression); + + tor_gzip_uncompress(&body, &body_used, comp_body, comp_body_used, + compression, 0, LOG_PROTOCOL_WARN); + + tt_str_op(NETWORK_STATUS, OP_EQ, body); + tt_int_op(strlen(NETWORK_STATUS), OP_EQ, body_used); + + stats = geoip_format_dirreq_stats(time(NULL)); + tt_assert(stats); + + tt_assert(strstr(stats, "ok=8")); + tt_assert(strstr(stats, "dirreq-v3-ips ab=8")); + tt_assert(strstr(stats, "dirreq-v3-reqs ab=8")); + tt_assert(strstr(stats, "dirreq-v3-direct-dl" + " complete=0,timeout=0,running=4")); + + hist = geoip_get_request_history(); + tt_assert(hist); + tt_str_op("ab=8", OP_EQ, hist); + + done: + NS_UNMOCK(geoip_get_country_by_addr); + UNMOCK(get_options); + tor_free(header); + tor_free(comp_body); + tor_free(body); + tor_free(stats); + tor_free(hist); + or_options_free(mock_options); mock_options = NULL; + + dirserv_free_all(); + clear_geoip_db(); +} + +static void +test_dir_handle_get_status_vote_current_consensus_ns_busy(void* data) +{ + char *header = NULL; + char *body = NULL; + size_t body_used = 0; + char *stats = NULL; + (void) data; + + dirserv_free_all(); + clear_geoip_db(); + + MOCK(get_options, mock_get_options); + + // Make it busy + init_mock_options(); + mock_options->CountPrivateBandwidth = 1; + + status_vote_current_consensus_ns_test(&header, &body, &body_used); + tt_assert(header); + + tt_str_op(SERVER_BUSY, OP_EQ, header); + + stats = geoip_format_dirreq_stats(time(NULL)); + tt_assert(stats); + tt_assert(strstr(stats, "busy=8")); + + done: + UNMOCK(get_options); + tor_free(header); + tor_free(body); + or_options_free(mock_options); mock_options = NULL; + + tor_free(stats); + dirserv_free_all(); + clear_geoip_db(); +} + +static void +test_dir_handle_get_status_vote_current_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/current/" HEX1), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +#define VOTE_DIGEST "312A4890D4D832597ABBD3089C782DBBFB81E48D" + +static void +status_vote_current_d_test(char **header, char **body, size_t *body_l) +{ + dir_connection_t *conn = NULL; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/current/d/" VOTE_DIGEST), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, header, MAX_HEADERS_SIZE, + body, body_l, strlen(VOTE_BODY_V3)+1, 0); + tt_assert(header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); +} + +static void +status_vote_next_d_test(char **header, char **body, size_t *body_l) +{ + dir_connection_t *conn = NULL; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/next/d/" VOTE_DIGEST), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, header, MAX_HEADERS_SIZE, + body, body_l, strlen(VOTE_BODY_V3)+1, 0); + tt_assert(header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); +} + +static void +test_dir_handle_get_status_vote_current_d_not_found(void* data) +{ + char *header = NULL; + (void) data; + + status_vote_current_d_test(&header, NULL, NULL); + + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + tor_free(header); +} + +static void +test_dir_handle_get_status_vote_next_d_not_found(void* data) +{ + char *header = NULL; + (void) data; + + status_vote_next_d_test(&header, NULL, NULL); + + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + tor_free(header); +} + +static void +test_dir_handle_get_status_vote_d(void* data) +{ + char *header = NULL, *body = NULL; + size_t body_used = 0; + dir_server_t *ds = NULL; + const char digest[DIGEST_LEN] = ""; + (void) data; + + clear_dir_servers(); + dirvote_free_all(); + + /* create a trusted ds */ + ds = trusted_dir_server_new("ds", "127.0.0.1", 9059, 9060, digest, NULL, + V3_DIRINFO, 1.0); + tt_assert(ds); + dir_server_add(ds); + + /* ds v3_identity_digest is the certificate's identity_key */ + base16_decode(ds->v3_identity_digest, DIGEST_LEN, + TEST_CERT_IDENT_KEY, HEX_DIGEST_LEN); + + init_mock_options(); + mock_options->AuthoritativeDir = 1; + mock_options->V3AuthoritativeDir = 1; + mock_options->TestingV3AuthVotingStartOffset = 0; + mock_options->TestingV3AuthInitialVotingInterval = 1; + mock_options->TestingV3AuthInitialVoteDelay = 1; + mock_options->TestingV3AuthInitialDistDelay = 1; + + time_t now = 1441223455 -1; + dirvote_recalculate_timing(mock_options, now); + + const char *msg_out = NULL; + int status_out = 0; + struct pending_vote_t *pv = dirvote_add_vote(VOTE_BODY_V3, &msg_out, + &status_out); + tt_assert(pv); + + status_vote_current_d_test(&header, &body, &body_used); + + tt_assert(header); + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 4135\r\n")); + + tt_str_op(VOTE_BODY_V3, OP_EQ, body); + + tor_free(header); + tor_free(body); + + status_vote_next_d_test(&header, &body, &body_used); + + tt_assert(header); + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 4135\r\n")); + + tt_str_op(VOTE_BODY_V3, OP_EQ, body); + + done: + tor_free(header); + tor_free(body); + or_options_free(mock_options); mock_options = NULL; + + clear_dir_servers(); + dirvote_free_all(); +} + +static void +test_dir_handle_get_status_vote_next_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/next/" HEX1), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +status_vote_next_consensus_test(char **header, char **body, size_t *body_used) +{ + dir_connection_t *conn = NULL; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/next/consensus"), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, header, MAX_HEADERS_SIZE, + body, body_used, 18, 0); + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); +} + +static void +test_dir_handle_get_status_vote_next_consensus_not_found(void* data) +{ + char *header = NULL, *body = NULL; + size_t body_used; + (void) data; + + status_vote_next_consensus_test(&header, &body, &body_used); + + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + tor_free(header); + tor_free(body); +} + +static void +test_dir_handle_get_status_vote_current_authority_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/current/authority"), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +static void +test_dir_handle_get_status_vote_next_authority_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL; + (void) data; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/next/authority"), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + NULL, NULL, 1, 0); + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + UNMOCK(connection_write_to_buf_impl_); + connection_free_(TO_CONN(conn)); + tor_free(header); +} + +NS_DECL(const char*, +dirvote_get_pending_consensus, (consensus_flavor_t flav)); + +const char* +NS(dirvote_get_pending_consensus)(consensus_flavor_t flav) +{ + (void)flav; + return "pending consensus"; +} + +static void +test_dir_handle_get_status_vote_next_consensus(void* data) +{ + char *header = NULL, *body = NULL; + size_t body_used = 0; + (void) data; + + NS_MOCK(dirvote_get_pending_consensus); + + status_vote_next_consensus_test(&header, &body, &body_used); + tt_assert(header); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 17\r\n")); + + tt_str_op("pending consensus", OP_EQ, body); + + done: + NS_UNMOCK(dirvote_get_pending_consensus); + tor_free(header); + tor_free(body); +} + +static void +test_dir_handle_get_status_vote_next_consensus_busy(void* data) +{ + char *header = NULL, *body = NULL; + size_t body_used = 0; + (void) data; + + MOCK(get_options, mock_get_options); + NS_MOCK(dirvote_get_pending_consensus); + + //Make it busy + init_mock_options(); + mock_options->CountPrivateBandwidth = 1; + + status_vote_next_consensus_test(&header, &body, &body_used); + + tt_assert(header); + tt_str_op(SERVER_BUSY, OP_EQ, header); + + done: + NS_UNMOCK(dirvote_get_pending_consensus); + UNMOCK(get_options); + tor_free(header); + tor_free(body); + or_options_free(mock_options); mock_options = NULL; +} + +static void +status_vote_next_consensus_signatures_test(char **header, char **body, + size_t *body_used) +{ + dir_connection_t *conn = NULL; + + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/next/consensus-signatures"), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, header, MAX_HEADERS_SIZE, + body, body_used, 22, 0); + + done: + connection_free_(TO_CONN(conn)); + UNMOCK(connection_write_to_buf_impl_); +} + +static void +test_dir_handle_get_status_vote_next_consensus_signatures_not_found(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL, *body = NULL; + size_t body_used; + (void) data; + + status_vote_next_consensus_signatures_test(&header, &body, &body_used); + + tt_assert(header); + tt_str_op(NOT_FOUND, OP_EQ, header); + + done: + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); +} + +NS_DECL(const char*, +dirvote_get_pending_detached_signatures, (void)); + +const char* +NS(dirvote_get_pending_detached_signatures)(void) +{ + return "pending detached sigs"; +} + +static void +test_dir_handle_get_status_vote_next_consensus_signatures(void* data) +{ + char *header = NULL, *body = NULL; + size_t body_used = 0; + (void) data; + + NS_MOCK(dirvote_get_pending_detached_signatures); + + status_vote_next_consensus_signatures_test(&header, &body, &body_used); + tt_assert(header); + + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 21\r\n")); + + tt_str_op("pending detached sigs", OP_EQ, body); + + done: + NS_UNMOCK(dirvote_get_pending_detached_signatures); + tor_free(header); + tor_free(body); +} + +static void +test_dir_handle_get_status_vote_next_consensus_signatures_busy(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL, *body = NULL; + size_t body_used; + (void) data; + + NS_MOCK(dirvote_get_pending_detached_signatures); + MOCK(get_options, mock_get_options); + + //Make it busy + init_mock_options(); + mock_options->CountPrivateBandwidth = 1; + + status_vote_next_consensus_signatures_test(&header, &body, &body_used); + + tt_assert(header); + tt_str_op(SERVER_BUSY, OP_EQ, header); + + done: + UNMOCK(get_options); + NS_UNMOCK(dirvote_get_pending_detached_signatures); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + or_options_free(mock_options); mock_options = NULL; +} + +static void +test_dir_handle_get_status_vote_next_authority(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL, *body = NULL; + const char *msg_out = NULL; + int status_out = 0; + size_t body_used = 0; + dir_server_t *ds = NULL; + const char digest[DIGEST_LEN] = ""; + (void) data; + + clear_dir_servers(); + routerlist_free_all(); + dirvote_free_all(); + + mock_cert = authority_cert_parse_from_string(TEST_CERTIFICATE, NULL); + + /* create a trusted ds */ + ds = trusted_dir_server_new("ds", "127.0.0.1", 9059, 9060, digest, NULL, + V3_DIRINFO, 1.0); + tt_assert(ds); + dir_server_add(ds); + + /* ds v3_identity_digest is the certificate's identity_key */ + base16_decode(ds->v3_identity_digest, DIGEST_LEN, + TEST_CERT_IDENT_KEY, HEX_DIGEST_LEN); + tt_int_op(0, OP_EQ, trusted_dirs_load_certs_from_string(TEST_CERTIFICATE, + TRUSTED_DIRS_CERTS_SRC_DL_BY_ID_DIGEST, 1)); + + init_mock_options(); + mock_options->AuthoritativeDir = 1; + mock_options->V3AuthoritativeDir = 1; + mock_options->TestingV3AuthVotingStartOffset = 0; + mock_options->TestingV3AuthInitialVotingInterval = 1; + mock_options->TestingV3AuthInitialVoteDelay = 1; + mock_options->TestingV3AuthInitialDistDelay = 1; + + time_t now = 1441223455 -1; + dirvote_recalculate_timing(mock_options, now); + + struct pending_vote_t *vote = dirvote_add_vote(VOTE_BODY_V3, &msg_out, + &status_out); + tt_assert(vote); + + MOCK(get_my_v3_authority_cert, get_my_v3_authority_cert_m); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/next/authority"), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(VOTE_BODY_V3)+1, 0); + + tt_assert(header); + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 4135\r\n")); + + tt_str_op(VOTE_BODY_V3, OP_EQ, body); + + done: + UNMOCK(connection_write_to_buf_impl_); + UNMOCK(get_my_v3_authority_cert); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + authority_cert_free(mock_cert); mock_cert = NULL; + or_options_free(mock_options); mock_options = NULL; + + clear_dir_servers(); + routerlist_free_all(); + dirvote_free_all(); +} + +static void +test_dir_handle_get_status_vote_current_authority(void* data) +{ + dir_connection_t *conn = NULL; + char *header = NULL, *body = NULL; + const char *msg_out = NULL; + int status_out = 0; + size_t body_used = 0; + const char digest[DIGEST_LEN] = ""; + + dir_server_t *ds = NULL; + (void) data; + + clear_dir_servers(); + routerlist_free_all(); + dirvote_free_all(); + + mock_cert = authority_cert_parse_from_string(TEST_CERTIFICATE, NULL); + + /* create a trusted ds */ + ds = trusted_dir_server_new("ds", "127.0.0.1", 9059, 9060, digest, NULL, + V3_DIRINFO, 1.0); + tt_assert(ds); + dir_server_add(ds); + + /* ds v3_identity_digest is the certificate's identity_key */ + base16_decode(ds->v3_identity_digest, DIGEST_LEN, + TEST_CERT_IDENT_KEY, HEX_DIGEST_LEN); + + tt_int_op(0, OP_EQ, trusted_dirs_load_certs_from_string(TEST_CERTIFICATE, + TRUSTED_DIRS_CERTS_SRC_DL_BY_ID_DIGEST, 1)); + + init_mock_options(); + mock_options->AuthoritativeDir = 1; + mock_options->V3AuthoritativeDir = 1; + mock_options->TestingV3AuthVotingStartOffset = 0; + mock_options->TestingV3AuthInitialVotingInterval = 1; + mock_options->TestingV3AuthInitialVoteDelay = 1; + mock_options->TestingV3AuthInitialDistDelay = 1; + + time_t now = 1441223455; + dirvote_recalculate_timing(mock_options, now-1); + + struct pending_vote_t *vote = dirvote_add_vote(VOTE_BODY_V3, &msg_out, + &status_out); + tt_assert(vote); + + // move the pending vote to previous vote + dirvote_act(mock_options, now+1); + + MOCK(get_my_v3_authority_cert, get_my_v3_authority_cert_m); + MOCK(connection_write_to_buf_impl_, connection_write_to_buf_mock); + + conn = dir_connection_new(tor_addr_family(&MOCK_TOR_ADDR)); + tt_int_op(0, OP_EQ, directory_handle_command_get(conn, + GET("/tor/status-vote/current/authority"), NULL, 0)); + + fetch_from_buf_http(TO_CONN(conn)->outbuf, &header, MAX_HEADERS_SIZE, + &body, &body_used, strlen(VOTE_BODY_V3)+1, 0); + + tt_assert(header); + tt_ptr_op(strstr(header, "HTTP/1.0 200 OK\r\n"), OP_EQ, header); + tt_assert(strstr(header, "Content-Type: text/plain\r\n")); + tt_assert(strstr(header, "Content-Encoding: identity\r\n")); + tt_assert(strstr(header, "Content-Length: 4135\r\n")); + + tt_str_op(VOTE_BODY_V3, OP_EQ, body); + + done: + UNMOCK(connection_write_to_buf_impl_); + UNMOCK(get_my_v3_authority_cert); + connection_free_(TO_CONN(conn)); + tor_free(header); + tor_free(body); + authority_cert_free(mock_cert); mock_cert = NULL; + or_options_free(mock_options); mock_options = NULL; + + clear_dir_servers(); + routerlist_free_all(); + dirvote_free_all(); +} + +#define DIR_HANDLE_CMD(name,flags) \ + { #name, test_dir_handle_get_##name, (flags), NULL, NULL } + +struct testcase_t dir_handle_get_tests[] = { + DIR_HANDLE_CMD(not_found, 0), + DIR_HANDLE_CMD(bad_request, 0), + DIR_HANDLE_CMD(v1_command_not_found, 0), + DIR_HANDLE_CMD(v1_command, 0), + DIR_HANDLE_CMD(robots_txt, 0), + DIR_HANDLE_CMD(bytes_txt, 0), + DIR_HANDLE_CMD(rendezvous2_not_found_if_not_encrypted, 0), + DIR_HANDLE_CMD(rendezvous2_not_found, 0), + DIR_HANDLE_CMD(rendezvous2_on_encrypted_conn_with_invalid_desc_id, 0), + DIR_HANDLE_CMD(rendezvous2_on_encrypted_conn_not_well_formed, 0), + DIR_HANDLE_CMD(rendezvous2_on_encrypted_conn_success, 0), + DIR_HANDLE_CMD(micro_d_not_found, 0), + DIR_HANDLE_CMD(micro_d_server_busy, 0), + DIR_HANDLE_CMD(micro_d, 0), + DIR_HANDLE_CMD(networkstatus_bridges_not_found_without_auth, 0), + DIR_HANDLE_CMD(networkstatus_bridges_not_found_wrong_auth, 0), + DIR_HANDLE_CMD(networkstatus_bridges, 0), + DIR_HANDLE_CMD(server_descriptors_not_found, 0), + DIR_HANDLE_CMD(server_descriptors_busy, TT_FORK), + DIR_HANDLE_CMD(server_descriptors_all, TT_FORK), + DIR_HANDLE_CMD(server_descriptors_authority, TT_FORK), + DIR_HANDLE_CMD(server_descriptors_fp, TT_FORK), + DIR_HANDLE_CMD(server_descriptors_d, TT_FORK), + DIR_HANDLE_CMD(server_keys_bad_req, 0), + DIR_HANDLE_CMD(server_keys_busy, 0), + DIR_HANDLE_CMD(server_keys_all_not_found, 0), + DIR_HANDLE_CMD(server_keys_all, 0), + DIR_HANDLE_CMD(server_keys_authority_not_found, 0), + DIR_HANDLE_CMD(server_keys_authority, 0), + DIR_HANDLE_CMD(server_keys_fp_not_found, 0), + DIR_HANDLE_CMD(server_keys_fp, 0), + DIR_HANDLE_CMD(server_keys_sk_not_found, 0), + DIR_HANDLE_CMD(server_keys_sk, 0), + DIR_HANDLE_CMD(server_keys_fpsk_not_found, 0), + DIR_HANDLE_CMD(server_keys_fpsk, 0), + DIR_HANDLE_CMD(status_vote_current_not_found, 0), + DIR_HANDLE_CMD(status_vote_next_not_found, 0), + DIR_HANDLE_CMD(status_vote_current_authority_not_found, 0), + DIR_HANDLE_CMD(status_vote_current_authority, 0), + DIR_HANDLE_CMD(status_vote_next_authority_not_found, 0), + DIR_HANDLE_CMD(status_vote_next_authority, 0), + DIR_HANDLE_CMD(status_vote_current_consensus_ns_not_enough_sigs, 0), + DIR_HANDLE_CMD(status_vote_current_consensus_ns_not_found, 0), + DIR_HANDLE_CMD(status_vote_current_consensus_ns_busy, 0), + DIR_HANDLE_CMD(status_vote_current_consensus_ns, 0), + DIR_HANDLE_CMD(status_vote_current_d_not_found, 0), + DIR_HANDLE_CMD(status_vote_next_d_not_found, 0), + DIR_HANDLE_CMD(status_vote_d, 0), + DIR_HANDLE_CMD(status_vote_next_consensus_not_found, 0), + DIR_HANDLE_CMD(status_vote_next_consensus_busy, 0), + DIR_HANDLE_CMD(status_vote_next_consensus, 0), + DIR_HANDLE_CMD(status_vote_next_consensus_signatures_not_found, 0), + DIR_HANDLE_CMD(status_vote_next_consensus_signatures_busy, 0), + DIR_HANDLE_CMD(status_vote_next_consensus_signatures, 0), + END_OF_TESTCASES +}; + diff --git a/src/test/test_dns.c b/src/test/test_dns.c index ad81914ccb..b40a4825a6 100644 --- a/src/test/test_dns.c +++ b/src/test/test_dns.c @@ -5,9 +5,14 @@ #include "dns.h" #include "connection.h" +#include "router.h" + +#define NS_MODULE dns + +#define NS_SUBMODULE clip_ttl static void -test_dns_clip_ttl(void *arg) +NS(test_main)(void *arg) { (void)arg; @@ -21,8 +26,12 @@ test_dns_clip_ttl(void *arg) return; } +#undef NS_SUBMODULE + +#define NS_SUBMODULE expiry_ttl + static void -test_dns_expiry_ttl(void *arg) +NS(test_main)(void *arg) { (void)arg; @@ -36,6 +45,10 @@ test_dns_expiry_ttl(void *arg) return; } +#undef NS_SUBMODULE + +#define NS_SUBMODULE resolve + static int resolve_retval = 0; static int resolve_made_conn_pending = 0; static char *resolved_name = NULL; @@ -43,6 +56,11 @@ static cached_resolve_t *cache_entry = NULL; static int n_fake_impl = 0; +NS_DECL(int, dns_resolve_impl, (edge_connection_t *exitconn, int is_resolve, + or_circuit_t *oncirc, char **hostname_out, + int *made_connection_pending_out, + cached_resolve_t **resolve_out)); + /** This will be our configurable substitute for <b>dns_resolve_impl</b> in * dns.c. It will return <b>resolve_retval</b>, * and set <b>resolve_made_conn_pending</b> to @@ -52,10 +70,10 @@ static int n_fake_impl = 0; * 1. */ static int -dns_resolve_fake_impl(edge_connection_t *exitconn, int is_resolve, - or_circuit_t *oncirc, char **hostname_out, - int *made_connection_pending_out, - cached_resolve_t **resolve_out) +NS(dns_resolve_impl)(edge_connection_t *exitconn, int is_resolve, + or_circuit_t *oncirc, char **hostname_out, + int *made_connection_pending_out, + cached_resolve_t **resolve_out) { (void)oncirc; (void)exitconn; @@ -82,8 +100,8 @@ static uint8_t last_answer_type = 0; static cached_resolve_t *last_resolved; static void -send_resolved_cell_replacement(edge_connection_t *conn, uint8_t answer_type, - const cached_resolve_t *resolved) +NS(send_resolved_cell)(edge_connection_t *conn, uint8_t answer_type, + const cached_resolve_t *resolved) { conn_for_resolved_cell = conn; @@ -98,8 +116,8 @@ static int n_send_resolved_hostname_cell_replacement = 0; static char *last_resolved_hostname = NULL; static void -send_resolved_hostname_cell_replacement(edge_connection_t *conn, - const char *hostname) +NS(send_resolved_hostname_cell)(edge_connection_t *conn, + const char *hostname) { conn_for_resolved_cell = conn; @@ -112,7 +130,7 @@ send_resolved_hostname_cell_replacement(edge_connection_t *conn, static int n_dns_cancel_pending_resolve_replacement = 0; static void -dns_cancel_pending_resolve_replacement(const char *address) +NS(dns_cancel_pending_resolve)(const char *address) { (void) address; n_dns_cancel_pending_resolve_replacement++; @@ -122,7 +140,7 @@ static int n_connection_free = 0; static connection_t *last_freed_conn = NULL; static void -connection_free_replacement(connection_t *conn) +NS(connection_free)(connection_t *conn) { n_connection_free++; @@ -130,7 +148,7 @@ connection_free_replacement(connection_t *conn) } static void -test_dns_resolve_outer(void *arg) +NS(test_main)(void *arg) { (void) arg; int retval; @@ -149,9 +167,9 @@ test_dns_resolve_outer(void *arg) memset(exitconn,0,sizeof(edge_connection_t)); memset(nextconn,0,sizeof(edge_connection_t)); - MOCK(dns_resolve_impl,dns_resolve_fake_impl); - MOCK(send_resolved_cell,send_resolved_cell_replacement); - MOCK(send_resolved_hostname_cell,send_resolved_hostname_cell_replacement); + NS_MOCK(dns_resolve_impl); + NS_MOCK(send_resolved_cell); + NS_MOCK(send_resolved_hostname_cell); /* * CASE 1: dns_resolve_impl returns 1 and sets a hostname. purpose is @@ -264,8 +282,8 @@ test_dns_resolve_outer(void *arg) * on exitconn with type being RESOLVED_TYPE_ERROR. */ - MOCK(dns_cancel_pending_resolve,dns_cancel_pending_resolve_replacement); - MOCK(connection_free,connection_free_replacement); + NS_MOCK(dns_cancel_pending_resolve); + NS_MOCK(connection_free); exitconn->on_circuit = &(on_circuit->base_); exitconn->base_.purpose = EXIT_PURPOSE_RESOLVE; @@ -288,11 +306,11 @@ test_dns_resolve_outer(void *arg) tt_assert(last_freed_conn == TO_CONN(exitconn)); done: - UNMOCK(dns_resolve_impl); - UNMOCK(send_resolved_cell); - UNMOCK(send_resolved_hostname_cell); - UNMOCK(dns_cancel_pending_resolve); - UNMOCK(connection_free); + NS_UNMOCK(dns_resolve_impl); + NS_UNMOCK(send_resolved_cell); + NS_UNMOCK(send_resolved_hostname_cell); + NS_UNMOCK(dns_cancel_pending_resolve); + NS_UNMOCK(connection_free); tor_free(on_circuit); tor_free(exitconn); tor_free(nextconn); @@ -302,10 +320,443 @@ test_dns_resolve_outer(void *arg) return; } +#undef NS_SUBMODULE + +/** Create an <b>edge_connection_t</b> instance that is considered a + * valid exit connection by asserts in dns_resolve_impl. + */ +static edge_connection_t * +create_valid_exitconn(void) +{ + edge_connection_t *exitconn = tor_malloc_zero(sizeof(edge_connection_t)); + TO_CONN(exitconn)->type = CONN_TYPE_EXIT; + TO_CONN(exitconn)->magic = EDGE_CONNECTION_MAGIC; + TO_CONN(exitconn)->purpose = EXIT_PURPOSE_RESOLVE; + TO_CONN(exitconn)->state = EXIT_CONN_STATE_RESOLVING; + exitconn->base_.s = TOR_INVALID_SOCKET; + + return exitconn; +} + +#define NS_SUBMODULE ASPECT(resolve_impl, addr_is_ip_no_need_to_resolve) + +/* + * Given that <b>exitconn->base_.address</b> is IP address string, we + * want dns_resolve_impl() to parse it and store in + * <b>exitconn->base_.addr</b>. We expect dns_resolve_impl to return 1. + * Lastly, we want it to set the TTL value to default one for DNS queries. + */ + +static void +NS(test_main)(void *arg) +{ + int retval; + int made_pending; + const tor_addr_t *resolved_addr; + tor_addr_t addr_to_compare; + + (void)arg; + + tor_addr_parse(&addr_to_compare, "8.8.8.8"); + + or_circuit_t *on_circ = tor_malloc_zero(sizeof(or_circuit_t)); + + edge_connection_t *exitconn = create_valid_exitconn(); + + TO_CONN(exitconn)->address = tor_strdup("8.8.8.8"); + + retval = dns_resolve_impl(exitconn, 1, on_circ, NULL, &made_pending, + NULL); + + resolved_addr = &(exitconn->base_.addr); + + tt_int_op(retval,==,1); + tt_assert(tor_addr_eq(resolved_addr, (const tor_addr_t *)&addr_to_compare)); + tt_int_op(exitconn->address_ttl,==,DEFAULT_DNS_TTL); + + done: + tor_free(on_circ); + tor_free(TO_CONN(exitconn)->address); + tor_free(exitconn); + return; +} + +#undef NS_SUBMODULE + +#define NS_SUBMODULE ASPECT(resolve_impl, non_exit) + +/** Given that Tor instance is not configured as an exit node, we want + * dns_resolve_impl() to fail with return value -1. + */ +static int +NS(router_my_exit_policy_is_reject_star)(void) +{ + return 1; +} + +static void +NS(test_main)(void *arg) +{ + int retval; + int made_pending; + + edge_connection_t *exitconn = create_valid_exitconn(); + or_circuit_t *on_circ = tor_malloc_zero(sizeof(or_circuit_t)); + + (void)arg; + + TO_CONN(exitconn)->address = tor_strdup("torproject.org"); + + NS_MOCK(router_my_exit_policy_is_reject_star); + + retval = dns_resolve_impl(exitconn, 1, on_circ, NULL, &made_pending, + NULL); + + tt_int_op(retval,==,-1); + + done: + tor_free(TO_CONN(exitconn)->address); + tor_free(exitconn); + tor_free(on_circ); + NS_UNMOCK(router_my_exit_policy_is_reject_star); + return; +} + +#undef NS_SUBMODULE + +#define NS_SUBMODULE ASPECT(resolve_impl, addr_is_invalid_dest) + +/** Given that address is not a valid destination (as judged by + * address_is_invalid_destination() function), we want dns_resolve_impl() + * function to fail with return value -1. + */ + +static int +NS(router_my_exit_policy_is_reject_star)(void) +{ + return 0; +} + +static void +NS(test_main)(void *arg) +{ + int retval; + int made_pending; + + edge_connection_t *exitconn = create_valid_exitconn(); + or_circuit_t *on_circ = tor_malloc_zero(sizeof(or_circuit_t)); + + (void)arg; + + NS_MOCK(router_my_exit_policy_is_reject_star); + + TO_CONN(exitconn)->address = tor_strdup("invalid#@!.org"); + + retval = dns_resolve_impl(exitconn, 1, on_circ, NULL, &made_pending, + NULL); + + tt_int_op(retval,==,-1); + + done: + NS_UNMOCK(router_my_exit_policy_is_reject_star); + tor_free(TO_CONN(exitconn)->address); + tor_free(exitconn); + tor_free(on_circ); + return; +} + +#undef NS_SUBMODULE + +#define NS_SUBMODULE ASPECT(resolve_impl, malformed_ptr) + +/** Given that address is a malformed PTR name, we want dns_resolve_impl to + * fail. + */ + +static int +NS(router_my_exit_policy_is_reject_star)(void) +{ + return 0; +} + +static void +NS(test_main)(void *arg) +{ + int retval; + int made_pending; + + edge_connection_t *exitconn = create_valid_exitconn(); + or_circuit_t *on_circ = tor_malloc_zero(sizeof(or_circuit_t)); + + (void)arg; + + TO_CONN(exitconn)->address = tor_strdup("127.0.0.1.in-addr.arpa"); + + NS_MOCK(router_my_exit_policy_is_reject_star); + + retval = dns_resolve_impl(exitconn, 1, on_circ, NULL, &made_pending, + NULL); + + tt_int_op(retval,==,-1); + + tor_free(TO_CONN(exitconn)->address); + + TO_CONN(exitconn)->address = + tor_strdup("z01234567890123456789.in-addr.arpa"); + + retval = dns_resolve_impl(exitconn, 1, on_circ, NULL, &made_pending, + NULL); + + tt_int_op(retval,==,-1); + + done: + NS_UNMOCK(router_my_exit_policy_is_reject_star); + tor_free(TO_CONN(exitconn)->address); + tor_free(exitconn); + tor_free(on_circ); + return; +} + +#undef NS_SUBMODULE + +#define NS_SUBMODULE ASPECT(resolve_impl, cache_hit_pending) + +/* Given that there is already a pending resolve for the given address, + * we want dns_resolve_impl to append our exit connection to list + * of pending connections for the pending DNS request and return 0. + */ + +static int +NS(router_my_exit_policy_is_reject_star)(void) +{ + return 0; +} + +static void +NS(test_main)(void *arg) +{ + int retval; + int made_pending = 0; + + pending_connection_t *pending_conn = NULL; + + edge_connection_t *exitconn = create_valid_exitconn(); + or_circuit_t *on_circ = tor_malloc_zero(sizeof(or_circuit_t)); + + cached_resolve_t *cache_entry = tor_malloc_zero(sizeof(cached_resolve_t)); + cache_entry->magic = CACHED_RESOLVE_MAGIC; + cache_entry->state = CACHE_STATE_PENDING; + cache_entry->minheap_idx = -1; + cache_entry->expire = time(NULL) + 60 * 60; + + (void)arg; + + TO_CONN(exitconn)->address = tor_strdup("torproject.org"); + + strlcpy(cache_entry->address, TO_CONN(exitconn)->address, + sizeof(cache_entry->address)); + + NS_MOCK(router_my_exit_policy_is_reject_star); + + dns_init(); + + dns_insert_cache_entry(cache_entry); + + retval = dns_resolve_impl(exitconn, 1, on_circ, NULL, &made_pending, + NULL); + + tt_int_op(retval,==,0); + tt_int_op(made_pending,==,1); + + pending_conn = cache_entry->pending_connections; + + tt_assert(pending_conn != NULL); + tt_assert(pending_conn->conn == exitconn); + + done: + NS_UNMOCK(router_my_exit_policy_is_reject_star); + tor_free(on_circ); + tor_free(TO_CONN(exitconn)->address); + tor_free(cache_entry->pending_connections); + tor_free(cache_entry); + return; +} + +#undef NS_SUBMODULE + +#define NS_SUBMODULE ASPECT(resolve_impl, cache_hit_cached) + +/* Given that a finished DNS resolve is available in our cache, we want + * dns_resolve_impl() return it to called via resolve_out and pass the + * handling to set_exitconn_info_from_resolve function. + */ +static int +NS(router_my_exit_policy_is_reject_star)(void) +{ + return 0; +} + +static edge_connection_t *last_exitconn = NULL; +static cached_resolve_t *last_resolve = NULL; + +static int +NS(set_exitconn_info_from_resolve)(edge_connection_t *exitconn, + const cached_resolve_t *resolve, + char **hostname_out) +{ + last_exitconn = exitconn; + last_resolve = (cached_resolve_t *)resolve; + + (void)hostname_out; + + return 0; +} + +static void +NS(test_main)(void *arg) +{ + int retval; + int made_pending = 0; + + edge_connection_t *exitconn = create_valid_exitconn(); + or_circuit_t *on_circ = tor_malloc_zero(sizeof(or_circuit_t)); + + cached_resolve_t *resolve_out = NULL; + + cached_resolve_t *cache_entry = tor_malloc_zero(sizeof(cached_resolve_t)); + cache_entry->magic = CACHED_RESOLVE_MAGIC; + cache_entry->state = CACHE_STATE_CACHED; + cache_entry->minheap_idx = -1; + cache_entry->expire = time(NULL) + 60 * 60; + + (void)arg; + + TO_CONN(exitconn)->address = tor_strdup("torproject.org"); + + strlcpy(cache_entry->address, TO_CONN(exitconn)->address, + sizeof(cache_entry->address)); + + NS_MOCK(router_my_exit_policy_is_reject_star); + NS_MOCK(set_exitconn_info_from_resolve); + + dns_init(); + + dns_insert_cache_entry(cache_entry); + + retval = dns_resolve_impl(exitconn, 1, on_circ, NULL, &made_pending, + &resolve_out); + + tt_int_op(retval,==,0); + tt_int_op(made_pending,==,0); + tt_assert(resolve_out == cache_entry); + + tt_assert(last_exitconn == exitconn); + tt_assert(last_resolve == cache_entry); + + done: + NS_UNMOCK(router_my_exit_policy_is_reject_star); + NS_UNMOCK(set_exitconn_info_from_resolve); + tor_free(on_circ); + tor_free(TO_CONN(exitconn)->address); + tor_free(cache_entry->pending_connections); + tor_free(cache_entry); + return; +} + +#undef NS_SUBMODULE + +#define NS_SUBMODULE ASPECT(resolve_impl, cache_miss) + +/* Given that there are neither pending nor pre-cached resolve for a given + * address, we want dns_resolve_impl() to create a new cached_resolve_t + * object, mark it as pending, insert it into the cache, attach the exit + * connection to list of pending connections and call launch_resolve() + * with the cached_resolve_t object it created. + */ +static int +NS(router_my_exit_policy_is_reject_star)(void) +{ + return 0; +} + +static cached_resolve_t *last_launched_resolve = NULL; + +static int +NS(launch_resolve)(cached_resolve_t *resolve) +{ + last_launched_resolve = resolve; + + return 0; +} + +static void +NS(test_main)(void *arg) +{ + int retval; + int made_pending = 0; + + pending_connection_t *pending_conn = NULL; + + edge_connection_t *exitconn = create_valid_exitconn(); + or_circuit_t *on_circ = tor_malloc_zero(sizeof(or_circuit_t)); + + cached_resolve_t *cache_entry = NULL; + cached_resolve_t query; + + (void)arg; + + TO_CONN(exitconn)->address = tor_strdup("torproject.org"); + + strlcpy(query.address, TO_CONN(exitconn)->address, sizeof(query.address)); + + NS_MOCK(router_my_exit_policy_is_reject_star); + NS_MOCK(launch_resolve); + + dns_init(); + + retval = dns_resolve_impl(exitconn, 1, on_circ, NULL, &made_pending, + NULL); + + tt_int_op(retval,==,0); + tt_int_op(made_pending,==,1); + + cache_entry = dns_get_cache_entry(&query); + + tt_assert(cache_entry); + + pending_conn = cache_entry->pending_connections; + + tt_assert(pending_conn != NULL); + tt_assert(pending_conn->conn == exitconn); + + tt_assert(last_launched_resolve == cache_entry); + tt_str_op(cache_entry->address,==,TO_CONN(exitconn)->address); + + done: + NS_UNMOCK(router_my_exit_policy_is_reject_star); + NS_UNMOCK(launch_resolve); + tor_free(on_circ); + tor_free(TO_CONN(exitconn)->address); + tor_free(cache_entry->pending_connections); + tor_free(cache_entry); + return; +} + +#undef NS_SUBMODULE + struct testcase_t dns_tests[] = { - { "clip_ttl", test_dns_clip_ttl, 0, NULL, NULL }, - { "expiry_ttl", test_dns_expiry_ttl, 0, NULL, NULL }, - { "resolve_outer", test_dns_resolve_outer, TT_FORK, NULL, NULL }, + TEST_CASE(clip_ttl), + TEST_CASE(expiry_ttl), + TEST_CASE(resolve), + TEST_CASE_ASPECT(resolve_impl, addr_is_ip_no_need_to_resolve), + TEST_CASE_ASPECT(resolve_impl, non_exit), + TEST_CASE_ASPECT(resolve_impl, addr_is_invalid_dest), + TEST_CASE_ASPECT(resolve_impl, malformed_ptr), + TEST_CASE_ASPECT(resolve_impl, cache_hit_pending), + TEST_CASE_ASPECT(resolve_impl, cache_hit_cached), + TEST_CASE_ASPECT(resolve_impl, cache_miss), END_OF_TESTCASES }; +#undef NS_MODULE + diff --git a/src/test/test_extorport.c b/src/test/test_extorport.c index 2e5a32eef3..5d38ed8fa2 100644 --- a/src/test/test_extorport.c +++ b/src/test/test_extorport.c @@ -309,15 +309,14 @@ test_ext_or_cookie_auth(void *arg) tor_free(client_hash2); } -static int +static void crypto_rand_return_tse_str(char *to, size_t n) { if (n != 32) { TT_FAIL(("Asked for %d bytes, not 32", (int)n)); - return -1; + return; } memcpy(to, "te road There is always another ", 32); - return 0; } static void diff --git a/src/test/test_ntor_cl.c b/src/test/test_ntor_cl.c index bfbf13a476..915a5d0a7c 100644 --- a/src/test/test_ntor_cl.c +++ b/src/test/test_ntor_cl.c @@ -106,6 +106,7 @@ server1(int argc, char **argv) done: tor_free(keys); tor_free(hexkeys); + dimap_free(keymap, NULL); return result; } diff --git a/src/test/test_policy.c b/src/test/test_policy.c index 37c36fed99..b7d89a1445 100644 --- a/src/test/test_policy.c +++ b/src/test/test_policy.c @@ -2,8 +2,11 @@ /* See LICENSE for licensing information */ #include "or.h" +#define CONFIG_PRIVATE +#include "config.h" #include "router.h" #include "routerparse.h" +#define POLICIES_PRIVATE #include "policies.h" #include "test.h" @@ -49,7 +52,7 @@ test_policy_summary_helper(const char *policy_str, r = policies_parse_exit_policy(&line, &policy, EXIT_POLICY_IPV6_ENABLED | - EXIT_POLICY_ADD_DEFAULT, 0, NULL, 0); + EXIT_POLICY_ADD_DEFAULT, NULL); tt_int_op(r,OP_EQ, 0); summary = policy_summarize(policy, AF_INET); @@ -80,7 +83,8 @@ test_policies_general(void *arg) *policy7 = NULL, *policy8 = NULL, *policy9 = NULL, *policy10 = NULL, *policy11 = NULL, *policy12 = NULL; addr_policy_t *p; - tor_addr_t tar; + tor_addr_t tar, tar2; + smartlist_t *addr_list = NULL; config_line_t line; smartlist_t *sm = NULL; char *policy_str = NULL; @@ -115,17 +119,22 @@ test_policies_general(void *arg) tt_int_op(0, OP_EQ, policies_parse_exit_policy(NULL, &policy2, EXIT_POLICY_IPV6_ENABLED | EXIT_POLICY_REJECT_PRIVATE | - EXIT_POLICY_ADD_DEFAULT, 0, - NULL, 0)); + EXIT_POLICY_ADD_DEFAULT, NULL)); tt_assert(policy2); - tor_addr_parse(&tar, "[2000::1234]"); + tor_addr_from_ipv4h(&tar, 0x0306090cu); + tor_addr_parse(&tar2, "[2000::1234]"); + addr_list = smartlist_new(); + smartlist_add(addr_list, &tar); + smartlist_add(addr_list, &tar2); tt_int_op(0, OP_EQ, policies_parse_exit_policy(NULL, &policy12, EXIT_POLICY_IPV6_ENABLED | EXIT_POLICY_REJECT_PRIVATE | EXIT_POLICY_ADD_DEFAULT, - 0x0306090cu, &tar, 1)); + addr_list)); + smartlist_free(addr_list); + addr_list = NULL; tt_assert(policy12); @@ -206,15 +215,15 @@ test_policies_general(void *arg) tt_int_op(0, OP_EQ, policies_parse_exit_policy(NULL, &policy8, EXIT_POLICY_IPV6_ENABLED | EXIT_POLICY_REJECT_PRIVATE | - EXIT_POLICY_ADD_DEFAULT, 0, - NULL, 0)); + EXIT_POLICY_ADD_DEFAULT, + NULL)); tt_assert(policy8); tt_int_op(0, OP_EQ, policies_parse_exit_policy(NULL, &policy9, EXIT_POLICY_REJECT_PRIVATE | - EXIT_POLICY_ADD_DEFAULT, 0, - NULL, 0)); + EXIT_POLICY_ADD_DEFAULT, + NULL)); tt_assert(policy9); @@ -268,8 +277,7 @@ test_policies_general(void *arg) line.next = NULL; tt_int_op(0, OP_EQ, policies_parse_exit_policy(&line,&policy, EXIT_POLICY_IPV6_ENABLED | - EXIT_POLICY_ADD_DEFAULT, 0, - NULL, 0)); + EXIT_POLICY_ADD_DEFAULT, NULL)); tt_assert(policy); //test_streq(policy->string, "accept *:80"); @@ -489,6 +497,324 @@ test_policies_general(void *arg) short_policy_free(short_parsed); } +/** Helper: Check that policy_list contains address */ +static int +test_policy_has_address_helper(const smartlist_t *policy_list, + const tor_addr_t *addr) +{ + int found = 0; + + tt_assert(policy_list); + tt_assert(addr); + + SMARTLIST_FOREACH_BEGIN(policy_list, addr_policy_t*, p) { + if (tor_addr_eq(&p->addr, addr)) { + found = 1; + } + } SMARTLIST_FOREACH_END(p); + + return found; + + done: + return 0; +} + +#define TEST_IPV4_ADDR (0x01020304) +#define TEST_IPV6_ADDR ("2002::abcd") + +/** Run unit tests for rejecting the configured addresses on this exit relay + * using policies_parse_exit_policy_reject_private */ +static void +test_policies_reject_exit_address(void *arg) +{ + smartlist_t *policy = NULL; + tor_addr_t ipv4_addr, ipv6_addr; + smartlist_t *ipv4_list, *ipv6_list, *both_list, *dupl_list; + (void)arg; + + tor_addr_from_ipv4h(&ipv4_addr, TEST_IPV4_ADDR); + tor_addr_parse(&ipv6_addr, TEST_IPV6_ADDR); + + ipv4_list = smartlist_new(); + ipv6_list = smartlist_new(); + both_list = smartlist_new(); + dupl_list = smartlist_new(); + + smartlist_add(ipv4_list, &ipv4_addr); + smartlist_add(both_list, &ipv4_addr); + smartlist_add(dupl_list, &ipv4_addr); + smartlist_add(dupl_list, &ipv4_addr); + smartlist_add(dupl_list, &ipv4_addr); + + smartlist_add(ipv6_list, &ipv6_addr); + smartlist_add(both_list, &ipv6_addr); + smartlist_add(dupl_list, &ipv6_addr); + smartlist_add(dupl_list, &ipv6_addr); + + /* IPv4-Only Exits */ + + /* test that IPv4 addresses are rejected on an IPv4-only exit */ + policies_parse_exit_policy_reject_private(&policy, 0, ipv4_list, 0, 0); + tt_assert(policy); + tt_assert(smartlist_len(policy) == 1); + tt_assert(test_policy_has_address_helper(policy, &ipv4_addr)); + addr_policy_list_free(policy); + policy = NULL; + + /* test that IPv6 addresses are NOT rejected on an IPv4-only exit + * (all IPv6 addresses are rejected by policies_parse_exit_policy_internal + * on IPv4-only exits, so policies_parse_exit_policy_reject_private doesn't + * need to do anything) */ + policies_parse_exit_policy_reject_private(&policy, 0, ipv6_list, 0, 0); + tt_assert(policy == NULL); + + /* test that only IPv4 addresses are rejected on an IPv4-only exit */ + policies_parse_exit_policy_reject_private(&policy, 0, both_list, 0, 0); + tt_assert(policy); + tt_assert(smartlist_len(policy) == 1); + tt_assert(test_policy_has_address_helper(policy, &ipv4_addr)); + addr_policy_list_free(policy); + policy = NULL; + + /* Test that lists with duplicate entries produce the same results */ + policies_parse_exit_policy_reject_private(&policy, 0, dupl_list, 0, 0); + tt_assert(policy); + tt_assert(smartlist_len(policy) == 1); + tt_assert(test_policy_has_address_helper(policy, &ipv4_addr)); + addr_policy_list_free(policy); + policy = NULL; + + /* IPv4/IPv6 Exits */ + + /* test that IPv4 addresses are rejected on an IPv4/IPv6 exit */ + policies_parse_exit_policy_reject_private(&policy, 1, ipv4_list, 0, 0); + tt_assert(policy); + tt_assert(smartlist_len(policy) == 1); + tt_assert(test_policy_has_address_helper(policy, &ipv4_addr)); + addr_policy_list_free(policy); + policy = NULL; + + /* test that IPv6 addresses are rejected on an IPv4/IPv6 exit */ + policies_parse_exit_policy_reject_private(&policy, 1, ipv6_list, 0, 0); + tt_assert(policy); + tt_assert(smartlist_len(policy) == 1); + tt_assert(test_policy_has_address_helper(policy, &ipv6_addr)); + addr_policy_list_free(policy); + policy = NULL; + + /* test that IPv4 and IPv6 addresses are rejected on an IPv4/IPv6 exit */ + policies_parse_exit_policy_reject_private(&policy, 1, both_list, 0, 0); + tt_assert(policy); + tt_assert(smartlist_len(policy) == 2); + tt_assert(test_policy_has_address_helper(policy, &ipv4_addr)); + tt_assert(test_policy_has_address_helper(policy, &ipv6_addr)); + addr_policy_list_free(policy); + policy = NULL; + + /* Test that lists with duplicate entries produce the same results */ + policies_parse_exit_policy_reject_private(&policy, 1, dupl_list, 0, 0); + tt_assert(policy); + tt_assert(smartlist_len(policy) == 2); + tt_assert(test_policy_has_address_helper(policy, &ipv4_addr)); + tt_assert(test_policy_has_address_helper(policy, &ipv6_addr)); + addr_policy_list_free(policy); + policy = NULL; + + done: + addr_policy_list_free(policy); + smartlist_free(ipv4_list); + smartlist_free(ipv6_list); + smartlist_free(both_list); + smartlist_free(dupl_list); +} + +static smartlist_t *test_configured_ports = NULL; +const smartlist_t *mock_get_configured_ports(void); + +/** Returns test_configured_ports */ +const smartlist_t * +mock_get_configured_ports(void) +{ + return test_configured_ports; +} + +/** Run unit tests for rejecting publicly routable configured port addresses + * on this exit relay using policies_parse_exit_policy_reject_private */ +static void +test_policies_reject_port_address(void *arg) +{ + smartlist_t *policy = NULL; + port_cfg_t *ipv4_port = NULL; + port_cfg_t *ipv6_port = NULL; + (void)arg; + + test_configured_ports = smartlist_new(); + + ipv4_port = port_cfg_new(0); + tor_addr_from_ipv4h(&ipv4_port->addr, TEST_IPV4_ADDR); + smartlist_add(test_configured_ports, ipv4_port); + + ipv6_port = port_cfg_new(0); + tor_addr_parse(&ipv6_port->addr, TEST_IPV6_ADDR); + smartlist_add(test_configured_ports, ipv6_port); + + MOCK(get_configured_ports, mock_get_configured_ports); + + /* test that an IPv4 port is rejected on an IPv4-only exit, but an IPv6 port + * is NOT rejected (all IPv6 addresses are rejected by + * policies_parse_exit_policy_internal on IPv4-only exits, so + * policies_parse_exit_policy_reject_private doesn't need to do anything + * with IPv6 addresses on IPv4-only exits) */ + policies_parse_exit_policy_reject_private(&policy, 0, NULL, 0, 1); + tt_assert(policy); + tt_assert(smartlist_len(policy) == 1); + tt_assert(test_policy_has_address_helper(policy, &ipv4_port->addr)); + addr_policy_list_free(policy); + policy = NULL; + + /* test that IPv4 and IPv6 ports are rejected on an IPv4/IPv6 exit */ + policies_parse_exit_policy_reject_private(&policy, 1, NULL, 0, 1); + tt_assert(policy); + tt_assert(smartlist_len(policy) == 2); + tt_assert(test_policy_has_address_helper(policy, &ipv4_port->addr)); + tt_assert(test_policy_has_address_helper(policy, &ipv6_port->addr)); + addr_policy_list_free(policy); + policy = NULL; + + done: + addr_policy_list_free(policy); + if (test_configured_ports) { + SMARTLIST_FOREACH(test_configured_ports, + port_cfg_t *, p, port_cfg_free(p)); + smartlist_free(test_configured_ports); + test_configured_ports = NULL; + } + UNMOCK(get_configured_ports); +} + +smartlist_t *mock_ipv4_addrs = NULL; +smartlist_t *mock_ipv6_addrs = NULL; + +/* mock get_interface_address6_list, returning a deep copy of the template + * address list ipv4_interface_address_list or ipv6_interface_address_list */ +static smartlist_t * +mock_get_interface_address6_list(int severity, + sa_family_t family, + int include_internal) +{ + (void)severity; + (void)include_internal; + smartlist_t *clone_list = smartlist_new(); + smartlist_t *template_list = NULL; + + if (family == AF_INET) { + template_list = mock_ipv4_addrs; + } else if (family == AF_INET6) { + template_list = mock_ipv6_addrs; + } else { + return NULL; + } + + tt_assert(template_list); + + SMARTLIST_FOREACH_BEGIN(template_list, tor_addr_t *, src_addr) { + tor_addr_t *dest_addr = malloc(sizeof(tor_addr_t)); + memset(dest_addr, 0, sizeof(*dest_addr)); + tor_addr_copy_tight(dest_addr, src_addr); + smartlist_add(clone_list, dest_addr); + } SMARTLIST_FOREACH_END(src_addr); + + return clone_list; + + done: + free_interface_address6_list(clone_list); + return NULL; +} + +/** Run unit tests for rejecting publicly routable interface addresses on this + * exit relay using policies_parse_exit_policy_reject_private */ +static void +test_policies_reject_interface_address(void *arg) +{ + smartlist_t *policy = NULL; + smartlist_t *public_ipv4_addrs = + get_interface_address6_list(LOG_INFO, AF_INET, 0); + smartlist_t *public_ipv6_addrs = + get_interface_address6_list(LOG_INFO, AF_INET6, 0); + tor_addr_t ipv4_addr, ipv6_addr; + (void)arg; + + /* test that no addresses are rejected when none are supplied/requested */ + policies_parse_exit_policy_reject_private(&policy, 0, NULL, 0, 0); + tt_assert(policy == NULL); + + /* test that only IPv4 interface addresses are rejected on an IPv4-only exit + * (and allow for duplicates) + */ + policies_parse_exit_policy_reject_private(&policy, 0, NULL, 1, 0); + if (policy) { + tt_assert(smartlist_len(policy) <= smartlist_len(public_ipv4_addrs)); + addr_policy_list_free(policy); + policy = NULL; + } + + /* test that IPv4 and IPv6 interface addresses are rejected on an IPv4/IPv6 + * exit (and allow for duplicates) */ + policies_parse_exit_policy_reject_private(&policy, 1, NULL, 1, 0); + if (policy) { + tt_assert(smartlist_len(policy) <= (smartlist_len(public_ipv4_addrs) + + smartlist_len(public_ipv6_addrs))); + addr_policy_list_free(policy); + policy = NULL; + } + + /* Now do it all again, but mocked */ + tor_addr_from_ipv4h(&ipv4_addr, TEST_IPV4_ADDR); + mock_ipv4_addrs = smartlist_new(); + smartlist_add(mock_ipv4_addrs, (void *)&ipv4_addr); + + tor_addr_parse(&ipv6_addr, TEST_IPV6_ADDR); + mock_ipv6_addrs = smartlist_new(); + smartlist_add(mock_ipv6_addrs, (void *)&ipv6_addr); + + MOCK(get_interface_address6_list, mock_get_interface_address6_list); + + /* test that no addresses are rejected when none are supplied/requested */ + policies_parse_exit_policy_reject_private(&policy, 0, NULL, 0, 0); + tt_assert(policy == NULL); + + /* test that only IPv4 interface addresses are rejected on an IPv4-only exit + */ + policies_parse_exit_policy_reject_private(&policy, 0, NULL, 1, 0); + tt_assert(policy); + tt_assert(smartlist_len(policy) == smartlist_len(mock_ipv4_addrs)); + addr_policy_list_free(policy); + policy = NULL; + + /* test that IPv4 and IPv6 interface addresses are rejected on an IPv4/IPv6 + * exit */ + policies_parse_exit_policy_reject_private(&policy, 1, NULL, 1, 0); + tt_assert(policy); + tt_assert(smartlist_len(policy) == (smartlist_len(mock_ipv4_addrs) + + smartlist_len(mock_ipv6_addrs))); + addr_policy_list_free(policy); + policy = NULL; + + done: + addr_policy_list_free(policy); + free_interface_address6_list(public_ipv4_addrs); + free_interface_address6_list(public_ipv6_addrs); + + UNMOCK(get_interface_address6_list); + /* we don't use free_interface_address6_list on these lists because their + * address pointers are stack-based */ + smartlist_free(mock_ipv4_addrs); + smartlist_free(mock_ipv6_addrs); +} + +#undef TEST_IPV4_ADDR +#undef TEST_IPV6_ADDR + static void test_dump_exit_policy_to_string(void *arg) { @@ -578,10 +904,152 @@ test_dump_exit_policy_to_string(void *arg) tor_free(ep); } +static routerinfo_t *mock_desc_routerinfo = NULL; +static const routerinfo_t * +mock_router_get_my_routerinfo(void) +{ + return mock_desc_routerinfo; +} + +#define DEFAULT_POLICY_STRING "reject *:*" +#define TEST_IPV4_ADDR (0x02040608) +#define TEST_IPV6_ADDR ("2003::ef01") + +static or_options_t mock_options; + +static const or_options_t * +mock_get_options(void) +{ + return &mock_options; +} + +/** Run unit tests for generating summary lines of exit policies */ +static void +test_policies_getinfo_helper_policies(void *arg) +{ + (void)arg; + int rv = 0; + size_t ipv4_len = 0, ipv6_len = 0; + char *answer = NULL; + const char *errmsg = NULL; + routerinfo_t mock_my_routerinfo; + + rv = getinfo_helper_policies(NULL, "exit-policy/default", &answer, &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + tt_assert(strlen(answer) > 0); + tor_free(answer); + + rv = getinfo_helper_policies(NULL, "exit-policy/reject-private/default", + &answer, &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + tt_assert(strlen(answer) > 0); + tor_free(answer); + + memset(&mock_my_routerinfo, 0, sizeof(routerinfo_t)); + MOCK(router_get_my_routerinfo, mock_router_get_my_routerinfo); + mock_my_routerinfo.exit_policy = smartlist_new(); + mock_desc_routerinfo = &mock_my_routerinfo; + + memset(&mock_options, 0, sizeof(or_options_t)); + MOCK(get_options, mock_get_options); + + rv = getinfo_helper_policies(NULL, "exit-policy/reject-private/relay", + &answer, &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + tt_assert(strlen(answer) == 0); + tor_free(answer); + + rv = getinfo_helper_policies(NULL, "exit-policy/ipv4", &answer, + &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + ipv4_len = strlen(answer); + tt_assert(ipv4_len == 0 || ipv4_len == strlen(DEFAULT_POLICY_STRING)); + tt_assert(ipv4_len == 0 || !strcasecmp(answer, DEFAULT_POLICY_STRING)); + tor_free(answer); + + rv = getinfo_helper_policies(NULL, "exit-policy/ipv6", &answer, + &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + ipv6_len = strlen(answer); + tt_assert(ipv6_len == 0 || ipv6_len == strlen(DEFAULT_POLICY_STRING)); + tt_assert(ipv6_len == 0 || !strcasecmp(answer, DEFAULT_POLICY_STRING)); + tor_free(answer); + + rv = getinfo_helper_policies(NULL, "exit-policy/full", &answer, + &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + /* It's either empty or it's the default */ + tt_assert(strlen(answer) == 0 || !strcasecmp(answer, DEFAULT_POLICY_STRING)); + tor_free(answer); + + mock_my_routerinfo.addr = TEST_IPV4_ADDR; + tor_addr_parse(&mock_my_routerinfo.ipv6_addr, TEST_IPV6_ADDR); + append_exit_policy_string(&mock_my_routerinfo.exit_policy, "accept *4:*"); + append_exit_policy_string(&mock_my_routerinfo.exit_policy, "reject *6:*"); + + mock_options.IPv6Exit = 1; + mock_options.ExitPolicyRejectPrivate = 1; + tor_addr_from_ipv4h(&mock_options.OutboundBindAddressIPv4_, TEST_IPV4_ADDR); + tor_addr_parse(&mock_options.OutboundBindAddressIPv6_, TEST_IPV6_ADDR); + + rv = getinfo_helper_policies(NULL, "exit-policy/reject-private/relay", + &answer, &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + tt_assert(strlen(answer) > 0); + tor_free(answer); + + rv = getinfo_helper_policies(NULL, "exit-policy/ipv4", &answer, + &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + ipv4_len = strlen(answer); + tt_assert(ipv4_len > 0); + tor_free(answer); + + rv = getinfo_helper_policies(NULL, "exit-policy/ipv6", &answer, + &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + ipv6_len = strlen(answer); + tt_assert(ipv6_len > 0); + tor_free(answer); + + rv = getinfo_helper_policies(NULL, "exit-policy/full", &answer, + &errmsg); + tt_assert(rv == 0); + tt_assert(answer != NULL); + tt_assert(strlen(answer) > 0); + tt_assert(strlen(answer) == ipv4_len + ipv6_len + 1); + tor_free(answer); + + done: + tor_free(answer); + UNMOCK(get_options); + UNMOCK(router_get_my_routerinfo); + addr_policy_list_free(mock_my_routerinfo.exit_policy); +} + +#undef DEFAULT_POLICY_STRING +#undef TEST_IPV4_ADDR +#undef TEST_IPV6_ADDR + struct testcase_t policy_tests[] = { { "router_dump_exit_policy_to_string", test_dump_exit_policy_to_string, 0, NULL, NULL }, { "general", test_policies_general, 0, NULL, NULL }, + { "getinfo_helper_policies", test_policies_getinfo_helper_policies, 0, NULL, + NULL }, + { "reject_exit_address", test_policies_reject_exit_address, 0, NULL, NULL }, + { "reject_interface_address", test_policies_reject_interface_address, 0, + NULL, NULL }, + { "reject_port_address", test_policies_reject_port_address, 0, NULL, NULL }, END_OF_TESTCASES }; diff --git a/src/test/test_procmon.c b/src/test/test_procmon.c new file mode 100644 index 0000000000..2855178788 --- /dev/null +++ b/src/test/test_procmon.c @@ -0,0 +1,58 @@ +/* Copyright (c) 2010-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#define PROCMON_PRIVATE +#include "orconfig.h" +#include "or.h" +#include "test.h" + +#include "procmon.h" + +#include "log_test_helpers.h" + +#define NS_MODULE procmon + +struct event_base; + +static void +test_procmon_tor_process_monitor_new(void *ignored) +{ + (void)ignored; + tor_process_monitor_t *res; + const char *msg; + + res = tor_process_monitor_new(NULL, "probably invalid", 0, NULL, NULL, &msg); + tt_assert(!res); + tt_str_op(msg, OP_EQ, "invalid PID"); + + res = tor_process_monitor_new(NULL, "243443535345454", 0, NULL, NULL, &msg); + tt_assert(!res); + tt_str_op(msg, OP_EQ, "invalid PID"); + + res = tor_process_monitor_new(tor_libevent_get_base(), "43", 0, + NULL, NULL, &msg); + tt_assert(res); + tt_assert(!msg); + tor_process_monitor_free(res); + + res = tor_process_monitor_new(tor_libevent_get_base(), "44 hello", 0, + NULL, NULL, &msg); + tt_assert(res); + tt_assert(!msg); + tor_process_monitor_free(res); + + res = tor_process_monitor_new(tor_libevent_get_base(), "45:hello", 0, + NULL, NULL, &msg); + tt_assert(res); + tt_assert(!msg); + + done: + tor_process_monitor_free(res); +} + +struct testcase_t procmon_tests[] = { + { "tor_process_monitor_new", test_procmon_tor_process_monitor_new, + TT_FORK, NULL, NULL }, + END_OF_TESTCASES +}; + diff --git a/src/test/test_rendcache.c b/src/test/test_rendcache.c new file mode 100644 index 0000000000..77796994b4 --- /dev/null +++ b/src/test/test_rendcache.c @@ -0,0 +1,1340 @@ +/* Copyright (c) 2010-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#include "or.h" + +#include "test.h" +#define RENDCACHE_PRIVATE +#include "rendcache.h" +#include "router.h" +#include "routerlist.h" +#include "config.h" +#include <openssl/rsa.h> +#include "rend_test_helpers.h" + +#define NS_MODULE rend_cache + +static const int RECENT_TIME = -10; +static const int TIME_IN_THE_PAST = -(REND_CACHE_MAX_AGE + \ + REND_CACHE_MAX_SKEW + 10); +static const int TIME_IN_THE_FUTURE = REND_CACHE_MAX_SKEW + 10; + +extern strmap_t *rend_cache; +extern digestmap_t *rend_cache_v2_dir; +extern strmap_t *rend_cache_failure; +extern size_t rend_cache_total_allocation; + +static rend_data_t * +mock_rend_data(const char *onion_address) +{ + rend_data_t *rend_query = tor_malloc_zero(sizeof(rend_data_t)); + + strlcpy(rend_query->onion_address, onion_address, + sizeof(rend_query->onion_address)); + rend_query->auth_type = REND_NO_AUTH; + rend_query->hsdirs_fp = smartlist_new(); + smartlist_add(rend_query->hsdirs_fp, tor_memdup("aaaaaaaaaaaaaaaaaaaaaaaa", + DIGEST_LEN)); + + return rend_query; +} + +static void +test_rend_cache_lookup_entry(void *data) +{ + int ret; + rend_data_t *mock_rend_query = NULL; + char desc_id_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; + rend_cache_entry_t *entry = NULL; + rend_encoded_v2_service_descriptor_t *desc_holder = NULL; + char *service_id = NULL; + (void)data; + + rend_cache_init(); + + generate_desc(RECENT_TIME, &desc_holder, &service_id, 3); + + ret = rend_cache_lookup_entry("abababababababab", 0, NULL); + tt_int_op(ret, OP_EQ, -ENOENT); + + ret = rend_cache_lookup_entry("invalid query", 2, NULL); + tt_int_op(ret, OP_EQ, -EINVAL); + + ret = rend_cache_lookup_entry("abababababababab", 2, NULL); + tt_int_op(ret, OP_EQ, -ENOENT); + + ret = rend_cache_lookup_entry("abababababababab", 4224, NULL); + tt_int_op(ret, OP_EQ, -ENOENT); + + mock_rend_query = mock_rend_data(service_id); + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + rend_cache_store_v2_desc_as_client(desc_holder->desc_str, desc_id_base32, + mock_rend_query, NULL); + + ret = rend_cache_lookup_entry(service_id, 2, NULL); + tt_int_op(ret, OP_EQ, 0); + + ret = rend_cache_lookup_entry(service_id, 2, &entry); + tt_assert(entry); + tt_int_op(entry->len, OP_EQ, strlen(desc_holder->desc_str)); + tt_str_op(entry->desc, OP_EQ, desc_holder->desc_str); + + done: + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_cache_free_all(); + rend_data_free(mock_rend_query); +} + +static void +test_rend_cache_store_v2_desc_as_client(void *data) +{ + rend_cache_store_status_t ret; + rend_data_t *mock_rend_query; + char desc_id_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; + rend_cache_entry_t *entry = NULL; + rend_encoded_v2_service_descriptor_t *desc_holder = NULL; + char *service_id = NULL; + char client_cookie[REND_DESC_COOKIE_LEN]; + (void)data; + + rend_cache_init(); + + generate_desc(RECENT_TIME, &desc_holder, &service_id, 3); + + // Test success + mock_rend_query = mock_rend_data(service_id); + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, mock_rend_query, + &entry); + + tt_int_op(ret, OP_EQ, RCS_OKAY); + tt_assert(entry); + tt_int_op(entry->len, OP_EQ, strlen(desc_holder->desc_str)); + tt_str_op(entry->desc, OP_EQ, desc_holder->desc_str); + + // Test various failure modes + + // TODO: a too long desc_id_base32 argument crashes the function + /* ret = rend_cache_store_v2_desc_as_client( */ + /* desc_holder->desc_str, */ + /* "3TOOLONG3TOOLONG3TOOLONG3TOOLONG3TOOLONG3TOOLONG", */ + /* &mock_rend_query, NULL); */ + /* tt_int_op(ret, OP_EQ, RCS_BADDESC); */ + + // Test bad base32 failure + // This causes an assertion failure if we're running with assertions. + // But when doing coverage, we can test it. +#ifdef TOR_COVERAGE + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + "!xqunszqnaolrrfmtzgaki7mxelgvkj", mock_rend_query, NULL); + tt_int_op(ret, OP_EQ, RCS_BADDESC); +#endif + + // Test invalid descriptor + ret = rend_cache_store_v2_desc_as_client("invalid descriptor", + "3xqunszqnaolrrfmtzgaki7mxelgvkje", mock_rend_query, NULL); + tt_int_op(ret, OP_EQ, RCS_BADDESC); + + // TODO: it doesn't seem to be possible to test invalid service ID condition. + // that means it is likely not possible to have that condition without + // earlier conditions failing first (such as signature checking of the desc) + + rend_cache_free_all(); + + // Test mismatch between service ID and onion address + rend_cache_init(); + strncpy(mock_rend_query->onion_address, "abc", REND_SERVICE_ID_LEN_BASE32+1); + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, + mock_rend_query, NULL); + tt_int_op(ret, OP_EQ, RCS_BADDESC); + rend_cache_free_all(); + rend_data_free(mock_rend_query); + + // Test incorrect descriptor ID + rend_cache_init(); + mock_rend_query = mock_rend_data(service_id); + desc_id_base32[0]++; + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, mock_rend_query, + NULL); + tt_int_op(ret, OP_EQ, RCS_BADDESC); + desc_id_base32[0]--; + rend_cache_free_all(); + + // Test too old descriptor + rend_cache_init(); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_data_free(mock_rend_query); + + generate_desc(TIME_IN_THE_PAST, &desc_holder, &service_id, 3); + mock_rend_query = mock_rend_data(service_id); + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, + mock_rend_query, NULL); + tt_int_op(ret, OP_EQ, RCS_BADDESC); + rend_cache_free_all(); + + // Test too new descriptor (in the future) + rend_cache_init(); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_data_free(mock_rend_query); + + generate_desc(TIME_IN_THE_FUTURE, &desc_holder, &service_id, 3); + mock_rend_query = mock_rend_data(service_id); + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, mock_rend_query, + NULL); + tt_int_op(ret, OP_EQ, RCS_BADDESC); + rend_cache_free_all(); + + // Test when a descriptor is already in the cache + rend_cache_init(); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_data_free(mock_rend_query); + + generate_desc(RECENT_TIME, &desc_holder, &service_id, 3); + mock_rend_query = mock_rend_data(service_id); + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + + rend_cache_store_v2_desc_as_client(desc_holder->desc_str, desc_id_base32, + mock_rend_query, NULL); + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, mock_rend_query, + NULL); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, mock_rend_query, + &entry); + tt_int_op(ret, OP_EQ, RCS_OKAY); + tt_assert(entry); + rend_cache_free_all(); + + // Test unsuccessful decrypting of introduction points + rend_cache_init(); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_data_free(mock_rend_query); + + generate_desc(RECENT_TIME, &desc_holder, &service_id, 3); + mock_rend_query = mock_rend_data(service_id); + mock_rend_query->auth_type = REND_BASIC_AUTH; + client_cookie[0] = 'A'; + memcpy(mock_rend_query->descriptor_cookie, client_cookie, + REND_DESC_COOKIE_LEN); + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, mock_rend_query, + NULL); + tt_int_op(ret, OP_EQ, RCS_OKAY); + rend_cache_free_all(); + + // Test successful run when we have REND_BASIC_AUTH but not cookie + rend_cache_init(); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_data_free(mock_rend_query); + + generate_desc(RECENT_TIME, &desc_holder, &service_id, 3); + mock_rend_query = mock_rend_data(service_id); + mock_rend_query->auth_type = REND_BASIC_AUTH; + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, mock_rend_query, + NULL); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + rend_cache_free_all(); + + // Test when we have no introduction points + rend_cache_init(); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_data_free(mock_rend_query); + + generate_desc(RECENT_TIME, &desc_holder, &service_id, 0); + mock_rend_query = mock_rend_data(service_id); + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, mock_rend_query, + NULL); + tt_int_op(ret, OP_EQ, RCS_BADDESC); + rend_cache_free_all(); + + // Test when we have too many intro points + rend_cache_init(); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_data_free(mock_rend_query); + + generate_desc(RECENT_TIME, &desc_holder, &service_id, MAX_INTRO_POINTS+1); + mock_rend_query = mock_rend_data(service_id); + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + ret = rend_cache_store_v2_desc_as_client(desc_holder->desc_str, + desc_id_base32, mock_rend_query, + NULL); + tt_int_op(ret, OP_EQ, RCS_BADDESC); + + done: + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_cache_free_all(); + rend_data_free(mock_rend_query); +} + +static void +test_rend_cache_store_v2_desc_as_client_with_different_time(void *data) +{ + rend_cache_store_status_t ret; + rend_data_t *mock_rend_query; + char desc_id_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; + rend_service_descriptor_t *generated = NULL; + smartlist_t *descs = smartlist_new(); + time_t t; + char *service_id = NULL; + rend_encoded_v2_service_descriptor_t *desc_holder_newer; + rend_encoded_v2_service_descriptor_t *desc_holder_older; + + t = time(NULL); + rend_cache_init(); + + create_descriptor(&generated, &service_id, 3); + + generated->timestamp = t + RECENT_TIME; + rend_encode_v2_descriptors(descs, generated, t + RECENT_TIME, 0, + REND_NO_AUTH, NULL, NULL); + desc_holder_newer = ((rend_encoded_v2_service_descriptor_t *) + smartlist_get(descs, 0)); + smartlist_set(descs, 0, NULL); + + SMARTLIST_FOREACH(descs, rend_encoded_v2_service_descriptor_t *, d, + rend_encoded_v2_service_descriptor_free(d)); + smartlist_free(descs); + descs = smartlist_new(); + + generated->timestamp = (t + RECENT_TIME) - 20; + rend_encode_v2_descriptors(descs, generated, t + RECENT_TIME, 0, + REND_NO_AUTH, NULL, NULL); + desc_holder_older = ((rend_encoded_v2_service_descriptor_t *) + smartlist_get(descs, 0)); + smartlist_set(descs, 0, NULL); + (void)data; + + // Test when a descriptor is already in the cache and it is newer than the + // one we submit + mock_rend_query = mock_rend_data(service_id); + base32_encode(desc_id_base32, sizeof(desc_id_base32), + desc_holder_newer->desc_id, DIGEST_LEN); + rend_cache_store_v2_desc_as_client(desc_holder_newer->desc_str, + desc_id_base32, mock_rend_query, NULL); + ret = rend_cache_store_v2_desc_as_client(desc_holder_older->desc_str, + desc_id_base32, mock_rend_query, + NULL); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + rend_cache_free_all(); + + // Test when an old descriptor is in the cache and we submit a newer one + rend_cache_init(); + rend_cache_store_v2_desc_as_client(desc_holder_older->desc_str, + desc_id_base32, mock_rend_query, NULL); + ret = rend_cache_store_v2_desc_as_client(desc_holder_newer->desc_str, + desc_id_base32, mock_rend_query, + NULL); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + done: + rend_encoded_v2_service_descriptor_free(desc_holder_newer); + rend_encoded_v2_service_descriptor_free(desc_holder_older); + SMARTLIST_FOREACH(descs, rend_encoded_v2_service_descriptor_t *, d, + rend_encoded_v2_service_descriptor_free(d)); + smartlist_free(descs); + rend_service_descriptor_free(generated); + tor_free(service_id); + rend_cache_free_all(); + rend_data_free(mock_rend_query); +} + +#define NS_SUBMODULE lookup_v2_desc_as_dir +NS_DECL(const routerinfo_t *, router_get_my_routerinfo, (void)); +NS_DECL(int, hid_serv_responsible_for_desc_id, (const char *id)); + +static routerinfo_t *mock_routerinfo; +static int hid_serv_responsible_for_desc_id_response; + +static const routerinfo_t * +NS(router_get_my_routerinfo)(void) +{ + if (!mock_routerinfo) { + mock_routerinfo = tor_malloc(sizeof(routerinfo_t)); + } + + return mock_routerinfo; +} + +static int +NS(hid_serv_responsible_for_desc_id)(const char *id) +{ + (void)id; + return hid_serv_responsible_for_desc_id_response; +} + +static void +test_rend_cache_lookup_v2_desc_as_dir(void *data) +{ + int ret; + char desc_id_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; + rend_encoded_v2_service_descriptor_t *desc_holder = NULL; + char *service_id = NULL; + const char *ret_desc = NULL; + + (void)data; + + NS_MOCK(router_get_my_routerinfo); + NS_MOCK(hid_serv_responsible_for_desc_id); + + rend_cache_init(); + + // Test invalid base32 + ret = rend_cache_lookup_v2_desc_as_dir("!bababababababab", NULL); + tt_int_op(ret, OP_EQ, -1); + + // Test non-existent descriptor but well formed + ret = rend_cache_lookup_v2_desc_as_dir("3xqunszqnaolrrfmtzgaki7mxelgvkje", + NULL); + tt_int_op(ret, OP_EQ, 0); + + // Test existing descriptor + hid_serv_responsible_for_desc_id_response = 1; + generate_desc(RECENT_TIME, &desc_holder, &service_id, 3); + rend_cache_store_v2_desc_as_dir(desc_holder->desc_str); + base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_holder->desc_id, + DIGEST_LEN); + ret = rend_cache_lookup_v2_desc_as_dir(desc_id_base32, &ret_desc); + tt_int_op(ret, OP_EQ, 1); + tt_assert(ret_desc); + + done: + NS_UNMOCK(router_get_my_routerinfo); + NS_UNMOCK(hid_serv_responsible_for_desc_id); + tor_free(mock_routerinfo); + rend_cache_free_all(); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); +} + +#undef NS_SUBMODULE + +#define NS_SUBMODULE store_v2_desc_as_dir +NS_DECL(const routerinfo_t *, router_get_my_routerinfo, (void)); +NS_DECL(int, hid_serv_responsible_for_desc_id, (const char *id)); + +static const routerinfo_t * +NS(router_get_my_routerinfo)(void) +{ + return mock_routerinfo; +} + +static int +NS(hid_serv_responsible_for_desc_id)(const char *id) +{ + (void)id; + return hid_serv_responsible_for_desc_id_response; +} + +static void +test_rend_cache_store_v2_desc_as_dir(void *data) +{ + (void)data; + rend_cache_store_status_t ret; + rend_encoded_v2_service_descriptor_t *desc_holder = NULL; + char *service_id = NULL; + + NS_MOCK(router_get_my_routerinfo); + NS_MOCK(hid_serv_responsible_for_desc_id); + + rend_cache_init(); + + // Test when we are not an HS dir + mock_routerinfo = NULL; + ret = rend_cache_store_v2_desc_as_dir(""); + tt_int_op(ret, OP_EQ, RCS_NOTDIR); + + // Test when we can't parse the descriptor + mock_routerinfo = tor_malloc(sizeof(routerinfo_t)); + hid_serv_responsible_for_desc_id_response = 1; + ret = rend_cache_store_v2_desc_as_dir("unparseable"); + tt_int_op(ret, OP_EQ, RCS_BADDESC); + + // Test when we are not responsible for an HS + hid_serv_responsible_for_desc_id_response = 0; + generate_desc(RECENT_TIME, &desc_holder, &service_id, 3); + ret = rend_cache_store_v2_desc_as_dir(desc_holder->desc_str); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + + // Test when we have an old descriptor + hid_serv_responsible_for_desc_id_response = 1; + generate_desc(TIME_IN_THE_PAST, &desc_holder, &service_id, 3); + ret = rend_cache_store_v2_desc_as_dir(desc_holder->desc_str); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + + // Test when we have a descriptor in the future + generate_desc(TIME_IN_THE_FUTURE, &desc_holder, &service_id, 3); + ret = rend_cache_store_v2_desc_as_dir(desc_holder->desc_str); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + + // Test when two descriptors + generate_desc(TIME_IN_THE_FUTURE, &desc_holder, &service_id, 3); + ret = rend_cache_store_v2_desc_as_dir(desc_holder->desc_str); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + + // Test when asking for hidden service statistics HiddenServiceStatistics + rend_cache_purge(); + generate_desc(RECENT_TIME, &desc_holder, &service_id, 3); + get_options_mutable()->HiddenServiceStatistics = 1; + ret = rend_cache_store_v2_desc_as_dir(desc_holder->desc_str); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + done: + NS_UNMOCK(router_get_my_routerinfo); + NS_UNMOCK(hid_serv_responsible_for_desc_id); + rend_encoded_v2_service_descriptor_free(desc_holder); + tor_free(service_id); + rend_cache_free_all(); + tor_free(mock_routerinfo); +} + +static void +test_rend_cache_store_v2_desc_as_dir_with_different_time(void *data) +{ + (void)data; + + rend_cache_store_status_t ret; + rend_service_descriptor_t *generated = NULL; + smartlist_t *descs = smartlist_new(); + time_t t; + char *service_id = NULL; + rend_encoded_v2_service_descriptor_t *desc_holder_newer; + rend_encoded_v2_service_descriptor_t *desc_holder_older; + + NS_MOCK(router_get_my_routerinfo); + NS_MOCK(hid_serv_responsible_for_desc_id); + + rend_cache_init(); + + t = time(NULL); + + create_descriptor(&generated, &service_id, 3); + generated->timestamp = t + RECENT_TIME; + rend_encode_v2_descriptors(descs, generated, t + RECENT_TIME, 0, + REND_NO_AUTH, NULL, NULL); + desc_holder_newer = ((rend_encoded_v2_service_descriptor_t *) + smartlist_get(descs, 0)); + smartlist_set(descs, 0, NULL); + SMARTLIST_FOREACH(descs, rend_encoded_v2_service_descriptor_t *, d, + rend_encoded_v2_service_descriptor_free(d)); + smartlist_free(descs); + descs = smartlist_new(); + + generated->timestamp = (t + RECENT_TIME) - 20; + rend_encode_v2_descriptors(descs, generated, t + RECENT_TIME, 0, + REND_NO_AUTH, NULL, NULL); + desc_holder_older = ((rend_encoded_v2_service_descriptor_t *) + smartlist_get(descs, 0)); + smartlist_set(descs, 0, NULL); + + // Test when we have a newer descriptor stored + mock_routerinfo = tor_malloc(sizeof(routerinfo_t)); + hid_serv_responsible_for_desc_id_response = 1; + rend_cache_store_v2_desc_as_dir(desc_holder_newer->desc_str); + ret = rend_cache_store_v2_desc_as_dir(desc_holder_older->desc_str); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + // Test when we have an old descriptor stored + rend_cache_purge(); + rend_cache_store_v2_desc_as_dir(desc_holder_older->desc_str); + ret = rend_cache_store_v2_desc_as_dir(desc_holder_newer->desc_str); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + done: + NS_UNMOCK(router_get_my_routerinfo); + NS_UNMOCK(hid_serv_responsible_for_desc_id); + rend_cache_free_all(); + rend_service_descriptor_free(generated); + tor_free(service_id); + SMARTLIST_FOREACH(descs, rend_encoded_v2_service_descriptor_t *, d, + rend_encoded_v2_service_descriptor_free(d)); + smartlist_free(descs); + rend_encoded_v2_service_descriptor_free(desc_holder_newer); + rend_encoded_v2_service_descriptor_free(desc_holder_older); + tor_free(mock_routerinfo); +} + +static void +test_rend_cache_store_v2_desc_as_dir_with_different_content(void *data) +{ + (void)data; + + rend_cache_store_status_t ret; + rend_service_descriptor_t *generated = NULL; + smartlist_t *descs = smartlist_new(); + time_t t; + char *service_id = NULL; + rend_encoded_v2_service_descriptor_t *desc_holder_one = NULL; + rend_encoded_v2_service_descriptor_t *desc_holder_two = NULL; + + NS_MOCK(router_get_my_routerinfo); + NS_MOCK(hid_serv_responsible_for_desc_id); + + rend_cache_init(); + + t = time(NULL); + + create_descriptor(&generated, &service_id, 3); + generated->timestamp = t + RECENT_TIME; + rend_encode_v2_descriptors(descs, generated, t + RECENT_TIME, 0, + REND_NO_AUTH, NULL, NULL); + desc_holder_one = ((rend_encoded_v2_service_descriptor_t *) + smartlist_get(descs, 0)); + smartlist_set(descs, 0, NULL); + + SMARTLIST_FOREACH(descs, rend_encoded_v2_service_descriptor_t *, d, + rend_encoded_v2_service_descriptor_free(d)); + smartlist_free(descs); + descs = smartlist_new(); + + generated->timestamp = t + RECENT_TIME; + generated->protocols = 41; + rend_encode_v2_descriptors(descs, generated, t + RECENT_TIME, 0, + REND_NO_AUTH, NULL, NULL); + desc_holder_two = ((rend_encoded_v2_service_descriptor_t *) + smartlist_get(descs, 0)); + smartlist_set(descs, 0, NULL); + + // Test when we have another descriptor stored, with a different descriptor + mock_routerinfo = tor_malloc(sizeof(routerinfo_t)); + hid_serv_responsible_for_desc_id_response = 1; + rend_cache_store_v2_desc_as_dir(desc_holder_one->desc_str); + ret = rend_cache_store_v2_desc_as_dir(desc_holder_two->desc_str); + tt_int_op(ret, OP_EQ, RCS_OKAY); + + done: + NS_UNMOCK(router_get_my_routerinfo); + NS_UNMOCK(hid_serv_responsible_for_desc_id); + rend_cache_free_all(); + rend_service_descriptor_free(generated); + tor_free(service_id); + SMARTLIST_FOREACH(descs, rend_encoded_v2_service_descriptor_t *, d, + rend_encoded_v2_service_descriptor_free(d)); + smartlist_free(descs); + rend_encoded_v2_service_descriptor_free(desc_holder_one); + rend_encoded_v2_service_descriptor_free(desc_holder_two); +} + +#undef NS_SUBMODULE + +static void +test_rend_cache_init(void *data) +{ + (void)data; + + tt_assert_msg(!rend_cache, "rend_cache should be NULL when starting"); + tt_assert_msg(!rend_cache_v2_dir, "rend_cache_v2_dir should be NULL " + "when starting"); + tt_assert_msg(!rend_cache_failure, "rend_cache_failure should be NULL when " + "starting"); + + rend_cache_init(); + + tt_assert_msg(rend_cache, "rend_cache should not be NULL after initing"); + tt_assert_msg(rend_cache_v2_dir, "rend_cache_v2_dir should not be NULL " + "after initing"); + tt_assert_msg(rend_cache_failure, "rend_cache_failure should not be NULL " + "after initing"); + + tt_int_op(strmap_size(rend_cache), OP_EQ, 0); + tt_int_op(digestmap_size(rend_cache_v2_dir), OP_EQ, 0); + tt_int_op(strmap_size(rend_cache_failure), OP_EQ, 0); + + done: + rend_cache_free_all(); +} + +static void +test_rend_cache_decrement_allocation(void *data) +{ + (void)data; + + // Test when the cache has enough allocations + rend_cache_total_allocation = 10; + rend_cache_decrement_allocation(3); + tt_int_op(rend_cache_total_allocation, OP_EQ, 7); + + // Test when there are not enough allocations + rend_cache_total_allocation = 1; + rend_cache_decrement_allocation(2); + tt_int_op(rend_cache_total_allocation, OP_EQ, 0); + + // And again + rend_cache_decrement_allocation(2); + tt_int_op(rend_cache_total_allocation, OP_EQ, 0); + + done: + (void)0; +} + +static void +test_rend_cache_increment_allocation(void *data) +{ + (void)data; + + // Test when the cache is not overflowing + rend_cache_total_allocation = 5; + rend_cache_increment_allocation(3); + tt_int_op(rend_cache_total_allocation, OP_EQ, 8); + + // Test when there are too many allocations + rend_cache_total_allocation = SIZE_MAX-1; + rend_cache_increment_allocation(2); + tt_u64_op(rend_cache_total_allocation, OP_EQ, SIZE_MAX); + + // And again + rend_cache_increment_allocation(2); + tt_u64_op(rend_cache_total_allocation, OP_EQ, SIZE_MAX); + + done: + (void)0; +} + +static void +test_rend_cache_failure_intro_entry_new(void *data) +{ + time_t now; + rend_cache_failure_intro_t *entry; + rend_intro_point_failure_t failure; + + (void)data; + + failure = INTRO_POINT_FAILURE_TIMEOUT; + now = time(NULL); + entry = rend_cache_failure_intro_entry_new(failure); + + tt_int_op(entry->failure_type, OP_EQ, INTRO_POINT_FAILURE_TIMEOUT); + tt_int_op(entry->created_ts, OP_GE, now-5); + tt_int_op(entry->created_ts, OP_LE, now+5); + + done: + tor_free(entry); +} + +static void +test_rend_cache_failure_intro_lookup(void *data) +{ + (void)data; + int ret; + rend_cache_failure_t *failure; + rend_cache_failure_intro_t *ip; + rend_cache_failure_intro_t *entry; + const char key_ip_one[DIGEST_LEN] = "ip1"; + const char key_ip_two[DIGEST_LEN] = "ip2"; + const char key_foo[DIGEST_LEN] = "foo1"; + + rend_cache_init(); + + failure = rend_cache_failure_entry_new(); + ip = rend_cache_failure_intro_entry_new(INTRO_POINT_FAILURE_TIMEOUT); + digestmap_set(failure->intro_failures, key_ip_one, ip); + strmap_set_lc(rend_cache_failure, "foo1", failure); + + // Test not found + ret = cache_failure_intro_lookup((const uint8_t *) key_foo, "foo2", NULL); + tt_int_op(ret, OP_EQ, 0); + + // Test found with no intro failures in it + ret = cache_failure_intro_lookup((const uint8_t *) key_ip_two, "foo1", NULL); + tt_int_op(ret, OP_EQ, 0); + + // Test found + ret = cache_failure_intro_lookup((const uint8_t *) key_ip_one, "foo1", NULL); + tt_int_op(ret, OP_EQ, 1); + + // Test found and asking for entry + cache_failure_intro_lookup((const uint8_t *) key_ip_one, "foo1", &entry); + tt_assert(entry); + tt_assert(entry == ip); + + done: + rend_cache_free_all(); +} + +static void +test_rend_cache_clean(void *data) +{ + rend_cache_entry_t *one, *two; + rend_service_descriptor_t *desc_one, *desc_two; + strmap_iter_t *iter = NULL; + const char *key; + void *val; + + (void)data; + + rend_cache_init(); + + // Test with empty rendcache + rend_cache_clean(time(NULL), REND_CACHE_TYPE_CLIENT); + tt_int_op(strmap_size(rend_cache), OP_EQ, 0); + + // Test with two old entries + one = tor_malloc_zero(sizeof(rend_cache_entry_t)); + two = tor_malloc_zero(sizeof(rend_cache_entry_t)); + desc_one = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + desc_two = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + one->parsed = desc_one; + two->parsed = desc_two; + + desc_one->timestamp = time(NULL) + TIME_IN_THE_PAST; + desc_two->timestamp = (time(NULL) + TIME_IN_THE_PAST) - 10; + desc_one->pk = pk_generate(0); + desc_two->pk = pk_generate(1); + + strmap_set_lc(rend_cache, "foo1", one); + strmap_set_lc(rend_cache, "foo2", two); + + rend_cache_clean(time(NULL), REND_CACHE_TYPE_CLIENT); + tt_int_op(strmap_size(rend_cache), OP_EQ, 0); + + // Test with one old entry and one newer entry + one = tor_malloc_zero(sizeof(rend_cache_entry_t)); + two = tor_malloc_zero(sizeof(rend_cache_entry_t)); + desc_one = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + desc_two = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + one->parsed = desc_one; + two->parsed = desc_two; + + desc_one->timestamp = (time(NULL) + TIME_IN_THE_PAST) - 10; + desc_two->timestamp = time(NULL) - 100; + desc_one->pk = pk_generate(0); + desc_two->pk = pk_generate(1); + + strmap_set_lc(rend_cache, "foo1", one); + strmap_set_lc(rend_cache, "foo2", two); + + rend_cache_clean(time(NULL), REND_CACHE_TYPE_CLIENT); + tt_int_op(strmap_size(rend_cache), OP_EQ, 1); + + iter = strmap_iter_init(rend_cache); + strmap_iter_get(iter, &key, &val); + tt_str_op(key, OP_EQ, "foo2"); + + done: + rend_cache_free_all(); +} + +static void +test_rend_cache_failure_entry_new(void *data) +{ + rend_cache_failure_t *failure; + + (void)data; + + failure = rend_cache_failure_entry_new(); + tt_assert(failure); + tt_int_op(digestmap_size(failure->intro_failures), OP_EQ, 0); + + done: + rend_cache_failure_entry_free(failure); +} + +static void +test_rend_cache_failure_entry_free(void *data) +{ + (void)data; + + // Test that it can deal with a NULL argument + rend_cache_failure_entry_free(NULL); + + /* done: */ + /* (void)0; */ +} + +static void +test_rend_cache_failure_clean(void *data) +{ + rend_cache_failure_t *failure; + rend_cache_failure_intro_t *ip_one, *ip_two; + + const char key_one[DIGEST_LEN] = "ip1"; + const char key_two[DIGEST_LEN] = "ip2"; + + (void)data; + + rend_cache_init(); + + // Test with empty failure cache + rend_cache_failure_clean(time(NULL)); + tt_int_op(strmap_size(rend_cache_failure), OP_EQ, 0); + + // Test with one empty failure entry + failure = rend_cache_failure_entry_new(); + strmap_set_lc(rend_cache_failure, "foo1", failure); + rend_cache_failure_clean(time(NULL)); + tt_int_op(strmap_size(rend_cache_failure), OP_EQ, 0); + + // Test with one new intro point + failure = rend_cache_failure_entry_new(); + ip_one = rend_cache_failure_intro_entry_new(INTRO_POINT_FAILURE_TIMEOUT); + digestmap_set(failure->intro_failures, key_one, ip_one); + strmap_set_lc(rend_cache_failure, "foo1", failure); + rend_cache_failure_clean(time(NULL)); + tt_int_op(strmap_size(rend_cache_failure), OP_EQ, 1); + + // Test with one old intro point + rend_cache_failure_purge(); + failure = rend_cache_failure_entry_new(); + ip_one = rend_cache_failure_intro_entry_new(INTRO_POINT_FAILURE_TIMEOUT); + ip_one->created_ts = time(NULL) - 7*60; + digestmap_set(failure->intro_failures, key_one, ip_one); + strmap_set_lc(rend_cache_failure, "foo1", failure); + rend_cache_failure_clean(time(NULL)); + tt_int_op(strmap_size(rend_cache_failure), OP_EQ, 0); + + // Test with one old intro point and one new one + rend_cache_failure_purge(); + failure = rend_cache_failure_entry_new(); + ip_one = rend_cache_failure_intro_entry_new(INTRO_POINT_FAILURE_TIMEOUT); + ip_one->created_ts = time(NULL) - 7*60; + digestmap_set(failure->intro_failures, key_one, ip_one); + ip_two = rend_cache_failure_intro_entry_new(INTRO_POINT_FAILURE_TIMEOUT); + ip_two->created_ts = time(NULL) - 2*60; + digestmap_set(failure->intro_failures, key_two, ip_two); + strmap_set_lc(rend_cache_failure, "foo1", failure); + rend_cache_failure_clean(time(NULL)); + tt_int_op(strmap_size(rend_cache_failure), OP_EQ, 1); + tt_int_op(digestmap_size(failure->intro_failures), OP_EQ, 1); + + done: + rend_cache_free_all(); +} + +static void +test_rend_cache_failure_remove(void *data) +{ + rend_service_descriptor_t *desc; + (void)data; + + rend_cache_init(); + + // Test that it deals well with a NULL desc + rend_cache_failure_remove(NULL); + + // Test a descriptor that isn't in the cache + desc = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + desc->pk = pk_generate(0); + rend_cache_failure_remove(desc); + + // There seems to not exist any way of getting rend_cache_failure_remove() + // to fail because of a problem with rend_get_service_id from here + rend_cache_free_all(); + + rend_service_descriptor_free(desc); + /* done: */ + /* (void)0; */ +} + +static void +test_rend_cache_free_all(void *data) +{ + rend_cache_failure_t *failure; + rend_cache_entry_t *one; + rend_service_descriptor_t *desc_one; + + (void)data; + + rend_cache_init(); + + failure = rend_cache_failure_entry_new(); + strmap_set_lc(rend_cache_failure, "foo1", failure); + + one = tor_malloc_zero(sizeof(rend_cache_entry_t)); + desc_one = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + one->parsed = desc_one; + desc_one->timestamp = time(NULL) + TIME_IN_THE_PAST; + desc_one->pk = pk_generate(0); + strmap_set_lc(rend_cache, "foo1", one); + + rend_cache_free_all(); + + tt_assert(!rend_cache); + tt_assert(!rend_cache_v2_dir); + tt_assert(!rend_cache_failure); + tt_assert(!rend_cache_total_allocation); + + done: + rend_cache_free_all(); +} + +static void +test_rend_cache_entry_free(void *data) +{ + (void)data; + rend_cache_entry_t *e; + + // Handles NULL correctly + rend_cache_entry_free(NULL); + + // Handles NULL descriptor correctly + e = tor_malloc_zero(sizeof(rend_cache_entry_t)); + rend_cache_entry_free(e); + + // Handles non-NULL descriptor correctly + e = tor_malloc_zero(sizeof(rend_cache_entry_t)); + e->desc = (char *)malloc(10); + rend_cache_entry_free(e); + + /* done: */ + /* (void)0; */ +} + +static void +test_rend_cache_purge(void *data) +{ + (void)data; + + // Deals with a NULL rend_cache + rend_cache_purge(); + tt_assert(rend_cache); + tt_assert(strmap_size(rend_cache) == 0); + + // Deals with existing rend_cache + rend_cache_free_all(); + rend_cache_init(); + tt_assert(rend_cache); + tt_assert(strmap_size(rend_cache) == 0); + + rend_cache_purge(); + tt_assert(rend_cache); + tt_assert(strmap_size(rend_cache) == 0); + + done: + rend_cache_free_all(); +} + +static void +test_rend_cache_failure_intro_add(void *data) +{ + (void)data; + rend_cache_failure_t *fail_entry; + rend_cache_failure_intro_t *entry; + const char identity[DIGEST_LEN] = "foo1"; + + rend_cache_init(); + + // Adds non-existing entry + cache_failure_intro_add((const uint8_t *) identity, "foo2", + INTRO_POINT_FAILURE_TIMEOUT); + fail_entry = strmap_get_lc(rend_cache_failure, "foo2"); + tt_assert(fail_entry); + tt_int_op(digestmap_size(fail_entry->intro_failures), OP_EQ, 1); + entry = digestmap_get(fail_entry->intro_failures, identity); + tt_assert(entry); + + // Adds existing entry + cache_failure_intro_add((const uint8_t *) identity, "foo2", + INTRO_POINT_FAILURE_TIMEOUT); + fail_entry = strmap_get_lc(rend_cache_failure, "foo2"); + tt_assert(fail_entry); + tt_int_op(digestmap_size(fail_entry->intro_failures), OP_EQ, 1); + entry = digestmap_get(fail_entry->intro_failures, identity); + tt_assert(entry); + + done: + rend_cache_free_all(); +} + +static void +test_rend_cache_intro_failure_note(void *data) +{ + (void)data; + rend_cache_failure_t *fail_entry; + rend_cache_failure_intro_t *entry; + const char key[DIGEST_LEN] = "foo1"; + + rend_cache_init(); + + // Test not found + rend_cache_intro_failure_note(INTRO_POINT_FAILURE_TIMEOUT, + (const uint8_t *) key, "foo2"); + fail_entry = strmap_get_lc(rend_cache_failure, "foo2"); + tt_assert(fail_entry); + tt_int_op(digestmap_size(fail_entry->intro_failures), OP_EQ, 1); + entry = digestmap_get(fail_entry->intro_failures, key); + tt_assert(entry); + tt_int_op(entry->failure_type, OP_EQ, INTRO_POINT_FAILURE_TIMEOUT); + + // Test found + rend_cache_intro_failure_note(INTRO_POINT_FAILURE_UNREACHABLE, + (const uint8_t *) key, "foo2"); + tt_int_op(entry->failure_type, OP_EQ, INTRO_POINT_FAILURE_UNREACHABLE); + + done: + rend_cache_free_all(); +} + +#define NS_SUBMODULE clean_v2_descs_as_dir +NS_DECL(int, hid_serv_responsible_for_desc_id, (const char *id)); + +static int +NS(hid_serv_responsible_for_desc_id)(const char *id) +{ + (void)id; + return hid_serv_responsible_for_desc_id_response; +} + +static void +test_rend_cache_clean_v2_descs_as_dir(void *data) +{ + rend_cache_entry_t *e; + time_t now; + rend_service_descriptor_t *desc; + now = time(NULL); + const char key[DIGEST_LEN] = "abcde"; + + (void)data; + + NS_MOCK(hid_serv_responsible_for_desc_id); + rend_cache_init(); + + // Test running with an empty cache + rend_cache_clean_v2_descs_as_dir(now, 0); + tt_int_op(digestmap_size(rend_cache_v2_dir), OP_EQ, 0); + + // Test with only one new entry + e = tor_malloc_zero(sizeof(rend_cache_entry_t)); + e->last_served = now; + desc = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + desc->timestamp = now; + desc->pk = pk_generate(0); + e->parsed = desc; + digestmap_set(rend_cache_v2_dir, key, e); + + hid_serv_responsible_for_desc_id_response = 1; + rend_cache_clean_v2_descs_as_dir(now, 0); + tt_int_op(digestmap_size(rend_cache_v2_dir), OP_EQ, 1); + + // Test with one old entry + desc->timestamp = now - (REND_CACHE_MAX_AGE + REND_CACHE_MAX_SKEW + 1000); + rend_cache_clean_v2_descs_as_dir(now, 0); + tt_int_op(digestmap_size(rend_cache_v2_dir), OP_EQ, 0); + + // Test with one entry that is not under the responsibility of this + // hidden service + e = tor_malloc_zero(sizeof(rend_cache_entry_t)); + e->last_served = now; + desc = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + desc->timestamp = now; + desc->pk = pk_generate(0); + e->parsed = desc; + digestmap_set(rend_cache_v2_dir, key, e); + + hid_serv_responsible_for_desc_id_response = 0; + rend_cache_clean_v2_descs_as_dir(now, 0); + tt_int_op(digestmap_size(rend_cache_v2_dir), OP_EQ, 0); + + // Test with one entry that has an old last served + e = tor_malloc_zero(sizeof(rend_cache_entry_t)); + e->last_served = now - (REND_CACHE_MAX_AGE + REND_CACHE_MAX_SKEW + 1000); + desc = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + desc->timestamp = now; + desc->pk = pk_generate(0); + e->parsed = desc; + digestmap_set(rend_cache_v2_dir, key, e); + + hid_serv_responsible_for_desc_id_response = 1; + rend_cache_clean_v2_descs_as_dir(now, 0); + tt_int_op(digestmap_size(rend_cache_v2_dir), OP_EQ, 0); + + // Test a run through asking for a large force_remove + e = tor_malloc_zero(sizeof(rend_cache_entry_t)); + e->last_served = now; + desc = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + desc->timestamp = now; + desc->pk = pk_generate(0); + e->parsed = desc; + digestmap_set(rend_cache_v2_dir, key, e); + + hid_serv_responsible_for_desc_id_response = 1; + rend_cache_clean_v2_descs_as_dir(now, 20000); + tt_int_op(digestmap_size(rend_cache_v2_dir), OP_EQ, 1); + + done: + NS_UNMOCK(hid_serv_responsible_for_desc_id); + rend_cache_free_all(); +} + +#undef NS_SUBMODULE + +static void +test_rend_cache_entry_allocation(void *data) +{ + (void)data; + + size_t ret; + rend_cache_entry_t *e = NULL; + + // Handles a null argument + ret = rend_cache_entry_allocation(NULL); + tt_int_op(ret, OP_EQ, 0); + + // Handles a non-null argument + e = tor_malloc_zero(sizeof(rend_cache_entry_t)); + ret = rend_cache_entry_allocation(e); + tt_int_op(ret, OP_GT, sizeof(rend_cache_entry_t)); + + done: + tor_free(e); +} + +static void +test_rend_cache_failure_intro_entry_free(void *data) +{ + (void)data; + rend_cache_failure_intro_t *entry; + + // Handles a null argument + rend_cache_failure_intro_entry_free(NULL); + + // Handles a non-null argument + entry = rend_cache_failure_intro_entry_new(INTRO_POINT_FAILURE_TIMEOUT); + rend_cache_failure_intro_entry_free(entry); +} + +static void +test_rend_cache_failure_purge(void *data) +{ + (void)data; + + // Handles a null failure cache + strmap_free(rend_cache_failure, rend_cache_failure_entry_free_); + rend_cache_failure = NULL; + + rend_cache_failure_purge(); + + tt_ptr_op(rend_cache_failure, OP_NE, NULL); + tt_int_op(strmap_size(rend_cache_failure), OP_EQ, 0); + + done: + rend_cache_free_all(); +} + +static void +test_rend_cache_validate_intro_point_failure(void *data) +{ + (void)data; + rend_service_descriptor_t *desc = NULL; + char *service_id = NULL; + rend_intro_point_t *intro = NULL; + const char *identity = NULL; + rend_cache_failure_t *failure; + rend_cache_failure_intro_t *ip; + + rend_cache_init(); + + create_descriptor(&desc, &service_id, 3); + desc->timestamp = time(NULL) + RECENT_TIME; + + intro = (rend_intro_point_t *)smartlist_get(desc->intro_nodes, 0); + identity = intro->extend_info->identity_digest; + + failure = rend_cache_failure_entry_new(); + ip = rend_cache_failure_intro_entry_new(INTRO_POINT_FAILURE_TIMEOUT); + digestmap_set(failure->intro_failures, identity, ip); + strmap_set_lc(rend_cache_failure, service_id, failure); + + // Test when we have an intro point in our cache + validate_intro_point_failure(desc, service_id); + tt_int_op(smartlist_len(desc->intro_nodes), OP_EQ, 2); + + done: + rend_cache_free_all(); + rend_service_descriptor_free(desc); + tor_free(service_id); +} + +struct testcase_t rend_cache_tests[] = { + { "init", test_rend_cache_init, 0, NULL, NULL }, + { "decrement_allocation", test_rend_cache_decrement_allocation, 0, + NULL, NULL }, + { "increment_allocation", test_rend_cache_increment_allocation, 0, + NULL, NULL }, + { "clean", test_rend_cache_clean, TT_FORK, NULL, NULL }, + { "clean_v2_descs_as_dir", test_rend_cache_clean_v2_descs_as_dir, 0, + NULL, NULL }, + { "entry_allocation", test_rend_cache_entry_allocation, 0, NULL, NULL }, + { "entry_free", test_rend_cache_entry_free, 0, NULL, NULL }, + { "failure_intro_entry_free", test_rend_cache_failure_intro_entry_free, 0, + NULL, NULL }, + { "free_all", test_rend_cache_free_all, 0, NULL, NULL }, + { "purge", test_rend_cache_purge, 0, NULL, NULL }, + { "failure_clean", test_rend_cache_failure_clean, 0, NULL, NULL }, + { "failure_entry_new", test_rend_cache_failure_entry_new, 0, NULL, NULL }, + { "failure_entry_free", test_rend_cache_failure_entry_free, 0, NULL, NULL }, + { "failure_intro_add", test_rend_cache_failure_intro_add, 0, NULL, NULL }, + { "failure_intro_entry_new", test_rend_cache_failure_intro_entry_new, 0, + NULL, NULL }, + { "failure_intro_lookup", test_rend_cache_failure_intro_lookup, 0, + NULL, NULL }, + { "failure_purge", test_rend_cache_failure_purge, 0, NULL, NULL }, + { "failure_remove", test_rend_cache_failure_remove, 0, NULL, NULL }, + { "intro_failure_note", test_rend_cache_intro_failure_note, 0, NULL, NULL }, + { "lookup", test_rend_cache_lookup_entry, 0, NULL, NULL }, + { "lookup_v2_desc_as_dir", test_rend_cache_lookup_v2_desc_as_dir, 0, + NULL, NULL }, + { "store_v2_desc_as_client", test_rend_cache_store_v2_desc_as_client, 0, + NULL, NULL }, + { "store_v2_desc_as_client_with_different_time", + test_rend_cache_store_v2_desc_as_client_with_different_time, 0, + NULL, NULL }, + { "store_v2_desc_as_dir", test_rend_cache_store_v2_desc_as_dir, 0, + NULL, NULL }, + { "store_v2_desc_as_dir_with_different_time", + test_rend_cache_store_v2_desc_as_dir_with_different_time, 0, NULL, NULL }, + { "store_v2_desc_as_dir_with_different_content", + test_rend_cache_store_v2_desc_as_dir_with_different_content, 0, + NULL, NULL }, + { "validate_intro_point_failure", + test_rend_cache_validate_intro_point_failure, 0, NULL, NULL }, + END_OF_TESTCASES +}; + diff --git a/src/test/test_replay.c b/src/test/test_replay.c index a02c160365..7a0f098776 100644 --- a/src/test/test_replay.c +++ b/src/test/test_replay.c @@ -17,6 +17,20 @@ static const char *test_buffer = " occaecat cupidatat non proident, sunt in culpa qui officia deserunt" " mollit anim id est laborum."; +static const char *test_buffer_2 = + "At vero eos et accusamus et iusto odio dignissimos ducimus qui blanditiis" + " praesentium voluptatum deleniti atque corrupti quos dolores et quas" + " molestias excepturi sint occaecati cupiditate non provident, similique" + " sunt in culpa qui officia deserunt mollitia animi, id est laborum et" + " dolorum fuga. Et harum quidem rerum facilis est et expedita distinctio." + " Nam libero tempore, cum soluta nobis est eligendi optio cumque nihil" + " impedit quo minus id quod maxime placeat facere possimus, omnis voluptas" + " assumenda est, omnis dolor repellendus. Temporibus autem quibusdam et aut" + " officiis debitis aut rerum necessitatibus saepe eveniet ut et voluptates" + " repudiandae sint et molestiae non recusandae. Itaque earum rerum hic" + " tenetur a sapiente delectus, ut aut reiciendis voluptatibus maiores alias" + " consequatur aut perferendis doloribus asperiores repellat."; + static void test_replaycache_alloc(void *arg) { @@ -83,6 +97,12 @@ test_replaycache_miss(void *arg) strlen(test_buffer), NULL); tt_int_op(result,OP_EQ, 0); + /* make sure a different buffer misses as well */ + result = + replaycache_add_and_test_internal(1200, NULL, test_buffer_2, + strlen(test_buffer_2), NULL); + tt_int_op(result,OP_EQ, 0); + /* poke the bad-parameter error case too */ result = replaycache_add_and_test_internal(1200, NULL, test_buffer, @@ -115,6 +135,18 @@ test_replaycache_hit(void *arg) strlen(test_buffer), NULL); tt_int_op(result,OP_EQ, 1); + /* make sure a different buffer misses then hits as well */ + + result = + replaycache_add_and_test_internal(1200, r, test_buffer_2, + strlen(test_buffer_2), NULL); + tt_int_op(result,OP_EQ, 0); + + result = + replaycache_add_and_test_internal(1300, r, test_buffer_2, + strlen(test_buffer_2), NULL); + tt_int_op(result,OP_EQ, 1); + done: if (r) replaycache_free(r); @@ -245,7 +277,7 @@ test_replaycache_scrub(void *arg) /* Make sure we hit the aging-out case too */ replaycache_scrub_if_needed_internal(1500, r); /* Assert that we aged it */ - tt_int_op(digestmap_size(r->digests_seen),OP_EQ, 0); + tt_int_op(digest256map_size(r->digests_seen),OP_EQ, 0); done: if (r) replaycache_free(r); diff --git a/src/test/test_routerset.c b/src/test/test_routerset.c index 90dfb28c6b..74b39c0486 100644 --- a/src/test/test_routerset.c +++ b/src/test/test_routerset.c @@ -423,10 +423,10 @@ NS(test_main)(void *arg) } #undef NS_SUBMODULE -#define NS_SUBMODULE ASPECT(routerset_parse, policy) +#define NS_SUBMODULE ASPECT(routerset_parse, policy_wildcard) /* - * Structural test for routerset_parse, when given a valid policy. + * Structural test for routerset_parse, when given a valid wildcard policy. */ NS_DECL(addr_policy_t *, router_parse_addr_policy_item_from_string, @@ -470,6 +470,100 @@ NS(router_parse_addr_policy_item_from_string)(const char *s, } #undef NS_SUBMODULE +#define NS_SUBMODULE ASPECT(routerset_parse, policy_ipv4) + +/* + * Structural test for routerset_parse, when given a valid IPv4 address + * literal policy. + */ + +NS_DECL(addr_policy_t *, router_parse_addr_policy_item_from_string, + (const char *s, int assume_action, int *bogus)); + +addr_policy_t *NS(mock_addr_policy); + +static void +NS(test_main)(void *arg) +{ + routerset_t *set; + const char *s; + int r; + (void)arg; + + NS_MOCK(router_parse_addr_policy_item_from_string); + NS(mock_addr_policy) = tor_malloc_zero(sizeof(addr_policy_t)); + + set = routerset_new(); + s = "127.0.0.1"; + r = routerset_parse(set, s, ""); + tt_int_op(r, OP_EQ, 0); + tt_int_op(smartlist_len(set->policies), OP_NE, 0); + tt_int_op(CALLED(router_parse_addr_policy_item_from_string), OP_EQ, 1); + + done: + routerset_free(set); +} + +addr_policy_t * +NS(router_parse_addr_policy_item_from_string)(const char *s, int assume_action, + int *bogus) +{ + (void)s; + (void)assume_action; + CALLED(router_parse_addr_policy_item_from_string)++; + *bogus = 0; + + return NS(mock_addr_policy); +} + +#undef NS_SUBMODULE +#define NS_SUBMODULE ASPECT(routerset_parse, policy_ipv6) + +/* + * Structural test for routerset_parse, when given a valid IPv6 address + * literal policy. + */ + +NS_DECL(addr_policy_t *, router_parse_addr_policy_item_from_string, + (const char *s, int assume_action, int *bad)); + +addr_policy_t *NS(mock_addr_policy); + +static void +NS(test_main)(void *arg) +{ + routerset_t *set; + const char *s; + int r; + (void)arg; + + NS_MOCK(router_parse_addr_policy_item_from_string); + NS(mock_addr_policy) = tor_malloc_zero(sizeof(addr_policy_t)); + + set = routerset_new(); + s = "::1"; + r = routerset_parse(set, s, ""); + tt_int_op(r, OP_EQ, 0); + tt_int_op(smartlist_len(set->policies), OP_NE, 0); + tt_int_op(CALLED(router_parse_addr_policy_item_from_string), OP_EQ, 1); + + done: + routerset_free(set); +} + +addr_policy_t * +NS(router_parse_addr_policy_item_from_string)(const char *s, + int assume_action, int *bad) +{ + (void)s; + (void)assume_action; + CALLED(router_parse_addr_policy_item_from_string)++; + *bad = 0; + + return NS(mock_addr_policy); +} + +#undef NS_SUBMODULE #define NS_SUBMODULE ASPECT(routerset_union, source_bad) /* @@ -2109,7 +2203,9 @@ struct testcase_t routerset_tests[] = { TEST_CASE_ASPECT(routerset_parse, valid_hexdigest), TEST_CASE_ASPECT(routerset_parse, valid_nickname), TEST_CASE_ASPECT(routerset_parse, get_countryname), - TEST_CASE_ASPECT(routerset_parse, policy), + TEST_CASE_ASPECT(routerset_parse, policy_wildcard), + TEST_CASE_ASPECT(routerset_parse, policy_ipv4), + TEST_CASE_ASPECT(routerset_parse, policy_ipv6), TEST_CASE(routerset_subtract_nodes), TEST_CASE_ASPECT(routerset_subtract_nodes, null_routerset), TEST_CASE(routerset_to_string), diff --git a/src/test/test_threads.c b/src/test/test_threads.c index 35f5dc8ea3..fe88c9470f 100644 --- a/src/test/test_threads.c +++ b/src/test/test_threads.c @@ -73,6 +73,8 @@ thread_test_func_(void* _s) ++thread_fns_failed; tor_mutex_release(thread_test_mutex_); + tor_free(mycount); + tor_mutex_release(m); spawn_exit(); diff --git a/src/test/test_tortls.c b/src/test/test_tortls.c new file mode 100644 index 0000000000..b4a3435cfa --- /dev/null +++ b/src/test/test_tortls.c @@ -0,0 +1,2828 @@ +/* Copyright (c) 2010-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#define TORTLS_PRIVATE +#define LOG_PRIVATE +#include "orconfig.h" + +#ifdef _WIN32 +#include <winsock2.h> +#endif + +#ifdef __GNUC__ +#define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__) +#endif + +#if __GNUC__ && GCC_VERSION >= 402 +#if GCC_VERSION >= 406 +#pragma GCC diagnostic push +#endif +/* Some versions of OpenSSL declare SSL_get_selected_srtp_profile twice in + * srtp.h. Suppress the GCC warning so we can build with -Wredundant-decl. */ +#pragma GCC diagnostic ignored "-Wredundant-decls" +#endif + +#include <openssl/opensslv.h> + +#include <openssl/ssl.h> +#include <openssl/ssl3.h> +#include <openssl/err.h> +#include <openssl/asn1t.h> +#include <openssl/x509.h> +#include <openssl/rsa.h> +#include <openssl/evp.h> +#include <openssl/bn.h> + +#if __GNUC__ && GCC_VERSION >= 402 +#if GCC_VERSION >= 406 +#pragma GCC diagnostic pop +#else +#pragma GCC diagnostic warning "-Wredundant-decls" +#endif +#endif + +#include "or.h" +#include "torlog.h" +#include "config.h" +#include "tortls.h" + +#include "test.h" +#include "log_test_helpers.h" +#define NS_MODULE tortls + +extern tor_tls_context_t *server_tls_context; +extern tor_tls_context_t *client_tls_context; + +#if OPENSSL_VERSION_NUMBER >= OPENSSL_V_SERIES(1,1,0) \ + && !defined(LIBRESSL_VERSION_NUMBER) +#define OPENSSL_OPAQUE +#define SSL_STATE_STR "before SSL initialization" +#else +#define SSL_STATE_STR "before/accept initialization" +#endif + +#ifndef OPENSSL_OPAQUE +static SSL_METHOD * +give_me_a_test_method(void) +{ + SSL_METHOD *method = tor_malloc_zero(sizeof(SSL_METHOD)); + memcpy(method, TLSv1_method(), sizeof(SSL_METHOD)); + return method; +} + +static int +fake_num_ciphers(void) +{ + return 0; +} +#endif + +static void +test_tortls_errno_to_tls_error(void *data) +{ + (void) data; + tt_int_op(tor_errno_to_tls_error(SOCK_ERRNO(ECONNRESET)),OP_EQ, + TOR_TLS_ERROR_CONNRESET); + tt_int_op(tor_errno_to_tls_error(SOCK_ERRNO(ETIMEDOUT)),OP_EQ, + TOR_TLS_ERROR_TIMEOUT); + tt_int_op(tor_errno_to_tls_error(SOCK_ERRNO(EHOSTUNREACH)),OP_EQ, + TOR_TLS_ERROR_NO_ROUTE); + tt_int_op(tor_errno_to_tls_error(SOCK_ERRNO(ENETUNREACH)),OP_EQ, + TOR_TLS_ERROR_NO_ROUTE); + tt_int_op(tor_errno_to_tls_error(SOCK_ERRNO(ECONNREFUSED)),OP_EQ, + TOR_TLS_ERROR_CONNREFUSED); + tt_int_op(tor_errno_to_tls_error(0),OP_EQ,TOR_TLS_ERROR_MISC); + done: + (void)1; +} + +static void +test_tortls_err_to_string(void *data) +{ + (void) data; + tt_str_op(tor_tls_err_to_string(1),OP_EQ,"[Not an error.]"); + tt_str_op(tor_tls_err_to_string(TOR_TLS_ERROR_MISC),OP_EQ,"misc error"); + tt_str_op(tor_tls_err_to_string(TOR_TLS_ERROR_IO),OP_EQ,"unexpected close"); + tt_str_op(tor_tls_err_to_string(TOR_TLS_ERROR_CONNREFUSED),OP_EQ, + "connection refused"); + tt_str_op(tor_tls_err_to_string(TOR_TLS_ERROR_CONNRESET),OP_EQ, + "connection reset"); + tt_str_op(tor_tls_err_to_string(TOR_TLS_ERROR_NO_ROUTE),OP_EQ, + "host unreachable"); + tt_str_op(tor_tls_err_to_string(TOR_TLS_ERROR_TIMEOUT),OP_EQ, + "connection timed out"); + tt_str_op(tor_tls_err_to_string(TOR_TLS_CLOSE),OP_EQ,"closed"); + tt_str_op(tor_tls_err_to_string(TOR_TLS_WANTREAD),OP_EQ,"want to read"); + tt_str_op(tor_tls_err_to_string(TOR_TLS_WANTWRITE),OP_EQ,"want to write"); + tt_str_op(tor_tls_err_to_string(-100),OP_EQ,"(unknown error code)"); + done: + (void)1; +} + +static int +mock_tls_cert_matches_key(const tor_tls_t *tls, const tor_x509_cert_t *cert) +{ + (void) tls; + (void) cert; // XXXX look at this. + return 1; +} + +static void +test_tortls_tor_tls_new(void *data) +{ + (void) data; + MOCK(tor_tls_cert_matches_key, mock_tls_cert_matches_key); + crypto_pk_t *key1 = NULL, *key2 = NULL; + SSL_METHOD *method = NULL; + + key1 = pk_generate(2); + key2 = pk_generate(3); + + tor_tls_t *tls = NULL; + tt_int_op(tor_tls_context_init(TOR_TLS_CTX_IS_PUBLIC_SERVER, + key1, key2, 86400), OP_EQ, 0); + tls = tor_tls_new(-1, 0); + tt_want(tls); + tor_tls_free(tls); tls = NULL; + + client_tls_context->ctx = NULL; + tls = tor_tls_new(-1, 0); + tt_assert(!tls); + +#ifndef OPENSSL_OPAQUE + method = give_me_a_test_method(); + SSL_CTX *ctx = SSL_CTX_new(method); + method->num_ciphers = fake_num_ciphers; + client_tls_context->ctx = ctx; + tls = tor_tls_new(-1, 0); + tt_assert(!tls); +#endif + + done: + UNMOCK(tor_tls_cert_matches_key); + crypto_pk_free(key1); + crypto_pk_free(key2); + tor_tls_free(tls); + tor_free(method); + tor_tls_free_all(); +} + +#define NS_MODULE tortls +NS_DECL(void, logv, (int severity, log_domain_mask_t domain, + const char *funcname, const char *suffix, + const char *format, va_list ap)); + +static void +NS(logv)(int severity, log_domain_mask_t domain, + const char *funcname, const char *suffix, const char *format, + va_list ap) +{ + (void) severity; + (void) domain; + (void) funcname; + (void) suffix; + (void) format; + (void) ap; // XXXX look at this. + CALLED(logv)++; +} + +static void +test_tortls_tor_tls_get_error(void *data) +{ + (void) data; + MOCK(tor_tls_cert_matches_key, mock_tls_cert_matches_key); + crypto_pk_t *key1 = NULL, *key2 = NULL; + key1 = pk_generate(2); + key2 = pk_generate(3); + + tor_tls_t *tls = NULL; + tt_int_op(tor_tls_context_init(TOR_TLS_CTX_IS_PUBLIC_SERVER, + key1, key2, 86400), OP_EQ, 0); + tls = tor_tls_new(-1, 0); + NS_MOCK(logv); + tt_int_op(CALLED(logv), OP_EQ, 0); + tor_tls_get_error(tls, 0, 0, + (const char *)"test", 0, 0); + tt_int_op(CALLED(logv), OP_EQ, 1); + + done: + UNMOCK(tor_tls_cert_matches_key); + NS_UNMOCK(logv); + crypto_pk_free(key1); + crypto_pk_free(key2); + tor_tls_free(tls); +} + +static void +test_tortls_get_state_description(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + char *buf; + SSL_CTX *ctx; + + SSL_library_init(); + SSL_load_error_strings(); + + ctx = SSL_CTX_new(SSLv23_method()); + + buf = tor_malloc_zero(1000); + tls = tor_malloc_zero(sizeof(tor_tls_t)); + + tor_tls_get_state_description(NULL, buf, 20); + tt_str_op(buf, OP_EQ, "(No SSL object)"); + + SSL_free(tls->ssl); + tls->ssl = NULL; + tor_tls_get_state_description(tls, buf, 20); + tt_str_op(buf, OP_EQ, "(No SSL object)"); + + tls->ssl = SSL_new(ctx); + tor_tls_get_state_description(tls, buf, 200); + tt_str_op(buf, OP_EQ, SSL_STATE_STR " in HANDSHAKE"); + + tls->state = TOR_TLS_ST_OPEN; + tor_tls_get_state_description(tls, buf, 200); + tt_str_op(buf, OP_EQ, SSL_STATE_STR " in OPEN"); + + tls->state = TOR_TLS_ST_GOTCLOSE; + tor_tls_get_state_description(tls, buf, 200); + tt_str_op(buf, OP_EQ, SSL_STATE_STR " in GOTCLOSE"); + + tls->state = TOR_TLS_ST_SENTCLOSE; + tor_tls_get_state_description(tls, buf, 200); + tt_str_op(buf, OP_EQ, SSL_STATE_STR " in SENTCLOSE"); + + tls->state = TOR_TLS_ST_CLOSED; + tor_tls_get_state_description(tls, buf, 200); + tt_str_op(buf, OP_EQ, SSL_STATE_STR " in CLOSED"); + + tls->state = TOR_TLS_ST_RENEGOTIATE; + tor_tls_get_state_description(tls, buf, 200); + tt_str_op(buf, OP_EQ, SSL_STATE_STR " in RENEGOTIATE"); + + tls->state = TOR_TLS_ST_BUFFEREVENT; + tor_tls_get_state_description(tls, buf, 200); + tt_str_op(buf, OP_EQ, SSL_STATE_STR); + + tls->state = 7; + tor_tls_get_state_description(tls, buf, 200); + tt_str_op(buf, OP_EQ, SSL_STATE_STR " in unknown TLS state"); + + done: + SSL_CTX_free(ctx); + SSL_free(tls->ssl); + tor_free(buf); + tor_free(tls); +} + +extern int tor_tls_object_ex_data_index; + +static void +test_tortls_get_by_ssl(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + tor_tls_t *res; + SSL_CTX *ctx; + SSL *ssl; + + SSL_library_init(); + SSL_load_error_strings(); + tor_tls_allocate_tor_tls_object_ex_data_index(); + + ctx = SSL_CTX_new(SSLv23_method()); + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->magic = TOR_TLS_MAGIC; + + ssl = SSL_new(ctx); + + res = tor_tls_get_by_ssl(ssl); + tt_assert(!res); + + SSL_set_ex_data(ssl, tor_tls_object_ex_data_index, tls); + + res = tor_tls_get_by_ssl(ssl); + tt_assert(res == tls); + + done: + SSL_free(ssl); + SSL_CTX_free(ctx); + tor_free(tls); +} + +static void +test_tortls_allocate_tor_tls_object_ex_data_index(void *ignored) +{ + (void)ignored; + int first; + + tor_tls_allocate_tor_tls_object_ex_data_index(); + + first = tor_tls_object_ex_data_index; + tor_tls_allocate_tor_tls_object_ex_data_index(); + tt_int_op(first, OP_EQ, tor_tls_object_ex_data_index); + + done: + (void)0; +} + +static void +test_tortls_log_one_error(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + SSL_CTX *ctx; + SSL *ssl = NULL; + + SSL_library_init(); + SSL_load_error_strings(); + + ctx = SSL_CTX_new(SSLv23_method()); + tls = tor_malloc_zero(sizeof(tor_tls_t)); + int previous_log = setup_capture_of_logs(LOG_INFO); + + tor_tls_log_one_error(NULL, 0, LOG_WARN, 0, "something"); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "TLS error while something: " + "(null) (in (null):(null):---)\n"); + + mock_clean_saved_logs(); + tor_tls_log_one_error(tls, 0, LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "TLS error: (null) " + "(in (null):(null):---)\n"); + + mock_clean_saved_logs(); + tls->address = tor_strdup("127.hello"); + tor_tls_log_one_error(tls, 0, LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "TLS error with 127.hello: (null) " + "(in (null):(null):---)\n"); + tor_free(tls->address); + + mock_clean_saved_logs(); + tls->address = tor_strdup("127.hello"); + tor_tls_log_one_error(tls, 0, LOG_WARN, 0, "blarg"); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "TLS error while blarg with " + "127.hello: (null) (in (null):(null):---)\n"); + + mock_clean_saved_logs(); + tor_tls_log_one_error(tls, ERR_PACK(1, 2, 3), LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "TLS error with 127.hello: " + "BN lib (in unknown library:(null):---)\n"); + + mock_clean_saved_logs(); + tor_tls_log_one_error(tls, ERR_PACK(1, 2, SSL_R_HTTP_REQUEST), + LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_INFO); + + mock_clean_saved_logs(); + tor_tls_log_one_error(tls, ERR_PACK(1, 2, SSL_R_HTTPS_PROXY_REQUEST), + LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_INFO); + + mock_clean_saved_logs(); + tor_tls_log_one_error(tls, ERR_PACK(1, 2, SSL_R_RECORD_LENGTH_MISMATCH), + LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_INFO); + + mock_clean_saved_logs(); + tor_tls_log_one_error(tls, ERR_PACK(1, 2, SSL_R_RECORD_TOO_LARGE), + LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_INFO); + + mock_clean_saved_logs(); + tor_tls_log_one_error(tls, ERR_PACK(1, 2, SSL_R_UNKNOWN_PROTOCOL), + LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_INFO); + + mock_clean_saved_logs(); + tor_tls_log_one_error(tls, ERR_PACK(1, 2, SSL_R_UNSUPPORTED_PROTOCOL), + LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_INFO); + + tls->ssl = SSL_new(ctx); + + mock_clean_saved_logs(); + tor_tls_log_one_error(tls, 0, LOG_WARN, 0, NULL); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "TLS error with 127.hello: (null)" + " (in (null):(null):" SSL_STATE_STR ")\n"); + + done: + teardown_capture_of_logs(previous_log); + SSL_free(ssl); + SSL_CTX_free(ctx); + if (tls && tls->ssl) + SSL_free(tls->ssl); + if (tls) + tor_free(tls->address); + tor_free(tls); +} + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_get_error(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + int ret; + SSL_CTX *ctx; + + SSL_library_init(); + SSL_load_error_strings(); + + ctx = SSL_CTX_new(SSLv23_method()); + int previous_log = setup_capture_of_logs(LOG_INFO); + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = SSL_new(ctx); + SSL_set_bio(tls->ssl, BIO_new(BIO_s_mem()), NULL); + + ret = tor_tls_get_error(tls, 0, 0, "something", LOG_WARN, 0); + tt_int_op(ret, OP_EQ, TOR_TLS_ERROR_IO); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "TLS error: unexpected close while" + " something (before/accept initialization)\n"); + + mock_clean_saved_logs(); + ret = tor_tls_get_error(tls, 2, 0, "something", LOG_WARN, 0); + tt_int_op(ret, OP_EQ, 0); + tt_int_op(mock_saved_log_number(), OP_EQ, 0); + + mock_clean_saved_logs(); + ret = tor_tls_get_error(tls, 0, 1, "something", LOG_WARN, 0); + tt_int_op(ret, OP_EQ, -11); + tt_int_op(mock_saved_log_number(), OP_EQ, 0); + + mock_clean_saved_logs(); + ERR_clear_error(); + ERR_put_error(ERR_LIB_BN, 2, -1, "somewhere.c", 99); + ret = tor_tls_get_error(tls, 0, 0, "something", LOG_WARN, 0); + tt_int_op(ret, OP_EQ, TOR_TLS_ERROR_MISC); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, "TLS error while something: (null)" + " (in bignum routines:(null):before/accept initialization)\n"); + + mock_clean_saved_logs(); + ERR_clear_error(); + tls->ssl->rwstate = SSL_READING; + SSL_get_rbio(tls->ssl)->flags = BIO_FLAGS_READ; + ret = tor_tls_get_error(tls, -1, 0, "something", LOG_WARN, 0); + tt_int_op(ret, OP_EQ, TOR_TLS_WANTREAD); + tt_int_op(mock_saved_log_number(), OP_EQ, 0); + + mock_clean_saved_logs(); + ERR_clear_error(); + tls->ssl->rwstate = SSL_READING; + SSL_get_rbio(tls->ssl)->flags = BIO_FLAGS_WRITE; + ret = tor_tls_get_error(tls, -1, 0, "something", LOG_WARN, 0); + tt_int_op(ret, OP_EQ, TOR_TLS_WANTWRITE); + tt_int_op(mock_saved_log_number(), OP_EQ, 0); + + mock_clean_saved_logs(); + ERR_clear_error(); + tls->ssl->rwstate = 0; + tls->ssl->shutdown = SSL_RECEIVED_SHUTDOWN; + tls->ssl->s3->warn_alert =SSL_AD_CLOSE_NOTIFY; + ret = tor_tls_get_error(tls, 0, 0, "something", LOG_WARN, 0); + tt_int_op(ret, OP_EQ, TOR_TLS_CLOSE); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + + mock_clean_saved_logs(); + ret = tor_tls_get_error(tls, 0, 2, "something", LOG_WARN, 0); + tt_int_op(ret, OP_EQ, -10); + tt_int_op(mock_saved_log_number(), OP_EQ, 0); + + mock_clean_saved_logs(); + ERR_put_error(ERR_LIB_SYS, 2, -1, "somewhere.c", 99); + ret = tor_tls_get_error(tls, -1, 0, "something", LOG_WARN, 0); + tt_int_op(ret, OP_EQ, -9); + tt_int_op(mock_saved_log_number(), OP_EQ, 2); + tt_str_op(mock_saved_log_at(1), OP_EQ, + "TLS error while something: (null) (in system library:" + "connect:before/accept initialization)\n"); + + done: + teardown_capture_of_logs(previous_log); + SSL_free(tls->ssl); + tor_free(tls); + SSL_CTX_free(ctx); +} +#endif + +static void +test_tortls_always_accept_verify_cb(void *ignored) +{ + (void)ignored; + int ret; + + ret = always_accept_verify_cb(0, NULL); + tt_int_op(ret, OP_EQ, 1); + + done: + (void)0; +} + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_x509_cert_free(void *ignored) +{ + (void)ignored; + tor_x509_cert_t *cert; + + cert = tor_malloc_zero(sizeof(tor_x509_cert_t)); + tor_x509_cert_free(cert); + + cert = tor_malloc_zero(sizeof(tor_x509_cert_t)); + cert->cert = tor_malloc_zero(sizeof(X509)); + cert->encoded = tor_malloc_zero(1); + tor_x509_cert_free(cert); +} +#endif + +static void +test_tortls_x509_cert_get_id_digests(void *ignored) +{ + (void)ignored; + tor_x509_cert_t *cert; + digests_t *d; + const digests_t *res; + cert = tor_malloc_zero(sizeof(tor_x509_cert_t)); + d = tor_malloc_zero(sizeof(digests_t)); + d->d[0][0] = 42; + + res = tor_x509_cert_get_id_digests(cert); + tt_assert(!res); + + cert->pkey_digests_set = 1; + cert->pkey_digests = *d; + res = tor_x509_cert_get_id_digests(cert); + tt_int_op(res->d[0][0], OP_EQ, 42); + + done: + tor_free(cert); + tor_free(d); +} + +#ifndef OPENSSL_OPAQUE +static int +fixed_pub_cmp(const EVP_PKEY *a, const EVP_PKEY *b) +{ + (void) a; (void) b; + return 1; +} + +static void +fake_x509_free(X509 *cert) +{ + if (cert) { + if (cert->cert_info) { + if (cert->cert_info->key) { + if (cert->cert_info->key->pkey) { + tor_free(cert->cert_info->key->pkey); + } + tor_free(cert->cert_info->key); + } + tor_free(cert->cert_info); + } + tor_free(cert); + } +} + +static void +test_tortls_cert_matches_key(void *ignored) +{ + (void)ignored; + int res; + tor_tls_t *tls; + tor_x509_cert_t *cert; + X509 *one = NULL, *two = NULL; + EVP_PKEY_ASN1_METHOD *meth = EVP_PKEY_asn1_new(999, 0, NULL, NULL); + EVP_PKEY_asn1_set_public(meth, NULL, NULL, fixed_pub_cmp, NULL, NULL, NULL); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + cert = tor_malloc_zero(sizeof(tor_x509_cert_t)); + one = tor_malloc_zero(sizeof(X509)); + one->references = 1; + two = tor_malloc_zero(sizeof(X509)); + two->references = 1; + + res = tor_tls_cert_matches_key(tls, cert); + tt_int_op(res, OP_EQ, 0); + + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->session = tor_malloc_zero(sizeof(SSL_SESSION)); + tls->ssl->session->peer = one; + res = tor_tls_cert_matches_key(tls, cert); + tt_int_op(res, OP_EQ, 0); + + cert->cert = two; + res = tor_tls_cert_matches_key(tls, cert); + tt_int_op(res, OP_EQ, 0); + + one->cert_info = tor_malloc_zero(sizeof(X509_CINF)); + one->cert_info->key = tor_malloc_zero(sizeof(X509_PUBKEY)); + one->cert_info->key->pkey = tor_malloc_zero(sizeof(EVP_PKEY)); + one->cert_info->key->pkey->references = 1; + one->cert_info->key->pkey->ameth = meth; + one->cert_info->key->pkey->type = 1; + + two->cert_info = tor_malloc_zero(sizeof(X509_CINF)); + two->cert_info->key = tor_malloc_zero(sizeof(X509_PUBKEY)); + two->cert_info->key->pkey = tor_malloc_zero(sizeof(EVP_PKEY)); + two->cert_info->key->pkey->references = 1; + two->cert_info->key->pkey->ameth = meth; + two->cert_info->key->pkey->type = 2; + + res = tor_tls_cert_matches_key(tls, cert); + tt_int_op(res, OP_EQ, 0); + + one->cert_info->key->pkey->type = 1; + two->cert_info->key->pkey->type = 1; + res = tor_tls_cert_matches_key(tls, cert); + tt_int_op(res, OP_EQ, 1); + + done: + EVP_PKEY_asn1_free(meth); + tor_free(tls->ssl->session); + tor_free(tls->ssl); + tor_free(tls); + tor_free(cert); + fake_x509_free(one); + fake_x509_free(two); +} + +static void +test_tortls_cert_get_key(void *ignored) +{ + (void)ignored; + tor_x509_cert_t *cert = NULL; + crypto_pk_t *res = NULL; + cert = tor_malloc_zero(sizeof(tor_x509_cert_t)); + X509 *key = NULL; + key = tor_malloc_zero(sizeof(X509)); + key->references = 1; + + res = tor_tls_cert_get_key(cert); + tt_assert(!res); + + cert->cert = key; + key->cert_info = tor_malloc_zero(sizeof(X509_CINF)); + key->cert_info->key = tor_malloc_zero(sizeof(X509_PUBKEY)); + key->cert_info->key->pkey = tor_malloc_zero(sizeof(EVP_PKEY)); + key->cert_info->key->pkey->references = 1; + key->cert_info->key->pkey->type = 2; + res = tor_tls_cert_get_key(cert); + tt_assert(!res); + + done: + fake_x509_free(key); + tor_free(cert); + crypto_pk_free(res); +} +#endif + +static void +test_tortls_get_my_client_auth_key(void *ignored) +{ + (void)ignored; + crypto_pk_t *ret; + crypto_pk_t *expected; + tor_tls_context_t *ctx; + RSA *k = tor_malloc_zero(sizeof(RSA)); + + ctx = tor_malloc_zero(sizeof(tor_tls_context_t)); + expected = crypto_new_pk_from_rsa_(k); + ctx->auth_key = expected; + + client_tls_context = NULL; + ret = tor_tls_get_my_client_auth_key(); + tt_assert(!ret); + + client_tls_context = ctx; + ret = tor_tls_get_my_client_auth_key(); + tt_assert(ret == expected); + + done: + tor_free(expected); + tor_free(k); + tor_free(ctx); +} + +static void +test_tortls_get_my_certs(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_context_t *ctx; + const tor_x509_cert_t *link_cert_out = NULL; + const tor_x509_cert_t *id_cert_out = NULL; + + ctx = tor_malloc_zero(sizeof(tor_tls_context_t)); + + client_tls_context = NULL; + ret = tor_tls_get_my_certs(0, NULL, NULL); + tt_int_op(ret, OP_EQ, -1); + + server_tls_context = NULL; + ret = tor_tls_get_my_certs(1, NULL, NULL); + tt_int_op(ret, OP_EQ, -1); + + client_tls_context = ctx; + ret = tor_tls_get_my_certs(0, NULL, NULL); + tt_int_op(ret, OP_EQ, 0); + + client_tls_context = ctx; + ret = tor_tls_get_my_certs(0, &link_cert_out, &id_cert_out); + tt_int_op(ret, OP_EQ, 0); + + server_tls_context = ctx; + ret = tor_tls_get_my_certs(1, &link_cert_out, &id_cert_out); + tt_int_op(ret, OP_EQ, 0); + + done: + (void)1; +} + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_get_ciphersuite_name(void *ignored) +{ + (void)ignored; + const char *ret; + tor_tls_t *ctx; + ctx = tor_malloc_zero(sizeof(tor_tls_t)); + ctx->ssl = tor_malloc_zero(sizeof(SSL)); + + ret = tor_tls_get_ciphersuite_name(ctx); + tt_str_op(ret, OP_EQ, "(NONE)"); + + done: + tor_free(ctx->ssl); + tor_free(ctx); +} + +static SSL_CIPHER * +get_cipher_by_name(const char *name) +{ + int i; + const SSL_METHOD *method = SSLv23_method(); + int num = method->num_ciphers(); + for (i = 0; i < num; ++i) { + const SSL_CIPHER *cipher = method->get_cipher(i); + const char *ciphername = SSL_CIPHER_get_name(cipher); + if (!strcmp(ciphername, name)) { + return (SSL_CIPHER *)cipher; + } + } + + return NULL; +} + +static SSL_CIPHER * +get_cipher_by_id(uint16_t id) +{ + int i; + const SSL_METHOD *method = SSLv23_method(); + int num = method->num_ciphers(); + for (i = 0; i < num; ++i) { + const SSL_CIPHER *cipher = method->get_cipher(i); + if (id == (SSL_CIPHER_get_id(cipher) & 0xffff)) { + return (SSL_CIPHER *)cipher; + } + } + + return NULL; +} + +extern uint16_t v2_cipher_list[]; + +static void +test_tortls_classify_client_ciphers(void *ignored) +{ + (void)ignored; + int i; + int ret; + SSL_CTX *ctx; + SSL *ssl; + tor_tls_t *tls; + STACK_OF(SSL_CIPHER) *ciphers; + SSL_CIPHER *tmp_cipher; + + SSL_library_init(); + SSL_load_error_strings(); + tor_tls_allocate_tor_tls_object_ex_data_index(); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->magic = TOR_TLS_MAGIC; + + ctx = SSL_CTX_new(TLSv1_method()); + ssl = SSL_new(ctx); + tls->ssl = ssl; + + ciphers = sk_SSL_CIPHER_new_null(); + + ret = tor_tls_classify_client_ciphers(ssl, NULL); + tt_int_op(ret, OP_EQ, -1); + + SSL_set_ex_data(ssl, tor_tls_object_ex_data_index, tls); + tls->client_cipher_list_type = 42; + + ret = tor_tls_classify_client_ciphers(ssl, NULL); + tt_int_op(ret, OP_EQ, 42); + + tls->client_cipher_list_type = 0; + ret = tor_tls_classify_client_ciphers(ssl, ciphers); + tt_int_op(ret, OP_EQ, 1); + tt_int_op(tls->client_cipher_list_type, OP_EQ, 1); + + tls->client_cipher_list_type = 0; + ret = tor_tls_classify_client_ciphers(ssl, SSL_get_ciphers(ssl)); + tt_int_op(ret, OP_EQ, 3); + tt_int_op(tls->client_cipher_list_type, OP_EQ, 3); + + SSL_CIPHER *one = get_cipher_by_name(TLS1_TXT_DHE_RSA_WITH_AES_128_SHA), + *two = get_cipher_by_name(TLS1_TXT_DHE_RSA_WITH_AES_256_SHA), + *three = get_cipher_by_name(SSL3_TXT_EDH_RSA_DES_192_CBC3_SHA), + *four = NULL; + sk_SSL_CIPHER_push(ciphers, one); + sk_SSL_CIPHER_push(ciphers, two); + sk_SSL_CIPHER_push(ciphers, three); + sk_SSL_CIPHER_push(ciphers, four); + + tls->client_cipher_list_type = 0; + ret = tor_tls_classify_client_ciphers(ssl, ciphers); + tt_int_op(ret, OP_EQ, 1); + tt_int_op(tls->client_cipher_list_type, OP_EQ, 1); + + sk_SSL_CIPHER_zero(ciphers); + + one = get_cipher_by_name("ECDH-RSA-AES256-GCM-SHA384"); + one->id = 0x00ff; + two = get_cipher_by_name("ECDH-RSA-AES128-GCM-SHA256"); + two->id = 0x0000; + sk_SSL_CIPHER_push(ciphers, one); + tls->client_cipher_list_type = 0; + ret = tor_tls_classify_client_ciphers(ssl, ciphers); + tt_int_op(ret, OP_EQ, 3); + tt_int_op(tls->client_cipher_list_type, OP_EQ, 3); + + sk_SSL_CIPHER_push(ciphers, two); + tls->client_cipher_list_type = 0; + ret = tor_tls_classify_client_ciphers(ssl, ciphers); + tt_int_op(ret, OP_EQ, 3); + tt_int_op(tls->client_cipher_list_type, OP_EQ, 3); + + one->id = 0xC00A; + tls->client_cipher_list_type = 0; + ret = tor_tls_classify_client_ciphers(ssl, ciphers); + tt_int_op(ret, OP_EQ, 3); + tt_int_op(tls->client_cipher_list_type, OP_EQ, 3); + + sk_SSL_CIPHER_zero(ciphers); + for (i=0; v2_cipher_list[i]; i++) { + tmp_cipher = get_cipher_by_id(v2_cipher_list[i]); + tt_assert(tmp_cipher); + sk_SSL_CIPHER_push(ciphers, tmp_cipher); + } + tls->client_cipher_list_type = 0; + ret = tor_tls_classify_client_ciphers(ssl, ciphers); + tt_int_op(ret, OP_EQ, 2); + tt_int_op(tls->client_cipher_list_type, OP_EQ, 2); + + done: + sk_SSL_CIPHER_free(ciphers); + SSL_free(tls->ssl); + tor_free(tls); + SSL_CTX_free(ctx); +} +#endif + +static void +test_tortls_client_is_using_v2_ciphers(void *ignored) +{ + (void)ignored; + +#ifdef HAVE_SSL_GET_CLIENT_CIPHERS + tt_skip(); + done: + (void)1; +#else + int ret; + SSL_CTX *ctx; + SSL *ssl; + SSL_SESSION *sess; + STACK_OF(SSL_CIPHER) *ciphers; + + SSL_library_init(); + SSL_load_error_strings(); + + ctx = SSL_CTX_new(TLSv1_method()); + ssl = SSL_new(ctx); + sess = SSL_SESSION_new(); + + ret = tor_tls_client_is_using_v2_ciphers(ssl); + tt_int_op(ret, OP_EQ, -1); + + ssl->session = sess; + ret = tor_tls_client_is_using_v2_ciphers(ssl); + tt_int_op(ret, OP_EQ, 0); + + ciphers = sk_SSL_CIPHER_new_null(); + SSL_CIPHER *one = get_cipher_by_name("ECDH-RSA-AES256-GCM-SHA384"); + one->id = 0x00ff; + sk_SSL_CIPHER_push(ciphers, one); + sess->ciphers = ciphers; + ret = tor_tls_client_is_using_v2_ciphers(ssl); + tt_int_op(ret, OP_EQ, 1); + done: + SSL_free(ssl); + SSL_CTX_free(ctx); +#endif +} + +#ifndef OPENSSL_OPAQUE +static X509 *fixed_try_to_extract_certs_from_tls_cert_out_result = NULL; +static X509 *fixed_try_to_extract_certs_from_tls_id_cert_out_result = NULL; + +static void +fixed_try_to_extract_certs_from_tls(int severity, tor_tls_t *tls, + X509 **cert_out, X509 **id_cert_out) +{ + (void) severity; + (void) tls; + *cert_out = fixed_try_to_extract_certs_from_tls_cert_out_result; + *id_cert_out = fixed_try_to_extract_certs_from_tls_id_cert_out_result; +} +#endif + +#ifndef OPENSSL_OPAQUE +static const char* notCompletelyValidCertString = + "-----BEGIN CERTIFICATE-----\n" + "MIICVjCCAb8CAg37MA0GCSqGSIb3DQEBBQUAMIGbMQswCQYDVQQGEwJKUDEOMAwG\n" + "A1UECBMFVG9reW8xEDAOBgNVBAcTB0NodW8ta3UxETAPBgNVBAoTCEZyYW5rNERE\n" + "MRgwFgYDVQQLEw9XZWJDZXJ0IFN1cHBvcnQxGDAWBgNVBAMTD0ZyYW5rNEREIFdl\n" + "YiBDQTEjMCEGCSqGSIb3DQEJARYUc3VwcG9ydEBmcmFuazRkZC5jb20wHhcNMTIw\n" + "ODIyMDUyNzIzWhcNMTcwODIxMDUyNzIzWjBKMQswCQYDVQQGEwJKUDEOMAwGA1UE\n" + "CAwFVG9reW8xETAPBgNVBAoMCEZyYW5rNEREMRgwFgYDVQQDDA93d3cuZXhhbXBs\n" + "ZS5jb20wgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMYBBrx5PlP0WNI/ZdzD\n" + "+6Pktmurn+F2kQYbtc7XQh8/LTBvCo+P6iZoLEmUA9e7EXLRxgU1CVqeAi7QcAn9\n" + "MwBlc8ksFJHB0rtf9pmf8Oza9E0Bynlq/4/Kb1x+d+AyhL7oK9tQwB24uHOueHi1\n" + "C/iVv8CSWKiYe6hzN1txYe8rAgMBAAEwDQYJKoZIhvcNAQEFBQADgYEAASPdjigJ\n" + "kXCqKWpnZ/Oc75EUcMi6HztaW8abUMlYXPIgkV2F7YanHOB7K4f7OOLjiz8DTPFf\n" + "jC9UeuErhaA/zzWi8ewMTFZW/WshOrm3fNvcMrMLKtH534JKvcdMg6qIdjTFINIr\n" + "evnAhf0cwULaebn+lMs8Pdl7y37+sfluVok=\n" + "-----END CERTIFICATE-----\n"; +#endif + +static const char* validCertString = "-----BEGIN CERTIFICATE-----\n" + "MIIDpTCCAY0CAg3+MA0GCSqGSIb3DQEBBQUAMF4xCzAJBgNVBAYTAlVTMREwDwYD\n" + "VQQIDAhJbGxpbm9pczEQMA4GA1UEBwwHQ2hpY2FnbzEUMBIGA1UECgwLVG9yIFRl\n" + "c3RpbmcxFDASBgNVBAMMC1RvciBUZXN0aW5nMB4XDTE1MDkwNjEzMzk1OVoXDTQz\n" + "MDEyMjEzMzk1OVowVjELMAkGA1UEBhMCVVMxEDAOBgNVBAcMB0NoaWNhZ28xFDAS\n" + "BgNVBAoMC1RvciBUZXN0aW5nMR8wHQYDVQQDDBZ0ZXN0aW5nLnRvcnByb2plY3Qu\n" + "b3JnMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDoT6uyVVhWyOF3wkHjjYbd\n" + "nKaykyRv4JVtKQdZ4OpEErmX1zw4MmyzpQNV6iR4bQnWiyLfzyVJMZDIC/WILBfX\n" + "w2Pza/yuLgUvDc3twMuhOACzOQVO8PrEF/aVv2+hbCCy2udXvKhnYn+CCXl3ozc8\n" + "XcKYvujTXDyvGWY3xwAjlQIDAQABMA0GCSqGSIb3DQEBBQUAA4ICAQCUvnhzQWuQ\n" + "MrN+pERkE+zcTI/9dGS90rUMMLgu8VDNqTa0TUQh8uO0EQ6uDvI8Js6e8tgwS0BR\n" + "UBahqb7ZHv+rejGCBr5OudqD+x4STiiuPNJVs86JTLN8SpM9CHjIBH5WCCN2KOy3\n" + "mevNoRcRRyYJzSFULCunIK6FGulszigMYGscrO4oiTkZiHPh9KvWT40IMiHfL+Lw\n" + "EtEWiLex6064LcA2YQ1AMuSZyCexks63lcfaFmQbkYOKqXa1oLkIRuDsOaSVjTfe\n" + "vec+X6jvf12cFTKS5WIeqkKF2Irt+dJoiHEGTe5RscUMN/f+gqHPzfFz5dR23sxo\n" + "g+HC6MZHlFkLAOx3wW6epPS8A/m1mw3zMPoTnb2U2YYt8T0dJMMlUn/7Y1sEAa+a\n" + "dSTMaeUf6VnJ//11m454EZl1to9Z7oJOgqmFffSrdD4BGIWe8f7hhW6L1Enmqe/J\n" + "BKL3wbzZh80O1W0bndAwhnEEhlzneFY84cbBo9pmVxpODHkUcStpr5Z7pBDrcL21\n" + "Ss/aB/1YrsVXhdvJdOGxl3Mnl9dUY57CympLGlT8f0pPS6GAKOelECOhFMHmJd8L\n" + "dj3XQSmKtYHevZ6IvuMXSlB/fJvSjSlkCuLo5+kJoaqPuRu+i/S1qxeRy3CBwmnE\n" + "LdSNdcX4N79GQJ996PA8+mUCQG7YRtK+WA==\n" + "-----END CERTIFICATE-----\n"; + +static const char* caCertString = "-----BEGIN CERTIFICATE-----\n" + "MIIFjzCCA3egAwIBAgIJAKd5WgyfPMYRMA0GCSqGSIb3DQEBCwUAMF4xCzAJBgNV\n" + "BAYTAlVTMREwDwYDVQQIDAhJbGxpbm9pczEQMA4GA1UEBwwHQ2hpY2FnbzEUMBIG\n" + "A1UECgwLVG9yIFRlc3RpbmcxFDASBgNVBAMMC1RvciBUZXN0aW5nMB4XDTE1MDkw\n" + "NjEzMzc0MVoXDTQzMDEyMjEzMzc0MVowXjELMAkGA1UEBhMCVVMxETAPBgNVBAgM\n" + "CElsbGlub2lzMRAwDgYDVQQHDAdDaGljYWdvMRQwEgYDVQQKDAtUb3IgVGVzdGlu\n" + "ZzEUMBIGA1UEAwwLVG9yIFRlc3RpbmcwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAw\n" + "ggIKAoICAQCpLMUEiLW5leUgBZoEJms2V7lZRhIAjnJBhVMHD0e3UubNknmaQoxf\n" + "ARz3rvqOaRd0JlV+qM9qE0DjiYcCVP1cAfqAo9d83uS1vwY3YMVJzADlaIiHfyVW\n" + "uEgBy0vvkeUBqaua24dYlcwsemOiXYLu41yM1wkcGHW1AhBNHppY6cznb8TyLgNM\n" + "2x3SGUdzc5XMyAFx51faKGBA3wjs+Hg1PLY7d30nmCgEOBavpm5I1disM/0k+Mcy\n" + "YmAKEo/iHJX/rQzO4b9znP69juLlR8PDBUJEVIG/CYb6+uw8MjjUyiWXYoqfVmN2\n" + "hm/lH8b6rXw1a2Aa3VTeD0DxaWeacMYHY/i01fd5n7hCoDTRNdSw5KJ0L3Z0SKTu\n" + "0lzffKzDaIfyZGlpW5qdouACkWYzsaitQOePVE01PIdO30vUfzNTFDfy42ccx3Di\n" + "59UCu+IXB+eMtrBfsok0Qc63vtF1linJgjHW1z/8ujk8F7/qkOfODhk4l7wngc2A\n" + "EmwWFIFoGaiTEZHB9qteXr4unbXZ0AHpM02uGGwZEGohjFyebEb73M+J57WKKAFb\n" + "PqbLcGUksL1SHNBNAJcVLttX55sO4nbidOS/kA3m+F1R04MBTyQF9qA6YDDHqdI3\n" + "h/3pw0Z4fxVouTYT4/NfRnX4JTP4u+7Mpcoof28VME0qWqD1LnRhFQIDAQABo1Aw\n" + "TjAdBgNVHQ4EFgQUMoAgIXH7pZ3QMRwTjT+DM9Yo/v0wHwYDVR0jBBgwFoAUMoAg\n" + "IXH7pZ3QMRwTjT+DM9Yo/v0wDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOC\n" + "AgEAUJxacjXR9sT+Xs6ISFiUsyd0T6WVKMnV46xrYJHirGfx+krWHrjxMY+ZtxYD\n" + "DBDGlo11Qc4v6QrclNf5QUBfIiGQsP9Cm6hHcQ+Tpg9HHCgSqG1YNPwCPReCR4br\n" + "BLvLfrfkcBL2IWM0PdQdCze+59DBfipsULD2mEn9fjYRXQEwb2QWtQ9qRc20Yb/x\n" + "Q4b/+CvUodLkaq7B8MHz0BV8HHcBoph6DYaRmO/N+hPauIuSp6XyaGYcEefGKVKj\n" + "G2+fcsdyXsoijNdL8vNKwm4j2gVwCBnw16J00yfFoV46YcbfqEdJB2je0XSvwXqt\n" + "14AOTngxso2h9k9HLtrfpO1ZG/B5AcCMs1lzbZ2fp5DPHtjvvmvA2RJqgo3yjw4W\n" + "4DHAuTglYFlC3mDHNfNtcGP20JvepcQNzNP2UzwcpOc94hfKikOFw+gf9Vf1qd0y\n" + "h/Sk6OZHn2+JVUPiWHIQV98Vtoh4RmUZDJD+b55ia3fQGTGzt4z1XFzQYSva5sfs\n" + "wocS/papthqWldQU7x+3wofNd5CNU1x6WKXG/yw30IT/4F8ADJD6GeygNT8QJYvt\n" + "u/8lAkbOy6B9xGmSvr0Kk1oq9P2NshA6kalxp1Oz/DTNDdL4AeBXV3JmM6WWCjGn\n" + "Yy1RT69d0rwYc5u/vnqODz1IjvT90smsrkBumGt791FAFeg=\n" + "-----END CERTIFICATE-----\n"; + +static X509 * +read_cert_from(const char *str) +{ + BIO *bio = BIO_new(BIO_s_mem()); + BIO_write(bio, str, (int) strlen(str)); + X509 *res = PEM_read_bio_X509(bio, NULL, NULL, NULL); + BIO_free(bio); + return res; +} + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_verify(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + crypto_pk_t *k = NULL; + X509 *cert1 = NULL, *cert2 = NULL, *invalidCert = NULL, + *validCert = NULL, *caCert = NULL; + + cert1 = tor_malloc_zero(sizeof(X509)); + cert1->references = 10; + + cert2 = tor_malloc_zero(sizeof(X509)); + cert2->references = 10; + + validCert = read_cert_from(validCertString); + caCert = read_cert_from(caCertString); + invalidCert = read_cert_from(notCompletelyValidCertString); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + ret = tor_tls_verify(LOG_WARN, tls, &k); + tt_int_op(ret, OP_EQ, -1); + + MOCK(try_to_extract_certs_from_tls, fixed_try_to_extract_certs_from_tls); + + fixed_try_to_extract_certs_from_tls_cert_out_result = cert1; + ret = tor_tls_verify(LOG_WARN, tls, &k); + tt_int_op(ret, OP_EQ, -1); + + fixed_try_to_extract_certs_from_tls_id_cert_out_result = cert2; + ret = tor_tls_verify(LOG_WARN, tls, &k); + tt_int_op(ret, OP_EQ, -1); + + fixed_try_to_extract_certs_from_tls_cert_out_result = invalidCert; + fixed_try_to_extract_certs_from_tls_id_cert_out_result = invalidCert; + + ret = tor_tls_verify(LOG_WARN, tls, &k); + tt_int_op(ret, OP_EQ, -1); + + fixed_try_to_extract_certs_from_tls_cert_out_result = validCert; + fixed_try_to_extract_certs_from_tls_id_cert_out_result = caCert; + + ret = tor_tls_verify(LOG_WARN, tls, &k); + tt_int_op(ret, OP_EQ, 0); + tt_assert(k); + + done: + UNMOCK(try_to_extract_certs_from_tls); + tor_free(cert1); + tor_free(cert2); + tor_free(tls); + tor_free(k); +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_check_lifetime(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + X509 *validCert = read_cert_from(validCertString); + time_t now = time(NULL); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + ret = tor_tls_check_lifetime(LOG_WARN, tls, 0, 0); + tt_int_op(ret, OP_EQ, -1); + + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->session = tor_malloc_zero(sizeof(SSL_SESSION)); + tls->ssl->session->peer = validCert; + ret = tor_tls_check_lifetime(LOG_WARN, tls, 0, 0); + tt_int_op(ret, OP_EQ, 0); + + validCert->cert_info->validity->notBefore = ASN1_TIME_set(NULL, now-10); + validCert->cert_info->validity->notAfter = ASN1_TIME_set(NULL, now+60); + + ret = tor_tls_check_lifetime(LOG_WARN, tls, 0, -1000); + tt_int_op(ret, OP_EQ, -1); + + ret = tor_tls_check_lifetime(LOG_WARN, tls, -1000, 0); + tt_int_op(ret, OP_EQ, -1); + + done: + tor_free(tls->ssl->session); + tor_free(tls->ssl); + tor_free(tls); +} +#endif + +#ifndef OPENSSL_OPAQUE +static int fixed_ssl_pending_result = 0; + +static int +fixed_ssl_pending(const SSL *ignored) +{ + (void)ignored; + return fixed_ssl_pending_result; +} + +static void +test_tortls_get_pending_bytes(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + SSL_METHOD *method; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + method = tor_malloc_zero(sizeof(SSL_METHOD)); + method->ssl_pending = fixed_ssl_pending; + tls->ssl->method = method; + + fixed_ssl_pending_result = 42; + ret = tor_tls_get_pending_bytes(tls); + tt_int_op(ret, OP_EQ, 42); + + done: + tor_free(method); + tor_free(tls->ssl); + tor_free(tls); +} +#endif + +static void +test_tortls_get_forced_write_size(void *ignored) +{ + (void)ignored; + long ret; + tor_tls_t *tls; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + + tls->wantwrite_n = 43; + ret = tor_tls_get_forced_write_size(tls); + tt_int_op(ret, OP_EQ, 43); + + done: + tor_free(tls); +} + +extern uint64_t total_bytes_written_over_tls; +extern uint64_t total_bytes_written_by_tls; + +static void +test_tortls_get_write_overhead_ratio(void *ignored) +{ + (void)ignored; + double ret; + + total_bytes_written_over_tls = 0; + ret = tls_get_write_overhead_ratio(); + tt_int_op(ret, OP_EQ, 1.0); + + total_bytes_written_by_tls = 10; + total_bytes_written_over_tls = 1; + ret = tls_get_write_overhead_ratio(); + tt_int_op(ret, OP_EQ, 10.0); + + total_bytes_written_by_tls = 10; + total_bytes_written_over_tls = 2; + ret = tls_get_write_overhead_ratio(); + tt_int_op(ret, OP_EQ, 5.0); + + done: + (void)0; +} + +static void +test_tortls_used_v1_handshake(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + tls = tor_malloc_zero(sizeof(tor_tls_t)); + + // These tests assume both V2 handshake server and client are enabled + tls->wasV2Handshake = 0; + ret = tor_tls_used_v1_handshake(tls); + tt_int_op(ret, OP_EQ, 1); + + tls->wasV2Handshake = 1; + ret = tor_tls_used_v1_handshake(tls); + tt_int_op(ret, OP_EQ, 0); + + done: + tor_free(tls); +} + +static void +test_tortls_get_num_server_handshakes(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + + tls->server_handshake_count = 3; + ret = tor_tls_get_num_server_handshakes(tls); + tt_int_op(ret, OP_EQ, 3); + + done: + tor_free(tls); +} + +static void +test_tortls_server_got_renegotiate(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + + tls->got_renegotiate = 1; + ret = tor_tls_server_got_renegotiate(tls); + tt_int_op(ret, OP_EQ, 1); + + done: + tor_free(tls); +} + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_SSL_SESSION_get_master_key(void *ignored) +{ + (void)ignored; + size_t ret; + tor_tls_t *tls; + uint8_t *out; + out = tor_malloc_zero(1); + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->session = tor_malloc_zero(sizeof(SSL_SESSION)); + tls->ssl->session->master_key_length = 1; + +#ifndef HAVE_SSL_SESSION_GET_MASTER_KEY + tls->ssl->session->master_key[0] = 43; + ret = SSL_SESSION_get_master_key(tls->ssl->session, out, 0); + tt_int_op(ret, OP_EQ, 1); + tt_int_op(out[0], OP_EQ, 0); + + ret = SSL_SESSION_get_master_key(tls->ssl->session, out, 1); + tt_int_op(ret, OP_EQ, 1); + tt_int_op(out[0], OP_EQ, 43); + + done: +#endif + tor_free(tls->ssl->session); + tor_free(tls->ssl); + tor_free(tls); + tor_free(out); +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_get_tlssecrets(void *ignored) +{ + (void)ignored; + int ret; + uint8_t *secret_out = tor_malloc_zero(DIGEST256_LEN);; + tor_tls_t *tls; + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->session = tor_malloc_zero(sizeof(SSL_SESSION)); + tls->ssl->session->master_key_length = 1; + tls->ssl->s3 = tor_malloc_zero(sizeof(SSL3_STATE)); + + ret = tor_tls_get_tlssecrets(tls, secret_out); + tt_int_op(ret, OP_EQ, 0); + + done: + tor_free(secret_out); + tor_free(tls->ssl->s3); + tor_free(tls->ssl->session); + tor_free(tls->ssl); + tor_free(tls); +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_get_buffer_sizes(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + size_t rbuf_c=-1, rbuf_b=-1, wbuf_c=-1, wbuf_b=-1; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->s3 = tor_malloc_zero(sizeof(SSL3_STATE)); + + tls->ssl->s3->rbuf.buf = NULL; + tls->ssl->s3->rbuf.len = 1; + tls->ssl->s3->rbuf.offset = 0; + tls->ssl->s3->rbuf.left = 42; + + tls->ssl->s3->wbuf.buf = NULL; + tls->ssl->s3->wbuf.len = 2; + tls->ssl->s3->wbuf.offset = 0; + tls->ssl->s3->wbuf.left = 43; + +#if OPENSSL_VERSION_NUMBER >= OPENSSL_V_SERIES(1,1,0) + ret = tor_tls_get_buffer_sizes(NULL, NULL, NULL, NULL, NULL); + tt_int_op(ret, OP_EQ, -1); +#else + ret = tor_tls_get_buffer_sizes(tls, &rbuf_c, &rbuf_b, &wbuf_c, &wbuf_b); + tt_int_op(ret, OP_EQ, 0); + tt_int_op(rbuf_c, OP_EQ, 0); + tt_int_op(wbuf_c, OP_EQ, 0); + tt_int_op(rbuf_b, OP_EQ, 42); + tt_int_op(wbuf_b, OP_EQ, 43); + + tls->ssl->s3->rbuf.buf = tor_malloc_zero(1); + tls->ssl->s3->wbuf.buf = tor_malloc_zero(1); + ret = tor_tls_get_buffer_sizes(tls, &rbuf_c, &rbuf_b, &wbuf_c, &wbuf_b); + tt_int_op(ret, OP_EQ, 0); + tt_int_op(rbuf_c, OP_EQ, 1); + tt_int_op(wbuf_c, OP_EQ, 2); + +#endif + + done: + tor_free(tls->ssl->s3->rbuf.buf); + tor_free(tls->ssl->s3->wbuf.buf); + tor_free(tls->ssl->s3); + tor_free(tls->ssl); + tor_free(tls); +} +#endif + +static void +test_tortls_evaluate_ecgroup_for_tls(void *ignored) +{ + (void)ignored; + int ret; + + ret = evaluate_ecgroup_for_tls(NULL); + tt_int_op(ret, OP_EQ, 1); + + ret = evaluate_ecgroup_for_tls("foobar"); + tt_int_op(ret, OP_EQ, 0); + + ret = evaluate_ecgroup_for_tls("P256"); + tt_int_op(ret, OP_EQ, 1); + + ret = evaluate_ecgroup_for_tls("P224"); + // tt_int_op(ret, OP_EQ, 1); This varies between machines + + done: + (void)0; +} + +#ifndef OPENSSL_OPAQUE +typedef struct cert_pkey_st_local +{ + X509 *x509; + EVP_PKEY *privatekey; + const EVP_MD *digest; +} CERT_PKEY_local; + +typedef struct sess_cert_st_local +{ + STACK_OF(X509) *cert_chain; + int peer_cert_type; + CERT_PKEY_local *peer_key; + CERT_PKEY_local peer_pkeys[8]; + int references; +} SESS_CERT_local; + +static void +test_tortls_try_to_extract_certs_from_tls(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + X509 *cert = NULL, *id_cert = NULL, *c1 = NULL, *c2 = NULL; + SESS_CERT_local *sess = NULL; + + c1 = read_cert_from(validCertString); + c2 = read_cert_from(caCertString); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->session = tor_malloc_zero(sizeof(SSL_SESSION)); + sess = tor_malloc_zero(sizeof(SESS_CERT_local)); + tls->ssl->session->sess_cert = (void *)sess; + + try_to_extract_certs_from_tls(LOG_WARN, tls, &cert, &id_cert); + tt_assert(!cert); + tt_assert(!id_cert); + + tls->ssl->session->peer = c1; + try_to_extract_certs_from_tls(LOG_WARN, tls, &cert, &id_cert); + tt_assert(cert == c1); + tt_assert(!id_cert); + + sess->cert_chain = sk_X509_new_null(); + try_to_extract_certs_from_tls(LOG_WARN, tls, &cert, &id_cert); + tt_assert(cert == c1); + tt_assert(!id_cert); + + sk_X509_push(sess->cert_chain, c1); + sk_X509_push(sess->cert_chain, c2); + try_to_extract_certs_from_tls(LOG_WARN, tls, &cert, &id_cert); + tt_assert(cert == c1); + tt_assert(id_cert); + + done: + tor_free(sess); + tor_free(tls->ssl->session); + tor_free(tls->ssl); + tor_free(tls); +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_get_peer_cert(void *ignored) +{ + (void)ignored; + tor_x509_cert_t *ret; + tor_tls_t *tls; + X509 *cert = NULL; + + cert = read_cert_from(validCertString); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->session = tor_malloc_zero(sizeof(SSL_SESSION)); + + ret = tor_tls_get_peer_cert(tls); + tt_assert(!ret); + + tls->ssl->session->peer = cert; + ret = tor_tls_get_peer_cert(tls); + tt_assert(ret); + tt_assert(ret->cert == cert); + + done: + tor_x509_cert_free(ret); + tor_free(tls->ssl->session); + tor_free(tls->ssl); + tor_free(tls); + X509_free(cert); +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_peer_has_cert(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + X509 *cert = NULL; + + cert = read_cert_from(validCertString); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->session = tor_malloc_zero(sizeof(SSL_SESSION)); + + ret = tor_tls_peer_has_cert(tls); + tt_assert(!ret); + + tls->ssl->session->peer = cert; + ret = tor_tls_peer_has_cert(tls); + tt_assert(ret); + + done: + tor_free(tls->ssl->session); + tor_free(tls->ssl); + tor_free(tls); + X509_free(cert); +} +#endif + +static void +test_tortls_is_server(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + int ret; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->isServer = 1; + ret = tor_tls_is_server(tls); + tt_int_op(ret, OP_EQ, 1); + + done: + tor_free(tls); +} + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_session_secret_cb(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + SSL_CTX *ctx; + STACK_OF(SSL_CIPHER) *ciphers = NULL; + SSL_CIPHER *one; + + SSL_library_init(); + SSL_load_error_strings(); + tor_tls_allocate_tor_tls_object_ex_data_index(); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + + tls->magic = TOR_TLS_MAGIC; + + ctx = SSL_CTX_new(TLSv1_method()); + tls->ssl = SSL_new(ctx); + SSL_set_ex_data(tls->ssl, tor_tls_object_ex_data_index, tls); + + SSL_set_session_secret_cb(tls->ssl, tor_tls_session_secret_cb, NULL); + + tor_tls_session_secret_cb(tls->ssl, NULL, NULL, NULL, NULL, NULL); + tt_assert(!tls->ssl->tls_session_secret_cb); + + one = get_cipher_by_name("ECDH-RSA-AES256-GCM-SHA384"); + one->id = 0x00ff; + ciphers = sk_SSL_CIPHER_new_null(); + sk_SSL_CIPHER_push(ciphers, one); + + tls->client_cipher_list_type = 0; + tor_tls_session_secret_cb(tls->ssl, NULL, NULL, ciphers, NULL, NULL); + tt_assert(!tls->ssl->tls_session_secret_cb); + + done: + sk_SSL_CIPHER_free(ciphers); + SSL_free(tls->ssl); + SSL_CTX_free(ctx); + tor_free(tls); +} +#endif + +#ifndef OPENSSL_OPAQUE +/* TODO: It seems block_renegotiation and unblock_renegotiation and + * using different blags. This might not be correct */ +static void +test_tortls_block_renegotiation(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->s3 = tor_malloc_zero(sizeof(SSL3_STATE)); + tls->ssl->s3->flags = 0x0010; + + tor_tls_block_renegotiation(tls); + + tt_assert(!(SSL_get_options(tls->ssl) & 0x0010)); + + done: + tor_free(tls->ssl->s3); + tor_free(tls->ssl); + tor_free(tls); +} + +static void +test_tortls_unblock_renegotiation(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tor_tls_unblock_renegotiation(tls); + + tt_assert(SSL_get_options(tls->ssl) & 0x00040000L); + + done: + tor_free(tls->ssl); + tor_free(tls); +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_assert_renegotiation_unblocked(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tor_tls_unblock_renegotiation(tls); + tor_tls_assert_renegotiation_unblocked(tls); + /* No assertion here - this test will fail if tor_assert is turned on + * and things are bad. */ + + tor_free(tls->ssl); + tor_free(tls); +} +#endif + +static void +test_tortls_set_logged_address(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + + tor_tls_set_logged_address(tls, "foo bar"); + + tt_str_op(tls->address, OP_EQ, "foo bar"); + + tor_tls_set_logged_address(tls, "foo bar 2"); + tt_str_op(tls->address, OP_EQ, "foo bar 2"); + + done: + tor_free(tls->address); + tor_free(tls); +} + +#ifndef OPENSSL_OPAQUE +static void +example_cb(tor_tls_t *t, void *arg) +{ + (void)t; + (void)arg; +} + +static void +test_tortls_set_renegotiate_callback(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + const char *arg = "hello"; + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + + tor_tls_set_renegotiate_callback(tls, example_cb, (void*)arg); + tt_assert(tls->negotiated_callback == example_cb); + tt_assert(tls->callback_arg == arg); + tt_assert(!tls->got_renegotiate); + + /* Assumes V2_HANDSHAKE_SERVER */ + tt_assert(tls->ssl->info_callback == tor_tls_server_info_callback); + + tor_tls_set_renegotiate_callback(tls, NULL, (void*)arg); + tt_assert(tls->ssl->info_callback == tor_tls_debug_state_callback); + + done: + tor_free(tls->ssl); + tor_free(tls); +} +#endif + +#ifndef OPENSSL_OPAQUE +static SSL_CIPHER *fixed_cipher1 = NULL; +static SSL_CIPHER *fixed_cipher2 = NULL; +static const SSL_CIPHER * +fake_get_cipher(unsigned ncipher) +{ + + switch (ncipher) { + case 1: + return fixed_cipher1; + case 2: + return fixed_cipher2; + default: + return NULL; + } +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_find_cipher_by_id(void *ignored) +{ + (void)ignored; + int ret; + SSL *ssl; + SSL_CTX *ctx; + const SSL_METHOD *m = TLSv1_method(); + SSL_METHOD *empty_method = tor_malloc_zero(sizeof(SSL_METHOD)); + + fixed_cipher1 = tor_malloc_zero(sizeof(SSL_CIPHER)); + fixed_cipher2 = tor_malloc_zero(sizeof(SSL_CIPHER)); + fixed_cipher2->id = 0xC00A; + + SSL_library_init(); + SSL_load_error_strings(); + + ctx = SSL_CTX_new(m); + ssl = SSL_new(ctx); + + ret = find_cipher_by_id(ssl, NULL, 0xC00A); + tt_int_op(ret, OP_EQ, 1); + + ret = find_cipher_by_id(ssl, m, 0xC00A); + tt_int_op(ret, OP_EQ, 1); + + ret = find_cipher_by_id(ssl, m, 0xFFFF); + tt_int_op(ret, OP_EQ, 0); + + ret = find_cipher_by_id(ssl, empty_method, 0xC00A); + tt_int_op(ret, OP_EQ, 1); + + ret = find_cipher_by_id(ssl, empty_method, 0xFFFF); +#ifdef HAVE_SSL_CIPHER_FIND + tt_int_op(ret, OP_EQ, 0); +#else + tt_int_op(ret, OP_EQ, 1); +#endif + + empty_method->get_cipher = fake_get_cipher; + ret = find_cipher_by_id(ssl, empty_method, 0xC00A); + tt_int_op(ret, OP_EQ, 1); + + empty_method->get_cipher = m->get_cipher; + empty_method->num_ciphers = m->num_ciphers; + ret = find_cipher_by_id(ssl, empty_method, 0xC00A); + tt_int_op(ret, OP_EQ, 1); + + empty_method->get_cipher = fake_get_cipher; + empty_method->num_ciphers = m->num_ciphers; + ret = find_cipher_by_id(ssl, empty_method, 0xC00A); + tt_int_op(ret, OP_EQ, 1); + + empty_method->num_ciphers = fake_num_ciphers; + ret = find_cipher_by_id(ssl, empty_method, 0xC00A); +#ifdef HAVE_SSL_CIPHER_FIND + tt_int_op(ret, OP_EQ, 1); +#else + tt_int_op(ret, OP_EQ, 0); +#endif + + done: + tor_free(empty_method); + SSL_free(ssl); + SSL_CTX_free(ctx); + tor_free(fixed_cipher1); +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_debug_state_callback(void *ignored) +{ + (void)ignored; + SSL *ssl; + char *buf = tor_malloc_zero(1000); + int n; + + int previous_log = setup_capture_of_logs(LOG_DEBUG); + + ssl = tor_malloc_zero(sizeof(SSL)); + + tor_tls_debug_state_callback(ssl, 32, 45); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + n = snprintf(buf, 1000, "SSL %p is now in state unknown" + " state [type=32,val=45].\n", ssl); + buf[n]='\0'; + if (strcasecmp(mock_saved_log_at(0), buf)) + tt_str_op(mock_saved_log_at(0), OP_EQ, buf); + + done: + teardown_capture_of_logs(previous_log); + tor_free(buf); + tor_free(ssl); +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_server_info_callback(void *ignored) +{ + (void)ignored; + tor_tls_t *tls; + SSL_CTX *ctx; + SSL *ssl; + int previous_log = setup_capture_of_logs(LOG_WARN); + + SSL_library_init(); + SSL_load_error_strings(); + + ctx = SSL_CTX_new(TLSv1_method()); + ssl = SSL_new(ctx); + + tor_tls_allocate_tor_tls_object_ex_data_index(); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->magic = TOR_TLS_MAGIC; + tls->ssl = ssl; + + tor_tls_server_info_callback(NULL, 0, 0); + + SSL_set_state(ssl, SSL3_ST_SW_SRVR_HELLO_A); + mock_clean_saved_logs(); + tor_tls_server_info_callback(ssl, SSL_CB_ACCEPT_LOOP, 0); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Couldn't look up the tls for an SSL*. How odd!\n"); + + SSL_set_state(ssl, SSL3_ST_SW_SRVR_HELLO_B); + mock_clean_saved_logs(); + tor_tls_server_info_callback(ssl, SSL_CB_ACCEPT_LOOP, 0); + tt_int_op(mock_saved_log_number(), OP_EQ, 1); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Couldn't look up the tls for an SSL*. How odd!\n"); + + SSL_set_state(ssl, 99); + mock_clean_saved_logs(); + tor_tls_server_info_callback(ssl, SSL_CB_ACCEPT_LOOP, 0); + tt_int_op(mock_saved_log_number(), OP_EQ, 0); + + SSL_set_ex_data(tls->ssl, tor_tls_object_ex_data_index, tls); + SSL_set_state(ssl, SSL3_ST_SW_SRVR_HELLO_B); + tls->negotiated_callback = 0; + tls->server_handshake_count = 120; + tor_tls_server_info_callback(ssl, SSL_CB_ACCEPT_LOOP, 0); + tt_int_op(tls->server_handshake_count, OP_EQ, 121); + + tls->server_handshake_count = 127; + tls->negotiated_callback = (void *)1; + tor_tls_server_info_callback(ssl, SSL_CB_ACCEPT_LOOP, 0); + tt_int_op(tls->server_handshake_count, OP_EQ, 127); + tt_int_op(tls->got_renegotiate, OP_EQ, 1); + + tls->ssl->session = SSL_SESSION_new(); + tls->wasV2Handshake = 0; + tor_tls_server_info_callback(ssl, SSL_CB_ACCEPT_LOOP, 0); + tt_int_op(tls->wasV2Handshake, OP_EQ, 0); + + done: + teardown_capture_of_logs(previous_log); + SSL_free(ssl); + SSL_CTX_free(ctx); + tor_free(tls); +} +#endif + +#ifndef OPENSSL_OPAQUE +static int fixed_ssl_read_result_index; +static int fixed_ssl_read_result[5]; +static int fixed_ssl_shutdown_result; + +static int +fixed_ssl_read(SSL *s, void *buf, int len) +{ + (void)s; + (void)buf; + (void)len; + return fixed_ssl_read_result[fixed_ssl_read_result_index++]; +} + +static int +fixed_ssl_shutdown(SSL *s) +{ + (void)s; + return fixed_ssl_shutdown_result; +} + +static int fixed_ssl_state_to_set; +static tor_tls_t *fixed_tls; + +static int +setting_version_ssl_shutdown(SSL *s) +{ + s->version = SSL2_VERSION; + return fixed_ssl_shutdown_result; +} + +static int +setting_version_and_state_ssl_shutdown(SSL *s) +{ + fixed_tls->state = fixed_ssl_state_to_set; + s->version = SSL2_VERSION; + return fixed_ssl_shutdown_result; +} + +static int +dummy_handshake_func(SSL *s) +{ + (void)s; + return 1; +} + +static void +test_tortls_shutdown(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + SSL_METHOD *method = give_me_a_test_method(); + int previous_log = setup_capture_of_logs(LOG_WARN); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->ssl->method = method; + method->ssl_read = fixed_ssl_read; + method->ssl_shutdown = fixed_ssl_shutdown; + + ret = tor_tls_shutdown(tls); + tt_int_op(ret, OP_EQ, -9); + + tls->state = TOR_TLS_ST_SENTCLOSE; + fixed_ssl_read_result_index = 0; + fixed_ssl_read_result[0] = 10; + fixed_ssl_read_result[1] = -1; + ret = tor_tls_shutdown(tls); + tt_int_op(ret, OP_EQ, -9); + + tls->ssl->handshake_func = dummy_handshake_func; + + fixed_ssl_read_result_index = 0; + fixed_ssl_read_result[0] = 10; + fixed_ssl_read_result[1] = 42; + fixed_ssl_read_result[2] = 0; + fixed_ssl_shutdown_result = 1; + ERR_clear_error(); + tls->ssl->version = SSL2_VERSION; + ret = tor_tls_shutdown(tls); + tt_int_op(ret, OP_EQ, TOR_TLS_DONE); + tt_int_op(tls->state, OP_EQ, TOR_TLS_ST_CLOSED); + + fixed_ssl_read_result_index = 0; + fixed_ssl_read_result[0] = 10; + fixed_ssl_read_result[1] = 42; + fixed_ssl_read_result[2] = 0; + fixed_ssl_shutdown_result = 0; + ERR_clear_error(); + tls->ssl->version = 0; + ret = tor_tls_shutdown(tls); + tt_int_op(ret, OP_EQ, TOR_TLS_DONE); + tt_int_op(tls->state, OP_EQ, TOR_TLS_ST_CLOSED); + + fixed_ssl_read_result_index = 0; + fixed_ssl_read_result[0] = 10; + fixed_ssl_read_result[1] = 42; + fixed_ssl_read_result[2] = 0; + fixed_ssl_shutdown_result = 0; + ERR_clear_error(); + tls->ssl->version = 0; + method->ssl_shutdown = setting_version_ssl_shutdown; + ret = tor_tls_shutdown(tls); + tt_int_op(ret, OP_EQ, TOR_TLS_ERROR_MISC); + + fixed_ssl_read_result_index = 0; + fixed_ssl_read_result[0] = 10; + fixed_ssl_read_result[1] = 42; + fixed_ssl_read_result[2] = 0; + fixed_ssl_shutdown_result = 0; + fixed_tls = tls; + fixed_ssl_state_to_set = TOR_TLS_ST_GOTCLOSE; + ERR_clear_error(); + tls->ssl->version = 0; + method->ssl_shutdown = setting_version_and_state_ssl_shutdown; + ret = tor_tls_shutdown(tls); + tt_int_op(ret, OP_EQ, TOR_TLS_ERROR_MISC); + + fixed_ssl_read_result_index = 0; + fixed_ssl_read_result[0] = 10; + fixed_ssl_read_result[1] = 42; + fixed_ssl_read_result[2] = 0; + fixed_ssl_read_result[3] = -1; + fixed_ssl_shutdown_result = 0; + fixed_tls = tls; + fixed_ssl_state_to_set = 0; + ERR_clear_error(); + tls->ssl->version = 0; + method->ssl_shutdown = setting_version_and_state_ssl_shutdown; + ret = tor_tls_shutdown(tls); + tt_int_op(ret, OP_EQ, TOR_TLS_ERROR_MISC); + + done: + teardown_capture_of_logs(previous_log); + tor_free(method); + tor_free(tls->ssl); + tor_free(tls); +} + +static int negotiated_callback_called; + +static void +negotiated_callback_setter(tor_tls_t *t, void *arg) +{ + (void)t; + (void)arg; + negotiated_callback_called++; +} + +static void +test_tortls_read(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + char buf[100]; + SSL_METHOD *method = give_me_a_test_method(); + int previous_log = setup_capture_of_logs(LOG_WARN); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->state = TOR_TLS_ST_OPEN; + + ret = tor_tls_read(tls, buf, 10); + tt_int_op(ret, OP_EQ, -9); + + /* These tests assume that V2_HANDSHAKE_SERVER is set */ + tls->ssl->handshake_func = dummy_handshake_func; + tls->ssl->method = method; + method->ssl_read = fixed_ssl_read; + fixed_ssl_read_result_index = 0; + fixed_ssl_read_result[0] = 42; + tls->state = TOR_TLS_ST_OPEN; + ERR_clear_error(); + ret = tor_tls_read(tls, buf, 10); + tt_int_op(ret, OP_EQ, 42); + + tls->state = TOR_TLS_ST_OPEN; + tls->got_renegotiate = 1; + fixed_ssl_read_result_index = 0; + ERR_clear_error(); + ret = tor_tls_read(tls, buf, 10); + tt_int_op(tls->got_renegotiate, OP_EQ, 0); + + tls->state = TOR_TLS_ST_OPEN; + tls->got_renegotiate = 1; + negotiated_callback_called = 0; + tls->negotiated_callback = negotiated_callback_setter; + fixed_ssl_read_result_index = 0; + ERR_clear_error(); + ret = tor_tls_read(tls, buf, 10); + tt_int_op(negotiated_callback_called, OP_EQ, 1); + + fixed_ssl_read_result_index = 0; + fixed_ssl_read_result[0] = 0; + tls->ssl->version = SSL2_VERSION; + ERR_clear_error(); + ret = tor_tls_read(tls, buf, 10); + tt_int_op(ret, OP_EQ, TOR_TLS_CLOSE); + tt_int_op(tls->state, OP_EQ, TOR_TLS_ST_CLOSED); + + // TODO: fill up + + done: + teardown_capture_of_logs(previous_log); + tor_free(tls->ssl); + tor_free(tls); + tor_free(method); +} + +static int fixed_ssl_write_result; + +static int +fixed_ssl_write(SSL *s, const void *buf, int len) +{ + (void)s; + (void)buf; + (void)len; + return fixed_ssl_write_result; +} + +static void +test_tortls_write(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + SSL_METHOD *method = give_me_a_test_method(); + char buf[100]; + int previous_log = setup_capture_of_logs(LOG_WARN); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = tor_malloc_zero(sizeof(SSL)); + tls->state = TOR_TLS_ST_OPEN; + + ret = tor_tls_write(tls, buf, 0); + tt_int_op(ret, OP_EQ, 0); + + ret = tor_tls_write(tls, buf, 10); + tt_int_op(ret, OP_EQ, -9); + + tls->ssl->method = method; + tls->wantwrite_n = 1; + ret = tor_tls_write(tls, buf, 10); + tt_int_op(tls->wantwrite_n, OP_EQ, 0); + + method->ssl_write = fixed_ssl_write; + tls->ssl->handshake_func = dummy_handshake_func; + fixed_ssl_write_result = 1; + ERR_clear_error(); + ret = tor_tls_write(tls, buf, 10); + tt_int_op(ret, OP_EQ, 1); + + fixed_ssl_write_result = -1; + ERR_clear_error(); + tls->ssl->rwstate = SSL_READING; + SSL_set_bio(tls->ssl, BIO_new(BIO_s_mem()), NULL); + SSL_get_rbio(tls->ssl)->flags = BIO_FLAGS_READ; + ret = tor_tls_write(tls, buf, 10); + tt_int_op(ret, OP_EQ, TOR_TLS_WANTREAD); + + ERR_clear_error(); + tls->ssl->rwstate = SSL_READING; + SSL_set_bio(tls->ssl, BIO_new(BIO_s_mem()), NULL); + SSL_get_rbio(tls->ssl)->flags = BIO_FLAGS_WRITE; + ret = tor_tls_write(tls, buf, 10); + tt_int_op(ret, OP_EQ, TOR_TLS_WANTWRITE); + + done: + teardown_capture_of_logs(previous_log); + tor_free(tls->ssl); + tor_free(tls); + tor_free(method); +} +#endif + +#ifndef OPENSSL_OPAQUE +static int fixed_ssl_accept_result; +static int fixed_ssl_connect_result; + +static int +setting_error_ssl_accept(SSL *ssl) +{ + (void)ssl; + ERR_put_error(ERR_LIB_BN, 2, -1, "somewhere.c", 99); + ERR_put_error(ERR_LIB_SYS, 2, -1, "somewhere.c", 99); + return fixed_ssl_accept_result; +} + +static int +setting_error_ssl_connect(SSL *ssl) +{ + (void)ssl; + ERR_put_error(ERR_LIB_BN, 2, -1, "somewhere.c", 99); + ERR_put_error(ERR_LIB_SYS, 2, -1, "somewhere.c", 99); + return fixed_ssl_connect_result; +} + +static int +fixed_ssl_accept(SSL *ssl) +{ + (void) ssl; + return fixed_ssl_accept_result; +} + +static void +test_tortls_handshake(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + SSL_CTX *ctx; + SSL_METHOD *method = give_me_a_test_method(); + int previous_log = setup_capture_of_logs(LOG_INFO); + + SSL_library_init(); + SSL_load_error_strings(); + + ctx = SSL_CTX_new(TLSv1_method()); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = SSL_new(ctx); + tls->state = TOR_TLS_ST_HANDSHAKE; + + ret = tor_tls_handshake(tls); + tt_int_op(ret, OP_EQ, -9); + + tls->isServer = 1; + tls->state = TOR_TLS_ST_HANDSHAKE; + ret = tor_tls_handshake(tls); + tt_int_op(ret, OP_EQ, -9); + + tls->ssl->method = method; + method->ssl_accept = fixed_ssl_accept; + fixed_ssl_accept_result = 2; + ERR_clear_error(); + tls->state = TOR_TLS_ST_HANDSHAKE; + ret = tor_tls_handshake(tls); + tt_int_op(tls->state, OP_EQ, TOR_TLS_ST_OPEN); + + method->ssl_accept = setting_error_ssl_accept; + fixed_ssl_accept_result = 1; + ERR_clear_error(); + mock_clean_saved_logs(); + tls->state = TOR_TLS_ST_HANDSHAKE; + ret = tor_tls_handshake(tls); + tt_int_op(ret, OP_EQ, TOR_TLS_ERROR_MISC); + tt_int_op(mock_saved_log_number(), OP_EQ, 2); + /* This fails on jessie. Investigate why! */ +#if 0 + tt_str_op(mock_saved_log_at(0), OP_EQ, + "TLS error while handshaking: (null) (in bignum routines:" + "(null):SSLv3 write client hello B)\n"); + tt_str_op(mock_saved_log_at(1), OP_EQ, + "TLS error while handshaking: (null) (in system library:" + "connect:SSLv3 write client hello B)\n"); +#endif + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_INFO); + tt_int_op(mock_saved_severity_at(1), OP_EQ, LOG_INFO); + + tls->isServer = 0; + method->ssl_connect = setting_error_ssl_connect; + fixed_ssl_connect_result = 1; + ERR_clear_error(); + mock_clean_saved_logs(); + tls->state = TOR_TLS_ST_HANDSHAKE; + ret = tor_tls_handshake(tls); + tt_int_op(ret, OP_EQ, TOR_TLS_ERROR_MISC); + tt_int_op(mock_saved_log_number(), OP_EQ, 2); +#if 0 + /* See above */ + tt_str_op(mock_saved_log_at(0), OP_EQ, "TLS error while handshaking: " + "(null) (in bignum routines:(null):SSLv3 write client hello B)\n"); + tt_str_op(mock_saved_log_at(1), OP_EQ, "TLS error while handshaking: " + "(null) (in system library:connect:SSLv3 write client hello B)\n"); +#endif + tt_int_op(mock_saved_severity_at(0), OP_EQ, LOG_WARN); + tt_int_op(mock_saved_severity_at(1), OP_EQ, LOG_WARN); + + done: + teardown_capture_of_logs(previous_log); + SSL_free(tls->ssl); + SSL_CTX_free(ctx); + tor_free(tls); + tor_free(method); +} +#endif + +#ifndef OPENSSL_OPAQUE +static void +test_tortls_finish_handshake(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_t *tls; + SSL_CTX *ctx; + SSL_METHOD *method = give_me_a_test_method(); + SSL_library_init(); + SSL_load_error_strings(); + + X509 *c1 = read_cert_from(validCertString); + SESS_CERT_local *sess = NULL; + + ctx = SSL_CTX_new(method); + + tls = tor_malloc_zero(sizeof(tor_tls_t)); + tls->ssl = SSL_new(ctx); + tls->state = TOR_TLS_ST_OPEN; + + ret = tor_tls_finish_handshake(tls); + tt_int_op(ret, OP_EQ, 0); + + tls->isServer = 1; + tls->wasV2Handshake = 0; + ret = tor_tls_finish_handshake(tls); + tt_int_op(ret, OP_EQ, 0); + tt_int_op(tls->wasV2Handshake, OP_EQ, 1); + + tls->wasV2Handshake = 1; + ret = tor_tls_finish_handshake(tls); + tt_int_op(ret, OP_EQ, 0); + tt_int_op(tls->wasV2Handshake, OP_EQ, 1); + + tls->wasV2Handshake = 1; + tls->ssl->session = SSL_SESSION_new(); + ret = tor_tls_finish_handshake(tls); + tt_int_op(ret, OP_EQ, 0); + tt_int_op(tls->wasV2Handshake, OP_EQ, 0); + + tls->isServer = 0; + + sess = tor_malloc_zero(sizeof(SESS_CERT_local)); + tls->ssl->session->sess_cert = (void *)sess; + sess->cert_chain = sk_X509_new_null(); + sk_X509_push(sess->cert_chain, c1); + tls->ssl->session->peer = c1; + tls->wasV2Handshake = 0; + ret = tor_tls_finish_handshake(tls); + tt_int_op(ret, OP_EQ, 0); + tt_int_op(tls->wasV2Handshake, OP_EQ, 1); + + method->num_ciphers = fake_num_ciphers; + ret = tor_tls_finish_handshake(tls); + tt_int_op(ret, OP_EQ, -9); + + done: + if (sess) + sk_X509_free(sess->cert_chain); + if (tls->ssl && tls->ssl->session) { + tor_free(tls->ssl->session->sess_cert); + } + SSL_free(tls->ssl); + tor_free(tls); + SSL_CTX_free(ctx); + tor_free(method); +} +#endif + +static int fixed_crypto_pk_new_result_index; +static crypto_pk_t *fixed_crypto_pk_new_result[5]; + +static crypto_pk_t * +fixed_crypto_pk_new(void) +{ + return fixed_crypto_pk_new_result[fixed_crypto_pk_new_result_index++]; +} + +#ifndef OPENSSL_OPAQUE +static int fixed_crypto_pk_generate_key_with_bits_result_index; +static int fixed_crypto_pk_generate_key_with_bits_result[5]; +static int fixed_tor_tls_create_certificate_result_index; +static X509 *fixed_tor_tls_create_certificate_result[5]; +static int fixed_tor_x509_cert_new_result_index; +static tor_x509_cert_t *fixed_tor_x509_cert_new_result[5]; + +static int +fixed_crypto_pk_generate_key_with_bits(crypto_pk_t *env, int bits) +{ + (void)env; + (void)bits; + return fixed_crypto_pk_generate_key_with_bits_result[ + fixed_crypto_pk_generate_key_with_bits_result_index++]; +} + +static X509 * +fixed_tor_tls_create_certificate(crypto_pk_t *rsa, + crypto_pk_t *rsa_sign, + const char *cname, + const char *cname_sign, + unsigned int cert_lifetime) +{ + (void)rsa; + (void)rsa_sign; + (void)cname; + (void)cname_sign; + (void)cert_lifetime; + return fixed_tor_tls_create_certificate_result[ + fixed_tor_tls_create_certificate_result_index++]; +} + +static tor_x509_cert_t * +fixed_tor_x509_cert_new(X509 *x509_cert) +{ + (void) x509_cert; + return fixed_tor_x509_cert_new_result[ + fixed_tor_x509_cert_new_result_index++]; +} + +static void +test_tortls_context_new(void *ignored) +{ + (void)ignored; + tor_tls_context_t *ret; + crypto_pk_t *pk1, *pk2, *pk3, *pk4, *pk5, *pk6, *pk7, *pk8, *pk9, *pk10, + *pk11, *pk12, *pk13, *pk14, *pk15, *pk16, *pk17, *pk18; + + pk1 = crypto_pk_new(); + pk2 = crypto_pk_new(); + pk3 = crypto_pk_new(); + pk4 = crypto_pk_new(); + pk5 = crypto_pk_new(); + pk6 = crypto_pk_new(); + pk7 = crypto_pk_new(); + pk8 = crypto_pk_new(); + pk9 = crypto_pk_new(); + pk10 = crypto_pk_new(); + pk11 = crypto_pk_new(); + pk12 = crypto_pk_new(); + pk13 = crypto_pk_new(); + pk14 = crypto_pk_new(); + pk15 = crypto_pk_new(); + pk16 = crypto_pk_new(); + pk17 = crypto_pk_new(); + pk18 = crypto_pk_new(); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = NULL; + MOCK(crypto_pk_new, fixed_crypto_pk_new); + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + MOCK(crypto_pk_generate_key_with_bits, + fixed_crypto_pk_generate_key_with_bits); + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk1; + fixed_crypto_pk_new_result[1] = NULL; + fixed_crypto_pk_generate_key_with_bits_result[0] = -1; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk2; + fixed_crypto_pk_new_result[1] = NULL; + fixed_crypto_pk_generate_key_with_bits_result[0] = 0; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk3; + fixed_crypto_pk_new_result[1] = pk4; + fixed_crypto_pk_new_result[2] = NULL; + fixed_crypto_pk_generate_key_with_bits_result[0] = 0; + fixed_crypto_pk_generate_key_with_bits_result[1] = -1; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + MOCK(tor_tls_create_certificate, fixed_tor_tls_create_certificate); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk5; + fixed_crypto_pk_new_result[1] = pk6; + fixed_crypto_pk_new_result[2] = NULL; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + fixed_crypto_pk_generate_key_with_bits_result[1] = 0; + fixed_tor_tls_create_certificate_result_index = 0; + fixed_tor_tls_create_certificate_result[0] = NULL; + fixed_tor_tls_create_certificate_result[1] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[2] = tor_malloc_zero(sizeof(X509)); + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk7; + fixed_crypto_pk_new_result[1] = pk8; + fixed_crypto_pk_new_result[2] = NULL; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + fixed_tor_tls_create_certificate_result_index = 0; + fixed_tor_tls_create_certificate_result[0] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[1] = NULL; + fixed_tor_tls_create_certificate_result[2] = tor_malloc_zero(sizeof(X509)); + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk9; + fixed_crypto_pk_new_result[1] = pk10; + fixed_crypto_pk_new_result[2] = NULL; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + fixed_tor_tls_create_certificate_result_index = 0; + fixed_tor_tls_create_certificate_result[0] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[1] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[2] = NULL; + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + MOCK(tor_x509_cert_new, fixed_tor_x509_cert_new); + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk11; + fixed_crypto_pk_new_result[1] = pk12; + fixed_crypto_pk_new_result[2] = NULL; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + fixed_tor_tls_create_certificate_result_index = 0; + fixed_tor_tls_create_certificate_result[0] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[1] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[2] = tor_malloc_zero(sizeof(X509)); + fixed_tor_x509_cert_new_result_index = 0; + fixed_tor_x509_cert_new_result[0] = NULL; + fixed_tor_x509_cert_new_result[1] = NULL; + fixed_tor_x509_cert_new_result[2] = NULL; + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk13; + fixed_crypto_pk_new_result[1] = pk14; + fixed_crypto_pk_new_result[2] = NULL; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + fixed_tor_tls_create_certificate_result_index = 0; + fixed_tor_tls_create_certificate_result[0] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[1] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[2] = tor_malloc_zero(sizeof(X509)); + fixed_tor_x509_cert_new_result_index = 0; + fixed_tor_x509_cert_new_result[0] = tor_malloc_zero(sizeof(tor_x509_cert_t)); + fixed_tor_x509_cert_new_result[1] = NULL; + fixed_tor_x509_cert_new_result[2] = NULL; + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk15; + fixed_crypto_pk_new_result[1] = pk16; + fixed_crypto_pk_new_result[2] = NULL; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + fixed_tor_tls_create_certificate_result_index = 0; + fixed_tor_tls_create_certificate_result[0] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[1] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[2] = tor_malloc_zero(sizeof(X509)); + fixed_tor_x509_cert_new_result_index = 0; + fixed_tor_x509_cert_new_result[0] = tor_malloc_zero(sizeof(tor_x509_cert_t)); + fixed_tor_x509_cert_new_result[1] = tor_malloc_zero(sizeof(tor_x509_cert_t)); + fixed_tor_x509_cert_new_result[2] = NULL; + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = pk17; + fixed_crypto_pk_new_result[1] = pk18; + fixed_crypto_pk_new_result[2] = NULL; + fixed_crypto_pk_generate_key_with_bits_result_index = 0; + fixed_tor_tls_create_certificate_result_index = 0; + fixed_tor_tls_create_certificate_result[0] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[1] = tor_malloc_zero(sizeof(X509)); + fixed_tor_tls_create_certificate_result[2] = tor_malloc_zero(sizeof(X509)); + fixed_tor_x509_cert_new_result_index = 0; + fixed_tor_x509_cert_new_result[0] = tor_malloc_zero(sizeof(tor_x509_cert_t)); + fixed_tor_x509_cert_new_result[1] = tor_malloc_zero(sizeof(tor_x509_cert_t)); + fixed_tor_x509_cert_new_result[2] = tor_malloc_zero(sizeof(tor_x509_cert_t)); + ret = tor_tls_context_new(NULL, 0, 0, 0); + tt_assert(!ret); + + done: + UNMOCK(tor_x509_cert_new); + UNMOCK(tor_tls_create_certificate); + UNMOCK(crypto_pk_generate_key_with_bits); + UNMOCK(crypto_pk_new); +} +#endif + +static int fixed_crypto_pk_get_evp_pkey_result_index = 0; +static EVP_PKEY *fixed_crypto_pk_get_evp_pkey_result[5]; + +static EVP_PKEY * +fixed_crypto_pk_get_evp_pkey_(crypto_pk_t *env, int private) +{ + (void) env; + (void) private; + return fixed_crypto_pk_get_evp_pkey_result[ + fixed_crypto_pk_get_evp_pkey_result_index++]; +} + +static void +test_tortls_create_certificate(void *ignored) +{ + (void)ignored; + X509 *ret; + crypto_pk_t *pk1, *pk2; + + pk1 = crypto_pk_new(); + pk2 = crypto_pk_new(); + + MOCK(crypto_pk_get_evp_pkey_, fixed_crypto_pk_get_evp_pkey_); + fixed_crypto_pk_get_evp_pkey_result_index = 0; + fixed_crypto_pk_get_evp_pkey_result[0] = NULL; + ret = tor_tls_create_certificate(pk1, pk2, "hello", "hello2", 1); + tt_assert(!ret); + + fixed_crypto_pk_get_evp_pkey_result_index = 0; + fixed_crypto_pk_get_evp_pkey_result[0] = tor_malloc_zero(sizeof(EVP_PKEY)); + fixed_crypto_pk_get_evp_pkey_result[1] = NULL; + ret = tor_tls_create_certificate(pk1, pk2, "hello", "hello2", 1); + tt_assert(!ret); + + fixed_crypto_pk_get_evp_pkey_result_index = 0; + fixed_crypto_pk_get_evp_pkey_result[0] = tor_malloc_zero(sizeof(EVP_PKEY)); + fixed_crypto_pk_get_evp_pkey_result[1] = tor_malloc_zero(sizeof(EVP_PKEY)); + ret = tor_tls_create_certificate(pk1, pk2, "hello", "hello2", 1); + tt_assert(!ret); + + done: + UNMOCK(crypto_pk_get_evp_pkey_); + crypto_pk_free(pk1); + crypto_pk_free(pk2); +} + +static void +test_tortls_cert_new(void *ignored) +{ + (void)ignored; + tor_x509_cert_t *ret; + X509 *cert = read_cert_from(validCertString); + + ret = tor_x509_cert_new(NULL); + tt_assert(!ret); + + ret = tor_x509_cert_new(cert); + tt_assert(ret); + tor_x509_cert_free(ret); + ret = NULL; + +#if 0 + cert = read_cert_from(validCertString); + /* XXX this doesn't do what you think: it alters a copy of the pubkey. */ + X509_get_pubkey(cert)->type = EVP_PKEY_DSA; + ret = tor_x509_cert_new(cert); + tt_assert(ret); +#endif + +#ifndef OPENSSL_OPAQUE + cert = read_cert_from(validCertString); + X509_CINF_free(cert->cert_info); + cert->cert_info = NULL; + ret = tor_x509_cert_new(cert); + tt_assert(ret); +#endif + + done: + tor_x509_cert_free(ret); +} + +static void +test_tortls_cert_is_valid(void *ignored) +{ + (void)ignored; + int ret; + tor_x509_cert_t *cert = NULL, *scert = NULL; + + scert = tor_malloc_zero(sizeof(tor_x509_cert_t)); + ret = tor_tls_cert_is_valid(LOG_WARN, cert, scert, 0); + tt_int_op(ret, OP_EQ, 0); + tor_free(scert); + + cert = tor_x509_cert_new(read_cert_from(validCertString)); + scert = tor_x509_cert_new(read_cert_from(caCertString)); + ret = tor_tls_cert_is_valid(LOG_WARN, cert, scert, 0); + tt_int_op(ret, OP_EQ, 1); + +#ifndef OPENSSL_OPAQUE + tor_x509_cert_free(cert); + tor_x509_cert_free(scert); + cert = tor_x509_cert_new(read_cert_from(validCertString)); + scert = tor_x509_cert_new(read_cert_from(caCertString)); + ASN1_TIME_free(cert->cert->cert_info->validity->notAfter); + cert->cert->cert_info->validity->notAfter = + ASN1_TIME_set(NULL, time(NULL)-1000000); + ret = tor_tls_cert_is_valid(LOG_WARN, cert, scert, 0); + tt_int_op(ret, OP_EQ, 0); + + tor_x509_cert_free(cert); + tor_x509_cert_free(scert); + cert = tor_x509_cert_new(read_cert_from(validCertString)); + scert = tor_x509_cert_new(read_cert_from(caCertString)); + X509_PUBKEY_free(cert->cert->cert_info->key); + cert->cert->cert_info->key = NULL; + ret = tor_tls_cert_is_valid(LOG_WARN, cert, scert, 1); + tt_int_op(ret, OP_EQ, 0); +#endif + +#if 0 + tor_x509_cert_free(cert); + tor_x509_cert_free(scert); + cert = tor_x509_cert_new(read_cert_from(validCertString)); + scert = tor_x509_cert_new(read_cert_from(caCertString)); + /* This doesn't actually change the key in the cert. XXXXXX */ + BN_one(EVP_PKEY_get1_RSA(X509_get_pubkey(cert->cert))->n); + ret = tor_tls_cert_is_valid(LOG_WARN, cert, scert, 1); + tt_int_op(ret, OP_EQ, 0); + + tor_x509_cert_free(cert); + tor_x509_cert_free(scert); + cert = tor_x509_cert_new(read_cert_from(validCertString)); + scert = tor_x509_cert_new(read_cert_from(caCertString)); + /* This doesn't actually change the key in the cert. XXXXXX */ + X509_get_pubkey(cert->cert)->type = EVP_PKEY_EC; + ret = tor_tls_cert_is_valid(LOG_WARN, cert, scert, 1); + tt_int_op(ret, OP_EQ, 0); + + tor_x509_cert_free(cert); + tor_x509_cert_free(scert); + cert = tor_x509_cert_new(read_cert_from(validCertString)); + scert = tor_x509_cert_new(read_cert_from(caCertString)); + /* This doesn't actually change the key in the cert. XXXXXX */ + X509_get_pubkey(cert->cert)->type = EVP_PKEY_EC; + ret = tor_tls_cert_is_valid(LOG_WARN, cert, scert, 0); + tt_int_op(ret, OP_EQ, 1); + + tor_x509_cert_free(cert); + tor_x509_cert_free(scert); + cert = tor_x509_cert_new(read_cert_from(validCertString)); + scert = tor_x509_cert_new(read_cert_from(caCertString)); + /* This doesn't actually change the key in the cert. XXXXXX */ + X509_get_pubkey(cert->cert)->type = EVP_PKEY_EC; + X509_get_pubkey(cert->cert)->ameth = NULL; + ret = tor_tls_cert_is_valid(LOG_WARN, cert, scert, 0); + tt_int_op(ret, OP_EQ, 0); +#endif + + done: + tor_x509_cert_free(cert); + tor_x509_cert_free(scert); +} + +static void +test_tortls_context_init_one(void *ignored) +{ + (void)ignored; + int ret; + tor_tls_context_t *old = NULL; + + MOCK(crypto_pk_new, fixed_crypto_pk_new); + + fixed_crypto_pk_new_result_index = 0; + fixed_crypto_pk_new_result[0] = NULL; + ret = tor_tls_context_init_one(&old, NULL, 0, 0, 0); + tt_int_op(ret, OP_EQ, -1); + + done: + UNMOCK(crypto_pk_new); +} + +#define LOCAL_TEST_CASE(name, flags) \ + { #name, test_tortls_##name, (flags|TT_FORK), NULL, NULL } + +#ifdef OPENSSL_OPAQUE +#define INTRUSIVE_TEST_CASE(name, flags) \ + { #name, NULL, TT_SKIP, NULL, NULL } +#else +#define INTRUSIVE_TEST_CASE(name, flags) LOCAL_TEST_CASE(name, flags) +#endif + +struct testcase_t tortls_tests[] = { + LOCAL_TEST_CASE(errno_to_tls_error, 0), + LOCAL_TEST_CASE(err_to_string, 0), + LOCAL_TEST_CASE(tor_tls_new, TT_FORK), + LOCAL_TEST_CASE(tor_tls_get_error, 0), + LOCAL_TEST_CASE(get_state_description, TT_FORK), + LOCAL_TEST_CASE(get_by_ssl, TT_FORK), + LOCAL_TEST_CASE(allocate_tor_tls_object_ex_data_index, TT_FORK), + LOCAL_TEST_CASE(log_one_error, TT_FORK), + INTRUSIVE_TEST_CASE(get_error, TT_FORK), + LOCAL_TEST_CASE(always_accept_verify_cb, 0), + INTRUSIVE_TEST_CASE(x509_cert_free, 0), + LOCAL_TEST_CASE(x509_cert_get_id_digests, 0), + INTRUSIVE_TEST_CASE(cert_matches_key, 0), + INTRUSIVE_TEST_CASE(cert_get_key, 0), + LOCAL_TEST_CASE(get_my_client_auth_key, TT_FORK), + LOCAL_TEST_CASE(get_my_certs, TT_FORK), + INTRUSIVE_TEST_CASE(get_ciphersuite_name, 0), + INTRUSIVE_TEST_CASE(classify_client_ciphers, 0), + LOCAL_TEST_CASE(client_is_using_v2_ciphers, 0), + INTRUSIVE_TEST_CASE(verify, 0), + INTRUSIVE_TEST_CASE(check_lifetime, 0), + INTRUSIVE_TEST_CASE(get_pending_bytes, 0), + LOCAL_TEST_CASE(get_forced_write_size, 0), + LOCAL_TEST_CASE(get_write_overhead_ratio, TT_FORK), + LOCAL_TEST_CASE(used_v1_handshake, TT_FORK), + LOCAL_TEST_CASE(get_num_server_handshakes, 0), + LOCAL_TEST_CASE(server_got_renegotiate, 0), + INTRUSIVE_TEST_CASE(SSL_SESSION_get_master_key, 0), + INTRUSIVE_TEST_CASE(get_tlssecrets, 0), + INTRUSIVE_TEST_CASE(get_buffer_sizes, 0), + LOCAL_TEST_CASE(evaluate_ecgroup_for_tls, 0), + INTRUSIVE_TEST_CASE(try_to_extract_certs_from_tls, 0), + INTRUSIVE_TEST_CASE(get_peer_cert, 0), + INTRUSIVE_TEST_CASE(peer_has_cert, 0), + INTRUSIVE_TEST_CASE(shutdown, 0), + INTRUSIVE_TEST_CASE(finish_handshake, 0), + INTRUSIVE_TEST_CASE(handshake, 0), + INTRUSIVE_TEST_CASE(write, 0), + INTRUSIVE_TEST_CASE(read, 0), + INTRUSIVE_TEST_CASE(server_info_callback, 0), + LOCAL_TEST_CASE(is_server, 0), + INTRUSIVE_TEST_CASE(assert_renegotiation_unblocked, 0), + INTRUSIVE_TEST_CASE(block_renegotiation, 0), + INTRUSIVE_TEST_CASE(unblock_renegotiation, 0), + INTRUSIVE_TEST_CASE(set_renegotiate_callback, 0), + LOCAL_TEST_CASE(set_logged_address, 0), + INTRUSIVE_TEST_CASE(find_cipher_by_id, 0), + INTRUSIVE_TEST_CASE(session_secret_cb, 0), + INTRUSIVE_TEST_CASE(debug_state_callback, 0), + INTRUSIVE_TEST_CASE(context_new, 0), + LOCAL_TEST_CASE(create_certificate, 0), + LOCAL_TEST_CASE(cert_new, 0), + LOCAL_TEST_CASE(cert_is_valid, 0), + LOCAL_TEST_CASE(context_init_one, 0), + END_OF_TESTCASES +}; + diff --git a/src/test/test_util.c b/src/test/test_util.c index 0a5783e9f5..4cf2f9bda1 100644 --- a/src/test/test_util.c +++ b/src/test/test_util.c @@ -19,6 +19,7 @@ #endif #include <math.h> #include <ctype.h> +#include <float.h> /* XXXX this is a minimal wrapper to make the unit tests compile with the * changed tor_timegm interface. */ @@ -4097,6 +4098,9 @@ test_util_round_to_next_multiple_of(void *arg) tt_u64_op(round_uint64_to_next_multiple_of(99,7), ==, 105); tt_u64_op(round_uint64_to_next_multiple_of(99,9), ==, 99); + tt_u64_op(round_uint64_to_next_multiple_of(UINT64_MAX,2), ==, + UINT64_MAX); + tt_i64_op(round_int64_to_next_multiple_of(0,1), ==, 0); tt_i64_op(round_int64_to_next_multiple_of(0,7), ==, 0); @@ -4110,7 +4114,27 @@ test_util_round_to_next_multiple_of(void *arg) tt_i64_op(round_int64_to_next_multiple_of(INT64_MIN,2), ==, INT64_MIN); tt_i64_op(round_int64_to_next_multiple_of(INT64_MAX,2), ==, - INT64_MAX-INT64_MAX%2); + INT64_MAX); + + tt_int_op(round_uint32_to_next_multiple_of(0,1), ==, 0); + tt_int_op(round_uint32_to_next_multiple_of(0,7), ==, 0); + + tt_int_op(round_uint32_to_next_multiple_of(99,1), ==, 99); + tt_int_op(round_uint32_to_next_multiple_of(99,7), ==, 105); + tt_int_op(round_uint32_to_next_multiple_of(99,9), ==, 99); + + tt_int_op(round_uint32_to_next_multiple_of(UINT32_MAX,2), ==, + UINT32_MAX); + + tt_uint_op(round_to_next_multiple_of(0,1), ==, 0); + tt_uint_op(round_to_next_multiple_of(0,7), ==, 0); + + tt_uint_op(round_to_next_multiple_of(99,1), ==, 99); + tt_uint_op(round_to_next_multiple_of(99,7), ==, 105); + tt_uint_op(round_to_next_multiple_of(99,9), ==, 99); + + tt_uint_op(round_to_next_multiple_of(UINT_MAX,2), ==, + UINT_MAX); done: ; } @@ -4143,6 +4167,7 @@ test_util_laplace(void *arg) */ tt_i64_op(INT64_MIN + 20, ==, add_laplace_noise(20, 0.0, delta_f, epsilon)); + tt_i64_op(-60, ==, add_laplace_noise(20, 0.1, delta_f, epsilon)); tt_i64_op(-14, ==, add_laplace_noise(20, 0.25, delta_f, epsilon)); tt_i64_op(20, ==, add_laplace_noise(20, 0.5, delta_f, epsilon)); @@ -4150,6 +4175,143 @@ test_util_laplace(void *arg) tt_i64_op(100, ==, add_laplace_noise(20, 0.9, delta_f, epsilon)); tt_i64_op(215, ==, add_laplace_noise(20, 0.99, delta_f, epsilon)); + /* Test extreme values of signal with maximally negative values of noise + * 1.0000000000000002 is the smallest number > 1 + * 0.0000000000000002 is the double epsilon (error when calculating near 1) + * this is approximately 1/(2^52) + * per https://en.wikipedia.org/wiki/Double_precision + * (let's not descend into the world of subnormals) + * >>> laplace.ppf([0, 0.0000000000000002], loc = 0, scale = 1) + * array([ -inf, -35.45506713]) + */ + const double noscale_df = 1.0, noscale_eps = 1.0; + + tt_i64_op(INT64_MIN, ==, + add_laplace_noise(0, 0.0, noscale_df, noscale_eps)); + + /* is it clipped to INT64_MIN? */ + tt_i64_op(INT64_MIN, ==, + add_laplace_noise(-1, 0.0, noscale_df, noscale_eps)); + tt_i64_op(INT64_MIN, ==, + add_laplace_noise(INT64_MIN, 0.0, + noscale_df, noscale_eps)); + /* ... even when scaled? */ + tt_i64_op(INT64_MIN, ==, + add_laplace_noise(0, 0.0, delta_f, epsilon)); + tt_i64_op(INT64_MIN, ==, + add_laplace_noise(0, 0.0, + DBL_MAX, 1)); + tt_i64_op(INT64_MIN, ==, + add_laplace_noise(INT64_MIN, 0.0, + DBL_MAX, 1)); + + /* does it play nice with INT64_MAX? */ + tt_i64_op((INT64_MIN + INT64_MAX), ==, + add_laplace_noise(INT64_MAX, 0.0, + noscale_df, noscale_eps)); + + /* do near-zero fractional values work? */ + const double min_dbl_error = 0.0000000000000002; + + tt_i64_op(-35, ==, + add_laplace_noise(0, min_dbl_error, + noscale_df, noscale_eps)); + tt_i64_op(INT64_MIN, ==, + add_laplace_noise(INT64_MIN, min_dbl_error, + noscale_df, noscale_eps)); + tt_i64_op((-35 + INT64_MAX), ==, + add_laplace_noise(INT64_MAX, min_dbl_error, + noscale_df, noscale_eps)); + tt_i64_op(INT64_MIN, ==, + add_laplace_noise(0, min_dbl_error, + DBL_MAX, 1)); + tt_i64_op((INT64_MAX + INT64_MIN), ==, + add_laplace_noise(INT64_MAX, min_dbl_error, + DBL_MAX, 1)); + tt_i64_op(INT64_MIN, ==, + add_laplace_noise(INT64_MIN, min_dbl_error, + DBL_MAX, 1)); + + /* does it play nice with INT64_MAX? */ + tt_i64_op((INT64_MAX - 35), ==, + add_laplace_noise(INT64_MAX, min_dbl_error, + noscale_df, noscale_eps)); + + /* Test extreme values of signal with maximally positive values of noise + * 1.0000000000000002 is the smallest number > 1 + * 0.9999999999999998 is the greatest number < 1 by calculation + * per https://en.wikipedia.org/wiki/Double_precision + * >>> laplace.ppf([1.0, 0.9999999999999998], loc = 0, scale = 1) + * array([inf, 35.35050621]) + * but the function rejects p == 1.0, so we just use max_dbl_lt_one + */ + const double max_dbl_lt_one = 0.9999999999999998; + + /* do near-one fractional values work? */ + tt_i64_op(35, ==, + add_laplace_noise(0, max_dbl_lt_one, noscale_df, noscale_eps)); + + /* is it clipped to INT64_MAX? */ + tt_i64_op(INT64_MAX, ==, + add_laplace_noise(INT64_MAX - 35, max_dbl_lt_one, + noscale_df, noscale_eps)); + tt_i64_op(INT64_MAX, ==, + add_laplace_noise(INT64_MAX - 34, max_dbl_lt_one, + noscale_df, noscale_eps)); + tt_i64_op(INT64_MAX, ==, + add_laplace_noise(INT64_MAX, max_dbl_lt_one, + noscale_df, noscale_eps)); + /* ... even when scaled? */ + tt_i64_op(INT64_MAX, ==, + add_laplace_noise(INT64_MAX, max_dbl_lt_one, + delta_f, epsilon)); + tt_i64_op((INT64_MIN + INT64_MAX), ==, + add_laplace_noise(INT64_MIN, max_dbl_lt_one, + DBL_MAX, 1)); + tt_i64_op(INT64_MAX, ==, + add_laplace_noise(INT64_MAX, max_dbl_lt_one, + DBL_MAX, 1)); + /* does it play nice with INT64_MIN? */ + tt_i64_op((INT64_MIN + 35), ==, + add_laplace_noise(INT64_MIN, max_dbl_lt_one, + noscale_df, noscale_eps)); + + done: + ; +} + +static void +test_util_clamp_double_to_int64(void *arg) +{ + (void)arg; + + tt_i64_op(INT64_MIN, ==, clamp_double_to_int64(-INFINITY)); + tt_i64_op(INT64_MIN, ==, + clamp_double_to_int64(-1.0 * pow(2.0, 64.0) - 1.0)); + tt_i64_op(INT64_MIN, ==, + clamp_double_to_int64(-1.0 * pow(2.0, 63.0) - 1.0)); + tt_i64_op(((uint64_t) -1) << 53, ==, + clamp_double_to_int64(-1.0 * pow(2.0, 53.0))); + tt_i64_op((((uint64_t) -1) << 53) + 1, ==, + clamp_double_to_int64(-1.0 * pow(2.0, 53.0) + 1.0)); + tt_i64_op(-1, ==, clamp_double_to_int64(-1.0)); + tt_i64_op(0, ==, clamp_double_to_int64(-0.9)); + tt_i64_op(0, ==, clamp_double_to_int64(-0.1)); + tt_i64_op(0, ==, clamp_double_to_int64(0.0)); + tt_i64_op(0, ==, clamp_double_to_int64(NAN)); + tt_i64_op(0, ==, clamp_double_to_int64(0.1)); + tt_i64_op(0, ==, clamp_double_to_int64(0.9)); + tt_i64_op(1, ==, clamp_double_to_int64(1.0)); + tt_i64_op((((int64_t) 1) << 53) - 1, ==, + clamp_double_to_int64(pow(2.0, 53.0) - 1.0)); + tt_i64_op(((int64_t) 1) << 53, ==, + clamp_double_to_int64(pow(2.0, 53.0))); + tt_i64_op(INT64_MAX, ==, + clamp_double_to_int64(pow(2.0, 63.0))); + tt_i64_op(INT64_MAX, ==, + clamp_double_to_int64(pow(2.0, 64.0))); + tt_i64_op(INT64_MAX, ==, clamp_double_to_int64(INFINITY)); + done: ; } @@ -4180,9 +4342,14 @@ fd_is_nonblocking(tor_socket_t fd) } #endif +#define ERRNO_IS_EPROTO(e) (e == SOCK_ERRNO(EPROTONOSUPPORT)) +#define SOCK_ERR_IS_EPROTO(s) ERRNO_IS_EPROTO(tor_socket_errno(s)) + +/* Test for tor_open_socket*, using IPv4 or IPv6 depending on arg. */ static void test_util_socket(void *arg) { + const int domain = !strcmp(arg, "4") ? AF_INET : AF_INET6; tor_socket_t fd1 = TOR_INVALID_SOCKET; tor_socket_t fd2 = TOR_INVALID_SOCKET; tor_socket_t fd3 = TOR_INVALID_SOCKET; @@ -4193,15 +4360,19 @@ test_util_socket(void *arg) (void)arg; - fd1 = tor_open_socket_with_extensions(AF_INET, SOCK_STREAM, 0, 0, 0); - fd2 = tor_open_socket_with_extensions(AF_INET, SOCK_STREAM, 0, 0, 1); + fd1 = tor_open_socket_with_extensions(domain, SOCK_STREAM, 0, 0, 0); + if (SOCK_ERR_IS_EPROTO(fd1)) { + /* Assume we're on an IPv4-only or IPv6-only system, and give up now. */ + goto done; + } + fd2 = tor_open_socket_with_extensions(domain, SOCK_STREAM, 0, 0, 1); tt_assert(SOCKET_OK(fd1)); tt_assert(SOCKET_OK(fd2)); tt_int_op(get_n_open_sockets(), OP_EQ, n + 2); - //fd3 = tor_open_socket_with_extensions(AF_INET, SOCK_STREAM, 0, 1, 0); - //fd4 = tor_open_socket_with_extensions(AF_INET, SOCK_STREAM, 0, 1, 1); - fd3 = tor_open_socket(AF_INET, SOCK_STREAM, 0); - fd4 = tor_open_socket_nonblocking(AF_INET, SOCK_STREAM, 0); + //fd3 = tor_open_socket_with_extensions(domain, SOCK_STREAM, 0, 1, 0); + //fd4 = tor_open_socket_with_extensions(domain, SOCK_STREAM, 0, 1, 1); + fd3 = tor_open_socket(domain, SOCK_STREAM, 0); + fd4 = tor_open_socket_nonblocking(domain, SOCK_STREAM, 0); tt_assert(SOCKET_OK(fd3)); tt_assert(SOCKET_OK(fd4)); tt_int_op(get_n_open_sockets(), OP_EQ, n + 4); @@ -4250,8 +4421,20 @@ test_util_socketpair(void *arg) int n = get_n_open_sockets(); tor_socket_t fds[2] = {TOR_INVALID_SOCKET, TOR_INVALID_SOCKET}; const int family = AF_UNIX; + int socketpair_result = 0; + + socketpair_result = tor_socketpair_fn(family, SOCK_STREAM, 0, fds); + /* If there is no 127.0.0.1 or ::1, tor_ersatz_socketpair will and must fail. + * Otherwise, we risk exposing a socketpair on a routable IP address. (Some + * BSD jails use a routable address for localhost. Fortunately, they have + * the real AF_UNIX socketpair.) */ + if (ersatz && ERRNO_IS_EPROTO(-socketpair_result)) { + /* In my testing, an IPv6-only FreeBSD jail without ::1 returned EINVAL. + * Assume we're on a machine without 127.0.0.1 or ::1 and give up now. */ + goto done; + } + tt_int_op(0, OP_EQ, socketpair_result); - tt_int_op(0, OP_EQ, tor_socketpair_fn(family, SOCK_STREAM, 0, fds)); tt_assert(SOCKET_OK(fds[0])); tt_assert(SOCKET_OK(fds[1])); tt_int_op(get_n_open_sockets(), OP_EQ, n + 2); @@ -4271,6 +4454,8 @@ test_util_socketpair(void *arg) tor_close_socket(fds[1]); } +#undef SOCKET_EPROTO + static void test_util_max_mem(void *arg) { @@ -4441,6 +4626,7 @@ struct testcase_t util_tests[] = { UTIL_TEST(di_map, 0), UTIL_TEST(round_to_next_multiple_of, 0), UTIL_TEST(laplace, 0), + UTIL_TEST(clamp_double_to_int64, 0), UTIL_TEST(find_str_at_start_of_line, 0), UTIL_TEST(string_is_C_identifier, 0), UTIL_TEST(asprintf, 0), @@ -4473,7 +4659,10 @@ struct testcase_t util_tests[] = { UTIL_TEST(write_chunks_to_file, 0), UTIL_TEST(mathlog, 0), UTIL_TEST(weak_random, 0), - UTIL_TEST(socket, TT_FORK), + { "socket_ipv4", test_util_socket, TT_FORK, &passthrough_setup, + (void*)"4" }, + { "socket_ipv6", test_util_socket, TT_FORK, + &passthrough_setup, (void*)"6" }, { "socketpair", test_util_socketpair, TT_FORK, &passthrough_setup, (void*)"0" }, { "socketpair_ersatz", test_util_socketpair, TT_FORK, diff --git a/src/test/test_util_format.c b/src/test/test_util_format.c new file mode 100644 index 0000000000..705dfcf605 --- /dev/null +++ b/src/test/test_util_format.c @@ -0,0 +1,262 @@ +/* Copyright (c) 2010-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#include "or.h" + +#include "test.h" + +#define UTIL_FORMAT_PRIVATE +#include "util_format.h" + +#define NS_MODULE util_format + +static void +test_util_format_base64_encode(void *ignored) +{ + (void)ignored; + int res; + int i; + char *src; + char *dst; + + src = tor_malloc_zero(256); + dst = tor_malloc_zero(1000); + + for (i=0;i<256;i++) { + src[i] = (char)i; + } + + res = base64_encode(NULL, 1, src, 1, 0); + tt_int_op(res, OP_EQ, -1); + + res = base64_encode(dst, 1, NULL, 1, 0); + tt_int_op(res, OP_EQ, -1); + + res = base64_encode(dst, 1, src, 10, 0); + tt_int_op(res, OP_EQ, -1); + + res = base64_encode(dst, SSIZE_MAX-1, src, 1, 0); + tt_int_op(res, OP_EQ, -1); + + res = base64_encode(dst, SSIZE_MAX-1, src, 10, 0); + tt_int_op(res, OP_EQ, -1); + + res = base64_encode(dst, 1000, src, 256, 0); + tt_int_op(res, OP_EQ, 344); + tt_str_op(dst, OP_EQ, "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh" + "8gISIjJCUmJygpKissLS4vMDEyMzQ1Njc4OTo7PD0+P0BBQkNERUZH" + "SElKS0xNTk9QUVJTVFVWV1hZWltcXV5fYGFiY2RlZmdoaWprbG1ub3" + "BxcnN0dXZ3eHl6e3x9fn+AgYKDhIWGh4iJiouMjY6PkJGSk5SVlpeY" + "mZqbnJ2en6ChoqOkpaanqKmqq6ytrq+wsbKztLW2t7i5uru8vb6/wM" + "HCw8TFxsfIycrLzM3Oz9DR0tPU1dbX2Nna29zd3t/g4eLj5OXm5+jp" + "6uvs7e7v8PHy8/T19vf4+fr7/P3+/w=="); + + res = base64_encode(dst, 1000, src, 256, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 350); + tt_str_op(dst, OP_EQ, + "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8gISIjJCUmJygpKissLS4v\n" + "MDEyMzQ1Njc4OTo7PD0+P0BBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWltcXV5f\n" + "YGFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6e3x9fn+AgYKDhIWGh4iJiouMjY6P\n" + "kJGSk5SVlpeYmZqbnJ2en6ChoqOkpaanqKmqq6ytrq+wsbKztLW2t7i5uru8vb6/\n" + "wMHCw8TFxsfIycrLzM3Oz9DR0tPU1dbX2Nna29zd3t/g4eLj5OXm5+jp6uvs7e7v\n" + "8PHy8/T19vf4+fr7/P3+/w==\n"); + + res = base64_encode(dst, 1000, src+1, 255, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 346); + + for (i = 0;i<50;i++) { + src[i] = 0; + } + src[50] = 255; + src[51] = 255; + src[52] = 255; + src[53] = 255; + + res = base64_encode(dst, 1000, src, 54, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 74); + + res = base64_encode(dst, 1000, src+1, 53, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 74); + + res = base64_encode(dst, 1000, src+2, 52, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 74); + + res = base64_encode(dst, 1000, src+3, 51, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 70); + + res = base64_encode(dst, 1000, src+4, 50, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 70); + + res = base64_encode(dst, 1000, src+5, 49, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 70); + + res = base64_encode(dst, 1000, src+6, 48, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 65); + + res = base64_encode(dst, 1000, src+7, 47, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 65); + + res = base64_encode(dst, 1000, src+8, 46, BASE64_ENCODE_MULTILINE); + tt_int_op(res, OP_EQ, 65); + + done: + tor_free(src); + tor_free(dst); +} + +static void +test_util_format_base64_decode_nopad(void *ignored) +{ + (void)ignored; + int res; + int i; + char *src; + uint8_t *dst, *real_dst; + uint8_t expected[] = {0x65, 0x78, 0x61, 0x6D, 0x70, 0x6C, 0x65}; + char real_src[] = "ZXhhbXBsZQ"; + + src = tor_malloc_zero(256); + dst = tor_malloc_zero(1000); + real_dst = tor_malloc_zero(10); + + for (i=0;i<256;i++) { + src[i] = (char)i; + } + + res = base64_decode_nopad(dst, 1, src, SIZE_T_CEILING); + tt_int_op(res, OP_EQ, -1); + + res = base64_decode_nopad(dst, 1, src, 5); + tt_int_op(res, OP_EQ, -1); + + const char *s = "SGVsbG8gd29ybGQ"; + res = base64_decode_nopad(dst, 1000, s, strlen(s)); + tt_int_op(res, OP_EQ, 11); + tt_mem_op(dst, OP_EQ, "Hello world", 11); + + s = "T3BhIG11bmRv"; + res = base64_decode_nopad(dst, 9, s, strlen(s)); + tt_int_op(res, OP_EQ, 9); + tt_mem_op(dst, OP_EQ, "Opa mundo", 9); + + res = base64_decode_nopad(real_dst, 10, real_src, 10); + tt_int_op(res, OP_EQ, 7); + tt_mem_op(real_dst, OP_EQ, expected, 7); + + done: + tor_free(src); + tor_free(dst); + tor_free(real_dst); +} + +static void +test_util_format_base64_decode(void *ignored) +{ + (void)ignored; + int res; + int i; + char *src; + char *dst, *real_dst; + uint8_t expected[] = {0x65, 0x78, 0x61, 0x6D, 0x70, 0x6C, 0x65}; + char real_src[] = "ZXhhbXBsZQ=="; + + src = tor_malloc_zero(256); + dst = tor_malloc_zero(1000); + real_dst = tor_malloc_zero(10); + + for (i=0;i<256;i++) { + src[i] = (char)i; + } + + res = base64_decode(dst, 1, src, SIZE_T_CEILING); + tt_int_op(res, OP_EQ, -1); + + res = base64_decode(dst, SIZE_T_CEILING+1, src, 10); + tt_int_op(res, OP_EQ, -1); + + const char *s = "T3BhIG11bmRv"; + res = base64_decode(dst, 9, s, strlen(s)); + tt_int_op(res, OP_EQ, 9); + tt_mem_op(dst, OP_EQ, "Opa mundo", 9); + + memset(dst, 0, 1000); + res = base64_decode(dst, 100, s, strlen(s)); + tt_int_op(res, OP_EQ, 9); + tt_mem_op(dst, OP_EQ, "Opa mundo", 9); + + s = "SGVsbG8gd29ybGQ="; + res = base64_decode(dst, 100, s, strlen(s)); + tt_int_op(res, OP_EQ, 11); + tt_mem_op(dst, OP_EQ, "Hello world", 11); + + res = base64_decode(real_dst, 10, real_src, 10); + tt_int_op(res, OP_EQ, 7); + tt_mem_op(real_dst, OP_EQ, expected, 7); + + done: + tor_free(src); + tor_free(dst); + tor_free(real_dst); +} + +static void +test_util_format_base16_decode(void *ignored) +{ + (void)ignored; + int res; + int i; + char *src; + char *dst, *real_dst; + char expected[] = {0x65, 0x78, 0x61, 0x6D, 0x70, 0x6C, 0x65}; + char real_src[] = "6578616D706C65"; + + src = tor_malloc_zero(256); + dst = tor_malloc_zero(1000); + real_dst = tor_malloc_zero(10); + + for (i=0;i<256;i++) { + src[i] = (char)i; + } + + res = base16_decode(dst, 3, src, 3); + tt_int_op(res, OP_EQ, -1); + + res = base16_decode(dst, 1, src, 10); + tt_int_op(res, OP_EQ, -1); + + res = base16_decode(dst, SIZE_T_CEILING+2, src, 10); + tt_int_op(res, OP_EQ, -1); + + res = base16_decode(dst, 1000, "", 0); + tt_int_op(res, OP_EQ, 0); + + res = base16_decode(dst, 1000, "aabc", 4); + tt_int_op(res, OP_EQ, 0); + tt_mem_op(dst, OP_EQ, "\xaa\xbc", 2); + + res = base16_decode(dst, 1000, "aabcd", 6); + tt_int_op(res, OP_EQ, -1); + + res = base16_decode(dst, 1000, "axxx", 4); + tt_int_op(res, OP_EQ, -1); + + res = base16_decode(real_dst, 10, real_src, 14); + tt_int_op(res, OP_EQ, 0); + tt_mem_op(real_dst, OP_EQ, expected, 7); + + done: + tor_free(src); + tor_free(dst); + tor_free(real_dst); +} + +struct testcase_t util_format_tests[] = { + { "base64_encode", test_util_format_base64_encode, 0, NULL, NULL }, + { "base64_decode_nopad", test_util_format_base64_decode_nopad, 0, + NULL, NULL }, + { "base64_decode", test_util_format_base64_decode, 0, NULL, NULL }, + { "base16_decode", test_util_format_base16_decode, 0, NULL, NULL }, + END_OF_TESTCASES +}; + diff --git a/src/test/test_util_process.c b/src/test/test_util_process.c new file mode 100644 index 0000000000..cb1d5b2ebb --- /dev/null +++ b/src/test/test_util_process.c @@ -0,0 +1,84 @@ +/* Copyright (c) 2010-2015, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#define UTIL_PROCESS_PRIVATE +#include "orconfig.h" +#include "or.h" + +#include "test.h" + +#include "util_process.h" + +#include "log_test_helpers.h" + +#ifndef _WIN32 +#define NS_MODULE util_process + +static void +temp_callback(int r, void *s) +{ + (void)r; + (void)s; +} + +static void +test_util_process_set_waitpid_callback(void *ignored) +{ + (void)ignored; + waitpid_callback_t *res1 = NULL, *res2 = NULL; + int previous_log = setup_capture_of_logs(LOG_WARN); + pid_t pid = (pid_t)42; + + res1 = set_waitpid_callback(pid, temp_callback, NULL); + tt_assert(res1); + + res2 = set_waitpid_callback(pid, temp_callback, NULL); + tt_assert(res2); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Replaced a waitpid monitor on pid 42. That should be " + "impossible.\n"); + + done: + teardown_capture_of_logs(previous_log); + clear_waitpid_callback(res1); + clear_waitpid_callback(res2); +} + +static void +test_util_process_clear_waitpid_callback(void *ignored) +{ + (void)ignored; + waitpid_callback_t *res; + int previous_log = setup_capture_of_logs(LOG_WARN); + pid_t pid = (pid_t)43; + + clear_waitpid_callback(NULL); + + res = set_waitpid_callback(pid, temp_callback, NULL); + clear_waitpid_callback(res); + tt_int_op(mock_saved_log_number(), OP_EQ, 0); + +#if 0 + /* No. This is use-after-free. We don't _do_ that. XXXX */ + clear_waitpid_callback(res); + tt_str_op(mock_saved_log_at(0), OP_EQ, + "Couldn't remove waitpid monitor for pid 43.\n"); +#endif + + done: + teardown_capture_of_logs(previous_log); +} +#endif /* _WIN32 */ + +#ifndef _WIN32 +#define TEST(name) { #name, test_util_process_##name, 0, NULL, NULL } +#else +#define TEST(name) { #name, NULL, TT_SKIP, NULL, NULL } +#endif + +struct testcase_t util_process_tests[] = { + TEST(set_waitpid_callback), + TEST(clear_waitpid_callback), + END_OF_TESTCASES +}; + diff --git a/src/test/test_workqueue.c b/src/test/test_workqueue.c index 0d79733cf0..1202f80fa3 100644 --- a/src/test/test_workqueue.c +++ b/src/test/test_workqueue.c @@ -390,8 +390,14 @@ main(int argc, char **argv) init_logging(1); network_init(); - crypto_global_init(1, NULL, NULL); - crypto_seed_rng(); + if (crypto_global_init(1, NULL, NULL) < 0) { + printf("Couldn't initialize crypto subsystem; exiting.\n"); + return 1; + } + if (crypto_seed_rng() < 0) { + printf("Couldn't seed RNG; exiting.\n"); + return 1; + } rq = replyqueue_new(as_flags); tor_assert(rq); diff --git a/src/test/testing_common.c b/src/test/testing_common.c index 441024bd7d..2ea158fddd 100644 --- a/src/test/testing_common.c +++ b/src/test/testing_common.c @@ -272,7 +272,10 @@ main(int c, const char **v) return 1; } crypto_set_tls_dh_prime(); - crypto_seed_rng(); + if (crypto_seed_rng() < 0) { + printf("Couldn't seed RNG; exiting.\n"); + return 1; + } rep_hist_init(); network_init(); setup_directory(); diff --git a/src/test/vote_descriptors.inc b/src/test/vote_descriptors.inc new file mode 100644 index 0000000000..c5ce21f744 --- /dev/null +++ b/src/test/vote_descriptors.inc @@ -0,0 +1,94 @@ +const char* VOTE_BODY_V3 = +"network-status-version 3\n" +"vote-status vote\n" +"consensus-methods 13 14 15 16 17 18 19 20 21\n" +"published 2015-09-02 19:34:15\n" +"valid-after 2015-09-02 19:50:55\n" +"fresh-until 2015-09-02 20:07:38\n" +"valid-until 2015-09-02 20:24:15\n" +"voting-delay 100 250\n" +"client-versions 0.1.2.14,0.1.2.17\n" +"server-versions 0.1.2.10,0.1.2.15,0.1.2.16\n" +"known-flags Authority Exit Fast Guard MadeOfCheese MadeOfTin Running Stable V2Dir Valid\n" +"flag-thresholds stable-uptime=0 stable-mtbf=0 fast-speed=0 guard-wfu=0.000% guard-tk=0 guard-bw-inc-exits=0 guard-bw-exc-exits=0 enough-mtbf=0 ignoring-advertised-bws=0\n" +"params circuitwindow=80 foo=660\n" +"dir-source Voter3 D867ACF56A9D229B35C25F0090BC9867E906BE69 3.4.5.6 3.4.5.6 80 9000\n" +"contact voter@example.com\n" +"legacy-dir-key 4141414141414141414141414141414141414141\n" +"dir-key-certificate-version 3\n" +"fingerprint D867ACF56A9D229B35C25F0090BC9867E906BE69\n" +"dir-key-published 2008-12-12 18:07:24\n" +"dir-key-expires 2009-12-12 18:07:24\n" +"dir-identity-key\n" +"-----BEGIN RSA PUBLIC KEY-----\n" +"MIIBigKCAYEAveMpKlw8oD1YqFqpJchuwSR82BDhutbqgHiez3QO9FmzOctJpV+Y\n" +"mpTYIJLS/qC+4GBKFF1VK0C4SoBrS3zri0qdXdE+vBGcyrxrjMklpxoqSKRY2011\n" +"4eqYPghKlo5RzuqteBclGCHyNxWjUJeRKDWgvh+U/gr2uYM6fRm5q0fCzg4aECE7\n" +"VP6fDGZrMbQI8jHpiMSoC9gkUASNEa6chLInlnP8/H5qUEW4TB9CN/q095pefuwL\n" +"P+F+1Nz5hnM7fa5XmeMB8iM4RriUmOQlLBZgpQBMpEfWMIPcR9F1Gh3MxERqqUcH\n" +"tmij+IZdeXg9OkCXykcabaYIhZD3meErn9Tax4oA/THduLfgli9zM0ExwzH1OooN\n" +"L8rIcJ+2eBo3bQiQUbdYW71sl9w7nSPtircbJUa1mUvWYLPWQxFliPiQSetgJLMj\n" +"VQqtPmV2hvN2Xk3lLfJO50qMTK7w7Gsaw8UtV4YDM1Hcjp/hQaIB1xfwhXgl+eUU\n" +"btUa4c+cUTjHAgMBAAE=\n" +"-----END RSA PUBLIC KEY-----\n" +"dir-signing-key\n" +"-----BEGIN RSA PUBLIC KEY-----\n" +"MIGJAoGBALPSUInyuEu6NV3NjozplaniIEBzQXEjv1x9/+mqnwZABpYVmuy9A8nx\n" +"eoyY3sZFsnYwNW/IZjAgG23pEmevu3F+L4myMjjaa6ORl3MgRYQ4gmuFqpefrGdm\n" +"ywRCleh2JerkQ4VxOuq10dn/abITzLyaZzMw30KXWp5pxKXOLtxFAgMBAAE=\n" +"-----END RSA PUBLIC KEY-----\n" +"dir-key-crosscert\n" +"-----BEGIN ID SIGNATURE-----\n" +"FTBJNR/Hlt4T53yUMp1r/QCSMCpkHJCbYBT0R0pvYqhqFfYN5qHRSICRXaFFImIF\n" +"0DGWmwRza6DxPKNzkm5/b7I0de9zJW1jNNdQAQK5xppAtQcAafRdu8cBonnmh9KX\n" +"k1NrAK/X00FYywju3yl/SxCn1GddVNkHYexEudmJMPM=\n" +"-----END ID SIGNATURE-----\n" +"dir-key-certification\n" +"-----BEGIN SIGNATURE-----\n" +"pjWguLFBfELZDc6DywL6Do21SCl7LcutfpM92MEn4WYeSNcTXNR6lRX7reOEJk4e\n" +"NwEaMt+Hl7slgeR5wjnW3OmMmRPZK9bquNWbfD+sAOV9bRFZTpXIdleAQFPlwvMF\n" +"z/Gzwspzn4i2Yh6hySShrctMmW8YL3OM8LsBXzBhp/rG2uHlsxmIsc13DA6HWt61\n" +"ffY72uNE6KckDGsQ4wPGP9q69y6g+X+TNio1KPbsILbePv6EjbO+rS8FiS4njPlg\n" +"SPYry1RaUvxzxTkswIzdE1tjJrUiqpbWlTGxrH9N4OszoLm45Pc784KLULrjKIoi\n" +"Q+vRsGrcMBAa+kDowWU6H1ryKR7KOhzRTcf2uqLE/W3ezaRwmOG+ETmoVFwbhk2X\n" +"OlbXEM9fWP+INvFkr6Z93VYL2jGkCjV7e3xXmre/Lb92fUcYi6t5dwzfV8gJnIoG\n" +"eCHd0K8NrQK0ipVk/7zcPDKOPeo9Y5aj/f6X/pDHtb+Dd5sT+l82G/Tqy4DIYUYR\n" +"-----END SIGNATURE-----\n" +"r router2 AwMDAwMDAwMDAwMDAwMDAwMDAwM Tk5OTk5OTk5OTk5OTk5OTk5OTk4 2015-09-02 19:09:15 153.0.136.1 443 8000\n" +"s Running V2Dir\n" +"v 0.1.2.14\n" +"w Bandwidth=30 Measured=30\n" +"p reject 1-65535\n" +"id ed25519 none\n" +"m 9,10,11,12,13,14,15,16,17 sha256=xyzajkldsdsajdadlsdjaslsdksdjlsdjsdaskdaaa0\n" +"r router1 BQUFBQUFBQUFBQUFBQUFBQUFBQU TU1NTU1NTU1NTU1NTU1NTU1NTU0 2015-09-02 19:17:35 153.0.153.1 443 0\n" +"a [1:2:3::4]:4711\n" +"s Exit Fast Guard Running Stable Valid\n" +"v 0.2.0.5\n" +"w Bandwidth=120 Measured=120\n" +"p reject 1-65535\n" +"id ed25519 none\n" +"m 9,10,11,12,13,14,15,16,17 sha256=xyzajkldsdsajdadlsdjaslsdksdjlsdjsdaskdaaa1\n" +"r router3 MzMzMzMzMzMzMzMzMzMzMzMzMzM T09PT09PT09PT09PT09PT09PT08 2015-09-02 19:17:35 170.0.153.1 400 9999\n" +"s Authority Exit Fast Guard Running Stable V2Dir Valid\n" +"v 0.1.0.3\n" +"w Bandwidth=120\n" +"p reject 1-65535\n" +"id ed25519 none\n" +"m 9,10,11,12,13,14,15,16,17 " +"sha256=xyzajkldsdsajdadlsdjaslsdksdjlsdjsdaskdaaa2\n" +"r router4 NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ Ly8vLy8vLy8vLy8vLy8vLy8vLy8 2015-09-02 19:17:35 192.0.2.3 500 1999\n" +"s Running V2Dir\n" +"v 0.1.6.3\n" +"w Bandwidth=30\n" +"p reject 1-65535\n" +"id ed25519 none\n" +"m 9,10,11,12,13,14,15,16,17 sha256=xyzajkldsdsajdadlsdjaslsdksdjlsdjsdaskdaaa3\n" +"directory-footer\n" +"directory-signature D867ACF56A9D229B35C25F0090BC9867E906BE69 CBF56A83368A5150F1A9AAADAFB4D77F8C4170E2\n" +"-----BEGIN SIGNATURE-----\n" +"AHiWcHe+T3XbnlQqvqSAk6RY3XmEy1+hM2u9Xk6BNi7BpQkEQM1f0vzRpgn5Dnf2\n" +"TXQWGUq9Z7jdSVnzWT3xqPA4zjw6eZkj+DKUtwq+oEDZGlf8eHTFmr0NAWfwZbk9\n" +"NAjbMTUXUP37N2XAZwkoCWwFCrrfMwXrL7OhZbj7ifo=\n" +"-----END SIGNATURE-----\n"; + diff --git a/src/trunnel/README b/src/trunnel/README new file mode 100644 index 0000000000..e24aea0764 --- /dev/null +++ b/src/trunnel/README @@ -0,0 +1,21 @@ +This directory contains code for use with, and code made by, the +automatic code generation tool "Trunnel". + +Trunnel generates binary parsers and formatters for simple data +structures. It aims for human-readable, obviously-correct outputs over +maximum efficiency or flexibility. + +The .trunnel files are the inputs here; the .c and .h files are the outputs. + +To add a new structure: + - Add a new .trunnel file or expand an existing one to describe the format + of the structure. + - Regenerate the .c and .h files. To do this, you run + "scripts/codegen/run_trunnel.sh". You'll need trunnel installed. + - Add the .trunnel, .c, and .h files to include.am + +For the Trunnel source code, and more documentation about using Trunnel, +see https://gitweb.torproject.org/trunnel.git , especially + https://gitweb.torproject.org/trunnel.git/tree/README +and https://gitweb.torproject.org/trunnel.git/tree/doc/trunnel.md + diff --git a/src/trunnel/include.am b/src/trunnel/include.am index 9bf37fe58b..b1448b7cb2 100644 --- a/src/trunnel/include.am +++ b/src/trunnel/include.am @@ -36,3 +36,7 @@ src_trunnel_libor_trunnel_testing_a_CPPFLAGS = -DTRUNNEL_LOCAL_H $(AM_CPPFLAGS) src_trunnel_libor_trunnel_testing_a_CFLAGS = $(AM_CFLAGS) $(TEST_CFLAGS) noinst_HEADERS+= $(TRUNNELHEADERS) + +EXTRA_DIST += \ + src/trunnel/README + diff --git a/src/win32/orconfig.h b/src/win32/orconfig.h index 8b687c8234..c0b14e5304 100644 --- a/src/win32/orconfig.h +++ b/src/win32/orconfig.h @@ -232,7 +232,7 @@ #define USING_TWOS_COMPLEMENT /* Version number of package */ -#define VERSION "0.2.7.6-dev" +#define VERSION "0.2.8.0-alpha-dev" |