diff options
397 files changed, 38865 insertions, 11099 deletions
diff --git a/.gitignore b/.gitignore index 673bb7871d..3a28ecfd44 100644 --- a/.gitignore +++ b/.gitignore @@ -5,6 +5,7 @@ *.swp # C stuff *.o +*.obj # Diff droppings *.orig *.rej @@ -29,6 +30,7 @@ /aclocal.m4 /autom4te.cache /build-stamp +/compile /configure /Doxyfile /orconfig.h @@ -87,6 +89,11 @@ /doc/tor.html /doc/tor.html.in /doc/tor.1.xml +/doc/tor-fw-helper.1 +/doc/tor-fw-helper.1.in +/doc/tor-fw-helper.html +/doc/tor-fw-helper.html.in +/doc/tor-fw-helper.1.xml /doc/tor-gencert.1 /doc/tor-gencert.1.in /doc/tor-gencert.html @@ -116,8 +123,11 @@ /src/common/Makefile.in /src/common/common_sha1.i /src/common/libor.a +/src/common/libor.lib /src/common/libor-crypto.a +/src/common/libor-crypto.lib /src/common/libor-event.a +/src/common/libor-event.lib # /src/config/ /src/config/Makefile @@ -134,11 +144,17 @@ /src/or/tor /src/or/tor.exe /src/or/libtor.a +/src/or/libtor.lib # /src/test /src/test/Makefile /src/test/Makefile.in +/src/test/bench +/src/test/bench.exe /src/test/test +/src/test/test-child +/src/test/test.exe +/src/test/test-child.exe # /src/tools/ @@ -151,6 +167,12 @@ /src/tools/Makefile /src/tools/Makefile.in +# /src/tools/tor-fw-helper/ +/src/tools/tor-fw-helper/tor-fw-helper +/src/tools/tor-fw-helper/tor-fw-helper.exe +/src/tools/tor-fw-helper/Makefile +/src/tools/tor-fw-helper/Makefile.in + # /src/win32/ /src/win32/Makefile /src/win32/Makefile.in @@ -1,3 +1,3804 @@ +Changes in version 0.2.3.18-rc - 2012-06-28 + Tor 0.2.3.18-rc is the first release candidate for the Tor 0.2.3.x + series. It fixes a few smaller bugs, but generally appears stable. + Please test it and let us know whether it is! + + o Major bugfixes: + - Allow wildcarded mapaddress targets to be specified on the + controlport. Partially fixes bug 6244; bugfix on 0.2.3.9-alpha. + - Make our linker option detection code more robust against linkers + such as on FreeBSD 8, where a bad combination of options completes + successfully but makes an unrunnable binary. Fixes bug 6173; + bugfix on 0.2.3.17-beta. + + o Minor bugfixes (on 0.2.2.x and earlier): + - Avoid a false positive in the util/threads unit test by increasing + the maximum timeout time. Fixes bug 6227; bugfix on 0.2.0.4-alpha. + - Replace "Sending publish request" log messages with "Launching + upload", so that they no longer confusingly imply that we're + sending something to a directory we might not even be connected + to yet. Fixes bug 3311; bugfix on 0.2.0.10-alpha. + - Make sure to set *socket_error in all error cases in + connection_connect(), so it can't produce a warning about + errno being zero from errno_to_orconn_end_reason(). Bugfix on + 0.2.1.1-alpha; resolves ticket 6028. + - Downgrade "Got a certificate, but we already have it" log messages + from warning to info, except when we're a dirauth. Fixes bug 5238; + bugfix on 0.2.1.7-alpha. + - When checking for requested signatures on the latest consensus + before serving it to a client, make sure to check the right + consensus flavor. Bugfix on 0.2.2.6-alpha. + - Downgrade "eventdns rejected address" message to LOG_PROTOCOL_WARN. + Fixes bug 5932; bugfix on 0.2.2.7-alpha. + + o Minor bugfixes (on 0.2.3.x): + - Make format_helper_exit_status() avoid unnecessary space padding + and stop confusing log_from_pipe(). Fixes ticket 5557; bugfix + on 0.2.3.1-alpha. + - Downgrade a message about cleaning the microdescriptor cache to + "info" from "notice". Fixes bug 6238; bugfix on 0.2.3.1-alpha. + - Log a BUG message at severity INFO if we have a networkstatus with + a missing entry for some microdescriptor. Continues on a patch + to 0.2.3.2-alpha. + - Improve the log message when a managed proxy fails to launch. Fixes + bug 5099; bugfix on 0.2.3.6-alpha. + - Don't do DNS lookups when parsing corrupted managed proxy protocol + messages. Fixes bug 6226; bugfix on 0.2.3.6-alpha. + - When formatting wildcarded address mappings for the controller, + be sure to include "*." as appropriate. Partially fixes bug 6244; + bugfix on 0.2.3.9-alpha. + - Avoid a warning caused by using strcspn() from glibc with clang 3.0. + Bugfix on 0.2.3.13-alpha. + - Stop logging messages about running with circuit timeout learning + enabled at severity LD_BUG. Fixes bug 6169; bugfix on 0.2.3.17-beta. + - Disable a spurious warning about reading on a marked and flushing + connection. We shouldn't be doing that, but apparently we + sometimes do. Fixes bug 6203; bugfix on 0.2.3.17-beta. + - Fix a bug that stopped AllowDotExit from working on addresses + that had an entry in the DNS cache. Fixes bug 6211; bugfix on + 0.2.3.17-beta. + + o Code simplification, refactoring, unit tests: + - Move tor_gettimeofday_cached() into compat_libevent.c, and use + Libevent's notion of cached time when possible. + - Remove duplicate code for invoking getrlimit() from control.c. + - Add a unit test for the environment_variable_names_equal function. + + o Documentation: + - Document the --defaults-torrc option, and the new (in 0.2.3) + semantics for overriding, extending, and clearing lists of + options. Closes bug 4748. + + +Changes in version 0.2.3.17-beta - 2012-06-15 + Tor 0.2.3.17-beta enables compiler and linker hardening by default, + gets our TLS handshake back on track for being able to blend in with + Firefox, fixes a big bug in 0.2.3.16-alpha that broke Tor's interaction + with Vidalia, and otherwise continues to get us closer to a release + candidate. + + o Major features: + - Enable gcc and ld hardening by default. Resolves ticket 5210. + - Update TLS cipher list to match Firefox 8 and later. Resolves + ticket 4744. + - Implement the client side of proposal 198: remove support for + clients falsely claiming to support standard ciphersuites that + they can actually provide. As of modern OpenSSL versions, it's not + necessary to fake any standard ciphersuite, and doing so prevents + us from using better ciphersuites in the future, since servers + can't know whether an advertised ciphersuite is really supported or + not. Some hosts -- notably, ones with very old versions of OpenSSL + or where OpenSSL has been built with ECC disabled -- will stand + out because of this change; TBB users should not be affected. + + o Major bugfixes: + - Change the default value for DynamicDHGroups (introduced in + 0.2.3.9-alpha) to 0. This feature can make Tor relays less + identifiable by their use of the mod_ssl DH group, but at + the cost of some usability (#4721) and bridge tracing (#6087) + regressions. Resolves ticket 5598. + - Send a CRLF at the end of each STATUS_* control protocol event. This + bug tickled a bug in Vidalia which would make it freeze. Fixes + bug 6094; bugfix on 0.2.3.16-alpha. + + o Minor bugfixes: + - Disable writing on marked-for-close connections when they are + blocked on bandwidth, to prevent busy-looping in Libevent. Fixes + bug 5263; bugfix on 0.0.2pre13, where we first added a special + case for flushing marked connections. + - Detect SSL handshake even when the initial attempt to write the + server hello fails. Fixes bug 4592; bugfix on 0.2.0.13-alpha. + - Change the AllowDotExit rules so they should actually work. + We now enforce AllowDotExit only immediately after receiving an + address via SOCKS or DNSPort: other sources are free to provide + .exit addresses after the resolution occurs. Fixes bug 3940; + bugfix on 0.2.2.1-alpha. + - Fix a (harmless) integer overflow in cell statistics reported by + some fast relays. Fixes bug 5849; bugfix on 0.2.2.1-alpha. + - Make sure circuitbuild.c checks LearnCircuitBuildTimeout in all the + right places and never depends on the consensus parameters or + computes adaptive timeouts when it is disabled. Fixes bug 5049; + bugfix on 0.2.2.14-alpha. + - When building Tor on Windows with -DUNICODE (not default), ensure + that error messages, filenames, and DNS server names are always + NUL-terminated when we convert them to a single-byte encoding. + Fixes bug 5909; bugfix on 0.2.2.16-alpha. + - Make Tor build correctly again with -DUNICODE -D_UNICODE defined. + Fixes bug 6097; bugfix on 0.2.2.16-alpha. + - Fix an edge case where TestingTorNetwork is set but the authorities + and relays all have an uptime of zero, where the private Tor network + could briefly lack support for hidden services. Fixes bug 3886; + bugfix on 0.2.2.18-alpha. + - Correct the manpage's descriptions for the default values of + DirReqStatistics and ExtraInfoStatistics. Fixes bug 2865; bugfix + on 0.2.3.1-alpha. + - Fix the documentation for the --hush and --quiet command line + options, which changed their behavior back in 0.2.3.3-alpha. + - Fix compilation warning with clang 3.1. Fixes bug 6141; bugfix on + 0.2.3.11-alpha. + + o Minor features: + - Rate-limit the "Weighted bandwidth is 0.000000" message, and add + more information to it, so that we can track it down in case it + returns again. Mitigates bug 5235. + - Check CircuitBuildTimeout and LearnCircuitBuildTimeout in + options_validate(); warn if LearnCircuitBuildTimeout is disabled and + CircuitBuildTimeout is set unreasonably low. Resolves ticket 5452. + - Warn the user when HTTPProxy, but no other proxy type, is + configured. This can cause surprising behavior: it doesn't send + all of Tor's traffic over the HTTPProxy -- it sends unencrypted + directory traffic only. Resolves ticket 4663. + - Issue a notice if a guard completes less than 40% of your circuits. + Threshold is configurable by torrc option PathBiasNoticeRate and + consensus parameter pb_noticepct. There is additional, off-by- + default code to disable guards which fail too many circuits. + Addresses ticket 5458. + - Update to the June 6 2012 Maxmind GeoLite Country database. + + o Code simplifications and refactoring: + - Remove validate_pluggable_transports_config(): its warning + message is now handled by connection_or_connect(). + + +Changes in version 0.2.2.37 - 2012-06-06 + Tor 0.2.2.37 introduces a workaround for a critical renegotiation + bug in OpenSSL 1.0.1 (where 20% of the Tor network can't talk to itself + currently). + + o Major bugfixes: + - Work around a bug in OpenSSL that broke renegotiation with TLS + 1.1 and TLS 1.2. Without this workaround, all attempts to speak + the v2 Tor connection protocol when both sides were using OpenSSL + 1.0.1 would fail. Resolves ticket 6033. + - When waiting for a client to renegotiate, don't allow it to add + any bytes to the input buffer. This fixes a potential DoS issue. + Fixes bugs 5934 and 6007; bugfix on 0.2.0.20-rc. + - Fix an edge case where if we fetch or publish a hidden service + descriptor, we might build a 4-hop circuit and then use that circuit + for exiting afterwards -- even if the new last hop doesn't obey our + ExitNodes config option. Fixes bug 5283; bugfix on 0.2.0.10-alpha. + + o Minor bugfixes: + - Fix a build warning with Clang 3.1 related to our use of vasprintf. + Fixes bug 5969. Bugfix on 0.2.2.11-alpha. + + o Minor features: + - Tell GCC and Clang to check for any errors in format strings passed + to the tor_v*(print|scan)f functions. + + +Changes in version 0.2.3.16-alpha - 2012-06-05 + Tor 0.2.3.16-alpha introduces a workaround for a critical renegotiation + bug in OpenSSL 1.0.1 (where 20% of the Tor network can't talk to itself + currently). It also fixes a variety of smaller bugs and other cleanups + that get us closer to a release candidate. + + o Major bugfixes (general): + - Work around a bug in OpenSSL that broke renegotiation with TLS + 1.1 and TLS 1.2. Without this workaround, all attempts to speak + the v2 Tor connection protocol when both sides were using OpenSSL + 1.0.1 would fail. Resolves ticket 6033. + - When waiting for a client to renegotiate, don't allow it to add + any bytes to the input buffer. This fixes a potential DoS issue. + Fixes bugs 5934 and 6007; bugfix on 0.2.0.20-rc. + - Pass correct OR address to managed proxies (like obfsproxy), + even when ORListenAddress is used. Fixes bug 4865; bugfix on + 0.2.3.9-alpha. + - The advertised platform of a router now includes only its operating + system's name (e.g., "Linux", "Darwin", "Windows 7"), and not its + service pack level (for Windows) or its CPU architecture (for Unix). + We also no longer include the "git-XYZ" tag in the version. Resolves + part of bug 2988. + + o Major bugfixes (clients): + - If we are unable to find any exit that supports our predicted ports, + stop calling them predicted, so that we don't loop and build + hopeless circuits indefinitely. Fixes bug 3296; bugfix on 0.0.9pre6, + which introduced predicted ports. + - Fix an edge case where if we fetch or publish a hidden service + descriptor, we might build a 4-hop circuit and then use that circuit + for exiting afterwards -- even if the new last hop doesn't obey our + ExitNodes config option. Fixes bug 5283; bugfix on 0.2.0.10-alpha. + - Check at each new consensus whether our entry guards were picked + long enough ago that we should rotate them. Previously, we only + did this check at startup, which could lead to us holding a guard + indefinitely. Fixes bug 5380; bugfix on 0.2.1.14-rc. + - When fetching a bridge descriptor from a bridge authority, + always do so anonymously, whether we have been able to open + circuits or not. Partial fix for bug 1938; bugfix on 2.0.7-alpha. + This behavior makes it *safer* to use UpdateBridgesFromAuthority, + but we'll need to wait for bug 6010 before it's actually usable. + + o Major bugfixes (directory authorities): + - When computing weight parameters, behave more robustly in the + presence of a bad bwweightscale value. Previously, the authorities + would crash if they agreed on a sufficiently broken weight_scale + value: now, they use a reasonable default and carry on. Partial + fix for 5786; bugfix on 0.2.2.17-alpha. + - Check more thoroughly to prevent a rogue authority from + double-voting on any consensus directory parameter. Previously, + authorities would crash in this case if the total number of + votes for any parameter exceeded the number of active voters, + but would let it pass otherwise. Partial fix for bug 5786; bugfix + on 0.2.2.2-alpha. + + o Minor features: + - Rate-limit log messages when asked to connect anonymously to + a private address. When these hit, they tended to hit fast and + often. Also, don't bother trying to connect to addresses that we + are sure will resolve to 127.0.0.1: getting 127.0.0.1 in a directory + reply makes us think we have been lied to, even when the address the + client tried to connect to was "localhost." Resolves ticket 2822. + - Allow packagers to insert an extra string in server descriptor + platform lines by setting the preprocessor variable TOR_BUILD_TAG. + Resolves the rest of ticket 2988. + - Raise the threshold of server descriptors needed (75%) and exit + server descriptors needed (50%) before we will declare ourselves + bootstrapped. This will make clients start building circuits a + little later, but makes the initially constructed circuits less + skewed and less in conflict with further directory fetches. Fixes + ticket 3196. + - Close any connection that sends unrecognized junk before the + handshake. Solves an issue noted in bug 4369. + - Improve log messages about managed transports. Resolves ticket 5070. + - Tag a bridge's descriptor as "never to be sent unencrypted". + This shouldn't matter, since bridges don't open non-anonymous + connections to the bridge authority and don't allow unencrypted + directory connections from clients, but we might as well make + sure. Closes bug 5139. + - Expose our view of whether we have gone dormant to the controller, + via a new "GETINFO dormant" value. Torbutton and other controllers + can use this to avoid doing periodic requests through Tor while + it's dormant (bug 4718). Fixes bug 5954. + - Tell GCC and Clang to check for any errors in format strings passed + to the tor_v*(print|scan)f functions. + - Update to the May 1 2012 Maxmind GeoLite Country database. + + o Minor bugfixes (already included in 0.2.2.36): + - Reject out-of-range times like 23:59:61 in parse_rfc1123_time(). + Fixes bug 5346; bugfix on 0.0.8pre3. + - Correct parsing of certain date types in parse_http_time(). + Without this patch, If-Modified-Since would behave + incorrectly. Fixes bug 5346; bugfix on 0.2.0.2-alpha. Patch from + Esteban Manchado Velázques. + - Make our number-parsing functions always treat too-large values + as an error, even when those values exceed the width of the + underlying type. Previously, if the caller provided these + functions with minima or maxima set to the extreme values of the + underlying integer type, these functions would return those + values on overflow rather than treating overflow as an error. + Fixes part of bug 5786; bugfix on 0.0.9. + - If we hit the error case where routerlist_insert() replaces an + existing (old) server descriptor, make sure to remove that + server descriptor from the old_routers list. Fix related to bug + 1776. Bugfix on 0.2.2.18-alpha. + - Clarify the behavior of MaxCircuitDirtiness with hidden service + circuits. Fixes issue 5259. + + o Minor bugfixes (coding cleanup, on 0.2.2.x and earlier): + - Prevent a null-pointer dereference when receiving a data cell + for a nonexistent stream when the circuit in question has an + empty deliver window. We don't believe this is triggerable, + since we don't currently allow deliver windows to become empty, + but the logic is tricky enough that it's better to make the code + robust. Fixes bug 5541; bugfix on 0.0.2pre14. + - Fix a memory leak when trying to launch a DNS request when the + network is disabled or the nameservers are unconfigurable. Fixes + bug 5916; bugfix on Tor 0.1.2.1-alpha (for the unconfigurable + nameserver case) and on 0.2.3.9-alpha (for the DisableNetwork case). + - Don't hold a Windows file handle open for every file mapping; + the file mapping handle is sufficient. Fixes bug 5951; bugfix on + 0.1.2.1-alpha. + - Avoid O(n^2) performance characteristics when parsing a large + extrainfo cache. Fixes bug 5828; bugfix on 0.2.0.1-alpha. + - Format more doubles with %f, not %lf. Patch from grarpamp to make + Tor build correctly on older BSDs again. Fixes bug 3894; bugfix on + Tor 0.2.0.8-alpha. + - Make our replacement implementation of strtok_r() compatible with + the standard behavior of strtok_r(). Patch by nils. Fixes bug 5091; + bugfix on 0.2.2.1-alpha. + - Fix a NULL-pointer dereference on a badly formed + SETCIRCUITPURPOSE command. Found by mikeyc. Fixes bug 5796; + bugfix on 0.2.2.9-alpha. + - Fix a build warning with Clang 3.1 related to our use of vasprintf. + Fixes bug 5969. Bugfix on 0.2.2.11-alpha. + - Defensively refactor rend_mid_rendezvous() so that protocol + violations and length checks happen in the beginning. Fixes + bug 5645. + - Set _WIN32_WINNT to 0x0501 consistently throughout the code, so + that IPv6 stuff will compile on MSVC, and compilation issues + will be easier to track down. Fixes bug 5861. + + o Minor bugfixes (correctness, on 0.2.2.x and earlier): + - Exit nodes now correctly report EADDRINUSE and EADDRNOTAVAIL as + resource exhaustion, so that clients can adjust their load to + try other exits. Fixes bug 4710; bugfix on 0.1.0.1-rc, which + started using END_STREAM_REASON_RESOURCELIMIT. + - Don't check for whether the address we're using for outbound + connections has changed until after the outbound connection has + completed. On Windows, getsockname() doesn't succeed until the + connection is finished. Fixes bug 5374; bugfix on 0.1.1.14-alpha. + - If the configuration tries to set MyFamily on a bridge, refuse to + do so, and warn about the security implications. Fixes bug 4657; + bugfix on 0.2.0.3-alpha. + - If the client fails to set a reasonable set of ciphersuites + during its v2 handshake renegotiation, allow the renegotiation to + continue nevertheless (i.e. send all the required certificates). + Fixes bug 4591; bugfix on 0.2.0.20-rc. + - When we receive a SIGHUP and the controller __ReloadTorrcOnSIGHUP + option is set to 0 (which Vidalia version 0.2.16 now does when + a SAVECONF attempt fails), perform other actions that SIGHUP + usually causes (like reopening the logs). Fixes bug 5095; bugfix + on 0.2.1.9-alpha. + - If we fail to write a microdescriptor to the disk cache, do not + continue replacing the old microdescriptor file. Fixes bug 2954; + bugfix on 0.2.2.6-alpha. + - Exit nodes don't need to fetch certificates for authorities that + they don't recognize; only directory authorities, bridges, + and caches need to do that. Fixes part of bug 2297; bugfix on + 0.2.2.11-alpha. + - Correctly handle checking the permissions on the parent + directory of a control socket in the root directory. Bug found + by Esteban Manchado Velázquez. Fixes bug 5089; bugfix on Tor + 0.2.2.26-beta. + - When told to add a bridge with the same digest as a preexisting + bridge but a different addr:port, change the addr:port as + requested. Previously we would not notice the change. Fixes half + of bug 5603; fix on 0.2.2.26-beta. + - End AUTHCHALLENGE error messages (in the control protocol) with + a CRLF. Fixes bug 5760; bugfix on 0.2.2.36 and 0.2.3.13-alpha. + + o Minor bugfixes (on 0.2.3.x): + - Turn an assertion (that the number of handshakes received as a + server is not < 1) into a warning. Fixes bug 4873; bugfix on + 0.2.3.1-alpha. + - Format IPv4 addresses correctly in ADDRMAP events. (Previously, + we had reversed them when the answer was cached.) Fixes bug + 5723; bugfix on 0.2.3.1-alpha. + - Work correctly on Linux systems with accept4 support advertised in + their headers, but without accept4 support in the kernel. Fix + by murb. Fixes bug 5762; bugfix on 0.2.3.1-alpha. + - When told to add a bridge with the same addr:port as a preexisting + bridge but a different transport, change the transport as + requested. Previously we would not notice the change. Fixes half + of bug 5603; fix on 0.2.3.2-alpha. + - Avoid a "double-reply" warning when replying to a SOCKS request + with a parse error. Patch from Fabian Keil. Fixes bug 4108; + bugfix on 0.2.3.4-alpha. + - Fix a bug where a bridge authority crashes if it has seen no + directory requests when it's time to write statistics to disk. + Fixes bug 5891; bugfix on 0.2.3.6-alpha. Also fixes bug 5508 in + a better way. + - Don't try to open non-control listeners when DisableNetwork is set. + Previously, we'd open all listeners, then immediately close them. + Fixes bug 5604; bugfix on 0.2.3.9-alpha. + - Don't abort the managed proxy protocol if the managed proxy + sends us an unrecognized line; ignore it instead. Fixes bug + 5910; bugfix on 0.2.3.9-alpha. + - Fix a compile warning in crypto.c when compiling with clang 3.1. + Fixes bug 5969, bugfix on 0.2.3.9-alpha. + - Fix a compilation issue on GNU Hurd, which doesn't have PATH_MAX. + Fixes bug 5355; bugfix on 0.2.3.11-alpha. + - Remove bogus definition of "_WIN32" from src/win32/orconfig.h, to + unbreak the MSVC build. Fixes bug 5858; bugfix on 0.2.3.12-alpha. + - Resolve numerous small warnings and build issues with MSVC. Resolves + bug 5859. + + o Documentation fixes: + - Improve the manual's documentation for the NT Service command-line + options. Addresses ticket 3964. + - Clarify SessionGroup documentation slightly; resolves ticket 5437. + - Document the changes to the ORPort and DirPort options, and the + fact that {OR/Dir}ListenAddress is now unnecessary (and + therefore deprecated). Resolves ticket 5597. + + o Removed files: + - Remove the torrc.bridge file: we don't use it for anything, and + it had become badly desynchronized from torrc.sample. Resolves + bug 5622. + + +Changes in version 0.2.2.36 - 2012-05-24 + Tor 0.2.2.36 updates the addresses for two of the eight directory + authorities, fixes some potential anonymity and security issues, + and fixes several crash bugs. + + Tor 0.2.1.x has reached its end-of-life. Those Tor versions have many + known flaws, and nobody should be using them. You should upgrade. If + you're using a Linux or BSD and its packages are obsolete, stop using + those packages and upgrade anyway. + + o Directory authority changes: + - Change IP address for maatuska (v3 directory authority). + - Change IP address for ides (v3 directory authority), and rename + it to turtles. + + o Security fixes: + - When building or running with any version of OpenSSL earlier + than 0.9.8s or 1.0.0f, disable SSLv3 support. These OpenSSL + versions have a bug (CVE-2011-4576) in which their block cipher + padding includes uninitialized data, potentially leaking sensitive + information to any peer with whom they make a SSLv3 connection. Tor + does not use SSL v3 by default, but a hostile client or server + could force an SSLv3 connection in order to gain information that + they shouldn't have been able to get. The best solution here is to + upgrade to OpenSSL 0.9.8s or 1.0.0f (or later). But when building + or running with a non-upgraded OpenSSL, we disable SSLv3 entirely + to make sure that the bug can't happen. + - Never use a bridge or a controller-supplied node as an exit, even + if its exit policy allows it. Found by wanoskarnet. Fixes bug + 5342. Bugfix on 0.1.1.15-rc (for controller-purpose descriptors) + and 0.2.0.3-alpha (for bridge-purpose descriptors). + - Only build circuits if we have a sufficient threshold of the total + descriptors that are marked in the consensus with the "Exit" + flag. This mitigates an attack proposed by wanoskarnet, in which + all of a client's bridges collude to restrict the exit nodes that + the client knows about. Fixes bug 5343. + - Provide controllers with a safer way to implement the cookie + authentication mechanism. With the old method, if another locally + running program could convince a controller that it was the Tor + process, then that program could trick the controller into telling + it the contents of an arbitrary 32-byte file. The new "SAFECOOKIE" + authentication method uses a challenge-response approach to prevent + this attack. Fixes bug 5185; implements proposal 193. + + o Major bugfixes: + - Avoid logging uninitialized data when unable to decode a hidden + service descriptor cookie. Fixes bug 5647; bugfix on 0.2.1.5-alpha. + - Avoid a client-side assertion failure when receiving an INTRODUCE2 + cell on a general purpose circuit. Fixes bug 5644; bugfix on + 0.2.1.6-alpha. + - Fix builds when the path to sed, openssl, or sha1sum contains + spaces, which is pretty common on Windows. Fixes bug 5065; bugfix + on 0.2.2.1-alpha. + - Correct our replacements for the timeradd() and timersub() functions + on platforms that lack them (for example, Windows). The timersub() + function is used when expiring circuits, while timeradd() is + currently unused. Bug report and patch by Vektor. Fixes bug 4778; + bugfix on 0.2.2.24-alpha. + - Fix the SOCKET_OK test that we use to tell when socket + creation fails so that it works on Win64. Fixes part of bug 4533; + bugfix on 0.2.2.29-beta. Bug found by wanoskarnet. + + o Minor bugfixes: + - Reject out-of-range times like 23:59:61 in parse_rfc1123_time(). + Fixes bug 5346; bugfix on 0.0.8pre3. + - Make our number-parsing functions always treat too-large values + as an error, even when those values exceed the width of the + underlying type. Previously, if the caller provided these + functions with minima or maxima set to the extreme values of the + underlying integer type, these functions would return those + values on overflow rather than treating overflow as an error. + Fixes part of bug 5786; bugfix on 0.0.9. + - Older Linux kernels erroneously respond to strange nmap behavior + by having accept() return successfully with a zero-length + socket. When this happens, just close the connection. Previously, + we would try harder to learn the remote address: but there was + no such remote address to learn, and our method for trying to + learn it was incorrect. Fixes bugs 1240, 4745, and 4747. Bugfix + on 0.1.0.3-rc. Reported and diagnosed by "r1eo". + - Correct parsing of certain date types in parse_http_time(). + Without this patch, If-Modified-Since would behave + incorrectly. Fixes bug 5346; bugfix on 0.2.0.2-alpha. Patch from + Esteban Manchado Velázques. + - Change the BridgePassword feature (part of the "bridge community" + design, which is not yet implemented) to use a time-independent + comparison. The old behavior might have allowed an adversary + to use timing to guess the BridgePassword value. Fixes bug 5543; + bugfix on 0.2.0.14-alpha. + - Detect and reject certain misformed escape sequences in + configuration values. Previously, these values would cause us + to crash if received in a torrc file or over an authenticated + control port. Bug found by Esteban Manchado Velázquez, and + independently by Robert Connolly from Matta Consulting who further + noted that it allows a post-authentication heap overflow. Patch + by Alexander Schrijver. Fixes bugs 5090 and 5402 (CVE 2012-1668); + bugfix on 0.2.0.16-alpha. + - Fix a compile warning when using the --enable-openbsd-malloc + configure option. Fixes bug 5340; bugfix on 0.2.0.20-rc. + - During configure, detect when we're building with clang version + 3.0 or lower and disable the -Wnormalized=id and -Woverride-init + CFLAGS. clang doesn't support them yet. + - When sending an HTTP/1.1 proxy request, include a Host header. + Fixes bug 5593; bugfix on 0.2.2.1-alpha. + - Fix a NULL-pointer dereference on a badly formed SETCIRCUITPURPOSE + command. Found by mikeyc. Fixes bug 5796; bugfix on 0.2.2.9-alpha. + - If we hit the error case where routerlist_insert() replaces an + existing (old) server descriptor, make sure to remove that + server descriptor from the old_routers list. Fix related to bug + 1776. Bugfix on 0.2.2.18-alpha. + + o Minor bugfixes (documentation and log messages): + - Fix a typo in a log message in rend_service_rendezvous_has_opened(). + Fixes bug 4856; bugfix on Tor 0.0.6. + - Update "ClientOnly" man page entry to explain that there isn't + really any point to messing with it. Resolves ticket 5005. + - Document the GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays + directory authority option (introduced in Tor 0.2.2.34). + - Downgrade the "We're missing a certificate" message from notice + to info: people kept mistaking it for a real problem, whereas it + is seldom the problem even when we are failing to bootstrap. Fixes + bug 5067; bugfix on 0.2.0.10-alpha. + - Correctly spell "connect" in a log message on failure to create a + controlsocket. Fixes bug 4803; bugfix on 0.2.2.26-beta. + - Clarify the behavior of MaxCircuitDirtiness with hidden service + circuits. Fixes issue 5259. + + o Minor features: + - Directory authorities now reject versions of Tor older than + 0.2.1.30, and Tor versions between 0.2.2.1-alpha and 0.2.2.20-alpha + inclusive. These versions accounted for only a small fraction of + the Tor network, and have numerous known security issues. Resolves + issue 4788. + - Update to the May 1 2012 Maxmind GeoLite Country database. + + - Feature removal: + - When sending or relaying a RELAY_EARLY cell, we used to convert + it to a RELAY cell if the connection was using the v1 link + protocol. This was a workaround for older versions of Tor, which + didn't handle RELAY_EARLY cells properly. Now that all supported + versions can handle RELAY_EARLY cells, and now that we're enforcing + the "no RELAY_EXTEND commands except in RELAY_EARLY cells" rule, + remove this workaround. Addresses bug 4786. + + +Changes in version 0.2.3.15-alpha - 2012-04-30 + Tor 0.2.3.15-alpha fixes a variety of smaller bugs, including making + the development branch build on Windows again. + + o Minor bugfixes (on 0.2.2.x and earlier): + - Make sure that there are no unhandled pending TLS errors before + reading from a TLS stream. We had checks in 0.1.0.3-rc, but + lost them in 0.1.0.5-rc when we refactored read_to_buf_tls(). + Bugfix on 0.1.0.5-rc; fixes bug 4528. + - Fix an assert that directory authorities could trigger on sighup + during some configuration state transitions. We now don't treat + it as a fatal error when the new descriptor we just generated in + init_keys() isn't accepted. Fixes bug 4438; bugfix on 0.2.1.9-alpha. + - After we pick a directory mirror, we would refuse to use it if + it's in our ExcludeExitNodes list, resulting in mysterious failures + to bootstrap for people who just wanted to avoid exiting from + certain locations. Fixes bug 5623; bugfix on 0.2.2.25-alpha. + - When building with --enable-static-tor on OpenBSD, do not + erroneously attempt to link -lrt. Fixes bug 5103. + + o Minor bugfixes (on 0.2.3.x): + - When Tor is built with kernel headers from a recent (last few + years) Linux kernel, do not fail to run on older (pre-2.6.28 + Linux kernels). Fixes bug 5112; bugfix on 0.2.3.1-alpha. + - Fix cross-compilation issues with mingw. Bugfixes on 0.2.3.6-alpha + and 0.2.3.12-alpha. + - Fix compilation with miniupnpc version 1.6; patch from + Anthony G. Basile. Fixes bug 5434; bugfix on 0.2.3.12-alpha. + - Fix compilation with MSVC, which had defined MS_WINDOWS. Bugfix + on 0.2.3.13-alpha; found and fixed by Gisle Vanem. + - Fix compilation on platforms without unistd.h, or where environ + is defined in stdlib.h. Fixes bug 5704; bugfix on 0.2.3.13-alpha. + + o Minor features: + - Directory authorities are now a little more lenient at accepting + older router descriptors, or newer router descriptors that don't + make big changes. This should help ameliorate past and future + issues where routers think they have uploaded valid descriptors, + but the authorities don't think so. Fix for ticket 2479. + - Make the code that clients use to detect an address change be + IPv6-aware, so that it won't fill clients' logs with error + messages when trying to get the IPv4 address of an IPv6 + connection. Implements ticket 5537. + + o Removed features: + - Remove the GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays option; + authorities needed to use it for a while to keep the network working + as people upgraded to 0.2.1.31, 0.2.2.34, or 0.2.3.6-alpha, but + that was six months ago. As of now, it should no longer be needed + or used. + + +Changes in version 0.2.3.14-alpha - 2012-04-23 + Tor 0.2.3.14-alpha fixes yet more bugs to get us closer to a release + candidate. It also dramatically speeds up AES: fast relays should + consider switching to the newer OpenSSL library. + + o Directory authority changes: + - Change IP address for ides (v3 directory authority), and rename + it to turtles. + + o Major bugfixes: + - Avoid logging uninitialized data when unable to decode a hidden + service descriptor cookie. Fixes bug 5647; bugfix on 0.2.1.5-alpha. + - Avoid a client-side assertion failure when receiving an INTRODUCE2 + cell on a general purpose circuit. Fixes bug 5644; bugfix on + 0.2.1.6-alpha. + - If authorities are unable to get a v2 consensus document from other + directory authorities, they no longer fall back to fetching + them from regular directory caches. Fixes bug 5635; bugfix on + 0.2.2.26-beta, where routers stopped downloading v2 consensus + documents entirely. + - When we start a Tor client with a normal consensus already cached, + be willing to download a microdescriptor consensus. Fixes bug 4011; + fix on 0.2.3.1-alpha. + + o Major features (performance): + - When built to use OpenSSL 1.0.1, and built for an x86 or x86_64 + instruction set, take advantage of OpenSSL's AESNI, bitsliced, or + vectorized AES implementations as appropriate. These can be much, + much faster than other AES implementations. + + o Minor bugfixes (0.2.2.x and earlier): + - Don't launch more than 10 service-side introduction-point circuits + for a hidden service in five minutes. Previously, we would consider + launching more introduction-point circuits if at least one second + had passed without any introduction-point circuits failing. Fixes + bug 4607; bugfix on 0.0.7pre1. + - Change the BridgePassword feature (part of the "bridge community" + design, which is not yet implemented) to use a time-independent + comparison. The old behavior might have allowed an adversary + to use timing to guess the BridgePassword value. Fixes bug 5543; + bugfix on 0.2.0.14-alpha. + - Enforce correct return behavior of tor_vsscanf() when the '%%' + pattern is used. Fixes bug 5558. Bugfix on 0.2.1.13. + - When sending an HTTP/1.1 proxy request, include a Host header. + Fixes bug 5593; bugfix on 0.2.2.1-alpha. + - Don't log that we have "decided to publish new relay descriptor" + unless we are actually publishing a descriptor. Fixes bug 3942; + bugfix on 0.2.2.28-beta. + + o Minor bugfixes (0.2.3.x): + - Fix a bug where a bridge authority crashes (on a failed assert) + if it has seen no directory requests when it's time to write + statistics to disk. Fixes bug 5508. Bugfix on 0.2.3.6-alpha. + - Fix bug stomping on ORPort option NoListen and ignoring option + NoAdvertise. Fixes bug 5151; bugfix on 0.2.3.9-alpha. + - In the testsuite, provide a large enough buffer in the tor_sscanf + unit test. Otherwise we'd overrun that buffer and crash during + the unit tests. Found by weasel. Fixes bug 5449; bugfix on + 0.2.3.12-alpha. + - Make sure we create the keys directory if it doesn't exist and we're + about to store the dynamic Diffie-Hellman parameters. Fixes bug + 5572; bugfix on 0.2.3.13-alpha. + - Fix a small memory leak when trying to decode incorrect base16 + authenticator during SAFECOOKIE authentication. Found by + Coverity Scan. Fixes CID 507. Bugfix on 0.2.3.13-alpha. + + o Minor features: + - Add more information to a log statement that might help track down + bug 4091. If you're seeing "Bug: tor_addr_is_internal() called with a + non-IP address" messages (or any Bug messages, for that matter!), + please let us know about it. + - Relays now understand an IPv6 address when they get one from a + directory server. Resolves ticket 4875. + - Resolve IPv6 addresses in bridge and entry statistics to country + code "??" which means we at least count them. Resolves ticket 5053; + improves on 0.2.3.9-alpha. + - Update to the April 3 2012 Maxmind GeoLite Country database. + - Begin a doc/state-contents.txt file to explain the contents of + the Tor state file. Fixes bug 2987. + + o Default torrc changes: + - Stop listing "socksport 9050" in torrc.sample. We open a socks + port on 9050 by default anyway, so this should not change anything + in practice. + - Stop mentioning the deprecated *ListenAddress options in + torrc.sample. Fixes bug 5438. + - Document unit of bandwidth related options in sample torrc. + Fixes bug 5621. + + o Removed features: + - The "torify" script no longer supports the "tsocks" socksifier + tool, since tsocks doesn't support DNS and UDP right for Tor. + Everyone should be using torsocks instead. Fixes bugs 3530 and + 5180. Based on a patch by "ugh". + + o Code refactoring: + - Change the symmetric cipher interface so that creating and + initializing a stream cipher are no longer separate functions. + - Remove all internal support for unpadded RSA. We never used it, and + it would be a bad idea to start. + + +Changes in version 0.2.3.13-alpha - 2012-03-26 + Tor 0.2.3.13-alpha fixes a variety of stability and correctness bugs + in managed pluggable transports, as well as providing other cleanups + that get us closer to a release candidate. + + o Directory authority changes: + - Change IP address for maatuska (v3 directory authority). + + o Security fixes: + - Provide controllers with a safer way to implement the cookie + authentication mechanism. With the old method, if another locally + running program could convince a controller that it was the Tor + process, then that program could trick the controller into telling + it the contents of an arbitrary 32-byte file. The new "SAFECOOKIE" + authentication method uses a challenge-response approach to prevent + this attack. Fixes bug 5185, implements proposal 193. + - Never use a bridge or a controller-supplied node as an exit, even + if its exit policy allows it. Found by wanoskarnet. Fixes bug + 5342. Bugfix on 0.1.1.15-rc (for controller-purpose descriptors) + and 0.2.0.3-alpha (for bridge-purpose descriptors). + - Only build circuits if we have a sufficient threshold of the total + descriptors that are marked in the consensus with the "Exit" + flag. This mitigates an attack proposed by wanoskarnet, in which + all of a client's bridges collude to restrict the exit nodes that + the client knows about. Fixes bug 5343. + + o Major bugfixes (on Tor 0.2.3.x): + - Avoid an assert when managed proxies like obfsproxy are configured, + and we receive HUP signals or setconf attempts too rapidly. This + situation happens most commonly when Vidalia tries to attach to + Tor or tries to configure the Tor it's attached to. Fixes bug 5084; + bugfix on 0.2.3.6-alpha. + - Fix a relay-side pluggable transports bug where managed proxies were + unreachable from the Internet, because Tor asked them to bind on + localhost. Fixes bug 4725; bugfix on 0.2.3.9-alpha. + - Stop discarding command-line arguments when TestingTorNetwork + is set. Discovered by Kevin Bauer. Fixes bug 5373; bugfix on + 0.2.3.9-alpha, where task 4552 added support for two layers of + torrc files. + - Resume allowing the unit tests to run in gdb. This was accidentally + made impossible when the DisableDebuggerAttachment option was + introduced. Fixes bug 5448; bugfix on 0.2.3.9-alpha. + - Resume building with nat-pmp support. Fixes bug 4955; bugfix on + 0.2.3.11-alpha. Reported by Anthony G. Basile. + + o Minor bugfixes (on 0.2.2.x and earlier): + - Ensure we don't cannibalize circuits that are longer than three hops + already, so we don't end up making circuits with 5 or more + hops. Patch contributed by wanoskarnet. Fixes bug 5231; bugfix on + 0.1.0.1-rc which introduced cannibalization. + - Detect and reject certain misformed escape sequences in + configuration values. Previously, these values would cause us + to crash if received in a torrc file or over an authenticated + control port. Bug found by Esteban Manchado Velázquez, and + independently by Robert Connolly from Matta Consulting who further + noted that it allows a post-authentication heap overflow. Patch + by Alexander Schrijver. Fixes bugs 5090 and 5402 (CVE 2012-1668); + bugfix on 0.2.0.16-alpha. + - Fix a compile warning when using the --enable-openbsd-malloc + configure option. Fixes bug 5340; bugfix on 0.2.0.20-rc. + - Directory caches no longer refuse to clean out descriptors because + of missing v2 networkstatus documents, unless they're configured + to retrieve v2 networkstatus documents. Fixes bug 4838; bugfix on + 0.2.2.26-beta. Patch by Daniel Bryg. + - Update to the latest version of the tinytest unit testing framework. + This includes a couple of bugfixes that can be relevant for + running forked unit tests on Windows, and removes all reserved + identifiers. + + o Minor bugfixes (on 0.2.3.x): + - On a failed pipe() call, don't leak file descriptors. Fixes bug + 4296; bugfix on 0.2.3.1-alpha. + - Spec conformance: on a v3 handshake, do not send a NETINFO cell + until after we have received a CERTS cell. Fixes bug 4361; bugfix + on 0.2.3.6-alpha. Patch by "frosty". + - When binding to an IPv6 address, set the IPV6_V6ONLY socket + option, so that the IP stack doesn't decide to use it for IPv4 + too. Fixes bug 4760; bugfix on 0.2.3.9-alpha. + - Ensure that variables set in Tor's environment cannot override + environment variables that Tor passes to a managed + pluggable-transport proxy. Previously, Tor would pass every + variable in its environment to managed proxies along with the new + ones, in such a way that on many operating systems, the inherited + environment variables would override those which Tor tried to + explicitly set. Bugfix on 0.2.3.12-alpha for most Unixoid systems; + bugfix on 0.2.3.9-alpha for Windows. + + o Minor features: + - A wide variety of new unit tests by Esteban Manchado Velázquez. + - Shorten links in the tor-exit-notice file. Patch by Christian Kujau. + - Update to the March 6 2012 Maxmind GeoLite Country database. + + +Changes in version 0.2.3.12-alpha - 2012-02-13 + Tor 0.2.3.12-alpha lets fast exit relays scale better, allows clients + to use bridges that run Tor 0.2.2.x, and resolves several big bugs + when Tor is configured to use a pluggable transport like obfsproxy. + + o Major bugfixes: + - Fix builds when the path to sed, openssl, or sha1sum contains + spaces, which is pretty common on Windows. Fixes bug 5065; bugfix + on 0.2.2.1-alpha. + - Set the SO_REUSEADDR socket option before we call bind() on outgoing + connections. This change should allow busy exit relays to stop + running out of available sockets as quickly. Fixes bug 4950; + bugfix on 0.2.2.26-beta. + - Allow 0.2.3.x clients to use 0.2.2.x bridges. Previously the client + would ask the bridge for microdescriptors, which are only supported + in 0.2.3.x, and then fail to bootstrap when it didn't get the + answers it wanted. Fixes bug 4013; bugfix on 0.2.3.2-alpha. + - Properly set up obfsproxy's environment when in managed mode. The + Tor Browser Bundle needs LD_LIBRARY_PATH to be passed to obfsproxy, + and when you run your Tor as a daemon, there's no HOME. Fixes bugs + 5076 and 5082; bugfix on 0.2.3.6-alpha. + + o Minor features: + - Use the dead_strip option when building Tor on OS X. This reduces + binary size by almost 19% when linking openssl and libevent + statically, which we do for Tor Browser Bundle. + - Fix broken URLs in the sample torrc file, and tell readers about + the OutboundBindAddress, ExitPolicyRejectPrivate, and + PublishServerDescriptor options. Addresses bug 4652. + - Update to the February 7 2012 Maxmind GeoLite Country database. + + o Minor bugfixes: + - Downgrade the "We're missing a certificate" message from notice + to info: people kept mistaking it for a real problem, whereas it + is seldom the problem even when we are failing to bootstrap. Fixes + bug 5067; bugfix on 0.2.0.10-alpha. + - Don't put "TOR_PT_EXTENDED_SERVER_PORT=127.0.0.1:4200" in a + managed pluggable transport server proxy's environment. + Previously, we would put it there, even though Tor doesn't + implement an 'extended server port' yet, and even though Tor + almost certainly isn't listening at that address. For now, we set + it to an empty string to avoid crashing older obfsproxies. Bugfix + on 0.2.3.6-alpha. + - Log the heartbeat message every HeartbeatPeriod seconds, not every + HeartbeatPeriod + 1 seconds. Fixes bug 4942; bugfix on + 0.2.3.1-alpha. Bug reported by Scott Bennett. + - Calculate absolute paths correctly on Windows. Fixes bug 4973; + bugfix on 0.2.3.11-alpha. + - Update "ClientOnly" man page entry to explain that there isn't + really any point to messing with it. Resolves ticket 5005. + - Use the correct CVE number for CVE-2011-4576 in our comments and + log messages. Found by "fermenthor". Resolves bug 5066; bugfix on + 0.2.3.11-alpha. + + o Code simplifications and refactoring: + - Use the _WIN32 macro throughout our code to detect Windows. + (Previously we had used the obsolete 'WIN32' and the idiosyncratic + 'MS_WINDOWS'.) + + +Changes in version 0.2.3.11-alpha - 2012-01-22 + Tor 0.2.3.11-alpha marks feature-freeze for the 0.2.3 tree. It deploys + the last step of the plan to limit maximum circuit length, includes + a wide variety of hidden service performance and correctness fixes, + works around an OpenSSL security flaw if your distro is too stubborn + to upgrade, and fixes a bunch of smaller issues. + + o Major features: + - Now that Tor 0.2.0.x is completely deprecated, enable the final + part of "Proposal 110: Avoiding infinite length circuits" by + refusing all circuit-extend requests that do not use a relay_early + cell. This change helps Tor resist a class of denial-of-service + attacks by limiting the maximum circuit length. + - Adjust the number of introduction points that a hidden service + will try to maintain based on how long its introduction points + remain in use and how many introductions they handle. Fixes + part of bug 3825. + - Try to use system facilities for enumerating local interface + addresses, before falling back to our old approach (which was + binding a UDP socket, and calling getsockname() on it). That + approach was scaring OS X users whose draconian firewall + software warned about binding to UDP sockets, regardless of + whether packets were sent. Now we try to use getifaddrs(), + SIOCGIFCONF, or GetAdaptersAddresses(), depending on what the + system supports. Resolves ticket 1827. + + o Major security workaround: + - When building or running with any version of OpenSSL earlier + than 0.9.8s or 1.0.0f, disable SSLv3 support. These OpenSSL + versions have a bug (CVE-2011-4576) in which their block cipher + padding includes uninitialized data, potentially leaking sensitive + information to any peer with whom they make a SSLv3 connection. Tor + does not use SSL v3 by default, but a hostile client or server + could force an SSLv3 connection in order to gain information that + they shouldn't have been able to get. The best solution here is to + upgrade to OpenSSL 0.9.8s or 1.0.0f (or later). But when building + or running with a non-upgraded OpenSSL, we disable SSLv3 entirely + to make sure that the bug can't happen. + + o Major bugfixes: + - Fix the SOCKET_OK test that we use to tell when socket + creation fails so that it works on Win64. Fixes part of bug 4533; + bugfix on 0.2.2.29-beta. Bug found by wanoskarnet. + - Correct our replacements for the timeradd() and timersub() functions + on platforms that lack them (for example, Windows). The timersub() + function is used when expiring circuits, while timeradd() is + currently unused. Bug report and patch by Vektor. Fixes bug 4778; + bugfix on 0.2.2.24-alpha and 0.2.3.1-alpha. + - Do not use OpenSSL 1.0.0's counter mode: it has a critical bug + that was fixed in OpenSSL 1.0.0a. We test for the counter mode + bug at runtime, not compile time, because some distributions hack + their OpenSSL to mis-report its version. Fixes bug 4779; bugfix + on 0.2.3.9-alpha. Found by Pascal. + + o Minor features (controller): + - Use absolute path names when reporting the torrc filename in the + control protocol, so a controller can more easily find the torrc + file. Resolves bug 1101. + - Extend the control protocol to report flags that control a circuit's + path selection in CIRC events and in replies to 'GETINFO + circuit-status'. Implements part of ticket 2411. + - Extend the control protocol to report the hidden service address + and current state of a hidden-service-related circuit in CIRC + events and in replies to 'GETINFO circuit-status'. Implements part + of ticket 2411. + - When reporting the path to the cookie file to the controller, + give an absolute path. Resolves ticket 4881. + - Allow controllers to request an event notification whenever a + circuit is cannibalized or its purpose is changed. Implements + part of ticket 3457. + - Include the creation time of a circuit in CIRC and CIRC2 + control-port events and the list produced by the 'GETINFO + circuit-status' control-port command. + + o Minor features (directory authorities): + - Directory authorities now reject versions of Tor older than + 0.2.1.30, and Tor versions between 0.2.2.1-alpha and 0.2.2.20-alpha + inclusive. These versions accounted for only a small fraction of + the Tor network, and have numerous known security issues. Resolves + issue 4788. + - Authority operators can now vote for all relays in a given + set of countries to be BadDir/BadExit/Invalid/Rejected. + - Provide two consensus parameters (FastFlagMinThreshold and + FastFlagMaxThreshold) to control the range of allowable bandwidths + for the Fast directory flag. These allow authorities to run + experiments on appropriate requirements for being a "Fast" node. + The AuthDirFastGuarantee config value still applies. + - Document the GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays + directory authority option (introduced in Tor 0.2.2.34). + + o Minor features (other): + - Don't disable the DirPort when we cannot exceed our AccountingMax + limit during this interval because the effective bandwidthrate is + low enough. This is useful in a situation where AccountMax is only + used as an additional safeguard or to provide statistics. + - Prepend an informative header to generated dynamic_dh_params files. + - If EntryNodes are given, but UseEntryGuards is set to 0, warn that + EntryNodes will have no effect. Resolves issue 2571. + - Log more useful messages when we fail to disable debugger + attachment. + - Log which authority we're missing votes from when we go to fetch + them from the other auths. + - Log (at debug level) whenever a circuit's purpose is changed. + - Add missing documentation for the MaxClientCircuitsPending, + UseMicrodescriptors, UserspaceIOCPBuffers, and + _UseFilteringSSLBufferevents options, all introduced during + the 0.2.3.x series. + - Update to the January 3 2012 Maxmind GeoLite Country database. + + o Minor bugfixes (hidden services): + - Don't close hidden service client circuits which have almost + finished connecting to their destination when they reach + the normal circuit-build timeout. Previously, we would close + introduction circuits which are waiting for an acknowledgement + from the introduction point, and rendezvous circuits which have + been specified in an INTRODUCE1 cell sent to a hidden service, + after the normal CBT. Now, we mark them as 'timed out', and launch + another rendezvous attempt in parallel. This behavior change can + be disabled using the new CloseHSClientCircuitsImmediatelyOnTimeout + option. Fixes part of bug 1297; bugfix on 0.2.2.2-alpha. + - Don't close hidden-service-side rendezvous circuits when they + reach the normal circuit-build timeout. This behaviour change can + be disabled using the new + CloseHSServiceRendCircuitsImmediatelyOnTimeout option. Fixes the + remaining part of bug 1297; bugfix on 0.2.2.2-alpha. + - Make sure we never mark the wrong rendezvous circuit as having + had its introduction cell acknowleged by the introduction-point + relay. Previously, when we received an INTRODUCE_ACK cell on a + client-side hidden-service introduction circuit, we might have + marked a rendezvous circuit other than the one we specified in + the INTRODUCE1 cell as INTRO_ACKED, which would have produced + a warning message and interfered with the hidden service + connection-establishment process. Fixes bug 4759; bugfix on + 0.2.3.3-alpha, when we added the stream-isolation feature which + might cause Tor to open multiple rendezvous circuits for the same + hidden service. + - Don't trigger an assertion failure when we mark a new client-side + hidden-service introduction circuit for close during the process + of creating it. Fixes bug 4796; bugfix on 0.2.3.6-alpha. Reported + by murb. + + o Minor bugfixes (log messages): + - Correctly spell "connect" in a log message on failure to create a + controlsocket. Fixes bug 4803; bugfix on 0.2.2.26-beta and + 0.2.3.2-alpha. + - Fix a typo in a log message in rend_service_rendezvous_has_opened(). + Fixes bug 4856; bugfix on Tor 0.0.6. + - Fix the log message describing how we work around discovering + that our version is the ill-fated OpenSSL 0.9.8l. Fixes bug + 4837; bugfix on 0.2.2.9-alpha. + - When logging about a disallowed .exit name, do not also call it + an "invalid onion address". Fixes bug 3325; bugfix on 0.2.2.9-alpha. + + o Minor bugfixes (build fixes): + - During configure, detect when we're building with clang version + 3.0 or lower and disable the -Wnormalized=id and -Woverride-init + CFLAGS. clang doesn't support them yet. + - During configure, search for library containing cos function as + libm lives in libcore on some platforms (BeOS/Haiku). Linking + against libm was hard-coded before. Fixes the first part of bug + 4727; bugfix on 0.2.2.2-alpha. Patch and analysis by Martin Hebnes + Pedersen. + - Detect attempts to build Tor on (as yet hypothetical) versions + of Windows where sizeof(intptr_t) != sizeof(SOCKET). Partial + fix for bug 4533. Bugfix on 0.2.2.28-beta. + - Preprocessor directives should not be put inside the arguments + of a macro. This would break compilation with GCC releases prior + to version 3.3. We would never recommend such an old GCC version, + but it is apparently required for binary compatibility on some + platforms (namely, certain builds of Haiku). Fixes the other part + of bug 4727; bugfix on 0.2.3.3-alpha. Patch and analysis by Martin + Hebnes Pedersen. + + o Minor bugfixes (other): + - Older Linux kernels erroneously respond to strange nmap behavior + by having accept() return successfully with a zero-length + socket. When this happens, just close the connection. Previously, + we would try harder to learn the remote address: but there was + no such remote address to learn, and our method for trying to + learn it was incorrect. Fixes bugs 1240, 4745, and 4747. Bugfix + on 0.1.0.3-rc. Reported and diagnosed by "r1eo". + - Fix null-pointer access that could occur if TLS allocation failed. + Fixes bug 4531; bugfix on 0.2.0.20-rc. Found by "troll_un". This was + erroneously listed as fixed in 0.2.3.9-alpha, but the fix had + accidentally been reverted. + - Fix our implementation of crypto_random_hostname() so it can't + overflow on ridiculously large inputs. (No Tor version has ever + provided this kind of bad inputs, but let's be correct in depth.) + Fixes bug 4413; bugfix on 0.2.2.9-alpha. Fix by Stephen Palmateer. + - Find more places in the code that should have been testing for + invalid sockets using the SOCKET_OK macro. Required for a fix + for bug 4533. Bugfix on 0.2.2.28-beta. + - Fix an assertion failure when, while running with bufferevents, a + connection finishes connecting after it is marked for close, but + before it is closed. Fixes bug 4697; bugfix on 0.2.3.1-alpha. + - test_util_spawn_background_ok() hardcoded the expected value + for ENOENT to 2. This isn't portable as error numbers are + platform specific, and particularly the hurd has ENOENT at + 0x40000002. Construct expected string at runtime, using the correct + value for ENOENT. Fixes bug 4733; bugfix on 0.2.3.1-alpha. + - Reject attempts to disable DisableDebuggerAttachment while Tor is + running. Fixes bug 4650; bugfix on 0.2.3.9-alpha. + - Use an appropriate-width type for sockets in tor-fw-helper on + win64. Fixes bug 1983 at last. Bugfix on 0.2.3.9-alpha. + + o Feature removal: + - When sending or relaying a RELAY_EARLY cell, we used to convert + it to a RELAY cell if the connection was using the v1 link + protocol. This was a workaround for older versions of Tor, which + didn't handle RELAY_EARLY cells properly. Now that all supported + versions can handle RELAY_EARLY cells, and now that we're enforcing + the "no RELAY_EXTEND commands except in RELAY_EARLY cells" rule, + remove this workaround. Addresses bug 4786. + + o Code simplifications and refactoring: + - Use OpenSSL's built-in SSL_state_string_long() instead of our + own homebrewed ssl_state_to_string() replacement. Patch from + Emile Snyder. Fixes bug 4653. + - Use macros to indicate OpenSSL versions, so we don't need to worry + about accidental hexadecimal bit shifts. + - Remove some workaround code for OpenSSL 0.9.6 (which is no longer + supported). + - Convert more instances of tor_snprintf+tor_strdup into tor_asprintf. + - Use the smartlist_add_asprintf() alias more consistently. + - Use a TOR_INVALID_SOCKET macro when initializing a socket to an + invalid value, rather than just -1. + - Rename a handful of old identifiers, mostly related to crypto + structures and crypto functions. By convention, our "create an + object" functions are called "type_new()", our "free an object" + functions are called "type_free()", and our types indicate that + they are types only with a final "_t". But a handful of older + types and functions broke these rules, with function names like + "type_create" or "subsystem_op_type", or with type names like + type_env_t. + + +Changes in version 0.2.3.10-alpha - 2011-12-16 + Tor 0.2.3.10-alpha fixes a critical heap-overflow security issue in + Tor's buffers code. Absolutely everybody should upgrade. + + The bug relied on an incorrect calculation when making data continuous + in one of our IO buffers, if the first chunk of the buffer was + misaligned by just the wrong amount. The miscalculation would allow an + attacker to overflow a piece of heap-allocated memory. To mount this + attack, the attacker would need to either open a SOCKS connection to + Tor's SocksPort (usually restricted to localhost), or target a Tor + instance configured to make its connections through a SOCKS proxy + (which Tor does not do by default). + + Good security practice requires that all heap-overflow bugs should be + presumed to be exploitable until proven otherwise, so we are treating + this as a potential code execution attack. Please upgrade immediately! + This bug does not affect bufferevents-based builds of Tor. Special + thanks to "Vektor" for reporting this issue to us! + + This release also contains a few minor bugfixes for issues discovered + in 0.2.3.9-alpha. + + o Major bugfixes: + - Fix a heap overflow bug that could occur when trying to pull + data into the first chunk of a buffer, when that chunk had + already had some data drained from it. Fixes CVE-2011-2778; + bugfix on 0.2.0.16-alpha. Reported by "Vektor". + + o Minor bugfixes: + - If we can't attach streams to a rendezvous circuit when we + finish connecting to a hidden service, clear the rendezvous + circuit's stream-isolation state and try to attach streams + again. Previously, we cleared rendezvous circuits' isolation + state either too early (if they were freshly built) or not at all + (if they had been built earlier and were cannibalized). Bugfix on + 0.2.3.3-alpha; fixes bug 4655. + - Fix compilation of the libnatpmp helper on non-Windows. Bugfix on + 0.2.3.9-alpha; fixes bug 4691. Reported by Anthony G. Basile. + - Fix an assertion failure when a relay with accounting enabled + starts up while dormant. Fixes bug 4702; bugfix on 0.2.3.9-alpha. + + o Minor features: + - Update to the December 6 2011 Maxmind GeoLite Country database. + + +Changes in version 0.2.2.35 - 2011-12-16 + Tor 0.2.2.35 fixes a critical heap-overflow security issue in Tor's + buffers code. Absolutely everybody should upgrade. + + The bug relied on an incorrect calculation when making data continuous + in one of our IO buffers, if the first chunk of the buffer was + misaligned by just the wrong amount. The miscalculation would allow an + attacker to overflow a piece of heap-allocated memory. To mount this + attack, the attacker would need to either open a SOCKS connection to + Tor's SocksPort (usually restricted to localhost), or target a Tor + instance configured to make its connections through a SOCKS proxy + (which Tor does not do by default). + + Good security practice requires that all heap-overflow bugs should be + presumed to be exploitable until proven otherwise, so we are treating + this as a potential code execution attack. Please upgrade immediately! + This bug does not affect bufferevents-based builds of Tor. Special + thanks to "Vektor" for reporting this issue to us! + + Tor 0.2.2.35 also fixes several bugs in previous versions, including + crash bugs for unusual configurations, and a long-term bug that + would prevent Tor from starting on Windows machines with draconian + AV software. + + With this release, we remind everyone that 0.2.0.x has reached its + formal end-of-life. Those Tor versions have many known flaws, and + nobody should be using them. You should upgrade -- ideally to the + 0.2.2.x series. If you're using a Linux or BSD and its packages are + obsolete, stop using those packages and upgrade anyway. + + The Tor 0.2.1.x series is also approaching its end-of-life: it will no + longer receive support after some time in early 2012. + + o Major bugfixes: + - Fix a heap overflow bug that could occur when trying to pull + data into the first chunk of a buffer, when that chunk had + already had some data drained from it. Fixes CVE-2011-2778; + bugfix on 0.2.0.16-alpha. Reported by "Vektor". + - Initialize Libevent with the EVENT_BASE_FLAG_NOLOCK flag enabled, so + that it doesn't attempt to allocate a socketpair. This could cause + some problems on Windows systems with overzealous firewalls. Fix for + bug 4457; workaround for Libevent versions 2.0.1-alpha through + 2.0.15-stable. + - If we mark an OR connection for close based on a cell we process, + don't process any further cells on it. We already avoid further + reads on marked-for-close connections, but now we also discard the + cells we'd already read. Fixes bug 4299; bugfix on 0.2.0.10-alpha, + which was the first version where we might mark a connection for + close based on processing a cell on it. + - Correctly sanity-check that we don't underflow on a memory + allocation (and then assert) for hidden service introduction + point decryption. Bug discovered by Dan Rosenberg. Fixes bug 4410; + bugfix on 0.2.1.5-alpha. + - Fix a memory leak when we check whether a hidden service + descriptor has any usable introduction points left. Fixes bug + 4424. Bugfix on 0.2.2.25-alpha. + - Don't crash when we're running as a relay and don't have a GeoIP + file. Bugfix on 0.2.2.34; fixes bug 4340. This backports a fix + we've had in the 0.2.3.x branch already. + - When running as a client, do not print a misleading (and plain + wrong) log message that we're collecting "directory request" + statistics: clients don't collect statistics. Also don't create a + useless (because empty) stats file in the stats/ directory. Fixes + bug 4353; bugfix on 0.2.2.34. + + o Minor bugfixes: + - Detect failure to initialize Libevent. This fix provides better + detection for future instances of bug 4457. + - Avoid frequent calls to the fairly expensive cull_wedged_cpuworkers + function. This was eating up hideously large amounts of time on some + busy servers. Fixes bug 4518; bugfix on 0.0.9.8. + - Resolve an integer overflow bug in smartlist_ensure_capacity(). + Fixes bug 4230; bugfix on Tor 0.1.0.1-rc. Based on a patch by + Mansour Moufid. + - Don't warn about unused log_mutex in log.c when building with + --disable-threads using a recent GCC. Fixes bug 4437; bugfix on + 0.1.0.6-rc which introduced --disable-threads. + - When configuring, starting, or stopping an NT service, stop + immediately after the service configuration attempt has succeeded + or failed. Fixes bug 3963; bugfix on 0.2.0.7-alpha. + - When sending a NETINFO cell, include the original address + received for the other side, not its canonical address. Found + by "troll_un"; fixes bug 4349; bugfix on 0.2.0.10-alpha. + - Fix a typo in a hibernation-related log message. Fixes bug 4331; + bugfix on 0.2.2.23-alpha; found by "tmpname0901". + - Fix a memory leak in launch_direct_bridge_descriptor_fetch() that + occurred when a client tried to fetch a descriptor for a bridge + in ExcludeNodes. Fixes bug 4383; bugfix on 0.2.2.25-alpha. + - Backport fixes for a pair of compilation warnings on Windows. + Fixes bug 4521; bugfix on 0.2.2.28-beta and on 0.2.2.29-beta. + - If we had ever tried to call tor_addr_to_str on an address of + unknown type, we would have done a strdup on an uninitialized + buffer. Now we won't. Fixes bug 4529; bugfix on 0.2.1.3-alpha. + Reported by "troll_un". + - Correctly detect and handle transient lookup failures from + tor_addr_lookup. Fixes bug 4530; bugfix on 0.2.1.5-alpha. + Reported by "troll_un". + - Fix null-pointer access that could occur if TLS allocation failed. + Fixes bug 4531; bugfix on 0.2.0.20-rc. Found by "troll_un". + - Use tor_socket_t type for listener argument to accept(). Fixes bug + 4535; bugfix on 0.2.2.28-beta. Found by "troll_un". + + o Minor features: + - Add two new config options for directory authorities: + AuthDirFastGuarantee sets a bandwidth threshold for guaranteeing the + Fast flag, and AuthDirGuardBWGuarantee sets a bandwidth threshold + that is always sufficient to satisfy the bandwidth requirement for + the Guard flag. Now it will be easier for researchers to simulate + Tor networks with different values. Resolves ticket 4484. + - When Tor ignores a hidden service specified in its configuration, + include the hidden service's directory in the warning message. + Previously, we would only tell the user that some hidden service + was ignored. Bugfix on 0.0.6; fixes bug 4426. + - Update to the December 6 2011 Maxmind GeoLite Country database. + + o Packaging changes: + - Make it easier to automate expert package builds on Windows, + by removing an absolute path from makensis.exe command. + + +Changes in version 0.2.3.9-alpha - 2011-12-08 + Tor 0.2.3.9-alpha introduces initial IPv6 support for bridges, adds + a "DisableNetwork" security feature that bundles can use to avoid + touching the network until bridges are configured, moves forward on + the pluggable transport design, fixes a flaw in the hidden service + design that unnecessarily prevented clients with wrong clocks from + reaching hidden services, and fixes a wide variety of other issues. + + o Major features: + - Clients can now connect to private bridges over IPv6. Bridges + still need at least one IPv4 address in order to connect to + other relays. Note that we don't yet handle the case where the + user has two bridge lines for the same bridge (one IPv4, one + IPv6). Implements parts of proposal 186. + - New "DisableNetwork" config option to prevent Tor from launching any + connections or accepting any connections except on a control port. + Bundles and controllers can set this option before letting Tor talk + to the rest of the network, for example to prevent any connections + to a non-bridge address. Packages like Orbot can also use this + option to instruct Tor to save power when the network is off. + - Clients and bridges can now be configured to use a separate + "transport" proxy. This approach makes the censorship arms race + easier by allowing bridges to use protocol obfuscation plugins. It + implements the "managed proxy" part of proposal 180 (ticket 3472). + - When using OpenSSL 1.0.0 or later, use OpenSSL's counter mode + implementation. It makes AES_CTR about 7% faster than our old one + (which was about 10% faster than the one OpenSSL used to provide). + Resolves ticket 4526. + - Add a "tor2web mode" for clients that want to connect to hidden + services non-anonymously (and possibly more quickly). As a safety + measure to try to keep users from turning this on without knowing + what they are doing, tor2web mode must be explicitly enabled at + compile time, and a copy of Tor compiled to run in tor2web mode + cannot be used as a normal Tor client. Implements feature 2553. + - Add experimental support for running on Windows with IOCP and no + kernel-space socket buffers. This feature is controlled by a new + "UserspaceIOCPBuffers" config option (off by default), which has + no effect unless Tor has been built with support for bufferevents, + is running on Windows, and has enabled IOCP. This may, in the long + run, help solve or mitigate bug 98. + - Use a more secure consensus parameter voting algorithm. Now at + least three directory authorities or a majority of them must + vote on a given parameter before it will be included in the + consensus. Implements proposal 178. + + o Major bugfixes: + - Hidden services now ignore the timestamps on INTRODUCE2 cells. + They used to check that the timestamp was within 30 minutes + of their system clock, so they could cap the size of their + replay-detection cache, but that approach unnecessarily refused + service to clients with wrong clocks. Bugfix on 0.2.1.6-alpha, when + the v3 intro-point protocol (the first one which sent a timestamp + field in the INTRODUCE2 cell) was introduced; fixes bug 3460. + - Only use the EVP interface when AES acceleration is enabled, + to avoid a 5-7% performance regression. Resolves issue 4525; + bugfix on 0.2.3.8-alpha. + + o Privacy/anonymity features (bridge detection): + - Make bridge SSL certificates a bit more stealthy by using random + serial numbers, in the same fashion as OpenSSL when generating + self-signed certificates. Implements ticket 4584. + - Introduce a new config option "DynamicDHGroups", enabled by + default, which provides each bridge with a unique prime DH modulus + to be used during SSL handshakes. This option attempts to help + against censors who might use the Apache DH modulus as a static + identifier for bridges. Addresses ticket 4548. + + o Minor features (new/different config options): + - New configuration option "DisableDebuggerAttachment" (on by default) + to prevent basic debugging attachment attempts by other processes. + Supports Mac OS X and Gnu/Linux. Resolves ticket 3313. + - Allow MapAddress directives to specify matches against super-domains, + as in "MapAddress *.torproject.org *.torproject.org.torserver.exit". + Implements issue 933. + - Slightly change behavior of "list" options (that is, config + options that can appear more than once) when they appear both in + torrc and on the command line. Previously, the command-line options + would be appended to the ones from torrc. Now, the command-line + options override the torrc options entirely. This new behavior + allows the user to override list options (like exit policies and + ports to listen on) from the command line, rather than simply + appending to the list. + - You can get the old (appending) command-line behavior for "list" + options by prefixing the option name with a "+". + - You can remove all the values for a "list" option from the command + line without adding any new ones by prefixing the option name + with a "/". + - Add experimental support for a "defaults" torrc file to be parsed + before the regular torrc. Torrc options override the defaults file's + options in the same way that the command line overrides the torrc. + The SAVECONF controller command saves only those options which + differ between the current configuration and the defaults file. HUP + reloads both files. (Note: This is an experimental feature; its + behavior will probably be refined in future 0.2.3.x-alpha versions + to better meet packagers' needs.) Implements task 4552. + + o Minor features: + - Try to make the introductory warning message that Tor prints on + startup more useful for actually finding help and information. + Resolves ticket 2474. + - Running "make version" now displays the version of Tor that + we're about to build. Idea from katmagic; resolves issue 4400. + - Expire old or over-used hidden service introduction points. + Required by fix for bug 3460. + - Move the replay-detection cache for the RSA-encrypted parts of + INTRODUCE2 cells to the introduction point data structures. + Previously, we would use one replay-detection cache per hidden + service. Required by fix for bug 3460. + - Reduce the lifetime of elements of hidden services' Diffie-Hellman + public key replay-detection cache from 60 minutes to 5 minutes. This + replay-detection cache is now used only to detect multiple + INTRODUCE2 cells specifying the same rendezvous point, so we can + avoid launching multiple simultaneous attempts to connect to it. + + o Minor bugfixes (on Tor 0.2.2.x and earlier): + - Resolve an integer overflow bug in smartlist_ensure_capacity(). + Fixes bug 4230; bugfix on Tor 0.1.0.1-rc. Based on a patch by + Mansour Moufid. + - Fix a minor formatting issue in one of tor-gencert's error messages. + Fixes bug 4574. + - Prevent a false positive from the check-spaces script, by disabling + the "whitespace between function name and (" check for functions + named 'op()'. + - Fix a log message suggesting that people contact a non-existent + email address. Fixes bug 3448. + - Fix null-pointer access that could occur if TLS allocation failed. + Fixes bug 4531; bugfix on 0.2.0.20-rc. Found by "troll_un". + - Report a real bootstrap problem to the controller on router + identity mismatch. Previously we just said "foo", which probably + made a lot of sense at the time. Fixes bug 4169; bugfix on + 0.2.1.1-alpha. + - If we had ever tried to call tor_addr_to_str() on an address of + unknown type, we would have done a strdup() on an uninitialized + buffer. Now we won't. Fixes bug 4529; bugfix on 0.2.1.3-alpha. + Reported by "troll_un". + - Correctly detect and handle transient lookup failures from + tor_addr_lookup(). Fixes bug 4530; bugfix on 0.2.1.5-alpha. + Reported by "troll_un". + - Use tor_socket_t type for listener argument to accept(). Fixes bug + 4535; bugfix on 0.2.2.28-beta. Found by "troll_un". + - Initialize conn->addr to a valid state in spawn_cpuworker(). Fixes + bug 4532; found by "troll_un". + + o Minor bugfixes (on Tor 0.2.3.x): + - Fix a compile warning in tor_inet_pton(). Bugfix on 0.2.3.8-alpha; + fixes bug 4554. + - Don't send two ESTABLISH_RENDEZVOUS cells when opening a new + circuit for use as a hidden service client's rendezvous point. + Fixes bugs 4641 and 4171; bugfix on 0.2.3.3-alpha. Diagnosed + with help from wanoskarnet. + - Restore behavior of overriding SocksPort, ORPort, and similar + options from the command line. Bugfix on 0.2.3.3-alpha. + + o Build fixes: + - Properly handle the case where the build-tree is not the same + as the source tree when generating src/common/common_sha1.i, + src/or/micro-revision.i, and src/or/or_sha1.i. Fixes bug 3953; + bugfix on 0.2.0.1-alpha. + + o Code simplifications, cleanups, and refactorings: + - Remove the pure attribute from all functions that used it + previously. In many cases we assigned it incorrectly, because the + functions might assert or call impure functions, and we don't have + evidence that keeping the pure attribute is worthwhile. Implements + changes suggested in ticket 4421. + - Remove some dead code spotted by coverity. Fixes cid 432. + Bugfix on 0.2.3.1-alpha, closes bug 4637. + + +Changes in version 0.2.3.8-alpha - 2011-11-22 + Tor 0.2.3.8-alpha fixes some crash and assert bugs, including a + socketpair-related bug that has been bothering Windows users. It adds + support to serve microdescriptors to controllers, so Vidalia's network + map can resume listing relays (once Vidalia implements its side), + and adds better support for hardware AES acceleration. Finally, it + starts the process of adjusting the bandwidth cutoff for getting the + "Fast" flag from 20KB to (currently) 32KB -- preliminary results show + that tiny relays harm performance more than they help network capacity. + + o Major bugfixes: + - Initialize Libevent with the EVENT_BASE_FLAG_NOLOCK flag enabled, so + that it doesn't attempt to allocate a socketpair. This could cause + some problems on Windows systems with overzealous firewalls. Fix for + bug 4457; workaround for Libevent versions 2.0.1-alpha through + 2.0.15-stable. + - Correctly sanity-check that we don't underflow on a memory + allocation (and then assert) for hidden service introduction + point decryption. Bug discovered by Dan Rosenberg. Fixes bug 4410; + bugfix on 0.2.1.5-alpha. + - Remove the artificially low cutoff of 20KB to guarantee the Fast + flag. In the past few years the average relay speed has picked + up, and while the "top 7/8 of the network get the Fast flag" and + "all relays with 20KB or more of capacity get the Fast flag" rules + used to have the same result, now the top 7/8 of the network has + a capacity more like 32KB. Bugfix on 0.2.1.14-rc. Fixes bug 4489. + - Fix a rare assertion failure when checking whether a v0 hidden + service descriptor has any usable introduction points left, and + we don't have enough information to build a circuit to the first + intro point named in the descriptor. The HS client code in + 0.2.3.x no longer uses v0 HS descriptors, but this assertion can + trigger on (and crash) v0 HS authorities. Fixes bug 4411. + Bugfix on 0.2.3.1-alpha; diagnosed by frosty_un. + - Make bridge authorities not crash when they are asked for their own + descriptor. Bugfix on 0.2.3.7-alpha, reported by Lucky Green. + - When running as a client, do not print a misleading (and plain + wrong) log message that we're collecting "directory request" + statistics: clients don't collect statistics. Also don't create a + useless (because empty) stats file in the stats/ directory. Fixes + bug 4353; bugfix on 0.2.2.34 and 0.2.3.7-alpha. + + o Major features: + - Allow Tor controllers like Vidalia to obtain the microdescriptor + for a relay by identity digest or nickname. Previously, + microdescriptors were only available by their own digests, so a + controller would have to ask for and parse the whole microdescriptor + consensus in order to look up a single relay's microdesc. Fixes + bug 3832; bugfix on 0.2.3.1-alpha. + - Use OpenSSL's EVP interface for AES encryption, so that all AES + operations can use hardware acceleration (if present). Resolves + ticket 4442. + + o Minor bugfixes (on 0.2.2.x and earlier): + - Detect failure to initialize Libevent. This fix provides better + detection for future instances of bug 4457. + - Avoid frequent calls to the fairly expensive cull_wedged_cpuworkers + function. This was eating up hideously large amounts of time on some + busy servers. Fixes bug 4518; bugfix on 0.0.9.8. + - Don't warn about unused log_mutex in log.c when building with + --disable-threads using a recent GCC. Fixes bug 4437; bugfix on + 0.1.0.6-rc which introduced --disable-threads. + - Allow manual 'authenticate' commands to the controller interface + from netcat (nc) as well as telnet. We were rejecting them because + they didn't come with the expected whitespace at the end of the + command. Bugfix on 0.1.1.1-alpha; fixes bug 2893. + - Fix some (not actually triggerable) buffer size checks in usage of + tor_inet_ntop. Fixes bug 4434; bugfix on Tor 0.2.0.1-alpha. Patch + by Anders Sundman. + - Fix parsing of some corner-cases with tor_inet_pton(). Fixes + bug 4515; bugfix on 0.2.0.1-alpha; fix by Anders Sundman. + - When configuring, starting, or stopping an NT service, stop + immediately after the service configuration attempt has succeeded + or failed. Fixes bug 3963; bugfix on 0.2.0.7-alpha. + - When sending a NETINFO cell, include the original address + received for the other side, not its canonical address. Found + by "troll_un"; fixes bug 4349; bugfix on 0.2.0.10-alpha. + - Rename the bench_{aes,dmap} functions to test_*, so that tinytest + can pick them up when the tests aren't disabled. Bugfix on + 0.2.2.4-alpha which introduced tinytest. + - Fix a memory leak when we check whether a hidden service + descriptor has any usable introduction points left. Fixes bug + 4424. Bugfix on 0.2.2.25-alpha. + - Fix a memory leak in launch_direct_bridge_descriptor_fetch() that + occurred when a client tried to fetch a descriptor for a bridge + in ExcludeNodes. Fixes bug 4383; bugfix on 0.2.2.25-alpha. + + o Minor bugfixes (on 0.2.3.x): + - Make util unit tests build correctly with MSVC. Bugfix on + 0.2.3.3-alpha. Patch by Gisle Vanem. + - Successfully detect AUTH_CHALLENGE cells with no recognized + authentication type listed. Fixes bug 4367; bugfix on 0.2.3.6-alpha. + Found by frosty_un. + - If a relay receives an AUTH_CHALLENGE cell it can't answer, + it should still send a NETINFO cell to allow the connection to + become open. Fixes bug 4368; fix on 0.2.3.6-alpha; bug found by + "frosty". + - Log less loudly when we get an invalid authentication certificate + from a source other than a directory authority: it's not unusual + to see invalid certs because of clock skew. Fixes bug 4370; bugfix + on 0.2.3.6-alpha. + - Tolerate servers with more clock skew in their authentication + certificates than previously. Fixes bug 4371; bugfix on + 0.2.3.6-alpha. + - Fix a couple of compile warnings on Windows. Fixes bug 4469; bugfix + on 0.2.3.4-alpha and 0.2.3.6-alpha. + + o Minor features: + - Add two new config options for directory authorities: + AuthDirFastGuarantee sets a bandwidth threshold for guaranteeing the + Fast flag, and AuthDirGuardBWGuarantee sets a bandwidth threshold + that is always sufficient to satisfy the bandwidth requirement for + the Guard flag. Now it will be easier for researchers to simulate + Tor networks with different values. Resolves ticket 4484. + - When Tor ignores a hidden service specified in its configuration, + include the hidden service's directory in the warning message. + Previously, we would only tell the user that some hidden service + was ignored. Bugfix on 0.0.6; fixes bug 4426. + - When we fail to initialize Libevent, retry with IOCP disabled so we + don't need to turn on multi-threading support in Libevent, which in + turn requires a working socketpair(). This is a workaround for bug + 4457, which affects Libevent versions from 2.0.1-alpha through + 2.0.15-stable. + - Detect when we try to build on a platform that doesn't define + AF_UNSPEC to 0. We don't work there, so refuse to compile. + - Update to the November 1 2011 Maxmind GeoLite Country database. + + o Packaging changes: + - Make it easier to automate expert package builds on Windows, + by removing an absolute path from makensis.exe command. + + o Code simplifications and refactoring: + - Remove some redundant #include directives throughout the code. + Patch from Andrea Gelmini. + - Unconditionally use OpenSSL's AES implementation instead of our + old built-in one. OpenSSL's AES has been better for a while, and + relatively few servers should still be on any version of OpenSSL + that doesn't have good optimized assembly AES. + - Use the name "CERTS" consistently to refer to the new cell type; + we were calling it CERT in some places and CERTS in others. + + o Testing: + - Numerous new unit tests for functions in util.c and address.c by + Anders Sundman. + - The long-disabled benchmark tests are now split into their own + ./src/test/bench binary. + - The benchmark tests can now use more accurate timers than + gettimeofday() when such timers are available. + + +Changes in version 0.2.3.7-alpha - 2011-10-30 + Tor 0.2.3.7-alpha fixes a crash bug in 0.2.3.6-alpha introduced by + the new v3 handshake. It also resolves yet another bridge address + enumeration issue. + + o Major bugfixes: + - If we mark an OR connection for close based on a cell we process, + don't process any further cells on it. We already avoid further + reads on marked-for-close connections, but now we also discard the + cells we'd already read. Fixes bug 4299; bugfix on 0.2.0.10-alpha, + which was the first version where we might mark a connection for + close based on processing a cell on it. + - Fix a double-free bug that would occur when we received an invalid + certificate in a CERT cell in the new v3 handshake. Fixes bug 4343; + bugfix on 0.2.3.6-alpha. + - Bridges no longer include their address in NETINFO cells on outgoing + OR connections, to allow them to blend in better with clients. + Removes another avenue for enumerating bridges. Reported by + "troll_un". Fixes bug 4348; bugfix on 0.2.0.10-alpha, when NETINFO + cells were introduced. + + o Trivial fixes: + - Fixed a typo in a hibernation-related log message. Fixes bug 4331; + bugfix on 0.2.2.23-alpha; found by "tmpname0901". + + +Changes in version 0.2.3.6-alpha - 2011-10-26 + Tor 0.2.3.6-alpha includes the fix from 0.2.2.34 for a critical + anonymity vulnerability where an attacker can deanonymize Tor + users. Everybody should upgrade. + + This release also features support for a new v3 connection handshake + protocol, and fixes to make hidden service connections more robust. + + o Major features: + - Implement a new handshake protocol (v3) for authenticating Tors to + each other over TLS. It should be more resistant to fingerprinting + than previous protocols, and should require less TLS hacking for + future Tor implementations. Implements proposal 176. + - Allow variable-length padding cells to disguise the length of + Tor's TLS records. Implements part of proposal 184. + + o Privacy/anonymity fixes (clients): + - Clients and bridges no longer send TLS certificate chains on + outgoing OR connections. Previously, each client or bridge would + use the same cert chain for all outgoing OR connections until + its IP address changes, which allowed any relay that the client + or bridge contacted to determine which entry guards it is using. + Fixes CVE-2011-2768. Bugfix on 0.0.9pre5; found by "frosty_un". + - If a relay receives a CREATE_FAST cell on a TLS connection, it + no longer considers that connection as suitable for satisfying a + circuit EXTEND request. Now relays can protect clients from the + CVE-2011-2768 issue even if the clients haven't upgraded yet. + - Directory authorities no longer assign the Guard flag to relays + that haven't upgraded to the above "refuse EXTEND requests + to client connections" fix. Now directory authorities can + protect clients from the CVE-2011-2768 issue even if neither + the clients nor the relays have upgraded yet. There's a new + "GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays" config option + to let us transition smoothly, else tomorrow there would be no + guard relays. + + o Major bugfixes (hidden services): + - Improve hidden service robustness: when an attempt to connect to + a hidden service ends, be willing to refetch its hidden service + descriptors from each of the HSDir relays responsible for them + immediately. Previously, we would not consider refetching the + service's descriptors from each HSDir for 15 minutes after the last + fetch, which was inconvenient if the hidden service was not running + during the first attempt. Bugfix on 0.2.0.18-alpha; fixes bug 3335. + - When one of a hidden service's introduction points appears to be + unreachable, stop trying it. Previously, we would keep trying + to build circuits to the introduction point until we lost the + descriptor, usually because the user gave up and restarted Tor. + Partly fixes bug 3825. + - Don't launch a useless circuit after failing to use one of a + hidden service's introduction points. Previously, we would + launch a new introduction circuit, but not set the hidden service + which that circuit was intended to connect to, so it would never + actually be used. A different piece of code would then create a + new introduction circuit correctly. Bug reported by katmagic and + found by Sebastian Hahn. Bugfix on 0.2.1.13-alpha; fixes bug 4212. + + o Major bugfixes (other): + - Bridges now refuse CREATE or CREATE_FAST cells on OR connections + that they initiated. Relays could distinguish incoming bridge + connections from client connections, creating another avenue for + enumerating bridges. Fixes CVE-2011-2769. Bugfix on 0.2.0.3-alpha. + Found by "frosty_un". + - Don't update the AccountingSoftLimitHitAt state file entry whenever + tor gets started. This prevents a wrong average bandwidth + estimate, which would cause relays to always start a new accounting + interval at the earliest possible moment. Fixes bug 2003; bugfix + on 0.2.2.7-alpha. Reported by BryonEldridge, who also helped + immensely in tracking this bug down. + - Fix a crash bug when changing node restrictions while a DNS lookup + is in-progress. Fixes bug 4259; bugfix on 0.2.2.25-alpha. Bugfix + by "Tey'". + + o Minor bugfixes (on 0.2.2.x and earlier): + - When a hidden service turns an extra service-side introduction + circuit into a general-purpose circuit, free the rend_data and + intro_key fields first, so we won't leak memory if the circuit + is cannibalized for use as another service-side introduction + circuit. Bugfix on 0.2.1.7-alpha; fixes bug 4251. + - Rephrase the log message emitted if the TestSocks check is + successful. Patch from Fabian Keil; fixes bug 4094. + - Bridges now skip DNS self-tests, to act a little more stealthily. + Fixes bug 4201; bugfix on 0.2.0.3-alpha, which first introduced + bridges. Patch by "warms0x". + - Remove a confusing dollar sign from the example fingerprint in the + man page, and also make the example fingerprint a valid one. Fixes + bug 4309; bugfix on 0.2.1.3-alpha. + - Fix internal bug-checking logic that was supposed to catch + failures in digest generation so that it will fail more robustly + if we ask for a nonexistent algorithm. Found by Coverity Scan. + Bugfix on 0.2.2.1-alpha; fixes Coverity CID 479. + - Report any failure in init_keys() calls launched because our + IP address has changed. Spotted by Coverity Scan. Bugfix on + 0.1.1.4-alpha; fixes CID 484. + + o Minor bugfixes (on 0.2.3.x): + - Fix a bug in configure.in that kept it from building a configure + script with autoconf versions earlier than 2.61. Fixes bug 2430; + bugfix on 0.2.3.1-alpha. + - Don't warn users that they are exposing a client port to the + Internet if they have specified an RFC1918 address. Previously, + we would warn if the user had specified any non-loopback + address. Bugfix on 0.2.3.3-alpha. Fixes bug 4018; reported by Tas. + - Fix memory leaks in the failing cases of the new SocksPort and + ControlPort code. Found by Coverity Scan. Bugfix on 0.2.3.3-alpha; + fixes coverity CIDs 485, 486, and 487. + + o Minor features: + - When a hidden service's introduction point times out, consider + trying it again during the next attempt to connect to the + HS. Previously, we would not try it again unless a newly fetched + descriptor contained it. Required by fixes for bugs 1297 and 3825. + - The next version of Windows will be called Windows 8, and it has + a major version of 6, minor version of 2. Correctly identify that + version instead of calling it "Very recent version". Resolves + ticket 4153; reported by funkstar. + - The Bridge Authority now writes statistics on how many bridge + descriptors it gave out in total, and how many unique descriptors + it gave out. It also lists how often the most and least commonly + fetched descriptors were given out, as well as the median and + 25th/75th percentile. Implements tickets 4200 and 4294. + - Update to the October 4 2011 Maxmind GeoLite Country database. + + o Code simplifications and refactoring: + - Remove some old code to remember statistics about which descriptors + we've served as a directory mirror. The feature wasn't used and + is outdated now that microdescriptors are around. + - Rename Tor functions that turn strings into addresses, so that + "parse" indicates that no hostname resolution occurs, and + "lookup" indicates that hostname resolution may occur. This + should help prevent mistakes in the future. Fixes bug 3512. + + +Changes in version 0.2.2.34 - 2011-10-26 + Tor 0.2.2.34 fixes a critical anonymity vulnerability where an attacker + can deanonymize Tor users. Everybody should upgrade. + + The attack relies on four components: 1) Clients reuse their TLS cert + when talking to different relays, so relays can recognize a user by + the identity key in her cert. 2) An attacker who knows the client's + identity key can probe each guard relay to see if that identity key + is connected to that guard relay right now. 3) A variety of active + attacks in the literature (starting from "Low-Cost Traffic Analysis + of Tor" by Murdoch and Danezis in 2005) allow a malicious website to + discover the guard relays that a Tor user visiting the website is using. + 4) Clients typically pick three guards at random, so the set of guards + for a given user could well be a unique fingerprint for her. This + release fixes components #1 and #2, which is enough to block the attack; + the other two remain as open research problems. Special thanks to + "frosty_un" for reporting the issue to us! + + Clients should upgrade so they are no longer recognizable by the TLS + certs they present. Relays should upgrade so they no longer allow a + remote attacker to probe them to test whether unpatched clients are + currently connected to them. + + This release also fixes several vulnerabilities that allow an attacker + to enumerate bridge relays. Some bridge enumeration attacks still + remain; see for example proposal 188. + + o Privacy/anonymity fixes (clients): + - Clients and bridges no longer send TLS certificate chains on + outgoing OR connections. Previously, each client or bridge would + use the same cert chain for all outgoing OR connections until + its IP address changes, which allowed any relay that the client + or bridge contacted to determine which entry guards it is using. + Fixes CVE-2011-2768. Bugfix on 0.0.9pre5; found by "frosty_un". + - If a relay receives a CREATE_FAST cell on a TLS connection, it + no longer considers that connection as suitable for satisfying a + circuit EXTEND request. Now relays can protect clients from the + CVE-2011-2768 issue even if the clients haven't upgraded yet. + - Directory authorities no longer assign the Guard flag to relays + that haven't upgraded to the above "refuse EXTEND requests + to client connections" fix. Now directory authorities can + protect clients from the CVE-2011-2768 issue even if neither + the clients nor the relays have upgraded yet. There's a new + "GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays" config option + to let us transition smoothly, else tomorrow there would be no + guard relays. + + o Privacy/anonymity fixes (bridge enumeration): + - Bridge relays now do their directory fetches inside Tor TLS + connections, like all the other clients do, rather than connecting + directly to the DirPort like public relays do. Removes another + avenue for enumerating bridges. Fixes bug 4115; bugfix on 0.2.0.35. + - Bridges relays now build circuits for themselves in a more similar + way to how clients build them. Removes another avenue for + enumerating bridges. Fixes bug 4124; bugfix on 0.2.0.3-alpha, + when bridges were introduced. + - Bridges now refuse CREATE or CREATE_FAST cells on OR connections + that they initiated. Relays could distinguish incoming bridge + connections from client connections, creating another avenue for + enumerating bridges. Fixes CVE-2011-2769. Bugfix on 0.2.0.3-alpha. + Found by "frosty_un". + + o Major bugfixes: + - Fix a crash bug when changing node restrictions while a DNS lookup + is in-progress. Fixes bug 4259; bugfix on 0.2.2.25-alpha. Bugfix + by "Tey'". + - Don't launch a useless circuit after failing to use one of a + hidden service's introduction points. Previously, we would + launch a new introduction circuit, but not set the hidden service + which that circuit was intended to connect to, so it would never + actually be used. A different piece of code would then create a + new introduction circuit correctly. Bug reported by katmagic and + found by Sebastian Hahn. Bugfix on 0.2.1.13-alpha; fixes bug 4212. + + o Minor bugfixes: + - Change an integer overflow check in the OpenBSD_Malloc code so + that GCC is less likely to eliminate it as impossible. Patch + from Mansour Moufid. Fixes bug 4059. + - When a hidden service turns an extra service-side introduction + circuit into a general-purpose circuit, free the rend_data and + intro_key fields first, so we won't leak memory if the circuit + is cannibalized for use as another service-side introduction + circuit. Bugfix on 0.2.1.7-alpha; fixes bug 4251. + - Bridges now skip DNS self-tests, to act a little more stealthily. + Fixes bug 4201; bugfix on 0.2.0.3-alpha, which first introduced + bridges. Patch by "warms0x". + - Fix internal bug-checking logic that was supposed to catch + failures in digest generation so that it will fail more robustly + if we ask for a nonexistent algorithm. Found by Coverity Scan. + Bugfix on 0.2.2.1-alpha; fixes Coverity CID 479. + - Report any failure in init_keys() calls launched because our + IP address has changed. Spotted by Coverity Scan. Bugfix on + 0.1.1.4-alpha; fixes CID 484. + + o Minor bugfixes (log messages and documentation): + - Remove a confusing dollar sign from the example fingerprint in the + man page, and also make the example fingerprint a valid one. Fixes + bug 4309; bugfix on 0.2.1.3-alpha. + - The next version of Windows will be called Windows 8, and it has + a major version of 6, minor version of 2. Correctly identify that + version instead of calling it "Very recent version". Resolves + ticket 4153; reported by funkstar. + - Downgrade log messages about circuit timeout calibration from + "notice" to "info": they don't require or suggest any human + intervention. Patch from Tom Lowenthal. Fixes bug 4063; + bugfix on 0.2.2.14-alpha. + + o Minor features: + - Turn on directory request statistics by default and include them in + extra-info descriptors. Don't break if we have no GeoIP database. + Backported from 0.2.3.1-alpha; implements ticket 3951. + - Update to the October 4 2011 Maxmind GeoLite Country database. + + +Changes in version 0.2.1.31 - 2011-10-26 + Tor 0.2.1.31 backports important security and privacy fixes for + oldstable. This release is intended only for package maintainers and + others who cannot use the 0.2.2 stable series. All others should be + using Tor 0.2.2.x or newer. + + o Security fixes (also included in 0.2.2.x): + - Replace all potentially sensitive memory comparison operations + with versions whose runtime does not depend on the data being + compared. This will help resist a class of attacks where an + adversary can use variations in timing information to learn + sensitive data. Fix for one case of bug 3122. (Safe memcmp + implementation by Robert Ransom based partially on code by DJB.) + - Fix an assert in parsing router descriptors containing IPv6 + addresses. This one took down the directory authorities when + somebody tried some experimental code. Bugfix on 0.2.1.3-alpha. + + o Privacy/anonymity fixes (also included in 0.2.2.x): + - Clients and bridges no longer send TLS certificate chains on + outgoing OR connections. Previously, each client or bridge would + use the same cert chain for all outgoing OR connections until + its IP address changes, which allowed any relay that the client + or bridge contacted to determine which entry guards it is using. + Fixes CVE-2011-2768. Bugfix on 0.0.9pre5; found by "frosty_un". + - If a relay receives a CREATE_FAST cell on a TLS connection, it + no longer considers that connection as suitable for satisfying a + circuit EXTEND request. Now relays can protect clients from the + CVE-2011-2768 issue even if the clients haven't upgraded yet. + - Bridges now refuse CREATE or CREATE_FAST cells on OR connections + that they initiated. Relays could distinguish incoming bridge + connections from client connections, creating another avenue for + enumerating bridges. Fixes CVE-2011-2769. Bugfix on 0.2.0.3-alpha. + Found by "frosty_un". + - When receiving a hidden service descriptor, check that it is for + the hidden service we wanted. Previously, Tor would store any + hidden service descriptors that a directory gave it, whether it + wanted them or not. This wouldn't have let an attacker impersonate + a hidden service, but it did let directories pre-seed a client + with descriptors that it didn't want. Bugfix on 0.0.6. + - Avoid linkability based on cached hidden service descriptors: forget + all hidden service descriptors cached as a client when processing a + SIGNAL NEWNYM command. Fixes bug 3000; bugfix on 0.0.6. + - Make the bridge directory authority refuse to answer directory + requests for "all" descriptors. It used to include bridge + descriptors in its answer, which was a major information leak. + Found by "piebeer". Bugfix on 0.2.0.3-alpha. + - Don't attach new streams to old rendezvous circuits after SIGNAL + NEWNYM. Previously, we would keep using an existing rendezvous + circuit if it remained open (i.e. if it were kept open by a + long-lived stream, or if a new stream were attached to it before + Tor could notice that it was old and no longer in use). Bugfix on + 0.1.1.15-rc; fixes bug 3375. + + o Minor bugfixes (also included in 0.2.2.x): + - When we restart our relay, we might get a successful connection + from the outside before we've started our reachability tests, + triggering a warning: "ORPort found reachable, but I have no + routerinfo yet. Failing to inform controller of success." This + bug was harmless unless Tor is running under a controller + like Vidalia, in which case the controller would never get a + REACHABILITY_SUCCEEDED status event. Bugfix on 0.1.2.6-alpha; + fixes bug 1172. + - Build correctly on OSX with zlib 1.2.4 and higher with all warnings + enabled. Fixes bug 1526. + - Remove undocumented option "-F" from tor-resolve: it hasn't done + anything since 0.2.1.16-rc. + - Avoid signed/unsigned comparisons by making SIZE_T_CEILING unsigned. + None of the cases where we did this before were wrong, but by making + this change we avoid warnings. Fixes bug 2475; bugfix on 0.2.1.28. + - Fix a rare crash bug that could occur when a client was configured + with a large number of bridges. Fixes bug 2629; bugfix on + 0.2.1.2-alpha. Bugfix by trac user "shitlei". + - Correct the warning displayed when a rendezvous descriptor exceeds + the maximum size. Fixes bug 2750; bugfix on 0.2.1.5-alpha. Found by + John Brooks. + - Fix an uncommon assertion failure when running with DNSPort under + heavy load. Fixes bug 2933; bugfix on 0.2.0.1-alpha. + - When warning about missing zlib development packages during compile, + give the correct package names. Bugfix on 0.2.0.1-alpha. + - Require that introduction point keys and onion keys have public + exponent 65537. Bugfix on 0.2.0.10-alpha. + - Do not crash when our configuration file becomes unreadable, for + example due to a permissions change, between when we start up + and when a controller calls SAVECONF. Fixes bug 3135; bugfix + on 0.0.9pre6. + - Fix warnings from GCC 4.6's "-Wunused-but-set-variable" option. + Fixes bug 3208. + - Always NUL-terminate the sun_path field of a sockaddr_un before + passing it to the kernel. (Not a security issue: kernels are + smart enough to reject bad sockaddr_uns.) Found by Coverity; + CID #428. Bugfix on Tor 0.2.0.3-alpha. + - Don't stack-allocate the list of supplementary GIDs when we're + about to log them. Stack-allocating NGROUPS_MAX gid_t elements + could take up to 256K, which is way too much stack. Found by + Coverity; CID #450. Bugfix on 0.2.1.7-alpha. + + o Minor bugfixes (only in 0.2.1.x): + - Resume using micro-version numbers in 0.2.1.x: our Debian packages + rely on them. Bugfix on 0.2.1.30. + - Use git revisions instead of svn revisions when generating our + micro-version numbers. Bugfix on 0.2.1.15-rc; fixes bug 2402. + + o Minor features (also included in 0.2.2.x): + - Adjust the expiration time on our SSL session certificates to + better match SSL certs seen in the wild. Resolves ticket 4014. + - Allow nameservers with IPv6 address. Resolves bug 2574. + - Update to the October 4 2011 Maxmind GeoLite Country database. + + +Changes in version 0.2.3.5-alpha - 2011-09-28 + Tor 0.2.3.5-alpha fixes two bugs that make it possible to enumerate + bridge relays; fixes an assertion error that many users started hitting + today; and adds the ability to refill token buckets more often than + once per second, allowing significant performance improvements. + + o Security fixes: + - Bridge relays now do their directory fetches inside Tor TLS + connections, like all the other clients do, rather than connecting + directly to the DirPort like public relays do. Removes another + avenue for enumerating bridges. Fixes bug 4115; bugfix on 0.2.0.35. + - Bridges relays now build circuits for themselves in a more similar + way to how clients build them. Removes another avenue for + enumerating bridges. Fixes bug 4124; bugfix on 0.2.0.3-alpha, + when bridges were introduced. + + o Major bugfixes: + - Fix an "Assertion md->held_by_node == 1 failed" error that could + occur when the same microdescriptor was referenced by two node_t + objects at once. Fix for bug 4118; bugfix on Tor 0.2.3.1-alpha. + + o Major features (networking): + - Add a new TokenBucketRefillInterval option to refill token buckets + more frequently than once per second. This should improve network + performance, alleviate queueing problems, and make traffic less + bursty. Implements proposal 183; closes ticket 3630. Design by + Florian Tschorsch and Björn Scheuermann; implementation by + Florian Tschorsch. + + o Minor bugfixes: + - Change an integer overflow check in the OpenBSD_Malloc code so + that GCC is less likely to eliminate it as impossible. Patch + from Mansour Moufid. Fixes bug 4059. + + o Minor bugfixes (usability): + - Downgrade log messages about circuit timeout calibration from + "notice" to "info": they don't require or suggest any human + intervention. Patch from Tom Lowenthal. Fixes bug 4063; + bugfix on 0.2.2.14-alpha. + + o Minor features (diagnostics): + - When the system call to create a listener socket fails, log the + error message explaining why. This may help diagnose bug 4027. + + +Changes in version 0.2.3.4-alpha - 2011-09-13 + Tor 0.2.3.4-alpha includes the fixes from 0.2.2.33, including a slight + tweak to Tor's TLS handshake that makes relays and bridges that run + this new version reachable from Iran again. It also fixes a few new + bugs in 0.2.3.x, and teaches relays to recognize when they're not + listed in the network consensus and republish. + + o Major bugfixes (also part of 0.2.2.33): + - Avoid an assertion failure when reloading a configuration with + TrackExitHosts changes. Found and fixed by 'laruldan'. Fixes bug + 3923; bugfix on 0.2.2.25-alpha. + + o Minor features (security, also part of 0.2.2.33): + - Check for replays of the public-key encrypted portion of an + INTRODUCE1 cell, in addition to the current check for replays of + the g^x value. This prevents a possible class of active attacks + by an attacker who controls both an introduction point and a + rendezvous point, and who uses the malleability of AES-CTR to + alter the encrypted g^x portion of the INTRODUCE1 cell. We think + that these attacks is infeasible (requiring the attacker to send + on the order of zettabytes of altered cells in a short interval), + but we'd rather block them off in case there are any classes of + this attack that we missed. Reported by Willem Pinckaers. + + o Minor features (also part of 0.2.2.33): + - Adjust the expiration time on our SSL session certificates to + better match SSL certs seen in the wild. Resolves ticket 4014. + - Change the default required uptime for a relay to be accepted as + a HSDir (hidden service directory) from 24 hours to 25 hours. + Improves on 0.2.0.10-alpha; resolves ticket 2649. + - Add a VoteOnHidServDirectoriesV2 config option to allow directory + authorities to abstain from voting on assignment of the HSDir + consensus flag. Related to bug 2649. + - Update to the September 6 2011 Maxmind GeoLite Country database. + + o Minor bugfixes (also part of 0.2.2.33): + - Demote the 'replay detected' log message emitted when a hidden + service receives the same Diffie-Hellman public key in two different + INTRODUCE2 cells to info level. A normal Tor client can cause that + log message during its normal operation. Bugfix on 0.2.1.6-alpha; + fixes part of bug 2442. + - Demote the 'INTRODUCE2 cell is too {old,new}' log message to info + level. There is nothing that a hidden service's operator can do + to fix its clients' clocks. Bugfix on 0.2.1.6-alpha; fixes part + of bug 2442. + - Clarify a log message specifying the characters permitted in + HiddenServiceAuthorizeClient client names. Previously, the log + message said that "[A-Za-z0-9+-_]" were permitted; that could have + given the impression that every ASCII character between "+" and "_" + was permitted. Now we say "[A-Za-z0-9+_-]". Bugfix on 0.2.1.5-alpha. + + o Build fixes (also part of 0.2.2.33): + - Clean up some code issues that prevented Tor from building on older + BSDs. Fixes bug 3894; reported by "grarpamp". + - Search for a platform-specific version of "ar" when cross-compiling. + Should fix builds on iOS. Resolves bug 3909, found by Marco Bonetti. + + o Major bugfixes: + - Fix a bug where the SocksPort option (for example) would get + ignored and replaced by the default if a SocksListenAddress + option was set. Bugfix on 0.2.3.3-alpha; fixes bug 3936. Fix by + Fabian Keil. + + o Major features: + - Relays now try regenerating and uploading their descriptor more + frequently if they are not listed in the consensus, or if the + version of their descriptor listed in the consensus is too + old. This fix should prevent situations where a server declines + to re-publish itself because it has done so too recently, even + though the authorities decided not to list its recent-enough + descriptor. Fix for bug 3327. + + o Minor features: + - Relays now include a reason for regenerating their descriptors + in an HTTP header when uploading to the authorities. This will + make it easier to debug descriptor-upload issues in the future. + - When starting as root and then changing our UID via the User + control option, and we have a ControlSocket configured, make sure + that the ControlSocket is owned by the same account that Tor will + run under. Implements ticket 3421; fix by Jérémy Bobbio. + + o Minor bugfixes: + - Abort if tor_vasprintf fails in connection_printf_to_buf (a + utility function used in the control-port code). This shouldn't + ever happen unless Tor is completely out of memory, but if it did + happen and Tor somehow recovered from it, Tor could have sent a log + message to a control port in the middle of a reply to a controller + command. Fixes part of bug 3428; bugfix on 0.1.2.3-alpha. + - Make 'FetchUselessDescriptors' cause all descriptor types and + all consensus types (including microdescriptors) to get fetched. + Fixes bug 3851; bugfix on 0.2.3.1-alpha. + + o Code refactoring: + - Make a new "entry connection" struct as an internal subtype of "edge + connection", to simplify the code and make exit connections smaller. + + +Changes in version 0.2.2.33 - 2011-09-13 + Tor 0.2.2.33 fixes several bugs, and includes a slight tweak to Tor's + TLS handshake that makes relays and bridges that run this new version + reachable from Iran again. + + o Major bugfixes: + - Avoid an assertion failure when reloading a configuration with + TrackExitHosts changes. Found and fixed by 'laruldan'. Fixes bug + 3923; bugfix on 0.2.2.25-alpha. + + o Minor features (security): + - Check for replays of the public-key encrypted portion of an + INTRODUCE1 cell, in addition to the current check for replays of + the g^x value. This prevents a possible class of active attacks + by an attacker who controls both an introduction point and a + rendezvous point, and who uses the malleability of AES-CTR to + alter the encrypted g^x portion of the INTRODUCE1 cell. We think + that these attacks are infeasible (requiring the attacker to send + on the order of zettabytes of altered cells in a short interval), + but we'd rather block them off in case there are any classes of + this attack that we missed. Reported by Willem Pinckaers. + + o Minor features: + - Adjust the expiration time on our SSL session certificates to + better match SSL certs seen in the wild. Resolves ticket 4014. + - Change the default required uptime for a relay to be accepted as + a HSDir (hidden service directory) from 24 hours to 25 hours. + Improves on 0.2.0.10-alpha; resolves ticket 2649. + - Add a VoteOnHidServDirectoriesV2 config option to allow directory + authorities to abstain from voting on assignment of the HSDir + consensus flag. Related to bug 2649. + - Update to the September 6 2011 Maxmind GeoLite Country database. + + o Minor bugfixes (documentation and log messages): + - Correct the man page to explain that HashedControlPassword and + CookieAuthentication can both be set, in which case either method + is sufficient to authenticate to Tor. Bugfix on 0.2.0.7-alpha, + when we decided to allow these config options to both be set. Issue + raised by bug 3898. + - Demote the 'replay detected' log message emitted when a hidden + service receives the same Diffie-Hellman public key in two different + INTRODUCE2 cells to info level. A normal Tor client can cause that + log message during its normal operation. Bugfix on 0.2.1.6-alpha; + fixes part of bug 2442. + - Demote the 'INTRODUCE2 cell is too {old,new}' log message to info + level. There is nothing that a hidden service's operator can do + to fix its clients' clocks. Bugfix on 0.2.1.6-alpha; fixes part + of bug 2442. + - Clarify a log message specifying the characters permitted in + HiddenServiceAuthorizeClient client names. Previously, the log + message said that "[A-Za-z0-9+-_]" were permitted; that could have + given the impression that every ASCII character between "+" and "_" + was permitted. Now we say "[A-Za-z0-9+_-]". Bugfix on 0.2.1.5-alpha. + + o Build fixes: + - Provide a substitute implementation of lround() for MSVC, which + apparently lacks it. Patch from Gisle Vanem. + - Clean up some code issues that prevented Tor from building on older + BSDs. Fixes bug 3894; reported by "grarpamp". + - Search for a platform-specific version of "ar" when cross-compiling. + Should fix builds on iOS. Resolves bug 3909, found by Marco Bonetti. + + +Changes in version 0.2.3.3-alpha - 2011-09-01 + Tor 0.2.3.3-alpha adds a new "stream isolation" feature to improve Tor's + security, and provides client-side support for the microdescriptor + and optimistic data features introduced earlier in the 0.2.3.x + series. It also includes numerous critical bugfixes in the (optional) + bufferevent-based networking backend. + + o Major features (stream isolation): + - You can now configure Tor so that streams from different + applications are isolated on different circuits, to prevent an + attacker who sees your streams as they leave an exit node from + linking your sessions to one another. To do this, choose some way + to distinguish the applications: have them connect to different + SocksPorts, or have one of them use SOCKS4 while the other uses + SOCKS5, or have them pass different authentication strings to the + SOCKS proxy. Then, use the new SocksPort syntax to configure the + degree of isolation you need. This implements Proposal 171. + - There's a new syntax for specifying multiple client ports (such as + SOCKSPort, TransPort, DNSPort, NATDPort): you can now just declare + multiple *Port entries with full addr:port syntax on each. + The old *ListenAddress format is still supported, but you can't + mix it with the new *Port syntax. + + o Major features (other): + - Enable microdescriptor fetching by default for clients. This allows + clients to download a much smaller amount of directory information. + To disable it (and go back to the old-style consensus and + descriptors), set "UseMicrodescriptors 0" in your torrc file. + - Tor's firewall-helper feature, introduced in 0.2.3.1-alpha (see the + "PortForwarding" config option), now supports Windows. + - When using an exit relay running 0.2.3.x, clients can now + "optimistically" send data before the exit relay reports that + the stream has opened. This saves a round trip when starting + connections where the client speaks first (such as web browsing). + This behavior is controlled by a consensus parameter (currently + disabled). To turn it on or off manually, use the "OptimisticData" + torrc option. Implements proposal 181; code by Ian Goldberg. + + o Major bugfixes (bufferevents, fixes on 0.2.3.1-alpha): + - When using IOCP on Windows, we need to enable Libevent windows + threading support. + - The IOCP backend now works even when the user has not specified + the (internal, debugging-only) _UseFilteringSSLBufferevents option. + Fixes part of bug 3752. + - Correctly record the bytes we've read and written when using + bufferevents, so that we can include them in our bandwidth history + and advertised bandwidth. Fixes bug 3803. + - Apply rate-limiting only at the bottom of a chain of filtering + bufferevents. This prevents us from filling up internal read + buffers and violating rate-limits when filtering bufferevents + are enabled. Fixes part of bug 3804. + - Add high-watermarks to the output buffers for filtered + bufferevents. This prevents us from filling up internal write + buffers and wasting CPU cycles when filtering bufferevents are + enabled. Fixes part of bug 3804. + - Correctly notice when data has been written from a bufferevent + without flushing it completely. Fixes bug 3805. + - Fix a bug where server-side tunneled bufferevent-based directory + streams would get closed prematurely. Fixes bug 3814. + - Fix a use-after-free error with per-connection rate-limiting + buckets. Fixes bug 3888. + + o Major bugfixes (also part of 0.2.2.31-rc): + - If we're configured to write our ControlPorts to disk, only write + them after switching UID and creating the data directory. This way, + we don't fail when starting up with a nonexistent DataDirectory + and a ControlPortWriteToFile setting based on that directory. Fixes + bug 3747; bugfix on Tor 0.2.2.26-beta. + + o Minor features: + - Added a new CONF_CHANGED event so that controllers can be notified + of any configuration changes made by other controllers, or by the + user. Implements ticket 1692. + - Use evbuffer_copyout() in inspect_evbuffer(). This fixes a memory + leak when using bufferevents, and lets Libevent worry about how to + best copy data out of a buffer. + - Replace files in stats/ rather than appending to them. Now that we + include statistics in extra-info descriptors, it makes no sense to + keep old statistics forever. Implements ticket 2930. + + o Minor features (build compatibility): + - Limited, experimental support for building with nmake and MSVC. + - Provide a substitute implementation of lround() for MSVC, which + apparently lacks it. Patch from Gisle Vanem. + + o Minor features (also part of 0.2.2.31-rc): + - Update to the August 2 2011 Maxmind GeoLite Country database. + + o Minor bugfixes (on 0.2.3.x-alpha): + - Fix a spurious warning when parsing SOCKS requests with + bufferevents enabled. Fixes bug 3615; bugfix on 0.2.3.2-alpha. + - Get rid of a harmless warning that could happen on relays running + with bufferevents. The warning was caused by someone doing an http + request to a relay's orport. Also don't warn for a few related + non-errors. Fixes bug 3700; bugfix on 0.2.3.1-alpha. + + o Minor bugfixes (on 2.2.x and earlier): + - Correct the man page to explain that HashedControlPassword and + CookieAuthentication can both be set, in which case either method + is sufficient to authenticate to Tor. Bugfix on 0.2.0.7-alpha, + when we decided to allow these config options to both be set. Issue + raised by bug 3898. + - The "--quiet" and "--hush" options now apply not only to Tor's + behavior before logs are configured, but also to Tor's behavior in + the absense of configured logs. Fixes bug 3550; bugfix on + 0.2.0.10-alpha. + + o Minor bugfixes (also part of 0.2.2.31-rc): + - Write several files in text mode, on OSes that distinguish text + mode from binary mode (namely, Windows). These files are: + 'buffer-stats', 'dirreq-stats', and 'entry-stats' on relays + that collect those statistics; 'client_keys' and 'hostname' for + hidden services that use authentication; and (in the tor-gencert + utility) newly generated identity and signing keys. Previously, + we wouldn't specify text mode or binary mode, leading to an + assertion failure. Fixes bug 3607. Bugfix on 0.2.1.1-alpha (when + the DirRecordUsageByCountry option which would have triggered + the assertion failure was added), although this assertion failure + would have occurred in tor-gencert on Windows in 0.2.0.1-alpha. + - Selectively disable deprecation warnings on OS X because Lion + started deprecating the shipped copy of openssl. Fixes bug 3643. + - Remove an extra pair of quotation marks around the error + message in control-port STATUS_GENERAL BUG events. Bugfix on + 0.1.2.6-alpha; fixes bug 3732. + - When unable to format an address as a string, report its value + as "???" rather than reusing the last formatted address. Bugfix + on 0.2.1.5-alpha. + + o Code simplifications and refactoring: + - Rewrite the listener-selection logic so that parsing which ports + we want to listen on is now separate from binding to the ports + we want. + + o Build changes: + - Building Tor with bufferevent support now requires Libevent + 2.0.13-stable or later. Previous versions of Libevent had bugs in + SSL-related bufferevents and related issues that would make Tor + work badly with bufferevents. Requiring 2.0.13-stable also allows + Tor with bufferevents to take advantage of Libevent APIs + introduced after 2.0.8-rc. + + +Changes in version 0.2.2.32 - 2011-08-27 + The Tor 0.2.2 release series is dedicated to the memory of Andreas + Pfitzmann (1958-2010), a pioneer in anonymity and privacy research, + a founder of the PETS community, a leader in our field, a mentor, + and a friend. He left us with these words: "I had the possibility + to contribute to this world that is not as it should be. I hope I + could help in some areas to make the world a better place, and that + I could also encourage other people to be engaged in improving the + world. Please, stay engaged. This world needs you, your love, your + initiative -- now I cannot be part of that anymore." + + Tor 0.2.2.32, the first stable release in the 0.2.2 branch, is finally + ready. More than two years in the making, this release features improved + client performance and hidden service reliability, better compatibility + for Android, correct behavior for bridges that listen on more than + one address, more extensible and flexible directory object handling, + better reporting of network statistics, improved code security, and + many many other features and bugfixes. + + +Changes in version 0.2.2.31-rc - 2011-08-17 + Tor 0.2.2.31-rc is the second and hopefully final release candidate + for the Tor 0.2.2.x series. + + o Major bugfixes: + - Remove an extra pair of quotation marks around the error + message in control-port STATUS_GENERAL BUG events. Bugfix on + 0.1.2.6-alpha; fixes bug 3732. + - If we're configured to write our ControlPorts to disk, only write + them after switching UID and creating the data directory. This way, + we don't fail when starting up with a nonexistent DataDirectory + and a ControlPortWriteToFile setting based on that directory. Fixes + bug 3747; bugfix on Tor 0.2.2.26-beta. + + o Minor features: + - Update to the August 2 2011 Maxmind GeoLite Country database. + + o Minor bugfixes: + - Allow GETINFO fingerprint to return a fingerprint even when + we have not yet built a router descriptor. Fixes bug 3577; + bugfix on 0.2.0.1-alpha. + - Write several files in text mode, on OSes that distinguish text + mode from binary mode (namely, Windows). These files are: + 'buffer-stats', 'dirreq-stats', and 'entry-stats' on relays + that collect those statistics; 'client_keys' and 'hostname' for + hidden services that use authentication; and (in the tor-gencert + utility) newly generated identity and signing keys. Previously, + we wouldn't specify text mode or binary mode, leading to an + assertion failure. Fixes bug 3607. Bugfix on 0.2.1.1-alpha (when + the DirRecordUsageByCountry option which would have triggered + the assertion failure was added), although this assertion failure + would have occurred in tor-gencert on Windows in 0.2.0.1-alpha. + - Selectively disable deprecation warnings on OS X because Lion + started deprecating the shipped copy of openssl. Fixes bug 3643. + - When unable to format an address as a string, report its value + as "???" rather than reusing the last formatted address. Bugfix + on 0.2.1.5-alpha. + + +Changes in version 0.2.3.2-alpha - 2011-07-18 + Tor 0.2.3.2-alpha introduces two new experimental features: + microdescriptors and pluggable transports. It also continues cleaning + up a variety of recently introduced features. + + o Major features: + - Clients can now use microdescriptors instead of regular descriptors + to build circuits. Microdescriptors are authority-generated + summaries of regular descriptors' contents, designed to change + very rarely (see proposal 158 for details). This feature is + designed to save bandwidth, especially for clients on slow internet + connections. It's off by default for now, since nearly no caches + support it, but it will be on-by-default for clients in a future + version. You can use the UseMicrodescriptors option to turn it on. + - Tor clients using bridges can now be configured to use a separate + 'transport' proxy for each bridge. This approach helps to resist + censorship by allowing bridges to use protocol obfuscation + plugins. It implements part of proposal 180. Implements ticket 2841. + - While we're trying to bootstrap, record how many TLS connections + fail in each state, and report which states saw the most failures + in response to any bootstrap failures. This feature may speed up + diagnosis of censorship events. Implements ticket 3116. + + o Major bugfixes (on 0.2.3.1-alpha): + - When configuring a large set of nodes in EntryNodes (as with + 'EntryNodes {cc}' or 'EntryNodes 1.1.1.1/16'), choose only a + random subset to be guards, and choose them in random + order. Fixes bug 2798. + - Tor could crash when remembering a consensus in a non-used consensus + flavor without having a current consensus set. Fixes bug 3361. + - Comparing an unknown address to a microdescriptor's shortened exit + policy would always give a "rejected" result. Fixes bug 3599. + - Using microdescriptors as a client no longer prevents Tor from + uploading and downloading hidden service descriptors. Fixes + bug 3601. + + o Minor features: + - Allow nameservers with IPv6 address. Resolves bug 2574. + - Accept attempts to include a password authenticator in the + handshake, as supported by SOCKS5. This handles SOCKS clients that + don't know how to omit a password when authenticating. Resolves + bug 1666. + - When configuring a large set of nodes in EntryNodes, and there are + enough of them listed as Guard so that we don't need to consider + the non-guard entries, prefer the ones listed with the Guard flag. + - Check for and recover from inconsistency in the microdescriptor + cache. This will make it harder for us to accidentally free a + microdescriptor without removing it from the appropriate data + structures. Fixes issue 3135; issue noted by "wanoskarnet". + - Log SSL state transitions at log level DEBUG, log domain + HANDSHAKE. This can be useful for debugging censorship events. + Implements ticket 3264. + - Add port 6523 (Gobby) to LongLivedPorts. Patch by intrigeri; + implements ticket 3439. + + o Minor bugfixes (on 0.2.3.1-alpha): + - Do not free all general-purpose regular descriptors just + because microdescriptor use is enabled. Fixes bug 3113. + - Correctly link libevent_openssl when --enable-static-libevent + is passed to configure. Fixes bug 3118. + - Bridges should not complain during their heartbeat log messages that + they are unlisted in the consensus: that's more or less the point + of being a bridge. Fixes bug 3183. + - Report a SIGNAL event to controllers when acting on a delayed + SIGNAL NEWNYM command. Previously, we would report a SIGNAL + event to the controller if we acted on a SIGNAL NEWNYM command + immediately, and otherwise not report a SIGNAL event for the + command at all. Fixes bug 3349. + - Fix a crash when handling the SIGNAL controller command or + reporting ERR-level status events with bufferevents enabled. Found + by Robert Ransom. Fixes bug 3367. + - Always ship the tor-fw-helper manpage in our release tarballs. + Fixes bug 3389. Reported by Stephen Walker. + - Fix a class of double-mark-for-close bugs when bufferevents + are enabled. Fixes bug 3403. + - Update tor-fw-helper to support libnatpmp-20110618. Fixes bug 3434. + - Add SIGNAL to the list returned by the 'GETINFO events/names' + control-port command. Fixes part of bug 3465. + - Prevent using negative indices during unit test runs when read_all() + fails. Spotted by coverity. + - Fix a rare memory leak when checking the nodelist without it being + present. Found by coverity. + - Only try to download a microdescriptor-flavored consensus from + a directory cache that provides them. + + o Minor bugfixes (on 0.2.2.x and earlier): + - Assert that hidden-service-related operations are not performed + using single-hop circuits. Previously, Tor would assert that + client-side streams are not attached to single-hop circuits, + but not that other sensitive operations on the client and service + side are not performed using single-hop circuits. Fixes bug 3332; + bugfix on 0.0.6. + - Don't publish a new relay descriptor when we reload our onion key, + unless the onion key has actually changed. Fixes bug 3263 and + resolves another cause of bug 1810. Bugfix on 0.1.1.11-alpha. + - Allow GETINFO fingerprint to return a fingerprint even when + we have not yet built a router descriptor. Fixes bug 3577; + bugfix on 0.2.0.1-alpha. + - Make 'tor --digests' list hashes of all Tor source files. Bugfix + on 0.2.2.4-alpha; fixes bug 3427. + + o Code simplification and refactoring: + - Use tor_sscanf() in place of scanf() in more places through the + code. This makes us a little more locale-independent, and + should help shut up code-analysis tools that can't tell + a safe sscanf string from a dangerous one. + - Use tt_assert(), not tor_assert(), for checking for test failures. + This makes the unit tests more able to go on in the event that + one of them fails. + - Split connection_about_to_close() into separate functions for each + connection type. + + o Build changes: + - On Windows, we now define the _WIN32_WINNT macros only if they + are not already defined. This lets the person building Tor decide, + if they want, to require a later version of Windows. + + +Changes in version 0.2.2.30-rc - 2011-07-07 + Tor 0.2.2.30-rc is the first release candidate for the Tor 0.2.2.x + series. It fixes a few smaller bugs, but generally appears stable. + Please test it and let us know whether it is! + + o Minor bugfixes: + - Send a SUCCEEDED stream event to the controller when a reverse + resolve succeeded. Fixes bug 3536; bugfix on 0.0.8pre1. Issue + discovered by katmagic. + - Always NUL-terminate the sun_path field of a sockaddr_un before + passing it to the kernel. (Not a security issue: kernels are + smart enough to reject bad sockaddr_uns.) Found by Coverity; + CID #428. Bugfix on Tor 0.2.0.3-alpha. + - Don't stack-allocate the list of supplementary GIDs when we're + about to log them. Stack-allocating NGROUPS_MAX gid_t elements + could take up to 256K, which is way too much stack. Found by + Coverity; CID #450. Bugfix on 0.2.1.7-alpha. + - Add BUILDTIMEOUT_SET to the list returned by the 'GETINFO + events/names' control-port command. Bugfix on 0.2.2.9-alpha; + fixes part of bug 3465. + - Fix a memory leak when receiving a descriptor for a hidden + service we didn't ask for. Found by Coverity; CID #30. Bugfix + on 0.2.2.26-beta. + + o Minor features: + - Update to the July 1 2011 Maxmind GeoLite Country database. + + +Changes in version 0.2.2.29-beta - 2011-06-20 + Tor 0.2.2.29-beta reverts an accidental behavior change for users who + have bridge lines in their torrc but don't want to use them; gets + us closer to having the control socket feature working on Debian; + and fixes a variety of smaller bugs. + + o Major bugfixes: + - Revert the UseBridges option to its behavior before 0.2.2.28-beta. + When we changed the default behavior to "use bridges if any + are listed in the torrc", we surprised users who had bridges + in their torrc files but who didn't actually want to use them. + Partial resolution for bug 3354. + + o Privacy fixes: + - Don't attach new streams to old rendezvous circuits after SIGNAL + NEWNYM. Previously, we would keep using an existing rendezvous + circuit if it remained open (i.e. if it were kept open by a + long-lived stream, or if a new stream were attached to it before + Tor could notice that it was old and no longer in use). Bugfix on + 0.1.1.15-rc; fixes bug 3375. + + o Minor bugfixes: + - Fix a bug when using ControlSocketsGroupWritable with User. The + directory's group would be checked against the current group, not + the configured group. Patch by Jérémy Bobbio. Fixes bug 3393; + bugfix on 0.2.2.26-beta. + - Make connection_printf_to_buf()'s behaviour sane. Its callers + expect it to emit a CRLF iff the format string ends with CRLF; + it actually emitted a CRLF iff (a) the format string ended with + CRLF or (b) the resulting string was over 1023 characters long or + (c) the format string did not end with CRLF *and* the resulting + string was 1021 characters long or longer. Bugfix on 0.1.1.9-alpha; + fixes part of bug 3407. + - Make send_control_event_impl()'s behaviour sane. Its callers + expect it to always emit a CRLF at the end of the string; it + might have emitted extra control characters as well. Bugfix on + 0.1.1.9-alpha; fixes another part of bug 3407. + - Make crypto_rand_int() check the value of its input correctly. + Previously, it accepted values up to UINT_MAX, but could return a + negative number if given a value above INT_MAX+1. Found by George + Kadianakis. Fixes bug 3306; bugfix on 0.2.2pre14. + - Avoid a segfault when reading a malformed circuit build state + with more than INT_MAX entries. Found by wanoskarnet. Bugfix on + 0.2.2.4-alpha. + - When asked about a DNS record type we don't support via a + client DNSPort, reply with NOTIMPL rather than an empty + reply. Patch by intrigeri. Fixes bug 3369; bugfix on 2.0.1-alpha. + - Fix a rare memory leak during stats writing. Found by coverity. + + o Minor features: + - Update to the June 1 2011 Maxmind GeoLite Country database. + + o Code simplifications and refactoring: + - Remove some dead code as indicated by coverity. + - Remove a few dead assignments during router parsing. Found by + coverity. + - Add some forgotten return value checks during unit tests. Found + by coverity. + - Don't use 1-bit wide signed bit fields. Found by coverity. + + +Changes in version 0.2.2.28-beta - 2011-06-04 + Tor 0.2.2.28-beta makes great progress towards a new stable release: we + fixed a big bug in whether relays stay in the consensus consistently, + we moved closer to handling bridges and hidden services correctly, + and we started the process of better handling the dreaded "my Vidalia + died, and now my Tor demands a password when I try to reconnect to it" + usability issue. + + o Major bugfixes: + - Don't decide to make a new descriptor when receiving a HUP signal. + This bug has caused a lot of 0.2.2.x relays to disappear from the + consensus periodically. Fixes the most common case of triggering + bug 1810; bugfix on 0.2.2.7-alpha. + - Actually allow nameservers with IPv6 addresses. Fixes bug 2574. + - Don't try to build descriptors if "ORPort auto" is set and we + don't know our actual ORPort yet. Fix for bug 3216; bugfix on + 0.2.2.26-beta. + - Resolve a crash that occurred when setting BridgeRelay to 1 with + accounting enabled. Fixes bug 3228; bugfix on 0.2.2.18-alpha. + - Apply circuit timeouts to opened hidden-service-related circuits + based on the correct start time. Previously, we would apply the + circuit build timeout based on time since the circuit's creation; + it was supposed to be applied based on time since the circuit + entered its current state. Bugfix on 0.0.6; fixes part of bug 1297. + - Use the same circuit timeout for client-side introduction + circuits as for other four-hop circuits, rather than the timeout + for single-hop directory-fetch circuits; the shorter timeout may + have been appropriate with the static circuit build timeout in + 0.2.1.x and earlier, but caused many hidden service access attempts + to fail with the adaptive CBT introduced in 0.2.2.2-alpha. Bugfix + on 0.2.2.2-alpha; fixes another part of bug 1297. + - In ticket 2511 we fixed a case where you could use an unconfigured + bridge if you had configured it as a bridge the last time you ran + Tor. Now fix another edge case: if you had configured it as a bridge + but then switched to a different bridge via the controller, you + would still be willing to use the old one. Bugfix on 0.2.0.1-alpha; + fixes bug 3321. + + o Major features: + - Add an __OwningControllerProcess configuration option and a + TAKEOWNERSHIP control-port command. Now a Tor controller can ensure + that when it exits, Tor will shut down. Implements feature 3049. + - If "UseBridges 1" is set and no bridges are configured, Tor will + now refuse to build any circuits until some bridges are set. + If "UseBridges auto" is set, Tor will use bridges if they are + configured and we are not running as a server, but otherwise will + make circuits as usual. The new default is "auto". Patch by anonym, + so the Tails LiveCD can stop automatically revealing you as a Tor + user on startup. + + o Minor bugfixes: + - Fix warnings from GCC 4.6's "-Wunused-but-set-variable" option. + - Remove a trailing asterisk from "exit-policy/default" in the + output of the control port command "GETINFO info/names". Bugfix + on 0.1.2.5-alpha. + - Use a wide type to hold sockets when built for 64-bit Windows builds. + Fixes bug 3270. + - Warn when the user configures two HiddenServiceDir lines that point + to the same directory. Bugfix on 0.0.6 (the version introducing + HiddenServiceDir); fixes bug 3289. + - Remove dead code from rend_cache_lookup_v2_desc_as_dir. Fixes + part of bug 2748; bugfix on 0.2.0.10-alpha. + - Log malformed requests for rendezvous descriptors as protocol + warnings, not warnings. Also, use a more informative log message + in case someone sees it at log level warning without prior + info-level messages. Fixes the other part of bug 2748; bugfix + on 0.2.0.10-alpha. + - Clear the table recording the time of the last request for each + hidden service descriptor from each HS directory on SIGNAL NEWNYM. + Previously, we would clear our HS descriptor cache on SIGNAL + NEWNYM, but if we had previously retrieved a descriptor (or tried + to) from every directory responsible for it, we would refuse to + fetch it again for up to 15 minutes. Bugfix on 0.2.2.25-alpha; + fixes bug 3309. + - Fix a log message that said "bits" while displaying a value in + bytes. Found by wanoskarnet. Fixes bug 3318; bugfix on + 0.2.0.1-alpha. + - When checking for 1024-bit keys, check for 1024 bits, not 128 + bytes. This allows Tor to correctly discard keys of length 1017 + through 1023. Bugfix on 0.0.9pre5. + + o Minor features: + - Relays now log the reason for publishing a new relay descriptor, + so we have a better chance of hunting down instances of bug 1810. + Resolves ticket 3252. + - Revise most log messages that refer to nodes by nickname to + instead use the "$key=nickname at address" format. This should be + more useful, especially since nicknames are less and less likely + to be unique. Resolves ticket 3045. + - Log (at info level) when purging pieces of hidden-service-client + state because of SIGNAL NEWNYM. + + o Removed options: + - Remove undocumented option "-F" from tor-resolve: it hasn't done + anything since 0.2.1.16-rc. + + +Changes in version 0.2.2.27-beta - 2011-05-18 + Tor 0.2.2.27-beta fixes a bridge-related stability bug in the previous + release, and also adds a few more general bugfixes. + + o Major bugfixes: + - Fix a crash bug when changing bridges in a running Tor process. + Fixes bug 3213; bugfix on 0.2.2.26-beta. + - When the controller configures a new bridge, don't wait 10 to 60 + seconds before trying to fetch its descriptor. Bugfix on + 0.2.0.3-alpha; fixes bug 3198 (suggested by 2355). + + o Minor bugfixes: + - Require that onion keys have exponent 65537 in microdescriptors too. + Fixes more of bug 3207; bugfix on 0.2.2.26-beta. + - Tor used to limit HttpProxyAuthenticator values to 48 characters. + Changed the limit to 512 characters by removing base64 newlines. + Fixes bug 2752. Fix by Michael Yakubovich. + - When a client starts or stops using bridges, never use a circuit + that was built before the configuration change. This behavior could + put at risk a user who uses bridges to ensure that her traffic + only goes to the chosen addresses. Bugfix on 0.2.0.3-alpha; fixes + bug 3200. + + +Changes in version 0.2.2.26-beta - 2011-05-17 + Tor 0.2.2.26-beta fixes a variety of potential privacy problems. It + also introduces a new "socksport auto" approach that should make it + easier to run multiple Tors on the same system, and does a lot of + cleanup to get us closer to a release candidate. + + o Security/privacy fixes: + - Replace all potentially sensitive memory comparison operations + with versions whose runtime does not depend on the data being + compared. This will help resist a class of attacks where an + adversary can use variations in timing information to learn + sensitive data. Fix for one case of bug 3122. (Safe memcmp + implementation by Robert Ransom based partially on code by DJB.) + - When receiving a hidden service descriptor, check that it is for + the hidden service we wanted. Previously, Tor would store any + hidden service descriptors that a directory gave it, whether it + wanted them or not. This wouldn't have let an attacker impersonate + a hidden service, but it did let directories pre-seed a client + with descriptors that it didn't want. Bugfix on 0.0.6. + - On SIGHUP, do not clear out all TrackHostExits mappings, client + DNS cache entries, and virtual address mappings: that's what + NEWNYM is for. Fixes bug 1345; bugfix on 0.1.0.1-rc. + + o Major features: + - The options SocksPort, ControlPort, and so on now all accept a + value "auto" that opens a socket on an OS-selected port. A + new ControlPortWriteToFile option tells Tor to write its + actual control port or ports to a chosen file. If the option + ControlPortFileGroupReadable is set, the file is created as + group-readable. Now users can run two Tor clients on the same + system without needing to manually mess with parameters. Resolves + part of ticket 3076. + - Set SO_REUSEADDR on all sockets, not just listeners. This should + help busy exit nodes avoid running out of useable ports just + because all the ports have been used in the near past. Resolves + issue 2850. + + o Minor features: + - New "GETINFO net/listeners/(type)" controller command to return + a list of addresses and ports that are bound for listeners for a + given connection type. This is useful when the user has configured + "SocksPort auto" and the controller needs to know which port got + chosen. Resolves another part of ticket 3076. + - Add a new ControlSocketsGroupWritable configuration option: when + it is turned on, ControlSockets are group-writeable by the default + group of the current user. Patch by Jérémy Bobbio; implements + ticket 2972. + - Tor now refuses to create a ControlSocket in a directory that is + world-readable (or group-readable if ControlSocketsGroupWritable + is 0). This is necessary because some operating systems do not + enforce permissions on an AF_UNIX sockets. Permissions on the + directory holding the socket, however, seems to work everywhere. + - Rate-limit a warning about failures to download v2 networkstatus + documents. Resolves part of bug 1352. + - Backport code from 0.2.3.x that allows directory authorities to + clean their microdescriptor caches. Needed to resolve bug 2230. + - When an HTTPS proxy reports "403 Forbidden", we now explain + what it means rather than calling it an unexpected status code. + Closes bug 2503. Patch from Michael Yakubovich. + - Update to the May 1 2011 Maxmind GeoLite Country database. + + o Minor bugfixes: + - Authorities now clean their microdesc cache periodically and when + reading from disk initially, not only when adding new descriptors. + This prevents a bug where we could lose microdescriptors. Bugfix + on 0.2.2.6-alpha. Fixes bug 2230. + - Do not crash when our configuration file becomes unreadable, for + example due to a permissions change, between when we start up + and when a controller calls SAVECONF. Fixes bug 3135; bugfix + on 0.0.9pre6. + - Avoid a bug that would keep us from replacing a microdescriptor + cache on Windows. (We would try to replace the file while still + holding it open. That's fine on Unix, but Windows doesn't let us + do that.) Bugfix on 0.2.2.6-alpha; bug found by wanoskarnet. + - Add missing explanations for the authority-related torrc options + RephistTrackTime, BridgePassword, and V3AuthUseLegacyKey in the + man page. Resolves issue 2379. + - As an authority, do not upload our own vote or signature set to + ourself. It would tell us nothing new, and as of 0.2.2.24-alpha, + it would get flagged as a duplicate. Resolves bug 3026. + - Accept hidden service descriptors if we think we might be a hidden + service directory, regardless of what our consensus says. This + helps robustness, since clients and hidden services can sometimes + have a more up-to-date view of the network consensus than we do, + and if they think that the directory authorities list us a HSDir, + we might actually be one. Related to bug 2732; bugfix on + 0.2.0.10-alpha. + - When a controller changes TrackHostExits, remove mappings for + hosts that should no longer have their exits tracked. Bugfix on + 0.1.0.1-rc. + - When a controller changes VirtualAddrNetwork, remove any mappings + for hosts that were automapped to the old network. Bugfix on + 0.1.1.19-rc. + - When a controller changes one of the AutomapHosts* options, remove + any mappings for hosts that should no longer be automapped. Bugfix + on 0.2.0.1-alpha. + - Do not reset the bridge descriptor download status every time we + re-parse our configuration or get a configuration change. Fixes + bug 3019; bugfix on 0.2.0.3-alpha. + + o Minor bugfixes (code cleanup): + - When loading the microdesc journal, remember its current size. + In 0.2.2, this helps prevent the microdesc journal from growing + without limit on authorities (who are the only ones to use it in + 0.2.2). Fixes a part of bug 2230; bugfix on 0.2.2.6-alpha. + Fix posted by "cypherpunks." + - The microdesc journal is supposed to get rebuilt only if it is + at least _half_ the length of the store, not _twice_ the length + of the store. Bugfix on 0.2.2.6-alpha; fixes part of bug 2230. + - Fix a potential null-pointer dereference while computing a + consensus. Bugfix on tor-0.2.0.3-alpha, found with the help of + clang's analyzer. + - Avoid a possible null-pointer dereference when rebuilding the mdesc + cache without actually having any descriptors to cache. Bugfix on + 0.2.2.6-alpha. Issue discovered using clang's static analyzer. + - If we fail to compute the identity digest of a v3 legacy keypair, + warn, and don't use a buffer-full of junk instead. Bugfix on + 0.2.1.1-alpha; fixes bug 3106. + - Resolve an untriggerable issue in smartlist_string_num_isin(), + where if the function had ever in the future been used to check + for the presence of a too-large number, it would have given an + incorrect result. (Fortunately, we only used it for 16-bit + values.) Fixes bug 3175; bugfix on 0.1.0.1-rc. + - Require that introduction point keys and onion handshake keys + have a public exponent of 65537. Starts to fix bug 3207; bugfix + on 0.2.0.10-alpha. + + o Removed features: + - Caches no longer download and serve v2 networkstatus documents + unless FetchV2Networkstatus flag is set: these documents haven't + haven't been used by clients or relays since 0.2.0.x. Resolves + bug 3022. + + +Changes in version 0.2.3.1-alpha - 2011-05-05 + Tor 0.2.3.1-alpha adds some new experimental features, including support + for an improved network IO backend, IOCP networking on Windows, + microdescriptor caching, "fast-start" support for streams, and automatic + home router configuration. There are also numerous internal improvements + to try to make the code easier for developers to work with. + + This is the first alpha release in a new series, so expect there to be + bugs. Users who would rather test out a more stable branch should + stay with 0.2.2.x for now. + + o Major features: + - Tor can now optionally build with the "bufferevents" buffered IO + backend provided by Libevent 2. To use this feature, make sure you + have the latest possible version of Libevent, and pass the + --enable-bufferevents flag to configure when building Tor from + source. This feature will make our networking code more flexible, + let us stack layers on each other, and let us use more efficient + zero-copy transports where available. + - As an experimental feature, Tor can use IOCP for networking on Windows. + Once this code is tuned and optimized, it promises much better + performance than the select-based backend we've used in the past. To + try this feature, you must build Tor with Libevent 2, configure Tor + with the "bufferevents" buffered IO backend, and add "DisableIOCP 0" to + your torrc. There are known bugs here: only try this if you can help + debug it as it breaks. + - The EntryNodes option can now include country codes like {de} or IP + addresses or network masks. Previously we had disallowed these options + because we didn't have an efficient way to keep the list up to + date. Fixes bug 1982, but see bug 2798 for an unresolved issue here. + - Exit nodes now accept and queue data on not-yet-connected streams. + Previously, the client wasn't allowed to send data until the stream was + connected, which slowed down all connections. This change will enable + clients to perform a "fast-start" on streams and send data without + having to wait for a confirmation that the stream has opened. (Patch + from Ian Goldberg; implements the server side of Proposal 174.) + - Tor now has initial support for automatic port mapping on the many + home routers that support NAT-PMP or UPnP. (Not yet supported on + Windows). To build the support code, you'll need to have libnatpnp + library and/or the libminiupnpc library, and you'll need to enable the + feature specifically by passing "--enable-upnp" and/or + "--enable-natpnp" to configure. To turn it on, use the new + PortForwarding option. + - Caches now download, cache, and serve multiple "flavors" of the + consensus, including a flavor that describes microdescriptors. + - Caches now download, cache, and serve microdescriptors -- small + summaries of router descriptors that are authenticated by all of the + directory authorities. Once enough caches are running this code, + clients will be able to save significant amounts of directory bandwidth + by downloading microdescriptors instead of router descriptors. + + o Minor features: + - Make logging resolution configurable with a new LogTimeGranularity + option, and change the default from 1 millisecond to 1 second. + Implements enhancement 1668. + - We log which torrc file we're using on startup. Implements ticket + 2444. + - Ordinarily, Tor does not count traffic from private addresses (like + 127.0.0.1 or 10.0.0.1) when calculating rate limits or accounting. + There is now a new option, CountPrivateBandwidth, to disable this + behavior. Patch from Daniel Cagara. + - New --enable-static-tor configure option for building Tor as + statically as possible. Idea, general hackery and thoughts from + Alexei Czeskis, John Gilmore, Jacob Appelbaum. Implements ticket + 2702. + - If you set the NumCPUs option to 0, Tor will now try to detect how + many CPUs you have. This is the new default behavior. + - Turn on directory request statistics by default and include them in + extra-info descriptors. Don't break if we have no GeoIP database. + - Relays that set "ConnDirectionStatistics 1" write statistics on the + bidirectional use of connections to disk every 24 hours. + - Add a GeoIP file digest to the extra-info descriptor. Implements + enhancement 1883. + - The NodeFamily option -- which let you declare that you want to + consider nodes to be part of a family whether they list themselves + that way or not -- now allows IP address ranges and country codes. + - Add a new 'Heartbeat' log message type to periodically log a message + describing Tor's status at level Notice. This feature is meant for + operators who log at notice, and want to make sure that their Tor + server is still working. Implementation by George Kadianakis. + + o Minor bugfixes (on 0.2.2.25-alpha): + - When loading the microdesc journal, remember its current size. + In 0.2.2, this helps prevent the microdesc journal from growing + without limit on authorities (who are the only ones to use it in + 0.2.2). Fixes a part of bug 2230; bugfix on 0.2.2.6-alpha. + Fix posted by "cypherpunks." + - The microdesc journal is supposed to get rebuilt only if it is + at least _half_ the length of the store, not _twice_ the length + of the store. Bugfix on 0.2.2.6-alpha; fixes part of bug 2230. + - If as an authority we fail to compute the identity digest of a v3 + legacy keypair, warn, and don't use a buffer-full of junk instead. + Bugfix on 0.2.1.1-alpha; fixes bug 3106. + - Authorities now clean their microdesc cache periodically and when + reading from disk initially, not only when adding new descriptors. + This prevents a bug where we could lose microdescriptors. Bugfix + on 0.2.2.6-alpha. + + o Minor features (controller): + - Add a new SIGNAL event to the controller interface so that + controllers can be notified when Tor handles a signal. Resolves + issue 1955. Patch by John Brooks. + - Add a new GETINFO option to get total bytes read and written. Patch + from pipe, revised by atagar. Resolves ticket 2345. + - Implement some GETINFO controller fields to provide information about + the Tor process's pid, euid, username, and resource limits. + + o Build changes: + - Our build system requires automake 1.6 or later to create the + Makefile.in files. Previously, you could have used 1.4. + This only affects developers and people building Tor from git; + people who build Tor from the source distribution without changing + the Makefile.am files should be fine. + - Our autogen.sh script uses autoreconf to launch autoconf, automake, and + so on. This is more robust against some of the failure modes + associated with running the autotools pieces on their own. + + o Minor packaging issues: + - On OpenSUSE, create the /var/run/tor directory on startup if it is not + already created. Patch from Andreas Stieger. Fixes bug 2573. + + o Code simplifications and refactoring: + - A major revision to our internal node-selecting and listing logic. + Tor already had at least two major ways to look at the question of + "which Tor servers do we know about": a list of router descriptors, + and a list of entries in the current consensus. With + microdescriptors, we're adding a third. Having so many systems + without an abstraction layer over them was hurting the codebase. + Now, we have a new "node_t" abstraction that presents a consistent + interface to a client's view of a Tor node, and holds (nearly) all + of the mutable state formerly in routerinfo_t and routerstatus_t. + - The helper programs tor-gencert, tor-resolve, and tor-checkkey + no longer link against Libevent: they never used it, but + our library structure used to force them to link it. + + o Removed features: + - Remove some old code to work around even older versions of Tor that + used forked processes to handle DNS requests. Such versions of Tor + are no longer in use as servers. + + o Documentation fixes: + - Correct a broken faq link in the INSTALL file. Fixes bug 2307. + - Add missing documentation for the authority-related torrc options + RephistTrackTime, BridgePassword, and V3AuthUseLegacyKey. Resolves + issue 2379. + + +Changes in version 0.2.2.25-alpha - 2011-04-29 + Tor 0.2.2.25-alpha fixes many bugs: hidden service clients are more + robust, routers no longer overreport their bandwidth, Win7 should crash + a little less, and NEWNYM (as used by Vidalia's "new identity" button) + now prevents hidden service-related activity from being linkable. It + provides more information to Vidalia so you can see if your bridge is + working. Also, 0.2.2.25-alpha revamps the Entry/Exit/ExcludeNodes and + StrictNodes configuration options to make them more reliable, more + understandable, and more regularly applied. If you use those options, + please see the revised documentation for them in the manual page. + + o Major bugfixes: + - Relays were publishing grossly inflated bandwidth values because + they were writing their state files wrong--now they write the + correct value. Also, resume reading bandwidth history from the + state file correctly. Fixes bug 2704; bugfix on 0.2.2.23-alpha. + - Improve hidden service robustness: When we find that we have + extended a hidden service's introduction circuit to a relay not + listed as an introduction point in the HS descriptor we currently + have, retry with an introduction point from the current + descriptor. Previously we would just give up. Fixes bugs 1024 and + 1930; bugfix on 0.2.0.10-alpha. + - Clients now stop trying to use an exit node associated with a given + destination by TrackHostExits if they fail to reach that exit node. + Fixes bug 2999. Bugfix on 0.2.0.20-rc. + - Fix crash bug on platforms where gmtime and localtime can return + NULL. Windows 7 users were running into this one. Fixes part of bug + 2077. Bugfix on all versions of Tor. Found by boboper. + + o Security and stability fixes: + - Don't double-free a parsable, but invalid, microdescriptor, even if + it is followed in the blob we're parsing by an unparsable + microdescriptor. Fixes an issue reported in a comment on bug 2954. + Bugfix on 0.2.2.6-alpha; fix by "cypherpunks". + - If the Nickname configuration option isn't given, Tor would pick a + nickname based on the local hostname as the nickname for a relay. + Because nicknames are not very important in today's Tor and the + "Unnamed" nickname has been implemented, this is now problematic + behavior: It leaks information about the hostname without being + useful at all. Fixes bug 2979; bugfix on 0.1.2.2-alpha, which + introduced the Unnamed nickname. Reported by tagnaq. + - Fix an uncommon assertion failure when running with DNSPort under + heavy load. Fixes bug 2933; bugfix on 0.2.0.1-alpha. + - Avoid linkability based on cached hidden service descriptors: forget + all hidden service descriptors cached as a client when processing a + SIGNAL NEWNYM command. Fixes bug 3000; bugfix on 0.0.6. + + o Major features: + - Export GeoIP information on bridge usage to controllers even if we + have not yet been running for 24 hours. Now Vidalia bridge operators + can get more accurate and immediate feedback about their + contributions to the network. + + o Major features and bugfixes (node selection): + - Revise and reconcile the meaning of the ExitNodes, EntryNodes, + ExcludeEntryNodes, ExcludeExitNodes, ExcludeNodes, and StrictNodes + options. Previously, we had been ambiguous in describing what + counted as an "exit" node, and what operations exactly "StrictNodes + 0" would permit. This created confusion when people saw nodes built + through unexpected circuits, and made it hard to tell real bugs from + surprises. Now the intended behavior is: + . "Exit", in the context of ExitNodes and ExcludeExitNodes, means + a node that delivers user traffic outside the Tor network. + . "Entry", in the context of EntryNodes, means a node used as the + first hop of a multihop circuit. It doesn't include direct + connections to directory servers. + . "ExcludeNodes" applies to all nodes. + . "StrictNodes" changes the behavior of ExcludeNodes only. When + StrictNodes is set, Tor should avoid all nodes listed in + ExcludeNodes, even when it will make user requests fail. When + StrictNodes is *not* set, then Tor should follow ExcludeNodes + whenever it can, except when it must use an excluded node to + perform self-tests, connect to a hidden service, provide a + hidden service, fulfill a .exit request, upload directory + information, or fetch directory information. + Collectively, the changes to implement the behavior fix bug 1090. + - ExcludeNodes now takes precedence over EntryNodes and ExitNodes: if + a node is listed in both, it's treated as excluded. + - ExcludeNodes now applies to directory nodes -- as a preference if + StrictNodes is 0, or an absolute requirement if StrictNodes is 1. + Don't exclude all the directory authorities and set StrictNodes to 1 + unless you really want your Tor to break. + - ExcludeNodes and ExcludeExitNodes now override exit enclaving. + - ExcludeExitNodes now overrides .exit requests. + - We don't use bridges listed in ExcludeNodes. + - When StrictNodes is 1: + . We now apply ExcludeNodes to hidden service introduction points + and to rendezvous points selected by hidden service users. This + can make your hidden service less reliable: use it with caution! + . If we have used ExcludeNodes on ourself, do not try relay + reachability self-tests. + . If we have excluded all the directory authorities, we will not + even try to upload our descriptor if we're a relay. + . Do not honor .exit requests to an excluded node. + - Remove a misfeature that caused us to ignore the Fast/Stable flags + when ExitNodes is set. Bugfix on 0.2.2.7-alpha. + - When the set of permitted nodes changes, we now remove any mappings + introduced via TrackExitHosts to now-excluded nodes. Bugfix on + 0.1.0.1-rc. + - We never cannibalize a circuit that had excluded nodes on it, even + if StrictNodes is 0. Bugfix on 0.1.0.1-rc. + - Revert a change where we would be laxer about attaching streams to + circuits than when building the circuits. This was meant to prevent + a set of bugs where streams were never attachable, but our improved + code here should make this unnecessary. Bugfix on 0.2.2.7-alpha. + - Keep track of how many times we launch a new circuit to handle a + given stream. Too many launches could indicate an inconsistency + between our "launch a circuit to handle this stream" logic and our + "attach this stream to one of the available circuits" logic. + - Improve log messages related to excluded nodes. + + o Minor bugfixes: + - Fix a spurious warning when moving from a short month to a long + month on relays with month-based BandwidthAccounting. Bugfix on + 0.2.2.17-alpha; fixes bug 3020. + - When a client finds that an origin circuit has run out of 16-bit + stream IDs, we now mark it as unusable for new streams. Previously, + we would try to close the entire circuit. Bugfix on 0.0.6. + - Add a forgotten cast that caused a compile warning on OS X 10.6. + Bugfix on 0.2.2.24-alpha. + - Be more careful about reporting the correct error from a failed + connect() system call. Under some circumstances, it was possible to + look at an incorrect value for errno when sending the end reason. + Bugfix on 0.1.0.1-rc. + - Correctly handle an "impossible" overflow cases in connection byte + counting, where we write or read more than 4GB on an edge connection + in a single second. Bugfix on 0.1.2.8-beta. + - Correct the warning displayed when a rendezvous descriptor exceeds + the maximum size. Fixes bug 2750; bugfix on 0.2.1.5-alpha. Found by + John Brooks. + - Clients and hidden services now use HSDir-flagged relays for hidden + service descriptor downloads and uploads even if the relays have no + DirPort set and the client has disabled TunnelDirConns. This will + eventually allow us to give the HSDir flag to relays with no + DirPort. Fixes bug 2722; bugfix on 0.2.1.6-alpha. + - Downgrade "no current certificates known for authority" message from + Notice to Info. Fixes bug 2899; bugfix on 0.2.0.10-alpha. + - Make the SIGNAL DUMP control-port command work on FreeBSD. Fixes bug + 2917. Bugfix on 0.1.1.1-alpha. + - Only limit the lengths of single HS descriptors, even when multiple + HS descriptors are published to an HSDir relay in a single POST + operation. Fixes bug 2948; bugfix on 0.2.1.5-alpha. Found by hsdir. + - Write the current time into the LastWritten line in our state file, + rather than the time from the previous write attempt. Also, stop + trying to use a time of -1 in our log statements. Fixes bug 3039; + bugfix on 0.2.2.14-alpha. + - Be more consistent in our treatment of file system paths. "~" should + get expanded to the user's home directory in the Log config option. + Fixes bug 2971; bugfix on 0.2.0.1-alpha, which introduced the + feature for the -f and --DataDirectory options. + + o Minor features: + - Make sure every relay writes a state file at least every 12 hours. + Previously, a relay could go for weeks without writing its state + file, and on a crash could lose its bandwidth history, capacity + estimates, client country statistics, and so on. Addresses bug 3012. + - Send END_STREAM_REASON_NOROUTE in response to EHOSTUNREACH errors. + Clients before 0.2.1.27 didn't handle NOROUTE correctly, but such + clients are already deprecated because of security bugs. + - Don't allow v0 hidden service authorities to act as clients. + Required by fix for bug 3000. + - Ignore SIGNAL NEWNYM commands on relay-only Tor instances. Required + by fix for bug 3000. + - Ensure that no empty [dirreq-](read|write)-history lines are added + to an extrainfo document. Implements ticket 2497. + + o Code simplification and refactoring: + - Remove workaround code to handle directory responses from servers + that had bug 539 (they would send HTTP status 503 responses _and_ + send a body too). Since only server versions before + 0.2.0.16-alpha/0.1.2.19 were affected, there is no longer reason to + keep the workaround in place. + - Remove the old 'fuzzy time' logic. It was supposed to be used for + handling calculations where we have a known amount of clock skew and + an allowed amount of unknown skew. But we only used it in three + places, and we never adjusted the known/unknown skew values. This is + still something we might want to do someday, but if we do, we'll + want to do it differently. + - Avoid signed/unsigned comparisons by making SIZE_T_CEILING unsigned. + None of the cases where we did this before were wrong, but by making + this change we avoid warnings. Fixes bug 2475; bugfix on 0.2.1.28. + - Use GetTempDir to find the proper temporary directory location on + Windows when generating temporary files for the unit tests. Patch by + Gisle Vanem. + + +Changes in version 0.2.2.24-alpha - 2011-04-08 + Tor 0.2.2.24-alpha fixes a variety of bugs, including a big bug that + prevented Tor clients from effectively using "multihomed" bridges, + that is, bridges that listen on multiple ports or IP addresses so users + can continue to use some of their addresses even if others get blocked. + + o Major bugfixes: + - Fix a bug where bridge users who configure the non-canonical + address of a bridge automatically switch to its canonical + address. If a bridge listens at more than one address, it should be + able to advertise those addresses independently and any non-blocked + addresses should continue to work. Bugfix on Tor 0.2.0.x. Fixes + bug 2510. + - If you configured Tor to use bridge A, and then quit and + configured Tor to use bridge B instead, it would happily continue + to use bridge A if it's still reachable. While this behavior is + a feature if your goal is connectivity, in some scenarios it's a + dangerous bug. Bugfix on Tor 0.2.0.1-alpha; fixes bug 2511. + - Directory authorities now use data collected from their own + uptime observations when choosing whether to assign the HSDir flag + to relays, instead of trusting the uptime value the relay reports in + its descriptor. This change helps prevent an attack where a small + set of nodes with frequently-changing identity keys can blackhole + a hidden service. (Only authorities need upgrade; others will be + fine once they do.) Bugfix on 0.2.0.10-alpha; fixes bug 2709. + + o Minor bugfixes: + - When we restart our relay, we might get a successful connection + from the outside before we've started our reachability tests, + triggering a warning: "ORPort found reachable, but I have no + routerinfo yet. Failing to inform controller of success." This + bug was harmless unless Tor is running under a controller + like Vidalia, in which case the controller would never get a + REACHABILITY_SUCCEEDED status event. Bugfix on 0.1.2.6-alpha; + fixes bug 1172. + - Make directory authorities more accurate at recording when + relays that have failed several reachability tests became + unreachable, so we can provide more accuracy at assigning Stable, + Guard, HSDir, etc flags. Bugfix on 0.2.0.6-alpha. Resolves bug 2716. + - Fix an issue that prevented static linking of libevent on + some platforms (notably Linux). Fixes bug 2698; bugfix on + versions 0.2.1.23/0.2.2.8-alpha (the versions introducing + the --with-static-libevent configure option). + - We now ask the other side of a stream (the client or the exit) + for more data on that stream when the amount of queued data on + that stream dips low enough. Previously, we wouldn't ask the + other side for more data until either it sent us more data (which + it wasn't supposed to do if it had exhausted its window!) or we + had completely flushed all our queued data. This flow control fix + should improve throughput. Fixes bug 2756; bugfix on the earliest + released versions of Tor (svn commit r152). + - Avoid a double-mark-for-free warning when failing to attach a + transparent proxy connection. (We thought we had fixed this in + 0.2.2.23-alpha, but it turns out our fix was checking the wrong + connection.) Fixes bug 2757; bugfix on 0.1.2.1-alpha (the original + bug) and 0.2.2.23-alpha (the incorrect fix). + - When warning about missing zlib development packages during compile, + give the correct package names. Bugfix on 0.2.0.1-alpha. + + o Minor features: + - Directory authorities now log the source of a rejected POSTed v3 + networkstatus vote. + - Make compilation with clang possible when using + --enable-gcc-warnings by removing two warning options that clang + hasn't implemented yet and by fixing a few warnings. Implements + ticket 2696. + - When expiring circuits, use microsecond timers rather than + one-second timers. This can avoid an unpleasant situation where a + circuit is launched near the end of one second and expired right + near the beginning of the next, and prevent fluctuations in circuit + timeout values. + - Use computed circuit-build timeouts to decide when to launch + parallel introduction circuits for hidden services. (Previously, + we would retry after 15 seconds.) + - Update to the April 1 2011 Maxmind GeoLite Country database. + + o Packaging fixes: + - Create the /var/run/tor directory on startup on OpenSUSE if it is + not already created. Patch from Andreas Stieger. Fixes bug 2573. + + o Documentation changes: + - Modernize the doxygen configuration file slightly. Fixes bug 2707. + - Resolve all doxygen warnings except those for missing documentation. + Fixes bug 2705. + - Add doxygen documentation for more functions, fields, and types. + + +Changes in version 0.2.2.23-alpha - 2011-03-08 + Tor 0.2.2.23-alpha lets relays record their bandwidth history so when + they restart they don't lose their bandwidth capacity estimate. This + release also fixes a diverse set of user-facing bugs, ranging from + relays overrunning their rate limiting to clients falsely warning about + clock skew to bridge descriptor leaks by our bridge directory authority. + + o Major bugfixes: + - Stop sending a CLOCK_SKEW controller status event whenever + we fetch directory information from a relay that has a wrong clock. + Instead, only inform the controller when it's a trusted authority + that claims our clock is wrong. Bugfix on 0.1.2.6-alpha; fixes + the rest of bug 1074. + - Fix an assert in parsing router descriptors containing IPv6 + addresses. This one took down the directory authorities when + somebody tried some experimental code. Bugfix on 0.2.1.3-alpha. + - Make the bridge directory authority refuse to answer directory + requests for "all" descriptors. It used to include bridge + descriptors in its answer, which was a major information leak. + Found by "piebeer". Bugfix on 0.2.0.3-alpha. + - If relays set RelayBandwidthBurst but not RelayBandwidthRate, + Tor would ignore their RelayBandwidthBurst setting, + potentially using more bandwidth than expected. Bugfix on + 0.2.0.1-alpha. Reported by Paul Wouters. Fixes bug 2470. + - Ignore and warn if the user mistakenly sets "PublishServerDescriptor + hidserv" in her torrc. The 'hidserv' argument never controlled + publication of hidden service descriptors. Bugfix on 0.2.0.1-alpha. + + o Major features: + - Relays now save observed peak bandwidth throughput rates to their + state file (along with total usage, which was already saved) + so that they can determine their correct estimated bandwidth on + restart. Resolves bug 1863, where Tor relays would reset their + estimated bandwidth to 0 after restarting. + - Directory authorities now take changes in router IP address and + ORPort into account when determining router stability. Previously, + if a router changed its IP or ORPort, the authorities would not + treat it as having any downtime for the purposes of stability + calculation, whereas clients would experience downtime since the + change could take a while to propagate to them. Resolves issue 1035. + - Enable Address Space Layout Randomization (ASLR) and Data Execution + Prevention (DEP) by default on Windows to make it harder for + attackers to exploit vulnerabilities. Patch from John Brooks. + + o Minor bugfixes (on 0.2.1.x and earlier): + - Fix a rare crash bug that could occur when a client was configured + with a large number of bridges. Fixes bug 2629; bugfix on + 0.2.1.2-alpha. Bugfix by trac user "shitlei". + - Avoid a double mark-for-free warning when failing to attach a + transparent proxy connection. Bugfix on 0.1.2.1-alpha. Fixes + bug 2279. + - Correctly detect failure to allocate an OpenSSL BIO. Fixes bug 2378; + found by "cypherpunks". This bug was introduced before the first + Tor release, in svn commit r110. + - Country codes aren't supported in EntryNodes until 0.2.3.x, so + don't mention them in the manpage. Fixes bug 2450; issue + spotted by keb and G-Lo. + - Fix a bug in bandwidth history state parsing that could have been + triggered if a future version of Tor ever changed the timing + granularity at which bandwidth history is measured. Bugfix on + Tor 0.1.1.11-alpha. + - When a relay decides that its DNS is too broken for it to serve + as an exit server, it advertised itself as a non-exit, but + continued to act as an exit. This could create accidental + partitioning opportunities for users. Instead, if a relay is + going to advertise reject *:* as its exit policy, it should + really act with exit policy "reject *:*". Fixes bug 2366. + Bugfix on Tor 0.1.2.5-alpha. Bugfix by user "postman" on trac. + - In the special case where you configure a public exit relay as your + bridge, Tor would be willing to use that exit relay as the last + hop in your circuit as well. Now we fail that circuit instead. + Bugfix on 0.2.0.12-alpha. Fixes bug 2403. Reported by "piebeer". + - Fix a bug with our locking implementation on Windows that couldn't + correctly detect when a file was already locked. Fixes bug 2504, + bugfix on 0.2.1.6-alpha. + - Fix IPv6-related connect() failures on some platforms (BSD, OS X). + Bugfix on 0.2.0.3-alpha; fixes first part of bug 2660. Patch by + "piebeer". + - Set target port in get_interface_address6() correctly. Bugfix + on 0.1.1.4-alpha and 0.2.0.3-alpha; fixes second part of bug 2660. + - Directory authorities are now more robust to hops back in time + when calculating router stability. Previously, if a run of uptime + or downtime appeared to be negative, the calculation could give + incorrect results. Bugfix on 0.2.0.6-alpha; noticed when fixing + bug 1035. + - Fix an assert that got triggered when using the TestingTorNetwork + configuration option and then issuing a GETINFO config-text control + command. Fixes bug 2250; bugfix on 0.2.1.2-alpha. + + o Minor bugfixes (on 0.2.2.x): + - Clients should not weight BadExit nodes as Exits in their node + selection. Similarly, directory authorities should not count BadExit + bandwidth as Exit bandwidth when computing bandwidth-weights. + Bugfix on 0.2.2.10-alpha; fixes bug 2203. + - Correctly clear our dir_read/dir_write history when there is an + error parsing any bw history value from the state file. Bugfix on + Tor 0.2.2.15-alpha. + - Resolve a bug in verifying signatures of directory objects + with digests longer than SHA1. Bugfix on 0.2.2.20-alpha. + Fixes bug 2409. Found by "piebeer". + - Bridge authorities no longer crash on SIGHUP when they try to + publish their relay descriptor to themselves. Fixes bug 2572. Bugfix + on 0.2.2.22-alpha. + + o Minor features: + - Log less aggressively about circuit timeout changes, and improve + some other circuit timeout messages. Resolves bug 2004. + - Log a little more clearly about the times at which we're no longer + accepting new connections. Resolves bug 2181. + - Reject attempts at the client side to open connections to private + IP addresses (like 127.0.0.1, 10.0.0.1, and so on) with + a randomly chosen exit node. Attempts to do so are always + ill-defined, generally prevented by exit policies, and usually + in error. This will also help to detect loops in transparent + proxy configurations. You can disable this feature by setting + "ClientRejectInternalAddresses 0" in your torrc. + - Always treat failure to allocate an RSA key as an unrecoverable + allocation error. + - Update to the March 1 2011 Maxmind GeoLite Country database. + + o Minor features (log subsystem): + - Add documentation for configuring logging at different severities in + different log domains. We've had this feature since 0.2.1.1-alpha, + but for some reason it never made it into the manpage. Fixes + bug 2215. + - Make it simpler to specify "All log domains except for A and B". + Previously you needed to say "[*,~A,~B]". Now you can just say + "[~A,~B]". + - Add a "LogMessageDomains 1" option to include the domains of log + messages along with the messages. Without this, there's no way + to use log domains without reading the source or doing a lot + of guessing. + + o Packaging changes: + - Stop shipping the Tor specs files and development proposal documents + in the tarball. They are now in a separate git repository at + git://git.torproject.org/torspec.git + + +Changes in version 0.2.1.30 - 2011-02-23 + Tor 0.2.1.30 fixes a variety of less critical bugs. The main other + change is a slight tweak to Tor's TLS handshake that makes relays + and bridges that run this new version reachable from Iran again. + We don't expect this tweak will win the arms race long-term, but it + buys us time until we roll out a better solution. + + o Major bugfixes: + - Stop sending a CLOCK_SKEW controller status event whenever + we fetch directory information from a relay that has a wrong clock. + Instead, only inform the controller when it's a trusted authority + that claims our clock is wrong. Bugfix on 0.1.2.6-alpha; fixes + the rest of bug 1074. + - Fix a bounds-checking error that could allow an attacker to + remotely crash a directory authority. Bugfix on 0.2.1.5-alpha. + Found by "piebeer". + - If relays set RelayBandwidthBurst but not RelayBandwidthRate, + Tor would ignore their RelayBandwidthBurst setting, + potentially using more bandwidth than expected. Bugfix on + 0.2.0.1-alpha. Reported by Paul Wouters. Fixes bug 2470. + - Ignore and warn if the user mistakenly sets "PublishServerDescriptor + hidserv" in her torrc. The 'hidserv' argument never controlled + publication of hidden service descriptors. Bugfix on 0.2.0.1-alpha. + + o Minor features: + - Adjust our TLS Diffie-Hellman parameters to match those used by + Apache's mod_ssl. + - Update to the February 1 2011 Maxmind GeoLite Country database. + + o Minor bugfixes: + - Check for and reject overly long directory certificates and + directory tokens before they have a chance to hit any assertions. + Bugfix on 0.2.1.28. Found by "doorss". + - Bring the logic that gathers routerinfos and assesses the + acceptability of circuits into line. This prevents a Tor OP from + getting locked in a cycle of choosing its local OR as an exit for a + path (due to a .exit request) and then rejecting the circuit because + its OR is not listed yet. It also prevents Tor clients from using an + OR running in the same instance as an exit (due to a .exit request) + if the OR does not meet the same requirements expected of an OR + running elsewhere. Fixes bug 1859; bugfix on 0.1.0.1-rc. + + o Packaging changes: + - Stop shipping the Tor specs files and development proposal documents + in the tarball. They are now in a separate git repository at + git://git.torproject.org/torspec.git + - Do not include Git version tags as though they are SVN tags when + generating a tarball from inside a repository that has switched + between branches. Bugfix on 0.2.1.15-rc; fixes bug 2402. + + +Changes in version 0.2.2.22-alpha - 2011-01-25 + Tor 0.2.2.22-alpha fixes a few more less-critical security issues. The + main other change is a slight tweak to Tor's TLS handshake that makes + relays and bridges that run this new version reachable from Iran again. + We don't expect this tweak will win the arms race long-term, but it + will buy us a bit more time until we roll out a better solution. + + o Major bugfixes: + - Fix a bounds-checking error that could allow an attacker to + remotely crash a directory authority. Bugfix on 0.2.1.5-alpha. + Found by "piebeer". + - Don't assert when changing from bridge to relay or vice versa + via the controller. The assert happened because we didn't properly + initialize our keys in this case. Bugfix on 0.2.2.18-alpha; fixes + bug 2433. Reported by bastik. + + o Minor features: + - Adjust our TLS Diffie-Hellman parameters to match those used by + Apache's mod_ssl. + - Provide a log message stating which geoip file we're parsing + instead of just stating that we're parsing the geoip file. + Implements ticket 2432. + + o Minor bugfixes: + - Check for and reject overly long directory certificates and + directory tokens before they have a chance to hit any assertions. + Bugfix on 0.2.1.28 / 0.2.2.20-alpha. Found by "doorss". + + +Changes in version 0.2.2.21-alpha - 2011-01-15 + Tor 0.2.2.21-alpha includes all the patches from Tor 0.2.1.29, which + continues our recent code security audit work. The main fix resolves + a remote heap overflow vulnerability that can allow remote code + execution (CVE-2011-0427). Other fixes address a variety of assert + and crash bugs, most of which we think are hard to exploit remotely. + + o Major bugfixes (security), also included in 0.2.1.29: + - Fix a heap overflow bug where an adversary could cause heap + corruption. This bug probably allows remote code execution + attacks. Reported by "debuger". Fixes CVE-2011-0427. Bugfix on + 0.1.2.10-rc. + - Prevent a denial-of-service attack by disallowing any + zlib-compressed data whose compression factor is implausibly + high. Fixes part of bug 2324; reported by "doorss". + - Zero out a few more keys in memory before freeing them. Fixes + bug 2384 and part of bug 2385. These key instances found by + "cypherpunks", based on Andrew Case's report about being able + to find sensitive data in Tor's memory space if you have enough + permissions. Bugfix on 0.0.2pre9. + + o Major bugfixes (crashes), also included in 0.2.1.29: + - Prevent calls to Libevent from inside Libevent log handlers. + This had potential to cause a nasty set of crashes, especially + if running Libevent with debug logging enabled, and running + Tor with a controller watching for low-severity log messages. + Bugfix on 0.1.0.2-rc. Fixes bug 2190. + - Add a check for SIZE_T_MAX to tor_realloc() to try to avoid + underflow errors there too. Fixes the other part of bug 2324. + - Fix a bug where we would assert if we ever had a + cached-descriptors.new file (or another file read directly into + memory) of exactly SIZE_T_CEILING bytes. Fixes bug 2326; bugfix + on 0.2.1.25. Found by doorss. + - Fix some potential asserts and parsing issues with grossly + malformed router caches. Fixes bug 2352; bugfix on Tor 0.2.1.27. + Found by doorss. + + o Minor bugfixes (other), also included in 0.2.1.29: + - Fix a bug with handling misformed replies to reverse DNS lookup + requests in DNSPort. Bugfix on Tor 0.2.0.1-alpha. Related to a + bug reported by doorss. + - Fix compilation on mingw when a pthreads compatibility library + has been installed. (We don't want to use it, so we shouldn't + be including pthread.h.) Fixes bug 2313; bugfix on 0.1.0.1-rc. + - Fix a bug where we would declare that we had run out of virtual + addresses when the address space was only half-exhausted. Bugfix + on 0.1.2.1-alpha. + - Correctly handle the case where AutomapHostsOnResolve is set but + no virtual addresses are available. Fixes bug 2328; bugfix on + 0.1.2.1-alpha. Bug found by doorss. + - Correctly handle wrapping around when we run out of virtual + address space. Found by cypherpunks; bugfix on 0.2.0.5-alpha. + + o Minor features, also included in 0.2.1.29: + - Update to the January 1 2011 Maxmind GeoLite Country database. + - Introduce output size checks on all of our decryption functions. + + o Build changes, also included in 0.2.1.29: + - Tor does not build packages correctly with Automake 1.6 and earlier; + added a check to Makefile.am to make sure that we're building with + Automake 1.7 or later. + - The 0.2.1.28 tarball was missing src/common/OpenBSD_malloc_Linux.c + because we built it with a too-old version of automake. Thus that + release broke ./configure --enable-openbsd-malloc, which is popular + among really fast exit relays on Linux. + + o Major bugfixes, new in 0.2.2.21-alpha: + - Prevent crash/heap corruption when the cbtnummodes consensus + parameter is set to 0 or large values. Fixes bug 2317; bugfix + on 0.2.2.14-alpha. + + o Major features, new in 0.2.2.21-alpha: + - Introduce minimum/maximum values that clients will believe + from the consensus. Now we'll have a better chance to avoid crashes + or worse when a consensus param has a weird value. + + o Minor features, new in 0.2.2.21-alpha: + - Make sure to disable DirPort if running as a bridge. DirPorts aren't + used on bridges, and it makes bridge scanning somewhat easier. + - If writing the state file to disk fails, wait up to an hour before + retrying again, rather than trying again each second. Fixes bug + 2346; bugfix on Tor 0.1.1.3-alpha. + - Make Libevent log messages get delivered to controllers later, + and not from inside the Libevent log handler. This prevents unsafe + reentrant Libevent calls while still letting the log messages + get through. + - Detect platforms that brokenly use a signed size_t, and refuse to + build there. Found and analyzed by doorss and rransom. + - Fix a bunch of compile warnings revealed by mingw with gcc 4.5. + Resolves bug 2314. + + o Minor bugfixes, new in 0.2.2.21-alpha: + - Handle SOCKS messages longer than 128 bytes long correctly, rather + than waiting forever for them to finish. Fixes bug 2330; bugfix + on 0.2.0.16-alpha. Found by doorss. + - Add assertions to check for overflow in arguments to + base32_encode() and base32_decode(); fix a signed-unsigned + comparison there too. These bugs are not actually reachable in Tor, + but it's good to prevent future errors too. Found by doorss. + - Correctly detect failures to create DNS requests when using Libevent + versions before v2. (Before Libevent 2, we used our own evdns + implementation. Its return values for Libevent's evdns_resolve_*() + functions are not consistent with those from Libevent.) Fixes bug + 2363; bugfix on 0.2.2.6-alpha. Found by "lodger". + + o Documentation, new in 0.2.2.21-alpha: + - Document the default socks host and port (127.0.0.1:9050) for + tor-resolve. + + +Changes in version 0.2.1.29 - 2011-01-15 + Tor 0.2.1.29 continues our recent code security audit work. The main + fix resolves a remote heap overflow vulnerability that can allow remote + code execution. Other fixes address a variety of assert and crash bugs, + most of which we think are hard to exploit remotely. + + o Major bugfixes (security): + - Fix a heap overflow bug where an adversary could cause heap + corruption. This bug probably allows remote code execution + attacks. Reported by "debuger". Fixes CVE-2011-0427. Bugfix on + 0.1.2.10-rc. + - Prevent a denial-of-service attack by disallowing any + zlib-compressed data whose compression factor is implausibly + high. Fixes part of bug 2324; reported by "doorss". + - Zero out a few more keys in memory before freeing them. Fixes + bug 2384 and part of bug 2385. These key instances found by + "cypherpunks", based on Andrew Case's report about being able + to find sensitive data in Tor's memory space if you have enough + permissions. Bugfix on 0.0.2pre9. + + o Major bugfixes (crashes): + - Prevent calls to Libevent from inside Libevent log handlers. + This had potential to cause a nasty set of crashes, especially + if running Libevent with debug logging enabled, and running + Tor with a controller watching for low-severity log messages. + Bugfix on 0.1.0.2-rc. Fixes bug 2190. + - Add a check for SIZE_T_MAX to tor_realloc() to try to avoid + underflow errors there too. Fixes the other part of bug 2324. + - Fix a bug where we would assert if we ever had a + cached-descriptors.new file (or another file read directly into + memory) of exactly SIZE_T_CEILING bytes. Fixes bug 2326; bugfix + on 0.2.1.25. Found by doorss. + - Fix some potential asserts and parsing issues with grossly + malformed router caches. Fixes bug 2352; bugfix on Tor 0.2.1.27. + Found by doorss. + + o Minor bugfixes (other): + - Fix a bug with handling misformed replies to reverse DNS lookup + requests in DNSPort. Bugfix on Tor 0.2.0.1-alpha. Related to a + bug reported by doorss. + - Fix compilation on mingw when a pthreads compatibility library + has been installed. (We don't want to use it, so we shouldn't + be including pthread.h.) Fixes bug 2313; bugfix on 0.1.0.1-rc. + - Fix a bug where we would declare that we had run out of virtual + addresses when the address space was only half-exhausted. Bugfix + on 0.1.2.1-alpha. + - Correctly handle the case where AutomapHostsOnResolve is set but + no virtual addresses are available. Fixes bug 2328; bugfix on + 0.1.2.1-alpha. Bug found by doorss. + - Correctly handle wrapping around to when we run out of virtual + address space. Found by cypherpunks, bugfix on 0.2.0.5-alpha. + - The 0.2.1.28 tarball was missing src/common/OpenBSD_malloc_Linux.c + because we built it with a too-old version of automake. Thus that + release broke ./configure --enable-openbsd-malloc, which is popular + among really fast exit relays on Linux. + + o Minor features: + - Update to the January 1 2011 Maxmind GeoLite Country database. + - Introduce output size checks on all of our decryption functions. + + o Build changes: + - Tor does not build packages correctly with Automake 1.6 and earlier; + added a check to Makefile.am to make sure that we're building with + Automake 1.7 or later. + + +Changes in version 0.2.2.20-alpha - 2010-12-17 + Tor 0.2.2.20-alpha does some code cleanup to reduce the risk of remotely + exploitable bugs. We also fix a variety of other significant bugs, + change the IP address for one of our directory authorities, and update + the minimum version that Tor relays must run to join the network. + + o Major bugfixes: + - Fix a remotely exploitable bug that could be used to crash instances + of Tor remotely by overflowing on the heap. Remote-code execution + hasn't been confirmed, but can't be ruled out. Everyone should + upgrade. Bugfix on the 0.1.1 series and later. + - Fix a bug that could break accounting on 64-bit systems with large + time_t values, making them hibernate for impossibly long intervals. + Fixes bug 2146. Bugfix on 0.0.9pre6; fix by boboper. + - Fix a logic error in directory_fetches_from_authorities() that + would cause all _non_-exits refusing single-hop-like circuits + to fetch from authorities, when we wanted to have _exits_ fetch + from authorities. Fixes more of 2097. Bugfix on 0.2.2.16-alpha; + fix by boboper. + - Fix a stream fairness bug that would cause newer streams on a given + circuit to get preference when reading bytes from the origin or + destination. Fixes bug 2210. Fix by Mashael AlSabah. This bug was + introduced before the first Tor release, in svn revision r152. + + o Directory authority changes: + - Change IP address and ports for gabelmoo (v3 directory authority). + + o Minor bugfixes: + - Avoid crashes when AccountingMax is set on clients. Fixes bug 2235. + Bugfix on 0.2.2.18-alpha. Diagnosed by boboper. + - Fix an off-by-one error in calculating some controller command + argument lengths. Fortunately, this mistake is harmless since + the controller code does redundant NUL termination too. Found by + boboper. Bugfix on 0.1.1.1-alpha. + - Do not dereference NULL if a bridge fails to build its + extra-info descriptor. Found by an anonymous commenter on + Trac. Bugfix on 0.2.2.19-alpha. + + o Minor features: + - Update to the December 1 2010 Maxmind GeoLite Country database. + - Directory authorities now reject relays running any versions of + Tor between 0.2.1.3-alpha and 0.2.1.18 inclusive; they have + known bugs that keep RELAY_EARLY cells from working on rendezvous + circuits. Followup to fix for bug 2081. + - Directory authorities now reject relays running any version of Tor + older than 0.2.0.26-rc. That version is the earliest that fetches + current directory information correctly. Fixes bug 2156. + - Report only the top 10 ports in exit-port stats in order not to + exceed the maximum extra-info descriptor length of 50 KB. Implements + task 2196. + + +Changes in version 0.2.1.28 - 2010-12-17 + Tor 0.2.1.28 does some code cleanup to reduce the risk of remotely + exploitable bugs. We also took this opportunity to change the IP address + for one of our directory authorities, and to update the geoip database + we ship. + + o Major bugfixes: + - Fix a remotely exploitable bug that could be used to crash instances + of Tor remotely by overflowing on the heap. Remote-code execution + hasn't been confirmed, but can't be ruled out. Everyone should + upgrade. Bugfix on the 0.1.1 series and later. + + o Directory authority changes: + - Change IP address and ports for gabelmoo (v3 directory authority). + + o Minor features: + - Update to the December 1 2010 Maxmind GeoLite Country database. + + +Changes in version 0.2.1.27 - 2010-11-23 + Yet another OpenSSL security patch broke its compatibility with Tor: + Tor 0.2.1.27 makes relays work with openssl 0.9.8p and 1.0.0.b. We + also took this opportunity to fix several crash bugs, integrate a new + directory authority, and update the bundled GeoIP database. + + o Major bugfixes: + - Resolve an incompatibility with OpenSSL 0.9.8p and OpenSSL 1.0.0b: + No longer set the tlsext_host_name extension on server SSL objects; + but continue to set it on client SSL objects. Our goal in setting + it was to imitate a browser, not a vhosting server. Fixes bug 2204; + bugfix on 0.2.1.1-alpha. + - Do not log messages to the controller while shrinking buffer + freelists. Doing so would sometimes make the controller connection + try to allocate a buffer chunk, which would mess up the internals + of the freelist and cause an assertion failure. Fixes bug 1125; + fixed by Robert Ransom. Bugfix on 0.2.0.16-alpha. + - Learn our external IP address when we're a relay or bridge, even if + we set PublishServerDescriptor to 0. Bugfix on 0.2.0.3-alpha, + where we introduced bridge relays that don't need to publish to + be useful. Fixes bug 2050. + - Do even more to reject (and not just ignore) annotations on + router descriptors received anywhere but from the cache. Previously + we would ignore such annotations at first, but cache them to disk + anyway. Bugfix on 0.2.0.8-alpha. Found by piebeer. + - When you're using bridges and your network goes away and your + bridges get marked as down, recover when you attempt a new socks + connection (if the network is back), rather than waiting up to an + hour to try fetching new descriptors for your bridges. Bugfix on + 0.2.0.3-alpha; fixes bug 1981. + + o Major features: + - Move to the November 2010 Maxmind GeoLite country db (rather + than the June 2009 ip-to-country GeoIP db) for our statistics that + count how many users relays are seeing from each country. Now we'll + have more accurate data, especially for many African countries. + + o New directory authorities: + - Set up maatuska (run by Linus Nordberg) as the eighth v3 directory + authority. + + o Minor bugfixes: + - Fix an assertion failure that could occur in directory caches or + bridge users when using a very short voting interval on a testing + network. Diagnosed by Robert Hogan. Fixes bug 1141; bugfix on + 0.2.0.8-alpha. + - Enforce multiplicity rules when parsing annotations. Bugfix on + 0.2.0.8-alpha. Found by piebeer. + - Allow handshaking OR connections to take a full KeepalivePeriod + seconds to handshake. Previously, we would close them after + IDLE_OR_CONN_TIMEOUT (180) seconds, the same timeout as if they + were open. Bugfix on 0.2.1.26; fixes bug 1840. Thanks to mingw-san + for analysis help. + - When building with --enable-gcc-warnings on OpenBSD, disable + warnings in system headers. This makes --enable-gcc-warnings + pass on OpenBSD 4.8. + + o Minor features: + - Exit nodes didn't recognize EHOSTUNREACH as a plausible error code, + and so sent back END_STREAM_REASON_MISC. Clients now recognize a new + stream ending reason for this case: END_STREAM_REASON_NOROUTE. + Servers can start sending this code when enough clients recognize + it. Bugfix on 0.1.0.1-rc; fixes part of bug 1793. + - Build correctly on mingw with more recent versions of OpenSSL 0.9.8. + Patch from mingw-san. + + o Removed files: + - Remove the old debian/ directory from the main Tor distribution. + The official Tor-for-debian git repository lives at the URL + https://git.torproject.org/debian/tor.git + - Stop shipping the old doc/website/ directory in the tarball. We + changed the website format in late 2010, and what we shipped in + 0.2.1.26 really wasn't that useful anyway. + + Changes in version 0.2.2.19-alpha - 2010-11-22 Yet another OpenSSL security patch broke its compatibility with Tor: Tor 0.2.2.19-alpha makes relays work with OpenSSL 0.9.8p and 1.0.0.b. @@ -543,9 +4344,10 @@ Changes in version 0.2.2.14-alpha - 2010-07-12 o Minor features: - New config option "WarnUnsafeSocks 0" disables the warning that - occurs whenever Tor receives only an IP address instead of a - hostname. Setups that do DNS locally over Tor are fine, and we - shouldn't spam the logs in that case. + occurs whenever Tor receives a socks handshake using a version of + the socks protocol that can only provide an IP address (rather + than a hostname). Setups that do DNS locally over Tor are fine, + and we shouldn't spam the logs in that case. - Convert the HACKING file to asciidoc, and add a few new sections to it, explaining how we use Git, how we make changelogs, and what should go in a patch. @@ -1307,8 +5109,9 @@ Changes in version 0.2.2.5-alpha - 2009-10-11 o Major bugfixes: - Make the tarball compile again. Oops. Bugfix on 0.2.2.4-alpha. - o New directory authorities: - - Move dizum to an alternate IP address. + o Directory authorities: + - Temporarily (just for this release) move dizum to an alternate + IP address. Changes in version 0.2.2.4-alpha - 2009-10-10 @@ -1478,8 +5281,8 @@ Changes in version 0.2.2.1-alpha - 2009-08-26 oldest-bug prize. o New options for gathering stats safely: - - Directories that set "DirReqStatistics 1" write statistics on - directory request to disk every 24 hours. As compared to the + - Directory mirrors that set "DirReqStatistics 1" write statistics + about directory requests to disk every 24 hours. As compared to the --enable-geoip-stats flag in 0.2.1.x, there are a few improvements: 1) stats are written to disk exactly every 24 hours; 2) estimated shares of v2 and v3 requests are determined as mean values, not at @@ -1517,9 +5320,9 @@ Changes in version 0.2.2.1-alpha - 2009-08-26 the git commit (when we're building from a git checkout). o Minor bugfixes: - - If any the v3 certs we download are unparseable, we should actually - notice the failure so we don't retry indefinitely. Bugfix on - 0.2.0.x; reported by "rotator". + - If any of the v3 certs we download are unparseable, we should + actually notice the failure so we don't retry indefinitely. Bugfix + on 0.2.0.x; reported by "rotator". - If the cached cert file is unparseable, warn but don't exit. - Fix possible segmentation fault on directory authorities. Bugfix on 0.2.1.14-rc. @@ -2306,7 +6109,7 @@ Changes in version 0.2.1.9-alpha - 2008-12-25 - When a download gets us zero good descriptors, do not notify Tor that new directory information has arrived. - Avoid some nasty corner cases in the logic for marking connections - as too old or obsolete or noncanonical for circuits. Partial + as too old or obsolete or noncanonical for circuits. Partial bugfix on bug 891. o Minor features (controller): @@ -3378,7 +7181,7 @@ Changes in version 0.2.0.22-rc - 2008-03-18 bugs. Bug found by lodger. Bugfix on 0.2.0.16-alpha. - Fix the implementation of ClientDNSRejectInternalAddresses so that it actually works, and doesn't warn about every single reverse lookup. - Fixes the other part of bug 617. Bugfix on 0.2.0.1-alpha. + Fixes the other part of bug 617. Bugfix on 0.2.0.1-alpha. o Minor features: - Only log guard node status when guard node status has changed. @@ -3561,7 +7364,7 @@ Changes in version 0.2.0.19-alpha - 2008-02-09 bandwidthburst values. o Minor features (controller): - - Reject controller commands over 1MB in length. This keeps rogue + - Reject controller commands over 1MB in length. This keeps rogue processes from running us out of memory. o Minor features (misc): @@ -3605,7 +7408,7 @@ Changes in version 0.2.0.19-alpha - 2008-02-09 - Fix shell error when warning about missing packages in configure script, on Fedora or Red Hat machines. Bugfix on 0.2.0.x. - Do not become confused when receiving a spurious VERSIONS-like - cell from a confused v1 client. Bugfix on 0.2.0.x. + cell from a confused v1 client. Bugfix on 0.2.0.x. - Re-fetch v2 (as well as v0) rendezvous descriptors when all introduction points for a hidden service have failed. Patch from Karsten Loesing. Bugfix on 0.2.0.x. @@ -3893,8 +7696,8 @@ Changes in version 0.2.0.14-alpha - 2007-12-23 - Make PublishServerDescriptor default to 1, so the default doesn't have to change as we invent new directory protocol versions. - Fix test for rlim_t on OSX 10.3: sys/resource.h doesn't want to - be included unless sys/time.h is already included. Fixes - bug 553. Bugfix on 0.2.0.x. + be included unless sys/time.h is already included. Fixes + bug 553. Bugfix on 0.2.0.x. - If we receive a general-purpose descriptor and then receive an identical bridge-purpose descriptor soon after, don't discard the next one as a duplicate. @@ -4001,7 +7804,7 @@ Changes in version 0.2.0.13-alpha - 2007-12-21 in really weird results on platforms whose sys/types.h files define nonstandard integer types. Bugfix on 0.1.2.x. - Fix compilation with --disable-threads set. Bugfix on 0.2.0.x. - - Don't crash on name lookup when we have no current consensus. Fixes + - Don't crash on name lookup when we have no current consensus. Fixes bug 538; bugfix on 0.2.0.x. - Only Tors that want to mirror the v2 directory info should create the "cached-status" directory in their datadir. (All Tors @@ -4185,7 +7988,7 @@ Changes in version 0.2.0.10-alpha - 2007-11-10 - Use "If-Modified-Since" to avoid retrieving consensus networkstatuses that we already have. - When we have no consensus, check FallbackNetworkstatusFile (defaults - to $PREFIX/share/tor/fallback-consensus) for a consensus. This way + to $PREFIX/share/tor/fallback-consensus) for a consensus. This way we start knowing some directory caches. - When we receive a consensus from the future, warn about skew. - Improve skew reporting: try to give the user a better log message @@ -4195,7 +7998,7 @@ Changes in version 0.2.0.10-alpha - 2007-11-10 - New --quiet command-line option to suppress the default console log. Good in combination with --hash-password. - Authorities send back an X-Descriptor-Not-New header in response to - an accepted-but-discarded descriptor upload. Partially implements + an accepted-but-discarded descriptor upload. Partially implements fix for bug 535. - Make the log message for "tls error. breaking." more useful. - Better log messages about certificate downloads, to attempt to @@ -4232,15 +8035,15 @@ Changes in version 0.2.0.10-alpha - 2007-11-10 on 0.2.0.1-alpha. - Stop leaking memory every time we load a v3 certificate. Bugfix on 0.2.0.1-alpha. Fixes bug 536. - - Stop leaking a cached networkstatus on exit. Bugfix on + - Stop leaking a cached networkstatus on exit. Bugfix on 0.2.0.3-alpha. - Stop leaking voter information every time we free a consensus. Bugfix on 0.2.0.3-alpha. - Stop leaking signed data every time we check a voter signature. Bugfix on 0.2.0.3-alpha. - Stop leaking a signature every time we fail to parse a consensus or - a vote. Bugfix on 0.2.0.3-alpha. - - Stop leaking v2_download_status_map on shutdown. Bugfix on + a vote. Bugfix on 0.2.0.3-alpha. + - Stop leaking v2_download_status_map on shutdown. Bugfix on 0.2.0.9-alpha. - Stop leaking conn->nickname every time we make a connection to a Tor relay without knowing its expected identity digest (e.g. when @@ -4371,7 +8174,7 @@ Changes in version 0.2.0.9-alpha - 2007-10-24 - Reattempt certificate downloads immediately on failure, as long as we haven't failed a threshold number of times yet. - Delay retrying consensus downloads while we're downloading - certificates to verify the one we just got. Also, count getting a + certificates to verify the one we just got. Also, count getting a consensus that we already have (or one that isn't valid) as a failure, and count failing to get the certificates after 20 minutes as a failure. @@ -4463,13 +8266,13 @@ Changes in version 0.2.0.9-alpha - 2007-10-24 yet. Bug found by spending four hours without a v3 consensus. Bugfix on 0.1.2.x. - Detect the reason for failing to mmap a descriptor file we just - wrote, and give a more useful log message. Fixes bug 533. Bugfix + wrote, and give a more useful log message. Fixes bug 533. Bugfix on 0.1.2.x. o Code simplifications and refactoring: - Remove support for the old bw_accounting file: we've been storing bandwidth accounting information in the state file since - 0.1.2.5-alpha. This may result in bandwidth accounting errors + 0.1.2.5-alpha. This may result in bandwidth accounting errors if you try to upgrade from 0.1.1.x or earlier, or if you try to downgrade to 0.1.1.x or earlier. - New convenience code to locate a file within the DataDirectory. @@ -4509,9 +8312,9 @@ Changes in version 0.2.0.8-alpha - 2007-10-12 - When a v3 authority is missing votes or signatures, it now tries to fetch them. - Directory authorities track weighted fractional uptime as well as - weighted mean-time-between failures. WFU is suitable for deciding + weighted mean-time-between failures. WFU is suitable for deciding whether a node is "usually up", while MTBF is suitable for deciding - whether a node is "likely to stay up." We need both, because + whether a node is "likely to stay up." We need both, because "usually up" is a good requirement for guards, while "likely to stay up" is a good requirement for long-lived connections. @@ -4932,7 +8735,7 @@ Changes in version 0.2.0.5-alpha - 2007-08-19 - If we require CookieAuthentication, stop generating a new cookie every time we change any piece of our config. - When loading bandwidth history, do not believe any information in - the future. Fixes bug 434. + the future. Fixes bug 434. - When loading entry guard information, do not believe any information in the future. - When we have our clock set far in the future and generate an @@ -5237,7 +9040,7 @@ Changes in version 0.2.0.1-alpha - 2007-06-01 - Change the way that Tor buffers data that it is waiting to write. Instead of queueing data cells in an enormous ring buffer for each client->OR or OR->OR connection, we now queue cells on a separate - queue for each circuit. This lets us use less slack memory, and + queue for each circuit. This lets us use less slack memory, and will eventually let us be smarter about prioritizing different kinds of traffic. - Use memory pools to allocate cells with better speed and memory @@ -5297,7 +9100,7 @@ Changes in version 0.2.0.1-alpha - 2007-06-01 authorities should set. This will let future authorities choose not to serve V2 directory information. - Directory authorities allow multiple router descriptors and/or extra - info documents to be uploaded in a single go. This will make + info documents to be uploaded in a single go. This will make implementing proposal 104 simpler. o Minor features (controller): @@ -5314,15 +9117,15 @@ Changes in version 0.2.0.1-alpha - 2007-06-01 o Minor features (hidden services): - Allow multiple HiddenServicePort directives with the same virtual port; when they occur, the user is sent round-robin to one - of the target ports chosen at random. Partially fixes bug 393 by + of the target ports chosen at random. Partially fixes bug 393 by adding limited ad-hoc round-robining. o Minor features (other): - More unit tests. - Add a new AutomapHostsOnResolve option: when it is enabled, any resolve request for hosts matching a given pattern causes Tor to - generate an internal virtual address mapping for that host. This - allows DNSPort to work sensibly with hidden service users. By + generate an internal virtual address mapping for that host. This + allows DNSPort to work sensibly with hidden service users. By default, .exit and .onion addresses are remapped; the list of patterns can be reconfigured with AutomapHostsSuffixes. - Add an "-F" option to tor-resolve to force a resolve for a .onion @@ -5336,7 +9139,7 @@ Changes in version 0.2.0.1-alpha - 2007-06-01 o Removed features: - Removed support for the old binary "version 0" controller protocol. This has been deprecated since 0.1.1, and warnings have been issued - since 0.1.2. When we encounter a v0 control message, we now send + since 0.1.2. When we encounter a v0 control message, we now send back an error and close the connection. - Remove the old "dns worker" server DNS code: it hasn't been default since 0.1.2.2-alpha, and all the servers seem to be using the new @@ -5359,7 +9162,7 @@ Changes in version 0.2.0.1-alpha - 2007-06-01 o Minor bugfixes (logging): - When we hit an EOF on a log (probably because we're shutting down), don't try to remove the log from the list: just mark it as - unusable. (Bulletproofs against bug 222.) + unusable. (Bulletproofs against bug 222.) o Minor bugfixes (other): - In the exitlist script, only consider the most recently published @@ -5369,7 +9172,7 @@ Changes in version 0.2.0.1-alpha - 2007-06-01 connections to that address. (Resolves bug 405.) - Stop allowing hibernating servers to be "stable" or "fast". - On Windows, we were preventing other processes from reading - cached-routers while Tor was running. (Reported by janbar) + cached-routers while Tor was running. (Reported by janbar) - Make the NodeFamilies config option work. (Reported by lodger -- it has never actually worked, even though we added it in Oct 2004.) @@ -5542,10 +9345,10 @@ Changes in version 0.1.2.9-rc - 2007-03-02 o Minor bugfixes (other): - Fix an assert that could trigger if a controller quickly set then - cleared EntryNodes. (Bug found by Udo van den Heuvel.) + cleared EntryNodes. Bug found by Udo van den Heuvel. - On architectures where sizeof(int)>4, still clamp declarable bandwidth to INT32_MAX. - - Fix a potential race condition in the rpm installer. Found by + - Fix a potential race condition in the rpm installer. Found by Stefan Nordhausen. - Try to fix eventdns warnings once and for all: do not treat a dns rcode of 2 as indicating that the server is completely bad; it sometimes @@ -5578,8 +9381,8 @@ Changes in version 0.1.2.8-beta - 2007-02-26 o Minor bugfixes (controller): - Give the controller END_STREAM_REASON_DESTROY events _before_ we clear the corresponding on_circuit variable, and remember later - that we don't need to send a redundant CLOSED event. (Resolves part - 3 of bug 367.) + that we don't need to send a redundant CLOSED event. Resolves part + 3 of bug 367. - Report events where a resolve succeeded or where we got a socks protocol error correctly, rather than calling both of them "INTERNAL". @@ -5613,7 +9416,7 @@ Changes in version 0.1.2.8-beta - 2007-02-26 tor_munmap() for systems with no mmap() call. - When Tor receives a router descriptor that it asked for, but no longer wants (because it has received fresh networkstatuses - in the meantime), do not warn the user. Cache the descriptor if + in the meantime), do not warn the user. Cache the descriptor if we're a cache; drop it if we aren't. - Make earlier entry guards _really_ get retried when the network comes back online. @@ -5624,7 +9427,7 @@ Changes in version 0.1.2.8-beta - 2007-02-26 o Minor features (controller): - Warn the user when an application uses the obsolete binary v0 - control protocol. We're planning to remove support for it during + control protocol. We're planning to remove support for it during the next development series, so it's good to give people some advance warning. - Add STREAM_BW events to report per-entry-stream bandwidth @@ -5644,7 +9447,7 @@ Changes in version 0.1.2.8-beta - 2007-02-26 (This is showing up in some profiles, but not others.) o Minor features: - - Remove some never-implemented options. Mark PathlenCoinWeight as + - Remove some never-implemented options. Mark PathlenCoinWeight as obsolete. - Implement proposal 106: Stop requiring clients to have well-formed certificates; stop checking nicknames in certificates. (Clients @@ -5685,10 +9488,10 @@ Changes in version 0.1.2.7-alpha - 2007-02-06 o Major bugfixes (NT services): - Install as NT_AUTHORITY\LocalService rather than as SYSTEM; add a command-line flag so that admins can override the default by saying - "tor --service install --user "SomeUser"". This will not affect - existing installed services. Also, warn the user that the service + "tor --service install --user "SomeUser"". This will not affect + existing installed services. Also, warn the user that the service will look for its configuration file in the service user's - %appdata% directory. (We can't do the 'hardwire the user's appdata + %appdata% directory. (We can't do the 'hardwire the user's appdata directory' trick any more, since we may not have read access to that directory.) @@ -5735,7 +9538,7 @@ Changes in version 0.1.2.7-alpha - 2007-02-06 "DNS resolve socks failed" handshake reply; just close it. - Stop using C functions that OpenBSD's linker doesn't like. - Don't launch requests for descriptors unless we have networkstatuses - from at least half of the authorities. This delays the first + from at least half of the authorities. This delays the first download slightly under pathological circumstances, but can prevent us from downloading a bunch of descriptors we don't need. - Do not log IPs with TLS failures for incoming TLS @@ -6113,7 +9916,7 @@ Changes in version 0.1.2.3-alpha - 2006-10-29 o Minor features, controller: - Add a REASON field to CIRC events; for backward compatibility, this field is sent only to controllers that have enabled the extended - event format. Also, add additional reason codes to explain why + event format. Also, add additional reason codes to explain why a given circuit has been destroyed or truncated. (Patches from Mike Perry) - Add a REMOTE_REASON field to extended CIRC events to tell the @@ -6142,14 +9945,14 @@ Changes in version 0.1.2.3-alpha - 2006-10-29 - When the controller does a "GETINFO network-status", tell it about even those routers whose descriptors are very old, and use long nicknames where appropriate. - - Change NT service functions to be loaded on demand. This lets us + - Change NT service functions to be loaded on demand. This lets us build with MinGW without breaking Tor for Windows 98 users. - Do DirPort reachability tests less often, since a single test chews through many circuits before giving up. - In the hidden service example in torrc.sample, stop recommending esoteric and discouraged hidden service options. - When stopping an NT service, wait up to 10 sec for it to actually - stop. (Patch from Matt Edman; resolves bug 295.) + stop. Patch from Matt Edman; resolves bug 295. - Fix handling of verbose nicknames with ORCONN controller events: make them show up exactly when requested, rather than exactly when not requested. @@ -6805,7 +10608,7 @@ Changes in version 0.1.1.15-rc - 2006-03-11 - Add a new circuit purpose 'controller' to let the controller ask for a circuit that Tor won't try to use. Extend the EXTENDCIRCUIT controller command to let you specify the purpose if you're - starting a new circuit. Add a new SETCIRCUITPURPOSE controller + starting a new circuit. Add a new SETCIRCUITPURPOSE controller command to let you change a circuit's purpose after it's been created. - Accept "private:*" in routerdesc exit policies; not generated yet @@ -6849,7 +10652,7 @@ Changes in version 0.1.1.14-alpha - 2006-02-20 be forward-compatible. - Generate 18.0.0.0/8 address policy format in descs when we can; warn when the mask is not reducible to a bit-prefix. - - Let the user set ControlListenAddress in the torrc. This can be + - Let the user set ControlListenAddress in the torrc. This can be dangerous, but there are some cases (like a secured LAN) where it makes sense. - Split ReachableAddresses into ReachableDirAddresses and @@ -7308,7 +11111,7 @@ Changes in version 0.1.1.9-alpha - 2005-11-15 - Start making directory caches retain old routerinfos, so soon clients can start asking by digest of descriptor rather than by fingerprint of server. - - Add half our entropy from RAND_poll in OpenSSL. This knows how + - Add half our entropy from RAND_poll in OpenSSL. This knows how to use egd (if present), openbsd weirdness (if present), vms/os2 weirdness (if we ever port there), and more in the future. @@ -8154,7 +11957,7 @@ Changes in version 0.1.0.1-rc - 2005-03-28 o Robustness/stability fixes: - Make Tor use Niels Provos's libevent instead of its current - poll-but-sometimes-select mess. This will let us use faster async + poll-but-sometimes-select mess. This will let us use faster async cores (like epoll, kpoll, and /dev/poll), and hopefully work better on Windows too. - pthread support now too. This was forced because when we forked, @@ -8272,7 +12075,7 @@ Changes in version 0.1.0.1-rc - 2005-03-28 Changes in version 0.0.9.6 - 2005-03-24 o Bugfixes on 0.0.9.x (crashes and asserts): - Add new end stream reasons to maintainance branch. Fix bug where - reason (8) could trigger an assert. Prevent bug from recurring. + reason (8) could trigger an assert. Prevent bug from recurring. - Apparently win32 stat wants paths to not end with a slash. - Fix assert triggers in assert_cpath_layer_ok(), where we were blowing away the circuit that conn->cpath_layer points to, then @@ -8495,7 +12298,7 @@ Changes in version 0.0.9rc5 - 2004-12-01 but doesn't seem to be currently; thanks to Ilja van Sprundel for finding it. - If anybody set DirFetchPostPeriod, give them StatusFetchPeriod - instead. Impose minima and maxima for all *Period options; impose + instead. Impose minima and maxima for all *Period options; impose even tighter maxima for fetching if we are a caching dirserver. Clip rather than rejecting. - Fetch cached running-routers from servers that serve it (that is, @@ -9476,7 +13279,7 @@ Changes in version 0.0.2pre20 - 2004-01-30 - I've split the TotalBandwidth option into BandwidthRate (how many bytes per second you want to allow, long-term) and BandwidthBurst (how many bytes you will allow at once before the cap - kicks in). This better token bucket approach lets you, say, set + kicks in). This better token bucket approach lets you, say, set BandwidthRate to 10KB/s and BandwidthBurst to 10MB, allowing good performance while not exceeding your monthly bandwidth quota. - Push out a tls record's worth of data once you've got it, rather @@ -3,7 +3,7 @@ Most users who realize that INSTALL files still exist should simply follow the directions at https://www.torproject.org/docs/tor-doc-unix -If you got the source from Subversion, run "./autogen.sh", which will +If you got the source from git, run "./autogen.sh", which will run the various auto* programs. Then you can run ./configure, and refer to the above instructions. @@ -21,5 +21,32 @@ If it doesn't build for you: For example, "setenv LD_LIBRARY_PATH /usr/athena/lib". Lastly, check out - http://wiki.noreply.org/noreply/TheOnionRouter/TorFAQ#ItDoesntWork + https://www.torproject.org/docs/faq#DoesntWork + +How to do static builds of tor: + +Tor supports linking each of the libraries it needs statically. Use the +--enable-static-X ./configure option in conjunction with the --with-X-dir +option for libevent, zlib, and openssl. For this to work sanely, libevent +should be built with --disable-shared --enable-static --with-pic, and +OpenSSL should be built with no-shared no-dso. + +If you need to build tor so that system libraries are also statically linked, +use the --enable-static-tor ./configure option. This won't work on OS X +unless you build the required crt0.o yourself. It is also incompatible with +the --enable-gcc-hardening option. + +An example of how to build a mostly static tor: +./configure --enable-static-libevent \ + --enable-static-openssl \ + --enable-static-zlib \ + --with-libevent-dir=/tmp/static-tor/libevent-1.4.14b-stable \ + --with-openssl-dir=/tmp/static-tor/openssl-0.9.8r/ \ + --with-zlib-dir=/tmp/static-tor/zlib-1.2.5 + +An example of how to build an entirely static tor: +./configure --enable-static-tor \ + --with-libevent-dir=/tmp/static-tor/libevent-1.4.14b-stable \ + --with-openssl-dir=/tmp/static-tor/openssl-0.9.8r/ \ + --with-zlib-dir=/tmp/static-tor/zlib-1.2.5 @@ -13,7 +13,7 @@ Tor is distributed under this license: Copyright (c) 2001-2004, Roger Dingledine Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson -Copyright (c) 2007-2011, The Tor Project, Inc. +Copyright (c) 2007-2012, The Tor Project, Inc. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are diff --git a/Makefile.am b/Makefile.am index 5f15926183..29bba715bd 100644 --- a/Makefile.am +++ b/Makefile.am @@ -3,36 +3,43 @@ # Copyright (c) 2007-2011, The Tor Project, Inc. # See LICENSE for licensing information +# "foreign" means we don't follow GNU package layout standards +# 1.7 means we require automake vesion 1.7 AUTOMAKE_OPTIONS = foreign 1.7 - # else it keeps trying to put COPYING back in SUBDIRS = src doc contrib DIST_SUBDIRS = src doc contrib -EXTRA_DIST = INSTALL README LICENSE ChangeLog \ - ReleaseNotes tor.spec tor.spec.in +EXTRA_DIST = \ + ChangeLog \ + INSTALL \ + LICENSE \ + Makefile.nmake \ + README \ + ReleaseNotes \ + tor.spec \ + tor.spec.in #install-data-local: # $(INSTALL) -m 755 -d $(LOCALSTATEDIR)/lib/tor -# Assume a tarball is in .. for now. -dist-rpm: - RPM_BUILD_DIR="/tmp/tor-rpm-build-$$$$"; \ - rm -rf $$RPM_BUILD_DIR; \ - mkdir $$RPM_BUILD_DIR || exit 1; \ - for subdir in BUILD RPMS SOURCES SPECS SRPMS; do \ - mkdir $$RPM_BUILD_DIR/$$subdir; \ - done; \ - mkdir $$RPM_BUILD_DIR/SOURCES/tor-$(VERSION); \ - cp -R ./ $$RPM_BUILD_DIR/SOURCES/tor-$(VERSION)/; \ - pushd $$RPM_BUILD_DIR/SOURCES/; \ - tar zcf tor-$(VERSION).tar.gz ./; \ - popd; \ - LIBS=-lrt rpmbuild -ba --define "_topdir $$RPM_BUILD_DIR" tor.spec; \ - mv $$RPM_BUILD_DIR/SRPMS/* .; \ - mv $$RPM_BUILD_DIR/RPMS/* .; \ - rm -rf $$RPM_BUILD_DIR +# Allows to override rpmbuild with rpmbuild-md5 from fedora-packager so that +# building for EL5 won't fail on https://bugzilla.redhat.com/show_bug.cgi?id=490613 +RPMBUILD ?= rpmbuild + +# Use automake's dist-gzip target to build the tarball +dist-rpm: dist-gzip + TIMESTAMP=$$(date +"%Y-%m-%d_%H.%M.%S"); \ + RPM_BUILD_DIR=$$(mktemp -d "/tmp/tor-rpm-build-$$TIMESTAMP-XXXX"); \ + mkdir -p "$$RPM_BUILD_DIR"/{BUILD,RPMS,SOURCES/"tor-$(VERSION)",SPECS,SRPMS}; \ + cp -fa "$(distdir).tar.gz" "$$RPM_BUILD_DIR"/SOURCES/; \ + LIBS=-lrt $(RPMBUILD) -ba --define "_topdir $$RPM_BUILD_DIR" tor.spec; \ + cp -fa "$$RPM_BUILD_DIR"/SRPMS/* .; \ + cp -fa "$$RPM_BUILD_DIR"/RPMS/* .; \ + rm -rf "$$RPM_BUILD_DIR"; \ + echo "RPM build finished"; \ + #end of dist-rpm dist: check @@ -46,10 +53,15 @@ test: all # eventdns.[hc], tinytest*.[ch] check-spaces: ./contrib/checkSpace.pl -C \ - src/common/*.h \ - src/common/[^asO]*.c src/common/address.c \ - src/or/[^e]*.[ch] src/or/eventdns_tor.h \ - src/test/test*.[ch] src/tools/*.[ch] + src/common/*.h \ + src/common/[^asO]*.c \ + src/common/address.c \ + src/or/[^e]*.[ch] \ + src/or/eventdns_tor.h \ + src/test/test*.[ch] \ + src/test/[^t]*.[ch] \ + src/tools/*.[ch] \ + src/tools/tor-fw-helper/*.[ch] check-docs: ./contrib/checkOptionDocs.pl @@ -58,3 +70,10 @@ check-logs: ./contrib/checkLogs.pl \ src/*/*.[ch] | sort -n +version: + @echo "Tor @VERSION@" + @if test -d "$(top_srcdir)/.git" && test -x "`which git 2>&1;true`"; then \ + echo -n "git: " ;\ + (cd "$(top_srcdir)" && git rev-parse --short=16 HEAD); \ + fi + diff --git a/Makefile.nmake b/Makefile.nmake new file mode 100644 index 0000000000..425f1ec262 --- /dev/null +++ b/Makefile.nmake @@ -0,0 +1,5 @@ +all:
+ cd src/common
+ $(MAKE) /F Makefile.nmake
+ cd ../../src/or
+ $(MAKE) /F Makefile.nmake
diff --git a/ReleaseNotes b/ReleaseNotes index 7ba473e907..93e38faf3f 100644 --- a/ReleaseNotes +++ b/ReleaseNotes @@ -3,6 +3,1789 @@ This document summarizes new features and bugfixes in each stable release of Tor. If you want to see more detailed descriptions of the changes in each development snapshot, see the ChangeLog file. +Changes in version 0.2.2.37 - 2012-06-06 + Tor 0.2.2.37 introduces a workaround for a critical renegotiation + bug in OpenSSL 1.0.1 (where 20% of the Tor network can't talk to itself + currently). + + o Major bugfixes: + - Work around a bug in OpenSSL that broke renegotiation with TLS + 1.1 and TLS 1.2. Without this workaround, all attempts to speak + the v2 Tor connection protocol when both sides were using OpenSSL + 1.0.1 would fail. Resolves ticket 6033. + - When waiting for a client to renegotiate, don't allow it to add + any bytes to the input buffer. This fixes a potential DoS issue. + Fixes bugs 5934 and 6007; bugfix on 0.2.0.20-rc. + - Fix an edge case where if we fetch or publish a hidden service + descriptor, we might build a 4-hop circuit and then use that circuit + for exiting afterwards -- even if the new last hop doesn't obey our + ExitNodes config option. Fixes bug 5283; bugfix on 0.2.0.10-alpha. + + o Minor bugfixes: + - Fix a build warning with Clang 3.1 related to our use of vasprintf. + Fixes bug 5969. Bugfix on 0.2.2.11-alpha. + + o Minor features: + - Tell GCC and Clang to check for any errors in format strings passed + to the tor_v*(print|scan)f functions. + + +Changes in version 0.2.2.36 - 2012-05-24 + Tor 0.2.2.36 updates the addresses for two of the eight directory + authorities, fixes some potential anonymity and security issues, + and fixes several crash bugs. + + Tor 0.2.1.x has reached its end-of-life. Those Tor versions have many + known flaws, and nobody should be using them. You should upgrade. If + you're using a Linux or BSD and its packages are obsolete, stop using + those packages and upgrade anyway. + + o Directory authority changes: + - Change IP address for maatuska (v3 directory authority). + - Change IP address for ides (v3 directory authority), and rename + it to turtles. + + o Security fixes: + - When building or running with any version of OpenSSL earlier + than 0.9.8s or 1.0.0f, disable SSLv3 support. These OpenSSL + versions have a bug (CVE-2011-4576) in which their block cipher + padding includes uninitialized data, potentially leaking sensitive + information to any peer with whom they make a SSLv3 connection. Tor + does not use SSL v3 by default, but a hostile client or server + could force an SSLv3 connection in order to gain information that + they shouldn't have been able to get. The best solution here is to + upgrade to OpenSSL 0.9.8s or 1.0.0f (or later). But when building + or running with a non-upgraded OpenSSL, we disable SSLv3 entirely + to make sure that the bug can't happen. + - Never use a bridge or a controller-supplied node as an exit, even + if its exit policy allows it. Found by wanoskarnet. Fixes bug + 5342. Bugfix on 0.1.1.15-rc (for controller-purpose descriptors) + and 0.2.0.3-alpha (for bridge-purpose descriptors). + - Only build circuits if we have a sufficient threshold of the total + descriptors that are marked in the consensus with the "Exit" + flag. This mitigates an attack proposed by wanoskarnet, in which + all of a client's bridges collude to restrict the exit nodes that + the client knows about. Fixes bug 5343. + - Provide controllers with a safer way to implement the cookie + authentication mechanism. With the old method, if another locally + running program could convince a controller that it was the Tor + process, then that program could trick the controller into telling + it the contents of an arbitrary 32-byte file. The new "SAFECOOKIE" + authentication method uses a challenge-response approach to prevent + this attack. Fixes bug 5185; implements proposal 193. + + o Major bugfixes: + - Avoid logging uninitialized data when unable to decode a hidden + service descriptor cookie. Fixes bug 5647; bugfix on 0.2.1.5-alpha. + - Avoid a client-side assertion failure when receiving an INTRODUCE2 + cell on a general purpose circuit. Fixes bug 5644; bugfix on + 0.2.1.6-alpha. + - Fix builds when the path to sed, openssl, or sha1sum contains + spaces, which is pretty common on Windows. Fixes bug 5065; bugfix + on 0.2.2.1-alpha. + - Correct our replacements for the timeradd() and timersub() functions + on platforms that lack them (for example, Windows). The timersub() + function is used when expiring circuits, while timeradd() is + currently unused. Bug report and patch by Vektor. Fixes bug 4778; + bugfix on 0.2.2.24-alpha. + - Fix the SOCKET_OK test that we use to tell when socket + creation fails so that it works on Win64. Fixes part of bug 4533; + bugfix on 0.2.2.29-beta. Bug found by wanoskarnet. + + o Minor bugfixes: + - Reject out-of-range times like 23:59:61 in parse_rfc1123_time(). + Fixes bug 5346; bugfix on 0.0.8pre3. + - Make our number-parsing functions always treat too-large values + as an error, even when those values exceed the width of the + underlying type. Previously, if the caller provided these + functions with minima or maxima set to the extreme values of the + underlying integer type, these functions would return those + values on overflow rather than treating overflow as an error. + Fixes part of bug 5786; bugfix on 0.0.9. + - Older Linux kernels erroneously respond to strange nmap behavior + by having accept() return successfully with a zero-length + socket. When this happens, just close the connection. Previously, + we would try harder to learn the remote address: but there was + no such remote address to learn, and our method for trying to + learn it was incorrect. Fixes bugs 1240, 4745, and 4747. Bugfix + on 0.1.0.3-rc. Reported and diagnosed by "r1eo". + - Correct parsing of certain date types in parse_http_time(). + Without this patch, If-Modified-Since would behave + incorrectly. Fixes bug 5346; bugfix on 0.2.0.2-alpha. Patch from + Esteban Manchado Velázques. + - Change the BridgePassword feature (part of the "bridge community" + design, which is not yet implemented) to use a time-independent + comparison. The old behavior might have allowed an adversary + to use timing to guess the BridgePassword value. Fixes bug 5543; + bugfix on 0.2.0.14-alpha. + - Detect and reject certain misformed escape sequences in + configuration values. Previously, these values would cause us + to crash if received in a torrc file or over an authenticated + control port. Bug found by Esteban Manchado Velázquez, and + independently by Robert Connolly from Matta Consulting who further + noted that it allows a post-authentication heap overflow. Patch + by Alexander Schrijver. Fixes bugs 5090 and 5402 (CVE 2012-1668); + bugfix on 0.2.0.16-alpha. + - Fix a compile warning when using the --enable-openbsd-malloc + configure option. Fixes bug 5340; bugfix on 0.2.0.20-rc. + - During configure, detect when we're building with clang version + 3.0 or lower and disable the -Wnormalized=id and -Woverride-init + CFLAGS. clang doesn't support them yet. + - When sending an HTTP/1.1 proxy request, include a Host header. + Fixes bug 5593; bugfix on 0.2.2.1-alpha. + - Fix a NULL-pointer dereference on a badly formed SETCIRCUITPURPOSE + command. Found by mikeyc. Fixes bug 5796; bugfix on 0.2.2.9-alpha. + - If we hit the error case where routerlist_insert() replaces an + existing (old) server descriptor, make sure to remove that + server descriptor from the old_routers list. Fix related to bug + 1776. Bugfix on 0.2.2.18-alpha. + + o Minor bugfixes (documentation and log messages): + - Fix a typo in a log message in rend_service_rendezvous_has_opened(). + Fixes bug 4856; bugfix on Tor 0.0.6. + - Update "ClientOnly" man page entry to explain that there isn't + really any point to messing with it. Resolves ticket 5005. + - Document the GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays + directory authority option (introduced in Tor 0.2.2.34). + - Downgrade the "We're missing a certificate" message from notice + to info: people kept mistaking it for a real problem, whereas it + is seldom the problem even when we are failing to bootstrap. Fixes + bug 5067; bugfix on 0.2.0.10-alpha. + - Correctly spell "connect" in a log message on failure to create a + controlsocket. Fixes bug 4803; bugfix on 0.2.2.26-beta. + - Clarify the behavior of MaxCircuitDirtiness with hidden service + circuits. Fixes issue 5259. + + o Minor features: + - Directory authorities now reject versions of Tor older than + 0.2.1.30, and Tor versions between 0.2.2.1-alpha and 0.2.2.20-alpha + inclusive. These versions accounted for only a small fraction of + the Tor network, and have numerous known security issues. Resolves + issue 4788. + - Update to the May 1 2012 Maxmind GeoLite Country database. + + - Feature removal: + - When sending or relaying a RELAY_EARLY cell, we used to convert + it to a RELAY cell if the connection was using the v1 link + protocol. This was a workaround for older versions of Tor, which + didn't handle RELAY_EARLY cells properly. Now that all supported + versions can handle RELAY_EARLY cells, and now that we're enforcing + the "no RELAY_EXTEND commands except in RELAY_EARLY cells" rule, + remove this workaround. Addresses bug 4786. + + +Changes in version 0.2.2.35 - 2011-12-16 + Tor 0.2.2.35 fixes a critical heap-overflow security issue in Tor's + buffers code. Absolutely everybody should upgrade. + + The bug relied on an incorrect calculation when making data continuous + in one of our IO buffers, if the first chunk of the buffer was + misaligned by just the wrong amount. The miscalculation would allow an + attacker to overflow a piece of heap-allocated memory. To mount this + attack, the attacker would need to either open a SOCKS connection to + Tor's SocksPort (usually restricted to localhost), or target a Tor + instance configured to make its connections through a SOCKS proxy + (which Tor does not do by default). + + Good security practice requires that all heap-overflow bugs should be + presumed to be exploitable until proven otherwise, so we are treating + this as a potential code execution attack. Please upgrade immediately! + This bug does not affect bufferevents-based builds of Tor. Special + thanks to "Vektor" for reporting this issue to us! + + Tor 0.2.2.35 also fixes several bugs in previous versions, including + crash bugs for unusual configurations, and a long-term bug that + would prevent Tor from starting on Windows machines with draconian + AV software. + + With this release, we remind everyone that 0.2.0.x has reached its + formal end-of-life. Those Tor versions have many known flaws, and + nobody should be using them. You should upgrade -- ideally to the + 0.2.2.x series. If you're using a Linux or BSD and its packages are + obsolete, stop using those packages and upgrade anyway. + + The Tor 0.2.1.x series is also approaching its end-of-life: it will no + longer receive support after some time in early 2012. + + o Major bugfixes: + - Fix a heap overflow bug that could occur when trying to pull + data into the first chunk of a buffer, when that chunk had + already had some data drained from it. Fixes CVE-2011-2778; + bugfix on 0.2.0.16-alpha. Reported by "Vektor". + - Initialize Libevent with the EVENT_BASE_FLAG_NOLOCK flag enabled, so + that it doesn't attempt to allocate a socketpair. This could cause + some problems on Windows systems with overzealous firewalls. Fix for + bug 4457; workaround for Libevent versions 2.0.1-alpha through + 2.0.15-stable. + - If we mark an OR connection for close based on a cell we process, + don't process any further cells on it. We already avoid further + reads on marked-for-close connections, but now we also discard the + cells we'd already read. Fixes bug 4299; bugfix on 0.2.0.10-alpha, + which was the first version where we might mark a connection for + close based on processing a cell on it. + - Correctly sanity-check that we don't underflow on a memory + allocation (and then assert) for hidden service introduction + point decryption. Bug discovered by Dan Rosenberg. Fixes bug 4410; + bugfix on 0.2.1.5-alpha. + - Fix a memory leak when we check whether a hidden service + descriptor has any usable introduction points left. Fixes bug + 4424. Bugfix on 0.2.2.25-alpha. + - Don't crash when we're running as a relay and don't have a GeoIP + file. Bugfix on 0.2.2.34; fixes bug 4340. This backports a fix + we've had in the 0.2.3.x branch already. + - When running as a client, do not print a misleading (and plain + wrong) log message that we're collecting "directory request" + statistics: clients don't collect statistics. Also don't create a + useless (because empty) stats file in the stats/ directory. Fixes + bug 4353; bugfix on 0.2.2.34. + + o Minor bugfixes: + - Detect failure to initialize Libevent. This fix provides better + detection for future instances of bug 4457. + - Avoid frequent calls to the fairly expensive cull_wedged_cpuworkers + function. This was eating up hideously large amounts of time on some + busy servers. Fixes bug 4518; bugfix on 0.0.9.8. + - Resolve an integer overflow bug in smartlist_ensure_capacity(). + Fixes bug 4230; bugfix on Tor 0.1.0.1-rc. Based on a patch by + Mansour Moufid. + - Don't warn about unused log_mutex in log.c when building with + --disable-threads using a recent GCC. Fixes bug 4437; bugfix on + 0.1.0.6-rc which introduced --disable-threads. + - When configuring, starting, or stopping an NT service, stop + immediately after the service configuration attempt has succeeded + or failed. Fixes bug 3963; bugfix on 0.2.0.7-alpha. + - When sending a NETINFO cell, include the original address + received for the other side, not its canonical address. Found + by "troll_un"; fixes bug 4349; bugfix on 0.2.0.10-alpha. + - Fix a typo in a hibernation-related log message. Fixes bug 4331; + bugfix on 0.2.2.23-alpha; found by "tmpname0901". + - Fix a memory leak in launch_direct_bridge_descriptor_fetch() that + occurred when a client tried to fetch a descriptor for a bridge + in ExcludeNodes. Fixes bug 4383; bugfix on 0.2.2.25-alpha. + - Backport fixes for a pair of compilation warnings on Windows. + Fixes bug 4521; bugfix on 0.2.2.28-beta and on 0.2.2.29-beta. + - If we had ever tried to call tor_addr_to_str on an address of + unknown type, we would have done a strdup on an uninitialized + buffer. Now we won't. Fixes bug 4529; bugfix on 0.2.1.3-alpha. + Reported by "troll_un". + - Correctly detect and handle transient lookup failures from + tor_addr_lookup. Fixes bug 4530; bugfix on 0.2.1.5-alpha. + Reported by "troll_un". + - Fix null-pointer access that could occur if TLS allocation failed. + Fixes bug 4531; bugfix on 0.2.0.20-rc. Found by "troll_un". + - Use tor_socket_t type for listener argument to accept(). Fixes bug + 4535; bugfix on 0.2.2.28-beta. Found by "troll_un". + + o Minor features: + - Add two new config options for directory authorities: + AuthDirFastGuarantee sets a bandwidth threshold for guaranteeing the + Fast flag, and AuthDirGuardBWGuarantee sets a bandwidth threshold + that is always sufficient to satisfy the bandwidth requirement for + the Guard flag. Now it will be easier for researchers to simulate + Tor networks with different values. Resolves ticket 4484. + - When Tor ignores a hidden service specified in its configuration, + include the hidden service's directory in the warning message. + Previously, we would only tell the user that some hidden service + was ignored. Bugfix on 0.0.6; fixes bug 4426. + - Update to the December 6 2011 Maxmind GeoLite Country database. + + o Packaging changes: + - Make it easier to automate expert package builds on Windows, + by removing an absolute path from makensis.exe command. + + +Changes in version 0.2.2.34 - 2011-10-26 + Tor 0.2.2.34 fixes a critical anonymity vulnerability where an attacker + can deanonymize Tor users. Everybody should upgrade. + + The attack relies on four components: 1) Clients reuse their TLS cert + when talking to different relays, so relays can recognize a user by + the identity key in her cert. 2) An attacker who knows the client's + identity key can probe each guard relay to see if that identity key + is connected to that guard relay right now. 3) A variety of active + attacks in the literature (starting from "Low-Cost Traffic Analysis + of Tor" by Murdoch and Danezis in 2005) allow a malicious website to + discover the guard relays that a Tor user visiting the website is using. + 4) Clients typically pick three guards at random, so the set of guards + for a given user could well be a unique fingerprint for her. This + release fixes components #1 and #2, which is enough to block the attack; + the other two remain as open research problems. Special thanks to + "frosty_un" for reporting the issue to us! + + Clients should upgrade so they are no longer recognizable by the TLS + certs they present. Relays should upgrade so they no longer allow a + remote attacker to probe them to test whether unpatched clients are + currently connected to them. + + This release also fixes several vulnerabilities that allow an attacker + to enumerate bridge relays. Some bridge enumeration attacks still + remain; see for example proposal 188. + + o Privacy/anonymity fixes (clients): + - Clients and bridges no longer send TLS certificate chains on + outgoing OR connections. Previously, each client or bridge would + use the same cert chain for all outgoing OR connections until + its IP address changes, which allowed any relay that the client + or bridge contacted to determine which entry guards it is using. + Fixes CVE-2011-2768. Bugfix on 0.0.9pre5; found by "frosty_un". + - If a relay receives a CREATE_FAST cell on a TLS connection, it + no longer considers that connection as suitable for satisfying a + circuit EXTEND request. Now relays can protect clients from the + CVE-2011-2768 issue even if the clients haven't upgraded yet. + - Directory authorities no longer assign the Guard flag to relays + that haven't upgraded to the above "refuse EXTEND requests + to client connections" fix. Now directory authorities can + protect clients from the CVE-2011-2768 issue even if neither + the clients nor the relays have upgraded yet. There's a new + "GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays" config option + to let us transition smoothly, else tomorrow there would be no + guard relays. + + o Privacy/anonymity fixes (bridge enumeration): + - Bridge relays now do their directory fetches inside Tor TLS + connections, like all the other clients do, rather than connecting + directly to the DirPort like public relays do. Removes another + avenue for enumerating bridges. Fixes bug 4115; bugfix on 0.2.0.35. + - Bridges relays now build circuits for themselves in a more similar + way to how clients build them. Removes another avenue for + enumerating bridges. Fixes bug 4124; bugfix on 0.2.0.3-alpha, + when bridges were introduced. + - Bridges now refuse CREATE or CREATE_FAST cells on OR connections + that they initiated. Relays could distinguish incoming bridge + connections from client connections, creating another avenue for + enumerating bridges. Fixes CVE-2011-2769. Bugfix on 0.2.0.3-alpha. + Found by "frosty_un". + + o Major bugfixes: + - Fix a crash bug when changing node restrictions while a DNS lookup + is in-progress. Fixes bug 4259; bugfix on 0.2.2.25-alpha. Bugfix + by "Tey'". + - Don't launch a useless circuit after failing to use one of a + hidden service's introduction points. Previously, we would + launch a new introduction circuit, but not set the hidden service + which that circuit was intended to connect to, so it would never + actually be used. A different piece of code would then create a + new introduction circuit correctly. Bug reported by katmagic and + found by Sebastian Hahn. Bugfix on 0.2.1.13-alpha; fixes bug 4212. + + o Minor bugfixes: + - Change an integer overflow check in the OpenBSD_Malloc code so + that GCC is less likely to eliminate it as impossible. Patch + from Mansour Moufid. Fixes bug 4059. + - When a hidden service turns an extra service-side introduction + circuit into a general-purpose circuit, free the rend_data and + intro_key fields first, so we won't leak memory if the circuit + is cannibalized for use as another service-side introduction + circuit. Bugfix on 0.2.1.7-alpha; fixes bug 4251. + - Bridges now skip DNS self-tests, to act a little more stealthily. + Fixes bug 4201; bugfix on 0.2.0.3-alpha, which first introduced + bridges. Patch by "warms0x". + - Fix internal bug-checking logic that was supposed to catch + failures in digest generation so that it will fail more robustly + if we ask for a nonexistent algorithm. Found by Coverity Scan. + Bugfix on 0.2.2.1-alpha; fixes Coverity CID 479. + - Report any failure in init_keys() calls launched because our + IP address has changed. Spotted by Coverity Scan. Bugfix on + 0.1.1.4-alpha; fixes CID 484. + + o Minor bugfixes (log messages and documentation): + - Remove a confusing dollar sign from the example fingerprint in the + man page, and also make the example fingerprint a valid one. Fixes + bug 4309; bugfix on 0.2.1.3-alpha. + - The next version of Windows will be called Windows 8, and it has + a major version of 6, minor version of 2. Correctly identify that + version instead of calling it "Very recent version". Resolves + ticket 4153; reported by funkstar. + - Downgrade log messages about circuit timeout calibration from + "notice" to "info": they don't require or suggest any human + intervention. Patch from Tom Lowenthal. Fixes bug 4063; + bugfix on 0.2.2.14-alpha. + + o Minor features: + - Turn on directory request statistics by default and include them in + extra-info descriptors. Don't break if we have no GeoIP database. + Backported from 0.2.3.1-alpha; implements ticket 3951. + - Update to the October 4 2011 Maxmind GeoLite Country database. + + +Changes in version 0.2.1.31 - 2011-10-26 + Tor 0.2.1.31 backports important security and privacy fixes for + oldstable. This release is intended only for package maintainers and + others who cannot use the 0.2.2 stable series. All others should be + using Tor 0.2.2.x or newer. + + o Security fixes (also included in 0.2.2.x): + - Replace all potentially sensitive memory comparison operations + with versions whose runtime does not depend on the data being + compared. This will help resist a class of attacks where an + adversary can use variations in timing information to learn + sensitive data. Fix for one case of bug 3122. (Safe memcmp + implementation by Robert Ransom based partially on code by DJB.) + - Fix an assert in parsing router descriptors containing IPv6 + addresses. This one took down the directory authorities when + somebody tried some experimental code. Bugfix on 0.2.1.3-alpha. + + o Privacy/anonymity fixes (also included in 0.2.2.x): + - Clients and bridges no longer send TLS certificate chains on + outgoing OR connections. Previously, each client or bridge would + use the same cert chain for all outgoing OR connections until + its IP address changes, which allowed any relay that the client + or bridge contacted to determine which entry guards it is using. + Fixes CVE-2011-2768. Bugfix on 0.0.9pre5; found by "frosty_un". + - If a relay receives a CREATE_FAST cell on a TLS connection, it + no longer considers that connection as suitable for satisfying a + circuit EXTEND request. Now relays can protect clients from the + CVE-2011-2768 issue even if the clients haven't upgraded yet. + - Bridges now refuse CREATE or CREATE_FAST cells on OR connections + that they initiated. Relays could distinguish incoming bridge + connections from client connections, creating another avenue for + enumerating bridges. Fixes CVE-2011-2769. Bugfix on 0.2.0.3-alpha. + Found by "frosty_un". + - When receiving a hidden service descriptor, check that it is for + the hidden service we wanted. Previously, Tor would store any + hidden service descriptors that a directory gave it, whether it + wanted them or not. This wouldn't have let an attacker impersonate + a hidden service, but it did let directories pre-seed a client + with descriptors that it didn't want. Bugfix on 0.0.6. + - Avoid linkability based on cached hidden service descriptors: forget + all hidden service descriptors cached as a client when processing a + SIGNAL NEWNYM command. Fixes bug 3000; bugfix on 0.0.6. + - Make the bridge directory authority refuse to answer directory + requests for "all" descriptors. It used to include bridge + descriptors in its answer, which was a major information leak. + Found by "piebeer". Bugfix on 0.2.0.3-alpha. + - Don't attach new streams to old rendezvous circuits after SIGNAL + NEWNYM. Previously, we would keep using an existing rendezvous + circuit if it remained open (i.e. if it were kept open by a + long-lived stream, or if a new stream were attached to it before + Tor could notice that it was old and no longer in use). Bugfix on + 0.1.1.15-rc; fixes bug 3375. + + o Minor bugfixes (also included in 0.2.2.x): + - When we restart our relay, we might get a successful connection + from the outside before we've started our reachability tests, + triggering a warning: "ORPort found reachable, but I have no + routerinfo yet. Failing to inform controller of success." This + bug was harmless unless Tor is running under a controller + like Vidalia, in which case the controller would never get a + REACHABILITY_SUCCEEDED status event. Bugfix on 0.1.2.6-alpha; + fixes bug 1172. + - Build correctly on OSX with zlib 1.2.4 and higher with all warnings + enabled. Fixes bug 1526. + - Remove undocumented option "-F" from tor-resolve: it hasn't done + anything since 0.2.1.16-rc. + - Avoid signed/unsigned comparisons by making SIZE_T_CEILING unsigned. + None of the cases where we did this before were wrong, but by making + this change we avoid warnings. Fixes bug 2475; bugfix on 0.2.1.28. + - Fix a rare crash bug that could occur when a client was configured + with a large number of bridges. Fixes bug 2629; bugfix on + 0.2.1.2-alpha. Bugfix by trac user "shitlei". + - Correct the warning displayed when a rendezvous descriptor exceeds + the maximum size. Fixes bug 2750; bugfix on 0.2.1.5-alpha. Found by + John Brooks. + - Fix an uncommon assertion failure when running with DNSPort under + heavy load. Fixes bug 2933; bugfix on 0.2.0.1-alpha. + - When warning about missing zlib development packages during compile, + give the correct package names. Bugfix on 0.2.0.1-alpha. + - Require that introduction point keys and onion keys have public + exponent 65537. Bugfix on 0.2.0.10-alpha. + - Do not crash when our configuration file becomes unreadable, for + example due to a permissions change, between when we start up + and when a controller calls SAVECONF. Fixes bug 3135; bugfix + on 0.0.9pre6. + - Fix warnings from GCC 4.6's "-Wunused-but-set-variable" option. + Fixes bug 3208. + - Always NUL-terminate the sun_path field of a sockaddr_un before + passing it to the kernel. (Not a security issue: kernels are + smart enough to reject bad sockaddr_uns.) Found by Coverity; + CID #428. Bugfix on Tor 0.2.0.3-alpha. + - Don't stack-allocate the list of supplementary GIDs when we're + about to log them. Stack-allocating NGROUPS_MAX gid_t elements + could take up to 256K, which is way too much stack. Found by + Coverity; CID #450. Bugfix on 0.2.1.7-alpha. + + o Minor bugfixes (only in 0.2.1.x): + - Resume using micro-version numbers in 0.2.1.x: our Debian packages + rely on them. Bugfix on 0.2.1.30. + - Use git revisions instead of svn revisions when generating our + micro-version numbers. Bugfix on 0.2.1.15-rc; fixes bug 2402. + + o Minor features (also included in 0.2.2.x): + - Adjust the expiration time on our SSL session certificates to + better match SSL certs seen in the wild. Resolves ticket 4014. + - Allow nameservers with IPv6 address. Resolves bug 2574. + - Update to the October 4 2011 Maxmind GeoLite Country database. + + +Changes in version 0.2.2.33 - 2011-09-13 + Tor 0.2.2.33 fixes several bugs, and includes a slight tweak to Tor's + TLS handshake that makes relays and bridges that run this new version + reachable from Iran again. + + o Major bugfixes: + - Avoid an assertion failure when reloading a configuration with + TrackExitHosts changes. Found and fixed by 'laruldan'. Fixes bug + 3923; bugfix on 0.2.2.25-alpha. + + o Minor features (security): + - Check for replays of the public-key encrypted portion of an + INTRODUCE1 cell, in addition to the current check for replays of + the g^x value. This prevents a possible class of active attacks + by an attacker who controls both an introduction point and a + rendezvous point, and who uses the malleability of AES-CTR to + alter the encrypted g^x portion of the INTRODUCE1 cell. We think + that these attacks are infeasible (requiring the attacker to send + on the order of zettabytes of altered cells in a short interval), + but we'd rather block them off in case there are any classes of + this attack that we missed. Reported by Willem Pinckaers. + + o Minor features: + - Adjust the expiration time on our SSL session certificates to + better match SSL certs seen in the wild. Resolves ticket 4014. + - Change the default required uptime for a relay to be accepted as + a HSDir (hidden service directory) from 24 hours to 25 hours. + Improves on 0.2.0.10-alpha; resolves ticket 2649. + - Add a VoteOnHidServDirectoriesV2 config option to allow directory + authorities to abstain from voting on assignment of the HSDir + consensus flag. Related to bug 2649. + - Update to the September 6 2011 Maxmind GeoLite Country database. + + o Minor bugfixes (documentation and log messages): + - Correct the man page to explain that HashedControlPassword and + CookieAuthentication can both be set, in which case either method + is sufficient to authenticate to Tor. Bugfix on 0.2.0.7-alpha, + when we decided to allow these config options to both be set. Issue + raised by bug 3898. + - Demote the 'replay detected' log message emitted when a hidden + service receives the same Diffie-Hellman public key in two different + INTRODUCE2 cells to info level. A normal Tor client can cause that + log message during its normal operation. Bugfix on 0.2.1.6-alpha; + fixes part of bug 2442. + - Demote the 'INTRODUCE2 cell is too {old,new}' log message to info + level. There is nothing that a hidden service's operator can do + to fix its clients' clocks. Bugfix on 0.2.1.6-alpha; fixes part + of bug 2442. + - Clarify a log message specifying the characters permitted in + HiddenServiceAuthorizeClient client names. Previously, the log + message said that "[A-Za-z0-9+-_]" were permitted; that could have + given the impression that every ASCII character between "+" and "_" + was permitted. Now we say "[A-Za-z0-9+_-]". Bugfix on 0.2.1.5-alpha. + + o Build fixes: + - Provide a substitute implementation of lround() for MSVC, which + apparently lacks it. Patch from Gisle Vanem. + - Clean up some code issues that prevented Tor from building on older + BSDs. Fixes bug 3894; reported by "grarpamp". + - Search for a platform-specific version of "ar" when cross-compiling. + Should fix builds on iOS. Resolves bug 3909, found by Marco Bonetti. + + +Changes in version 0.2.2.32 - 2011-08-27 + The Tor 0.2.2 release series is dedicated to the memory of Andreas + Pfitzmann (1958-2010), a pioneer in anonymity and privacy research, + a founder of the PETS community, a leader in our field, a mentor, + and a friend. He left us with these words: "I had the possibility + to contribute to this world that is not as it should be. I hope I + could help in some areas to make the world a better place, and that + I could also encourage other people to be engaged in improving the + world. Please, stay engaged. This world needs you, your love, your + initiative -- now I cannot be part of that anymore." + + Tor 0.2.2.32, the first stable release in the 0.2.2 branch, is finally + ready. More than two years in the making, this release features improved + client performance and hidden service reliability, better compatibility + for Android, correct behavior for bridges that listen on more than + one address, more extensible and flexible directory object handling, + better reporting of network statistics, improved code security, and + many many other features and bugfixes. + + o Major features (client performance): + - When choosing which cells to relay first, relays now favor circuits + that have been quiet recently, to provide lower latency for + low-volume circuits. By default, relays enable or disable this + feature based on a setting in the consensus. They can override + this default by using the new "CircuitPriorityHalflife" config + option. Design and code by Ian Goldberg, Can Tang, and Chris + Alexander. + - Directory authorities now compute consensus weightings that instruct + clients how to weight relays flagged as Guard, Exit, Guard+Exit, + and no flag. Clients use these weightings to distribute network load + more evenly across these different relay types. The weightings are + in the consensus so we can change them globally in the future. Extra + thanks to "outofwords" for finding some nasty security bugs in + the first implementation of this feature. + + o Major features (client performance, circuit build timeout): + - Tor now tracks how long it takes to build client-side circuits + over time, and adapts its timeout to local network performance. + Since a circuit that takes a long time to build will also provide + bad performance, we get significant latency improvements by + discarding the slowest 20% of circuits. Specifically, Tor creates + circuits more aggressively than usual until it has enough data + points for a good timeout estimate. Implements proposal 151. + - Circuit build timeout constants can be controlled by consensus + parameters. We set good defaults for these parameters based on + experimentation on broadband and simulated high-latency links. + - Circuit build time learning can be disabled via consensus parameter + or by the client via a LearnCircuitBuildTimeout config option. We + also automatically disable circuit build time calculation if either + AuthoritativeDirectory is set, or if we fail to write our state + file. Implements ticket 1296. + + o Major features (relays use their capacity better): + - Set SO_REUSEADDR socket option on all sockets, not just + listeners. This should help busy exit nodes avoid running out of + useable ports just because all the ports have been used in the + near past. Resolves issue 2850. + - Relays now save observed peak bandwidth throughput rates to their + state file (along with total usage, which was already saved), + so that they can determine their correct estimated bandwidth on + restart. Resolves bug 1863, where Tor relays would reset their + estimated bandwidth to 0 after restarting. + - Lower the maximum weighted-fractional-uptime cutoff to 98%. This + should give us approximately 40-50% more Guard-flagged nodes, + improving the anonymity the Tor network can provide and also + decreasing the dropoff in throughput that relays experience when + they first get the Guard flag. + - Directory authorities now take changes in router IP address and + ORPort into account when determining router stability. Previously, + if a router changed its IP or ORPort, the authorities would not + treat it as having any downtime for the purposes of stability + calculation, whereas clients would experience downtime since the + change would take a while to propagate to them. Resolves issue 1035. + - New AccelName and AccelDir options add support for dynamic OpenSSL + hardware crypto acceleration engines. + + o Major features (relays control their load better): + - Exit relays now try harder to block exit attempts from unknown + relays, to make it harder for people to use them as one-hop proxies + a la tortunnel. Controlled by the refuseunknownexits consensus + parameter (currently enabled), or you can override it on your + relay with the RefuseUnknownExits torrc option. Resolves bug 1751; + based on a variant of proposal 163. + - Add separate per-conn write limiting to go with the per-conn read + limiting. We added a global write limit in Tor 0.1.2.5-alpha, + but never per-conn write limits. + - New consensus params "bwconnrate" and "bwconnburst" to let us + rate-limit client connections as they enter the network. It's + controlled in the consensus so we can turn it on and off for + experiments. It's starting out off. Based on proposal 163. + + o Major features (controllers): + - Export GeoIP information on bridge usage to controllers even if we + have not yet been running for 24 hours. Now Vidalia bridge operators + can get more accurate and immediate feedback about their + contributions to the network. + - Add an __OwningControllerProcess configuration option and a + TAKEOWNERSHIP control-port command. Now a Tor controller can ensure + that when it exits, Tor will shut down. Implements feature 3049. + + o Major features (directory authorities): + - Directory authorities now create, vote on, and serve multiple + parallel formats of directory data as part of their voting process. + Partially implements Proposal 162: "Publish the consensus in + multiple flavors". + - Directory authorities now agree on and publish small summaries + of router information that clients can use in place of regular + server descriptors. This transition will allow Tor 0.2.3 clients + to use far less bandwidth for downloading information about the + network. Begins the implementation of Proposal 158: "Clients + download consensus + microdescriptors". + - The directory voting system is now extensible to use multiple hash + algorithms for signatures and resource selection. Newer formats + are signed with SHA256, with a possibility for moving to a better + hash algorithm in the future. + - Directory authorities can now vote on arbitary integer values as + part of the consensus process. This is designed to help set + network-wide parameters. Implements proposal 167. + + o Major features and bugfixes (node selection): + - Revise and reconcile the meaning of the ExitNodes, EntryNodes, + ExcludeEntryNodes, ExcludeExitNodes, ExcludeNodes, and Strict*Nodes + options. Previously, we had been ambiguous in describing what + counted as an "exit" node, and what operations exactly "StrictNodes + 0" would permit. This created confusion when people saw nodes built + through unexpected circuits, and made it hard to tell real bugs from + surprises. Now the intended behavior is: + . "Exit", in the context of ExitNodes and ExcludeExitNodes, means + a node that delivers user traffic outside the Tor network. + . "Entry", in the context of EntryNodes, means a node used as the + first hop of a multihop circuit. It doesn't include direct + connections to directory servers. + . "ExcludeNodes" applies to all nodes. + . "StrictNodes" changes the behavior of ExcludeNodes only. When + StrictNodes is set, Tor should avoid all nodes listed in + ExcludeNodes, even when it will make user requests fail. When + StrictNodes is *not* set, then Tor should follow ExcludeNodes + whenever it can, except when it must use an excluded node to + perform self-tests, connect to a hidden service, provide a + hidden service, fulfill a .exit request, upload directory + information, or fetch directory information. + Collectively, the changes to implement the behavior fix bug 1090. + - If EntryNodes, ExitNodes, ExcludeNodes, or ExcludeExitNodes + change during a config reload, mark and discard all our origin + circuits. This fix should address edge cases where we change the + config options and but then choose a circuit that we created before + the change. + - Make EntryNodes config option much more aggressive even when + StrictNodes is not set. Before it would prepend your requested + entrynodes to your list of guard nodes, but feel free to use others + after that. Now it chooses only from your EntryNodes if any of + those are available, and only falls back to others if a) they're + all down and b) StrictNodes is not set. + - Now we refresh your entry guards from EntryNodes at each consensus + fetch -- rather than just at startup and then they slowly rot as + the network changes. + - Add support for the country code "{??}" in torrc options like + ExcludeNodes, to indicate all routers of unknown country. Closes + bug 1094. + - ExcludeNodes now takes precedence over EntryNodes and ExitNodes: if + a node is listed in both, it's treated as excluded. + - ExcludeNodes now applies to directory nodes -- as a preference if + StrictNodes is 0, or an absolute requirement if StrictNodes is 1. + Don't exclude all the directory authorities and set StrictNodes to 1 + unless you really want your Tor to break. + - ExcludeNodes and ExcludeExitNodes now override exit enclaving. + - ExcludeExitNodes now overrides .exit requests. + - We don't use bridges listed in ExcludeNodes. + - When StrictNodes is 1: + . We now apply ExcludeNodes to hidden service introduction points + and to rendezvous points selected by hidden service users. This + can make your hidden service less reliable: use it with caution! + . If we have used ExcludeNodes on ourself, do not try relay + reachability self-tests. + . If we have excluded all the directory authorities, we will not + even try to upload our descriptor if we're a relay. + . Do not honor .exit requests to an excluded node. + - When the set of permitted nodes changes, we now remove any mappings + introduced via TrackExitHosts to now-excluded nodes. Bugfix on + 0.1.0.1-rc. + - We never cannibalize a circuit that had excluded nodes on it, even + if StrictNodes is 0. Bugfix on 0.1.0.1-rc. + - Improve log messages related to excluded nodes. + + o Major features (misc): + - Numerous changes, bugfixes, and workarounds from Nathan Freitas + to help Tor build correctly for Android phones. + - The options SocksPort, ControlPort, and so on now all accept a + value "auto" that opens a socket on an OS-selected port. A + new ControlPortWriteToFile option tells Tor to write its + actual control port or ports to a chosen file. If the option + ControlPortFileGroupReadable is set, the file is created as + group-readable. Now users can run two Tor clients on the same + system without needing to manually mess with parameters. Resolves + part of ticket 3076. + - Tor now supports tunneling all of its outgoing connections over + a SOCKS proxy, using the SOCKS4Proxy and/or SOCKS5Proxy + configuration options. Code by Christopher Davis. + + o Code security improvements: + - Replace all potentially sensitive memory comparison operations + with versions whose runtime does not depend on the data being + compared. This will help resist a class of attacks where an + adversary can use variations in timing information to learn + sensitive data. Fix for one case of bug 3122. (Safe memcmp + implementation by Robert Ransom based partially on code by DJB.) + - Enable Address Space Layout Randomization (ASLR) and Data Execution + Prevention (DEP) by default on Windows to make it harder for + attackers to exploit vulnerabilities. Patch from John Brooks. + - New "--enable-gcc-hardening" ./configure flag (off by default) + to turn on gcc compile time hardening options. It ensures + that signed ints have defined behavior (-fwrapv), enables + -D_FORTIFY_SOURCE=2 (requiring -O2), adds stack smashing protection + with canaries (-fstack-protector-all), turns on ASLR protection if + supported by the kernel (-fPIE, -pie), and adds additional security + related warnings. Verified to work on Mac OS X and Debian Lenny. + - New "--enable-linker-hardening" ./configure flag (off by default) + to turn on ELF specific hardening features (relro, now). This does + not work with Mac OS X or any other non-ELF binary format. + - Always search the Windows system directory for system DLLs, and + nowhere else. Bugfix on 0.1.1.23; fixes bug 1954. + - New DisableAllSwap option. If set to 1, Tor will attempt to lock all + current and future memory pages via mlockall(). On supported + platforms (modern Linux and probably BSD but not Windows or OS X), + this should effectively disable any and all attempts to page out + memory. This option requires that you start your Tor as root -- + if you use DisableAllSwap, please consider using the User option + to properly reduce the privileges of your Tor. + + o Major bugfixes (crashes): + - Fix crash bug on platforms where gmtime and localtime can return + NULL. Windows 7 users were running into this one. Fixes part of bug + 2077. Bugfix on all versions of Tor. Found by boboper. + - Introduce minimum/maximum values that clients will believe + from the consensus. Now we'll have a better chance to avoid crashes + or worse when a consensus param has a weird value. + - Fix a rare crash bug that could occur when a client was configured + with a large number of bridges. Fixes bug 2629; bugfix on + 0.2.1.2-alpha. Bugfix by trac user "shitlei". + - Do not crash when our configuration file becomes unreadable, for + example due to a permissions change, between when we start up + and when a controller calls SAVECONF. Fixes bug 3135; bugfix + on 0.0.9pre6. + - If we're in the pathological case where there's no exit bandwidth + but there is non-exit bandwidth, or no guard bandwidth but there + is non-guard bandwidth, don't crash during path selection. Bugfix + on 0.2.0.3-alpha. + - Fix a crash bug when trying to initialize the evdns module in + Libevent 2. Bugfix on 0.2.1.16-rc. + + o Major bugfixes (stability): + - Fix an assert in parsing router descriptors containing IPv6 + addresses. This one took down the directory authorities when + somebody tried some experimental code. Bugfix on 0.2.1.3-alpha. + - Fix an uncommon assertion failure when running with DNSPort under + heavy load. Fixes bug 2933; bugfix on 0.2.0.1-alpha. + - Treat an unset $HOME like an empty $HOME rather than triggering an + assert. Bugfix on 0.0.8pre1; fixes bug 1522. + - More gracefully handle corrupt state files, removing asserts + in favor of saving a backup and resetting state. + - Instead of giving an assertion failure on an internal mismatch + on estimated freelist size, just log a BUG warning and try later. + Mitigates but does not fix bug 1125. + - Fix an assert that got triggered when using the TestingTorNetwork + configuration option and then issuing a GETINFO config-text control + command. Fixes bug 2250; bugfix on 0.2.1.2-alpha. + - If the cached cert file is unparseable, warn but don't exit. + + o Privacy fixes (relays/bridges): + - Don't list Windows capabilities in relay descriptors. We never made + use of them, and maybe it's a bad idea to publish them. Bugfix + on 0.1.1.8-alpha. + - If the Nickname configuration option isn't given, Tor would pick a + nickname based on the local hostname as the nickname for a relay. + Because nicknames are not very important in today's Tor and the + "Unnamed" nickname has been implemented, this is now problematic + behavior: It leaks information about the hostname without being + useful at all. Fixes bug 2979; bugfix on 0.1.2.2-alpha, which + introduced the Unnamed nickname. Reported by tagnaq. + - Maintain separate TLS contexts and certificates for incoming and + outgoing connections in bridge relays. Previously we would use the + same TLS contexts and certs for incoming and outgoing connections. + Bugfix on 0.2.0.3-alpha; addresses bug 988. + - Maintain separate identity keys for incoming and outgoing TLS + contexts in bridge relays. Previously we would use the same + identity keys for incoming and outgoing TLS contexts. Bugfix on + 0.2.0.3-alpha; addresses the other half of bug 988. + - Make the bridge directory authority refuse to answer directory + requests for "all descriptors". It used to include bridge + descriptors in its answer, which was a major information leak. + Found by "piebeer". Bugfix on 0.2.0.3-alpha. + + o Privacy fixes (clients): + - When receiving a hidden service descriptor, check that it is for + the hidden service we wanted. Previously, Tor would store any + hidden service descriptors that a directory gave it, whether it + wanted them or not. This wouldn't have let an attacker impersonate + a hidden service, but it did let directories pre-seed a client + with descriptors that it didn't want. Bugfix on 0.0.6. + - Start the process of disabling ".exit" address notation, since it + can be used for a variety of esoteric application-level attacks + on users. To reenable it, set "AllowDotExit 1" in your torrc. Fix + on 0.0.9rc5. + - Reject attempts at the client side to open connections to private + IP addresses (like 127.0.0.1, 10.0.0.1, and so on) with + a randomly chosen exit node. Attempts to do so are always + ill-defined, generally prevented by exit policies, and usually + in error. This will also help to detect loops in transparent + proxy configurations. You can disable this feature by setting + "ClientRejectInternalAddresses 0" in your torrc. + - Log a notice when we get a new control connection. Now it's easier + for security-conscious users to recognize when a local application + is knocking on their controller door. Suggested by bug 1196. + + o Privacy fixes (newnym): + - Avoid linkability based on cached hidden service descriptors: forget + all hidden service descriptors cached as a client when processing a + SIGNAL NEWNYM command. Fixes bug 3000; bugfix on 0.0.6. + - On SIGHUP, do not clear out all TrackHostExits mappings, client + DNS cache entries, and virtual address mappings: that's what + NEWNYM is for. Fixes bug 1345; bugfix on 0.1.0.1-rc. + - Don't attach new streams to old rendezvous circuits after SIGNAL + NEWNYM. Previously, we would keep using an existing rendezvous + circuit if it remained open (i.e. if it were kept open by a + long-lived stream, or if a new stream were attached to it before + Tor could notice that it was old and no longer in use). Bugfix on + 0.1.1.15-rc; fixes bug 3375. + + o Major bugfixes (relay bandwidth accounting): + - Fix a bug that could break accounting on 64-bit systems with large + time_t values, making them hibernate for impossibly long intervals. + Fixes bug 2146. Bugfix on 0.0.9pre6; fix by boboper. + - Fix a bug in bandwidth accounting that could make us use twice + the intended bandwidth when our interval start changes due to + daylight saving time. Now we tolerate skew in stored vs computed + interval starts: if the start of the period changes by no more than + 50% of the period's duration, we remember bytes that we transferred + in the old period. Fixes bug 1511; bugfix on 0.0.9pre5. + + o Major bugfixes (bridges): + - Bridges now use "reject *:*" as their default exit policy. Bugfix + on 0.2.0.3-alpha. Fixes bug 1113. + - If you configure your bridge with a known identity fingerprint, + and the bridge authority is unreachable (as it is in at least + one country now), fall back to directly requesting the descriptor + from the bridge. Finishes the feature started in 0.2.0.10-alpha; + closes bug 1138. + - Fix a bug where bridge users who configure the non-canonical + address of a bridge automatically switch to its canonical + address. If a bridge listens at more than one address, it + should be able to advertise those addresses independently and + any non-blocked addresses should continue to work. Bugfix on Tor + 0.2.0.3-alpha. Fixes bug 2510. + - If you configure Tor to use bridge A, and then quit and + configure Tor to use bridge B instead (or if you change Tor + to use bridge B via the controller), it would happily continue + to use bridge A if it's still reachable. While this behavior is + a feature if your goal is connectivity, in some scenarios it's a + dangerous bug. Bugfix on Tor 0.2.0.1-alpha; fixes bug 2511. + - When the controller configures a new bridge, don't wait 10 to 60 + seconds before trying to fetch its descriptor. Bugfix on + 0.2.0.3-alpha; fixes bug 3198 (suggested by 2355). + + o Major bugfixes (directory authorities): + - Many relays have been falling out of the consensus lately because + not enough authorities know about their descriptor for them to get + a majority of votes. When we deprecated the v2 directory protocol, + we got rid of the only way that v3 authorities can hear from each + other about other descriptors. Now authorities examine every v3 + vote for new descriptors, and fetch them from that authority. Bugfix + on 0.2.1.23. + - Authorities could be tricked into giving out the Exit flag to relays + that didn't allow exiting to any ports. This bug could screw + with load balancing and stats. Bugfix on 0.1.1.6-alpha; fixes bug + 1238. Bug discovered by Martin Kowalczyk. + - If all authorities restart at once right before a consensus vote, + nobody will vote about "Running", and clients will get a consensus + with no usable relays. Instead, authorities refuse to build a + consensus if this happens. Bugfix on 0.2.0.10-alpha; fixes bug 1066. + + o Major bugfixes (stream-level fairness): + - When receiving a circuit-level SENDME for a blocked circuit, try + to package cells fairly from all the streams that had previously + been blocked on that circuit. Previously, we had started with the + oldest stream, and allowed each stream to potentially exhaust + the circuit's package window. This gave older streams on any + given circuit priority over newer ones. Fixes bug 1937. Detected + originally by Camilo Viecco. This bug was introduced before the + first Tor release, in svn commit r152: it is the new winner of + the longest-lived bug prize. + - Fix a stream fairness bug that would cause newer streams on a given + circuit to get preference when reading bytes from the origin or + destination. Fixes bug 2210. Fix by Mashael AlSabah. This bug was + introduced before the first Tor release, in svn revision r152. + - When the exit relay got a circuit-level sendme cell, it started + reading on the exit streams, even if had 500 cells queued in the + circuit queue already, so the circuit queue just grew and grew in + some cases. We fix this by not re-enabling reading on receipt of a + sendme cell when the cell queue is blocked. Fixes bug 1653. Bugfix + on 0.2.0.1-alpha. Detected by Mashael AlSabah. Original patch by + "yetonetime". + - Newly created streams were allowed to read cells onto circuits, + even if the circuit's cell queue was blocked and waiting to drain. + This created potential unfairness, as older streams would be + blocked, but newer streams would gladly fill the queue completely. + We add code to detect this situation and prevent any stream from + getting more than one free cell. Bugfix on 0.2.0.1-alpha. Partially + fixes bug 1298. + + o Major bugfixes (hidden services): + - Apply circuit timeouts to opened hidden-service-related circuits + based on the correct start time. Previously, we would apply the + circuit build timeout based on time since the circuit's creation; + it was supposed to be applied based on time since the circuit + entered its current state. Bugfix on 0.0.6; fixes part of bug 1297. + - Improve hidden service robustness: When we find that we have + extended a hidden service's introduction circuit to a relay not + listed as an introduction point in the HS descriptor we currently + have, retry with an introduction point from the current + descriptor. Previously we would just give up. Fixes bugs 1024 and + 1930; bugfix on 0.2.0.10-alpha. + - Directory authorities now use data collected from their own + uptime observations when choosing whether to assign the HSDir flag + to relays, instead of trusting the uptime value the relay reports in + its descriptor. This change helps prevent an attack where a small + set of nodes with frequently-changing identity keys can blackhole + a hidden service. (Only authorities need upgrade; others will be + fine once they do.) Bugfix on 0.2.0.10-alpha; fixes bug 2709. + - Stop assigning the HSDir flag to relays that disable their + DirPort (and thus will refuse to answer directory requests). This + fix should dramatically improve the reachability of hidden services: + hidden services and hidden service clients pick six HSDir relays + to store and retrieve the hidden service descriptor, and currently + about half of the HSDir relays will refuse to work. Bugfix on + 0.2.0.10-alpha; fixes part of bug 1693. + + o Major bugfixes (misc): + - Clients now stop trying to use an exit node associated with a given + destination by TrackHostExits if they fail to reach that exit node. + Fixes bug 2999. Bugfix on 0.2.0.20-rc. + - Fix a regression that caused Tor to rebind its ports if it receives + SIGHUP while hibernating. Bugfix in 0.1.1.6-alpha; closes bug 919. + - Remove an extra pair of quotation marks around the error + message in control-port STATUS_GENERAL BUG events. Bugfix on + 0.1.2.6-alpha; fixes bug 3732. + + o Minor features (relays): + - Ensure that no empty [dirreq-](read|write)-history lines are added + to an extrainfo document. Implements ticket 2497. + - When bandwidth accounting is enabled, be more generous with how + much bandwidth we'll use up before entering "soft hibernation". + Previously, we'd refuse new connections and circuits once we'd + used up 95% of our allotment. Now, we use up 95% of our allotment, + AND make sure that we have no more than 500MB (or 3 hours of + expected traffic, whichever is lower) remaining before we enter + soft hibernation. + - Relays now log the reason for publishing a new relay descriptor, + so we have a better chance of hunting down instances of bug 1810. + Resolves ticket 3252. + - Log a little more clearly about the times at which we're no longer + accepting new connections (e.g. due to hibernating). Resolves + bug 2181. + - When AllowSingleHopExits is set, print a warning to explain to the + relay operator why most clients are avoiding her relay. + - Send END_STREAM_REASON_NOROUTE in response to EHOSTUNREACH errors. + Clients before 0.2.1.27 didn't handle NOROUTE correctly, but such + clients are already deprecated because of security bugs. + + o Minor features (network statistics): + - Directory mirrors that set "DirReqStatistics 1" write statistics + about directory requests to disk every 24 hours. As compared to the + "--enable-geoip-stats" ./configure flag in 0.2.1.x, there are a few + improvements: 1) stats are written to disk exactly every 24 hours; + 2) estimated shares of v2 and v3 requests are determined as mean + values, not at the end of a measurement period; 3) unresolved + requests are listed with country code '??'; 4) directories also + measure download times. + - Exit nodes that set "ExitPortStatistics 1" write statistics on the + number of exit streams and transferred bytes per port to disk every + 24 hours. + - Relays that set "CellStatistics 1" write statistics on how long + cells spend in their circuit queues to disk every 24 hours. + - Entry nodes that set "EntryStatistics 1" write statistics on the + rough number and origins of connecting clients to disk every 24 + hours. + - Relays that write any of the above statistics to disk and set + "ExtraInfoStatistics 1" include the past 24 hours of statistics in + their extra-info documents. Implements proposal 166. + + o Minor features (GeoIP and statistics): + - Provide a log message stating which geoip file we're parsing + instead of just stating that we're parsing the geoip file. + Implements ticket 2432. + - Make sure every relay writes a state file at least every 12 hours. + Previously, a relay could go for weeks without writing its state + file, and on a crash could lose its bandwidth history, capacity + estimates, client country statistics, and so on. Addresses bug 3012. + - Relays report the number of bytes spent on answering directory + requests in extra-info descriptors similar to {read,write}-history. + Implements enhancement 1790. + - Report only the top 10 ports in exit-port stats in order not to + exceed the maximum extra-info descriptor length of 50 KB. Implements + task 2196. + - If writing the state file to disk fails, wait up to an hour before + retrying again, rather than trying again each second. Fixes bug + 2346; bugfix on Tor 0.1.1.3-alpha. + - Delay geoip stats collection by bridges for 6 hours, not 2 hours, + when we switch from being a public relay to a bridge. Otherwise + there will still be clients that see the relay in their consensus, + and the stats will end up wrong. Bugfix on 0.2.1.15-rc; fixes + bug 932. + - Update to the August 2 2011 Maxmind GeoLite Country database. + + o Minor features (clients): + - When expiring circuits, use microsecond timers rather than + one-second timers. This can avoid an unpleasant situation where a + circuit is launched near the end of one second and expired right + near the beginning of the next, and prevent fluctuations in circuit + timeout values. + - If we've configured EntryNodes and our network goes away and/or all + our entrynodes get marked down, optimistically retry them all when + a new socks application request appears. Fixes bug 1882. + - Always perform router selections using weighted relay bandwidth, + even if we don't need a high capacity circuit at the time. Non-fast + circuits now only differ from fast ones in that they can use relays + not marked with the Fast flag. This "feature" could turn out to + be a horrible bug; we should investigate more before it goes into + a stable release. + - When we run out of directory information such that we can't build + circuits, but then get enough that we can build circuits, log when + we actually construct a circuit, so the user has a better chance of + knowing what's going on. Fixes bug 1362. + - Log SSL state transitions at debug level during handshake, and + include SSL states in error messages. This may help debug future + SSL handshake issues. + + o Minor features (directory authorities): + - When a router changes IP address or port, authorities now launch + a new reachability test for it. Implements ticket 1899. + - Directory authorities now reject relays running any versions of + Tor between 0.2.1.3-alpha and 0.2.1.18 inclusive; they have + known bugs that keep RELAY_EARLY cells from working on rendezvous + circuits. Followup to fix for bug 2081. + - Directory authorities now reject relays running any version of Tor + older than 0.2.0.26-rc. That version is the earliest that fetches + current directory information correctly. Fixes bug 2156. + - Directory authorities now do an immediate reachability check as soon + as they hear about a new relay. This change should slightly reduce + the time between setting up a relay and getting listed as running + in the consensus. It should also improve the time between setting + up a bridge and seeing use by bridge users. + - Directory authorities no longer launch a TLS connection to every + relay as they startup. Now that we have 2k+ descriptors cached, + the resulting network hiccup is becoming a burden. Besides, + authorities already avoid voting about Running for the first half + hour of their uptime. + - Directory authorities now log the source of a rejected POSTed v3 + networkstatus vote, so we can track failures better. + - Backport code from 0.2.3.x that allows directory authorities to + clean their microdescriptor caches. Needed to resolve bug 2230. + + o Minor features (hidden services): + - Use computed circuit-build timeouts to decide when to launch + parallel introduction circuits for hidden services. (Previously, + we would retry after 15 seconds.) + - Don't allow v0 hidden service authorities to act as clients. + Required by fix for bug 3000. + - Ignore SIGNAL NEWNYM commands on relay-only Tor instances. Required + by fix for bug 3000. + - Make hidden services work better in private Tor networks by not + requiring any uptime to join the hidden service descriptor + DHT. Implements ticket 2088. + - Log (at info level) when purging pieces of hidden-service-client + state because of SIGNAL NEWNYM. + + o Minor features (controller interface): + - New "GETINFO net/listeners/(type)" controller command to return + a list of addresses and ports that are bound for listeners for a + given connection type. This is useful when the user has configured + "SocksPort auto" and the controller needs to know which port got + chosen. Resolves another part of ticket 3076. + - Have the controller interface give a more useful message than + "Internal Error" in response to failed GETINFO requests. + - Add a TIMEOUT_RATE keyword to the BUILDTIMEOUT_SET control port + event, to give information on the current rate of circuit timeouts + over our stored history. + - The 'EXTENDCIRCUIT' control port command can now be used with + a circ id of 0 and no path. This feature will cause Tor to build + a new 'fast' general purpose circuit using its own path selection + algorithms. + - Added a BUILDTIMEOUT_SET controller event to describe changes + to the circuit build timeout. + - New controller command "getinfo config-text". It returns the + contents that Tor would write if you send it a SAVECONF command, + so the controller can write the file to disk itself. + + o Minor features (controller protocol): + - Add a new ControlSocketsGroupWritable configuration option: when + it is turned on, ControlSockets are group-writeable by the default + group of the current user. Patch by Jérémy Bobbio; implements + ticket 2972. + - Tor now refuses to create a ControlSocket in a directory that is + world-readable (or group-readable if ControlSocketsGroupWritable + is 0). This is necessary because some operating systems do not + enforce permissions on an AF_UNIX sockets. Permissions on the + directory holding the socket, however, seems to work everywhere. + - Warn when CookieAuthFileGroupReadable is set but CookieAuthFile is + not. This would lead to a cookie that is still not group readable. + Closes bug 1843. Suggested by katmagic. + - Future-proof the controller protocol a bit by ignoring keyword + arguments we do not recognize. + + o Minor features (more useful logging): + - Revise most log messages that refer to nodes by nickname to + instead use the "$key=nickname at address" format. This should be + more useful, especially since nicknames are less and less likely + to be unique. Resolves ticket 3045. + - When an HTTPS proxy reports "403 Forbidden", we now explain + what it means rather than calling it an unexpected status code. + Closes bug 2503. Patch from Michael Yakubovich. + - Rate-limit a warning about failures to download v2 networkstatus + documents. Resolves part of bug 1352. + - Rate-limit the "your application is giving Tor only an IP address" + warning. Addresses bug 2000; bugfix on 0.0.8pre2. + - Rate-limit "Failed to hand off onionskin" warnings. + - When logging a rate-limited warning, we now mention how many messages + got suppressed since the last warning. + - Make the formerly ugly "2 unknown, 7 missing key, 0 good, 0 bad, + 2 no signature, 4 required" messages about consensus signatures + easier to read, and make sure they get logged at the same severity + as the messages explaining which keys are which. Fixes bug 1290. + - Don't warn when we have a consensus that we can't verify because + of missing certificates, unless those certificates are ones + that we have been trying and failing to download. Fixes bug 1145. + + o Minor features (log domains): + - Add documentation for configuring logging at different severities in + different log domains. We've had this feature since 0.2.1.1-alpha, + but for some reason it never made it into the manpage. Fixes + bug 2215. + - Make it simpler to specify "All log domains except for A and B". + Previously you needed to say "[*,~A,~B]". Now you can just say + "[~A,~B]". + - Add a "LogMessageDomains 1" option to include the domains of log + messages along with the messages. Without this, there's no way + to use log domains without reading the source or doing a lot + of guessing. + - Add a new "Handshake" log domain for activities that happen + during the TLS handshake. + + o Minor features (build process): + - Make compilation with clang possible when using + "--enable-gcc-warnings" by removing two warning options that clang + hasn't implemented yet and by fixing a few warnings. Resolves + ticket 2696. + - Detect platforms that brokenly use a signed size_t, and refuse to + build there. Found and analyzed by doorss and rransom. + - Fix a bunch of compile warnings revealed by mingw with gcc 4.5. + Resolves bug 2314. + - Add support for statically linking zlib by specifying + "--enable-static-zlib", to go with our support for statically + linking openssl and libevent. Resolves bug 1358. + - Instead of adding the svn revision to the Tor version string, report + the git commit (when we're building from a git checkout). + - Rename the "log.h" header to "torlog.h" so as to conflict with fewer + system headers. + - New --digests command-line switch to output the digests of the + source files Tor was built with. + - Generate our manpage and HTML documentation using Asciidoc. This + change should make it easier to maintain the documentation, and + produce nicer HTML. The build process fails if asciidoc cannot + be found and building with asciidoc isn't disabled (via the + "--disable-asciidoc" argument to ./configure. Skipping the manpage + speeds up the build considerably. + + o Minor features (options / torrc): + - Warn when the same option is provided more than once in a torrc + file, on the command line, or in a single SETCONF statement, and + the option is one that only accepts a single line. Closes bug 1384. + - Warn when the user configures two HiddenServiceDir lines that point + to the same directory. Bugfix on 0.0.6 (the version introducing + HiddenServiceDir); fixes bug 3289. + - Add new "perconnbwrate" and "perconnbwburst" consensus params to + do individual connection-level rate limiting of clients. The torrc + config options with the same names trump the consensus params, if + both are present. Replaces the old "bwconnrate" and "bwconnburst" + consensus params which were broken from 0.2.2.7-alpha through + 0.2.2.14-alpha. Closes bug 1947. + - New config option "WarnUnsafeSocks 0" disables the warning that + occurs whenever Tor receives a socks handshake using a version of + the socks protocol that can only provide an IP address (rather + than a hostname). Setups that do DNS locally over Tor are fine, + and we shouldn't spam the logs in that case. + - New config option "CircuitStreamTimeout" to override our internal + timeout schedule for how many seconds until we detach a stream from + a circuit and try a new circuit. If your network is particularly + slow, you might want to set this to a number like 60. + - New options for SafeLogging to allow scrubbing only log messages + generated while acting as a relay. Specify "SafeLogging relay" if + you want to ensure that only messages known to originate from + client use of the Tor process will be logged unsafely. + - Time and memory units in the configuration file can now be set to + fractional units. For example, "2.5 GB" is now a valid value for + AccountingMax. + - Support line continuations in the torrc config file. If a line + ends with a single backslash character, the newline is ignored, and + the configuration value is treated as continuing on the next line. + Resolves bug 1929. + + o Minor features (unit tests): + - Revise our unit tests to use the "tinytest" framework, so we + can run tests in their own processes, have smarter setup/teardown + code, and so on. The unit test code has moved to its own + subdirectory, and has been split into multiple modules. + - Add a unit test for cross-platform directory-listing code. + - Add some forgotten return value checks during unit tests. Found + by coverity. + - Use GetTempDir to find the proper temporary directory location on + Windows when generating temporary files for the unit tests. Patch + by Gisle Vanem. + + o Minor features (misc): + - The "torify" script now uses torsocks where available. + - Make Libevent log messages get delivered to controllers later, + and not from inside the Libevent log handler. This prevents unsafe + reentrant Libevent calls while still letting the log messages + get through. + - Certain Tor clients (such as those behind check.torproject.org) may + want to fetch the consensus in an extra early manner. To enable this + a user may now set FetchDirInfoExtraEarly to 1. This also depends on + setting FetchDirInfoEarly to 1. Previous behavior will stay the same + as only certain clients who must have this information sooner should + set this option. + - Expand homedirs passed to tor-checkkey. This should silence a + coverity complaint about passing a user-supplied string into + open() without checking it. + - Make sure to disable DirPort if running as a bridge. DirPorts aren't + used on bridges, and it makes bridge scanning somewhat easier. + - Create the /var/run/tor directory on startup on OpenSUSE if it is + not already created. Patch from Andreas Stieger. Fixes bug 2573. + + o Minor bugfixes (relays): + - When a relay decides that its DNS is too broken for it to serve + as an exit server, it advertised itself as a non-exit, but + continued to act as an exit. This could create accidental + partitioning opportunities for users. Instead, if a relay is + going to advertise reject *:* as its exit policy, it should + really act with exit policy "reject *:*". Fixes bug 2366. + Bugfix on Tor 0.1.2.5-alpha. Bugfix by user "postman" on trac. + - Publish a router descriptor even if generating an extra-info + descriptor fails. Previously we would not publish a router + descriptor without an extra-info descriptor; this can cause fast + exit relays collecting exit-port statistics to drop from the + consensus. Bugfix on 0.1.2.9-rc; fixes bug 2195. + - When we're trying to guess whether we know our IP address as + a relay, we would log various ways that we failed to guess + our address, but never log that we ended up guessing it + successfully. Now add a log line to help confused and anxious + relay operators. Bugfix on 0.1.2.1-alpha; fixes bug 1534. + - For bandwidth accounting, calculate our expected bandwidth rate + based on the time during which we were active and not in + soft-hibernation during the last interval. Previously, we were + also considering the time spent in soft-hibernation. If this + was a long time, we would wind up underestimating our bandwidth + by a lot, and skewing our wakeup time towards the start of the + accounting interval. Fixes bug 1789. Bugfix on 0.0.9pre5. + - Demote a confusing TLS warning that relay operators might get when + someone tries to talk to their ORPort. It is not the operator's + fault, nor can they do anything about it. Fixes bug 1364; bugfix + on 0.2.0.14-alpha. + - Change "Application request when we're believed to be offline." + notice to "Application request when we haven't used client + functionality lately.", to clarify that it's not an error. Bugfix + on 0.0.9.3; fixes bug 1222. + + o Minor bugfixes (bridges): + - When a client starts or stops using bridges, never use a circuit + that was built before the configuration change. This behavior could + put at risk a user who uses bridges to ensure that her traffic + only goes to the chosen addresses. Bugfix on 0.2.0.3-alpha; fixes + bug 3200. + - Do not reset the bridge descriptor download status every time we + re-parse our configuration or get a configuration change. Fixes + bug 3019; bugfix on 0.2.0.3-alpha. + - Users couldn't configure a regular relay to be their bridge. It + didn't work because when Tor fetched the bridge descriptor, it found + that it already had it, and didn't realize that the purpose of the + descriptor had changed. Now we replace routers with a purpose other + than bridge with bridge descriptors when fetching them. Bugfix on + 0.1.1.9-alpha. Fixes bug 1776. + - In the special case where you configure a public exit relay as your + bridge, Tor would be willing to use that exit relay as the last + hop in your circuit as well. Now we fail that circuit instead. + Bugfix on 0.2.0.12-alpha. Fixes bug 2403. Reported by "piebeer". + + o Minor bugfixes (clients): + - We now ask the other side of a stream (the client or the exit) + for more data on that stream when the amount of queued data on + that stream dips low enough. Previously, we wouldn't ask the + other side for more data until either it sent us more data (which + it wasn't supposed to do if it had exhausted its window!) or we + had completely flushed all our queued data. This flow control fix + should improve throughput. Fixes bug 2756; bugfix on the earliest + released versions of Tor (svn commit r152). + - When a client finds that an origin circuit has run out of 16-bit + stream IDs, we now mark it as unusable for new streams. Previously, + we would try to close the entire circuit. Bugfix on 0.0.6. + - Make it explicit that we don't cannibalize one-hop circuits. This + happens in the wild, but doesn't turn out to be a problem because + we fortunately don't use those circuits. Many thanks to outofwords + for the initial analysis and to swissknife who confirmed that + two-hop circuits are actually created. + - Resolve an edge case in path weighting that could make us misweight + our relay selection. Fixes bug 1203; bugfix on 0.0.8rc1. + - Make the DNSPort option work with libevent 2.x. Don't alter the + behaviour for libevent 1.x. Fixes bug 1143. Found by SwissTorExit. + + o Minor bugfixes (directory authorities): + - Make directory authorities more accurate at recording when + relays that have failed several reachability tests became + unreachable, so we can provide more accuracy at assigning Stable, + Guard, HSDir, etc flags. Bugfix on 0.2.0.6-alpha. Resolves bug 2716. + - Directory authorities are now more robust to hops back in time + when calculating router stability. Previously, if a run of uptime + or downtime appeared to be negative, the calculation could give + incorrect results. Bugfix on 0.2.0.6-alpha; noticed when fixing + bug 1035. + - Directory authorities will now attempt to download consensuses + if their own efforts to make a live consensus have failed. This + change means authorities that restart will fetch a valid + consensus, and it means authorities that didn't agree with the + current consensus will still fetch and serve it if it has enough + signatures. Bugfix on 0.2.0.9-alpha; fixes bug 1300. + - Never vote for a server as "Running" if we have a descriptor for + it claiming to be hibernating, and that descriptor was published + more recently than our last contact with the server. Bugfix on + 0.2.0.3-alpha; fixes bug 911. + - Directory authorities no longer change their opinion of, or vote on, + whether a router is Running, unless they have themselves been + online long enough to have some idea. Bugfix on 0.2.0.6-alpha. + Fixes bug 1023. + + o Minor bugfixes (hidden services): + - Log malformed requests for rendezvous descriptors as protocol + warnings, not warnings. Also, use a more informative log message + in case someone sees it at log level warning without prior + info-level messages. Fixes bug 2748; bugfix on 0.2.0.10-alpha. + - Accept hidden service descriptors if we think we might be a hidden + service directory, regardless of what our consensus says. This + helps robustness, since clients and hidden services can sometimes + have a more up-to-date view of the network consensus than we do, + and if they think that the directory authorities list us a HSDir, + we might actually be one. Related to bug 2732; bugfix on + 0.2.0.10-alpha. + - Correct the warning displayed when a rendezvous descriptor exceeds + the maximum size. Fixes bug 2750; bugfix on 0.2.1.5-alpha. Found by + John Brooks. + - Clients and hidden services now use HSDir-flagged relays for hidden + service descriptor downloads and uploads even if the relays have no + DirPort set and the client has disabled TunnelDirConns. This will + eventually allow us to give the HSDir flag to relays with no + DirPort. Fixes bug 2722; bugfix on 0.2.1.6-alpha. + - Only limit the lengths of single HS descriptors, even when multiple + HS descriptors are published to an HSDir relay in a single POST + operation. Fixes bug 2948; bugfix on 0.2.1.5-alpha. Found by hsdir. + + o Minor bugfixes (controllers): + - Allow GETINFO fingerprint to return a fingerprint even when + we have not yet built a router descriptor. Fixes bug 3577; + bugfix on 0.2.0.1-alpha. + - Send a SUCCEEDED stream event to the controller when a reverse + resolve succeeded. Fixes bug 3536; bugfix on 0.0.8pre1. Issue + discovered by katmagic. + - Remove a trailing asterisk from "exit-policy/default" in the + output of the control port command "GETINFO info/names". Bugfix + on 0.1.2.5-alpha. + - Make the SIGNAL DUMP controller command work on FreeBSD. Fixes bug + 2917. Bugfix on 0.1.1.1-alpha. + - When we restart our relay, we might get a successful connection + from the outside before we've started our reachability tests, + triggering a warning: "ORPort found reachable, but I have no + routerinfo yet. Failing to inform controller of success." This + bug was harmless unless Tor is running under a controller + like Vidalia, in which case the controller would never get a + REACHABILITY_SUCCEEDED status event. Bugfix on 0.1.2.6-alpha; + fixes bug 1172. + - When a controller changes TrackHostExits, remove mappings for + hosts that should no longer have their exits tracked. Bugfix on + 0.1.0.1-rc. + - When a controller changes VirtualAddrNetwork, remove any mappings + for hosts that were automapped to the old network. Bugfix on + 0.1.1.19-rc. + - When a controller changes one of the AutomapHosts* options, remove + any mappings for hosts that should no longer be automapped. Bugfix + on 0.2.0.1-alpha. + - Fix an off-by-one error in calculating some controller command + argument lengths. Fortunately, this mistake is harmless since + the controller code does redundant NUL termination too. Found by + boboper. Bugfix on 0.1.1.1-alpha. + - Fix a bug in the controller interface where "GETINFO ns/asdaskljkl" + would return "551 Internal error" rather than "552 Unrecognized key + ns/asdaskljkl". Bugfix on 0.1.2.3-alpha. + - Don't spam the controller with events when we have no file + descriptors available. Bugfix on 0.2.1.5-alpha. (Rate-limiting + for log messages was already solved from bug 748.) + - Emit a GUARD DROPPED controller event for a case we missed. + - Ensure DNS requests launched by "RESOLVE" commands from the + controller respect the __LeaveStreamsUnattached setconf options. The + same goes for requests launched via DNSPort or transparent + proxying. Bugfix on 0.2.0.1-alpha; fixes bug 1525. + + o Minor bugfixes (config options): + - Tor used to limit HttpProxyAuthenticator values to 48 characters. + Change the limit to 512 characters by removing base64 newlines. + Fixes bug 2752. Fix by Michael Yakubovich. + - Complain if PublishServerDescriptor is given multiple arguments that + include 0 or 1. This configuration will be rejected in the future. + Bugfix on 0.2.0.1-alpha; closes bug 1107. + - Disallow BridgeRelay 1 and ORPort 0 at once in the configuration. + Bugfix on 0.2.0.13-alpha; closes bug 928. + + o Minor bugfixes (log subsystem fixes): + - When unable to format an address as a string, report its value + as "???" rather than reusing the last formatted address. Bugfix + on 0.2.1.5-alpha. + - Be more consistent in our treatment of file system paths. "~" should + get expanded to the user's home directory in the Log config option. + Fixes bug 2971; bugfix on 0.2.0.1-alpha, which introduced the + feature for the -f and --DataDirectory options. + + o Minor bugfixes (memory management): + - Don't stack-allocate the list of supplementary GIDs when we're + about to log them. Stack-allocating NGROUPS_MAX gid_t elements + could take up to 256K, which is way too much stack. Found by + Coverity; CID #450. Bugfix on 0.2.1.7-alpha. + - Save a couple bytes in memory allocation every time we escape + certain characters in a string. Patch from Florian Zumbiehl. + + o Minor bugfixes (protocol correctness): + - When checking for 1024-bit keys, check for 1024 bits, not 128 + bytes. This allows Tor to correctly discard keys of length 1017 + through 1023. Bugfix on 0.0.9pre5. + - Require that introduction point keys and onion handshake keys + have a public exponent of 65537. Starts to fix bug 3207; bugfix + on 0.2.0.10-alpha. + - Handle SOCKS messages longer than 128 bytes long correctly, rather + than waiting forever for them to finish. Fixes bug 2330; bugfix + on 0.2.0.16-alpha. Found by doorss. + - Never relay a cell for a circuit we have already destroyed. + Between marking a circuit as closeable and finally closing it, + it may have been possible for a few queued cells to get relayed, + even though they would have been immediately dropped by the next + OR in the circuit. Fixes bug 1184; bugfix on 0.2.0.1-alpha. + - Never queue a cell for a circuit that's already been marked + for close. + - Fix a spec conformance issue: the network-status-version token + must be the first token in a v3 consensus or vote. Discovered by + "parakeep". Bugfix on 0.2.0.3-alpha. + - A networkstatus vote must contain exactly one signature. Spec + conformance issue. Bugfix on 0.2.0.3-alpha. + - When asked about a DNS record type we don't support via a + client DNSPort, reply with NOTIMPL rather than an empty + reply. Patch by intrigeri. Fixes bug 3369; bugfix on 2.0.1-alpha. + - Make more fields in the controller protocol case-insensitive, since + control-spec.txt said they were. + + o Minor bugfixes (log messages): + - Fix a log message that said "bits" while displaying a value in + bytes. Found by wanoskarnet. Fixes bug 3318; bugfix on + 0.2.0.1-alpha. + - Downgrade "no current certificates known for authority" message from + Notice to Info. Fixes bug 2899; bugfix on 0.2.0.10-alpha. + - Correctly describe errors that occur when generating a TLS object. + Previously we would attribute them to a failure while generating a + TLS context. Patch by Robert Ransom. Bugfix on 0.1.0.4-rc; fixes + bug 1994. + - Fix an instance where a Tor directory mirror might accidentally + log the IP address of a misbehaving Tor client. Bugfix on + 0.1.0.1-rc. + - Stop logging at severity 'warn' when some other Tor client tries + to establish a circuit with us using weak DH keys. It's a protocol + violation, but that doesn't mean ordinary users need to hear about + it. Fixes the bug part of bug 1114. Bugfix on 0.1.0.13. + - If your relay can't keep up with the number of incoming create + cells, it would log one warning per failure into your logs. Limit + warnings to 1 per minute. Bugfix on 0.0.2pre10; fixes bug 1042. + + o Minor bugfixes (build fixes): + - Fix warnings from GCC 4.6's "-Wunused-but-set-variable" option. + - When warning about missing zlib development packages during compile, + give the correct package names. Bugfix on 0.2.0.1-alpha. + - Fix warnings that newer versions of autoconf produce during + ./autogen.sh. These warnings appear to be harmless in our case, + but they were extremely verbose. Fixes bug 2020. + - Squash a compile warning on OpenBSD. Reported by Tas; fixes + bug 1848. + + o Minor bugfixes (portability): + - Write several files in text mode, on OSes that distinguish text + mode from binary mode (namely, Windows). These files are: + 'buffer-stats', 'dirreq-stats', and 'entry-stats' on relays + that collect those statistics; 'client_keys' and 'hostname' for + hidden services that use authentication; and (in the tor-gencert + utility) newly generated identity and signing keys. Previously, + we wouldn't specify text mode or binary mode, leading to an + assertion failure. Fixes bug 3607. Bugfix on 0.2.1.1-alpha (when + the DirRecordUsageByCountry option which would have triggered + the assertion failure was added), although this assertion failure + would have occurred in tor-gencert on Windows in 0.2.0.1-alpha. + - Selectively disable deprecation warnings on OS X because Lion + started deprecating the shipped copy of openssl. Fixes bug 3643. + - Use a wide type to hold sockets when built for 64-bit Windows. + Fixes bug 3270. + - Fix an issue that prevented static linking of libevent on + some platforms (notably Linux). Fixes bug 2698; bugfix on 0.2.1.23, + where we introduced the "--with-static-libevent" configure option. + - Fix a bug with our locking implementation on Windows that couldn't + correctly detect when a file was already locked. Fixes bug 2504, + bugfix on 0.2.1.6-alpha. + - Build correctly on OSX with zlib 1.2.4 and higher with all warnings + enabled. + - Fix IPv6-related connect() failures on some platforms (BSD, OS X). + Bugfix on 0.2.0.3-alpha; fixes first part of bug 2660. Patch by + "piebeer". + + o Minor bugfixes (code correctness): + - Always NUL-terminate the sun_path field of a sockaddr_un before + passing it to the kernel. (Not a security issue: kernels are + smart enough to reject bad sockaddr_uns.) Found by Coverity; + CID #428. Bugfix on Tor 0.2.0.3-alpha. + - Make connection_printf_to_buf()'s behaviour sane. Its callers + expect it to emit a CRLF iff the format string ends with CRLF; + it actually emitted a CRLF iff (a) the format string ended with + CRLF or (b) the resulting string was over 1023 characters long or + (c) the format string did not end with CRLF *and* the resulting + string was 1021 characters long or longer. Bugfix on 0.1.1.9-alpha; + fixes part of bug 3407. + - Make send_control_event_impl()'s behaviour sane. Its callers + expect it to always emit a CRLF at the end of the string; it + might have emitted extra control characters as well. Bugfix on + 0.1.1.9-alpha; fixes another part of bug 3407. + - Make crypto_rand_int() check the value of its input correctly. + Previously, it accepted values up to UINT_MAX, but could return a + negative number if given a value above INT_MAX+1. Found by George + Kadianakis. Fixes bug 3306; bugfix on 0.2.2pre14. + - Fix a potential null-pointer dereference while computing a + consensus. Bugfix on tor-0.2.0.3-alpha, found with the help of + clang's analyzer. + - If we fail to compute the identity digest of a v3 legacy keypair, + warn, and don't use a buffer-full of junk instead. Bugfix on + 0.2.1.1-alpha; fixes bug 3106. + - Resolve an untriggerable issue in smartlist_string_num_isin(), + where if the function had ever in the future been used to check + for the presence of a too-large number, it would have given an + incorrect result. (Fortunately, we only used it for 16-bit + values.) Fixes bug 3175; bugfix on 0.1.0.1-rc. + - Be more careful about reporting the correct error from a failed + connect() system call. Under some circumstances, it was possible to + look at an incorrect value for errno when sending the end reason. + Bugfix on 0.1.0.1-rc. + - Correctly handle an "impossible" overflow cases in connection byte + counting, where we write or read more than 4GB on an edge connection + in a single second. Bugfix on 0.1.2.8-beta. + - Avoid a double mark-for-free warning when failing to attach a + transparent proxy connection. Bugfix on 0.1.2.1-alpha. Fixes + bug 2279. + - Correctly detect failure to allocate an OpenSSL BIO. Fixes bug 2378; + found by "cypherpunks". This bug was introduced before the first + Tor release, in svn commit r110. + - Fix a bug in bandwidth history state parsing that could have been + triggered if a future version of Tor ever changed the timing + granularity at which bandwidth history is measured. Bugfix on + Tor 0.1.1.11-alpha. + - Add assertions to check for overflow in arguments to + base32_encode() and base32_decode(); fix a signed-unsigned + comparison there too. These bugs are not actually reachable in Tor, + but it's good to prevent future errors too. Found by doorss. + - Avoid a bogus overlapped memcpy in tor_addr_copy(). Reported by + "memcpyfail". + - Set target port in get_interface_address6() correctly. Bugfix + on 0.1.1.4-alpha and 0.2.0.3-alpha; fixes second part of bug 2660. + - Fix an impossible-to-actually-trigger buffer overflow in relay + descriptor generation. Bugfix on 0.1.0.15. + - Fix numerous small code-flaws found by Coverity Scan Rung 3. + + o Minor bugfixes (code improvements): + - After we free an internal connection structure, overwrite it + with a different memory value than we use for overwriting a freed + internal circuit structure. Should help with debugging. Suggested + by bug 1055. + - If OpenSSL fails to make a duplicate of a private or public key, log + an error message and try to exit cleanly. May help with debugging + if bug 1209 ever remanifests. + - Some options used different conventions for uppercasing of acronyms + when comparing manpage and source. Fix those in favor of the + manpage, as it makes sense to capitalize acronyms. + - Take a first step towards making or.h smaller by splitting out + function definitions for all source files in src/or/. Leave + structures and defines in or.h for now. + - Remove a few dead assignments during router parsing. Found by + coverity. + - Don't use 1-bit wide signed bit fields. Found by coverity. + - Avoid signed/unsigned comparisons by making SIZE_T_CEILING unsigned. + None of the cases where we did this before were wrong, but by making + this change we avoid warnings. Fixes bug 2475; bugfix on 0.2.1.28. + - The memarea code now uses a sentinel value at the end of each area + to make sure nothing writes beyond the end of an area. This might + help debug some conceivable causes of bug 930. + - Always treat failure to allocate an RSA key as an unrecoverable + allocation error. + - Add some more defensive programming for architectures that can't + handle unaligned integer accesses. We don't know of any actual bugs + right now, but that's the best time to fix them. Fixes bug 1943. + + o Minor bugfixes (misc): + - Fix a rare bug in rend_fn unit tests: we would fail a test when + a randomly generated port is 0. Diagnosed by Matt Edman. Bugfix + on 0.2.0.10-alpha; fixes bug 1808. + - Where available, use Libevent 2.0's periodic timers so that our + once-per-second cleanup code gets called even more closely to + once per second than it would otherwise. Fixes bug 943. + - Ignore OutboundBindAddress when connecting to localhost. + Connections to localhost need to come _from_ localhost, or else + local servers (like DNS and outgoing HTTP/SOCKS proxies) will often + refuse to listen. + - Update our OpenSSL 0.9.8l fix so that it works with OpenSSL 0.9.8m + too. + - If any of the v3 certs we download are unparseable, we should + actually notice the failure so we don't retry indefinitely. Bugfix + on 0.2.0.x; reported by "rotator". + - When Tor fails to parse a descriptor of any kind, dump it to disk. + Might help diagnosing bug 1051. + - Make our 'torify' script more portable; if we have only one of + 'torsocks' or 'tsocks' installed, don't complain to the user; + and explain our warning about tsocks better. + - Fix some urls in the exit notice file and make it XHTML1.1 strict + compliant. Based on a patch from Christian Kujau. + + o Documentation changes: + - Modernize the doxygen configuration file slightly. Fixes bug 2707. + - Resolve all doxygen warnings except those for missing documentation. + Fixes bug 2705. + - Add doxygen documentation for more functions, fields, and types. + - Convert the HACKING file to asciidoc, and add a few new sections + to it, explaining how we use Git, how we make changelogs, and + what should go in a patch. + - Document the default socks host and port (127.0.0.1:9050) for + tor-resolve. + - Removed some unnecessary files from the source distribution. The + AUTHORS file has now been merged into the people page on the + website. The roadmaps and design doc can now be found in the + projects directory in svn. + + o Deprecated and removed features (config): + - Remove the torrc.complete file. It hasn't been kept up to date + and users will have better luck checking out the manpage. + - Remove the HSAuthorityRecordStats option that version 0 hidden + service authorities could use to track statistics of overall v0 + hidden service usage. + - Remove the obsolete "NoPublish" option; it has been flagged + as obsolete and has produced a warning since 0.1.1.18-rc. + - Caches no longer download and serve v2 networkstatus documents + unless FetchV2Networkstatus flag is set: these documents haven't + haven't been used by clients or relays since 0.2.0.x. Resolves + bug 3022. + + o Deprecated and removed features (controller): + - The controller no longer accepts the old obsolete "addr-mappings/" + or "unregistered-servers-" GETINFO values. + - The EXTENDED_EVENTS and VERBOSE_NAMES controller features are now + always on; using them is necessary for correct forward-compatible + controllers. + + o Deprecated and removed features (misc): + - Hidden services no longer publish version 0 descriptors, and clients + do not request or use version 0 descriptors. However, the old hidden + service authorities still accept and serve version 0 descriptors + when contacted by older hidden services/clients. + - Remove undocumented option "-F" from tor-resolve: it hasn't done + anything since 0.2.1.16-rc. + - Remove everything related to building the expert bundle for OS X. + It has confused many users, doesn't work right on OS X 10.6, + and is hard to get rid of once installed. Resolves bug 1274. + - Remove support for .noconnect style addresses. Nobody was using + them, and they provided another avenue for detecting Tor users + via application-level web tricks. + - When we fixed bug 1038 we had to put in a restriction not to send + RELAY_EARLY cells on rend circuits. This was necessary as long + as relays using Tor 0.2.1.3-alpha through 0.2.1.18-alpha were + active. Now remove this obsolete check. Resolves bug 2081. + - Remove workaround code to handle directory responses from servers + that had bug 539 (they would send HTTP status 503 responses _and_ + send a body too). Since only server versions before + 0.2.0.16-alpha/0.1.2.19 were affected, there is no longer reason to + keep the workaround in place. + - Remove the old 'fuzzy time' logic. It was supposed to be used for + handling calculations where we have a known amount of clock skew and + an allowed amount of unknown skew. But we only used it in three + places, and we never adjusted the known/unknown skew values. This is + still something we might want to do someday, but if we do, we'll + want to do it differently. + - Remove the "--enable-iphone" option to ./configure. According to + reports from Marco Bonetti, Tor builds fine without any special + tweaking on recent iPhone SDK versions. + + Changes in version 0.2.1.30 - 2011-02-23 Tor 0.2.1.30 fixes a variety of less critical bugs. The main other change is a slight tweak to Tor's TLS handshake that makes relays @@ -1972,6 +3755,8 @@ Changes in version 0.2.0.30 - 2008-07-15 warning "-Wshorten-64-to-32" is available. - Support compilation to target iPhone; patch from cjacker huang. To build for iPhone, pass the --enable-iphone option to configure. + - Port Tor to build and run correctly on Windows CE systems, using + the wcecompat library. Contributed by Valerio Lupi. - Detect non-ASCII platforms (if any still exist) and refuse to build there: some of our code assumes that 'A' is 65 and so on. - Clear up some MIPSPro compiler warnings. diff --git a/acinclude.m4 b/acinclude.m4 index ccfecff28d..9ab684877a 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -2,6 +2,7 @@ dnl Helper macros for Tor configure.in dnl Copyright (c) 2001-2004, Roger Dingledine dnl Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson dnl Copyright (c) 2007-2008, Roger Dingledine, Nick Mathewson +dnl Copyright (c) 2007-2012, The Tor Project, Inc. dnl See LICENSE for licensing information AC_DEFUN([TOR_EXTEND_CODEPATH], @@ -41,6 +42,51 @@ AC_DEFUN([TOR_DEFINE_CODEPATH], AC_SUBST(TOR_LDFLAGS_$2) ]) +dnl 1:flags +AC_DEFUN([TOR_CHECK_CFLAGS], [ + AS_VAR_PUSHDEF([VAR],[tor_cv_cflags_$1]) + AC_CACHE_CHECK([whether the compiler accepts $1], VAR, [ + tor_saved_CFLAGS="$CFLAGS" + CFLAGS="$CFLAGS -pedantic -Werror $1" + AC_TRY_COMPILE([], [return 0;], + [AS_VAR_SET(VAR,yes)], + [AS_VAR_SET(VAR,no)]) + CFLAGS="$tor_saved_CFLAGS" + ]) + if test x$VAR = xyes; then + CFLAGS="$CFLAGS $1" + fi + AS_VAR_POPDEF([VAR]) +]) + +dnl 1:flags +dnl 2:extra ldflags +dnl 3:extra libraries +AC_DEFUN([TOR_CHECK_LDFLAGS], [ + AS_VAR_PUSHDEF([VAR],[tor_cv_ldflags_$1]) + AC_CACHE_CHECK([whether the linker accepts $1], VAR, [ + tor_saved_CFLAGS="$CFLAGS" + tor_saved_LDFLAGS="$LDFLAGS" + tor_saved_LIBS="$LIBS" + CFLAGS="$CFLAGS -pedantic -Werror" + LDFLAGS="$LDFLAGS $2 $1" + LIBS="$LIBS $3" + AC_RUN_IFELSE([AC_LANG_PROGRAM([#include <stdio.h>], [fputs("", stdout)])], + [AS_VAR_SET(VAR,yes)], + [AS_VAR_SET(VAR,no)], + [AC_TRY_LINK([], [return 0;], + [AS_VAR_SET(VAR,yes)], + [AS_VAR_SET(VAR,no)])]) + CFLAGS="$tor_saved_CFLAGS" + LDFLAGS="$tor_saved_LDFLAGS" + LIBS="$tor_saved_LIBS" + ]) + if test x$VAR = xyes; then + LDFLAGS="$LDFLAGS $1" + fi + AS_VAR_POPDEF([VAR]) +]) + dnl 1:libname AC_DEFUN([TOR_WARN_MISSING_LIB], [ h="" diff --git a/autogen.sh b/autogen.sh index eb9395c719..0592f16c2e 100755 --- a/autogen.sh +++ b/autogen.sh @@ -1,5 +1,9 @@ #!/bin/sh +if [ -x "`which autoreconf 2>/dev/null`" ] ; then + exec autoreconf -ivf +fi + set -e # Run this to generate all the initial makefiles, etc. diff --git a/changes/abandon-rend-circs-on-newnym b/changes/abandon-rend-circs-on-newnym deleted file mode 100644 index 67cb2dce2f..0000000000 --- a/changes/abandon-rend-circs-on-newnym +++ /dev/null @@ -1,8 +0,0 @@ - o Security fixes: - - Don't attach new streams to old rendezvous circuits after SIGNAL - NEWNYM. Previously, we would keep using an existing rendezvous - circuit if it remained open (i.e. if it were kept open by a - long-lived stream or if a new stream were attached to it before - Tor could notice that it was old and no longer in use and close - it). Bugfix on 0.1.1.15-rc; fixes bug 3375. - diff --git a/changes/bridgepassword b/changes/bridgepassword deleted file mode 100644 index 5f0e250ff6..0000000000 --- a/changes/bridgepassword +++ /dev/null @@ -1,11 +0,0 @@ - o Security fixes: - - When using the debuging BridgePassword field, a bridge authority - now compares alleged passwords by hashing them, then comparing - the result to a digest of the expected authenticator. This avoids - a potential side-channel attack in the previous code, which - had foolishly used strcmp(). Fortunately, the BridgePassword field - *is not in use*, but if it had been, the timing - behavior of strcmp() might have allowed an adversary to guess the - BridgePassword value, and enumerate the bridges. Bugfix on - 0.2.0.14-alpha. Fixes bug 5543. - diff --git a/changes/buffer_bug b/changes/buffer_bug deleted file mode 100644 index 634f609533..0000000000 --- a/changes/buffer_bug +++ /dev/null @@ -1,7 +0,0 @@ - - o Major bugfixes: - - Fix a heap overflow bug that could occur when trying to pull - data into the first chunk of a buffer, when that chunk had - already had some data drained from it. Fixes CVE-2011-2778; - bugfix on 0.2.0.16-alpha. Reported by "Vektor". - diff --git a/changes/bug1240 b/changes/bug1240 deleted file mode 100644 index 657066491c..0000000000 --- a/changes/bug1240 +++ /dev/null @@ -1,8 +0,0 @@ - o Minor bugfixes: - - When running with an older Linux kernel that erroneously responds - to strange nmap behavior by having accept() return successfully - with a zero-length socket, just close the connection. Previously, - we would try harder to learn the remote address: but there was no - such remote address to learn, and our method for trying to learn - it was incorrect. Fixes bugs #1240, #4745, and #4747. Bugfix on - 0.1.0.3-rc. Reported and diagnosed by "r1eo". diff --git a/changes/bug1297a b/changes/bug1297a deleted file mode 100644 index 140b94e3b0..0000000000 --- a/changes/bug1297a +++ /dev/null @@ -1,16 +0,0 @@ - o Major bugfixes: - - Apply circuit timeouts to opened hidden-service-related circuits - based on the correct start time. Previously, we would apply the - circuit build timeout based on time since the circuit's - creation; it was supposed to be applied based on time since the - circuit entered its current state. Bugfix on 0.0.6; fixes part - of bug 1297. - - Use the same circuit timeout for client-side introduction - circuits as for other four-hop circuits. Previously, - client-side introduction circuits were closed after the same - timeout as single-hop directory-fetch circuits; this was - appropriate with the static circuit build timeout in 0.2.1.x and - earlier, but caused many hidden service access attempts to fail - with the adaptive CBT introduced in 0.2.2.2-alpha. Bugfix on - 0.2.2.2-alpha; fixes another part of bug 1297. - diff --git a/changes/bug1345 b/changes/bug1345 deleted file mode 100644 index 0c9375a35d..0000000000 --- a/changes/bug1345 +++ /dev/null @@ -1,13 +0,0 @@ - o Minor bugfixes: - - On SIGHUP, do not clear out all TrackHostExits mappings, client DNS - cache entries, and virtual address mappings: that's what NEWNYM is - for. Bugfix on Tor 0.1.0.1-rc; fixes bug 1345. - - When TrackHostExits is changed from a controller, remove any - mappings for hosts that should no longer have their exits tracked. - Bugfix on Tor 0.1.0.1-rc. - - When VirtualAddrNetwork option is changed from a controller, - remove any mappings for hosts that were automapped to - that network. Bugfix on 0.1.1.19-rc. - - When one of the AutomapHosts* options is changed from a - controller, remove any mappings for hosts that should no longer be - automapped. Bugfix on 0.2.0.1-alpha. diff --git a/changes/bug1352 b/changes/bug1352 deleted file mode 100644 index bde0192401..0000000000 --- a/changes/bug1352 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features - - Rate-limit a warning about failures to download v2 networkstatus - documents. Resolves part of bug 1352. - diff --git a/changes/bug1810 b/changes/bug1810 deleted file mode 100644 index 11e561f7cf..0000000000 --- a/changes/bug1810 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes: - - Don't decide to make a new descriptor when receiving a HUP signal. - This bug has caused a lot of relays to disappear from the consensus - periodically. Fixes the most common case of triggering bug 1810; - bugfix on 0.2.2.7-alpha. - diff --git a/changes/bug2355 b/changes/bug2355 deleted file mode 100644 index ee0ae4b96a..0000000000 --- a/changes/bug2355 +++ /dev/null @@ -1,8 +0,0 @@ - o Major features: - - If "UseBridges 1" is set and no bridges are configured, Tor will - now refuse to build any circuits until some bridges are set. - If "UseBridges auto" is set, Tor will use bridges if they are - configured and we are not running as a server, but otherwise - will make circuits as usual. The new default is "auto". Patch - by anonym. - diff --git a/changes/bug2355_revert b/changes/bug2355_revert deleted file mode 100644 index 2ded40ad8e..0000000000 --- a/changes/bug2355_revert +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes: - - Revert the UseBridges option to its behavior before 0.2.2.28-beta. - When we changed the default behavior to "use bridges if any are - listed in the torrc", we broke a number of users who had bridges - in their torrc files but who didn't actually want to use them. - Partial resolution for bug 3354. - diff --git a/changes/bug2442 b/changes/bug2442 deleted file mode 100644 index cbcc22bb80..0000000000 --- a/changes/bug2442 +++ /dev/null @@ -1,8 +0,0 @@ - * Minor bugfixes: - - - Demote the 'replay detected' log message emitted when a hidden - service receives the same Diffie-Hellman public key in two - different INTRODUCE2 cells to info level. A normal Tor client - can cause that log message during its normal operation. Bugfix - on 0.2.1.6-alpha; fixes part of bug 2442. - diff --git a/changes/bug2442b b/changes/bug2442b deleted file mode 100644 index 02e1636e91..0000000000 --- a/changes/bug2442b +++ /dev/null @@ -1,8 +0,0 @@ - * Minor bugfixes: - - - Demote the 'INTRODUCE2 cell is too {old,new}' log message to - info level. There is nothing that a hidden service's operator - can do to fix its clients' clocks. Bugfix on 0.2.1.6-alpha; - fixes part of bug 2442. - - diff --git a/changes/bug2503 b/changes/bug2503 deleted file mode 100644 index 50b8bf50c2..0000000000 --- a/changes/bug2503 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features: - - When an HTTPS proxy reports "403 Forbidden", we now explain - what it means rather than calling it an unexpected status code. - Closes bug 2503. Patch from "mikey". diff --git a/changes/bug2574 b/changes/bug2574 deleted file mode 100644 index 5cf2daebfa..0000000000 --- a/changes/bug2574 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Allow nameservers with IPv6 address. Fixes bug 2574. - diff --git a/changes/bug2649a b/changes/bug2649a deleted file mode 100644 index 4ee31ebdb6..0000000000 --- a/changes/bug2649a +++ /dev/null @@ -1,5 +0,0 @@ - o Minor features: - - Add a VoteOnHidServDirectoriesV2 configuration option to allow - directory authorities to abstain from voting on assignment of - the HSDir consensus flag. Related to bug 2649. - diff --git a/changes/bug2649b b/changes/bug2649b deleted file mode 100644 index 1ff14e5569..0000000000 --- a/changes/bug2649b +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Change the default required uptime for a relay to be accepted as - a HSDir from 24 hours to 25 hours. Bugfix on 0.2.0.10-alpha; - fixes bug 2649. - diff --git a/changes/bug2732-simple b/changes/bug2732-simple deleted file mode 100644 index 367836152d..0000000000 --- a/changes/bug2732-simple +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes - - Do not reject hidden service descriptors simply because we don't - think we have not been assigned the HSDir flag. Clients and - hidden services can have a more up-to-date view of the network - consensus, and if they think that the directory authorities - list us a HSDir, we might actually be one. Related to bug 2732; - bugfix on 0.2.0.10-alpha. diff --git a/changes/bug2748 b/changes/bug2748 deleted file mode 100644 index b522560a92..0000000000 --- a/changes/bug2748 +++ /dev/null @@ -1,10 +0,0 @@ - o Minor bugfixes - - Remove dead code from rend_cache_lookup_v2_desc_as_dir. Fixes - part of bug 2748; bugfix on 0.2.0.10-alpha. - - Log malformed requests for rendezvous descriptors as protocol - warnings, not warnings. Also, use a more informative log - message in case someone sees it at log level warning without - prior info-level messages. Fixes the other part of bug 2748; - bugfix on 0.2.0.10-alpha. - - diff --git a/changes/bug2752 b/changes/bug2752 deleted file mode 100644 index b872d3374a..0000000000 --- a/changes/bug2752 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor features: - - Tor used to limit HttpProxyAuthenticator values to 48 characters. - Changed the limit to 512 characters by removing base64 newlines. - Fixes bug 2752. Fix by Michael Yakubovich. - diff --git a/changes/bug2792_checkdir b/changes/bug2792_checkdir deleted file mode 100644 index 10de1deb2d..0000000000 --- a/changes/bug2792_checkdir +++ /dev/null @@ -1,8 +0,0 @@ - o Minor features: - - Tor now refuses to create a ControlSocket in a directory that is - world-readable (or group-readable if ControlSocketsGroupWritable - is 0). This is necessary because some operating systems do not - check the permissions on an AF_UNIX socket when programs try to - connect to it. Checking permissions on the directory holding - the socket, however, seems to work everywhere. - diff --git a/changes/bug2850 b/changes/bug2850 deleted file mode 100644 index 77ccbfa25d..0000000000 --- a/changes/bug2850 +++ /dev/null @@ -1,5 +0,0 @@ - - Minor features - o Set SO_REUSEADDR on all sockets, not just listeners. This should - help busy exit nodes avoid running out of useable ports just because - all the ports have been used in the near past. Resolves issue 2850. - diff --git a/changes/bug2972 b/changes/bug2972 deleted file mode 100644 index 26afcca421..0000000000 --- a/changes/bug2972 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor features: - - Allow ControlSockets to be group-writable when the - ControlSocksGroupWritable configuration option is turned on. Patch - by Jérémy Bobbio; implements ticket 2972. - diff --git a/changes/bug3019 b/changes/bug3019 deleted file mode 100644 index 4df709fb3b..0000000000 --- a/changes/bug3019 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Do not reset the bridge descriptor download status every time we - re-parse our configuration or get a configuration change. Fixes - bug 3019; bugfix on Tor 0.2.0.3-alpha. diff --git a/changes/bug3022 b/changes/bug3022 deleted file mode 100644 index 9472e6d196..0000000000 --- a/changes/bug3022 +++ /dev/null @@ -1,6 +0,0 @@ - o Removed features - - Caches no longer download and serve v2 networkstatus documents - unless FetchV2Networkstatus flag is set: these documents haven't - haven't been used by clients or relays since 0.2.0.x. Resolves - bug 3022. - diff --git a/changes/bug3026 b/changes/bug3026 deleted file mode 100644 index c0c0a3860a..0000000000 --- a/changes/bug3026 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (directory authority) - - Do not upload our own vote or signature set to ourself. It would - tell us nothing new. Also, as of Tor 0.2.2.24-alpha, we started - to warn about receiving duplicate votes. Resolves bug 3026. diff --git a/changes/bug3045 b/changes/bug3045 deleted file mode 100644 index 1cbcabaff6..0000000000 --- a/changes/bug3045 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor features: - - Revise most log messages that refer to nodes by nickname to - instead use the "$key=nickname at address" format. This should be - more useful, especially since nicknames are less and less likely - to be unique. Fixes bug 3045. - diff --git a/changes/bug3122_memcmp b/changes/bug3122_memcmp deleted file mode 100644 index a049476743..0000000000 --- a/changes/bug3122_memcmp +++ /dev/null @@ -1,7 +0,0 @@ - o Security fixes - - Replace all potentially sensitive memory comparison operations - with versions whose runtime does not depend on the data being - compared. This will help resist a class of attacks where an - adversary can use variations in timing information to learn - sensitive data. Fix for one case of bug 3122. (Safe memcmp - implementation by Robert Ransom based partially on code by DJB.) diff --git a/changes/bug3135 b/changes/bug3135 deleted file mode 100644 index d761123480..0000000000 --- a/changes/bug3135 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes - - Do not crash when our configuration file becomes unreadable - (usually due to a permissions change) between when we start - up and when a controller calls SAVECONF. Fixes bug 3135; - bugfix on 0.0.9pre6. - diff --git a/changes/bug3175 b/changes/bug3175 deleted file mode 100644 index 3360fbce00..0000000000 --- a/changes/bug3175 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes: - - Resolve an untriggerable issue in smartlist_string_num_isin(), - where if the function had ever in the future been used to check - for the presence of a too-large number, it would have given an - incorrect result. (Fortunately, we only used it for 16-bit - values.) Fixes bug 3175; bugfix on Tor 0.1.0.1-rc. - diff --git a/changes/bug3198 b/changes/bug3198 deleted file mode 100644 index 29c16852e1..0000000000 --- a/changes/bug3198 +++ /dev/null @@ -1,4 +0,0 @@ - o Major bugfixes: - - When we configure a new bridge via the controller, don't wait up - to ten seconds before trying to fetch its descriptor. Bugfix on - 0.2.0.3-alpha; fixes bug 3198 (suggested by 2355). diff --git a/changes/bug3200 b/changes/bug3200 deleted file mode 100644 index a80d51633e..0000000000 --- a/changes/bug3200 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes: - - When a client starts or stops using bridges, never use a circuit - that was built before the configuration change. This behavior could - put at risk a user who uses bridges to ensure that her traffic - only goes to the chosen addresses. Bugfix on 0.2.0.3-alpha; fixes - bug 3200. diff --git a/changes/bug3207 b/changes/bug3207 deleted file mode 100644 index 65a7dac1ab..0000000000 --- a/changes/bug3207 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Require that onion keys have exponent 65537 in microdescriptors too. - Fixes part of bug 3207; bugfix on 0.2.2.25-alpha - diff --git a/changes/bug3208 b/changes/bug3208 deleted file mode 100644 index fd737ba695..0000000000 --- a/changes/bug3208 +++ /dev/null @@ -1,6 +0,0 @@ - o Removed options: - - Remove undocumented option "-F" from tor-resolve: it hasn't done - anything since 0.2.1.16-rc. - - o Minor bugfixes: - - Fix warnings from GCC 4.6's "-Wunused-but-set-variable" option. diff --git a/changes/bug3213 b/changes/bug3213 deleted file mode 100644 index ab7de2d629..0000000000 --- a/changes/bug3213 +++ /dev/null @@ -1,4 +0,0 @@ - o Major bugfixes: - - Fix a crash bug when changing bridges in a running Tor process. - Fixes bug 3213; bugfix on 0.2.2.26-beta. - diff --git a/changes/bug3216 b/changes/bug3216 deleted file mode 100644 index 599b5e162f..0000000000 --- a/changes/bug3216 +++ /dev/null @@ -1,4 +0,0 @@ - o Major bugfixes: - - Don't try to build descriptors if "ORPort auto" is set and we - don't know our actual ORPort yet. Fix for bug 3216; bugfix on - 0.2.2.26-beta. diff --git a/changes/bug3228 b/changes/bug3228 deleted file mode 100644 index 4aca810d3c..0000000000 --- a/changes/bug3228 +++ /dev/null @@ -1,3 +0,0 @@ - o Major bugfixes: - - Resolve a crash that occured when setting BridgeRelay to 1 with - accounting enabled. Fixes bug 3228; bugfix on 0.2.2.18-alpha. diff --git a/changes/bug3252 b/changes/bug3252 deleted file mode 100644 index f85f633fbd..0000000000 --- a/changes/bug3252 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features: - - Relays now log the reason for publishing a new relay descriptor, - so we have a better chance of hunting down the root cause of bug - 1810. Resolves ticket 3252. diff --git a/changes/bug3270 b/changes/bug3270 deleted file mode 100644 index b37bb983cc..0000000000 --- a/changes/bug3270 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes - - Use a wide type to hold sockets when built for 64-bit Windows builds. - Fixes bug 3270. - diff --git a/changes/bug3289 b/changes/bug3289 deleted file mode 100644 index c469796d6e..0000000000 --- a/changes/bug3289 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Warn when the user configures two HiddenServiceDir lines that point - to the same directory. Bugfix on 0.0.6 (the version introducing - HiddenServiceDir); fixes bug 3289. - diff --git a/changes/bug3306 b/changes/bug3306 deleted file mode 100644 index f868a24af0..0000000000 --- a/changes/bug3306 +++ /dev/null @@ -1,9 +0,0 @@ - o Minor bugfixes: - - Make our crypto_rand_int() function check the value of its input - correctly. Previously, it accepted values up to UINT_MAX, but - could return a negative number if given a value above INT_MAX+1. - Found by George Kadianakis. Fixes bug 3306; bugfix on 0.2.2pre14. - - - Avoid a segfault when reading a malformed circuit build state - with more than INT_MAX entries. Found by wanoskarnet. Bugfix on - 0.2.2.4-alpha. diff --git a/changes/bug3309 b/changes/bug3309 deleted file mode 100644 index 104056d8e3..0000000000 --- a/changes/bug3309 +++ /dev/null @@ -1,13 +0,0 @@ - o Minor bugfixes: - - Clear the table recording the time of the last request for each - hidden service descriptor from each HS directory on SIGNAL - NEWNYM. Previously, we would clear our HS descriptor cache on - SIGNAL NEWNYM, but if we had previously retrieved a descriptor - (or tried to) from every directory responsible for it, we would - refuse to fetch it again for up to 15 minutes. Bugfix on - 0.2.2.25-alpha; fixes bug 3309. - - o Minor features: - - Log (at info level) when purging pieces of hidden-service-client - state on SIGNAL NEWNYM. - diff --git a/changes/bug3318 b/changes/bug3318 deleted file mode 100644 index 8a3c27825f..0000000000 --- a/changes/bug3318 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes: - - Fix a log message that said "bits" while displaying a value in - bytes. Found by wanoskarnet. Fixes bug 3318; bugfix on - 0.2.0.1-alpha. - - When checking for 1024-bit keys, check for 1024 bits, not 128 - bytes. This allows Tor to correctly discard keys of length - 1017 through 1023. Bugfix on 0.0.9pre5. diff --git a/changes/bug3321 b/changes/bug3321 deleted file mode 100644 index 3605efce2d..0000000000 --- a/changes/bug3321 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes: - - In bug 2511 we fixed a case where you could use an unconfigured - bridge if you had configured it as a bridge the last time you ran - Tor. Now fix another edge case: if you had configured it as a bridge - but then switched to a different bridge via the controller, you - would still be willing to use the old one. Bugfix on 0.2.0.1-alpha; - fixes bug 3321. diff --git a/changes/bug3369 b/changes/bug3369 deleted file mode 100644 index 9c0d0e699a..0000000000 --- a/changes/bug3369 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - When asked about a DNS record type we don't support via a - client DNSPort, reply with NOTIMPL rather than an empty - reply. Patch by intrigeri. Fixes bug 3369; bugfix on 2.0.1-alpha. diff --git a/changes/bug3393 b/changes/bug3393 deleted file mode 100644 index 677bcb7be2..0000000000 --- a/changes/bug3393 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Fix a bug when using ControlSocketsGroupWritable with User. The - directory's group would be checked against the current group, not - the configured group. Patch by Jérémy Bobbio. Fixes bug3393; bugfix - on Tor 0.2.2.26-beta.
\ No newline at end of file diff --git a/changes/bug3465-022 b/changes/bug3465-022 deleted file mode 100644 index 2d226162aa..0000000000 --- a/changes/bug3465-022 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes: - - - Add BUILDTIMEOUT_SET to the list returned by the 'GETINFO - events/names' control-port command. Bugfix on 0.2.2.9-alpha; - fixes part of bug 3465. - diff --git a/changes/bug3536 b/changes/bug3536 deleted file mode 100644 index d3cec131ba..0000000000 --- a/changes/bug3536 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Send a SUCCEEDED stream event to the controller when a reverse - resolve succeeded. Fixes bug 3536; bugfix on 0.0.8pre1. Issue - discovered by katmagic. - diff --git a/changes/bug3577 b/changes/bug3577 deleted file mode 100644 index 6335272752..0000000000 --- a/changes/bug3577 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Allow GETINFO fingerprint to return a fingerprint even when - we have not yet built a router descriptor. Fixes bug 3577; - bugfix on 0.2.0.1-alpha. diff --git a/changes/bug3607 b/changes/bug3607 deleted file mode 100644 index 5ece21934b..0000000000 --- a/changes/bug3607 +++ /dev/null @@ -1,15 +0,0 @@ - o Minor bugfixes: - - - Write several files in text mode, on OSes that distinguish text - mode from binary mode (namely, Windows). These files are: - buffer-stats, dirreq-stats, and entry-stats on relays that collect - those statistics; client_keys and hostname files for hidden - services that use authentication; and (in the tor-gencert utility) - newly generated identity and signing keys. Previously, we - wouldn't specify text mode or binary mode, leading to an assertion - failure. Fixes bug 3607. Bugfix on 0.2.1.1-alpha (when the - DirRecordUsageByCountry option which would have triggered the - assertion failure was added), although this assertion failure - would have occurred in tor-gencert on Windows in 0.2.0.1-alpha. - - diff --git a/changes/bug3643 b/changes/bug3643 deleted file mode 100644 index 86bd920cac..0000000000 --- a/changes/bug3643 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Selectively disable deprecation warnings on OS X because Lion started - deprecating the shipped copy of openssl. Fixes bug 3643. - diff --git a/changes/bug3732 b/changes/bug3732 deleted file mode 100644 index 7a71d1aef3..0000000000 --- a/changes/bug3732 +++ /dev/null @@ -1,7 +0,0 @@ - o Major bugfixes: - - - Remove an extra pair of quotation marks around the error - message in control-port STATUS_GENERAL BUG events. Bugfix on - 0.1.2.6-alpha; fixes bug 3732. - - diff --git a/changes/bug3747 b/changes/bug3747 deleted file mode 100644 index 052dab1bd0..0000000000 --- a/changes/bug3747 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes: - - Write control ports to disk only after switching UID and - creating the data directory. This way, we don't fail when - starting up with a nonexistant DataDirectory and a - ControlPortWriteToFile setting based on that directory. Fixes - bug 3747; bugfix on Tor 0.2.2.26-beta.
\ No newline at end of file diff --git a/changes/bug3894 b/changes/bug3894 deleted file mode 100644 index 4c2220aba8..0000000000 --- a/changes/bug3894 +++ /dev/null @@ -1,4 +0,0 @@ - o Build fixes: - - Clean up some code issues that prevented Tor from building on older - BSDs. Fixes bug 3894; reported by grarpamp. - diff --git a/changes/bug3898a b/changes/bug3898a deleted file mode 100644 index d40445e340..0000000000 --- a/changes/bug3898a +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes: - - Correct the man page to explain that HashedControlPassword and - CookieAuthentication can both be set, in which case either method - is sufficient to authenticate to Tor. Bugfix on 0.2.0.7-alpha, - when we decided to allow these config options to both be set. Issue - raised by bug 3898. diff --git a/changes/bug3909 b/changes/bug3909 deleted file mode 100644 index 0b4b292030..0000000000 --- a/changes/bug3909 +++ /dev/null @@ -1,3 +0,0 @@ - o Build fixes: - - Search for a platform-specific version of "ar" when cross-compiling. - Should fix builds on iOS. Found by Marco Bonetti. diff --git a/changes/bug3923 b/changes/bug3923 deleted file mode 100644 index 9c0e138826..0000000000 --- a/changes/bug3923 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfies: - - Avoid an assertion failure when reloading a configuration with - TrackExitHosts changes. Found and fixed by 'laruldan'. Fixes - bug 3923; bugfix on 0.2.2.25-alpha. - diff --git a/changes/bug3963 b/changes/bug3963 deleted file mode 100644 index 2fc44a095c..0000000000 --- a/changes/bug3963 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - When configuring, starting, or stopping an NT service, stop - immediately after the service configuration attempt has succeeded - or failed. Fixes bug3963; bugfix on 0.2.0.7-alpha. - diff --git a/changes/bug4012_022 b/changes/bug4012_022 deleted file mode 100644 index f101db5535..0000000000 --- a/changes/bug4012_022 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (documentation): - - Document the GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays - directory authority option (introduced in Tor 0.2.2.34). diff --git a/changes/bug4014 b/changes/bug4014 deleted file mode 100644 index 9c20c6c337..0000000000 --- a/changes/bug4014 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Adjust the expiration time on our SSL session certificates to - better match SSL certs seen in the wild. Resolves ticket 4014. diff --git a/changes/bug4059 b/changes/bug4059 deleted file mode 100644 index 82a4b1a10c..0000000000 --- a/changes/bug4059 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Change an integer overflow check in the OpenBSD_Malloc code so - that GCC is less likely to eliminate it as impossible. Patch - from Mansour Moufid. Fixes bug 4059. - diff --git a/changes/bug4115 b/changes/bug4115 deleted file mode 100644 index 626791a806..0000000000 --- a/changes/bug4115 +++ /dev/null @@ -1,7 +0,0 @@ - o Security fixes: - - Bridge relays now do their directory fetches inside Tor TLS - connections, like all the other clients do, rather than connecting - directly to the DirPort like public relays do. Removes another - avenue for enumerating bridges. Fixes part of bug 4115; bugfix - on 0.2.0.35. - diff --git a/changes/bug4124 b/changes/bug4124 deleted file mode 100644 index abe93ccdd8..0000000000 --- a/changes/bug4124 +++ /dev/null @@ -1,6 +0,0 @@ - o Security fixes: - - Bridges relays now build circuits for themselves in a more similar - way to how clients build them. Removes another avenue for - enumerating bridges. Fixes bug 4124; bugfix on 0.2.0.3-alpha, - when bridges were introduced. - diff --git a/changes/bug4201 b/changes/bug4201 deleted file mode 100644 index 6f7d715af2..0000000000 --- a/changes/bug4201 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Bridges now skip DNS self-tests, to act a little more stealthily. - Fixes bug 4201; bugfix on 0.2.0.3-alpha, which first introduced - bridges. Patch by "warms0x". - diff --git a/changes/bug4212 b/changes/bug4212 deleted file mode 100644 index 6222a59978..0000000000 --- a/changes/bug4212 +++ /dev/null @@ -1,13 +0,0 @@ - o Major bugfixes: - - - Don't launch a useless circuit after failing to use one of a - hidden service's introduction points. Previously, we would - launch a new introduction circuit, but not set the hidden - service which that circuit was intended to connect to, so it - would never actually be used. A different piece of code would - then create a new introduction circuit correctly, so this bug - was harmless until it caused an assertion in the client-side - part of the #3825 fix to fail. Bug reported by katmagic and - found by Sebastian Hahn. Bugfix on 0.2.1.13-alpha; fixes bug - 4212. - diff --git a/changes/bug4230 b/changes/bug4230 deleted file mode 100644 index c1ba5847fc..0000000000 --- a/changes/bug4230 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Resolve an integer overflow bug in smartlist_ensure_capacity. - Fixes bug 4230; bugfix on Tor 0.1.0.1-rc. Based on a patch by - Mansour Moufid. - diff --git a/changes/bug4251 b/changes/bug4251 deleted file mode 100644 index 303c9e6364..0000000000 --- a/changes/bug4251 +++ /dev/null @@ -1,8 +0,0 @@ - o Minor bugfixes: - - - When a hidden service turns an extra service-side introduction - circuit into a general-purpose circuit, free the rend_data and - intro_key fields first, so they won't be leaked if the circuit - is cannibalized for use as another service-side introduction - circuit. Bugfix on 0.2.1.7-alpha; fixes bug 4251. - diff --git a/changes/bug4259 b/changes/bug4259 deleted file mode 100644 index bfccd3aee8..0000000000 --- a/changes/bug4259 +++ /dev/null @@ -1,4 +0,0 @@ - o Major bugfixes: - - Fix a crash bug when changing node restrictions while a DNS lookup - is in-progress. Fixes bug 4259; bugfix on 0.2.2.25-alpha. Bugfix - by "Tey'". diff --git a/changes/bug4299 b/changes/bug4299 deleted file mode 100644 index c43d81460a..0000000000 --- a/changes/bug4299 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfix: - - Do not process cells on a marked-for-close connection. We previously - avoided this by not calling read handlers on marked connections, but - that's not adequate for the case when cells are very small. Fixes - bug 4299; bugfix on 0.2.0.20-rc which first made small cells possible. diff --git a/changes/bug4309 b/changes/bug4309 deleted file mode 100644 index f4f910e7ff..0000000000 --- a/changes/bug4309 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Remove a confusing dollar sign from the example fingerprint in the - man page, and also make example fingerprint a valid one. Fixes bug - 4309; bugfix on 0.2.1.3-alpha. - diff --git a/changes/bug4331 b/changes/bug4331 deleted file mode 100644 index 011238a962..0000000000 --- a/changes/bug4331 +++ /dev/null @@ -1,4 +0,0 @@ - o Trivial fixes: - - Fixed a typo in a hibernation-related log message. Fixes bug 4331; - bugfix on 0.2.2.23-alpha; found by "tmpname0901". - diff --git a/changes/bug4340 b/changes/bug4340 deleted file mode 100644 index 08098b1cd5..0000000000 --- a/changes/bug4340 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes: - - Don't crash when we're running as a relay and don't have a geoip - file. Bugfix on tor-0.2.2.34; fixes bug 4340. This backports a fix - we've had in master already. - diff --git a/changes/bug4349 b/changes/bug4349 deleted file mode 100644 index 633916bdfd..0000000000 --- a/changes/bug4349 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - When sending a NETINFO cell, include the original address - received for the other side, not its canonical address. Found - by "troll_un"; fixes bug 4349; bugfix on 0.2.0.10-alpha. diff --git a/changes/bug4353 b/changes/bug4353 deleted file mode 100644 index 5e80c902c8..0000000000 --- a/changes/bug4353 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes: - - When running as client without a geoip database, do not print a - misleading (and plain wrong) log message that we're collecting - dirreq statistics - we're not collecting statistics as clients. - Also don't create a useless (because empty) stats file in the - stats/ directory. Fixes bug 4353, bugfix on 0.2.2.34. - diff --git a/changes/bug4383 b/changes/bug4383 deleted file mode 100644 index e618b8c8fb..0000000000 --- a/changes/bug4383 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Fix a memleak in launch_direct_bridge_descriptor_fetch() that - occured when a client tried to fetch a descriptor for a bridge - in ExcludeNodes. Fixes #4383; bugfix on 0.2.2.25-alpha. - diff --git a/changes/bug4410 b/changes/bug4410 deleted file mode 100644 index f42893adf4..0000000000 --- a/changes/bug4410 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes: - - Correctly sanity-check that we don't underflow on a memory allocation - for introduction point decryption. Bug discovered by Dan Rosenberg. - Fixes bug 4410; bugfix on 0.2.1.5-alpha. - diff --git a/changes/bug4424 b/changes/bug4424 deleted file mode 100644 index 443625dca6..0000000000 --- a/changes/bug4424 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes - - - Don't leak memory when we check whether a hidden service - descriptor has any usable introduction points left. Fixes bug - 4424. Bugfix on 0.2.2.25-alpha. - diff --git a/changes/bug4426 b/changes/bug4426 deleted file mode 100644 index 1322243d09..0000000000 --- a/changes/bug4426 +++ /dev/null @@ -1,8 +0,0 @@ - o Minor features: - - - When Tor ignores a hidden service specified in its - configuration, include the hidden service's directory in the - warning message. Previously, we would only tell the user that - some hidden service was ignored. Bugfix on 0.0.6; fixes bug - 4426. - diff --git a/changes/bug4437 b/changes/bug4437 deleted file mode 100644 index 985c670b15..0000000000 --- a/changes/bug4437 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Don't warn about unused log_mutex in log.c when building with - --disable-threads using a recent GCC. Fixes bug 4437; bugfix on - 0.1.0.6-rc which introduced --disable-threads. - diff --git a/changes/bug4457 b/changes/bug4457 deleted file mode 100644 index fe7c95ff80..0000000000 --- a/changes/bug4457 +++ /dev/null @@ -1,9 +0,0 @@ - o Minor bugfixes: - - Initialize Libevent with the EVENT_BASE_FLAG_NOLOCK flag enabled, so - that it doesn't attempt to allocate a socketpair. This could cause - some problems on windows systems with overzealous firewalls. Fix for - bug 4457; workaround for Libevent versions 2.0.1-alpha through - 2.0.15-stable. - - - Detect failure to initialize Libevent. Better detection for bug 4457. - diff --git a/changes/bug4518 b/changes/bug4518 deleted file mode 100644 index 8dcb93bf72..0000000000 --- a/changes/bug4518 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (performance): - - Avoid frequent calls to the fairly expensive cull_wedged_cpuworkers - function. This was eating up hideously large amounts of time on some - busy servers. Fixes bug 4518. diff --git a/changes/bug4521 b/changes/bug4521 deleted file mode 100644 index 9b0bae9b00..0000000000 --- a/changes/bug4521 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes: - - Backport fixes for a pair of compilation warnings on Windows. - Fixes bug 4521; bugfix on 0.2.2.28-beta and on 0.2.2.29-beta. diff --git a/changes/bug4529 b/changes/bug4529 deleted file mode 100644 index 89d10b2f6b..0000000000 --- a/changes/bug4529 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bufixes: - - If we had ever tried to call tor_addr_to_str on an address of - unknown type, we would have done a strdup on an uninitialized - buffer. Now we won't. Fixes bug 4529; bugfix on 0.2.1.3-alpha. - Reported by "troll_un". diff --git a/changes/bug4530 b/changes/bug4530 deleted file mode 100644 index 7cd4726e57..0000000000 --- a/changes/bug4530 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes: - - - Correctly detect and handle transient lookup failures from - tor_addr_lookup. Fixes bug 4530; bugfix on 0.2.1.5-alpha. - Reported by "troll_un". - diff --git a/changes/bug4531 b/changes/bug4531 deleted file mode 100644 index 6209f9a058..0000000000 --- a/changes/bug4531 +++ /dev/null @@ -1,4 +0,0 @@ - o Major bugfixes: - - Fix null-pointer access that could occur if TLS allocation failed. - Fixes bug 4531; bugfix on 0.2.0.20-rc. Found by "troll_un". - diff --git a/changes/bug4533_part2 b/changes/bug4533_part2 deleted file mode 100644 index 7e0f7c313e..0000000000 --- a/changes/bug4533_part2 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes: - - Fix the SOCKET_OK test that we use to tell when socket - creation fails so that it works on Win64. Fixes part of bug - 4533; bugfix on 0.2.2.29-beta. Bug found by wanoskarnet. - diff --git a/changes/bug4535 b/changes/bug4535 deleted file mode 100644 index 57ced29d0b..0000000000 --- a/changes/bug4535 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes: - - Use tor_socket_t type for listener argument to accept(). Fixes bug - 4535; bugfix on 0.2.2.28-beta. Found by "troll_un". diff --git a/changes/bug4786 b/changes/bug4786 deleted file mode 100644 index 7c1c60f632..0000000000 --- a/changes/bug4786 +++ /dev/null @@ -1,9 +0,0 @@ - - Feature removal: - - When sending or relaying a RELAY_EARLY cell, we used to convert - it to a RELAY cell if the connection was using the v1 link - protocol. This was a workaround for older versions of Tor, which - didn't handle RELAY_EARLY cells properly. Now that all supported - versions can handle RELAY_EARLY cells, and now that we're - enforcing the "no RELAY_EXTEND commands except in RELAY_EARLY - cells" rule, we're removing this workaround. Addresses bug 4786. - diff --git a/changes/bug4788 b/changes/bug4788 deleted file mode 100644 index d65c0015a0..0000000000 --- a/changes/bug4788 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor features (directory server): - - Directory servers now reject versions of Tor older than 0.2.1.30, - and Tor versions between 0.2.2.1-alpha and 0.2.2.20-alpha - (inclusive). These versions accounted for only a small fraction of - the Tor network, and have numerous known security issues. Resolves - issue #4788. diff --git a/changes/bug4803 b/changes/bug4803 deleted file mode 100644 index cd25266c75..0000000000 --- a/changes/bug4803 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Correctly spell "connect" in a log message when creating a controlsocket - fails. Fixes bug 4803; bugfix on 0.2.2.26-beta/0.2.3.2-alpha. - diff --git a/changes/bug4822 b/changes/bug4822 deleted file mode 100644 index 73f43f0452..0000000000 --- a/changes/bug4822 +++ /dev/null @@ -1,13 +0,0 @@ - o Major security workaround: - - When building or running with any version of OpenSSL earlier - than 0.9.8s or 1.0.0f, disable SSLv3 support. These versions had - a bug (CVE-2011-4576) in which their block cipher padding - included uninitialized data, potentially leaking sensitive - information to any peer with whom they made a SSLv3 - connection. Tor does not use SSL v3 by default, but a hostile - client or server could force an SSLv3 connection in order to - gain information that they shouldn't have been able to get. The - best solution here is to upgrade to OpenSSL 0.9.8s or 1.0.0f (or - later). But when building or running with a non-upgraded - OpenSSL, we should instead make sure that the bug can't happen - by disabling SSLv3 entirely. diff --git a/changes/bug4856 b/changes/bug4856 deleted file mode 100644 index fa284a09f5..0000000000 --- a/changes/bug4856 +++ /dev/null @@ -1,3 +0,0 @@ - o Trivial bugfixes - - Fix a typo in a log message in rend_service_rendezvous_has_opened(). - Fixes bug 4856; bugfix on Tor 0.0.6. diff --git a/changes/bug5005 b/changes/bug5005 deleted file mode 100644 index 04d8dfe6a5..0000000000 --- a/changes/bug5005 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes: - - Update "ClientOnly" man page entry to explain that there isn't - really any point to messing with it. Resolves ticket 5005. diff --git a/changes/bug5065 b/changes/bug5065 deleted file mode 100644 index d195313623..0000000000 --- a/changes/bug5065 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes: - - Fix build if path to sed, openssl or sha1sum contains spaces. - This is pretty common on Windows. Fixes bug 5065; bugfix on - 0.2.2.1-alpha. - diff --git a/changes/bug5067 b/changes/bug5067 deleted file mode 100644 index d94b921ce9..0000000000 --- a/changes/bug5067 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (usability): - - Downgrade the "We're missing a certificate" message from notice - to info: people kept mistaking it for a real problem, whereas it - is only a problem when we are failing to bootstrap. Fixes bug - 5067; bugfix on 0.2.10-alpha. diff --git a/changes/bug5090 b/changes/bug5090 deleted file mode 100644 index d47858cb11..0000000000 --- a/changes/bug5090 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes: - - Detect and reject certain misformed escape sequences in configuration - values. Previously, these values would cause us to crash if received - in a torrc file or over an (authenticated) control port. Bug found by - Esteban Manchado Velázquez. Patch by Alexander Schrijver. Fix for - bug 5090; bugfix on 0.2.0.16-alpha. - diff --git a/changes/bug5259 b/changes/bug5259 deleted file mode 100644 index bdada586e9..0000000000 --- a/changes/bug5259 +++ /dev/null @@ -1,3 +0,0 @@ - o Documentation fixes: - - Clarify the behavior of MaxCircuitDirtiness with hidden service - circuits. Fix for issue 5259. diff --git a/changes/bug5283 b/changes/bug5283 deleted file mode 100644 index f0325cf26c..0000000000 --- a/changes/bug5283 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes: - - Fix an edge case where if we fetch or publish a hidden service - descriptor, we might build a 4-hop circuit and then use that circuit - for exiting afterwards -- even if the new last hop doesn't obey our - ExitNodes config option. Fixes bug 5283; bugfix on 0.2.0.10-alpha. - diff --git a/changes/bug5340 b/changes/bug5340 deleted file mode 100644 index 708988af08..0000000000 --- a/changes/bug5340 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes: - - Fix a compile warning when using the --enable-openbsd-malloc configure - option. Fixes bug 5340; bugfix on 0.2.0.20-rc. diff --git a/changes/bug5342 b/changes/bug5342 deleted file mode 100644 index b2ae4515a9..0000000000 --- a/changes/bug5342 +++ /dev/null @@ -1,3 +0,0 @@ - o Security fixes: - - Never use a bridge as an exit, even if it claims to be one. Found by - wanoskarnet. Fixes bug 5342. Bugfix on ????. diff --git a/changes/bug5343 b/changes/bug5343 deleted file mode 100644 index e4e14897f6..0000000000 --- a/changes/bug5343 +++ /dev/null @@ -1,7 +0,0 @@ - o Security fixes: - - Only build circuits if we have a sufficient threshold of the total - descriptors marked in the consensus with the "Exit" flag. This - mitigates an attack proposed by wanoskarnet, in which all of a - client's bridges collude to restrict the exit nodes that the - client knows about. Fixes bug 5343. - diff --git a/changes/bug5346 b/changes/bug5346 deleted file mode 100644 index 3d21f90144..0000000000 --- a/changes/bug5346 +++ /dev/null @@ -1,8 +0,0 @@ - o Minor bugfixes: - - Correct parsing of certain date types in parse_http_time(). - Without this patch, If-Modified-Since would behave - incorrectly. Fix for bug 5346; bugfix on 0.2.0.2-alpha. Patch from - Esteban Manchado Velázques. - - Reject out-of-range times like 23:59:61. Fix for bug 5346; - bugfix on 0.0.8pre3. - diff --git a/changes/bug5593 b/changes/bug5593 deleted file mode 100644 index 358e8de60d..0000000000 --- a/changes/bug5593 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes: - - When sending an HTTP/1.1 proxy request, include a Host header. - Fixes bug 5593; bugfix on 0.2.2.1-alpha. diff --git a/changes/bug5644 b/changes/bug5644 deleted file mode 100644 index a390eba996..0000000000 --- a/changes/bug5644 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes - - Prevent a client-side assertion failure when receiving an - INTRODUCE2 cell by an exit relay, in a general purpose - circuit. Fixes bug 5644; bugfix on tor-0.2.1.6-alpha - diff --git a/changes/bug5647 b/changes/bug5647 deleted file mode 100644 index 92f41c8559..0000000000 --- a/changes/bug5647 +++ /dev/null @@ -1,4 +0,0 @@ - o Major bugfixes: - - Avoid logging uninitialized data when unable to decode a hidden - service descriptor cookie. Fixes bug 5647; bugfix on 0.2.1.5-alpha. - diff --git a/changes/bug5650 b/changes/bug5650 new file mode 100644 index 0000000000..401e317074 --- /dev/null +++ b/changes/bug5650 @@ -0,0 +1,5 @@ + o Major bugfixes: + - Avoid a bug where our response to TLS renegotation under certain + network conditions could lead to a busy-loop, with 100% CPU + consumption. Fixes bug 5650; bugfix on 0.2.0.16-alpha. + diff --git a/changes/bug5760 b/changes/bug5760 deleted file mode 100644 index a26407b588..0000000000 --- a/changes/bug5760 +++ /dev/null @@ -1,3 +0,0 @@ - o Major bugfixes: - - End AUTHCHALLENGE error response messages with a CRLF. Fixes bug 5760; - bugfix on 0.2.3.16-alpha, and backported to maint-0.2.2 diff --git a/changes/bug5786_range b/changes/bug5786_range deleted file mode 100644 index 40ac4d2467..0000000000 --- a/changes/bug5786_range +++ /dev/null @@ -1,8 +0,0 @@ - o Minor bugfixes: - - Make our number-parsing functions always treat too-large values - as an error, even when those values exceed the width of the - underlying type. Previously, if the caller provided these - functions with minima or maxima set to the extreme values of the - underlying integer type, these functions would return those - values on overflow rather than treating overflow as an error. - Fix for part of bug 5786; bugfix on Tor 0.0.9.
\ No newline at end of file diff --git a/changes/bug5796 b/changes/bug5796 deleted file mode 100644 index b92659f74a..0000000000 --- a/changes/bug5796 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (controller): - - Fix a NULL-pointer derefernce on a badly formed - SETCIRCUITPURPOSE command. Found by mikeyc. Fixes bug 5796; - bugfix on 0.2.2.9-alpha. diff --git a/changes/bug5969_022 b/changes/bug5969_022 deleted file mode 100644 index 57c8744267..0000000000 --- a/changes/bug5969_022 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes - - Fix a build warning with Clang 3.1 related to our use of vasprint. - Fix for bug 5969. Bugfix on 0.2.2.11-alpha. - - o Compilation improvements: - - Tell GCC and Clang to check for any errors in format strings passed - to the tor_v*(print|scan)f functions. diff --git a/changes/bug6007 b/changes/bug6007 deleted file mode 100644 index 4e815754aa..0000000000 --- a/changes/bug6007 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes (security): - - When waiting for a client to renegotiate, don't allow it to add - any bytes to the input buffer. This fixes a DoS issue. Fix for - bugs 6007 and 5934; bugfix on 0.2.0.20-rc. - diff --git a/changes/bug6024 b/changes/bug6024 new file mode 100644 index 0000000000..743e6ef1fe --- /dev/null +++ b/changes/bug6024 @@ -0,0 +1,2 @@ + o Documentation fixes: + - Clarify that hidden services are TCP only. Fixes bug 6024. diff --git a/changes/bug6033 b/changes/bug6033 deleted file mode 100644 index 56cffd68b7..0000000000 --- a/changes/bug6033 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes: - - Work around a bug in OpenSSL that broke renegotiation with - TLS 1.1 and TLS 1.2. Without this workaround, all attempts - to speak the v2 Tor network protocol when both sides were - using OpenSSL 1.0.1 would fail. Fix for bug 6033, which is - not a bug in Tor. diff --git a/changes/bug6043 b/changes/bug6043 new file mode 100644 index 0000000000..b88bafb788 --- /dev/null +++ b/changes/bug6043 @@ -0,0 +1,6 @@ + o Packaging (RPM): + - Our default RPM spec files have been updated to work with mock + and rpmbuild on RHEL/Fedora. They have an updated set of + dependencies and conflicts, a fix for an ancient typo when creating + the "_tor" user, and better instructions. Thanks to Ondrej + Mikle for the patch series; fix for bug 6043. diff --git a/changes/bug6094 b/changes/bug6094 deleted file mode 100644 index 835d127dad..0000000000 --- a/changes/bug6094 +++ /dev/null @@ -1,8 +0,0 @@ - o Major bugfixes: - - - Send a CRLF at the end of each STATUS_* control-port event. A - change introduced in 0.2.3.16-alpha caused Tor to emit the CRLF - in the middle of a STATUS_* event; among other things, this - caused Vidalia to freeze. Fixes bug 6094; bugfix on - 0.2.3.16-alpha. - diff --git a/changes/bug6218 b/changes/bug6218 new file mode 100644 index 0000000000..5d5d108b00 --- /dev/null +++ b/changes/bug6218 @@ -0,0 +1,3 @@ + o Minor bugfixes: + - Fix wrong TCP port range in parse_port_range(). Fixes bug 6218; + bugfix on 0.2.1.10-alpha. diff --git a/changes/bug6244_part_c b/changes/bug6244_part_c new file mode 100644 index 0000000000..dea6e7b69e --- /dev/null +++ b/changes/bug6244_part_c @@ -0,0 +1,6 @@ + o Major bugfixes (controller): + - Make wildcarded addresses (that is, ones beginning with *.) work when + provided via the controller's MapAddress command. Previously, they + were accepted, but we never actually noticed that they were wildcards. + Fix for bug 6244; bugfix on 0.2.3.9-alpha. + diff --git a/changes/bug6251 b/changes/bug6251 new file mode 100644 index 0000000000..c782a93e49 --- /dev/null +++ b/changes/bug6251 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Downgrade "set buildtimeout to low value" messages to INFO + severity; they were never an actual problem, there was never + anything reasonable to do about them, and they tended to spam + logs from time to time. Fix for bug 6251; bugfix on + 0.2.2.2-alpha.
\ No newline at end of file diff --git a/changes/bug6252_again b/changes/bug6252_again new file mode 100644 index 0000000000..f7fd00cb38 --- /dev/null +++ b/changes/bug6252_again @@ -0,0 +1,11 @@ + o Security fixes: + - Tear down the circuit if we get an unexpected SENDME cell. Clients + could use this trick to make their circuits receive cells faster + than our flow control would have allowed, or to gum up the network, + or possibly to do targeted memory denial-of-service attacks on + entry nodes. Fixes bug 6252. Bugfix on the 54th commit on Tor -- + from July 2002, before the release of Tor 0.0.0. We had committed + this patch previously, but we had to revert it because of bug 6271. + Now that 6271 is fixed, this appears to work. + + diff --git a/changes/bug6271 b/changes/bug6271 new file mode 100644 index 0000000000..06b129f73f --- /dev/null +++ b/changes/bug6271 @@ -0,0 +1,7 @@ + o Major bugfixes + + - Fix a bug handling SENDME cells on nonexistent streams that + could result in bizarre window values. Report and patch + contributed pseudymously. Fixes part of bug 6271. This bug + was introduced before the first Tor release, in svn commit + r152. diff --git a/changes/bug6274 b/changes/bug6274 new file mode 100644 index 0000000000..ad1abcde54 --- /dev/null +++ b/changes/bug6274 @@ -0,0 +1,3 @@ + o Minor bugfixes: + - Ignore ServerTransportPlugin lines when Tor is not configured as + a relay. Fixes bug 6274; bugfix on 0.2.3.6-alpha. diff --git a/changes/bug6274_2 b/changes/bug6274_2 new file mode 100644 index 0000000000..89576f9328 --- /dev/null +++ b/changes/bug6274_2 @@ -0,0 +1,3 @@ + o Minor features: + - Terminate active server managed proxies if Tor stops being a + relay. Addresses parts of bug 6274; bugfix on 0.2.3.6-alpha. diff --git a/changes/bug6296 b/changes/bug6296 new file mode 100644 index 0000000000..b452b1745d --- /dev/null +++ b/changes/bug6296 @@ -0,0 +1,4 @@ + o Minor bugfixes: + + - Instead of ENOBUFS on Windows, say WSAENOBUFS. Fixes + compilation on Windows. Fixes bug 6296; bugfix on 0.2.3.18-rc. diff --git a/changes/bug6341 b/changes/bug6341 new file mode 100644 index 0000000000..04e52c7cd3 --- /dev/null +++ b/changes/bug6341 @@ -0,0 +1,5 @@ + o Major bugfixes: + - Fix a possible crash bug when checking for deactivated circuits + in connection_or_flush_from_first_active_circuit(). Fixes bug + 6341; bugfix on 0.2.2.7-alpha. Bug report and fix received + pseudonymously. diff --git a/changes/bug6377 b/changes/bug6377 new file mode 100644 index 0000000000..a3a3672783 --- /dev/null +++ b/changes/bug6377 @@ -0,0 +1,4 @@ + o Testing: + - Make it possible to set the TestingTorNetwork configuration + option using AlternateDirAuthority and AlternateBridgeAuthority + as an alternative to setting DirServer. diff --git a/changes/bug6379 b/changes/bug6379 new file mode 100644 index 0000000000..1f2b6941cd --- /dev/null +++ b/changes/bug6379 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Fix build warnings from --enable-openbsd-malloc with gcc warnings + enabled. Fixes bug 6379. + - Fix 64-bit warnings from --enable-openbsd-malloc. Fixes bug 6379. + Bugfix on 0.2.0.20-rc. + diff --git a/changes/bug6387 b/changes/bug6387 new file mode 100644 index 0000000000..73fc4f7cfe --- /dev/null +++ b/changes/bug6387 @@ -0,0 +1,3 @@ + o Documentation: + - Clarify the documentation for the Alternate*Authority options. + Fixes bug 6387. diff --git a/changes/bug6397 b/changes/bug6397 new file mode 100644 index 0000000000..23d8359bd2 --- /dev/null +++ b/changes/bug6397 @@ -0,0 +1,4 @@ + o Major bugfixes: + - When disabling guards for having too high a proportion of failed + circuits, make sure to look at each guard. Fix for bug 6397; bugfix + on 0.2.3.17-beta. diff --git a/changes/bug6404 b/changes/bug6404 new file mode 100644 index 0000000000..948f00b92e --- /dev/null +++ b/changes/bug6404 @@ -0,0 +1,16 @@ + o Minor bugfixes: + + - Remove the maximum length of microdescriptor we are willing to + generate. Occasionally this is needed for routers + with complex policies or family declarations. Partial fix for + bug 6404; fix on 0.2.2.6-alpha. + + - Authorities no longer include any router in their + microdescriptor consensuses for which they couldn't generate or + agree on a microdescriptor. Partial fix for bug 6404; fix on + 0.2.2.6-alpha. + + - Move log message when unable to find a microdesc in a + routerstatus entry to parse time. Previously we'd spam this + warning every time we tried to figure out which microdescriptors + to download. Partial fix for bug 6404; fix on 0.2.3.18-rc. diff --git a/changes/bug6423 b/changes/bug6423 new file mode 100644 index 0000000000..2ea4f1410d --- /dev/null +++ b/changes/bug6423 @@ -0,0 +1,3 @@ + o Minor features: + - Consider new, removed or changed IPv6 OR ports a non cosmetic + change. diff --git a/changes/bug6436 b/changes/bug6436 new file mode 100644 index 0000000000..2c163df105 --- /dev/null +++ b/changes/bug6436 @@ -0,0 +1,3 @@ + o Minor features: + - Provide a better error message about possible OSX Asciidoc failure + reasons. Fix for bug 6436. diff --git a/changes/bug6472 b/changes/bug6472 new file mode 100644 index 0000000000..dcd42ebe68 --- /dev/null +++ b/changes/bug6472 @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Avoid a pair of double-free and use-after-mark bugs that can + occur with certain timings in canceled and re-received DNS + requests. Fix for bug 6472; bugfix on 0.0.7rc1. diff --git a/changes/bug6475 b/changes/bug6475 new file mode 100644 index 0000000000..67bab99622 --- /dev/null +++ b/changes/bug6475 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Add internal circuit construction state to protect against + the noisy warn message "Unexpectedly high circuit_successes". + Also add some additional rate-limited notice messages to help + determine the root cause of the warn. Fixes bug 6475. + Bugfix against 0.2.3.17-beta. diff --git a/changes/bug6480 b/changes/bug6480 new file mode 100644 index 0000000000..83ae00b251 --- /dev/null +++ b/changes/bug6480 @@ -0,0 +1,5 @@ + o Major bugfixes: + - Avoid read-from-freed-RAM bug and related double-free bug that + could occur when a DNS request fails while launching it. Fixes + bug 6480; bugfix on 0.2.0.1-alpha. + diff --git a/changes/bug6490 b/changes/bug6490 new file mode 100644 index 0000000000..c92daad8f4 --- /dev/null +++ b/changes/bug6490 @@ -0,0 +1,4 @@ + o Minor features: + - Warn when Tor is configured to use accounting in a way that will + link a hidden service to some other hidden service or public + address. Fix for bug 6490. diff --git a/changes/bug6500 b/changes/bug6500 new file mode 100644 index 0000000000..cac2054a3a --- /dev/null +++ b/changes/bug6500 @@ -0,0 +1,2 @@ + o Minor bugfixes: + - Fix some typos in the manpages. Patch from A. Costa. Fixes bug 6500. diff --git a/changes/bug6507 b/changes/bug6507 new file mode 100644 index 0000000000..89940cbf7b --- /dev/null +++ b/changes/bug6507 @@ -0,0 +1,15 @@ + o Major bugfixes: + - Detect 'ORPort 0' as meaning, uniformly, that we're not running + as a server. Previously, some of our code would treat the + presence of any ORPort line as meaning that we should act like a + server, even though our new listener code would correctly not + open any ORPorts for ORPort 0. Similar bugs in other Port + options are also fixed. Fixes bug 6507; bugfix on 0.2.3.3-alpha. + + o Minor features: + + - Detect and reject attempts to specify both 'FooPort' and + 'FooPort 0' in the same configuration domain. (It's still okay + to have a FooPort in your configuration file,and use 'FooPort 0' + on the command line to disable it.) Fixes another case of + bug6507; bugfix on 0.2.3.3-alpha. diff --git a/changes/bug6514 b/changes/bug6514 new file mode 100644 index 0000000000..84633bd279 --- /dev/null +++ b/changes/bug6514 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Add a (probably redundant) memory clear between iterations of + the router status voting loop, to prevent future coding errors + where data might leak between iterations of the loop. Resolves + ticket 6514. diff --git a/changes/bug6710 b/changes/bug6710 new file mode 100644 index 0000000000..2c89346114 --- /dev/null +++ b/changes/bug6710 @@ -0,0 +1,6 @@ + o Major bugfixes (security): + - Reject any attempt to extend to an internal address. Without + this fix, a router could be used to probe addresses on an + internal network to see whether they were accepting + connections. Fix for bug 6710; bugfix on 0.0.8pre1. + diff --git a/changes/bug6732 b/changes/bug6732 new file mode 100644 index 0000000000..7a744e014a --- /dev/null +++ b/changes/bug6732 @@ -0,0 +1,3 @@ + o Documentation: + - Add missing documentation for consensus and microdesc files. Fix for + bug 6732. diff --git a/changes/bug6743 b/changes/bug6743 new file mode 100644 index 0000000000..6ec78f853a --- /dev/null +++ b/changes/bug6743 @@ -0,0 +1,9 @@ + o Minor bugfixes: + - Allow one-hop directory fetching circuits the full "circuit build + timeout" period, rather than just half of it, before failing them + and marking the relay down. This fix should help reduce cases where + clients declare relays (or worse, bridges) unreachable because + the TLS handshake takes a few seconds to complete. Fixes bug 6743; + bugfix on 0.2.2.2-alpha, where we changed the timeout from a static + 30 seconds. + diff --git a/changes/bug6774 b/changes/bug6774 new file mode 100644 index 0000000000..0c137fd678 --- /dev/null +++ b/changes/bug6774 @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Avoid crashing on a malformed state file where EntryGuardPathBias + precedes EntryGuard. Fix for bug 6774; bugfix on 0.2.3.17-beta. + diff --git a/changes/bug6801 b/changes/bug6801 new file mode 100644 index 0000000000..ef21acc98f --- /dev/null +++ b/changes/bug6801 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Avoid segfault when starting up having run with an extremely old + version of Tor and parsing its state file. Fixes bug 6801; bugfix on + 0.2.2.23-alpha. + diff --git a/changes/bug6827 b/changes/bug6827 new file mode 100644 index 0000000000..bf71d2b97c --- /dev/null +++ b/changes/bug6827 @@ -0,0 +1,9 @@ + o Minor bugfixes: + + - Avoid undefined behaviour when parsing the list of supported + rendezvous/introduction protocols in a hidden service + descriptor. Previously, Tor would have confused (as-yet-unused) + protocol version numbers greater than 32 with lower ones on many + platforms. Fixes bug 6827; bugfix on 0.2.0.10-alpha; found by + George Kadianakis. + diff --git a/changes/bug6844 b/changes/bug6844 new file mode 100644 index 0000000000..338e19d9a5 --- /dev/null +++ b/changes/bug6844 @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Correct file sizes when reading binary files on + Cygwin, to avoid a bug where Tor would fail to read its state file. + Fixes bug 6844; bugfix on 0.1.2.7-alpha. diff --git a/changes/bug6866 b/changes/bug6866 new file mode 100644 index 0000000000..561676b765 --- /dev/null +++ b/changes/bug6866 @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Convert an assert in the pathbias code to a log message. Assert + appears to only be triggerable by Tor2Web mode. Fixes bug 6866; + bugfix on 0.2.3.17-beta. diff --git a/changes/bug7014 b/changes/bug7014 new file mode 100644 index 0000000000..1d39103a50 --- /dev/null +++ b/changes/bug7014 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Fix two cases in src/or/transports.c where we were calling + fmt_addr() twice in a parameter list. Bug found by David + Fifield. Fixes bug 7014; bugfix on 0.2.3.9-alpha. + diff --git a/changes/bug7022 b/changes/bug7022 new file mode 100644 index 0000000000..10ac354724 --- /dev/null +++ b/changes/bug7022 @@ -0,0 +1,3 @@ + o Minor bugfixes: + - Fix memory leaks whenever we logged any message about the "path + bias" detection. Fixes bug 7022; bugfix on 0.2.3.21-rc. diff --git a/changes/bug7037 b/changes/bug7037 new file mode 100644 index 0000000000..fc3a1ad1c5 --- /dev/null +++ b/changes/bug7037 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - When relays refuse a "create" cell because their queue of pending + create cells is too big (typically because their cpu can't keep up + with the arrival rate), send back reason "resource limit" rather + than reason "internal", so network measurement scripts can get a + more accurate picture. Bugfix on 0.1.1.11-alpha; fixes bug 7037. diff --git a/changes/bug7190 b/changes/bug7190 new file mode 100644 index 0000000000..1607f79442 --- /dev/null +++ b/changes/bug7190 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Clients now consider the ClientRejectInternalAddresses config option + when using a microdescriptor consensus stanza to decide whether + an exit relay would allow exiting to an internal address. Fixes + bug 7190; bugfix on 0.2.3.1-alpha. + diff --git a/changes/bug7192 b/changes/bug7192 new file mode 100644 index 0000000000..10cbc2469a --- /dev/null +++ b/changes/bug7192 @@ -0,0 +1,10 @@ + o Major bugfixes: + - When parsing exit policy summaries from microdescriptors, we had + previously been ignoring the last character in each one, so that + "accept 80,443,8080" would be treated by clients as indicating a + node that allows access to ports 80, 443, and 808. That would lead + to clients attempting connections that could never work, and + ignoring exit nodes that would support their connections. Now clients + parse these exit policy summaries correctly. Fixes bug 7192; + bugfix on 0.2.3.1-alpha. + diff --git a/changes/bug7352 b/changes/bug7352 new file mode 100644 index 0000000000..74a878dbe0 --- /dev/null +++ b/changes/bug7352 @@ -0,0 +1,12 @@ + o Major bugfixes: + - Tor tries to wipe potentially sensitive data after using it, so + that if some subsequent security failure exposes Tor's memory, + the damage will be limited. But we had a bug where the compiler + was eliminating these wipe operations when it decided that the + memory was no longer visible to a (correctly running) program, + hence defeating our attempt at defense in depth. We fix that + by using OpenSSL's OPENSSL_cleanse() operation, which a compiler + is unlikely to optimize away. Future versions of Tor may use + a less ridiculously heavy approach for this. Fixes bug 7352. + Reported in an article by Andrey Karpov. + diff --git a/changes/bug7464 b/changes/bug7464 new file mode 100644 index 0000000000..9259cc74a3 --- /dev/null +++ b/changes/bug7464 @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Fix a harmless bug when opting against publishing a relay descriptor + because DisableNetwork is set. Fixes bug 7464; bugfix on + 0.2.3.9-alpha. diff --git a/changes/bug7889 b/changes/bug7889 new file mode 100644 index 0000000000..ce99a59ce5 --- /dev/null +++ b/changes/bug7889 @@ -0,0 +1,8 @@ + o Major bugfixes: + - Reject bogus create and relay cells with 0 circuit ID or 0 stream + ID: these could be used to create unexpected streams and circuits + which would count as "present" to some parts of Tor but "absent" + to others, leading to zombie circuits and streams or to a + bandwidth DOS. Fixes bug 7889; bugfix on every released version of + Tor. Reported by "oftc_must_be_destroyed". + diff --git a/changes/bug8208 b/changes/bug8208 new file mode 100644 index 0000000000..c85db90b52 --- /dev/null +++ b/changes/bug8208 @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Avoid a crash if we fail to generate an extrinfo descriptor. + Fixes bug 8208; bugfix on 0.2.3.16-alpha. Found by Coverity; + this is CID 718634. diff --git a/changes/bug8377 b/changes/bug8377 new file mode 100644 index 0000000000..c9ad151bc9 --- /dev/null +++ b/changes/bug8377 @@ -0,0 +1,3 @@ + o Minor bugfixes: + - Correctly recognize that [::1] is a loopback address. Fixes bug #8377; + bugfix on 0.2.1.3-alpha. diff --git a/changes/bug8844 b/changes/bug8844 new file mode 100644 index 0000000000..320e5f2845 --- /dev/null +++ b/changes/bug8844 @@ -0,0 +1,6 @@ + o Major bugfixes: + - Prevent the get_freelists() function from running off the end of + the list of freelists if it somehow gets an unrecognized + allocation. Fixes bug 8844; bugfix on 0.2.0.16-alpha. Reported by + eugenis. + diff --git a/changes/bug9002 b/changes/bug9002 new file mode 100644 index 0000000000..c41ace394a --- /dev/null +++ b/changes/bug9002 @@ -0,0 +1,4 @@ + o Major bugfixes: + - Limit hidden service descriptors to at most ten introduction + points, to slow one kind of guard enumeration. Fixes bug 9002; + bugfix on 0.1.1.11-alpha. diff --git a/changes/bug9017 b/changes/bug9017 new file mode 100644 index 0000000000..359c526b00 --- /dev/null +++ b/changes/bug9017 @@ -0,0 +1,6 @@ + o Major bugfixes: + - Avoid an assertion failure on OpenBSD (and perhaps other BSDs) + when an exit connection with optimistic data succeeds immediately + rather than returning EINPROGRESS. Fixes bug 9017; bugfix on + 0.2.3.1-alpha. + diff --git a/changes/bug9063 b/changes/bug9063 new file mode 100644 index 0000000000..af3b1a87f4 --- /dev/null +++ b/changes/bug9063 @@ -0,0 +1,3 @@ + o Normal bugfixes: + - Close any circuit that has more cells queued than the spec permits. + Fixes bug #9063; bugfix on 0.2.3.25. diff --git a/changes/bug9063_redux b/changes/bug9063_redux new file mode 100644 index 0000000000..e6fae72efc --- /dev/null +++ b/changes/bug9063_redux @@ -0,0 +1,15 @@ + o Major bugfixes: + - When we have too much memory queued in circuits (according to a new + MaxMemInCellQueues option), close the circuits consuming the most + memory. This prevents us from running out of memory as a relay if + circuits fill up faster than they can be drained. Fixes + bug 9063; bugfix on the 54th commit of Tor. This bug is a further + fix beyond bug 6252, whose fix was merged into 0.2.3.21-rc. + + Also fixes an earlier approach taken in 0.2.4.13-alpha, where we + tried to solve this issue simply by imposing an upper limit on the + number of queued cells for a single circuit. That approach proved to + be problematic, since there are ways to provoke clients to send a + number of cells in excess of any such reasonable limit. + Fixes bug 9072; bugfix on 0.2.4.13-alpha. + diff --git a/changes/bug9072 b/changes/bug9072 new file mode 100644 index 0000000000..e594a38335 --- /dev/null +++ b/changes/bug9072 @@ -0,0 +1,3 @@ + o Critical bugfixes: + - Disable middle relay queue overfill detection code due to possible + guard discovery attack, pending further analysis. Fixes bug #9072. diff --git a/changes/check-fetched-rend-desc-service-id b/changes/check-fetched-rend-desc-service-id deleted file mode 100644 index 2f37c30216..0000000000 --- a/changes/check-fetched-rend-desc-service-id +++ /dev/null @@ -1,7 +0,0 @@ - o Security fixes: - - When fetching a hidden service descriptor, check that it is for - the hidden service we were trying to connect to, in order to - stop a directory from pre-seeding a client with a descriptor for - a hidden service that they didn't want. Bugfix on 0.0.6. - - diff --git a/changes/check-public-key-exponents b/changes/check-public-key-exponents deleted file mode 100644 index a8d00673be..0000000000 --- a/changes/check-public-key-exponents +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Require that introduction point keys and onion keys have public - exponent 65537. Bugfix on 0.2.0.10-alpha. - - diff --git a/changes/cid_428 b/changes/cid_428 deleted file mode 100644 index cb0fc8c2b2..0000000000 --- a/changes/cid_428 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Always NUL-terminate the sun_path field of a sockaddr_un before - passing it to the kernel. (Not a security issue: kernels are - smart enough to reject bad sockaddr_uns.) Found by Coverity; CID - # 428. Bugfix on Tor 0.2.0.3-alpha. diff --git a/changes/cid_450 b/changes/cid_450 deleted file mode 100644 index 2045fca239..0000000000 --- a/changes/cid_450 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Don't stack-allocate the list of supplementary GIDs when we're - about to log them. Stack-allocating NGROUPS_MAX gid_t elements - could take up to 256K, which is way too much stack. Found by - Coverity; CID #450. Bugfix on 0.2.1.7-alpha. diff --git a/changes/clang_30_options b/changes/clang_30_options deleted file mode 100644 index e8e34c8e3e..0000000000 --- a/changes/clang_30_options +++ /dev/null @@ -1,5 +0,0 @@ - o Code simplifications and refactoring: - - During configure, detect when we're building with clang version 3.0 or - lower and disable the -Wnormalized=id and -Woverride-init CFLAGS. - clang doesn't support them yet. - diff --git a/changes/cov479 b/changes/cov479 deleted file mode 100644 index afbaffc63b..0000000000 --- a/changes/cov479 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Fix internal bug-checking logic that was supposed to catch - failures in digest generation so that it will fail more robustly - if we ask for a nonexistent algorithm. Found by Coverity Scan. - Bugfix on 0.2.2.1-alpha; fixes Coverity CID 479. diff --git a/changes/cov484 b/changes/cov484 deleted file mode 100644 index 33adbda18c..0000000000 --- a/changes/cov484 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Report any failure in init_keys() calls done because our IP address - has changed. Spotted by Coverity Scan. Bugfix on 0.1.1.4-alpha; - fixes CID 484. diff --git a/changes/cov709056 b/changes/cov709056 new file mode 100644 index 0000000000..64a75ad8a2 --- /dev/null +++ b/changes/cov709056 @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Check return value of fputs() when writing authority certificate + file. Fixes Coverity issue 709056; bugfix on 0.2.0.1-alpha. + diff --git a/changes/coverity_maint b/changes/coverity_maint deleted file mode 100644 index e7be90a485..0000000000 --- a/changes/coverity_maint +++ /dev/null @@ -1,9 +0,0 @@ - o Code simplifications and refactoring: - - Remove some dead code as indicated by coverity. - - Remove a few dead assignments during router parsing. Found by coverity. - o Minor bugfixes: - - Add some forgotten return value checks during unit tests. Found - by coverity. - - Don't use 1-bit wide signed bit fields. Found by coverity. - - Fix a rare memory leak during stats writing. Found by coverity. - diff --git a/changes/cve-2012-2249 b/changes/cve-2012-2249 new file mode 100644 index 0000000000..625bfa2f58 --- /dev/null +++ b/changes/cve-2012-2249 @@ -0,0 +1,5 @@ + o Major bugfixes (security): + - Discard extraneous renegotiation attempts once the V3 link + protocol has been initiated. Failure to do so left us open to + a remotely triggerable assertion failure. Fixes CVE-2012-2249; + bugfix on 0.2.3.6-alpha. Reported by "some guy from France". diff --git a/changes/dirreq-stats-default b/changes/dirreq-stats-default deleted file mode 100644 index df7ac11425..0000000000 --- a/changes/dirreq-stats-default +++ /dev/null @@ -1,5 +0,0 @@ - o Minor features: - - Turn on directory request statistics by default and include them in - extra-info descriptors. Don't break if we have no GeoIP database. - Backported from 0.2.3.1-alpha; implements ticket 3951. - diff --git a/changes/dirserv-BUGGY-a b/changes/dirserv-BUGGY-a new file mode 100644 index 0000000000..35b492a2d7 --- /dev/null +++ b/changes/dirserv-BUGGY-a @@ -0,0 +1,7 @@ + o Minor bugfixes: + + - Don't serve or accept v2 hidden service descriptors over a + relay's DirPort. It's never correct to do so, and disabling it + might make it more annoying to exploit any bugs that turn up in the + descriptor-parsing code. Fixes bug 7149. + diff --git a/changes/dirvote_null_deref b/changes/dirvote_null_deref deleted file mode 100644 index 65dc519f52..0000000000 --- a/changes/dirvote_null_deref +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Fix a potential null-pointer dereference while computing a consensus. - Bugfix on tor-0.2.0.3-alpha, found with the help of clang's analyzer. - diff --git a/changes/disable_pathbias_messages b/changes/disable_pathbias_messages new file mode 100644 index 0000000000..3bc996347b --- /dev/null +++ b/changes/disable_pathbias_messages @@ -0,0 +1,3 @@ + o Disabeled features + - Downgrade path-bias warning messages to INFO. We'll try to get them + working better in 0.2.4. Fixes bug 6475; bugfix on 0.2.3.17-beta. diff --git a/changes/exit-policy-default-is-not-a-prefix b/changes/exit-policy-default-is-not-a-prefix deleted file mode 100644 index 6eb1e8df99..0000000000 --- a/changes/exit-policy-default-is-not-a-prefix +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Remove a trailing asterisk from "exit-policy/default" in the - output of the control port command "GETINFO info/names". Bugfix - on 0.1.2.5-alpha. - diff --git a/changes/feature3049 b/changes/feature3049 deleted file mode 100644 index 7960a1f475..0000000000 --- a/changes/feature3049 +++ /dev/null @@ -1,6 +0,0 @@ - o Major features: - - Add an __OwningControllerProcess configuration option and a - TAKEOWNERSHIP control-port command, so that a Tor controller can - ensure that when it exits, Tor will shut down. Implements - feature 3049. - diff --git a/changes/feature3076 b/changes/feature3076 deleted file mode 100644 index a3dcec8741..0000000000 --- a/changes/feature3076 +++ /dev/null @@ -1,14 +0,0 @@ - o Minor features - - The options SocksPort, ControlPort, and so on now all accept an - optional value "auto" that opens a socket on an OS-selected port. - o Minor features (controller) - - GETINFO net/listeners/(type) now returns a list of the addresses - and ports that are bound for listeners for a given connection - type. This is useful for if the user has selected SocksPort - "auto", and you need to know which port got chosen. - - There is a ControlPortWriteToFile option that tells Tor to write - its actual control port or ports to a chosen file. If the option - ControlPortFileGroupReadable is set, the file is created as - group-readable. - - diff --git a/changes/feature4484 b/changes/feature4484 deleted file mode 100644 index 78154e9649..0000000000 --- a/changes/feature4484 +++ /dev/null @@ -1,8 +0,0 @@ - o Minor features: - - Add two new config options for directory authorities: - AuthDirFastGuarantee sets a bandwidth threshold for guaranteeing the - Fast flag, and AuthDirGuardBWGuarantee sets a bandwidth threshold - that is always sufficient to satisfy the bandwidth requirement for - the Guard flag. Now it will be easier for researchers to simulate - Tor networks with different values. Resolves ticket 4484. - diff --git a/changes/fix-connection_printf_to_buf b/changes/fix-connection_printf_to_buf deleted file mode 100644 index e191eac8a5..0000000000 --- a/changes/fix-connection_printf_to_buf +++ /dev/null @@ -1,15 +0,0 @@ - * Code simplifications and refactoring: - - - Make connection_printf_to_buf's behaviour sane. Its callers - expect it to emit a CRLF iff the format string ends with CRLF; - it actually emits a CRLF iff (a) the format string ends with - CRLF or (b) the resulting string is over 1023 characters long or - (c) the format string does not end with CRLF ''and'' the - resulting string is 1021 characters long or longer. Bugfix on - 0.1.1.9-alpha; fixes part of bug 3407. - - - Make send_control_event_impl's behaviour sane. Its callers - expect it to always emit a CRLF at the end of the string; it - might emit extra control characters as well. Bugfix on - 0.1.1.9-alpha; fixes another part of bug 3407. - diff --git a/changes/fmt_addr b/changes/fmt_addr deleted file mode 100644 index b88c9e1bf4..0000000000 --- a/changes/fmt_addr +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - When unable to format an address as a string, report its value - as "???" rather than reusing the last formatted address. Bugfix - on 0.2.1.5-alpha. diff --git a/changes/geoip-april2012 b/changes/geoip-april2012 deleted file mode 100644 index 66720c6d69..0000000000 --- a/changes/geoip-april2012 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the April 3 2012 Maxmind GeoLite Country database. - diff --git a/changes/geoip-august2011 b/changes/geoip-august2011 deleted file mode 100644 index 6de8b0f29c..0000000000 --- a/changes/geoip-august2011 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the August 2 2011 Maxmind GeoLite Country database. - diff --git a/changes/geoip-december2011 b/changes/geoip-december2011 deleted file mode 100644 index 82a708de62..0000000000 --- a/changes/geoip-december2011 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the December 6 2011 Maxmind GeoLite Country database. - diff --git a/changes/geoip-february2012 b/changes/geoip-february2012 deleted file mode 100644 index 0711654021..0000000000 --- a/changes/geoip-february2012 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the February 7 2012 Maxmind GeoLite Country database. - diff --git a/changes/geoip-january2012 b/changes/geoip-january2012 deleted file mode 100644 index 2f4180e578..0000000000 --- a/changes/geoip-january2012 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the January 3 2012 Maxmind GeoLite Country database. - diff --git a/changes/geoip-july2011 b/changes/geoip-july2011 deleted file mode 100644 index 7a9f119be0..0000000000 --- a/changes/geoip-july2011 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the July 1 2011 Maxmind GeoLite Country database. - diff --git a/changes/geoip-june2011 b/changes/geoip-june2011 deleted file mode 100644 index 8cf011b723..0000000000 --- a/changes/geoip-june2011 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the June 1 2011 Maxmind GeoLite Country database. - diff --git a/changes/geoip-june2012 b/changes/geoip-june2012 deleted file mode 100644 index f73bf35529..0000000000 --- a/changes/geoip-june2012 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the June 6 2012 Maxmind GeoLite Country database. - diff --git a/changes/geoip-march2012 b/changes/geoip-march2012 deleted file mode 100644 index 0f66d8fae2..0000000000 --- a/changes/geoip-march2012 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the March 6 2012 Maxmind GeoLite Country database. - diff --git a/changes/geoip-may2011 b/changes/geoip-may2011 deleted file mode 100644 index c908f24b45..0000000000 --- a/changes/geoip-may2011 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the May 1 2011 Maxmind GeoLite Country database. - diff --git a/changes/geoip-may2012 b/changes/geoip-may2012 deleted file mode 100644 index 1575eced87..0000000000 --- a/changes/geoip-may2012 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the May 1 2012 Maxmind GeoLite Country database. - diff --git a/changes/geoip-november2011 b/changes/geoip-november2011 deleted file mode 100644 index 3aa8dc05c2..0000000000 --- a/changes/geoip-november2011 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the November 1 2011 Maxmind GeoLite Country database. - diff --git a/changes/geoip-october2011 b/changes/geoip-october2011 deleted file mode 100644 index d5b6910edb..0000000000 --- a/changes/geoip-october2011 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the October 4 2011 Maxmind GeoLite Country database. - diff --git a/changes/geoip-september2011 b/changes/geoip-september2011 deleted file mode 100644 index c41314b1f0..0000000000 --- a/changes/geoip-september2011 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the September 6 2011 Maxmind GeoLite Country database. - diff --git a/changes/ides-becomes-turtles b/changes/ides-becomes-turtles deleted file mode 100644 index 09d964fe9b..0000000000 --- a/changes/ides-becomes-turtles +++ /dev/null @@ -1,4 +0,0 @@ - o Directory authority changes: - - Change IP address for ides (v3 directory authority), and rename it to - turtles. - diff --git a/changes/issue-2011-10-19L b/changes/issue-2011-10-19L deleted file mode 100644 index b879c9d401..0000000000 --- a/changes/issue-2011-10-19L +++ /dev/null @@ -1,28 +0,0 @@ - o Security fixes: - - - Don't send TLS certificate chains on outgoing OR connections - from clients and bridges. Previously, each client or bridge - would use a single cert chain for all outgoing OR connections - for up to 24 hours, which allowed any relay connected to by a - client or bridge to determine which entry guards it is using. - This is a potential user-tracing bug for *all* users; everyone - who uses Tor's client or hidden service functionality should - upgrade. Fixes CVE-2011-2768. Bugfix on FIXME; found by - frosty_un. - - - Don't use any OR connection on which we have received a - CREATE_FAST cell to satisfy an EXTEND request. Previously, we - would not consider whether a connection appears to be from a - client or bridge when deciding whether to use that connection to - satisfy an EXTEND request. Mitigates CVE-2011-2768, by - preventing an attacker from determining whether an unpatched - client is connected to a patched relay. Bugfix on FIXME; found - by frosty_un. - - - Don't assign the Guard flag to relays running a version of Tor - which would use an OR connection on which it has received a - CREATE_FAST cell to satisfy an EXTEND request. Mitigates - CVE-2011-2768, by ensuring that clients will not connect - directly to any relay which an attacker could probe for an - unpatched client's connections. - diff --git a/changes/issue-2011-10-23G b/changes/issue-2011-10-23G deleted file mode 100644 index 45f86754f0..0000000000 --- a/changes/issue-2011-10-23G +++ /dev/null @@ -1,9 +0,0 @@ - o Security fixes: - - - Reject CREATE and CREATE_FAST cells on outgoing OR connections - from a bridge to a relay. Previously, we would accept them and - handle them normally, thereby allowing a malicious relay to - easily distinguish bridges which connect to it from clients. - Fixes CVE-2011-2769. Bugfix on 0.2.0.3-alpha, when bridges were - implemented; found by frosty_un. - diff --git a/changes/link_negotiation_assert b/changes/link_negotiation_assert new file mode 100644 index 0000000000..398a545573 --- /dev/null +++ b/changes/link_negotiation_assert @@ -0,0 +1,6 @@ + o Major bugfixs (security): + - Fix a group of remotely triggerable assertion failures related to + incorrect link protocol negotiation. Found, diagnosed, and fixed + by "some guy from France." Fix for CVE-2012-2250; bugfix on + 0.2.3.6-alpha. + diff --git a/changes/maatuska-ip b/changes/maatuska-ip deleted file mode 100644 index a00b43f866..0000000000 --- a/changes/maatuska-ip +++ /dev/null @@ -1,3 +0,0 @@ - o Directory authority changes: - - Change IP address for maatuska (v3 directory authority). - diff --git a/changes/md_cache_replace b/changes/md_cache_replace deleted file mode 100644 index 88e029c00a..0000000000 --- a/changes/md_cache_replace +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes - - Avoid a bug that would keep us from replacing a microdescriptor - cache on Windows. (We would try to replace the file while still - holding it open. That's fine on Unix, but Windows doesn't let us - do that.) Bugfix on 0.2.2.6-alpha; bug found by wanoskarnet. - diff --git a/changes/mdesc_null_deref b/changes/mdesc_null_deref deleted file mode 100644 index 30f0280536..0000000000 --- a/changes/mdesc_null_deref +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Avoid a possible null-pointer dereference when rebuilding the mdesc - cache without actually having any descriptors to cache. Bugfix on - 0.2.2.6-alpha. Issue discovered using clang's static analyzer. - diff --git a/changes/memleak_rendcache b/changes/memleak_rendcache deleted file mode 100644 index 93b1f6141b..0000000000 --- a/changes/memleak_rendcache +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Fix a memory leak when receiving a descriptor for a hidden - service we didn't ask for. Found by Coverity; CID#30. Bugfix on - 0.2.2.26-beta. diff --git a/changes/msvc_lround b/changes/msvc_lround deleted file mode 100644 index e4aea95351..0000000000 --- a/changes/msvc_lround +++ /dev/null @@ -1,4 +0,0 @@ - o Build fixes: - - Provide a substitute implementation of lround() for MSVC, which - apparently lacks it. Patch from Gisle Vanem. - diff --git a/changes/port_doc b/changes/port_doc new file mode 100644 index 0000000000..0e8662f0ab --- /dev/null +++ b/changes/port_doc @@ -0,0 +1,3 @@ + o Minor features (usability): + - Try to make the warning when giving an obsolete SOCKSListenAddress + a littel more useful. diff --git a/changes/replay-firstpart b/changes/replay-firstpart deleted file mode 100644 index f4a7767fb1..0000000000 --- a/changes/replay-firstpart +++ /dev/null @@ -1,13 +0,0 @@ - o Minor features (security): - - - Check for replays of the public-key encrypted portion of an - INTRODUCE1 cell, in addition to the current check for replays of - the g^x value. This prevents a possible class of active attacks - by an attacker who controls both an introduction point and a - rendezvous point, and who uses the malleability of AES-CTR to - alter the encrypted g^x portion of the INTRODUCE1 cell. We - think that these attacks is infeasible (requiring the attacker - to send on the order of zettabytes of altered cells in a short - interval), but we'd rather block them off in case there are any - classes of this attack that we missed. Reported by dvorak. - diff --git a/changes/routerlist_ins_replace b/changes/routerlist_ins_replace deleted file mode 100644 index 7b442a2f5d..0000000000 --- a/changes/routerlist_ins_replace +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - If we hit the error case where routerlist_insert() replaces an - existing (old) server descriptor, make sure to remove that - server descriptor from the old_routers list. Fix related to bug - 1776. Bugfix on 0.2.2.18-alpha. diff --git a/changes/safecookie b/changes/safecookie deleted file mode 100644 index fd7d7af2b0..0000000000 --- a/changes/safecookie +++ /dev/null @@ -1,9 +0,0 @@ - o Security Features: - - Provide controllers with a safer way to implement the cookie - authentication mechanism. With the old method, if another locally - running program could convince a controller that it was the Tor - process, then that program could trick the contoller into - telling it the contents of an arbitrary 32-byte file. The new - "SAFECOOKIE" authentication method uses a challenge-response - approach to prevent this. Fixes bug 5185, implements proposal 193. - diff --git a/changes/smartlist_foreach b/changes/smartlist_foreach new file mode 100644 index 0000000000..2fd3a1a85c --- /dev/null +++ b/changes/smartlist_foreach @@ -0,0 +1,8 @@ + o Code simplification and refactoring: + - Do not use SMARTLIST_FOREACH for any loop whose body exceeds + 10 lines. Doing so in the past has led to hard-to-debug code. + The new style is to use the SMARTLIST_FOREACH_{BEGIN,END} pair. + Issue 6400. + - Do not nest SMARTLIST_FOREACH blocks within one another. Any + nested block ought to be using SMARTLIST_FOREACH_{BEGIN,END}. + Issue 6400. diff --git a/changes/ticket-4063 b/changes/ticket-4063 deleted file mode 100644 index 6a985b8c25..0000000000 --- a/changes/ticket-4063 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (usability): - - Downgrade log messages about circuit timeout calibration from - "notice" to "info": they don't require or suggest any human - intervention. Patch from Tom Lowenthal. Fixes bug 4063; - bugfix on 0.2.2.14-alpha. - diff --git a/changes/ticket5749 b/changes/ticket5749 new file mode 100644 index 0000000000..0237241981 --- /dev/null +++ b/changes/ticket5749 @@ -0,0 +1,3 @@ + o New directory authorities: + - Add Faravahar (run by Sina Rabbani) as the ninth v3 directory + authority. Closes ticket 5749. diff --git a/changes/timersub_bug b/changes/timersub_bug deleted file mode 100644 index 9183862677..0000000000 --- a/changes/timersub_bug +++ /dev/null @@ -1,7 +0,0 @@ - o Major bugfixes: - - Provide correct replacements for the timeradd() and timersub() functions - for platforms that lack them (for example, windows). The timersub() - function is used when expiring circuits, timeradd() is currently unused. - Patch written by Vektor, who also reported the bug. Thanks! Bugfix - on 0.2.2.24-alpha/0.2.3.1-alpha, fixes bug 4778. - diff --git a/changes/typo-fix-ohkah8Ah b/changes/typo-fix-ohkah8Ah deleted file mode 100644 index 9b4e5c08cc..0000000000 --- a/changes/typo-fix-ohkah8Ah +++ /dev/null @@ -1,9 +0,0 @@ - * Minor bugfixes: - - - Clarify a log message specifying the characters permitted in - HiddenServiceAuthorizeClient client names. Previously, the log - message said that "[A-Za-z0-9+-_]" were permitted; that could - have given the impression that every ASCII character between "+" - and "_" was permitted. Now we say "[A-Za-z0-9+_-]". Bugfix on - 0.2.1.5-alpha. - diff --git a/changes/win-bundle-path b/changes/win-bundle-path deleted file mode 100644 index 32ff514ef2..0000000000 --- a/changes/win-bundle-path +++ /dev/null @@ -1,4 +0,0 @@ - o Packaging changes: - - Remove absolute path from makensis.exe command to build Tor expert bundle - in order to make it easier to automate package builds - diff --git a/changes/windows_8 b/changes/windows_8 deleted file mode 100644 index 405e4fa158..0000000000 --- a/changes/windows_8 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - The next version of Windows will be called Windows 8, and it has a major - version of 6, minor version of 2. Correctly identify that version instead - of calling it "Very recent version". Fixes bug 4153; reported by funkstar. - diff --git a/configure.in b/configure.in index 424cccb874..8aa90f6414 100644 --- a/configure.in +++ b/configure.in @@ -1,10 +1,10 @@ dnl Copyright (c) 2001-2004, Roger Dingledine dnl Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson -dnl Copyright (c) 2007-2008, The Tor Project, Inc. +dnl Copyright (c) 2007-2012, The Tor Project, Inc. dnl See LICENSE for licensing information AC_INIT -AM_INIT_AUTOMAKE(tor, 0.2.2.34-dev) +AM_INIT_AUTOMAKE(tor, 0.2.3.18-rc-dev) AM_CONFIG_HEADER(orconfig.h) AC_CANONICAL_HOST @@ -32,6 +32,15 @@ AC_ARG_ENABLE(static-libevent, AS_HELP_STRING(--enable-static-libevent, Link against a static libevent library. Requires --with-libevent-dir)) AC_ARG_ENABLE(static-zlib, AS_HELP_STRING(--enable-static-zlib, Link against a static zlib library. Requires --with-zlib-dir)) +AC_ARG_ENABLE(static-tor, + AS_HELP_STRING(--enable-static-tor, Create an entirely static Tor binary. Requires --with-openssl-dir and --with-libevent-dir and --with-zlib-dir)) + +if test "$enable_static_tor" = "yes"; then + enable_static_libevent="yes"; + enable_static_openssl="yes"; + enable_static_zlib="yes"; + CFLAGS="$CFLAGS -static" +fi if test x$enable_buf_freelists != xno; then AC_DEFINE(ENABLE_BUF_FREELISTS, 1, @@ -59,6 +68,24 @@ AC_ARG_ENABLE(asciidoc, *) AC_MSG_ERROR(bad value for --disable-asciidoc) ;; esac], [asciidoc=true]) +# By default, we're not ready to ship a NAT-PMP aware Tor +AC_ARG_ENABLE(nat-pmp, + AS_HELP_STRING(--enable-nat-pmp, enable NAT-PMP support), + [case "${enableval}" in + yes) natpmp=true ;; + no) natpmp=false ;; + * ) AC_MSG_ERROR(bad value for --enable-nat-pmp) ;; + esac], [natpmp=false]) + +# By default, we're not ready to ship a UPnP aware Tor +AC_ARG_ENABLE(upnp, + AS_HELP_STRING(--enable-upnp, enable UPnP support), + [case "${enableval}" in + yes) upnp=true ;; + no) upnp=false ;; + * ) AC_MSG_ERROR(bad value for --enable-upnp) ;; + esac], [upnp=false]) + AC_ARG_ENABLE(threads, AS_HELP_STRING(--disable-threads, disable multi-threading support)) @@ -95,21 +122,12 @@ dnl -D_FORTIFY_SOURCE=2 -fstack-protector-all dnl Others suggest '/gs /safeseh /nxcompat /dynamicbase' for non-gcc on Windows dnl This requires that we use gcc and that we add -O2 to the CFLAGS. AC_ARG_ENABLE(gcc-hardening, - AS_HELP_STRING(--enable-gcc-hardening, enable compiler security checks), -[if test x$enableval = xyes; then - CFLAGS="$CFLAGS -D_FORTIFY_SOURCE=2 -fstack-protector-all" - CFLAGS="$CFLAGS -fwrapv -fPIE -Wstack-protector" - CFLAGS="$CFLAGS --param ssp-buffer-size=1" - LDFLAGS="$LDFLAGS -pie" -fi]) + AS_HELP_STRING(--disable-gcc-hardening, disable compiler security checks)) dnl Linker hardening options dnl Currently these options are ELF specific - you can't use this with MacOSX AC_ARG_ENABLE(linker-hardening, - AS_HELP_STRING(--enable-linker-hardening, enable linker security fixups), -[if test x$enableval = xyes; then - LDFLAGS="$LDFLAGS -z relro -z now" -fi]) + AS_HELP_STRING(--disable-linker-hardening, disable linker security fixups)) AC_ARG_ENABLE(local-appdata, AS_HELP_STRING(--enable-local-appdata, default to host local application data paths on Windows)) @@ -118,6 +136,16 @@ if test "$enable_local_appdata" = "yes"; then [Defined if we default to host local appdata paths on Windows]) fi +# Tor2web mode flag +AC_ARG_ENABLE(tor2web-mode, + AS_HELP_STRING(--enable-tor2web-mode, support tor2web non-anonymous mode), +[if test x$enableval = xyes; then + CFLAGS="$CFLAGS -D ENABLE_TOR2WEB_MODE=1" +fi]) + +AC_ARG_ENABLE(bufferevents, + AS_HELP_STRING(--enable-bufferevents, use Libevent's buffered IO.)) + dnl check for the correct "ar" when cross-compiling AN_MAKEVAR([AR], [AC_PROG_AR]) AN_PROGRAM([ar], [AC_PROG_AR]) @@ -138,6 +166,33 @@ AC_PATH_PROG([A2X], [a2x], none) AM_CONDITIONAL(USE_ASCIIDOC, test x$asciidoc = xtrue) +AM_CONDITIONAL(USE_FW_HELPER, test x$natpmp = xtrue || test x$upnp = xtrue) +AM_CONDITIONAL(NAT_PMP, test x$natpmp = xtrue) +AM_CONDITIONAL(MINIUPNPC, test x$upnp = xtrue) +AM_PROG_CC_C_O + +ifdef([AC_C_FLEXIBLE_ARRAY_MEMBER], [ +AC_C_FLEXIBLE_ARRAY_MEMBER +], [ + dnl Maybe we've got an old autoconf... + AC_CACHE_CHECK([for flexible array members], + tor_cv_c_flexarray, + [AC_COMPILE_IFELSE( + AC_LANG_PROGRAM([ + struct abc { int a; char b[]; }; +], [ + struct abc *def = malloc(sizeof(struct abc)+sizeof(char)); + def->b[0] = 33; +]), + [tor_cv_c_flexarray=yes], + [tor_cv_c_flexarray=no])]) + if test $tor_cv_flexarray = yes ; then + AC_DEFINE([FLEXIBLE_ARRAY_MEMBER], [], [Define to nothing if C supports flexible array members, and to 1 if it does not.]) + else + AC_DEFINE([FLEXIBLE_ARRAY_MEMBER], [1], [Define to nothing if C supports flexible array members, and to 1 if it does not.]) + fi +]) + AC_PATH_PROG([SHA1SUM], [sha1sum], none) AC_PATH_PROG([OPENSSL], [openssl], none) @@ -160,12 +215,12 @@ AC_ARG_WITH(tor-group, AC_SUBST(TORGROUP) -dnl If WIN32 is defined and non-zero, we are building for win32 +dnl If _WIN32 is defined and non-zero, we are building for win32 AC_MSG_CHECKING([for win32]) AC_RUN_IFELSE([AC_LANG_SOURCE([ int main(int c, char **v) { -#ifdef WIN32 -#if WIN32 +#ifdef _WIN32 +#if _WIN32 return 0; #else return 1; @@ -182,7 +237,7 @@ bwin32=cross; AC_MSG_RESULT([cross]) if test "$bwin32" = cross; then AC_MSG_CHECKING([for win32 (cross)]) AC_COMPILE_IFELSE([AC_LANG_SOURCE([ -#ifdef WIN32 +#ifdef _WIN32 int main(int c, char **v) {return 0;} #else #error @@ -193,9 +248,6 @@ bwin32=true; AC_MSG_RESULT([yes]), bwin32=false; AC_MSG_RESULT([no])) fi -if test "$bwin32" = true; then -AC_DEFINE(MS_WINDOWS, 1, [Define to 1 if we are building for Windows.]) -fi AM_CONDITIONAL(BUILD_NT_SERVICES, test x$bwin32 = xtrue) dnl Enable C99 when compiling with MIPSpro @@ -215,10 +267,16 @@ fi AC_C_BIGENDIAN -AC_SEARCH_LIBS(socket, [socket]) +AC_SEARCH_LIBS(socket, [socket network]) AC_SEARCH_LIBS(gethostbyname, [nsl]) AC_SEARCH_LIBS(dlopen, [dl]) AC_SEARCH_LIBS(inet_aton, [resolv]) +saved_LIBS="$LIBS" +AC_SEARCH_LIBS([clock_gettime], [rt]) +if test "$LIBS" != "$saved_LIBS"; then + # Looks like we need -lrt for clock_gettime(). + have_rt=yes +fi if test "$enable_threads" = "yes"; then AC_SEARCH_LIBS(pthread_create, [pthread]) @@ -229,7 +287,35 @@ dnl ------------------------------------------------------------------- dnl Check for functions before libevent, since libevent-1.2 apparently dnl exports strlcpy without defining it in a header. -AC_CHECK_FUNCS(gettimeofday ftime socketpair uname inet_aton strptime getrlimit strlcat strlcpy strtoull getaddrinfo localtime_r gmtime_r memmem strtok_r writev readv flock prctl vasprintf lround rint) +AC_CHECK_FUNCS( + _NSGetEnviron \ + accept4 \ + clock_gettime \ + flock \ + ftime \ + getaddrinfo \ + getifaddrs \ + getrlimit \ + gettimeofday \ + gmtime_r \ + inet_aton \ + ioctl \ + issetugid \ + localtime_r \ + lround \ + memmem \ + prctl \ + rint \ + socketpair \ + strlcat \ + strlcpy \ + strptime \ + strtok_r \ + strtoull \ + sysconf \ + uname \ + vasprintf \ +) using_custom_malloc=no if test x$enable_openbsd_malloc = xyes ; then @@ -253,6 +339,7 @@ dnl Where do you live, libevent? And how do we call you? if test "$bwin32" = true; then TOR_LIB_WS32=-lws2_32 + TOR_LIB_IPHLPAPI=-liphlpapi # Some of the cargo-cults recommend -lwsock32 as well, but I don't # think it's actually necessary. TOR_LIB_GDI=-lgdi32 @@ -262,6 +349,7 @@ else fi AC_SUBST(TOR_LIB_WS32) AC_SUBST(TOR_LIB_GDI) +AC_SUBST(TOR_LIB_IPHLPAPI) dnl We need to do this before we try our disgusting hack below. AC_CHECK_HEADERS([sys/types.h]) @@ -281,28 +369,25 @@ dnl On Gnu/Linux or any place we require it, we'll add librt to the Libevent dnl linking for static builds. STATIC_LIBEVENT_FLAGS="" if test "$enable_static_libevent" = "yes"; then - dnl Determine if we have clock_gettime in librt - AC_SEARCH_LIBS([clock_gettime], [rt], - [have_rt=yes]) if test "$have_rt" = yes; then STATIC_LIBEVENT_FLAGS=" -lrt " fi fi TOR_SEARCH_LIBRARY(libevent, $trylibeventdir, [-levent $STATIC_LIBEVENT_FLAGS $TOR_LIB_WS32], [ -#ifdef WIN32 +#ifdef _WIN32 #include <winsock2.h> #endif #include <stdlib.h> #include <sys/time.h> #include <sys/types.h> #include <event.h>], [ -#ifdef WIN32 +#ifdef _WIN32 #include <winsock2.h> #endif void exit(int); void *event_init(void);], [ -#ifdef WIN32 +#ifdef _WIN32 {WSADATA d; WSAStartup(0x101,&d); } #endif event_init(); exit(0); @@ -320,7 +405,7 @@ AC_CHECK_MEMBERS([struct event.min_heap_idx], , , [#include <event.h> ]) -AC_CHECK_HEADERS(event2/event.h event2/dns.h) +AC_CHECK_HEADERS(event2/event.h event2/dns.h event2/bufferevent_ssl.h) LIBS="$save_LIBS" LDFLAGS="$save_LDFLAGS" @@ -338,8 +423,75 @@ if test "$enable_static_libevent" = "yes"; then else TOR_LIBEVENT_LIBS="-levent" fi + +dnl This isn't the best test for Libevent 2.0.3-alpha. Once it's released, +dnl we can do much better. +if test "$enable_bufferevents" = "yes" ; then + if test "$ac_cv_header_event2_bufferevent_ssl_h" != "yes" ; then + AC_MSG_ERROR([You've asked for bufferevent support, but you're using a version of Libevent without SSL support. This won't work. We need Libevent 2.0.8-rc or later, and you don't seem to even have Libevent 2.0.3-alpha.]) + else + + CPPFLAGS="$CPPFLAGS $TOR_CPPFLAGS_libevent" + + # Check for the right version. First see if version detection works. + AC_MSG_CHECKING([whether we can detect the Libevent version]) + AC_COMPILE_IFELSE([AC_LANG_SOURCE([ +#include <event2/event.h> +#if !defined(LIBEVENT_VERSION_NUMBER) || LIBEVENT_VERSION_NUMBER < 10 +#error +int x = y(zz); +#else +int x = 1; +#endif + ])], [event_version_number_works=yes; AC_MSG_RESULT([yes]) ], + [event_version_number_works=no; AC_MSG_RESULT([no])]) + if test "$event_version_number_works" != 'yes'; then + AC_MSG_WARN([Version detection on Libevent seems broken. Your Libevent installation is probably screwed up or very old.]) + else + AC_MSG_CHECKING([whether Libevent is new enough for bufferevents]) + AC_COMPILE_IFELSE([AC_LANG_SOURCE([ +#include <event2/event.h> +#if !defined(LIBEVENT_VERSION_NUMBER) || LIBEVENT_VERSION_NUMBER < 0x02000d00 +#error +int x = y(zz); +#else +int x = 1; +#endif + ])], [ AC_MSG_RESULT([yes]) ], + [ AC_MSG_RESULT([no]) + AC_MSG_ERROR([Libevent does not seem new enough to support bufferevents. We require 2.0.13-stable or later]) ] ) + fi + fi +fi + +LIBS="$save_LIBS" +LDFLAGS="$save_LDFLAGS" +CPPFLAGS="$save_CPPFLAGS" + +AM_CONDITIONAL(USE_BUFFEREVENTS, test "$enable_bufferevents" = "yes") +if test "$enable_bufferevents" = "yes"; then + AC_DEFINE(USE_BUFFEREVENTS, 1, [Defined if we're going to use Libevent's buffered IO API]) + if test "$enable_static_libevent" = "yes"; then + TOR_LIBEVENT_LIBS="$TOR_LIBDIR_libevent/libevent_openssl.a $TOR_LIBEVENT_LIBS" + else + TOR_LIBEVENT_LIBS="-levent_openssl $TOR_LIBEVENT_LIBS" + fi +fi AC_SUBST(TOR_LIBEVENT_LIBS) +dnl ------------------------------------------------------ +dnl Where do you live, libm? + +dnl On some platforms (Haiku/BeOS) the math library is +dnl part of libroot. In which case don't link against lm +TOR_LIB_MATH="" +save_LIBS="$LIBS" +AC_SEARCH_LIBS(pow, [m], , AC_MSG_ERROR([Could not find pow in libm or libc.])) +if test "$ac_cv_search_pow" != "none required"; then + TOR_LIB_MATH="$ac_cv_search_pow" +fi +LIBS="$save_LIBS" +AC_SUBST(TOR_LIB_MATH) dnl ------------------------------------------------------ dnl Where do you live, openssl? And how do we call you? @@ -366,7 +518,6 @@ TOR_SEARCH_LIBRARY(openssl, $tryssldir, [-lssl -lcrypto $TOR_LIB_GDI], dnl XXXX check for OPENSSL_VERSION_NUMBER == SSLeay() -echo "tor_cv_library_openssl_dir is $tor_cv_library_openssl_dir" if test "$enable_static_openssl" = "yes"; then if test "$tor_cv_library_openssl_dir" = "(system)"; then AC_MSG_ERROR("You must specify an explicit --with-openssl-dir=x option when using --enable-static-openssl") @@ -398,24 +549,159 @@ if test "$enable_static_zlib" = "yes"; then using --enable-static-zlib") else TOR_ZLIB_LIBS="$TOR_LIBDIR_zlib/libz.a" - echo "$TOR_LIBDIR_zlib/libz.a" fi else TOR_ZLIB_LIBS="-lz" fi AC_SUBST(TOR_ZLIB_LIBS) -dnl Make sure to enable support for large off_t if available. +dnl --------------------------------------------------------------------- +dnl Now that we know about our major libraries, we can check for compiler +dnl and linker hardening options. We need to do this with the libraries known, +dnl since sometimes the linker will like an option but not be willing to +dnl use it with a build of a library. -AC_SYS_LARGEFILE +all_ldflags_for_check="$TOR_LDFLAGS_zlib $TOR_LDFLAGS_openssl $TOR_LDFLAGS_libevent" +all_libs_for_check="$TOR_ZLIB_LIBS $TOR_LIB_MATH $TOR_LIBEVENT_LIBS $TOR_OPENSSL_LIBS $TOR_LIB_WS32 $TOR_LIB_GDI" -AC_CHECK_HEADERS(unistd.h string.h signal.h sys/stat.h sys/types.h fcntl.h sys/fcntl.h sys/time.h errno.h assert.h time.h, , AC_MSG_WARN(Some headers were not found, compilation may fail. If compilation succeeds, please send your orconfig.h to the developers so we can fix this warning.)) +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ +#if !defined(__clang__) +#error +#endif])], have_clang=yes, have_clang=no) -AC_CHECK_HEADERS(netdb.h sys/ioctl.h sys/socket.h arpa/inet.h netinet/in.h pwd.h grp.h sys/un.h sys/uio.h) +if test x$enable_gcc_hardening != xno; then + CFLAGS="$CFLAGS -D_FORTIFY_SOURCE=2" + if test x$have_clang = xyes; then + TOR_CHECK_CFLAGS(-Qunused-arguments) + fi + TOR_CHECK_CFLAGS(-fstack-protector-all) + TOR_CHECK_CFLAGS(-Wstack-protector) + TOR_CHECK_CFLAGS(-fwrapv) + TOR_CHECK_CFLAGS(--param ssp-buffer-size=1) + if test "$bwin32" = "false"; then + TOR_CHECK_CFLAGS(-fPIE) + TOR_CHECK_LDFLAGS(-pie, "$all_ldflags_for_check", "$all_libs_for_check") + fi +fi + +if test x$enable_linker_hardening != xno; then + TOR_CHECK_LDFLAGS(-z relro -z now, "$all_ldflags_for_check", "$all_libs_for_check") +fi + +dnl ------------------------------------------------------ +dnl Where do you live, libnatpmp? And how do we call you? +dnl There are no packages for Debian or Redhat as of this patch + +if test "$natpmp" = "true"; then + AC_DEFINE(NAT_PMP, 1, [Define to 1 if we are building with nat-pmp.]) + TOR_SEARCH_LIBRARY(libnatpmp, $trylibnatpmpdir, [-lnatpmp $TOR_LIB_WS32 $TOR_LIB_IPHLPAPI], + [#include <natpmp.h>], + [#ifdef _WIN32 + #define STATICLIB + #endif + #include <natpmp.h>], + [ int r; + natpmp_t natpmp; + natpmpresp_t response; + r = initnatpmp(&natpmp, 0, 0);], + [printf("initnatpmp() returned %d (%s)\n", r, r?"FAILED":"SUCCESS"); + exit(0);], + [--with-libnatpmp-dir], + [/usr/lib/]) +fi + + +dnl ------------------------------------------------------ +dnl Where do you live, libminiupnpc? And how do we call you? +dnl There are no packages for Debian or Redhat as of this patch + +if test "$upnp" = "true"; then + AC_DEFINE(MINIUPNPC, 1, [Define to 1 if we are building with UPnP.]) + + dnl Before we call TOR_SEARCH_LIBRARY we'll do a quick compile test + dnl to see if we have miniupnpc-1.5 or -1.6 + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([#include <miniupnpc/miniupnpc.h>], + [upnpDiscover(1, 0, 0, 0);exit(0);])],[miniupnpc15="true"],[miniupnpc15="false"]) + + if test "$miniupnpc15" = "true" ; then + AC_DEFINE([MINIUPNPC15],[1],[libminiupnpc version 1.5 found]) + TOR_SEARCH_LIBRARY(libminiupnpc, $trylibminiupnpcdir, [-lminiupnpc $TOR_LIB_WS32 $TOR_LIB_IPHLPAPI], + [#include <miniupnpc/miniwget.h> + #include <miniupnpc/miniupnpc.h> + #include <miniupnpc/upnpcommands.h>], + [void upnpDiscover(int delay, const char * multicastif, + const char * minissdpdsock, int sameport);], + [upnpDiscover(1, 0, 0, 0); exit(0);], + [--with-libminiupnpc-dir], + [/usr/lib/]) + else + TOR_SEARCH_LIBRARY(libminiupnpc, $trylibminiupnpcdir, [-lminiupnpc $TOR_LIB_WS32 $TOR_LIB_IPHLPAPI], + [#include <miniupnpc/miniwget.h> + #include <miniupnpc/miniupnpc.h> + #include <miniupnpc/upnpcommands.h>], + [void upnpDiscover(int delay, const char * multicastif, + const char * minissdpdsock, int sameport, int ipv6, int * error);], + [upnpDiscover(1, 0, 0, 0, 0, 0); exit(0);], + [--with-libminiupnpc-dir], + [/usr/lib/]) + fi +fi + +dnl Make sure to enable support for large off_t if available. +AC_SYS_LARGEFILE + +AC_CHECK_HEADERS( + assert.h \ + errno.h \ + fcntl.h \ + signal.h \ + string.h \ + sys/fcntl.h \ + sys/stat.h \ + sys/time.h \ + sys/types.h \ + time.h \ + unistd.h + , , AC_MSG_WARN(Some headers were not found, compilation may fail. If compilation succeeds, please send your orconfig.h to the developers so we can fix this warning.)) dnl These headers are not essential -AC_CHECK_HEADERS(stdint.h sys/types.h inttypes.h sys/param.h sys/wait.h limits.h sys/limits.h netinet/in.h arpa/inet.h machine/limits.h syslog.h sys/time.h sys/resource.h inttypes.h utime.h sys/utime.h sys/mman.h netinet/in6.h malloc.h sys/syslimits.h malloc/malloc.h linux/types.h sys/file.h malloc_np.h sys/prctl.h) +AC_CHECK_HEADERS( + arpa/inet.h \ + crt_externs.h \ + grp.h \ + ifaddrs.h \ + inttypes.h \ + limits.h \ + linux/types.h \ + machine/limits.h \ + malloc.h \ + malloc/malloc.h \ + malloc_np.h \ + netdb.h \ + netinet/in.h \ + netinet/in6.h \ + pwd.h \ + stdint.h \ + sys/file.h \ + sys/ioctl.h \ + sys/limits.h \ + sys/mman.h \ + sys/param.h \ + sys/prctl.h \ + sys/resource.h \ + sys/socket.h \ + sys/syslimits.h \ + sys/time.h \ + sys/types.h \ + sys/un.h \ + sys/utime.h \ + sys/wait.h \ + syslog.h \ + utime.h +) + +AC_CHECK_HEADERS(sys/param.h) TOR_CHECK_PROTOTYPE(malloc_good_size, HAVE_MALLOC_GOOD_SIZE_PROTOTYPE, [#ifdef HAVE_MALLOC_H @@ -532,9 +818,8 @@ AC_CHECK_TYPES([struct in6_addr, struct sockaddr_in6, sa_family_t], , , #ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> #endif -#ifdef MS_WINDOWS -#define WIN32_WINNT 0x400 -#define _WIN32_WINNT 0x400 +#ifdef _WIN32 +#define _WIN32_WINNT 0x0501 #define WIN32_LEAN_AND_MEAN #if defined(_MSC_VER) && (_MSC_VER < 1300) #include <winsock.h> @@ -557,9 +842,8 @@ AC_CHECK_MEMBERS([struct in6_addr.s6_addr32, struct in6_addr.s6_addr16, struct s #ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> #endif -#ifdef MS_WINDOWS -#define WIN32_WINNT 0x400 -#define _WIN32_WINNT 0x400 +#ifdef _WIN32 +#define _WIN32_WINNT 0x0501 #define WIN32_LEAN_AND_MEAN #if defined(_MSC_VER) && (_MSC_VER < 1300) #include <winsock.h> @@ -847,6 +1131,20 @@ int main(int c, char **v) { puts(__FUNCTION__); }])], tor_cv_have_FUNCTION_macro=yes, tor_cv_have_FUNCTION_macro=no)) +AC_CACHE_CHECK([whether we have extern char **environ already declared], + tor_cv_have_environ_declared, + AC_COMPILE_IFELSE([AC_LANG_SOURCE([ +/* We define _GNU_SOURCE here because it is also defined in compat.c. + * Without it environ doesn't get declared. */ +#define _GNU_SOURCE +#ifdef HAVE_UNISTD_H +#include <unistd.h> +#endif +#include <stdlib.h> +int main(int c, char **v) { char **t = environ; }])], + tor_cv_have_environ_declared=yes, + tor_cv_have_environ_declared=no)) + if test "$tor_cv_have_func_macro" = 'yes'; then AC_DEFINE(HAVE_MACRO__func__, 1, [Defined if the compiler supports __func__]) fi @@ -860,6 +1158,11 @@ if test "$tor_cv_have_FUNCTION_macro" = 'yes'; then [Defined if the compiler supports __FUNCTION__]) fi +if test "$tor_cv_have_environ_declared" = 'yes'; then + AC_DEFINE(HAVE_EXTERN_ENVIRON_DECLARED, 1, + [Defined if we have extern char **environ already declared]) +fi + # $prefix stores the value of the --prefix command line option, or # NONE if the option wasn't set. In the case that it wasn't set, make # it be the default, so that we can use it to expand directories now. @@ -872,6 +1175,13 @@ if test "x$exec_prefix" = "xNONE"; then exec_prefix=$prefix fi +if test "x$BUILDDIR" = "x"; then + BUILDDIR=`pwd` +fi +AC_SUBST(BUILDDIR) +AH_TEMPLATE([BUILDDIR],[tor's build directory]) +AC_DEFINE_UNQUOTED(BUILDDIR,"$BUILDDIR") + if test "x$CONFDIR" = "x"; then CONFDIR=`eval echo $sysconfdir/tor` fi @@ -913,12 +1223,13 @@ else fi # OS X Lion started deprecating the system openssl. Let's just disable -# all deprecation warnings on OS X. +# all deprecation warnings on OS X. Also, to potentially make the binary +# a little smaller, let's enable dead_strip. case "$host_os" in darwin*) CFLAGS="$CFLAGS -Wno-deprecated-declarations" - ;; + LDFLAGS="$LDFLAGS -dead_strip" ;; esac # Add some more warnings which we use in development but not in the @@ -940,11 +1251,6 @@ if test x$enable_gcc_warnings = xyes || test x$enable_gcc_warnings_advisory = xy #error #endif])], have_gcc43=yes, have_gcc43=no) -AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ -#if !defined(__clang__) -#error -#endif])], have_clang=yes, have_clang=no) - save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Wshorten-64-to-32" AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [])], have_shorten64_flag=yes, @@ -960,7 +1266,12 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ CFLAGS="$CFLAGS -Wno-system-headers" ;; esac - CFLAGS="$CFLAGS -W -Wfloat-equal -Wundef -Wpointer-arith -Wstrict-prototypes -Wmissing-prototypes -Wwrite-strings -Wredundant-decls -Wchar-subscripts -Wcomment -Wformat=2 -Wwrite-strings -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wbad-function-cast -Wswitch-enum" + CFLAGS="$CFLAGS -W -Wfloat-equal -Wundef -Wpointer-arith" + CFLAGS="$CFLAGS -Wstrict-prototypes -Wmissing-prototypes -Wwrite-strings" + CFLAGS="$CFLAGS -Wredundant-decls -Wchar-subscripts -Wcomment -Wformat=2" + CFLAGS="$CFLAGS -Wwrite-strings -Wmissing-declarations -Wredundant-decls" + CFLAGS="$CFLAGS -Wnested-externs -Wbad-function-cast -Wswitch-enum" + if test x$enable_gcc_warnings = xyes; then CFLAGS="$CFLAGS -Werror" fi @@ -972,7 +1283,7 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ CFLAGS="$CFLAGS -Winit-self -Wmissing-field-initializers -Wdeclaration-after-statement -Wold-style-definition" fi - if test x$have_gcc42 = xyes ; then + if test x$have_gcc42 = xyes ; then # These warnings break gcc 4.0.2 and work on gcc 4.2 # XXXX020 See if any of these work with earlier versions. CFLAGS="$CFLAGS -Waddress -Wmissing-noreturn -Wstrict-overflow=1" @@ -1003,7 +1314,29 @@ fi CPPFLAGS="$CPPFLAGS $TOR_CPPFLAGS_libevent $TOR_CPPFLAGS_openssl $TOR_CPPFLAGS_zlib" -AC_CONFIG_FILES([Makefile tor.spec Doxyfile contrib/tor.sh contrib/torctl contrib/torify contrib/tor.logrotate contrib/Makefile src/config/torrc.sample src/Makefile doc/Makefile src/config/Makefile src/common/Makefile src/or/Makefile src/test/Makefile src/win32/Makefile src/tools/Makefile contrib/suse/Makefile contrib/suse/tor.sh]) +AC_CONFIG_FILES([ + Doxyfile + Makefile + contrib/Makefile + contrib/suse/Makefile + contrib/suse/tor.sh + contrib/tor.logrotate + contrib/tor.sh + contrib/torctl + contrib/torify + doc/Makefile + src/Makefile + src/common/Makefile + src/config/Makefile + src/config/torrc.sample + src/or/Makefile + src/test/Makefile + src/tools/Makefile + src/tools/tor-fw-helper/Makefile + src/win32/Makefile + tor.spec +]) + AC_OUTPUT if test -x /usr/bin/perl && test -x ./contrib/updateVersions.pl ; then diff --git a/contrib/Makefile.am b/contrib/Makefile.am index 5aae2c819e..795c351f3a 100644 --- a/contrib/Makefile.am +++ b/contrib/Makefile.am @@ -3,7 +3,20 @@ DIST_SUBDIRS = suse confdir = $(sysconfdir)/tor -EXTRA_DIST = exitlist tor-tsocks.conf tor.nsi.in tor.sh torctl rc.subr cross.sh tor-mingw.nsi.in package_nsis-mingw.sh tor.ico tor-ctrl.sh linux-tor-prio.sh tor-exit-notice.html +EXTRA_DIST = \ + cross.sh \ + exitlist \ + linux-tor-prio.sh \ + package_nsis-mingw.sh \ + rc.subr \ + tor-ctrl.sh \ + tor-exit-notice.html \ + tor-mingw.nsi.in \ + tor-tsocks.conf \ + tor.ico \ + tor.nsi.in \ + tor.sh \ + torctl conf_DATA = tor-tsocks.conf diff --git a/contrib/checkSpace.pl b/contrib/checkSpace.pl index 6eb32e5620..682dbced00 100755 --- a/contrib/checkSpace.pl +++ b/contrib/checkSpace.pl @@ -32,10 +32,18 @@ for $fn (@ARGV) { if ($C && /\s(?:if|while|for|switch)\(/) { print " KW(:$fn:$.\n"; } - ## Warn about #else #if instead of #elif. + ## Warn about #else #if instead of #elif. if (($lastline =~ /^\# *else/) and ($_ =~ /^\# *if/)) { print " #else#if:$fn:$.\n"; } + ## Warn about some K&R violations + if (/^\s+\{/ and $lastline =~ /^\s*(if|while|for|else if)/ and + $lastline !~ /\{$/) { + print "non-K&R {:$fn:$.\n"; + } + if (/^\s*else/ and $lastline =~ /\}$/) { + print " }\\nelse:$fn:$.\n"; + } $lastline = $_; ## Warn about unnecessary empty lines. if ($lastnil && /^\s*}\n/) { @@ -97,7 +105,7 @@ for $fn (@ARGV) { if ($1 ne "if" and $1 ne "while" and $1 ne "for" and $1 ne "switch" and $1 ne "return" and $1 ne "int" and $1 ne "elsif" and $1 ne "WINAPI" and $2 ne "WINAPI" and - $1 ne "void" and $1 ne "__attribute__") { + $1 ne "void" and $1 ne "__attribute__" and $1 ne "op") { print " fn ():$fn:$.\n"; } } diff --git a/contrib/exitlist b/contrib/exitlist index ba682b5c06..3fd26b5166 100755 --- a/contrib/exitlist +++ b/contrib/exitlist @@ -8,23 +8,25 @@ exitlist -- Given a Tor directory on stdin, lists the Tor servers that accept connections to given addreses. - example usage (Tor 0.2.0.7-alpha and earlier): - - cat ~/.tor/cached-routers* | python exitlist 18.244.0.188:80 - - example usage (Tor 0.2.0.8-alpha and later): + example usage: cat ~/.tor/cached-descriptors* | python exitlist 18.244.0.188:80 - You should look at the "FetchUselessDescriptors" config option in the - man page. For 0.2.0.13-alpha and later, also look at the - "FetchDirInfoEarly" config option. + You should look at the "FetchUselessDescriptors" and "FetchDirInfoEarly" + config options in the man page. Note that this script won't give you a perfect list of IP addresses - that might connect to you using Tor, since some Tor servers might exit - from other addresses than the one they publish. See - https://check.torproject.org/ for an alternative (more - accurate!) approach. + that might connect to you using Tor. + False negatives: + - Some Tor servers might exit from other addresses than the one they + publish in their descriptor. + False positives: + - This script just looks at the descriptor lists, so it counts relays + that were running a day in the past and aren't running now (or are + now running at a different address). + + See https://check.torproject.org/ for an alternative (more accurate!) + approach. """ diff --git a/contrib/findMergedChanges.pl b/contrib/findMergedChanges.pl new file mode 100755 index 0000000000..46e070f943 --- /dev/null +++ b/contrib/findMergedChanges.pl @@ -0,0 +1,70 @@ +#!/usr/bin/perl + +use warnings; +use strict; + +sub nChanges { + my ($branches, $fname) = @_; + local *F; + # requires perl 5.8. Avoids shell issues if we ever get a changes + # file named by the parents of Little Johnny Tables. + open F, "-|", "git", "log", "--pretty=format:%H", $branches, "--", $fname + or die "$!"; + my @changes = <F>; + return scalar @changes +} + +my $look_for_type = "merged"; + +if (! @ARGV) { + print <<EOF +Usage: + findMergedChanges.pl [--merged/--unmerged/--weird/--list] changes/* + +A change is "merged" if it has ever been merged to release-0.2.2 and it has had +no subsequent changes in master. + +A change is "unmerged" if it has never been merged to release-0.2.2 and it +has had changes in master. + +A change is "weird" if it has been merged to release-0.2.2 and it *has* had +subsequent changes in master. + +Suggested application: + findMergedChanges.pl --merged changes/* | xargs -n 1 git rm + +EOF +} + +my $target_branch = "origin/release-0.2.2"; + +while (@ARGV and $ARGV[0] =~ /^--/) { + my $flag = shift @ARGV; + if ($flag =~ /^--(weird|merged|unmerged|list)/) { + $look_for_type = $1; + } elsif ($flag =~ /^--branch=(\S+)/) { + $target_branch = $1; + } else { + die "Unrecognized flag $flag"; + } +} + +for my $changefile (@ARGV) { + my $n_merged = nChanges($target_branch, $changefile); + my $n_postmerged = nChanges("${target_branch}..origin/master", $changefile); + my $type; + + if ($n_merged != 0 and $n_postmerged == 0) { + $type = "merged"; + } elsif ($n_merged == 0 and $n_postmerged != 0) { + $type = "unmerged"; + } else { + $type = "weird"; + } + + if ($type eq $look_for_type) { + print "$changefile\n"; + } elsif ($look_for_type eq 'list') { + printf "% 8s: %s\n", $type, $changefile; + } +} diff --git a/contrib/make-signature.sh b/contrib/make-signature.sh new file mode 100755 index 0000000000..4aba08b754 --- /dev/null +++ b/contrib/make-signature.sh @@ -0,0 +1,79 @@ +#!/bin/sh + +set -eu + +if test "$1" = "" ; then + echo "I need a package as an argument." + exit 1 +fi + +PACKAGEFILE=$1 + +if test ! -f "$PACKAGEFILE" ; then + echo "$PACKAGEFILE is not a file." + exit 1 +fi + +DIGESTNAME=sha256 +DIGESTOUTPUT=`gpg --print-md $DIGESTNAME $PACKAGEFILE` + +RAWDIGEST=`gpg --print-md $DIGESTNAME $PACKAGEFILE | sed -e 's/^[^ ]*: //' ` + +# These regexes are a little fragile, but I think they work for us. +VERSION=`echo $PACKAGEFILE | sed -e 's/^[a-z\-]*//' -e 's/\.[\.a-z]*$//' ` +PACKAGE=`echo $PACKAGEFILE | sed -e 's/-[0-9].*//'` +SIGFILE_UNSIGNED="$PACKAGE-$VERSION-signature" +SIGNATUREFILE="$SIGFILE_UNSIGNED.asc" + +cat >$SIGFILE_UNSIGNED <<EOF +This is the signature file for "$PACKAGEFILE", +which contains version "$VERSION" of "$PACKAGE". + +Here's how to check this signature. + +1) Make sure that this is really a signature file, and not a forgery, + with: + + "gpg --verify $SIGNATUREFILE" + + The key should be one of the keys that signs the Tor release; the + official Tor website has more information on those. + + If this step fails, then either you are missing the correct key, or + this signature file was not really signed by a Tor packager. + Beware! + +2) Make sure that the package you wanted is indeed "$PACKAGE", and that + its version you wanted is indeed "$VERSION". If you wanted a + different package, or a different version, this signature file is + not the right one! + +3) Now that you're sure you have the right signature file, make sure + that you got the right package. Check its $DIGESTNAME digest with + + "gpg --print-md $DIGESTNAME $PACKAGEFILE" + + The output should match this, exactly: + +$DIGESTOUTPUT + + Make sure that every part of the output matches: don't just check the + first few characters. If the digest does not match, you do not have + the right package file. It could even be a forgery. + +Frequently asked questions: + +Q: Why not just sign the package file, like you used to do? +A: GPG signatures authenticate file contents, but not file names. If + somebody gave you a renamed file with a matching renamed signature + file, the signature would still be given as "valid". + +-- +FILENAME: $PACKAGEFILE +PACKAGE: $PACKAGE +VERSION: $VERSION +DIGESTALG: $DIGESTNAME +DIGEST: $RAWDIGEST +EOF + +gpg --clearsign $SIGFILE_UNSIGNED diff --git a/contrib/redox.py b/contrib/redox.py index e18a92f44d..43200dd076 100755 --- a/contrib/redox.py +++ b/contrib/redox.py @@ -1,6 +1,6 @@ #!/usr/bin/python # -# Copyright (c) 2008 The Tor Project, Inc. +# Copyright (c) 2008-2012 The Tor Project, Inc. # See LICENSE for licensing information. # # Hi! @@ -37,6 +37,8 @@ SKIP_FILES = [ "OpenBSD_malloc_Linux.c", "eventdns.h", "strlcat.c", "strlcpy.c", + "sha256.c", + "sha256.h", "aes.c", "aes.h" ] @@ -46,8 +48,8 @@ SKIP_NAME_PATTERNS = [ r'^.*_c_id$', # Which types of things should get DOCDOC comments added if they are # missing documentation? Recognized types are in KINDS below. -#ADD_DOCDOCS_TO_TYPES = [ 'function', 'type', 'typedef' ] -ADD_DOCDOCS_TO_TYPES = [ 'variable' ] +ADD_DOCDOCS_TO_TYPES = [ 'function', 'type', 'typedef' ] +ADD_DOCDOCS_TO_TYPES += [ 'variable', ] # ==================== # The rest of this should not need hacking. @@ -55,11 +57,12 @@ ADD_DOCDOCS_TO_TYPES = [ 'variable' ] import re import sys -KINDS = [ "type", "field", "typedef", "define", "function", "variable" ] +KINDS = [ "type", "field", "typedef", "define", "function", "variable", + "enumeration" ] NODOC_LINE_RE = re.compile(r'^([^:]+):(\d+): (\w+): (.*) is not documented\.$') -THING_RE = re.compile(r'^Member ([a-zA-Z0-9_]+).*\((typedef|define|function|variable)\) of (file|class) ') +THING_RE = re.compile(r'^Member ([a-zA-Z0-9_]+).*\((typedef|define|function|variable|enumeration)\) of (file|class) ') SKIP_NAMES = [re.compile(s) for s in SKIP_NAME_PATTERNS] @@ -72,7 +75,7 @@ def parsething(thing): else: m = THING_RE.match(thing) if not m: - print thing + print thing, "???? Format didn't match." return None, None else: name, tp, parent = m.groups() @@ -92,7 +95,7 @@ def read(): m = NODOC_LINE_RE.match(line) if m: file, line, tp, thing = m.groups() - assert tp == 'Warning' + assert tp.lower() == 'warning' name, kind = parsething(thing) errs.setdefault(file, []).append((int(line), name, kind)) diff --git a/contrib/tor-exit-notice.html b/contrib/tor-exit-notice.html index de3be1744b..8cf5c294f2 100644 --- a/contrib/tor-exit-notice.html +++ b/contrib/tor-exit-notice.html @@ -87,7 +87,7 @@ seize this router will accomplish nothing.</p> <p> Furthermore, this machine also serves as a carrier of email, which means that its contents are further protected under the ECPA. <a -href="http://www4.law.cornell.edu/uscode/html/uscode18/usc_sec_18_00002707----000-.html">18 +href="http://www.law.cornell.edu/uscode/text/18/2707">18 USC 2707</a> explicitly allows for civil remedies ($1000/account <i><b>plus</b></i> legal fees) in the event of a seizure executed without good faith or probable cause (it @@ -105,7 +105,7 @@ used to violate the DMCA, please be aware that this machine does not host or contain any illegal content. Also be aware that network infrastructure maintainers are not liable for the type of content that passes over their equipment, in accordance with <a -href="http://www4.law.cornell.edu/uscode/html/uscode17/usc_sec_17_00000512----000-.html">DMCA +href="http://www.law.cornell.edu/uscode/text/17/512">DMCA "safe harbor" provisions</a>. In other words, you will have just as much luck sending a takedown notice to the Internet backbone providers. Please consult <a href="https://www.torproject.org/eff/tor-dmca-response">EFF's prepared diff --git a/contrib/tor-mingw.nsi.in b/contrib/tor-mingw.nsi.in index 6b359ad8e0..2133a471b3 100644 --- a/contrib/tor-mingw.nsi.in +++ b/contrib/tor-mingw.nsi.in @@ -8,7 +8,7 @@ !include "LogicLib.nsh" !include "FileFunc.nsh" !insertmacro GetParameters -!define VERSION "0.2.2.34-dev" +!define VERSION "0.2.3.18-rc-dev" !define INSTALLER "tor-${VERSION}-win32.exe" !define WEBSITE "https://www.torproject.org/" !define LICENSE "LICENSE" diff --git a/contrib/torify.in b/contrib/torify.in index d430da8ce7..54acfed654 100755 --- a/contrib/torify.in +++ b/contrib/torify.in @@ -1,71 +1,61 @@ #! /bin/sh -# Wrapper script for use of the tsocks(8) transparent socksification library -# See the tsocks(1) and torify(1) manpages. - +# This script used to call (the now deprecated) tsocks as a fallback in case +# torsocks wasn't installed. +# Now, it's just a backwards compatible shim around torsocks with reasonable +# behavior if -v/--verbose or -h/--help arguments are passed. +# # Copyright (c) 2004, 2006, 2009 Peter Palfrader # Modified by Jacob Appelbaum <jacob@appelbaum.net> April 16th 2006 +# Stripped of all the tsocks cruft by ugh on February 22nd 2012 # May be distributed under the same terms as Tor itself -# taken from Debian's Developer's Reference, 6.4 -pathfind() { - OLDIFS="$IFS" - IFS=: - for p in $PATH; do - if [ -x "$p/$*" ]; then - IFS="$OLDIFS" - return 0 - fi - done - IFS="$OLDIFS" - return 1 -} -# Check for any argument list -if [ "$#" = 0 ]; then - echo "Usage: $0 [-hv] <command> [<options>...]" >&2 - exit 1 -fi +compat() { + echo "torify is now just a wrapper around torsocks(1) for backwards compatibility." +} -if [ "$#" = 1 ] && ( [ "$1" = "-h" ] || [ "$1" = "--help" ] ); then +usage() { + compat echo "Usage: $0 [-hv] <command> [<options>...]" - exit 0 -fi - -if [ "$1" = "-v" ] || [ "$1" = "--verbose" ]; then - verbose=1 - shift 1 -else - verbose=0 -fi - -if pathfind torsocks; then - ! [ "$verbose" -ge 1 ] || echo "Using torsocks as socksifier." >&2 +} - exec torsocks "$@" - echo "$0: Failed to exec torsocks $@" >&2 +case $# in 0) + usage >&2 exit 1 +esac -elif pathfind tsocks; then - ! [ "$verbose" -ge 1 ] || echo "Using tsocks as socksifier." >&2 +case $# in 1) + case $1 in -h|--help) + usage + exit 0 + esac +esac - # Define our tsocks config file - TSOCKS_CONF_FILE="/etc/tor/tor-tsocks.conf" - export TSOCKS_CONF_FILE +case $1 in -v|--verbose) + compat >&2 + shift +esac - # Check that we've got a tsocks config file - if [ -r "$TSOCKS_CONF_FILE" ] - then - echo "WARNING: tsocks is known to leak DNS and UDP data. If you had torsocks we would use that." >&2 - exec tsocks "$@" - echo "$0: Failed to exec tsocks $@" >&2 - exit 1 - else - echo "$0: Missing tsocks configuration file \"$TSOCKS_CONF_FILE\"." >&2 - exit 1 - fi +# taken from Debian's Developer's Reference, 6.4 +pathfind() { + OLDIFS="$IFS" + IFS=: + for p in $PATH; do + if [ -x "$p/$*" ]; then + IFS="$OLDIFS" + return 0 + fi + done + IFS="$OLDIFS" + return 1 +} +if pathfind torsocks; then + exec torsocks "$@" + echo "$0: Failed to exec torsocks $@" >&2 + exit 1 else - echo "$0: Can't find either tsocks or torsocks in your PATH. Perhaps you haven't installed either?" >&2 - exit 1 + echo "$0: torsocks not found in your PATH. Perhaps it isn't installed? (tsocks is no longer supported, for security reasons.)" >&2 fi + diff --git a/doc/HACKING b/doc/HACKING index 9747f22651..bc409dc0d0 100644 --- a/doc/HACKING +++ b/doc/HACKING @@ -426,10 +426,10 @@ interesting and understandable. first entry or two and the last entry most interesting: they're the ones that skimmers tend to read. - 2.4) Clean them up + 2.4) Clean them up: Standard idioms: - "Fixes bug 9999; Bugfix on 0.3.3.3-alpha." + "Fixes bug 9999; bugfix on 0.3.3.3-alpha." One period after a space. @@ -446,6 +446,11 @@ interesting and understandable. Present and imperative tense: not past. + Try not to let any given section be longer than about a page. Break up + long sections into subsections by some sort of common subtopic. This + guideline is especially important when organizing Release Notes for + new stable releases. + If a given changes stanza showed up in a different release (e.g. maint-0.2.1), be sure to make the stanzas identical (so people can distinguish if these are the same change). @@ -468,7 +473,7 @@ git branches too. a while to see if anybody has problems building it. Try to get Sebastian or somebody to try building it on Windows. -6) Get at least two of weasel/arma/karsten to put the new version number +6) Get at least two of weasel/arma/sebastian to put the new version number in their approved versions list. 7) Sign the tarball, then sign and push the git tag: diff --git a/doc/Makefile.am b/doc/Makefile.am index bc3d8df475..6cdd66d517 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -12,15 +12,21 @@ # part of the source distribution, so that people without asciidoc can # just use the .1 and .html files. +regular_mans = tor tor-gencert tor-resolve torify +all_mans = $(regular_mans) tor-fw-helper + if USE_ASCIIDOC -asciidoc_files = tor tor-gencert tor-resolve torify -html_in = $(asciidoc_files:=.html.in) -man_in = $(asciidoc_files:=.1.in) -txt_in = $(asciidoc_files:=.1.txt) -nodist_man_MANS = $(asciidoc_files:=.1) -doc_DATA = $(asciidoc_files:=.html) +if USE_FW_HELPER +nodist_man_MANS = $(all_mans:=.1) +doc_DATA = $(all_mans:=.html) +else +nodist_man_MANS = $(regular_mans:=.1) +doc_DATA = $(regular_mans:=.html) +endif +html_in = $(all_mans:=.html.in) +man_in = $(all_mans:=.1.in) +txt_in = $(all_mans:=.1.txt) else -asciidoc_files = html_in = man_in = txt_in = @@ -31,7 +37,8 @@ endif EXTRA_DIST = HACKING asciidoc-helper.sh \ $(html_in) $(man_in) $(txt_in) \ tor-rpm-creation.txt \ - tor-win32-mingw-creation.txt spec/README + tor-win32-mingw-creation.txt spec/README \ + state-contents.txt docdir = @docdir@ @@ -46,6 +53,7 @@ tor.html.in : tor.1.txt torify.html.in : torify.1.txt tor-gencert.html.in : tor-gencert.1.txt tor-resolve.html.in : tor-resolve.1.txt +tor-fw-helper.html.in : tor-fw-helper.1.txt # Generate the manpage from asciidoc, but don't do # machine-specific replacements yet @@ -56,6 +64,7 @@ tor.1.in : tor.1.txt torify.1.in : torify.1.txt tor-gencert.1.in : tor-gencert.1.txt tor-resolve.1.in : tor-resolve.1.txt +tor-fw-helper.1.in : tor-fw-helper.1.txt # use ../config.status to swap all machine-specific magic strings # in the asciidoc with their replacements. @@ -69,10 +78,12 @@ tor.1 : tor.1.in torify.1 : torify.1.in tor-gencert.1 : tor-gencert.1.in tor-resolve.1 : tor-resolve.1.in +tor-fw-helper.1 : tor-fw-helper.1.in tor.html : tor.html.in torify.html : torify.html.in tor-gencert.html : tor-gencert.html.in tor-resolve.html : tor-resolve.html.in +tor-fw-helper.html : tor-fw-helper.html.in CLEANFILES = $(asciidoc_product) config.log DISTCLEANFILES = $(html_in) $(man_in) diff --git a/doc/asciidoc-helper.sh b/doc/asciidoc-helper.sh index 00f8b8d07f..dd420f7c4f 100755 --- a/doc/asciidoc-helper.sh +++ b/doc/asciidoc-helper.sh @@ -17,6 +17,7 @@ output=$3 if [ "$1" = "html" ]; then input=${output%%.html.in}.1.txt base=${output%%.html.in} + if [ "$2" != none ]; then "$2" -d manpage -o $output $input; else @@ -32,7 +33,7 @@ if [ "$1" = "html" ]; then elif [ "$1" = "man" ]; then input=${output%%.1.in}.1.txt base=${output%%.1.in} - + if test "$2" = none; then echo "=================================="; echo; @@ -53,7 +54,9 @@ You need a working asciidoc installed to be able to build the manpage. a2x is installed, but for some reason it isn't working. Sometimes This happens because required docbook support files are missing. Please install docbook-xsl, docbook-xml, and libxml2-utils (Debian) or -similar. +similar. If you use homebrew on Mac OS X, install the docbook formula +and add "export XML_CATALOG_FILES=/usr/local/etc/xml/catalog" to your +.bashrc Alternatively, to build without manpages, use the --disable-asciidoc argument when calling configure. diff --git a/doc/state-contents.txt b/doc/state-contents.txt new file mode 100644 index 0000000000..44716efc0c --- /dev/null +++ b/doc/state-contents.txt @@ -0,0 +1,105 @@ + +Contents of the Tor state file +============================== + +The state file is structured with more or less the same rules as torrc. +Recognized fields are: + + TorVersion + + The version of Tor that wrote this file + + LastWritten + + Time when this state file was written. + Given in ISO format (YYYY-MM-DD HH:MM:SS) + + AccountingBytesReadInInterval (memory unit) + AccountingBytesWrittenInInterval (memory unit) + AccountingExpectedUsage (memory unit) + AccountingIntervalStart (ISO time) + AccountingSecondsActive (time interval) + AccountingSecondsToReachSoftLimit (time interval) + AccountingSoftLimitHitAt (ISO time) + AccountingBytesAtSoftLimit (memory unit) + + These fields describe the state of the accounting subsystem. + + The IntervalStart is the time at which the current accounting + interval began. We were expecting to use ExpectedUsage over the + course of the interval. BytesRead/BytesWritten are the total + number of bytes transferred over the whole interval. If Tor has + been active during the interval, then AccountingSecondsActive is + the amount of time for which it has been active. We were expecting + to hit the bandwidth soft limit in SecondsToReachSoftLimit after we + became active. When we hit the soft limit, we record + BytesAtSoftLimit. If we hit the soft limit already, we did so at + SoftLimitHitAt. + + EntryGuard + EntryGuardDownSince + EntryGuardUnlistedSince + EntryGuardAddedBy + + These lines form sections related to entry guards. Each section + starts with a single EntryGuard line, and is then followed by + information on the state of the Entry guard. + + The EntryGuard line contains a nickname, then an identity digest, of + the guard. + + The EntryGuardDownSince and EntryGuardUnlistedSince lines are present + if the entry guard is believed to be non-running or non-listed. If + present, they contain a line in ISO format (YYYY-MM-DD HH:MM:SS). + + The EntryGuardAddedBy line is optional. It contains three + space-separated fields: the identity of the entry guard, the version of + Tor that added it, and the ISO time at which it was added. + + TransportProxy + + One or more of these may be present. + + The format is "transportname addr:port", to remember the address + at which a pluggable transport was listening. Tor bridges use + this information to spawn pluggable transport listeners in the + same IP address and TCP port even after tor client restarts. + + BWHistoryReadEnds (ISO time) + BWHistoryReadInterval (integer, number of seconds) + BWHistoryReadValues (comma-separated list of integer) + BWHistoryReadMaxima (comma-separated list of integer) + BWHistoryWriteEnds + BWHistoryWriteInterval + BWHistoryWriteValues + BWHistoryWriteMaxima + BWHistoryDirReadEnds + BWHistoryDirReadInterval + BWHistoryDirReadValues + BWHistoryDirReadMaxima + BWHistoryDirWriteEnds + BWHistoryDirWriteInterval + BWHistoryDirWriteValues + BWHistoryDirWriteMaxima + + These values record bandwidth history. The "Values" fields are a list, for + some number of "Intervals", of the total amount read/written during that + integer. The "Maxima" are the highest burst for each interval. + + Interval duration is set by the "Interval" field, in seconds. The + "Ends" field is the ending time of the last interval in each list. + + The *Read* and *Write* fields are the total amount read and + written; the *DirRead* and *DirWrite* variants are for directory + traffic only. + + LastRotatedOnionKey + + The last time that we changed our onion key for a new one. + Given in ISO format (YYYY-MM-DD HH:MM:SS) + + TotalBuildTimes + CircuitBuildAbandonedCount + CircuitBuildTimeBin + + XXXX writeme. diff --git a/doc/tor-fw-helper.1.txt b/doc/tor-fw-helper.1.txt new file mode 100644 index 0000000000..49b0910380 --- /dev/null +++ b/doc/tor-fw-helper.1.txt @@ -0,0 +1,68 @@ +// Copyright (c) The Tor Project, Inc. +// See LICENSE for licensing information +// This is an asciidoc file used to generate the manpage/html reference. +// Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +tor-fw-helper(1) +================ +Jacob Appelbaum + +NAME +---- +tor-fw-helper - Manage upstream firewall/NAT devices + +SYNOPSIS +-------- +**tor-fw-helper** [-h|--help] [-T|--test] [-v|--verbose] [-g|--fetch-public-ip] + -i|--internal-or-port __TCP port__ [-e|--external-or-port _TCP port_] + [-d|--internal-dir-port _TCP port_] [-p|--external-dir-port _TCP port_] + +DESCRIPTION +----------- +**tor-fw-helper** currently supports Apple's NAT-PMP protocol and the UPnP +standard for TCP port mapping. It is written as the reference implementation of +tor-fw-helper-spec.txt and conforms to that loose plugin API. If your network +supports either NAT-PMP or UPnP, tor-fw-helper will attempt to automatically +map the required TCP ports for Tor's Or and Dir ports. + + +OPTIONS +------- +**-h** or **--help**:: + Display help text and exit. + +**-v**:: + Display verbose output. + +**-T** or **--test**:: + Display test information and print the test information in + tor-fw-helper.log + +**-g** or **--fetch-public-ip**:: + Fetch the the public ip address for each supported NAT helper method. + +**-i** or **--internal-or-port** __port__:: + Inform **tor-fw-helper** of your internal OR port. This is the only + required argument. + +**-e** or **--external-or-port** __port__:: + Inform **tor-fw-helper** of your external OR port. + +**-d** or **--internal-dir-port** __port__:: + Inform **tor-fw-helper** of your internal Dir port. + +**-p** or **--external-dir-port** __port__:: + Inform **tor-fw-helper** of your external Dir port. + +BUGS +---- +This probably doesn't run on Windows. That's not a big issue, since we don't +really want to deal with Windows before October 2010 anyway. + +SEE ALSO +-------- +**tor**(1) + + +See also the "tor-fw-helper-spec.txt" file, distributed with Tor. + +AUTHORS +------- + Jacob Appelbaum <jacob@torproject.org>, Steven J. Murdoch <Steven.Murdoch@cl.cam.ac.uk> diff --git a/doc/tor-rpm-creation.txt b/doc/tor-rpm-creation.txt index 3fc6bde79c..a03891e2b9 100644 --- a/doc/tor-rpm-creation.txt +++ b/doc/tor-rpm-creation.txt @@ -2,18 +2,9 @@ ## The process used to create the official rpms is as follows: -Download latest stable libevent from -http://www.monkey.org/~provos/libevent/ - -The first step of compiling libevent is to configure it as follows: -./configure --enable-static --disable-shared - -Complete the "make" and "make install". You will need to be root, -or sudo -s, to complete the "make install". - -Check for a successful universal binary of libevent.a in, by default, -/usr/local/lib by using the following command: - "file /usr/local/lib/libevent.a" +You'll need to install libevent headers, usually located in package named +libevent-devel. Alternatively, you could download latest libevent from +http://libevent.org/ but that shouldn't be necessary. Download and Extract the latest tor source code from https://www.torproject.org/download @@ -23,4 +14,43 @@ LIBS=-lrt ./configure make dist-rpm You should have at least two, maybe three, rpms. There should be the binary -i386.rpm, a src.rpm, and on redhat/centos machines, a debuginfo.rpm. +(i686|x86_64).rpm, a src.rpm, and on redhat/centos machines, a debuginfo.rpm. +The debuginfo rpms are created if package redhat-rpm-config is installed (case +of redhat distros). + +This step suffices unless you want to create RPMs for distros other than the +one you used for building. + + +## Instructions for building RPMs for multiple architectures or distributions +## using 'mock' on Fedora or RHEL (and clones) + +Make sure you have mock installed and configured, see following HOWTOs for setup: +https://fedoraproject.org/wiki/How_to_create_an_RPM_package +https://fedoraproject.org/wiki/Using_Mock_to_test_package_builds + +Take the source RPM generated by previous step, and execute mock for every +target architecture (the names come from files in /etc/mock, strip the .cfg +extension in the -r parameter): + +mock --rebuild -r fedora-17-x86_64 tor-X.Y.Z.src.rpm + +Building for EL5 from newer distro (e.g. EL6 or Fedora 17) will fail due to bug +(https://bugzilla.redhat.com/show_bug.cgi?id=490613). +Here's a workaround: + +Before even building the source RPM, install fedora-packager and instruct +the build system to use rpmbuild-md5 like this: + +yum install fedora-packager +export RPMBUILD=rpmbuild-md5 + +Then proceed as usual to create the source RPM and binary RPMs: + +LIBS=-lrt ./configure +make dist-rpm +mock --rebuild -r epel-5-x86_64 tor-X.Y.Z.src.rpm + + +(Note: don't build under OpenVZ - it breaks unshare() syscall, which in turn +breaks mock. It could save you several hours.) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 5f10f68712..1f3afef2ac 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -28,13 +28,21 @@ Users bounce their TCP streams -- web traffic, ftp, ssh, etc -- around the routers, and recipients, observers, and even the routers themselves have difficulty tracking the source of the stream. -OPTIONS -------- +COMMAND-LINE OPTIONS +-------------------- **-h**, **-help**:: Display a short help message and exit. **-f** __FILE__:: - FILE contains further "option value" pairs. (Default: @CONFDIR@/torrc) + Specify a new configuration file to contain further Tor configuration + options. (Default: $HOME/.torrc, or @CONFDIR@/torrc if that file is not + found) + +**--defaults-torrc** __FILE__:: + Specify a file in which to find default values for Tor options. The + contents of this file are overridden by those in the regular + configuration file, and by those on the command line. (Default: + @CONFDIR@/torrc-defaults.) **--hash-password**:: Generates a hashed password for control port access. @@ -45,10 +53,16 @@ OPTIONS **--verify-config**:: Verify the configuration file is valid. +**--service install** [**--options** __command-line options__]:: + Install an instance of Tor as a Windows service, with the provided + command-line options. Current instructions can be found at + https://trac.torproject.org/projects/tor/wiki/doc/TorFAQ#HowdoIrunmyTorrelayasanNTservice + +**--service** **remove**|**start**|**stop**:: + Remove, start, or stop a configured Tor Windows service. + **--nt-service**:: - **--service [install|remove|start|stop]** Manage the Tor Windows - NT/2000/XP service. Current instructions can be found at - https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#WinNTService + Used internally to implement a Windows service. **--list-torrc-options**:: List all valid options. @@ -56,26 +70,64 @@ OPTIONS **--version**:: Display Tor version and exit. -**--quiet**:: - Do not start Tor with a console log unless explicitly requested to do so. - (By default, Tor starts out logging messages at level "notice" or higher to - the console, until it has parsed its configuration.) - -Other options can be specified either on the command-line (--option - value), or in the configuration file (option value or option "value"). - Options are case-insensitive. C-style escaped characters are allowed inside - quoted values. Options on the command line take precedence over - options found in the configuration file, except indicated otherwise. To - split one configuration entry into multiple lines, use a single \ before - the end of the line. Comments can be used in such multiline entries, but - they must start at the beginning of a line. +**--quiet**|**--hush**:: + Override the default console log. By default, Tor starts out logging + messages at level "notice" and higher to the console. It stops doing so + after it parses its configuration, if the configuration tells it to log + anywhere else. You can override this behavior with the **--hush** option, + which tells Tor to only send warnings and errors to the console, or with + the **--quiet** option, which tells Tor not to log to the console at all. + +Other options can be specified on the command-line in the format "--option +value", in the format "option value", or in a configuration file. For +instance, you can tell Tor to start listening for SOCKS connections on port +9999 by passing --SOCKSPort 9999 or SOCKSPort 9999 to it on the command line, +or by putting "SOCKSPort 9999" in the configuration file. You will need to +quote options with spaces in them: if you want Tor to log all debugging +messages to debug.log, you will probably need to say --Log 'debug file +debug.log'. + +Options on the command line override those in configuration files. See the +next section for more information. + +THE CONFIGURATION FILE FORMAT +----------------------------- + +All configuration options in a configuration are written on a single line by +default. They take the form of an option name and a value, or an option name +and a quoted value (option value or option "value"). Anything after a # +character is treated as a comment. Options are +case-insensitive. C-style escaped characters are allowed inside quoted +values. To split one configuration entry into multiple lines, use a single +backslash character (\) before the end of the line. Comments can be used in +such multiline entries, but they must start at the beginning of a line. + +By default, an option on the command line overrides an option found in the +configuration file, and an option in a configuration file overrides one in +the defaults file. + +This rule is simple for options that take a single value, but it can become +complicated for options that are allowed to occur more than once: if you +specify four SOCKSPorts in your configuration file, and one more SOCKSPort on +the command line, the option on the command line will replace __all__ of the +SOCKSPorts in the configuration file. If this isn't what you want, prefix +the option name with a plus sign, and it will be appended to the previous set +of options instead. + +Alternatively, you might want to remove every instance of an option in the +configuration file, and not replace it at all: you might want to say on the +command line that you want no SOCKSPorts at all. To do that, prefix the +option name with a forward slash. + +GENERAL OPTIONS +--------------- **BandwidthRate** __N__ **bytes**|**KB**|**MB**|**GB**:: A token bucket limits the average incoming bandwidth usage on this node to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. If you want to run a relay in the - public network, this needs to be _at the very least_ 20 KB (that is, - 20480 bytes). (Default: 5 MB) + public network, this needs to be _at the very least_ 30 KB (that is, + 30720 bytes). (Default: 5 MB) **BandwidthBurst** __N__ **bytes**|**KB**|**MB**|**GB**:: Limit the maximum token bucket size (also known as the burst) to the given @@ -110,6 +162,23 @@ Other options can be specified either on the command-line (--option You should never need to change this value, since a network-wide value is published in the consensus and your relay will use that value. (Default: 0) +**ClientTransportPlugin** __transport__ socks4|socks5 __IP__:__PORT__:: +**ClientTransportPlugin** __transport__ exec __path-to-binary__ [options]:: + In its first form, when set along with a corresponding Bridge line, the Tor + client forwards its traffic to a SOCKS-speaking proxy on "IP:PORT". It's the + duty of that proxy to properly forward the traffic to the bridge. + + + + In its second form, when set along with a corresponding Bridge line, the Tor + client launches the pluggable transport proxy executable in + __path-to-binary__ using __options__ as its command-line options, and + forwards its traffic to it. It's the duty of that proxy to properly forward + the traffic to the bridge. + +**ServerTransportPlugin** __transport__ exec __path-to-binary__ [options]:: + The Tor relay launches the pluggable transport proxy in __path-to-binary__ + using __options__ as its command-line options, and expects to receive + proxied client traffic from it. + **ConnLimit** __NUM__:: The minimum number of file descriptors that must be available to the Tor process before it will start. Tor will ask the OS for as many file @@ -119,6 +188,12 @@ Other options can be specified either on the command-line (--option You probably don't need to adjust this. It has no effect on Windows since that platform lacks getrlimit(). (Default: 1000) +**DisableNetwork** **0**|**1**:: + When this option is set, we don't listen for or accept any connections + other than controller connections, and we don't make any outbound + connections. Controllers sometimes use this option to avoid using + the network until Tor is fully configured. (Default: 0) + **ConstrainedSockets** **0**|**1**:: If set, Tor will tell the kernel to attempt to shrink the buffers for all sockets to the size specified in **ConstrainedSockSize**. This is useful for @@ -138,7 +213,7 @@ Other options can be specified either on the command-line (--option You should **not** enable this feature unless you encounter the "no buffer space available" issue. Reducing the TCP buffers affects window size for the TCP stream and will reduce throughput in proportion to round trip - time on long paths. (Default: 0.) + time on long paths. (Default: 0) **ConstrainedSockSize** __N__ **bytes**|**KB**:: When **ConstrainedSockets** is enabled the receive and transmit buffers for @@ -154,15 +229,15 @@ Other options can be specified either on the command-line (--option host to control it. (Setting both authentication methods means either method is sufficient to authenticate to Tor.) This option is required for many Tor controllers; most use the value of 9051. - Set it to "auto" to have Tor pick a port for you. (Default: 0). + Set it to "auto" to have Tor pick a port for you. (Default: 0) **ControlListenAddress** __IP__[:__PORT__]:: Bind the controller listener to this address. If you specify a port, bind to this port rather than the one specified in ControlPort. We strongly recommend that you leave this alone unless you know what you're doing, since giving attackers access to your control listener is really - dangerous. (Default: 127.0.0.1) This directive can be specified multiple - times to bind to multiple addresses/ports. + dangerous. This directive can be specified multiple + times to bind to multiple addresses/ports. (Default: 127.0.0.1) **ControlSocket** __Path__:: Like ControlPort, but listens on a Unix domain socket, rather than a TCP @@ -195,7 +270,7 @@ Other options can be specified either on the command-line (--option If this option is set to 0, don't allow the filesystem group to read the cookie file. If the option is set to 1, make the cookie file readable by the default GID. [Making the file readable by other groups is not yet - implemented; let us know if you need this for some reason.] (Default: 0). + implemented; let us know if you need this for some reason.] (Default: 0) **ControlPortWriteToFile** __Path__:: If set, Tor writes the address and port of any control port it opens to @@ -205,7 +280,7 @@ Other options can be specified either on the command-line (--option **ControlPortFileGroupReadable** **0**|**1**:: If this option is set to 0, don't allow the filesystem group to read the control port file. If the option is set to 1, make the control port - file readable by the default GID. (Default: 0). + file readable by the default GID. (Default: 0) **DataDirectory** __DIR__:: Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor) @@ -233,16 +308,25 @@ Other options can be specified either on the command-line (--option distinguishable from other users, because you won't believe the same authorities they do. +**DynamicDHGroups** **0**|**1**:: + If this option is set to 1, when running as a server, generate our + own Diffie-Hellman group instead of using the one from Apache's mod_ssl. + This option may help circumvent censorship based on static + Diffie-Hellman parameters. (Default: 0) + **AlternateDirAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__ + **AlternateHSAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__ + **AlternateBridgeAuthority** [__nickname__] [**flags**] __address__:__port__ __ fingerprint__:: - As DirServer, but replaces less of the default directory authorities. Using + These options behave as DirServer, but they replace fewer of the + default directory authorities. Using AlternateDirAuthority replaces the default Tor directory authorities, but - leaves the hidden service authorities and bridge authorities in place. - Similarly, Using AlternateHSAuthority replaces the default hidden service - authorities, but not the directory or bridge authorities. + leaves the default hidden service authorities and bridge authorities in + place. Similarly, AlternateHSAuthority replaces the default hidden + service authorities, but not the directory or bridge authorities; and + AlternateBridgeAuthority replaces the default bridge authority, + but leaves the directory and hidden service authorities alone. **DisableAllSwap** **0**|**1**:: If set to 1, Tor will attempt to lock all current and future memory pages, @@ -252,6 +336,20 @@ Other options can be specified either on the command-line (--option option requires that you start your Tor as root, and you should use the **User** option to properly reduce Tor's privileges. (Default: 0) +**DisableDebuggerAttachment** **0**|**1**:: + If set to 1, Tor will attempt to prevent basic debugging attachment attempts + by other processes. It has no impact for users who wish to attach if they + have CAP_SYS_PTRACE or if they are root. We believe that this feature + works on modern Gnu/Linux distributions, and that it may also work on *BSD + systems (untested). Some modern Gnu/Linux systems such as Ubuntu have the + kernel.yama.ptrace_scope sysctl and by default enable it as an attempt to + limit the PTRACE scope for all user processes by default. This feature will + attempt to limit the PTRACE scope for Tor specifically - it will not attempt + to alter the system wide ptrace scope as it may not even exist. If you wish + to attach to Tor with a debugger such as gdb or strace you will want to set + this to 0 for the duration of your debugging. Normal users should leave it + on. Disabling this option while Tor is running is prohibited. (Default: 1) + **FetchDirInfoEarly** **0**|**1**:: If set to 1, Tor will always fetch directory information like other directory caches, even if you don't meet the normal criteria for fetching @@ -389,6 +487,13 @@ Other options can be specified either on the command-line (--option on Windows; instead you should use the --service command-line option. (Default: 0) +**LogTimeGranularity** __NUM__:: + Set the resolution of timestamps in Tor's logs to NUM milliseconds. + NUM must be positive and either a divisor or a multiple of 1 second. + Note that this option only controls the granularity written by Tor to + a file or console log. Tor does not (for example) "batch up" log + messages to affect times logged by a controller, times attached to + syslog messages, or the mtime fields on log files. (Default: 1 second) **SafeLogging** **0**|**1**|**relay**:: Tor can scrub potentially sensitive strings from log messages (e.g. @@ -441,13 +546,39 @@ Other options can be specified either on the command-line (--option CircuitPriorityHalflife value (in seconds). If this option is not set at all, we use the behavior recommended in the current consensus networkstatus. This is an advanced option; you generally shouldn't have - to mess with it. (Default: not set.) + to mess with it. (Default: not set) + +**DisableIOCP** **0**|**1**:: + If Tor was built to use the Libevent's "bufferevents" networking code + and you're running on Windows, setting this option to 1 will tell Libevent + not to use the Windows IOCP networking API. (Default: 1) + +**UserspaceIOCPBuffers** **0**|**1**:: + If IOCP is enabled (see DisableIOCP above), setting this option to 1 + will tell Tor to disable kernel-space TCP buffers, in order to avoid + needless copy operations and try not to run out of non-paged RAM. + This feature is experimental; don't use it yet unless you're eager to + help tracking down bugs. (Default: 0) + +**_UseFilteringSSLBufferevents** **0**|**1**:: + Tells Tor to do its SSL communication using a chain of + bufferevents: one for SSL and one for networking. This option has no + effect if bufferevents are disabled (in which case it can't turn on), or + if IOCP bufferevents are enabled (in which case it can't turn off). This + option is useful for debugging only; most users shouldn't touch it. + (Default: 0) + +**CountPrivateBandwidth** **0**|**1**:: + If this option is set, then Tor's rate-limiting applies not only to + remote connections, but also to connections to private addresses like + 127.0.0.1 or 10.0.0.1. This is mostly useful for debugging + rate-limiting. (Default: 0) CLIENT OPTIONS -------------- The following options are useful only for clients (that is, if -**SocksPort** is non-zero): +**SocksPort**, **TransPort**, **DNSPort**, or **NATDPort** is non-zero): **AllowInvalidNodes** **entry**|**exit**|**middle**|**introduction**|**rendezvous**|**...**:: If some Tor servers are obviously not working right, the directory @@ -465,13 +596,17 @@ The following options are useful only for clients (that is, if so using these relays might make your client stand out. (Default: 1) -**Bridge** __IP__:__ORPort__ [fingerprint]:: +**Bridge** [__transport__] __IP__:__ORPort__ [__fingerprint__]:: When set along with UseBridges, instructs Tor to use the relay at "IP:ORPort" as a "bridge" relaying into the Tor network. If "fingerprint" is provided (using the same format as for DirServer), we will verify that the relay running at that location has the right fingerprint. We also use fingerprint to look up the bridge descriptor at the bridge authority, if - it's provided and if UpdateBridgesFromAuthority is set too. + it's provided and if UpdateBridgesFromAuthority is set too. + + + + If "transport" is provided, and matches to a ClientTransportPlugin + line, we use that pluggable transports proxy to transfer data to + the bridge. **LearnCircuitBuildTimeout** **0**|**1**:: If 0, CircuitBuildTimeout adaptive learning is disabled. (Default: 1) @@ -482,7 +617,7 @@ The following options are useful only for clients (that is, if open in that time, give up on it. If LearnCircuitBuildTimeout is 1, this value serves as the initial value to use before a timeout is learned. If LearnCircuitBuildTimeout is 0, this value is the only value used. - (Default: 60 seconds.) + (Default: 60 seconds) **CircuitIdleTimeout** __NUM__:: If we have kept a clean (never used) circuit around for NUM seconds, then @@ -490,7 +625,7 @@ The following options are useful only for clients (that is, if of its circuits, and then expire its TLS connections. Also, if we end up making a circuit that is not useful for exiting any of the requests we're receiving, it won't forever take up a slot in the circuit list. (Default: 1 - hour.) + hour) **CircuitStreamTimeout** __NUM__:: If non-zero, this option overrides our internal timeout schedule for how @@ -532,7 +667,6 @@ The following options are useful only for clients (that is, if node listed in ExcludeNodes is automatically considered to be part of this list too. See also the caveats on the "ExitNodes" option below. - **ExitNodes** __node__,__node__,__...__:: A list of identity fingerprints, nicknames, country codes and address patterns of nodes to use as exit node---that is, a @@ -557,9 +691,9 @@ The following options are useful only for clients (that is, if this option. **EntryNodes** __node__,__node__,__...__:: - A list of identity fingerprints and nicknames of nodes - to use for the first hop in your normal circuits. (Country codes and - address patterns are not yet supported.) Normal circuits include all + A list of identity fingerprints, nicknames, and country codes of nodes + to use for the first hop in your normal circuits. + Normal circuits include all circuits except for direct connections to directory servers. The Bridge option overrides this option; if you have configured bridges and UseBridges is 1, the Bridges are used as your entry nodes. + @@ -592,16 +726,6 @@ The following options are useful only for clients (that is, if **FascistFirewall** is set. This option is deprecated; use ReachableAddresses instead. (Default: 80, 443) -**HidServAuth** __onion-address__ __auth-cookie__ [__service-name__]:: - Client authorization for a hidden service. Valid onion addresses contain 16 - characters in a-z2-7 plus ".onion", and valid auth cookies contain 22 - characters in A-Za-z0-9+/. The service name is only used for internal - purposes, e.g., for Tor controllers. This option may be used multiple times - for different hidden services. If a hidden service uses authorization and - this option is not set, the hidden service is not accessible. Hidden - services can be configured to require authorization using the - **HiddenServiceAuthorizeClient** option. - **ReachableAddresses** __ADDR__[/__MASK__][:__PORT__]...:: A comma-separated list of IP addresses and ports that your firewall allows you to connect to. The format is as for the addresses in ExitPolicy, except @@ -631,19 +755,82 @@ The following options are useful only for clients (that is, if and some limit HTTP GET requests (which Tor uses for fetching directory information) to port 80. +**HidServAuth** __onion-address__ __auth-cookie__ [__service-name__]:: + Client authorization for a hidden service. Valid onion addresses contain 16 + characters in a-z2-7 plus ".onion", and valid auth cookies contain 22 + characters in A-Za-z0-9+/. The service name is only used for internal + purposes, e.g., for Tor controllers. This option may be used multiple times + for different hidden services. If a hidden service uses authorization and + this option is not set, the hidden service is not accessible. Hidden + services can be configured to require authorization using the + **HiddenServiceAuthorizeClient** option. + +**CloseHSClientCircuitsImmediatelyOnTimeout** **0**|**1**:: + If 1, Tor will close unfinished hidden service client circuits + which have not moved closer to connecting to their destination + hidden service when their internal state has not changed for the + duration of the current circuit-build timeout. Otherwise, such + circuits will be left open, in the hope that they will finish + connecting to their destination hidden services. In either case, + another set of introduction and rendezvous circuits for the same + destination hidden service will be launched. (Default: 0) + +**CloseHSServiceRendCircuitsImmediatelyOnTimeout** **0**|**1**:: + If 1, Tor will close unfinished hidden-service-side rendezvous + circuits after the current circuit-build timeout. Otherwise, such + circuits will be left open, in the hope that they will finish + connecting to their destinations. In either case, another + rendezvous circuit for the same destination client will be + launched. (Default: 0) + **LongLivedPorts** __PORTS__:: A list of ports for services that tend to have long-running connections (e.g. chat and interactive shells). Circuits for streams that use these ports will contain only high-uptime nodes, to reduce the chance that a node - will go down before the stream is finished. (Default: 21, 22, 706, 1863, - 5050, 5190, 5222, 5223, 6667, 6697, 8300) + will go down before the stream is finished. Note that the list is also + honored for circuits (both client and service side) involving hidden + services whose virtual port is in this list. (Default: 21, 22, 706, + 1863, 5050, 5190, 5222, 5223, 6523, 6667, 6697, 8300) **MapAddress** __address__ __newaddress__:: - When a request for address arrives to Tor, it will rewrite it to newaddress + When a request for address arrives to Tor, it will transform to newaddress before processing it. For example, if you always want connections to - www.indymedia.org to exit via __torserver__ (where __torserver__ is the - nickname of the server), use "MapAddress www.indymedia.org - www.indymedia.org.torserver.exit". + www.example.com to exit via __torserver__ (where __torserver__ is the + nickname of the server), use "MapAddress www.example.com + www.example.com.torserver.exit". If the value is prefixed with a + "\*.", matches an entire domain. For example, if you + always want connections to example.com and any if its subdomains + to exit via + __torserver__ (where __torserver__ is the nickname of the server), use + "MapAddress \*.example.com \*.example.com.torserver.exit". (Note the + leading "*." in each part of the directive.) You can also redirect all + subdomains of a domain to a single address. For example, "MapAddress + *.example.com www.example.com". + + + + NOTES: + + 1. When evaluating MapAddress expressions Tor stops when it hits the most + recently added expression that matches the requested address. So if you + have the following in your torrc, www.torproject.org will map to 1.1.1.1: + + MapAddress www.torproject.org 2.2.2.2 + MapAddress www.torproject.org 1.1.1.1 + + 2. Tor evaluates the MapAddress configuration until it finds no matches. So + if you have the following in your torrc, www.torproject.org will map to + 2.2.2.2: + + MapAddress 1.1.1.1 2.2.2.2 + MapAddress www.torproject.org 1.1.1.1 + + 3. The following MapAddress expression is invalid (and will be + ignored) because you cannot map from a specific address to a wildcard + address: + + MapAddress www.torproject.org *.torproject.org.torserver.exit + + 4. Using a wildcard to match only part of a string (as in *ample.com) is + also invalid. **NewCircuitPeriod** __NUM__:: Every NUM seconds consider whether to build a new circuit. (Default: 30 @@ -655,29 +842,68 @@ The following options are useful only for clients (that is, if services, this applies to the __last__ time a circuit was used, not the first. (Default: 10 minutes) +**MaxClientCircuitsPending** __NUM__:: + Do not allow more than NUM circuits to be pending at a time for handling + client streams. A circuit is pending if we have begun constructing it, + but it has not yet been completely constructed. (Default: 32) + **NodeFamily** __node__,__node__,__...__:: The Tor servers, defined by their identity fingerprints or nicknames, constitute a "family" of similar or co-administered servers, so never use any two of them in the same circuit. Defining a NodeFamily is only needed when a server doesn't list the family itself (with MyFamily). This option - can be used multiple times. + can be used multiple times. In addition to nodes, you can also list + IP address and ranges and country codes in {curly braces}. **EnforceDistinctSubnets** **0**|**1**:: If 1, Tor will not put two servers whose IP addresses are "too close" on the same circuit. Currently, two addresses are "too close" if they lie in the same /16 range. (Default: 1) -**SocksPort** __PORT__|**auto**:: - Advertise this port to listen for connections from Socks-speaking +**SOCKSPort** \['address':]__port__|**auto** [_isolation flags_]:: + Open this port to listen for connections from SOCKS-speaking applications. Set this to 0 if you don't want to allow application connections via SOCKS. Set it to "auto" to have Tor pick a port for - you. (Default: 9050) - -**SocksListenAddress** __IP__[:__PORT__]:: + you. This directive can be specified multiple times to bind + to multiple addresses/ports. (Default: 9050) + + + + The _isolation flags_ arguments give Tor rules for which streams + received on this SOCKSPort are allowed to share circuits with one + another. Recognized isolation flags are: + **IsolateClientAddr**;; + Don't share circuits with streams from a different + client address. (On by default and strongly recommended; + you can disable it with **NoIsolateClientAddr**.) + **IsolateSOCKSAuth**;; + Don't share circuits with streams for which different + SOCKS authentication was provided. (On by default; + you can disable it with **NoIsolateSOCKSAuth**.) + **IsolateClientProtocol**;; + Don't share circuits with streams using a different protocol. + (SOCKS 4, SOCKS 5, TransPort connections, NATDPort connections, + and DNSPort requests are all considered to be different protocols.) + **IsolateDestPort**;; + Don't share circuits with streams targetting a different + destination port. + **IsolateDestAddr**;; + Don't share circuits with streams targetting a different + destination address. + **SessionGroup=**__INT__;; + If no other isolation rules would prevent it, allow streams + on this port to share circuits with streams from every other + port with the same session group. (By default, streams received + on different SOCKSPorts, TransPorts, etc are always isolated from one + another. This option overrides that behavior.) + +**SOCKSListenAddress** __IP__[:__PORT__]:: Bind to this address to listen for connections from Socks-speaking applications. (Default: 127.0.0.1) You can also specify a port (e.g. 192.168.0.1:9100). This directive can be specified multiple times to bind - to multiple addresses/ports. + to multiple addresses/ports. (DEPRECATED: As of 0.2.3.x-alpha, you can + now use multiple SOCKSPort entries, and provide addresses for SOCKSPort + entries, so SOCKSListenAddress no longer has a purpose. For backward + compatibility, SOCKSListenAddress is only allowed when SOCKSPort is just + a port number.) **SocksPolicy** __policy__,__policy__,__...__:: Set an entrance policy for this server, to limit who can connect to the @@ -687,7 +913,14 @@ The following options are useful only for clients (that is, if **SocksTimeout** __NUM__:: Let a socks connection wait NUM seconds handshaking, and NUM seconds unattached waiting for an appropriate circuit, before we fail it. (Default: - 2 minutes.) + 2 minutes) + +**TokenBucketRefillInterval** __NUM__ [**msec**|**second**]:: + Set the refill interval of Tor's token bucket to NUM milliseconds. + NUM must be between 1 and 1000, inclusive. Note that the configured + bandwidth limits are still expressed in bytes per second: this + option only affects the frequency with which Tor checks to see whether + previously exhausted connections may read again. (Default: 100 msec) **TrackHostExits** __host__,__.domain__,__...__:: For each value in the comma separated list, Tor will track recent @@ -720,18 +953,18 @@ The following options are useful only for clients (that is, if If this option is set to 1, we pick a few long-term entry servers, and try to stick with them. This is desirable because constantly changing servers increases the odds that an adversary who owns some servers will observe a - fraction of your paths. (Defaults to 1.) + fraction of your paths. (Default: 1) **NumEntryGuards** __NUM__:: If UseEntryGuards is set to 1, we will try to pick a total of NUM routers - as long-term entries for our circuits. (Defaults to 3.) + as long-term entries for our circuits. (Default: 3) **SafeSocks** **0**|**1**:: When this option is enabled, Tor will reject application connections that use unsafe variants of the socks protocol -- ones that only provide an IP address, meaning the application is doing a DNS resolve first. Specifically, these are socks4 and socks5 when not doing remote DNS. - (Defaults to 0.) + (Default: 0) **TestSocks** **0**|**1**:: When this option is enabled, Tor will make a notice-level log entry for @@ -780,87 +1013,163 @@ The following options are useful only for clients (that is, if operating as a relay, and it will never use the public key step if it doesn't yet know the onion key of the first hop. (Default: 1) -**TransPort** __PORT__|**auto**:: - If non-zero, enables transparent proxy support on __PORT__ (by convention, - 9040). Requires OS support for transparent proxies, such as BSDs' pf or +**TransPort** \['address':]__port__|**auto** [_isolation flags_]:: + Open this port to listen for transparent proxy connections. Set this to + 0 if you don't want to allow transparent proxy connections. Set the port + to "auto" to have Tor pick a port for you. This directive can be + specified multiple times to bind to multiple addresses/ports. See + SOCKSPort for an explanation of isolation flags. + + + + TransPort requires OS support for transparent proxies, such as BSDs' pf or Linux's IPTables. If you're planning to use Tor as a transparent proxy for a network, you'll want to examine and change VirtualAddrNetwork from the default setting. You'll also want to set the TransListenAddress option for - the network you'd like to proxy. Set it to "auto" to have Tor pick a - port for you. (Default: 0). + the network you'd like to proxy. (Default: 0) **TransListenAddress** __IP__[:__PORT__]:: Bind to this address to listen for transparent proxy connections. (Default: 127.0.0.1). This is useful for exporting a transparent proxy server to an - entire network. - -**NATDPort** __PORT__|**auto**:: - Allow old versions of ipfw (as included in old versions of FreeBSD, etc.) - to send connections through Tor using the NATD protocol. This option is - only for people who cannot use TransPort. Set it to "auto" to have Tor - pick a port for you. (Default: 0) + entire network. (DEPRECATED: As of 0.2.3.x-alpha, you can + now use multiple TransPort entries, and provide addresses for TransPort + entries, so TransListenAddress no longer has a purpose. For backward + compatibility, TransListenAddress is only allowed when TransPort is just + a port number.) + +**NATDPort** \['address':]__port__|**auto** [_isolation flags_]:: + Open this port to listen for connections from old versions of ipfw (as + included in old versions of FreeBSD, etc) using the NATD protocol. + Use 0 if you don't want to allow NATD connections. Set the port + to "auto" to have Tor pick a port for you. This directive can be + specified multiple times to bind to multiple addresses/ports. See + SOCKSPort for an explanation of isolation flags. + + + + This option is only for people who cannot use TransPort. (Default: 0) **NATDListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for NATD connections. (Default: 127.0.0.1). + Bind to this address to listen for NATD connections. (DEPRECATED: As of + 0.2.3.x-alpha, you can now use multiple NATDPort entries, and provide + addresses for NATDPort entries, so NATDListenAddress no longer has a + purpose. For backward compatibility, NATDListenAddress is only allowed + when NATDPort is just a port number.) **AutomapHostsOnResolve** **0**|**1**:: When this option is enabled, and we get a request to resolve an address that ends with one of the suffixes in **AutomapHostsSuffixes**, we map an - unused virtual address to that address, and return the new virtual address. + unused virtual address to that address, and return the new virtual address. This is handy for making ".onion" addresses work with applications that - resolve an address and then connect to it. (Default: 0). + resolve an address and then connect to it. (Default: 0) **AutomapHostsSuffixes** __SUFFIX__,__SUFFIX__,__...__:: A comma-separated list of suffixes to use with **AutomapHostsOnResolve**. The "." suffix is equivalent to "all addresses." (Default: .exit,.onion). -**DNSPort** __PORT__|**auto**:: - If non-zero, Tor listens for UDP DNS requests on this port and resolves - them anonymously. Set it to "auto" to have Tor pick a port for - you. (Default: 0). +**DNSPort** \['address':]__port__|**auto** [_isolation flags_]:: + If non-zero, open this port to listen for UDP DNS requests, and resolve + them anonymously. Set the port to "auto" to have Tor pick a port for + you. This directive can be specified multiple times to bind to multiple + addresses/ports. See SOCKSPort for an explanation of isolation + flags. (Default: 0) **DNSListenAddress** __IP__[:__PORT__]:: - Bind to this address to listen for DNS connections. (Default: 127.0.0.1). + Bind to this address to listen for DNS connections. (DEPRECATED: As of + 0.2.3.x-alpha, you can now use multiple DNSPort entries, and provide + addresses for DNSPort entries, so DNSListenAddress no longer has a + purpose. For backward compatibility, DNSListenAddress is only allowed + when DNSPort is just a port number.) **ClientDNSRejectInternalAddresses** **0**|**1**:: If true, Tor does not believe any anonymously retrieved DNS answer that tells it that an address resolves to an internal address (like 127.0.0.1 or 192.168.0.1). This option prevents certain browser-based attacks; don't - turn it off unless you know what you're doing. (Default: 1). + turn it off unless you know what you're doing. (Default: 1) **ClientRejectInternalAddresses** **0**|**1**:: If true, Tor does not try to fulfill requests to connect to an internal address (like 127.0.0.1 or 192.168.0.1) __unless a exit node is specifically requested__ (for example, via a .exit hostname, or a - controller request). (Default: 1). + controller request). (Default: 1) **DownloadExtraInfo** **0**|**1**:: If true, Tor downloads and caches "extra-info" documents. These documents contain information about servers other than the information in their regular router descriptors. Tor does not use this information for anything - itself; to save bandwidth, leave this option turned off. (Default: 0). + itself; to save bandwidth, leave this option turned off. (Default: 0) **FallbackNetworkstatusFile** __FILENAME__:: If Tor doesn't have a cached networkstatus file, it starts out using this one instead. Even if this file is out of date, Tor can still use it to learn about directory mirrors, so it doesn't need to put load on the - authorities. (Default: None). + authorities. (Default: None) **WarnPlaintextPorts** __port__,__port__,__...__:: Tells Tor to issue a warnings whenever the user tries to make an anonymous connection to one of these ports. This option is designed to alert users to services that risk sending passwords in the clear. (Default: - 23,109,110,143). + 23,109,110,143) **RejectPlaintextPorts** __port__,__port__,__...__:: Like WarnPlaintextPorts, but instead of warning about risky port uses, Tor - will instead refuse to make the connection. (Default: None). + will instead refuse to make the connection. (Default: None) **AllowSingleHopCircuits** **0**|**1**:: When this option is set, the attached Tor controller can use relays that have the **AllowSingleHopExits** option turned on to build one-hop Tor connections. (Default: 0) +**OptimisticData** **0**|**1**|**auto**:: + When this option is set, and Tor is using an exit node that supports + the feature, it will try optimistically to send data to the exit node + without waiting for the exit node to report whether the connection + succeeded. This can save a round-trip time for protocols like HTTP + where the client talks first. If OptimisticData is set to **auto**, + Tor will look at the UseOptimisticData parameter in the networkstatus. + (Default: auto) + +**Tor2webMode** **0**|**1**:: + When this option is set, Tor connects to hidden services + **non-anonymously**. This option also disables client connections to + non-hidden-service hostnames through Tor. It **must only** be used when + running a tor2web Hidden Service web proxy. + To enable this option the compile time flag --enable-tor2webmode must be + specified. (Default: 0) + +**UseMicrodescriptors** **0**|**1**|**auto**:: + Microdescriptors are a smaller version of the information that Tor needs + in order to build its circuits. Using microdescriptors makes Tor clients + download less directory information, thus saving bandwidth. Directory + caches need to fetch regular descriptors and microdescriptors, so this + option doesn't save any bandwidth for them. If this option is set to + "auto" (recommended) then it is on for all clients that do not set + FetchUselessDescriptors. (Default: auto) + +**PathBiasCircThreshold** __NUM__ + + +**PathBiasNoticeRate** __NUM__ + + +**PathBiasDisableRate** __NUM__ + + +**PathBiasScaleThreshold** __NUM__ + + +**PathBiasScaleFactor** __NUM__:: + These options override the default behavior of Tor's (**currently + experimental**) path bias detection algorithm. To try to find broken or + misbehaving guard nodes, Tor looks for nodes where more than a certain + fraction of circuits through that node fail after the first hop. The + PathBiasCircThreshold option controls how many circuits we need to build + through a guard before we make these checks. The PathBiasNoticeRate and + PathBiasDisableRate options control what fraction of circuits must + succeed through a guard so we won't warn about it or disable it, + respectively. When we have seen more than PathBiasScaleThreshold + circuits through a guard, we divide our observations by + PathBiasScaleFactor, so that new observations don't get swamped by old + ones. + + + + By default, or if a negative value is provided for one of these options, + Tor uses reasonable defaults from the networkstatus consensus document. + If no defaults are available there, these options default to 20, .70, + 0.0, 200, and 4 respectively. + + SERVER OPTIONS -------------- @@ -960,25 +1269,65 @@ is non-zero): their identity fingerprints or nicknames. When two servers both declare that they are in the same \'family', Tor clients will not use them in the same circuit. (Each server only needs to list the other servers in its - family; it doesn't need to list itself, but it won't hurt.) + family; it doesn't need to list itself, but it won't hurt.) Do not list + any bridge relay as it would compromise its concealment. **Nickname** __name__:: Set the server's nickname to \'name'. Nicknames must be between 1 and 19 characters inclusive, and must contain only the characters [a-zA-Z0-9]. **NumCPUs** __num__:: - How many processes to use at once for decrypting onionskins. (Default: 1) + How many processes to use at once for decrypting onionskins and other + parallelizable operations. If this is set to 0, Tor will try to detect + how many CPUs you have, defaulting to 1 if it can't tell. (Default: 0) -**ORPort** __PORT__|**auto**:: +**ORPort** \['address':]__PORT__|**auto** [_flags_]:: Advertise this port to listen for connections from Tor clients and servers. This option is required to be a Tor server. - Set it to "auto" to have Tor pick a port for you. (Default: 0). + Set it to "auto" to have Tor pick a port for you. Set it to 0 to not + run an ORPort at all. This option can occur more than once. (Default: 0) ++ + Tor recognizes these flags on each ORPort: + **NoAdvertise**:: + By default, we bind to a port and tell our users about it. If + NoAdvertise is specified, we don't advertise, but listen anyway. This + can be useful if the port everybody will be connecting to (for + example, one that's opened on our firewall) is somewhere else. + **NoListen**:: + By default, we bind to a port and tell our users about it. If + NoListen is specified, we don't bind, but advertise anyway. This + can be useful if something else (for example, a firewall's port + forwarding configuration) is causing connections to reach us. + **IPv4Only**:: + If the address is absent, or resolves to both an IPv4 and an IPv6 + address, only listen to the IPv4 address. + **IPv6Only**:: + If the address is absent, or resolves to both an IPv4 and an IPv6 + address, only listen to the IPv6 address. ++ + For obvious reasons, NoAdvertise and NoListen are mutually exclusive, and + IPv4Only and IPv6Only are mutually exclusive. **ORListenAddress** __IP__[:__PORT__]:: Bind to this IP address to listen for connections from Tor clients and servers. If you specify a port, bind to this port rather than the one specified in ORPort. (Default: 0.0.0.0) This directive can be specified multiple times to bind to multiple addresses/ports. ++ + This option is deprecated; you can get the same behavior with ORPort now + that it supports NoAdvertise and explicit addresses. + +**PortForwarding** **0**|**1**:: + Attempt to automatically forward the DirPort and ORPort on a NAT router + connecting this Tor server to the Internet. If set, Tor will try both + NAT-PMP (common on Apple routers) and UPnP (common on routers from other + manufacturers). (Default: 0) + +**PortForwardingHelper** __filename__|__pathname__:: + If PortForwarding is set, use this executable to configure the forwarding. + If set to a filename, the system path will be searched for the executable. + If set to a path, only the specified path will be executed. + (Default: tor-fw-helper) **PublishServerDescriptor** **0**|**1**|**v1**|**v2**|**v3**|**bridge**,**...**:: This option specifies which descriptors Tor will publish when acting as @@ -999,6 +1348,11 @@ is non-zero): seconds, we exit. If we get a second SIGINT, we exit immedi- ately. (Default: 30 seconds) +**HeartbeatPeriod** __N__ **minutes**|**hours**|**days**|**weeks**:: + Log a heartbeat message every **HeartbeatPeriod** seconds. This is + a log level __info__ message, designed to let you know your Tor + server is still alive and doing useful things. Settings this + to 0 will disable the heartbeat. (Default: 6 hours) **AccountingMax** __N__ **bytes**|**KB**|**MB**|**GB**|**TB**:: Never send more than the specified number of bytes in a given accounting @@ -1023,14 +1377,14 @@ is non-zero): of the __dayth__ day of one week to the same day and time of the next week, with Monday as day 1 and Sunday as day 7. If **day** is given, each accounting period runs from the time __HH:MM__ each day to the same time on - the next day. All times are local, and given in 24-hour time. (Defaults to - "month 1 0:00".) + the next day. All times are local, and given in 24-hour time. (Default: + "month 1 0:00") **RefuseUnknownExits** **0**|**1**|**auto**:: Prevent nodes that don't appear in the consensus from exiting using this relay. If the option is 1, we always block exit attempts from such nodes; if it's 0, we never do, and if the option is "auto", then we do - whatever the authorities suggest in the consensus. (Defaults to auto.) + whatever the authorities suggest in the consensus. (Default: auto) **ServerDNSResolvConfFile** __filename__:: Overrides the default DNS configuration with the configuration in @@ -1043,28 +1397,28 @@ is non-zero): If this option is false, Tor exits immediately if there are problems parsing the system DNS configuration or connecting to nameservers. Otherwise, Tor continues to periodically retry the system nameservers until - it eventually succeeds. (Defaults to "1".) + it eventually succeeds. (Default: 1) **ServerDNSSearchDomains** **0**|**1**:: If set to 1, then we will search for addresses in the local search domain. For example, if this system is configured to believe it is in "example.com", and a client tries to connect to "www", the client will be connected to "www.example.com". This option only affects name lookups that - your server does on behalf of clients. (Defaults to "0".) + your server does on behalf of clients. (Default: 0) **ServerDNSDetectHijacking** **0**|**1**:: When this option is set to 1, we will test periodically to determine whether our local nameservers have been configured to hijack failing DNS requests (usually to an advertising site). If they are, we will attempt to correct this. This option only affects name lookups that your server does - on behalf of clients. (Defaults to "1".) + on behalf of clients. (Default: 1) **ServerDNSTestAddresses** __address__,__address__,__...__:: When we're detecting DNS hijacking, make sure that these __valid__ addresses aren't getting redirected. If they are, then our DNS is completely useless, and we'll reset our exit policy to "reject *:*". This option only affects - name lookups that your server does on behalf of clients. (Defaults to - "www.google.com, www.mit.edu, www.yahoo.com, www.slashdot.org".) + name lookups that your server does on behalf of clients. (Default: + "www.google.com, www.mit.edu, www.yahoo.com, www.slashdot.org") **ServerDNSAllowNonRFC953Hostnames** **0**|**1**:: When this option is disabled, Tor does not try to resolve hostnames @@ -1095,9 +1449,9 @@ is non-zero): cells spend in circuit queues to disk every 24 hours. (Default: 0) **DirReqStatistics** **0**|**1**:: - When this option is enabled, Tor writes statistics on the number and - response time of network status requests to disk every 24 hours. - (Default: 0) + When this option is enabled, a Tor directory writes statistics on the + number and response time of network status requests to disk every 24 + hours. (Default: 1) **EntryStatistics** **0**|**1**:: When this option is enabled, Tor writes statistics on the number of @@ -1107,10 +1461,28 @@ is non-zero): When this option is enabled, Tor writes statistics on the number of relayed bytes and opened stream per exit port to disk every 24 hours. (Default: 0) +**ConnDirectionStatistics** **0**|**1**:: + When this option is enabled, Tor writes statistics on the bidirectional use + of connections to disk every 24 hours. (Default: 0) + **ExtraInfoStatistics** **0**|**1**:: When this option is enabled, Tor includes previously gathered statistics in its extra-info documents that it uploads to the directory authorities. - (Default: 0) + (Default: 1) + +**ExtendAllowPrivateAddresses** **0**|**1**:: + When this option is enabled, Tor routers allow EXTEND request to + localhost, RFC1918 addresses, and so on. This can create security issues; + you should probably leave it off. (Default: 0) + +**MaxMemInCellQueues** __N__ **bytes**|**KB**|**MB**|**GB**:: + This option configures a threshold above which Tor will assume that it + needs to stop queueing cells because it's about to run out of memory. + If it hits this threshold, it will begin killing circuits until it + has recovered at least 10% of this memory. Do not set this option too + low, or your relay may be unreliable under load. This option only + effects circuit queues, so the actual process size will be larger than + this. (Default: 8GB) DIRECTORY SERVER OPTIONS ------------------------ @@ -1184,15 +1556,21 @@ if DirPort is non-zero): Minimum uptime of a v2 hidden service directory to be accepted as such by authoritative directories. (Default: 25 hours) -**DirPort** __PORT__|**auto**:: +**DirPort** \['address':]__PORT__|**auto** [_flags_]:: If this option is nonzero, advertise the directory service on this port. - Set it to "auto" to have Tor pick a port for you. (Default: 0) + Set it to "auto" to have Tor pick a port for you. This option can occur + more than once. (Default: 0) ++ + The same flags are supported here as are supported by ORPort. **DirListenAddress** __IP__[:__PORT__]:: Bind the directory service to this address. If you specify a port, bind to this port rather than the one specified in DirPort. (Default: 0.0.0.0) This directive can be specified multiple times to bind to multiple addresses/ports. ++ + This option is deprecated; you can get the same behavior with DirPort now + that it supports NoAdvertise and explicit addresses. **DirPolicy** __policy__,__policy__,__...__:: Set an entrance policy for this server, to limit who can connect to the @@ -1201,7 +1579,7 @@ if DirPort is non-zero): **FetchV2Networkstatus** **0**|**1**:: If set, we try to fetch the (obsolete, unused) version 2 network status consensus documents from the directory authorities. No currently - supported Tor version uses them. (Default: 0.) + supported Tor version uses them. (Default: 0) DIRECTORY AUTHORITY SERVER OPTIONS @@ -1235,7 +1613,7 @@ DIRECTORY AUTHORITY SERVER OPTIONS **DirAllowPrivateAddresses** **0**|**1**:: If set to 1, Tor will accept router descriptors with arbitrary "Address" elements. Otherwise, if the address is not an IP address or is a private IP - address, it will reject the router descriptor. Defaults to 0. + address, it will reject the router descriptor. (Default: 0) **AuthDirBadDir** __AddressPattern...__:: Authoritative directories only. A set of address patterns for servers that @@ -1258,6 +1636,18 @@ DIRECTORY AUTHORITY SERVER OPTIONS authority publishes, or accepted as an OR address in any descriptor submitted for publication by this authority. +**AuthDirBadDirCCs** __CC__,... + + +**AuthDirBadExitCCs** __CC__,... + + +**AuthDirInvalidCCs** __CC__,... + + +**AuthDirRejectCCs** __CC__,...:: + Authoritative directories only. These options contain a comma-separated + list of country codes such that any server in one of those country codes + will be marked as a bad directory/bad exit/invalid for use, or rejected + entirely. + **AuthDirListBadDirs** **0**|**1**:: Authoritative directories only. If set to 1, this directory has some opinion about which nodes are unsuitable as directory caches. (Do not set @@ -1289,7 +1679,7 @@ DIRECTORY AUTHORITY SERVER OPTIONS **AuthDirFastGuarantee** __N__ **bytes**|**KB**|**MB**|**GB**:: Authoritative directories only. If non-zero, always vote the Fast flag for any relay advertising this amount of capacity or - more. (Default: 20 KB) + more. (Default: 100 KB) **AuthDirGuardBWGuarantee** __N__ **bytes**|**KB**|**MB**|**GB**:: Authoritative directories only. If non-zero, this advertised capacity @@ -1302,7 +1692,7 @@ DIRECTORY AUTHORITY SERVER OPTIONS implemented) "bridge community" design, where a community of bridge relay operators all use an alternate bridge directory authority, and their target user audience can periodically fetch the list of - available community bridges to stay up-to-date. (Default: not set.) + available community bridges to stay up-to-date. (Default: not set) **V3AuthVotingInterval** __N__ **minutes**|**hours**:: V3 authoritative directories only. Configures the server's preferred voting @@ -1314,14 +1704,14 @@ DIRECTORY AUTHORITY SERVER OPTIONS V3 authoritative directories only. Configures the server's preferred delay between publishing its vote and assuming it has all the votes from all the other authorities. Note that the actual time used is not the server's - preferred time, but the consensus of all preferences. (Default: 5 minutes.) + preferred time, but the consensus of all preferences. (Default: 5 minutes) **V3AuthDistDelay** __N__ **minutes**|**hours**:: V3 authoritative directories only. Configures the server's preferred delay between publishing its consensus and signature and assuming it has all the signatures from all the other authorities. Note that the actual time used is not the server's preferred time, but the consensus of all preferences. - (Default: 5 minutes.) + (Default: 5 minutes) **V3AuthNIntervalsValid** __NUM__:: V3 authoritative directories only. Configures the number of VotingIntervals @@ -1329,18 +1719,18 @@ DIRECTORY AUTHORITY SERVER OPTIONS increases network partitioning risks; choosing low numbers increases directory traffic. Note that the actual number of intervals used is not the server's preferred number, but the consensus of all preferences. Must be at - least 2. (Default: 3.) + least 2. (Default: 3) **V3BandwidthsFile** __FILENAME__:: V3 authoritative directories only. Configures the location of the - bandiwdth-authority generated file storing information on relays' measured - bandwidth capacities. (Default: unset.) + bandwidth-authority generated file storing information on relays' measured + bandwidth capacities. (Default: unset) **V3AuthUseLegacyKey** **0**|**1**:: If set, the directory authority will sign consensuses not only with its own signing key, but also with a "legacy" key and certificate with a different identity. This feature is used to migrate directory authority - keys in the event of a compromise. (Default: 0.) + keys in the event of a compromise. (Default: 0) **RephistTrackTime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: Tells an authority, or other node tracking node reliability and history, @@ -1352,10 +1742,6 @@ DIRECTORY AUTHORITY SERVER OPTIONS votes on whether to accept relays as hidden service directories. (Default: 1) -GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays **0**|**1**:: - When this option is set to 0, do not vote to give the Guard flag to any - version of Tor vulnerable to CVE-2011-2769. (Default: 0) - HIDDEN SERVICE OPTIONS ---------------------- @@ -1370,10 +1756,11 @@ The following options are used to configure a hidden service. Configure a virtual port VIRTPORT for a hidden service. You may use this option multiple times; each time applies to the service using the most recent hiddenservicedir. By default, this option maps the virtual port to - the same port on 127.0.0.1. You may override the target port, address, or - both by specifying a target of addr, port, or addr:port. You may also have - multiple lines with the same VIRTPORT: when a user connects to that - VIRTPORT, one of the TARGETs from those lines will be chosen at random. + the same port on 127.0.0.1 over TCP. You may override the target port, + address, or both by specifying a target of addr, port, or addr:port. + You may also have multiple lines with the same VIRTPORT: when a user + connects to that VIRTPORT, one of the TARGETs from those lines will be + chosen at random. **PublishHidServDescriptors** **0**|**1**:: If set to 0, Tor will run any hidden services you configure, but it won't @@ -1398,7 +1785,7 @@ The following options are used to configure a hidden service. their configuration file using **HidServAuth**. **RendPostPeriod** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: - Every time the specified period elapses, Tor uploads any rendezvous + Every time the specified period elapses, Tor uploads any rendezvous service descriptors to the directory servers. This information is also uploaded whenever it changes. (Default: 1 hour) @@ -1421,7 +1808,9 @@ The following options are used for running a testing Tor network. AuthDirMaxServersPerAuthAddr 0 ClientDNSRejectInternalAddresses 0 ClientRejectInternalAddresses 0 + CountPrivateBandwidth 1 ExitPolicyRejectPrivate 0 + ExtendAllowPrivateAddresses 1 V3AuthVotingInterval 5 minutes V3AuthVoteDelay 20 seconds V3AuthDistDelay 20 seconds @@ -1503,7 +1892,10 @@ FILES __DataDirectory__**/cached-status/**:: The most recently downloaded network status document for each authority. Each file holds one such document; the filenames are the hexadecimal - identity key fingerprints of the directory authorities. + identity key fingerprints of the directory authorities. Mostly obsolete. + +__DataDirectory__**/cached-consensus** and/or **cached-microdesc-consensus**:: + The most recent consensus network status document we've downloaded. __DataDirectory__**/cached-descriptors** and **cached-descriptors.new**:: These files hold downloaded router statuses. Some routers may appear more @@ -1512,6 +1904,12 @@ __DataDirectory__**/cached-descriptors** and **cached-descriptors.new**:: a given router. The ".new" file is an append-only journal; when it gets too large, all entries are merged into a new cached-descriptors file. +__DataDirectory__**/cached-microdescs** and **cached-microdescs.new**:: + These files hold downloaded microdescriptors. Lines beginning with + @-signs are annotations that contain more information about a given + router. The ".new" file is an append-only journal; when it gets too + large, all entries are merged into a new cached-microdescs file. + __DataDirectory__**/cached-routers** and **cached-routers.new**:: Obsolete versions of cached-descriptors and cached-descriptors.new. When Tor can't find the newer files, it looks here instead. diff --git a/doc/torify.1.txt b/doc/torify.1.txt index ca2c385c94..4a4be1250a 100644 --- a/doc/torify.1.txt +++ b/doc/torify.1.txt @@ -21,7 +21,7 @@ DESCRIPTION wrapper available on a system. It calls torsocks or tsocks with a tor specific configuration file. + -torsocks is an improved wrapper that explictly rejects UDP, safely resolves DNS +torsocks is an improved wrapper that explicitly rejects UDP, safely resolves DNS lookups and properly socksifies your TCP connections. + tsocks itself is a wrapper between the tsocks library and the application that diff --git a/src/common/Makefile.am b/src/common/Makefile.am index 6da1782c8e..5e7684259a 100644 --- a/src/common/Makefile.am +++ b/src/common/Makefile.am @@ -1,7 +1,7 @@ noinst_LIBRARIES = libor.a libor-crypto.a libor-event.a -EXTRA_DIST = common_sha1.i sha256.c +EXTRA_DIST = common_sha1.i sha256.c Makefile.nmake #CFLAGS = -Wall -Wpointer-arith -O2 @@ -11,18 +11,53 @@ else libor_extra_source= endif -libor_a_SOURCES = address.c log.c util.c compat.c container.c mempool.c \ - memarea.c di_ops.c procmon.c util_codedigest.c $(libor_extra_source) -libor_crypto_a_SOURCES = crypto.c aes.c tortls.c torgzip.c +libor_a_SOURCES = \ + address.c \ + compat.c \ + container.c \ + di_ops.c \ + log.c \ + memarea.c \ + mempool.c \ + procmon.c \ + util.c \ + util_codedigest.c \ + $(libor_extra_source) + +libor_crypto_a_SOURCES = \ + aes.c \ + crypto.c \ + torgzip.c \ + tortls.c + libor_event_a_SOURCES = compat_libevent.c -noinst_HEADERS = address.h torlog.h crypto.h util.h compat.h aes.h torint.h tortls.h strlcpy.c strlcat.c torgzip.h container.h ht.h mempool.h memarea.h ciphers.inc compat_libevent.h tortls_states.h di_ops.h procmon.h +noinst_HEADERS = \ + address.h \ + aes.h \ + ciphers.inc \ + compat.h \ + compat_libevent.h \ + container.h \ + crypto.h \ + di_ops.h \ + ht.h \ + memarea.h \ + mempool.h \ + procmon.h \ + strlcat.c \ + strlcpy.c \ + torgzip.h \ + torint.h \ + torlog.h \ + tortls.h \ + util.h common_sha1.i: $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS) if test "@SHA1SUM@" != none; then \ - "@SHA1SUM@" $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS) | "@SED@" -n 's/^\(.*\)$$/"\1\\n"/p' > common_sha1.i; \ + (cd "$(srcdir)" && "@SHA1SUM@" $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS)) | "@SED@" -n 's/^\(.*\)$$/"\1\\n"/p' > common_sha1.i; \ elif test "@OPENSSL@" != none; then \ - "@OPENSSL@" sha1 $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS) | "@SED@" -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > common_sha1.i; \ + (cd "$(srcdir)" && "@OPENSSL@" sha1 $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS)) | "@SED@" -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > common_sha1.i; \ else \ rm common_sha1.i; \ touch common_sha1.i; \ diff --git a/src/common/Makefile.nmake b/src/common/Makefile.nmake new file mode 100644 index 0000000000..e548273670 --- /dev/null +++ b/src/common/Makefile.nmake @@ -0,0 +1,20 @@ +all: libor.lib libor-crypto.lib libor-event.lib + +CFLAGS = /I ..\win32 /I ..\..\..\build-alpha\include + +LIBOR_OBJECTS = address.obj compat.obj container.obj di_ops.obj \ + log.obj memarea.obj mempool.obj procmon.obj util.obj \ + util_codedigest.obj + +LIBOR_CRYPTO_OBJECTS = aes.obj crypto.obj torgzip.obj tortls.obj + +LIBOR_EVENT_OBJECTS = compat_libevent.obj + +libor.lib: $(LIBOR_OBJECTS) + lib $(LIBOR_OBJECTS) /out:libor.lib + +libor-crypto.lib: $(LIBOR_CRYPTO_OBJECTS) + lib $(LIBOR_CRYPTO_OBJECTS) /out:libor-crypto.lib + +libor-event.lib: $(LIBOR_EVENT_OBJECTS) + lib $(LIBOR_EVENT_OBJECTS) /out:libor-event.lib diff --git a/src/common/OpenBSD_malloc_Linux.c b/src/common/OpenBSD_malloc_Linux.c index 92ca9c0066..da82729811 100644 --- a/src/common/OpenBSD_malloc_Linux.c +++ b/src/common/OpenBSD_malloc_Linux.c @@ -14,6 +14,10 @@ * ---------------------------------------------------------------------------- */ +/* We use this macro to remove some code that we don't actually want, + * rather than to fix its warnings. */ +#define BUILDING_FOR_TOR + /* * Defining MALLOC_EXTRA_SANITY will enable extra checks which are * related to internal conditions and consistency in malloc.c. This has @@ -79,6 +83,7 @@ static size_t g_alignment = 0; extern int __libc_enable_secure; +#ifndef HAVE_ISSETUGID static int issetugid(void) { if (__libc_enable_secure) return 1; @@ -86,8 +91,10 @@ static int issetugid(void) if (getgid() != getegid()) return 1; return 0; } +#endif #define PGSHIFT 12 +#undef MADV_FREE #define MADV_FREE MADV_DONTNEED #include <pthread.h> static pthread_mutex_t gen_mutex = PTHREAD_MUTEX_INITIALIZER; @@ -443,6 +450,7 @@ extern char *__progname; static void wrterror(const char *p) { +#ifndef BUILDING_FOR_TOR const char *q = " error: "; struct iovec iov[5]; @@ -457,7 +465,9 @@ wrterror(const char *p) iov[4].iov_base = (char*)"\n"; iov[4].iov_len = 1; writev(STDERR_FILENO, iov, 5); - +#else + (void)p; +#endif suicide = 1; #ifdef MALLOC_STATS if (malloc_stats) @@ -471,14 +481,17 @@ wrterror(const char *p) static void wrtwarning(const char *p) { +#ifndef BUILDING_FOR_TOR const char *q = " warning: "; struct iovec iov[5]; +#endif if (malloc_abort) wrterror(p); else if (malloc_silent) return; +#ifndef BUILDING_FOR_TOR iov[0].iov_base = __progname; iov[0].iov_len = strlen(__progname); iov[1].iov_base = (char*)malloc_func; @@ -489,8 +502,11 @@ wrtwarning(const char *p) iov[3].iov_len = strlen(p); iov[4].iov_base = (char*)"\n"; iov[4].iov_len = 1; - - writev(STDERR_FILENO, iov, 5); + + (void) writev(STDERR_FILENO, iov, 5); +#else + (void)p; +#endif } #ifdef MALLOC_STATS @@ -665,7 +681,7 @@ malloc_init(void) for (i = 0; i < 3; i++) { switch (i) { case 0: - j = readlink("/etc/malloc.conf", b, sizeof b - 1); + j = (int) readlink("/etc/malloc.conf", b, sizeof b - 1); if (j <= 0) continue; b[j] = '\0'; @@ -1145,9 +1161,10 @@ malloc_bytes(size_t size) if (size == 0) j = 0; else { + size_t ii; j = 1; - i = size - 1; - while (i >>= 1) + ii = size - 1; + while (ii >>= 1) j++; } @@ -1971,6 +1988,7 @@ calloc(size_t num, size_t size) return(p); } +#ifndef BUILDING_FOR_TOR static int ispowerof2 (size_t a) { size_t b; for (b = 1ULL << (sizeof(size_t)*NBBY - 1); b > 1; b >>= 1) @@ -1978,7 +1996,9 @@ static int ispowerof2 (size_t a) { return 1; return 0; } +#endif +#ifndef BUILDING_FOR_TOR int posix_memalign(void **memptr, size_t alignment, size_t size) { void *r; @@ -2015,18 +2035,20 @@ void *valloc(size_t size) posix_memalign(&r, malloc_pagesize, size); return r; } +#endif size_t malloc_good_size(size_t size) { if (size == 0) { return 1; } else if (size <= malloc_maxsize) { - int i, j; + int j; + size_t ii; /* round up to the nearest power of 2, with same approach * as malloc_bytes() uses. */ j = 1; - i = size - 1; - while (i >>= 1) + ii = size - 1; + while (ii >>= 1) j++; return ((size_t)1) << j; } else { diff --git a/src/common/address.c b/src/common/address.c index 17bdea9238..df26f61f8f 100644 --- a/src/common/address.c +++ b/src/common/address.c @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -13,10 +13,16 @@ #include "util.h" #include "address.h" #include "torlog.h" +#include "container.h" -#ifdef MS_WINDOWS +#ifdef _WIN32 #include <process.h> #include <windows.h> +#include <winsock2.h> +/* For access to structs needed by GetAdaptersAddresses */ +#undef _WIN32_WINNT +#define _WIN32_WINNT 0x0501 +#include <iphlpapi.h> #endif #ifdef HAVE_SYS_TIME_H @@ -46,12 +52,28 @@ #ifdef HAVE_SYS_UN_H #include <sys/un.h> #endif +#ifdef HAVE_IFADDRS_H +#include <ifaddrs.h> +#endif +#ifdef HAVE_SYS_IOCTL_H +#include <sys/ioctl.h> +#endif +#ifdef HAVE_NET_IF_H +#include <net/if.h> +#endif #include <stdarg.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <assert.h> +/* tor_addr_is_null() and maybe other functions rely on AF_UNSPEC being 0 to + * work correctly. Bail out here if we've found a platform where AF_UNSPEC + * isn't 0. */ +#if AF_UNSPEC != 0 +#error We rely on AF_UNSPEC being 0. Let us know about your platform, please! +#endif + /** Convert the tor_addr_t in <b>a</b>, with port in <b>port</b>, into a * sockaddr object in *<b>sa_out</b> of object size <b>len</b>. If not enough * room is available in sa_out, or on error, return 0. On success, return @@ -253,7 +275,7 @@ tor_addr_lookup(const char *name, uint16_t family, tor_addr_t *addr) ent = err ? NULL : &hent; #else ent = gethostbyname(name); -#ifdef MS_WINDOWS +#ifdef _WIN32 err = WSAGetLastError(); #else err = h_errno; @@ -269,7 +291,7 @@ tor_addr_lookup(const char *name, uint16_t family, tor_addr_t *addr) } return 0; } -#ifdef MS_WINDOWS +#ifdef _WIN32 return (err == WSATRY_AGAIN) ? 1 : -1; #else return (err == TRY_AGAIN) ? 1 : -1; @@ -333,7 +355,9 @@ tor_addr_is_internal(const tor_addr_t *addr, int for_listening) /* unknown address family... assume it's not safe for external use */ /* rather than tor_assert(0) */ - log_warn(LD_BUG, "tor_addr_is_internal() called with a non-IP address."); + log_warn(LD_BUG, "tor_addr_is_internal() called with a non-IP address of " + "type %d", (int)v_family); + tor_fragile_assert(); return 1; } @@ -343,22 +367,28 @@ tor_addr_is_internal(const tor_addr_t *addr, int for_listening) * brackets. */ const char * -tor_addr_to_str(char *dest, const tor_addr_t *addr, int len, int decorate) +tor_addr_to_str(char *dest, const tor_addr_t *addr, size_t len, int decorate) { const char *ptr; tor_assert(addr && dest); switch (tor_addr_family(addr)) { case AF_INET: - if (len<3) + /* Shortest addr x.x.x.x + \0 */ + if (len < 8) return NULL; - ptr = tor_inet_ntop(AF_INET, &addr->addr.in_addr, dest, len); + ptr = tor_inet_ntop(AF_INET, &addr->addr.in_addr, dest, len); break; case AF_INET6: + /* Shortest addr [ :: ] + \0 */ + if (len < (3 + (decorate ? 2 : 0))) + return NULL; + if (decorate) ptr = tor_inet_ntop(AF_INET6, &addr->addr.in6_addr, dest+1, len-2); else ptr = tor_inet_ntop(AF_INET6, &addr->addr.in6_addr, dest, len); + if (ptr && decorate) { *dest = '['; memcpy(dest+strlen(dest), "]", 2); @@ -384,7 +414,7 @@ tor_addr_to_str(char *dest, const tor_addr_t *addr, int len, int decorate) * IPv4 or IPv6 address too. */ int -tor_addr_parse_reverse_lookup_name(tor_addr_t *result, const char *address, +tor_addr_parse_PTR_name(tor_addr_t *result, const char *address, int family, int accept_regular) { if (!strcasecmpend(address, ".in-addr.arpa")) { @@ -455,7 +485,7 @@ tor_addr_parse_reverse_lookup_name(tor_addr_t *result, const char *address, if (accept_regular) { tor_addr_t tmp; - int r = tor_addr_from_str(&tmp, address); + int r = tor_addr_parse(&tmp, address); if (r < 0) return 0; if (r != family && family != AF_UNSPEC) @@ -470,13 +500,17 @@ tor_addr_parse_reverse_lookup_name(tor_addr_t *result, const char *address, return 0; } -/** Convert <b>addr</b> to an in-addr.arpa name or a .ip6.arpa name, and store - * the result in the <b>outlen</b>-byte buffer at <b>out</b>. Return 0 on - * success, -1 on failure. */ +/** Convert <b>addr</b> to an in-addr.arpa name or a .ip6.arpa name, + * and store the result in the <b>outlen</b>-byte buffer at + * <b>out</b>. Return the number of chars written to <b>out</b>, not + * including the trailing \0, on success. Returns -1 on failure. */ int -tor_addr_to_reverse_lookup_name(char *out, size_t outlen, - const tor_addr_t *addr) +tor_addr_to_PTR_name(char *out, size_t outlen, + const tor_addr_t *addr) { + tor_assert(out); + tor_assert(addr); + if (addr->family == AF_INET) { uint32_t a = tor_addr_to_ipv4h(addr); @@ -499,7 +533,7 @@ tor_addr_to_reverse_lookup_name(char *out, size_t outlen, *cp++ = '.'; } memcpy(cp, "ip6.arpa", 9); /* 8 characters plus NUL */ - return 0; + return 32 * 2 + 8; } return -1; } @@ -745,7 +779,8 @@ tor_addr_is_loopback(const tor_addr_t *addr) case AF_INET6: { /* ::1 */ uint32_t *a32 = tor_addr_to_in6_addr32(addr); - return (a32[0] == 0) && (a32[1] == 0) && (a32[2] == 0) && (a32[3] == 1); + return (a32[0] == 0) && (a32[1] == 0) && (a32[2] == 0) && + (ntohl(a32[3]) == 1); } case AF_INET: /* 127.0.0.1 */ @@ -952,21 +987,39 @@ tor_dup_addr(const tor_addr_t *addr) } } -/** Return a string representing the address <b>addr</b>. This string is - * statically allocated, and must not be freed. Each call to - * <b>fmt_addr</b> invalidates the last result of the function. This - * function is not thread-safe. */ +/** Return a string representing the address <b>addr</b>. This string + * is statically allocated, and must not be freed. Each call to + * <b>fmt_addr_impl</b> invalidates the last result of the function. + * This function is not thread-safe. If <b>decorate</b> is set, add + * brackets to IPv6 addresses. + * + * It's better to use the wrapper macros of this function: + * <b>fmt_addr()</b> and <b>fmt_and_decorate_addr()</b>. + */ const char * -fmt_addr(const tor_addr_t *addr) +fmt_addr_impl(const tor_addr_t *addr, int decorate) { static char buf[TOR_ADDR_BUF_LEN]; if (!addr) return "<null>"; - if (tor_addr_to_str(buf, addr, sizeof(buf), 0)) + if (tor_addr_to_str(buf, addr, sizeof(buf), decorate)) return buf; else return "???"; } +/** Like fmt_addr(), but takes <b>addr</b> as a host-order IPv4 + * addresses. Also not thread-safe, also clobbers its return buffer on + * repeated calls. */ +const char * +fmt_addr32(uint32_t addr) +{ + static char buf[INET_NTOA_BUF_LEN]; + struct in_addr in; + in.s_addr = htonl(addr); + tor_inet_ntoa(&in, buf, sizeof(buf)); + return buf; +} + /** Convert the string in <b>src</b> to a tor_addr_t <b>addr</b>. The string * may be an IPv4 address, an IPv6 address, or an IPv6 address surrounded by * square brackets. @@ -974,7 +1027,7 @@ fmt_addr(const tor_addr_t *addr) * Return an address family on success, or -1 if an invalid address string is * provided. */ int -tor_addr_from_str(tor_addr_t *addr, const char *src) +tor_addr_parse(tor_addr_t *addr, const char *src) { char *tmp = NULL; /* Holds substring if we got a dotted quad. */ int result; @@ -1002,7 +1055,7 @@ tor_addr_from_str(tor_addr_t *addr, const char *src) * address as needed, and put the result in <b>addr_out</b> and (optionally) * <b>port_out</b>. Return 0 on success, negative on failure. */ int -tor_addr_port_parse(const char *s, tor_addr_t *addr_out, uint16_t *port_out) +tor_addr_port_lookup(const char *s, tor_addr_t *addr_out, uint16_t *port_out) { const char *port; tor_addr_t addr; @@ -1056,6 +1109,169 @@ tor_addr_port_parse(const char *s, tor_addr_t *addr_out, uint16_t *port_out) return -1; } +#ifdef _WIN32 +typedef ULONG (WINAPI *GetAdaptersAddresses_fn_t)( + ULONG, ULONG, PVOID, PIP_ADAPTER_ADDRESSES, PULONG); +#endif + +/** Try to ask our network interfaces what addresses they are bound to. + * Return a new smartlist of tor_addr_t on success, and NULL on failure. + * (An empty smartlist indicates that we successfully learned that we have no + * addresses.) Log failure messages at <b>severity</b>. */ +static smartlist_t * +get_interface_addresses_raw(int severity) +{ +#if defined(HAVE_GETIFADDRS) + /* Most free Unixy systems provide getifaddrs, which gives us a linked list + * of struct ifaddrs. */ + struct ifaddrs *ifa = NULL; + const struct ifaddrs *i; + smartlist_t *result; + if (getifaddrs(&ifa) < 0) { + log_fn(severity, LD_NET, "Unable to call getifaddrs(): %s", + strerror(errno)); + return NULL; + } + + result = smartlist_new(); + for (i = ifa; i; i = i->ifa_next) { + tor_addr_t tmp; + if (!i->ifa_addr) + continue; + if (i->ifa_addr->sa_family != AF_INET && + i->ifa_addr->sa_family != AF_INET6) + continue; + if (tor_addr_from_sockaddr(&tmp, i->ifa_addr, NULL) < 0) + continue; + smartlist_add(result, tor_memdup(&tmp, sizeof(tmp))); + } + + freeifaddrs(ifa); + return result; +#elif defined(_WIN32) + /* Windows XP began to provide GetAdaptersAddresses. Windows 2000 had a + "GetAdaptersInfo", but that's deprecated; let's just try + GetAdaptersAddresses and fall back to connect+getsockname. + */ + HANDLE lib = load_windows_system_library(TEXT("iphlpapi.dll")); + smartlist_t *result = NULL; + GetAdaptersAddresses_fn_t fn; + ULONG size, res; + IP_ADAPTER_ADDRESSES *addresses = NULL, *address; + + (void) severity; + +#define FLAGS (GAA_FLAG_SKIP_ANYCAST | \ + GAA_FLAG_SKIP_MULTICAST | \ + GAA_FLAG_SKIP_DNS_SERVER) + + if (!lib) { + log_fn(severity, LD_NET, "Unable to load iphlpapi.dll"); + goto done; + } + + if (!(fn = (GetAdaptersAddresses_fn_t) + GetProcAddress(lib, "GetAdaptersAddresses"))) { + log_fn(severity, LD_NET, "Unable to obtain pointer to " + "GetAdaptersAddresses"); + goto done; + } + + /* Guess how much space we need. */ + size = 15*1024; + addresses = tor_malloc(size); + res = fn(AF_UNSPEC, FLAGS, NULL, addresses, &size); + if (res == ERROR_BUFFER_OVERFLOW) { + /* we didn't guess that we needed enough space; try again */ + tor_free(addresses); + addresses = tor_malloc(size); + res = fn(AF_UNSPEC, FLAGS, NULL, addresses, &size); + } + if (res != NO_ERROR) { + log_fn(severity, LD_NET, "GetAdaptersAddresses failed (result: %lu)", res); + goto done; + } + + result = smartlist_new(); + for (address = addresses; address; address = address->Next) { + IP_ADAPTER_UNICAST_ADDRESS *a; + for (a = address->FirstUnicastAddress; a; a = a->Next) { + /* Yes, it's a linked list inside a linked list */ + struct sockaddr *sa = a->Address.lpSockaddr; + tor_addr_t tmp; + if (sa->sa_family != AF_INET && sa->sa_family != AF_INET6) + continue; + if (tor_addr_from_sockaddr(&tmp, sa, NULL) < 0) + continue; + smartlist_add(result, tor_memdup(&tmp, sizeof(tmp))); + } + } + + done: + if (lib) + FreeLibrary(lib); + tor_free(addresses); + return result; +#elif defined(SIOCGIFCONF) && defined(HAVE_IOCTL) + /* Some older unixy systems make us use ioctl(SIOCGIFCONF) */ + struct ifconf ifc; + int fd, i, sz, n; + smartlist_t *result = NULL; + /* This interface, AFAICT, only supports AF_INET addresses */ + fd = socket(AF_INET, SOCK_DGRAM, 0); + if (fd < 0) { + log(severity, LD_NET, "socket failed: %s", strerror(errno)); + goto done; + } + /* Guess how much space we need. */ + ifc.ifc_len = sz = 15*1024; + ifc.ifc_ifcu.ifcu_req = tor_malloc(sz); + if (ioctl(fd, SIOCGIFCONF, &ifc) < 0) { + log(severity, LD_NET, "ioctl failed: %s", strerror(errno)); + close(fd); + goto done; + } + close(fd); + result = smartlist_new(); + if (ifc.ifc_len < sz) + sz = ifc.ifc_len; + n = sz / sizeof(struct ifreq); + for (i = 0; i < n ; ++i) { + struct ifreq *r = &ifc.ifc_ifcu.ifcu_req[i]; + struct sockaddr *sa = &r->ifr_addr; + tor_addr_t tmp; + if (sa->sa_family != AF_INET && sa->sa_family != AF_INET6) + continue; /* should be impossible */ + if (tor_addr_from_sockaddr(&tmp, sa, NULL) < 0) + continue; + smartlist_add(result, tor_memdup(&tmp, sizeof(tmp))); + } + done: + tor_free(ifc.ifc_ifcu.ifcu_req); + return result; +#else + (void) severity; + return NULL; +#endif +} + +/** Return true iff <b>a</b> is a multicast address. */ +static int +tor_addr_is_multicast(const tor_addr_t *a) +{ + sa_family_t family = tor_addr_family(a); + if (family == AF_INET) { + uint32_t ipv4h = tor_addr_to_ipv4h(a); + if ((ipv4h >> 24) == 0xe0) + return 1; /* Multicast */ + } else if (family == AF_INET6) { + const uint8_t *a32 = tor_addr_to_in6_addr8(a); + if (a32[0] == 0xff) + return 1; + } + return 0; +} + /** Set *<b>addr</b> to the IP address (if any) of whatever interface * connects to the Internet. This address should only be used in checking * whether our address has changed. Return 0 on success, -1 on failure. @@ -1063,12 +1279,38 @@ tor_addr_port_parse(const char *s, tor_addr_t *addr_out, uint16_t *port_out) int get_interface_address6(int severity, sa_family_t family, tor_addr_t *addr) { + /* XXX really, this function should yield a smartlist of addresses. */ + smartlist_t *addrs; int sock=-1, r=-1; struct sockaddr_storage my_addr, target_addr; socklen_t addr_len; - tor_assert(addr); + /* Try to do this the smart way if possible. */ + if ((addrs = get_interface_addresses_raw(severity))) { + int rv = -1; + SMARTLIST_FOREACH_BEGIN(addrs, tor_addr_t *, a) { + if (family != AF_UNSPEC && family != tor_addr_family(a)) + continue; + if (tor_addr_is_loopback(a) || + tor_addr_is_multicast(a)) + continue; + + tor_addr_copy(addr, a); + rv = 0; + + /* If we found a non-internal address, declare success. Otherwise, + * keep looking. */ + if (!tor_addr_is_internal(a, 0)) + break; + } SMARTLIST_FOREACH_END(a); + + SMARTLIST_FOREACH(addrs, tor_addr_t *, a, tor_free(a)); + smartlist_free(addrs); + return rv; + } + + /* Okay, the smart way is out. */ memset(addr, 0, sizeof(tor_addr_t)); memset(&target_addr, 0, sizeof(target_addr)); /* Don't worry: no packets are sent. We just need to use a real address @@ -1122,7 +1364,7 @@ get_interface_address6(int severity, sa_family_t family, tor_addr_t *addr) /* ====== * IPv4 helpers - * XXXX023 IPv6 deprecate some of these. + * XXXX024 IPv6 deprecate some of these. */ /** Return true iff <b>ip</b> (in host order) is an IP reserved to localhost, @@ -1138,6 +1380,20 @@ is_internal_IP(uint32_t ip, int for_listening) return tor_addr_is_internal(&myaddr, for_listening); } +/** Given an address of the form "host:port", try to divide it into its host + * ane port portions, setting *<b>address_out</b> to a newly allocated string + * holding the address portion and *<b>port_out</b> to the port (or 0 if no + * port is given). Return 0 on success, -1 on failure. */ +int +tor_addr_port_split(int severity, const char *addrport, + char **address_out, uint16_t *port_out) +{ + tor_assert(addrport); + tor_assert(address_out); + tor_assert(port_out); + return addr_port_lookup(severity, addrport, address_out, NULL, port_out); +} + /** Parse a string of the form "host[:port]" from <b>addrport</b>. If * <b>address</b> is provided, set *<b>address</b> to a copy of the * host portion of the string. If <b>addr</b> is provided, try to @@ -1149,7 +1405,7 @@ is_internal_IP(uint32_t ip, int for_listening) * Return 0 on success, -1 on failure. */ int -parse_addr_port(int severity, const char *addrport, char **address, +addr_port_lookup(int severity, const char *addrport, char **address, uint32_t *addr, uint16_t *port_out) { const char *colon; @@ -1159,7 +1415,7 @@ parse_addr_port(int severity, const char *addrport, char **address, tor_assert(addrport); - colon = strchr(addrport, ':'); + colon = strrchr(addrport, ':'); if (colon) { _address = tor_strndup(addrport, colon-addrport); _port = (int) tor_parse_long(colon+1,10,1,65535,NULL,NULL); @@ -1271,7 +1527,7 @@ parse_port_range(const char *port, uint16_t *port_min_out, } else if (endptr && *endptr == '-') { port = endptr+1; endptr = NULL; - port_max = (int)tor_parse_long(port, 10, 1, 65536, &ok, &endptr); + port_max = (int)tor_parse_long(port, 10, 1, 65535, &ok, &endptr); if (!ok) { log_warn(LD_GENERAL, "Malformed port %s on address range; rejecting.", @@ -1432,3 +1688,13 @@ get_interface_address(int severity, uint32_t *addr) return r; } +/** Return true if we can tell that <b>name</b> is a canonical name for the + * loopback address. */ +int +tor_addr_hostname_is_local(const char *name) +{ + return !strcasecmp(name, "localhost") || + !strcasecmp(name, "local") || + !strcasecmpend(name, ".local"); +} + diff --git a/src/common/address.h b/src/common/address.h index 9a7656f69b..c6c126862a 100644 --- a/src/common/address.h +++ b/src/common/address.h @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -26,11 +26,22 @@ typedef struct tor_addr_t { sa_family_t family; union { + uint32_t dummy_; /* This field is here so we have something to initialize + * with a reliable cross-platform type. */ struct in_addr in_addr; struct in6_addr in6_addr; } addr; } tor_addr_t; +/** Holds an IP address and a TCP/UDP port. */ +typedef struct tor_addr_port_t +{ + tor_addr_t addr; + uint16_t port; +} tor_addr_port_t; + +#define TOR_ADDR_NULL {AF_UNSPEC, {0}}; + static INLINE const struct in6_addr *tor_addr_to_in6(const tor_addr_t *a); static INLINE uint32_t tor_addr_to_ipv4n(const tor_addr_t *a); static INLINE uint32_t tor_addr_to_ipv4h(const tor_addr_t *a); @@ -126,7 +137,15 @@ tor_addr_eq_ipv4h(const tor_addr_t *a, uint32_t u) int tor_addr_lookup(const char *name, uint16_t family, tor_addr_t *addr_out); char *tor_dup_addr(const tor_addr_t *addr) ATTR_MALLOC; -const char *fmt_addr(const tor_addr_t *addr); + +/** Wrapper function of fmt_addr_impl(). It does not decorate IPv6 + * addresses. */ +#define fmt_addr(a) fmt_addr_impl((a), 0) +/** Wrapper function of fmt_addr_impl(). It decorates IPv6 + * addresses. */ +#define fmt_and_decorate_addr(a) fmt_addr_impl((a), 1) +const char *fmt_addr_impl(const tor_addr_t *addr, int decorate); +const char * fmt_addr32(uint32_t addr); int get_interface_address6(int severity, sa_family_t family, tor_addr_t *addr); /** Flag to specify how to do a comparison between addresses. In an "exact" @@ -148,24 +167,24 @@ int tor_addr_compare_masked(const tor_addr_t *addr1, const tor_addr_t *addr2, unsigned int tor_addr_hash(const tor_addr_t *addr); int tor_addr_is_v4(const tor_addr_t *addr); -int tor_addr_is_internal(const tor_addr_t *ip, int for_listening) ATTR_PURE; +int tor_addr_is_internal(const tor_addr_t *ip, int for_listening); /** Longest length that can be required for a reverse lookup name. */ /* 32 nybbles, 32 dots, 8 characters of "ip6.arpa", 1 NUL: 73 characters. */ #define REVERSE_LOOKUP_NAME_BUF_LEN 73 -int tor_addr_to_reverse_lookup_name(char *out, size_t outlen, +int tor_addr_to_PTR_name(char *out, size_t outlen, const tor_addr_t *addr); -int tor_addr_parse_reverse_lookup_name(tor_addr_t *result, const char *address, +int tor_addr_parse_PTR_name(tor_addr_t *result, const char *address, int family, int accept_regular); -int tor_addr_port_parse(const char *s, tor_addr_t *addr_out, +int tor_addr_port_lookup(const char *s, tor_addr_t *addr_out, uint16_t *port_out); int tor_addr_parse_mask_ports(const char *s, tor_addr_t *addr_out, maskbits_t *mask_out, uint16_t *port_min_out, uint16_t *port_max_out); -const char * tor_addr_to_str(char *dest, const tor_addr_t *addr, int len, +const char * tor_addr_to_str(char *dest, const tor_addr_t *addr, size_t len, int decorate); -int tor_addr_from_str(tor_addr_t *addr, const char *src); +int tor_addr_parse(tor_addr_t *addr, const char *src); void tor_addr_copy(tor_addr_t *dest, const tor_addr_t *src); void tor_addr_from_ipv4n(tor_addr_t *dest, uint32_t v4addr); /** Set <b>dest</b> to the IPv4 address encoded in <b>v4addr</b> in host @@ -180,9 +199,14 @@ void tor_addr_from_in6(tor_addr_t *dest, const struct in6_addr *in6); int tor_addr_is_null(const tor_addr_t *addr); int tor_addr_is_loopback(const tor_addr_t *addr); +int tor_addr_port_split(int severity, const char *addrport, + char **address_out, uint16_t *port_out); + +int tor_addr_hostname_is_local(const char *name); + /* IPv4 helpers */ -int is_internal_IP(uint32_t ip, int for_listening) ATTR_PURE; -int parse_addr_port(int severity, const char *addrport, char **address, +int is_internal_IP(uint32_t ip, int for_listening); +int addr_port_lookup(int severity, const char *addrport, char **address, uint32_t *addr, uint16_t *port_out); int parse_port_range(const char *port, uint16_t *port_min_out, uint16_t *port_max_out); diff --git a/src/common/aes.c b/src/common/aes.c index c2fdeb594a..295a90749a 100644 --- a/src/common/aes.c +++ b/src/common/aes.c @@ -1,137 +1,149 @@ /* Copyright (c) 2001, Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** * \file aes.c - * \brief Implements the AES cipher (with 128-bit keys and blocks), - * and a counter-mode stream cipher on top of AES. This code is - * taken from the main Rijndael distribution. (We include this - * because many people are running older versions of OpenSSL without - * AES support.) + * \brief Implements a counter-mode stream cipher on top of AES. **/ #include "orconfig.h" + +#ifdef _WIN32 /*wrkard for dtls1.h >= 0.9.8m of "#include <winsock.h>"*/ + #ifndef _WIN32_WINNT + #define _WIN32_WINNT 0x0501 + #endif + #define WIN32_LEAN_AND_MEAN + #if defined(_MSC_VER) && (_MSC_VER < 1300) + #include <winsock.h> + #else + #include <winsock2.h> + #include <ws2tcpip.h> + #endif +#endif + #include <openssl/opensslv.h> #include <assert.h> #include <stdlib.h> #include <string.h> +#include <openssl/aes.h> +#include <openssl/evp.h> +#include <openssl/engine.h> +#include "crypto.h" +#if OPENSSL_VERSION_NUMBER >= OPENSSL_V_SERIES(1,0,0) +/* See comments about which counter mode implementation to use below. */ +#include <openssl/modes.h> +#define CAN_USE_OPENSSL_CTR +#endif #include "compat.h" #include "aes.h" #include "util.h" #include "torlog.h" -/* We have 3 strategies for getting AES: Via OpenSSL's AES_encrypt function, - * via OpenSSL's EVP_EncryptUpdate function, or via the built-in AES - * implementation below. */ - -/** Defined iff we're using OpenSSL's AES functions for AES. */ -#undef USE_OPENSSL_AES -/** Defined iff we're using OpenSSL's EVP code for AES. */ -#undef USE_OPENSSL_EVP -/** Defined iff we're using Tor's internal AES implementation, defined - * below. */ -#undef USE_BUILTIN_AES +#ifdef ANDROID +/* Android's OpenSSL seems to have removed all of its Engine support. */ +#define DISABLE_ENGINES +#endif -/* Figure out our CPU type. We use this to pick an AES implementation. - * Macros are as listed at http://predef.sourceforge.net/prearch.html +/* We have five strategies for implementing AES counter mode. + * + * Best with x86 and x86_64: Use EVP_aes_ctr128() and EVP_EncryptUpdate(). + * This is possible with OpenSSL 1.0.1, where the counter-mode implementation + * can use bit-sliced or vectorized AES or AESNI as appropriate. + * + * Otherwise: Pick the best possible AES block implementation that OpenSSL + * gives us, and the best possible counter-mode implementation, and combine + * them. */ -#if (defined(i386) || defined(__i386__) || defined(__i386) || defined(_X86_) \ - || defined(_M_IX86) || defined(__THW_INTEL__) || defined(__I86__)) -# define CPU_IS_X86 -#elif (defined(__amd64__) || defined(__amd64) || \ - defined(__x86_64__) || defined(__x86_64) || \ - defined(_M_X64)) -# define CPU_IS_X86_64 -#elif (defined(__ia64__) || defined(__ia64) || defined(_IA64) || \ - defined(_M_IA64)) -# define CPU_IS_IA64 -#elif (defined(__sparc__) || defined(__sparc)) -# define CPU_IS_SPARC -#elif (defined(__arm__) || defined (__TARGET_ARCH_ARM)) -# define CPU_IS_ARM -#endif +#if OPENSSL_VERSION_NUMBER >= OPENSSL_V_NOPATCH(1,0,1) && \ + (defined(__i386) || defined(__i386__) || defined(_M_IX86) || \ + defined(__x86_64) || defined(__x86_64__) || \ + defined(_M_AMD64) || defined(_M_X64) || defined(__INTEL__)) \ -/* Here we pick which to use, if none is force-defined. See - * http://archives.seul.org/or/dev/Feb-2007/msg00045.html - * for a summary of the most recent benchmarking results that led to this - * nutty decision tree. -*/ -#if (!defined(USE_BUILTIN_AES) && \ - !defined(USE_OPENSSL_AES) && \ - !defined(USE_OPENSSL_EVP)) +#define USE_EVP_AES_CTR -/* OpenSSL 0.9.7 was the first to support AES. It was slower than our - * built-in implementation. - * OpenSSL 0.9.8 added assembly implementations for i386 and ia64. - * Either the i386 stuff isn't used for x86-64, or it isn't faster. - * OpenSSL 0.9.9 (not yet out) has added assembly implementations for - * x86_64 (aka amd64), sparc9, and arm +#endif + +/* We have 2 strategies for getting the AES block cipher: Via OpenSSL's + * AES_encrypt function, or via OpenSSL's EVP_EncryptUpdate function. * - * Note: the "f" at the end of OpenSSL version numbers below means - * "release". */ -# if defined(CPU_IS_X86) || defined(CPU_IS_IA64) -# if OPENSSL_VERSION_NUMBER >= 0x0090800fL -# define USE_OPENSSL_AES -# endif -# endif + * If there's any hardware acceleration in play, we want to be using EVP_* so + * we can get it. Otherwise, we'll want AES_*, which seems to be about 5% + * faster than indirecting through the EVP layer. + */ -# if defined(CPU_IS_X86_64) || defined(CPU_IS_ARM) || defined(CPU_IS_SPARC) -# if OPENSSL_VERSION_NUMBER >= 0x0090900fL -# define USE_OPENSSL_AES -# endif -# endif +/* We have 2 strategies for getting a plug-in counter mode: use our own, or + * use OpenSSL's. + * + * Here we have a counter mode that's faster than the one shipping with + * OpenSSL pre-1.0 (by about 10%!). But OpenSSL 1.0.0 added a counter mode + * implementation faster than the one here (by about 7%). So we pick which + * one to used based on the Openssl version above. (OpenSSL 1.0.0a fixed a + * critical bug in that counter mode implementation, so we need to test to + * make sure that we have a fixed version.) + */ -/* Otherwise, use the built-in implementation below. */ -# ifndef USE_OPENSSL_AES -# define USE_BUILTIN_AES -# endif -#endif /* endif need to pick a method */ +#ifdef USE_EVP_AES_CTR -/* Include OpenSSL headers as needed. */ -#ifdef USE_OPENSSL_AES -# include <openssl/aes.h> -#endif -#ifdef USE_OPENSSL_EVP -# include <openssl/evp.h> -#endif +struct aes_cnt_cipher { + EVP_CIPHER_CTX evp; +}; -/* Figure out which AES optimizations to use. */ -#ifdef USE_BUILTIN_AES -/** If this is defined, we take advantage of the fact that AES treats its - * input as a set of 4 32-bit words, so that there is no need to encode and - * decode the 128-bit counter before every block encryption */ -# define USE_RIJNDAEL_COUNTER_OPTIMIZATION -# if 0 && (defined(__powerpc__) || defined(__powerpc64__)) -/* XXXX do more experimentation before concluding this is actually - * a good idea. */ -# define FULL_UNROLL -# endif -#endif +aes_cnt_cipher_t * +aes_new_cipher(const char *key, const char *iv) +{ + aes_cnt_cipher_t *cipher; + cipher = tor_malloc_zero(sizeof(aes_cnt_cipher_t)); + EVP_EncryptInit(&cipher->evp, EVP_aes_128_ctr(), + (const unsigned char*)key, (const unsigned char *)iv); + return cipher; +} +void +aes_cipher_free(aes_cnt_cipher_t *cipher) +{ + if (!cipher) + return; + EVP_CIPHER_CTX_cleanup(&cipher->evp); + memwipe(cipher, 0, sizeof(aes_cnt_cipher_t)); + tor_free(cipher); +} +void +aes_crypt(aes_cnt_cipher_t *cipher, const char *input, size_t len, + char *output) +{ + int outl; -/*======================================================================*/ -/* From rijndael-alg-fst.h */ + tor_assert(len < INT_MAX); -typedef uint64_t u64; -typedef uint32_t u32; -typedef uint8_t u8; + EVP_EncryptUpdate(&cipher->evp, (unsigned char*)output, + &outl, (const unsigned char *)input, (int)len); +} +void +aes_crypt_inplace(aes_cnt_cipher_t *cipher, char *data, size_t len) +{ + int outl; -#ifdef USE_BUILTIN_AES -#define MAXNR 14 + tor_assert(len < INT_MAX); -static int rijndaelKeySetupEnc(u32 rk[/*4*(Nr + 1)*/], - const u8 cipherKey[], int keyBits); -#ifdef USE_RIJNDAEL_COUNTER_OPTIMIZATION -static void rijndaelEncrypt(const u32 rk[/*4*(Nr + 1)*/], int Nr, - u32 ctr3, u32 ctr2, - u32 ctr1, u32 ctr0, u8 ct[16]); + EVP_EncryptUpdate(&cipher->evp, (unsigned char*)data, + &outl, (unsigned char*)data, (int)len); +} +int +evaluate_evp_for_aes(int force_val) +{ + (void) force_val; + log_notice(LD_CRYPTO, "This version of OpenSSL has a known-good EVP " + "counter-mode implementation. Using it."); + return 0; +} +int +evaluate_ctr_for_aes(void) +{ + return 0; +} #else -static void rijndaelEncrypt(const u32 rk[/*4*(Nr + 1)*/], int Nr, - const u8 pt[16], u8 ct[16]); -#endif -#endif /*======================================================================*/ /* Interface to AES code, and counter implementation */ @@ -139,42 +151,129 @@ static void rijndaelEncrypt(const u32 rk[/*4*(Nr + 1)*/], int Nr, /** Implements an AES counter-mode cipher. */ struct aes_cnt_cipher { /** This next element (however it's defined) is the AES key. */ -#if defined(USE_OPENSSL_EVP) - EVP_CIPHER_CTX key; -#elif defined(USE_OPENSSL_AES) - AES_KEY key; -#else - u32 rk[4*(MAXNR+1)]; - int nr; -#endif + union { + EVP_CIPHER_CTX evp; + AES_KEY aes; + } key; -#if !defined(WORDS_BIGENDIAN) || defined(USE_RIJNDAEL_COUNTER_OPTIMIZATION) +#if !defined(WORDS_BIGENDIAN) #define USING_COUNTER_VARS /** These four values, together, implement a 128-bit counter, with * counter0 as the low-order word and counter3 as the high-order word. */ - u32 counter3; - u32 counter2; - u32 counter1; - u32 counter0; + uint32_t counter3; + uint32_t counter2; + uint32_t counter1; + uint32_t counter0; #endif -#ifndef USE_RIJNDAEL_COUNTER_OPTIMIZATION -#define USING_COUNTER_BUFS union { /** The counter, in big-endian order, as bytes. */ - u8 buf[16]; + uint8_t buf[16]; /** The counter, in big-endian order, as big-endian words. Note that * on big-endian platforms, this is redundant with counter3...0, * so we just use these values instead. */ - u32 buf32[4]; + uint32_t buf32[4]; } ctr_buf; -#endif + /** The encrypted value of ctr_buf. */ - u8 buf[16]; + uint8_t buf[16]; /** Our current stream position within buf. */ - u8 pos; + unsigned int pos; + + /** True iff we're using the evp implementation of this cipher. */ + uint8_t using_evp; }; +/** True iff we should prefer the EVP implementation for AES, either because + * we're testing it or because we have hardware acceleration configured */ +static int should_use_EVP = 0; + +#ifdef CAN_USE_OPENSSL_CTR +/** True iff we have tested the counter-mode implementation and found that it + * doesn't have the counter-mode bug from OpenSSL 1.0.0. */ +static int should_use_openssl_CTR = 0; +#endif + +/** Check whether we should use the EVP interface for AES. If <b>force_val</b> + * is nonnegative, we use use EVP iff it is true. Otherwise, we use EVP + * if there is an engine enabled for aes-ecb. */ +int +evaluate_evp_for_aes(int force_val) +{ + ENGINE *e; + + if (force_val >= 0) { + should_use_EVP = force_val; + return 0; + } +#ifdef DISABLE_ENGINES + should_use_EVP = 0; +#else + e = ENGINE_get_cipher_engine(NID_aes_128_ecb); + + if (e) { + log_notice(LD_CRYPTO, "AES engine \"%s\" found; using EVP_* functions.", + ENGINE_get_name(e)); + should_use_EVP = 1; + } else { + log_notice(LD_CRYPTO, "No AES engine found; using AES_* functions."); + should_use_EVP = 0; + } +#endif + + return 0; +} + +/** Test the OpenSSL counter mode implementation to see whether it has the + * counter-mode bug from OpenSSL 1.0.0. If the implementation works, then + * we will use it for future encryption/decryption operations. + * + * We can't just look at the OpenSSL version, since some distributions update + * their OpenSSL packages without changing the version number. + **/ +int +evaluate_ctr_for_aes(void) +{ +#ifdef CAN_USE_OPENSSL_CTR + /* Result of encrypting an all-zero block with an all-zero 128-bit AES key. + * This should be the same as encrypting an all-zero block with an all-zero + * 128-bit AES key in counter mode, starting at position 0 of the stream. + */ + static const unsigned char encrypt_zero[] = + "\x66\xe9\x4b\xd4\xef\x8a\x2c\x3b\x88\x4c\xfa\x59\xca\x34\x2b\x2e"; + unsigned char zero[16]; + unsigned char output[16]; + unsigned char ivec[16]; + unsigned char ivec_tmp[16]; + unsigned int pos, i; + AES_KEY key; + memset(zero, 0, sizeof(zero)); + memset(ivec, 0, sizeof(ivec)); + AES_set_encrypt_key(zero, 128, &key); + + pos = 0; + /* Encrypting a block one byte at a time should make the error manifest + * itself for known bogus openssl versions. */ + for (i=0; i<16; ++i) + AES_ctr128_encrypt(&zero[i], &output[i], 1, &key, ivec, ivec_tmp, &pos); + + if (memcmp(output, encrypt_zero, 16)) { + /* Counter mode is buggy */ + log_notice(LD_CRYPTO, "This OpenSSL has a buggy version of counter mode; " + "not using it."); + } else { + /* Counter mode is okay */ + log_notice(LD_CRYPTO, "This OpenSSL has a good implementation of counter " + "mode; using it."); + should_use_openssl_CTR = 1; + } +#else + log_notice(LD_CRYPTO, "This version of OpenSSL has a slow implementation of " + "counter mode; not using it."); +#endif + return 0; +} + #if !defined(USING_COUNTER_VARS) #define COUNTER(c, n) ((c)->ctr_buf.buf32[3-(n)]) #else @@ -194,34 +293,32 @@ _aes_fill_buf(aes_cnt_cipher_t *cipher) * 3) changing the counter position was not trivial, last time I looked. * None of these issues are insurmountable in principle. */ -#if defined(USE_BUILTIN_AES) && defined(USE_RIJNDAEL_COUNTER_OPTIMIZATION) - rijndaelEncrypt(cipher->rk, cipher->nr, - cipher->counter3, cipher->counter2, - cipher->counter1, cipher->counter0, cipher->buf); -#else -#if defined(USE_OPENSSL_EVP) - { + if (cipher->using_evp) { int outl=16, inl=16; - EVP_EncryptUpdate(&cipher->key, cipher->buf, &outl, + EVP_EncryptUpdate(&cipher->key.evp, cipher->buf, &outl, cipher->ctr_buf.buf, inl); + } else { + AES_encrypt(cipher->ctr_buf.buf, cipher->buf, &cipher->key.aes); } -#elif defined(USE_OPENSSL_AES) - AES_encrypt(cipher->ctr_buf.buf, cipher->buf, &cipher->key); -#else - rijndaelEncrypt(cipher->rk, cipher->nr, cipher->ctr_buf.buf, cipher->buf); -#endif -#endif } +static void aes_set_key(aes_cnt_cipher_t *cipher, const char *key, + int key_bits); +static void aes_set_iv(aes_cnt_cipher_t *cipher, const char *iv); + /** - * Return a newly allocated counter-mode AES128 cipher implementation. + * Return a newly allocated counter-mode AES128 cipher implementation, + * using the 128-bit key <b>key</b> and the 128-bit IV <b>iv</b>. */ aes_cnt_cipher_t* -aes_new_cipher(void) +aes_new_cipher(const char *key, const char *iv) { aes_cnt_cipher_t* result = tor_malloc_zero(sizeof(aes_cnt_cipher_t)); + aes_set_key(result, key, 128); + aes_set_iv(result, iv); + return result; } @@ -229,53 +326,58 @@ aes_new_cipher(void) * <b>key_bits</b> bits long (must be 128, 192, or 256). Also resets * the counter to 0. */ -void +static void aes_set_key(aes_cnt_cipher_t *cipher, const char *key, int key_bits) { -#if defined(USE_OPENSSL_EVP) - const EVP_CIPHER *c; - switch (key_bits) { - case 128: c = EVP_aes_128_ecb(); break; - case 192: c = EVP_aes_192_ecb(); break; - case 256: c = EVP_aes_256_ecb(); break; - default: tor_assert(0); + if (should_use_EVP) { + const EVP_CIPHER *c; + switch (key_bits) { + case 128: c = EVP_aes_128_ecb(); break; + case 192: c = EVP_aes_192_ecb(); break; + case 256: c = EVP_aes_256_ecb(); break; + default: tor_assert(0); + } + EVP_EncryptInit(&cipher->key.evp, c, (const unsigned char*)key, NULL); + cipher->using_evp = 1; + } else { + AES_set_encrypt_key((const unsigned char *)key, key_bits, &cipher->key.aes); + cipher->using_evp = 0; } - EVP_EncryptInit(&cipher->key, c, (const unsigned char*)key, NULL); -#elif defined(USE_OPENSSL_AES) - AES_set_encrypt_key((const unsigned char *)key, key_bits, &(cipher->key)); -#else - cipher->nr = rijndaelKeySetupEnc(cipher->rk, (const unsigned char*)key, - key_bits); -#endif + #ifdef USING_COUNTER_VARS cipher->counter0 = 0; cipher->counter1 = 0; cipher->counter2 = 0; cipher->counter3 = 0; #endif -#ifdef USING_COUNTER_BUFS + memset(cipher->ctr_buf.buf, 0, sizeof(cipher->ctr_buf.buf)); -#endif cipher->pos = 0; - _aes_fill_buf(cipher); + +#ifdef CAN_USE_OPENSSL_CTR + if (should_use_openssl_CTR) + memset(cipher->buf, 0, sizeof(cipher->buf)); + else +#endif + _aes_fill_buf(cipher); } /** Release storage held by <b>cipher</b> */ void -aes_free_cipher(aes_cnt_cipher_t *cipher) +aes_cipher_free(aes_cnt_cipher_t *cipher) { if (!cipher) return; -#ifdef USE_OPENSSL_EVP - EVP_CIPHER_CTX_cleanup(&cipher->key); -#endif - memset(cipher, 0, sizeof(aes_cnt_cipher_t)); + if (cipher->using_evp) { + EVP_CIPHER_CTX_cleanup(&cipher->key.evp); + } + memwipe(cipher, 0, sizeof(aes_cnt_cipher_t)); tor_free(cipher); } -#if defined(USING_COUNTER_VARS) && defined(USING_COUNTER_BUFS) +#if defined(USING_COUNTER_VARS) #define UPDATE_CTR_BUF(c, n) STMT_BEGIN \ (c)->ctr_buf.buf32[3-(n)] = htonl((c)->counter ## n); \ STMT_END @@ -283,6 +385,18 @@ aes_free_cipher(aes_cnt_cipher_t *cipher) #define UPDATE_CTR_BUF(c, n) #endif +#ifdef CAN_USE_OPENSSL_CTR +/* Helper function to use EVP with openssl's counter-mode wrapper. */ +static void evp_block128_fn(const uint8_t in[16], + uint8_t out[16], + const void *key) +{ + EVP_CIPHER_CTX *ctx = (void*)key; + int inl=16, outl=16; + EVP_EncryptUpdate(ctx, out, &outl, in, inl); +} +#endif + /** Encrypt <b>len</b> bytes from <b>input</b>, storing the result in * <b>output</b>. Uses the key in <b>cipher</b>, and advances the counter * by <b>len</b> bytes as it encrypts. @@ -291,32 +405,56 @@ void aes_crypt(aes_cnt_cipher_t *cipher, const char *input, size_t len, char *output) { - - /* XXXX This function is up to 5% of our runtime in some profiles; - * we should look into unrolling some of the loops; taking advantage - * of alignment, using a bigger buffer, and so on. Not till after 0.1.2.x, - * though. */ - int c = cipher->pos; - if (PREDICT_UNLIKELY(!len)) return; - - while (1) { - do { - if (len-- == 0) { cipher->pos = c; return; } - *(output++) = *(input++) ^ cipher->buf[c]; - } while (++c != 16); - cipher->pos = c = 0; - if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 0))) { - if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 1))) { - if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 2))) { - ++COUNTER(cipher, 3); - UPDATE_CTR_BUF(cipher, 3); +#ifdef CAN_USE_OPENSSL_CTR + if (should_use_openssl_CTR) { + if (cipher->using_evp) { + /* In openssl 1.0.0, there's an if'd out EVP_aes_128_ctr in evp.h. If + * it weren't disabled, it might be better just to use that. + */ + CRYPTO_ctr128_encrypt((const unsigned char *)input, + (unsigned char *)output, + len, + &cipher->key.evp, + cipher->ctr_buf.buf, + cipher->buf, + &cipher->pos, + evp_block128_fn); + } else { + AES_ctr128_encrypt((const unsigned char *)input, + (unsigned char *)output, + len, + &cipher->key.aes, + cipher->ctr_buf.buf, + cipher->buf, + &cipher->pos); + } + return; + } + else +#endif + { + int c = cipher->pos; + if (PREDICT_UNLIKELY(!len)) return; + + while (1) { + do { + if (len-- == 0) { cipher->pos = c; return; } + *(output++) = *(input++) ^ cipher->buf[c]; + } while (++c != 16); + cipher->pos = c = 0; + if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 0))) { + if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 1))) { + if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 2))) { + ++COUNTER(cipher, 3); + UPDATE_CTR_BUF(cipher, 3); + } + UPDATE_CTR_BUF(cipher, 2); } - UPDATE_CTR_BUF(cipher, 2); + UPDATE_CTR_BUF(cipher, 1); } - UPDATE_CTR_BUF(cipher, 1); + UPDATE_CTR_BUF(cipher, 0); + _aes_fill_buf(cipher); } - UPDATE_CTR_BUF(cipher, 0); - _aes_fill_buf(cipher); } } @@ -327,38 +465,42 @@ aes_crypt(aes_cnt_cipher_t *cipher, const char *input, size_t len, void aes_crypt_inplace(aes_cnt_cipher_t *cipher, char *data, size_t len) { - - /* XXXX This function is up to 5% of our runtime in some profiles; - * we should look into unrolling some of the loops; taking advantage - * of alignment, using a bigger buffer, and so on. Not till after 0.1.2.x, - * though. */ - int c = cipher->pos; - if (PREDICT_UNLIKELY(!len)) return; - - while (1) { - do { - if (len-- == 0) { cipher->pos = c; return; } - *(data++) ^= cipher->buf[c]; - } while (++c != 16); - cipher->pos = c = 0; - if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 0))) { - if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 1))) { - if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 2))) { - ++COUNTER(cipher, 3); - UPDATE_CTR_BUF(cipher, 3); +#ifdef CAN_USE_OPENSSL_CTR + if (should_use_openssl_CTR) { + aes_crypt(cipher, data, len, data); + return; + } + else +#endif + { + int c = cipher->pos; + if (PREDICT_UNLIKELY(!len)) return; + + while (1) { + do { + if (len-- == 0) { cipher->pos = c; return; } + *(data++) ^= cipher->buf[c]; + } while (++c != 16); + cipher->pos = c = 0; + if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 0))) { + if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 1))) { + if (PREDICT_UNLIKELY(! ++COUNTER(cipher, 2))) { + ++COUNTER(cipher, 3); + UPDATE_CTR_BUF(cipher, 3); + } + UPDATE_CTR_BUF(cipher, 2); } - UPDATE_CTR_BUF(cipher, 2); + UPDATE_CTR_BUF(cipher, 1); } - UPDATE_CTR_BUF(cipher, 1); + UPDATE_CTR_BUF(cipher, 0); + _aes_fill_buf(cipher); } - UPDATE_CTR_BUF(cipher, 0); - _aes_fill_buf(cipher); } } /** Reset the 128-bit counter of <b>cipher</b> to the 16-bit big-endian value * in <b>iv</b>. */ -void +static void aes_set_iv(aes_cnt_cipher_t *cipher, const char *iv) { #ifdef USING_COUNTER_VARS @@ -368,705 +510,12 @@ aes_set_iv(aes_cnt_cipher_t *cipher, const char *iv) cipher->counter0 = ntohl(get_uint32(iv+12)); #endif cipher->pos = 0; -#ifndef USE_RIJNDAEL_COUNTER_OPTIMIZATION memcpy(cipher->ctr_buf.buf, iv, 16); -#endif - - _aes_fill_buf(cipher); -} - -#ifdef USE_BUILTIN_AES -/*======================================================================*/ -/* From rijndael-alg-fst.c */ - -/** - * rijndael-alg-fst.c - * - * @version 3.0 (December 2000) - * - * Optimized ANSI C code for the Rijndael cipher (now AES) - * - * @author Vincent Rijmen <vincent.rijmen@esat.kuleuven.ac.be> - * @author Antoon Bosselaers <antoon.bosselaers@esat.kuleuven.ac.be> - * @author Paulo Barreto <paulo.barreto@terra.com.br> - * - * This code is hereby placed in the public domain. - * - * THIS SOFTWARE IS PROVIDED BY THE AUTHORS ''AS IS'' AND ANY EXPRESS - * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR - * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, - * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE - * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, - * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -/* -Te0[x] = S [x].[02, 01, 01, 03]; -Te1[x] = S [x].[03, 02, 01, 01]; -Te2[x] = S [x].[01, 03, 02, 01]; -Te3[x] = S [x].[01, 01, 03, 02]; -Te4[x] = S [x].[01, 01, 01, 01]; - -Td0[x] = Si[x].[0e, 09, 0d, 0b]; -Td1[x] = Si[x].[0b, 0e, 09, 0d]; -Td2[x] = Si[x].[0d, 0b, 0e, 09]; -Td3[x] = Si[x].[09, 0d, 0b, 0e]; -Td4[x] = Si[x].[01, 01, 01, 01]; -*/ - -static const u32 Te0[256] = { - 0xc66363a5U, 0xf87c7c84U, 0xee777799U, 0xf67b7b8dU, - 0xfff2f20dU, 0xd66b6bbdU, 0xde6f6fb1U, 0x91c5c554U, - 0x60303050U, 0x02010103U, 0xce6767a9U, 0x562b2b7dU, - 0xe7fefe19U, 0xb5d7d762U, 0x4dababe6U, 0xec76769aU, - 0x8fcaca45U, 0x1f82829dU, 0x89c9c940U, 0xfa7d7d87U, - 0xeffafa15U, 0xb25959ebU, 0x8e4747c9U, 0xfbf0f00bU, - 0x41adadecU, 0xb3d4d467U, 0x5fa2a2fdU, 0x45afafeaU, - 0x239c9cbfU, 0x53a4a4f7U, 0xe4727296U, 0x9bc0c05bU, - 0x75b7b7c2U, 0xe1fdfd1cU, 0x3d9393aeU, 0x4c26266aU, - 0x6c36365aU, 0x7e3f3f41U, 0xf5f7f702U, 0x83cccc4fU, - 0x6834345cU, 0x51a5a5f4U, 0xd1e5e534U, 0xf9f1f108U, - 0xe2717193U, 0xabd8d873U, 0x62313153U, 0x2a15153fU, - 0x0804040cU, 0x95c7c752U, 0x46232365U, 0x9dc3c35eU, - 0x30181828U, 0x379696a1U, 0x0a05050fU, 0x2f9a9ab5U, - 0x0e070709U, 0x24121236U, 0x1b80809bU, 0xdfe2e23dU, - 0xcdebeb26U, 0x4e272769U, 0x7fb2b2cdU, 0xea75759fU, - 0x1209091bU, 0x1d83839eU, 0x582c2c74U, 0x341a1a2eU, - 0x361b1b2dU, 0xdc6e6eb2U, 0xb45a5aeeU, 0x5ba0a0fbU, - 0xa45252f6U, 0x763b3b4dU, 0xb7d6d661U, 0x7db3b3ceU, - 0x5229297bU, 0xdde3e33eU, 0x5e2f2f71U, 0x13848497U, - 0xa65353f5U, 0xb9d1d168U, 0x00000000U, 0xc1eded2cU, - 0x40202060U, 0xe3fcfc1fU, 0x79b1b1c8U, 0xb65b5bedU, - 0xd46a6abeU, 0x8dcbcb46U, 0x67bebed9U, 0x7239394bU, - 0x944a4adeU, 0x984c4cd4U, 0xb05858e8U, 0x85cfcf4aU, - 0xbbd0d06bU, 0xc5efef2aU, 0x4faaaae5U, 0xedfbfb16U, - 0x864343c5U, 0x9a4d4dd7U, 0x66333355U, 0x11858594U, - 0x8a4545cfU, 0xe9f9f910U, 0x04020206U, 0xfe7f7f81U, - 0xa05050f0U, 0x783c3c44U, 0x259f9fbaU, 0x4ba8a8e3U, - 0xa25151f3U, 0x5da3a3feU, 0x804040c0U, 0x058f8f8aU, - 0x3f9292adU, 0x219d9dbcU, 0x70383848U, 0xf1f5f504U, - 0x63bcbcdfU, 0x77b6b6c1U, 0xafdada75U, 0x42212163U, - 0x20101030U, 0xe5ffff1aU, 0xfdf3f30eU, 0xbfd2d26dU, - 0x81cdcd4cU, 0x180c0c14U, 0x26131335U, 0xc3ecec2fU, - 0xbe5f5fe1U, 0x359797a2U, 0x884444ccU, 0x2e171739U, - 0x93c4c457U, 0x55a7a7f2U, 0xfc7e7e82U, 0x7a3d3d47U, - 0xc86464acU, 0xba5d5de7U, 0x3219192bU, 0xe6737395U, - 0xc06060a0U, 0x19818198U, 0x9e4f4fd1U, 0xa3dcdc7fU, - 0x44222266U, 0x542a2a7eU, 0x3b9090abU, 0x0b888883U, - 0x8c4646caU, 0xc7eeee29U, 0x6bb8b8d3U, 0x2814143cU, - 0xa7dede79U, 0xbc5e5ee2U, 0x160b0b1dU, 0xaddbdb76U, - 0xdbe0e03bU, 0x64323256U, 0x743a3a4eU, 0x140a0a1eU, - 0x924949dbU, 0x0c06060aU, 0x4824246cU, 0xb85c5ce4U, - 0x9fc2c25dU, 0xbdd3d36eU, 0x43acacefU, 0xc46262a6U, - 0x399191a8U, 0x319595a4U, 0xd3e4e437U, 0xf279798bU, - 0xd5e7e732U, 0x8bc8c843U, 0x6e373759U, 0xda6d6db7U, - 0x018d8d8cU, 0xb1d5d564U, 0x9c4e4ed2U, 0x49a9a9e0U, - 0xd86c6cb4U, 0xac5656faU, 0xf3f4f407U, 0xcfeaea25U, - 0xca6565afU, 0xf47a7a8eU, 0x47aeaee9U, 0x10080818U, - 0x6fbabad5U, 0xf0787888U, 0x4a25256fU, 0x5c2e2e72U, - 0x381c1c24U, 0x57a6a6f1U, 0x73b4b4c7U, 0x97c6c651U, - 0xcbe8e823U, 0xa1dddd7cU, 0xe874749cU, 0x3e1f1f21U, - 0x964b4bddU, 0x61bdbddcU, 0x0d8b8b86U, 0x0f8a8a85U, - 0xe0707090U, 0x7c3e3e42U, 0x71b5b5c4U, 0xcc6666aaU, - 0x904848d8U, 0x06030305U, 0xf7f6f601U, 0x1c0e0e12U, - 0xc26161a3U, 0x6a35355fU, 0xae5757f9U, 0x69b9b9d0U, - 0x17868691U, 0x99c1c158U, 0x3a1d1d27U, 0x279e9eb9U, - 0xd9e1e138U, 0xebf8f813U, 0x2b9898b3U, 0x22111133U, - 0xd26969bbU, 0xa9d9d970U, 0x078e8e89U, 0x339494a7U, - 0x2d9b9bb6U, 0x3c1e1e22U, 0x15878792U, 0xc9e9e920U, - 0x87cece49U, 0xaa5555ffU, 0x50282878U, 0xa5dfdf7aU, - 0x038c8c8fU, 0x59a1a1f8U, 0x09898980U, 0x1a0d0d17U, - 0x65bfbfdaU, 0xd7e6e631U, 0x844242c6U, 0xd06868b8U, - 0x824141c3U, 0x299999b0U, 0x5a2d2d77U, 0x1e0f0f11U, - 0x7bb0b0cbU, 0xa85454fcU, 0x6dbbbbd6U, 0x2c16163aU, -}; -static const u32 Te1[256] = { - 0xa5c66363U, 0x84f87c7cU, 0x99ee7777U, 0x8df67b7bU, - 0x0dfff2f2U, 0xbdd66b6bU, 0xb1de6f6fU, 0x5491c5c5U, - 0x50603030U, 0x03020101U, 0xa9ce6767U, 0x7d562b2bU, - 0x19e7fefeU, 0x62b5d7d7U, 0xe64dababU, 0x9aec7676U, - 0x458fcacaU, 0x9d1f8282U, 0x4089c9c9U, 0x87fa7d7dU, - 0x15effafaU, 0xebb25959U, 0xc98e4747U, 0x0bfbf0f0U, - 0xec41adadU, 0x67b3d4d4U, 0xfd5fa2a2U, 0xea45afafU, - 0xbf239c9cU, 0xf753a4a4U, 0x96e47272U, 0x5b9bc0c0U, - 0xc275b7b7U, 0x1ce1fdfdU, 0xae3d9393U, 0x6a4c2626U, - 0x5a6c3636U, 0x417e3f3fU, 0x02f5f7f7U, 0x4f83ccccU, - 0x5c683434U, 0xf451a5a5U, 0x34d1e5e5U, 0x08f9f1f1U, - 0x93e27171U, 0x73abd8d8U, 0x53623131U, 0x3f2a1515U, - 0x0c080404U, 0x5295c7c7U, 0x65462323U, 0x5e9dc3c3U, - 0x28301818U, 0xa1379696U, 0x0f0a0505U, 0xb52f9a9aU, - 0x090e0707U, 0x36241212U, 0x9b1b8080U, 0x3ddfe2e2U, - 0x26cdebebU, 0x694e2727U, 0xcd7fb2b2U, 0x9fea7575U, - 0x1b120909U, 0x9e1d8383U, 0x74582c2cU, 0x2e341a1aU, - 0x2d361b1bU, 0xb2dc6e6eU, 0xeeb45a5aU, 0xfb5ba0a0U, - 0xf6a45252U, 0x4d763b3bU, 0x61b7d6d6U, 0xce7db3b3U, - 0x7b522929U, 0x3edde3e3U, 0x715e2f2fU, 0x97138484U, - 0xf5a65353U, 0x68b9d1d1U, 0x00000000U, 0x2cc1ededU, - 0x60402020U, 0x1fe3fcfcU, 0xc879b1b1U, 0xedb65b5bU, - 0xbed46a6aU, 0x468dcbcbU, 0xd967bebeU, 0x4b723939U, - 0xde944a4aU, 0xd4984c4cU, 0xe8b05858U, 0x4a85cfcfU, - 0x6bbbd0d0U, 0x2ac5efefU, 0xe54faaaaU, 0x16edfbfbU, - 0xc5864343U, 0xd79a4d4dU, 0x55663333U, 0x94118585U, - 0xcf8a4545U, 0x10e9f9f9U, 0x06040202U, 0x81fe7f7fU, - 0xf0a05050U, 0x44783c3cU, 0xba259f9fU, 0xe34ba8a8U, - 0xf3a25151U, 0xfe5da3a3U, 0xc0804040U, 0x8a058f8fU, - 0xad3f9292U, 0xbc219d9dU, 0x48703838U, 0x04f1f5f5U, - 0xdf63bcbcU, 0xc177b6b6U, 0x75afdadaU, 0x63422121U, - 0x30201010U, 0x1ae5ffffU, 0x0efdf3f3U, 0x6dbfd2d2U, - 0x4c81cdcdU, 0x14180c0cU, 0x35261313U, 0x2fc3ececU, - 0xe1be5f5fU, 0xa2359797U, 0xcc884444U, 0x392e1717U, - 0x5793c4c4U, 0xf255a7a7U, 0x82fc7e7eU, 0x477a3d3dU, - 0xacc86464U, 0xe7ba5d5dU, 0x2b321919U, 0x95e67373U, - 0xa0c06060U, 0x98198181U, 0xd19e4f4fU, 0x7fa3dcdcU, - 0x66442222U, 0x7e542a2aU, 0xab3b9090U, 0x830b8888U, - 0xca8c4646U, 0x29c7eeeeU, 0xd36bb8b8U, 0x3c281414U, - 0x79a7dedeU, 0xe2bc5e5eU, 0x1d160b0bU, 0x76addbdbU, - 0x3bdbe0e0U, 0x56643232U, 0x4e743a3aU, 0x1e140a0aU, - 0xdb924949U, 0x0a0c0606U, 0x6c482424U, 0xe4b85c5cU, - 0x5d9fc2c2U, 0x6ebdd3d3U, 0xef43acacU, 0xa6c46262U, - 0xa8399191U, 0xa4319595U, 0x37d3e4e4U, 0x8bf27979U, - 0x32d5e7e7U, 0x438bc8c8U, 0x596e3737U, 0xb7da6d6dU, - 0x8c018d8dU, 0x64b1d5d5U, 0xd29c4e4eU, 0xe049a9a9U, - 0xb4d86c6cU, 0xfaac5656U, 0x07f3f4f4U, 0x25cfeaeaU, - 0xafca6565U, 0x8ef47a7aU, 0xe947aeaeU, 0x18100808U, - 0xd56fbabaU, 0x88f07878U, 0x6f4a2525U, 0x725c2e2eU, - 0x24381c1cU, 0xf157a6a6U, 0xc773b4b4U, 0x5197c6c6U, - 0x23cbe8e8U, 0x7ca1ddddU, 0x9ce87474U, 0x213e1f1fU, - 0xdd964b4bU, 0xdc61bdbdU, 0x860d8b8bU, 0x850f8a8aU, - 0x90e07070U, 0x427c3e3eU, 0xc471b5b5U, 0xaacc6666U, - 0xd8904848U, 0x05060303U, 0x01f7f6f6U, 0x121c0e0eU, - 0xa3c26161U, 0x5f6a3535U, 0xf9ae5757U, 0xd069b9b9U, - 0x91178686U, 0x5899c1c1U, 0x273a1d1dU, 0xb9279e9eU, - 0x38d9e1e1U, 0x13ebf8f8U, 0xb32b9898U, 0x33221111U, - 0xbbd26969U, 0x70a9d9d9U, 0x89078e8eU, 0xa7339494U, - 0xb62d9b9bU, 0x223c1e1eU, 0x92158787U, 0x20c9e9e9U, - 0x4987ceceU, 0xffaa5555U, 0x78502828U, 0x7aa5dfdfU, - 0x8f038c8cU, 0xf859a1a1U, 0x80098989U, 0x171a0d0dU, - 0xda65bfbfU, 0x31d7e6e6U, 0xc6844242U, 0xb8d06868U, - 0xc3824141U, 0xb0299999U, 0x775a2d2dU, 0x111e0f0fU, - 0xcb7bb0b0U, 0xfca85454U, 0xd66dbbbbU, 0x3a2c1616U, -}; -static const u32 Te2[256] = { - 0x63a5c663U, 0x7c84f87cU, 0x7799ee77U, 0x7b8df67bU, - 0xf20dfff2U, 0x6bbdd66bU, 0x6fb1de6fU, 0xc55491c5U, - 0x30506030U, 0x01030201U, 0x67a9ce67U, 0x2b7d562bU, - 0xfe19e7feU, 0xd762b5d7U, 0xabe64dabU, 0x769aec76U, - 0xca458fcaU, 0x829d1f82U, 0xc94089c9U, 0x7d87fa7dU, - 0xfa15effaU, 0x59ebb259U, 0x47c98e47U, 0xf00bfbf0U, - 0xadec41adU, 0xd467b3d4U, 0xa2fd5fa2U, 0xafea45afU, - 0x9cbf239cU, 0xa4f753a4U, 0x7296e472U, 0xc05b9bc0U, - 0xb7c275b7U, 0xfd1ce1fdU, 0x93ae3d93U, 0x266a4c26U, - 0x365a6c36U, 0x3f417e3fU, 0xf702f5f7U, 0xcc4f83ccU, - 0x345c6834U, 0xa5f451a5U, 0xe534d1e5U, 0xf108f9f1U, - 0x7193e271U, 0xd873abd8U, 0x31536231U, 0x153f2a15U, - 0x040c0804U, 0xc75295c7U, 0x23654623U, 0xc35e9dc3U, - 0x18283018U, 0x96a13796U, 0x050f0a05U, 0x9ab52f9aU, - 0x07090e07U, 0x12362412U, 0x809b1b80U, 0xe23ddfe2U, - 0xeb26cdebU, 0x27694e27U, 0xb2cd7fb2U, 0x759fea75U, - 0x091b1209U, 0x839e1d83U, 0x2c74582cU, 0x1a2e341aU, - 0x1b2d361bU, 0x6eb2dc6eU, 0x5aeeb45aU, 0xa0fb5ba0U, - 0x52f6a452U, 0x3b4d763bU, 0xd661b7d6U, 0xb3ce7db3U, - 0x297b5229U, 0xe33edde3U, 0x2f715e2fU, 0x84971384U, - 0x53f5a653U, 0xd168b9d1U, 0x00000000U, 0xed2cc1edU, - 0x20604020U, 0xfc1fe3fcU, 0xb1c879b1U, 0x5bedb65bU, - 0x6abed46aU, 0xcb468dcbU, 0xbed967beU, 0x394b7239U, - 0x4ade944aU, 0x4cd4984cU, 0x58e8b058U, 0xcf4a85cfU, - 0xd06bbbd0U, 0xef2ac5efU, 0xaae54faaU, 0xfb16edfbU, - 0x43c58643U, 0x4dd79a4dU, 0x33556633U, 0x85941185U, - 0x45cf8a45U, 0xf910e9f9U, 0x02060402U, 0x7f81fe7fU, - 0x50f0a050U, 0x3c44783cU, 0x9fba259fU, 0xa8e34ba8U, - 0x51f3a251U, 0xa3fe5da3U, 0x40c08040U, 0x8f8a058fU, - 0x92ad3f92U, 0x9dbc219dU, 0x38487038U, 0xf504f1f5U, - 0xbcdf63bcU, 0xb6c177b6U, 0xda75afdaU, 0x21634221U, - 0x10302010U, 0xff1ae5ffU, 0xf30efdf3U, 0xd26dbfd2U, - 0xcd4c81cdU, 0x0c14180cU, 0x13352613U, 0xec2fc3ecU, - 0x5fe1be5fU, 0x97a23597U, 0x44cc8844U, 0x17392e17U, - 0xc45793c4U, 0xa7f255a7U, 0x7e82fc7eU, 0x3d477a3dU, - 0x64acc864U, 0x5de7ba5dU, 0x192b3219U, 0x7395e673U, - 0x60a0c060U, 0x81981981U, 0x4fd19e4fU, 0xdc7fa3dcU, - 0x22664422U, 0x2a7e542aU, 0x90ab3b90U, 0x88830b88U, - 0x46ca8c46U, 0xee29c7eeU, 0xb8d36bb8U, 0x143c2814U, - 0xde79a7deU, 0x5ee2bc5eU, 0x0b1d160bU, 0xdb76addbU, - 0xe03bdbe0U, 0x32566432U, 0x3a4e743aU, 0x0a1e140aU, - 0x49db9249U, 0x060a0c06U, 0x246c4824U, 0x5ce4b85cU, - 0xc25d9fc2U, 0xd36ebdd3U, 0xacef43acU, 0x62a6c462U, - 0x91a83991U, 0x95a43195U, 0xe437d3e4U, 0x798bf279U, - 0xe732d5e7U, 0xc8438bc8U, 0x37596e37U, 0x6db7da6dU, - 0x8d8c018dU, 0xd564b1d5U, 0x4ed29c4eU, 0xa9e049a9U, - 0x6cb4d86cU, 0x56faac56U, 0xf407f3f4U, 0xea25cfeaU, - 0x65afca65U, 0x7a8ef47aU, 0xaee947aeU, 0x08181008U, - 0xbad56fbaU, 0x7888f078U, 0x256f4a25U, 0x2e725c2eU, - 0x1c24381cU, 0xa6f157a6U, 0xb4c773b4U, 0xc65197c6U, - 0xe823cbe8U, 0xdd7ca1ddU, 0x749ce874U, 0x1f213e1fU, - 0x4bdd964bU, 0xbddc61bdU, 0x8b860d8bU, 0x8a850f8aU, - 0x7090e070U, 0x3e427c3eU, 0xb5c471b5U, 0x66aacc66U, - 0x48d89048U, 0x03050603U, 0xf601f7f6U, 0x0e121c0eU, - 0x61a3c261U, 0x355f6a35U, 0x57f9ae57U, 0xb9d069b9U, - 0x86911786U, 0xc15899c1U, 0x1d273a1dU, 0x9eb9279eU, - 0xe138d9e1U, 0xf813ebf8U, 0x98b32b98U, 0x11332211U, - 0x69bbd269U, 0xd970a9d9U, 0x8e89078eU, 0x94a73394U, - 0x9bb62d9bU, 0x1e223c1eU, 0x87921587U, 0xe920c9e9U, - 0xce4987ceU, 0x55ffaa55U, 0x28785028U, 0xdf7aa5dfU, - 0x8c8f038cU, 0xa1f859a1U, 0x89800989U, 0x0d171a0dU, - 0xbfda65bfU, 0xe631d7e6U, 0x42c68442U, 0x68b8d068U, - 0x41c38241U, 0x99b02999U, 0x2d775a2dU, 0x0f111e0fU, - 0xb0cb7bb0U, 0x54fca854U, 0xbbd66dbbU, 0x163a2c16U, -}; -static const u32 Te3[256] = { - - 0x6363a5c6U, 0x7c7c84f8U, 0x777799eeU, 0x7b7b8df6U, - 0xf2f20dffU, 0x6b6bbdd6U, 0x6f6fb1deU, 0xc5c55491U, - 0x30305060U, 0x01010302U, 0x6767a9ceU, 0x2b2b7d56U, - 0xfefe19e7U, 0xd7d762b5U, 0xababe64dU, 0x76769aecU, - 0xcaca458fU, 0x82829d1fU, 0xc9c94089U, 0x7d7d87faU, - 0xfafa15efU, 0x5959ebb2U, 0x4747c98eU, 0xf0f00bfbU, - 0xadadec41U, 0xd4d467b3U, 0xa2a2fd5fU, 0xafafea45U, - 0x9c9cbf23U, 0xa4a4f753U, 0x727296e4U, 0xc0c05b9bU, - 0xb7b7c275U, 0xfdfd1ce1U, 0x9393ae3dU, 0x26266a4cU, - 0x36365a6cU, 0x3f3f417eU, 0xf7f702f5U, 0xcccc4f83U, - 0x34345c68U, 0xa5a5f451U, 0xe5e534d1U, 0xf1f108f9U, - 0x717193e2U, 0xd8d873abU, 0x31315362U, 0x15153f2aU, - 0x04040c08U, 0xc7c75295U, 0x23236546U, 0xc3c35e9dU, - 0x18182830U, 0x9696a137U, 0x05050f0aU, 0x9a9ab52fU, - 0x0707090eU, 0x12123624U, 0x80809b1bU, 0xe2e23ddfU, - 0xebeb26cdU, 0x2727694eU, 0xb2b2cd7fU, 0x75759feaU, - 0x09091b12U, 0x83839e1dU, 0x2c2c7458U, 0x1a1a2e34U, - 0x1b1b2d36U, 0x6e6eb2dcU, 0x5a5aeeb4U, 0xa0a0fb5bU, - 0x5252f6a4U, 0x3b3b4d76U, 0xd6d661b7U, 0xb3b3ce7dU, - 0x29297b52U, 0xe3e33eddU, 0x2f2f715eU, 0x84849713U, - 0x5353f5a6U, 0xd1d168b9U, 0x00000000U, 0xeded2cc1U, - 0x20206040U, 0xfcfc1fe3U, 0xb1b1c879U, 0x5b5bedb6U, - 0x6a6abed4U, 0xcbcb468dU, 0xbebed967U, 0x39394b72U, - 0x4a4ade94U, 0x4c4cd498U, 0x5858e8b0U, 0xcfcf4a85U, - 0xd0d06bbbU, 0xefef2ac5U, 0xaaaae54fU, 0xfbfb16edU, - 0x4343c586U, 0x4d4dd79aU, 0x33335566U, 0x85859411U, - 0x4545cf8aU, 0xf9f910e9U, 0x02020604U, 0x7f7f81feU, - 0x5050f0a0U, 0x3c3c4478U, 0x9f9fba25U, 0xa8a8e34bU, - 0x5151f3a2U, 0xa3a3fe5dU, 0x4040c080U, 0x8f8f8a05U, - 0x9292ad3fU, 0x9d9dbc21U, 0x38384870U, 0xf5f504f1U, - 0xbcbcdf63U, 0xb6b6c177U, 0xdada75afU, 0x21216342U, - 0x10103020U, 0xffff1ae5U, 0xf3f30efdU, 0xd2d26dbfU, - 0xcdcd4c81U, 0x0c0c1418U, 0x13133526U, 0xecec2fc3U, - 0x5f5fe1beU, 0x9797a235U, 0x4444cc88U, 0x1717392eU, - 0xc4c45793U, 0xa7a7f255U, 0x7e7e82fcU, 0x3d3d477aU, - 0x6464acc8U, 0x5d5de7baU, 0x19192b32U, 0x737395e6U, - 0x6060a0c0U, 0x81819819U, 0x4f4fd19eU, 0xdcdc7fa3U, - 0x22226644U, 0x2a2a7e54U, 0x9090ab3bU, 0x8888830bU, - 0x4646ca8cU, 0xeeee29c7U, 0xb8b8d36bU, 0x14143c28U, - 0xdede79a7U, 0x5e5ee2bcU, 0x0b0b1d16U, 0xdbdb76adU, - 0xe0e03bdbU, 0x32325664U, 0x3a3a4e74U, 0x0a0a1e14U, - 0x4949db92U, 0x06060a0cU, 0x24246c48U, 0x5c5ce4b8U, - 0xc2c25d9fU, 0xd3d36ebdU, 0xacacef43U, 0x6262a6c4U, - 0x9191a839U, 0x9595a431U, 0xe4e437d3U, 0x79798bf2U, - 0xe7e732d5U, 0xc8c8438bU, 0x3737596eU, 0x6d6db7daU, - 0x8d8d8c01U, 0xd5d564b1U, 0x4e4ed29cU, 0xa9a9e049U, - 0x6c6cb4d8U, 0x5656faacU, 0xf4f407f3U, 0xeaea25cfU, - 0x6565afcaU, 0x7a7a8ef4U, 0xaeaee947U, 0x08081810U, - 0xbabad56fU, 0x787888f0U, 0x25256f4aU, 0x2e2e725cU, - 0x1c1c2438U, 0xa6a6f157U, 0xb4b4c773U, 0xc6c65197U, - 0xe8e823cbU, 0xdddd7ca1U, 0x74749ce8U, 0x1f1f213eU, - 0x4b4bdd96U, 0xbdbddc61U, 0x8b8b860dU, 0x8a8a850fU, - 0x707090e0U, 0x3e3e427cU, 0xb5b5c471U, 0x6666aaccU, - 0x4848d890U, 0x03030506U, 0xf6f601f7U, 0x0e0e121cU, - 0x6161a3c2U, 0x35355f6aU, 0x5757f9aeU, 0xb9b9d069U, - 0x86869117U, 0xc1c15899U, 0x1d1d273aU, 0x9e9eb927U, - 0xe1e138d9U, 0xf8f813ebU, 0x9898b32bU, 0x11113322U, - 0x6969bbd2U, 0xd9d970a9U, 0x8e8e8907U, 0x9494a733U, - 0x9b9bb62dU, 0x1e1e223cU, 0x87879215U, 0xe9e920c9U, - 0xcece4987U, 0x5555ffaaU, 0x28287850U, 0xdfdf7aa5U, - 0x8c8c8f03U, 0xa1a1f859U, 0x89898009U, 0x0d0d171aU, - 0xbfbfda65U, 0xe6e631d7U, 0x4242c684U, 0x6868b8d0U, - 0x4141c382U, 0x9999b029U, 0x2d2d775aU, 0x0f0f111eU, - 0xb0b0cb7bU, 0x5454fca8U, 0xbbbbd66dU, 0x16163a2cU, -}; -static const u32 Te4[256] = { - 0x63636363U, 0x7c7c7c7cU, 0x77777777U, 0x7b7b7b7bU, - 0xf2f2f2f2U, 0x6b6b6b6bU, 0x6f6f6f6fU, 0xc5c5c5c5U, - 0x30303030U, 0x01010101U, 0x67676767U, 0x2b2b2b2bU, - 0xfefefefeU, 0xd7d7d7d7U, 0xababababU, 0x76767676U, - 0xcacacacaU, 0x82828282U, 0xc9c9c9c9U, 0x7d7d7d7dU, - 0xfafafafaU, 0x59595959U, 0x47474747U, 0xf0f0f0f0U, - 0xadadadadU, 0xd4d4d4d4U, 0xa2a2a2a2U, 0xafafafafU, - 0x9c9c9c9cU, 0xa4a4a4a4U, 0x72727272U, 0xc0c0c0c0U, - 0xb7b7b7b7U, 0xfdfdfdfdU, 0x93939393U, 0x26262626U, - 0x36363636U, 0x3f3f3f3fU, 0xf7f7f7f7U, 0xccccccccU, - 0x34343434U, 0xa5a5a5a5U, 0xe5e5e5e5U, 0xf1f1f1f1U, - 0x71717171U, 0xd8d8d8d8U, 0x31313131U, 0x15151515U, - 0x04040404U, 0xc7c7c7c7U, 0x23232323U, 0xc3c3c3c3U, - 0x18181818U, 0x96969696U, 0x05050505U, 0x9a9a9a9aU, - 0x07070707U, 0x12121212U, 0x80808080U, 0xe2e2e2e2U, - 0xebebebebU, 0x27272727U, 0xb2b2b2b2U, 0x75757575U, - 0x09090909U, 0x83838383U, 0x2c2c2c2cU, 0x1a1a1a1aU, - 0x1b1b1b1bU, 0x6e6e6e6eU, 0x5a5a5a5aU, 0xa0a0a0a0U, - 0x52525252U, 0x3b3b3b3bU, 0xd6d6d6d6U, 0xb3b3b3b3U, - 0x29292929U, 0xe3e3e3e3U, 0x2f2f2f2fU, 0x84848484U, - 0x53535353U, 0xd1d1d1d1U, 0x00000000U, 0xededededU, - 0x20202020U, 0xfcfcfcfcU, 0xb1b1b1b1U, 0x5b5b5b5bU, - 0x6a6a6a6aU, 0xcbcbcbcbU, 0xbebebebeU, 0x39393939U, - 0x4a4a4a4aU, 0x4c4c4c4cU, 0x58585858U, 0xcfcfcfcfU, - 0xd0d0d0d0U, 0xefefefefU, 0xaaaaaaaaU, 0xfbfbfbfbU, - 0x43434343U, 0x4d4d4d4dU, 0x33333333U, 0x85858585U, - 0x45454545U, 0xf9f9f9f9U, 0x02020202U, 0x7f7f7f7fU, - 0x50505050U, 0x3c3c3c3cU, 0x9f9f9f9fU, 0xa8a8a8a8U, - 0x51515151U, 0xa3a3a3a3U, 0x40404040U, 0x8f8f8f8fU, - 0x92929292U, 0x9d9d9d9dU, 0x38383838U, 0xf5f5f5f5U, - 0xbcbcbcbcU, 0xb6b6b6b6U, 0xdadadadaU, 0x21212121U, - 0x10101010U, 0xffffffffU, 0xf3f3f3f3U, 0xd2d2d2d2U, - 0xcdcdcdcdU, 0x0c0c0c0cU, 0x13131313U, 0xececececU, - 0x5f5f5f5fU, 0x97979797U, 0x44444444U, 0x17171717U, - 0xc4c4c4c4U, 0xa7a7a7a7U, 0x7e7e7e7eU, 0x3d3d3d3dU, - 0x64646464U, 0x5d5d5d5dU, 0x19191919U, 0x73737373U, - 0x60606060U, 0x81818181U, 0x4f4f4f4fU, 0xdcdcdcdcU, - 0x22222222U, 0x2a2a2a2aU, 0x90909090U, 0x88888888U, - 0x46464646U, 0xeeeeeeeeU, 0xb8b8b8b8U, 0x14141414U, - 0xdedededeU, 0x5e5e5e5eU, 0x0b0b0b0bU, 0xdbdbdbdbU, - 0xe0e0e0e0U, 0x32323232U, 0x3a3a3a3aU, 0x0a0a0a0aU, - 0x49494949U, 0x06060606U, 0x24242424U, 0x5c5c5c5cU, - 0xc2c2c2c2U, 0xd3d3d3d3U, 0xacacacacU, 0x62626262U, - 0x91919191U, 0x95959595U, 0xe4e4e4e4U, 0x79797979U, - 0xe7e7e7e7U, 0xc8c8c8c8U, 0x37373737U, 0x6d6d6d6dU, - 0x8d8d8d8dU, 0xd5d5d5d5U, 0x4e4e4e4eU, 0xa9a9a9a9U, - 0x6c6c6c6cU, 0x56565656U, 0xf4f4f4f4U, 0xeaeaeaeaU, - 0x65656565U, 0x7a7a7a7aU, 0xaeaeaeaeU, 0x08080808U, - 0xbabababaU, 0x78787878U, 0x25252525U, 0x2e2e2e2eU, - 0x1c1c1c1cU, 0xa6a6a6a6U, 0xb4b4b4b4U, 0xc6c6c6c6U, - 0xe8e8e8e8U, 0xddddddddU, 0x74747474U, 0x1f1f1f1fU, - 0x4b4b4b4bU, 0xbdbdbdbdU, 0x8b8b8b8bU, 0x8a8a8a8aU, - 0x70707070U, 0x3e3e3e3eU, 0xb5b5b5b5U, 0x66666666U, - 0x48484848U, 0x03030303U, 0xf6f6f6f6U, 0x0e0e0e0eU, - 0x61616161U, 0x35353535U, 0x57575757U, 0xb9b9b9b9U, - 0x86868686U, 0xc1c1c1c1U, 0x1d1d1d1dU, 0x9e9e9e9eU, - 0xe1e1e1e1U, 0xf8f8f8f8U, 0x98989898U, 0x11111111U, - 0x69696969U, 0xd9d9d9d9U, 0x8e8e8e8eU, 0x94949494U, - 0x9b9b9b9bU, 0x1e1e1e1eU, 0x87878787U, 0xe9e9e9e9U, - 0xcecececeU, 0x55555555U, 0x28282828U, 0xdfdfdfdfU, - 0x8c8c8c8cU, 0xa1a1a1a1U, 0x89898989U, 0x0d0d0d0dU, - 0xbfbfbfbfU, 0xe6e6e6e6U, 0x42424242U, 0x68686868U, - 0x41414141U, 0x99999999U, 0x2d2d2d2dU, 0x0f0f0f0fU, - 0xb0b0b0b0U, 0x54545454U, 0xbbbbbbbbU, 0x16161616U, -}; - -static const u32 rcon[] = { - 0x01000000, 0x02000000, 0x04000000, 0x08000000, - 0x10000000, 0x20000000, 0x40000000, 0x80000000, - 0x1B000000, 0x36000000, /* for 128-bit blocks, Rijndael never uses more than 10 rcon values */ -}; - -#define SWAP(x) (_lrotl(x, 8) & 0x00ff00ff | _lrotr(x, 8) & 0xff00ff00) - -#ifdef _MSC_VER -#define GETU32(p) SWAP(*((u32 *)(p))) -#define PUTU32(ct, st) { *((u32 *)(ct)) = SWAP((st)); } -#else -#define GETU32(pt) (((u32)(pt)[0] << 24) ^ ((u32)(pt)[1] << 16) ^ ((u32)(pt)[2] << 8) ^ ((u32)(pt)[3])) -#define PUTU32(ct, st) { (ct)[0] = (u8)((st) >> 24); (ct)[1] = (u8)((st) >> 16); (ct)[2] = (u8)((st) >> 8); (ct)[3] = (u8)(st); } -#endif - -/** - * Expand the cipher key into the encryption key schedule. - * - * @return the number of rounds for the given cipher key size. - */ -static int -rijndaelKeySetupEnc(u32 rk[/*4*(Nr + 1)*/], const u8 cipherKey[], int keyBits) -{ - int i = 0; - u32 temp; - - rk[0] = GETU32(cipherKey ); - rk[1] = GETU32(cipherKey + 4); - rk[2] = GETU32(cipherKey + 8); - rk[3] = GETU32(cipherKey + 12); - if (keyBits == 128) { - for (;;) { - temp = rk[3]; - rk[4] = rk[0] ^ - (Te4[(temp >> 16) & 0xff] & 0xff000000) ^ - (Te4[(temp >> 8) & 0xff] & 0x00ff0000) ^ - (Te4[(temp ) & 0xff] & 0x0000ff00) ^ - (Te4[(temp >> 24) ] & 0x000000ff) ^ - rcon[i]; - rk[5] = rk[1] ^ rk[4]; - rk[6] = rk[2] ^ rk[5]; - rk[7] = rk[3] ^ rk[6]; - if (++i == 10) { - return 10; - } - rk += 4; - } - } - rk[4] = GETU32(cipherKey + 16); - rk[5] = GETU32(cipherKey + 20); - if (keyBits == 192) { - for (;;) { - temp = rk[ 5]; - rk[ 6] = rk[ 0] ^ - (Te4[(temp >> 16) & 0xff] & 0xff000000) ^ - (Te4[(temp >> 8) & 0xff] & 0x00ff0000) ^ - (Te4[(temp ) & 0xff] & 0x0000ff00) ^ - (Te4[(temp >> 24) ] & 0x000000ff) ^ - rcon[i]; - rk[ 7] = rk[ 1] ^ rk[ 6]; - rk[ 8] = rk[ 2] ^ rk[ 7]; - rk[ 9] = rk[ 3] ^ rk[ 8]; - if (++i == 8) { - return 12; - } - rk[10] = rk[ 4] ^ rk[ 9]; - rk[11] = rk[ 5] ^ rk[10]; - rk += 6; - } - } - rk[6] = GETU32(cipherKey + 24); - rk[7] = GETU32(cipherKey + 28); - if (keyBits == 256) { - for (;;) { - temp = rk[ 7]; - rk[ 8] = rk[ 0] ^ - (Te4[(temp >> 16) & 0xff] & 0xff000000) ^ - (Te4[(temp >> 8) & 0xff] & 0x00ff0000) ^ - (Te4[(temp ) & 0xff] & 0x0000ff00) ^ - (Te4[(temp >> 24) ] & 0x000000ff) ^ - rcon[i]; - rk[ 9] = rk[ 1] ^ rk[ 8]; - rk[10] = rk[ 2] ^ rk[ 9]; - rk[11] = rk[ 3] ^ rk[10]; - if (++i == 7) { - return 14; - } - temp = rk[11]; - rk[12] = rk[ 4] ^ - (Te4[(temp >> 24) ] & 0xff000000) ^ - (Te4[(temp >> 16) & 0xff] & 0x00ff0000) ^ - (Te4[(temp >> 8) & 0xff] & 0x0000ff00) ^ - (Te4[(temp ) & 0xff] & 0x000000ff); - rk[13] = rk[ 5] ^ rk[12]; - rk[14] = rk[ 6] ^ rk[13]; - rk[15] = rk[ 7] ^ rk[14]; - - rk += 8; - } - } - return 0; -} -#ifdef USE_RIJNDAEL_COUNTER_OPTIMIZATION -static void -rijndaelEncrypt(const u32 rk[/*4*(Nr + 1)*/], int Nr, u32 ctr3, u32 ctr2, u32 ctr1, u32 ctr0, u8 ct[16]) -#else -static void -rijndaelEncrypt(const u32 rk[/*4*(Nr + 1)*/], int Nr, const u8 pt[16], u8 ct[16]) -#endif -{ - u32 s0, s1, s2, s3, t0, t1, t2, t3; -#ifndef FULL_UNROLL - int r; -#endif /* ?FULL_UNROLL */ - - /* - * map byte array block to cipher state - * and add initial round key: - */ -#ifdef USE_RIJNDAEL_COUNTER_OPTIMIZATION - s0 = ctr3 ^ rk[0]; - s1 = ctr2 ^ rk[1]; - s2 = ctr1 ^ rk[2]; - s3 = ctr0 ^ rk[3]; -#else - s0 = GETU32(pt ) ^ rk[0]; - s1 = GETU32(pt + 4) ^ rk[1]; - s2 = GETU32(pt + 8) ^ rk[2]; - s3 = GETU32(pt + 12) ^ rk[3]; +#ifdef CAN_USE_OPENSSL_CTR + if (!should_use_openssl_CTR) #endif - -#ifdef FULL_UNROLL - /* round 1: */ - t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[ 4]; - t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[ 5]; - t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[ 6]; - t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[ 7]; - /* round 2: */ - s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[ 8]; - s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[ 9]; - s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[10]; - s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[11]; - /* round 3: */ - t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[12]; - t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[13]; - t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[14]; - t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[15]; - /* round 4: */ - s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[16]; - s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[17]; - s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[18]; - s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[19]; - /* round 5: */ - t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[20]; - t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[21]; - t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[22]; - t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[23]; - /* round 6: */ - s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[24]; - s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[25]; - s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[26]; - s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[27]; - /* round 7: */ - t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[28]; - t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[29]; - t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[30]; - t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[31]; - /* round 8: */ - s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[32]; - s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[33]; - s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[34]; - s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[35]; - /* round 9: */ - t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[36]; - t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[37]; - t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[38]; - t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[39]; - if (Nr > 10) { - /* round 10: */ - s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[40]; - s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[41]; - s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[42]; - s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[43]; - /* round 11: */ - t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[44]; - t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[45]; - t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[46]; - t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[47]; - if (Nr > 12) { - /* round 12: */ - s0 = Te0[t0 >> 24] ^ Te1[(t1 >> 16) & 0xff] ^ Te2[(t2 >> 8) & 0xff] ^ Te3[t3 & 0xff] ^ rk[48]; - s1 = Te0[t1 >> 24] ^ Te1[(t2 >> 16) & 0xff] ^ Te2[(t3 >> 8) & 0xff] ^ Te3[t0 & 0xff] ^ rk[49]; - s2 = Te0[t2 >> 24] ^ Te1[(t3 >> 16) & 0xff] ^ Te2[(t0 >> 8) & 0xff] ^ Te3[t1 & 0xff] ^ rk[50]; - s3 = Te0[t3 >> 24] ^ Te1[(t0 >> 16) & 0xff] ^ Te2[(t1 >> 8) & 0xff] ^ Te3[t2 & 0xff] ^ rk[51]; - /* round 13: */ - t0 = Te0[s0 >> 24] ^ Te1[(s1 >> 16) & 0xff] ^ Te2[(s2 >> 8) & 0xff] ^ Te3[s3 & 0xff] ^ rk[52]; - t1 = Te0[s1 >> 24] ^ Te1[(s2 >> 16) & 0xff] ^ Te2[(s3 >> 8) & 0xff] ^ Te3[s0 & 0xff] ^ rk[53]; - t2 = Te0[s2 >> 24] ^ Te1[(s3 >> 16) & 0xff] ^ Te2[(s0 >> 8) & 0xff] ^ Te3[s1 & 0xff] ^ rk[54]; - t3 = Te0[s3 >> 24] ^ Te1[(s0 >> 16) & 0xff] ^ Te2[(s1 >> 8) & 0xff] ^ Te3[s2 & 0xff] ^ rk[55]; - } - } - rk += Nr << 2; -#else /* !FULL_UNROLL */ - /* - * Nr - 1 full rounds: - */ - r = Nr >> 1; - for (;;) { - t0 = - Te0[(s0 >> 24) ] ^ - Te1[(s1 >> 16) & 0xff] ^ - Te2[(s2 >> 8) & 0xff] ^ - Te3[(s3 ) & 0xff] ^ - rk[4]; - t1 = - Te0[(s1 >> 24) ] ^ - Te1[(s2 >> 16) & 0xff] ^ - Te2[(s3 >> 8) & 0xff] ^ - Te3[(s0 ) & 0xff] ^ - rk[5]; - t2 = - Te0[(s2 >> 24) ] ^ - Te1[(s3 >> 16) & 0xff] ^ - Te2[(s0 >> 8) & 0xff] ^ - Te3[(s1 ) & 0xff] ^ - rk[6]; - t3 = - Te0[(s3 >> 24) ] ^ - Te1[(s0 >> 16) & 0xff] ^ - Te2[(s1 >> 8) & 0xff] ^ - Te3[(s2 ) & 0xff] ^ - rk[7]; - - rk += 8; - if (--r == 0) { - break; - } - - s0 = - Te0[(t0 >> 24) ] ^ - Te1[(t1 >> 16) & 0xff] ^ - Te2[(t2 >> 8) & 0xff] ^ - Te3[(t3 ) & 0xff] ^ - rk[0]; - s1 = - Te0[(t1 >> 24) ] ^ - Te1[(t2 >> 16) & 0xff] ^ - Te2[(t3 >> 8) & 0xff] ^ - Te3[(t0 ) & 0xff] ^ - rk[1]; - s2 = - Te0[(t2 >> 24) ] ^ - Te1[(t3 >> 16) & 0xff] ^ - Te2[(t0 >> 8) & 0xff] ^ - Te3[(t1 ) & 0xff] ^ - rk[2]; - s3 = - Te0[(t3 >> 24) ] ^ - Te1[(t0 >> 16) & 0xff] ^ - Te2[(t1 >> 8) & 0xff] ^ - Te3[(t2 ) & 0xff] ^ - rk[3]; - } -#endif /* ?FULL_UNROLL */ - /* - * apply last round and - * map cipher state to byte array block: - */ - s0 = - (Te4[(t0 >> 24) ] & 0xff000000) ^ - (Te4[(t1 >> 16) & 0xff] & 0x00ff0000) ^ - (Te4[(t2 >> 8) & 0xff] & 0x0000ff00) ^ - (Te4[(t3 ) & 0xff] & 0x000000ff) ^ - rk[0]; - PUTU32(ct , s0); - s1 = - (Te4[(t1 >> 24) ] & 0xff000000) ^ - (Te4[(t2 >> 16) & 0xff] & 0x00ff0000) ^ - (Te4[(t3 >> 8) & 0xff] & 0x0000ff00) ^ - (Te4[(t0 ) & 0xff] & 0x000000ff) ^ - rk[1]; - PUTU32(ct + 4, s1); - s2 = - (Te4[(t2 >> 24) ] & 0xff000000) ^ - (Te4[(t3 >> 16) & 0xff] & 0x00ff0000) ^ - (Te4[(t0 >> 8) & 0xff] & 0x0000ff00) ^ - (Te4[(t1 ) & 0xff] & 0x000000ff) ^ - rk[2]; - PUTU32(ct + 8, s2); - s3 = - (Te4[(t3 >> 24) ] & 0xff000000) ^ - (Te4[(t0 >> 16) & 0xff] & 0x00ff0000) ^ - (Te4[(t1 >> 8) & 0xff] & 0x0000ff00) ^ - (Te4[(t2 ) & 0xff] & 0x000000ff) ^ - rk[3]; - PUTU32(ct + 12, s3); + _aes_fill_buf(cipher); } -#endif -#ifdef AES_BENCHMARK -int -main(int c, char **v) -{ - int i; - char blob[509]; /* the size of a cell payload. */ - char blob_out[509]; - aes_cnt_cipher_t *cipher = aes_new_cipher(); - aes_set_key(cipher, "aesbenchmarkkey!", 128); - memset(blob, 'z', sizeof(blob)); - - for (i=0;i<1000000; ++i) { - aes_crypt(cipher, blob, sizeof(blob), blob_out); - } - return 0; -} #endif - diff --git a/src/common/aes.h b/src/common/aes.h index eb633dbcce..bde567f87f 100644 --- a/src/common/aes.h +++ b/src/common/aes.h @@ -1,6 +1,6 @@ /* Copyright (c) 2003, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /* Implements a minimal interface to counter-mode AES. */ @@ -13,18 +13,17 @@ * \brief Headers for aes.c */ -#include "torint.h" - struct aes_cnt_cipher; typedef struct aes_cnt_cipher aes_cnt_cipher_t; -aes_cnt_cipher_t* aes_new_cipher(void); -void aes_free_cipher(aes_cnt_cipher_t *cipher); -void aes_set_key(aes_cnt_cipher_t *cipher, const char *key, int key_bits); +aes_cnt_cipher_t* aes_new_cipher(const char *key, const char *iv); +void aes_cipher_free(aes_cnt_cipher_t *cipher); void aes_crypt(aes_cnt_cipher_t *cipher, const char *input, size_t len, char *output); void aes_crypt_inplace(aes_cnt_cipher_t *cipher, char *data, size_t len); -void aes_set_iv(aes_cnt_cipher_t *cipher, const char *iv); + +int evaluate_evp_for_aes(int force_value); +int evaluate_ctr_for_aes(void); #endif diff --git a/src/common/ciphers.inc b/src/common/ciphers.inc index c84620d49e..137d78b117 100644 --- a/src/common/ciphers.inc +++ b/src/common/ciphers.inc @@ -1,6 +1,9 @@ /* This is an include file used to define the list of ciphers clients should * advertise. Before including it, you should define the CIPHER and XCIPHER - * macros. */ + * macros. + * + * This file was automatically generated by get_mozilla_ciphers.py. + */ #ifdef TLS1_TXT_ECDHE_ECDSA_WITH_AES_256_CBC_SHA CIPHER(0xc00a, TLS1_TXT_ECDHE_ECDSA_WITH_AES_256_CBC_SHA) #else @@ -11,6 +14,16 @@ #else XCIPHER(0xc014, TLS1_TXT_ECDHE_RSA_WITH_AES_256_CBC_SHA) #endif +#ifdef TLS1_TXT_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA + CIPHER(0x0088, TLS1_TXT_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA) +#else + XCIPHER(0x0088, TLS1_TXT_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA) +#endif +#ifdef TLS1_TXT_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA + CIPHER(0x0087, TLS1_TXT_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA) +#else + XCIPHER(0x0087, TLS1_TXT_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA) +#endif #ifdef TLS1_TXT_DHE_RSA_WITH_AES_256_SHA CIPHER(0x0039, TLS1_TXT_DHE_RSA_WITH_AES_256_SHA) #else @@ -31,6 +44,11 @@ #else XCIPHER(0xc005, TLS1_TXT_ECDH_ECDSA_WITH_AES_256_CBC_SHA) #endif +#ifdef TLS1_TXT_RSA_WITH_CAMELLIA_256_CBC_SHA + CIPHER(0x0084, TLS1_TXT_RSA_WITH_CAMELLIA_256_CBC_SHA) +#else + XCIPHER(0x0084, TLS1_TXT_RSA_WITH_CAMELLIA_256_CBC_SHA) +#endif #ifdef TLS1_TXT_RSA_WITH_AES_256_SHA CIPHER(0x0035, TLS1_TXT_RSA_WITH_AES_256_SHA) #else @@ -56,6 +74,16 @@ #else XCIPHER(0xc013, TLS1_TXT_ECDHE_RSA_WITH_AES_128_CBC_SHA) #endif +#ifdef TLS1_TXT_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA + CIPHER(0x0045, TLS1_TXT_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA) +#else + XCIPHER(0x0045, TLS1_TXT_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA) +#endif +#ifdef TLS1_TXT_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA + CIPHER(0x0044, TLS1_TXT_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA) +#else + XCIPHER(0x0044, TLS1_TXT_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA) +#endif #ifdef TLS1_TXT_DHE_RSA_WITH_AES_128_SHA CIPHER(0x0033, TLS1_TXT_DHE_RSA_WITH_AES_128_SHA) #else @@ -86,6 +114,16 @@ #else XCIPHER(0xc004, TLS1_TXT_ECDH_ECDSA_WITH_AES_128_CBC_SHA) #endif +#ifdef TLS1_TXT_RSA_WITH_SEED_SHA + CIPHER(0x0096, TLS1_TXT_RSA_WITH_SEED_SHA) +#else + XCIPHER(0x0096, TLS1_TXT_RSA_WITH_SEED_SHA) +#endif +#ifdef TLS1_TXT_RSA_WITH_CAMELLIA_128_CBC_SHA + CIPHER(0x0041, TLS1_TXT_RSA_WITH_CAMELLIA_128_CBC_SHA) +#else + XCIPHER(0x0041, TLS1_TXT_RSA_WITH_CAMELLIA_128_CBC_SHA) +#endif #ifdef SSL3_TXT_RSA_RC4_128_MD5 CIPHER(0x0004, SSL3_TXT_RSA_RC4_128_MD5) #else @@ -131,10 +169,11 @@ #else XCIPHER(0xc003, TLS1_TXT_ECDH_ECDSA_WITH_DES_192_CBC3_SHA) #endif -#ifdef SSL3_TXT_RSA_FIPS_WITH_3DES_EDE_CBC_SHA - CIPHER(0xfeff, SSL3_TXT_RSA_FIPS_WITH_3DES_EDE_CBC_SHA) +/* No openssl macro found for 0xfeff */ +#ifdef SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA + CIPHER(0xfeff, SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA) #else - XCIPHER(0xfeff, SSL3_TXT_RSA_FIPS_WITH_3DES_EDE_CBC_SHA) + XCIPHER(0xfeff, SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA) #endif #ifdef SSL3_TXT_RSA_DES_192_CBC3_SHA CIPHER(0x000a, SSL3_TXT_RSA_DES_192_CBC3_SHA) diff --git a/src/common/compat.c b/src/common/compat.c index a4e50747cd..59e3898deb 100644 --- a/src/common/compat.c +++ b/src/common/compat.c @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -15,13 +15,17 @@ /* This is required on rh7 to make strptime not complain. * We also need it to make memmem get defined (where available) */ -/* XXXX023 We should just use AC_USE_SYSTEM_EXTENSIONS in our autoconf, - * and get this (and other important stuff!) automatically */ +/* XXXX024 We should just use AC_USE_SYSTEM_EXTENSIONS in our autoconf, + * and get this (and other important stuff!) automatically. Once we do that, + * make sure to also change the extern char **environ detection in + * configure.in, because whether that is declared or not depends on whether + * we have _GNU_SOURCE defined! Maybe that means that once we take this out, + * we can also take out the configure check. */ #define _GNU_SOURCE #include "compat.h" -#ifdef MS_WINDOWS +#ifdef _WIN32 #include <process.h> #include <windows.h> #include <sys/locking.h> @@ -51,6 +55,9 @@ #ifdef HAVE_ARPA_INET_H #include <arpa/inet.h> #endif +#ifdef HAVE_CRT_EXTERNS_H +#include <crt_externs.h> +#endif #ifndef HAVE_GETTIMEOFDAY #ifdef HAVE_FTIME @@ -58,6 +65,14 @@ #endif #endif +/* Includes for the process attaching prevention */ +#if defined(HAVE_SYS_PRCTL_H) && defined(__linux__) +#include <sys/prctl.h> +#elif defined(__APPLE__) +#include <sys/types.h> +#include <sys/ptrace.h> +#endif + #ifdef HAVE_NETDB_H #include <netdb.h> #endif @@ -103,7 +118,45 @@ #include "strlcat.c" #endif -#ifdef HAVE_SYS_MMAN_H +/** As open(path, flags, mode), but return an fd with the close-on-exec mode + * set. */ +int +tor_open_cloexec(const char *path, int flags, unsigned mode) +{ + int fd; +#ifdef O_CLOEXEC + fd = open(path, flags|O_CLOEXEC, mode); + if (fd >= 0) + return fd; + /* If we got an error, see if it is EINVAL. EINVAL might indicate that, + * even though we were built on a system with O_CLOEXEC support, we + * are running on one without. */ + if (errno != EINVAL) + return -1; +#endif + + fd = open(path, flags, mode); +#ifdef FD_CLOEXEC + if (fd >= 0) + fcntl(fd, F_SETFD, FD_CLOEXEC); +#endif + return fd; +} + +/** As fopen(path,mode), but ensures that the O_CLOEXEC bit is set on the + * underlying file handle. */ +FILE * +tor_fopen_cloexec(const char *path, const char *mode) +{ + FILE *result = fopen(path, mode); +#ifdef FD_CLOEXEC + if (result != NULL) + fcntl(fileno(result), F_SETFD, FD_CLOEXEC); +#endif + return result; +} + +#if defined(HAVE_SYS_MMAN_H) || defined(RUNNING_DOXYGEN) /** Try to create a memory mapping for <b>filename</b> and return it. On * failure, return NULL. Sets errno properly, using ERANGE to mean * "empty file". */ @@ -118,7 +171,7 @@ tor_mmap_file(const char *filename) tor_assert(filename); - fd = open(filename, O_RDONLY, 0); + fd = tor_open_cloexec(filename, O_RDONLY, 0); if (fd<0) { int save_errno = errno; int severity = (errno == ENOENT) ? LOG_INFO : LOG_WARN; @@ -168,47 +221,55 @@ tor_munmap_file(tor_mmap_t *handle) munmap((char*)handle->data, handle->mapping_size); tor_free(handle); } -#elif defined(MS_WINDOWS) +#elif defined(_WIN32) tor_mmap_t * tor_mmap_file(const char *filename) { TCHAR tfilename[MAX_PATH]= {0}; tor_mmap_t *res = tor_malloc_zero(sizeof(tor_mmap_t)); int empty = 0; - res->file_handle = INVALID_HANDLE_VALUE; + HANDLE file_handle = INVALID_HANDLE_VALUE; + DWORD size_low, size_high; + uint64_t real_size; res->mmap_handle = NULL; #ifdef UNICODE mbstowcs(tfilename,filename,MAX_PATH); #else strlcpy(tfilename,filename,MAX_PATH); #endif - res->file_handle = CreateFile(tfilename, - GENERIC_READ, FILE_SHARE_READ, - NULL, - OPEN_EXISTING, - FILE_ATTRIBUTE_NORMAL, - 0); + file_handle = CreateFile(tfilename, + GENERIC_READ, FILE_SHARE_READ, + NULL, + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL, + 0); - if (res->file_handle == INVALID_HANDLE_VALUE) + if (file_handle == INVALID_HANDLE_VALUE) goto win_err; - res->size = GetFileSize(res->file_handle, NULL); + size_low = GetFileSize(file_handle, &size_high); - if (res->size == 0) { + if (size_low == INVALID_FILE_SIZE && GetLastError() != NO_ERROR) { + log_warn(LD_FS,"Error getting size of \"%s\".",filename); + goto win_err; + } + if (size_low == 0 && size_high == 0) { log_info(LD_FS,"File \"%s\" is empty. Ignoring.",filename); empty = 1; goto err; } + real_size = (((uint64_t)size_high)<<32) | size_low; + if (real_size > SIZE_MAX) { + log_warn(LD_FS,"File \"%s\" is too big to map; not trying.",filename); + goto err; + } + res->size = real_size; - res->mmap_handle = CreateFileMapping(res->file_handle, + res->mmap_handle = CreateFileMapping(file_handle, NULL, PAGE_READONLY, -#if SIZEOF_SIZE_T > 4 - (res->base.size >> 32), -#else - 0, -#endif - (res->size & 0xfffffffful), + size_high, + size_low, NULL); if (res->mmap_handle == NULL) goto win_err; @@ -218,6 +279,7 @@ tor_mmap_file(const char *filename) if (!res->data) goto win_err; + CloseHandle(file_handle); return res; win_err: { DWORD e = GetLastError(); @@ -234,6 +296,8 @@ tor_mmap_file(const char *filename) err: if (empty) errno = ERANGE; + if (file_handle != INVALID_HANDLE_VALUE) + CloseHandle(file_handle); tor_munmap_file(res); return NULL; } @@ -247,8 +311,6 @@ tor_munmap_file(tor_mmap_t *handle) if (handle->mmap_handle != NULL) CloseHandle(handle->mmap_handle); - if (handle->file_handle != INVALID_HANDLE_VALUE) - CloseHandle(handle->file_handle); tor_free(handle); } #else @@ -270,7 +332,7 @@ tor_munmap_file(tor_mmap_t *handle) { char *d = (char*)handle->data; tor_free(d); - memset(handle, 0, sizeof(tor_mmap_t)); + memwipe(handle, 0, sizeof(tor_mmap_t)); tor_free(handle); } #endif @@ -304,7 +366,7 @@ tor_vsnprintf(char *str, size_t size, const char *format, va_list args) return -1; /* no place for the NUL */ if (size > SIZE_T_CEILING) return -1; -#ifdef MS_WINDOWS +#ifdef _WIN32 r = _vsnprintf(str, size, format, args); #else r = vsnprintf(str, size, format, args); @@ -447,10 +509,13 @@ tor_memmem(const void *_haystack, size_t hlen, #endif } -/* Tables to implement ctypes-replacement TOR_IS*() functions. Each table +/** + * Tables to implement ctypes-replacement TOR_IS*() functions. Each table * has 256 bits to look up whether a character is in some set or not. This * fails on non-ASCII platforms, but it is hard to find a platform whose * character set is not a superset of ASCII nowadays. */ + +/**@{*/ const uint32_t TOR_ISALPHA_TABLE[8] = { 0, 0, 0x7fffffe, 0x7fffffe, 0, 0, 0, 0 }; const uint32_t TOR_ISALNUM_TABLE[8] = @@ -463,8 +528,10 @@ const uint32_t TOR_ISPRINT_TABLE[8] = { 0, 0xffffffff, 0xffffffff, 0x7fffffff, 0, 0, 0, 0x0 }; const uint32_t TOR_ISUPPER_TABLE[8] = { 0, 0, 0x7fffffe, 0, 0, 0, 0, 0 }; const uint32_t TOR_ISLOWER_TABLE[8] = { 0, 0, 0, 0x7fffffe, 0, 0, 0, 0 }; -/* Upper-casing and lowercasing tables to map characters to upper/lowercase - * equivalents. */ + +/** Upper-casing and lowercasing tables to map characters to upper/lowercase + * equivalents. Used by tor_toupper() and tor_tolower(). */ +/**@{*/ const char TOR_TOUPPER_TABLE[256] = { 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15, 16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31, @@ -501,6 +568,22 @@ const char TOR_TOLOWER_TABLE[256] = { 224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239, 240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255, }; +/**@}*/ + +/** Helper for tor_strtok_r_impl: Advances cp past all characters in + * <b>sep</b>, and returns its new value. */ +static char * +strtok_helper(char *cp, const char *sep) +{ + if (sep[1]) { + while (*cp && strchr(sep, *cp)) + ++cp; + } else { + while (*cp && *cp == *sep) + ++cp; + } + return cp; +} /** Implementation of strtok_r for platforms whose coders haven't figured out * how to write one. Hey guys! You can use this code here for free! */ @@ -508,19 +591,22 @@ char * tor_strtok_r_impl(char *str, const char *sep, char **lasts) { char *cp, *start; - if (str) + tor_assert(*sep); + if (str) { + str = strtok_helper(str, sep); + if (!*str) + return NULL; start = cp = *lasts = str; - else if (!*lasts) + } else if (!*lasts || !**lasts) { return NULL; - else + } else { start = cp = *lasts; + } - tor_assert(*sep); if (sep[1]) { while (*cp && !strchr(sep, *cp)) ++cp; } else { - tor_assert(strlen(sep) == 1); cp = strchr(cp, *sep); } @@ -528,12 +614,12 @@ tor_strtok_r_impl(char *str, const char *sep, char **lasts) *lasts = NULL; } else { *cp++ = '\0'; - *lasts = cp; + *lasts = strtok_helper(cp, sep); } return start; } -#ifdef MS_WINDOWS +#ifdef _WIN32 /** Take a filename and return a pointer to its final element. This * function is called on __FILE__ to fix a MSVC nit where __FILE__ * contains the full path to the file. This is bad, because it @@ -633,7 +719,7 @@ set_uint64(void *cp, uint64_t v) int replace_file(const char *from, const char *to) { -#ifndef MS_WINDOWS +#ifndef _WIN32 return rename(from,to); #else switch (file_status(to)) @@ -695,14 +781,14 @@ tor_lockfile_lock(const char *filename, int blocking, int *locked_out) *locked_out = 0; log_info(LD_FS, "Locking \"%s\"", filename); - fd = open(filename, O_RDWR|O_CREAT|O_TRUNC, 0600); + fd = tor_open_cloexec(filename, O_RDWR|O_CREAT|O_TRUNC, 0600); if (fd < 0) { log_warn(LD_FS,"Couldn't open \"%s\" for locking: %s", filename, strerror(errno)); return NULL; } -#ifdef WIN32 +#ifdef _WIN32 _lseek(fd, 0, SEEK_SET); if (_locking(fd, blocking ? _LK_LOCK : _LK_NBLCK, 1) < 0) { if (errno != EACCES && errno != EDEADLOCK) @@ -751,7 +837,7 @@ tor_lockfile_unlock(tor_lockfile_t *lockfile) tor_assert(lockfile); log_info(LD_FS, "Unlocking \"%s\"", lockfile->filename); -#ifdef WIN32 +#ifdef _WIN32 _lseek(lockfile->fd, 0, SEEK_SET); if (_locking(lockfile->fd, _LK_UNLCK, 1) < 0) { log_warn(LD_FS,"Error unlocking \"%s\": %s", lockfile->filename, @@ -787,7 +873,7 @@ tor_lockfile_unlock(tor_lockfile_t *lockfile) off_t tor_fd_getpos(int fd) { -#ifdef WIN32 +#ifdef _WIN32 return (off_t) _lseek(fd, 0, SEEK_CUR); #else return (off_t) lseek(fd, 0, SEEK_CUR); @@ -798,7 +884,7 @@ tor_fd_getpos(int fd) int tor_fd_seekend(int fd) { -#ifdef WIN32 +#ifdef _WIN32 return _lseek(fd, 0, SEEK_END) < 0 ? -1 : 0; #else return lseek(fd, 0, SEEK_END) < 0 ? -1 : 0; @@ -851,7 +937,7 @@ tor_close_socket(tor_socket_t s) * tor_close_socket to close sockets, and always using close() on * files. */ -#if defined(MS_WINDOWS) +#if defined(_WIN32) r = closesocket(s); #else r = close(s); @@ -872,7 +958,7 @@ tor_close_socket(tor_socket_t s) } else { int err = tor_socket_errno(-1); log_info(LD_NET, "Close returned an error: %s", tor_socket_strerror(err)); -#ifdef WIN32 +#ifdef _WIN32 if (err != WSAENOTSOCK) --n_sockets_open; #else @@ -922,13 +1008,33 @@ mark_socket_open(tor_socket_t s) tor_socket_t tor_open_socket(int domain, int type, int protocol) { - tor_socket_t s = socket(domain, type, protocol); - if (SOCKET_OK(s)) { - socket_accounting_lock(); - ++n_sockets_open; - mark_socket_open(s); - socket_accounting_unlock(); - } + tor_socket_t s; +#ifdef SOCK_CLOEXEC + s = socket(domain, type|SOCK_CLOEXEC, protocol); + if (SOCKET_OK(s)) + goto socket_ok; + /* If we got an error, see if it is EINVAL. EINVAL might indicate that, + * even though we were built on a system with SOCK_CLOEXEC support, we + * are running on one without. */ + if (errno != EINVAL) + return s; +#endif /* SOCK_CLOEXEC */ + + s = socket(domain, type, protocol); + if (! SOCKET_OK(s)) + return s; + +#if defined(FD_CLOEXEC) + fcntl(s, F_SETFD, FD_CLOEXEC); +#endif + + goto socket_ok; /* So that socket_ok will not be unused. */ + + socket_ok: + socket_accounting_lock(); + ++n_sockets_open; + mark_socket_open(s); + socket_accounting_unlock(); return s; } @@ -936,13 +1042,34 @@ tor_open_socket(int domain, int type, int protocol) tor_socket_t tor_accept_socket(tor_socket_t sockfd, struct sockaddr *addr, socklen_t *len) { - tor_socket_t s = accept(sockfd, addr, len); - if (SOCKET_OK(s)) { - socket_accounting_lock(); - ++n_sockets_open; - mark_socket_open(s); - socket_accounting_unlock(); - } + tor_socket_t s; +#if defined(HAVE_ACCEPT4) && defined(SOCK_CLOEXEC) + s = accept4(sockfd, addr, len, SOCK_CLOEXEC); + if (SOCKET_OK(s)) + goto socket_ok; + /* If we got an error, see if it is ENOSYS. ENOSYS indicates that, + * even though we were built on a system with accept4 support, we + * are running on one without. Also, check for EINVAL, which indicates that + * we are missing SOCK_CLOEXEC support. */ + if (errno != EINVAL && errno != ENOSYS) + return s; +#endif + + s = accept(sockfd, addr, len); + if (!SOCKET_OK(s)) + return s; + +#if defined(FD_CLOEXEC) + fcntl(s, F_SETFD, FD_CLOEXEC); +#endif + + goto socket_ok; /* So that socket_ok will not be unused. */ + + socket_ok: + socket_accounting_lock(); + ++n_sockets_open; + mark_socket_open(s); + socket_accounting_unlock(); return s; } @@ -962,7 +1089,7 @@ get_n_open_sockets(void) void set_socket_nonblocking(tor_socket_t socket) { -#if defined(MS_WINDOWS) +#if defined(_WIN32) unsigned long nonblocking = 1; ioctlsocket(socket, FIONBIO, (unsigned long*) &nonblocking); #else @@ -991,22 +1118,45 @@ int tor_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) { //don't use win32 socketpairs (they are always bad) -#if defined(HAVE_SOCKETPAIR) && !defined(MS_WINDOWS) +#if defined(HAVE_SOCKETPAIR) && !defined(_WIN32) int r; + +#ifdef SOCK_CLOEXEC + r = socketpair(family, type|SOCK_CLOEXEC, protocol, fd); + if (r == 0) + goto sockets_ok; + /* If we got an error, see if it is EINVAL. EINVAL might indicate that, + * even though we were built on a system with SOCK_CLOEXEC support, we + * are running on one without. */ + if (errno != EINVAL) + return -errno; +#endif + r = socketpair(family, type, protocol, fd); - if (r == 0) { - socket_accounting_lock(); - if (fd[0] >= 0) { - ++n_sockets_open; - mark_socket_open(fd[0]); - } - if (fd[1] >= 0) { - ++n_sockets_open; - mark_socket_open(fd[1]); - } - socket_accounting_unlock(); + if (r < 0) + return -errno; + +#if defined(FD_CLOEXEC) + if (SOCKET_OK(fd[0])) + fcntl(fd[0], F_SETFD, FD_CLOEXEC); + if (SOCKET_OK(fd[1])) + fcntl(fd[1], F_SETFD, FD_CLOEXEC); +#endif + goto sockets_ok; /* So that sockets_ok will not be unused. */ + + sockets_ok: + socket_accounting_lock(); + if (SOCKET_OK(fd[0])) { + ++n_sockets_open; + mark_socket_open(fd[0]); } - return r < 0 ? -errno : r; + if (SOCKET_OK(fd[1])) { + ++n_sockets_open; + mark_socket_open(fd[1]); + } + socket_accounting_unlock(); + + return 0; #else /* This socketpair does not work when localhost is down. So * it's really not the same thing at all. But it's close enough @@ -1026,7 +1176,7 @@ tor_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) || family != AF_UNIX #endif ) { -#ifdef MS_WINDOWS +#ifdef _WIN32 return -WSAEAFNOSUPPORT; #else return -EAFNOSUPPORT; @@ -1037,7 +1187,7 @@ tor_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) } listener = tor_open_socket(AF_INET, type, 0); - if (listener < 0) + if (!SOCKET_OK(listener)) return -tor_socket_errno(-1); memset(&listen_addr, 0, sizeof(listen_addr)); listen_addr.sin_family = AF_INET; @@ -1050,7 +1200,7 @@ tor_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) goto tidy_up_and_fail; connector = tor_open_socket(AF_INET, type, 0); - if (connector < 0) + if (!SOCKET_OK(connector)) goto tidy_up_and_fail; /* We want to find out the port number to connect to. */ size = sizeof(connect_addr); @@ -1065,7 +1215,7 @@ tor_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) size = sizeof(listen_addr); acceptor = tor_accept_socket(listener, (struct sockaddr *) &listen_addr, &size); - if (acceptor < 0) + if (!SOCKET_OK(acceptor)) goto tidy_up_and_fail; if (size != sizeof(listen_addr)) goto abort_tidy_up_and_fail; @@ -1086,7 +1236,7 @@ tor_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) return 0; abort_tidy_up_and_fail: -#ifdef MS_WINDOWS +#ifdef _WIN32 saved_errno = WSAECONNABORTED; #else saved_errno = ECONNABORTED; /* I hope this is portable and appropriate. */ @@ -1108,13 +1258,16 @@ tor_socketpair(int family, int type, int protocol, tor_socket_t fd[2]) * tell Tor it's allowed to use. */ #define ULIMIT_BUFFER 32 /* keep 32 extra fd's beyond _ConnLimit */ -/** Learn the maximum allowed number of file descriptors. (Some systems - * have a low soft limit. +/** Learn the maximum allowed number of file descriptors, and tell the system + * we want to use up to that number. (Some systems have a low soft limit, and + * let us set it higher.) * * We compute this by finding the largest number that we can use. * If we can't find a number greater than or equal to <b>limit</b>, * then we fail: return -1. * + * If <b>limit</b> is 0, then do not adjust the current maximum. + * * Otherwise, return 0 and store the maximum we found inside <b>max_out</b>.*/ int set_max_file_descriptors(rlim_t limit, int *max_out) @@ -1128,7 +1281,7 @@ set_max_file_descriptors(rlim_t limit, int *max_out) #if defined(CYGWIN) || defined(__CYGWIN__) const char *platform = "Cygwin"; const unsigned long MAX_CONNECTIONS = 3200; -#elif defined(MS_WINDOWS) +#elif defined(_WIN32) const char *platform = "Windows"; const unsigned long MAX_CONNECTIONS = 15000; #else @@ -1147,14 +1300,20 @@ set_max_file_descriptors(rlim_t limit, int *max_out) limit = MAX_CONNECTIONS; #else /* HAVE_GETRLIMIT */ struct rlimit rlim; - tor_assert(limit > 0); if (getrlimit(RLIMIT_NOFILE, &rlim) != 0) { log_warn(LD_NET, "Could not get maximum number of file descriptors: %s", strerror(errno)); return -1; } - + if (limit == 0) { + /* If limit == 0, return the maximum value without setting it. */ + limit = rlim.rlim_max; + if (limit > INT_MAX) + limit = INT_MAX; + *max_out = (int)limit - ULIMIT_BUFFER; + return 0; + } if (rlim.rlim_max < limit) { log_warn(LD_CONFIG,"We need %lu file descriptors available, and we're " "limited to %lu. Please change your ulimit -n.", @@ -1211,7 +1370,7 @@ set_max_file_descriptors(rlim_t limit, int *max_out) return 0; } -#ifndef MS_WINDOWS +#ifndef _WIN32 /** Log details of current user and group credentials. Return 0 on * success. Logs and return -1 on failure. */ @@ -1288,31 +1447,19 @@ log_credential_status(void) return -1; } else { int i, retval = 0; - char *strgid; char *s = NULL; - smartlist_t *elts = smartlist_create(); + smartlist_t *elts = smartlist_new(); for (i = 0; i<ngids; i++) { - strgid = tor_malloc(11); - if (tor_snprintf(strgid, 11, "%u", (unsigned)sup_gids[i]) < 0) { - log_warn(LD_GENERAL, "Error printing supplementary GIDs"); - tor_free(strgid); - retval = -1; - goto error; - } - smartlist_add(elts, strgid); + smartlist_add_asprintf(elts, "%u", (unsigned)sup_gids[i]); } s = smartlist_join_strings(elts, " ", 0, NULL); log_fn(CREDENTIAL_LOG_LEVEL, LD_GENERAL, "Supplementary groups are: %s",s); - error: tor_free(s); - SMARTLIST_FOREACH(elts, char *, cp, - { - tor_free(cp); - }); + SMARTLIST_FOREACH(elts, char *, cp, tor_free(cp)); smartlist_free(elts); tor_free(sup_gids); @@ -1329,7 +1476,7 @@ log_credential_status(void) int switch_id(const char *user) { -#ifndef MS_WINDOWS +#ifndef _WIN32 struct passwd *pw = NULL; uid_t old_uid; gid_t old_gid; @@ -1464,6 +1611,58 @@ switch_id(const char *user) #endif } +/* We only use the linux prctl for now. There is no Win32 support; this may + * also work on various BSD systems and Mac OS X - send testing feedback! + * + * On recent Gnu/Linux kernels it is possible to create a system-wide policy + * that will prevent non-root processes from attaching to other processes + * unless they are the parent process; thus gdb can attach to programs that + * they execute but they cannot attach to other processes running as the same + * user. The system wide policy may be set with the sysctl + * kernel.yama.ptrace_scope or by inspecting + * /proc/sys/kernel/yama/ptrace_scope and it is 1 by default on Ubuntu 11.04. + * + * This ptrace scope will be ignored on Gnu/Linux for users with + * CAP_SYS_PTRACE and so it is very likely that root will still be able to + * attach to the Tor process. + */ +/** Attempt to disable debugger attachment: return 1 on success, -1 on + * failure, and 0 if we don't know how to try on this platform. */ +int +tor_disable_debugger_attach(void) +{ + int r, attempted; + r = -1; + attempted = 0; + log_debug(LD_CONFIG, + "Attemping to disable debugger attachment to Tor for " + "unprivileged users."); +#if defined(__linux__) && defined(HAVE_SYS_PRCTL_H) && defined(HAVE_PRCTL) +#ifdef PR_SET_DUMPABLE + attempted = 1; + r = prctl(PR_SET_DUMPABLE, 0); +#endif +#endif +#if defined(__APPLE__) && defined(PT_DENY_ATTACH) + if (r < 0) { + attempted = 1; + r = ptrace(PT_DENY_ATTACH, 0, 0, 0); + } +#endif + + // XXX: TODO - Mac OS X has dtrace and this may be disabled. + // XXX: TODO - Windows probably has something similar + if (r == 0 && attempted) { + log_debug(LD_CONFIG,"Debugger attachment disabled for " + "unprivileged users."); + return 1; + } else if (attempted) { + log_warn(LD_CONFIG, "Unable to disable debugger attaching: %s", + strerror(errno)); + } + return r; +} + #ifdef HAVE_PWD_H /** Allocate and return a string containing the home directory for the * user <b>username</b>. Only works on posix-like systems. */ @@ -1481,14 +1680,18 @@ get_user_homedir(const char *username) } #endif -/** Modify <b>fname</b> to contain the name of the directory */ +/** Modify <b>fname</b> to contain the name of its parent directory. Doesn't + * actually examine the filesystem; does a purely syntactic modification. + * + * The parent of the root director is considered to be iteself. + * */ int get_parent_directory(char *fname) { char *cp; int at_end = 1; tor_assert(fname); -#ifdef MS_WINDOWS +#ifdef _WIN32 /* If we start with, say, c:, then don't consider that the start of the path */ if (fname[0] && fname[1] == ':') { @@ -1503,13 +1706,18 @@ get_parent_directory(char *fname) */ cp = fname + strlen(fname); at_end = 1; - while (--cp > fname) { + while (--cp >= fname) { int is_sep = (*cp == '/' -#ifdef MS_WINDOWS +#ifdef _WIN32 || *cp == '\\' #endif ); if (is_sep) { + if (cp == fname) { + /* This is the first separator in the file name; don't remove it! */ + cp[1] = '\0'; + return 0; + } *cp = '\0'; if (! at_end) return 0; @@ -1520,6 +1728,101 @@ get_parent_directory(char *fname) return -1; } +#ifndef _WIN32 +/** Return a newly allocated string containing the output of getcwd(). Return + * NULL on failure. (We can't just use getcwd() into a PATH_MAX buffer, since + * Hurd hasn't got a PATH_MAX.) + */ +static char * +alloc_getcwd(void) +{ + int saved_errno = errno; +/* We use this as a starting path length. Not too large seems sane. */ +#define START_PATH_LENGTH 128 +/* Nobody has a maxpath longer than this, as far as I know. And if they + * do, they shouldn't. */ +#define MAX_SANE_PATH_LENGTH 4096 + size_t path_length = START_PATH_LENGTH; + char *path = tor_malloc(path_length); + + errno = 0; + while (getcwd(path, path_length) == NULL) { + if (errno == ERANGE && path_length < MAX_SANE_PATH_LENGTH) { + path_length*=2; + path = tor_realloc(path, path_length); + } else { + tor_free(path); + path = NULL; + break; + } + } + errno = saved_errno; + return path; +} +#endif + +/** Expand possibly relative path <b>fname</b> to an absolute path. + * Return a newly allocated string, possibly equal to <b>fname</b>. */ +char * +make_path_absolute(char *fname) +{ +#ifdef _WIN32 + char *absfname_malloced = _fullpath(NULL, fname, 1); + + /* We don't want to assume that tor_free can free a string allocated + * with malloc. On failure, return fname (it's better than nothing). */ + char *absfname = tor_strdup(absfname_malloced ? absfname_malloced : fname); + if (absfname_malloced) free(absfname_malloced); + + return absfname; +#else + char *absfname = NULL, *path = NULL; + + tor_assert(fname); + + if (fname[0] == '/') { + absfname = tor_strdup(fname); + } else { + path = alloc_getcwd(); + if (path) { + tor_asprintf(&absfname, "%s/%s", path, fname); + tor_free(path); + } else { + /* If getcwd failed, the best we can do here is keep using the + * relative path. (Perhaps / isn't readable by this UID/GID.) */ + log_warn(LD_GENERAL, "Unable to find current working directory: %s", + strerror(errno)); + absfname = tor_strdup(fname); + } + } + return absfname; +#endif +} + +#ifndef HAVE__NSGETENVIRON +#ifndef HAVE_EXTERN_ENVIRON_DECLARED +/* Some platforms declare environ under some circumstances, others don't. */ +#ifndef RUNNING_DOXYGEN +extern char **environ; +#endif +#endif +#endif + +/** Return the current environment. This is a portable replacement for + * 'environ'. */ +char ** +get_environment(void) +{ +#ifdef HAVE__NSGETENVIRON + /* This is for compatibility between OSX versions. Otherwise (for example) + * when we do a mostly-static build on OSX 10.7, the resulting binary won't + * work on OSX 10.6. */ + return *_NSGetEnviron(); +#else + return environ; +#endif +} + /** Set *addr to the IP address (in dotted-quad notation) stored in c. * Return 1 on success, 0 if c is badly formatted. (Like inet_aton(c,addr), * but works on Windows and Solaris.) @@ -1577,7 +1880,7 @@ tor_inet_ntop(int af, const void *src, char *dst, size_t len) addr->s6_addr[12], addr->s6_addr[13], addr->s6_addr[14], addr->s6_addr[15]); } - if (strlen(buf) > len) + if ((strlen(buf) + 1) > len) /* +1 for \0 */ return NULL; strlcpy(dst, buf, len); return dst; @@ -1618,7 +1921,7 @@ tor_inet_ntop(int af, const void *src, char *dst, size_t len) } } *cp = '\0'; - if (strlen(buf) > len) + if ((strlen(buf) + 1) > len) /* +1 for \0 */ return NULL; strlcpy(dst, buf, len); return dst; @@ -1678,24 +1981,30 @@ tor_inet_pton(int af, const char *src, void *dst) return 0; if (TOR_ISXDIGIT(*src)) { char *next; + ssize_t len; long r = strtol(src, &next, 16); - if (next > 4+src) - return 0; - if (next == src) - return 0; - if (r<0 || r>65536) + tor_assert(next != NULL); + tor_assert(next != src); + + len = *next == '\0' ? eow - src : next - src; + if (len > 4) return 0; + if (len > 1 && !TOR_ISXDIGIT(src[1])) + return 0; /* 0x is not valid */ + tor_assert(r >= 0); + tor_assert(r < 65536); words[i++] = (uint16_t)r; setWords++; src = next; if (*src != ':' && src != eow) return 0; ++src; - } else if (*src == ':' && i > 0 && gapPos==-1) { + } else if (*src == ':' && i > 0 && gapPos == -1) { gapPos = i; ++src; - } else if (*src == ':' && i == 0 && src[1] == ':' && gapPos==-1) { + } else if (*src == ':' && i == 0 && src+1 < eow && src[1] == ':' && + gapPos == -1) { gapPos = i; src += 2; } else { @@ -1755,7 +2064,7 @@ tor_lookup_hostname(const char *name, uint32_t *addr) void tor_init_weak_random(unsigned seed) { -#ifdef MS_WINDOWS +#ifdef _WIN32 srand(seed); #else srandom(seed); @@ -1768,7 +2077,7 @@ tor_init_weak_random(unsigned seed) long tor_weak_random(void) { -#ifdef MS_WINDOWS +#ifdef _WIN32 return rand(); #else return random(); @@ -1792,17 +2101,14 @@ get_uname(void) #ifdef HAVE_UNAME if (uname(&u) != -1) { /* (Linux says 0 is success, Solaris says 1 is success) */ - tor_snprintf(uname_result, sizeof(uname_result), "%s %s", - u.sysname, u.machine); + strlcpy(uname_result, u.sysname, sizeof(uname_result)); } else #endif { -#ifdef MS_WINDOWS +#ifdef _WIN32 OSVERSIONINFOEX info; int i; const char *plat = NULL; - const char *extra = NULL; - char acsd[MAX_PATH] = {0}; static struct { unsigned major; unsigned minor; const char *version; } win_version_table[] = { @@ -1827,20 +2133,11 @@ get_uname(void) uname_result_is_set = 1; return uname_result; } -#ifdef UNICODE - wcstombs(acsd, info.szCSDVersion, MAX_PATH); -#else - strlcpy(acsd, info.szCSDVersion, sizeof(acsd)); -#endif if (info.dwMajorVersion == 4 && info.dwMinorVersion == 0) { if (info.dwPlatformId == VER_PLATFORM_WIN32_NT) plat = "Windows NT 4.0"; else plat = "Windows 95"; - if (acsd[1] == 'B') - extra = "OSR2 (B)"; - else if (acsd[1] == 'C') - extra = "OSR2 (C)"; } else { for (i=0; win_version_table[i].major>0; ++i) { if (win_version_table[i].major == info.dwMajorVersion && @@ -1850,39 +2147,25 @@ get_uname(void) } } } - if (plat && !strcmp(plat, "Windows 98")) { - if (acsd[1] == 'A') - extra = "SE (A)"; - else if (acsd[1] == 'B') - extra = "SE (B)"; - } if (plat) { - if (!extra) - extra = acsd; - tor_snprintf(uname_result, sizeof(uname_result), "%s %s", - plat, extra); + strlcpy(uname_result, plat, sizeof(uname_result)); } else { if (info.dwMajorVersion > 6 || (info.dwMajorVersion==6 && info.dwMinorVersion>2)) tor_snprintf(uname_result, sizeof(uname_result), - "Very recent version of Windows [major=%d,minor=%d] %s", - (int)info.dwMajorVersion,(int)info.dwMinorVersion, - acsd); + "Very recent version of Windows [major=%d,minor=%d]", + (int)info.dwMajorVersion,(int)info.dwMinorVersion); else tor_snprintf(uname_result, sizeof(uname_result), - "Unrecognized version of Windows [major=%d,minor=%d] %s", - (int)info.dwMajorVersion,(int)info.dwMinorVersion, - acsd); + "Unrecognized version of Windows [major=%d,minor=%d]", + (int)info.dwMajorVersion,(int)info.dwMinorVersion); } #if !defined (WINCE) -#ifdef VER_SUITE_BACKOFFICE - if (info.wProductType == VER_NT_DOMAIN_CONTROLLER) { - strlcat(uname_result, " [domain controller]", sizeof(uname_result)); - } else if (info.wProductType == VER_NT_SERVER) { - strlcat(uname_result, " [server]", sizeof(uname_result)); - } else if (info.wProductType == VER_NT_WORKSTATION) { - strlcat(uname_result, " [workstation]", sizeof(uname_result)); - } +#ifdef VER_NT_SERVER + if (info.wProductType == VER_NT_SERVER || + info.wProductType == VER_NT_DOMAIN_CONTROLLER) { + strlcat(uname_result, " [server]", sizeof(uname_result)); + } #endif #endif #else @@ -1995,13 +2278,59 @@ spawn_exit(void) #endif } +/** Implementation logic for compute_num_cpus(). */ +static int +compute_num_cpus_impl(void) +{ +#ifdef _WIN32 + SYSTEM_INFO info; + memset(&info, 0, sizeof(info)); + GetSystemInfo(&info); + if (info.dwNumberOfProcessors >= 1 && info.dwNumberOfProcessors < INT_MAX) + return (int)info.dwNumberOfProcessors; + else + return -1; +#elif defined(HAVE_SYSCONF) && defined(_SC_NPROCESSORS_CONF) + long cpus = sysconf(_SC_NPROCESSORS_CONF); + if (cpus >= 1 && cpus < INT_MAX) + return (int)cpus; + else + return -1; +#else + return -1; +#endif +} + +#define MAX_DETECTABLE_CPUS 16 + +/** Return how many CPUs we are running with. We assume that nobody is + * using hot-swappable CPUs, so we don't recompute this after the first + * time. Return -1 if we don't know how to tell the number of CPUs on this + * system. + */ +int +compute_num_cpus(void) +{ + static int num_cpus = -2; + if (num_cpus == -2) { + num_cpus = compute_num_cpus_impl(); + tor_assert(num_cpus != -2); + if (num_cpus > MAX_DETECTABLE_CPUS) + log_notice(LD_GENERAL, "Wow! I detected that you have %d CPUs. I " + "will not autodetect any more than %d, though. If you " + "want to configure more, set NumCPUs in your torrc", + num_cpus, MAX_DETECTABLE_CPUS); + } + return num_cpus; +} + /** Set *timeval to the current time of day. On error, log and terminate. * (Same as gettimeofday(timeval,NULL), but never returns -1.) */ void tor_gettimeofday(struct timeval *timeval) { -#ifdef MS_WINDOWS +#ifdef _WIN32 /* Epoch bias copied from perl: number of units between windows epoch and * Unix epoch. */ #define EPOCH_BIAS U64_LITERAL(116444736000000000) @@ -2046,12 +2375,18 @@ tor_gettimeofday(struct timeval *timeval) return; } -#if defined(TOR_IS_MULTITHREADED) && !defined(MS_WINDOWS) +#if defined(TOR_IS_MULTITHREADED) && !defined(_WIN32) /** Defined iff we need to add locks when defining fake versions of reentrant * versions of time-related functions. */ #define TIME_FNS_NEED_LOCKS #endif +/** Helper: Deal with confused or out-of-bounds values from localtime_r and + * friends. (On some platforms, they can give out-of-bounds values or can + * return NULL.) If <b>islocal</b>, this is a localtime result; otherwise + * it's from gmtime. The function returned <b>r</b>, when given <b>timep</b> + * as its input. If we need to store new results, store them in + * <b>resultbuf</b>. */ static struct tm * correct_tm(int islocal, const time_t *timep, struct tm *resultbuf, struct tm *r) @@ -2396,7 +2731,7 @@ tor_cond_new(void) { tor_cond_t *cond = tor_malloc_zero(sizeof(tor_cond_t)); InitializeCriticalSection(&cond->mutex); - cond->events = smartlist_create(); + cond->events = smartlist_new(); return cond; } void @@ -2590,7 +2925,7 @@ in_main_thread(void) * should call tor_socket_errno <em>at most once</em> on the failing * socket to get the error. */ -#if defined(MS_WINDOWS) +#if defined(_WIN32) int tor_socket_errno(tor_socket_t sock) { @@ -2606,7 +2941,7 @@ tor_socket_errno(tor_socket_t sock) } #endif -#if defined(MS_WINDOWS) +#if defined(_WIN32) #define E(code, s) { code, (s " [" #code " ]") } struct { int code; const char *msg; } windows_socket_errors[] = { E(WSAEINTR, "Interrupted function call"), @@ -2688,7 +3023,7 @@ tor_socket_strerror(int e) int network_init(void) { -#ifdef MS_WINDOWS +#ifdef _WIN32 /* This silly exercise is necessary before windows will allow * gethostbyname to work. */ WSADATA WSAData; @@ -2698,6 +3033,11 @@ network_init(void) log_warn(LD_NET,"Error initializing windows network layer: code was %d",r); return -1; } + if (sizeof(SOCKET) != sizeof(tor_socket_t)) { + log_warn(LD_BUG,"The tor_socket_t type does not match SOCKET in size; Tor " + "might not work. (Sizes are %d and %d respectively.)", + (int)sizeof(tor_socket_t), (int)sizeof(SOCKET)); + } /* WSAData.iMaxSockets might show the max sockets we're allowed to use. * We might use it to complain if we're trying to be a server but have * too few sockets available. */ @@ -2705,7 +3045,7 @@ network_init(void) return 0; } -#ifdef MS_WINDOWS +#ifdef _WIN32 /** Return a newly allocated string describing the windows system error code * <b>err</b>. Note that error codes are different from errno. Error codes * come from GetLastError() when a winapi call fails. errno is set only when @@ -2715,28 +3055,37 @@ format_win32_error(DWORD err) { TCHAR *str = NULL; char *result; + DWORD n; /* Somebody once decided that this interface was better than strerror(). */ - FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER | + n = FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL, err, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), - (LPVOID)&str, + (LPVOID)&str, 0, NULL); - if (str) { + if (str && n) { #ifdef UNICODE - char abuf[1024] = {0}; - wcstombs(abuf,str,1024); - result = tor_strdup(abuf); + size_t len; + if (n > 128*1024) + len = (128 * 1024) * 2 + 1; /* This shouldn't be possible, but let's + * make sure. */ + else + len = n * 2 + 1; + result = tor_malloc(len); + wcstombs(result,str,len); + result[len-1] = '\0'; #else result = tor_strdup(str); #endif - LocalFree(str); /* LocalFree != free() */ } else { result = tor_strdup("<unformattable error>"); } + if (str) { + LocalFree(str); /* LocalFree != free() */ + } return result; } #endif diff --git a/src/common/compat.h b/src/common/compat.h index fc70caf50c..42648bb04b 100644 --- a/src/common/compat.h +++ b/src/common/compat.h @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #ifndef _TOR_COMPAT_H @@ -8,9 +8,10 @@ #include "orconfig.h" #include "torint.h" -#ifdef MS_WINDOWS -#define WIN32_WINNT 0x400 -#define _WIN32_WINNT 0x400 +#ifdef _WIN32 +#ifndef _WIN32_WINNT +#define _WIN32_WINNT 0x0501 +#endif #define WIN32_LEAN_AND_MEAN #if defined(_MSC_VER) && (_MSC_VER < 1300) #include <winsock.h> @@ -19,6 +20,9 @@ #include <ws2tcpip.h> #endif #endif +#ifdef HAVE_SYS_PARAM_H +#include <sys/param.h> +#endif #ifdef HAVE_SYS_TYPES_H #include <sys/types.h> #endif @@ -31,7 +35,7 @@ #ifdef HAVE_STRING_H #include <string.h> #endif -#if defined(HAVE_PTHREAD_H) && !defined(MS_WINDOWS) +#if defined(HAVE_PTHREAD_H) && !defined(_WIN32) #include <pthread.h> #endif #include <stdarg.h> @@ -41,9 +45,6 @@ #ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> #endif -#ifdef HAVE_SYS_TYPES_H -#include <sys/types.h> -#endif #ifdef HAVE_NETINET_IN_H #include <netinet/in.h> #endif @@ -51,6 +52,8 @@ #include <netinet6/in6.h> #endif +#include <stdio.h> + #if defined (WINCE) #include <fcntl.h> #include <io.h> @@ -86,7 +89,7 @@ #endif /* inline is __inline on windows. */ -#ifdef MS_WINDOWS +#ifdef _WIN32 #define INLINE __inline #else #define INLINE inline @@ -132,7 +135,6 @@ extern INLINE double U64_TO_DBL(uint64_t x) { /* GCC has several useful attributes. */ #if defined(__GNUC__) && __GNUC__ >= 3 #define ATTR_NORETURN __attribute__((noreturn)) -#define ATTR_PURE __attribute__((pure)) #define ATTR_CONST __attribute__((const)) #define ATTR_MALLOC __attribute__((malloc)) #define ATTR_NORETURN __attribute__((noreturn)) @@ -165,7 +167,6 @@ extern INLINE double U64_TO_DBL(uint64_t x) { #define PREDICT_UNLIKELY(exp) __builtin_expect(!!(exp), 0) #else #define ATTR_NORETURN -#define ATTR_PURE #define ATTR_CONST #define ATTR_MALLOC #define ATTR_NORETURN @@ -195,7 +196,7 @@ extern INLINE double U64_TO_DBL(uint64_t x) { #endif /* ===== String compatibility */ -#ifdef MS_WINDOWS +#ifdef _WIN32 /* Windows names string functions differently from most other platforms. */ #define strncasecmp _strnicmp #define strcasecmp _stricmp @@ -248,8 +249,7 @@ typedef struct tor_mmap_t { #ifdef HAVE_SYS_MMAN_H size_t mapping_size; /**< Size of the actual mapping. (This is this file * size, rounded up to the nearest page.) */ -#elif defined MS_WINDOWS - HANDLE file_handle; +#elif defined _WIN32 HANDLE mmap_handle; #endif @@ -269,9 +269,9 @@ int tor_vasprintf(char **strp, const char *fmt, va_list args) CHECK_PRINTF(2,0); const void *tor_memmem(const void *haystack, size_t hlen, const void *needle, - size_t nlen) ATTR_PURE ATTR_NONNULL((1,3)); + size_t nlen) ATTR_NONNULL((1,3)); static const void *tor_memstr(const void *haystack, size_t hlen, - const char *needle) ATTR_PURE ATTR_NONNULL((1,3)); + const char *needle) ATTR_NONNULL((1,3)); static INLINE const void * tor_memstr(const void *haystack, size_t hlen, const char *needle) { @@ -307,7 +307,7 @@ char *tor_strtok_r_impl(char *str, const char *sep, char **lasts); #define tor_strtok_r(str, sep, lasts) tor_strtok_r_impl(str, sep, lasts) #endif -#ifdef MS_WINDOWS +#ifdef _WIN32 #define _SHORT_FILE_ (tor_fix_source_file(__FILE__)) const char *tor_fix_source_file(const char *fname); #else @@ -372,6 +372,9 @@ struct tm *tor_gmtime_r(const time_t *timep, struct tm *result); #endif /* ===== File compatibility */ +int tor_open_cloexec(const char *path, int flags, unsigned mode); +FILE *tor_fopen_cloexec(const char *path, const char *mode); + int replace_file(const char *from, const char *to); int touch_file(const char *fname); @@ -383,7 +386,7 @@ void tor_lockfile_unlock(tor_lockfile_t *lockfile); off_t tor_fd_getpos(int fd); int tor_fd_seekend(int fd); -#ifdef MS_WINDOWS +#ifdef _WIN32 #define PATH_SEPARATOR "\\" #else #define PATH_SEPARATOR "/" @@ -395,12 +398,20 @@ int tor_fd_seekend(int fd); typedef int socklen_t; #endif -#ifdef MS_WINDOWS +#ifdef _WIN32 +/* XXX Actually, this should arguably be SOCKET; we use intptr_t here so that + * any inadvertant checks for the socket being <= 0 or > 0 will probably + * still work. */ #define tor_socket_t intptr_t #define SOCKET_OK(s) ((SOCKET)(s) != INVALID_SOCKET) +#define TOR_INVALID_SOCKET INVALID_SOCKET #else +/** Type used for a network socket. */ #define tor_socket_t int +/** Macro: true iff 's' is a possible value for a valid initialized socket. */ #define SOCKET_OK(s) ((s) >= 0) +/** Error/uninitialized value for a tor_socket_t. */ +#define TOR_INVALID_SOCKET (-1) #endif int tor_close_socket(tor_socket_t s); @@ -488,7 +499,9 @@ int network_init(void); * errnos against expected values, and use tor_socket_errno to find * the actual errno after a socket operation fails. */ -#if defined(MS_WINDOWS) +#if defined(_WIN32) +/** Expands to WSA<b>e</b> on Windows, and to <b>e</b> elsewhere. */ +#define SOCK_ERRNO(e) WSA##e /** Return true if e is EAGAIN or the local equivalent. */ #define ERRNO_IS_EAGAIN(e) ((e) == EAGAIN || (e) == WSAEWOULDBLOCK) /** Return true if e is EINPROGRESS or the local equivalent. */ @@ -509,6 +522,7 @@ int network_init(void); int tor_socket_errno(tor_socket_t sock); const char *tor_socket_strerror(int e); #else +#define SOCK_ERRNO(e) e #define ERRNO_IS_EAGAIN(e) ((e) == EAGAIN) #define ERRNO_IS_EINPROGRESS(e) ((e) == EINPROGRESS) #define ERRNO_IS_CONN_EINPROGRESS(e) ((e) == EINPROGRESS) @@ -541,9 +555,9 @@ long tor_weak_random(void); /* ===== OS compatibility */ const char *get_uname(void); -uint16_t get_uint16(const void *cp) ATTR_PURE ATTR_NONNULL((1)); -uint32_t get_uint32(const void *cp) ATTR_PURE ATTR_NONNULL((1)); -uint64_t get_uint64(const void *cp) ATTR_PURE ATTR_NONNULL((1)); +uint16_t get_uint16(const void *cp) ATTR_NONNULL((1)); +uint32_t get_uint32(const void *cp) ATTR_NONNULL((1)); +uint64_t get_uint64(const void *cp) ATTR_NONNULL((1)); void set_uint16(void *cp, uint16_t v) ATTR_NONNULL((1)); void set_uint32(void *cp, uint32_t v) ATTR_NONNULL((1)); void set_uint64(void *cp, uint64_t v) ATTR_NONNULL((1)); @@ -561,17 +575,21 @@ set_uint8(void *cp, uint8_t v) typedef unsigned long rlim_t; #endif int set_max_file_descriptors(rlim_t limit, int *max); +int tor_disable_debugger_attach(void); int switch_id(const char *user); #ifdef HAVE_PWD_H char *get_user_homedir(const char *username); #endif int get_parent_directory(char *fname); +char *make_path_absolute(char *fname); + +char **get_environment(void); int spawn_func(void (*func)(void *), void *data); void spawn_exit(void) ATTR_NORETURN; -#if defined(ENABLE_THREADS) && defined(MS_WINDOWS) +#if defined(ENABLE_THREADS) && defined(_WIN32) #define USE_WIN32_THREADS #define TOR_IS_MULTITHREADED 1 #elif (defined(ENABLE_THREADS) && defined(HAVE_PTHREAD_H) && \ @@ -582,6 +600,8 @@ void spawn_exit(void) ATTR_NORETURN; #undef TOR_IS_MULTITHREADED #endif +int compute_num_cpus(void); + /* Because we use threads instead of processes on most platforms (Windows, * Linux, etc), we need locking for them. On platforms with poor thread * support or broken gethostbyname_r, these functions are no-ops. */ @@ -651,14 +671,14 @@ void tor_cond_signal_all(tor_cond_t *cond); #endif /* Platform-specific helpers. */ -#ifdef MS_WINDOWS +#ifdef _WIN32 char *format_win32_error(DWORD err); #endif /*for some reason my compiler doesn't have these version flags defined a nice homework assignment for someone one day is to define the rest*/ //these are the values as given on MSDN -#ifdef MS_WINDOWS +#ifdef _WIN32 #ifndef VER_SUITE_EMBEDDEDNT #define VER_SUITE_EMBEDDEDNT 0x00000040 diff --git a/src/common/compat_libevent.c b/src/common/compat_libevent.c index ddb2da68aa..6655ca87d3 100644 --- a/src/common/compat_libevent.c +++ b/src/common/compat_libevent.c @@ -1,4 +1,4 @@ -/* Copyright (c) 2009-2011, The Tor Project, Inc. */ +/* Copyright (c) 2009-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -6,9 +6,8 @@ * \brief Wrappers to handle porting between different versions of libevent. * * In an ideal world, we'd just use Libevent 2.0 from now on. But as of June - * 2009, Libevent 2.0 is still in alpha, and we will have old versions of - * Libevent for the forseeable future. - **/ + * 2012, Libevent 1.4 is still all over, and some poor souls are stuck on + * Libevent 1.3e. */ #include "orconfig.h" #include "compat.h" @@ -19,6 +18,10 @@ #ifdef HAVE_EVENT2_EVENT_H #include <event2/event.h> +#include <event2/thread.h> +#ifdef USE_BUFFEREVENTS +#include <event2/bufferevent.h> +#endif #else #include <event.h> #endif @@ -53,7 +56,7 @@ typedef uint32_t le_version_t; static le_version_t tor_get_libevent_version(const char **v_out); -#ifdef HAVE_EVENT_SET_LOG_CALLBACK +#if defined(HAVE_EVENT_SET_LOG_CALLBACK) || defined(RUNNING_DOXYGEN) /** A string which, if it appears in a libevent log, should be ignored. */ static const char *suppress_msg = NULL; /** Callback function passed to event_set_log() so we can intercept @@ -163,11 +166,24 @@ struct event_base *the_event_base = NULL; #endif #endif +#ifdef USE_BUFFEREVENTS +static int using_iocp_bufferevents = 0; +static void tor_libevent_set_tick_timeout(int msec_per_tick); + +int +tor_libevent_using_iocp_bufferevents(void) +{ + return using_iocp_bufferevents; +} +#endif + /** Initialize the Libevent library and set up the event base. */ void -tor_libevent_initialize(void) +tor_libevent_initialize(tor_libevent_cfg *torcfg) { tor_assert(the_event_base == NULL); + /* some paths below don't use torcfg, so avoid unused variable warnings */ + (void)torcfg; #ifdef __APPLE__ if (MACOSX_KQUEUE_IS_BROKEN || @@ -178,17 +194,65 @@ tor_libevent_initialize(void) #ifdef HAVE_EVENT2_EVENT_H { - struct event_config *cfg = event_config_new(); + int attempts = 0; + int using_threads; + struct event_config *cfg; + + retry: + ++attempts; + using_threads = 0; + cfg = event_config_new(); tor_assert(cfg); - /* In 0.2.2, we don't use locking at all. Telling Libevent not to try to - * turn it on can avoid a needless socketpair() attempt. - */ - event_config_set_flag(cfg, EVENT_BASE_FLAG_NOLOCK); +#if defined(_WIN32) && defined(USE_BUFFEREVENTS) + if (! torcfg->disable_iocp) { + evthread_use_windows_threads(); + event_config_set_flag(cfg, EVENT_BASE_FLAG_STARTUP_IOCP); + using_iocp_bufferevents = 1; + using_threads = 1; + } else { + using_iocp_bufferevents = 0; + } +#endif + + if (!using_threads) { + /* Telling Libevent not to try to turn locking on can avoid a needless + * socketpair() attempt. */ + event_config_set_flag(cfg, EVENT_BASE_FLAG_NOLOCK); + } + +#if defined(LIBEVENT_VERSION_NUMBER) && LIBEVENT_VERSION_NUMBER >= V(2,0,7) + if (torcfg->num_cpus > 0) + event_config_set_num_cpus_hint(cfg, torcfg->num_cpus); +#endif + +#if LIBEVENT_VERSION_NUMBER >= V(2,0,9) + /* We can enable changelist support with epoll, since we don't give + * Libevent any dup'd fds. This lets us avoid some syscalls. */ + event_config_set_flag(cfg, EVENT_BASE_FLAG_EPOLL_USE_CHANGELIST); +#endif the_event_base = event_base_new_with_config(cfg); event_config_free(cfg); + + if (using_threads && the_event_base == NULL && attempts < 2) { + /* This could be a socketpair() failure, which can happen sometimes on + * windows boxes with obnoxious firewall rules. Downgrade and try + * again. */ +#if defined(_WIN32) && defined(USE_BUFFEREVENTS) + if (torcfg->disable_iocp == 0) { + log_warn(LD_GENERAL, "Unable to initialize Libevent. Trying again " + "with IOCP disabled."); + } else +#endif + { + log_warn(LD_GENERAL, "Unable to initialize Libevent. Trying again."); + } + + torcfg->disable_iocp = 1; + goto retry; + } } #else the_event_base = event_init(); @@ -212,6 +276,10 @@ tor_libevent_initialize(void) "You have a *VERY* old version of libevent. It is likely to be buggy; " "please build Tor with a more recent version."); #endif + +#ifdef USE_BUFFEREVENTS + tor_libevent_set_tick_timeout(torcfg->msec_per_tick); +#endif } /** Return the current Libevent event base that we're set up to use. */ @@ -257,7 +325,7 @@ tor_decode_libevent_version(const char *v) /* Try the new preferred "1.4.11-stable" format. * Also accept "1.4.14b-stable". */ - fields = sscanf(v, "%u.%u.%u%c%c", &major, &minor, &patchlevel, &c, &e); + fields = tor_sscanf(v, "%u.%u.%u%c%c", &major, &minor, &patchlevel, &c, &e); if (fields == 3 || ((fields == 4 || fields == 5 ) && (c == '-' || c == '_')) || (fields == 5 && TOR_ISALPHA(c) && (e == '-' || e == '_'))) { @@ -265,7 +333,7 @@ tor_decode_libevent_version(const char *v) } /* Try the old "1.3e" format. */ - fields = sscanf(v, "%u.%u%c%c", &major, &minor, &c, &extra); + fields = tor_sscanf(v, "%u.%u%c%c", &major, &minor, &c, &extra); if (fields == 3 && TOR_ISALPHA(c)) { return V_OLD(major, minor, c); } else if (fields == 2) { @@ -569,3 +637,90 @@ periodic_timer_free(periodic_timer_t *timer) tor_free(timer); } +#ifdef USE_BUFFEREVENTS +static const struct timeval *one_tick = NULL; +/** + * Return a special timeout to be passed whenever libevent's O(1) timeout + * implementation should be used. Only use this when the timer is supposed + * to fire after msec_per_tick ticks have elapsed. +*/ +const struct timeval * +tor_libevent_get_one_tick_timeout(void) +{ + tor_assert(one_tick); + return one_tick; +} + +/** Initialize the common timeout that we'll use to refill the buckets every + * time a tick elapses. */ +static void +tor_libevent_set_tick_timeout(int msec_per_tick) +{ + struct event_base *base = tor_libevent_get_base(); + struct timeval tv; + + tor_assert(! one_tick); + tv.tv_sec = msec_per_tick / 1000; + tv.tv_usec = (msec_per_tick % 1000) * 1000; + one_tick = event_base_init_common_timeout(base, &tv); +} + +static struct bufferevent * +tor_get_root_bufferevent(struct bufferevent *bev) +{ + struct bufferevent *u; + while ((u = bufferevent_get_underlying(bev)) != NULL) + bev = u; + return bev; +} + +int +tor_set_bufferevent_rate_limit(struct bufferevent *bev, + struct ev_token_bucket_cfg *cfg) +{ + return bufferevent_set_rate_limit(tor_get_root_bufferevent(bev), cfg); +} + +int +tor_add_bufferevent_to_rate_limit_group(struct bufferevent *bev, + struct bufferevent_rate_limit_group *g) +{ + return bufferevent_add_to_rate_limit_group(tor_get_root_bufferevent(bev), g); +} +#endif + +#if defined(LIBEVENT_VERSION_NUMBER) && LIBEVENT_VERSION_NUMBER >= V(2,1,1) +void +tor_gettimeofday_cached(struct timeval *tv) +{ + event_base_gettimeofday_cached(the_event_base, tv); +} +void +tor_gettimeofday_cache_clear(void) +{ + event_base_update_cache_time(the_event_base); +} +#else +/** Cache the current hi-res time; the cache gets reset when libevent + * calls us. */ +static struct timeval cached_time_hires = {0, 0}; + +/** Return a fairly recent view of the current time. */ +void +tor_gettimeofday_cached(struct timeval *tv) +{ + if (cached_time_hires.tv_sec == 0) { + tor_gettimeofday(&cached_time_hires); + } + *tv = cached_time_hires; +} + +/** Reset the cached view of the current time, so that the next time we try + * to learn it, we will get an up-to-date value. */ +void +tor_gettimeofday_cache_clear(void) +{ + cached_time_hires.tv_sec = 0; +} +#endif + diff --git a/src/common/compat_libevent.h b/src/common/compat_libevent.h index 89b256396b..56285ef80d 100644 --- a/src/common/compat_libevent.h +++ b/src/common/compat_libevent.h @@ -1,4 +1,4 @@ -/* Copyright (c) 2009, The Tor Project, Inc. */ +/* Copyright (c) 2009-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #ifndef _TOR_COMPAT_LIBEVENT_H @@ -8,6 +8,11 @@ struct event; struct event_base; +#ifdef USE_BUFFEREVENTS +struct bufferevent; +struct ev_token_bucket_cfg; +struct bufferevent_rate_limit_group; +#endif #ifdef HAVE_EVENT2_EVENT_H #include <event2/util.h> @@ -54,7 +59,19 @@ struct timeval; int tor_event_base_loopexit(struct event_base *base, struct timeval *tv); #endif -void tor_libevent_initialize(void); +/** Defines a configuration for using libevent with Tor: passed as an argument + * to tor_libevent_initialize() to describe how we want to set up. */ +typedef struct tor_libevent_cfg { + /** Flag: if true, disable IOCP (assuming that it could be enabled). */ + int disable_iocp; + /** How many CPUs should we use (relevant only with IOCP). */ + int num_cpus; + /** How many milliseconds should we allow between updating bandwidth limits? + * (relevant only with bufferevents). */ + int msec_per_tick; +} tor_libevent_cfg; + +void tor_libevent_initialize(tor_libevent_cfg *cfg); struct event_base *tor_libevent_get_base(void); const char *tor_libevent_get_method(void); void tor_check_libevent_version(const char *m, int server, @@ -62,5 +79,17 @@ void tor_check_libevent_version(const char *m, int server, void tor_check_libevent_header_compatibility(void); const char *tor_libevent_get_version_str(void); +#ifdef USE_BUFFEREVENTS +const struct timeval *tor_libevent_get_one_tick_timeout(void); +int tor_libevent_using_iocp_bufferevents(void); +int tor_set_bufferevent_rate_limit(struct bufferevent *bev, + struct ev_token_bucket_cfg *cfg); +int tor_add_bufferevent_to_rate_limit_group(struct bufferevent *bev, + struct bufferevent_rate_limit_group *g); +#endif + +void tor_gettimeofday_cached(struct timeval *tv); +void tor_gettimeofday_cache_clear(void); + #endif diff --git a/src/common/container.c b/src/common/container.c index c047562cd0..ede98eca5a 100644 --- a/src/common/container.c +++ b/src/common/container.c @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -29,7 +29,7 @@ /** Allocate and return an empty smartlist. */ smartlist_t * -smartlist_create(void) +smartlist_new(void) { smartlist_t *sl = tor_malloc(sizeof(smartlist_t)); sl->num_used = 0; @@ -296,7 +296,6 @@ smartlist_subtract(smartlist_t *sl1, const smartlist_t *sl2) /** Remove the <b>idx</b>th element of sl; if idx is not the last * element, swap the last element of sl into the <b>idx</b>th space. - * Return the old value of the <b>idx</b>th element. */ void smartlist_del(smartlist_t *sl, int idx) @@ -1049,7 +1048,7 @@ digestmap_set(digestmap_t *map, const char *key, void *val) * the hash table that we do in the unoptimized code above. (Each of * HT_INSERT and HT_FIND calls HT_SET_HASH and HT_FIND_P.) */ - _HT_FIND_OR_INSERT(digestmap_impl, node, digestmap_entry_hash, &(map->head), + HT_FIND_OR_INSERT_(digestmap_impl, node, digestmap_entry_hash, &(map->head), digestmap_entry_t, &search, ptr, { /* we found an entry. */ @@ -1063,7 +1062,7 @@ digestmap_set(digestmap_t *map, const char *key, void *val) tor_malloc_zero(sizeof(digestmap_entry_t)); memcpy(newent->key, key, DIGEST_LEN); newent->val = val; - _HT_FOI_INSERT(node, &(map->head), &search, newent, ptr); + HT_FOI_INSERT_(node, &(map->head), &search, newent, ptr); return NULL; }); #endif @@ -1376,14 +1375,14 @@ digestmap_free(digestmap_t *map, void (*free_val)(void*)) void strmap_assert_ok(const strmap_t *map) { - tor_assert(!_strmap_impl_HT_REP_IS_BAD(&map->head)); + tor_assert(!strmap_impl_HT_REP_IS_BAD_(&map->head)); } /** Fail with an assertion error if anything has gone wrong with the internal * representation of <b>map</b>. */ void digestmap_assert_ok(const digestmap_t *map) { - tor_assert(!_digestmap_impl_HT_REP_IS_BAD(&map->head)); + tor_assert(!digestmap_impl_HT_REP_IS_BAD_(&map->head)); } /** Return true iff <b>map</b> has no entries. */ diff --git a/src/common/container.h b/src/common/container.h index 4a6eba789d..dab3b83f37 100644 --- a/src/common/container.h +++ b/src/common/container.h @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #ifndef _TOR_CONTAINER_H @@ -26,7 +26,7 @@ typedef struct smartlist_t { /** @} */ } smartlist_t; -smartlist_t *smartlist_create(void); +smartlist_t *smartlist_new(void); void smartlist_free(smartlist_t *sl); void smartlist_clear(smartlist_t *sl); void smartlist_add(smartlist_t *sl, void *element); @@ -35,19 +35,14 @@ void smartlist_remove(smartlist_t *sl, const void *element); void *smartlist_pop_last(smartlist_t *sl); void smartlist_reverse(smartlist_t *sl); void smartlist_string_remove(smartlist_t *sl, const char *element); -int smartlist_isin(const smartlist_t *sl, const void *element) ATTR_PURE; -int smartlist_string_isin(const smartlist_t *sl, const char *element) - ATTR_PURE; -int smartlist_string_pos(const smartlist_t *, const char *elt) ATTR_PURE; -int smartlist_string_isin_case(const smartlist_t *sl, const char *element) - ATTR_PURE; -int smartlist_string_num_isin(const smartlist_t *sl, int num) ATTR_PURE; -int smartlist_strings_eq(const smartlist_t *sl1, const smartlist_t *sl2) - ATTR_PURE; -int smartlist_digest_isin(const smartlist_t *sl, const char *element) - ATTR_PURE; -int smartlist_overlap(const smartlist_t *sl1, const smartlist_t *sl2) - ATTR_PURE; +int smartlist_isin(const smartlist_t *sl, const void *element); +int smartlist_string_isin(const smartlist_t *sl, const char *element); +int smartlist_string_pos(const smartlist_t *, const char *elt); +int smartlist_string_isin_case(const smartlist_t *sl, const char *element); +int smartlist_string_num_isin(const smartlist_t *sl, int num); +int smartlist_strings_eq(const smartlist_t *sl1, const smartlist_t *sl2); +int smartlist_digest_isin(const smartlist_t *sl, const char *element); +int smartlist_overlap(const smartlist_t *sl1, const smartlist_t *sl2); void smartlist_intersect(smartlist_t *sl1, const smartlist_t *sl2); void smartlist_subtract(smartlist_t *sl1, const smartlist_t *sl2); @@ -55,14 +50,14 @@ void smartlist_subtract(smartlist_t *sl1, const smartlist_t *sl2); #ifdef DEBUG_SMARTLIST /** Return the number of items in sl. */ -static INLINE int smartlist_len(const smartlist_t *sl) ATTR_PURE; +static INLINE int smartlist_len(const smartlist_t *sl); static INLINE int smartlist_len(const smartlist_t *sl) { tor_assert(sl); return (sl)->num_used; } /** Return the <b>idx</b>th element of sl. */ -static INLINE void *smartlist_get(const smartlist_t *sl, int idx) ATTR_PURE; +static INLINE void *smartlist_get(const smartlist_t *sl, int idx); static INLINE void *smartlist_get(const smartlist_t *sl, int idx) { tor_assert(sl); tor_assert(idx>=0); @@ -114,8 +109,7 @@ void smartlist_uniq_strings(smartlist_t *sl); void smartlist_uniq_digests(smartlist_t *sl); void smartlist_uniq_digests256(smartlist_t *sl); void *smartlist_bsearch(smartlist_t *sl, const void *key, - int (*compare)(const void *key, const void **member)) - ATTR_PURE; + int (*compare)(const void *key, const void **member)); int smartlist_bsearch_idx(const smartlist_t *sl, const void *key, int (*compare)(const void *key, const void **member), int *found_out); @@ -148,9 +142,9 @@ char *smartlist_join_strings2(smartlist_t *sl, const char *join, /** Iterate over the items in a smartlist <b>sl</b>, in order. For each item, * assign it to a new local variable of type <b>type</b> named <b>var</b>, and - * execute the statement <b>cmd</b>. Inside the loop, the loop index can - * be accessed as <b>var</b>_sl_idx and the length of the list can be accessed - * as <b>var</b>_sl_len. + * execute the statements inside the loop body. Inside the loop, the loop + * index can be accessed as <b>var</b>_sl_idx and the length of the list can + * be accessed as <b>var</b>_sl_len. * * NOTE: Do not change the length of the list while the loop is in progress, * unless you adjust the _sl_len variable correspondingly. See second example @@ -159,23 +153,21 @@ char *smartlist_join_strings2(smartlist_t *sl, const char *join, * Example use: * <pre> * smartlist_t *list = smartlist_split("A:B:C", ":", 0, 0); - * SMARTLIST_FOREACH(list, char *, cp, - * { + * SMARTLIST_FOREACH_BEGIN(list, char *, cp) { * printf("%d: %s\n", cp_sl_idx, cp); * tor_free(cp); - * }); + * } SMARTLIST_FOREACH_END(cp); * smartlist_free(list); * </pre> * * Example use (advanced): * <pre> - * SMARTLIST_FOREACH(list, char *, cp, - * { + * SMARTLIST_FOREACH_BEGIN(list, char *, cp) { * if (!strcmp(cp, "junk")) { * tor_free(cp); * SMARTLIST_DEL_CURRENT(list, cp); * } - * }); + * } SMARTLIST_FOREACH_END(cp); * </pre> */ /* Note: these macros use token pasting, and reach into smartlist internals. @@ -224,6 +216,14 @@ char *smartlist_join_strings2(smartlist_t *sl, const char *join, var = NULL; \ } STMT_END +/** + * An alias for SMARTLIST_FOREACH_BEGIN and SMARTLIST_FOREACH_END, using + * <b>cmd</b> as the loop body. This wrapper is here for convenience with + * very short loops. + * + * By convention, we do not use this for loops which nest, or for loops over + * 10 lines or so. Use SMARTLIST_FOREACH_{BEGIN,END} for those. + */ #define SMARTLIST_FOREACH(sl, type, var, cmd) \ SMARTLIST_FOREACH_BEGIN(sl,type,var) { \ cmd; \ diff --git a/src/common/crypto.c b/src/common/crypto.c index 88e48ef843..30990ecc8f 100644 --- a/src/common/crypto.c +++ b/src/common/crypto.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001, Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -12,9 +12,10 @@ #include "orconfig.h" -#ifdef MS_WINDOWS -#define WIN32_WINNT 0x400 -#define _WIN32_WINNT 0x400 +#ifdef _WIN32 +#ifndef _WIN32_WINNT +#define _WIN32_WINNT 0x0501 +#endif #define WIN32_LEAN_AND_MEAN #include <windows.h> #include <wincrypt.h> @@ -56,18 +57,20 @@ #include "container.h" #include "compat.h" -#if OPENSSL_VERSION_NUMBER < 0x00907000l +#if OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(0,9,7) #error "We require OpenSSL >= 0.9.7" #endif -#include <openssl/engine.h> - #ifdef ANDROID /* Android's OpenSSL seems to have removed all of its Engine support. */ #define DISABLE_ENGINES #endif -#if OPENSSL_VERSION_NUMBER < 0x00908000l +/** Longest recognized */ +#define MAX_DNS_LABEL_SIZE 63 + +#if OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(0,9,8) && \ + !defined(RUNNING_DOXYGEN) /** @{ */ /** On OpenSSL versions before 0.9.8, there is no working SHA256 * implementation, so we use Tom St Denis's nice speedy one, slightly adapted @@ -104,23 +107,24 @@ static int _n_openssl_mutexes = 0; #endif /** A public key, or a public/private key-pair. */ -struct crypto_pk_env_t +struct crypto_pk_t { int refs; /**< reference count, so we don't have to copy keys */ RSA *key; /**< The key itself */ }; /** Key and stream information for a stream cipher. */ -struct crypto_cipher_env_t +struct crypto_cipher_t { char key[CIPHER_KEY_LEN]; /**< The raw key. */ + char iv[CIPHER_IV_LEN]; /**< The initial IV. */ aes_cnt_cipher_t *cipher; /**< The key in format usable for counter-mode AES * encryption */ }; /** A structure to hold the first half (x, g^x) of a Diffie-Hellman handshake * while we're waiting for the second.*/ -struct crypto_dh_env_t { +struct crypto_dh_t { DH *dh; /**< The openssl DH object */ }; @@ -134,7 +138,6 @@ crypto_get_rsa_padding_overhead(int padding) { switch (padding) { - case RSA_NO_PADDING: return 0; case RSA_PKCS1_OAEP_PADDING: return 42; case RSA_PKCS1_PADDING: return 11; default: tor_assert(0); return -1; @@ -148,7 +151,6 @@ crypto_get_rsa_padding(int padding) { switch (padding) { - case PK_NO_PADDING: return RSA_NO_PADDING; case PK_PKCS1_PADDING: return RSA_PKCS1_PADDING; case PK_PKCS1_OAEP_PADDING: return RSA_PKCS1_OAEP_PADDING; default: tor_assert(0); return -1; @@ -274,6 +276,10 @@ crypto_global_init(int useAccel, const char *accelName, const char *accelDir) } else { log_info(LD_CRYPTO, "NOT using OpenSSL engine support."); } + + evaluate_evp_for_aes(-1); + evaluate_ctr_for_aes(); + return crypto_seed_rng(1); } return 0; @@ -286,61 +292,30 @@ crypto_thread_cleanup(void) ERR_remove_state(0); } -/** Uninitialize the crypto library. Return 0 on success, -1 on failure. - */ -int -crypto_global_cleanup(void) -{ - EVP_cleanup(); - ERR_remove_state(0); - ERR_free_strings(); - -#ifndef DISABLE_ENGINES - ENGINE_cleanup(); -#endif - - CONF_modules_unload(1); - CRYPTO_cleanup_all_ex_data(); -#ifdef TOR_IS_MULTITHREADED - if (_n_openssl_mutexes) { - int n = _n_openssl_mutexes; - tor_mutex_t **ms = _openssl_mutexes; - int i; - _openssl_mutexes = NULL; - _n_openssl_mutexes = 0; - for (i=0;i<n;++i) { - tor_mutex_free(ms[i]); - } - tor_free(ms); - } -#endif - return 0; -} - -/** used by tortls.c: wrap an RSA* in a crypto_pk_env_t. */ -crypto_pk_env_t * -_crypto_new_pk_env_rsa(RSA *rsa) +/** used by tortls.c: wrap an RSA* in a crypto_pk_t. */ +crypto_pk_t * +_crypto_new_pk_from_rsa(RSA *rsa) { - crypto_pk_env_t *env; + crypto_pk_t *env; tor_assert(rsa); - env = tor_malloc(sizeof(crypto_pk_env_t)); + env = tor_malloc(sizeof(crypto_pk_t)); env->refs = 1; env->key = rsa; return env; } /** Helper, used by tor-checkkey.c and tor-gencert.c. Return the RSA from a - * crypto_pk_env_t. */ + * crypto_pk_t. */ RSA * -_crypto_pk_env_get_rsa(crypto_pk_env_t *env) +_crypto_pk_get_rsa(crypto_pk_t *env) { return env->key; } -/** used by tortls.c: get an equivalent EVP_PKEY* for a crypto_pk_env_t. Iff +/** used by tortls.c: get an equivalent EVP_PKEY* for a crypto_pk_t. Iff * private is set, include the private-key portion of the key. */ EVP_PKEY * -_crypto_pk_env_get_evp_pkey(crypto_pk_env_t *env, int private) +_crypto_pk_get_evp_pkey(crypto_pk_t *env, int private) { RSA *key = NULL; EVP_PKEY *pkey = NULL; @@ -365,10 +340,10 @@ _crypto_pk_env_get_evp_pkey(crypto_pk_env_t *env, int private) return NULL; } -/** Used by tortls.c: Get the DH* from a crypto_dh_env_t. +/** Used by tortls.c: Get the DH* from a crypto_dh_t. */ DH * -_crypto_dh_env_get_dh(crypto_dh_env_t *dh) +_crypto_dh_get_dh(crypto_dh_t *dh) { return dh->dh; } @@ -376,21 +351,21 @@ _crypto_dh_env_get_dh(crypto_dh_env_t *dh) /** Allocate and return storage for a public key. The key itself will not yet * be set. */ -crypto_pk_env_t * -crypto_new_pk_env(void) +crypto_pk_t * +crypto_pk_new(void) { RSA *rsa; rsa = RSA_new(); tor_assert(rsa); - return _crypto_new_pk_env_rsa(rsa); + return _crypto_new_pk_from_rsa(rsa); } /** Release a reference to an asymmetric key; when all the references * are released, free the key. */ void -crypto_free_pk_env(crypto_pk_env_t *env) +crypto_pk_free(crypto_pk_t *env) { if (!env) return; @@ -405,61 +380,52 @@ crypto_free_pk_env(crypto_pk_env_t *env) tor_free(env); } -/** Create a new symmetric cipher for a given key and encryption flag - * (1=encrypt, 0=decrypt). Return the crypto object on success; NULL - * on failure. +/** Allocate and return a new symmetric cipher using the provided key and iv. + * The key is CIPHER_KEY_LEN bytes; the IV is CIPHER_IV_LEN bytes. If you + * provide NULL in place of either one, it is generated at random. */ -crypto_cipher_env_t * -crypto_create_init_cipher(const char *key, int encrypt_mode) +crypto_cipher_t * +crypto_cipher_new_with_iv(const char *key, const char *iv) { - int r; - crypto_cipher_env_t *crypto = NULL; + crypto_cipher_t *env; - if (! (crypto = crypto_new_cipher_env())) { - log_warn(LD_CRYPTO, "Unable to allocate crypto object"); - return NULL; - } + env = tor_malloc_zero(sizeof(crypto_cipher_t)); - crypto_cipher_set_key(crypto, key); - - if (encrypt_mode) - r = crypto_cipher_encrypt_init_cipher(crypto); + if (key == NULL) + crypto_rand(env->key, CIPHER_KEY_LEN); + else + memcpy(env->key, key, CIPHER_KEY_LEN); + if (iv == NULL) + crypto_rand(env->iv, CIPHER_IV_LEN); else - r = crypto_cipher_decrypt_init_cipher(crypto); + memcpy(env->iv, iv, CIPHER_IV_LEN); - if (r) - goto error; - return crypto; + env->cipher = aes_new_cipher(env->key, env->iv); - error: - if (crypto) - crypto_free_cipher_env(crypto); - return NULL; + return env; } -/** Allocate and return a new symmetric cipher. - */ -crypto_cipher_env_t * -crypto_new_cipher_env(void) +/** Return a new crypto_cipher_t with the provided <b>key</b> and an IV of all + * zero bytes. */ +crypto_cipher_t * +crypto_cipher_new(const char *key) { - crypto_cipher_env_t *env; - - env = tor_malloc_zero(sizeof(crypto_cipher_env_t)); - env->cipher = aes_new_cipher(); - return env; + char zeroiv[CIPHER_IV_LEN]; + memset(zeroiv, 0, sizeof(zeroiv)); + return crypto_cipher_new_with_iv(key, zeroiv); } /** Free a symmetric cipher. */ void -crypto_free_cipher_env(crypto_cipher_env_t *env) +crypto_cipher_free(crypto_cipher_t *env) { if (!env) return; tor_assert(env->cipher); - aes_free_cipher(env->cipher); - memset(env, 0, sizeof(crypto_cipher_env_t)); + aes_cipher_free(env->cipher); + memwipe(env, 0, sizeof(crypto_cipher_t)); tor_free(env); } @@ -469,13 +435,13 @@ crypto_free_cipher_env(crypto_cipher_env_t *env) * Return 0 on success, -1 on failure. */ int -crypto_pk_generate_key_with_bits(crypto_pk_env_t *env, int bits) +crypto_pk_generate_key_with_bits(crypto_pk_t *env, int bits) { tor_assert(env); if (env->key) RSA_free(env->key); -#if OPENSSL_VERSION_NUMBER < 0x00908000l +#if OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(0,9,8) /* In OpenSSL 0.9.7, RSA_generate_key is all we have. */ env->key = RSA_generate_key(bits, 65537, NULL, NULL); #else @@ -516,7 +482,7 @@ crypto_pk_generate_key_with_bits(crypto_pk_env_t *env, int bits) */ /* Used here, and used for testing. */ int -crypto_pk_read_private_key_from_string(crypto_pk_env_t *env, +crypto_pk_read_private_key_from_string(crypto_pk_t *env, const char *s, ssize_t len) { BIO *b; @@ -548,7 +514,7 @@ crypto_pk_read_private_key_from_string(crypto_pk_env_t *env, * <b>keyfile</b> into <b>env</b>. Return 0 on success, -1 on failure. */ int -crypto_pk_read_private_key_from_filename(crypto_pk_env_t *env, +crypto_pk_read_private_key_from_filename(crypto_pk_t *env, const char *keyfile) { char *contents; @@ -563,7 +529,7 @@ crypto_pk_read_private_key_from_filename(crypto_pk_env_t *env, /* Try to parse it. */ r = crypto_pk_read_private_key_from_string(env, contents, -1); - memset(contents, 0, strlen(contents)); + memwipe(contents, 0, strlen(contents)); tor_free(contents); if (r) return -1; /* read_private_key_from_string already warned, so we don't.*/ @@ -577,7 +543,7 @@ crypto_pk_read_private_key_from_filename(crypto_pk_env_t *env, /** Helper function to implement crypto_pk_write_*_key_to_string. */ static int -crypto_pk_write_key_to_string_impl(crypto_pk_env_t *env, char **dest, +crypto_pk_write_key_to_string_impl(crypto_pk_t *env, char **dest, size_t *len, int is_public) { BUF_MEM *buf; @@ -625,7 +591,7 @@ crypto_pk_write_key_to_string_impl(crypto_pk_env_t *env, char **dest, * failure, return -1. */ int -crypto_pk_write_public_key_to_string(crypto_pk_env_t *env, char **dest, +crypto_pk_write_public_key_to_string(crypto_pk_t *env, char **dest, size_t *len) { return crypto_pk_write_key_to_string_impl(env, dest, len, 1); @@ -637,7 +603,7 @@ crypto_pk_write_public_key_to_string(crypto_pk_env_t *env, char **dest, * failure, return -1. */ int -crypto_pk_write_private_key_to_string(crypto_pk_env_t *env, char **dest, +crypto_pk_write_private_key_to_string(crypto_pk_t *env, char **dest, size_t *len) { return crypto_pk_write_key_to_string_impl(env, dest, len, 0); @@ -648,7 +614,7 @@ crypto_pk_write_private_key_to_string(crypto_pk_env_t *env, char **dest, * failure. */ int -crypto_pk_read_public_key_from_string(crypto_pk_env_t *env, const char *src, +crypto_pk_read_public_key_from_string(crypto_pk_t *env, const char *src, size_t len) { BIO *b; @@ -679,7 +645,7 @@ crypto_pk_read_public_key_from_string(crypto_pk_env_t *env, const char *src, * PEM-encoded. Return 0 on success, -1 on failure. */ int -crypto_pk_write_private_key_to_filename(crypto_pk_env_t *env, +crypto_pk_write_private_key_to_filename(crypto_pk_t *env, const char *fname) { BIO *bio; @@ -705,7 +671,7 @@ crypto_pk_write_private_key_to_filename(crypto_pk_env_t *env, s[len]='\0'; r = write_str_to_file(fname, s, 0); BIO_free(bio); - memset(s, 0, strlen(s)); + memwipe(s, 0, strlen(s)); tor_free(s); return r; } @@ -713,7 +679,7 @@ crypto_pk_write_private_key_to_filename(crypto_pk_env_t *env, /** Return true iff <b>env</b> has a valid key. */ int -crypto_pk_check_key(crypto_pk_env_t *env) +crypto_pk_check_key(crypto_pk_t *env) { int r; tor_assert(env); @@ -727,7 +693,7 @@ crypto_pk_check_key(crypto_pk_env_t *env) /** Return true iff <b>key</b> contains the private-key portion of the RSA * key. */ int -crypto_pk_key_is_private(const crypto_pk_env_t *key) +crypto_pk_key_is_private(const crypto_pk_t *key) { tor_assert(key); return PRIVATE_KEY_OK(key); @@ -737,7 +703,7 @@ crypto_pk_key_is_private(const crypto_pk_env_t *key) * equals 65537. */ int -crypto_pk_public_exponent_ok(crypto_pk_env_t *env) +crypto_pk_public_exponent_ok(crypto_pk_t *env) { tor_assert(env); tor_assert(env->key); @@ -749,7 +715,7 @@ crypto_pk_public_exponent_ok(crypto_pk_env_t *env) * if a==b, and 1 if a\>b. */ int -crypto_pk_cmp_keys(crypto_pk_env_t *a, crypto_pk_env_t *b) +crypto_pk_cmp_keys(crypto_pk_t *a, crypto_pk_t *b) { int result; @@ -769,7 +735,7 @@ crypto_pk_cmp_keys(crypto_pk_env_t *a, crypto_pk_env_t *b) /** Return the size of the public key modulus in <b>env</b>, in bytes. */ size_t -crypto_pk_keysize(crypto_pk_env_t *env) +crypto_pk_keysize(crypto_pk_t *env) { tor_assert(env); tor_assert(env->key); @@ -779,7 +745,7 @@ crypto_pk_keysize(crypto_pk_env_t *env) /** Return the size of the public key modulus of <b>env</b>, in bits. */ int -crypto_pk_num_bits(crypto_pk_env_t *env) +crypto_pk_num_bits(crypto_pk_t *env) { tor_assert(env); tor_assert(env->key); @@ -790,8 +756,8 @@ crypto_pk_num_bits(crypto_pk_env_t *env) /** Increase the reference count of <b>env</b>, and return it. */ -crypto_pk_env_t * -crypto_pk_dup_key(crypto_pk_env_t *env) +crypto_pk_t * +crypto_pk_dup_key(crypto_pk_t *env) { tor_assert(env); tor_assert(env->key); @@ -801,8 +767,8 @@ crypto_pk_dup_key(crypto_pk_env_t *env) } /** Make a real honest-to-goodness copy of <b>env</b>, and return it. */ -crypto_pk_env_t * -crypto_pk_copy_full(crypto_pk_env_t *env) +crypto_pk_t * +crypto_pk_copy_full(crypto_pk_t *env) { RSA *new_key; int privatekey = 0; @@ -825,7 +791,7 @@ crypto_pk_copy_full(crypto_pk_env_t *env) return NULL; } - return _crypto_new_pk_env_rsa(new_key); + return _crypto_new_pk_from_rsa(new_key); } /** Encrypt <b>fromlen</b> bytes from <b>from</b> with the public key @@ -837,7 +803,7 @@ crypto_pk_copy_full(crypto_pk_env_t *env) * at least the length of the modulus of <b>env</b>. */ int -crypto_pk_public_encrypt(crypto_pk_env_t *env, char *to, size_t tolen, +crypto_pk_public_encrypt(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen, int padding) { int r; @@ -866,7 +832,7 @@ crypto_pk_public_encrypt(crypto_pk_env_t *env, char *to, size_t tolen, * at least the length of the modulus of <b>env</b>. */ int -crypto_pk_private_decrypt(crypto_pk_env_t *env, char *to, +crypto_pk_private_decrypt(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen, int padding, int warnOnFailure) @@ -903,7 +869,7 @@ crypto_pk_private_decrypt(crypto_pk_env_t *env, char *to, * at least the length of the modulus of <b>env</b>. */ int -crypto_pk_public_checksig(crypto_pk_env_t *env, char *to, +crypto_pk_public_checksig(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen) { @@ -930,7 +896,7 @@ crypto_pk_public_checksig(crypto_pk_env_t *env, char *to, * SHA1(data). Else return -1. */ int -crypto_pk_public_checksig_digest(crypto_pk_env_t *env, const char *data, +crypto_pk_public_checksig_digest(crypto_pk_t *env, const char *data, size_t datalen, const char *sig, size_t siglen) { char digest[DIGEST_LEN]; @@ -948,7 +914,7 @@ crypto_pk_public_checksig_digest(crypto_pk_env_t *env, const char *data, log_warn(LD_BUG, "couldn't compute digest"); return -1; } - buflen = crypto_pk_keysize(env)+1; + buflen = crypto_pk_keysize(env); buf = tor_malloc(buflen); r = crypto_pk_public_checksig(env,buf,buflen,sig,siglen); if (r != DIGEST_LEN) { @@ -975,7 +941,7 @@ crypto_pk_public_checksig_digest(crypto_pk_env_t *env, const char *data, * at least the length of the modulus of <b>env</b>. */ int -crypto_pk_private_sign(crypto_pk_env_t *env, char *to, size_t tolen, +crypto_pk_private_sign(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen) { int r; @@ -1007,7 +973,7 @@ crypto_pk_private_sign(crypto_pk_env_t *env, char *to, size_t tolen, * at least the length of the modulus of <b>env</b>. */ int -crypto_pk_private_sign_digest(crypto_pk_env_t *env, char *to, size_t tolen, +crypto_pk_private_sign_digest(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen) { int r; @@ -1015,7 +981,7 @@ crypto_pk_private_sign_digest(crypto_pk_env_t *env, char *to, size_t tolen, if (crypto_digest(digest,from,fromlen)<0) return -1; r = crypto_pk_private_sign(env,to,tolen,digest,DIGEST_LEN); - memset(digest, 0, sizeof(digest)); + memwipe(digest, 0, sizeof(digest)); return r; } @@ -1023,9 +989,6 @@ crypto_pk_private_sign_digest(crypto_pk_env_t *env, char *to, size_t tolen, * bytes of data from <b>from</b>, with padding type 'padding', * storing the results on <b>to</b>. * - * If no padding is used, the public key must be at least as large as - * <b>from</b>. - * * Returns the number of bytes written on success, -1 on failure. * * The encrypted data consists of: @@ -1037,7 +1000,7 @@ crypto_pk_private_sign_digest(crypto_pk_env_t *env, char *to, size_t tolen, * the source data encrypted in AES-CTR mode with the symmetric key. */ int -crypto_pk_public_hybrid_encrypt(crypto_pk_env_t *env, +crypto_pk_public_hybrid_encrypt(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen, @@ -1045,7 +1008,7 @@ crypto_pk_public_hybrid_encrypt(crypto_pk_env_t *env, { int overhead, outlen, r; size_t pkeylen, symlen; - crypto_cipher_env_t *cipher = NULL; + crypto_cipher_t *cipher = NULL; char *buf = NULL; tor_assert(env); @@ -1056,9 +1019,6 @@ crypto_pk_public_hybrid_encrypt(crypto_pk_env_t *env, overhead = crypto_get_rsa_padding_overhead(crypto_get_rsa_padding(padding)); pkeylen = crypto_pk_keysize(env); - if (padding == PK_NO_PADDING && fromlen < pkeylen) - return -1; - if (!force && fromlen+overhead <= pkeylen) { /* It all fits in a single encrypt. */ return crypto_pk_public_encrypt(env,to, @@ -1068,20 +1028,8 @@ crypto_pk_public_hybrid_encrypt(crypto_pk_env_t *env, tor_assert(tolen >= fromlen + overhead + CIPHER_KEY_LEN); tor_assert(tolen >= pkeylen); - cipher = crypto_new_cipher_env(); - if (!cipher) return -1; - if (crypto_cipher_generate_key(cipher)<0) - goto err; - /* You can't just run around RSA-encrypting any bitstream: if it's - * greater than the RSA key, then OpenSSL will happily encrypt, and - * later decrypt to the wrong value. So we set the first bit of - * 'cipher->key' to 0 if we aren't padding. This means that our - * symmetric key is really only 127 bits. - */ - if (padding == PK_NO_PADDING) - cipher->key[0] &= 0x7f; - if (crypto_cipher_encrypt_init_cipher(cipher)<0) - goto err; + cipher = crypto_cipher_new(NULL); /* generate a new key. */ + buf = tor_malloc(pkeylen+1); memcpy(buf, cipher->key, CIPHER_KEY_LEN); memcpy(buf+CIPHER_KEY_LEN, from, pkeylen-overhead-CIPHER_KEY_LEN); @@ -1097,23 +1045,22 @@ crypto_pk_public_hybrid_encrypt(crypto_pk_env_t *env, from+pkeylen-overhead-CIPHER_KEY_LEN, symlen); if (r<0) goto err; - memset(buf, 0, pkeylen); + memwipe(buf, 0, pkeylen); tor_free(buf); - crypto_free_cipher_env(cipher); + crypto_cipher_free(cipher); tor_assert(outlen+symlen < INT_MAX); return (int)(outlen + symlen); err: - if (buf) { - memset(buf, 0, pkeylen); - tor_free(buf); - } - if (cipher) crypto_free_cipher_env(cipher); + + memwipe(buf, 0, pkeylen); + tor_free(buf); + crypto_cipher_free(cipher); return -1; } /** Invert crypto_pk_public_hybrid_encrypt. */ int -crypto_pk_private_hybrid_decrypt(crypto_pk_env_t *env, +crypto_pk_private_hybrid_decrypt(crypto_pk_t *env, char *to, size_t tolen, const char *from, @@ -1122,7 +1069,7 @@ crypto_pk_private_hybrid_decrypt(crypto_pk_env_t *env, { int outlen, r; size_t pkeylen; - crypto_cipher_env_t *cipher = NULL; + crypto_cipher_t *cipher = NULL; char *buf = NULL; tor_assert(fromlen < SIZE_T_CEILING); @@ -1133,8 +1080,8 @@ crypto_pk_private_hybrid_decrypt(crypto_pk_env_t *env, warnOnFailure); } - buf = tor_malloc(pkeylen+1); - outlen = crypto_pk_private_decrypt(env,buf,pkeylen+1,from,pkeylen,padding, + buf = tor_malloc(pkeylen); + outlen = crypto_pk_private_decrypt(env,buf,pkeylen,from,pkeylen,padding, warnOnFailure); if (outlen<0) { log_fn(warnOnFailure?LOG_WARN:LOG_DEBUG, LD_CRYPTO, @@ -1146,7 +1093,7 @@ crypto_pk_private_hybrid_decrypt(crypto_pk_env_t *env, "No room for a symmetric key"); goto err; } - cipher = crypto_create_init_cipher(buf, 0); + cipher = crypto_cipher_new(buf); if (!cipher) { goto err; } @@ -1156,15 +1103,15 @@ crypto_pk_private_hybrid_decrypt(crypto_pk_env_t *env, r = crypto_cipher_decrypt(cipher, to+outlen, from+pkeylen, fromlen-pkeylen); if (r<0) goto err; - memset(buf,0,pkeylen); + memwipe(buf,0,pkeylen); tor_free(buf); - crypto_free_cipher_env(cipher); + crypto_cipher_free(cipher); tor_assert(outlen + fromlen < INT_MAX); return (int)(outlen + (fromlen-pkeylen)); err: - memset(buf,0,pkeylen); + memwipe(buf,0,pkeylen); tor_free(buf); - if (cipher) crypto_free_cipher_env(cipher); + crypto_cipher_free(cipher); return -1; } @@ -1172,7 +1119,7 @@ crypto_pk_private_hybrid_decrypt(crypto_pk_env_t *env, * Return -1 on error, or the number of characters used on success. */ int -crypto_pk_asn1_encode(crypto_pk_env_t *pk, char *dest, size_t dest_len) +crypto_pk_asn1_encode(crypto_pk_t *pk, char *dest, size_t dest_len) { int len; unsigned char *buf, *cp; @@ -1197,14 +1144,11 @@ crypto_pk_asn1_encode(crypto_pk_env_t *pk, char *dest, size_t dest_len) /** Decode an ASN.1-encoded public key from <b>str</b>; return the result on * success and NULL on failure. */ -crypto_pk_env_t * +crypto_pk_t * crypto_pk_asn1_decode(const char *str, size_t len) { RSA *rsa; unsigned char *buf; - /* This ifdef suppresses a type warning. Take out the first case once - * everybody is using OpenSSL 0.9.7 or later. - */ const unsigned char *cp; cp = buf = tor_malloc(len); memcpy(buf,str,len); @@ -1214,7 +1158,7 @@ crypto_pk_asn1_decode(const char *str, size_t len) crypto_log_errors(LOG_WARN,"decoding public key"); return NULL; } - return _crypto_new_pk_env_rsa(rsa); + return _crypto_new_pk_from_rsa(rsa); } /** Given a private or public key <b>pk</b>, put a SHA1 hash of the @@ -1222,7 +1166,7 @@ crypto_pk_asn1_decode(const char *str, size_t len) * Return 0 on success, -1 on failure. */ int -crypto_pk_get_digest(crypto_pk_env_t *pk, char *digest_out) +crypto_pk_get_digest(crypto_pk_t *pk, char *digest_out) { unsigned char *buf, *bufp; int len; @@ -1245,6 +1189,32 @@ crypto_pk_get_digest(crypto_pk_env_t *pk, char *digest_out) return 0; } +/** Compute all digests of the DER encoding of <b>pk</b>, and store them + * in <b>digests_out</b>. Return 0 on success, -1 on failure. */ +int +crypto_pk_get_all_digests(crypto_pk_t *pk, digests_t *digests_out) +{ + unsigned char *buf, *bufp; + int len; + + len = i2d_RSAPublicKey(pk->key, NULL); + if (len < 0) + return -1; + buf = bufp = tor_malloc(len+1); + len = i2d_RSAPublicKey(pk->key, &bufp); + if (len < 0) { + crypto_log_errors(LOG_WARN,"encoding public key"); + tor_free(buf); + return -1; + } + if (crypto_digest_all(digests_out, (char*)buf, len) < 0) { + tor_free(buf); + return -1; + } + tor_free(buf); + return 0; +} + /** Copy <b>in</b> to the <b>outlen</b>-byte buffer <b>out</b>, adding spaces * every four spaces. */ /* static */ void @@ -1276,7 +1246,7 @@ add_spaces_to_fp(char *out, size_t outlen, const char *in) * If <b>add_space</b> is false, omit the spaces. */ int -crypto_pk_get_fingerprint(crypto_pk_env_t *pk, char *fp_out, int add_space) +crypto_pk_get_fingerprint(crypto_pk_t *pk, char *fp_out, int add_space) { char digest[DIGEST_LEN]; char hexdigest[HEX_DIGEST_LEN+1]; @@ -1311,87 +1281,20 @@ crypto_pk_check_fingerprint_syntax(const char *s) /* symmetric crypto */ -/** Generate a new random key for the symmetric cipher in <b>env</b>. - * Return 0 on success, -1 on failure. Does not initialize the cipher. - */ -int -crypto_cipher_generate_key(crypto_cipher_env_t *env) -{ - tor_assert(env); - - return crypto_rand(env->key, CIPHER_KEY_LEN); -} - -/** Set the symmetric key for the cipher in <b>env</b> to the first - * CIPHER_KEY_LEN bytes of <b>key</b>. Does not initialize the cipher. - */ -void -crypto_cipher_set_key(crypto_cipher_env_t *env, const char *key) -{ - tor_assert(env); - tor_assert(key); - - memcpy(env->key, key, CIPHER_KEY_LEN); -} - -/** Generate an initialization vector for our AES-CTR cipher; store it - * in the first CIPHER_IV_LEN bytes of <b>iv_out</b>. */ -void -crypto_cipher_generate_iv(char *iv_out) -{ - crypto_rand(iv_out, CIPHER_IV_LEN); -} - -/** Adjust the counter of <b>env</b> to point to the first byte of the block - * corresponding to the encryption of the CIPHER_IV_LEN bytes at - * <b>iv</b>. */ -int -crypto_cipher_set_iv(crypto_cipher_env_t *env, const char *iv) -{ - tor_assert(env); - tor_assert(iv); - aes_set_iv(env->cipher, iv); - return 0; -} - /** Return a pointer to the key set for the cipher in <b>env</b>. */ const char * -crypto_cipher_get_key(crypto_cipher_env_t *env) +crypto_cipher_get_key(crypto_cipher_t *env) { return env->key; } -/** Initialize the cipher in <b>env</b> for encryption. Return 0 on - * success, -1 on failure. - */ -int -crypto_cipher_encrypt_init_cipher(crypto_cipher_env_t *env) -{ - tor_assert(env); - - aes_set_key(env->cipher, env->key, CIPHER_KEY_LEN*8); - return 0; -} - -/** Initialize the cipher in <b>env</b> for decryption. Return 0 on - * success, -1 on failure. - */ -int -crypto_cipher_decrypt_init_cipher(crypto_cipher_env_t *env) -{ - tor_assert(env); - - aes_set_key(env->cipher, env->key, CIPHER_KEY_LEN*8); - return 0; -} - /** Encrypt <b>fromlen</b> bytes from <b>from</b> using the cipher * <b>env</b>; on success, store the result to <b>to</b> and return 0. * On failure, return -1. */ int -crypto_cipher_encrypt(crypto_cipher_env_t *env, char *to, +crypto_cipher_encrypt(crypto_cipher_t *env, char *to, const char *from, size_t fromlen) { tor_assert(env); @@ -1410,7 +1313,7 @@ crypto_cipher_encrypt(crypto_cipher_env_t *env, char *to, * On failure, return -1. */ int -crypto_cipher_decrypt(crypto_cipher_env_t *env, char *to, +crypto_cipher_decrypt(crypto_cipher_t *env, char *to, const char *from, size_t fromlen) { tor_assert(env); @@ -1426,7 +1329,7 @@ crypto_cipher_decrypt(crypto_cipher_env_t *env, char *to, * on success, return 0. On failure, return -1. */ int -crypto_cipher_crypt_inplace(crypto_cipher_env_t *env, char *buf, size_t len) +crypto_cipher_crypt_inplace(crypto_cipher_t *env, char *buf, size_t len) { tor_assert(len < SIZE_T_CEILING); aes_crypt_inplace(env->cipher, buf, len); @@ -1434,20 +1337,17 @@ crypto_cipher_crypt_inplace(crypto_cipher_env_t *env, char *buf, size_t len) } /** Encrypt <b>fromlen</b> bytes (at least 1) from <b>from</b> with the key in - * <b>cipher</b> to the buffer in <b>to</b> of length + * <b>key</b> to the buffer in <b>to</b> of length * <b>tolen</b>. <b>tolen</b> must be at least <b>fromlen</b> plus * CIPHER_IV_LEN bytes for the initialization vector. On success, return the * number of bytes written, on failure, return -1. - * - * This function adjusts the current position of the counter in <b>cipher</b> - * to immediately after the encrypted data. */ int -crypto_cipher_encrypt_with_iv(crypto_cipher_env_t *cipher, +crypto_cipher_encrypt_with_iv(const char *key, char *to, size_t tolen, const char *from, size_t fromlen) { - tor_assert(cipher); + crypto_cipher_t *cipher; tor_assert(from); tor_assert(to); tor_assert(fromlen < INT_MAX); @@ -1457,28 +1357,27 @@ crypto_cipher_encrypt_with_iv(crypto_cipher_env_t *cipher, if (tolen < fromlen + CIPHER_IV_LEN) return -1; - crypto_cipher_generate_iv(to); - if (crypto_cipher_set_iv(cipher, to)<0) - return -1; + cipher = crypto_cipher_new_with_iv(key, NULL); + + memcpy(to, cipher->iv, CIPHER_IV_LEN); crypto_cipher_encrypt(cipher, to+CIPHER_IV_LEN, from, fromlen); + crypto_cipher_free(cipher); return (int)(fromlen + CIPHER_IV_LEN); } /** Decrypt <b>fromlen</b> bytes (at least 1+CIPHER_IV_LEN) from <b>from</b> - * with the key in <b>cipher</b> to the buffer in <b>to</b> of length + * with the key in <b>key</b> to the buffer in <b>to</b> of length * <b>tolen</b>. <b>tolen</b> must be at least <b>fromlen</b> minus * CIPHER_IV_LEN bytes for the initialization vector. On success, return the * number of bytes written, on failure, return -1. - * - * This function adjusts the current position of the counter in <b>cipher</b> - * to immediately after the decrypted data. */ int -crypto_cipher_decrypt_with_iv(crypto_cipher_env_t *cipher, +crypto_cipher_decrypt_with_iv(const char *key, char *to, size_t tolen, const char *from, size_t fromlen) { - tor_assert(cipher); + crypto_cipher_t *cipher; + tor_assert(key); tor_assert(from); tor_assert(to); tor_assert(fromlen < INT_MAX); @@ -1488,9 +1387,10 @@ crypto_cipher_decrypt_with_iv(crypto_cipher_env_t *cipher, if (tolen < fromlen - CIPHER_IV_LEN) return -1; - if (crypto_cipher_set_iv(cipher, from)<0) - return -1; + cipher = crypto_cipher_new_with_iv(key, from); + crypto_cipher_encrypt(cipher, to, from+CIPHER_IV_LEN, fromlen-CIPHER_IV_LEN); + crypto_cipher_free(cipher); return (int)(fromlen - CIPHER_IV_LEN); } @@ -1568,7 +1468,7 @@ crypto_digest_algorithm_parse_name(const char *name) } /** Intermediate information about the digest of a stream of data. */ -struct crypto_digest_env_t { +struct crypto_digest_t { union { SHA_CTX sha1; /**< state for SHA1 */ SHA256_CTX sha2; /**< state for SHA256 */ @@ -1579,11 +1479,11 @@ struct crypto_digest_env_t { /** Allocate and return a new digest object to compute SHA1 digests. */ -crypto_digest_env_t * -crypto_new_digest_env(void) +crypto_digest_t * +crypto_digest_new(void) { - crypto_digest_env_t *r; - r = tor_malloc(sizeof(crypto_digest_env_t)); + crypto_digest_t *r; + r = tor_malloc(sizeof(crypto_digest_t)); SHA1_Init(&r->d.sha1); r->algorithm = DIGEST_SHA1; return r; @@ -1591,12 +1491,12 @@ crypto_new_digest_env(void) /** Allocate and return a new digest object to compute 256-bit digests * using <b>algorithm</b>. */ -crypto_digest_env_t * -crypto_new_digest256_env(digest_algorithm_t algorithm) +crypto_digest_t * +crypto_digest256_new(digest_algorithm_t algorithm) { - crypto_digest_env_t *r; + crypto_digest_t *r; tor_assert(algorithm == DIGEST_SHA256); - r = tor_malloc(sizeof(crypto_digest_env_t)); + r = tor_malloc(sizeof(crypto_digest_t)); SHA256_Init(&r->d.sha2); r->algorithm = algorithm; return r; @@ -1605,18 +1505,18 @@ crypto_new_digest256_env(digest_algorithm_t algorithm) /** Deallocate a digest object. */ void -crypto_free_digest_env(crypto_digest_env_t *digest) +crypto_digest_free(crypto_digest_t *digest) { if (!digest) return; - memset(digest, 0, sizeof(crypto_digest_env_t)); + memwipe(digest, 0, sizeof(crypto_digest_t)); tor_free(digest); } /** Add <b>len</b> bytes from <b>data</b> to the digest object. */ void -crypto_digest_add_bytes(crypto_digest_env_t *digest, const char *data, +crypto_digest_add_bytes(crypto_digest_t *digest, const char *data, size_t len) { tor_assert(digest); @@ -1644,15 +1544,15 @@ crypto_digest_add_bytes(crypto_digest_env_t *digest, const char *data, * <b>out_len</b> must be \<= DIGEST256_LEN. */ void -crypto_digest_get_digest(crypto_digest_env_t *digest, +crypto_digest_get_digest(crypto_digest_t *digest, char *out, size_t out_len) { unsigned char r[DIGEST256_LEN]; - crypto_digest_env_t tmpenv; + crypto_digest_t tmpenv; tor_assert(digest); tor_assert(out); /* memcpy into a temporary ctx, since SHA*_Final clears the context */ - memcpy(&tmpenv, digest, sizeof(crypto_digest_env_t)); + memcpy(&tmpenv, digest, sizeof(crypto_digest_t)); switch (digest->algorithm) { case DIGEST_SHA1: tor_assert(out_len <= DIGEST_LEN); @@ -1671,19 +1571,19 @@ crypto_digest_get_digest(crypto_digest_env_t *digest, break; } memcpy(out, r, out_len); - memset(r, 0, sizeof(r)); + memwipe(r, 0, sizeof(r)); } /** Allocate and return a new digest object with the same state as * <b>digest</b> */ -crypto_digest_env_t * -crypto_digest_dup(const crypto_digest_env_t *digest) +crypto_digest_t * +crypto_digest_dup(const crypto_digest_t *digest) { - crypto_digest_env_t *r; + crypto_digest_t *r; tor_assert(digest); - r = tor_malloc(sizeof(crypto_digest_env_t)); - memcpy(r,digest,sizeof(crypto_digest_env_t)); + r = tor_malloc(sizeof(crypto_digest_t)); + memcpy(r,digest,sizeof(crypto_digest_t)); return r; } @@ -1691,12 +1591,12 @@ crypto_digest_dup(const crypto_digest_env_t *digest) * of the digest object <b>from</b>. */ void -crypto_digest_assign(crypto_digest_env_t *into, - const crypto_digest_env_t *from) +crypto_digest_assign(crypto_digest_t *into, + const crypto_digest_t *from) { tor_assert(into); tor_assert(from); - memcpy(into,from,sizeof(crypto_digest_env_t)); + memcpy(into,from,sizeof(crypto_digest_t)); } /** Compute the HMAC-SHA-1 of the <b>msg_len</b> bytes in <b>msg</b>, using @@ -1715,15 +1615,15 @@ crypto_hmac_sha1(char *hmac_out, } /** Compute the HMAC-SHA-256 of the <b>msg_len</b> bytes in <b>msg</b>, using - * the <b>key</b> of length <b>key_len</b>. Store the DIGEST_LEN-byte result - * in <b>hmac_out</b>. + * the <b>key</b> of length <b>key_len</b>. Store the DIGEST256_LEN-byte + * result in <b>hmac_out</b>. */ void crypto_hmac_sha256(char *hmac_out, const char *key, size_t key_len, const char *msg, size_t msg_len) { -#if (OPENSSL_VERSION_NUMBER >= 0x00908000l) +#if OPENSSL_VERSION_NUMBER >= OPENSSL_V_SERIES(0,9,8) /* If we've got OpenSSL >=0.9.8 we can use its hmac implementation. */ tor_assert(key_len < INT_MAX); tor_assert(msg_len < INT_MAX); @@ -1773,10 +1673,10 @@ crypto_hmac_sha256(char *hmac_out, SHA256_Final((uint8_t*)hmac_out, &st); /* Now clear everything. */ - memset(k, 0, sizeof(k)); - memset(pad, 0, sizeof(pad)); - memset(d, 0, sizeof(d)); - memset(&st, 0, sizeof(st)); + memwipe(k, 0, sizeof(k)); + memwipe(pad, 0, sizeof(pad)); + memwipe(d, 0, sizeof(d)); + memwipe(&st, 0, sizeof(st)); #undef BLOCKSIZE #undef DIGESTSIZE #endif @@ -1784,6 +1684,9 @@ crypto_hmac_sha256(char *hmac_out, /* DH */ +/** Our DH 'g' parameter */ +#define DH_GENERATOR 2 + /** Shared P parameter for our circuit-crypto DH key exchanges. */ static BIGNUM *dh_param_p = NULL; /** Shared P parameter for our TLS DH key exchanges. */ @@ -1791,49 +1694,324 @@ static BIGNUM *dh_param_p_tls = NULL; /** Shared G parameter for our DH key exchanges. */ static BIGNUM *dh_param_g = NULL; +/** Generate and return a reasonable and safe DH parameter p. */ +static BIGNUM * +crypto_generate_dynamic_dh_modulus(void) +{ + BIGNUM *dynamic_dh_modulus; + DH *dh_parameters; + int r, dh_codes; + char *s; + + dynamic_dh_modulus = BN_new(); + tor_assert(dynamic_dh_modulus); + + dh_parameters = DH_generate_parameters(DH_BYTES*8, DH_GENERATOR, NULL, NULL); + tor_assert(dh_parameters); + + r = DH_check(dh_parameters, &dh_codes); + tor_assert(r && !dh_codes); + + BN_copy(dynamic_dh_modulus, dh_parameters->p); + tor_assert(dynamic_dh_modulus); + + DH_free(dh_parameters); + + { /* log the dynamic DH modulus: */ + s = BN_bn2hex(dynamic_dh_modulus); + tor_assert(s); + log_info(LD_OR, "Dynamic DH modulus generated: [%s]", s); + OPENSSL_free(s); + } + + return dynamic_dh_modulus; +} + +/** Store our dynamic DH modulus (and its group parameters) to + <b>fname</b> for future use. */ +static int +crypto_store_dynamic_dh_modulus(const char *fname) +{ + int len, new_len; + DH *dh = NULL; + unsigned char *dh_string_repr = NULL, *cp = NULL; + char *base64_encoded_dh = NULL; + char *file_string = NULL; + int retval = -1; + static const char file_header[] = "# This file contains stored Diffie-" + "Hellman parameters for future use.\n# You *do not* need to edit this " + "file.\n\n"; + + tor_assert(fname); + + if (!dh_param_p_tls) { + log_info(LD_CRYPTO, "Tried to store a DH modulus that does not exist."); + goto done; + } + + if (!(dh = DH_new())) + goto done; + if (!(dh->p = BN_dup(dh_param_p_tls))) + goto done; + if (!(dh->g = BN_new())) + goto done; + if (!BN_set_word(dh->g, DH_GENERATOR)) + goto done; + + len = i2d_DHparams(dh, NULL); + if (len < 0) { + log_warn(LD_CRYPTO, "Error occured while DER encoding DH modulus (1)."); + goto done; + } + + cp = dh_string_repr = tor_malloc_zero(len+1); + len = i2d_DHparams(dh, &cp); + if ((len < 0) || ((cp - dh_string_repr) != len)) { + log_warn(LD_CRYPTO, "Error occured while DER encoding DH modulus (2)."); + goto done; + } + + base64_encoded_dh = tor_malloc_zero(len * 2); /* should be enough */ + new_len = base64_encode(base64_encoded_dh, len * 2, + (char *)dh_string_repr, len); + if (new_len < 0) { + log_warn(LD_CRYPTO, "Error occured while base64-encoding DH modulus."); + goto done; + } + + /* concatenate file header and the dh parameters blob */ + new_len = tor_asprintf(&file_string, "%s%s", file_header, base64_encoded_dh); + + /* write to file */ + if (write_bytes_to_new_file(fname, file_string, new_len, 0) < 0) { + log_info(LD_CRYPTO, "'%s' was already occupied.", fname); + goto done; + } + + retval = 0; + + done: + if (dh) + DH_free(dh); + tor_free(dh_string_repr); + tor_free(base64_encoded_dh); + tor_free(file_string); + + return retval; +} + +/** Return the dynamic DH modulus stored in <b>fname</b>. If there is no + dynamic DH modulus stored in <b>fname</b>, return NULL. */ +static BIGNUM * +crypto_get_stored_dynamic_dh_modulus(const char *fname) +{ + int retval; + char *contents = NULL; + const char *contents_tmp = NULL; + int dh_codes; + DH *stored_dh = NULL; + BIGNUM *dynamic_dh_modulus = NULL; + int length = 0; + unsigned char *base64_decoded_dh = NULL; + const unsigned char *cp = NULL; + + tor_assert(fname); + + contents = read_file_to_str(fname, RFTS_IGNORE_MISSING, NULL); + if (!contents) { + log_info(LD_CRYPTO, "Could not open file '%s'", fname); + goto done; /*usually means that ENOENT. don't try to move file to broken.*/ + } + + /* skip the file header */ + contents_tmp = eat_whitespace(contents); + if (!*contents_tmp) { + log_warn(LD_CRYPTO, "Stored dynamic DH modulus file " + "seems corrupted (eat_whitespace)."); + goto err; + } + + /* 'fname' contains the DH parameters stored in base64-ed DER + * format. We are only interested in the DH modulus. + * NOTE: We allocate more storage here than we need. Since we're already + * doing that, we can also add 1 byte extra to appease Coverity's + * scanner. */ + + cp = base64_decoded_dh = tor_malloc_zero(strlen(contents_tmp) + 1); + length = base64_decode((char *)base64_decoded_dh, strlen(contents_tmp), + contents_tmp, strlen(contents_tmp)); + if (length < 0) { + log_warn(LD_CRYPTO, "Stored dynamic DH modulus seems corrupted (base64)."); + goto err; + } + + stored_dh = d2i_DHparams(NULL, &cp, length); + if ((!stored_dh) || (cp - base64_decoded_dh != length)) { + log_warn(LD_CRYPTO, "Stored dynamic DH modulus seems corrupted (d2i)."); + goto err; + } + + { /* check the cryptographic qualities of the stored dynamic DH modulus: */ + retval = DH_check(stored_dh, &dh_codes); + if (!retval || dh_codes) { + log_warn(LD_CRYPTO, "Stored dynamic DH modulus is not a safe prime."); + goto err; + } + + retval = DH_size(stored_dh); + if (retval < DH_BYTES) { + log_warn(LD_CRYPTO, "Stored dynamic DH modulus is smaller " + "than '%d' bits.", DH_BYTES*8); + goto err; + } + + if (!BN_is_word(stored_dh->g, 2)) { + log_warn(LD_CRYPTO, "Stored dynamic DH parameters do not use '2' " + "as the group generator."); + goto err; + } + } + + { /* log the dynamic DH modulus: */ + char *s = BN_bn2hex(stored_dh->p); + tor_assert(s); + log_info(LD_OR, "Found stored dynamic DH modulus: [%s]", s); + OPENSSL_free(s); + } + + goto done; + + err: + + { + /* move broken prime to $filename.broken */ + char *fname_new=NULL; + tor_asprintf(&fname_new, "%s.broken", fname); + + log_warn(LD_CRYPTO, "Moving broken dynamic DH prime to '%s'.", fname_new); + + if (replace_file(fname, fname_new)) + log_notice(LD_CRYPTO, "Error while moving '%s' to '%s'.", + fname, fname_new); + + tor_free(fname_new); + } + + if (stored_dh) { + DH_free(stored_dh); + stored_dh = NULL; + } + + done: + tor_free(contents); + tor_free(base64_decoded_dh); + + if (stored_dh) { + dynamic_dh_modulus = BN_dup(stored_dh->p); + DH_free(stored_dh); + } + + return dynamic_dh_modulus; +} + +/** Set the global TLS Diffie-Hellman modulus. + * If <b>dynamic_dh_modulus_fname</b> is set, try to read a dynamic DH modulus + * off it and use it as the DH modulus. If that's not possible, + * generate a new dynamic DH modulus. + * If <b>dynamic_dh_modulus_fname</b> is NULL, use the Apache mod_ssl DH + * modulus. */ +void +crypto_set_tls_dh_prime(const char *dynamic_dh_modulus_fname) +{ + BIGNUM *tls_prime = NULL; + int store_dh_prime_afterwards = 0; + int r; + + /* If the space is occupied, free the previous TLS DH prime */ + if (dh_param_p_tls) { + BN_free(dh_param_p_tls); + dh_param_p_tls = NULL; + } + + if (dynamic_dh_modulus_fname) { /* use dynamic DH modulus: */ + log_info(LD_OR, "Using stored dynamic DH modulus."); + tls_prime = crypto_get_stored_dynamic_dh_modulus(dynamic_dh_modulus_fname); + + if (!tls_prime) { + log_notice(LD_OR, "Generating fresh dynamic DH modulus. " + "This might take a while..."); + tls_prime = crypto_generate_dynamic_dh_modulus(); + + store_dh_prime_afterwards++; + } + } else { /* use the static DH prime modulus used by Apache in mod_ssl: */ + tls_prime = BN_new(); + tor_assert(tls_prime); + + /* This is the 1024-bit safe prime that Apache uses for its DH stuff; see + * modules/ssl/ssl_engine_dh.c; Apache also uses a generator of 2 with this + * prime. + */ + r =BN_hex2bn(&tls_prime, + "D67DE440CBBBDC1936D693D34AFD0AD50C84D239A45F520BB88174CB98" + "BCE951849F912E639C72FB13B4B4D7177E16D55AC179BA420B2A29FE324A" + "467A635E81FF5901377BEDDCFD33168A461AAD3B72DAE8860078045B07A7" + "DBCA7874087D1510EA9FCC9DDD330507DD62DB88AEAA747DE0F4D6E2BD68" + "B0E7393E0F24218EB3"); + tor_assert(r); + } + + tor_assert(tls_prime); + + dh_param_p_tls = tls_prime; + + if (store_dh_prime_afterwards) + /* save the new dynamic DH modulus to disk. */ + if (crypto_store_dynamic_dh_modulus(dynamic_dh_modulus_fname)) { + log_notice(LD_CRYPTO, "Failed while storing dynamic DH modulus. " + "Make sure your data directory is sane."); + } +} + /** Initialize dh_param_p and dh_param_g if they are not already * set. */ static void init_dh_param(void) { - BIGNUM *p, *p2, *g; + BIGNUM *circuit_dh_prime, *generator; int r; - if (dh_param_p && dh_param_g && dh_param_p_tls) + if (dh_param_p && dh_param_g) return; - p = BN_new(); - p2 = BN_new(); - g = BN_new(); - tor_assert(p); - tor_assert(p2); - tor_assert(g); + circuit_dh_prime = BN_new(); + generator = BN_new(); + tor_assert(circuit_dh_prime && generator); + + /* Set our generator for all DH parameters */ + r = BN_set_word(generator, DH_GENERATOR); + tor_assert(r); /* This is from rfc2409, section 6.2. It's a safe prime, and supposedly it equals: 2^1024 - 2^960 - 1 + 2^64 * { [2^894 pi] + 129093 }. */ - r = BN_hex2bn(&p, + r = BN_hex2bn(&circuit_dh_prime, "FFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD129024E08" "8A67CC74020BBEA63B139B22514A08798E3404DDEF9519B3CD3A431B" "302B0A6DF25F14374FE1356D6D51C245E485B576625E7EC6F44C42E9" "A637ED6B0BFF5CB6F406B7EDEE386BFB5A899FA5AE9F24117C4B1FE6" "49286651ECE65381FFFFFFFFFFFFFFFF"); tor_assert(r); - /* This is the 1024-bit safe prime that Apache uses for its DH stuff; see - * modules/ssl/ssl_engine_dh.c */ - r = BN_hex2bn(&p2, - "D67DE440CBBBDC1936D693D34AFD0AD50C84D239A45F520BB88174CB98" - "BCE951849F912E639C72FB13B4B4D7177E16D55AC179BA420B2A29FE324A" - "467A635E81FF5901377BEDDCFD33168A461AAD3B72DAE8860078045B07A7" - "DBCA7874087D1510EA9FCC9DDD330507DD62DB88AEAA747DE0F4D6E2BD68" - "B0E7393E0F24218EB3"); - tor_assert(r); - r = BN_set_word(g, 2); - tor_assert(r); - dh_param_p = p; - dh_param_p_tls = p2; - dh_param_g = g; + /* Set the new values as the global DH parameters. */ + dh_param_p = circuit_dh_prime; + dh_param_g = generator; + + /* Ensure that we have TLS DH parameters set up, too, even if we're + going to change them soon. */ + if (!dh_param_p_tls) { + crypto_set_tls_dh_prime(NULL); + } } /** Number of bits to use when choosing the x or y value in a Diffie-Hellman @@ -1844,10 +2022,10 @@ init_dh_param(void) /** Allocate and return a new DH object for a key exchange. */ -crypto_dh_env_t * +crypto_dh_t * crypto_dh_new(int dh_type) { - crypto_dh_env_t *res = tor_malloc_zero(sizeof(crypto_dh_env_t)); + crypto_dh_t *res = tor_malloc_zero(sizeof(crypto_dh_t)); tor_assert(dh_type == DH_TYPE_CIRCUIT || dh_type == DH_TYPE_TLS || dh_type == DH_TYPE_REND); @@ -1882,7 +2060,7 @@ crypto_dh_new(int dh_type) /** Return the length of the DH key in <b>dh</b>, in bytes. */ int -crypto_dh_get_bytes(crypto_dh_env_t *dh) +crypto_dh_get_bytes(crypto_dh_t *dh) { tor_assert(dh); return DH_size(dh->dh); @@ -1892,7 +2070,7 @@ crypto_dh_get_bytes(crypto_dh_env_t *dh) * success, -1 on failure. */ int -crypto_dh_generate_public(crypto_dh_env_t *dh) +crypto_dh_generate_public(crypto_dh_t *dh) { again: if (!DH_generate_key(dh->dh)) { @@ -1916,7 +2094,7 @@ crypto_dh_generate_public(crypto_dh_env_t *dh) * success, -1 on failure. <b>pubkey_len</b> must be \>= DH_BYTES. */ int -crypto_dh_get_public(crypto_dh_env_t *dh, char *pubkey, size_t pubkey_len) +crypto_dh_get_public(crypto_dh_t *dh, char *pubkey, size_t pubkey_len) { int bytes; tor_assert(dh); @@ -1989,7 +2167,7 @@ tor_check_dh_key(int severity, BIGNUM *bn) * where || is concatenation.) */ ssize_t -crypto_dh_compute_secret(int severity, crypto_dh_env_t *dh, +crypto_dh_compute_secret(int severity, crypto_dh_t *dh, const char *pubkey, size_t pubkey_len, char *secret_out, size_t secret_bytes_out) { @@ -2030,7 +2208,7 @@ crypto_dh_compute_secret(int severity, crypto_dh_env_t *dh, if (pubkey_bn) BN_free(pubkey_bn); if (secret_tmp) { - memset(secret_tmp, 0, secret_tmp_len); + memwipe(secret_tmp, 0, secret_tmp_len); tor_free(secret_tmp); } if (result < 0) @@ -2065,22 +2243,22 @@ crypto_expand_key_material(const char *key_in, size_t key_in_len, goto err; memcpy(cp, digest, MIN(DIGEST_LEN, key_out_len-(cp-key_out))); } - memset(tmp, 0, key_in_len+1); + memwipe(tmp, 0, key_in_len+1); tor_free(tmp); - memset(digest, 0, sizeof(digest)); + memwipe(digest, 0, sizeof(digest)); return 0; err: - memset(tmp, 0, key_in_len+1); + memwipe(tmp, 0, key_in_len+1); tor_free(tmp); - memset(digest, 0, sizeof(digest)); + memwipe(digest, 0, sizeof(digest)); return -1; } /** Free a DH key exchange object. */ void -crypto_dh_free(crypto_dh_env_t *dh) +crypto_dh_free(crypto_dh_t *dh) { if (!dh) return; @@ -2097,13 +2275,6 @@ crypto_dh_free(crypto_dh_env_t *dh) * work for us too. */ #define ADD_ENTROPY 32 -/** True iff we should use OpenSSL's RAND_poll function to add entropy to its - * pool. - * - * Use RAND_poll if OpenSSL is 0.9.6 release or later. (The "f" means - *"release".) */ -#define HAVE_RAND_POLL (OPENSSL_VERSION_NUMBER >= 0x0090600fl) - /** True iff it's safe to use RAND_poll after setup. * * Versions of OpenSSL prior to 0.9.7k and 0.9.8c had a bug where RAND_poll @@ -2111,9 +2282,9 @@ crypto_dh_free(crypto_dh_env_t *dh) * that fd without checking whether it fit in the fd_set. Thus, if the * system has not just been started up, it is unsafe to call */ #define RAND_POLL_IS_SAFE \ - ((OPENSSL_VERSION_NUMBER >= 0x009070afl && \ - OPENSSL_VERSION_NUMBER <= 0x00907fffl) || \ - (OPENSSL_VERSION_NUMBER >= 0x0090803fl)) + ((OPENSSL_VERSION_NUMBER >= OPENSSL_V(0,9,7,'j') && \ + OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(0,9,8)) || \ + OPENSSL_VERSION_NUMBER >= OPENSSL_V(0,9,8,'c')) /** Set the seed of the weak RNG to a random value. */ static void @@ -2134,7 +2305,7 @@ crypto_seed_rng(int startup) int rand_poll_status = 0; /* local variables */ -#ifdef MS_WINDOWS +#ifdef _WIN32 unsigned char buf[ADD_ENTROPY]; static int provider_set = 0; static HCRYPTPROV provider; @@ -2147,8 +2318,7 @@ crypto_seed_rng(int startup) size_t n; #endif -#if HAVE_RAND_POLL - /* OpenSSL 0.9.6 adds a RAND_poll function that knows about more kinds of + /* OpenSSL has a RAND_poll function that knows about more kinds of * entropy than we do. We'll try calling that, *and* calling our own entropy * functions. If one succeeds, we'll accept the RNG as seeded. */ if (startup || RAND_POLL_IS_SAFE) { @@ -2156,9 +2326,8 @@ crypto_seed_rng(int startup) if (rand_poll_status == 0) log_warn(LD_CRYPTO, "RAND_poll() failed."); } -#endif -#ifdef MS_WINDOWS +#ifdef _WIN32 if (!provider_set) { if (!CryptAcquireContext(&provider, NULL, NULL, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT)) { @@ -2174,7 +2343,7 @@ crypto_seed_rng(int startup) return rand_poll_status ? 0 : -1; } RAND_seed(buf, sizeof(buf)); - memset(buf, 0, sizeof(buf)); + memwipe(buf, 0, sizeof(buf)); seed_weak_rng(); return 0; #else @@ -2191,7 +2360,7 @@ crypto_seed_rng(int startup) return -1; } RAND_seed(buf, (int)sizeof(buf)); - memset(buf, 0, sizeof(buf)); + memwipe(buf, 0, sizeof(buf)); seed_weak_rng(); return 0; } @@ -2282,9 +2451,12 @@ crypto_rand_double(void) } /** Generate and return a new random hostname starting with <b>prefix</b>, - * ending with <b>suffix</b>, and containing no less than + * ending with <b>suffix</b>, and containing no fewer than * <b>min_rand_len</b> and no more than <b>max_rand_len</b> random base32 - * characters between. */ + * characters between. + * + * Clip <b>max_rand_len</b> to MAX_DNS_LABEL_SIZE. + **/ char * crypto_random_hostname(int min_rand_len, int max_rand_len, const char *prefix, const char *suffix) @@ -2293,8 +2465,13 @@ crypto_random_hostname(int min_rand_len, int max_rand_len, const char *prefix, int randlen, rand_bytes_len; size_t resultlen, prefixlen; - tor_assert(max_rand_len >= min_rand_len); + if (max_rand_len > MAX_DNS_LABEL_SIZE) + max_rand_len = MAX_DNS_LABEL_SIZE; + if (min_rand_len > max_rand_len) + min_rand_len = max_rand_len; + randlen = min_rand_len + crypto_rand_int(max_rand_len - min_rand_len + 1); + prefixlen = strlen(prefix); resultlen = prefixlen + strlen(suffix) + randlen + 16; @@ -2666,7 +2843,7 @@ base32_decode(char *dest, size_t destlen, const char *src, size_t srclen) } } - memset(tmp, 0, srclen); + memwipe(tmp, 0, srclen); tor_free(tmp); tmp = NULL; return 0; @@ -2682,7 +2859,7 @@ void secret_to_key(char *key_out, size_t key_out_len, const char *secret, size_t secret_len, const char *s2k_specifier) { - crypto_digest_env_t *d; + crypto_digest_t *d; uint8_t c; size_t count, tmplen; char *tmp; @@ -2695,7 +2872,7 @@ secret_to_key(char *key_out, size_t key_out_len, const char *secret, tor_assert(key_out_len <= DIGEST_LEN); - d = crypto_new_digest_env(); + d = crypto_digest_new(); tmplen = 8+secret_len; tmp = tor_malloc(tmplen); memcpy(tmp,s2k_specifier,8); @@ -2711,9 +2888,52 @@ secret_to_key(char *key_out, size_t key_out_len, const char *secret, } } crypto_digest_get_digest(d, key_out, key_out_len); - memset(tmp, 0, tmplen); + memwipe(tmp, 0, tmplen); tor_free(tmp); - crypto_free_digest_env(d); + crypto_digest_free(d); +} + +/** + * Destroy the <b>sz</b> bytes of data stored at <b>mem</b>, setting them to + * the value <b>byte</b>. + * + * This function is preferable to memset, since many compilers will happily + * optimize out memset() when they can convince themselves that the data being + * cleared will never be read. + * + * Right now, our convention is to use this function when we are wiping data + * that's about to become inaccessible, such as stack buffers that are about + * to go out of scope or structures that are about to get freed. (In + * practice, it appears that the compilers we're currently using will optimize + * out the memset()s for stack-allocated buffers, but not those for + * about-to-be-freed structures. That could change, though, so we're being + * wary.) If there are live reads for the data, then you can just use + * memset(). + */ +void +memwipe(void *mem, uint8_t byte, size_t sz) +{ + /* Because whole-program-optimization exists, we may not be able to just + * have this function call "memset". A smart compiler could inline it, then + * eliminate dead memsets, and declare itself to be clever. */ + + /* This is a slow and ugly function from OpenSSL that fills 'mem' with junk + * based on the pointer value, then uses that junk to update a global + * variable. It's an elaborate ruse to trick the compiler into not + * optimizing out the "wipe this memory" code. Read it if you like zany + * programming tricks! In later versions of Tor, we should look for better + * not-optimized-out memory wiping stuff. */ + OPENSSL_cleanse(mem, sz); + /* Just in case some caller of memwipe() is relying on getting a buffer + * filled with a particular value, fill the buffer. + * + * If this function gets inlined, this memset might get eliminated, but + * that's okay: We only care about this particular memset in the case where + * the caller should have been using memset(), and the memset() wouldn't get + * eliminated. In other words, this is here so that we won't break anything + * if somebody accidentally calls memwipe() instead of memset(). + **/ + memset(mem, byte, sz); } #ifdef TOR_IS_MULTITHREADED @@ -2805,5 +3025,44 @@ setup_openssl_threading(void) return 0; } #endif + +/** Uninitialize the crypto library. Return 0 on success, -1 on failure. + */ +int +crypto_global_cleanup(void) +{ + EVP_cleanup(); + ERR_remove_state(0); + ERR_free_strings(); + + if (dh_param_p) + BN_free(dh_param_p); + if (dh_param_p_tls) + BN_free(dh_param_p_tls); + if (dh_param_g) + BN_free(dh_param_g); + +#ifndef DISABLE_ENGINES + ENGINE_cleanup(); +#endif + + CONF_modules_unload(1); + CRYPTO_cleanup_all_ex_data(); +#ifdef TOR_IS_MULTITHREADED + if (_n_openssl_mutexes) { + int n = _n_openssl_mutexes; + tor_mutex_t **ms = _openssl_mutexes; + int i; + _openssl_mutexes = NULL; + _n_openssl_mutexes = 0; + for (i=0;i<n;++i) { + tor_mutex_free(ms[i]); + } + tor_free(ms); + } +#endif + return 0; +} + /** @} */ diff --git a/src/common/crypto.h b/src/common/crypto.h index a0ee075ba0..7d56271785 100644 --- a/src/common/crypto.h +++ b/src/common/crypto.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001, Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -16,6 +16,38 @@ #include <stdio.h> #include "torint.h" +/* + Macro to create an arbitrary OpenSSL version number as used by + OPENSSL_VERSION_NUMBER or SSLeay(), since the actual numbers are a bit hard + to read. + + Don't use this directly, instead use one of the other OPENSSL_V macros + below. + + The format is: 4 bits major, 8 bits minor, 8 bits fix, 8 bits patch, 4 bit + status. + */ +#define OPENSSL_VER(a,b,c,d,e) \ + (((a)<<28) | \ + ((b)<<20) | \ + ((c)<<12) | \ + ((d)<< 4) | \ + (e)) +/** An openssl release number. For example, OPENSSL_V(0,9,8,'j') is the + * version for the released version of 0.9.8j */ +#define OPENSSL_V(a,b,c,d) \ + OPENSSL_VER((a),(b),(c),(d)-'a'+1,0xf) +/** An openssl release number for the first release in the series. For + * example, OPENSSL_V_NOPATCH(1,0,0) is the first released version of OpenSSL + * 1.0.0. */ +#define OPENSSL_V_NOPATCH(a,b,c) \ + OPENSSL_VER((a),(b),(c),0,0xf) +/** The first version that would occur for any alpha or beta in an openssl + * series. For example, OPENSSL_V_SERIES(0,9,8) is greater than any released + * 0.9.7, and less than any released 0.9.8. */ +#define OPENSSL_V_SERIES(a,b,c) \ + OPENSSL_VER((a),(b),(c),0,0) + /** Length of the output of our message digest. */ #define DIGEST_LEN 20 /** Length of the output of our second (improved) message digests. (For now @@ -37,11 +69,9 @@ * signs removed. */ #define BASE64_DIGEST256_LEN 43 -/** Constants used to indicate no padding for public-key encryption */ -#define PK_NO_PADDING 60000 -/** Constants used to indicate PKCS1 padding for public-key encryption */ +/** Constant used to indicate PKCS1 padding for public-key encryption */ #define PK_PKCS1_PADDING 60001 -/** Constants used to indicate OAEP padding for public-key encryption */ +/** Constant used to indicate OAEP padding for public-key encryption */ #define PK_PKCS1_OAEP_PADDING 60002 /** Number of bytes added for PKCS1 padding. */ @@ -75,10 +105,10 @@ typedef struct { char d[N_DIGEST_ALGORITHMS][DIGEST256_LEN]; } digests_t; -typedef struct crypto_pk_env_t crypto_pk_env_t; -typedef struct crypto_cipher_env_t crypto_cipher_env_t; -typedef struct crypto_digest_env_t crypto_digest_env_t; -typedef struct crypto_dh_env_t crypto_dh_env_t; +typedef struct crypto_pk_t crypto_pk_t; +typedef struct crypto_cipher_t crypto_cipher_t; +typedef struct crypto_digest_t crypto_digest_t; +typedef struct crypto_dh_t crypto_dh_t; /* global state */ int crypto_global_init(int hardwareAccel, @@ -88,90 +118,84 @@ void crypto_thread_cleanup(void); int crypto_global_cleanup(void); /* environment setup */ -crypto_pk_env_t *crypto_new_pk_env(void); -void crypto_free_pk_env(crypto_pk_env_t *env); +crypto_pk_t *crypto_pk_new(void); +void crypto_pk_free(crypto_pk_t *env); -/* convenience function: wraps crypto_create_crypto_env, set_key, and init. */ -crypto_cipher_env_t *crypto_create_init_cipher(const char *key, - int encrypt_mode); +void crypto_set_tls_dh_prime(const char *dynamic_dh_modulus_fname); -crypto_cipher_env_t *crypto_new_cipher_env(void); -void crypto_free_cipher_env(crypto_cipher_env_t *env); +crypto_cipher_t *crypto_cipher_new(const char *key); +crypto_cipher_t *crypto_cipher_new_with_iv(const char *key, const char *iv); +void crypto_cipher_free(crypto_cipher_t *env); /* public key crypto */ -int crypto_pk_generate_key_with_bits(crypto_pk_env_t *env, int bits); +int crypto_pk_generate_key_with_bits(crypto_pk_t *env, int bits); #define crypto_pk_generate_key(env) \ crypto_pk_generate_key_with_bits((env), (PK_BYTES*8)) -int crypto_pk_read_private_key_from_filename(crypto_pk_env_t *env, +int crypto_pk_read_private_key_from_filename(crypto_pk_t *env, const char *keyfile); -int crypto_pk_write_public_key_to_string(crypto_pk_env_t *env, +int crypto_pk_write_public_key_to_string(crypto_pk_t *env, char **dest, size_t *len); -int crypto_pk_write_private_key_to_string(crypto_pk_env_t *env, +int crypto_pk_write_private_key_to_string(crypto_pk_t *env, char **dest, size_t *len); -int crypto_pk_read_public_key_from_string(crypto_pk_env_t *env, +int crypto_pk_read_public_key_from_string(crypto_pk_t *env, const char *src, size_t len); -int crypto_pk_read_private_key_from_string(crypto_pk_env_t *env, +int crypto_pk_read_private_key_from_string(crypto_pk_t *env, const char *s, ssize_t len); -int crypto_pk_write_private_key_to_filename(crypto_pk_env_t *env, +int crypto_pk_write_private_key_to_filename(crypto_pk_t *env, const char *fname); -int crypto_pk_check_key(crypto_pk_env_t *env); -int crypto_pk_cmp_keys(crypto_pk_env_t *a, crypto_pk_env_t *b); -size_t crypto_pk_keysize(crypto_pk_env_t *env); -int crypto_pk_num_bits(crypto_pk_env_t *env); -crypto_pk_env_t *crypto_pk_dup_key(crypto_pk_env_t *orig); -crypto_pk_env_t *crypto_pk_copy_full(crypto_pk_env_t *orig); -int crypto_pk_key_is_private(const crypto_pk_env_t *key); -int crypto_pk_public_exponent_ok(crypto_pk_env_t *env); +int crypto_pk_check_key(crypto_pk_t *env); +int crypto_pk_cmp_keys(crypto_pk_t *a, crypto_pk_t *b); +size_t crypto_pk_keysize(crypto_pk_t *env); +int crypto_pk_num_bits(crypto_pk_t *env); +crypto_pk_t *crypto_pk_dup_key(crypto_pk_t *orig); +crypto_pk_t *crypto_pk_copy_full(crypto_pk_t *orig); +int crypto_pk_key_is_private(const crypto_pk_t *key); +int crypto_pk_public_exponent_ok(crypto_pk_t *env); -int crypto_pk_public_encrypt(crypto_pk_env_t *env, char *to, size_t tolen, +int crypto_pk_public_encrypt(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen, int padding); -int crypto_pk_private_decrypt(crypto_pk_env_t *env, char *to, size_t tolen, +int crypto_pk_private_decrypt(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen, int padding, int warnOnFailure); -int crypto_pk_public_checksig(crypto_pk_env_t *env, char *to, size_t tolen, +int crypto_pk_public_checksig(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen); -int crypto_pk_public_checksig_digest(crypto_pk_env_t *env, const char *data, +int crypto_pk_public_checksig_digest(crypto_pk_t *env, const char *data, size_t datalen, const char *sig, size_t siglen); -int crypto_pk_private_sign(crypto_pk_env_t *env, char *to, size_t tolen, +int crypto_pk_private_sign(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen); -int crypto_pk_private_sign_digest(crypto_pk_env_t *env, char *to, size_t tolen, +int crypto_pk_private_sign_digest(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen); -int crypto_pk_public_hybrid_encrypt(crypto_pk_env_t *env, char *to, +int crypto_pk_public_hybrid_encrypt(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen, int padding, int force); -int crypto_pk_private_hybrid_decrypt(crypto_pk_env_t *env, char *to, +int crypto_pk_private_hybrid_decrypt(crypto_pk_t *env, char *to, size_t tolen, const char *from, size_t fromlen, int padding, int warnOnFailure); -int crypto_pk_asn1_encode(crypto_pk_env_t *pk, char *dest, size_t dest_len); -crypto_pk_env_t *crypto_pk_asn1_decode(const char *str, size_t len); -int crypto_pk_get_digest(crypto_pk_env_t *pk, char *digest_out); -int crypto_pk_get_fingerprint(crypto_pk_env_t *pk, char *fp_out,int add_space); +int crypto_pk_asn1_encode(crypto_pk_t *pk, char *dest, size_t dest_len); +crypto_pk_t *crypto_pk_asn1_decode(const char *str, size_t len); +int crypto_pk_get_digest(crypto_pk_t *pk, char *digest_out); +int crypto_pk_get_all_digests(crypto_pk_t *pk, digests_t *digests_out); +int crypto_pk_get_fingerprint(crypto_pk_t *pk, char *fp_out,int add_space); int crypto_pk_check_fingerprint_syntax(const char *s); /* symmetric crypto */ -int crypto_cipher_generate_key(crypto_cipher_env_t *env); -void crypto_cipher_set_key(crypto_cipher_env_t *env, const char *key); -void crypto_cipher_generate_iv(char *iv_out); -int crypto_cipher_set_iv(crypto_cipher_env_t *env, const char *iv); -const char *crypto_cipher_get_key(crypto_cipher_env_t *env); -int crypto_cipher_encrypt_init_cipher(crypto_cipher_env_t *env); -int crypto_cipher_decrypt_init_cipher(crypto_cipher_env_t *env); - -int crypto_cipher_encrypt(crypto_cipher_env_t *env, char *to, +const char *crypto_cipher_get_key(crypto_cipher_t *env); + +int crypto_cipher_encrypt(crypto_cipher_t *env, char *to, const char *from, size_t fromlen); -int crypto_cipher_decrypt(crypto_cipher_env_t *env, char *to, +int crypto_cipher_decrypt(crypto_cipher_t *env, char *to, const char *from, size_t fromlen); -int crypto_cipher_crypt_inplace(crypto_cipher_env_t *env, char *d, size_t len); +int crypto_cipher_crypt_inplace(crypto_cipher_t *env, char *d, size_t len); -int crypto_cipher_encrypt_with_iv(crypto_cipher_env_t *env, +int crypto_cipher_encrypt_with_iv(const char *key, char *to, size_t tolen, const char *from, size_t fromlen); -int crypto_cipher_decrypt_with_iv(crypto_cipher_env_t *env, +int crypto_cipher_decrypt_with_iv(const char *key, char *to, size_t tolen, const char *from, size_t fromlen); @@ -182,16 +206,16 @@ int crypto_digest256(char *digest, const char *m, size_t len, int crypto_digest_all(digests_t *ds_out, const char *m, size_t len); const char *crypto_digest_algorithm_get_name(digest_algorithm_t alg); int crypto_digest_algorithm_parse_name(const char *name); -crypto_digest_env_t *crypto_new_digest_env(void); -crypto_digest_env_t *crypto_new_digest256_env(digest_algorithm_t algorithm); -void crypto_free_digest_env(crypto_digest_env_t *digest); -void crypto_digest_add_bytes(crypto_digest_env_t *digest, const char *data, +crypto_digest_t *crypto_digest_new(void); +crypto_digest_t *crypto_digest256_new(digest_algorithm_t algorithm); +void crypto_digest_free(crypto_digest_t *digest); +void crypto_digest_add_bytes(crypto_digest_t *digest, const char *data, size_t len); -void crypto_digest_get_digest(crypto_digest_env_t *digest, +void crypto_digest_get_digest(crypto_digest_t *digest, char *out, size_t out_len); -crypto_digest_env_t *crypto_digest_dup(const crypto_digest_env_t *digest); -void crypto_digest_assign(crypto_digest_env_t *into, - const crypto_digest_env_t *from); +crypto_digest_t *crypto_digest_dup(const crypto_digest_t *digest); +void crypto_digest_assign(crypto_digest_t *into, + const crypto_digest_t *from); void crypto_hmac_sha1(char *hmac_out, const char *key, size_t key_len, const char *msg, size_t msg_len); @@ -203,15 +227,15 @@ void crypto_hmac_sha256(char *hmac_out, #define DH_TYPE_CIRCUIT 1 #define DH_TYPE_REND 2 #define DH_TYPE_TLS 3 -crypto_dh_env_t *crypto_dh_new(int dh_type); -int crypto_dh_get_bytes(crypto_dh_env_t *dh); -int crypto_dh_generate_public(crypto_dh_env_t *dh); -int crypto_dh_get_public(crypto_dh_env_t *dh, char *pubkey_out, +crypto_dh_t *crypto_dh_new(int dh_type); +int crypto_dh_get_bytes(crypto_dh_t *dh); +int crypto_dh_generate_public(crypto_dh_t *dh); +int crypto_dh_get_public(crypto_dh_t *dh, char *pubkey_out, size_t pubkey_out_len); -ssize_t crypto_dh_compute_secret(int severity, crypto_dh_env_t *dh, +ssize_t crypto_dh_compute_secret(int severity, crypto_dh_t *dh, const char *pubkey, size_t pubkey_len, char *secret_out, size_t secret_out_len); -void crypto_dh_free(crypto_dh_env_t *dh); +void crypto_dh_free(crypto_dh_t *dh); int crypto_expand_key_material(const char *key_in, size_t in_len, char *key_out, size_t key_out_len); @@ -247,16 +271,20 @@ int digest256_from_base64(char *digest, const char *d64); void secret_to_key(char *key_out, size_t key_out_len, const char *secret, size_t secret_len, const char *s2k_specifier); +/** OpenSSL-based utility functions. */ +void memwipe(void *mem, uint8_t byte, size_t sz); + #ifdef CRYPTO_PRIVATE -/* Prototypes for private functions only used by tortls.c and crypto.c */ +/* Prototypes for private functions only used by tortls.c, crypto.c, and the + * unit tests. */ struct rsa_st; struct evp_pkey_st; struct dh_st; -struct rsa_st *_crypto_pk_env_get_rsa(crypto_pk_env_t *env); -crypto_pk_env_t *_crypto_new_pk_env_rsa(struct rsa_st *rsa); -struct evp_pkey_st *_crypto_pk_env_get_evp_pkey(crypto_pk_env_t *env, +struct rsa_st *_crypto_pk_get_rsa(crypto_pk_t *env); +crypto_pk_t *_crypto_new_pk_from_rsa(struct rsa_st *rsa); +struct evp_pkey_st *_crypto_pk_get_evp_pkey(crypto_pk_t *env, int private); -struct dh_st *_crypto_dh_env_get_dh(crypto_dh_env_t *dh); +struct dh_st *_crypto_dh_get_dh(crypto_dh_t *dh); /* Prototypes for private functions only used by crypto.c and test.c*/ void add_spaces_to_fp(char *out, size_t outlen, const char *in); #endif diff --git a/src/common/di_ops.c b/src/common/di_ops.c index b22a58d1b1..7683c59dee 100644 --- a/src/common/di_ops.c +++ b/src/common/di_ops.c @@ -1,4 +1,4 @@ -/* Copyright (c) 2011, The Tor Project, Inc. */ +/* Copyright (c) 2011-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** diff --git a/src/common/di_ops.h b/src/common/di_ops.h index fa7d86806a..8f0bb698f9 100644 --- a/src/common/di_ops.h +++ b/src/common/di_ops.h @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** diff --git a/src/common/get_mozilla_ciphers.py b/src/common/get_mozilla_ciphers.py new file mode 100644 index 0000000000..c7e9a84a0e --- /dev/null +++ b/src/common/get_mozilla_ciphers.py @@ -0,0 +1,192 @@ +#!/usr/bin/python +# coding=utf-8 +# Copyright 2011, The Tor Project, Inc +# original version by Arturo Filastò +# See LICENSE for licensing information + +# This script parses Firefox and OpenSSL sources, and uses this information +# to generate a ciphers.inc file. +# +# It takes two arguments: the location of a firefox source directory, and the +# location of an openssl source directory. + +import os +import re +import sys + +if len(sys.argv) != 3: + print >>sys.stderr, "Syntax: get_mozilla_ciphers.py <firefox-source-dir> <openssl-source-dir>" + sys.exit(1) + +ff_root = sys.argv[1] +ossl_root = sys.argv[2] + +def ff(s): + return os.path.join(ff_root, s) +def ossl(s): + return os.path.join(ossl_root, s) + +##### +# Read the cpp file to understand what Ciphers map to what name : +# Make "ciphers" a map from name used in the javascript to a cipher macro name +fileA = open(ff('security/manager/ssl/src/nsNSSComponent.cpp'),'r') + +# The input format is a file containing exactly one section of the form: +# static CipherPref CipherPrefs[] = { +# {"name", MACRO_NAME}, // comment +# ... +# {NULL, 0} +# } + +inCipherSection = False +cipherLines = [] +for line in fileA: + if line.startswith('static CipherPref CipherPrefs'): + # Get the starting boundary of the Cipher Preferences + inCipherSection = True + elif inCipherSection: + line = line.strip() + if line.startswith('{NULL, 0}'): + # At the ending boundary of the Cipher Prefs + break + else: + cipherLines.append(line) +fileA.close() + +# Parse the lines and put them into a dict +ciphers = {} +cipher_pref = {} +for line in cipherLines: + m = re.search(r'^{\s*\"([^\"]+)\",\s*(\S*)\s*}', line) + if m: + key,value = m.groups() + ciphers[key] = value + cipher_pref[value] = key + +#### +# Now find the correct order for the ciphers +fileC = open(ff('security/nss/lib/ssl/ssl3con.c'), 'r') +firefox_ciphers = [] +inEnum=False +for line in fileC: + if not inEnum: + if "ssl3CipherSuiteCfg cipherSuites[" in line: + inEnum = True + continue + + if line.startswith("};"): + break + + m = re.match(r'^\s*\{\s*([A-Z_0-9]+),', line) + if m: + firefox_ciphers.append(m.group(1)) + +fileC.close() + +##### +# Read the JS file to understand what ciphers are enabled. The format is +# pref("name", true/false); +# Build a map enabled_ciphers from javascript name to "true" or "false", +# and an (unordered!) list of the macro names for those ciphers that are +# enabled. +fileB = open(ff('netwerk/base/public/security-prefs.js'), 'r') + +enabled_ciphers = {} +for line in fileB: + m = re.match(r'pref\(\"([^\"]+)\"\s*,\s*(\S*)\s*\)', line) + if not m: + continue + key, val = m.groups() + if key.startswith("security.ssl3"): + enabled_ciphers[key] = val +fileB.close() + +used_ciphers = [] +for k, v in enabled_ciphers.items(): + if v == "true": + used_ciphers.append(ciphers[k]) + +#oSSLinclude = ('/usr/include/openssl/ssl3.h', '/usr/include/openssl/ssl.h', +# '/usr/include/openssl/ssl2.h', '/usr/include/openssl/ssl23.h', +# '/usr/include/openssl/tls1.h') +oSSLinclude = ('ssl/ssl3.h', 'ssl/ssl.h', + 'ssl/ssl2.h', 'ssl/ssl23.h', + 'ssl/tls1.h') + +##### +# This reads the hex code for the ciphers that are used by firefox. +# sslProtoD is set to a map from macro name to macro value in sslproto.h; +# cipher_codes is set to an (unordered!) list of these hex values. +sslProto = open(ff('security/nss/lib/ssl/sslproto.h'), 'r') +sslProtoD = {} + +for line in sslProto: + m = re.match('#define\s+(\S+)\s+(\S+)', line) + if m: + key, value = m.groups() + sslProtoD[key] = value +sslProto.close() + +cipher_codes = [] +for x in used_ciphers: + cipher_codes.append(sslProtoD[x].lower()) + +#### +# Now read through all the openssl include files, and try to find the openssl +# macro names for those files. +openssl_macro_by_hex = {} +all_openssl_macros = {} +for fl in oSSLinclude: + fp = open(ossl(fl), 'r') + for line in fp.readlines(): + m = re.match('#define\s+(\S+)\s+(\S+)', line) + if m: + value,key = m.groups() + if key.startswith('0x') and "_CK_" in value: + key = key.replace('0x0300','0x').lower() + #print "%s %s" % (key, value) + openssl_macro_by_hex[key] = value + all_openssl_macros[value]=key + fp.close() + +# Now generate the output. +print """\ +/* This is an include file used to define the list of ciphers clients should + * advertise. Before including it, you should define the CIPHER and XCIPHER + * macros. + * + * This file was automatically generated by get_mozilla_ciphers.py. + */""" +# Go in order by the order in CipherPrefs +for firefox_macro in firefox_ciphers: + + try: + js_cipher_name = cipher_pref[firefox_macro] + except KeyError: + # This one has no javascript preference. + continue + + # The cipher needs to be enabled in security-prefs.js + if enabled_ciphers.get(js_cipher_name, 'false') != 'true': + continue + + hexval = sslProtoD[firefox_macro].lower() + + try: + openssl_macro = openssl_macro_by_hex[hexval.lower()] + openssl_macro = openssl_macro.replace("_CK_", "_TXT_") + if openssl_macro not in all_openssl_macros: + raise KeyError() + format = {'hex':hexval, 'macro':openssl_macro, 'note':""} + except KeyError: + # openssl doesn't have a macro for this. + format = {'hex':hexval, 'macro':firefox_macro, + 'note':"/* No openssl macro found for "+hexval+" */\n"} + + res = """\ +%(note)s#ifdef %(macro)s + CIPHER(%(hex)s, %(macro)s) +#else + XCIPHER(%(hex)s, %(macro)s) +#endif""" % format + print res diff --git a/src/common/ht.h b/src/common/ht.h index 0850c07092..25156c4165 100644 --- a/src/common/ht.h +++ b/src/common/ht.h @@ -1,6 +1,6 @@ /* Copyright (c) 2002, Christopher Clark. * Copyright (c) 2005-2006, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See license at end. */ /* Based on ideas by Christopher Clark and interfaces from Niels Provos. */ @@ -25,19 +25,22 @@ #define HT_INITIALIZER() \ { NULL, 0, 0, 0, -1 } +#ifdef HT_NO_CACHE_HASH_VALUES +#define HT_ENTRY(type) \ + struct { \ + struct type *hte_next; \ + } +#else #define HT_ENTRY(type) \ struct { \ struct type *hte_next; \ unsigned hte_hash; \ } +#endif #define HT_EMPTY(head) \ ((head)->hth_n_entries == 0) -/* Helper: alias for the bucket containing 'elm'. */ -#define _HT_BUCKET(head, field, elm) \ - ((head)->hth_table[elm->field.hte_hash % head->hth_table_length]) - /* How many elements in 'head'? */ #define HT_SIZE(head) \ ((head)->hth_n_entries) @@ -98,8 +101,26 @@ ht_string_hash(const char *s) return h; } -#define _HT_SET_HASH(elm, field, hashfn) \ - (elm)->field.hte_hash = hashfn(elm) +#ifndef HT_NO_CACHE_HASH_VALUES +#define HT_SET_HASH_(elm, field, hashfn) \ + do { (elm)->field.hte_hash = hashfn(elm); } while (0) +#define HT_SET_HASHVAL_(elm, field, val) \ + do { (elm)->field.hte_hash = (val); } while (0) +#define HT_ELT_HASH_(elm, field, hashfn) \ + ((elm)->field.hte_hash) +#else +#define HT_SET_HASH_(elm, field, hashfn) \ + ((void)0) +#define HT_ELT_HASH_(elm, field, hashfn) \ + (hashfn(elm)) +#define HT_SET_HASHVAL_(elm, field, val) \ + ((void)0) +#endif + +/* Helper: alias for the bucket containing 'elm'. */ +#define HT_BUCKET_(head, field, elm, hashfn) \ + ((head)->hth_table[HT_ELT_HASH_(elm,field,hashfn) \ + % head->hth_table_length]) #define HT_FOREACH(x, name, head) \ for ((x) = HT_START(name, head); \ @@ -109,7 +130,7 @@ ht_string_hash(const char *s) #define HT_PROTOTYPE(name, type, field, hashfn, eqfn) \ int name##_HT_GROW(struct name *ht, unsigned min_capacity); \ void name##_HT_CLEAR(struct name *ht); \ - int _##name##_HT_REP_IS_BAD(const struct name *ht); \ + int name##_HT_REP_IS_BAD_(const struct name *ht); \ static INLINE void \ name##_HT_INIT(struct name *head) { \ head->hth_table_length = 0; \ @@ -121,12 +142,12 @@ ht_string_hash(const char *s) /* Helper: returns a pointer to the right location in the table \ * 'head' to find or insert the element 'elm'. */ \ static INLINE struct type ** \ - _##name##_HT_FIND_P(struct name *head, struct type *elm) \ + name##_HT_FIND_P_(struct name *head, struct type *elm) \ { \ struct type **p; \ if (!head->hth_table) \ return NULL; \ - p = &_HT_BUCKET(head, field, elm); \ + p = &HT_BUCKET_(head, field, elm, hashfn); \ while (*p) { \ if (eqfn(*p, elm)) \ return p; \ @@ -141,8 +162,8 @@ ht_string_hash(const char *s) { \ struct type **p; \ struct name *h = (struct name *) head; \ - _HT_SET_HASH(elm, field, hashfn); \ - p = _##name##_HT_FIND_P(h, elm); \ + HT_SET_HASH_(elm, field, hashfn); \ + p = name##_HT_FIND_P_(h, elm); \ return p ? *p : NULL; \ } \ /* Insert the element 'elm' into the table 'head'. Do not call this \ @@ -154,8 +175,8 @@ ht_string_hash(const char *s) if (!head->hth_table || head->hth_n_entries >= head->hth_load_limit) \ name##_HT_GROW(head, head->hth_n_entries+1); \ ++head->hth_n_entries; \ - _HT_SET_HASH(elm, field, hashfn); \ - p = &_HT_BUCKET(head, field, elm); \ + HT_SET_HASH_(elm, field, hashfn); \ + p = &HT_BUCKET_(head, field, elm, hashfn); \ elm->field.hte_next = *p; \ *p = elm; \ } \ @@ -168,8 +189,8 @@ ht_string_hash(const char *s) struct type **p, *r; \ if (!head->hth_table || head->hth_n_entries >= head->hth_load_limit) \ name##_HT_GROW(head, head->hth_n_entries+1); \ - _HT_SET_HASH(elm, field, hashfn); \ - p = _##name##_HT_FIND_P(head, elm); \ + HT_SET_HASH_(elm, field, hashfn); \ + p = name##_HT_FIND_P_(head, elm); \ r = *p; \ *p = elm; \ if (r && (r!=elm)) { \ @@ -187,8 +208,8 @@ ht_string_hash(const char *s) name##_HT_REMOVE(struct name *head, struct type *elm) \ { \ struct type **p, *r; \ - _HT_SET_HASH(elm, field, hashfn); \ - p = _##name##_HT_FIND_P(head,elm); \ + HT_SET_HASH_(elm, field, hashfn); \ + p = name##_HT_FIND_P_(head,elm); \ if (!p || !*p) \ return NULL; \ r = *p; \ @@ -207,7 +228,6 @@ ht_string_hash(const char *s) void *data) \ { \ unsigned idx; \ - int remove; \ struct type **p, **nextp, *next; \ if (!head->hth_table) \ return; \ @@ -216,8 +236,7 @@ ht_string_hash(const char *s) while (*p) { \ nextp = &(*p)->field.hte_next; \ next = *nextp; \ - remove = fn(*p, data); \ - if (remove) { \ + if (fn(*p, data)) { \ --head->hth_n_entries; \ *p = next; \ } else { \ @@ -251,7 +270,8 @@ ht_string_hash(const char *s) if ((*elm)->field.hte_next) { \ return &(*elm)->field.hte_next; \ } else { \ - unsigned b = ((*elm)->field.hte_hash % head->hth_table_length)+1; \ + unsigned b = (HT_ELT_HASH_(*elm, field, hashfn) \ + % head->hth_table_length)+1; \ while (b < head->hth_table_length) { \ if (head->hth_table[b]) \ return &head->hth_table[b]; \ @@ -263,7 +283,7 @@ ht_string_hash(const char *s) static INLINE struct type ** \ name##_HT_NEXT_RMV(struct name *head, struct type **elm) \ { \ - unsigned h = (*elm)->field.hte_hash; \ + unsigned h = HT_ELT_HASH_(*elm, field, hashfn); \ *elm = (*elm)->field.hte_next; \ --head->hth_n_entries; \ if (*elm) { \ @@ -320,7 +340,7 @@ ht_string_hash(const char *s) elm = head->hth_table[b]; \ while (elm) { \ next = elm->field.hte_next; \ - b2 = elm->field.hte_hash % new_len; \ + b2 = HT_ELT_HASH_(elm, field, hashfn) % new_len; \ elm->field.hte_next = new_table[b2]; \ new_table[b2] = elm; \ elm = next; \ @@ -338,7 +358,7 @@ ht_string_hash(const char *s) for (b=0; b < head->hth_table_length; ++b) { \ struct type *e, **pE; \ for (pE = &new_table[b], e = *pE; e != NULL; e = *pE) { \ - b2 = e->field.hte_hash % new_len; \ + b2 = HT_ELT_HASH_(e, field, hashfn) % new_len; \ if (b2 == b) { \ pE = &e->field.hte_next; \ } else { \ @@ -368,7 +388,7 @@ ht_string_hash(const char *s) /* Debugging helper: return false iff the representation of 'head' is \ * internally consistent. */ \ int \ - _##name##_HT_REP_IS_BAD(const struct name *head) \ + name##_HT_REP_IS_BAD_(const struct name *head) \ { \ unsigned n, i; \ struct type *elm; \ @@ -390,9 +410,9 @@ ht_string_hash(const char *s) return 5; \ for (n = i = 0; i < head->hth_table_length; ++i) { \ for (elm = head->hth_table[i]; elm; elm = elm->field.hte_next) { \ - if (elm->field.hte_hash != hashfn(elm)) \ + if (HT_ELT_HASH_(elm, field, hashfn) != hashfn(elm)) \ return 1000 + i; \ - if ((elm->field.hte_hash % head->hth_table_length) != i) \ + if ((HT_ELT_HASH_(elm, field, hashfn) % head->hth_table_length) != i) \ return 10000 + i; \ ++n; \ } \ @@ -405,24 +425,24 @@ ht_string_hash(const char *s) /** Implements an over-optimized "find and insert if absent" block; * not meant for direct usage by typical code, or usage outside the critical * path.*/ -#define _HT_FIND_OR_INSERT(name, field, hashfn, head, eltype, elm, var, y, n) \ +#define HT_FIND_OR_INSERT_(name, field, hashfn, head, eltype, elm, var, y, n) \ { \ - struct name *_##var##_head = head; \ - eltype **var; \ - if (!_##var##_head->hth_table || \ - _##var##_head->hth_n_entries >= _##var##_head->hth_load_limit) \ - name##_HT_GROW(_##var##_head, _##var##_head->hth_n_entries+1); \ - _HT_SET_HASH((elm), field, hashfn); \ - var = _##name##_HT_FIND_P(_##var##_head, (elm)); \ + struct name *var##_head_ = head; \ + struct eltype **var; \ + if (!var##_head_->hth_table || \ + var##_head_->hth_n_entries >= var##_head_->hth_load_limit) \ + name##_HT_GROW(var##_head_, var##_head_->hth_n_entries+1); \ + HT_SET_HASH_((elm), field, hashfn); \ + var = name##_HT_FIND_P_(var##_head_, (elm)); \ if (*var) { \ y; \ } else { \ n; \ } \ } -#define _HT_FOI_INSERT(field, head, elm, newent, var) \ +#define HT_FOI_INSERT_(field, head, elm, newent, var) \ { \ - newent->field.hte_hash = (elm)->field.hte_hash; \ + HT_SET_HASHVAL_(newent, field, (elm)->field.hte_hash); \ newent->field.hte_next = NULL; \ *var = newent; \ ++((head)->hth_n_entries); \ @@ -431,9 +451,8 @@ ht_string_hash(const char *s) /* * Copyright 2005, Nick Mathewson. Implementation logic is adapted from code * by Christopher Clark, retrofit to allow drop-in memory management, and to - * use the same interface as Niels Provos's HT_H. I'm not sure whether this - * is a derived work any more, but whether it is or not, the license below - * applies. + * use the same interface as Niels Provos's tree.h. This is probably still + * a derived work, so the original license below still applies. * * Copyright (c) 2002, Christopher Clark * All rights reserved. diff --git a/src/common/log.c b/src/common/log.c index f2999f4e66..5e2e6b5b50 100644 --- a/src/common/log.c +++ b/src/common/log.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001, Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -97,6 +97,7 @@ should_log_function_name(log_domain_mask_t domain, int severity) /** A mutex to guard changes to logfiles and logging. */ static tor_mutex_t log_mutex; +/** True iff we have initialized log_mutex */ static int log_mutex_initialized = 0; /** Linked list of logfile_t. */ @@ -161,6 +162,17 @@ log_set_application_name(const char *name) appname = name ? tor_strdup(name) : NULL; } +/** Log time granularity in milliseconds. */ +static int log_time_granularity = 1; + +/** Define log time granularity for all logs to be <b>granularity_msec</b> + * milliseconds. */ +void +set_log_time_granularity(int granularity_msec) +{ + log_time_granularity = granularity_msec; +} + /** Helper: Write the standard prefix for log lines to a * <b>buf_len</b> character buffer in <b>buf</b>. */ @@ -171,14 +183,22 @@ _log_prefix(char *buf, size_t buf_len, int severity) struct timeval now; struct tm tm; size_t n; - int r; + int r, ms; tor_gettimeofday(&now); t = (time_t)now.tv_sec; + ms = (int)now.tv_usec / 1000; + if (log_time_granularity >= 1000) { + t -= t % (log_time_granularity / 1000); + ms = 0; + } else { + ms -= ((int)now.tv_usec / 1000) % log_time_granularity; + } n = strftime(buf, buf_len, "%b %d %H:%M:%S", tor_localtime_r(&t, &tm)); - r = tor_snprintf(buf+n, buf_len-n, ".%.3i [%s] ", - (int)now.tv_usec / 1000, sev_to_string(severity)); + r = tor_snprintf(buf+n, buf_len-n, ".%.3i [%s] ", ms, + sev_to_string(severity)); + if (r<0) return buf_len-1; else @@ -641,7 +661,7 @@ init_logging(void) log_mutex_initialized = 1; } if (pending_cb_messages == NULL) - pending_cb_messages = smartlist_create(); + pending_cb_messages = smartlist_new(); } /** Set whether we report logging domains as a part of our log messages. @@ -710,7 +730,7 @@ change_callback_log_severity(int loglevelMin, int loglevelMax, UNLOCK_LOGS(); } -/** If there are any log messages that were genered with LD_NOCB waiting to +/** If there are any log messages that were generated with LD_NOCB waiting to * be sent to callback-based loggers, send them now. */ void flush_pending_log_callbacks(void) @@ -725,7 +745,7 @@ flush_pending_log_callbacks(void) } messages = pending_cb_messages; - pending_cb_messages = smartlist_create(); + pending_cb_messages = smartlist_new(); do { SMARTLIST_FOREACH_BEGIN(messages, pending_cb_message_t *, msg) { const int severity = msg->severity; @@ -810,7 +830,7 @@ add_file_log(const log_severity_list_t *severity, const char *filename) int fd; logfile_t *lf; - fd = open(filename, O_WRONLY|O_CREAT|O_APPEND, 0644); + fd = tor_open_cloexec(filename, O_WRONLY|O_CREAT|O_APPEND, 0644); if (fd<0) return -1; if (tor_fd_seekend(fd)<0) @@ -887,7 +907,7 @@ log_level_to_string(int level) static const char *domain_list[] = { "GENERAL", "CRYPTO", "NET", "CONFIG", "FS", "PROTOCOL", "MM", "HTTP", "APP", "CONTROL", "CIRC", "REND", "BUG", "DIR", "DIRSERV", - "OR", "EDGE", "ACCT", "HIST", "HANDSHAKE", NULL + "OR", "EDGE", "ACCT", "HIST", "HANDSHAKE", "HEARTBEAT", NULL }; /** Return a bitmask for the log domain for which <b>domain</b> is the name, @@ -981,12 +1001,11 @@ parse_log_severity_config(const char **cfg_ptr, return -1; domains = 0; domains_str = tor_strndup(cfg+1, closebracket-cfg-1); - domains_list = smartlist_create(); + domains_list = smartlist_new(); smartlist_split_string(domains_list, domains_str, ",", SPLIT_SKIP_SPACE, -1); tor_free(domains_str); - SMARTLIST_FOREACH(domains_list, const char *, domain, - { + SMARTLIST_FOREACH_BEGIN(domains_list, const char *, domain) { if (!strcmp(domain, "*")) { domains = ~0u; } else { @@ -1007,7 +1026,7 @@ parse_log_severity_config(const char **cfg_ptr, domains |= d; } } - }); + } SMARTLIST_FOREACH_END(domain); SMARTLIST_FOREACH(domains_list, char *, d, tor_free(d)); smartlist_free(domains_list); if (err) diff --git a/src/common/memarea.c b/src/common/memarea.c index a6b8c4ee9c..07bd593cc9 100644 --- a/src/common/memarea.c +++ b/src/common/memarea.c @@ -1,4 +1,4 @@ -/* Copyright (c) 2008-2011, The Tor Project, Inc. */ +/* Copyright (c) 2008-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** \file memarea.c diff --git a/src/common/memarea.h b/src/common/memarea.h index 4d31dc95a8..b3c76d8d0c 100644 --- a/src/common/memarea.h +++ b/src/common/memarea.h @@ -1,4 +1,4 @@ -/* Copyright (c) 2008-2011, The Tor Project, Inc. */ +/* Copyright (c) 2008-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /* Tor dependencies */ diff --git a/src/common/mempool.c b/src/common/mempool.c index c444923189..637f081c88 100644 --- a/src/common/mempool.c +++ b/src/common/mempool.c @@ -1,4 +1,4 @@ -/* Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #if 1 /* Tor dependencies */ @@ -8,6 +8,7 @@ #include <stdlib.h> #include <string.h> #include "torint.h" +#include "crypto.h" #define MEMPOOL_PRIVATE #include "mempool.h" @@ -62,7 +63,6 @@ #if 1 /* Tor dependencies */ -#include "orconfig.h" #include "util.h" #include "compat.h" #include "torlog.h" @@ -137,7 +137,7 @@ struct mp_chunk_t { int capacity; /**< Number of items that can be fit into this chunk. */ size_t mem_size; /**< Number of usable bytes in mem. */ char *next_mem; /**< Pointer into part of <b>mem</b> not yet carved up. */ - char mem[1]; /**< Storage for this chunk. (Not actual size.) */ + char mem[FLEXIBLE_ARRAY_MEMBER]; /**< Storage for this chunk. */ }; /** Number of extra bytes needed beyond mem_size to allocate a chunk. */ @@ -520,7 +520,7 @@ mp_pool_destroy(mp_pool_t *pool) destroy_chunks(pool->empty_chunks); destroy_chunks(pool->used_chunks); destroy_chunks(pool->full_chunks); - memset(pool, 0xe0, sizeof(mp_pool_t)); + memwipe(pool, 0xe0, sizeof(mp_pool_t)); FREE(pool); } diff --git a/src/common/mempool.h b/src/common/mempool.h index fb1e9e8b42..bc424acdeb 100644 --- a/src/common/mempool.h +++ b/src/common/mempool.h @@ -1,4 +1,4 @@ -/* Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -22,6 +22,8 @@ void mp_pool_destroy(mp_pool_t *pool); void mp_pool_assert_ok(mp_pool_t *pool); void mp_pool_log_status(mp_pool_t *pool, int severity); +#define MP_POOL_ITEM_OVERHEAD (sizeof(void*)) + #define MEMPOOL_STATS #ifdef MEMPOOL_PRIVATE diff --git a/src/common/procmon.c b/src/common/procmon.c index 5c10e9a22b..36b1a48553 100644 --- a/src/common/procmon.c +++ b/src/common/procmon.c @@ -1,3 +1,5 @@ +/* Copyright (c) 2011-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ /** * \file procmon.c @@ -21,7 +23,7 @@ #include <errno.h> #endif -#ifdef MS_WINDOWS +#ifdef _WIN32 #include <windows.h> /* Windows does not define pid_t, but _getpid() returns an int. */ @@ -41,6 +43,7 @@ static void tor_process_monitor_poll_cb(evutil_socket_t unused1, short unused2, /* This struct may contain pointers into the original process * specifier string, but it should *never* contain anything which * needs to be freed. */ +/* DOCDOC parsed_process_specifier_t */ struct parsed_process_specifier_t { pid_t pid; }; @@ -81,6 +84,7 @@ parse_process_specifier(const char *process_spec, return -1; } +/* DOCDOC tor_process_monitor_t */ struct tor_process_monitor_t { /** Log domain for warning messages. */ log_domain_mask_t log_domain; @@ -91,7 +95,7 @@ struct tor_process_monitor_t { * polls. */ pid_t pid; -#ifdef MS_WINDOWS +#ifdef _WIN32 /** Windows-only: Should we poll hproc? If false, poll pid * instead. */ int poll_hproc; @@ -152,6 +156,7 @@ tor_validate_process_specifier(const char *process_spec, #define PERIODIC_TIMER_FLAGS (0) #endif +/* DOCDOC poll_interval_tv */ static struct timeval poll_interval_tv = {15, 0}; /* Note: If you port this file to plain Libevent 2, you can make * poll_interval_tv const. It has to be non-const here because in @@ -192,7 +197,7 @@ tor_process_monitor_new(struct event_base *base, procmon->pid = ppspec.pid; -#ifdef MS_WINDOWS +#ifdef _WIN32 procmon->hproc = OpenProcess(PROCESS_QUERY_INFORMATION | SYNCHRONIZE, FALSE, procmon->pid); @@ -246,7 +251,7 @@ tor_process_monitor_poll_cb(evutil_socket_t unused1, short unused2, tor_assert(procmon != NULL); -#ifdef MS_WINDOWS +#ifdef _WIN32 if (procmon->poll_hproc) { DWORD exit_code; if (!GetExitCodeProcess(procmon->hproc, &exit_code)) { @@ -323,7 +328,7 @@ tor_process_monitor_free(tor_process_monitor_t *procmon) if (procmon == NULL) return; -#ifdef MS_WINDOWS +#ifdef _WIN32 if (procmon->hproc != NULL) CloseHandle(procmon->hproc); #endif diff --git a/src/common/procmon.h b/src/common/procmon.h index 02eb2da61c..88d64d6a15 100644 --- a/src/common/procmon.h +++ b/src/common/procmon.h @@ -1,3 +1,5 @@ +/* Copyright (c) 2011-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ /** * \file procmon.h @@ -14,6 +16,7 @@ typedef struct tor_process_monitor_t tor_process_monitor_t; +/* DOCDOC tor_procmon_callback_t */ typedef void (*tor_procmon_callback_t)(void *); int tor_validate_process_specifier(const char *process_spec, diff --git a/src/common/sha256.c b/src/common/sha256.c index 258b7e062a..813c68d2a3 100644 --- a/src/common/sha256.c +++ b/src/common/sha256.c @@ -1,4 +1,4 @@ -/* Copyright (c) 2009-2011, The Tor Project, Inc. */ +/* Copyright (c) 2009-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /* This SHA256 implementation is adapted from the public domain one in LibTomCrypt, version 1.6. Tor uses it on platforms where OpenSSL doesn't diff --git a/src/common/torgzip.c b/src/common/torgzip.c index a26e5b21e5..da4136228e 100644 --- a/src/common/torgzip.c +++ b/src/common/torgzip.c @@ -1,6 +1,6 @@ /* Copyright (c) 2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -44,11 +44,7 @@ #define off64_t int64_t #endif -#ifdef _MSC_VER -#include "..\..\contrib\zlib\zlib.h" -#else #include <zlib.h> -#endif /** Set to 1 if zlib is a version that supports gzip; set to 0 if it doesn't; * set to -1 if we haven't checked yet. */ diff --git a/src/common/torgzip.h b/src/common/torgzip.h index 15e09eb700..d3ded81f9c 100644 --- a/src/common/torgzip.h +++ b/src/common/torgzip.h @@ -1,6 +1,6 @@ /* Copyright (c) 2003, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** diff --git a/src/common/torint.h b/src/common/torint.h index 0b5c29adc0..8771802d70 100644 --- a/src/common/torint.h +++ b/src/common/torint.h @@ -1,6 +1,6 @@ /* Copyright (c) 2003, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -111,6 +111,15 @@ typedef signed int int32_t; typedef unsigned int uint32_t; #define HAVE_UINT32_T #endif +#ifndef UINT16_MAX +#define UINT16_MAX 0xffffu +#endif +#ifndef INT16_MAX +#define INT16_MAX 0x7fff +#endif +#ifndef INT16_MIN +#define INT16_MIN (-INT16_MAX-1) +#endif #ifndef UINT32_MAX #define UINT32_MAX 0xffffffffu #endif @@ -182,6 +191,16 @@ typedef unsigned __int64 uint64_t; #endif #endif +#ifndef SIZE_MAX +#if SIZEOF_SIZE_T == 8 +#define SIZE_MAX UINT64_MAX +#elif SIZEOF_SIZE_T == 4 +#define SIZE_MAX UINT32_MAX +#else +#error "Can't define SIZE_MAX" +#endif +#endif + #ifndef HAVE_SSIZE_T #if SIZEOF_SIZE_T == 8 typedef int64_t ssize_t; diff --git a/src/common/torlog.h b/src/common/torlog.h index 541a0d1738..28890a44af 100644 --- a/src/common/torlog.h +++ b/src/common/torlog.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001, Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -92,8 +92,10 @@ #define LD_HIST (1u<<18) /** OR handshaking */ #define LD_HANDSHAKE (1u<<19) +/** Heartbeat messages */ +#define LD_HEARTBEAT (1u<<20) /** Number of logging domains in the code. */ -#define N_LOGGING_DOMAINS 20 +#define N_LOGGING_DOMAINS 21 /** This log message is not safe to send to a callback-based logger * immediately. Used as a flag, not a log domain. */ @@ -145,12 +147,13 @@ void change_callback_log_severity(int loglevelMin, int loglevelMax, log_callback cb); void flush_pending_log_callbacks(void); void log_set_application_name(const char *name); +void set_log_time_granularity(int granularity_msec); void tor_log(int severity, log_domain_mask_t domain, const char *format, ...) CHECK_PRINTF(3,4); #define log tor_log /* hack it so we don't conflict with log() as much */ -#ifdef __GNUC__ +#if defined(__GNUC__) || defined(RUNNING_DOXYGEN) extern int _log_global_min_severity; void _log_fn(int severity, log_domain_mask_t domain, diff --git a/src/common/tortls.c b/src/common/tortls.c index fc0bcb9524..60aac64929 100644 --- a/src/common/tortls.c +++ b/src/common/tortls.c @@ -1,6 +1,6 @@ /* Copyright (c) 2003, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -21,9 +21,10 @@ #endif #include <assert.h> -#ifdef MS_WINDOWS /*wrkard for dtls1.h >= 0.9.8m of "#include <winsock.h>"*/ - #define WIN32_WINNT 0x400 - #define _WIN32_WINNT 0x400 +#ifdef _WIN32 /*wrkard for dtls1.h >= 0.9.8m of "#include <winsock.h>"*/ + #ifndef _WIN32_WINNT + #define _WIN32_WINNT 0x0501 + #endif #define WIN32_LEAN_AND_MEAN #if defined(_MSC_VER) && (_MSC_VER < 1300) #include <winsock.h> @@ -40,20 +41,27 @@ #include <openssl/bio.h> #include <openssl/opensslv.h> -#if OPENSSL_VERSION_NUMBER < 0x00907000l -#error "We require OpenSSL >= 0.9.7" +#ifdef USE_BUFFEREVENTS +#include <event2/bufferevent_ssl.h> +#include <event2/buffer.h> +#include <event2/event.h> +#include "compat_libevent.h" #endif #define CRYPTO_PRIVATE /* to import prototypes from crypto.h */ +#define TORTLS_PRIVATE #include "crypto.h" #include "tortls.h" #include "util.h" #include "torlog.h" #include "container.h" -#include "ht.h" #include <string.h> +#if OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(0,9,7) +#error "We require OpenSSL >= 0.9.7" +#endif + /* Enable the "v2" TLS handshake. */ #define V2_HANDSHAKE_SERVER @@ -68,9 +76,9 @@ #define ADDR(tls) (((tls) && (tls)->address) ? tls->address : "peer") -#if (OPENSSL_VERSION_NUMBER < 0x0090813fL || \ - (OPENSSL_VERSION_NUMBER >= 0x00909000L && \ - OPENSSL_VERSION_NUMBER < 0x1000006fL)) +#if (OPENSSL_VERSION_NUMBER < OPENSSL_V(0,9,8,'s') || \ + (OPENSSL_VERSION_NUMBER >= OPENSSL_V_SERIES(0,9,9) && \ + OPENSSL_VERSION_NUMBER < OPENSSL_V(1,0,0,'f'))) /* This is a version of OpenSSL before 0.9.8s/1.0.0f. It does not have * the CVE-2011-4576 fix, and as such it can't use RELEASE_BUFFERS and * SSL3 safely at the same time. @@ -96,22 +104,36 @@ static int use_unsafe_renegotiation_op = 0; * SSL3_FLAGS_ALLOW_UNSAFE_LEGACY_RENEGOTIATION? */ static int use_unsafe_renegotiation_flag = 0; +/** Structure that we use for a single certificate. */ +struct tor_cert_t { + X509 *cert; + uint8_t *encoded; + size_t encoded_len; + unsigned pkey_digests_set : 1; + digests_t cert_digests; + digests_t pkey_digests; +}; + /** Holds a SSL_CTX object and related state used to configure TLS * connections. */ typedef struct tor_tls_context_t { int refcnt; SSL_CTX *ctx; - X509 *my_cert; - X509 *my_id_cert; - crypto_pk_env_t *key; + tor_cert_t *my_link_cert; + tor_cert_t *my_id_cert; + tor_cert_t *my_auth_cert; + crypto_pk_t *link_key; + crypto_pk_t *auth_key; } tor_tls_context_t; +#define TOR_TLS_MAGIC 0x71571571 + /** Holds a SSL object and its associated data. Members are only * accessed from within tortls.c. */ struct tor_tls_t { - HT_ENTRY(tor_tls_t) node; + uint32_t magic; tor_tls_context_t *context; /** A link to the context object for this tls. */ SSL *ssl; /**< An OpenSSL SSL object. */ int socket; /**< The underlying file descriptor for this TLS connection. */ @@ -119,6 +141,7 @@ struct tor_tls_t { enum { TOR_TLS_ST_HANDSHAKE, TOR_TLS_ST_OPEN, TOR_TLS_ST_GOTCLOSE, TOR_TLS_ST_SENTCLOSE, TOR_TLS_ST_CLOSED, TOR_TLS_ST_RENEGOTIATE, + TOR_TLS_ST_BUFFEREVENT } state : 3; /**< The current SSL state, depending on which operations have * completed successfully. */ unsigned int isServer:1; /**< True iff this is a server-side connection */ @@ -127,8 +150,10 @@ struct tor_tls_t { * of the connection protocol (client sends * different cipher list, server sends only * one certificate). */ - /** True iff we should call negotiated_callback when we're done reading. */ + /** True iff we should call negotiated_callback when we're done reading. */ unsigned int got_renegotiate:1; + /** Incremented every time we start the server side of a handshake. */ + uint8_t server_handshake_count; size_t wantwrite_n; /**< 0 normally, >0 if we returned wantwrite last * time. */ /** Last values retrieved from BIO_number_read()/write(); see @@ -153,65 +178,58 @@ static SSL_CIPHER *CLIENT_CIPHER_DUMMIES = NULL; static STACK_OF(SSL_CIPHER) *CLIENT_CIPHER_STACK = NULL; #endif -/** Helper: compare tor_tls_t objects by its SSL. */ -static INLINE int -tor_tls_entries_eq(const tor_tls_t *a, const tor_tls_t *b) -{ - return a->ssl == b->ssl; -} +/** The ex_data index in which we store a pointer to an SSL object's + * corresponding tor_tls_t object. */ +static int tor_tls_object_ex_data_index = -1; -/** Helper: return a hash value for a tor_tls_t by its SSL. */ -static INLINE unsigned int -tor_tls_entry_hash(const tor_tls_t *a) +/** Helper: Allocate tor_tls_object_ex_data_index. */ +static void +tor_tls_allocate_tor_tls_object_ex_data_index(void) { -#if SIZEOF_INT == SIZEOF_VOID_P - return ((unsigned int)(uintptr_t)a->ssl); -#else - return (unsigned int) ((((uint64_t)a->ssl)>>2) & UINT_MAX); -#endif + if (tor_tls_object_ex_data_index == -1) { + tor_tls_object_ex_data_index = + SSL_get_ex_new_index(0, NULL, NULL, NULL, NULL); + tor_assert(tor_tls_object_ex_data_index != -1); + } } -/** Map from SSL* pointers to tor_tls_t objects using those pointers. - */ -static HT_HEAD(tlsmap, tor_tls_t) tlsmap_root = HT_INITIALIZER(); - -HT_PROTOTYPE(tlsmap, tor_tls_t, node, tor_tls_entry_hash, - tor_tls_entries_eq) -HT_GENERATE(tlsmap, tor_tls_t, node, tor_tls_entry_hash, - tor_tls_entries_eq, 0.6, malloc, realloc, free) - /** Helper: given a SSL* pointer, return the tor_tls_t object using that * pointer. */ static INLINE tor_tls_t * tor_tls_get_by_ssl(const SSL *ssl) { - tor_tls_t search, *result; - memset(&search, 0, sizeof(search)); - search.ssl = (SSL*)ssl; - result = HT_FIND(tlsmap, &tlsmap_root, &search); + tor_tls_t *result = SSL_get_ex_data(ssl, tor_tls_object_ex_data_index); + if (result) + tor_assert(result->magic == TOR_TLS_MAGIC); return result; } static void tor_tls_context_decref(tor_tls_context_t *ctx); static void tor_tls_context_incref(tor_tls_context_t *ctx); -static X509* tor_tls_create_certificate(crypto_pk_env_t *rsa, - crypto_pk_env_t *rsa_sign, +static X509* tor_tls_create_certificate(crypto_pk_t *rsa, + crypto_pk_t *rsa_sign, const char *cname, const char *cname_sign, unsigned int lifetime); -static void tor_tls_unblock_renegotiation(tor_tls_t *tls); + static int tor_tls_context_init_one(tor_tls_context_t **ppcontext, - crypto_pk_env_t *identity, + crypto_pk_t *identity, unsigned int key_lifetime, int is_client); -static tor_tls_context_t *tor_tls_context_new(crypto_pk_env_t *identity, +static tor_tls_context_t *tor_tls_context_new(crypto_pk_t *identity, unsigned int key_lifetime, int is_client); +static int check_cert_lifetime_internal(int severity, const X509 *cert, + int past_tolerance, int future_tolerance); /** Global TLS contexts. We keep them here because nobody else needs - * to touch them. */ + * to touch them. + * + * @{ */ static tor_tls_context_t *server_tls_context = NULL; static tor_tls_context_t *client_tls_context = NULL; +/**@}*/ + /** True iff tor_tls_init() has been called. */ static int tls_library_is_initialized = 0; @@ -219,52 +237,97 @@ static int tls_library_is_initialized = 0; #define _TOR_TLS_SYSCALL (_MIN_TOR_TLS_ERROR_VAL - 2) #define _TOR_TLS_ZERORETURN (_MIN_TOR_TLS_ERROR_VAL - 1) -#include "tortls_states.h" +/** Write a description of the current state of <b>tls</b> into the + * <b>sz</b>-byte buffer at <b>buf</b>. */ +void +tor_tls_get_state_description(tor_tls_t *tls, char *buf, size_t sz) +{ + const char *ssl_state; + const char *tortls_state; + + if (PREDICT_UNLIKELY(!tls || !tls->ssl)) { + strlcpy(buf, "(No SSL object)", sz); + return; + } -/** Return the symbolic name of an OpenSSL state. */ -static const char * -ssl_state_to_string(int ssl_state) + ssl_state = SSL_state_string_long(tls->ssl); + switch (tls->state) { +#define CASE(st) case TOR_TLS_ST_##st: tortls_state = " in "#st ; break + CASE(HANDSHAKE); + CASE(OPEN); + CASE(GOTCLOSE); + CASE(SENTCLOSE); + CASE(CLOSED); + CASE(RENEGOTIATE); +#undef CASE + case TOR_TLS_ST_BUFFEREVENT: + tortls_state = ""; + break; + default: + tortls_state = " in unknown TLS state"; + break; + } + + tor_snprintf(buf, sz, "%s%s", ssl_state, tortls_state); +} + +/** Log a single error <b>err</b> as returned by ERR_get_error(), which was + * received while performing an operation <b>doing</b> on <b>tls</b>. Log + * the message at <b>severity</b>, in log domain <b>domain</b>. */ +void +tor_tls_log_one_error(tor_tls_t *tls, unsigned long err, + int severity, int domain, const char *doing) { - static char buf[40]; - int i; - for (i = 0; state_map[i].name; ++i) { - if (state_map[i].state == ssl_state) - return state_map[i].name; + const char *state = NULL, *addr; + const char *msg, *lib, *func; + + state = (tls && tls->ssl)?SSL_state_string_long(tls->ssl):"---"; + + addr = tls ? tls->address : NULL; + + /* Some errors are known-benign, meaning they are the fault of the other + * side of the connection. The caller doesn't know this, so override the + * priority for those cases. */ + switch (ERR_GET_REASON(err)) { + case SSL_R_HTTP_REQUEST: + case SSL_R_HTTPS_PROXY_REQUEST: + case SSL_R_RECORD_LENGTH_MISMATCH: + case SSL_R_RECORD_TOO_LARGE: + case SSL_R_UNKNOWN_PROTOCOL: + case SSL_R_UNSUPPORTED_PROTOCOL: + severity = LOG_INFO; + break; + default: + break; + } + + msg = (const char*)ERR_reason_error_string(err); + lib = (const char*)ERR_lib_error_string(err); + func = (const char*)ERR_func_error_string(err); + if (!msg) msg = "(null)"; + if (!lib) lib = "(null)"; + if (!func) func = "(null)"; + if (doing) { + log(severity, domain, "TLS error while %s%s%s: %s (in %s:%s:%s)", + doing, addr?" with ":"", addr?addr:"", + msg, lib, func, state); + } else { + log(severity, domain, "TLS error%s%s: %s (in %s:%s:%s)", + addr?" with ":"", addr?addr:"", + msg, lib, func, state); } - tor_snprintf(buf, sizeof(buf), "Unknown state %d", ssl_state); - return buf; } -/** Log all pending tls errors at level <b>severity</b>. Use - * <b>doing</b> to describe our current activities. +/** Log all pending tls errors at level <b>severity</b> in log domain + * <b>domain</b>. Use <b>doing</b> to describe our current activities. */ static void tls_log_errors(tor_tls_t *tls, int severity, int domain, const char *doing) { - const char *state = NULL; - int st; unsigned long err; - const char *msg, *lib, *func, *addr; - addr = tls ? tls->address : NULL; - st = (tls && tls->ssl) ? tls->ssl->state : -1; + while ((err = ERR_get_error()) != 0) { - msg = (const char*)ERR_reason_error_string(err); - lib = (const char*)ERR_lib_error_string(err); - func = (const char*)ERR_func_error_string(err); - if (!state) - state = (st>=0)?ssl_state_to_string(st):"---"; - if (!msg) msg = "(null)"; - if (!lib) lib = "(null)"; - if (!func) func = "(null)"; - if (doing) { - log(severity, domain, "TLS error while %s%s%s: %s (in %s:%s:%s)", - doing, addr?" with ":"", addr?addr:"", - msg, lib, func, state); - } else { - log(severity, domain, "TLS error%s%s: %s (in %s:%s:%s)", - addr?" with ":"", addr?addr:"", - msg, lib, func, state); - } + tor_tls_log_one_error(tls, err, severity, domain, doing); } } @@ -273,7 +336,7 @@ tls_log_errors(tor_tls_t *tls, int severity, int domain, const char *doing) static int tor_errno_to_tls_error(int e) { -#if defined(MS_WINDOWS) +#if defined(_WIN32) switch (e) { case WSAECONNRESET: // most common return TOR_TLS_ERROR_CONNRESET; @@ -355,14 +418,14 @@ tor_tls_get_error(tor_tls_t *tls, int r, int extra, return _TOR_TLS_SYSCALL; if (r == 0) { log(severity, LD_NET, "TLS error: unexpected close while %s (%s)", - doing, ssl_state_to_string(tls->ssl->state)); + doing, SSL_state_string_long(tls->ssl)); tor_error = TOR_TLS_ERROR_IO; } else { int e = tor_socket_errno(tls->socket); log(severity, LD_NET, "TLS error: <syscall error while %s> (errno=%d: %s; state=%s)", doing, e, tor_socket_strerror(e), - ssl_state_to_string(tls->ssl->state)); + SSL_state_string_long(tls->ssl)); tor_error = tor_errno_to_tls_error(e); } tls_log_errors(tls, severity, domain, doing); @@ -371,7 +434,7 @@ tor_tls_get_error(tor_tls_t *tls, int r, int extra, if (extra&CATCH_ZERO) return _TOR_TLS_ZERORETURN; log(severity, LD_NET, "TLS connection closed while %s in state %s", - doing, ssl_state_to_string(tls->ssl->state)); + doing, SSL_state_string_long(tls->ssl)); tls_log_errors(tls, severity, domain, doing); return TOR_TLS_CLOSE; default: @@ -414,18 +477,20 @@ tor_tls_init(void) * program should be allowed to use renegotiation unless it first passed * a test of intelligence and determination. */ - if (version >= 0x009080c0L && version < 0x009080d0L) { - log_notice(LD_GENERAL, "OpenSSL %s looks like version 0.9.8l; " - "I will try SSL3_FLAGS to enable renegotation.", + if (version > OPENSSL_V(0,9,8,'k') && version <= OPENSSL_V(0,9,8,'l')) { + log_notice(LD_GENERAL, "OpenSSL %s looks like version 0.9.8l, but " + "some vendors have backported renegotiation code from " + "0.9.8m without updating the version number. " + "I will try SSL3_FLAGS and SSL_OP to enable renegotation.", SSLeay_version(SSLEAY_VERSION)); use_unsafe_renegotiation_flag = 1; use_unsafe_renegotiation_op = 1; - } else if (version >= 0x009080d0L) { + } else if (version > OPENSSL_V(0,9,8,'l')) { log_notice(LD_GENERAL, "OpenSSL %s looks like version 0.9.8m or later; " "I will try SSL_OP to enable renegotiation", SSLeay_version(SSLEAY_VERSION)); use_unsafe_renegotiation_op = 1; - } else if (version < 0x009080c0L) { + } else if (version <= OPENSSL_V(0,9,8,'k')) { log_notice(LD_GENERAL, "OpenSSL %s [%lx] looks like it's older than " "0.9.8l, but some vendors have backported 0.9.8l's " "renegotiation code to earlier versions, and some have " @@ -435,10 +500,13 @@ tor_tls_init(void) use_unsafe_renegotiation_flag = 1; use_unsafe_renegotiation_op = 1; } else { + /* this is dead code, yes? */ log_info(LD_GENERAL, "OpenSSL %s has version %lx", SSLeay_version(SSLEAY_VERSION), version); } + tor_tls_allocate_tor_tls_object_ex_data_index(); + tls_library_is_initialized = 1; } } @@ -457,10 +525,6 @@ tor_tls_free_all(void) client_tls_context = NULL; tor_tls_context_decref(ctx); } - if (!HT_EMPTY(&tlsmap_root)) { - log_warn(LD_MM, "Still have entries in the tlsmap at shutdown."); - } - HT_CLEAR(tlsmap, &tlsmap_root); #ifdef V2_HANDSHAKE_CLIENT if (CLIENT_CIPHER_DUMMIES) tor_free(CLIENT_CIPHER_DUMMIES); @@ -508,13 +572,19 @@ tor_x509_name_new(const char *cname) * failure. */ static X509 * -tor_tls_create_certificate(crypto_pk_env_t *rsa, - crypto_pk_env_t *rsa_sign, +tor_tls_create_certificate(crypto_pk_t *rsa, + crypto_pk_t *rsa_sign, const char *cname, const char *cname_sign, unsigned int cert_lifetime) { + /* OpenSSL generates self-signed certificates with random 64-bit serial + * numbers, so let's do that too. */ +#define SERIAL_NUMBER_SIZE 8 + time_t start_time, end_time; + BIGNUM *serial_number = NULL; + unsigned char serial_tmp[SERIAL_NUMBER_SIZE]; EVP_PKEY *sign_pkey = NULL, *pkey=NULL; X509 *x509 = NULL; X509_NAME *name = NULL, *name_issuer=NULL; @@ -527,16 +597,23 @@ tor_tls_create_certificate(crypto_pk_env_t *rsa, tor_assert(cname); tor_assert(rsa_sign); tor_assert(cname_sign); - if (!(sign_pkey = _crypto_pk_env_get_evp_pkey(rsa_sign,1))) + if (!(sign_pkey = _crypto_pk_get_evp_pkey(rsa_sign,1))) goto error; - if (!(pkey = _crypto_pk_env_get_evp_pkey(rsa,0))) + if (!(pkey = _crypto_pk_get_evp_pkey(rsa,0))) goto error; if (!(x509 = X509_new())) goto error; if (!(X509_set_version(x509, 2))) goto error; - if (!(ASN1_INTEGER_set(X509_get_serialNumber(x509), (long)start_time))) - goto error; + + { /* our serial number is 8 random bytes. */ + if (crypto_rand((char *)serial_tmp, sizeof(serial_tmp)) < 0) + goto error; + if (!(serial_number = BN_bin2bn(serial_tmp, sizeof(serial_tmp), NULL))) + goto error; + if (!(BN_to_ASN1_INTEGER(serial_number, X509_get_serialNumber(x509)))) + goto error; + } if (!(name = tor_x509_name_new(cname))) goto error; @@ -569,11 +646,15 @@ tor_tls_create_certificate(crypto_pk_env_t *rsa, EVP_PKEY_free(sign_pkey); if (pkey) EVP_PKEY_free(pkey); + if (serial_number) + BN_free(serial_number); if (name) X509_NAME_free(name); if (name_issuer) X509_NAME_free(name_issuer); return x509; + +#undef SERIAL_NUMBER_SIZE } /** List of ciphers that servers should select from.*/ @@ -593,6 +674,9 @@ tor_tls_create_certificate(crypto_pk_env_t *rsa, * our OpenSSL doesn't know about. */ static const char CLIENT_CIPHER_LIST[] = #include "./ciphers.inc" + /* Tell it not to use SSLv2 ciphers, so that it can select an SSLv3 version + * of any cipher we say. */ + "!SSLv2" ; #undef CIPHER #undef XCIPHER @@ -620,6 +704,137 @@ static const int N_CLIENT_CIPHERS = SSL3_TXT_EDH_RSA_DES_192_CBC3_SHA) #endif +/** Free all storage held in <b>cert</b> */ +void +tor_cert_free(tor_cert_t *cert) +{ + if (! cert) + return; + if (cert->cert) + X509_free(cert->cert); + tor_free(cert->encoded); + memwipe(cert, 0x03, sizeof(*cert)); + tor_free(cert); +} + +/** + * Allocate a new tor_cert_t to hold the certificate "x509_cert". + * + * Steals a reference to x509_cert. + */ +static tor_cert_t * +tor_cert_new(X509 *x509_cert) +{ + tor_cert_t *cert; + EVP_PKEY *pkey; + RSA *rsa; + int length, length2; + unsigned char *cp; + + if (!x509_cert) + return NULL; + + length = i2d_X509(x509_cert, NULL); + cert = tor_malloc_zero(sizeof(tor_cert_t)); + if (length <= 0) { + tor_free(cert); + log_err(LD_CRYPTO, "Couldn't get length of encoded x509 certificate"); + X509_free(x509_cert); + return NULL; + } + cert->encoded_len = (size_t) length; + cp = cert->encoded = tor_malloc(length); + length2 = i2d_X509(x509_cert, &cp); + tor_assert(length2 == length); + + cert->cert = x509_cert; + + crypto_digest_all(&cert->cert_digests, + (char*)cert->encoded, cert->encoded_len); + + if ((pkey = X509_get_pubkey(x509_cert)) && + (rsa = EVP_PKEY_get1_RSA(pkey))) { + crypto_pk_t *pk = _crypto_new_pk_from_rsa(rsa); + crypto_pk_get_all_digests(pk, &cert->pkey_digests); + cert->pkey_digests_set = 1; + crypto_pk_free(pk); + EVP_PKEY_free(pkey); + } + + return cert; +} + +/** Read a DER-encoded X509 cert, of length exactly <b>certificate_len</b>, + * from a <b>certificate</b>. Return a newly allocated tor_cert_t on success + * and NULL on failure. */ +tor_cert_t * +tor_cert_decode(const uint8_t *certificate, size_t certificate_len) +{ + X509 *x509; + const unsigned char *cp = (const unsigned char *)certificate; + tor_cert_t *newcert; + tor_assert(certificate); + + if (certificate_len > INT_MAX) + return NULL; + +#if OPENSSL_VERSION_NUMBER < OPENSSL_V_SERIES(0,9,8) + /* This ifdef suppresses a type warning. Take out this case once everybody + * is using OpenSSL 0.9.8 or later. */ + x509 = d2i_X509(NULL, (unsigned char**)&cp, (int)certificate_len); +#else + x509 = d2i_X509(NULL, &cp, (int)certificate_len); +#endif + if (!x509) + return NULL; /* Couldn't decode */ + if (cp - certificate != (int)certificate_len) { + X509_free(x509); + return NULL; /* Didn't use all the bytes */ + } + newcert = tor_cert_new(x509); + if (!newcert) { + return NULL; + } + if (newcert->encoded_len != certificate_len || + fast_memneq(newcert->encoded, certificate, certificate_len)) { + /* Cert wasn't in DER */ + tor_cert_free(newcert); + return NULL; + } + return newcert; +} + +/** Set *<b>encoded_out</b> and *<b>size_out</b> to <b>cert</b>'s encoded DER + * representation and length, respectively. */ +void +tor_cert_get_der(const tor_cert_t *cert, + const uint8_t **encoded_out, size_t *size_out) +{ + tor_assert(cert); + tor_assert(encoded_out); + tor_assert(size_out); + *encoded_out = cert->encoded; + *size_out = cert->encoded_len; +} + +/** Return a set of digests for the public key in <b>cert</b>, or NULL if this + * cert's public key is not one we know how to take the digest of. */ +const digests_t * +tor_cert_get_id_digests(const tor_cert_t *cert) +{ + if (cert->pkey_digests_set) + return &cert->pkey_digests; + else + return NULL; +} + +/** Return a set of digests for the public key in <b>cert</b>. */ +const digests_t * +tor_cert_get_cert_digests(const tor_cert_t *cert) +{ + return &cert->cert_digests; +} + /** Remove a reference to <b>ctx</b>, and free it if it has no more * references. */ static void @@ -628,13 +843,172 @@ tor_tls_context_decref(tor_tls_context_t *ctx) tor_assert(ctx); if (--ctx->refcnt == 0) { SSL_CTX_free(ctx->ctx); - X509_free(ctx->my_cert); - X509_free(ctx->my_id_cert); - crypto_free_pk_env(ctx->key); + tor_cert_free(ctx->my_link_cert); + tor_cert_free(ctx->my_id_cert); + tor_cert_free(ctx->my_auth_cert); + crypto_pk_free(ctx->link_key); + crypto_pk_free(ctx->auth_key); tor_free(ctx); } } +/** Set *<b>link_cert_out</b> and *<b>id_cert_out</b> to the link certificate + * and ID certificate that we're currently using for our V3 in-protocol + * handshake's certificate chain. If <b>server</b> is true, provide the certs + * that we use in server mode; otherwise, provide the certs that we use in + * client mode. */ +int +tor_tls_get_my_certs(int server, + const tor_cert_t **link_cert_out, + const tor_cert_t **id_cert_out) +{ + tor_tls_context_t *ctx = server ? server_tls_context : client_tls_context; + if (! ctx) + return -1; + if (link_cert_out) + *link_cert_out = server ? ctx->my_link_cert : ctx->my_auth_cert; + if (id_cert_out) + *id_cert_out = ctx->my_id_cert; + return 0; +} + +/** + * Return the authentication key that we use to authenticate ourselves as a + * client in the V3 in-protocol handshake. + */ +crypto_pk_t * +tor_tls_get_my_client_auth_key(void) +{ + if (! client_tls_context) + return NULL; + return client_tls_context->auth_key; +} + +/** + * Return a newly allocated copy of the public key that a certificate + * certifies. Return NULL if the cert's key is not RSA. + */ +crypto_pk_t * +tor_tls_cert_get_key(tor_cert_t *cert) +{ + crypto_pk_t *result = NULL; + EVP_PKEY *pkey = X509_get_pubkey(cert->cert); + RSA *rsa; + if (!pkey) + return NULL; + rsa = EVP_PKEY_get1_RSA(pkey); + if (!rsa) { + EVP_PKEY_free(pkey); + return NULL; + } + result = _crypto_new_pk_from_rsa(rsa); + EVP_PKEY_free(pkey); + return result; +} + +/** Return true iff <b>a</b> and <b>b</b> represent the same public key. */ +static int +pkey_eq(EVP_PKEY *a, EVP_PKEY *b) +{ + /* We'd like to do this, but openssl 0.9.7 doesn't have it: + return EVP_PKEY_cmp(a,b) == 1; + */ + unsigned char *a_enc=NULL, *b_enc=NULL, *a_ptr, *b_ptr; + int a_len1, b_len1, a_len2, b_len2, result; + a_len1 = i2d_PublicKey(a, NULL); + b_len1 = i2d_PublicKey(b, NULL); + if (a_len1 != b_len1) + return 0; + a_ptr = a_enc = tor_malloc(a_len1); + b_ptr = b_enc = tor_malloc(b_len1); + a_len2 = i2d_PublicKey(a, &a_ptr); + b_len2 = i2d_PublicKey(b, &b_ptr); + tor_assert(a_len2 == a_len1); + tor_assert(b_len2 == b_len1); + result = tor_memeq(a_enc, b_enc, a_len1); + tor_free(a_enc); + tor_free(b_enc); + return result; +} + +/** Return true iff the other side of <b>tls</b> has authenticated to us, and + * the key certified in <b>cert</b> is the same as the key they used to do it. + */ +int +tor_tls_cert_matches_key(const tor_tls_t *tls, const tor_cert_t *cert) +{ + X509 *peercert = SSL_get_peer_certificate(tls->ssl); + EVP_PKEY *link_key = NULL, *cert_key = NULL; + int result; + + if (!peercert) + return 0; + link_key = X509_get_pubkey(peercert); + cert_key = X509_get_pubkey(cert->cert); + + result = link_key && cert_key && pkey_eq(cert_key, link_key); + + X509_free(peercert); + if (link_key) + EVP_PKEY_free(link_key); + if (cert_key) + EVP_PKEY_free(cert_key); + + return result; +} + +/** Check whether <b>cert</b> is well-formed, currently live, and correctly + * signed by the public key in <b>signing_cert</b>. If <b>check_rsa_1024</b>, + * make sure that it has an RSA key with 1024 bits; otherwise, just check that + * the key is long enough. Return 1 if the cert is good, and 0 if it's bad or + * we couldn't check it. */ +int +tor_tls_cert_is_valid(int severity, + const tor_cert_t *cert, + const tor_cert_t *signing_cert, + int check_rsa_1024) +{ + EVP_PKEY *cert_key; + EVP_PKEY *signing_key = X509_get_pubkey(signing_cert->cert); + int r, key_ok = 0; + if (!signing_key) + return 0; + r = X509_verify(cert->cert, signing_key); + EVP_PKEY_free(signing_key); + if (r <= 0) + return 0; + + /* okay, the signature checked out right. Now let's check the check the + * lifetime. */ + if (check_cert_lifetime_internal(severity, cert->cert, + 48*60*60, 30*24*60*60) < 0) + return 0; + + cert_key = X509_get_pubkey(cert->cert); + if (check_rsa_1024 && cert_key) { + RSA *rsa = EVP_PKEY_get1_RSA(cert_key); + if (rsa && BN_num_bits(rsa->n) == 1024) + key_ok = 1; + if (rsa) + RSA_free(rsa); + } else if (cert_key) { + int min_bits = 1024; +#ifdef EVP_PKEY_EC + if (EVP_PKEY_type(cert_key->type) == EVP_PKEY_EC) + min_bits = 128; +#endif + if (EVP_PKEY_bits(cert_key) >= min_bits) + key_ok = 1; + } + EVP_PKEY_free(cert_key); + if (!key_ok) + return 0; + + /* XXXX compare DNs or anything? */ + + return 1; +} + /** Increase the reference count of <b>ctx</b>. */ static void tor_tls_context_incref(tor_tls_context_t *ctx) @@ -650,8 +1024,8 @@ tor_tls_context_incref(tor_tls_context_t *ctx) * ignore <b>client_identity</b>. */ int tor_tls_context_init(int is_public_server, - crypto_pk_env_t *client_identity, - crypto_pk_env_t *server_identity, + crypto_pk_t *client_identity, + crypto_pk_t *server_identity, unsigned int key_lifetime) { int rv1 = 0; @@ -709,7 +1083,7 @@ tor_tls_context_init(int is_public_server, */ static int tor_tls_context_init_one(tor_tls_context_t **ppcontext, - crypto_pk_env_t *identity, + crypto_pk_t *identity, unsigned int key_lifetime, int is_client) { @@ -737,32 +1111,45 @@ tor_tls_context_init_one(tor_tls_context_t **ppcontext, * certificate. */ static tor_tls_context_t * -tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime, +tor_tls_context_new(crypto_pk_t *identity, unsigned int key_lifetime, int is_client) { - crypto_pk_env_t *rsa = NULL; + crypto_pk_t *rsa = NULL, *rsa_auth = NULL; EVP_PKEY *pkey = NULL; tor_tls_context_t *result = NULL; - X509 *cert = NULL, *idcert = NULL; + X509 *cert = NULL, *idcert = NULL, *authcert = NULL; char *nickname = NULL, *nn2 = NULL; tor_tls_init(); nickname = crypto_random_hostname(8, 20, "www.", ".net"); +#ifdef DISABLE_V3_LINKPROTO_SERVERSIDE nn2 = crypto_random_hostname(8, 20, "www.", ".net"); +#else + nn2 = crypto_random_hostname(8, 20, "www.", ".com"); +#endif - /* Generate short-term RSA key. */ - if (!(rsa = crypto_new_pk_env())) + /* Generate short-term RSA key for use with TLS. */ + if (!(rsa = crypto_pk_new())) goto error; if (crypto_pk_generate_key(rsa)<0) goto error; if (!is_client) { - /* Create certificate signed by identity key. */ + /* Generate short-term RSA key for use in the in-protocol ("v3") + * authentication handshake. */ + if (!(rsa_auth = crypto_pk_new())) + goto error; + if (crypto_pk_generate_key(rsa_auth)<0) + goto error; + /* Create a link certificate signed by identity key. */ cert = tor_tls_create_certificate(rsa, identity, nickname, nn2, key_lifetime); /* Create self-signed certificate for identity key. */ idcert = tor_tls_create_certificate(identity, identity, nn2, nn2, IDENTITY_CERT_LIFETIME); - if (!cert || !idcert) { + /* Create an authentication certificate signed by identity key. */ + authcert = tor_tls_create_certificate(rsa_auth, identity, nickname, nn2, + key_lifetime); + if (!cert || !idcert || !authcert) { log(LOG_WARN, LD_CRYPTO, "Error creating certificate"); goto error; } @@ -771,9 +1158,13 @@ tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime, result = tor_malloc_zero(sizeof(tor_tls_context_t)); result->refcnt = 1; if (!is_client) { - result->my_cert = X509_dup(cert); - result->my_id_cert = X509_dup(idcert); - result->key = crypto_pk_dup_key(rsa); + result->my_link_cert = tor_cert_new(X509_dup(cert)); + result->my_id_cert = tor_cert_new(X509_dup(idcert)); + result->my_auth_cert = tor_cert_new(X509_dup(authcert)); + if (!result->my_link_cert || !result->my_id_cert || !result->my_auth_cert) + goto error; + result->link_key = crypto_pk_dup_key(rsa); + result->auth_key = crypto_pk_dup_key(rsa_auth); } #if 0 @@ -817,9 +1208,9 @@ tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime, #ifdef DISABLE_SSL3_HANDSHAKE 1 || #endif - SSLeay() < 0x0090813fL || - (SSLeay() >= 0x00909000L && - SSLeay() < 0x1000006fL)) { + SSLeay() < OPENSSL_V(0,9,8,'s') || + (SSLeay() >= OPENSSL_V_SERIES(0,9,9) && + SSLeay() < OPENSSL_V(1,0,0,'f'))) { /* And not SSL3 if it's subject to CVE-2011-4576. */ log_info(LD_NET, "Disabling SSLv3 because this OpenSSL version " "might otherwise be vulnerable to CVE-2011-4576 " @@ -866,7 +1257,7 @@ tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime, SSL_CTX_set_session_cache_mode(result->ctx, SSL_SESS_CACHE_OFF); if (!is_client) { tor_assert(rsa); - if (!(pkey = _crypto_pk_env_get_evp_pkey(rsa,1))) + if (!(pkey = _crypto_pk_get_evp_pkey(rsa,1))) goto error; if (!SSL_CTX_use_PrivateKey(result->ctx, pkey)) goto error; @@ -876,9 +1267,9 @@ tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime, goto error; } { - crypto_dh_env_t *dh = crypto_dh_new(DH_TYPE_TLS); + crypto_dh_t *dh = crypto_dh_new(DH_TYPE_TLS); tor_assert(dh); - SSL_CTX_set_tmp_dh(result->ctx, _crypto_dh_env_get_dh(dh)); + SSL_CTX_set_tmp_dh(result->ctx, _crypto_dh_get_dh(dh)); crypto_dh_free(dh); } SSL_CTX_set_verify(result->ctx, SSL_VERIFY_PEER, @@ -887,7 +1278,10 @@ tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime, SSL_CTX_set_mode(result->ctx, SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER); if (rsa) - crypto_free_pk_env(rsa); + crypto_pk_free(rsa); + if (rsa_auth) + crypto_pk_free(rsa_auth); + X509_free(authcert); tor_free(nickname); tor_free(nn2); return result; @@ -899,13 +1293,17 @@ tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime, if (pkey) EVP_PKEY_free(pkey); if (rsa) - crypto_free_pk_env(rsa); + crypto_pk_free(rsa); + if (rsa_auth) + crypto_pk_free(rsa_auth); if (result) tor_tls_context_decref(result); if (cert) X509_free(cert); if (idcert) X509_free(idcert); + if (authcert) + X509_free(authcert); return NULL; } @@ -945,7 +1343,7 @@ tor_tls_client_is_using_v2_ciphers(const SSL *ssl, const char *address) return 0; dump_list: { - smartlist_t *elts = smartlist_create(); + smartlist_t *elts = smartlist_new(); char *s; for (i = 0; i < sk_SSL_CIPHER_num(session->ciphers); ++i) { SSL_CIPHER *cipher = sk_SSL_CIPHER_value(session->ciphers, i); @@ -961,6 +1359,14 @@ tor_tls_client_is_using_v2_ciphers(const SSL *ssl, const char *address) return 1; } +/** Invoked when a TLS state changes: log the change at severity 'debug' */ +static void +tor_tls_debug_state_callback(const SSL *ssl, int type, int val) +{ + log_debug(LD_HANDSHAKE, "SSL %p is now in state %s [type=%d,val=%d].", + ssl, SSL_state_string_long(ssl), type, val); +} + /** Invoked when we're accepting a connection on <b>ssl</b>, and the connection * changes state. We use this: * <ul><li>To alter the state of the handshake partway through, so we @@ -972,9 +1378,13 @@ tor_tls_server_info_callback(const SSL *ssl, int type, int val) { tor_tls_t *tls; (void) val; + + tor_tls_debug_state_callback(ssl, type, val); + if (type != SSL_CB_ACCEPT_LOOP) return; - if (ssl->state != SSL3_ST_SW_SRVR_HELLO_A) + if ((ssl->state != SSL3_ST_SW_SRVR_HELLO_A) && + (ssl->state != SSL3_ST_SW_SRVR_HELLO_B)) return; tls = tor_tls_get_by_ssl(ssl); @@ -982,13 +1392,18 @@ tor_tls_server_info_callback(const SSL *ssl, int type, int val) /* Check whether we're watching for renegotiates. If so, this is one! */ if (tls->negotiated_callback) tls->got_renegotiate = 1; + if (tls->server_handshake_count < 127) /*avoid any overflow possibility*/ + ++tls->server_handshake_count; } else { log_warn(LD_BUG, "Couldn't look up the tls for an SSL*. How odd!"); + return; } /* Now check the cipher list. */ if (tor_tls_client_is_using_v2_ciphers(ssl, ADDR(tls))) { - /*XXXX_TLS keep this from happening more than once! */ + if (tls->wasV2Handshake) + return; /* We already turned this stuff off for the first handshake; + * This is a renegotiation. */ /* Yes, we're casting away the const from ssl. This is very naughty of us. * Let's hope openssl doesn't notice! */ @@ -1000,6 +1415,10 @@ tor_tls_server_info_callback(const SSL *ssl, int type, int val) if (tls) { tls->wasV2Handshake = 1; +#ifdef USE_BUFFEREVENTS + if (use_unsafe_renegotiation_flag) + tls->ssl->s3->flags |= SSL3_FLAGS_ALLOW_UNSAFE_LEGACY_RENEGOTIATION; +#endif } else { log_warn(LD_BUG, "Couldn't look up the tls for an SSL*. How odd!"); } @@ -1007,11 +1426,35 @@ tor_tls_server_info_callback(const SSL *ssl, int type, int val) } #endif +/** Explain which ciphers we're missing. */ +static void +log_unsupported_ciphers(smartlist_t *unsupported) +{ + char *joined; + + log_notice(LD_NET, "We weren't able to find support for all of the " + "TLS ciphersuites that we wanted to advertise. This won't " + "hurt security, but it might make your Tor (if run as a client) " + "more easy for censors to block."); + + if (SSLeay() < 0x10000000L) { + log_notice(LD_NET, "To correct this, use a more recent OpenSSL, " + "built without disabling any secure ciphers or features."); + } else { + log_notice(LD_NET, "To correct this, use a version of OpenSSL " + "built with none of its ciphers disabled."); + } + + joined = smartlist_join_strings(unsupported, ":", 0, NULL); + log_info(LD_NET, "The unsupported ciphers were: %s", joined); + tor_free(joined); +} + /** Replace *<b>ciphers</b> with a new list of SSL ciphersuites: specifically, - * a list designed to mimic a common web browser. Some of the ciphers in the - * list won't actually be implemented by OpenSSL: that's okay so long as the - * server doesn't select them, and the server won't select anything besides - * what's in SERVER_CIPHER_LIST. + * a list designed to mimic a common web browser. We might not be able to do + * that if OpenSSL doesn't support all the ciphers we want. Some of the + * ciphers in the list won't actually be implemented by OpenSSL: that's okay + * so long as the server doesn't select them. * * [If the server <b>does</b> select a bogus cipher, we won't crash or * anything; we'll just fail later when we try to look up the cipher in @@ -1023,14 +1466,17 @@ rectify_client_ciphers(STACK_OF(SSL_CIPHER) **ciphers) #ifdef V2_HANDSHAKE_CLIENT if (PREDICT_UNLIKELY(!CLIENT_CIPHER_STACK)) { /* We need to set CLIENT_CIPHER_STACK to an array of the ciphers - * we want.*/ + * we want to use/advertise. */ int i = 0, j = 0; + smartlist_t *unsupported = smartlist_new(); /* First, create a dummy SSL_CIPHER for every cipher. */ CLIENT_CIPHER_DUMMIES = tor_malloc_zero(sizeof(SSL_CIPHER)*N_CLIENT_CIPHERS); for (i=0; i < N_CLIENT_CIPHERS; ++i) { CLIENT_CIPHER_DUMMIES[i].valid = 1; + /* The "3<<24" here signifies that the cipher is supposed to work with + * SSL3 and TLS1. */ CLIENT_CIPHER_DUMMIES[i].id = CLIENT_CIPHER_INFO_LIST[i].id | (3<<24); CLIENT_CIPHER_DUMMIES[i].name = CLIENT_CIPHER_INFO_LIST[i].name; } @@ -1045,27 +1491,49 @@ rectify_client_ciphers(STACK_OF(SSL_CIPHER) **ciphers) } /* Then copy as many ciphers as we can from the good list, inserting - * dummies as needed. */ - j=0; - for (i = 0; i < N_CLIENT_CIPHERS; ) { + * dummies as needed. Let j be an index into list of ciphers we have + * (*ciphers) and let i be an index into the ciphers we want + * (CLIENT_INFO_CIPHER_LIST). We are building a list of ciphers in + * CLIENT_CIPHER_STACK. + */ + for (i = j = 0; i < N_CLIENT_CIPHERS; ) { SSL_CIPHER *cipher = NULL; if (j < sk_SSL_CIPHER_num(*ciphers)) cipher = sk_SSL_CIPHER_value(*ciphers, j); if (cipher && ((cipher->id >> 24) & 0xff) != 3) { - log_debug(LD_NET, "Skipping v2 cipher %s", cipher->name); + /* Skip over non-v3 ciphers entirely. (This should no longer be + * needed, thanks to saying !SSLv2 above.) */ + log_debug(LD_NET, "Skipping v%d cipher %s", + (int)((cipher->id>>24) & 0xff), + cipher->name); ++j; } else if (cipher && (cipher->id & 0xffff) == CLIENT_CIPHER_INFO_LIST[i].id) { + /* "cipher" is the cipher we expect. Put it on the list. */ log_debug(LD_NET, "Found cipher %s", cipher->name); sk_SSL_CIPHER_push(CLIENT_CIPHER_STACK, cipher); ++j; ++i; - } else { + } else if (!strcmp(CLIENT_CIPHER_DUMMIES[i].name, + "SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA")) { + /* We found bogus cipher 0xfeff, which OpenSSL doesn't support and + * never has. For this one, we need a dummy. */ log_debug(LD_NET, "Inserting fake %s", CLIENT_CIPHER_DUMMIES[i].name); sk_SSL_CIPHER_push(CLIENT_CIPHER_STACK, &CLIENT_CIPHER_DUMMIES[i]); ++i; + } else { + /* OpenSSL doesn't have this one. */ + log_debug(LD_NET, "Completely omitting unsupported cipher %s", + CLIENT_CIPHER_INFO_LIST[i].name); + smartlist_add(unsupported, (char*) CLIENT_CIPHER_INFO_LIST[i].name); + ++i; } } + + if (smartlist_len(unsupported)) + log_unsupported_ciphers(unsupported); + + smartlist_free(unsupported); } sk_SSL_CIPHER_free(*ciphers); @@ -1087,6 +1555,7 @@ tor_tls_new(int sock, int isServer) tor_tls_t *result = tor_malloc_zero(sizeof(tor_tls_t)); tor_tls_context_t *context = isServer ? server_tls_context : client_tls_context; + result->magic = TOR_TLS_MAGIC; tor_assert(context); /* make sure somebody made it first */ if (!(result->ssl = SSL_new(context->ctx))) { @@ -1127,7 +1596,14 @@ tor_tls_new(int sock, int isServer) tor_free(result); return NULL; } - HT_INSERT(tlsmap, &tlsmap_root, result); + { + int set_worked = + SSL_set_ex_data(result->ssl, tor_tls_object_ex_data_index, result); + if (!set_worked) { + log_warn(LD_BUG, + "Couldn't set the tls for an SSL*; connection will fail"); + } + } SSL_set_bio(result->ssl, bio, bio); tor_tls_context_incref(context); result->context = context; @@ -1143,8 +1619,11 @@ tor_tls_new(int sock, int isServer) #ifdef V2_HANDSHAKE_SERVER if (isServer) { SSL_set_info_callback(result->ssl, tor_tls_server_info_callback); - } + } else #endif + { + SSL_set_info_callback(result->ssl, tor_tls_debug_state_callback); + } /* Not expected to get called. */ tls_log_errors(NULL, LOG_WARN, LD_NET, "creating tor_tls_t object"); @@ -1178,7 +1657,7 @@ tor_tls_set_renegotiate_callback(tor_tls_t *tls, if (cb) { SSL_set_info_callback(tls->ssl, tor_tls_server_info_callback); } else { - SSL_set_info_callback(tls->ssl, NULL); + SSL_set_info_callback(tls->ssl, tor_tls_debug_state_callback); } #endif } @@ -1186,7 +1665,7 @@ tor_tls_set_renegotiate_callback(tor_tls_t *tls, /** If this version of openssl requires it, turn on renegotiation on * <b>tls</b>. */ -static void +void tor_tls_unblock_renegotiation(tor_tls_t *tls) { /* Yes, we know what we are doing here. No, we do not treat a renegotiation @@ -1210,6 +1689,20 @@ tor_tls_block_renegotiation(tor_tls_t *tls) tls->ssl->s3->flags &= ~SSL3_FLAGS_ALLOW_UNSAFE_LEGACY_RENEGOTIATION; } +/** Assert that the flags that allow legacy renegotiation are still set */ +void +tor_tls_assert_renegotiation_unblocked(tor_tls_t *tls) +{ + if (use_unsafe_renegotiation_flag) { + tor_assert(0 != (tls->ssl->s3->flags & + SSL3_FLAGS_ALLOW_UNSAFE_LEGACY_RENEGOTIATION)); + } + if (use_unsafe_renegotiation_op) { + long options = SSL_get_options(tls->ssl); + tor_assert(0 != (options & SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION)); + } +} + /** Return whether this tls initiated the connect (client) or * received it (server). */ int @@ -1225,14 +1718,9 @@ tor_tls_is_server(tor_tls_t *tls) void tor_tls_free(tor_tls_t *tls) { - tor_tls_t *removed; if (!tls) return; tor_assert(tls->ssl); - removed = HT_REMOVE(tlsmap, &tlsmap_root, tls); - if (!removed) { - log_warn(LD_BUG, "Freeing a TLS that was not in the ssl->tls map."); - } #ifdef SSL_set_tlsext_host_name SSL_set_tlsext_host_name(tls->ssl, NULL); #endif @@ -1242,6 +1730,7 @@ tor_tls_free(tor_tls_t *tls) if (tls->context) tor_tls_context_decref(tls->context); tor_free(tls->address); + tls->magic = 0x99999999; tor_free(tls); } @@ -1333,16 +1822,16 @@ tor_tls_handshake(tor_tls_t *tls) oldstate = tls->ssl->state; if (tls->isServer) { log_debug(LD_HANDSHAKE, "About to call SSL_accept on %p (%s)", tls, - ssl_state_to_string(tls->ssl->state)); + SSL_state_string_long(tls->ssl)); r = SSL_accept(tls->ssl); } else { log_debug(LD_HANDSHAKE, "About to call SSL_connect on %p (%s)", tls, - ssl_state_to_string(tls->ssl->state)); + SSL_state_string_long(tls->ssl)); r = SSL_connect(tls->ssl); } if (oldstate != tls->ssl->state) log_debug(LD_HANDSHAKE, "After call, %p was in state %s", - tls, ssl_state_to_string(tls->ssl->state)); + tls, SSL_state_string_long(tls->ssl)); /* We need to call this here and not earlier, since OpenSSL has a penchant * for clearing its flags when you say accept or connect. */ tor_tls_unblock_renegotiation(tls); @@ -1354,56 +1843,86 @@ tor_tls_handshake(tor_tls_t *tls) } if (r == TOR_TLS_DONE) { tls->state = TOR_TLS_ST_OPEN; - if (tls->isServer) { - SSL_set_info_callback(tls->ssl, NULL); - SSL_set_verify(tls->ssl, SSL_VERIFY_PEER, always_accept_verify_cb); - /* There doesn't seem to be a clear OpenSSL API to clear mode flags. */ - tls->ssl->mode &= ~SSL_MODE_NO_AUTO_CHAIN; + return tor_tls_finish_handshake(tls); + } + return r; +} + +/** Perform the final part of the intial TLS handshake on <b>tls</b>. This + * should be called for the first handshake only: it determines whether the v1 + * or the v2 handshake was used, and adjusts things for the renegotiation + * handshake as appropriate. + * + * tor_tls_handshake() calls this on its own; you only need to call this if + * bufferevent is doing the handshake for you. + */ +int +tor_tls_finish_handshake(tor_tls_t *tls) +{ + int r = TOR_TLS_DONE; + if (tls->isServer) { + SSL_set_info_callback(tls->ssl, NULL); + SSL_set_verify(tls->ssl, SSL_VERIFY_PEER, always_accept_verify_cb); + /* There doesn't seem to be a clear OpenSSL API to clear mode flags. */ + tls->ssl->mode &= ~SSL_MODE_NO_AUTO_CHAIN; #ifdef V2_HANDSHAKE_SERVER - if (tor_tls_client_is_using_v2_ciphers(tls->ssl, ADDR(tls))) { - /* This check is redundant, but back when we did it in the callback, - * we might have not been able to look up the tor_tls_t if the code - * was buggy. Fixing that. */ - if (!tls->wasV2Handshake) { - log_warn(LD_BUG, "For some reason, wasV2Handshake didn't" - " get set. Fixing that."); - } - tls->wasV2Handshake = 1; - log_debug(LD_HANDSHAKE, - "Completed V2 TLS handshake with client; waiting " - "for renegotiation."); - } else { - tls->wasV2Handshake = 0; + if (tor_tls_client_is_using_v2_ciphers(tls->ssl, ADDR(tls))) { + /* This check is redundant, but back when we did it in the callback, + * we might have not been able to look up the tor_tls_t if the code + * was buggy. Fixing that. */ + if (!tls->wasV2Handshake) { + log_warn(LD_BUG, "For some reason, wasV2Handshake didn't" + " get set. Fixing that."); } -#endif + tls->wasV2Handshake = 1; + log_debug(LD_HANDSHAKE, "Completed V2 TLS handshake with client; waiting" + " for renegotiation."); } else { + tls->wasV2Handshake = 0; + } +#endif + } else { #ifdef V2_HANDSHAKE_CLIENT - /* If we got no ID cert, we're a v2 handshake. */ - X509 *cert = SSL_get_peer_certificate(tls->ssl); - STACK_OF(X509) *chain = SSL_get_peer_cert_chain(tls->ssl); - int n_certs = sk_X509_num(chain); - if (n_certs > 1 || (n_certs == 1 && cert != sk_X509_value(chain, 0))) { - log_debug(LD_HANDSHAKE, "Server sent back multiple certificates; it " - "looks like a v1 handshake on %p", tls); - tls->wasV2Handshake = 0; - } else { - log_debug(LD_HANDSHAKE, - "Server sent back a single certificate; looks like " - "a v2 handshake on %p.", tls); - tls->wasV2Handshake = 1; - } - if (cert) - X509_free(cert); + /* If we got no ID cert, we're a v2 handshake. */ + X509 *cert = SSL_get_peer_certificate(tls->ssl); + STACK_OF(X509) *chain = SSL_get_peer_cert_chain(tls->ssl); + int n_certs = sk_X509_num(chain); + if (n_certs > 1 || (n_certs == 1 && cert != sk_X509_value(chain, 0))) { + log_debug(LD_HANDSHAKE, "Server sent back multiple certificates; it " + "looks like a v1 handshake on %p", tls); + tls->wasV2Handshake = 0; + } else { + log_debug(LD_HANDSHAKE, + "Server sent back a single certificate; looks like " + "a v2 handshake on %p.", tls); + tls->wasV2Handshake = 1; + } + if (cert) + X509_free(cert); #endif - if (SSL_set_cipher_list(tls->ssl, SERVER_CIPHER_LIST) == 0) { - tls_log_errors(NULL, LOG_WARN, LD_HANDSHAKE, "re-setting ciphers"); - r = TOR_TLS_ERROR_MISC; - } + if (SSL_set_cipher_list(tls->ssl, SERVER_CIPHER_LIST) == 0) { + tls_log_errors(NULL, LOG_WARN, LD_HANDSHAKE, "re-setting ciphers"); + r = TOR_TLS_ERROR_MISC; } } return r; } +#ifdef USE_BUFFEREVENTS +/** Put <b>tls</b>, which must be a client connection, into renegotiation + * mode. */ +int +tor_tls_start_renegotiating(tor_tls_t *tls) +{ + int r = SSL_renegotiate(tls->ssl); + if (r <= 0) { + return tor_tls_get_error(tls, r, 0, "renegotiating", LOG_WARN, + LD_HANDSHAKE); + } + return 0; +} +#endif + /** Client only: Renegotiate a TLS session. When finished, returns * TOR_TLS_DONE. On failure, returns TOR_TLS_ERROR, TOR_TLS_WANTREAD, or * TOR_TLS_WANTWRITE. @@ -1509,9 +2028,21 @@ tor_tls_peer_has_cert(tor_tls_t *tls) return 1; } +/** Return the peer certificate, or NULL if there isn't one. */ +tor_cert_t * +tor_tls_get_peer_cert(tor_tls_t *tls) +{ + X509 *cert; + cert = SSL_get_peer_certificate(tls->ssl); + tls_log_errors(tls, LOG_WARN, LD_HANDSHAKE, "getting peer certificate"); + if (!cert) + return NULL; + return tor_cert_new(cert); +} + /** Warn that a certificate lifetime extends through a certain range. */ static void -log_cert_lifetime(X509 *cert, const char *problem) +log_cert_lifetime(int severity, const X509 *cert, const char *problem) { BIO *bio = NULL; BUF_MEM *buf; @@ -1521,9 +2052,10 @@ log_cert_lifetime(X509 *cert, const char *problem) struct tm tm; if (problem) - log_warn(LD_GENERAL, - "Certificate %s: is your system clock set incorrectly?", - problem); + log(severity, LD_GENERAL, + "Certificate %s. Either their clock is set wrong, or your clock " + "is wrong.", + problem); if (!(bio = BIO_new(BIO_s_mem()))) { log_warn(LD_GENERAL, "Couldn't allocate BIO!"); goto end; @@ -1545,9 +2077,9 @@ log_cert_lifetime(X509 *cert, const char *problem) strftime(mytime, 32, "%b %d %H:%M:%S %Y GMT", tor_gmtime_r(&now, &tm)); - log_warn(LD_GENERAL, - "(certificate lifetime runs from %s through %s. Your time is %s.)", - s1,s2,mytime); + log(severity, LD_GENERAL, + "(certificate lifetime runs from %s through %s. Your time is %s.)", + s1,s2,mytime); end: /* Not expected to get invoked */ @@ -1604,7 +2136,7 @@ try_to_extract_certs_from_tls(int severity, tor_tls_t *tls, * 0. Else, return -1 and log complaints with log-level <b>severity</b>. */ int -tor_tls_verify(int severity, tor_tls_t *tls, crypto_pk_env_t **identity_key) +tor_tls_verify(int severity, tor_tls_t *tls, crypto_pk_t **identity_key) { X509 *cert = NULL, *id_cert = NULL; EVP_PKEY *id_pkey = NULL; @@ -1620,6 +2152,8 @@ tor_tls_verify(int severity, tor_tls_t *tls, crypto_pk_env_t **identity_key) log_fn(severity,LD_PROTOCOL,"No distinct identity certificate found"); goto done; } + tls_log_errors(tls, severity, LD_HANDSHAKE, "before verifying certificate"); + if (!(id_pkey = X509_get_pubkey(id_cert)) || X509_verify(cert, id_pkey) <= 0) { log_fn(severity,LD_PROTOCOL,"X509_verify on cert and pkey returned <= 0"); @@ -1630,7 +2164,7 @@ tor_tls_verify(int severity, tor_tls_t *tls, crypto_pk_env_t **identity_key) rsa = EVP_PKEY_get1_RSA(id_pkey); if (!rsa) goto done; - *identity_key = _crypto_new_pk_env_rsa(rsa); + *identity_key = _crypto_new_pk_from_rsa(rsa); r = 0; @@ -1647,34 +2181,25 @@ tor_tls_verify(int severity, tor_tls_t *tls, crypto_pk_env_t **identity_key) return r; } -/** Check whether the certificate set on the connection <b>tls</b> is - * expired or not-yet-valid, give or take <b>tolerance</b> - * seconds. Return 0 for valid, -1 for failure. +/** Check whether the certificate set on the connection <b>tls</b> is expired + * give or take <b>past_tolerance</b> seconds, or not-yet-valid give or take + * <b>future_tolerance</b> seconds. Return 0 for valid, -1 for failure. * * NOTE: you should call tor_tls_verify before tor_tls_check_lifetime. */ int -tor_tls_check_lifetime(tor_tls_t *tls, int tolerance) +tor_tls_check_lifetime(int severity, tor_tls_t *tls, + int past_tolerance, int future_tolerance) { - time_t now, t; X509 *cert; int r = -1; - now = time(NULL); - if (!(cert = SSL_get_peer_certificate(tls->ssl))) goto done; - t = now + tolerance; - if (X509_cmp_time(X509_get_notBefore(cert), &t) > 0) { - log_cert_lifetime(cert, "not yet valid"); + if (check_cert_lifetime_internal(severity, cert, + past_tolerance, future_tolerance) < 0) goto done; - } - t = now - tolerance; - if (X509_cmp_time(X509_get_notAfter(cert), &t) < 0) { - log_cert_lifetime(cert, "already expired"); - goto done; - } r = 0; done: @@ -1686,6 +2211,32 @@ tor_tls_check_lifetime(tor_tls_t *tls, int tolerance) return r; } +/** Helper: check whether <b>cert</b> is expired give or take + * <b>past_tolerance</b> seconds, or not-yet-valid give or take + * <b>future_tolerance</b> seconds. If it is live, return 0. If it is not + * live, log a message and return -1. */ +static int +check_cert_lifetime_internal(int severity, const X509 *cert, + int past_tolerance, int future_tolerance) +{ + time_t now, t; + + now = time(NULL); + + t = now + future_tolerance; + if (X509_cmp_time(X509_get_notBefore(cert), &t) > 0) { + log_cert_lifetime(severity, cert, "not yet valid"); + return -1; + } + t = now - past_tolerance; + if (X509_cmp_time(X509_get_notAfter(cert), &t) < 0) { + log_cert_lifetime(severity, cert, "already expired"); + return -1; + } + + return 0; +} + /** Return the number of bytes available for reading from <b>tls</b>. */ int @@ -1769,6 +2320,138 @@ tor_tls_used_v1_handshake(tor_tls_t *tls) return 1; } +/** Return true iff <b>name</b> is a DN of a kind that could only + * occur in a v3-handshake-indicating certificate */ +static int +dn_indicates_v3_cert(X509_NAME *name) +{ +#ifdef DISABLE_V3_LINKPROTO_CLIENTSIDE + (void)name; + return 0; +#else + X509_NAME_ENTRY *entry; + int n_entries; + ASN1_OBJECT *obj; + ASN1_STRING *str; + unsigned char *s; + int len, r; + + n_entries = X509_NAME_entry_count(name); + if (n_entries != 1) + return 1; /* More than one entry in the DN. */ + entry = X509_NAME_get_entry(name, 0); + + obj = X509_NAME_ENTRY_get_object(entry); + if (OBJ_obj2nid(obj) != OBJ_txt2nid("commonName")) + return 1; /* The entry isn't a commonName. */ + + str = X509_NAME_ENTRY_get_data(entry); + len = ASN1_STRING_to_UTF8(&s, str); + if (len < 0) + return 0; + r = fast_memneq(s + len - 4, ".net", 4); + OPENSSL_free(s); + return r; +#endif +} + +/** Return true iff the peer certificate we're received on <b>tls</b> + * indicates that this connection should use the v3 (in-protocol) + * authentication handshake. + * + * Only the connection initiator should use this, and only once the initial + * handshake is done; the responder detects a v1 handshake by cipher types, + * and a v3/v2 handshake by Versions cell vs renegotiation. + */ +int +tor_tls_received_v3_certificate(tor_tls_t *tls) +{ + X509 *cert = SSL_get_peer_certificate(tls->ssl); + EVP_PKEY *key = NULL; + X509_NAME *issuer_name, *subject_name; + int is_v3 = 0; + + if (!cert) { + log_warn(LD_BUG, "Called on a connection with no peer certificate"); + goto done; + } + + subject_name = X509_get_subject_name(cert); + issuer_name = X509_get_issuer_name(cert); + + if (X509_name_cmp(subject_name, issuer_name) == 0) { + is_v3 = 1; /* purportedly self signed */ + goto done; + } + + if (dn_indicates_v3_cert(subject_name) || + dn_indicates_v3_cert(issuer_name)) { + is_v3 = 1; /* DN is fancy */ + goto done; + } + + key = X509_get_pubkey(cert); + if (EVP_PKEY_bits(key) != 1024 || + EVP_PKEY_type(key->type) != EVP_PKEY_RSA) { + is_v3 = 1; /* Key is fancy */ + goto done; + } + + done: + if (key) + EVP_PKEY_free(key); + if (cert) + X509_free(cert); + + return is_v3; +} + +/** Return the number of server handshakes that we've noticed doing on + * <b>tls</b>. */ +int +tor_tls_get_num_server_handshakes(tor_tls_t *tls) +{ + return tls->server_handshake_count; +} + +/** Return true iff the server TLS connection <b>tls</b> got the renegotiation + * request it was waiting for. */ +int +tor_tls_server_got_renegotiate(tor_tls_t *tls) +{ + return tls->got_renegotiate; +} + +/** Set the DIGEST256_LEN buffer at <b>secrets_out</b> to the value used in + * the v3 handshake to prove that the client knows the TLS secrets for the + * connection <b>tls</b>. Return 0 on success, -1 on failure. + */ +int +tor_tls_get_tlssecrets(tor_tls_t *tls, uint8_t *secrets_out) +{ +#define TLSSECRET_MAGIC "Tor V3 handshake TLS cross-certification" + char buf[128]; + size_t len; + tor_assert(tls); + tor_assert(tls->ssl); + tor_assert(tls->ssl->s3); + tor_assert(tls->ssl->session); + /* + The value is an HMAC, using the TLS master key as the HMAC key, of + client_random | server_random | TLSSECRET_MAGIC + */ + memcpy(buf + 0, tls->ssl->s3->client_random, 32); + memcpy(buf + 32, tls->ssl->s3->server_random, 32); + memcpy(buf + 64, TLSSECRET_MAGIC, strlen(TLSSECRET_MAGIC) + 1); + len = 64 + strlen(TLSSECRET_MAGIC) + 1; + crypto_hmac_sha256((char*)secrets_out, + (char*)tls->ssl->session->master_key, + tls->ssl->session->master_key_length, + buf, len); + memwipe(buf, 0, sizeof(buf)); + return 0; +} + /** Examine the amount of memory used and available for buffers in <b>tls</b>. * Set *<b>rbuf_capacity</b> to the amount of storage allocated for the read * buffer and *<b>rbuf_bytes</b> to the amount actually used. @@ -1791,3 +2474,75 @@ tor_tls_get_buffer_sizes(tor_tls_t *tls, *wbuf_bytes = tls->ssl->s3->wbuf.left; } +#ifdef USE_BUFFEREVENTS +/** Construct and return an TLS-encrypting bufferevent to send data over + * <b>socket</b>, which must match the socket of the underlying bufferevent + * <b>bufev_in</b>. The TLS object <b>tls</b> is used for encryption. + * + * This function will either create a filtering bufferevent that wraps around + * <b>bufev_in</b>, or it will free bufev_in and return a new bufferevent that + * uses the <b>tls</b> to talk to the network directly. Do not use + * <b>bufev_in</b> after calling this function. + * + * The connection will start out doing a server handshake if <b>receiving</b> + * is strue, and a client handshake otherwise. + * + * Returns NULL on failure. + */ +struct bufferevent * +tor_tls_init_bufferevent(tor_tls_t *tls, struct bufferevent *bufev_in, + evutil_socket_t socket, int receiving, + int filter) +{ + struct bufferevent *out; + const enum bufferevent_ssl_state state = receiving ? + BUFFEREVENT_SSL_ACCEPTING : BUFFEREVENT_SSL_CONNECTING; + + if (filter || tor_libevent_using_iocp_bufferevents()) { + /* Grab an extra reference to the SSL, since BEV_OPT_CLOSE_ON_FREE + means that the SSL will get freed too. + + This increment makes our SSL usage not-threadsafe, BTW. We should + see if we're allowed to use CRYPTO_add from outside openssl. */ + tls->ssl->references += 1; + out = bufferevent_openssl_filter_new(tor_libevent_get_base(), + bufev_in, + tls->ssl, + state, + BEV_OPT_DEFER_CALLBACKS| + BEV_OPT_CLOSE_ON_FREE); + /* Tell the underlying bufferevent when to accept more data from the SSL + filter (only when it's got less than 32K to write), and when to notify + the SSL filter that it could write more (when it drops under 24K). */ + bufferevent_setwatermark(bufev_in, EV_WRITE, 24*1024, 32*1024); + } else { + if (bufev_in) { + evutil_socket_t s = bufferevent_getfd(bufev_in); + tor_assert(s == -1 || s == socket); + tor_assert(evbuffer_get_length(bufferevent_get_input(bufev_in)) == 0); + tor_assert(evbuffer_get_length(bufferevent_get_output(bufev_in)) == 0); + tor_assert(BIO_number_read(SSL_get_rbio(tls->ssl)) == 0); + tor_assert(BIO_number_written(SSL_get_rbio(tls->ssl)) == 0); + bufferevent_free(bufev_in); + } + + /* Current versions (as of 2.0.x) of Libevent need to defer + * bufferevent_openssl callbacks, or else our callback functions will + * get called reentrantly, which is bad for us. + */ + out = bufferevent_openssl_socket_new(tor_libevent_get_base(), + socket, + tls->ssl, + state, + BEV_OPT_DEFER_CALLBACKS); + } + tls->state = TOR_TLS_ST_BUFFEREVENT; + + /* Unblock _after_ creating the bufferevent, since accept/connect tend to + * clear flags. */ + tor_tls_unblock_renegotiation(tls); + + return out; +} +#endif + diff --git a/src/common/tortls.h b/src/common/tortls.h index 55fee81aea..491a5419df 100644 --- a/src/common/tortls.h +++ b/src/common/tortls.h @@ -1,6 +1,6 @@ /* Copyright (c) 2003, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #ifndef _TOR_TORTLS_H @@ -17,6 +17,9 @@ /* Opaque structure to hold a TLS connection. */ typedef struct tor_tls_t tor_tls_t; +/* Opaque structure to hold an X509 certificate. */ +typedef struct tor_cert_t tor_cert_t; + /* Possible return values for most tor_tls_* functions. */ #define _MIN_TOR_TLS_ERROR_VAL -9 #define TOR_TLS_ERROR_MISC -9 @@ -48,11 +51,12 @@ typedef struct tor_tls_t tor_tls_t; #define TOR_TLS_IS_ERROR(rv) ((rv) < TOR_TLS_CLOSE) const char *tor_tls_err_to_string(int err); +void tor_tls_get_state_description(tor_tls_t *tls, char *buf, size_t sz); void tor_tls_free_all(void); int tor_tls_context_init(int is_public_server, - crypto_pk_env_t *client_identity, - crypto_pk_env_t *server_identity, + crypto_pk_t *client_identity, + crypto_pk_t *server_identity, unsigned int key_lifetime); tor_tls_t *tor_tls_new(int sock, int is_server); void tor_tls_set_logged_address(tor_tls_t *tls, const char *address); @@ -62,13 +66,19 @@ void tor_tls_set_renegotiate_callback(tor_tls_t *tls, int tor_tls_is_server(tor_tls_t *tls); void tor_tls_free(tor_tls_t *tls); int tor_tls_peer_has_cert(tor_tls_t *tls); -int tor_tls_verify(int severity, tor_tls_t *tls, crypto_pk_env_t **identity); -int tor_tls_check_lifetime(tor_tls_t *tls, int tolerance); +tor_cert_t *tor_tls_get_peer_cert(tor_tls_t *tls); +int tor_tls_verify(int severity, tor_tls_t *tls, crypto_pk_t **identity); +int tor_tls_check_lifetime(int severity, + tor_tls_t *tls, int past_tolerance, + int future_tolerance); int tor_tls_read(tor_tls_t *tls, char *cp, size_t len); int tor_tls_write(tor_tls_t *tls, const char *cp, size_t n); int tor_tls_handshake(tor_tls_t *tls); +int tor_tls_finish_handshake(tor_tls_t *tls); int tor_tls_renegotiate(tor_tls_t *tls); +void tor_tls_unblock_renegotiation(tor_tls_t *tls); void tor_tls_block_renegotiation(tor_tls_t *tls); +void tor_tls_assert_renegotiation_unblocked(tor_tls_t *tls); int tor_tls_shutdown(tor_tls_t *tls); int tor_tls_get_pending_bytes(tor_tls_t *tls); size_t tor_tls_get_forced_write_size(tor_tls_t *tls); @@ -81,12 +91,44 @@ void tor_tls_get_buffer_sizes(tor_tls_t *tls, size_t *wbuf_capacity, size_t *wbuf_bytes); int tor_tls_used_v1_handshake(tor_tls_t *tls); +int tor_tls_received_v3_certificate(tor_tls_t *tls); +int tor_tls_get_num_server_handshakes(tor_tls_t *tls); +int tor_tls_server_got_renegotiate(tor_tls_t *tls); +int tor_tls_get_tlssecrets(tor_tls_t *tls, uint8_t *secrets_out); /* Log and abort if there are unhandled TLS errors in OpenSSL's error stack. */ #define check_no_tls_errors() _check_no_tls_errors(__FILE__,__LINE__) void _check_no_tls_errors(const char *fname, int line); +void tor_tls_log_one_error(tor_tls_t *tls, unsigned long err, + int severity, int domain, const char *doing); + +#ifdef USE_BUFFEREVENTS +int tor_tls_start_renegotiating(tor_tls_t *tls); +struct bufferevent *tor_tls_init_bufferevent(tor_tls_t *tls, + struct bufferevent *bufev_in, + evutil_socket_t socket, int receiving, + int filter); +#endif + +void tor_cert_free(tor_cert_t *cert); +tor_cert_t *tor_cert_decode(const uint8_t *certificate, + size_t certificate_len); +void tor_cert_get_der(const tor_cert_t *cert, + const uint8_t **encoded_out, size_t *size_out); +const digests_t *tor_cert_get_id_digests(const tor_cert_t *cert); +const digests_t *tor_cert_get_cert_digests(const tor_cert_t *cert); +int tor_tls_get_my_certs(int server, + const tor_cert_t **link_cert_out, + const tor_cert_t **id_cert_out); +crypto_pk_t *tor_tls_get_my_client_auth_key(void); +crypto_pk_t *tor_tls_cert_get_key(tor_cert_t *cert); +int tor_tls_cert_matches_key(const tor_tls_t *tls, const tor_cert_t *cert); +int tor_tls_cert_is_valid(int severity, + const tor_cert_t *cert, + const tor_cert_t *signing_cert, + int check_rsa_1024); #endif diff --git a/src/common/tortls_states.h b/src/common/tortls_states.h deleted file mode 100644 index dcff2479f6..0000000000 --- a/src/common/tortls_states.h +++ /dev/null @@ -1,414 +0,0 @@ -/* Copyright (c) 2003, Roger Dingledine - * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ -/* See LICENSE for licensing information */ - -/* Helper file: included only in tortls.c */ - -#ifndef _TORTLS_STATES_H -#define _TORTLS_STATES_H - -/* The main body of this file was mechanically generated with this - perl script: - - my %keys = (); - for $fn (@ARGV) { - open(F, $fn); - while (<F>) { - next unless /^#define ((?:SSL|DTLS)\w*_ST_\w*)/; - $keys{$1} = 1; - } - close(F); - } - for $k (sort keys %keys) { - print "#ifdef $k\n S($k),\n#endif\n" - } -*/ - -/** Mapping from allowed value of SSL.state to the name of C macro for that - * state. Used for debugging an openssl connection. */ -static const struct { int state; const char *name; } state_map[] = { -#define S(state) { state, #state } -#ifdef DTLS1_ST_CR_HELLO_VERIFY_REQUEST_A - S(DTLS1_ST_CR_HELLO_VERIFY_REQUEST_A), -#endif -#ifdef DTLS1_ST_CR_HELLO_VERIFY_REQUEST_B - S(DTLS1_ST_CR_HELLO_VERIFY_REQUEST_B), -#endif -#ifdef DTLS1_ST_SW_HELLO_VERIFY_REQUEST_A - S(DTLS1_ST_SW_HELLO_VERIFY_REQUEST_A), -#endif -#ifdef DTLS1_ST_SW_HELLO_VERIFY_REQUEST_B - S(DTLS1_ST_SW_HELLO_VERIFY_REQUEST_B), -#endif -#ifdef SSL23_ST_CR_SRVR_HELLO_A - S(SSL23_ST_CR_SRVR_HELLO_A), -#endif -#ifdef SSL23_ST_CR_SRVR_HELLO_B - S(SSL23_ST_CR_SRVR_HELLO_B), -#endif -#ifdef SSL23_ST_CW_CLNT_HELLO_A - S(SSL23_ST_CW_CLNT_HELLO_A), -#endif -#ifdef SSL23_ST_CW_CLNT_HELLO_B - S(SSL23_ST_CW_CLNT_HELLO_B), -#endif -#ifdef SSL23_ST_SR_CLNT_HELLO_A - S(SSL23_ST_SR_CLNT_HELLO_A), -#endif -#ifdef SSL23_ST_SR_CLNT_HELLO_B - S(SSL23_ST_SR_CLNT_HELLO_B), -#endif -#ifdef SSL2_ST_CLIENT_START_ENCRYPTION - S(SSL2_ST_CLIENT_START_ENCRYPTION), -#endif -#ifdef SSL2_ST_GET_CLIENT_FINISHED_A - S(SSL2_ST_GET_CLIENT_FINISHED_A), -#endif -#ifdef SSL2_ST_GET_CLIENT_FINISHED_B - S(SSL2_ST_GET_CLIENT_FINISHED_B), -#endif -#ifdef SSL2_ST_GET_CLIENT_HELLO_A - S(SSL2_ST_GET_CLIENT_HELLO_A), -#endif -#ifdef SSL2_ST_GET_CLIENT_HELLO_B - S(SSL2_ST_GET_CLIENT_HELLO_B), -#endif -#ifdef SSL2_ST_GET_CLIENT_HELLO_C - S(SSL2_ST_GET_CLIENT_HELLO_C), -#endif -#ifdef SSL2_ST_GET_CLIENT_MASTER_KEY_A - S(SSL2_ST_GET_CLIENT_MASTER_KEY_A), -#endif -#ifdef SSL2_ST_GET_CLIENT_MASTER_KEY_B - S(SSL2_ST_GET_CLIENT_MASTER_KEY_B), -#endif -#ifdef SSL2_ST_GET_SERVER_FINISHED_A - S(SSL2_ST_GET_SERVER_FINISHED_A), -#endif -#ifdef SSL2_ST_GET_SERVER_FINISHED_B - S(SSL2_ST_GET_SERVER_FINISHED_B), -#endif -#ifdef SSL2_ST_GET_SERVER_HELLO_A - S(SSL2_ST_GET_SERVER_HELLO_A), -#endif -#ifdef SSL2_ST_GET_SERVER_HELLO_B - S(SSL2_ST_GET_SERVER_HELLO_B), -#endif -#ifdef SSL2_ST_GET_SERVER_VERIFY_A - S(SSL2_ST_GET_SERVER_VERIFY_A), -#endif -#ifdef SSL2_ST_GET_SERVER_VERIFY_B - S(SSL2_ST_GET_SERVER_VERIFY_B), -#endif -#ifdef SSL2_ST_SEND_CLIENT_CERTIFICATE_A - S(SSL2_ST_SEND_CLIENT_CERTIFICATE_A), -#endif -#ifdef SSL2_ST_SEND_CLIENT_CERTIFICATE_B - S(SSL2_ST_SEND_CLIENT_CERTIFICATE_B), -#endif -#ifdef SSL2_ST_SEND_CLIENT_CERTIFICATE_C - S(SSL2_ST_SEND_CLIENT_CERTIFICATE_C), -#endif -#ifdef SSL2_ST_SEND_CLIENT_CERTIFICATE_D - S(SSL2_ST_SEND_CLIENT_CERTIFICATE_D), -#endif -#ifdef SSL2_ST_SEND_CLIENT_FINISHED_A - S(SSL2_ST_SEND_CLIENT_FINISHED_A), -#endif -#ifdef SSL2_ST_SEND_CLIENT_FINISHED_B - S(SSL2_ST_SEND_CLIENT_FINISHED_B), -#endif -#ifdef SSL2_ST_SEND_CLIENT_HELLO_A - S(SSL2_ST_SEND_CLIENT_HELLO_A), -#endif -#ifdef SSL2_ST_SEND_CLIENT_HELLO_B - S(SSL2_ST_SEND_CLIENT_HELLO_B), -#endif -#ifdef SSL2_ST_SEND_CLIENT_MASTER_KEY_A - S(SSL2_ST_SEND_CLIENT_MASTER_KEY_A), -#endif -#ifdef SSL2_ST_SEND_CLIENT_MASTER_KEY_B - S(SSL2_ST_SEND_CLIENT_MASTER_KEY_B), -#endif -#ifdef SSL2_ST_SEND_REQUEST_CERTIFICATE_A - S(SSL2_ST_SEND_REQUEST_CERTIFICATE_A), -#endif -#ifdef SSL2_ST_SEND_REQUEST_CERTIFICATE_B - S(SSL2_ST_SEND_REQUEST_CERTIFICATE_B), -#endif -#ifdef SSL2_ST_SEND_REQUEST_CERTIFICATE_C - S(SSL2_ST_SEND_REQUEST_CERTIFICATE_C), -#endif -#ifdef SSL2_ST_SEND_REQUEST_CERTIFICATE_D - S(SSL2_ST_SEND_REQUEST_CERTIFICATE_D), -#endif -#ifdef SSL2_ST_SEND_SERVER_FINISHED_A - S(SSL2_ST_SEND_SERVER_FINISHED_A), -#endif -#ifdef SSL2_ST_SEND_SERVER_FINISHED_B - S(SSL2_ST_SEND_SERVER_FINISHED_B), -#endif -#ifdef SSL2_ST_SEND_SERVER_HELLO_A - S(SSL2_ST_SEND_SERVER_HELLO_A), -#endif -#ifdef SSL2_ST_SEND_SERVER_HELLO_B - S(SSL2_ST_SEND_SERVER_HELLO_B), -#endif -#ifdef SSL2_ST_SEND_SERVER_VERIFY_A - S(SSL2_ST_SEND_SERVER_VERIFY_A), -#endif -#ifdef SSL2_ST_SEND_SERVER_VERIFY_B - S(SSL2_ST_SEND_SERVER_VERIFY_B), -#endif -#ifdef SSL2_ST_SEND_SERVER_VERIFY_C - S(SSL2_ST_SEND_SERVER_VERIFY_C), -#endif -#ifdef SSL2_ST_SERVER_START_ENCRYPTION - S(SSL2_ST_SERVER_START_ENCRYPTION), -#endif -#ifdef SSL2_ST_X509_GET_CLIENT_CERTIFICATE - S(SSL2_ST_X509_GET_CLIENT_CERTIFICATE), -#endif -#ifdef SSL2_ST_X509_GET_SERVER_CERTIFICATE - S(SSL2_ST_X509_GET_SERVER_CERTIFICATE), -#endif -#ifdef SSL3_ST_CR_CERT_A - S(SSL3_ST_CR_CERT_A), -#endif -#ifdef SSL3_ST_CR_CERT_B - S(SSL3_ST_CR_CERT_B), -#endif -#ifdef SSL3_ST_CR_CERT_REQ_A - S(SSL3_ST_CR_CERT_REQ_A), -#endif -#ifdef SSL3_ST_CR_CERT_REQ_B - S(SSL3_ST_CR_CERT_REQ_B), -#endif -#ifdef SSL3_ST_CR_CERT_STATUS_A - S(SSL3_ST_CR_CERT_STATUS_A), -#endif -#ifdef SSL3_ST_CR_CERT_STATUS_B - S(SSL3_ST_CR_CERT_STATUS_B), -#endif -#ifdef SSL3_ST_CR_CHANGE_A - S(SSL3_ST_CR_CHANGE_A), -#endif -#ifdef SSL3_ST_CR_CHANGE_B - S(SSL3_ST_CR_CHANGE_B), -#endif -#ifdef SSL3_ST_CR_FINISHED_A - S(SSL3_ST_CR_FINISHED_A), -#endif -#ifdef SSL3_ST_CR_FINISHED_B - S(SSL3_ST_CR_FINISHED_B), -#endif -#ifdef SSL3_ST_CR_KEY_EXCH_A - S(SSL3_ST_CR_KEY_EXCH_A), -#endif -#ifdef SSL3_ST_CR_KEY_EXCH_B - S(SSL3_ST_CR_KEY_EXCH_B), -#endif -#ifdef SSL3_ST_CR_SESSION_TICKET_A - S(SSL3_ST_CR_SESSION_TICKET_A), -#endif -#ifdef SSL3_ST_CR_SESSION_TICKET_B - S(SSL3_ST_CR_SESSION_TICKET_B), -#endif -#ifdef SSL3_ST_CR_SRVR_DONE_A - S(SSL3_ST_CR_SRVR_DONE_A), -#endif -#ifdef SSL3_ST_CR_SRVR_DONE_B - S(SSL3_ST_CR_SRVR_DONE_B), -#endif -#ifdef SSL3_ST_CR_SRVR_HELLO_A - S(SSL3_ST_CR_SRVR_HELLO_A), -#endif -#ifdef SSL3_ST_CR_SRVR_HELLO_B - S(SSL3_ST_CR_SRVR_HELLO_B), -#endif -#ifdef SSL3_ST_CW_CERT_A - S(SSL3_ST_CW_CERT_A), -#endif -#ifdef SSL3_ST_CW_CERT_B - S(SSL3_ST_CW_CERT_B), -#endif -#ifdef SSL3_ST_CW_CERT_C - S(SSL3_ST_CW_CERT_C), -#endif -#ifdef SSL3_ST_CW_CERT_D - S(SSL3_ST_CW_CERT_D), -#endif -#ifdef SSL3_ST_CW_CERT_VRFY_A - S(SSL3_ST_CW_CERT_VRFY_A), -#endif -#ifdef SSL3_ST_CW_CERT_VRFY_B - S(SSL3_ST_CW_CERT_VRFY_B), -#endif -#ifdef SSL3_ST_CW_CHANGE_A - S(SSL3_ST_CW_CHANGE_A), -#endif -#ifdef SSL3_ST_CW_CHANGE_B - S(SSL3_ST_CW_CHANGE_B), -#endif -#ifdef SSL3_ST_CW_CLNT_HELLO_A - S(SSL3_ST_CW_CLNT_HELLO_A), -#endif -#ifdef SSL3_ST_CW_CLNT_HELLO_B - S(SSL3_ST_CW_CLNT_HELLO_B), -#endif -#ifdef SSL3_ST_CW_FINISHED_A - S(SSL3_ST_CW_FINISHED_A), -#endif -#ifdef SSL3_ST_CW_FINISHED_B - S(SSL3_ST_CW_FINISHED_B), -#endif -#ifdef SSL3_ST_CW_FLUSH - S(SSL3_ST_CW_FLUSH), -#endif -#ifdef SSL3_ST_CW_KEY_EXCH_A - S(SSL3_ST_CW_KEY_EXCH_A), -#endif -#ifdef SSL3_ST_CW_KEY_EXCH_B - S(SSL3_ST_CW_KEY_EXCH_B), -#endif -#ifdef SSL3_ST_SR_CERT_A - S(SSL3_ST_SR_CERT_A), -#endif -#ifdef SSL3_ST_SR_CERT_B - S(SSL3_ST_SR_CERT_B), -#endif -#ifdef SSL3_ST_SR_CERT_VRFY_A - S(SSL3_ST_SR_CERT_VRFY_A), -#endif -#ifdef SSL3_ST_SR_CERT_VRFY_B - S(SSL3_ST_SR_CERT_VRFY_B), -#endif -#ifdef SSL3_ST_SR_CHANGE_A - S(SSL3_ST_SR_CHANGE_A), -#endif -#ifdef SSL3_ST_SR_CHANGE_B - S(SSL3_ST_SR_CHANGE_B), -#endif -#ifdef SSL3_ST_SR_CLNT_HELLO_A - S(SSL3_ST_SR_CLNT_HELLO_A), -#endif -#ifdef SSL3_ST_SR_CLNT_HELLO_B - S(SSL3_ST_SR_CLNT_HELLO_B), -#endif -#ifdef SSL3_ST_SR_CLNT_HELLO_C - S(SSL3_ST_SR_CLNT_HELLO_C), -#endif -#ifdef SSL3_ST_SR_FINISHED_A - S(SSL3_ST_SR_FINISHED_A), -#endif -#ifdef SSL3_ST_SR_FINISHED_B - S(SSL3_ST_SR_FINISHED_B), -#endif -#ifdef SSL3_ST_SR_KEY_EXCH_A - S(SSL3_ST_SR_KEY_EXCH_A), -#endif -#ifdef SSL3_ST_SR_KEY_EXCH_B - S(SSL3_ST_SR_KEY_EXCH_B), -#endif -#ifdef SSL3_ST_SW_CERT_A - S(SSL3_ST_SW_CERT_A), -#endif -#ifdef SSL3_ST_SW_CERT_B - S(SSL3_ST_SW_CERT_B), -#endif -#ifdef SSL3_ST_SW_CERT_REQ_A - S(SSL3_ST_SW_CERT_REQ_A), -#endif -#ifdef SSL3_ST_SW_CERT_REQ_B - S(SSL3_ST_SW_CERT_REQ_B), -#endif -#ifdef SSL3_ST_SW_CERT_STATUS_A - S(SSL3_ST_SW_CERT_STATUS_A), -#endif -#ifdef SSL3_ST_SW_CERT_STATUS_B - S(SSL3_ST_SW_CERT_STATUS_B), -#endif -#ifdef SSL3_ST_SW_CHANGE_A - S(SSL3_ST_SW_CHANGE_A), -#endif -#ifdef SSL3_ST_SW_CHANGE_B - S(SSL3_ST_SW_CHANGE_B), -#endif -#ifdef SSL3_ST_SW_FINISHED_A - S(SSL3_ST_SW_FINISHED_A), -#endif -#ifdef SSL3_ST_SW_FINISHED_B - S(SSL3_ST_SW_FINISHED_B), -#endif -#ifdef SSL3_ST_SW_FLUSH - S(SSL3_ST_SW_FLUSH), -#endif -#ifdef SSL3_ST_SW_HELLO_REQ_A - S(SSL3_ST_SW_HELLO_REQ_A), -#endif -#ifdef SSL3_ST_SW_HELLO_REQ_B - S(SSL3_ST_SW_HELLO_REQ_B), -#endif -#ifdef SSL3_ST_SW_HELLO_REQ_C - S(SSL3_ST_SW_HELLO_REQ_C), -#endif -#ifdef SSL3_ST_SW_KEY_EXCH_A - S(SSL3_ST_SW_KEY_EXCH_A), -#endif -#ifdef SSL3_ST_SW_KEY_EXCH_B - S(SSL3_ST_SW_KEY_EXCH_B), -#endif -#ifdef SSL3_ST_SW_SESSION_TICKET_A - S(SSL3_ST_SW_SESSION_TICKET_A), -#endif -#ifdef SSL3_ST_SW_SESSION_TICKET_B - S(SSL3_ST_SW_SESSION_TICKET_B), -#endif -#ifdef SSL3_ST_SW_SRVR_DONE_A - S(SSL3_ST_SW_SRVR_DONE_A), -#endif -#ifdef SSL3_ST_SW_SRVR_DONE_B - S(SSL3_ST_SW_SRVR_DONE_B), -#endif -#ifdef SSL3_ST_SW_SRVR_HELLO_A - S(SSL3_ST_SW_SRVR_HELLO_A), -#endif -#ifdef SSL3_ST_SW_SRVR_HELLO_B - S(SSL3_ST_SW_SRVR_HELLO_B), -#endif -#ifdef SSL_ST_ACCEPT - S(SSL_ST_ACCEPT), -#endif -#ifdef SSL_ST_BEFORE - S(SSL_ST_BEFORE), -#endif -#ifdef SSL_ST_CONNECT - S(SSL_ST_CONNECT), -#endif -#ifdef SSL_ST_INIT - S(SSL_ST_INIT), -#endif -#ifdef SSL_ST_MASK - S(SSL_ST_MASK), -#endif -#ifdef SSL_ST_OK - S(SSL_ST_OK), -#endif -#ifdef SSL_ST_READ_BODY - S(SSL_ST_READ_BODY), -#endif -#ifdef SSL_ST_READ_DONE - S(SSL_ST_READ_DONE), -#endif -#ifdef SSL_ST_READ_HEADER - S(SSL_ST_READ_HEADER), -#endif -#ifdef SSL_ST_RENEGOTIATE - S(SSL_ST_RENEGOTIATE), -#endif - { 0, NULL } -}; - -#endif - diff --git a/src/common/util.c b/src/common/util.c index 698f989ec1..6fb597a3a5 100644 --- a/src/common/util.c +++ b/src/common/util.c @@ -1,6 +1,6 @@ /* Copyright (c) 2003, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -14,6 +14,10 @@ #define _GNU_SOURCE #include "orconfig.h" +#ifdef HAVE_FCNTL_H +#include <fcntl.h> +#endif +#define UTIL_PRIVATE #include "util.h" #include "torlog.h" #undef log @@ -22,11 +26,12 @@ #include "container.h" #include "address.h" -#ifdef MS_WINDOWS +#ifdef _WIN32 #include <io.h> #include <direct.h> #include <process.h> #include <tchar.h> +#include <winbase.h> #else #include <dirent.h> #include <pwd.h> @@ -42,6 +47,7 @@ #include <stdio.h> #include <string.h> #include <assert.h> +#include <signal.h> #ifdef HAVE_NETINET_IN_H #include <netinet/in.h> @@ -67,9 +73,6 @@ #ifdef HAVE_SYS_FCNTL_H #include <sys/fcntl.h> #endif -#ifdef HAVE_FCNTL_H -#include <fcntl.h> -#endif #ifdef HAVE_TIME_H #include <time.h> #endif @@ -77,7 +80,7 @@ #include <malloc/malloc.h> #endif #ifdef HAVE_MALLOC_H -#ifndef OPENBSD +#if !defined(OPENBSD) && !defined(__FreeBSD__) /* OpenBSD has a malloc.h, but for our purposes, it only exists in order to * scold us for being so stupid as to autodetect its presence. To be fair, * they've done this since 1996, when autoconf was only 5 years old. */ @@ -87,6 +90,9 @@ #ifdef HAVE_MALLOC_NP_H #include <malloc_np.h> #endif +#ifdef HAVE_SYS_WAIT_H +#include <sys/wait.h> +#endif /* ===== * Memory management @@ -166,6 +172,35 @@ _tor_malloc_zero(size_t size DMALLOC_PARAMS) return result; } +/** Allocate a chunk of <b>nmemb</b>*<b>size</b> bytes of memory, fill + * the memory with zero bytes, and return a pointer to the result. + * Log and terminate the process on error. (Same as + * calloc(<b>nmemb</b>,<b>size</b>), but never returns NULL.) + * + * XXXX This implementation probably asserts in cases where it could + * work, because it only tries dividing SIZE_MAX by size (according to + * the calloc(3) man page, the size of an element of the nmemb-element + * array to be allocated), not by nmemb (which could in theory be + * smaller than size). Don't do that then. + */ +void * +_tor_calloc(size_t nmemb, size_t size DMALLOC_PARAMS) +{ + /* You may ask yourself, "wouldn't it be smart to use calloc instead of + * malloc+memset? Perhaps libc's calloc knows some nifty optimization trick + * we don't!" Indeed it does, but its optimizations are only a big win when + * we're allocating something very big (it knows if it just got the memory + * from the OS in a pre-zeroed state). We don't want to use tor_malloc_zero + * for big stuff, so we don't bother with calloc. */ + void *result; + size_t max_nmemb = (size == 0) ? SIZE_MAX : SIZE_MAX/size; + + tor_assert(nmemb < max_nmemb); + + result = _tor_malloc_zero((nmemb * size) DMALLOC_FN_ARGS); + return result; +} + /** Change the size of the memory block pointed to by <b>ptr</b> to <b>size</b> * bytes long; return the new memory block. On error, log and * terminate. (Like realloc(ptr,size), but never returns NULL.) @@ -417,6 +452,32 @@ round_uint64_to_next_multiple_of(uint64_t number, uint64_t divisor) return number; } +/** Return the number of bits set in <b>v</b>. */ +int +n_bits_set_u8(uint8_t v) +{ + static const int nybble_table[] = { + 0, /* 0000 */ + 1, /* 0001 */ + 1, /* 0010 */ + 2, /* 0011 */ + 1, /* 0100 */ + 2, /* 0101 */ + 2, /* 0110 */ + 3, /* 0111 */ + 1, /* 1000 */ + 2, /* 1001 */ + 2, /* 1010 */ + 3, /* 1011 */ + 2, /* 1100 */ + 3, /* 1101 */ + 3, /* 1110 */ + 4, /* 1111 */ + }; + + return nybble_table[v & 15] + nybble_table[v>>4]; +} + /* ===== * String manipulation * ===== */ @@ -500,6 +561,23 @@ tor_strisnonupper(const char *s) return 1; } +/** As strcmp, except that either string may be NULL. The NULL string is + * considered to be before any non-NULL string. */ +int +strcmp_opt(const char *s1, const char *s2) +{ + if (!s1) { + if (!s2) + return 0; + else + return -1; + } else if (!s2) { + return 1; + } else { + return strcmp(s1, s2); + } +} + /** Compares the first strlen(s2) characters of s1 with s2. Returns as for * strcmp. */ @@ -722,6 +800,34 @@ find_str_at_start_of_line(const char *haystack, const char *needle) return NULL; } +/** Returns true if <b>string</b> could be a C identifier. + A C identifier must begin with a letter or an underscore and the + rest of its characters can be letters, numbers or underscores. No + length limit is imposed. */ +int +string_is_C_identifier(const char *string) +{ + size_t iter; + size_t length = strlen(string); + if (!length) + return 0; + + for (iter = 0; iter < length ; iter++) { + if (iter == 0) { + if (!(TOR_ISALPHA(string[iter]) || + string[iter] == '_')) + return 0; + } else { + if (!(TOR_ISALPHA(string[iter]) || + TOR_ISDIGIT(string[iter]) || + string[iter] == '_')) + return 0; + } + } + + return 1; +} + /** Return true iff the 'len' bytes at 'mem' are all zero. */ int tor_mem_is_zero(const char *mem, size_t len) @@ -803,6 +909,12 @@ tor_parse_long(const char *s, int base, long min, long max, char *endptr; long r; + if (base < 0) { + if (ok) + *ok = 0; + return 0; + } + errno = 0; r = strtol(s, &endptr, base); CHECK_STRTOX_RESULT(); @@ -816,6 +928,12 @@ tor_parse_ulong(const char *s, int base, unsigned long min, char *endptr; unsigned long r; + if (base < 0) { + if (ok) + *ok = 0; + return 0; + } + errno = 0; r = strtoul(s, &endptr, base); CHECK_STRTOX_RESULT(); @@ -842,10 +960,16 @@ tor_parse_uint64(const char *s, int base, uint64_t min, char *endptr; uint64_t r; + if (base < 0) { + if (ok) + *ok = 0; + return 0; + } + errno = 0; #ifdef HAVE_STRTOULL r = (uint64_t)strtoull(s, &endptr, base); -#elif defined(MS_WINDOWS) +#elif defined(_WIN32) #if defined(_MSC_VER) && _MSC_VER < 1300 tor_assert(base <= 10); r = (uint64_t)_atoi64(s); @@ -965,7 +1089,7 @@ esc_for_log(const char *s) char *result, *outp; size_t len = 3; if (!s) { - return tor_strdup(""); + return tor_strdup("(null)"); } for (cp = s; *cp; ++cp) { @@ -1361,6 +1485,26 @@ format_iso_time(char *buf, time_t t) strftime(buf, ISO_TIME_LEN+1, "%Y-%m-%d %H:%M:%S", tor_gmtime_r(&t, &tm)); } +/** As format_iso_time, but use the yyyy-mm-ddThh:mm:ss format to avoid + * embedding an internal space. */ +void +format_iso_time_nospace(char *buf, time_t t) +{ + format_iso_time(buf, t); + buf[10] = 'T'; +} + +/** As format_iso_time_nospace, but include microseconds in decimal + * fixed-point format. Requires that buf be at least ISO_TIME_USEC_LEN+1 + * bytes long. */ +void +format_iso_time_nospace_usec(char *buf, const struct timeval *tv) +{ + tor_assert(tv); + format_iso_time_nospace(buf, tv->tv_sec); + tor_snprintf(buf+ISO_TIME_LEN, 8, ".%06d", (int)tv->tv_usec); +} + /** Given an ISO-formatted UTC time value (after the epoch) in <b>cp</b>, * parse it and store its value in *<b>t</b>. Return 0 on success, -1 on * failure. Ignore extraneous stuff in <b>cp</b> separated by whitespace from @@ -1646,7 +1790,7 @@ read_all(tor_socket_t fd, char *buf, size_t count, int isSocket) static void clean_name_for_stat(char *name) { -#ifdef MS_WINDOWS +#ifdef _WIN32 size_t len = strlen(name); if (!len) return; @@ -1705,7 +1849,7 @@ check_private_dir(const char *dirname, cpd_check_t check, int r; struct stat st; char *f; -#ifndef MS_WINDOWS +#ifndef _WIN32 int mask; struct passwd *pw = NULL; uid_t running_uid; @@ -1727,7 +1871,7 @@ check_private_dir(const char *dirname, cpd_check_t check, } if (check & CPD_CREATE) { log_info(LD_GENERAL, "Creating directory %s", dirname); -#if defined (MS_WINDOWS) && !defined (WINCE) +#if defined (_WIN32) && !defined (WINCE) r = mkdir(dirname); #else r = mkdir(dirname, 0700); @@ -1749,7 +1893,7 @@ check_private_dir(const char *dirname, cpd_check_t check, log_warn(LD_FS, "%s is not a directory", dirname); return -1; } -#ifndef MS_WINDOWS +#ifndef _WIN32 if (effective_user) { /* Look up the user and group information. * If we have a problem, bail out. */ @@ -1836,7 +1980,7 @@ check_private_dir(const char *dirname, cpd_check_t check, int write_str_to_file(const char *fname, const char *str, int bin) { -#ifdef MS_WINDOWS +#ifdef _WIN32 if (!bin && strchr(str, '\r')) { log_warn(LD_BUG, "We're writing a text string that already contains a CR."); @@ -1879,7 +2023,6 @@ int start_writing_to_file(const char *fname, int open_flags, int mode, open_file_t **data_out) { - size_t tempname_len = strlen(fname)+16; open_file_t *new_file = tor_malloc_zero(sizeof(open_file_t)); const char *open_name; int append = 0; @@ -1890,7 +2033,6 @@ start_writing_to_file(const char *fname, int open_flags, int mode, tor_assert((open_flags & (O_BINARY|O_TEXT)) != 0); #endif new_file->fd = -1; - tor_assert(tempname_len > strlen(fname)); /*check for overflow*/ new_file->filename = tor_strdup(fname); if (open_flags & O_APPEND) { open_name = fname; @@ -1898,11 +2040,8 @@ start_writing_to_file(const char *fname, int open_flags, int mode, append = 1; open_flags &= ~O_APPEND; } else { - open_name = new_file->tempname = tor_malloc(tempname_len); - if (tor_snprintf(new_file->tempname, tempname_len, "%s.tmp", fname)<0) { - log_warn(LD_GENERAL, "Failed to generate filename"); - goto err; - } + tor_asprintf(&new_file->tempname, "%s.tmp", fname); + open_name = new_file->tempname; /* We always replace an existing temporary file if there is one. */ open_flags |= O_CREAT|O_TRUNC; open_flags &= ~O_EXCL; @@ -1911,7 +2050,7 @@ start_writing_to_file(const char *fname, int open_flags, int mode, if (open_flags & O_BINARY) new_file->binary = 1; - new_file->fd = open(open_name, open_flags, mode); + new_file->fd = tor_open_cloexec(open_name, open_flags, mode); if (new_file->fd < 0) { log_warn(LD_FS, "Couldn't open \"%s\" (%s) for writing: %s", open_name, fname, strerror(errno)); @@ -2072,36 +2211,50 @@ write_chunks_to_file(const char *fname, const smartlist_t *chunks, int bin) return write_chunks_to_file_impl(fname, chunks, flags); } -/** As write_str_to_file, but does not assume a NUL-terminated - * string. Instead, we write <b>len</b> bytes, starting at <b>str</b>. */ -int -write_bytes_to_file(const char *fname, const char *str, size_t len, - int bin) +/** Write <b>len</b> bytes, starting at <b>str</b>, to <b>fname</b> + using the open() flags passed in <b>flags</b>. */ +static int +write_bytes_to_file_impl(const char *fname, const char *str, size_t len, + int flags) { - int flags = OPEN_FLAGS_REPLACE|(bin?O_BINARY:O_TEXT); int r; sized_chunk_t c = { str, len }; - smartlist_t *chunks = smartlist_create(); + smartlist_t *chunks = smartlist_new(); smartlist_add(chunks, &c); r = write_chunks_to_file_impl(fname, chunks, flags); smartlist_free(chunks); return r; } +/** As write_str_to_file, but does not assume a NUL-terminated + * string. Instead, we write <b>len</b> bytes, starting at <b>str</b>. */ +int +write_bytes_to_file(const char *fname, const char *str, size_t len, + int bin) +{ + return write_bytes_to_file_impl(fname, str, len, + OPEN_FLAGS_REPLACE|(bin?O_BINARY:O_TEXT)); +} + /** As write_bytes_to_file, but if the file already exists, append the bytes * to the end of the file instead of overwriting it. */ int append_bytes_to_file(const char *fname, const char *str, size_t len, int bin) { - int flags = OPEN_FLAGS_APPEND|(bin?O_BINARY:O_TEXT); - int r; - sized_chunk_t c = { str, len }; - smartlist_t *chunks = smartlist_create(); - smartlist_add(chunks, &c); - r = write_chunks_to_file_impl(fname, chunks, flags); - smartlist_free(chunks); - return r; + return write_bytes_to_file_impl(fname, str, len, + OPEN_FLAGS_APPEND|(bin?O_BINARY:O_TEXT)); +} + +/** Like write_str_to_file(), but also return -1 if there was a file + already residing in <b>fname</b>. */ +int +write_bytes_to_new_file(const char *fname, const char *str, size_t len, + int bin) +{ + return write_bytes_to_file_impl(fname, str, len, + OPEN_FLAGS_DONT_REPLACE| + (bin?O_BINARY:O_TEXT)); } /** Read the contents of <b>filename</b> into a newly allocated @@ -2132,7 +2285,7 @@ read_file_to_str(const char *filename, int flags, struct stat *stat_out) tor_assert(filename); - fd = open(filename,O_RDONLY|(bin?O_BINARY:O_TEXT),0); + fd = tor_open_cloexec(filename,O_RDONLY|(bin?O_BINARY:O_TEXT),0); if (fd<0) { int severity = LOG_WARN; int save_errno = errno; @@ -2169,7 +2322,7 @@ read_file_to_str(const char *filename, int flags, struct stat *stat_out) } string[r] = '\0'; /* NUL-terminate the result. */ -#ifdef MS_WINDOWS +#if defined(_WIN32) || defined(__CYGWIN__) if (!bin && strchr(string, '\r')) { log_debug(LD_FS, "We didn't convert CRLF to LF as well as we hoped " "when reading %s. Coping.", @@ -2334,7 +2487,7 @@ parse_config_line_from_str(const char *line, char **key_out, char **value_out) KEYCHAR = Any character except ' ', '\r', '\n', '\t', '#', "\" VALUES = QUOTEDVALUE | NORMALVALUE - QUOTEDVALUE = QUOTE QVITEM* QUOTE EOLSPACE? + QUOTEDVALUE = QUOTE QVCHAR* QUOTE EOLSPACE? QUOTE = '"' QVCHAR = KEYCHAR | ESC ('n' | 't' | 'r' | '"' | ESC |'\'' | OCTAL | HEX) ESC = "\\" @@ -2462,7 +2615,7 @@ char * expand_filename(const char *filename) { tor_assert(filename); -#ifdef MS_WINDOWS +#ifdef _WIN32 return tor_strdup(filename); #else if (*filename == '~') { @@ -2530,18 +2683,21 @@ digit_to_num(char d) * success, store the result in <b>out</b>, advance bufp to the next * character, and return 0. On failure, return -1. */ static int -scan_unsigned(const char **bufp, unsigned *out, int width) +scan_unsigned(const char **bufp, unsigned *out, int width, int base) { unsigned result = 0; int scanned_so_far = 0; + const int hex = base==16; + tor_assert(base == 10 || base == 16); if (!bufp || !*bufp || !out) return -1; if (width<0) width=MAX_SCANF_WIDTH; - while (**bufp && TOR_ISDIGIT(**bufp) && scanned_so_far < width) { - int digit = digit_to_num(*(*bufp)++); - unsigned new_result = result * 10 + digit; + while (**bufp && (hex?TOR_ISXDIGIT(**bufp):TOR_ISDIGIT(**bufp)) + && scanned_so_far < width) { + int digit = hex?hex_decode_digit(*(*bufp)++):digit_to_num(*(*bufp)++); + unsigned new_result = result * base + digit; if (new_result > UINT32_MAX || new_result < result) return -1; /* over/underflow. */ result = new_result; @@ -2603,11 +2759,12 @@ tor_vsscanf(const char *buf, const char *pattern, va_list ap) if (!width) /* No zero-width things. */ return -1; } - if (*pattern == 'u') { + if (*pattern == 'u' || *pattern == 'x') { unsigned *u = va_arg(ap, unsigned *); + const int base = (*pattern == 'u') ? 10 : 16; if (!*buf) return n_matched; - if (scan_unsigned(&buf, u, width)<0) + if (scan_unsigned(&buf, u, width, base)<0) return n_matched; ++pattern; ++n_matched; @@ -2630,7 +2787,7 @@ tor_vsscanf(const char *buf, const char *pattern, va_list ap) ++n_matched; } else if (*pattern == '%') { if (*buf != '%') - return -1; + return n_matched; ++buf; ++pattern; } else { @@ -2644,9 +2801,9 @@ tor_vsscanf(const char *buf, const char *pattern, va_list ap) /** Minimal sscanf replacement: parse <b>buf</b> according to <b>pattern</b> * and store the results in the corresponding argument fields. Differs from - * sscanf in that it: Only handles %u and %Ns. Does not handle arbitrarily - * long widths. %u does not consume any space. Is locale-independent. - * Returns -1 on malformed patterns. + * sscanf in that it: Only handles %u, %x, %c and %Ns. Does not handle + * arbitrarily long widths. %u and %x do not consume any space. Is + * locale-independent. Returns -1 on malformed patterns. * * (As with other locale-independent functions, we need this to parse data that * is in ASCII without worrying that the C library's locale-handling will make @@ -2663,6 +2820,30 @@ tor_sscanf(const char *buf, const char *pattern, ...) return r; } +/** Append the string produced by tor_asprintf(<b>pattern</b>, <b>...</b>) + * to <b>sl</b>. */ +void +smartlist_add_asprintf(struct smartlist_t *sl, const char *pattern, ...) +{ + va_list ap; + va_start(ap, pattern); + smartlist_add_vasprintf(sl, pattern, ap); + va_end(ap); +} + +/** va_list-based backend of smartlist_add_asprintf. */ +void +smartlist_add_vasprintf(struct smartlist_t *sl, const char *pattern, + va_list args) +{ + char *str = NULL; + + tor_vasprintf(&str, pattern, args); + tor_assert(str != NULL); + + smartlist_add(sl, str); +} + /** Return a new list containing the filenames in the directory <b>dirname</b>. * Return NULL on error or if <b>dirname</b> is not a directory. */ @@ -2670,15 +2851,13 @@ smartlist_t * tor_listdir(const char *dirname) { smartlist_t *result; -#ifdef MS_WINDOWS - char *pattern; +#ifdef _WIN32 + char *pattern=NULL; TCHAR tpattern[MAX_PATH] = {0}; - char name[MAX_PATH] = {0}; + char name[MAX_PATH*2+1] = {0}; HANDLE handle; WIN32_FIND_DATA findData; - size_t pattern_len = strlen(dirname)+16; - pattern = tor_malloc(pattern_len); - tor_snprintf(pattern, pattern_len, "%s\\*", dirname); + tor_asprintf(&pattern, "%s\\*", dirname); #ifdef UNICODE mbstowcs(tpattern,pattern,MAX_PATH); #else @@ -2688,10 +2867,11 @@ tor_listdir(const char *dirname) tor_free(pattern); return NULL; } - result = smartlist_create(); + result = smartlist_new(); while (1) { #ifdef UNICODE wcstombs(name,findData.cFileName,MAX_PATH); + name[sizeof(name)-1] = '\0'; #else strlcpy(name,findData.cFileName,sizeof(name)); #endif @@ -2717,7 +2897,7 @@ tor_listdir(const char *dirname) if (!(d = opendir(dirname))) return NULL; - result = smartlist_create(); + result = smartlist_new(); while ((de = readdir(d))) { if (!strcmp(de->d_name, ".") || !strcmp(de->d_name, "..")) @@ -2735,7 +2915,7 @@ path_is_relative(const char *filename) { if (filename && filename[0] == '/') return 0; -#ifdef MS_WINDOWS +#ifdef _WIN32 else if (filename && filename[0] == '\\') return 0; else if (filename && strlen(filename)>3 && TOR_ISALPHA(filename[0]) && @@ -2750,7 +2930,7 @@ path_is_relative(const char *filename) * Process helpers * ===== */ -#ifndef MS_WINDOWS +#ifndef _WIN32 /* Based on code contributed by christian grothoff */ /** True iff we've called start_daemon(). */ static int start_daemon_called = 0; @@ -2840,7 +3020,7 @@ finish_daemon(const char *desired_cwd) exit(1); } - nullfd = open("/dev/null", O_RDWR); + nullfd = tor_open_cloexec("/dev/null", O_RDWR, 0); if (nullfd < 0) { log_err(LD_GENERAL,"/dev/null can't be opened. Exiting."); exit(1); @@ -2864,7 +3044,7 @@ finish_daemon(const char *desired_cwd) close(daemon_filedes[1]); } #else -/* defined(MS_WINDOWS) */ +/* defined(_WIN32) */ void start_daemon(void) { @@ -2887,7 +3067,7 @@ write_pidfile(char *filename) log_warn(LD_FS, "Unable to open \"%s\" for writing: %s", filename, strerror(errno)); } else { -#ifdef MS_WINDOWS +#ifdef _WIN32 fprintf(pidfile, "%d\n", (int)_getpid()); #else fprintf(pidfile, "%d\n", (int)getpid()); @@ -2896,7 +3076,7 @@ write_pidfile(char *filename) } } -#ifdef MS_WINDOWS +#ifdef _WIN32 HANDLE load_windows_system_library(const TCHAR *library_name) { @@ -2911,3 +3091,1463 @@ load_windows_system_library(const TCHAR *library_name) } #endif +/** Format a single argument for being put on a Windows command line. + * Returns a newly allocated string */ +static char * +format_win_cmdline_argument(const char *arg) +{ + char *formatted_arg; + char need_quotes; + const char *c; + int i; + int bs_counter = 0; + /* Backslash we can point to when one is inserted into the string */ + const char backslash = '\\'; + + /* Smartlist of *char */ + smartlist_t *arg_chars; + arg_chars = smartlist_new(); + + /* Quote string if it contains whitespace or is empty */ + need_quotes = (strchr(arg, ' ') || strchr(arg, '\t') || '\0' == arg[0]); + + /* Build up smartlist of *chars */ + for (c=arg; *c != '\0'; c++) { + if ('"' == *c) { + /* Double up backslashes preceding a quote */ + for (i=0; i<(bs_counter*2); i++) + smartlist_add(arg_chars, (void*)&backslash); + bs_counter = 0; + /* Escape the quote */ + smartlist_add(arg_chars, (void*)&backslash); + smartlist_add(arg_chars, (void*)c); + } else if ('\\' == *c) { + /* Count backslashes until we know whether to double up */ + bs_counter++; + } else { + /* Don't double up slashes preceding a non-quote */ + for (i=0; i<bs_counter; i++) + smartlist_add(arg_chars, (void*)&backslash); + bs_counter = 0; + smartlist_add(arg_chars, (void*)c); + } + } + /* Don't double up trailing backslashes */ + for (i=0; i<bs_counter; i++) + smartlist_add(arg_chars, (void*)&backslash); + + /* Allocate space for argument, quotes (if needed), and terminator */ + formatted_arg = tor_malloc(sizeof(char) * + (smartlist_len(arg_chars) + (need_quotes?2:0) + 1)); + + /* Add leading quote */ + i=0; + if (need_quotes) + formatted_arg[i++] = '"'; + + /* Add characters */ + SMARTLIST_FOREACH(arg_chars, char*, c, + { + formatted_arg[i++] = *c; + }); + + /* Add trailing quote */ + if (need_quotes) + formatted_arg[i++] = '"'; + formatted_arg[i] = '\0'; + + smartlist_free(arg_chars); + return formatted_arg; +} + +/** Format a command line for use on Windows, which takes the command as a + * string rather than string array. Follows the rules from "Parsing C++ + * Command-Line Arguments" in MSDN. Algorithm based on list2cmdline in the + * Python subprocess module. Returns a newly allocated string */ +char * +tor_join_win_cmdline(const char *argv[]) +{ + smartlist_t *argv_list; + char *joined_argv; + int i; + + /* Format each argument and put the result in a smartlist */ + argv_list = smartlist_new(); + for (i=0; argv[i] != NULL; i++) { + smartlist_add(argv_list, (void *)format_win_cmdline_argument(argv[i])); + } + + /* Join the arguments with whitespace */ + joined_argv = smartlist_join_strings(argv_list, " ", 0, NULL); + + /* Free the newly allocated arguments, and the smartlist */ + SMARTLIST_FOREACH(argv_list, char *, arg, + { + tor_free(arg); + }); + smartlist_free(argv_list); + + return joined_argv; +} + +/** + * Helper function to output hex numbers, called by + * format_helper_exit_status(). This writes the hexadecimal digits of x into + * buf, up to max_len digits, and returns the actual number of digits written. + * If there is insufficient space, it will write nothing and return 0. + * + * This function DOES NOT add a terminating NUL character to its output: be + * careful! + * + * This accepts an unsigned int because format_helper_exit_status() needs to + * call it with a signed int and an unsigned char, and since the C standard + * does not guarantee that an int is wider than a char (an int must be at + * least 16 bits but it is permitted for a char to be that wide as well), we + * can't assume a signed int is sufficient to accomodate an unsigned char. + * Thus, format_helper_exit_status() will still need to emit any require '-' + * on its own. + * + * For most purposes, you'd want to use tor_snprintf("%x") instead of this + * function; it's designed to be used in code paths where you can't call + * arbitrary C functions. + */ +int +format_hex_number_for_helper_exit_status(unsigned int x, char *buf, + int max_len) +{ + int len; + unsigned int tmp; + char *cur; + + /* Sanity check */ + if (!buf || max_len <= 0) + return 0; + + /* How many chars do we need for x? */ + if (x > 0) { + len = 0; + tmp = x; + while (tmp > 0) { + tmp >>= 4; + ++len; + } + } else { + len = 1; + } + + /* Bail if we would go past the end of the buffer */ + if (len > max_len) + return 0; + + /* Point to last one */ + cur = buf + len - 1; + + /* Convert x to hex */ + do { + *cur-- = "0123456789ABCDEF"[x & 0xf]; + x >>= 4; + } while (x != 0 && cur >= buf); + + /* Return len */ + return len; +} + +/** Format <b>child_state</b> and <b>saved_errno</b> as a hex string placed in + * <b>hex_errno</b>. Called between fork and _exit, so must be signal-handler + * safe. + * + * <b>hex_errno</b> must have at least HEX_ERRNO_SIZE bytes available. + * + * The format of <b>hex_errno</b> is: "CHILD_STATE/ERRNO\n", left-padded + * with spaces. Note that there is no trailing \0. CHILD_STATE indicates where + * in the processs of starting the child process did the failure occur (see + * CHILD_STATE_* macros for definition), and SAVED_ERRNO is the value of + * errno when the failure occurred. + * + * On success return the number of characters added to hex_errno, not counting + * the terminating NUL; return -1 on error. + */ +int +format_helper_exit_status(unsigned char child_state, int saved_errno, + char *hex_errno) +{ + unsigned int unsigned_errno; + int written, left; + char *cur; + size_t i; + int res = -1; + + /* Fill hex_errno with spaces, and a trailing newline (memset may + not be signal handler safe, so we can't use it) */ + for (i = 0; i < (HEX_ERRNO_SIZE - 1); i++) + hex_errno[i] = ' '; + hex_errno[HEX_ERRNO_SIZE - 1] = '\n'; + + /* Convert errno to be unsigned for hex conversion */ + if (saved_errno < 0) { + unsigned_errno = (unsigned int) -saved_errno; + } else { + unsigned_errno = (unsigned int) saved_errno; + } + + /* + * Count how many chars of space we have left, and keep a pointer into the + * current point in the buffer. + */ + left = HEX_ERRNO_SIZE; + cur = hex_errno; + + /* Emit child_state */ + written = format_hex_number_for_helper_exit_status(child_state, + cur, left); + if (written <= 0) + goto err; + + /* Adjust left and cur */ + left -= written; + cur += written; + if (left <= 0) + goto err; + + /* Now the '/' */ + *cur = '/'; + + /* Adjust left and cur */ + ++cur; + --left; + if (left <= 0) + goto err; + + /* Need minus? */ + if (saved_errno < 0) { + *cur = '-'; + ++cur; + --left; + if (left <= 0) + goto err; + } + + /* Emit unsigned_errno */ + written = format_hex_number_for_helper_exit_status(unsigned_errno, + cur, left); + + if (written <= 0) + goto err; + + /* Adjust left and cur */ + left -= written; + cur += written; + + /* Check that we have enough space left for a newline */ + if (left <= 0) + goto err; + + /* Emit the newline and NUL */ + *cur++ = '\n'; + *cur++ = '\0'; + + res = (int)(cur - hex_errno - 1); + + goto done; + + err: + /* + * In error exit, just write a '\0' in the first char so whatever called + * this at least won't fall off the end. + */ + *hex_errno = '\0'; + + done: + return res; +} + +/* Maximum number of file descriptors, if we cannot get it via sysconf() */ +#define DEFAULT_MAX_FD 256 + +/** Terminate the process of <b>process_handle</b>. + * Code borrowed from Python's os.kill. */ +int +tor_terminate_process(process_handle_t *process_handle) +{ +#ifdef _WIN32 + if (tor_get_exit_code(process_handle, 0, NULL) == PROCESS_EXIT_RUNNING) { + HANDLE handle; + /* If the signal is outside of what GenerateConsoleCtrlEvent can use, + attempt to open and terminate the process. */ + handle = OpenProcess(PROCESS_ALL_ACCESS, FALSE, + process_handle->pid.dwProcessId); + if (!handle) + return -1; + + if (!TerminateProcess(handle, 0)) + return -1; + else + return 0; + } +#else /* Unix */ + return kill(process_handle->pid, SIGTERM); +#endif + + return -1; +} + +/** Return the Process ID of <b>process_handle</b>. */ +int +tor_process_get_pid(process_handle_t *process_handle) +{ +#ifdef _WIN32 + return (int) process_handle->pid.dwProcessId; +#else + return (int) process_handle->pid; +#endif +} + +#ifdef _WIN32 +HANDLE +tor_process_get_stdout_pipe(process_handle_t *process_handle) +{ + return process_handle->stdout_pipe; +} +#else +/* DOCDOC tor_process_get_stdout_pipe */ +FILE * +tor_process_get_stdout_pipe(process_handle_t *process_handle) +{ + return process_handle->stdout_handle; +} +#endif + +/* DOCDOC process_handle_new */ +static process_handle_t * +process_handle_new(void) +{ + process_handle_t *out = tor_malloc_zero(sizeof(process_handle_t)); + +#ifdef _WIN32 + out->stdout_pipe = INVALID_HANDLE_VALUE; + out->stderr_pipe = INVALID_HANDLE_VALUE; +#else + out->stdout_pipe = -1; + out->stderr_pipe = -1; +#endif + + return out; +} + +/** + * @name child-process states + * + * Each of these values represents a possible state that a child process can + * be in. They're used to determine what to say when telling the parent how + * far along we were before failure. + * + * @{ + */ +#define CHILD_STATE_INIT 0 +#define CHILD_STATE_PIPE 1 +#define CHILD_STATE_MAXFD 2 +#define CHILD_STATE_FORK 3 +#define CHILD_STATE_DUPOUT 4 +#define CHILD_STATE_DUPERR 5 +#define CHILD_STATE_REDIRECT 6 +#define CHILD_STATE_CLOSEFD 7 +#define CHILD_STATE_EXEC 8 +#define CHILD_STATE_FAILEXEC 9 +/** @} */ +/** Start a program in the background. If <b>filename</b> contains a '/', then + * it will be treated as an absolute or relative path. Otherwise, on + * non-Windows systems, the system path will be searched for <b>filename</b>. + * On Windows, only the current directory will be searched. Here, to search the + * system path (as well as the application directory, current working + * directory, and system directories), set filename to NULL. + * + * The strings in <b>argv</b> will be passed as the command line arguments of + * the child program (following convention, argv[0] should normally be the + * filename of the executable, and this must be the case if <b>filename</b> is + * NULL). The last element of argv must be NULL. A handle to the child process + * will be returned in process_handle (which must be non-NULL). Read + * process_handle.status to find out if the process was successfully launched. + * For convenience, process_handle.status is returned by this function. + * + * Some parts of this code are based on the POSIX subprocess module from + * Python, and example code from + * http://msdn.microsoft.com/en-us/library/ms682499%28v=vs.85%29.aspx. + */ +int +tor_spawn_background(const char *const filename, const char **argv, + process_environment_t *env, + process_handle_t **process_handle_out) +{ +#ifdef _WIN32 + HANDLE stdout_pipe_read = NULL; + HANDLE stdout_pipe_write = NULL; + HANDLE stderr_pipe_read = NULL; + HANDLE stderr_pipe_write = NULL; + process_handle_t *process_handle; + int status; + + STARTUPINFOA siStartInfo; + BOOL retval = FALSE; + + SECURITY_ATTRIBUTES saAttr; + char *joined_argv; + + saAttr.nLength = sizeof(SECURITY_ATTRIBUTES); + saAttr.bInheritHandle = TRUE; + /* TODO: should we set explicit security attributes? (#2046, comment 5) */ + saAttr.lpSecurityDescriptor = NULL; + + /* Assume failure to start process */ + status = PROCESS_STATUS_ERROR; + + /* Set up pipe for stdout */ + if (!CreatePipe(&stdout_pipe_read, &stdout_pipe_write, &saAttr, 0)) { + log_warn(LD_GENERAL, + "Failed to create pipe for stdout communication with child process: %s", + format_win32_error(GetLastError())); + return status; + } + if (!SetHandleInformation(stdout_pipe_read, HANDLE_FLAG_INHERIT, 0)) { + log_warn(LD_GENERAL, + "Failed to configure pipe for stdout communication with child " + "process: %s", format_win32_error(GetLastError())); + return status; + } + + /* Set up pipe for stderr */ + if (!CreatePipe(&stderr_pipe_read, &stderr_pipe_write, &saAttr, 0)) { + log_warn(LD_GENERAL, + "Failed to create pipe for stderr communication with child process: %s", + format_win32_error(GetLastError())); + return status; + } + if (!SetHandleInformation(stderr_pipe_read, HANDLE_FLAG_INHERIT, 0)) { + log_warn(LD_GENERAL, + "Failed to configure pipe for stderr communication with child " + "process: %s", format_win32_error(GetLastError())); + return status; + } + + /* Create the child process */ + + /* Windows expects argv to be a whitespace delimited string, so join argv up + */ + joined_argv = tor_join_win_cmdline(argv); + + process_handle = process_handle_new(); + process_handle->status = status; + + ZeroMemory(&(process_handle->pid), sizeof(PROCESS_INFORMATION)); + ZeroMemory(&siStartInfo, sizeof(STARTUPINFO)); + siStartInfo.cb = sizeof(STARTUPINFO); + siStartInfo.hStdError = stderr_pipe_write; + siStartInfo.hStdOutput = stdout_pipe_write; + siStartInfo.hStdInput = NULL; + siStartInfo.dwFlags |= STARTF_USESTDHANDLES; + + /* Create the child process */ + + retval = CreateProcessA(filename, // module name + joined_argv, // command line + /* TODO: should we set explicit security attributes? (#2046, comment 5) */ + NULL, // process security attributes + NULL, // primary thread security attributes + TRUE, // handles are inherited + /*(TODO: set CREATE_NEW CONSOLE/PROCESS_GROUP to make GetExitCodeProcess() + * work?) */ + 0, // creation flags + (env==NULL) ? NULL : env->windows_environment_block, + NULL, // use parent's current directory + &siStartInfo, // STARTUPINFO pointer + &(process_handle->pid)); // receives PROCESS_INFORMATION + + tor_free(joined_argv); + + if (!retval) { + log_warn(LD_GENERAL, + "Failed to create child process %s: %s", filename?filename:argv[0], + format_win32_error(GetLastError())); + tor_free(process_handle); + } else { + /* TODO: Close hProcess and hThread in process_handle->pid? */ + process_handle->stdout_pipe = stdout_pipe_read; + process_handle->stderr_pipe = stderr_pipe_read; + status = process_handle->status = PROCESS_STATUS_RUNNING; + } + + /* TODO: Close pipes on exit */ + *process_handle_out = process_handle; + return status; +#else // _WIN32 + pid_t pid; + int stdout_pipe[2]; + int stderr_pipe[2]; + int fd, retval; + ssize_t nbytes; + process_handle_t *process_handle; + int status; + + const char *error_message = SPAWN_ERROR_MESSAGE; + size_t error_message_length; + + /* Represents where in the process of spawning the program is; + this is used for printing out the error message */ + unsigned char child_state = CHILD_STATE_INIT; + + char hex_errno[HEX_ERRNO_SIZE]; + + static int max_fd = -1; + + status = PROCESS_STATUS_ERROR; + + /* We do the strlen here because strlen() is not signal handler safe, + and we are not allowed to use unsafe functions between fork and exec */ + error_message_length = strlen(error_message); + + child_state = CHILD_STATE_PIPE; + + /* Set up pipe for redirecting stdout and stderr of child */ + retval = pipe(stdout_pipe); + if (-1 == retval) { + log_warn(LD_GENERAL, + "Failed to set up pipe for stdout communication with child process: %s", + strerror(errno)); + return status; + } + + retval = pipe(stderr_pipe); + if (-1 == retval) { + log_warn(LD_GENERAL, + "Failed to set up pipe for stderr communication with child process: %s", + strerror(errno)); + + close(stdout_pipe[0]); + close(stdout_pipe[1]); + + return status; + } + + child_state = CHILD_STATE_MAXFD; + +#ifdef _SC_OPEN_MAX + if (-1 != max_fd) { + max_fd = (int) sysconf(_SC_OPEN_MAX); + if (max_fd == -1) + max_fd = DEFAULT_MAX_FD; + log_warn(LD_GENERAL, + "Cannot find maximum file descriptor, assuming %d", max_fd); + } +#else + max_fd = DEFAULT_MAX_FD; +#endif + + child_state = CHILD_STATE_FORK; + + pid = fork(); + if (0 == pid) { + /* In child */ + + child_state = CHILD_STATE_DUPOUT; + + /* Link child stdout to the write end of the pipe */ + retval = dup2(stdout_pipe[1], STDOUT_FILENO); + if (-1 == retval) + goto error; + + child_state = CHILD_STATE_DUPERR; + + /* Link child stderr to the write end of the pipe */ + retval = dup2(stderr_pipe[1], STDERR_FILENO); + if (-1 == retval) + goto error; + + child_state = CHILD_STATE_REDIRECT; + + /* Link stdin to /dev/null */ + fd = open("/dev/null", O_RDONLY); /* NOT cloexec, obviously. */ + if (fd != -1) + dup2(fd, STDIN_FILENO); + else + goto error; + + child_state = CHILD_STATE_CLOSEFD; + + close(stderr_pipe[0]); + close(stderr_pipe[1]); + close(stdout_pipe[0]); + close(stdout_pipe[1]); + close(fd); + + /* Close all other fds, including the read end of the pipe */ + /* XXX: We should now be doing enough FD_CLOEXEC setting to make + * this needless. */ + for (fd = STDERR_FILENO + 1; fd < max_fd; fd++) { + close(fd); + } + + child_state = CHILD_STATE_EXEC; + + /* Call the requested program. We need the cast because + execvp doesn't define argv as const, even though it + does not modify the arguments */ + if (env) + execve(filename, (char *const *) argv, env->unixoid_environment_block); + else + execvp(filename, (char *const *) argv); + + /* If we got here, the exec or open(/dev/null) failed */ + + child_state = CHILD_STATE_FAILEXEC; + + error: + { + /* XXX: are we leaking fds from the pipe? */ + int n; + + n = format_helper_exit_status(child_state, errno, hex_errno); + + if (n >= 0) { + /* Write the error message. GCC requires that we check the return + value, but there is nothing we can do if it fails */ + /* TODO: Don't use STDOUT, use a pipe set up just for this purpose */ + nbytes = write(STDOUT_FILENO, error_message, error_message_length); + nbytes = write(STDOUT_FILENO, hex_errno, n); + } + } + + (void) nbytes; + + _exit(255); + /* Never reached, but avoids compiler warning */ + return status; + } + + /* In parent */ + + if (-1 == pid) { + log_warn(LD_GENERAL, "Failed to fork child process: %s", strerror(errno)); + close(stdout_pipe[0]); + close(stdout_pipe[1]); + close(stderr_pipe[0]); + close(stderr_pipe[1]); + return status; + } + + process_handle = process_handle_new(); + process_handle->status = status; + process_handle->pid = pid; + + /* TODO: If the child process forked but failed to exec, waitpid it */ + + /* Return read end of the pipes to caller, and close write end */ + process_handle->stdout_pipe = stdout_pipe[0]; + retval = close(stdout_pipe[1]); + + if (-1 == retval) { + log_warn(LD_GENERAL, + "Failed to close write end of stdout pipe in parent process: %s", + strerror(errno)); + } + + process_handle->stderr_pipe = stderr_pipe[0]; + retval = close(stderr_pipe[1]); + + if (-1 == retval) { + log_warn(LD_GENERAL, + "Failed to close write end of stderr pipe in parent process: %s", + strerror(errno)); + } + + status = process_handle->status = PROCESS_STATUS_RUNNING; + /* Set stdout/stderr pipes to be non-blocking */ + fcntl(process_handle->stdout_pipe, F_SETFL, O_NONBLOCK); + fcntl(process_handle->stderr_pipe, F_SETFL, O_NONBLOCK); + /* Open the buffered IO streams */ + process_handle->stdout_handle = fdopen(process_handle->stdout_pipe, "r"); + process_handle->stderr_handle = fdopen(process_handle->stderr_pipe, "r"); + + *process_handle_out = process_handle; + return process_handle->status; +#endif // _WIN32 +} + +/** Destroy all resources allocated by the process handle in + * <b>process_handle</b>. + * If <b>also_terminate_process</b> is true, also terminate the + * process of the process handle. */ +void +tor_process_handle_destroy(process_handle_t *process_handle, + int also_terminate_process) +{ + if (!process_handle) + return; + + if (also_terminate_process) { + if (tor_terminate_process(process_handle) < 0) { + log_notice(LD_GENERAL, "Failed to terminate process with PID '%d'", + tor_process_get_pid(process_handle)); + } else { + log_info(LD_GENERAL, "Terminated process with PID '%d'", + tor_process_get_pid(process_handle)); + } + } + + process_handle->status = PROCESS_STATUS_NOTRUNNING; + +#ifdef _WIN32 + if (process_handle->stdout_pipe) + CloseHandle(process_handle->stdout_pipe); + + if (process_handle->stderr_pipe) + CloseHandle(process_handle->stderr_pipe); +#else + if (process_handle->stdout_handle) + fclose(process_handle->stdout_handle); + + if (process_handle->stderr_handle) + fclose(process_handle->stderr_handle); +#endif + + memset(process_handle, 0x0f, sizeof(process_handle_t)); + tor_free(process_handle); +} + +/** Get the exit code of a process specified by <b>process_handle</b> and store + * it in <b>exit_code</b>, if set to a non-NULL value. If <b>block</b> is set + * to true, the call will block until the process has exited. Otherwise if + * the process is still running, the function will return + * PROCESS_EXIT_RUNNING, and exit_code will be left unchanged. Returns + * PROCESS_EXIT_EXITED if the process did exit. If there is a failure, + * PROCESS_EXIT_ERROR will be returned and the contents of exit_code (if + * non-NULL) will be undefined. N.B. Under *nix operating systems, this will + * probably not work in Tor, because waitpid() is called in main.c to reap any + * terminated child processes.*/ +int +tor_get_exit_code(const process_handle_t *process_handle, + int block, int *exit_code) +{ +#ifdef _WIN32 + DWORD retval; + BOOL success; + + if (block) { + /* Wait for the process to exit */ + retval = WaitForSingleObject(process_handle->pid.hProcess, INFINITE); + if (retval != WAIT_OBJECT_0) { + log_warn(LD_GENERAL, "WaitForSingleObject() failed (%d): %s", + (int)retval, format_win32_error(GetLastError())); + return PROCESS_EXIT_ERROR; + } + } else { + retval = WaitForSingleObject(process_handle->pid.hProcess, 0); + if (WAIT_TIMEOUT == retval) { + /* Process has not exited */ + return PROCESS_EXIT_RUNNING; + } else if (retval != WAIT_OBJECT_0) { + log_warn(LD_GENERAL, "WaitForSingleObject() failed (%d): %s", + (int)retval, format_win32_error(GetLastError())); + return PROCESS_EXIT_ERROR; + } + } + + if (exit_code != NULL) { + success = GetExitCodeProcess(process_handle->pid.hProcess, + (PDWORD)exit_code); + if (!success) { + log_warn(LD_GENERAL, "GetExitCodeProcess() failed: %s", + format_win32_error(GetLastError())); + return PROCESS_EXIT_ERROR; + } + } +#else + int stat_loc; + int retval; + + retval = waitpid(process_handle->pid, &stat_loc, block?0:WNOHANG); + if (!block && 0 == retval) { + /* Process has not exited */ + return PROCESS_EXIT_RUNNING; + } else if (retval != process_handle->pid) { + log_warn(LD_GENERAL, "waitpid() failed for PID %d: %s", + process_handle->pid, strerror(errno)); + return PROCESS_EXIT_ERROR; + } + + if (!WIFEXITED(stat_loc)) { + log_warn(LD_GENERAL, "Process %d did not exit normally", + process_handle->pid); + return PROCESS_EXIT_ERROR; + } + + if (exit_code != NULL) + *exit_code = WEXITSTATUS(stat_loc); +#endif // _WIN32 + + return PROCESS_EXIT_EXITED; +} + +/** Helper: return the number of characters in <b>s</b> preceding the first + * occurrence of <b>ch</b>. If <b>ch</b> does not occur in <b>s</b>, return + * the length of <b>s</b>. Should be equivalent to strspn(s, "ch"). */ +static INLINE size_t +str_num_before(const char *s, char ch) +{ + const char *cp = strchr(s, ch); + if (cp) + return cp - s; + else + return strlen(s); +} + +/** Return non-zero iff getenv would consider <b>s1</b> and <b>s2</b> + * to have the same name as strings in a process's environment. */ +int +environment_variable_names_equal(const char *s1, const char *s2) +{ + size_t s1_name_len = str_num_before(s1, '='); + size_t s2_name_len = str_num_before(s2, '='); + + return (s1_name_len == s2_name_len && + tor_memeq(s1, s2, s1_name_len)); +} + +/** Free <b>env</b> (assuming it was produced by + * process_environment_make). */ +void +process_environment_free(process_environment_t *env) +{ + if (env == NULL) return; + + /* As both an optimization hack to reduce consing on Unixoid systems + * and a nice way to ensure that some otherwise-Windows-specific + * code will always get tested before changes to it get merged, the + * strings which env->unixoid_environment_block points to are packed + * into env->windows_environment_block. */ + tor_free(env->unixoid_environment_block); + tor_free(env->windows_environment_block); + + tor_free(env); +} + +/** Make a process_environment_t containing the environment variables + * specified in <b>env_vars</b> (as C strings of the form + * "NAME=VALUE"). */ +process_environment_t * +process_environment_make(struct smartlist_t *env_vars) +{ + process_environment_t *env = tor_malloc_zero(sizeof(process_environment_t)); + size_t n_env_vars = smartlist_len(env_vars); + size_t i; + size_t total_env_length; + smartlist_t *env_vars_sorted; + + tor_assert(n_env_vars + 1 != 0); + env->unixoid_environment_block = tor_calloc(n_env_vars + 1, sizeof(char *)); + /* env->unixoid_environment_block is already NULL-terminated, + * because we assume that NULL == 0 (and check that during compilation). */ + + total_env_length = 1; /* terminating NUL of terminating empty string */ + for (i = 0; i < n_env_vars; ++i) { + const char *s = smartlist_get(env_vars, i); + size_t slen = strlen(s); + + tor_assert(slen + 1 != 0); + tor_assert(slen + 1 < SIZE_MAX - total_env_length); + total_env_length += slen + 1; + } + + env->windows_environment_block = tor_malloc_zero(total_env_length); + /* env->windows_environment_block is already + * (NUL-terminated-empty-string)-terminated. */ + + /* Some versions of Windows supposedly require that environment + * blocks be sorted. Or maybe some Windows programs (or their + * runtime libraries) fail to look up strings in non-sorted + * environment blocks. + * + * Also, sorting strings makes it easy to find duplicate environment + * variables and environment-variable strings without an '=' on all + * OSes, and they can cause badness. Let's complain about those. */ + env_vars_sorted = smartlist_new(); + smartlist_add_all(env_vars_sorted, env_vars); + smartlist_sort_strings(env_vars_sorted); + + /* Now copy the strings into the environment blocks. */ + { + char *cp = env->windows_environment_block; + const char *prev_env_var = NULL; + + for (i = 0; i < n_env_vars; ++i) { + const char *s = smartlist_get(env_vars_sorted, i); + size_t slen = strlen(s); + size_t s_name_len = str_num_before(s, '='); + + if (s_name_len == slen) { + log_warn(LD_GENERAL, + "Preparing an environment containing a variable " + "without a value: %s", + s); + } + if (prev_env_var != NULL && + environment_variable_names_equal(s, prev_env_var)) { + log_warn(LD_GENERAL, + "Preparing an environment containing two variables " + "with the same name: %s and %s", + prev_env_var, s); + } + + prev_env_var = s; + + /* Actually copy the string into the environment. */ + memcpy(cp, s, slen+1); + env->unixoid_environment_block[i] = cp; + cp += slen+1; + } + + tor_assert(cp == env->windows_environment_block + total_env_length - 1); + } + + smartlist_free(env_vars_sorted); + + return env; +} + +/** Return a newly allocated smartlist containing every variable in + * this process's environment, as a NUL-terminated string of the form + * "NAME=VALUE". Note that on some/many/most/all OSes, the parent + * process can put strings not of that form in our environment; + * callers should try to not get crashed by that. + * + * The returned strings are heap-allocated, and must be freed by the + * caller. */ +struct smartlist_t * +get_current_process_environment_variables(void) +{ + smartlist_t *sl = smartlist_new(); + + char **environ_tmp; /* Not const char ** ? Really? */ + for (environ_tmp = get_environment(); *environ_tmp; ++environ_tmp) { + smartlist_add(sl, tor_strdup(*environ_tmp)); + } + + return sl; +} + +/** For each string s in <b>env_vars</b> such that + * environment_variable_names_equal(s, <b>new_var</b>), remove it; if + * <b>free_p</b> is non-zero, call <b>free_old</b>(s). If + * <b>new_var</b> contains '=', insert it into <b>env_vars</b>. */ +void +set_environment_variable_in_smartlist(struct smartlist_t *env_vars, + const char *new_var, + void (*free_old)(void*), + int free_p) +{ + SMARTLIST_FOREACH_BEGIN(env_vars, const char *, s) { + if (environment_variable_names_equal(s, new_var)) { + SMARTLIST_DEL_CURRENT(env_vars, s); + if (free_p) { + free_old((void *)s); + } + } + } SMARTLIST_FOREACH_END(s); + + if (strchr(new_var, '=') != NULL) { + smartlist_add(env_vars, (void *)new_var); + } +} + +#ifdef _WIN32 +/** Read from a handle <b>h</b> into <b>buf</b>, up to <b>count</b> bytes. If + * <b>hProcess</b> is NULL, the function will return immediately if there is + * nothing more to read. Otherwise <b>hProcess</b> should be set to the handle + * to the process owning the <b>h</b>. In this case, the function will exit + * only once the process has exited, or <b>count</b> bytes are read. Returns + * the number of bytes read, or -1 on error. */ +ssize_t +tor_read_all_handle(HANDLE h, char *buf, size_t count, + const process_handle_t *process) +{ + size_t numread = 0; + BOOL retval; + DWORD byte_count; + BOOL process_exited = FALSE; + + if (count > SIZE_T_CEILING || count > SSIZE_T_MAX) + return -1; + + while (numread != count) { + /* Check if there is anything to read */ + retval = PeekNamedPipe(h, NULL, 0, NULL, &byte_count, NULL); + if (!retval) { + log_warn(LD_GENERAL, + "Failed to peek from handle: %s", + format_win32_error(GetLastError())); + return -1; + } else if (0 == byte_count) { + /* Nothing available: process exited or it is busy */ + + /* Exit if we don't know whether the process is running */ + if (NULL == process) + break; + + /* The process exited and there's nothing left to read from it */ + if (process_exited) + break; + + /* If process is not running, check for output one more time in case + it wrote something after the peek was performed. Otherwise keep on + waiting for output */ + tor_assert(process != NULL); + byte_count = WaitForSingleObject(process->pid.hProcess, 0); + if (WAIT_TIMEOUT != byte_count) + process_exited = TRUE; + + continue; + } + + /* There is data to read; read it */ + retval = ReadFile(h, buf+numread, count-numread, &byte_count, NULL); + tor_assert(byte_count + numread <= count); + if (!retval) { + log_warn(LD_GENERAL, "Failed to read from handle: %s", + format_win32_error(GetLastError())); + return -1; + } else if (0 == byte_count) { + /* End of file */ + break; + } + numread += byte_count; + } + return (ssize_t)numread; +} +#else +/** Read from a handle <b>h</b> into <b>buf</b>, up to <b>count</b> bytes. If + * <b>process</b> is NULL, the function will return immediately if there is + * nothing more to read. Otherwise data will be read until end of file, or + * <b>count</b> bytes are read. Returns the number of bytes read, or -1 on + * error. Sets <b>eof</b> to true if <b>eof</b> is not NULL and the end of the + * file has been reached. */ +ssize_t +tor_read_all_handle(FILE *h, char *buf, size_t count, + const process_handle_t *process, + int *eof) +{ + size_t numread = 0; + char *retval; + + if (eof) + *eof = 0; + + if (count > SIZE_T_CEILING || count > SSIZE_T_MAX) + return -1; + + while (numread != count) { + /* Use fgets because that is what we use in log_from_pipe() */ + retval = fgets(buf+numread, (int)(count-numread), h); + if (NULL == retval) { + if (feof(h)) { + log_debug(LD_GENERAL, "fgets() reached end of file"); + if (eof) + *eof = 1; + break; + } else { + if (EAGAIN == errno) { + if (process) + continue; + else + break; + } else { + log_warn(LD_GENERAL, "fgets() from handle failed: %s", + strerror(errno)); + return -1; + } + } + } + tor_assert(retval != NULL); + tor_assert(strlen(retval) + numread <= count); + numread += strlen(retval); + } + + log_debug(LD_GENERAL, "fgets() read %d bytes from handle", (int)numread); + return (ssize_t)numread; +} +#endif + +/** Read from stdout of a process until the process exits. */ +ssize_t +tor_read_all_from_process_stdout(const process_handle_t *process_handle, + char *buf, size_t count) +{ +#ifdef _WIN32 + return tor_read_all_handle(process_handle->stdout_pipe, buf, count, + process_handle); +#else + return tor_read_all_handle(process_handle->stdout_handle, buf, count, + process_handle, NULL); +#endif +} + +/** Read from stdout of a process until the process exits. */ +ssize_t +tor_read_all_from_process_stderr(const process_handle_t *process_handle, + char *buf, size_t count) +{ +#ifdef _WIN32 + return tor_read_all_handle(process_handle->stderr_pipe, buf, count, + process_handle); +#else + return tor_read_all_handle(process_handle->stderr_handle, buf, count, + process_handle, NULL); +#endif +} + +/** Split buf into lines, and add to smartlist. The buffer <b>buf</b> will be + * modified. The resulting smartlist will consist of pointers to buf, so there + * is no need to free the contents of sl. <b>buf</b> must be a NUL-terminated + * string. <b>len</b> should be set to the length of the buffer excluding the + * NUL. Non-printable characters (including NUL) will be replaced with "." */ +int +tor_split_lines(smartlist_t *sl, char *buf, int len) +{ + /* Index in buf of the start of the current line */ + int start = 0; + /* Index in buf of the current character being processed */ + int cur = 0; + /* Are we currently in a line */ + char in_line = 0; + + /* Loop over string */ + while (cur < len) { + /* Loop until end of line or end of string */ + for (; cur < len; cur++) { + if (in_line) { + if ('\r' == buf[cur] || '\n' == buf[cur]) { + /* End of line */ + buf[cur] = '\0'; + /* Point cur to the next line */ + cur++; + /* Line starts at start and ends with a nul */ + break; + } else { + if (!TOR_ISPRINT(buf[cur])) + buf[cur] = '.'; + } + } else { + if ('\r' == buf[cur] || '\n' == buf[cur]) { + /* Skip leading vertical space */ + ; + } else { + in_line = 1; + start = cur; + if (!TOR_ISPRINT(buf[cur])) + buf[cur] = '.'; + } + } + } + /* We are at the end of the line or end of string. If in_line is true there + * is a line which starts at buf+start and ends at a NUL. cur points to + * the character after the NUL. */ + if (in_line) + smartlist_add(sl, (void *)(buf+start)); + in_line = 0; + } + return smartlist_len(sl); +} + +#ifdef _WIN32 +/** Read from stream, and send lines to log at the specified log level. + * Returns -1 if there is a error reading, and 0 otherwise. + * If the generated stream is flushed more often than on new lines, or + * a read exceeds 256 bytes, lines will be truncated. This should be fixed, + * along with the corresponding problem on *nix (see bug #2045). + */ +static int +log_from_handle(HANDLE *pipe, int severity) +{ + char buf[256]; + int pos; + smartlist_t *lines; + + pos = tor_read_all_handle(pipe, buf, sizeof(buf) - 1, NULL); + if (pos < 0) { + /* Error */ + log_warn(LD_GENERAL, "Failed to read data from subprocess"); + return -1; + } + + if (0 == pos) { + /* There's nothing to read (process is busy or has exited) */ + log_debug(LD_GENERAL, "Subprocess had nothing to say"); + return 0; + } + + /* End with a null even if there isn't a \r\n at the end */ + /* TODO: What if this is a partial line? */ + buf[pos] = '\0'; + log_debug(LD_GENERAL, "Subprocess had %d bytes to say", pos); + + /* Split up the buffer */ + lines = smartlist_new(); + tor_split_lines(lines, buf, pos); + + /* Log each line */ + SMARTLIST_FOREACH(lines, char *, line, + { + log_fn(severity, LD_GENERAL, "Port forwarding helper says: %s", line); + }); + smartlist_free(lines); + + return 0; +} + +#else + +/** Read from stream, and send lines to log at the specified log level. + * Returns 1 if stream is closed normally, -1 if there is a error reading, and + * 0 otherwise. Handles lines from tor-fw-helper and + * tor_spawn_background() specially. + */ +static int +log_from_pipe(FILE *stream, int severity, const char *executable, + int *child_status) +{ + char buf[256]; + enum stream_status r; + + for (;;) { + r = get_string_from_pipe(stream, buf, sizeof(buf) - 1); + + if (r == IO_STREAM_CLOSED) { + return 1; + } else if (r == IO_STREAM_EAGAIN) { + return 0; + } else if (r == IO_STREAM_TERM) { + return -1; + } + + tor_assert(r == IO_STREAM_OKAY); + + /* Check if buf starts with SPAWN_ERROR_MESSAGE */ + if (strcmpstart(buf, SPAWN_ERROR_MESSAGE) == 0) { + /* Parse error message */ + int retval, child_state, saved_errno; + retval = tor_sscanf(buf, SPAWN_ERROR_MESSAGE "%x/%x", + &child_state, &saved_errno); + if (retval == 2) { + log_warn(LD_GENERAL, + "Failed to start child process \"%s\" in state %d: %s", + executable, child_state, strerror(saved_errno)); + if (child_status) + *child_status = 1; + } else { + /* Failed to parse message from child process, log it as a + warning */ + log_warn(LD_GENERAL, + "Unexpected message from port forwarding helper \"%s\": %s", + executable, buf); + } + } else { + log_fn(severity, LD_GENERAL, "Port forwarding helper says: %s", buf); + } + } + + /* We should never get here */ + return -1; +} +#endif + +/** Reads from <b>stream</b> and stores input in <b>buf_out</b> making + * sure it's below <b>count</b> bytes. + * If the string has a trailing newline, we strip it off. + * + * This function is specifically created to handle input from managed + * proxies, according to the pluggable transports spec. Make sure it + * fits your needs before using it. + * + * Returns: + * IO_STREAM_CLOSED: If the stream is closed. + * IO_STREAM_EAGAIN: If there is nothing to read and we should check back + * later. + * IO_STREAM_TERM: If something is wrong with the stream. + * IO_STREAM_OKAY: If everything went okay and we got a string + * in <b>buf_out</b>. */ +enum stream_status +get_string_from_pipe(FILE *stream, char *buf_out, size_t count) +{ + char *retval; + size_t len; + + tor_assert(count <= INT_MAX); + + retval = fgets(buf_out, (int)count, stream); + + if (!retval) { + if (feof(stream)) { + /* Program has closed stream (probably it exited) */ + /* TODO: check error */ + return IO_STREAM_CLOSED; + } else { + if (EAGAIN == errno) { + /* Nothing more to read, try again next time */ + return IO_STREAM_EAGAIN; + } else { + /* There was a problem, abandon this child process */ + return IO_STREAM_TERM; + } + } + } else { + len = strlen(buf_out); + if (len == 0) { + /* this probably means we got a NUL at the start of the string. */ + return IO_STREAM_EAGAIN; + } + + if (buf_out[len - 1] == '\n') { + /* Remove the trailing newline */ + buf_out[len - 1] = '\0'; + } else { + /* No newline; check whether we overflowed the buffer */ + if (!feof(stream)) + log_info(LD_GENERAL, + "Line from stream was truncated: %s", buf_out); + /* TODO: What to do with this error? */ + } + + return IO_STREAM_OKAY; + } + + /* We should never get here */ + return IO_STREAM_TERM; +} + +/* DOCDOC tor_check_port_forwarding */ +void +tor_check_port_forwarding(const char *filename, int dir_port, int or_port, + time_t now) +{ +/* When fw-helper succeeds, how long do we wait until running it again */ +#define TIME_TO_EXEC_FWHELPER_SUCCESS 300 +/* When fw-helper failed to start, how long do we wait until running it again + */ +#define TIME_TO_EXEC_FWHELPER_FAIL 60 + + /* Static variables are initialized to zero, so child_handle.status=0 + * which corresponds to it not running on startup */ + static process_handle_t *child_handle=NULL; + + static time_t time_to_run_helper = 0; + int stdout_status, stderr_status, retval; + const char *argv[10]; + char s_dirport[6], s_orport[6]; + + tor_assert(filename); + + /* Set up command line for tor-fw-helper */ + snprintf(s_dirport, sizeof s_dirport, "%d", dir_port); + snprintf(s_orport, sizeof s_orport, "%d", or_port); + + /* TODO: Allow different internal and external ports */ + argv[0] = filename; + argv[1] = "--internal-or-port"; + argv[2] = s_orport; + argv[3] = "--external-or-port"; + argv[4] = s_orport; + argv[5] = "--internal-dir-port"; + argv[6] = s_dirport; + argv[7] = "--external-dir-port"; + argv[8] = s_dirport; + argv[9] = NULL; + + /* Start the child, if it is not already running */ + if ((!child_handle || child_handle->status != PROCESS_STATUS_RUNNING) && + time_to_run_helper < now) { + int status; + + /* Assume tor-fw-helper will succeed, start it later*/ + time_to_run_helper = now + TIME_TO_EXEC_FWHELPER_SUCCESS; + + if (child_handle) { + tor_process_handle_destroy(child_handle, 1); + child_handle = NULL; + } + +#ifdef _WIN32 + /* Passing NULL as lpApplicationName makes Windows search for the .exe */ + status = tor_spawn_background(NULL, argv, NULL, &child_handle); +#else + status = tor_spawn_background(filename, argv, NULL, &child_handle); +#endif + + if (PROCESS_STATUS_ERROR == status) { + log_warn(LD_GENERAL, "Failed to start port forwarding helper %s", + filename); + time_to_run_helper = now + TIME_TO_EXEC_FWHELPER_FAIL; + return; + } + + log_info(LD_GENERAL, + "Started port forwarding helper (%s) with pid '%d'", + filename, tor_process_get_pid(child_handle)); + } + + /* If child is running, read from its stdout and stderr) */ + if (child_handle && PROCESS_STATUS_RUNNING == child_handle->status) { + /* Read from stdout/stderr and log result */ + retval = 0; +#ifdef _WIN32 + stdout_status = log_from_handle(child_handle->stdout_pipe, LOG_INFO); + stderr_status = log_from_handle(child_handle->stderr_pipe, LOG_WARN); + /* If we got this far (on Windows), the process started */ + retval = 0; +#else + stdout_status = log_from_pipe(child_handle->stdout_handle, + LOG_INFO, filename, &retval); + stderr_status = log_from_pipe(child_handle->stderr_handle, + LOG_WARN, filename, &retval); +#endif + if (retval) { + /* There was a problem in the child process */ + time_to_run_helper = now + TIME_TO_EXEC_FWHELPER_FAIL; + } + + /* Combine the two statuses in order of severity */ + if (-1 == stdout_status || -1 == stderr_status) + /* There was a failure */ + retval = -1; +#ifdef _WIN32 + else if (!child_handle || tor_get_exit_code(child_handle, 0, NULL) != + PROCESS_EXIT_RUNNING) { + /* process has exited or there was an error */ + /* TODO: Do something with the process return value */ + /* TODO: What if the process output something since + * between log_from_handle and tor_get_exit_code? */ + retval = 1; + } +#else + else if (1 == stdout_status || 1 == stderr_status) + /* stdout or stderr was closed, the process probably + * exited. It will be reaped by waitpid() in main.c */ + /* TODO: Do something with the process return value */ + retval = 1; +#endif + else + /* Both are fine */ + retval = 0; + + /* If either pipe indicates a failure, act on it */ + if (0 != retval) { + if (1 == retval) { + log_info(LD_GENERAL, "Port forwarding helper terminated"); + child_handle->status = PROCESS_STATUS_NOTRUNNING; + } else { + log_warn(LD_GENERAL, "Failed to read from port forwarding helper"); + child_handle->status = PROCESS_STATUS_ERROR; + } + + /* TODO: The child might not actually be finished (maybe it failed or + closed stdout/stderr), so maybe we shouldn't start another? */ + } + } +} + diff --git a/src/common/util.h b/src/common/util.h index 75de3422d7..8977d273c5 100644 --- a/src/common/util.h +++ b/src/common/util.h @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -17,6 +17,10 @@ #include "di_ops.h" #include <stdio.h> #include <stdlib.h> +#ifdef _WIN32 +/* for the correct alias to struct stat */ +#include <sys/stat.h> +#endif #ifndef O_BINARY #define O_BINARY 0 @@ -73,6 +77,7 @@ void *_tor_malloc(size_t size DMALLOC_PARAMS) ATTR_MALLOC; void *_tor_malloc_zero(size_t size DMALLOC_PARAMS) ATTR_MALLOC; void *_tor_malloc_roundup(size_t *size DMALLOC_PARAMS) ATTR_MALLOC; +void *_tor_calloc(size_t nmemb, size_t size DMALLOC_PARAMS) ATTR_MALLOC; void *_tor_realloc(void *ptr, size_t size DMALLOC_PARAMS); char *_tor_strdup(const char *s DMALLOC_PARAMS) ATTR_MALLOC ATTR_NONNULL((1)); char *_tor_strndup(const char *s, size_t n DMALLOC_PARAMS) @@ -107,6 +112,7 @@ extern int dmalloc_free(const char *file, const int line, void *pnt, #define tor_malloc(size) _tor_malloc(size DMALLOC_ARGS) #define tor_malloc_zero(size) _tor_malloc_zero(size DMALLOC_ARGS) +#define tor_calloc(nmemb,size) _tor_calloc(nmemb, size DMALLOC_ARGS) #define tor_malloc_roundup(szp) _tor_malloc_roundup(szp DMALLOC_ARGS) #define tor_realloc(ptr, size) _tor_realloc(ptr, size DMALLOC_ARGS) #define tor_strdup(s) _tor_strdup(s DMALLOC_ARGS) @@ -160,6 +166,7 @@ uint64_t round_to_power_of_2(uint64_t u64); unsigned round_to_next_multiple_of(unsigned number, unsigned divisor); uint32_t round_uint32_to_next_multiple_of(uint32_t number, uint32_t divisor); uint64_t round_uint64_to_next_multiple_of(uint64_t number, uint64_t divisor); +int n_bits_set_u8(uint8_t v); /* Compute the CEIL of <b>a</b> divided by <b>b</b>, for nonnegative <b>a</b> * and positive <b>b</b>. Works on integer types only. Not defined if a+b can @@ -172,18 +179,15 @@ uint64_t round_uint64_to_next_multiple_of(uint64_t number, uint64_t divisor); #define HEX_CHARACTERS "0123456789ABCDEFabcdef" void tor_strlower(char *s) ATTR_NONNULL((1)); void tor_strupper(char *s) ATTR_NONNULL((1)); -int tor_strisprint(const char *s) ATTR_PURE ATTR_NONNULL((1)); -int tor_strisnonupper(const char *s) ATTR_PURE ATTR_NONNULL((1)); -int strcmpstart(const char *s1, const char *s2) ATTR_PURE ATTR_NONNULL((1,2)); -int strcmp_len(const char *s1, const char *s2, size_t len) - ATTR_PURE ATTR_NONNULL((1,2)); -int strcasecmpstart(const char *s1, const char *s2) - ATTR_PURE ATTR_NONNULL((1,2)); -int strcmpend(const char *s1, const char *s2) ATTR_PURE ATTR_NONNULL((1,2)); -int strcasecmpend(const char *s1, const char *s2) - ATTR_PURE ATTR_NONNULL((1,2)); -int fast_memcmpstart(const void *mem, size_t memlen, - const char *prefix) ATTR_PURE; +int tor_strisprint(const char *s) ATTR_NONNULL((1)); +int tor_strisnonupper(const char *s) ATTR_NONNULL((1)); +int strcmp_opt(const char *s1, const char *s2); +int strcmpstart(const char *s1, const char *s2) ATTR_NONNULL((1,2)); +int strcmp_len(const char *s1, const char *s2, size_t len) ATTR_NONNULL((1,2)); +int strcasecmpstart(const char *s1, const char *s2) ATTR_NONNULL((1,2)); +int strcmpend(const char *s1, const char *s2) ATTR_NONNULL((1,2)); +int strcasecmpend(const char *s1, const char *s2) ATTR_NONNULL((1,2)); +int fast_memcmpstart(const void *mem, size_t memlen, const char *prefix); void tor_strstrip(char *s, const char *strip) ATTR_NONNULL((1,2)); long tor_parse_long(const char *s, int base, long min, @@ -195,17 +199,19 @@ double tor_parse_double(const char *s, double min, double max, int *ok, uint64_t tor_parse_uint64(const char *s, int base, uint64_t min, uint64_t max, int *ok, char **next); const char *hex_str(const char *from, size_t fromlen) ATTR_NONNULL((1)); -const char *eat_whitespace(const char *s) ATTR_PURE; -const char *eat_whitespace_eos(const char *s, const char *eos) ATTR_PURE; -const char *eat_whitespace_no_nl(const char *s) ATTR_PURE; -const char *eat_whitespace_eos_no_nl(const char *s, const char *eos) ATTR_PURE; -const char *find_whitespace(const char *s) ATTR_PURE; -const char *find_whitespace_eos(const char *s, const char *eos) ATTR_PURE; -const char *find_str_at_start_of_line(const char *haystack, const char *needle) - ATTR_PURE; -int tor_mem_is_zero(const char *mem, size_t len) ATTR_PURE; -int tor_digest_is_zero(const char *digest) ATTR_PURE; -int tor_digest256_is_zero(const char *digest) ATTR_PURE; +const char *eat_whitespace(const char *s); +const char *eat_whitespace_eos(const char *s, const char *eos); +const char *eat_whitespace_no_nl(const char *s); +const char *eat_whitespace_eos_no_nl(const char *s, const char *eos); +const char *find_whitespace(const char *s); +const char *find_whitespace_eos(const char *s, const char *eos); +const char *find_str_at_start_of_line(const char *haystack, + const char *needle); +int string_is_C_identifier(const char *string); + +int tor_mem_is_zero(const char *mem, size_t len); +int tor_digest_is_zero(const char *digest); +int tor_digest256_is_zero(const char *digest); char *esc_for_log(const char *string) ATTR_MALLOC; const char *escaped(const char *string); struct smartlist_t; @@ -222,6 +228,12 @@ int tor_sscanf(const char *buf, const char *pattern, ...) #endif ; +void smartlist_add_asprintf(struct smartlist_t *sl, const char *pattern, ...) + CHECK_PRINTF(2, 3); +void smartlist_add_vasprintf(struct smartlist_t *sl, const char *pattern, + va_list args) + CHECK_PRINTF(2, 0); + int hex_decode_digit(char c); void base16_encode(char *dest, size_t destlen, const char *src, size_t srclen); int base16_decode(char *dest, size_t destlen, const char *src, size_t srclen); @@ -237,8 +249,11 @@ int tor_timegm(const struct tm *tm, time_t *time_out); void format_rfc1123_time(char *buf, time_t t); int parse_rfc1123_time(const char *buf, time_t *t); #define ISO_TIME_LEN 19 +#define ISO_TIME_USEC_LEN (ISO_TIME_LEN+7) void format_local_iso_time(char *buf, time_t t); void format_iso_time(char *buf, time_t t); +void format_iso_time_nospace(char *buf, time_t t); +void format_iso_time_nospace_usec(char *buf, const struct timeval *tv); int parse_iso_time(const char *buf, time_t *t); int parse_http_time(const char *buf, struct tm *tm); int format_time_interval(char *out, size_t out_len, long interval); @@ -283,6 +298,16 @@ char *rate_limit_log(ratelim_t *lim, time_t now); ssize_t write_all(tor_socket_t fd, const char *buf, size_t count,int isSocket); ssize_t read_all(tor_socket_t fd, char *buf, size_t count, int isSocket); +/** Status of an I/O stream. */ +enum stream_status { + IO_STREAM_OKAY, + IO_STREAM_EAGAIN, + IO_STREAM_TERM, + IO_STREAM_CLOSED +}; + +enum stream_status get_string_from_pipe(FILE *stream, char *buf, size_t count); + /** Return values from file_status(); see that function's documentation * for details. */ typedef enum { FN_ERROR, FN_NOENT, FN_FILE, FN_DIR } file_status_t; @@ -300,6 +325,7 @@ int check_private_dir(const char *dirname, cpd_check_t check, const char *effective_user); #define OPEN_FLAGS_REPLACE (O_WRONLY|O_CREAT|O_TRUNC) #define OPEN_FLAGS_APPEND (O_WRONLY|O_CREAT|O_APPEND) +#define OPEN_FLAGS_DONT_REPLACE (O_CREAT|O_EXCL|O_APPEND|O_WRONLY) typedef struct open_file_t open_file_t; int start_writing_to_file(const char *fname, int open_flags, int mode, open_file_t **data_out); @@ -321,30 +347,141 @@ int write_chunks_to_file(const char *fname, const struct smartlist_t *chunks, int bin); int append_bytes_to_file(const char *fname, const char *str, size_t len, int bin); +int write_bytes_to_new_file(const char *fname, const char *str, size_t len, + int bin); /** Flag for read_file_to_str: open the file in binary mode. */ #define RFTS_BIN 1 /** Flag for read_file_to_str: it's okay if the file doesn't exist. */ #define RFTS_IGNORE_MISSING 2 +#ifndef _WIN32 struct stat; +#endif char *read_file_to_str(const char *filename, int flags, struct stat *stat_out) ATTR_MALLOC; const char *parse_config_line_from_str(const char *line, char **key_out, char **value_out); char *expand_filename(const char *filename); struct smartlist_t *tor_listdir(const char *dirname); -int path_is_relative(const char *filename) ATTR_PURE; +int path_is_relative(const char *filename); /* Process helpers */ void start_daemon(void); void finish_daemon(const char *desired_cwd); void write_pidfile(char *filename); -#ifdef MS_WINDOWS +/* Port forwarding */ +void tor_check_port_forwarding(const char *filename, + int dir_port, int or_port, time_t now); + +typedef struct process_handle_t process_handle_t; +typedef struct process_environment_t process_environment_t; +int tor_spawn_background(const char *const filename, const char **argv, + process_environment_t *env, + process_handle_t **process_handle_out); + +#define SPAWN_ERROR_MESSAGE "ERR: Failed to spawn background process - code " + +#ifdef _WIN32 HANDLE load_windows_system_library(const TCHAR *library_name); #endif +int environment_variable_names_equal(const char *s1, const char *s2); + +/* DOCDOC process_environment_t */ +struct process_environment_t { + /** A pointer to a sorted empty-string-terminated sequence of + * NUL-terminated strings of the form "NAME=VALUE". */ + char *windows_environment_block; + /** A pointer to a NULL-terminated array of pointers to + * NUL-terminated strings of the form "NAME=VALUE". */ + char **unixoid_environment_block; +}; + +process_environment_t *process_environment_make(struct smartlist_t *env_vars); +void process_environment_free(process_environment_t *env); + +struct smartlist_t *get_current_process_environment_variables(void); + +void set_environment_variable_in_smartlist(struct smartlist_t *env_vars, + const char *new_var, + void (*free_old)(void*), + int free_p); + +/* Values of process_handle_t.status. PROCESS_STATUS_NOTRUNNING must be + * 0 because tor_check_port_forwarding depends on this being the initial + * statue of the static instance of process_handle_t */ +#define PROCESS_STATUS_NOTRUNNING 0 +#define PROCESS_STATUS_RUNNING 1 +#define PROCESS_STATUS_ERROR -1 + +#ifdef UTIL_PRIVATE +/** Structure to represent the state of a process with which Tor is + * communicating. The contents of this structure are private to util.c */ +struct process_handle_t { + /** One of the PROCESS_STATUS_* values */ + int status; +#ifdef _WIN32 + HANDLE stdout_pipe; + HANDLE stderr_pipe; + PROCESS_INFORMATION pid; +#else + int stdout_pipe; + int stderr_pipe; + FILE *stdout_handle; + FILE *stderr_handle; + pid_t pid; +#endif // _WIN32 +}; +#endif + +/* Return values of tor_get_exit_code() */ +#define PROCESS_EXIT_RUNNING 1 +#define PROCESS_EXIT_EXITED 0 +#define PROCESS_EXIT_ERROR -1 +int tor_get_exit_code(const process_handle_t *process_handle, + int block, int *exit_code); +int tor_split_lines(struct smartlist_t *sl, char *buf, int len); +#ifdef _WIN32 +ssize_t tor_read_all_handle(HANDLE h, char *buf, size_t count, + const process_handle_t *process); +#else +ssize_t tor_read_all_handle(FILE *h, char *buf, size_t count, + const process_handle_t *process, + int *eof); +#endif +ssize_t tor_read_all_from_process_stdout( + const process_handle_t *process_handle, char *buf, size_t count); +ssize_t tor_read_all_from_process_stderr( + const process_handle_t *process_handle, char *buf, size_t count); +char *tor_join_win_cmdline(const char *argv[]); + +int tor_process_get_pid(process_handle_t *process_handle); +#ifdef _WIN32 +HANDLE tor_process_get_stdout_pipe(process_handle_t *process_handle); +#else +FILE *tor_process_get_stdout_pipe(process_handle_t *process_handle); +#endif + +int tor_terminate_process(process_handle_t *process_handle); +void tor_process_handle_destroy(process_handle_t *process_handle, + int also_terminate_process); + +#ifdef UTIL_PRIVATE +/* Prototypes for private functions only used by util.c (and unit tests) */ + +int format_hex_number_for_helper_exit_status(unsigned int x, char *buf, + int max_len); +int format_helper_exit_status(unsigned char child_state, + int saved_errno, char *hex_errno); + +/* Space for hex values of child state, a slash, saved_errno (with + leading minus) and newline (no null) */ +#define HEX_ERRNO_SIZE (sizeof(char) * 2 + 1 + \ + 1 + sizeof(int) * 2 + 1) +#endif + const char *libor_get_digests(void); #endif diff --git a/src/common/util_codedigest.c b/src/common/util_codedigest.c index 88fe508b92..7384f7dc1a 100644 --- a/src/common/util_codedigest.c +++ b/src/common/util_codedigest.c @@ -1,6 +1,8 @@ #include "util.h" +/** Return a string describing the digest of the source files in src/common/ + */ const char * libor_get_digests(void) { diff --git a/src/config/torrc.bridge.in b/src/config/torrc.bridge.in deleted file mode 100644 index 557b7adf46..0000000000 --- a/src/config/torrc.bridge.in +++ /dev/null @@ -1,171 +0,0 @@ -## Configuration file for a typical Tor user -## Last updated 16 July 2009 for Tor 0.2.2.1-alpha. -## (May or may not work for much older or much newer versions of Tor.) -## -## Lines that begin with "## " try to explain what's going on. Lines -## that begin with just "#" are disabled commands: you can enable them -## by removing the "#" symbol. -## -## See 'man tor', or https://www.torproject.org/tor-manual.html, -## for more options you can use in this file. -## -## Tor will look for this file in various places based on your platform: -## https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#torrc - - -## Replace this with "SocksPort 0" if you plan to run Tor only as a -## relay, and not make any local application connections yourself. -SocksPort 9050 # what port to open for local application connections -SocksListenAddress 127.0.0.1 # accept connections only from localhost -#SocksListenAddress 192.168.0.1:9100 # listen on this IP:port also - -## Entry policies to allow/deny SOCKS requests based on IP address. -## First entry that matches wins. If no SocksPolicy is set, we accept -## all (and only) requests from SocksListenAddress. -#SocksPolicy accept 192.168.0.0/16 -#SocksPolicy reject * - -## Logs go to stdout at level "notice" unless redirected by something -## else, like one of the below lines. You can have as many Log lines as -## you want. -## -## We advise using "notice" in most cases, since anything more verbose -## may provide sensitive information to an attacker who obtains the logs. -## -## Send all messages of level 'notice' or higher to @LOCALSTATEDIR@/log/tor/notices.log -#Log notice file @LOCALSTATEDIR@/log/tor/notices.log -## Send every possible message to @LOCALSTATEDIR@/log/tor/debug.log -#Log debug file @LOCALSTATEDIR@/log/tor/debug.log -## Use the system log instead of Tor's logfiles -#Log notice syslog -## To send all messages to stderr: -#Log debug stderr - -## Uncomment this to start the process in the background... or use -## --runasdaemon 1 on the command line. This is ignored on Windows; -## see the FAQ entry if you want Tor to run as an NT service. -#RunAsDaemon 1 - -## The directory for keeping all the keys/etc. By default, we store -## things in $HOME/.tor on Unix, and in Application Data\tor on Windows. -#DataDirectory @LOCALSTATEDIR@/lib/tor - -## The port on which Tor will listen for local connections from Tor -## controller applications, as documented in control-spec.txt. -#ControlPort 9051 -## If you enable the controlport, be sure to enable one of these -## authentication methods, to prevent attackers from accessing it. -#HashedControlPassword 16:872860B76453A77D60CA2BB8C1A7042072093276A3D701AD684053EC4C -#CookieAuthentication 1 - -############### This section is just for location-hidden services ### - -## Once you have configured a hidden service, you can look at the -## contents of the file ".../hidden_service/hostname" for the address -## to tell people. -## -## HiddenServicePort x y:z says to redirect requests on port x to the -## address y:z. - -#HiddenServiceDir @LOCALSTATEDIR@/lib/tor/hidden_service/ -#HiddenServicePort 80 127.0.0.1:80 - -#HiddenServiceDir @LOCALSTATEDIR@/lib/tor/other_hidden_service/ -#HiddenServicePort 80 127.0.0.1:80 -#HiddenServicePort 22 127.0.0.1:22 - -################ This section is just for relays ##################### -# -## See https://www.torproject.org/docs/tor-doc-relay for details. - -## Required: what port to advertise for incoming Tor connections. -ORPort 9001 -## If you want to listen on a port other than the one advertised -## in ORPort (e.g. to advertise 443 but bind to 9090), uncomment the -## line below too. You'll need to do ipchains or other port forwarding -## yourself to make this work. -#ORListenAddress 0.0.0.0:9090 - -## A handle for your relay, so people don't have to refer to it by key. -Nickname Unnamed - -## The IP address or full DNS name for your relay. Leave commented out -## and Tor will guess. -#Address noname.example.com - -## Define these to limit how much relayed traffic you will allow. Your -## own traffic is still unthrottled. Note that RelayBandwidthRate must -## be at least 20 KB. -#RelayBandwidthRate 100 KB # Throttle traffic to 100KB/s (800Kbps) -#RelayBandwidthBurst 200 KB # But allow bursts up to 200KB/s (1600Kbps) -RelayBandwidthBurst 10485760 -RelayBandwidthRate 5242880 - -## Use these to restrict the maximum traffic per day, week, or month. -## Note that this threshold applies to sent _and_ to received bytes, -## not to their sum: Setting "4 GB" may allow up to 8 GB -## total before hibernating. -## -## Set a maximum of 4 gigabytes each way per period. -#AccountingMax 4 GB -## Each period starts daily at midnight (AccountingMax is per day) -#AccountingStart day 00:00 -## Each period starts on the 3rd of the month at 15:00 (AccountingMax -## is per month) -#AccountingStart month 3 15:00 - -## Contact info to be published in the directory, so we can contact you -## if your relay is misconfigured or something else goes wrong. Google -## indexes this, so spammers might also collect it. -#ContactInfo Random Person <nobody AT example dot com> -## You might also include your PGP or GPG fingerprint if you have one: -#ContactInfo 1234D/FFFFFFFF Random Person <nobody AT example dot com> - -## Uncomment this to mirror directory information for others. Please do -## if you have enough bandwidth. -DirPort 9030 # what port to advertise for directory connections -## If you want to listen on a port other than the one advertised -## in DirPort (e.g. to advertise 80 but bind to 9091), uncomment the line -## below too. You'll need to do ipchains or other port forwarding yourself -## to make this work. -#DirListenAddress 0.0.0.0:9091 -## Uncomment to return an arbitrary blob of html on your DirPort. Now you -## can explain what Tor is if anybody wonders why your IP address is -## contacting them. See contrib/tor-exit-notice.html in Tor's source -## distribution for a sample. -#DirPortFrontPage @CONFDIR@/tor-exit-notice.html - -## Uncomment this if you run more than one Tor relay, and add the identity -## key fingerprint of each Tor relay you control, even if they're on -## different networks. You declare it here so Tor clients can avoid -## using more than one of your relays in a single circuit. See -## https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#MultipleServers -#MyFamily $keyid,$keyid,... - -## A comma-separated list of exit policies. They're considered first -## to last, and the first match wins. If you want to _replace_ -## the default exit policy, end this with either a reject *:* or an -## accept *:*. Otherwise, you're _augmenting_ (prepending to) the -## default exit policy. Leave commented to just use the default, which is -## described in the man page or at -## https://www.torproject.org/documentation.html -## -## Look at https://www.torproject.org/faq-abuse.html#TypicalAbuses -## for issues you might encounter if you use the default exit policy. -## -## If certain IPs and ports are blocked externally, e.g. by your firewall, -## you should update your exit policy to reflect this -- otherwise Tor -## users will be told that those destinations are down. -## -#ExitPolicy accept *:6660-6667,reject *:* # allow irc ports but no more -#ExitPolicy accept *:119 # accept nntp as well as default exit policy -#ExitPolicy reject *:* # no exits allowed -# -## Bridge relays (or "bridges") are Tor relays that aren't listed in the -## main directory. Since there is no complete public list of them, even if an -## ISP is filtering connections to all the known Tor relays, they probably -## won't be able to block all the bridges. Also, websites won't treat you -## differently because they won't know you're running Tor. If you can -## be a real relay, please do; but if not, be a bridge! -BridgeRelay 1 -ExitPolicy reject *:* diff --git a/src/config/torrc.sample.in b/src/config/torrc.sample.in index f0c78ce5a9..a1a08aa8f9 100644 --- a/src/config/torrc.sample.in +++ b/src/config/torrc.sample.in @@ -1,27 +1,28 @@ ## Configuration file for a typical Tor user -## Last updated 16 July 2009 for Tor 0.2.2.1-alpha. -## (May or may not work for much older or much newer versions of Tor.) +## Last updated 22 April 2012 for Tor 0.2.3.14-alpha. +## (may or may not work for much older or much newer versions of Tor.) ## ## Lines that begin with "## " try to explain what's going on. Lines ## that begin with just "#" are disabled commands: you can enable them ## by removing the "#" symbol. ## -## See 'man tor', or https://www.torproject.org/tor-manual.html, +## See 'man tor', or https://www.torproject.org/docs/tor-manual.html, ## for more options you can use in this file. ## ## Tor will look for this file in various places based on your platform: -## https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#torrc +## https://www.torproject.org/docs/faq#torrc - -## Replace this with "SocksPort 0" if you plan to run Tor only as a -## relay, and not make any local application connections yourself. -SocksPort 9050 # what port to open for local application connections -SocksListenAddress 127.0.0.1 # accept connections only from localhost -#SocksListenAddress 192.168.0.1:9100 # listen on this IP:port also +## Tor opens a socks proxy on port 9050 by default -- even if you don't +## configure one below. Set "SocksPort 0" if you plan to run Tor only +## as a relay, and not make any local application connections yourself. +#SocksPort 9050 # Default: Bind to localhost:9050 for local connections. +#SocksPort 192.168.0.1:9100 # Bind to this adddress:port too. ## Entry policies to allow/deny SOCKS requests based on IP address. ## First entry that matches wins. If no SocksPolicy is set, we accept -## all (and only) requests from SocksListenAddress. +## all (and only) requests that reach a SocksPort. Untrusted users who +## can access your SocksPort may be able to learn about the connections +## you make. #SocksPolicy accept 192.168.0.0/16 #SocksPolicy reject * @@ -80,29 +81,36 @@ SocksListenAddress 127.0.0.1 # accept connections only from localhost ## Required: what port to advertise for incoming Tor connections. #ORPort 9001 -## If you want to listen on a port other than the one advertised -## in ORPort (e.g. to advertise 443 but bind to 9090), uncomment the -## line below too. You'll need to do ipchains or other port forwarding +## If you want to listen on a port other than the one advertised in +## ORPort (e.g. to advertise 443 but bind to 9090), you can do it as +## follows. You'll need to do ipchains or other port forwarding ## yourself to make this work. -#ORListenAddress 0.0.0.0:9090 +#ORPort 443 NoListen +#ORPort 127.0.0.1:9090 NoAdvertise + +## The IP address or full DNS name for incoming connections to your +## relay. Leave commented out and Tor will guess. +#Address noname.example.com + +## If you have multiple network interfaces, you can specify one for +## outgoing traffic to use. +# OutboundBindAddress 10.0.0.5 ## A handle for your relay, so people don't have to refer to it by key. #Nickname ididnteditheconfig -## The IP address or full DNS name for your relay. Leave commented out -## and Tor will guess. -#Address noname.example.com - ## Define these to limit how much relayed traffic you will allow. Your ## own traffic is still unthrottled. Note that RelayBandwidthRate must ## be at least 20 KB. +## Note that units for these config options are bytes per second, not bits +## per second, and that prefixes are binary prefixes, i.e. 2^10, 2^20, etc. #RelayBandwidthRate 100 KB # Throttle traffic to 100KB/s (800Kbps) #RelayBandwidthBurst 200 KB # But allow bursts up to 200KB/s (1600Kbps) ## Use these to restrict the maximum traffic per day, week, or month. -## Note that this threshold applies to sent _and_ to received bytes, -## not to their sum: Setting "4 GB" may allow up to 8 GB -## total before hibernating. +## Note that this threshold applies separately to sent and received bytes, +## not to their sum: setting "4 GB" may allow up to 8 GB total before +## hibernating. ## ## Set a maximum of 4 gigabytes each way per period. #AccountingMax 4 GB @@ -117,16 +125,17 @@ SocksListenAddress 127.0.0.1 # accept connections only from localhost ## indexes this, so spammers might also collect it. #ContactInfo Random Person <nobody AT example dot com> ## You might also include your PGP or GPG fingerprint if you have one: -#ContactInfo 1234D/FFFFFFFF Random Person <nobody AT example dot com> +#ContactInfo 0xFFFFFFFF Random Person <nobody AT example dot com> ## Uncomment this to mirror directory information for others. Please do ## if you have enough bandwidth. #DirPort 9030 # what port to advertise for directory connections -## If you want to listen on a port other than the one advertised -## in DirPort (e.g. to advertise 80 but bind to 9091), uncomment the line -## below too. You'll need to do ipchains or other port forwarding yourself -## to make this work. -#DirListenAddress 0.0.0.0:9091 +## If you want to listen on a port other than the one advertised in +## DirPort (e.g. to advertise 80 but bind to 9091), you can do it as +## follows. below too. You'll need to do ipchains or other port +## forwarding yourself to make this work. +#DirPort 80 NoListen +#DirPort 127.0.0.1:9091 NoAdvertise ## Uncomment to return an arbitrary blob of html on your DirPort. Now you ## can explain what Tor is if anybody wonders why your IP address is ## contacting them. See contrib/tor-exit-notice.html in Tor's source @@ -137,7 +146,9 @@ SocksListenAddress 127.0.0.1 # accept connections only from localhost ## key fingerprint of each Tor relay you control, even if they're on ## different networks. You declare it here so Tor clients can avoid ## using more than one of your relays in a single circuit. See -## https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#MultipleServers +## https://www.torproject.org/docs/faq#MultipleRelays +## However, you should never include a bridge's fingerprint here, as it would +## break its concealability and potentionally reveal its IP/TCP address. #MyFamily $keyid,$keyid,... ## A comma-separated list of exit policies. They're considered first @@ -155,16 +166,24 @@ SocksListenAddress 127.0.0.1 # accept connections only from localhost ## you should update your exit policy to reflect this -- otherwise Tor ## users will be told that those destinations are down. ## +## For security, by default Tor rejects connections to private (local) +## networks, including to your public IP address. See the man page entry +## for ExitPolicyRejectPrivate if you want to allow "exit enclaving". +## #ExitPolicy accept *:6660-6667,reject *:* # allow irc ports but no more #ExitPolicy accept *:119 # accept nntp as well as default exit policy #ExitPolicy reject *:* # no exits allowed -# + ## Bridge relays (or "bridges") are Tor relays that aren't listed in the -## main directory. Since there is no complete public list of them, even if an -## ISP is filtering connections to all the known Tor relays, they probably +## main directory. Since there is no complete public list of them, even an +## ISP that filters connections to all the known Tor relays probably ## won't be able to block all the bridges. Also, websites won't treat you ## differently because they won't know you're running Tor. If you can ## be a real relay, please do; but if not, be a bridge! #BridgeRelay 1 -#ExitPolicy reject *:* +## By default, Tor will advertise your bridge to users through various +## mechanisms like https://bridges.torproject.org/. If you want to run +## a private bridge, for example because you'll give out your bridge +## address manually to your friends, uncomment this line: +#PublishServerDescriptor 0 diff --git a/src/or/Makefile.am b/src/or/Makefile.am index 4f4e9c9731..3cc789a1be 100644 --- a/src/or/Makefile.am +++ b/src/or/Makefile.am @@ -7,7 +7,7 @@ else tor_platform_source= endif -EXTRA_DIST=ntmain.c or_sha1.i +EXTRA_DIST=ntmain.c or_sha1.i Makefile.nmake if USE_EXTERNAL_EVDNS evdns_source= @@ -15,16 +15,46 @@ else evdns_source=eventdns.c endif -libtor_a_SOURCES = buffers.c circuitbuild.c circuitlist.c \ - circuituse.c command.c config.c \ - connection.c connection_edge.c connection_or.c control.c \ - cpuworker.c directory.c dirserv.c dirvote.c \ - dns.c dnsserv.c geoip.c hibernate.c main.c $(tor_platform_source) \ - microdesc.c \ - networkstatus.c onion.c policies.c \ - reasons.c relay.c rendcommon.c rendclient.c rendmid.c \ - rendservice.c rephist.c router.c routerlist.c routerparse.c \ - $(evdns_source) config_codedigest.c +libtor_a_SOURCES = \ + buffers.c \ + circuitbuild.c \ + circuitlist.c \ + circuituse.c \ + command.c \ + config.c \ + connection.c \ + connection_edge.c \ + connection_or.c \ + control.c \ + cpuworker.c \ + directory.c \ + dirserv.c \ + dirvote.c \ + dns.c \ + dnsserv.c \ + geoip.c \ + hibernate.c \ + main.c \ + microdesc.c \ + networkstatus.c \ + nodelist.c \ + onion.c \ + transports.c \ + policies.c \ + reasons.c \ + relay.c \ + rendclient.c \ + rendcommon.c \ + rendmid.c \ + rendservice.c \ + rephist.c \ + router.c \ + routerlist.c \ + routerparse.c \ + status.c \ + $(evdns_source) \ + $(tor_platform_source) \ + config_codedigest.c #libtor_a_LIBADD = ../common/libor.a ../common/libor-crypto.a \ # ../common/libor-event.a @@ -40,18 +70,55 @@ AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ # This seems to matter nowhere but on windows, but I assure you that it # matters a lot there, and is quite hard to debug if you forget to do it. + tor_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ @TOR_LDFLAGS_libevent@ tor_LDADD = ./libtor.a ../common/libor.a ../common/libor-crypto.a \ ../common/libor-event.a \ - @TOR_ZLIB_LIBS@ -lm @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ - -noinst_HEADERS = buffers.h circuitbuild.h circuitlist.h circuituse.h \ - command.h config.h connection_edge.h connection.h connection_or.h \ - control.h cpuworker.h directory.h dirserv.h dirvote.h dns.h \ - dnsserv.h geoip.h hibernate.h main.h microdesc.h networkstatus.h \ - ntmain.h onion.h policies.h reasons.h relay.h rendclient.h \ - rendcommon.h rendmid.h rendservice.h rephist.h router.h routerlist.h \ - routerparse.h or.h eventdns.h eventdns_tor.h micro-revision.i + @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ \ + @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +noinst_HEADERS = \ + buffers.h \ + circuitbuild.h \ + circuitlist.h \ + circuituse.h \ + command.h \ + config.h \ + connection.h \ + connection_edge.h \ + connection_or.h \ + control.h \ + cpuworker.h \ + directory.h \ + dirserv.h \ + dirvote.h \ + dns.h \ + dnsserv.h \ + eventdns.h \ + eventdns_tor.h \ + geoip.h \ + hibernate.h \ + main.h \ + microdesc.h \ + networkstatus.h \ + nodelist.h \ + ntmain.h \ + onion.h \ + or.h \ + transports.h \ + policies.h \ + reasons.h \ + relay.h \ + rendclient.h \ + rendcommon.h \ + rendmid.h \ + rendservice.h \ + rephist.h \ + router.h \ + routerlist.h \ + routerparse.h \ + status.h \ + micro-revision.i config_codedigest.o: or_sha1.i @@ -59,8 +126,9 @@ tor_main.o: micro-revision.i micro-revision.i: FORCE @rm -f micro-revision.tmp; \ - if test -d ../../.git && test -x "`which git 2>&1;true`"; then \ - HASH="`git rev-parse --short=16 HEAD`"; \ + if test -d "$(top_srcdir)/.git" && \ + test -x "`which git 2>&1;true`"; then \ + HASH="`cd "$(top_srcdir)" && git rev-parse --short=16 HEAD`"; \ echo \"$$HASH\" > micro-revision.tmp; \ fi; \ if test ! -f micro-revision.tmp ; then \ @@ -72,11 +140,13 @@ micro-revision.i: FORCE mv micro-revision.tmp micro-revision.i; \ fi; true -or_sha1.i: $(tor_SOURCES) +or_sha1.i: $(tor_SOURCES) $(libtor_a_SOURCES) if test "@SHA1SUM@" != none; then \ - "@SHA1SUM@" $(tor_SOURCES) | "@SED@" -n 's/^\(.*\)$$/"\1\\n"/p' > or_sha1.i; \ + (cd "$(srcdir)" && "@SHA1SUM@" $(tor_SOURCES) $(libtor_a_SOURCES)) | \ + "@SED@" -n 's/^\(.*\)$$/"\1\\n"/p' > or_sha1.i; \ elif test "@OPENSSL@" != none; then \ - "@OPENSSL@" sha1 $(tor_SOURCES) | "@SED@" -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > or_sha1.i; \ + (cd "$(srcdir)" && "@OPENSSL@" sha1 $(tor_SOURCES) $(libtor_a_SOURCES)) | \ + "@SED@" -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > or_sha1.i; \ else \ rm or_sha1.i; \ touch or_sha1.i; \ diff --git a/src/or/Makefile.nmake b/src/or/Makefile.nmake new file mode 100644 index 0000000000..3181e79c20 --- /dev/null +++ b/src/or/Makefile.nmake @@ -0,0 +1,28 @@ +all: tor.exe + +CFLAGS = /I ..\win32 /I ..\..\..\build-alpha\include /I ..\common + +LIBS = ..\..\..\build-alpha\lib\libevent.a \ + ..\..\..\build-alpha\lib\libcrypto.a \ + ..\..\..\build-alpha\lib\libssl.a \ + ..\..\..\build-alpha\lib\libz.a \ + ws2_32.lib advapi32.lib shell32.lib + +LIBTOR_OBJECTS = buffers.obj circuitbuild.obj circuitlist.obj circuituse.obj \ + command.obj config.obj connection.obj connection_edge.obj \ + connection_or.obj control.obj cpuworker.obj directory.obj \ + dirserv.obj dirvote.obj dns.obj dnsserv.obj geoip.obj \ + hibernate.obj main.obj microdesc.obj networkstatus.obj \ + nodelist.obj onion.obj policies.obj reasons.obj relay.obj \ + rendclient.obj rendcommon.obj rendmid.obj rendservice.obj \ + rephist.obj router.obj routerlist.obj routerparse.obj status.obj \ + config_codedigest.obj ntmain.obj + +libtor.lib: $(LIBTOR_OBJECTS) + lib $(LIBTOR_OBJECTS) /out:libtor.lib + +tor.exe: libtor.lib tor_main.obj + $(CC) $(CFLAGS) $(LIBS) libtor.lib ..\common\*.lib tor_main.obj + +clean: + del $(LIBTOR_OBJECTS) *.lib tor.exe diff --git a/src/or/buffers.c b/src/or/buffers.c index 58e5f658b8..9be0476f64 100644 --- a/src/or/buffers.c +++ b/src/or/buffers.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -23,9 +23,6 @@ #ifdef HAVE_UNISTD_H #include <unistd.h> #endif -#ifdef HAVE_SYS_UIO_H -#include <sys/uio.h> -#endif //#define PARANOIA @@ -56,6 +53,13 @@ * forever. */ +static int parse_socks(const char *data, size_t datalen, socks_request_t *req, + int log_sockstype, int safe_socks, ssize_t *drain_out, + size_t *want_length_out); +static int parse_socks_client(const uint8_t *data, size_t datalen, + int state, char **reason, + ssize_t *drain_out); + /* Chunk manipulation functions */ /** A single chunk on a buffer or in a freelist. */ @@ -64,8 +68,8 @@ typedef struct chunk_t { size_t datalen; /**< The number of bytes stored in this chunk */ size_t memlen; /**< The number of usable bytes of storage in <b>mem</b>. */ char *data; /**< A pointer to the first byte of data stored in <b>mem</b>. */ - char mem[1]; /**< The actual memory used for storage in this chunk. May be - * more than one byte long. */ + char mem[FLEXIBLE_ARRAY_MEMBER]; /**< The actual memory used for storage in + * this chunk. */ } chunk_t; #define CHUNK_HEADER_LEN STRUCT_OFFSET(chunk_t, mem[0]) @@ -104,7 +108,7 @@ chunk_repack(chunk_t *chunk) chunk->data = &chunk->mem[0]; } -#ifdef ENABLE_BUF_FREELISTS +#if defined(ENABLE_BUF_FREELISTS) || defined(RUNNING_DOXYGEN) /** A freelist of chunks. */ typedef struct chunk_freelist_t { size_t alloc_size; /**< What size chunks does this freelist hold? */ @@ -143,7 +147,8 @@ static INLINE chunk_freelist_t * get_freelist(size_t alloc) { int i; - for (i=0; freelists[i].alloc_size <= alloc; ++i) { + for (i=0; (freelists[i].alloc_size <= alloc && + freelists[i].alloc_size); ++i ) { if (freelists[i].alloc_size == alloc) { return &freelists[i]; } @@ -548,11 +553,45 @@ buf_free(buf_t *buf) { if (!buf) return; + buf_clear(buf); buf->magic = 0xdeadbeef; tor_free(buf); } +/** Return a new copy of <b>in_chunk</b> */ +static chunk_t * +chunk_copy(const chunk_t *in_chunk) +{ + chunk_t *newch = tor_memdup(in_chunk, CHUNK_ALLOC_SIZE(in_chunk->memlen)); + newch->next = NULL; + if (in_chunk->data) { + off_t offset = in_chunk->data - in_chunk->mem; + newch->data = newch->mem + offset; + } + return newch; +} + +/** Return a new copy of <b>buf</b> */ +buf_t * +buf_copy(const buf_t *buf) +{ + chunk_t *ch; + buf_t *out = buf_new(); + out->default_chunk_size = buf->default_chunk_size; + for (ch = buf->head; ch; ch = ch->next) { + chunk_t *newch = chunk_copy(ch); + if (out->tail) { + out->tail->next = newch; + out->tail = newch; + } else { + out->head = out->tail = newch; + } + } + out->datalen = buf->datalen; + return out; +} + /** Append a new chunk with enough capacity to hold <b>capacity</b> bytes to * the tail of <b>buf</b>. If <b>capped</b>, don't allocate a chunk bigger * than MAX_CHUNK_ALLOC. */ @@ -579,10 +618,6 @@ buf_add_chunk_with_capacity(buf_t *buf, size_t capacity, int capped) return chunk; } -/** If we're using readv and writev, how many chunks are we willing to - * read/write at a time? */ -#define N_IOV 3 - /** Read up to <b>at_most</b> bytes from the socket <b>fd</b> into * <b>chunk</b> (which must be on <b>buf</b>). If we get an EOF, set * *<b>reached_eof</b> to 1. Return -1 on error, 0 on eof or blocking, @@ -592,30 +627,14 @@ read_to_chunk(buf_t *buf, chunk_t *chunk, tor_socket_t fd, size_t at_most, int *reached_eof, int *socket_error) { ssize_t read_result; -#if 0 && defined(HAVE_READV) && !defined(WIN32) - struct iovec iov[N_IOV]; - int i; - size_t remaining = at_most; - for (i=0; chunk && i < N_IOV && remaining; ++i) { - iov[i].iov_base = CHUNK_WRITE_PTR(chunk); - if (remaining > CHUNK_REMAINING_CAPACITY(chunk)) - iov[i].iov_len = CHUNK_REMAINING_CAPACITY(chunk); - else - iov[i].iov_len = remaining; - remaining -= iov[i].iov_len; - chunk = chunk->next; - } - read_result = readv(fd, iov, i); -#else if (at_most > CHUNK_REMAINING_CAPACITY(chunk)) at_most = CHUNK_REMAINING_CAPACITY(chunk); read_result = tor_socket_recv(fd, CHUNK_WRITE_PTR(chunk), at_most, 0); -#endif if (read_result < 0) { int e = tor_socket_errno(fd); if (!ERRNO_IS_EAGAIN(e)) { /* it's a real error */ -#ifdef MS_WINDOWS +#ifdef _WIN32 if (e == WSAENOBUFS) log_warn(LD_NET,"recv() failed: WSAENOBUFS. Not enough ram?"); #endif @@ -629,14 +648,6 @@ read_to_chunk(buf_t *buf, chunk_t *chunk, tor_socket_t fd, size_t at_most, return 0; } else { /* actually got bytes. */ buf->datalen += read_result; -#if 0 && defined(HAVE_READV) && !defined(WIN32) - while ((size_t)read_result > CHUNK_REMAINING_CAPACITY(chunk)) { - chunk->datalen += CHUNK_REMAINING_CAPACITY(chunk); - read_result -= CHUNK_REMAINING_CAPACITY(chunk); - chunk = chunk->next; - tor_assert(chunk); - } -#endif chunk->datalen += read_result; log_debug(LD_NET,"Read %ld bytes. %d on inbuf.", (long)read_result, (int)buf->datalen); @@ -667,12 +678,12 @@ read_to_chunk_tls(buf_t *buf, chunk_t *chunk, tor_tls_t *tls, * (because of EOF), set *<b>reached_eof</b> to 1 and return 0. Return -1 on * error; else return the number of bytes read. */ -/* XXXX023 indicate "read blocked" somehow? */ +/* XXXX024 indicate "read blocked" somehow? */ int read_to_buf(tor_socket_t s, size_t at_most, buf_t *buf, int *reached_eof, int *socket_error) { - /* XXXX023 It's stupid to overload the return values for these functions: + /* XXXX024 It's stupid to overload the return values for these functions: * "error status" and "number of bytes read" are not mutually exclusive. */ int r = 0; @@ -680,7 +691,7 @@ read_to_buf(tor_socket_t s, size_t at_most, buf_t *buf, int *reached_eof, check(); tor_assert(reached_eof); - tor_assert(s >= 0); + tor_assert(SOCKET_OK(s)); while (at_most > total_read) { size_t readlen = at_most - total_read; @@ -734,6 +745,9 @@ read_to_buf_tls(tor_tls_t *tls, size_t at_most, buf_t *buf) { int r = 0; size_t total_read = 0; + + check_no_tls_errors(); + check(); while (at_most > total_read) { @@ -772,30 +786,15 @@ flush_chunk(tor_socket_t s, buf_t *buf, chunk_t *chunk, size_t sz, size_t *buf_flushlen) { ssize_t write_result; -#if 0 && defined(HAVE_WRITEV) && !defined(WIN32) - struct iovec iov[N_IOV]; - int i; - size_t remaining = sz; - for (i=0; chunk && i < N_IOV && remaining; ++i) { - iov[i].iov_base = chunk->data; - if (remaining > chunk->datalen) - iov[i].iov_len = chunk->datalen; - else - iov[i].iov_len = remaining; - remaining -= iov[i].iov_len; - chunk = chunk->next; - } - write_result = writev(s, iov, i); -#else + if (sz > chunk->datalen) sz = chunk->datalen; write_result = tor_socket_send(s, chunk->data, sz, 0); -#endif if (write_result < 0) { int e = tor_socket_errno(s); if (!ERRNO_IS_EAGAIN(e)) { /* it's a real error */ -#ifdef MS_WINDOWS +#ifdef _WIN32 if (e == WSAENOBUFS) log_warn(LD_NET,"write() failed: WSAENOBUFS. Not enough ram?"); #endif @@ -857,13 +856,13 @@ flush_chunk_tls(tor_tls_t *tls, buf_t *buf, chunk_t *chunk, int flush_buf(tor_socket_t s, buf_t *buf, size_t sz, size_t *buf_flushlen) { - /* XXXX023 It's stupid to overload the return values for these functions: + /* XXXX024 It's stupid to overload the return values for these functions: * "error status" and "number of bytes flushed" are not mutually exclusive. */ int r; size_t flushed = 0; tor_assert(buf_flushlen); - tor_assert(s >= 0); + tor_assert(SOCKET_OK(s)); tor_assert(*buf_flushlen <= buf->datalen); tor_assert(sz <= *buf_flushlen); @@ -1011,6 +1010,33 @@ fetch_from_buf(char *string, size_t string_len, buf_t *buf) return (int)buf->datalen; } +/** True iff the cell command <b>command</b> is one that implies a + * variable-length cell in Tor link protocol <b>linkproto</b>. */ +static INLINE int +cell_command_is_var_length(uint8_t command, int linkproto) +{ + /* If linkproto is v2 (2), CELL_VERSIONS is the only variable-length cells + * work as implemented here. If it's 1, there are no variable-length cells. + * Tor does not support other versions right now, and so can't negotiate + * them. + */ + switch (linkproto) { + case 1: + /* Link protocol version 1 has no variable-length cells. */ + return 0; + case 2: + /* In link protocol version 2, VERSIONS is the only variable-length cell */ + return command == CELL_VERSIONS; + case 0: + case 3: + default: + /* In link protocol version 3 and later, and in version "unknown", + * commands 128 and higher indicate variable-length. VERSIONS is + * grandfathered in. */ + return command == CELL_VERSIONS || command >= 128; + } +} + /** Check <b>buf</b> for a variable-length cell according to the rules of link * protocol version <b>linkproto</b>. If one is found, pull it off the buffer * and assign a newly allocated var_cell_t to *<b>out</b>, and return 1. @@ -1025,12 +1051,6 @@ fetch_var_cell_from_buf(buf_t *buf, var_cell_t **out, int linkproto) var_cell_t *result; uint8_t command; uint16_t length; - /* If linkproto is unknown (0) or v2 (2), variable-length cells work as - * implemented here. If it's 1, there are no variable-length cells. Tor - * does not support other versions right now, and so can't negotiate them. - */ - if (linkproto == 1) - return 0; check(); *out = NULL; if (buf->datalen < VAR_CELL_HEADER_SIZE) @@ -1038,7 +1058,7 @@ fetch_var_cell_from_buf(buf_t *buf, var_cell_t **out, int linkproto) peek_from_buf(hdr, sizeof(hdr), buf); command = get_uint8(hdr+2); - if (!(CELL_COMMAND_IS_VAR_LENGTH(command))) + if (!(cell_command_is_var_length(command, linkproto))) return 0; length = ntohs(get_uint16(hdr+3)); @@ -1057,6 +1077,91 @@ fetch_var_cell_from_buf(buf_t *buf, var_cell_t **out, int linkproto) return 1; } +#ifdef USE_BUFFEREVENTS +/** Try to read <b>n</b> bytes from <b>buf</b> at <b>pos</b> (which may be + * NULL for the start of the buffer), copying the data only if necessary. Set + * *<b>data_out</b> to a pointer to the desired bytes. Set <b>free_out</b> + * to 1 if we needed to malloc *<b>data</b> because the original bytes were + * noncontiguous; 0 otherwise. Return the number of bytes actually available + * at *<b>data_out</b>. + */ +static ssize_t +inspect_evbuffer(struct evbuffer *buf, char **data_out, size_t n, + int *free_out, struct evbuffer_ptr *pos) +{ + int n_vecs, i; + + if (evbuffer_get_length(buf) < n) + n = evbuffer_get_length(buf); + if (n == 0) + return 0; + n_vecs = evbuffer_peek(buf, n, pos, NULL, 0); + tor_assert(n_vecs > 0); + if (n_vecs == 1) { + struct evbuffer_iovec v; + i = evbuffer_peek(buf, n, pos, &v, 1); + tor_assert(i == 1); + *data_out = v.iov_base; + *free_out = 0; + return v.iov_len; + } else { + ev_ssize_t copied; + *data_out = tor_malloc(n); + *free_out = 1; + copied = evbuffer_copyout(buf, *data_out, n); + tor_assert(copied >= 0 && (size_t)copied == n); + return copied; + } +} + +/** As fetch_var_cell_from_buf, buf works on an evbuffer. */ +int +fetch_var_cell_from_evbuffer(struct evbuffer *buf, var_cell_t **out, + int linkproto) +{ + char *hdr = NULL; + int free_hdr = 0; + size_t n; + size_t buf_len; + uint8_t command; + uint16_t cell_length; + var_cell_t *cell; + int result = 0; + + *out = NULL; + buf_len = evbuffer_get_length(buf); + if (buf_len < VAR_CELL_HEADER_SIZE) + return 0; + + n = inspect_evbuffer(buf, &hdr, VAR_CELL_HEADER_SIZE, &free_hdr, NULL); + tor_assert(n >= VAR_CELL_HEADER_SIZE); + + command = get_uint8(hdr+2); + if (!(cell_command_is_var_length(command, linkproto))) { + goto done; + } + + cell_length = ntohs(get_uint16(hdr+3)); + if (buf_len < (size_t)(VAR_CELL_HEADER_SIZE+cell_length)) { + result = 1; /* Not all here yet. */ + goto done; + } + + cell = var_cell_new(cell_length); + cell->command = command; + cell->circ_id = ntohs(get_uint16(hdr)); + evbuffer_drain(buf, VAR_CELL_HEADER_SIZE); + evbuffer_remove(buf, cell->payload, cell_length); + *out = cell; + result = 1; + + done: + if (free_hdr && hdr) + tor_free(hdr); + return result; +} +#endif + /** Move up to *<b>buf_flushlen</b> bytes from <b>buf_in</b> to * <b>buf_out</b>, and modify *<b>buf_flushlen</b> appropriately. * Return the number of bytes actually copied. @@ -1064,8 +1169,7 @@ fetch_var_cell_from_buf(buf_t *buf, var_cell_t **out, int linkproto) int move_buf_to_buf(buf_t *buf_out, buf_t *buf_in, size_t *buf_flushlen) { - /* XXXX we can do way better here, but this doesn't turn up in any - * profiles. */ + /* We can do way better here, but this doesn't turn up in any profiles. */ char b[4096]; size_t cp, len; len = *buf_flushlen; @@ -1300,6 +1404,94 @@ fetch_from_buf_http(buf_t *buf, return 1; } +#ifdef USE_BUFFEREVENTS +/** As fetch_from_buf_http, buf works on an evbuffer. */ +int +fetch_from_evbuffer_http(struct evbuffer *buf, + char **headers_out, size_t max_headerlen, + char **body_out, size_t *body_used, size_t max_bodylen, + int force_complete) +{ + struct evbuffer_ptr crlf, content_length; + size_t headerlen, bodylen, contentlen; + + /* Find the first \r\n\r\n in the buffer */ + crlf = evbuffer_search(buf, "\r\n\r\n", 4, NULL); + if (crlf.pos < 0) { + /* We didn't find one. */ + if (evbuffer_get_length(buf) > max_headerlen) + return -1; /* Headers too long. */ + return 0; /* Headers not here yet. */ + } else if (crlf.pos > (int)max_headerlen) { + return -1; /* Headers too long. */ + } + + headerlen = crlf.pos + 4; /* Skip over the \r\n\r\n */ + bodylen = evbuffer_get_length(buf) - headerlen; + if (bodylen > max_bodylen) + return -1; /* body too long */ + + /* Look for the first occurrence of CONTENT_LENGTH insize buf before the + * crlfcrlf */ + content_length = evbuffer_search_range(buf, CONTENT_LENGTH, + strlen(CONTENT_LENGTH), NULL, &crlf); + + if (content_length.pos >= 0) { + /* We found a content_length: parse it and figure out if the body is here + * yet. */ + struct evbuffer_ptr eol; + char *data = NULL; + int free_data = 0; + int n, i; + n = evbuffer_ptr_set(buf, &content_length, strlen(CONTENT_LENGTH), + EVBUFFER_PTR_ADD); + tor_assert(n == 0); + eol = evbuffer_search_eol(buf, &content_length, NULL, EVBUFFER_EOL_CRLF); + tor_assert(eol.pos > content_length.pos); + tor_assert(eol.pos <= crlf.pos); + inspect_evbuffer(buf, &data, eol.pos - content_length.pos, &free_data, + &content_length); + + i = atoi(data); + if (free_data) + tor_free(data); + if (i < 0) { + log_warn(LD_PROTOCOL, "Content-Length is less than zero; it looks like " + "someone is trying to crash us."); + return -1; + } + contentlen = i; + /* if content-length is malformed, then our body length is 0. fine. */ + log_debug(LD_HTTP,"Got a contentlen of %d.",(int)contentlen); + if (bodylen < contentlen) { + if (!force_complete) { + log_debug(LD_HTTP,"body not all here yet."); + return 0; /* not all there yet */ + } + } + if (bodylen > contentlen) { + bodylen = contentlen; + log_debug(LD_HTTP,"bodylen reduced to %d.",(int)bodylen); + } + } + + if (headers_out) { + *headers_out = tor_malloc(headerlen+1); + evbuffer_remove(buf, *headers_out, headerlen); + (*headers_out)[headerlen] = '\0'; + } + if (body_out) { + tor_assert(headers_out); + tor_assert(body_used); + *body_used = bodylen; + *body_out = tor_malloc(bodylen+1); + evbuffer_remove(buf, *body_out, bodylen); + (*body_out)[bodylen] = '\0'; + } + return 1; +} +#endif + /** * Wait this many seconds before warning the user about using SOCKS unsafely * again (requires that WarnUnsafeSocks is turned on). */ @@ -1314,7 +1506,7 @@ log_unsafe_socks_warning(int socks_protocol, const char *address, { static ratelim_t socks_ratelim = RATELIM_INIT(SOCKS_WARN_INTERVAL); - or_options_t *options = get_options(); + const or_options_t *options = get_options(); char *m = NULL; if (! options->WarnUnsafeSocks) return; @@ -1341,6 +1533,31 @@ log_unsafe_socks_warning(int socks_protocol, const char *address, * actually significantly higher than the longest possible socks message. */ #define MAX_SOCKS_MESSAGE_LEN 512 +/** Return a new socks_request_t. */ +socks_request_t * +socks_request_new(void) +{ + return tor_malloc_zero(sizeof(socks_request_t)); +} + +/** Free all storage held in the socks_request_t <b>req</b>. */ +void +socks_request_free(socks_request_t *req) +{ + if (!req) + return; + if (req->username) { + memwipe(req->username, 0x10, req->usernamelen); + tor_free(req->username); + } + if (req->password) { + memwipe(req->password, 0x04, req->passwordlen); + tor_free(req->password); + } + memwipe(req, 0xCC, sizeof(socks_request_t)); + tor_free(req); +} + /** There is a (possibly incomplete) socks handshake on <b>buf</b>, of one * of the forms * - socks4: "socksheader username\\0" @@ -1370,59 +1587,241 @@ int fetch_from_buf_socks(buf_t *buf, socks_request_t *req, int log_sockstype, int safe_socks) { + int res; + ssize_t n_drain; + size_t want_length = 128; + + if (buf->datalen < 2) /* version and another byte */ + return 0; + + do { + n_drain = 0; + buf_pullup(buf, want_length, 0); + tor_assert(buf->head && buf->head->datalen >= 2); + want_length = 0; + + res = parse_socks(buf->head->data, buf->head->datalen, req, log_sockstype, + safe_socks, &n_drain, &want_length); + + if (n_drain < 0) + buf_clear(buf); + else if (n_drain > 0) + buf_remove_from_front(buf, n_drain); + + } while (res == 0 && buf->head && want_length < buf->datalen && + buf->datalen >= 2); + + return res; +} + +#ifdef USE_BUFFEREVENTS +/* As fetch_from_buf_socks(), but targets an evbuffer instead. */ +int +fetch_from_evbuffer_socks(struct evbuffer *buf, socks_request_t *req, + int log_sockstype, int safe_socks) +{ + char *data; + ssize_t n_drain; + size_t datalen, buflen, want_length; + int res; + + buflen = evbuffer_get_length(buf); + if (buflen < 2) + return 0; + + { + /* See if we can find the socks request in the first chunk of the buffer. + */ + struct evbuffer_iovec v; + int i; + n_drain = 0; + i = evbuffer_peek(buf, -1, NULL, &v, 1); + tor_assert(i == 1); + data = v.iov_base; + datalen = v.iov_len; + want_length = 0; + + res = parse_socks(data, datalen, req, log_sockstype, + safe_socks, &n_drain, &want_length); + + if (n_drain < 0) + evbuffer_drain(buf, evbuffer_get_length(buf)); + else if (n_drain > 0) + evbuffer_drain(buf, n_drain); + + if (res) + return res; + } + + /* Okay, the first chunk of the buffer didn't have a complete socks request. + * That means that either we don't have a whole socks request at all, or + * it's gotten split up. We're going to try passing parse_socks() bigger + * and bigger chunks until either it says "Okay, I got it", or it says it + * will need more data than we currently have. */ + + /* Loop while we have more data that we haven't given parse_socks() yet. */ + do { + int free_data = 0; + const size_t last_wanted = want_length; + n_drain = 0; + data = NULL; + datalen = inspect_evbuffer(buf, &data, want_length, &free_data, NULL); + + want_length = 0; + res = parse_socks(data, datalen, req, log_sockstype, + safe_socks, &n_drain, &want_length); + + if (free_data) + tor_free(data); + + if (n_drain < 0) + evbuffer_drain(buf, evbuffer_get_length(buf)); + else if (n_drain > 0) + evbuffer_drain(buf, n_drain); + + if (res == 0 && n_drain == 0 && want_length <= last_wanted) { + /* If we drained nothing, and we didn't ask for more than last time, + * then we probably wanted more data than the buffer actually had, + * and we're finding out that we're not satisified with it. It's + * time to break until we have more data. */ + break; + } + + buflen = evbuffer_get_length(buf); + } while (res == 0 && want_length <= buflen && buflen >= 2); + + return res; +} +#endif + +/** Implementation helper to implement fetch_from_*_socks. Instead of looking + * at a buffer's contents, we look at the <b>datalen</b> bytes of data in + * <b>data</b>. Instead of removing data from the buffer, we set + * <b>drain_out</b> to the amount of data that should be removed (or -1 if the + * buffer should be cleared). Instead of pulling more data into the first + * chunk of the buffer, we set *<b>want_length_out</b> to the number of bytes + * we'd like to see in the input buffer, if they're available. */ +static int +parse_socks(const char *data, size_t datalen, socks_request_t *req, + int log_sockstype, int safe_socks, ssize_t *drain_out, + size_t *want_length_out) +{ unsigned int len; char tmpbuf[TOR_ADDR_BUF_LEN+1]; tor_addr_t destaddr; uint32_t destip; uint8_t socksver; - enum {socks4, socks4a} socks4_prot = socks4a; char *next, *startaddr; + unsigned char usernamelen, passlen; struct in_addr in; - if (buf->datalen < 2) /* version and another byte */ + if (datalen < 2) { + /* We always need at least 2 bytes. */ + *want_length_out = 2; return 0; + } - buf_pullup(buf, MAX_SOCKS_MESSAGE_LEN, 0); - tor_assert(buf->head && buf->head->datalen >= 2); + if (req->socks_version == 5 && !req->got_auth) { + /* See if we have received authentication. Strictly speaking, we should + also check whether we actually negotiated username/password + authentication. But some broken clients will send us authentication + even if we negotiated SOCKS_NO_AUTH. */ + if (*data == 1) { /* username/pass version 1 */ + /* Format is: authversion [1 byte] == 1 + usernamelen [1 byte] + username [usernamelen bytes] + passlen [1 byte] + password [passlen bytes] */ + usernamelen = (unsigned char)*(data + 1); + if (datalen < 2u + usernamelen + 1u) { + *want_length_out = 2u + usernamelen + 1u; + return 0; + } + passlen = (unsigned char)*(data + 2u + usernamelen); + if (datalen < 2u + usernamelen + 1u + passlen) { + *want_length_out = 2u + usernamelen + 1u + passlen; + return 0; + } + req->replylen = 2; /* 2 bytes of response */ + req->reply[0] = 5; + req->reply[1] = 0; /* authentication successful */ + log_debug(LD_APP, + "socks5: Accepted username/password without checking."); + if (usernamelen) { + req->username = tor_memdup(data+2u, usernamelen); + req->usernamelen = usernamelen; + } + if (passlen) { + req->password = tor_memdup(data+3u+usernamelen, passlen); + req->passwordlen = passlen; + } + *drain_out = 2u + usernamelen + 1u + passlen; + req->got_auth = 1; + *want_length_out = 7; /* Minimal socks5 sommand. */ + return 0; + } else if (req->auth_type == SOCKS_USER_PASS) { + /* unknown version byte */ + log_warn(LD_APP, "Socks5 username/password version %d not recognized; " + "rejecting.", (int)*data); + return -1; + } + } - socksver = *buf->head->data; + socksver = *data; switch (socksver) { /* which version of socks? */ - case 5: /* socks5 */ if (req->socks_version != 5) { /* we need to negotiate a method */ - unsigned char nummethods = (unsigned char)*(buf->head->data+1); + unsigned char nummethods = (unsigned char)*(data+1); + int r=0; tor_assert(!req->socks_version); - if (buf->datalen < 2u+nummethods) + if (datalen < 2u+nummethods) { + *want_length_out = 2u+nummethods; return 0; - buf_pullup(buf, 2u+nummethods, 0); - if (!nummethods || !memchr(buf->head->data+2, 0, nummethods)) { + } + if (!nummethods) + return -1; + req->replylen = 2; /* 2 bytes of response */ + req->reply[0] = 5; /* socks5 reply */ + if (memchr(data+2, SOCKS_NO_AUTH, nummethods)) { + req->reply[1] = SOCKS_NO_AUTH; /* tell client to use "none" auth + method */ + req->socks_version = 5; /* remember we've already negotiated auth */ + log_debug(LD_APP,"socks5: accepted method 0 (no authentication)"); + r=0; + } else if (memchr(data+2, SOCKS_USER_PASS, nummethods)) { + req->auth_type = SOCKS_USER_PASS; + req->reply[1] = SOCKS_USER_PASS; /* tell client to use "user/pass" + auth method */ + req->socks_version = 5; /* remember we've already negotiated auth */ + log_debug(LD_APP,"socks5: accepted method 2 (username/password)"); + r=0; + } else { log_warn(LD_APP, - "socks5: offered methods don't include 'no auth'. " - "Rejecting."); - req->replylen = 2; /* 2 bytes of response */ - req->reply[0] = 5; + "socks5: offered methods don't include 'no auth' or " + "username/password. Rejecting."); req->reply[1] = '\xFF'; /* reject all methods */ - return -1; + r=-1; } - /* remove packet from buf. also remove any other extraneous - * bytes, to support broken socks clients. */ - buf_clear(buf); + /* Remove packet from buf. Some SOCKS clients will have sent extra + * junk at this point; let's hope it's an authentication message. */ + *drain_out = 2u + nummethods; - req->replylen = 2; /* 2 bytes of response */ - req->reply[0] = 5; /* socks5 reply */ - req->reply[1] = 0; /* tell client to use "none" auth method */ - req->socks_version = 5; /* remember we've already negotiated auth */ - log_debug(LD_APP,"socks5: accepted method 0"); - return 0; + return r; + } + if (req->auth_type != SOCKS_NO_AUTH && !req->got_auth) { + log_warn(LD_APP, + "socks5: negotiated authentication, but none provided"); + return -1; } /* we know the method; read in the request */ log_debug(LD_APP,"socks5: checking request"); - if (buf->datalen < 8) /* basic info plus >=2 for addr plus 2 for port */ + if (datalen < 7) {/* basic info plus >=1 for addr plus 2 for port */ + *want_length_out = 7; return 0; /* not yet */ - tor_assert(buf->head->datalen >= 8); - req->command = (unsigned char) *(buf->head->data+1); + } + req->command = (unsigned char) *(data+1); if (req->command != SOCKS_COMMAND_CONNECT && req->command != SOCKS_COMMAND_RESOLVE && req->command != SOCKS_COMMAND_RESOLVE_PTR) { @@ -1431,19 +1830,21 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, req->command); return -1; } - switch (*(buf->head->data+3)) { /* address type */ + switch (*(data+3)) { /* address type */ case 1: /* IPv4 address */ case 4: /* IPv6 address */ { - const int is_v6 = *(buf->head->data+3) == 4; + const int is_v6 = *(data+3) == 4; const unsigned addrlen = is_v6 ? 16 : 4; log_debug(LD_APP,"socks5: ipv4 address type"); - if (buf->datalen < 6+addrlen) /* ip/port there? */ + if (datalen < 6+addrlen) {/* ip/port there? */ + *want_length_out = 6+addrlen; return 0; /* not yet */ + } if (is_v6) - tor_addr_from_ipv6_bytes(&destaddr, buf->head->data+4); + tor_addr_from_ipv6_bytes(&destaddr, data+4); else - tor_addr_from_ipv4n(&destaddr, get_uint32(buf->head->data+4)); + tor_addr_from_ipv4n(&destaddr, get_uint32(data+4)); tor_addr_to_str(tmpbuf, &destaddr, sizeof(tmpbuf), 1); @@ -1455,8 +1856,8 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, return -1; } strlcpy(req->address,tmpbuf,sizeof(req->address)); - req->port = ntohs(get_uint16(buf->head->data+4+addrlen)); - buf_remove_from_front(buf, 6+addrlen); + req->port = ntohs(get_uint16(data+4+addrlen)); + *drain_out = 6+addrlen; if (req->command != SOCKS_COMMAND_RESOLVE_PTR && !addressmap_have_mapping(req->address,0)) { log_unsafe_socks_warning(5, req->address, req->port, safe_socks); @@ -1472,21 +1873,21 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, "hostname type. Rejecting."); return -1; } - len = (unsigned char)*(buf->head->data+4); - if (buf->datalen < 7+len) /* addr/port there? */ + len = (unsigned char)*(data+4); + if (datalen < 7+len) { /* addr/port there? */ + *want_length_out = 7+len; return 0; /* not yet */ - buf_pullup(buf, 7+len, 0); - tor_assert(buf->head->datalen >= 7+len); + } if (len+1 > MAX_SOCKS_ADDR_LEN) { log_warn(LD_APP, "socks5 hostname is %d bytes, which doesn't fit in " "%d. Rejecting.", len+1,MAX_SOCKS_ADDR_LEN); return -1; } - memcpy(req->address,buf->head->data+5,len); + memcpy(req->address,data+5,len); req->address[len] = 0; - req->port = ntohs(get_uint16(buf->head->data+5+len)); - buf_remove_from_front(buf, 5+len+2); + req->port = ntohs(get_uint16(data+5+len)); + *drain_out = 5+len+2; if (!tor_strisprint(req->address) || strchr(req->address,'\"')) { log_warn(LD_PROTOCOL, "Your application (using socks5 to port %d) gave Tor " @@ -1496,25 +1897,29 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, } if (log_sockstype) log_notice(LD_APP, - "Your application (using socks5 to port %d) gave " - "Tor a hostname, which means Tor will do the DNS resolve " - "for you. This is good.", req->port); + "Your application (using socks5 to port %d) instructed " + "Tor to take care of the DNS resolution itself if " + "necessary. This is good.", req->port); return 1; default: /* unsupported */ log_warn(LD_APP,"socks5: unsupported address type %d. Rejecting.", - (int) *(buf->head->data+3)); + (int) *(data+3)); return -1; } tor_assert(0); - case 4: /* socks4 */ - /* http://archive.socks.permeo.com/protocol/socks4.protocol */ - /* http://archive.socks.permeo.com/protocol/socks4a.protocol */ + case 4: { /* socks4 */ + enum {socks4, socks4a} socks4_prot = socks4a; + const char *authstart, *authend; + /* http://ss5.sourceforge.net/socks4.protocol.txt */ + /* http://ss5.sourceforge.net/socks4A.protocol.txt */ req->socks_version = 4; - if (buf->datalen < SOCKS4_NETWORK_LEN) /* basic info available? */ + if (datalen < SOCKS4_NETWORK_LEN) {/* basic info available? */ + *want_length_out = SOCKS4_NETWORK_LEN; return 0; /* not yet */ - buf_pullup(buf, 1280, 0); - req->command = (unsigned char) *(buf->head->data+1); + } + // buf_pullup(buf, 1280, 0); + req->command = (unsigned char) *(data+1); if (req->command != SOCKS_COMMAND_CONNECT && req->command != SOCKS_COMMAND_RESOLVE) { /* not a connect or resolve? we don't support it. (No resolve_ptr with @@ -1524,8 +1929,8 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, return -1; } - req->port = ntohs(get_uint16(buf->head->data+2)); - destip = ntohl(get_uint32(buf->head->data+4)); + req->port = ntohs(get_uint16(data+2)); + destip = ntohl(get_uint32(data+4)); if ((!req->port && req->command!=SOCKS_COMMAND_RESOLVE) || !destip) { log_warn(LD_APP,"socks4: Port or DestIP is zero. Rejecting."); return -1; @@ -1545,17 +1950,20 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, socks4_prot = socks4; } - next = memchr(buf->head->data+SOCKS4_NETWORK_LEN, 0, - buf->head->datalen-SOCKS4_NETWORK_LEN); + authstart = data + SOCKS4_NETWORK_LEN; + next = memchr(authstart, 0, + datalen-SOCKS4_NETWORK_LEN); if (!next) { - if (buf->head->datalen >= 1024) { + if (datalen >= 1024) { log_debug(LD_APP, "Socks4 user name too long; rejecting."); return -1; } log_debug(LD_APP,"socks4: Username not here yet."); + *want_length_out = datalen+1024; /* More than we need, but safe */ return 0; } - tor_assert(next < CHUNK_WRITE_PTR(buf->head)); + authend = next; + tor_assert(next < data+datalen); startaddr = NULL; if (socks4_prot != socks4a && @@ -1566,18 +1974,20 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, return -1; } if (socks4_prot == socks4a) { - if (next+1 == CHUNK_WRITE_PTR(buf->head)) { + if (next+1 == data+datalen) { log_debug(LD_APP,"socks4: No part of destaddr here yet."); + *want_length_out = datalen + 1024; /* More than we need, but safe */ return 0; } startaddr = next+1; - next = memchr(startaddr, 0, CHUNK_WRITE_PTR(buf->head)-startaddr); + next = memchr(startaddr, 0, data + datalen - startaddr); if (!next) { - if (buf->head->datalen >= 1024) { + if (datalen >= 1024) { log_debug(LD_APP,"socks4: Destaddr too long."); return -1; } log_debug(LD_APP,"socks4: Destaddr not all here yet."); + *want_length_out = datalen + 1024; /* More than we need, but safe */ return 0; } if (MAX_SOCKS_ADDR_LEN <= next-startaddr) { @@ -1588,9 +1998,9 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, if (log_sockstype) log_notice(LD_APP, - "Your application (using socks4a to port %d) gave " - "Tor a hostname, which means Tor will do the DNS resolve " - "for you. This is good.", req->port); + "Your application (using socks4a to port %d) instructed " + "Tor to take care of the DNS resolution itself if " + "necessary. This is good.", req->port); } log_debug(LD_APP,"socks4: Everything is here. Success."); strlcpy(req->address, startaddr ? startaddr : tmpbuf, @@ -1602,15 +2012,20 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, req->port, escaped(req->address)); return -1; } + if (authend != authstart) { + req->got_auth = 1; + req->usernamelen = authend - authstart; + req->username = tor_memdup(authstart, authend - authstart); + } /* next points to the final \0 on inbuf */ - buf_remove_from_front(buf, next - buf->head->data + 1); + *drain_out = next - data + 1; return 1; - + } case 'G': /* get */ case 'H': /* head */ case 'P': /* put/post */ case 'C': /* connect */ - strlcpy(req->reply, + strlcpy((char*)req->reply, "HTTP/1.0 501 Tor is not an HTTP Proxy\r\n" "Content-Type: text/html; charset=iso-8859-1\r\n\r\n" "<html>\n" @@ -1636,14 +2051,15 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, "</body>\n" "</html>\n" , MAX_SOCKS_REPLY_LEN); - req->replylen = strlen(req->reply)+1; + req->replylen = strlen((char*)req->reply)+1; /* fall through */ default: /* version is not socks4 or socks5 */ log_warn(LD_APP, "Socks version %d not recognized. (Tor is not an http proxy.)", - *(buf->head->data)); + *(data)); { - char *tmp = tor_strndup(buf->head->data, 8); /*XXXX what if longer?*/ + /* Tell the controller the first 8 bytes. */ + char *tmp = tor_strndup(data, datalen < 8 ? datalen : 8); control_event_client_status(LOG_WARN, "SOCKS_UNKNOWN_PROTOCOL DATA=\"%s\"", escaped(tmp)); @@ -1665,21 +2081,67 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, int fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) { - unsigned char *data; - size_t addrlen; - + ssize_t drain = 0; + int r; if (buf->datalen < 2) return 0; buf_pullup(buf, MAX_SOCKS_MESSAGE_LEN, 0); tor_assert(buf->head && buf->head->datalen >= 2); - data = (unsigned char *) buf->head->data; + r = parse_socks_client((uint8_t*)buf->head->data, buf->head->datalen, + state, reason, &drain); + if (drain > 0) + buf_remove_from_front(buf, drain); + else if (drain < 0) + buf_clear(buf); + + return r; +} + +#ifdef USE_BUFFEREVENTS +/** As fetch_from_buf_socks_client, buf works on an evbuffer */ +int +fetch_from_evbuffer_socks_client(struct evbuffer *buf, int state, + char **reason) +{ + ssize_t drain = 0; + uint8_t *data; + size_t datalen; + int r; + + /* Linearize the SOCKS response in the buffer, up to 128 bytes. + * (parse_socks_client shouldn't need to see anything beyond that.) */ + datalen = evbuffer_get_length(buf); + if (datalen > MAX_SOCKS_MESSAGE_LEN) + datalen = MAX_SOCKS_MESSAGE_LEN; + data = evbuffer_pullup(buf, datalen); + + r = parse_socks_client(data, datalen, state, reason, &drain); + if (drain > 0) + evbuffer_drain(buf, drain); + else if (drain < 0) + evbuffer_drain(buf, evbuffer_get_length(buf)); + + return r; +} +#endif + +/** Implementation logic for fetch_from_*_socks_client. */ +static int +parse_socks_client(const uint8_t *data, size_t datalen, + int state, char **reason, + ssize_t *drain_out) +{ + unsigned int addrlen; + *drain_out = 0; + if (datalen < 2) + return 0; switch (state) { case PROXY_SOCKS4_WANT_CONNECT_OK: /* Wait for the complete response */ - if (buf->head->datalen < 8) + if (datalen < 8) return 0; if (data[1] != 0x5a) { @@ -1688,7 +2150,7 @@ fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) } /* Success */ - buf_remove_from_front(buf, 8); + *drain_out = 8; return 1; case PROXY_SOCKS5_WANT_AUTH_METHOD_NONE: @@ -1700,7 +2162,7 @@ fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) } log_info(LD_NET, "SOCKS 5 client: continuing without authentication"); - buf_clear(buf); + *drain_out = -1; return 1; case PROXY_SOCKS5_WANT_AUTH_METHOD_RFC1929: @@ -1710,11 +2172,11 @@ fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) case 0x00: log_info(LD_NET, "SOCKS 5 client: we have auth details but server " "doesn't require authentication."); - buf_clear(buf); + *drain_out = -1; return 1; case 0x02: log_info(LD_NET, "SOCKS 5 client: need authentication."); - buf_clear(buf); + *drain_out = -1; return 2; /* fall through */ } @@ -1731,7 +2193,7 @@ fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) } log_info(LD_NET, "SOCKS 5 client: authentication successful."); - buf_clear(buf); + *drain_out = -1; return 1; case PROXY_SOCKS5_WANT_CONNECT_OK: @@ -1740,7 +2202,7 @@ fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) * the data used */ /* wait for address type field to arrive */ - if (buf->datalen < 4) + if (datalen < 4) return 0; switch (data[3]) { @@ -1751,7 +2213,7 @@ fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) addrlen = 16; break; case 0x03: /* fqdn (can this happen here?) */ - if (buf->datalen < 5) + if (datalen < 5) return 0; addrlen = 1 + data[4]; break; @@ -1761,7 +2223,7 @@ fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) } /* wait for address and port */ - if (buf->datalen < 6 + addrlen) + if (datalen < 6 + addrlen) return 0; if (data[1] != 0x00) { @@ -1769,7 +2231,7 @@ fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) return -1; } - buf_remove_from_front(buf, 6 + addrlen); + *drain_out = 6 + addrlen; return 1; } @@ -1795,6 +2257,27 @@ peek_buf_has_control0_command(buf_t *buf) return 0; } +#ifdef USE_BUFFEREVENTS +int +peek_evbuffer_has_control0_command(struct evbuffer *buf) +{ + int result = 0; + if (evbuffer_get_length(buf) >= 4) { + int free_out = 0; + char *data = NULL; + size_t n = inspect_evbuffer(buf, &data, 4, &free_out, NULL); + uint16_t cmd; + tor_assert(n >= 4); + cmd = ntohs(get_uint16(data+2)); + if (cmd <= 0x14) + result = 1; + if (free_out) + tor_free(data); + } + return result; +} +#endif + /** Return the index within <b>buf</b> at which <b>ch</b> first appears, * or -1 if <b>ch</b> does not appear on buf. */ static off_t @@ -1812,12 +2295,12 @@ buf_find_offset_of_char(buf_t *buf, char ch) return -1; } -/** Try to read a single LF-terminated line from <b>buf</b>, and write it, - * NUL-terminated, into the *<b>data_len</b> byte buffer at <b>data_out</b>. - * Set *<b>data_len</b> to the number of bytes in the line, not counting the - * terminating NUL. Return 1 if we read a whole line, return 0 if we don't - * have a whole line yet, and return -1 if the line length exceeds - * *<b>data_len</b>. +/** Try to read a single LF-terminated line from <b>buf</b>, and write it + * (including the LF), NUL-terminated, into the *<b>data_len</b> byte buffer + * at <b>data_out</b>. Set *<b>data_len</b> to the number of bytes in the + * line, not counting the terminating NUL. Return 1 if we read a whole line, + * return 0 if we don't have a whole line yet, and return -1 if the line + * length exceeds *<b>data_len</b>. */ int fetch_from_buf_line(buf_t *buf, char *data_out, size_t *data_len) @@ -1891,6 +2374,96 @@ write_to_buf_zlib(buf_t *buf, tor_zlib_state_t *state, return 0; } +#ifdef USE_BUFFEREVENTS +int +write_to_evbuffer_zlib(struct evbuffer *buf, tor_zlib_state_t *state, + const char *data, size_t data_len, + int done) +{ + char *next; + size_t old_avail, avail; + int over = 0, n; + struct evbuffer_iovec vec[1]; + do { + { + size_t cap = data_len / 4; + if (cap < 128) + cap = 128; + /* XXXX NM this strategy is fragmentation-prone. We should really have + * two iovecs, and write first into the one, and then into the + * second if the first gets full. */ + n = evbuffer_reserve_space(buf, cap, vec, 1); + tor_assert(n == 1); + } + + next = vec[0].iov_base; + avail = old_avail = vec[0].iov_len; + + switch (tor_zlib_process(state, &next, &avail, &data, &data_len, done)) { + case TOR_ZLIB_DONE: + over = 1; + break; + case TOR_ZLIB_ERR: + return -1; + case TOR_ZLIB_OK: + if (data_len == 0) + over = 1; + break; + case TOR_ZLIB_BUF_FULL: + if (avail) { + /* Zlib says we need more room (ZLIB_BUF_FULL). Start a new chunk + * automatically, whether were going to or not. */ + } + break; + } + + /* XXXX possible infinite loop on BUF_FULL. */ + vec[0].iov_len = old_avail - avail; + evbuffer_commit_space(buf, vec, 1); + + } while (!over); + check(); + return 0; +} +#endif + +/** Set *<b>output</b> to contain a copy of the data in *<b>input</b> */ +int +generic_buffer_set_to_copy(generic_buffer_t **output, + const generic_buffer_t *input) +{ +#ifdef USE_BUFFEREVENTS + struct evbuffer_ptr ptr; + size_t remaining = evbuffer_get_length(input); + if (*output) { + evbuffer_drain(*output, evbuffer_get_length(*output)); + } else { + if (!(*output = evbuffer_new())) + return -1; + } + evbuffer_ptr_set((struct evbuffer*)input, &ptr, 0, EVBUFFER_PTR_SET); + while (remaining) { + struct evbuffer_iovec v[4]; + int n_used, i; + n_used = evbuffer_peek((struct evbuffer*)input, -1, &ptr, v, 4); + if (n_used < 0) + return -1; + for (i=0;i<n_used;++i) { + evbuffer_add(*output, v[i].iov_base, v[i].iov_len); + tor_assert(v[i].iov_len <= remaining); + remaining -= v[i].iov_len; + evbuffer_ptr_set((struct evbuffer*)input, + &ptr, v[i].iov_len, EVBUFFER_PTR_ADD); + } + } +#else + if (*output) + buf_free(*output); + *output = buf_copy(input); +#endif + return 0; +} + /** Log an error and exit if <b>buf</b> is corrupted. */ void diff --git a/src/or/buffers.h b/src/or/buffers.h index 63fab4957a..a5886adc7a 100644 --- a/src/or/buffers.h +++ b/src/or/buffers.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -16,6 +16,7 @@ buf_t *buf_new(void); buf_t *buf_new_with_capacity(size_t size); void buf_free(buf_t *buf); void buf_clear(buf_t *buf); +buf_t *buf_copy(const buf_t *buf); void buf_shrink(buf_t *buf); void buf_shrink_freelists(int free_all); void buf_dump_freelist_sizes(int severity); @@ -41,6 +42,8 @@ int fetch_from_buf_http(buf_t *buf, char **headers_out, size_t max_headerlen, char **body_out, size_t *body_used, size_t max_bodylen, int force_complete); +socks_request_t *socks_request_new(void); +void socks_request_free(socks_request_t *req); int fetch_from_buf_socks(buf_t *buf, socks_request_t *req, int log_sockstype, int safe_socks); int fetch_from_buf_socks_client(buf_t *buf, int state, char **reason); @@ -48,6 +51,41 @@ int fetch_from_buf_line(buf_t *buf, char *data_out, size_t *data_len); int peek_buf_has_control0_command(buf_t *buf); +#ifdef USE_BUFFEREVENTS +int fetch_var_cell_from_evbuffer(struct evbuffer *buf, var_cell_t **out, + int linkproto); +int fetch_from_evbuffer_socks(struct evbuffer *buf, socks_request_t *req, + int log_sockstype, int safe_socks); +int fetch_from_evbuffer_socks_client(struct evbuffer *buf, int state, + char **reason); +int fetch_from_evbuffer_http(struct evbuffer *buf, + char **headers_out, size_t max_headerlen, + char **body_out, size_t *body_used, size_t max_bodylen, + int force_complete); +int peek_evbuffer_has_control0_command(struct evbuffer *buf); +int write_to_evbuffer_zlib(struct evbuffer *buf, tor_zlib_state_t *state, + const char *data, size_t data_len, + int done); +#endif + +#ifdef USE_BUFFEREVENTS +#define generic_buffer_new() evbuffer_new() +#define generic_buffer_len(b) evbuffer_get_length((b)) +#define generic_buffer_add(b,dat,len) evbuffer_add((b),(dat),(len)) +#define generic_buffer_get(b,buf,buflen) evbuffer_remove((b),(buf),(buflen)) +#define generic_buffer_clear(b) evbuffer_drain((b), evbuffer_get_length((b))) +#define generic_buffer_free(b) evbuffer_free((b)) +#else +#define generic_buffer_new() buf_new() +#define generic_buffer_len(b) buf_datalen((b)) +#define generic_buffer_add(b,dat,len) write_to_buf((dat),(len),(b)) +#define generic_buffer_get(b,buf,buflen) fetch_from_buf((buf),(buflen),(b)) +#define generic_buffer_clear(b) buf_clear((b)) +#define generic_buffer_free(b) buf_free((b)) +#endif +int generic_buffer_set_to_copy(generic_buffer_t **output, + const generic_buffer_t *input); + void assert_buf_ok(buf_t *buf); #ifdef BUFFERS_PRIVATE diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index decb18fc11..f8521c5cff 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -23,8 +23,10 @@ #include "directory.h" #include "main.h" #include "networkstatus.h" +#include "nodelist.h" #include "onion.h" #include "policies.h" +#include "transports.h" #include "relay.h" #include "rephist.h" #include "router.h" @@ -42,12 +44,12 @@ /********* START VARIABLES **********/ /** Global list of circuit build times */ -// XXXX023: Add this as a member for entry_guard_t instead of global? +// XXXX: Add this as a member for entry_guard_t instead of global? // Then we could do per-guard statistics, as guards are likely to // vary in their own latency. The downside of this is that guards // can change frequently, so we'd be building a lot more circuits // most likely. -/* XXXX023 Make this static; add accessor functions. */ +/* XXXX024 Make this static; add accessor functions. */ circuit_build_times_t circ_times; /** A global list of all circuits at this hop. */ @@ -55,8 +57,8 @@ extern circuit_t *global_circuitlist; /** An entry_guard_t represents our information about a chosen long-term * first hop, known as a "helper" node in the literature. We can't just - * use a routerinfo_t, since we want to remember these even when we - * don't have a directory. */ + * use a node_t, since we want to remember these even when we + * don't have any directory info. */ typedef struct { char nickname[MAX_NICKNAME_LEN+1]; char identity[DIGEST_LEN]; @@ -68,6 +70,10 @@ typedef struct { * router, 1 if we have. */ unsigned int can_retry : 1; /**< Should we retry connecting to this entry, * in spite of having it marked as unreachable?*/ + unsigned int path_bias_notice : 1; /**< Did we alert the user about path bias + * for this node already? */ + unsigned int path_bias_disabled : 1; /**< Have we disabled this node because + * of path bias issues? */ time_t bad_since; /**< 0 if this guard is currently usable, or the time at * which it was observed to become (according to the * directory or the user configuration) unusable. */ @@ -76,8 +82,34 @@ typedef struct { * connect to it. */ time_t last_attempted; /**< 0 if we can connect to this guard, or the time * at which we last failed to connect to it. */ + + unsigned first_hops; /**< Number of first hops this guard has completed */ + unsigned circuit_successes; /**< Number of successfully built circuits using + * this guard as first hop. */ } entry_guard_t; +/** Information about a configured bridge. Currently this just matches the + * ones in the torrc file, but one day we may be able to learn about new + * bridges on our own, and remember them in the state file. */ +typedef struct { + /** Address of the bridge. */ + tor_addr_t addr; + /** TLS port for the bridge. */ + uint16_t port; + /** Boolean: We are re-parsing our bridge list, and we are going to remove + * this one if we don't find it in the list of configured bridges. */ + unsigned marked_for_removal : 1; + /** Expected identity digest, or all zero bytes if we don't know what the + * digest should be. */ + char identity[DIGEST_LEN]; + + /** Name of pluggable transport protocol taken from its config line. */ + char *transport_name; + + /** When should we next try to fetch a descriptor for this bridge? */ + download_status_t fetch_status; +} bridge_info_t; + /** A list of our chosen entry guards. */ static smartlist_t *entry_guards = NULL; /** A value of 1 means that the entry_guards list has changed @@ -95,10 +127,16 @@ static int circuit_deliver_create_cell(circuit_t *circ, static int onion_pick_cpath_exit(origin_circuit_t *circ, extend_info_t *exit); static crypt_path_t *onion_next_hop_in_cpath(crypt_path_t *cpath); static int onion_extend_cpath(origin_circuit_t *circ); -static int count_acceptable_routers(smartlist_t *routers); +static int count_acceptable_nodes(smartlist_t *routers); static int onion_append_hop(crypt_path_t **head_ptr, extend_info_t *choice); static void entry_guards_changed(void); +static entry_guard_t *entry_guard_get_by_id_digest(const char *digest); + +static void bridge_free(bridge_info_t *bridge); + +static int entry_guard_inc_first_hop_count(entry_guard_t *guard); +static void pathbias_count_success(origin_circuit_t *circ); /** * This function decides if CBT learning should be disabled. It returns @@ -123,13 +161,18 @@ circuit_build_times_disabled(void) if (consensus_disabled || config_disabled || dirauth_disabled || state_disabled) { - log_info(LD_CIRC, + log_debug(LD_CIRC, "CircuitBuildTime learning is disabled. " "Consensus=%d, Config=%d, AuthDir=%d, StateFile=%d", consensus_disabled, config_disabled, dirauth_disabled, state_disabled); return 1; } else { + log_debug(LD_CIRC, + "CircuitBuildTime learning is not disabled. " + "Consensus=%d, Config=%d, AuthDir=%d, StateFile=%d", + consensus_disabled, config_disabled, dirauth_disabled, + state_disabled); return 0; } } @@ -145,10 +188,21 @@ circuit_build_times_disabled(void) static int32_t circuit_build_times_max_timeouts(void) { - return networkstatus_get_param(NULL, "cbtmaxtimeouts", + int32_t cbt_maxtimeouts; + + cbt_maxtimeouts = networkstatus_get_param(NULL, "cbtmaxtimeouts", CBT_DEFAULT_MAX_RECENT_TIMEOUT_COUNT, CBT_MIN_MAX_RECENT_TIMEOUT_COUNT, CBT_MAX_MAX_RECENT_TIMEOUT_COUNT); + + if (!(get_options()->LearnCircuitBuildTimeout)) { + log_debug(LD_BUG, + "circuit_build_times_max_timeouts() called, cbtmaxtimeouts is" + " %d", + cbt_maxtimeouts); + } + + return cbt_maxtimeouts; } /** @@ -167,6 +221,14 @@ circuit_build_times_default_num_xm_modes(void) CBT_DEFAULT_NUM_XM_MODES, CBT_MIN_NUM_XM_MODES, CBT_MAX_NUM_XM_MODES); + + if (!(get_options()->LearnCircuitBuildTimeout)) { + log_debug(LD_BUG, + "circuit_build_times_default_num_xm_modes() called, cbtnummodes" + " is %d", + num); + } + return num; } @@ -183,6 +245,14 @@ circuit_build_times_min_circs_to_observe(void) CBT_DEFAULT_MIN_CIRCUITS_TO_OBSERVE, CBT_MIN_MIN_CIRCUITS_TO_OBSERVE, CBT_MAX_MIN_CIRCUITS_TO_OBSERVE); + + if (!(get_options()->LearnCircuitBuildTimeout)) { + log_debug(LD_BUG, + "circuit_build_times_min_circs_to_observe() called, cbtmincircs" + " is %d", + num); + } + return num; } @@ -207,9 +277,18 @@ circuit_build_times_quantile_cutoff(void) CBT_DEFAULT_QUANTILE_CUTOFF, CBT_MIN_QUANTILE_CUTOFF, CBT_MAX_QUANTILE_CUTOFF); + + if (!(get_options()->LearnCircuitBuildTimeout)) { + log_debug(LD_BUG, + "circuit_build_times_quantile_cutoff() called, cbtquantile" + " is %d", + num); + } + return num/100.0; } +/* DOCDOC circuit_build_times_get_bw_scale */ int circuit_build_times_get_bw_scale(networkstatus_t *ns) { @@ -236,6 +315,13 @@ circuit_build_times_close_quantile(void) CBT_DEFAULT_CLOSE_QUANTILE, CBT_MIN_CLOSE_QUANTILE, CBT_MAX_CLOSE_QUANTILE); + + if (!(get_options()->LearnCircuitBuildTimeout)) { + log_debug(LD_BUG, + "circuit_build_times_close_quantile() called, cbtclosequantile" + " is %d", param); + } + if (param < min) { log_warn(LD_DIR, "Consensus parameter cbtclosequantile is " "too small, raising to %d", min); @@ -258,6 +344,13 @@ circuit_build_times_test_frequency(void) CBT_DEFAULT_TEST_FREQUENCY, CBT_MIN_TEST_FREQUENCY, CBT_MAX_TEST_FREQUENCY); + + if (!(get_options()->LearnCircuitBuildTimeout)) { + log_debug(LD_BUG, + "circuit_build_times_test_frequency() called, cbttestfreq is %d", + num); + } + return num; } @@ -275,6 +368,13 @@ circuit_build_times_min_timeout(void) CBT_DEFAULT_TIMEOUT_MIN_VALUE, CBT_MIN_TIMEOUT_MIN_VALUE, CBT_MAX_TIMEOUT_MIN_VALUE); + + if (!(get_options()->LearnCircuitBuildTimeout)) { + log_debug(LD_BUG, + "circuit_build_times_min_timeout() called, cbtmintimeout is %d", + num); + } + return num; } @@ -292,6 +392,14 @@ circuit_build_times_initial_timeout(void) CBT_DEFAULT_TIMEOUT_INITIAL_VALUE, CBT_MIN_TIMEOUT_INITIAL_VALUE, CBT_MAX_TIMEOUT_INITIAL_VALUE); + + if (!(get_options()->LearnCircuitBuildTimeout)) { + log_debug(LD_BUG, + "circuit_build_times_initial_timeout() called, " + "cbtinitialtimeout is %d", + param); + } + if (param < min) { log_warn(LD_DIR, "Consensus parameter cbtinitialtimeout is too small, " "raising to %d", min); @@ -310,10 +418,20 @@ circuit_build_times_initial_timeout(void) static int32_t circuit_build_times_recent_circuit_count(networkstatus_t *ns) { - return networkstatus_get_param(ns, "cbtrecentcount", - CBT_DEFAULT_RECENT_CIRCUITS, - CBT_MIN_RECENT_CIRCUITS, - CBT_MAX_RECENT_CIRCUITS); + int32_t num; + num = networkstatus_get_param(ns, "cbtrecentcount", + CBT_DEFAULT_RECENT_CIRCUITS, + CBT_MIN_RECENT_CIRCUITS, + CBT_MAX_RECENT_CIRCUITS); + + if (!(get_options()->LearnCircuitBuildTimeout)) { + log_debug(LD_BUG, + "circuit_build_times_recent_circuit_count() called, " + "cbtrecentcount is %d", + num); + } + + return num; } /** @@ -326,41 +444,79 @@ void circuit_build_times_new_consensus_params(circuit_build_times_t *cbt, networkstatus_t *ns) { - int32_t num = circuit_build_times_recent_circuit_count(ns); + int32_t num; + + /* + * First check if we're doing adaptive timeouts at all; nothing to + * update if we aren't. + */ - if (num > 0 && num != cbt->liveness.num_recent_circs) { - int8_t *recent_circs; - log_notice(LD_CIRC, "The Tor Directory Consensus has changed how many " - "circuits we must track to detect network failures from %d " - "to %d.", cbt->liveness.num_recent_circs, num); + if (!circuit_build_times_disabled()) { + num = circuit_build_times_recent_circuit_count(ns); - tor_assert(cbt->liveness.timeouts_after_firsthop); + if (num > 0) { + if (num != cbt->liveness.num_recent_circs) { + int8_t *recent_circs; + log_notice(LD_CIRC, "The Tor Directory Consensus has changed how many " + "circuits we must track to detect network failures from %d " + "to %d.", cbt->liveness.num_recent_circs, num); + tor_assert(cbt->liveness.timeouts_after_firsthop || + cbt->liveness.num_recent_circs == 0); + + /* + * Technically this is a circular array that we are reallocating + * and memcopying. However, since it only consists of either 1s + * or 0s, and is only used in a statistical test to determine when + * we should discard our history after a sufficient number of 1's + * have been reached, it is fine if order is not preserved or + * elements are lost. + * + * cbtrecentcount should only be changing in cases of severe network + * distress anyway, so memory correctness here is paramount over + * doing acrobatics to preserve the array. + */ + recent_circs = tor_malloc_zero(sizeof(int8_t)*num); + if (cbt->liveness.timeouts_after_firsthop && + cbt->liveness.num_recent_circs > 0) { + memcpy(recent_circs, cbt->liveness.timeouts_after_firsthop, + sizeof(int8_t)*MIN(num, cbt->liveness.num_recent_circs)); + } + + // Adjust the index if it needs it. + if (num < cbt->liveness.num_recent_circs) { + cbt->liveness.after_firsthop_idx = MIN(num-1, + cbt->liveness.after_firsthop_idx); + } + + tor_free(cbt->liveness.timeouts_after_firsthop); + cbt->liveness.timeouts_after_firsthop = recent_circs; + cbt->liveness.num_recent_circs = num; + } + /* else no change, nothing to do */ + } else { /* num == 0 */ + /* + * Weird. This probably shouldn't happen, so log a warning, but try + * to do something sensible anyway. + */ + + log_warn(LD_CIRC, + "The cbtrecentcircs consensus parameter came back zero! " + "This disables adaptive timeouts since we can't keep track of " + "any recent circuits."); + + circuit_build_times_free_timeouts(cbt); + } + } else { /* - * Technically this is a circular array that we are reallocating - * and memcopying. However, since it only consists of either 1s - * or 0s, and is only used in a statistical test to determine when - * we should discard our history after a sufficient number of 1's - * have been reached, it is fine if order is not preserved or - * elements are lost. - * - * cbtrecentcount should only be changing in cases of severe network - * distress anyway, so memory correctness here is paramount over - * doing acrobatics to preserve the array. + * Adaptive timeouts are disabled; this might be because of the + * LearnCircuitBuildTimes config parameter, and hence permanent, or + * the cbtdisabled consensus parameter, so it may be a new condition. + * Treat it like getting num == 0 above and free the circuit history + * if we have any. */ - recent_circs = tor_malloc_zero(sizeof(int8_t)*num); - memcpy(recent_circs, cbt->liveness.timeouts_after_firsthop, - sizeof(int8_t)*MIN(num, cbt->liveness.num_recent_circs)); - - // Adjust the index if it needs it. - if (num < cbt->liveness.num_recent_circs) { - cbt->liveness.after_firsthop_idx = MIN(num-1, - cbt->liveness.after_firsthop_idx); - } - tor_free(cbt->liveness.timeouts_after_firsthop); - cbt->liveness.timeouts_after_firsthop = recent_circs; - cbt->liveness.num_recent_circs = num; + circuit_build_times_free_timeouts(cbt); } } @@ -379,16 +535,26 @@ static double circuit_build_times_get_initial_timeout(void) { double timeout; - if (!unit_tests && get_options()->CircuitBuildTimeout) { - timeout = get_options()->CircuitBuildTimeout*1000; - if (timeout < circuit_build_times_min_timeout()) { - log_warn(LD_CIRC, "Config CircuitBuildTimeout too low. Setting to %ds", - circuit_build_times_min_timeout()/1000); - timeout = circuit_build_times_min_timeout(); + + /* + * Check if we have LearnCircuitBuildTimeout, and if we don't, + * always use CircuitBuildTimeout, no questions asked. + */ + if (get_options()->LearnCircuitBuildTimeout) { + if (!unit_tests && get_options()->CircuitBuildTimeout) { + timeout = get_options()->CircuitBuildTimeout*1000; + if (timeout < circuit_build_times_min_timeout()) { + log_warn(LD_CIRC, "Config CircuitBuildTimeout too low. Setting to %ds", + circuit_build_times_min_timeout()/1000); + timeout = circuit_build_times_min_timeout(); + } + } else { + timeout = circuit_build_times_initial_timeout(); } } else { - timeout = circuit_build_times_initial_timeout(); + timeout = get_options()->CircuitBuildTimeout*1000; } + return timeout; } @@ -417,14 +583,40 @@ void circuit_build_times_init(circuit_build_times_t *cbt) { memset(cbt, 0, sizeof(*cbt)); - cbt->liveness.num_recent_circs = + /* + * Check if we really are using adaptive timeouts, and don't keep + * track of this stuff if not. + */ + if (!circuit_build_times_disabled()) { + cbt->liveness.num_recent_circs = circuit_build_times_recent_circuit_count(NULL); - cbt->liveness.timeouts_after_firsthop = tor_malloc_zero(sizeof(int8_t)* - cbt->liveness.num_recent_circs); + cbt->liveness.timeouts_after_firsthop = + tor_malloc_zero(sizeof(int8_t)*cbt->liveness.num_recent_circs); + } else { + cbt->liveness.num_recent_circs = 0; + cbt->liveness.timeouts_after_firsthop = NULL; + } cbt->close_ms = cbt->timeout_ms = circuit_build_times_get_initial_timeout(); control_event_buildtimeout_set(cbt, BUILDTIMEOUT_SET_EVENT_RESET); } +/** + * Free the saved timeouts, if the cbtdisabled consensus parameter got turned + * on or something. + */ + +void +circuit_build_times_free_timeouts(circuit_build_times_t *cbt) +{ + if (!cbt) return; + + if (cbt->liveness.timeouts_after_firsthop) { + tor_free(cbt->liveness.timeouts_after_firsthop); + } + + cbt->liveness.num_recent_circs = 0; +} + #if 0 /** * Rewind our build time history by n positions. @@ -649,8 +841,7 @@ circuit_build_times_update_state(circuit_build_times_t *cbt, if (histogram[i] == 0) continue; *next = line = tor_malloc_zero(sizeof(config_line_t)); line->key = tor_strdup("CircuitBuildTimeBin"); - line->value = tor_malloc(25); - tor_snprintf(line->value, 25, "%d %d", + tor_asprintf(&line->value, "%d %d", CBT_BIN_TO_MS(i), histogram[i]); next = &(line->next); } @@ -768,7 +959,7 @@ circuit_build_times_parse_state(circuit_build_times_t *cbt, loaded_times = tor_malloc_zero(sizeof(build_time_t)*state->TotalBuildTimes); for (line = state->BuildtimeHistogram; line; line = line->next) { - smartlist_t *args = smartlist_create(); + smartlist_t *args = smartlist_new(); smartlist_split_string(args, line->value, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); if (smartlist_len(args) < 2) { @@ -1108,9 +1299,14 @@ circuit_build_times_network_is_live(circuit_build_times_t *cbt) void circuit_build_times_network_circ_success(circuit_build_times_t *cbt) { - cbt->liveness.timeouts_after_firsthop[cbt->liveness.after_firsthop_idx] = 0; - cbt->liveness.after_firsthop_idx++; - cbt->liveness.after_firsthop_idx %= cbt->liveness.num_recent_circs; + /* Check for NULLness because we might not be using adaptive timeouts */ + if (cbt->liveness.timeouts_after_firsthop && + cbt->liveness.num_recent_circs > 0) { + cbt->liveness.timeouts_after_firsthop[cbt->liveness.after_firsthop_idx] + = 0; + cbt->liveness.after_firsthop_idx++; + cbt->liveness.after_firsthop_idx %= cbt->liveness.num_recent_circs; + } } /** @@ -1125,10 +1321,15 @@ static void circuit_build_times_network_timeout(circuit_build_times_t *cbt, int did_onehop) { - if (did_onehop) { - cbt->liveness.timeouts_after_firsthop[cbt->liveness.after_firsthop_idx]=1; - cbt->liveness.after_firsthop_idx++; - cbt->liveness.after_firsthop_idx %= cbt->liveness.num_recent_circs; + /* Check for NULLness because we might not be using adaptive timeouts */ + if (cbt->liveness.timeouts_after_firsthop && + cbt->liveness.num_recent_circs > 0) { + if (did_onehop) { + cbt->liveness.timeouts_after_firsthop[cbt->liveness.after_firsthop_idx] + = 1; + cbt->liveness.after_firsthop_idx++; + cbt->liveness.after_firsthop_idx %= cbt->liveness.num_recent_circs; + } } } @@ -1214,10 +1415,13 @@ circuit_build_times_network_check_changed(circuit_build_times_t *cbt) int timeout_count=0; int i; - /* how many of our recent circuits made it to the first hop but then - * timed out? */ - for (i = 0; i < cbt->liveness.num_recent_circs; i++) { - timeout_count += cbt->liveness.timeouts_after_firsthop[i]; + if (cbt->liveness.timeouts_after_firsthop && + cbt->liveness.num_recent_circs > 0) { + /* how many of our recent circuits made it to the first hop but then + * timed out? */ + for (i = 0; i < cbt->liveness.num_recent_circs; i++) { + timeout_count += cbt->liveness.timeouts_after_firsthop[i]; + } } /* If 80% of our recent circuits are timing out after the first hop, @@ -1227,9 +1431,12 @@ circuit_build_times_network_check_changed(circuit_build_times_t *cbt) } circuit_build_times_reset(cbt); - memset(cbt->liveness.timeouts_after_firsthop, 0, - sizeof(*cbt->liveness.timeouts_after_firsthop)* - cbt->liveness.num_recent_circs); + if (cbt->liveness.timeouts_after_firsthop && + cbt->liveness.num_recent_circs > 0) { + memset(cbt->liveness.timeouts_after_firsthop, 0, + sizeof(*cbt->liveness.timeouts_after_firsthop)* + cbt->liveness.num_recent_circs); + } cbt->liveness.after_firsthop_idx = 0; /* Check to see if this has happened before. If so, double the timeout @@ -1410,11 +1617,17 @@ circuit_build_times_set_timeout(circuit_build_times_t *cbt) long prev_timeout = tor_lround(cbt->timeout_ms/1000); double timeout_rate; + /* + * Just return if we aren't using adaptive timeouts + */ + if (circuit_build_times_disabled()) + return; + if (!circuit_build_times_set_timeout_worker(cbt)) return; if (cbt->timeout_ms < circuit_build_times_min_timeout()) { - log_warn(LD_CIRC, "Set buildtimeout to low value %fms. Setting to %dms", + log_info(LD_CIRC, "Set buildtimeout to low value %fms. Setting to %dms", cbt->timeout_ms, circuit_build_times_min_timeout()); cbt->timeout_ms = circuit_build_times_min_timeout(); if (cbt->close_ms < cbt->timeout_ms) { @@ -1515,27 +1728,24 @@ circuit_list_path_impl(origin_circuit_t *circ, int verbose, int verbose_names) const char *states[] = {"closed", "waiting for keys", "open"}; char *s; - elements = smartlist_create(); + elements = smartlist_new(); if (verbose) { const char *nickname = build_state_get_exit_nickname(circ->build_state); - char *cp; - tor_asprintf(&cp, "%s%s circ (length %d%s%s):", + smartlist_add_asprintf(elements, "%s%s circ (length %d%s%s):", circ->build_state->is_internal ? "internal" : "exit", circ->build_state->need_uptime ? " (high-uptime)" : "", circ->build_state->desired_path_len, circ->_base.state == CIRCUIT_STATE_OPEN ? "" : ", last hop ", circ->_base.state == CIRCUIT_STATE_OPEN ? "" : (nickname?nickname:"*unnamed*")); - smartlist_add(elements, cp); } hop = circ->cpath; do { - routerinfo_t *ri; - routerstatus_t *rs; char *elt; const char *id; + const node_t *node; if (!hop) break; if (!verbose && hop->state != CPATH_STATE_OPEN) @@ -1545,10 +1755,8 @@ circuit_list_path_impl(origin_circuit_t *circ, int verbose, int verbose_names) id = hop->extend_info->identity_digest; if (verbose_names) { elt = tor_malloc(MAX_VERBOSE_NICKNAME_LEN+1); - if ((ri = router_get_by_digest(id))) { - router_get_verbose_nickname(elt, ri); - } else if ((rs = router_get_consensus_status_by_id(id))) { - routerstatus_get_verbose_nickname(elt, rs); + if ((node = node_get_by_id(id))) { + node_get_verbose_nickname(node, elt); } else if (is_legal_nickname(hop->extend_info->nickname)) { elt[0] = '$'; base16_encode(elt+1, HEX_DIGEST_LEN+1, id, DIGEST_LEN); @@ -1560,9 +1768,9 @@ circuit_list_path_impl(origin_circuit_t *circ, int verbose, int verbose_names) base16_encode(elt+1, HEX_DIGEST_LEN+1, id, DIGEST_LEN); } } else { /* ! verbose_names */ - if ((ri = router_get_by_digest(id)) && - ri->is_named) { - elt = tor_strdup(hop->extend_info->nickname); + node = node_get_by_id(id); + if (node && node_is_named(node)) { + elt = tor_strdup(node_get_nickname(node)); } else { elt = tor_malloc(HEX_DIGEST_LEN+2); elt[0] = '$'; @@ -1571,11 +1779,8 @@ circuit_list_path_impl(origin_circuit_t *circ, int verbose, int verbose_names) } tor_assert(elt); if (verbose) { - size_t len = strlen(elt)+2+strlen(states[hop->state])+1; - char *v = tor_malloc(len); tor_assert(hop->state <= 2); - tor_snprintf(v,len,"%s(%s)",elt,states[hop->state]); - smartlist_add(elements, v); + smartlist_add_asprintf(elements,"%s(%s)",elt,states[hop->state]); tor_free(elt); } else { smartlist_add(elements, elt); @@ -1631,31 +1836,28 @@ void circuit_rep_hist_note_result(origin_circuit_t *circ) { crypt_path_t *hop; - char *prev_digest = NULL; - routerinfo_t *router; + const char *prev_digest = NULL; hop = circ->cpath; if (!hop) /* circuit hasn't started building yet. */ return; if (server_mode(get_options())) { - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); if (!me) return; prev_digest = me->cache_info.identity_digest; } do { - router = router_get_by_digest(hop->extend_info->identity_digest); - if (router) { + const node_t *node = node_get_by_id(hop->extend_info->identity_digest); + if (node) { /* Why do we check this? We know the identity. -NM XXXX */ if (prev_digest) { if (hop->state == CPATH_STATE_OPEN) - rep_hist_note_extend_succeeded(prev_digest, - router->cache_info.identity_digest); + rep_hist_note_extend_succeeded(prev_digest, node->identity); else { - rep_hist_note_extend_failed(prev_digest, - router->cache_info.identity_digest); + rep_hist_note_extend_failed(prev_digest, node->identity); break; } } - prev_digest = router->cache_info.identity_digest; + prev_digest = node->identity; } else { prev_digest = NULL; } @@ -1811,7 +2013,7 @@ circuit_n_conn_done(or_connection_t *or_conn, int status) or_conn->nickname ? or_conn->nickname : "NULL", or_conn->_base.address, status); - pending_circs = smartlist_create(); + pending_circs = smartlist_new(); circuit_get_all_pending_on_or_conn(pending_circs, or_conn); SMARTLIST_FOREACH_BEGIN(pending_circs, circuit_t *, circ) @@ -1924,7 +2126,7 @@ int inform_testing_reachability(void) { char dirbuf[128]; - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); if (!me) return 0; control_event_server_status(LOG_NOTICE, @@ -1953,7 +2155,7 @@ inform_testing_reachability(void) static INLINE int should_use_create_fast_for_circuit(origin_circuit_t *circ) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); tor_assert(circ->cpath); tor_assert(circ->cpath->extend_info); @@ -1997,7 +2199,7 @@ int circuit_send_next_onion_skin(origin_circuit_t *circ) { crypt_path_t *hop; - routerinfo_t *router; + const node_t *node; char payload[2+4+DIGEST_LEN+ONIONSKIN_CHALLENGE_LEN]; char *onionskin; size_t payload_len; @@ -2013,7 +2215,7 @@ circuit_send_next_onion_skin(origin_circuit_t *circ) else control_event_bootstrap(BOOTSTRAP_STATUS_CIRCUIT_CREATE, 0); - router = router_get_by_digest(circ->_base.n_conn->identity_digest); + node = node_get_by_id(circ->_base.n_conn->identity_digest); fast = should_use_create_fast_for_circuit(circ); if (!fast) { /* We are an OR and we know the right onion key: we should @@ -2047,7 +2249,7 @@ circuit_send_next_onion_skin(origin_circuit_t *circ) circuit_set_state(TO_CIRCUIT(circ), CIRCUIT_STATE_BUILDING); log_info(LD_CIRC,"First hop: finished sending %s cell to '%s'", fast ? "CREATE_FAST" : "CREATE", - router ? router_describe(router) : "<unnamed>"); + node ? node_describe(node) : "<unnamed>"); } else { tor_assert(circ->cpath->state == CPATH_STATE_OPEN); tor_assert(circ->_base.state == CIRCUIT_STATE_BUILDING); @@ -2086,10 +2288,13 @@ circuit_send_next_onion_skin(origin_circuit_t *circ) } log_info(LD_CIRC,"circuit built!"); circuit_reset_failure_count(0); - if (circ->build_state->onehop_tunnel) + + if (circ->build_state->onehop_tunnel || circ->has_opened) { control_event_bootstrap(BOOTSTRAP_STATUS_REQUESTING_STATUS, 0); + } + if (!can_complete_circuit && !circ->build_state->onehop_tunnel) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); can_complete_circuit=1; /* FFFF Log a count of known routers here */ log_notice(LD_GENERAL, @@ -2097,11 +2302,14 @@ circuit_send_next_onion_skin(origin_circuit_t *circ) "Looks like client functionality is working."); control_event_bootstrap(BOOTSTRAP_STATUS_DONE, 0); control_event_client_status(LOG_NOTICE, "CIRCUIT_ESTABLISHED"); + clear_broken_connection_map(1); if (server_mode(options) && !check_whether_orport_reachable()) { inform_testing_reachability(); consider_testing_reachability(1, 1); } } + + pathbias_count_success(circ); circuit_rep_hist_note_result(circ); circuit_has_opened(circ); /* do other actions as necessary */ @@ -2224,6 +2432,13 @@ circuit_extend(cell_t *cell, circuit_t *circ) return -1; } + if (tor_addr_is_internal(&n_addr, 0) && + !get_options()->ExtendAllowPrivateAddresses) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Client asked me to extend to a private address"); + return -1; + } + /* Check if they asked us for 0000..0000. We support using * an empty fingerprint for the first hop (e.g. for a bridge relay), * but we don't want to let people send us extend cells for empty @@ -2305,26 +2520,26 @@ int circuit_init_cpath_crypto(crypt_path_t *cpath, const char *key_data, int reverse) { - crypto_digest_env_t *tmp_digest; - crypto_cipher_env_t *tmp_crypto; + crypto_digest_t *tmp_digest; + crypto_cipher_t *tmp_crypto; tor_assert(cpath); tor_assert(key_data); tor_assert(!(cpath->f_crypto || cpath->b_crypto || cpath->f_digest || cpath->b_digest)); - cpath->f_digest = crypto_new_digest_env(); + cpath->f_digest = crypto_digest_new(); crypto_digest_add_bytes(cpath->f_digest, key_data, DIGEST_LEN); - cpath->b_digest = crypto_new_digest_env(); + cpath->b_digest = crypto_digest_new(); crypto_digest_add_bytes(cpath->b_digest, key_data+DIGEST_LEN, DIGEST_LEN); if (!(cpath->f_crypto = - crypto_create_init_cipher(key_data+(2*DIGEST_LEN),1))) { + crypto_cipher_new(key_data+(2*DIGEST_LEN)))) { log_warn(LD_BUG,"Forward cipher initialization failed."); return -1; } if (!(cpath->b_crypto = - crypto_create_init_cipher(key_data+(2*DIGEST_LEN)+CIPHER_KEY_LEN,0))) { + crypto_cipher_new(key_data+(2*DIGEST_LEN)+CIPHER_KEY_LEN))) { log_warn(LD_BUG,"Backward cipher initialization failed."); return -1; } @@ -2341,6 +2556,346 @@ circuit_init_cpath_crypto(crypt_path_t *cpath, const char *key_data, return 0; } +/** The minimum number of first hop completions before we start + * thinking about warning about path bias and dropping guards */ +static int +pathbias_get_min_circs(const or_options_t *options) +{ +#define DFLT_PATH_BIAS_MIN_CIRC 20 + if (options->PathBiasCircThreshold >= 5) + return options->PathBiasCircThreshold; + else + return networkstatus_get_param(NULL, "pb_mincircs", + DFLT_PATH_BIAS_MIN_CIRC, + 5, INT32_MAX); +} + +static double +pathbias_get_notice_rate(const or_options_t *options) +{ +#define DFLT_PATH_BIAS_NOTICE_PCT 40 + if (options->PathBiasNoticeRate >= 0.0) + return options->PathBiasNoticeRate; + else + return networkstatus_get_param(NULL, "pb_noticepct", + DFLT_PATH_BIAS_NOTICE_PCT, 0, 100)/100.0; +} + +static double +pathbias_get_disable_rate(const or_options_t *options) +{ +// XXX: This needs tuning based on use + experimentation before we set it +#define DFLT_PATH_BIAS_DISABLE_PCT 0 + if (options->PathBiasDisableRate >= 0.0) + return options->PathBiasDisableRate; + else + return networkstatus_get_param(NULL, "pb_disablepct", + DFLT_PATH_BIAS_DISABLE_PCT, 0, 100)/100.0; +} + +static int +pathbias_get_scale_threshold(const or_options_t *options) +{ +#define DFLT_PATH_BIAS_SCALE_THRESHOLD 200 + if (options->PathBiasScaleThreshold >= 2) + return options->PathBiasScaleThreshold; + else + return networkstatus_get_param(NULL, "pb_scalecircs", + DFLT_PATH_BIAS_SCALE_THRESHOLD, 10, + INT32_MAX); +} + +static int +pathbias_get_scale_factor(const or_options_t *options) +{ +#define DFLT_PATH_BIAS_SCALE_FACTOR 4 + if (options->PathBiasScaleFactor >= 1) + return options->PathBiasScaleFactor; + else + return networkstatus_get_param(NULL, "pb_scalefactor", + DFLT_PATH_BIAS_SCALE_THRESHOLD, 1, INT32_MAX); +} + +static const char * +pathbias_state_to_string(path_state_t state) +{ + switch (state) { + case PATH_STATE_NEW_CIRC: + return "new"; + case PATH_STATE_DID_FIRST_HOP: + return "first hop"; + case PATH_STATE_SUCCEEDED: + return "succeeded"; + } + + return "unknown"; +} + +/** + * Check our circuit state to see if this is a successful first hop. + * If so, record it in the current guard's path bias first_hop count. + * + * Also check for several potential error cases for bug #6475. + */ +static int +pathbias_count_first_hop(origin_circuit_t *circ) +{ +#define FIRST_HOP_NOTICE_INTERVAL (600) + static ratelim_t first_hop_notice_limit = + RATELIM_INIT(FIRST_HOP_NOTICE_INTERVAL); + char *rate_msg = NULL; + + /* Completely ignore one hop circuits */ + if (circ->build_state->onehop_tunnel || + circ->build_state->desired_path_len == 1) { + /* Check for inconsistency */ + if (circ->build_state->desired_path_len != 1 || + !circ->build_state->onehop_tunnel) { + if ((rate_msg = rate_limit_log(&first_hop_notice_limit, + approx_time()))) { + log_info(LD_BUG, + "One-hop circuit has length %d. Path state is %s. " + "Circuit is a %s currently %s.%s", + circ->build_state->desired_path_len, + pathbias_state_to_string(circ->path_state), + circuit_purpose_to_string(circ->_base.purpose), + circuit_state_to_string(circ->_base.state), + rate_msg); + tor_free(rate_msg); + } + tor_fragile_assert(); + } + return 0; + } + + if (circ->cpath->state == CPATH_STATE_AWAITING_KEYS) { + /* Help track down the real cause of bug #6475: */ + if (circ->has_opened && circ->path_state != PATH_STATE_DID_FIRST_HOP) { + if ((rate_msg = rate_limit_log(&first_hop_notice_limit, + approx_time()))) { + log_info(LD_BUG, + "Opened circuit is in strange path state %s. " + "Circuit is a %s currently %s.%s", + pathbias_state_to_string(circ->path_state), + circuit_purpose_to_string(circ->_base.purpose), + circuit_state_to_string(circ->_base.state), + rate_msg); + tor_free(rate_msg); + } + } + + /* Don't count cannibalized circs for path bias */ + if (!circ->has_opened) { + entry_guard_t *guard; + + guard = entry_guard_get_by_id_digest( + circ->_base.n_conn->identity_digest); + if (guard) { + if (circ->path_state == PATH_STATE_NEW_CIRC) { + circ->path_state = PATH_STATE_DID_FIRST_HOP; + + if (entry_guard_inc_first_hop_count(guard) < 0) { + /* Bogus guard; we already warned. */ + return -END_CIRC_REASON_TORPROTOCOL; + } + } else { + if ((rate_msg = rate_limit_log(&first_hop_notice_limit, + approx_time()))) { + log_info(LD_BUG, + "Unopened circuit has strange path state %s. " + "Circuit is a %s currently %s.%s", + pathbias_state_to_string(circ->path_state), + circuit_purpose_to_string(circ->_base.purpose), + circuit_state_to_string(circ->_base.state), + rate_msg); + tor_free(rate_msg); + } + } + } else { + if ((rate_msg = rate_limit_log(&first_hop_notice_limit, + approx_time()))) { + log_info(LD_BUG, + "Unopened circuit has no known guard. " + "Circuit is a %s currently %s.%s", + circuit_purpose_to_string(circ->_base.purpose), + circuit_state_to_string(circ->_base.state), + rate_msg); + tor_free(rate_msg); + } + } + } + } else { + /* Help track down the real cause of bug #6475: */ + if (circ->path_state == PATH_STATE_NEW_CIRC) { + if ((rate_msg = rate_limit_log(&first_hop_notice_limit, + approx_time()))) { + log_info(LD_BUG, + "A %s circuit is in cpath state %d (opened: %d). " + "Circuit is a %s currently %s.%s", + pathbias_state_to_string(circ->path_state), + circ->cpath->state, circ->has_opened, + circuit_purpose_to_string(circ->_base.purpose), + circuit_state_to_string(circ->_base.state), + rate_msg); + tor_free(rate_msg); + } + } + } + + return 0; +} + +/** + * Check our circuit state to see if this is a successful circuit + * completion. If so, record it in the current guard's path bias + * success count. + * + * Also check for several potential error cases for bug #6475. + */ +static void +pathbias_count_success(origin_circuit_t *circ) +{ +#define SUCCESS_NOTICE_INTERVAL (600) + static ratelim_t success_notice_limit = + RATELIM_INIT(SUCCESS_NOTICE_INTERVAL); + char *rate_msg = NULL; + + /* Ignore one hop circuits */ + if (circ->build_state->onehop_tunnel || + circ->build_state->desired_path_len == 1) { + /* Check for consistency */ + if (circ->build_state->desired_path_len != 1 || + !circ->build_state->onehop_tunnel) { + if ((rate_msg = rate_limit_log(&success_notice_limit, + approx_time()))) { + log_info(LD_BUG, + "One-hop circuit has length %d. Path state is %s. " + "Circuit is a %s currently %s.%s", + circ->build_state->desired_path_len, + pathbias_state_to_string(circ->path_state), + circuit_purpose_to_string(circ->_base.purpose), + circuit_state_to_string(circ->_base.state), + rate_msg); + tor_free(rate_msg); + } + tor_fragile_assert(); + } + return; + } + + /* Don't count cannibalized/reused circs for path bias */ + if (!circ->has_opened) { + entry_guard_t *guard = + entry_guard_get_by_id_digest(circ->_base.n_conn->identity_digest); + + if (guard) { + if (circ->path_state == PATH_STATE_DID_FIRST_HOP) { + circ->path_state = PATH_STATE_SUCCEEDED; + guard->circuit_successes++; + + log_info(LD_PROTOCOL, "Got success count %u/%u for guard %s=%s", + guard->circuit_successes, guard->first_hops, + guard->nickname, hex_str(guard->identity, DIGEST_LEN)); + } else { + if ((rate_msg = rate_limit_log(&success_notice_limit, + approx_time()))) { + log_info(LD_BUG, + "Succeeded circuit is in strange path state %s. " + "Circuit is a %s currently %s.%s", + pathbias_state_to_string(circ->path_state), + circuit_purpose_to_string(circ->_base.purpose), + circuit_state_to_string(circ->_base.state), + rate_msg); + tor_free(rate_msg); + } + } + + if (guard->first_hops < guard->circuit_successes) { + log_info(LD_BUG, "Unexpectedly high circuit_successes (%u/%u) " + "for guard %s=%s", + guard->circuit_successes, guard->first_hops, + guard->nickname, hex_str(guard->identity, DIGEST_LEN)); + } + } else { + if ((rate_msg = rate_limit_log(&success_notice_limit, + approx_time()))) { + log_info(LD_BUG, + "Completed circuit has no known guard. " + "Circuit is a %s currently %s.%s", + circuit_purpose_to_string(circ->_base.purpose), + circuit_state_to_string(circ->_base.state), + rate_msg); + tor_free(rate_msg); + } + } + } else { + if (circ->path_state != PATH_STATE_SUCCEEDED) { + if ((rate_msg = rate_limit_log(&success_notice_limit, + approx_time()))) { + log_info(LD_BUG, + "Opened circuit is in strange path state %s. " + "Circuit is a %s currently %s.%s", + pathbias_state_to_string(circ->path_state), + circuit_purpose_to_string(circ->_base.purpose), + circuit_state_to_string(circ->_base.state), + rate_msg); + tor_free(rate_msg); + } + } + } +} + +/** Increment the number of times we successfully extended a circuit to + * 'guard', first checking if the failure rate is high enough that we should + * eliminate the guard. Return -1 if the guard looks no good; return 0 if the + * guard looks fine. */ +static int +entry_guard_inc_first_hop_count(entry_guard_t *guard) +{ + const or_options_t *options = get_options(); + + entry_guards_changed(); + + if (guard->first_hops > (unsigned)pathbias_get_min_circs(options)) { + /* Note: We rely on the < comparison here to allow us to set a 0 + * rate and disable the feature entirely. If refactoring, don't + * change to <= */ + if (guard->circuit_successes/((double)guard->first_hops) + < pathbias_get_disable_rate(options)) { + + log_info(LD_PROTOCOL, + "Extremely low circuit success rate %u/%u for guard %s=%s. " + "This might indicate an attack, or a bug.", + guard->circuit_successes, guard->first_hops, guard->nickname, + hex_str(guard->identity, DIGEST_LEN)); + + guard->path_bias_disabled = 1; + guard->bad_since = approx_time(); + return -1; + } else if (guard->circuit_successes/((double)guard->first_hops) + < pathbias_get_notice_rate(options) + && !guard->path_bias_notice) { + guard->path_bias_notice = 1; + log_info(LD_PROTOCOL, + "Low circuit success rate %u/%u for guard %s=%s.", + guard->circuit_successes, guard->first_hops, guard->nickname, + hex_str(guard->identity, DIGEST_LEN)); + } + } + + /* If we get a ton of circuits, just scale everything down */ + if (guard->first_hops > (unsigned)pathbias_get_scale_threshold(options)) { + const int scale_factor = pathbias_get_scale_factor(options); + guard->first_hops /= scale_factor; + guard->circuit_successes /= scale_factor; + } + guard->first_hops++; + log_info(LD_PROTOCOL, "Got success count %u/%u for guard %s=%s", + guard->circuit_successes, guard->first_hops, guard->nickname, + hex_str(guard->identity, DIGEST_LEN)); + return 0; +} + /** A created or extended cell came back to us on the circuit, and it included * <b>reply</b> as its body. (If <b>reply_type</b> is CELL_CREATED, the body * contains (the second DH key, plus KH). If <b>reply_type</b> is @@ -2357,10 +2912,14 @@ circuit_finish_handshake(origin_circuit_t *circ, uint8_t reply_type, { char keys[CPATH_KEY_MATERIAL_LEN]; crypt_path_t *hop; + int rv; + + if ((rv = pathbias_count_first_hop(circ)) < 0) + return rv; - if (circ->cpath->state == CPATH_STATE_AWAITING_KEYS) + if (circ->cpath->state == CPATH_STATE_AWAITING_KEYS) { hop = circ->cpath; - else { + } else { hop = onion_next_hop_in_cpath(circ->cpath); if (!hop) { /* got an extended when we're all done? */ log_warn(LD_PROTOCOL,"got extended when circ already built? Closing."); @@ -2503,7 +3062,8 @@ onionskin_answer(or_circuit_t *circ, uint8_t cell_type, const char *payload, append_cell_to_circuit_queue(TO_CIRCUIT(circ), circ->p_conn, &cell, CELL_DIRECTION_IN, 0); - log_debug(LD_CIRC,"Finished sending 'created' cell."); + log_debug(LD_CIRC,"Finished sending '%s' cell.", + circ->is_first_hop ? "created_fast" : "created"); if (!is_local_addr(&circ->p_conn->_base.addr) && !connection_or_nonopen_was_started_here(circ->p_conn)) { @@ -2524,12 +3084,12 @@ onionskin_answer(or_circuit_t *circ, uint8_t cell_type, const char *payload, */ static int new_route_len(uint8_t purpose, extend_info_t *exit, - smartlist_t *routers) + smartlist_t *nodes) { int num_acceptable_routers; int routelen; - tor_assert(routers); + tor_assert(nodes); routelen = DEFAULT_ROUTE_LEN; if (exit && @@ -2537,10 +3097,10 @@ new_route_len(uint8_t purpose, extend_info_t *exit, purpose != CIRCUIT_PURPOSE_S_ESTABLISH_INTRO) routelen++; - num_acceptable_routers = count_acceptable_routers(routers); + num_acceptable_routers = count_acceptable_nodes(nodes); log_debug(LD_CIRC,"Chosen route length %d (%d/%d routers suitable).", - routelen, num_acceptable_routers, smartlist_len(routers)); + routelen, num_acceptable_routers, smartlist_len(nodes)); if (num_acceptable_routers < 2) { log_info(LD_CIRC, @@ -2558,24 +3118,12 @@ new_route_len(uint8_t purpose, extend_info_t *exit, return routelen; } -/** Fetch the list of predicted ports, dup it into a smartlist of - * uint16_t's, remove the ones that are already handled by an - * existing circuit, and return it. - */ +/** Return a newly allocated list of uint16_t * for each predicted port not + * handled by a current circuit. */ static smartlist_t * circuit_get_unhandled_ports(time_t now) { - smartlist_t *source = rep_hist_get_predicted_ports(now); - smartlist_t *dest = smartlist_create(); - uint16_t *tmp; - int i; - - for (i = 0; i < smartlist_len(source); ++i) { - tmp = tor_malloc(sizeof(uint16_t)); - memcpy(tmp, smartlist_get(source, i), sizeof(uint16_t)); - smartlist_add(dest, tmp); - } - + smartlist_t *dest = rep_hist_get_predicted_ports(now); circuit_remove_handled_ports(dest); return dest; } @@ -2609,12 +3157,12 @@ circuit_all_predicted_ports_handled(time_t now, int *need_uptime, return enough; } -/** Return 1 if <b>router</b> can handle one or more of the ports in +/** Return 1 if <b>node</b> can handle one or more of the ports in * <b>needed_ports</b>, else return 0. */ static int -router_handles_some_port(routerinfo_t *router, smartlist_t *needed_ports) -{ +node_handles_some_port(const node_t *node, smartlist_t *needed_ports) +{ /* XXXX MOVE */ int i; uint16_t port; @@ -2624,7 +3172,10 @@ router_handles_some_port(routerinfo_t *router, smartlist_t *needed_ports) needed_ports is explicitly a smartlist of uint16_t's */ port = *(uint16_t *)smartlist_get(needed_ports, i); tor_assert(port); - r = compare_addr_to_addr_policy(0, port, router->exit_policy); + if (node) + r = compare_tor_addr_to_node_policy(NULL, port, node); + else + continue; if (r != ADDR_POLICY_REJECTED && r != ADDR_POLICY_PROBABLY_REJECTED) return 1; } @@ -2636,14 +3187,18 @@ router_handles_some_port(routerinfo_t *router, smartlist_t *needed_ports) static int ap_stream_wants_exit_attention(connection_t *conn) { - if (conn->type == CONN_TYPE_AP && - conn->state == AP_CONN_STATE_CIRCUIT_WAIT && + entry_connection_t *entry; + if (conn->type != CONN_TYPE_AP) + return 0; + entry = TO_ENTRY_CONN(conn); + + if (conn->state == AP_CONN_STATE_CIRCUIT_WAIT && !conn->marked_for_close && - !(TO_EDGE_CONN(conn)->want_onehop) && /* ignore one-hop streams */ - !(TO_EDGE_CONN(conn)->use_begindir) && /* ignore targeted dir fetches */ - !(TO_EDGE_CONN(conn)->chosen_exit_name) && /* ignore defined streams */ + !(entry->want_onehop) && /* ignore one-hop streams */ + !(entry->use_begindir) && /* ignore targeted dir fetches */ + !(entry->chosen_exit_name) && /* ignore defined streams */ !connection_edge_is_rendezvous_stream(TO_EDGE_CONN(conn)) && - !circuit_stream_is_being_handled(TO_EDGE_CONN(conn), 0, + !circuit_stream_is_being_handled(TO_ENTRY_CONN(conn), 0, MIN_CIRCUITS_HANDLING_STREAM)) return 1; return 0; @@ -2657,18 +3212,17 @@ ap_stream_wants_exit_attention(connection_t *conn) * * Return NULL if we can't find any suitable routers. */ -static routerinfo_t * -choose_good_exit_server_general(routerlist_t *dir, int need_uptime, - int need_capacity) +static const node_t * +choose_good_exit_server_general(int need_uptime, int need_capacity) { int *n_supported; - int i; int n_pending_connections = 0; smartlist_t *connections; int best_support = -1; int n_best_support=0; - routerinfo_t *router; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); + const smartlist_t *the_nodes; + const node_t *node=NULL; connections = get_connection_array(); @@ -2689,10 +3243,11 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, * * -1 means "Don't use this router at all." */ - n_supported = tor_malloc(sizeof(int)*smartlist_len(dir->routers)); - for (i = 0; i < smartlist_len(dir->routers); ++i) {/* iterate over routers */ - router = smartlist_get(dir->routers, i); - if (router_is_me(router)) { + the_nodes = nodelist_get_list(); + n_supported = tor_malloc(sizeof(int)*smartlist_len(the_nodes)); + SMARTLIST_FOREACH_BEGIN(the_nodes, const node_t *, node) { + const int i = node_sl_idx; + if (router_digest_is_me(node->identity)) { n_supported[i] = -1; // log_fn(LOG_DEBUG,"Skipping node %s -- it's me.", router->nickname); /* XXX there's probably a reverse predecessor attack here, but @@ -2700,45 +3255,49 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, */ continue; } - if (!router->is_running || router->is_bad_exit) { + if (!node_has_descriptor(node)) { + n_supported[i] = -1; + continue; + } + if (!node->is_running || node->is_bad_exit) { n_supported[i] = -1; continue; /* skip routers that are known to be down or bad exits */ } - if (router->purpose != ROUTER_PURPOSE_GENERAL) { + if (node_get_purpose(node) != ROUTER_PURPOSE_GENERAL) { /* never pick a non-general node as a random exit. */ n_supported[i] = -1; continue; } - if (options->_ExcludeExitNodesUnion && - routerset_contains_router(options->_ExcludeExitNodesUnion, router)) { + if (routerset_contains_node(options->_ExcludeExitNodesUnion, node)) { n_supported[i] = -1; continue; /* user asked us not to use it, no matter what */ } if (options->ExitNodes && - !routerset_contains_router(options->ExitNodes, router)) { + !routerset_contains_node(options->ExitNodes, node)) { n_supported[i] = -1; continue; /* not one of our chosen exit nodes */ } - if (router_is_unreliable(router, need_uptime, need_capacity, 0)) { + if (node_is_unreliable(node, need_uptime, need_capacity, 0)) { n_supported[i] = -1; continue; /* skip routers that are not suitable. Don't worry if * this makes us reject all the possible routers: if so, * we'll retry later in this function with need_update and * need_capacity set to 0. */ } - if (!(router->is_valid || options->_AllowInvalid & ALLOW_INVALID_EXIT)) { + if (!(node->is_valid || options->_AllowInvalid & ALLOW_INVALID_EXIT)) { /* if it's invalid and we don't want it */ n_supported[i] = -1; // log_fn(LOG_DEBUG,"Skipping node %s (index %d) -- invalid router.", // router->nickname, i); continue; /* skip invalid routers */ } - if (options->ExcludeSingleHopRelays && router->allow_single_hop_exits) { + if (options->ExcludeSingleHopRelays && + node_allows_single_hop_exits(node)) { n_supported[i] = -1; continue; } - if (router_exit_policy_rejects_all(router)) { + if (node_exit_policy_rejects_all(node)) { n_supported[i] = -1; // log_fn(LOG_DEBUG,"Skipping node %s (index %d) -- it rejects all.", // router->nickname, i); @@ -2746,11 +3305,10 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, } n_supported[i] = 0; /* iterate over connections */ - SMARTLIST_FOREACH(connections, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(connections, connection_t *, conn) { if (!ap_stream_wants_exit_attention(conn)) continue; /* Skip everything but APs in CIRCUIT_WAIT */ - if (connection_ap_can_use_exit(TO_EDGE_CONN(conn), router)) { + if (connection_ap_can_use_exit(TO_ENTRY_CONN(conn), node)) { ++n_supported[i]; // log_fn(LOG_DEBUG,"%s is supported. n_supported[%d] now %d.", // router->nickname, i, n_supported[i]); @@ -2758,7 +3316,7 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, // log_fn(LOG_DEBUG,"%s (index %d) would reject this stream.", // router->nickname, i); } - }); /* End looping over connections. */ + } SMARTLIST_FOREACH_END(conn); if (n_pending_connections > 0 && n_supported[i] == 0) { /* Leave best_support at -1 if that's where it is, so we can * distinguish it later. */ @@ -2775,7 +3333,7 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, * count of equally good routers.*/ ++n_best_support; } - } + } SMARTLIST_FOREACH_END(node); log_info(LD_CIRC, "Found %d servers that might support %d/%d pending connections.", n_best_support, best_support >= 0 ? best_support : 0, @@ -2784,13 +3342,14 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, /* If any routers definitely support any pending connections, choose one * at random. */ if (best_support > 0) { - smartlist_t *supporting = smartlist_create(); + smartlist_t *supporting = smartlist_new(); - for (i = 0; i < smartlist_len(dir->routers); i++) - if (n_supported[i] == best_support) - smartlist_add(supporting, smartlist_get(dir->routers, i)); + SMARTLIST_FOREACH(the_nodes, const node_t *, node, { + if (n_supported[node_sl_idx] == best_support) + smartlist_add(supporting, (void*)node); + }); - router = routerlist_sl_choose_by_bandwidth(supporting, WEIGHT_FOR_EXIT); + node = node_sl_choose_by_bandwidth(supporting, WEIGHT_FOR_EXIT); smartlist_free(supporting); } else { /* Either there are no pending connections, or no routers even seem to @@ -2808,31 +3367,34 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, need_capacity?", fast":"", need_uptime?", stable":""); tor_free(n_supported); - return choose_good_exit_server_general(dir, 0, 0); + return choose_good_exit_server_general(0, 0); } log_notice(LD_CIRC, "All routers are down or won't exit%s -- " "choosing a doomed exit at random.", options->_ExcludeExitNodesUnion ? " or are Excluded" : ""); } - supporting = smartlist_create(); + supporting = smartlist_new(); needed_ports = circuit_get_unhandled_ports(time(NULL)); for (attempt = 0; attempt < 2; attempt++) { /* try once to pick only from routers that satisfy a needed port, * then if there are none, pick from any that support exiting. */ - for (i = 0; i < smartlist_len(dir->routers); i++) { - router = smartlist_get(dir->routers, i); - if (n_supported[i] != -1 && - (attempt || router_handles_some_port(router, needed_ports))) { + SMARTLIST_FOREACH_BEGIN(the_nodes, const node_t *, node) { + if (n_supported[node_sl_idx] != -1 && + (attempt || node_handles_some_port(node, needed_ports))) { // log_fn(LOG_DEBUG,"Try %d: '%s' is a possibility.", // try, router->nickname); - smartlist_add(supporting, router); + smartlist_add(supporting, (void*)node); } - } + } SMARTLIST_FOREACH_END(node); - router = routerlist_sl_choose_by_bandwidth(supporting, WEIGHT_FOR_EXIT); - if (router) + node = node_sl_choose_by_bandwidth(supporting, WEIGHT_FOR_EXIT); + if (node) break; smartlist_clear(supporting); + /* If we reach this point, we can't actually support any unhandled + * predicted ports, so clear all the remaining ones. */ + if (smartlist_len(needed_ports)) + rep_hist_remove_predicted_ports(needed_ports); } SMARTLIST_FOREACH(needed_ports, uint16_t *, cp, tor_free(cp)); smartlist_free(needed_ports); @@ -2840,9 +3402,9 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, } tor_free(n_supported); - if (router) { - log_info(LD_CIRC, "Chose exit server '%s'", router_describe(router)); - return router; + if (node) { + log_info(LD_CIRC, "Chose exit server '%s'", node_describe(node)); + return node; } if (options->ExitNodes) { log_warn(LD_CIRC, @@ -2863,12 +3425,12 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, * For client-side rendezvous circuits, choose a random node, weighted * toward the preferences in 'options'. */ -static routerinfo_t * -choose_good_exit_server(uint8_t purpose, routerlist_t *dir, +static const node_t * +choose_good_exit_server(uint8_t purpose, int need_uptime, int need_capacity, int is_internal) { - or_options_t *options = get_options(); - router_crn_flags_t flags = 0; + const or_options_t *options = get_options(); + router_crn_flags_t flags = CRN_NEED_DESC; if (need_uptime) flags |= CRN_NEED_UPTIME; if (need_capacity) @@ -2881,7 +3443,7 @@ choose_good_exit_server(uint8_t purpose, routerlist_t *dir, if (is_internal) /* pick it like a middle hop */ return router_choose_random_node(NULL, options->ExcludeNodes, flags); else - return choose_good_exit_server_general(dir,need_uptime,need_capacity); + return choose_good_exit_server_general(need_uptime,need_capacity); case CIRCUIT_PURPOSE_C_ESTABLISH_REND: if (options->_AllowInvalid & ALLOW_INVALID_RENDEZVOUS) flags |= CRN_ALLOW_INVALID; @@ -2897,7 +3459,7 @@ choose_good_exit_server(uint8_t purpose, routerlist_t *dir, static void warn_if_last_router_excluded(origin_circuit_t *circ, const extend_info_t *exit) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); routerset_t *rs = options->ExcludeNodes; const char *description; uint8_t purpose = circ->_base.purpose; @@ -2974,13 +3536,12 @@ static int onion_pick_cpath_exit(origin_circuit_t *circ, extend_info_t *exit) { cpath_build_state_t *state = circ->build_state; - routerlist_t *rl = router_get_routerlist(); if (state->onehop_tunnel) { log_debug(LD_CIRC, "Launching a one-hop circuit for dir tunnel."); state->desired_path_len = 1; } else { - int r = new_route_len(circ->_base.purpose, exit, rl->routers); + int r = new_route_len(circ->_base.purpose, exit, nodelist_get_list()); if (r < 1) /* must be at least 1 */ return -1; state->desired_path_len = r; @@ -2992,14 +3553,15 @@ onion_pick_cpath_exit(origin_circuit_t *circ, extend_info_t *exit) extend_info_describe(exit)); exit = extend_info_dup(exit); } else { /* we have to decide one */ - routerinfo_t *router = - choose_good_exit_server(circ->_base.purpose, rl, state->need_uptime, + const node_t *node = + choose_good_exit_server(circ->_base.purpose, state->need_uptime, state->need_capacity, state->is_internal); - if (!router) { + if (!node) { log_warn(LD_CIRC,"failed to choose an exit server"); return -1; } - exit = extend_info_from_router(router); + exit = extend_info_from_node(node, 0); + tor_assert(exit); } state->chosen_exit = exit; return 0; @@ -3050,35 +3612,30 @@ circuit_extend_to_new_exit(origin_circuit_t *circ, extend_info_t *exit) * and available for building circuits through. */ static int -count_acceptable_routers(smartlist_t *routers) +count_acceptable_nodes(smartlist_t *nodes) { - int i, n; int num=0; - routerinfo_t *r; - n = smartlist_len(routers); - for (i=0;i<n;i++) { - r = smartlist_get(routers, i); -// log_debug(LD_CIRC, + SMARTLIST_FOREACH_BEGIN(nodes, const node_t *, node) { + // log_debug(LD_CIRC, // "Contemplating whether router %d (%s) is a new option.", // i, r->nickname); - if (r->is_running == 0) { + if (! node->is_running) // log_debug(LD_CIRC,"Nope, the directory says %d is not running.",i); - goto next_i_loop; - } - if (r->is_valid == 0) { + continue; + if (! node->is_valid) // log_debug(LD_CIRC,"Nope, the directory says %d is not valid.",i); - goto next_i_loop; + continue; + if (! node_has_descriptor(node)) + continue; /* XXX This clause makes us count incorrectly: if AllowInvalidRouters * allows this node in some places, then we're getting an inaccurate * count. For now, be conservative and don't count it. But later we * should try to be smarter. */ - } - num++; + ++num; + } SMARTLIST_FOREACH_END(node); + // log_debug(LD_CIRC,"I like %d. num_acceptable_routers now %d.",i, num); - next_i_loop: - ; /* C requires an explicit statement after the label */ - } return num; } @@ -3106,31 +3663,29 @@ onion_append_to_cpath(crypt_path_t **head_ptr, crypt_path_t *new_hop) * circuit. In particular, make sure we don't pick the exit node or its * family, and make sure we don't duplicate any previous nodes or their * families. */ -static routerinfo_t * +static const node_t * choose_good_middle_server(uint8_t purpose, cpath_build_state_t *state, crypt_path_t *head, int cur_len) { int i; - routerinfo_t *r, *choice; + const node_t *r, *choice; crypt_path_t *cpath; smartlist_t *excluded; - or_options_t *options = get_options(); - router_crn_flags_t flags = 0; + const or_options_t *options = get_options(); + router_crn_flags_t flags = CRN_NEED_DESC; tor_assert(_CIRCUIT_PURPOSE_MIN <= purpose && purpose <= _CIRCUIT_PURPOSE_MAX); log_debug(LD_CIRC, "Contemplating intermediate hop: random choice."); - excluded = smartlist_create(); - if ((r = build_state_get_exit_router(state))) { - smartlist_add(excluded, r); - routerlist_add_family(excluded, r); + excluded = smartlist_new(); + if ((r = build_state_get_exit_node(state))) { + nodelist_add_node_and_family(excluded, r); } for (i = 0, cpath = head; i < cur_len; ++i, cpath=cpath->next) { - if ((r = router_get_by_digest(cpath->extend_info->identity_digest))) { - smartlist_add(excluded, r); - routerlist_add_family(excluded, r); + if ((r = node_get_by_id(cpath->extend_info->identity_digest))) { + nodelist_add_node_and_family(excluded, r); } } @@ -3153,44 +3708,43 @@ choose_good_middle_server(uint8_t purpose, * If <b>state</b> is NULL, we're choosing a router to serve as an entry * guard, not for any particular circuit. */ -static routerinfo_t * +static const node_t * choose_good_entry_server(uint8_t purpose, cpath_build_state_t *state) { - routerinfo_t *r, *choice; + const node_t *choice; smartlist_t *excluded; - or_options_t *options = get_options(); - router_crn_flags_t flags = CRN_NEED_GUARD; + const or_options_t *options = get_options(); + router_crn_flags_t flags = CRN_NEED_GUARD|CRN_NEED_DESC; + const node_t *node; if (state && options->UseEntryGuards && (purpose != CIRCUIT_PURPOSE_TESTING || options->BridgeRelay)) { + /* This request is for an entry server to use for a regular circuit, + * and we use entry guard nodes. Just return one of the guard nodes. */ return choose_random_entry(state); } - excluded = smartlist_create(); + excluded = smartlist_new(); - if (state && (r = build_state_get_exit_router(state))) { - smartlist_add(excluded, r); - routerlist_add_family(excluded, r); + if (state && (node = build_state_get_exit_node(state))) { + /* Exclude the exit node from the state, if we have one. Also exclude its + * family. */ + nodelist_add_node_and_family(excluded, node); } if (firewall_is_fascist_or()) { - /*XXXX This could slow things down a lot; use a smarter implementation */ - /* exclude all ORs that listen on the wrong port, if anybody notices. */ - routerlist_t *rl = router_get_routerlist(); - int i; - - for (i=0; i < smartlist_len(rl->routers); i++) { - r = smartlist_get(rl->routers, i); - if (!fascist_firewall_allows_or(r)) - smartlist_add(excluded, r); - } + /* Exclude all ORs that we can't reach through our firewall */ + smartlist_t *nodes = nodelist_get_list(); + SMARTLIST_FOREACH(nodes, const node_t *, node, { + if (!fascist_firewall_allows_node(node)) + smartlist_add(excluded, (void*)node); + }); } - /* and exclude current entry guards, if applicable */ + /* and exclude current entry guards and their families, if applicable */ if (options->UseEntryGuards && entry_guards) { SMARTLIST_FOREACH(entry_guards, entry_guard_t *, entry, { - if ((r = router_get_by_digest(entry->identity))) { - smartlist_add(excluded, r); - routerlist_add_family(excluded, r); + if ((node = node_get_by_id(entry->identity))) { + nodelist_add_node_and_family(excluded, node); } }); } @@ -3246,14 +3800,23 @@ onion_extend_cpath(origin_circuit_t *circ) if (cur_len == state->desired_path_len - 1) { /* Picking last node */ info = extend_info_dup(state->chosen_exit); } else if (cur_len == 0) { /* picking first node */ - routerinfo_t *r = choose_good_entry_server(purpose, state); - if (r) - info = extend_info_from_router(r); + const node_t *r = choose_good_entry_server(purpose, state); + if (r) { + /* If we're extending to a bridge, use the preferred address + rather than the primary, for potentially extending to an IPv6 + bridge. */ + int use_pref_addr = (r->ri != NULL && + r->ri->purpose == ROUTER_PURPOSE_BRIDGE); + info = extend_info_from_node(r, use_pref_addr); + tor_assert(info); + } } else { - routerinfo_t *r = + const node_t *r = choose_good_middle_server(purpose, state, circ->cpath, cur_len); - if (r) - info = extend_info_from_router(r); + if (r) { + info = extend_info_from_node(r, 0); + tor_assert(info); + } } if (!info) { @@ -3296,7 +3859,7 @@ onion_append_hop(crypt_path_t **head_ptr, extend_info_t *choice) /** Allocate a new extend_info object based on the various arguments. */ extend_info_t * extend_info_alloc(const char *nickname, const char *digest, - crypto_pk_env_t *onion_key, + crypto_pk_t *onion_key, const tor_addr_t *addr, uint16_t port) { extend_info_t *info = tor_malloc_zero(sizeof(extend_info_t)); @@ -3310,16 +3873,47 @@ extend_info_alloc(const char *nickname, const char *digest, return info; } -/** Allocate and return a new extend_info_t that can be used to build a - * circuit to or through the router <b>r</b>. */ +/** Allocate and return a new extend_info_t that can be used to build + * a circuit to or through the router <b>r</b>. Use the primary + * address of the router unless <b>for_direct_connect</b> is true, in + * which case the preferred address is used instead. */ extend_info_t * -extend_info_from_router(routerinfo_t *r) +extend_info_from_router(const routerinfo_t *r, int for_direct_connect) { - tor_addr_t addr; + tor_addr_port_t ap; tor_assert(r); - tor_addr_from_ipv4h(&addr, r->addr); + + if (for_direct_connect) + router_get_pref_orport(r, &ap); + else + router_get_prim_orport(r, &ap); return extend_info_alloc(r->nickname, r->cache_info.identity_digest, - r->onion_pkey, &addr, r->or_port); + r->onion_pkey, &ap.addr, ap.port); +} + +/** Allocate and return a new extend_info that can be used to build a + * circuit to or through the node <b>node</b>. Use the primary address + * of the node unless <b>for_direct_connect</b> is true, in which case + * the preferred address is used instead. May return NULL if there is + * not enough info about <b>node</b> to extend to it--for example, if + * there is no routerinfo_t or microdesc_t. + **/ +extend_info_t * +extend_info_from_node(const node_t *node, int for_direct_connect) +{ + if (node->ri) { + return extend_info_from_router(node->ri, for_direct_connect); + } else if (node->rs && node->md) { + tor_addr_t addr; + tor_addr_from_ipv4h(&addr, node->rs->addr); + return extend_info_alloc(node->rs->nickname, + node->identity, + node->md->onion_pkey, + &addr, + node->rs->or_port); + } else { + return NULL; + } } /** Release storage held by an extend_info_t struct. */ @@ -3328,7 +3922,7 @@ extend_info_free(extend_info_t *info) { if (!info) return; - crypto_free_pk_env(info->onion_key); + crypto_pk_free(info->onion_key); tor_free(info); } @@ -3352,12 +3946,12 @@ extend_info_dup(extend_info_t *info) * If there is no chosen exit, or if we don't know the routerinfo_t for * the chosen exit, return NULL. */ -routerinfo_t * -build_state_get_exit_router(cpath_build_state_t *state) +const node_t * +build_state_get_exit_node(cpath_build_state_t *state) { if (!state || !state->chosen_exit) return NULL; - return router_get_by_digest(state->chosen_exit->identity_digest); + return node_get_by_id(state->chosen_exit->identity_digest); } /** Return the nickname for the chosen exit router in <b>state</b>. If @@ -3379,10 +3973,10 @@ build_state_get_exit_nickname(cpath_build_state_t *state) * * If it's not usable, set *<b>reason</b> to a static string explaining why. */ -/*XXXX take a routerstatus, not a routerinfo. */ static int -entry_guard_set_status(entry_guard_t *e, routerinfo_t *ri, - time_t now, or_options_t *options, const char **reason) +entry_guard_set_status(entry_guard_t *e, const node_t *node, + time_t now, const or_options_t *options, + const char **reason) { char buf[HEX_DIGEST_LEN+1]; int changed = 0; @@ -3390,19 +3984,22 @@ entry_guard_set_status(entry_guard_t *e, routerinfo_t *ri, *reason = NULL; /* Do we want to mark this guard as bad? */ - if (!ri) + if (!node) *reason = "unlisted"; - else if (!ri->is_running) + else if (!node->is_running) *reason = "down"; - else if (options->UseBridges && ri->purpose != ROUTER_PURPOSE_BRIDGE) + else if (options->UseBridges && (!node->ri || + node->ri->purpose != ROUTER_PURPOSE_BRIDGE)) *reason = "not a bridge"; - else if (options->UseBridges && !routerinfo_is_a_configured_bridge(ri)) + else if (options->UseBridges && !node_is_a_configured_bridge(node)) *reason = "not a configured bridge"; - else if (!options->UseBridges && !ri->is_possible_guard && - !routerset_contains_router(options->EntryNodes,ri)) + else if (!options->UseBridges && !node->is_possible_guard && + !routerset_contains_node(options->EntryNodes,node)) *reason = "not recommended as a guard"; - else if (routerset_contains_router(options->ExcludeNodes, ri)) + else if (routerset_contains_node(options->ExcludeNodes, node)) *reason = "excluded"; + else if (e->path_bias_disabled) + *reason = "path-biased"; if (*reason && ! e->bad_since) { /* Router is newly bad. */ @@ -3445,7 +4042,7 @@ entry_is_time_to_retry(entry_guard_t *e, time_t now) return now > (e->last_attempted + 36*60*60); } -/** Return the router corresponding to <b>e</b>, if <b>e</b> is +/** Return the node corresponding to <b>e</b>, if <b>e</b> is * working well enough that we are willing to use it as an entry * right now. (Else return NULL.) In particular, it must be * - Listed as either up or never yet contacted; @@ -3459,14 +4056,18 @@ entry_is_time_to_retry(entry_guard_t *e, time_t now) * * If the answer is no, set *<b>msg</b> to an explanation of why. */ -static INLINE routerinfo_t * +static INLINE const node_t * entry_is_live(entry_guard_t *e, int need_uptime, int need_capacity, int assume_reachable, const char **msg) { - routerinfo_t *r; - or_options_t *options = get_options(); + const node_t *node; + const or_options_t *options = get_options(); tor_assert(msg); + if (e->path_bias_disabled) { + *msg = "path-biased"; + return NULL; + } if (e->bad_since) { *msg = "bad"; return NULL; @@ -3477,38 +4078,39 @@ entry_is_live(entry_guard_t *e, int need_uptime, int need_capacity, *msg = "unreachable"; return NULL; } - r = router_get_by_digest(e->identity); - if (!r) { + node = node_get_by_id(e->identity); + if (!node || !node_has_descriptor(node)) { *msg = "no descriptor"; return NULL; } - if (options->UseBridges) { - if (r->purpose != ROUTER_PURPOSE_BRIDGE) { + if (get_options()->UseBridges) { + if (node_get_purpose(node) != ROUTER_PURPOSE_BRIDGE) { *msg = "not a bridge"; return NULL; } - if (!routerinfo_is_a_configured_bridge(r)) { + if (!node_is_a_configured_bridge(node)) { *msg = "not a configured bridge"; return NULL; } - } else if (r->purpose != ROUTER_PURPOSE_GENERAL) { - *msg = "not general-purpose"; - return NULL; + } else { /* !get_options()->UseBridges */ + if (node_get_purpose(node) != ROUTER_PURPOSE_GENERAL) { + *msg = "not general-purpose"; + return NULL; + } } - if (options->EntryNodes && - routerset_contains_router(options->EntryNodes, r)) { + if (routerset_contains_node(options->EntryNodes, node)) { /* they asked for it, they get it */ need_uptime = need_capacity = 0; } - if (router_is_unreliable(r, need_uptime, need_capacity, 0)) { + if (node_is_unreliable(node, need_uptime, need_capacity, 0)) { *msg = "not fast/stable"; return NULL; } - if (!fascist_firewall_allows_or(r)) { + if (!fascist_firewall_allows_node(node)) { *msg = "unreachable by config"; return NULL; } - return r; + return node; } /** Return the number of entry guards that we think are usable. */ @@ -3529,8 +4131,8 @@ num_live_entry_guards(void) /** If <b>digest</b> matches the identity of any node in the * entry_guards list, return that node. Else return NULL. */ -static INLINE entry_guard_t * -is_an_entry_guard(const char *digest) +static entry_guard_t * +entry_guard_get_by_id_digest(const char *digest) { SMARTLIST_FOREACH(entry_guards, entry_guard_t *, entry, if (tor_memeq(digest, entry->identity, DIGEST_LEN)) @@ -3544,25 +4146,23 @@ is_an_entry_guard(const char *digest) static void log_entry_guards(int severity) { - smartlist_t *elements = smartlist_create(); + smartlist_t *elements = smartlist_new(); char *s; SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, e) { const char *msg = NULL; - char *cp; if (entry_is_live(e, 0, 1, 0, &msg)) - tor_asprintf(&cp, "%s [%s] (up %s)", + smartlist_add_asprintf(elements, "%s [%s] (up %s)", e->nickname, hex_str(e->identity, DIGEST_LEN), e->made_contact ? "made-contact" : "never-contacted"); else - tor_asprintf(&cp, "%s [%s] (%s, %s)", + smartlist_add_asprintf(elements, "%s [%s] (%s, %s)", e->nickname, hex_str(e->identity, DIGEST_LEN), msg, e->made_contact ? "made-contact" : "never-contacted"); - smartlist_add(elements, cp); } SMARTLIST_FOREACH_END(e); @@ -3588,7 +4188,7 @@ control_event_guard_deferred(void) #if 0 int n = 0; const char *msg; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (!entry_guards) return; SMARTLIST_FOREACH(entry_guards, entry_guard_t *, entry, @@ -3610,15 +4210,15 @@ control_event_guard_deferred(void) * If <b>chosen</b> is defined, use that one, and if it's not * already in our entry_guards list, put it at the *beginning*. * Else, put the one we pick at the end of the list. */ -static routerinfo_t * -add_an_entry_guard(routerinfo_t *chosen, int reset_status) +static const node_t * +add_an_entry_guard(const node_t *chosen, int reset_status, int prepend) { - routerinfo_t *router; + const node_t *node; entry_guard_t *entry; if (chosen) { - router = chosen; - entry = is_an_entry_guard(router->cache_info.identity_digest); + node = chosen; + entry = entry_guard_get_by_id_digest(node->identity); if (entry) { if (reset_status) { entry->bad_since = 0; @@ -3627,15 +4227,15 @@ add_an_entry_guard(routerinfo_t *chosen, int reset_status) return NULL; } } else { - router = choose_good_entry_server(CIRCUIT_PURPOSE_C_GENERAL, NULL); - if (!router) + node = choose_good_entry_server(CIRCUIT_PURPOSE_C_GENERAL, NULL); + if (!node) return NULL; } entry = tor_malloc_zero(sizeof(entry_guard_t)); - log_info(LD_CIRC, "Chose '%s' as new entry guard.", - router_describe(router)); - strlcpy(entry->nickname, router->nickname, sizeof(entry->nickname)); - memcpy(entry->identity, router->cache_info.identity_digest, DIGEST_LEN); + log_info(LD_CIRC, "Chose %s as new entry guard.", + node_describe(node)); + strlcpy(entry->nickname, node_get_nickname(node), sizeof(entry->nickname)); + memcpy(entry->identity, node->identity, DIGEST_LEN); /* Choose expiry time smudged over the past month. The goal here * is to a) spread out when Tor clients rotate their guards, so they * don't all select them on the same day, and b) avoid leaving a @@ -3643,27 +4243,27 @@ add_an_entry_guard(routerinfo_t *chosen, int reset_status) * this guard. For details, see the Jan 2010 or-dev thread. */ entry->chosen_on_date = time(NULL) - crypto_rand_int(3600*24*30); entry->chosen_by_version = tor_strdup(VERSION); - if (chosen) /* prepend */ + if (prepend) smartlist_insert(entry_guards, 0, entry); - else /* append */ + else smartlist_add(entry_guards, entry); control_event_guard(entry->nickname, entry->identity, "NEW"); control_event_guard_deferred(); log_entry_guards(LOG_INFO); - return router; + return node; } /** If the use of entry guards is configured, choose more entry guards * until we have enough in the list. */ static void -pick_entry_guards(or_options_t *options) +pick_entry_guards(const or_options_t *options) { int changed = 0; tor_assert(entry_guards); while (num_live_entry_guards() < options->NumEntryGuards) { - if (!add_an_entry_guard(NULL, 0)) + if (!add_an_entry_guard(NULL, 0, 0)) break; changed = 1; } @@ -3687,7 +4287,9 @@ entry_guard_free(entry_guard_t *e) /** Remove any entry guard which was selected by an unknown version of Tor, * or which was selected by a version of Tor that's known to select - * entry guards badly. */ + * entry guards badly, or which was selected more 2 months ago. */ +/* XXXX The "obsolete guards" and "chosen long ago guards" things should + * probably be different functions. */ static int remove_obsolete_entry_guards(time_t now) { @@ -3706,9 +4308,8 @@ remove_obsolete_entry_guards(time_t now) msg = "does not seem to be from any recognized version of Tor"; version_is_bad = 1; } else { - size_t len = strlen(ver)+5; - char *tor_ver = tor_malloc(len); - tor_snprintf(tor_ver, len, "Tor %s", ver); + char *tor_ver = NULL; + tor_asprintf(&tor_ver, "Tor %s", ver); if ((tor_version_as_new_as(tor_ver, "0.1.0.10-alpha") && !tor_version_as_new_as(tor_ver, "0.1.2.16-dev")) || (tor_version_as_new_as(tor_ver, "0.2.0.0-alpha") && @@ -3761,6 +4362,7 @@ remove_dead_entry_guards(time_t now) for (i = 0; i < smartlist_len(entry_guards); ) { entry_guard_t *entry = smartlist_get(entry_guards, i); if (entry->bad_since && + ! entry->path_bias_disabled && entry->bad_since + ENTRY_GUARD_REMOVE_AFTER < now) { base16_encode(dbuf, sizeof(dbuf), entry->identity, DIGEST_LEN); @@ -3789,7 +4391,7 @@ remove_dead_entry_guards(time_t now) * think that things are unlisted. */ void -entry_guards_compute_status(or_options_t *options, time_t now) +entry_guards_compute_status(const or_options_t *options, time_t now) { int changed = 0; digestmap_t *reasons; @@ -3803,7 +4405,7 @@ entry_guards_compute_status(or_options_t *options, time_t now) reasons = digestmap_new(); SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, entry) { - routerinfo_t *r = router_get_by_digest(entry->identity); + const node_t *r = node_get_by_id(entry->identity); const char *reason = NULL; if (entry_guard_set_status(entry, r, now, options, &reason)) changed = 1; @@ -3817,12 +4419,14 @@ entry_guards_compute_status(or_options_t *options, time_t now) if (remove_dead_entry_guards(now)) changed = 1; + if (remove_obsolete_entry_guards(now)) + changed = 1; if (changed) { SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, entry) { const char *reason = digestmap_get(reasons, entry->identity); const char *live_msg = ""; - routerinfo_t *r = entry_is_live(entry, 0, 1, 0, &live_msg); + const node_t *r = entry_is_live(entry, 0, 1, 0, &live_msg); log_info(LD_CIRC, "Summary: Entry %s [%s] is %s, %s%s%s, and %s%s.", entry->nickname, hex_str(entry->identity, DIGEST_LEN), @@ -3850,7 +4454,7 @@ entry_guards_compute_status(or_options_t *options, time_t now) * If <b>mark_relay_status</b>, also call router_set_status() on this * relay. * - * XXX023 change succeeded and mark_relay_status into 'int flags'. + * XXX024 change succeeded and mark_relay_status into 'int flags'. */ int entry_guard_register_connect_status(const char *digest, int succeeded, @@ -3866,14 +4470,14 @@ entry_guard_register_connect_status(const char *digest, int succeeded, if (! entry_guards) return 0; - SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, - { - if (tor_memeq(e->identity, digest, DIGEST_LEN)) { - entry = e; - idx = e_sl_idx; - break; - } - }); + SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, e) { + tor_assert(e); + if (tor_memeq(e->identity, digest, DIGEST_LEN)) { + entry = e; + idx = e_sl_idx; + break; + } + } SMARTLIST_FOREACH_END(e); if (!entry) return 0; @@ -3935,18 +4539,18 @@ entry_guard_register_connect_status(const char *digest, int succeeded, * came back? We should give our earlier entries another try too, * and close this connection so we don't use it before we've given * the others a shot. */ - SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, { + SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, e) { if (e == entry) break; if (e->made_contact) { const char *msg; - routerinfo_t *r = entry_is_live(e, 0, 1, 1, &msg); + const node_t *r = entry_is_live(e, 0, 1, 1, &msg); if (r && e->unreachable_since) { refuse_conn = 1; e->can_retry = 1; } } - }); + } SMARTLIST_FOREACH_END(e); if (refuse_conn) { log_info(LD_CIRC, "Connected to new entry guard '%s' (%s). Marking earlier " @@ -3976,12 +4580,12 @@ entry_nodes_should_be_added(void) should_add_entry_nodes = 1; } -/** Add all nodes in EntryNodes that aren't currently guard nodes to the list - * of guard nodes, at the front. */ +/** Adjust the entry guards list so that it only contains entries from + * EntryNodes, adding new entries from EntryNodes to the list as needed. */ static void -entry_guards_prepend_from_config(or_options_t *options) +entry_guards_set_from_config(const or_options_t *options) { - smartlist_t *entry_routers, *entry_fps; + smartlist_t *entry_nodes, *worse_entry_nodes, *entry_fps; smartlist_t *old_entry_guards_on_list, *old_entry_guards_not_on_list; tor_assert(entry_guards); @@ -4001,23 +4605,19 @@ entry_guards_prepend_from_config(or_options_t *options) tor_free(string); } - entry_routers = smartlist_create(); - entry_fps = smartlist_create(); - old_entry_guards_on_list = smartlist_create(); - old_entry_guards_not_on_list = smartlist_create(); + entry_nodes = smartlist_new(); + worse_entry_nodes = smartlist_new(); + entry_fps = smartlist_new(); + old_entry_guards_on_list = smartlist_new(); + old_entry_guards_not_on_list = smartlist_new(); /* Split entry guards into those on the list and those not. */ - /* XXXX023 Now that we allow countries and IP ranges in EntryNodes, this is - * potentially an enormous list. For now, we disable such values for - * EntryNodes in options_validate(); really, this wants a better solution. - * Perhaps we should do this calculation once whenever the list of routers - * changes or the entrynodes setting changes. - */ - routerset_get_all_routers(entry_routers, options->EntryNodes, - options->ExcludeNodes, 0); - SMARTLIST_FOREACH(entry_routers, routerinfo_t *, ri, - smartlist_add(entry_fps,ri->cache_info.identity_digest)); + routerset_get_all_nodes(entry_nodes, options->EntryNodes, + options->ExcludeNodes, 0); + SMARTLIST_FOREACH(entry_nodes, const node_t *,node, + smartlist_add(entry_fps, (void*)node->identity)); + SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, { if (smartlist_digest_isin(entry_fps, e->identity)) smartlist_add(old_entry_guards_on_list, e); @@ -4025,27 +4625,47 @@ entry_guards_prepend_from_config(or_options_t *options) smartlist_add(old_entry_guards_not_on_list, e); }); - /* Remove all currently configured entry guards from entry_routers. */ - SMARTLIST_FOREACH(entry_routers, routerinfo_t *, ri, { - if (is_an_entry_guard(ri->cache_info.identity_digest)) { - SMARTLIST_DEL_CURRENT(entry_routers, ri); + /* Remove all currently configured guard nodes, excluded nodes, unreachable + * nodes, or non-Guard nodes from entry_nodes. */ + SMARTLIST_FOREACH_BEGIN(entry_nodes, const node_t *, node) { + if (entry_guard_get_by_id_digest(node->identity)) { + SMARTLIST_DEL_CURRENT(entry_nodes, node); + continue; + } else if (routerset_contains_node(options->ExcludeNodes, node)) { + SMARTLIST_DEL_CURRENT(entry_nodes, node); + continue; + } else if (!fascist_firewall_allows_node(node)) { + SMARTLIST_DEL_CURRENT(entry_nodes, node); + continue; + } else if (! node->is_possible_guard) { + smartlist_add(worse_entry_nodes, (node_t*)node); + SMARTLIST_DEL_CURRENT(entry_nodes, node); } - }); + } SMARTLIST_FOREACH_END(node); /* Now build the new entry_guards list. */ smartlist_clear(entry_guards); /* First, the previously configured guards that are in EntryNodes. */ smartlist_add_all(entry_guards, old_entry_guards_on_list); + /* Next, scramble the rest of EntryNodes, putting the guards first. */ + smartlist_shuffle(entry_nodes); + smartlist_shuffle(worse_entry_nodes); + smartlist_add_all(entry_nodes, worse_entry_nodes); + /* Next, the rest of EntryNodes */ - SMARTLIST_FOREACH(entry_routers, routerinfo_t *, ri, { - add_an_entry_guard(ri, 0); - }); + SMARTLIST_FOREACH_BEGIN(entry_nodes, const node_t *, node) { + add_an_entry_guard(node, 0, 0); + if (smartlist_len(entry_guards) > options->NumEntryGuards * 10) + break; + } SMARTLIST_FOREACH_END(node); + log_notice(LD_GENERAL, "%d entries in guards", smartlist_len(entry_guards)); /* Finally, free the remaining previously configured guards that are not in * EntryNodes. */ SMARTLIST_FOREACH(old_entry_guards_not_on_list, entry_guard_t *, e, entry_guard_free(e)); - smartlist_free(entry_routers); + smartlist_free(entry_nodes); + smartlist_free(worse_entry_nodes); smartlist_free(entry_fps); smartlist_free(old_entry_guards_on_list); smartlist_free(old_entry_guards_not_on_list); @@ -4057,7 +4677,7 @@ entry_guards_prepend_from_config(or_options_t *options) * list already and we must stick to it. */ int -entry_list_is_constrained(or_options_t *options) +entry_list_is_constrained(const or_options_t *options) { if (options->EntryNodes) return 1; @@ -4071,28 +4691,29 @@ entry_list_is_constrained(or_options_t *options) * make sure not to pick this circuit's exit or any node in the * exit's family. If <b>state</b> is NULL, we're looking for a random * guard (likely a bridge). */ -routerinfo_t * +const node_t * choose_random_entry(cpath_build_state_t *state) { - or_options_t *options = get_options(); - smartlist_t *live_entry_guards = smartlist_create(); - smartlist_t *exit_family = smartlist_create(); - routerinfo_t *chosen_exit = state?build_state_get_exit_router(state) : NULL; - routerinfo_t *r = NULL; + const or_options_t *options = get_options(); + smartlist_t *live_entry_guards = smartlist_new(); + smartlist_t *exit_family = smartlist_new(); + const node_t *chosen_exit = + state?build_state_get_exit_node(state) : NULL; + const node_t *node = NULL; int need_uptime = state ? state->need_uptime : 0; int need_capacity = state ? state->need_capacity : 0; int preferred_min, consider_exit_family = 0; if (chosen_exit) { - routerlist_add_family(exit_family, chosen_exit); + nodelist_add_node_and_family(exit_family, chosen_exit); consider_exit_family = 1; } if (!entry_guards) - entry_guards = smartlist_create(); + entry_guards = smartlist_new(); if (should_add_entry_nodes) - entry_guards_prepend_from_config(options); + entry_guards_set_from_config(options); if (!entry_list_is_constrained(options) && smartlist_len(entry_guards) < options->NumEntryGuards) @@ -4100,25 +4721,24 @@ choose_random_entry(cpath_build_state_t *state) retry: smartlist_clear(live_entry_guards); - SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, entry) - { + SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, entry) { const char *msg; - r = entry_is_live(entry, need_uptime, need_capacity, 0, &msg); - if (!r) + node = entry_is_live(entry, need_uptime, need_capacity, 0, &msg); + if (!node) continue; /* down, no point */ - if (r == chosen_exit) + if (node == chosen_exit) continue; /* don't pick the same node for entry and exit */ - if (consider_exit_family && smartlist_isin(exit_family, r)) + if (consider_exit_family && smartlist_isin(exit_family, node)) continue; /* avoid relays that are family members of our exit */ #if 0 /* since EntryNodes is always strict now, this clause is moot */ if (options->EntryNodes && - !routerset_contains_router(options->EntryNodes, r)) { + !routerset_contains_node(options->EntryNodes, node)) { /* We've come to the end of our preferred entry nodes. */ if (smartlist_len(live_entry_guards)) goto choose_and_finish; /* only choose from the ones we like */ if (options->StrictNodes) { /* in theory this case should never happen, since - * entry_guards_prepend_from_config() drops unwanted relays */ + * entry_guards_set_from_config() drops unwanted relays */ tor_fragile_assert(); } else { log_info(LD_CIRC, @@ -4126,7 +4746,7 @@ choose_random_entry(cpath_build_state_t *state) } } #endif - smartlist_add(live_entry_guards, r); + smartlist_add(live_entry_guards, (void*)node); if (!entry->made_contact) { /* Always start with the first not-yet-contacted entry * guard. Otherwise we might add several new ones, pick @@ -4135,9 +4755,8 @@ choose_random_entry(cpath_build_state_t *state) goto choose_and_finish; } if (smartlist_len(live_entry_guards) >= options->NumEntryGuards) - break; /* we have enough */ - } - SMARTLIST_FOREACH_END(entry); + goto choose_and_finish; /* we have enough */ + } SMARTLIST_FOREACH_END(entry); if (entry_list_is_constrained(options)) { /* If we prefer the entry nodes we've got, and we have at least @@ -4157,8 +4776,8 @@ choose_random_entry(cpath_build_state_t *state) /* XXX if guard doesn't imply fast and stable, then we need * to tell add_an_entry_guard below what we want, or it might * be a long time til we get it. -RD */ - r = add_an_entry_guard(NULL, 0); - if (r) { + node = add_an_entry_guard(NULL, 0, 0); + if (node) { entry_guards_changed(); /* XXX we start over here in case the new node we added shares * a family with our exit node. There's a chance that we'll just @@ -4168,11 +4787,11 @@ choose_random_entry(cpath_build_state_t *state) goto retry; } } - if (!r && need_uptime) { + if (!node && need_uptime) { need_uptime = 0; /* try without that requirement */ goto retry; } - if (!r && need_capacity) { + if (!node && need_capacity) { /* still no? last attempt, try without requiring capacity */ need_capacity = 0; goto retry; @@ -4181,10 +4800,10 @@ choose_random_entry(cpath_build_state_t *state) /* Removing this retry logic: if we only allow one exit, and it is in the same family as all our entries, then we are just plain not going to win here. */ - if (!r && entry_list_is_constrained(options) && consider_exit_family) { - /* still no? if we're using bridges, - * and our chosen exit is in the same family as all our - * bridges, then be flexible about families. */ + if (!node && entry_list_is_constrained(options) && consider_exit_family) { + /* still no? if we're using bridges or have strictentrynodes + * set, and our chosen exit is in the same family as all our + * bridges/entry guards, then be flexible about families. */ consider_exit_family = 0; goto retry; } @@ -4196,16 +4815,16 @@ choose_random_entry(cpath_build_state_t *state) if (entry_list_is_constrained(options)) { /* We need to weight by bandwidth, because our bridges or entryguards * were not already selected proportional to their bandwidth. */ - r = routerlist_sl_choose_by_bandwidth(live_entry_guards, WEIGHT_FOR_GUARD); + node = node_sl_choose_by_bandwidth(live_entry_guards, WEIGHT_FOR_GUARD); } else { /* We choose uniformly at random here, because choose_good_entry_server() * already weights its choices by bandwidth, so we don't want to * *double*-weight our guard selection. */ - r = smartlist_choose(live_entry_guards); + node = smartlist_choose(live_entry_guards); } smartlist_free(live_entry_guards); smartlist_free(exit_family); - return r; + return node; } /** Parse <b>state</b> and learn about the entry guards it describes. @@ -4218,7 +4837,7 @@ int entry_guards_parse_state(or_state_t *state, int set, char **msg) { entry_guard_t *node = NULL; - smartlist_t *new_entry_guards = smartlist_create(); + smartlist_t *new_entry_guards = smartlist_new(); config_line_t *line; time_t now = time(NULL); const char *state_version = state->TorVersion; @@ -4227,7 +4846,7 @@ entry_guards_parse_state(or_state_t *state, int set, char **msg) *msg = NULL; for (line = state->EntryGuards; line; line = line->next) { if (!strcasecmp(line->key, "EntryGuard")) { - smartlist_t *args = smartlist_create(); + smartlist_t *args = smartlist_new(); node = tor_malloc_zero(sizeof(entry_guard_t)); /* all entry guards on disk have been contacted */ node->made_contact = 1; @@ -4295,13 +4914,43 @@ entry_guards_parse_state(or_state_t *state, int set, char **msg) continue; } digestmap_set(added_by, d, tor_strdup(line->value+HEX_DIGEST_LEN+1)); + } else if (!strcasecmp(line->key, "EntryGuardPathBias")) { + const or_options_t *options = get_options(); + unsigned hop_cnt, success_cnt; + + if (!node) { + *msg = tor_strdup("Unable to parse entry nodes: " + "EntryGuardPathBias without EntryGuard"); + break; + } + + if (tor_sscanf(line->value, "%u %u", &success_cnt, &hop_cnt) != 2) { + log_warn(LD_GENERAL, "Unable to parse guard path bias info: " + "Misformated EntryGuardPathBias %s", escaped(line->value)); + continue; + } + + node->first_hops = hop_cnt; + node->circuit_successes = success_cnt; + log_info(LD_GENERAL, "Read %u/%u path bias for node %s", + node->circuit_successes, node->first_hops, node->nickname); + /* Note: We rely on the < comparison here to allow us to set a 0 + * rate and disable the feature entirely. If refactoring, don't + * change to <= */ + if (node->circuit_successes/((double)node->first_hops) + < pathbias_get_disable_rate(options)) { + node->path_bias_disabled = 1; + log_info(LD_GENERAL, + "Path bias is too high (%u/%u); disabling node %s", + node->circuit_successes, node->first_hops, node->nickname); + } + } else { log_warn(LD_BUG, "Unexpected key %s", line->key); } } - SMARTLIST_FOREACH(new_entry_guards, entry_guard_t *, e, - { + SMARTLIST_FOREACH_BEGIN(new_entry_guards, entry_guard_t *, e) { char *sp; char *val = digestmap_get(added_by, e->identity); if (val && (sp = strchr(val, ' '))) { @@ -4319,7 +4968,10 @@ entry_guards_parse_state(or_state_t *state, int set, char **msg) e->chosen_on_date = time(NULL) - crypto_rand_int(3600*24*30); } } - }); + if (e->path_bias_disabled && !e->bad_since) + e->bad_since = time(NULL); + } + SMARTLIST_FOREACH_END(e); if (*msg || !set) { SMARTLIST_FOREACH(new_entry_guards, entry_guard_t *, e, @@ -4333,7 +4985,7 @@ entry_guards_parse_state(or_state_t *state, int set, char **msg) } entry_guards = new_entry_guards; entry_guards_dirty = 0; - /* XXX023 hand new_entry_guards to this func, and move it up a + /* XXX024 hand new_entry_guards to this func, and move it up a * few lines, so we don't have to re-dirty it */ if (remove_obsolete_entry_guards(now)) entry_guards_dirty = 1; @@ -4373,18 +5025,15 @@ entry_guards_update_state(or_state_t *state) next = &state->EntryGuards; *next = NULL; if (!entry_guards) - entry_guards = smartlist_create(); - SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, - { + entry_guards = smartlist_new(); + SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, e) { char dbuf[HEX_DIGEST_LEN+1]; if (!e->made_contact) continue; /* don't write this one to disk */ *next = line = tor_malloc_zero(sizeof(config_line_t)); line->key = tor_strdup("EntryGuard"); - line->value = tor_malloc(HEX_DIGEST_LEN+MAX_NICKNAME_LEN+2); base16_encode(dbuf, sizeof(dbuf), e->identity, DIGEST_LEN); - tor_snprintf(line->value,HEX_DIGEST_LEN+MAX_NICKNAME_LEN+2, - "%s %s", e->nickname, dbuf); + tor_asprintf(&line->value, "%s %s", e->nickname, dbuf); next = &(line->next); if (e->unreachable_since) { *next = line = tor_malloc_zero(sizeof(config_line_t)); @@ -4408,19 +5057,23 @@ entry_guards_update_state(or_state_t *state) !strchr(e->chosen_by_version, ' ')) { char d[HEX_DIGEST_LEN+1]; char t[ISO_TIME_LEN+1]; - size_t val_len; *next = line = tor_malloc_zero(sizeof(config_line_t)); line->key = tor_strdup("EntryGuardAddedBy"); - val_len = (HEX_DIGEST_LEN+1+strlen(e->chosen_by_version) - +1+ISO_TIME_LEN+1); - line->value = tor_malloc(val_len); base16_encode(d, sizeof(d), e->identity, DIGEST_LEN); format_iso_time(t, e->chosen_on_date); - tor_snprintf(line->value, val_len, "%s %s %s", + tor_asprintf(&line->value, "%s %s %s", d, e->chosen_by_version, t); next = &(line->next); } - }); + if (e->first_hops) { + *next = line = tor_malloc_zero(sizeof(config_line_t)); + line->key = tor_strdup("EntryGuardPathBias"); + tor_asprintf(&line->value, "%u %u", + e->circuit_successes, e->first_hops); + next = &(line->next); + } + + } SMARTLIST_FOREACH_END(e); if (!get_options()->AvoidDiskWrites) or_state_mark_dirty(get_or_state(), 0); entry_guards_dirty = 0; @@ -4442,17 +5095,15 @@ getinfo_helper_entry_guards(control_connection_t *conn, if (!strcmp(question,"entry-guards") || !strcmp(question,"helper-nodes")) { - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); char tbuf[ISO_TIME_LEN+1]; char nbuf[MAX_VERBOSE_NICKNAME_LEN+1]; if (!entry_guards) - entry_guards = smartlist_create(); + entry_guards = smartlist_new(); SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, e) { - size_t len = MAX_VERBOSE_NICKNAME_LEN+ISO_TIME_LEN+32; - char *c = tor_malloc(len); const char *status = NULL; time_t when = 0; - routerinfo_t *ri; + const node_t *node; if (!e->made_contact) { status = "never-connected"; @@ -4463,9 +5114,9 @@ getinfo_helper_entry_guards(control_connection_t *conn, status = "up"; } - ri = router_get_by_digest(e->identity); - if (ri) { - router_get_verbose_nickname(nbuf, ri); + node = node_get_by_id(e->identity); + if (node) { + node_get_verbose_nickname(node, nbuf); } else { nbuf[0] = '$'; base16_encode(nbuf+1, sizeof(nbuf)-1, e->identity, DIGEST_LEN); @@ -4475,11 +5126,10 @@ getinfo_helper_entry_guards(control_connection_t *conn, if (when) { format_iso_time(tbuf, when); - tor_snprintf(c, len, "%s %s %s\n", nbuf, status, tbuf); + smartlist_add_asprintf(sl, "%s %s %s\n", nbuf, status, tbuf); } else { - tor_snprintf(c, len, "%s %s\n", nbuf, status); + smartlist_add_asprintf(sl, "%s %s\n", nbuf, status); } - smartlist_add(sl, c); } SMARTLIST_FOREACH_END(e); *answer = smartlist_join_strings(sl, "", 0, NULL); SMARTLIST_FOREACH(sl, char *, c, tor_free(c)); @@ -4488,24 +5138,6 @@ getinfo_helper_entry_guards(control_connection_t *conn, return 0; } -/** Information about a configured bridge. Currently this just matches the - * ones in the torrc file, but one day we may be able to learn about new - * bridges on our own, and remember them in the state file. */ -typedef struct { - /** Address of the bridge. */ - tor_addr_t addr; - /** TLS port for the bridge. */ - uint16_t port; - /** Boolean: We are re-parsing our bridge list, and we are going to remove - * this one if we don't find it in the list of configured bridges. */ - unsigned marked_for_removal : 1; - /** Expected identity digest, or all zero bytes if we don't know what the - * digest should be. */ - char identity[DIGEST_LEN]; - /** When should we next try to fetch a descriptor for this bridge? */ - download_status_t fetch_status; -} bridge_info_t; - /** A list of configured bridges. Whenever we actually get a descriptor * for one, we add it as an entry guard. Note that the order of bridges * in this list does not necessarily correspond to the order of bridges @@ -4518,7 +5150,7 @@ void mark_bridge_list(void) { if (!bridge_list) - bridge_list = smartlist_create(); + bridge_list = smartlist_new(); SMARTLIST_FOREACH(bridge_list, bridge_info_t *, b, b->marked_for_removal = 1); } @@ -4529,11 +5161,11 @@ void sweep_bridge_list(void) { if (!bridge_list) - bridge_list = smartlist_create(); + bridge_list = smartlist_new(); SMARTLIST_FOREACH_BEGIN(bridge_list, bridge_info_t *, b) { if (b->marked_for_removal) { SMARTLIST_DEL_CURRENT(bridge_list, b); - tor_free(b); + bridge_free(b); } } SMARTLIST_FOREACH_END(b); } @@ -4543,11 +5175,219 @@ static void clear_bridge_list(void) { if (!bridge_list) - bridge_list = smartlist_create(); - SMARTLIST_FOREACH(bridge_list, bridge_info_t *, b, tor_free(b)); + bridge_list = smartlist_new(); + SMARTLIST_FOREACH(bridge_list, bridge_info_t *, b, bridge_free(b)); smartlist_clear(bridge_list); } +/** Free the bridge <b>bridge</b>. */ +static void +bridge_free(bridge_info_t *bridge) +{ + if (!bridge) + return; + + tor_free(bridge->transport_name); + tor_free(bridge); +} + +/** A list of pluggable transports found in torrc. */ +static smartlist_t *transport_list = NULL; + +/** Mark every entry of the transport list to be removed on our next call to + * sweep_transport_list unless it has first been un-marked. */ +void +mark_transport_list(void) +{ + if (!transport_list) + transport_list = smartlist_new(); + SMARTLIST_FOREACH(transport_list, transport_t *, t, + t->marked_for_removal = 1); +} + +/** Remove every entry of the transport list that was marked with + * mark_transport_list if it has not subsequently been un-marked. */ +void +sweep_transport_list(void) +{ + if (!transport_list) + transport_list = smartlist_new(); + SMARTLIST_FOREACH_BEGIN(transport_list, transport_t *, t) { + if (t->marked_for_removal) { + SMARTLIST_DEL_CURRENT(transport_list, t); + transport_free(t); + } + } SMARTLIST_FOREACH_END(t); +} + +/** Initialize the pluggable transports list to empty, creating it if + * needed. */ +void +clear_transport_list(void) +{ + if (!transport_list) + transport_list = smartlist_new(); + SMARTLIST_FOREACH(transport_list, transport_t *, t, transport_free(t)); + smartlist_clear(transport_list); +} + +/** Free the pluggable transport struct <b>transport</b>. */ +void +transport_free(transport_t *transport) +{ + if (!transport) + return; + + tor_free(transport->name); + tor_free(transport); +} + +/** Returns the transport in our transport list that has the name <b>name</b>. + * Else returns NULL. */ +transport_t * +transport_get_by_name(const char *name) +{ + tor_assert(name); + + if (!transport_list) + return NULL; + + SMARTLIST_FOREACH_BEGIN(transport_list, transport_t *, transport) { + if (!strcmp(transport->name, name)) + return transport; + } SMARTLIST_FOREACH_END(transport); + + return NULL; +} + +/** Returns a transport_t struct for a transport proxy supporting the + protocol <b>name</b> listening at <b>addr</b>:<b>port</b> using + SOCKS version <b>socks_ver</b>. */ +transport_t * +transport_new(const tor_addr_t *addr, uint16_t port, + const char *name, int socks_ver) +{ + transport_t *t = tor_malloc_zero(sizeof(transport_t)); + + tor_addr_copy(&t->addr, addr); + t->port = port; + t->name = tor_strdup(name); + t->socks_version = socks_ver; + + return t; +} + +/** Resolve any conflicts that the insertion of transport <b>t</b> + * might cause. + * Return 0 if <b>t</b> is OK and should be registered, 1 if there is + * a transport identical to <b>t</b> already registered and -1 if + * <b>t</b> cannot be added due to conflicts. */ +static int +transport_resolve_conflicts(transport_t *t) +{ + /* This is how we resolve transport conflicts: + + If there is already a transport with the same name and addrport, + we either have duplicate torrc lines OR we are here post-HUP and + this transport was here pre-HUP as well. In any case, mark the + old transport so that it doesn't get removed and ignore the new + one. Our caller has to free the new transport so we return '1' to + signify this. + + If there is already a transport with the same name but different + addrport: + * if it's marked for removal, it means that it either has a lower + priority than 't' in torrc (otherwise the mark would have been + cleared by the paragraph above), or it doesn't exist at all in + the post-HUP torrc. We destroy the old transport and register 't'. + * if it's *not* marked for removal, it means that it was newly + added in the post-HUP torrc or that it's of higher priority, in + this case we ignore 't'. */ + transport_t *t_tmp = transport_get_by_name(t->name); + if (t_tmp) { /* same name */ + if (tor_addr_eq(&t->addr, &t_tmp->addr) && (t->port == t_tmp->port)) { + /* same name *and* addrport */ + t_tmp->marked_for_removal = 0; + return 1; + } else { /* same name but different addrport */ + char *new_transport_addr = tor_strdup(fmt_addr(&t->addr)); + if (t_tmp->marked_for_removal) { /* marked for removal */ + log_notice(LD_GENERAL, "You tried to add transport '%s' at '%s:%u' " + "but there was already a transport marked for deletion at " + "'%s:%u'. We deleted the old transport and registered the " + "new one.", t->name, new_transport_addr, t->port, + fmt_addr(&t_tmp->addr), t_tmp->port); + smartlist_remove(transport_list, t_tmp); + transport_free(t_tmp); + tor_free(new_transport_addr); + } else { /* *not* marked for removal */ + log_notice(LD_GENERAL, "You tried to add transport '%s' at '%s:%u' " + "but the same transport already exists at '%s:%u'. " + "Skipping.", t->name, new_transport_addr, t->port, + fmt_addr(&t_tmp->addr), t_tmp->port); + tor_free(new_transport_addr); + return -1; + } + } + } + + return 0; +} + +/** Add transport <b>t</b> to the internal list of pluggable + * transports. + * Returns 0 if the transport was added correctly, 1 if the same + * transport was already registered (in this case the caller must + * free the transport) and -1 if there was an error. */ +int +transport_add(transport_t *t) +{ + int r; + tor_assert(t); + + r = transport_resolve_conflicts(t); + + switch (r) { + case 0: /* should register transport */ + if (!transport_list) + transport_list = smartlist_new(); + smartlist_add(transport_list, t); + return 0; + default: /* let our caller know the return code */ + return r; + } +} + +/** Remember a new pluggable transport proxy at <b>addr</b>:<b>port</b>. + * <b>name</b> is set to the name of the protocol this proxy uses. + * <b>socks_ver</b> is set to the SOCKS version of the proxy. */ +int +transport_add_from_config(const tor_addr_t *addr, uint16_t port, + const char *name, int socks_ver) +{ + transport_t *t = transport_new(addr, port, name, socks_ver); + + int r = transport_add(t); + + switch (r) { + case -1: + default: + log_notice(LD_GENERAL, "Could not add transport %s at %s:%u. Skipping.", + t->name, fmt_addr(&t->addr), t->port); + transport_free(t); + return -1; + case 1: + log_info(LD_GENERAL, "Succesfully registered transport %s at %s:%u.", + t->name, fmt_addr(&t->addr), t->port); + transport_free(t); /* falling */ + return 0; + case 0: + log_info(LD_GENERAL, "Succesfully registered transport %s at %s:%u.", + t->name, fmt_addr(&t->addr), t->port); + return 0; + } +} + /** Return a bridge pointer if <b>ri</b> is one of our known bridges * (either by comparing keys if possible, else by comparing addr/port). * Else return NULL. */ @@ -4574,21 +5414,53 @@ get_configured_bridge_by_addr_port_digest(const tor_addr_t *addr, /** Wrapper around get_configured_bridge_by_addr_port_digest() to look * it up via router descriptor <b>ri</b>. */ static bridge_info_t * -get_configured_bridge_by_routerinfo(routerinfo_t *ri) +get_configured_bridge_by_routerinfo(const routerinfo_t *ri) { - tor_addr_t addr; - tor_addr_from_ipv4h(&addr, ri->addr); - return get_configured_bridge_by_addr_port_digest(&addr, - ri->or_port, ri->cache_info.identity_digest); + tor_addr_port_t ap; + + router_get_pref_orport(ri, &ap); + return get_configured_bridge_by_addr_port_digest(&ap.addr, ap.port, + ri->cache_info.identity_digest); } /** Return 1 if <b>ri</b> is one of our known bridges, else 0. */ int -routerinfo_is_a_configured_bridge(routerinfo_t *ri) +routerinfo_is_a_configured_bridge(const routerinfo_t *ri) { return get_configured_bridge_by_routerinfo(ri) ? 1 : 0; } +/** Return 1 if <b>node</b> is one of our configured bridges, else 0. */ +int +node_is_a_configured_bridge(const node_t *node) +{ + int retval = 0; /* Negative. */ + smartlist_t *orports = NULL; + + if (!node) + goto out; + + orports = node_get_all_orports(node); + if (orports == NULL) + goto out; + + SMARTLIST_FOREACH_BEGIN(orports, tor_addr_port_t *, orport) { + if (get_configured_bridge_by_addr_port_digest(&orport->addr, orport->port, + node->identity) != NULL) { + retval = 1; + goto out; + } + } SMARTLIST_FOREACH_END(orport); + + out: + if (orports != NULL) { + SMARTLIST_FOREACH(orports, tor_addr_port_t *, p, tor_free(p)); + smartlist_free(orports); + orports = NULL; + } + return retval; +} + /** We made a connection to a router at <b>addr</b>:<b>port</b> * without knowing its digest. Its digest turned out to be <b>digest</b>. * If it was a bridge, and we still don't know its digest, record it. @@ -4606,28 +5478,94 @@ learned_router_identity(const tor_addr_t *addr, uint16_t port, } } +/** Return true if <b>bridge</b> has the same identity digest as + * <b>digest</b>. If <b>digest</b> is NULL, it matches + * bridges with unspecified identity digests. */ +static int +bridge_has_digest(const bridge_info_t *bridge, const char *digest) +{ + if (digest) + return tor_memeq(digest, bridge->identity, DIGEST_LEN); + else + return tor_digest_is_zero(bridge->identity); +} + +/** We are about to add a new bridge at <b>addr</b>:<b>port</b>, with optional + * <b>digest</b> and <b>transport_name</b>. Mark for removal any previously + * existing bridge with the same address and port, and warn the user as + * appropriate. + */ +static void +bridge_resolve_conflicts(const tor_addr_t *addr, uint16_t port, + const char *digest, const char *transport_name) +{ + /* Iterate the already-registered bridge list: + + If you find a bridge with the same adress and port, mark it for + removal. It doesn't make sense to have two active bridges with + the same IP:PORT. If the bridge in question has a different + digest or transport than <b>digest</b>/<b>transport_name</b>, + it's probably a misconfiguration and we should warn the user. + */ + SMARTLIST_FOREACH_BEGIN(bridge_list, bridge_info_t *, bridge) { + if (bridge->marked_for_removal) + continue; + + if (tor_addr_eq(&bridge->addr, addr) && (bridge->port == port)) { + + bridge->marked_for_removal = 1; + + if (!bridge_has_digest(bridge, digest) || + strcmp_opt(bridge->transport_name, transport_name)) { + /* warn the user */ + char *bridge_description_new, *bridge_description_old; + tor_asprintf(&bridge_description_new, "%s:%u:%s:%s", + fmt_addr(addr), port, + digest ? hex_str(digest, DIGEST_LEN) : "", + transport_name ? transport_name : ""); + tor_asprintf(&bridge_description_old, "%s:%u:%s:%s", + fmt_addr(&bridge->addr), bridge->port, + tor_digest_is_zero(bridge->identity) ? + "" : hex_str(bridge->identity,DIGEST_LEN), + bridge->transport_name ? bridge->transport_name : ""); + + log_warn(LD_GENERAL,"Tried to add bridge '%s', but we found a conflict" + " with the already registered bridge '%s'. We will discard" + " the old bridge and keep '%s'. If this is not what you" + " wanted, please change your configuration file accordingly.", + bridge_description_new, bridge_description_old, + bridge_description_new); + + tor_free(bridge_description_new); + tor_free(bridge_description_old); + } + } + } SMARTLIST_FOREACH_END(bridge); +} + /** Remember a new bridge at <b>addr</b>:<b>port</b>. If <b>digest</b> * is set, it tells us the identity key too. If we already had the - * bridge in our list, unmark it, and don't actually add anything new. */ + * bridge in our list, unmark it, and don't actually add anything new. + * If <b>transport_name</b> is non-NULL - the bridge is associated with a + * pluggable transport - we assign the transport to the bridge. */ void bridge_add_from_config(const tor_addr_t *addr, uint16_t port, - const char *digest) + const char *digest, const char *transport_name) { bridge_info_t *b; - if ((b = get_configured_bridge_by_addr_port_digest(addr, port, digest))) { - b->marked_for_removal = 0; - return; - } + bridge_resolve_conflicts(addr, port, digest, transport_name); b = tor_malloc_zero(sizeof(bridge_info_t)); tor_addr_copy(&b->addr, addr); b->port = port; if (digest) memcpy(b->identity, digest, DIGEST_LEN); + if (transport_name) + b->transport_name = tor_strdup(transport_name); b->fetch_status.schedule = DL_SCHED_BRIDGE; if (!bridge_list) - bridge_list = smartlist_create(); + bridge_list = smartlist_new(); smartlist_add(bridge_list, b); } @@ -4662,12 +5600,64 @@ find_bridge_by_digest(const char *digest) return NULL; } +/* DOCDOC find_transport_name_by_bridge_addrport */ +const char * +find_transport_name_by_bridge_addrport(const tor_addr_t *addr, uint16_t port) +{ + if (!bridge_list) + return NULL; + + SMARTLIST_FOREACH_BEGIN(bridge_list, const bridge_info_t *, bridge) { + if (tor_addr_eq(&bridge->addr, addr) && + (bridge->port == port)) + return bridge->transport_name; + } SMARTLIST_FOREACH_END(bridge); + + return NULL; +} + +/** If <b>addr</b> and <b>port</b> match the address and port of a + * bridge of ours that uses pluggable transports, place its transport + * in <b>transport</b>. + * + * Return 0 on success (found a transport, or found a bridge with no + * transport, or found no bridge); return -1 if we should be using a + * transport, but the transport could not be found. + */ +int +find_transport_by_bridge_addrport(const tor_addr_t *addr, uint16_t port, + const transport_t **transport) +{ + *transport = NULL; + if (!bridge_list) + return 0; + + SMARTLIST_FOREACH_BEGIN(bridge_list, const bridge_info_t *, bridge) { + if (tor_addr_eq(&bridge->addr, addr) && + (bridge->port == port)) { /* bridge matched */ + if (bridge->transport_name) { /* it also uses pluggable transports */ + *transport = transport_get_by_name(bridge->transport_name); + if (*transport == NULL) { /* it uses pluggable transports, but + the transport could not be found! */ + return -1; + } + return 0; + } else { /* bridge matched, but it doesn't use transports. */ + break; + } + } + } SMARTLIST_FOREACH_END(bridge); + + *transport = NULL; + return 0; +} + /** We need to ask <b>bridge</b> for its server descriptor. */ static void launch_direct_bridge_descriptor_fetch(bridge_info_t *bridge) { char *address; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (connection_get_by_type_addr_port_purpose( CONN_TYPE_DIR, &bridge->addr, bridge->port, @@ -4709,15 +5699,20 @@ retry_bridge_descriptor_fetch_directly(const char *digest) * descriptor, fetch a new copy of its descriptor -- either directly * from the bridge or via a bridge authority. */ void -fetch_bridge_descriptors(or_options_t *options, time_t now) +fetch_bridge_descriptors(const or_options_t *options, time_t now) { - int num_bridge_auths = get_n_authorities(BRIDGE_AUTHORITY); + int num_bridge_auths = get_n_authorities(BRIDGE_DIRINFO); int ask_bridge_directly; int can_use_bridge_authority; if (!bridge_list) return; + /* If we still have unconfigured managed proxies, don't go and + connect to a bridge. */ + if (pt_proxies_configuration_pending()) + return; + SMARTLIST_FOREACH_BEGIN(bridge_list, bridge_info_t *, bridge) { if (!download_status_is_ready(&bridge->fetch_status, now, @@ -4776,26 +5771,88 @@ fetch_bridge_descriptors(or_options_t *options, time_t now) } /** If our <b>bridge</b> is configured to be a different address than - * the bridge gives in its routerinfo <b>ri</b>, rewrite the routerinfo + * the bridge gives in <b>node</b>, rewrite the routerinfo * we received to use the address we meant to use. Now we handle * multihomed bridges better. */ static void -rewrite_routerinfo_address_for_bridge(bridge_info_t *bridge, routerinfo_t *ri) +rewrite_node_address_for_bridge(const bridge_info_t *bridge, node_t *node) { + /* XXXX move this function. */ + /* XXXX overridden addresses should really live in the node_t, so that the + * routerinfo_t and the microdesc_t can be immutable. But we can only + * do that safely if we know that no function that connects to an OR + * does so through an address from any source other than node_get_addr(). + */ tor_addr_t addr; - tor_addr_from_ipv4h(&addr, ri->addr); - if (!tor_addr_compare(&bridge->addr, &addr, CMP_EXACT) && - bridge->port == ri->or_port) - return; /* they match, so no need to do anything */ + if (node->ri) { + routerinfo_t *ri = node->ri; + tor_addr_from_ipv4h(&addr, ri->addr); - ri->addr = tor_addr_to_ipv4h(&bridge->addr); - tor_free(ri->address); - ri->address = tor_dup_ip(ri->addr); - ri->or_port = bridge->port; - log_info(LD_DIR, "Adjusted bridge '%s' to match configured address %s:%d.", - ri->nickname, ri->address, ri->or_port); + if ((!tor_addr_compare(&bridge->addr, &addr, CMP_EXACT) && + bridge->port == ri->or_port) || + (!tor_addr_compare(&bridge->addr, &ri->ipv6_addr, CMP_EXACT) && + bridge->port == ri->ipv6_orport)) { + /* they match, so no need to do anything */ + } else { + if (tor_addr_family(&bridge->addr) == AF_INET) { + ri->addr = tor_addr_to_ipv4h(&bridge->addr); + tor_free(ri->address); + ri->address = tor_dup_ip(ri->addr); + ri->or_port = bridge->port; + log_info(LD_DIR, + "Adjusted bridge routerinfo for '%s' to match configured " + "address %s:%d.", + ri->nickname, ri->address, ri->or_port); + } else if (tor_addr_family(&bridge->addr) == AF_INET6) { + tor_addr_copy(&ri->ipv6_addr, &bridge->addr); + ri->ipv6_orport = bridge->port; + log_info(LD_DIR, + "Adjusted bridge routerinfo for '%s' to match configured " + "address %s:%d.", + ri->nickname, fmt_addr(&ri->ipv6_addr), ri->ipv6_orport); + } else { + log_err(LD_BUG, "Address family not supported: %d.", + tor_addr_family(&bridge->addr)); + return; + } + } + + /* Indicate that we prefer connecting to this bridge over the + protocol that the bridge address indicates. Last bridge + descriptor handled wins. */ + ri->ipv6_preferred = tor_addr_family(&bridge->addr) == AF_INET6; + + /* XXXipv6 we lack support for falling back to another address for + the same relay, warn the user */ + if (!tor_addr_is_null(&ri->ipv6_addr)) { + tor_addr_port_t ap; + router_get_pref_orport(ri, &ap); + log_notice(LD_CONFIG, + "Bridge '%s' has both an IPv4 and an IPv6 address. " + "Will prefer using its %s address (%s:%d).", + ri->nickname, + ri->ipv6_preferred ? "IPv6" : "IPv4", + fmt_addr(&ap.addr), ap.port); + } + } + if (node->rs) { + routerstatus_t *rs = node->rs; + tor_addr_from_ipv4h(&addr, rs->addr); + + if (!tor_addr_compare(&bridge->addr, &addr, CMP_EXACT) && + bridge->port == rs->or_port) { + /* they match, so no need to do anything */ + } else { + rs->addr = tor_addr_to_ipv4h(&bridge->addr); + rs->or_port = bridge->port; + log_info(LD_DIR, + "Adjusted bridge routerstatus for '%s' to match " + "configured address %s:%d.", + rs->nickname, fmt_addr(&bridge->addr), rs->or_port); + } + } } /** We just learned a descriptor for a bridge. See if that @@ -4809,18 +5866,21 @@ learned_bridge_descriptor(routerinfo_t *ri, int from_cache) int first = !any_bridge_descriptors_known(); bridge_info_t *bridge = get_configured_bridge_by_routerinfo(ri); time_t now = time(NULL); - ri->is_running = 1; + router_set_status(ri->cache_info.identity_digest, 1); if (bridge) { /* if we actually want to use this one */ + node_t *node; /* it's here; schedule its re-fetch for a long time from now. */ if (!from_cache) download_status_reset(&bridge->fetch_status); - rewrite_routerinfo_address_for_bridge(bridge, ri); + node = node_get_mutable_by_id(ri->cache_info.identity_digest); + tor_assert(node); + rewrite_node_address_for_bridge(bridge, node); + add_an_entry_guard(node, 1, 1); - add_an_entry_guard(ri, 1); - log_notice(LD_DIR, "new bridge descriptor '%s' (%s)", ri->nickname, - from_cache ? "cached" : "fresh"); + log_notice(LD_DIR, "new bridge descriptor '%s' (%s): %s", ri->nickname, + from_cache ? "cached" : "fresh", router_describe(ri)); /* set entry->made_contact so if it goes down we don't drop it from * our entry node list */ entry_guard_register_connect_status(ri->cache_info.identity_digest, @@ -4851,8 +5911,7 @@ int any_pending_bridge_descriptor_fetches(void) { smartlist_t *conns = get_connection_array(); - SMARTLIST_FOREACH(conns, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { if (conn->type == CONN_TYPE_DIR && conn->purpose == DIR_PURPOSE_FETCH_SERVERDESC && TO_DIR_CONN(conn)->router_purpose == ROUTER_PURPOSE_BRIDGE && @@ -4862,7 +5921,7 @@ any_pending_bridge_descriptor_fetches(void) log_debug(LD_DIR, "found one: %s", conn->address); return 1; } - }); + } SMARTLIST_FOREACH_END(conn); return 0; } @@ -4871,21 +5930,20 @@ any_pending_bridge_descriptor_fetches(void) * down. Else return 0. If <b>act</b> is 1, then mark the down guards * up; else just observe and report. */ static int -entries_retry_helper(or_options_t *options, int act) +entries_retry_helper(const or_options_t *options, int act) { - routerinfo_t *ri; + const node_t *node; int any_known = 0; int any_running = 0; - int purpose = options->UseBridges ? - ROUTER_PURPOSE_BRIDGE : ROUTER_PURPOSE_GENERAL; + int need_bridges = options->UseBridges != 0; if (!entry_guards) - entry_guards = smartlist_create(); - SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, - { - ri = router_get_by_digest(e->identity); - if (ri && ri->purpose == purpose) { + entry_guards = smartlist_new(); + SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, e) { + node = node_get_by_id(e->identity); + if (node && node_has_descriptor(node) && + node_is_bridge(node) == need_bridges) { any_known = 1; - if (ri->is_running) + if (node->is_running) any_running = 1; /* some entry is both known and running */ else if (act) { /* Mark all current connections to this OR as unhealthy, since @@ -4894,15 +5952,15 @@ entries_retry_helper(or_options_t *options, int act) * the node down and undermine the retry attempt. We mark even * the established conns, since if the network just came back * we'll want to attach circuits to fresh conns. */ - connection_or_set_bad_connections(ri->cache_info.identity_digest, 1); + connection_or_set_bad_connections(node->identity, 1); /* mark this entry node for retry */ - router_set_status(ri->cache_info.identity_digest, 1); + router_set_status(node->identity, 1); e->can_retry = 1; e->bad_since = 0; } } - }); + } SMARTLIST_FOREACH_END(e); log_debug(LD_DIR, "%d: any_known %d, any_running %d", act, any_known, any_running); return any_known && !any_running; @@ -4911,7 +5969,7 @@ entries_retry_helper(or_options_t *options, int act) /** Do we know any descriptors for our bridges / entrynodes, and are * all the ones we have descriptors for down? */ int -entries_known_but_down(or_options_t *options) +entries_known_but_down(const or_options_t *options) { tor_assert(entry_list_is_constrained(options)); return entries_retry_helper(options, 0); @@ -4919,12 +5977,40 @@ entries_known_but_down(or_options_t *options) /** Mark all down known bridges / entrynodes up. */ void -entries_retry_all(or_options_t *options) +entries_retry_all(const or_options_t *options) { tor_assert(entry_list_is_constrained(options)); entries_retry_helper(options, 1); } +/** Return true if we've ever had a bridge running a Tor version that can't + * provide microdescriptors to us. In that case fall back to asking for + * full descriptors. Eventually all bridges will support microdescriptors + * and we can take this check out; see bug 4013. */ +int +any_bridges_dont_support_microdescriptors(void) +{ + const node_t *node; + static int ever_answered_yes = 0; + if (!get_options()->UseBridges || !entry_guards) + return 0; + if (ever_answered_yes) + return 1; /* if we ever answer 'yes', always answer 'yes' */ + SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, e) { + node = node_get_by_id(e->identity); + if (node && node->ri && + node_is_bridge(node) && node_is_a_configured_bridge(node) && + !tor_version_supports_microdescriptors(node->ri->platform)) { + /* This is one of our current bridges, and we know enough about + * it to know that it won't be able to answer our microdescriptor + * questions. */ + ever_answered_yes = 1; + return 1; + } + } SMARTLIST_FOREACH_END(e); + return 0; +} + /** Release all storage held by the list of entry guards and related * memory structs. */ void @@ -4937,7 +6023,10 @@ entry_guards_free_all(void) entry_guards = NULL; } clear_bridge_list(); + clear_transport_list(); smartlist_free(bridge_list); + smartlist_free(transport_list); bridge_list = NULL; + transport_list = NULL; } diff --git a/src/or/circuitbuild.h b/src/or/circuitbuild.h index 0e673e1c05..984d04a99e 100644 --- a/src/or/circuitbuild.h +++ b/src/or/circuitbuild.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -12,6 +12,21 @@ #ifndef _TOR_CIRCUITBUILD_H #define _TOR_CIRCUITBUILD_H +/** Represents a pluggable transport proxy used by a bridge. */ +typedef struct { + /** SOCKS version: One of PROXY_SOCKS4, PROXY_SOCKS5. */ + int socks_version; + /** Name of pluggable transport protocol */ + char *name; + /** Address of proxy */ + tor_addr_t addr; + /** Port of proxy */ + uint16_t port; + /** Boolean: We are re-parsing our transport list, and we are going to remove + * this one if we don't find it in the list of configured transports. */ + unsigned marked_for_removal : 1; +} transport_t; + char *circuit_list_path(origin_circuit_t *circ, int verbose); char *circuit_list_path_for_controller(origin_circuit_t *circ); void circuit_log_path(int severity, unsigned int domain, @@ -42,20 +57,23 @@ int circuit_append_new_exit(origin_circuit_t *circ, extend_info_t *info); int circuit_extend_to_new_exit(origin_circuit_t *circ, extend_info_t *info); void onion_append_to_cpath(crypt_path_t **head_ptr, crypt_path_t *new_hop); extend_info_t *extend_info_alloc(const char *nickname, const char *digest, - crypto_pk_env_t *onion_key, + crypto_pk_t *onion_key, const tor_addr_t *addr, uint16_t port); -extend_info_t *extend_info_from_router(routerinfo_t *r); +extend_info_t *extend_info_from_router(const routerinfo_t *r, + int for_direct_connect); +extend_info_t *extend_info_from_node(const node_t *node, + int for_direct_connect); extend_info_t *extend_info_dup(extend_info_t *info); void extend_info_free(extend_info_t *info); -routerinfo_t *build_state_get_exit_router(cpath_build_state_t *state); +const node_t *build_state_get_exit_node(cpath_build_state_t *state); const char *build_state_get_exit_nickname(cpath_build_state_t *state); -void entry_guards_compute_status(or_options_t *options, time_t now); +void entry_guards_compute_status(const or_options_t *options, time_t now); int entry_guard_register_connect_status(const char *digest, int succeeded, int mark_relay_status, time_t now); void entry_nodes_should_be_added(void); -int entry_list_is_constrained(or_options_t *options); -routerinfo_t *choose_random_entry(cpath_build_state_t *state); +int entry_list_is_constrained(const or_options_t *options); +const node_t *choose_random_entry(cpath_build_state_t *state); int entry_guards_parse_state(or_state_t *state, int set, char **msg); void entry_guards_update_state(or_state_t *state); int getinfo_helper_entry_guards(control_connection_t *conn, @@ -64,18 +82,25 @@ int getinfo_helper_entry_guards(control_connection_t *conn, void mark_bridge_list(void); void sweep_bridge_list(void); -int routerinfo_is_a_configured_bridge(routerinfo_t *ri); +void mark_transport_list(void); +void sweep_transport_list(void); + +int routerinfo_is_a_configured_bridge(const routerinfo_t *ri); +int node_is_a_configured_bridge(const node_t *node); void learned_router_identity(const tor_addr_t *addr, uint16_t port, const char *digest); void bridge_add_from_config(const tor_addr_t *addr, uint16_t port, - const char *digest); + const char *digest, + const char *transport_name); void retry_bridge_descriptor_fetch_directly(const char *digest); -void fetch_bridge_descriptors(or_options_t *options, time_t now); +void fetch_bridge_descriptors(const or_options_t *options, time_t now); void learned_bridge_descriptor(routerinfo_t *ri, int from_cache); int any_bridge_descriptors_known(void); int any_pending_bridge_descriptor_fetches(void); -int entries_known_but_down(or_options_t *options); -void entries_retry_all(or_options_t *options); +int entries_known_but_down(const or_options_t *options); +void entries_retry_all(const or_options_t *options); + +int any_bridges_dont_support_microdescriptors(void); void entry_guards_free_all(void); @@ -96,6 +121,7 @@ int circuit_build_times_needs_circuits(circuit_build_times_t *cbt); int circuit_build_times_needs_circuits_now(circuit_build_times_t *cbt); void circuit_build_times_init(circuit_build_times_t *cbt); +void circuit_build_times_free_timeouts(circuit_build_times_t *cbt); void circuit_build_times_new_consensus_params(circuit_build_times_t *cbt, networkstatus_t *ns); double circuit_build_times_timeout_rate(const circuit_build_times_t *cbt); @@ -122,7 +148,24 @@ void circuit_build_times_network_is_live(circuit_build_times_t *cbt); int circuit_build_times_network_check_live(circuit_build_times_t *cbt); void circuit_build_times_network_circ_success(circuit_build_times_t *cbt); +/* DOCDOC circuit_build_times_get_bw_scale */ int circuit_build_times_get_bw_scale(networkstatus_t *ns); +void clear_transport_list(void); +int transport_add_from_config(const tor_addr_t *addr, uint16_t port, + const char *name, int socks_ver); +int transport_add(transport_t *t); +void transport_free(transport_t *transport); +transport_t *transport_new(const tor_addr_t *addr, uint16_t port, + const char *name, int socks_ver); + +/* DOCDOC find_transport_name_by_bridge_addrport */ +const char *find_transport_name_by_bridge_addrport(const tor_addr_t *addr, + uint16_t port); + +int find_transport_by_bridge_addrport(const tor_addr_t *addr, uint16_t port, + const transport_t **transport); +transport_t *transport_get_by_name(const char *name); + #endif diff --git a/src/or/circuitlist.c b/src/or/circuitlist.c index e9cc9eb1f4..d9ea4d1b51 100644 --- a/src/or/circuitlist.c +++ b/src/or/circuitlist.c @@ -1,7 +1,7 @@ /* Copyright 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -19,6 +19,7 @@ #include "connection_or.h" #include "control.h" #include "networkstatus.h" +#include "nodelist.h" #include "onion.h" #include "relay.h" #include "rendclient.h" @@ -38,6 +39,7 @@ static smartlist_t *circuits_pending_or_conns=NULL; static void circuit_free(circuit_t *circ); static void circuit_free_cpath(crypt_path_t *cpath); static void circuit_free_cpath_node(crypt_path_t *victim); +static void cpath_ref_decref(crypt_path_reference_t *cpath_ref); /********* END VARIABLES ************/ @@ -86,10 +88,7 @@ orconn_circid_circuit_map_t *_last_circid_orconn_ent = NULL; /** Implementation helper for circuit_set_{p,n}_circid_orconn: A circuit ID * and/or or_connection for circ has just changed from <b>old_conn, old_id</b> * to <b>conn, id</b>. Adjust the conn,circid map as appropriate, removing - * the old entry (if any) and adding a new one. If <b>active</b> is true, - * remove the circuit from the list of active circuits on old_conn and add it - * to the list of active circuits on conn. - * XXX "active" isn't an arg anymore */ + * the old entry (if any) and adding a new one. */ static void circuit_set_circid_orconn_helper(circuit_t *circ, int direction, circid_t id, @@ -203,7 +202,7 @@ circuit_set_state(circuit_t *circ, uint8_t state) if (state == circ->state) return; if (!circuits_pending_or_conns) - circuits_pending_or_conns = smartlist_create(); + circuits_pending_or_conns = smartlist_new(); if (circ->state == CIRCUIT_STATE_OR_WAIT) { /* remove from waiting-circuit list. */ smartlist_remove(circuits_pending_or_conns, circ); @@ -270,7 +269,7 @@ int circuit_count_pending_on_or_conn(or_connection_t *or_conn) { int cnt; - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); circuit_get_all_pending_on_or_conn(sl, or_conn); cnt = smartlist_len(sl); smartlist_free(sl); @@ -380,6 +379,63 @@ circuit_purpose_to_controller_string(uint8_t purpose) } } +/** Return a string specifying the state of the hidden-service circuit + * purpose <b>purpose</b>, or NULL if <b>purpose</b> is not a + * hidden-service-related circuit purpose. */ +const char * +circuit_purpose_to_controller_hs_state_string(uint8_t purpose) +{ + switch (purpose) + { + default: + log_fn(LOG_WARN, LD_BUG, + "Unrecognized circuit purpose: %d", + (int)purpose); + tor_fragile_assert(); + /* fall through */ + + case CIRCUIT_PURPOSE_OR: + case CIRCUIT_PURPOSE_C_GENERAL: + case CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT: + case CIRCUIT_PURPOSE_TESTING: + case CIRCUIT_PURPOSE_CONTROLLER: + return NULL; + + case CIRCUIT_PURPOSE_INTRO_POINT: + return "OR_HSSI_ESTABLISHED"; + case CIRCUIT_PURPOSE_REND_POINT_WAITING: + return "OR_HSCR_ESTABLISHED"; + case CIRCUIT_PURPOSE_REND_ESTABLISHED: + return "OR_HS_R_JOINED"; + + case CIRCUIT_PURPOSE_C_INTRODUCING: + return "HSCI_CONNECTING"; + case CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT: + return "HSCI_INTRO_SENT"; + case CIRCUIT_PURPOSE_C_INTRODUCE_ACKED: + return "HSCI_DONE"; + + case CIRCUIT_PURPOSE_C_ESTABLISH_REND: + return "HSCR_CONNECTING"; + case CIRCUIT_PURPOSE_C_REND_READY: + return "HSCR_ESTABLISHED_IDLE"; + case CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED: + return "HSCR_ESTABLISHED_WAITING"; + case CIRCUIT_PURPOSE_C_REND_JOINED: + return "HSCR_JOINED"; + + case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO: + return "HSSI_CONNECTING"; + case CIRCUIT_PURPOSE_S_INTRO: + return "HSSI_ESTABLISHED"; + + case CIRCUIT_PURPOSE_S_CONNECT_REND: + return "HSSR_CONNECTING"; + case CIRCUIT_PURPOSE_S_REND_JOINED: + return "HSSR_JOINED"; + } +} + /** Return a human-readable string for the circuit purpose <b>purpose</b>. */ const char * circuit_purpose_to_string(uint8_t purpose) @@ -545,13 +601,24 @@ circuit_free(circuit_t *circ) if (ocirc->build_state) { extend_info_free(ocirc->build_state->chosen_exit); circuit_free_cpath_node(ocirc->build_state->pending_final_cpath); + cpath_ref_decref(ocirc->build_state->service_pending_final_cpath_ref); } tor_free(ocirc->build_state); circuit_free_cpath(ocirc->cpath); - crypto_free_pk_env(ocirc->intro_key); + crypto_pk_free(ocirc->intro_key); rend_data_free(ocirc->rend_data); + + tor_free(ocirc->dest_address); + if (ocirc->socks_username) { + memwipe(ocirc->socks_username, 0x12, ocirc->socks_username_len); + tor_free(ocirc->socks_username); + } + if (ocirc->socks_password) { + memwipe(ocirc->socks_password, 0x06, ocirc->socks_password_len); + tor_free(ocirc->socks_password); + } } else { or_circuit_t *ocirc = TO_OR_CIRCUIT(circ); /* Remember cell statistics for this circuit before deallocating. */ @@ -561,10 +628,10 @@ circuit_free(circuit_t *circ) memlen = sizeof(or_circuit_t); tor_assert(circ->magic == OR_CIRCUIT_MAGIC); - crypto_free_cipher_env(ocirc->p_crypto); - crypto_free_digest_env(ocirc->p_digest); - crypto_free_cipher_env(ocirc->n_crypto); - crypto_free_digest_env(ocirc->n_digest); + crypto_cipher_free(ocirc->p_crypto); + crypto_digest_free(ocirc->p_digest); + crypto_cipher_free(ocirc->n_crypto); + crypto_digest_free(ocirc->n_digest); if (ocirc->rend_splice) { or_circuit_t *other = ocirc->rend_splice; @@ -590,7 +657,7 @@ circuit_free(circuit_t *circ) * "active" checks will be violated. */ cell_queue_clear(&circ->n_conn_cells); - memset(mem, 0xAA, memlen); /* poison memory */ + memwipe(mem, 0xAA, memlen); /* poison memory */ tor_free(mem); } @@ -647,17 +714,29 @@ circuit_free_cpath_node(crypt_path_t *victim) if (!victim) return; - crypto_free_cipher_env(victim->f_crypto); - crypto_free_cipher_env(victim->b_crypto); - crypto_free_digest_env(victim->f_digest); - crypto_free_digest_env(victim->b_digest); + crypto_cipher_free(victim->f_crypto); + crypto_cipher_free(victim->b_crypto); + crypto_digest_free(victim->f_digest); + crypto_digest_free(victim->b_digest); crypto_dh_free(victim->dh_handshake_state); extend_info_free(victim->extend_info); - memset(victim, 0xBB, sizeof(crypt_path_t)); /* poison memory */ + memwipe(victim, 0xBB, sizeof(crypt_path_t)); /* poison memory */ tor_free(victim); } +/** Release a crypt_path_reference_t*, which may be NULL. */ +static void +cpath_ref_decref(crypt_path_reference_t *cpath_ref) +{ + if (cpath_ref != NULL) { + if (--(cpath_ref->refcount) == 0) { + circuit_free_cpath_node(cpath_ref->cpath); + tor_free(cpath_ref); + } + } +} + /** A helper function for circuit_dump_by_conn() below. Log a bunch * of information about circuit <b>circ</b>. */ @@ -865,26 +944,30 @@ circuit_unlink_all_from_or_conn(or_connection_t *conn, int reason) } } -/** Return a circ such that: - * - circ-\>rend_data-\>onion_address is equal to <b>rend_query</b>, and - * - circ-\>purpose is equal to <b>purpose</b>. +/** Return a circ such that + * - circ-\>rend_data-\>onion_address is equal to + * <b>rend_data</b>-\>onion_address, + * - circ-\>rend_data-\>rend_cookie is equal to + * <b>rend_data</b>-\>rend_cookie, and + * - circ-\>purpose is equal to CIRCUIT_PURPOSE_C_REND_READY. * * Return NULL if no such circuit exists. */ origin_circuit_t * -circuit_get_by_rend_query_and_purpose(const char *rend_query, uint8_t purpose) +circuit_get_ready_rend_circ_by_rend_data(const rend_data_t *rend_data) { circuit_t *circ; - tor_assert(CIRCUIT_PURPOSE_IS_ORIGIN(purpose)); - for (circ = global_circuitlist; circ; circ = circ->next) { if (!circ->marked_for_close && - circ->purpose == purpose) { + circ->purpose == CIRCUIT_PURPOSE_C_REND_READY) { origin_circuit_t *ocirc = TO_ORIGIN_CIRCUIT(circ); if (ocirc->rend_data && - !rend_cmp_service_ids(rend_query, - ocirc->rend_data->onion_address)) + !rend_cmp_service_ids(rend_data->onion_address, + ocirc->rend_data->onion_address) && + tor_memeq(ocirc->rend_data->rend_cookie, + rend_data->rend_cookie, + REND_COOKIE_LEN)) return ocirc; } } @@ -981,7 +1064,7 @@ circuit_find_to_cannibalize(uint8_t purpose, extend_info_t *info, int need_uptime = (flags & CIRCLAUNCH_NEED_UPTIME) != 0; int need_capacity = (flags & CIRCLAUNCH_NEED_CAPACITY) != 0; int internal = (flags & CIRCLAUNCH_IS_INTERNAL) != 0; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); /* Make sure we're not trying to create a onehop circ by * cannibalization. */ @@ -1003,19 +1086,21 @@ circuit_find_to_cannibalize(uint8_t purpose, extend_info_t *info, (!need_capacity || circ->build_state->need_capacity) && (internal == circ->build_state->is_internal) && circ->remaining_relay_early_cells && - !circ->build_state->onehop_tunnel) { + circ->build_state->desired_path_len == DEFAULT_ROUTE_LEN && + !circ->build_state->onehop_tunnel && + !circ->isolation_values_set) { if (info) { /* need to make sure we don't duplicate hops */ crypt_path_t *hop = circ->cpath; - routerinfo_t *ri1 = router_get_by_digest(info->identity_digest); + const node_t *ri1 = node_get_by_id(info->identity_digest); do { - routerinfo_t *ri2; + const node_t *ri2; if (tor_memeq(hop->extend_info->identity_digest, info->identity_digest, DIGEST_LEN)) goto next; if (ri1 && - (ri2 = router_get_by_digest(hop->extend_info->identity_digest)) - && routers_in_same_family(ri1, ri2)) + (ri2 = node_get_by_id(hop->extend_info->identity_digest)) + && nodes_in_same_family(ri1, ri2)) goto next; hop=hop->next; } while (hop!=circ->cpath); @@ -1095,18 +1180,18 @@ circuit_mark_all_unused_circs(void) * This is useful for letting the user change pseudonyms, so new * streams will not be linkable to old streams. */ -/* XXX023 this is a bad name for what this function does */ +/* XXX024 this is a bad name for what this function does */ void circuit_expire_all_dirty_circs(void) { circuit_t *circ; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); for (circ=global_circuitlist; circ; circ = circ->next) { if (CIRCUIT_IS_ORIGIN(circ) && !circ->marked_for_close && circ->timestamp_dirty) - /* XXXX023 This is a screwed-up way to say "This is too dirty + /* XXXX024 This is a screwed-up way to say "This is too dirty * for new circuits. */ circ->timestamp_dirty -= options->MaxCircuitDirtiness; } @@ -1119,9 +1204,11 @@ circuit_expire_all_dirty_circs(void) * - If circ isn't open yet: call circuit_build_failed() if we're * the origin, and in either case call circuit_rep_hist_note_result() * to note stats. - * - If purpose is C_INTRODUCE_ACK_WAIT, remove the intro point we - * just tried from our list of intro points for that service - * descriptor. + * - If purpose is C_INTRODUCE_ACK_WAIT, report the intro point + * failure we just had to the hidden service client module. + * - If purpose is C_INTRODUCING and <b>reason</b> isn't TIMEOUT, + * report to the hidden service client module that the intro point + * we just tried may be unreachable. * - Send appropriate destroys and edge_destroys for conns and * streams attached to circ. * - If circ->rend_splice is set (we are the midpoint of a joined @@ -1190,16 +1277,33 @@ _circuit_mark_for_close(circuit_t *circ, int reason, int line, } if (circ->purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT) { origin_circuit_t *ocirc = TO_ORIGIN_CIRCUIT(circ); + int timed_out = (reason == END_CIRC_REASON_TIMEOUT); tor_assert(circ->state == CIRCUIT_STATE_OPEN); tor_assert(ocirc->build_state->chosen_exit); tor_assert(ocirc->rend_data); /* treat this like getting a nack from it */ - log_info(LD_REND, "Failed intro circ %s to %s (awaiting ack). " - "Removing from descriptor.", + log_info(LD_REND, "Failed intro circ %s to %s (awaiting ack). %s", safe_str_client(ocirc->rend_data->onion_address), + safe_str_client(build_state_get_exit_nickname(ocirc->build_state)), + timed_out ? "Recording timeout." : "Removing from descriptor."); + rend_client_report_intro_point_failure(ocirc->build_state->chosen_exit, + ocirc->rend_data, + timed_out ? + INTRO_POINT_FAILURE_TIMEOUT : + INTRO_POINT_FAILURE_GENERIC); + } else if (circ->purpose == CIRCUIT_PURPOSE_C_INTRODUCING && + reason != END_CIRC_REASON_TIMEOUT) { + origin_circuit_t *ocirc = TO_ORIGIN_CIRCUIT(circ); + if (ocirc->build_state->chosen_exit && ocirc->rend_data) { + log_info(LD_REND, "Failed intro circ %s to %s " + "(building circuit to intro point). " + "Marking intro point as possibly unreachable.", + safe_str_client(ocirc->rend_data->onion_address), safe_str_client(build_state_get_exit_nickname(ocirc->build_state))); - rend_client_remove_intro_point(ocirc->build_state->chosen_exit, - ocirc->rend_data); + rend_client_report_intro_point_failure(ocirc->build_state->chosen_exit, + ocirc->rend_data, + INTRO_POINT_FAILURE_UNREACHABLE); + } } if (circ->n_conn) { circuit_clear_cell_queue(circ, circ->n_conn); @@ -1255,6 +1359,108 @@ _circuit_mark_for_close(circuit_t *circ, int reason, int line, } } +/** Given a marked circuit <b>circ</b>, aggressively free its cell queues to + * recover memory. */ +static void +marked_circuit_free_cells(circuit_t *circ) +{ + if (!circ->marked_for_close) { + log_warn(LD_BUG, "Called on non-marked circuit"); + return; + } + cell_queue_clear(&circ->n_conn_cells); + if (! CIRCUIT_IS_ORIGIN(circ)) + cell_queue_clear(& TO_OR_CIRCUIT(circ)->p_conn_cells); +} + +/** Return the number of cells used by the circuit <b>c</b>'s cell queues. */ +static size_t +n_cells_in_circ_queues(const circuit_t *c) +{ + size_t n = c->n_conn_cells.n; + if (! CIRCUIT_IS_ORIGIN(c)) + n += TO_OR_CIRCUIT((circuit_t*)c)->p_conn_cells.n; + return n; +} + +/** helper to sort a list of circuit_q by total queue lengths, in descending + * order. */ +static int +circuits_compare_by_queue_len_(const void **a_, const void **b_) +{ + const circuit_t *a = *a_; + const circuit_t *b = *b_; + size_t a_n = n_cells_in_circ_queues(a); + size_t b_n = n_cells_in_circ_queues(b); + + if (a_n < b_n) + return 1; + else if (a_n == b_n) + return 0; + else + return -1; +} + +#define FRACTION_OF_CIRCS_TO_RETAIN_ON_OOM 0.90 + +/** We're out of memory for cells, having allocated <b>current_allocation</b> + * bytes' worth. Kill the 'worst' circuits until we're under + * FRACTION_OF_CIRCS_TO_RETAIN_ON_OOM of our maximum usage. */ +void +circuits_handle_oom(size_t current_allocation) +{ + /* Let's hope there's enough slack space for this allocation here... */ + smartlist_t *circlist = smartlist_new(); + circuit_t *circ; + size_t n_cells_removed=0, n_cells_to_remove; + int n_circuits_killed=0; + log_notice(LD_GENERAL, "We're low on memory. Killing circuits with " + "over-long queues. (This behavior is controlled by " + "MaxMemInCellQueues.)"); + + { + size_t mem_target = (size_t)(get_options()->MaxMemInCellQueues * + FRACTION_OF_CIRCS_TO_RETAIN_ON_OOM); + size_t mem_to_recover; + if (current_allocation <= mem_target) + return; + mem_to_recover = current_allocation - mem_target; + n_cells_to_remove = CEIL_DIV(mem_to_recover, packed_cell_mem_cost()); + } + + /* This algorithm itself assumes that you've got enough memory slack + * to actually run it. */ + for (circ = global_circuitlist; circ; circ = circ->next) + smartlist_add(circlist, circ); + + /* This is O(n log n); there are faster algorithms we could use instead. + * Let's hope this doesn't happen enough to be in the critical path. */ + smartlist_sort(circlist, circuits_compare_by_queue_len_); + + /* Okay, now the worst circuits are at the front of the list. Let's mark + * them, and reclaim their storage aggressively. */ + SMARTLIST_FOREACH_BEGIN(circlist, circuit_t *, circ) { + size_t n = n_cells_in_circ_queues(circ); + if (! circ->marked_for_close) { + circuit_mark_for_close(circ, END_CIRC_REASON_RESOURCELIMIT); + } + marked_circuit_free_cells(circ); + + ++n_circuits_killed; + n_cells_removed += n; + if (n_cells_removed >= n_cells_to_remove) + break; + } SMARTLIST_FOREACH_END(circ); + + clean_cell_pool(); /* In case this helps. */ + + log_notice(LD_GENERAL, "Removed "U64_FORMAT" bytes by killing %d circuits.", + U64_PRINTF_ARG(n_cells_removed * packed_cell_mem_cost()), + n_circuits_killed); + + smartlist_free(circlist); +} + /** Verify that cpath layer <b>cp</b> has all of its invariants * correct. Trigger an assert if anything is invalid. */ diff --git a/src/or/circuitlist.h b/src/or/circuitlist.h index 7b01ca3ae2..44f0c1fe3b 100644 --- a/src/or/circuitlist.h +++ b/src/or/circuitlist.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -15,6 +15,7 @@ circuit_t * _circuit_get_global_list(void); const char *circuit_state_to_string(int state); const char *circuit_purpose_to_controller_string(uint8_t purpose); +const char *circuit_purpose_to_controller_hs_state_string(uint8_t purpose); const char *circuit_purpose_to_string(uint8_t purpose); void circuit_dump_by_conn(connection_t *conn, int severity); void circuit_set_p_circid_orconn(or_circuit_t *circ, circid_t id, @@ -32,8 +33,8 @@ int circuit_id_in_use_on_orconn(circid_t circ_id, or_connection_t *conn); circuit_t *circuit_get_by_edge_conn(edge_connection_t *conn); void circuit_unlink_all_from_or_conn(or_connection_t *conn, int reason); origin_circuit_t *circuit_get_by_global_id(uint32_t id); -origin_circuit_t *circuit_get_by_rend_query_and_purpose(const char *rend_query, - uint8_t purpose); +origin_circuit_t *circuit_get_ready_rend_circ_by_rend_data( + const rend_data_t *rend_data); origin_circuit_t *circuit_get_next_by_pk_and_purpose(origin_circuit_t *start, const char *digest, uint8_t purpose); or_circuit_t *circuit_get_rendezvous(const char *cookie); @@ -56,6 +57,7 @@ int circuit_count_pending_on_or_conn(or_connection_t *or_conn); void assert_cpath_layer_ok(const crypt_path_t *cp); void assert_circuit_ok(const circuit_t *c); void circuit_free_all(void); +void circuits_handle_oom(size_t current_allocation); #endif diff --git a/src/or/circuituse.c b/src/or/circuituse.c index df33f63bb9..20f124eb4e 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -17,6 +17,8 @@ #include "connection.h" #include "connection_edge.h" #include "control.h" +#include "nodelist.h" +#include "networkstatus.h" #include "policies.h" #include "rendclient.h" #include "rendcommon.h" @@ -38,19 +40,19 @@ static void circuit_increment_failure_count(void); * Else return 0. */ static int -circuit_is_acceptable(circuit_t *circ, edge_connection_t *conn, +circuit_is_acceptable(const origin_circuit_t *origin_circ, + const entry_connection_t *conn, int must_be_open, uint8_t purpose, int need_uptime, int need_internal, time_t now) { - routerinfo_t *exitrouter; + const circuit_t *circ = TO_CIRCUIT(origin_circ); + const node_t *exitnode; cpath_build_state_t *build_state; tor_assert(circ); tor_assert(conn); tor_assert(conn->socks_request); - if (!CIRCUIT_IS_ORIGIN(circ)) - return 0; /* this circ doesn't start at us */ if (must_be_open && (circ->state != CIRCUIT_STATE_OPEN || !circ->n_conn)) return 0; /* ignore non-open circs */ if (circ->marked_for_close) @@ -73,6 +75,11 @@ circuit_is_acceptable(circuit_t *circ, edge_connection_t *conn, return 0; } + /* If this is a timed-out hidden service circuit, skip it. */ + if (origin_circ->hs_circ_has_timed_out) { + return 0; + } + if (purpose == CIRCUIT_PURPOSE_C_GENERAL || purpose == CIRCUIT_PURPOSE_C_REND_JOINED) if (circ->timestamp_dirty && @@ -85,8 +92,8 @@ circuit_is_acceptable(circuit_t *circ, edge_connection_t *conn, * circuit, it's the magical extra bob hop. so just check the nickname * of the one we meant to finish at. */ - build_state = TO_ORIGIN_CIRCUIT(circ)->build_state; - exitrouter = build_state_get_exit_router(build_state); + build_state = origin_circ->build_state; + exitnode = build_state_get_exit_node(build_state); if (need_uptime && !build_state->need_uptime) return 0; @@ -94,7 +101,7 @@ circuit_is_acceptable(circuit_t *circ, edge_connection_t *conn, return 0; if (purpose == CIRCUIT_PURPOSE_C_GENERAL) { - if (!exitrouter && !build_state->onehop_tunnel) { + if (!exitnode && !build_state->onehop_tunnel) { log_debug(LD_CIRC,"Not considering circuit with unknown router."); return 0; /* this circuit is screwed and doesn't know it yet, * or is a rendezvous circuit. */ @@ -115,7 +122,7 @@ circuit_is_acceptable(circuit_t *circ, edge_connection_t *conn, if (tor_digest_is_zero(digest)) { /* we don't know the digest; have to compare addr:port */ tor_addr_t addr; - int r = tor_addr_from_str(&addr, conn->socks_request->address); + int r = tor_addr_parse(&addr, conn->socks_request->address); if (r < 0 || !tor_addr_eq(&build_state->chosen_exit->addr, &addr) || build_state->chosen_exit->port != conn->socks_request->port) @@ -128,30 +135,43 @@ circuit_is_acceptable(circuit_t *circ, edge_connection_t *conn, return 0; } } - if (exitrouter && !connection_ap_can_use_exit(conn, exitrouter)) { + if (exitnode && !connection_ap_can_use_exit(conn, exitnode)) { /* can't exit from this router */ return 0; } } else { /* not general */ - origin_circuit_t *ocirc = TO_ORIGIN_CIRCUIT(circ); - if ((conn->rend_data && !ocirc->rend_data) || - (!conn->rend_data && ocirc->rend_data) || - (conn->rend_data && ocirc->rend_data && - rend_cmp_service_ids(conn->rend_data->onion_address, - ocirc->rend_data->onion_address))) { + const edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(conn); + if ((edge_conn->rend_data && !origin_circ->rend_data) || + (!edge_conn->rend_data && origin_circ->rend_data) || + (edge_conn->rend_data && origin_circ->rend_data && + rend_cmp_service_ids(edge_conn->rend_data->onion_address, + origin_circ->rend_data->onion_address))) { /* this circ is not for this conn */ return 0; } } + + if (!connection_edge_compatible_with_circuit(conn, origin_circ)) { + /* conn needs to be isolated from other conns that have already used + * origin_circ */ + return 0; + } + return 1; } /** Return 1 if circuit <b>a</b> is better than circuit <b>b</b> for - * <b>purpose</b>, and return 0 otherwise. Used by circuit_get_best. + * <b>conn</b>, and return 0 otherwise. Used by circuit_get_best. */ static int -circuit_is_better(circuit_t *a, circuit_t *b, uint8_t purpose) +circuit_is_better(const origin_circuit_t *oa, const origin_circuit_t *ob, + const entry_connection_t *conn) { + const circuit_t *a = TO_CIRCUIT(oa); + const circuit_t *b = TO_CIRCUIT(ob); + const uint8_t purpose = ENTRY_TO_CONN(conn)->purpose; + int a_bits, b_bits; + switch (purpose) { case CIRCUIT_PURPOSE_C_GENERAL: /* if it's used but less dirty it's best; @@ -165,8 +185,7 @@ circuit_is_better(circuit_t *a, circuit_t *b, uint8_t purpose) if (a->timestamp_dirty || timercmp(&a->timestamp_created, &b->timestamp_created, >)) return 1; - if (CIRCUIT_IS_ORIGIN(b) && - TO_ORIGIN_CIRCUIT(b)->build_state->is_internal) + if (ob->build_state->is_internal) /* XXX023 what the heck is this internal thing doing here. I * think we can get rid of it. circuit_is_acceptable() already * makes sure that is_internal is exactly what we need it to @@ -185,6 +204,29 @@ circuit_is_better(circuit_t *a, circuit_t *b, uint8_t purpose) return 1; break; } + + /* XXXX023 Maybe this check should get a higher priority to avoid + * using up circuits too rapidly. */ + + a_bits = connection_edge_update_circuit_isolation(conn, + (origin_circuit_t*)oa, 1); + b_bits = connection_edge_update_circuit_isolation(conn, + (origin_circuit_t*)ob, 1); + /* if x_bits < 0, then we have not used x for anything; better not to dirty + * a connection if we can help it. */ + if (a_bits < 0) { + return 0; + } else if (b_bits < 0) { + return 1; + } + a_bits &= ~ oa->isolation_flags_mixed; + a_bits &= ~ ob->isolation_flags_mixed; + if (n_bits_set_u8(a_bits) < n_bits_set_u8(b_bits)) { + /* The fewer new restrictions we need to make on a circuit for stream + * isolation, the better. */ + return 1; + } + return 0; } @@ -205,10 +247,12 @@ circuit_is_better(circuit_t *a, circuit_t *b, uint8_t purpose) * closest introduce-purposed circuit that you can find. */ static origin_circuit_t * -circuit_get_best(edge_connection_t *conn, int must_be_open, uint8_t purpose, +circuit_get_best(const entry_connection_t *conn, + int must_be_open, uint8_t purpose, int need_uptime, int need_internal) { - circuit_t *circ, *best=NULL; + circuit_t *circ; + origin_circuit_t *best=NULL; struct timeval now; int intro_going_on_but_too_old = 0; @@ -221,7 +265,11 @@ circuit_get_best(edge_connection_t *conn, int must_be_open, uint8_t purpose, tor_gettimeofday(&now); for (circ=global_circuitlist;circ;circ = circ->next) { - if (!circuit_is_acceptable(circ,conn,must_be_open,purpose, + origin_circuit_t *origin_circ; + if (!CIRCUIT_IS_ORIGIN(circ)) + continue; + origin_circ = TO_ORIGIN_CIRCUIT(circ); + if (!circuit_is_acceptable(origin_circ,conn,must_be_open,purpose, need_uptime,need_internal,now.tv_sec)) continue; @@ -235,8 +283,8 @@ circuit_get_best(edge_connection_t *conn, int must_be_open, uint8_t purpose, /* now this is an acceptable circ to hand back. but that doesn't * mean it's the *best* circ to hand back. try to decide. */ - if (!best || circuit_is_better(circ,best,purpose)) - best = circ; + if (!best || circuit_is_better(origin_circ,best,conn)) + best = origin_circ; } if (!best && intro_going_on_but_too_old) @@ -244,7 +292,28 @@ circuit_get_best(edge_connection_t *conn, int must_be_open, uint8_t purpose, "right now, but it has already taken quite a while. Starting " "one in parallel."); - return best ? TO_ORIGIN_CIRCUIT(best) : NULL; + return best; +} + +/** Return the number of not-yet-open general-purpose origin circuits. */ +static int +count_pending_general_client_circuits(void) +{ + const circuit_t *circ; + + int count = 0; + + for (circ = global_circuitlist; circ; circ = circ->next) { + if (circ->marked_for_close || + circ->state == CIRCUIT_STATE_OPEN || + circ->purpose != CIRCUIT_PURPOSE_C_GENERAL || + !CIRCUIT_IS_ORIGIN(circ)) + continue; + + ++count; + } + + return count; } #if 0 @@ -287,7 +356,9 @@ circuit_expire_building(void) * circuit_build_times_get_initial_timeout() if we haven't computed * custom timeouts yet */ struct timeval general_cutoff, begindir_cutoff, fourhop_cutoff, - cannibalize_cutoff, close_cutoff, extremely_old_cutoff; + cannibalize_cutoff, close_cutoff, extremely_old_cutoff, + hs_extremely_old_cutoff; + const or_options_t *options = get_options(); struct timeval now; cpath_build_state_t *build_state; @@ -301,12 +372,16 @@ circuit_expire_building(void) } while (0) SET_CUTOFF(general_cutoff, circ_times.timeout_ms); - SET_CUTOFF(begindir_cutoff, circ_times.timeout_ms / 2.0); + SET_CUTOFF(begindir_cutoff, circ_times.timeout_ms); SET_CUTOFF(fourhop_cutoff, circ_times.timeout_ms * (4/3.0)); SET_CUTOFF(cannibalize_cutoff, circ_times.timeout_ms / 2.0); SET_CUTOFF(close_cutoff, circ_times.close_ms); SET_CUTOFF(extremely_old_cutoff, circ_times.close_ms*2 + 1000); + SET_CUTOFF(hs_extremely_old_cutoff, + MAX(circ_times.close_ms*2 + 1000, + options->SocksTimeout * 1000)); + while (next_circ) { struct timeval cutoff; victim = next_circ; @@ -328,6 +403,9 @@ circuit_expire_building(void) else cutoff = general_cutoff; + if (TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out) + cutoff = hs_extremely_old_cutoff; + if (timercmp(&victim->timestamp_created, &cutoff, >)) continue; /* it's still young, leave it alone */ @@ -403,7 +481,7 @@ circuit_expire_building(void) control_event_circuit_status(TO_ORIGIN_CIRCUIT(victim), CIRC_EVENT_FAILED, END_CIRC_REASON_TIMEOUT); - victim->purpose = CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT; + circuit_change_purpose(victim, CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT); /* Record this failure to check for too many timeouts * in a row. This function does not record a time value yet * (we do that later); it only counts the fact that we did @@ -433,6 +511,60 @@ circuit_expire_building(void) } } + /* If this is a hidden service client circuit which is far enough + * along in connecting to its destination, and we haven't already + * flagged it as 'timed out', and the user has not told us to + * close such circs immediately on timeout, flag it as 'timed out' + * so we'll launch another intro or rend circ, but don't mark it + * for close yet. + * + * (Circs flagged as 'timed out' are given a much longer timeout + * period above, so we won't close them in the next call to + * circuit_expire_building.) */ + if (!(options->CloseHSClientCircuitsImmediatelyOnTimeout) && + !(TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out)) { + switch (victim->purpose) { + case CIRCUIT_PURPOSE_C_REND_READY: + /* We only want to spare a rend circ if it has been specified in + * an INTRODUCE1 cell sent to a hidden service. A circ's + * pending_final_cpath field is non-NULL iff it is a rend circ + * and we have tried to send an INTRODUCE1 cell specifying it. + * Thus, if the pending_final_cpath field *is* NULL, then we + * want to not spare it. */ + if (TO_ORIGIN_CIRCUIT(victim)->build_state->pending_final_cpath == + NULL) + break; + case CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT: + case CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED: + /* If we have reached this line, we want to spare the circ for now. */ + log_info(LD_CIRC,"Marking circ %d (state %d:%s, purpose %d) " + "as timed-out HS circ", + victim->n_circ_id, + victim->state, circuit_state_to_string(victim->state), + victim->purpose); + TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out = 1; + continue; + default: + break; + } + } + + /* If this is a service-side rendezvous circuit which is far + * enough along in connecting to its destination, consider sparing + * it. */ + if (!(options->CloseHSServiceRendCircuitsImmediatelyOnTimeout) && + !(TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out) && + victim->purpose == CIRCUIT_PURPOSE_S_CONNECT_REND) { + log_info(LD_CIRC,"Marking circ %d (state %d:%s, purpose %d) " + "as timed-out HS circ; relaunching rendezvous attempt.", + victim->n_circ_id, + victim->state, circuit_state_to_string(victim->state), + victim->purpose); + TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out = 1; + rend_service_relaunch_rendezvous(TO_ORIGIN_CIRCUIT(victim)); + continue; + } + if (victim->n_conn) log_info(LD_CIRC,"Abandoning circ %s:%d:%d (state %d:%s, purpose %d)", victim->n_conn->_base.address, victim->n_conn->_base.port, @@ -481,11 +613,11 @@ circuit_remove_handled_ports(smartlist_t *needed_ports) * Else return 0. */ int -circuit_stream_is_being_handled(edge_connection_t *conn, +circuit_stream_is_being_handled(entry_connection_t *conn, uint16_t port, int min) { circuit_t *circ; - routerinfo_t *exitrouter; + const node_t *exitnode; int num=0; time_t now = time(NULL); int need_uptime = smartlist_string_num_isin(get_options()->LongLivedPorts, @@ -501,14 +633,14 @@ circuit_stream_is_being_handled(edge_connection_t *conn, if (build_state->is_internal || build_state->onehop_tunnel) continue; - exitrouter = build_state_get_exit_router(build_state); - if (exitrouter && (!need_uptime || build_state->need_uptime)) { + exitnode = build_state_get_exit_node(build_state); + if (exitnode && (!need_uptime || build_state->need_uptime)) { int ok; if (conn) { - ok = connection_ap_can_use_exit(conn, exitrouter); + ok = connection_ap_can_use_exit(conn, exitnode); } else { - addr_policy_result_t r = compare_addr_to_addr_policy( - 0, port, exitrouter->exit_policy); + addr_policy_result_t r; + r = compare_tor_addr_to_node_policy(NULL, port, exitnode); ok = r != ADDR_POLICY_REJECTED && r != ADDR_POLICY_PROBABLY_REJECTED; } if (ok) { @@ -575,7 +707,7 @@ circuit_predict_and_launch_new(void) log_info(LD_CIRC, "Have %d clean circs (%d internal), need another exit circ.", num, num_internal); - circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags); + circuit_launch(CIRCUIT_PURPOSE_C_GENERAL, flags); return; } @@ -587,7 +719,7 @@ circuit_predict_and_launch_new(void) "Have %d clean circs (%d internal), need another internal " "circ for my hidden service.", num, num_internal); - circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags); + circuit_launch(CIRCUIT_PURPOSE_C_GENERAL, flags); return; } @@ -605,7 +737,7 @@ circuit_predict_and_launch_new(void) "Have %d clean circs (%d uptime-internal, %d internal), need" " another hidden service circ.", num, num_uptime_internal, num_internal); - circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags); + circuit_launch(CIRCUIT_PURPOSE_C_GENERAL, flags); return; } @@ -614,11 +746,12 @@ circuit_predict_and_launch_new(void) * want, don't do another -- we want to leave a few slots open so * we can still build circuits preemptively as needed. */ if (num < MAX_UNUSED_OPEN_CIRCUITS-2 && + get_options()->LearnCircuitBuildTimeout && circuit_build_times_needs_circuits_now(&circ_times)) { flags = CIRCLAUNCH_NEED_CAPACITY; log_info(LD_CIRC, "Have %d clean circs need another buildtime test circ.", num); - circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags); + circuit_launch(CIRCUIT_PURPOSE_C_GENERAL, flags); return; } } @@ -635,7 +768,7 @@ void circuit_build_needed_circs(time_t now) { static time_t time_to_new_circuit = 0; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); /* launch a new circ for any pending streams that need one */ connection_ap_attach_pending(); @@ -654,9 +787,9 @@ circuit_build_needed_circs(time_t now) circ = circuit_get_youngest_clean_open(CIRCUIT_PURPOSE_C_GENERAL); if (get_options()->RunTesting && circ && - circ->timestamp_created + TESTING_CIRCUIT_INTERVAL < now) { + circ->timestamp_created.tv_sec + TESTING_CIRCUIT_INTERVAL < now) { log_fn(LOG_INFO,"Creating a new testing circuit."); - circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, 0); + circuit_launch(CIRCUIT_PURPOSE_C_GENERAL, 0); } #endif } @@ -675,7 +808,11 @@ circuit_detach_stream(circuit_t *circ, edge_connection_t *conn) tor_assert(circ); tor_assert(conn); - conn->cpath_layer = NULL; /* make sure we don't keep a stale pointer */ + if (conn->_base.type == CONN_TYPE_AP) { + entry_connection_t *entry_conn = EDGE_TO_ENTRY_CONN(conn); + entry_conn->may_use_optimistic_data = 0; + } + conn->cpath_layer = NULL; /* don't keep a stale pointer */ conn->on_circuit = NULL; if (CIRCUIT_IS_ORIGIN(circ)) { @@ -745,7 +882,8 @@ circuit_expire_old_circuits_clientside(void) tor_gettimeofday(&now); cutoff = now; - if (circuit_build_times_needs_circuits(&circ_times)) { + if (get_options()->LearnCircuitBuildTimeout && + circuit_build_times_needs_circuits(&circ_times)) { /* Circuits should be shorter lived if we need more of them * for learning a good build timeout */ cutoff.tv_sec -= IDLE_TIMEOUT_WHILE_LEARNING; @@ -946,7 +1084,15 @@ circuit_has_opened(origin_circuit_t *circ) switch (TO_CIRCUIT(circ)->purpose) { case CIRCUIT_PURPOSE_C_ESTABLISH_REND: rend_client_rendcirc_has_opened(circ); + /* Start building an intro circ if we don't have one yet. */ connection_ap_attach_pending(); + /* This isn't a call to circuit_try_attaching_streams because a + * circuit in _C_ESTABLISH_REND state isn't connected to its + * hidden service yet, thus we can't attach streams to it yet, + * thus circuit_try_attaching_streams would always clear the + * circuit's isolation state. circuit_try_attaching_streams is + * called later, when the rend circ enters _C_REND_JOINED + * state. */ break; case CIRCUIT_PURPOSE_C_INTRODUCING: rend_client_introcirc_has_opened(circ); @@ -954,7 +1100,7 @@ circuit_has_opened(origin_circuit_t *circ) case CIRCUIT_PURPOSE_C_GENERAL: /* Tell any AP connections that have been waiting for a new * circuit that one is ready. */ - connection_ap_attach_pending(); + circuit_try_attaching_streams(circ); break; case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO: /* at Bob, waiting for introductions */ @@ -973,6 +1119,45 @@ circuit_has_opened(origin_circuit_t *circ) } } +/** If the stream-isolation state of <b>circ</b> can be cleared, clear + * it. Return non-zero iff <b>circ</b>'s isolation state was cleared. */ +static int +circuit_try_clearing_isolation_state(origin_circuit_t *circ) +{ + if (/* The circuit may have become non-open if it was cannibalized.*/ + circ->_base.state == CIRCUIT_STATE_OPEN && + /* If !isolation_values_set, there is nothing to clear. */ + circ->isolation_values_set && + /* It's not legal to clear a circuit's isolation info if it's ever had + * streams attached */ + !circ->isolation_any_streams_attached) { + /* If we have any isolation information set on this circuit, and + * we didn't manage to attach any streams to it, then we can + * and should clear it and try again. */ + circuit_clear_isolation(circ); + return 1; + } else { + return 0; + } +} + +/** Called when a circuit becomes ready for streams to be attached to + * it. */ +void +circuit_try_attaching_streams(origin_circuit_t *circ) +{ + /* Attach streams to this circuit if we can. */ + connection_ap_attach_pending(); + + /* The call to circuit_try_clearing_isolation_state here will do + * nothing and return 0 if we didn't attach any streams to circ + * above. */ + if (circuit_try_clearing_isolation_state(circ)) { + /* Maybe *now* we can attach some streams to this circuit. */ + connection_ap_attach_pending(); + } +} + /** Called whenever a circuit could not be successfully built. */ void @@ -1091,17 +1276,9 @@ static int did_circs_fail_last_period = 0; /** Launch a new circuit; see circuit_launch_by_extend_info() for * details on arguments. */ origin_circuit_t * -circuit_launch_by_router(uint8_t purpose, - routerinfo_t *exit, int flags) +circuit_launch(uint8_t purpose, int flags) { - origin_circuit_t *circ; - extend_info_t *info = NULL; - if (exit) - info = extend_info_from_router(exit); - circ = circuit_launch_by_extend_info(purpose, info, flags); - - extend_info_free(info); - return circ; + return circuit_launch_by_extend_info(purpose, NULL, flags); } /** Launch a new circuit with purpose <b>purpose</b> and exit node @@ -1132,14 +1309,22 @@ circuit_launch_by_extend_info(uint8_t purpose, * internal circs rather than exit circs? -RD */ circ = circuit_find_to_cannibalize(purpose, extend_info, flags); if (circ) { + uint8_t old_purpose = circ->_base.purpose; + struct timeval old_timestamp_created; + log_info(LD_CIRC,"Cannibalizing circ '%s' for purpose %d (%s)", build_state_get_exit_nickname(circ->build_state), purpose, circuit_purpose_to_string(purpose)); - circ->_base.purpose = purpose; + + circuit_change_purpose(TO_CIRCUIT(circ), purpose); /* reset the birth date of this circ, else expire_building * will see it and think it's been trying to build since it * began. */ tor_gettimeofday(&circ->_base.timestamp_created); + + control_event_circuit_cannibalized(circ, old_purpose, + &old_timestamp_created); + switch (purpose) { case CIRCUIT_PURPOSE_C_ESTABLISH_REND: case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO: @@ -1207,7 +1392,7 @@ circuit_reset_failure_count(int timeout) * Write the found or in-progress or launched circ into *circp. */ static int -circuit_get_open_circ_or_launch(edge_connection_t *conn, +circuit_get_open_circ_or_launch(entry_connection_t *conn, uint8_t desired_circuit_purpose, origin_circuit_t **circp) { @@ -1215,15 +1400,15 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, int check_exit_policy; int need_uptime, need_internal; int want_onehop; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); tor_assert(conn); tor_assert(circp); - tor_assert(conn->_base.state == AP_CONN_STATE_CIRCUIT_WAIT); + tor_assert(ENTRY_TO_CONN(conn)->state == AP_CONN_STATE_CIRCUIT_WAIT); check_exit_policy = conn->socks_request->command == SOCKS_COMMAND_CONNECT && !conn->use_begindir && - !connection_edge_is_rendezvous_stream(conn); + !connection_edge_is_rendezvous_stream(ENTRY_TO_EDGE_CONN(conn)); want_onehop = conn->want_onehop; need_uptime = !conn->want_onehop && !conn->use_begindir && @@ -1275,12 +1460,14 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, if (check_exit_policy) { if (!conn->chosen_exit_name) { struct in_addr in; - uint32_t addr = 0; - if (tor_inet_aton(conn->socks_request->address, &in)) - addr = ntohl(in.s_addr); - if (router_exit_policy_all_routers_reject(addr, - conn->socks_request->port, - need_uptime)) { + tor_addr_t addr, *addrp=NULL; + if (tor_inet_aton(conn->socks_request->address, &in)) { + tor_addr_from_in(&addr, &in); + addrp = &addr; + } + if (router_exit_policy_all_nodes_reject(addrp, + conn->socks_request->port, + need_uptime)) { log_notice(LD_APP, "No Tor server allows exit to %s:%d. Rejecting.", safe_str_client(conn->socks_request->address), @@ -1288,11 +1475,11 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, return -1; } } else { - /* XXXX023 Duplicates checks in connection_ap_handshake_attach_circuit: + /* XXXX024 Duplicates checks in connection_ap_handshake_attach_circuit: * refactor into a single function? */ - routerinfo_t *router = router_get_by_nickname(conn->chosen_exit_name, 1); + const node_t *node = node_get_by_nickname(conn->chosen_exit_name, 1); int opt = conn->chosen_exit_optional; - if (router && !connection_ap_can_use_exit(conn, router)) { + if (node && !connection_ap_can_use_exit(conn, node)) { log_fn(opt ? LOG_INFO : LOG_WARN, LD_APP, "Requested exit point '%s' is excluded or " "would refuse request. %s.", @@ -1318,22 +1505,37 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, if (!circ) { extend_info_t *extend_info=NULL; uint8_t new_circ_purpose; + const int n_pending = count_pending_general_client_circuits(); + + if (n_pending >= options->MaxClientCircuitsPending) { + static ratelim_t delay_limit = RATELIM_INIT(10*60); + char *m; + if ((m = rate_limit_log(&delay_limit, approx_time()))) { + log_notice(LD_APP, "We'd like to launch a circuit to handle a " + "connection, but we already have %d general-purpose client " + "circuits pending. Waiting until some finish.", + n_pending); + tor_free(m); + } + return 0; + } if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT) { /* need to pick an intro point */ - tor_assert(conn->rend_data); - extend_info = rend_client_get_random_intro(conn->rend_data); + rend_data_t *rend_data = ENTRY_TO_EDGE_CONN(conn)->rend_data; + tor_assert(rend_data); + extend_info = rend_client_get_random_intro(rend_data); if (!extend_info) { log_info(LD_REND, "No intro points for '%s': re-fetching service descriptor.", - safe_str_client(conn->rend_data->onion_address)); - rend_client_refetch_v2_renddesc(conn->rend_data); - conn->_base.state = AP_CONN_STATE_RENDDESC_WAIT; + safe_str_client(rend_data->onion_address)); + rend_client_refetch_v2_renddesc(rend_data); + ENTRY_TO_CONN(conn)->state = AP_CONN_STATE_RENDDESC_WAIT; return 0; } log_info(LD_REND,"Chose %s as intro point for '%s'.", extend_info_describe(extend_info), - safe_str_client(conn->rend_data->onion_address)); + safe_str_client(rend_data->onion_address)); } /* If we have specified a particular exit node for our @@ -1341,11 +1543,14 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, */ if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_GENERAL) { if (conn->chosen_exit_name) { - routerinfo_t *r; + const node_t *r; int opt = conn->chosen_exit_optional; - r = router_get_by_nickname(conn->chosen_exit_name, 1); - if (r) { - extend_info = extend_info_from_router(r); + r = node_get_by_nickname(conn->chosen_exit_name, 1); + if (r && node_has_descriptor(r)) { + /* We might want to connect to an IPv6 bridge for loading + descriptors so we use the preferred address rather than + the primary. */ + extend_info = extend_info_from_node(r, conn->want_onehop ? 1 : 0); } else { log_debug(LD_DIR, "considering %d, %s", want_onehop, conn->chosen_exit_name); @@ -1360,7 +1565,7 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, log_info(LD_DIR, "Broken exit digest on tunnel conn. Closing."); return -1; } - if (tor_addr_from_str(&addr, conn->socks_request->address) < 0) { + if (tor_addr_parse(&addr, conn->socks_request->address) < 0) { log_info(LD_DIR, "Broken address %s on tunnel conn. Closing.", escaped_safe_str_client(conn->socks_request->address)); return -1; @@ -1395,6 +1600,12 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, else new_circ_purpose = desired_circuit_purpose; + if (options->Tor2webMode && + (new_circ_purpose == CIRCUIT_PURPOSE_C_ESTABLISH_REND || + new_circ_purpose == CIRCUIT_PURPOSE_C_INTRODUCING)) { + want_onehop = 1; + } + { int flags = CIRCLAUNCH_NEED_CAPACITY; if (want_onehop) flags |= CIRCLAUNCH_ONEHOP_TUNNEL; @@ -1422,18 +1633,26 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, rep_hist_note_used_internal(time(NULL), need_uptime, 1); if (circ) { /* write the service_id into circ */ - circ->rend_data = rend_data_dup(conn->rend_data); + circ->rend_data = rend_data_dup(ENTRY_TO_EDGE_CONN(conn)->rend_data); if (circ->_base.purpose == CIRCUIT_PURPOSE_C_ESTABLISH_REND && circ->_base.state == CIRCUIT_STATE_OPEN) rend_client_rendcirc_has_opened(circ); } } - } - if (!circ) + } /* endif (!circ) */ + if (circ) { + /* Mark the circuit with the isolation fields for this connection. + * When the circuit arrives, we'll clear these flags: this is + * just some internal bookkeeping to make sure that we have + * launched enough circuits. + */ + connection_edge_update_circuit_isolation(conn, circ, 0); + } else { log_info(LD_APP, "No safe circuit (purpose %d) ready for edge " "connection; delaying.", desired_circuit_purpose); + } *circp = circ; return 0; } @@ -1452,39 +1671,86 @@ cpath_is_on_circuit(origin_circuit_t *circ, crypt_path_t *crypt_path) return 0; } +/** Return true iff client-side optimistic data is supported. */ +static int +optimistic_data_enabled(void) +{ + const or_options_t *options = get_options(); + if (options->OptimisticData < 0) { + /* XXX023 consider having auto default to 1 rather than 0 before + * the 0.2.3 branch goes stable. See bug 3617. -RD */ + const int32_t enabled = + networkstatus_get_param(NULL, "UseOptimisticData", 0, 0, 1); + return (int)enabled; + } + return options->OptimisticData; +} + /** Attach the AP stream <b>apconn</b> to circ's linked list of * p_streams. Also set apconn's cpath_layer to <b>cpath</b>, or to the last * hop in circ's cpath if <b>cpath</b> is NULL. */ static void -link_apconn_to_circ(edge_connection_t *apconn, origin_circuit_t *circ, +link_apconn_to_circ(entry_connection_t *apconn, origin_circuit_t *circ, crypt_path_t *cpath) { + const node_t *exitnode; + /* add it into the linked list of streams on this circuit */ log_debug(LD_APP|LD_CIRC, "attaching new conn to circ. n_circ_id %d.", circ->_base.n_circ_id); /* reset it, so we can measure circ timeouts */ - apconn->_base.timestamp_lastread = time(NULL); - apconn->next_stream = circ->p_streams; - apconn->on_circuit = TO_CIRCUIT(circ); + ENTRY_TO_CONN(apconn)->timestamp_lastread = time(NULL); + ENTRY_TO_EDGE_CONN(apconn)->next_stream = circ->p_streams; + ENTRY_TO_EDGE_CONN(apconn)->on_circuit = TO_CIRCUIT(circ); /* assert_connection_ok(conn, time(NULL)); */ - circ->p_streams = apconn; + circ->p_streams = ENTRY_TO_EDGE_CONN(apconn); + + if (connection_edge_is_rendezvous_stream(ENTRY_TO_EDGE_CONN(apconn))) { + /* We are attaching a stream to a rendezvous circuit. That means + * that an attempt to connect to a hidden service just + * succeeded. Tell rendclient.c. */ + rend_client_note_connection_attempt_ended( + ENTRY_TO_EDGE_CONN(apconn)->rend_data->onion_address); + } if (cpath) { /* we were given one; use it */ tor_assert(cpath_is_on_circuit(circ, cpath)); - apconn->cpath_layer = cpath; - } else { /* use the last hop in the circuit */ + } else { + /* use the last hop in the circuit */ tor_assert(circ->cpath); tor_assert(circ->cpath->prev); tor_assert(circ->cpath->prev->state == CPATH_STATE_OPEN); - apconn->cpath_layer = circ->cpath->prev; + cpath = circ->cpath->prev; + } + ENTRY_TO_EDGE_CONN(apconn)->cpath_layer = cpath; + + circ->isolation_any_streams_attached = 1; + connection_edge_update_circuit_isolation(apconn, circ, 0); + + /* See if we can use optimistic data on this circuit */ + if (cpath->extend_info && + (exitnode = node_get_by_id(cpath->extend_info->identity_digest)) && + exitnode->rs) { + /* Okay; we know what exit node this is. */ + if (optimistic_data_enabled() && + circ->_base.purpose == CIRCUIT_PURPOSE_C_GENERAL && + exitnode->rs->version_supports_optimistic_data) + apconn->may_use_optimistic_data = 1; + else + apconn->may_use_optimistic_data = 0; + log_info(LD_APP, "Looks like completed circuit to %s %s allow " + "optimistic data for connection to %s", + safe_str_client(node_describe(exitnode)), + apconn->may_use_optimistic_data ? "does" : "doesn't", + safe_str_client(apconn->socks_request->address)); } } /** Return true iff <b>address</b> is matched by one of the entries in * TrackHostExits. */ int -hostname_in_track_host_exits(or_options_t *options, const char *address) +hostname_in_track_host_exits(const or_options_t *options, const char *address) { if (!options->TrackHostExits) return 0; @@ -1506,9 +1772,10 @@ hostname_in_track_host_exits(or_options_t *options, const char *address) * <b>conn</b>'s destination. */ static void -consider_recording_trackhost(edge_connection_t *conn, origin_circuit_t *circ) +consider_recording_trackhost(const entry_connection_t *conn, + const origin_circuit_t *circ) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); char *new_address = NULL; char fp[HEX_DIGEST_LEN+1]; @@ -1534,7 +1801,7 @@ consider_recording_trackhost(edge_connection_t *conn, origin_circuit_t *circ) addressmap_register(conn->socks_request->address, new_address, time(NULL) + options->TrackHostExitsExpire, - ADDRMAPSRC_TRACKEXIT); + ADDRMAPSRC_TRACKEXIT, 0, 0); } /** Attempt to attach the connection <b>conn</b> to <b>circ</b>, and send a @@ -1543,18 +1810,19 @@ consider_recording_trackhost(edge_connection_t *conn, origin_circuit_t *circ) * indicated by <b>cpath</b>, or from the last hop in circ's cpath if * <b>cpath</b> is NULL. */ int -connection_ap_handshake_attach_chosen_circuit(edge_connection_t *conn, +connection_ap_handshake_attach_chosen_circuit(entry_connection_t *conn, origin_circuit_t *circ, crypt_path_t *cpath) { + connection_t *base_conn = ENTRY_TO_CONN(conn); tor_assert(conn); - tor_assert(conn->_base.state == AP_CONN_STATE_CIRCUIT_WAIT || - conn->_base.state == AP_CONN_STATE_CONTROLLER_WAIT); + tor_assert(base_conn->state == AP_CONN_STATE_CIRCUIT_WAIT || + base_conn->state == AP_CONN_STATE_CONTROLLER_WAIT); tor_assert(conn->socks_request); tor_assert(circ); tor_assert(circ->_base.state == CIRCUIT_STATE_OPEN); - conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; + base_conn->state = AP_CONN_STATE_CIRCUIT_WAIT; if (!circ->_base.timestamp_dirty) circ->_base.timestamp_dirty = time(NULL); @@ -1584,21 +1852,22 @@ connection_ap_handshake_attach_chosen_circuit(edge_connection_t *conn, /* XXXX this function should mark for close whenever it returns -1; * its callers shouldn't have to worry about that. */ int -connection_ap_handshake_attach_circuit(edge_connection_t *conn) +connection_ap_handshake_attach_circuit(entry_connection_t *conn) { + connection_t *base_conn = ENTRY_TO_CONN(conn); int retval; int conn_age; int want_onehop; tor_assert(conn); - tor_assert(conn->_base.state == AP_CONN_STATE_CIRCUIT_WAIT); + tor_assert(base_conn->state == AP_CONN_STATE_CIRCUIT_WAIT); tor_assert(conn->socks_request); want_onehop = conn->want_onehop; - conn_age = (int)(time(NULL) - conn->_base.timestamp_created); + conn_age = (int)(time(NULL) - base_conn->timestamp_created); if (conn_age >= get_options()->SocksTimeout) { - int severity = (tor_addr_is_null(&conn->_base.addr) && !conn->_base.port) ? + int severity = (tor_addr_is_null(&base_conn->addr) && !base_conn->port) ? LOG_INFO : LOG_NOTICE; log_fn(severity, LD_APP, "Tried for %d seconds to get a connection to %s:%d. Giving up.", @@ -1607,13 +1876,14 @@ connection_ap_handshake_attach_circuit(edge_connection_t *conn) return -1; } - if (!connection_edge_is_rendezvous_stream(conn)) { /* we're a general conn */ + if (!connection_edge_is_rendezvous_stream(ENTRY_TO_EDGE_CONN(conn))) { + /* we're a general conn */ origin_circuit_t *circ=NULL; if (conn->chosen_exit_name) { - routerinfo_t *router = router_get_by_nickname(conn->chosen_exit_name, 1); + const node_t *node = node_get_by_nickname(conn->chosen_exit_name, 1); int opt = conn->chosen_exit_optional; - if (!router && !want_onehop) { + if (!node && !want_onehop) { /* We ran into this warning when trying to extend a circuit to a * hidden service directory for which we didn't have a router * descriptor. See flyspray task 767 for more details. We should @@ -1629,7 +1899,7 @@ connection_ap_handshake_attach_circuit(edge_connection_t *conn) } return -1; } - if (router && !connection_ap_can_use_exit(conn, router)) { + if (node && !connection_ap_can_use_exit(conn, node)) { log_fn(opt ? LOG_INFO : LOG_WARN, LD_APP, "Requested exit point '%s' is excluded or " "would refuse request. %s.", @@ -1646,7 +1916,7 @@ connection_ap_handshake_attach_circuit(edge_connection_t *conn) /* find the circuit that we should use, if there is one. */ retval = circuit_get_open_circ_or_launch( conn, CIRCUIT_PURPOSE_C_GENERAL, &circ); - if (retval < 1) // XXX022 if we totally fail, this still returns 0 -RD + if (retval < 1) // XXX023 if we totally fail, this still returns 0 -RD return retval; log_debug(LD_APP|LD_CIRC, @@ -1662,7 +1932,7 @@ connection_ap_handshake_attach_circuit(edge_connection_t *conn) } else { /* we're a rendezvous conn */ origin_circuit_t *rendcirc=NULL, *introcirc=NULL; - tor_assert(!conn->cpath_layer); + tor_assert(!ENTRY_TO_EDGE_CONN(conn)->cpath_layer); /* start by finding a rendezvous circuit for us */ @@ -1718,8 +1988,9 @@ connection_ap_handshake_attach_circuit(edge_connection_t *conn) !c->marked_for_close && CIRCUIT_IS_ORIGIN(c)) { origin_circuit_t *oc = TO_ORIGIN_CIRCUIT(c); if (oc->rend_data && - !rend_cmp_service_ids(conn->rend_data->onion_address, - oc->rend_data->onion_address)) { + !rend_cmp_service_ids( + ENTRY_TO_EDGE_CONN(conn)->rend_data->onion_address, + oc->rend_data->onion_address)) { log_info(LD_REND|LD_CIRC, "Closing introduction circuit that we " "built in parallel."); circuit_mark_for_close(c, END_CIRC_REASON_TIMEOUT); @@ -1771,3 +2042,39 @@ connection_ap_handshake_attach_circuit(edge_connection_t *conn) } } +/** Change <b>circ</b>'s purpose to <b>new_purpose</b>. */ +void +circuit_change_purpose(circuit_t *circ, uint8_t new_purpose) +{ + uint8_t old_purpose; + /* Don't allow an OR circ to become an origin circ or vice versa. */ + tor_assert(!!(CIRCUIT_IS_ORIGIN(circ)) == + !!(CIRCUIT_PURPOSE_IS_ORIGIN(new_purpose))); + + if (circ->purpose == new_purpose) return; + + if (CIRCUIT_IS_ORIGIN(circ)) { + char old_purpose_desc[80] = ""; + + strncpy(old_purpose_desc, circuit_purpose_to_string(circ->purpose), 80-1); + old_purpose_desc[80-1] = '\0'; + + log_debug(LD_CIRC, + "changing purpose of origin circ %d " + "from \"%s\" (%d) to \"%s\" (%d)", + TO_ORIGIN_CIRCUIT(circ)->global_identifier, + old_purpose_desc, + circ->purpose, + circuit_purpose_to_string(new_purpose), + new_purpose); + } + + old_purpose = circ->purpose; + circ->purpose = new_purpose; + + if (CIRCUIT_IS_ORIGIN(circ)) { + control_event_circuit_purpose_changed(TO_ORIGIN_CIRCUIT(circ), + old_purpose); + } +} + diff --git a/src/or/circuituse.h b/src/or/circuituse.h index bfeaea20dc..be2bd7ec51 100644 --- a/src/or/circuituse.h +++ b/src/or/circuituse.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -14,7 +14,7 @@ void circuit_expire_building(void); void circuit_remove_handled_ports(smartlist_t *needed_ports); -int circuit_stream_is_being_handled(edge_connection_t *conn, uint16_t port, +int circuit_stream_is_being_handled(entry_connection_t *conn, uint16_t port, int min); #if 0 int circuit_conforms_to_options(const origin_circuit_t *circ, @@ -29,6 +29,7 @@ void reset_bandwidth_test(void); int circuit_enough_testing_circs(void); void circuit_has_opened(origin_circuit_t *circ); +void circuit_try_attaching_streams(origin_circuit_t *circ); void circuit_build_failed(origin_circuit_t *circ); /** Flag to set when a circuit should have only a single hop. */ @@ -43,15 +44,17 @@ void circuit_build_failed(origin_circuit_t *circ); origin_circuit_t *circuit_launch_by_extend_info(uint8_t purpose, extend_info_t *info, int flags); -origin_circuit_t *circuit_launch_by_router(uint8_t purpose, - routerinfo_t *exit, int flags); +origin_circuit_t *circuit_launch(uint8_t purpose, int flags); void circuit_reset_failure_count(int timeout); -int connection_ap_handshake_attach_chosen_circuit(edge_connection_t *conn, +int connection_ap_handshake_attach_chosen_circuit(entry_connection_t *conn, origin_circuit_t *circ, crypt_path_t *cpath); -int connection_ap_handshake_attach_circuit(edge_connection_t *conn); +int connection_ap_handshake_attach_circuit(entry_connection_t *conn); -int hostname_in_track_host_exits(or_options_t *options, const char *address); +void circuit_change_purpose(circuit_t *circ, uint8_t new_purpose); + +int hostname_in_track_host_exits(const or_options_t *options, + const char *address); #endif diff --git a/src/or/command.c b/src/or/command.c index 5271a7a907..8321e261e0 100644 --- a/src/or/command.c +++ b/src/or/command.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -25,6 +25,7 @@ #include "control.h" #include "cpuworker.h" #include "hibernate.h" +#include "nodelist.h" #include "onion.h" #include "relay.h" #include "router.h" @@ -45,6 +46,17 @@ uint64_t stats_n_versions_cells_processed = 0; /** How many CELL_NETINFO cells have we received, ever? */ uint64_t stats_n_netinfo_cells_processed = 0; +/** How many CELL_VPADDING cells have we received, ever? */ +uint64_t stats_n_vpadding_cells_processed = 0; +/** How many CELL_CERTS cells have we received, ever? */ +uint64_t stats_n_certs_cells_processed = 0; +/** How many CELL_AUTH_CHALLENGE cells have we received, ever? */ +uint64_t stats_n_auth_challenge_cells_processed = 0; +/** How many CELL_AUTHENTICATE cells have we received, ever? */ +uint64_t stats_n_authenticate_cells_processed = 0; +/** How many CELL_AUTHORIZE cells have we received, ever? */ +uint64_t stats_n_authorize_cells_processed = 0; + /* These are the main functions for processing cells */ static void command_process_create_cell(cell_t *cell, or_connection_t *conn); static void command_process_created_cell(cell_t *cell, or_connection_t *conn); @@ -53,6 +65,14 @@ static void command_process_destroy_cell(cell_t *cell, or_connection_t *conn); static void command_process_versions_cell(var_cell_t *cell, or_connection_t *conn); static void command_process_netinfo_cell(cell_t *cell, or_connection_t *conn); +static void command_process_certs_cell(var_cell_t *cell, + or_connection_t *conn); +static void command_process_auth_challenge_cell(var_cell_t *cell, + or_connection_t *conn); +static void command_process_authenticate_cell(var_cell_t *cell, + or_connection_t *conn); +static int enter_v3_handshake_with_cell(var_cell_t *cell, + or_connection_t *conn); #ifdef KEEP_TIMING_STATS /** This is a wrapper function around the actual function that processes the @@ -92,7 +112,7 @@ command_time_process_cell(cell_t *cell, or_connection_t *conn, int *time, void command_process_cell(cell_t *cell, or_connection_t *conn) { - int handshaking = (conn->_base.state == OR_CONN_STATE_OR_HANDSHAKING); + int handshaking = (conn->_base.state != OR_CONN_STATE_OPEN); #ifdef KEEP_TIMING_STATS /* how many of each cell have we seen so far this second? needs better * name. */ @@ -136,9 +156,20 @@ command_process_cell(cell_t *cell, or_connection_t *conn) return; /* Reject all but VERSIONS and NETINFO when handshaking. */ + /* (VERSIONS should actually be impossible; it's variable-length.) */ if (handshaking && cell->command != CELL_VERSIONS && - cell->command != CELL_NETINFO) + cell->command != CELL_NETINFO) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Received unexpected cell command %d in state %s; closing the " + "connection.", + (int)cell->command, + conn_state_to_string(CONN_TYPE_OR,conn->_base.state)); + connection_mark_for_close(TO_CONN(conn)); return; + } + + if (conn->_base.state == OR_CONN_STATE_OR_HANDSHAKING_V3) + or_handshake_state_record_cell(conn->handshake_state, cell, 1); switch (cell->command) { case CELL_PADDING: @@ -178,6 +209,21 @@ command_process_cell(cell_t *cell, or_connection_t *conn) } } +/** Return true if <b>command</b> is a cell command that's allowed to start a + * V3 handshake. */ +static int +command_allowed_before_handshake(uint8_t command) +{ + switch (command) { + case CELL_VERSIONS: + case CELL_VPADDING: + case CELL_AUTHORIZE: + return 1; + default: + return 0; + } +} + /** Process a <b>cell</b> that was just received on <b>conn</b>. Keep internal * statistics about how many of each cell we've processed so far * this second, and the total number of microseconds it took to @@ -189,19 +235,19 @@ command_process_var_cell(var_cell_t *cell, or_connection_t *conn) #ifdef KEEP_TIMING_STATS /* how many of each cell have we seen so far this second? needs better * name. */ - static int num_versions=0, num_cert=0; + static int num_versions=0, num_certs=0; time_t now = time(NULL); if (now > current_second) { /* the second has rolled over */ /* print stats */ log_info(LD_OR, - "At end of second: %d versions (%d ms), %d cert (%d ms)", + "At end of second: %d versions (%d ms), %d certs (%d ms)", num_versions, versions_time/1000, - cert, cert_time/1000); + num_certs, certs_time/1000); - num_versions = num_cert = 0; - versions_time = cert_time = 0; + num_versions = num_certs = 0; + versions_time = certs_time = 0; /* remember which second it is, for next time */ current_second = now; @@ -211,20 +257,94 @@ command_process_var_cell(var_cell_t *cell, or_connection_t *conn) if (conn->_base.marked_for_close) return; - /* reject all when not handshaking. */ - if (conn->_base.state != OR_CONN_STATE_OR_HANDSHAKING) - return; + switch (conn->_base.state) + { + case OR_CONN_STATE_OR_HANDSHAKING_V2: + if (cell->command != CELL_VERSIONS) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Received a cell with command %d in state %s; " + "closing the connection.", + (int)cell->command, + conn_state_to_string(CONN_TYPE_OR,conn->_base.state)); + connection_mark_for_close(TO_CONN(conn)); + return; + } + break; + case OR_CONN_STATE_TLS_HANDSHAKING: + /* If we're using bufferevents, it's entirely possible for us to + * notice "hey, data arrived!" before we notice "hey, the handshake + * finished!" And we need to be accepting both at once to handle both + * the v2 and v3 handshakes. */ + + /* fall through */ + case OR_CONN_STATE_TLS_SERVER_RENEGOTIATING: + if (! command_allowed_before_handshake(cell->command)) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Received a cell with command %d in state %s; " + "closing the connection.", + (int)cell->command, + conn_state_to_string(CONN_TYPE_OR,conn->_base.state)); + connection_mark_for_close(TO_CONN(conn)); + return; + } else { + if (enter_v3_handshake_with_cell(cell, conn)<0) + return; + } + break; + case OR_CONN_STATE_OR_HANDSHAKING_V3: + if (cell->command != CELL_AUTHENTICATE) + or_handshake_state_record_var_cell(conn->handshake_state, cell, 1); + break; /* Everything is allowed */ + case OR_CONN_STATE_OPEN: + if (conn->link_proto < 3) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Received a variable-length cell with command %d in state %s " + "with link protocol %d; ignoring it.", + (int)cell->command, + conn_state_to_string(CONN_TYPE_OR,conn->_base.state), + (int)conn->link_proto); + return; + } + break; + default: + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Received var-length cell with command %d in unexpected state " + "%s [%d]; ignoring it.", + (int)cell->command, + conn_state_to_string(CONN_TYPE_OR,conn->_base.state), + (int)conn->_base.state); + return; + } switch (cell->command) { case CELL_VERSIONS: ++stats_n_versions_cells_processed; PROCESS_CELL(versions, cell, conn); break; + case CELL_VPADDING: + ++stats_n_vpadding_cells_processed; + /* Do nothing */ + break; + case CELL_CERTS: + ++stats_n_certs_cells_processed; + PROCESS_CELL(certs, cell, conn); + break; + case CELL_AUTH_CHALLENGE: + ++stats_n_auth_challenge_cells_processed; + PROCESS_CELL(auth_challenge, cell, conn); + break; + case CELL_AUTHENTICATE: + ++stats_n_authenticate_cells_processed; + PROCESS_CELL(authenticate, cell, conn); + break; + case CELL_AUTHORIZE: + ++stats_n_authorize_cells_processed; + /* Ignored so far. */ + break; default: - log_warn(LD_BUG, + log_fn(LOG_INFO, LD_PROTOCOL, "Variable-length cell of unknown type (%d) received.", cell->command); - tor_fragile_assert(); break; } } @@ -238,7 +358,7 @@ static void command_process_create_cell(cell_t *cell, or_connection_t *conn) { or_circuit_t *circ; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int id_is_high; if (we_are_hibernating()) { @@ -262,6 +382,14 @@ command_process_create_cell(cell_t *cell, or_connection_t *conn) return; } + if (cell->circ_id == 0) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Received a create cell (type %d) from %s:%d with zero circID; " + " ignoring.", (int)cell->command, conn->_base.address, + conn->_base.port); + return; + } + /* If the high bit of the circuit ID is not as expected, close the * circ. */ id_is_high = cell->circ_id & (1<<15); @@ -276,15 +404,18 @@ command_process_create_cell(cell_t *cell, or_connection_t *conn) } if (circuit_id_in_use_on_orconn(cell->circ_id, conn)) { - routerinfo_t *router = router_get_by_digest(conn->identity_digest); + const node_t *node = node_get_by_id(conn->identity_digest); log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, "Received CREATE cell (circID %d) for known circ. " "Dropping (age %d).", cell->circ_id, (int)(time(NULL) - conn->_base.timestamp_created)); - if (router) + if (node) { + char *p = esc_for_log(node_get_platform(node)); log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, "Details: router %s, platform %s.", - router_describe(router), escaped(router->platform)); + node_describe(node), p); + tor_free(p); + } return; } @@ -305,7 +436,7 @@ command_process_create_cell(cell_t *cell, or_connection_t *conn) log_warn(LD_GENERAL,"Failed to hand off onionskin. Closing.%s",m); tor_free(m); } - circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_INTERNAL); + circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_RESOURCELIMIT); return; } log_debug(LD_OR,"success: handed off onionskin."); @@ -478,7 +609,6 @@ command_process_destroy_cell(cell_t *cell, or_connection_t *conn) int reason; circ = circuit_get_by_circid_orconn(cell->circ_id, conn); - reason = (uint8_t)cell->payload[0]; if (!circ) { log_info(LD_OR,"unknown circuit %d on connection from %s:%d. Dropping.", cell->circ_id, conn->_base.address, conn->_base.port); @@ -486,6 +616,8 @@ command_process_destroy_cell(cell_t *cell, or_connection_t *conn) } log_debug(LD_OR,"Received for circID %d.",cell->circ_id); + reason = (uint8_t)cell->payload[0]; + if (!CIRCUIT_IS_ORIGIN(circ) && cell->circ_id == TO_OR_CIRCUIT(circ)->p_circ_id) { /* the destroy came from behind */ @@ -505,6 +637,36 @@ command_process_destroy_cell(cell_t *cell, or_connection_t *conn) } } +/** Called when we as a server receive an appropriate cell while waiting + * either for a cell or a TLS handshake. Set the connection's state to + * "handshaking_v3', initializes the or_handshake_state field as needed, + * and add the cell to the hash of incoming cells.) + * + * Return 0 on success; return -1 and mark the connection on failure. + */ +static int +enter_v3_handshake_with_cell(var_cell_t *cell, or_connection_t *conn) +{ + const int started_here = connection_or_nonopen_was_started_here(conn); + + tor_assert(conn->_base.state == OR_CONN_STATE_TLS_HANDSHAKING || + conn->_base.state == OR_CONN_STATE_TLS_SERVER_RENEGOTIATING); + + if (started_here) { + log_fn(LOG_PROTOCOL_WARN, LD_OR, + "Received a cell while TLS-handshaking, not in " + "OR_HANDSHAKING_V3, on a connection we originated."); + } + connection_or_block_renegotiation(conn); + conn->_base.state = OR_CONN_STATE_OR_HANDSHAKING_V3; + if (connection_init_or_handshake_state(conn, started_here) < 0) { + connection_mark_for_close(TO_CONN(conn)); + return -1; + } + or_handshake_state_record_var_cell(conn->handshake_state, cell, 1); + return 0; +} + /** Process a 'versions' cell. The current link protocol version must be 0 * to indicate that no version has yet been negotiated. We compare the * versions in the cell to the list of versions we support, pick the @@ -516,14 +678,27 @@ command_process_versions_cell(var_cell_t *cell, or_connection_t *conn) { int highest_supported_version = 0; const uint8_t *cp, *end; + const int started_here = connection_or_nonopen_was_started_here(conn); if (conn->link_proto != 0 || - conn->_base.state != OR_CONN_STATE_OR_HANDSHAKING || (conn->handshake_state && conn->handshake_state->received_versions)) { log_fn(LOG_PROTOCOL_WARN, LD_OR, "Received a VERSIONS cell on a connection with its version " "already set to %d; dropping", (int) conn->link_proto); return; } + switch (conn->_base.state) + { + case OR_CONN_STATE_OR_HANDSHAKING_V2: + case OR_CONN_STATE_OR_HANDSHAKING_V3: + break; + case OR_CONN_STATE_TLS_HANDSHAKING: + case OR_CONN_STATE_TLS_SERVER_RENEGOTIATING: + default: + log_fn(LOG_PROTOCOL_WARN, LD_OR, + "VERSIONS cell while in unexpected state"); + return; + } + tor_assert(conn->handshake_state); end = cell->payload + cell->payload_len; for (cp = cell->payload; cp+1 < end; ++cp) { @@ -545,19 +720,95 @@ command_process_versions_cell(var_cell_t *cell, or_connection_t *conn) "That's crazily non-compliant. Closing connection."); connection_mark_for_close(TO_CONN(conn)); return; + } else if (highest_supported_version < 3 && + conn->_base.state == OR_CONN_STATE_OR_HANDSHAKING_V3) { + log_fn(LOG_PROTOCOL_WARN, LD_OR, + "Negotiated link protocol 2 or lower after doing a v3 TLS " + "handshake. Closing connection."); + connection_mark_for_close(TO_CONN(conn)); + return; + } else if (highest_supported_version != 2 && + conn->_base.state == OR_CONN_STATE_OR_HANDSHAKING_V2) { + /* XXXX This should eventually be a log_protocol_warn */ + log_fn(LOG_WARN, LD_OR, + "Negotiated link with non-2 protocol after doing a v2 TLS " + "handshake with %s. Closing connection.", + fmt_addr(&conn->_base.addr)); + connection_mark_for_close(TO_CONN(conn)); + return; } + conn->link_proto = highest_supported_version; conn->handshake_state->received_versions = 1; - log_info(LD_OR, "Negotiated version %d with %s:%d; sending NETINFO.", - highest_supported_version, - safe_str_client(conn->_base.address), - conn->_base.port); - tor_assert(conn->link_proto >= 2); + if (conn->link_proto == 2) { + log_info(LD_OR, "Negotiated version %d with %s:%d; sending NETINFO.", + highest_supported_version, + safe_str_client(conn->_base.address), + conn->_base.port); - if (connection_or_send_netinfo(conn) < 0) { - connection_mark_for_close(TO_CONN(conn)); - return; + if (connection_or_send_netinfo(conn) < 0) { + connection_mark_for_close(TO_CONN(conn)); + return; + } + } else { + const int send_versions = !started_here; + /* If we want to authenticate, send a CERTS cell */ + const int send_certs = !started_here || public_server_mode(get_options()); + /* If we're a relay that got a connection, ask for authentication. */ + const int send_chall = !started_here && public_server_mode(get_options()); + /* If our certs cell will authenticate us, we can send a netinfo cell + * right now. */ + const int send_netinfo = !started_here; + const int send_any = + send_versions || send_certs || send_chall || send_netinfo; + tor_assert(conn->link_proto >= 3); + + log_info(LD_OR, "Negotiated version %d with %s:%d; %s%s%s%s%s", + highest_supported_version, + safe_str_client(conn->_base.address), + conn->_base.port, + send_any ? "Sending cells:" : "Waiting for CERTS cell", + send_versions ? " VERSIONS" : "", + send_certs ? " CERTS" : "", + send_chall ? " AUTH_CHALLENGE" : "", + send_netinfo ? " NETINFO" : ""); + +#ifdef DISABLE_V3_LINKPROTO_SERVERSIDE + if (1) { + connection_mark_for_close(TO_CONN(conn)); + return; + } +#endif + + if (send_versions) { + if (connection_or_send_versions(conn, 1) < 0) { + log_warn(LD_OR, "Couldn't send versions cell"); + connection_mark_for_close(TO_CONN(conn)); + return; + } + } + if (send_certs) { + if (connection_or_send_certs_cell(conn) < 0) { + log_warn(LD_OR, "Couldn't send certs cell"); + connection_mark_for_close(TO_CONN(conn)); + return; + } + } + if (send_chall) { + if (connection_or_send_auth_challenge_cell(conn) < 0) { + log_warn(LD_OR, "Couldn't send auth_challenge cell"); + connection_mark_for_close(TO_CONN(conn)); + return; + } + } + if (send_netinfo) { + if (connection_or_send_netinfo(conn) < 0) { + log_warn(LD_OR, "Couldn't send netinfo cell"); + connection_mark_for_close(TO_CONN(conn)); + return; + } + } } } @@ -583,13 +834,41 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn) conn->link_proto == 0 ? "non-versioned" : "a v1"); return; } - if (conn->_base.state != OR_CONN_STATE_OR_HANDSHAKING) { + if (conn->_base.state != OR_CONN_STATE_OR_HANDSHAKING_V2 && + conn->_base.state != OR_CONN_STATE_OR_HANDSHAKING_V3) { log_fn(LOG_PROTOCOL_WARN, LD_OR, "Received a NETINFO cell on non-handshaking connection; dropping."); return; } tor_assert(conn->handshake_state && conn->handshake_state->received_versions); + + if (conn->_base.state == OR_CONN_STATE_OR_HANDSHAKING_V3) { + tor_assert(conn->link_proto >= 3); + if (conn->handshake_state->started_here) { + if (!conn->handshake_state->authenticated) { + log_fn(LOG_PROTOCOL_WARN, LD_OR, "Got a NETINFO cell from server, " + "but no authentication. Closing the connection."); + connection_mark_for_close(TO_CONN(conn)); + return; + } + } else { + /* we're the server. If the client never authenticated, we have + some housekeeping to do.*/ + if (!conn->handshake_state->authenticated) { + tor_assert(tor_digest_is_zero( + (const char*)conn->handshake_state->authenticated_peer_id)); + connection_or_set_circid_type(conn, NULL); + + connection_or_init_conn_from_address(conn, + &conn->_base.addr, + conn->_base.port, + (const char*)conn->handshake_state->authenticated_peer_id, + 0); + } + } + } + /* Decode the cell. */ timestamp = ntohl(get_uint32(cell->payload)); if (labs(now - conn->handshake_state->sent_versions_at) < 180) { @@ -635,7 +914,7 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn) /** Warn when we get a netinfo skew with at least this value. */ #define NETINFO_NOTICE_SKEW 3600 if (labs(apparent_skew) > NETINFO_NOTICE_SKEW && - router_get_by_digest(conn->identity_digest)) { + router_get_by_id_digest(conn->identity_digest)) { char dbuf[64]; int severity; /*XXXX be smarter about when everybody says we are skewed. */ @@ -662,13 +941,426 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn) * trustworthy. */ (void)my_apparent_addr; - if (connection_or_set_state_open(conn)<0) + if (connection_or_set_state_open(conn)<0) { + log_fn(LOG_PROTOCOL_WARN, LD_OR, "Got good NETINFO cell from %s:%d; but " + "was unable to make the OR connection become open.", + safe_str_client(conn->_base.address), + conn->_base.port); connection_mark_for_close(TO_CONN(conn)); - else + } else { log_info(LD_OR, "Got good NETINFO cell from %s:%d; OR connection is now " - "open, using protocol version %d", + "open, using protocol version %d. Its ID digest is %s", safe_str_client(conn->_base.address), - conn->_base.port, (int)conn->link_proto); + conn->_base.port, (int)conn->link_proto, + hex_str(conn->identity_digest, DIGEST_LEN)); + } assert_connection_ok(TO_CONN(conn),time(NULL)); } +/** Process a CERTS cell from an OR connection. + * + * If the other side should not have sent us a CERTS cell, or the cell is + * malformed, or it is supposed to authenticate the TLS key but it doesn't, + * then mark the connection. + * + * If the cell has a good cert chain and we're doing a v3 handshake, then + * store the certificates in or_handshake_state. If this is the client side + * of the connection, we then authenticate the server or mark the connection. + * If it's the server side, wait for an AUTHENTICATE cell. + */ +static void +command_process_certs_cell(var_cell_t *cell, or_connection_t *conn) +{ +#define ERR(s) \ + do { \ + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, \ + "Received a bad CERTS cell from %s:%d: %s", \ + safe_str(conn->_base.address), conn->_base.port, (s)); \ + connection_mark_for_close(TO_CONN(conn)); \ + goto err; \ + } while (0) + + tor_cert_t *link_cert = NULL; + tor_cert_t *id_cert = NULL; + tor_cert_t *auth_cert = NULL; + + uint8_t *ptr; + int n_certs, i; + int send_netinfo = 0; + + if (conn->_base.state != OR_CONN_STATE_OR_HANDSHAKING_V3) + ERR("We're not doing a v3 handshake!"); + if (conn->link_proto < 3) + ERR("We're not using link protocol >= 3"); + if (conn->handshake_state->received_certs_cell) + ERR("We already got one"); + if (conn->handshake_state->authenticated) { + /* Should be unreachable, but let's make sure. */ + ERR("We're already authenticated!"); + } + if (cell->payload_len < 1) + ERR("It had no body"); + if (cell->circ_id) + ERR("It had a nonzero circuit ID"); + + n_certs = cell->payload[0]; + ptr = cell->payload + 1; + for (i = 0; i < n_certs; ++i) { + uint8_t cert_type; + uint16_t cert_len; + if (ptr + 3 > cell->payload + cell->payload_len) { + goto truncated; + } + cert_type = *ptr; + cert_len = ntohs(get_uint16(ptr+1)); + if (ptr + 3 + cert_len > cell->payload + cell->payload_len) { + goto truncated; + } + if (cert_type == OR_CERT_TYPE_TLS_LINK || + cert_type == OR_CERT_TYPE_ID_1024 || + cert_type == OR_CERT_TYPE_AUTH_1024) { + tor_cert_t *cert = tor_cert_decode(ptr + 3, cert_len); + if (!cert) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Received undecodable certificate in CERTS cell from %s:%d", + safe_str(conn->_base.address), conn->_base.port); + } else { + if (cert_type == OR_CERT_TYPE_TLS_LINK) { + if (link_cert) { + tor_cert_free(cert); + ERR("Too many TLS_LINK certificates"); + } + link_cert = cert; + } else if (cert_type == OR_CERT_TYPE_ID_1024) { + if (id_cert) { + tor_cert_free(cert); + ERR("Too many ID_1024 certificates"); + } + id_cert = cert; + } else if (cert_type == OR_CERT_TYPE_AUTH_1024) { + if (auth_cert) { + tor_cert_free(cert); + ERR("Too many AUTH_1024 certificates"); + } + auth_cert = cert; + } else { + tor_cert_free(cert); + } + } + } + ptr += 3 + cert_len; + continue; + + truncated: + ERR("It ends in the middle of a certificate"); + } + + if (conn->handshake_state->started_here) { + int severity; + if (! (id_cert && link_cert)) + ERR("The certs we wanted were missing"); + /* Okay. We should be able to check the certificates now. */ + if (! tor_tls_cert_matches_key(conn->tls, link_cert)) { + ERR("The link certificate didn't match the TLS public key"); + } + /* Note that this warns more loudly about time and validity if we were + * _trying_ to connect to an authority, not necessarily if we _did_ connect + * to one. */ + if (router_digest_is_trusted_dir(conn->identity_digest)) + severity = LOG_WARN; + else + severity = LOG_PROTOCOL_WARN; + + if (! tor_tls_cert_is_valid(severity, link_cert, id_cert, 0)) + ERR("The link certificate was not valid"); + if (! tor_tls_cert_is_valid(severity, id_cert, id_cert, 1)) + ERR("The ID certificate was not valid"); + + conn->handshake_state->authenticated = 1; + { + const digests_t *id_digests = tor_cert_get_id_digests(id_cert); + crypto_pk_t *identity_rcvd; + if (!id_digests) + ERR("Couldn't compute digests for key in ID cert"); + + identity_rcvd = tor_tls_cert_get_key(id_cert); + if (!identity_rcvd) + ERR("Internal error: Couldn't get RSA key from ID cert."); + memcpy(conn->handshake_state->authenticated_peer_id, + id_digests->d[DIGEST_SHA1], DIGEST_LEN); + connection_or_set_circid_type(conn, identity_rcvd); + crypto_pk_free(identity_rcvd); + } + + if (connection_or_client_learned_peer_id(conn, + conn->handshake_state->authenticated_peer_id) < 0) + ERR("Problem setting or checking peer id"); + + log_info(LD_OR, "Got some good certificates from %s:%d: Authenticated it.", + safe_str(conn->_base.address), conn->_base.port); + + conn->handshake_state->id_cert = id_cert; + id_cert = NULL; + + if (!public_server_mode(get_options())) { + /* If we initiated the connection and we are not a public server, we + * aren't planning to authenticate at all. At this point we know who we + * are talking to, so we can just send a netinfo now. */ + send_netinfo = 1; + } + } else { + if (! (id_cert && auth_cert)) + ERR("The certs we wanted were missing"); + + /* Remember these certificates so we can check an AUTHENTICATE cell */ + if (! tor_tls_cert_is_valid(LOG_PROTOCOL_WARN, auth_cert, id_cert, 1)) + ERR("The authentication certificate was not valid"); + if (! tor_tls_cert_is_valid(LOG_PROTOCOL_WARN, id_cert, id_cert, 1)) + ERR("The ID certificate was not valid"); + + log_info(LD_OR, "Got some good certificates from %s:%d: " + "Waiting for AUTHENTICATE.", + safe_str(conn->_base.address), conn->_base.port); + /* XXXX check more stuff? */ + + conn->handshake_state->id_cert = id_cert; + conn->handshake_state->auth_cert = auth_cert; + id_cert = auth_cert = NULL; + } + + conn->handshake_state->received_certs_cell = 1; + + if (send_netinfo) { + if (connection_or_send_netinfo(conn) < 0) { + log_warn(LD_OR, "Couldn't send netinfo cell"); + connection_mark_for_close(TO_CONN(conn)); + goto err; + } + } + + err: + tor_cert_free(id_cert); + tor_cert_free(link_cert); + tor_cert_free(auth_cert); +#undef ERR +} + +/** Process an AUTH_CHALLENGE cell from an OR connection. + * + * If we weren't supposed to get one (for example, because we're not the + * originator of the connection), or it's ill-formed, or we aren't doing a v3 + * handshake, mark the connection. If the cell is well-formed but we don't + * want to authenticate, just drop it. If the cell is well-formed *and* we + * want to authenticate, send an AUTHENTICATE cell and then a NETINFO cell. */ +static void +command_process_auth_challenge_cell(var_cell_t *cell, or_connection_t *conn) +{ + int n_types, i, use_type = -1; + uint8_t *cp; + +#define ERR(s) \ + do { \ + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, \ + "Received a bad AUTH_CHALLENGE cell from %s:%d: %s", \ + safe_str(conn->_base.address), conn->_base.port, (s)); \ + connection_mark_for_close(TO_CONN(conn)); \ + return; \ + } while (0) + + if (conn->_base.state != OR_CONN_STATE_OR_HANDSHAKING_V3) + ERR("We're not currently doing a v3 handshake"); + if (conn->link_proto < 3) + ERR("We're not using link protocol >= 3"); + if (! conn->handshake_state->started_here) + ERR("We didn't originate this connection"); + if (conn->handshake_state->received_auth_challenge) + ERR("We already received one"); + if (! conn->handshake_state->received_certs_cell) + ERR("We haven't gotten a CERTS cell yet"); + if (cell->payload_len < OR_AUTH_CHALLENGE_LEN + 2) + ERR("It was too short"); + if (cell->circ_id) + ERR("It had a nonzero circuit ID"); + + n_types = ntohs(get_uint16(cell->payload + OR_AUTH_CHALLENGE_LEN)); + if (cell->payload_len < OR_AUTH_CHALLENGE_LEN + 2 + 2*n_types) + ERR("It looks truncated"); + + /* Now see if there is an authentication type we can use */ + cp=cell->payload+OR_AUTH_CHALLENGE_LEN+2; + for (i=0; i < n_types; ++i, cp += 2) { + uint16_t authtype = ntohs(get_uint16(cp)); + if (authtype == AUTHTYPE_RSA_SHA256_TLSSECRET) + use_type = authtype; + } + + conn->handshake_state->received_auth_challenge = 1; + + if (! public_server_mode(get_options())) { + /* If we're not a public server then we don't want to authenticate on a + connection we originated, and we already sent a NETINFO cell when we + got the CERTS cell. We have nothing more to do. */ + return; + } + + if (use_type >= 0) { + log_info(LD_OR, "Got an AUTH_CHALLENGE cell from %s:%d: Sending " + "authentication", + safe_str(conn->_base.address), conn->_base.port); + + if (connection_or_send_authenticate_cell(conn, use_type) < 0) { + log_warn(LD_OR, "Couldn't send authenticate cell"); + connection_mark_for_close(TO_CONN(conn)); + return; + } + } else { + log_info(LD_OR, "Got an AUTH_CHALLENGE cell from %s:%d, but we don't " + "know any of its authentication types. Not authenticating.", + safe_str(conn->_base.address), conn->_base.port); + } + + if (connection_or_send_netinfo(conn) < 0) { + log_warn(LD_OR, "Couldn't send netinfo cell"); + connection_mark_for_close(TO_CONN(conn)); + return; + } + +#undef ERR +} + +/** Process an AUTHENTICATE cell from an OR connection. + * + * If it's ill-formed or we weren't supposed to get one or we're not doing a + * v3 handshake, then mark the connection. If it does not authenticate the + * other side of the connection successfully (because it isn't signed right, + * we didn't get a CERTS cell, etc) mark the connection. Otherwise, accept + * the identity of the router on the other side of the connection. + */ +static void +command_process_authenticate_cell(var_cell_t *cell, or_connection_t *conn) +{ + uint8_t expected[V3_AUTH_FIXED_PART_LEN]; + const uint8_t *auth; + int authlen; + +#define ERR(s) \ + do { \ + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, \ + "Received a bad AUTHENTICATE cell from %s:%d: %s", \ + safe_str(conn->_base.address), conn->_base.port, (s)); \ + connection_mark_for_close(TO_CONN(conn)); \ + return; \ + } while (0) + + if (conn->_base.state != OR_CONN_STATE_OR_HANDSHAKING_V3) + ERR("We're not doing a v3 handshake"); + if (conn->link_proto < 3) + ERR("We're not using link protocol >= 3"); + if (conn->handshake_state->started_here) + ERR("We originated this connection"); + if (conn->handshake_state->received_authenticate) + ERR("We already got one!"); + if (conn->handshake_state->authenticated) { + /* Should be impossible given other checks */ + ERR("The peer is already authenticated"); + } + if (! conn->handshake_state->received_certs_cell) + ERR("We never got a certs cell"); + if (conn->handshake_state->auth_cert == NULL) + ERR("We never got an authentication certificate"); + if (conn->handshake_state->id_cert == NULL) + ERR("We never got an identity certificate"); + if (cell->payload_len < 4) + ERR("Cell was way too short"); + + auth = cell->payload; + { + uint16_t type = ntohs(get_uint16(auth)); + uint16_t len = ntohs(get_uint16(auth+2)); + if (4 + len > cell->payload_len) + ERR("Authenticator was truncated"); + + if (type != AUTHTYPE_RSA_SHA256_TLSSECRET) + ERR("Authenticator type was not recognized"); + + auth += 4; + authlen = len; + } + + if (authlen < V3_AUTH_BODY_LEN + 1) + ERR("Authenticator was too short"); + + if (connection_or_compute_authenticate_cell_body( + conn, expected, sizeof(expected), NULL, 1) < 0) + ERR("Couldn't compute expected AUTHENTICATE cell body"); + + if (tor_memneq(expected, auth, sizeof(expected))) + ERR("Some field in the AUTHENTICATE cell body was not as expected"); + + { + crypto_pk_t *pk = tor_tls_cert_get_key( + conn->handshake_state->auth_cert); + char d[DIGEST256_LEN]; + char *signed_data; + size_t keysize; + int signed_len; + + if (!pk) + ERR("Internal error: couldn't get RSA key from AUTH cert."); + crypto_digest256(d, (char*)auth, V3_AUTH_BODY_LEN, DIGEST_SHA256); + + keysize = crypto_pk_keysize(pk); + signed_data = tor_malloc(keysize); + signed_len = crypto_pk_public_checksig(pk, signed_data, keysize, + (char*)auth + V3_AUTH_BODY_LEN, + authlen - V3_AUTH_BODY_LEN); + crypto_pk_free(pk); + if (signed_len < 0) { + tor_free(signed_data); + ERR("Signature wasn't valid"); + } + if (signed_len < DIGEST256_LEN) { + tor_free(signed_data); + ERR("Not enough data was signed"); + } + /* Note that we deliberately allow *more* than DIGEST256_LEN bytes here, + * in case they're later used to hold a SHA3 digest or something. */ + if (tor_memneq(signed_data, d, DIGEST256_LEN)) { + tor_free(signed_data); + ERR("Signature did not match data to be signed."); + } + tor_free(signed_data); + } + + /* Okay, we are authenticated. */ + conn->handshake_state->received_authenticate = 1; + conn->handshake_state->authenticated = 1; + conn->handshake_state->digest_received_data = 0; + { + crypto_pk_t *identity_rcvd = + tor_tls_cert_get_key(conn->handshake_state->id_cert); + const digests_t *id_digests = + tor_cert_get_id_digests(conn->handshake_state->id_cert); + + /* This must exist; we checked key type when reading the cert. */ + tor_assert(id_digests); + + memcpy(conn->handshake_state->authenticated_peer_id, + id_digests->d[DIGEST_SHA1], DIGEST_LEN); + + connection_or_set_circid_type(conn, identity_rcvd); + crypto_pk_free(identity_rcvd); + + connection_or_init_conn_from_address(conn, + &conn->_base.addr, + conn->_base.port, + (const char*)conn->handshake_state->authenticated_peer_id, + 0); + + log_info(LD_OR, "Got an AUTHENTICATE cell from %s:%d: Looks good.", + safe_str(conn->_base.address), conn->_base.port); + } + +#undef ERR +} + diff --git a/src/or/command.h b/src/or/command.h index 95b0f3a931..078ccc9f5d 100644 --- a/src/or/command.h +++ b/src/or/command.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** diff --git a/src/or/config.c b/src/or/config.c index 090aba1c7a..d5c5689474 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -33,24 +33,34 @@ #include "rendservice.h" #include "rephist.h" #include "router.h" +#include "util.h" #include "routerlist.h" -#ifdef MS_WINDOWS +#include "transports.h" +#ifdef _WIN32 #include <shlobj.h> #endif #include "procmon.h" +/* From main.c */ +extern int quiet_level; + /** Enumeration of types which option values can take */ typedef enum config_type_t { CONFIG_TYPE_STRING = 0, /**< An arbitrary string. */ CONFIG_TYPE_FILENAME, /**< A filename: some prefixes get expanded. */ CONFIG_TYPE_UINT, /**< A non-negative integer less than MAX_INT */ + CONFIG_TYPE_INT, /**< Any integer. */ CONFIG_TYPE_PORT, /**< A port from 1...65535, 0 for "not set", or * "auto". */ CONFIG_TYPE_INTERVAL, /**< A number of seconds, with optional units*/ + CONFIG_TYPE_MSEC_INTERVAL,/**< A number of milliseconds, with optional + * units */ CONFIG_TYPE_MEMUNIT, /**< A number of bytes, with optional units*/ CONFIG_TYPE_DOUBLE, /**< A floating-point value */ CONFIG_TYPE_BOOL, /**< A boolean value, expressed as 0 or 1. */ + CONFIG_TYPE_AUTOBOOL, /**< A boolean+auto value, expressed 0 for false, + * 1 for true, and -1 for auto */ CONFIG_TYPE_ISOTIME, /**< An ISO-formatted time relative to GMT. */ CONFIG_TYPE_CSV, /**< A list of strings, separated by commas and * optional whitespace. */ @@ -80,6 +90,10 @@ typedef struct config_abbrev_t { /** A list of abbreviations and aliases to map command-line options, obsolete * option names, or alternative option names, to their current values. */ static config_abbrev_t _option_abbrevs[] = { + PLURAL(AuthDirBadDirCC), + PLURAL(AuthDirBadExitCC), + PLURAL(AuthDirInvalidCC), + PLURAL(AuthDirRejectCC), PLURAL(ExitNode), PLURAL(EntryNode), PLURAL(ExcludeNode), @@ -154,6 +168,9 @@ typedef struct config_var_t { /** An entry for config_vars: "The option <b>name</b> is obsolete." */ #define OBSOLETE(name) { name, CONFIG_TYPE_OBSOLETE, 0, NULL } +#define VPORT(member,conftype,initvalue) \ + VAR(#member, conftype, member ## _lines, initvalue) + /** Array of configuration options. Until we disallow nonstandard * abbreviations, order is significant, since the first matching option will * be chosen first. @@ -173,11 +190,15 @@ static config_var_t _option_vars[] = { V(AlternateHSAuthority, LINELIST, NULL), V(AssumeReachable, BOOL, "0"), V(AuthDirBadDir, LINELIST, NULL), + V(AuthDirBadDirCCs, CSV, ""), V(AuthDirBadExit, LINELIST, NULL), + V(AuthDirBadExitCCs, CSV, ""), V(AuthDirInvalid, LINELIST, NULL), - V(AuthDirFastGuarantee, MEMUNIT, "20 KB"), + V(AuthDirInvalidCCs, CSV, ""), + V(AuthDirFastGuarantee, MEMUNIT, "100 KB"), V(AuthDirGuardBWGuarantee, MEMUNIT, "250 KB"), V(AuthDirReject, LINELIST, NULL), + V(AuthDirRejectCCs, CSV, ""), V(AuthDirRejectUnlisted, BOOL, "0"), V(AuthDirListBadDirs, BOOL, "0"), V(AuthDirListBadExits, BOOL, "0"), @@ -201,15 +222,17 @@ static config_var_t _option_vars[] = { V(CircuitStreamTimeout, INTERVAL, "0"), V(CircuitPriorityHalflife, DOUBLE, "-100.0"), /*negative:'Use default'*/ V(ClientDNSRejectInternalAddresses, BOOL,"1"), - V(ClientRejectInternalAddresses, BOOL, "1"), V(ClientOnly, BOOL, "0"), + V(ClientRejectInternalAddresses, BOOL, "1"), + V(ClientTransportPlugin, LINELIST, NULL), V(ConsensusParams, STRING, NULL), V(ConnLimit, UINT, "1000"), + V(ConnDirectionStatistics, BOOL, "0"), V(ConstrainedSockets, BOOL, "0"), V(ConstrainedSockSize, MEMUNIT, "8192"), V(ContactInfo, STRING, NULL), V(ControlListenAddress, LINELIST, NULL), - V(ControlPort, PORT, "0"), + VPORT(ControlPort, LINELIST, NULL), V(ControlPortFileGroupReadable,BOOL, "0"), V(ControlPortWriteToFile, FILENAME, NULL), V(ControlSocket, LINELIST, NULL), @@ -217,14 +240,16 @@ static config_var_t _option_vars[] = { V(CookieAuthentication, BOOL, "0"), V(CookieAuthFileGroupReadable, BOOL, "0"), V(CookieAuthFile, STRING, NULL), + V(CountPrivateBandwidth, BOOL, "0"), V(DataDirectory, FILENAME, NULL), OBSOLETE("DebugLogFile"), - V(DirAllowPrivateAddresses, BOOL, NULL), + V(DisableNetwork, BOOL, "0"), + V(DirAllowPrivateAddresses, BOOL, "0"), V(TestingAuthDirTimeToLearnReachability, INTERVAL, "30 minutes"), V(DirListenAddress, LINELIST, NULL), OBSOLETE("DirFetchPeriod"), V(DirPolicy, LINELIST, NULL), - V(DirPort, PORT, "0"), + VPORT(DirPort, LINELIST, NULL), V(DirPortFrontPage, FILENAME, NULL), OBSOLETE("DirPostPeriod"), OBSOLETE("DirRecordUsageByCountry"), @@ -234,7 +259,10 @@ static config_var_t _option_vars[] = { V(DirReqStatistics, BOOL, "1"), VAR("DirServer", LINELIST, DirServers, NULL), V(DisableAllSwap, BOOL, "0"), - V(DNSPort, PORT, "0"), + V(DisableDebuggerAttachment, BOOL, "1"), + V(DisableIOCP, BOOL, "1"), + V(DynamicDHGroups, BOOL, "0"), + VPORT(DNSPort, LINELIST, NULL), V(DNSListenAddress, LINELIST, NULL), V(DownloadExtraInfo, BOOL, "0"), V(EnforceDistinctSubnets, BOOL, "1"), @@ -248,6 +276,7 @@ static config_var_t _option_vars[] = { V(ExitPolicy, LINELIST, NULL), V(ExitPolicyRejectPrivate, BOOL, "1"), V(ExitPortStatistics, BOOL, "0"), + V(ExtendAllowPrivateAddresses, BOOL, "0"), V(ExtraInfoStatistics, BOOL, "1"), #if defined (WINCE) @@ -265,16 +294,16 @@ static config_var_t _option_vars[] = { V(FetchHidServDescriptors, BOOL, "1"), V(FetchUselessDescriptors, BOOL, "0"), V(FetchV2Networkstatus, BOOL, "0"), -#ifdef WIN32 +#ifdef _WIN32 V(GeoIPFile, FILENAME, "<default>"), #else V(GeoIPFile, FILENAME, SHARE_DATADIR PATH_SEPARATOR "tor" PATH_SEPARATOR "geoip"), #endif - V(GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays, - BOOL, "0"), + OBSOLETE("GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays"), OBSOLETE("Group"), V(HardwareAccel, BOOL, "0"), + V(HeartbeatPeriod, INTERVAL, "6 hours"), V(AccelName, STRING, NULL), V(AccelDir, FILENAME, NULL), V(HashedControlPassword, LINELIST, NULL), @@ -289,10 +318,13 @@ static config_var_t _option_vars[] = { V(HidServAuth, LINELIST, NULL), V(HSAuthoritativeDir, BOOL, "0"), OBSOLETE("HSAuthorityRecordStats"), + V(CloseHSClientCircuitsImmediatelyOnTimeout, BOOL, "0"), + V(CloseHSServiceRendCircuitsImmediatelyOnTimeout, BOOL, "0"), V(HTTPProxy, STRING, NULL), V(HTTPProxyAuthenticator, STRING, NULL), V(HTTPSProxy, STRING, NULL), V(HTTPSProxyAuthenticator, STRING, NULL), + VAR("ServerTransportPlugin", LINELIST, ServerTransportPlugin, NULL), V(Socks4Proxy, STRING, NULL), V(Socks5Proxy, STRING, NULL), V(Socks5ProxyUsername, STRING, NULL), @@ -304,32 +336,45 @@ static config_var_t _option_vars[] = { OBSOLETE("LinkPadding"), OBSOLETE("LogLevel"), OBSOLETE("LogFile"), + V(LogTimeGranularity, MSEC_INTERVAL, "1 second"), V(LongLivedPorts, CSV, - "21,22,706,1863,5050,5190,5222,5223,6667,6697,8300"), + "21,22,706,1863,5050,5190,5222,5223,6523,6667,6697,8300"), VAR("MapAddress", LINELIST, AddressMap, NULL), V(MaxAdvertisedBandwidth, MEMUNIT, "1 GB"), V(MaxCircuitDirtiness, INTERVAL, "10 minutes"), + V(MaxClientCircuitsPending, UINT, "32"), + V(MaxMemInCellQueues, MEMUNIT, "8 GB"), V(MaxOnionsPending, UINT, "100"), OBSOLETE("MonthlyAccountingStart"), V(MyFamily, STRING, NULL), V(NewCircuitPeriod, INTERVAL, "30 seconds"), VAR("NamingAuthoritativeDirectory",BOOL, NamingAuthoritativeDir, "0"), V(NATDListenAddress, LINELIST, NULL), - V(NATDPort, PORT, "0"), + VPORT(NATDPort, LINELIST, NULL), V(Nickname, STRING, NULL), V(WarnUnsafeSocks, BOOL, "1"), OBSOLETE("NoPublish"), VAR("NodeFamily", LINELIST, NodeFamilies, NULL), - V(NumCPUs, UINT, "1"), + V(NumCPUs, UINT, "0"), V(NumEntryGuards, UINT, "3"), V(ORListenAddress, LINELIST, NULL), - V(ORPort, PORT, "0"), + VPORT(ORPort, LINELIST, NULL), V(OutboundBindAddress, STRING, NULL), + + V(PathBiasCircThreshold, INT, "-1"), + V(PathBiasNoticeRate, DOUBLE, "-1"), + V(PathBiasDisableRate, DOUBLE, "-1"), + V(PathBiasScaleThreshold, INT, "-1"), + V(PathBiasScaleFactor, INT, "-1"), + OBSOLETE("PathlenCoinWeight"), V(PerConnBWBurst, MEMUNIT, "0"), V(PerConnBWRate, MEMUNIT, "0"), V(PidFile, STRING, NULL), V(TestingTorNetwork, BOOL, "0"), + V(OptimisticData, AUTOBOOL, "auto"), + V(PortForwarding, BOOL, "0"), + V(PortForwardingHelper, FILENAME, "tor-fw-helper"), V(PreferTunneledDirConns, BOOL, "1"), V(ProtocolWarnings, BOOL, "0"), V(PublishServerDescriptor, CSV, "1"), @@ -341,7 +386,7 @@ static config_var_t _option_vars[] = { V(RecommendedClientVersions, LINELIST, NULL), V(RecommendedServerVersions, LINELIST, NULL), OBSOLETE("RedirectExit"), - V(RefuseUnknownExits, STRING, "auto"), + V(RefuseUnknownExits, AUTOBOOL, "auto"), V(RejectPlaintextPorts, CSV, ""), V(RelayBandwidthBurst, MEMUNIT, "0"), V(RelayBandwidthRate, MEMUNIT, "0"), @@ -366,23 +411,27 @@ static config_var_t _option_vars[] = { V(ShutdownWaitLength, INTERVAL, "30 seconds"), V(SocksListenAddress, LINELIST, NULL), V(SocksPolicy, LINELIST, NULL), - V(SocksPort, PORT, "9050"), + VPORT(SocksPort, LINELIST, NULL), V(SocksTimeout, INTERVAL, "2 minutes"), OBSOLETE("StatusFetchPeriod"), V(StrictNodes, BOOL, "0"), OBSOLETE("SysLog"), V(TestSocks, BOOL, "0"), OBSOLETE("TestVia"), + V(TokenBucketRefillInterval, MSEC_INTERVAL, "100 msec"), + V(Tor2webMode, BOOL, "0"), V(TrackHostExits, CSV, NULL), V(TrackHostExitsExpire, INTERVAL, "30 minutes"), OBSOLETE("TrafficShaping"), V(TransListenAddress, LINELIST, NULL), - V(TransPort, PORT, "0"), + VPORT(TransPort, LINELIST, NULL), V(TunnelDirConns, BOOL, "1"), V(UpdateBridgesFromAuthority, BOOL, "0"), V(UseBridges, BOOL, "0"), V(UseEntryGuards, BOOL, "1"), + V(UseMicrodescriptors, AUTOBOOL, "auto"), V(User, STRING, NULL), + V(UserspaceIOCPBuffers, BOOL, "0"), VAR("V1AuthoritativeDirectory",BOOL, V1AuthoritativeDir, "0"), VAR("V2AuthoritativeDirectory",BOOL, V2AuthoritativeDir, "0"), VAR("V3AuthoritativeDirectory",BOOL, V3AuthoritativeDir, "0"), @@ -398,6 +447,7 @@ static config_var_t _option_vars[] = { VAR("VersioningAuthoritativeDirectory",BOOL,VersioningAuthoritativeDir, "0"), V(VirtualAddrNetwork, STRING, "127.192.0.0/10"), V(WarnPlaintextPorts, CSV, "23,109,110,143"), + V(_UseFilteringSSLBufferevents, BOOL, "0"), VAR("__ReloadTorrcOnSIGHUP", BOOL, ReloadTorrcOnSIGHUP, "1"), VAR("__AllDirActionsPrivate", BOOL, AllDirActionsPrivate, "0"), VAR("__DisablePredictedCircuits",BOOL,DisablePredictedCircuits, "0"), @@ -414,8 +464,8 @@ static config_var_t _option_vars[] = { /** Override default values with these if the user sets the TestingTorNetwork * option. */ -static config_var_t testing_tor_network_defaults[] = { - V(ServerDNSAllowBrokenConfig, BOOL, "1"), +static const config_var_t testing_tor_network_defaults[] = { + V(ServerDNSAllowBrokenConfig, BOOL, "1"), V(DirAllowPrivateAddresses, BOOL, "1"), V(EnforceDistinctSubnets, BOOL, "0"), V(AssumeReachable, BOOL, "1"), @@ -423,7 +473,9 @@ static config_var_t testing_tor_network_defaults[] = { V(AuthDirMaxServersPerAuthAddr,UINT, "0"), V(ClientDNSRejectInternalAddresses, BOOL,"0"), V(ClientRejectInternalAddresses, BOOL, "0"), + V(CountPrivateBandwidth, BOOL, "1"), V(ExitPolicyRejectPrivate, BOOL, "0"), + V(ExtendAllowPrivateAddresses, BOOL, "1"), V(V3AuthVotingInterval, INTERVAL, "5 minutes"), V(V3AuthVoteDelay, INTERVAL, "20 seconds"), V(V3AuthDistDelay, INTERVAL, "20 seconds"), @@ -434,6 +486,7 @@ static config_var_t testing_tor_network_defaults[] = { V(TestingEstimatedDescriptorPropagationTime, INTERVAL, "0 minutes"), V(MinUptimeHidServDirectoryV2, INTERVAL, "0 minutes"), V(_UsingTestNetworkDefaults, BOOL, "1"), + { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } }; #undef VAR @@ -444,6 +497,8 @@ static config_var_t testing_tor_network_defaults[] = { /** Array of "state" variables saved to the ~/.tor/state file. */ static config_var_t _state_vars[] = { + /* Remember to document these in state-contents.txt ! */ + V(AccountingBytesReadInInterval, MEMUNIT, NULL), V(AccountingBytesWrittenInInterval, MEMUNIT, NULL), V(AccountingExpectedUsage, MEMUNIT, NULL), @@ -457,8 +512,12 @@ static config_var_t _state_vars[] = { VAR("EntryGuardDownSince", LINELIST_S, EntryGuards, NULL), VAR("EntryGuardUnlistedSince", LINELIST_S, EntryGuards, NULL), VAR("EntryGuardAddedBy", LINELIST_S, EntryGuards, NULL), + VAR("EntryGuardPathBias", LINELIST_S, EntryGuards, NULL), V(EntryGuards, LINELIST_V, NULL), + VAR("TransportProxy", LINELIST_S, TransportProxies, NULL), + V(TransportProxies, LINELIST_V, NULL), + V(BWHistoryReadEnds, ISOTIME, NULL), V(BWHistoryReadInterval, UINT, "900"), V(BWHistoryReadValues, CSV, ""), @@ -485,7 +544,6 @@ static config_var_t _state_vars[] = { V(CircuitBuildAbandonedCount, UINT, "0"), VAR("CircuitBuildTimeBin", LINELIST_S, BuildtimeHistogram, NULL), VAR("BuildtimeHistogram", LINELIST_V, BuildtimeHistogram, NULL), - { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } }; @@ -531,55 +589,66 @@ typedef struct { *(uint32_t*)STRUCT_VAR_P(cfg,fmt->magic_offset)); \ STMT_END -#ifdef MS_WINDOWS +#ifdef _WIN32 static char *get_windows_conf_root(void); #endif static void config_line_append(config_line_t **lst, const char *key, const char *val); -static void option_clear(config_format_t *fmt, or_options_t *options, - config_var_t *var); -static void option_reset(config_format_t *fmt, or_options_t *options, - config_var_t *var, int use_defaults); -static void config_free(config_format_t *fmt, void *options); +static void option_clear(const config_format_t *fmt, or_options_t *options, + const config_var_t *var); +static void option_reset(const config_format_t *fmt, or_options_t *options, + const config_var_t *var, int use_defaults); +static void config_free(const config_format_t *fmt, void *options); static int config_lines_eq(config_line_t *a, config_line_t *b); -static int option_is_same(config_format_t *fmt, - or_options_t *o1, or_options_t *o2, +static int config_count_key(const config_line_t *a, const char *key); +static int option_is_same(const config_format_t *fmt, + const or_options_t *o1, const or_options_t *o2, const char *name); -static or_options_t *options_dup(config_format_t *fmt, or_options_t *old); -static int options_validate(or_options_t *old_options, or_options_t *options, +static or_options_t *options_dup(const config_format_t *fmt, + const or_options_t *old); +static int options_validate(or_options_t *old_options, + or_options_t *options, int from_setconf, char **msg); -static int options_act_reversible(or_options_t *old_options, char **msg); -static int options_act(or_options_t *old_options); -static int options_transition_allowed(or_options_t *old, or_options_t *new, +static int options_act_reversible(const or_options_t *old_options, char **msg); +static int options_act(const or_options_t *old_options); +static int options_transition_allowed(const or_options_t *old, + const or_options_t *new, char **msg); -static int options_transition_affects_workers(or_options_t *old_options, - or_options_t *new_options); -static int options_transition_affects_descriptor(or_options_t *old_options, - or_options_t *new_options); +static int options_transition_affects_workers( + const or_options_t *old_options, const or_options_t *new_options); +static int options_transition_affects_descriptor( + const or_options_t *old_options, const or_options_t *new_options); static int check_nickname_list(const char *lst, const char *name, char **msg); -static void config_register_addressmaps(or_options_t *options); static int parse_bridge_line(const char *line, int validate_only); +static int parse_client_transport_line(const char *line, int validate_only); + +static int parse_server_transport_line(const char *line, int validate_only); static int parse_dir_server_line(const char *line, - authority_type_t required_type, + dirinfo_type_t required_type, int validate_only); +static void port_cfg_free(port_cfg_t *port); +static int parse_ports(or_options_t *options, int validate_only, + char **msg_out, int *n_ports_out); +static int check_server_ports(const smartlist_t *ports, + const or_options_t *options); + static int validate_data_directory(or_options_t *options); -static int write_configuration_file(const char *fname, or_options_t *options); -static config_line_t *get_assigned_option(config_format_t *fmt, - void *options, const char *key, - int escape_val); -static void config_init(config_format_t *fmt, void *options); +static int write_configuration_file(const char *fname, + const or_options_t *options); +static config_line_t *get_assigned_option(const config_format_t *fmt, + const void *options, const char *key, + int escape_val); +static void config_init(const config_format_t *fmt, void *options); static int or_state_validate(or_state_t *old_options, or_state_t *options, int from_setconf, char **msg); static int or_state_load(void); static int options_init_logs(or_options_t *options, int validate_only); -static int is_listening_on_low_port(int port_option, - const config_line_t *listen_options); - static uint64_t config_parse_memunit(const char *s, int *ok); +static int config_parse_msec_interval(const char *s, int *ok); static int config_parse_interval(const char *s, int *ok); -static void init_libevent(void); +static void init_libevent(const or_options_t *options); static int opt_streq(const char *s1, const char *s2); /** Magic value for or_options_t. */ @@ -606,7 +675,7 @@ static config_var_t state_extra_var = { }; /** Configuration format for or_state_t. */ -static config_format_t state_format = { +static const config_format_t state_format = { sizeof(or_state_t), OR_STATE_MAGIC, STRUCT_OFFSET(or_state_t, _magic), @@ -622,14 +691,21 @@ static config_format_t state_format = { /** Command-line and config-file options. */ static or_options_t *global_options = NULL; +/** The fallback options_t object; this is where we look for options not + * in torrc before we fall back to Tor's defaults. */ +static or_options_t *global_default_options = NULL; /** Name of most recently read torrc file. */ static char *torrc_fname = NULL; +/** Name of the most recently read torrc-defaults file.*/ +static char *torrc_defaults_fname; /** Persistent serialized state. */ static or_state_t *global_state = NULL; /** Configuration Options set by command line. */ static config_line_t *global_cmdline_options = NULL; /** Contents of most recently read DirPortFrontPage file. */ static char *global_dirfrontpagecontents = NULL; +/** List of port_cfg_t for all configured ports. */ +static smartlist_t *configured_ports = NULL; /** Return the contents of our frontpage string, or NULL if not configured. */ const char * @@ -640,7 +716,7 @@ get_dirportfrontpage(void) /** Allocate an empty configuration object of a given format type. */ static void * -config_alloc(config_format_t *fmt) +config_alloc(const config_format_t *fmt) { void *opts = tor_malloc_zero(fmt->size); *(uint32_t*)STRUCT_VAR_P(opts, fmt->magic_offset) = fmt->magic; @@ -650,12 +726,19 @@ config_alloc(config_format_t *fmt) /** Return the currently configured options. */ or_options_t * -get_options(void) +get_options_mutable(void) { tor_assert(global_options); return global_options; } +/** Returns the currently configured options */ +const or_options_t * +get_options(void) +{ + return get_options_mutable(); +} + /** Change the current global options to contain <b>new_val</b> instead of * their current value; take action based on the new value; free the old value * as necessary. Returns 0 on success, -1 on failure. @@ -663,6 +746,9 @@ get_options(void) int set_options(or_options_t *new_val, char **msg) { + int i; + smartlist_t *elements; + config_line_t *line; or_options_t *old_options = global_options; global_options = new_val; /* Note that we pass the *old* options below, for comparison. It @@ -677,8 +763,37 @@ set_options(or_options_t *new_val, char **msg) "Acting on config options left us in a broken state. Dying."); exit(1); } + /* Issues a CONF_CHANGED event to notify controller of the change. If Tor is + * just starting up then the old_options will be undefined. */ + if (old_options && old_options != global_options) { + elements = smartlist_new(); + for (i=0; options_format.vars[i].name; ++i) { + const config_var_t *var = &options_format.vars[i]; + const char *var_name = var->name; + if (var->type == CONFIG_TYPE_LINELIST_S || + var->type == CONFIG_TYPE_OBSOLETE) { + continue; + } + if (!option_is_same(&options_format, new_val, old_options, var_name)) { + line = get_assigned_option(&options_format, new_val, var_name, 1); + + if (line) { + for (; line; line = line->next) { + smartlist_add(elements, line->key); + smartlist_add(elements, line->value); + } + } else { + smartlist_add(elements, (char*)options_format.vars[i].name); + smartlist_add(elements, NULL); + } + } + } + control_event_conf_changed(elements); + smartlist_free(elements); + } - config_free(&options_format, old_options); + if (old_options != global_options) + config_free(&options_format, old_options); return 0; } @@ -686,22 +801,39 @@ set_options(or_options_t *new_val, char **msg) extern const char tor_git_revision[]; /* from tor_main.c */ /** The version of this Tor process, as parsed. */ -static char *_version = NULL; +static char *the_tor_version = NULL; +/** A shorter version of this Tor process's version, for export in our router + * descriptor. (Does not include the git version, if any.) */ +static char *the_short_tor_version = NULL; /** Return the current Tor version. */ const char * get_version(void) { - if (_version == NULL) { + if (the_tor_version == NULL) { if (strlen(tor_git_revision)) { - size_t len = strlen(VERSION)+strlen(tor_git_revision)+16; - _version = tor_malloc(len); - tor_snprintf(_version, len, "%s (git-%s)", VERSION, tor_git_revision); + tor_asprintf(&the_tor_version, "%s (git-%s)", get_short_version(), + tor_git_revision); } else { - _version = tor_strdup(VERSION); + the_tor_version = tor_strdup(get_short_version()); } } - return _version; + return the_tor_version; +} + +/** Return the current Tor version, without any git tag. */ +const char * +get_short_version(void) +{ + + if (the_short_tor_version == NULL) { +#ifdef TOR_BUILD_TAG + tor_asprintf(&the_short_tor_version, "%s (%s)", VERSION, TOR_BUILD_TAG); +#else + the_short_tor_version = tor_strdup(VERSION); +#endif + } + return the_short_tor_version; } /** Release additional memory allocated in options @@ -713,6 +845,11 @@ or_options_free(or_options_t *options) return; routerset_free(options->_ExcludeExitNodesUnion); + if (options->NodeFamilySets) { + SMARTLIST_FOREACH(options->NodeFamilySets, routerset_t *, + rs, routerset_free(rs)); + smartlist_free(options->NodeFamilySets); + } tor_free(options->_BridgePassword_AuthDigest); config_free(&options_format, options); } @@ -724,6 +861,8 @@ config_free_all(void) { or_options_free(global_options); global_options = NULL; + or_options_free(global_default_options); + global_default_options = NULL; config_free(&state_format, global_state); global_state = NULL; @@ -731,8 +870,16 @@ config_free_all(void) config_free_lines(global_cmdline_options); global_cmdline_options = NULL; + if (configured_ports) { + SMARTLIST_FOREACH(configured_ports, + port_cfg_t *, p, tor_free(p)); + smartlist_free(configured_ports); + configured_ports = NULL; + } + tor_free(torrc_fname); - tor_free(_version); + tor_free(torrc_defaults_fname); + tor_free(the_tor_version); tor_free(global_dirfrontpagecontents); } @@ -796,7 +943,7 @@ escaped_safe_str(const char *address) /** Add the default directory authorities directly into the trusted dir list, * but only add them insofar as they share bits with <b>type</b>. */ static void -add_default_trusted_dir_authorities(authority_type_t type) +add_default_trusted_dir_authorities(dirinfo_type_t type) { int i; const char *dirservers[] = { @@ -823,6 +970,9 @@ add_default_trusted_dir_authorities(authority_type_t type) "maatuska orport=80 no-v2 " "v3ident=49015F787433103580E3B66A1707A00E60F2D15B " "171.25.193.9:443 BD6A 8292 55CB 08E6 6FBE 7D37 4836 3586 E46B 3810", + "Faravahar orport=443 no-v2 " + "v3ident=EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97 " + "154.35.32.5:80 CF6D 0AAF B385 BE71 B8E1 11FC 5CFF 4B47 9237 33BC", NULL }; for (i=0; dirservers[i]; i++) { @@ -871,16 +1021,16 @@ validate_dir_authorities(or_options_t *options, or_options_t *old_options) /* Now go through the four ways you can configure an alternate * set of directory authorities, and make sure none are broken. */ for (cl = options->DirServers; cl; cl = cl->next) - if (parse_dir_server_line(cl->value, NO_AUTHORITY, 1)<0) + if (parse_dir_server_line(cl->value, NO_DIRINFO, 1)<0) return -1; for (cl = options->AlternateBridgeAuthority; cl; cl = cl->next) - if (parse_dir_server_line(cl->value, NO_AUTHORITY, 1)<0) + if (parse_dir_server_line(cl->value, NO_DIRINFO, 1)<0) return -1; for (cl = options->AlternateDirAuthority; cl; cl = cl->next) - if (parse_dir_server_line(cl->value, NO_AUTHORITY, 1)<0) + if (parse_dir_server_line(cl->value, NO_DIRINFO, 1)<0) return -1; for (cl = options->AlternateHSAuthority; cl; cl = cl->next) - if (parse_dir_server_line(cl->value, NO_AUTHORITY, 1)<0) + if (parse_dir_server_line(cl->value, NO_DIRINFO, 1)<0) return -1; return 0; } @@ -889,8 +1039,8 @@ validate_dir_authorities(or_options_t *options, or_options_t *old_options) * as appropriate. */ static int -consider_adding_dir_authorities(or_options_t *options, - or_options_t *old_options) +consider_adding_dir_authorities(const or_options_t *options, + const or_options_t *old_options) { config_line_t *cl; int need_to_update = @@ -911,27 +1061,28 @@ consider_adding_dir_authorities(or_options_t *options, if (!options->DirServers) { /* then we may want some of the defaults */ - authority_type_t type = NO_AUTHORITY; + dirinfo_type_t type = NO_DIRINFO; if (!options->AlternateBridgeAuthority) - type |= BRIDGE_AUTHORITY; + type |= BRIDGE_DIRINFO; if (!options->AlternateDirAuthority) - type |= V1_AUTHORITY | V2_AUTHORITY | V3_AUTHORITY; + type |= V1_DIRINFO | V2_DIRINFO | V3_DIRINFO | EXTRAINFO_DIRINFO | + MICRODESC_DIRINFO; if (!options->AlternateHSAuthority) - type |= HIDSERV_AUTHORITY; + type |= HIDSERV_DIRINFO; add_default_trusted_dir_authorities(type); } for (cl = options->DirServers; cl; cl = cl->next) - if (parse_dir_server_line(cl->value, NO_AUTHORITY, 0)<0) + if (parse_dir_server_line(cl->value, NO_DIRINFO, 0)<0) return -1; for (cl = options->AlternateBridgeAuthority; cl; cl = cl->next) - if (parse_dir_server_line(cl->value, NO_AUTHORITY, 0)<0) + if (parse_dir_server_line(cl->value, NO_DIRINFO, 0)<0) return -1; for (cl = options->AlternateDirAuthority; cl; cl = cl->next) - if (parse_dir_server_line(cl->value, NO_AUTHORITY, 0)<0) + if (parse_dir_server_line(cl->value, NO_DIRINFO, 0)<0) return -1; for (cl = options->AlternateHSAuthority; cl; cl = cl->next) - if (parse_dir_server_line(cl->value, NO_AUTHORITY, 0)<0) + if (parse_dir_server_line(cl->value, NO_DIRINFO, 0)<0) return -1; return 0; } @@ -943,12 +1094,12 @@ consider_adding_dir_authorities(or_options_t *options, * Return 0 if all goes well, return -1 if things went badly. */ static int -options_act_reversible(or_options_t *old_options, char **msg) +options_act_reversible(const or_options_t *old_options, char **msg) { - smartlist_t *new_listeners = smartlist_create(); - smartlist_t *replaced_listeners = smartlist_create(); + smartlist_t *new_listeners = smartlist_new(); + smartlist_t *replaced_listeners = smartlist_new(); static int libevent_initialized = 0; - or_options_t *options = get_options(); + or_options_t *options = get_options_mutable(); int running_tor = options->command == CMD_RUN_TOR; int set_conn_limit = 0; int r = -1; @@ -977,6 +1128,7 @@ options_act_reversible(or_options_t *old_options, char **msg) #endif if (running_tor) { + int n_ports=0; /* We need to set the connection limit before we can open the listeners. */ if (set_max_file_descriptors((unsigned)options->ConnLimit, &options->_ConnLimit) < 0) { @@ -988,23 +1140,43 @@ options_act_reversible(or_options_t *old_options, char **msg) /* Set up libevent. (We need to do this before we can register the * listeners as listeners.) */ if (running_tor && !libevent_initialized) { - init_libevent(); + init_libevent(options); libevent_initialized = 1; } + /* Adjust the port configuration so we can launch listeners. */ + if (parse_ports(options, 0, msg, &n_ports)) { + if (!*msg) + *msg = tor_strdup("Unexpected problem parsing port config"); + goto rollback; + } + + /* Set the hibernation state appropriately.*/ + consider_hibernation(time(NULL)); + /* Launch the listeners. (We do this before we setuid, so we can bind to - * ports under 1024.) We don't want to rebind if we're hibernating. */ + * ports under 1024.) We don't want to rebind if we're hibernating. If + * networking is disabled, this will close all but the control listeners, + * but disable those. */ if (!we_are_hibernating()) { - if (retry_all_listeners(replaced_listeners, new_listeners) < 0) { + if (retry_all_listeners(replaced_listeners, new_listeners, + options->DisableNetwork) < 0) { *msg = tor_strdup("Failed to bind one of the listener ports."); goto rollback; } } + if (options->DisableNetwork) { + /* Aggressively close non-controller stuff, NOW */ + log_notice(LD_NET, "DisableNetwork is set. Tor will not make or accept " + "non-control network connections. Shutting down all existing " + "connections."); + connection_mark_all_noncontrol_connections(); + } } #if defined(HAVE_NET_IF_H) && defined(HAVE_NET_PFVAR_H) /* Open /dev/pf before dropping privileges. */ - if (options->TransPort) { + if (options->TransPort_set) { if (get_pf_socket() < 0) { *msg = tor_strdup("Unable to open /dev/pf for transparent proxy."); goto rollback; @@ -1044,10 +1216,9 @@ options_act_reversible(or_options_t *old_options, char **msg) /* Write control ports to disk as appropriate */ control_ports_write_to_file(); - if (directory_caches_v2_dir_info(options)) { - size_t len = strlen(options->DataDirectory)+32; - char *fn = tor_malloc(len); - tor_snprintf(fn, len, "%s"PATH_SEPARATOR"cached-status", + if (directory_caches_v2_dir_info(options)) { + char *fn = NULL; + tor_asprintf(&fn, "%s"PATH_SEPARATOR"cached-status", options->DataDirectory); if (check_private_dir(fn, running_tor ? CPD_CREATE : CPD_CHECK, options->User) < 0) { @@ -1105,7 +1276,7 @@ options_act_reversible(or_options_t *old_options, char **msg) SMARTLIST_FOREACH(new_listeners, connection_t *, conn, { - log_notice(LD_NET, "Closing partially-constructed listener %s on %s:%d", + log_notice(LD_NET, "Closing partially-constructed %s on %s:%d", conn_type_to_string(conn->type), conn->address, conn->port); connection_close_immediate(conn); connection_mark_for_close(conn); @@ -1120,7 +1291,7 @@ options_act_reversible(or_options_t *old_options, char **msg) /** If we need to have a GEOIP ip-to-country map to run with our configured * options, return 1 and set *<b>reason_out</b> to a description of why. */ int -options_need_geoip_info(or_options_t *options, const char **reason_out) +options_need_geoip_info(const or_options_t *options, const char **reason_out) { int bridge_usage = options->BridgeRelay && options->BridgeRecordUsageByCountry; @@ -1145,7 +1316,7 @@ options_need_geoip_info(or_options_t *options, const char **reason_out) /** Return the bandwidthrate that we are going to report to the authorities * based on the config options. */ uint32_t -get_effective_bwrate(or_options_t *options) +get_effective_bwrate(const or_options_t *options) { uint64_t bw = options->BandwidthRate; if (bw > options->MaxAdvertisedBandwidth) @@ -1159,7 +1330,7 @@ get_effective_bwrate(or_options_t *options) /** Return the bandwidthburst that we are going to report to the authorities * based on the config options. */ uint32_t -get_effective_bwburst(or_options_t *options) +get_effective_bwburst(const or_options_t *options) { uint64_t bw = options->BandwidthBurst; if (options->RelayBandwidthBurst > 0 && bw > options->RelayBandwidthBurst) @@ -1168,6 +1339,24 @@ get_effective_bwburst(or_options_t *options) return (uint32_t)bw; } +/** Return True if any changes from <b>old_options</b> to + * <b>new_options</b> needs us to refresh our TLS context. */ +static int +options_transition_requires_fresh_tls_context(const or_options_t *old_options, + const or_options_t *new_options) +{ + tor_assert(new_options); + + if (!old_options) + return 0; + + if ((old_options->DynamicDHGroups != new_options->DynamicDHGroups)) { + return 1; + } + + return 0; +} + /** Fetch the active option list, and take actions based on it. All of the * things we do should survive being done repeatedly. If present, * <b>old_options</b> contains the previous value of the options. @@ -1178,15 +1367,39 @@ get_effective_bwburst(or_options_t *options) * here yet. Some is still in do_hup() and other places. */ static int -options_act(or_options_t *old_options) +options_act(const or_options_t *old_options) { config_line_t *cl; - or_options_t *options = get_options(); + or_options_t *options = get_options_mutable(); int running_tor = options->command == CMD_RUN_TOR; char *msg; const int transition_affects_workers = old_options && options_transition_affects_workers(old_options, options); + /* disable ptrace and later, other basic debugging techniques */ + { + /* Remember if we already disabled debugger attachment */ + static int disabled_debugger_attach = 0; + /* Remember if we already warned about being configured not to disable + * debugger attachment */ + static int warned_debugger_attach = 0; + /* Don't disable debugger attachment when we're running the unit tests. */ + if (options->DisableDebuggerAttachment && !disabled_debugger_attach && + running_tor) { + int ok = tor_disable_debugger_attach(); + if (warned_debugger_attach && ok == 1) { + log_notice(LD_CONFIG, "Disabled attaching debuggers for unprivileged " + "users."); + } + disabled_debugger_attach = (ok == 1); + } else if (!options->DisableDebuggerAttachment && + !warned_debugger_attach) { + log_notice(LD_CONFIG, "Not disabling debugger attaching for " + "unprivileged users."); + warned_debugger_attach = 1; + } + } + if (running_tor && !have_lockfile()) { if (try_locking(options, 1) < 0) return -1; @@ -1195,6 +1408,28 @@ options_act(or_options_t *old_options) if (consider_adding_dir_authorities(options, old_options) < 0) return -1; +#ifdef NON_ANONYMOUS_MODE_ENABLED + log(LOG_WARN, LD_GENERAL, "This copy of Tor was compiled to run in a " + "non-anonymous mode. It will provide NO ANONYMITY."); +#endif + +#ifdef ENABLE_TOR2WEB_MODE + if (!options->Tor2webMode) { + log_err(LD_CONFIG, "This copy of Tor was compiled to run in " + "'tor2web mode'. It can only be run with the Tor2webMode torrc " + "option enabled."); + return -1; + } +#else + if (options->Tor2webMode) { + log_err(LD_CONFIG, "This copy of Tor was not compiled to run in " + "'tor2web mode'. It cannot be run with the Tor2webMode torrc " + "option enabled. To enable Tor2webMode recompile with the " + "--enable-tor2webmode option."); + return -1; + } +#endif + if (options->Bridges) { mark_bridge_list(); for (cl = options->Bridges; cl; cl = cl->next) { @@ -1226,6 +1461,32 @@ options_act(or_options_t *old_options) rep_hist_load_mtbf_data(time(NULL)); } + mark_transport_list(); + pt_prepare_proxy_list_for_config_read(); + if (options->ClientTransportPlugin) { + for (cl = options->ClientTransportPlugin; cl; cl = cl->next) { + if (parse_client_transport_line(cl->value, 0)<0) { + log_warn(LD_BUG, + "Previously validated ClientTransportPlugin line " + "could not be added!"); + return -1; + } + } + } + + if (options->ServerTransportPlugin && server_mode(options)) { + for (cl = options->ServerTransportPlugin; cl; cl = cl->next) { + if (parse_server_transport_line(cl->value, 0)<0) { + log_warn(LD_BUG, + "Previously validated ServerTransportPlugin line " + "could not be added!"); + return -1; + } + } + } + sweep_transport_list(); + sweep_proxy_list(); + /* Bail out at this point if we're not going to be a client or server: * we want to not fork, and to log stuff to stderr. */ if (!running_tor) @@ -1237,6 +1498,24 @@ options_act(or_options_t *old_options) finish_daemon(options->DataDirectory); } + /* If needed, generate a new TLS DH prime according to the current torrc. */ + if (server_mode(options) && options->DynamicDHGroups) { + char *keydir = get_datadir_fname("keys"); + if (check_private_dir(keydir, CPD_CREATE, options->User)) { + tor_free(keydir); + return -1; + } + tor_free(keydir); + + if (!old_options || !old_options->DynamicDHGroups) { + char *fname = get_datadir_fname2("keys", "dynamic_dh_params"); + crypto_set_tls_dh_prime(fname); + tor_free(fname); + } + } else { /* clients don't need a dynamic DH prime. */ + crypto_set_tls_dh_prime(NULL); + } + /* We want to reinit keys as needed before we do much of anything else: keys are important, and other things can depend on them. */ if (transition_affects_workers || @@ -1246,6 +1525,13 @@ options_act(or_options_t *old_options) log_warn(LD_BUG,"Error initializing keys; exiting"); return -1; } + } else if (old_options && + options_transition_requires_fresh_tls_context(old_options, + options)) { + if (router_initialize_tls_context() < 0) { + log_warn(LD_BUG,"Error initializing TLS context."); + return -1; + } } /* Write our PID to the PID file. If we do not have write permissions we @@ -1285,17 +1571,16 @@ options_act(or_options_t *old_options) if (accounting_is_enabled(options)) configure_accounting(time(NULL)); - /* parse RefuseUnknownExits tristate */ - if (!strcmp(options->RefuseUnknownExits, "0")) - options->RefuseUnknownExits_ = 0; - else if (!strcmp(options->RefuseUnknownExits, "1")) - options->RefuseUnknownExits_ = 1; - else if (!strcmp(options->RefuseUnknownExits, "auto")) - options->RefuseUnknownExits_ = -1; - else { - /* Should have caught this in options_validate */ - return -1; - } +#ifdef USE_BUFFEREVENTS + /* If we're using the bufferevents implementation and our rate limits + * changed, we need to tell the rate-limiting system about it. */ + if (!old_options || + old_options->BandwidthRate != options->BandwidthRate || + old_options->BandwidthBurst != options->BandwidthBurst || + old_options->RelayBandwidthRate != options->RelayBandwidthRate || + old_options->RelayBandwidthBurst != options->RelayBandwidthBurst) + connection_bucket_init(); +#endif /* Change the cell EWMA settings */ cell_ewma_set_scale_factor(options, networkstatus_get_latest_consensus()); @@ -1374,7 +1659,7 @@ options_act(or_options_t *old_options) int was_relay = 0; if (options->BridgeRelay) { time_t int_start = time(NULL); - if (old_options->ORPort == options->ORPort) { + if (config_lines_eq(old_options->ORPort_lines,options->ORPort_lines)) { int_start += RELAY_BRIDGE_STATS_DELAY; was_relay = 1; } @@ -1417,15 +1702,13 @@ options_act(or_options_t *old_options) || !geoip_is_loaded())) { /* XXXX Don't use this "<default>" junk; make our filename options * understand prefixes somehow. -NM */ - /* XXXX023 Reload GeoIPFile on SIGHUP. -NM */ + /* XXXX024 Reload GeoIPFile on SIGHUP. -NM */ char *actual_fname = tor_strdup(options->GeoIPFile); -#ifdef WIN32 +#ifdef _WIN32 if (!strcmp(actual_fname, "<default>")) { const char *conf_root = get_windows_conf_root(); - size_t len = strlen(conf_root)+16; tor_free(actual_fname); - actual_fname = tor_malloc(len+1); - tor_snprintf(actual_fname, len, "%s\\geoip", conf_root); + tor_asprintf(&actual_fname, "%s\\geoip", conf_root); } #endif geoip_load_file(actual_fname, options); @@ -1433,7 +1716,9 @@ options_act(or_options_t *old_options) } if (options->CellStatistics || options->DirReqStatistics || - options->EntryStatistics || options->ExitPortStatistics) { + options->EntryStatistics || options->ExitPortStatistics || + options->ConnDirectionStatistics || + options->BridgeAuthoritativeDir) { time_t now = time(NULL); int print_notice = 0; @@ -1457,10 +1742,12 @@ options_act(or_options_t *old_options) print_notice = 1; } else { options->DirReqStatistics = 0; - log_notice(LD_CONFIG, "Configured to measure directory request " - "statistics, but no GeoIP database found! " - "Please specify a GeoIP database using the " - "GeoIPFile option!"); + /* Don't warn Tor clients, they don't use statistics */ + if (options->ORPort_set) + log_notice(LD_CONFIG, "Configured to measure directory request " + "statistics, but no GeoIP database found. " + "Please specify a GeoIP database using the " + "GeoIPFile option."); } } if ((!old_options || !old_options->EntryStatistics) && @@ -1471,9 +1758,9 @@ options_act(or_options_t *old_options) } else { options->EntryStatistics = 0; log_notice(LD_CONFIG, "Configured to measure entry node " - "statistics, but no GeoIP database found! " + "statistics, but no GeoIP database found. " "Please specify a GeoIP database using the " - "GeoIPFile option!"); + "GeoIPFile option."); } } if ((!old_options || !old_options->ExitPortStatistics) && @@ -1481,6 +1768,15 @@ options_act(or_options_t *old_options) rep_hist_exit_stats_init(now); print_notice = 1; } + if ((!old_options || !old_options->ConnDirectionStatistics) && + options->ConnDirectionStatistics) { + rep_hist_conn_stats_init(now); + } + if ((!old_options || !old_options->BridgeAuthoritativeDir) && + options->BridgeAuthoritativeDir) { + rep_hist_desc_stats_init(now); + print_notice = 1; + } if (print_notice) log_notice(LD_CONFIG, "Configured to measure statistics. Look for " "the *-stats files that will first be written to the " @@ -1499,6 +1795,12 @@ options_act(or_options_t *old_options) if (old_options && old_options->ExitPortStatistics && !options->ExitPortStatistics) rep_hist_exit_stats_term(); + if (old_options && old_options->ConnDirectionStatistics && + !options->ConnDirectionStatistics) + rep_hist_conn_stats_term(); + if (old_options && old_options->BridgeAuthoritativeDir && + !options->BridgeAuthoritativeDir) + rep_hist_desc_stats_term(); /* Check if we need to parse and add the EntryNodes config option. */ if (options->EntryNodes && @@ -1556,7 +1858,7 @@ options_act(or_options_t *old_options) * apply abbreviations that work for the config file and the command line. * If <b>warn_obsolete</b> is set, warn about deprecated names. */ static const char * -expand_abbrev(config_format_t *fmt, const char *option, int command_line, +expand_abbrev(const config_format_t *fmt, const char *option, int command_line, int warn_obsolete) { int i; @@ -1594,7 +1896,11 @@ config_get_commandlines(int argc, char **argv, config_line_t **result) int i = 1; while (i < argc) { + unsigned command = CONFIG_LINE_NORMAL; + int want_arg = 1; + if (!strcmp(argv[i],"-f") || + !strcmp(argv[i],"--defaults-torrc") || !strcmp(argv[i],"--hash-password")) { i += 2; /* command-line option with argument. ignore them. */ continue; @@ -1611,13 +1917,6 @@ config_get_commandlines(int argc, char **argv, config_line_t **result) continue; } - if (i == argc-1) { - log_warn(LD_CONFIG,"Command-line option '%s' with no value. Failing.", - argv[i]); - config_free_lines(front); - return -1; - } - *new = tor_malloc_zero(sizeof(config_line_t)); s = argv[i]; @@ -1626,15 +1925,33 @@ config_get_commandlines(int argc, char **argv, config_line_t **result) s++; if (*s == '-') s++; + /* Figure out the command, if any. */ + if (*s == '+') { + s++; + command = CONFIG_LINE_APPEND; + } else if (*s == '/') { + s++; + command = CONFIG_LINE_CLEAR; + /* A 'clear' command has no argument. */ + want_arg = 0; + } + + if (want_arg && i == argc-1) { + log_warn(LD_CONFIG,"Command-line option '%s' with no value. Failing.", + argv[i]); + config_free_lines(front); + return -1; + } (*new)->key = tor_strdup(expand_abbrev(&options_format, s, 1, 1)); - (*new)->value = tor_strdup(argv[i+1]); + (*new)->value = want_arg ? tor_strdup(argv[i+1]) : tor_strdup(""); + (*new)->command = command; (*new)->next = NULL; log(LOG_DEBUG, LD_CONFIG, "command line: parsed keyword '%s', value '%s'", (*new)->key, (*new)->value); new = &((*new)->next); - i += 2; + i += want_arg ? 2 : 1; } *result = front; return 0; @@ -1649,7 +1966,7 @@ config_line_append(config_line_t **lst, { config_line_t *newline; - newline = tor_malloc(sizeof(config_line_t)); + newline = tor_malloc_zero(sizeof(config_line_t)); newline->key = tor_strdup(key); newline->value = tor_strdup(val); newline->next = NULL; @@ -1662,9 +1979,12 @@ config_line_append(config_line_t **lst, /** Helper: parse the config string and strdup into key/value * strings. Set *result to the list, or NULL if parsing the string * failed. Return 0 on success, -1 on failure. Warn and ignore any - * misformatted lines. */ + * misformatted lines. + * + * If <b>extended</b> is set, then treat keys beginning with / and with + as + * indicating "clear" and "append" respectively. */ int -config_get_lines(const char *string, config_line_t **result) +config_get_lines(const char *string, config_line_t **result, int extended) { config_line_t *list = NULL, **next; char *k, *v; @@ -1680,13 +2000,30 @@ config_get_lines(const char *string, config_line_t **result) return -1; } if (k && v) { + unsigned command = CONFIG_LINE_NORMAL; + if (extended) { + if (k[0] == '+') { + char *k_new = tor_strdup(k+1); + tor_free(k); + k = k_new; + command = CONFIG_LINE_APPEND; + } else if (k[0] == '/') { + char *k_new = tor_strdup(k+1); + tor_free(k); + k = k_new; + tor_free(v); + v = tor_strdup(""); + command = CONFIG_LINE_CLEAR; + } + } /* This list can get long, so we keep a pointer to the end of it * rather than using config_line_append over and over and getting * n^2 performance. */ - *next = tor_malloc(sizeof(config_line_t)); + *next = tor_malloc_zero(sizeof(config_line_t)); (*next)->key = k; (*next)->value = v; (*next)->next = NULL; + (*next)->command = command; next = &((*next)->next); } else { tor_free(k); @@ -1716,12 +2053,9 @@ config_free_lines(config_line_t *front) } } -/** If <b>key</b> is a configuration option, return the corresponding - * config_var_t. Otherwise, if <b>key</b> is a non-standard abbreviation, - * warn, and return the corresponding config_var_t. Otherwise return NULL. - */ +/** As config_find_option, but return a non-const pointer. */ static config_var_t * -config_find_option(config_format_t *fmt, const char *key) +config_find_option_mutable(config_format_t *fmt, const char *key) { int i; size_t keylen = strlen(key); @@ -1746,9 +2080,20 @@ config_find_option(config_format_t *fmt, const char *key) return NULL; } +/** If <b>key</b> is a configuration option, return the corresponding const + * config_var_t. Otherwise, if <b>key</b> is a non-standard abbreviation, + * warn, and return the corresponding const config_var_t. Otherwise return + * NULL. + */ +static const config_var_t * +config_find_option(const config_format_t *fmt, const char *key) +{ + return config_find_option_mutable((config_format_t*)fmt, key); +} + /** Return the number of option entries in <b>fmt</b>. */ static int -config_count_options(config_format_t *fmt) +config_count_options(const config_format_t *fmt) { int i; for (i=0; fmt->vars[i].name; ++i) @@ -1766,11 +2111,11 @@ config_count_options(config_format_t *fmt) * Called from config_assign_line() and option_reset(). */ static int -config_assign_value(config_format_t *fmt, or_options_t *options, +config_assign_value(const config_format_t *fmt, or_options_t *options, config_line_t *c, char **msg) { int i, ok; - config_var_t *var; + const config_var_t *var; void *lvalue; CHECK(fmt, options); @@ -1788,8 +2133,10 @@ config_assign_value(config_format_t *fmt, or_options_t *options, break; } /* fall through */ + case CONFIG_TYPE_INT: case CONFIG_TYPE_UINT: - i = (int)tor_parse_long(c->value, 10, 0, + i = (int)tor_parse_long(c->value, 10, + var->type==CONFIG_TYPE_INT ? INT_MIN : 0, var->type==CONFIG_TYPE_PORT ? 65535 : INT_MAX, &ok, NULL); if (!ok) { @@ -1813,6 +2160,18 @@ config_assign_value(config_format_t *fmt, or_options_t *options, break; } + case CONFIG_TYPE_MSEC_INTERVAL: { + i = config_parse_msec_interval(c->value, &ok); + if (!ok) { + tor_asprintf(msg, + "Msec interval '%s %s' is malformed or out of bounds.", + c->key, c->value); + return -1; + } + *(int *)lvalue = i; + break; + } + case CONFIG_TYPE_MEMUNIT: { uint64_t u64 = config_parse_memunit(c->value, &ok); if (!ok) { @@ -1836,6 +2195,20 @@ config_assign_value(config_format_t *fmt, or_options_t *options, *(int *)lvalue = i; break; + case CONFIG_TYPE_AUTOBOOL: + if (!strcmp(c->value, "auto")) + *(int *)lvalue = -1; + else if (!strcmp(c->value, "0")) + *(int *)lvalue = 0; + else if (!strcmp(c->value, "1")) + *(int *)lvalue = 1; + else { + tor_asprintf(msg, "Boolean '%s %s' expects 0, 1, or 'auto'.", + c->key, c->value); + return -1; + } + break; + case CONFIG_TYPE_STRING: case CONFIG_TYPE_FILENAME: tor_free(*(char **)lvalue); @@ -1871,7 +2244,7 @@ config_assign_value(config_format_t *fmt, or_options_t *options, SMARTLIST_FOREACH(*(smartlist_t**)lvalue, char *, cp, tor_free(cp)); smartlist_clear(*(smartlist_t**)lvalue); } else { - *(smartlist_t**)lvalue = smartlist_create(); + *(smartlist_t**)lvalue = smartlist_new(); } smartlist_split_string(*(smartlist_t**)lvalue, c->value, ",", @@ -1880,7 +2253,19 @@ config_assign_value(config_format_t *fmt, or_options_t *options, case CONFIG_TYPE_LINELIST: case CONFIG_TYPE_LINELIST_S: - config_line_append((config_line_t**)lvalue, c->key, c->value); + { + config_line_t *lastval = *(config_line_t**)lvalue; + if (lastval && lastval->fragile) { + if (c->command != CONFIG_LINE_APPEND) { + config_free_lines(lastval); + *(config_line_t**)lvalue = NULL; + } else { + lastval->fragile = 0; + } + } + + config_line_append((config_line_t**)lvalue, c->key, c->value); + } break; case CONFIG_TYPE_OBSOLETE: log_warn(LD_CONFIG, "Skipping obsolete configuration option '%s'", c->key); @@ -1896,6 +2281,28 @@ config_assign_value(config_format_t *fmt, or_options_t *options, return 0; } +/** Mark every linelist in <b>options</b> "fragile", so that fresh assignments + * to it will replace old ones. */ +static void +config_mark_lists_fragile(const config_format_t *fmt, or_options_t *options) +{ + int i; + tor_assert(fmt); + tor_assert(options); + + for (i = 0; fmt->vars[i].name; ++i) { + const config_var_t *var = &fmt->vars[i]; + config_line_t *list; + if (var->type != CONFIG_TYPE_LINELIST && + var->type != CONFIG_TYPE_LINELIST_V) + continue; + + list = *(config_line_t **)STRUCT_VAR_P(options, var->var_offset); + if (list) + list->fragile = 1; + } +} + /** If <b>c</b> is a syntactically valid configuration line, update * <b>options</b> with its value and return 0. Otherwise return -1 for bad * key, -2 for bad value. @@ -1906,11 +2313,11 @@ config_assign_value(config_format_t *fmt, or_options_t *options, * Called from config_assign(). */ static int -config_assign_line(config_format_t *fmt, or_options_t *options, +config_assign_line(const config_format_t *fmt, or_options_t *options, config_line_t *c, int use_defaults, int clear_first, bitarray_t *options_seen, char **msg) { - config_var_t *var; + const config_var_t *var; CHECK(fmt, options); @@ -1938,8 +2345,9 @@ config_assign_line(config_format_t *fmt, or_options_t *options, if (!strlen(c->value)) { /* reset or clear it, then return */ if (!clear_first) { - if (var->type == CONFIG_TYPE_LINELIST || - var->type == CONFIG_TYPE_LINELIST_S) { + if ((var->type == CONFIG_TYPE_LINELIST || + var->type == CONFIG_TYPE_LINELIST_S) && + c->command != CONFIG_LINE_CLEAR) { /* We got an empty linelist from the torrc or command line. As a special case, call this an error. Warn and ignore. */ log_warn(LD_CONFIG, @@ -1949,6 +2357,8 @@ config_assign_line(config_format_t *fmt, or_options_t *options, } } return 0; + } else if (c->command == CONFIG_LINE_CLEAR && !clear_first) { + option_reset(fmt, options, var, use_defaults); } if (options_seen && (var->type != CONFIG_TYPE_LINELIST && @@ -1971,10 +2381,10 @@ config_assign_line(config_format_t *fmt, or_options_t *options, /** Restore the option named <b>key</b> in options to its default value. * Called from config_assign(). */ static void -config_reset_line(config_format_t *fmt, or_options_t *options, +config_reset_line(const config_format_t *fmt, or_options_t *options, const char *key, int use_defaults) { - config_var_t *var; + const config_var_t *var; CHECK(fmt, options); @@ -1989,7 +2399,7 @@ config_reset_line(config_format_t *fmt, or_options_t *options, int option_is_recognized(const char *key) { - config_var_t *var = config_find_option(&options_format, key); + const config_var_t *var = config_find_option(&options_format, key); return (var != NULL); } @@ -1998,14 +2408,14 @@ option_is_recognized(const char *key) const char * option_get_canonical_name(const char *key) { - config_var_t *var = config_find_option(&options_format, key); + const config_var_t *var = config_find_option(&options_format, key); return var ? var->name : NULL; } /** Return a canonical list of the options assigned for key. */ config_line_t * -option_get_assignment(or_options_t *options, const char *key) +option_get_assignment(const or_options_t *options, const char *key) { return get_assigned_option(&options_format, options, key, 1); } @@ -2043,7 +2453,7 @@ config_lines_dup(const config_line_t *inp) config_line_t *result = NULL; config_line_t **next_out = &result; while (inp) { - *next_out = tor_malloc(sizeof(config_line_t)); + *next_out = tor_malloc_zero(sizeof(config_line_t)); (*next_out)->key = tor_strdup(inp->key); (*next_out)->value = tor_strdup(inp->value); inp = inp->next; @@ -2058,10 +2468,10 @@ config_lines_dup(const config_line_t *inp) * value needs to be quoted before it's put in a config file, quote and * escape that value. Return NULL if no such key exists. */ static config_line_t * -get_assigned_option(config_format_t *fmt, void *options, +get_assigned_option(const config_format_t *fmt, const void *options, const char *key, int escape_val) { - config_var_t *var; + const config_var_t *var; const void *value; config_line_t *result; tor_assert(options && key); @@ -2107,7 +2517,9 @@ get_assigned_option(config_format_t *fmt, void *options, } /* fall through */ case CONFIG_TYPE_INTERVAL: + case CONFIG_TYPE_MSEC_INTERVAL: case CONFIG_TYPE_UINT: + case CONFIG_TYPE_INT: /* This means every or_options_t uint or bool element * needs to be an int. Not, say, a uint16_t or char. */ tor_asprintf(&result->value, "%d", *(int*)value); @@ -2122,6 +2534,14 @@ get_assigned_option(config_format_t *fmt, void *options, tor_asprintf(&result->value, "%f", *(double*)value); escape_val = 0; /* Can't need escape. */ break; + + case CONFIG_TYPE_AUTOBOOL: + if (*(int*)value == -1) { + result->value = tor_strdup("auto"); + escape_val = 0; + break; + } + /* fall through */ case CONFIG_TYPE_BOOL: result->value = tor_strdup(*(int*)value ? "1" : "0"); escape_val = 0; /* Can't need escape. */ @@ -2234,7 +2654,7 @@ options_trial_assign() calls config_assign(1, 1) returns. */ static int -config_assign(config_format_t *fmt, void *options, config_line_t *list, +config_assign(const config_format_t *fmt, void *options, config_line_t *list, int use_defaults, int clear_first, char **msg) { config_line_t *p; @@ -2271,6 +2691,12 @@ config_assign(config_format_t *fmt, void *options, config_line_t *list, list = list->next; } bitarray_free(options_seen); + + /** Now we're done assigning a group of options to the configuration. + * Subsequent group assignments should _replace_ linelists, not extend + * them. */ + config_mark_lists_fragile(fmt, options); + return 0; } @@ -2296,7 +2722,7 @@ options_trial_assign(config_line_t *list, int use_defaults, return r; } - if (options_validate(get_options(), trial_options, 1, msg) < 0) { + if (options_validate(get_options_mutable(), trial_options, 1, msg) < 0) { config_free(&options_format, trial_options); return SETOPT_ERR_PARSE; /*XXX make this a separate return value. */ } @@ -2318,7 +2744,8 @@ options_trial_assign(config_line_t *list, int use_defaults, /** Reset config option <b>var</b> to 0, 0.0, NULL, or the equivalent. * Called from option_reset() and config_free(). */ static void -option_clear(config_format_t *fmt, or_options_t *options, config_var_t *var) +option_clear(const config_format_t *fmt, or_options_t *options, + const config_var_t *var) { void *lvalue = STRUCT_VAR_P(options, var->var_offset); (void)fmt; /* unused */ @@ -2334,11 +2761,16 @@ option_clear(config_format_t *fmt, or_options_t *options, config_var_t *var) *(time_t*)lvalue = 0; break; case CONFIG_TYPE_INTERVAL: + case CONFIG_TYPE_MSEC_INTERVAL: case CONFIG_TYPE_UINT: + case CONFIG_TYPE_INT: case CONFIG_TYPE_PORT: case CONFIG_TYPE_BOOL: *(int*)lvalue = 0; break; + case CONFIG_TYPE_AUTOBOOL: + *(int*)lvalue = -1; + break; case CONFIG_TYPE_MEMUNIT: *(uint64_t*)lvalue = 0; break; @@ -2372,8 +2804,8 @@ option_clear(config_format_t *fmt, or_options_t *options, config_var_t *var) * <b>use_defaults</b>, set it to its default value. * Called by config_init() and option_reset_line() and option_assign_line(). */ static void -option_reset(config_format_t *fmt, or_options_t *options, - config_var_t *var, int use_defaults) +option_reset(const config_format_t *fmt, or_options_t *options, + const config_var_t *var, int use_defaults) { config_line_t *c; char *msg = NULL; @@ -2400,7 +2832,7 @@ print_usage(void) printf( "Copyright (c) 2001-2004, Roger Dingledine\n" "Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson\n" -"Copyright (c) 2007-2011, The Tor Project, Inc.\n\n" +"Copyright (c) 2007-2012, The Tor Project, Inc.\n\n" "tor -f <torrc> [args]\n" "See man page for options, or https://www.torproject.org/ for " "documentation.\n"); @@ -2411,9 +2843,9 @@ static void list_torrc_options(void) { int i; - smartlist_t *lines = smartlist_create(); + smartlist_t *lines = smartlist_new(); for (i = 0; _option_vars[i].name; ++i) { - config_var_t *var = &_option_vars[i]; + const config_var_t *var = &_option_vars[i]; if (var->type == CONFIG_TYPE_OBSOLETE || var->type == CONFIG_TYPE_LINELIST_V) continue; @@ -2430,9 +2862,10 @@ static uint32_t last_resolved_addr = 0; * set *<b>hostname_out</b> to a new string holding the hostname we used to * get the address. Return 0 if all is well, or -1 if we can't find a suitable * public IP address. + * XXXX ipv6 */ int -resolve_my_address(int warn_severity, or_options_t *options, +resolve_my_address(int warn_severity, const or_options_t *options, uint32_t *addr_out, char **hostname_out) { struct in_addr in; @@ -2441,7 +2874,7 @@ resolve_my_address(int warn_severity, or_options_t *options, int explicit_ip=1; int explicit_hostname=1; int from_interface=0; - char tmpbuf[INET_NTOA_BUF_LEN]; + char *addr_string = NULL; const char *address = options->Address; int notice_severity = warn_severity <= LOG_NOTICE ? LOG_NOTICE : warn_severity; @@ -2483,48 +2916,43 @@ resolve_my_address(int warn_severity, or_options_t *options, return -1; } from_interface = 1; - in.s_addr = htonl(interface_ip); - tor_inet_ntoa(&in,tmpbuf,sizeof(tmpbuf)); + addr = interface_ip; log_fn(notice_severity, LD_CONFIG, "Learned IP address '%s' for " - "local interface. Using that.", tmpbuf); + "local interface. Using that.", fmt_addr32(addr)); strlcpy(hostname, "<guessed from interfaces>", sizeof(hostname)); } else { /* resolved hostname into addr */ - in.s_addr = htonl(addr); - if (!explicit_hostname && - is_internal_IP(ntohl(in.s_addr), 0)) { + is_internal_IP(addr, 0)) { uint32_t interface_ip; - tor_inet_ntoa(&in,tmpbuf,sizeof(tmpbuf)); log_fn(notice_severity, LD_CONFIG, "Guessed local hostname '%s' " - "resolves to a private IP address (%s). Trying something " - "else.", hostname, tmpbuf); + "resolves to a private IP address (%s). Trying something " + "else.", hostname, fmt_addr32(addr)); if (get_interface_address(warn_severity, &interface_ip)) { log_fn(warn_severity, LD_CONFIG, "Could not get local interface IP address. Too bad."); } else if (is_internal_IP(interface_ip, 0)) { - struct in_addr in2; - in2.s_addr = htonl(interface_ip); - tor_inet_ntoa(&in2,tmpbuf,sizeof(tmpbuf)); log_fn(notice_severity, LD_CONFIG, "Interface IP address '%s' is a private address too. " - "Ignoring.", tmpbuf); + "Ignoring.", fmt_addr32(interface_ip)); } else { from_interface = 1; - in.s_addr = htonl(interface_ip); - tor_inet_ntoa(&in,tmpbuf,sizeof(tmpbuf)); + addr = interface_ip; log_fn(notice_severity, LD_CONFIG, "Learned IP address '%s' for local interface." - " Using that.", tmpbuf); + " Using that.", fmt_addr32(addr)); strlcpy(hostname, "<guessed from interfaces>", sizeof(hostname)); } } } + } else { + addr = ntohl(in.s_addr); /* set addr so that addr_string is not + * illformed */ } - tor_inet_ntoa(&in,tmpbuf,sizeof(tmpbuf)); - if (is_internal_IP(ntohl(in.s_addr), 0)) { + addr_string = tor_dup_ip(addr); + if (is_internal_IP(addr, 0)) { /* make sure we're ok with publishing an internal IP */ if (!options->DirServers && !options->AlternateDirAuthority) { /* if they are using the default dirservers, disallow internal IPs @@ -2532,7 +2960,8 @@ resolve_my_address(int warn_severity, or_options_t *options, log_fn(warn_severity, LD_CONFIG, "Address '%s' resolves to private IP address '%s'. " "Tor servers that use the default DirServers must have public " - "IP addresses.", hostname, tmpbuf); + "IP addresses.", hostname, addr_string); + tor_free(addr_string); return -1; } if (!explicit_ip) { @@ -2540,19 +2969,20 @@ resolve_my_address(int warn_severity, or_options_t *options, * they're using an internal address. */ log_fn(warn_severity, LD_CONFIG, "Address '%s' resolves to private " "IP address '%s'. Please set the Address config option to be " - "the IP address you want to use.", hostname, tmpbuf); + "the IP address you want to use.", hostname, addr_string); + tor_free(addr_string); return -1; } } - log_debug(LD_CONFIG, "Resolved Address to '%s'.", tmpbuf); - *addr_out = ntohl(in.s_addr); + log_debug(LD_CONFIG, "Resolved Address to '%s'.", fmt_addr32(addr)); + *addr_out = addr; if (last_resolved_addr && last_resolved_addr != *addr_out) { /* Leave this as a notice, regardless of the requested severity, * at least until dynamic IP address support becomes bulletproof. */ log_notice(LD_NET, "Your IP address seems to have changed to %s. Updating.", - tmpbuf); + addr_string); ip_address_changed(0); } if (last_resolved_addr != *addr_out) { @@ -2571,11 +3001,12 @@ resolve_my_address(int warn_severity, or_options_t *options, } control_event_server_status(LOG_NOTICE, "EXTERNAL_ADDRESS ADDRESS=%s METHOD=%s %s%s", - tmpbuf, method, h?"HOSTNAME=":"", h); + addr_string, method, h?"HOSTNAME=":"", h); } last_resolved_addr = *addr_out; if (hostname_out) *hostname_out = tor_strdup(hostname); + tor_free(addr_string); return 0; } @@ -2610,7 +3041,7 @@ is_local_addr(const tor_addr_t *addr) /** Release storage held by <b>options</b>. */ static void -config_free(config_format_t *fmt, void *options) +config_free(const config_format_t *fmt, void *options) { int i; @@ -2645,12 +3076,27 @@ config_lines_eq(config_line_t *a, config_line_t *b) return 1; } +/** Return the number of lines in <b>a</b> whose key is <b>key</b>. */ +static int +config_count_key(const config_line_t *a, const char *key) +{ + int n = 0; + while (a) { + if (!strcasecmp(a->key, key)) { + ++n; + } + a = a->next; + } + return n; +} + /** Return true iff the option <b>name</b> has the same value in <b>o1</b> * and <b>o2</b>. Must not be called for LINELIST_S or OBSOLETE options. */ static int -option_is_same(config_format_t *fmt, - or_options_t *o1, or_options_t *o2, const char *name) +option_is_same(const config_format_t *fmt, + const or_options_t *o1, const or_options_t *o2, + const char *name) { config_line_t *c1, *c2; int r = 1; @@ -2667,7 +3113,7 @@ option_is_same(config_format_t *fmt, /** Copy storage held by <b>old</b> into a new or_options_t and return it. */ static or_options_t * -options_dup(config_format_t *fmt, or_options_t *old) +options_dup(const config_format_t *fmt, const or_options_t *old) { or_options_t *newopts; int i; @@ -2709,44 +3155,13 @@ options_init(or_options_t *options) config_init(&options_format, options); } -/* Check if the port number given in <b>port_option</b> in combination with - * the specified port in <b>listen_options</b> will result in Tor actually - * opening a low port (meaning a port lower than 1024). Return 1 if - * it is, or 0 if it isn't or the concept of a low port isn't applicable for - * the platform we're on. */ -static int -is_listening_on_low_port(int port_option, - const config_line_t *listen_options) -{ -#ifdef MS_WINDOWS - (void) port_option; - (void) listen_options; - return 0; /* No port is too low for windows. */ -#else - const config_line_t *l; - uint16_t p; - if (port_option == 0) - return 0; /* We're not listening */ - if (listen_options == NULL) - return (port_option < 1024); - - for (l = listen_options; l; l = l->next) { - parse_addr_port(LOG_WARN, l->value, NULL, NULL, &p); - if (p<1024) { - return 1; - } - } - return 0; -#endif -} - /** Set all vars in the configuration object <b>options</b> to their default * values. */ static void -config_init(config_format_t *fmt, void *options) +config_init(const config_format_t *fmt, void *options) { int i; - config_var_t *var; + const config_var_t *var; CHECK(fmt, options); for (i=0; fmt->vars[i].name; ++i) { @@ -2762,27 +3177,33 @@ config_init(config_format_t *fmt, void *options) * Else, if comment_defaults, write default values as comments. */ static char * -config_dump(config_format_t *fmt, void *options, int minimal, +config_dump(const config_format_t *fmt, const void *default_options, + const void *options, int minimal, int comment_defaults) { smartlist_t *elements; - or_options_t *defaults; + const or_options_t *defaults = default_options; + void *defaults_tmp = NULL; config_line_t *line, *assigned; char *result; int i; char *msg = NULL; - defaults = config_alloc(fmt); - config_init(fmt, defaults); + if (defaults == NULL) { + defaults = defaults_tmp = config_alloc(fmt); + config_init(fmt, defaults_tmp); + } /* XXX use a 1 here so we don't add a new log line while dumping */ - if (fmt->validate_fn(NULL,defaults, 1, &msg) < 0) { - log_err(LD_BUG, "Failed to validate default config."); - tor_free(msg); - tor_assert(0); + if (default_options == NULL) { + if (fmt->validate_fn(NULL, defaults_tmp, 1, &msg) < 0) { + log_err(LD_BUG, "Failed to validate default config."); + tor_free(msg); + tor_assert(0); + } } - elements = smartlist_create(); + elements = smartlist_new(); for (i=0; fmt->vars[i].name; ++i) { int comment_option = 0; if (fmt->vars[i].type == CONFIG_TYPE_OBSOLETE || @@ -2800,11 +3221,9 @@ config_dump(config_format_t *fmt, void *options, int minimal, line = assigned = get_assigned_option(fmt, options, fmt->vars[i].name, 1); for (; line; line = line->next) { - char *tmp; - tor_asprintf(&tmp, "%s%s %s\n", + smartlist_add_asprintf(elements, "%s%s %s\n", comment_option ? "# " : "", line->key, line->value); - smartlist_add(elements, tmp); } config_free_lines(assigned); } @@ -2812,16 +3231,15 @@ config_dump(config_format_t *fmt, void *options, int minimal, if (fmt->extra) { line = *(config_line_t**)STRUCT_VAR_P(options, fmt->extra->var_offset); for (; line; line = line->next) { - char *tmp; - tor_asprintf(&tmp, "%s %s\n", line->key, line->value); - smartlist_add(elements, tmp); + smartlist_add_asprintf(elements, "%s %s\n", line->key, line->value); } } result = smartlist_join_strings(elements, "", 0, NULL); SMARTLIST_FOREACH(elements, char *, cp, tor_free(cp)); smartlist_free(elements); - config_free(fmt, defaults); + if (defaults_tmp) + config_free(fmt, defaults_tmp); return result; } @@ -2830,9 +3248,10 @@ config_dump(config_format_t *fmt, void *options, int minimal, * include options that are the same as Tor's defaults. */ char * -options_dump(or_options_t *options, int minimal) +options_dump(const or_options_t *options, int minimal) { - return config_dump(&options_format, options, minimal, 0); + return config_dump(&options_format, global_default_options, + options, minimal, 0); } /** Return 0 if every element of sl is a string holding a decimal @@ -2889,24 +3308,24 @@ static int compute_publishserverdescriptor(or_options_t *options) { smartlist_t *list = options->PublishServerDescriptor; - authority_type_t *auth = &options->_PublishServerDescriptor; - *auth = NO_AUTHORITY; + dirinfo_type_t *auth = &options->_PublishServerDescriptor; + *auth = NO_DIRINFO; if (!list) /* empty list, answer is none */ return 0; - SMARTLIST_FOREACH(list, const char *, string, { + SMARTLIST_FOREACH_BEGIN(list, const char *, string) { if (!strcasecmp(string, "v1")) - *auth |= V1_AUTHORITY; + *auth |= V1_DIRINFO; else if (!strcmp(string, "1")) if (options->BridgeRelay) - *auth |= BRIDGE_AUTHORITY; + *auth |= BRIDGE_DIRINFO; else - *auth |= V2_AUTHORITY | V3_AUTHORITY; + *auth |= V2_DIRINFO | V3_DIRINFO; else if (!strcasecmp(string, "v2")) - *auth |= V2_AUTHORITY; + *auth |= V2_DIRINFO; else if (!strcasecmp(string, "v3")) - *auth |= V3_AUTHORITY; + *auth |= V3_DIRINFO; else if (!strcasecmp(string, "bridge")) - *auth |= BRIDGE_AUTHORITY; + *auth |= BRIDGE_DIRINFO; else if (!strcasecmp(string, "hidserv")) log_warn(LD_CONFIG, "PublishServerDescriptor hidserv is invalid. See " @@ -2915,7 +3334,7 @@ compute_publishserverdescriptor(or_options_t *options) /* no authority */; else return -1; - }); + } SMARTLIST_FOREACH_END(string); return 0; } @@ -2934,6 +3353,17 @@ compute_publishserverdescriptor(or_options_t *options) * will generate too many circuits and potentially overload the network. */ #define MIN_CIRCUIT_STREAM_TIMEOUT 10 +/** Lowest allowable value for HeartbeatPeriod; if this is too low, we might + * expose more information than we're comfortable with. */ +#define MIN_HEARTBEAT_PERIOD (30*60) + +/** Lowest recommended value for CircuitBuildTimeout; if it is set too low + * and LearnCircuitBuildTimeout is off, the failure rate for circuit + * construction may be very high. In that case, if it is set below this + * threshold emit a warning. + * */ +#define RECOMMENDED_MIN_CIRCUIT_BUILD_TIMEOUT (10) + /** Return 0 if every setting in <b>options</b> is reasonable, and a * permissible transition from <b>old_options</b>. Else return -1. * Should have no side effects, except for normalizing the contents of @@ -2953,6 +3383,7 @@ options_validate(or_options_t *old_options, or_options_t *options, int i; config_line_t *cl; const char *uname = get_uname(); + int n_ports=0; #define REJECT(arg) \ STMT_BEGIN *msg = tor_strdup(arg); return -1; STMT_END #define COMPLAIN(arg) STMT_BEGIN log(LOG_WARN, LD_CONFIG, arg); STMT_END @@ -2970,63 +3401,8 @@ options_validate(or_options_t *old_options, or_options_t *options, "for details.", uname); } - if (options->ORPort == 0 && options->ORListenAddress != NULL) - REJECT("ORPort must be defined if ORListenAddress is defined."); - - if (options->DirPort == 0 && options->DirListenAddress != NULL) - REJECT("DirPort must be defined if DirListenAddress is defined."); - - if (options->DNSPort == 0 && options->DNSListenAddress != NULL) - REJECT("DNSPort must be defined if DNSListenAddress is defined."); - - if (options->ControlPort == 0 && options->ControlListenAddress != NULL) - REJECT("ControlPort must be defined if ControlListenAddress is defined."); - - if (options->TransPort == 0 && options->TransListenAddress != NULL) - REJECT("TransPort must be defined if TransListenAddress is defined."); - - if (options->NATDPort == 0 && options->NATDListenAddress != NULL) - REJECT("NATDPort must be defined if NATDListenAddress is defined."); - - /* Don't gripe about SocksPort 0 with SocksListenAddress set; a standard - * configuration does this. */ - - for (i = 0; i < 3; ++i) { - int is_socks = i==0; - int is_trans = i==1; - config_line_t *line, *opt, *old; - const char *tp; - if (is_socks) { - opt = options->SocksListenAddress; - old = old_options ? old_options->SocksListenAddress : NULL; - tp = "SOCKS proxy"; - } else if (is_trans) { - opt = options->TransListenAddress; - old = old_options ? old_options->TransListenAddress : NULL; - tp = "transparent proxy"; - } else { - opt = options->NATDListenAddress; - old = old_options ? old_options->NATDListenAddress : NULL; - tp = "natd proxy"; - } - - for (line = opt; line; line = line->next) { - char *address = NULL; - uint16_t port; - uint32_t addr; - if (parse_addr_port(LOG_WARN, line->value, &address, &addr, &port)<0) - continue; /* We'll warn about this later. */ - if (!is_internal_IP(addr, 1) && - (!old_options || !config_lines_eq(old, opt))) { - log_warn(LD_CONFIG, - "You specified a public address '%s' for a %s. Other " - "people on the Internet might find your computer and use it as " - "an open %s. Please don't allow this unless you have " - "a good reason.", address, tp, tp); - } - tor_free(address); - } - } + if (parse_ports(options, 1, msg, &n_ports) < 0) + return -1; if (validate_data_directory(options)<0) REJECT("Invalid DataDirectory"); @@ -3050,8 +3426,12 @@ options_validate(or_options_t *old_options, or_options_t *options, "misconfigured or something else goes wrong."); /* Special case on first boot if no Log options are given. */ - if (!options->Logs && !options->RunAsDaemon && !from_setconf) - config_line_append(&options->Logs, "Log", "notice stdout"); + if (!options->Logs && !options->RunAsDaemon && !from_setconf) { + if (quiet_level == 0) + config_line_append(&options->Logs, "Log", "notice stdout"); + else if (quiet_level == 1) + config_line_append(&options->Logs, "Log", "warn stdout"); + } if (options_init_logs(options, 1)<0) /* Validate the log(s) */ REJECT("Failed to validate Log options. See logs for details."); @@ -3063,41 +3443,28 @@ options_validate(or_options_t *old_options, or_options_t *options, REJECT("Failed to resolve/guess local address. See logs for details."); } - if (strcmp(options->RefuseUnknownExits, "0") && - strcmp(options->RefuseUnknownExits, "1") && - strcmp(options->RefuseUnknownExits, "auto")) { - REJECT("RefuseUnknownExits must be 0, 1, or auto"); - } - -#ifndef MS_WINDOWS +#ifndef _WIN32 if (options->RunAsDaemon && torrc_fname && path_is_relative(torrc_fname)) REJECT("Can't use a relative path to torrc when RunAsDaemon is set."); #endif - if (options->SocksPort == 0 && options->TransPort == 0 && - options->NATDPort == 0 && options->ORPort == 0 && - options->DNSPort == 0 && !options->RendConfigLines) + /* XXXX require that the only port not be DirPort? */ + /* XXXX require that at least one port be listened-upon. */ + if (n_ports == 0 && !options->RendConfigLines) log(LOG_WARN, LD_CONFIG, "SocksPort, TransPort, NATDPort, DNSPort, and ORPort are all " "undefined, and there aren't any hidden services configured. " "Tor will still run, but probably won't do anything."); #ifndef USE_TRANSPARENT - if (options->TransPort || options->TransListenAddress) + /* XXXX024 I think we can remove this TransListenAddress */ + if (options->TransPort_set || options->TransListenAddress) REJECT("TransPort and TransListenAddress are disabled in this build."); #endif - if (options->AccountingMax && - (is_listening_on_low_port(options->ORPort, options->ORListenAddress) || - is_listening_on_low_port(options->DirPort, options->DirListenAddress))) - { - log(LOG_WARN, LD_CONFIG, - "You have set AccountingMax to use hibernation. You have also " - "chosen a low DirPort or OrPort. This combination can make Tor stop " - "working when it tries to re-attach the port after a period of " - "hibernation. Please choose a different port or turn off " - "hibernation unless you know this combination will work on your " - "platform."); + if (options->TokenBucketRefillInterval <= 0 + || options->TokenBucketRefillInterval > 1000) { + REJECT("TokenBucketRefillInterval must be between 1 and 1000 inclusive."); } if (options->ExcludeExitNodes || options->ExcludeNodes) { @@ -3106,17 +3473,24 @@ options_validate(or_options_t *old_options, or_options_t *options, routerset_union(options->_ExcludeExitNodesUnion,options->ExcludeNodes); } + if (options->NodeFamilies) { + options->NodeFamilySets = smartlist_new(); + for (cl = options->NodeFamilies; cl; cl = cl->next) { + routerset_t *rs = routerset_new(); + if (routerset_parse(rs, cl->value, cl->key) == 0) { + smartlist_add(options->NodeFamilySets, rs); + } else { + routerset_free(rs); + } + } + } + if (options->ExcludeNodes && options->StrictNodes) { COMPLAIN("You have asked to exclude certain relays from all positions " "in your circuits. Expect hidden services and other Tor " "features to be broken in unpredictable ways."); } - if (options->EntryNodes && !routerset_is_list(options->EntryNodes)) { - /* XXXX fix this; see entry_guards_prepend_from_config(). */ - REJECT("IPs or countries are not yet supported in EntryNodes."); - } - if (options->AuthoritativeDir) { if (!options->ContactInfo && !options->TestingTorNetwork) REJECT("Authoritative directory servers must set ContactInfo"); @@ -3154,10 +3528,10 @@ options_validate(or_options_t *old_options, or_options_t *options, } } - if (options->AuthoritativeDir && !options->DirPort) + if (options->AuthoritativeDir && !options->DirPort_set) REJECT("Running as authoritative directory, but no DirPort set."); - if (options->AuthoritativeDir && !options->ORPort) + if (options->AuthoritativeDir && !options->ORPort_set) REJECT("Running as authoritative directory, but no ORPort set."); if (options->AuthoritativeDir && options->ClientOnly) @@ -3178,6 +3552,15 @@ options_validate(or_options_t *old_options, or_options_t *options, return -1; } + if (options->MaxClientCircuitsPending <= 0 || + options->MaxClientCircuitsPending > MAX_MAX_CLIENT_CIRCUITS_PENDING) { + tor_asprintf(msg, + "MaxClientCircuitsPending must be between 1 and %d, but " + "was set to %d", MAX_MAX_CLIENT_CIRCUITS_PENDING, + options->MaxClientCircuitsPending); + return -1; + } + if (validate_ports_csv(options->FirewallPorts, "FirewallPorts", msg) < 0) return -1; @@ -3197,7 +3580,7 @@ options_validate(or_options_t *old_options, or_options_t *options, /* We already have firewall ports set, so migrate them to * ReachableAddresses, which will set ReachableORAddresses and * ReachableDirAddresses if they aren't set explicitly. */ - smartlist_t *instead = smartlist_create(); + smartlist_t *instead = smartlist_new(); config_line_t *new_line = tor_malloc_zero(sizeof(config_line_t)); new_line->key = tor_strdup("ReachableAddresses"); /* If we're configured with the old format, we need to prepend some @@ -3205,11 +3588,8 @@ options_validate(or_options_t *old_options, or_options_t *options, SMARTLIST_FOREACH(options->FirewallPorts, const char *, portno, { int p = atoi(portno); - char *s; if (p<0) continue; - s = tor_malloc(16); - tor_snprintf(s, 16, "*:%d", p); - smartlist_add(instead, s); + smartlist_add_asprintf(instead, "*:%d", p); }); new_line->value = smartlist_join_strings(instead,",",0,NULL); /* These have been deprecated since 0.1.1.5-alpha-cvs */ @@ -3285,9 +3665,19 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options->UseBridges && options->EntryNodes) REJECT("You cannot set both UseBridges and EntryNodes."); + if (options->EntryNodes && !options->UseEntryGuards) + log_warn(LD_CONFIG, "EntryNodes is set, but UseEntryGuards is disabled. " + "EntryNodes will be ignored."); + + if (options->MaxMemInCellQueues < (500 << 20)) { + log_warn(LD_CONFIG, "MaxMemInCellQueues must be at least 500 MB for now. " + "Ideally, have it as large as you can afford."); + options->MaxMemInCellQueues = (500 << 20); + } + options->_AllowInvalid = 0; if (options->AllowInvalidNodes) { - SMARTLIST_FOREACH(options->AllowInvalidNodes, const char *, cp, { + SMARTLIST_FOREACH_BEGIN(options->AllowInvalidNodes, const char *, cp) { if (!strcasecmp(cp, "entry")) options->_AllowInvalid |= ALLOW_INVALID_ENTRY; else if (!strcasecmp(cp, "exit")) @@ -3303,7 +3693,7 @@ options_validate(or_options_t *old_options, or_options_t *options, "Unrecognized value '%s' in AllowInvalidNodes", cp); return -1; } - }); + } SMARTLIST_FOREACH_END(cp); } if (!options->SafeLogging || @@ -3326,18 +3716,20 @@ options_validate(or_options_t *old_options, or_options_t *options, } if ((options->BridgeRelay - || options->_PublishServerDescriptor & BRIDGE_AUTHORITY) + || options->_PublishServerDescriptor & BRIDGE_DIRINFO) && (options->_PublishServerDescriptor - & (V1_AUTHORITY|V2_AUTHORITY|V3_AUTHORITY))) { + & (V1_DIRINFO|V2_DIRINFO|V3_DIRINFO))) { REJECT("Bridges are not supposed to publish router descriptors to the " "directory authorities. Please correct your " "PublishServerDescriptor line."); } - if (options->BridgeRelay && options->DirPort) { + if (options->BridgeRelay && options->DirPort_set) { log_warn(LD_CONFIG, "Can't set a DirPort on a bridge relay; disabling " "DirPort"); - options->DirPort = 0; + config_free_lines(options->DirPort_lines); + options->DirPort_lines = NULL; + options->DirPort_set = 0; } if (options->MinUptimeHidServDirectoryV2 < 0) { @@ -3358,6 +3750,35 @@ options_validate(or_options_t *old_options, or_options_t *options, options->RendPostPeriod = MAX_DIR_PERIOD; } + if (options->Tor2webMode && options->LearnCircuitBuildTimeout) { + /* LearnCircuitBuildTimeout and Tor2webMode are incompatible in + * two ways: + * + * - LearnCircuitBuildTimeout results in a low CBT, which + * Tor2webMode's use of one-hop rendezvous circuits lowers + * much further, producing *far* too many timeouts. + * + * - The adaptive CBT code does not update its timeout estimate + * using build times for single-hop circuits. + * + * If we fix both of these issues someday, we should test + * Tor2webMode with LearnCircuitBuildTimeout on again. */ + log_notice(LD_CONFIG,"Tor2webMode is enabled; turning " + "LearnCircuitBuildTimeout off."); + options->LearnCircuitBuildTimeout = 0; + } + + if (!(options->LearnCircuitBuildTimeout) && + options->CircuitBuildTimeout < RECOMMENDED_MIN_CIRCUIT_BUILD_TIMEOUT) { + log_warn(LD_CONFIG, + "CircuitBuildTimeout is shorter (%d seconds) than recommended " + "(%d seconds), and LearnCircuitBuildTimeout is disabled. " + "If tor isn't working, raise this value or enable " + "LearnCircuitBuildTimeout.", + options->CircuitBuildTimeout, + RECOMMENDED_MIN_CIRCUIT_BUILD_TIMEOUT ); + } + if (options->MaxCircuitDirtiness < MIN_MAX_CIRCUIT_DIRTINESS) { log_warn(LD_CONFIG, "MaxCircuitDirtiness option is too short; " "raising to %d seconds.", MIN_MAX_CIRCUIT_DIRTINESS); @@ -3371,6 +3792,13 @@ options_validate(or_options_t *old_options, or_options_t *options, options->CircuitStreamTimeout = MIN_CIRCUIT_STREAM_TIMEOUT; } + if (options->HeartbeatPeriod && + options->HeartbeatPeriod < MIN_HEARTBEAT_PERIOD) { + log_warn(LD_CONFIG, "HeartbeatPeriod option is too short; " + "raising to %d seconds.", MIN_HEARTBEAT_PERIOD); + options->HeartbeatPeriod = MIN_HEARTBEAT_PERIOD; + } + if (options->KeepalivePeriod < 1) REJECT("KeepalivePeriod option must be positive."); @@ -3452,8 +3880,22 @@ options_validate(or_options_t *old_options, or_options_t *options, if (accounting_parse_options(options, 1)<0) REJECT("Failed to parse accounting options. See logs for details."); + if (options->AccountingMax) { + if (options->RendConfigLines && server_mode(options)) { + log_warn(LD_CONFIG, "Using accounting with a hidden service and an " + "ORPort is risky: your hidden service(s) and your public " + "address will all turn off at the same time, which may alert " + "observers that they are being run by the same party."); + } else if (config_count_key(options->RendConfigLines, + "HiddenServiceDir") > 1) { + log_warn(LD_CONFIG, "Using accounting with multiple hidden services is " + "risky: they will all turn off at the same time, which may " + "alert observers that they are being run by the same party."); + } + } + if (options->HTTPProxy) { /* parse it now */ - if (tor_addr_port_parse(options->HTTPProxy, + if (tor_addr_port_lookup(options->HTTPProxy, &options->HTTPProxyAddr, &options->HTTPProxyPort) < 0) REJECT("HTTPProxy failed to parse or resolve. Please fix."); if (options->HTTPProxyPort == 0) { /* give it a default */ @@ -3467,7 +3909,7 @@ options_validate(or_options_t *old_options, or_options_t *options, } if (options->HTTPSProxy) { /* parse it now */ - if (tor_addr_port_parse(options->HTTPSProxy, + if (tor_addr_port_lookup(options->HTTPSProxy, &options->HTTPSProxyAddr, &options->HTTPSProxyPort) <0) REJECT("HTTPSProxy failed to parse or resolve. Please fix."); if (options->HTTPSProxyPort == 0) { /* give it a default */ @@ -3481,7 +3923,7 @@ options_validate(or_options_t *old_options, or_options_t *options, } if (options->Socks4Proxy) { /* parse it now */ - if (tor_addr_port_parse(options->Socks4Proxy, + if (tor_addr_port_lookup(options->Socks4Proxy, &options->Socks4ProxyAddr, &options->Socks4ProxyPort) <0) REJECT("Socks4Proxy failed to parse or resolve. Please fix."); @@ -3491,7 +3933,7 @@ options_validate(or_options_t *old_options, or_options_t *options, } if (options->Socks5Proxy) { /* parse it now */ - if (tor_addr_port_parse(options->Socks5Proxy, + if (tor_addr_port_lookup(options->Socks5Proxy, &options->Socks5ProxyAddr, &options->Socks5ProxyPort) <0) REJECT("Socks5Proxy failed to parse or resolve. Please fix."); @@ -3500,8 +3942,20 @@ options_validate(or_options_t *old_options, or_options_t *options, } } - if (options->Socks4Proxy && options->Socks5Proxy) - REJECT("You cannot specify both Socks4Proxy and SOCKS5Proxy"); + /* Check if more than one proxy type has been enabled. */ + if (!!options->Socks4Proxy + !!options->Socks5Proxy + + !!options->HTTPSProxy + !!options->ClientTransportPlugin > 1) + REJECT("You have configured more than one proxy type. " + "(Socks4Proxy|Socks5Proxy|HTTPSProxy|ClientTransportPlugin)"); + + /* Check if the proxies will give surprising behavior. */ + if (options->HTTPProxy && !(options->Socks4Proxy || + options->Socks5Proxy || + options->HTTPSProxy)) { + log_warn(LD_CONFIG, "HTTPProxy configured, but no SOCKS proxy or " + "HTTPS proxy configured. Watch out: this configuration will " + "proxy unencrypted directory connections only."); + } if (options->Socks5ProxyUsername) { size_t len; @@ -3550,40 +4004,7 @@ options_validate(or_options_t *old_options, or_options_t *options, } } - if (options->ControlListenAddress) { - int all_are_local = 1; - config_line_t *ln; - for (ln = options->ControlListenAddress; ln; ln = ln->next) { - if (strcmpstart(ln->value, "127.")) - all_are_local = 0; - } - if (!all_are_local) { - if (!options->HashedControlPassword && - !options->HashedControlSessionPassword && - !options->CookieAuthentication) { - log_warn(LD_CONFIG, - "You have a ControlListenAddress set to accept " - "unauthenticated connections from a non-local address. " - "This means that programs not running on your computer " - "can reconfigure your Tor, without even having to guess a " - "password. That's so bad that I'm closing your ControlPort " - "for you. If you need to control your Tor remotely, try " - "enabling authentication and using a tool like stunnel or " - "ssh to encrypt remote access."); - options->ControlPort = 0; - } else { - log_warn(LD_CONFIG, "You have a ControlListenAddress set to accept " - "connections from a non-local address. This means that " - "programs not running on your computer can reconfigure your " - "Tor. That's pretty bad, since the controller " - "protocol isn't encrypted! Maybe you should just listen on " - "127.0.0.1 and use a tool like stunnel or ssh to encrypt " - "remote connections to your control port."); - } - } - } - - if (options->ControlPort && !options->HashedControlPassword && + if (options->ControlPort_set && !options->HashedControlPassword && !options->HashedControlSessionPassword && !options->CookieAuthentication) { log_warn(LD_CONFIG, "ControlPort is open, but no authentication method " @@ -3601,11 +4022,21 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options->UseEntryGuards && ! options->NumEntryGuards) REJECT("Cannot enable UseEntryGuards with NumEntryGuards set to 0"); + if (options->MyFamily && options->BridgeRelay) { + log_warn(LD_CONFIG, "Listing a family for a bridge relay is not " + "supported: it can reveal bridge fingerprints to censors. " + "You should also make sure you aren't listing this bridge's " + "fingerprint in any other MyFamily."); + } if (check_nickname_list(options->MyFamily, "MyFamily", msg)) return -1; for (cl = options->NodeFamilies; cl; cl = cl->next) { - if (check_nickname_list(cl->value, "NodeFamily", msg)) + routerset_t *rs = routerset_new(); + if (routerset_parse(rs, cl->value, cl->key)) { + routerset_free(rs); return -1; + } + routerset_free(rs); } if (validate_addr_policies(options, msg) < 0) @@ -3618,11 +4049,27 @@ options_validate(or_options_t *old_options, or_options_t *options, REJECT("If you set UseBridges, you must specify at least one bridge."); if (options->UseBridges && !options->TunnelDirConns) REJECT("If you set UseBridges, you must set TunnelDirConns."); - if (options->Bridges) { - for (cl = options->Bridges; cl; cl = cl->next) { - if (parse_bridge_line(cl->value, 1)<0) - REJECT("Bridge line did not parse. See logs for details."); - } + + for (cl = options->Bridges; cl; cl = cl->next) { + if (parse_bridge_line(cl->value, 1)<0) + REJECT("Bridge line did not parse. See logs for details."); + } + + for (cl = options->ClientTransportPlugin; cl; cl = cl->next) { + if (parse_client_transport_line(cl->value, 1)<0) + REJECT("Transport line did not parse. See logs for details."); + } + + for (cl = options->ServerTransportPlugin; cl; cl = cl->next) { + if (parse_server_transport_line(cl->value, 1)<0) + REJECT("Server transport line did not parse. See logs for details."); + } + + if (options->ServerTransportPlugin && !server_mode(options)) { + log_notice(LD_GENERAL, "Tor is not configured as a relay but you specified" + " a ServerTransportPlugin line (%s). The ServerTransportPlugin " + "line will be ignored.", + esc_for_log(options->ServerTransportPlugin->value)); } if (options->ConstrainedSockets) { @@ -3637,7 +4084,7 @@ options_validate(or_options_t *old_options, or_options_t *options, MIN_CONSTRAINED_TCP_BUFFER, MAX_CONSTRAINED_TCP_BUFFER); return -1; } - if (options->DirPort) { + if (options->DirPort_set) { /* Providing cached directory entries while system TCP buffers are scarce * will exacerbate the socket errors. Suggest that this be disabled. */ COMPLAIN("You have requested constrained socket buffers while also " @@ -3697,9 +4144,13 @@ options_validate(or_options_t *old_options, or_options_t *options, }); } - if (options->TestingTorNetwork && !options->DirServers) { + if (options->TestingTorNetwork && + !(options->DirServers || + (options->AlternateDirAuthority && + options->AlternateBridgeAuthority))) { REJECT("TestingTorNetwork may only be configured in combination with " - "a non-default set of DirServers."); + "a non-default set of DirServer or both of AlternateDirAuthority " + "and AlternateBridgeAuthority configured."); } if (options->AllowSingleHopExits && !options->DirServers) { @@ -3709,7 +4160,7 @@ options_validate(or_options_t *old_options, or_options_t *options, "ignore you."); } - /*XXXX023 checking for defaults manually like this is a bit fragile.*/ + /*XXXX checking for defaults manually like this is a bit fragile.*/ /* Keep changes to hard-coded values synchronous to man page and default * values table. */ @@ -3792,8 +4243,9 @@ options_validate(or_options_t *old_options, or_options_t *options, } }); - if (options->BridgeRelay == 1 && options->ORPort == 0) - REJECT("BridgeRelay is 1, ORPort is 0. This is an invalid combination."); + if (options->BridgeRelay == 1 && ! options->ORPort_set) + REJECT("BridgeRelay is 1, ORPort is not set. This is an invalid " + "combination."); return 0; #undef REJECT @@ -3805,17 +4257,13 @@ options_validate(or_options_t *old_options, or_options_t *options, static int opt_streq(const char *s1, const char *s2) { - if (!s1 && !s2) - return 1; - else if (s1 && s2 && !strcmp(s1,s2)) - return 1; - else - return 0; + return 0 == strcmp_opt(s1, s2); } /** Check if any of the previous options have changed but aren't allowed to. */ static int -options_transition_allowed(or_options_t *old, or_options_t *new_val, +options_transition_allowed(const or_options_t *old, + const or_options_t *new_val, char **msg) { if (!old) @@ -3865,18 +4313,37 @@ options_transition_allowed(or_options_t *old, or_options_t *new_val, return -1; } + if (old->TokenBucketRefillInterval != new_val->TokenBucketRefillInterval) { + *msg = tor_strdup("While Tor is running, changing TokenBucketRefill" + "Interval is not allowed"); + return -1; + } + + if (old->DisableIOCP != new_val->DisableIOCP) { + *msg = tor_strdup("While Tor is running, changing DisableIOCP " + "is not allowed."); + return -1; + } + + if (old->DisableDebuggerAttachment && + !new_val->DisableDebuggerAttachment) { + *msg = tor_strdup("While Tor is running, disabling " + "DisableDebuggerAttachment is not allowed."); + return -1; + } + return 0; } /** Return 1 if any change from <b>old_options</b> to <b>new_options</b> * will require us to rotate the CPU and DNS workers; else return 0. */ static int -options_transition_affects_workers(or_options_t *old_options, - or_options_t *new_options) +options_transition_affects_workers(const or_options_t *old_options, + const or_options_t *new_options) { if (!opt_streq(old_options->DataDirectory, new_options->DataDirectory) || old_options->NumCPUs != new_options->NumCPUs || - old_options->ORPort != new_options->ORPort || + !config_lines_eq(old_options->ORPort_lines, new_options->ORPort_lines) || old_options->ServerDNSSearchDomains != new_options->ServerDNSSearchDomains || old_options->_SafeLogging != new_options->_SafeLogging || @@ -3895,8 +4362,8 @@ options_transition_affects_workers(or_options_t *old_options, /** Return 1 if any change from <b>old_options</b> to <b>new_options</b> * will require us to generate a new descriptor; else return 0. */ static int -options_transition_affects_descriptor(or_options_t *old_options, - or_options_t *new_options) +options_transition_affects_descriptor(const or_options_t *old_options, + const or_options_t *new_options) { /* XXX We can be smarter here. If your DirPort isn't being * published and you just turned it off, no need to republish. Etc. */ @@ -3906,9 +4373,12 @@ options_transition_affects_descriptor(or_options_t *old_options, !config_lines_eq(old_options->ExitPolicy,new_options->ExitPolicy) || old_options->ExitPolicyRejectPrivate != new_options->ExitPolicyRejectPrivate || - old_options->ORPort != new_options->ORPort || - old_options->DirPort != new_options->DirPort || + !config_lines_eq(old_options->ORPort_lines, + new_options->ORPort_lines) || + !config_lines_eq(old_options->DirPort_lines, + new_options->DirPort_lines) || old_options->ClientOnly != new_options->ClientOnly || + old_options->DisableNetwork != new_options->DisableNetwork || old_options->_PublishServerDescriptor != new_options->_PublishServerDescriptor || get_effective_bwrate(old_options) != get_effective_bwrate(new_options) || @@ -3917,20 +4387,21 @@ options_transition_affects_descriptor(or_options_t *old_options, !opt_streq(old_options->ContactInfo, new_options->ContactInfo) || !opt_streq(old_options->MyFamily, new_options->MyFamily) || !opt_streq(old_options->AccountingStart, new_options->AccountingStart) || - old_options->AccountingMax != new_options->AccountingMax) + old_options->AccountingMax != new_options->AccountingMax || + public_server_mode(old_options) != public_server_mode(new_options)) return 1; return 0; } -#ifdef MS_WINDOWS +#ifdef _WIN32 /** Return the directory on windows where we expect to find our application * data. */ static char * get_windows_conf_root(void) { static int is_set = 0; - static char path[MAX_PATH+1]; + static char path[MAX_PATH*2+1]; TCHAR tpath[MAX_PATH] = {0}; LPITEMIDLIST idl; @@ -3960,7 +4431,8 @@ get_windows_conf_root(void) /* Convert the path from an "ID List" (whatever that is!) to a path. */ result = SHGetPathFromIDList(idl, tpath); #ifdef UNICODE - wcstombs(path,tpath,MAX_PATH); + wcstombs(path,tpath,sizeof(path)); + path[sizeof(path)-1] = '\0'; #else strlcpy(path,tpath,sizeof(path)); #endif @@ -3981,17 +4453,25 @@ get_windows_conf_root(void) } #endif -/** Return the default location for our torrc file. */ +/** Return the default location for our torrc file (if <b>defaults_file</b> is + * false), or for the torrc-defaults file (if <b>defaults_file</b> is true). */ static const char * -get_default_conf_file(void) +get_default_conf_file(int defaults_file) { -#ifdef MS_WINDOWS - static char path[MAX_PATH+1]; - strlcpy(path, get_windows_conf_root(), MAX_PATH); - strlcat(path,"\\torrc",MAX_PATH); - return path; +#ifdef _WIN32 + if (defaults_file) { + static char defaults_path[MAX_PATH+1]; + tor_snprintf(defaults_path, MAX_PATH, "%s\\torrc-defaults", + get_windows_conf_root()); + return defaults_path; + } else { + static char path[MAX_PATH+1]; + tor_snprintf(path, MAX_PATH, "%s\\torrc", + get_windows_conf_root()); + return path; + } #else - return (CONFDIR "/torrc"); + return defaults_file ? CONFDIR "/torrc-defaults" : CONFDIR "/torrc"; #endif } @@ -4006,7 +4486,7 @@ check_nickname_list(const char *lst, const char *name, char **msg) if (!lst) return 0; - sl = smartlist_create(); + sl = smartlist_new(); smartlist_split_string(sl, lst, ",", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK|SPLIT_STRIP_SPACE, 0); @@ -4024,37 +4504,63 @@ check_nickname_list(const char *lst, const char *name, char **msg) return r; } -/** Learn config file name from command line arguments, or use the default */ +/** Learn config file name from command line arguments, or use the default. + * + * If <b>defaults_file</b> is true, we're looking for torrc-defaults; + * otherwise, we're looking for the regular torrc_file. + * + * Set *<b>using_default_fname</b> to true if we're using the default + * configuration file name; or false if we've set it from the command line. + * + * Set *<b>ignore_missing_torrc</b> to true if we should ignore the resulting + * filename if it doesn't exist. + */ static char * find_torrc_filename(int argc, char **argv, - int *using_default_torrc, int *ignore_missing_torrc) + int defaults_file, + int *using_default_fname, int *ignore_missing_torrc) { char *fname=NULL; int i; + const char *fname_opt = defaults_file ? "--defaults-torrc" : "-f"; + const char *ignore_opt = defaults_file ? NULL : "--ignore-missing-torrc"; + + if (defaults_file) + *ignore_missing_torrc = 1; for (i = 1; i < argc; ++i) { - if (i < argc-1 && !strcmp(argv[i],"-f")) { + if (i < argc-1 && !strcmp(argv[i],fname_opt)) { if (fname) { - log(LOG_WARN, LD_CONFIG, "Duplicate -f options on command line."); + log(LOG_WARN, LD_CONFIG, "Duplicate %s options on command line.", + fname_opt); tor_free(fname); } fname = expand_filename(argv[i+1]); - *using_default_torrc = 0; + + { + char *absfname; + absfname = make_path_absolute(fname); + tor_free(fname); + fname = absfname; + } + + *using_default_fname = 0; ++i; - } else if (!strcmp(argv[i],"--ignore-missing-torrc")) { + } else if (ignore_opt && !strcmp(argv[i],ignore_opt)) { *ignore_missing_torrc = 1; } } - if (*using_default_torrc) { + if (*using_default_fname) { /* didn't find one, try CONFDIR */ - const char *dflt = get_default_conf_file(); + const char *dflt = get_default_conf_file(defaults_file); if (dflt && file_status(dflt) == FN_FILE) { fname = tor_strdup(dflt); } else { -#ifndef MS_WINDOWS - char *fn; - fn = expand_filename("~/.torrc"); +#ifndef _WIN32 + char *fn = NULL; + if (!defaults_file) + fn = expand_filename("~/.torrc"); if (fn && file_status(fn) == FN_FILE) { fname = fn; } else { @@ -4069,43 +4575,53 @@ find_torrc_filename(int argc, char **argv, return fname; } -/** Load torrc from disk, setting torrc_fname if successful */ +/** Load a configuration file from disk, setting torrc_fname or + * torrc_defaults_fname if successful. + * + * If <b>defaults_file</b> is true, load torrc-defaults; otherwise load torrc. + * + * Return the contents of the file on success, and NULL on failure. + */ static char * -load_torrc_from_disk(int argc, char **argv) +load_torrc_from_disk(int argc, char **argv, int defaults_file) { char *fname=NULL; char *cf = NULL; int using_default_torrc = 1; int ignore_missing_torrc = 0; + char **fname_var = defaults_file ? &torrc_defaults_fname : &torrc_fname; - fname = find_torrc_filename(argc, argv, + fname = find_torrc_filename(argc, argv, defaults_file, &using_default_torrc, &ignore_missing_torrc); tor_assert(fname); log(LOG_DEBUG, LD_CONFIG, "Opening config file \"%s\"", fname); - tor_free(torrc_fname); - torrc_fname = fname; + tor_free(*fname_var); + *fname_var = fname; /* Open config file */ if (file_status(fname) != FN_FILE || !(cf = read_file_to_str(fname,0,NULL))) { - if (using_default_torrc == 1 || ignore_missing_torrc ) { - log(LOG_NOTICE, LD_CONFIG, "Configuration file \"%s\" not present, " - "using reasonable defaults.", fname); + if (using_default_torrc == 1 || ignore_missing_torrc) { + if (!defaults_file) + log(LOG_NOTICE, LD_CONFIG, "Configuration file \"%s\" not present, " + "using reasonable defaults.", fname); tor_free(fname); /* sets fname to NULL */ - torrc_fname = NULL; + *fname_var = NULL; cf = tor_strdup(""); } else { log(LOG_WARN, LD_CONFIG, "Unable to open configuration file \"%s\".", fname); goto err; } + } else { + log(LOG_NOTICE, LD_CONFIG, "Read configuration file \"%s\".", fname); } return cf; err: tor_free(fname); - torrc_fname = NULL; + *fname_var = NULL; return NULL; } @@ -4116,8 +4632,9 @@ load_torrc_from_disk(int argc, char **argv) int options_init_from_torrc(int argc, char **argv) { - char *cf=NULL; - int i, retval, command; + char *cf=NULL, *cf_defaults=NULL; + int i, command; + int retval = -1; static char **backup_argv; static int backup_argc; char *command_arg = NULL; @@ -4176,24 +4693,24 @@ options_init_from_torrc(int argc, char **argv) if (command == CMD_HASH_PASSWORD) { cf = tor_strdup(""); } else { - cf = load_torrc_from_disk(argc, argv); + cf_defaults = load_torrc_from_disk(argc, argv, 1); + cf = load_torrc_from_disk(argc, argv, 0); if (!cf) goto err; } - retval = options_init_from_string(cf, command, command_arg, &errmsg); - tor_free(cf); - if (retval < 0) - goto err; - - return 0; + retval = options_init_from_string(cf_defaults, cf, command, command_arg, + &errmsg); err: + + tor_free(cf); + tor_free(cf_defaults); if (errmsg) { log(LOG_WARN,LD_CONFIG,"%s", errmsg); tor_free(errmsg); } - return -1; + return retval < 0 ? -1 : 0; } /** Load the options from the configuration in <b>cf</b>, validate @@ -4206,13 +4723,13 @@ options_init_from_torrc(int argc, char **argv) * * -4 for error while setting the new options */ setopt_err_t -options_init_from_string(const char *cf, +options_init_from_string(const char *cf_defaults, const char *cf, int command, const char *command_arg, char **msg) { - or_options_t *oldoptions, *newoptions; + or_options_t *oldoptions, *newoptions, *newdefaultoptions=NULL; config_line_t *cl; - int retval; + int retval, i; setopt_err_t err = SETOPT_ERR_MISC; tor_assert(msg); @@ -4225,17 +4742,24 @@ options_init_from_string(const char *cf, newoptions->command = command; newoptions->command_arg = command_arg; - /* get config lines, assign them */ - retval = config_get_lines(cf, &cl); - if (retval < 0) { - err = SETOPT_ERR_PARSE; - goto err; - } - retval = config_assign(&options_format, newoptions, cl, 0, 0, msg); - config_free_lines(cl); - if (retval < 0) { - err = SETOPT_ERR_PARSE; - goto err; + for (i = 0; i < 2; ++i) { + const char *body = i==0 ? cf_defaults : cf; + if (!body) + continue; + /* get config lines, assign them */ + retval = config_get_lines(body, &cl, 1); + if (retval < 0) { + err = SETOPT_ERR_PARSE; + goto err; + } + retval = config_assign(&options_format, newoptions, cl, 0, 0, msg); + config_free_lines(cl); + if (retval < 0) { + err = SETOPT_ERR_PARSE; + goto err; + } + if (i==0) + newdefaultoptions = options_dup(&options_format, newoptions); } /* Go through command-line variables too */ @@ -4260,9 +4784,9 @@ options_init_from_string(const char *cf, /* Change defaults. */ int i; for (i = 0; testing_tor_network_defaults[i].name; ++i) { - config_var_t *new_var = &testing_tor_network_defaults[i]; + const config_var_t *new_var = &testing_tor_network_defaults[i]; config_var_t *old_var = - config_find_option(&options_format, new_var->name); + config_find_option_mutable(&options_format, new_var->name); tor_assert(new_var); tor_assert(old_var); old_var->initvalue = new_var->initvalue; @@ -4270,6 +4794,8 @@ options_init_from_string(const char *cf, /* Clear newoptions and re-initialize them with new defaults. */ config_free(&options_format, newoptions); + config_free(&options_format, newdefaultoptions); + newdefaultoptions = NULL; newoptions = tor_malloc_zero(sizeof(or_options_t)); newoptions->_magic = OR_OPTIONS_MAGIC; options_init(newoptions); @@ -4277,17 +4803,26 @@ options_init_from_string(const char *cf, newoptions->command_arg = command_arg; /* Assign all options a second time. */ - retval = config_get_lines(cf, &cl); - if (retval < 0) { - err = SETOPT_ERR_PARSE; - goto err; - } - retval = config_assign(&options_format, newoptions, cl, 0, 0, msg); - config_free_lines(cl); - if (retval < 0) { - err = SETOPT_ERR_PARSE; - goto err; + for (i = 0; i < 2; ++i) { + const char *body = i==0 ? cf_defaults : cf; + if (!body) + continue; + /* get config lines, assign them */ + retval = config_get_lines(body, &cl, 1); + if (retval < 0) { + err = SETOPT_ERR_PARSE; + goto err; + } + retval = config_assign(&options_format, newoptions, cl, 0, 0, msg); + config_free_lines(cl); + if (retval < 0) { + err = SETOPT_ERR_PARSE; + goto err; + } + if (i==0) + newdefaultoptions = options_dup(&options_format, newoptions); } + /* Assign command-line variables a second time too */ retval = config_assign(&options_format, newoptions, global_cmdline_options, 0, 0, msg); if (retval < 0) { @@ -4311,11 +4846,14 @@ options_init_from_string(const char *cf, err = SETOPT_ERR_SETTING; goto err; /* frees and replaces old options */ } + config_free(&options_format, global_default_options); + global_default_options = newdefaultoptions; return SETOPT_OK; err: config_free(&options_format, newoptions); + config_free(&options_format, newdefaultoptions); if (*msg) { char *old_msg = *msg; tor_asprintf(msg, "Failed to parse/validate config: %s", old_msg); @@ -4327,51 +4865,106 @@ options_init_from_string(const char *cf, /** Return the location for our configuration file. */ const char * -get_torrc_fname(void) +get_torrc_fname(int defaults_fname) { - if (torrc_fname) - return torrc_fname; + const char *fname = defaults_fname ? torrc_defaults_fname : torrc_fname; + + if (fname) + return fname; else - return get_default_conf_file(); + return get_default_conf_file(defaults_fname); } /** Adjust the address map based on the MapAddress elements in the * configuration <b>options</b> */ -static void -config_register_addressmaps(or_options_t *options) +void +config_register_addressmaps(const or_options_t *options) { smartlist_t *elts; config_line_t *opt; - char *from, *to; + const char *from, *to, *msg; addressmap_clear_configured(); - elts = smartlist_create(); + elts = smartlist_new(); for (opt = options->AddressMap; opt; opt = opt->next) { smartlist_split_string(elts, opt->value, NULL, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 2); - if (smartlist_len(elts) >= 2) { - from = smartlist_get(elts,0); - to = smartlist_get(elts,1); - if (address_is_invalid_destination(to, 1)) { - log_warn(LD_CONFIG, - "Skipping invalid argument '%s' to MapAddress", to); - } else { - addressmap_register(from, tor_strdup(to), 0, ADDRMAPSRC_TORRC); - if (smartlist_len(elts)>2) { - log_warn(LD_CONFIG,"Ignoring extra arguments to MapAddress."); - } - } - } else { + if (smartlist_len(elts) < 2) { log_warn(LD_CONFIG,"MapAddress '%s' has too few arguments. Ignoring.", opt->value); + goto cleanup; } + + from = smartlist_get(elts,0); + to = smartlist_get(elts,1); + + if (to[0] == '.' || from[0] == '.') { + log_warn(LD_CONFIG,"MapAddress '%s' is ambiguous - address starts with a" + "'.'. Ignoring.",opt->value); + goto cleanup; + } + + if (addressmap_register_auto(from, to, 0, ADDRMAPSRC_TORRC, &msg) < 0) { + log_warn(LD_CONFIG,"MapAddress '%s' failed: %s. Ignoring.", opt->value, + msg); + goto cleanup; + } + + if (smartlist_len(elts) > 2) + log_warn(LD_CONFIG,"Ignoring extra arguments to MapAddress."); + + cleanup: SMARTLIST_FOREACH(elts, char*, cp, tor_free(cp)); smartlist_clear(elts); } smartlist_free(elts); } +/** As addressmap_register(), but detect the wildcarded status of "from" and + * "to", and do not steal a reference to <b>to</b>. */ +/* XXXX024 move to connection_edge.c */ +int +addressmap_register_auto(const char *from, const char *to, + time_t expires, + addressmap_entry_source_t addrmap_source, + const char **msg) +{ + int from_wildcard = 0, to_wildcard = 0; + + *msg = "whoops, forgot the error message"; + if (1) { + if (!strcmp(to, "*") || !strcmp(from, "*")) { + *msg = "can't remap from or to *"; + return -1; + } + /* Detect asterisks in expressions of type: '*.example.com' */ + if (!strncmp(from,"*.",2)) { + from += 2; + from_wildcard = 1; + } + if (!strncmp(to,"*.",2)) { + to += 2; + to_wildcard = 1; + } + + if (to_wildcard && !from_wildcard) { + *msg = "can only use wildcard (i.e. '*.') if 'from' address " + "uses wildcard also"; + return -1; + } + + if (address_is_invalid_destination(to, 1)) { + *msg = "destination is invalid"; + return -1; + } + + addressmap_register(from, tor_strdup(to), expires, addrmap_source, + from_wildcard, to_wildcard); + } + return 0; +} + /** * Initialize the logs based on the configuration file. */ @@ -4382,14 +4975,43 @@ options_init_logs(or_options_t *options, int validate_only) int ok; smartlist_t *elts; int daemon = -#ifdef MS_WINDOWS +#ifdef _WIN32 0; #else options->RunAsDaemon; #endif + if (options->LogTimeGranularity <= 0) { + log_warn(LD_CONFIG, "Log time granularity '%d' has to be positive.", + options->LogTimeGranularity); + return -1; + } else if (1000 % options->LogTimeGranularity != 0 && + options->LogTimeGranularity % 1000 != 0) { + int granularity = options->LogTimeGranularity; + if (granularity < 40) { + do granularity++; + while (1000 % granularity != 0); + } else if (granularity < 1000) { + granularity = 1000 / granularity; + while (1000 % granularity != 0) + granularity--; + granularity = 1000 / granularity; + } else { + granularity = 1000 * ((granularity / 1000) + 1); + } + log_warn(LD_CONFIG, "Log time granularity '%d' has to be either a " + "divisor or a multiple of 1 second. Changing to " + "'%d'.", + options->LogTimeGranularity, granularity); + if (!validate_only) + set_log_time_granularity(granularity); + } else { + if (!validate_only) + set_log_time_granularity(options->LogTimeGranularity); + } + ok = 1; - elts = smartlist_create(); + elts = smartlist_new(); for (opt = options->Logs; opt; opt = opt->next) { log_severity_list_t *severity; @@ -4477,20 +5099,38 @@ parse_bridge_line(const char *line, int validate_only) smartlist_t *items = NULL; int r; char *addrport=NULL, *fingerprint=NULL; + char *transport_name=NULL; + char *field1=NULL; tor_addr_t addr; uint16_t port = 0; char digest[DIGEST_LEN]; - items = smartlist_create(); + items = smartlist_new(); smartlist_split_string(items, line, NULL, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); if (smartlist_len(items) < 1) { log_warn(LD_CONFIG, "Too few arguments to Bridge line."); goto err; } - addrport = smartlist_get(items, 0); + + /* field1 is either a transport name or addrport */ + field1 = smartlist_get(items, 0); smartlist_del_keeporder(items, 0); - if (tor_addr_port_parse(addrport, &addr, &port)<0) { + + if (!(strstr(field1, ".") || strstr(field1, ":"))) { + /* new-style bridge line */ + transport_name = field1; + if (smartlist_len(items) < 1) { + log_warn(LD_CONFIG, "Too few items to Bridge line."); + goto err; + } + addrport = smartlist_get(items, 0); + smartlist_del_keeporder(items, 0); + } else { + addrport = field1; + } + + if (tor_addr_port_lookup(addrport, &addr, &port)<0) { log_warn(LD_CONFIG, "Error parsing Bridge address '%s'", addrport); goto err; } @@ -4514,26 +5154,282 @@ parse_bridge_line(const char *line, int validate_only) } if (!validate_only) { - log_debug(LD_DIR, "Bridge at %s:%d (%s)", fmt_addr(&addr), - (int)port, + log_debug(LD_DIR, "Bridge at %s:%d (transport: %s) (%s)", + fmt_addr(&addr), (int)port, + transport_name ? transport_name : "no transport", fingerprint ? fingerprint : "no key listed"); - bridge_add_from_config(&addr, port, fingerprint ? digest : NULL); + bridge_add_from_config(&addr, port, + fingerprint ? digest : NULL, transport_name); } r = 0; goto done; - err: + err: r = -1; - done: + done: SMARTLIST_FOREACH(items, char*, s, tor_free(s)); smartlist_free(items); tor_free(addrport); + tor_free(transport_name); tor_free(fingerprint); return r; } +/** Read the contents of a ClientTransportPlugin line from + * <b>line</b>. Return 0 if the line is well-formed, and -1 if it + * isn't. + * + * If <b>validate_only</b> is 0, and the line is well-formed: + * - If it's an external proxy line, add the transport described in the line to + * our internal transport list. + * - If it's a managed proxy line, launch the managed proxy. */ +static int +parse_client_transport_line(const char *line, int validate_only) +{ + smartlist_t *items = NULL; + int r; + char *field2=NULL; + + const char *transports=NULL; + smartlist_t *transport_list=NULL; + char *addrport=NULL; + tor_addr_t addr; + uint16_t port = 0; + int socks_ver=PROXY_NONE; + + /* managed proxy options */ + int is_managed=0; + char **proxy_argv=NULL; + char **tmp=NULL; + int proxy_argc,i; + + int line_length; + + items = smartlist_new(); + smartlist_split_string(items, line, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); + + line_length = smartlist_len(items); + if (line_length < 3) { + log_warn(LD_CONFIG, "Too few arguments on ClientTransportPlugin line."); + goto err; + } + + /* Get the first line element, split it to commas into + transport_list (in case it's multiple transports) and validate + the transport names. */ + transports = smartlist_get(items, 0); + transport_list = smartlist_new(); + smartlist_split_string(transport_list, transports, ",", + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + SMARTLIST_FOREACH_BEGIN(transport_list, const char *, transport_name) { + if (!string_is_C_identifier(transport_name)) { + log_warn(LD_CONFIG, "Transport name is not a C identifier (%s).", + transport_name); + goto err; + } + } SMARTLIST_FOREACH_END(transport_name); + + /* field2 is either a SOCKS version or "exec" */ + field2 = smartlist_get(items, 1); + + if (!strcmp(field2,"socks4")) { + socks_ver = PROXY_SOCKS4; + } else if (!strcmp(field2,"socks5")) { + socks_ver = PROXY_SOCKS5; + } else if (!strcmp(field2,"exec")) { + is_managed=1; + } else { + log_warn(LD_CONFIG, "Strange ClientTransportPlugin field '%s'.", + field2); + goto err; + } + + if (is_managed) { /* managed */ + if (!validate_only) { /* if we are not just validating, use the + rest of the line as the argv of the proxy + to be launched */ + proxy_argc = line_length-2; + tor_assert(proxy_argc > 0); + proxy_argv = tor_malloc_zero(sizeof(char*)*(proxy_argc+1)); + tmp = proxy_argv; + for (i=0;i<proxy_argc;i++) { /* store arguments */ + *tmp++ = smartlist_get(items, 2); + smartlist_del_keeporder(items, 2); + } + *tmp = NULL; /*terminated with NULL, just like execve() likes it*/ + + /* kickstart the thing */ + pt_kickstart_client_proxy(transport_list, proxy_argv); + } + } else { /* external */ + if (smartlist_len(transport_list) != 1) { + log_warn(LD_CONFIG, "You can't have an external proxy with " + "more than one transports."); + goto err; + } + + addrport = smartlist_get(items, 2); + + if (tor_addr_port_lookup(addrport, &addr, &port)<0) { + log_warn(LD_CONFIG, "Error parsing transport " + "address '%s'", addrport); + goto err; + } + if (!port) { + log_warn(LD_CONFIG, + "Transport address '%s' has no port.", addrport); + goto err; + } + + if (!validate_only) { + transport_add_from_config(&addr, port, smartlist_get(transport_list, 0), + socks_ver); + + log_info(LD_DIR, "Transport '%s' found at %s:%d", + transports, fmt_addr(&addr), (int)port); + } + } + + r = 0; + goto done; + + err: + r = -1; + + done: + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + if (transport_list) { + SMARTLIST_FOREACH(transport_list, char*, s, tor_free(s)); + smartlist_free(transport_list); + } + + return r; +} + +/** Read the contents of a ServerTransportPlugin line from + * <b>line</b>. Return 0 if the line is well-formed, and -1 if it + * isn't. + * If <b>validate_only</b> is 0, the line is well-formed, and it's a + * managed proxy line, launch the managed proxy. */ +static int +parse_server_transport_line(const char *line, int validate_only) +{ + smartlist_t *items = NULL; + int r; + const char *transports=NULL; + smartlist_t *transport_list=NULL; + char *type=NULL; + char *addrport=NULL; + tor_addr_t addr; + uint16_t port = 0; + + /* managed proxy options */ + int is_managed=0; + char **proxy_argv=NULL; + char **tmp=NULL; + int proxy_argc,i; + + int line_length; + + items = smartlist_new(); + smartlist_split_string(items, line, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); + + line_length = smartlist_len(items); + if (line_length < 3) { + log_warn(LD_CONFIG, "Too few arguments on ServerTransportPlugin line."); + goto err; + } + + /* Get the first line element, split it to commas into + transport_list (in case it's multiple transports) and validate + the transport names. */ + transports = smartlist_get(items, 0); + transport_list = smartlist_new(); + smartlist_split_string(transport_list, transports, ",", + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + SMARTLIST_FOREACH_BEGIN(transport_list, const char *, transport_name) { + if (!string_is_C_identifier(transport_name)) { + log_warn(LD_CONFIG, "Transport name is not a C identifier (%s).", + transport_name); + goto err; + } + } SMARTLIST_FOREACH_END(transport_name); + + type = smartlist_get(items, 1); + + if (!strcmp(type, "exec")) { + is_managed=1; + } else if (!strcmp(type, "proxy")) { + is_managed=0; + } else { + log_warn(LD_CONFIG, "Strange ServerTransportPlugin type '%s'", type); + goto err; + } + + if (is_managed) { /* managed */ + if (!validate_only) { + proxy_argc = line_length-2; + tor_assert(proxy_argc > 0); + proxy_argv = tor_malloc_zero(sizeof(char*)*(proxy_argc+1)); + tmp = proxy_argv; + + for (i=0;i<proxy_argc;i++) { /* store arguments */ + *tmp++ = smartlist_get(items, 2); + smartlist_del_keeporder(items, 2); + } + *tmp = NULL; /*terminated with NULL, just like execve() likes it*/ + + /* kickstart the thing */ + pt_kickstart_server_proxy(transport_list, proxy_argv); + } + } else { /* external */ + if (smartlist_len(transport_list) != 1) { + log_warn(LD_CONFIG, "You can't have an external proxy with " + "more than one transports."); + goto err; + } + + addrport = smartlist_get(items, 2); + + if (tor_addr_port_lookup(addrport, &addr, &port)<0) { + log_warn(LD_CONFIG, "Error parsing transport " + "address '%s'", addrport); + goto err; + } + if (!port) { + log_warn(LD_CONFIG, + "Transport address '%s' has no port.", addrport); + goto err; + } + + if (!validate_only) { + log_info(LD_DIR, "Server transport '%s' at %s:%d.", + transports, fmt_addr(&addr), (int)port); + } + } + + r = 0; + goto done; + + err: + r = -1; + + done: + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + if (transport_list) { + SMARTLIST_FOREACH(transport_list, char*, s, tor_free(s)); + smartlist_free(transport_list); + } + + return r; +} + /** Read the contents of a DirServer line from <b>line</b>. If * <b>validate_only</b> is 0, and the line is well-formed, and it * shares any bits with <b>required_type</b> or <b>required_type</b> @@ -4541,7 +5437,7 @@ parse_bridge_line(const char *line, int validate_only) * bits it's missing) as a valid authority. Return 0 on success, * or -1 if the line isn't well-formed or if we can't add it. */ static int -parse_dir_server_line(const char *line, authority_type_t required_type, +parse_dir_server_line(const char *line, dirinfo_type_t required_type, int validate_only) { smartlist_t *items = NULL; @@ -4550,10 +5446,10 @@ parse_dir_server_line(const char *line, authority_type_t required_type, uint16_t dir_port = 0, or_port = 0; char digest[DIGEST_LEN]; char v3_digest[DIGEST_LEN]; - authority_type_t type = V2_AUTHORITY; + dirinfo_type_t type = V2_DIRINFO; int is_not_hidserv_authority = 0, is_not_v2_authority = 0; - items = smartlist_create(); + items = smartlist_new(); smartlist_split_string(items, line, NULL, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); if (smartlist_len(items) < 1) { @@ -4571,13 +5467,13 @@ parse_dir_server_line(const char *line, authority_type_t required_type, if (TOR_ISDIGIT(flag[0])) break; if (!strcasecmp(flag, "v1")) { - type |= (V1_AUTHORITY | HIDSERV_AUTHORITY); + type |= (V1_DIRINFO | HIDSERV_DIRINFO); } else if (!strcasecmp(flag, "hs")) { - type |= HIDSERV_AUTHORITY; + type |= HIDSERV_DIRINFO; } else if (!strcasecmp(flag, "no-hs")) { is_not_hidserv_authority = 1; } else if (!strcasecmp(flag, "bridge")) { - type |= BRIDGE_AUTHORITY; + type |= BRIDGE_DIRINFO; } else if (!strcasecmp(flag, "no-v2")) { is_not_v2_authority = 1; } else if (!strcasecmpstart(flag, "orport=")) { @@ -4594,7 +5490,7 @@ parse_dir_server_line(const char *line, authority_type_t required_type, log_warn(LD_CONFIG, "Bad v3 identity digest '%s' on DirServer line", flag); } else { - type |= V3_AUTHORITY; + type |= V3_DIRINFO|EXTRAINFO_DIRINFO|MICRODESC_DIRINFO; } } else { log_warn(LD_CONFIG, "Unrecognized flag '%s' on DirServer line", @@ -4604,9 +5500,9 @@ parse_dir_server_line(const char *line, authority_type_t required_type, smartlist_del_keeporder(items, 0); } if (is_not_hidserv_authority) - type &= ~HIDSERV_AUTHORITY; + type &= ~HIDSERV_DIRINFO; if (is_not_v2_authority) - type &= ~V2_AUTHORITY; + type &= ~V2_DIRINFO; if (smartlist_len(items) < 2) { log_warn(LD_CONFIG, "Too few arguments to DirServer line."); @@ -4614,7 +5510,7 @@ parse_dir_server_line(const char *line, authority_type_t required_type, } addrport = smartlist_get(items, 0); smartlist_del_keeporder(items, 0); - if (parse_addr_port(LOG_WARN, addrport, &address, NULL, &dir_port)<0) { + if (addr_port_lookup(LOG_WARN, addrport, &address, NULL, &dir_port)<0) { log_warn(LD_CONFIG, "Error parsing DirServer address '%s'", addrport); goto err; } @@ -4634,7 +5530,7 @@ parse_dir_server_line(const char *line, authority_type_t required_type, * clause once Tor 0.1.2.17 is obsolete. */ log_warn(LD_CONFIG, "Dangerous dirserver line. To correct, erase your " "torrc file (%s), or reinstall Tor and use the default torrc.", - get_torrc_fname()); + get_torrc_fname(0)); goto err; } if (base16_decode(digest, DIGEST_LEN, fingerprint, HEX_DIGEST_LEN)<0) { @@ -4669,12 +5565,784 @@ parse_dir_server_line(const char *line, authority_type_t required_type, return r; } +/** Free all storage held in <b>port</b> */ +static void +port_cfg_free(port_cfg_t *port) +{ + tor_free(port); +} + +/** Warn for every port in <b>ports</b> that is on a publicly routable + * address. */ +static void +warn_nonlocal_client_ports(const smartlist_t *ports, const char *portname) +{ + SMARTLIST_FOREACH_BEGIN(ports, const port_cfg_t *, port) { + if (port->is_unix_addr) { + /* Unix sockets aren't accessible over a network. */ + } else if (!tor_addr_is_internal(&port->addr, 1)) { + log_warn(LD_CONFIG, "You specified a public address for %sPort. " + "Other people on the Internet might find your computer and " + "use it as an open proxy. Please don't allow this unless you " + "have a good reason.", portname); + } else if (!tor_addr_is_loopback(&port->addr)) { + log_notice(LD_CONFIG, "You configured a non-loopback address for " + "%sPort. This allows everybody on your local network to use " + "your machine as a proxy. Make sure this is what you wanted.", + portname); + } + } SMARTLIST_FOREACH_END(port); +} + +/** Given a list of port_cfg_t in <b>ports</b>, warn any controller port there + * is listening on any non-loopback address. If <b>forbid</b> is true, + * then emit a stronger warning and remove the port from the list. + */ +static void +warn_nonlocal_controller_ports(smartlist_t *ports, unsigned forbid) +{ + int warned = 0; + SMARTLIST_FOREACH_BEGIN(ports, port_cfg_t *, port) { + if (port->type != CONN_TYPE_CONTROL_LISTENER) + continue; + if (port->is_unix_addr) + continue; + if (!tor_addr_is_loopback(&port->addr)) { + if (forbid) { + if (!warned) + log_warn(LD_CONFIG, + "You have a ControlPort set to accept " + "unauthenticated connections from a non-local address. " + "This means that programs not running on your computer " + "can reconfigure your Tor, without even having to guess a " + "password. That's so bad that I'm closing your ControlPort " + "for you. If you need to control your Tor remotely, try " + "enabling authentication and using a tool like stunnel or " + "ssh to encrypt remote access."); + warned = 1; + port_cfg_free(port); + SMARTLIST_DEL_CURRENT(ports, port); + } else { + log_warn(LD_CONFIG, "You have a ControlPort set to accept " + "connections from a non-local address. This means that " + "programs not running on your computer can reconfigure your " + "Tor. That's pretty bad, since the controller " + "protocol isn't encrypted! Maybe you should just listen on " + "127.0.0.1 and use a tool like stunnel or ssh to encrypt " + "remote connections to your control port."); + return; /* No point in checking the rest */ + } + } + } SMARTLIST_FOREACH_END(port); +} + +#define CL_PORT_NO_OPTIONS (1u<<0) +#define CL_PORT_WARN_NONLOCAL (1u<<1) +#define CL_PORT_ALLOW_EXTRA_LISTENADDR (1u<<2) +#define CL_PORT_SERVER_OPTIONS (1u<<3) +#define CL_PORT_FORBID_NONLOCAL (1u<<4) + +/** + * Parse port configuration for a single port type. + * + * Read entries of the "FooPort" type from the list <b>ports</b>, and + * entries of the "FooListenAddress" type from the list + * <b>listenaddrs</b>. Two syntaxes are supported: a legacy syntax + * where FooPort is at most a single entry containing a port number and + * where FooListenAddress has any number of address:port combinations; + * and a new syntax where there are no FooListenAddress entries and + * where FooPort can have any number of entries of the format + * "[Address:][Port] IsolationOptions". + * + * In log messages, describe the port type as <b>portname</b>. + * + * If no address is specified, default to <b>defaultaddr</b>. If no + * FooPort is given, default to defaultport (if 0, there is no default). + * + * If CL_PORT_NO_OPTIONS is set in <b>flags</b>, do not allow stream + * isolation options in the FooPort entries. + * + * If CL_PORT_WARN_NONLOCAL is set in <b>flags</b>, warn if any of the + * ports are not on a local address. If CL_PORT_FORBID_NONLOCAL is set, + * this is a contrl port with no password set: don't even allow it. + * + * Unless CL_PORT_ALLOW_EXTRA_LISTENADDR is set in <b>flags</b>, warn + * if FooListenAddress is set but FooPort is 0. + * + * If CL_PORT_SERVER_OPTIONS is set in <b>flags</b>, do not allow stream + * isolation options in the FooPort entries; instead allow the + * server-port option set. + * + * On success, if <b>out</b> is given, add a new port_cfg_t entry to + * <b>out</b> for every port that the client should listen on. Return 0 + * on success, -1 on failure. + */ +static int +parse_port_config(smartlist_t *out, + const config_line_t *ports, + const config_line_t *listenaddrs, + const char *portname, + int listener_type, + const char *defaultaddr, + int defaultport, + unsigned flags) +{ + smartlist_t *elts; + int retval = -1; + const unsigned is_control = (listener_type == CONN_TYPE_CONTROL_LISTENER); + const unsigned allow_no_options = flags & CL_PORT_NO_OPTIONS; + const unsigned use_server_options = flags & CL_PORT_SERVER_OPTIONS; + const unsigned warn_nonlocal = flags & CL_PORT_WARN_NONLOCAL; + const unsigned forbid_nonlocal = flags & CL_PORT_FORBID_NONLOCAL; + const unsigned allow_spurious_listenaddr = + flags & CL_PORT_ALLOW_EXTRA_LISTENADDR; + int got_zero_port=0, got_nonzero_port=0; + + /* FooListenAddress is deprecated; let's make it work like it used to work, + * though. */ + if (listenaddrs) { + int mainport = defaultport; + + if (ports && ports->next) { + log_warn(LD_CONFIG, "%sListenAddress can't be used when there are " + "multiple %sPort lines", portname, portname); + return -1; + } else if (ports) { + if (!strcmp(ports->value, "auto")) { + mainport = CFG_AUTO_PORT; + } else { + int ok; + mainport = (int)tor_parse_long(ports->value, 10, 0, 65535, &ok, NULL); + if (!ok) { + log_warn(LD_CONFIG, "%sListenAddress can only be used with a single " + "%sPort with value \"auto\" or 1-65535 and no options set.", + portname, portname); + return -1; + } + } + } + + if (mainport == 0) { + if (allow_spurious_listenaddr) + return 1; /*DOCDOC*/ + log_warn(LD_CONFIG, "%sPort must be defined if %sListenAddress is used", + portname, portname); + return -1; + } + + if (use_server_options && out) { + /* Add a no_listen port. */ + port_cfg_t *cfg = tor_malloc_zero(sizeof(port_cfg_t)); + cfg->type = listener_type; + cfg->port = mainport; + tor_addr_make_unspec(&cfg->addr); /* Server ports default to 0.0.0.0 */ + cfg->no_listen = 1; + cfg->ipv4_only = 1; + smartlist_add(out, cfg); + } + + for (; listenaddrs; listenaddrs = listenaddrs->next) { + tor_addr_t addr; + uint16_t port = 0; + if (tor_addr_port_lookup(listenaddrs->value, &addr, &port) < 0) { + log_warn(LD_CONFIG, "Unable to parse %sListenAddress '%s'", + portname, listenaddrs->value); + return -1; + } + if (out) { + port_cfg_t *cfg = tor_malloc_zero(sizeof(port_cfg_t)); + cfg->type = listener_type; + cfg->port = port ? port : mainport; + tor_addr_copy(&cfg->addr, &addr); + cfg->session_group = SESSION_GROUP_UNSET; + cfg->isolation_flags = ISO_DEFAULT; + cfg->no_advertise = 1; + smartlist_add(out, cfg); + } + } + + if (warn_nonlocal && out) { + if (is_control) + warn_nonlocal_controller_ports(out, forbid_nonlocal); + else + warn_nonlocal_client_ports(out, portname); + } + return 0; + } /* end if (listenaddrs) */ + + /* No ListenAddress lines. If there's no FooPort, then maybe make a default + * one. */ + if (! ports) { + if (defaultport && out) { + port_cfg_t *cfg = tor_malloc_zero(sizeof(port_cfg_t)); + cfg->type = listener_type; + cfg->port = defaultport; + tor_addr_parse(&cfg->addr, defaultaddr); + cfg->session_group = SESSION_GROUP_UNSET; + cfg->isolation_flags = ISO_DEFAULT; + smartlist_add(out, cfg); + } + return 0; + } + + /* At last we can actually parse the FooPort lines. The syntax is: + * [Addr:](Port|auto) [Options].*/ + elts = smartlist_new(); + + for (; ports; ports = ports->next) { + tor_addr_t addr; + int port; + int sessiongroup = SESSION_GROUP_UNSET; + unsigned isolation = ISO_DEFAULT; + + char *addrport; + uint16_t ptmp=0; + int ok; + int no_listen = 0, no_advertise = 0, all_addrs = 0, + ipv4_only = 0, ipv6_only = 0; + + smartlist_split_string(elts, ports->value, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + if (smartlist_len(elts) == 0) { + log_warn(LD_CONFIG, "Invalid %sPort line with no value", portname); + goto err; + } + + if (allow_no_options && smartlist_len(elts) > 1) { + log_warn(LD_CONFIG, "Too many options on %sPort line", portname); + goto err; + } + + /* Now parse the addr/port value */ + addrport = smartlist_get(elts, 0); + if (!strcmp(addrport, "auto")) { + port = CFG_AUTO_PORT; + tor_addr_parse(&addr, defaultaddr); + } else if (!strcasecmpend(addrport, ":auto")) { + char *addrtmp = tor_strndup(addrport, strlen(addrport)-5); + port = CFG_AUTO_PORT; + if (tor_addr_port_lookup(addrtmp, &addr, &ptmp)<0 || ptmp) { + log_warn(LD_CONFIG, "Invalid address '%s' for %sPort", + escaped(addrport), portname); + tor_free(addrtmp); + goto err; + } + } else { + /* Try parsing integer port before address, because, who knows? + "9050" might be a valid address. */ + port = (int) tor_parse_long(addrport, 10, 0, 65535, &ok, NULL); + if (ok) { + tor_addr_parse(&addr, defaultaddr); + } else if (tor_addr_port_lookup(addrport, &addr, &ptmp) == 0) { + if (ptmp == 0) { + log_warn(LD_CONFIG, "%sPort line has address but no port", portname); + goto err; + } + port = ptmp; + } else { + log_warn(LD_CONFIG, "Couldn't parse address '%s' for %sPort", + escaped(addrport), portname); + goto err; + } + } + + /* Now parse the rest of the options, if any. */ + if (use_server_options) { + /* This is a server port; parse advertising options */ + SMARTLIST_FOREACH_BEGIN(elts, char *, elt) { + if (elt_sl_idx == 0) + continue; /* Skip addr:port */ + + if (!strcasecmp(elt, "NoAdvertise")) { + no_advertise = 1; + } else if (!strcasecmp(elt, "NoListen")) { + no_listen = 1; +#if 0 + /* not implemented yet. */ + } else if (!strcasecmp(elt, "AllAddrs")) { + + all_addrs = 1; +#endif + } else if (!strcasecmp(elt, "IPv4Only")) { + ipv4_only = 1; + } else if (!strcasecmp(elt, "IPv6Only")) { + ipv6_only = 1; + } else { + log_warn(LD_CONFIG, "Unrecognized %sPort option '%s'", + portname, escaped(elt)); + } + } SMARTLIST_FOREACH_END(elt); + + if (no_advertise && no_listen) { + log_warn(LD_CONFIG, "Tried to set both NoListen and NoAdvertise " + "on %sPort line '%s'", + portname, escaped(ports->value)); + goto err; + } + if (ipv4_only && ipv6_only) { + log_warn(LD_CONFIG, "Tried to set both IPv4Only and IPv6Only " + "on %sPort line '%s'", + portname, escaped(ports->value)); + goto err; + } + if (ipv4_only && tor_addr_family(&addr) == AF_INET6) { + log_warn(LD_CONFIG, "Could not interpret %sPort address as IPv6", + portname); + goto err; + } + if (ipv6_only && tor_addr_family(&addr) == AF_INET) { + log_warn(LD_CONFIG, "Could not interpret %sPort address as IPv4", + portname); + goto err; + } + } else { + /* This is a client port; parse isolation options */ + SMARTLIST_FOREACH_BEGIN(elts, char *, elt) { + int no = 0, isoflag = 0; + const char *elt_orig = elt; + if (elt_sl_idx == 0) + continue; /* Skip addr:port */ + if (!strcasecmpstart(elt, "SessionGroup=")) { + int group = (int)tor_parse_long(elt+strlen("SessionGroup="), + 10, 0, INT_MAX, &ok, NULL); + if (!ok) { + log_warn(LD_CONFIG, "Invalid %sPort option '%s'", + portname, escaped(elt)); + goto err; + } + if (sessiongroup >= 0) { + log_warn(LD_CONFIG, "Multiple SessionGroup options on %sPort", + portname); + goto err; + } + sessiongroup = group; + continue; + } + + if (!strcasecmpstart(elt, "No")) { + no = 1; + elt += 2; + } + if (!strcasecmpend(elt, "s")) + elt[strlen(elt)-1] = '\0'; /* kill plurals. */ + + if (!strcasecmp(elt, "IsolateDestPort")) { + isoflag = ISO_DESTPORT; + } else if (!strcasecmp(elt, "IsolateDestAddr")) { + isoflag = ISO_DESTADDR; + } else if (!strcasecmp(elt, "IsolateSOCKSAuth")) { + isoflag = ISO_SOCKSAUTH; + } else if (!strcasecmp(elt, "IsolateClientProtocol")) { + isoflag = ISO_CLIENTPROTO; + } else if (!strcasecmp(elt, "IsolateClientAddr")) { + isoflag = ISO_CLIENTADDR; + } else { + log_warn(LD_CONFIG, "Unrecognized %sPort option '%s'", + portname, escaped(elt_orig)); + } + + if (no) { + isolation &= ~isoflag; + } else { + isolation |= isoflag; + } + } SMARTLIST_FOREACH_END(elt); + } + + if (port) + got_nonzero_port = 1; + else + got_zero_port = 1; + + if (out && port) { + port_cfg_t *cfg = tor_malloc_zero(sizeof(port_cfg_t)); + tor_addr_copy(&cfg->addr, &addr); + cfg->port = port; + cfg->type = listener_type; + cfg->isolation_flags = isolation; + cfg->session_group = sessiongroup; + cfg->no_advertise = no_advertise; + cfg->no_listen = no_listen; + cfg->all_addrs = all_addrs; + cfg->ipv4_only = ipv4_only; + cfg->ipv6_only = ipv6_only; + + smartlist_add(out, cfg); + } + SMARTLIST_FOREACH(elts, char *, cp, tor_free(cp)); + smartlist_clear(elts); + } + + if (warn_nonlocal && out) { + if (is_control) + warn_nonlocal_controller_ports(out, forbid_nonlocal); + else + warn_nonlocal_client_ports(out, portname); + } + + if (got_zero_port && got_nonzero_port) { + log_warn(LD_CONFIG, "You specified a nonzero %sPort along with '%sPort 0' " + "in the same configuration. Did you mean to disable %sPort or " + "not?", portname, portname, portname); + goto err; + } + + retval = 0; + err: + SMARTLIST_FOREACH(elts, char *, cp, tor_free(cp)); + smartlist_free(elts); + return retval; +} + +/** Parse a list of config_line_t for an AF_UNIX unix socket listener option + * from <b>cfg</b> and add them to <b>out</b>. No fancy options are + * supported: the line contains nothing but the path to the AF_UNIX socket. */ +static int +parse_unix_socket_config(smartlist_t *out, const config_line_t *cfg, + int listener_type) +{ + + if (!out) + return 0; + + for ( ; cfg; cfg = cfg->next) { + size_t len = strlen(cfg->value); + port_cfg_t *port = tor_malloc_zero(sizeof(port_cfg_t) + len + 1); + port->is_unix_addr = 1; + memcpy(port->unix_addr, cfg->value, len+1); + port->type = listener_type; + smartlist_add(out, port); + } + + return 0; +} + +/** Return the number of ports which are actually going to listen with type + * <b>listenertype</b>. Do not count no_listen ports. Do not count unix + * sockets. */ +static int +count_real_listeners(const smartlist_t *ports, int listenertype) +{ + int n = 0; + SMARTLIST_FOREACH_BEGIN(ports, port_cfg_t *, port) { + if (port->no_listen || port->is_unix_addr) + continue; + if (port->type != listenertype) + continue; + ++n; + } SMARTLIST_FOREACH_END(port); + return n; +} + +/** Parse all client port types (Socks, DNS, Trans, NATD) from + * <b>options</b>. On success, set *<b>n_ports_out</b> to the number + * of ports that are listed, update the *Port_set values in + * <b>options</b>, and return 0. On failure, set *<b>msg</b> to a + * description of the problem and return -1. + * + * If <b>validate_only</b> is false, set configured_client_ports to the + * new list of ports parsed from <b>options</b>. + **/ +static int +parse_ports(or_options_t *options, int validate_only, + char **msg, int *n_ports_out) +{ + smartlist_t *ports; + int retval = -1; + + ports = smartlist_new(); + + *n_ports_out = 0; + + if (parse_port_config(ports, + options->SocksPort_lines, options->SocksListenAddress, + "Socks", CONN_TYPE_AP_LISTENER, + "127.0.0.1", 9050, + CL_PORT_WARN_NONLOCAL|CL_PORT_ALLOW_EXTRA_LISTENADDR) < 0) { + *msg = tor_strdup("Invalid SocksPort/SocksListenAddress configuration"); + goto err; + } + if (parse_port_config(ports, + options->DNSPort_lines, options->DNSListenAddress, + "DNS", CONN_TYPE_AP_DNS_LISTENER, + "127.0.0.1", 0, + CL_PORT_WARN_NONLOCAL) < 0) { + *msg = tor_strdup("Invalid DNSPort/DNSListenAddress configuration"); + goto err; + } + if (parse_port_config(ports, + options->TransPort_lines, options->TransListenAddress, + "Trans", CONN_TYPE_AP_TRANS_LISTENER, + "127.0.0.1", 0, + CL_PORT_WARN_NONLOCAL) < 0) { + *msg = tor_strdup("Invalid TransPort/TransListenAddress configuration"); + goto err; + } + if (parse_port_config(ports, + options->NATDPort_lines, options->NATDListenAddress, + "NATD", CONN_TYPE_AP_NATD_LISTENER, + "127.0.0.1", 0, + CL_PORT_WARN_NONLOCAL) < 0) { + *msg = tor_strdup("Invalid NatdPort/NatdListenAddress configuration"); + goto err; + } + { + unsigned control_port_flags = CL_PORT_NO_OPTIONS | CL_PORT_WARN_NONLOCAL; + const int any_passwords = (options->HashedControlPassword || + options->HashedControlSessionPassword || + options->CookieAuthentication); + if (! any_passwords) + control_port_flags |= CL_PORT_FORBID_NONLOCAL; + + if (parse_port_config(ports, + options->ControlPort_lines, + options->ControlListenAddress, + "Control", CONN_TYPE_CONTROL_LISTENER, + "127.0.0.1", 0, + control_port_flags) < 0) { + *msg = tor_strdup("Invalid ControlPort/ControlListenAddress " + "configuration"); + goto err; + } + if (parse_unix_socket_config(ports, + options->ControlSocket, + CONN_TYPE_CONTROL_LISTENER) < 0) { + *msg = tor_strdup("Invalid ControlSocket configuration"); + goto err; + } + } + if (! options->ClientOnly) { + if (parse_port_config(ports, + options->ORPort_lines, options->ORListenAddress, + "OR", CONN_TYPE_OR_LISTENER, + "0.0.0.0", 0, + CL_PORT_SERVER_OPTIONS) < 0) { + *msg = tor_strdup("Invalid ORPort/ORListenAddress configuration"); + goto err; + } + if (parse_port_config(ports, + options->DirPort_lines, options->DirListenAddress, + "Dir", CONN_TYPE_DIR_LISTENER, + "0.0.0.0", 0, + CL_PORT_SERVER_OPTIONS) < 0) { + *msg = tor_strdup("Invalid DirPort/DirListenAddress configuration"); + goto err; + } + } + + if (check_server_ports(ports, options) < 0) { + *msg = tor_strdup("Misconfigured server ports"); + goto err; + } + + *n_ports_out = smartlist_len(ports); + + retval = 0; + + /* Update the *Port_set options. The !! here is to force a boolean out of + an integer. */ + options->ORPort_set = + !! count_real_listeners(ports, CONN_TYPE_OR_LISTENER); + options->SocksPort_set = + !! count_real_listeners(ports, CONN_TYPE_AP_LISTENER); + options->TransPort_set = + !! count_real_listeners(ports, CONN_TYPE_AP_TRANS_LISTENER); + options->NATDPort_set = + !! count_real_listeners(ports, CONN_TYPE_AP_NATD_LISTENER); + options->ControlPort_set = + !! count_real_listeners(ports, CONN_TYPE_CONTROL_LISTENER); + options->DirPort_set = + !! count_real_listeners(ports, CONN_TYPE_DIR_LISTENER); + options->DNSPort_set = + !! count_real_listeners(ports, CONN_TYPE_AP_DNS_LISTENER); + + if (!validate_only) { + if (configured_ports) { + SMARTLIST_FOREACH(configured_ports, + port_cfg_t *, p, port_cfg_free(p)); + smartlist_free(configured_ports); + } + configured_ports = ports; + ports = NULL; /* prevent free below. */ + } + + err: + if (ports) { + SMARTLIST_FOREACH(ports, port_cfg_t *, p, port_cfg_free(p)); + smartlist_free(ports); + } + return retval; +} + +/** Given a list of <b>port_cfg_t</b> in <b>ports</b>, check them for internal + * consistency and warn as appropriate. */ +static int +check_server_ports(const smartlist_t *ports, + const or_options_t *options) +{ + int n_orport_advertised = 0; + int n_orport_advertised_ipv4 = 0; + int n_orport_listeners = 0; + int n_dirport_advertised = 0; + int n_dirport_listeners = 0; + int n_low_port = 0; + int r = 0; + + SMARTLIST_FOREACH_BEGIN(ports, const port_cfg_t *, port) { + if (port->type == CONN_TYPE_DIR_LISTENER) { + if (! port->no_advertise) + ++n_dirport_advertised; + if (! port->no_listen) + ++n_dirport_listeners; + } else if (port->type == CONN_TYPE_OR_LISTENER) { + if (! port->no_advertise) { + ++n_orport_advertised; + if (tor_addr_family(&port->addr) == AF_INET || + (tor_addr_family(&port->addr) == AF_UNSPEC && !port->ipv6_only)) + ++n_orport_advertised_ipv4; + } + if (! port->no_listen) + ++n_orport_listeners; + } else { + continue; + } +#ifndef _WIN32 + if (!port->no_advertise && port->port < 1024) + ++n_low_port; +#endif + } SMARTLIST_FOREACH_END(port); + + if (n_orport_advertised && !n_orport_listeners) { + log_warn(LD_CONFIG, "We are advertising an ORPort, but not actually " + "listening on one."); + r = -1; + } + if (n_dirport_advertised && !n_dirport_listeners) { + log_warn(LD_CONFIG, "We are advertising a DirPort, but not actually " + "listening on one."); + r = -1; + } + if (n_dirport_advertised > 1) { + log_warn(LD_CONFIG, "Can't advertise more than one DirPort."); + r = -1; + } + if (n_orport_advertised && !n_orport_advertised_ipv4 && + !options->BridgeRelay) { + log_warn(LD_CONFIG, "Configured non-bridge only to listen on an IPv6 " + "address."); + r = -1; + } + + if (n_low_port && options->AccountingMax) { + log(LOG_WARN, LD_CONFIG, + "You have set AccountingMax to use hibernation. You have also " + "chosen a low DirPort or OrPort. This combination can make Tor stop " + "working when it tries to re-attach the port after a period of " + "hibernation. Please choose a different port or turn off " + "hibernation unless you know this combination will work on your " + "platform."); + } + + return r; +} + +/** Return a list of port_cfg_t for client ports parsed from the + * options. */ +const smartlist_t * +get_configured_ports(void) +{ + if (!configured_ports) + configured_ports = smartlist_new(); + return configured_ports; +} + +/** Return an address:port string representation of the address + * where the first <b>listener_type</b> listener waits for + * connections. Return NULL if we couldn't find a listener. The + * string is allocated on the heap and it's the responsibility of the + * caller to free it after use. + * + * This function is meant to be used by the pluggable transport proxy + * spawning code, please make sure that it fits your purposes before + * using it. */ +char * +get_first_listener_addrport_string(int listener_type) +{ + static const char *ipv4_localhost = "127.0.0.1"; + static const char *ipv6_localhost = "[::1]"; + const char *address; + uint16_t port; + char *string = NULL; + + if (!configured_ports) + return NULL; + + SMARTLIST_FOREACH_BEGIN(configured_ports, const port_cfg_t *, cfg) { + if (cfg->no_listen) + continue; + + if (cfg->type == listener_type && + tor_addr_family(&cfg->addr) != AF_UNSPEC) { + + /* We found the first listener of the type we are interested in! */ + + /* If a listener is listening on INADDR_ANY, assume that it's + also listening on 127.0.0.1, and point the transport proxy + there: */ + if (tor_addr_is_null(&cfg->addr)) + address = tor_addr_is_v4(&cfg->addr) ? ipv4_localhost : ipv6_localhost; + else + address = fmt_and_decorate_addr(&cfg->addr); + + /* If a listener is configured with port 'auto', we are forced + to iterate all listener connections and find out in which + port it ended up listening: */ + if (cfg->port == CFG_AUTO_PORT) { + port = router_get_active_listener_port_by_type(listener_type); + if (!port) + return NULL; + } else { + port = cfg->port; + } + + tor_asprintf(&string, "%s:%u", address, port); + + return string; + } + + } SMARTLIST_FOREACH_END(cfg); + + return NULL; +} + +/** Return the first advertised port of type <b>listener_type</b> in + <b>address_family</b>. */ +int +get_first_advertised_port_by_type_af(int listener_type, int address_family) +{ + if (!configured_ports) + return 0; + SMARTLIST_FOREACH_BEGIN(configured_ports, const port_cfg_t *, cfg) { + if (cfg->type == listener_type && + !cfg->no_advertise && + (tor_addr_family(&cfg->addr) == address_family || + tor_addr_family(&cfg->addr) == AF_UNSPEC)) { + if (tor_addr_family(&cfg->addr) != AF_UNSPEC || + (address_family == AF_INET && !cfg->ipv6_only) || + (address_family == AF_INET6 && !cfg->ipv4_only)) { + return cfg->port; + } + } + } SMARTLIST_FOREACH_END(cfg); + return 0; +} + /** Adjust the value of options->DataDirectory, or fill it in if it's * absent. Return 0 on success, -1 on failure. */ static int normalize_data_directory(or_options_t *options) { -#ifdef MS_WINDOWS +#ifdef _WIN32 char *p; if (options->DataDirectory) return 0; /* all set */ @@ -4740,7 +6408,7 @@ validate_data_directory(or_options_t *options) * doesn't begin with GENERATED_FILE_PREFIX, rename it. Otherwise * replace it. Return 0 on success, -1 on failure. */ static int -write_configuration_file(const char *fname, or_options_t *options) +write_configuration_file(const char *fname, const or_options_t *options) { char *old_val=NULL, *new_val=NULL, *new_conf=NULL; int rename_old = 0, r; @@ -4775,18 +6443,12 @@ write_configuration_file(const char *fname, or_options_t *options) if (rename_old) { int i = 1; - size_t fn_tmp_len = strlen(fname)+32; - char *fn_tmp; - tor_assert(fn_tmp_len > strlen(fname)); /*check for overflow*/ - fn_tmp = tor_malloc(fn_tmp_len); + char *fn_tmp = NULL; while (1) { - if (tor_snprintf(fn_tmp, fn_tmp_len, "%s.orig.%d", fname, i)<0) { - log_warn(LD_BUG, "tor_snprintf failed inexplicably"); - tor_free(fn_tmp); - goto err; - } + tor_asprintf(&fn_tmp, "%s.orig.%d", fname, i); if (file_status(fn_tmp) == FN_NOENT) break; + tor_free(fn_tmp); ++i; } log_notice(LD_CONFIG, "Renaming old configuration file to \"%s\"", fn_tmp); @@ -4825,7 +6487,7 @@ options_save_current(void) * If we try falling back to datadirectory or something, we have a better * chance of saving the configuration, but a better chance of doing * something the user never expected. */ - return write_configuration_file(get_torrc_fname(), get_options()); + return write_configuration_file(get_torrc_fname(0), get_options()); } /** Mapping from a unit name to a multiplier for converting that unit into a @@ -4881,6 +6543,26 @@ static struct unit_table_t time_units[] = { { NULL, 0 }, }; +/** Table to map the names of time units to the number of milliseconds + * they contain. */ +static struct unit_table_t time_msec_units[] = { + { "", 1 }, + { "msec", 1 }, + { "millisecond", 1 }, + { "milliseconds", 1 }, + { "second", 1000 }, + { "seconds", 1000 }, + { "minute", 60*1000 }, + { "minutes", 60*1000 }, + { "hour", 60*60*1000 }, + { "hours", 60*60*1000 }, + { "day", 24*60*60*1000 }, + { "days", 24*60*60*1000 }, + { "week", 7*24*60*60*1000 }, + { "weeks", 7*24*60*60*1000 }, + { NULL, 0 }, +}; + /** Parse a string <b>val</b> containing a number, zero or more * spaces, and an optional unit string. If the unit appears in the * table <b>u</b>, then multiply the number by the unit multiplier. @@ -4944,6 +6626,25 @@ config_parse_memunit(const char *s, int *ok) return u; } +/** Parse a string in the format "number unit", where unit is a unit of + * time in milliseconds. On success, set *<b>ok</b> to true and return + * the number of milliseconds in the provided interval. Otherwise, set + * *<b>ok</b> to 0 and return -1. */ +static int +config_parse_msec_interval(const char *s, int *ok) +{ + uint64_t r; + r = config_parse_units(s, time_msec_units, ok); + if (!ok) + return -1; + if (r > INT_MAX) { + log_warn(LD_CONFIG, "Msec interval '%s' is too long", s); + *ok = 0; + return -1; + } + return (int)r; +} + /** Parse a string in the format "number unit", where unit is a unit of time. * On success, set *<b>ok</b> to true and return the number of seconds in * the provided interval. Otherwise, set *<b>ok</b> to 0 and return -1. @@ -4963,13 +6664,29 @@ config_parse_interval(const char *s, int *ok) return (int)r; } +/** Return the number of cpus configured in <b>options</b>. If we are + * told to auto-detect the number of cpus, return the auto-detected number. */ +int +get_num_cpus(const or_options_t *options) +{ + if (options->NumCPUs == 0) { + int n = compute_num_cpus(); + return (n >= 1) ? n : 1; + } else { + return options->NumCPUs; + } +} + /** * Initialize the libevent library. */ static void -init_libevent(void) +init_libevent(const or_options_t *options) { const char *badness=NULL; + tor_libevent_cfg cfg; + + tor_assert(options); configure_libevent_logging(); /* If the kernel complains that some method (say, epoll) doesn't @@ -4979,12 +6696,17 @@ init_libevent(void) tor_check_libevent_header_compatibility(); - tor_libevent_initialize(); + memset(&cfg, 0, sizeof(cfg)); + cfg.disable_iocp = options->DisableIOCP; + cfg.num_cpus = get_num_cpus(options); + cfg.msec_per_tick = options->TokenBucketRefillInterval; + + tor_libevent_initialize(&cfg); suppress_libevent_log_msg(NULL); tor_check_libevent_version(tor_libevent_get_method(), - get_options()->ORPort != 0, + server_mode(get_options()), &badness); if (badness) { const char *v = tor_libevent_get_version_str(); @@ -5018,7 +6740,7 @@ get_or_state(void) * Note: Consider using the get_datadir_fname* macros in or.h. */ char * -options_get_datadir_fname2_suffix(or_options_t *options, +options_get_datadir_fname2_suffix(const or_options_t *options, const char *sub1, const char *sub2, const char *suffix) { @@ -5053,6 +6775,69 @@ options_get_datadir_fname2_suffix(or_options_t *options, return fname; } +/** Return true if <b>line</b> is a valid state TransportProxy line. + * Return false otherwise. */ +static int +state_transport_line_is_valid(const char *line) +{ + smartlist_t *items = NULL; + char *addrport=NULL; + tor_addr_t addr; + uint16_t port = 0; + int r; + + items = smartlist_new(); + smartlist_split_string(items, line, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); + + if (smartlist_len(items) != 2) { + log_warn(LD_CONFIG, "state: Not enough arguments in TransportProxy line."); + goto err; + } + + addrport = smartlist_get(items, 1); + if (tor_addr_port_lookup(addrport, &addr, &port) < 0) { + log_warn(LD_CONFIG, "state: Could not parse addrport."); + goto err; + } + + if (!port) { + log_warn(LD_CONFIG, "state: Transport line did not contain port."); + goto err; + } + + r = 1; + goto done; + + err: + r = 0; + + done: + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + return r; +} + +/** Return 0 if all TransportProxy lines in <b>state</b> are well + * formed. Otherwise, return -1. */ +static int +validate_transports_in_state(or_state_t *state) +{ + int broken = 0; + config_line_t *line; + + for (line = state->TransportProxies ; line ; line = line->next) { + tor_assert(!strcmp(line->key, "TransportProxy")); + if (!state_transport_line_is_valid(line->value)) + broken = 1; + } + + if (broken) + log_warn(LD_CONFIG, "state: State file seems to be broken."); + + return 0; +} + /** Return 0 if every setting in <b>state</b> is reasonable, and a * permissible transition from <b>old_state</b>. Else warn and return -1. * Should have no side effects, except for normalizing the contents of @@ -5071,6 +6856,9 @@ or_state_validate(or_state_t *old_state, or_state_t *state, if (entry_guards_parse_state(state, 0, msg)<0) return -1; + if (validate_transports_in_state(state)<0) + return -1; + return 0; } @@ -5107,13 +6895,13 @@ or_state_save_broken(char *fname) { int i; file_status_t status; - size_t len = strlen(fname)+16; - char *fname2 = tor_malloc(len); + char *fname2 = NULL; for (i = 0; i < 100; ++i) { - tor_snprintf(fname2, len, "%s.%d", fname, i); + tor_asprintf(&fname2, "%s.%d", fname, i); status = file_status(fname2); if (status == FN_NOENT) break; + tor_free(fname2); } if (i == 100) { log_warn(LD_BUG, "Unable to parse state in \"%s\"; too many saved bad " @@ -5165,7 +6953,7 @@ or_state_load(void) if (contents) { config_line_t *lines=NULL; int assign_retval; - if (config_get_lines(contents, &lines)<0) + if (config_get_lines(contents, &lines, 0)<0) goto done; assign_retval = config_assign(&state_format, new_state, lines, 0, 0, &errmsg); @@ -5269,7 +7057,7 @@ or_state_save(time_t now) tor_free(global_state->TorVersion); tor_asprintf(&global_state->TorVersion, "Tor %s", get_version()); - state = config_dump(&state_format, global_state, 1, 0); + state = config_dump(&state_format, NULL, global_state, 1, 0); format_local_iso_time(tbuf, now); tor_asprintf(&contents, "# Tor state file last generated on %s local time\n" @@ -5303,6 +7091,158 @@ or_state_save(time_t now) return 0; } +/** Return the config line for transport <b>transport</b> in the current state. + * Return NULL if there is no config line for <b>transport</b>. */ +static config_line_t * +get_transport_in_state_by_name(const char *transport) +{ + or_state_t *or_state = get_or_state(); + config_line_t *line; + config_line_t *ret = NULL; + smartlist_t *items = NULL; + + for (line = or_state->TransportProxies ; line ; line = line->next) { + tor_assert(!strcmp(line->key, "TransportProxy")); + + items = smartlist_new(); + smartlist_split_string(items, line->value, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); + if (smartlist_len(items) != 2) /* broken state */ + goto done; + + if (!strcmp(smartlist_get(items, 0), transport)) { + ret = line; + goto done; + } + + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + items = NULL; + } + + done: + if (items) { + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + } + return ret; +} + +/** Return string containing the address:port part of the + * TransportProxy <b>line</b> for transport <b>transport</b>. + * If the line is corrupted, return NULL. */ +static const char * +get_transport_bindaddr(const char *line, const char *transport) +{ + char *line_tmp = NULL; + + if (strlen(line) < strlen(transport) + 2) { + goto broken_state; + } else { + /* line should start with the name of the transport and a space. + (for example, "obfs2 127.0.0.1:47245") */ + tor_asprintf(&line_tmp, "%s ", transport); + if (strcmpstart(line, line_tmp)) + goto broken_state; + + tor_free(line_tmp); + return (line+strlen(transport)+1); + } + + broken_state: + tor_free(line_tmp); + return NULL; +} + +/** Return a string containing the address:port that a proxy transport + * should bind on. The string is stored on the heap and must be freed + * by the caller of this function. */ +char * +get_stored_bindaddr_for_server_transport(const char *transport) +{ + char *default_addrport = NULL; + const char *stored_bindaddr = NULL; + + config_line_t *line = get_transport_in_state_by_name(transport); + if (!line) /* Found no references in state for this transport. */ + goto no_bindaddr_found; + + stored_bindaddr = get_transport_bindaddr(line->value, transport); + if (stored_bindaddr) /* found stored bindaddr in state file. */ + return tor_strdup(stored_bindaddr); + + no_bindaddr_found: + /** If we didn't find references for this pluggable transport in the + state file, we should instruct the pluggable transport proxy to + listen on INADDR_ANY on a random ephemeral port. */ + tor_asprintf(&default_addrport, "%s:%s", fmt_addr32(INADDR_ANY), "0"); + return default_addrport; +} + +/** Save <b>transport</b> listening on <b>addr</b>:<b>port</b> to + state */ +void +save_transport_to_state(const char *transport, + const tor_addr_t *addr, uint16_t port) +{ + or_state_t *state = get_or_state(); + + char *transport_addrport=NULL; + + /** find where to write on the state */ + config_line_t **next, *line; + + /* see if this transport is already stored in state */ + config_line_t *transport_line = + get_transport_in_state_by_name(transport); + + if (transport_line) { /* if transport already exists in state... */ + const char *prev_bindaddr = /* get its addrport... */ + get_transport_bindaddr(transport_line->value, transport); + tor_asprintf(&transport_addrport, "%s:%d", fmt_addr(addr), (int)port); + + /* if transport in state has the same address as this one, life is good */ + if (!strcmp(prev_bindaddr, transport_addrport)) { + log_info(LD_CONFIG, "Transport seems to have spawned on its usual " + "address:port."); + goto done; + } else { /* if addrport in state is different than the one we got */ + log_info(LD_CONFIG, "Transport seems to have spawned on different " + "address:port. Let's update the state file with the new " + "address:port"); + tor_free(transport_line->value); /* free the old line */ + tor_asprintf(&transport_line->value, "%s %s:%d", transport, + fmt_addr(addr), + (int) port); /* replace old addrport line with new line */ + } + } else { /* never seen this one before; save it in state for next time */ + log_info(LD_CONFIG, "It's the first time we see this transport. " + "Let's save its address:port"); + next = &state->TransportProxies; + /* find the last TransportProxy line in the state and point 'next' + right after it */ + line = state->TransportProxies; + while (line) { + next = &(line->next); + line = line->next; + } + + /* allocate space for the new line and fill it in */ + *next = line = tor_malloc_zero(sizeof(config_line_t)); + line->key = tor_strdup("TransportProxy"); + tor_asprintf(&line->value, "%s %s:%d", transport, + fmt_addr(addr), (int) port); + + next = &(line->next); + } + + if (!get_options()->AvoidDiskWrites) + or_state_mark_dirty(state, 0); + + done: + tor_free(transport_addrport); +} + /** Given a file name check to see whether the file exists but has not been * modified for a very long time. If so, remove it. */ void @@ -5332,21 +7272,23 @@ getinfo_helper_config(control_connection_t *conn, (void) conn; (void) errmsg; if (!strcmp(question, "config/names")) { - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); int i; for (i = 0; _option_vars[i].name; ++i) { - config_var_t *var = &_option_vars[i]; + const config_var_t *var = &_option_vars[i]; const char *type; - char *line; switch (var->type) { case CONFIG_TYPE_STRING: type = "String"; break; case CONFIG_TYPE_FILENAME: type = "Filename"; break; case CONFIG_TYPE_UINT: type = "Integer"; break; + case CONFIG_TYPE_INT: type = "SignedInteger"; break; case CONFIG_TYPE_PORT: type = "Port"; break; case CONFIG_TYPE_INTERVAL: type = "TimeInterval"; break; + case CONFIG_TYPE_MSEC_INTERVAL: type = "TimeMsecInterval"; break; case CONFIG_TYPE_MEMUNIT: type = "DataSize"; break; case CONFIG_TYPE_DOUBLE: type = "Float"; break; case CONFIG_TYPE_BOOL: type = "Boolean"; break; + case CONFIG_TYPE_AUTOBOOL: type = "Boolean+Auto"; break; case CONFIG_TYPE_ISOTIME: type = "Time"; break; case CONFIG_TYPE_ROUTERSET: type = "RouterList"; break; case CONFIG_TYPE_CSV: type = "CommaList"; break; @@ -5359,8 +7301,7 @@ getinfo_helper_config(control_connection_t *conn, } if (!type) continue; - tor_asprintf(&line, "%s %s\n",var->name,type); - smartlist_add(sl, line); + smartlist_add_asprintf(sl, "%s %s\n",var->name,type); } *answer = smartlist_join_strings(sl, "", 0, NULL); SMARTLIST_FOREACH(sl, char *, c, tor_free(c)); diff --git a/src/or/config.h b/src/or/config.h index 78a67dddf5..dd76edcf1d 100644 --- a/src/or/config.h +++ b/src/or/config.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -13,7 +13,8 @@ #define _TOR_CONFIG_H const char *get_dirportfrontpage(void); -or_options_t *get_options(void); +const or_options_t *get_options(void); +or_options_t *get_options_mutable(void); int set_options(or_options_t *new_val, char **msg); void config_free_all(void); const char *safe_str_client(const char *address); @@ -21,26 +22,27 @@ const char *safe_str(const char *address); const char *escaped_safe_str_client(const char *address); const char *escaped_safe_str(const char *address); const char *get_version(void); +const char *get_short_version(void); -int config_get_lines(const char *string, config_line_t **result); +int config_get_lines(const char *string, config_line_t **result, int extended); void config_free_lines(config_line_t *front); setopt_err_t options_trial_assign(config_line_t *list, int use_defaults, int clear_first, char **msg); -int resolve_my_address(int warn_severity, or_options_t *options, +int resolve_my_address(int warn_severity, const or_options_t *options, uint32_t *addr, char **hostname_out); -int is_local_addr(const tor_addr_t *addr) ATTR_PURE; +int is_local_addr(const tor_addr_t *addr); void options_init(or_options_t *options); -char *options_dump(or_options_t *options, int minimal); +char *options_dump(const or_options_t *options, int minimal); int options_init_from_torrc(int argc, char **argv); -setopt_err_t options_init_from_string(const char *cf, +setopt_err_t options_init_from_string(const char *cf_defaults, const char *cf, int command, const char *command_arg, char **msg); int option_is_recognized(const char *key); const char *option_get_canonical_name(const char *key); -config_line_t *option_get_assignment(or_options_t *options, +config_line_t *option_get_assignment(const or_options_t *options, const char *key); int options_save_current(void); -const char *get_torrc_fname(void); -char *options_get_datadir_fname2_suffix(or_options_t *options, +const char *get_torrc_fname(int defaults_fname); +char *options_get_datadir_fname2_suffix(const or_options_t *options, const char *sub1, const char *sub2, const char *suffix); #define get_datadir_fname2_suffix(sub1, sub2, suffix) \ @@ -57,23 +59,48 @@ char *options_get_datadir_fname2_suffix(or_options_t *options, #define get_datadir_fname_suffix(sub1, suffix) \ get_datadir_fname2_suffix((sub1), NULL, (suffix)) +int get_num_cpus(const or_options_t *options); + or_state_t *get_or_state(void); int did_last_state_file_write_fail(void); int or_state_save(time_t now); -int options_need_geoip_info(or_options_t *options, const char **reason_out); +const smartlist_t *get_configured_ports(void); +int get_first_advertised_port_by_type_af(int listener_type, + int address_family); +#define get_primary_or_port() \ + (get_first_advertised_port_by_type_af(CONN_TYPE_OR_LISTENER, AF_INET)) +#define get_primary_dir_port() \ + (get_first_advertised_port_by_type_af(CONN_TYPE_DIR_LISTENER, AF_INET)) + +char *get_first_listener_addrport_string(int listener_type); + +int options_need_geoip_info(const or_options_t *options, + const char **reason_out); + +void save_transport_to_state(const char *transport_name, + const tor_addr_t *addr, uint16_t port); +char *get_stored_bindaddr_for_server_transport(const char *transport); + int getinfo_helper_config(control_connection_t *conn, const char *question, char **answer, const char **errmsg); const char *tor_get_digests(void); -uint32_t get_effective_bwrate(or_options_t *options); -uint32_t get_effective_bwburst(or_options_t *options); +uint32_t get_effective_bwrate(const or_options_t *options); +uint32_t get_effective_bwburst(const or_options_t *options); #ifdef CONFIG_PRIVATE /* Used only by config.c and test.c */ or_options_t *options_new(void); #endif +void config_register_addressmaps(const or_options_t *options); +/* XXXX024 move to connection_edge.h */ +int addressmap_register_auto(const char *from, const char *to, + time_t expires, + addressmap_entry_source_t addrmap_source, + const char **msg); + #endif diff --git a/src/or/config_codedigest.c b/src/or/config_codedigest.c index be9eaa331d..86d14bacef 100644 --- a/src/or/config_codedigest.c +++ b/src/or/config_codedigest.c @@ -1,6 +1,8 @@ const char *tor_get_digests(void); +/** Return a string describing the digest of the source files in src/or/ + */ const char * tor_get_digests(void) { diff --git a/src/or/connection.c b/src/or/connection.c index dc75601ab4..e366e0e77f 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -36,17 +36,28 @@ #include "router.h" #include "routerparse.h" -static connection_t *connection_create_listener( +#ifdef USE_BUFFEREVENTS +#include <event2/event.h> +#endif + +#ifdef HAVE_PWD_H +#include <pwd.h> +#endif + +static connection_t *connection_listener_new( const struct sockaddr *listensockaddr, socklen_t listensocklen, int type, - char* address); + const char *address, + const port_cfg_t *portcfg); static void connection_init(time_t now, connection_t *conn, int type, int socket_family); static int connection_init_accepted_conn(connection_t *conn, - uint8_t listener_type); + const listener_connection_t *listener); static int connection_handle_listener_read(connection_t *conn, int new_type); +#ifndef USE_BUFFEREVENTS static int connection_bucket_should_increase(int bucket, or_connection_t *conn); +#endif static int connection_finished_flushing(connection_t *conn); static int connection_flushed_some(connection_t *conn); static int connection_finished_connecting(connection_t *conn); @@ -60,14 +71,30 @@ static void set_constrained_socket_buffers(tor_socket_t sock, int size); static const char *connection_proxy_state_to_string(int state); static int connection_read_https_proxy_response(connection_t *conn); static void connection_send_socks5_connect(connection_t *conn); +static const char *proxy_type_to_string(int proxy_type); +static int get_proxy_type(void); -/** The last IPv4 address that our network interface seemed to have been - * binding to, in host order. We use this to detect when our IP changes. */ -static uint32_t last_interface_ip = 0; -/** A list of uint32_ts for addresses we've used in outgoing connections. +/** The last addresses that our network interface seemed to have been + * binding to. We use this as one way to detect when our IP changes. + * + * XXX024 We should really use the entire list of interfaces here. + **/ +static tor_addr_t *last_interface_ipv4 = NULL; +/* DOCDOC last_interface_ipv6 */ +static tor_addr_t *last_interface_ipv6 = NULL; +/** A list of tor_addr_t for addresses we've used in outgoing connections. * Used to detect IP address changes. */ static smartlist_t *outgoing_addrs = NULL; +#define CASE_ANY_LISTENER_TYPE \ + case CONN_TYPE_OR_LISTENER: \ + case CONN_TYPE_AP_LISTENER: \ + case CONN_TYPE_DIR_LISTENER: \ + case CONN_TYPE_CONTROL_LISTENER: \ + case CONN_TYPE_AP_TRANS_LISTENER: \ + case CONN_TYPE_AP_NATD_LISTENER: \ + case CONN_TYPE_AP_DNS_LISTENER + /**************************************************************/ /** @@ -108,13 +135,7 @@ conn_state_to_string(int type, int state) { static char buf[96]; switch (type) { - case CONN_TYPE_OR_LISTENER: - case CONN_TYPE_AP_LISTENER: - case CONN_TYPE_AP_TRANS_LISTENER: - case CONN_TYPE_AP_NATD_LISTENER: - case CONN_TYPE_AP_DNS_LISTENER: - case CONN_TYPE_DIR_LISTENER: - case CONN_TYPE_CONTROL_LISTENER: + CASE_ANY_LISTENER_TYPE: if (state == LISTENER_STATE_READY) return "ready"; break; @@ -124,10 +145,13 @@ conn_state_to_string(int type, int state) case OR_CONN_STATE_PROXY_HANDSHAKING: return "handshaking (proxy)"; case OR_CONN_STATE_TLS_HANDSHAKING: return "handshaking (TLS)"; case OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING: - return "renegotiating (TLS)"; + return "renegotiating (TLS, v2 handshake)"; case OR_CONN_STATE_TLS_SERVER_RENEGOTIATING: - return "waiting for renegotiation (TLS)"; - case OR_CONN_STATE_OR_HANDSHAKING: return "handshaking (Tor)"; + return "waiting for renegotiation or V3 handshake"; + case OR_CONN_STATE_OR_HANDSHAKING_V2: + return "handshaking (Tor, v2 handshake)"; + case OR_CONN_STATE_OR_HANDSHAKING_V3: + return "handshaking (Tor, v3 handshake)"; case OR_CONN_STATE_OPEN: return "open"; } break; @@ -183,6 +207,26 @@ conn_state_to_string(int type, int state) return buf; } +#ifdef USE_BUFFEREVENTS +/** Return true iff the connection's type is one that can use a + bufferevent-based implementation. */ +int +connection_type_uses_bufferevent(connection_t *conn) +{ + switch (conn->type) { + case CONN_TYPE_AP: + case CONN_TYPE_EXIT: + case CONN_TYPE_DIR: + case CONN_TYPE_CONTROL: + case CONN_TYPE_OR: + case CONN_TYPE_CPUWORKER: + return 1; + default: + return 0; + } +} +#endif + /** Allocate and return a new dir_connection_t, initialized as by * connection_init(). */ dir_connection_t * @@ -205,22 +249,32 @@ or_connection_new(int socket_family) or_conn->timestamp_last_added_nonpadding = time(NULL); or_conn->next_circ_id = crypto_rand_int(1<<15); - or_conn->active_circuit_pqueue = smartlist_create(); + or_conn->active_circuit_pqueue = smartlist_new(); or_conn->active_circuit_pqueue_last_recalibrated = cell_ewma_get_tick(); return or_conn; } +/** Allocate and return a new entry_connection_t, initialized as by + * connection_init(). */ +entry_connection_t * +entry_connection_new(int type, int socket_family) +{ + entry_connection_t *entry_conn = tor_malloc_zero(sizeof(entry_connection_t)); + tor_assert(type == CONN_TYPE_AP); + connection_init(time(NULL), ENTRY_TO_CONN(entry_conn), type, socket_family); + entry_conn->socks_request = socks_request_new(); + return entry_conn; +} + /** Allocate and return a new edge_connection_t, initialized as by * connection_init(). */ edge_connection_t * edge_connection_new(int type, int socket_family) { edge_connection_t *edge_conn = tor_malloc_zero(sizeof(edge_connection_t)); - tor_assert(type == CONN_TYPE_EXIT || type == CONN_TYPE_AP); + tor_assert(type == CONN_TYPE_EXIT); connection_init(time(NULL), TO_CONN(edge_conn), type, socket_family); - if (type == CONN_TYPE_AP) - edge_conn->socks_request = tor_malloc_zero(sizeof(socks_request_t)); return edge_conn; } @@ -237,6 +291,17 @@ control_connection_new(int socket_family) return control_conn; } +/** Allocate and return a new listener_connection_t, initialized as by + * connection_init(). */ +listener_connection_t * +listener_connection_new(int type, int socket_family) +{ + listener_connection_t *listener_conn = + tor_malloc_zero(sizeof(listener_connection_t)); + connection_init(time(NULL), TO_CONN(listener_conn), type, socket_family); + return listener_conn; +} + /** Allocate, initialize, and return a new connection_t subtype of <b>type</b> * to make or receive connections of address family <b>socket_family</b>. The * type should be one of the CONN_TYPE_* constants. */ @@ -248,15 +313,20 @@ connection_new(int type, int socket_family) return TO_CONN(or_connection_new(socket_family)); case CONN_TYPE_EXIT: - case CONN_TYPE_AP: return TO_CONN(edge_connection_new(type, socket_family)); + case CONN_TYPE_AP: + return ENTRY_TO_CONN(entry_connection_new(type, socket_family)); + case CONN_TYPE_DIR: return TO_CONN(dir_connection_new(socket_family)); case CONN_TYPE_CONTROL: return TO_CONN(control_connection_new(socket_family)); + CASE_ANY_LISTENER_TYPE: + return TO_CONN(listener_connection_new(type, socket_family)); + default: { connection_t *conn = tor_malloc_zero(sizeof(connection_t)); connection_init(time(NULL), conn, type, socket_family); @@ -288,30 +358,38 @@ connection_init(time_t now, connection_t *conn, int type, int socket_family) conn->magic = OR_CONNECTION_MAGIC; break; case CONN_TYPE_EXIT: - case CONN_TYPE_AP: conn->magic = EDGE_CONNECTION_MAGIC; break; + case CONN_TYPE_AP: + conn->magic = ENTRY_CONNECTION_MAGIC; + break; case CONN_TYPE_DIR: conn->magic = DIR_CONNECTION_MAGIC; break; case CONN_TYPE_CONTROL: conn->magic = CONTROL_CONNECTION_MAGIC; break; + CASE_ANY_LISTENER_TYPE: + conn->magic = LISTENER_CONNECTION_MAGIC; + break; default: conn->magic = BASE_CONNECTION_MAGIC; break; } - conn->s = -1; /* give it a default of 'not used' */ + conn->s = TOR_INVALID_SOCKET; /* give it a default of 'not used' */ conn->conn_array_index = -1; /* also default to 'not used' */ conn->global_identifier = n_connections_allocated++; conn->type = type; conn->socket_family = socket_family; - if (!connection_is_listener(conn)) { /* listeners never use their buf */ +#ifndef USE_BUFFEREVENTS + if (!connection_is_listener(conn)) { + /* listeners never use their buf */ conn->inbuf = buf_new(); conn->outbuf = buf_new(); } +#endif conn->timestamp_created = now; conn->timestamp_lastread = now; @@ -322,8 +400,8 @@ connection_init(time_t now, connection_t *conn, int type, int socket_family) void connection_link_connections(connection_t *conn_a, connection_t *conn_b) { - tor_assert(conn_a->s < 0); - tor_assert(conn_b->s < 0); + tor_assert(! SOCKET_OK(conn_a->s)); + tor_assert(! SOCKET_OK(conn_b->s)); conn_a->linked = 1; conn_b->linked = 1; @@ -350,6 +428,10 @@ _connection_free(connection_t *conn) memlen = sizeof(or_connection_t); break; case CONN_TYPE_AP: + tor_assert(conn->magic == ENTRY_CONNECTION_MAGIC); + mem = TO_ENTRY_CONN(conn); + memlen = sizeof(entry_connection_t); + break; case CONN_TYPE_EXIT: tor_assert(conn->magic == EDGE_CONNECTION_MAGIC); mem = TO_EDGE_CONN(conn); @@ -365,6 +447,11 @@ _connection_free(connection_t *conn) mem = TO_CONTROL_CONN(conn); memlen = sizeof(control_connection_t); break; + CASE_ANY_LISTENER_TYPE: + tor_assert(conn->magic == LISTENER_CONNECTION_MAGIC); + mem = TO_LISTENER_CONN(conn); + memlen = sizeof(listener_connection_t); + break; default: tor_assert(conn->magic == BASE_CONNECTION_MAGIC); mem = conn; @@ -377,7 +464,8 @@ _connection_free(connection_t *conn) "bytes on inbuf, %d on outbuf.", conn_type_to_string(conn->type), conn_state_to_string(conn->type, conn->state), - (int)buf_datalen(conn->inbuf), (int)buf_datalen(conn->outbuf)); + (int)connection_get_inbuf_len(conn), + (int)connection_get_outbuf_len(conn)); } if (!connection_is_listener(conn)) { @@ -407,15 +495,21 @@ _connection_free(connection_t *conn) smartlist_free(or_conn->active_circuit_pqueue); tor_free(or_conn->nickname); } - if (CONN_IS_EDGE(conn)) { - edge_connection_t *edge_conn = TO_EDGE_CONN(conn); - tor_free(edge_conn->chosen_exit_name); - if (edge_conn->socks_request) { - memset(edge_conn->socks_request, 0xcc, sizeof(socks_request_t)); - tor_free(edge_conn->socks_request); + if (conn->type == CONN_TYPE_AP) { + entry_connection_t *entry_conn = TO_ENTRY_CONN(conn); + tor_free(entry_conn->chosen_exit_name); + tor_free(entry_conn->original_dest_address); + if (entry_conn->socks_request) + socks_request_free(entry_conn->socks_request); + if (entry_conn->pending_optimistic_data) { + generic_buffer_free(entry_conn->pending_optimistic_data); } - - rend_data_free(edge_conn->rend_data); + if (entry_conn->sending_optimistic_data) { + generic_buffer_free(entry_conn->sending_optimistic_data); + } + } + if (CONN_IS_EDGE(conn)) { + rend_data_free(TO_EDGE_CONN(conn)->rend_data); } if (conn->type == CONN_TYPE_CONTROL) { control_connection_t *control_conn = TO_CONTROL_CONN(conn); @@ -425,6 +519,15 @@ _connection_free(connection_t *conn) tor_free(conn->read_event); /* Probably already freed by connection_free. */ tor_free(conn->write_event); /* Probably already freed by connection_free. */ + IF_HAS_BUFFEREVENT(conn, { + /* This was a workaround to handle bugs in some old versions of libevent + * where callbacks can occur after calling bufferevent_free(). Setting + * the callbacks to NULL prevented this. It shouldn't be necessary any + * more, but let's not tempt fate for now. */ + bufferevent_setcb(conn->bufev, NULL, NULL, NULL, NULL); + bufferevent_free(conn->bufev); + conn->bufev = NULL; + }); if (conn->type == CONN_TYPE_DIR) { dir_connection_t *dir_conn = TO_DIR_CONN(conn); @@ -443,7 +546,7 @@ _connection_free(connection_t *conn) if (SOCKET_OK(conn->s)) { log_debug(LD_NET,"closing fd %d.",(int)conn->s); tor_close_socket(conn->s); - conn->s = -1; + conn->s = TOR_INVALID_SOCKET; } if (conn->type == CONN_TYPE_OR && @@ -451,8 +554,14 @@ _connection_free(connection_t *conn) log_warn(LD_BUG, "called on OR conn with non-zeroed identity_digest"); connection_or_remove_from_identity_map(TO_OR_CONN(conn)); } +#ifdef USE_BUFFEREVENTS + if (conn->type == CONN_TYPE_OR && TO_OR_CONN(conn)->bucket_cfg) { + ev_token_bucket_cfg_free(TO_OR_CONN(conn)->bucket_cfg); + TO_OR_CONN(conn)->bucket_cfg = NULL; + } +#endif - memset(mem, 0xCC, memlen); /* poison memory */ + memwipe(mem, 0xCC, memlen); /* poison memory */ tor_free(mem); } @@ -486,39 +595,9 @@ connection_free(connection_t *conn) _connection_free(conn); } -/** Call _connection_free() on every connection in our array, and release all - * storage held by connection.c. This is used by cpuworkers and dnsworkers - * when they fork, so they don't keep resources held open (especially - * sockets). - * - * Don't do the checks in connection_free(), because they will - * fail. - */ -void -connection_free_all(void) -{ - smartlist_t *conns = get_connection_array(); - - /* We don't want to log any messages to controllers. */ - SMARTLIST_FOREACH(conns, connection_t *, conn, - if (conn->type == CONN_TYPE_CONTROL) - TO_CONTROL_CONN(conn)->event_mask = 0); - - control_update_global_event_mask(); - - /* Unlink everything from the identity map. */ - connection_or_clear_identity_map(); - - SMARTLIST_FOREACH(conns, connection_t *, conn, _connection_free(conn)); - - if (outgoing_addrs) { - SMARTLIST_FOREACH(outgoing_addrs, void*, addr, tor_free(addr)); - smartlist_free(outgoing_addrs); - outgoing_addrs = NULL; - } -} - -/** Do any cleanup needed: +/** + * Called when we're about to finally unlink and free a connection: + * perform necessary accounting and cleanup * - Directory conns that failed to fetch a rendezvous descriptor * need to inform pending rendezvous streams. * - OR conns need to call rep_hist_note_*() to record status. @@ -531,114 +610,20 @@ connection_free_all(void) void connection_about_to_close_connection(connection_t *conn) { - circuit_t *circ; - dir_connection_t *dir_conn; - or_connection_t *or_conn; - edge_connection_t *edge_conn; - time_t now = time(NULL); - tor_assert(conn->marked_for_close); - if (CONN_IS_EDGE(conn)) { - edge_conn = TO_EDGE_CONN(conn); - if (!edge_conn->edge_has_sent_end) { - log_warn(LD_BUG, "(Harmless.) Edge connection (marked at %s:%d) " - "hasn't sent end yet?", - conn->marked_for_close_file, conn->marked_for_close); - tor_fragile_assert(); - } - } - switch (conn->type) { case CONN_TYPE_DIR: - dir_conn = TO_DIR_CONN(conn); - if (conn->state < DIR_CONN_STATE_CLIENT_FINISHED) { - /* It's a directory connection and connecting or fetching - * failed: forget about this router, and maybe try again. */ - connection_dir_request_failed(dir_conn); - } - /* If we were trying to fetch a v2 rend desc and did not succeed, - * retry as needed. (If a fetch is successful, the connection state - * is changed to DIR_PURPOSE_HAS_FETCHED_RENDDESC to mark that - * refetching is unnecessary.) */ - if (conn->purpose == DIR_PURPOSE_FETCH_RENDDESC_V2 && - dir_conn->rend_data && - strlen(dir_conn->rend_data->onion_address) == - REND_SERVICE_ID_LEN_BASE32) - rend_client_refetch_v2_renddesc(dir_conn->rend_data); + connection_dir_about_to_close(TO_DIR_CONN(conn)); break; case CONN_TYPE_OR: - or_conn = TO_OR_CONN(conn); - /* Remember why we're closing this connection. */ - if (conn->state != OR_CONN_STATE_OPEN) { - /* Inform any pending (not attached) circs that they should - * give up. */ - circuit_n_conn_done(TO_OR_CONN(conn), 0); - /* now mark things down as needed */ - if (connection_or_nonopen_was_started_here(or_conn)) { - or_options_t *options = get_options(); - rep_hist_note_connect_failed(or_conn->identity_digest, now); - entry_guard_register_connect_status(or_conn->identity_digest,0, - !options->HTTPSProxy, now); - if (conn->state >= OR_CONN_STATE_TLS_HANDSHAKING) { - int reason = tls_error_to_orconn_end_reason(or_conn->tls_error); - control_event_or_conn_status(or_conn, OR_CONN_EVENT_FAILED, - reason); - if (!authdir_mode_tests_reachability(options)) - control_event_bootstrap_problem( - orconn_end_reason_to_control_string(reason), reason); - } - } - } else if (conn->hold_open_until_flushed) { - /* We only set hold_open_until_flushed when we're intentionally - * closing a connection. */ - rep_hist_note_disconnect(or_conn->identity_digest, now); - control_event_or_conn_status(or_conn, OR_CONN_EVENT_CLOSED, - tls_error_to_orconn_end_reason(or_conn->tls_error)); - } else if (!tor_digest_is_zero(or_conn->identity_digest)) { - rep_hist_note_connection_died(or_conn->identity_digest, now); - control_event_or_conn_status(or_conn, OR_CONN_EVENT_CLOSED, - tls_error_to_orconn_end_reason(or_conn->tls_error)); - } - /* Now close all the attached circuits on it. */ - circuit_unlink_all_from_or_conn(TO_OR_CONN(conn), - END_CIRC_REASON_OR_CONN_CLOSED); + connection_or_about_to_close(TO_OR_CONN(conn)); break; case CONN_TYPE_AP: - edge_conn = TO_EDGE_CONN(conn); - if (edge_conn->socks_request->has_finished == 0) { - /* since conn gets removed right after this function finishes, - * there's no point trying to send back a reply at this point. */ - log_warn(LD_BUG,"Closing stream (marked at %s:%d) without sending" - " back a socks reply.", - conn->marked_for_close_file, conn->marked_for_close); - } - if (!edge_conn->end_reason) { - log_warn(LD_BUG,"Closing stream (marked at %s:%d) without having" - " set end_reason.", - conn->marked_for_close_file, conn->marked_for_close); - } - if (edge_conn->dns_server_request) { - log_warn(LD_BUG,"Closing stream (marked at %s:%d) without having" - " replied to DNS request.", - conn->marked_for_close_file, conn->marked_for_close); - dnsserv_reject_request(edge_conn); - } - control_event_stream_bandwidth(edge_conn); - control_event_stream_status(edge_conn, STREAM_EVENT_CLOSED, - edge_conn->end_reason); - circ = circuit_get_by_edge_conn(edge_conn); - if (circ) - circuit_detach_stream(circ, edge_conn); + connection_ap_about_to_close(TO_ENTRY_CONN(conn)); break; case CONN_TYPE_EXIT: - edge_conn = TO_EDGE_CONN(conn); - circ = circuit_get_by_edge_conn(edge_conn); - if (circ) - circuit_detach_stream(circ, edge_conn); - if (conn->state == EXIT_CONN_STATE_RESOLVING) { - connection_dns_remove(edge_conn); - } + connection_exit_about_to_close(TO_EDGE_CONN(conn)); break; } } @@ -646,7 +631,7 @@ connection_about_to_close_connection(connection_t *conn) /** Return true iff connection_close_immediate() has been called on this * connection. */ #define CONN_IS_CLOSED(c) \ - ((c)->linked ? ((c)->linked_conn_is_closed) : ((c)->s < 0)) + ((c)->linked ? ((c)->linked_conn_is_closed) : (! SOCKET_OK(c->s))) /** Close the underlying socket for <b>conn</b>, so we don't try to * flush it. Must be used in conjunction with (right before) @@ -672,13 +657,12 @@ connection_close_immediate(connection_t *conn) if (SOCKET_OK(conn->s)) tor_close_socket(conn->s); - conn->s = -1; + conn->s = TOR_INVALID_SOCKET; if (conn->linked) conn->linked_conn_is_closed = 1; - if (!connection_is_listener(conn)) { + if (conn->outbuf) buf_clear(conn->outbuf); - conn->outbuf_flushlen = 0; - } + conn->outbuf_flushlen = 0; } /** Mark <b>conn</b> to be closed next time we loop through @@ -722,8 +706,7 @@ connection_expire_held_open(void) now = time(NULL); - SMARTLIST_FOREACH(conns, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { /* If we've been holding the connection open, but we haven't written * for 15 seconds... */ @@ -745,52 +728,10 @@ connection_expire_held_open(void) conn->hold_open_until_flushed = 0; } } - }); + } SMARTLIST_FOREACH_END(conn); } -/** Create an AF_INET listenaddr struct. - * <b>listenaddress</b> provides the host and optionally the port information - * for the new structure. If no port is provided in <b>listenaddress</b> then - * <b>listenport</b> is used. - * - * If not NULL <b>readable_address</b> will contain a copy of the host part of - * <b>listenaddress</b>. - * - * The listenaddr struct has to be freed by the caller. - */ -static struct sockaddr_in * -create_inet_sockaddr(const char *listenaddress, int listenport, - char **readable_address, socklen_t *socklen_out) { - struct sockaddr_in *listenaddr = NULL; - uint32_t addr; - uint16_t usePort = 0; - - if (parse_addr_port(LOG_WARN, - listenaddress, readable_address, &addr, &usePort)<0) { - log_warn(LD_CONFIG, - "Error parsing/resolving ListenAddress %s", listenaddress); - goto err; - } - if (usePort==0) { - if (listenport != CFG_AUTO_PORT) - usePort = listenport; - } - - listenaddr = tor_malloc_zero(sizeof(struct sockaddr_in)); - listenaddr->sin_addr.s_addr = htonl(addr); - listenaddr->sin_family = AF_INET; - listenaddr->sin_port = htons((uint16_t) usePort); - - *socklen_out = sizeof(struct sockaddr_in); - - return listenaddr; - - err: - tor_free(listenaddr); - return NULL; -} - -#ifdef HAVE_SYS_UN_H +#if defined(HAVE_SYS_UN_H) || defined(RUNNING_DOXYGEN) /** Create an AF_UNIX listenaddr struct. * <b>listenaddress</b> provides the path to the Unix socket. * @@ -836,8 +777,9 @@ create_unix_sockaddr(const char *listenaddress, char **readable_address, log_fn(LOG_ERR, LD_BUG, "Unix domain sockets not supported, yet we tried to create one."); *len_out = 0; - tor_assert(0); -}; + tor_fragile_assert(); + return NULL; +} #endif /* HAVE_SYS_UN_H */ /** Warn that an accept or a connect has failed because we're running up @@ -863,7 +805,7 @@ warn_too_many_conns(void) /** Check whether we should be willing to open an AF_UNIX socket in * <b>path</b>. Return 0 if we should go ahead and -1 if we shouldn't. */ static int -check_location_for_unix_socket(or_options_t *options, const char *path) +check_location_for_unix_socket(const or_options_t *options, const char *path) { int r = -1; char *p = tor_strdup(path); @@ -901,7 +843,7 @@ check_location_for_unix_socket(or_options_t *options, const char *path) static void make_socket_reuseable(tor_socket_t sock) { -#ifdef MS_WINDOWS +#ifdef _WIN32 (void) sock; #else int one=1; @@ -922,22 +864,30 @@ make_socket_reuseable(tor_socket_t sock) * to the conn. */ static connection_t * -connection_create_listener(const struct sockaddr *listensockaddr, +connection_listener_new(const struct sockaddr *listensockaddr, socklen_t socklen, - int type, char* address) + int type, const char *address, + const port_cfg_t *port_cfg) { + listener_connection_t *lis_conn; connection_t *conn; tor_socket_t s; /* the socket we're going to make */ + or_options_t const *options = get_options(); +#if defined(HAVE_PWD_H) && defined(HAVE_SYS_UN_H) + struct passwd *pw = NULL; +#endif uint16_t usePort = 0, gotPort = 0; int start_reading = 0; + static int global_next_session_group = SESSION_GROUP_FIRST_AUTO; + tor_addr_t addr; if (get_n_open_sockets() >= get_options()->_ConnLimit-1) { warn_too_many_conns(); return NULL; } - if (listensockaddr->sa_family == AF_INET) { - tor_addr_t addr; + if (listensockaddr->sa_family == AF_INET || + listensockaddr->sa_family == AF_INET6) { int is_tcp = (type != CONN_TYPE_AP_DNS_LISTENER); if (is_tcp) start_reading = 1; @@ -947,16 +897,37 @@ connection_create_listener(const struct sockaddr *listensockaddr, log_notice(LD_NET, "Opening %s on %s:%d", conn_type_to_string(type), fmt_addr(&addr), usePort); - s = tor_open_socket(PF_INET, + s = tor_open_socket(tor_addr_family(&addr), is_tcp ? SOCK_STREAM : SOCK_DGRAM, is_tcp ? IPPROTO_TCP: IPPROTO_UDP); if (!SOCKET_OK(s)) { - log_warn(LD_NET,"Socket creation failed."); + log_warn(LD_NET,"Socket creation failed: %s", + tor_socket_strerror(tor_socket_errno(-1))); goto err; } make_socket_reuseable(s); +#ifdef IPV6_V6ONLY + if (listensockaddr->sa_family == AF_INET6) { +#ifdef _WIN32 + /* In Redmond, this kind of thing passes for standards-conformance. */ + DWORD one = 1; +#else + int one = 1; +#endif + /* We need to set IPV6_V6ONLY so that this socket can't get used for + * IPv4 connections. */ + if (setsockopt(s,IPPROTO_IPV6, IPV6_V6ONLY, + (void*)&one, sizeof(one))<0) { + int e = tor_socket_errno(s); + log_warn(LD_NET, "Error setting IPV6_V6ONLY flag: %s", + tor_socket_strerror(e)); + /* Keep going; probably not harmful. */ + } + } +#endif + if (bind(s,listensockaddr,socklen) < 0) { const char *helpfulhint = ""; int e = tor_socket_errno(s); @@ -999,19 +970,21 @@ connection_create_listener(const struct sockaddr *listensockaddr, * and listeners at the same time */ tor_assert(type == CONN_TYPE_CONTROL_LISTENER); - if (check_location_for_unix_socket(get_options(), address) < 0) + if (check_location_for_unix_socket(options, address) < 0) goto err; log_notice(LD_NET, "Opening %s on %s", conn_type_to_string(type), address); + tor_addr_make_unspec(&addr); + if (unlink(address) < 0 && errno != ENOENT) { log_warn(LD_NET, "Could not unlink %s: %s", address, strerror(errno)); goto err; } s = tor_open_socket(AF_UNIX, SOCK_STREAM, 0); - if (s < 0) { + if (! SOCKET_OK(s)) { log_warn(LD_NET,"Socket creation failed: %s.", strerror(errno)); goto err; } @@ -1021,7 +994,20 @@ connection_create_listener(const struct sockaddr *listensockaddr, tor_socket_strerror(tor_socket_errno(s))); goto err; } - if (get_options()->ControlSocketsGroupWritable) { +#ifdef HAVE_PWD_H + if (options->User) { + pw = getpwnam(options->User); + if (pw == NULL) { + log_warn(LD_NET,"Unable to chown() %s socket: user %s not found.", + address, options->User); + } else if (chown(address, pw->pw_uid, pw->pw_gid) < 0) { + log_warn(LD_NET,"Unable to chown() %s socket: %s.", + address, strerror(errno)); + goto err; + } + } +#endif + if (options->ControlSocketsGroupWritable) { /* We need to use chmod; fchmod doesn't work on sockets on all * platforms. */ if (chmod(address, 0660) < 0) { @@ -1037,6 +1023,8 @@ connection_create_listener(const struct sockaddr *listensockaddr, tor_close_socket(s); goto err; } +#else + (void)options; #endif /* HAVE_SYS_UN_H */ } else { log_err(LD_BUG,"Got unexpected address family %d.", @@ -1046,11 +1034,28 @@ connection_create_listener(const struct sockaddr *listensockaddr, set_socket_nonblocking(s); - conn = connection_new(type, listensockaddr->sa_family); + lis_conn = listener_connection_new(type, listensockaddr->sa_family); + conn = TO_CONN(lis_conn); conn->socket_family = listensockaddr->sa_family; conn->s = s; conn->address = tor_strdup(address); conn->port = gotPort; + tor_addr_copy(&conn->addr, &addr); + + if (port_cfg->isolation_flags) { + lis_conn->isolation_flags = port_cfg->isolation_flags; + if (port_cfg->session_group >= 0) { + lis_conn->session_group = port_cfg->session_group; + } else { + /* This can wrap after around INT_MAX listeners are opened. But I don't + * believe that matters, since you would need to open a ridiculous + * number of listeners while keeping the early ones open before you ever + * hit this. An OR with a dozen ports open, for example, would have to + * close and re-open its listeners every second for 4 years nonstop. + */ + lis_conn->session_group = global_next_session_group--; + } + } if (connection_add(conn) < 0) { /* no space, forget it */ log_warn(LD_NET,"connection_add for listener failed. Giving up."); @@ -1148,14 +1153,14 @@ connection_handle_listener_read(connection_t *conn, int new_type) tor_socket_t news; /* the new socket */ connection_t *newconn; /* information about the remote peer when connecting to other routers */ - char addrbuf[256]; /*XXX023 use sockaddr_storage instead*/ - struct sockaddr *remote = (struct sockaddr*)addrbuf; + struct sockaddr_storage addrbuf; + struct sockaddr *remote = (struct sockaddr*)&addrbuf; /* length of the remote address. Must be whatever accept() needs. */ socklen_t remotelen = (socklen_t)sizeof(addrbuf); - or_options_t *options = get_options(); + const or_options_t *options = get_options(); tor_assert((size_t)remotelen >= sizeof(struct sockaddr_in)); - memset(addrbuf, 0, sizeof(addrbuf)); + memset(&addrbuf, 0, sizeof(addrbuf)); news = tor_accept_socket(conn->s,remote,&remotelen); if (!SOCKET_OK(news)) { /* accept() error */ @@ -1254,7 +1259,7 @@ connection_handle_listener_read(connection_t *conn, int new_type) return 0; /* no need to tear down the parent */ } - if (connection_init_accepted_conn(newconn, conn->type) < 0) { + if (connection_init_accepted_conn(newconn, TO_LISTENER_CONN(conn)) < 0) { if (! newconn->marked_for_close) connection_mark_for_close(newconn); return 0; @@ -1268,7 +1273,8 @@ connection_handle_listener_read(connection_t *conn, int new_type) * and place it in circuit_wait. */ static int -connection_init_accepted_conn(connection_t *conn, uint8_t listener_type) +connection_init_accepted_conn(connection_t *conn, + const listener_connection_t *listener) { connection_start_reading(conn); @@ -1277,16 +1283,20 @@ connection_init_accepted_conn(connection_t *conn, uint8_t listener_type) control_event_or_conn_status(TO_OR_CONN(conn), OR_CONN_EVENT_NEW, 0); return connection_tls_start_handshake(TO_OR_CONN(conn), 1); case CONN_TYPE_AP: - switch (listener_type) { + TO_ENTRY_CONN(conn)->isolation_flags = listener->isolation_flags; + TO_ENTRY_CONN(conn)->session_group = listener->session_group; + TO_ENTRY_CONN(conn)->nym_epoch = get_signewnym_epoch(); + TO_ENTRY_CONN(conn)->socks_request->listener_type = listener->_base.type; + switch (TO_CONN(listener)->type) { case CONN_TYPE_AP_LISTENER: conn->state = AP_CONN_STATE_SOCKS_WAIT; break; case CONN_TYPE_AP_TRANS_LISTENER: - TO_EDGE_CONN(conn)->is_transparent_ap = 1; + TO_ENTRY_CONN(conn)->is_transparent_ap = 1; conn->state = AP_CONN_STATE_CIRCUIT_WAIT; - return connection_ap_process_transparent(TO_EDGE_CONN(conn)); + return connection_ap_process_transparent(TO_ENTRY_CONN(conn)); case CONN_TYPE_AP_NATD_LISTENER: - TO_EDGE_CONN(conn)->is_transparent_ap = 1; + TO_ENTRY_CONN(conn)->is_transparent_ap = 1; conn->state = AP_CONN_STATE_NATD_WAIT; break; } @@ -1317,14 +1327,15 @@ connection_connect(connection_t *conn, const char *address, { tor_socket_t s; int inprogress = 0; - char addrbuf[256]; + struct sockaddr_storage addrbuf; struct sockaddr *dest_addr; - socklen_t dest_addr_len; - or_options_t *options = get_options(); + int dest_addr_len; + const or_options_t *options = get_options(); int protocol_family; if (get_n_open_sockets() >= get_options()->_ConnLimit-1) { warn_too_many_conns(); + *socket_error = SOCK_ERRNO(ENOBUFS); return -1; } @@ -1333,14 +1344,34 @@ connection_connect(connection_t *conn, const char *address, else protocol_family = PF_INET; + if (get_options()->DisableNetwork) { + /* We should never even try to connect anyplace if DisableNetwork is set. + * Warn if we do, and refuse to make the connection. */ + static ratelim_t disablenet_violated = RATELIM_INIT(30*60); + char *m; +#ifdef _WIN32 + *socket_error = WSAENETUNREACH; +#else + *socket_error = ENETUNREACH; +#endif + if ((m = rate_limit_log(&disablenet_violated, approx_time()))) { + log_warn(LD_BUG, "Tried to open a socket with DisableNetwork set.%s", m); + tor_free(m); + } + tor_fragile_assert(); + return -1; + } + s = tor_open_socket(protocol_family,SOCK_STREAM,IPPROTO_TCP); - if (s < 0) { + if (! SOCKET_OK(s)) { *socket_error = tor_socket_errno(-1); log_warn(LD_NET,"Error creating network socket: %s", tor_socket_strerror(*socket_error)); return -1; } + make_socket_reuseable(s); + if (options->OutboundBindAddress && !tor_addr_is_loopback(addr)) { struct sockaddr_in ext_addr; @@ -1367,17 +1398,15 @@ connection_connect(connection_t *conn, const char *address, if (options->ConstrainedSockets) set_constrained_socket_buffers(s, (int)options->ConstrainedSockSize); - memset(addrbuf,0,sizeof(addrbuf)); - dest_addr = (struct sockaddr*) addrbuf; + memset(&addrbuf,0,sizeof(addrbuf)); + dest_addr = (struct sockaddr*) &addrbuf; dest_addr_len = tor_addr_to_sockaddr(addr, port, dest_addr, sizeof(addrbuf)); tor_assert(dest_addr_len > 0); log_debug(LD_NET, "Connecting to %s:%u.", escaped_safe_str_client(address), port); - make_socket_reuseable(s); - - if (connect(s, dest_addr, dest_addr_len) < 0) { + if (connect(s, dest_addr, (socklen_t)dest_addr_len) < 0) { int e = tor_socket_errno(s); if (!ERRNO_IS_CONN_EINPROGRESS(e)) { /* yuck. kill it. */ @@ -1393,17 +1422,17 @@ connection_connect(connection_t *conn, const char *address, } } - if (!server_mode(options)) - client_check_address_changed(s); - /* it succeeded. we're connected. */ log_fn(inprogress?LOG_DEBUG:LOG_INFO, LD_NET, "Connection to %s:%u %s (sock %d).", escaped_safe_str_client(address), port, inprogress?"in progress":"established", s); conn->s = s; - if (connection_add(conn) < 0) /* no space, forget it */ + if (connection_add_connecting(conn) < 0) { + /* no space, forget it */ + *socket_error = SOCK_ERRNO(ENOBUFS); return -1; + } return inprogress ? 0 : 1; } @@ -1415,6 +1444,7 @@ connection_proxy_state_to_string(int state) static const char *unknown = "???"; static const char *states[] = { "PROXY_NONE", + "PROXY_INFANT", "PROXY_HTTPS_WANT_CONNECT_OK", "PROXY_SOCKS4_WANT_CONNECT_OK", "PROXY_SOCKS5_WANT_AUTH_METHOD_NONE", @@ -1443,7 +1473,7 @@ connection_proxy_state_to_string(int state) int connection_proxy_connect(connection_t *conn, int type) { - or_options_t *options; + const or_options_t *options; tor_assert(conn); @@ -1638,6 +1668,20 @@ connection_send_socks5_connect(connection_t *conn) conn->proxy_state = PROXY_SOCKS5_WANT_CONNECT_OK; } +/** Wrapper around fetch_from_(buf/evbuffer)_socks_client: see those functions + * for documentation of its behavior. */ +static int +connection_fetch_from_buf_socks_client(connection_t *conn, + int state, char **reason) +{ + IF_HAS_BUFFEREVENT(conn, { + struct evbuffer *input = bufferevent_get_input(conn->bufev); + return fetch_from_evbuffer_socks_client(input, state, reason); + }) ELSE_IF_NO_BUFFEREVENT { + return fetch_from_buf_socks_client(conn->inbuf, state, reason); + } +} + /** Call this from connection_*_process_inbuf() to advance the proxy * handshake. * @@ -1665,17 +1709,17 @@ connection_read_proxy_handshake(connection_t *conn) break; case PROXY_SOCKS4_WANT_CONNECT_OK: - ret = fetch_from_buf_socks_client(conn->inbuf, - conn->proxy_state, - &reason); + ret = connection_fetch_from_buf_socks_client(conn, + conn->proxy_state, + &reason); if (ret == 1) conn->proxy_state = PROXY_CONNECTED; break; case PROXY_SOCKS5_WANT_AUTH_METHOD_NONE: - ret = fetch_from_buf_socks_client(conn->inbuf, - conn->proxy_state, - &reason); + ret = connection_fetch_from_buf_socks_client(conn, + conn->proxy_state, + &reason); /* no auth needed, do connect */ if (ret == 1) { connection_send_socks5_connect(conn); @@ -1684,9 +1728,9 @@ connection_read_proxy_handshake(connection_t *conn) break; case PROXY_SOCKS5_WANT_AUTH_METHOD_RFC1929: - ret = fetch_from_buf_socks_client(conn->inbuf, - conn->proxy_state, - &reason); + ret = connection_fetch_from_buf_socks_client(conn, + conn->proxy_state, + &reason); /* send auth if needed, otherwise do connect */ if (ret == 1) { @@ -1721,9 +1765,9 @@ connection_read_proxy_handshake(connection_t *conn) break; case PROXY_SOCKS5_WANT_AUTH_RFC1929_OK: - ret = fetch_from_buf_socks_client(conn->inbuf, - conn->proxy_state, - &reason); + ret = connection_fetch_from_buf_socks_client(conn, + conn->proxy_state, + &reason); /* send the connect request */ if (ret == 1) { connection_send_socks5_connect(conn); @@ -1732,9 +1776,9 @@ connection_read_proxy_handshake(connection_t *conn) break; case PROXY_SOCKS5_WANT_CONNECT_OK: - ret = fetch_from_buf_socks_client(conn->inbuf, - conn->proxy_state, - &reason); + ret = connection_fetch_from_buf_socks_client(conn, + conn->proxy_state, + &reason); if (ret == 1) conn->proxy_state = PROXY_CONNECTED; break; @@ -1767,172 +1811,124 @@ connection_read_proxy_handshake(connection_t *conn) return ret; } -/** - * Launch any configured listener connections of type <b>type</b>. (A - * listener is configured if <b>port_option</b> is non-zero. If any - * ListenAddress configuration options are given in <b>cfg</b>, create a - * connection binding to each one. Otherwise, create a single - * connection binding to the address <b>default_addr</b>.) +/** Given a list of listener connections in <b>old_conns</b>, and list of + * port_cfg_t entries in <b>ports</b>, open a new listener for every port in + * <b>ports</b> that does not already have a listener in <b>old_conns</b>. * - * Only launch the listeners of this type that are not already open, and - * only close listeners that are no longer wanted. Existing listeners - * that are still configured are not touched. + * Remove from <b>old_conns</b> every connection that has a corresponding + * entry in <b>ports</b>. Add to <b>new_conns</b> new every connection we + * launch. * - * If <b>disable_all_conns</b> is set, then never open new conns, and - * close the existing ones. + * If <b>control_listeners_only</b> is true, then we only open control + * listeners, and we do not remove any noncontrol listeners from old_conns. * - * Add all old conns that should be closed to <b>replaced_conns</b>. - * Add all new connections to <b>new_conns</b>. - */ + * Return 0 on success, -1 on failure. + **/ static int -retry_listeners(int type, config_line_t *cfg, - int port_option, const char *default_addr, - smartlist_t *replaced_conns, - smartlist_t *new_conns, - int disable_all_conns, - int socket_family) -{ - smartlist_t *launch = smartlist_create(), *conns; - int free_launch_elts = 1; - int r; - config_line_t *c; - connection_t *conn; - config_line_t *line; - - tor_assert(socket_family == AF_INET || socket_family == AF_UNIX); +retry_listener_ports(smartlist_t *old_conns, + const smartlist_t *ports, + smartlist_t *new_conns, + int control_listeners_only) +{ + smartlist_t *launch = smartlist_new(); + int r = 0; - if (cfg && port_option) { - for (c = cfg; c; c = c->next) { - smartlist_add(launch, c); - } - free_launch_elts = 0; - } else if (port_option) { - line = tor_malloc_zero(sizeof(config_line_t)); - line->key = tor_strdup(""); - line->value = tor_strdup(default_addr); - smartlist_add(launch, line); + if (control_listeners_only) { + SMARTLIST_FOREACH(ports, port_cfg_t *, p, { + if (p->type == CONN_TYPE_CONTROL_LISTENER) + smartlist_add(launch, p); + }); + } else { + smartlist_add_all(launch, ports); } - /* - SMARTLIST_FOREACH(launch, config_line_t *, l, - log_fn(LOG_NOTICE, "#%s#%s", l->key, l->value)); - */ + /* Iterate through old_conns, comparing it to launch: remove from both lists + * each pair of elements that corresponds to the same port. */ + SMARTLIST_FOREACH_BEGIN(old_conns, connection_t *, conn) { + const port_cfg_t *found_port = NULL; - conns = get_connection_array(); - SMARTLIST_FOREACH(conns, connection_t *, conn, - { - if (conn->type != type || - conn->socket_family != socket_family || - conn->marked_for_close) - continue; /* Okay, so this is a listener. Is it configured? */ - line = NULL; - SMARTLIST_FOREACH(launch, config_line_t *, wanted, - { - char *address=NULL; - uint16_t port; - switch (socket_family) { - case AF_INET: - if (!parse_addr_port(LOG_WARN, - wanted->value, &address, NULL, &port)) { - int addr_matches = !strcasecmp(address, conn->address); - int port_matches; - tor_free(address); - if (port) { - /* The Listener line has a port */ - port_matches = (port == conn->port); - } else if (port_option == CFG_AUTO_PORT) { - /* The Listener line has no port, and the Port line is "auto". - * "auto" matches anything; transitions from any port to - * "auto" succeed. */ - port_matches = 1; - } else { - /* The Listener line has no port, and the Port line is "auto". - * "auto" matches anything; transitions from any port to - * "auto" succeed. */ - port_matches = (port_option == conn->port); - } - if (port_matches && addr_matches) { - line = wanted; - break; - } - } - break; - case AF_UNIX: - if (!strcasecmp(wanted->value, conn->address)) { - line = wanted; - break; - } - break; - default: - tor_assert(0); + SMARTLIST_FOREACH_BEGIN(launch, const port_cfg_t *, wanted) { + if (conn->type != wanted->type) + continue; + if ((conn->socket_family != AF_UNIX && wanted->is_unix_addr) || + (conn->socket_family == AF_UNIX && ! wanted->is_unix_addr)) + continue; + + if (wanted->no_listen) + continue; /* We don't want to open a listener for this one */ + + if (wanted->is_unix_addr) { + if (conn->socket_family == AF_UNIX && + !strcmp(wanted->unix_addr, conn->address)) { + found_port = wanted; + break; } - }); - if (!line || disable_all_conns) { - /* This one isn't configured. Close it. */ - log_notice(LD_NET, "Closing no-longer-configured %s on %s:%d", - conn_type_to_string(type), conn->address, conn->port); - if (replaced_conns) { - smartlist_add(replaced_conns, conn); } else { - connection_close_immediate(conn); - connection_mark_for_close(conn); + int port_matches; + if (wanted->port == CFG_AUTO_PORT) { + port_matches = 1; + } else { + port_matches = (wanted->port == conn->port); + } + if (port_matches && tor_addr_eq(&wanted->addr, &conn->addr)) { + found_port = wanted; + break; + } } - } else { - /* It's configured; we don't need to launch it. */ -// log_debug(LD_NET, "Already have %s on %s:%d", -// conn_type_to_string(type), conn->address, conn->port); - smartlist_remove(launch, line); - if (free_launch_elts) - config_free_lines(line); + } SMARTLIST_FOREACH_END(wanted); + + if (found_port) { + /* This listener is already running; we don't need to launch it. */ + //log_debug(LD_NET, "Already have %s on %s:%d", + // conn_type_to_string(found_port->type), conn->address, conn->port); + smartlist_remove(launch, found_port); + /* And we can remove the connection from old_conns too. */ + SMARTLIST_DEL_CURRENT(old_conns, conn); } - }); + } SMARTLIST_FOREACH_END(conn); /* Now open all the listeners that are configured but not opened. */ - r = 0; - if (!disable_all_conns) { - SMARTLIST_FOREACH_BEGIN(launch, config_line_t *, cfg_line) { - char *address = NULL; - struct sockaddr *listensockaddr; - socklen_t listensocklen = 0; - - switch (socket_family) { - case AF_INET: - listensockaddr = (struct sockaddr *) - create_inet_sockaddr(cfg_line->value, - port_option, - &address, &listensocklen); - break; - case AF_UNIX: - listensockaddr = (struct sockaddr *) - create_unix_sockaddr(cfg_line->value, - &address, &listensocklen); - break; - default: - tor_assert(0); - } + SMARTLIST_FOREACH_BEGIN(launch, const port_cfg_t *, port) { + struct sockaddr *listensockaddr; + socklen_t listensocklen = 0; + char *address=NULL; + connection_t *conn; + int real_port = port->port == CFG_AUTO_PORT ? 0 : port->port; + tor_assert(real_port <= UINT16_MAX); + if (port->no_listen) + continue; + + if (port->is_unix_addr) { + listensockaddr = (struct sockaddr *) + create_unix_sockaddr(port->unix_addr, + &address, &listensocklen); + } else { + listensockaddr = tor_malloc(sizeof(struct sockaddr_storage)); + listensocklen = tor_addr_to_sockaddr(&port->addr, + real_port, + listensockaddr, + sizeof(struct sockaddr_storage)); + address = tor_dup_addr(&port->addr); + } - if (listensockaddr) { - conn = connection_create_listener(listensockaddr, listensocklen, - type, address); - tor_free(listensockaddr); - tor_free(address); - } else - conn = NULL; + if (listensockaddr) { + conn = connection_listener_new(listensockaddr, listensocklen, + port->type, address, port); + tor_free(listensockaddr); + tor_free(address); + } else { + conn = NULL; + } - if (!conn) { - r = -1; - } else { - if (new_conns) - smartlist_add(new_conns, conn); - } - } SMARTLIST_FOREACH_END(cfg_line); - } + if (!conn) { + r = -1; + } else { + if (new_conns) + smartlist_add(new_conns, conn); + } + } SMARTLIST_FOREACH_END(port); - if (free_launch_elts) { - SMARTLIST_FOREACH(launch, config_line_t *, cfg_line, - config_free_lines(cfg_line)); - } smartlist_free(launch); return r; @@ -1944,59 +1940,47 @@ retry_listeners(int type, config_line_t *cfg, * * Add all old conns that should be closed to <b>replaced_conns</b>. * Add all new connections to <b>new_conns</b>. + * + * If <b>close_all_noncontrol</b> is true, then we only open control + * listeners, and we close all other listeners. */ int retry_all_listeners(smartlist_t *replaced_conns, - smartlist_t *new_conns) + smartlist_t *new_conns, int close_all_noncontrol) { - or_options_t *options = get_options(); + smartlist_t *listeners = smartlist_new(); + const or_options_t *options = get_options(); int retval = 0; const uint16_t old_or_port = router_get_advertised_or_port(options); const uint16_t old_dir_port = router_get_advertised_dir_port(options, 0); - if (retry_listeners(CONN_TYPE_OR_LISTENER, options->ORListenAddress, - options->ORPort, "0.0.0.0", - replaced_conns, new_conns, options->ClientOnly, - AF_INET)<0) - retval = -1; - if (retry_listeners(CONN_TYPE_DIR_LISTENER, options->DirListenAddress, - options->DirPort, "0.0.0.0", - replaced_conns, new_conns, options->ClientOnly, - AF_INET)<0) - retval = -1; - if (retry_listeners(CONN_TYPE_AP_LISTENER, options->SocksListenAddress, - options->SocksPort, "127.0.0.1", - replaced_conns, new_conns, 0, - AF_INET)<0) - retval = -1; - if (retry_listeners(CONN_TYPE_AP_TRANS_LISTENER, options->TransListenAddress, - options->TransPort, "127.0.0.1", - replaced_conns, new_conns, 0, - AF_INET)<0) - retval = -1; - if (retry_listeners(CONN_TYPE_AP_NATD_LISTENER, options->NATDListenAddress, - options->NATDPort, "127.0.0.1", - replaced_conns, new_conns, 0, - AF_INET)<0) - retval = -1; - if (retry_listeners(CONN_TYPE_AP_DNS_LISTENER, options->DNSListenAddress, - options->DNSPort, "127.0.0.1", - replaced_conns, new_conns, 0, - AF_INET)<0) + SMARTLIST_FOREACH_BEGIN(get_connection_array(), connection_t *, conn) { + if (connection_is_listener(conn) && !conn->marked_for_close) + smartlist_add(listeners, conn); + } SMARTLIST_FOREACH_END(conn); + + if (retry_listener_ports(listeners, + get_configured_ports(), + new_conns, + close_all_noncontrol) < 0) retval = -1; - if (retry_listeners(CONN_TYPE_CONTROL_LISTENER, - options->ControlListenAddress, - options->ControlPort, "127.0.0.1", - replaced_conns, new_conns, 0, - AF_INET)<0) - return -1; - if (retry_listeners(CONN_TYPE_CONTROL_LISTENER, - options->ControlSocket, - options->ControlSocket ? 1 : 0, NULL, - replaced_conns, new_conns, 0, - AF_UNIX)<0) - return -1; + /* Any members that were still in 'listeners' don't correspond to + * any configured port. Kill 'em. */ + SMARTLIST_FOREACH_BEGIN(listeners, connection_t *, conn) { + log_notice(LD_NET, "Closing no-longer-configured %s on %s:%d", + conn_type_to_string(conn->type), conn->address, conn->port); + if (replaced_conns) { + smartlist_add(replaced_conns, conn); + } else { + connection_close_immediate(conn); + connection_mark_for_close(conn); + } + } SMARTLIST_FOREACH_END(conn); + + smartlist_free(listeners); + + /* XXXprop186 should take all advertised ports into account */ if (old_or_port != router_get_advertised_or_port(options) || old_dir_port != router_get_advertised_dir_port(options, 0)) { /* Our chosen ORPort or DirPort is not what it used to be: the @@ -2009,20 +1993,66 @@ retry_all_listeners(smartlist_t *replaced_conns, return retval; } -/** Return 1 if we should apply rate limiting to <b>conn</b>, - * and 0 otherwise. Right now this just checks if it's an internal - * IP address or an internal connection. */ +/** Mark every listener of type other than CONTROL_LISTENER to be closed. */ +void +connection_mark_all_noncontrol_listeners(void) +{ + SMARTLIST_FOREACH_BEGIN(get_connection_array(), connection_t *, conn) { + if (conn->marked_for_close) + continue; + if (conn->type == CONN_TYPE_CONTROL_LISTENER) + continue; + if (connection_is_listener(conn)) + connection_mark_for_close(conn); + } SMARTLIST_FOREACH_END(conn); +} + +/** Mark every external connection not used for controllers for close. */ +void +connection_mark_all_noncontrol_connections(void) +{ + SMARTLIST_FOREACH_BEGIN(get_connection_array(), connection_t *, conn) { + if (conn->marked_for_close) + continue; + switch (conn->type) { + case CONN_TYPE_CPUWORKER: + case CONN_TYPE_CONTROL_LISTENER: + case CONN_TYPE_CONTROL: + break; + case CONN_TYPE_AP: + connection_mark_unattached_ap(TO_ENTRY_CONN(conn), + END_STREAM_REASON_HIBERNATING); + break; + default: + connection_mark_for_close(conn); + break; + } + } SMARTLIST_FOREACH_END(conn); +} + +/** Return 1 if we should apply rate limiting to <b>conn</b>, and 0 + * otherwise. + * Right now this just checks if it's an internal IP address or an + * internal connection. We also check if the connection uses pluggable + * transports, since we should then limit it even if it comes from an + * internal IP address. */ static int connection_is_rate_limited(connection_t *conn) { - if (conn->linked || /* internal connection */ - tor_addr_family(&conn->addr) == AF_UNSPEC || /* no address */ - tor_addr_is_internal(&conn->addr, 0)) /* internal address */ - return 0; + const or_options_t *options = get_options(); + if (conn->linked) + return 0; /* Internal connection */ + else if (! options->CountPrivateBandwidth && + (tor_addr_family(&conn->addr) == AF_UNSPEC || /* no address */ + tor_addr_is_internal(&conn->addr, 0))) + return 0; /* Internal address */ else return 1; } +#ifdef USE_BUFFEREVENTS +static struct bufferevent_rate_limit_group *global_rate_limit = NULL; +#else extern int global_read_bucket, global_write_bucket; extern int global_relayed_read_bucket, global_relayed_write_bucket; @@ -2030,11 +2060,13 @@ extern int global_relayed_read_bucket, global_relayed_write_bucket; * we are likely to run dry again this second, so be stingy with the * tokens we just put in. */ static int write_buckets_empty_last_second = 0; +#endif /** How many seconds of no active local circuits will make the * connection revert to the "relayed" bandwidth class? */ #define CLIENT_IDLE_TIME_FOR_PRIORITY 30 +#ifndef USE_BUFFEREVENTS /** Return 1 if <b>conn</b> should use tokens from the "relayed" * bandwidth rates, else 0. Currently, only OR conns with bandwidth * class 1, and directory conns that are serving data out, count. @@ -2145,6 +2177,20 @@ connection_bucket_write_limit(connection_t *conn, time_t now) return connection_bucket_round_robin(base, priority, global_bucket, conn_bucket); } +#else +static ssize_t +connection_bucket_read_limit(connection_t *conn, time_t now) +{ + (void) now; + return bufferevent_get_max_to_read(conn->bufev); +} +ssize_t +connection_bucket_write_limit(connection_t *conn, time_t now) +{ + (void) now; + return bufferevent_get_max_to_write(conn->bufev); +} +#endif /** Return 1 if the global write buckets are low enough that we * shouldn't send <b>attempt</b> bytes of low-priority directory stuff @@ -2169,8 +2215,12 @@ connection_bucket_write_limit(connection_t *conn, time_t now) int global_write_bucket_low(connection_t *conn, size_t attempt, int priority) { +#ifdef USE_BUFFEREVENTS + ssize_t smaller_bucket = bufferevent_get_max_to_write(conn->bufev); +#else int smaller_bucket = global_write_bucket < global_relayed_write_bucket ? global_write_bucket : global_relayed_write_bucket; +#endif if (authdir_mode(get_options()) && priority>1) return 0; /* there's always room to answer v2 if we're an auth dir */ @@ -2180,12 +2230,14 @@ global_write_bucket_low(connection_t *conn, size_t attempt, int priority) if (smaller_bucket < (int)attempt) return 1; /* not enough space no matter the priority */ +#ifndef USE_BUFFEREVENTS if (write_buckets_empty_last_second) return 1; /* we're already hitting our limits, no more please */ +#endif if (priority == 1) { /* old-style v1 query */ /* Could we handle *two* of these requests within the next two seconds? */ - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int64_t can_write = (int64_t)smaller_bucket + 2*(options->RelayBandwidthRate ? options->RelayBandwidthRate : options->BandwidthRate); @@ -2197,23 +2249,13 @@ global_write_bucket_low(connection_t *conn, size_t attempt, int priority) return 0; } -/** We just read <b>num_read</b> and wrote <b>num_written</b> bytes - * onto <b>conn</b>. Decrement buckets appropriately. */ +/** Helper: adjusts our bandwidth history and informs the controller as + * appropriate, given that we have just read <b>num_read</b> bytes and written + * <b>num_written</b> bytes on <b>conn</b>. */ static void -connection_buckets_decrement(connection_t *conn, time_t now, - size_t num_read, size_t num_written) +record_num_bytes_transferred_impl(connection_t *conn, + time_t now, size_t num_read, size_t num_written) { - if (num_written >= INT_MAX || num_read >= INT_MAX) { - log_err(LD_BUG, "Value out of range. num_read=%lu, num_written=%lu, " - "connection type=%s, state=%s", - (unsigned long)num_read, (unsigned long)num_written, - conn_type_to_string(conn->type), - conn_state_to_string(conn->type, conn->state)); - if (num_written >= INT_MAX) num_written = 1; - if (num_read >= INT_MAX) num_read = 1; - tor_fragile_assert(); - } - /* Count bytes of answering direct and tunneled directory requests */ if (conn->type == CONN_TYPE_DIR && conn->purpose == DIR_PURPOSE_SERVER) { if (num_read > 0) @@ -2224,6 +2266,11 @@ connection_buckets_decrement(connection_t *conn, time_t now, if (!connection_is_rate_limited(conn)) return; /* local IPs are free */ + + if (conn->type == CONN_TYPE_OR) + rep_hist_note_or_conn_bytes(conn->global_identifier, num_read, + num_written, now); + if (num_read > 0) { rep_hist_note_bytes_read(num_read, now); } @@ -2232,6 +2279,53 @@ connection_buckets_decrement(connection_t *conn, time_t now, } if (conn->type == CONN_TYPE_EXIT) rep_hist_note_exit_bytes(conn->port, num_written, num_read); +} + +#ifdef USE_BUFFEREVENTS +/** Wrapper around fetch_from_(buf/evbuffer)_socks_client: see those functions + * for documentation of its behavior. */ +static void +record_num_bytes_transferred(connection_t *conn, + time_t now, size_t num_read, size_t num_written) +{ + /* XXX024 check if this is necessary */ + if (num_written >= INT_MAX || num_read >= INT_MAX) { + log_err(LD_BUG, "Value out of range. num_read=%lu, num_written=%lu, " + "connection type=%s, state=%s", + (unsigned long)num_read, (unsigned long)num_written, + conn_type_to_string(conn->type), + conn_state_to_string(conn->type, conn->state)); + if (num_written >= INT_MAX) num_written = 1; + if (num_read >= INT_MAX) num_read = 1; + tor_fragile_assert(); + } + + record_num_bytes_transferred_impl(conn,now,num_read,num_written); +} +#endif + +#ifndef USE_BUFFEREVENTS +/** We just read <b>num_read</b> and wrote <b>num_written</b> bytes + * onto <b>conn</b>. Decrement buckets appropriately. */ +static void +connection_buckets_decrement(connection_t *conn, time_t now, + size_t num_read, size_t num_written) +{ + if (num_written >= INT_MAX || num_read >= INT_MAX) { + log_err(LD_BUG, "Value out of range. num_read=%lu, num_written=%lu, " + "connection type=%s, state=%s", + (unsigned long)num_read, (unsigned long)num_written, + conn_type_to_string(conn->type), + conn_state_to_string(conn->type, conn->state)); + if (num_written >= INT_MAX) num_written = 1; + if (num_read >= INT_MAX) num_read = 1; + tor_fragile_assert(); + } + + record_num_bytes_transferred_impl(conn, now, num_read, num_written); + + if (!connection_is_rate_limited(conn)) + return; /* local IPs are free */ if (connection_counts_as_relayed_traffic(conn, now)) { global_relayed_read_bucket -= (int)num_read; @@ -2297,7 +2391,7 @@ connection_consider_empty_write_buckets(connection_t *conn) void connection_bucket_init(void) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); /* start it at max traffic */ global_read_bucket = (int)options->BandwidthBurst; global_write_bucket = (int)options->BandwidthBurst; @@ -2310,22 +2404,23 @@ connection_bucket_init(void) } } -/** Refill a single <b>bucket</b> called <b>name</b> with bandwidth rate - * <b>rate</b> and bandwidth burst <b>burst</b>, assuming that - * <b>seconds_elapsed</b> seconds have passed since the last call. - **/ +/** Refill a single <b>bucket</b> called <b>name</b> with bandwidth rate per + * second <b>rate</b> and bandwidth burst <b>burst</b>, assuming that + * <b>milliseconds_elapsed</b> milliseconds have passed since the last + * call. */ static void connection_bucket_refill_helper(int *bucket, int rate, int burst, - int seconds_elapsed, const char *name) + int milliseconds_elapsed, + const char *name) { int starting_bucket = *bucket; - if (starting_bucket < burst && seconds_elapsed) { - if (((burst - starting_bucket)/seconds_elapsed) < rate) { + if (starting_bucket < burst && milliseconds_elapsed > 0) { + int64_t incr = (((int64_t)rate) * milliseconds_elapsed) / 1000; + if ((burst - starting_bucket) < incr) { *bucket = burst; /* We would overflow the bucket; just set it to * the maximum. */ } else { - int incr = rate*seconds_elapsed; - *bucket += incr; + *bucket += (int)incr; if (*bucket > burst || *bucket < starting_bucket) { /* If we overflow the burst, or underflow our starting bucket, * cap the bucket value to burst. */ @@ -2338,60 +2433,66 @@ connection_bucket_refill_helper(int *bucket, int rate, int burst, } } -/** A second has rolled over; increment buckets appropriately. */ +/** Time has passed; increment buckets appropriately. */ void -connection_bucket_refill(int seconds_elapsed, time_t now) +connection_bucket_refill(int milliseconds_elapsed, time_t now) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); smartlist_t *conns = get_connection_array(); - int relayrate, relayburst; + int bandwidthrate, bandwidthburst, relayrate, relayburst; + + bandwidthrate = (int)options->BandwidthRate; + bandwidthburst = (int)options->BandwidthBurst; if (options->RelayBandwidthRate) { relayrate = (int)options->RelayBandwidthRate; relayburst = (int)options->RelayBandwidthBurst; } else { - relayrate = (int)options->BandwidthRate; - relayburst = (int)options->BandwidthBurst; + relayrate = bandwidthrate; + relayburst = bandwidthburst; } - tor_assert(seconds_elapsed >= 0); + tor_assert(milliseconds_elapsed >= 0); write_buckets_empty_last_second = global_relayed_write_bucket <= 0 || global_write_bucket <= 0; /* refill the global buckets */ connection_bucket_refill_helper(&global_read_bucket, - (int)options->BandwidthRate, - (int)options->BandwidthBurst, - seconds_elapsed, "global_read_bucket"); + bandwidthrate, bandwidthburst, + milliseconds_elapsed, + "global_read_bucket"); connection_bucket_refill_helper(&global_write_bucket, - (int)options->BandwidthRate, - (int)options->BandwidthBurst, - seconds_elapsed, "global_write_bucket"); + bandwidthrate, bandwidthburst, + milliseconds_elapsed, + "global_write_bucket"); connection_bucket_refill_helper(&global_relayed_read_bucket, - relayrate, relayburst, seconds_elapsed, + relayrate, relayburst, + milliseconds_elapsed, "global_relayed_read_bucket"); connection_bucket_refill_helper(&global_relayed_write_bucket, - relayrate, relayburst, seconds_elapsed, + relayrate, relayburst, + milliseconds_elapsed, "global_relayed_write_bucket"); /* refill the per-connection buckets */ - SMARTLIST_FOREACH(conns, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { if (connection_speaks_cells(conn)) { or_connection_t *or_conn = TO_OR_CONN(conn); + int orbandwidthrate = or_conn->bandwidthrate; + int orbandwidthburst = or_conn->bandwidthburst; if (connection_bucket_should_increase(or_conn->read_bucket, or_conn)) { connection_bucket_refill_helper(&or_conn->read_bucket, - or_conn->bandwidthrate, - or_conn->bandwidthburst, - seconds_elapsed, + orbandwidthrate, + orbandwidthburst, + milliseconds_elapsed, "or_conn->read_bucket"); } if (connection_bucket_should_increase(or_conn->write_bucket, or_conn)) { connection_bucket_refill_helper(&or_conn->write_bucket, - or_conn->bandwidthrate, - or_conn->bandwidthburst, - seconds_elapsed, + orbandwidthrate, + orbandwidthburst, + milliseconds_elapsed, "or_conn->write_bucket"); } } @@ -2422,7 +2523,7 @@ connection_bucket_refill(int seconds_elapsed, time_t now) conn->write_blocked_on_bw = 0; connection_start_writing(conn); } - }); + } SMARTLIST_FOREACH_END(conn); } /** Is the <b>bucket</b> for connection <b>conn</b> low enough that we @@ -2440,13 +2541,95 @@ connection_bucket_should_increase(int bucket, or_connection_t *conn) return 1; } +#else +static void +connection_buckets_decrement(connection_t *conn, time_t now, + size_t num_read, size_t num_written) +{ + (void) conn; + (void) now; + (void) num_read; + (void) num_written; + /* Libevent does this for us. */ +} + +void +connection_bucket_refill(int seconds_elapsed, time_t now) +{ + (void) seconds_elapsed; + (void) now; + /* Libevent does this for us. */ +} +void +connection_bucket_init(void) +{ + const or_options_t *options = get_options(); + const struct timeval *tick = tor_libevent_get_one_tick_timeout(); + struct ev_token_bucket_cfg *bucket_cfg; + + uint64_t rate, burst; + if (options->RelayBandwidthRate) { + rate = options->RelayBandwidthRate; + burst = options->RelayBandwidthBurst; + } else { + rate = options->BandwidthRate; + burst = options->BandwidthBurst; + } + + /* This can't overflow, since TokenBucketRefillInterval <= 1000, + * and rate started out less than INT32_MAX. */ + rate = (rate * options->TokenBucketRefillInterval) / 1000; + + bucket_cfg = ev_token_bucket_cfg_new((uint32_t)rate, (uint32_t)burst, + (uint32_t)rate, (uint32_t)burst, + tick); + + if (!global_rate_limit) { + global_rate_limit = + bufferevent_rate_limit_group_new(tor_libevent_get_base(), bucket_cfg); + } else { + bufferevent_rate_limit_group_set_cfg(global_rate_limit, bucket_cfg); + } + ev_token_bucket_cfg_free(bucket_cfg); +} + +void +connection_get_rate_limit_totals(uint64_t *read_out, uint64_t *written_out) +{ + if (global_rate_limit == NULL) { + *read_out = *written_out = 0; + } else { + bufferevent_rate_limit_group_get_totals( + global_rate_limit, read_out, written_out); + } +} + +/** Perform whatever operations are needed on <b>conn</b> to enable + * rate-limiting. */ +void +connection_enable_rate_limiting(connection_t *conn) +{ + if (conn->bufev) { + if (!global_rate_limit) + connection_bucket_init(); + tor_add_bufferevent_to_rate_limit_group(conn->bufev, global_rate_limit); + } +} + +static void +connection_consider_empty_write_buckets(connection_t *conn) +{ + (void) conn; +} +static void +connection_consider_empty_read_buckets(connection_t *conn) +{ + (void) conn; +} +#endif /** Read bytes from conn-\>s and process them. * - * This function gets called from conn_read() in main.c, either - * when poll() has declared that conn wants to read, or (for OR conns) - * when there are pending TLS bytes. - * * It calls connection_read_to_buf() to bring in any new bytes, * and then calls connection_process_inbuf() to process them. * @@ -2498,8 +2681,10 @@ connection_handle_read_impl(connection_t *conn) if (CONN_IS_EDGE(conn)) { edge_connection_t *edge_conn = TO_EDGE_CONN(conn); connection_edge_end_errno(edge_conn); - if (edge_conn->socks_request) /* broken, don't send a socks reply back */ - edge_conn->socks_request->has_finished = 1; + if (conn->type == CONN_TYPE_AP && TO_ENTRY_CONN(conn)->socks_request) { + /* broken, don't send a socks reply back */ + TO_ENTRY_CONN(conn)->socks_request->has_finished = 1; + } } connection_close_immediate(conn); /* Don't flush; connection is dead. */ connection_mark_for_close(conn); @@ -2553,6 +2738,7 @@ connection_handle_read_impl(connection_t *conn) return 0; } +/* DOCDOC connection_handle_read */ int connection_handle_read(connection_t *conn) { @@ -2639,7 +2825,20 @@ connection_read_to_buf(connection_t *conn, ssize_t *max_to_read, case TOR_TLS_WANTWRITE: connection_start_writing(conn); return 0; - case TOR_TLS_WANTREAD: /* we're already reading */ + case TOR_TLS_WANTREAD: + if (conn->in_connection_handle_write) { + /* We've been invoked from connection_handle_write, because we're + * waiting for a TLS renegotiation, the renegotiation started, and + * SSL_read returned WANTWRITE. But now SSL_read is saying WANTREAD + * again. Stop waiting for write events now, or else we'll + * busy-loop until data arrives for us to read. */ + connection_stop_writing(conn); + if (!connection_is_reading(conn)) + connection_start_reading(conn); + } + /* we're already reading, one hopes */ + result = 0; + break; case TOR_TLS_DONE: /* no data read, so nothing to process */ result = 0; break; /* so we call bucket_decrement below */ @@ -2694,7 +2893,7 @@ connection_read_to_buf(connection_t *conn, ssize_t *max_to_read, } if (n_read > 0) { - /* change *max_to_read */ + /* change *max_to_read */ *max_to_read = at_most - n_read; /* Update edge_conn->n_read */ @@ -2726,11 +2925,215 @@ connection_read_to_buf(connection_t *conn, ssize_t *max_to_read, return 0; } +#ifdef USE_BUFFEREVENTS +/* XXXX These generic versions could be simplified by making them + type-specific */ + +/** Callback: Invoked whenever bytes are added to or drained from an input + * evbuffer. Used to track the number of bytes read. */ +static void +evbuffer_inbuf_callback(struct evbuffer *buf, + const struct evbuffer_cb_info *info, void *arg) +{ + connection_t *conn = arg; + (void) buf; + /* XXXX These need to get real counts on the non-nested TLS case. - NM */ + if (info->n_added) { + time_t now = approx_time(); + conn->timestamp_lastread = now; + record_num_bytes_transferred(conn, now, info->n_added, 0); + connection_consider_empty_read_buckets(conn); + if (conn->type == CONN_TYPE_AP) { + edge_connection_t *edge_conn = TO_EDGE_CONN(conn); + /*XXXX024 check for overflow*/ + edge_conn->n_read += (int)info->n_added; + } + } +} + +/** Callback: Invoked whenever bytes are added to or drained from an output + * evbuffer. Used to track the number of bytes written. */ +static void +evbuffer_outbuf_callback(struct evbuffer *buf, + const struct evbuffer_cb_info *info, void *arg) +{ + connection_t *conn = arg; + (void)buf; + if (info->n_deleted) { + time_t now = approx_time(); + conn->timestamp_lastwritten = now; + record_num_bytes_transferred(conn, now, 0, info->n_deleted); + connection_consider_empty_write_buckets(conn); + if (conn->type == CONN_TYPE_AP) { + edge_connection_t *edge_conn = TO_EDGE_CONN(conn); + /*XXXX024 check for overflow*/ + edge_conn->n_written += (int)info->n_deleted; + } + } +} + +/** Callback: invoked whenever a bufferevent has read data. */ +void +connection_handle_read_cb(struct bufferevent *bufev, void *arg) +{ + connection_t *conn = arg; + (void) bufev; + if (!conn->marked_for_close) { + if (connection_process_inbuf(conn, 1)<0) /* XXXX Always 1? */ + if (!conn->marked_for_close) + connection_mark_for_close(conn); + } +} + +/** Callback: invoked whenever a bufferevent has written data. */ +void +connection_handle_write_cb(struct bufferevent *bufev, void *arg) +{ + connection_t *conn = arg; + struct evbuffer *output; + if (connection_flushed_some(conn)<0) { + if (!conn->marked_for_close) + connection_mark_for_close(conn); + return; + } + + output = bufferevent_get_output(bufev); + if (!evbuffer_get_length(output)) { + connection_finished_flushing(conn); + if (conn->marked_for_close && conn->hold_open_until_flushed) { + conn->hold_open_until_flushed = 0; + if (conn->linked) { + /* send eof */ + bufferevent_flush(conn->bufev, EV_WRITE, BEV_FINISHED); + } + } + } +} + +/** Callback: invoked whenever a bufferevent has had an event (like a + * connection, or an eof, or an error) occur. */ +void +connection_handle_event_cb(struct bufferevent *bufev, short event, void *arg) +{ + connection_t *conn = arg; + (void) bufev; + if (conn->marked_for_close) + return; + + if (event & BEV_EVENT_CONNECTED) { + tor_assert(connection_state_is_connecting(conn)); + if (connection_finished_connecting(conn)<0) + return; + } + if (event & BEV_EVENT_EOF) { + if (!conn->marked_for_close) { + conn->inbuf_reached_eof = 1; + if (connection_reached_eof(conn)<0) + return; + } + } + if (event & BEV_EVENT_ERROR) { + int socket_error = evutil_socket_geterror(conn->s); + if (conn->type == CONN_TYPE_OR && + conn->state == OR_CONN_STATE_CONNECTING) { + connection_or_connect_failed(TO_OR_CONN(conn), + errno_to_orconn_end_reason(socket_error), + tor_socket_strerror(socket_error)); + } else if (CONN_IS_EDGE(conn)) { + edge_connection_t *edge_conn = TO_EDGE_CONN(conn); + if (!edge_conn->edge_has_sent_end) + connection_edge_end_errno(edge_conn); + if (conn->type == CONN_TYPE_AP && TO_ENTRY_CONN(conn)->socks_request) { + /* broken, don't send a socks reply back */ + TO_ENTRY_CONN(conn)->socks_request->has_finished = 1; + } + } + connection_close_immediate(conn); /* Connection is dead. */ + if (!conn->marked_for_close) + connection_mark_for_close(conn); + } +} + +/** Set up the generic callbacks for the bufferevent on <b>conn</b>. */ +void +connection_configure_bufferevent_callbacks(connection_t *conn) +{ + struct bufferevent *bufev; + struct evbuffer *input, *output; + tor_assert(conn->bufev); + bufev = conn->bufev; + bufferevent_setcb(bufev, + connection_handle_read_cb, + connection_handle_write_cb, + connection_handle_event_cb, + conn); + /* Set a fairly high write low-watermark so that we get the write callback + called whenever data is written to bring us under 128K. Leave the + high-watermark at 0. + */ + bufferevent_setwatermark(bufev, EV_WRITE, 128*1024, 0); + + input = bufferevent_get_input(bufev); + output = bufferevent_get_output(bufev); + evbuffer_add_cb(input, evbuffer_inbuf_callback, conn); + evbuffer_add_cb(output, evbuffer_outbuf_callback, conn); +} +#endif + /** A pass-through to fetch_from_buf. */ int connection_fetch_from_buf(char *string, size_t len, connection_t *conn) { - return fetch_from_buf(string, len, conn->inbuf); + IF_HAS_BUFFEREVENT(conn, { + /* XXX overflow -seb */ + return (int)bufferevent_read(conn->bufev, string, len); + }) ELSE_IF_NO_BUFFEREVENT { + return fetch_from_buf(string, len, conn->inbuf); + } +} + +/** As fetch_from_buf_line(), but read from a connection's input buffer. */ +int +connection_fetch_from_buf_line(connection_t *conn, char *data, + size_t *data_len) +{ + IF_HAS_BUFFEREVENT(conn, { + int r; + size_t eol_len=0; + struct evbuffer *input = bufferevent_get_input(conn->bufev); + struct evbuffer_ptr ptr = + evbuffer_search_eol(input, NULL, &eol_len, EVBUFFER_EOL_LF); + if (ptr.pos == -1) + return 0; /* No EOL found. */ + if ((size_t)ptr.pos+eol_len >= *data_len) { + return -1; /* Too long */ + } + *data_len = ptr.pos+eol_len; + r = evbuffer_remove(input, data, ptr.pos+eol_len); + tor_assert(r >= 0); + data[ptr.pos+eol_len] = '\0'; + return 1; + }) ELSE_IF_NO_BUFFEREVENT { + return fetch_from_buf_line(conn->inbuf, data, data_len); + } +} + +/** As fetch_from_buf_http, but fetches from a conncetion's input buffer_t or + * its bufferevent as appropriate. */ +int +connection_fetch_from_buf_http(connection_t *conn, + char **headers_out, size_t max_headerlen, + char **body_out, size_t *body_used, + size_t max_bodylen, int force_complete) +{ + IF_HAS_BUFFEREVENT(conn, { + struct evbuffer *input = bufferevent_get_input(conn->bufev); + return fetch_from_evbuffer_http(input, headers_out, max_headerlen, + body_out, body_used, max_bodylen, force_complete); + }) ELSE_IF_NO_BUFFEREVENT { + return fetch_from_buf_http(conn->inbuf, headers_out, max_headerlen, + body_out, body_used, max_bodylen, force_complete); + } } /** Return conn-\>outbuf_flushlen: how many bytes conn wants to flush @@ -2794,8 +3197,7 @@ connection_handle_write_impl(connection_t *conn, int force) /* Sometimes, "writable" means "connected". */ if (connection_state_is_connecting(conn)) { if (getsockopt(conn->s, SOL_SOCKET, SO_ERROR, (void*)&e, &len) < 0) { - log_warn(LD_BUG, - "getsockopt() syscall failed?! Please report to tor-ops."); + log_warn(LD_BUG, "getsockopt() syscall failed"); if (CONN_IS_EDGE(conn)) connection_edge_end_errno(TO_EDGE_CONN(conn)); connection_mark_for_close(conn); @@ -2851,6 +3253,7 @@ connection_handle_write_impl(connection_t *conn, int force) /* If we just flushed the last bytes, check if this tunneled dir * request is done. */ + /* XXXX move this to flushed_some or finished_flushing -NM */ if (buf_datalen(conn->outbuf) == 0 && conn->dirreq_id) geoip_change_dirreq_state(conn->dirreq_id, DIRREQ_TUNNELED, DIRREQ_OR_CONN_BUFFER_FLUSHED); @@ -2906,6 +3309,7 @@ connection_handle_write_impl(connection_t *conn, int force) if (n_written && conn->type == CONN_TYPE_AP) { edge_connection_t *edge_conn = TO_EDGE_CONN(conn); + /* Check for overflow: */ if (PREDICT_LIKELY(UINT32_MAX - edge_conn->n_written > n_written)) edge_conn->n_written += (int)n_written; @@ -2940,15 +3344,37 @@ connection_handle_write_impl(connection_t *conn, int force) return 0; } +/* DOCDOC connection_handle_write */ int connection_handle_write(connection_t *conn, int force) { int res; tor_gettimeofday_cache_clear(); + conn->in_connection_handle_write = 1; res = connection_handle_write_impl(conn, force); + conn->in_connection_handle_write = 0; return res; } +/** + * Try to flush data that's waiting for a write on <b>conn</b>. Return + * -1 on failure, 0 on success. + * + * Don't use this function for regular writing; the buffers/bufferevents + * system should be good enough at scheduling writes there. Instead, this + * function is for cases when we're about to exit or something and we want + * to report it right away. + */ +int +connection_flush(connection_t *conn) +{ + IF_HAS_BUFFEREVENT(conn, { + int r = bufferevent_flush(conn->bufev, EV_WRITE, BEV_FLUSH); + return (r < 0) ? -1 : 0; + }); + return connection_handle_write(conn, 1); +} + /** OpenSSL TLS record size is 16383; this is close. The goal here is to * push data out as soon as we know there's enough for a TLS record, so * during periods of high load we won't read entire megabytes from @@ -2982,6 +3408,22 @@ _connection_write_to_buf_impl(const char *string, size_t len, if (conn->marked_for_close && !conn->hold_open_until_flushed) return; + IF_HAS_BUFFEREVENT(conn, { + if (zlib) { + int done = zlib < 0; + r = write_to_evbuffer_zlib(bufferevent_get_output(conn->bufev), + TO_DIR_CONN(conn)->zlib_state, + string, len, done); + } else { + r = bufferevent_write(conn->bufev, string, len); + } + if (r < 0) { + /* XXXX mark for close? */ + log_warn(LD_NET, "bufferevent_write failed! That shouldn't happen."); + } + return; + }); + old_datalen = buf_datalen(conn->outbuf); if (zlib) { dir_connection_t *dir_conn = TO_DIR_CONN(conn); @@ -3009,7 +3451,13 @@ _connection_write_to_buf_impl(const char *string, size_t len, return; } - connection_start_writing(conn); + /* If we receive optimistic data in the EXIT_CONN_STATE_RESOLVING + * state, we don't want to try to write it right away, since + * conn->write_event won't be set yet. Otherwise, write data from + * this conn as the socket is available. */ + if (conn->write_event) { + connection_start_writing(conn); + } if (zlib) { conn->outbuf_flushlen += buf_datalen(conn->outbuf) - old_datalen; } else { @@ -3135,8 +3583,7 @@ connection_get_by_type_state_rendquery(int type, int state, type == CONN_TYPE_AP || type == CONN_TYPE_EXIT); tor_assert(rendquery); - SMARTLIST_FOREACH(conns, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { if (conn->type == type && !conn->marked_for_close && (!state || state == conn->state)) { @@ -3151,7 +3598,33 @@ connection_get_by_type_state_rendquery(int type, int state, TO_EDGE_CONN(conn)->rend_data->onion_address)) return conn; } - }); + } SMARTLIST_FOREACH_END(conn); + return NULL; +} + +/** Return a directory connection (if any one exists) that is fetching + * the item described by <b>state</b>/<b>resource</b> */ +dir_connection_t * +connection_dir_get_by_purpose_and_resource(int purpose, + const char *resource) +{ + smartlist_t *conns = get_connection_array(); + + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { + dir_connection_t *dirconn; + if (conn->type != CONN_TYPE_DIR || conn->marked_for_close || + conn->purpose != purpose) + continue; + dirconn = TO_DIR_CONN(conn); + if (dirconn->requested_resource == NULL) { + if (resource == NULL) + return dirconn; + } else if (resource) { + if (0 == strcmp(resource, dirconn->requested_resource)) + return dirconn; + } + } SMARTLIST_FOREACH_END(conn); + return NULL; } @@ -3268,47 +3741,62 @@ alloc_http_authenticator(const char *authenticator) static void client_check_address_changed(tor_socket_t sock) { - uint32_t iface_ip, ip_out; /* host order */ - struct sockaddr_in out_addr; - socklen_t out_addr_len = (socklen_t) sizeof(out_addr); - uint32_t *ip; /* host order */ + struct sockaddr_storage out_sockaddr; + socklen_t out_addr_len = (socklen_t) sizeof(out_sockaddr); + tor_addr_t out_addr, iface_addr; + tor_addr_t **last_interface_ip_ptr; + sa_family_t family; - if (!last_interface_ip) - get_interface_address(LOG_INFO, &last_interface_ip); if (!outgoing_addrs) - outgoing_addrs = smartlist_create(); + outgoing_addrs = smartlist_new(); - if (getsockname(sock, (struct sockaddr*)&out_addr, &out_addr_len)<0) { + if (getsockname(sock, (struct sockaddr*)&out_sockaddr, &out_addr_len)<0) { int e = tor_socket_errno(sock); log_warn(LD_NET, "getsockname() to check for address change failed: %s", tor_socket_strerror(e)); return; } + tor_addr_from_sockaddr(&out_addr, (struct sockaddr*)&out_sockaddr, NULL); + family = tor_addr_family(&out_addr); + + if (family == AF_INET) + last_interface_ip_ptr = &last_interface_ipv4; + else if (family == AF_INET6) + last_interface_ip_ptr = &last_interface_ipv6; + else + return; + + if (! *last_interface_ip_ptr) { + tor_addr_t *a = tor_malloc_zero(sizeof(tor_addr_t)); + if (get_interface_address6(LOG_INFO, family, a)==0) { + *last_interface_ip_ptr = a; + } else { + tor_free(a); + } + } /* If we've used this address previously, we're okay. */ - ip_out = ntohl(out_addr.sin_addr.s_addr); - SMARTLIST_FOREACH(outgoing_addrs, uint32_t*, ip_ptr, - if (*ip_ptr == ip_out) return; + SMARTLIST_FOREACH(outgoing_addrs, const tor_addr_t *, a_ptr, + if (tor_addr_eq(a_ptr, &out_addr)) + return; ); /* Uh-oh. We haven't connected from this address before. Has the interface * address changed? */ - if (get_interface_address(LOG_INFO, &iface_ip)<0) + if (get_interface_address6(LOG_INFO, family, &iface_addr)<0) return; - ip = tor_malloc(sizeof(uint32_t)); - *ip = ip_out; - if (iface_ip == last_interface_ip) { + if (tor_addr_eq(&iface_addr, *last_interface_ip_ptr)) { /* Nope, it hasn't changed. Add this address to the list. */ - smartlist_add(outgoing_addrs, ip); + smartlist_add(outgoing_addrs, tor_memdup(&out_addr, sizeof(tor_addr_t))); } else { /* The interface changed. We're a client, so we need to regenerate our * keys. First, reset the state. */ log(LOG_NOTICE, LD_NET, "Our IP address has changed. Rotating keys..."); - last_interface_ip = iface_ip; - SMARTLIST_FOREACH(outgoing_addrs, void*, ip_ptr, tor_free(ip_ptr)); + tor_addr_copy(*last_interface_ip_ptr, &iface_addr); + SMARTLIST_FOREACH(outgoing_addrs, tor_addr_t*, a_ptr, tor_free(a_ptr)); smartlist_clear(outgoing_addrs); - smartlist_add(outgoing_addrs, ip); + smartlist_add(outgoing_addrs, tor_memdup(&out_addr, sizeof(tor_addr_t))); /* Okay, now change our keys. */ ip_address_changed(1); } @@ -3405,6 +3893,9 @@ connection_finished_flushing(connection_t *conn) // log_fn(LOG_DEBUG,"entered. Socket %u.", conn->s); + IF_HAS_NO_BUFFEREVENT(conn) + connection_stop_writing(conn); + switch (conn->type) { case CONN_TYPE_OR: return connection_or_finished_flushing(TO_OR_CONN(conn)); @@ -3434,6 +3925,14 @@ static int connection_finished_connecting(connection_t *conn) { tor_assert(conn); + + if (!server_mode(get_options())) { + /* See whether getsockname() says our address changed. We need to do this + * now that the connection has finished, because getsockname() on Windows + * won't work until then. */ + client_check_address_changed(conn->s); + } + switch (conn->type) { case CONN_TYPE_OR: @@ -3488,8 +3987,7 @@ connection_dump_buffer_mem_stats(int severity) memset(alloc_by_type, 0, sizeof(alloc_by_type)); memset(n_conns_by_type, 0, sizeof(n_conns_by_type)); - SMARTLIST_FOREACH(conns, connection_t *, c, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, c) { int tp = c->type; ++n_conns_by_type[tp]; if (c->inbuf) { @@ -3500,7 +3998,7 @@ connection_dump_buffer_mem_stats(int severity) used_by_type[tp] += buf_datalen(c->outbuf); alloc_by_type[tp] += buf_allocation(c->outbuf); } - }); + } SMARTLIST_FOREACH_END(c); for (i=0; i <= _CONN_TYPE_MAX; ++i) { total_used += used_by_type[i]; total_alloc += alloc_by_type[i]; @@ -3530,11 +4028,23 @@ assert_connection_ok(connection_t *conn, time_t now) tor_assert(conn); tor_assert(conn->type >= _CONN_TYPE_MIN); tor_assert(conn->type <= _CONN_TYPE_MAX); + +#ifdef USE_BUFFEREVENTS + if (conn->bufev) { + tor_assert(conn->read_event == NULL); + tor_assert(conn->write_event == NULL); + tor_assert(conn->inbuf == NULL); + tor_assert(conn->outbuf == NULL); + } +#endif + switch (conn->type) { case CONN_TYPE_OR: tor_assert(conn->magic == OR_CONNECTION_MAGIC); break; case CONN_TYPE_AP: + tor_assert(conn->magic == ENTRY_CONNECTION_MAGIC); + break; case CONN_TYPE_EXIT: tor_assert(conn->magic == EDGE_CONNECTION_MAGIC); break; @@ -3544,6 +4054,9 @@ assert_connection_ok(connection_t *conn, time_t now) case CONN_TYPE_CONTROL: tor_assert(conn->magic == CONTROL_CONNECTION_MAGIC); break; + CASE_ANY_LISTENER_TYPE: + tor_assert(conn->magic == LISTENER_CONNECTION_MAGIC); + break; default: tor_assert(conn->magic == BASE_CONNECTION_MAGIC); break; @@ -3557,8 +4070,15 @@ assert_connection_ok(connection_t *conn, time_t now) tor_assert(!SOCKET_OK(conn->s)); if (conn->outbuf_flushlen > 0) { - tor_assert(connection_is_writing(conn) || conn->write_blocked_on_bw || - (CONN_IS_EDGE(conn) && TO_EDGE_CONN(conn)->edge_blocked_on_circ)); + /* With optimistic data, we may have queued data in + * EXIT_CONN_STATE_RESOLVING while the conn is not yet marked to writing. + * */ + tor_assert((conn->type == CONN_TYPE_EXIT && + conn->state == EXIT_CONN_STATE_RESOLVING) || + connection_is_writing(conn) || + conn->write_blocked_on_bw || + (CONN_IS_EDGE(conn) && + TO_EDGE_CONN(conn)->edge_blocked_on_circ)); } if (conn->hold_open_until_flushed) @@ -3568,10 +4088,10 @@ assert_connection_ok(connection_t *conn, time_t now) * marked_for_close. */ /* buffers */ - if (!connection_is_listener(conn)) { + if (conn->inbuf) assert_buf_ok(conn->inbuf); + if (conn->outbuf) assert_buf_ok(conn->outbuf); - } if (conn->type == CONN_TYPE_OR) { or_connection_t *or_conn = TO_OR_CONN(conn); @@ -3591,21 +4111,18 @@ assert_connection_ok(connection_t *conn, time_t now) } if (CONN_IS_EDGE(conn)) { - edge_connection_t *edge_conn = TO_EDGE_CONN(conn); - if (edge_conn->chosen_exit_optional || edge_conn->chosen_exit_retries) { - tor_assert(conn->type == CONN_TYPE_AP); - tor_assert(edge_conn->chosen_exit_name); - } - /* XXX unchecked: package window, deliver window. */ if (conn->type == CONN_TYPE_AP) { + entry_connection_t *entry_conn = TO_ENTRY_CONN(conn); + if (entry_conn->chosen_exit_optional || entry_conn->chosen_exit_retries) + tor_assert(entry_conn->chosen_exit_name); - tor_assert(edge_conn->socks_request); + tor_assert(entry_conn->socks_request); if (conn->state == AP_CONN_STATE_OPEN) { - tor_assert(edge_conn->socks_request->has_finished); + tor_assert(entry_conn->socks_request->has_finished); if (!conn->marked_for_close) { - tor_assert(edge_conn->cpath_layer); - assert_cpath_layer_ok(edge_conn->cpath_layer); + tor_assert(ENTRY_TO_EDGE_CONN(entry_conn)->cpath_layer); + assert_cpath_layer_ok(ENTRY_TO_EDGE_CONN(entry_conn)->cpath_layer); } } } @@ -3621,13 +4138,7 @@ assert_connection_ok(connection_t *conn, time_t now) switch (conn->type) { - case CONN_TYPE_OR_LISTENER: - case CONN_TYPE_AP_LISTENER: - case CONN_TYPE_AP_TRANS_LISTENER: - case CONN_TYPE_AP_NATD_LISTENER: - case CONN_TYPE_DIR_LISTENER: - case CONN_TYPE_CONTROL_LISTENER: - case CONN_TYPE_AP_DNS_LISTENER: + CASE_ANY_LISTENER_TYPE: tor_assert(conn->state == LISTENER_STATE_READY); break; case CONN_TYPE_OR: @@ -3644,7 +4155,7 @@ assert_connection_ok(connection_t *conn, time_t now) case CONN_TYPE_AP: tor_assert(conn->state >= _AP_CONN_STATE_MIN); tor_assert(conn->state <= _AP_CONN_STATE_MAX); - tor_assert(TO_EDGE_CONN(conn)->socks_request); + tor_assert(TO_ENTRY_CONN(conn)->socks_request); break; case CONN_TYPE_DIR: tor_assert(conn->state >= _DIR_CONN_STATE_MIN); @@ -3665,3 +4176,141 @@ assert_connection_ok(connection_t *conn, time_t now) } } +/** Fills <b>addr</b> and <b>port</b> with the details of the global + * proxy server we are using. + * <b>conn</b> contains the connection we are using the proxy for. + * + * Return 0 on success, -1 on failure. + */ +int +get_proxy_addrport(tor_addr_t *addr, uint16_t *port, int *proxy_type, + const connection_t *conn) +{ + const or_options_t *options = get_options(); + + if (options->HTTPSProxy) { + tor_addr_copy(addr, &options->HTTPSProxyAddr); + *port = options->HTTPSProxyPort; + *proxy_type = PROXY_CONNECT; + return 0; + } else if (options->Socks4Proxy) { + tor_addr_copy(addr, &options->Socks4ProxyAddr); + *port = options->Socks4ProxyPort; + *proxy_type = PROXY_SOCKS4; + return 0; + } else if (options->Socks5Proxy) { + tor_addr_copy(addr, &options->Socks5ProxyAddr); + *port = options->Socks5ProxyPort; + *proxy_type = PROXY_SOCKS5; + return 0; + } else if (options->ClientTransportPlugin || + options->Bridges) { + const transport_t *transport = NULL; + int r; + r = find_transport_by_bridge_addrport(&conn->addr, conn->port, &transport); + if (r<0) + return -1; + if (transport) { /* transport found */ + tor_addr_copy(addr, &transport->addr); + *port = transport->port; + *proxy_type = transport->socks_version; + return 0; + } + } + + *proxy_type = PROXY_NONE; + return 0; +} + +/** Returns the global proxy type used by tor. */ +static int +get_proxy_type(void) +{ + const or_options_t *options = get_options(); + + if (options->HTTPSProxy) + return PROXY_CONNECT; + else if (options->Socks4Proxy) + return PROXY_SOCKS4; + else if (options->Socks5Proxy) + return PROXY_SOCKS5; + else if (options->ClientTransportPlugin) + return PROXY_PLUGGABLE; + else + return PROXY_NONE; +} + +/** Log a failed connection to a proxy server. + * <b>conn</b> is the connection we use the proxy server for. */ +void +log_failed_proxy_connection(connection_t *conn) +{ + tor_addr_t proxy_addr; + uint16_t proxy_port; + int proxy_type; + + if (get_proxy_addrport(&proxy_addr, &proxy_port, &proxy_type, conn) != 0) + return; /* if we have no proxy set up, leave this function. */ + + log_warn(LD_NET, + "The connection to the %s proxy server at %s:%u just failed. " + "Make sure that the proxy server is up and running.", + proxy_type_to_string(get_proxy_type()), fmt_addr(&proxy_addr), + proxy_port); +} + +/** Return string representation of <b>proxy_type</b>. */ +static const char * +proxy_type_to_string(int proxy_type) +{ + switch (proxy_type) { + case PROXY_CONNECT: return "HTTP"; + case PROXY_SOCKS4: return "SOCKS4"; + case PROXY_SOCKS5: return "SOCKS5"; + case PROXY_PLUGGABLE: return "pluggable transports SOCKS"; + case PROXY_NONE: return "NULL"; + default: tor_assert(0); + } + return NULL; /*Unreached*/ +} + +/** Call _connection_free() on every connection in our array, and release all + * storage held by connection.c. This is used by cpuworkers and dnsworkers + * when they fork, so they don't keep resources held open (especially + * sockets). + * + * Don't do the checks in connection_free(), because they will + * fail. + */ +void +connection_free_all(void) +{ + smartlist_t *conns = get_connection_array(); + + /* We don't want to log any messages to controllers. */ + SMARTLIST_FOREACH(conns, connection_t *, conn, + if (conn->type == CONN_TYPE_CONTROL) + TO_CONTROL_CONN(conn)->event_mask = 0); + + control_update_global_event_mask(); + + /* Unlink everything from the identity map. */ + connection_or_clear_identity_map(); + + /* Clear out our list of broken connections */ + clear_broken_connection_map(0); + + SMARTLIST_FOREACH(conns, connection_t *, conn, _connection_free(conn)); + + if (outgoing_addrs) { + SMARTLIST_FOREACH(outgoing_addrs, tor_addr_t *, addr, tor_free(addr)); + smartlist_free(outgoing_addrs); + outgoing_addrs = NULL; + } + +#ifdef USE_BUFFEREVENTS + if (global_rate_limit) + bufferevent_rate_limit_group_free(global_rate_limit); +#endif +} + diff --git a/src/or/connection.h b/src/or/connection.h index 576d3a63e1..785625e44b 100644 --- a/src/or/connection.h +++ b/src/or/connection.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -12,13 +12,18 @@ #ifndef _TOR_CONNECTION_H #define _TOR_CONNECTION_H +/* XXXX For buf_datalen in inline function */ +#include "buffers.h" + const char *conn_type_to_string(int type); const char *conn_state_to_string(int type, int state); dir_connection_t *dir_connection_new(int socket_family); or_connection_t *or_connection_new(int socket_family); edge_connection_t *edge_connection_new(int type, int socket_family); +entry_connection_t *entry_connection_new(int type, int socket_family); control_connection_t *control_connection_new(int socket_family); +listener_connection_t *listener_connection_new(int type, int socket_family); connection_t *connection_new(int type, int socket_family); void connection_link_connections(connection_t *conn_a, connection_t *conn_b); @@ -31,6 +36,21 @@ void _connection_mark_for_close(connection_t *conn,int line, const char *file); #define connection_mark_for_close(c) \ _connection_mark_for_close((c), __LINE__, _SHORT_FILE_) +/** + * Mark 'c' for close, but try to hold it open until all the data is written. + */ +#define _connection_mark_and_flush(c,line,file) \ + do { \ + connection_t *tmp_conn_ = (c); \ + _connection_mark_for_close(tmp_conn_, (line), (file)); \ + tmp_conn_->hold_open_until_flushed = 1; \ + IF_HAS_BUFFEREVENT(tmp_conn_, \ + connection_start_writing(tmp_conn_)); \ + } while (0) + +#define connection_mark_and_flush(c) \ + _connection_mark_and_flush((c), __LINE__, _SHORT_FILE_) + void connection_expire_held_open(void); int connection_connect(connection_t *conn, const char *address, @@ -39,9 +59,16 @@ int connection_connect(connection_t *conn, const char *address, int connection_proxy_connect(connection_t *conn, int type); int connection_read_proxy_handshake(connection_t *conn); +void log_failed_proxy_connection(connection_t *conn); +int get_proxy_addrport(tor_addr_t *addr, uint16_t *port, int *proxy_type, + const connection_t *conn); int retry_all_listeners(smartlist_t *replaced_conns, - smartlist_t *new_conns); + smartlist_t *new_conns, + int close_all_noncontrol); + +void connection_mark_all_noncontrol_listeners(void); +void connection_mark_all_noncontrol_connections(void); ssize_t connection_bucket_write_limit(connection_t *conn, time_t now); int global_write_bucket_low(connection_t *conn, size_t attempt, int priority); @@ -51,14 +78,24 @@ void connection_bucket_refill(int seconds_elapsed, time_t now); int connection_handle_read(connection_t *conn); int connection_fetch_from_buf(char *string, size_t len, connection_t *conn); +int connection_fetch_from_buf_line(connection_t *conn, char *data, + size_t *data_len); +int connection_fetch_from_buf_http(connection_t *conn, + char **headers_out, size_t max_headerlen, + char **body_out, size_t *body_used, + size_t max_bodylen, int force_complete); int connection_wants_to_flush(connection_t *conn); int connection_outbuf_too_full(connection_t *conn); int connection_handle_write(connection_t *conn, int force); +int connection_flush(connection_t *conn); + void _connection_write_to_buf_impl(const char *string, size_t len, connection_t *conn, int zlib); +/* DOCDOC connection_write_to_buf */ static void connection_write_to_buf(const char *string, size_t len, connection_t *conn); +/* DOCDOC connection_write_to_buf_zlib */ static void connection_write_to_buf_zlib(const char *string, size_t len, dir_connection_t *conn, int done); static INLINE void @@ -73,6 +110,31 @@ connection_write_to_buf_zlib(const char *string, size_t len, _connection_write_to_buf_impl(string, len, TO_CONN(conn), done ? -1 : 1); } +/* DOCDOC connection_get_inbuf_len */ +static size_t connection_get_inbuf_len(connection_t *conn); +/* DOCDOC connection_get_outbuf_len */ +static size_t connection_get_outbuf_len(connection_t *conn); + +static INLINE size_t +connection_get_inbuf_len(connection_t *conn) +{ + IF_HAS_BUFFEREVENT(conn, { + return evbuffer_get_length(bufferevent_get_input(conn->bufev)); + }) ELSE_IF_NO_BUFFEREVENT { + return conn->inbuf ? buf_datalen(conn->inbuf) : 0; + } +} + +static INLINE size_t +connection_get_outbuf_len(connection_t *conn) +{ + IF_HAS_BUFFEREVENT(conn, { + return evbuffer_get_length(bufferevent_get_output(conn->bufev)); + }) ELSE_IF_NO_BUFFEREVENT { + return conn->outbuf ? buf_datalen(conn->outbuf) : 0; + } +} + connection_t *connection_get_by_global_id(uint64_t id); connection_t *connection_get_by_type(int type); @@ -83,6 +145,8 @@ connection_t *connection_get_by_type_addr_port_purpose(int type, connection_t *connection_get_by_type_state(int type, int state); connection_t *connection_get_by_type_state_rendquery(int type, int state, const char *rendquery); +dir_connection_t *connection_dir_get_by_purpose_and_resource( + int state, const char *resource); #define connection_speaks_cells(conn) ((conn)->type == CONN_TYPE_OR) int connection_is_listener(connection_t *conn); @@ -96,5 +160,19 @@ int connection_or_nonopen_was_started_here(or_connection_t *conn); void connection_dump_buffer_mem_stats(int severity); void remove_file_if_very_old(const char *fname, time_t now); +#ifdef USE_BUFFEREVENTS +int connection_type_uses_bufferevent(connection_t *conn); +void connection_configure_bufferevent_callbacks(connection_t *conn); +void connection_handle_read_cb(struct bufferevent *bufev, void *arg); +void connection_handle_write_cb(struct bufferevent *bufev, void *arg); +void connection_handle_event_cb(struct bufferevent *bufev, short event, + void *arg); +void connection_get_rate_limit_totals(uint64_t *read_out, + uint64_t *written_out); +void connection_enable_rate_limiting(connection_t *conn); +#else +#define connection_type_uses_bufferevent(c) (0) +#endif + #endif diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index 4763bf59a2..362ad9af95 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -23,6 +23,7 @@ #include "dirserv.h" #include "hibernate.h" #include "main.h" +#include "nodelist.h" #include "policies.h" #include "reasons.h" #include "relay.h" @@ -50,27 +51,43 @@ #define SOCKS4_GRANTED 90 #define SOCKS4_REJECT 91 -static int connection_ap_handshake_process_socks(edge_connection_t *conn); -static int connection_ap_process_natd(edge_connection_t *conn); +static int connection_ap_handshake_process_socks(entry_connection_t *conn); +static int connection_ap_process_natd(entry_connection_t *conn); static int connection_exit_connect_dir(edge_connection_t *exitconn); static int address_is_in_virtual_range(const char *addr); -static int consider_plaintext_ports(edge_connection_t *conn, uint16_t port); +static int consider_plaintext_ports(entry_connection_t *conn, uint16_t port); static void clear_trackexithost_mappings(const char *exitname); +static int connection_ap_supports_optimistic_data(const entry_connection_t *); /** An AP stream has failed/finished. If it hasn't already sent back * a socks reply, send one now (based on endreason). Also set * has_sent_end to 1, and mark the conn. */ void -_connection_mark_unattached_ap(edge_connection_t *conn, int endreason, +_connection_mark_unattached_ap(entry_connection_t *conn, int endreason, int line, const char *file) { - tor_assert(conn->_base.type == CONN_TYPE_AP); - conn->edge_has_sent_end = 1; /* no circ yet */ - - if (conn->_base.marked_for_close) { + connection_t *base_conn = ENTRY_TO_CONN(conn); + edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(conn); + tor_assert(base_conn->type == CONN_TYPE_AP); + ENTRY_TO_EDGE_CONN(conn)->edge_has_sent_end = 1; /* no circ yet */ + + /* If this is a rendezvous stream and it is failing without ever + * being attached to a circuit, assume that an attempt to connect to + * the destination hidden service has just ended. + * + * XXXX This condition doesn't limit to only streams failing + * without ever being attached. That sloppiness should be harmless, + * but we should fix it someday anyway. */ + if ((edge_conn->on_circuit != NULL || edge_conn->edge_has_sent_end) && + connection_edge_is_rendezvous_stream(edge_conn)) { + rend_client_note_connection_attempt_ended( + edge_conn->rend_data->onion_address); + } + + if (base_conn->marked_for_close) { /* This call will warn as appropriate. */ - _connection_mark_for_close(TO_CONN(conn), line, file); + _connection_mark_for_close(base_conn, line, file); return; } @@ -90,9 +107,9 @@ _connection_mark_unattached_ap(edge_connection_t *conn, int endreason, conn->socks_request->has_finished = 1; } - _connection_mark_for_close(TO_CONN(conn), line, file); - conn->_base.hold_open_until_flushed = 1; - conn->end_reason = endreason; + _connection_mark_and_flush(base_conn, line, file); + + ENTRY_TO_EDGE_CONN(conn)->end_reason = endreason; } /** There was an EOF. Send an end and mark the connection for close. @@ -100,7 +117,7 @@ _connection_mark_unattached_ap(edge_connection_t *conn, int endreason, int connection_edge_reached_eof(edge_connection_t *conn) { - if (buf_datalen(conn->_base.inbuf) && + if (connection_get_inbuf_len(TO_CONN(conn)) && connection_state_is_open(TO_CONN(conn))) { /* it still has stuff to process. don't let it die yet. */ return 0; @@ -110,8 +127,11 @@ connection_edge_reached_eof(edge_connection_t *conn) /* only mark it if not already marked. it's possible to * get the 'end' right around when the client hangs up on us. */ connection_edge_end(conn, END_STREAM_REASON_DONE); - if (conn->socks_request) /* eof, so don't send a socks reply back */ - conn->socks_request->has_finished = 1; + if (conn->_base.type == CONN_TYPE_AP) { + /* eof, so don't send a socks reply back */ + if (EDGE_TO_ENTRY_CONN(conn)->socks_request) + EDGE_TO_ENTRY_CONN(conn)->socks_request->has_finished = 1; + } connection_mark_for_close(TO_CONN(conn)); } return 0; @@ -134,13 +154,13 @@ connection_edge_process_inbuf(edge_connection_t *conn, int package_partial) switch (conn->_base.state) { case AP_CONN_STATE_SOCKS_WAIT: - if (connection_ap_handshake_process_socks(conn) < 0) { + if (connection_ap_handshake_process_socks(EDGE_TO_ENTRY_CONN(conn)) <0) { /* already marked */ return -1; } return 0; case AP_CONN_STATE_NATD_WAIT: - if (connection_ap_process_natd(conn) < 0) { + if (connection_ap_process_natd(EDGE_TO_ENTRY_CONN(conn)) < 0) { /* already marked */ return -1; } @@ -153,10 +173,26 @@ connection_edge_process_inbuf(edge_connection_t *conn, int package_partial) return -1; } return 0; + case AP_CONN_STATE_CONNECT_WAIT: + if (connection_ap_supports_optimistic_data(EDGE_TO_ENTRY_CONN(conn))) { + log_info(LD_EDGE, + "data from edge while in '%s' state. Sending it anyway. " + "package_partial=%d, buflen=%ld", + conn_state_to_string(conn->_base.type, conn->_base.state), + package_partial, + (long)connection_get_inbuf_len(TO_CONN(conn))); + if (connection_edge_package_raw_inbuf(conn, package_partial, NULL)<0) { + /* (We already sent an end cell if possible) */ + connection_mark_for_close(TO_CONN(conn)); + return -1; + } + return 0; + } + /* Fall through if the connection is on a circuit without optimistic + * data support. */ case EXIT_CONN_STATE_CONNECTING: case AP_CONN_STATE_RENDDESC_WAIT: case AP_CONN_STATE_CIRCUIT_WAIT: - case AP_CONN_STATE_CONNECT_WAIT: case AP_CONN_STATE_RESOLVE_WAIT: case AP_CONN_STATE_CONTROLLER_WAIT: log_info(LD_EDGE, @@ -181,9 +217,10 @@ connection_edge_destroy(circid_t circ_id, edge_connection_t *conn) log_info(LD_EDGE, "CircID %d: At an edge. Marking connection for close.", circ_id); if (conn->_base.type == CONN_TYPE_AP) { - connection_mark_unattached_ap(conn, END_STREAM_REASON_DESTROY); + entry_connection_t *entry_conn = EDGE_TO_ENTRY_CONN(conn); + connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_DESTROY); control_event_stream_bandwidth(conn); - control_event_stream_status(conn, STREAM_EVENT_CLOSED, + control_event_stream_status(entry_conn, STREAM_EVENT_CLOSED, END_STREAM_REASON_DESTROY); conn->end_reason |= END_STREAM_REASON_FLAG_ALREADY_SENT_CLOSED; } else { @@ -191,8 +228,7 @@ connection_edge_destroy(circid_t circ_id, edge_connection_t *conn) conn->edge_has_sent_end = 1; conn->end_reason = END_STREAM_REASON_DESTROY; conn->end_reason |= END_STREAM_REASON_FLAG_ALREADY_SENT_CLOSED; - connection_mark_for_close(TO_CONN(conn)); - conn->_base.hold_open_until_flushed = 1; + connection_mark_and_flush(TO_CONN(conn)); } } conn->cpath_layer = NULL; @@ -336,7 +372,6 @@ connection_edge_finished_flushing(edge_connection_t *conn) switch (conn->_base.state) { case AP_CONN_STATE_OPEN: case EXIT_CONN_STATE_OPEN: - connection_stop_writing(TO_CONN(conn)); connection_edge_consider_sending_sendme(conn); return 0; case AP_CONN_STATE_SOCKS_WAIT: @@ -345,7 +380,7 @@ connection_edge_finished_flushing(edge_connection_t *conn) case AP_CONN_STATE_CIRCUIT_WAIT: case AP_CONN_STATE_CONNECT_WAIT: case AP_CONN_STATE_CONTROLLER_WAIT: - connection_stop_writing(TO_CONN(conn)); + case AP_CONN_STATE_RESOLVE_WAIT: return 0; default: log_warn(LD_BUG, "Called in unexpected state %d.",conn->_base.state); @@ -375,8 +410,9 @@ connection_edge_finished_connecting(edge_connection_t *edge_conn) rep_hist_note_exit_stream_opened(conn->port); conn->state = EXIT_CONN_STATE_OPEN; - connection_watch_events(conn, READ_EVENT); /* stop writing, keep reading */ - if (connection_wants_to_flush(conn)) /* in case there are any queued relay + IF_HAS_NO_BUFFEREVENT(conn) + connection_watch_events(conn, READ_EVENT); /* stop writing, keep reading */ + if (connection_get_outbuf_len(conn)) /* in case there are any queued relay * cells */ connection_start_writing(conn); /* deliver a 'connected' relay cell back through the circuit. */ @@ -408,13 +444,79 @@ connection_edge_finished_connecting(edge_connection_t *edge_conn) return connection_edge_process_inbuf(edge_conn, 1); } +/** Common code to connection_(ap|exit)_about_to_close. */ +static void +connection_edge_about_to_close(edge_connection_t *edge_conn) +{ + if (!edge_conn->edge_has_sent_end) { + connection_t *conn = TO_CONN(edge_conn); + log_warn(LD_BUG, "(Harmless.) Edge connection (marked at %s:%d) " + "hasn't sent end yet?", + conn->marked_for_close_file, conn->marked_for_close); + tor_fragile_assert(); + } +} + +/** Called when we're about to finally unlink and free an AP (client) + * connection: perform necessary accounting and cleanup */ +void +connection_ap_about_to_close(entry_connection_t *entry_conn) +{ + circuit_t *circ; + edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(entry_conn); + connection_t *conn = ENTRY_TO_CONN(entry_conn); + + if (entry_conn->socks_request->has_finished == 0) { + /* since conn gets removed right after this function finishes, + * there's no point trying to send back a reply at this point. */ + log_warn(LD_BUG,"Closing stream (marked at %s:%d) without sending" + " back a socks reply.", + conn->marked_for_close_file, conn->marked_for_close); + } + if (!edge_conn->end_reason) { + log_warn(LD_BUG,"Closing stream (marked at %s:%d) without having" + " set end_reason.", + conn->marked_for_close_file, conn->marked_for_close); + } + if (entry_conn->dns_server_request) { + log_warn(LD_BUG,"Closing stream (marked at %s:%d) without having" + " replied to DNS request.", + conn->marked_for_close_file, conn->marked_for_close); + dnsserv_reject_request(entry_conn); + } + control_event_stream_bandwidth(edge_conn); + control_event_stream_status(entry_conn, STREAM_EVENT_CLOSED, + edge_conn->end_reason); + circ = circuit_get_by_edge_conn(edge_conn); + if (circ) + circuit_detach_stream(circ, edge_conn); +} + +/** Called when we're about to finally unlink and free an exit + * connection: perform necessary accounting and cleanup */ +void +connection_exit_about_to_close(edge_connection_t *edge_conn) +{ + circuit_t *circ; + connection_t *conn = TO_CONN(edge_conn); + + connection_edge_about_to_close(edge_conn); + + circ = circuit_get_by_edge_conn(edge_conn); + if (circ) + circuit_detach_stream(circ, edge_conn); + if (conn->state == EXIT_CONN_STATE_RESOLVING) { + connection_dns_remove(edge_conn); + } +} + /** Define a schedule for how long to wait between retrying * application connections. Rather than waiting a fixed amount of * time between each retry, we wait 10 seconds each for the first * two tries, and 15 seconds for each retry after * that. Hopefully this will improve the expected user experience. */ static int -compute_retry_timeout(edge_connection_t *conn) +compute_retry_timeout(entry_connection_t *conn) { int timeout = get_options()->CircuitStreamTimeout; if (timeout) /* if our config options override the default, use them */ @@ -437,41 +539,43 @@ void connection_ap_expire_beginning(void) { edge_connection_t *conn; + entry_connection_t *entry_conn; circuit_t *circ; time_t now = time(NULL); - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int severity; int cutoff; int seconds_idle, seconds_since_born; smartlist_t *conns = get_connection_array(); - SMARTLIST_FOREACH_BEGIN(conns, connection_t *, c) { - if (c->type != CONN_TYPE_AP || c->marked_for_close) + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, base_conn) { + if (base_conn->type != CONN_TYPE_AP || base_conn->marked_for_close) continue; - conn = TO_EDGE_CONN(c); + entry_conn = TO_ENTRY_CONN(base_conn); + conn = ENTRY_TO_EDGE_CONN(entry_conn); /* if it's an internal linked connection, don't yell its status. */ - severity = (tor_addr_is_null(&conn->_base.addr) && !conn->_base.port) + severity = (tor_addr_is_null(&base_conn->addr) && !base_conn->port) ? LOG_INFO : LOG_NOTICE; - seconds_idle = (int)( now - conn->_base.timestamp_lastread ); - seconds_since_born = (int)( now - conn->_base.timestamp_created ); + seconds_idle = (int)( now - base_conn->timestamp_lastread ); + seconds_since_born = (int)( now - base_conn->timestamp_created ); - if (conn->_base.state == AP_CONN_STATE_OPEN) + if (base_conn->state == AP_CONN_STATE_OPEN) continue; /* We already consider SocksTimeout in * connection_ap_handshake_attach_circuit(), but we need to consider * it here too because controllers that put streams in controller_wait * state never ask Tor to attach the circuit. */ - if (AP_CONN_STATE_IS_UNATTACHED(conn->_base.state)) { + if (AP_CONN_STATE_IS_UNATTACHED(base_conn->state)) { if (seconds_since_born >= options->SocksTimeout) { log_fn(severity, LD_APP, "Tried for %d seconds to get a connection to %s:%d. " "Giving up. (%s)", seconds_since_born, - safe_str_client(conn->socks_request->address), - conn->socks_request->port, - conn_state_to_string(CONN_TYPE_AP, conn->_base.state)); - connection_mark_unattached_ap(conn, END_STREAM_REASON_TIMEOUT); + safe_str_client(entry_conn->socks_request->address), + entry_conn->socks_request->port, + conn_state_to_string(CONN_TYPE_AP, base_conn->state)); + connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_TIMEOUT); } continue; } @@ -479,14 +583,14 @@ connection_ap_expire_beginning(void) /* We're in state connect_wait or resolve_wait now -- waiting for a * reply to our relay cell. See if we want to retry/give up. */ - cutoff = compute_retry_timeout(conn); + cutoff = compute_retry_timeout(entry_conn); if (seconds_idle < cutoff) continue; circ = circuit_get_by_edge_conn(conn); if (!circ) { /* it's vanished? */ log_info(LD_APP,"Conn is waiting (address %s), but lost its circ.", - safe_str_client(conn->socks_request->address)); - connection_mark_unattached_ap(conn, END_STREAM_REASON_TIMEOUT); + safe_str_client(entry_conn->socks_request->address)); + connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_TIMEOUT); continue; } if (circ->purpose == CIRCUIT_PURPOSE_C_REND_JOINED) { @@ -495,9 +599,9 @@ connection_ap_expire_beginning(void) "Rend stream is %d seconds late. Giving up on address" " '%s.onion'.", seconds_idle, - safe_str_client(conn->socks_request->address)); + safe_str_client(entry_conn->socks_request->address)); connection_edge_end(conn, END_STREAM_REASON_TIMEOUT); - connection_mark_unattached_ap(conn, END_STREAM_REASON_TIMEOUT); + connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_TIMEOUT); } continue; } @@ -506,9 +610,10 @@ connection_ap_expire_beginning(void) "We tried for %d seconds to connect to '%s' using exit %s." " Retrying on a new circuit.", seconds_idle, - safe_str_client(conn->socks_request->address), + safe_str_client(entry_conn->socks_request->address), conn->cpath_layer ? - extend_info_describe(conn->cpath_layer->extend_info): "*unnamed*"); + extend_info_describe(conn->cpath_layer->extend_info): + "*unnamed*"); /* send an end down the circuit */ connection_edge_end(conn, END_STREAM_REASON_TIMEOUT); /* un-mark it as ending, since we're going to reuse it */ @@ -517,20 +622,21 @@ connection_ap_expire_beginning(void) /* kludge to make us not try this circuit again, yet to allow * current streams on it to survive if they can: make it * unattractive to use for new streams */ - /* XXXX023 this is a kludgy way to do this. */ + /* XXXX024 this is a kludgy way to do this. */ tor_assert(circ->timestamp_dirty); circ->timestamp_dirty -= options->MaxCircuitDirtiness; /* give our stream another 'cutoff' seconds to try */ conn->_base.timestamp_lastread += cutoff; - if (conn->num_socks_retries < 250) /* avoid overflow */ - conn->num_socks_retries++; + if (entry_conn->num_socks_retries < 250) /* avoid overflow */ + entry_conn->num_socks_retries++; /* move it back into 'pending' state, and try to attach. */ - if (connection_ap_detach_retriable(conn, TO_ORIGIN_CIRCUIT(circ), + if (connection_ap_detach_retriable(entry_conn, TO_ORIGIN_CIRCUIT(circ), END_STREAM_REASON_TIMEOUT)<0) { - if (!conn->_base.marked_for_close) - connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); + if (!base_conn->marked_for_close) + connection_mark_unattached_ap(entry_conn, + END_STREAM_REASON_CANT_ATTACH); } - } SMARTLIST_FOREACH_END(conn); + } SMARTLIST_FOREACH_END(base_conn); } /** Tell any AP streams that are waiting for a new circuit to try again, @@ -539,33 +645,32 @@ connection_ap_expire_beginning(void) void connection_ap_attach_pending(void) { - edge_connection_t *edge_conn; + entry_connection_t *entry_conn; smartlist_t *conns = get_connection_array(); - SMARTLIST_FOREACH(conns, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { if (conn->marked_for_close || conn->type != CONN_TYPE_AP || conn->state != AP_CONN_STATE_CIRCUIT_WAIT) continue; - edge_conn = TO_EDGE_CONN(conn); - if (connection_ap_handshake_attach_circuit(edge_conn) < 0) { - if (!edge_conn->_base.marked_for_close) - connection_mark_unattached_ap(edge_conn, + entry_conn = TO_ENTRY_CONN(conn); + if (connection_ap_handshake_attach_circuit(entry_conn) < 0) { + if (!conn->marked_for_close) + connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_CANT_ATTACH); } - }); + } SMARTLIST_FOREACH_END(conn); } /** Tell any AP streams that are waiting for a one-hop tunnel to * <b>failed_digest</b> that they are going to fail. */ -/* XXX023 We should get rid of this function, and instead attach +/* XXX024 We should get rid of this function, and instead attach * one-hop streams to circ->p_streams so they get marked in * circuit_mark_for_close like normal p_streams. */ void connection_ap_fail_onehop(const char *failed_digest, cpath_build_state_t *build_state) { - edge_connection_t *edge_conn; + entry_connection_t *entry_conn; char digest[DIGEST_LEN]; smartlist_t *conns = get_connection_array(); SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { @@ -573,27 +678,27 @@ connection_ap_fail_onehop(const char *failed_digest, conn->type != CONN_TYPE_AP || conn->state != AP_CONN_STATE_CIRCUIT_WAIT) continue; - edge_conn = TO_EDGE_CONN(conn); - if (!edge_conn->want_onehop) + entry_conn = TO_ENTRY_CONN(conn); + if (!entry_conn->want_onehop) continue; - if (hexdigest_to_digest(edge_conn->chosen_exit_name, digest) < 0 || + if (hexdigest_to_digest(entry_conn->chosen_exit_name, digest) < 0 || tor_memneq(digest, failed_digest, DIGEST_LEN)) continue; if (tor_digest_is_zero(digest)) { /* we don't know the digest; have to compare addr:port */ tor_addr_t addr; if (!build_state || !build_state->chosen_exit || - !edge_conn->socks_request || !edge_conn->socks_request->address) + !entry_conn->socks_request || !entry_conn->socks_request->address) continue; - if (tor_addr_from_str(&addr, edge_conn->socks_request->address)<0 || + if (tor_addr_parse(&addr, entry_conn->socks_request->address)<0 || !tor_addr_eq(&build_state->chosen_exit->addr, &addr) || - build_state->chosen_exit->port != edge_conn->socks_request->port) + build_state->chosen_exit->port != entry_conn->socks_request->port) continue; } log_info(LD_APP, "Closing one-hop stream to '%s/%s' because the OR conn " - "just failed.", edge_conn->chosen_exit_name, - edge_conn->socks_request->address); - connection_mark_unattached_ap(edge_conn, END_STREAM_REASON_TIMEOUT); + "just failed.", entry_conn->chosen_exit_name, + entry_conn->socks_request->address); + connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_TIMEOUT); } SMARTLIST_FOREACH_END(conn); } @@ -604,8 +709,8 @@ connection_ap_fail_onehop(const char *failed_digest, void circuit_discard_optional_exit_enclaves(extend_info_t *info) { - edge_connection_t *edge_conn; - routerinfo_t *r1, *r2; + entry_connection_t *entry_conn; + const node_t *r1, *r2; smartlist_t *conns = get_connection_array(); SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { @@ -613,32 +718,32 @@ circuit_discard_optional_exit_enclaves(extend_info_t *info) conn->type != CONN_TYPE_AP || conn->state != AP_CONN_STATE_CIRCUIT_WAIT) continue; - edge_conn = TO_EDGE_CONN(conn); - if (!edge_conn->chosen_exit_optional && - !edge_conn->chosen_exit_retries) + entry_conn = TO_ENTRY_CONN(conn); + if (!entry_conn->chosen_exit_optional && + !entry_conn->chosen_exit_retries) continue; - r1 = router_get_by_nickname(edge_conn->chosen_exit_name, 0); - r2 = router_get_by_digest(info->identity_digest); + r1 = node_get_by_nickname(entry_conn->chosen_exit_name, 0); + r2 = node_get_by_id(info->identity_digest); if (!r1 || !r2 || r1 != r2) continue; - tor_assert(edge_conn->socks_request); - if (edge_conn->chosen_exit_optional) { + tor_assert(entry_conn->socks_request); + if (entry_conn->chosen_exit_optional) { log_info(LD_APP, "Giving up on enclave exit '%s' for destination %s.", - safe_str_client(edge_conn->chosen_exit_name), - escaped_safe_str_client(edge_conn->socks_request->address)); - edge_conn->chosen_exit_optional = 0; - tor_free(edge_conn->chosen_exit_name); /* clears it */ + safe_str_client(entry_conn->chosen_exit_name), + escaped_safe_str_client(entry_conn->socks_request->address)); + entry_conn->chosen_exit_optional = 0; + tor_free(entry_conn->chosen_exit_name); /* clears it */ /* if this port is dangerous, warn or reject it now that we don't * think it'll be using an enclave. */ - consider_plaintext_ports(edge_conn, edge_conn->socks_request->port); + consider_plaintext_ports(entry_conn, entry_conn->socks_request->port); } - if (edge_conn->chosen_exit_retries) { - if (--edge_conn->chosen_exit_retries == 0) { /* give up! */ - clear_trackexithost_mappings(edge_conn->chosen_exit_name); - tor_free(edge_conn->chosen_exit_name); /* clears it */ + if (entry_conn->chosen_exit_retries) { + if (--entry_conn->chosen_exit_retries == 0) { /* give up! */ + clear_trackexithost_mappings(entry_conn->chosen_exit_name); + tor_free(entry_conn->chosen_exit_name); /* clears it */ /* if this port is dangerous, warn or reject it now that we don't * think it'll be using an enclave. */ - consider_plaintext_ports(edge_conn, edge_conn->socks_request->port); + consider_plaintext_ports(entry_conn, entry_conn->socks_request->port); } } } SMARTLIST_FOREACH_END(conn); @@ -652,20 +757,27 @@ circuit_discard_optional_exit_enclaves(extend_info_t *info) * Returns -1 on err, 1 on success, 0 on not-yet-sure. */ int -connection_ap_detach_retriable(edge_connection_t *conn, origin_circuit_t *circ, +connection_ap_detach_retriable(entry_connection_t *conn, + origin_circuit_t *circ, int reason) { control_event_stream_status(conn, STREAM_EVENT_FAILED_RETRIABLE, reason); - conn->_base.timestamp_lastread = time(NULL); + ENTRY_TO_CONN(conn)->timestamp_lastread = time(NULL); + + if (conn->pending_optimistic_data) { + generic_buffer_set_to_copy(&conn->sending_optimistic_data, + conn->pending_optimistic_data); + } + if (!get_options()->LeaveStreamsUnattached || conn->use_begindir) { /* If we're attaching streams ourself, or if this connection is * a tunneled directory connection, then just attach it. */ - conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; - circuit_detach_stream(TO_CIRCUIT(circ),conn); + ENTRY_TO_CONN(conn)->state = AP_CONN_STATE_CIRCUIT_WAIT; + circuit_detach_stream(TO_CIRCUIT(circ),ENTRY_TO_EDGE_CONN(conn)); return connection_ap_handshake_attach_circuit(conn); } else { - conn->_base.state = AP_CONN_STATE_CONTROLLER_WAIT; - circuit_detach_stream(TO_CIRCUIT(circ),conn); + ENTRY_TO_CONN(conn)->state = AP_CONN_STATE_CONTROLLER_WAIT; + circuit_detach_stream(TO_CIRCUIT(circ),ENTRY_TO_EDGE_CONN(conn)); return 0; } } @@ -690,11 +802,18 @@ connection_ap_detach_retriable(edge_connection_t *conn, origin_circuit_t *circ, * the configuration file, "1" for mappings set from the control * interface, and other values for DNS and TrackHostExit mappings that can * expire.) + * + * A mapping may be 'wildcarded'. If "src_wildcard" is true, then + * any address that ends with a . followed by the key for this entry will + * get remapped by it. If "dst_wildcard" is also true, then only the + * matching suffix of such addresses will get replaced by new_address. */ typedef struct { char *new_address; time_t expires; addressmap_entry_source_t source:3; + unsigned src_wildcard:1; + unsigned dst_wildcard:1; short num_resolve_failures; } addressmap_entry_t; @@ -789,13 +908,10 @@ addressmap_ent_remove(const char *address, addressmap_entry_t *ent) static void clear_trackexithost_mappings(const char *exitname) { - char *suffix; - size_t suffix_len; + char *suffix = NULL; if (!addressmap || !exitname) return; - suffix_len = strlen(exitname) + 16; - suffix = tor_malloc(suffix_len); - tor_snprintf(suffix, suffix_len, ".%s.exit", exitname); + tor_asprintf(&suffix, ".%s.exit", exitname); tor_strlower(suffix); STRMAP_FOREACH_MODIFY(addressmap, address, addressmap_entry_t *, ent) { @@ -813,7 +929,7 @@ clear_trackexithost_mappings(const char *exitname) * host is unknown or no longer allowed, or for which the source address * is no longer in trackexithosts. */ void -addressmap_clear_excluded_trackexithosts(or_options_t *options) +addressmap_clear_excluded_trackexithosts(const or_options_t *options) { const routerset_t *allow_nodes = options->ExitNodes; const routerset_t *exclude_nodes = options->_ExcludeExitNodesUnion; @@ -829,7 +945,7 @@ addressmap_clear_excluded_trackexithosts(or_options_t *options) size_t len; const char *target = ent->new_address, *dot; char *nodename; - routerinfo_t *ri; /* XXX023 Use node_t. */ + const node_t *node; if (!target) { /* DNS resolving in progress */ @@ -849,11 +965,11 @@ addressmap_clear_excluded_trackexithosts(or_options_t *options) dot--; if (*dot == '.') dot++; nodename = tor_strndup(dot, len-5-(dot-target));; - ri = router_get_by_nickname(nodename, 0); + node = node_get_by_nickname(nodename, 0); tor_free(nodename); - if (!ri || - (allow_nodes && !routerset_contains_router(allow_nodes, ri)) || - routerset_contains_router(exclude_nodes, ri) || + if (!node || + (allow_nodes && !routerset_contains_node(allow_nodes, node)) || + routerset_contains_node(exclude_nodes, node) || !hostname_in_track_host_exits(options, address)) { /* We don't know this one, or we want to be rid of it. */ addressmap_ent_remove(address, ent); @@ -867,7 +983,7 @@ addressmap_clear_excluded_trackexithosts(or_options_t *options) * no longer allowed by AutomapHostsOnResolve, or for which the * target address is no longer in the virtual network. */ void -addressmap_clear_invalid_automaps(or_options_t *options) +addressmap_clear_invalid_automaps(const or_options_t *options) { int clear_all = !options->AutomapHostsOnResolve; const smartlist_t *suffixes = options->AutomapHostsSuffixes; @@ -941,45 +1057,118 @@ addressmap_free_all(void) virtaddress_reversemap = NULL; } +/** Try to find a match for AddressMap expressions that use + * wildcard notation such as '*.c.d *.e.f' (so 'a.c.d' will map to 'a.e.f') or + * '*.c.d a.b.c' (so 'a.c.d' will map to a.b.c). + * Return the matching entry in AddressMap or NULL if no match is found. + * For expressions such as '*.c.d *.e.f', truncate <b>address</b> 'a.c.d' + * to 'a' before we return the matching AddressMap entry. + * + * This function does not handle the case where a pattern of the form "*.c.d" + * matches the address c.d -- that's done by the main addressmap_rewrite + * function. + */ +static addressmap_entry_t * +addressmap_match_superdomains(char *address) +{ + addressmap_entry_t *val; + char *cp; + + cp = address; + while ((cp = strchr(cp, '.'))) { + /* cp now points to a suffix of address that begins with a . */ + val = strmap_get_lc(addressmap, cp+1); + if (val && val->src_wildcard) { + if (val->dst_wildcard) + *cp = '\0'; + return val; + } + ++cp; + } + return NULL; +} + /** Look at address, and rewrite it until it doesn't want any * more rewrites; but don't get into an infinite loop. * Don't write more than maxlen chars into address. Return true if the * address changed; false otherwise. Set *<b>expires_out</b> to the * expiry time of the result, or to <b>time_max</b> if the result does * not expire. + * + * If <b>exit_source_out</b> is non-null, we set it as follows. If we the + * address starts out as a non-exit address, and we remap it to an .exit + * address at any point, then set *<b>exit_source_out</b> to the + * address_entry_source_t of the first such rule. Set *<b>exit_source_out</b> + * to ADDRMAPSRC_NONE if there is no such rewrite, or if the original address + * was a .exit. */ int -addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out) +addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out, + addressmap_entry_source_t *exit_source_out) { addressmap_entry_t *ent; int rewrites; - char *cp; time_t expires = TIME_MAX; + addressmap_entry_source_t exit_source = ADDRMAPSRC_NONE; + char *addr_orig = tor_strdup(address); + char *log_addr_orig = NULL; for (rewrites = 0; rewrites < 16; rewrites++) { + int exact_match = 0; + log_addr_orig = tor_strdup(escaped_safe_str_client(address)); + ent = strmap_get(addressmap, address); if (!ent || !ent->new_address) { - if (expires_out) - *expires_out = expires; - return (rewrites > 0); /* done, no rewrite needed */ + ent = addressmap_match_superdomains(address); + } else { + if (ent->src_wildcard && !ent->dst_wildcard && + !strcasecmp(address, ent->new_address)) { + /* This is a rule like *.example.com example.com, and we just got + * "example.com" */ + goto done; + } + + exact_match = 1; + } + + if (!ent || !ent->new_address) { + goto done; + } + + if (ent->dst_wildcard && !exact_match) { + strlcat(address, ".", maxlen); + strlcat(address, ent->new_address, maxlen); + } else { + strlcpy(address, ent->new_address, maxlen); + } + + if (!strcmpend(address, ".exit") && + strcmpend(addr_orig, ".exit") && + exit_source == ADDRMAPSRC_NONE) { + exit_source = ent->source; } - cp = tor_strdup(escaped_safe_str_client(ent->new_address)); log_info(LD_APP, "Addressmap: rewriting %s to %s", - escaped_safe_str_client(address), cp); + log_addr_orig, escaped_safe_str_client(address)); if (ent->expires > 1 && ent->expires < expires) expires = ent->expires; - tor_free(cp); - strlcpy(address, ent->new_address, maxlen); + + tor_free(log_addr_orig); } log_warn(LD_CONFIG, "Loop detected: we've rewritten %s 16 times! Using it as-is.", escaped_safe_str_client(address)); /* it's fine to rewrite a rewrite, but don't loop forever */ + + done: + tor_free(addr_orig); + tor_free(log_addr_orig); + if (exit_source_out) + *exit_source_out = exit_source; if (expires_out) *expires_out = TIME_MAX; - return 1; + return (rewrites > 0); } /** If we have a cached reverse DNS entry for the address stored in the @@ -990,11 +1179,10 @@ addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out) static int addressmap_rewrite_reverse(char *address, size_t maxlen, time_t *expires_out) { - size_t len = maxlen + 16; - char *s = tor_malloc(len), *cp; + char *s, *cp; addressmap_entry_t *ent; int r = 0; - tor_snprintf(s, len, "REVERSE[%s]", address); + tor_asprintf(&s, "REVERSE[%s]", address); ent = strmap_get(addressmap, s); if (ent) { cp = tor_strdup(escaped_safe_str_client(ent->new_address)); @@ -1035,17 +1223,34 @@ addressmap_have_mapping(const char *address, int update_expiry) * <b>new_address</b> should be a newly dup'ed string, which we'll use or * free as appropriate. We will leave address alone. * - * If <b>new_address</b> is NULL, or equal to <b>address</b>, remove - * any mappings that exist from <b>address</b>. - */ + * If <b>wildcard_addr</b> is true, then the mapping will match any address + * equal to <b>address</b>, or any address ending with a period followed by + * <b>address</b>. If <b>wildcard_addr</b> and <b>wildcard_new_addr</b> are + * both true, the mapping will rewrite addresses that end with + * ".<b>address</b>" into ones that end with ".<b>new_address</b>." + * + * If <b>new_address</b> is NULL, or <b>new_address</b> is equal to + * <b>address</b> and <b>wildcard_addr</b> is equal to + * <b>wildcard_new_addr</b>, remove any mappings that exist from + * <b>address</b>. + * + * + * It is an error to set <b>wildcard_new_addr</b> if <b>wildcard_addr</b> is + * not set. */ void addressmap_register(const char *address, char *new_address, time_t expires, - addressmap_entry_source_t source) + addressmap_entry_source_t source, + const int wildcard_addr, + const int wildcard_new_addr) { addressmap_entry_t *ent; + if (wildcard_new_addr) + tor_assert(wildcard_addr); + ent = strmap_get(addressmap, address); - if (!new_address || !strcasecmp(address,new_address)) { + if (!new_address || (!strcasecmp(address,new_address) && + wildcard_addr == wildcard_new_addr)) { /* Remove the mapping, if any. */ tor_free(new_address); if (ent) { @@ -1080,6 +1285,8 @@ addressmap_register(const char *address, char *new_address, time_t expires, ent->expires = expires==2 ? 1 : expires; ent->num_resolve_failures = 0; ent->source = source; + ent->src_wildcard = wildcard_addr ? 1 : 0; + ent->dst_wildcard = wildcard_new_addr ? 1 : 0; log_info(LD_CONFIG, "Addressmap: (re)mapped '%s' to '%s'", safe_str_client(address), @@ -1164,7 +1371,7 @@ client_dns_set_addressmap_impl(const char *address, const char *name, "%s", name); } addressmap_register(extendedaddress, tor_strdup(extendedval), - time(NULL) + ttl, ADDRMAPSRC_DNS); + time(NULL) + ttl, ADDRMAPSRC_DNS, 0, 0); } /** Record the fact that <b>address</b> resolved to <b>val</b>. @@ -1209,9 +1416,8 @@ client_dns_set_reverse_addressmap(const char *address, const char *v, const char *exitname, int ttl) { - size_t len = strlen(address) + 16; - char *s = tor_malloc(len); - tor_snprintf(s, len, "REVERSE[%s]", address); + char *s = NULL; + tor_asprintf(&s, "REVERSE[%s]", address); client_dns_set_addressmap_impl(s, v, exitname, ttl); tor_free(s); } @@ -1314,7 +1520,6 @@ static char * addressmap_get_virtual_address(int type) { char buf[64]; - struct in_addr in; tor_assert(addressmap); if (type == RESOLVED_TYPE_HOSTNAME) { @@ -1328,6 +1533,7 @@ addressmap_get_virtual_address(int type) } else if (type == RESOLVED_TYPE_IPV4) { // This is an imperfect estimate of how many addresses are available, but // that's ok. + struct in_addr in; uint32_t available = 1u << (32-virtual_addr_netmask_bits); while (available) { /* Don't hand out any .0 or .255 address. */ @@ -1416,7 +1622,7 @@ addressmap_register_virtual_address(int type, char *new_address) log_info(LD_APP, "Registering map from %s to %s", *addrp, new_address); if (vent_needs_to_be_added) strmap_set(virtaddress_reversemap, new_address, vent); - addressmap_register(*addrp, new_address, 2, ADDRMAPSRC_AUTOMAP); + addressmap_register(*addrp, new_address, 2, ADDRMAPSRC_AUTOMAP, 0, 0); #if 0 { @@ -1495,21 +1701,23 @@ addressmap_get_mappings(smartlist_t *sl, time_t min_expires, addressmap_ent_remove(key, val); continue; } else if (val->new_address) { - size_t len = strlen(key)+strlen(val->new_address)+ISO_TIME_LEN+5; - char *line = tor_malloc(len); + const char *src_wc = val->src_wildcard ? "*." : ""; + const char *dst_wc = val->dst_wildcard ? "*." : ""; if (want_expiry) { if (val->expires < 3 || val->expires == TIME_MAX) - tor_snprintf(line, len, "%s %s NEVER", key, val->new_address); + smartlist_add_asprintf(sl, "%s%s %s%s NEVER", + src_wc, key, dst_wc, val->new_address); else { char time[ISO_TIME_LEN+1]; format_iso_time(time, val->expires); - tor_snprintf(line, len, "%s %s \"%s\"", key, val->new_address, - time); + smartlist_add_asprintf(sl, "%s%s %s%s \"%s\"", + src_wc, key, dst_wc, val->new_address, + time); } } else { - tor_snprintf(line, len, "%s %s", key, val->new_address); + smartlist_add_asprintf(sl, "%s%s %s%s", + src_wc, key, dst_wc, val->new_address); } - smartlist_add(sl, line); } } iter = strmap_iter_next(addressmap,iter); @@ -1519,9 +1727,9 @@ addressmap_get_mappings(smartlist_t *sl, time_t min_expires, /** Check if <b>conn</b> is using a dangerous port. Then warn and/or * reject depending on our config options. */ static int -consider_plaintext_ports(edge_connection_t *conn, uint16_t port) +consider_plaintext_ports(entry_connection_t *conn, uint16_t port) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int reject = smartlist_string_num_isin(options->RejectPlaintextPorts, port); if (smartlist_string_num_isin(options->WarnPlaintextPorts, port)) { @@ -1554,14 +1762,14 @@ consider_plaintext_ports(edge_connection_t *conn, uint16_t port) * documentation for arguments and return value. */ int -connection_ap_rewrite_and_attach_if_allowed(edge_connection_t *conn, +connection_ap_rewrite_and_attach_if_allowed(entry_connection_t *conn, origin_circuit_t *circ, crypt_path_t *cpath) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (options->LeaveStreamsUnattached) { - conn->_base.state = AP_CONN_STATE_CONTROLLER_WAIT; + ENTRY_TO_CONN(conn)->state = AP_CONN_STATE_CONTROLLER_WAIT; return 0; } return connection_ap_handshake_rewrite_and_attach(conn, circ, cpath); @@ -1583,23 +1791,22 @@ connection_ap_rewrite_and_attach_if_allowed(edge_connection_t *conn, * <b>cpath</b> is NULL. */ int -connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, +connection_ap_handshake_rewrite_and_attach(entry_connection_t *conn, origin_circuit_t *circ, crypt_path_t *cpath) { socks_request_t *socks = conn->socks_request; hostname_type_t addresstype; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); struct in_addr addr_tmp; /* We set this to true if this is an address we should automatically * remap to a local address in VirtualAddrNetwork */ int automap = 0; char orig_address[MAX_SOCKS_ADDR_LEN]; time_t map_expires = TIME_MAX; - /* This will be set to true iff the address starts out as a non-.exit - address, and we remap it to one because of an entry in the addressmap. */ - int remapped_to_exit = 0; time_t now = time(NULL); + connection_t *base_conn = ENTRY_TO_CONN(conn); + addressmap_entry_source_t exit_source = ADDRMAPSRC_NONE; tor_strlower(socks->address); /* normalize it */ strlcpy(orig_address, socks->address, sizeof(orig_address)); @@ -1607,6 +1814,19 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, safe_str_client(socks->address), socks->port); + if (!strcmpend(socks->address, ".exit") && !options->AllowDotExit) { + log_warn(LD_APP, "The \".exit\" notation is disabled in Tor due to " + "security risks. Set AllowDotExit in your torrc to enable " + "it (at your own risk)."); + control_event_client_status(LOG_WARN, "SOCKS_BAD_HOSTNAME HOSTNAME=%s", + escaped(socks->address)); + connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); + return -1; + } + + if (! conn->original_dest_address) + conn->original_dest_address = tor_strdup(conn->socks_request->address); + if (socks->command == SOCKS_COMMAND_RESOLVE && !tor_inet_aton(socks->address, &addr_tmp) && options->AutomapHostsOnResolve && options->AutomapHostsSuffixes) { @@ -1652,7 +1872,7 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, /* Don't let people try to do a reverse lookup on 10.0.0.1. */ tor_addr_t addr; int ok; - ok = tor_addr_parse_reverse_lookup_name( + ok = tor_addr_parse_PTR_name( &addr, socks->address, AF_UNSPEC, 1); if (ok == 1 && tor_addr_is_internal(&addr, 0)) { connection_ap_handshake_socks_resolved(conn, RESOLVED_TYPE_ERROR, @@ -1664,16 +1884,11 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, } } } else if (!automap) { - int started_without_chosen_exit = strcasecmpend(socks->address, ".exit"); /* For address map controls, remap the address. */ if (addressmap_rewrite(socks->address, sizeof(socks->address), - &map_expires)) { + &map_expires, &exit_source)) { control_event_stream_status(conn, STREAM_EVENT_REMAP, REMAP_STREAM_SOURCE_CACHE); - if (started_without_chosen_exit && - !strcasecmpend(socks->address, ".exit") && - map_expires < TIME_MAX) - remapped_to_exit = 1; } } @@ -1692,12 +1907,9 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, /* Parse the address provided by SOCKS. Modify it in-place if it * specifies a hidden-service (.onion) or particular exit node (.exit). */ - addresstype = parse_extended_hostname(socks->address, - remapped_to_exit || options->AllowDotExit); + addresstype = parse_extended_hostname(socks->address); if (addresstype == BAD_HOSTNAME) { - log_warn(LD_APP, "Invalid onion hostname %s; rejecting", - safe_str_client(socks->address)); control_event_client_status(LOG_WARN, "SOCKS_BAD_HOSTNAME HOSTNAME=%s", escaped(socks->address)); connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); @@ -1712,17 +1924,44 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, /* If StrictNodes is not set, then .exit overrides ExcludeNodes. */ routerset_t *excludeset = options->StrictNodes ? options->_ExcludeExitNodesUnion : options->ExcludeExitNodes; - /*XXX023 make this a node_t. */ - routerinfo_t *router; + const node_t *node; + + if (exit_source == ADDRMAPSRC_AUTOMAP && !options->AllowDotExit) { + /* Whoops; this one is stale. It must have gotten added earlier, + * when AllowDotExit was on. */ + log_warn(LD_APP,"Stale automapped address for '%s.exit', with " + "AllowDotExit disabled. Refusing.", + safe_str_client(socks->address)); + control_event_client_status(LOG_WARN, "SOCKS_BAD_HOSTNAME HOSTNAME=%s", + escaped(socks->address)); + connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); + return -1; + } + + if (exit_source == ADDRMAPSRC_DNS || + (exit_source == ADDRMAPSRC_NONE && !options->AllowDotExit)) { + /* It shouldn't be possible to get a .exit address from any of these + * sources. */ + log_warn(LD_BUG,"Address '%s.exit', with impossible source for the " + ".exit part. Refusing.", + safe_str_client(socks->address)); + control_event_client_status(LOG_WARN, "SOCKS_BAD_HOSTNAME HOSTNAME=%s", + escaped(socks->address)); + connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); + return -1; + } tor_assert(!automap); if (s) { /* The address was of the form "(stuff).(name).exit */ if (s[1] != '\0') { conn->chosen_exit_name = tor_strdup(s+1); - router = router_get_by_nickname(conn->chosen_exit_name, 1); - if (remapped_to_exit) /* 5 tries before it expires the addressmap */ + node = node_get_by_nickname(conn->chosen_exit_name, 1); + + if (exit_source == ADDRMAPSRC_TRACKEXIT) { + /* We 5 tries before it expires the addressmap */ conn->chosen_exit_retries = TRACKHOSTEXITS_RETRIES; + } *s = 0; } else { /* Oops, the address was (stuff)..exit. That's not okay. */ @@ -1735,15 +1974,16 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, } } else { /* It looks like they just asked for "foo.exit". */ + conn->chosen_exit_name = tor_strdup(socks->address); - router = router_get_by_nickname(conn->chosen_exit_name, 1); - if (router) { + node = node_get_by_nickname(conn->chosen_exit_name, 1); + if (node) { *socks->address = 0; - strlcpy(socks->address, router->address, sizeof(socks->address)); + node_get_address_string(node, socks->address, sizeof(socks->address)); } } /* Now make sure that the chosen exit exists... */ - if (!router) { + if (!node) { log_warn(LD_APP, "Unrecognized relay in exit address '%s.exit'. Refusing.", safe_str_client(socks->address)); @@ -1751,14 +1991,14 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, return -1; } /* ...and make sure that it isn't excluded. */ - if (routerset_contains_router(excludeset, router)) { + if (routerset_contains_node(excludeset, node)) { log_warn(LD_APP, "Excluded relay in exit address '%s.exit'. Refusing.", safe_str_client(socks->address)); connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); return -1; } - /* XXXX022-1090 Should we also allow foo.bar.exit if ExitNodes is set and + /* XXXX024-1090 Should we also allow foo.bar.exit if ExitNodes is set and Bar is not listed in it? I say yes, but our revised manpage branch implies no. */ } @@ -1775,6 +2015,14 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, return -1; } + if (options->Tor2webMode) { + log_warn(LD_APP, "Refusing to connect to non-hidden-service hostname %s " + "because tor2web mode is enabled.", + safe_str_client(socks->address)); + connection_mark_unattached_ap(conn, END_STREAM_REASON_ENTRYPOLICY); + return -1; + } + if (socks->command == SOCKS_COMMAND_RESOLVE) { uint32_t answer; struct in_addr in; @@ -1804,20 +2052,35 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, if (options->ClientRejectInternalAddresses && !conn->use_begindir && !conn->chosen_exit_name && !circ) { tor_addr_t addr; - if (tor_addr_from_str(&addr, socks->address) >= 0 && - tor_addr_is_internal(&addr, 0)) { + if (tor_addr_hostname_is_local(socks->address) || + (tor_addr_parse(&addr, socks->address) >= 0 && + tor_addr_is_internal(&addr, 0))) { /* If this is an explicit private address with no chosen exit node, * then we really don't want to try to connect to it. That's * probably an error. */ if (conn->is_transparent_ap) { - log_warn(LD_NET, - "Rejecting request for anonymous connection to private " - "address %s on a TransPort or NATDPort. Possible loop " - "in your NAT rules?", safe_str_client(socks->address)); +#define WARN_INTRVL_LOOP 300 + static ratelim_t loop_warn_limit = RATELIM_INIT(WARN_INTRVL_LOOP); + char *m; + if ((m = rate_limit_log(&loop_warn_limit, approx_time()))) { + log_warn(LD_NET, + "Rejecting request for anonymous connection to private " + "address %s on a TransPort or NATDPort. Possible loop " + "in your NAT rules?%s", safe_str_client(socks->address), + m); + tor_free(m); + } } else { - log_warn(LD_NET, - "Rejecting SOCKS request for anonymous connection to " - "private address %s", safe_str_client(socks->address)); +#define WARN_INTRVL_PRIV 300 + static ratelim_t priv_warn_limit = RATELIM_INIT(WARN_INTRVL_PRIV); + char *m; + if ((m = rate_limit_log(&priv_warn_limit, approx_time()))) { + log_warn(LD_NET, + "Rejecting SOCKS request for anonymous connection to " + "private address %s.%s", + safe_str_client(socks->address),m); + tor_free(m); + } } connection_mark_unattached_ap(conn, END_STREAM_REASON_PRIVATE_ADDR); return -1; @@ -1826,17 +2089,16 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, if (!conn->use_begindir && !conn->chosen_exit_name && !circ) { /* see if we can find a suitable enclave exit */ - routerinfo_t *r = + const node_t *r = router_find_exact_exit_enclave(socks->address, socks->port); if (r) { log_info(LD_APP, "Redirecting address %s to exit at enclave router %s", - safe_str_client(socks->address), - router_describe(r)); + safe_str_client(socks->address), node_describe(r)); /* use the hex digest, not nickname, in case there are two routers with this nickname */ conn->chosen_exit_name = - tor_strdup(hex_str(r->cache_info.identity_digest, DIGEST_LEN)); + tor_strdup(hex_str(r->identity, DIGEST_LEN)); conn->chosen_exit_optional = 1; } } @@ -1856,12 +2118,12 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, } else { tor_fragile_assert(); } - conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; + base_conn->state = AP_CONN_STATE_CIRCUIT_WAIT; if ((circ && connection_ap_handshake_attach_chosen_circuit( conn, circ, cpath) < 0) || (!circ && connection_ap_handshake_attach_circuit(conn) < 0)) { - if (!conn->_base.marked_for_close) + if (!base_conn->marked_for_close) connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); return -1; } @@ -1871,6 +2133,7 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, rend_cache_entry_t *entry; int r; rend_service_authorization_t *client_auth; + rend_data_t *rend_data; tor_assert(!automap); if (SOCKS_COMMAND_IS_RESOLVE(socks->command)) { /* if it's a resolve request, fail it right now, rather than @@ -1892,16 +2155,17 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, return -1; } - conn->rend_data = tor_malloc_zero(sizeof(rend_data_t)); - strlcpy(conn->rend_data->onion_address, socks->address, - sizeof(conn->rend_data->onion_address)); + ENTRY_TO_EDGE_CONN(conn)->rend_data = rend_data = + tor_malloc_zero(sizeof(rend_data_t)); + strlcpy(rend_data->onion_address, socks->address, + sizeof(rend_data->onion_address)); log_info(LD_REND,"Got a hidden service request for ID '%s'", - safe_str_client(conn->rend_data->onion_address)); + safe_str_client(rend_data->onion_address)); /* see if we already have it cached */ - r = rend_cache_lookup_entry(conn->rend_data->onion_address, -1, &entry); + r = rend_cache_lookup_entry(rend_data->onion_address, -1, &entry); if (r<0) { log_warn(LD_BUG,"Invalid service name '%s'", - safe_str_client(conn->rend_data->onion_address)); + safe_str_client(rend_data->onion_address)); connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); return -1; } @@ -1912,24 +2176,24 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, /* Look up if we have client authorization for it. */ client_auth = rend_client_lookup_service_authorization( - conn->rend_data->onion_address); + rend_data->onion_address); if (client_auth) { log_info(LD_REND, "Using previously configured client authorization " "for hidden service request."); - memcpy(conn->rend_data->descriptor_cookie, + memcpy(rend_data->descriptor_cookie, client_auth->descriptor_cookie, REND_DESC_COOKIE_LEN); - conn->rend_data->auth_type = client_auth->auth_type; + rend_data->auth_type = client_auth->auth_type; } if (r==0) { - conn->_base.state = AP_CONN_STATE_RENDDESC_WAIT; + base_conn->state = AP_CONN_STATE_RENDDESC_WAIT; log_info(LD_REND, "Unknown descriptor %s. Fetching.", - safe_str_client(conn->rend_data->onion_address)); - rend_client_refetch_v2_renddesc(conn->rend_data); + safe_str_client(rend_data->onion_address)); + rend_client_refetch_v2_renddesc(rend_data); } else { /* r > 0 */ - conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; + base_conn->state = AP_CONN_STATE_CIRCUIT_WAIT; log_info(LD_REND, "Descriptor is here. Great."); if (connection_ap_handshake_attach_circuit(conn) < 0) { - if (!conn->_base.marked_for_close) + if (!base_conn->marked_for_close) connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); return -1; } @@ -1951,10 +2215,10 @@ get_pf_socket(void) #ifdef OPENBSD /* only works on OpenBSD */ - pf = open("/dev/pf", O_RDONLY); + pf = tor_open_cloexec("/dev/pf", O_RDONLY, 0); #else /* works on NetBSD and FreeBSD */ - pf = open("/dev/pf", O_RDWR); + pf = tor_open_cloexec("/dev/pf", O_RDWR, 0); #endif if (pf < 0) { @@ -1975,7 +2239,7 @@ get_pf_socket(void) * else return 0. */ static int -connection_ap_get_original_destination(edge_connection_t *conn, +connection_ap_get_original_destination(entry_connection_t *conn, socks_request_t *req) { #ifdef TRANS_NETFILTER @@ -1984,9 +2248,9 @@ connection_ap_get_original_destination(edge_connection_t *conn, socklen_t orig_dst_len = sizeof(orig_dst); tor_addr_t addr; - if (getsockopt(conn->_base.s, SOL_IP, SO_ORIGINAL_DST, + if (getsockopt(ENTRY_TO_CONN(conn)->s, SOL_IP, SO_ORIGINAL_DST, (struct sockaddr*)&orig_dst, &orig_dst_len) < 0) { - int e = tor_socket_errno(conn->_base.s); + int e = tor_socket_errno(ENTRY_TO_CONN(conn)->s); log_warn(LD_NET, "getsockopt() failed: %s", tor_socket_strerror(e)); return -1; } @@ -2003,9 +2267,9 @@ connection_ap_get_original_destination(edge_connection_t *conn, tor_addr_t addr; int pf = -1; - if (getsockname(conn->_base.s, (struct sockaddr*)&proxy_addr, + if (getsockname(ENTRY_TO_CONN(conn)->s, (struct sockaddr*)&proxy_addr, &proxy_addr_len) < 0) { - int e = tor_socket_errno(conn->_base.s); + int e = tor_socket_errno(ENTRY_TO_CONN(conn)->s); log_warn(LD_NET, "getsockname() to determine transocks destination " "failed: %s", tor_socket_strerror(e)); return -1; @@ -2017,16 +2281,16 @@ connection_ap_get_original_destination(edge_connection_t *conn, if (proxy_sa->sa_family == AF_INET) { struct sockaddr_in *sin = (struct sockaddr_in *)proxy_sa; pnl.af = AF_INET; - pnl.saddr.v4.s_addr = tor_addr_to_ipv4n(&conn->_base.addr); - pnl.sport = htons(conn->_base.port); + pnl.saddr.v4.s_addr = tor_addr_to_ipv4n(&ENTRY_TO_CONN(conn)->addr); + pnl.sport = htons(ENTRY_TO_CONN(conn)->port); pnl.daddr.v4.s_addr = sin->sin_addr.s_addr; pnl.dport = sin->sin_port; } else if (proxy_sa->sa_family == AF_INET6) { struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)proxy_sa; pnl.af = AF_INET6; - memcpy(&pnl.saddr.v6, tor_addr_to_in6(&conn->_base.addr), + memcpy(&pnl.saddr.v6, tor_addr_to_in6(&ENTRY_TO_CONN(conn)->addr), sizeof(struct in6_addr)); - pnl.sport = htons(conn->_base.port); + pnl.sport = htons(ENTRY_TO_CONN(conn)->port); memcpy(&pnl.daddr.v6, &sin6->sin6_addr, sizeof(struct in6_addr)); pnl.dport = sin6->sin6_port; } else { @@ -2077,38 +2341,48 @@ connection_ap_get_original_destination(edge_connection_t *conn, * else return 0. */ static int -connection_ap_handshake_process_socks(edge_connection_t *conn) +connection_ap_handshake_process_socks(entry_connection_t *conn) { socks_request_t *socks; int sockshere; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); + int had_reply = 0; + connection_t *base_conn = ENTRY_TO_CONN(conn); tor_assert(conn); - tor_assert(conn->_base.type == CONN_TYPE_AP); - tor_assert(conn->_base.state == AP_CONN_STATE_SOCKS_WAIT); + tor_assert(base_conn->type == CONN_TYPE_AP); + tor_assert(base_conn->state == AP_CONN_STATE_SOCKS_WAIT); tor_assert(conn->socks_request); socks = conn->socks_request; log_debug(LD_APP,"entered."); - sockshere = fetch_from_buf_socks(conn->_base.inbuf, socks, - options->TestSocks, options->SafeSocks); - if (sockshere == 0) { - if (socks->replylen) { - connection_write_to_buf(socks->reply, socks->replylen, TO_CONN(conn)); - /* zero it out so we can do another round of negotiation */ - socks->replylen = 0; - } else { - log_debug(LD_APP,"socks handshake not all here yet."); + IF_HAS_BUFFEREVENT(base_conn, { + struct evbuffer *input = bufferevent_get_input(base_conn->bufev); + sockshere = fetch_from_evbuffer_socks(input, socks, + options->TestSocks, options->SafeSocks); + }) ELSE_IF_NO_BUFFEREVENT { + sockshere = fetch_from_buf_socks(base_conn->inbuf, socks, + options->TestSocks, options->SafeSocks); + }; + + if (socks->replylen) { + had_reply = 1; + connection_write_to_buf((const char*)socks->reply, socks->replylen, + base_conn); + socks->replylen = 0; + if (sockshere == -1) { + /* An invalid request just got a reply, no additional + * one is necessary. */ + socks->has_finished = 1; } + } + + if (sockshere == 0) { + log_debug(LD_APP,"socks handshake not all here yet."); return 0; } else if (sockshere == -1) { - if (socks->replylen) { /* we should send reply back */ - log_debug(LD_APP,"reply is already set for us. Using it."); - connection_ap_handshake_socks_reply(conn, socks->reply, socks->replylen, - END_STREAM_REASON_SOCKSPROTOCOL); - - } else { + if (!had_reply) { log_warn(LD_APP,"Fetching socks handshake failed. Closing."); connection_ap_handshake_socks_reply(conn, NULL, 0, END_STREAM_REASON_SOCKSPROTOCOL); @@ -2135,12 +2409,11 @@ connection_ap_handshake_process_socks(edge_connection_t *conn) * for close), else return 0. */ int -connection_ap_process_transparent(edge_connection_t *conn) +connection_ap_process_transparent(entry_connection_t *conn) { socks_request_t *socks; tor_assert(conn); - tor_assert(conn->_base.type == CONN_TYPE_AP); tor_assert(conn->socks_request); socks = conn->socks_request; @@ -2176,7 +2449,7 @@ connection_ap_process_transparent(edge_connection_t *conn) * for close), else return 0. */ static int -connection_ap_process_natd(edge_connection_t *conn) +connection_ap_process_natd(entry_connection_t *conn) { char tmp_buf[36], *tbuf, *daddr; size_t tlen = 30; @@ -2184,8 +2457,7 @@ connection_ap_process_natd(edge_connection_t *conn) socks_request_t *socks; tor_assert(conn); - tor_assert(conn->_base.type == CONN_TYPE_AP); - tor_assert(conn->_base.state == AP_CONN_STATE_NATD_WAIT); + tor_assert(ENTRY_TO_CONN(conn)->state == AP_CONN_STATE_NATD_WAIT); tor_assert(conn->socks_request); socks = conn->socks_request; @@ -2193,7 +2465,7 @@ connection_ap_process_natd(edge_connection_t *conn) /* look for LF-terminated "[DEST ip_addr port]" * where ip_addr is a dotted-quad and port is in string form */ - err = fetch_from_buf_line(conn->_base.inbuf, tmp_buf, &tlen); + err = connection_fetch_from_buf_line(ENTRY_TO_CONN(conn), tmp_buf, &tlen); if (err == 0) return 0; if (err < 0) { @@ -2237,7 +2509,7 @@ connection_ap_process_natd(edge_connection_t *conn) control_event_stream_status(conn, STREAM_EVENT_NEW, 0); - conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; + ENTRY_TO_CONN(conn)->state = AP_CONN_STATE_CIRCUIT_WAIT; return connection_ap_rewrite_and_attach_if_allowed(conn, NULL, NULL); } @@ -2267,34 +2539,52 @@ get_unique_stream_id_by_circ(origin_circuit_t *circ) return test_stream_id; } +/** Return true iff <b>conn</b> is linked to a circuit and configured to use + * an exit that supports optimistic data. */ +static int +connection_ap_supports_optimistic_data(const entry_connection_t *conn) +{ + const edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(conn); + /* We can only send optimistic data if we're connected to an open + general circuit. */ + if (edge_conn->on_circuit == NULL || + edge_conn->on_circuit->state != CIRCUIT_STATE_OPEN || + edge_conn->on_circuit->purpose != CIRCUIT_PURPOSE_C_GENERAL) + return 0; + + return conn->may_use_optimistic_data; +} + /** Write a relay begin cell, using destaddr and destport from ap_conn's * socks_request field, and send it down circ. * * If ap_conn is broken, mark it for close and return -1. Else return 0. */ int -connection_ap_handshake_send_begin(edge_connection_t *ap_conn) +connection_ap_handshake_send_begin(entry_connection_t *ap_conn) { char payload[CELL_PAYLOAD_SIZE]; int payload_len; int begin_type; origin_circuit_t *circ; - tor_assert(ap_conn->on_circuit); - circ = TO_ORIGIN_CIRCUIT(ap_conn->on_circuit); + edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(ap_conn); + connection_t *base_conn = TO_CONN(edge_conn); + tor_assert(edge_conn->on_circuit); + circ = TO_ORIGIN_CIRCUIT(edge_conn->on_circuit); - tor_assert(ap_conn->_base.type == CONN_TYPE_AP); - tor_assert(ap_conn->_base.state == AP_CONN_STATE_CIRCUIT_WAIT); + tor_assert(base_conn->type == CONN_TYPE_AP); + tor_assert(base_conn->state == AP_CONN_STATE_CIRCUIT_WAIT); tor_assert(ap_conn->socks_request); tor_assert(SOCKS_COMMAND_IS_CONNECT(ap_conn->socks_request->command)); - ap_conn->stream_id = get_unique_stream_id_by_circ(circ); - if (ap_conn->stream_id==0) { - /* XXXX023 Instead of closing this stream, we should make it get + edge_conn->stream_id = get_unique_stream_id_by_circ(circ); + if (edge_conn->stream_id==0) { + /* XXXX024 Instead of closing this stream, we should make it get * retried on another circuit. */ connection_mark_unattached_ap(ap_conn, END_STREAM_REASON_INTERNAL); /* Mark this circuit "unusable for new streams". */ - /* XXXX023 this is a kludgy way to do this. */ + /* XXXX024 this is a kludgy way to do this. */ tor_assert(circ->_base.timestamp_dirty); circ->_base.timestamp_dirty -= get_options()->MaxCircuitDirtiness; return -1; @@ -2306,26 +2596,44 @@ connection_ap_handshake_send_begin(edge_connection_t *ap_conn) ap_conn->socks_request->port); payload_len = (int)strlen(payload)+1; - log_debug(LD_APP, - "Sending relay cell to begin stream %d.", ap_conn->stream_id); + log_info(LD_APP, + "Sending relay cell %d to begin stream %d.", + (int)ap_conn->use_begindir, + edge_conn->stream_id); begin_type = ap_conn->use_begindir ? RELAY_COMMAND_BEGIN_DIR : RELAY_COMMAND_BEGIN; if (begin_type == RELAY_COMMAND_BEGIN) { +#ifndef NON_ANONYMOUS_MODE_ENABLED tor_assert(circ->build_state->onehop_tunnel == 0); +#endif } - if (connection_edge_send_command(ap_conn, begin_type, + if (connection_edge_send_command(edge_conn, begin_type, begin_type == RELAY_COMMAND_BEGIN ? payload : NULL, begin_type == RELAY_COMMAND_BEGIN ? payload_len : 0) < 0) return -1; /* circuit is closed, don't continue */ - ap_conn->package_window = STREAMWINDOW_START; - ap_conn->deliver_window = STREAMWINDOW_START; - ap_conn->_base.state = AP_CONN_STATE_CONNECT_WAIT; + edge_conn->package_window = STREAMWINDOW_START; + edge_conn->deliver_window = STREAMWINDOW_START; + base_conn->state = AP_CONN_STATE_CONNECT_WAIT; log_info(LD_APP,"Address/port sent, ap socket %d, n_circ_id %d", - ap_conn->_base.s, circ->_base.n_circ_id); + base_conn->s, circ->_base.n_circ_id); control_event_stream_status(ap_conn, STREAM_EVENT_SENT_CONNECT, 0); + + /* If there's queued-up data, send it now */ + if ((connection_get_inbuf_len(base_conn) || + ap_conn->sending_optimistic_data) && + connection_ap_supports_optimistic_data(ap_conn)) { + log_info(LD_APP, "Sending up to %ld + %ld bytes of queued-up data", + (long)connection_get_inbuf_len(base_conn), + ap_conn->sending_optimistic_data ? + (long)generic_buffer_len(ap_conn->sending_optimistic_data) : 0); + if (connection_edge_package_raw_inbuf(edge_conn, 1, NULL) < 0) { + connection_mark_for_close(base_conn); + } + } + return 0; } @@ -2335,31 +2643,33 @@ connection_ap_handshake_send_begin(edge_connection_t *ap_conn) * If ap_conn is broken, mark it for close and return -1. Else return 0. */ int -connection_ap_handshake_send_resolve(edge_connection_t *ap_conn) +connection_ap_handshake_send_resolve(entry_connection_t *ap_conn) { int payload_len, command; const char *string_addr; char inaddr_buf[REVERSE_LOOKUP_NAME_BUF_LEN]; origin_circuit_t *circ; - tor_assert(ap_conn->on_circuit); - circ = TO_ORIGIN_CIRCUIT(ap_conn->on_circuit); + edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(ap_conn); + connection_t *base_conn = TO_CONN(edge_conn); + tor_assert(edge_conn->on_circuit); + circ = TO_ORIGIN_CIRCUIT(edge_conn->on_circuit); - tor_assert(ap_conn->_base.type == CONN_TYPE_AP); - tor_assert(ap_conn->_base.state == AP_CONN_STATE_CIRCUIT_WAIT); + tor_assert(base_conn->type == CONN_TYPE_AP); + tor_assert(base_conn->state == AP_CONN_STATE_CIRCUIT_WAIT); tor_assert(ap_conn->socks_request); tor_assert(circ->_base.purpose == CIRCUIT_PURPOSE_C_GENERAL); command = ap_conn->socks_request->command; tor_assert(SOCKS_COMMAND_IS_RESOLVE(command)); - ap_conn->stream_id = get_unique_stream_id_by_circ(circ); - if (ap_conn->stream_id==0) { - /* XXXX023 Instead of closing this stream, we should make it get + edge_conn->stream_id = get_unique_stream_id_by_circ(circ); + if (edge_conn->stream_id==0) { + /* XXXX024 Instead of closing this stream, we should make it get * retried on another circuit. */ connection_mark_unattached_ap(ap_conn, END_STREAM_REASON_INTERNAL); /* Mark this circuit "unusable for new streams". */ - /* XXXX023 this is a kludgy way to do this. */ + /* XXXX024 this is a kludgy way to do this. */ tor_assert(circ->_base.timestamp_dirty); circ->_base.timestamp_dirty -= get_options()->MaxCircuitDirtiness; return -1; @@ -2376,7 +2686,7 @@ connection_ap_handshake_send_resolve(edge_connection_t *ap_conn) /* We're doing a reverse lookup. The input could be an IP address, or * could be an .in-addr.arpa or .ip6.arpa address */ - r = tor_addr_parse_reverse_lookup_name(&addr, a, AF_INET, 1); + r = tor_addr_parse_PTR_name(&addr, a, AF_INET, 1); if (r <= 0) { log_warn(LD_APP, "Rejecting ill-formed reverse lookup of %s", safe_str_client(a)); @@ -2384,7 +2694,7 @@ connection_ap_handshake_send_resolve(edge_connection_t *ap_conn) return -1; } - r = tor_addr_to_reverse_lookup_name(inaddr_buf, sizeof(inaddr_buf), &addr); + r = tor_addr_to_PTR_name(inaddr_buf, sizeof(inaddr_buf), &addr); if (r < 0) { log_warn(LD_BUG, "Couldn't generate reverse lookup hostname of %s", safe_str_client(a)); @@ -2398,41 +2708,47 @@ connection_ap_handshake_send_resolve(edge_connection_t *ap_conn) } log_debug(LD_APP, - "Sending relay cell to begin stream %d.", ap_conn->stream_id); + "Sending relay cell to begin stream %d.", edge_conn->stream_id); - if (connection_edge_send_command(ap_conn, + if (connection_edge_send_command(edge_conn, RELAY_COMMAND_RESOLVE, string_addr, payload_len) < 0) return -1; /* circuit is closed, don't continue */ - tor_free(ap_conn->_base.address); /* Maybe already set by dnsserv. */ - ap_conn->_base.address = tor_strdup("(Tor_internal)"); - ap_conn->_base.state = AP_CONN_STATE_RESOLVE_WAIT; + tor_free(base_conn->address); /* Maybe already set by dnsserv. */ + base_conn->address = tor_strdup("(Tor_internal)"); + base_conn->state = AP_CONN_STATE_RESOLVE_WAIT; log_info(LD_APP,"Address sent for resolve, ap socket %d, n_circ_id %d", - ap_conn->_base.s, circ->_base.n_circ_id); + base_conn->s, circ->_base.n_circ_id); control_event_stream_status(ap_conn, STREAM_EVENT_NEW, 0); control_event_stream_status(ap_conn, STREAM_EVENT_SENT_RESOLVE, 0); return 0; } -/** Make an AP connection_t, make a new linked connection pair, and attach - * one side to the conn, connection_add it, initialize it to circuit_wait, - * and call connection_ap_handshake_attach_circuit(conn) on it. +/** Make an AP connection_t linked to the connection_t <b>partner</b>. make a + * new linked connection pair, and attach one side to the conn, connection_add + * it, initialize it to circuit_wait, and call + * connection_ap_handshake_attach_circuit(conn) on it. * - * Return the other end of the linked connection pair, or -1 if error. + * Return the newly created end of the linked connection pair, or -1 if error. */ -edge_connection_t * -connection_ap_make_link(char *address, uint16_t port, - const char *digest, int use_begindir, int want_onehop) +entry_connection_t * +connection_ap_make_link(connection_t *partner, + char *address, uint16_t port, + const char *digest, + int session_group, int isolation_flags, + int use_begindir, int want_onehop) { - edge_connection_t *conn; + entry_connection_t *conn; + connection_t *base_conn; log_info(LD_APP,"Making internal %s tunnel to %s:%d ...", want_onehop ? "direct" : "anonymized", safe_str_client(address), port); - conn = edge_connection_new(CONN_TYPE_AP, AF_INET); - conn->_base.linked = 1; /* so that we can add it safely below. */ + conn = entry_connection_new(CONN_TYPE_AP, tor_addr_family(&partner->addr)); + base_conn = ENTRY_TO_CONN(conn); + base_conn->linked = 1; /* so that we can add it safely below. */ /* populate conn->socks_request */ @@ -2453,22 +2769,30 @@ connection_ap_make_link(char *address, uint16_t port, digest, DIGEST_LEN); } - conn->_base.address = tor_strdup("(Tor_internal)"); - tor_addr_make_unspec(&conn->_base.addr); - conn->_base.port = 0; + /* Populate isolation fields. */ + conn->socks_request->listener_type = CONN_TYPE_DIR_LISTENER; + conn->original_dest_address = tor_strdup(address); + conn->session_group = session_group; + conn->isolation_flags = isolation_flags; - if (connection_add(TO_CONN(conn)) < 0) { /* no space, forget it */ - connection_free(TO_CONN(conn)); + base_conn->address = tor_strdup("(Tor_internal)"); + tor_addr_make_unspec(&base_conn->addr); + base_conn->port = 0; + + connection_link_connections(partner, base_conn); + + if (connection_add(base_conn) < 0) { /* no space, forget it */ + connection_free(base_conn); return NULL; } - conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; + base_conn->state = AP_CONN_STATE_CIRCUIT_WAIT; control_event_stream_status(conn, STREAM_EVENT_NEW, 0); /* attaching to a dirty circuit is fine */ if (connection_ap_handshake_attach_circuit(conn) < 0) { - if (!conn->_base.marked_for_close) + if (!base_conn->marked_for_close) connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); return NULL; } @@ -2481,7 +2805,7 @@ connection_ap_make_link(char *address, uint16_t port, * or resolve error. Takes the same arguments as does * connection_ap_handshake_socks_resolved(). */ static void -tell_controller_about_resolved_result(edge_connection_t *conn, +tell_controller_about_resolved_result(entry_connection_t *conn, int answer_type, size_t answer_len, const char *answer, @@ -2493,13 +2817,11 @@ tell_controller_about_resolved_result(edge_connection_t *conn, answer_type == RESOLVED_TYPE_HOSTNAME)) { return; /* we already told the controller. */ } else if (answer_type == RESOLVED_TYPE_IPV4 && answer_len >= 4) { - struct in_addr in; - char buf[INET_NTOA_BUF_LEN]; - in.s_addr = get_uint32(answer); - tor_inet_ntoa(&in, buf, sizeof(buf)); + char *cp = tor_dup_ip(ntohl(get_uint32(answer))); control_event_address_mapped(conn->socks_request->address, - buf, expires, NULL); - } else if (answer_type == RESOLVED_TYPE_HOSTNAME && answer_len <256) { + cp, expires, NULL); + tor_free(cp); + } else if (answer_type == RESOLVED_TYPE_HOSTNAME && answer_len < 256) { char *cp = tor_strndup(answer, answer_len); control_event_address_mapped(conn->socks_request->address, cp, expires, NULL); @@ -2519,10 +2841,10 @@ tell_controller_about_resolved_result(edge_connection_t *conn, * certain errors or for values that didn't come via DNS. <b>expires</b> is * a time when the answer expires, or -1 or TIME_MAX if there's a good TTL. **/ -/* XXXX023 the use of the ttl and expires fields is nutty. Let's make this +/* XXXX the use of the ttl and expires fields is nutty. Let's make this * interface and those that use it less ugly. */ void -connection_ap_handshake_socks_resolved(edge_connection_t *conn, +connection_ap_handshake_socks_resolved(entry_connection_t *conn, int answer_type, size_t answer_len, const uint8_t *answer, @@ -2547,7 +2869,7 @@ connection_ap_handshake_socks_resolved(edge_connection_t *conn, } } - if (conn->is_dns_request) { + if (ENTRY_TO_EDGE_CONN(conn)->is_dns_request) { if (conn->dns_server_request) { /* We had a request on our DNS port: answer it. */ dnsserv_resolved(conn, answer_type, answer_len, (char*)answer, ttl); @@ -2627,7 +2949,7 @@ connection_ap_handshake_socks_resolved(edge_connection_t *conn, * be 0 or REASON_DONE. Send endreason to the controller, if appropriate. */ void -connection_ap_handshake_socks_reply(edge_connection_t *conn, char *reply, +connection_ap_handshake_socks_reply(entry_connection_t *conn, char *reply, size_t replylen, int endreason) { char buf[256]; @@ -2646,7 +2968,7 @@ connection_ap_handshake_socks_reply(edge_connection_t *conn, char *reply, return; } if (replylen) { /* we already have a reply in mind */ - connection_write_to_buf(reply, replylen, TO_CONN(conn)); + connection_write_to_buf(reply, replylen, ENTRY_TO_CONN(conn)); conn->socks_request->has_finished = 1; return; } @@ -2654,7 +2976,7 @@ connection_ap_handshake_socks_reply(edge_connection_t *conn, char *reply, memset(buf,0,SOCKS4_NETWORK_LEN); buf[1] = (status==SOCKS5_SUCCEEDED ? SOCKS4_GRANTED : SOCKS4_REJECT); /* leave version, destport, destip zero */ - connection_write_to_buf(buf, SOCKS4_NETWORK_LEN, TO_CONN(conn)); + connection_write_to_buf(buf, SOCKS4_NETWORK_LEN, ENTRY_TO_CONN(conn)); } else if (conn->socks_request->socks_version == 5) { buf[0] = 5; /* version 5 */ buf[1] = (char)status; @@ -2662,7 +2984,7 @@ connection_ap_handshake_socks_reply(edge_connection_t *conn, char *reply, buf[3] = 1; /* ipv4 addr */ memset(buf+4,0,6); /* Set external addr/port to 0. The spec doesn't seem to say what to do here. -RD */ - connection_write_to_buf(buf,10,TO_CONN(conn)); + connection_write_to_buf(buf,10,ENTRY_TO_CONN(conn)); } /* If socks_version isn't 4 or 5, don't send anything. * This can happen in the case of AP bridges. */ @@ -2696,7 +3018,7 @@ connection_exit_begin_conn(cell_t *cell, circuit_t *circ) char *address=NULL; uint16_t port; or_circuit_t *or_circ = NULL; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); assert_circuit_ok(circ); if (!CIRCUIT_IS_ORIGIN(circ)) @@ -2728,9 +3050,9 @@ connection_exit_begin_conn(cell_t *cell, circuit_t *circ) END_STREAM_REASON_TORPROTOCOL, NULL); return 0; } - if (parse_addr_port(LOG_PROTOCOL_WARN, - (char*)(cell->payload+RELAY_HEADER_SIZE), - &address,NULL,&port)<0) { + if (tor_addr_port_split(LOG_PROTOCOL_WARN, + (char*)(cell->payload+RELAY_HEADER_SIZE), + &address,&port)<0) { log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, "Unable to parse addr:port in relay begin cell. Closing."); relay_send_end_cell_from_edge(rh.stream_id, circ, @@ -2979,12 +3301,14 @@ connection_exit_connect(edge_connection_t *edge_conn) } conn->state = EXIT_CONN_STATE_OPEN; - if (connection_wants_to_flush(conn)) { - /* in case there are any queued data cells */ - log_warn(LD_BUG,"newly connected conn had data waiting!"); -// connection_start_writing(conn); + if (connection_get_outbuf_len(conn)) { + /* in case there are any queued data cells, from e.g. optimistic data */ + IF_HAS_NO_BUFFEREVENT(conn) + connection_watch_events(conn, READ_EVENT|WRITE_EVENT); + } else { + IF_HAS_NO_BUFFEREVENT(conn) + connection_watch_events(conn, READ_EVENT); } - connection_watch_events(conn, READ_EVENT); /* also, deliver a 'connected' cell back through the circuit. */ if (connection_edge_is_rendezvous_stream(edge_conn)) { @@ -3029,7 +3353,7 @@ connection_exit_connect_dir(edge_connection_t *exitconn) exitconn->_base.state = EXIT_CONN_STATE_OPEN; - dirconn = dir_connection_new(AF_INET); + dirconn = dir_connection_new(tor_addr_family(&exitconn->_base.addr)); tor_addr_copy(&dirconn->_base.addr, &exitconn->_base.addr); dirconn->_base.port = 0; @@ -3094,12 +3418,11 @@ connection_edge_is_rendezvous_stream(edge_connection_t *conn) * resolved.) */ int -connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit) +connection_ap_can_use_exit(const entry_connection_t *conn, const node_t *exit) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); tor_assert(conn); - tor_assert(conn->_base.type == CONN_TYPE_AP); tor_assert(conn->socks_request); tor_assert(exit); @@ -3107,10 +3430,10 @@ connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit) * make sure the exit node of the existing circuit matches exactly. */ if (conn->chosen_exit_name) { - routerinfo_t *chosen_exit = - router_get_by_nickname(conn->chosen_exit_name, 1); - if (!chosen_exit || tor_memneq(chosen_exit->cache_info.identity_digest, - exit->cache_info.identity_digest, DIGEST_LEN)) { + const node_t *chosen_exit = + node_get_by_nickname(conn->chosen_exit_name, 1); + if (!chosen_exit || tor_memneq(chosen_exit->identity, + exit->identity, DIGEST_LEN)) { /* doesn't match */ // log_debug(LD_APP,"Requested node '%s', considering node '%s'. No.", // conn->chosen_exit_name, exit->nickname); @@ -3118,15 +3441,20 @@ connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit) } } - if (conn->socks_request->command == SOCKS_COMMAND_CONNECT && - !conn->use_begindir) { + if (conn->use_begindir) { + /* Internal directory fetches do not count as exiting. */ + return 1; + } + + if (conn->socks_request->command == SOCKS_COMMAND_CONNECT) { struct in_addr in; - uint32_t addr = 0; + tor_addr_t addr, *addrp = NULL; addr_policy_result_t r; - if (tor_inet_aton(conn->socks_request->address, &in)) - addr = ntohl(in.s_addr); - r = compare_addr_to_addr_policy(addr, conn->socks_request->port, - exit->exit_policy); + if (tor_inet_aton(conn->socks_request->address, &in)) { + tor_addr_from_in(&addr, &in); + addrp = &addr; + } + r = compare_tor_addr_to_node_policy(addrp, conn->socks_request->port,exit); if (r == ADDR_POLICY_REJECTED) return 0; /* We know the address, and the exit policy rejects it. */ if (r == ADDR_POLICY_PROBABLY_REJECTED && !conn->chosen_exit_name) @@ -3134,17 +3462,11 @@ connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit) * addresses with this port. Since the user didn't ask for * this node, err on the side of caution. */ } else if (SOCKS_COMMAND_IS_RESOLVE(conn->socks_request->command)) { - /* Can't support reverse lookups without eventdns. */ - if (conn->socks_request->command == SOCKS_COMMAND_RESOLVE_PTR && - exit->has_old_dnsworkers) - return 0; - /* Don't send DNS requests to non-exit servers by default. */ - if (!conn->chosen_exit_name && policy_is_reject_star(exit->exit_policy)) + if (!conn->chosen_exit_name && node_exit_policy_rejects_all(exit)) return 0; } - if (options->_ExcludeExitNodesUnion && - routerset_contains_router(options->_ExcludeExitNodesUnion, exit)) { + if (routerset_contains_node(options->_ExcludeExitNodesUnion, exit)) { /* Not a suitable exit. Refuse it. */ return 0; } @@ -3155,14 +3477,17 @@ connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit) /** If address is of the form "y.onion" with a well-formed handle y: * Put a NUL after y, lower-case it, and return ONION_HOSTNAME. * - * If address is of the form "y.exit" and <b>allowdotexit</b> is true: + * If address is of the form "y.onion" with a badly-formed handle y: + * Return BAD_HOSTNAME and log a message. + * + * If address is of the form "y.exit": * Put a NUL after y and return EXIT_HOSTNAME. * * Otherwise: * Return NORMAL_HOSTNAME and change nothing. */ hostname_type_t -parse_extended_hostname(char *address, int allowdotexit) +parse_extended_hostname(char *address) { char *s; char query[REND_SERVICE_ID_LEN_BASE32+1]; @@ -3171,16 +3496,8 @@ parse_extended_hostname(char *address, int allowdotexit) if (!s) return NORMAL_HOSTNAME; /* no dot, thus normal */ if (!strcmp(s+1,"exit")) { - if (allowdotexit) { - *s = 0; /* NUL-terminate it */ - return EXIT_HOSTNAME; /* .exit */ - } else { - log_warn(LD_APP, "The \".exit\" notation is disabled in Tor due to " - "security risks. Set AllowDotExit in your torrc to enable " - "it."); - /* FFFF send a controller event too to notify Vidalia users */ - return BAD_HOSTNAME; - } + *s = 0; /* NUL-terminate it */ + return EXIT_HOSTNAME; /* .exit */ } if (strcmp(s+1,"onion")) return NORMAL_HOSTNAME; /* neither .exit nor .onion, thus normal */ @@ -3196,6 +3513,210 @@ parse_extended_hostname(char *address, int allowdotexit) failed: /* otherwise, return to previous state and return 0 */ *s = '.'; + log_warn(LD_APP, "Invalid onion hostname %s; rejecting", + safe_str_client(address)); return BAD_HOSTNAME; } +/** Return true iff the (possibly NULL) <b>alen</b>-byte chunk of memory at + * <b>a</b> is equal to the (possibly NULL) <b>blen</b>-byte chunk of memory + * at <b>b</b>. */ +static int +memeq_opt(const char *a, size_t alen, const char *b, size_t blen) +{ + if (a == NULL) { + return (b == NULL); + } else if (b == NULL) { + return 0; + } else if (alen != blen) { + return 0; + } else { + return tor_memeq(a, b, alen); + } +} + +/** + * Return true iff none of the isolation flags and fields in <b>conn</b> + * should prevent it from being attached to <b>circ</b>. + */ +int +connection_edge_compatible_with_circuit(const entry_connection_t *conn, + const origin_circuit_t *circ) +{ + const uint8_t iso = conn->isolation_flags; + const socks_request_t *sr = conn->socks_request; + + /* If circ has never been used for an isolated connection, we can + * totally use it for this one. */ + if (!circ->isolation_values_set) + return 1; + + /* If circ has been used for connections having more than one value + * for some field f, it will have the corresponding bit set in + * isolation_flags_mixed. If isolation_flags_mixed has any bits + * in common with iso, then conn must be isolated from at least + * one stream that has been attached to circ. */ + if ((iso & circ->isolation_flags_mixed) != 0) { + /* For at least one field where conn is isolated, the circuit + * already has mixed streams. */ + return 0; + } + + if (! conn->original_dest_address) { + log_warn(LD_BUG, "Reached connection_edge_compatible_with_circuit without " + "having set conn->original_dest_address"); + ((entry_connection_t*)conn)->original_dest_address = + tor_strdup(conn->socks_request->address); + } + + if ((iso & ISO_STREAM) && + (circ->associated_isolated_stream_global_id != + ENTRY_TO_CONN(conn)->global_identifier)) + return 0; + + if ((iso & ISO_DESTPORT) && conn->socks_request->port != circ->dest_port) + return 0; + if ((iso & ISO_DESTADDR) && + strcasecmp(conn->original_dest_address, circ->dest_address)) + return 0; + if ((iso & ISO_SOCKSAUTH) && + (! memeq_opt(sr->username, sr->usernamelen, + circ->socks_username, circ->socks_username_len) || + ! memeq_opt(sr->password, sr->passwordlen, + circ->socks_password, circ->socks_password_len))) + return 0; + if ((iso & ISO_CLIENTPROTO) && + (conn->socks_request->listener_type != circ->client_proto_type || + conn->socks_request->socks_version != circ->client_proto_socksver)) + return 0; + if ((iso & ISO_CLIENTADDR) && + !tor_addr_eq(&ENTRY_TO_CONN(conn)->addr, &circ->client_addr)) + return 0; + if ((iso & ISO_SESSIONGRP) && conn->session_group != circ->session_group) + return 0; + if ((iso & ISO_NYM_EPOCH) && conn->nym_epoch != circ->nym_epoch) + return 0; + + return 1; +} + +/** + * If <b>dry_run</b> is false, update <b>circ</b>'s isolation flags and fields + * to reflect having had <b>conn</b> attached to it, and return 0. Otherwise, + * if <b>dry_run</b> is true, then make no changes to <b>circ</b>, and return + * a bitfield of isolation flags that we would have to set in + * isolation_flags_mixed to add <b>conn</b> to <b>circ</b>, or -1 if + * <b>circ</b> has had no streams attached to it. + */ +int +connection_edge_update_circuit_isolation(const entry_connection_t *conn, + origin_circuit_t *circ, + int dry_run) +{ + const socks_request_t *sr = conn->socks_request; + if (! conn->original_dest_address) { + log_warn(LD_BUG, "Reached connection_update_circuit_isolation without " + "having set conn->original_dest_address"); + ((entry_connection_t*)conn)->original_dest_address = + tor_strdup(conn->socks_request->address); + } + + if (!circ->isolation_values_set) { + if (dry_run) + return -1; + circ->associated_isolated_stream_global_id = + ENTRY_TO_CONN(conn)->global_identifier; + circ->dest_port = conn->socks_request->port; + circ->dest_address = tor_strdup(conn->original_dest_address); + circ->client_proto_type = conn->socks_request->listener_type; + circ->client_proto_socksver = conn->socks_request->socks_version; + tor_addr_copy(&circ->client_addr, &ENTRY_TO_CONN(conn)->addr); + circ->session_group = conn->session_group; + circ->nym_epoch = conn->nym_epoch; + circ->socks_username = sr->username ? + tor_memdup(sr->username, sr->usernamelen) : NULL; + circ->socks_password = sr->password ? + tor_memdup(sr->password, sr->passwordlen) : NULL; + circ->socks_username_len = sr->usernamelen; + circ->socks_password_len = sr->passwordlen; + + circ->isolation_values_set = 1; + return 0; + } else { + uint8_t mixed = 0; + if (conn->socks_request->port != circ->dest_port) + mixed |= ISO_DESTPORT; + if (strcasecmp(conn->original_dest_address, circ->dest_address)) + mixed |= ISO_DESTADDR; + if (!memeq_opt(sr->username, sr->usernamelen, + circ->socks_username, circ->socks_username_len) || + !memeq_opt(sr->password, sr->passwordlen, + circ->socks_password, circ->socks_password_len)) + mixed |= ISO_SOCKSAUTH; + if ((conn->socks_request->listener_type != circ->client_proto_type || + conn->socks_request->socks_version != circ->client_proto_socksver)) + mixed |= ISO_CLIENTPROTO; + if (!tor_addr_eq(&ENTRY_TO_CONN(conn)->addr, &circ->client_addr)) + mixed |= ISO_CLIENTADDR; + if (conn->session_group != circ->session_group) + mixed |= ISO_SESSIONGRP; + if (conn->nym_epoch != circ->nym_epoch) + mixed |= ISO_NYM_EPOCH; + + if (dry_run) + return mixed; + + if ((mixed & conn->isolation_flags) != 0) { + log_warn(LD_BUG, "Updating a circuit with seemingly incompatible " + "isolation flags."); + } + circ->isolation_flags_mixed |= mixed; + return 0; + } +} + +/** + * Clear the isolation settings on <b>circ</b>. + * + * This only works on an open circuit that has never had a stream attached to + * it, and whose isolation settings are hypothetical. (We set hypothetical + * isolation settings on circuits as we're launching them, so that we + * know whether they can handle more streams or whether we need to launch + * even more circuits. Once the circuit is open, if it turns out that + * we no longer have any streams to attach to it, we clear the isolation flags + * and data so that other streams can have a chance.) + */ +void +circuit_clear_isolation(origin_circuit_t *circ) +{ + if (circ->isolation_any_streams_attached) { + log_warn(LD_BUG, "Tried to clear the isolation status of a dirty circuit"); + return; + } + if (TO_CIRCUIT(circ)->state != CIRCUIT_STATE_OPEN) { + log_warn(LD_BUG, "Tried to clear the isolation status of a non-open " + "circuit"); + return; + } + + circ->isolation_values_set = 0; + circ->isolation_flags_mixed = 0; + circ->associated_isolated_stream_global_id = 0; + circ->client_proto_type = 0; + circ->client_proto_socksver = 0; + circ->dest_port = 0; + tor_addr_make_unspec(&circ->client_addr); + tor_free(circ->dest_address); + circ->session_group = -1; + circ->nym_epoch = 0; + if (circ->socks_username) { + memwipe(circ->socks_username, 0x11, circ->socks_username_len); + tor_free(circ->socks_username); + } + if (circ->socks_password) { + memwipe(circ->socks_password, 0x05, circ->socks_password_len); + tor_free(circ->socks_password); + } + circ->socks_username_len = circ->socks_password_len = 0; +} + diff --git a/src/or/connection_edge.h b/src/or/connection_edge.h index 8ba2fafd08..c320d6ba49 100644 --- a/src/or/connection_edge.h +++ b/src/or/connection_edge.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -15,7 +15,7 @@ #define connection_mark_unattached_ap(conn, endreason) \ _connection_mark_unattached_ap((conn), (endreason), __LINE__, _SHORT_FILE_) -void _connection_mark_unattached_ap(edge_connection_t *conn, int endreason, +void _connection_mark_unattached_ap(entry_connection_t *conn, int endreason, int line, const char *file); int connection_edge_reached_eof(edge_connection_t *conn); int connection_edge_process_inbuf(edge_connection_t *conn, @@ -27,16 +27,22 @@ int connection_edge_flushed_some(edge_connection_t *conn); int connection_edge_finished_flushing(edge_connection_t *conn); int connection_edge_finished_connecting(edge_connection_t *conn); -int connection_ap_handshake_send_begin(edge_connection_t *ap_conn); -int connection_ap_handshake_send_resolve(edge_connection_t *ap_conn); +void connection_ap_about_to_close(entry_connection_t *edge_conn); +void connection_exit_about_to_close(edge_connection_t *edge_conn); -edge_connection_t *connection_ap_make_link(char *address, uint16_t port, +int connection_ap_handshake_send_begin(entry_connection_t *ap_conn); +int connection_ap_handshake_send_resolve(entry_connection_t *ap_conn); + +entry_connection_t *connection_ap_make_link(connection_t *partner, + char *address, uint16_t port, const char *digest, + int session_group, + int isolation_flags, int use_begindir, int want_onehop); -void connection_ap_handshake_socks_reply(edge_connection_t *conn, char *reply, +void connection_ap_handshake_socks_reply(entry_connection_t *conn, char *reply, size_t replylen, int endreason); -void connection_ap_handshake_socks_resolved(edge_connection_t *conn, +void connection_ap_handshake_socks_resolved(entry_connection_t *conn, int answer_type, size_t answer_len, const uint8_t *answer, @@ -47,31 +53,35 @@ int connection_exit_begin_conn(cell_t *cell, circuit_t *circ); int connection_exit_begin_resolve(cell_t *cell, or_circuit_t *circ); void connection_exit_connect(edge_connection_t *conn); int connection_edge_is_rendezvous_stream(edge_connection_t *conn); -int connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit); +int connection_ap_can_use_exit(const entry_connection_t *conn, + const node_t *exit); void connection_ap_expire_beginning(void); void connection_ap_attach_pending(void); void connection_ap_fail_onehop(const char *failed_digest, cpath_build_state_t *build_state); void circuit_discard_optional_exit_enclaves(extend_info_t *info); -int connection_ap_detach_retriable(edge_connection_t *conn, +int connection_ap_detach_retriable(entry_connection_t *conn, origin_circuit_t *circ, int reason); -int connection_ap_process_transparent(edge_connection_t *conn); +int connection_ap_process_transparent(entry_connection_t *conn); int address_is_invalid_destination(const char *address, int client); void addressmap_init(void); -void addressmap_clear_excluded_trackexithosts(or_options_t *options); -void addressmap_clear_invalid_automaps(or_options_t *options); +void addressmap_clear_excluded_trackexithosts(const or_options_t *options); +void addressmap_clear_invalid_automaps(const or_options_t *options); void addressmap_clean(time_t now); void addressmap_clear_configured(void); void addressmap_clear_transient(void); void addressmap_free_all(void); -int addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out); +int addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out, + addressmap_entry_source_t *exit_source_out); int addressmap_have_mapping(const char *address, int update_timeout); void addressmap_register(const char *address, char *new_address, - time_t expires, addressmap_entry_source_t source); + time_t expires, addressmap_entry_source_t source, + const int address_wildcard, + const int new_address_wildcard); int parse_virtual_addr_network(const char *val, int validate_only, char **msg); int client_dns_incr_failures(const char *address); @@ -81,10 +91,10 @@ void client_dns_set_addressmap(const char *address, uint32_t val, const char *addressmap_register_virtual_address(int type, char *new_address); void addressmap_get_mappings(smartlist_t *sl, time_t min_expires, time_t max_expires, int want_expiry); -int connection_ap_rewrite_and_attach_if_allowed(edge_connection_t *conn, +int connection_ap_rewrite_and_attach_if_allowed(entry_connection_t *conn, origin_circuit_t *circ, crypt_path_t *cpath); -int connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, +int connection_ap_handshake_rewrite_and_attach(entry_connection_t *conn, origin_circuit_t *circ, crypt_path_t *cpath); @@ -92,11 +102,18 @@ int connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, typedef enum hostname_type_t { NORMAL_HOSTNAME, ONION_HOSTNAME, EXIT_HOSTNAME, BAD_HOSTNAME } hostname_type_t; -hostname_type_t parse_extended_hostname(char *address, int allowdotexit); +hostname_type_t parse_extended_hostname(char *address); #if defined(HAVE_NET_IF_H) && defined(HAVE_NET_PFVAR_H) int get_pf_socket(void); #endif +int connection_edge_compatible_with_circuit(const entry_connection_t *conn, + const origin_circuit_t *circ); +int connection_edge_update_circuit_isolation(const entry_connection_t *conn, + origin_circuit_t *circ, + int dry_run); +void circuit_clear_isolation(origin_circuit_t *circ); + #endif diff --git a/src/or/connection_or.c b/src/or/connection_or.c index cb0082bdc2..5eecee0740 100644 --- a/src/or/connection_or.c +++ b/src/or/connection_or.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -13,6 +13,7 @@ #include "or.h" #include "buffers.h" #include "circuitbuild.h" +#include "circuitlist.h" #include "command.h" #include "config.h" #include "connection.h" @@ -22,21 +23,32 @@ #include "geoip.h" #include "main.h" #include "networkstatus.h" +#include "nodelist.h" #include "reasons.h" #include "relay.h" #include "rephist.h" #include "router.h" #include "routerlist.h" +#ifdef USE_BUFFEREVENTS +#include <event2/bufferevent_ssl.h> +#endif + static int connection_tls_finish_handshake(or_connection_t *conn); +static int connection_or_launch_v3_or_handshake(or_connection_t *conn); static int connection_or_process_cells_from_inbuf(or_connection_t *conn); -static int connection_or_send_versions(or_connection_t *conn); -static int connection_init_or_handshake_state(or_connection_t *conn, - int started_here); static int connection_or_check_valid_tls_handshake(or_connection_t *conn, int started_here, char *digest_rcvd_out); +static void connection_or_tls_renegotiated_cb(tor_tls_t *tls, void *_conn); + +#ifdef USE_BUFFEREVENTS +static void connection_or_handle_event_cb(struct bufferevent *bufev, + short event, void *arg); +#include <event2/buffer.h>/*XXXX REMOVE */ +#endif + /**************************************************************/ /** Map from identity digest of connected OR or desired OR to a connection_t @@ -136,6 +148,142 @@ connection_or_set_identity_digest(or_connection_t *conn, const char *digest) #endif } +/**************************************************************/ + +/** Map from a string describing what a non-open OR connection was doing when + * failed, to an intptr_t describing the count of connections that failed that + * way. Note that the count is stored _as_ the pointer. + */ +static strmap_t *broken_connection_counts; + +/** If true, do not record information in <b>broken_connection_counts</b>. */ +static int disable_broken_connection_counts = 0; + +/** Record that an OR connection failed in <b>state</b>. */ +static void +note_broken_connection(const char *state) +{ + void *ptr; + intptr_t val; + if (disable_broken_connection_counts) + return; + + if (!broken_connection_counts) + broken_connection_counts = strmap_new(); + + ptr = strmap_get(broken_connection_counts, state); + val = (intptr_t)ptr; + val++; + ptr = (void*)val; + strmap_set(broken_connection_counts, state, ptr); +} + +/** Forget all recorded states for failed connections. If + * <b>stop_recording</b> is true, don't record any more. */ +void +clear_broken_connection_map(int stop_recording) +{ + if (broken_connection_counts) + strmap_free(broken_connection_counts, NULL); + broken_connection_counts = NULL; + if (stop_recording) + disable_broken_connection_counts = 1; +} + +/** Write a detailed description the state of <b>orconn</b> into the + * <b>buflen</b>-byte buffer at <b>buf</b>. This description includes not + * only the OR-conn level state but also the TLS state. It's useful for + * diagnosing broken handshakes. */ +static void +connection_or_get_state_description(or_connection_t *orconn, + char *buf, size_t buflen) +{ + connection_t *conn = TO_CONN(orconn); + const char *conn_state; + char tls_state[256]; + + tor_assert(conn->type == CONN_TYPE_OR); + + conn_state = conn_state_to_string(conn->type, conn->state); + tor_tls_get_state_description(orconn->tls, tls_state, sizeof(tls_state)); + + tor_snprintf(buf, buflen, "%s with SSL state %s", conn_state, tls_state); +} + +/** Record the current state of <b>orconn</b> as the state of a broken + * connection. */ +static void +connection_or_note_state_when_broken(or_connection_t *orconn) +{ + char buf[256]; + if (disable_broken_connection_counts) + return; + connection_or_get_state_description(orconn, buf, sizeof(buf)); + log_info(LD_HANDSHAKE,"Connection died in state '%s'", buf); + note_broken_connection(buf); +} + +/** Helper type used to sort connection states and find the most frequent. */ +typedef struct broken_state_count_t { + intptr_t count; + const char *state; +} broken_state_count_t; + +/** Helper function used to sort broken_state_count_t by frequency. */ +static int +broken_state_count_compare(const void **a_ptr, const void **b_ptr) +{ + const broken_state_count_t *a = *a_ptr, *b = *b_ptr; + if (b->count < a->count) + return -1; + else if (b->count == a->count) + return 0; + else + return 1; +} + +/** Upper limit on the number of different states to report for connection + * failure. */ +#define MAX_REASONS_TO_REPORT 10 + +/** Report a list of the top states for failed OR connections at log level + * <b>severity</b>, in log domain <b>domain</b>. */ +void +connection_or_report_broken_states(int severity, int domain) +{ + int total = 0; + smartlist_t *items; + + if (!broken_connection_counts || disable_broken_connection_counts) + return; + + items = smartlist_new(); + STRMAP_FOREACH(broken_connection_counts, state, void *, countptr) { + broken_state_count_t *c = tor_malloc(sizeof(broken_state_count_t)); + c->count = (intptr_t)countptr; + total += (int)c->count; + c->state = state; + smartlist_add(items, c); + } STRMAP_FOREACH_END; + + smartlist_sort(items, broken_state_count_compare); + + log(severity, domain, "%d connections have failed%s", total, + smartlist_len(items) > MAX_REASONS_TO_REPORT ? ". Top reasons:" : ":"); + + SMARTLIST_FOREACH_BEGIN(items, const broken_state_count_t *, c) { + if (c_sl_idx > MAX_REASONS_TO_REPORT) + break; + log(severity, domain, + " %d connections died in state %s", (int)c->count, c->state); + } SMARTLIST_FOREACH_END(c); + + SMARTLIST_FOREACH(items, broken_state_count_t *, c, tor_free(c)); + smartlist_free(items); +} + +/**************************************************************/ + /** Pack the cell_t host-order structure <b>src</b> into network-order * in the buffer <b>dest</b>. See tor-spec.txt for details about the * wire format. @@ -178,7 +326,8 @@ var_cell_pack_header(const var_cell_t *cell, char *hdr_out) var_cell_t * var_cell_new(uint16_t payload_len) { - var_cell_t *cell = tor_malloc(sizeof(var_cell_t)+payload_len-1); + size_t size = STRUCT_OFFSET(var_cell_t, payload) + payload_len; + var_cell_t *cell = tor_malloc(size); cell->payload_len = payload_len; cell->command = 0; cell->circ_id = 0; @@ -232,20 +381,35 @@ connection_or_process_inbuf(or_connection_t *conn) } return ret; + case OR_CONN_STATE_TLS_SERVER_RENEGOTIATING: +#ifdef USE_BUFFEREVENTS + if (tor_tls_server_got_renegotiate(conn->tls)) + connection_or_tls_renegotiated_cb(conn->tls, conn); + if (conn->_base.marked_for_close) + return 0; + /* fall through. */ +#endif case OR_CONN_STATE_OPEN: - case OR_CONN_STATE_OR_HANDSHAKING: + case OR_CONN_STATE_OR_HANDSHAKING_V2: + case OR_CONN_STATE_OR_HANDSHAKING_V3: return connection_or_process_cells_from_inbuf(conn); default: break; /* don't do anything */ } + /* This check was necessary with 0.2.2, when the TLS_SERVER_RENEGOTIATING + * check would otherwise just let data accumulate. It serves no purpose + * in 0.2.3. + * + * XXX024 Remove this check once we verify that the above paragraph is + * 100% true. */ if (buf_datalen(conn->_base.inbuf) > MAX_OR_INBUF_WHEN_NONOPEN) { log_fn(LOG_PROTOCOL_WARN, LD_NET, "Accumulated too much data (%d bytes) " - "on nonopen OR connection %s %s:%u in state %s; closing.", - (int)buf_datalen(conn->_base.inbuf), - connection_or_nonopen_was_started_here(conn) ? "to" : "from", - conn->_base.address, conn->_base.port, - conn_state_to_string(conn->_base.type, conn->_base.state)); + "on nonopen OR connection %s %s:%u in state %s; closing.", + (int)buf_datalen(conn->_base.inbuf), + connection_or_nonopen_was_started_here(conn) ? "to" : "from", + conn->_base.address, conn->_base.port, + conn_state_to_string(conn->_base.type, conn->_base.state)); connection_mark_for_close(TO_CONN(conn)); ret = -1; } @@ -266,7 +430,7 @@ connection_or_process_inbuf(or_connection_t *conn) int connection_or_flushed_some(or_connection_t *conn) { - size_t datalen = buf_datalen(conn->_base.outbuf); + size_t datalen = connection_get_outbuf_len(TO_CONN(conn)); /* If we're under the low water mark, add cells until we're just over the * high water mark. */ if (datalen < OR_CONN_LOWWATER) { @@ -298,8 +462,8 @@ connection_or_finished_flushing(or_connection_t *conn) switch (conn->_base.state) { case OR_CONN_STATE_PROXY_HANDSHAKING: case OR_CONN_STATE_OPEN: - case OR_CONN_STATE_OR_HANDSHAKING: - connection_stop_writing(TO_CONN(conn)); + case OR_CONN_STATE_OR_HANDSHAKING_V2: + case OR_CONN_STATE_OR_HANDSHAKING_V3: break; default: log_err(LD_BUG,"Called in unexpected state %d.", conn->_base.state); @@ -314,7 +478,7 @@ connection_or_finished_flushing(or_connection_t *conn) int connection_or_finished_connecting(or_connection_t *or_conn) { - int proxy_type; + const int proxy_type = or_conn->proxy_type; connection_t *conn; tor_assert(or_conn); conn = TO_CONN(or_conn); @@ -324,15 +488,6 @@ connection_or_finished_connecting(or_connection_t *or_conn) conn->address,conn->port); control_event_bootstrap(BOOTSTRAP_STATUS_HANDSHAKE, 0); - proxy_type = PROXY_NONE; - - if (get_options()->HTTPSProxy) - proxy_type = PROXY_CONNECT; - else if (get_options()->Socks4Proxy) - proxy_type = PROXY_SOCKS4; - else if (get_options()->Socks5Proxy) - proxy_type = PROXY_SOCKS5; - if (proxy_type != PROXY_NONE) { /* start proxy handshake */ if (connection_proxy_connect(conn, proxy_type) < 0) { @@ -353,6 +508,51 @@ connection_or_finished_connecting(or_connection_t *or_conn) return 0; } +/** Called when we're about to finally unlink and free an OR connection: + * perform necessary accounting and cleanup */ +void +connection_or_about_to_close(or_connection_t *or_conn) +{ + time_t now = time(NULL); + connection_t *conn = TO_CONN(or_conn); + + /* Remember why we're closing this connection. */ + if (conn->state != OR_CONN_STATE_OPEN) { + /* Inform any pending (not attached) circs that they should + * give up. */ + circuit_n_conn_done(TO_OR_CONN(conn), 0); + /* now mark things down as needed */ + if (connection_or_nonopen_was_started_here(or_conn)) { + const or_options_t *options = get_options(); + connection_or_note_state_when_broken(or_conn); + rep_hist_note_connect_failed(or_conn->identity_digest, now); + entry_guard_register_connect_status(or_conn->identity_digest,0, + !options->HTTPSProxy, now); + if (conn->state >= OR_CONN_STATE_TLS_HANDSHAKING) { + int reason = tls_error_to_orconn_end_reason(or_conn->tls_error); + control_event_or_conn_status(or_conn, OR_CONN_EVENT_FAILED, + reason); + if (!authdir_mode_tests_reachability(options)) + control_event_bootstrap_problem( + orconn_end_reason_to_control_string(reason), reason); + } + } + } else if (conn->hold_open_until_flushed) { + /* We only set hold_open_until_flushed when we're intentionally + * closing a connection. */ + rep_hist_note_disconnect(or_conn->identity_digest, now); + control_event_or_conn_status(or_conn, OR_CONN_EVENT_CLOSED, + tls_error_to_orconn_end_reason(or_conn->tls_error)); + } else if (!tor_digest_is_zero(or_conn->identity_digest)) { + rep_hist_note_connection_died(or_conn->identity_digest, now); + control_event_or_conn_status(or_conn, OR_CONN_EVENT_CLOSED, + tls_error_to_orconn_end_reason(or_conn->tls_error)); + } + /* Now close all the attached circuits on it. */ + circuit_unlink_all_from_or_conn(TO_OR_CONN(conn), + END_CIRC_REASON_OR_CONN_CLOSED); +} + /** Return 1 if identity digest <b>id_digest</b> is known to be a * currently or recently running relay. Otherwise return 0. */ int @@ -360,7 +560,7 @@ connection_or_digest_is_known_relay(const char *id_digest) { if (router_get_consensus_status_by_id(id_digest)) return 1; /* It's in the consensus: "yes" */ - if (router_get_by_digest(id_digest)) + if (router_get_by_id_digest(id_digest)) return 1; /* Not in the consensus, but we have a descriptor for * it. Probably it was in a recent consensus. "Yes". */ return 0; @@ -377,7 +577,7 @@ connection_or_digest_is_known_relay(const char *id_digest) */ static void connection_or_update_token_buckets_helper(or_connection_t *conn, int reset, - or_options_t *options) + const or_options_t *options) { int rate, burst; /* per-connection rate limiting params */ if (connection_or_digest_is_known_relay(conn->identity_digest)) { @@ -400,6 +600,27 @@ connection_or_update_token_buckets_helper(or_connection_t *conn, int reset, conn->bandwidthrate = rate; conn->bandwidthburst = burst; +#ifdef USE_BUFFEREVENTS + { + const struct timeval *tick = tor_libevent_get_one_tick_timeout(); + struct ev_token_bucket_cfg *cfg, *old_cfg; + int64_t rate64 = (((int64_t)rate) * options->TokenBucketRefillInterval) + / 1000; + /* This can't overflow, since TokenBucketRefillInterval <= 1000, + * and rate started out less than INT_MAX. */ + int rate_per_tick = (int) rate64; + + cfg = ev_token_bucket_cfg_new(rate_per_tick, burst, rate_per_tick, + burst, tick); + old_cfg = conn->bucket_cfg; + if (conn->_base.bufev) + tor_set_bufferevent_rate_limit(conn->_base.bufev, cfg); + if (old_cfg) + ev_token_bucket_cfg_free(old_cfg); + conn->bucket_cfg = cfg; + (void) reset; /* No way to do this with libevent yet. */ + } +#else if (reset) { /* set up the token buckets to be full */ conn->read_bucket = conn->write_bucket = burst; return; @@ -410,13 +631,15 @@ connection_or_update_token_buckets_helper(or_connection_t *conn, int reset, conn->read_bucket = burst; if (conn->write_bucket > burst) conn->write_bucket = burst; +#endif } /** Either our set of relays or our per-conn rate limits have changed. * Go through all the OR connections and update their token buckets to make * sure they don't exceed their maximum values. */ void -connection_or_update_token_buckets(smartlist_t *conns, or_options_t *options) +connection_or_update_token_buckets(smartlist_t *conns, + const or_options_t *options) { SMARTLIST_FOREACH(conns, connection_t *, conn, { @@ -427,14 +650,16 @@ connection_or_update_token_buckets(smartlist_t *conns, or_options_t *options) /** If we don't necessarily know the router we're connecting to, but we * have an addr/port/id_digest, then fill in as much as we can. Start - * by checking to see if this describes a router we know. */ -static void + * by checking to see if this describes a router we know. + * <b>started_here</b> is 1 if we are the initiator of <b>conn</b> and + * 0 if it's an incoming connection. */ +void connection_or_init_conn_from_address(or_connection_t *conn, const tor_addr_t *addr, uint16_t port, const char *id_digest, int started_here) { - routerinfo_t *r = router_get_by_digest(id_digest); + const node_t *r = node_get_by_id(id_digest); connection_or_set_identity_digest(conn, id_digest); connection_or_update_token_buckets_helper(conn, 1, get_options()); @@ -442,8 +667,11 @@ connection_or_init_conn_from_address(or_connection_t *conn, tor_addr_copy(&conn->_base.addr, addr); tor_addr_copy(&conn->real_addr, addr); if (r) { - /* XXXX proposal 118 will make this more complex. */ - if (tor_addr_eq_ipv4h(&conn->_base.addr, r->addr)) + tor_addr_port_t node_ap; + node_get_pref_orport(r, &node_ap); + /* XXXX proposal 186 is making this more complex. For now, a conn + is canonical when it uses the _preferred_ address. */ + if (tor_addr_eq(&conn->_base.addr, &node_ap.addr)) conn->is_canonical = 1; if (!started_here) { /* Override the addr/port, so our log messages will make sense. @@ -456,12 +684,12 @@ connection_or_init_conn_from_address(or_connection_t *conn, * right IP address and port 56244, that wouldn't be as helpful. now we * log the "right" port too, so we know if it's moria1 or moria2. */ - tor_addr_from_ipv4h(&conn->_base.addr, r->addr); - conn->_base.port = r->or_port; + tor_addr_copy(&conn->_base.addr, &node_ap.addr); + conn->_base.port = node_ap.port; } - conn->nickname = tor_strdup(r->nickname); + conn->nickname = tor_strdup(node_get_nickname(r)); tor_free(conn->_base.address); - conn->_base.address = tor_strdup(r->address); + conn->_base.address = tor_dup_addr(&node_ap.addr); } else { const char *n; /* If we're an authoritative directory server, we may know a @@ -810,11 +1038,15 @@ connection_or_connect(const tor_addr_t *_addr, uint16_t port, const char *id_digest) { or_connection_t *conn; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int socket_error = 0; - int using_proxy = 0; tor_addr_t addr; + int r; + tor_addr_t proxy_addr; + uint16_t proxy_port; + int proxy_type; + tor_assert(_addr); tor_assert(id_digest); tor_addr_copy(&addr, _addr); @@ -824,7 +1056,7 @@ connection_or_connect(const tor_addr_t *_addr, uint16_t port, return NULL; } - conn = or_connection_new(AF_INET); + conn = or_connection_new(tor_addr_family(&addr)); /* set up conn so it's got all the data we need to remember */ connection_or_init_conn_from_address(conn, &addr, port, id_digest, 1); @@ -833,19 +1065,40 @@ connection_or_connect(const tor_addr_t *_addr, uint16_t port, conn->is_outgoing = 1; - /* use a proxy server if available */ - if (options->HTTPSProxy) { - using_proxy = 1; - tor_addr_copy(&addr, &options->HTTPSProxyAddr); - port = options->HTTPSProxyPort; - } else if (options->Socks4Proxy) { - using_proxy = 1; - tor_addr_copy(&addr, &options->Socks4ProxyAddr); - port = options->Socks4ProxyPort; - } else if (options->Socks5Proxy) { - using_proxy = 1; - tor_addr_copy(&addr, &options->Socks5ProxyAddr); - port = options->Socks5ProxyPort; + /* If we are using a proxy server, find it and use it. */ + r = get_proxy_addrport(&proxy_addr, &proxy_port, &proxy_type, TO_CONN(conn)); + if (r == 0) { + conn->proxy_type = proxy_type; + if (proxy_type != PROXY_NONE) { + tor_addr_copy(&addr, &proxy_addr); + port = proxy_port; + conn->_base.proxy_state = PROXY_INFANT; + } + } else { + /* get_proxy_addrport() might fail if we have a Bridge line that + references a transport, but no ClientTransportPlugin lines + defining its transport proxy. If this is the case, let's try to + output a useful log message to the user. */ + const char *transport_name = + find_transport_name_by_bridge_addrport(&TO_CONN(conn)->addr, + TO_CONN(conn)->port); + + if (transport_name) { + log_warn(LD_GENERAL, "We were supposed to connect to bridge '%s:%u' " + "using pluggable transport '%s', but we can't find a pluggable " + "transport proxy supporting '%s'. This can happen if you " + "haven't provided a ClientTransportPlugin line, or if " + "your pluggable transport proxy stopped running.", + fmt_addr(&TO_CONN(conn)->addr), TO_CONN(conn)->port, + transport_name, transport_name); + } else { + log_warn(LD_GENERAL, "Tried to connect to '%s:%u' through a proxy, but " + "the proxy address could not be found.", + fmt_addr(&TO_CONN(conn)->addr), TO_CONN(conn)->port); + } + + connection_free(TO_CONN(conn)); + return NULL; } switch (connection_connect(TO_CONN(conn), conn->_base.address, @@ -853,7 +1106,7 @@ connection_or_connect(const tor_addr_t *_addr, uint16_t port, case -1: /* If the connection failed immediately, and we're using * a proxy, our proxy is down. Don't blame the Tor server. */ - if (!using_proxy) + if (conn->_base.proxy_state == PROXY_INFANT) entry_guard_register_connect_status(conn->identity_digest, 0, 1, time(NULL)); connection_or_connect_failed(conn, @@ -888,6 +1141,7 @@ int connection_tls_start_handshake(or_connection_t *conn, int receiving) { conn->_base.state = OR_CONN_STATE_TLS_HANDSHAKING; + tor_assert(!conn->tls); conn->tls = tor_tls_new(conn->_base.s, receiving); if (!conn->tls) { log_warn(LD_BUG,"tor_tls_new failed. Closing."); @@ -895,16 +1149,54 @@ connection_tls_start_handshake(or_connection_t *conn, int receiving) } tor_tls_set_logged_address(conn->tls, // XXX client and relay? escaped_safe_str(conn->_base.address)); + +#ifdef USE_BUFFEREVENTS + if (connection_type_uses_bufferevent(TO_CONN(conn))) { + const int filtering = get_options()->_UseFilteringSSLBufferevents; + struct bufferevent *b = + tor_tls_init_bufferevent(conn->tls, conn->_base.bufev, conn->_base.s, + receiving, filtering); + if (!b) { + log_warn(LD_BUG,"tor_tls_init_bufferevent failed. Closing."); + return -1; + } + conn->_base.bufev = b; + if (conn->bucket_cfg) + tor_set_bufferevent_rate_limit(conn->_base.bufev, conn->bucket_cfg); + connection_enable_rate_limiting(TO_CONN(conn)); + + connection_configure_bufferevent_callbacks(TO_CONN(conn)); + bufferevent_setcb(b, + connection_handle_read_cb, + connection_handle_write_cb, + connection_or_handle_event_cb,/* overriding this one*/ + TO_CONN(conn)); + } +#endif connection_start_reading(TO_CONN(conn)); log_debug(LD_HANDSHAKE,"starting TLS handshake on fd %d", conn->_base.s); note_crypto_pk_op(receiving ? TLS_HANDSHAKE_S : TLS_HANDSHAKE_C); - if (connection_tls_continue_handshake(conn) < 0) { - return -1; + IF_HAS_BUFFEREVENT(TO_CONN(conn), { + /* ???? */; + }) ELSE_IF_NO_BUFFEREVENT { + if (connection_tls_continue_handshake(conn) < 0) + return -1; } return 0; } +/** Block all future attempts to renegotiate on 'conn' */ +void +connection_or_block_renegotiation(or_connection_t *conn) +{ + tor_tls_t *tls = conn->tls; + if (!tls) + return; + tor_tls_set_renegotiate_callback(tls, NULL, NULL); + tor_tls_block_renegotiation(tls); +} + /** Invoked on the server side from inside tor_tls_read() when the server * gets a successful TLS renegotiation from the client. */ static void @@ -914,8 +1206,7 @@ connection_or_tls_renegotiated_cb(tor_tls_t *tls, void *_conn) (void)tls; /* Don't invoke this again. */ - tor_tls_set_renegotiate_callback(tls, NULL, NULL); - tor_tls_block_renegotiation(tls); + connection_or_block_renegotiation(conn); if (connection_tls_finish_handshake(conn) < 0) { /* XXXX_TLS double-check that it's ok to do this from inside read. */ @@ -954,16 +1245,22 @@ connection_tls_continue_handshake(or_connection_t *conn) if (! tor_tls_used_v1_handshake(conn->tls)) { if (!tor_tls_is_server(conn->tls)) { if (conn->_base.state == OR_CONN_STATE_TLS_HANDSHAKING) { - log_debug(LD_OR, "Done with initial SSL handshake (client-side). " - "Requesting renegotiation."); - conn->_base.state = OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING; - goto again; + if (tor_tls_received_v3_certificate(conn->tls)) { + log_info(LD_OR, "Client got a v3 cert! Moving on to v3 " + "handshake."); + return connection_or_launch_v3_or_handshake(conn); + } else { + log_debug(LD_OR, "Done with initial SSL handshake (client-side)." + " Requesting renegotiation."); + conn->_base.state = OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING; + goto again; + } } // log_notice(LD_OR,"Done. state was %d.", conn->_base.state); } else { - /* improved handshake, but not a client. */ + /* v2/v3 handshake, but not a client. */ log_debug(LD_OR, "Done with initial SSL handshake (server-side). " - "Expecting renegotiation."); + "Expecting renegotiation or VERSIONS cell"); tor_tls_set_renegotiate_callback(conn->tls, connection_or_tls_renegotiated_cb, conn); @@ -988,6 +1285,87 @@ connection_tls_continue_handshake(or_connection_t *conn) return 0; } +#ifdef USE_BUFFEREVENTS +static void +connection_or_handle_event_cb(struct bufferevent *bufev, short event, + void *arg) +{ + struct or_connection_t *conn = TO_OR_CONN(arg); + + /* XXXX cut-and-paste code; should become a function. */ + if (event & BEV_EVENT_CONNECTED) { + if (conn->_base.state == OR_CONN_STATE_TLS_HANDSHAKING) { + if (tor_tls_finish_handshake(conn->tls) < 0) { + log_warn(LD_OR, "Problem finishing handshake"); + connection_mark_for_close(TO_CONN(conn)); + return; + } + } + + if (! tor_tls_used_v1_handshake(conn->tls)) { + if (!tor_tls_is_server(conn->tls)) { + if (conn->_base.state == OR_CONN_STATE_TLS_HANDSHAKING) { + if (tor_tls_received_v3_certificate(conn->tls)) { + log_info(LD_OR, "Client got a v3 cert!"); + if (connection_or_launch_v3_or_handshake(conn) < 0) + connection_mark_for_close(TO_CONN(conn)); + return; + } else { + conn->_base.state = OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING; + tor_tls_unblock_renegotiation(conn->tls); + if (bufferevent_ssl_renegotiate(conn->_base.bufev)<0) { + log_warn(LD_OR, "Start_renegotiating went badly."); + connection_mark_for_close(TO_CONN(conn)); + } + tor_tls_unblock_renegotiation(conn->tls); + return; /* ???? */ + } + } + } else { + const int handshakes = tor_tls_get_num_server_handshakes(conn->tls); + + if (handshakes == 1) { + /* v2 or v3 handshake, as a server. Only got one handshake, so + * wait for the next one. */ + tor_tls_set_renegotiate_callback(conn->tls, + connection_or_tls_renegotiated_cb, + conn); + conn->_base.state = OR_CONN_STATE_TLS_SERVER_RENEGOTIATING; + } else if (handshakes == 2) { + /* v2 handshake, as a server. Two handshakes happened already, + * so we treat renegotiation as done. + */ + connection_or_tls_renegotiated_cb(conn->tls, conn); + } else if (handshakes > 2) { + log_warn(LD_OR, "More than two handshakes done on connection. " + "Closing."); + connection_mark_for_close(TO_CONN(conn)); + } else { + log_warn(LD_BUG, "We were unexpectedly told that a connection " + "got %d handshakes. Closing.", handshakes); + connection_mark_for_close(TO_CONN(conn)); + } + return; + } + } + connection_watch_events(TO_CONN(conn), READ_EVENT|WRITE_EVENT); + if (connection_tls_finish_handshake(conn) < 0) + connection_mark_for_close(TO_CONN(conn)); /* ???? */ + return; + } + + if (event & BEV_EVENT_ERROR) { + unsigned long err; + while ((err = bufferevent_get_openssl_error(bufev))) { + tor_tls_log_one_error(conn->tls, err, LOG_WARN, LD_OR, + "handshaking (with bufferevent)"); + } + } + + connection_handle_event_cb(bufev, event, arg); +} +#endif + /** Return 1 if we initiated this connection, or 0 if it started * out as an incoming connection. */ @@ -1002,6 +1380,29 @@ connection_or_nonopen_was_started_here(or_connection_t *conn) return !tor_tls_is_server(conn->tls); } +/** Set the circid_type field of <b>conn</b> (which determines which part of + * the circuit ID space we're willing to use) based on comparing our ID to + * <b>identity_rcvd</b> */ +void +connection_or_set_circid_type(or_connection_t *conn, + crypto_pk_t *identity_rcvd) +{ + const int started_here = connection_or_nonopen_was_started_here(conn); + crypto_pk_t *our_identity = + started_here ? get_tlsclient_identity_key() : + get_server_identity_key(); + + if (identity_rcvd) { + if (crypto_pk_cmp_keys(our_identity, identity_rcvd)<0) { + conn->circ_id_type = CIRC_ID_TYPE_LOWER; + } else { + conn->circ_id_type = CIRC_ID_TYPE_HIGHER; + } + } else { + conn->circ_id_type = CIRC_ID_TYPE_NEITHER; + } +} + /** <b>Conn</b> just completed its handshake. Return 0 if all is well, and * return -1 if he is lying, broken, or otherwise something is wrong. * @@ -1032,17 +1433,14 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, int started_here, char *digest_rcvd_out) { - crypto_pk_env_t *identity_rcvd=NULL; - or_options_t *options = get_options(); + crypto_pk_t *identity_rcvd=NULL; + const or_options_t *options = get_options(); int severity = server_mode(options) ? LOG_PROTOCOL_WARN : LOG_WARN; const char *safe_address = started_here ? conn->_base.address : safe_str_client(conn->_base.address); const char *conn_type = started_here ? "outgoing" : "incoming"; - crypto_pk_env_t *our_identity = - started_here ? get_tlsclient_identity_key() : - get_server_identity_key(); - int has_cert = 0, has_identity=0; + int has_cert = 0; check_no_tls_errors(); has_cert = tor_tls_peer_has_cert(conn->tls); @@ -1077,21 +1475,45 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, } if (identity_rcvd) { - has_identity = 1; crypto_pk_get_digest(identity_rcvd, digest_rcvd_out); - if (crypto_pk_cmp_keys(our_identity, identity_rcvd)<0) { - conn->circ_id_type = CIRC_ID_TYPE_LOWER; - } else { - conn->circ_id_type = CIRC_ID_TYPE_HIGHER; - } - crypto_free_pk_env(identity_rcvd); } else { memset(digest_rcvd_out, 0, DIGEST_LEN); - conn->circ_id_type = CIRC_ID_TYPE_NEITHER; } - if (started_here && tor_digest_is_zero(conn->identity_digest)) { - connection_or_set_identity_digest(conn, digest_rcvd_out); + connection_or_set_circid_type(conn, identity_rcvd); + crypto_pk_free(identity_rcvd); + + if (started_here) + return connection_or_client_learned_peer_id(conn, + (const uint8_t*)digest_rcvd_out); + + return 0; +} + +/** Called when we (as a connection initiator) have definitively, + * authenticatedly, learned that ID of the Tor instance on the other + * side of <b>conn</b> is <b>peer_id</b>. For v1 and v2 handshakes, + * this is right after we get a certificate chain in a TLS handshake + * or renegotiation. For v3 handshakes, this is right after we get a + * certificate chain in a CERTS cell. + * + * If we want any particular ID before, record the one we got. + * + * If we wanted an ID, but we didn't get it, log a warning and return -1. + * + * If we're testing reachability, remember what we learned. + * + * Return 0 on success, -1 on failure. + */ +int +connection_or_client_learned_peer_id(or_connection_t *conn, + const uint8_t *peer_id) +{ + const or_options_t *options = get_options(); + int severity = server_mode(options) ? LOG_PROTOCOL_WARN : LOG_WARN; + + if (tor_digest_is_zero(conn->identity_digest)) { + connection_or_set_identity_digest(conn, (const char*)peer_id); tor_free(conn->nickname); conn->nickname = tor_malloc(HEX_DIGEST_LEN+2); conn->nickname[0] = '$'; @@ -1103,43 +1525,39 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, /* if it's a bridge and we didn't know its identity fingerprint, now * we do -- remember it for future attempts. */ learned_router_identity(&conn->_base.addr, conn->_base.port, - digest_rcvd_out); + (const char*)peer_id); } - if (started_here) { - int as_advertised = 1; - tor_assert(has_cert); - tor_assert(has_identity); - if (tor_memneq(digest_rcvd_out, conn->identity_digest, DIGEST_LEN)) { - /* I was aiming for a particular digest. I didn't get it! */ - char seen[HEX_DIGEST_LEN+1]; - char expected[HEX_DIGEST_LEN+1]; - base16_encode(seen, sizeof(seen), digest_rcvd_out, DIGEST_LEN); - base16_encode(expected, sizeof(expected), conn->identity_digest, - DIGEST_LEN); - log_fn(severity, LD_HANDSHAKE, - "Tried connecting to router at %s:%d, but identity key was not " - "as expected: wanted %s but got %s.", - conn->_base.address, conn->_base.port, expected, seen); - entry_guard_register_connect_status(conn->identity_digest, 0, 1, - time(NULL)); - control_event_or_conn_status(conn, OR_CONN_EVENT_FAILED, - END_OR_CONN_REASON_OR_IDENTITY); - if (!authdir_mode_tests_reachability(options)) - control_event_bootstrap_problem("foo", END_OR_CONN_REASON_OR_IDENTITY); - as_advertised = 0; - } - if (authdir_mode_tests_reachability(options)) { - dirserv_orconn_tls_done(conn->_base.address, conn->_base.port, - digest_rcvd_out, as_advertised); - } - if (!as_advertised) - return -1; + if (tor_memneq(peer_id, conn->identity_digest, DIGEST_LEN)) { + /* I was aiming for a particular digest. I didn't get it! */ + char seen[HEX_DIGEST_LEN+1]; + char expected[HEX_DIGEST_LEN+1]; + base16_encode(seen, sizeof(seen), (const char*)peer_id, DIGEST_LEN); + base16_encode(expected, sizeof(expected), conn->identity_digest, + DIGEST_LEN); + log_fn(severity, LD_HANDSHAKE, + "Tried connecting to router at %s:%d, but identity key was not " + "as expected: wanted %s but got %s.", + conn->_base.address, conn->_base.port, expected, seen); + entry_guard_register_connect_status(conn->identity_digest, 0, 1, + time(NULL)); + control_event_or_conn_status(conn, OR_CONN_EVENT_FAILED, + END_OR_CONN_REASON_OR_IDENTITY); + if (!authdir_mode_tests_reachability(options)) + control_event_bootstrap_problem( + "Unexpected identity in router certificate", + END_OR_CONN_REASON_OR_IDENTITY); + return -1; } + if (authdir_mode_tests_reachability(options)) { + dirserv_orconn_tls_done(conn->_base.address, conn->_base.port, + (const char*)peer_id); + } + return 0; } -/** The tls handshake is finished. +/** The v1/v2 TLS handshake is finished. * * Make sure we are happy with the person we just handshaked with. * @@ -1149,6 +1567,8 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, * If all is successful, call circuit_n_conn_done() to handle events * that have been pending on the <tls handshake completion. Also set the * directory to be dirty (only matters if I'm an authdirserver). + * + * If this is a v2 TLS handshake, send a versions cell. */ static int connection_tls_finish_handshake(or_connection_t *conn) @@ -1156,7 +1576,9 @@ connection_tls_finish_handshake(or_connection_t *conn) char digest_rcvd[DIGEST_LEN]; int started_here = connection_or_nonopen_was_started_here(conn); - log_debug(LD_HANDSHAKE,"tls handshake with %s done. verifying.", + log_debug(LD_HANDSHAKE,"%s tls handshake on %p with %s done. verifying.", + started_here?"outgoing":"incoming", + conn, safe_str_client(conn->_base.address)); directory_set_dirty(); @@ -1176,25 +1598,52 @@ connection_tls_finish_handshake(or_connection_t *conn) tor_tls_block_renegotiation(conn->tls); return connection_or_set_state_open(conn); } else { - conn->_base.state = OR_CONN_STATE_OR_HANDSHAKING; + conn->_base.state = OR_CONN_STATE_OR_HANDSHAKING_V2; if (connection_init_or_handshake_state(conn, started_here) < 0) return -1; if (!started_here) { connection_or_init_conn_from_address(conn, &conn->_base.addr, conn->_base.port, digest_rcvd, 0); } - return connection_or_send_versions(conn); + return connection_or_send_versions(conn, 0); } } +/** + * Called as client when initial TLS handshake is done, and we notice + * that we got a v3-handshake signalling certificate from the server. + * Set up structures, do bookkeeping, and send the versions cell. + * Return 0 on success and -1 on failure. + */ +static int +connection_or_launch_v3_or_handshake(or_connection_t *conn) +{ + tor_assert(connection_or_nonopen_was_started_here(conn)); + tor_assert(tor_tls_received_v3_certificate(conn->tls)); + + circuit_build_times_network_is_live(&circ_times); + + conn->_base.state = OR_CONN_STATE_OR_HANDSHAKING_V3; + if (connection_init_or_handshake_state(conn, 1) < 0) + return -1; + + return connection_or_send_versions(conn, 1); +} + /** Allocate a new connection handshake state for the connection * <b>conn</b>. Return 0 on success, -1 on failure. */ -static int +int connection_init_or_handshake_state(or_connection_t *conn, int started_here) { or_handshake_state_t *s; + if (conn->handshake_state) { + log_warn(LD_BUG, "Duplicate call to connection_init_or_handshake_state!"); + return 0; + } s = conn->handshake_state = tor_malloc_zero(sizeof(or_handshake_state_t)); s->started_here = started_here ? 1 : 0; + s->digest_sent_data = 1; + s->digest_received_data = 1; return 0; } @@ -1204,10 +1653,89 @@ or_handshake_state_free(or_handshake_state_t *state) { if (!state) return; - memset(state, 0xBE, sizeof(or_handshake_state_t)); + crypto_digest_free(state->digest_sent); + crypto_digest_free(state->digest_received); + tor_cert_free(state->auth_cert); + tor_cert_free(state->id_cert); + memwipe(state, 0xBE, sizeof(or_handshake_state_t)); tor_free(state); } +/** + * Remember that <b>cell</b> has been transmitted (if <b>incoming</b> is + * false) or received (if <b>incoming</b> is true) during a V3 handshake using + * <b>state</b>. + * + * (We don't record the cell, but we keep a digest of everything sent or + * received during the v3 handshake, and the client signs it in an + * authenticate cell.) + */ +void +or_handshake_state_record_cell(or_handshake_state_t *state, + const cell_t *cell, + int incoming) +{ + crypto_digest_t *d, **dptr; + packed_cell_t packed; + if (incoming) { + if (!state->digest_received_data) + return; + } else { + if (!state->digest_sent_data) + return; + } + if (!incoming) { + log_warn(LD_BUG, "We shouldn't be sending any non-variable-length cells " + "while making a handshake digest. But we think we are sending " + "one with type %d.", (int)cell->command); + } + dptr = incoming ? &state->digest_received : &state->digest_sent; + if (! *dptr) + *dptr = crypto_digest256_new(DIGEST_SHA256); + + d = *dptr; + /* Re-packing like this is a little inefficient, but we don't have to do + this very often at all. */ + cell_pack(&packed, cell); + crypto_digest_add_bytes(d, packed.body, sizeof(packed.body)); + memwipe(&packed, 0, sizeof(packed)); +} + +/** Remember that a variable-length <b>cell</b> has been transmitted (if + * <b>incoming</b> is false) or received (if <b>incoming</b> is true) during a + * V3 handshake using <b>state</b>. + * + * (We don't record the cell, but we keep a digest of everything sent or + * received during the v3 handshake, and the client signs it in an + * authenticate cell.) + */ +void +or_handshake_state_record_var_cell(or_handshake_state_t *state, + const var_cell_t *cell, + int incoming) +{ + crypto_digest_t *d, **dptr; + char buf[VAR_CELL_HEADER_SIZE]; + if (incoming) { + if (!state->digest_received_data) + return; + } else { + if (!state->digest_sent_data) + return; + } + dptr = incoming ? &state->digest_received : &state->digest_sent; + if (! *dptr) + *dptr = crypto_digest256_new(DIGEST_SHA256); + + d = *dptr; + + var_cell_pack_header(cell, buf); + crypto_digest_add_bytes(d, buf, sizeof(buf)); + crypto_digest_add_bytes(d, (const char *)cell->payload, cell->payload_len); + + memwipe(buf, 0, sizeof(buf)); +} + /** Set <b>conn</b>'s state to OR_CONN_STATE_OPEN, and tell other subsystems * as appropriate. Called when we are done with all TLS and OR handshaking. */ @@ -1237,19 +1765,20 @@ connection_or_set_state_open(or_connection_t *conn) router_set_status(conn->identity_digest, 1); } else { /* only report it to the geoip module if it's not a known router */ - if (!router_get_by_digest(conn->identity_digest)) { - if (tor_addr_family(&TO_CONN(conn)->addr) == AF_INET) { - /*XXXX IP6 support ipv6 geoip.*/ - uint32_t a = tor_addr_to_ipv4h(&TO_CONN(conn)->addr); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, a, now); - } + if (!router_get_by_id_digest(conn->identity_digest)) { + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &TO_CONN(conn)->addr, + now); } } or_handshake_state_free(conn->handshake_state); conn->handshake_state = NULL; + IF_HAS_BUFFEREVENT(TO_CONN(conn), { + connection_watch_events(TO_CONN(conn), READ_EVENT|WRITE_EVENT); + }) ELSE_IF_NO_BUFFEREVENT { + connection_start_reading(TO_CONN(conn)); + } - connection_start_reading(TO_CONN(conn)); circuit_n_conn_done(conn, 1); /* send the pending creates, if any. */ return 0; @@ -1271,6 +1800,9 @@ connection_or_write_cell_to_buf(const cell_t *cell, or_connection_t *conn) connection_write_to_buf(networkcell.body, CELL_NETWORK_SIZE, TO_CONN(conn)); + if (conn->_base.state == OR_CONN_STATE_OR_HANDSHAKING_V3) + or_handshake_state_record_cell(conn->handshake_state, cell, 0); + if (cell->command != CELL_PADDING) conn->timestamp_last_added_nonpadding = approx_time(); } @@ -1290,16 +1822,24 @@ connection_or_write_var_cell_to_buf(const var_cell_t *cell, connection_write_to_buf(hdr, sizeof(hdr), TO_CONN(conn)); connection_write_to_buf((char*)cell->payload, cell->payload_len, TO_CONN(conn)); + if (conn->_base.state == OR_CONN_STATE_OR_HANDSHAKING_V3) + or_handshake_state_record_var_cell(conn->handshake_state, cell, 0); if (cell->command != CELL_PADDING) conn->timestamp_last_added_nonpadding = approx_time(); } -/** See whether there's a variable-length cell waiting on <b>conn</b>'s +/** See whether there's a variable-length cell waiting on <b>or_conn</b>'s * inbuf. Return values as for fetch_var_cell_from_buf(). */ static int -connection_fetch_var_cell_from_buf(or_connection_t *conn, var_cell_t **out) +connection_fetch_var_cell_from_buf(or_connection_t *or_conn, var_cell_t **out) { - return fetch_var_cell_from_buf(conn->_base.inbuf, out, conn->link_proto); + connection_t *conn = TO_CONN(or_conn); + IF_HAS_BUFFEREVENT(conn, { + struct evbuffer *input = bufferevent_get_input(conn->bufev); + return fetch_var_cell_from_evbuffer(input, out, or_conn->link_proto); + }) ELSE_IF_NO_BUFFEREVENT { + return fetch_var_cell_from_buf(conn->inbuf, out, or_conn->link_proto); + } } /** Process cells from <b>conn</b>'s inbuf. @@ -1317,7 +1857,7 @@ connection_or_process_cells_from_inbuf(or_connection_t *conn) while (1) { log_debug(LD_OR, "%d: starting, inbuf_datalen %d (%d pending in tls object).", - conn->_base.s,(int)buf_datalen(conn->_base.inbuf), + conn->_base.s,(int)connection_get_inbuf_len(TO_CONN(conn)), tor_tls_get_pending_bytes(conn->tls)); if (connection_fetch_var_cell_from_buf(conn, &var_cell)) { if (!var_cell) @@ -1328,8 +1868,8 @@ connection_or_process_cells_from_inbuf(or_connection_t *conn) } else { char buf[CELL_NETWORK_SIZE]; cell_t cell; - if (buf_datalen(conn->_base.inbuf) < CELL_NETWORK_SIZE) /* whole response - available? */ + if (connection_get_inbuf_len(TO_CONN(conn)) + < CELL_NETWORK_SIZE) /* whole response available? */ return 0; /* not yet */ circuit_build_times_network_is_live(&circ_times); @@ -1368,7 +1908,7 @@ connection_or_send_destroy(circid_t circ_id, or_connection_t *conn, int reason) } /** Array of recognized link protocol versions. */ -static const uint16_t or_protocol_versions[] = { 1, 2 }; +static const uint16_t or_protocol_versions[] = { 1, 2, 3 }; /** Number of versions in <b>or_protocol_versions</b>. */ static const int n_or_protocol_versions = (int)( sizeof(or_protocol_versions)/sizeof(uint16_t) ); @@ -1387,20 +1927,33 @@ is_or_protocol_version_known(uint16_t v) } /** Send a VERSIONS cell on <b>conn</b>, telling the other host about the - * link protocol versions that this Tor can support. */ -static int -connection_or_send_versions(or_connection_t *conn) + * link protocol versions that this Tor can support. + * + * If <b>v3_plus</b>, this is part of a V3 protocol handshake, so only + * allow protocol version v3 or later. If not <b>v3_plus</b>, this is + * not part of a v3 protocol handshake, so don't allow protocol v3 or + * later. + **/ +int +connection_or_send_versions(or_connection_t *conn, int v3_plus) { var_cell_t *cell; int i; + int n_versions = 0; + const int min_version = v3_plus ? 3 : 0; + const int max_version = v3_plus ? UINT16_MAX : 2; tor_assert(conn->handshake_state && !conn->handshake_state->sent_versions_at); cell = var_cell_new(n_or_protocol_versions * 2); cell->command = CELL_VERSIONS; for (i = 0; i < n_or_protocol_versions; ++i) { uint16_t v = or_protocol_versions[i]; - set_uint16(cell->payload+(2*i), htons(v)); + if (v < min_version || v > max_version) + continue; + set_uint16(cell->payload+(2*n_versions), htons(v)); + ++n_versions; } + cell->payload_len = n_versions * 2; connection_or_write_var_cell_to_buf(cell, conn); conn->handshake_state->sent_versions_at = time(NULL); @@ -1416,10 +1969,12 @@ connection_or_send_netinfo(or_connection_t *conn) { cell_t cell; time_t now = time(NULL); - routerinfo_t *me; + const routerinfo_t *me; int len; uint8_t *out; + tor_assert(conn->handshake_state); + memset(&cell, 0, sizeof(cell_t)); cell.command = CELL_NETINFO; @@ -1428,13 +1983,20 @@ connection_or_send_netinfo(or_connection_t *conn) /* Their address. */ out = cell.payload + 4; - len = append_address_to_payload(out, &conn->real_addr); + /* We use &conn->real_addr below, unless it hasn't yet been set. If it + * hasn't yet been set, we know that _base.addr hasn't been tampered with + * yet either. */ + len = append_address_to_payload(out, !tor_addr_is_null(&conn->real_addr) + ? &conn->real_addr : &conn->_base.addr); if (len<0) return -1; out += len; - /* My address. */ - if ((me = router_get_my_routerinfo())) { + /* My address -- only include it if I'm a public relay, or if I'm a + * bridge and this is an incoming connection. If I'm a bridge and this + * is an outgoing connection, act like a normal client and omit it. */ + if ((public_server_mode(get_options()) || !conn->is_outgoing) && + (me = router_get_my_routerinfo())) { tor_addr_t my_addr; *out++ = 1; /* only one address is supported. */ @@ -1446,8 +2008,286 @@ connection_or_send_netinfo(or_connection_t *conn) *out = 0; } + conn->handshake_state->digest_sent_data = 0; connection_or_write_cell_to_buf(&cell, conn); return 0; } +/** Send a CERTS cell on the connection <b>conn</b>. Return 0 on success, -1 + * on failure. */ +int +connection_or_send_certs_cell(or_connection_t *conn) +{ + const tor_cert_t *link_cert = NULL, *id_cert = NULL; + const uint8_t *link_encoded = NULL, *id_encoded = NULL; + size_t link_len, id_len; + var_cell_t *cell; + size_t cell_len; + ssize_t pos; + int server_mode; + + tor_assert(conn->_base.state == OR_CONN_STATE_OR_HANDSHAKING_V3); + + if (! conn->handshake_state) + return -1; + server_mode = ! conn->handshake_state->started_here; + if (tor_tls_get_my_certs(server_mode, &link_cert, &id_cert) < 0) + return -1; + tor_cert_get_der(link_cert, &link_encoded, &link_len); + tor_cert_get_der(id_cert, &id_encoded, &id_len); + + cell_len = 1 /* 1 byte: num certs in cell */ + + 2 * ( 1 + 2 ) /* For each cert: 1 byte for type, 2 for length */ + + link_len + id_len; + cell = var_cell_new(cell_len); + cell->command = CELL_CERTS; + cell->payload[0] = 2; + pos = 1; + + if (server_mode) + cell->payload[pos] = OR_CERT_TYPE_TLS_LINK; /* Link cert */ + else + cell->payload[pos] = OR_CERT_TYPE_AUTH_1024; /* client authentication */ + set_uint16(&cell->payload[pos+1], htons(link_len)); + memcpy(&cell->payload[pos+3], link_encoded, link_len); + pos += 3 + link_len; + + cell->payload[pos] = OR_CERT_TYPE_ID_1024; /* ID cert */ + set_uint16(&cell->payload[pos+1], htons(id_len)); + memcpy(&cell->payload[pos+3], id_encoded, id_len); + pos += 3 + id_len; + + tor_assert(pos == (int)cell_len); /* Otherwise we just smashed the heap */ + + connection_or_write_var_cell_to_buf(cell, conn); + var_cell_free(cell); + + return 0; +} + +/** Send an AUTH_CHALLENGE cell on the connection <b>conn</b>. Return 0 + * on success, -1 on failure. */ +int +connection_or_send_auth_challenge_cell(or_connection_t *conn) +{ + var_cell_t *cell; + uint8_t *cp; + uint8_t challenge[OR_AUTH_CHALLENGE_LEN]; + tor_assert(conn->_base.state == OR_CONN_STATE_OR_HANDSHAKING_V3); + + if (! conn->handshake_state) + return -1; + + if (crypto_rand((char*)challenge, OR_AUTH_CHALLENGE_LEN) < 0) + return -1; + cell = var_cell_new(OR_AUTH_CHALLENGE_LEN + 4); + cell->command = CELL_AUTH_CHALLENGE; + memcpy(cell->payload, challenge, OR_AUTH_CHALLENGE_LEN); + cp = cell->payload + OR_AUTH_CHALLENGE_LEN; + set_uint16(cp, htons(1)); /* We recognize one authentication type. */ + set_uint16(cp+2, htons(AUTHTYPE_RSA_SHA256_TLSSECRET)); + + connection_or_write_var_cell_to_buf(cell, conn); + var_cell_free(cell); + memwipe(challenge, 0, sizeof(challenge)); + + return 0; +} + +/** Compute the main body of an AUTHENTICATE cell that a client can use + * to authenticate itself on a v3 handshake for <b>conn</b>. Write it to the + * <b>outlen</b>-byte buffer at <b>out</b>. + * + * If <b>server</b> is true, only calculate the first + * V3_AUTH_FIXED_PART_LEN bytes -- the part of the authenticator that's + * determined by the rest of the handshake, and which match the provided value + * exactly. + * + * If <b>server</b> is false and <b>signing_key</b> is NULL, calculate the + * first V3_AUTH_BODY_LEN bytes of the authenticator (that is, everything + * that should be signed), but don't actually sign it. + * + * If <b>server</b> is false and <b>signing_key</b> is provided, calculate the + * entire authenticator, signed with <b>signing_key</b>. + * + * Return the length of the cell body on success, and -1 on failure. + */ +int +connection_or_compute_authenticate_cell_body(or_connection_t *conn, + uint8_t *out, size_t outlen, + crypto_pk_t *signing_key, + int server) +{ + uint8_t *ptr; + + /* assert state is reasonable XXXX */ + + if (outlen < V3_AUTH_FIXED_PART_LEN || + (!server && outlen < V3_AUTH_BODY_LEN)) + return -1; + + ptr = out; + + /* Type: 8 bytes. */ + memcpy(ptr, "AUTH0001", 8); + ptr += 8; + + { + const tor_cert_t *id_cert=NULL, *link_cert=NULL; + const digests_t *my_digests, *their_digests; + const uint8_t *my_id, *their_id, *client_id, *server_id; + if (tor_tls_get_my_certs(0, &link_cert, &id_cert)) + return -1; + my_digests = tor_cert_get_id_digests(id_cert); + their_digests = tor_cert_get_id_digests(conn->handshake_state->id_cert); + tor_assert(my_digests); + tor_assert(their_digests); + my_id = (uint8_t*)my_digests->d[DIGEST_SHA256]; + their_id = (uint8_t*)their_digests->d[DIGEST_SHA256]; + + client_id = server ? their_id : my_id; + server_id = server ? my_id : their_id; + + /* Client ID digest: 32 octets. */ + memcpy(ptr, client_id, 32); + ptr += 32; + + /* Server ID digest: 32 octets. */ + memcpy(ptr, server_id, 32); + ptr += 32; + } + + { + crypto_digest_t *server_d, *client_d; + if (server) { + server_d = conn->handshake_state->digest_sent; + client_d = conn->handshake_state->digest_received; + } else { + client_d = conn->handshake_state->digest_sent; + server_d = conn->handshake_state->digest_received; + } + + /* Server log digest : 32 octets */ + crypto_digest_get_digest(server_d, (char*)ptr, 32); + ptr += 32; + + /* Client log digest : 32 octets */ + crypto_digest_get_digest(client_d, (char*)ptr, 32); + ptr += 32; + } + + { + /* Digest of cert used on TLS link : 32 octets. */ + const tor_cert_t *cert = NULL; + tor_cert_t *freecert = NULL; + if (server) { + tor_tls_get_my_certs(1, &cert, NULL); + } else { + freecert = tor_tls_get_peer_cert(conn->tls); + cert = freecert; + } + if (!cert) + return -1; + memcpy(ptr, tor_cert_get_cert_digests(cert)->d[DIGEST_SHA256], 32); + + if (freecert) + tor_cert_free(freecert); + ptr += 32; + } + + /* HMAC of clientrandom and serverrandom using master key : 32 octets */ + tor_tls_get_tlssecrets(conn->tls, ptr); + ptr += 32; + + tor_assert(ptr - out == V3_AUTH_FIXED_PART_LEN); + + if (server) + return V3_AUTH_FIXED_PART_LEN; // ptr-out + + /* Time: 8 octets. */ + { + uint64_t now = time(NULL); + if ((time_t)now < 0) + return -1; + set_uint32(ptr, htonl((uint32_t)(now>>32))); + set_uint32(ptr+4, htonl((uint32_t)now)); + ptr += 8; + } + + /* Nonce: 16 octets. */ + crypto_rand((char*)ptr, 16); + ptr += 16; + + tor_assert(ptr - out == V3_AUTH_BODY_LEN); + + if (!signing_key) + return V3_AUTH_BODY_LEN; // ptr - out + + { + int siglen; + char d[32]; + crypto_digest256(d, (char*)out, ptr-out, DIGEST_SHA256); + siglen = crypto_pk_private_sign(signing_key, + (char*)ptr, outlen - (ptr-out), + d, 32); + if (siglen < 0) + return -1; + + ptr += siglen; + tor_assert(ptr <= out+outlen); + return (int)(ptr - out); + } +} + +/** Send an AUTHENTICATE cell on the connection <b>conn</b>. Return 0 on + * success, -1 on failure */ +int +connection_or_send_authenticate_cell(or_connection_t *conn, int authtype) +{ + var_cell_t *cell; + crypto_pk_t *pk = tor_tls_get_my_client_auth_key(); + int authlen; + size_t cell_maxlen; + /* XXXX make sure we're actually supposed to send this! */ + + if (!pk) { + log_warn(LD_BUG, "Can't compute authenticate cell: no client auth key"); + return -1; + } + if (authtype != AUTHTYPE_RSA_SHA256_TLSSECRET) { + log_warn(LD_BUG, "Tried to send authenticate cell with unknown " + "authentication type %d", authtype); + return -1; + } + + cell_maxlen = 4 + /* overhead */ + V3_AUTH_BODY_LEN + /* Authentication body */ + crypto_pk_keysize(pk) + /* Max signature length */ + 16 /* add a few extra bytes just in case. */; + + cell = var_cell_new(cell_maxlen); + cell->command = CELL_AUTHENTICATE; + set_uint16(cell->payload, htons(AUTHTYPE_RSA_SHA256_TLSSECRET)); + /* skip over length ; we don't know that yet. */ + + authlen = connection_or_compute_authenticate_cell_body(conn, + cell->payload+4, + cell_maxlen-4, + pk, + 0 /* not server */); + if (authlen < 0) { + log_warn(LD_BUG, "Unable to compute authenticate cell!"); + var_cell_free(cell); + return -1; + } + tor_assert(authlen + 4 <= cell->payload_len); + set_uint16(cell->payload+2, htons(authlen)); + cell->payload_len = authlen + 4; + + connection_or_write_var_cell_to_buf(cell, conn); + var_cell_free(cell); + + return 0; +} + diff --git a/src/or/connection_or.h b/src/or/connection_or.h index 70ef96a335..b78c444921 100644 --- a/src/or/connection_or.h +++ b/src/or/connection_or.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -14,30 +14,53 @@ void connection_or_remove_from_identity_map(or_connection_t *conn); void connection_or_clear_identity_map(void); +void clear_broken_connection_map(int disable); or_connection_t *connection_or_get_for_extend(const char *digest, const tor_addr_t *target_addr, const char **msg_out, int *launch_out); void connection_or_set_bad_connections(const char *digest, int force); +void connection_or_block_renegotiation(or_connection_t *conn); int connection_or_reached_eof(or_connection_t *conn); int connection_or_process_inbuf(or_connection_t *conn); int connection_or_flushed_some(or_connection_t *conn); int connection_or_finished_flushing(or_connection_t *conn); int connection_or_finished_connecting(or_connection_t *conn); +void connection_or_about_to_close(or_connection_t *conn); int connection_or_digest_is_known_relay(const char *id_digest); void connection_or_update_token_buckets(smartlist_t *conns, - or_options_t *options); + const or_options_t *options); void connection_or_connect_failed(or_connection_t *conn, int reason, const char *msg); or_connection_t *connection_or_connect(const tor_addr_t *addr, uint16_t port, const char *id_digest); +void connection_or_report_broken_states(int severity, int domain); + int connection_tls_start_handshake(or_connection_t *conn, int receiving); int connection_tls_continue_handshake(or_connection_t *conn); +int connection_init_or_handshake_state(or_connection_t *conn, + int started_here); +void connection_or_init_conn_from_address(or_connection_t *conn, + const tor_addr_t *addr, + uint16_t port, + const char *id_digest, + int started_here); +int connection_or_client_learned_peer_id(or_connection_t *conn, + const uint8_t *peer_id); +void connection_or_set_circid_type(or_connection_t *conn, + crypto_pk_t *identity_rcvd); void or_handshake_state_free(or_handshake_state_t *state); +void or_handshake_state_record_cell(or_handshake_state_t *state, + const cell_t *cell, + int incoming); +void or_handshake_state_record_var_cell(or_handshake_state_t *state, + const var_cell_t *cell, + int incoming); + int connection_or_set_state_open(or_connection_t *conn); void connection_or_write_cell_to_buf(const cell_t *cell, or_connection_t *conn); @@ -45,7 +68,16 @@ void connection_or_write_var_cell_to_buf(const var_cell_t *cell, or_connection_t *conn); int connection_or_send_destroy(circid_t circ_id, or_connection_t *conn, int reason); +int connection_or_send_versions(or_connection_t *conn, int v3_plus); int connection_or_send_netinfo(or_connection_t *conn); +int connection_or_send_certs_cell(or_connection_t *conn); +int connection_or_send_auth_challenge_cell(or_connection_t *conn); +int connection_or_compute_authenticate_cell_body(or_connection_t *conn, + uint8_t *out, size_t outlen, + crypto_pk_t *signing_key, + int server); +int connection_or_send_authenticate_cell(or_connection_t *conn, int type); + int is_or_protocol_version_known(uint16_t version); void cell_pack(packed_cell_t *dest, const cell_t *src); diff --git a/src/or/control.c b/src/or/control.c index 782468f6fe..913d18a7fc 100644 --- a/src/or/control.c +++ b/src/or/control.c @@ -1,5 +1,5 @@ /* Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -18,6 +18,7 @@ #include "config.h" #include "connection.h" #include "connection_edge.h" +#include "connection_or.h" #include "control.h" #include "directory.h" #include "dirserv.h" @@ -26,12 +27,19 @@ #include "hibernate.h" #include "main.h" #include "networkstatus.h" +#include "nodelist.h" #include "policies.h" #include "reasons.h" +#include "rephist.h" #include "router.h" #include "routerlist.h" #include "routerparse.h" +#ifndef _WIN32 +#include <pwd.h> +#include <sys/resource.h> +#endif + #include "procmon.h" /** Yield true iff <b>s</b> is the state of a control_connection_t that has @@ -47,7 +55,7 @@ #define EVENT_STREAM_STATUS 0x0002 #define EVENT_OR_CONN_STATUS 0x0003 #define EVENT_BANDWIDTH_USED 0x0004 -#define EVENT_LOG_OBSOLETE 0x0005 /* Can reclaim this. */ +#define EVENT_CIRCUIT_STATUS_MINOR 0x0005 #define EVENT_NEW_DESC 0x0006 #define EVENT_DEBUG_MSG 0x0007 #define EVENT_INFO_MSG 0x0008 @@ -66,7 +74,9 @@ #define EVENT_CLIENTS_SEEN 0x0015 #define EVENT_NEWCONSENSUS 0x0016 #define EVENT_BUILDTIMEOUT_SET 0x0017 -#define _EVENT_MAX 0x0017 +#define EVENT_SIGNAL 0x0018 +#define EVENT_CONF_CHANGED 0x0019 +#define _EVENT_MAX 0x0019 /* If _EVENT_MAX ever hits 0x0020, we need to make the mask wider. */ /** Bitfield: The bit 1<<e is set if <b>any</b> open control @@ -181,7 +191,7 @@ static int handle_control_resolve(control_connection_t *conn, uint32_t len, static int handle_control_usefeature(control_connection_t *conn, uint32_t len, const char *body); -static int write_stream_target_to_buf(edge_connection_t *conn, char *buf, +static int write_stream_target_to_buf(entry_connection_t *conn, char *buf, size_t len); static void orconn_target_get_name(char *buf, size_t len, or_connection_t *conn); @@ -276,8 +286,7 @@ control_adjust_event_log_severity(void) break; } } - if (EVENT_IS_INTERESTING(EVENT_LOG_OBSOLETE) || - EVENT_IS_INTERESTING(EVENT_STATUS_GENERAL)) { + if (EVENT_IS_INTERESTING(EVENT_STATUS_GENERAL)) { if (min_log_event > EVENT_NOTICE_MSG) min_log_event = EVENT_NOTICE_MSG; if (max_log_event < EVENT_ERR_MSG) @@ -507,8 +516,8 @@ connection_printf_to_buf(control_connection_t *conn, const char *format, ...) va_end(ap); if (len < 0) { - log_warn(LD_BUG, "Unable to format string for controller."); - return; + log_err(LD_BUG, "Unable to format string for controller."); + tor_assert(0); } connection_write_to_buf(buf, (size_t)len, TO_CONN(conn)); @@ -522,26 +531,23 @@ control_ports_write_to_file(void) { smartlist_t *lines; char *joined = NULL; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (!options->ControlPortWriteToFile) return; - lines = smartlist_create(); + lines = smartlist_new(); SMARTLIST_FOREACH_BEGIN(get_connection_array(), const connection_t *, conn) { - char *port_str = NULL; if (conn->type != CONN_TYPE_CONTROL_LISTENER || conn->marked_for_close) continue; #ifdef AF_UNIX if (conn->socket_family == AF_UNIX) { - tor_asprintf(&port_str, "UNIX_PORT=%s\n", conn->address); - smartlist_add(lines, port_str); + smartlist_add_asprintf(lines, "UNIX_PORT=%s\n", conn->address); continue; } #endif - tor_asprintf(&port_str, "PORT=%s:%d\n", conn->address, conn->port); - smartlist_add(lines, port_str); + smartlist_add_asprintf(lines, "PORT=%s:%d\n", conn->address, conn->port); } SMARTLIST_FOREACH_END(conn); joined = smartlist_join_strings(lines, "", 0, NULL); @@ -550,7 +556,7 @@ control_ports_write_to_file(void) log_warn(LD_CONTROL, "Writing %s failed: %s", options->ControlPortWriteToFile, strerror(errno)); } -#ifndef MS_WINDOWS +#ifndef _WIN32 if (options->ControlPortFileGroupReadable) { if (chmod(options->ControlPortWriteToFile, 0640)) { log_warn(LD_FS,"Unable to make %s group-readable.", @@ -606,7 +612,7 @@ send_control_event_string(uint16_t event, event_format_t which, else if (event == EVENT_STATUS_SERVER) is_err = !strcmpstart(msg, "STATUS_SERVER ERR "); if (is_err) - connection_handle_write(TO_CONN(control_conn), 1); + connection_flush(TO_CONN(control_conn)); } } } SMARTLIST_FOREACH_END(conn); @@ -660,7 +666,7 @@ get_circ(const char *id) } /** Given a text stream <b>id</b>, return the corresponding AP connection. */ -static edge_connection_t * +static entry_connection_t * get_stream(const char *id) { uint64_t n_id; @@ -672,7 +678,7 @@ get_stream(const char *id) conn = connection_get_by_global_id(n_id); if (!conn || conn->type != CONN_TYPE_AP || conn->marked_for_close) return NULL; - return TO_EDGE_CONN(conn); + return TO_ENTRY_CONN(conn); } /** Helper for setconf and resetconf. Acts like setconf, except @@ -690,7 +696,7 @@ control_setconf_helper(control_connection_t *conn, uint32_t len, char *body, const int clear_first = 1; char *config; - smartlist_t *entries = smartlist_create(); + smartlist_t *entries = smartlist_new(); /* We have a string, "body", of the format '(key(=val|="val")?)' entries * separated by space. break it into a list of configuration entries. */ @@ -705,7 +711,6 @@ control_setconf_helper(control_connection_t *conn, uint32_t len, char *body, if (*eq == '=') { char *val=NULL; size_t val_len=0; - size_t ent_len; if (*body != '\"') { char *val_start = body; while (!TOR_ISSPACE(*body)) @@ -723,9 +728,7 @@ control_setconf_helper(control_connection_t *conn, uint32_t len, char *body, return 0; } } - ent_len = strlen(key)+val_len+3; - entry = tor_malloc(ent_len+1); - tor_snprintf(entry, ent_len, "%s %s", key, val); + tor_asprintf(&entry, "%s %s", key, val); tor_free(key); tor_free(val); } else { @@ -741,7 +744,7 @@ control_setconf_helper(control_connection_t *conn, uint32_t len, char *body, SMARTLIST_FOREACH(entries, char *, cp, tor_free(cp)); smartlist_free(entries); - if (config_get_lines(config, &lines) < 0) { + if (config_get_lines(config, &lines, 0) < 0) { log_warn(LD_CONTROL,"Controller gave us config lines we can't parse."); connection_write_str_to_buf("551 Couldn't parse configuration\r\n", conn); @@ -806,38 +809,31 @@ static int handle_control_getconf(control_connection_t *conn, uint32_t body_len, const char *body) { - smartlist_t *questions = smartlist_create(); - smartlist_t *answers = smartlist_create(); - smartlist_t *unrecognized = smartlist_create(); + smartlist_t *questions = smartlist_new(); + smartlist_t *answers = smartlist_new(); + smartlist_t *unrecognized = smartlist_new(); char *msg = NULL; size_t msg_len; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int i, len; (void) body_len; /* body is NUL-terminated; so we can ignore len. */ smartlist_split_string(questions, body, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - SMARTLIST_FOREACH(questions, const char *, q, - { + SMARTLIST_FOREACH_BEGIN(questions, const char *, q) { if (!option_is_recognized(q)) { smartlist_add(unrecognized, (char*) q); } else { config_line_t *answer = option_get_assignment(options,q); if (!answer) { const char *name = option_get_canonical_name(q); - size_t alen = strlen(name)+8; - char *astr = tor_malloc(alen); - tor_snprintf(astr, alen, "250-%s\r\n", name); - smartlist_add(answers, astr); + smartlist_add_asprintf(answers, "250-%s\r\n", name); } while (answer) { config_line_t *next; - size_t alen = strlen(answer->key)+strlen(answer->value)+8; - char *astr = tor_malloc(alen); - tor_snprintf(astr, alen, "250-%s=%s\r\n", + smartlist_add_asprintf(answers, "250-%s=%s\r\n", answer->key, answer->value); - smartlist_add(answers, astr); next = answer->next; tor_free(answer->key); @@ -846,7 +842,7 @@ handle_control_getconf(control_connection_t *conn, uint32_t body_len, answer = next; } } - }); + } SMARTLIST_FOREACH_END(q); if ((len = smartlist_len(unrecognized))) { for (i=0; i < len-1; ++i) @@ -887,7 +883,7 @@ handle_control_loadconf(control_connection_t *conn, uint32_t len, const char *msg = NULL; (void) len; - retval = options_init_from_string(body, CMD_RUN_TOR, NULL, &errstring); + retval = options_init_from_string(NULL, body, CMD_RUN_TOR, NULL, &errstring); if (retval != SETOPT_OK) log_warn(LD_CONTROL, @@ -923,15 +919,51 @@ handle_control_loadconf(control_connection_t *conn, uint32_t len, return 0; } +/** Helper structure: maps event values to their names. */ +struct control_event_t { + uint16_t event_code; + const char *event_name; +}; +/** Table mapping event values to their names. Used to implement SETEVENTS + * and GETINFO events/names, and to keep they in sync. */ +static const struct control_event_t control_event_table[] = { + { EVENT_CIRCUIT_STATUS, "CIRC" }, + { EVENT_CIRCUIT_STATUS_MINOR, "CIRC_MINOR" }, + { EVENT_STREAM_STATUS, "STREAM" }, + { EVENT_OR_CONN_STATUS, "ORCONN" }, + { EVENT_BANDWIDTH_USED, "BW" }, + { EVENT_DEBUG_MSG, "DEBUG" }, + { EVENT_INFO_MSG, "INFO" }, + { EVENT_NOTICE_MSG, "NOTICE" }, + { EVENT_WARN_MSG, "WARN" }, + { EVENT_ERR_MSG, "ERR" }, + { EVENT_NEW_DESC, "NEWDESC" }, + { EVENT_ADDRMAP, "ADDRMAP" }, + { EVENT_AUTHDIR_NEWDESCS, "AUTHDIR_NEWDESCS" }, + { EVENT_DESCCHANGED, "DESCCHANGED" }, + { EVENT_NS, "NS" }, + { EVENT_STATUS_GENERAL, "STATUS_GENERAL" }, + { EVENT_STATUS_CLIENT, "STATUS_CLIENT" }, + { EVENT_STATUS_SERVER, "STATUS_SERVER" }, + { EVENT_GUARD, "GUARD" }, + { EVENT_STREAM_BANDWIDTH_USED, "STREAM_BW" }, + { EVENT_CLIENTS_SEEN, "CLIENTS_SEEN" }, + { EVENT_NEWCONSENSUS, "NEWCONSENSUS" }, + { EVENT_BUILDTIMEOUT_SET, "BUILDTIMEOUT_SET" }, + { EVENT_SIGNAL, "SIGNAL" }, + { EVENT_CONF_CHANGED, "CONF_CHANGED"}, + { 0, NULL }, +}; + /** Called when we get a SETEVENTS message: update conn->event_mask, * and reply with DONE or ERROR. */ static int handle_control_setevents(control_connection_t *conn, uint32_t len, const char *body) { - uint16_t event_code; + int event_code = -1; uint32_t event_mask = 0; - smartlist_t *events = smartlist_create(); + smartlist_t *events = smartlist_new(); (void) len; @@ -941,56 +973,22 @@ handle_control_setevents(control_connection_t *conn, uint32_t len, { if (!strcasecmp(ev, "EXTENDED")) { continue; - } else if (!strcasecmp(ev, "CIRC")) - event_code = EVENT_CIRCUIT_STATUS; - else if (!strcasecmp(ev, "STREAM")) - event_code = EVENT_STREAM_STATUS; - else if (!strcasecmp(ev, "ORCONN")) - event_code = EVENT_OR_CONN_STATUS; - else if (!strcasecmp(ev, "BW")) - event_code = EVENT_BANDWIDTH_USED; - else if (!strcasecmp(ev, "DEBUG")) - event_code = EVENT_DEBUG_MSG; - else if (!strcasecmp(ev, "INFO")) - event_code = EVENT_INFO_MSG; - else if (!strcasecmp(ev, "NOTICE")) - event_code = EVENT_NOTICE_MSG; - else if (!strcasecmp(ev, "WARN")) - event_code = EVENT_WARN_MSG; - else if (!strcasecmp(ev, "ERR")) - event_code = EVENT_ERR_MSG; - else if (!strcasecmp(ev, "NEWDESC")) - event_code = EVENT_NEW_DESC; - else if (!strcasecmp(ev, "ADDRMAP")) - event_code = EVENT_ADDRMAP; - else if (!strcasecmp(ev, "AUTHDIR_NEWDESCS")) - event_code = EVENT_AUTHDIR_NEWDESCS; - else if (!strcasecmp(ev, "DESCCHANGED")) - event_code = EVENT_DESCCHANGED; - else if (!strcasecmp(ev, "NS")) - event_code = EVENT_NS; - else if (!strcasecmp(ev, "STATUS_GENERAL")) - event_code = EVENT_STATUS_GENERAL; - else if (!strcasecmp(ev, "STATUS_CLIENT")) - event_code = EVENT_STATUS_CLIENT; - else if (!strcasecmp(ev, "STATUS_SERVER")) - event_code = EVENT_STATUS_SERVER; - else if (!strcasecmp(ev, "GUARD")) - event_code = EVENT_GUARD; - else if (!strcasecmp(ev, "STREAM_BW")) - event_code = EVENT_STREAM_BANDWIDTH_USED; - else if (!strcasecmp(ev, "CLIENTS_SEEN")) - event_code = EVENT_CLIENTS_SEEN; - else if (!strcasecmp(ev, "NEWCONSENSUS")) - event_code = EVENT_NEWCONSENSUS; - else if (!strcasecmp(ev, "BUILDTIMEOUT_SET")) - event_code = EVENT_BUILDTIMEOUT_SET; - else { - connection_printf_to_buf(conn, "552 Unrecognized event \"%s\"\r\n", - ev); - SMARTLIST_FOREACH(events, char *, e, tor_free(e)); - smartlist_free(events); - return 0; + } else { + int i; + for (i = 0; control_event_table[i].event_name != NULL; ++i) { + if (!strcasecmp(ev, control_event_table[i].event_name)) { + event_code = control_event_table[i].event_code; + break; + } + } + + if (event_code == -1) { + connection_printf_to_buf(conn, "552 Unrecognized event \"%s\"\r\n", + ev); + SMARTLIST_FOREACH(events, char *, e, tor_free(e)); + smartlist_free(events); + return 0; + } } event_mask |= (1 << event_code); } @@ -1014,7 +1012,7 @@ decode_hashed_passwords(config_line_t *passwords) { char decoded[64]; config_line_t *cl; - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); tor_assert(passwords); @@ -1052,7 +1050,7 @@ handle_control_authenticate(control_connection_t *conn, uint32_t len, const char *body) { int used_quoted_string = 0; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); const char *errstr = NULL; char *password; size_t password_len; @@ -1061,7 +1059,10 @@ handle_control_authenticate(control_connection_t *conn, uint32_t len, int bad_cookie=0, bad_password=0; smartlist_t *sl = NULL; - if (TOR_ISXDIGIT(body[0])) { + if (!len) { + password = tor_strdup(""); + password_len = 0; + } else if (TOR_ISXDIGIT(body[0])) { cp = body; while (TOR_ISXDIGIT(*cp)) ++cp; @@ -1078,9 +1079,6 @@ handle_control_authenticate(control_connection_t *conn, uint32_t len, tor_free(password); return 0; } - } else if (TOR_ISSPACE(body[0])) { - password = tor_strdup(""); - password_len = 0; } else { if (!decode_escaped_string(body, len, &password, &password_len)) { connection_write_str_to_buf("551 Invalid quoted string. You need " @@ -1153,7 +1151,7 @@ handle_control_authenticate(control_connection_t *conn, uint32_t len, smartlist_t *sl_tmp; char received[DIGEST_LEN]; int also_cookie = options->CookieAuthentication; - sl = smartlist_create(); + sl = smartlist_new(); if (options->HashedControlPassword) { sl_tmp = decode_hashed_passwords(options->HashedControlPassword); if (!sl_tmp) @@ -1287,7 +1285,7 @@ handle_control_signal(control_connection_t *conn, uint32_t len, send_control_done(conn); /* Flush the "done" first if the signal might make us shut down. */ if (sig == SIGTERM || sig == SIGINT) - connection_handle_write(TO_CONN(conn), 1); + connection_flush(TO_CONN(conn)); process_signal(sig); @@ -1314,6 +1312,17 @@ handle_control_takeownership(control_connection_t *conn, uint32_t len, return 0; } +/** Return true iff <b>addr</b> is unusable as a mapaddress target because of + * containing funny characters. */ +static int +address_is_invalid_mapaddress_target(const char *addr) +{ + if (!strcmpstart(addr, "*.")) + return address_is_invalid_destination(addr+2, 1); + else + return address_is_invalid_destination(addr, 1); +} + /** Called when we get a MAPADDRESS command; try to bind all listed addresses, * and report success or failure. */ static int @@ -1327,24 +1336,20 @@ handle_control_mapaddress(control_connection_t *conn, uint32_t len, size_t sz; (void) len; /* body is NUL-terminated, so it's safe to ignore the length. */ - lines = smartlist_create(); - elts = smartlist_create(); - reply = smartlist_create(); + lines = smartlist_new(); + elts = smartlist_new(); + reply = smartlist_new(); smartlist_split_string(lines, body, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - SMARTLIST_FOREACH(lines, char *, line, - { + SMARTLIST_FOREACH_BEGIN(lines, char *, line) { tor_strlower(line); smartlist_split_string(elts, line, "=", 0, 2); if (smartlist_len(elts) == 2) { const char *from = smartlist_get(elts,0); const char *to = smartlist_get(elts,1); - size_t anslen = strlen(line)+512; - char *ans = tor_malloc(anslen); - if (address_is_invalid_destination(to, 1)) { - tor_snprintf(ans, anslen, + if (address_is_invalid_mapaddress_target(to)) { + smartlist_add_asprintf(reply, "512-syntax error: invalid address '%s'", to); - smartlist_add(reply, ans); log_warn(LD_CONTROL, "Skipping invalid argument '%s' in MapAddress msg", to); } else if (!strcmp(from, ".") || !strcmp(from, "0.0.0.0")) { @@ -1352,34 +1357,38 @@ handle_control_mapaddress(control_connection_t *conn, uint32_t len, !strcmp(from,".") ? RESOLVED_TYPE_HOSTNAME : RESOLVED_TYPE_IPV4, tor_strdup(to)); if (!address) { - tor_snprintf(ans, anslen, + smartlist_add_asprintf(reply, "451-resource exhausted: skipping '%s'", line); - smartlist_add(reply, ans); log_warn(LD_CONTROL, "Unable to allocate address for '%s' in MapAddress msg", safe_str_client(line)); } else { - tor_snprintf(ans, anslen, "250-%s=%s", address, to); - smartlist_add(reply, ans); + smartlist_add_asprintf(reply, "250-%s=%s", address, to); } } else { - addressmap_register(from, tor_strdup(to), 1, ADDRMAPSRC_CONTROLLER); - tor_snprintf(ans, anslen, "250-%s", line); - smartlist_add(reply, ans); + const char *msg; + if (addressmap_register_auto(from, to, 1, + ADDRMAPSRC_CONTROLLER, &msg) < 0) { + smartlist_add_asprintf(reply, + "512-syntax error: invalid address mapping " + " '%s': %s", line, msg); + log_warn(LD_CONTROL, + "Skipping invalid argument '%s' in MapAddress msg: %s", + line, msg); + } else { + smartlist_add_asprintf(reply, "250-%s", line); + } } } else { - size_t anslen = strlen(line)+256; - char *ans = tor_malloc(anslen); - tor_snprintf(ans, anslen, "512-syntax error: mapping '%s' is " + smartlist_add_asprintf(reply, "512-syntax error: mapping '%s' is " "not of expected form 'foo=bar'.", line); - smartlist_add(reply, ans); log_info(LD_CONTROL, "Skipping MapAddress '%s': wrong " "number of items.", safe_str_client(line)); } SMARTLIST_FOREACH(elts, char *, cp, tor_free(cp)); smartlist_clear(elts); - }); + } SMARTLIST_FOREACH_END(line); SMARTLIST_FOREACH(lines, char *, cp, tor_free(cp)); smartlist_free(lines); smartlist_free(elts); @@ -1410,17 +1419,27 @@ getinfo_helper_misc(control_connection_t *conn, const char *question, if (!strcmp(question, "version")) { *answer = tor_strdup(get_version()); } else if (!strcmp(question, "config-file")) { - *answer = tor_strdup(get_torrc_fname()); + *answer = tor_strdup(get_torrc_fname(0)); + } else if (!strcmp(question, "config-defaults-file")) { + *answer = tor_strdup(get_torrc_fname(1)); } else if (!strcmp(question, "config-text")) { *answer = options_dump(get_options(), 1); } else if (!strcmp(question, "info/names")) { *answer = list_getinfo_options(); + } else if (!strcmp(question, "dormant")) { + int dormant = rep_hist_circbuilding_dormant(time(NULL)); + *answer = tor_strdup(dormant ? "1" : "0"); } else if (!strcmp(question, "events/names")) { - *answer = tor_strdup("CIRC STREAM ORCONN BW DEBUG INFO NOTICE WARN ERR " - "NEWDESC ADDRMAP AUTHDIR_NEWDESCS DESCCHANGED " - "NS STATUS_GENERAL STATUS_CLIENT STATUS_SERVER " - "GUARD STREAM_BW CLIENTS_SEEN NEWCONSENSUS " - "BUILDTIMEOUT_SET"); + int i; + smartlist_t *event_names = smartlist_new(); + + for (i = 0; control_event_table[i].event_name != NULL; ++i) { + smartlist_add(event_names, (char *)control_event_table[i].event_name); + } + + *answer = smartlist_join_strings(event_names, " ", 0, NULL); + + smartlist_free(event_names); } else if (!strcmp(question, "features/names")) { *answer = tor_strdup("VERBOSE_NAMES EXTENDED_EVENTS"); } else if (!strcmp(question, "address")) { @@ -1430,10 +1449,48 @@ getinfo_helper_misc(control_connection_t *conn, const char *question, return -1; } *answer = tor_dup_ip(addr); + } else if (!strcmp(question, "traffic/read")) { + tor_asprintf(answer, U64_FORMAT, U64_PRINTF_ARG(get_bytes_read())); + } else if (!strcmp(question, "traffic/written")) { + tor_asprintf(answer, U64_FORMAT, U64_PRINTF_ARG(get_bytes_written())); + } else if (!strcmp(question, "process/pid")) { + int myPid = -1; + + #ifdef _WIN32 + myPid = _getpid(); + #else + myPid = getpid(); + #endif + + tor_asprintf(answer, "%d", myPid); + } else if (!strcmp(question, "process/uid")) { + #ifdef _WIN32 + *answer = tor_strdup("-1"); + #else + int myUid = geteuid(); + tor_asprintf(answer, "%d", myUid); + #endif + } else if (!strcmp(question, "process/user")) { + #ifdef _WIN32 + *answer = tor_strdup(""); + #else + int myUid = geteuid(); + struct passwd *myPwEntry = getpwuid(myUid); + + if (myPwEntry) { + *answer = tor_strdup(myPwEntry->pw_name); + } else { + *answer = tor_strdup(""); + } + #endif + } else if (!strcmp(question, "process/descriptor-limit")) { + int max_fds=-1; + set_max_file_descriptors(0, &max_fds); + tor_asprintf(answer, "%d", max_fds); } else if (!strcmp(question, "dir-usage")) { *answer = directory_dump_request_log(); } else if (!strcmp(question, "fingerprint")) { - crypto_pk_env_t *server_key; + crypto_pk_t *server_key; if (!server_mode(get_options())) { *errmsg = "Not running in server mode"; return -1; @@ -1455,8 +1512,9 @@ getinfo_helper_misc(control_connection_t *conn, const char *question, * NOTE: <b>ri_body</b> is as returned by signed_descriptor_get_body: it might * not be NUL-terminated. */ static char * -munge_extrainfo_into_routerinfo(const char *ri_body, signed_descriptor_t *ri, - signed_descriptor_t *ei) +munge_extrainfo_into_routerinfo(const char *ri_body, + const signed_descriptor_t *ri, + const signed_descriptor_t *ei) { char *out = NULL, *outp; int i; @@ -1524,9 +1582,8 @@ getinfo_helper_listeners(control_connection_t *control_conn, else return 0; /* unknown key */ - res = smartlist_create(); + res = smartlist_new(); SMARTLIST_FOREACH_BEGIN(get_connection_array(), connection_t *, conn) { - char *addr; struct sockaddr_storage ss; socklen_t ss_len = sizeof(ss); @@ -1534,14 +1591,13 @@ getinfo_helper_listeners(control_connection_t *control_conn, continue; if (getsockname(conn->s, (struct sockaddr *)&ss, &ss_len) < 0) { - tor_asprintf(&addr, "%s:%d", conn->address, (int)conn->port); + smartlist_add_asprintf(res, "%s:%d", conn->address, (int)conn->port); } else { char *tmp = tor_sockaddr_to_str((struct sockaddr *)&ss); - addr = esc_for_log(tmp); + smartlist_add(res, esc_for_log(tmp)); tor_free(tmp); } - if (addr) - smartlist_add(res, addr); + } SMARTLIST_FOREACH_END(conn); *answer = smartlist_join_strings(res, " ", 0, NULL); @@ -1558,16 +1614,19 @@ getinfo_helper_dir(control_connection_t *control_conn, const char *question, char **answer, const char **errmsg) { + const routerinfo_t *ri; (void) control_conn; if (!strcmpstart(question, "desc/id/")) { - routerinfo_t *ri = router_get_by_hexdigest(question+strlen("desc/id/")); + ri = router_get_by_hexdigest(question+strlen("desc/id/")); if (ri) { const char *body = signed_descriptor_get_body(&ri->cache_info); if (body) *answer = tor_strndup(body, ri->cache_info.signed_descriptor_len); } } else if (!strcmpstart(question, "desc/name/")) { - routerinfo_t *ri = router_get_by_nickname(question+strlen("desc/name/"),1); + /* XXX023 Setting 'warn_if_unnamed' here is a bit silly -- the + * warning goes to the user, not to the controller. */ + ri = router_get_by_nickname(question+strlen("desc/name/"),1); if (ri) { const char *body = signed_descriptor_get_body(&ri->cache_info); if (body) @@ -1575,9 +1634,9 @@ getinfo_helper_dir(control_connection_t *control_conn, } } else if (!strcmp(question, "desc/all-recent")) { routerlist_t *routerlist = router_get_routerlist(); - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); if (routerlist && routerlist->routers) { - SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, ri, + SMARTLIST_FOREACH(routerlist->routers, const routerinfo_t *, ri, { const char *body = signed_descriptor_get_body(&ri->cache_info); if (body) @@ -1591,10 +1650,9 @@ getinfo_helper_dir(control_connection_t *control_conn, } else if (!strcmp(question, "desc/all-recent-extrainfo-hack")) { /* XXXX Remove this once Torstat asks for extrainfos. */ routerlist_t *routerlist = router_get_routerlist(); - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); if (routerlist && routerlist->routers) { - SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, ri, - { + SMARTLIST_FOREACH_BEGIN(routerlist->routers, const routerinfo_t *, ri) { const char *body = signed_descriptor_get_body(&ri->cache_info); signed_descriptor_t *ei = extrainfo_get_by_descriptor_digest( ri->cache_info.extra_info_digest); @@ -1605,14 +1663,33 @@ getinfo_helper_dir(control_connection_t *control_conn, smartlist_add(sl, tor_strndup(body, ri->cache_info.signed_descriptor_len)); } - }); + } SMARTLIST_FOREACH_END(ri); } *answer = smartlist_join_strings(sl, "", 0, NULL); SMARTLIST_FOREACH(sl, char *, c, tor_free(c)); smartlist_free(sl); + } else if (!strcmpstart(question, "md/id/")) { + const node_t *node = node_get_by_hex_id(question+strlen("md/id/")); + const microdesc_t *md = NULL; + if (node) md = node->md; + if (md) { + tor_assert(md->body); + *answer = tor_strndup(md->body, md->bodylen); + } + } else if (!strcmpstart(question, "md/name/")) { + /* XXX023 Setting 'warn_if_unnamed' here is a bit silly -- the + * warning goes to the user, not to the controller. */ + const node_t *node = node_get_by_nickname(question+strlen("md/name/"), 1); + /* XXXX duplicated code */ + const microdesc_t *md = NULL; + if (node) md = node->md; + if (md) { + tor_assert(md->body); + *answer = tor_strndup(md->body, md->bodylen); + } } else if (!strcmpstart(question, "desc-annotations/id/")) { - routerinfo_t *ri = router_get_by_hexdigest(question+ - strlen("desc-annotations/id/")); + ri = router_get_by_hexdigest(question+ + strlen("desc-annotations/id/")); if (ri) { const char *annotations = signed_descriptor_get_annotations(&ri->cache_info); @@ -1621,13 +1698,13 @@ getinfo_helper_dir(control_connection_t *control_conn, ri->cache_info.annotations_len); } } else if (!strcmpstart(question, "dir/server/")) { - size_t answer_len = 0, url_len = strlen(question)+2; - char *url = tor_malloc(url_len); - smartlist_t *descs = smartlist_create(); + size_t answer_len = 0; + char *url = NULL; + smartlist_t *descs = smartlist_new(); const char *msg; int res; char *cp; - tor_snprintf(url, url_len, "/tor/%s", question+4); + tor_asprintf(&url, "/tor/%s", question+4); res = dirserv_get_routerdescs(descs, url, &msg); if (res) { log_warn(LD_CONTROL, "getinfo '%s': %s", question, msg); @@ -1652,7 +1729,7 @@ getinfo_helper_dir(control_connection_t *control_conn, if (directory_permits_controller_requests(get_options())) { size_t len=0; char *cp; - smartlist_t *status_list = smartlist_create(); + smartlist_t *status_list = smartlist_new(); dirserv_get_networkstatus_v2(status_list, question+strlen("dir/status/")); SMARTLIST_FOREACH(status_list, cached_dir_t *, d, len += d->dir_len); @@ -1664,8 +1741,8 @@ getinfo_helper_dir(control_connection_t *control_conn, *cp = '\0'; smartlist_free(status_list); } else { - smartlist_t *fp_list = smartlist_create(); - smartlist_t *status_list = smartlist_create(); + smartlist_t *fp_list = smartlist_new(); + smartlist_t *status_list = smartlist_new(); dirserv_get_networkstatus_v2_fingerprints( fp_list, question+strlen("dir/status/")); SMARTLIST_FOREACH(fp_list, const char *, fp, { @@ -1726,6 +1803,81 @@ getinfo_helper_dir(control_connection_t *control_conn, return 0; } +/** Allocate and return a description of <b>circ</b>'s current status, + * including its path (if any). */ +static char * +circuit_describe_status_for_controller(origin_circuit_t *circ) +{ + char *rv; + smartlist_t *descparts = smartlist_new(); + + { + char *vpath = circuit_list_path_for_controller(circ); + if (*vpath) { + smartlist_add(descparts, vpath); + } else { + tor_free(vpath); /* empty path; don't put an extra space in the result */ + } + } + + { + cpath_build_state_t *build_state = circ->build_state; + smartlist_t *flaglist = smartlist_new(); + char *flaglist_joined; + + if (build_state->onehop_tunnel) + smartlist_add(flaglist, (void *)"ONEHOP_TUNNEL"); + if (build_state->is_internal) + smartlist_add(flaglist, (void *)"IS_INTERNAL"); + if (build_state->need_capacity) + smartlist_add(flaglist, (void *)"NEED_CAPACITY"); + if (build_state->need_uptime) + smartlist_add(flaglist, (void *)"NEED_UPTIME"); + + /* Only emit a BUILD_FLAGS argument if it will have a non-empty value. */ + if (smartlist_len(flaglist)) { + flaglist_joined = smartlist_join_strings(flaglist, ",", 0, NULL); + + smartlist_add_asprintf(descparts, "BUILD_FLAGS=%s", flaglist_joined); + + tor_free(flaglist_joined); + } + + smartlist_free(flaglist); + } + + smartlist_add_asprintf(descparts, "PURPOSE=%s", + circuit_purpose_to_controller_string(circ->_base.purpose)); + + { + const char *hs_state = + circuit_purpose_to_controller_hs_state_string(circ->_base.purpose); + + if (hs_state != NULL) { + smartlist_add_asprintf(descparts, "HS_STATE=%s", hs_state); + } + } + + if (circ->rend_data != NULL) { + smartlist_add_asprintf(descparts, "REND_QUERY=%s", + circ->rend_data->onion_address); + } + + { + char tbuf[ISO_TIME_USEC_LEN+1]; + format_iso_time_nospace_usec(tbuf, &circ->_base.timestamp_created); + + smartlist_add_asprintf(descparts, "TIME_CREATED=%s", tbuf); + } + + rv = smartlist_join_strings(descparts, " ", 0, NULL); + + SMARTLIST_FOREACH(descparts, char *, cp, tor_free(cp)); + smartlist_free(descparts); + + return rv; +} + /** Implementation helper for GETINFO: knows how to generate summaries of the * current states of things we send events about. */ static int @@ -1735,46 +1887,40 @@ getinfo_helper_events(control_connection_t *control_conn, { (void) control_conn; if (!strcmp(question, "circuit-status")) { - circuit_t *circ; - smartlist_t *status = smartlist_create(); - for (circ = _circuit_get_global_list(); circ; circ = circ->next) { - char *s, *path; - size_t slen; + circuit_t *circ_; + smartlist_t *status = smartlist_new(); + for (circ_ = _circuit_get_global_list(); circ_; circ_ = circ_->next) { + origin_circuit_t *circ; + char *circdesc; const char *state; - const char *purpose; - if (! CIRCUIT_IS_ORIGIN(circ) || circ->marked_for_close) + if (! CIRCUIT_IS_ORIGIN(circ_) || circ_->marked_for_close) continue; + circ = TO_ORIGIN_CIRCUIT(circ_); - path = circuit_list_path_for_controller(TO_ORIGIN_CIRCUIT(circ)); - - if (circ->state == CIRCUIT_STATE_OPEN) + if (circ->_base.state == CIRCUIT_STATE_OPEN) state = "BUILT"; - else if (strlen(path)) + else if (circ->cpath) state = "EXTENDED"; else state = "LAUNCHED"; - purpose = circuit_purpose_to_controller_string(circ->purpose); - slen = strlen(path)+strlen(state)+strlen(purpose)+30; - s = tor_malloc(slen+1); - tor_snprintf(s, slen, "%lu %s%s%s PURPOSE=%s", - (unsigned long)TO_ORIGIN_CIRCUIT(circ)->global_identifier, - state, *path ? " " : "", path, purpose); - smartlist_add(status, s); - tor_free(path); + circdesc = circuit_describe_status_for_controller(circ); + + smartlist_add_asprintf(status, "%lu %s%s%s", + (unsigned long)circ->global_identifier, + state, *circdesc ? " " : "", circdesc); + tor_free(circdesc); } *answer = smartlist_join_strings(status, "\r\n", 0, NULL); SMARTLIST_FOREACH(status, char *, cp, tor_free(cp)); smartlist_free(status); } else if (!strcmp(question, "stream-status")) { smartlist_t *conns = get_connection_array(); - smartlist_t *status = smartlist_create(); + smartlist_t *status = smartlist_new(); char buf[256]; SMARTLIST_FOREACH_BEGIN(conns, connection_t *, base_conn) { const char *state; - edge_connection_t *conn; - char *s; - size_t slen; + entry_connection_t *conn; circuit_t *circ; origin_circuit_t *origin_circ = NULL; if (base_conn->type != CONN_TYPE_AP || @@ -1782,8 +1928,8 @@ getinfo_helper_events(control_connection_t *control_conn, base_conn->state == AP_CONN_STATE_SOCKS_WAIT || base_conn->state == AP_CONN_STATE_NATD_WAIT) continue; - conn = TO_EDGE_CONN(base_conn); - switch (conn->_base.state) + conn = TO_ENTRY_CONN(base_conn); + switch (base_conn->state) { case AP_CONN_STATE_CONTROLLER_WAIT: case AP_CONN_STATE_CIRCUIT_WAIT: @@ -1802,33 +1948,28 @@ getinfo_helper_events(control_connection_t *control_conn, state = "SUCCEEDED"; break; default: log_warn(LD_BUG, "Asked for stream in unknown state %d", - conn->_base.state); + base_conn->state); continue; } - circ = circuit_get_by_edge_conn(conn); + circ = circuit_get_by_edge_conn(ENTRY_TO_EDGE_CONN(conn)); if (circ && CIRCUIT_IS_ORIGIN(circ)) origin_circ = TO_ORIGIN_CIRCUIT(circ); write_stream_target_to_buf(conn, buf, sizeof(buf)); - slen = strlen(buf)+strlen(state)+32; - s = tor_malloc(slen+1); - tor_snprintf(s, slen, "%lu %s %lu %s", - (unsigned long) conn->_base.global_identifier,state, + smartlist_add_asprintf(status, "%lu %s %lu %s", + (unsigned long) base_conn->global_identifier,state, origin_circ? (unsigned long)origin_circ->global_identifier : 0ul, buf); - smartlist_add(status, s); } SMARTLIST_FOREACH_END(base_conn); *answer = smartlist_join_strings(status, "\r\n", 0, NULL); SMARTLIST_FOREACH(status, char *, cp, tor_free(cp)); smartlist_free(status); } else if (!strcmp(question, "orconn-status")) { smartlist_t *conns = get_connection_array(); - smartlist_t *status = smartlist_create(); + smartlist_t *status = smartlist_new(); SMARTLIST_FOREACH_BEGIN(conns, connection_t *, base_conn) { const char *state; - char *s; char name[128]; - size_t slen; or_connection_t *conn; if (base_conn->type != CONN_TYPE_OR || base_conn->marked_for_close) continue; @@ -1840,10 +1981,7 @@ getinfo_helper_events(control_connection_t *control_conn, else state = "NEW"; orconn_target_get_name(name, sizeof(name), conn); - slen = strlen(name)+strlen(state)+2; - s = tor_malloc(slen+1); - tor_snprintf(s, slen, "%s %s", name, state); - smartlist_add(status, s); + smartlist_add_asprintf(status, "%s %s", name, state); } SMARTLIST_FOREACH_END(base_conn); *answer = smartlist_join_strings(status, "\r\n", 0, NULL); SMARTLIST_FOREACH(status, char *, cp, tor_free(cp)); @@ -1863,7 +2001,7 @@ getinfo_helper_events(control_connection_t *control_conn, } else { return 0; } - mappings = smartlist_create(); + mappings = smartlist_new(); addressmap_get_mappings(mappings, min_e, max_e, 1); *answer = smartlist_join_strings(mappings, "\r\n", 0, NULL); SMARTLIST_FOREACH(mappings, char *, cp, tor_free(cp)); @@ -1887,8 +2025,7 @@ getinfo_helper_events(control_connection_t *control_conn, } else if (!strcmp(question, "status/reachability-succeeded/dir")) { *answer = tor_strdup(check_whether_dirport_reachable() ? "1" : "0"); } else if (!strcmp(question, "status/reachability-succeeded")) { - *answer = tor_malloc(16); - tor_snprintf(*answer, 16, "OR=%d DIR=%d", + tor_asprintf(answer, "OR=%d DIR=%d", check_whether_orport_reachable() ? 1 : 0, check_whether_dirport_reachable() ? 1 : 0); } else if (!strcmp(question, "status/bootstrap-phase")) { @@ -1924,9 +2061,7 @@ getinfo_helper_events(control_connection_t *control_conn, } } else if (!strcmp(question, "status/version/num-versioning") || !strcmp(question, "status/version/num-concurring")) { - char s[33]; - tor_snprintf(s, sizeof(s), "%d", get_n_authorities(V3_AUTHORITY)); - *answer = tor_strdup(s); + tor_asprintf(answer, "%d", get_n_authorities(V3_DIRINFO)); log_warn(LD_GENERAL, "%s is deprecated; it no longer gives useful " "information", question); } @@ -1973,6 +2108,7 @@ typedef struct getinfo_item_t { static const getinfo_item_t getinfo_items[] = { ITEM("version", misc, "The current version of Tor."), ITEM("config-file", misc, "Current location of the \"torrc\" file."), + ITEM("config-defaults-file", misc, "Current location of the defaults file."), ITEM("config-text", misc, "Return the string that would be written by a saveconf command."), ITEM("accounting/bytes", accounting, @@ -2004,6 +2140,8 @@ static const getinfo_item_t getinfo_items[] = { ITEM("desc/all-recent", dir, "All non-expired, non-superseded router descriptors."), ITEM("desc/all-recent-extrainfo-hack", dir, NULL), /* Hack. */ + PREFIX("md/id/", dir, "Microdescriptors by ID"), + PREFIX("md/name/", dir, "Microdescriptors by name"), PREFIX("extra-info/digest/", dir, "Extra-info documents by digest."), PREFIX("net/listeners/", listeners, "Bound addresses by type"), ITEM("ns/all", networkstatus, @@ -2014,12 +2152,13 @@ static const getinfo_item_t getinfo_items[] = { "Brief summary of router status by nickname (v2 directory format)."), PREFIX("ns/purpose/", networkstatus, "Brief summary of router status by purpose (v2 directory format)."), - ITEM("network-status", dir, "Brief summary of router status (v1 directory format)"), ITEM("circuit-status", events, "List of current circuits originating here."), ITEM("stream-status", events,"List of current streams."), ITEM("orconn-status", events, "A list of current OR connections."), + ITEM("dormant", misc, + "Is Tor dormant (not building circuits because it's idle)?"), PREFIX("address-mappings/", events, NULL), DOC("address-mappings/all", "Current address mappings."), DOC("address-mappings/cache", "Current cached DNS replies."), @@ -2043,6 +2182,14 @@ static const getinfo_item_t getinfo_items[] = { "Number of versioning authorities agreeing on the status of the " "current version"), ITEM("address", misc, "IP address of this Tor host, if we can guess it."), + ITEM("traffic/read", misc,"Bytes read since the process was started."), + ITEM("traffic/written", misc, + "Bytes written since the process was started."), + ITEM("process/pid", misc, "Process id belonging to the main tor process."), + ITEM("process/uid", misc, "User id running the tor process."), + ITEM("process/user", misc, + "Username under which the tor process is running."), + ITEM("process/descriptor-limit", misc, "File descriptor limit."), ITEM("dir-usage", misc, "Breakdown of bytes transferred over DirPort."), PREFIX("desc-annotations/id/", dir, "Router annotations by hexdigest."), PREFIX("dir/server/", dir,"Router descriptors as retrieved from a DirPort."), @@ -2061,18 +2208,16 @@ static char * list_getinfo_options(void) { int i; - char *buf=NULL; - smartlist_t *lines = smartlist_create(); + smartlist_t *lines = smartlist_new(); char *ans; for (i = 0; getinfo_items[i].varname; ++i) { if (!getinfo_items[i].desc) continue; - tor_asprintf(&buf, "%s%s -- %s\n", + smartlist_add_asprintf(lines, "%s%s -- %s\n", getinfo_items[i].varname, getinfo_items[i].is_prefix ? "*" : "", getinfo_items[i].desc); - smartlist_add(lines, buf); } smartlist_sort_strings(lines); @@ -2116,9 +2261,9 @@ static int handle_control_getinfo(control_connection_t *conn, uint32_t len, const char *body) { - smartlist_t *questions = smartlist_create(); - smartlist_t *answers = smartlist_create(); - smartlist_t *unrecognized = smartlist_create(); + smartlist_t *questions = smartlist_new(); + smartlist_t *answers = smartlist_new(); + smartlist_t *unrecognized = smartlist_new(); char *msg = NULL, *ans = NULL; int i; (void) len; /* body is NUL-terminated, so it's safe to ignore the length. */ @@ -2204,7 +2349,7 @@ static smartlist_t * getargs_helper(const char *command, control_connection_t *conn, const char *body, int min_args, int max_args) { - smartlist_t *args = smartlist_create(); + smartlist_t *args = smartlist_new(); smartlist_split_string(args, body, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); if (smartlist_len(args) < min_args) { @@ -2252,14 +2397,14 @@ static int handle_control_extendcircuit(control_connection_t *conn, uint32_t len, const char *body) { - smartlist_t *router_nicknames=NULL, *routers=NULL; + smartlist_t *router_nicknames=NULL, *nodes=NULL; origin_circuit_t *circ = NULL; int zero_circ; uint8_t intended_purpose = CIRCUIT_PURPOSE_C_GENERAL; smartlist_t *args; (void) len; - router_nicknames = smartlist_create(); + router_nicknames = smartlist_new(); args = getargs_helper("EXTENDCIRCUIT", conn, body, 1, -1); if (!args) @@ -2283,8 +2428,7 @@ handle_control_extendcircuit(control_connection_t *conn, uint32_t len, if ((smartlist_len(args) == 1) || (smartlist_len(args) >= 2 && is_keyval_pair(smartlist_get(args, 1)))) { // "EXTENDCIRCUIT 0" || EXTENDCIRCUIT 0 foo=bar" - circ = circuit_launch_by_router(intended_purpose, NULL, - CIRCLAUNCH_NEED_CAPACITY); + circ = circuit_launch(intended_purpose, CIRCLAUNCH_NEED_CAPACITY); if (!circ) { connection_write_str_to_buf("551 Couldn't start circuit\r\n", conn); } else { @@ -2312,17 +2456,20 @@ handle_control_extendcircuit(control_connection_t *conn, uint32_t len, SMARTLIST_FOREACH(args, char *, cp, tor_free(cp)); smartlist_free(args); - routers = smartlist_create(); - SMARTLIST_FOREACH(router_nicknames, const char *, n, - { - routerinfo_t *r = router_get_by_nickname(n, 1); - if (!r) { + nodes = smartlist_new(); + SMARTLIST_FOREACH_BEGIN(router_nicknames, const char *, n) { + const node_t *node = node_get_by_nickname(n, 1); + if (!node) { connection_printf_to_buf(conn, "552 No such router \"%s\"\r\n", n); goto done; } - smartlist_add(routers, r); - }); - if (!smartlist_len(routers)) { + if (!node_has_descriptor(node)) { + connection_printf_to_buf(conn, "552 descriptor for \"%s\"\r\n", n); + goto done; + } + smartlist_add(nodes, (void*)node); + } SMARTLIST_FOREACH_END(n); + if (!smartlist_len(nodes)) { connection_write_str_to_buf("512 No router names provided\r\n", conn); goto done; } @@ -2333,9 +2480,10 @@ handle_control_extendcircuit(control_connection_t *conn, uint32_t len, } /* now circ refers to something that is ready to be extended */ - SMARTLIST_FOREACH(routers, routerinfo_t *, r, + SMARTLIST_FOREACH(nodes, const node_t *, node, { - extend_info_t *info = extend_info_from_router(r); + extend_info_t *info = extend_info_from_node(node, 0); + tor_assert(info); /* True, since node_has_descriptor(node) == true */ circuit_append_new_exit(circ, info); extend_info_free(info); }); @@ -2369,7 +2517,7 @@ handle_control_extendcircuit(control_connection_t *conn, uint32_t len, done: SMARTLIST_FOREACH(router_nicknames, char *, n, tor_free(n)); smartlist_free(router_nicknames); - smartlist_free(routers); + smartlist_free(nodes); return 0; } @@ -2407,7 +2555,7 @@ handle_control_setcircuitpurpose(control_connection_t *conn, } } - circ->_base.purpose = new_purpose; + circuit_change_purpose(TO_CIRCUIT(circ), new_purpose); connection_write_str_to_buf("250 OK\r\n", conn); done: @@ -2424,7 +2572,7 @@ static int handle_control_attachstream(control_connection_t *conn, uint32_t len, const char *body) { - edge_connection_t *ap_conn = NULL; + entry_connection_t *ap_conn = NULL; origin_circuit_t *circ = NULL; int zero_circ; smartlist_t *args; @@ -2460,9 +2608,9 @@ handle_control_attachstream(control_connection_t *conn, uint32_t len, if (!ap_conn || (!zero_circ && !circ) || !hop_line_ok) return 0; - if (ap_conn->_base.state != AP_CONN_STATE_CONTROLLER_WAIT && - ap_conn->_base.state != AP_CONN_STATE_CONNECT_WAIT && - ap_conn->_base.state != AP_CONN_STATE_RESOLVE_WAIT) { + if (ENTRY_TO_CONN(ap_conn)->state != AP_CONN_STATE_CONTROLLER_WAIT && + ENTRY_TO_CONN(ap_conn)->state != AP_CONN_STATE_CONNECT_WAIT && + ENTRY_TO_CONN(ap_conn)->state != AP_CONN_STATE_RESOLVE_WAIT) { connection_write_str_to_buf( "555 Connection is not managed by controller.\r\n", conn); @@ -2470,15 +2618,16 @@ handle_control_attachstream(control_connection_t *conn, uint32_t len, } /* Do we need to detach it first? */ - if (ap_conn->_base.state != AP_CONN_STATE_CONTROLLER_WAIT) { - circuit_t *tmpcirc = circuit_get_by_edge_conn(ap_conn); - connection_edge_end(ap_conn, END_STREAM_REASON_TIMEOUT); + if (ENTRY_TO_CONN(ap_conn)->state != AP_CONN_STATE_CONTROLLER_WAIT) { + edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(ap_conn); + circuit_t *tmpcirc = circuit_get_by_edge_conn(edge_conn); + connection_edge_end(edge_conn, END_STREAM_REASON_TIMEOUT); /* Un-mark it as ending, since we're going to reuse it. */ - ap_conn->edge_has_sent_end = 0; - ap_conn->end_reason = 0; + edge_conn->edge_has_sent_end = 0; + edge_conn->end_reason = 0; if (tmpcirc) - circuit_detach_stream(tmpcirc,ap_conn); - ap_conn->_base.state = AP_CONN_STATE_CONTROLLER_WAIT; + circuit_detach_stream(tmpcirc, edge_conn); + TO_CONN(edge_conn)->state = AP_CONN_STATE_CONTROLLER_WAIT; } if (circ && (circ->_base.state != CIRCUIT_STATE_OPEN)) { @@ -2489,16 +2638,17 @@ handle_control_attachstream(control_connection_t *conn, uint32_t len, } /* Is this a single hop circuit? */ if (circ && (circuit_get_cpath_len(circ)<2 || hop==1)) { - routerinfo_t *r = NULL; - char* exit_digest; + const node_t *node = NULL; + char *exit_digest; if (circ->build_state && circ->build_state->chosen_exit && !tor_digest_is_zero(circ->build_state->chosen_exit->identity_digest)) { exit_digest = circ->build_state->chosen_exit->identity_digest; - r = router_get_by_digest(exit_digest); + node = node_get_by_id(exit_digest); } /* Do both the client and relay allow one-hop exit circuits? */ - if (!r || !r->allow_single_hop_exits || + if (!node || + !node_allows_single_hop_exits(node) || !get_options()->AllowSingleHopCircuits) { connection_write_str_to_buf( "551 Can't attach stream to this one-hop circuit.\r\n", conn); @@ -2536,14 +2686,13 @@ handle_control_postdescriptor(control_connection_t *conn, uint32_t len, int cache = 0; /* eventually, we may switch this to 1 */ char *cp = memchr(body, '\n', len); - smartlist_t *args = smartlist_create(); + smartlist_t *args = smartlist_new(); tor_assert(cp); *cp++ = '\0'; smartlist_split_string(args, body, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - SMARTLIST_FOREACH(args, char *, option, - { + SMARTLIST_FOREACH_BEGIN(args, char *, option) { if (!strcasecmpstart(option, "purpose=")) { option += strlen("purpose="); purpose = router_purpose_from_string(option); @@ -2568,7 +2717,7 @@ handle_control_postdescriptor(control_connection_t *conn, uint32_t len, "512 Unexpected argument \"%s\" to postdescriptor\r\n", option); goto done; } - }); + } SMARTLIST_FOREACH_END(option); read_escaped_data(cp, len-(cp-body), &desc); @@ -2599,7 +2748,7 @@ static int handle_control_redirectstream(control_connection_t *conn, uint32_t len, const char *body) { - edge_connection_t *ap_conn = NULL; + entry_connection_t *ap_conn = NULL; char *new_addr = NULL; uint16_t new_port = 0; smartlist_t *args; @@ -2647,7 +2796,7 @@ static int handle_control_closestream(control_connection_t *conn, uint32_t len, const char *body) { - edge_connection_t *ap_conn=NULL; + entry_connection_t *ap_conn=NULL; uint8_t reason=0; smartlist_t *args; int ok; @@ -2735,7 +2884,7 @@ handle_control_resolve(control_connection_t *conn, uint32_t len, "isn't listening for ADDRMAP events. It probably won't see " "the answer."); } - args = smartlist_create(); + args = smartlist_new(); smartlist_split_string(args, body, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); { @@ -2743,7 +2892,7 @@ handle_control_resolve(control_connection_t *conn, uint32_t len, if (modearg && !strcasecmp(modearg, "mode=reverse")) is_reverse = 1; } - failed = smartlist_create(); + failed = smartlist_new(); SMARTLIST_FOREACH(args, const char *, arg, { if (!is_keyval_pair(arg)) { if (dnsserv_launch_request(arg, is_reverse)<0) @@ -2773,7 +2922,7 @@ handle_control_protocolinfo(control_connection_t *conn, uint32_t len, (void)len; conn->have_sent_protocolinfo = 1; - args = smartlist_create(); + args = smartlist_new(); smartlist_split_string(args, body, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); SMARTLIST_FOREACH(args, const char *, arg, { @@ -2792,15 +2941,18 @@ handle_control_protocolinfo(control_connection_t *conn, uint32_t len, connection_mark_for_close(TO_CONN(conn)); goto done; } else { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int cookies = options->CookieAuthentication; char *cfile = get_cookie_file(); - char *esc_cfile = esc_for_log(cfile); + char *abs_cfile; + char *esc_cfile; char *methods; + abs_cfile = make_path_absolute(cfile); + esc_cfile = esc_for_log(abs_cfile); { int passwd = (options->HashedControlPassword != NULL || options->HashedControlSessionPassword != NULL); - smartlist_t *mlist = smartlist_create(); + smartlist_t *mlist = smartlist_new(); if (cookies) { smartlist_add(mlist, (char*)"COOKIE"); smartlist_add(mlist, (char*)"SAFECOOKIE"); @@ -2824,6 +2976,7 @@ handle_control_protocolinfo(control_connection_t *conn, uint32_t len, escaped(VERSION)); tor_free(methods); tor_free(cfile); + tor_free(abs_cfile); tor_free(esc_cfile); } done: @@ -2885,6 +3038,7 @@ handle_control_authchallenge(control_connection_t *conn, uint32_t len, connection_write_str_to_buf("513 Invalid base16 client nonce\r\n", conn); connection_mark_for_close(TO_CONN(conn)); + tor_free(client_nonce); return -1; } @@ -2958,10 +3112,10 @@ handle_control_usefeature(control_connection_t *conn, smartlist_t *args; int bad = 0; (void) len; /* body is nul-terminated; it's safe to ignore the length */ - args = smartlist_create(); + args = smartlist_new(); smartlist_split_string(args, body, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - SMARTLIST_FOREACH(args, const char *, arg, { + SMARTLIST_FOREACH_BEGIN(args, const char *, arg) { if (!strcasecmp(arg, "VERBOSE_NAMES")) ; else if (!strcasecmp(arg, "EXTENDED_EVENTS")) @@ -2972,7 +3126,7 @@ handle_control_usefeature(control_connection_t *conn, bad = 1; break; } - }); + } SMARTLIST_FOREACH_END(arg); if (!bad) { send_control_done(conn); @@ -2988,8 +3142,6 @@ int connection_control_finished_flushing(control_connection_t *conn) { tor_assert(conn); - - connection_stop_writing(TO_CONN(conn)); return 0; } @@ -3064,6 +3216,21 @@ is_valid_initial_command(control_connection_t *conn, const char *cmd) * interfaces is broken. */ #define MAX_COMMAND_LINE_LENGTH (1024*1024) +/** Wrapper around peek_(evbuffer|buf)_has_control0 command: presents the same + * interface as those underlying functions, but takes a connection_t intead of + * an evbuffer or a buf_t. + */ +static int +peek_connection_has_control0_command(connection_t *conn) +{ + IF_HAS_BUFFEREVENT(conn, { + struct evbuffer *input = bufferevent_get_input(conn->bufev); + return peek_evbuffer_has_control0_command(input); + }) ELSE_IF_NO_BUFFEREVENT { + return peek_buf_has_control0_command(conn->inbuf); + } +} + /** Called when data has arrived on a v1 control connection: Try to fetch * commands from conn->inbuf, and execute them. */ @@ -3086,7 +3253,7 @@ connection_control_process_inbuf(control_connection_t *conn) } if (conn->_base.state == CONTROL_CONN_STATE_NEEDAUTH && - peek_buf_has_control0_command(conn->_base.inbuf)) { + peek_connection_has_control0_command(TO_CONN(conn))) { /* Detect v0 commands and send a "no more v0" message. */ size_t body_len; char buf[128]; @@ -3098,8 +3265,8 @@ connection_control_process_inbuf(control_connection_t *conn) body_len = 2+strlen(buf+6)+2; /* code, msg, nul. */ set_uint16(buf+0, htons(body_len)); connection_write_to_buf(buf, 4+body_len, TO_CONN(conn)); - connection_mark_for_close(TO_CONN(conn)); - conn->_base.hold_open_until_flushed = 1; + + connection_mark_and_flush(TO_CONN(conn)); return 0; } @@ -3110,7 +3277,7 @@ connection_control_process_inbuf(control_connection_t *conn) /* First, fetch a line. */ do { data_len = conn->incoming_cmd_len - conn->incoming_cmd_cur_len; - r = fetch_from_buf_line(conn->_base.inbuf, + r = connection_fetch_from_buf_line(TO_CONN(conn), conn->incoming_cmd+conn->incoming_cmd_cur_len, &data_len); if (r == 0) @@ -3120,8 +3287,7 @@ connection_control_process_inbuf(control_connection_t *conn) if (data_len + conn->incoming_cmd_cur_len > MAX_COMMAND_LINE_LENGTH) { connection_write_str_to_buf("500 Line too long.\r\n", conn); connection_stop_reading(TO_CONN(conn)); - connection_mark_for_close(TO_CONN(conn)); - conn->_base.hold_open_until_flushed = 1; + connection_mark_and_flush(TO_CONN(conn)); } while (conn->incoming_cmd_len < data_len+conn->incoming_cmd_cur_len) conn->incoming_cmd_len *= 2; @@ -3168,7 +3334,7 @@ connection_control_process_inbuf(control_connection_t *conn) args = conn->incoming_cmd+cmd_len+1; tor_assert(data_len>(size_t)cmd_len); data_len -= (cmd_len+1); /* skip the command and NUL we added after it */ - while (*args == ' ' || *args == '\t') { + while (TOR_ISSPACE(*args)) { ++args; --data_len; } @@ -3181,8 +3347,7 @@ connection_control_process_inbuf(control_connection_t *conn) /* Otherwise, Quit is always valid. */ if (!strcasecmp(conn->incoming_cmd, "QUIT")) { connection_write_str_to_buf("250 closing connection\r\n", conn); - connection_mark_for_close(TO_CONN(conn)); - conn->_base.hold_open_until_flushed = 1; + connection_mark_and_flush(TO_CONN(conn)); return 0; } @@ -3280,14 +3445,14 @@ connection_control_process_inbuf(control_connection_t *conn) goto again; } -/** Something has happened to circuit <b>circ</b>: tell any interested - * control connections. */ +/** Something major has happened to circuit <b>circ</b>: tell any + * interested control connections. */ int control_event_circuit_status(origin_circuit_t *circ, circuit_status_event_t tp, int reason_code) { const char *status; - char extended_buf[96]; + char reasons[64] = ""; if (!EVENT_IS_INTERESTING(EVENT_CIRCUIT_STATUS)) return 0; tor_assert(circ); @@ -3301,50 +3466,147 @@ control_event_circuit_status(origin_circuit_t *circ, circuit_status_event_t tp, case CIRC_EVENT_CLOSED: status = "CLOSED"; break; default: log_warn(LD_BUG, "Unrecognized status code %d", (int)tp); + tor_fragile_assert(); return 0; } - tor_snprintf(extended_buf, sizeof(extended_buf), "PURPOSE=%s", - circuit_purpose_to_controller_string(circ->_base.purpose)); - if (tp == CIRC_EVENT_FAILED || tp == CIRC_EVENT_CLOSED) { const char *reason_str = circuit_end_reason_to_control_string(reason_code); - char *reason = NULL; - size_t n=strlen(extended_buf); + char unk_reason_buf[16]; if (!reason_str) { - reason = tor_malloc(16); - tor_snprintf(reason, 16, "UNKNOWN_%d", reason_code); - reason_str = reason; + tor_snprintf(unk_reason_buf, 16, "UNKNOWN_%d", reason_code); + reason_str = unk_reason_buf; } if (reason_code > 0 && reason_code & END_CIRC_REASON_FLAG_REMOTE) { - tor_snprintf(extended_buf+n, sizeof(extended_buf)-n, + tor_snprintf(reasons, sizeof(reasons), " REASON=DESTROYED REMOTE_REASON=%s", reason_str); } else { - tor_snprintf(extended_buf+n, sizeof(extended_buf)-n, + tor_snprintf(reasons, sizeof(reasons), " REASON=%s", reason_str); } - tor_free(reason); } { - char *vpath = circuit_list_path_for_controller(circ); - const char *sp = strlen(vpath) ? " " : ""; + char *circdesc = circuit_describe_status_for_controller(circ); + const char *sp = strlen(circdesc) ? " " : ""; send_control_event(EVENT_CIRCUIT_STATUS, ALL_FORMATS, - "650 CIRC %lu %s%s%s %s\r\n", + "650 CIRC %lu %s%s%s%s\r\n", (unsigned long)circ->global_identifier, - status, sp, vpath, extended_buf); - tor_free(vpath); + status, sp, + circdesc, + reasons); + tor_free(circdesc); + } + + return 0; +} + +/** Something minor has happened to circuit <b>circ</b>: tell any + * interested control connections. */ +static int +control_event_circuit_status_minor(origin_circuit_t *circ, + circuit_status_minor_event_t e, + int purpose, const struct timeval *tv) +{ + const char *event_desc; + char event_tail[160] = ""; + if (!EVENT_IS_INTERESTING(EVENT_CIRCUIT_STATUS_MINOR)) + return 0; + tor_assert(circ); + + switch (e) + { + case CIRC_MINOR_EVENT_PURPOSE_CHANGED: + event_desc = "PURPOSE_CHANGED"; + + { + /* event_tail can currently be up to 68 chars long */ + const char *hs_state_str = + circuit_purpose_to_controller_hs_state_string(purpose); + tor_snprintf(event_tail, sizeof(event_tail), + " OLD_PURPOSE=%s%s%s", + circuit_purpose_to_controller_string(purpose), + (hs_state_str != NULL) ? " OLD_HS_STATE=" : "", + (hs_state_str != NULL) ? hs_state_str : ""); + } + + break; + case CIRC_MINOR_EVENT_CANNIBALIZED: + event_desc = "CANNIBALIZED"; + + { + /* event_tail can currently be up to 130 chars long */ + const char *hs_state_str = + circuit_purpose_to_controller_hs_state_string(purpose); + const struct timeval *old_timestamp_created = tv; + char tbuf[ISO_TIME_USEC_LEN+1]; + format_iso_time_nospace_usec(tbuf, old_timestamp_created); + + tor_snprintf(event_tail, sizeof(event_tail), + " OLD_PURPOSE=%s%s%s OLD_TIME_CREATED=%s", + circuit_purpose_to_controller_string(purpose), + (hs_state_str != NULL) ? " OLD_HS_STATE=" : "", + (hs_state_str != NULL) ? hs_state_str : "", + tbuf); + } + + break; + default: + log_warn(LD_BUG, "Unrecognized status code %d", (int)e); + tor_fragile_assert(); + return 0; + } + + { + char *circdesc = circuit_describe_status_for_controller(circ); + const char *sp = strlen(circdesc) ? " " : ""; + send_control_event(EVENT_CIRCUIT_STATUS_MINOR, ALL_FORMATS, + "650 CIRC_MINOR %lu %s%s%s%s\r\n", + (unsigned long)circ->global_identifier, + event_desc, sp, + circdesc, + event_tail); + tor_free(circdesc); } return 0; } +/** + * <b>circ</b> has changed its purpose from <b>old_purpose</b>: tell any + * interested controllers. + */ +int +control_event_circuit_purpose_changed(origin_circuit_t *circ, + int old_purpose) +{ + return control_event_circuit_status_minor(circ, + CIRC_MINOR_EVENT_PURPOSE_CHANGED, + old_purpose, + NULL); +} + +/** + * <b>circ</b> has changed its purpose from <b>old_purpose</b>, and its + * created-time from <b>old_tv_created</b>: tell any interested controllers. + */ +int +control_event_circuit_cannibalized(origin_circuit_t *circ, + int old_purpose, + const struct timeval *old_tv_created) +{ + return control_event_circuit_status_minor(circ, + CIRC_MINOR_EVENT_CANNIBALIZED, + old_purpose, + old_tv_created); +} + /** Given an AP connection <b>conn</b> and a <b>len</b>-character buffer * <b>buf</b>, determine the address:port combination requested on * <b>conn</b>, and write it to <b>buf</b>. Return 0 on success, -1 on * failure. */ static int -write_stream_target_to_buf(edge_connection_t *conn, char *buf, size_t len) +write_stream_target_to_buf(entry_connection_t *conn, char *buf, size_t len) { char buf2[256]; if (conn->chosen_exit_name) @@ -3355,8 +3617,8 @@ write_stream_target_to_buf(edge_connection_t *conn, char *buf, size_t len) if (tor_snprintf(buf, len, "%s%s%s:%d", conn->socks_request->address, conn->chosen_exit_name ? buf2 : "", - !conn->chosen_exit_name && - connection_edge_is_rendezvous_stream(conn) ? ".onion" : "", + !conn->chosen_exit_name && connection_edge_is_rendezvous_stream( + ENTRY_TO_EDGE_CONN(conn)) ? ".onion" : "", conn->socks_request->port)<0) return -1; return 0; @@ -3365,7 +3627,7 @@ write_stream_target_to_buf(edge_connection_t *conn, char *buf, size_t len) /** Something has happened to the stream associated with AP connection * <b>conn</b>: tell any interested control connections. */ int -control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, +control_event_stream_status(entry_connection_t *conn, stream_status_event_t tp, int reason_code) { char reason_buf[64]; @@ -3408,8 +3670,7 @@ control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, const char *reason_str = stream_end_reason_to_control_string(reason_code); char *r = NULL; if (!reason_str) { - r = tor_malloc(16); - tor_snprintf(r, 16, " UNKNOWN_%d", reason_code); + tor_asprintf(&r, " UNKNOWN_%d", reason_code); reason_str = r; } if (reason_code & END_STREAM_REASON_FLAG_REMOTE) @@ -3437,7 +3698,7 @@ control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, if (tp == STREAM_EVENT_NEW) { tor_snprintf(addrport_buf,sizeof(addrport_buf), " SOURCE_ADDR=%s:%d", - TO_CONN(conn)->address, TO_CONN(conn)->port ); + ENTRY_TO_CONN(conn)->address, ENTRY_TO_CONN(conn)->port); } else { addrport_buf[0] = '\0'; } @@ -3445,12 +3706,12 @@ control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, if (tp == STREAM_EVENT_NEW_RESOLVE) { purpose = " PURPOSE=DNS_REQUEST"; } else if (tp == STREAM_EVENT_NEW) { - if (conn->is_dns_request || + if (ENTRY_TO_EDGE_CONN(conn)->is_dns_request || (conn->socks_request && SOCKS_COMMAND_IS_RESOLVE(conn->socks_request->command))) purpose = " PURPOSE=DNS_REQUEST"; else if (conn->use_begindir) { - connection_t *linked = TO_CONN(conn)->linked_conn; + connection_t *linked = ENTRY_TO_CONN(conn)->linked_conn; int linked_dir_purpose = -1; if (linked && linked->type == CONN_TYPE_DIR) linked_dir_purpose = linked->purpose; @@ -3462,12 +3723,13 @@ control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, purpose = " PURPOSE=USER"; } - circ = circuit_get_by_edge_conn(conn); + circ = circuit_get_by_edge_conn(ENTRY_TO_EDGE_CONN(conn)); if (circ && CIRCUIT_IS_ORIGIN(circ)) origin_circ = TO_ORIGIN_CIRCUIT(circ); send_control_event(EVENT_STREAM_STATUS, ALL_FORMATS, "650 STREAM "U64_FORMAT" %s %lu %s%s%s%s\r\n", - U64_PRINTF_ARG(conn->_base.global_identifier), status, + U64_PRINTF_ARG(ENTRY_TO_CONN(conn)->global_identifier), + status, origin_circ? (unsigned long)origin_circ->global_identifier : 0ul, buf, reason_buf, addrport_buf, purpose); @@ -3483,10 +3745,10 @@ control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, static void orconn_target_get_name(char *name, size_t len, or_connection_t *conn) { - routerinfo_t *ri = router_get_by_digest(conn->identity_digest); - if (ri) { + const node_t *node = node_get_by_id(conn->identity_digest); + if (node) { tor_assert(len > MAX_VERBOSE_NICKNAME_LEN); - router_get_verbose_nickname(name, ri); + node_get_verbose_nickname(node, name); } else if (! tor_digest_is_zero(conn->identity_digest)) { name[0] = '$'; base16_encode(name+1, len-1, conn->identity_digest, @@ -3691,18 +3953,15 @@ control_event_descriptors_changed(smartlist_t *routers) return 0; { - smartlist_t *names = smartlist_create(); + smartlist_t *names = smartlist_new(); char *ids; - size_t names_len; SMARTLIST_FOREACH(routers, routerinfo_t *, ri, { char *b = tor_malloc(MAX_VERBOSE_NICKNAME_LEN+1); router_get_verbose_nickname(b, ri); smartlist_add(names, b); }); - ids = smartlist_join_strings(names, " ", 0, &names_len); - names_len = strlen(ids)+32; - msg = tor_malloc(names_len); - tor_snprintf(msg, names_len, "650 NEWDESC %s\r\n", ids); + ids = smartlist_join_strings(names, " ", 0, NULL); + tor_asprintf(&msg, "650 NEWDESC %s\r\n", ids); send_control_event_string(EVENT_NEW_DESC, ALL_FORMATS, msg); tor_free(ids); tor_free(msg); @@ -3799,11 +4058,11 @@ control_event_networkstatus_changed_helper(smartlist_t *statuses, if (!EVENT_IS_INTERESTING(event) || !smartlist_len(statuses)) return 0; - strs = smartlist_create(); + strs = smartlist_new(); smartlist_add(strs, tor_strdup("650+")); smartlist_add(strs, tor_strdup(event_string)); smartlist_add(strs, tor_strdup("\r\n")); - SMARTLIST_FOREACH(statuses, routerstatus_t *, rs, + SMARTLIST_FOREACH(statuses, const routerstatus_t *, rs, { s = networkstatus_getinfo_helper_single(rs); if (!s) continue; @@ -3848,11 +4107,13 @@ control_event_buildtimeout_set(const circuit_build_times_t *cbt, buildtimeout_set_event_t type) { const char *type_string = NULL; - double qnt = circuit_build_times_quantile_cutoff(); + double qnt; if (!control_event_is_interesting(EVENT_BUILDTIMEOUT_SET)) return 0; + qnt = circuit_build_times_quantile_cutoff(); + switch (type) { case BUILDTIMEOUT_SET_EVENT_COMPUTED: type_string = "COMPUTED"; @@ -3891,10 +4152,46 @@ control_event_buildtimeout_set(const circuit_build_times_t *cbt, return 0; } +/** Called when a signal has been processed from signal_callback */ +int +control_event_signal(uintptr_t signal) +{ + const char *signal_string = NULL; + + if (!control_event_is_interesting(EVENT_SIGNAL)) + return 0; + + switch (signal) { + case SIGHUP: + signal_string = "RELOAD"; + break; + case SIGUSR1: + signal_string = "DUMP"; + break; + case SIGUSR2: + signal_string = "DEBUG"; + break; + case SIGNEWNYM: + signal_string = "NEWNYM"; + break; + case SIGCLEARDNSCACHE: + signal_string = "CLEARDNSCACHE"; + break; + default: + log_warn(LD_BUG, "Unrecognized signal %lu in control_event_signal", + (unsigned long)signal); + return -1; + } + + send_control_event(EVENT_SIGNAL, ALL_FORMATS, "650 SIGNAL %s\r\n", + signal_string); + return 0; +} + /** Called when a single local_routerstatus_t has changed: Sends an NS event * to any controller that cares. */ int -control_event_networkstatus_changed_single(routerstatus_t *rs) +control_event_networkstatus_changed_single(const routerstatus_t *rs) { smartlist_t *statuses; int r; @@ -3902,8 +4199,8 @@ control_event_networkstatus_changed_single(routerstatus_t *rs) if (!EVENT_IS_INTERESTING(EVENT_NS)) return 0; - statuses = smartlist_create(); - smartlist_add(statuses, rs); + statuses = smartlist_new(); + smartlist_add(statuses, (void*)rs); r = control_event_networkstatus_changed(statuses); smartlist_free(statuses); return r; @@ -4031,9 +4328,9 @@ control_event_guard(const char *nickname, const char *digest, { char buf[MAX_VERBOSE_NICKNAME_LEN+1]; - routerinfo_t *ri = router_get_by_digest(digest); - if (ri) { - router_get_verbose_nickname(buf, ri); + const node_t *node = node_get_by_id(digest); + if (node) { + node_get_verbose_nickname(node, buf); } else { tor_snprintf(buf, sizeof(buf), "$%s~%s", hbuf, nickname); } @@ -4043,12 +4340,45 @@ control_event_guard(const char *nickname, const char *digest, return 0; } +/** Called when a configuration option changes. This is generally triggered + * by SETCONF requests and RELOAD/SIGHUP signals. The <b>elements</b> is + * a smartlist_t containing (key, value, ...) pairs in sequence. + * <b>value</b> can be NULL. */ +int +control_event_conf_changed(smartlist_t *elements) +{ + int i; + char *result; + smartlist_t *lines; + if (!EVENT_IS_INTERESTING(EVENT_CONF_CHANGED) || + smartlist_len(elements) == 0) { + return 0; + } + lines = smartlist_new(); + for (i = 0; i < smartlist_len(elements); i += 2) { + char *k = smartlist_get(elements, i); + char *v = smartlist_get(elements, i+1); + if (v == NULL) { + smartlist_add_asprintf(lines, "650-%s", k); + } else { + smartlist_add_asprintf(lines, "650-%s=%s", k, v); + } + } + result = smartlist_join_strings(lines, "\r\n", 0, NULL); + send_control_event(EVENT_CONF_CHANGED, 0, + "650-CONF_CHANGED\r\n%s\r\n650 OK\r\n", result); + tor_free(result); + SMARTLIST_FOREACH(lines, char *, cp, tor_free(cp)); + smartlist_free(lines); + return 0; +} + /** Helper: Return a newly allocated string containing a path to the * file where we store our authentication cookie. */ static char * get_cookie_file(void) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (options->CookieAuthFile && strlen(options->CookieAuthFile)) { return tor_strdup(options->CookieAuthFile); } else { @@ -4084,7 +4414,7 @@ init_cookie_authentication(int enabled) tor_free(fname); return -1; } -#ifndef MS_WINDOWS +#ifndef _WIN32 if (get_options()->CookieAuthFileGroupReadable) { if (chmod(fname, 0640)) { log_warn(LD_FS,"Unable to make %s group-readable.", escaped(fname)); @@ -4316,6 +4646,7 @@ control_event_bootstrap_problem(const char *warn, int reason) const char *tag, *summary; char buf[BOOTSTRAP_MSG_LEN]; const char *recommendation = "ignore"; + int severity; /* bootstrap_percent must not be in "undefined" state here. */ tor_assert(status >= 0); @@ -4340,12 +4671,17 @@ control_event_bootstrap_problem(const char *warn, int reason) status--; /* find a recognized status string based on current progress */ status = bootstrap_percent; /* set status back to the actual number */ - log_fn(!strcmp(recommendation, "warn") ? LOG_WARN : LOG_INFO, + severity = !strcmp(recommendation, "warn") ? LOG_WARN : LOG_INFO; + + log_fn(severity, LD_CONTROL, "Problem bootstrapping. Stuck at %d%%: %s. (%s; %s; " "count %d; recommendation %s)", status, summary, warn, orconn_end_reason_to_control_string(reason), bootstrap_problems, recommendation); + + connection_or_report_broken_states(severity, LD_HANDSHAKE); + tor_snprintf(buf, sizeof(buf), "BOOTSTRAP PROGRESS=%d TAG=%s SUMMARY=\"%s\" WARNING=\"%s\" REASON=%s " "COUNT=%d RECOMMENDATION=%s", diff --git a/src/or/control.h b/src/or/control.h index ddea4cd548..f301ce91be 100644 --- a/src/or/control.h +++ b/src/or/control.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -37,7 +37,12 @@ int control_event_is_interesting(int event); int control_event_circuit_status(origin_circuit_t *circ, circuit_status_event_t e, int reason); -int control_event_stream_status(edge_connection_t *conn, +int control_event_circuit_purpose_changed(origin_circuit_t *circ, + int old_purpose); +int control_event_circuit_cannibalized(origin_circuit_t *circ, + int old_purpose, + const struct timeval *old_tv_created); +int control_event_stream_status(entry_connection_t *conn, stream_status_event_t e, int reason); int control_event_or_conn_status(or_connection_t *conn, @@ -45,7 +50,7 @@ int control_event_or_conn_status(or_connection_t *conn, int control_event_bandwidth_used(uint32_t n_read, uint32_t n_written); int control_event_stream_bandwidth(edge_connection_t *edge_conn); int control_event_stream_bandwidth_used(void); -void control_event_logmsg(int severity, unsigned int domain, const char *msg); +void control_event_logmsg(int severity, uint32_t domain, const char *msg); int control_event_descriptors_changed(smartlist_t *routers); int control_event_address_mapped(const char *from, const char *to, time_t expires, const char *error); @@ -57,7 +62,7 @@ int control_event_my_descriptor_changed(void); int control_event_networkstatus_changed(smartlist_t *statuses); int control_event_newconsensus(const networkstatus_t *consensus); -int control_event_networkstatus_changed_single(routerstatus_t *rs); +int control_event_networkstatus_changed_single(const routerstatus_t *rs); int control_event_general_status(int severity, const char *format, ...) CHECK_PRINTF(2,3); int control_event_client_status(int severity, const char *format, ...) @@ -66,8 +71,10 @@ int control_event_server_status(int severity, const char *format, ...) CHECK_PRINTF(2,3); int control_event_guard(const char *nickname, const char *digest, const char *status); +int control_event_conf_changed(smartlist_t *elements); int control_event_buildtimeout_set(const circuit_build_times_t *cbt, buildtimeout_set_event_t type); +int control_event_signal(uintptr_t signal); int init_cookie_authentication(int enabled); smartlist_t *decode_hashed_passwords(config_line_t *passwords); diff --git a/src/or/cpuworker.c b/src/or/cpuworker.c index 9d196d36eb..0255227e7b 100644 --- a/src/or/cpuworker.c +++ b/src/or/cpuworker.c @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -62,7 +62,6 @@ connection_cpu_finished_flushing(connection_t *conn) { tor_assert(conn); tor_assert(conn->type == CONN_TYPE_CPUWORKER); - connection_stop_writing(conn); return 0; } @@ -141,13 +140,13 @@ connection_cpu_process_inbuf(connection_t *conn) tor_assert(conn); tor_assert(conn->type == CONN_TYPE_CPUWORKER); - if (!buf_datalen(conn->inbuf)) + if (!connection_get_inbuf_len(conn)) return 0; if (conn->state == CPUWORKER_STATE_BUSY_ONION) { - if (buf_datalen(conn->inbuf) < LEN_ONION_RESPONSE) /* answer available? */ + if (connection_get_inbuf_len(conn) < LEN_ONION_RESPONSE) return 0; /* not yet */ - tor_assert(buf_datalen(conn->inbuf) == LEN_ONION_RESPONSE); + tor_assert(connection_get_inbuf_len(conn) == LEN_ONION_RESPONSE); connection_fetch_from_buf(&success,1,conn); connection_fetch_from_buf(buf,LEN_ONION_RESPONSE-1,conn); @@ -234,7 +233,7 @@ cpuworker_main(void *data) char reply_to_proxy[ONIONSKIN_REPLY_LEN]; char buf[LEN_ONION_RESPONSE]; char tag[TAG_LEN]; - crypto_pk_env_t *onion_key = NULL, *last_onion_key = NULL; + crypto_pk_t *onion_key = NULL, *last_onion_key = NULL; fd = fdarray[1]; /* this side is ours */ #ifndef TOR_IS_MULTITHREADED @@ -304,9 +303,9 @@ cpuworker_main(void *data) } end: if (onion_key) - crypto_free_pk_env(onion_key); + crypto_pk_free(onion_key); if (last_onion_key) - crypto_free_pk_env(last_onion_key); + crypto_pk_free(last_onion_key); tor_close_socket(fd); crypto_thread_cleanup(); spawn_exit(); @@ -330,8 +329,8 @@ spawn_cpuworker(void) return -1; } - tor_assert(fdarray[0] >= 0); - tor_assert(fdarray[1] >= 0); + tor_assert(SOCKET_OK(fdarray[0])); + tor_assert(SOCKET_OK(fdarray[1])); fd = fdarray[0]; spawn_func(cpuworker_main, (void*)fdarray); @@ -348,6 +347,7 @@ spawn_cpuworker(void) /* set up conn so it's got all the data we need to remember */ conn->s = fd; conn->address = tor_strdup("localhost"); + tor_addr_make_unspec(&conn->addr); if (connection_add(conn) < 0) { /* no space, forget it */ log_warn(LD_NET,"connection_add for cpuworker failed. Giving up."); @@ -367,7 +367,7 @@ spawn_cpuworker(void) static void spawn_enough_cpuworkers(void) { - int num_cpuworkers_needed = get_options()->NumCPUs; + int num_cpuworkers_needed = get_num_cpus(get_options()); if (num_cpuworkers_needed < MIN_CPUWORKERS) num_cpuworkers_needed = MIN_CPUWORKERS; @@ -417,8 +417,7 @@ cull_wedged_cpuworkers(void) { time_t now = time(NULL); smartlist_t *conns = get_connection_array(); - SMARTLIST_FOREACH(conns, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { if (!conn->marked_for_close && conn->type == CONN_TYPE_CPUWORKER && conn->state == CPUWORKER_STATE_BUSY_ONION && @@ -429,7 +428,7 @@ cull_wedged_cpuworkers(void) num_cpuworkers--; connection_mark_for_close(conn); } - }); + } SMARTLIST_FOREACH_END(conn); } /** Try to tell a cpuworker to perform the public key operations necessary to diff --git a/src/or/cpuworker.h b/src/or/cpuworker.h index 04e37ee459..91172caa56 100644 --- a/src/or/cpuworker.h +++ b/src/or/cpuworker.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** diff --git a/src/or/directory.c b/src/or/directory.c index 975f8910cd..f235bf3b41 100644 --- a/src/or/directory.c +++ b/src/or/directory.c @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #include "or.h" @@ -15,7 +15,9 @@ #include "dirvote.h" #include "geoip.h" #include "main.h" +#include "microdesc.h" #include "networkstatus.h" +#include "nodelist.h" #include "policies.h" #include "rendclient.h" #include "rendcommon.h" @@ -78,6 +80,8 @@ static void dir_routerdesc_download_failed(smartlist_t *failed, int router_purpose, int was_extrainfo, int was_descriptor_digests); +static void dir_microdesc_download_failed(smartlist_t *failed, + int status_code); static void note_client_request(int purpose, int compressed, size_t bytes); static int client_likes_consensus(networkstatus_t *v, const char *want_url); @@ -126,8 +130,9 @@ purpose_needs_anonymity(uint8_t dir_purpose, uint8_t router_purpose) { if (get_options()->AllDirActionsPrivate) return 1; - if (router_purpose == ROUTER_PURPOSE_BRIDGE && can_complete_circuit) - return 1; /* if no circuits yet, we may need this info to bootstrap. */ + if (router_purpose == ROUTER_PURPOSE_BRIDGE) + return 1; /* if no circuits yet, this might break bootstrapping, but it's + * needed to be safe. */ if (dir_purpose == DIR_PURPOSE_UPLOAD_DIR || dir_purpose == DIR_PURPOSE_UPLOAD_VOTE || dir_purpose == DIR_PURPOSE_UPLOAD_SIGNATURES || @@ -137,26 +142,28 @@ purpose_needs_anonymity(uint8_t dir_purpose, uint8_t router_purpose) dir_purpose == DIR_PURPOSE_FETCH_CONSENSUS || dir_purpose == DIR_PURPOSE_FETCH_CERTIFICATE || dir_purpose == DIR_PURPOSE_FETCH_SERVERDESC || - dir_purpose == DIR_PURPOSE_FETCH_EXTRAINFO) + dir_purpose == DIR_PURPOSE_FETCH_EXTRAINFO || + dir_purpose == DIR_PURPOSE_FETCH_MICRODESC) return 0; return 1; } -/** Return a newly allocated string describing <b>auth</b>. */ -char * -authority_type_to_string(authority_type_t auth) +/** Return a newly allocated string describing <b>auth</b>. Only describes + * authority features. */ +static char * +authdir_type_to_string(dirinfo_type_t auth) { char *result; - smartlist_t *lst = smartlist_create(); - if (auth & V1_AUTHORITY) + smartlist_t *lst = smartlist_new(); + if (auth & V1_DIRINFO) smartlist_add(lst, (void*)"V1"); - if (auth & V2_AUTHORITY) + if (auth & V2_DIRINFO) smartlist_add(lst, (void*)"V2"); - if (auth & V3_AUTHORITY) + if (auth & V3_DIRINFO) smartlist_add(lst, (void*)"V3"); - if (auth & BRIDGE_AUTHORITY) + if (auth & BRIDGE_DIRINFO) smartlist_add(lst, (void*)"Bridge"); - if (auth & HIDSERV_AUTHORITY) + if (auth & HIDSERV_DIRINFO) smartlist_add(lst, (void*)"Hidden service"); if (smartlist_len(lst)) { result = smartlist_join_strings(lst, ", ", 0, NULL); @@ -201,6 +208,8 @@ dir_conn_purpose_to_string(int purpose) return "hidden-service v2 descriptor fetch"; case DIR_PURPOSE_UPLOAD_RENDDESC_V2: return "hidden-service v2 descriptor upload"; + case DIR_PURPOSE_FETCH_MICRODESC: + return "microdescriptor fetch"; } log_warn(LD_BUG, "Called with unknown purpose %d", purpose); @@ -213,17 +222,19 @@ dir_conn_purpose_to_string(int purpose) int router_supports_extrainfo(const char *identity_digest, int is_authority) { - routerinfo_t *ri = router_get_by_digest(identity_digest); + const node_t *node = node_get_by_id(identity_digest); - if (ri) { - if (ri->caches_extra_info) + if (node && node->ri) { + if (node->ri->caches_extra_info) return 1; - if (is_authority && ri->platform && - tor_version_as_new_as(ri->platform, "Tor 0.2.0.0-alpha-dev (r10070)")) + if (is_authority && node->ri->platform && + tor_version_as_new_as(node->ri->platform, + "Tor 0.2.0.0-alpha-dev (r10070)")) return 1; } if (is_authority) { - routerstatus_t *rs = router_get_consensus_status_by_id(identity_digest); + const routerstatus_t *rs = + router_get_consensus_status_by_id(identity_digest); if (rs && rs->version_supports_extrainfo_upload) return 1; } @@ -242,7 +253,7 @@ int directories_have_accepted_server_descriptor(void) { smartlist_t *servers = router_get_trusted_dir_servers(); - or_options_t *options = get_options(); + const or_options_t *options = get_options(); SMARTLIST_FOREACH(servers, trusted_dir_server_t *, d, { if ((d->type & options->_PublishServerDescriptor) && d->has_accepted_serverdesc) { @@ -271,11 +282,11 @@ directories_have_accepted_server_descriptor(void) */ void directory_post_to_dirservers(uint8_t dir_purpose, uint8_t router_purpose, - authority_type_t type, + dirinfo_type_t type, const char *payload, size_t payload_len, size_t extrainfo_len) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int post_via_tor; smartlist_t *dirservers = router_get_trusted_dir_servers(); int found = 0; @@ -296,8 +307,8 @@ directory_post_to_dirservers(uint8_t dir_purpose, uint8_t router_purpose, if (exclude_self && router_digest_is_me(ds->digest)) continue; - if (options->ExcludeNodes && options->StrictNodes && - routerset_contains_routerstatus(options->ExcludeNodes, rs)) { + if (options->StrictNodes && + routerset_contains_routerstatus(options->ExcludeNodes, rs, -1)) { log_warn(LD_DIR, "Wanted to contact authority '%s' for %s, but " "it's in our ExcludedNodes list and StrictNodes is set. " "Skipping.", @@ -324,7 +335,7 @@ directory_post_to_dirservers(uint8_t dir_purpose, uint8_t router_purpose, NULL, payload, upload_len, 0); } SMARTLIST_FOREACH_END(ds); if (!found) { - char *s = authority_type_to_string(type); + char *s = authdir_type_to_string(type); log_warn(LD_DIR, "Publishing server descriptor to directory authorities " "of type '%s', but no authorities of that type listed!", s); tor_free(s); @@ -341,39 +352,46 @@ void directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, const char *resource, int pds_flags) { - routerstatus_t *rs = NULL; - or_options_t *options = get_options(); + const routerstatus_t *rs = NULL; + const or_options_t *options = get_options(); int prefer_authority = directory_fetches_from_authorities(options); + int require_authority = 0; int get_via_tor = purpose_needs_anonymity(dir_purpose, router_purpose); - authority_type_t type; + dirinfo_type_t type; time_t if_modified_since = 0; /* FFFF we could break this switch into its own function, and call * it elsewhere in directory.c. -RD */ switch (dir_purpose) { case DIR_PURPOSE_FETCH_EXTRAINFO: - type = EXTRAINFO_CACHE | - (router_purpose == ROUTER_PURPOSE_BRIDGE ? BRIDGE_AUTHORITY : - V3_AUTHORITY); + type = EXTRAINFO_DIRINFO | + (router_purpose == ROUTER_PURPOSE_BRIDGE ? BRIDGE_DIRINFO : + V3_DIRINFO); break; case DIR_PURPOSE_FETCH_V2_NETWORKSTATUS: - type = V2_AUTHORITY; + type = V2_DIRINFO; prefer_authority = 1; /* Only v2 authorities have these anyway. */ + require_authority = 1; /* Don't fallback to asking a non-authority */ break; case DIR_PURPOSE_FETCH_SERVERDESC: - type = (router_purpose == ROUTER_PURPOSE_BRIDGE ? BRIDGE_AUTHORITY : - V3_AUTHORITY); + type = (router_purpose == ROUTER_PURPOSE_BRIDGE ? BRIDGE_DIRINFO : + V3_DIRINFO); break; case DIR_PURPOSE_FETCH_RENDDESC: - type = HIDSERV_AUTHORITY; + type = HIDSERV_DIRINFO; break; case DIR_PURPOSE_FETCH_STATUS_VOTE: case DIR_PURPOSE_FETCH_DETACHED_SIGNATURES: - type = V3_AUTHORITY; + case DIR_PURPOSE_FETCH_CERTIFICATE: + type = V3_DIRINFO; break; case DIR_PURPOSE_FETCH_CONSENSUS: - case DIR_PURPOSE_FETCH_CERTIFICATE: - type = V3_AUTHORITY; + type = V3_DIRINFO; + if (resource && !strcmp(resource,"microdesc")) + type |= MICRODESC_DIRINFO; + break; + case DIR_PURPOSE_FETCH_MICRODESC: + type = MICRODESC_DIRINFO; break; default: log_warn(LD_BUG, "Unexpected purpose %d", (int)dir_purpose); @@ -381,25 +399,46 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, } if (dir_purpose == DIR_PURPOSE_FETCH_CONSENSUS) { - networkstatus_t *v = networkstatus_get_latest_consensus(); - if (v) - if_modified_since = v->valid_after + 180; + int flav = FLAV_NS; + networkstatus_t *v; + if (resource) + flav = networkstatus_parse_flavor_name(resource); + + if (flav != -1) { + /* IF we have a parsed consensus of this type, we can do an + * if-modified-time based on it. */ + v = networkstatus_get_latest_consensus_by_flavor(flav); + if (v) + if_modified_since = v->valid_after + 180; + } else { + /* Otherwise it might be a consensus we don't parse, but which we + * do cache. Look at the cached copy, perhaps. */ + cached_dir_t *cd = dirserv_get_consensus(resource); + if (cd) + if_modified_since = cd->published + 180; + } } - if (!options->FetchServerDescriptors && type != HIDSERV_AUTHORITY) + if (!options->FetchServerDescriptors && type != HIDSERV_DIRINFO) return; if (!get_via_tor) { - if (options->UseBridges && type != BRIDGE_AUTHORITY) { - /* want to ask a running bridge for which we have a descriptor. */ - /* XXX023 we assume that all of our bridges can answer any - * possible directory question. This won't be true forever. -RD */ - /* It certainly is not true with conditional consensus downloading, - * so, for now, never assume the server supports that. */ - routerinfo_t *ri = choose_random_entry(NULL); - if (ri) { + if (options->UseBridges && type != BRIDGE_DIRINFO) { + /* We want to ask a running bridge for which we have a descriptor. + * + * Be careful here: we should only ask questions that we know our + * bridges can answer. So far we're solving that by backing off to + * the behavior supported by our oldest bridge; see for example + * any_bridges_dont_support_microdescriptors(). + */ + /* XXX024 Not all bridges handle conditional consensus downloading, + * so, for now, never assume the server supports that. -PP */ + const node_t *node = choose_random_entry(NULL); + if (node && node->ri) { + /* every bridge has a routerinfo. */ tor_addr_t addr; - tor_addr_from_ipv4h(&addr, ri->addr); + routerinfo_t *ri = node->ri; + node_get_addr(node, &addr); directory_initiate_command(ri->address, &addr, ri->or_port, 0, 0, /* don't use conditional consensus url */ @@ -412,10 +451,11 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, "nodes are available yet."); return; } else { - if (prefer_authority || type == BRIDGE_AUTHORITY) { + if (prefer_authority || type == BRIDGE_DIRINFO) { /* only ask authdirservers, and don't ask myself */ rs = router_pick_trusteddirserver(type, pds_flags); - if (rs == NULL && (pds_flags & PDS_NO_EXISTING_SERVERDESC_FETCH)) { + if (rs == NULL && (pds_flags & (PDS_NO_EXISTING_SERVERDESC_FETCH| + PDS_NO_EXISTING_MICRODESC_FETCH))) { /* We don't want to fetch from any authorities that we're currently * fetching server descriptors from, and we got no match. Did we * get no match because all the authorities have connections @@ -423,7 +463,8 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, * return,) or because all the authorities are down or on fire or * unreachable or something (in which case we should go on with * our fallback code)? */ - pds_flags &= ~PDS_NO_EXISTING_SERVERDESC_FETCH; + pds_flags &= ~(PDS_NO_EXISTING_SERVERDESC_FETCH| + PDS_NO_EXISTING_MICRODESC_FETCH); rs = router_pick_trusteddirserver(type, pds_flags); if (rs) { log_debug(LD_DIR, "Deferring serverdesc fetch: all authorities " @@ -431,8 +472,13 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, return; } } + if (rs == NULL && require_authority) { + log_info(LD_DIR, "No authorities were available for %s: will try " + "later.", dir_conn_purpose_to_string(dir_purpose)); + return; + } } - if (!rs && type != BRIDGE_AUTHORITY) { + if (!rs && type != BRIDGE_DIRINFO) { /* anybody with a non-zero dirport will do */ rs = router_pick_directory_server(type, pds_flags); if (!rs) { @@ -449,7 +495,7 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, if (dir_purpose == DIR_PURPOSE_FETCH_RENDDESC) { /* only ask hidserv authorities, any of them will do */ pds_flags |= PDS_IGNORE_FASCISTFIREWALL|PDS_ALLOW_SELF; - rs = router_pick_trusteddirserver(HIDSERV_AUTHORITY, pds_flags); + rs = router_pick_trusteddirserver(HIDSERV_DIRINFO, pds_flags); } else { /* anybody with a non-zero dirport will do. Disregard firewalls. */ pds_flags |= PDS_IGNORE_FASCISTFIREWALL; @@ -489,24 +535,23 @@ directory_get_from_all_authorities(uint8_t dir_purpose, tor_assert(dir_purpose == DIR_PURPOSE_FETCH_STATUS_VOTE || dir_purpose == DIR_PURPOSE_FETCH_DETACHED_SIGNATURES); - SMARTLIST_FOREACH(router_get_trusted_dir_servers(), - trusted_dir_server_t *, ds, - { + SMARTLIST_FOREACH_BEGIN(router_get_trusted_dir_servers(), + trusted_dir_server_t *, ds) { routerstatus_t *rs; if (router_digest_is_me(ds->digest)) continue; - if (!(ds->type & V3_AUTHORITY)) + if (!(ds->type & V3_DIRINFO)) continue; rs = &ds->fake_status; directory_initiate_command_routerstatus(rs, dir_purpose, router_purpose, 0, resource, NULL, 0, 0); - }); + } SMARTLIST_FOREACH_END(ds); } /** Same as directory_initiate_command_routerstatus(), but accepts * rendezvous data to fetch a hidden service descriptor. */ void -directory_initiate_command_routerstatus_rend(routerstatus_t *status, +directory_initiate_command_routerstatus_rend(const routerstatus_t *status, uint8_t dir_purpose, uint8_t router_purpose, int anonymized_connection, @@ -516,21 +561,22 @@ directory_initiate_command_routerstatus_rend(routerstatus_t *status, time_t if_modified_since, const rend_data_t *rend_query) { - or_options_t *options = get_options(); - routerinfo_t *router; + const or_options_t *options = get_options(); + const node_t *node; char address_buf[INET_NTOA_BUF_LEN+1]; struct in_addr in; const char *address; tor_addr_t addr; - router = router_get_by_digest(status->identity_digest); + node = node_get_by_id(status->identity_digest); - if (!router && anonymized_connection) { - log_info(LD_DIR, "Not sending anonymized request to directory %s; we " + if (!node && anonymized_connection) { + log_info(LD_DIR, "Not sending anonymized request to directory '%s'; we " "don't have its router descriptor.", routerstatus_describe(status)); return; - } else if (router) { - address = router->address; + } else if (node) { + node_get_address_string(node, address_buf, sizeof(address_buf)); + address = address_buf; } else { in.s_addr = htonl(status->addr); tor_inet_ntoa(&in, address_buf, sizeof(address_buf)); @@ -539,7 +585,7 @@ directory_initiate_command_routerstatus_rend(routerstatus_t *status, tor_addr_from_ipv4h(&addr, status->addr); if (options->ExcludeNodes && options->StrictNodes && - routerset_contains_routerstatus(options->ExcludeNodes, status)) { + routerset_contains_routerstatus(options->ExcludeNodes, status, -1)) { log_warn(LD_DIR, "Wanted to contact directory mirror %s for %s, but " "it's in our ExcludedNodes list and StrictNodes is set. " "Skipping. This choice might make your Tor not work.", @@ -574,7 +620,7 @@ directory_initiate_command_routerstatus_rend(routerstatus_t *status, * want to fetch. */ void -directory_initiate_command_routerstatus(routerstatus_t *status, +directory_initiate_command_routerstatus(const routerstatus_t *status, uint8_t dir_purpose, uint8_t router_purpose, int anonymized_connection, @@ -598,7 +644,7 @@ directory_conn_is_self_reachability_test(dir_connection_t *conn) { if (conn->requested_resource && !strcmpstart(conn->requested_resource,"authority")) { - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); if (me && router_digest_is_me(conn->identity_digest) && tor_addr_eq_ipv4h(&conn->_base.addr, me->addr) && /*XXXX prop 118*/ @@ -612,7 +658,7 @@ directory_conn_is_self_reachability_test(dir_connection_t *conn) * server due to a network error: Mark the router as down and try again if * possible. */ -void +static void connection_dir_request_failed(dir_connection_t *conn) { if (directory_conn_is_self_reachability_test(conn)) { @@ -626,15 +672,18 @@ connection_dir_request_failed(dir_connection_t *conn) connection_dir_download_v2_networkstatus_failed(conn, -1); } else if (conn->_base.purpose == DIR_PURPOSE_FETCH_SERVERDESC || conn->_base.purpose == DIR_PURPOSE_FETCH_EXTRAINFO) { - log_info(LD_DIR, "Giving up on directory server at '%s'; retrying", + log_info(LD_DIR, "Giving up on serverdesc/extrainfo fetch from " + "directory server at '%s'; retrying", conn->_base.address); if (conn->router_purpose == ROUTER_PURPOSE_BRIDGE) connection_dir_bridge_routerdesc_failed(conn); connection_dir_download_routerdesc_failed(conn); } else if (conn->_base.purpose == DIR_PURPOSE_FETCH_CONSENSUS) { - networkstatus_consensus_download_failed(0); + if (conn->requested_resource) + networkstatus_consensus_download_failed(0, conn->requested_resource); } else if (conn->_base.purpose == DIR_PURPOSE_FETCH_CERTIFICATE) { - log_info(LD_DIR, "Giving up on directory server at '%s'; retrying", + log_info(LD_DIR, "Giving up on certificate fetch from directory server " + "at '%s'; retrying", conn->_base.address); connection_dir_download_cert_failed(conn, 0); } else if (conn->_base.purpose == DIR_PURPOSE_FETCH_DETACHED_SIGNATURES) { @@ -643,6 +692,10 @@ connection_dir_request_failed(dir_connection_t *conn) } else if (conn->_base.purpose == DIR_PURPOSE_FETCH_STATUS_VOTE) { log_info(LD_DIR, "Giving up downloading votes from '%s'", conn->_base.address); + } else if (conn->_base.purpose == DIR_PURPOSE_FETCH_MICRODESC) { + log_info(LD_DIR, "Giving up on downloading microdescriptors from " + " directory server at '%s'; will retry", conn->_base.address); + connection_dir_download_routerdesc_failed(conn); } } @@ -672,7 +725,7 @@ connection_dir_download_v2_networkstatus_failed(dir_connection_t *conn, } else if (!strcmpstart(conn->requested_resource, "fp/")) { /* We were trying to download by fingerprint; mark them all as having * failed, and possibly retry them later.*/ - smartlist_t *failed = smartlist_create(); + smartlist_t *failed = smartlist_new(); dir_split_resource_into_fingerprints(conn->requested_resource+3, failed, NULL, 0); if (smartlist_len(failed)) { @@ -713,7 +766,8 @@ connection_dir_download_routerdesc_failed(dir_connection_t *conn) /* No need to relaunch descriptor downloads here: we already do it * every 10 or 60 seconds (FOO_DESCRIPTOR_RETRY_INTERVAL) in main.c. */ tor_assert(conn->_base.purpose == DIR_PURPOSE_FETCH_SERVERDESC || - conn->_base.purpose == DIR_PURPOSE_FETCH_EXTRAINFO); + conn->_base.purpose == DIR_PURPOSE_FETCH_EXTRAINFO || + conn->_base.purpose == DIR_PURPOSE_FETCH_MICRODESC); (void) conn; } @@ -732,7 +786,7 @@ connection_dir_bridge_routerdesc_failed(dir_connection_t *conn) if (!conn->requested_resource || strcmpstart(conn->requested_resource,"fp/")) return; - which = smartlist_create(); + which = smartlist_new(); dir_split_resource_into_fingerprints(conn->requested_resource + strlen("fp/"), which, NULL, 0); @@ -754,7 +808,7 @@ connection_dir_download_cert_failed(dir_connection_t *conn, int status) if (!conn->requested_resource) return; - failed = smartlist_create(); + failed = smartlist_new(); dir_split_resource_into_fingerprints(conn->requested_resource+3, failed, NULL, DSR_HEX); SMARTLIST_FOREACH(failed, char *, cp, @@ -776,7 +830,7 @@ connection_dir_download_cert_failed(dir_connection_t *conn, int status) * 3) Else yes. */ static int -directory_command_should_use_begindir(or_options_t *options, +directory_command_should_use_begindir(const or_options_t *options, const tor_addr_t *addr, int or_port, uint8_t router_purpose, int anonymized_connection) @@ -816,6 +870,20 @@ directory_initiate_command(const char *address, const tor_addr_t *_addr, if_modified_since, NULL); } +/** Return non-zero iff a directory connection with purpose + * <b>dir_purpose</b> reveals sensitive information about a Tor + * instance's client activities. (Such connections must be performed + * through normal three-hop Tor circuits.) */ +static int +is_sensitive_dir_purpose(uint8_t dir_purpose) +{ + return ((dir_purpose == DIR_PURPOSE_FETCH_RENDDESC) || + (dir_purpose == DIR_PURPOSE_HAS_FETCHED_RENDDESC) || + (dir_purpose == DIR_PURPOSE_UPLOAD_RENDDESC) || + (dir_purpose == DIR_PURPOSE_UPLOAD_RENDDESC_V2) || + (dir_purpose == DIR_PURPOSE_FETCH_RENDDESC_V2)); +} + /** Same as directory_initiate_command(), but accepts rendezvous data to * fetch a hidden service descriptor. */ static void @@ -831,7 +899,7 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, const rend_data_t *rend_query) { dir_connection_t *conn; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int socket_error = 0; int use_begindir = supports_begindir && directory_command_should_use_begindir(options, _addr, @@ -850,6 +918,13 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, log_debug(LD_DIR, "Initiating %s", dir_conn_purpose_to_string(dir_purpose)); +#ifndef NON_ANONYMOUS_MODE_ENABLED + tor_assert(!(is_sensitive_dir_purpose(dir_purpose) && + !anonymized_connection)); +#else + (void)is_sensitive_dir_purpose; +#endif + /* ensure that we don't make direct connections when a SOCKS server is * configured. */ if (!anonymized_connection && !use_begindir && !options->HTTPProxy && @@ -859,7 +934,7 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, return; } - conn = dir_connection_new(AF_INET); + conn = dir_connection_new(tor_addr_family(&addr)); /* set up conn so it's got all the data we need to remember */ tor_addr_copy(&conn->_base.addr, &addr); @@ -911,7 +986,11 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, error indicates broken link in windowsland. */ } } else { /* we want to connect via a tor connection */ - edge_connection_t *linked_conn; + entry_connection_t *linked_conn; + /* Anonymized tunneled connections can never share a circuit. + * One-hop directory connections can share circuits with each other + * but nothing else. */ + int iso_flags = anonymized_connection ? ISO_STREAM : ISO_SESSIONGRP; /* If it's an anonymized connection, remember the fact that we * wanted it for later: maybe we'll want it again soon. */ @@ -925,14 +1004,16 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, * hook up both sides */ linked_conn = - connection_ap_make_link(conn->_base.address, conn->_base.port, - digest, use_begindir, conn->dirconn_direct); + connection_ap_make_link(TO_CONN(conn), + conn->_base.address, conn->_base.port, + digest, + SESSION_GROUP_DIRCONN, iso_flags, + use_begindir, conn->dirconn_direct); if (!linked_conn) { log_warn(LD_NET,"Making tunnel to dirserver failed."); connection_mark_for_close(TO_CONN(conn)); return; } - connection_link_connections(TO_CONN(conn), TO_CONN(linked_conn)); if (connection_add(TO_CONN(conn)) < 0) { log_warn(LD_NET,"Unable to add connection for link to dirserver."); @@ -945,8 +1026,13 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, payload, payload_len, supports_conditional_consensus, if_modified_since); + connection_watch_events(TO_CONN(conn), READ_EVENT|WRITE_EVENT); - connection_start_reading(TO_CONN(linked_conn)); + IF_HAS_BUFFEREVENT(ENTRY_TO_CONN(linked_conn), { + connection_watch_events(ENTRY_TO_CONN(linked_conn), + READ_EVENT|WRITE_EVENT); + }) ELSE_IF_NO_BUFFEREVENT + connection_start_reading(ENTRY_TO_CONN(linked_conn)); } } @@ -984,43 +1070,51 @@ _compare_strs(const void **a, const void **b) * This url depends on whether or not the server we go to * is sufficiently new to support conditional consensus downloading, * i.e. GET .../consensus/<b>fpr</b>+<b>fpr</b>+<b>fpr</b> + * + * If 'resource' is provided, it is the name of a consensus flavor to request. */ static char * -directory_get_consensus_url(int supports_conditional_consensus) +directory_get_consensus_url(int supports_conditional_consensus, + const char *resource) { - char *url; - size_t len; + char *url = NULL; + const char *hyphen, *flavor; + if (resource==NULL || strcmp(resource, "ns")==0) { + flavor = ""; /* Request ns consensuses as "", so older servers will work*/ + hyphen = ""; + } else { + flavor = resource; + hyphen = "-"; + } if (supports_conditional_consensus) { char *authority_id_list; - smartlist_t *authority_digests = smartlist_create(); + smartlist_t *authority_digests = smartlist_new(); - SMARTLIST_FOREACH(router_get_trusted_dir_servers(), - trusted_dir_server_t *, ds, - { + SMARTLIST_FOREACH_BEGIN(router_get_trusted_dir_servers(), + trusted_dir_server_t *, ds) { char *hex; - if (!(ds->type & V3_AUTHORITY)) + if (!(ds->type & V3_DIRINFO)) continue; hex = tor_malloc(2*CONDITIONAL_CONSENSUS_FPR_LEN+1); base16_encode(hex, 2*CONDITIONAL_CONSENSUS_FPR_LEN+1, ds->v3_identity_digest, CONDITIONAL_CONSENSUS_FPR_LEN); smartlist_add(authority_digests, hex); - }); + } SMARTLIST_FOREACH_END(ds); smartlist_sort(authority_digests, _compare_strs); authority_id_list = smartlist_join_strings(authority_digests, "+", 0, NULL); - len = strlen(authority_id_list)+64; - url = tor_malloc(len); - tor_snprintf(url, len, "/tor/status-vote/current/consensus/%s.z", - authority_id_list); + tor_asprintf(&url, "/tor/status-vote/current/consensus%s%s/%s.z", + hyphen, flavor, authority_id_list); SMARTLIST_FOREACH(authority_digests, char *, cp, tor_free(cp)); smartlist_free(authority_digests); tor_free(authority_id_list); } else { - url = tor_strdup("/tor/status-vote/current/consensus.z"); + tor_asprintf(&url, "/tor/status-vote/current/consensus%s%s.z", + hyphen, flavor); } return url; } @@ -1036,13 +1130,11 @@ directory_send_command(dir_connection_t *conn, time_t if_modified_since) { char proxystring[256]; - char proxyauthstring[256]; char hoststring[128]; - char imsstring[RFC1123_TIME_LEN+32]; + smartlist_t *headers = smartlist_new(); char *url; char request[8192]; const char *httpcommand = NULL; - size_t len; tor_assert(conn); tor_assert(conn->_base.type == CONN_TYPE_DIR); @@ -1060,12 +1152,10 @@ directory_send_command(dir_connection_t *conn, } /* Format if-modified-since */ - if (!if_modified_since) { - imsstring[0] = '\0'; - } else { + if (if_modified_since) { char b[RFC1123_TIME_LEN+1]; format_rfc1123_time(b, if_modified_since); - tor_snprintf(imsstring, sizeof(imsstring), "\r\nIf-Modified-Since: %s", b); + smartlist_add_asprintf(headers, "If-Modified-Since: %s\r\n", b); } /* come up with some proxy lines, if we're using one. */ @@ -1080,31 +1170,27 @@ directory_send_command(dir_connection_t *conn, log_warn(LD_BUG, "Encoding http authenticator failed"); } if (base64_authenticator) { - tor_snprintf(proxyauthstring, sizeof(proxyauthstring), - "\r\nProxy-Authorization: Basic %s", + smartlist_add_asprintf(headers, + "Proxy-Authorization: Basic %s\r\n", base64_authenticator); tor_free(base64_authenticator); - } else { - proxyauthstring[0] = 0; } } else { proxystring[0] = 0; - proxyauthstring[0] = 0; } switch (purpose) { case DIR_PURPOSE_FETCH_V2_NETWORKSTATUS: tor_assert(resource); httpcommand = "GET"; - len = strlen(resource)+32; - url = tor_malloc(len); - tor_snprintf(url, len, "/tor/status/%s", resource); + tor_asprintf(&url, "/tor/status/%s", resource); break; case DIR_PURPOSE_FETCH_CONSENSUS: - tor_assert(!resource); + /* resource is optional. If present, it's a flavor name */ tor_assert(!payload); httpcommand = "GET"; - url = directory_get_consensus_url(supports_conditional_consensus); + url = directory_get_consensus_url(supports_conditional_consensus, + resource); log_info(LD_DIR, "Downloading consensus from %s using %s", hoststring, url); break; @@ -1112,17 +1198,13 @@ directory_send_command(dir_connection_t *conn, tor_assert(resource); tor_assert(!payload); httpcommand = "GET"; - len = strlen(resource)+32; - url = tor_malloc(len); - tor_snprintf(url, len, "/tor/keys/%s", resource); + tor_asprintf(&url, "/tor/keys/%s", resource); break; case DIR_PURPOSE_FETCH_STATUS_VOTE: tor_assert(resource); tor_assert(!payload); httpcommand = "GET"; - len = strlen(resource)+32; - url = tor_malloc(len); - tor_snprintf(url, len, "/tor/status-vote/next/%s.z", resource); + tor_asprintf(&url, "/tor/status-vote/next/%s.z", resource); break; case DIR_PURPOSE_FETCH_DETACHED_SIGNATURES: tor_assert(!resource); @@ -1133,23 +1215,29 @@ directory_send_command(dir_connection_t *conn, case DIR_PURPOSE_FETCH_SERVERDESC: tor_assert(resource); httpcommand = "GET"; - len = strlen(resource)+32; - url = tor_malloc(len); - tor_snprintf(url, len, "/tor/server/%s", resource); + tor_asprintf(&url, "/tor/server/%s", resource); break; case DIR_PURPOSE_FETCH_EXTRAINFO: tor_assert(resource); httpcommand = "GET"; - len = strlen(resource)+32; - url = tor_malloc(len); - tor_snprintf(url, len, "/tor/extra/%s", resource); + tor_asprintf(&url, "/tor/extra/%s", resource); break; - case DIR_PURPOSE_UPLOAD_DIR: + case DIR_PURPOSE_FETCH_MICRODESC: + tor_assert(resource); + httpcommand = "GET"; + tor_asprintf(&url, "/tor/micro/%s", resource); + break; + case DIR_PURPOSE_UPLOAD_DIR: { + const char *why = router_get_descriptor_gen_reason(); tor_assert(!resource); tor_assert(payload); httpcommand = "POST"; url = tor_strdup("/tor/"); + if (why) { + smartlist_add_asprintf(headers, "X-Desc-Gen-Reason: %s\r\n", why); + } break; + } case DIR_PURPOSE_UPLOAD_VOTE: tor_assert(!resource); tor_assert(payload); @@ -1167,9 +1255,7 @@ directory_send_command(dir_connection_t *conn, tor_assert(strlen(resource) <= REND_DESC_ID_V2_LEN_BASE32); tor_assert(!payload); httpcommand = "GET"; - len = strlen(resource) + 32; - url = tor_malloc(len); - tor_snprintf(url, len, "/tor/rendezvous2/%s", resource); + tor_asprintf(&url, "/tor/rendezvous2/%s", resource); break; case DIR_PURPOSE_UPLOAD_RENDDESC: tor_assert(!resource); @@ -1200,26 +1286,27 @@ directory_send_command(dir_connection_t *conn, connection_write_to_buf(url, strlen(url), TO_CONN(conn)); tor_free(url); - if (!strcmp(httpcommand, "GET") && !payload) { - tor_snprintf(request, sizeof(request), - " HTTP/1.0\r\nHost: %s%s%s\r\n\r\n", - hoststring, - imsstring, - proxyauthstring); - } else { - tor_snprintf(request, sizeof(request), - " HTTP/1.0\r\nContent-Length: %lu\r\nHost: %s%s%s\r\n\r\n", - payload ? (unsigned long)payload_len : 0, - hoststring, - imsstring, - proxyauthstring); + if (!strcmp(httpcommand, "POST") || payload) { + smartlist_add_asprintf(headers, "Content-Length: %lu\r\n", + payload ? (unsigned long)payload_len : 0); } + + { + char *header = smartlist_join_strings(headers, "", 0, NULL); + tor_snprintf(request, sizeof(request), " HTTP/1.0\r\nHost: %s\r\n%s\r\n", + hoststring, header); + tor_free(header); + } + connection_write_to_buf(request, strlen(request), TO_CONN(conn)); if (payload) { /* then send the payload afterwards too */ connection_write_to_buf(payload, payload_len, TO_CONN(conn)); } + + SMARTLIST_FOREACH(headers, char *, h, tor_free(h)); + smartlist_free(headers); } /** Parse an HTTP request string <b>headers</b> of the form @@ -1355,11 +1442,11 @@ parse_http_response(const char *headers, int *code, time_t *date, } *code = n2; - parsed_headers = smartlist_create(); + parsed_headers = smartlist_new(); smartlist_split_string(parsed_headers, headers, "\n", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); if (reason) { - smartlist_t *status_line_elements = smartlist_create(); + smartlist_t *status_line_elements = smartlist_new(); tor_assert(smartlist_len(parsed_headers)); smartlist_split_string(status_line_elements, smartlist_get(parsed_headers, 0), @@ -1419,6 +1506,9 @@ body_is_plausible(const char *body, size_t len, int purpose) return 1; /* empty bodies don't need decompression */ if (len < 32) return 0; + if (purpose == DIR_PURPOSE_FETCH_MICRODESC) { + return (!strcmpstart(body,"onion-key")); + } if (purpose != DIR_PURPOSE_FETCH_RENDDESC) { if (!strcmpstart(body,"router") || !strcmpstart(body,"signed-directory") || @@ -1495,11 +1585,12 @@ connection_dir_client_reached_eof(dir_connection_t *conn) int plausible; int skewed=0; int allow_partial = (conn->_base.purpose == DIR_PURPOSE_FETCH_SERVERDESC || - conn->_base.purpose == DIR_PURPOSE_FETCH_EXTRAINFO); + conn->_base.purpose == DIR_PURPOSE_FETCH_EXTRAINFO || + conn->_base.purpose == DIR_PURPOSE_FETCH_MICRODESC); int was_compressed=0; time_t now = time(NULL); - switch (fetch_from_buf_http(conn->_base.inbuf, + switch (connection_fetch_from_buf_http(TO_CONN(conn), &headers, MAX_HEADERS_SIZE, &body, &body_len, MAX_DIR_DL_SIZE, allow_partial)) { @@ -1526,9 +1617,11 @@ connection_dir_client_reached_eof(dir_connection_t *conn) if (!reason) reason = tor_strdup("[no reason given]"); log_debug(LD_DIR, - "Received response from directory server '%s:%d': %d %s", + "Received response from directory server '%s:%d': %d %s " + "(purpose: %d)", conn->_base.address, conn->_base.port, status_code, - escaped(reason)); + escaped(reason), + conn->_base.purpose); /* now check if it's got any hints for us about our IP address. */ if (conn->dirconn_direct) { @@ -1574,13 +1667,14 @@ connection_dir_client_reached_eof(dir_connection_t *conn) if (status_code == 503) { routerstatus_t *rs; trusted_dir_server_t *ds; + const char *id_digest = conn->identity_digest; log_info(LD_DIR,"Received http status code %d (%s) from server " "'%s:%d'. I'll try again soon.", status_code, escaped(reason), conn->_base.address, conn->_base.port); - if ((rs = router_get_consensus_status_by_id(conn->identity_digest))) + if ((rs = router_get_mutable_consensus_status_by_id(id_digest))) rs->last_dir_503_at = now; - if ((ds = router_get_trusteddirserver_by_digest(conn->identity_digest))) + if ((ds = router_get_trusteddirserver_by_digest(id_digest))) ds->fake_status.last_dir_503_at = now; tor_free(body); tor_free(headers); tor_free(reason); @@ -1671,13 +1765,13 @@ connection_dir_client_reached_eof(dir_connection_t *conn) if (conn->requested_resource && !strcmpstart(conn->requested_resource,"fp/")) { source = NS_FROM_DIR_BY_FP; - which = smartlist_create(); + which = smartlist_new(); dir_split_resource_into_fingerprints(conn->requested_resource+3, which, NULL, 0); } else if (conn->requested_resource && !strcmpstart(conn->requested_resource, "all")) { source = NS_FROM_DIR_ALL; - which = smartlist_create(); + which = smartlist_new(); SMARTLIST_FOREACH(router_get_trusted_dir_servers(), trusted_dir_server_t *, ds, { @@ -1719,6 +1813,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) if (conn->_base.purpose == DIR_PURPOSE_FETCH_CONSENSUS) { int r; + const char *flavname = conn->requested_resource; if (status_code != 200) { int severity = (status_code == 304) ? LOG_INFO : LOG_WARN; log(severity, LD_DIR, @@ -1727,22 +1822,24 @@ connection_dir_client_reached_eof(dir_connection_t *conn) status_code, escaped(reason), conn->_base.address, conn->_base.port); tor_free(body); tor_free(headers); tor_free(reason); - networkstatus_consensus_download_failed(status_code); + networkstatus_consensus_download_failed(status_code, flavname); return -1; } log_info(LD_DIR,"Received consensus directory (size %d) from server " "'%s:%d'", (int)body_len, conn->_base.address, conn->_base.port); - if ((r=networkstatus_set_current_consensus(body, "ns", 0))<0) { + if ((r=networkstatus_set_current_consensus(body, flavname, 0))<0) { log_fn(r<-1?LOG_WARN:LOG_INFO, LD_DIR, - "Unable to load consensus directory downloaded from " + "Unable to load %s consensus directory downloaded from " "server '%s:%d'. I'll try again soon.", - conn->_base.address, conn->_base.port); + flavname, conn->_base.address, conn->_base.port); tor_free(body); tor_free(headers); tor_free(reason); - networkstatus_consensus_download_failed(0); + networkstatus_consensus_download_failed(0, flavname); return -1; } /* launches router downloads as needed */ routers_update_all_from_networkstatus(now, 3); + update_microdescs_from_networkstatus(now); + update_microdesc_downloads(now); directory_info_has_arrived(now, 0); log_info(LD_DIR, "Successfully loaded consensus."); } @@ -1823,7 +1920,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) if (conn->requested_resource && (!strcmpstart(conn->requested_resource,"d/") || !strcmpstart(conn->requested_resource,"fp/"))) { - which = smartlist_create(); + which = smartlist_new(); dir_split_resource_into_fingerprints(conn->requested_resource + (descriptor_digests ? 2 : 3), which, NULL, 0); @@ -1892,6 +1989,43 @@ connection_dir_client_reached_eof(dir_connection_t *conn) if (directory_conn_is_self_reachability_test(conn)) router_dirport_found_reachable(); } + if (conn->_base.purpose == DIR_PURPOSE_FETCH_MICRODESC) { + smartlist_t *which = NULL; + log_info(LD_DIR,"Received answer to microdescriptor request (status %d, " + "size %d) from server '%s:%d'", + status_code, (int)body_len, conn->_base.address, + conn->_base.port); + tor_assert(conn->requested_resource && + !strcmpstart(conn->requested_resource, "d/")); + which = smartlist_new(); + dir_split_resource_into_fingerprints(conn->requested_resource+2, + which, NULL, + DSR_DIGEST256|DSR_BASE64); + if (status_code != 200) { + log_info(LD_DIR, "Received status code %d (%s) from server " + "'%s:%d' while fetching \"/tor/micro/%s\". I'll try again " + "soon.", + status_code, escaped(reason), conn->_base.address, + (int)conn->_base.port, conn->requested_resource); + dir_microdesc_download_failed(which, status_code); + SMARTLIST_FOREACH(which, char *, cp, tor_free(cp)); + smartlist_free(which); + tor_free(body); tor_free(headers); tor_free(reason); + return 0; + } else { + smartlist_t *mds; + mds = microdescs_add_to_cache(get_microdesc_cache(), + body, body+body_len, SAVED_NOWHERE, 0, + now, which); + if (smartlist_len(which)) { + /* Mark remaining ones as failed. */ + dir_microdesc_download_failed(which, status_code); + } + SMARTLIST_FOREACH(which, char *, cp, tor_free(cp)); + smartlist_free(which); + smartlist_free(mds); + } + } if (conn->_base.purpose == DIR_PURPOSE_UPLOAD_DIR) { switch (status_code) { @@ -2166,7 +2300,7 @@ connection_dir_process_inbuf(dir_connection_t *conn) return 0; } - if (buf_datalen(conn->_base.inbuf) > MAX_DIRECTORY_OBJECT_SIZE) { + if (connection_get_inbuf_len(TO_CONN(conn)) > MAX_DIRECTORY_OBJECT_SIZE) { log_warn(LD_HTTP, "Too much data received from directory connection: " "denial of service attempt, or you need to upgrade?"); connection_mark_for_close(TO_CONN(conn)); @@ -2178,6 +2312,28 @@ connection_dir_process_inbuf(dir_connection_t *conn) return 0; } +/** Called when we're about to finally unlink and free a directory connection: + * perform necessary accounting and cleanup */ +void +connection_dir_about_to_close(dir_connection_t *dir_conn) +{ + connection_t *conn = TO_CONN(dir_conn); + + if (conn->state < DIR_CONN_STATE_CLIENT_FINISHED) { + /* It's a directory connection and connecting or fetching + * failed: forget about this router, and maybe try again. */ + connection_dir_request_failed(dir_conn); + } + /* If we were trying to fetch a v2 rend desc and did not succeed, + * retry as needed. (If a fetch is successful, the connection state + * is changed to DIR_PURPOSE_HAS_FETCHED_RENDDESC to mark that + * refetching is unnecessary.) */ + if (conn->purpose == DIR_PURPOSE_FETCH_RENDDESC_V2 && + dir_conn->rend_data && + strlen(dir_conn->rend_data->onion_address) == REND_SERVICE_ID_LEN_BASE32) + rend_client_refetch_v2_renddesc(dir_conn->rend_data); +} + /** Create an http response for the client <b>conn</b> out of * <b>status</b> and <b>reason_phrase</b>. Write it to <b>conn</b>. */ @@ -2280,7 +2436,8 @@ write_http_response_header(dir_connection_t *conn, ssize_t length, cache_lifetime); } -#ifdef INSTRUMENT_DOWNLOADS +#if defined(INSTRUMENT_DOWNLOADS) || defined(RUNNING_DOXYGEN) +/* DOCDOC */ typedef struct request_t { uint64_t bytes; /**< How many bytes have we transferred? */ uint64_t count; /**< How many requests have we made? */ @@ -2317,11 +2474,9 @@ note_client_request(int purpose, int compressed, size_t bytes) case DIR_PURPOSE_UPLOAD_RENDDESC_V2: kind = "dl/ul-rend2"; break; } if (kind) { - key = tor_malloc(256); - tor_snprintf(key, 256, "%s%s", kind, compressed?".z":""); + tor_asprintf(&key, "%s%s", kind, compressed?".z":""); } else { - key = tor_malloc(256); - tor_snprintf(key, 256, "unknown purpose (%d)%s", + tor_asprintf(&key, "unknown purpose (%d)%s", purpose, compressed?".z":""); } note_request(key, bytes); @@ -2360,13 +2515,12 @@ char * directory_dump_request_log(void) { smartlist_t *lines; - char tmp[256]; char *result; strmap_iter_t *iter; ensure_request_map_initialized(); - lines = smartlist_create(); + lines = smartlist_new(); for (iter = strmap_iter_init(request_map); !strmap_iter_done(iter); @@ -2376,9 +2530,8 @@ directory_dump_request_log(void) request_t *r; strmap_iter_get(iter, &key, &val); r = val; - tor_snprintf(tmp, sizeof(tmp), "%s "U64_FORMAT" "U64_FORMAT"\n", + smartlist_add_asprintf(lines, "%s "U64_FORMAT" "U64_FORMAT"\n", key, U64_PRINTF_ARG(r->bytes), U64_PRINTF_ARG(r->count)); - smartlist_add(lines, tor_strdup(tmp)); } smartlist_sort_strings(lines); result = smartlist_join_strings(lines, "", 0, NULL); @@ -2426,7 +2579,7 @@ directory_dump_request_log(void) int client_likes_consensus(networkstatus_t *v, const char *want_url) { - smartlist_t *want_authorities = smartlist_create(); + smartlist_t *want_authorities = smartlist_new(); int need_at_least; int have = 0; @@ -2469,18 +2622,18 @@ client_likes_consensus(networkstatus_t *v, const char *want_url) * Always return 0. */ static int directory_handle_command_get(dir_connection_t *conn, const char *headers, - const char *body, size_t body_len) + const char *req_body, size_t req_body_len) { size_t dlen; char *url, *url_mem, *header; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); time_t if_modified_since = 0; int compressed; size_t url_len; /* We ignore the body of a GET request. */ - (void)body; - (void)body_len; + (void)req_body; + (void)req_body_len; log_debug(LD_DIRSERV,"Received GET command."); @@ -2601,7 +2754,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, if (!strcmpstart(url,"/tor/status/") || !strcmpstart(url, "/tor/status-vote/current/consensus")) { /* v2 or v3 network status fetch. */ - smartlist_t *dir_fps = smartlist_create(); + smartlist_t *dir_fps = smartlist_new(); int is_v3 = !strcmpstart(url, "/tor/status-vote"); geoip_client_action_t act = is_v3 ? GEOIP_CLIENT_NETWORKSTATUS : GEOIP_CLIENT_NETWORKSTATUS_V2; @@ -2621,10 +2774,11 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, else request_type = "/tor/status/?"; } else { - networkstatus_t *v = networkstatus_get_latest_consensus(); + networkstatus_t *v; time_t now = time(NULL); const char *want_fps = NULL; char *flavor = NULL; + int flav = FLAV_NS; #define CONSENSUS_URL_PREFIX "/tor/status-vote/current/consensus/" #define CONSENSUS_FLAVORED_PREFIX "/tor/status-vote/current/consensus-" /* figure out the flavor if any, and who we wanted to sign the thing */ @@ -2638,12 +2792,16 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, } else { flavor = tor_strdup(f); } + flav = networkstatus_parse_flavor_name(flavor); + if (flav < 0) + flav = FLAV_NS; } else { if (!strcmpstart(url, CONSENSUS_URL_PREFIX)) want_fps = url+strlen(CONSENSUS_URL_PREFIX); } - /* XXXX MICRODESC NM NM should check document of correct flavor */ + v = networkstatus_get_latest_consensus_by_flavor(flav); + if (v && want_fps && !client_likes_consensus(v, want_fps)) { write_http_status_line(conn, 404, "Consensus not signed by sufficient " @@ -2700,8 +2858,10 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, { struct in_addr in; + tor_addr_t addr; if (tor_inet_aton((TO_CONN(conn))->address, &in)) { - geoip_note_client_seen(act, ntohl(in.s_addr), time(NULL)); + tor_addr_from_ipv4h(&addr, ntohl(in.s_addr)); + geoip_note_client_seen(act, &addr, time(NULL)); geoip_note_ns_response(act, GEOIP_SUCCESS); /* Note that a request for a network status has started, so that we * can measure the download time later on. */ @@ -2736,8 +2896,8 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, int current; ssize_t body_len = 0; ssize_t estimated_len = 0; - smartlist_t *items = smartlist_create(); - smartlist_t *dir_items = smartlist_create(); + smartlist_t *items = smartlist_new(); + smartlist_t *dir_items = smartlist_new(); int lifetime = 60; /* XXXX023 should actually use vote intervals. */ url += strlen("/tor/status-vote/"); current = !strcmpstart(url, "current/"); @@ -2765,7 +2925,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, smartlist_add(dir_items, (cached_dir_t*)d); } else { const cached_dir_t *d; - smartlist_t *fps = smartlist_create(); + smartlist_t *fps = smartlist_new(); int flags; if (!strcmpstart(url, "d/")) { url += 2; @@ -2829,7 +2989,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, } if (!strcmpstart(url, "/tor/micro/d/")) { - smartlist_t *fps = smartlist_create(); + smartlist_t *fps = smartlist_new(); dir_split_resource_into_fingerprints(url+strlen("/tor/micro/d/"), fps, NULL, @@ -2872,7 +3032,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, int cache_lifetime = 0; int is_extra = !strcmpstart(url,"/tor/extra/"); url += is_extra ? strlen("/tor/extra/") : strlen("/tor/server/"); - conn->fingerprint_stack = smartlist_create(); + conn->fingerprint_stack = smartlist_new(); res = dirserv_get_routerdesc_fingerprints(conn->fingerprint_stack, url, &msg, !connection_dir_is_encrypted(conn), @@ -2933,7 +3093,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, } if (!strcmpstart(url,"/tor/keys/")) { - smartlist_t *certs = smartlist_create(); + smartlist_t *certs = smartlist_new(); ssize_t len = -1; if (!strcmp(url, "/tor/keys/all")) { authority_cert_get_all(certs); @@ -2942,7 +3102,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, if (cert) smartlist_add(certs, cert); } else if (!strcmpstart(url, "/tor/keys/fp/")) { - smartlist_t *fps = smartlist_create(); + smartlist_t *fps = smartlist_new(); dir_split_resource_into_fingerprints(url+strlen("/tor/keys/fp/"), fps, NULL, DSR_HEX|DSR_SORT_UNIQ); @@ -2953,7 +3113,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, }); smartlist_free(fps); } else if (!strcmpstart(url, "/tor/keys/sk/")) { - smartlist_t *fps = smartlist_create(); + smartlist_t *fps = smartlist_new(); dir_split_resource_into_fingerprints(url+strlen("/tor/keys/sk/"), fps, NULL, DSR_HEX|DSR_SORT_UNIQ); @@ -2964,7 +3124,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, }); smartlist_free(fps); } else if (!strcmpstart(url, "/tor/keys/fp-sk/")) { - smartlist_t *fp_sks = smartlist_create(); + smartlist_t *fp_sks = smartlist_new(); dir_split_resource_into_fingerprint_pairs(url+strlen("/tor/keys/fp-sk/"), fp_sks); SMARTLIST_FOREACH(fp_sks, fp_pair_t *, pair, { @@ -3018,6 +3178,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, } if (options->HidServDirectoryV2 && + connection_dir_is_encrypted(conn) && !strcmpstart(url,"/tor/rendezvous2/")) { /* Handle v2 rendezvous descriptor fetch request. */ const char *descp; @@ -3135,8 +3296,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, #if defined(EXPORTMALLINFO) && defined(HAVE_MALLOC_H) && defined(HAVE_MALLINFO) #define ADD_MALLINFO_LINE(x) do { \ - tor_snprintf(tmp, sizeof(tmp), "%s %d\n", #x, mi.x); \ - smartlist_add(lines, tor_strdup(tmp)); \ + smartlist_add_asprintf(lines, "%s %d\n", #x, mi.x); \ }while(0); if (!strcmp(url,"/tor/mallinfo.txt") && @@ -3145,11 +3305,10 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, size_t len; struct mallinfo mi; smartlist_t *lines; - char tmp[256]; memset(&mi, 0, sizeof(mi)); mi = mallinfo(); - lines = smartlist_create(); + lines = smartlist_new(); ADD_MALLINFO_LINE(arena) ADD_MALLINFO_LINE(ordblks) @@ -3192,7 +3351,7 @@ directory_handle_command_post(dir_connection_t *conn, const char *headers, const char *body, size_t body_len) { char *url = NULL; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); log_debug(LD_DIRSERV,"Received POST command."); @@ -3206,6 +3365,7 @@ directory_handle_command_post(dir_connection_t *conn, const char *headers, /* Handle v2 rendezvous service publish request. */ if (options->HidServDirectoryV2 && + connection_dir_is_encrypted(conn) && !strcmpstart(url,"/tor/rendezvous2/publish")) { switch (rend_cache_store_v2_desc_as_dir(body)) { case -2: @@ -3336,7 +3496,7 @@ directory_handle_command(dir_connection_t *conn) tor_assert(conn); tor_assert(conn->_base.type == CONN_TYPE_DIR); - switch (fetch_from_buf_http(conn->_base.inbuf, + switch (connection_fetch_from_buf_http(TO_CONN(conn), &headers, MAX_HEADERS_SIZE, &body, &body_len, MAX_DIR_UL_SIZE, 0)) { case -1: /* overflow */ @@ -3389,14 +3549,27 @@ connection_dir_finished_flushing(dir_connection_t *conn) DIRREQ_DIRECT, DIRREQ_FLUSHING_DIR_CONN_FINISHED); switch (conn->_base.state) { + case DIR_CONN_STATE_CONNECTING: case DIR_CONN_STATE_CLIENT_SENDING: log_debug(LD_DIR,"client finished sending command."); conn->_base.state = DIR_CONN_STATE_CLIENT_READING; - connection_stop_writing(TO_CONN(conn)); return 0; case DIR_CONN_STATE_SERVER_WRITING: - log_debug(LD_DIRSERV,"Finished writing server response. Closing."); - connection_mark_for_close(TO_CONN(conn)); + if (conn->dir_spool_src != DIR_SPOOL_NONE) { +#ifdef USE_BUFFEREVENTS + /* This can happen with paired bufferevents, since a paired connection + * can flush immediately when you write to it, making the subsequent + * check in connection_handle_write_cb() decide that the connection + * is flushed. */ + log_debug(LD_DIRSERV, "Emptied a dirserv buffer, but still spooling."); +#else + log_warn(LD_BUG, "Emptied a dirserv buffer, but it's still spooling!"); + connection_mark_for_close(TO_CONN(conn)); +#endif + } else { + log_debug(LD_DIRSERV, "Finished writing server response. Closing."); + connection_mark_for_close(TO_CONN(conn)); + } return 0; default: log_warn(LD_BUG,"called in unexpected state %d.", @@ -3431,8 +3604,7 @@ dir_networkstatus_download_failed(smartlist_t *failed, int status_code) { if (status_code == 503) return; - SMARTLIST_FOREACH(failed, const char *, fp, - { + SMARTLIST_FOREACH_BEGIN(failed, const char *, fp) { char digest[DIGEST_LEN]; trusted_dir_server_t *dir; if (base16_decode(digest, DIGEST_LEN, fp, strlen(fp))<0) { @@ -3444,7 +3616,7 @@ dir_networkstatus_download_failed(smartlist_t *failed, int status_code) if (dir) download_status_failed(&dir->v2_ns_dl_status, status_code); - }); + } SMARTLIST_FOREACH_END(fp); } /** Schedule for when servers should download things in general. */ @@ -3563,7 +3735,7 @@ download_status_reset(download_status_t *dls) const int *schedule; size_t schedule_len; - find_dl_schedule_and_len(dls, get_options()->DirPort, + find_dl_schedule_and_len(dls, get_options()->DirPort_set, &schedule, &schedule_len); dls->n_download_failures = 0; @@ -3598,8 +3770,7 @@ dir_routerdesc_download_failed(smartlist_t *failed, int status_code, } return; /* FFFF should implement for other-than-router-purpose someday */ } - SMARTLIST_FOREACH(failed, const char *, cp, - { + SMARTLIST_FOREACH_BEGIN(failed, const char *, cp) { download_status_t *dls = NULL; if (base16_decode(digest, DIGEST_LEN, cp, strlen(cp)) < 0) { log_warn(LD_BUG, "Malformed fingerprint in list: %s", escaped(cp)); @@ -3616,12 +3787,45 @@ dir_routerdesc_download_failed(smartlist_t *failed, int status_code, if (!dls || dls->n_download_failures >= MAX_ROUTERDESC_DOWNLOAD_FAILURES) continue; download_status_increment_failure(dls, status_code, cp, server, now); - }); + } SMARTLIST_FOREACH_END(cp); /* No need to relaunch descriptor downloads here: we already do it * every 10 or 60 seconds (FOO_DESCRIPTOR_RETRY_INTERVAL) in main.c. */ } +/** Called when a connection to download microdescriptors has failed in whole + * or in part. <b>failed</b> is a list of every microdesc digest we didn't + * get. <b>status_code</b> is the http status code we received. Reschedule the + * microdesc downloads as appropriate. */ +static void +dir_microdesc_download_failed(smartlist_t *failed, + int status_code) +{ + networkstatus_t *consensus + = networkstatus_get_latest_consensus_by_flavor(FLAV_MICRODESC); + routerstatus_t *rs; + download_status_t *dls; + time_t now = time(NULL); + int server = directory_fetches_from_authorities(get_options()); + + if (! consensus) + return; + SMARTLIST_FOREACH_BEGIN(failed, const char *, d) { + rs = router_get_mutable_consensus_status_by_descriptor_digest(consensus,d); + if (!rs) + continue; + dls = &rs->dl_status; + if (dls->n_download_failures >= MAX_MICRODESC_DOWNLOAD_FAILURES) + continue; + { + char buf[BASE64_DIGEST256_LEN+1]; + digest256_to_base64(buf, d); + download_status_increment_failure(dls, status_code, buf, + server, now); + } + } SMARTLIST_FOREACH_END(d); +} + /** Helper. Compare two fp_pair_t objects, and return negative, 0, or * positive as appropriate. */ static int @@ -3643,8 +3847,8 @@ int dir_split_resource_into_fingerprint_pairs(const char *res, smartlist_t *pairs_out) { - smartlist_t *pairs_tmp = smartlist_create(); - smartlist_t *pairs_result = smartlist_create(); + smartlist_t *pairs_tmp = smartlist_new(); + smartlist_t *pairs_result = smartlist_new(); smartlist_split_string(pairs_tmp, res, "+", 0, 0); if (smartlist_len(pairs_tmp)) { @@ -3712,7 +3916,7 @@ dir_split_resource_into_fingerprints(const char *resource, HEX_DIGEST256_LEN : HEX_DIGEST_LEN; const int base64_digest_len = digests_are_256 ? BASE64_DIGEST256_LEN : BASE64_DIGEST_LEN; - smartlist_t *fp_tmp = smartlist_create(); + smartlist_t *fp_tmp = smartlist_new(); tor_assert(!(decode_hex && decode_base64)); tor_assert(fp_out); diff --git a/src/or/directory.h b/src/or/directory.h index 94dfb17644..1ca1c5a6e0 100644 --- a/src/or/directory.h +++ b/src/or/directory.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -13,9 +13,8 @@ #define _TOR_DIRECTORY_H int directories_have_accepted_server_descriptor(void); -char *authority_type_to_string(authority_type_t auth); void directory_post_to_dirservers(uint8_t dir_purpose, uint8_t router_purpose, - authority_type_t type, const char *payload, + dirinfo_type_t type, const char *payload, size_t payload_len, size_t extrainfo_len); void directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, const char *resource, @@ -23,7 +22,7 @@ void directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, void directory_get_from_all_authorities(uint8_t dir_purpose, uint8_t router_purpose, const char *resource); -void directory_initiate_command_routerstatus(routerstatus_t *status, +void directory_initiate_command_routerstatus(const routerstatus_t *status, uint8_t dir_purpose, uint8_t router_purpose, int anonymized_connection, @@ -31,7 +30,7 @@ void directory_initiate_command_routerstatus(routerstatus_t *status, const char *payload, size_t payload_len, time_t if_modified_since); -void directory_initiate_command_routerstatus_rend(routerstatus_t *status, +void directory_initiate_command_routerstatus_rend(const routerstatus_t *status, uint8_t dir_purpose, uint8_t router_purpose, int anonymized_connection, @@ -49,7 +48,7 @@ int connection_dir_reached_eof(dir_connection_t *conn); int connection_dir_process_inbuf(dir_connection_t *conn); int connection_dir_finished_flushing(dir_connection_t *conn); int connection_dir_finished_connecting(dir_connection_t *conn); -void connection_dir_request_failed(dir_connection_t *conn); +void connection_dir_about_to_close(dir_connection_t *dir_conn); void directory_initiate_command(const char *address, const tor_addr_t *addr, uint16_t or_port, uint16_t dir_port, int supports_conditional_consensus, @@ -81,7 +80,7 @@ time_t download_status_increment_failure(download_status_t *dls, * the optional status code <b>sc</b>. */ #define download_status_failed(dls, sc) \ download_status_increment_failure((dls), (sc), NULL, \ - get_options()->DirPort, time(NULL)) + get_options()->DirPort_set, time(NULL)) void download_status_reset(download_status_t *dls); static int download_status_is_ready(download_status_t *dls, time_t now, diff --git a/src/or/dirserv.c b/src/or/dirserv.c index 0308d4dbc1..f1c9c6232d 100644 --- a/src/or/dirserv.c +++ b/src/or/dirserv.c @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #define DIRSERV_PRIVATE @@ -16,6 +16,7 @@ #include "hibernate.h" #include "microdesc.h" #include "networkstatus.h" +#include "nodelist.h" #include "policies.h" #include "rephist.h" #include "router.h" @@ -67,8 +68,6 @@ static char *format_versions_list(config_line_t *ln); struct authdir_config_t; static int add_fingerprint_to_dir(const char *nickname, const char *fp, struct authdir_config_t *list); -static uint32_t dirserv_router_get_status(const routerinfo_t *router, - const char **msg); static uint32_t dirserv_get_status_impl(const char *fp, const char *nickname, const char *address, @@ -76,7 +75,8 @@ dirserv_get_status_impl(const char *fp, const char *nickname, const char *platform, const char *contact, const char **msg, int should_log); static void clear_cached_dir(cached_dir_t *d); -static signed_descriptor_t *get_signed_descriptor_by_fp(const char *fp, +static const signed_descriptor_t *get_signed_descriptor_by_fp( + const char *fp, int extrainfo, time_t publish_cutoff); static int dirserv_add_extrainfo(extrainfo_t *ei, const char **msg); @@ -185,7 +185,7 @@ add_fingerprint_to_dir(const char *nickname, const char *fp, /** Add the nickname and fingerprint for this OR to the * global list of recognized identity key fingerprints. */ int -dirserv_add_own_fingerprint(const char *nickname, crypto_pk_env_t *pk) +dirserv_add_own_fingerprint(const char *nickname, crypto_pk_t *pk) { char fp[FINGERPRINT_LEN+1]; if (crypto_pk_get_fingerprint(pk, fp, 0)<0) { @@ -212,7 +212,7 @@ dirserv_load_fingerprint_file(void) authdir_config_t *fingerprint_list_new; int result; config_line_t *front=NULL, *list; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); fname = get_datadir_fname("approved-routers"); log_info(LD_GENERAL, @@ -232,7 +232,7 @@ dirserv_load_fingerprint_file(void) } tor_free(fname); - result = config_get_lines(cf, &front); + result = config_get_lines(cf, &front, 0); tor_free(cf); if (result < 0) { log_warn(LD_CONFIG, "Error reading from fingerprint file"); @@ -305,7 +305,7 @@ dirserv_load_fingerprint_file(void) * * If the status is 'FP_REJECT' and <b>msg</b> is provided, set * *<b>msg</b> to an explanation of why. */ -static uint32_t +uint32_t dirserv_router_get_status(const routerinfo_t *router, const char **msg) { char d[DIGEST_LEN]; @@ -327,7 +327,7 @@ dirserv_router_get_status(const routerinfo_t *router, const char **msg) /** Return true if there is no point in downloading the router described by * <b>rs</b> because this directory would reject it. */ int -dirserv_would_reject_router(routerstatus_t *rs) +dirserv_would_reject_router(const routerstatus_t *rs) { uint32_t res; @@ -362,7 +362,7 @@ dirserv_get_name_status(const char *id_digest, const char *nickname) return 0; } -/** Helper: As dirserv_get_router_status, but takes the router fingerprint +/** Helper: As dirserv_router_get_status, but takes the router fingerprint * (hex, no spaces), nickname, address (used for logging only), IP address, OR * port, platform (logging only) and contact info (logging only) as arguments. * @@ -377,7 +377,7 @@ dirserv_get_status_impl(const char *id_digest, const char *nickname, const char **msg, int should_log) { int reject_unlisted = get_options()->AuthDirRejectUnlisted; - uint32_t result = 0; + uint32_t result; router_status_t *status_by_digest; if (!fingerprint_list) @@ -543,7 +543,7 @@ dirserv_router_has_valid_address(routerinfo_t *ri) */ int authdir_wants_to_reject_router(routerinfo_t *ri, const char **msg, - int complain) + int complain, int *valid_out) { /* Okay. Now check whether the fingerprint is recognized. */ uint32_t status = dirserv_router_get_status(ri, msg); @@ -587,15 +587,24 @@ authdir_wants_to_reject_router(routerinfo_t *ri, const char **msg, *msg = "Rejected: Address is not an IP, or IP is a private address."; return -1; } - /* Okay, looks like we're willing to accept this one. */ - ri->is_named = (status & FP_NAMED) ? 1 : 0; - ri->is_valid = (status & FP_INVALID) ? 0 : 1; - ri->is_bad_directory = (status & FP_BADDIR) ? 1 : 0; - ri->is_bad_exit = (status & FP_BADEXIT) ? 1 : 0; + + *valid_out = ! (status & FP_INVALID); return 0; } +/** Update the relevant flags of <b>node</b> based on our opinion as a + * directory authority in <b>authstatus</b>, as returned by + * dirserv_router_get_status or equivalent. */ +void +dirserv_set_node_flags_from_authoritative_status(node_t *node, + uint32_t authstatus) +{ + node->is_valid = (authstatus & FP_INVALID) ? 0 : 1; + node->is_bad_directory = (authstatus & FP_BADDIR) ? 1 : 0; + node->is_bad_exit = (authstatus & FP_BADEXIT) ? 1 : 0; +} + /** True iff <b>a</b> is more severe than <b>b</b>. */ static int WRA_MORE_SEVERE(was_router_added_t a, was_router_added_t b) @@ -636,7 +645,7 @@ dirserv_add_multiple_descriptors(const char *desc, uint8_t purpose, } s = desc; - list = smartlist_create(); + list = smartlist_new(); if (!router_parse_list_from_string(&s, NULL, list, SAVED_NOWHERE, 0, 0, annotation_buf)) { SMARTLIST_FOREACH(list, routerinfo_t *, ri, { @@ -720,7 +729,7 @@ dirserv_add_descriptor(routerinfo_t *ri, const char **msg, const char *source) * from this server. (We do this here and not in router_add_to_routerlist * because we want to be able to accept the newest router descriptor that * another authority has, so we all converge on the same one.) */ - ri_old = router_get_by_digest(ri->cache_info.identity_digest); + ri_old = router_get_mutable_by_digest(ri->cache_info.identity_digest); if (ri_old && ri_old->cache_info.published_on < ri->cache_info.published_on && router_differences_are_cosmetic(ri_old, ri) && !router_is_me(ri)) { @@ -759,13 +768,12 @@ dirserv_add_descriptor(routerinfo_t *ri, const char **msg, const char *source) smartlist_t *changed; control_event_or_authdir_new_descriptor("ACCEPTED", desc, desclen, *msg); - changed = smartlist_create(); + changed = smartlist_new(); smartlist_add(changed, ri); routerlist_descriptors_added(changed, 0); smartlist_free(changed); if (!*msg) { - *msg = ri->is_valid ? "Descriptor for valid server accepted" : - "Descriptor for invalid server accepted"; + *msg = "Descriptor accepted"; } log_info(LD_DIRSERV, "Added descriptor from '%s' (source: %s): %s.", @@ -780,12 +788,12 @@ dirserv_add_descriptor(routerinfo_t *ri, const char **msg, const char *source) static was_router_added_t dirserv_add_extrainfo(extrainfo_t *ei, const char **msg) { - routerinfo_t *ri; + const routerinfo_t *ri; int r; tor_assert(msg); *msg = NULL; - ri = router_get_by_digest(ei->cache_info.identity_digest); + ri = router_get_by_id_digest(ei->cache_info.identity_digest); if (!ri) { *msg = "No corresponding router descriptor for extra-info descriptor"; extrainfo_free(ei); @@ -820,56 +828,67 @@ dirserv_add_extrainfo(extrainfo_t *ei, const char **msg) static void directory_remove_invalid(void) { - int i; int changed = 0; routerlist_t *rl = router_get_routerlist(); + smartlist_t *nodes = smartlist_new(); + smartlist_add_all(nodes, nodelist_get_list()); - routerlist_assert_ok(rl); - - for (i = 0; i < smartlist_len(rl->routers); ++i) { + SMARTLIST_FOREACH_BEGIN(nodes, node_t *, node) { const char *msg; - routerinfo_t *ent = smartlist_get(rl->routers, i); + routerinfo_t *ent = node->ri; char description[NODE_DESC_BUF_LEN]; - uint32_t r = dirserv_router_get_status(ent, &msg); + uint32_t r; + if (!ent) + continue; + r = dirserv_router_get_status(ent, &msg); router_get_description(description, ent); if (r & FP_REJECT) { log_info(LD_DIRSERV, "Router %s is now rejected: %s", description, msg?msg:""); routerlist_remove(rl, ent, 0, time(NULL)); - i--; changed = 1; continue; } - if (bool_neq((r & FP_NAMED), ent->is_named)) { +#if 0 + if (bool_neq((r & FP_NAMED), ent->auth_says_is_named)) { log_info(LD_DIRSERV, "Router %s is now %snamed.", description, (r&FP_NAMED)?"":"un"); ent->is_named = (r&FP_NAMED)?1:0; changed = 1; } - if (bool_neq((r & FP_INVALID), !ent->is_valid)) { + if (bool_neq((r & FP_UNNAMED), ent->auth_says_is_unnamed)) { + log_info(LD_DIRSERV, + "Router '%s' is now %snamed. (FP_UNNAMED)", description, + (r&FP_NAMED)?"":"un"); + ent->is_named = (r&FP_NUNAMED)?0:1; + changed = 1; + } +#endif + if (bool_neq((r & FP_INVALID), !node->is_valid)) { log_info(LD_DIRSERV, "Router '%s' is now %svalid.", description, (r&FP_INVALID) ? "in" : ""); - ent->is_valid = (r&FP_INVALID)?0:1; + node->is_valid = (r&FP_INVALID)?0:1; changed = 1; } - if (bool_neq((r & FP_BADDIR), ent->is_bad_directory)) { + if (bool_neq((r & FP_BADDIR), node->is_bad_directory)) { log_info(LD_DIRSERV, "Router '%s' is now a %s directory", description, (r & FP_BADDIR) ? "bad" : "good"); - ent->is_bad_directory = (r&FP_BADDIR) ? 1: 0; + node->is_bad_directory = (r&FP_BADDIR) ? 1: 0; changed = 1; } - if (bool_neq((r & FP_BADEXIT), ent->is_bad_exit)) { + if (bool_neq((r & FP_BADEXIT), node->is_bad_exit)) { log_info(LD_DIRSERV, "Router '%s' is now a %s exit", description, (r & FP_BADEXIT) ? "bad" : "good"); - ent->is_bad_exit = (r&FP_BADEXIT) ? 1: 0; + node->is_bad_exit = (r&FP_BADEXIT) ? 1: 0; changed = 1; } - } + } SMARTLIST_FOREACH_END(node); if (changed) directory_set_dirty(); routerlist_assert_ok(rl); + smartlist_free(nodes); } /** Mark the directory as <b>dirty</b> -- when we're next asked for a @@ -908,10 +927,11 @@ directory_set_dirty(void) * as running iff <b>is_live</b> is true. */ static char * -list_single_server_status(routerinfo_t *desc, int is_live) +list_single_server_status(const routerinfo_t *desc, int is_live) { char buf[MAX_NICKNAME_LEN+HEX_DIGEST_LEN+4]; /* !nickname=$hexdigest\0 */ char *cp; + const node_t *node; tor_assert(desc); @@ -919,7 +939,8 @@ list_single_server_status(routerinfo_t *desc, int is_live) if (!is_live) { *cp++ = '!'; } - if (desc->is_valid) { + node = node_get_by_id(desc->cache_info.identity_digest); + if (node && node->is_valid) { strlcpy(cp, desc->nickname, sizeof(buf)-(cp-buf)); cp += strlen(cp); *cp++ = '='; @@ -930,6 +951,7 @@ list_single_server_status(routerinfo_t *desc, int is_live) return tor_strdup(buf); } +/* DOCDOC running_long_enough_to_decide_unreachable */ static INLINE int running_long_enough_to_decide_unreachable(void) { @@ -953,11 +975,13 @@ running_long_enough_to_decide_unreachable(void) void dirserv_set_router_is_running(routerinfo_t *router, time_t now) { - /*XXXX023 This function is a mess. Separate out the part that calculates + /*XXXX024 This function is a mess. Separate out the part that calculates whether it's reachable and the part that tells rephist that the router was unreachable. */ int answer; + node_t *node = node_get_mutable_by_id(router->cache_info.identity_digest); + tor_assert(node); if (router_is_me(router)) { /* We always know if we are down ourselves. */ @@ -992,7 +1016,7 @@ dirserv_set_router_is_running(routerinfo_t *router, time_t now) rep_hist_note_router_unreachable(router->cache_info.identity_digest, when); } - router->is_running = answer; + node->is_running = answer; } /** Based on the routerinfo_ts in <b>routers</b>, allocate the @@ -1010,16 +1034,18 @@ list_server_status_v1(smartlist_t *routers, char **router_status_out, smartlist_t *rs_entries; time_t now = time(NULL); time_t cutoff = now - ROUTER_MAX_AGE_TO_PUBLISH; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); /* We include v2 dir auths here too, because they need to answer * controllers. Eventually we'll deprecate this whole function; * see also networkstatus_getinfo_by_purpose(). */ int authdir = authdir_mode_publishes_statuses(options); tor_assert(router_status_out); - rs_entries = smartlist_create(); + rs_entries = smartlist_new(); SMARTLIST_FOREACH_BEGIN(routers, routerinfo_t *, ri) { + const node_t *node = node_get_by_id(ri->cache_info.identity_digest); + tor_assert(node); if (authdir) { /* Update router status in routerinfo_t. */ dirserv_set_router_is_running(ri, now); @@ -1027,12 +1053,13 @@ list_server_status_v1(smartlist_t *routers, char **router_status_out, if (for_controller) { char name_buf[MAX_VERBOSE_NICKNAME_LEN+2]; char *cp = name_buf; - if (!ri->is_running) + if (!node->is_running) *cp++ = '!'; router_get_verbose_nickname(cp, ri); smartlist_add(rs_entries, tor_strdup(name_buf)); } else if (ri->cache_info.published_on >= cutoff) { - smartlist_add(rs_entries, list_single_server_status(ri, ri->is_running)); + smartlist_add(rs_entries, list_single_server_status(ri, + node->is_running)); } } SMARTLIST_FOREACH_END(ri); @@ -1054,7 +1081,7 @@ format_versions_list(config_line_t *ln) { smartlist_t *versions; char *result; - versions = smartlist_create(); + versions = smartlist_new(); for ( ; ln; ln = ln->next) { smartlist_split_string(versions, ln->value, ",", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); @@ -1070,12 +1097,12 @@ format_versions_list(config_line_t *ln) * not hibernating, and not too old. Else return 0. */ static int -router_is_active(routerinfo_t *ri, time_t now) +router_is_active(const routerinfo_t *ri, const node_t *node, time_t now) { time_t cutoff = now - ROUTER_MAX_AGE_TO_PUBLISH; if (ri->cache_info.published_on < cutoff) return 0; - if (!ri->is_running || !ri->is_valid || ri->is_hibernating) + if (!node->is_running || !node->is_valid || ri->is_hibernating) return 0; return 1; } @@ -1088,7 +1115,7 @@ router_is_active(routerinfo_t *ri, time_t now) */ int dirserv_dump_directory_to_string(char **dir_out, - crypto_pk_env_t *private_key) + crypto_pk_t *private_key) { char *cp; char *identity_pkey; /* Identity key, DER64-encoded. */ @@ -1174,9 +1201,9 @@ dirserv_dump_directory_to_string(char **dir_out, /** Return 1 if we fetch our directory material directly from the * authorities, rather than from a mirror. */ int -directory_fetches_from_authorities(or_options_t *options) +directory_fetches_from_authorities(const or_options_t *options) { - routerinfo_t *me; + const routerinfo_t *me; uint32_t addr; int refuseunknown; if (options->FetchDirInfoEarly) @@ -1187,7 +1214,7 @@ directory_fetches_from_authorities(or_options_t *options) return 1; /* we don't know our IP address; ask an authority. */ refuseunknown = ! router_my_exit_policy_is_reject_star() && should_refuse_unknown_exits(options); - if (options->DirPort == 0 && !refuseunknown) + if (!options->DirPort_set && !refuseunknown) return 0; if (!server_mode(options) || !advertised_server_mode()) return 0; @@ -1201,7 +1228,7 @@ directory_fetches_from_authorities(or_options_t *options) * on the "mirror" schedule rather than the "client" schedule. */ int -directory_fetches_dir_info_early(or_options_t *options) +directory_fetches_dir_info_early(const or_options_t *options) { return directory_fetches_from_authorities(options); } @@ -1213,7 +1240,7 @@ directory_fetches_dir_info_early(or_options_t *options) * client as a directory guard. */ int -directory_fetches_dir_info_later(or_options_t *options) +directory_fetches_dir_info_later(const or_options_t *options) { return options->UseBridges != 0; } @@ -1221,18 +1248,27 @@ directory_fetches_dir_info_later(or_options_t *options) /** Return 1 if we want to cache v2 dir info (each status file). */ int -directory_caches_v2_dir_info(or_options_t *options) +directory_caches_v2_dir_info(const or_options_t *options) { - return options->DirPort != 0; + return options->DirPort_set; +} + +/** Return true iff we want to fetch and keep certificates for authorities + * that we don't acknowledge as aurthorities ourself. + */ +int +directory_caches_unknown_auth_certs(const or_options_t *options) +{ + return options->DirPort_set || options->BridgeRelay; } /** Return 1 if we want to keep descriptors, networkstatuses, etc around * and we're willing to serve them to others. Else return 0. */ int -directory_caches_dir_info(or_options_t *options) +directory_caches_dir_info(const or_options_t *options) { - if (options->BridgeRelay || options->DirPort) + if (options->BridgeRelay || options->DirPort_set) return 1; if (!server_mode(options) || !advertised_server_mode()) return 0; @@ -1246,18 +1282,18 @@ directory_caches_dir_info(or_options_t *options) * requests via the "begin_dir" interface, which doesn't require * having any separate port open. */ int -directory_permits_begindir_requests(or_options_t *options) +directory_permits_begindir_requests(const or_options_t *options) { - return options->BridgeRelay != 0 || options->DirPort != 0; + return options->BridgeRelay != 0 || options->DirPort_set; } /** Return 1 if we want to allow controllers to ask us directory * requests via the controller interface, which doesn't require * having any separate port open. */ int -directory_permits_controller_requests(or_options_t *options) +directory_permits_controller_requests(const or_options_t *options) { - return options->DirPort != 0; + return options->DirPort_set; } /** Return 1 if we have no need to fetch new descriptors. This generally @@ -1265,7 +1301,8 @@ directory_permits_controller_requests(or_options_t *options) * lately. */ int -directory_too_idle_to_fetch_descriptors(or_options_t *options, time_t now) +directory_too_idle_to_fetch_descriptors(const or_options_t *options, + time_t now) { return !directory_caches_dir_info(options) && !options->FetchUselessDescriptors && @@ -1288,7 +1325,8 @@ static cached_dir_t cached_runningrouters; * cached_dir_t. */ static digestmap_t *cached_v2_networkstatus = NULL; -/** Map from flavor name to the v3 consensuses that we're currently serving. */ +/** Map from flavor name to the cached_dir_t for the v3 consensuses that we're + * currently serving. */ static strmap_t *cached_consensuses = NULL; /** Possibly replace the contents of <b>d</b> with the value of @@ -1532,11 +1570,11 @@ dirserv_pick_cached_dir_obj(cached_dir_t *cache_src, cached_dir_t *auth_src, time_t dirty, cached_dir_t *(*regenerate)(void), const char *name, - authority_type_t auth_type) + dirinfo_type_t auth_type) { - or_options_t *options = get_options(); - int authority = (auth_type == V1_AUTHORITY && authdir_mode_v1(options)) || - (auth_type == V2_AUTHORITY && authdir_mode_v2(options)); + const or_options_t *options = get_options(); + int authority = (auth_type == V1_DIRINFO && authdir_mode_v1(options)) || + (auth_type == V2_DIRINFO && authdir_mode_v2(options)); if (!authority || authdir_mode_bridge(options)) { return cache_src; @@ -1565,7 +1603,7 @@ dirserv_get_directory(void) return dirserv_pick_cached_dir_obj(cached_directory, the_directory, the_directory_is_dirty, dirserv_regenerate_directory, - "v1 server directory", V1_AUTHORITY); + "v1 server directory", V1_DIRINFO); } /** Only called by v1 auth dirservers. @@ -1608,7 +1646,7 @@ generate_runningrouters(void) char digest[DIGEST_LEN]; char published[ISO_TIME_LEN+1]; size_t len; - crypto_pk_env_t *private_key = get_server_identity_key(); + crypto_pk_t *private_key = get_server_identity_key(); char *identity_pkey; /* Identity key, DER64-encoded. */ size_t identity_pkey_len; @@ -1658,7 +1696,7 @@ dirserv_get_runningrouters(void) &cached_runningrouters, &the_runningrouters, runningrouters_is_dirty, generate_runningrouters, - "v1 network status list", V1_AUTHORITY); + "v1 network status list", V1_DIRINFO); } /** Return the latest downloaded consensus networkstatus in encoded, signed, @@ -1735,7 +1773,7 @@ static uint64_t total_exit_bandwidth = 0; /** Helper: estimate the uptime of a router given its stated uptime and the * amount of time since it last stated its stated uptime. */ static INLINE long -real_uptime(routerinfo_t *router, time_t now) +real_uptime(const routerinfo_t *router, time_t now) { if (now < router->cache_info.published_on) return router->uptime; @@ -1755,7 +1793,7 @@ dirserv_thinks_router_is_unreliable(time_t now, { if (need_uptime) { if (!enough_mtbf_info) { - /* XXX023 Once most authorities are on v3, we should change the rule from + /* XXX024 Once most authorities are on v3, we should change the rule from * "use uptime if we don't have mtbf data" to "don't advertise Stable on * v3 if we don't have enough mtbf data." Or maybe not, since if we ever * hit a point where we need to reset a lot of authorities at once, @@ -1789,7 +1827,8 @@ dirserv_thinks_router_is_unreliable(time_t now, * been set. */ static int -dirserv_thinks_router_is_hs_dir(routerinfo_t *router, time_t now) +dirserv_thinks_router_is_hs_dir(const routerinfo_t *router, + const node_t *node, time_t now) { long uptime; @@ -1816,8 +1855,8 @@ dirserv_thinks_router_is_hs_dir(routerinfo_t *router, time_t now) * *clients* suffering from bug 2722 are obsolete. The first version * to fix the bug was 0.2.2.25-alpha. */ return (router->wants_to_be_hs_dir && router->dir_port && - uptime > get_options()->MinUptimeHidServDirectoryV2 && - router->is_running); + uptime >= get_options()->MinUptimeHidServDirectoryV2 && + node->is_running); } /** Look through the routerlist, the Mean Time Between Failure history, and @@ -1836,7 +1875,7 @@ dirserv_compute_performance_thresholds(routerlist_t *rl) long *tks; double *mtbfs, *wfus; time_t now = time(NULL); - or_options_t *options = get_options(); + const or_options_t *options = get_options(); /* initialize these all here, in case there are no routers */ stable_uptime = 0; @@ -1866,19 +1905,22 @@ dirserv_compute_performance_thresholds(routerlist_t *rl) /* Weighted fractional uptime for each active router. */ wfus = tor_malloc(sizeof(double)*smartlist_len(rl->routers)); + nodelist_assert_ok(); + /* Now, fill in the arrays. */ - SMARTLIST_FOREACH(rl->routers, routerinfo_t *, ri, { - if (router_is_active(ri, now)) { + SMARTLIST_FOREACH_BEGIN(nodelist_get_list(), node_t *, node) { + routerinfo_t *ri = node->ri; + if (ri && router_is_active(ri, node, now)) { const char *id = ri->cache_info.identity_digest; uint32_t bw; - ri->is_exit = (!router_exit_policy_rejects_all(ri) && - exit_policy_is_general_exit(ri->exit_policy)); + node->is_exit = (!router_exit_policy_rejects_all(ri) && + exit_policy_is_general_exit(ri->exit_policy)); uptimes[n_active] = (uint32_t)real_uptime(ri, now); mtbfs[n_active] = rep_hist_get_stability(id, now); tks [n_active] = rep_hist_get_weighted_time_known(id, now); bandwidths[n_active] = bw = router_get_advertised_bandwidth(ri); total_bandwidth += bw; - if (ri->is_exit && !ri->is_bad_exit) { + if (node->is_exit && !node->is_bad_exit) { total_exit_bandwidth += bw; } else { bandwidths_excluding_exits[n_active_nonexit] = bw; @@ -1886,7 +1928,7 @@ dirserv_compute_performance_thresholds(routerlist_t *rl) } ++n_active; } - }); + } SMARTLIST_FOREACH_END(node); /* Now, compute thresholds. */ if (n_active) { @@ -1906,6 +1948,18 @@ dirserv_compute_performance_thresholds(routerlist_t *rl) if (guard_tk > TIME_KNOWN_TO_GUARANTEE_FAMILIAR) guard_tk = TIME_KNOWN_TO_GUARANTEE_FAMILIAR; + { + /* We can vote on a parameter for the minimum and maximum. */ + int32_t min_fast, max_fast; + min_fast = networkstatus_get_param(NULL, "FastFlagMinThreshold", + 0, 0, INT32_MAX); + max_fast = networkstatus_get_param(NULL, "FastFlagMaxThreshold", + INT32_MAX, min_fast, INT32_MAX); + if (fast_bandwidth < (uint32_t)min_fast) + fast_bandwidth = min_fast; + if (fast_bandwidth > (uint32_t)max_fast) + fast_bandwidth = max_fast; + } /* Protect sufficiently fast nodes from being pushed out of the set * of Fast nodes. */ if (options->AuthDirFastGuarantee && @@ -1915,15 +1969,17 @@ dirserv_compute_performance_thresholds(routerlist_t *rl) /* Now that we have a time-known that 7/8 routers are known longer than, * fill wfus with the wfu of every such "familiar" router. */ n_familiar = 0; - SMARTLIST_FOREACH(rl->routers, routerinfo_t *, ri, { - if (router_is_active(ri, now)) { + + SMARTLIST_FOREACH_BEGIN(nodelist_get_list(), node_t *, node) { + routerinfo_t *ri = node->ri; + if (ri && router_is_active(ri, node, now)) { const char *id = ri->cache_info.identity_digest; long tk = rep_hist_get_weighted_time_known(id, now); if (tk < guard_tk) continue; wfus[n_familiar++] = rep_hist_get_weighted_fractional_uptime(id, now); } - }); + } SMARTLIST_FOREACH_END(node); if (n_familiar) guard_wfu = median_double(wfus, n_familiar); if (guard_wfu > WFU_TO_GUARANTEE_GUARD) @@ -1939,7 +1995,7 @@ dirserv_compute_performance_thresholds(routerlist_t *rl) log(LOG_INFO, LD_DIRSERV, "Cutoffs: For Stable, %lu sec uptime, %lu sec MTBF. " "For Fast: %lu bytes/sec. " - "For Guard: WFU %.03lf%%, time-known %lu sec, " + "For Guard: WFU %.03f%%, time-known %lu sec, " "and bandwidth %lu or %lu bytes/sec. We%s have enough stability data.", (unsigned long)stable_uptime, (unsigned long)stable_mtbf, @@ -1994,24 +2050,20 @@ version_from_platform(const char *platform) */ int routerstatus_format_entry(char *buf, size_t buf_len, - routerstatus_t *rs, const char *version, + const routerstatus_t *rs, const char *version, routerstatus_format_type_t format) { int r; - struct in_addr in; char *cp; char *summary; char published[ISO_TIME_LEN+1]; - char ipaddr[INET_NTOA_BUF_LEN]; char identity64[BASE64_DIGEST_LEN+1]; char digest64[BASE64_DIGEST_LEN+1]; format_iso_time(published, rs->published_on); digest_to_base64(identity64, rs->identity_digest); digest_to_base64(digest64, rs->descriptor_digest); - in.s_addr = htonl(rs->addr); - tor_inet_ntoa(&in, ipaddr, sizeof(ipaddr)); r = tor_snprintf(buf, buf_len, "r %s %s %s%s%s %s %d %d\n", @@ -2020,7 +2072,7 @@ routerstatus_format_entry(char *buf, size_t buf_len, (format==NS_V3_CONSENSUS_MICRODESC)?"":digest64, (format==NS_V3_CONSENSUS_MICRODESC)?"":" ", published, - ipaddr, + fmt_addr32(rs->addr), (int)rs->or_port, (int)rs->dir_port); if (r<0) { @@ -2048,7 +2100,7 @@ routerstatus_format_entry(char *buf, size_t buf_len, rs->is_possible_guard?" Guard":"", rs->is_hs_dir?" HSDir":"", rs->is_named?" Named":"", - rs->is_running?" Running":"", + rs->is_flagged_running?" Running":"", rs->is_stable?" Stable":"", rs->is_unnamed?" Unnamed":"", rs->is_v2_dir?" V2Dir":"", @@ -2070,7 +2122,7 @@ routerstatus_format_entry(char *buf, size_t buf_len, } if (format != NS_V2) { - routerinfo_t* desc = router_get_by_digest(rs->identity_digest); + const routerinfo_t* desc = router_get_by_id_digest(rs->identity_digest); uint32_t bw; if (format != NS_CONTROL_PORT) { @@ -2166,6 +2218,8 @@ _compare_routerinfo_by_ip_and_bw(const void **a, const void **b) routerinfo_t *first = *(routerinfo_t **)a, *second = *(routerinfo_t **)b; int first_is_auth, second_is_auth; uint32_t bw_first, bw_second; + const node_t *node_first, *node_second; + int first_is_running, second_is_running; /* we return -1 if first should appear before second... that is, * if first is a better router. */ @@ -2188,9 +2242,14 @@ _compare_routerinfo_by_ip_and_bw(const void **a, const void **b) else if (!first_is_auth && second_is_auth) return 1; - else if (first->is_running && !second->is_running) + node_first = node_get_by_id(first->cache_info.identity_digest); + node_second = node_get_by_id(second->cache_info.identity_digest); + first_is_running = node_first && node_first->is_running; + second_is_running = node_second && node_second->is_running; + + if (first_is_running && !second_is_running) return -1; - else if (!first->is_running && second->is_running) + else if (!first_is_running && second_is_running) return 1; bw_first = router_get_advertised_bandwidth(first); @@ -2214,9 +2273,9 @@ _compare_routerinfo_by_ip_and_bw(const void **a, const void **b) static digestmap_t * get_possible_sybil_list(const smartlist_t *routers) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); digestmap_t *omit_as_sybil; - smartlist_t *routers_by_ip = smartlist_create(); + smartlist_t *routers_by_ip = smartlist_new(); uint32_t last_addr; int addr_count; /* Allow at most this number of Tor servers on a single IP address, ... */ @@ -2234,8 +2293,7 @@ get_possible_sybil_list(const smartlist_t *routers) last_addr = 0; addr_count = 0; - SMARTLIST_FOREACH(routers_by_ip, routerinfo_t *, ri, - { + SMARTLIST_FOREACH_BEGIN(routers_by_ip, routerinfo_t *, ri) { if (last_addr != ri->addr) { last_addr = ri->addr; addr_count = 1; @@ -2244,7 +2302,7 @@ get_possible_sybil_list(const smartlist_t *routers) addr_count > max_with_same_addr_on_authority) digestmap_set(omit_as_sybil, ri->cache_info.identity_digest, ri); } - }); + } SMARTLIST_FOREACH_END(ri); smartlist_free(routers_by_ip); return omit_as_sybil; @@ -2262,7 +2320,7 @@ is_router_version_good_for_possible_guard(const char *platform) tor_version_t router_version; - /* XXX023 This block should be extracted into its own function. */ + /* XXX024 This block should be extracted into its own function. */ /* XXXX Begin code copied from tor_version_as_new_as (in routerparse.c) */ { char *s, *s2, *start; @@ -2270,7 +2328,8 @@ is_router_version_good_for_possible_guard(const char *platform) tor_assert(platform); - if (strcmpstart(platform,"Tor ")) /* nonstandard Tor; be safe and say yes */ + /* nonstandard Tor; be safe and say yes */ + if (strcmpstart(platform,"Tor ")) return 1; start = (char *)eat_whitespace(platform+3); @@ -2327,7 +2386,9 @@ is_router_version_good_for_possible_guard(const char *platform) */ void set_routerstatus_from_routerinfo(routerstatus_t *rs, - routerinfo_t *ri, time_t now, + node_t *node, + routerinfo_t *ri, + time_t now, int naming, int listbadexits, int listbaddirs, int vote_on_hsdirs) { @@ -2342,50 +2403,50 @@ set_routerstatus_from_routerinfo(routerstatus_t *rs, router_digest_is_trusted_dir(ri->cache_info.identity_digest); /* Already set by compute_performance_thresholds. */ - rs->is_exit = ri->is_exit; - rs->is_stable = ri->is_stable = - router_is_active(ri, now) && + rs->is_exit = node->is_exit; + rs->is_stable = node->is_stable = + router_is_active(ri, node, now) && !dirserv_thinks_router_is_unreliable(now, ri, 1, 0) && !unstable_version; - rs->is_fast = ri->is_fast = - router_is_active(ri, now) && + rs->is_fast = node->is_fast = + router_is_active(ri, node, now) && !dirserv_thinks_router_is_unreliable(now, ri, 0, 1); - rs->is_running = ri->is_running; /* computed above */ + rs->is_flagged_running = node->is_running; /* computed above */ if (naming) { uint32_t name_status = dirserv_get_name_status( - ri->cache_info.identity_digest, ri->nickname); + node->identity, ri->nickname); rs->is_named = (naming && (name_status & FP_NAMED)) ? 1 : 0; rs->is_unnamed = (naming && (name_status & FP_UNNAMED)) ? 1 : 0; } - rs->is_valid = ri->is_valid; + rs->is_valid = node->is_valid; - if (rs->is_fast && + if (node->is_fast && ((options->AuthDirGuardBWGuarantee && routerbw >= options->AuthDirGuardBWGuarantee) || routerbw >= MIN(guard_bandwidth_including_exits, guard_bandwidth_excluding_exits)) && - (options->GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays || - is_router_version_good_for_possible_guard(ri->platform))) { + is_router_version_good_for_possible_guard(ri->platform)) { long tk = rep_hist_get_weighted_time_known( - ri->cache_info.identity_digest, now); + node->identity, now); double wfu = rep_hist_get_weighted_fractional_uptime( - ri->cache_info.identity_digest, now); + node->identity, now); rs->is_possible_guard = (wfu >= guard_wfu && tk >= guard_tk) ? 1 : 0; } else { rs->is_possible_guard = 0; } - rs->is_bad_directory = listbaddirs && ri->is_bad_directory; - rs->is_bad_exit = listbadexits && ri->is_bad_exit; - ri->is_hs_dir = dirserv_thinks_router_is_hs_dir(ri, now); - rs->is_hs_dir = vote_on_hsdirs && ri->is_hs_dir; + + rs->is_bad_directory = listbaddirs && node->is_bad_directory; + rs->is_bad_exit = listbadexits && node->is_bad_exit; + node->is_hs_dir = dirserv_thinks_router_is_hs_dir(ri, node, now); + rs->is_hs_dir = vote_on_hsdirs && node->is_hs_dir; rs->is_v2_dir = ri->dir_port != 0; if (!strcasecmp(ri->nickname, UNNAMED_ROUTER_NICKNAME)) rs->is_named = rs->is_unnamed = 0; rs->published_on = ri->cache_info.published_on; - memcpy(rs->identity_digest, ri->cache_info.identity_digest, DIGEST_LEN); + memcpy(rs->identity_digest, node->identity, DIGEST_LEN); memcpy(rs->descriptor_digest, ri->cache_info.signed_descriptor_digest, DIGEST_LEN); rs->addr = ri->addr; @@ -2402,7 +2463,7 @@ static void clear_status_flags_on_sybil(routerstatus_t *rs) { rs->is_authority = rs->is_exit = rs->is_stable = rs->is_fast = - rs->is_running = rs->is_named = rs->is_valid = rs->is_v2_dir = + rs->is_flagged_running = rs->is_named = rs->is_valid = rs->is_v2_dir = rs->is_hs_dir = rs->is_possible_guard = rs->is_bad_exit = rs->is_bad_directory = 0; /* FFFF we might want some mechanism to check later on if we @@ -2410,18 +2471,6 @@ clear_status_flags_on_sybil(routerstatus_t *rs) * forget to add it to this clause. */ } -/** Clear all the status flags in routerinfo <b>router</b>. We put this - * function here because it's eerily similar to - * clear_status_flags_on_sybil() above. One day we should merge them. */ -void -router_clear_status_flags(routerinfo_t *router) -{ - router->is_valid = router->is_running = router->is_hs_dir = - router->is_fast = router->is_stable = - router->is_possible_guard = router->is_exit = - router->is_bad_exit = router->is_bad_directory = 0; -} - /** * Helper function to parse out a line in the measured bandwidth file * into a measured_bw_line_t output structure. Returns -1 on failure @@ -2539,7 +2588,7 @@ dirserv_read_measured_bandwidths(const char *from_file, smartlist_t *routerstatuses) { char line[256]; - FILE *fp = fopen(from_file, "r"); + FILE *fp = tor_fopen_cloexec(from_file, "r"); int applied_lines = 0; time_t file_time; int ok; @@ -2596,10 +2645,10 @@ dirserv_read_measured_bandwidths(const char *from_file, /** Return a new networkstatus_t* containing our current opinion. (For v3 * authorities) */ networkstatus_t * -dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, +dirserv_generate_networkstatus_vote_obj(crypto_pk_t *private_key, authority_cert_t *cert) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); networkstatus_t *v3_out = NULL; uint32_t addr; char *hostname = NULL, *client_versions = NULL, *server_versions = NULL; @@ -2657,23 +2706,26 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, dirserv_compute_performance_thresholds(rl); - routers = smartlist_create(); + routers = smartlist_new(); smartlist_add_all(routers, rl->routers); routers_sort_by_identity(routers); omit_as_sybil = get_possible_sybil_list(routers); - routerstatuses = smartlist_create(); - microdescriptors = smartlist_create(); + routerstatuses = smartlist_new(); + microdescriptors = smartlist_new(); SMARTLIST_FOREACH_BEGIN(routers, routerinfo_t *, ri) { if (ri->cache_info.published_on >= cutoff) { routerstatus_t *rs; vote_routerstatus_t *vrs; microdesc_t *md; + node_t *node = node_get_mutable_by_id(ri->cache_info.identity_digest); + if (!node) + continue; vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; - set_routerstatus_from_routerinfo(rs, ri, now, + set_routerstatus_from_routerinfo(rs, node, ri, now, naming, listbadexits, listbaddirs, vote_on_hsdirs); @@ -2681,7 +2733,7 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, clear_status_flags_on_sybil(rs); if (!vote_on_reachability) - rs->is_running = 0; + rs->is_flagged_running = 0; vrs->version = version_from_platform(ri->platform); md = dirvote_create_microdescriptor(ri); @@ -2750,7 +2802,7 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, v3_out->client_versions = client_versions; v3_out->server_versions = server_versions; - v3_out->known_flags = smartlist_create(); + v3_out->known_flags = smartlist_new(); smartlist_split_string(v3_out->known_flags, "Authority Exit Fast Guard Stable V2Dir Valid", 0, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); @@ -2769,7 +2821,7 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, smartlist_sort_strings(v3_out->known_flags); if (options->ConsensusParams) { - v3_out->net_params = smartlist_create(); + v3_out->net_params = smartlist_new(); smartlist_split_string(v3_out->net_params, options->ConsensusParams, NULL, 0, 0); smartlist_sort_strings(v3_out->net_params); @@ -2778,7 +2830,7 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t)); voter->nickname = tor_strdup(options->Nickname); memcpy(voter->identity_digest, identity_digest, DIGEST_LEN); - voter->sigs = smartlist_create(); + voter->sigs = smartlist_new(); voter->address = hostname; voter->addr = addr; voter->dir_port = router_get_advertised_dir_port(options, 0); @@ -2794,7 +2846,7 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, } } - v3_out->voters = smartlist_create(); + v3_out->voters = smartlist_new(); smartlist_add(v3_out->voters, voter); v3_out->cert = authority_cert_dup(cert); v3_out->routerstatus_list = routerstatuses; @@ -2815,14 +2867,12 @@ generate_v2_networkstatus_opinion(void) char *status = NULL, *client_versions = NULL, *server_versions = NULL, *identity_pkey = NULL, *hostname = NULL; char *outp, *endp; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); char fingerprint[FINGERPRINT_LEN+1]; - char ipaddr[INET_NTOA_BUF_LEN]; char published[ISO_TIME_LEN+1]; char digest[DIGEST_LEN]; - struct in_addr in; uint32_t addr; - crypto_pk_env_t *private_key; + crypto_pk_t *private_key; routerlist_t *rl = router_get_routerlist(); time_t now = time(NULL); time_t cutoff = now - ROUTER_MAX_AGE_TO_PUBLISH; @@ -2842,8 +2892,6 @@ generate_v2_networkstatus_opinion(void) log_warn(LD_NET, "Couldn't resolve my hostname"); goto done; } - in.s_addr = htonl(addr); - tor_inet_ntoa(&in, ipaddr, sizeof(ipaddr)); format_iso_time(published, now); @@ -2861,14 +2909,12 @@ generate_v2_networkstatus_opinion(void) goto done; } - contact = get_options()->ContactInfo; + contact = options->ContactInfo; if (!contact) contact = "(none)"; if (versioning) { - size_t v_len = 64+strlen(client_versions)+strlen(server_versions); - version_lines = tor_malloc(v_len); - tor_snprintf(version_lines, v_len, + tor_asprintf(&version_lines, "client-versions %s\nserver-versions %s\n", client_versions, server_versions); } else { @@ -2889,7 +2935,7 @@ generate_v2_networkstatus_opinion(void) "dir-options%s%s%s%s\n" "%s" /* client version line, server version line. */ "dir-signing-key\n%s", - hostname, ipaddr, + hostname, fmt_addr32(addr), (int)router_get_advertised_dir_port(options, 0), fingerprint, contact, @@ -2911,18 +2957,22 @@ generate_v2_networkstatus_opinion(void) dirserv_compute_performance_thresholds(rl); - routers = smartlist_create(); + routers = smartlist_new(); smartlist_add_all(routers, rl->routers); routers_sort_by_identity(routers); omit_as_sybil = get_possible_sybil_list(routers); - SMARTLIST_FOREACH(routers, routerinfo_t *, ri, { + SMARTLIST_FOREACH_BEGIN(routers, routerinfo_t *, ri) { if (ri->cache_info.published_on >= cutoff) { routerstatus_t rs; char *version = version_from_platform(ri->platform); - - set_routerstatus_from_routerinfo(&rs, ri, now, + node_t *node = node_get_mutable_by_id(ri->cache_info.identity_digest); + if (!node) { + tor_free(version); + continue; + } + set_routerstatus_from_routerinfo(&rs, node, ri, now, naming, listbadexits, listbaddirs, vote_on_hsdirs); @@ -2937,10 +2987,10 @@ generate_v2_networkstatus_opinion(void) tor_free(version); outp += strlen(outp); } - }); + } SMARTLIST_FOREACH_END(ri); if (tor_snprintf(outp, endp-outp, "directory-signature %s\n", - get_options()->Nickname)<0) { + options->Nickname)<0) { log_warn(LD_BUG, "Unable to write signature line."); goto done; } @@ -3006,7 +3056,7 @@ dirserv_get_networkstatus_v2_fingerprints(smartlist_t *result, if (!strcmp(key,"authority")) { if (authdir_mode_v2(get_options())) { - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); if (me) smartlist_add(result, tor_memdup(me->cache_info.identity_digest, DIGEST_LEN)); @@ -3025,7 +3075,7 @@ dirserv_get_networkstatus_v2_fingerprints(smartlist_t *result, } else { SMARTLIST_FOREACH(router_get_trusted_dir_servers(), trusted_dir_server_t *, ds, - if (ds->type & V2_AUTHORITY) + if (ds->type & V2_DIRINFO) smartlist_add(result, tor_memdup(ds->digest, DIGEST_LEN))); } smartlist_sort_digests(result); @@ -3048,15 +3098,14 @@ dirserv_get_networkstatus_v2(smartlist_t *result, const char *key) { cached_dir_t *cached; - smartlist_t *fingerprints = smartlist_create(); + smartlist_t *fingerprints = smartlist_new(); tor_assert(result); if (!cached_v2_networkstatus) cached_v2_networkstatus = digestmap_new(); dirserv_get_networkstatus_v2_fingerprints(fingerprints, key); - SMARTLIST_FOREACH(fingerprints, const char *, fp, - { + SMARTLIST_FOREACH_BEGIN(fingerprints, const char *, fp) { if (router_digest_is_me(fp) && should_generate_v2_networkstatus()) generate_v2_networkstatus_opinion(); cached = digestmap_get(cached_v2_networkstatus, fp); @@ -3068,7 +3117,7 @@ dirserv_get_networkstatus_v2(smartlist_t *result, log_info(LD_DIRSERV, "Don't know about any network status with " "fingerprint '%s'", hexbuf); } - }); + } SMARTLIST_FOREACH_END(fp); SMARTLIST_FOREACH(fingerprints, char *, cp, tor_free(cp)); smartlist_free(fingerprints); } @@ -3094,7 +3143,7 @@ dirserv_get_routerdesc_fingerprints(smartlist_t *fps_out, const char *key, /* Treat "all" requests as if they were unencrypted */ for_unencrypted_conn = 1; } else if (!strcmp(key, "authority")) { - routerinfo_t *ri = router_get_my_routerinfo(); + const routerinfo_t *ri = router_get_my_routerinfo(); if (ri) smartlist_add(fps_out, tor_memdup(ri->cache_info.identity_digest, DIGEST_LEN)); @@ -3114,8 +3163,8 @@ dirserv_get_routerdesc_fingerprints(smartlist_t *fps_out, const char *key, if (for_unencrypted_conn) { /* Remove anything that insists it not be sent unencrypted. */ - SMARTLIST_FOREACH(fps_out, char *, cp, { - signed_descriptor_t *sd; + SMARTLIST_FOREACH_BEGIN(fps_out, char *, cp) { + const signed_descriptor_t *sd; if (by_id) sd = get_signed_descriptor_by_fp(cp,is_extrainfo,0); else if (is_extrainfo) @@ -3126,7 +3175,7 @@ dirserv_get_routerdesc_fingerprints(smartlist_t *fps_out, const char *key, tor_free(cp); SMARTLIST_DEL_CURRENT(fps_out, cp); } - }); + } SMARTLIST_FOREACH_END(cp); } if (!smartlist_len(fps_out)) { @@ -3165,11 +3214,11 @@ dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, SMARTLIST_FOREACH(rl->routers, routerinfo_t *, r, smartlist_add(descs_out, &(r->cache_info))); } else if (!strcmp(key, "/tor/server/authority")) { - routerinfo_t *ri = router_get_my_routerinfo(); + const routerinfo_t *ri = router_get_my_routerinfo(); if (ri) - smartlist_add(descs_out, &(ri->cache_info)); + smartlist_add(descs_out, (void*) &(ri->cache_info)); } else if (!strcmpstart(key, "/tor/server/d/")) { - smartlist_t *digests = smartlist_create(); + smartlist_t *digests = smartlist_new(); key += strlen("/tor/server/d/"); dir_split_resource_into_fingerprints(key, digests, NULL, DSR_HEX|DSR_SORT_UNIQ); @@ -3182,28 +3231,27 @@ dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, SMARTLIST_FOREACH(digests, char *, d, tor_free(d)); smartlist_free(digests); } else if (!strcmpstart(key, "/tor/server/fp/")) { - smartlist_t *digests = smartlist_create(); + smartlist_t *digests = smartlist_new(); time_t cutoff = time(NULL) - ROUTER_MAX_AGE_TO_PUBLISH; key += strlen("/tor/server/fp/"); dir_split_resource_into_fingerprints(key, digests, NULL, DSR_HEX|DSR_SORT_UNIQ); - SMARTLIST_FOREACH(digests, const char *, d, - { + SMARTLIST_FOREACH_BEGIN(digests, const char *, d) { if (router_digest_is_me(d)) { /* make sure desc_routerinfo exists */ - routerinfo_t *ri = router_get_my_routerinfo(); + const routerinfo_t *ri = router_get_my_routerinfo(); if (ri) - smartlist_add(descs_out, &(ri->cache_info)); + smartlist_add(descs_out, (void*) &(ri->cache_info)); } else { - routerinfo_t *ri = router_get_by_digest(d); + const routerinfo_t *ri = router_get_by_id_digest(d); /* Don't actually serve a descriptor that everyone will think is * expired. This is an (ugly) workaround to keep buggy 0.1.1.10 * Tors from downloading descriptors that they will throw away. */ if (ri && ri->cache_info.published_on > cutoff) - smartlist_add(descs_out, &(ri->cache_info)); + smartlist_add(descs_out, (void*) &(ri->cache_info)); } - }); + } SMARTLIST_FOREACH_END(d); SMARTLIST_FOREACH(digests, char *, d, tor_free(d)); smartlist_free(digests); } else { @@ -3222,53 +3270,49 @@ dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, * router listening at <b>address</b>:<b>or_port</b>, and has yielded * a certificate with digest <b>digest_rcvd</b>. * - * Also, if as_advertised is 1, then inform the reachability checker - * that we could get to this guy. + * Inform the reachability checker that we could get to this guy. */ void dirserv_orconn_tls_done(const char *address, uint16_t or_port, - const char *digest_rcvd, - int as_advertised) + const char *digest_rcvd) { - routerlist_t *rl = router_get_routerlist(); + routerinfo_t *ri; time_t now = time(NULL); - int bridge_auth = authdir_mode_bridge(get_options()); tor_assert(address); tor_assert(digest_rcvd); - /* XXX023 Doing a loop like this is stupid. We should just look up the - * router by digest_rcvd, and see if address, orport, and as_advertised - * match up. -NM */ - SMARTLIST_FOREACH_BEGIN(rl->routers, routerinfo_t *, ri) { - if (!strcasecmp(address, ri->address) && or_port == ri->or_port && - as_advertised && - fast_memeq(ri->cache_info.identity_digest, digest_rcvd, DIGEST_LEN)) { - /* correct digest. mark this router reachable! */ - if (!bridge_auth || ri->purpose == ROUTER_PURPOSE_BRIDGE) { - tor_addr_t addr, *addrp=NULL; - log_info(LD_DIRSERV, "Found router %s to be reachable at %s:%d. Yay.", - router_describe(ri), - address, ri->or_port); - if (tor_addr_from_str(&addr, ri->address) != -1) - addrp = &addr; - else - log_warn(LD_BUG, "Couldn't parse IP address \"%s\"", ri->address); - rep_hist_note_router_reachable(digest_rcvd, addrp, or_port, now); - ri->last_reachable = now; - } + ri = router_get_mutable_by_digest(digest_rcvd); + + if (ri == NULL) + return; + + if (!strcasecmp(address, ri->address) && or_port == ri->or_port) { + /* Found the right router. */ + if (!authdir_mode_bridge(get_options()) || + ri->purpose == ROUTER_PURPOSE_BRIDGE) { + /* This is a bridge or we're not a bridge authorititative -- + mark it as reachable. */ + tor_addr_t addr, *addrp=NULL; + log_info(LD_DIRSERV, "Found router %s to be reachable at %s:%d. Yay.", + router_describe(ri), + address, ri->or_port); + if (tor_addr_parse(&addr, ri->address) != -1) + addrp = &addr; + else + log_warn(LD_BUG, "Couldn't parse IP address \"%s\"", ri->address); + rep_hist_note_router_reachable(digest_rcvd, addrp, or_port, now); + ri->last_reachable = now; } - } SMARTLIST_FOREACH_END(ri); - /* FFFF Maybe we should reinstate the code that dumps routers with the same - * addr/port but with nonmatching keys, but instead of dumping, we should - * skip testing. */ + } } /** Called when we, as an authority, receive a new router descriptor either as * an upload or a download. Used to decide whether to relaunch reachability * testing for the server. */ int -dirserv_should_launch_reachability_test(routerinfo_t *ri, routerinfo_t *ri_old) +dirserv_should_launch_reachability_test(const routerinfo_t *ri, + const routerinfo_t *ri_old) { if (!authdir_mode_handles_descs(get_options(), ri->purpose)) return 0; @@ -3355,7 +3399,7 @@ lookup_cached_dir_by_fp(const char *fp) d = strmap_get(cached_consensuses, "ns"); else if (memchr(fp, '\0', DIGEST_LEN) && cached_consensuses && (d = strmap_get(cached_consensuses, fp))) { - /* this here interface is a nasty hack XXXX023 */; + /* this here interface is a nasty hack XXXX024 */; } else if (router_digest_is_me(fp) && the_v2_networkstatus) d = the_v2_networkstatus; else if (cached_v2_networkstatus) @@ -3373,8 +3417,7 @@ int dirserv_remove_old_statuses(smartlist_t *fps, time_t cutoff) { int found_any = 0; - SMARTLIST_FOREACH(fps, char *, digest, - { + SMARTLIST_FOREACH_BEGIN(fps, char *, digest) { cached_dir_t *d = lookup_cached_dir_by_fp(digest); if (!d) continue; @@ -3383,7 +3426,7 @@ dirserv_remove_old_statuses(smartlist_t *fps, time_t cutoff) tor_free(digest); SMARTLIST_DEL_CURRENT(fps, digest); } - }); + } SMARTLIST_FOREACH_END(digest); return found_any; } @@ -3392,7 +3435,7 @@ dirserv_remove_old_statuses(smartlist_t *fps, time_t cutoff) * its extra-info document if <b>extrainfo</b> is true. Return * NULL if not found or if the descriptor is older than * <b>publish_cutoff</b>. */ -static signed_descriptor_t * +static const signed_descriptor_t * get_signed_descriptor_by_fp(const char *fp, int extrainfo, time_t publish_cutoff) { @@ -3402,7 +3445,7 @@ get_signed_descriptor_by_fp(const char *fp, int extrainfo, else return &(router_get_my_routerinfo()->cache_info); } else { - routerinfo_t *ri = router_get_by_digest(fp); + const routerinfo_t *ri = router_get_by_id_digest(fp); if (ri && ri->cache_info.published_on > publish_cutoff) { if (extrainfo) @@ -3422,7 +3465,7 @@ int dirserv_have_any_serverdesc(smartlist_t *fps, int spool_src) { time_t publish_cutoff = time(NULL)-ROUTER_MAX_AGE_TO_PUBLISH; - SMARTLIST_FOREACH(fps, const char *, fp, { + SMARTLIST_FOREACH_BEGIN(fps, const char *, fp) { switch (spool_src) { case DIR_SPOOL_EXTRA_BY_DIGEST: @@ -3438,7 +3481,7 @@ dirserv_have_any_serverdesc(smartlist_t *fps, int spool_src) return 1; break; } - }); + } SMARTLIST_FOREACH_END(fp); return 0; } @@ -3470,7 +3513,7 @@ dirserv_estimate_data_size(smartlist_t *fps, int is_serverdescs, tor_assert(fps); if (is_serverdescs) { int n = smartlist_len(fps); - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); result = (me?me->cache_info.signed_descriptor_len:2048) * n; if (compressed) result /= 2; /* observed compressibility is between 35 and 55%. */ @@ -3524,20 +3567,19 @@ connection_dirserv_finish_spooling(dir_connection_t *conn) static int connection_dirserv_add_servers_to_outbuf(dir_connection_t *conn) { -#ifdef TRACK_SERVED_TIME - time_t now = time(NULL); -#endif int by_fp = (conn->dir_spool_src == DIR_SPOOL_SERVER_BY_FP || conn->dir_spool_src == DIR_SPOOL_EXTRA_BY_FP); int extra = (conn->dir_spool_src == DIR_SPOOL_EXTRA_BY_FP || conn->dir_spool_src == DIR_SPOOL_EXTRA_BY_DIGEST); time_t publish_cutoff = time(NULL)-ROUTER_MAX_AGE_TO_PUBLISH; + const or_options_t *options = get_options(); + while (smartlist_len(conn->fingerprint_stack) && - buf_datalen(conn->_base.outbuf) < DIRSERV_BUFFER_MIN) { + connection_get_outbuf_len(TO_CONN(conn)) < DIRSERV_BUFFER_MIN) { const char *body; char *fp = smartlist_pop_last(conn->fingerprint_stack); - signed_descriptor_t *sd = NULL; + const signed_descriptor_t *sd = NULL; if (by_fp) { sd = get_signed_descriptor_by_fp(fp, extra, publish_cutoff); } else { @@ -3554,12 +3596,20 @@ connection_dirserv_add_servers_to_outbuf(dir_connection_t *conn) * unknown bridge descriptor has shown up between then and now. */ continue; } -#ifdef TRACK_SERVED_TIME - sd->last_served_at = now; -#endif + + /** If we are the bridge authority and the descriptor is a bridge + * descriptor, remember that we served this descriptor for desc stats. */ + if (options->BridgeAuthoritativeDir && by_fp) { + const routerinfo_t *router = + router_get_by_id_digest(sd->identity_digest); + /* router can be NULL here when the bridge auth is asked for its own + * descriptor. */ + if (router && router->purpose == ROUTER_PURPOSE_BRIDGE) + rep_hist_note_desc_served(sd->identity_digest); + } body = signed_descriptor_get_body(sd); if (conn->zlib_state) { - /* XXXX022 This 'last' business should actually happen on the last + /* XXXX024 This 'last' business should actually happen on the last * routerinfo, not on the last fingerprint. */ int last = ! smartlist_len(conn->fingerprint_stack); connection_write_to_buf_zlib(body, sd->signed_descriptor_len, conn, @@ -3595,14 +3645,14 @@ connection_dirserv_add_microdescs_to_outbuf(dir_connection_t *conn) { microdesc_cache_t *cache = get_microdesc_cache(); while (smartlist_len(conn->fingerprint_stack) && - buf_datalen(conn->_base.outbuf) < DIRSERV_BUFFER_MIN) { + connection_get_outbuf_len(TO_CONN(conn)) < DIRSERV_BUFFER_MIN) { char *fp256 = smartlist_pop_last(conn->fingerprint_stack); microdesc_t *md = microdesc_cache_lookup_by_digest256(cache, fp256); tor_free(fp256); if (!md) continue; if (conn->zlib_state) { - /* XXXX022 This 'last' business should actually happen on the last + /* XXXX024 This 'last' business should actually happen on the last * routerinfo, not on the last fingerprint. */ int last = !smartlist_len(conn->fingerprint_stack); connection_write_to_buf_zlib(md->body, md->bodylen, conn, last); @@ -3634,7 +3684,7 @@ connection_dirserv_add_dir_bytes_to_outbuf(dir_connection_t *conn) ssize_t bytes; int64_t remaining; - bytes = DIRSERV_BUFFER_MIN - buf_datalen(conn->_base.outbuf); + bytes = DIRSERV_BUFFER_MIN - connection_get_outbuf_len(TO_CONN(conn)); tor_assert(bytes > 0); tor_assert(conn->cached_dir); if (bytes < 8192) @@ -3673,7 +3723,7 @@ static int connection_dirserv_add_networkstatus_bytes_to_outbuf(dir_connection_t *conn) { - while (buf_datalen(conn->_base.outbuf) < DIRSERV_BUFFER_MIN) { + while (connection_get_outbuf_len(TO_CONN(conn)) < DIRSERV_BUFFER_MIN) { if (conn->cached_dir) { int uncompressing = (conn->zlib_state != NULL); int r = connection_dirserv_add_dir_bytes_to_outbuf(conn); @@ -3719,7 +3769,7 @@ connection_dirserv_flushed_some(dir_connection_t *conn) { tor_assert(conn->_base.state == DIR_CONN_STATE_SERVER_WRITING); - if (buf_datalen(conn->_base.outbuf) >= DIRSERV_BUFFER_MIN) + if (connection_get_outbuf_len(TO_CONN(conn)) >= DIRSERV_BUFFER_MIN) return 0; switch (conn->dir_spool_src) { diff --git a/src/or/dirserv.h b/src/or/dirserv.h index 569abfca2e..22269b2009 100644 --- a/src/or/dirserv.h +++ b/src/or/dirserv.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -52,11 +52,9 @@ MAX_V_LINE_LEN \ ) -#define UNNAMED_ROUTER_NICKNAME "Unnamed" - int connection_dirserv_flushed_some(dir_connection_t *conn); -int dirserv_add_own_fingerprint(const char *nickname, crypto_pk_env_t *pk); +int dirserv_add_own_fingerprint(const char *nickname, crypto_pk_t *pk); int dirserv_load_fingerprint_file(void); void dirserv_free_fingerprint_list(void); const char *dirserv_get_nickname_by_digest(const char *digest); @@ -71,17 +69,19 @@ void dirserv_set_router_is_running(routerinfo_t *router, time_t now); int list_server_status_v1(smartlist_t *routers, char **router_status_out, int for_controller); int dirserv_dump_directory_to_string(char **dir_out, - crypto_pk_env_t *private_key); + crypto_pk_t *private_key); -int directory_fetches_from_authorities(or_options_t *options); -int directory_fetches_dir_info_early(or_options_t *options); -int directory_fetches_dir_info_later(or_options_t *options); -int directory_caches_v2_dir_info(or_options_t *options); +int directory_fetches_from_authorities(const or_options_t *options); +int directory_fetches_dir_info_early(const or_options_t *options); +int directory_fetches_dir_info_later(const or_options_t *options); +int directory_caches_v2_dir_info(const or_options_t *options); #define directory_caches_v1_dir_info(o) directory_caches_v2_dir_info(o) -int directory_caches_dir_info(or_options_t *options); -int directory_permits_begindir_requests(or_options_t *options); -int directory_permits_controller_requests(or_options_t *options); -int directory_too_idle_to_fetch_descriptors(or_options_t *options, time_t now); +int directory_caches_unknown_auth_certs(const or_options_t *options); +int directory_caches_dir_info(const or_options_t *options); +int directory_permits_begindir_requests(const or_options_t *options); +int directory_permits_controller_requests(const or_options_t *options); +int directory_too_idle_to_fetch_descriptors(const or_options_t *options, + time_t now); void directory_set_dirty(void); cached_dir_t *dirserv_get_directory(void); @@ -109,15 +109,20 @@ int dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, const char **msg); void dirserv_orconn_tls_done(const char *address, uint16_t or_port, - const char *digest_rcvd, - int as_advertised); -int dirserv_should_launch_reachability_test(routerinfo_t *ri, - routerinfo_t *ri_old); + const char *digest_rcvd); +int dirserv_should_launch_reachability_test(const routerinfo_t *ri, + const routerinfo_t *ri_old); void dirserv_single_reachability_test(time_t now, routerinfo_t *router); void dirserv_test_reachability(time_t now); int authdir_wants_to_reject_router(routerinfo_t *ri, const char **msg, - int complain); -int dirserv_would_reject_router(routerstatus_t *rs); + int complain, + int *valid_out); +uint32_t dirserv_router_get_status(const routerinfo_t *router, + const char **msg); +void dirserv_set_node_flags_from_authoritative_status(node_t *node, + uint32_t authstatus); + +int dirserv_would_reject_router(const routerstatus_t *rs); int dirserv_remove_old_statuses(smartlist_t *fps, time_t cutoff); int dirserv_have_any_serverdesc(smartlist_t *fps, int spool_src); int dirserv_have_any_microdesc(const smartlist_t *fps); @@ -126,7 +131,7 @@ size_t dirserv_estimate_data_size(smartlist_t *fps, int is_serverdescs, size_t dirserv_estimate_microdesc_size(const smartlist_t *fps, int compressed); int routerstatus_format_entry(char *buf, size_t buf_len, - routerstatus_t *rs, const char *platform, + const routerstatus_t *rs, const char *platform, routerstatus_format_type_t format); void dirserv_free_all(void); void cached_dir_decref(cached_dir_t *d); diff --git a/src/or/dirvote.c b/src/or/dirvote.c index a557fc782e..144859ae04 100644 --- a/src/or/dirvote.c +++ b/src/or/dirvote.c @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #define DIRVOTE_PRIVATE @@ -33,6 +33,7 @@ typedef struct pending_consensus_t { networkstatus_t *consensus; } pending_consensus_t; +/* DOCDOC dirvote_add_signatures_to_all_pending_consensuses */ static int dirvote_add_signatures_to_all_pending_consensuses( const char *detached_signatures_body, const char *source, @@ -53,7 +54,7 @@ static int dirvote_publish_consensus(void); static char *make_consensus_method_list(int low, int high, const char *sep); /** The highest consensus method that we currently support. */ -#define MAX_SUPPORTED_CONSENSUS_METHOD 11 +#define MAX_SUPPORTED_CONSENSUS_METHOD 13 /** Lowest consensus method that contains a 'directory-footer' marker */ #define MIN_METHOD_FOR_FOOTER 9 @@ -67,6 +68,14 @@ static char *make_consensus_method_list(int low, int high, const char *sep); /** Lowest consensus method that generates microdescriptors */ #define MIN_METHOD_FOR_MICRODESC 8 +/** Lowest consensus method that ensures a majority of authorities voted + * for a param. */ +#define MIN_METHOD_FOR_MAJORITY_PARAMS 12 + +/** Lowest consensus method where microdesc consensuses omit any entry + * with no microdesc. */ +#define MIN_METHOD_FOR_MANDATORY_MICRODESC 13 + /* ===== * Voting * =====*/ @@ -78,7 +87,7 @@ static char *make_consensus_method_list(int low, int high, const char *sep); * <b>v3_ns</b>, signed with our v3 signing key <b>private_signing_key</b>. * For v3 authorities. */ char * -format_networkstatus_vote(crypto_pk_env_t *private_signing_key, +format_networkstatus_vote(crypto_pk_t *private_signing_key, networkstatus_t *v3_ns) { size_t len; @@ -86,9 +95,7 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key, const char *client_versions = NULL, *server_versions = NULL; char *outp, *endp; char fingerprint[FINGERPRINT_LEN+1]; - char ipaddr[INET_NTOA_BUF_LEN]; char digest[DIGEST_LEN]; - struct in_addr in; uint32_t addr; routerlist_t *rl = router_get_routerlist(); char *version_lines = NULL; @@ -101,8 +108,6 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key, voter = smartlist_get(v3_ns->voters, 0); addr = voter->addr; - in.s_addr = htonl(addr); - tor_inet_ntoa(&in, ipaddr, sizeof(ipaddr)); base16_encode(fingerprint, sizeof(fingerprint), v3_ns->cert->cache_info.identity_digest, DIGEST_LEN); @@ -189,7 +194,8 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key, flags, params, voter->nickname, fingerprint, voter->address, - ipaddr, voter->dir_port, voter->or_port, voter->contact); + fmt_addr32(addr), voter->dir_port, voter->or_port, + voter->contact); if (r < 0) { log_err(LD_BUG, "Insufficient memory for network status line"); @@ -370,8 +376,7 @@ get_frequent_members(smartlist_t *out, smartlist_t *in, int min) { char *cur = NULL; int count = 0; - SMARTLIST_FOREACH(in, char *, cp, - { + SMARTLIST_FOREACH_BEGIN(in, char *, cp) { if (cur && !strcmp(cp, cur)) { ++count; } else { @@ -380,7 +385,7 @@ get_frequent_members(smartlist_t *out, smartlist_t *in, int min) cur = cp; count = 1; } - }); + } SMARTLIST_FOREACH_END(cp); if (count > min) smartlist_add(out, cur); } @@ -443,8 +448,7 @@ compute_routerstatus_consensus(smartlist_t *votes, int consensus_method, * date cannot tie, we use the descriptor with the smaller digest. */ smartlist_sort(votes, _compare_vote_rs); - SMARTLIST_FOREACH(votes, vote_routerstatus_t *, rs, - { + SMARTLIST_FOREACH_BEGIN(votes, vote_routerstatus_t *, rs) { if (cur && !compare_vote_rs(cur, rs)) { ++cur_n; } else { @@ -458,7 +462,7 @@ compute_routerstatus_consensus(smartlist_t *votes, int consensus_method, cur_n = 1; cur = rs; } - }); + } SMARTLIST_FOREACH_END(rs); if (cur_n > most_n || (cur && cur_n == most_n && cur->status.published_on > most_published)) { @@ -471,7 +475,7 @@ compute_routerstatus_consensus(smartlist_t *votes, int consensus_method, if (consensus_method >= MIN_METHOD_FOR_MICRODESC && microdesc_digest256_out) { - smartlist_t *digests = smartlist_create(); + smartlist_t *digests = smartlist_new(); const char *best_microdesc_digest; SMARTLIST_FOREACH_BEGIN(votes, vote_routerstatus_t *, rs) { char d[DIGEST256_LEN]; @@ -499,15 +503,15 @@ static void hash_list_members(char *digest_out, size_t len_out, smartlist_t *lst, digest_algorithm_t alg) { - crypto_digest_env_t *d; + crypto_digest_t *d; if (alg == DIGEST_SHA1) - d = crypto_new_digest_env(); + d = crypto_digest_new(); else - d = crypto_new_digest256_env(alg); + d = crypto_digest256_new(alg); SMARTLIST_FOREACH(lst, const char *, cp, crypto_digest_add_bytes(d, cp, strlen(cp))); crypto_digest_get_digest(d, digest_out, len_out); - crypto_free_digest_env(d); + crypto_digest_free(d); } /** Sorting helper: compare two strings based on their values as base-ten @@ -535,9 +539,9 @@ _cmp_int_strings(const void **_a, const void **_b) static int compute_consensus_method(smartlist_t *votes) { - smartlist_t *all_methods = smartlist_create(); - smartlist_t *acceptable_methods = smartlist_create(); - smartlist_t *tmp = smartlist_create(); + smartlist_t *all_methods = smartlist_new(); + smartlist_t *acceptable_methods = smartlist_new(); + smartlist_t *tmp = smartlist_new(); int min = (smartlist_len(votes) * 2) / 3; int n_ok; int result; @@ -580,15 +584,13 @@ make_consensus_method_list(int low, int high, const char *separator) { char *list; - char b[32]; int i; smartlist_t *lst; - lst = smartlist_create(); + lst = smartlist_new(); for (i = low; i <= high; ++i) { if (!consensus_method_is_supported(i)) continue; - tor_snprintf(b, sizeof(b), "%d", i); - smartlist_add(lst, tor_strdup(b)); + smartlist_add_asprintf(lst, "%d", i); } list = smartlist_join_strings(lst, separator, 0, NULL); tor_assert(list); @@ -605,7 +607,7 @@ static char * compute_consensus_versions_list(smartlist_t *lst, int n_versioning) { int min = n_versioning / 2; - smartlist_t *good = smartlist_create(); + smartlist_t *good = smartlist_new(); char *result; sort_version_list(lst, 0); get_frequent_members(good, lst, min); @@ -614,11 +616,16 @@ compute_consensus_versions_list(smartlist_t *lst, int n_versioning) return result; } +/** Minimum number of directory authorities voting for a parameter to + * include it in the consensus, if consensus method 12 or later is to be + * used. See proposal 178 for details. */ +#define MIN_VOTES_FOR_PARAM 3 + /** Helper: given a list of valid networkstatus_t, return a new string * containing the contents of the consensus network parameter set. */ /* private */ char * -dirvote_compute_params(smartlist_t *votes) +dirvote_compute_params(smartlist_t *votes, int method, int total_authorities) { int i; int32_t *vals; @@ -630,7 +637,7 @@ dirvote_compute_params(smartlist_t *votes) const int n_votes = smartlist_len(votes); smartlist_t *output; - smartlist_t *param_list = smartlist_create(); + smartlist_t *param_list = smartlist_new(); /* We require that the parameter lists in the votes are well-formed: that is, that their keywords are unique and sorted, and that their values are @@ -658,16 +665,16 @@ dirvote_compute_params(smartlist_t *votes) tor_assert(eq); cur_param_len = (int)(eq+1 - cur_param); - output = smartlist_create(); + output = smartlist_new(); SMARTLIST_FOREACH_BEGIN(param_list, const char *, param) { const char *next_param; int ok=0; eq = strchr(param, '='); - tor_assert(i<n_votes); + tor_assert(i<n_votes); /* Make sure we prevented vote-stuffing. */ vals[i++] = (int32_t) tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, NULL); - tor_assert(ok); + tor_assert(ok); /* Already checked these when parsing. */ if (param_sl_idx+1 == smartlist_len(param_list)) next_param = NULL; @@ -675,11 +682,17 @@ dirvote_compute_params(smartlist_t *votes) next_param = smartlist_get(param_list, param_sl_idx+1); if (!next_param || strncmp(next_param, param, cur_param_len)) { /* We've reached the end of a series. */ - int32_t median = median_int32(vals, i); - char *out_string = tor_malloc(64+cur_param_len); - memcpy(out_string, param, cur_param_len); - tor_snprintf(out_string+cur_param_len,64, "%ld", (long)median); - smartlist_add(output, out_string); + /* Make sure enough authorities voted on this param, unless the + * the consensus method we use is too old for that. */ + if (method < MIN_METHOD_FOR_MAJORITY_PARAMS || + i > total_authorities/2 || + i >= MIN_VOTES_FOR_PARAM) { + int32_t median = median_int32(vals, i); + char *out_string = tor_malloc(64+cur_param_len); + memcpy(out_string, param, cur_param_len); + tor_snprintf(out_string+cur_param_len,64, "%ld", (long)median); + smartlist_add(output, out_string); + } i = 0; if (next_param) { @@ -798,8 +811,6 @@ networkstatus_compute_bw_weights_v10(smartlist_t *chunks, int64_t G, int64_t Wmg = -1, Wme = -1, Wmd = -1; int64_t Wed = -1, Wee = -1; const char *casename; - char buf[512]; - int r; if (G <= 0 || M <= 0 || E <= 0 || D <= 0) { log_warn(LD_DIR, "Consensus with empty bandwidth: " @@ -997,7 +1008,7 @@ networkstatus_compute_bw_weights_v10(smartlist_t *chunks, int64_t G, /* We cast down the weights to 32 bit ints on the assumption that * weight_scale is ~= 10000. We need to ensure a rogue authority * doesn't break this assumption to rig our weights */ - tor_assert(0 < weight_scale && weight_scale < INT32_MAX); + tor_assert(0 < weight_scale && weight_scale <= INT32_MAX); /* * Provide Wgm=Wgg, Wmm=1, Wem=Wee, Weg=Wed. May later determine @@ -1007,7 +1018,7 @@ networkstatus_compute_bw_weights_v10(smartlist_t *chunks, int64_t G, * * NOTE: This list is sorted. */ - r = tor_snprintf(buf, sizeof(buf), + smartlist_add_asprintf(chunks, "bandwidth-weights Wbd=%d Wbe=%d Wbg=%d Wbm=%d " "Wdb=%d " "Web=%d Wed=%d Wee=%d Weg=%d Wem=%d " @@ -1018,13 +1029,6 @@ networkstatus_compute_bw_weights_v10(smartlist_t *chunks, int64_t G, (int)weight_scale, (int)Wed, (int)Wee, (int)Wed, (int)Wee, (int)weight_scale, (int)Wgd, (int)Wgg, (int)Wgg, (int)weight_scale, (int)Wmd, (int)Wme, (int)Wmg, (int)weight_scale); - if (r<0) { - log_warn(LD_BUG, - "Not enough space in buffer for bandwidth-weights line."); - *buf = '\0'; - return 0; - } - smartlist_add(chunks, tor_strdup(buf)); log_notice(LD_CIRC, "Computed bandwidth weights for %s with v10: " "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT @@ -1048,8 +1052,6 @@ networkstatus_compute_bw_weights_v9(smartlist_t *chunks, int64_t G, int64_t M, int64_t Wmg = -1, Wme = -1, Wmd = -1; int64_t Wed = -1, Wee = -1; const char *casename; - char buf[512]; - int r; if (G <= 0 || M <= 0 || E <= 0 || D <= 0) { log_warn(LD_DIR, "Consensus with empty bandwidth: " @@ -1234,7 +1236,7 @@ networkstatus_compute_bw_weights_v9(smartlist_t *chunks, int64_t G, int64_t M, /* We cast down the weights to 32 bit ints on the assumption that * weight_scale is ~= 10000. We need to ensure a rogue authority * doesn't break this assumption to rig our weights */ - tor_assert(0 < weight_scale && weight_scale < INT32_MAX); + tor_assert(0 < weight_scale && weight_scale <= INT32_MAX); if (Wgg < 0 || Wgg > weight_scale) { log_warn(LD_DIR, "Bw %s: Wgg="I64_FORMAT"! G="I64_FORMAT @@ -1311,7 +1313,7 @@ networkstatus_compute_bw_weights_v9(smartlist_t *chunks, int64_t G, int64_t M, * * NOTE: This list is sorted. */ - r = tor_snprintf(buf, sizeof(buf), + smartlist_add_asprintf(chunks, "Wbd=%d Wbe=%d Wbg=%d Wbm=%d " "Wdb=%d " "Web=%d Wed=%d Wee=%d Weg=%d Wem=%d " @@ -1322,12 +1324,7 @@ networkstatus_compute_bw_weights_v9(smartlist_t *chunks, int64_t G, int64_t M, (int)weight_scale, (int)Wed, (int)Wee, (int)Wed, (int)Wee, (int)weight_scale, (int)Wgd, (int)Wgg, (int)Wgg, (int)weight_scale, (int)Wmd, (int)Wme, (int)Wmg, (int)weight_scale); - if (r<0) { - log_warn(LD_BUG, - "Not enough space in buffer for bandwidth-weights line."); - *buf = '\0'; - } - smartlist_add(chunks, tor_strdup(buf)); + log_notice(LD_CIRC, "Computed bandwidth weights for %s with v9: " "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT " T="I64_FORMAT, @@ -1347,10 +1344,10 @@ networkstatus_compute_bw_weights_v9(smartlist_t *chunks, int64_t G, int64_t M, char * networkstatus_compute_consensus(smartlist_t *votes, int total_authorities, - crypto_pk_env_t *identity_key, - crypto_pk_env_t *signing_key, + crypto_pk_t *identity_key, + crypto_pk_t *signing_key, const char *legacy_id_key_digest, - crypto_pk_env_t *legacy_signing_key, + crypto_pk_t *legacy_signing_key, consensus_flavor_t flavor) { smartlist_t *chunks; @@ -1375,7 +1372,7 @@ networkstatus_compute_consensus(smartlist_t *votes, log_warn(LD_DIR, "Can't compute a consensus from no votes."); return NULL; } - flags = smartlist_create(); + flags = smartlist_new(); consensus_method = compute_consensus_method(votes); if (consensus_method_is_supported(consensus_method)) { @@ -1398,8 +1395,8 @@ networkstatus_compute_consensus(smartlist_t *votes, int *votesec_list = tor_malloc(n_votes * sizeof(int)); int *distsec_list = tor_malloc(n_votes * sizeof(int)); int n_versioning_clients = 0, n_versioning_servers = 0; - smartlist_t *combined_client_versions = smartlist_create(); - smartlist_t *combined_server_versions = smartlist_create(); + smartlist_t *combined_client_versions = smartlist_new(); + smartlist_t *combined_server_versions = smartlist_new(); SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) { tor_assert(v->type == NS_TYPE_VOTE); @@ -1409,7 +1406,7 @@ networkstatus_compute_consensus(smartlist_t *votes, votesec_list[v_sl_idx] = v->vote_seconds; distsec_list[v_sl_idx] = v->dist_seconds; if (v->client_versions) { - smartlist_t *cv = smartlist_create(); + smartlist_t *cv = smartlist_new(); ++n_versioning_clients; smartlist_split_string(cv, v->client_versions, ",", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); @@ -1418,7 +1415,7 @@ networkstatus_compute_consensus(smartlist_t *votes, smartlist_free(cv); /* elements get freed later. */ } if (v->server_versions) { - smartlist_t *sv = smartlist_create(); + smartlist_t *sv = smartlist_new(); ++n_versioning_servers; smartlist_split_string(sv, v->server_versions, ",", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); @@ -1460,10 +1457,9 @@ networkstatus_compute_consensus(smartlist_t *votes, tor_free(distsec_list); } - chunks = smartlist_create(); + chunks = smartlist_new(); { - char *buf=NULL; char va_buf[ISO_TIME_LEN+1], fu_buf[ISO_TIME_LEN+1], vu_buf[ISO_TIME_LEN+1]; char *flaglist; @@ -1472,20 +1468,17 @@ networkstatus_compute_consensus(smartlist_t *votes, format_iso_time(vu_buf, valid_until); flaglist = smartlist_join_strings(flags, " ", 0, NULL); - tor_asprintf(&buf, "network-status-version 3%s%s\n" + smartlist_add_asprintf(chunks, "network-status-version 3%s%s\n" "vote-status consensus\n", flavor == FLAV_NS ? "" : " ", flavor == FLAV_NS ? "" : flavor_name); - smartlist_add(chunks, buf); - if (consensus_method >= 2) { - tor_asprintf(&buf, "consensus-method %d\n", + smartlist_add_asprintf(chunks, "consensus-method %d\n", consensus_method); - smartlist_add(chunks, buf); } - tor_asprintf(&buf, + smartlist_add_asprintf(chunks, "valid-after %s\n" "fresh-until %s\n" "valid-until %s\n" @@ -1496,13 +1489,13 @@ networkstatus_compute_consensus(smartlist_t *votes, va_buf, fu_buf, vu_buf, vote_seconds, dist_seconds, client_versions, server_versions, flaglist); - smartlist_add(chunks, buf); tor_free(flaglist); } if (consensus_method >= MIN_METHOD_FOR_PARAMS) { - params = dirvote_compute_params(votes); + params = dirvote_compute_params(votes, consensus_method, + total_authorities); if (params) { smartlist_add(chunks, tor_strdup("params ")); smartlist_add(chunks, params); @@ -1514,7 +1507,7 @@ networkstatus_compute_consensus(smartlist_t *votes, smartlist_sort(votes, _compare_votes_by_authority_id); /* Add the authority sections. */ { - smartlist_t *dir_sources = smartlist_create(); + smartlist_t *dir_sources = smartlist_new(); SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) { dir_src_ent_t *e = tor_malloc_zero(sizeof(dir_src_ent_t)); e->v = v; @@ -1533,37 +1526,30 @@ networkstatus_compute_consensus(smartlist_t *votes, smartlist_sort(dir_sources, _compare_dir_src_ents_by_authority_id); SMARTLIST_FOREACH_BEGIN(dir_sources, const dir_src_ent_t *, e) { - struct in_addr in; - char ip[INET_NTOA_BUF_LEN]; char fingerprint[HEX_DIGEST_LEN+1]; char votedigest[HEX_DIGEST_LEN+1]; networkstatus_t *v = e->v; networkstatus_voter_info_t *voter = get_voter(v); - char *buf = NULL; if (e->is_legacy) tor_assert(consensus_method >= 2); - in.s_addr = htonl(voter->addr); - tor_inet_ntoa(&in, ip, sizeof(ip)); base16_encode(fingerprint, sizeof(fingerprint), e->digest, DIGEST_LEN); base16_encode(votedigest, sizeof(votedigest), voter->vote_digest, DIGEST_LEN); - tor_asprintf(&buf, + smartlist_add_asprintf(chunks, "dir-source %s%s %s %s %s %d %d\n", voter->nickname, e->is_legacy ? "-legacy" : "", - fingerprint, voter->address, ip, + fingerprint, voter->address, fmt_addr32(voter->addr), voter->dir_port, voter->or_port); - smartlist_add(chunks, buf); if (! e->is_legacy) { - tor_asprintf(&buf, + smartlist_add_asprintf(chunks, "contact %s\n" "vote-digest %s\n", voter->contact, votedigest); - smartlist_add(chunks, buf); } } SMARTLIST_FOREACH_END(e); SMARTLIST_FOREACH(dir_sources, dir_src_ent_t *, e, tor_free(e)); @@ -1577,10 +1563,10 @@ networkstatus_compute_consensus(smartlist_t *votes, int *flag_counts; /* The number of voters that list flag[j] for the * currently considered router. */ int i; - smartlist_t *matching_descs = smartlist_create(); - smartlist_t *chosen_flags = smartlist_create(); - smartlist_t *versions = smartlist_create(); - smartlist_t *exitsummaries = smartlist_create(); + smartlist_t *matching_descs = smartlist_new(); + smartlist_t *chosen_flags = smartlist_new(); + smartlist_t *versions = smartlist_new(); + smartlist_t *exitsummaries = smartlist_new(); uint32_t *bandwidths = tor_malloc(sizeof(uint32_t) * smartlist_len(votes)); uint32_t *measured_bws = tor_malloc(sizeof(uint32_t) * smartlist_len(votes)); @@ -1615,12 +1601,10 @@ networkstatus_compute_consensus(smartlist_t *votes, chosen_named_idx = smartlist_string_pos(flags, "Named"); /* Build the flag index. */ - SMARTLIST_FOREACH(votes, networkstatus_t *, v, - { + SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) { flag_map[v_sl_idx] = tor_malloc_zero( sizeof(int)*smartlist_len(v->known_flags)); - SMARTLIST_FOREACH(v->known_flags, const char *, fl, - { + SMARTLIST_FOREACH_BEGIN(v->known_flags, const char *, fl) { int p = smartlist_string_pos(flags, fl); tor_assert(p >= 0); flag_map[v_sl_idx][fl_sl_idx] = p; @@ -1629,21 +1613,21 @@ networkstatus_compute_consensus(smartlist_t *votes, named_flag[v_sl_idx] = fl_sl_idx; if (!strcmp(fl, "Unnamed")) unnamed_flag[v_sl_idx] = fl_sl_idx; - }); + } SMARTLIST_FOREACH_END(fl); n_voter_flags[v_sl_idx] = smartlist_len(v->known_flags); size[v_sl_idx] = smartlist_len(v->routerstatus_list); - }); + } SMARTLIST_FOREACH_END(v); /* Named and Unnamed get treated specially */ if (consensus_method >= 2) { - SMARTLIST_FOREACH(votes, networkstatus_t *, v, - { + SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) { uint64_t nf; if (named_flag[v_sl_idx]<0) continue; nf = U64_LITERAL(1) << named_flag[v_sl_idx]; - SMARTLIST_FOREACH(v->routerstatus_list, vote_routerstatus_t *, rs, - { + SMARTLIST_FOREACH_BEGIN(v->routerstatus_list, + vote_routerstatus_t *, rs) { + if ((rs->flags & nf) != 0) { const char *d = strmap_get_lc(name_to_id_map, rs->status.nickname); if (!d) { @@ -1658,16 +1642,16 @@ networkstatus_compute_consensus(smartlist_t *votes, /* It's already a conflict, or it's already this ID. */ } } - }); - }); - SMARTLIST_FOREACH(votes, networkstatus_t *, v, - { + } SMARTLIST_FOREACH_END(rs); + } SMARTLIST_FOREACH_END(v); + + SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) { uint64_t uf; if (unnamed_flag[v_sl_idx]<0) continue; uf = U64_LITERAL(1) << unnamed_flag[v_sl_idx]; - SMARTLIST_FOREACH(v->routerstatus_list, vote_routerstatus_t *, rs, - { + SMARTLIST_FOREACH_BEGIN(v->routerstatus_list, + vote_routerstatus_t *, rs) { if ((rs->flags & uf) != 0) { const char *d = strmap_get_lc(name_to_id_map, rs->status.nickname); if (d == conflict || d == unknown) { @@ -1682,8 +1666,8 @@ networkstatus_compute_consensus(smartlist_t *votes, /* It's mapped to a different name. */ } } - }); - }); + } SMARTLIST_FOREACH_END(rs); + } SMARTLIST_FOREACH_END(v); } /* Now go through all the votes */ @@ -1700,7 +1684,6 @@ networkstatus_compute_consensus(smartlist_t *votes, int naming_conflict = 0; int n_listing = 0; int i; - char *buf=NULL; char microdesc_digest[DIGEST256_LEN]; /* Of the next-to-be-considered digest in each voter, which is first? */ @@ -1773,6 +1756,7 @@ networkstatus_compute_consensus(smartlist_t *votes, rs = compute_routerstatus_consensus(matching_descs, consensus_method, microdesc_digest); /* Copy bits of that into rs_out. */ + memset(&rs_out, 0, sizeof(rs_out)); tor_assert(fast_memeq(lowest_id, rs->status.identity_digest,DIGEST_LEN)); memcpy(rs_out.identity_digest, lowest_id, DIGEST_LEN); memcpy(rs_out.descriptor_digest, rs->status.descriptor_digest, @@ -1806,8 +1790,7 @@ networkstatus_compute_consensus(smartlist_t *votes, /* Set the flags. */ smartlist_add(chosen_flags, (char*)"s"); /* for the start of the line. */ - SMARTLIST_FOREACH(flags, const char *, fl, - { + SMARTLIST_FOREACH_BEGIN(flags, const char *, fl) { if (!strcmp(fl, "Named")) { if (is_named) smartlist_add(chosen_flags, (char*)fl); @@ -1827,7 +1810,7 @@ networkstatus_compute_consensus(smartlist_t *votes, is_bad_exit = 1; } } - }); + } SMARTLIST_FOREACH_END(fl); /* Starting with consensus method 4 we do not list servers * that are not running in a consensus. See Proposal 138 */ @@ -1892,7 +1875,7 @@ networkstatus_compute_consensus(smartlist_t *votes, * that list previously */ const char *chosen_exitsummary = NULL; smartlist_clear(exitsummaries); - SMARTLIST_FOREACH(matching_descs, vote_routerstatus_t *, vsr, { + SMARTLIST_FOREACH_BEGIN(matching_descs, vote_routerstatus_t *, vsr) { /* Check if the vote where this status comes from had the * proper descriptor */ tor_assert(fast_memeq(rs_out.identity_digest, @@ -1912,7 +1895,7 @@ networkstatus_compute_consensus(smartlist_t *votes, exitsummary_disagreement = 1; } } - }); + } SMARTLIST_FOREACH_END(vsr); if (exitsummary_disagreement) { char id[HEX_DIGEST_LEN+1]; @@ -1957,6 +1940,13 @@ networkstatus_compute_consensus(smartlist_t *votes, } } + if (flavor == FLAV_MICRODESC && + consensus_method >= MIN_METHOD_FOR_MANDATORY_MICRODESC && + tor_digest256_is_zero(microdesc_digest)) { + /* With no microdescriptor digest, we omit the entry entirely. */ + continue; + } + { char buf[4096]; /* Okay!! Now we can write the descriptor... */ @@ -1968,10 +1958,9 @@ networkstatus_compute_consensus(smartlist_t *votes, /* Now an m line, if applicable. */ if (flavor == FLAV_MICRODESC && !tor_digest256_is_zero(microdesc_digest)) { - char m[BASE64_DIGEST256_LEN+1], *cp; + char m[BASE64_DIGEST256_LEN+1]; digest256_to_base64(m, microdesc_digest); - tor_asprintf(&cp, "m %s\n", m); - smartlist_add(chunks, cp); + smartlist_add_asprintf(chunks, "m %s\n", m); } /* Next line is all flags. The "\n" is missing. */ smartlist_add(chunks, @@ -1984,15 +1973,12 @@ networkstatus_compute_consensus(smartlist_t *votes, smartlist_add(chunks, tor_strdup("\n")); /* Now the weight line. */ if (rs_out.has_bandwidth) { - char *cp=NULL; - tor_asprintf(&cp, "w Bandwidth=%d\n", rs_out.bandwidth); - smartlist_add(chunks, cp); + smartlist_add_asprintf(chunks, "w Bandwidth=%d\n", rs_out.bandwidth); } /* Now the exitpolicy summary line. */ if (rs_out.has_exitsummary && flavor == FLAV_NS) { - tor_asprintf(&buf, "p %s\n", rs_out.exitsummary); - smartlist_add(chunks, buf); + smartlist_add_asprintf(chunks, "p %s\n", rs_out.exitsummary); } /* And the loop is over and we move on to the next router */ @@ -2041,7 +2027,7 @@ networkstatus_compute_consensus(smartlist_t *votes, int ok=0; char *eq = strchr(bw_weight_param, '='); if (eq) { - weight_scale = tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, + weight_scale = tor_parse_long(eq+1, 10, 1, INT32_MAX, &ok, NULL); if (!ok) { log_warn(LD_DIR, "Bad element '%s' in bw weight param", @@ -2074,7 +2060,6 @@ networkstatus_compute_consensus(smartlist_t *votes, size_t digest_len = flavor == FLAV_NS ? DIGEST_LEN : DIGEST256_LEN; const char *algname = crypto_digest_algorithm_get_name(digest_alg); - char *buf = NULL; char sigbuf[4096]; smartlist_add(chunks, tor_strdup("directory-signature ")); @@ -2088,14 +2073,13 @@ networkstatus_compute_consensus(smartlist_t *votes, /* add the junk that will go at the end of the line. */ if (flavor == FLAV_NS) { - tor_asprintf(&buf, "%s %s\n", fingerprint, + smartlist_add_asprintf(chunks, "%s %s\n", fingerprint, signing_key_fingerprint); } else { - tor_asprintf(&buf, "%s %s %s\n", + smartlist_add_asprintf(chunks, "%s %s %s\n", algname, fingerprint, signing_key_fingerprint); } - smartlist_add(chunks, buf); /* And the signature. */ sigbuf[0] = '\0'; if (router_append_dirobj_signature(sigbuf, sizeof(sigbuf), @@ -2113,14 +2097,13 @@ networkstatus_compute_consensus(smartlist_t *votes, crypto_pk_get_fingerprint(legacy_signing_key, signing_key_fingerprint, 0); if (flavor == FLAV_NS) { - tor_asprintf(&buf, "%s %s\n", fingerprint, + smartlist_add_asprintf(chunks, "%s %s\n", fingerprint, signing_key_fingerprint); } else { - tor_asprintf(&buf, "%s %s %s\n", + smartlist_add_asprintf(chunks, "%s %s %s\n", algname, fingerprint, signing_key_fingerprint); } - smartlist_add(chunks, buf); sigbuf[0] = '\0'; if (router_append_dirobj_signature(sigbuf, sizeof(sigbuf), digest, digest_len, @@ -2313,7 +2296,7 @@ networkstatus_format_signatures(networkstatus_t *consensus, else keyword = "directory-signature"; - elements = smartlist_create(); + elements = smartlist_new(); SMARTLIST_FOREACH_BEGIN(consensus->voters, networkstatus_voter_info_t *, v) { SMARTLIST_FOREACH_BEGIN(v->sigs, document_signature_t *, sig) { @@ -2325,20 +2308,19 @@ networkstatus_format_signatures(networkstatus_t *consensus, base16_encode(sk, sizeof(sk), sig->signing_key_digest, DIGEST_LEN); base16_encode(id, sizeof(id), sig->identity_digest, DIGEST_LEN); if (flavor == FLAV_NS) { - tor_snprintf(buf, sizeof(buf), + smartlist_add_asprintf(elements, "%s %s %s\n-----BEGIN SIGNATURE-----\n", keyword, id, sk); } else { const char *digest_name = crypto_digest_algorithm_get_name(sig->alg); - tor_snprintf(buf, sizeof(buf), + smartlist_add_asprintf(elements, "%s%s%s %s %s %s\n-----BEGIN SIGNATURE-----\n", keyword, for_detached_signatures ? " " : "", for_detached_signatures ? flavor_name : "", digest_name, id, sk); } - smartlist_add(elements, tor_strdup(buf)); base64_encode(buf, sizeof(buf), sig->signature, sig->signature_len); strlcat(buf, "-----END SIGNATURE-----\n", sizeof(buf)); smartlist_add(elements, tor_strdup(buf)); @@ -2361,7 +2343,6 @@ char * networkstatus_get_detached_signatures(smartlist_t *consensuses) { smartlist_t *elements; - char buf[4096]; char *result = NULL, *sigs = NULL; networkstatus_t *consensus_ns = NULL; tor_assert(consensuses); @@ -2377,7 +2358,7 @@ networkstatus_get_detached_signatures(smartlist_t *consensuses) return NULL; } - elements = smartlist_create(); + elements = smartlist_new(); { char va_buf[ISO_TIME_LEN+1], fu_buf[ISO_TIME_LEN+1], @@ -2390,12 +2371,11 @@ networkstatus_get_detached_signatures(smartlist_t *consensuses) format_iso_time(fu_buf, consensus_ns->fresh_until); format_iso_time(vu_buf, consensus_ns->valid_until); - tor_snprintf(buf, sizeof(buf), + smartlist_add_asprintf(elements, "consensus-digest %s\n" "valid-after %s\n" "fresh-until %s\n" "valid-until %s\n", d, va_buf, fu_buf, vu_buf); - smartlist_add(elements, tor_strdup(buf)); } /* Get all the digests for the non-FLAV_NS consensuses */ @@ -2414,9 +2394,8 @@ networkstatus_get_detached_signatures(smartlist_t *consensuses) if (tor_mem_is_zero(ns->digests.d[alg], DIGEST256_LEN)) continue; base16_encode(d, sizeof(d), ns->digests.d[alg], DIGEST256_LEN); - tor_snprintf(buf, sizeof(buf), "additional-digest %s %s %s\n", + smartlist_add_asprintf(elements, "additional-digest %s %s %s\n", flavor_name, alg_name, d); - smartlist_add(elements, tor_strdup(buf)); } } SMARTLIST_FOREACH_END(ns); @@ -2455,7 +2434,7 @@ get_detached_signatures_from_pending_consensuses(pending_consensus_t *pending, { int flav; char *signatures; - smartlist_t *c = smartlist_create(); + smartlist_t *c = smartlist_new(); for (flav = 0; flav < n_flavors; ++flav) { if (pending[flav].consensus) smartlist_add(c, pending[flav].consensus); @@ -2518,7 +2497,7 @@ authority_cert_dup(authority_cert_t *cert) void dirvote_get_preferred_voting_intervals(vote_timing_t *timing_out) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); tor_assert(timing_out); @@ -2594,7 +2573,7 @@ static struct { /** Set voting_schedule to hold the timing for the next vote we should be * doing. */ void -dirvote_recalculate_timing(or_options_t *options, time_t now) +dirvote_recalculate_timing(const or_options_t *options, time_t now) { int interval, vote_delay, dist_delay; time_t start; @@ -2645,7 +2624,7 @@ dirvote_recalculate_timing(or_options_t *options, time_t now) /** Entry point: Take whatever voting actions are pending as of <b>now</b>. */ void -dirvote_act(or_options_t *options, time_t now) +dirvote_act(const or_options_t *options, time_t now) { if (!authdir_mode_v3(options)) return; @@ -2710,6 +2689,7 @@ static smartlist_t *pending_vote_list = NULL; * build a consensus, the votes go here for the next period. */ static smartlist_t *previous_vote_list = NULL; +/* DOCDOC pending_consensuses */ static pending_consensus_t pending_consensuses[N_CONSENSUS_FLAVORS]; /** The detached signatures for the consensus that we're currently @@ -2725,7 +2705,7 @@ static smartlist_t *pending_consensus_signature_list = NULL; static int dirvote_perform_vote(void) { - crypto_pk_env_t *key = get_my_v3_authority_signing_key(); + crypto_pk_t *key = get_my_v3_authority_signing_key(); authority_cert_t *cert = get_my_v3_authority_cert(); networkstatus_t *ns; char *contents; @@ -2760,7 +2740,7 @@ dirvote_perform_vote(void) directory_post_to_dirservers(DIR_PURPOSE_UPLOAD_VOTE, ROUTER_PURPOSE_GENERAL, - V3_AUTHORITY, + V3_DIRINFO, pending_vote->vote_body->dir, pending_vote->vote_body->dir_len, 0); log_notice(LD_DIR, "Vote posted."); @@ -2773,13 +2753,12 @@ dirvote_perform_vote(void) static void dirvote_fetch_missing_votes(void) { - smartlist_t *missing_fps = smartlist_create(); + smartlist_t *missing_fps = smartlist_new(); char *resource; - SMARTLIST_FOREACH(router_get_trusted_dir_servers(), - trusted_dir_server_t *, ds, - { - if (!(ds->type & V3_AUTHORITY)) + SMARTLIST_FOREACH_BEGIN(router_get_trusted_dir_servers(), + trusted_dir_server_t *, ds) { + if (!(ds->type & V3_DIRINFO)) continue; if (!dirvote_get_vote(ds->v3_identity_digest, DGV_BY_ID|DGV_INCLUDE_PENDING)) { @@ -2788,14 +2767,19 @@ dirvote_fetch_missing_votes(void) DIGEST_LEN); smartlist_add(missing_fps, cp); } - }); + } SMARTLIST_FOREACH_END(ds); if (!smartlist_len(missing_fps)) { smartlist_free(missing_fps); return; } - log_notice(LOG_NOTICE, "We're missing votes from %d authorities. Asking " - "every other authority for a copy.", smartlist_len(missing_fps)); + { + char *tmp = smartlist_join_strings(missing_fps, " ", 0, NULL); + log_notice(LOG_NOTICE, "We're missing votes from %d authorities (%s). " + "Asking every other authority for a copy.", + smartlist_len(missing_fps), tmp); + tor_free(tmp); + } resource = smartlist_join_strings(missing_fps, "+", 0, NULL); directory_get_from_all_authorities(DIR_PURPOSE_FETCH_STATUS_VOTE, 0, resource); @@ -2847,9 +2831,9 @@ static void dirvote_clear_votes(int all_votes) { if (!previous_vote_list) - previous_vote_list = smartlist_create(); + previous_vote_list = smartlist_new(); if (!pending_vote_list) - pending_vote_list = smartlist_create(); + pending_vote_list = smartlist_new(); /* All "previous" votes are now junk. */ SMARTLIST_FOREACH(previous_vote_list, pending_vote_t *, v, { @@ -2888,11 +2872,11 @@ dirvote_clear_votes(int all_votes) static char * list_v3_auth_ids(void) { - smartlist_t *known_v3_keys = smartlist_create(); + smartlist_t *known_v3_keys = smartlist_new(); char *keys; SMARTLIST_FOREACH(router_get_trusted_dir_servers(), trusted_dir_server_t *, ds, - if ((ds->type & V3_AUTHORITY) && + if ((ds->type & V3_DIRINFO) && !tor_digest_is_zero(ds->v3_identity_digest)) smartlist_add(known_v3_keys, tor_strdup(hex_str(ds->v3_identity_digest, DIGEST_LEN)))); @@ -2921,7 +2905,7 @@ dirvote_add_vote(const char *vote_body, const char **msg_out, int *status_out) tor_assert(status_out); if (!pending_vote_list) - pending_vote_list = smartlist_create(); + pending_vote_list = smartlist_new(); *status_out = 0; *msg_out = NULL; @@ -2985,7 +2969,7 @@ dirvote_add_vote(const char *vote_body, const char **msg_out, int *status_out) update_consensus_router_descriptor_downloads(time(NULL), 1, vote); /* Now see whether we already have a vote from this authority. */ - SMARTLIST_FOREACH(pending_vote_list, pending_vote_t *, v, { + SMARTLIST_FOREACH_BEGIN(pending_vote_list, pending_vote_t *, v) { if (fast_memeq(v->vote->cert->cache_info.identity_digest, vote->cert->cache_info.identity_digest, DIGEST_LEN)) { @@ -3020,7 +3004,7 @@ dirvote_add_vote(const char *vote_body, const char **msg_out, int *status_out) goto err; } } - }); + } SMARTLIST_FOREACH_END(v); pending_vote = tor_malloc_zero(sizeof(pending_vote_t)); pending_vote->vote_body = new_cached_dir(tor_strndup(vote_body, @@ -3085,9 +3069,9 @@ dirvote_compute_consensuses(void) memset(pending, 0, sizeof(pending)); if (!pending_vote_list) - pending_vote_list = smartlist_create(); + pending_vote_list = smartlist_new(); - n_voters = get_n_authorities(V3_AUTHORITY); + n_voters = get_n_authorities(V3_DIRINFO); n_votes = smartlist_len(pending_vote_list); if (n_votes <= n_voters/2) { log_warn(LD_DIR, "We don't have enough votes to generate a consensus: " @@ -3113,8 +3097,8 @@ dirvote_compute_consensuses(void) goto err; } - votes = smartlist_create(); - votestrings = smartlist_create(); + votes = smartlist_new(); + votestrings = smartlist_new(); SMARTLIST_FOREACH(pending_vote_list, pending_vote_t *, v, { sized_chunk_t *c = tor_malloc(sizeof(sized_chunk_t)); @@ -3133,7 +3117,7 @@ dirvote_compute_consensuses(void) { char legacy_dbuf[DIGEST_LEN]; - crypto_pk_env_t *legacy_sign=NULL; + crypto_pk_t *legacy_sign=NULL; char *legacy_id_digest = NULL; int n_generated = 0; if (get_options()->V3AuthUseLegacyKey) { @@ -3204,8 +3188,7 @@ dirvote_compute_consensuses(void) int n_sigs = 0; /* we may have gotten signatures for this consensus before we built * it ourself. Add them now. */ - SMARTLIST_FOREACH(pending_consensus_signature_list, char *, sig, - { + SMARTLIST_FOREACH_BEGIN(pending_consensus_signature_list, char *, sig) { const char *msg = NULL; int r = dirvote_add_signatures_to_all_pending_consensuses(sig, "pending", &msg); @@ -3216,7 +3199,7 @@ dirvote_compute_consensuses(void) "Could not add queued signature to new consensus: %s", msg); tor_free(sig); - }); + } SMARTLIST_FOREACH_END(sig); if (n_sigs) log_notice(LD_DIR, "Added %d pending signatures while building " "consensus.", n_sigs); @@ -3227,7 +3210,7 @@ dirvote_compute_consensuses(void) directory_post_to_dirservers(DIR_PURPOSE_UPLOAD_SIGNATURES, ROUTER_PURPOSE_GENERAL, - V3_AUTHORITY, + V3_DIRINFO, pending_consensus_signatures, strlen(pending_consensus_signatures), 0); log_notice(LD_DIR, "Signature(s) posted."); @@ -3394,7 +3377,7 @@ dirvote_add_signatures(const char *detached_signatures_body, log_notice(LD_DIR, "Got a signature from %s. " "Queuing it for the next consensus.", source); if (!pending_consensus_signature_list) - pending_consensus_signature_list = smartlist_create(); + pending_consensus_signature_list = smartlist_new(); smartlist_add(pending_consensus_signature_list, tor_strdup(detached_signatures_body)); *msg = "Signature queued"; @@ -3533,9 +3516,9 @@ dirvote_create_microdescriptor(const routerinfo_t *ri) { microdesc_t *result = NULL; char *key = NULL, *summary = NULL, *family = NULL; - char buf[1024]; size_t keylen; - char *out = buf, *end = buf+sizeof(buf); + smartlist_t *chunks = smartlist_new(); + char *output = NULL; if (crypto_pk_write_public_key_to_string(ri->onion_pkey, &key, &keylen)<0) goto done; @@ -3543,23 +3526,19 @@ dirvote_create_microdescriptor(const routerinfo_t *ri) if (ri->declared_family) family = smartlist_join_strings(ri->declared_family, " ", 0, NULL); - if (tor_snprintf(out, end-out, "onion-key\n%s", key)<0) - goto done; - out += strlen(out); - if (family) { - if (tor_snprintf(out, end-out, "family %s\n", family)<0) - goto done; - out += strlen(out); - } - if (summary && strcmp(summary, "reject 1-65535")) { - if (tor_snprintf(out, end-out, "p %s\n", summary)<0) - goto done; - out += strlen(out); - } - *out = '\0'; /* Make sure it's nul-terminated. This should be a no-op */ + smartlist_add_asprintf(chunks, "onion-key\n%s", key); + + if (family) + smartlist_add_asprintf(chunks, "family %s\n", family); + + if (summary && strcmp(summary, "reject 1-65535")) + smartlist_add_asprintf(chunks, "p %s\n", summary); + + output = smartlist_join_strings(chunks, "", 0, NULL); { - smartlist_t *lst = microdescs_parse_from_string(buf, out, 0, 1); + smartlist_t *lst = microdescs_parse_from_string(output, + output+strlen(output), 0, 1); if (smartlist_len(lst) != 1) { log_warn(LD_DIR, "We generated a microdescriptor we couldn't parse."); SMARTLIST_FOREACH(lst, microdesc_t *, md, microdesc_free(md)); @@ -3571,9 +3550,14 @@ dirvote_create_microdescriptor(const routerinfo_t *ri) } done: + tor_free(output); tor_free(key); tor_free(summary); tor_free(family); + if (chunks) { + SMARTLIST_FOREACH(chunks, char *, cp, tor_free(cp)); + smartlist_free(chunks); + } return result; } diff --git a/src/or/dirvote.h b/src/or/dirvote.h index b8acd6a32f..e6f9700614 100644 --- a/src/or/dirvote.h +++ b/src/or/dirvote.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -24,10 +24,10 @@ void dirvote_free_all(void); /* vote manipulation */ char *networkstatus_compute_consensus(smartlist_t *votes, int total_authorities, - crypto_pk_env_t *identity_key, - crypto_pk_env_t *signing_key, + crypto_pk_t *identity_key, + crypto_pk_t *signing_key, const char *legacy_identity_key_digest, - crypto_pk_env_t *legacy_signing_key, + crypto_pk_t *legacy_signing_key, consensus_flavor_t flavor); int networkstatus_add_detached_signatures(networkstatus_t *target, ns_detached_signatures_t *sigs, @@ -43,8 +43,8 @@ authority_cert_t *authority_cert_dup(authority_cert_t *cert); /* vote scheduling */ void dirvote_get_preferred_voting_intervals(vote_timing_t *timing_out); time_t dirvote_get_start_of_next_interval(time_t now, int interval); -void dirvote_recalculate_timing(or_options_t *options, time_t now); -void dirvote_act(or_options_t *options, time_t now); +void dirvote_recalculate_timing(const or_options_t *options, time_t now); +void dirvote_act(const or_options_t *options, time_t now); /* invoked on timers and by outside triggers. */ struct pending_vote_t * dirvote_add_vote(const char *vote_body, @@ -62,12 +62,12 @@ const char *dirvote_get_pending_detached_signatures(void); #define DGV_INCLUDE_PREVIOUS 4 const cached_dir_t *dirvote_get_vote(const char *fp, int flags); void set_routerstatus_from_routerinfo(routerstatus_t *rs, + node_t *node, routerinfo_t *ri, time_t now, int naming, int listbadexits, int listbaddirs, int vote_on_hsdirs); -void router_clear_status_flags(routerinfo_t *ri); networkstatus_t * -dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, +dirserv_generate_networkstatus_vote_obj(crypto_pk_t *private_key, authority_cert_t *cert); microdesc_t *dirvote_create_microdescriptor(const routerinfo_t *ri); @@ -83,9 +83,10 @@ document_signature_t *voter_get_sig_by_algorithm( digest_algorithm_t alg); #ifdef DIRVOTE_PRIVATE -char *format_networkstatus_vote(crypto_pk_env_t *private_key, +char *format_networkstatus_vote(crypto_pk_t *private_key, networkstatus_t *v3_ns); -char *dirvote_compute_params(smartlist_t *votes); +char *dirvote_compute_params(smartlist_t *votes, int method, + int total_authorities); #endif #endif diff --git a/src/or/dns.c b/src/or/dns.c index 9b6b98afaf..78893bfbed 100644 --- a/src/or/dns.c +++ b/src/or/dns.c @@ -1,6 +1,6 @@ /* Copyright (c) 2003-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -168,7 +168,8 @@ static void add_wildcarded_test_address(const char *address); static int configure_nameservers(int force); static int answer_is_wildcarded(const char *ip); static int dns_resolve_impl(edge_connection_t *exitconn, int is_resolve, - or_circuit_t *oncirc, char **resolved_to_hostname); + or_circuit_t *oncirc, char **resolved_to_hostname, + int *made_connection_pending_out); #ifdef DEBUG_DNS_CACHE static void _assert_cache_ok(void); #define assert_cache_ok() _assert_cache_ok() @@ -276,7 +277,7 @@ dns_init(void) int dns_reset(void) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (! server_mode(options)) { if (!the_evdns_base) { @@ -376,7 +377,7 @@ set_expiry(cached_resolve_t *resolve, time_t expires) { tor_assert(resolve && resolve->expire == 0); if (!cached_resolve_pqueue) - cached_resolve_pqueue = smartlist_create(); + cached_resolve_pqueue = smartlist_new(); resolve->expire = expires; smartlist_pqueue_add(cached_resolve_pqueue, _compare_cached_resolves_by_expiry, @@ -449,16 +450,17 @@ purge_expired_resolves(time_t now) if (resolve->pending_connections) { log_debug(LD_EXIT, "Closing pending connections on timed-out DNS resolve!"); - tor_fragile_assert(); while (resolve->pending_connections) { pend = resolve->pending_connections; resolve->pending_connections = pend->next; /* Connections should only be pending if they have no socket. */ - tor_assert(pend->conn->_base.s == -1); + tor_assert(!SOCKET_OK(pend->conn->_base.s)); pendconn = pend->conn; - connection_edge_end(pendconn, END_STREAM_REASON_TIMEOUT); - circuit_detach_stream(circuit_get_by_edge_conn(pendconn), pendconn); - connection_free(TO_CONN(pendconn)); + if (!pendconn->_base.marked_for_close) { + connection_edge_end(pendconn, END_STREAM_REASON_TIMEOUT); + circuit_detach_stream(circuit_get_by_edge_conn(pendconn), pendconn); + connection_free(TO_CONN(pendconn)); + } tor_free(pend); } } @@ -567,7 +569,7 @@ send_resolved_hostname_cell(edge_connection_t *conn, const char *hostname) // log_notice(LD_EXIT, "Sent"); } -/** See if we have a cache entry for <b>exitconn</b>-\>address. if so, +/** See if we have a cache entry for <b>exitconn</b>-\>address. If so, * if resolve valid, put it into <b>exitconn</b>-\>addr and return 1. * If resolve failed, free exitconn and return -1. * @@ -596,10 +598,12 @@ dns_resolve(edge_connection_t *exitconn) { or_circuit_t *oncirc = TO_OR_CIRCUIT(exitconn->on_circuit); int is_resolve, r; + int made_connection_pending = 0; char *hostname = NULL; is_resolve = exitconn->_base.purpose == EXIT_PURPOSE_RESOLVE; - r = dns_resolve_impl(exitconn, is_resolve, oncirc, &hostname); + r = dns_resolve_impl(exitconn, is_resolve, oncirc, &hostname, + &made_connection_pending); switch (r) { case 1: @@ -639,16 +643,11 @@ dns_resolve(edge_connection_t *exitconn) dns_cancel_pending_resolve(exitconn->_base.address); - if (!exitconn->_base.marked_for_close) { + if (!made_connection_pending && !exitconn->_base.marked_for_close) { + /* If we made the connection pending, then we freed it already in + * dns_cancel_pending_resolve(). If we marked it for close, it'll + * get freed from the main loop. Otherwise, can free it now. */ connection_free(TO_CONN(exitconn)); - // XXX ... and we just leak exitconn otherwise? -RD - // If it's marked for close, it's on closeable_connection_lst in - // main.c. If it's on the closeable list, it will get freed from - // main.c. -NM - // "<armadev> If that's true, there are other bugs around, where we - // don't check if it's marked, and will end up double-freeing." - // On the other hand, I don't know of any actual bugs here, so this - // shouldn't be holding up the rc. -RD } break; default: @@ -667,27 +666,33 @@ dns_resolve(edge_connection_t *exitconn) * Return -2 on a transient error. If it's a reverse resolve and it's * successful, sets *<b>hostname_out</b> to a newly allocated string * holding the cached reverse DNS value. + * + * Set *<b>made_connection_pending_out</b> to true if we have placed + * <b>exitconn</b> on the list of pending connections for some resolve; set it + * to false otherwise. */ static int dns_resolve_impl(edge_connection_t *exitconn, int is_resolve, - or_circuit_t *oncirc, char **hostname_out) + or_circuit_t *oncirc, char **hostname_out, + int *made_connection_pending_out) { cached_resolve_t *resolve; cached_resolve_t search; pending_connection_t *pending_connection; - routerinfo_t *me; + const routerinfo_t *me; tor_addr_t addr; time_t now = time(NULL); uint8_t is_reverse = 0; int r; assert_connection_ok(TO_CONN(exitconn), 0); - tor_assert(exitconn->_base.s == -1); + tor_assert(!SOCKET_OK(exitconn->_base.s)); assert_cache_ok(); tor_assert(oncirc); + *made_connection_pending_out = 0; /* first check if exitconn->_base.address is an IP. If so, we already * know the answer. */ - if (tor_addr_from_str(&addr, exitconn->_base.address) >= 0) { + if (tor_addr_parse(&addr, exitconn->_base.address) >= 0) { if (tor_addr_family(&addr) == AF_INET) { tor_addr_copy(&exitconn->_base.addr, &addr); exitconn->address_ttl = DEFAULT_DNS_TTL; @@ -721,7 +726,7 @@ dns_resolve_impl(edge_connection_t *exitconn, int is_resolve, * .in-addr.arpa address but this isn't a resolve request, kill the * connection. */ - if ((r = tor_addr_parse_reverse_lookup_name(&addr, exitconn->_base.address, + if ((r = tor_addr_parse_PTR_name(&addr, exitconn->_base.address, AF_UNSPEC, 0)) != 0) { if (r == 1) { is_reverse = 1; @@ -757,6 +762,7 @@ dns_resolve_impl(edge_connection_t *exitconn, int is_resolve, pending_connection->conn = exitconn; pending_connection->next = resolve->pending_connections; resolve->pending_connections = pending_connection; + *made_connection_pending_out = 1; log_debug(LD_EXIT,"Connection (fd %d) waiting for pending DNS " "resolve of %s", exitconn->_base.s, escaped_safe_str(exitconn->_base.address)); @@ -797,6 +803,7 @@ dns_resolve_impl(edge_connection_t *exitconn, int is_resolve, pending_connection = tor_malloc_zero(sizeof(pending_connection_t)); pending_connection->conn = exitconn; resolve->pending_connections = pending_connection; + *made_connection_pending_out = 1; /* Add this resolve to the cache and priority queue. */ HT_INSERT(cache_map, &cache_root, resolve); @@ -849,7 +856,7 @@ assert_all_pending_dns_resolves_ok(void) pend; pend = pend->next) { assert_connection_ok(TO_CONN(pend->conn), 0); - tor_assert(pend->conn->_base.s == -1); + tor_assert(!SOCKET_OK(pend->conn->_base.s)); tor_assert(!connection_in_array(TO_CONN(pend->conn))); } } @@ -955,7 +962,7 @@ dns_cancel_pending_resolve(const char *address) pend->conn->_base.state = EXIT_CONN_STATE_RESOLVEFAILED; pendconn = pend->conn; assert_connection_ok(TO_CONN(pendconn), 0); - tor_assert(pendconn->_base.s == -1); + tor_assert(!SOCKET_OK(pendconn->_base.s)); if (!pendconn->_base.marked_for_close) { connection_edge_end(pendconn, END_STREAM_REASON_RESOLVEFAILED); } @@ -1026,7 +1033,7 @@ add_answer_to_cache(const char *address, uint8_t is_reverse, uint32_t addr, static INLINE int is_test_address(const char *address) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); return options->ServerDNSTestAddresses && smartlist_string_isin_case(options->ServerDNSTestAddresses, address); } @@ -1085,6 +1092,13 @@ dns_found_answer(const char *address, uint8_t is_reverse, uint32_t addr, pendconn = pend->conn; /* don't pass complex things to the connection_mark_for_close macro */ assert_connection_ok(TO_CONN(pendconn),time(NULL)); + if (pendconn->_base.marked_for_close) { + /* prevent double-remove. */ + pendconn->_base.state = EXIT_CONN_STATE_RESOLVEFAILED; + resolve->pending_connections = pend->next; + tor_free(pend); + continue; + } tor_addr_from_ipv4h(&pendconn->_base.addr, addr); pendconn->address_ttl = ttl; @@ -1177,13 +1191,13 @@ evdns_err_is_transient(int err) static int configure_nameservers(int force) { - or_options_t *options; + const or_options_t *options; const char *conf_fname; struct stat st; int r; options = get_options(); conf_fname = options->ServerDNSResolvConfFile; -#ifndef MS_WINDOWS +#ifndef _WIN32 if (!conf_fname) conf_fname = "/etc/resolv.conf"; #endif @@ -1198,7 +1212,7 @@ configure_nameservers(int force) #ifdef HAVE_EVDNS_SET_DEFAULT_OUTGOING_BIND_ADDRESS if (options->OutboundBindAddress) { tor_addr_t addr; - if (tor_addr_from_str(&addr, options->OutboundBindAddress) < 0) { + if (tor_addr_parse(&addr, options->OutboundBindAddress) < 0) { log_warn(LD_CONFIG,"Outbound bind address '%s' didn't parse. Ignoring.", options->OutboundBindAddress); } else { @@ -1251,7 +1265,7 @@ configure_nameservers(int force) if (nameservers_configured) evdns_base_resume(the_evdns_base); } -#ifdef MS_WINDOWS +#ifdef _WIN32 else { if (nameservers_configured) { evdns_base_search_clear(the_evdns_base); @@ -1389,12 +1403,16 @@ evdns_callback(int result, char type, int count, int ttl, void *addresses, static int launch_resolve(edge_connection_t *exitconn) { - char *addr = tor_strdup(exitconn->_base.address); + char *addr; struct evdns_request *req = NULL; tor_addr_t a; int r; int options = get_options()->ServerDNSSearchDomains ? 0 : DNS_QUERY_NO_SEARCH; + + if (get_options()->DisableNetwork) + return -1; + /* What? Nameservers not configured? Sounds like a bug. */ if (!nameservers_configured) { log_warn(LD_EXIT, "(Harmless.) Nameservers not configured, but resolve " @@ -1404,7 +1422,9 @@ launch_resolve(edge_connection_t *exitconn) } } - r = tor_addr_parse_reverse_lookup_name( + addr = tor_strdup(exitconn->_base.address); + + r = tor_addr_parse_PTR_name( &a, exitconn->_base.address, AF_UNSPEC, 0); tor_assert(the_evdns_base); @@ -1431,7 +1451,7 @@ launch_resolve(edge_connection_t *exitconn) r = 0; if (!req) { - log_warn(LD_EXIT, "eventdns rejected address %s.", + log_fn(LOG_PROTOCOL_WARN, LD_EXIT, "eventdns rejected address %s.", escaped_safe_str(addr)); r = -1; tor_free(addr); /* There is no evdns request in progress; stop @@ -1485,7 +1505,7 @@ wildcard_increment_answer(const char *id) ++*ip; if (*ip > 5 && n_wildcard_requests > 10) { - if (!dns_wildcard_list) dns_wildcard_list = smartlist_create(); + if (!dns_wildcard_list) dns_wildcard_list = smartlist_new(); if (!smartlist_string_isin(dns_wildcard_list, id)) { log(dns_wildcard_notice_given ? LOG_INFO : LOG_NOTICE, LD_EXIT, "Your DNS provider has given \"%s\" as an answer for %d different " @@ -1507,7 +1527,7 @@ add_wildcarded_test_address(const char *address) { int n, n_test_addrs; if (!dns_wildcarded_test_address_list) - dns_wildcarded_test_address_list = smartlist_create(); + dns_wildcarded_test_address_list = smartlist_new(); if (smartlist_string_isin_case(dns_wildcarded_test_address_list, address)) return; @@ -1595,12 +1615,15 @@ launch_wildcard_check(int min_len, int max_len, const char *suffix) static void launch_test_addresses(int fd, short event, void *args) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); struct evdns_request *req; (void)fd; (void)event; (void)args; + if (options->DisableNetwork) + return; + log_info(LD_EXIT, "Launching checks to see whether our nameservers like to " "hijack *everything*."); /* This situation is worse than the failure-hijacking situation. When this diff --git a/src/or/dns.h b/src/or/dns.h index 25ff86e2c6..8c8b476ac8 100644 --- a/src/or/dns.h +++ b/src/or/dns.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** diff --git a/src/or/dnsserv.c b/src/or/dnsserv.c index f2c473dfc5..69adcef9ea 100644 --- a/src/or/dnsserv.c +++ b/src/or/dnsserv.c @@ -1,4 +1,4 @@ -/* Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -19,7 +19,7 @@ #ifdef HAVE_EVENT2_DNS_H #include <event2/dns.h> #include <event2/dns_compat.h> -/* XXXX023 this implies we want an improved evdns */ +/* XXXX this implies we want an improved evdns */ #include <event2/dns_struct.h> #else #include "eventdns.h" @@ -29,8 +29,10 @@ * DNSPort. We need to eventually answer the request <b>req</b>. */ static void -evdns_server_callback(struct evdns_server_request *req, void *_data) +evdns_server_callback(struct evdns_server_request *req, void *data_) { + const listener_connection_t *listener = data_; + entry_connection_t *entry_conn; edge_connection_t *conn; int i = 0; struct evdns_server_question *q = NULL; @@ -43,7 +45,7 @@ evdns_server_callback(struct evdns_server_request *req, void *_data) char *q_name; tor_assert(req); - tor_assert(_data == NULL); + log_info(LD_APP, "Got a new DNS request!"); req->flags |= 0x80; /* set RA */ @@ -114,8 +116,9 @@ evdns_server_callback(struct evdns_server_request *req, void *_data) } /* Make a new dummy AP connection, and attach the request to it. */ - conn = edge_connection_new(CONN_TYPE_AP, AF_INET); - conn->_base.state = AP_CONN_STATE_RESOLVE_WAIT; + entry_conn = entry_connection_new(CONN_TYPE_AP, AF_INET); + conn = ENTRY_TO_EDGE_CONN(entry_conn); + TO_CONN(conn)->state = AP_CONN_STATE_RESOLVE_WAIT; conn->is_dns_request = 1; tor_addr_copy(&TO_CONN(conn)->addr, &tor_addr); @@ -123,23 +126,27 @@ evdns_server_callback(struct evdns_server_request *req, void *_data) TO_CONN(conn)->address = tor_dup_addr(&tor_addr); if (q->type == EVDNS_TYPE_A) - conn->socks_request->command = SOCKS_COMMAND_RESOLVE; + entry_conn->socks_request->command = SOCKS_COMMAND_RESOLVE; else - conn->socks_request->command = SOCKS_COMMAND_RESOLVE_PTR; + entry_conn->socks_request->command = SOCKS_COMMAND_RESOLVE_PTR; - strlcpy(conn->socks_request->address, q->name, - sizeof(conn->socks_request->address)); + strlcpy(entry_conn->socks_request->address, q->name, + sizeof(entry_conn->socks_request->address)); - conn->dns_server_request = req; + entry_conn->socks_request->listener_type = listener->_base.type; + entry_conn->dns_server_request = req; + entry_conn->isolation_flags = listener->isolation_flags; + entry_conn->session_group = listener->session_group; + entry_conn->nym_epoch = get_signewnym_epoch(); - if (connection_add(TO_CONN(conn)) < 0) { + if (connection_add(ENTRY_TO_CONN(entry_conn)) < 0) { log_warn(LD_APP, "Couldn't register dummy connection for DNS request"); evdns_server_request_respond(req, DNS_ERR_SERVERFAILED); - connection_free(TO_CONN(conn)); + connection_free(ENTRY_TO_CONN(entry_conn)); return; } - control_event_stream_status(conn, STREAM_EVENT_NEW, 0); + control_event_stream_status(entry_conn, STREAM_EVENT_NEW, 0); /* Now, unless a controller asked us to leave streams unattached, * throw the connection over to get rewritten (which will @@ -148,7 +155,7 @@ evdns_server_callback(struct evdns_server_request *req, void *_data) log_info(LD_APP, "Passing request for %s to rewrite_and_attach.", escaped_safe_str_client(q->name)); q_name = tor_strdup(q->name); /* q could be freed in rewrite_and_attach */ - connection_ap_rewrite_and_attach_if_allowed(conn, NULL, NULL); + connection_ap_rewrite_and_attach_if_allowed(entry_conn, NULL, NULL); /* Now, the connection is marked if it was bad. */ log_info(LD_APP, "Passed request for %s to rewrite_and_attach_if_allowed.", @@ -164,22 +171,30 @@ evdns_server_callback(struct evdns_server_request *req, void *_data) int dnsserv_launch_request(const char *name, int reverse) { + entry_connection_t *entry_conn; edge_connection_t *conn; char *q_name; /* Make a new dummy AP connection, and attach the request to it. */ - conn = edge_connection_new(CONN_TYPE_AP, AF_INET); + entry_conn = entry_connection_new(CONN_TYPE_AP, AF_INET); + conn = ENTRY_TO_EDGE_CONN(entry_conn); conn->_base.state = AP_CONN_STATE_RESOLVE_WAIT; if (reverse) - conn->socks_request->command = SOCKS_COMMAND_RESOLVE_PTR; + entry_conn->socks_request->command = SOCKS_COMMAND_RESOLVE_PTR; else - conn->socks_request->command = SOCKS_COMMAND_RESOLVE; + entry_conn->socks_request->command = SOCKS_COMMAND_RESOLVE; conn->is_dns_request = 1; - strlcpy(conn->socks_request->address, name, - sizeof(conn->socks_request->address)); + strlcpy(entry_conn->socks_request->address, name, + sizeof(entry_conn->socks_request->address)); + + entry_conn->socks_request->listener_type = CONN_TYPE_CONTROL_LISTENER; + entry_conn->original_dest_address = tor_strdup(name); + entry_conn->session_group = SESSION_GROUP_CONTROL_RESOLVE; + entry_conn->nym_epoch = get_signewnym_epoch(); + entry_conn->isolation_flags = ISO_DEFAULT; if (connection_add(TO_CONN(conn))<0) { log_warn(LD_APP, "Couldn't register dummy connection for RESOLVE request"); @@ -194,7 +209,7 @@ dnsserv_launch_request(const char *name, int reverse) log_info(LD_APP, "Passing request for %s to rewrite_and_attach.", escaped_safe_str_client(name)); q_name = tor_strdup(name); /* q could be freed in rewrite_and_attach */ - connection_ap_rewrite_and_attach_if_allowed(conn, NULL, NULL); + connection_ap_rewrite_and_attach_if_allowed(entry_conn, NULL, NULL); /* Now, the connection is marked if it was bad. */ log_info(LD_APP, "Passed request for %s to rewrite_and_attach_if_allowed.", @@ -206,7 +221,7 @@ dnsserv_launch_request(const char *name, int reverse) /** If there is a pending request on <b>conn</b> that's waiting for an answer, * send back an error and free the request. */ void -dnsserv_reject_request(edge_connection_t *conn) +dnsserv_reject_request(entry_connection_t *conn) { if (conn->dns_server_request) { evdns_server_request_respond(conn->dns_server_request, @@ -252,7 +267,7 @@ evdns_get_orig_address(const struct evdns_server_request *req, * <b>answer_len</b>, in <b>answer</b>, with TTL <b>ttl</b>. Doesn't do * any caching; that's handled elsewhere. */ void -dnsserv_resolved(edge_connection_t *conn, +dnsserv_resolved(entry_connection_t *conn, int answer_type, size_t answer_len, const char *answer, @@ -305,12 +320,15 @@ dnsserv_resolved(edge_connection_t *conn, void dnsserv_configure_listener(connection_t *conn) { + listener_connection_t *listener_conn; tor_assert(conn); tor_assert(SOCKET_OK(conn->s)); tor_assert(conn->type == CONN_TYPE_AP_DNS_LISTENER); - conn->dns_server_port = - tor_evdns_add_server_port(conn->s, 0, evdns_server_callback, NULL); + listener_conn = TO_LISTENER_CONN(conn); + listener_conn->dns_server_port = + tor_evdns_add_server_port(conn->s, 0, evdns_server_callback, + listener_conn); } /** Free the evdns server port for <b>conn</b>, which must be an @@ -318,12 +336,15 @@ dnsserv_configure_listener(connection_t *conn) void dnsserv_close_listener(connection_t *conn) { + listener_connection_t *listener_conn; tor_assert(conn); tor_assert(conn->type == CONN_TYPE_AP_DNS_LISTENER); - if (conn->dns_server_port) { - evdns_close_server_port(conn->dns_server_port); - conn->dns_server_port = NULL; + listener_conn = TO_LISTENER_CONN(conn); + + if (listener_conn->dns_server_port) { + evdns_close_server_port(listener_conn->dns_server_port); + listener_conn->dns_server_port = NULL; } } diff --git a/src/or/dnsserv.h b/src/or/dnsserv.h index fcca868885..3aaa038d2b 100644 --- a/src/or/dnsserv.h +++ b/src/or/dnsserv.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -14,12 +14,12 @@ void dnsserv_configure_listener(connection_t *conn); void dnsserv_close_listener(connection_t *conn); -void dnsserv_resolved(edge_connection_t *conn, +void dnsserv_resolved(entry_connection_t *conn, int answer_type, size_t answer_len, const char *answer, int ttl); -void dnsserv_reject_request(edge_connection_t *conn); +void dnsserv_reject_request(entry_connection_t *conn); int dnsserv_launch_request(const char *name, int is_reverse); #endif diff --git a/src/or/eventdns.c b/src/or/eventdns.c index 42e16aec7a..768693aba6 100644 --- a/src/or/eventdns.c +++ b/src/or/eventdns.c @@ -1,12 +1,18 @@ -/* The original version of this module was written by Adam Langley; for - * a history of modifications, check out the subversion logs. +/* READ THIS COMMENT BEFORE HACKING THIS FILE. * - * When editing this module, try to keep it re-mergeable by Adam. Don't - * reformat the whitespace, add Tor dependencies, or so on. + * This eventdns.c copy has diverged a bit from Libevent's version, and it's + * no longer easy to resynchronize them. Once Tor requires Libevent 2.0, we + * will just dump this file and use Libevent's evdns code. * - * TODO: - * - Replace all externally visible magic numbers with #defined constants. - * - Write documentation for APIs of all external functions. + * Therefore, you probably shouldn't make any change here without making it to + * Libevent as well: it's not good for the implementation to diverge even + * more. Also, we can't shouldn't wantonly the API here (since Libevent APIs + * can't change in ways that break user behavior). Also, we shouldn't bother + * with cosmetic changes: the whole module is slated for demolition, so + * there's no point dusting the linebreaks or re-painting the parser. + * + * (We can't just drop the Libevent 2.0 evdns implementation in here instead, + * since it depends pretty heavily on parts of Libevent 2.0.) */ /* Async DNS Library @@ -75,7 +81,6 @@ #include <stdint.h> #endif #include <stdlib.h> -#include <string.h> #include <errno.h> #include <assert.h> #ifdef HAVE_UNISTD_H @@ -91,7 +96,7 @@ #include "eventdns.h" -#ifdef WIN32 +#ifdef _WIN32 #include <windows.h> #include <winsock2.h> #include <iphlpapi.h> @@ -105,7 +110,7 @@ #include <netinet/in6.h> #endif -#ifdef WIN32 +#ifdef _WIN32 typedef int socklen_t; #endif @@ -338,7 +343,7 @@ static void server_port_ready_callback(int fd, short events, void *arg); static int strtoint(const char *const str); -#ifdef WIN32 +#ifdef _WIN32 static int last_error(int sock) { @@ -427,7 +432,7 @@ _evdns_log(int warn, const char *fmt, ...) if (!evdns_log_fn) return; va_start(args,fmt); -#ifdef WIN32 +#ifdef _WIN32 _vsnprintf(buf, sizeof(buf), fmt, args); #else vsnprintf(buf, sizeof(buf), fmt, args); @@ -1831,8 +1836,8 @@ evdns_server_request_respond(struct evdns_server_request *_req, int err) r = sendto(port->socket, req->response, req->response_len, 0, (struct sockaddr*) &req->addr, req->addrlen); if (r<0) { - int err = last_error(port->socket); - if (! error_is_eagain(err)) + int error = last_error(port->socket); + if (! error_is_eagain(error)) return -1; if (port->pending_replies) { @@ -2291,9 +2296,9 @@ _evdns_nameserver_add_impl(const struct sockaddr *address, evtimer_set(&ns->timeout_event, nameserver_prod_callback, ns); - ns->socket = socket(address->sa_family, SOCK_DGRAM, 0); + ns->socket = tor_open_socket(address->sa_family, SOCK_DGRAM, 0); if (ns->socket < 0) { err = 1; goto out1; } -#ifdef WIN32 +#ifdef _WIN32 { u_long nonblocking = 1; ioctlsocket(ns->socket, FIONBIO, &nonblocking); @@ -3040,7 +3045,7 @@ evdns_resolv_conf_parse(int flags, const char *const filename) { log(EVDNS_LOG_DEBUG, "Parsing resolv.conf file %s", filename); - fd = open(filename, O_RDONLY); + fd = tor_open_cloexec(filename, O_RDONLY, 0); if (fd < 0) { evdns_resolv_set_defaults(flags); return 1; @@ -3096,7 +3101,7 @@ out1: return err; } -#ifdef WIN32 +#ifdef _WIN32 /* Add multiple nameservers from a space-or-comma-separated list. */ static int evdns_nameserver_ip_add_line(const char *ips) { @@ -3208,7 +3213,7 @@ static int config_nameserver_from_reg_key(HKEY key, const TCHAR *subkey) { char *buf; - char ansibuf[MAX_PATH] = {0}; + char ansibuf[MAX_PATH] = {0}; DWORD bufsz = 0, type = 0; int status = 0; @@ -3221,6 +3226,7 @@ config_nameserver_from_reg_key(HKEY key, const TCHAR *subkey) if (RegQueryValueEx(key, subkey, 0, &type, (LPBYTE)buf, &bufsz) == ERROR_SUCCESS && bufsz > 1) { wcstombs(ansibuf,(wchar_t*)buf,MAX_PATH);/*XXXX UNICODE */ + abuf[MAX_PATH-1] = '\0'; status = evdns_nameserver_ip_add_line(ansibuf); } @@ -3304,7 +3310,7 @@ int evdns_init(void) { int res = 0; -#ifdef WIN32 +#ifdef _WIN32 evdns_config_windows_nameservers(); #else res = evdns_resolv_conf_parse(DNS_OPTIONS_ALL, "/etc/resolv.conf"); @@ -3460,7 +3466,7 @@ main(int c, char **v) { if (servertest) { int sock; struct sockaddr_in my_addr; - sock = socket(PF_INET, SOCK_DGRAM, 0); + sock = tor_open_socket(PF_INET, SOCK_DGRAM, 0); fcntl(sock, F_SETFL, O_NONBLOCK); my_addr.sin_family = AF_INET; my_addr.sin_port = htons(10053); diff --git a/src/or/eventdns.h b/src/or/eventdns.h index 3ff8bba4b6..1c130b2a12 100644 --- a/src/or/eventdns.h +++ b/src/or/eventdns.h @@ -273,7 +273,7 @@ int evdns_resolve_reverse(const struct in_addr *in, int flags, evdns_callback_ty int evdns_resolve_reverse_ipv6(const struct in6_addr *in, int flags, evdns_callback_type callback, void *ptr); int evdns_set_option(const char *option, const char *val, int flags); int evdns_resolv_conf_parse(int flags, const char *); -#ifdef MS_WINDOWS +#ifdef _WIN32 int evdns_config_windows_nameservers(void); #endif void evdns_search_clear(void); diff --git a/src/or/eventdns_tor.h b/src/or/eventdns_tor.h index 6a178938b6..4c40b3524b 100644 --- a/src/or/eventdns_tor.h +++ b/src/or/eventdns_tor.h @@ -1,4 +1,4 @@ -/* Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #include "orconfig.h" @@ -9,16 +9,11 @@ typedef unsigned int uint; #ifndef HAVE_U_CHAR typedef unsigned char u_char; #endif -#ifdef MS_WINDOWS +#ifdef _WIN32 #define inline __inline #endif #include "torint.h" -#if defined(MS_WINDOWS) && !defined(WIN32) -/* How did _this_ happen? */ -#define WIN32 -#endif - /* These are for debugging possible memory leaks. */ #include "util.h" #include "compat.h" diff --git a/src/or/geoip.c b/src/or/geoip.c index c51142c82e..6b7cc82b82 100644 --- a/src/or/geoip.c +++ b/src/or/geoip.c @@ -1,4 +1,4 @@ -/* Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -44,6 +44,9 @@ static strmap_t *country_idxplus1_by_lc_code = NULL; /** A list of all known geoip_entry_t, sorted by ip_low. */ static smartlist_t *geoip_entries = NULL; +/** SHA1 digest of the GeoIP file to include in extra-info descriptors. */ +static char geoip_digest[DIGEST_LEN]; + /** Return the index of the <b>country</b>'s entry in the GeoIP DB * if it is a valid 2-letter country code, otherwise return -1. */ @@ -107,16 +110,16 @@ geoip_parse_entry(const char *line) if (!geoip_countries) init_geoip_countries(); if (!geoip_entries) - geoip_entries = smartlist_create(); + geoip_entries = smartlist_new(); while (TOR_ISSPACE(*line)) ++line; if (*line == '#') return 0; - if (sscanf(line,"%u,%u,%2s", &low, &high, b) == 3) { + if (tor_sscanf(line,"%u,%u,%2s", &low, &high, b) == 3) { geoip_add_entry(low, high, b); return 0; - } else if (sscanf(line,"\"%u\",\"%u\",\"%2s\",", &low, &high, b) == 3) { + } else if (tor_sscanf(line,"\"%u\",\"%u\",\"%2s\",", &low, &high, b) == 3) { geoip_add_entry(low, high, b); return 0; } else { @@ -159,7 +162,7 @@ _geoip_compare_key_to_entry(const void *_key, const void **_member) /** Return 1 if we should collect geoip stats on bridge users, and * include them in our extrainfo descriptor. Else return 0. */ int -should_record_bridge_info(or_options_t *options) +should_record_bridge_info(const or_options_t *options) { return options->BridgeRelay && options->BridgeRecordUsageByCountry; } @@ -171,7 +174,7 @@ static void init_geoip_countries(void) { geoip_country_t *geoip_unresolved; - geoip_countries = smartlist_create(); + geoip_countries = smartlist_new(); /* Add a geoip_country_t for requests that could not be resolved to a * country as first element (index 0) to geoip_countries. */ geoip_unresolved = tor_malloc_zero(sizeof(geoip_country_t)); @@ -196,13 +199,14 @@ init_geoip_countries(void) * with '#' (comments). */ int -geoip_load_file(const char *filename, or_options_t *options) +geoip_load_file(const char *filename, const or_options_t *options) { FILE *f; const char *msg = ""; int severity = options_need_geoip_info(options, &msg) ? LOG_WARN : LOG_INFO; + crypto_digest_t *geoip_digest_env = NULL; clear_geoip_db(); - if (!(f = fopen(filename, "r"))) { + if (!(f = tor_fopen_cloexec(filename, "r"))) { log_fn(severity, LD_GENERAL, "Failed to open GEOIP file %s. %s", filename, msg); return -1; @@ -213,12 +217,14 @@ geoip_load_file(const char *filename, or_options_t *options) SMARTLIST_FOREACH(geoip_entries, geoip_entry_t *, e, tor_free(e)); smartlist_free(geoip_entries); } - geoip_entries = smartlist_create(); + geoip_entries = smartlist_new(); + geoip_digest_env = crypto_digest_new(); log_notice(LD_GENERAL, "Parsing GEOIP file %s.", filename); while (!feof(f)) { char buf[512]; if (fgets(buf, (int)sizeof(buf), f) == NULL) break; + crypto_digest_add_bytes(geoip_digest_env, buf, strlen(buf)); /* FFFF track full country name. */ geoip_parse_entry(buf); } @@ -231,6 +237,11 @@ geoip_load_file(const char *filename, or_options_t *options) * country. */ refresh_all_country_info(); + /* Remember file digest so that we can include it in our extra-info + * descriptors. */ + crypto_digest_get_digest(geoip_digest_env, geoip_digest, DIGEST_LEN); + crypto_digest_free(geoip_digest_env); + return 0; } @@ -250,6 +261,21 @@ geoip_get_country_by_ip(uint32_t ipaddr) return ent ? (int)ent->country : 0; } +/** Given an IP address, return a number representing the country to which + * that address belongs, -1 for "No geoip information available", or 0 for + * the 'unknown country'. The return value will always be less than + * geoip_get_n_countries(). To decode it, call geoip_get_country_name(). + */ +int +geoip_get_country_by_addr(const tor_addr_t *addr) +{ + if (tor_addr_family(addr) != AF_INET) { + /*XXXX IP6 support ipv6 geoip.*/ + return -1; + } + return geoip_get_country_by_ip(tor_addr_to_ipv4h(addr)); +} + /** Return the number of countries recognized by the GeoIP database. */ int geoip_get_n_countries(void) @@ -278,12 +304,21 @@ geoip_is_loaded(void) return geoip_countries != NULL && geoip_entries != NULL; } +/** Return the hex-encoded SHA1 digest of the loaded GeoIP file. The + * result does not need to be deallocated, but will be overwritten by the + * next call of hex_str(). */ +const char * +geoip_db_digest(void) +{ + return hex_str(geoip_digest, DIGEST_LEN); +} + /** Entry in a map from IP address to the last time we've seen an incoming * connection from that IP address. Used by bridges only, to track which * countries have them blocked. */ typedef struct clientmap_entry_t { HT_ENTRY(clientmap_entry_t) node; - uint32_t ipaddr; + tor_addr_t addr; /** Time when we last saw this IP address, in MINUTES since the epoch. * * (This will run out of space around 4011 CE. If Tor is still in use around @@ -305,13 +340,14 @@ static HT_HEAD(clientmap, clientmap_entry_t) client_history = static INLINE unsigned clientmap_entry_hash(const clientmap_entry_t *a) { - return ht_improve_hash((unsigned) a->ipaddr); + return ht_improve_hash(tor_addr_hash(&a->addr)); } /** Hashtable helper: compare two clientmap_entry_t values for equality. */ static INLINE int clientmap_entries_eq(const clientmap_entry_t *a, const clientmap_entry_t *b) { - return a->ipaddr == b->ipaddr && a->action == b->action; + return !tor_addr_compare(&a->addr, &b->addr, CMP_EXACT) && + a->action == b->action; } HT_PROTOTYPE(clientmap, clientmap_entry_t, node, clientmap_entry_hash, @@ -397,14 +433,14 @@ geoip_get_mean_shares(time_t now, double *v2_share_out, return 0; } -/** Note that we've seen a client connect from the IP <b>addr</b> (host order) +/** Note that we've seen a client connect from the IP <b>addr</b> * at time <b>now</b>. Ignored by all but bridges and directories if * configured accordingly. */ void geoip_note_client_seen(geoip_client_action_t action, - uint32_t addr, time_t now) + const tor_addr_t *addr, time_t now) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); clientmap_entry_t lookup, *ent; if (action == GEOIP_CLIENT_CONNECT) { /* Only remember statistics as entry guard or as bridge. */ @@ -417,12 +453,12 @@ geoip_note_client_seen(geoip_client_action_t action, return; } - lookup.ipaddr = addr; + tor_addr_copy(&lookup.addr, addr); lookup.action = (int)action; ent = HT_FIND(clientmap, &client_history, &lookup); if (! ent) { ent = tor_malloc_zero(sizeof(clientmap_entry_t)); - ent->ipaddr = addr; + tor_addr_copy(&ent->addr, addr); ent->action = (int)action; HT_INSERT(clientmap, &client_history, ent); } @@ -433,7 +469,7 @@ geoip_note_client_seen(geoip_client_action_t action, if (action == GEOIP_CLIENT_NETWORKSTATUS || action == GEOIP_CLIENT_NETWORKSTATUS_V2) { - int country_idx = geoip_get_country_by_ip(addr); + int country_idx = geoip_get_country_by_addr(addr); if (country_idx < 0) country_idx = 0; /** unresolved requests are stored at index 0. */ if (country_idx >= 0 && country_idx < smartlist_len(geoip_countries)) { @@ -577,6 +613,7 @@ dirreq_map_ent_eq(const dirreq_map_entry_t *a, return a->dirreq_id == b->dirreq_id && a->type == b->type; } +/* DOCDOC dirreq_map_ent_hash */ static unsigned dirreq_map_ent_hash(const dirreq_map_entry_t *entry) { @@ -694,7 +731,7 @@ geoip_get_dirreq_history(geoip_client_action_t action, if (action != GEOIP_CLIENT_NETWORKSTATUS && action != GEOIP_CLIENT_NETWORKSTATUS_V2) return NULL; - dirreq_completed = smartlist_create(); + dirreq_completed = smartlist_new(); for (ptr = HT_START(dirreqmap, &dirreq_map); ptr; ptr = next) { ent = *ptr; if (ent->action != action || ent->type != type) { @@ -803,7 +840,7 @@ geoip_get_client_history(geoip_client_action_t action) int country; if ((*ent)->action != (int)action) continue; - country = geoip_get_country_by_ip((*ent)->ipaddr); + country = geoip_get_country_by_addr(&(*ent)->addr); if (country < 0) country = 0; /** unresolved requests are stored at index 0. */ tor_assert(0 <= country && country < n_countries); @@ -814,7 +851,7 @@ geoip_get_client_history(geoip_client_action_t action) if (total < MIN_IPS_TO_NOTE_ANYTHING) goto done; /* Make a list of c_hist_t */ - entries = smartlist_create(); + entries = smartlist_new(); for (i = 0; i < n_countries; ++i) { unsigned c = counts[i]; const char *countrycode; @@ -834,11 +871,9 @@ geoip_get_client_history(geoip_client_action_t action) smartlist_sort(entries, _c_hist_compare); /* Build the result. */ - chunks = smartlist_create(); + chunks = smartlist_new(); SMARTLIST_FOREACH(entries, c_hist_t *, ch, { - char *buf=NULL; - tor_asprintf(&buf, "%s=%u", ch->country, ch->total); - smartlist_add(chunks, buf); + smartlist_add_asprintf(chunks, "%s=%u", ch->country, ch->total); }); result = smartlist_join_strings(chunks, ",", 0, NULL); done: @@ -870,8 +905,8 @@ geoip_get_request_history(geoip_client_action_t action) if (!geoip_countries) return NULL; - entries = smartlist_create(); - SMARTLIST_FOREACH(geoip_countries, geoip_country_t *, c, { + entries = smartlist_new(); + SMARTLIST_FOREACH_BEGIN(geoip_countries, geoip_country_t *, c) { uint32_t tot = 0; c_hist_t *ent; tot = (action == GEOIP_CLIENT_NETWORKSTATUS) ? @@ -882,15 +917,13 @@ geoip_get_request_history(geoip_client_action_t action) strlcpy(ent->country, c->countrycode, sizeof(ent->country)); ent->total = round_to_next_multiple_of(tot, granularity); smartlist_add(entries, ent); - }); + } SMARTLIST_FOREACH_END(c); smartlist_sort(entries, _c_hist_compare); - strings = smartlist_create(); + strings = smartlist_new(); SMARTLIST_FOREACH(entries, c_hist_t *, ent, { - char *buf = NULL; - tor_asprintf(&buf, "%s=%u", ent->country, ent->total); - smartlist_add(strings, buf); - }); + smartlist_add_asprintf(strings, "%s=%u", ent->country, ent->total); + }); result = smartlist_join_strings(strings, ",", 0, NULL); SMARTLIST_FOREACH(strings, char *, cp, tor_free(cp)); SMARTLIST_FOREACH(entries, c_hist_t *, ent, tor_free(ent)); @@ -910,10 +943,9 @@ geoip_dirreq_stats_init(time_t now) start_of_dirreq_stats_interval = now; } -/** Stop collecting directory request stats in a way that we can re-start - * doing so in geoip_dirreq_stats_init(). */ +/** Reset counters for dirreq stats. */ void -geoip_dirreq_stats_term(void) +geoip_reset_dirreq_stats(time_t now) { SMARTLIST_FOREACH(geoip_countries, geoip_country_t *, c, { c->n_v2_ns_requests = c->n_v3_ns_requests = 0; @@ -945,59 +977,44 @@ geoip_dirreq_stats_term(void) tor_free(this); } } - start_of_dirreq_stats_interval = 0; + start_of_dirreq_stats_interval = now; } -/** Write dirreq statistics to $DATADIR/stats/dirreq-stats and return when - * we would next want to write. */ -time_t -geoip_dirreq_stats_write(time_t now) +/** Stop collecting directory request stats in a way that we can re-start + * doing so in geoip_dirreq_stats_init(). */ +void +geoip_dirreq_stats_term(void) { - char *statsdir = NULL, *filename = NULL; - char *data_v2 = NULL, *data_v3 = NULL; - char written[ISO_TIME_LEN+1]; - open_file_t *open_file = NULL; + geoip_reset_dirreq_stats(0); +} + +/** Return a newly allocated string containing the dirreq statistics + * until <b>now</b>, or NULL if we're not collecting dirreq stats. Caller + * must ensure start_of_dirreq_stats_interval is in the past. */ +char * +geoip_format_dirreq_stats(time_t now) +{ + char t[ISO_TIME_LEN+1]; double v2_share = 0.0, v3_share = 0.0; - FILE *out; int i; + char *v3_ips_string, *v2_ips_string, *v3_reqs_string, *v2_reqs_string, + *v2_share_string = NULL, *v3_share_string = NULL, + *v3_direct_dl_string, *v2_direct_dl_string, + *v3_tunneled_dl_string, *v2_tunneled_dl_string; + char *result; if (!start_of_dirreq_stats_interval) - return 0; /* Not initialized. */ - if (start_of_dirreq_stats_interval + WRITE_STATS_INTERVAL > now) - goto done; /* Not ready to write. */ + return NULL; /* Not initialized. */ - /* Discard all items in the client history that are too old. */ - geoip_remove_old_clients(start_of_dirreq_stats_interval); + tor_assert(now >= start_of_dirreq_stats_interval); - statsdir = get_datadir_fname("stats"); - if (check_private_dir(statsdir, CPD_CREATE, get_options()->User) < 0) - goto done; - filename = get_datadir_fname2("stats", "dirreq-stats"); - data_v2 = geoip_get_client_history(GEOIP_CLIENT_NETWORKSTATUS_V2); - data_v3 = geoip_get_client_history(GEOIP_CLIENT_NETWORKSTATUS); - format_iso_time(written, now); - out = start_writing_to_stdio_file(filename, OPEN_FLAGS_APPEND | O_TEXT, - 0600, &open_file); - if (!out) - goto done; - if (fprintf(out, "dirreq-stats-end %s (%d s)\ndirreq-v3-ips %s\n" - "dirreq-v2-ips %s\n", written, - (unsigned) (now - start_of_dirreq_stats_interval), - data_v3 ? data_v3 : "", data_v2 ? data_v2 : "") < 0) - goto done; - tor_free(data_v2); - tor_free(data_v3); + format_iso_time(t, now); + v2_ips_string = geoip_get_client_history(GEOIP_CLIENT_NETWORKSTATUS_V2); + v3_ips_string = geoip_get_client_history(GEOIP_CLIENT_NETWORKSTATUS); + v2_reqs_string = geoip_get_request_history( + GEOIP_CLIENT_NETWORKSTATUS_V2); + v3_reqs_string = geoip_get_request_history(GEOIP_CLIENT_NETWORKSTATUS); - data_v2 = geoip_get_request_history(GEOIP_CLIENT_NETWORKSTATUS_V2); - data_v3 = geoip_get_request_history(GEOIP_CLIENT_NETWORKSTATUS); - if (fprintf(out, "dirreq-v3-reqs %s\ndirreq-v2-reqs %s\n", - data_v3 ? data_v3 : "", data_v2 ? data_v2 : "") < 0) - goto done; - tor_free(data_v2); - tor_free(data_v3); - SMARTLIST_FOREACH(geoip_countries, geoip_country_t *, c, { - c->n_v2_ns_requests = c->n_v3_ns_requests = 0; - }); #define RESPONSE_GRANULARITY 8 for (i = 0; i < GEOIP_NS_RESPONSE_NUM; i++) { ns_v2_responses[i] = round_uint32_to_next_multiple_of( @@ -1006,61 +1023,117 @@ geoip_dirreq_stats_write(time_t now) ns_v3_responses[i], RESPONSE_GRANULARITY); } #undef RESPONSE_GRANULARITY - if (fprintf(out, "dirreq-v3-resp ok=%u,not-enough-sigs=%u,unavailable=%u," - "not-found=%u,not-modified=%u,busy=%u\n", - ns_v3_responses[GEOIP_SUCCESS], - ns_v3_responses[GEOIP_REJECT_NOT_ENOUGH_SIGS], - ns_v3_responses[GEOIP_REJECT_UNAVAILABLE], - ns_v3_responses[GEOIP_REJECT_NOT_FOUND], - ns_v3_responses[GEOIP_REJECT_NOT_MODIFIED], - ns_v3_responses[GEOIP_REJECT_BUSY]) < 0) - goto done; - if (fprintf(out, "dirreq-v2-resp ok=%u,unavailable=%u," - "not-found=%u,not-modified=%u,busy=%u\n", - ns_v2_responses[GEOIP_SUCCESS], - ns_v2_responses[GEOIP_REJECT_UNAVAILABLE], - ns_v2_responses[GEOIP_REJECT_NOT_FOUND], - ns_v2_responses[GEOIP_REJECT_NOT_MODIFIED], - ns_v2_responses[GEOIP_REJECT_BUSY]) < 0) - goto done; - memset(ns_v2_responses, 0, sizeof(ns_v2_responses)); - memset(ns_v3_responses, 0, sizeof(ns_v3_responses)); + if (!geoip_get_mean_shares(now, &v2_share, &v3_share)) { - if (fprintf(out, "dirreq-v2-share %0.2lf%%\n", v2_share*100) < 0) - goto done; - if (fprintf(out, "dirreq-v3-share %0.2lf%%\n", v3_share*100) < 0) - goto done; + tor_asprintf(&v2_share_string, "dirreq-v2-share %0.2f%%\n", + v2_share*100); + tor_asprintf(&v3_share_string, "dirreq-v3-share %0.2f%%\n", + v3_share*100); } - data_v2 = geoip_get_dirreq_history(GEOIP_CLIENT_NETWORKSTATUS_V2, - DIRREQ_DIRECT); - data_v3 = geoip_get_dirreq_history(GEOIP_CLIENT_NETWORKSTATUS, - DIRREQ_DIRECT); - if (fprintf(out, "dirreq-v3-direct-dl %s\ndirreq-v2-direct-dl %s\n", - data_v3 ? data_v3 : "", data_v2 ? data_v2 : "") < 0) - goto done; - tor_free(data_v2); - tor_free(data_v3); - data_v2 = geoip_get_dirreq_history(GEOIP_CLIENT_NETWORKSTATUS_V2, - DIRREQ_TUNNELED); - data_v3 = geoip_get_dirreq_history(GEOIP_CLIENT_NETWORKSTATUS, - DIRREQ_TUNNELED); - if (fprintf(out, "dirreq-v3-tunneled-dl %s\ndirreq-v2-tunneled-dl %s\n", - data_v3 ? data_v3 : "", data_v2 ? data_v2 : "") < 0) - goto done; + v2_direct_dl_string = geoip_get_dirreq_history( + GEOIP_CLIENT_NETWORKSTATUS_V2, DIRREQ_DIRECT); + v3_direct_dl_string = geoip_get_dirreq_history( + GEOIP_CLIENT_NETWORKSTATUS, DIRREQ_DIRECT); + + v2_tunneled_dl_string = geoip_get_dirreq_history( + GEOIP_CLIENT_NETWORKSTATUS_V2, DIRREQ_TUNNELED); + v3_tunneled_dl_string = geoip_get_dirreq_history( + GEOIP_CLIENT_NETWORKSTATUS, DIRREQ_TUNNELED); + + /* Put everything together into a single string. */ + tor_asprintf(&result, "dirreq-stats-end %s (%d s)\n" + "dirreq-v3-ips %s\n" + "dirreq-v2-ips %s\n" + "dirreq-v3-reqs %s\n" + "dirreq-v2-reqs %s\n" + "dirreq-v3-resp ok=%u,not-enough-sigs=%u,unavailable=%u," + "not-found=%u,not-modified=%u,busy=%u\n" + "dirreq-v2-resp ok=%u,unavailable=%u," + "not-found=%u,not-modified=%u,busy=%u\n" + "%s" + "%s" + "dirreq-v3-direct-dl %s\n" + "dirreq-v2-direct-dl %s\n" + "dirreq-v3-tunneled-dl %s\n" + "dirreq-v2-tunneled-dl %s\n", + t, + (unsigned) (now - start_of_dirreq_stats_interval), + v3_ips_string ? v3_ips_string : "", + v2_ips_string ? v2_ips_string : "", + v3_reqs_string ? v3_reqs_string : "", + v2_reqs_string ? v2_reqs_string : "", + ns_v3_responses[GEOIP_SUCCESS], + ns_v3_responses[GEOIP_REJECT_NOT_ENOUGH_SIGS], + ns_v3_responses[GEOIP_REJECT_UNAVAILABLE], + ns_v3_responses[GEOIP_REJECT_NOT_FOUND], + ns_v3_responses[GEOIP_REJECT_NOT_MODIFIED], + ns_v3_responses[GEOIP_REJECT_BUSY], + ns_v2_responses[GEOIP_SUCCESS], + ns_v2_responses[GEOIP_REJECT_UNAVAILABLE], + ns_v2_responses[GEOIP_REJECT_NOT_FOUND], + ns_v2_responses[GEOIP_REJECT_NOT_MODIFIED], + ns_v2_responses[GEOIP_REJECT_BUSY], + v2_share_string ? v2_share_string : "", + v3_share_string ? v3_share_string : "", + v3_direct_dl_string ? v3_direct_dl_string : "", + v2_direct_dl_string ? v2_direct_dl_string : "", + v3_tunneled_dl_string ? v3_tunneled_dl_string : "", + v2_tunneled_dl_string ? v2_tunneled_dl_string : ""); + + /* Free partial strings. */ + tor_free(v3_ips_string); + tor_free(v2_ips_string); + tor_free(v3_reqs_string); + tor_free(v2_reqs_string); + tor_free(v2_share_string); + tor_free(v3_share_string); + tor_free(v3_direct_dl_string); + tor_free(v2_direct_dl_string); + tor_free(v3_tunneled_dl_string); + tor_free(v2_tunneled_dl_string); - finish_writing_to_file(open_file); - open_file = NULL; + return result; +} - start_of_dirreq_stats_interval = now; +/** If 24 hours have passed since the beginning of the current dirreq + * stats period, write dirreq stats to $DATADIR/stats/dirreq-stats + * (possibly overwriting an existing file) and reset counters. Return + * when we would next want to write dirreq stats or 0 if we never want to + * write. */ +time_t +geoip_dirreq_stats_write(time_t now) +{ + char *statsdir = NULL, *filename = NULL, *str = NULL; + + if (!start_of_dirreq_stats_interval) + return 0; /* Not initialized. */ + if (start_of_dirreq_stats_interval + WRITE_STATS_INTERVAL > now) + goto done; /* Not ready to write. */ + + /* Discard all items in the client history that are too old. */ + geoip_remove_old_clients(start_of_dirreq_stats_interval); + + /* Generate history string .*/ + str = geoip_format_dirreq_stats(now); + + /* Write dirreq-stats string to disk. */ + statsdir = get_datadir_fname("stats"); + if (check_private_dir(statsdir, CPD_CREATE, get_options()->User) < 0) { + log_warn(LD_HIST, "Unable to create stats/ directory!"); + goto done; + } + filename = get_datadir_fname2("stats", "dirreq-stats"); + if (write_str_to_file(filename, str, 0) < 0) + log_warn(LD_HIST, "Unable to write dirreq statistics to disk!"); + + /* Reset measurement interval start. */ + geoip_reset_dirreq_stats(now); done: - if (open_file) - abort_writing_to_file(open_file); - tor_free(filename); tor_free(statsdir); - tor_free(data_v2); - tor_free(data_v3); + tor_free(filename); + tor_free(str); return start_of_dirreq_stats_interval + WRITE_STATS_INTERVAL; } @@ -1140,8 +1213,8 @@ static char *bridge_stats_extrainfo = NULL; /** Return a newly allocated string holding our bridge usage stats by country * in a format suitable for inclusion in an extrainfo document. Return NULL on * failure. */ -static char * -format_bridge_stats_extrainfo(time_t now) +char * +geoip_format_bridge_stats(time_t now) { char *out = NULL, *data = NULL; long duration = now - start_of_bridge_stats_interval; @@ -1149,6 +1222,8 @@ format_bridge_stats_extrainfo(time_t now) if (duration < 0) return NULL; + if (!start_of_bridge_stats_interval) + return NULL; /* Not initialized. */ format_iso_time(written, now); data = geoip_get_client_history(GEOIP_CLIENT_CONNECT); @@ -1198,7 +1273,7 @@ geoip_bridge_stats_write(time_t now) geoip_remove_old_clients(start_of_bridge_stats_interval); /* Generate formatted string */ - val = format_bridge_stats_extrainfo(now); + val = geoip_format_bridge_stats(now); if (val == NULL) goto done; @@ -1275,25 +1350,54 @@ geoip_entry_stats_init(time_t now) start_of_entry_stats_interval = now; } +/** Reset counters for entry stats. */ +void +geoip_reset_entry_stats(time_t now) +{ + client_history_clear(); + start_of_entry_stats_interval = now; +} + /** Stop collecting entry stats in a way that we can re-start doing so in * geoip_entry_stats_init(). */ void geoip_entry_stats_term(void) { - client_history_clear(); - start_of_entry_stats_interval = 0; + geoip_reset_entry_stats(0); +} + +/** Return a newly allocated string containing the entry statistics + * until <b>now</b>, or NULL if we're not collecting entry stats. Caller + * must ensure start_of_entry_stats_interval lies in the past. */ +char * +geoip_format_entry_stats(time_t now) +{ + char t[ISO_TIME_LEN+1]; + char *data = NULL; + char *result; + + if (!start_of_entry_stats_interval) + return NULL; /* Not initialized. */ + + tor_assert(now >= start_of_entry_stats_interval); + + data = geoip_get_client_history(GEOIP_CLIENT_CONNECT); + format_iso_time(t, now); + tor_asprintf(&result, "entry-stats-end %s (%u s)\nentry-ips %s\n", + t, (unsigned) (now - start_of_entry_stats_interval), + data ? data : ""); + tor_free(data); + return result; } -/** Write entry statistics to $DATADIR/stats/entry-stats and return time - * when we would next want to write. */ +/** If 24 hours have passed since the beginning of the current entry stats + * period, write entry stats to $DATADIR/stats/entry-stats (possibly + * overwriting an existing file) and reset counters. Return when we would + * next want to write entry stats or 0 if we never want to write. */ time_t geoip_entry_stats_write(time_t now) { - char *statsdir = NULL, *filename = NULL; - char *data = NULL; - char written[ISO_TIME_LEN+1]; - open_file_t *open_file = NULL; - FILE *out; + char *statsdir = NULL, *filename = NULL, *str = NULL; if (!start_of_entry_stats_interval) return 0; /* Not initialized. */ @@ -1303,31 +1407,26 @@ geoip_entry_stats_write(time_t now) /* Discard all items in the client history that are too old. */ geoip_remove_old_clients(start_of_entry_stats_interval); + /* Generate history string .*/ + str = geoip_format_entry_stats(now); + + /* Write entry-stats string to disk. */ statsdir = get_datadir_fname("stats"); - if (check_private_dir(statsdir, CPD_CREATE, get_options()->User) < 0) + if (check_private_dir(statsdir, CPD_CREATE, get_options()->User) < 0) { + log_warn(LD_HIST, "Unable to create stats/ directory!"); goto done; + } filename = get_datadir_fname2("stats", "entry-stats"); - data = geoip_get_client_history(GEOIP_CLIENT_CONNECT); - format_iso_time(written, now); - out = start_writing_to_stdio_file(filename, OPEN_FLAGS_APPEND | O_TEXT, - 0600, &open_file); - if (!out) - goto done; - if (fprintf(out, "entry-stats-end %s (%u s)\nentry-ips %s\n", - written, (unsigned) (now - start_of_entry_stats_interval), - data ? data : "") < 0) - goto done; + if (write_str_to_file(filename, str, 0) < 0) + log_warn(LD_HIST, "Unable to write entry statistics to disk!"); - start_of_entry_stats_interval = now; + /* Reset measurement interval start. */ + geoip_reset_entry_stats(now); - finish_writing_to_file(open_file); - open_file = NULL; done: - if (open_file) - abort_writing_to_file(open_file); - tor_free(filename); tor_free(statsdir); - tor_free(data); + tor_free(filename); + tor_free(str); return start_of_entry_stats_interval + WRITE_STATS_INTERVAL; } diff --git a/src/or/geoip.h b/src/or/geoip.h index 24f7c5b931..4aed4e07bb 100644 --- a/src/or/geoip.h +++ b/src/or/geoip.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -15,16 +15,18 @@ #ifdef GEOIP_PRIVATE int geoip_parse_entry(const char *line); #endif -int should_record_bridge_info(or_options_t *options); -int geoip_load_file(const char *filename, or_options_t *options); +int should_record_bridge_info(const or_options_t *options); +int geoip_load_file(const char *filename, const or_options_t *options); int geoip_get_country_by_ip(uint32_t ipaddr); +int geoip_get_country_by_addr(const tor_addr_t *addr); int geoip_get_n_countries(void); const char *geoip_get_country_name(country_t num); int geoip_is_loaded(void); +const char *geoip_db_digest(void); country_t geoip_get_country(const char *countrycode); void geoip_note_client_seen(geoip_client_action_t action, - uint32_t addr, time_t now); + const tor_addr_t *addr, time_t now); void geoip_remove_old_clients(time_t cutoff); void geoip_note_ns_response(geoip_client_action_t action, @@ -42,12 +44,17 @@ void geoip_change_dirreq_state(uint64_t dirreq_id, dirreq_type_t type, dirreq_state_t new_state); void geoip_dirreq_stats_init(time_t now); +void geoip_reset_dirreq_stats(time_t now); +char *geoip_format_dirreq_stats(time_t now); time_t geoip_dirreq_stats_write(time_t now); void geoip_dirreq_stats_term(void); void geoip_entry_stats_init(time_t now); time_t geoip_entry_stats_write(time_t now); void geoip_entry_stats_term(void); +void geoip_reset_entry_stats(time_t now); +char *geoip_format_entry_stats(time_t now); void geoip_bridge_stats_init(time_t now); +char *geoip_format_bridge_stats(time_t now); time_t geoip_bridge_stats_write(time_t now); void geoip_bridge_stats_term(void); const char *geoip_get_bridge_stats_extrainfo(time_t); diff --git a/src/or/hibernate.c b/src/or/hibernate.c index 2f7170fa24..3a9c1e4224 100644 --- a/src/or/hibernate.c +++ b/src/or/hibernate.c @@ -1,5 +1,5 @@ /* Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -21,6 +21,7 @@ hibernating, phase 2: - close all OR/AP/exit conns) */ +#define HIBERNATE_PRIVATE #include "or.h" #include "config.h" #include "connection.h" @@ -29,26 +30,11 @@ hibernating, phase 2: #include "main.h" #include "router.h" -/** Possible values of hibernate_state */ -typedef enum { - /** We are running normally. */ - HIBERNATE_STATE_LIVE=1, - /** We're trying to shut down cleanly, and we'll kill all active connections - * at shutdown_time. */ - HIBERNATE_STATE_EXITING=2, - /** We're running low on allocated bandwidth for this period, so we won't - * accept any new connections. */ - HIBERNATE_STATE_LOWBANDWIDTH=3, - /** We are hibernating, and we won't wake up till there's more bandwidth to - * use. */ - HIBERNATE_STATE_DORMANT=4 -} hibernate_state_t; - extern long stats_n_seconds_working; /* published uptime */ /** Are we currently awake, asleep, running out of bandwidth, or shutting * down? */ -static hibernate_state_t hibernate_state = HIBERNATE_STATE_LIVE; +static hibernate_state_t hibernate_state = HIBERNATE_STATE_INITIAL; /** If are hibernating, when do we plan to wake up? Set to 0 if we * aren't hibernating. */ static time_t hibernate_end_time = 0; @@ -116,9 +102,11 @@ static time_unit_t cfg_unit = UNIT_MONTH; /** How many days,hours,minutes into each unit does our accounting interval * start? */ +/** @{ */ static int cfg_start_day = 0, cfg_start_hour = 0, cfg_start_min = 0; +/** @} */ static void reset_accounting(time_t now); static int read_bandwidth_usage(void); @@ -134,7 +122,7 @@ static void accounting_set_wakeup_time(void); * options->AccountingStart. Return 0 on success, -1 on failure. If * <b>validate_only</b> is true, do not change the current settings. */ int -accounting_parse_options(or_options_t *options, int validate_only) +accounting_parse_options(const or_options_t *options, int validate_only) { time_unit_t unit; int ok, idx; @@ -154,7 +142,7 @@ accounting_parse_options(or_options_t *options, int validate_only) return 0; } - items = smartlist_create(); + items = smartlist_new(); smartlist_split_string(items, v, NULL, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK,0); if (smartlist_len(items)<2) { @@ -249,13 +237,21 @@ accounting_parse_options(or_options_t *options, int validate_only) * hibernate, return 1, else return 0. */ int -accounting_is_enabled(or_options_t *options) +accounting_is_enabled(const or_options_t *options) { if (options->AccountingMax) return 1; return 0; } +/** If accounting is enabled, return how long (in seconds) this + * interval lasts. */ +int +accounting_get_interval_length(void) +{ + return (int)(interval_end_time - interval_start_time); +} + /** Called from main.c to tell us that <b>seconds</b> seconds have * passed, <b>n_read</b> bytes have been read, and <b>n_written</b> * bytes have been written. */ @@ -411,7 +407,7 @@ static void update_expected_bandwidth(void) { uint64_t expected; - or_options_t *options= get_options(); + const or_options_t *options= get_options(); uint64_t max_configured = (options->RelayBandwidthRate > 0 ? options->RelayBandwidthRate : options->BandwidthRate) * 60; @@ -517,8 +513,7 @@ static void accounting_set_wakeup_time(void) { char digest[DIGEST_LEN]; - crypto_digest_env_t *d_env; - int time_in_interval; + crypto_digest_t *d_env; uint64_t time_to_exhaust_bw; int time_to_consider; @@ -535,11 +530,11 @@ accounting_set_wakeup_time(void) crypto_pk_get_digest(get_server_identity_key(), digest); - d_env = crypto_new_digest_env(); + d_env = crypto_digest_new(); crypto_digest_add_bytes(d_env, buf, ISO_TIME_LEN); crypto_digest_add_bytes(d_env, digest, DIGEST_LEN); crypto_digest_get_digest(d_env, digest, DIGEST_LEN); - crypto_free_digest_env(d_env); + crypto_digest_free(d_env); } else { crypto_rand(digest, DIGEST_LEN); } @@ -552,22 +547,21 @@ accounting_set_wakeup_time(void) interval_wakeup_time = interval_start_time; log_notice(LD_ACCT, - "Configured hibernation. This interval begins at %s " - "and ends at %s. We have no prior estimate for bandwidth, so " + "Configured hibernation. This interval begins at %s " + "and ends at %s. We have no prior estimate for bandwidth, so " "we will start out awake and hibernate when we exhaust our quota.", buf1, buf2); return; } - time_in_interval = (int)(interval_end_time - interval_start_time); - time_to_exhaust_bw = (get_options()->AccountingMax/expected_bandwidth_usage)*60; if (time_to_exhaust_bw > INT_MAX) { time_to_exhaust_bw = INT_MAX; time_to_consider = 0; } else { - time_to_consider = time_in_interval - (int)time_to_exhaust_bw; + time_to_consider = accounting_get_interval_length() - + (int)time_to_exhaust_bw; } if (time_to_consider<=0) { @@ -749,8 +743,7 @@ hibernate_soft_limit_reached(void) static void hibernate_begin(hibernate_state_t new_state, time_t now) { - connection_t *conn; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (new_state == HIBERNATE_STATE_EXITING && hibernate_state != HIBERNATE_STATE_LIVE) { @@ -770,15 +763,7 @@ hibernate_begin(hibernate_state_t new_state, time_t now) } /* close listeners. leave control listener(s). */ - while ((conn = connection_get_by_type(CONN_TYPE_OR_LISTENER)) || - (conn = connection_get_by_type(CONN_TYPE_AP_LISTENER)) || - (conn = connection_get_by_type(CONN_TYPE_AP_TRANS_LISTENER)) || - (conn = connection_get_by_type(CONN_TYPE_AP_DNS_LISTENER)) || - (conn = connection_get_by_type(CONN_TYPE_AP_NATD_LISTENER)) || - (conn = connection_get_by_type(CONN_TYPE_DIR_LISTENER))) { - log_info(LD_NET,"Closing listener type %d", conn->type); - connection_mark_for_close(conn); - } + connection_mark_all_noncontrol_listeners(); /* XXX kill intro point circs */ /* XXX upload rendezvous service descriptors with no intro points */ @@ -804,10 +789,12 @@ static void hibernate_end(hibernate_state_t new_state) { tor_assert(hibernate_state == HIBERNATE_STATE_LOWBANDWIDTH || - hibernate_state == HIBERNATE_STATE_DORMANT); + hibernate_state == HIBERNATE_STATE_DORMANT || + hibernate_state == HIBERNATE_STATE_INITIAL); /* listeners will be relaunched in run_scheduled_events() in main.c */ - log_notice(LD_ACCT,"Hibernation period ended. Resuming normal activity."); + if (hibernate_state != HIBERNATE_STATE_INITIAL) + log_notice(LD_ACCT,"Hibernation period ended. Resuming normal activity."); hibernate_state = new_state; hibernate_end_time = 0; /* no longer hibernating */ @@ -856,7 +843,7 @@ hibernate_go_dormant(time_t now) connection_edge_end(TO_EDGE_CONN(conn), END_STREAM_REASON_HIBERNATING); log_info(LD_NET,"Closing conn type %d", conn->type); if (conn->type == CONN_TYPE_AP) /* send socks failure if needed */ - connection_mark_unattached_ap(TO_EDGE_CONN(conn), + connection_mark_unattached_ap(TO_ENTRY_CONN(conn), END_STREAM_REASON_HIBERNATING); else connection_mark_for_close(conn); @@ -939,7 +926,8 @@ consider_hibernation(time_t now) /* Else, we aren't hibernating. See if it's time to start hibernating, or to * go dormant. */ - if (hibernate_state == HIBERNATE_STATE_LIVE) { + if (hibernate_state == HIBERNATE_STATE_LIVE || + hibernate_state == HIBERNATE_STATE_INITIAL) { if (hibernate_soft_limit_reached()) { log_notice(LD_ACCT, "Bandwidth soft limit reached; commencing hibernation. " @@ -951,6 +939,8 @@ consider_hibernation(time_t now) "Commencing hibernation. We will wake up at %s local time.", buf); hibernate_go_dormant(now); + } else if (hibernate_state == HIBERNATE_STATE_INITIAL) { + hibernate_end(HIBERNATE_STATE_LIVE); } } @@ -988,8 +978,7 @@ getinfo_helper_accounting(control_connection_t *conn, else *answer = tor_strdup("awake"); } else if (!strcmp(question, "accounting/bytes")) { - *answer = tor_malloc(32); - tor_snprintf(*answer, 32, U64_FORMAT" "U64_FORMAT, + tor_asprintf(answer, U64_FORMAT" "U64_FORMAT, U64_PRINTF_ARG(n_bytes_read_in_interval), U64_PRINTF_ARG(n_bytes_written_in_interval)); } else if (!strcmp(question, "accounting/bytes-left")) { @@ -999,8 +988,7 @@ getinfo_helper_accounting(control_connection_t *conn, read_left = limit - n_bytes_read_in_interval; if (n_bytes_written_in_interval < limit) write_left = limit - n_bytes_written_in_interval; - *answer = tor_malloc(64); - tor_snprintf(*answer, 64, U64_FORMAT" "U64_FORMAT, + tor_asprintf(answer, U64_FORMAT" "U64_FORMAT, U64_PRINTF_ARG(read_left), U64_PRINTF_ARG(write_left)); } else if (!strcmp(question, "accounting/interval-start")) { *answer = tor_malloc(ISO_TIME_LEN+1); @@ -1017,3 +1005,13 @@ getinfo_helper_accounting(control_connection_t *conn, return 0; } +/** + * Manually change the hibernation state. Private; used only by the unit + * tests. + */ +void +hibernate_set_state_for_testing_(hibernate_state_t newstate) +{ + hibernate_state = newstate; +} + diff --git a/src/or/hibernate.h b/src/or/hibernate.h index 2aea0fab0c..9aa026b7b0 100644 --- a/src/or/hibernate.h +++ b/src/or/hibernate.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -12,8 +12,9 @@ #ifndef _TOR_HIBERNATE_H #define _TOR_HIBERNATE_H -int accounting_parse_options(or_options_t *options, int validate_only); -int accounting_is_enabled(or_options_t *options); +int accounting_parse_options(const or_options_t *options, int validate_only); +int accounting_is_enabled(const or_options_t *options); +int accounting_get_interval_length(void); void configure_accounting(time_t now); void accounting_run_housekeeping(time_t now); void accounting_add_bytes(size_t n_read, size_t n_written, int seconds); @@ -25,5 +26,27 @@ int getinfo_helper_accounting(control_connection_t *conn, const char *question, char **answer, const char **errmsg); +#ifdef HIBERNATE_PRIVATE +/** Possible values of hibernate_state */ +typedef enum { + /** We are running normally. */ + HIBERNATE_STATE_LIVE=1, + /** We're trying to shut down cleanly, and we'll kill all active connections + * at shutdown_time. */ + HIBERNATE_STATE_EXITING=2, + /** We're running low on allocated bandwidth for this period, so we won't + * accept any new connections. */ + HIBERNATE_STATE_LOWBANDWIDTH=3, + /** We are hibernating, and we won't wake up till there's more bandwidth to + * use. */ + HIBERNATE_STATE_DORMANT=4, + /** We start out in state default, which means we havent decided which state + * we're in. */ + HIBERNATE_STATE_INITIAL=5 +} hibernate_state_t; + +void hibernate_set_state_for_testing_(hibernate_state_t newstate); +#endif + #endif diff --git a/src/or/main.c b/src/or/main.c index b1159746a2..34bf3e50f5 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -33,9 +33,11 @@ #include "main.h" #include "microdesc.h" #include "networkstatus.h" +#include "nodelist.h" #include "ntmain.h" #include "onion.h" #include "policies.h" +#include "transports.h" #include "relay.h" #include "rendclient.h" #include "rendcommon.h" @@ -44,6 +46,7 @@ #include "router.h" #include "routerlist.h" #include "routerparse.h" +#include "status.h" #ifdef USE_DMALLOC #include <dmalloc.h> #include <openssl/crypto.h> @@ -56,6 +59,10 @@ #include <event.h> #endif +#ifdef USE_BUFFEREVENTS +#include <event2/bufferevent.h> +#endif + void evdns_shutdown(int); /********* PROTOTYPES **********/ @@ -71,6 +78,7 @@ static int connection_should_read_from_linked_conn(connection_t *conn); /********* START VARIABLES **********/ +#ifndef USE_BUFFEREVENTS int global_read_bucket; /**< Max number of bytes I can read this second. */ int global_write_bucket; /**< Max number of bytes I can write this second. */ @@ -78,13 +86,19 @@ int global_write_bucket; /**< Max number of bytes I can write this second. */ int global_relayed_read_bucket; /** Max number of relayed (bandwidth class 1) bytes I can write this second. */ int global_relayed_write_bucket; - /** What was the read bucket before the last second_elapsed_callback() call? * (used to determine how many bytes we've read). */ static int stats_prev_global_read_bucket; /** What was the write bucket before the last second_elapsed_callback() call? * (used to determine how many bytes we've written). */ static int stats_prev_global_write_bucket; +#endif + +/* DOCDOC stats_prev_n_read */ +static uint64_t stats_prev_n_read = 0; +/* DOCDOC stats_prev_n_written */ +static uint64_t stats_prev_n_written = 0; + /* XXX we might want to keep stats about global_relayed_*_bucket too. Or not.*/ /** How many bytes have we read since we started the process? */ static uint64_t stats_n_bytes_read = 0; @@ -103,6 +117,8 @@ static time_t time_to_check_for_correct_dns = 0; static time_t time_of_last_signewnym = 0; /** Is there a signewnym request we're currently waiting to handle? */ static int signewnym_is_pending = 0; +/** How many times have we called newnym? */ +static unsigned newnym_epoch = 0; /** Smartlist of all open connections. */ static smartlist_t *connection_array = NULL; @@ -141,6 +157,12 @@ int can_complete_circuit=0; * they are obsolete? */ #define TLS_HANDSHAKE_TIMEOUT (60) +/** Decides our behavior when no logs are configured/before any + * logs have been configured. For 0, we log notice to stdout as normal. + * For 1, we log warnings only. For 2, we log nothing. + */ +int quiet_level = 0; + /********* END VARIABLES ************/ /**************************************************************************** @@ -150,12 +172,58 @@ int can_complete_circuit=0; * ****************************************************************************/ +#if 0 && defined(USE_BUFFEREVENTS) +static void +free_old_inbuf(connection_t *conn) +{ + if (! conn->inbuf) + return; + + tor_assert(conn->outbuf); + tor_assert(buf_datalen(conn->inbuf) == 0); + tor_assert(buf_datalen(conn->outbuf) == 0); + buf_free(conn->inbuf); + buf_free(conn->outbuf); + conn->inbuf = conn->outbuf = NULL; + + if (conn->read_event) { + event_del(conn->read_event); + tor_event_free(conn->read_event); + } + if (conn->write_event) { + event_del(conn->read_event); + tor_event_free(conn->write_event); + } + conn->read_event = conn->write_event = NULL; +} +#endif + +#if defined(_WIN32) && defined(USE_BUFFEREVENTS) +/** Remove the kernel-space send and receive buffers for <b>s</b>. For use + * with IOCP only. */ +static int +set_buffer_lengths_to_zero(tor_socket_t s) +{ + int zero = 0; + int r = 0; + if (setsockopt(s, SOL_SOCKET, SO_SNDBUF, (void*)&zero, sizeof(zero))) { + log_warn(LD_NET, "Unable to clear SO_SNDBUF"); + r = -1; + } + if (setsockopt(s, SOL_SOCKET, SO_RCVBUF, (void*)&zero, sizeof(zero))) { + log_warn(LD_NET, "Unable to clear SO_RCVBUF"); + r = -1; + } + return r; +} +#endif + /** Add <b>conn</b> to the array of connections that we can poll on. The * connection's socket must be set; the connection starts out * non-reading and non-writing. */ int -connection_add(connection_t *conn) +connection_add_impl(connection_t *conn, int is_connecting) { tor_assert(conn); tor_assert(SOCKET_OK(conn->s) || @@ -167,11 +235,71 @@ connection_add(connection_t *conn) conn->conn_array_index = smartlist_len(connection_array); smartlist_add(connection_array, conn); - if (SOCKET_OK(conn->s) || conn->linked) { +#ifdef USE_BUFFEREVENTS + if (connection_type_uses_bufferevent(conn)) { + if (SOCKET_OK(conn->s) && !conn->linked) { + +#ifdef _WIN32 + if (tor_libevent_using_iocp_bufferevents() && + get_options()->UserspaceIOCPBuffers) { + set_buffer_lengths_to_zero(conn->s); + } +#endif + + conn->bufev = bufferevent_socket_new( + tor_libevent_get_base(), + conn->s, + BEV_OPT_DEFER_CALLBACKS); + if (!conn->bufev) { + log_warn(LD_BUG, "Unable to create socket bufferevent"); + smartlist_del(connection_array, conn->conn_array_index); + conn->conn_array_index = -1; + return -1; + } + if (is_connecting) { + /* Put the bufferevent into a "connecting" state so that we'll get + * a "connected" event callback on successful write. */ + bufferevent_socket_connect(conn->bufev, NULL, 0); + } + connection_configure_bufferevent_callbacks(conn); + } else if (conn->linked && conn->linked_conn && + connection_type_uses_bufferevent(conn->linked_conn)) { + tor_assert(!(SOCKET_OK(conn->s))); + if (!conn->bufev) { + struct bufferevent *pair[2] = { NULL, NULL }; + if (bufferevent_pair_new(tor_libevent_get_base(), + BEV_OPT_DEFER_CALLBACKS, + pair) < 0) { + log_warn(LD_BUG, "Unable to create bufferevent pair"); + smartlist_del(connection_array, conn->conn_array_index); + conn->conn_array_index = -1; + return -1; + } + tor_assert(pair[0]); + conn->bufev = pair[0]; + conn->linked_conn->bufev = pair[1]; + } /* else the other side already was added, and got a bufferevent_pair */ + connection_configure_bufferevent_callbacks(conn); + } else { + tor_assert(!conn->linked); + } + + if (conn->bufev) + tor_assert(conn->inbuf == NULL); + + if (conn->linked_conn && conn->linked_conn->bufev) + tor_assert(conn->linked_conn->inbuf == NULL); + } +#else + (void) is_connecting; +#endif + + if (!HAS_BUFFEREVENT(conn) && (SOCKET_OK(conn->s) || conn->linked)) { conn->read_event = tor_event_new(tor_libevent_get_base(), conn->s, EV_READ|EV_PERSIST, conn_read_callback, conn); conn->write_event = tor_event_new(tor_libevent_get_base(), conn->s, EV_WRITE|EV_PERSIST, conn_write_callback, conn); + /* XXXX CHECK FOR NULL RETURN! */ } log_debug(LD_NET,"new conn type %s, socket %d, address %s, n_conns %d.", @@ -195,7 +323,13 @@ connection_unregister_events(connection_t *conn) log_warn(LD_BUG, "Error removing write event for %d", (int)conn->s); tor_free(conn->write_event); } - if (conn->dns_server_port) { +#ifdef USE_BUFFEREVENTS + if (conn->bufev) { + bufferevent_free(conn->bufev); + conn->bufev = NULL; + } +#endif + if (conn->type == CONN_TYPE_AP_DNS_LISTENER) { dnsserv_close_listener(conn); } } @@ -299,16 +433,42 @@ smartlist_t * get_connection_array(void) { if (!connection_array) - connection_array = smartlist_create(); + connection_array = smartlist_new(); return connection_array; } +/** Provides the traffic read and written over the life of the process. */ + +uint64_t +get_bytes_read(void) +{ + return stats_n_bytes_read; +} + +/* DOCDOC get_bytes_written */ +uint64_t +get_bytes_written(void) +{ + return stats_n_bytes_written; +} + /** Set the event mask on <b>conn</b> to <b>events</b>. (The event * mask is a bitmask whose bits are READ_EVENT and WRITE_EVENT) */ void connection_watch_events(connection_t *conn, watchable_events_t events) { + IF_HAS_BUFFEREVENT(conn, { + short ev = ((short)events) & (EV_READ|EV_WRITE); + short old_ev = bufferevent_get_enabled(conn->bufev); + if ((ev & ~old_ev) != 0) { + bufferevent_enable(conn->bufev, ev); + } + if ((old_ev & ~ev) != 0) { + bufferevent_disable(conn->bufev, old_ev & ~ev); + } + return; + }); if (events & READ_EVENT) connection_start_reading(conn); else @@ -326,6 +486,9 @@ connection_is_reading(connection_t *conn) { tor_assert(conn); + IF_HAS_BUFFEREVENT(conn, + return (bufferevent_get_enabled(conn->bufev) & EV_READ) != 0; + ); return conn->reading_from_linked_conn || (conn->read_event && event_pending(conn->read_event, EV_READ, NULL)); } @@ -335,6 +498,12 @@ void connection_stop_reading(connection_t *conn) { tor_assert(conn); + + IF_HAS_BUFFEREVENT(conn, { + bufferevent_disable(conn->bufev, EV_READ); + return; + }); + tor_assert(conn->read_event); if (conn->linked) { @@ -354,6 +523,12 @@ void connection_start_reading(connection_t *conn) { tor_assert(conn); + + IF_HAS_BUFFEREVENT(conn, { + bufferevent_enable(conn->bufev, EV_READ); + return; + }); + tor_assert(conn->read_event); if (conn->linked) { @@ -375,6 +550,10 @@ connection_is_writing(connection_t *conn) { tor_assert(conn); + IF_HAS_BUFFEREVENT(conn, + return (bufferevent_get_enabled(conn->bufev) & EV_WRITE) != 0; + ); + return conn->writing_to_linked_conn || (conn->write_event && event_pending(conn->write_event, EV_WRITE, NULL)); } @@ -384,6 +563,12 @@ void connection_stop_writing(connection_t *conn) { tor_assert(conn); + + IF_HAS_BUFFEREVENT(conn, { + bufferevent_disable(conn->bufev, EV_WRITE); + return; + }); + tor_assert(conn->write_event); if (conn->linked) { @@ -404,6 +589,12 @@ void connection_start_writing(connection_t *conn) { tor_assert(conn); + + IF_HAS_BUFFEREVENT(conn, { + bufferevent_enable(conn->bufev, EV_WRITE); + return; + }); + tor_assert(conn->write_event); if (conn->linked) { @@ -513,7 +704,7 @@ conn_read_callback(evutil_socket_t fd, short event, void *_conn) if (connection_handle_read(conn) < 0) { if (!conn->marked_for_close) { -#ifndef MS_WINDOWS +#ifndef _WIN32 log_warn(LD_BUG,"Unhandled error on read for %s connection " "(fd %d); removing", conn_type_to_string(conn->type), (int)conn->s); @@ -590,9 +781,32 @@ conn_close_if_marked(int i) assert_connection_ok(conn, now); /* assert_all_pending_dns_resolves_ok(); */ - log_debug(LD_NET,"Cleaning up connection (fd %d).",(int)conn->s); - if ((SOCKET_OK(conn->s) || conn->linked_conn) - && connection_wants_to_flush(conn)) { +#ifdef USE_BUFFEREVENTS + if (conn->bufev) { + if (conn->hold_open_until_flushed && + evbuffer_get_length(bufferevent_get_output(conn->bufev))) { + /* don't close yet. */ + return 0; + } + if (conn->linked_conn && ! conn->linked_conn->marked_for_close) { + /* We need to do this explicitly so that the linked connection + * notices that there was an EOF. */ + bufferevent_flush(conn->bufev, EV_WRITE, BEV_FINISHED); + } + } +#endif + + log_debug(LD_NET,"Cleaning up connection (fd %d).",conn->s); + + /* If the connection we are about to close was trying to connect to + a proxy server and failed, the client won't be able to use that + proxy. We should warn the user about this. */ + if (conn->proxy_state == PROXY_INFANT) + log_failed_proxy_connection(conn); + + IF_HAS_BUFFEREVENT(conn, goto unlink); + if ((SOCKET_OK(conn->s) || conn->linked_conn) && + connection_wants_to_flush(conn)) { /* s == -1 means it's an incomplete edge connection, or that the socket * has already been closed as unflushable. */ ssize_t sz = connection_bucket_write_limit(conn, now); @@ -614,8 +828,8 @@ conn_close_if_marked(int i) } log_debug(LD_GENERAL, "Flushed last %d bytes from a linked conn; " "%d left; flushlen %d; wants-to-flush==%d", retval, - (int)buf_datalen(conn->outbuf), - (int)conn->outbuf_flushlen, + (int)connection_get_outbuf_len(conn), + (int)conn->outbuf_flushlen, connection_wants_to_flush(conn)); } else if (connection_speaks_cells(conn)) { if (conn->state == OR_CONN_STATE_OPEN) { @@ -634,6 +848,42 @@ conn_close_if_marked(int i) "Holding conn (fd %d) open for more flushing.", (int)conn->s)); conn->timestamp_lastwritten = now; /* reset so we can flush more */ + } else if (sz == 0) { + /* Also, retval==0. If we get here, we didn't want to write anything + * (because of rate-limiting) and we didn't. */ + + /* Connection must flush before closing, but it's being rate-limited. + * Let's remove from Libevent, and mark it as blocked on bandwidth + * so it will be re-added on next token bucket refill. Prevents + * busy Libevent loops where we keep ending up here and returning + * 0 until we are no longer blocked on bandwidth. + */ + if (connection_is_writing(conn)) { + conn->write_blocked_on_bw = 1; + connection_stop_writing(conn); + } + if (connection_is_reading(conn)) { + /* XXXX024 We should make this code unreachable; if a connection is + * marked for close and flushing, there is no point in reading to it + * at all. Further, checking at this point is a bit of a hack: it + * would make much more sense to react in + * connection_handle_read_impl, or to just stop reading in + * mark_and_flush */ +#if 0 +#define MARKED_READING_RATE 180 + static ratelim_t marked_read_lim = RATELIM_INIT(MARKED_READING_RATE); + char *m; + if ((m = rate_limit_log(&marked_read_lim, now))) { + log_warn(LD_BUG, "Marked connection (fd %d, type %s, state %s) " + "is still reading; that shouldn't happen.%s", + (int)conn->s, conn_type_to_string(conn->type), + conn_state_to_string(conn->type, conn->state), m); + tor_free(m); + } +#endif + conn->read_blocked_on_bw = 1; + connection_stop_reading(conn); + } } return 0; } @@ -652,13 +902,17 @@ conn_close_if_marked(int i) "something is wrong with your network connection, or " "something is wrong with theirs. " "(fd %d, type %s, state %d, marked at %s:%d).", - (int)buf_datalen(conn->outbuf), + (int)connection_get_outbuf_len(conn), escaped_safe_str_client(conn->address), (int)conn->s, conn_type_to_string(conn->type), conn->state, conn->marked_for_close_file, conn->marked_for_close); } } + +#ifdef USE_BUFFEREVENTS + unlink: +#endif connection_unlink(conn); /* unlink, remove, free */ return 1; } @@ -679,13 +933,13 @@ directory_all_unreachable(time_t now) while ((conn = connection_get_by_type_state(CONN_TYPE_AP, AP_CONN_STATE_CIRCUIT_WAIT))) { - edge_connection_t *edge_conn = TO_EDGE_CONN(conn); + entry_connection_t *entry_conn = TO_ENTRY_CONN(conn); log_notice(LD_NET, "Is your network connection down? " "Failing connection to '%s:%d'.", - safe_str_client(edge_conn->socks_request->address), - edge_conn->socks_request->port); - connection_mark_unattached_ap(edge_conn, + safe_str_client(entry_conn->socks_request->address), + entry_conn->socks_request->port); + connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_NET_UNREACHABLE); } control_event_general_status(LOG_ERR, "DIR_ALL_UNREACHABLE"); @@ -696,18 +950,19 @@ directory_all_unreachable(time_t now) void directory_info_has_arrived(time_t now, int from_cache) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (!router_have_minimum_dir_info()) { int quiet = directory_too_idle_to_fetch_descriptors(options, now); log(quiet ? LOG_INFO : LOG_NOTICE, LD_DIR, "I learned some more directory information, but not enough to " "build a circuit: %s", get_dir_info_status_string()); - update_router_descriptor_downloads(now); + update_all_descriptor_downloads(now); return; } else { - if (directory_fetches_from_authorities(options)) - update_router_descriptor_downloads(now); + if (directory_fetches_from_authorities(options)) { + update_all_descriptor_downloads(now); + } /* if we have enough dir info, then update our guard status with * whatever we just learned. */ @@ -718,7 +973,7 @@ directory_info_has_arrived(time_t now, int from_cache) update_extrainfo_downloads(now); } - if (server_mode(options) && !we_are_hibernating() && !from_cache && + if (server_mode(options) && !net_is_disabled() && !from_cache && (can_complete_circuit || !any_predicted_circuits(now))) consider_testing_reachability(1, 1); } @@ -740,12 +995,13 @@ run_connection_housekeeping(int i, time_t now) { cell_t cell; connection_t *conn = smartlist_get(connection_array, i); - or_options_t *options = get_options(); + const or_options_t *options = get_options(); or_connection_t *or_conn; int past_keepalive = now >= conn->timestamp_lastwritten + options->KeepalivePeriod; - if (conn->outbuf && !buf_datalen(conn->outbuf) && conn->type == CONN_TYPE_OR) + if (conn->outbuf && !connection_get_outbuf_len(conn) && + conn->type == CONN_TYPE_OR) TO_OR_CONN(conn)->timestamp_lastempty = now; if (conn->marked_for_close) { @@ -765,7 +1021,7 @@ run_connection_housekeeping(int i, time_t now) /* This check is temporary; it's to let us know whether we should consider * parsing partial serverdesc responses. */ if (conn->purpose == DIR_PURPOSE_FETCH_SERVERDESC && - buf_datalen(conn->inbuf)>=1024) { + connection_get_inbuf_len(conn) >= 1024) { log_info(LD_DIR,"Trying to extract information from wedged server desc " "download."); connection_dir_reached_eof(TO_DIR_CONN(conn)); @@ -782,7 +1038,11 @@ run_connection_housekeeping(int i, time_t now) the connection or send a keepalive, depending. */ or_conn = TO_OR_CONN(conn); +#ifdef USE_BUFFEREVENTS + tor_assert(conn->bufev); +#else tor_assert(conn->outbuf); +#endif if (or_conn->is_bad_for_new_circs && !or_conn->n_circuits) { /* It's bad for new circuits, and has no unmarked circuits on it: @@ -794,8 +1054,7 @@ run_connection_housekeeping(int i, time_t now) connection_or_connect_failed(TO_OR_CONN(conn), END_OR_CONN_REASON_TIMEOUT, "Tor gave up on the connection"); - connection_mark_for_close(conn); - conn->hold_open_until_flushed = 1; + connection_mark_and_flush(conn); } else if (!connection_state_is_open(conn)) { if (past_keepalive) { /* We never managed to actually get this connection open and happy. */ @@ -804,13 +1063,12 @@ run_connection_housekeeping(int i, time_t now) connection_mark_for_close(conn); } } else if (we_are_hibernating() && !or_conn->n_circuits && - !buf_datalen(conn->outbuf)) { + !connection_get_outbuf_len(conn)) { /* We're hibernating, there's no circuits, and nothing to flush.*/ log_info(LD_OR,"Expiring non-used OR connection to fd %d (%s:%d) " "[Hibernating or exiting].", (int)conn->s,conn->address, conn->port); - connection_mark_for_close(conn); - conn->hold_open_until_flushed = 1; + connection_mark_and_flush(conn); } else if (!or_conn->n_circuits && now >= or_conn->timestamp_last_added_nonpadding + IDLE_OR_CONN_TIMEOUT) { @@ -818,7 +1076,6 @@ run_connection_housekeeping(int i, time_t now) "[idle %d].", (int)conn->s,conn->address, conn->port, (int)(now - or_conn->timestamp_last_added_nonpadding)); connection_mark_for_close(conn); - conn->hold_open_until_flushed = 1; } else if ( now >= or_conn->timestamp_lastempty + options->KeepalivePeriod*10 && now >= conn->timestamp_lastwritten + options->KeepalivePeriod*10) { @@ -826,10 +1083,10 @@ run_connection_housekeeping(int i, time_t now) "Expiring stuck OR connection to fd %d (%s:%d). (%d bytes to " "flush; %d seconds since last write)", (int)conn->s, conn->address, conn->port, - (int)buf_datalen(conn->outbuf), + (int)connection_get_outbuf_len(conn), (int)(now-conn->timestamp_lastwritten)); connection_mark_for_close(conn); - } else if (past_keepalive && !buf_datalen(conn->outbuf)) { + } else if (past_keepalive && !connection_get_outbuf_len(conn)) { /* send a padding cell */ log_fn(LOG_DEBUG,LD_OR,"Sending keepalive to (%s:%d)", conn->address, conn->port); @@ -844,7 +1101,7 @@ run_connection_housekeeping(int i, time_t now) static void signewnym_impl(time_t now) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (!proxy_mode(options)) { log_info(LD_CONTROL, "Ignoring SIGNAL NEWNYM because client functionality " "is disabled."); @@ -856,6 +1113,17 @@ signewnym_impl(time_t now) rend_client_purge_state(); time_of_last_signewnym = now; signewnym_is_pending = 0; + + ++newnym_epoch; + + control_event_signal(SIGNEWNYM); +} + +/** Return the number of times that signewnym has been called. */ +unsigned +get_signewnym_epoch(void) +{ + return newnym_epoch; } /** Perform regular maintenance tasks. This function gets run once per @@ -881,10 +1149,13 @@ run_scheduled_events(time_t now) static time_t time_to_check_for_expired_networkstatus = 0; static time_t time_to_write_stats_files = 0; static time_t time_to_write_bridge_stats = 0; + static time_t time_to_check_port_forwarding = 0; static time_t time_to_launch_reachability_tests = 0; static int should_init_bridge_stats = 1; static time_t time_to_retry_dns_init = 0; - or_options_t *options = get_options(); + static time_t time_to_next_heartbeat = 0; + const or_options_t *options = get_options(); + int is_server = server_mode(options); int i; int have_dir_info; @@ -895,6 +1166,16 @@ run_scheduled_events(time_t now) */ consider_hibernation(now); +#if 0 + { + static time_t nl_check_time = 0; + if (nl_check_time <= now) { + nodelist_assert_ok(); + nl_check_time = now + 30; + } + } +#endif + /* 0b. If we've deferred a signewnym, make sure it gets handled * eventually. */ if (signewnym_is_pending && @@ -918,12 +1199,12 @@ run_scheduled_events(time_t now) if (router_rebuild_descriptor(1)<0) { log_info(LD_CONFIG, "Couldn't rebuild router descriptor"); } - if (advertised_server_mode()) + if (advertised_server_mode() && !options->DisableNetwork) router_upload_dir_desc_to_dirservers(0); } - if (time_to_try_getting_descriptors < now) { - update_router_descriptor_downloads(now); + if (!options->DisableNetwork && time_to_try_getting_descriptors < now) { + update_all_descriptor_downloads(now); update_extrainfo_downloads(now); if (router_have_minimum_dir_info()) time_to_try_getting_descriptors = now + LAZY_DESCRIPTOR_RETRY_INTERVAL; @@ -946,10 +1227,7 @@ run_scheduled_events(time_t now) last_rotated_x509_certificate = now; if (last_rotated_x509_certificate+MAX_SSL_KEY_LIFETIME_INTERNAL < now) { log_info(LD_GENERAL,"Rotating tls context."); - if (tor_tls_context_init(public_server_mode(options), - get_tlsclient_identity_key(), - is_server ? get_server_identity_key() : NULL, - MAX_SSL_KEY_LIFETIME_ADVERTISED) < 0) { + if (router_initialize_tls_context() < 0) { log_warn(LD_BUG, "Error reinitializing TLS context"); /* XXX is it a bug here, that we just keep going? -RD */ } @@ -976,7 +1254,7 @@ run_scheduled_events(time_t now) if (time_to_launch_reachability_tests < now && (authdir_mode_tests_reachability(options)) && - !we_are_hibernating()) { + !net_is_disabled()) { time_to_launch_reachability_tests = now + REACHABILITY_TEST_INTERVAL; /* try to determine reachability of the other Tor relays */ dirserv_test_reachability(now); @@ -1048,6 +1326,16 @@ run_scheduled_events(time_t now) if (next_write && next_write < next_time_to_write_stats_files) next_time_to_write_stats_files = next_write; } + if (options->ConnDirectionStatistics) { + time_t next_write = rep_hist_conn_stats_write(time_to_write_stats_files); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->BridgeAuthoritativeDir) { + time_t next_write = rep_hist_desc_stats_write(time_to_write_stats_files); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } time_to_write_stats_files = next_time_to_write_stats_files; } @@ -1076,10 +1364,9 @@ run_scheduled_events(time_t now) /* Remove old information from rephist and the rend cache. */ if (time_to_clean_caches < now) { rep_history_clean(now - options->RephistTrackTime); - rend_cache_clean(); - rend_cache_clean_v2_descs_as_dir(); - if (authdir_mode_v3(options)) - microdesc_cache_rebuild(NULL, 0); + rend_cache_clean(now); + rend_cache_clean_v2_descs_as_dir(now); + microdesc_cache_rebuild(NULL, 0); #define CLEAN_CACHES_INTERVAL (30*60) time_to_clean_caches = now + CLEAN_CACHES_INTERVAL; } @@ -1088,7 +1375,7 @@ run_scheduled_events(time_t now) /* If we're a server and initializing dns failed, retry periodically. */ if (time_to_retry_dns_init < now) { time_to_retry_dns_init = now + RETRY_DNS_INTERVAL; - if (server_mode(options) && has_dns_init_failed()) + if (is_server && has_dns_init_failed()) dns_init(); } @@ -1103,7 +1390,7 @@ run_scheduled_events(time_t now) /* 2b. Once per minute, regenerate and upload the descriptor if the old * one is inaccurate. */ - if (time_to_check_descriptor < now) { + if (time_to_check_descriptor < now && !options->DisableNetwork) { static int dirport_reachability_count = 0; time_to_check_descriptor = now + CHECK_DESCRIPTOR_INTERVAL; check_descriptor_bandwidth_changed(now); @@ -1111,15 +1398,11 @@ run_scheduled_events(time_t now) time_to_check_ipaddress = now + CHECK_IPADDRESS_INTERVAL; check_descriptor_ipaddress_changed(now); } -/** If our router descriptor ever goes this long without being regenerated - * because something changed, we force an immediate regenerate-and-upload. */ -#define FORCE_REGENERATE_DESCRIPTOR_INTERVAL (18*60*60) - mark_my_descriptor_dirty_if_older_than( - now - FORCE_REGENERATE_DESCRIPTOR_INTERVAL); + mark_my_descriptor_dirty_if_too_old(now); consider_publishable_server(0); /* also, check religiously for reachability, if it's within the first * 20 minutes of our uptime. */ - if (server_mode(options) && + if (is_server && (can_complete_circuit || !any_predicted_circuits(now)) && !we_are_hibernating()) { if (stats_n_seconds_working < TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { @@ -1130,7 +1413,7 @@ run_scheduled_events(time_t now) /* If we haven't checked for 12 hours and our bandwidth estimate is * low, do another bandwidth test. This is especially important for * bridges, since they might go long periods without much use. */ - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); if (time_to_recheck_bandwidth && me && me->bandwidthcapacity < me->bandwidthrate && me->bandwidthcapacity < 51200) { @@ -1163,11 +1446,8 @@ run_scheduled_events(time_t now) * We do this before step 4, so it can try building more if * it's not comfortable with the number of available circuits. */ - /* XXXX022 If our circuit build timeout is much lower than a second, maybe - * we should do this more often? -NM - * It can't be lower than 1.5 seconds currently; see - * circuit_build_times_min_timeout(). -RD - */ + /* (If our circuit build timeout can ever become lower than a second (which + * it can't, currently), we should do this more often.) */ circuit_expire_building(); /** 3b. Also look at pending streams and prune the ones that 'began' @@ -1182,8 +1462,8 @@ run_scheduled_events(time_t now) connection_expire_held_open(); /** 3d. And every 60 seconds, we relaunch listeners if any died. */ - if (!we_are_hibernating() && time_to_check_listeners < now) { - retry_all_listeners(NULL, NULL); + if (!net_is_disabled() && time_to_check_listeners < now) { + retry_all_listeners(NULL, NULL, 0); time_to_check_listeners = now+60; } @@ -1193,7 +1473,7 @@ run_scheduled_events(time_t now) * and we make a new circ if there are no clean circuits. */ have_dir_info = router_have_minimum_dir_info(); - if (have_dir_info && !we_are_hibernating()) + if (have_dir_info && !net_is_disabled()) circuit_build_needed_circs(now); /* every 10 seconds, but not at the same second as other such events */ @@ -1224,7 +1504,7 @@ run_scheduled_events(time_t now) circuit_close_all_marked(); /** 7. And upload service descriptors if necessary. */ - if (can_complete_circuit && !we_are_hibernating()) { + if (can_complete_circuit && !net_is_disabled()) { rend_consider_services_upload(now); rend_consider_descriptor_republication(); } @@ -1241,7 +1521,8 @@ run_scheduled_events(time_t now) /** 9. and if we're a server, check whether our DNS is telling stories to * us. */ - if (public_server_mode(options) && time_to_check_for_correct_dns < now) { + if (!net_is_disabled() && + public_server_mode(options) && time_to_check_for_correct_dns < now) { if (!time_to_check_for_correct_dns) { time_to_check_for_correct_dns = now + 60 + crypto_rand_int(120); } else { @@ -1251,13 +1532,38 @@ run_scheduled_events(time_t now) } } - /** 10b. write bridge networkstatus file to disk */ + /** 10. write bridge networkstatus file to disk */ if (options->BridgeAuthoritativeDir && time_to_write_bridge_status_file < now) { networkstatus_dump_bridge_status_to_file(now); #define BRIDGE_STATUSFILE_INTERVAL (30*60) time_to_write_bridge_status_file = now+BRIDGE_STATUSFILE_INTERVAL; } + + /** 11. check the port forwarding app */ + if (!net_is_disabled() && + time_to_check_port_forwarding < now && + options->PortForwarding && + is_server) { +#define PORT_FORWARDING_CHECK_INTERVAL 5 + /* XXXXX this should take a list of ports, not just two! */ + tor_check_port_forwarding(options->PortForwardingHelper, + get_primary_dir_port(), + get_primary_or_port(), + now); + time_to_check_port_forwarding = now+PORT_FORWARDING_CHECK_INTERVAL; + } + + /** 11b. check pending unconfigured managed proxies */ + if (!net_is_disabled() && pt_proxies_configuration_pending()) + pt_configure_remaining_proxies(); + + /** 12. write the heartbeat message */ + if (options->HeartbeatPeriod && + time_to_next_heartbeat <= now) { + log_heartbeat(now); + time_to_next_heartbeat = now+options->HeartbeatPeriod; + } } /** Timer: used to invoke second_elapsed_callback() once per second. */ @@ -1277,7 +1583,7 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) size_t bytes_written; size_t bytes_read; int seconds_elapsed; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); (void)timer; (void)arg; @@ -1288,30 +1594,39 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) update_approx_time(now); /* the second has rolled over. check more stuff. */ - bytes_written = stats_prev_global_write_bucket - global_write_bucket; - bytes_read = stats_prev_global_read_bucket - global_read_bucket; seconds_elapsed = current_second ? (int)(now - current_second) : 0; - stats_n_bytes_read += bytes_read; - stats_n_bytes_written += bytes_written; - if (accounting_is_enabled(options) && seconds_elapsed >= 0) - accounting_add_bytes(bytes_read, bytes_written, seconds_elapsed); +#ifdef USE_BUFFEREVENTS + { + uint64_t cur_read,cur_written; + connection_get_rate_limit_totals(&cur_read, &cur_written); + bytes_written = (size_t)(cur_written - stats_prev_n_written); + bytes_read = (size_t)(cur_read - stats_prev_n_read); + stats_n_bytes_read += bytes_read; + stats_n_bytes_written += bytes_written; + if (accounting_is_enabled(options) && seconds_elapsed >= 0) + accounting_add_bytes(bytes_read, bytes_written, seconds_elapsed); + stats_prev_n_written = cur_written; + stats_prev_n_read = cur_read; + } +#else + bytes_read = (size_t)(stats_n_bytes_read - stats_prev_n_read); + bytes_written = (size_t)(stats_n_bytes_written - stats_prev_n_written); + stats_prev_n_read = stats_n_bytes_read; + stats_prev_n_written = stats_n_bytes_written; +#endif + control_event_bandwidth_used((uint32_t)bytes_read,(uint32_t)bytes_written); control_event_stream_bandwidth_used(); - if (seconds_elapsed > 0) - connection_bucket_refill(seconds_elapsed, now); - stats_prev_global_read_bucket = global_read_bucket; - stats_prev_global_write_bucket = global_write_bucket; - if (server_mode(options) && - !we_are_hibernating() && + !net_is_disabled() && seconds_elapsed > 0 && can_complete_circuit && stats_n_seconds_working / TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT != (stats_n_seconds_working+seconds_elapsed) / TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { /* every 20 minutes, check and complain if necessary */ - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); if (me && !check_whether_orport_reachable()) { log_warn(LD_CONFIG,"Your server (%s:%d) has not managed to confirm that " "its ORPort is reachable. Please check your firewalls, ports, " @@ -1350,7 +1665,58 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) current_second = now; /* remember which second it is, for next time */ } -#ifndef MS_WINDOWS +#ifndef USE_BUFFEREVENTS +/** Timer: used to invoke refill_callback(). */ +static periodic_timer_t *refill_timer = NULL; + +/** Libevent callback: invoked periodically to refill token buckets + * and count r/w bytes. It is only used when bufferevents are disabled. */ +static void +refill_callback(periodic_timer_t *timer, void *arg) +{ + static struct timeval current_millisecond; + struct timeval now; + + size_t bytes_written; + size_t bytes_read; + int milliseconds_elapsed = 0; + int seconds_rolled_over = 0; + + const or_options_t *options = get_options(); + + (void)timer; + (void)arg; + + tor_gettimeofday(&now); + + /* If this is our first time, no time has passed. */ + if (current_millisecond.tv_sec) { + long mdiff = tv_mdiff(¤t_millisecond, &now); + if (mdiff > INT_MAX) + mdiff = INT_MAX; + milliseconds_elapsed = (int)mdiff; + seconds_rolled_over = (int)(now.tv_sec - current_millisecond.tv_sec); + } + + bytes_written = stats_prev_global_write_bucket - global_write_bucket; + bytes_read = stats_prev_global_read_bucket - global_read_bucket; + + stats_n_bytes_read += bytes_read; + stats_n_bytes_written += bytes_written; + if (accounting_is_enabled(options) && milliseconds_elapsed >= 0) + accounting_add_bytes(bytes_read, bytes_written, seconds_rolled_over); + + if (milliseconds_elapsed > 0) + connection_bucket_refill(milliseconds_elapsed, now.tv_sec); + + stats_prev_global_read_bucket = global_read_bucket; + stats_prev_global_write_bucket = global_write_bucket; + + current_millisecond = now; /* remember what time it is, for next time */ +} +#endif + +#ifndef _WIN32 /** Called when a possibly ignorable libevent error occurs; ensures that we * don't get into an infinite loop by ignoring too many errors from * libevent. */ @@ -1410,7 +1776,7 @@ dns_servers_relaunch_checks(void) static int do_hup(void) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); #ifdef USE_DMALLOC dmalloc_log_stats(); @@ -1434,8 +1800,16 @@ do_hup(void) } options = get_options(); /* they have changed now */ } else { + char *msg = NULL; log_notice(LD_GENERAL, "Not reloading config file: the controller told " "us not to."); + /* Make stuff get rescanned, reloaded, etc. */ + if (set_options((or_options_t*)options, &msg) < 0) { + if (!msg) + msg = tor_strdup("Unknown error"); + log_warn(LD_GENERAL, "Unable to re-set previous options: %s", msg); + tor_free(msg); + } } if (authdir_mode_handles_descs(options, -1)) { /* reload the approved-routers file */ @@ -1454,7 +1828,8 @@ do_hup(void) /* retry appropriate downloads */ router_reset_status_download_failures(); router_reset_descriptor_download_failures(); - update_networkstatus_downloads(time(NULL)); + if (!options->DisableNetwork) + update_networkstatus_downloads(time(NULL)); /* We'll retry routerstatus downloads in about 10 seconds; no need to * force a retry there. */ @@ -1502,8 +1877,10 @@ do_main_loop(void) /* Set up our buckets */ connection_bucket_init(); +#ifndef USE_BUFFEREVENTS stats_prev_global_read_bucket = global_read_bucket; stats_prev_global_write_bucket = global_write_bucket; +#endif /* initialize the bootstrap status events to know we're starting up */ control_event_bootstrap(BOOTSTRAP_STATUS_STARTING, 0); @@ -1546,11 +1923,27 @@ do_main_loop(void) tor_assert(second_timer); } +#ifndef USE_BUFFEREVENTS + if (!refill_timer) { + struct timeval refill_interval; + int msecs = get_options()->TokenBucketRefillInterval; + + refill_interval.tv_sec = msecs/1000; + refill_interval.tv_usec = (msecs%1000)*1000; + + refill_timer = periodic_timer_new(tor_libevent_get_base(), + &refill_interval, + refill_callback, + NULL); + tor_assert(refill_timer); + } +#endif + for (;;) { if (nt_service_is_stopping()) return 0; -#ifndef MS_WINDOWS +#ifndef _WIN32 /* Make it easier to tell whether libevent failure is our fault or not. */ errno = 0; #endif @@ -1574,7 +1967,7 @@ do_main_loop(void) log_err(LD_NET,"libevent call with %s failed: %s [%d]", tor_libevent_get_method(), tor_socket_strerror(e), e); return -1; -#ifndef MS_WINDOWS +#ifndef _WIN32 } else if (e == EINVAL) { log_warn(LD_NET, "EINVAL from libevent: should you upgrade libevent?"); if (got_libevent_error()) @@ -1593,7 +1986,7 @@ do_main_loop(void) } } -#ifndef MS_WINDOWS /* Only called when we're willing to use signals */ +#ifndef _WIN32 /* Only called when we're willing to use signals */ /** Libevent callback: invoked when we get a signal. */ static void @@ -1634,11 +2027,13 @@ process_signal(uintptr_t sig) case SIGUSR1: /* prefer to log it at INFO, but make sure we always see it */ dumpstats(get_min_log_level()<LOG_INFO ? get_min_log_level() : LOG_INFO); + control_event_signal(sig); break; case SIGUSR2: switch_logs_debug(); log_debug(LD_GENERAL,"Caught USR2, going to loglevel debug. " "Send HUP to change back."); + control_event_signal(sig); break; case SIGHUP: if (do_hup() < 0) { @@ -1646,6 +2041,7 @@ process_signal(uintptr_t sig) tor_cleanup(); exit(1); } + control_event_signal(sig); break; #ifdef SIGCHLD case SIGCHLD: @@ -1667,10 +2063,18 @@ process_signal(uintptr_t sig) } case SIGCLEARDNSCACHE: addressmap_clear_transient(); + control_event_signal(sig); break; } } +/** Returns Tor's uptime. */ +long +get_uptime(void) +{ + return stats_n_seconds_working; +} + extern uint64_t rephist_total_alloc; extern uint32_t rephist_total_num; @@ -1701,8 +2105,7 @@ dumpstats(int severity) log(severity, LD_GENERAL, "Dumping stats:"); - SMARTLIST_FOREACH(connection_array, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(connection_array, connection_t *, conn) { int i = conn_sl_idx; log(severity, LD_GENERAL, "Conn %d (socket %d) type %d (%s), state %d (%s), created %d secs ago", @@ -1717,13 +2120,13 @@ dumpstats(int severity) log(severity,LD_GENERAL, "Conn %d: %d bytes waiting on inbuf (len %d, last read %d secs ago)", i, - (int)buf_datalen(conn->inbuf), + (int)connection_get_inbuf_len(conn), (int)buf_allocation(conn->inbuf), (int)(now - conn->timestamp_lastread)); log(severity,LD_GENERAL, "Conn %d: %d bytes waiting on outbuf " "(len %d, last written %d secs ago)",i, - (int)buf_datalen(conn->outbuf), + (int)connection_get_outbuf_len(conn), (int)buf_allocation(conn->outbuf), (int)(now - conn->timestamp_lastwritten)); if (conn->type == CONN_TYPE_OR) { @@ -1740,7 +2143,7 @@ dumpstats(int severity) } circuit_dump_by_conn(conn, severity); /* dump info about all the circuits * using this conn */ - }); + } SMARTLIST_FOREACH_END(conn); log(severity, LD_NET, "Cells processed: "U64_FORMAT" padding\n" " "U64_FORMAT" create\n" @@ -1799,7 +2202,7 @@ exit_function(void) { /* NOTE: If we ever daemonize, this gets called immediately. That's * okay for now, because we only use this on Windows. */ -#ifdef MS_WINDOWS +#ifdef _WIN32 WSACleanup(); #endif } @@ -1808,7 +2211,7 @@ exit_function(void) void handle_signals(int is_parent) { -#ifndef MS_WINDOWS /* do signal stuff only on Unix */ +#ifndef _WIN32 /* do signal stuff only on Unix */ int i; static const int signals[] = { SIGINT, /* do a controlled slow shutdown */ @@ -1861,11 +2264,11 @@ tor_init(int argc, char *argv[]) int i, quiet = 0; time_of_process_start = time(NULL); if (!connection_array) - connection_array = smartlist_create(); + connection_array = smartlist_new(); if (!closeable_connection_lst) - closeable_connection_lst = smartlist_create(); + closeable_connection_lst = smartlist_new(); if (!active_linked_connection_lst) - active_linked_connection_lst = smartlist_create(); + active_linked_connection_lst = smartlist_new(); /* Have the log set up with our application name. */ tor_snprintf(buf, sizeof(buf), "Tor %s", get_version()); log_set_application_name(buf); @@ -1895,10 +2298,30 @@ tor_init(int argc, char *argv[]) default: add_temp_log(LOG_NOTICE); } + quiet_level = quiet; + + { + const char *version = get_version(); +#ifdef USE_BUFFEREVENTS + log_notice(LD_GENERAL, "Tor v%s (with bufferevents) running on %s.", + version, get_uname()); +#else + log_notice(LD_GENERAL, "Tor v%s running on %s.", version, get_uname()); +#endif + + log_notice(LD_GENERAL, "Tor can't help you if you use it wrong! " + "Learn how to be safe at " + "https://www.torproject.org/download/download#warning"); + + if (strstr(version, "alpha") || strstr(version, "beta")) + log_notice(LD_GENERAL, "This version is not a stable Tor release. " + "Expect more bugs than usual."); + } - log(LOG_NOTICE, LD_GENERAL, "Tor v%s. This is experimental software. " - "Do not rely on it for strong anonymity. (Running on %s)",get_version(), - get_uname()); +#ifdef NON_ANONYMOUS_MODE_ENABLED + log(LOG_WARN, LD_GENERAL, "This copy of Tor was compiled to run in a " + "non-anonymous mode. It will provide NO ANONYMITY."); +#endif if (network_init()<0) { log_err(LD_BUG,"Error initializing network; exiting."); @@ -1911,7 +2334,7 @@ tor_init(int argc, char *argv[]) return -1; } -#ifndef MS_WINDOWS +#ifndef _WIN32 if (geteuid()==0) log_warn(LD_GENERAL,"You are running Tor as root. You don't need to, " "and you probably shouldn't."); @@ -1938,7 +2361,7 @@ static tor_lockfile_t *lockfile = NULL; * return -1 if we can't get the lockfile. Return 0 on success. */ int -try_locking(or_options_t *options, int err_if_locked) +try_locking(const or_options_t *options, int err_if_locked) { if (lockfile) return 0; @@ -1953,7 +2376,7 @@ try_locking(or_options_t *options, int err_if_locked) log_warn(LD_GENERAL, "It looks like another Tor process is running " "with the same data directory. Waiting 5 seconds to see " "if it goes away."); -#ifndef WIN32 +#ifndef _WIN32 sleep(5); #else Sleep(5000); @@ -2017,9 +2440,11 @@ tor_free_all(int postfork) clear_pending_onions(); circuit_free_all(); entry_guards_free_all(); + pt_free_all(); connection_free_all(); buf_shrink_freelists(1); memarea_clear_freelist(); + nodelist_free_all(); microdesc_free_all(); if (!postfork) { config_free_all(); @@ -2051,7 +2476,7 @@ tor_free_all(int postfork) void tor_cleanup(void) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (options->command == CMD_RUN_TOR) { time_t now = time(NULL); /* Remove our pid file. We don't care if there was an error when we @@ -2086,7 +2511,7 @@ tor_cleanup(void) do_list_fingerprint(void) { char buf[FINGERPRINT_LEN+1]; - crypto_pk_env_t *k; + crypto_pk_t *k; const char *nickname = get_options()->Nickname; if (!server_mode(get_options())) { log_err(LD_GENERAL, @@ -2171,8 +2596,7 @@ tor_main(int argc, char *argv[]) // a file on a folder shared by the wm emulator. // if no flashcard (real or emulated) is present, // log files will be written in the root folder - if (find_flashcard_path(path,MAX_PATH) == -1) - { + if (find_flashcard_path(path,MAX_PATH) == -1) { redir = _wfreopen( L"\\stdout.log", L"w", stdout ); redirdbg = _wfreopen( L"\\stderr.log", L"w", stderr ); } else { @@ -2187,7 +2611,7 @@ tor_main(int argc, char *argv[]) } #endif -#ifdef MS_WINDOWS +#ifdef _WIN32 /* Call SetProcessDEPPolicy to permanently enable DEP. The function will not resolve on earlier versions of Windows, and failure is not dangerous. */ diff --git a/src/or/main.h b/src/or/main.h index 0551f7aaf9..f843b6f9fc 100644 --- a/src/or/main.h +++ b/src/or/main.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -14,7 +14,9 @@ extern int can_complete_circuit; -int connection_add(connection_t *conn); +int connection_add_impl(connection_t *conn, int is_connecting); +#define connection_add(conn) connection_add_impl((conn), 0) +#define connection_add_connecting(conn) connection_add_impl((conn), 1) int connection_remove(connection_t *conn); void connection_unregister_events(connection_t *conn); int connection_in_array(connection_t *conn); @@ -22,10 +24,13 @@ void add_connection_to_closeable_list(connection_t *conn); int connection_is_on_closeable_list(connection_t *conn); smartlist_t *get_connection_array(void); +uint64_t get_bytes_read(void); +uint64_t get_bytes_written(void); /** Bitmask for events that we can turn on and off with * connection_watch_events. */ typedef enum watchable_events { + /* Yes, it is intentional that these match Libevent's EV_READ and EV_WRITE */ READ_EVENT=0x02, /**< We want to know when a connection is readable */ WRITE_EVENT=0x04 /**< We want to know when a connection is writable */ } watchable_events_t; @@ -46,10 +51,13 @@ void directory_info_has_arrived(time_t now, int from_cache); void ip_address_changed(int at_interface); void dns_servers_relaunch_checks(void); +long get_uptime(void); +unsigned get_signewnym_epoch(void); + void handle_signals(int is_parent); void process_signal(uintptr_t sig); -int try_locking(or_options_t *options, int err_if_locked); +int try_locking(const or_options_t *options, int err_if_locked); int have_lockfile(void); void release_lockfile(void); diff --git a/src/or/microdesc.c b/src/or/microdesc.c index 7c67d51448..b4d22c1c62 100644 --- a/src/or/microdesc.c +++ b/src/or/microdesc.c @@ -1,9 +1,17 @@ -/* Copyright (c) 2009-2011, The Tor Project, Inc. */ +/* Copyright (c) 2009-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #include "or.h" +#include "circuitbuild.h" #include "config.h" +#include "directory.h" +#include "dirserv.h" #include "microdesc.h" +#include "networkstatus.h" +#include "nodelist.h" +#include "policies.h" +#include "router.h" +#include "routerlist.h" #include "routerparse.h" /** A data structure to hold a bunch of cached microdescriptors. There are @@ -72,7 +80,12 @@ dump_microdescriptor(FILE *f, microdesc_t *md, size_t *annotation_len_out) char annotation[ISO_TIME_LEN+32]; format_iso_time(buf, md->last_listed); tor_snprintf(annotation, sizeof(annotation), "@last-listed %s\n", buf); - fputs(annotation, f); + if (fputs(annotation, f) < 0) { + log_warn(LD_DIR, + "Couldn't write microdescriptor annotation: %s", + strerror(ferror(f))); + return -1; + } r += strlen(annotation); *annotation_len_out = r; } else { @@ -121,15 +134,19 @@ get_microdesc_cache(void) * ending at <b>eos</b>, and store them in <b>cache</b>. If <b>no-save</b>, * mark them as non-writable to disk. If <b>where</b> is SAVED_IN_CACHE, * leave their bodies as pointers to the mmap'd cache. If where is - * <b>SAVED_NOWHERE</b>, do not allow annotations. Return a list of the added - * microdescriptors. */ + * <b>SAVED_NOWHERE</b>, do not allow annotations. If listed_at is positive, + * set the last_listed field of every microdesc to listed_at. If + * requested_digests is non-null, then it contains a list of digests we mean + * to allow, so we should reject any non-requested microdesc with a different + * digest, and alter the list to contain only the digests of those microdescs + * we didn't find. + * Return a newly allocated list of the added microdescriptors, or NULL */ smartlist_t * microdescs_add_to_cache(microdesc_cache_t *cache, const char *s, const char *eos, saved_location_t where, - int no_save) + int no_save, time_t listed_at, + smartlist_t *requested_digests256) { - /*XXXX need an argument that sets last_listed as appropriate. */ - smartlist_t *descriptors, *added; const int allow_annotations = (where != SAVED_NOWHERE); const int copy_body = (where != SAVED_IN_CACHE); @@ -137,14 +154,41 @@ microdescs_add_to_cache(microdesc_cache_t *cache, descriptors = microdescs_parse_from_string(s, eos, allow_annotations, copy_body); + if (listed_at > 0) { + SMARTLIST_FOREACH(descriptors, microdesc_t *, md, + md->last_listed = listed_at); + } + if (requested_digests256) { + digestmap_t *requested; /* XXXX actuqlly we should just use a + digest256map */ + requested = digestmap_new(); + SMARTLIST_FOREACH(requested_digests256, const char *, cp, + digestmap_set(requested, cp, (void*)1)); + SMARTLIST_FOREACH_BEGIN(descriptors, microdesc_t *, md) { + if (digestmap_get(requested, md->digest)) { + digestmap_set(requested, md->digest, (void*)2); + } else { + log_fn(LOG_PROTOCOL_WARN, LD_DIR, "Received non-requested microcdesc"); + microdesc_free(md); + SMARTLIST_DEL_CURRENT(descriptors, md); + } + } SMARTLIST_FOREACH_END(md); + SMARTLIST_FOREACH_BEGIN(requested_digests256, char *, cp) { + if (digestmap_get(requested, cp) == (void*)2) { + tor_free(cp); + SMARTLIST_DEL_CURRENT(requested_digests256, cp); + } + } SMARTLIST_FOREACH_END(cp); + digestmap_free(requested, NULL); + } added = microdescs_add_list_to_cache(cache, descriptors, where, no_save); smartlist_free(descriptors); return added; } -/* As microdescs_add_to_cache, but takes a list of micrdescriptors instead of - * a string to encode. Frees any members of <b>descriptors</b> that it does +/** As microdescs_add_to_cache, but takes a list of micrdescriptors instead of + * a string to decode. Frees any members of <b>descriptors</b> that it does * not add. */ smartlist_t * microdescs_add_list_to_cache(microdesc_cache_t *cache, @@ -168,7 +212,7 @@ microdescs_add_list_to_cache(microdesc_cache_t *cache, } } - added = smartlist_create(); + added = smartlist_new(); SMARTLIST_FOREACH_BEGIN(descriptors, microdesc_t *, md) { microdesc_t *md2; md2 = HT_FIND(microdesc_map, &cache->map, md); @@ -187,9 +231,10 @@ microdescs_add_list_to_cache(microdesc_cache_t *cache, size_t annotation_len; size = dump_microdescriptor(f, md, &annotation_len); if (size < 0) { - /* XXX handle errors from dump_microdescriptor() */ - /* log? return -1? die? coredump the universe? */ - continue; + /* we already warned in dump_microdescriptor; */ + abort_writing_to_file(open_file); + smartlist_clear(added); + return added; } md->saved_location = SAVED_IN_JOURNAL; cache->journal_len += size; @@ -200,6 +245,7 @@ microdescs_add_list_to_cache(microdesc_cache_t *cache, md->no_save = no_save; HT_INSERT(microdesc_map, &cache->map, md); + md->held_in_map = 1; smartlist_add(added, md); ++cache->n_seen; cache->total_len_seen += md->bodylen; @@ -208,6 +254,15 @@ microdescs_add_list_to_cache(microdesc_cache_t *cache, if (f) finish_writing_to_file(open_file); /*XXX Check me.*/ + { + networkstatus_t *ns = networkstatus_get_latest_consensus(); + if (ns && ns->flavor == FLAV_MICRODESC) + SMARTLIST_FOREACH(added, microdesc_t *, md, nodelist_add_microdesc(md)); + } + + if (smartlist_len(added)) + router_dir_info_changed(); + return added; } @@ -219,6 +274,7 @@ microdesc_cache_clear(microdesc_cache_t *cache) for (entry = HT_START(microdesc_map, &cache->map); entry; entry = next) { microdesc_t *md = *entry; next = HT_NEXT_RMV(microdesc_map, &cache->map, entry); + md->held_in_map = 0; microdesc_free(md); } HT_CLEAR(microdesc_map, &cache->map); @@ -247,7 +303,7 @@ microdesc_cache_reload(microdesc_cache_t *cache) mm = cache->cache_content = tor_mmap_file(cache->cache_fname); if (mm) { added = microdescs_add_to_cache(cache, mm->data, mm->data+mm->size, - SAVED_IN_CACHE, 0); + SAVED_IN_CACHE, 0, -1, NULL); if (added) { total += smartlist_len(added); smartlist_free(added); @@ -260,7 +316,7 @@ microdesc_cache_reload(microdesc_cache_t *cache) cache->journal_len = (size_t) st.st_size; added = microdescs_add_to_cache(cache, journal_content, journal_content+st.st_size, - SAVED_IN_JOURNAL, 0); + SAVED_IN_JOURNAL, 0, -1, NULL); if (added) { total += smartlist_len(added); smartlist_free(added); @@ -293,9 +349,11 @@ microdesc_cache_clean(microdesc_cache_t *cache, time_t cutoff, int force) size_t bytes_dropped = 0; time_t now = time(NULL); - (void) force; - /* In 0.2.2, we let this proceed unconditionally: only authorities have - * microdesc caches. */ + /* If we don't know a live consensus, don't believe last_listed values: we + * might be starting up after being down for a while. */ + if (! force && + ! networkstatus_get_reasonably_live_consensus(now, FLAV_MICRODESC)) + return; if (cutoff <= 0) cutoff = now - TOLERATE_MICRODESC_AGE; @@ -305,6 +363,7 @@ microdesc_cache_clean(microdesc_cache_t *cache, time_t cutoff, int force) ++dropped; victim = *mdp; mdp = HT_NEXT_RMV(microdesc_map, &cache->map, mdp); + victim->held_in_map = 0; bytes_dropped += victim->bodylen; microdesc_free(victim); } else { @@ -314,8 +373,8 @@ microdesc_cache_clean(microdesc_cache_t *cache, time_t cutoff, int force) } if (dropped) { - log_notice(LD_DIR, "Removed %d/%d microdescriptors as old.", - dropped,dropped+kept); + log_info(LD_DIR, "Removed %d/%d microdescriptors as old.", + dropped,dropped+kept); cache->bytes_dropped += bytes_dropped; } } @@ -376,7 +435,7 @@ microdesc_cache_rebuild(microdesc_cache_t *cache, int force) if (!f) return -1; - wrote = smartlist_create(); + wrote = smartlist_new(); HT_FOREACH(mdp, microdesc_map, &cache->map) { microdesc_t *md = *mdp; @@ -390,6 +449,7 @@ microdesc_cache_rebuild(microdesc_cache_t *cache, int force) /* log? return -1? die? coredump the universe? */ continue; } + tor_assert(((size_t)size) == annotation_len + md->bodylen); md->off = off + annotation_len; off += size; if (md->saved_location != SAVED_IN_CACHE) { @@ -415,7 +475,21 @@ microdesc_cache_rebuild(microdesc_cache_t *cache, int force) SMARTLIST_FOREACH_BEGIN(wrote, microdesc_t *, md) { tor_assert(md->saved_location == SAVED_IN_CACHE); md->body = (char*)cache->cache_content->data + md->off; - tor_assert(fast_memeq(md->body, "onion-key", 9)); + if (PREDICT_UNLIKELY( + md->bodylen < 9 || fast_memneq(md->body, "onion-key", 9) != 0)) { + /* XXXX once bug 2022 is solved, we can kill this block and turn it + * into just the tor_assert(!memcmp) */ + off_t avail = cache->cache_content->size - md->off; + char *bad_str; + tor_assert(avail >= 0); + bad_str = tor_strndup(md->body, MIN(128, (size_t)avail)); + log_err(LD_BUG, "After rebuilding microdesc cache, offsets seem wrong. " + " At offset %d, I expected to find a microdescriptor starting " + " with \"onion-key\". Instead I got %s.", + (int)md->off, escaped(bad_str)); + tor_free(bad_str); + tor_assert(fast_memeq(md->body, "onion-key", 9)); + } } SMARTLIST_FOREACH_END(md); smartlist_free(wrote); @@ -432,6 +506,28 @@ microdesc_cache_rebuild(microdesc_cache_t *cache, int force) return 0; } +/** Make sure that the reference count of every microdescriptor in cache is + * accurate. */ +void +microdesc_check_counts(void) +{ + microdesc_t **mdp; + if (!the_microdesc_cache) + return; + + HT_FOREACH(mdp, microdesc_map, &the_microdesc_cache->map) { + microdesc_t *md = *mdp; + unsigned int found=0; + const smartlist_t *nodes = nodelist_get_list(); + SMARTLIST_FOREACH(nodes, node_t *, node, { + if (node->md == md) { + ++found; + } + }); + tor_assert(found == md->held_by_nodes); + } +} + /** Deallocate a single microdescriptor. Note: the microdescriptor MUST have * previously been removed from the cache if it had ever been inserted. */ void @@ -439,9 +535,45 @@ microdesc_free(microdesc_t *md) { if (!md) return; - /* Must be removed from hash table! */ + + /* Make sure that the microdesc was really removed from the appropriate data + structures. */ + if (md->held_in_map) { + microdesc_cache_t *cache = get_microdesc_cache(); + microdesc_t *md2 = HT_FIND(microdesc_map, &cache->map, md); + if (md2 == md) { + log_warn(LD_BUG, "microdesc_free() called, but md was still in " + "microdesc_map"); + HT_REMOVE(microdesc_map, &cache->map, md); + } else { + log_warn(LD_BUG, "microdesc_free() called with held_in_map set, but " + "microdesc was not in the map."); + } + tor_fragile_assert(); + } + if (md->held_by_nodes) { + int found=0; + const smartlist_t *nodes = nodelist_get_list(); + SMARTLIST_FOREACH(nodes, node_t *, node, { + if (node->md == md) { + ++found; + node->md = NULL; + } + }); + if (found) { + log_warn(LD_BUG, "microdesc_free() called, but md was still referenced " + "%d node(s); held_by_nodes == %u", found, md->held_by_nodes); + } else { + log_warn(LD_BUG, "microdesc_free() called with held_by_nodes set to %u, " + "but md was not referenced by any nodes", md->held_by_nodes); + } + tor_fragile_assert(); + } + //tor_assert(md->held_in_map == 0); + //tor_assert(md->held_by_nodes == 0); + if (md->onion_pkey) - crypto_free_pk_env(md->onion_pkey); + crypto_pk_free(md->onion_pkey); if (md->body && md->saved_location != SAVED_IN_CACHE) tor_free(md->body); @@ -449,7 +581,7 @@ microdesc_free(microdesc_t *md) SMARTLIST_FOREACH(md->family, char *, cp, tor_free(cp)); smartlist_free(md->family); } - tor_free(md->exitsummary); + short_policy_free(md->exit_policy); tor_free(md); } @@ -491,3 +623,153 @@ microdesc_average_size(microdesc_cache_t *cache) return (size_t)(cache->total_len_seen / cache->n_seen); } +/** Return a smartlist of all the sha256 digest of the microdescriptors that + * are listed in <b>ns</b> but not present in <b>cache</b>. Returns pointers + * to internals of <b>ns</b>; you should not free the members of the resulting + * smartlist. Omit all microdescriptors whose digest appear in <b>skip</b>. */ +smartlist_t * +microdesc_list_missing_digest256(networkstatus_t *ns, microdesc_cache_t *cache, + int downloadable_only, digestmap_t *skip) +{ + smartlist_t *result = smartlist_new(); + time_t now = time(NULL); + tor_assert(ns->flavor == FLAV_MICRODESC); + SMARTLIST_FOREACH_BEGIN(ns->routerstatus_list, routerstatus_t *, rs) { + if (microdesc_cache_lookup_by_digest256(cache, rs->descriptor_digest)) + continue; + if (downloadable_only && + !download_status_is_ready(&rs->dl_status, now, + MAX_MICRODESC_DOWNLOAD_FAILURES)) + continue; + if (skip && digestmap_get(skip, rs->descriptor_digest)) + continue; + if (tor_mem_is_zero(rs->descriptor_digest, DIGEST256_LEN)) + continue; + /* XXXX Also skip if we're a noncache and wouldn't use this router. + * XXXX NM Microdesc + */ + smartlist_add(result, rs->descriptor_digest); + } SMARTLIST_FOREACH_END(rs); + return result; +} + +/** Launch download requests for microdescriptors as appropriate. + * + * Specifically, we should launch download requests if we are configured to + * download mirodescriptors, and there are some microdescriptors listed the + * current microdesc consensus that we don't have, and either we never asked + * for them, or we failed to download them but we're willing to retry. + */ +void +update_microdesc_downloads(time_t now) +{ + const or_options_t *options = get_options(); + networkstatus_t *consensus; + smartlist_t *missing; + digestmap_t *pending; + + if (should_delay_dir_fetches(options)) + return; + if (directory_too_idle_to_fetch_descriptors(options, now)) + return; + + consensus = networkstatus_get_reasonably_live_consensus(now, FLAV_MICRODESC); + if (!consensus) + return; + + if (!we_fetch_microdescriptors(options)) + return; + + pending = digestmap_new(); + list_pending_microdesc_downloads(pending); + + missing = microdesc_list_missing_digest256(consensus, + get_microdesc_cache(), + 1, + pending); + digestmap_free(pending, NULL); + + launch_descriptor_downloads(DIR_PURPOSE_FETCH_MICRODESC, + missing, NULL, now); + + smartlist_free(missing); +} + +/** For every microdescriptor listed in the current microdecriptor consensus, + * update its last_listed field to be at least as recent as the publication + * time of the current microdescriptor consensus. + */ +void +update_microdescs_from_networkstatus(time_t now) +{ + microdesc_cache_t *cache = get_microdesc_cache(); + microdesc_t *md; + networkstatus_t *ns = + networkstatus_get_reasonably_live_consensus(now, FLAV_MICRODESC); + + if (! ns) + return; + + tor_assert(ns->flavor == FLAV_MICRODESC); + + SMARTLIST_FOREACH_BEGIN(ns->routerstatus_list, routerstatus_t *, rs) { + md = microdesc_cache_lookup_by_digest256(cache, rs->descriptor_digest); + if (md && ns->valid_after > md->last_listed) + md->last_listed = ns->valid_after; + } SMARTLIST_FOREACH_END(rs); +} + +/** Return true iff we should prefer to use microdescriptors rather than + * routerdescs for building circuits. */ +int +we_use_microdescriptors_for_circuits(const or_options_t *options) +{ + int ret = options->UseMicrodescriptors; + if (ret == -1) { + /* UseMicrodescriptors is "auto"; we need to decide: */ + /* If we are configured to use bridges and one of our bridges doesn't + * know what a microdescriptor is, the answer is no. */ + if (options->UseBridges && any_bridges_dont_support_microdescriptors()) + return 0; + /* Otherwise, we decide that we'll use microdescriptors iff we are + * not a server, and we're not autofetching everything. */ + /* XXX023 what does not being a server have to do with it? also there's + * a partitioning issue here where bridges differ from clients. */ + ret = !server_mode(options) && !options->FetchUselessDescriptors; + } + return ret; +} + +/** Return true iff we should try to download microdescriptors at all. */ +int +we_fetch_microdescriptors(const or_options_t *options) +{ + if (directory_caches_dir_info(options)) + return 1; + if (options->FetchUselessDescriptors) + return 1; + return we_use_microdescriptors_for_circuits(options); +} + +/** Return true iff we should try to download router descriptors at all. */ +int +we_fetch_router_descriptors(const or_options_t *options) +{ + if (directory_caches_dir_info(options)) + return 1; + if (options->FetchUselessDescriptors) + return 1; + return ! we_use_microdescriptors_for_circuits(options); +} + +/** Return the consensus flavor we actually want to use to build circuits. */ +int +usable_consensus_flavor(void) +{ + if (we_use_microdescriptors_for_circuits(get_options())) { + return FLAV_MICRODESC; + } else { + return FLAV_NS; + } +} + diff --git a/src/or/microdesc.h b/src/or/microdesc.h index 77ce8536bc..5646fc7a85 100644 --- a/src/or/microdesc.h +++ b/src/or/microdesc.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -14,9 +14,12 @@ microdesc_cache_t *get_microdesc_cache(void); +void microdesc_check_counts(void); + smartlist_t *microdescs_add_to_cache(microdesc_cache_t *cache, const char *s, const char *eos, saved_location_t where, - int no_save); + int no_save, time_t listed_at, + smartlist_t *requested_digests256); smartlist_t *microdescs_add_list_to_cache(microdesc_cache_t *cache, smartlist_t *descriptors, saved_location_t where, int no_save); @@ -31,8 +34,21 @@ microdesc_t *microdesc_cache_lookup_by_digest256(microdesc_cache_t *cache, size_t microdesc_average_size(microdesc_cache_t *cache); +smartlist_t *microdesc_list_missing_digest256(networkstatus_t *ns, + microdesc_cache_t *cache, + int downloadable_only, + digestmap_t *skip); + void microdesc_free(microdesc_t *md); void microdesc_free_all(void); +void update_microdesc_downloads(time_t now); +void update_microdescs_from_networkstatus(time_t now); + +int usable_consensus_flavor(void); +int we_fetch_microdescriptors(const or_options_t *options); +int we_fetch_router_descriptors(const or_options_t *options); +int we_use_microdescriptors_for_circuits(const or_options_t *options); + #endif diff --git a/src/or/networkstatus.c b/src/or/networkstatus.c index d5637b2c93..2553a74e50 100644 --- a/src/or/networkstatus.c +++ b/src/or/networkstatus.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -20,7 +20,9 @@ #include "dirserv.h" #include "dirvote.h" #include "main.h" +#include "microdesc.h" #include "networkstatus.h" +#include "nodelist.h" #include "relay.h" #include "router.h" #include "routerlist.h" @@ -44,8 +46,21 @@ static strmap_t *named_server_map = NULL; * as unnamed for some server in the consensus. */ static strmap_t *unnamed_server_map = NULL; -/** Most recently received and validated v3 consensus network status. */ -static networkstatus_t *current_consensus = NULL; +/** Most recently received and validated v3 consensus network status, + * of whichever type we are using for our own circuits. This will be the same + * as one of current_ns_consensus or current_md_consensus. + */ +#define current_consensus \ + (we_use_microdescriptors_for_circuits(get_options()) ? \ + current_md_consensus : current_ns_consensus) + +/** Most recently received and validated v3 "ns"-flavored consensus network + * status. */ +static networkstatus_t *current_ns_consensus = NULL; + +/** Most recently received and validated v3 "microdec"-flavored consensus + * network status. */ +static networkstatus_t *current_md_consensus = NULL; /** A v3 consensus networkstatus that we've received, but which we don't * have enough certificates to be happy about. */ @@ -63,6 +78,9 @@ typedef struct consensus_waiting_for_certs_t { int dl_failed; } consensus_waiting_for_certs_t; +/** An array, for each flavor of consensus we might want, of consensuses that + * we have downloaded, but which we cannot verify due to having insufficient + * authority certificates. */ static consensus_waiting_for_certs_t consensus_waiting_for_certs[N_CONSENSUS_FLAVORS]; @@ -74,7 +92,7 @@ static time_t last_networkstatus_download_attempted = 0; /** A time before which we shouldn't try to replace the current consensus: * this will be at some point after the next consensus becomes valid, but * before the current consensus becomes invalid. */ -static time_t time_to_download_next_consensus = 0; +static time_t time_to_download_next_consensus[N_CONSENSUS_FLAVORS]; /** Download status for the current consensus networkstatus. */ static download_status_t consensus_dl_status[N_CONSENSUS_FLAVORS]; @@ -94,9 +112,8 @@ void networkstatus_reset_warnings(void) { if (current_consensus) { - SMARTLIST_FOREACH(current_consensus->routerstatus_list, - routerstatus_t *, rs, - rs->name_lookup_warned = 0); + SMARTLIST_FOREACH(nodelist_get_list(), node_t *, node, + node->name_lookup_warned = 0); } have_warned_about_old_version = 0; @@ -111,12 +128,12 @@ networkstatus_reset_download_failures(void) { int i; const smartlist_t *networkstatus_v2_list = networkstatus_get_v2_list(); - SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, - SMARTLIST_FOREACH(ns->entries, routerstatus_t *, rs, - { + SMARTLIST_FOREACH_BEGIN(networkstatus_v2_list, networkstatus_v2_t *, ns) { + SMARTLIST_FOREACH_BEGIN(ns->entries, routerstatus_t *, rs) { if (!router_get_by_descriptor_digest(rs->descriptor_digest)) rs->need_to_mirror = 1; - }));; + } SMARTLIST_FOREACH_END(rs); + } SMARTLIST_FOREACH_END(ns); for (i=0; i < N_CONSENSUS_FLAVORS; ++i) download_status_reset(&consensus_dl_status[i]); @@ -147,7 +164,7 @@ router_reload_v2_networkstatus(void) int maybe_delete = !directory_caches_v2_dir_info(get_options()); time_t now = time(NULL); if (!networkstatus_v2_list) - networkstatus_v2_list = smartlist_create(); + networkstatus_v2_list = smartlist_new(); entries = tor_listdir(filename); if (!entries) { /* dir doesn't exist */ @@ -160,7 +177,7 @@ router_reload_v2_networkstatus(void) return 0; } tor_free(filename); - SMARTLIST_FOREACH(entries, const char *, fn, { + SMARTLIST_FOREACH_BEGIN(entries, const char *, fn) { char buf[DIGEST_LEN]; if (maybe_delete) { filename = get_datadir_fname2("cached-status", fn); @@ -184,7 +201,7 @@ router_reload_v2_networkstatus(void) tor_free(s); } tor_free(filename); - }); + } SMARTLIST_FOREACH_END(fn); SMARTLIST_FOREACH(entries, char *, fn, tor_free(fn)); smartlist_free(entries); networkstatus_v2_list_clean(time(NULL)); @@ -199,7 +216,7 @@ router_reload_consensus_networkstatus(void) char *filename; char *s; struct stat st; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); const unsigned int flags = NSSET_FROM_CACHE | NSSET_DONT_DOWNLOAD_CERTS; int flav; @@ -271,6 +288,7 @@ router_reload_consensus_networkstatus(void) update_certificate_downloads(time(NULL)); routers_update_all_from_networkstatus(time(NULL), 3); + update_microdescs_from_networkstatus(time(NULL)); return 0; } @@ -311,7 +329,7 @@ networkstatus_v2_free(networkstatus_v2_t *ns) tor_free(ns->source_address); tor_free(ns->contact); if (ns->signing_key) - crypto_free_pk_env(ns->signing_key); + crypto_pk_free(ns->signing_key); tor_free(ns->client_versions); tor_free(ns->server_versions); if (ns->entries) { @@ -395,7 +413,7 @@ networkstatus_vote_free(networkstatus_t *ns) digestmap_free(ns->desc_digest_map, NULL); - memset(ns, 11, sizeof(*ns)); + memwipe(ns, 11, sizeof(*ns)); tor_free(ns); } @@ -469,13 +487,13 @@ networkstatus_check_consensus_signature(networkstatus_t *consensus, int n_bad = 0; int n_unknown = 0; int n_no_signature = 0; - int n_v3_authorities = get_n_authorities(V3_AUTHORITY); + int n_v3_authorities = get_n_authorities(V3_DIRINFO); int n_required = n_v3_authorities/2 + 1; - smartlist_t *list_good = smartlist_create(); - smartlist_t *list_no_signature = smartlist_create(); - smartlist_t *need_certs_from = smartlist_create(); - smartlist_t *unrecognized = smartlist_create(); - smartlist_t *missing_authorities = smartlist_create(); + smartlist_t *list_good = smartlist_new(); + smartlist_t *list_no_signature = smartlist_new(); + smartlist_t *need_certs_from = smartlist_new(); + smartlist_t *unrecognized = smartlist_new(); + smartlist_t *missing_authorities = smartlist_new(); int severity; time_t now = time(NULL); @@ -545,7 +563,7 @@ networkstatus_check_consensus_signature(networkstatus_t *consensus, SMARTLIST_FOREACH(router_get_trusted_dir_servers(), trusted_dir_server_t *, ds, { - if ((ds->type & V3_AUTHORITY) && + if ((ds->type & V3_DIRINFO) && !networkstatus_get_voter_by_id(consensus, ds->v3_identity_digest)) smartlist_add(missing_authorities, ds); }); @@ -583,39 +601,38 @@ networkstatus_check_consensus_signature(networkstatus_t *consensus, hex_str(ds->v3_identity_digest, DIGEST_LEN)); }); { - smartlist_t *sl = smartlist_create(); - char *cp; + char *joined; + smartlist_t *sl = smartlist_new(); char *tmp = smartlist_join_strings(list_good, " ", 0, NULL); - tor_asprintf(&cp, "A consensus needs %d good signatures from recognized " + smartlist_add_asprintf(sl, + "A consensus needs %d good signatures from recognized " "authorities for us to accept it. This one has %d (%s).", n_required, n_good, tmp); tor_free(tmp); - smartlist_add(sl,cp); if (n_no_signature) { tmp = smartlist_join_strings(list_no_signature, " ", 0, NULL); - tor_asprintf(&cp, "%d (%s) of the authorities we know didn't sign it.", + smartlist_add_asprintf(sl, + "%d (%s) of the authorities we know didn't sign it.", n_no_signature, tmp); tor_free(tmp); - smartlist_add(sl,cp); } if (n_unknown) { - tor_asprintf(&cp, "It has %d signatures from authorities we don't " + smartlist_add_asprintf(sl, + "It has %d signatures from authorities we don't " "recognize.", n_unknown); - smartlist_add(sl,cp); } if (n_bad) { - tor_asprintf(&cp, "%d of the signatures on it didn't verify " + smartlist_add_asprintf(sl, "%d of the signatures on it didn't verify " "correctly.", n_bad); - smartlist_add(sl,cp); } if (n_missing_key) { - tor_asprintf(&cp, "We were unable to check %d of the signatures, " + smartlist_add_asprintf(sl, + "We were unable to check %d of the signatures, " "because we were missing the keys.", n_missing_key); - smartlist_add(sl,cp); } - cp = smartlist_join_strings(sl, " ", 0, NULL); - log(severity, LD_DIR, "%s", cp); - tor_free(cp); + joined = smartlist_join_strings(sl, " ", 0, NULL); + log(severity, LD_DIR, "%s", joined); + tor_free(joined); SMARTLIST_FOREACH(sl, char *, c, tor_free(c)); smartlist_free(sl); } @@ -734,7 +751,7 @@ router_set_networkstatus_v2(const char *s, time_t arrived_at, base16_encode(fp, HEX_DIGEST_LEN+1, ns->identity_digest, DIGEST_LEN); if (!(trusted_dir = router_get_trusteddirserver_by_digest(ns->identity_digest)) || - !(trusted_dir->type & V2_AUTHORITY)) { + !(trusted_dir->type & V2_DIRINFO)) { log_info(LD_DIR, "Network status was signed, but not by an authoritative " "directory we recognize."); source_desc = fp; @@ -764,7 +781,7 @@ router_set_networkstatus_v2(const char *s, time_t arrived_at, } if (!networkstatus_v2_list) - networkstatus_v2_list = smartlist_create(); + networkstatus_v2_list = smartlist_new(); if ( (source == NS_FROM_DIR_BY_FP || source == NS_FROM_DIR_ALL) && router_digest_is_me(ns->identity_digest)) { @@ -864,8 +881,7 @@ router_set_networkstatus_v2(const char *s, time_t arrived_at, { time_t live_until = ns->published_on + V2_NETWORKSTATUS_ROUTER_LIFETIME; - SMARTLIST_FOREACH(ns->entries, routerstatus_t *, rs, - { + SMARTLIST_FOREACH_BEGIN(ns->entries, routerstatus_t *, rs) { signed_descriptor_t *sd = router_get_by_descriptor_digest(rs->descriptor_digest); if (sd) { @@ -874,7 +890,7 @@ router_set_networkstatus_v2(const char *s, time_t arrived_at, } else { rs->need_to_mirror = 1; } - }); + } SMARTLIST_FOREACH_END(rs); } log_info(LD_DIR, "Setting networkstatus %s %s (published %s)", @@ -938,10 +954,9 @@ compare_digest_to_routerstatus_entry(const void *_key, const void **_member) return tor_memcmp(key, rs->identity_digest, DIGEST_LEN); } -/** Return the entry in <b>ns</b> for the identity digest <b>digest</b>, or - * NULL if none was found. */ +/** As networkstatus_v2_find_entry, but do not return a const pointer */ routerstatus_t * -networkstatus_v2_find_entry(networkstatus_v2_t *ns, const char *digest) +networkstatus_v2_find_mutable_entry(networkstatus_v2_t *ns, const char *digest) { return smartlist_bsearch(ns->entries, digest, compare_digest_to_routerstatus_entry); @@ -949,14 +964,29 @@ networkstatus_v2_find_entry(networkstatus_v2_t *ns, const char *digest) /** Return the entry in <b>ns</b> for the identity digest <b>digest</b>, or * NULL if none was found. */ +const routerstatus_t * +networkstatus_v2_find_entry(networkstatus_v2_t *ns, const char *digest) +{ + return networkstatus_v2_find_mutable_entry(ns, digest); +} + +/** As networkstatus_find_entry, but do not return a const pointer */ routerstatus_t * -networkstatus_vote_find_entry(networkstatus_t *ns, const char *digest) +networkstatus_vote_find_mutable_entry(networkstatus_t *ns, const char *digest) { return smartlist_bsearch(ns->routerstatus_list, digest, compare_digest_to_routerstatus_entry); } -/*XXXX make this static once functions are moved into this file. */ +/** Return the entry in <b>ns</b> for the identity digest <b>digest</b>, or + * NULL if none was found. */ +const routerstatus_t * +networkstatus_vote_find_entry(networkstatus_t *ns, const char *digest) +{ + return networkstatus_vote_find_mutable_entry(ns, digest); +} + +/*XXXX MOVE make this static once functions are moved into this file. */ /** Search the routerstatuses in <b>ns</b> for one whose identity digest is * <b>digest</b>. Return value and set *<b>found_out</b> as for * smartlist_bsearch_idx(). */ @@ -974,26 +1004,41 @@ const smartlist_t * networkstatus_get_v2_list(void) { if (!networkstatus_v2_list) - networkstatus_v2_list = smartlist_create(); + networkstatus_v2_list = smartlist_new(); return networkstatus_v2_list; } -/** Return the consensus view of the status of the router whose current - * <i>descriptor</i> digest is <b>digest</b>, or NULL if no such router is - * known. */ +/** As router_get_consensus_status_by_descriptor_digest, but does not return + * a const pointer. */ routerstatus_t * -router_get_consensus_status_by_descriptor_digest(const char *digest) +router_get_mutable_consensus_status_by_descriptor_digest( + networkstatus_t *consensus, + const char *digest) { - if (!current_consensus) return NULL; - if (!current_consensus->desc_digest_map) { - digestmap_t * m = current_consensus->desc_digest_map = digestmap_new(); - SMARTLIST_FOREACH(current_consensus->routerstatus_list, + if (!consensus) + consensus = current_consensus; + if (!consensus) + return NULL; + if (!consensus->desc_digest_map) { + digestmap_t *m = consensus->desc_digest_map = digestmap_new(); + SMARTLIST_FOREACH(consensus->routerstatus_list, routerstatus_t *, rs, { digestmap_set(m, rs->descriptor_digest, rs); }); } - return digestmap_get(current_consensus->desc_digest_map, digest); + return digestmap_get(consensus->desc_digest_map, digest); +} + +/** Return the consensus view of the status of the router whose current + * <i>descriptor</i> digest in <b>consensus</b> is <b>digest</b>, or NULL if + * no such router is known. */ +const routerstatus_t * +router_get_consensus_status_by_descriptor_digest(networkstatus_t *consensus, + const char *digest) +{ + return router_get_mutable_consensus_status_by_descriptor_digest( + consensus, digest); } /** Given the digest of a router descriptor, return its current download @@ -1002,7 +1047,10 @@ download_status_t * router_get_dl_status_by_descriptor_digest(const char *d) { routerstatus_t *rs; - if ((rs = router_get_consensus_status_by_descriptor_digest(d))) + if (!current_ns_consensus) + return NULL; + if ((rs = router_get_mutable_consensus_status_by_descriptor_digest( + current_ns_consensus, d))) return &rs->dl_status; if (v2_download_status_map) return digestmap_get(v2_download_status_map, d); @@ -1010,10 +1058,9 @@ router_get_dl_status_by_descriptor_digest(const char *d) return NULL; } -/** Return the consensus view of the status of the router whose identity - * digest is <b>digest</b>, or NULL if we don't know about any such router. */ +/** As router_get_consensus_status_by_id, but do not return a const pointer */ routerstatus_t * -router_get_consensus_status_by_id(const char *digest) +router_get_mutable_consensus_status_by_id(const char *digest) { if (!current_consensus) return NULL; @@ -1021,100 +1068,27 @@ router_get_consensus_status_by_id(const char *digest) compare_digest_to_routerstatus_entry); } +/** Return the consensus view of the status of the router whose identity + * digest is <b>digest</b>, or NULL if we don't know about any such router. */ +const routerstatus_t * +router_get_consensus_status_by_id(const char *digest) +{ + return router_get_mutable_consensus_status_by_id(digest); +} + /** Given a nickname (possibly verbose, possibly a hexadecimal digest), return * the corresponding routerstatus_t, or NULL if none exists. Warn the * user if <b>warn_if_unnamed</b> is set, and they have specified a router by * nickname, but the Named flag isn't set for that router. */ -routerstatus_t * +const routerstatus_t * router_get_consensus_status_by_nickname(const char *nickname, int warn_if_unnamed) { - char digest[DIGEST_LEN]; - routerstatus_t *best=NULL; - smartlist_t *matches=NULL; - const char *named_id=NULL; - - if (!current_consensus || !nickname) - return NULL; - - /* Is this name really a hexadecimal identity digest? */ - if (nickname[0] == '$') { - if (base16_decode(digest, DIGEST_LEN, nickname+1, strlen(nickname+1))<0) - return NULL; - return networkstatus_vote_find_entry(current_consensus, digest); - } else if (strlen(nickname) == HEX_DIGEST_LEN && - (base16_decode(digest, DIGEST_LEN, nickname, strlen(nickname))==0)) { - return networkstatus_vote_find_entry(current_consensus, digest); - } - - /* Is there a server that is Named with this name? */ - if (named_server_map) - named_id = strmap_get_lc(named_server_map, nickname); - if (named_id) - return networkstatus_vote_find_entry(current_consensus, named_id); - - /* Okay; is this name listed as Unnamed? */ - if (unnamed_server_map && - strmap_get_lc(unnamed_server_map, nickname)) { - log_info(LD_GENERAL, "The name %s is listed as Unnamed; it is not the " - "canonical name of any server we know.", escaped(nickname)); + const node_t *node = node_get_by_nickname(nickname, warn_if_unnamed); + if (node) + return node->rs; + else return NULL; - } - - /* This name is not canonical for any server; go through the list and - * see who it matches. */ - /*XXXX This is inefficient; optimize it if it matters. */ - matches = smartlist_create(); - SMARTLIST_FOREACH(current_consensus->routerstatus_list, - routerstatus_t *, lrs, - { - if (!strcasecmp(lrs->nickname, nickname)) { - if (lrs->is_named) { - tor_fragile_assert() /* This should never happen. */ - smartlist_free(matches); - return lrs; - } else { - if (lrs->is_unnamed) { - tor_fragile_assert(); /* nor should this. */ - smartlist_clear(matches); - best=NULL; - break; - } - smartlist_add(matches, lrs); - best = lrs; - } - } - }); - - if (smartlist_len(matches)>1 && warn_if_unnamed) { - int any_unwarned=0; - SMARTLIST_FOREACH(matches, routerstatus_t *, lrs, - { - if (! lrs->name_lookup_warned) { - lrs->name_lookup_warned=1; - any_unwarned=1; - } - }); - if (any_unwarned) { - log_warn(LD_CONFIG,"There are multiple matches for the nickname \"%s\"," - " but none is listed as named by the directory authorities. " - "Choosing one arbitrarily.", nickname); - } - } else if (warn_if_unnamed && best && !best->name_lookup_warned) { - char fp[HEX_DIGEST_LEN+1]; - base16_encode(fp, sizeof(fp), - best->identity_digest, DIGEST_LEN); - log_warn(LD_CONFIG, - "When looking up a status, you specified a server \"%s\" by name, " - "but the directory authorities do not have any key registered for " - "this nickname -- so it could be used by any server, " - "not just the one you meant. " - "To make sure you get the same server in the future, refer to " - "it by key, as \"$%s\".", nickname, fp); - best->name_lookup_warned = 1; - } - smartlist_free(matches); - return best; } /** Return the identity digest that's mapped to officially by @@ -1170,7 +1144,7 @@ update_v2_networkstatus_cache_downloads(time_t now) { char resource[HEX_DIGEST_LEN+6]; /* fp/hexdigit.z\0 */ tor_addr_t addr; - if (!(ds->type & V2_AUTHORITY)) + if (!(ds->type & V2_DIRINFO)) continue; if (router_digest_is_me(ds->digest)) continue; @@ -1211,6 +1185,30 @@ update_v2_networkstatus_cache_downloads(time_t now) } } +/** Return true iff, given the options listed in <b>options</b>, <b>flavor</b> + * is the flavor of a consensus networkstatus that we would like to fetch. */ +static int +we_want_to_fetch_flavor(const or_options_t *options, int flavor) +{ + if (flavor < 0 || flavor > N_CONSENSUS_FLAVORS) { + /* This flavor is crazy; we don't want it */ + /*XXXX handle unrecognized flavors later */ + return 0; + } + if (authdir_mode_v3(options) || directory_caches_dir_info(options)) { + /* We want to serve all flavors to others, regardless if we would use + * it ourselves. */ + return 1; + } + if (options->FetchUselessDescriptors) { + /* In order to get all descriptors, we need to fetch all consensuses. */ + return 1; + } + /* Otherwise, we want the flavor only if we want to use it to build + * circuits. */ + return flavor == usable_consensus_flavor(); +} + /** How many times will we try to fetch a consensus before we give up? */ #define CONSENSUS_NETWORKSTATUS_MAX_DL_TRIES 8 /** How long will we hang onto a possibly live consensus for which we're @@ -1223,48 +1221,70 @@ static void update_consensus_networkstatus_downloads(time_t now) { int i; - if (!networkstatus_get_live_consensus(now)) - time_to_download_next_consensus = now; /* No live consensus? Get one now!*/ - if (time_to_download_next_consensus > now) - return; /* Wait until the current consensus is older. */ - /* XXXXNM Microdescs: may need to download more types. */ - if (!download_status_is_ready(&consensus_dl_status[FLAV_NS], now, - CONSENSUS_NETWORKSTATUS_MAX_DL_TRIES)) - return; /* We failed downloading a consensus too recently. */ - if (connection_get_by_type_purpose(CONN_TYPE_DIR, - DIR_PURPOSE_FETCH_CONSENSUS)) - return; /* There's an in-progress download.*/ + const or_options_t *options = get_options(); for (i=0; i < N_CONSENSUS_FLAVORS; ++i) { - consensus_waiting_for_certs_t *waiting = &consensus_waiting_for_certs[i]; + /* XXXX need some way to download unknown flavors if we are caching. */ + const char *resource; + consensus_waiting_for_certs_t *waiting; + networkstatus_t *c; + + if (! we_want_to_fetch_flavor(options, i)) + continue; + + c = networkstatus_get_latest_consensus_by_flavor(i); + if (! (c && c->valid_after <= now && now <= c->valid_until)) { + /* No live consensus? Get one now!*/ + time_to_download_next_consensus[i] = now; + } + + if (time_to_download_next_consensus[i] > now) + return; /* Wait until the current consensus is older. */ + + resource = networkstatus_get_flavor_name(i); + + if (!download_status_is_ready(&consensus_dl_status[i], now, + CONSENSUS_NETWORKSTATUS_MAX_DL_TRIES)) + continue; /* We failed downloading a consensus too recently. */ + if (connection_dir_get_by_purpose_and_resource( + DIR_PURPOSE_FETCH_CONSENSUS, resource)) + continue; /* There's an in-progress download.*/ + + waiting = &consensus_waiting_for_certs[i]; if (waiting->consensus) { /* XXXX make sure this doesn't delay sane downloads. */ - if (waiting->set_at + DELAY_WHILE_FETCHING_CERTS > now) - return; /* We're still getting certs for this one. */ - else { + if (waiting->set_at + DELAY_WHILE_FETCHING_CERTS > now) { + continue; /* We're still getting certs for this one. */ + } else { if (!waiting->dl_failed) { - download_status_failed(&consensus_dl_status[FLAV_NS], 0); + download_status_failed(&consensus_dl_status[i], 0); waiting->dl_failed=1; } } } - } - log_info(LD_DIR, "Launching networkstatus consensus download."); - directory_get_from_dirserver(DIR_PURPOSE_FETCH_CONSENSUS, - ROUTER_PURPOSE_GENERAL, NULL, - PDS_RETRY_IF_NO_SERVERS); + log_info(LD_DIR, "Launching %s networkstatus consensus download.", + networkstatus_get_flavor_name(i)); + + directory_get_from_dirserver(DIR_PURPOSE_FETCH_CONSENSUS, + ROUTER_PURPOSE_GENERAL, resource, + PDS_RETRY_IF_NO_SERVERS); + } } /** Called when an attempt to download a consensus fails: note that the * failure occurred, and possibly retry. */ void -networkstatus_consensus_download_failed(int status_code) +networkstatus_consensus_download_failed(int status_code, const char *flavname) { - /* XXXXNM Microdescs: may need to handle more types. */ - download_status_failed(&consensus_dl_status[FLAV_NS], status_code); - /* Retry immediately, if appropriate. */ - update_consensus_networkstatus_downloads(time(NULL)); + int flav = networkstatus_parse_flavor_name(flavname); + if (flav >= 0) { + tor_assert(flav < N_CONSENSUS_FLAVORS); + /* XXXX handle unrecognized flavors */ + download_status_failed(&consensus_dl_status[flav], status_code); + /* Retry immediately, if appropriate. */ + update_consensus_networkstatus_downloads(time(NULL)); + } } /** How long do we (as a cache) wait after a consensus becomes non-fresh @@ -1272,13 +1292,17 @@ networkstatus_consensus_download_failed(int status_code) #define CONSENSUS_MIN_SECONDS_BEFORE_CACHING 120 /** Update the time at which we'll consider replacing the current - * consensus. */ -void -update_consensus_networkstatus_fetch_time(time_t now) + * consensus of flavor <b>flav</b> */ +static void +update_consensus_networkstatus_fetch_time_impl(time_t now, int flav) { - or_options_t *options = get_options(); - networkstatus_t *c = networkstatus_get_live_consensus(now); - if (c) { + const or_options_t *options = get_options(); + networkstatus_t *c = networkstatus_get_latest_consensus_by_flavor(flav); + const char *flavor = networkstatus_get_flavor_name(flav); + if (! we_want_to_fetch_flavor(get_options(), flav)) + return; + + if (c && c->valid_after <= now && now <= c->valid_until) { long dl_interval; long interval = c->fresh_until - c->valid_after; long min_sec_before_caching = CONSENSUS_MIN_SECONDS_BEFORE_CACHING; @@ -1327,22 +1351,36 @@ update_consensus_networkstatus_fetch_time(time_t now) tor_assert(c->fresh_until < start); /* We must download the next one before c is invalid: */ tor_assert(start+dl_interval < c->valid_until); - time_to_download_next_consensus = start +crypto_rand_int((int)dl_interval); + time_to_download_next_consensus[flav] = + start + crypto_rand_int((int)dl_interval); { char tbuf1[ISO_TIME_LEN+1]; char tbuf2[ISO_TIME_LEN+1]; char tbuf3[ISO_TIME_LEN+1]; format_local_iso_time(tbuf1, c->fresh_until); format_local_iso_time(tbuf2, c->valid_until); - format_local_iso_time(tbuf3, time_to_download_next_consensus); - log_info(LD_DIR, "Live consensus %s the most recent until %s and will " - "expire at %s; fetching the next one at %s.", - (c->fresh_until > now) ? "will be" : "was", + format_local_iso_time(tbuf3, time_to_download_next_consensus[flav]); + log_info(LD_DIR, "Live %s consensus %s the most recent until %s and " + "will expire at %s; fetching the next one at %s.", + flavor, (c->fresh_until > now) ? "will be" : "was", tbuf1, tbuf2, tbuf3); } } else { - time_to_download_next_consensus = now; - log_info(LD_DIR, "No live consensus; we should fetch one immediately."); + time_to_download_next_consensus[flav] = now; + log_info(LD_DIR, "No live %s consensus; we should fetch one immediately.", + flavor); + } +} + +/** Update the time at which we'll consider replacing the current + * consensus of flavor 'flavor' */ +void +update_consensus_networkstatus_fetch_time(time_t now) +{ + int i; + for (i = 0; i < N_CONSENSUS_FLAVORS; ++i) { + if (we_want_to_fetch_flavor(get_options(), i)) + update_consensus_networkstatus_fetch_time_impl(now, i); } } @@ -1350,7 +1388,7 @@ update_consensus_networkstatus_fetch_time(time_t now) * fetches yet (e.g. we demand bridges and none are yet known). * Else return 0. */ int -should_delay_dir_fetches(or_options_t *options) +should_delay_dir_fetches(const or_options_t *options) { if (options->UseBridges && !any_bridge_descriptors_known()) { log_info(LD_DIR, "delaying dir fetches (no running bridges known)"); @@ -1364,7 +1402,7 @@ should_delay_dir_fetches(or_options_t *options) void update_networkstatus_downloads(time_t now) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (should_delay_dir_fetches(options)) return; if (authdir_mode_any_main(options) || options->FetchV2Networkstatus) @@ -1385,7 +1423,10 @@ update_certificate_downloads(time_t now) now); } - authority_certs_fetch_missing(current_consensus, now); + if (current_ns_consensus) + authority_certs_fetch_missing(current_ns_consensus, now); + if (current_md_consensus) + authority_certs_fetch_missing(current_md_consensus, now); } /** Return 1 if we have a consensus but we don't have enough certificates @@ -1393,7 +1434,7 @@ update_certificate_downloads(time_t now) int consensus_is_waiting_for_certs(void) { - return consensus_waiting_for_certs[USABLE_CONSENSUS_FLAVOR].consensus + return consensus_waiting_for_certs[usable_consensus_flavor()].consensus ? 1 : 0; } @@ -1417,6 +1458,21 @@ networkstatus_get_latest_consensus(void) return current_consensus; } +/** Return the latest consensus we have whose flavor matches <b>f</b>, or NULL + * if we don't have one. */ +networkstatus_t * +networkstatus_get_latest_consensus_by_flavor(consensus_flavor_t f) +{ + if (f == FLAV_NS) + return current_ns_consensus; + else if (f == FLAV_MICRODESC) + return current_md_consensus; + else { + tor_assert(0); + return NULL; + } +} + /** Return the most recent consensus that we have downloaded, or NULL if it is * no longer live. */ networkstatus_t * @@ -1436,13 +1492,15 @@ networkstatus_get_live_consensus(time_t now) /** As networkstatus_get_live_consensus(), but is way more tolerant of expired * consensuses. */ networkstatus_t * -networkstatus_get_reasonably_live_consensus(time_t now) +networkstatus_get_reasonably_live_consensus(time_t now, int flavor) { #define REASONABLY_LIVE_TIME (24*60*60) - if (current_consensus && - current_consensus->valid_after <= now && - now <= current_consensus->valid_until+REASONABLY_LIVE_TIME) - return current_consensus; + networkstatus_t *consensus = + networkstatus_get_latest_consensus_by_flavor(flavor); + if (consensus && + consensus->valid_after <= now && + now <= consensus->valid_until+REASONABLY_LIVE_TIME) + return consensus; else return NULL; } @@ -1463,7 +1521,7 @@ routerstatus_has_changed(const routerstatus_t *a, const routerstatus_t *b) a->is_exit != b->is_exit || a->is_stable != b->is_stable || a->is_fast != b->is_fast || - a->is_running != b->is_running || + a->is_flagged_running != b->is_flagged_running || a->is_named != b->is_named || a->is_unnamed != b->is_unnamed || a->is_valid != b->is_valid || @@ -1502,15 +1560,16 @@ notify_control_networkstatus_changed(const networkstatus_t *old_c, control_event_networkstatus_changed(new_c->routerstatus_list); return; } - changed = smartlist_create(); + changed = smartlist_new(); - SMARTLIST_FOREACH_JOIN(old_c->routerstatus_list, routerstatus_t *, rs_old, - new_c->routerstatus_list, routerstatus_t *, rs_new, - tor_memcmp(rs_old->identity_digest, - rs_new->identity_digest, DIGEST_LEN), - smartlist_add(changed, rs_new)) { + SMARTLIST_FOREACH_JOIN( + old_c->routerstatus_list, const routerstatus_t *, rs_old, + new_c->routerstatus_list, const routerstatus_t *, rs_new, + tor_memcmp(rs_old->identity_digest, + rs_new->identity_digest, DIGEST_LEN), + smartlist_add(changed, (void*) rs_new)) { if (routerstatus_has_changed(rs_old, rs_new)) - smartlist_add(changed, rs_new); + smartlist_add(changed, (void*)rs_new); } SMARTLIST_FOREACH_JOIN_END(rs_old, rs_new); control_event_networkstatus_changed(changed); @@ -1534,7 +1593,6 @@ networkstatus_copy_old_consensus_info(networkstatus_t *new_c, rs_new->identity_digest, DIGEST_LEN), STMT_NIL) { /* Okay, so we're looking at the same identity. */ - rs_new->name_lookup_warned = rs_old->name_lookup_warned; rs_new->last_dir_503_at = rs_old->last_dir_503_at; if (tor_memeq(rs_old->descriptor_digest, rs_new->descriptor_digest, @@ -1570,7 +1628,7 @@ networkstatus_set_current_consensus(const char *consensus, networkstatus_t *c=NULL; int r, result = -1; time_t now = time(NULL); - or_options_t *options = get_options(); + const or_options_t *options = get_options(); char *unverified_fname = NULL, *consensus_fname = NULL; int flav = networkstatus_parse_flavor_name(flavor); const unsigned from_cache = flags & NSSET_FROM_CACHE; @@ -1608,7 +1666,7 @@ networkstatus_set_current_consensus(const char *consensus, flavor = networkstatus_get_flavor_name(flav); } - if (flav != USABLE_CONSENSUS_FLAVOR && + if (flav != usable_consensus_flavor() && !directory_caches_dir_info(options)) { /* This consensus is totally boring to us: we won't use it, and we won't * serve it. Drop it. */ @@ -1627,9 +1685,16 @@ networkstatus_set_current_consensus(const char *consensus, if (!strcmp(flavor, "ns")) { consensus_fname = get_datadir_fname("cached-consensus"); unverified_fname = get_datadir_fname("unverified-consensus"); - if (current_consensus) { - current_digests = ¤t_consensus->digests; - current_valid_after = current_consensus->valid_after; + if (current_ns_consensus) { + current_digests = ¤t_ns_consensus->digests; + current_valid_after = current_ns_consensus->valid_after; + } + } else if (!strcmp(flavor, "microdesc")) { + consensus_fname = get_datadir_fname("cached-microdesc-consensus"); + unverified_fname = get_datadir_fname("unverified-microdesc-consensus"); + if (current_md_consensus) { + current_digests = ¤t_md_consensus->digests; + current_valid_after = current_md_consensus->valid_after; } } else { cached_dir_t *cur; @@ -1706,24 +1771,36 @@ networkstatus_set_current_consensus(const char *consensus, } } - if (!from_cache && flav == USABLE_CONSENSUS_FLAVOR) + if (!from_cache && flav == usable_consensus_flavor()) control_event_client_status(LOG_NOTICE, "CONSENSUS_ARRIVED"); /* Are we missing any certificates at all? */ if (r != 1 && dl_certs) authority_certs_fetch_missing(c, now); - if (flav == USABLE_CONSENSUS_FLAVOR) { + if (flav == usable_consensus_flavor()) { notify_control_networkstatus_changed(current_consensus, c); - - if (current_consensus) { - networkstatus_copy_old_consensus_info(c, current_consensus); - networkstatus_vote_free(current_consensus); + } + if (flav == FLAV_NS) { + if (current_ns_consensus) { + networkstatus_copy_old_consensus_info(c, current_ns_consensus); + networkstatus_vote_free(current_ns_consensus); /* Defensive programming : we should set current_consensus very soon, * but we're about to call some stuff in the meantime, and leaving this * dangling pointer around has proven to be trouble. */ - current_consensus = NULL; + current_ns_consensus = NULL; } + current_ns_consensus = c; + free_consensus = 0; /* avoid free */ + } else if (flav == FLAV_MICRODESC) { + if (current_md_consensus) { + networkstatus_copy_old_consensus_info(c, current_md_consensus); + networkstatus_vote_free(current_md_consensus); + /* more defensive programming */ + current_md_consensus = NULL; + } + current_md_consensus = c; + free_consensus = 0; /* avoid free */ } waiting = &consensus_waiting_for_certs[flav]; @@ -1748,17 +1825,17 @@ networkstatus_set_current_consensus(const char *consensus, download_status_failed(&consensus_dl_status[flav], 0); } - if (flav == USABLE_CONSENSUS_FLAVOR) { - current_consensus = c; - free_consensus = 0; /* Prevent free. */ - - /* XXXXNM Microdescs: needs a non-ns variant. */ + if (flav == usable_consensus_flavor()) { + /* XXXXNM Microdescs: needs a non-ns variant. ???? NM*/ update_consensus_networkstatus_fetch_time(now); + + nodelist_set_consensus(current_consensus); + dirvote_recalculate_timing(options, now); routerstatus_list_update_named_server_map(); cell_ewma_set_scale_factor(options, current_consensus); - /* XXXX023 this call might be unnecessary here: can changing the + /* XXXX024 this call might be unnecessary here: can changing the * current consensus really alter our view of any OR's rate limits? */ connection_or_update_token_buckets(get_connection_array(), options); @@ -1780,11 +1857,11 @@ networkstatus_set_current_consensus(const char *consensus, * valid-after time, declare that our clock is skewed. */ #define EARLY_CONSENSUS_NOTICE_SKEW 60 - if (now < current_consensus->valid_after - EARLY_CONSENSUS_NOTICE_SKEW) { + if (now < c->valid_after - EARLY_CONSENSUS_NOTICE_SKEW) { char tbuf[ISO_TIME_LEN+1]; char dbuf[64]; - long delta = now - current_consensus->valid_after; - format_iso_time(tbuf, current_consensus->valid_after); + long delta = now - c->valid_after; + format_iso_time(tbuf, c->valid_after); format_time_interval(dbuf, sizeof(dbuf), delta); log_warn(LD_GENERAL, "Our clock is %s behind the time published in the " "consensus network status document (%s GMT). Tor needs an " @@ -1835,7 +1912,8 @@ void routers_update_all_from_networkstatus(time_t now, int dir_version) { routerlist_t *rl = router_get_routerlist(); - networkstatus_t *consensus = networkstatus_get_live_consensus(now); + networkstatus_t *consensus = networkstatus_get_reasonably_live_consensus(now, + FLAV_NS); if (networkstatus_v2_list_has_changed) download_status_map_update_from_v2_networkstatus(); @@ -1907,7 +1985,7 @@ download_status_map_update_from_v2_networkstatus(void) dl_status = digestmap_new(); SMARTLIST_FOREACH_BEGIN(networkstatus_v2_list, networkstatus_v2_t *, ns) { - SMARTLIST_FOREACH_BEGIN(ns->entries, routerstatus_t *, rs) { + SMARTLIST_FOREACH_BEGIN(ns->entries, const routerstatus_t *, rs) { const char *d = rs->descriptor_digest; download_status_t *s; if (digestmap_get(dl_status, d)) @@ -1935,8 +2013,8 @@ routerstatus_list_update_named_server_map(void) named_server_map = strmap_new(); strmap_free(unnamed_server_map, NULL); unnamed_server_map = strmap_new(); - SMARTLIST_FOREACH(current_consensus->routerstatus_list, routerstatus_t *, rs, - { + SMARTLIST_FOREACH_BEGIN(current_consensus->routerstatus_list, + const routerstatus_t *, rs) { if (rs->is_named) { strmap_set_lc(named_server_map, rs->nickname, tor_memdup(rs->identity_digest, DIGEST_LEN)); @@ -1944,7 +2022,7 @@ routerstatus_list_update_named_server_map(void) if (rs->is_unnamed) { strmap_set_lc(unnamed_server_map, rs->nickname, (void*)1); } - }); + } SMARTLIST_FOREACH_END(rs); } /** Given a list <b>routers</b> of routerinfo_t *, update each status field @@ -1955,14 +2033,13 @@ routers_update_status_from_consensus_networkstatus(smartlist_t *routers, int reset_failures) { trusted_dir_server_t *ds; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int authdir = authdir_mode_v2(options) || authdir_mode_v3(options); - int namingdir = authdir && options->NamingAuthoritativeDir; networkstatus_t *ns = current_consensus; if (!ns || !smartlist_len(ns->routerstatus_list)) return; if (!networkstatus_v2_list) - networkstatus_v2_list = smartlist_create(); + networkstatus_v2_list = smartlist_new(); routers_sort_by_identity(routers); @@ -1971,25 +2048,12 @@ routers_update_status_from_consensus_networkstatus(smartlist_t *routers, tor_memcmp(rs->identity_digest, router->cache_info.identity_digest, DIGEST_LEN), { - /* We have no routerstatus for this router. Clear flags and skip it. */ - if (!namingdir) - router->is_named = 0; - if (!authdir) { - if (router->purpose == ROUTER_PURPOSE_GENERAL) - router_clear_status_flags(router); - } }) { /* We have a routerstatus for this router. */ const char *digest = router->cache_info.identity_digest; ds = router_get_trusteddirserver_by_digest(digest); - if (!namingdir) { - if (rs->is_named && !strcasecmp(router->nickname, rs->nickname)) - router->is_named = 1; - else - router->is_named = 0; - } /* Is it the same descriptor, or only the same identity? */ if (tor_memeq(router->cache_info.signed_descriptor_digest, rs->descriptor_digest, DIGEST_LEN)) { @@ -1997,28 +2061,17 @@ routers_update_status_from_consensus_networkstatus(smartlist_t *routers, router->cache_info.last_listed_as_valid_until = ns->valid_until; } - if (!authdir) { - /* If we're not an authdir, believe others. */ - router->is_valid = rs->is_valid; - router->is_running = rs->is_running; - router->is_fast = rs->is_fast; - router->is_stable = rs->is_stable; - router->is_possible_guard = rs->is_possible_guard; - router->is_exit = rs->is_exit; - router->is_bad_directory = rs->is_bad_directory; - router->is_bad_exit = rs->is_bad_exit; - router->is_hs_dir = rs->is_hs_dir; - } else { + if (authdir) { /* If we _are_ an authority, we should check whether this router * is one that will cause us to need a reachability test. */ routerinfo_t *old_router = - router_get_by_digest(router->cache_info.identity_digest); + router_get_mutable_by_digest(router->cache_info.identity_digest); if (old_router != router) { router->needs_retest_if_added = dirserv_should_launch_reachability_test(router, old_router); } } - if (router->is_running && ds) { + if (rs->is_flagged_running && ds) { download_status_reset(&ds->v2_ns_dl_status); } if (reset_failures) { @@ -2027,10 +2080,9 @@ routers_update_status_from_consensus_networkstatus(smartlist_t *routers, } SMARTLIST_FOREACH_JOIN_END(rs, router); /* Now update last_listed_as_valid_until from v2 networkstatuses. */ - /* XXXX If this is slow, we need to rethink the code. */ - SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, { + SMARTLIST_FOREACH_BEGIN(networkstatus_v2_list, networkstatus_v2_t *, ns) { time_t live_until = ns->published_on + V2_NETWORKSTATUS_ROUTER_LIFETIME; - SMARTLIST_FOREACH_JOIN(ns->entries, routerstatus_t *, rs, + SMARTLIST_FOREACH_JOIN(ns->entries, const routerstatus_t *, rs, routers, routerinfo_t *, ri, tor_memcmp(rs->identity_digest, ri->cache_info.identity_digest, DIGEST_LEN), @@ -2041,7 +2093,7 @@ routers_update_status_from_consensus_networkstatus(smartlist_t *routers, ri->cache_info.last_listed_as_valid_until = live_until; } } SMARTLIST_FOREACH_JOIN_END(rs, ri); - }); + } SMARTLIST_FOREACH_END(ns); router_dir_info_changed(); } @@ -2051,7 +2103,7 @@ routers_update_status_from_consensus_networkstatus(smartlist_t *routers, void signed_descs_update_status_from_consensus_networkstatus(smartlist_t *descs) { - networkstatus_t *ns = current_consensus; + networkstatus_t *ns = current_ns_consensus; if (!ns) return; @@ -2059,11 +2111,11 @@ signed_descs_update_status_from_consensus_networkstatus(smartlist_t *descs) char dummy[DIGEST_LEN]; /* instantiates the digest map. */ memset(dummy, 0, sizeof(dummy)); - router_get_consensus_status_by_descriptor_digest(dummy); + router_get_consensus_status_by_descriptor_digest(ns, dummy); } SMARTLIST_FOREACH(descs, signed_descriptor_t *, d, { - routerstatus_t *rs = digestmap_get(ns->desc_digest_map, + const routerstatus_t *rs = digestmap_get(ns->desc_digest_map, d->signed_descriptor_digest); if (rs) { if (ns->valid_until > d->last_listed_as_valid_until) @@ -2076,7 +2128,7 @@ signed_descs_update_status_from_consensus_networkstatus(smartlist_t *descs) * return the result in a newly allocated string. Used only by controller * interface (for now.) */ char * -networkstatus_getinfo_helper_single(routerstatus_t *rs) +networkstatus_getinfo_helper_single(const routerstatus_t *rs) { char buf[RS_ENTRY_LEN+1]; routerstatus_format_entry(buf, sizeof(buf), rs, NULL, NS_CONTROL_PORT); @@ -2107,8 +2159,11 @@ networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now) return NULL; } - statuses = smartlist_create(); - SMARTLIST_FOREACH(rl->routers, routerinfo_t *, ri, { + statuses = smartlist_new(); + SMARTLIST_FOREACH_BEGIN(rl->routers, routerinfo_t *, ri) { + node_t *node = node_get_mutable_by_id(ri->cache_info.identity_digest); + if (!node) + continue; if (ri->cache_info.published_on < cutoff) continue; if (ri->purpose != purpose) @@ -2116,9 +2171,9 @@ networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now) if (bridge_auth && ri->purpose == ROUTER_PURPOSE_BRIDGE) dirserv_set_router_is_running(ri, now); /* then generate and write out status lines for each of them */ - set_routerstatus_from_routerinfo(&rs, ri, now, 0, 0, 0, 0); + set_routerstatus_from_routerinfo(&rs, node, ri, now, 0, 0, 0, 0); smartlist_add(statuses, networkstatus_getinfo_helper_single(&rs)); - }); + } SMARTLIST_FOREACH_END(ri); answer = smartlist_join_strings(statuses, "", 0, NULL); SMARTLIST_FOREACH(statuses, char *, cp, tor_free(cp)); @@ -2131,16 +2186,16 @@ void networkstatus_dump_bridge_status_to_file(time_t now) { char *status = networkstatus_getinfo_by_purpose("bridge", now); - or_options_t *options = get_options(); - size_t len = strlen(options->DataDirectory) + 32; - char *fname = tor_malloc(len); - tor_snprintf(fname, len, "%s"PATH_SEPARATOR"networkstatus-bridges", + const or_options_t *options = get_options(); + char *fname = NULL; + tor_asprintf(&fname, "%s"PATH_SEPARATOR"networkstatus-bridges", options->DataDirectory); write_str_to_file(fname,status,0); tor_free(fname); tor_free(status); } +/* DOCDOC get_net_param_from_list */ static int32_t get_net_param_from_list(smartlist_t *net_params, const char *param_name, int32_t default_val, int32_t min_val, int32_t max_val) @@ -2185,7 +2240,7 @@ get_net_param_from_list(smartlist_t *net_params, const char *param_name, * <b>min_val</b> and at most <b>max_val</b> and raise/cap the parsed value * if necessary. */ int32_t -networkstatus_get_param(networkstatus_t *ns, const char *param_name, +networkstatus_get_param(const networkstatus_t *ns, const char *param_name, int32_t default_val, int32_t min_val, int32_t max_val) { if (!ns) /* if they pass in null, go find it ourselves */ @@ -2264,7 +2319,7 @@ getinfo_helper_networkstatus(control_connection_t *conn, const char *question, char **answer, const char **errmsg) { - routerstatus_t *status; + const routerstatus_t *status; (void) conn; if (!current_consensus) { @@ -2273,9 +2328,9 @@ getinfo_helper_networkstatus(control_connection_t *conn, } if (!strcmp(question, "ns/all")) { - smartlist_t *statuses = smartlist_create(); + smartlist_t *statuses = smartlist_new(); SMARTLIST_FOREACH(current_consensus->routerstatus_list, - routerstatus_t *, rs, + const routerstatus_t *, rs, { smartlist_add(statuses, networkstatus_getinfo_helper_single(rs)); }); @@ -2319,8 +2374,9 @@ networkstatus_free_all(void) digestmap_free(v2_download_status_map, _tor_free); v2_download_status_map = NULL; - networkstatus_vote_free(current_consensus); - current_consensus = NULL; + networkstatus_vote_free(current_ns_consensus); + networkstatus_vote_free(current_md_consensus); + current_md_consensus = current_ns_consensus = NULL; for (i=0; i < N_CONSENSUS_FLAVORS; ++i) { consensus_waiting_for_certs_t *waiting = &consensus_waiting_for_certs[i]; diff --git a/src/or/networkstatus.h b/src/or/networkstatus.h index ec2e8f884d..0af17512dd 100644 --- a/src/or/networkstatus.h +++ b/src/or/networkstatus.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -38,31 +38,46 @@ int router_set_networkstatus_v2(const char *s, time_t arrived_at, void networkstatus_v2_list_clean(time_t now); int compare_digest_to_routerstatus_entry(const void *_key, const void **_member); -routerstatus_t *networkstatus_v2_find_entry(networkstatus_v2_t *ns, +const routerstatus_t *networkstatus_v2_find_entry(networkstatus_v2_t *ns, const char *digest); -routerstatus_t *networkstatus_vote_find_entry(networkstatus_t *ns, +const routerstatus_t *networkstatus_vote_find_entry(networkstatus_t *ns, + const char *digest); +routerstatus_t *networkstatus_v2_find_mutable_entry(networkstatus_v2_t *ns, + const char *digest); +routerstatus_t *networkstatus_vote_find_mutable_entry(networkstatus_t *ns, const char *digest); int networkstatus_vote_find_entry_idx(networkstatus_t *ns, const char *digest, int *found_out); const smartlist_t *networkstatus_get_v2_list(void); download_status_t *router_get_dl_status_by_descriptor_digest(const char *d); -routerstatus_t *router_get_consensus_status_by_id(const char *digest); -routerstatus_t *router_get_consensus_status_by_descriptor_digest( - const char *digest); -routerstatus_t *router_get_consensus_status_by_nickname(const char *nickname, - int warn_if_unnamed); +const routerstatus_t *router_get_consensus_status_by_id(const char *digest); +routerstatus_t *router_get_mutable_consensus_status_by_id( + const char *digest); +const routerstatus_t *router_get_consensus_status_by_descriptor_digest( + networkstatus_t *consensus, + const char *digest); +routerstatus_t *router_get_mutable_consensus_status_by_descriptor_digest( + networkstatus_t *consensus, + const char *digest); +const routerstatus_t *router_get_consensus_status_by_nickname( + const char *nickname, + int warn_if_unnamed); const char *networkstatus_get_router_digest_by_nickname(const char *nickname); int networkstatus_nickname_is_unnamed(const char *nickname); -void networkstatus_consensus_download_failed(int status_code); +void networkstatus_consensus_download_failed(int status_code, + const char *flavname); void update_consensus_networkstatus_fetch_time(time_t now); -int should_delay_dir_fetches(or_options_t *options); +int should_delay_dir_fetches(const or_options_t *options); void update_networkstatus_downloads(time_t now); void update_certificate_downloads(time_t now); int consensus_is_waiting_for_certs(void); networkstatus_v2_t *networkstatus_v2_get_by_digest(const char *digest); networkstatus_t *networkstatus_get_latest_consensus(void); +networkstatus_t *networkstatus_get_latest_consensus_by_flavor( + consensus_flavor_t f); networkstatus_t *networkstatus_get_live_consensus(time_t now); -networkstatus_t *networkstatus_get_reasonably_live_consensus(time_t now); +networkstatus_t *networkstatus_get_reasonably_live_consensus(time_t now, + int flavor); #define NSSET_FROM_CACHE 1 #define NSSET_WAS_WAITING_FOR_CERTS 2 #define NSSET_DONT_DOWNLOAD_CERTS 4 @@ -78,10 +93,11 @@ void routers_update_status_from_consensus_networkstatus(smartlist_t *routers, void signed_descs_update_status_from_consensus_networkstatus( smartlist_t *descs); -char *networkstatus_getinfo_helper_single(routerstatus_t *rs); +char *networkstatus_getinfo_helper_single(const routerstatus_t *rs); char *networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now); void networkstatus_dump_bridge_status_to_file(time_t now); -int32_t networkstatus_get_param(networkstatus_t *ns, const char *param_name, +int32_t networkstatus_get_param(const networkstatus_t *ns, + const char *param_name, int32_t default_val, int32_t min_val, int32_t max_val); int getinfo_helper_networkstatus(control_connection_t *conn, diff --git a/src/or/nodelist.c b/src/or/nodelist.c new file mode 100644 index 0000000000..d17850888d --- /dev/null +++ b/src/or/nodelist.c @@ -0,0 +1,820 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "config.h" +#include "dirserv.h" +#include "microdesc.h" +#include "networkstatus.h" +#include "nodelist.h" +#include "policies.h" +#include "router.h" +#include "routerlist.h" + +#include <string.h> + +static void nodelist_drop_node(node_t *node, int remove_from_ht); +static void node_free(node_t *node); + +/** A nodelist_t holds a node_t object for every router we're "willing to use + * for something". Specifically, it should hold a node_t for every node that + * is currently in the routerlist, or currently in the consensus we're using. + */ +typedef struct nodelist_t { + /* A list of all the nodes. */ + smartlist_t *nodes; + /* Hash table to map from node ID digest to node. */ + HT_HEAD(nodelist_map, node_t) nodes_by_id; + +} nodelist_t; + +static INLINE unsigned int +node_id_hash(const node_t *node) +{ +#if SIZEOF_INT == 4 + const uint32_t *p = (const uint32_t*)node->identity; + return p[0] ^ p[1] ^ p[2] ^ p[3] ^ p[4]; +#elif SIZEOF_INT == 8 + const uint64_t *p = (const uint32_t*)node->identity; + const uint32_t *p32 = (const uint32_t*)node->identity; + return p[0] ^ p[1] ^ p32[4]; +#endif +} + +static INLINE unsigned int +node_id_eq(const node_t *node1, const node_t *node2) +{ + return tor_memeq(node1->identity, node2->identity, DIGEST_LEN); +} + +HT_PROTOTYPE(nodelist_map, node_t, ht_ent, node_id_hash, node_id_eq); +HT_GENERATE(nodelist_map, node_t, ht_ent, node_id_hash, node_id_eq, + 0.6, malloc, realloc, free); + +/** The global nodelist. */ +static nodelist_t *the_nodelist=NULL; + +/** Create an empty nodelist if we haven't done so already. */ +static void +init_nodelist(void) +{ + if (PREDICT_UNLIKELY(the_nodelist == NULL)) { + the_nodelist = tor_malloc_zero(sizeof(nodelist_t)); + HT_INIT(nodelist_map, &the_nodelist->nodes_by_id); + the_nodelist->nodes = smartlist_new(); + } +} + +/** As node_get_by_id, but returns a non-const pointer */ +node_t * +node_get_mutable_by_id(const char *identity_digest) +{ + node_t search, *node; + if (PREDICT_UNLIKELY(the_nodelist == NULL)) + return NULL; + + memcpy(&search.identity, identity_digest, DIGEST_LEN); + node = HT_FIND(nodelist_map, &the_nodelist->nodes_by_id, &search); + return node; +} + +/** Return the node_t whose identity is <b>identity_digest</b>, or NULL + * if no such node exists. */ +const node_t * +node_get_by_id(const char *identity_digest) +{ + return node_get_mutable_by_id(identity_digest); +} + +/** Internal: return the node_t whose identity_digest is + * <b>identity_digest</b>. If none exists, create a new one, add it to the + * nodelist, and return it. + * + * Requires that the nodelist be initialized. + */ +static node_t * +node_get_or_create(const char *identity_digest) +{ + node_t *node; + + if ((node = node_get_mutable_by_id(identity_digest))) + return node; + + node = tor_malloc_zero(sizeof(node_t)); + memcpy(node->identity, identity_digest, DIGEST_LEN); + HT_INSERT(nodelist_map, &the_nodelist->nodes_by_id, node); + + smartlist_add(the_nodelist->nodes, node); + node->nodelist_idx = smartlist_len(the_nodelist->nodes) - 1; + + node->country = -1; + + return node; +} + +/** Add <b>ri</b> to the nodelist. */ +node_t * +nodelist_add_routerinfo(routerinfo_t *ri) +{ + node_t *node; + init_nodelist(); + node = node_get_or_create(ri->cache_info.identity_digest); + node->ri = ri; + + if (node->country == -1) + node_set_country(node); + + if (authdir_mode(get_options())) { + const char *discard=NULL; + uint32_t status = dirserv_router_get_status(ri, &discard); + dirserv_set_node_flags_from_authoritative_status(node, status); + } + + return node; +} + +/** Set the appropriate node_t to use <b>md</b> as its microdescriptor. + * + * Called when a new microdesc has arrived and the usable consensus flavor + * is "microdesc". + **/ +node_t * +nodelist_add_microdesc(microdesc_t *md) +{ + networkstatus_t *ns = + networkstatus_get_latest_consensus_by_flavor(FLAV_MICRODESC); + const routerstatus_t *rs; + node_t *node; + if (ns == NULL) + return NULL; + init_nodelist(); + + /* Microdescriptors don't carry an identity digest, so we need to figure + * it out by looking up the routerstatus. */ + rs = router_get_consensus_status_by_descriptor_digest(ns, md->digest); + if (rs == NULL) + return NULL; + node = node_get_mutable_by_id(rs->identity_digest); + if (node) { + if (node->md) + node->md->held_by_nodes--; + node->md = md; + md->held_by_nodes++; + } + return node; +} + +/** Tell the nodelist that the current usable consensus to <b>ns</b>. + * This makes the nodelist change all of the routerstatus entries for + * the nodes, drop nodes that no longer have enough info to get used, + * and grab microdescriptors into nodes as appropriate. + */ +void +nodelist_set_consensus(networkstatus_t *ns) +{ + const or_options_t *options = get_options(); + int authdir = authdir_mode_v2(options) || authdir_mode_v3(options); + + init_nodelist(); + if (ns->flavor == FLAV_MICRODESC) + (void) get_microdesc_cache(); /* Make sure it exists first. */ + + SMARTLIST_FOREACH(the_nodelist->nodes, node_t *, node, + node->rs = NULL); + + SMARTLIST_FOREACH_BEGIN(ns->routerstatus_list, routerstatus_t *, rs) { + node_t *node = node_get_or_create(rs->identity_digest); + node->rs = rs; + if (ns->flavor == FLAV_MICRODESC) { + if (node->md == NULL || + tor_memneq(node->md->digest,rs->descriptor_digest,DIGEST256_LEN)) { + if (node->md) + node->md->held_by_nodes--; + node->md = microdesc_cache_lookup_by_digest256(NULL, + rs->descriptor_digest); + if (node->md) + node->md->held_by_nodes++; + } + } + + node_set_country(node); + + /* If we're not an authdir, believe others. */ + if (!authdir) { + node->is_valid = rs->is_valid; + node->is_running = rs->is_flagged_running; + node->is_fast = rs->is_fast; + node->is_stable = rs->is_stable; + node->is_possible_guard = rs->is_possible_guard; + node->is_exit = rs->is_exit; + node->is_bad_directory = rs->is_bad_directory; + node->is_bad_exit = rs->is_bad_exit; + node->is_hs_dir = rs->is_hs_dir; + } + + } SMARTLIST_FOREACH_END(rs); + + nodelist_purge(); + + if (! authdir) { + SMARTLIST_FOREACH_BEGIN(the_nodelist->nodes, node_t *, node) { + /* We have no routerstatus for this router. Clear flags so we can skip + * it, maybe.*/ + if (!node->rs) { + tor_assert(node->ri); /* if it had only an md, or nothing, purge + * would have removed it. */ + if (node->ri->purpose == ROUTER_PURPOSE_GENERAL) { + /* Clear all flags. */ + node->is_valid = node->is_running = node->is_hs_dir = + node->is_fast = node->is_stable = + node->is_possible_guard = node->is_exit = + node->is_bad_exit = node->is_bad_directory = 0; + } + } + } SMARTLIST_FOREACH_END(node); + } +} + +/** Helper: return true iff a node has a usable amount of information*/ +static INLINE int +node_is_usable(const node_t *node) +{ + return (node->rs) || (node->ri); +} + +/** Tell the nodelist that <b>md</b> is no longer a microdescriptor for the + * node with <b>identity_digest</b>. */ +void +nodelist_remove_microdesc(const char *identity_digest, microdesc_t *md) +{ + node_t *node = node_get_mutable_by_id(identity_digest); + if (node && node->md == md) { + node->md = NULL; + md->held_by_nodes--; + } +} + +/** Tell the nodelist that <b>ri</b> is no longer in the routerlist. */ +void +nodelist_remove_routerinfo(routerinfo_t *ri) +{ + node_t *node = node_get_mutable_by_id(ri->cache_info.identity_digest); + if (node && node->ri == ri) { + node->ri = NULL; + if (! node_is_usable(node)) { + nodelist_drop_node(node, 1); + node_free(node); + } + } +} + +/** Remove <b>node</b> from the nodelist. (Asserts that it was there to begin + * with.) */ +static void +nodelist_drop_node(node_t *node, int remove_from_ht) +{ + node_t *tmp; + int idx; + if (remove_from_ht) { + tmp = HT_REMOVE(nodelist_map, &the_nodelist->nodes_by_id, node); + tor_assert(tmp == node); + } + + idx = node->nodelist_idx; + tor_assert(idx >= 0); + + tor_assert(node == smartlist_get(the_nodelist->nodes, idx)); + smartlist_del(the_nodelist->nodes, idx); + if (idx < smartlist_len(the_nodelist->nodes)) { + tmp = smartlist_get(the_nodelist->nodes, idx); + tmp->nodelist_idx = idx; + } + node->nodelist_idx = -1; +} + +/** Release storage held by <b>node</b> */ +static void +node_free(node_t *node) +{ + if (!node) + return; + if (node->md) + node->md->held_by_nodes--; + tor_assert(node->nodelist_idx == -1); + tor_free(node); +} + +/** Remove all entries from the nodelist that don't have enough info to be + * usable for anything. */ +void +nodelist_purge(void) +{ + node_t **iter; + if (PREDICT_UNLIKELY(the_nodelist == NULL)) + return; + + /* Remove the non-usable nodes. */ + for (iter = HT_START(nodelist_map, &the_nodelist->nodes_by_id); iter; ) { + node_t *node = *iter; + + if (node->md && !node->rs) { + /* An md is only useful if there is an rs. */ + node->md->held_by_nodes--; + node->md = NULL; + } + + if (node_is_usable(node)) { + iter = HT_NEXT(nodelist_map, &the_nodelist->nodes_by_id, iter); + } else { + iter = HT_NEXT_RMV(nodelist_map, &the_nodelist->nodes_by_id, iter); + nodelist_drop_node(node, 0); + node_free(node); + } + } + nodelist_assert_ok(); +} + +/** Release all storage held by the nodelist. */ +void +nodelist_free_all(void) +{ + if (PREDICT_UNLIKELY(the_nodelist == NULL)) + return; + + HT_CLEAR(nodelist_map, &the_nodelist->nodes_by_id); + SMARTLIST_FOREACH_BEGIN(the_nodelist->nodes, node_t *, node) { + node->nodelist_idx = -1; + node_free(node); + } SMARTLIST_FOREACH_END(node); + + smartlist_free(the_nodelist->nodes); + + tor_free(the_nodelist); +} + +/** Check that the nodelist is internally consistent, and consistent with + * the directory info it's derived from. + */ +void +nodelist_assert_ok(void) +{ + routerlist_t *rl = router_get_routerlist(); + networkstatus_t *ns = networkstatus_get_latest_consensus(); + digestmap_t *dm; + + if (!the_nodelist) + return; + + dm = digestmap_new(); + + /* every routerinfo in rl->routers should be in the nodelist. */ + if (rl) { + SMARTLIST_FOREACH_BEGIN(rl->routers, routerinfo_t *, ri) { + const node_t *node = node_get_by_id(ri->cache_info.identity_digest); + tor_assert(node && node->ri == ri); + tor_assert(fast_memeq(ri->cache_info.identity_digest, + node->identity, DIGEST_LEN)); + tor_assert(! digestmap_get(dm, node->identity)); + digestmap_set(dm, node->identity, (void*)node); + } SMARTLIST_FOREACH_END(ri); + } + + /* every routerstatus in ns should be in the nodelist */ + if (ns) { + SMARTLIST_FOREACH_BEGIN(ns->routerstatus_list, routerstatus_t *, rs) { + const node_t *node = node_get_by_id(rs->identity_digest); + tor_assert(node && node->rs == rs); + tor_assert(fast_memeq(rs->identity_digest, node->identity, DIGEST_LEN)); + digestmap_set(dm, node->identity, (void*)node); + if (ns->flavor == FLAV_MICRODESC) { + /* If it's a microdesc consensus, every entry that has a + * microdescriptor should be in the nodelist. + */ + microdesc_t *md = + microdesc_cache_lookup_by_digest256(NULL, rs->descriptor_digest); + tor_assert(md == node->md); + if (md) + tor_assert(md->held_by_nodes >= 1); + } + } SMARTLIST_FOREACH_END(rs); + } + + /* The nodelist should have no other entries, and its entries should be + * well-formed. */ + SMARTLIST_FOREACH_BEGIN(the_nodelist->nodes, node_t *, node) { + tor_assert(digestmap_get(dm, node->identity) != NULL); + tor_assert(node_sl_idx == node->nodelist_idx); + } SMARTLIST_FOREACH_END(node); + + tor_assert((long)smartlist_len(the_nodelist->nodes) == + (long)HT_SIZE(&the_nodelist->nodes_by_id)); + + digestmap_free(dm, NULL); +} + +/** Return a list of a node_t * for every node we know about. The caller + * MUST NOT modify the list. (You can set and clear flags in the nodes if + * you must, but you must not add or remove nodes.) */ +smartlist_t * +nodelist_get_list(void) +{ + init_nodelist(); + return the_nodelist->nodes; +} + +/** Given a hex-encoded nickname of the format DIGEST, $DIGEST, $DIGEST=name, + * or $DIGEST~name, return the node with the matching identity digest and + * nickname (if any). Return NULL if no such node exists, or if <b>hex_id</b> + * is not well-formed. */ +const node_t * +node_get_by_hex_id(const char *hex_id) +{ + char digest_buf[DIGEST_LEN]; + char nn_buf[MAX_NICKNAME_LEN+1]; + char nn_char='\0'; + + if (hex_digest_nickname_decode(hex_id, digest_buf, &nn_char, nn_buf)==0) { + const node_t *node = node_get_by_id(digest_buf); + if (!node) + return NULL; + if (nn_char) { + const char *real_name = node_get_nickname(node); + if (!real_name || strcasecmp(real_name, nn_buf)) + return NULL; + if (nn_char == '=') { + const char *named_id = + networkstatus_get_router_digest_by_nickname(nn_buf); + if (!named_id || tor_memneq(named_id, digest_buf, DIGEST_LEN)) + return NULL; + } + } + return node; + } + + return NULL; +} + +/** Given a nickname (possibly verbose, possibly a hexadecimal digest), return + * the corresponding node_t, or NULL if none exists. Warn the user if + * <b>warn_if_unnamed</b> is set, and they have specified a router by + * nickname, but the Named flag isn't set for that router. */ +const node_t * +node_get_by_nickname(const char *nickname, int warn_if_unnamed) +{ + const node_t *node; + if (!the_nodelist) + return NULL; + + /* Handle these cases: DIGEST, $DIGEST, $DIGEST=name, $DIGEST~name. */ + if ((node = node_get_by_hex_id(nickname)) != NULL) + return node; + + if (!strcasecmp(nickname, UNNAMED_ROUTER_NICKNAME)) + return NULL; + + /* Okay, so if we get here, the nickname is just a nickname. Is there + * a binding for it in the consensus? */ + { + const char *named_id = + networkstatus_get_router_digest_by_nickname(nickname); + if (named_id) + return node_get_by_id(named_id); + } + + /* Is it marked as owned-by-someone-else? */ + if (networkstatus_nickname_is_unnamed(nickname)) { + log_info(LD_GENERAL, "The name %s is listed as Unnamed: there is some " + "router that holds it, but not one listed in the current " + "consensus.", escaped(nickname)); + return NULL; + } + + /* Okay, so the name is not canonical for anybody. */ + { + smartlist_t *matches = smartlist_new(); + const node_t *choice = NULL; + + SMARTLIST_FOREACH_BEGIN(the_nodelist->nodes, node_t *, node) { + if (!strcasecmp(node_get_nickname(node), nickname)) + smartlist_add(matches, node); + } SMARTLIST_FOREACH_END(node); + + if (smartlist_len(matches)>1 && warn_if_unnamed) { + int any_unwarned = 0; + SMARTLIST_FOREACH_BEGIN(matches, node_t *, node) { + if (!node->name_lookup_warned) { + node->name_lookup_warned = 1; + any_unwarned = 1; + } + } SMARTLIST_FOREACH_END(node); + + if (any_unwarned) { + log_warn(LD_CONFIG, "There are multiple matches for the name %s, " + "but none is listed as Named in the directory consensus. " + "Choosing one arbitrarily.", nickname); + } + } else if (smartlist_len(matches)>1 && warn_if_unnamed) { + char fp[HEX_DIGEST_LEN+1]; + node_t *node = smartlist_get(matches, 0); + if (node->name_lookup_warned) { + base16_encode(fp, sizeof(fp), node->identity, DIGEST_LEN); + log_warn(LD_CONFIG, + "You specified a server \"%s\" by name, but the directory " + "authorities do not have any key registered for this " + "nickname -- so it could be used by any server, not just " + "the one you meant. " + "To make sure you get the same server in the future, refer " + "to it by key, as \"$%s\".", nickname, fp); + node->name_lookup_warned = 1; + } + } + + if (smartlist_len(matches)) + choice = smartlist_get(matches, 0); + + smartlist_free(matches); + return choice; + } +} + +/** Return the nickname of <b>node</b>, or NULL if we can't find one. */ +const char * +node_get_nickname(const node_t *node) +{ + tor_assert(node); + if (node->rs) + return node->rs->nickname; + else if (node->ri) + return node->ri->nickname; + else + return NULL; +} + +/** Return true iff the nickname of <b>node</b> is canonical, based on the + * latest consensus. */ +int +node_is_named(const node_t *node) +{ + const char *named_id; + const char *nickname = node_get_nickname(node); + if (!nickname) + return 0; + named_id = networkstatus_get_router_digest_by_nickname(nickname); + if (!named_id) + return 0; + return tor_memeq(named_id, node->identity, DIGEST_LEN); +} + +/** Return true iff <b>node</b> appears to be a directory authority or + * directory cache */ +int +node_is_dir(const node_t *node) +{ + if (node->rs) + return node->rs->dir_port != 0; + else if (node->ri) + return node->ri->dir_port != 0; + else + return 0; +} + +/** Return true iff <b>node</b> has either kind of usable descriptor -- that + * is, a routerdecriptor or a microdescriptor. */ +int +node_has_descriptor(const node_t *node) +{ + return (node->ri || + (node->rs && node->md)); +} + +/** Return the router_purpose of <b>node</b>. */ +int +node_get_purpose(const node_t *node) +{ + if (node->ri) + return node->ri->purpose; + else + return ROUTER_PURPOSE_GENERAL; +} + +/** Compute the verbose ("extended") nickname of <b>node</b> and store it + * into the MAX_VERBOSE_NICKNAME_LEN+1 character buffer at + * <b>verbose_nickname_out</b> */ +void +node_get_verbose_nickname(const node_t *node, + char *verbose_name_out) +{ + const char *nickname = node_get_nickname(node); + int is_named = node_is_named(node); + verbose_name_out[0] = '$'; + base16_encode(verbose_name_out+1, HEX_DIGEST_LEN+1, node->identity, + DIGEST_LEN); + if (!nickname) + return; + verbose_name_out[1+HEX_DIGEST_LEN] = is_named ? '=' : '~'; + strlcpy(verbose_name_out+1+HEX_DIGEST_LEN+1, nickname, MAX_NICKNAME_LEN+1); +} + +/** Return true iff it seems that <b>node</b> allows circuits to exit + * through it directlry from the client. */ +int +node_allows_single_hop_exits(const node_t *node) +{ + if (node && node->ri) + return node->ri->allow_single_hop_exits; + else + return 0; +} + +/** Return true iff it seems that <b>node</b> has an exit policy that doesn't + * actually permit anything to exit, or we don't know its exit policy */ +int +node_exit_policy_rejects_all(const node_t *node) +{ + if (node->rejects_all) + return 1; + + if (node->ri) + return node->ri->policy_is_reject_star; + else if (node->md) + return node->md->exit_policy == NULL || + short_policy_is_reject_star(node->md->exit_policy); + else + return 1; +} + +/** Return list of tor_addr_port_t with all OR ports (in the sense IP + * addr + TCP port) for <b>node</b>. Caller must free all elements + * using tor_free() and free the list using smartlist_free(). + * + * XXX this is potentially a memory fragmentation hog -- if on + * critical path consider the option of having the caller allocate the + * memory + */ +smartlist_t * +node_get_all_orports(const node_t *node) +{ + smartlist_t *sl = smartlist_new(); + + if (node->ri != NULL) { + if (node->ri->addr != 0) { + tor_addr_port_t *ap = tor_malloc(sizeof(tor_addr_port_t)); + tor_addr_from_ipv4h(&ap->addr, node->ri->addr); + ap->port = node->ri->or_port; + smartlist_add(sl, ap); + } + if (!tor_addr_is_null(&node->ri->ipv6_addr)) { + tor_addr_port_t *ap = tor_malloc(sizeof(tor_addr_port_t)); + tor_addr_copy(&ap->addr, &node->ri->ipv6_addr); + ap->port = node->ri->or_port; + smartlist_add(sl, ap); + } + } else if (node->rs != NULL) { + tor_addr_port_t *ap = tor_malloc(sizeof(tor_addr_port_t)); + tor_addr_from_ipv4h(&ap->addr, node->rs->addr); + ap->port = node->rs->or_port; + smartlist_add(sl, ap); + } + + return sl; +} + +/** Copy the primary (IPv4) OR port (IP address and TCP port) for + * <b>node</b> into *<b>ap_out</b>. */ +void +node_get_prim_orport(const node_t *node, tor_addr_port_t *ap_out) +{ + if (node->ri) { + router_get_prim_orport(node->ri, ap_out); + } else if (node->rs) { + tor_addr_from_ipv4h(&ap_out->addr, node->rs->addr); + ap_out->port = node->rs->or_port; + } +} + +/** Wrapper around node_get_prim_orport for backward + compatibility. */ +void +node_get_addr(const node_t *node, tor_addr_t *addr_out) +{ + tor_addr_port_t ap; + node_get_prim_orport(node, &ap); + tor_addr_copy(addr_out, &ap.addr); +} + +/** Return the host-order IPv4 address for <b>node</b>, or 0 if it doesn't + * seem to have one. */ +uint32_t +node_get_prim_addr_ipv4h(const node_t *node) +{ + if (node->ri) { + return node->ri->addr; + } else if (node->rs) { + return node->rs->addr; + } + return 0; +} + +/** Copy the preferred OR port (IP address and TCP port) for + * <b>node</b> into <b>ap_out</b>. */ +void +node_get_pref_orport(const node_t *node, tor_addr_port_t *ap_out) +{ + if (node->ri) { + router_get_pref_orport(node->ri, ap_out); + } else if (node->rs) { + /* No IPv6 in routerstatus_t yet. XXXprop186 ok for private + bridges but needs fixing */ + tor_addr_from_ipv4h(&ap_out->addr, node->rs->addr); + ap_out->port = node->rs->or_port; + } +} + +/** Copy the preferred IPv6 OR port (address and TCP port) for + * <b>node</b> into *<b>ap_out</b>. */ +void +node_get_pref_ipv6_orport(const node_t *node, tor_addr_port_t *ap_out) +{ + if (node->ri) { + router_get_pref_ipv6_orport(node->ri, ap_out); + } else if (node->rs) { + /* No IPv6 in routerstatus_t yet. XXXprop186 ok for private + bridges but needs fixing */ + tor_addr_make_unspec(&ap_out->addr); + ap_out->port = 0; + } +} + +/** Copy a string representation of an IP address for <b>node</b> into + * the <b>len</b>-byte buffer at <b>buf</b>. */ +void +node_get_address_string(const node_t *node, char *buf, size_t len) +{ + if (node->ri) { + strlcpy(buf, node->ri->address, len); + } else if (node->rs) { + tor_addr_t addr; + tor_addr_from_ipv4h(&addr, node->rs->addr); + tor_addr_to_str(buf, &addr, len, 0); + } else { + buf[0] = '\0'; + } +} + +/** Return <b>node</b>'s declared uptime, or -1 if it doesn't seem to have + * one. */ +long +node_get_declared_uptime(const node_t *node) +{ + if (node->ri) + return node->ri->uptime; + else + return -1; +} + +/** Return <b>node</b>'s platform string, or NULL if we don't know it. */ +const char * +node_get_platform(const node_t *node) +{ + /* If we wanted, we could record the version in the routerstatus_t, since + * the consensus lists it. We don't, though, so this function just won't + * work with microdescriptors. */ + if (node->ri) + return node->ri->platform; + else + return NULL; +} + +/** Return <b>node</b>'s time of publication, or 0 if we don't have one. */ +time_t +node_get_published_on(const node_t *node) +{ + if (node->ri) + return node->ri->cache_info.published_on; + else + return 0; +} + +/** Return true iff <b>node</b> is one representing this router. */ +int +node_is_me(const node_t *node) +{ + return router_digest_is_me(node->identity); +} + +/** Return <b>node</b> declared family (as a list of names), or NULL if + * the node didn't declare a family. */ +const smartlist_t * +node_get_declared_family(const node_t *node) +{ + if (node->ri && node->ri->declared_family) + return node->ri->declared_family; + else if (node->md && node->md->family) + return node->md->family; + else + return NULL; +} + diff --git a/src/or/nodelist.h b/src/or/nodelist.h new file mode 100644 index 0000000000..1e9da88d4e --- /dev/null +++ b/src/or/nodelist.h @@ -0,0 +1,67 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file microdesc.h + * \brief Header file for microdesc.c. + **/ + +#ifndef _TOR_NODELIST_H +#define _TOR_NODELIST_H + +node_t *node_get_mutable_by_id(const char *identity_digest); +const node_t *node_get_by_id(const char *identity_digest); +const node_t *node_get_by_hex_id(const char *identity_digest); +node_t *nodelist_add_routerinfo(routerinfo_t *ri); +node_t *nodelist_add_microdesc(microdesc_t *md); +void nodelist_set_consensus(networkstatus_t *ns); + +void nodelist_remove_microdesc(const char *identity_digest, microdesc_t *md); +void nodelist_remove_routerinfo(routerinfo_t *ri); +void nodelist_purge(void); + +void nodelist_free_all(void); +void nodelist_assert_ok(void); + +const node_t *node_get_by_nickname(const char *nickname, int warn_if_unnamed); +void node_get_verbose_nickname(const node_t *node, + char *verbose_name_out); +int node_is_named(const node_t *node); +int node_is_dir(const node_t *node); +int node_has_descriptor(const node_t *node); +int node_get_purpose(const node_t *node); +#define node_is_bridge(node) \ + (node_get_purpose((node)) == ROUTER_PURPOSE_BRIDGE) +int node_is_me(const node_t *node); +int node_exit_policy_rejects_all(const node_t *node); +smartlist_t *node_get_all_orports(const node_t *node); +void node_get_prim_orport(const node_t *node, tor_addr_port_t *addr_port_out); +void node_get_pref_orport(const node_t *node, tor_addr_port_t *addr_port_out); +void node_get_pref_ipv6_orport(const node_t *node, + tor_addr_port_t *addr_port_out); +uint32_t node_get_prim_addr_ipv4h(const node_t *node); +int node_allows_single_hop_exits(const node_t *node); +const char *node_get_nickname(const node_t *node); +const char *node_get_platform(const node_t *node); +void node_get_address_string(const node_t *node, char *cp, size_t len); +long node_get_declared_uptime(const node_t *node); +time_t node_get_published_on(const node_t *node); +const smartlist_t *node_get_declared_family(const node_t *node); + +smartlist_t *nodelist_get_list(void); + +/* Temporary during transition to multiple addresses. */ +void node_get_addr(const node_t *node, tor_addr_t *addr_out); +#define node_get_addr_ipv4h(n) node_get_prim_addr_ipv4h((n)) + +/* XXXX These need to move out of routerlist.c */ +void nodelist_refresh_countries(void); +void node_set_country(node_t *node); +void nodelist_add_node_and_family(smartlist_t *nodes, const node_t *node); +int nodes_in_same_family(const node_t *node1, const node_t *node2); + +#endif + diff --git a/src/or/ntmain.c b/src/or/ntmain.c index 985fab73ca..d001f7be13 100644 --- a/src/or/ntmain.c +++ b/src/or/ntmain.c @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #define MAIN_PRIVATE @@ -193,7 +193,6 @@ nt_service_loadlibrary(void) */ int nt_service_is_stopping(void) -/* XXXX this function would probably _love_ to be inline, in 0.2.0. */ { /* If we haven't loaded the function pointers, we can't possibly be an NT * service trying to shut down. */ @@ -456,10 +455,10 @@ static char * nt_service_command_line(int *using_default_torrc) { TCHAR tor_exe[MAX_PATH+1]; - char tor_exe_ascii[MAX_PATH+1]; - char *command, *options=NULL; + char tor_exe_ascii[MAX_PATH*2+1]; + char *command=NULL, *options=NULL; smartlist_t *sl; - int i, cmdlen; + int i; *using_default_torrc = 1; /* Get the location of tor.exe */ @@ -467,7 +466,7 @@ nt_service_command_line(int *using_default_torrc) return NULL; /* Get the service arguments */ - sl = smartlist_create(); + sl = smartlist_new(); for (i = 1; i < backup_argc; ++i) { if (!strcmp(backup_argv[i], "--options") || !strcmp(backup_argv[i], "-options")) { @@ -484,25 +483,18 @@ nt_service_command_line(int *using_default_torrc) #ifdef UNICODE wcstombs(tor_exe_ascii, tor_exe, sizeof(tor_exe_ascii)); + tor_exe_ascii[sizeof(tor_exe_ascii)-1] = '\0'; #else strlcpy(tor_exe_ascii, tor_exe, sizeof(tor_exe_ascii)); #endif - /* Allocate a string for the NT service command line */ - cmdlen = strlen(tor_exe_ascii) + (options?strlen(options):0) + 32; - command = tor_malloc(cmdlen); - + /* Allocate a string for the NT service command line and */ /* Format the service command */ if (options) { - if (tor_snprintf(command, cmdlen, "\"%s\" --nt-service \"%s\"", - tor_exe_ascii, options)<0) { - tor_free(command); /* sets command to NULL. */ - } + tor_asprintf(&command, "\"%s\" --nt-service \"%s\"", + tor_exe_ascii, options); } else { /* ! options */ - if (tor_snprintf(command, cmdlen, "\"%s\" --nt-service", - tor_exe_ascii)<0) { - tor_free(command); /* sets command to NULL. */ - } + tor_asprintf(&command, "\"%s\" --nt-service", tor_exe_ascii); } tor_free(options); diff --git a/src/or/ntmain.h b/src/or/ntmain.h index acd0e1d7eb..07fdcf466b 100644 --- a/src/or/ntmain.h +++ b/src/or/ntmain.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -12,7 +12,7 @@ #ifndef _TOR_NTMAIN_H #define _TOR_NTMAIN_H -#ifdef MS_WINDOWS +#ifdef _WIN32 #if !defined (WINCE) #define NT_SERVICE #endif diff --git a/src/or/onion.c b/src/or/onion.c index 211d14c1e1..f8c4d72b5a 100644 --- a/src/or/onion.c +++ b/src/or/onion.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -29,9 +29,11 @@ typedef struct onion_queue_t { #define ONIONQUEUE_WAIT_CUTOFF 5 /** First and last elements in the linked list of circuits waiting for CPU - * workers, or NULL if the list is empty. */ + * workers, or NULL if the list is empty. + * @{ */ static onion_queue_t *ol_list=NULL; static onion_queue_t *ol_tail=NULL; +/**@}*/ /** Length of ol_list */ static int ol_length=0; @@ -170,12 +172,12 @@ onion_pending_remove(or_circuit_t *circ) * The meeting point/cookies and auth are zeroed out for now. */ int -onion_skin_create(crypto_pk_env_t *dest_router_key, - crypto_dh_env_t **handshake_state_out, +onion_skin_create(crypto_pk_t *dest_router_key, + crypto_dh_t **handshake_state_out, char *onion_skin_out) /* ONIONSKIN_CHALLENGE_LEN bytes */ { char challenge[DH_KEY_LEN]; - crypto_dh_env_t *dh = NULL; + crypto_dh_t *dh = NULL; int dhbytes, pkbytes; tor_assert(dest_router_key); @@ -204,12 +206,12 @@ onion_skin_create(crypto_pk_env_t *dest_router_key, PK_PKCS1_OAEP_PADDING, 1)<0) goto err; - memset(challenge, 0, sizeof(challenge)); + memwipe(challenge, 0, sizeof(challenge)); *handshake_state_out = dh; return 0; err: - memset(challenge, 0, sizeof(challenge)); + memwipe(challenge, 0, sizeof(challenge)); if (dh) crypto_dh_free(dh); return -1; } @@ -221,19 +223,19 @@ onion_skin_create(crypto_pk_env_t *dest_router_key, */ int onion_skin_server_handshake(const char *onion_skin, /*ONIONSKIN_CHALLENGE_LEN*/ - crypto_pk_env_t *private_key, - crypto_pk_env_t *prev_private_key, + crypto_pk_t *private_key, + crypto_pk_t *prev_private_key, char *handshake_reply_out, /*ONIONSKIN_REPLY_LEN*/ char *key_out, size_t key_out_len) { char challenge[ONIONSKIN_CHALLENGE_LEN]; - crypto_dh_env_t *dh = NULL; + crypto_dh_t *dh = NULL; ssize_t len; char *key_material=NULL; size_t key_material_len=0; int i; - crypto_pk_env_t *k; + crypto_pk_t *k; len = -1; for (i=0;i<2;++i) { @@ -284,15 +286,15 @@ onion_skin_server_handshake(const char *onion_skin, /*ONIONSKIN_CHALLENGE_LEN*/ /* use the rest of the key material for our shared keys, digests, etc */ memcpy(key_out, key_material+DIGEST_LEN, key_out_len); - memset(challenge, 0, sizeof(challenge)); - memset(key_material, 0, key_material_len); + memwipe(challenge, 0, sizeof(challenge)); + memwipe(key_material, 0, key_material_len); tor_free(key_material); crypto_dh_free(dh); return 0; err: - memset(challenge, 0, sizeof(challenge)); + memwipe(challenge, 0, sizeof(challenge)); if (key_material) { - memset(key_material, 0, key_material_len); + memwipe(key_material, 0, key_material_len); tor_free(key_material); } if (dh) crypto_dh_free(dh); @@ -310,7 +312,7 @@ onion_skin_server_handshake(const char *onion_skin, /*ONIONSKIN_CHALLENGE_LEN*/ * After the invocation, call crypto_dh_free on handshake_state. */ int -onion_skin_client_handshake(crypto_dh_env_t *handshake_state, +onion_skin_client_handshake(crypto_dh_t *handshake_state, const char *handshake_reply, /* ONIONSKIN_REPLY_LEN bytes */ char *key_out, size_t key_out_len) @@ -338,11 +340,11 @@ onion_skin_client_handshake(crypto_dh_env_t *handshake_state, /* use the rest of the key material for our shared keys, digests, etc */ memcpy(key_out, key_material+DIGEST_LEN, key_out_len); - memset(key_material, 0, key_material_len); + memwipe(key_material, 0, key_material_len); tor_free(key_material); return 0; err: - memset(key_material, 0, key_material_len); + memwipe(key_material, 0, key_material_len); tor_free(key_material); return -1; } @@ -379,8 +381,8 @@ fast_server_handshake(const uint8_t *key_in, /* DIGEST_LEN bytes */ memcpy(key_out, out+DIGEST_LEN, key_out_len); r = 0; done: - memset(tmp, 0, sizeof(tmp)); - memset(out, 0, out_len); + memwipe(tmp, 0, sizeof(tmp)); + memwipe(out, 0, out_len); tor_free(out); return r; } @@ -424,8 +426,8 @@ fast_client_handshake(const uint8_t *handshake_state,/*DIGEST_LEN bytes*/ memcpy(key_out, out+DIGEST_LEN, key_out_len); r = 0; done: - memset(tmp, 0, sizeof(tmp)); - memset(out, 0, out_len); + memwipe(tmp, 0, sizeof(tmp)); + memwipe(out, 0, out_len); tor_free(out); return r; } diff --git a/src/or/onion.h b/src/or/onion.h index 7f603b8147..7e0f873c73 100644 --- a/src/or/onion.h +++ b/src/or/onion.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -16,18 +16,18 @@ int onion_pending_add(or_circuit_t *circ, char *onionskin); or_circuit_t *onion_next_task(char **onionskin_out); void onion_pending_remove(or_circuit_t *circ); -int onion_skin_create(crypto_pk_env_t *router_key, - crypto_dh_env_t **handshake_state_out, +int onion_skin_create(crypto_pk_t *router_key, + crypto_dh_t **handshake_state_out, char *onion_skin_out); int onion_skin_server_handshake(const char *onion_skin, - crypto_pk_env_t *private_key, - crypto_pk_env_t *prev_private_key, + crypto_pk_t *private_key, + crypto_pk_t *prev_private_key, char *handshake_reply_out, char *key_out, size_t key_out_len); -int onion_skin_client_handshake(crypto_dh_env_t *handshake_state, +int onion_skin_client_handshake(crypto_dh_t *handshake_state, const char *handshake_reply, char *key_out, size_t key_out_len); diff --git a/src/or/or.h b/src/or/or.h index a4e8e4985b..dd95c349c0 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -22,9 +22,10 @@ #endif #endif -#ifdef MS_WINDOWS -#define WIN32_WINNT 0x400 -#define _WIN32_WINNT 0x400 +#ifdef _WIN32 +#ifndef _WIN32_WINNT +#define _WIN32_WINNT 0x0501 +#endif #define WIN32_LEAN_AND_MEAN #endif @@ -75,7 +76,7 @@ #include <time.h> #endif -#ifdef MS_WINDOWS +#ifdef _WIN32 #include <io.h> #include <process.h> #include <direct.h> @@ -83,6 +84,13 @@ #define snprintf _snprintf #endif +#ifdef USE_BUFFEREVENTS +#include <event2/bufferevent.h> +#include <event2/buffer.h> +#include <event2/util.h> +#endif + +#include "crypto.h" #include "tortls.h" #include "../common/torlog.h" #include "container.h" @@ -119,6 +127,10 @@ #define cell_t tor_cell_t #endif +#ifdef ENABLE_TOR2WEB_MODE +#define NON_ANONYMOUS_MODE_ENABLED 1 +#endif + /** Length of longest allowable configured nickname. */ #define MAX_NICKNAME_LEN 19 /** Length of a router identity encoded as a hexadecimal digest, plus @@ -172,7 +184,7 @@ #define ROUTER_MAX_AGE (60*60*48) /** How old can a router get before we (as a server) will no longer * consider it live? In seconds. */ -#define ROUTER_MAX_AGE_TO_PUBLISH (60*60*20) +#define ROUTER_MAX_AGE_TO_PUBLISH (60*60*24) /** How old do we let a saved descriptor get before force-removing it? */ #define OLD_ROUTER_DESC_MAX_AGE (60*60*24*5) @@ -225,15 +237,30 @@ typedef enum { #define PROXY_CONNECT 1 #define PROXY_SOCKS4 2 #define PROXY_SOCKS5 3 +/* !!!! If there is ever a PROXY_* type over 2, we must grow the proxy_type + * field in or_connection_t */ +/* pluggable transports proxy type */ +#define PROXY_PLUGGABLE 4 /* Proxy client handshake states */ -#define PROXY_HTTPS_WANT_CONNECT_OK 1 -#define PROXY_SOCKS4_WANT_CONNECT_OK 2 -#define PROXY_SOCKS5_WANT_AUTH_METHOD_NONE 3 -#define PROXY_SOCKS5_WANT_AUTH_METHOD_RFC1929 4 -#define PROXY_SOCKS5_WANT_AUTH_RFC1929_OK 5 -#define PROXY_SOCKS5_WANT_CONNECT_OK 6 -#define PROXY_CONNECTED 7 +/* We use a proxy but we haven't even connected to it yet. */ +#define PROXY_INFANT 1 +/* We use an HTTP proxy and we've sent the CONNECT command. */ +#define PROXY_HTTPS_WANT_CONNECT_OK 2 +/* We use a SOCKS4 proxy and we've sent the CONNECT command. */ +#define PROXY_SOCKS4_WANT_CONNECT_OK 3 +/* We use a SOCKS5 proxy and we try to negotiate without + any authentication . */ +#define PROXY_SOCKS5_WANT_AUTH_METHOD_NONE 4 +/* We use a SOCKS5 proxy and we try to negotiate with + Username/Password authentication . */ +#define PROXY_SOCKS5_WANT_AUTH_METHOD_RFC1929 5 +/* We use a SOCKS5 proxy and we just sent our credentials. */ +#define PROXY_SOCKS5_WANT_AUTH_RFC1929_OK 6 +/* We use a SOCKS5 proxy and we just sent our CONNECT command. */ +#define PROXY_SOCKS5_WANT_CONNECT_OK 7 +/* We use a proxy and we CONNECTed successfully!. */ +#define PROXY_CONNECTED 8 /** True iff <b>x</b> is an edge connection. */ #define CONN_IS_EDGE(x) \ @@ -257,22 +284,27 @@ typedef enum { #define OR_CONN_STATE_CONNECTING 1 /** State for a connection to an OR: waiting for proxy handshake to complete */ #define OR_CONN_STATE_PROXY_HANDSHAKING 2 -/** State for a connection to an OR or client: SSL is handshaking, not done +/** State for an OR connection client: SSL is handshaking, not done * yet. */ #define OR_CONN_STATE_TLS_HANDSHAKING 3 /** State for a connection to an OR: We're doing a second SSL handshake for - * renegotiation purposes. */ + * renegotiation purposes. (V2 handshake only.) */ #define OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING 4 /** State for a connection at an OR: We're waiting for the client to - * renegotiate. */ + * renegotiate (to indicate a v2 handshake) or send a versions cell (to + * indicate a v3 handshake) */ #define OR_CONN_STATE_TLS_SERVER_RENEGOTIATING 5 -/** State for a connection to an OR: We're done with our SSL handshake, but we - * haven't yet negotiated link protocol versions and sent a netinfo cell. - */ -#define OR_CONN_STATE_OR_HANDSHAKING 6 -/** State for a connection to an OR: Ready to send/receive cells. */ -#define OR_CONN_STATE_OPEN 7 -#define _OR_CONN_STATE_MAX 7 +/** State for an OR connection: We're done with our SSL handshake, we've done + * renegotiation, but we haven't yet negotiated link protocol versions and + * sent a netinfo cell. */ +#define OR_CONN_STATE_OR_HANDSHAKING_V2 6 +/** State for an OR connection: We're done with our SSL handshake, but we + * haven't yet negotiated link protocol versions, done a V3 handshake, and + * sent a netinfo cell. */ +#define OR_CONN_STATE_OR_HANDSHAKING_V3 7 +/** State for an OR connection: Ready to send/receive cells. */ +#define OR_CONN_STATE_OPEN 8 +#define _OR_CONN_STATE_MAX 8 #define _EXIT_CONN_STATE_MIN 1 /** State for an exit connection: waiting for response from DNS farm. */ @@ -386,7 +418,9 @@ typedef enum { /** A connection to a hidden service directory server: download a v2 rendezvous * descriptor. */ #define DIR_PURPOSE_FETCH_RENDDESC_V2 18 -#define _DIR_PURPOSE_MAX 18 +/** A connection to a directory server: download a microdescriptor. */ +#define DIR_PURPOSE_FETCH_MICRODESC 19 +#define _DIR_PURPOSE_MAX 19 /** True iff <b>p</b> is a purpose corresponding to uploading data to a * directory server. */ @@ -505,6 +539,8 @@ typedef enum { #define CIRCUIT_PURPOSE_IS_ESTABLISHED_REND(p) \ ((p) == CIRCUIT_PURPOSE_C_REND_JOINED || \ (p) == CIRCUIT_PURPOSE_S_REND_JOINED) +/** True iff the circuit_t c is actually an or_circuit_t */ +#define CIRCUIT_IS_ORCIRC(c) (((circuit_t *)(c))->magic == OR_CIRCUIT_MAGIC) /** How many circuits do we want simultaneously in-progress to handle * a given stream? */ @@ -756,10 +792,10 @@ typedef struct rend_data_t { char rend_cookie[REND_COOKIE_LEN]; } rend_data_t; -/** Time interval for tracking possible replays of INTRODUCE2 cells. - * Incoming cells with timestamps half of this interval in the past or - * future are dropped immediately. */ -#define REND_REPLAY_TIME_INTERVAL (60 * 60) +/** Time interval for tracking replays of DH public keys received in + * INTRODUCE2 cells. Used only to avoid launching multiple + * simultaneous attempts to connect to the same rendezvous point. */ +#define REND_REPLAY_TIME_INTERVAL (5 * 60) /** Used to indicate which way a cell is going on a circuit. */ typedef enum { @@ -780,6 +816,13 @@ typedef enum { /** Amount to increment a stream window when we get a stream SENDME. */ #define STREAMWINDOW_INCREMENT 50 +/** Maximum number of queued cells on a circuit for which we are the + * midpoint before we give up and kill it. This must be >= circwindow + * to avoid killing innocent circuits, and >= circwindow*2 to give + * leaky-pipe a chance for being useful someday. + */ +#define ORCIRC_MAX_MIDDLE_CELLS (21*(CIRCWINDOW_START_MAX)/10) + /* Cell commands. These values are defined in tor-spec.txt. */ #define CELL_PADDING 0 #define CELL_CREATE 1 @@ -792,9 +835,11 @@ typedef enum { #define CELL_NETINFO 8 #define CELL_RELAY_EARLY 9 -/** True iff the cell command <b>x</b> is one that implies a variable-length - * cell. */ -#define CELL_COMMAND_IS_VAR_LENGTH(x) ((x) == CELL_VERSIONS) +#define CELL_VPADDING 128 +#define CELL_CERTS 129 +#define CELL_AUTH_CHALLENGE 130 +#define CELL_AUTHENTICATE 131 +#define CELL_AUTHORIZE 132 /** How long to test reachability before complaining to the user. */ #define TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT (20*60) @@ -807,6 +852,9 @@ typedef enum { * Tor 0.1.2.x is obsolete, we can remove this. */ #define DEFAULT_CLIENT_NICKNAME "client" +/** Name chosen by routers that don't configure nicknames */ +#define UNNAMED_ROUTER_NICKNAME "Unnamed" + /** Number of bytes in a SOCKS4 header. */ #define SOCKS4_NETWORK_LEN 8 @@ -857,7 +905,7 @@ typedef struct var_cell_t { /** Number of bytes actually stored in <b>payload</b> */ uint16_t payload_len; /** Payload of this cell */ - uint8_t payload[1]; + uint8_t payload[FLEXIBLE_ARRAY_MEMBER]; } var_cell_t; /** A cell as packed for writing to the network. */ @@ -901,14 +949,21 @@ typedef struct { typedef struct buf_t buf_t; typedef struct socks_request_t socks_request_t; +#ifdef USE_BUFFEREVENTS +#define generic_buffer_t struct evbuffer +#else +#define generic_buffer_t buf_t +#endif /* Values for connection_t.magic: used to make sure that downcasts (casts from * connection_t to foo_connection_t) are safe. */ #define BASE_CONNECTION_MAGIC 0x7C3C304Eu #define OR_CONNECTION_MAGIC 0x7D31FF03u #define EDGE_CONNECTION_MAGIC 0xF0374013u +#define ENTRY_CONNECTION_MAGIC 0xbb4a5703 #define DIR_CONNECTION_MAGIC 0x9988ffeeu #define CONTROL_CONNECTION_MAGIC 0x8abc765du +#define LISTENER_CONNECTION_MAGIC 0x1a1ac741u /** Description of a connection to another host or process, and associated * data. @@ -952,6 +1007,9 @@ typedef struct connection_t { /** Set to 1 when we're inside connection_flushed_some to keep us from * calling connection_handle_write() recursively. */ unsigned int in_flushed_some:1; + /** True if connection_handle_write is currently running on this connection. + */ + unsigned int in_connection_handle_write:1; /* For linked connections: */ @@ -971,9 +1029,11 @@ typedef struct connection_t { /** CONNECT/SOCKS proxy client handshake state (for outgoing connections). */ unsigned int proxy_state:4; - /** Our socket; -1 if this connection is closed, or has no socket. */ + /** Our socket; set to TOR_INVALID_SOCKET if this connection is closed, + * or has no socket. */ tor_socket_t s; int conn_array_index; /**< Index into the global connection array. */ + struct event *read_event; /**< Libevent event structure. */ struct event *write_event; /**< Libevent event structure. */ buf_t *inbuf; /**< Buffer holding data read over this connection. */ @@ -984,6 +1044,11 @@ typedef struct connection_t { * read? */ time_t timestamp_lastwritten; /**< When was the last time libevent said we * could write? */ + +#ifdef USE_BUFFEREVENTS + struct bufferevent *bufev; /**< A Libevent buffered IO structure. */ +#endif + time_t timestamp_created; /**< When was this connection_t created? */ /* XXXX_IP6 make this IPv6-capable */ @@ -1008,17 +1073,74 @@ typedef struct connection_t { /** Unique identifier for this connection on this Tor instance. */ uint64_t global_identifier; - /* XXXX023 move this field, and all the listener-only fields (just - socket_family, I think), into a new listener_connection_t subtype. */ + /** Unique ID for measuring tunneled network status requests. */ + uint64_t dirreq_id; +} connection_t; + +/** Subtype of connection_t; used for a listener socket. */ +typedef struct listener_connection_t { + connection_t _base; + /** If the connection is a CONN_TYPE_AP_DNS_LISTENER, this field points * to the evdns_server_port it uses to listen to and answer connections. */ struct evdns_server_port *dns_server_port; - /** Unique ID for measuring tunneled network status requests. */ - uint64_t dirreq_id; -} connection_t; + /** @name Isolation parameters + * + * For an AP listener, these fields describe how to isolate streams that + * arrive on the listener. + * + * @{ + */ + /** The session group for this listener. */ + int session_group; + /** One or more ISO_ flags to describe how to isolate streams. */ + uint8_t isolation_flags; + /**@}*/ -/** Stores flags and information related to the portion of a v2 Tor OR +} listener_connection_t; + +/** Minimum length of the random part of an AUTH_CHALLENGE cell. */ +#define OR_AUTH_CHALLENGE_LEN 32 + +/** + * @name Certificate types for CERTS cells. + * + * These values are defined by the protocol, and affect how an X509 + * certificate in a CERTS cell is interpreted and used. + * + * @{ */ +/** A certificate that authenticates a TLS link key. The subject key + * must match the key used in the TLS handshake; it must be signed by + * the identity key. */ +#define OR_CERT_TYPE_TLS_LINK 1 +/** A self-signed identity certificate. The subject key must be a + * 1024-bit RSA key. */ +#define OR_CERT_TYPE_ID_1024 2 +/** A certificate that authenticates a key used in an AUTHENTICATE cell + * in the v3 handshake. The subject key must be a 1024-bit RSA key; it + * must be signed by the identity key */ +#define OR_CERT_TYPE_AUTH_1024 3 +/**@}*/ + +/** The one currently supported type of AUTHENTICATE cell. It contains + * a bunch of structures signed with an RSA1024 key. The signed + * structures include a HMAC using negotiated TLS secrets, and a digest + * of all cells sent or received before the AUTHENTICATE cell (including + * the random server-generated AUTH_CHALLENGE cell). + */ +#define AUTHTYPE_RSA_SHA256_TLSSECRET 1 + +/** The length of the part of the AUTHENTICATE cell body that the client and + * server can generate independently (when using RSA_SHA256_TLSSECRET). It + * contains everything except the client's timestamp, the client's randomly + * generated nonce, and the signature. */ +#define V3_AUTH_FIXED_PART_LEN (8+(32*6)) +/** The length of the part of the AUTHENTICATE cell body that the client + * signs. */ +#define V3_AUTH_BODY_LEN (V3_AUTH_FIXED_PART_LEN + 8 + 16) + +/** Stores flags and information related to the portion of a v2/v3 Tor OR * connection handshake that happens after the TLS handshake is finished. */ typedef struct or_handshake_state_t { @@ -1029,6 +1151,52 @@ typedef struct or_handshake_state_t { unsigned int started_here : 1; /** True iff we have received and processed a VERSIONS cell. */ unsigned int received_versions : 1; + /** True iff we have received and processed an AUTH_CHALLENGE cell */ + unsigned int received_auth_challenge : 1; + /** True iff we have received and processed a CERTS cell. */ + unsigned int received_certs_cell : 1; + /** True iff we have received and processed an AUTHENTICATE cell */ + unsigned int received_authenticate : 1; + + /* True iff we've received valid authentication to some identity. */ + unsigned int authenticated : 1; + + /** True iff we should feed outgoing cells into digest_sent and + * digest_received respectively. + * + * From the server's side of the v3 handshake, we want to capture everything + * from the VERSIONS cell through and including the AUTH_CHALLENGE cell. + * From the client's, we want to capture everything from the VERSIONS cell + * through but *not* including the AUTHENTICATE cell. + * + * @{ */ + unsigned int digest_sent_data : 1; + unsigned int digest_received_data : 1; + /**@}*/ + + /** Identity digest that we have received and authenticated for our peer + * on this connection. */ + uint8_t authenticated_peer_id[DIGEST_LEN]; + + /** Digests of the cells that we have sent or received as part of a V3 + * handshake. Used for making and checking AUTHENTICATE cells. + * + * @{ + */ + crypto_digest_t *digest_sent; + crypto_digest_t *digest_received; + /** @} */ + + /** Certificates that a connection initiator sent us in a CERTS cell; we're + * holding on to them until we get an AUTHENTICATE cell. + * + * @{ + */ + /** The cert for the key that's supposed to sign the AUTHENTICATE cell */ + tor_cert_t *auth_cert; + /** A self-signed identity certificate */ + tor_cert_t *id_cert; + /**@}*/ } or_handshake_state_t; /** Subtype of connection_t for an "OR connection" -- that is, one that speaks @@ -1074,6 +1242,7 @@ typedef struct or_connection_t { unsigned int is_connection_with_client:1; /** True iff this is an outgoing connection. */ unsigned int is_outgoing:1; + unsigned int proxy_type:2; /**< One of PROXY_NONE...PROXY_SOCKS5 */ uint8_t link_proto; /**< What protocol version are we using? 0 for * "none negotiated yet." */ circid_t next_circ_id; /**< Which circ_id do we try to use next on @@ -1089,10 +1258,17 @@ typedef struct or_connection_t { /* bandwidth* and *_bucket only used by ORs in OPEN state: */ int bandwidthrate; /**< Bytes/s added to the bucket. (OPEN ORs only.) */ int bandwidthburst; /**< Max bucket size for this conn. (OPEN ORs only.) */ +#ifndef USE_BUFFEREVENTS int read_bucket; /**< When this hits 0, stop receiving. Every second we * add 'bandwidthrate' to this, capping it at * bandwidthburst. (OPEN ORs only) */ int write_bucket; /**< When this hits 0, stop writing. Like read_bucket. */ +#else + /** A rate-limiting configuration object to determine how this connection + * set its read- and write- limits. */ + /* XXXX we could share this among all connections. */ + struct ev_token_bucket_cfg *bucket_cfg; +#endif int n_circuits; /**< How many circuits use this connection as p_conn or * n_conn ? */ @@ -1115,27 +1291,27 @@ typedef struct or_connection_t { * identity digest as this one. */ } or_connection_t; -/** Subtype of connection_t for an "edge connection" -- that is, a socks (ap) +/** Subtype of connection_t for an "edge connection" -- that is, an entry (ap) * connection, or an exit. */ typedef struct edge_connection_t { connection_t _base; struct edge_connection_t *next_stream; /**< Points to the next stream at this * edge, if any */ - struct crypt_path_t *cpath_layer; /**< A pointer to which node in the circ - * this conn exits at. */ int package_window; /**< How many more relay cells can I send into the * circuit? */ int deliver_window; /**< How many more relay cells can end at me? */ - /** Nickname of planned exit node -- used with .exit support. */ - char *chosen_exit_name; - - socks_request_t *socks_request; /**< SOCKS structure describing request (AP - * only.) */ struct circuit_t *on_circuit; /**< The circuit (if any) that this edge * connection is using. */ + /** A pointer to which node in the circ this conn exits at. Set for AP + * connections and for hidden service exit connections. */ + struct crypt_path_t *cpath_layer; + /** What rendezvous service are we querying for (if an AP) or providing (if + * an exit)? */ + rend_data_t *rend_data; + uint32_t address_ttl; /**< TTL for address-to-addr mapping on exit * connection. Exit connections only. */ @@ -1151,14 +1327,62 @@ typedef struct edge_connection_t { /** Bytes written since last call to control_event_stream_bandwidth_used() */ uint32_t n_written; - /** What rendezvous service are we querying for? (AP only) */ - rend_data_t *rend_data; + /** True iff this connection is for a DNS request only. */ + unsigned int is_dns_request:1; + + unsigned int edge_has_sent_end:1; /**< For debugging; only used on edge + * connections. Set once we've set the stream end, + * and check in connection_about_to_close_connection(). + */ + /** True iff we've blocked reading until the circuit has fewer queued + * cells. */ + unsigned int edge_blocked_on_circ:1; + +} edge_connection_t; + +/** Subtype of edge_connection_t for an "entry connection" -- that is, a SOCKS + * connection, a DNS request, a TransPort connection or a NATD connection */ +typedef struct entry_connection_t { + edge_connection_t _edge; + + /** Nickname of planned exit node -- used with .exit support. */ + char *chosen_exit_name; + + socks_request_t *socks_request; /**< SOCKS structure describing request (AP + * only.) */ + + /* === Isolation related, AP only. === */ + /** AP only: based on which factors do we isolate this stream? */ + uint8_t isolation_flags; + /** AP only: what session group is this stream in? */ + int session_group; + /** AP only: The newnym epoch in which we created this connection. */ + unsigned nym_epoch; + /** AP only: The original requested address before we rewrote it. */ + char *original_dest_address; + /* Other fields to isolate on already exist. The ClientAddr is addr. The + ClientProtocol is a combination of type and socks_request-> + socks_version. SocksAuth is socks_request->username/password. + DestAddr is in socks_request->address. */ /** Number of times we've reassigned this application connection to * a new circuit. We keep track because the timeout is longer if we've * already retried several times. */ uint8_t num_socks_retries; + /** For AP connections only: buffer for data that we have sent + * optimistically, which we might need to re-send if we have to + * retry this connection. */ + generic_buffer_t *pending_optimistic_data; + /* For AP connections only: buffer for data that we previously sent + * optimistically which we are currently re-sending as we retry this + * connection. */ + generic_buffer_t *sending_optimistic_data; + + /** If this is a DNSPort connection, this field holds the pending DNS + * request that we're going to try to answer. */ + struct evdns_server_request *dns_server_request; + #define NUM_CIRCUITS_LAUNCHED_THRESHOLD 10 /** Number of times we've launched a circuit to handle this stream. If * it gets too high, that could indicate an inconsistency between our @@ -1166,9 +1390,6 @@ typedef struct edge_connection_t { * stream to one of the available circuits" logic. */ unsigned int num_circuits_launched:4; - /** True iff this connection is for a DNS request only. */ - unsigned int is_dns_request:1; - /** True iff this stream must attach to a one-hop circuit (e.g. for * begin_dir). */ unsigned int want_onehop:1; @@ -1176,13 +1397,6 @@ typedef struct edge_connection_t { * itself rather than BEGIN (either via onehop or via a whole circuit). */ unsigned int use_begindir:1; - unsigned int edge_has_sent_end:1; /**< For debugging; only used on edge - * connections. Set once we've set the stream end, - * and check in connection_about_to_close_connection(). - */ - /** True iff we've blocked reading until the circuit has fewer queued - * cells. */ - unsigned int edge_blocked_on_circ:1; /** For AP connections only. If 1, and we fail to reach the chosen exit, * stop requiring it. */ unsigned int chosen_exit_optional:1; @@ -1196,19 +1410,26 @@ typedef struct edge_connection_t { * NATd connection */ unsigned int is_transparent_ap:1; - /** If this is a DNSPort connection, this field holds the pending DNS - * request that we're going to try to answer. */ - struct evdns_server_request *dns_server_request; + /** For AP connections only: Set if this connection's target exit node + * allows optimistic data (that is, data sent on this stream before + * the exit has sent a CONNECTED cell) and we have chosen to use it. + */ + unsigned int may_use_optimistic_data : 1; -} edge_connection_t; +} entry_connection_t; /** Subtype of connection_t for an "directory connection" -- that is, an HTTP * connection to retrieve or serve directory material. */ typedef struct dir_connection_t { connection_t _base; - char *requested_resource; /**< Which 'resource' did we ask the directory - * for? */ + /** Which 'resource' did we ask the directory for? This is typically the part + * of the URL string that defines, relative to the directory conn purpose, + * what thing we want. For example, in router descriptor downloads by + * descriptor digest, it contains "d/", then one ore more +-separated + * fingerprints. + **/ + char *requested_resource; unsigned int dirconn_direct:1; /**< Is this dirconn direct, or via Tor? */ /* Used only for server sides of some dir connections, to implement @@ -1274,6 +1495,11 @@ typedef struct control_connection_t { /** Helper macro: Given a pointer to to._base, of type from*, return &to. */ #define DOWNCAST(to, ptr) ((to*)SUBTYPE_P(ptr, to, _base)) +/** Cast a entry_connection_t subtype pointer to a edge_connection_t **/ +#define ENTRY_TO_EDGE_CONN(c) (&(((c))->_edge)) +/** Cast a entry_connection_t subtype pointer to a connection_t **/ +#define ENTRY_TO_CONN(c) (TO_CONN(ENTRY_TO_EDGE_CONN(c))) + /** Convert a connection_t* to an or_connection_t*; assert if the cast is * invalid. */ static or_connection_t *TO_OR_CONN(connection_t *); @@ -1283,9 +1509,18 @@ static dir_connection_t *TO_DIR_CONN(connection_t *); /** Convert a connection_t* to an edge_connection_t*; assert if the cast is * invalid. */ static edge_connection_t *TO_EDGE_CONN(connection_t *); +/** Convert a connection_t* to an entry_connection_t*; assert if the cast is + * invalid. */ +static entry_connection_t *TO_ENTRY_CONN(connection_t *); +/** Convert a edge_connection_t* to an entry_connection_t*; assert if the cast + * is invalid. */ +static entry_connection_t *EDGE_TO_ENTRY_CONN(edge_connection_t *); /** Convert a connection_t* to an control_connection_t*; assert if the cast is * invalid. */ static control_connection_t *TO_CONTROL_CONN(connection_t *); +/** Convert a connection_t* to an listener_connection_t*; assert if the cast is + * invalid. */ +static listener_connection_t *TO_LISTENER_CONN(connection_t *); static INLINE or_connection_t *TO_OR_CONN(connection_t *c) { @@ -1299,14 +1534,75 @@ static INLINE dir_connection_t *TO_DIR_CONN(connection_t *c) } static INLINE edge_connection_t *TO_EDGE_CONN(connection_t *c) { - tor_assert(c->magic == EDGE_CONNECTION_MAGIC); + tor_assert(c->magic == EDGE_CONNECTION_MAGIC || + c->magic == ENTRY_CONNECTION_MAGIC); return DOWNCAST(edge_connection_t, c); } +static INLINE entry_connection_t *TO_ENTRY_CONN(connection_t *c) +{ + tor_assert(c->magic == ENTRY_CONNECTION_MAGIC); + return (entry_connection_t*) SUBTYPE_P(c, entry_connection_t, _edge._base); +} +static INLINE entry_connection_t *EDGE_TO_ENTRY_CONN(edge_connection_t *c) +{ + tor_assert(c->_base.magic == ENTRY_CONNECTION_MAGIC); + return (entry_connection_t*) SUBTYPE_P(c, entry_connection_t, _edge); +} static INLINE control_connection_t *TO_CONTROL_CONN(connection_t *c) { tor_assert(c->magic == CONTROL_CONNECTION_MAGIC); return DOWNCAST(control_connection_t, c); } +static INLINE listener_connection_t *TO_LISTENER_CONN(connection_t *c) +{ + tor_assert(c->magic == LISTENER_CONNECTION_MAGIC); + return DOWNCAST(listener_connection_t, c); +} + +/* Conditional macros to help write code that works whether bufferevents are + disabled or not. + + We can't just write: + if (conn->bufev) { + do bufferevent stuff; + } else { + do other stuff; + } + because the bufferevent stuff won't even compile unless we have a fairly + new version of Libevent. Instead, we say: + IF_HAS_BUFFEREVENT(conn, { do_bufferevent_stuff } ); + or: + IF_HAS_BUFFEREVENT(conn, { + do bufferevent stuff; + }) ELSE_IF_NO_BUFFEREVENT { + do non-bufferevent stuff; + } + If we're compiling with bufferevent support, then the macros expand more or + less to: + if (conn->bufev) { + do_bufferevent_stuff; + } else { + do non-bufferevent stuff; + } + and if we aren't using bufferevents, they expand more or less to: + { do non-bufferevent stuff; } +*/ +#ifdef USE_BUFFEREVENTS +#define HAS_BUFFEREVENT(c) (((c)->bufev) != NULL) +#define IF_HAS_BUFFEREVENT(c, stmt) \ + if ((c)->bufev) do { \ + stmt ; \ + } while (0) +#define ELSE_IF_NO_BUFFEREVENT ; else +#define IF_HAS_NO_BUFFEREVENT(c) \ + if (NULL == (c)->bufev) +#else +#define HAS_BUFFEREVENT(c) (0) +#define IF_HAS_BUFFEREVENT(c, stmt) (void)0 +#define ELSE_IF_NO_BUFFEREVENT ; +#define IF_HAS_NO_BUFFEREVENT(c) \ + if (1) +#endif /** What action type does an address policy indicate: accept or reject? */ typedef enum { @@ -1423,11 +1719,6 @@ typedef struct signed_descriptor_t { * networkstatus that listed it. 0 for "never listed in a consensus or * status, so far as we know." */ time_t last_listed_as_valid_until; -#ifdef TRACK_SERVED_TIME - /** The last time we served anybody this descriptor. Used for internal - * testing to see whether we're holding on to descriptors too long. */ - time_t last_served_at; /*XXXX remove if not useful. */ -#endif /* If true, we do not ever try to save this object in the cache. */ unsigned int do_not_cache : 1; /* If true, this item is meant to represent an extrainfo. */ @@ -1452,8 +1743,15 @@ typedef struct { uint16_t or_port; /**< Port for TLS connections. */ uint16_t dir_port; /**< Port for HTTP directory connections. */ - crypto_pk_env_t *onion_pkey; /**< Public RSA key for onions. */ - crypto_pk_env_t *identity_pkey; /**< Public RSA key for signing. */ + /** A router's IPv6 address, if it has one. */ + /* XXXXX187 Actually these should probably be part of a list of addresses, + * not just a special case. Use abstractions to access these; don't do it + * directly. */ + tor_addr_t ipv6_addr; + uint16_t ipv6_orport; + + crypto_pk_t *onion_pkey; /**< Public RSA key for onions. */ + crypto_pk_t *identity_pkey; /**< Public RSA key for signing. */ char *platform; /**< What software/operating system is this OR using? */ @@ -1471,59 +1769,51 @@ typedef struct { char *contact_info; /**< Declared contact info for this router. */ unsigned int is_hibernating:1; /**< Whether the router claims to be * hibernating */ - unsigned int has_old_dnsworkers:1; /**< Whether the router is using - * dnsworker code. */ - unsigned int caches_extra_info:1; /**< Whether the router caches and serves - * extrainfo documents. */ - unsigned int allow_single_hop_exits:1; /**< Whether the router allows - * single hop exits. */ - - /* local info */ - unsigned int is_running:1; /**< As far as we know, is this OR currently - * running? */ - unsigned int is_valid:1; /**< Has a trusted dirserver validated this OR? - * (For Authdir: Have we validated this OR?) - */ - unsigned int is_named:1; /**< Do we believe the nickname that this OR gives - * us? */ - unsigned int is_fast:1; /** Do we think this is a fast OR? */ - unsigned int is_stable:1; /** Do we think this is a stable OR? */ - unsigned int is_possible_guard:1; /**< Do we think this is an OK guard? */ - unsigned int is_exit:1; /**< Do we think this is an OK exit? */ - unsigned int is_bad_exit:1; /**< Do we think this exit is censored, borked, - * or otherwise nasty? */ - unsigned int is_bad_directory:1; /**< Do we think this directory is junky, - * underpowered, or otherwise useless? */ + unsigned int caches_extra_info:1; /**< Whether the router says it caches and + * serves extrainfo documents. */ + unsigned int allow_single_hop_exits:1; /**< Whether the router says + * it allows single hop exits. */ + unsigned int wants_to_be_hs_dir:1; /**< True iff this router claims to be * a hidden service directory. */ - unsigned int is_hs_dir:1; /**< True iff this router is a hidden service - * directory according to the authorities. */ unsigned int policy_is_reject_star:1; /**< True iff the exit policy for this * router rejects everything. */ /** True if, after we have added this router, we should re-launch * tests for it. */ unsigned int needs_retest_if_added:1; + /** True if ipv6_addr:ipv6_orport is preferred. */ + unsigned int ipv6_preferred:1; -/** Tor can use this router for general positions in circuits. */ +/** Tor can use this router for general positions in circuits; we got it + * from a directory server as usual, or we're an authority and a server + * uploaded it. */ #define ROUTER_PURPOSE_GENERAL 0 -/** Tor should avoid using this router for circuit-building. */ +/** Tor should avoid using this router for circuit-building: we got it + * from a crontroller. If the controller wants to use it, it'll have to + * ask for it by identity. */ #define ROUTER_PURPOSE_CONTROLLER 1 -/** Tor should use this router only for bridge positions in circuits. */ +/** Tor should use this router only for bridge positions in circuits: we got + * it via a directory request from the bridge itself, or a bridge + * authority. x*/ #define ROUTER_PURPOSE_BRIDGE 2 /** Tor should not use this router; it was marked in cached-descriptors with * a purpose we didn't recognize. */ #define ROUTER_PURPOSE_UNKNOWN 255 - uint8_t purpose; /** What positions in a circuit is this router good for? */ + /* In what way did we find out about this router? One of ROUTER_PURPOSE_*. + * Routers of different purposes are kept segregated and used for different + * things; see notes on ROUTER_PURPOSE_* macros above. + */ + uint8_t purpose; /* The below items are used only by authdirservers for * reachability testing. */ + /** When was the last time we could reach this OR? */ time_t last_reachable; /** When did we start testing reachability for this OR? */ time_t testing_since; - /** According to the geoip db what country is this router in? */ - country_t country; + } routerinfo_t; /** Information needed to keep and cache a signed extra-info document. */ @@ -1549,8 +1839,9 @@ typedef struct routerstatus_t { * has. */ char identity_digest[DIGEST_LEN]; /**< Digest of the router's identity * key. */ - char descriptor_digest[DIGEST_LEN]; /**< Digest of the router's most recent - * descriptor. */ + /** Digest of the router's most recent descriptor or microdescriptor. + * If it's a descriptor, we only use the first DIGEST_LEN bytes. */ + char descriptor_digest[DIGEST256_LEN]; uint32_t addr; /**< IPv4 address for this router. */ uint16_t or_port; /**< OR port for this router. */ uint16_t dir_port; /**< Directory port for this router. */ @@ -1558,7 +1849,11 @@ typedef struct routerstatus_t { unsigned int is_exit:1; /**< True iff this router is a good exit. */ unsigned int is_stable:1; /**< True iff this router stays up a long time. */ unsigned int is_fast:1; /**< True iff this router has good bandwidth. */ - unsigned int is_running:1; /**< True iff this router is up. */ + /** True iff this router is called 'running' in the consensus. We give it + * this funny name so that we don't accidentally use this bit as a view of + * whether we think the router is *currently* running. If that's what you + * want to know, look at is_running in node_t. */ + unsigned int is_flagged_running:1; unsigned int is_named:1; /**< True iff "nickname" belongs to this router. */ unsigned int is_unnamed:1; /**< True iff "nickname" belongs to another * router. */ @@ -1589,6 +1884,12 @@ typedef struct routerstatus_t { /** True iff this router is a version that, if it caches directory info, * we can get v3 downloads from. */ unsigned int version_supports_v3_dir:1; + /** True iff this router is a version that, if it caches directory info, + * we can get microdescriptors from. */ + unsigned int version_supports_microdesc_cache:1; + /** True iff this router is a version that allows DATA cells to arrive on + * a stream before it has sent a CONNECTED cell. */ + unsigned int version_supports_optimistic_data:1; unsigned int has_bandwidth:1; /**< The vote/consensus had bw info */ unsigned int has_exitsummary:1; /**< The vote/consensus had exit summaries */ @@ -1610,15 +1911,31 @@ typedef struct routerstatus_t { * from this authority.) Applies in v2 networkstatus document only. */ unsigned int need_to_mirror:1; - unsigned int name_lookup_warned:1; /**< Have we warned the user for referring - * to this (unnamed) router by nickname? - */ time_t last_dir_503_at; /**< When did this router last tell us that it * was too busy to serve directory info? */ download_status_t dl_status; } routerstatus_t; +/** A single entry in a parsed policy summary, describing a range of ports. */ +typedef struct short_policy_entry_t { + uint16_t min_port, max_port; +} short_policy_entry_t; + +/** A short_poliy_t is the parsed version of a policy summary. */ +typedef struct short_policy_t { + /** True if the members of 'entries' are port ranges to accept; false if + * they are port ranges to reject */ + unsigned int is_accept : 1; + /** The actual number of values in 'entries'. */ + unsigned int n_entries : 31; + /** An array of 0 or more short_policy_entry_t values, each describing a + * range of ports that this policy accepts or rejects (depending on the + * value of is_accept). + */ + short_policy_entry_t entries[FLEXIBLE_ARRAY_MEMBER]; +} short_policy_t; + /** A microdescriptor is the smallest amount of information needed to build a * circuit through a router. They are generated by the directory authorities, * using information from the uploaded routerinfo documents. They are not @@ -1638,6 +1955,11 @@ typedef struct microdesc_t { saved_location_t saved_location : 3; /** If true, do not attempt to cache this microdescriptor on disk. */ unsigned int no_save : 1; + /** If true, this microdesc has an entry in the microdesc_map */ + unsigned int held_in_map : 1; + /** Reference count: how many node_ts have a reference to this microdesc? */ + unsigned int held_by_nodes; + /** If saved_location == SAVED_IN_CACHE, this field holds the offset of the * microdescriptor in the cache. */ off_t off; @@ -1657,18 +1979,86 @@ typedef struct microdesc_t { /* Fields in the microdescriptor. */ /** As routerinfo_t.onion_pkey */ - crypto_pk_env_t *onion_pkey; + crypto_pk_t *onion_pkey; /** As routerinfo_t.family */ smartlist_t *family; - /** Encoded exit policy summary */ - char *exitsummary; /**< exit policy summary - - * XXX this probably should not stay a string. */ + /** Exit policy summary */ + short_policy_t *exit_policy; } microdesc_t; +/** A node_t represents a Tor router. + * + * Specifically, a node_t is a Tor router as we are using it: a router that + * we are considering for circuits, connections, and so on. A node_t is a + * thin wrapper around the routerstatus, routerinfo, and microdesc for a + * single wrapper, and provides a consistent interface for all of them. + * + * Also, a node_t has mutable state. While a routerinfo, a routerstatus, + * and a microdesc have[*] only the information read from a router + * descriptor, a consensus entry, and a microdescriptor (respectively)... + * a node_t has flags based on *our own current opinion* of the node. + * + * [*] Actually, there is some leftover information in each that is mutable. + * We should try to excise that. + */ +typedef struct node_t { + /* Indexing information */ + + /** Used to look up the node_t by its identity digest. */ + HT_ENTRY(node_t) ht_ent; + /** Position of the node within the list of nodes */ + int nodelist_idx; + + /** The identity digest of this node_t. No more than one node_t per + * identity may exist at a time. */ + char identity[DIGEST_LEN]; + + microdesc_t *md; + routerinfo_t *ri; + routerstatus_t *rs; + + /* local info: copied from routerstatus, then possibly frobbed based + * on experience. Authorities set this stuff directly. */ + + unsigned int is_running:1; /**< As far as we know, is this OR currently + * running? */ + unsigned int is_valid:1; /**< Has a trusted dirserver validated this OR? + * (For Authdir: Have we validated this OR?) + */ + unsigned int is_fast:1; /** Do we think this is a fast OR? */ + unsigned int is_stable:1; /** Do we think this is a stable OR? */ + unsigned int is_possible_guard:1; /**< Do we think this is an OK guard? */ + unsigned int is_exit:1; /**< Do we think this is an OK exit? */ + unsigned int is_bad_exit:1; /**< Do we think this exit is censored, borked, + * or otherwise nasty? */ + unsigned int is_bad_directory:1; /**< Do we think this directory is junky, + * underpowered, or otherwise useless? */ + unsigned int is_hs_dir:1; /**< True iff this router is a hidden service + * directory according to the authorities. */ + + /* Local info: warning state. */ + + unsigned int name_lookup_warned:1; /**< Have we warned the user for referring + * to this (unnamed) router by nickname? + */ + + /** Local info: we treat this node as if it rejects everything */ + unsigned int rejects_all:1; + + /* Local info: derived. */ + + /** According to the geoip db what country is this router in? */ + country_t country; +} node_t; + /** How many times will we try to download a router's descriptor before giving * up? */ #define MAX_ROUTERDESC_DOWNLOAD_FAILURES 8 +/** How many times will we try to download a microdescriptor before giving + * up? */ +#define MAX_MICRODESC_DOWNLOAD_FAILURES 8 + /** Contents of a v2 (non-consensus, non-vote) network status object. */ typedef struct networkstatus_v2_t { /** When did we receive the network-status document? */ @@ -1697,7 +2087,7 @@ typedef struct networkstatus_v2_t { char identity_digest[DIGEST_LEN]; /**< Digest of signing key. */ char *contact; /**< How to contact directory admin? (may be NULL). */ - crypto_pk_env_t *signing_key; /**< Key used to sign this directory. */ + crypto_pk_t *signing_key; /**< Key used to sign this directory. */ char *client_versions; /**< comma-separated list of recommended client * versions. */ char *server_versions; /**< comma-separated list of recommended server @@ -1782,9 +2172,6 @@ typedef enum { FLAV_MICRODESC = 1, } consensus_flavor_t; -/** Which consensus flavor do we actually want to use to build circuits? */ -#define USABLE_CONSENSUS_FLAVOR FLAV_NS - /** How many different consensus flavors are there? */ #define N_CONSENSUS_FLAVORS ((int)(FLAV_MICRODESC)+1) @@ -1929,7 +2316,7 @@ typedef struct extend_info_t { char identity_digest[DIGEST_LEN]; /**< Hash of this router's identity key. */ uint16_t port; /**< OR port. */ tor_addr_t addr; /**< IP address. */ - crypto_pk_env_t *onion_key; /**< Current onionskin key. */ + crypto_pk_t *onion_key; /**< Current onionskin key. */ } extend_info_t; /** Certificate for v3 directory protocol: binds long-term authority identity @@ -1938,9 +2325,9 @@ typedef struct authority_cert_t { /** Information relating to caching this cert on disk and looking it up. */ signed_descriptor_t cache_info; /** This authority's long-term authority identity key. */ - crypto_pk_env_t *identity_key; + crypto_pk_t *identity_key; /** This authority's medium-term signing key. */ - crypto_pk_env_t *signing_key; + crypto_pk_t *signing_key; /** The digest of <b>signing_key</b> */ char signing_key_digest[DIGEST_LEN]; /** The listed expiration time of this certificate. */ @@ -1954,24 +2341,33 @@ typedef struct authority_cert_t { uint8_t is_cross_certified; } authority_cert_t; -/** Bitfield enum type listing types of directory authority/directory - * server. */ +/** Bitfield enum type listing types of information that directory authorities + * can be authoritative about, and that directory caches may or may not cache. + * + * Note that the granularity here is based on authority granularity and on + * cache capabilities. Thus, one particular bit may correspond in practice to + * a few types of directory info, so long as every authority that pronounces + * officially about one of the types prounounces officially about all of them, + * and so long as every cache that caches one of them caches all of them. + */ typedef enum { - NO_AUTHORITY = 0, + NO_DIRINFO = 0, /** Serves/signs v1 directory information: Big lists of routers, and short * routerstatus documents. */ - V1_AUTHORITY = 1 << 0, + V1_DIRINFO = 1 << 0, /** Serves/signs v2 directory information: i.e. v2 networkstatus documents */ - V2_AUTHORITY = 1 << 1, + V2_DIRINFO = 1 << 1, /** Serves/signs v3 directory information: votes, consensuses, certs */ - V3_AUTHORITY = 1 << 2, + V3_DIRINFO = 1 << 2, /** Serves hidden service descriptors. */ - HIDSERV_AUTHORITY = 1 << 3, + HIDSERV_DIRINFO = 1 << 3, /** Serves bridge descriptors. */ - BRIDGE_AUTHORITY = 1 << 4, - /** Serves extrainfo documents. (XXX Not precisely an authority type)*/ - EXTRAINFO_CACHE = 1 << 5, -} authority_type_t; + BRIDGE_DIRINFO = 1 << 4, + /** Serves extrainfo documents. */ + EXTRAINFO_DIRINFO=1 << 5, + /** Serves microdescriptors. */ + MICRODESC_DIRINFO=1 << 6, +} dirinfo_type_t; #define CRYPT_PATH_MAGIC 0x70127012u @@ -1983,19 +2379,19 @@ typedef struct crypt_path_t { /* crypto environments */ /** Encryption key and counter for cells heading towards the OR at this * step. */ - crypto_cipher_env_t *f_crypto; + crypto_cipher_t *f_crypto; /** Encryption key and counter for cells heading back from the OR at this * step. */ - crypto_cipher_env_t *b_crypto; + crypto_cipher_t *b_crypto; /** Digest state for cells heading towards the OR at this step. */ - crypto_digest_env_t *f_digest; /* for integrity checking */ + crypto_digest_t *f_digest; /* for integrity checking */ /** Digest state for cells heading away from the OR at this step. */ - crypto_digest_env_t *b_digest; + crypto_digest_t *b_digest; /** Current state of Diffie-Hellman key negotiation with the OR at this * step. */ - crypto_dh_env_t *dh_handshake_state; + crypto_dh_t *dh_handshake_state; /** Current state of 'fast' (non-PK) key negotiation with the OR at this * step. Used to save CPU when TLS is already providing all the * authentication, secrecy, and integrity we need, and we're already @@ -2029,6 +2425,18 @@ typedef struct crypt_path_t { * at this step? */ } crypt_path_t; +/** A reference-counted pointer to a crypt_path_t, used only to share + * the final rendezvous cpath to be used on a service-side rendezvous + * circuit among multiple circuits built in parallel to the same + * destination rendezvous point. */ +typedef struct { + /** The reference count. */ + unsigned int refcount; + /** The pointer. Set to NULL when the crypt_path_t is put into use + * on an opened rendezvous circuit. */ + crypt_path_t *cpath; +} crypt_path_reference_t; + #define CPATH_KEY_MATERIAL_LEN (20*2+16*2) #define DH_KEY_LEN DH_BYTES @@ -2044,17 +2452,20 @@ typedef struct { /** How to extend to the planned exit node. */ extend_info_t *chosen_exit; /** Whether every node in the circ must have adequate uptime. */ - int need_uptime; + unsigned int need_uptime : 1; /** Whether every node in the circ must have adequate capacity. */ - int need_capacity; + unsigned int need_capacity : 1; /** Whether the last hop was picked with exiting in mind. */ - int is_internal; - /** Did we pick this as a one-hop tunnel (not safe for other conns)? - * These are for encrypted connections that exit to this router, not + unsigned int is_internal : 1; + /** Did we pick this as a one-hop tunnel (not safe for other streams)? + * These are for encrypted dir conns that exit to this router, not * for arbitrary exits from the circuit. */ - int onehop_tunnel; + unsigned int onehop_tunnel : 1; /** The crypt_path_t to append after rendezvous: used for rendezvous. */ crypt_path_t *pending_final_cpath; + /** A ref-counted reference to the crypt_path_t to append after + * rendezvous; used on the service side. */ + crypt_path_reference_t *service_pending_final_cpath_ref; /** How many times has building a circuit for this task failed? */ int failure_count; /** At what time should we give up on this task? */ @@ -2150,7 +2561,10 @@ typedef struct circuit_t { * length ONIONSKIN_CHALLENGE_LEN. */ char *n_conn_onionskin; - struct timeval timestamp_created; /**< When was the circuit created? */ + /** When was this circuit created? We keep this timestamp with a higher + * resolution than most so that the circuit-build-time tracking code can + * get millisecond resolution. */ + struct timeval timestamp_created; /** When the circuit was first used, or 0 if the circuit is clean. * * XXXX023 Note that some code will artifically adjust this value backward @@ -2194,6 +2608,23 @@ typedef struct circuit_t { * circuit. */ #define MAX_RELAY_EARLY_CELLS_PER_CIRCUIT 8 +/** + * Describes the circuit building process in simplified terms based + * on the path bias accounting state for a circuit. Created to prevent + * overcounting due to unknown cases of circuit reuse. See Bug #6475. + */ +typedef enum { + /** This circuit is "new". It has not yet completed a first hop + * or been counted by the path bias code. */ + PATH_STATE_NEW_CIRC = 0, + /** This circuit has completed a first hop, and has been counted by + * the path bias logic. */ + PATH_STATE_DID_FIRST_HOP = 1, + /** This circuit has been completely built, and has been counted as + * successful by the path bias logic. */ + PATH_STATE_SUCCEEDED = 2, +} path_state_t; + /** An origin_circuit_t holds data necessary to build and use a circuit. */ typedef struct origin_circuit_t { @@ -2227,6 +2658,34 @@ typedef struct origin_circuit_t { * cannibalized circuits. */ unsigned int has_opened : 1; + /** Kludge to help us prevent the warn in bug #6475 and eventually + * debug why we are not seeing first hops in some cases. */ + path_state_t path_state : 2; + + /** Set iff this is a hidden-service circuit which has timed out + * according to our current circuit-build timeout, but which has + * been kept around because it might still succeed in connecting to + * its destination, and which is not a fully-connected rendezvous + * circuit. + * + * (We clear this flag for client-side rendezvous circuits when they + * are 'joined' to the other side's rendezvous circuit, so that + * connection_ap_handshake_attach_circuit can put client streams on + * the circuit. We also clear this flag for service-side rendezvous + * circuits when they are 'joined' to a client's rend circ, but only + * for symmetry with the client case. Client-side introduction + * circuits are closed when we get a joined rend circ, and + * service-side introduction circuits never have this flag set.) */ + unsigned int hs_circ_has_timed_out : 1; + + /** Set iff this is a service-side rendezvous circuit for which a + * new connection attempt has been launched. We consider launching + * a new service-side rend circ to a client when the previous one + * fails; now that we don't necessarily close a service-side rend + * circ when we launch a new one to the same client, this flag keeps + * us from launching two retries for the same failed rend circ. */ + unsigned int hs_service_side_rend_circ_has_been_relaunched : 1; + /** What commands were sent over this circuit that decremented the * RELAY_EARLY counter? This is for debugging task 878. */ uint8_t relay_early_commands[MAX_RELAY_EARLY_CELLS_PER_CIRCUIT]; @@ -2242,12 +2701,61 @@ typedef struct origin_circuit_t { /* The intro key replaces the hidden service's public key if purpose is * S_ESTABLISH_INTRO or S_INTRO, provided that no unversioned rendezvous * descriptor is used. */ - crypto_pk_env_t *intro_key; + crypto_pk_t *intro_key; /** Quasi-global identifier for this circuit; used for control.c */ /* XXXX NM This can get re-used after 2**32 circuits. */ uint32_t global_identifier; + /** True if we have associated one stream to this circuit, thereby setting + * the isolation paramaters for this circuit. Note that this doesn't + * necessarily mean that we've <em>attached</em> any streams to the circuit: + * we may only have marked up this circuit during the launch process. + */ + unsigned int isolation_values_set : 1; + /** True iff any stream has <em>ever</em> been attached to this circuit. + * + * In a better world we could use timestamp_dirty for this, but + * timestamp_dirty is far too overloaded at the moment. + */ + unsigned int isolation_any_streams_attached : 1; + + /** A bitfield of ISO_* flags for every isolation field such that this + * circuit has had streams with more than one value for that field + * attached to it. */ + uint8_t isolation_flags_mixed; + + /** @name Isolation parameters + * + * If any streams have been associated with this circ (isolation_values_set + * == 1), and all streams associated with the circuit have had the same + * value for some field ((isolation_flags_mixed & ISO_FOO) == 0), then these + * elements hold the value for that field. + * + * Note again that "associated" is not the same as "attached": we + * preliminarily associate streams with a circuit while the circuit is being + * launched, so that we can tell whether we need to launch more circuits. + * + * @{ + */ + uint8_t client_proto_type; + uint8_t client_proto_socksver; + uint16_t dest_port; + tor_addr_t client_addr; + char *dest_address; + int session_group; + unsigned nym_epoch; + size_t socks_username_len; + uint8_t socks_password_len; + /* Note that the next two values are NOT NUL-terminated; see + socks_username_len and socks_password_len for their lengths. */ + char *socks_username; + char *socks_password; + /** Global identifier for the first stream attached here; used by + * ISO_STREAM. */ + uint64_t associated_isolated_stream_global_id; + /**@}*/ + } origin_circuit_t; /** An or_circuit_t holds information needed to implement a circuit at an @@ -2277,19 +2785,19 @@ typedef struct or_circuit_t { edge_connection_t *resolving_streams; /** The cipher used by intermediate hops for cells heading toward the * OP. */ - crypto_cipher_env_t *p_crypto; + crypto_cipher_t *p_crypto; /** The cipher used by intermediate hops for cells heading away from * the OP. */ - crypto_cipher_env_t *n_crypto; + crypto_cipher_t *n_crypto; /** The integrity-checking digest used by intermediate hops, for * cells packaged here and heading towards the OP. */ - crypto_digest_env_t *p_digest; + crypto_digest_t *p_digest; /** The integrity-checking digest used by intermediate hops, for * cells packaged at the OP and arriving here. */ - crypto_digest_env_t *n_digest; + crypto_digest_t *n_digest; /** Points to spliced circuit if purpose is REND_ESTABLISHED, and circuit * is not marked for close. */ @@ -2366,11 +2874,86 @@ typedef enum invalid_router_usage_t { #define MIN_CONSTRAINED_TCP_BUFFER 2048 #define MAX_CONSTRAINED_TCP_BUFFER 262144 /* 256k */ +/** @name Isolation flags + + Ways to isolate client streams + + @{ +*/ +/** Isolate based on destination port */ +#define ISO_DESTPORT (1u<<0) +/** Isolate based on destination address */ +#define ISO_DESTADDR (1u<<1) +/** Isolate based on SOCKS authentication */ +#define ISO_SOCKSAUTH (1u<<2) +/** Isolate based on client protocol choice */ +#define ISO_CLIENTPROTO (1u<<3) +/** Isolate based on client address */ +#define ISO_CLIENTADDR (1u<<4) +/** Isolate based on session group (always on). */ +#define ISO_SESSIONGRP (1u<<5) +/** Isolate based on newnym epoch (always on). */ +#define ISO_NYM_EPOCH (1u<<6) +/** Isolate all streams (Internal only). */ +#define ISO_STREAM (1u<<7) +/**@}*/ + +/** Default isolation level for ports. */ +#define ISO_DEFAULT (ISO_CLIENTADDR|ISO_SOCKSAUTH|ISO_SESSIONGRP|ISO_NYM_EPOCH) + +/** Indicates that we haven't yet set a session group on a port_cfg_t. */ +#define SESSION_GROUP_UNSET -1 +/** Session group reserved for directory connections */ +#define SESSION_GROUP_DIRCONN -2 +/** Session group reserved for resolve requests launched by a controller */ +#define SESSION_GROUP_CONTROL_RESOLVE -3 +/** First automatically allocated session group number */ +#define SESSION_GROUP_FIRST_AUTO -4 + +/** Configuration for a single port that we're listening on. */ +typedef struct port_cfg_t { + tor_addr_t addr; /**< The actual IP to listen on, if !is_unix_addr. */ + int port; /**< The configured port, or CFG_AUTO_PORT to tell Tor to pick its + * own port. */ + uint8_t type; /**< One of CONN_TYPE_*_LISTENER */ + unsigned is_unix_addr : 1; /**< True iff this is an AF_UNIX address. */ + + /* Client port types (socks, dns, trans, natd) only: */ + uint8_t isolation_flags; /**< Zero or more isolation flags */ + int session_group; /**< A session group, or -1 if this port is not in a + * session group. */ + + /* Server port types (or, dir) only: */ + unsigned int no_advertise : 1; + unsigned int no_listen : 1; + unsigned int all_addrs : 1; + unsigned int ipv4_only : 1; + unsigned int ipv6_only : 1; + + /* Unix sockets only: */ + /** Path for an AF_UNIX address */ + char unix_addr[FLEXIBLE_ARRAY_MEMBER]; +} port_cfg_t; + +/** Ordinary configuration line. */ +#define CONFIG_LINE_NORMAL 0 +/** Appends to previous configuration for the same option, even if we + * would ordinary replace it. */ +#define CONFIG_LINE_APPEND 1 +/* Removes all previous configuration for an option. */ +#define CONFIG_LINE_CLEAR 2 + /** A linked list of lines in a config file. */ typedef struct config_line_t { char *key; char *value; struct config_line_t *next; + /** What special treatment (if any) does this line require? */ + unsigned int command:2; + /** If true, subsequent assignments to this linelist should replace + * it, not extend it. Set only on the first item in a linelist in an + * or_options_t. */ + unsigned int fragile:1; } config_line_t; typedef struct routerset_t routerset_t; @@ -2392,6 +2975,7 @@ typedef struct { config_line_t *Logs; /**< New-style list of configuration lines * for logs */ + int LogTimeGranularity; /**< Log resolution in milliseconds. */ int LogMessageDomains; /**< Boolean: Should we log the domain(s) in which * each log message occurs? */ @@ -2402,6 +2986,8 @@ typedef struct { char *Address; /**< OR only: configured address for this onion router. */ char *PidFile; /**< Where to store PID of Tor process. */ + int DynamicDHGroups; /**< Dynamic generation of prime moduli for use in DH.*/ + routerset_t *ExitNodes; /**< Structure containing nicknames, digests, * country codes and IP address patterns of ORs to * consider as exits. */ @@ -2455,21 +3041,50 @@ typedef struct { config_line_t *RecommendedVersions; config_line_t *RecommendedClientVersions; config_line_t *RecommendedServerVersions; - /** Whether dirservers refuse router descriptors with private IPs. */ + /** Whether dirservers allow router descriptors with private IPs. */ int DirAllowPrivateAddresses; + /** Whether routers accept EXTEND cells to routers with private IPs. */ + int ExtendAllowPrivateAddresses; char *User; /**< Name of user to run Tor as. */ char *Group; /**< Name of group to run Tor as. */ - int ORPort; /**< Port to listen on for OR connections. */ - int SocksPort; /**< Port to listen on for SOCKS connections. */ - /** Port to listen on for transparent pf/netfilter connections. */ - int TransPort; - int NATDPort; /**< Port to listen on for transparent natd connections. */ - int ControlPort; /**< Port to listen on for control connections. */ + config_line_t *ORPort_lines; /**< Ports to listen on for OR connections. */ + /** Ports to listen on for SOCKS connections. */ + config_line_t *SocksPort_lines; + /** Ports to listen on for transparent pf/netfilter connections. */ + config_line_t *TransPort_lines; + config_line_t *NATDPort_lines; /**< Ports to listen on for transparent natd + * connections. */ + config_line_t *ControlPort_lines; /**< Ports to listen on for control + * connections. */ config_line_t *ControlSocket; /**< List of Unix Domain Sockets to listen on * for control connections. */ + int ControlSocketsGroupWritable; /**< Boolean: Are control sockets g+rw? */ - int DirPort; /**< Port to listen on for directory connections. */ - int DNSPort; /**< Port to listen on for DNS requests. */ + /** Ports to listen on for directory connections. */ + config_line_t *DirPort_lines; + config_line_t *DNSPort_lines; /**< Ports to listen on for DNS requests. */ + + uint64_t MaxMemInCellQueues; /**< If we have more memory than this allocated + * for circuit cell queues, run the OOM handler + */ + + /** @name port booleans + * + * Derived booleans: True iff there is a non-listener port on an AF_INET or + * AF_INET6 address of the given type configured in one of the _lines + * options above. + * + * @{ + */ + unsigned int ORPort_set : 1; + unsigned int SocksPort_set : 1; + unsigned int TransPort_set : 1; + unsigned int NATDPort_set : 1; + unsigned int ControlPort_set : 1; + unsigned int DirPort_set : 1; + unsigned int DNSPort_set : 1; + /**@}*/ + int AssumeReachable; /**< Whether to publish our descriptor regardless. */ int AuthoritativeDir; /**< Boolean: is this an authoritative directory? */ int V1AuthoritativeDir; /**< Boolean: is this an authoritative directory @@ -2498,6 +3113,12 @@ typedef struct { int UseBridges; /**< Boolean: should we start all circuits with a bridge? */ config_line_t *Bridges; /**< List of bootstrap bridge addresses. */ + config_line_t *ClientTransportPlugin; /**< List of client + transport plugins. */ + + config_line_t *ServerTransportPlugin; /**< List of client + transport plugins. */ + int BridgeRelay; /**< Boolean: are we acting as a bridge relay? We make * this explicit so we can change how we behave in the * future. */ @@ -2512,8 +3133,8 @@ typedef struct { /** To what authority types do we publish our descriptor? Choices are * "v1", "v2", "v3", "bridge", or "". */ smartlist_t *PublishServerDescriptor; - /** An authority type, derived from PublishServerDescriptor. */ - authority_type_t _PublishServerDescriptor; + /** A bitfield of authority types, derived from PublishServerDescriptor. */ + dirinfo_type_t _PublishServerDescriptor; /** Boolean: do we publish hidden service descriptors to the HS auths? */ int PublishHidServDescriptors; int FetchServerDescriptors; /**< Do we fetch server descriptors as normal? */ @@ -2531,6 +3152,20 @@ typedef struct { int AllDirActionsPrivate; /**< Should every directory action be sent * through a Tor circuit? */ + /** Run in 'tor2web mode'? (I.e. only make client connections to hidden + * services, and use a single hop for all hidden-service-related + * circuits.) */ + int Tor2webMode; + + /** Close hidden service client circuits immediately when they reach + * the normal circuit-build timeout, even if they have already sent + * an INTRODUCE1 cell on its way to the service. */ + int CloseHSClientCircuitsImmediatelyOnTimeout; + + /** Close hidden-service-side rendezvous circuits immediately when + * they reach the normal circuit-build timeout. */ + int CloseHSServiceRendCircuitsImmediatelyOnTimeout; + int ConnLimit; /**< Demanded minimum number of simultaneous connections. */ int _ConnLimit; /**< Maximum allowed number of simultaneous connections. */ int RunAsDaemon; /**< If true, run in the background. (Unix only) */ @@ -2545,12 +3180,10 @@ typedef struct { uint64_t ConstrainedSockSize; /**< Size of constrained buffers. */ /** Whether we should drop exit streams from Tors that we don't know are - * relays. One of "0" (never refuse), "1" (always refuse), or "auto" (do + * relays. One of "0" (never refuse), "1" (always refuse), or "-1" (do * what the consensus says, defaulting to 'refuse' if the consensus says * nothing). */ - char *RefuseUnknownExits; - /** Parsed version of RefuseUnknownExits. -1 for auto. */ - int RefuseUnknownExits_; + int RefuseUnknownExits; /** Application ports that require all nodes in circ to have sufficient * uptime. */ @@ -2621,6 +3254,9 @@ typedef struct { * authorizations for hidden services */ char *ContactInfo; /**< Contact info to be published in the directory. */ + int HeartbeatPeriod; /**< Log heartbeat messages after this many seconds + * have passed. */ + char *HTTPProxy; /**< hostname[:port] to use as http proxy, if any. */ tor_addr_t HTTPProxyAddr; /**< Parsed IPv4 addr for http proxy, if any. */ uint16_t HTTPProxyPort; /**< Parsed port for http proxy, if any. */ @@ -2658,7 +3294,8 @@ typedef struct { char *MyFamily; /**< Declared family for this OR. */ config_line_t *NodeFamilies; /**< List of config lines for - * node families */ + * node families */ + smartlist_t *NodeFamilySets; /**< List of parsed NodeFamilies values. */ config_line_t *AuthDirBadDir; /**< Address policy for descriptors to * mark as bad dir mirrors. */ config_line_t *AuthDirBadExit; /**< Address policy for descriptors to @@ -2667,6 +3304,19 @@ typedef struct { * reject. */ config_line_t *AuthDirInvalid; /**< Address policy for descriptors to * never mark as valid. */ + /** @name AuthDir...CC + * + * Lists of country codes to mark as BadDir, BadExit, or Invalid, or to + * reject entirely. + * + * @{ + */ + smartlist_t *AuthDirBadDirCCs; + smartlist_t *AuthDirBadExitCCs; + smartlist_t *AuthDirInvalidCCs; + smartlist_t *AuthDirRejectCCs; + /**@}*/ + int AuthDirListBadDirs; /**< True iff we should list bad dirs, * and vote for all other dir mirrors as good. */ int AuthDirListBadExits; /**< True iff we should list bad exits, @@ -2679,10 +3329,6 @@ typedef struct { * number of servers per IP address shared * with an authority. */ - /** Should we assign the Guard flag to relays which would allow - * exploitation of CVE-2011-2768 against their clients? */ - int GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays; - /** If non-zero, always vote the Fast flag for any relay advertising * this amount of capacity or more. */ uint64_t AuthDirFastGuarantee; @@ -2736,6 +3382,8 @@ typedef struct { * log whether it was DNS-leaking or not? */ int HardwareAccel; /**< Boolean: Should we enable OpenSSL hardware * acceleration where available? */ + /** Token Bucket Refill resolution in milliseconds. */ + int TokenBucketRefillInterval; char *AccelName; /**< Optional hardware acceleration engine name. */ char *AccelDir; /**< Optional hardware acceleration engine search dir. */ int UseEntryGuards; /**< Boolean: Do we try to enter from a smallish number @@ -2767,10 +3415,14 @@ typedef struct { disclaimer. This allows a server administrator to show that they're running Tor and anyone visiting their server will know this without any specialized knowledge. */ + int DisableDebuggerAttachment; /**< Currently Linux only specific attempt to + disable ptrace; needs BSD testing. */ /** Boolean: if set, we start even if our resolv.conf file is missing * or broken. */ int ServerDNSAllowBrokenConfig; - + /** Boolean: if set, then even connections to private addresses will get + * rate-limited. */ + int CountPrivateBandwidth; smartlist_t *ServerDNSTestAddresses; /**< A list of addresses that definitely * should be resolvable. Used for * testing our DNS server. */ @@ -2780,6 +3432,10 @@ typedef struct { * possible. */ int PreferTunneledDirConns; /**< If true, avoid dirservers that don't * support BEGIN_DIR, when possible. */ + int PortForwarding; /**< If true, use NAT-PMP or UPnP to automatically + * forward the DirPort and ORPort on the NAT device */ + char *PortForwardingHelper; /** < Filename or full path of the port + forwarding helper executable */ int AllowNonRFC953Hostnames; /**< If true, we allow connections to hostnames * with weird characters. */ /** If true, we try resolving hostnames with weird characters. */ @@ -2817,6 +3473,9 @@ typedef struct { /** If true, the user wants us to collect statistics on port usage. */ int ExitPortStatistics; + /** If true, the user wants us to collect connection statistics. */ + int ConnDirectionStatistics; + /** If true, the user wants us to collect cell statistics. */ int CellStatistics; @@ -2913,16 +3572,55 @@ typedef struct { */ double CircuitPriorityHalflife; + /** If true, do not enable IOCP on windows with bufferevents, even if + * we think we could. */ + int DisableIOCP; + /** For testing only: will go away in 0.2.3.x. */ + int _UseFilteringSSLBufferevents; + /** Set to true if the TestingTorNetwork configuration option is set. * This is used so that options_validate() has a chance to realize that * the defaults have changed. */ int _UsingTestNetworkDefaults; + /** If 1, we try to use microdescriptors to build circuits. If 0, we don't. + * If -1, Tor decides. */ + int UseMicrodescriptors; + /** File where we should write the ControlPort. */ char *ControlPortWriteToFile; /** Should that file be group-readable? */ int ControlPortFileGroupReadable; +#define MAX_MAX_CLIENT_CIRCUITS_PENDING 1024 + /** Maximum number of non-open general-purpose origin circuits to allow at + * once. */ + int MaxClientCircuitsPending; + + /** If 1, we always send optimistic data when it's supported. If 0, we + * never use it. If -1, we do what the consensus says. */ + int OptimisticData; + + /** If 1, and we are using IOCP, we set the kernel socket SNDBUF and RCVBUF + * to 0 to try to save kernel memory and avoid the dread "Out of buffers" + * issue. */ + int UserspaceIOCPBuffers; + + /** If 1, we accept and launch no external network connections, except on + * control ports. */ + int DisableNetwork; + + /** + * Parameters for path-bias detection. + * @{ + */ + int PathBiasCircThreshold; + double PathBiasNoticeRate; + double PathBiasDisableRate; + int PathBiasScaleThreshold; + int PathBiasScaleFactor; + /** @} */ + } or_options_t; /** Persistent state for an onion router, as saved to disk. */ @@ -2949,6 +3647,8 @@ typedef struct { /** A list of Entry Guard-related configuration lines. */ config_line_t *EntryGuards; + config_line_t *TransportProxies; + /** These fields hold information on the history of bandwidth usage for * servers. The "Ends" fields hold the time when we last updated the * bandwidth usage. The "Interval" fields hold the granularity, in seconds, @@ -3001,6 +3701,8 @@ static INLINE void or_state_mark_dirty(or_state_t *state, time_t when) #define MAX_SOCKS_REPLY_LEN 1024 #define MAX_SOCKS_ADDR_LEN 256 +#define SOCKS_NO_AUTH 0x00 +#define SOCKS_USER_PASS 0x02 /** Please open a TCP connection to this addr:port. */ #define SOCKS_COMMAND_CONNECT 0x01 @@ -3020,10 +3722,17 @@ struct socks_request_t { /** Which version of SOCKS did the client use? One of "0, 4, 5" -- where * 0 means that no socks handshake ever took place, and this is just a * stub connection (e.g. see connection_ap_make_link()). */ - char socks_version; - int command; /**< What is this stream's goal? One from the above list. */ + uint8_t socks_version; + /** If using socks5 authentication, which authentication type did we + * negotiate? currently we support 0 (no authentication) and 2 + * (username/password). */ + uint8_t auth_type; + /** What is this stream's goal? One of the SOCKS_COMMAND_* values */ + uint8_t command; + /** Which kind of listener created this stream? */ + uint8_t listener_type; size_t replylen; /**< Length of <b>reply</b>. */ - char reply[MAX_SOCKS_REPLY_LEN]; /**< Write an entry into this string if + uint8_t reply[MAX_SOCKS_REPLY_LEN]; /**< Write an entry into this string if * we want to specify our own socks reply, * rather than using the default socks4 or * socks5 socks reply. We use this for the @@ -3035,10 +3744,21 @@ struct socks_request_t { unsigned int has_finished : 1; /**< Has the SOCKS handshake finished? Used to * make sure we send back a socks reply for * every connection. */ + unsigned int got_auth : 1; /**< Have we received any authentication data? */ + + /** Number of bytes in username; 0 if username is NULL */ + size_t usernamelen; + /** Number of bytes in password; 0 if password is NULL */ + uint8_t passwordlen; + /** The negotiated username value if any (for socks5), or the entire + * authentication string (for socks4). This value is NOT nul-terminated; + * see usernamelen for its length. */ + char *username; + /** The negotiated password value if any (for socks5). This value is NOT + * nul-terminated; see passwordlen for its length. */ + char *password; }; -/* all the function prototypes go here */ - /********************************* circuitbuild.c **********************/ /** How many hops does a general-purpose circuit have by default? */ @@ -3206,6 +3926,11 @@ typedef enum { /** We're remapping this address because we got a DNS resolution from a * Tor server that told us what its value was. */ ADDRMAPSRC_DNS, + + /** No remapping has occurred. This isn't a possible value for an + * addrmap_entry_t; it's used as a null value when we need to answer "Why + * did this remapping happen." */ + ADDRMAPSRC_NONE } addressmap_entry_source_t; /********************************* control.c ***************************/ @@ -3220,6 +3945,13 @@ typedef enum circuit_status_event_t { CIRC_EVENT_CLOSED = 4, } circuit_status_event_t; +/** Used to indicate the type of a CIRC_MINOR event passed to the controller. + * The various types are defined in control-spec.txt . */ +typedef enum circuit_status_minor_event_t { + CIRC_MINOR_EVENT_PURPOSE_CHANGED, + CIRC_MINOR_EVENT_CANNIBALIZED, +} circuit_status_minor_event_t; + /** Used to indicate the type of a stream event passed to the controller. * The various types are defined in control-spec.txt */ typedef enum stream_status_event_t { @@ -3451,7 +4183,7 @@ typedef enum { ADDR_POLICY_PROBABLY_ACCEPTED=1, /** Part of the address was unknown, but as far as we can tell, it was * rejected. */ - ADDR_POLICY_PROBABLY_REJECTED=2 + ADDR_POLICY_PROBABLY_REJECTED=2, } addr_policy_result_t; /********************************* rephist.c ***************************/ @@ -3472,7 +4204,7 @@ typedef enum { typedef struct rend_authorized_client_t { char *client_name; char descriptor_cookie[REND_DESC_COOKIE_LEN]; - crypto_pk_env_t *client_key; + crypto_pk_t *client_key; } rend_authorized_client_t; /** ASCII-encoded v2 hidden service descriptor. */ @@ -3481,23 +4213,99 @@ typedef struct rend_encoded_v2_service_descriptor_t { char *desc_str; /**< Descriptor string. */ } rend_encoded_v2_service_descriptor_t; +/** The maximum number of non-circuit-build-timeout failures a hidden + * service client will tolerate while trying to build a circuit to an + * introduction point. See also rend_intro_point_t.unreachable_count. */ +#define MAX_INTRO_POINT_REACHABILITY_FAILURES 5 + +/** The maximum number of distinct INTRODUCE2 cells which a hidden + * service's introduction point will receive before it begins to + * expire. + * + * XXX023 Is this number at all sane? */ +#define INTRO_POINT_LIFETIME_INTRODUCTIONS 16384 + +/** The minimum number of seconds that an introduction point will last + * before expiring due to old age. (If it receives + * INTRO_POINT_LIFETIME_INTRODUCTIONS INTRODUCE2 cells, it may expire + * sooner.) + * + * XXX023 Should this be configurable? */ +#define INTRO_POINT_LIFETIME_MIN_SECONDS 18*60*60 +/** The maximum number of seconds that an introduction point will last + * before expiring due to old age. + * + * XXX023 Should this be configurable? */ +#define INTRO_POINT_LIFETIME_MAX_SECONDS 24*60*60 + /** Introduction point information. Used both in rend_service_t (on * the service side) and in rend_service_descriptor_t (on both the * client and service side). */ typedef struct rend_intro_point_t { extend_info_t *extend_info; /**< Extend info of this introduction point. */ - crypto_pk_env_t *intro_key; /**< Introduction key that replaces the service + crypto_pk_t *intro_key; /**< Introduction key that replaces the service * key, if this descriptor is V2. */ + + /** (Client side only) Flag indicating that a timeout has occurred + * after sending an INTRODUCE cell to this intro point. After a + * timeout, an intro point should not be tried again during the same + * hidden service connection attempt, but it may be tried again + * during a future connection attempt. */ + unsigned int timed_out : 1; + + /** (Client side only) The number of times we have failed to build a + * circuit to this intro point for some reason other than our + * circuit-build timeout. See also MAX_INTRO_POINT_REACHABILITY_FAILURES. */ + unsigned int unreachable_count : 3; + + /** (Service side only) Flag indicating that this intro point was + * included in the last HS descriptor we generated. */ + unsigned int listed_in_last_desc : 1; + + /** (Service side only) Flag indicating that + * rend_service_note_removing_intro_point has been called for this + * intro point. */ + unsigned int rend_service_note_removing_intro_point_called : 1; + + /** (Service side only) A digestmap recording the INTRODUCE2 cells + * this intro point's circuit has received. Each key is the digest + * of the RSA-encrypted part of a received INTRODUCE2 cell; each + * value is a pointer to the time_t at which the cell was received. + * This digestmap is used to prevent replay attacks. */ + digestmap_t *accepted_intro_rsa_parts; + + /** (Service side only) The time at which this intro point was first + * published, or -1 if this intro point has not yet been + * published. */ + time_t time_published; + + /** (Service side only) The time at which this intro point should + * (start to) expire, or -1 if we haven't decided when this intro + * point should expire. */ + time_t time_to_expire; + + /** (Service side only) The time at which we decided that this intro + * point should start expiring, or -1 if this intro point is not yet + * expiring. + * + * This field also serves as a flag to indicate that we have decided + * to expire this intro point, in case intro_point_should_expire_now + * flaps (perhaps due to a clock jump; perhaps due to other + * weirdness, or even a (present or future) bug). */ + time_t time_expiring; } rend_intro_point_t; +#define REND_PROTOCOL_VERSION_BITMASK_WIDTH 16 + /** Information used to connect to a hidden service. Used on both the * service side and the client side. */ typedef struct rend_service_descriptor_t { - crypto_pk_env_t *pk; /**< This service's public key. */ + crypto_pk_t *pk; /**< This service's public key. */ int version; /**< Version of the descriptor format: 0 or 2. */ time_t timestamp; /**< Time when the descriptor was generated. */ - uint16_t protocols; /**< Bitmask: which rendezvous protocols are supported? - * (We allow bits '0', '1', and '2' to be set.) */ + /** Bitmask: which rendezvous protocols are supported? + * (We allow bits '0', '1', and '2' to be set.) */ + int protocols : REND_PROTOCOL_VERSION_BITMASK_WIDTH; /** List of the service's introduction points. Elements are removed if * introduction attempts fail. */ smartlist_t *intro_nodes; @@ -3539,7 +4347,7 @@ typedef struct trusted_dir_server_t { unsigned int has_accepted_serverdesc:1; /** What kind of authority is this? (Bitfield.) */ - authority_type_t type; + dirinfo_type_t type; download_status_t v2_ns_dl_status; /**< Status of downloading this server's * v2 network status. */ @@ -3585,6 +4393,8 @@ typedef struct trusted_dir_server_t { * fetches to _any_ single directory server.] */ #define PDS_NO_EXISTING_SERVERDESC_FETCH (1<<3) +#define PDS_NO_EXISTING_MICRODESC_FETCH (1<<4) + #define _PDS_PREFER_TUNNELED_DIR_CONNS (1<<16) /** Possible ways to weight routers when choosing one randomly. See @@ -3602,7 +4412,8 @@ typedef enum { CRN_NEED_GUARD = 1<<2, CRN_ALLOW_INVALID = 1<<3, /* XXXX not used, apparently. */ - CRN_WEIGHT_AS_EXIT = 1<<5 + CRN_WEIGHT_AS_EXIT = 1<<5, + CRN_NEED_DESC = 1<<6 } router_crn_flags_t; /** Return value for router_add_to_routerlist() and dirserv_add_descriptor() */ diff --git a/src/or/policies.c b/src/or/policies.c index 55d08afc81..81e4809687 100644 --- a/src/or/policies.c +++ b/src/or/policies.c @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -11,8 +11,10 @@ #include "or.h" #include "config.h" #include "dirserv.h" +#include "nodelist.h" #include "policies.h" #include "routerparse.h" +#include "geoip.h" #include "ht.h" /** Policy that addresses for incoming SOCKS connections must match. */ @@ -73,27 +75,26 @@ policy_expand_private(smartlist_t **policy) if (!*policy) /*XXXX disallow NULL policies? */ return; - tmp = smartlist_create(); + tmp = smartlist_new(); - SMARTLIST_FOREACH(*policy, addr_policy_t *, p, - { + SMARTLIST_FOREACH_BEGIN(*policy, addr_policy_t *, p) { if (! p->is_private) { smartlist_add(tmp, p); continue; } for (i = 0; private_nets[i]; ++i) { - addr_policy_t policy; - memcpy(&policy, p, sizeof(addr_policy_t)); - policy.is_private = 0; - policy.is_canonical = 0; - if (tor_addr_parse_mask_ports(private_nets[i], &policy.addr, - &policy.maskbits, &port_min, &port_max)<0) { + addr_policy_t newpolicy; + memcpy(&newpolicy, p, sizeof(addr_policy_t)); + newpolicy.is_private = 0; + newpolicy.is_canonical = 0; + if (tor_addr_parse_mask_ports(private_nets[i], &newpolicy.addr, + &newpolicy.maskbits, &port_min, &port_max)<0) { tor_assert(0); } - smartlist_add(tmp, addr_policy_get_canonical_entry(&policy)); + smartlist_add(tmp, addr_policy_get_canonical_entry(&newpolicy)); } addr_policy_free(p); - }); + } SMARTLIST_FOREACH_END(p); smartlist_free(*policy); *policy = tmp; @@ -120,13 +121,12 @@ parse_addr_policy(config_line_t *cfg, smartlist_t **dest, if (!cfg) return 0; - result = smartlist_create(); - entries = smartlist_create(); + result = smartlist_new(); + entries = smartlist_new(); for (; cfg; cfg = cfg->next) { smartlist_split_string(entries, cfg->value, ",", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - SMARTLIST_FOREACH(entries, const char *, ent, - { + SMARTLIST_FOREACH_BEGIN(entries, const char *, ent) { log_debug(LD_CONFIG,"Adding new entry '%s'",ent); item = router_parse_addr_policy_item_from_string(ent, assume_action); if (item) { @@ -135,7 +135,7 @@ parse_addr_policy(config_line_t *cfg, smartlist_t **dest, log_warn(LD_CONFIG,"Malformed policy '%s'.", ent); r = -1; } - }); + } SMARTLIST_FOREACH_END(ent); SMARTLIST_FOREACH(entries, char *, ent, tor_free(ent)); smartlist_clear(entries); } @@ -163,7 +163,7 @@ parse_addr_policy(config_line_t *cfg, smartlist_t **dest, static int parse_reachable_addresses(void) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int ret = 0; if (options->ReachableDirAddresses && @@ -261,7 +261,7 @@ fascist_firewall_allows_address_or(const tor_addr_t *addr, uint16_t port) /** Return true iff we think our firewall will let us make an OR connection to * <b>ri</b>. */ int -fascist_firewall_allows_or(routerinfo_t *ri) +fascist_firewall_allows_or(const routerinfo_t *ri) { /* XXXX proposal 118 */ tor_addr_t addr; @@ -269,6 +269,22 @@ fascist_firewall_allows_or(routerinfo_t *ri) return fascist_firewall_allows_address_or(&addr, ri->or_port); } +/** Return true iff we think our firewall will let us make an OR connection to + * <b>node</b>. */ +int +fascist_firewall_allows_node(const node_t *node) +{ + if (node->ri) { + return fascist_firewall_allows_or(node->ri); + } else if (node->rs) { + tor_addr_t addr; + tor_addr_from_ipv4h(&addr, node->rs->addr); + return fascist_firewall_allows_address_or(&addr, node->rs->or_port); + } else { + return 1; + } +} + /** Return true iff we think our firewall will let us make a directory * connection to addr:port. */ int @@ -296,13 +312,29 @@ socks_policy_permits_address(const tor_addr_t *addr) return addr_policy_permits_tor_addr(addr, 1, socks_policy); } +/** Return true iff the address <b>addr</b> is in a country listed in the + * case-insensitive list of country codes <b>cc_list</b>. */ +static int +addr_is_in_cc_list(uint32_t addr, const smartlist_t *cc_list) +{ + country_t country; + const char *name; + if (!cc_list) + return 0; + country = geoip_get_country_by_ip(addr); + name = geoip_get_country_name(country); + return smartlist_string_isin_case(cc_list, name); +} + /** Return 1 if <b>addr</b>:<b>port</b> is permitted to publish to our * directory, based on <b>authdir_reject_policy</b>. Else return 0. */ int authdir_policy_permits_address(uint32_t addr, uint16_t port) { - return addr_policy_permits_address(addr, port, authdir_reject_policy); + if (! addr_policy_permits_address(addr, port, authdir_reject_policy)) + return 0; + return !addr_is_in_cc_list(addr, get_options()->AuthDirRejectCCs); } /** Return 1 if <b>addr</b>:<b>port</b> is considered valid in our @@ -311,7 +343,9 @@ authdir_policy_permits_address(uint32_t addr, uint16_t port) int authdir_policy_valid_address(uint32_t addr, uint16_t port) { - return addr_policy_permits_address(addr, port, authdir_invalid_policy); + if (! addr_policy_permits_address(addr, port, authdir_invalid_policy)) + return 0; + return !addr_is_in_cc_list(addr, get_options()->AuthDirInvalidCCs); } /** Return 1 if <b>addr</b>:<b>port</b> should be marked as a bad dir, @@ -320,7 +354,9 @@ authdir_policy_valid_address(uint32_t addr, uint16_t port) int authdir_policy_baddir_address(uint32_t addr, uint16_t port) { - return ! addr_policy_permits_address(addr, port, authdir_baddir_policy); + if (! addr_policy_permits_address(addr, port, authdir_baddir_policy)) + return 1; + return addr_is_in_cc_list(addr, get_options()->AuthDirBadDirCCs); } /** Return 1 if <b>addr</b>:<b>port</b> should be marked as a bad exit, @@ -329,7 +365,9 @@ authdir_policy_baddir_address(uint32_t addr, uint16_t port) int authdir_policy_badexit_address(uint32_t addr, uint16_t port) { - return ! addr_policy_permits_address(addr, port, authdir_badexit_policy); + if (! addr_policy_permits_address(addr, port, authdir_badexit_policy)) + return 1; + return addr_is_in_cc_list(addr, get_options()->AuthDirBadExitCCs); } #define REJECT(arg) \ @@ -339,7 +377,7 @@ authdir_policy_badexit_address(uint32_t addr, uint16_t port) * options in <b>options</b>, return -1 and set <b>msg</b> to a newly * allocated description of the error. Else return 0. */ int -validate_addr_policies(or_options_t *options, char **msg) +validate_addr_policies(const or_options_t *options, char **msg) { /* XXXX Maybe merge this into parse_policies_from_options, to make sure * that the two can't go out of sync. */ @@ -423,7 +461,7 @@ load_policy_from_option(config_line_t *config, smartlist_t **policy, /** Set all policies based on <b>options</b>, which should have been validated * first by validate_addr_policies. */ int -policies_parse_from_options(or_options_t *options) +policies_parse_from_options(const or_options_t *options) { int ret = 0; if (load_policy_from_option(options->SocksPolicy, &socks_policy, -1) < 0) @@ -495,6 +533,7 @@ typedef struct policy_map_ent_t { addr_policy_t *policy; } policy_map_ent_t; +/* DOCDOC policy_root */ static HT_HEAD(policy_map, policy_map_ent_t) policy_root = HT_INITIALIZER(); /** Return true iff a and b are equal. */ @@ -553,18 +592,6 @@ addr_policy_get_canonical_entry(addr_policy_t *e) return found->policy; } -/** As compare_tor_addr_to_addr_policy, but instead of a tor_addr_t, takes - * in host order. */ -addr_policy_result_t -compare_addr_to_addr_policy(uint32_t addr, uint16_t port, - const smartlist_t *policy) -{ - /*XXXX deprecate this function when possible. */ - tor_addr_t a; - tor_addr_from_ipv4h(&a, addr); - return compare_tor_addr_to_addr_policy(&a, port, policy); -} - /** Helper for compare_tor_addr_to_addr_policy. Implements the case where * addr and port are both known. */ static addr_policy_result_t @@ -641,7 +668,7 @@ compare_unknown_tor_addr_to_addr_policy(uint16_t port, SMARTLIST_FOREACH_BEGIN(policy, addr_policy_t *, tmpe) { if (tmpe->prt_min <= port && port <= tmpe->prt_max) { - if (tmpe->maskbits == 0) { + if (tmpe->maskbits == 0) { /* Definitely matches, since it covers all addresses. */ if (tmpe->policy_type == ADDR_POLICY_ACCEPT) { /* If we already hit a clause that might trigger a 'reject', than we @@ -684,7 +711,7 @@ compare_tor_addr_to_addr_policy(const tor_addr_t *addr, uint16_t port, if (!policy) { /* no policy? accept all. */ return ADDR_POLICY_ACCEPTED; - } else if (tor_addr_is_null(addr)) { + } else if (addr == NULL || tor_addr_is_null(addr)) { if (port == 0) { log_info(LD_BUG, "Rejecting null address with 0 port (family %d)", addr ? tor_addr_family(addr) : -1); @@ -870,15 +897,11 @@ policies_exit_policy_append_reject_star(smartlist_t **dest) append_exit_policy_string(dest, "reject *:*"); } -/** Replace the exit policy of <b>r</b> with reject *:*. */ +/** Replace the exit policy of <b>node</b> with reject *:* */ void -policies_set_router_exitpolicy_to_reject_all(routerinfo_t *r) +policies_set_node_exitpolicy_to_reject_all(node_t *node) { - addr_policy_t *item; - addr_policy_list_free(r->exit_policy); - r->exit_policy = smartlist_create(); - item = router_parse_addr_policy_item_from_string("reject *:*", -1); - smartlist_add(r->exit_policy, item); + node->rejects_all = 1; } /** Return 1 if there is at least one /8 subnet in <b>policy</b> that @@ -891,7 +914,7 @@ exit_policy_is_general_exit_helper(smartlist_t *policy, int port) char subnet_status[256]; memset(subnet_status, 0, sizeof(subnet_status)); - SMARTLIST_FOREACH(policy, addr_policy_t *, p, { + SMARTLIST_FOREACH_BEGIN(policy, addr_policy_t *, p) { if (tor_addr_family(&p->addr) != AF_INET) continue; /* IPv4 only for now */ if (p->prt_min > port || p->prt_max < port) @@ -922,7 +945,7 @@ exit_policy_is_general_exit_helper(smartlist_t *policy, int port) subnet_status[i] = 1; } } - }); + } SMARTLIST_FOREACH_END(p); return 0; } @@ -1044,7 +1067,7 @@ policy_summary_create(void) item->reject_count = 0; item->accepted = 0; - summary = smartlist_create(); + summary = smartlist_new(); smartlist_add(summary, item); return summary; @@ -1089,7 +1112,7 @@ policy_summary_split(smartlist_t *summary, int start_at_index; int i = 0; - /* XXXX Do a binary search if run time matters */ + while (AT(i)->prt_max < prt_min) i++; if (AT(i)->prt_min != prt_min) { @@ -1198,7 +1221,7 @@ policy_summarize(smartlist_t *policy) smartlist_t *summary = policy_summary_create(); smartlist_t *accepts, *rejects; int i, last, start_prt; - size_t accepts_len, rejects_len, shorter_len, final_size; + size_t accepts_len, rejects_len; char *accepts_str = NULL, *rejects_str = NULL, *shorter_str, *result; const char *prefix; @@ -1216,8 +1239,8 @@ policy_summarize(smartlist_t *policy) */ i = 0; start_prt = 1; - accepts = smartlist_create(); - rejects = smartlist_create(); + accepts = smartlist_new(); + rejects = smartlist_new(); while (1) { last = i == smartlist_len(summary)-1; if (last || @@ -1270,21 +1293,15 @@ policy_summarize(smartlist_t *policy) tor_assert(*c == ','); *c = '\0'; - shorter_len = strlen(shorter_str); } else if (rejects_len < accepts_len) { shorter_str = rejects_str; - shorter_len = rejects_len; prefix = "reject"; } else { shorter_str = accepts_str; - shorter_len = accepts_len; prefix = "accept"; } - final_size = strlen(prefix)+1+shorter_len+1; - tor_assert(final_size <= MAX_EXITPOLICY_SUMMARY_LEN+1); - result = tor_malloc(final_size); - tor_snprintf(result, final_size, "%s %s", prefix, shorter_str); + tor_asprintf(&result, "%s %s", prefix, shorter_str); cleanup: /* cleanup */ @@ -1302,6 +1319,226 @@ policy_summarize(smartlist_t *policy) return result; } +/** Convert a summarized policy string into a short_policy_t. Return NULL + * if the string is not well-formed. */ +short_policy_t * +parse_short_policy(const char *summary) +{ + const char *orig_summary = summary; + short_policy_t *result; + int is_accept; + int n_entries; + short_policy_entry_t entries[MAX_EXITPOLICY_SUMMARY_LEN]; /* overkill */ + const char *next; + + if (!strcmpstart(summary, "accept ")) { + is_accept = 1; + summary += strlen("accept "); + } else if (!strcmpstart(summary, "reject ")) { + is_accept = 0; + summary += strlen("reject "); + } else { + log_fn(LOG_PROTOCOL_WARN, LD_DIR, "Unrecognized policy summary keyword"); + return NULL; + } + + n_entries = 0; + for ( ; *summary; summary = next) { + const char *comma = strchr(summary, ','); + unsigned low, high; + char dummy; + char ent_buf[32]; + size_t len; + + next = comma ? comma+1 : strchr(summary, '\0'); + len = comma ? (size_t)(comma - summary) : strlen(summary); + + if (n_entries == MAX_EXITPOLICY_SUMMARY_LEN) { + log_fn(LOG_PROTOCOL_WARN, LD_DIR, "Impossibly long policy summary %s", + escaped(orig_summary)); + return NULL; + } + + if (! TOR_ISDIGIT(*summary) || len > (sizeof(ent_buf)-1)) { + /* unrecognized entry format. skip it. */ + continue; + } + if (len < 1) { + /* empty; skip it. */ + /* XXX This happens to be unreachable, since if len==0, then *summary is + * ',' or '\0', and the TOR_ISDIGIT test above would have failed. */ + continue; + } + + memcpy(ent_buf, summary, len); + ent_buf[len] = '\0'; + + if (tor_sscanf(ent_buf, "%u-%u%c", &low, &high, &dummy) == 2) { + if (low<1 || low>65535 || high<1 || high>65535 || low>high) { + log_fn(LOG_PROTOCOL_WARN, LD_DIR, + "Found bad entry in policy summary %s", escaped(orig_summary)); + return NULL; + } + } else if (tor_sscanf(ent_buf, "%u%c", &low, &dummy) == 1) { + if (low<1 || low>65535) { + log_fn(LOG_PROTOCOL_WARN, LD_DIR, + "Found bad entry in policy summary %s", escaped(orig_summary)); + return NULL; + } + high = low; + } else { + log_fn(LOG_PROTOCOL_WARN, LD_DIR,"Found bad entry in policy summary %s", + escaped(orig_summary)); + return NULL; + } + + entries[n_entries].min_port = low; + entries[n_entries].max_port = high; + n_entries++; + } + + if (n_entries == 0) { + log_fn(LOG_PROTOCOL_WARN, LD_DIR, + "Found no port-range entries in summary %s", escaped(orig_summary)); + return NULL; + } + + { + size_t size = STRUCT_OFFSET(short_policy_t, entries) + + sizeof(short_policy_entry_t)*(n_entries); + result = tor_malloc_zero(size); + + tor_assert( (char*)&result->entries[n_entries-1] < ((char*)result)+size); + } + + result->is_accept = is_accept; + result->n_entries = n_entries; + memcpy(result->entries, entries, sizeof(short_policy_entry_t)*n_entries); + return result; +} + +/** Write <b>policy</b> back out into a string. Used only for unit tests + * currently. */ +char * +write_short_policy(const short_policy_t *policy) +{ + int i; + char *answer; + smartlist_t *sl = smartlist_new(); + + smartlist_add_asprintf(sl, "%s", policy->is_accept ? "accept " : "reject "); + + for (i=0; i < policy->n_entries; i++) { + const short_policy_entry_t *e = &policy->entries[i]; + if (e->min_port == e->max_port) { + smartlist_add_asprintf(sl, "%d", e->min_port); + } else { + smartlist_add_asprintf(sl, "%d-%d", e->min_port, e->max_port); + } + if (i < policy->n_entries-1) + smartlist_add(sl, tor_strdup(",")); + } + answer = smartlist_join_strings(sl, "", 0, NULL); + SMARTLIST_FOREACH(sl, char *, a, tor_free(a)); + smartlist_free(sl); + return answer; +} + +/** Release all storage held in <b>policy</b>. */ +void +short_policy_free(short_policy_t *policy) +{ + tor_free(policy); +} + +/** See whether the <b>addr</b>:<b>port</b> address is likely to be accepted + * or rejected by the summarized policy <b>policy</b>. Return values are as + * for compare_tor_addr_to_addr_policy. Unlike the regular addr_policy + * functions, requires the <b>port</b> be specified. */ +addr_policy_result_t +compare_tor_addr_to_short_policy(const tor_addr_t *addr, uint16_t port, + const short_policy_t *policy) +{ + int i; + int found_match = 0; + int accept; + + tor_assert(port != 0); + + if (addr && tor_addr_is_null(addr)) + addr = NULL; /* Unspec means 'no address at all,' in this context. */ + + if (addr && get_options()->ClientRejectInternalAddresses && + (tor_addr_is_internal(addr, 0) || tor_addr_is_loopback(addr))) + return ADDR_POLICY_REJECTED; + + for (i=0; i < policy->n_entries; ++i) { + const short_policy_entry_t *e = &policy->entries[i]; + if (e->min_port <= port && port <= e->max_port) { + found_match = 1; + break; + } + } + + if (found_match) + accept = policy->is_accept; + else + accept = ! policy->is_accept; + + /* ???? are these right? -NM */ + /* We should be sure not to return ADDR_POLICY_ACCEPTED in the accept + * case here, because it would cause clients to believe that the node + * allows exit enclaving. Trying it anyway would open up a cool attack + * where the node refuses due to exitpolicy, the client reacts in + * surprise by rewriting the node's exitpolicy to reject *:*, and then + * a bad guy targets users by causing them to attempt such connections + * to 98% of the exits. + * + * Once microdescriptors can handle addresses in special cases (e.g. if + * we ever solve ticket 1774), we can provide certainty here. -RD */ + if (accept) + return ADDR_POLICY_PROBABLY_ACCEPTED; + else + return ADDR_POLICY_REJECTED; +} + +/** Return true iff <b>policy</b> seems reject all ports */ +int +short_policy_is_reject_star(const short_policy_t *policy) +{ + /* This doesn't need to be as much on the lookout as policy_is_reject_star, + * since policy summaries are from the consensus or from consensus + * microdescs. + */ + tor_assert(policy); + /* Check for an exact match of "reject 1-65535". */ + return (policy->is_accept == 0 && policy->n_entries == 1 && + policy->entries[0].min_port == 1 && + policy->entries[0].max_port == 65535); +} + +/** Decides whether addr:port is probably or definitely accepted or rejcted by + * <b>node</b>. See compare_tor_addr_to_addr_policy for details on addr/port + * interpretation. */ +addr_policy_result_t +compare_tor_addr_to_node_policy(const tor_addr_t *addr, uint16_t port, + const node_t *node) +{ + if (node->rejects_all) + return ADDR_POLICY_REJECTED; + + if (node->ri) + return compare_tor_addr_to_addr_policy(addr, port, node->ri->exit_policy); + else if (node->md) { + if (node->md->exit_policy == NULL) + return ADDR_POLICY_REJECTED; + else + return compare_tor_addr_to_short_policy(addr, port, + node->md->exit_policy); + } else + return ADDR_POLICY_PROBABLY_REJECTED; +} + /** Implementation for GETINFO control command: knows the answer for questions * about "exit-policy/..." */ int diff --git a/src/or/policies.h b/src/or/policies.h index b2947c67e7..f00d8299b8 100644 --- a/src/or/policies.h +++ b/src/or/policies.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -19,7 +19,8 @@ int firewall_is_fascist_or(void); int fascist_firewall_allows_address_or(const tor_addr_t *addr, uint16_t port); -int fascist_firewall_allows_or(routerinfo_t *ri); +int fascist_firewall_allows_or(const routerinfo_t *ri); +int fascist_firewall_allows_node(const node_t *node); int fascist_firewall_allows_address_dir(const tor_addr_t *addr, uint16_t port); int dir_policy_permits_address(const tor_addr_t *addr); int socks_policy_permits_address(const tor_addr_t *addr); @@ -28,21 +29,23 @@ int authdir_policy_valid_address(uint32_t addr, uint16_t port); int authdir_policy_baddir_address(uint32_t addr, uint16_t port); int authdir_policy_badexit_address(uint32_t addr, uint16_t port); -int validate_addr_policies(or_options_t *options, char **msg); +int validate_addr_policies(const or_options_t *options, char **msg); void policy_expand_private(smartlist_t **policy); -int policies_parse_from_options(or_options_t *options); +int policies_parse_from_options(const or_options_t *options); addr_policy_t *addr_policy_get_canonical_entry(addr_policy_t *ent); int cmp_addr_policies(smartlist_t *a, smartlist_t *b); addr_policy_result_t compare_tor_addr_to_addr_policy(const tor_addr_t *addr, uint16_t port, const smartlist_t *policy); -addr_policy_result_t compare_addr_to_addr_policy(uint32_t addr, - uint16_t port, const smartlist_t *policy); + +addr_policy_result_t compare_tor_addr_to_node_policy(const tor_addr_t *addr, + uint16_t port, const node_t *node); + int policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, int rejectprivate, const char *local_address, int add_default_policy); void policies_exit_policy_append_reject_star(smartlist_t **dest); -void policies_set_router_exitpolicy_to_reject_all(routerinfo_t *exitrouter); +void policies_set_node_exitpolicy_to_reject_all(node_t *exitrouter); int exit_policy_is_general_exit(smartlist_t *policy); int policy_is_reject_star(const smartlist_t *policy); int getinfo_helper_policies(control_connection_t *conn, @@ -57,5 +60,13 @@ void policies_free_all(void); char *policy_summarize(smartlist_t *policy); +short_policy_t *parse_short_policy(const char *summary); +char *write_short_policy(const short_policy_t *policy); +void short_policy_free(short_policy_t *policy); +int short_policy_is_reject_star(const short_policy_t *policy); +addr_policy_result_t compare_tor_addr_to_short_policy( + const tor_addr_t *addr, uint16_t port, + const short_policy_t *policy); + #endif diff --git a/src/or/reasons.c b/src/or/reasons.c index 319e6c055a..c51d8ee6f5 100644 --- a/src/or/reasons.c +++ b/src/or/reasons.c @@ -1,5 +1,5 @@ /* Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -143,7 +143,7 @@ stream_end_reason_to_socks5_response(int reason) * E_CASE is for errors where windows has both a EFOO and a WSAEFOO * version, and S_CASE is for errors where windows has only a WSAEFOO * version. (The E is for 'error', the S is for 'socket'). */ -#ifdef MS_WINDOWS +#ifdef _WIN32 #define E_CASE(s) case s: case WSA ## s #define S_CASE(s) case WSA ## s #else @@ -184,6 +184,8 @@ errno_to_stream_end_reason(int e) S_CASE(ENOBUFS): case ENOMEM: case ENFILE: + S_CASE(EADDRINUSE): + S_CASE(EADDRNOTAVAIL): E_CASE(EMFILE): return END_STREAM_REASON_RESOURCELIMIT; default: diff --git a/src/or/reasons.h b/src/or/reasons.h index 01f9717948..377b61b113 100644 --- a/src/or/reasons.h +++ b/src/or/reasons.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** diff --git a/src/or/relay.c b/src/or/relay.c index b637fadf59..fda9e89ca4 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -11,6 +11,7 @@ **/ #include <math.h> +#define RELAY_PRIVATE #include "or.h" #include "buffers.h" #include "circuitbuild.h" @@ -24,6 +25,7 @@ #include "main.h" #include "mempool.h" #include "networkstatus.h" +#include "nodelist.h" #include "policies.h" #include "reasons.h" #include "relay.h" @@ -32,9 +34,6 @@ #include "routerlist.h" #include "routerparse.h" -static int relay_crypt(circuit_t *circ, cell_t *cell, - cell_direction_t cell_direction, - crypt_path_t **layer_hint, char *recognized); static edge_connection_t *relay_lookup_conn(circuit_t *circ, cell_t *cell, cell_direction_t cell_direction, crypt_path_t *layer_hint); @@ -53,11 +52,6 @@ static int circuit_consider_stop_edge_reading(circuit_t *circ, crypt_path_t *layer_hint); static int circuit_queue_streams_are_blocked(circuit_t *circ); -/* XXXX023 move this all to compat_libevent */ -/** Cache the current hi-res time; the cache gets reset when libevent - * calls us. */ -static struct timeval cached_time_hires = {0, 0}; - /** Stop reading on edge connections when we have this many cells * waiting on the appropriate queue. */ #define CELL_QUEUE_HIGHWATER_SIZE 256 @@ -65,21 +59,6 @@ static struct timeval cached_time_hires = {0, 0}; * cells. */ #define CELL_QUEUE_LOWWATER_SIZE 64 -static void -tor_gettimeofday_cached(struct timeval *tv) -{ - if (cached_time_hires.tv_sec == 0) { - tor_gettimeofday(&cached_time_hires); - } - *tv = cached_time_hires; -} - -void -tor_gettimeofday_cache_clear(void) -{ - cached_time_hires.tv_sec = 0; -} - /** Stats: how many relay cells have originated at this hop, or have * been relayed onward (not recognized at this hop)? */ @@ -93,7 +72,7 @@ uint64_t stats_n_relay_cells_delivered = 0; * cell. */ static void -relay_set_digest(crypto_digest_env_t *digest, cell_t *cell) +relay_set_digest(crypto_digest_t *digest, cell_t *cell) { char integrity[4]; relay_header_t rh; @@ -114,11 +93,11 @@ relay_set_digest(crypto_digest_env_t *digest, cell_t *cell) * and cell to their original state and return 0. */ static int -relay_digest_matches(crypto_digest_env_t *digest, cell_t *cell) +relay_digest_matches(crypto_digest_t *digest, cell_t *cell) { char received_integrity[4], calculated_integrity[4]; relay_header_t rh; - crypto_digest_env_t *backup_digest=NULL; + crypto_digest_t *backup_digest=NULL; backup_digest = crypto_digest_dup(digest); @@ -142,10 +121,10 @@ relay_digest_matches(crypto_digest_env_t *digest, cell_t *cell) /* restore the relay header */ memcpy(rh.integrity, received_integrity, 4); relay_header_pack(cell->payload, &rh); - crypto_free_digest_env(backup_digest); + crypto_digest_free(backup_digest); return 0; } - crypto_free_digest_env(backup_digest); + crypto_digest_free(backup_digest); return 1; } @@ -157,7 +136,7 @@ relay_digest_matches(crypto_digest_env_t *digest, cell_t *cell) * Return -1 if the crypto fails, else return 0. */ static int -relay_crypt_one_payload(crypto_cipher_env_t *cipher, uint8_t *in, +relay_crypt_one_payload(crypto_cipher_t *cipher, uint8_t *in, int encrypt_mode) { int r; @@ -296,7 +275,7 @@ circuit_receive_relay_cell(cell_t *cell, circuit_t *circ, * Return -1 to indicate that we should mark the circuit for close, * else return 0. */ -static int +int relay_crypt(circuit_t *circ, cell_t *cell, cell_direction_t cell_direction, crypt_path_t **layer_hint, char *recognized) { @@ -608,7 +587,7 @@ relay_send_command_from_edge(streamid_t stream_id, circuit_t *circ, /* If no RELAY_EARLY cells can be sent over this circuit, log which * commands have been sent as RELAY_EARLY cells before; helps debug * task 878. */ - smartlist_t *commands_list = smartlist_create(); + smartlist_t *commands_list = smartlist_new(); int i = 0; char *commands = NULL; for (; i < origin_circ->relay_early_cells_sent; i++) @@ -648,6 +627,7 @@ connection_edge_send_command(edge_connection_t *fromconn, { /* XXXX NM Split this function into a separate versions per circuit type? */ circuit_t *circ; + crypt_path_t *cpath_layer = fromconn->cpath_layer; tor_assert(fromconn); circ = fromconn->on_circuit; @@ -662,7 +642,8 @@ connection_edge_send_command(edge_connection_t *fromconn, if (!circ) { if (fromconn->_base.type == CONN_TYPE_AP) { log_info(LD_APP,"no circ. Closing conn."); - connection_mark_unattached_ap(fromconn, END_STREAM_REASON_INTERNAL); + connection_mark_unattached_ap(EDGE_TO_ENTRY_CONN(fromconn), + END_STREAM_REASON_INTERNAL); } else { log_info(LD_EXIT,"no circ. Closing conn."); fromconn->edge_has_sent_end = 1; /* no circ to send to */ @@ -674,7 +655,7 @@ connection_edge_send_command(edge_connection_t *fromconn, return relay_send_command_from_edge(fromconn->stream_id, circ, relay_command, payload, - payload_len, fromconn->cpath_layer); + payload_len, cpath_layer); } /** How many times will I retry a stream that fails due to DNS @@ -702,22 +683,24 @@ edge_reason_is_retriable(int reason) static int connection_ap_process_end_not_open( relay_header_t *rh, cell_t *cell, origin_circuit_t *circ, - edge_connection_t *conn, crypt_path_t *layer_hint) + entry_connection_t *conn, crypt_path_t *layer_hint) { struct in_addr in; - routerinfo_t *exitrouter; + node_t *exitrouter; int reason = *(cell->payload+RELAY_HEADER_SIZE); int control_reason = reason | END_STREAM_REASON_FLAG_REMOTE; + edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(conn); (void) layer_hint; /* unused */ if (rh->length > 0 && edge_reason_is_retriable(reason) && - !connection_edge_is_rendezvous_stream(conn) /* avoid retry if rend */ - ) { + /* avoid retry if rend */ + !connection_edge_is_rendezvous_stream(edge_conn)) { + const char *chosen_exit_digest = + circ->build_state->chosen_exit->identity_digest; log_info(LD_APP,"Address '%s' refused due to '%s'. Considering retrying.", safe_str(conn->socks_request->address), stream_end_reason_to_string(reason)); - exitrouter = - router_get_by_digest(circ->build_state->chosen_exit->identity_digest); + exitrouter = node_get_mutable_by_id(chosen_exit_digest); switch (reason) { case END_STREAM_REASON_EXITPOLICY: if (rh->length >= 5) { @@ -752,13 +735,13 @@ connection_ap_process_end_not_open( log_info(LD_APP, "Exitrouter %s seems to be more restrictive than its exit " "policy. Not using this router as exit for now.", - router_describe(exitrouter)); - policies_set_router_exitpolicy_to_reject_all(exitrouter); + node_describe(exitrouter)); + policies_set_node_exitpolicy_to_reject_all(exitrouter); } /* rewrite it to an IP if we learned one. */ if (addressmap_rewrite(conn->socks_request->address, sizeof(conn->socks_request->address), - NULL)) { + NULL, NULL)) { control_event_stream_status(conn, STREAM_EVENT_REMAP, 0); } if (conn->chosen_exit_optional || @@ -793,7 +776,7 @@ connection_ap_process_end_not_open( /* We haven't retried too many times; reattach the connection. */ circuit_log_path(LOG_INFO,LD_APP,circ); /* Mark this circuit "unusable for new streams". */ - /* XXXX023 this is a kludgy way to do this. */ + /* XXXX024 this is a kludgy way to do this. */ tor_assert(circ->_base.timestamp_dirty); circ->_base.timestamp_dirty -= get_options()->MaxCircuitDirtiness; @@ -818,7 +801,7 @@ connection_ap_process_end_not_open( case END_STREAM_REASON_HIBERNATING: case END_STREAM_REASON_RESOURCELIMIT: if (exitrouter) { - policies_set_router_exitpolicy_to_reject_all(exitrouter); + policies_set_node_exitpolicy_to_reject_all(exitrouter); } if (conn->chosen_exit_optional) { /* stop wanting a specific exit */ @@ -838,7 +821,7 @@ connection_ap_process_end_not_open( stream_end_reason_to_string(rh->length > 0 ? reason : -1)); circuit_log_path(LOG_INFO,LD_APP,circ); /* need to test because of detach_retriable */ - if (!conn->_base.marked_for_close) + if (!ENTRY_TO_CONN(conn)->marked_for_close) connection_mark_unattached_ap(conn, control_reason); return 0; } @@ -847,7 +830,7 @@ connection_ap_process_end_not_open( * dotted-quad representation of <b>new_addr</b> (given in host order), * and send an appropriate REMAP event. */ static void -remap_event_helper(edge_connection_t *conn, uint32_t new_addr) +remap_event_helper(entry_connection_t *conn, uint32_t new_addr) { struct in_addr in; @@ -873,7 +856,8 @@ connection_edge_process_relay_cell_not_open( if (rh->command == RELAY_COMMAND_END) { if (CIRCUIT_IS_ORIGIN(circ) && conn->_base.type == CONN_TYPE_AP) { return connection_ap_process_end_not_open(rh, cell, - TO_ORIGIN_CIRCUIT(circ), conn, + TO_ORIGIN_CIRCUIT(circ), + EDGE_TO_ENTRY_CONN(conn), layer_hint); } else { /* we just got an 'end', don't need to send one */ @@ -887,6 +871,7 @@ connection_edge_process_relay_cell_not_open( if (conn->_base.type == CONN_TYPE_AP && rh->command == RELAY_COMMAND_CONNECTED) { + entry_connection_t *entry_conn = EDGE_TO_ENTRY_CONN(conn); tor_assert(CIRCUIT_IS_ORIGIN(circ)); if (conn->_base.state != AP_CONN_STATE_CONNECT_WAIT) { log_fn(LOG_PROTOCOL_WARN, LD_APP, @@ -901,29 +886,27 @@ connection_edge_process_relay_cell_not_open( int ttl; if (!addr || (get_options()->ClientDNSRejectInternalAddresses && is_internal_IP(addr, 0))) { - char buf[INET_NTOA_BUF_LEN]; - struct in_addr a; - a.s_addr = htonl(addr); - tor_inet_ntoa(&a, buf, sizeof(buf)); - log_info(LD_APP, - "...but it claims the IP address was %s. Closing.", buf); + log_info(LD_APP, "...but it claims the IP address was %s. Closing.", + fmt_addr32(addr)); connection_edge_end(conn, END_STREAM_REASON_TORPROTOCOL); - connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); + connection_mark_unattached_ap(entry_conn, + END_STREAM_REASON_TORPROTOCOL); return 0; } if (rh->length >= 8) ttl = (int)ntohl(get_uint32(cell->payload+RELAY_HEADER_SIZE+4)); else ttl = -1; - client_dns_set_addressmap(conn->socks_request->address, addr, - conn->chosen_exit_name, ttl); + client_dns_set_addressmap(entry_conn->socks_request->address, addr, + entry_conn->chosen_exit_name, ttl); - remap_event_helper(conn, addr); + remap_event_helper(entry_conn, addr); } circuit_log_path(LOG_INFO,LD_APP,TO_ORIGIN_CIRCUIT(circ)); /* don't send a socks reply to transparent conns */ - if (!conn->socks_request->has_finished) - connection_ap_handshake_socks_reply(conn, NULL, 0, 0); + tor_assert(entry_conn->socks_request != NULL); + if (!entry_conn->socks_request->has_finished) + connection_ap_handshake_socks_reply(entry_conn, NULL, 0, 0); /* Was it a linked dir conn? If so, a dir request just started to * fetch something; this could be a bootstrap status milestone. */ @@ -946,6 +929,12 @@ connection_edge_process_relay_cell_not_open( break; } } + /* This is definitely a success, so forget about any pending data we + * had sent. */ + if (entry_conn->pending_optimistic_data) { + generic_buffer_free(entry_conn->pending_optimistic_data); + entry_conn->pending_optimistic_data = NULL; + } /* handle anything that might have queued */ if (connection_edge_package_raw_inbuf(conn, 1, NULL) < 0) { @@ -960,17 +949,18 @@ connection_edge_process_relay_cell_not_open( int ttl; int answer_len; uint8_t answer_type; + entry_connection_t *entry_conn = EDGE_TO_ENTRY_CONN(conn); if (conn->_base.state != AP_CONN_STATE_RESOLVE_WAIT) { log_fn(LOG_PROTOCOL_WARN, LD_APP, "Got a 'resolved' cell while " "not in state resolve_wait. Dropping."); return 0; } - tor_assert(SOCKS_COMMAND_IS_RESOLVE(conn->socks_request->command)); + tor_assert(SOCKS_COMMAND_IS_RESOLVE(entry_conn->socks_request->command)); answer_len = cell->payload[RELAY_HEADER_SIZE+1]; if (rh->length < 2 || answer_len+2>rh->length) { log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, "Dropping malformed 'resolved' cell"); - connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); + connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_TORPROTOCOL); return 0; } answer_type = cell->payload[RELAY_HEADER_SIZE]; @@ -983,19 +973,17 @@ connection_edge_process_relay_cell_not_open( uint32_t addr = ntohl(get_uint32(cell->payload+RELAY_HEADER_SIZE+2)); if (get_options()->ClientDNSRejectInternalAddresses && is_internal_IP(addr, 0)) { - char buf[INET_NTOA_BUF_LEN]; - struct in_addr a; - a.s_addr = htonl(addr); - tor_inet_ntoa(&a, buf, sizeof(buf)); - log_info(LD_APP,"Got a resolve with answer %s. Rejecting.", buf); - connection_ap_handshake_socks_resolved(conn, + log_info(LD_APP,"Got a resolve with answer %s. Rejecting.", + fmt_addr32(addr)); + connection_ap_handshake_socks_resolved(entry_conn, RESOLVED_TYPE_ERROR_TRANSIENT, 0, NULL, 0, TIME_MAX); - connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); + connection_mark_unattached_ap(entry_conn, + END_STREAM_REASON_TORPROTOCOL); return 0; } } - connection_ap_handshake_socks_resolved(conn, + connection_ap_handshake_socks_resolved(entry_conn, answer_type, cell->payload[RELAY_HEADER_SIZE+1], /*answer_len*/ cell->payload+RELAY_HEADER_SIZE+2, /*answer*/ @@ -1003,9 +991,9 @@ connection_edge_process_relay_cell_not_open( -1); if (answer_type == RESOLVED_TYPE_IPV4 && answer_len == 4) { uint32_t addr = ntohl(get_uint32(cell->payload+RELAY_HEADER_SIZE+2)); - remap_event_helper(conn, addr); + remap_event_helper(entry_conn, addr); } - connection_mark_unattached_ap(conn, + connection_mark_unattached_ap(entry_conn, END_STREAM_REASON_DONE | END_STREAM_REASON_FLAG_ALREADY_SOCKS_REPLIED); return 0; @@ -1039,6 +1027,9 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, relay_header_t rh; unsigned domain = layer_hint?LD_APP:LD_EXIT; int reason; + int optimistic_data = 0; /* Set to 1 if we receive data on a stream + * that's in the EXIT_CONN_STATE_RESOLVING + * or EXIT_CONN_STATE_CONNECTING states. */ tor_assert(cell); tor_assert(circ); @@ -1055,12 +1046,40 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, return - END_CIRC_REASON_TORPROTOCOL; } + if (rh.stream_id == 0) { + switch (rh.command) { + case RELAY_COMMAND_BEGIN: + case RELAY_COMMAND_CONNECTED: + case RELAY_COMMAND_DATA: + case RELAY_COMMAND_END: + case RELAY_COMMAND_RESOLVE: + case RELAY_COMMAND_RESOLVED: + case RELAY_COMMAND_BEGIN_DIR: + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, "Relay command %d with zero " + "stream_id. Dropping.", (int)rh.command); + return 0; + default: + ; + } + } + /* either conn is NULL, in which case we've got a control cell, or else * conn points to the recognized stream. */ - if (conn && !connection_state_is_open(TO_CONN(conn))) - return connection_edge_process_relay_cell_not_open( - &rh, cell, circ, conn, layer_hint); + if (conn && !connection_state_is_open(TO_CONN(conn))) { + if (conn->_base.type == CONN_TYPE_EXIT && + (conn->_base.state == EXIT_CONN_STATE_CONNECTING || + conn->_base.state == EXIT_CONN_STATE_RESOLVING) && + rh.command == RELAY_COMMAND_DATA) { + /* Allow DATA cells to be delivered to an exit node in state + * EXIT_CONN_STATE_CONNECTING or EXIT_CONN_STATE_RESOLVING. + * This speeds up HTTP, for example. */ + optimistic_data = 1; + } else { + return connection_edge_process_relay_cell_not_open( + &rh, cell, circ, conn, layer_hint); + } + } switch (rh.command) { case RELAY_COMMAND_DROP: @@ -1103,8 +1122,12 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, (!layer_hint && --circ->deliver_window < 0)) { log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, "(relay data) circ deliver_window below 0. Killing."); - connection_edge_end(conn, END_STREAM_REASON_TORPROTOCOL); - connection_mark_for_close(TO_CONN(conn)); + if (conn) { + /* XXXX Do we actually need to do this? Will killing the circuit + * not send an END and mark the stream for close as appropriate? */ + connection_edge_end(conn, END_STREAM_REASON_TORPROTOCOL); + connection_mark_for_close(TO_CONN(conn)); + } return -END_CIRC_REASON_TORPROTOCOL; } log_debug(domain,"circ deliver_window now %d.", layer_hint ? @@ -1127,7 +1150,14 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, stats_n_data_bytes_received += rh.length; connection_write_to_buf((char*)(cell->payload + RELAY_HEADER_SIZE), rh.length, TO_CONN(conn)); - connection_edge_consider_sending_sendme(conn); + + if (!optimistic_data) { + /* Only send a SENDME if we're not getting optimistic data; otherwise + * a SENDME could arrive before the CONNECTED. + */ + connection_edge_consider_sending_sendme(conn); + } + return 0; case RELAY_COMMAND_END: reason = rh.length > 0 ? @@ -1142,9 +1172,13 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, conn->_base.s, stream_end_reason_to_string(reason), conn->stream_id); - if (conn->socks_request && !conn->socks_request->has_finished) - log_warn(LD_BUG, - "open stream hasn't sent socks answer yet? Closing."); + if (conn->_base.type == CONN_TYPE_AP) { + entry_connection_t *entry_conn = EDGE_TO_ENTRY_CONN(conn); + if (entry_conn->socks_request && + !entry_conn->socks_request->has_finished) + log_warn(LD_BUG, + "open stream hasn't sent socks answer yet? Closing."); + } /* We just *got* an end; no reason to send one. */ conn->edge_has_sent_end = 1; if (!conn->end_reason) @@ -1152,17 +1186,43 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, if (!conn->_base.marked_for_close) { /* only mark it if not already marked. it's possible to * get the 'end' right around when the client hangs up on us. */ - connection_mark_for_close(TO_CONN(conn)); - conn->_base.hold_open_until_flushed = 1; + connection_mark_and_flush(TO_CONN(conn)); } return 0; - case RELAY_COMMAND_EXTEND: + case RELAY_COMMAND_EXTEND: { + static uint64_t total_n_extend=0, total_nonearly=0; + total_n_extend++; if (conn) { log_fn(LOG_PROTOCOL_WARN, domain, "'extend' cell received for non-zero stream. Dropping."); return 0; } + if (cell->command != CELL_RELAY_EARLY && + !networkstatus_get_param(NULL,"AllowNonearlyExtend",0,0,1)) { +#define EARLY_WARNING_INTERVAL 3600 + static ratelim_t early_warning_limit = + RATELIM_INIT(EARLY_WARNING_INTERVAL); + char *m; + if (cell->command == CELL_RELAY) { + ++total_nonearly; + if ((m = rate_limit_log(&early_warning_limit, approx_time()))) { + double percentage = ((double)total_nonearly)/total_n_extend; + percentage *= 100; + log_fn(LOG_PROTOCOL_WARN, domain, "EXTEND cell received, " + "but not via RELAY_EARLY. Dropping.%s", m); + log_fn(LOG_PROTOCOL_WARN, domain, " (We have dropped %.02f%% of " + "all EXTEND cells for this reason)", percentage); + tor_free(m); + } + } else { + log_fn(LOG_WARN, domain, + "EXTEND cell received, in a cell with type %d! Dropping.", + cell->command); + } + return 0; + } return circuit_extend(cell, circ); + } case RELAY_COMMAND_EXTENDED: if (!layer_hint) { log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, @@ -1220,13 +1280,27 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, "'connected' received, no conn attached anymore. Ignoring."); return 0; case RELAY_COMMAND_SENDME: - if (!conn) { + if (!rh.stream_id) { if (layer_hint) { + if (layer_hint->package_window + CIRCWINDOW_INCREMENT > + CIRCWINDOW_START_MAX) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Bug/attack: unexpected sendme cell from exit relay. " + "Closing circ."); + return -END_CIRC_REASON_TORPROTOCOL; + } layer_hint->package_window += CIRCWINDOW_INCREMENT; log_debug(LD_APP,"circ-level sendme at origin, packagewindow %d.", layer_hint->package_window); circuit_resume_edge_reading(circ, layer_hint); } else { + if (circ->package_window + CIRCWINDOW_INCREMENT > + CIRCWINDOW_START_MAX) { + log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + "Bug/attack: unexpected sendme cell from client. " + "Closing circ."); + return -END_CIRC_REASON_TORPROTOCOL; + } circ->package_window += CIRCWINDOW_INCREMENT; log_debug(LD_APP, "circ-level sendme at non-origin, packagewindow %d.", @@ -1235,6 +1309,11 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, } return 0; } + if (!conn) { + log_info(domain,"sendme cell dropped, unknown stream (streamid %d).", + rh.stream_id); + return 0; + } conn->package_window += STREAMWINDOW_INCREMENT; log_debug(domain,"stream-level sendme, packagewindow now %d.", conn->package_window); @@ -1324,10 +1403,17 @@ int connection_edge_package_raw_inbuf(edge_connection_t *conn, int package_partial, int *max_cells) { - size_t amount_to_process, length; + size_t bytes_to_process, length; char payload[CELL_PAYLOAD_SIZE]; circuit_t *circ; - unsigned domain = conn->cpath_layer ? LD_APP : LD_EXIT; + const unsigned domain = conn->_base.type == CONN_TYPE_AP ? LD_APP : LD_EXIT; + int sending_from_optimistic = 0; + const int sending_optimistically = + conn->_base.type == CONN_TYPE_AP && + conn->_base.state != AP_CONN_STATE_OPEN; + entry_connection_t *entry_conn = + conn->_base.type == CONN_TYPE_AP ? EDGE_TO_ENTRY_CONN(conn) : NULL; + crypt_path_t *cpath_layer = conn->cpath_layer; tor_assert(conn); @@ -1350,7 +1436,7 @@ connection_edge_package_raw_inbuf(edge_connection_t *conn, int package_partial, return -1; } - if (circuit_consider_stop_edge_reading(circ, conn->cpath_layer)) + if (circuit_consider_stop_edge_reading(circ, cpath_layer)) return 0; if (conn->package_window <= 0) { @@ -1360,44 +1446,75 @@ connection_edge_package_raw_inbuf(edge_connection_t *conn, int package_partial, return 0; } - amount_to_process = buf_datalen(conn->_base.inbuf); + sending_from_optimistic = entry_conn && + entry_conn->sending_optimistic_data != NULL; + + if (PREDICT_UNLIKELY(sending_from_optimistic)) { + bytes_to_process = generic_buffer_len(entry_conn->sending_optimistic_data); + if (PREDICT_UNLIKELY(!bytes_to_process)) { + log_warn(LD_BUG, "sending_optimistic_data was non-NULL but empty"); + bytes_to_process = connection_get_inbuf_len(TO_CONN(conn)); + sending_from_optimistic = 0; + } + } else { + bytes_to_process = connection_get_inbuf_len(TO_CONN(conn)); + } - if (!amount_to_process) + if (!bytes_to_process) return 0; - if (!package_partial && amount_to_process < RELAY_PAYLOAD_SIZE) + if (!package_partial && bytes_to_process < RELAY_PAYLOAD_SIZE) return 0; - if (amount_to_process > RELAY_PAYLOAD_SIZE) { + if (bytes_to_process > RELAY_PAYLOAD_SIZE) { length = RELAY_PAYLOAD_SIZE; } else { - length = amount_to_process; + length = bytes_to_process; } stats_n_data_bytes_packaged += length; stats_n_data_cells_packaged += 1; - connection_fetch_from_buf(payload, length, TO_CONN(conn)); + if (PREDICT_UNLIKELY(sending_from_optimistic)) { + /* XXXX We could be more efficient here by sometimes packing + * previously-sent optimistic data in the same cell with data + * from the inbuf. */ + generic_buffer_get(entry_conn->sending_optimistic_data, payload, length); + if (!generic_buffer_len(entry_conn->sending_optimistic_data)) { + generic_buffer_free(entry_conn->sending_optimistic_data); + entry_conn->sending_optimistic_data = NULL; + } + } else { + connection_fetch_from_buf(payload, length, TO_CONN(conn)); + } log_debug(domain,"(%d) Packaging %d bytes (%d waiting).", conn->_base.s, - (int)length, (int)buf_datalen(conn->_base.inbuf)); + (int)length, (int)connection_get_inbuf_len(TO_CONN(conn))); + + if (sending_optimistically && !sending_from_optimistic) { + /* This is new optimistic data; remember it in case we need to detach and + retry */ + if (!entry_conn->pending_optimistic_data) + entry_conn->pending_optimistic_data = generic_buffer_new(); + generic_buffer_add(entry_conn->pending_optimistic_data, payload, length); + } if (connection_edge_send_command(conn, RELAY_COMMAND_DATA, payload, length) < 0 ) /* circuit got marked for close, don't continue, don't need to mark conn */ return 0; - if (!conn->cpath_layer) { /* non-rendezvous exit */ + if (!cpath_layer) { /* non-rendezvous exit */ tor_assert(circ->package_window > 0); circ->package_window--; } else { /* we're an AP, or an exit on a rendezvous circ */ - tor_assert(conn->cpath_layer->package_window > 0); - conn->cpath_layer->package_window--; + tor_assert(cpath_layer->package_window > 0); + cpath_layer->package_window--; } if (--conn->package_window <= 0) { /* is it 0 after decrement? */ connection_stop_reading(TO_CONN(conn)); log_debug(domain,"conn->package_window reached 0."); - circuit_consider_stop_edge_reading(circ, conn->cpath_layer); + circuit_consider_stop_edge_reading(circ, cpath_layer); return 0; /* don't process the inbuf any more */ } log_debug(domain,"conn->package_window is now %d",conn->package_window); @@ -1436,7 +1553,7 @@ connection_edge_consider_sending_sendme(edge_connection_t *conn) } while (conn->deliver_window <= STREAMWINDOW_START - STREAMWINDOW_INCREMENT) { - log_debug(conn->cpath_layer?LD_APP:LD_EXIT, + log_debug(conn->_base.type == CONN_TYPE_AP ?LD_APP:LD_EXIT, "Outbuf %d, Queuing stream sendme.", (int)conn->_base.outbuf_flushlen); conn->deliver_window += STREAMWINDOW_INCREMENT; @@ -1532,7 +1649,7 @@ circuit_resume_edge_reading_helper(edge_connection_t *first_conn, if (!layer_hint || conn->cpath_layer == layer_hint) { connection_start_reading(TO_CONN(conn)); - if (buf_datalen(conn->_base.inbuf) > 0) + if (connection_get_inbuf_len(TO_CONN(conn)) > 0) ++n_packaging_streams; } } @@ -1543,7 +1660,7 @@ circuit_resume_edge_reading_helper(edge_connection_t *first_conn, if (!layer_hint || conn->cpath_layer == layer_hint) { connection_start_reading(TO_CONN(conn)); - if (buf_datalen(conn->_base.inbuf) > 0) + if (connection_get_inbuf_len(TO_CONN(conn)) > 0) ++n_packaging_streams; } } @@ -1582,7 +1699,7 @@ circuit_resume_edge_reading_helper(edge_connection_t *first_conn, } /* If there's still data to read, we'll be coming back to this stream. */ - if (buf_datalen(conn->_base.inbuf)) + if (connection_get_inbuf_len(TO_CONN(conn))) ++n_streams_left; /* If the circuit won't accept any more data, return without looking @@ -1638,9 +1755,10 @@ circuit_consider_stop_edge_reading(circuit_t *circ, crypt_path_t *layer_hint) if (layer_hint->package_window <= 0) { log_debug(domain,"yes, at-origin. stopped."); for (conn = TO_ORIGIN_CIRCUIT(circ)->p_streams; conn; - conn=conn->next_stream) + conn=conn->next_stream) { if (conn->cpath_layer == layer_hint) connection_stop_reading(TO_CONN(conn)); + } return 1; } return 0; @@ -1681,7 +1799,7 @@ circuit_consider_sending_sendme(circuit_t *circ, crypt_path_t *layer_hint) #endif /** The total number of cells we have allocated from the memory pool. */ -static int total_cells_allocated = 0; +static size_t total_cells_allocated = 0; /** A memory pool to allocate packed_cell_t objects. */ static mp_pool_t *cell_pool = NULL; @@ -1753,7 +1871,7 @@ dump_cell_pool_usage(int severity) ++n_circs; } log(severity, LD_MM, "%d cells allocated on %d circuits. %d cells leaked.", - n_cells, n_circs, total_cells_allocated - n_cells); + n_cells, n_circs, (int)total_cells_allocated - n_cells); mp_pool_log_status(cell_pool, severity); } @@ -1860,6 +1978,29 @@ cell_queue_pop(cell_queue_t *queue) return cell; } +/** Return the total number of bytes used for each packed_cell in a queue. + * Approximate. */ +size_t +packed_cell_mem_cost(void) +{ + return sizeof(packed_cell_t) + MP_POOL_ITEM_OVERHEAD + + get_options()->CellStatistics ? + (sizeof(insertion_time_elem_t)+MP_POOL_ITEM_OVERHEAD) : 0; +} + +/** Check whether we've got too much space used for cells. If so, + * call the OOM handler and return 1. Otherwise, return 0. */ +static int +cell_queues_check_size(void) +{ + size_t alloc = total_cells_allocated * packed_cell_mem_cost(); + if (alloc >= get_options()->MaxMemInCellQueues) { + circuits_handle_oom(alloc); + return 1; + } + return 0; +} + /** Return a pointer to the "next_active_on_{n,p}_conn" pointer of <b>circ</b>, * depending on whether <b>conn</b> matches n_conn or p_conn. */ static INLINE circuit_t ** @@ -1988,14 +2129,18 @@ cell_ewma_get_tick(void) * has value ewma_scale_factor ** N.) */ static double ewma_scale_factor = 0.1; +/* DOCDOC ewma_enabled */ static int ewma_enabled = 0; +/*DOCDOC*/ #define EPSILON 0.00001 +/*DOCDOC*/ #define LOG_ONEHALF -0.69314718055994529 /** Adjust the global cell scale factor based on <b>options</b> */ void -cell_ewma_set_scale_factor(or_options_t *options, networkstatus_t *consensus) +cell_ewma_set_scale_factor(const or_options_t *options, + const networkstatus_t *consensus) { int32_t halflife_ms; double halflife; @@ -2246,7 +2391,7 @@ set_streams_blocked_on_circ(circuit_t *circ, or_connection_t *orconn, edge->edge_blocked_on_circ = block; } - if (!conn->read_event) { + if (!conn->read_event && !HAS_BUFFEREVENT(conn)) { /* This connection is a placeholder for something; probably a DNS * request. It can't actually stop or start reading.*/ continue; @@ -2321,13 +2466,13 @@ connection_or_flush_from_first_active_circuit(or_connection_t *conn, int max, /* Calculate the exact time that this cell has spent in the queue. */ if (get_options()->CellStatistics && !CIRCUIT_IS_ORIGIN(circ)) { - struct timeval now; + struct timeval tvnow; uint32_t flushed; uint32_t cell_waiting_time; insertion_time_queue_t *it_queue = queue->insertion_times; - tor_gettimeofday_cached(&now); - flushed = (uint32_t)((now.tv_sec % SECONDS_IN_A_DAY) * 100L + - (uint32_t)now.tv_usec / (uint32_t)10000L); + tor_gettimeofday_cached(&tvnow); + flushed = (uint32_t)((tvnow.tv_sec % SECONDS_IN_A_DAY) * 100L + + (uint32_t)tvnow.tv_usec / (uint32_t)10000L); if (!it_queue || !it_queue->first) { log_info(LD_GENERAL, "Cannot determine insertion time of cell. " "Looks like the CellStatistics option was " @@ -2373,7 +2518,7 @@ connection_or_flush_from_first_active_circuit(or_connection_t *conn, int max, tor_assert(tmp == cell_ewma); add_cell_ewma_to_conn(conn, cell_ewma); } - if (circ != conn->active_circuits) { + if (!ewma_enabled && circ != conn->active_circuits) { /* If this happens, the current circuit just got made inactive by * a call in connection_write_to_buf(). That's nothing to worry about: * circuit_make_inactive_on_conn() already advanced conn->active_circuits @@ -2410,8 +2555,10 @@ append_cell_to_circuit_queue(circuit_t *circ, or_connection_t *orconn, cell_t *cell, cell_direction_t direction, streamid_t fromstream) { + or_circuit_t *orcirc = NULL; cell_queue_t *queue; int streams_blocked; + if (circ->marked_for_close) return; @@ -2419,13 +2566,43 @@ append_cell_to_circuit_queue(circuit_t *circ, or_connection_t *orconn, queue = &circ->n_conn_cells; streams_blocked = circ->streams_blocked_on_n_conn; } else { - or_circuit_t *orcirc = TO_OR_CIRCUIT(circ); + orcirc = TO_OR_CIRCUIT(circ); queue = &orcirc->p_conn_cells; streams_blocked = circ->streams_blocked_on_p_conn; } + /* + * Disabling this for now because of a possible guard discovery attack + */ +#if 0 + /* Are we a middle circuit about to exceed ORCIRC_MAX_MIDDLE_CELLS? */ + if ((circ->n_conn != NULL) && CIRCUIT_IS_ORCIRC(circ)) { + orcirc = TO_OR_CIRCUIT(circ); + if (orcirc->p_conn) { + if (queue->n + 1 >= ORCIRC_MAX_MIDDLE_CELLS) { + /* Queueing this cell would put queue over the cap */ + log_warn(LD_CIRC, + "Got a cell exceeding the cap of %u in the %s direction " + "on middle circ ID %u; killing the circuit.", + ORCIRC_MAX_MIDDLE_CELLS, + (direction == CELL_DIRECTION_OUT) ? "n" : "p", + (direction == CELL_DIRECTION_OUT) ? + circ->n_circ_id : orcirc->p_circ_id); + circuit_mark_for_close(circ, END_CIRC_REASON_RESOURCELIMIT); + return; + } + } + } +#endif + cell_queue_append_packed_copy(queue, cell); + if (PREDICT_UNLIKELY(cell_queues_check_size())) { + /* We ran the OOM handler */ + if (circ->marked_for_close) + return; + } + /* If we have too many cells on the circuit, we should stop reading from * the edge streams for a while. */ if (!streams_blocked && queue->n >= CELL_QUEUE_HIGHWATER_SIZE) @@ -2443,7 +2620,7 @@ append_cell_to_circuit_queue(circuit_t *circ, or_connection_t *orconn, make_circuit_active_on_conn(circ, orconn); } - if (! buf_datalen(orconn->_base.outbuf)) { + if (! connection_get_outbuf_len(TO_CONN(orconn))) { /* There is no data at all waiting to be sent on the outbuf. Add a * cell, so that we can notice when it gets flushed, flushed_some can * get called, and we can start putting more data onto the buffer then. diff --git a/src/or/relay.h b/src/or/relay.h index f64752da5d..c55813b33c 100644 --- a/src/or/relay.h +++ b/src/or/relay.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -40,6 +40,7 @@ void init_cell_pool(void); void free_cell_pool(void); void clean_cell_pool(void); void dump_cell_pool_usage(int severity); +size_t packed_cell_mem_cost(void); void cell_queue_clear(cell_queue_t *queue); void cell_queue_append(cell_queue_t *queue, packed_cell_t *cell); @@ -60,11 +61,14 @@ const uint8_t *decode_address_from_payload(tor_addr_t *addr_out, const uint8_t *payload, int payload_len); unsigned cell_ewma_get_tick(void); -void cell_ewma_set_scale_factor(or_options_t *options, - networkstatus_t *consensus); +void cell_ewma_set_scale_factor(const or_options_t *options, + const networkstatus_t *consensus); void circuit_clear_cell_queue(circuit_t *circ, or_connection_t *orconn); -void tor_gettimeofday_cache_clear(void); +#ifdef RELAY_PRIVATE +int relay_crypt(circuit_t *circ, cell_t *cell, cell_direction_t cell_direction, + crypt_path_t **layer_hint, char *recognized); +#endif #endif diff --git a/src/or/rendclient.c b/src/or/rendclient.c index f951dad611..3a0cd1a666 100644 --- a/src/or/rendclient.c +++ b/src/or/rendclient.c @@ -1,5 +1,5 @@ /* Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -16,6 +16,7 @@ #include "connection_edge.h" #include "directory.h" #include "main.h" +#include "nodelist.h" #include "relay.h" #include "rendclient.h" #include "rendcommon.h" @@ -130,7 +131,7 @@ rend_client_send_introduction(origin_circuit_t *introcirc, rend_cache_entry_t *entry; crypt_path_t *cpath; off_t dh_offset; - crypto_pk_env_t *intro_key = NULL; + crypto_pk_t *intro_key = NULL; tor_assert(introcirc->_base.purpose == CIRCUIT_PURPOSE_C_INTRODUCING); tor_assert(rendcirc->_base.purpose == CIRCUIT_PURPOSE_C_REND_READY); @@ -138,6 +139,10 @@ rend_client_send_introduction(origin_circuit_t *introcirc, tor_assert(rendcirc->rend_data); tor_assert(!rend_cmp_service_ids(introcirc->rend_data->onion_address, rendcirc->rend_data->onion_address)); +#ifndef NON_ANONYMOUS_MODE_ENABLED + tor_assert(!(introcirc->build_state->onehop_tunnel)); + tor_assert(!(rendcirc->build_state->onehop_tunnel)); +#endif if (rend_cache_lookup_entry(introcirc->rend_data->onion_address, -1, &entry) < 1) { @@ -270,6 +275,12 @@ rend_client_send_introduction(origin_circuit_t *introcirc, payload_len = DIGEST_LEN + r; tor_assert(payload_len <= RELAY_PAYLOAD_SIZE); /* we overran something */ + /* Copy the rendezvous cookie from rendcirc to introcirc, so that + * when introcirc gets an ack, we can change the state of the right + * rendezvous circuit. */ + memcpy(introcirc->rend_data->rend_cookie, rendcirc->rend_data->rend_cookie, + REND_COOKIE_LEN); + log_info(LD_REND, "Sending an INTRODUCE1 cell"); if (relay_send_command_from_edge(0, TO_CIRCUIT(introcirc), RELAY_COMMAND_INTRODUCE1, @@ -281,7 +292,8 @@ rend_client_send_introduction(origin_circuit_t *introcirc, } /* Now, we wait for an ACK or NAK on this circuit. */ - introcirc->_base.purpose = CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT; + circuit_change_purpose(TO_CIRCUIT(introcirc), + CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT); /* Set timestamp_dirty, because circuit_expire_building expects it * to specify when a circuit entered the _C_INTRODUCE_ACK_WAIT * state. */ @@ -328,6 +340,9 @@ rend_client_introduction_acked(origin_circuit_t *circ, } tor_assert(circ->build_state->chosen_exit); +#ifndef NON_ANONYMOUS_MODE_ENABLED + tor_assert(!(circ->build_state->onehop_tunnel)); +#endif tor_assert(circ->rend_data); if (request_len == 0) { @@ -336,10 +351,13 @@ rend_client_introduction_acked(origin_circuit_t *circ, * and tell it. */ log_info(LD_REND,"Received ack. Telling rend circ..."); - rendcirc = circuit_get_by_rend_query_and_purpose( - circ->rend_data->onion_address, CIRCUIT_PURPOSE_C_REND_READY); + rendcirc = circuit_get_ready_rend_circ_by_rend_data(circ->rend_data); if (rendcirc) { /* remember the ack */ - rendcirc->_base.purpose = CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED; +#ifndef NON_ANONYMOUS_MODE_ENABLED + tor_assert(!(rendcirc->build_state->onehop_tunnel)); +#endif + circuit_change_purpose(TO_CIRCUIT(rendcirc), + CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED); /* Set timestamp_dirty, because circuit_expire_building expects * it to specify when a circuit entered the * _C_REND_READY_INTRO_ACKED state. */ @@ -348,11 +366,12 @@ rend_client_introduction_acked(origin_circuit_t *circ, log_info(LD_REND,"...Found no rend circ. Dropping on the floor."); } /* close the circuit: we won't need it anymore. */ - circ->_base.purpose = CIRCUIT_PURPOSE_C_INTRODUCE_ACKED; + circuit_change_purpose(TO_CIRCUIT(circ), + CIRCUIT_PURPOSE_C_INTRODUCE_ACKED); circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_FINISHED); } else { /* It's a NAK; the introduction point didn't relay our request. */ - circ->_base.purpose = CIRCUIT_PURPOSE_C_INTRODUCING; + circuit_change_purpose(TO_CIRCUIT(circ), CIRCUIT_PURPOSE_C_INTRODUCING); /* Remove this intro point from the set of viable introduction * points. If any remain, extend to a new one and try again. * If none remain, refetch the service descriptor. @@ -360,8 +379,9 @@ rend_client_introduction_acked(origin_circuit_t *circ, log_info(LD_REND, "Got nack for %s from %s...", safe_str_client(circ->rend_data->onion_address), safe_str_client(extend_info_describe(circ->build_state->chosen_exit))); - if (rend_client_remove_intro_point(circ->build_state->chosen_exit, - circ->rend_data) > 0) { + if (rend_client_report_intro_point_failure(circ->build_state->chosen_exit, + circ->rend_data, + INTRO_POINT_FAILURE_GENERIC)>0) { /* There are introduction points left. Re-extend the circuit to * another intro point and try again. */ int result = rend_client_reextend_intro_circuit(circ); @@ -378,9 +398,12 @@ rend_client_introduction_acked(origin_circuit_t *circ, #define REND_HID_SERV_DIR_REQUERY_PERIOD (15 * 60) /** Contains the last request times to hidden service directories for - * certain queries; keys are strings consisting of base32-encoded - * hidden service directory identities and base32-encoded descriptor IDs; - * values are pointers to timestamps of the last requests. */ + * certain queries; each key is a string consisting of the + * concatenation of a base32-encoded HS directory identity digest, a + * base32-encoded HS descriptor ID, and a hidden service address + * (without the ".onion" part); each value is a pointer to a time_t + * holding the time of the last request for that descriptor ID to that + * HS directory. */ static strmap_t *last_hid_serv_requests_ = NULL; /** Returns last_hid_serv_requests_, initializing it to a new strmap if @@ -393,23 +416,34 @@ get_last_hid_serv_requests(void) return last_hid_serv_requests_; } +#define LAST_HID_SERV_REQUEST_KEY_LEN (REND_DESC_ID_V2_LEN_BASE32 + \ + REND_DESC_ID_V2_LEN_BASE32 + \ + REND_SERVICE_ID_LEN_BASE32) + /** Look up the last request time to hidden service directory <b>hs_dir</b> - * for descriptor ID <b>desc_id_base32</b>. If <b>set</b> is non-zero, + * for descriptor ID <b>desc_id_base32</b> for the service specified in + * <b>rend_query</b>. If <b>set</b> is non-zero, * assign the current time <b>now</b> and return that. Otherwise, return * the most recent request time, or 0 if no such request has been sent * before. */ static time_t lookup_last_hid_serv_request(routerstatus_t *hs_dir, - const char *desc_id_base32, time_t now, int set) + const char *desc_id_base32, + const rend_data_t *rend_query, + time_t now, int set) { char hsdir_id_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; - char hsdir_desc_comb_id[2 * REND_DESC_ID_V2_LEN_BASE32 + 1]; + char hsdir_desc_comb_id[LAST_HID_SERV_REQUEST_KEY_LEN + 1]; time_t *last_request_ptr; strmap_t *last_hid_serv_requests = get_last_hid_serv_requests(); base32_encode(hsdir_id_base32, sizeof(hsdir_id_base32), hs_dir->identity_digest, DIGEST_LEN); - tor_snprintf(hsdir_desc_comb_id, sizeof(hsdir_desc_comb_id), "%s%s", - hsdir_id_base32, desc_id_base32); + tor_snprintf(hsdir_desc_comb_id, sizeof(hsdir_desc_comb_id), "%s%s%s", + hsdir_id_base32, + desc_id_base32, + rend_query->onion_address); + /* XXX023 tor_assert(strlen(hsdir_desc_comb_id) == + LAST_HID_SERV_REQUEST_KEY_LEN); */ if (set) { time_t *oldptr; last_request_ptr = tor_malloc_zero(sizeof(time_t)); @@ -427,10 +461,10 @@ lookup_last_hid_serv_request(routerstatus_t *hs_dir, * it does not contain requests older than REND_HID_SERV_DIR_REQUERY_PERIOD * seconds any more. */ static void -directory_clean_last_hid_serv_requests(void) +directory_clean_last_hid_serv_requests(time_t now) { strmap_iter_t *iter; - time_t cutoff = time(NULL) - REND_HID_SERV_DIR_REQUERY_PERIOD; + time_t cutoff = now - REND_HID_SERV_DIR_REQUERY_PERIOD; strmap_t *last_hid_serv_requests = get_last_hid_serv_requests(); for (iter = strmap_iter_init(last_hid_serv_requests); !strmap_iter_done(iter); ) { @@ -448,6 +482,33 @@ directory_clean_last_hid_serv_requests(void) } } +/** Remove all requests related to the hidden service named + * <b>onion_address</b> from the history of times of requests to + * hidden service directories. */ +static void +purge_hid_serv_from_last_hid_serv_requests(const char *onion_address) +{ + strmap_iter_t *iter; + strmap_t *last_hid_serv_requests = get_last_hid_serv_requests(); + /* XXX023 tor_assert(strlen(onion_address) == REND_SERVICE_ID_LEN_BASE32); */ + for (iter = strmap_iter_init(last_hid_serv_requests); + !strmap_iter_done(iter); ) { + const char *key; + void *val; + strmap_iter_get(iter, &key, &val); + /* XXX023 tor_assert(strlen(key) == LAST_HID_SERV_REQUEST_KEY_LEN); */ + if (tor_memeq(key + LAST_HID_SERV_REQUEST_KEY_LEN - + REND_SERVICE_ID_LEN_BASE32, + onion_address, + REND_SERVICE_ID_LEN_BASE32)) { + iter = strmap_iter_next_rmv(last_hid_serv_requests, iter); + tor_free(val); + } else { + iter = strmap_iter_next(last_hid_serv_requests, iter); + } + } +} + /** Purge the history of request times to hidden service directories, * so that future lookups of an HS descriptor will not fail because we * accessed all of the HSDir relays responsible for the descriptor @@ -469,20 +530,20 @@ rend_client_purge_last_hid_serv_requests(void) } /** Determine the responsible hidden service directories for <b>desc_id</b> - * and fetch the descriptor belonging to that ID from one of them. Only - * send a request to hidden service directories that we did not try within - * the last REND_HID_SERV_DIR_REQUERY_PERIOD seconds; on success, return 1, + * and fetch the descriptor with that ID from one of them. Only + * send a request to a hidden service directory that we have not yet tried + * during this attempt to connect to this hidden service; on success, return 1, * in the case that no hidden service directory is left to ask for the - * descriptor, return 0, and in case of a failure -1. <b>query</b> is only - * passed for pretty log statements. */ + * descriptor, return 0, and in case of a failure -1. */ static int directory_get_from_hs_dir(const char *desc_id, const rend_data_t *rend_query) { - smartlist_t *responsible_dirs = smartlist_create(); + smartlist_t *responsible_dirs = smartlist_new(); routerstatus_t *hs_dir; char desc_id_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; time_t now = time(NULL); char descriptor_cookie_base64[3*REND_DESC_COOKIE_LEN_BASE64]; + int tor2web_mode = get_options()->Tor2webMode; tor_assert(desc_id); tor_assert(rend_query); /* Determine responsible dirs. Even if we can't get all we want, @@ -494,12 +555,16 @@ directory_get_from_hs_dir(const char *desc_id, const rend_data_t *rend_query) /* Only select those hidden service directories to which we did not send * a request recently and for which we have a router descriptor here. */ - directory_clean_last_hid_serv_requests(); /* Clean request history first. */ + + /* Clean request history first. */ + directory_clean_last_hid_serv_requests(now); SMARTLIST_FOREACH(responsible_dirs, routerstatus_t *, dir, { - if (lookup_last_hid_serv_request(dir, desc_id_base32, 0, 0) + - REND_HID_SERV_DIR_REQUERY_PERIOD >= now || - !router_get_by_digest(dir->identity_digest)) + time_t last = lookup_last_hid_serv_request( + dir, desc_id_base32, rend_query, 0, 0); + const node_t *node = node_get_by_id(dir->identity_digest); + if (last + REND_HID_SERV_DIR_REQUERY_PERIOD >= now || + !node || !node_has_descriptor(node)) SMARTLIST_DEL_CURRENT(responsible_dirs, dir); }); @@ -512,9 +577,9 @@ directory_get_from_hs_dir(const char *desc_id, const rend_data_t *rend_query) return 0; } - /* Remember, that we are requesting a descriptor from this hidden service + /* Remember that we are requesting a descriptor from this hidden service * directory now. */ - lookup_last_hid_serv_request(hs_dir, desc_id_base32, now, 1); + lookup_last_hid_serv_request(hs_dir, desc_id_base32, rend_query, now, 1); /* Encode descriptor cookie for logging purposes. */ if (rend_query->auth_type != REND_NO_AUTH) { @@ -537,7 +602,8 @@ directory_get_from_hs_dir(const char *desc_id, const rend_data_t *rend_query) directory_initiate_command_routerstatus_rend(hs_dir, DIR_PURPOSE_FETCH_RENDDESC_V2, ROUTER_PURPOSE_GENERAL, - 1, desc_id_base32, NULL, 0, 0, + !tor2web_mode, desc_id_base32, + NULL, 0, 0, rend_query); log_info(LD_REND, "Sending fetch request for v2 descriptor for " "service '%s' with descriptor ID '%s', auth type %d, " @@ -568,10 +634,11 @@ rend_client_refetch_v2_renddesc(const rend_data_t *rend_query) "service descriptor, but are not fetching service descriptors."); return; } - /* Before fetching, check if we already have the descriptor here. */ - if (rend_cache_lookup_entry(rend_query->onion_address, -1, &e) > 0) { + /* Before fetching, check if we already have a usable descriptor here. */ + if (rend_cache_lookup_entry(rend_query->onion_address, -1, &e) > 0 && + rend_client_any_intro_points_usable(e)) { log_info(LD_REND, "We would fetch a v2 rendezvous descriptor, but we " - "already have that descriptor here. Not fetching."); + "already have a usable descriptor here. Not fetching."); return; } log_debug(LD_REND, "Fetching v2 rendezvous descriptor for service %s", @@ -641,16 +708,31 @@ rend_client_cancel_descriptor_fetches(void) } SMARTLIST_FOREACH_END(conn); } -/** Remove failed_intro from ent. If ent now has no intro points, or - * service is unrecognized, then launch a new renddesc fetch. - +/** Mark <b>failed_intro</b> as a failed introduction point for the + * hidden service specified by <b>rend_query</b>. If the HS now has no + * usable intro points, or we do not have an HS descriptor for it, + * then launch a new renddesc fetch. + * + * If <b>failure_type</b> is INTRO_POINT_FAILURE_GENERIC, remove the + * intro point from (our parsed copy of) the HS descriptor. * - * Return -1 if error, 0 if no intro points remain or service + * If <b>failure_type</b> is INTRO_POINT_FAILURE_TIMEOUT, mark the + * intro point as 'timed out'; it will not be retried until the + * current hidden service connection attempt has ended or it has + * appeared in a newly fetched rendezvous descriptor. + * + * If <b>failure_type</b> is INTRO_POINT_FAILURE_UNREACHABLE, + * increment the intro point's reachability-failure count; if it has + * now failed MAX_INTRO_POINT_REACHABILITY_FAILURES or more times, + * remove the intro point from (our parsed copy of) the HS descriptor. + * + * Return -1 if error, 0 if no usable intro points remain or service * unrecognized, 1 if recognized and some intro points remain. */ int -rend_client_remove_intro_point(extend_info_t *failed_intro, - const rend_data_t *rend_query) +rend_client_report_intro_point_failure(extend_info_t *failed_intro, + const rend_data_t *rend_query, + unsigned int failure_type) { int i, r; rend_cache_entry_t *ent; @@ -673,8 +755,34 @@ rend_client_remove_intro_point(extend_info_t *failed_intro, rend_intro_point_t *intro = smartlist_get(ent->parsed->intro_nodes, i); if (tor_memeq(failed_intro->identity_digest, intro->extend_info->identity_digest, DIGEST_LEN)) { - rend_intro_point_free(intro); - smartlist_del(ent->parsed->intro_nodes, i); + switch (failure_type) { + default: + log_warn(LD_BUG, "Unknown failure type %u. Removing intro point.", + failure_type); + tor_fragile_assert(); + /* fall through */ + case INTRO_POINT_FAILURE_GENERIC: + rend_intro_point_free(intro); + smartlist_del(ent->parsed->intro_nodes, i); + break; + case INTRO_POINT_FAILURE_TIMEOUT: + intro->timed_out = 1; + break; + case INTRO_POINT_FAILURE_UNREACHABLE: + ++(intro->unreachable_count); + { + int zap_intro_point = + intro->unreachable_count >= MAX_INTRO_POINT_REACHABILITY_FAILURES; + log_info(LD_REND, "Failed to reach this intro point %u times.%s", + intro->unreachable_count, + zap_intro_point ? " Removing from descriptor.": ""); + if (zap_intro_point) { + rend_intro_point_free(intro); + smartlist_del(ent->parsed->intro_nodes, i); + } + } + break; + } break; } } @@ -718,11 +826,11 @@ rend_client_rendezvous_acked(origin_circuit_t *circ, const uint8_t *request, } log_info(LD_REND,"Got rendezvous ack. This circuit is now ready for " "rendezvous."); - circ->_base.purpose = CIRCUIT_PURPOSE_C_REND_READY; + circuit_change_purpose(TO_CIRCUIT(circ), CIRCUIT_PURPOSE_C_REND_READY); /* Set timestamp_dirty, because circuit_expire_building expects it * to specify when a circuit entered the _C_REND_READY state. */ circ->_base.timestamp_dirty = time(NULL); - /* XXXX023 This is a pretty brute-force approach. It'd be better to + /* XXXX This is a pretty brute-force approach. It'd be better to * attach only the connections that are waiting on this circuit, rather * than trying to attach them all. See comments bug 743. */ /* If we already have the introduction circuit built, make sure we send @@ -782,7 +890,7 @@ rend_client_receive_rendezvous(origin_circuit_t *circ, const uint8_t *request, hop->dh_handshake_state = NULL; /* All is well. Extend the circuit. */ - circ->_base.purpose = CIRCUIT_PURPOSE_C_REND_JOINED; + circuit_change_purpose(TO_CIRCUIT(circ), CIRCUIT_PURPOSE_C_REND_JOINED); hop->state = CPATH_STATE_OPEN; /* set the windows to default. these are the windows * that alice thinks bob has. @@ -790,16 +898,20 @@ rend_client_receive_rendezvous(origin_circuit_t *circ, const uint8_t *request, hop->package_window = circuit_initial_package_window(); hop->deliver_window = CIRCWINDOW_START; + /* Now that this circuit has finished connecting to its destination, + * make sure circuit_get_open_circ_or_launch is willing to return it + * so we can actually use it. */ + circ->hs_circ_has_timed_out = 0; + onion_append_to_cpath(&circ->cpath, hop); circ->build_state->pending_final_cpath = NULL; /* prevent double-free */ - /* XXXX023 This is a pretty brute-force approach. It'd be better to - * attach only the connections that are waiting on this circuit, rather - * than trying to attach them all. See comments bug 743. */ - connection_ap_attach_pending(); - memset(keys, 0, sizeof(keys)); + + circuit_try_attaching_streams(circ); + + memwipe(keys, 0, sizeof(keys)); return 0; err: - memset(keys, 0, sizeof(keys)); + memwipe(keys, 0, sizeof(keys)); circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_TORPROTOCOL); return -1; } @@ -810,40 +922,42 @@ rend_client_receive_rendezvous(origin_circuit_t *circ, const uint8_t *request, void rend_client_desc_trynow(const char *query) { - edge_connection_t *conn; + entry_connection_t *conn; rend_cache_entry_t *entry; + const rend_data_t *rend_data; time_t now = time(NULL); smartlist_t *conns = get_connection_array(); - SMARTLIST_FOREACH_BEGIN(conns, connection_t *, _conn) { - if (_conn->type != CONN_TYPE_AP || - _conn->state != AP_CONN_STATE_RENDDESC_WAIT || - _conn->marked_for_close) + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, base_conn) { + if (base_conn->type != CONN_TYPE_AP || + base_conn->state != AP_CONN_STATE_RENDDESC_WAIT || + base_conn->marked_for_close) continue; - conn = TO_EDGE_CONN(_conn); - if (!conn->rend_data) + conn = TO_ENTRY_CONN(base_conn); + rend_data = ENTRY_TO_EDGE_CONN(conn)->rend_data; + if (!rend_data) continue; - if (rend_cmp_service_ids(query, conn->rend_data->onion_address)) + if (rend_cmp_service_ids(query, rend_data->onion_address)) continue; - assert_connection_ok(TO_CONN(conn), now); - if (rend_cache_lookup_entry(conn->rend_data->onion_address, -1, + assert_connection_ok(base_conn, now); + if (rend_cache_lookup_entry(rend_data->onion_address, -1, &entry) == 1 && rend_client_any_intro_points_usable(entry)) { /* either this fetch worked, or it failed but there was a * valid entry from before which we should reuse */ log_info(LD_REND,"Rend desc is usable. Launching circuits."); - conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; + base_conn->state = AP_CONN_STATE_CIRCUIT_WAIT; /* restart their timeout values, so they get a fair shake at * connecting to the hidden service. */ - conn->_base.timestamp_created = now; - conn->_base.timestamp_lastread = now; - conn->_base.timestamp_lastwritten = now; + base_conn->timestamp_created = now; + base_conn->timestamp_lastread = now; + base_conn->timestamp_lastwritten = now; if (connection_ap_handshake_attach_circuit(conn) < 0) { /* it will never work */ log_warn(LD_REND,"Rendezvous attempt failed. Closing."); - if (!conn->_base.marked_for_close) + if (!base_conn->marked_for_close) connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); } } else { /* 404, or fetch didn't get that far */ @@ -851,8 +965,34 @@ rend_client_desc_trynow(const char *query) "unavailable (try again later).", safe_str_client(query)); connection_mark_unattached_ap(conn, END_STREAM_REASON_RESOLVEFAILED); + rend_client_note_connection_attempt_ended(query); } - } SMARTLIST_FOREACH_END(_conn); + } SMARTLIST_FOREACH_END(base_conn); +} + +/** Clear temporary state used only during an attempt to connect to + * the hidden service named <b>onion_address</b>. Called when a + * connection attempt has ended; may be called occasionally at other + * times, and should be reasonably harmless. */ +void +rend_client_note_connection_attempt_ended(const char *onion_address) +{ + rend_cache_entry_t *cache_entry = NULL; + rend_cache_lookup_entry(onion_address, -1, &cache_entry); + + log_info(LD_REND, "Connection attempt for %s has ended; " + "cleaning up temporary state.", + safe_str_client(onion_address)); + + /* Clear the timed_out flag on all remaining intro points for this HS. */ + if (cache_entry != NULL) { + SMARTLIST_FOREACH(cache_entry->parsed->intro_nodes, + rend_intro_point_t *, ip, + ip->timed_out = 0; ); + } + + /* Remove the HS's entries in last_hid_serv_requests. */ + purge_hid_serv_from_last_hid_serv_requests(onion_address); } /** Return a newly allocated extend_info_t* for a randomly chosen introduction @@ -894,16 +1034,22 @@ rend_client_get_random_intro_impl(const rend_cache_entry_t *entry, int i; rend_intro_point_t *intro; - routerinfo_t *router; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); smartlist_t *usable_nodes; int n_excluded = 0; /* We'll keep a separate list of the usable nodes. If this becomes empty, * no nodes are usable. */ - usable_nodes = smartlist_create(); + usable_nodes = smartlist_new(); smartlist_add_all(usable_nodes, entry->parsed->intro_nodes); + /* Remove the intro points that have timed out during this HS + * connection attempt from our list of usable nodes. */ + SMARTLIST_FOREACH(usable_nodes, rend_intro_point_t *, ip, + if (ip->timed_out) { + SMARTLIST_DEL_CURRENT(usable_nodes, ip); + }); + again: if (smartlist_len(usable_nodes) == 0) { if (n_excluded && get_options()->StrictNodes && warnings) { @@ -921,21 +1067,33 @@ rend_client_get_random_intro_impl(const rend_cache_entry_t *entry, intro = smartlist_get(usable_nodes, i); /* Do we need to look up the router or is the extend info complete? */ if (!intro->extend_info->onion_key) { + const node_t *node; + extend_info_t *new_extend_info; if (tor_digest_is_zero(intro->extend_info->identity_digest)) - router = router_get_by_hexdigest(intro->extend_info->nickname); + node = node_get_by_hex_id(intro->extend_info->nickname); else - router = router_get_by_digest(intro->extend_info->identity_digest); - if (!router) { + node = node_get_by_id(intro->extend_info->identity_digest); + if (!node) { log_info(LD_REND, "Unknown router with nickname '%s'; trying another.", intro->extend_info->nickname); smartlist_del(usable_nodes, i); goto again; } - extend_info_free(intro->extend_info); - intro->extend_info = extend_info_from_router(router); + new_extend_info = extend_info_from_node(node, 0); + if (!new_extend_info) { + log_info(LD_REND, "We don't have a descriptor for the intro-point relay " + "'%s'; trying another.", + extend_info_describe(intro->extend_info)); + smartlist_del(usable_nodes, i); + goto again; + } else { + extend_info_free(intro->extend_info); + intro->extend_info = new_extend_info; + } + tor_assert(intro->extend_info != NULL); } /* Check if we should refuse to talk to this router. */ - if (options->ExcludeNodes && strict && + if (strict && routerset_contains_extendinfo(options->ExcludeNodes, intro->extend_info)) { n_excluded++; @@ -1006,12 +1164,13 @@ rend_service_authorization_free_all(void) * service and add it to the local map of hidden service authorizations. * Return 0 for success and -1 for failure. */ int -rend_parse_service_authorization(or_options_t *options, int validate_only) +rend_parse_service_authorization(const or_options_t *options, + int validate_only) { config_line_t *line; int res = -1; strmap_t *parsed = strmap_new(); - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); rend_service_authorization_t *auth = NULL; for (line = options->HidServAuth; line; line = line->next) { diff --git a/src/or/rendclient.h b/src/or/rendclient.h index c6cf82b3dd..393b556e32 100644 --- a/src/or/rendclient.h +++ b/src/or/rendclient.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -22,8 +22,15 @@ int rend_client_introduction_acked(origin_circuit_t *circ, void rend_client_refetch_v2_renddesc(const rend_data_t *rend_query); void rend_client_cancel_descriptor_fetches(void); void rend_client_purge_last_hid_serv_requests(void); -int rend_client_remove_intro_point(extend_info_t *failed_intro, - const rend_data_t *rend_query); + +#define INTRO_POINT_FAILURE_GENERIC 0 +#define INTRO_POINT_FAILURE_TIMEOUT 1 +#define INTRO_POINT_FAILURE_UNREACHABLE 2 + +int rend_client_report_intro_point_failure(extend_info_t *failed_intro, + const rend_data_t *rend_query, + unsigned int failure_type); + int rend_client_rendezvous_acked(origin_circuit_t *circ, const uint8_t *request, size_t request_len); @@ -32,12 +39,14 @@ int rend_client_receive_rendezvous(origin_circuit_t *circ, size_t request_len); void rend_client_desc_trynow(const char *query); +void rend_client_note_connection_attempt_ended(const char *onion_address); + extend_info_t *rend_client_get_random_intro(const rend_data_t *rend_query); int rend_client_any_intro_points_usable(const rend_cache_entry_t *entry); int rend_client_send_introduction(origin_circuit_t *introcirc, origin_circuit_t *rendcirc); -int rend_parse_service_authorization(or_options_t *options, +int rend_parse_service_authorization(const or_options_t *options, int validate_only); rend_service_authorization_t *rend_client_lookup_service_authorization( const char *onion_address); diff --git a/src/or/rendcommon.c b/src/or/rendcommon.c index c5bf88163d..76786e0fd1 100644 --- a/src/or/rendcommon.c +++ b/src/or/rendcommon.c @@ -1,5 +1,5 @@ /* Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -34,7 +34,7 @@ rend_service_descriptor_free(rend_service_descriptor_t *desc) if (!desc) return; if (desc->pk) - crypto_free_pk_env(desc->pk); + crypto_pk_free(desc->pk); if (desc->intro_nodes) { SMARTLIST_FOREACH(desc->intro_nodes, rend_intro_point_t *, intro, rend_intro_point_free(intro);); @@ -64,11 +64,11 @@ rend_get_descriptor_id_bytes(char *descriptor_id_out, const char *service_id, const char *secret_id_part) { - crypto_digest_env_t *digest = crypto_new_digest_env(); + crypto_digest_t *digest = crypto_digest_new(); crypto_digest_add_bytes(digest, service_id, REND_SERVICE_ID_LEN); crypto_digest_add_bytes(digest, secret_id_part, DIGEST_LEN); crypto_digest_get_digest(digest, descriptor_id_out, DIGEST_LEN); - crypto_free_digest_env(digest); + crypto_digest_free(digest); } /** Compute the secret ID part for time_period, @@ -80,7 +80,7 @@ static void get_secret_id_part_bytes(char *secret_id_part, uint32_t time_period, const char *descriptor_cookie, uint8_t replica) { - crypto_digest_env_t *digest = crypto_new_digest_env(); + crypto_digest_t *digest = crypto_digest_new(); time_period = htonl(time_period); crypto_digest_add_bytes(digest, (char*)&time_period, sizeof(uint32_t)); if (descriptor_cookie) { @@ -89,7 +89,7 @@ get_secret_id_part_bytes(char *secret_id_part, uint32_t time_period, } crypto_digest_add_bytes(digest, (const char *)&replica, REND_REPLICA_LEN); crypto_digest_get_digest(digest, secret_id_part, DIGEST_LEN); - crypto_free_digest_env(digest); + crypto_digest_free(digest); } /** Return the time period for time <b>now</b> plus a potentially @@ -181,7 +181,7 @@ rend_encode_v2_intro_points(char **encoded, rend_service_descriptor_t *desc) char id_base32[REND_INTRO_POINT_ID_LEN_BASE32 + 1]; char *onion_key = NULL; size_t onion_key_len; - crypto_pk_env_t *intro_key; + crypto_pk_t *intro_key; char *service_key = NULL; char *address = NULL; size_t service_key_len; @@ -262,8 +262,8 @@ rend_encrypt_v2_intro_points_basic(char **encrypted_out, char *enc = NULL, iv[CIPHER_IV_LEN], *client_part = NULL, session_key[CIPHER_KEY_LEN]; smartlist_t *encrypted_session_keys = NULL; - crypto_digest_env_t *digest; - crypto_cipher_env_t *cipher; + crypto_digest_t *digest; + crypto_cipher_t *cipher; tor_assert(encoded); tor_assert(client_cookies && smartlist_len(client_cookies) > 0); @@ -290,11 +290,10 @@ rend_encrypt_v2_intro_points_basic(char **encrypted_out, enc[1] = (uint8_t)client_blocks; /* Encrypt with random session key. */ - cipher = crypto_create_init_cipher(session_key, 1); - enclen = crypto_cipher_encrypt_with_iv(cipher, + enclen = crypto_cipher_encrypt_with_iv(session_key, enc + 2 + client_entries_len, CIPHER_IV_LEN + strlen(encoded), encoded, strlen(encoded)); - crypto_free_cipher_env(cipher); + if (enclen < 0) { log_warn(LD_REND, "Could not encrypt introduction point string."); goto done; @@ -303,28 +302,28 @@ rend_encrypt_v2_intro_points_basic(char **encrypted_out, /* Encrypt session key for cookies, determine client IDs, and put both * in a smartlist. */ - encrypted_session_keys = smartlist_create(); + encrypted_session_keys = smartlist_new(); SMARTLIST_FOREACH_BEGIN(client_cookies, const char *, cookie) { client_part = tor_malloc_zero(REND_BASIC_AUTH_CLIENT_ENTRY_LEN); /* Encrypt session key. */ - cipher = crypto_create_init_cipher(cookie, 1); + cipher = crypto_cipher_new(cookie); if (crypto_cipher_encrypt(cipher, client_part + REND_BASIC_AUTH_CLIENT_ID_LEN, session_key, CIPHER_KEY_LEN) < 0) { log_warn(LD_REND, "Could not encrypt session key for client."); - crypto_free_cipher_env(cipher); + crypto_cipher_free(cipher); tor_free(client_part); goto done; } - crypto_free_cipher_env(cipher); + crypto_cipher_free(cipher); /* Determine client ID. */ - digest = crypto_new_digest_env(); + digest = crypto_digest_new(); crypto_digest_add_bytes(digest, cookie, REND_DESC_COOKIE_LEN); crypto_digest_add_bytes(digest, iv, CIPHER_IV_LEN); crypto_digest_get_digest(digest, client_part, REND_BASIC_AUTH_CLIENT_ID_LEN); - crypto_free_digest_env(digest); + crypto_digest_free(digest); /* Put both together. */ smartlist_add(encrypted_session_keys, client_part); @@ -374,18 +373,16 @@ rend_encrypt_v2_intro_points_stealth(char **encrypted_out, const char *descriptor_cookie) { int r = -1, enclen; - crypto_cipher_env_t *cipher; char *enc; tor_assert(encoded); tor_assert(descriptor_cookie); enc = tor_malloc_zero(1 + CIPHER_IV_LEN + strlen(encoded)); enc[0] = 0x02; /* Auth type */ - cipher = crypto_create_init_cipher(descriptor_cookie, 1); - enclen = crypto_cipher_encrypt_with_iv(cipher, enc + 1, + enclen = crypto_cipher_encrypt_with_iv(descriptor_cookie, + enc + 1, CIPHER_IV_LEN+strlen(encoded), encoded, strlen(encoded)); - crypto_free_cipher_env(cipher); if (enclen < 0) { log_warn(LD_REND, "Could not encrypt introduction point string."); goto done; @@ -439,7 +436,12 @@ rend_intro_point_free(rend_intro_point_t *intro) return; extend_info_free(intro->extend_info); - crypto_free_pk_env(intro->intro_key); + crypto_pk_free(intro->intro_key); + + if (intro->accepted_intro_rsa_parts != NULL) { + digestmap_free(intro->accepted_intro_rsa_parts, _tor_free); + } + tor_free(intro); } @@ -455,7 +457,7 @@ int rend_encode_v2_descriptors(smartlist_t *descs_out, rend_service_descriptor_t *desc, time_t now, uint8_t period, rend_auth_type_t auth_type, - crypto_pk_env_t *client_key, + crypto_pk_t *client_key, smartlist_t *client_cookies) { char service_id[DIGEST_LEN]; @@ -465,7 +467,7 @@ rend_encode_v2_descriptors(smartlist_t *descs_out, size_t ipos_len = 0, ipos_encrypted_len = 0; int k; uint32_t seconds_valid; - crypto_pk_env_t *service_key; + crypto_pk_t *service_key; if (!desc) { log_warn(LD_BUG, "Could not encode v2 descriptor: No desc given."); return -1; @@ -703,7 +705,7 @@ rend_parse_service_descriptor(const char *str, size_t len) n_intro_points = ntohs(get_uint16(cp)); cp += 2; - result->intro_nodes = smartlist_create(); + result->intro_nodes = smartlist_new(); for (i=0;i<n_intro_points;++i) { if (end-cp < 2) goto truncated; eos = (const char *)memchr(cp,'\0',end-cp); @@ -748,7 +750,7 @@ rend_parse_service_descriptor(const char *str, size_t len) * identify services in directory requests and .onion URLs.) */ int -rend_get_service_id(crypto_pk_env_t *pk, char *out) +rend_get_service_id(crypto_pk_t *pk, char *out) { char buf[DIGEST_LEN]; tor_assert(pk); @@ -795,6 +797,8 @@ rend_cache_entry_free(rend_cache_entry_t *e) tor_free(e); } +/** Helper: deallocate a rend_cache_entry_t. (Used with strmap_free(), which + * requires a function pointer whose argument is void*). */ static void _rend_cache_entry_free(void *p) { @@ -814,14 +818,13 @@ rend_cache_free_all(void) /** Removes all old entries from the service descriptor cache. */ void -rend_cache_clean(void) +rend_cache_clean(time_t now) { strmap_iter_t *iter; const char *key; void *val; rend_cache_entry_t *ent; - time_t cutoff; - cutoff = time(NULL) - REND_CACHE_MAX_AGE - REND_CACHE_MAX_SKEW; + time_t cutoff = now - REND_CACHE_MAX_AGE - REND_CACHE_MAX_SKEW; for (iter = strmap_iter_init(rend_cache); !strmap_iter_done(iter); ) { strmap_iter_get(iter, &key, &val); ent = (rend_cache_entry_t*)val; @@ -849,10 +852,10 @@ rend_cache_purge(void) /** Remove all old v2 descriptors and those for which this hidden service * directory is not responsible for any more. */ void -rend_cache_clean_v2_descs_as_dir(void) +rend_cache_clean_v2_descs_as_dir(time_t now) { digestmap_iter_t *iter; - time_t cutoff = time(NULL) - REND_CACHE_MAX_AGE - REND_CACHE_MAX_SKEW; + time_t cutoff = now - REND_CACHE_MAX_AGE - REND_CACHE_MAX_SKEW; for (iter = digestmap_iter_init(rend_cache_v2_dir); !digestmap_iter_done(iter); ) { const char *key; @@ -999,6 +1002,10 @@ rend_cache_lookup_v2_desc_as_dir(const char *desc_id, const char **desc) return 0; } +/* Do not allow more than this many introduction points in a hidden service + * descriptor */ +#define MAX_INTRO_POINTS 10 + /** Parse *desc, calculate its service id, and store it in the cache. * If we have a newer v0 descriptor with the same ID, ignore this one. * If we have an older descriptor with the same ID, replace it. @@ -1067,6 +1074,16 @@ rend_cache_store(const char *desc, size_t desc_len, int published, rend_service_descriptor_free(parsed); return -1; } + if (parsed->intro_nodes && + smartlist_len(parsed->intro_nodes) > MAX_INTRO_POINTS) { + log_warn(LD_REND, "Found too many introduction points on a hidden " + "service descriptor for %s. This is probably a (misguided) " + "attempt to improve reliability, but it could also be an " + "attempt to do a guard enumeration attack. Rejecting.", + safe_str_client(query)); + rend_service_descriptor_free(parsed); + return -2; + } tor_snprintf(key, sizeof(key), "0%s", query); e = (rend_cache_entry_t*) strmap_get_lc(rend_cache, key); if (e && e->parsed->timestamp > parsed->timestamp) { @@ -1285,6 +1302,7 @@ rend_cache_store_v2_desc_as_client(const char *desc, } /* Decode/decrypt introduction points. */ if (intro_content) { + int n_intro_points; if (rend_query->auth_type != REND_NO_AUTH && !tor_mem_is_zero(rend_query->descriptor_cookie, sizeof(rend_query->descriptor_cookie))) { @@ -1305,17 +1323,26 @@ rend_cache_store_v2_desc_as_client(const char *desc, intro_size = ipos_decrypted_size; } } - if (rend_parse_introduction_points(parsed, intro_content, - intro_size) <= 0) { + n_intro_points = rend_parse_introduction_points(parsed, intro_content, + intro_size); + if (n_intro_points <= 0) { log_warn(LD_REND, "Failed to parse introduction points. Either the " "service has published a corrupt descriptor or you have " "provided invalid authorization data."); retval = -2; goto err; + } else if (n_intro_points > MAX_INTRO_POINTS) { + log_warn(LD_REND, "Found too many introduction points on a hidden " + "service descriptor for %s. This is probably a (misguided) " + "attempt to improve reliability, but it could also be an " + "attempt to do a guard enumeration attack. Rejecting.", + safe_str_client(rend_query->onion_address)); + retval = -2; + goto err; } } else { log_info(LD_REND, "Descriptor does not contain any introduction points."); - parsed->intro_nodes = smartlist_create(); + parsed->intro_nodes = smartlist_new(); } /* We don't need the encoded/encrypted introduction points any longer. */ tor_free(intro_content); diff --git a/src/or/rendcommon.h b/src/or/rendcommon.h index c51039f1f2..be6bd13d2c 100644 --- a/src/or/rendcommon.h +++ b/src/or/rendcommon.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -28,14 +28,14 @@ void rend_process_relay_cell(circuit_t *circ, const crypt_path_t *layer_hint, void rend_service_descriptor_free(rend_service_descriptor_t *desc); rend_service_descriptor_t *rend_parse_service_descriptor(const char *str, size_t len); -int rend_get_service_id(crypto_pk_env_t *pk, char *out); +int rend_get_service_id(crypto_pk_t *pk, char *out); void rend_encoded_v2_service_descriptor_free( rend_encoded_v2_service_descriptor_t *desc); void rend_intro_point_free(rend_intro_point_t *intro); void rend_cache_init(void); -void rend_cache_clean(void); -void rend_cache_clean_v2_descs_as_dir(void); +void rend_cache_clean(time_t now); +void rend_cache_clean_v2_descs_as_dir(time_t now); void rend_cache_purge(void); void rend_cache_free_all(void); int rend_valid_service_id(const char *query); @@ -53,7 +53,7 @@ int rend_cache_size(void); int rend_encode_v2_descriptors(smartlist_t *descs_out, rend_service_descriptor_t *desc, time_t now, uint8_t period, rend_auth_type_t auth_type, - crypto_pk_env_t *client_key, + crypto_pk_t *client_key, smartlist_t *client_cookies); int rend_compute_v2_desc_id(char *desc_id_out, const char *service_id, const char *descriptor_cookie, diff --git a/src/or/rendmid.c b/src/or/rendmid.c index 04edd8e3e2..bacd0ef93e 100644 --- a/src/or/rendmid.c +++ b/src/or/rendmid.c @@ -1,5 +1,5 @@ /* Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -9,6 +9,7 @@ #include "or.h" #include "circuitlist.h" +#include "circuituse.h" #include "config.h" #include "relay.h" #include "rendmid.h" @@ -21,7 +22,7 @@ int rend_mid_establish_intro(or_circuit_t *circ, const uint8_t *request, size_t request_len) { - crypto_pk_env_t *pk = NULL; + crypto_pk_t *pk = NULL; char buf[DIGEST_LEN+9]; char expected_digest[DIGEST_LEN]; char pk_digest[DIGEST_LEN]; @@ -85,7 +86,7 @@ rend_mid_establish_intro(or_circuit_t *circ, const uint8_t *request, goto err; } - crypto_free_pk_env(pk); /* don't need it anymore */ + crypto_pk_free(pk); /* don't need it anymore */ pk = NULL; /* so we don't free it again if err */ base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32+1, @@ -109,7 +110,7 @@ rend_mid_establish_intro(or_circuit_t *circ, const uint8_t *request, } /* Now, set up this circuit. */ - circ->_base.purpose = CIRCUIT_PURPOSE_INTRO_POINT; + circuit_change_purpose(TO_CIRCUIT(circ), CIRCUIT_PURPOSE_INTRO_POINT); memcpy(circ->rend_token, pk_digest, DIGEST_LEN); log_info(LD_REND, @@ -121,7 +122,7 @@ rend_mid_establish_intro(or_circuit_t *circ, const uint8_t *request, log_warn(LD_PROTOCOL, "Rejecting truncated ESTABLISH_INTRO cell."); reason = END_CIRC_REASON_TORPROTOCOL; err: - if (pk) crypto_free_pk_env(pk); + if (pk) crypto_pk_free(pk); circuit_mark_for_close(TO_CIRCUIT(circ), reason); return -1; } @@ -249,7 +250,7 @@ rend_mid_establish_rendezvous(or_circuit_t *circ, const uint8_t *request, goto err; } - circ->_base.purpose = CIRCUIT_PURPOSE_REND_POINT_WAITING; + circuit_change_purpose(TO_CIRCUIT(circ), CIRCUIT_PURPOSE_REND_POINT_WAITING); memcpy(circ->rend_token, request, REND_COOKIE_LEN); base16_encode(hexid,9,(char*)request,4); @@ -275,13 +276,6 @@ rend_mid_rendezvous(or_circuit_t *circ, const uint8_t *request, or_circuit_t *rend_circ; char hexid[9]; int reason = END_CIRC_REASON_INTERNAL; - base16_encode(hexid,9,(char*)request,request_len<4?request_len:4); - - if (request_len>=4) { - log_info(LD_REND, - "Got request for rendezvous from circuit %d to cookie %s.", - circ->p_circ_id, hexid); - } if (circ->_base.purpose != CIRCUIT_PURPOSE_OR || circ->_base.n_conn) { log_info(LD_REND, @@ -299,6 +293,12 @@ rend_mid_rendezvous(or_circuit_t *circ, const uint8_t *request, goto err; } + base16_encode(hexid, sizeof(hexid), (const char*)request, 4); + + log_info(LD_REND, + "Got request for rendezvous from circuit %d to cookie %s.", + circ->p_circ_id, hexid); + rend_circ = circuit_get_rendezvous((char*)request); if (!rend_circ) { log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, @@ -324,8 +324,9 @@ rend_mid_rendezvous(or_circuit_t *circ, const uint8_t *request, "Completing rendezvous: circuit %d joins circuit %d (cookie %s)", circ->p_circ_id, rend_circ->p_circ_id, hexid); - circ->_base.purpose = CIRCUIT_PURPOSE_REND_ESTABLISHED; - rend_circ->_base.purpose = CIRCUIT_PURPOSE_REND_ESTABLISHED; + circuit_change_purpose(TO_CIRCUIT(circ), CIRCUIT_PURPOSE_REND_ESTABLISHED); + circuit_change_purpose(TO_CIRCUIT(rend_circ), + CIRCUIT_PURPOSE_REND_ESTABLISHED); memset(circ->rend_token, 0, REND_COOKIE_LEN); rend_circ->rend_splice = circ; diff --git a/src/or/rendmid.h b/src/or/rendmid.h index 5ed87fd2b1..0af6436dea 100644 --- a/src/or/rendmid.h +++ b/src/or/rendmid.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** diff --git a/src/or/rendservice.c b/src/or/rendservice.c index a1daa8a550..d235f089fc 100644 --- a/src/or/rendservice.c +++ b/src/or/rendservice.c @@ -1,5 +1,5 @@ /* Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -14,6 +14,7 @@ #include "config.h" #include "directory.h" #include "networkstatus.h" +#include "nodelist.h" #include "rendclient.h" #include "rendcommon.h" #include "rendservice.h" @@ -25,6 +26,11 @@ static origin_circuit_t *find_intro_circuit(rend_intro_point_t *intro, const char *pk_digest); +static rend_intro_point_t *find_intro_point(origin_circuit_t *circ); + +static int intro_point_accepted_intro_count(rend_intro_point_t *intro); +static int intro_point_should_expire_now(rend_intro_point_t *intro, + time_t now); /** Represents the mapping from a virtual port of a rendezvous service to * a real port on some IP. @@ -35,8 +41,10 @@ typedef struct rend_service_port_config_t { tor_addr_t real_addr; } rend_service_port_config_t; -/** Try to maintain this many intro points per service if possible. */ -#define NUM_INTRO_POINTS 3 +/** Try to maintain this many intro points per service by default. */ +#define NUM_INTRO_POINTS_DEFAULT 3 +/** Maintain no more than this many intro points per hidden service. */ +#define NUM_INTRO_POINTS_MAX 10 /** If we can't build our intro circuits, don't retry for this long. */ #define INTRO_CIRC_RETRY_PERIOD (60*5) @@ -50,6 +58,10 @@ typedef struct rend_service_port_config_t { * rendezvous point before giving up? */ #define MAX_REND_TIMEOUT 30 +/** How many seconds should we wait for new HS descriptors to reach + * our clients before we close an expiring intro point? */ +#define INTRO_POINT_EXPIRATION_GRACE_PERIOD 5*60 + /** Represents a single hidden service running at this OP. */ typedef struct rend_service_t { /* Fields specified in config file */ @@ -61,7 +73,7 @@ typedef struct rend_service_t { * clients that may access our service. Can be NULL * if no client authorization is performed. */ /* Other fields */ - crypto_pk_env_t *private_key; /**< Permanent hidden-service key. */ + crypto_pk_t *private_key; /**< Permanent hidden-service key. */ char service_id[REND_SERVICE_ID_LEN_BASE32+1]; /**< Onion address without * '.onion' */ char pk_digest[DIGEST_LEN]; /**< Hash of permanent hidden-service key. */ @@ -71,17 +83,24 @@ typedef struct rend_service_t { * introduction points. */ int n_intro_circuits_launched; /**< Count of intro circuits we have * established in this period. */ + unsigned int n_intro_points_wanted; /**< Number of intro points this + * service wants to have open. */ rend_service_descriptor_t *desc; /**< Current hidden service descriptor. */ time_t desc_is_dirty; /**< Time at which changes to the hidden service * descriptor content occurred, or 0 if it's * up-to-date. */ time_t next_upload_time; /**< Scheduled next hidden service descriptor * upload time. */ - /** Map from digests of Diffie-Hellman values INTRODUCE2 to time_t of when - * they were received; used to prevent replays. */ - digestmap_t *accepted_intros; - /** Time at which we last removed expired values from accepted_intros. */ - time_t last_cleaned_accepted_intros; + /** Map from digests of Diffie-Hellman values INTRODUCE2 to time_t + * of when they were received. Clients may send INTRODUCE1 cells + * for the same rendezvous point through two or more different + * introduction points; when they do, this digestmap keeps us from + * launching multiple simultaneous attempts to connect to the same + * rend point. */ + digestmap_t *accepted_intro_dh_parts; + /** Time at which we last removed expired values from + * accepted_intro_dh_parts. */ + time_t last_cleaned_accepted_intro_dh_parts; } rend_service_t; /** A list of rend_service_t's for services run on this OP. @@ -97,6 +116,17 @@ num_rend_services(void) return smartlist_len(rend_service_list); } +/** Return a string identifying <b>service</b>, suitable for use in a + * log message. The result does not need to be freed, but may be + * overwritten by the next call to this function. */ +static const char * +rend_service_describe_for_log(rend_service_t *service) +{ + /* XXX024 Use this function throughout rendservice.c. */ + /* XXX024 Return a more useful description? */ + return safe_str_client(service->service_id); +} + /** Helper: free storage held by a single service authorized client entry. */ static void rend_authorized_client_free(rend_authorized_client_t *client) @@ -104,7 +134,7 @@ rend_authorized_client_free(rend_authorized_client_t *client) if (!client) return; if (client->client_key) - crypto_free_pk_env(client->client_key); + crypto_pk_free(client->client_key); tor_free(client->client_name); tor_free(client); } @@ -128,7 +158,7 @@ rend_service_free(rend_service_t *service) SMARTLIST_FOREACH(service->ports, void*, p, tor_free(p)); smartlist_free(service->ports); if (service->private_key) - crypto_free_pk_env(service->private_key); + crypto_pk_free(service->private_key); if (service->intro_nodes) { SMARTLIST_FOREACH(service->intro_nodes, rend_intro_point_t *, intro, rend_intro_point_free(intro);); @@ -141,7 +171,7 @@ rend_service_free(rend_service_t *service) rend_authorized_client_free(c);); smartlist_free(service->clients); } - digestmap_free(service->accepted_intros, _tor_free); + digestmap_free(service->accepted_intro_dh_parts, _tor_free); tor_free(service); } @@ -167,13 +197,13 @@ rend_add_service(rend_service_t *service) int i; rend_service_port_config_t *p; - service->intro_nodes = smartlist_create(); + service->intro_nodes = smartlist_new(); if (service->auth_type != REND_NO_AUTH && smartlist_len(service->clients) == 0) { log_warn(LD_CONFIG, "Hidden service (%s) with client authorization but no " "clients; ignoring.", - esc_for_log(service->directory)); + escaped(service->directory)); rend_service_free(service); return; } @@ -181,7 +211,7 @@ rend_add_service(rend_service_t *service) if (!smartlist_len(service->ports)) { log_warn(LD_CONFIG, "Hidden service (%s) with no ports configured; " "ignoring.", - esc_for_log(service->directory)); + escaped(service->directory)); rend_service_free(service); } else { int dupe = 0; @@ -238,7 +268,7 @@ parse_port_config(const char *string) const char *addrport; rend_service_port_config_t *result = NULL; - sl = smartlist_create(); + sl = smartlist_new(); smartlist_split_string(sl, string, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); if (smartlist_len(sl) < 1 || smartlist_len(sl) > 2) { @@ -260,7 +290,7 @@ parse_port_config(const char *string) } else { addrport = smartlist_get(sl,1); if (strchr(addrport, ':') || strchr(addrport, '.')) { - if (tor_addr_port_parse(addrport, &addr, &p)<0) { + if (tor_addr_port_lookup(addrport, &addr, &p)<0) { log_warn(LD_CONFIG,"Unparseable address in hidden service port " "configuration."); goto err; @@ -294,7 +324,7 @@ parse_port_config(const char *string) * normal, but don't actually change the configured services.) */ int -rend_config_services(or_options_t *options, int validate_only) +rend_config_services(const or_options_t *options, int validate_only) { config_line_t *line; rend_service_t *service = NULL; @@ -303,7 +333,7 @@ rend_config_services(or_options_t *options, int validate_only) if (!validate_only) { old_service_list = rend_service_list; - rend_service_list = smartlist_create(); + rend_service_list = smartlist_new(); } for (line = options->RendConfigLines; line; line = line->next) { @@ -316,8 +346,9 @@ rend_config_services(or_options_t *options, int validate_only) } service = tor_malloc_zero(sizeof(rend_service_t)); service->directory = tor_strdup(line->value); - service->ports = smartlist_create(); + service->ports = smartlist_new(); service->intro_period_started = time(NULL); + service->n_intro_points_wanted = NUM_INTRO_POINTS_DEFAULT; continue; } if (!service) { @@ -346,7 +377,7 @@ rend_config_services(or_options_t *options, int validate_only) rend_service_free(service); return -1; } - type_names_split = smartlist_create(); + type_names_split = smartlist_new(); smartlist_split_string(type_names_split, line->value, " ", 0, 2); if (smartlist_len(type_names_split) < 1) { log_warn(LD_BUG, "HiddenServiceAuthorizeClient has no value. This " @@ -371,7 +402,7 @@ rend_config_services(or_options_t *options, int validate_only) rend_service_free(service); return -1; } - service->clients = smartlist_create(); + service->clients = smartlist_new(); if (smartlist_len(type_names_split) < 2) { log_warn(LD_CONFIG, "HiddenServiceAuthorizeClient contains " "auth-type '%s', but no client names.", @@ -380,7 +411,7 @@ rend_config_services(or_options_t *options, int validate_only) smartlist_free(type_names_split); continue; } - clients = smartlist_create(); + clients = smartlist_new(); smartlist_split_string(clients, smartlist_get(type_names_split, 1), ",", SPLIT_SKIP_SPACE, 0); SMARTLIST_FOREACH(type_names_split, char *, cp, tor_free(cp)); @@ -463,22 +494,22 @@ rend_config_services(or_options_t *options, int validate_only) * keep the introduction points that are still needed and close the * other ones. */ if (old_service_list && !validate_only) { - smartlist_t *surviving_services = smartlist_create(); + smartlist_t *surviving_services = smartlist_new(); circuit_t *circ; /* Copy introduction points to new services. */ /* XXXX This is O(n^2), but it's only called on reconfigure, so it's * probably ok? */ - SMARTLIST_FOREACH(rend_service_list, rend_service_t *, new, { - SMARTLIST_FOREACH(old_service_list, rend_service_t *, old, { + SMARTLIST_FOREACH_BEGIN(rend_service_list, rend_service_t *, new) { + SMARTLIST_FOREACH_BEGIN(old_service_list, rend_service_t *, old) { if (!strcmp(old->directory, new->directory)) { smartlist_add_all(new->intro_nodes, old->intro_nodes); smartlist_clear(old->intro_nodes); smartlist_add(surviving_services, old); break; } - }); - }); + } SMARTLIST_FOREACH_END(old); + } SMARTLIST_FOREACH_END(new); /* Close introduction circuits of services we don't serve anymore. */ /* XXXX it would be nicer if we had a nicer abstraction to use here, @@ -534,23 +565,45 @@ rend_service_update_descriptor(rend_service_t *service) d = service->desc = tor_malloc_zero(sizeof(rend_service_descriptor_t)); d->pk = crypto_pk_dup_key(service->private_key); d->timestamp = time(NULL); - d->intro_nodes = smartlist_create(); + d->intro_nodes = smartlist_new(); /* Support intro protocols 2 and 3. */ d->protocols = (1 << 2) + (1 << 3); for (i = 0; i < smartlist_len(service->intro_nodes); ++i) { rend_intro_point_t *intro_svc = smartlist_get(service->intro_nodes, i); rend_intro_point_t *intro_desc; + + /* This intro point won't be listed in the descriptor... */ + intro_svc->listed_in_last_desc = 0; + + if (intro_svc->time_expiring != -1) { + /* This intro point is expiring. Don't list it. */ + continue; + } + circ = find_intro_circuit(intro_svc, service->pk_digest); - if (!circ || circ->_base.purpose != CIRCUIT_PURPOSE_S_INTRO) + if (!circ || circ->_base.purpose != CIRCUIT_PURPOSE_S_INTRO) { + /* This intro point's circuit isn't finished yet. Don't list it. */ continue; + } - /* We have an entirely established intro circuit. */ + /* ...unless this intro point is listed in the descriptor. */ + intro_svc->listed_in_last_desc = 1; + + /* We have an entirely established intro circuit. Publish it in + * our descriptor. */ intro_desc = tor_malloc_zero(sizeof(rend_intro_point_t)); intro_desc->extend_info = extend_info_dup(intro_svc->extend_info); if (intro_svc->intro_key) intro_desc->intro_key = crypto_pk_dup_key(intro_svc->intro_key); smartlist_add(d->intro_nodes, intro_desc); + + if (intro_svc->time_published == -1) { + /* We are publishing this intro point in a descriptor for the + * first time -- note the current time in the service's copy of + * the intro point. */ + intro_svc->time_published = time(NULL); + } } } @@ -681,19 +734,19 @@ rend_service_load_keys(void) client->client_key = crypto_pk_dup_key(parsed->client_key); } else if (s->auth_type == REND_STEALTH_AUTH) { /* Create private key for client. */ - crypto_pk_env_t *prkey = NULL; - if (!(prkey = crypto_new_pk_env())) { + crypto_pk_t *prkey = NULL; + if (!(prkey = crypto_pk_new())) { log_warn(LD_BUG,"Error constructing client key"); goto err; } if (crypto_pk_generate_key(prkey)) { log_warn(LD_BUG,"Error generating client key"); - crypto_free_pk_env(prkey); + crypto_pk_free(prkey); goto err; } if (crypto_pk_check_key(prkey) <= 0) { log_warn(LD_BUG,"Generated client key seems invalid"); - crypto_free_pk_env(prkey); + crypto_pk_free(prkey); goto err; } client->client_key = prkey; @@ -856,15 +909,16 @@ rend_check_authorization(rend_service_t *service, /** Remove elements from <b>service</b>'s replay cache that are old enough to * be noticed by timestamp checking. */ static void -clean_accepted_intros(rend_service_t *service, time_t now) +clean_accepted_intro_dh_parts(rend_service_t *service, time_t now) { const time_t cutoff = now - REND_REPLAY_TIME_INTERVAL; - service->last_cleaned_accepted_intros = now; - if (!service->accepted_intros) + service->last_cleaned_accepted_intro_dh_parts = now; + if (!service->accepted_intro_dh_parts) return; - DIGESTMAP_FOREACH_MODIFY(service->accepted_intros, digest, time_t *, t) { + DIGESTMAP_FOREACH_MODIFY(service->accepted_intro_dh_parts, digest, + time_t *, t) { if (*t < cutoff) { tor_free(t); MAP_DEL_CURRENT(digest); @@ -872,6 +926,106 @@ clean_accepted_intros(rend_service_t *service, time_t now) } DIGESTMAP_FOREACH_END; } +/** Called when <b>intro</b> will soon be removed from + * <b>service</b>'s list of intro points. */ +static void +rend_service_note_removing_intro_point(rend_service_t *service, + rend_intro_point_t *intro) +{ + time_t now = time(NULL); + + /* Don't process an intro point twice here. */ + if (intro->rend_service_note_removing_intro_point_called) { + return; + } else { + intro->rend_service_note_removing_intro_point_called = 1; + } + + /* Update service->n_intro_points_wanted based on how long intro + * lasted and how many introductions it handled. */ + if (intro->time_published == -1) { + /* This intro point was never used. Don't change + * n_intro_points_wanted. */ + } else { + /* We want to increase the number of introduction points service + * operates if intro was heavily used, or decrease the number of + * intro points if intro was lightly used. + * + * We consider an intro point's target 'usage' to be + * INTRO_POINT_LIFETIME_INTRODUCTIONS introductions in + * INTRO_POINT_LIFETIME_MIN_SECONDS seconds. To calculate intro's + * fraction of target usage, we divide the fraction of + * _LIFETIME_INTRODUCTIONS introductions that it has handled by + * the fraction of _LIFETIME_MIN_SECONDS for which it existed. + * + * Then we multiply that fraction of desired usage by a fudge + * factor of 1.5, to decide how many new introduction points + * should ideally replace intro (which is now closed or soon to be + * closed). In theory, assuming that introduction load is + * distributed equally across all intro points and ignoring the + * fact that different intro points are established and closed at + * different times, that number of intro points should bring all + * of our intro points exactly to our target usage. + * + * Then we clamp that number to a number of intro points we might + * be willing to replace this intro point with and turn it into an + * integer. then we clamp it again to the number of new intro + * points we could establish now, then we adjust + * service->n_intro_points_wanted and let rend_services_introduce + * create the new intro points we want (if any). + */ + const double intro_point_usage = + intro_point_accepted_intro_count(intro) / + (double)(now - intro->time_published); + const double intro_point_target_usage = + INTRO_POINT_LIFETIME_INTRODUCTIONS / + (double)INTRO_POINT_LIFETIME_MIN_SECONDS; + const double fractional_n_intro_points_wanted_to_replace_this_one = + (1.5 * (intro_point_usage / intro_point_target_usage)); + unsigned int n_intro_points_wanted_to_replace_this_one; + unsigned int n_intro_points_wanted_now; + unsigned int n_intro_points_really_wanted_now; + int n_intro_points_really_replacing_this_one; + + if (fractional_n_intro_points_wanted_to_replace_this_one > + NUM_INTRO_POINTS_MAX) { + n_intro_points_wanted_to_replace_this_one = NUM_INTRO_POINTS_MAX; + } else if (fractional_n_intro_points_wanted_to_replace_this_one < 0) { + n_intro_points_wanted_to_replace_this_one = 0; + } else { + n_intro_points_wanted_to_replace_this_one = (unsigned) + fractional_n_intro_points_wanted_to_replace_this_one; + } + + n_intro_points_wanted_now = + service->n_intro_points_wanted + + n_intro_points_wanted_to_replace_this_one - 1; + + if (n_intro_points_wanted_now < NUM_INTRO_POINTS_DEFAULT) { + /* XXXX This should be NUM_INTRO_POINTS_MIN instead. Perhaps + * another use of NUM_INTRO_POINTS_DEFAULT should be, too. */ + n_intro_points_really_wanted_now = NUM_INTRO_POINTS_DEFAULT; + } else if (n_intro_points_wanted_now > NUM_INTRO_POINTS_MAX) { + n_intro_points_really_wanted_now = NUM_INTRO_POINTS_MAX; + } else { + n_intro_points_really_wanted_now = n_intro_points_wanted_now; + } + + n_intro_points_really_replacing_this_one = + n_intro_points_really_wanted_now - service->n_intro_points_wanted + 1; + + log_info(LD_REND, "Replacing closing intro point for service %s " + "with %d new intro points (wanted %g replacements); " + "service will now try to have %u intro points", + rend_service_describe_for_log(service), + n_intro_points_really_replacing_this_one, + fractional_n_intro_points_wanted_to_replace_this_one, + n_intro_points_really_wanted_now); + + service->n_intro_points_wanted = n_intro_points_really_wanted_now; + } +} + /****** * Handle cells ******/ @@ -879,7 +1033,7 @@ clean_accepted_intros(rend_service_t *service, time_t now) /** Respond to an INTRODUCE2 cell by launching a circuit to the chosen * rendezvous point. */ - /* XXX022 this function sure could use some organizing. -RD */ + /* XXXX024 this function sure could use some organizing. -RD */ int rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, size_t request_len) @@ -889,25 +1043,26 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, char buf[RELAY_PAYLOAD_SIZE]; char keys[DIGEST_LEN+CPATH_KEY_MATERIAL_LEN]; /* Holds KH, Df, Db, Kf, Kb */ rend_service_t *service; + rend_intro_point_t *intro_point; int r, i, v3_shift = 0; size_t len, keylen; - crypto_dh_env_t *dh = NULL; + crypto_dh_t *dh = NULL; origin_circuit_t *launched = NULL; crypt_path_t *cpath = NULL; char serviceid[REND_SERVICE_ID_LEN_BASE32+1]; char hexcookie[9]; int circ_needs_uptime; int reason = END_CIRC_REASON_TORPROTOCOL; - crypto_pk_env_t *intro_key; + crypto_pk_t *intro_key; char intro_key_digest[DIGEST_LEN]; int auth_type; size_t auth_len = 0; char auth_data[REND_DESC_COOKIE_LEN]; - crypto_digest_env_t *digest = NULL; + crypto_digest_t *digest = NULL; time_t now = time(NULL); char diffie_hellman_hash[DIGEST_LEN]; time_t *access_time; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (circuit->_base.purpose != CIRCUIT_PURPOSE_S_INTRO) { log_warn(LD_PROTOCOL, @@ -916,6 +1071,9 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, return -1; } +#ifndef NON_ANONYMOUS_MODE_ENABLED + tor_assert(!(circuit->build_state->onehop_tunnel)); +#endif tor_assert(circuit->rend_data); base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32+1, @@ -935,7 +1093,8 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, service = rend_service_get_by_pk_digest( circuit->rend_data->rend_pk_digest); if (!service) { - log_warn(LD_REND, "Got an INTRODUCE2 cell for an unrecognized service %s.", + log_warn(LD_BUG, "Internal error: Got an INTRODUCE2 cell on an intro " + "circ for an unrecognized service %s.", escaped(serviceid)); return -1; } @@ -960,17 +1119,26 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, return -1; } - if (!service->accepted_intros) - service->accepted_intros = digestmap_new(); + intro_point = find_intro_point(circuit); + if (intro_point == NULL) { + log_warn(LD_BUG, "Internal error: Got an INTRODUCE2 cell on an intro circ " + "(for service %s) with no corresponding rend_intro_point_t.", + escaped(serviceid)); + return -1; + } + + if (!service->accepted_intro_dh_parts) + service->accepted_intro_dh_parts = digestmap_new(); + + if (!intro_point->accepted_intro_rsa_parts) + intro_point->accepted_intro_rsa_parts = digestmap_new(); { char pkpart_digest[DIGEST_LEN]; - /* Check for replay of PK-encrypted portion. It is slightly naughty to - use the same digestmap to check for this and for g^x replays, but - collisions are tremendously unlikely. - */ + /* Check for replay of PK-encrypted portion. */ crypto_digest(pkpart_digest, (char*)request+DIGEST_LEN, keylen); - access_time = digestmap_get(service->accepted_intros, pkpart_digest); + access_time = digestmap_get(intro_point->accepted_intro_rsa_parts, + pkpart_digest); if (access_time != NULL) { log_warn(LD_REND, "Possible replay detected! We received an " "INTRODUCE2 cell with same PK-encrypted part %d seconds ago. " @@ -979,7 +1147,8 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, } access_time = tor_malloc(sizeof(time_t)); *access_time = now; - digestmap_set(service->accepted_intros, pkpart_digest, access_time); + digestmap_set(intro_point->accepted_intro_rsa_parts, + pkpart_digest, access_time); } /* Next N bytes is encrypted with service key */ @@ -995,7 +1164,6 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, len = r; if (*buf == 3) { /* Version 3 INTRODUCE2 cell. */ - time_t ts = 0; v3_shift = 1; auth_type = buf[1]; switch (auth_type) { @@ -1017,17 +1185,8 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, log_info(LD_REND, "Unknown authorization type '%d'", auth_type); } - /* Check timestamp. */ - ts = ntohl(get_uint32(buf+1+v3_shift)); + /* Skip the timestamp field. We no longer use it. */ v3_shift += 4; - if ((now - ts) < -1 * REND_REPLAY_TIME_INTERVAL / 2 || - (now - ts) > REND_REPLAY_TIME_INTERVAL / 2) { - /* This is far more likely to mean that a client's clock is - * skewed than that a replay attack is in progress. */ - log_info(LD_REND, "INTRODUCE2 cell is too %s. Discarding.", - (now - ts) < 0 ? "old" : "new"); - return -1; - } } if (*buf == 2 || *buf == 3) { /* Version 2 INTRODUCE2 cell. */ @@ -1061,7 +1220,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, } else { char *rp_nickname; size_t nickname_field_len; - routerinfo_t *router; + const node_t *node; int version; if (*buf == 1) { rp_nickname = buf+1; @@ -1088,8 +1247,8 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, len -= nickname_field_len; len -= rp_nickname - buf; /* also remove header space used by version, if * any */ - router = router_get_by_nickname(rp_nickname, 0); - if (!router) { + node = node_get_by_nickname(rp_nickname, 0); + if (!node) { log_info(LD_REND, "Couldn't find router %s named in introduce2 cell.", escaped_safe_str_client(rp_nickname)); /* XXXX Add a no-such-router reason? */ @@ -1097,7 +1256,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, goto err; } - extend_info = extend_info_from_router(router); + extend_info = extend_info_from_node(node, 0); } if (len != REND_COOKIE_LEN+DH_KEY_LEN) { @@ -1107,7 +1266,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, } /* Check if we'd refuse to talk to this router */ - if (options->ExcludeNodes && options->StrictNodes && + if (options->StrictNodes && routerset_contains_extendinfo(options->ExcludeNodes, extend_info)) { log_warn(LD_REND, "Client asked to rendezvous at a relay that we " "exclude, and StrictNodes is set. Refusing service."); @@ -1119,14 +1278,15 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, base16_encode(hexcookie,9,r_cookie,4); /* Determine hash of Diffie-Hellman, part 1 to detect replays. */ - digest = crypto_new_digest_env(); + digest = crypto_digest_new(); crypto_digest_add_bytes(digest, ptr+REND_COOKIE_LEN, DH_KEY_LEN); crypto_digest_get_digest(digest, diffie_hellman_hash, DIGEST_LEN); - crypto_free_digest_env(digest); + crypto_digest_free(digest); /* Check whether there is a past request with the same Diffie-Hellman, * part 1. */ - access_time = digestmap_get(service->accepted_intros, diffie_hellman_hash); + access_time = digestmap_get(service->accepted_intro_dh_parts, + diffie_hellman_hash); if (access_time != NULL) { /* A Tor client will send a new INTRODUCE1 cell with the same rend * cookie and DH public key as its previous one if its intro circ @@ -1148,9 +1308,11 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, * one hour. */ access_time = tor_malloc(sizeof(time_t)); *access_time = now; - digestmap_set(service->accepted_intros, diffie_hellman_hash, access_time); - if (service->last_cleaned_accepted_intros + REND_REPLAY_TIME_INTERVAL < now) - clean_accepted_intros(service, now); + digestmap_set(service->accepted_intro_dh_parts, + diffie_hellman_hash, access_time); + if (service->last_cleaned_accepted_intro_dh_parts + REND_REPLAY_TIME_INTERVAL + < now) + clean_accepted_intro_dh_parts(service, now); /* If the service performs client authorization, check included auth data. */ if (service->clients) { @@ -1225,7 +1387,12 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, memcpy(launched->rend_data->rend_cookie, r_cookie, REND_COOKIE_LEN); strlcpy(launched->rend_data->onion_address, service->service_id, sizeof(launched->rend_data->onion_address)); - launched->build_state->pending_final_cpath = cpath = + + launched->build_state->service_pending_final_cpath_ref = + tor_malloc_zero(sizeof(crypt_path_reference_t)); + launched->build_state->service_pending_final_cpath_ref->refcount = 1; + + launched->build_state->service_pending_final_cpath_ref->cpath = cpath = tor_malloc_zero(sizeof(crypt_path_t)); cpath->magic = CRYPT_PATH_MAGIC; launched->build_state->expiry_time = now + MAX_REND_TIMEOUT; @@ -1237,10 +1404,10 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, memcpy(cpath->handshake_digest, keys, DIGEST_LEN); if (extend_info) extend_info_free(extend_info); - memset(keys, 0, sizeof(keys)); + memwipe(keys, 0, sizeof(keys)); return 0; err: - memset(keys, 0, sizeof(keys)); + memwipe(keys, 0, sizeof(keys)); if (dh) crypto_dh_free(dh); if (launched) circuit_mark_for_close(TO_CIRCUIT(launched), reason); @@ -1259,6 +1426,17 @@ rend_service_relaunch_rendezvous(origin_circuit_t *oldcirc) tor_assert(oldcirc->_base.purpose == CIRCUIT_PURPOSE_S_CONNECT_REND); + /* Don't relaunch the same rend circ twice. */ + if (oldcirc->hs_service_side_rend_circ_has_been_relaunched) { + log_info(LD_REND, "Rendezvous circuit to %s has already been relaunched; " + "not relaunching it again.", + oldcirc->build_state ? + safe_str(extend_info_describe(oldcirc->build_state->chosen_exit)) + : "*unknown*"); + return; + } + oldcirc->hs_service_side_rend_circ_has_been_relaunched = 1; + if (!oldcirc->build_state || oldcirc->build_state->failure_count > MAX_REND_FAILURES || oldcirc->build_state->expiry_time < time(NULL)) { @@ -1274,7 +1452,7 @@ rend_service_relaunch_rendezvous(origin_circuit_t *oldcirc) oldstate = oldcirc->build_state; tor_assert(oldstate); - if (oldstate->pending_final_cpath == NULL) { + if (oldstate->service_pending_final_cpath_ref == NULL) { log_info(LD_REND,"Skipping relaunch of circ that failed on its first hop. " "Initiator will retry."); return; @@ -1296,8 +1474,9 @@ rend_service_relaunch_rendezvous(origin_circuit_t *oldcirc) tor_assert(newstate); newstate->failure_count = oldstate->failure_count+1; newstate->expiry_time = oldstate->expiry_time; - newstate->pending_final_cpath = oldstate->pending_final_cpath; - oldstate->pending_final_cpath = NULL; + newstate->service_pending_final_cpath_ref = + oldstate->service_pending_final_cpath_ref; + ++(newstate->service_pending_final_cpath_ref->refcount); newcirc->rend_data = rend_data_dup(oldcirc->rend_data); } @@ -1389,9 +1568,12 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) char auth[DIGEST_LEN + 9]; char serviceid[REND_SERVICE_ID_LEN_BASE32+1]; int reason = END_CIRC_REASON_TORPROTOCOL; - crypto_pk_env_t *intro_key; + crypto_pk_t *intro_key; tor_assert(circuit->_base.purpose == CIRCUIT_PURPOSE_S_ESTABLISH_INTRO); +#ifndef NON_ANONYMOUS_MODE_ENABLED + tor_assert(!(circuit->build_state->onehop_tunnel)); +#endif tor_assert(circuit->cpath); tor_assert(circuit->rend_data); @@ -1409,8 +1591,9 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) /* If we already have enough introduction circuits for this service, * redefine this one as a general circuit or close it, depending. */ - if (count_established_intro_points(serviceid) > NUM_INTRO_POINTS) { - or_options_t *options = get_options(); + if (count_established_intro_points(serviceid) > + (int)service->n_intro_points_wanted) { /* XXX023 remove cast */ + const or_options_t *options = get_options(); if (options->ExcludeNodes) { /* XXXX in some future version, we can test whether the transition is allowed or not given the actual nodes in the circuit. But for now, @@ -1425,7 +1608,7 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) "circuit, but we already have enough. Redefining purpose to " "general; leaving as internal."); - TO_CIRCUIT(circuit)->purpose = CIRCUIT_PURPOSE_C_GENERAL; + circuit_change_purpose(TO_CIRCUIT(circuit), CIRCUIT_PURPOSE_C_GENERAL); { rend_data_t *rend_data = circuit->rend_data; @@ -1433,9 +1616,9 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) rend_data_free(rend_data); } { - crypto_pk_env_t *intro_key = circuit->intro_key; + crypto_pk_t *intro_key = circuit->intro_key; circuit->intro_key = NULL; - crypto_free_pk_env(intro_key); + crypto_pk_free(intro_key); } circuit_has_opened(circuit); @@ -1517,7 +1700,7 @@ rend_service_intro_established(origin_circuit_t *circuit, goto err; } service->desc_is_dirty = time(NULL); - circuit->_base.purpose = CIRCUIT_PURPOSE_S_INTRO; + circuit_change_purpose(TO_CIRCUIT(circuit), CIRCUIT_PURPOSE_S_INTRO); base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32 + 1, circuit->rend_data->rend_pk_digest, REND_SERVICE_ID_LEN); @@ -1547,9 +1730,11 @@ rend_service_rendezvous_has_opened(origin_circuit_t *circuit) tor_assert(circuit->_base.purpose == CIRCUIT_PURPOSE_S_CONNECT_REND); tor_assert(circuit->cpath); tor_assert(circuit->build_state); +#ifndef NON_ANONYMOUS_MODE_ENABLED + tor_assert(!(circuit->build_state->onehop_tunnel)); +#endif tor_assert(circuit->rend_data); - hop = circuit->build_state->pending_final_cpath; - tor_assert(hop); + hop = circuit->build_state->service_pending_final_cpath_ref->cpath; base16_encode(hexcookie,9,circuit->rend_data->rend_cookie,4); base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32+1, @@ -1560,6 +1745,27 @@ rend_service_rendezvous_has_opened(origin_circuit_t *circuit) "cookie %s for service %s", circuit->_base.n_circ_id, hexcookie, serviceid); + /* Clear the 'in-progress HS circ has timed out' flag for + * consistency with what happens on the client side; this line has + * no effect on Tor's behaviour. */ + circuit->hs_circ_has_timed_out = 0; + + /* If hop is NULL, another rend circ has already connected to this + * rend point. Close this circ. */ + if (hop == NULL) { + log_info(LD_REND, "Another rend circ has already reached this rend point; " + "closing this rend circ."); + reason = END_CIRC_REASON_NONE; + goto err; + } + + /* Remove our final cpath element from the reference, so that no + * other circuit will try to use it. Store it in + * pending_final_cpath for now to ensure that it will be freed if + * our rendezvous attempt fails. */ + circuit->build_state->pending_final_cpath = hop; + circuit->build_state->service_pending_final_cpath_ref->cpath = NULL; + service = rend_service_get_by_pk_digest( circuit->rend_data->rend_pk_digest); if (!service) { @@ -1605,7 +1811,7 @@ rend_service_rendezvous_has_opened(origin_circuit_t *circuit) circuit->build_state->pending_final_cpath = NULL; /* prevent double-free */ /* Change the circuit purpose. */ - circuit->_base.purpose = CIRCUIT_PURPOSE_S_REND_JOINED; + circuit_change_purpose(TO_CIRCUIT(circuit), CIRCUIT_PURPOSE_S_REND_JOINED); return; err: @@ -1648,6 +1854,35 @@ find_intro_circuit(rend_intro_point_t *intro, const char *pk_digest) return NULL; } +/** Return a pointer to the rend_intro_point_t corresponding to the + * service-side introduction circuit <b>circ</b>. */ +static rend_intro_point_t * +find_intro_point(origin_circuit_t *circ) +{ + const char *serviceid; + rend_service_t *service = NULL; + + tor_assert(TO_CIRCUIT(circ)->purpose == CIRCUIT_PURPOSE_S_ESTABLISH_INTRO || + TO_CIRCUIT(circ)->purpose == CIRCUIT_PURPOSE_S_INTRO); + tor_assert(circ->rend_data); + serviceid = circ->rend_data->onion_address; + + SMARTLIST_FOREACH(rend_service_list, rend_service_t *, s, + if (tor_memeq(s->service_id, serviceid, REND_SERVICE_ID_LEN_BASE32)) { + service = s; + break; + }); + + if (service == NULL) return NULL; + + SMARTLIST_FOREACH(service->intro_nodes, rend_intro_point_t *, intro_point, + if (crypto_pk_cmp_keys(intro_point->intro_key, circ->intro_key) == 0) { + return intro_point; + }); + + return NULL; +} + /** Determine the responsible hidden service directories for the * rend_encoded_v2_service_descriptor_t's in <b>descs</b> and upload them; * <b>service_id</b> and <b>seconds_valid</b> are only passed for logging @@ -1658,8 +1893,8 @@ directory_post_to_hs_dir(rend_service_descriptor_t *renddesc, int seconds_valid) { int i, j, failed_upload = 0; - smartlist_t *responsible_dirs = smartlist_create(); - smartlist_t *successful_uploads = smartlist_create(); + smartlist_t *responsible_dirs = smartlist_new(); + smartlist_t *successful_uploads = smartlist_new(); routerstatus_t *hs_dir; for (i = 0; i < smartlist_len(descs); i++) { rend_encoded_v2_service_descriptor_t *desc = smartlist_get(descs, i); @@ -1675,13 +1910,15 @@ directory_post_to_hs_dir(rend_service_descriptor_t *renddesc, for (j = 0; j < smartlist_len(responsible_dirs); j++) { char desc_id_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; char *hs_dir_ip; + const node_t *node; hs_dir = smartlist_get(responsible_dirs, j); if (smartlist_digest_isin(renddesc->successful_uploads, hs_dir->identity_digest)) /* Don't upload descriptor if we succeeded in doing so last time. */ continue; - if (!router_get_by_digest(hs_dir->identity_digest)) { - log_info(LD_REND, "Not sending publish request for v2 descriptor to " + node = node_get_by_id(hs_dir->identity_digest); + if (!node || !node_has_descriptor(node)) { + log_info(LD_REND, "Not launching upload for for v2 descriptor to " "hidden service directory %s; we don't have its " "router descriptor. Queuing for later upload.", safe_str_client(routerstatus_describe(hs_dir))); @@ -1697,7 +1934,7 @@ directory_post_to_hs_dir(rend_service_descriptor_t *renddesc, base32_encode(desc_id_base32, sizeof(desc_id_base32), desc->desc_id, DIGEST_LEN); hs_dir_ip = tor_dup_ip(hs_dir->addr); - log_info(LD_REND, "Sending publish request for v2 descriptor for " + log_info(LD_REND, "Launching upload for v2 descriptor for " "service '%s' with descriptor ID '%s' with validity " "of %d seconds to hidden service directory '%s' on " "%s:%d.", @@ -1725,7 +1962,7 @@ directory_post_to_hs_dir(rend_service_descriptor_t *renddesc, /* Remember which routers worked this time, so that we don't upload the * descriptor to them again. */ if (!renddesc->successful_uploads) - renddesc->successful_uploads = smartlist_create(); + renddesc->successful_uploads = smartlist_new(); SMARTLIST_FOREACH(successful_uploads, const char *, c, { if (!smartlist_digest_isin(renddesc->successful_uploads, c)) { char *hsdir_id = tor_memdup(c, DIGEST_LEN); @@ -1755,15 +1992,15 @@ upload_service_descriptor(rend_service_t *service) networkstatus_t *c = networkstatus_get_latest_consensus(); if (c && smartlist_len(c->routerstatus_list) > 0) { int seconds_valid, i, j, num_descs; - smartlist_t *descs = smartlist_create(); - smartlist_t *client_cookies = smartlist_create(); + smartlist_t *descs = smartlist_new(); + smartlist_t *client_cookies = smartlist_new(); /* Either upload a single descriptor (including replicas) or one * descriptor for each authorized client in case of authorization * type 'stealth'. */ num_descs = service->auth_type == REND_STEALTH_AUTH ? smartlist_len(service->clients) : 1; for (j = 0; j < num_descs; j++) { - crypto_pk_env_t *client_key = NULL; + crypto_pk_t *client_key = NULL; rend_authorized_client_t *client = NULL; smartlist_clear(client_cookies); switch (service->auth_type) { @@ -1795,7 +2032,7 @@ upload_service_descriptor(rend_service_t *service) } /* Post the current descriptors to the hidden service directories. */ rend_get_service_id(service->desc->pk, serviceid); - log_info(LD_REND, "Sending publish request for hidden service %s", + log_info(LD_REND, "Launching upload for hidden service %s", serviceid); directory_post_to_hs_dir(service->desc, descs, serviceid, seconds_valid); @@ -1849,6 +2086,64 @@ upload_service_descriptor(rend_service_t *service) service->desc_is_dirty = 0; } +/** Return the number of INTRODUCE2 cells this hidden service has received + * from this intro point. */ +static int +intro_point_accepted_intro_count(rend_intro_point_t *intro) +{ + if (intro->accepted_intro_rsa_parts == NULL) { + return 0; + } else { + return digestmap_size(intro->accepted_intro_rsa_parts); + } +} + +/** Return non-zero iff <b>intro</b> should 'expire' now (i.e. we + * should stop publishing it in new descriptors and eventually close + * it). */ +static int +intro_point_should_expire_now(rend_intro_point_t *intro, + time_t now) +{ + tor_assert(intro != NULL); + + if (intro->time_published == -1) { + /* Don't expire an intro point if we haven't even published it yet. */ + return 0; + } + + if (intro->time_expiring != -1) { + /* We've already started expiring this intro point. *Don't* let + * this function's result 'flap'. */ + return 1; + } + + if (intro_point_accepted_intro_count(intro) >= + INTRO_POINT_LIFETIME_INTRODUCTIONS) { + /* This intro point has been used too many times. Expire it now. */ + return 1; + } + + if (intro->time_to_expire == -1) { + /* This intro point has been published, but we haven't picked an + * expiration time for it. Pick one now. */ + int intro_point_lifetime_seconds = + INTRO_POINT_LIFETIME_MIN_SECONDS + + crypto_rand_int(INTRO_POINT_LIFETIME_MAX_SECONDS - + INTRO_POINT_LIFETIME_MIN_SECONDS); + + /* Start the expiration timer now, rather than when the intro + * point was first published. There shouldn't be much of a time + * difference. */ + intro->time_to_expire = now + intro_point_lifetime_seconds; + + return 0; + } + + /* This intro point has a time to expire set already. Use it. */ + return (now >= intro->time_to_expire); +} + /** For every service, check how many intro points it currently has, and: * - Pick new intro points as necessary. * - Launch circuits to any new intro points. @@ -1857,23 +2152,34 @@ void rend_services_introduce(void) { int i,j,r; - routerinfo_t *router; + const node_t *node; rend_service_t *service; rend_intro_point_t *intro; - int changed, prev_intro_nodes; - smartlist_t *intro_routers; + int intro_point_set_changed, prev_intro_nodes; + unsigned int n_intro_points_unexpired; + unsigned int n_intro_points_to_open; + smartlist_t *intro_nodes; time_t now; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); - intro_routers = smartlist_create(); + intro_nodes = smartlist_new(); now = time(NULL); for (i=0; i < smartlist_len(rend_service_list); ++i) { - smartlist_clear(intro_routers); + smartlist_clear(intro_nodes); service = smartlist_get(rend_service_list, i); tor_assert(service); - changed = 0; + + /* intro_point_set_changed becomes non-zero iff the set of intro + * points to be published in service's descriptor has changed. */ + intro_point_set_changed = 0; + + /* n_intro_points_unexpired collects the number of non-expiring + * intro points we have, so that we know how many new intro + * circuits we need to launch for this service. */ + n_intro_points_unexpired = 0; + if (now > service->intro_period_started+INTRO_CIRC_RETRY_PERIOD) { /* One period has elapsed; we can try building circuits again. */ service->intro_period_started = now; @@ -1887,84 +2193,141 @@ rend_services_introduce(void) /* Find out which introduction points we have in progress for this service. */ - for (j=0; j < smartlist_len(service->intro_nodes); ++j) { - intro = smartlist_get(service->intro_nodes, j); - router = router_get_by_digest(intro->extend_info->identity_digest); - if (!router || !find_intro_circuit(intro, service->pk_digest)) { - log_info(LD_REND,"Giving up on %s as intro point for %s.", + SMARTLIST_FOREACH_BEGIN(service->intro_nodes, rend_intro_point_t *, + intro) { + origin_circuit_t *intro_circ = + find_intro_circuit(intro, service->pk_digest); + + if (intro->time_expiring + INTRO_POINT_EXPIRATION_GRACE_PERIOD > now) { + /* This intro point has completely expired. Remove it, and + * mark the circuit for close if it's still alive. */ + if (intro_circ != NULL) { + circuit_mark_for_close(TO_CIRCUIT(intro_circ), + END_CIRC_REASON_FINISHED); + } + rend_intro_point_free(intro); + intro = NULL; /* SMARTLIST_DEL_CURRENT takes a name, not a value. */ + SMARTLIST_DEL_CURRENT(service->intro_nodes, intro); + /* We don't need to set intro_point_set_changed here, because + * this intro point wouldn't have been published in a current + * descriptor anyway. */ + continue; + } + + node = node_get_by_id(intro->extend_info->identity_digest); + if (!node || !intro_circ) { + int removing_this_intro_point_changes_the_intro_point_set = 1; + log_info(LD_REND, "Giving up on %s as intro point for %s" + " (circuit disappeared).", safe_str_client(extend_info_describe(intro->extend_info)), safe_str_client(service->service_id)); - if (service->desc) { - SMARTLIST_FOREACH(service->desc->intro_nodes, rend_intro_point_t *, - dintro, { - if (tor_memeq(dintro->extend_info->identity_digest, - intro->extend_info->identity_digest, DIGEST_LEN)) { - log_info(LD_REND, "The intro point we are giving up on was " - "included in the last published descriptor. " - "Marking current descriptor as dirty."); - service->desc_is_dirty = now; - } - }); + rend_service_note_removing_intro_point(service, intro); + if (intro->time_expiring != -1) { + log_info(LD_REND, "We were already expiring the intro point; " + "no need to mark the HS descriptor as dirty over this."); + removing_this_intro_point_changes_the_intro_point_set = 0; + } else if (intro->listed_in_last_desc) { + log_info(LD_REND, "The intro point we are giving up on was " + "included in the last published descriptor. " + "Marking current descriptor as dirty."); + service->desc_is_dirty = now; } rend_intro_point_free(intro); - smartlist_del(service->intro_nodes,j--); - changed = 1; + intro = NULL; /* SMARTLIST_DEL_CURRENT takes a name, not a value. */ + SMARTLIST_DEL_CURRENT(service->intro_nodes, intro); + if (removing_this_intro_point_changes_the_intro_point_set) + intro_point_set_changed = 1; } - if (router) - smartlist_add(intro_routers, router); - } - /* We have enough intro points, and the intro points we thought we had were - * all connected. - */ - if (!changed && smartlist_len(service->intro_nodes) >= NUM_INTRO_POINTS) { - /* We have all our intro points! Start a fresh period and reset the - * circuit count. */ - service->intro_period_started = now; - service->n_intro_circuits_launched = 0; + if (intro != NULL && intro_point_should_expire_now(intro, now)) { + log_info(LD_REND, "Expiring %s as intro point for %s.", + safe_str_client(extend_info_describe(intro->extend_info)), + safe_str_client(service->service_id)); + + rend_service_note_removing_intro_point(service, intro); + + /* The polite (and generally Right) way to expire an intro + * point is to establish a new one to replace it, publish a + * new descriptor that doesn't list any expiring intro points, + * and *then*, once our upload attempts for the new descriptor + * have ended (whether in success or failure), close the + * expiring intro points. + * + * Unfortunately, we can't find out when the new descriptor + * has actually been uploaded, so we'll have to settle for a + * five-minute timer. Start it. XXXX024 This sucks. */ + intro->time_expiring = now; + + intro_point_set_changed = 1; + } + + if (intro != NULL && intro->time_expiring == -1) + ++n_intro_points_unexpired; + + if (node) + smartlist_add(intro_nodes, (void*)node); + } SMARTLIST_FOREACH_END(intro); + + if (!intro_point_set_changed && + (n_intro_points_unexpired >= service->n_intro_points_wanted)) { continue; } - /* Remember how many introduction circuits we started with. */ + /* Remember how many introduction circuits we started with. + * + * prev_intro_nodes serves a different purpose than + * n_intro_points_unexpired -- this variable tells us where our + * previously-created intro points end and our new ones begin in + * the intro-point list, so we don't have to launch the circuits + * at the same time as we create the intro points they correspond + * to. XXXX This is daft. */ prev_intro_nodes = smartlist_len(service->intro_nodes); + /* We have enough directory information to start establishing our - * intro points. We want to end up with three intro points, but if - * we're just starting, we launch five and pick the first three that - * complete. + * intro points. We want to end up with n_intro_points_wanted + * intro points, but if we're just starting, we launch two extra + * circuits and use the first n_intro_points_wanted that complete. * * The ones after the first three will be converted to 'general' * internal circuits in rend_service_intro_has_opened(), and then * we'll drop them from the list of intro points next time we * go through the above "find out which introduction points we have * in progress" loop. */ -#define NUM_INTRO_POINTS_INIT (NUM_INTRO_POINTS + 2) - for (j=prev_intro_nodes; j < (prev_intro_nodes == 0 ? - NUM_INTRO_POINTS_INIT : NUM_INTRO_POINTS); ++j) { - router_crn_flags_t flags = CRN_NEED_UPTIME; + n_intro_points_to_open = (service->n_intro_points_wanted + + (prev_intro_nodes == 0 ? 2 : 0)); + for (j = (int)n_intro_points_unexpired; + j < (int)n_intro_points_to_open; + ++j) { /* XXXX remove casts */ + router_crn_flags_t flags = CRN_NEED_UPTIME|CRN_NEED_DESC; if (get_options()->_AllowInvalid & ALLOW_INVALID_INTRODUCTION) flags |= CRN_ALLOW_INVALID; - router = router_choose_random_node(intro_routers, - options->ExcludeNodes, flags); - if (!router) { + node = router_choose_random_node(intro_nodes, + options->ExcludeNodes, flags); + if (!node) { log_warn(LD_REND, - "Could only establish %d introduction points for %s.", - smartlist_len(service->intro_nodes), service->service_id); + "Could only establish %d introduction points for %s; " + "wanted %u.", + smartlist_len(service->intro_nodes), service->service_id, + n_intro_points_to_open); break; } - changed = 1; - smartlist_add(intro_routers, router); + intro_point_set_changed = 1; + smartlist_add(intro_nodes, (void*)node); intro = tor_malloc_zero(sizeof(rend_intro_point_t)); - intro->extend_info = extend_info_from_router(router); - intro->intro_key = crypto_new_pk_env(); + intro->extend_info = extend_info_from_node(node, 0); + intro->intro_key = crypto_pk_new(); tor_assert(!crypto_pk_generate_key(intro->intro_key)); + intro->time_published = -1; + intro->time_to_expire = -1; + intro->time_expiring = -1; smartlist_add(service->intro_nodes, intro); log_info(LD_REND, "Picked router %s as an intro point for %s.", - safe_str_client(router_describe(router)), + safe_str_client(node_describe(node)), safe_str_client(service->service_id)); } /* If there's no need to launch new circuits, stop here. */ - if (!changed) + if (!intro_point_set_changed) continue; /* Establish new introduction points. */ @@ -1978,7 +2341,7 @@ rend_services_introduce(void) } } } - smartlist_free(intro_routers); + smartlist_free(intro_nodes); } /** Regenerate and upload rendezvous service descriptors for all @@ -2118,7 +2481,7 @@ rend_service_set_connection_addr_port(edge_connection_t *conn, serviceid, circ->_base.n_circ_id); return -1; } - matching_ports = smartlist_create(); + matching_ports = smartlist_new(); SMARTLIST_FOREACH(service->ports, rend_service_port_config_t *, p, { if (conn->_base.port == p->virtual_port) { diff --git a/src/or/rendservice.h b/src/or/rendservice.h index 70389afe9a..e5848785a8 100644 --- a/src/or/rendservice.h +++ b/src/or/rendservice.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -13,7 +13,7 @@ #define _TOR_RENDSERVICE_H int num_rend_services(void); -int rend_config_services(or_options_t *options, int validate_only); +int rend_config_services(const or_options_t *options, int validate_only); int rend_service_load_keys(void); void rend_services_introduce(void); void rend_consider_services_upload(time_t now); diff --git a/src/or/rephist.c b/src/or/rephist.c index 1dd3d94fd0..3b0d9dd35f 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -1,5 +1,5 @@ /* Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -7,7 +7,7 @@ * \brief Basic history and "reputation" functionality to remember * which servers have worked in the past, how much bandwidth we've * been using, which ports we tend to want, and so on; further, - * exit port statistics and cell statistics. + * exit port statistics, cell statistics, and connection statistics. **/ #include "or.h" @@ -15,6 +15,7 @@ #include "circuituse.h" #include "config.h" #include "networkstatus.h" +#include "nodelist.h" #include "rephist.h" #include "router.h" #include "routerlist.h" @@ -587,7 +588,7 @@ rep_hist_get_weighted_time_known(const char *id, time_t when) int rep_hist_have_measured_enough_stability(void) { - /* XXXX022 This doesn't do so well when we change our opinion + /* XXXX023 This doesn't do so well when we change our opinion * as to whether we're tracking router stability. */ return started_tracking_stability < time(NULL) - 4*60*60; } @@ -643,7 +644,7 @@ rep_hist_dump_stats(time_t now, int severity) size_t len; int ret; unsigned long upt, downt; - routerinfo_t *r; + const node_t *node; rep_history_clean(now - get_options()->RephistTrackTime); @@ -657,8 +658,8 @@ rep_hist_dump_stats(time_t now, int severity) digestmap_iter_get(orhist_it, &digest1, &or_history_p); or_history = (or_history_t*) or_history_p; - if ((r = router_get_by_digest(digest1))) - name1 = r->nickname; + if ((node = node_get_by_id(digest1)) && node_get_nickname(node)) + name1 = node_get_nickname(node); else name1 = "(unknown)"; base16_encode(hexdigest1, sizeof(hexdigest1), digest1, DIGEST_LEN); @@ -688,8 +689,8 @@ rep_hist_dump_stats(time_t now, int severity) lhist_it = digestmap_iter_next(or_history->link_history_map, lhist_it)) { digestmap_iter_get(lhist_it, &digest2, &link_history_p); - if ((r = router_get_by_digest(digest2))) - name2 = r->nickname; + if ((node = node_get_by_id(digest2)) && node_get_nickname(node)) + name2 = node_get_nickname(node); else name2 = "(unknown)"; @@ -823,7 +824,7 @@ rep_hist_record_mtbf_data(time_t now, int missing_means_down) base16_encode(dbuf, sizeof(dbuf), digest, DIGEST_LEN); if (missing_means_down && hist->start_of_run && - !router_get_by_digest(digest)) { + !router_get_by_id_digest(digest)) { /* We think this relay is running, but it's not listed in our * routerlist. Somehow it fell out without telling us it went * down. Complain and also correct it. */ @@ -838,7 +839,7 @@ rep_hist_record_mtbf_data(time_t now, int missing_means_down) format_iso_time(time_buf, hist->start_of_run); t = time_buf; } - PRINTF((f, "+MTBF %lu %.5lf%s%s\n", + PRINTF((f, "+MTBF %lu %.5f%s%s\n", hist->weighted_run_length, hist->total_run_weights, t ? " S=" : "", t ? t : "")); t = NULL; @@ -888,10 +889,10 @@ rep_hist_format_router_status(or_history_t *hist, time_t now) tor_asprintf(&cp, "%s%s%s" "%s%s%s" - "wfu %0.3lf\n" + "wfu %0.3f\n" " weighted-time %lu\n" " weighted-uptime %lu\n" - "mtbf %0.1lf\n" + "mtbf %0.1f\n" " weighted-run-length %lu\n" " total-run-weights %f\n", up?"uptime-started ":"", up?sor_buf:"", up?" UTC\n":"", @@ -929,7 +930,7 @@ rep_hist_get_router_stability_doc(time_t now) return NULL; tor_free(last_stability_doc); - chunks = smartlist_create(); + chunks = smartlist_new(); if (rep_hist_have_measured_enough_stability()) { smartlist_add(chunks, tor_strdup("we-have-enough-measurements\n")); @@ -938,33 +939,35 @@ rep_hist_get_router_stability_doc(time_t now) } DIGESTMAP_FOREACH(history_map, id, or_history_t *, hist) { - routerinfo_t *ri; + const node_t *node; char dbuf[BASE64_DIGEST_LEN+1]; - char header_buf[512]; char *info; digest_to_base64(dbuf, id); - ri = router_get_by_digest(id); - if (ri) { - char *ip = tor_dup_ip(ri->addr); + node = node_get_by_id(id); + if (node) { + char ip[INET_NTOA_BUF_LEN+1]; char tbuf[ISO_TIME_LEN+1]; - format_iso_time(tbuf, ri->cache_info.published_on); - tor_snprintf(header_buf, sizeof(header_buf), + time_t published = node_get_published_on(node); + node_get_address_string(node,ip,sizeof(ip)); + if (published > 0) + format_iso_time(tbuf, published); + else + strlcpy(tbuf, "???", sizeof(tbuf)); + smartlist_add_asprintf(chunks, "router %s %s %s\n" "published %s\n" "relevant-flags %s%s%s\n" "declared-uptime %ld\n", - dbuf, ri->nickname, ip, + dbuf, node_get_nickname(node), ip, tbuf, - ri->is_running ? "Running " : "", - ri->is_valid ? "Valid " : "", - ri->is_hibernating ? "Hibernating " : "", - ri->uptime); - tor_free(ip); + node->is_running ? "Running " : "", + node->is_valid ? "Valid " : "", + node->ri && node->ri->is_hibernating ? "Hibernating " : "", + node_get_declared_uptime(node)); } else { - tor_snprintf(header_buf, sizeof(header_buf), + smartlist_add_asprintf(chunks, "router %s {no descriptor}\n", dbuf); } - smartlist_add(chunks, tor_strdup(header_buf)); info = rep_hist_format_router_status(hist, now); if (info) smartlist_add(chunks, info); @@ -1058,7 +1061,7 @@ rep_hist_load_mtbf_data(time_t now) tor_free(filename); if (!d) return -1; - lines = smartlist_create(); + lines = smartlist_new(); smartlist_split_string(lines, d, "\n", SPLIT_SKIP_SPACE, 0); tor_free(d); } @@ -1474,7 +1477,7 @@ rep_hist_fill_bandwidth_history(char *buf, size_t len, const bw_array_t *b) { char *cp = buf; int i, n; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); uint64_t cutoff; if (b->num_maxes_set <= b->next_max_idx) { @@ -1583,7 +1586,6 @@ rep_hist_update_bwhist_state_section(or_state_t *state, time_t *s_begins, int *s_interval) { - char *cp; int i,j; uint64_t maxval; @@ -1607,31 +1609,31 @@ rep_hist_update_bwhist_state_section(or_state_t *state, } *s_begins = 0; *s_interval = 900; - *s_values = smartlist_create(); - *s_maxima = smartlist_create(); + *s_values = smartlist_new(); + *s_maxima = smartlist_new(); return; } *s_begins = b->next_period; *s_interval = NUM_SECS_BW_SUM_INTERVAL; - *s_values = smartlist_create(); - *s_maxima = smartlist_create(); + *s_values = smartlist_new(); + *s_maxima = smartlist_new(); /* Set i to first position in circular array */ i = (b->num_maxes_set <= b->next_max_idx) ? 0 : b->next_max_idx; for (j=0; j < b->num_maxes_set; ++j,++i) { if (i >= NUM_TOTALS) i = 0; - tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(b->totals[i] & ~0x3ff)); - smartlist_add(*s_values, cp); + smartlist_add_asprintf(*s_values, U64_FORMAT, + U64_PRINTF_ARG(b->totals[i] & ~0x3ff)); maxval = b->maxima[i] / NUM_SECS_ROLLING_MEASURE; - tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(maxval & ~0x3ff)); - smartlist_add(*s_maxima, cp); + smartlist_add_asprintf(*s_maxima, U64_FORMAT, + U64_PRINTF_ARG(maxval & ~0x3ff)); } - tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(b->total_in_period & ~0x3ff)); - smartlist_add(*s_values, cp); + smartlist_add_asprintf(*s_values, U64_FORMAT, + U64_PRINTF_ARG(b->total_in_period & ~0x3ff)); maxval = b->max_total / NUM_SECS_ROLLING_MEASURE; - tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(maxval & ~0x3ff)); - smartlist_add(*s_maxima, cp); + smartlist_add_asprintf(*s_maxima, U64_FORMAT, + U64_PRINTF_ARG(maxval & ~0x3ff)); } /** Update <b>state</b> with the newest bandwidth history. Done before @@ -1673,7 +1675,8 @@ rep_hist_load_bwhist_state_section(bw_array_t *b, uint64_t v, mv; int i,ok,ok_m; - int have_maxima = (smartlist_len(s_values) == smartlist_len(s_maxima)); + int have_maxima = s_maxima && s_values && + (smartlist_len(s_values) == smartlist_len(s_maxima)); if (s_values && s_begins >= now - NUM_SECS_BW_SUM_INTERVAL*NUM_TOTALS) { start = s_begins - s_interval*(smartlist_len(s_values)); @@ -1770,10 +1773,18 @@ rep_hist_load_state(or_state_t *state, char **err) /*********************************************************************/ +/** A single predicted port: used to remember which ports we've made + * connections to, so that we can try to keep making circuits that can handle + * those ports. */ +typedef struct predicted_port_t { + /** The port we connected to */ + uint16_t port; + /** The time at which we last used it */ + time_t time; +} predicted_port_t; + /** A list of port numbers that have been used recently. */ static smartlist_t *predicted_ports_list=NULL; -/** The corresponding most recently used time for each port. */ -static smartlist_t *predicted_ports_times=NULL; /** We just got an application request for a connection with * port <b>port</b>. Remember it for the future, so we can keep @@ -1782,14 +1793,11 @@ static smartlist_t *predicted_ports_times=NULL; static void add_predicted_port(time_t now, uint16_t port) { - /* XXXX we could just use uintptr_t here, I think. -NM */ - uint16_t *tmp_port = tor_malloc(sizeof(uint16_t)); - time_t *tmp_time = tor_malloc(sizeof(time_t)); - *tmp_port = port; - *tmp_time = now; - rephist_total_alloc += sizeof(uint16_t) + sizeof(time_t); - smartlist_add(predicted_ports_list, tmp_port); - smartlist_add(predicted_ports_times, tmp_time); + predicted_port_t *pp = tor_malloc(sizeof(predicted_port_t)); + pp->port = port; + pp->time = now; + rephist_total_alloc += sizeof(*pp); + smartlist_add(predicted_ports_list, pp); } /** Initialize whatever memory and structs are needed for predicting @@ -1799,8 +1807,7 @@ add_predicted_port(time_t now, uint16_t port) static void predicted_ports_init(void) { - predicted_ports_list = smartlist_create(); - predicted_ports_times = smartlist_create(); + predicted_ports_list = smartlist_new(); add_predicted_port(time(NULL), 80); /* add one to kickstart us */ } @@ -1810,12 +1817,11 @@ predicted_ports_init(void) static void predicted_ports_free(void) { - rephist_total_alloc -= smartlist_len(predicted_ports_list)*sizeof(uint16_t); - SMARTLIST_FOREACH(predicted_ports_list, char *, cp, tor_free(cp)); + rephist_total_alloc -= + smartlist_len(predicted_ports_list)*sizeof(predicted_port_t); + SMARTLIST_FOREACH(predicted_ports_list, predicted_port_t *, + pp, tor_free(pp)); smartlist_free(predicted_ports_list); - rephist_total_alloc -= smartlist_len(predicted_ports_times)*sizeof(time_t); - SMARTLIST_FOREACH(predicted_ports_times, char *, cp, tor_free(cp)); - smartlist_free(predicted_ports_times); } /** Remember that <b>port</b> has been asked for as of time <b>now</b>. @@ -1825,24 +1831,17 @@ predicted_ports_free(void) void rep_hist_note_used_port(time_t now, uint16_t port) { - int i; - uint16_t *tmp_port; - time_t *tmp_time; - tor_assert(predicted_ports_list); - tor_assert(predicted_ports_times); if (!port) /* record nothing */ return; - for (i = 0; i < smartlist_len(predicted_ports_list); ++i) { - tmp_port = smartlist_get(predicted_ports_list, i); - tmp_time = smartlist_get(predicted_ports_times, i); - if (*tmp_port == port) { - *tmp_time = now; + SMARTLIST_FOREACH_BEGIN(predicted_ports_list, predicted_port_t *, pp) { + if (pp->port == port) { + pp->time = now; return; } - } + } SMARTLIST_FOREACH_END(pp); /* it's not there yet; we need to add it */ add_predicted_port(now, port); } @@ -1851,36 +1850,48 @@ rep_hist_note_used_port(time_t now, uint16_t port) * we'll want to make connections to the same port in the future. */ #define PREDICTED_CIRCS_RELEVANCE_TIME (60*60) -/** Return a pointer to the list of port numbers that +/** Return a newly allocated pointer to a list of uint16_t * for ports that * are likely to be asked for in the near future. - * - * The caller promises not to mess with it. */ smartlist_t * rep_hist_get_predicted_ports(time_t now) { - int i; - uint16_t *tmp_port; - time_t *tmp_time; - + smartlist_t *out = smartlist_new(); tor_assert(predicted_ports_list); - tor_assert(predicted_ports_times); /* clean out obsolete entries */ - for (i = 0; i < smartlist_len(predicted_ports_list); ++i) { - tmp_time = smartlist_get(predicted_ports_times, i); - if (*tmp_time + PREDICTED_CIRCS_RELEVANCE_TIME < now) { - tmp_port = smartlist_get(predicted_ports_list, i); - log_debug(LD_CIRC, "Expiring predicted port %d", *tmp_port); - smartlist_del(predicted_ports_list, i); - smartlist_del(predicted_ports_times, i); - rephist_total_alloc -= sizeof(uint16_t)+sizeof(time_t); - tor_free(tmp_port); - tor_free(tmp_time); - i--; + SMARTLIST_FOREACH_BEGIN(predicted_ports_list, predicted_port_t *, pp) { + if (pp->time + PREDICTED_CIRCS_RELEVANCE_TIME < now) { + log_debug(LD_CIRC, "Expiring predicted port %d", pp->port); + + rephist_total_alloc -= sizeof(predicted_port_t); + tor_free(pp); + SMARTLIST_DEL_CURRENT(predicted_ports_list, pp); + } else { + smartlist_add(out, tor_memdup(&pp->port, sizeof(uint16_t))); } - } - return predicted_ports_list; + } SMARTLIST_FOREACH_END(pp); + return out; +} + +/** + * Take a list of uint16_t *, and remove every port in the list from the + * current list of predicted ports. + */ +void +rep_hist_remove_predicted_ports(const smartlist_t *rmv_ports) +{ + /* Let's do this on O(N), not O(N^2). */ + bitarray_t *remove_ports = bitarray_init_zero(UINT16_MAX); + SMARTLIST_FOREACH(rmv_ports, const uint16_t *, p, + bitarray_set(remove_ports, *p)); + SMARTLIST_FOREACH_BEGIN(predicted_ports_list, predicted_port_t *, pp) { + if (bitarray_is_set(remove_ports, pp->port)) { + tor_free(pp); + SMARTLIST_DEL_CURRENT(predicted_ports_list, pp); + } + } SMARTLIST_FOREACH_END(pp); + bitarray_free(remove_ports); } /** The user asked us to do a resolve. Rather than keeping track of @@ -2116,7 +2127,9 @@ rep_hist_exit_stats_term(void) tor_free(exit_streams); } -/** Helper for qsort: compare two ints. */ +/** Helper for qsort: compare two ints. Does not handle overflow properly, + * but works fine for sorting an array of port numbers, which is what we use + * it for. */ static int _compare_int(const void *x, const void *y) { @@ -2124,7 +2137,8 @@ _compare_int(const void *x, const void *y) } /** Return a newly allocated string containing the exit port statistics - * until <b>now</b>, or NULL if we're not collecting exit stats. */ + * until <b>now</b>, or NULL if we're not collecting exit stats. Caller + * must ensure start_of_exit_stats_interval is in the past. */ char * rep_hist_format_exit_stats(time_t now) { @@ -2134,7 +2148,6 @@ rep_hist_format_exit_stats(time_t now) uint64_t cur_bytes = 0, other_read = 0, other_written = 0, total_read = 0, total_written = 0; uint32_t total_streams = 0, other_streams = 0; - char *buf; smartlist_t *written_strings, *read_strings, *streams_strings; char *written_string, *read_string, *streams_string; char t[ISO_TIME_LEN+1]; @@ -2143,6 +2156,8 @@ rep_hist_format_exit_stats(time_t now) if (!start_of_exit_stats_interval) return NULL; /* Not initialized. */ + tor_assert(now >= start_of_exit_stats_interval); + /* Go through all ports to find the n ports that saw most written and * read bytes. * @@ -2195,9 +2210,9 @@ rep_hist_format_exit_stats(time_t now) } /* Add observations of top ports to smartlists. */ - written_strings = smartlist_create(); - read_strings = smartlist_create(); - streams_strings = smartlist_create(); + written_strings = smartlist_new(); + read_strings = smartlist_new(); + streams_strings = smartlist_new(); other_read = total_read; other_written = total_written; other_streams = total_streams; @@ -2211,9 +2226,8 @@ rep_hist_format_exit_stats(time_t now) exit_bytes_written[cur_port], EXIT_STATS_ROUND_UP_BYTES); num /= 1024; - buf = NULL; - tor_asprintf(&buf, "%d="U64_FORMAT, cur_port, U64_PRINTF_ARG(num)); - smartlist_add(written_strings, buf); + smartlist_add_asprintf(written_strings, "%d="U64_FORMAT, + cur_port, U64_PRINTF_ARG(num)); other_written -= exit_bytes_written[cur_port]; } if (exit_bytes_read[cur_port] > 0) { @@ -2221,18 +2235,15 @@ rep_hist_format_exit_stats(time_t now) exit_bytes_read[cur_port], EXIT_STATS_ROUND_UP_BYTES); num /= 1024; - buf = NULL; - tor_asprintf(&buf, "%d="U64_FORMAT, cur_port, U64_PRINTF_ARG(num)); - smartlist_add(read_strings, buf); + smartlist_add_asprintf(read_strings, "%d="U64_FORMAT, + cur_port, U64_PRINTF_ARG(num)); other_read -= exit_bytes_read[cur_port]; } if (exit_streams[cur_port] > 0) { uint32_t num = round_uint32_to_next_multiple_of( exit_streams[cur_port], EXIT_STATS_ROUND_UP_STREAMS); - buf = NULL; - tor_asprintf(&buf, "%d=%u", cur_port, num); - smartlist_add(streams_strings, buf); + smartlist_add_asprintf(streams_strings, "%d=%u", cur_port, num); other_streams -= exit_streams[cur_port]; } } @@ -2241,20 +2252,16 @@ rep_hist_format_exit_stats(time_t now) other_written = round_uint64_to_next_multiple_of(other_written, EXIT_STATS_ROUND_UP_BYTES); other_written /= 1024; - buf = NULL; - tor_asprintf(&buf, "other="U64_FORMAT, U64_PRINTF_ARG(other_written)); - smartlist_add(written_strings, buf); + smartlist_add_asprintf(written_strings, "other="U64_FORMAT, + U64_PRINTF_ARG(other_written)); other_read = round_uint64_to_next_multiple_of(other_read, EXIT_STATS_ROUND_UP_BYTES); other_read /= 1024; - buf = NULL; - tor_asprintf(&buf, "other="U64_FORMAT, U64_PRINTF_ARG(other_read)); - smartlist_add(read_strings, buf); + smartlist_add_asprintf(read_strings, "other="U64_FORMAT, + U64_PRINTF_ARG(other_read)); other_streams = round_uint32_to_next_multiple_of(other_streams, EXIT_STATS_ROUND_UP_STREAMS); - buf = NULL; - tor_asprintf(&buf, "other=%u", other_streams); - smartlist_add(streams_strings, buf); + smartlist_add_asprintf(streams_strings, "other=%u", other_streams); /* Join all observations in single strings. */ written_string = smartlist_join_strings(written_strings, ",", 0, NULL); @@ -2374,41 +2381,60 @@ typedef struct circ_buffer_stats_t { /** List of circ_buffer_stats_t. */ static smartlist_t *circuits_for_buffer_stats = NULL; +/** Remember cell statistics <b>mean_num_cells_in_queue</b>, + * <b>mean_time_cells_in_queue</b>, and <b>processed_cells</b> of a + * circuit. */ +void +rep_hist_add_buffer_stats(double mean_num_cells_in_queue, + double mean_time_cells_in_queue, uint32_t processed_cells) +{ + circ_buffer_stats_t *stat; + if (!start_of_buffer_stats_interval) + return; /* Not initialized. */ + stat = tor_malloc_zero(sizeof(circ_buffer_stats_t)); + stat->mean_num_cells_in_queue = mean_num_cells_in_queue; + stat->mean_time_cells_in_queue = mean_time_cells_in_queue; + stat->processed_cells = processed_cells; + if (!circuits_for_buffer_stats) + circuits_for_buffer_stats = smartlist_new(); + smartlist_add(circuits_for_buffer_stats, stat); +} + /** Remember cell statistics for circuit <b>circ</b> at time * <b>end_of_interval</b> and reset cell counters in case the circuit * remains open in the next measurement interval. */ void rep_hist_buffer_stats_add_circ(circuit_t *circ, time_t end_of_interval) { - circ_buffer_stats_t *stat; time_t start_of_interval; int interval_length; or_circuit_t *orcirc; + double mean_num_cells_in_queue, mean_time_cells_in_queue; + uint32_t processed_cells; if (CIRCUIT_IS_ORIGIN(circ)) return; orcirc = TO_OR_CIRCUIT(circ); if (!orcirc->processed_cells) return; - if (!circuits_for_buffer_stats) - circuits_for_buffer_stats = smartlist_create(); - start_of_interval = circ->timestamp_created.tv_sec > - start_of_buffer_stats_interval ? + start_of_interval = (circ->timestamp_created.tv_sec > + start_of_buffer_stats_interval) ? circ->timestamp_created.tv_sec : start_of_buffer_stats_interval; interval_length = (int) (end_of_interval - start_of_interval); if (interval_length <= 0) return; - stat = tor_malloc_zero(sizeof(circ_buffer_stats_t)); - stat->processed_cells = orcirc->processed_cells; + processed_cells = orcirc->processed_cells; /* 1000.0 for s -> ms; 2.0 because of app-ward and exit-ward queues */ - stat->mean_num_cells_in_queue = (double) orcirc->total_cell_waiting_time / + mean_num_cells_in_queue = (double) orcirc->total_cell_waiting_time / (double) interval_length / 1000.0 / 2.0; - stat->mean_time_cells_in_queue = + mean_time_cells_in_queue = (double) orcirc->total_cell_waiting_time / (double) orcirc->processed_cells; - smartlist_add(circuits_for_buffer_stats, stat); orcirc->total_cell_waiting_time = 0; orcirc->processed_cells = 0; + rep_hist_add_buffer_stats(mean_num_cells_in_queue, + mean_time_cells_in_queue, + processed_cells); } /** Sorting helper: return -1, 1, or 0 based on comparison of two @@ -2430,136 +2456,543 @@ _buffer_stats_compare_entries(const void **_a, const void **_b) void rep_hist_buffer_stats_term(void) { - start_of_buffer_stats_interval = 0; + rep_hist_reset_buffer_stats(0); +} + +/** Clear history of circuit statistics and set the measurement interval + * start to <b>now</b>. */ +void +rep_hist_reset_buffer_stats(time_t now) +{ if (!circuits_for_buffer_stats) - circuits_for_buffer_stats = smartlist_create(); + circuits_for_buffer_stats = smartlist_new(); SMARTLIST_FOREACH(circuits_for_buffer_stats, circ_buffer_stats_t *, stat, tor_free(stat)); smartlist_clear(circuits_for_buffer_stats); + start_of_buffer_stats_interval = now; } -/** Write buffer statistics to $DATADIR/stats/buffer-stats and return when - * we would next want to write exit stats. */ -time_t -rep_hist_buffer_stats_write(time_t now) +/** Return a newly allocated string containing the buffer statistics until + * <b>now</b>, or NULL if we're not collecting buffer stats. Caller must + * ensure start_of_buffer_stats_interval is in the past. */ +char * +rep_hist_format_buffer_stats(time_t now) { - char *statsdir = NULL, *filename = NULL; - char written[ISO_TIME_LEN+1]; - open_file_t *open_file = NULL; - FILE *out; #define SHARES 10 - int processed_cells[SHARES], circs_in_share[SHARES], - number_of_circuits, i; + uint64_t processed_cells[SHARES]; + uint32_t circs_in_share[SHARES]; + int number_of_circuits, i; double queued_cells[SHARES], time_in_queue[SHARES]; - smartlist_t *str_build = NULL; - char *str = NULL, *buf = NULL; - circuit_t *circ; + smartlist_t *processed_cells_strings, *queued_cells_strings, + *time_in_queue_strings; + char *processed_cells_string, *queued_cells_string, + *time_in_queue_string; + char t[ISO_TIME_LEN+1]; + char *result; if (!start_of_buffer_stats_interval) - return 0; /* Not initialized. */ - if (start_of_buffer_stats_interval + WRITE_STATS_INTERVAL > now) - goto done; /* Not ready to write */ + return NULL; /* Not initialized. */ - str_build = smartlist_create(); + tor_assert(now >= start_of_buffer_stats_interval); - /* add current circuits to stats */ - for (circ = _circuit_get_global_list(); circ; circ = circ->next) - rep_hist_buffer_stats_add_circ(circ, now); - /* calculate deciles */ - memset(processed_cells, 0, SHARES * sizeof(int)); - memset(circs_in_share, 0, SHARES * sizeof(int)); + /* Calculate deciles if we saw at least one circuit. */ + memset(processed_cells, 0, SHARES * sizeof(uint64_t)); + memset(circs_in_share, 0, SHARES * sizeof(uint32_t)); memset(queued_cells, 0, SHARES * sizeof(double)); memset(time_in_queue, 0, SHARES * sizeof(double)); if (!circuits_for_buffer_stats) - circuits_for_buffer_stats = smartlist_create(); - smartlist_sort(circuits_for_buffer_stats, - _buffer_stats_compare_entries); + circuits_for_buffer_stats = smartlist_new(); number_of_circuits = smartlist_len(circuits_for_buffer_stats); - if (number_of_circuits < 1) { - log_info(LD_HIST, "Attempt to write cell statistics to disk failed. " - "We haven't seen a single circuit to report about."); - goto done; + if (number_of_circuits > 0) { + smartlist_sort(circuits_for_buffer_stats, + _buffer_stats_compare_entries); + i = 0; + SMARTLIST_FOREACH_BEGIN(circuits_for_buffer_stats, + circ_buffer_stats_t *, stat) + { + int share = i++ * SHARES / number_of_circuits; + processed_cells[share] += stat->processed_cells; + queued_cells[share] += stat->mean_num_cells_in_queue; + time_in_queue[share] += stat->mean_time_cells_in_queue; + circs_in_share[share]++; + } + SMARTLIST_FOREACH_END(stat); } - i = 0; - SMARTLIST_FOREACH_BEGIN(circuits_for_buffer_stats, - circ_buffer_stats_t *, stat) - { - int share = i++ * SHARES / number_of_circuits; - processed_cells[share] += stat->processed_cells; - queued_cells[share] += stat->mean_num_cells_in_queue; - time_in_queue[share] += stat->mean_time_cells_in_queue; - circs_in_share[share]++; - } - SMARTLIST_FOREACH_END(stat); - /* clear buffer stats history */ - SMARTLIST_FOREACH(circuits_for_buffer_stats, circ_buffer_stats_t *, - stat, tor_free(stat)); - smartlist_clear(circuits_for_buffer_stats); - /* write to file */ - statsdir = get_datadir_fname("stats"); - if (check_private_dir(statsdir, CPD_CREATE, get_options()->User) < 0) - goto done; - filename = get_datadir_fname2("stats", "buffer-stats"); - out = start_writing_to_stdio_file(filename, OPEN_FLAGS_APPEND | O_TEXT, - 0600, &open_file); - if (!out) - goto done; - format_iso_time(written, now); - if (fprintf(out, "cell-stats-end %s (%d s)\n", written, - (unsigned) (now - start_of_buffer_stats_interval)) < 0) - goto done; + + /* Write deciles to strings. */ + processed_cells_strings = smartlist_new(); + queued_cells_strings = smartlist_new(); + time_in_queue_strings = smartlist_new(); for (i = 0; i < SHARES; i++) { - tor_asprintf(&buf,"%d", !circs_in_share[i] ? 0 : - processed_cells[i] / circs_in_share[i]); - smartlist_add(str_build, buf); + smartlist_add_asprintf(processed_cells_strings, + U64_FORMAT, !circs_in_share[i] ? 0 : + U64_PRINTF_ARG(processed_cells[i] / + circs_in_share[i])); } - str = smartlist_join_strings(str_build, ",", 0, NULL); - if (fprintf(out, "cell-processed-cells %s\n", str) < 0) - goto done; - tor_free(str); - SMARTLIST_FOREACH(str_build, char *, c, tor_free(c)); - smartlist_clear(str_build); for (i = 0; i < SHARES; i++) { - tor_asprintf(&buf, "%.2f", circs_in_share[i] == 0 ? 0.0 : - queued_cells[i] / (double) circs_in_share[i]); - smartlist_add(str_build, buf); + smartlist_add_asprintf(queued_cells_strings, "%.2f", + circs_in_share[i] == 0 ? 0.0 : + queued_cells[i] / (double) circs_in_share[i]); } - str = smartlist_join_strings(str_build, ",", 0, NULL); - if (fprintf(out, "cell-queued-cells %s\n", str) < 0) - goto done; - tor_free(str); - SMARTLIST_FOREACH(str_build, char *, c, tor_free(c)); - smartlist_clear(str_build); for (i = 0; i < SHARES; i++) { - tor_asprintf(&buf, "%.0f", circs_in_share[i] == 0 ? 0.0 : - time_in_queue[i] / (double) circs_in_share[i]); - smartlist_add(str_build, buf); + smartlist_add_asprintf(time_in_queue_strings, "%.0f", + circs_in_share[i] == 0 ? 0.0 : + time_in_queue[i] / (double) circs_in_share[i]); } - str = smartlist_join_strings(str_build, ",", 0, NULL); - if (fprintf(out, "cell-time-in-queue %s\n", str) < 0) - goto done; - tor_free(str); - SMARTLIST_FOREACH(str_build, char *, c, tor_free(c)); - smartlist_free(str_build); - str_build = NULL; - if (fprintf(out, "cell-circuits-per-decile %d\n", - (number_of_circuits + SHARES - 1) / SHARES) < 0) + + /* Join all observations in single strings. */ + processed_cells_string = smartlist_join_strings(processed_cells_strings, + ",", 0, NULL); + queued_cells_string = smartlist_join_strings(queued_cells_strings, + ",", 0, NULL); + time_in_queue_string = smartlist_join_strings(time_in_queue_strings, + ",", 0, NULL); + SMARTLIST_FOREACH(processed_cells_strings, char *, cp, tor_free(cp)); + SMARTLIST_FOREACH(queued_cells_strings, char *, cp, tor_free(cp)); + SMARTLIST_FOREACH(time_in_queue_strings, char *, cp, tor_free(cp)); + smartlist_free(processed_cells_strings); + smartlist_free(queued_cells_strings); + smartlist_free(time_in_queue_strings); + + /* Put everything together. */ + format_iso_time(t, now); + tor_asprintf(&result, "cell-stats-end %s (%d s)\n" + "cell-processed-cells %s\n" + "cell-queued-cells %s\n" + "cell-time-in-queue %s\n" + "cell-circuits-per-decile %d\n", + t, (unsigned) (now - start_of_buffer_stats_interval), + processed_cells_string, + queued_cells_string, + time_in_queue_string, + (number_of_circuits + SHARES - 1) / SHARES); + tor_free(processed_cells_string); + tor_free(queued_cells_string); + tor_free(time_in_queue_string); + return result; +#undef SHARES +} + +/** If 24 hours have passed since the beginning of the current buffer + * stats period, write buffer stats to $DATADIR/stats/buffer-stats + * (possibly overwriting an existing file) and reset counters. Return + * when we would next want to write buffer stats or 0 if we never want to + * write. */ +time_t +rep_hist_buffer_stats_write(time_t now) +{ + circuit_t *circ; + char *statsdir = NULL, *filename = NULL, *str = NULL; + + if (!start_of_buffer_stats_interval) + return 0; /* Not initialized. */ + if (start_of_buffer_stats_interval + WRITE_STATS_INTERVAL > now) + goto done; /* Not ready to write */ + + /* Add open circuits to the history. */ + for (circ = _circuit_get_global_list(); circ; circ = circ->next) { + rep_hist_buffer_stats_add_circ(circ, now); + } + + /* Generate history string. */ + str = rep_hist_format_buffer_stats(now); + + /* Reset both buffer history and counters of open circuits. */ + rep_hist_reset_buffer_stats(now); + + /* Try to write to disk. */ + statsdir = get_datadir_fname("stats"); + if (check_private_dir(statsdir, CPD_CREATE, get_options()->User) < 0) { + log_warn(LD_HIST, "Unable to create stats/ directory!"); goto done; - finish_writing_to_file(open_file); - open_file = NULL; - start_of_buffer_stats_interval = now; + } + filename = get_datadir_fname2("stats", "buffer-stats"); + if (write_str_to_file(filename, str, 0) < 0) + log_warn(LD_HIST, "Unable to write buffer stats to disk!"); + done: - if (open_file) - abort_writing_to_file(open_file); + tor_free(str); tor_free(filename); tor_free(statsdir); - if (str_build) { - SMARTLIST_FOREACH(str_build, char *, c, tor_free(c)); - smartlist_free(str_build); + return start_of_buffer_stats_interval + WRITE_STATS_INTERVAL; +} + +/*** Descriptor serving statistics ***/ + +/** Digestmap to track which descriptors were downloaded this stats + * collection interval. It maps descriptor digest to pointers to 1, + * effectively turning this into a list. */ +static digestmap_t *served_descs = NULL; + +/** Number of how many descriptors were downloaded in total during this + * interval. */ +static unsigned long total_descriptor_downloads; + +/** Start time of served descs stats or 0 if we're not collecting those. */ +static time_t start_of_served_descs_stats_interval; + +/** Initialize descriptor stats. */ +void +rep_hist_desc_stats_init(time_t now) +{ + if (served_descs) { + log_warn(LD_BUG, "Called rep_hist_desc_stats_init() when desc stats were " + "already initialized. This is probably harmless."); + return; // Already initialized + } + served_descs = digestmap_new(); + total_descriptor_downloads = 0; + start_of_served_descs_stats_interval = now; +} + +/** Reset served descs stats to empty, starting a new interval <b>now</b>. */ +static void +rep_hist_reset_desc_stats(time_t now) +{ + rep_hist_desc_stats_term(); + rep_hist_desc_stats_init(now); +} + +/** Stop collecting served descs stats, so that rep_hist_desc_stats_init() is + * safe to be called again. */ +void +rep_hist_desc_stats_term(void) +{ + digestmap_free(served_descs, NULL); + served_descs = NULL; + start_of_served_descs_stats_interval = 0; + total_descriptor_downloads = 0; +} + +/** Helper for rep_hist_desc_stats_write(). Return a newly allocated string + * containing the served desc statistics until now, or NULL if we're not + * collecting served desc stats. Caller must ensure that now is not before + * start_of_served_descs_stats_interval. */ +static char * +rep_hist_format_desc_stats(time_t now) +{ + char t[ISO_TIME_LEN+1]; + char *result; + + digestmap_iter_t *iter; + const char *key; + void *val; + unsigned size; + int *vals, max = 0, q3 = 0, md = 0, q1 = 0, min = 0; + int n = 0; + + if (!start_of_served_descs_stats_interval) + return NULL; + + size = digestmap_size(served_descs); + if (size > 0) { + vals = tor_malloc(size * sizeof(int)); + for (iter = digestmap_iter_init(served_descs); + !digestmap_iter_done(iter); + iter = digestmap_iter_next(served_descs, iter)) { + uintptr_t count; + digestmap_iter_get(iter, &key, &val); + count = (uintptr_t)val; + vals[n++] = (int)count; + (void)key; + } + max = find_nth_int(vals, size, size-1); + q3 = find_nth_int(vals, size, (3*size-1)/4); + md = find_nth_int(vals, size, (size-1)/2); + q1 = find_nth_int(vals, size, (size-1)/4); + min = find_nth_int(vals, size, 0); + tor_free(vals); + } + + format_iso_time(t, now); + + tor_asprintf(&result, + "served-descs-stats-end %s (%d s) total=%lu unique=%u " + "max=%d q3=%d md=%d q1=%d min=%d\n", + t, + (unsigned) (now - start_of_served_descs_stats_interval), + total_descriptor_downloads, + size, max, q3, md, q1, min); + + return result; +} + +/** If WRITE_STATS_INTERVAL seconds have passed since the beginning of + * the current served desc stats interval, write the stats to + * $DATADIR/stats/served-desc-stats (possibly appending to an existing file) + * and reset the state for the next interval. Return when we would next want + * to write served desc stats or 0 if we won't want to write. */ +time_t +rep_hist_desc_stats_write(time_t now) +{ + char *statsdir = NULL, *filename = NULL, *str = NULL; + + if (!start_of_served_descs_stats_interval) + return 0; /* We're not collecting stats. */ + if (start_of_served_descs_stats_interval + WRITE_STATS_INTERVAL > now) + return start_of_served_descs_stats_interval + WRITE_STATS_INTERVAL; + + str = rep_hist_format_desc_stats(now); + tor_assert(str != NULL); + + statsdir = get_datadir_fname("stats"); + if (check_private_dir(statsdir, CPD_CREATE, get_options()->User) < 0) { + log_warn(LD_HIST, "Unable to create stats/ directory!"); + goto done; + } + filename = get_datadir_fname2("stats", "served-desc-stats"); + if (append_bytes_to_file(filename, str, strlen(str), 0) < 0) + log_warn(LD_HIST, "Unable to write served descs statistics to disk!"); + + rep_hist_reset_desc_stats(now); + + done: + tor_free(statsdir); + tor_free(filename); + tor_free(str); + return start_of_served_descs_stats_interval + WRITE_STATS_INTERVAL; +} + +/* DOCDOC rep_hist_note_desc_served */ +void +rep_hist_note_desc_served(const char * desc) +{ + void *val; + uintptr_t count; + if (!served_descs) + return; // We're not collecting stats + val = digestmap_get(served_descs, desc); + count = (uintptr_t)val; + if (count != INT_MAX) + ++count; + digestmap_set(served_descs, desc, (void*)count); + total_descriptor_downloads++; +} + +/*** Connection statistics ***/ + +/** Start of the current connection stats interval or 0 if we're not + * collecting connection statistics. */ +static time_t start_of_conn_stats_interval; + +/** Initialize connection stats. */ +void +rep_hist_conn_stats_init(time_t now) +{ + start_of_conn_stats_interval = now; +} + +/* Count connections that we read and wrote less than these many bytes + * from/to as below threshold. */ +#define BIDI_THRESHOLD 20480 + +/* Count connections that we read or wrote at least this factor as many + * bytes from/to than we wrote or read to/from as mostly reading or + * writing. */ +#define BIDI_FACTOR 10 + +/* Interval length in seconds for considering read and written bytes for + * connection stats. */ +#define BIDI_INTERVAL 10 + +/** Start of next BIDI_INTERVAL second interval. */ +static time_t bidi_next_interval = 0; + +/** Number of connections that we read and wrote less than BIDI_THRESHOLD + * bytes from/to in BIDI_INTERVAL seconds. */ +static uint32_t below_threshold = 0; + +/** Number of connections that we read at least BIDI_FACTOR times more + * bytes from than we wrote to in BIDI_INTERVAL seconds. */ +static uint32_t mostly_read = 0; + +/** Number of connections that we wrote at least BIDI_FACTOR times more + * bytes to than we read from in BIDI_INTERVAL seconds. */ +static uint32_t mostly_written = 0; + +/** Number of connections that we read and wrote at least BIDI_THRESHOLD + * bytes from/to, but not BIDI_FACTOR times more in either direction in + * BIDI_INTERVAL seconds. */ +static uint32_t both_read_and_written = 0; + +/** Entry in a map from connection ID to the number of read and written + * bytes on this connection in a BIDI_INTERVAL second interval. */ +typedef struct bidi_map_entry_t { + HT_ENTRY(bidi_map_entry_t) node; + uint64_t conn_id; /**< Connection ID */ + size_t read; /**< Number of read bytes */ + size_t written; /**< Number of written bytes */ +} bidi_map_entry_t; + +/** Map of OR connections together with the number of read and written + * bytes in the current BIDI_INTERVAL second interval. */ +static HT_HEAD(bidimap, bidi_map_entry_t) bidi_map = + HT_INITIALIZER(); + +static int +bidi_map_ent_eq(const bidi_map_entry_t *a, const bidi_map_entry_t *b) +{ + return a->conn_id == b->conn_id; +} + +/* DOCDOC bidi_map_ent_hash */ +static unsigned +bidi_map_ent_hash(const bidi_map_entry_t *entry) +{ + return (unsigned) entry->conn_id; +} + +HT_PROTOTYPE(bidimap, bidi_map_entry_t, node, bidi_map_ent_hash, + bidi_map_ent_eq); +HT_GENERATE(bidimap, bidi_map_entry_t, node, bidi_map_ent_hash, + bidi_map_ent_eq, 0.6, malloc, realloc, free); + +/* DOCDOC bidi_map_free */ +static void +bidi_map_free(void) +{ + bidi_map_entry_t **ptr, **next, *ent; + for (ptr = HT_START(bidimap, &bidi_map); ptr; ptr = next) { + ent = *ptr; + next = HT_NEXT_RMV(bidimap, &bidi_map, ptr); + tor_free(ent); } + HT_CLEAR(bidimap, &bidi_map); +} + +/** Reset counters for conn statistics. */ +void +rep_hist_reset_conn_stats(time_t now) +{ + start_of_conn_stats_interval = now; + below_threshold = 0; + mostly_read = 0; + mostly_written = 0; + both_read_and_written = 0; + bidi_map_free(); +} + +/** Stop collecting connection stats in a way that we can re-start doing + * so in rep_hist_conn_stats_init(). */ +void +rep_hist_conn_stats_term(void) +{ + rep_hist_reset_conn_stats(0); +} + +/** We read <b>num_read</b> bytes and wrote <b>num_written</b> from/to OR + * connection <b>conn_id</b> in second <b>when</b>. If this is the first + * observation in a new interval, sum up the last observations. Add bytes + * for this connection. */ +void +rep_hist_note_or_conn_bytes(uint64_t conn_id, size_t num_read, + size_t num_written, time_t when) +{ + if (!start_of_conn_stats_interval) + return; + /* Initialize */ + if (bidi_next_interval == 0) + bidi_next_interval = when + BIDI_INTERVAL; + /* Sum up last period's statistics */ + if (when >= bidi_next_interval) { + bidi_map_entry_t **ptr, **next, *ent; + for (ptr = HT_START(bidimap, &bidi_map); ptr; ptr = next) { + ent = *ptr; + if (ent->read + ent->written < BIDI_THRESHOLD) + below_threshold++; + else if (ent->read >= ent->written * BIDI_FACTOR) + mostly_read++; + else if (ent->written >= ent->read * BIDI_FACTOR) + mostly_written++; + else + both_read_and_written++; + next = HT_NEXT_RMV(bidimap, &bidi_map, ptr); + tor_free(ent); + } + while (when >= bidi_next_interval) + bidi_next_interval += BIDI_INTERVAL; + log_info(LD_GENERAL, "%d below threshold, %d mostly read, " + "%d mostly written, %d both read and written.", + below_threshold, mostly_read, mostly_written, + both_read_and_written); + } + /* Add this connection's bytes. */ + if (num_read > 0 || num_written > 0) { + bidi_map_entry_t *entry, lookup; + lookup.conn_id = conn_id; + entry = HT_FIND(bidimap, &bidi_map, &lookup); + if (entry) { + entry->written += num_written; + entry->read += num_read; + } else { + entry = tor_malloc_zero(sizeof(bidi_map_entry_t)); + entry->conn_id = conn_id; + entry->written = num_written; + entry->read = num_read; + HT_INSERT(bidimap, &bidi_map, entry); + } + } +} + +/** Return a newly allocated string containing the connection statistics + * until <b>now</b>, or NULL if we're not collecting conn stats. Caller must + * ensure start_of_conn_stats_interval is in the past. */ +char * +rep_hist_format_conn_stats(time_t now) +{ + char *result, written[ISO_TIME_LEN+1]; + + if (!start_of_conn_stats_interval) + return NULL; /* Not initialized. */ + + tor_assert(now >= start_of_conn_stats_interval); + + format_iso_time(written, now); + tor_asprintf(&result, "conn-bi-direct %s (%d s) %d,%d,%d,%d\n", + written, + (unsigned) (now - start_of_conn_stats_interval), + below_threshold, + mostly_read, + mostly_written, + both_read_and_written); + return result; +} + +/** If 24 hours have passed since the beginning of the current conn stats + * period, write conn stats to $DATADIR/stats/conn-stats (possibly + * overwriting an existing file) and reset counters. Return when we would + * next want to write conn stats or 0 if we never want to write. */ +time_t +rep_hist_conn_stats_write(time_t now) +{ + char *statsdir = NULL, *filename = NULL, *str = NULL; + + if (!start_of_conn_stats_interval) + return 0; /* Not initialized. */ + if (start_of_conn_stats_interval + WRITE_STATS_INTERVAL > now) + goto done; /* Not ready to write */ + + /* Generate history string. */ + str = rep_hist_format_conn_stats(now); + + /* Reset counters. */ + rep_hist_reset_conn_stats(now); + + /* Try to write to disk. */ + statsdir = get_datadir_fname("stats"); + if (check_private_dir(statsdir, CPD_CREATE, get_options()->User) < 0) { + log_warn(LD_HIST, "Unable to create stats/ directory!"); + goto done; + } + filename = get_datadir_fname2("stats", "conn-stats"); + if (write_str_to_file(filename, str, 0) < 0) + log_warn(LD_HIST, "Unable to write conn stats to disk!"); + + done: tor_free(str); -#undef SHARES - return start_of_buffer_stats_interval + WRITE_STATS_INTERVAL; + tor_free(filename); + tor_free(statsdir); + return start_of_conn_stats_interval + WRITE_STATS_INTERVAL; } /** Free all storage held by the OR/link history caches, by the @@ -2576,11 +3009,15 @@ rep_hist_free_all(void) tor_free(exit_streams); built_last_stability_doc_at = 0; predicted_ports_free(); + bidi_map_free(); + if (circuits_for_buffer_stats) { SMARTLIST_FOREACH(circuits_for_buffer_stats, circ_buffer_stats_t *, s, tor_free(s)); smartlist_free(circuits_for_buffer_stats); circuits_for_buffer_stats = NULL; } + rep_hist_desc_stats_term(); + total_descriptor_downloads = 0; } diff --git a/src/or/rephist.h b/src/or/rephist.h index b06a39ed59..d47724edb5 100644 --- a/src/or/rephist.h +++ b/src/or/rephist.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -34,7 +34,7 @@ int rep_hist_load_state(or_state_t *state, char **err); void rep_history_clean(time_t before); void rep_hist_note_router_reachable(const char *id, const tor_addr_t *at_addr, - uint16_t at_port, time_t when); + const uint16_t at_port, time_t when); void rep_hist_note_router_unreachable(const char *id, time_t when); int rep_hist_record_mtbf_data(time_t now, int missing_means_down); int rep_hist_load_mtbf_data(time_t now); @@ -49,6 +49,7 @@ const char *rep_hist_get_router_stability_doc(time_t now); void rep_hist_note_used_port(time_t now, uint16_t port); smartlist_t *rep_hist_get_predicted_ports(time_t now); +void rep_hist_remove_predicted_ports(const smartlist_t *rmv_ports); void rep_hist_note_used_resolve(time_t now); void rep_hist_note_used_internal(time_t now, int need_uptime, int need_capacity); @@ -77,6 +78,23 @@ void rep_hist_buffer_stats_add_circ(circuit_t *circ, time_t end_of_interval); time_t rep_hist_buffer_stats_write(time_t now); void rep_hist_buffer_stats_term(void); +void rep_hist_add_buffer_stats(double mean_num_cells_in_queue, + double mean_time_cells_in_queue, uint32_t processed_cells); +char *rep_hist_format_buffer_stats(time_t now); +void rep_hist_reset_buffer_stats(time_t now); + +void rep_hist_desc_stats_init(time_t now); +void rep_hist_note_desc_served(const char * desc); +void rep_hist_desc_stats_term(void); +time_t rep_hist_desc_stats_write(time_t now); + +void rep_hist_conn_stats_init(time_t now); +void rep_hist_note_or_conn_bytes(uint64_t conn_id, size_t num_read, + size_t num_written, time_t when); +void rep_hist_reset_conn_stats(time_t now); +char *rep_hist_format_conn_stats(time_t now); +time_t rep_hist_conn_stats_write(time_t now); +void rep_hist_conn_stats_term(void); #endif diff --git a/src/or/router.c b/src/or/router.c index 365e888af9..1ace8e2492 100644 --- a/src/or/router.c +++ b/src/or/router.c @@ -1,12 +1,13 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #define ROUTER_PRIVATE #include "or.h" +#include "circuitbuild.h" #include "circuitlist.h" #include "circuituse.h" #include "config.h" @@ -19,6 +20,7 @@ #include "hibernate.h" #include "main.h" #include "networkstatus.h" +#include "nodelist.h" #include "policies.h" #include "relay.h" #include "rephist.h" @@ -45,28 +47,28 @@ extern long stats_n_seconds_working; static tor_mutex_t *key_lock=NULL; static time_t onionkey_set_at=0; /**< When was onionkey last changed? */ /** Current private onionskin decryption key: used to decode CREATE cells. */ -static crypto_pk_env_t *onionkey=NULL; +static crypto_pk_t *onionkey=NULL; /** Previous private onionskin decryption key: used to decode CREATE cells * generated by clients that have an older version of our descriptor. */ -static crypto_pk_env_t *lastonionkey=NULL; +static crypto_pk_t *lastonionkey=NULL; /** Private server "identity key": used to sign directory info and TLS * certificates. Never changes. */ -static crypto_pk_env_t *server_identitykey=NULL; +static crypto_pk_t *server_identitykey=NULL; /** Digest of server_identitykey. */ static char server_identitykey_digest[DIGEST_LEN]; /** Private client "identity key": used to sign bridges' and clients' * outbound TLS certificates. Regenerated on startup and on IP address * change. */ -static crypto_pk_env_t *client_identitykey=NULL; +static crypto_pk_t *client_identitykey=NULL; /** Signing key used for v3 directory material; only set for authorities. */ -static crypto_pk_env_t *authority_signing_key = NULL; +static crypto_pk_t *authority_signing_key = NULL; /** Key certificate to authenticate v3 directory material; only set for * authorities. */ static authority_cert_t *authority_key_certificate = NULL; /** For emergency V3 authority key migration: An extra signing key that we use * with our old (obsolete) identity key for a while. */ -static crypto_pk_env_t *legacy_signing_key = NULL; +static crypto_pk_t *legacy_signing_key = NULL; /** For emergency V3 authority key migration: An extra certificate to * authenticate legacy_signing_key with our obsolete identity key.*/ static authority_cert_t *legacy_key_certificate = NULL; @@ -80,10 +82,15 @@ static authority_cert_t *legacy_key_certificate = NULL; * lastonionkey; to update lastonionkey correctly, call rotate_onion_key(). */ static void -set_onion_key(crypto_pk_env_t *k) +set_onion_key(crypto_pk_t *k) { + if (onionkey && !crypto_pk_cmp_keys(onionkey, k)) { + /* k is already our onion key; free it and return */ + crypto_pk_free(k); + return; + } tor_mutex_acquire(key_lock); - crypto_free_pk_env(onionkey); + crypto_pk_free(onionkey); onionkey = k; tor_mutex_release(key_lock); mark_my_descriptor_dirty("set onion key"); @@ -91,7 +98,7 @@ set_onion_key(crypto_pk_env_t *k) /** Return the current onion key. Requires that the onion key has been * loaded or generated. */ -crypto_pk_env_t * +crypto_pk_t * get_onion_key(void) { tor_assert(onionkey); @@ -102,7 +109,7 @@ get_onion_key(void) * copy of the most recent onion key into *<b>last</b>. */ void -dup_onion_keys(crypto_pk_env_t **key, crypto_pk_env_t **last) +dup_onion_keys(crypto_pk_t **key, crypto_pk_t **last) { tor_assert(key); tor_assert(last); @@ -129,9 +136,9 @@ get_onion_key_set_at(void) /** Set the current server identity key to <b>k</b>. */ void -set_server_identity_key(crypto_pk_env_t *k) +set_server_identity_key(crypto_pk_t *k) { - crypto_free_pk_env(server_identitykey); + crypto_pk_free(server_identitykey); server_identitykey = k; crypto_pk_get_digest(server_identitykey, server_identitykey_digest); } @@ -149,15 +156,15 @@ assert_identity_keys_ok(void) } else { /* assert that we have set the client and server keys to be unequal */ if (server_identitykey) - tor_assert(0!=crypto_pk_cmp_keys(client_identitykey, - server_identitykey)); + tor_assert(0!=crypto_pk_cmp_keys(client_identitykey, + server_identitykey)); } } /** Returns the current server identity key; requires that the key has * been set, and that we are running as a Tor server. */ -crypto_pk_env_t * +crypto_pk_t * get_server_identity_key(void) { tor_assert(server_identitykey); @@ -176,16 +183,16 @@ server_identity_key_is_set(void) /** Set the current client identity key to <b>k</b>. */ void -set_client_identity_key(crypto_pk_env_t *k) +set_client_identity_key(crypto_pk_t *k) { - crypto_free_pk_env(client_identitykey); + crypto_pk_free(client_identitykey); client_identitykey = k; } /** Returns the current client identity key for use on outgoing TLS * connections; requires that the key has been set. */ -crypto_pk_env_t * +crypto_pk_t * get_tlsclient_identity_key(void) { tor_assert(client_identitykey); @@ -210,7 +217,7 @@ get_my_v3_authority_cert(void) /** Return the v3 signing key for this v3 (voting) authority, or NULL * if we have no such key. */ -crypto_pk_env_t * +crypto_pk_t * get_my_v3_authority_signing_key(void) { return authority_signing_key; @@ -227,7 +234,7 @@ get_my_v3_legacy_cert(void) /** If we're an authority, and we're using a legacy authority identity key for * emergency migration purposes, return that key. */ -crypto_pk_env_t * +crypto_pk_t * get_my_v3_legacy_signing_key(void) { return legacy_signing_key; @@ -244,12 +251,12 @@ void rotate_onion_key(void) { char *fname, *fname_prev; - crypto_pk_env_t *prkey; + crypto_pk_t *prkey; or_state_t *state = get_or_state(); time_t now; fname = get_datadir_fname2("keys", "secret_onion_key"); fname_prev = get_datadir_fname2("keys", "secret_onion_key.old"); - if (!(prkey = crypto_new_pk_env())) { + if (!(prkey = crypto_pk_new())) { log_err(LD_GENERAL,"Error constructing rotated onion key"); goto error; } @@ -267,7 +274,7 @@ rotate_onion_key(void) } log_info(LD_GENERAL, "Rotating onion key"); tor_mutex_acquire(key_lock); - crypto_free_pk_env(lastonionkey); + crypto_pk_free(lastonionkey); lastonionkey = onionkey; onionkey = prkey; now = time(NULL); @@ -279,7 +286,7 @@ rotate_onion_key(void) error: log_warn(LD_GENERAL, "Couldn't rotate onion key."); if (prkey) - crypto_free_pk_env(prkey); + crypto_pk_free(prkey); done: tor_free(fname); tor_free(fname_prev); @@ -290,12 +297,12 @@ rotate_onion_key(void) * <b>fname</b>. Return the read/created key, or NULL on error. Log all * errors at level <b>severity</b>. */ -crypto_pk_env_t * +crypto_pk_t * init_key_from_file(const char *fname, int generate, int severity) { - crypto_pk_env_t *prkey = NULL; + crypto_pk_t *prkey = NULL; - if (!(prkey = crypto_new_pk_env())) { + if (!(prkey = crypto_pk_new())) { log(severity, LD_GENERAL,"Error constructing key"); goto error; } @@ -350,7 +357,7 @@ init_key_from_file(const char *fname, int generate, int severity) error: if (prkey) - crypto_free_pk_env(prkey); + crypto_pk_free(prkey); return NULL; } @@ -360,13 +367,13 @@ init_key_from_file(const char *fname, int generate, int severity) * key/cert set. On success, store them into *<b>key_out</b> and * *<b>cert_out</b> respectively, and return 0. On failure, return -1. */ static int -load_authority_keyset(int legacy, crypto_pk_env_t **key_out, +load_authority_keyset(int legacy, crypto_pk_t **key_out, authority_cert_t **cert_out) { int r = -1; char *fname = NULL, *cert = NULL; const char *eos = NULL; - crypto_pk_env_t *signing_key = NULL; + crypto_pk_t *signing_key = NULL; authority_cert_t *parsed = NULL; fname = get_datadir_fname2("keys", @@ -396,7 +403,7 @@ load_authority_keyset(int legacy, crypto_pk_env_t **key_out, goto done; } - crypto_free_pk_env(*key_out); + crypto_pk_free(*key_out); authority_cert_free(*cert_out); *key_out = signing_key; @@ -408,7 +415,7 @@ load_authority_keyset(int legacy, crypto_pk_env_t **key_out, done: tor_free(fname); tor_free(cert); - crypto_free_pk_env(signing_key); + crypto_pk_free(signing_key); authority_cert_free(parsed); return r; } @@ -477,6 +484,18 @@ v3_authority_check_key_expiry(void) last_warned = now; } +/** Set up Tor's TLS contexts, based on our configuration and keys. Return 0 + * on success, and -1 on failure. */ +int +router_initialize_tls_context(void) +{ + return tor_tls_context_init(public_server_mode(get_options()), + get_tlsclient_identity_key(), + server_mode(get_options()) ? + get_server_identity_key() : NULL, + MAX_SSL_KEY_LIFETIME_ADVERTISED); +} + /** Initialize all OR private keys, and the TLS context, as necessary. * On OPs, this only initializes the tls context. Return 0 on success, * or -1 if Tor should die. @@ -489,12 +508,12 @@ init_keys(void) /*nickname<space>fp\n\0 */ char fingerprint_line[MAX_NICKNAME_LEN+FINGERPRINT_LEN+3]; const char *mydesc; - crypto_pk_env_t *prkey; + crypto_pk_t *prkey; char digest[DIGEST_LEN]; char v3_digest[DIGEST_LEN]; char *cp; - or_options_t *options = get_options(); - authority_type_t type; + const or_options_t *options = get_options(); + dirinfo_type_t type; time_t now = time(NULL); trusted_dir_server_t *ds; int v3_digest_set = 0; @@ -515,18 +534,15 @@ init_keys(void) /* OP's don't need persistent keys; just make up an identity and * initialize the TLS context. */ if (!server_mode(options)) { - if (!(prkey = crypto_new_pk_env())) + if (!(prkey = crypto_pk_new())) return -1; if (crypto_pk_generate_key(prkey)) { - crypto_free_pk_env(prkey); + crypto_pk_free(prkey); return -1; } set_client_identity_key(prkey); /* Create a TLS context. */ - if (tor_tls_context_init(0, - get_tlsclient_identity_key(), - NULL, - MAX_SSL_KEY_LIFETIME_ADVERTISED) < 0) { + if (router_initialize_tls_context() < 0) { log_err(LD_GENERAL,"Error creating TLS context for Tor client."); return -1; } @@ -575,10 +591,10 @@ init_keys(void) if (public_server_mode(options)) { set_client_identity_key(crypto_pk_dup_key(prkey)); /* set above */ } else { - if (!(prkey = crypto_new_pk_env())) + if (!(prkey = crypto_pk_new())) return -1; if (crypto_pk_generate_key(prkey)) { - crypto_free_pk_env(prkey); + crypto_pk_free(prkey); return -1; } set_client_identity_key(prkey); @@ -619,13 +635,11 @@ init_keys(void) tor_free(keydir); /* 3. Initialize link key and TLS context. */ - if (tor_tls_context_init(public_server_mode(options), - get_tlsclient_identity_key(), - get_server_identity_key(), - MAX_SSL_KEY_LIFETIME_ADVERTISED) < 0) { + if (router_initialize_tls_context() < 0) { log_err(LD_GENERAL,"Error initializing TLS context"); return -1; } + /* 4. Build our router descriptor. */ /* Must be called after keys are initialized. */ mydesc = router_get_my_descriptor(); @@ -639,15 +653,27 @@ init_keys(void) return -1; } if (mydesc) { + was_router_added_t added; ri = router_parse_entry_from_string(mydesc, NULL, 1, 0, NULL); if (!ri) { log_err(LD_GENERAL,"Generated a routerinfo we couldn't parse."); return -1; } - if (!WRA_WAS_ADDED(dirserv_add_descriptor(ri, &m, "self"))) { - log_err(LD_GENERAL,"Unable to add own descriptor to directory: %s", - m?m:"<unknown error>"); - return -1; + added = dirserv_add_descriptor(ri, &m, "self"); + if (!WRA_WAS_ADDED(added)) { + if (!WRA_WAS_OUTDATED(added)) { + log_err(LD_GENERAL, "Unable to add own descriptor to directory: %s", + m?m:"<unknown error>"); + return -1; + } else { + /* If the descriptor was outdated, that's ok. This can happen + * when some config options are toggled that affect workers, but + * we don't really need new keys yet so the descriptor doesn't + * change and the old one is still fresh. */ + log_info(LD_GENERAL, "Couldn't add own descriptor to directory " + "after key init: %s. This is usually not a problem.", + m?m:"<unknown error>"); + } } } } @@ -695,11 +721,12 @@ init_keys(void) } /* 6b. [authdirserver only] add own key to approved directories. */ crypto_pk_get_digest(get_server_identity_key(), digest); - type = ((options->V1AuthoritativeDir ? V1_AUTHORITY : NO_AUTHORITY) | - (options->V2AuthoritativeDir ? V2_AUTHORITY : NO_AUTHORITY) | - (options->V3AuthoritativeDir ? V3_AUTHORITY : NO_AUTHORITY) | - (options->BridgeAuthoritativeDir ? BRIDGE_AUTHORITY : NO_AUTHORITY) | - (options->HSAuthoritativeDir ? HIDSERV_AUTHORITY : NO_AUTHORITY)); + type = ((options->V1AuthoritativeDir ? V1_DIRINFO : NO_DIRINFO) | + (options->V2AuthoritativeDir ? V2_DIRINFO : NO_DIRINFO) | + (options->V3AuthoritativeDir ? + (V3_DIRINFO|MICRODESC_DIRINFO|EXTRAINFO_DIRINFO) : NO_DIRINFO) | + (options->BridgeAuthoritativeDir ? BRIDGE_DIRINFO : NO_DIRINFO) | + (options->HSAuthoritativeDir ? HIDSERV_DIRINFO : NO_DIRINFO)); ds = router_get_trusteddirserver_by_digest(digest); if (!ds) { @@ -721,7 +748,7 @@ init_keys(void) type, ds->type); ds->type = type; } - if (v3_digest_set && (ds->type & V3_AUTHORITY) && + if (v3_digest_set && (ds->type & V3_DIRINFO) && tor_memneq(v3_digest, ds->v3_identity_digest, DIGEST_LEN)) { log_warn(LD_DIR, "V3 identity key does not match identity declared in " "DirServer line. Adjusting."); @@ -760,7 +787,7 @@ router_reset_reachability(void) int check_whether_orport_reachable(void) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); return options->AssumeReachable || can_reach_or_port; } @@ -769,10 +796,10 @@ check_whether_orport_reachable(void) int check_whether_dirport_reachable(void) { - or_options_t *options = get_options(); - return !options->DirPort || + const or_options_t *options = get_options(); + return !options->DirPort_set || options->AssumeReachable || - we_are_hibernating() || + net_is_disabled() || can_reach_dir_port; } @@ -784,7 +811,7 @@ check_whether_dirport_reachable(void) * a DirPort. */ static int -decide_to_advertise_dirport(or_options_t *options, uint16_t dir_port) +decide_to_advertise_dirport(const or_options_t *options, uint16_t dir_port) { static int advertising=1; /* start out assuming we will advertise */ int new_choice=1; @@ -798,7 +825,7 @@ decide_to_advertise_dirport(or_options_t *options, uint16_t dir_port) return 0; if (authdir_mode(options)) /* always publish */ return dir_port; - if (we_are_hibernating()) + if (net_is_disabled()) return 0; if (!check_whether_dirport_reachable()) return 0; @@ -810,9 +837,26 @@ decide_to_advertise_dirport(or_options_t *options, uint16_t dir_port) * make us choose not to publish. */ if (accounting_is_enabled(options)) { - /* if we might potentially hibernate */ - new_choice = 0; - reason = "AccountingMax enabled"; + /* Don't spend bytes for directory traffic if we could end up hibernating, + * but allow DirPort otherwise. Some people set AccountingMax because + * they're confused or to get statistics. */ + int interval_length = accounting_get_interval_length(); + uint32_t effective_bw = get_effective_bwrate(options); + if (!interval_length) { + log_warn(LD_BUG, "An accounting interval is not allowed to be zero " + "seconds long. Raising to 1."); + interval_length = 1; + } + log_info(LD_GENERAL, "Calculating whether to disable dirport: effective " + "bwrate: %u, AccountingMax: "U64_FORMAT", " + "accounting interval length %d", effective_bw, + U64_PRINTF_ARG(options->AccountingMax), + interval_length); + if (effective_bw >= + options->AccountingMax / interval_length) { + new_choice = 0; + reason = "AccountingMax enabled"; + } #define MIN_BW_TO_ADVERTISE_DIRPORT 51200 } else if (options->BandwidthRate < MIN_BW_TO_ADVERTISE_DIRPORT || (options->RelayBandwidthRate > 0 && @@ -849,14 +893,14 @@ decide_to_advertise_dirport(or_options_t *options, uint16_t dir_port) void consider_testing_reachability(int test_or, int test_dir) { - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); int orport_reachable = check_whether_orport_reachable(); tor_addr_t addr; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (!me) return; - if (routerset_contains_router(options->ExcludeNodes, me) && + if (routerset_contains_router(options->ExcludeNodes, me, -1) && options->StrictNodes) { /* If we've excluded ourself, and StrictNodes is set, we can't test * ourself. */ @@ -876,11 +920,15 @@ consider_testing_reachability(int test_or, int test_dir) } if (test_or && (!orport_reachable || !circuit_enough_testing_circs())) { + extend_info_t *ei; log_info(LD_CIRC, "Testing %s of my ORPort: %s:%d.", !orport_reachable ? "reachability" : "bandwidth", me->address, me->or_port); - circuit_launch_by_router(CIRCUIT_PURPOSE_TESTING, me, - CIRCLAUNCH_NEED_CAPACITY|CIRCLAUNCH_IS_INTERNAL); + /* XXX IPv6 self testing IPv6 orports will need pref_addr */ + ei = extend_info_from_router(me, 0); + circuit_launch_by_extend_info(CIRCUIT_PURPOSE_TESTING, ei, + CIRCLAUNCH_NEED_CAPACITY|CIRCLAUNCH_IS_INTERNAL); + extend_info_free(ei); } tor_addr_from_ipv4h(&addr, me->addr); @@ -903,11 +951,11 @@ consider_testing_reachability(int test_or, int test_dir) void router_orport_found_reachable(void) { - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); if (!can_reach_or_port && me) { log_notice(LD_OR,"Self-testing indicates your ORPort is reachable from " "the outside. Excellent.%s", - get_options()->_PublishServerDescriptor != NO_AUTHORITY ? + get_options()->_PublishServerDescriptor != NO_DIRINFO ? " Publishing server descriptor." : ""); can_reach_or_port = 1; mark_my_descriptor_dirty("ORPort found reachable"); @@ -921,7 +969,7 @@ router_orport_found_reachable(void) void router_dirport_found_reachable(void) { - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); if (!can_reach_dir_port && me) { log_notice(LD_DIRSERV,"Self-testing indicates your DirPort is reachable " "from the outside. Excellent."); @@ -963,11 +1011,19 @@ router_perform_bandwidth_test(int num_circs, time_t now) } } +/** Return true iff our network is in some sense disabled: either we're + * hibernating, entering hibernation, or */ +int +net_is_disabled(void) +{ + return get_options()->DisableNetwork || we_are_hibernating(); +} + /** Return true iff we believe ourselves to be an authoritative * directory server. */ int -authdir_mode(or_options_t *options) +authdir_mode(const or_options_t *options) { return options->AuthoritativeDir != 0; } @@ -975,7 +1031,7 @@ authdir_mode(or_options_t *options) * directory server. */ int -authdir_mode_v1(or_options_t *options) +authdir_mode_v1(const or_options_t *options) { return authdir_mode(options) && options->V1AuthoritativeDir != 0; } @@ -983,7 +1039,7 @@ authdir_mode_v1(or_options_t *options) * directory server. */ int -authdir_mode_v2(or_options_t *options) +authdir_mode_v2(const or_options_t *options) { return authdir_mode(options) && options->V2AuthoritativeDir != 0; } @@ -991,13 +1047,13 @@ authdir_mode_v2(or_options_t *options) * directory server. */ int -authdir_mode_v3(or_options_t *options) +authdir_mode_v3(const or_options_t *options) { return authdir_mode(options) && options->V3AuthoritativeDir != 0; } /** Return true iff we are a v1, v2, or v3 directory authority. */ int -authdir_mode_any_main(or_options_t *options) +authdir_mode_any_main(const or_options_t *options) { return options->V1AuthoritativeDir || options->V2AuthoritativeDir || @@ -1006,16 +1062,16 @@ authdir_mode_any_main(or_options_t *options) /** Return true if we believe ourselves to be any kind of * authoritative directory beyond just a hidserv authority. */ int -authdir_mode_any_nonhidserv(or_options_t *options) +authdir_mode_any_nonhidserv(const or_options_t *options) { return options->BridgeAuthoritativeDir || authdir_mode_any_main(options); } /** Return true iff we are an authoritative directory server that is * authoritative about receiving and serving descriptors of type - * <b>purpose</b> its dirport. Use -1 for "any purpose". */ + * <b>purpose</b> on its dirport. Use -1 for "any purpose". */ int -authdir_mode_handles_descs(or_options_t *options, int purpose) +authdir_mode_handles_descs(const or_options_t *options, int purpose) { if (purpose < 0) return authdir_mode_any_nonhidserv(options); @@ -1030,7 +1086,7 @@ authdir_mode_handles_descs(or_options_t *options, int purpose) * publishes its own network statuses. */ int -authdir_mode_publishes_statuses(or_options_t *options) +authdir_mode_publishes_statuses(const or_options_t *options) { if (authdir_mode_bridge(options)) return 0; @@ -1040,7 +1096,7 @@ authdir_mode_publishes_statuses(or_options_t *options) * tests reachability of the descriptors it learns about. */ int -authdir_mode_tests_reachability(or_options_t *options) +authdir_mode_tests_reachability(const or_options_t *options) { return authdir_mode_handles_descs(options, -1); } @@ -1048,7 +1104,7 @@ authdir_mode_tests_reachability(or_options_t *options) * directory server. */ int -authdir_mode_bridge(or_options_t *options) +authdir_mode_bridge(const or_options_t *options) { return authdir_mode(options) && options->BridgeAuthoritativeDir != 0; } @@ -1056,16 +1112,17 @@ authdir_mode_bridge(or_options_t *options) /** Return true iff we are trying to be a server. */ int -server_mode(or_options_t *options) +server_mode(const or_options_t *options) { if (options->ClientOnly) return 0; - return (options->ORPort != 0 || options->ORListenAddress); + /* XXXX024 I believe we can kill off ORListenAddress here.*/ + return (options->ORPort_set || options->ORListenAddress); } /** Return true iff we are trying to be a non-bridge server. */ int -public_server_mode(or_options_t *options) +public_server_mode(const or_options_t *options) { if (!server_mode(options)) return 0; return (!options->BridgeRelay); @@ -1075,10 +1132,10 @@ public_server_mode(or_options_t *options) * in the consensus mean that we don't want to allow exits from circuits * we got from addresses not known to be servers. */ int -should_refuse_unknown_exits(or_options_t *options) +should_refuse_unknown_exits(const or_options_t *options) { - if (options->RefuseUnknownExits_ != -1) { - return options->RefuseUnknownExits_; + if (options->RefuseUnknownExits != -1) { + return options->RefuseUnknownExits; } else { return networkstatus_get_param(NULL, "refuseunknownexits", 1, 0, 1); } @@ -1105,14 +1162,19 @@ set_server_advertised(int s) server_is_advertised = s; } -/** Return true iff we are trying to be a socks proxy. */ +/** Return true iff we are trying to proxy client connections. */ int -proxy_mode(or_options_t *options) -{ - return (options->SocksPort != 0 || - options->TransPort != 0 || - options->NATDPort != 0 || - options->DNSPort != 0); +proxy_mode(const or_options_t *options) +{ + (void)options; + SMARTLIST_FOREACH_BEGIN(get_configured_ports(), const port_cfg_t *, p) { + if (p->type == CONN_TYPE_AP_LISTENER || + p->type == CONN_TYPE_AP_TRANS_LISTENER || + p->type == CONN_TYPE_AP_DNS_LISTENER || + p->type == CONN_TYPE_AP_NATD_LISTENER) + return 1; + } SMARTLIST_FOREACH_END(p); + return 0; } /** Decide if we're a publishable server. We are a publishable server if: @@ -1128,11 +1190,11 @@ proxy_mode(or_options_t *options) static int decide_if_publishable_server(void) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (options->ClientOnly) return 0; - if (options->_PublishServerDescriptor == NO_AUTHORITY) + if (options->_PublishServerDescriptor == NO_DIRINFO) return 0; if (!server_mode(options)) return 0; @@ -1169,19 +1231,38 @@ consider_publishable_server(int force) } } +/** Return the port of the first active listener of type + * <b>listener_type</b>. */ +/** XXX not a very good interface. it's not reliable when there are + multiple listeners. */ +uint16_t +router_get_active_listener_port_by_type(int listener_type) +{ + /* Iterate all connections, find one of the right kind and return + the port. Not very sophisticated or fast, but effective. */ + const connection_t *c = connection_get_by_type(listener_type); + if (c) + return c->port; + + return 0; +} + /** Return the port that we should advertise as our ORPort; this is either * the one configured in the ORPort option, or the one we actually bound to - * if ORPort is "auto". */ + * if ORPort is "auto". + */ uint16_t -router_get_advertised_or_port(or_options_t *options) +router_get_advertised_or_port(const or_options_t *options) { - if (options->ORPort == CFG_AUTO_PORT) { - connection_t *c = connection_get_by_type(CONN_TYPE_OR_LISTENER); - if (c) - return c->port; - return 0; - } - return options->ORPort; + int port = get_primary_or_port(); + (void)options; + + /* If the port is in 'auto' mode, we have to use + router_get_listener_port_by_type(). */ + if (port == CFG_AUTO_PORT) + return router_get_active_listener_port_by_type(CONN_TYPE_OR_LISTENER); + + return port; } /** Return the port that we should advertise as our DirPort; @@ -1190,17 +1271,18 @@ router_get_advertised_or_port(or_options_t *options) * the one configured in the DirPort option, * or the one we actually bound to if DirPort is "auto". */ uint16_t -router_get_advertised_dir_port(or_options_t *options, uint16_t dirport) +router_get_advertised_dir_port(const or_options_t *options, uint16_t dirport) { - if (!options->DirPort) + int dirport_configured = get_primary_dir_port(); + (void)options; + + if (!dirport_configured) return dirport; - if (options->DirPort == CFG_AUTO_PORT) { - connection_t *c = connection_get_by_type(CONN_TYPE_DIR_LISTENER); - if (c) - return c->port; - return 0; - } - return options->DirPort; + + if (dirport_configured == CFG_AUTO_PORT) + return router_get_active_listener_port_by_type(CONN_TYPE_DIR_LISTENER); + + return dirport_configured; } /* @@ -1211,9 +1293,14 @@ router_get_advertised_dir_port(or_options_t *options, uint16_t dirport) static routerinfo_t *desc_routerinfo = NULL; /** My extrainfo */ static extrainfo_t *desc_extrainfo = NULL; +/** Why did we most recently decide to regenerate our descriptor? Used to + * tell the authorities why we're sending it to them. */ +static const char *desc_gen_reason = NULL; /** Since when has our descriptor been "clean"? 0 if we need to regenerate it * now. */ static time_t desc_clean_since = 0; +/** Why did we mark the descriptor dirty? */ +static const char *desc_dirty_reason = NULL; /** Boolean: do we need to regenerate the above? */ static int desc_needs_upload = 0; @@ -1224,11 +1311,11 @@ static int desc_needs_upload = 0; void router_upload_dir_desc_to_dirservers(int force) { - routerinfo_t *ri; + const routerinfo_t *ri; extrainfo_t *ei; char *msg; size_t desc_len, extra_len = 0, total_len; - authority_type_t auth = get_options()->_PublishServerDescriptor; + dirinfo_type_t auth = get_options()->_PublishServerDescriptor; ri = router_get_my_routerinfo(); if (!ri) { @@ -1236,7 +1323,7 @@ router_upload_dir_desc_to_dirservers(int force) return; } ei = router_get_my_extrainfo(); - if (auth == NO_AUTHORITY) + if (auth == NO_DIRINFO) return; if (!force && !desc_needs_upload) return; @@ -1257,7 +1344,7 @@ router_upload_dir_desc_to_dirservers(int force) msg[desc_len+extra_len] = 0; directory_post_to_dirservers(DIR_PURPOSE_UPLOAD_DIR, - (auth & BRIDGE_AUTHORITY) ? + (auth & BRIDGE_DIRINFO) ? ROUTER_PURPOSE_BRIDGE : ROUTER_PURPOSE_GENERAL, auth, msg, desc_len, extra_len); @@ -1322,7 +1409,7 @@ router_extrainfo_digest_is_me(const char *digest) /** A wrapper around router_digest_is_me(). */ int -router_is_me(routerinfo_t *router) +router_is_me(const routerinfo_t *router) { return router_digest_is_me(router->cache_info.identity_digest); } @@ -1341,7 +1428,7 @@ router_fingerprint_is_me(const char *fp) /** Return a routerinfo for this OR, rebuilding a fresh one if * necessary. Return NULL on error, or if called on an OP. */ -routerinfo_t * +const routerinfo_t * router_get_my_routerinfo(void) { if (!server_mode(get_options())) @@ -1380,6 +1467,14 @@ router_get_my_extrainfo(void) return desc_extrainfo; } +/** Return a human-readable string describing what triggered us to generate + * our current descriptor, or NULL if we don't know. */ +const char * +router_get_descriptor_gen_reason(void) +{ + return desc_gen_reason; +} + /** A list of nicknames that we've warned about including in our family * declaration verbatim rather than as digests. */ static smartlist_t *warned_nonexistent_family = NULL; @@ -1391,10 +1486,8 @@ static int router_guess_address_from_dir_headers(uint32_t *guess); * dirserver headers. Place the answer in *<b>addr</b> and return * 0 on success, else return -1 if we have no guess. */ int -router_pick_published_address(or_options_t *options, uint32_t *addr) +router_pick_published_address(const or_options_t *options, uint32_t *addr) { - char buf[INET_NTOA_BUF_LEN]; - struct in_addr a; if (resolve_my_address(LOG_INFO, options, addr, NULL) < 0) { log_info(LD_CONFIG, "Could not determine our address locally. " "Checking if directory headers provide any hints."); @@ -1404,9 +1497,7 @@ router_pick_published_address(or_options_t *options, uint32_t *addr) return -1; } } - a.s_addr = htonl(*addr); - tor_inet_ntoa(&a, buf, sizeof(buf)); - log_info(LD_CONFIG,"Success: chose address '%s'.", buf); + log_info(LD_CONFIG,"Success: chose address '%s'.", fmt_addr32(*addr)); return 0; } @@ -1422,7 +1513,7 @@ router_rebuild_descriptor(int force) uint32_t addr; char platform[256]; int hibernating = we_are_hibernating(); - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (desc_clean_since && !force) return 0; @@ -1448,6 +1539,32 @@ router_rebuild_descriptor(int force) ri->cache_info.published_on = time(NULL); ri->onion_pkey = crypto_pk_dup_key(get_onion_key()); /* must invoke from * main thread */ + if (options->BridgeRelay) { + /* For now, only bridges advertise an ipv6 or-address. And only one. */ + const port_cfg_t *ipv6_orport = NULL; + SMARTLIST_FOREACH_BEGIN(get_configured_ports(), const port_cfg_t *, p) { + if (p->type == CONN_TYPE_OR_LISTENER && + ! p->no_advertise && + ! p->ipv4_only && + tor_addr_family(&p->addr) == AF_INET6) { + if (! tor_addr_is_internal(&p->addr, 0)) { + ipv6_orport = p; + break; + } else { + char addrbuf[TOR_ADDR_BUF_LEN]; + log_warn(LD_CONFIG, + "Unable to use configured IPv6 address \"%s\" in a " + "descriptor. Skipping it. " + "Try specifying a globally reachable address explicitly. ", + tor_addr_to_str(addrbuf, &p->addr, sizeof(addrbuf), 1)); + } + } + } SMARTLIST_FOREACH_END(p); + if (ipv6_orport) { + tor_addr_copy(&ri->ipv6_addr, &ipv6_orport->addr); + ri->ipv6_orport = ipv6_orport->port; + } + } ri->identity_pkey = crypto_pk_dup_key(get_server_identity_key()); if (crypto_pk_get_digest(ri->identity_pkey, ri->cache_info.identity_digest)<0) { @@ -1476,28 +1593,26 @@ router_rebuild_descriptor(int force) ri->policy_is_reject_star = policy_is_reject_star(ri->exit_policy); - if (desc_routerinfo) { /* inherit values */ - ri->is_valid = desc_routerinfo->is_valid; - ri->is_running = desc_routerinfo->is_running; - ri->is_named = desc_routerinfo->is_named; - } +#if 0 + /* XXXX NM NM I belive this is safe to remove */ if (authdir_mode(options)) ri->is_valid = ri->is_named = 1; /* believe in yourself */ - if (options->MyFamily) { +#endif + + if (options->MyFamily && ! options->BridgeRelay) { smartlist_t *family; if (!warned_nonexistent_family) - warned_nonexistent_family = smartlist_create(); - family = smartlist_create(); - ri->declared_family = smartlist_create(); + warned_nonexistent_family = smartlist_new(); + family = smartlist_new(); + ri->declared_family = smartlist_new(); smartlist_split_string(family, options->MyFamily, ",", SPLIT_SKIP_SPACE|SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - SMARTLIST_FOREACH(family, char *, name, - { - routerinfo_t *member; + SMARTLIST_FOREACH_BEGIN(family, char *, name) { + const node_t *member; if (!strcasecmp(name, options->Nickname)) - member = ri; + goto skip; /* Don't list ourself, that's redundant */ else - member = router_get_by_nickname(name, 1); + member = node_get_by_nickname(name, 1); if (!member) { int is_legal = is_legal_nickname_or_hexdigest(name); if (!smartlist_string_isin(warned_nonexistent_family, name) && @@ -1517,19 +1632,21 @@ router_rebuild_descriptor(int force) smartlist_add(ri->declared_family, name); name = NULL; } - } else if (router_is_me(member)) { + } else if (router_digest_is_me(member->identity)) { /* Don't list ourself in our own family; that's redundant */ + /* XXX shouldn't be possible */ } else { char *fp = tor_malloc(HEX_DIGEST_LEN+2); fp[0] = '$'; base16_encode(fp+1,HEX_DIGEST_LEN+1, - member->cache_info.identity_digest, DIGEST_LEN); + member->identity, DIGEST_LEN); smartlist_add(ri->declared_family, fp); if (smartlist_string_isin(warned_nonexistent_family, name)) smartlist_string_remove(warned_nonexistent_family, name); } + skip: tor_free(name); - }); + } SMARTLIST_FOREACH_END(name); /* remove duplicates from the list */ smartlist_sort_strings(ri->declared_family); @@ -1554,6 +1671,7 @@ router_rebuild_descriptor(int force) ei->cache_info.signed_descriptor_len = strlen(ei->cache_info.signed_descriptor_body); router_get_extrainfo_hash(ei->cache_info.signed_descriptor_body, + ei->cache_info.signed_descriptor_len, ei->cache_info.signed_descriptor_digest); } @@ -1579,19 +1697,24 @@ router_rebuild_descriptor(int force) ri->purpose = options->BridgeRelay ? ROUTER_PURPOSE_BRIDGE : ROUTER_PURPOSE_GENERAL; - ri->cache_info.send_unencrypted = 1; - /* Let bridges serve their own descriptors unencrypted, so they can - * pass reachability testing. (If they want to be harder to notice, - * they can always leave the DirPort off). */ - if (ei && !options->BridgeRelay) - ei->cache_info.send_unencrypted = 1; + if (options->BridgeRelay) { + /* Bridges shouldn't be able to send their descriptors unencrypted, + anyway, since they don't have a DirPort, and always connect to the + bridge authority anonymously. But just in case they somehow think of + sending them on an unencrypted connection, don't allow them to try. */ + ri->cache_info.send_unencrypted = 0; + if (ei) + ei->cache_info.send_unencrypted = 0; + } else { + ri->cache_info.send_unencrypted = 1; + if (ei) + ei->cache_info.send_unencrypted = 1; + } router_get_router_hash(ri->cache_info.signed_descriptor_body, strlen(ri->cache_info.signed_descriptor_body), ri->cache_info.signed_descriptor_digest); - routerinfo_set_country(ri); - if (ei) { tor_assert(! routerinfo_incompatible_with_extrainfo(ri, ei, NULL, NULL)); } @@ -1603,24 +1726,68 @@ router_rebuild_descriptor(int force) desc_clean_since = time(NULL); desc_needs_upload = 1; + desc_gen_reason = desc_dirty_reason; + desc_dirty_reason = NULL; control_event_my_descriptor_changed(); return 0; } -/** Mark descriptor out of date if it's older than <b>when</b> */ +/** If our router descriptor ever goes this long without being regenerated + * because something changed, we force an immediate regenerate-and-upload. */ +#define FORCE_REGENERATE_DESCRIPTOR_INTERVAL (18*60*60) + +/** If our router descriptor seems to be missing or unacceptable according + * to the authorities, regenerate and reupload it _this_ often. */ +#define FAST_RETRY_DESCRIPTOR_INTERVAL (90*60) + +/** Mark descriptor out of date if it's been "too long" since we last tried + * to upload one. */ void -mark_my_descriptor_dirty_if_older_than(time_t when) +mark_my_descriptor_dirty_if_too_old(time_t now) { - if (desc_clean_since < when) + networkstatus_t *ns; + const routerstatus_t *rs; + const char *retry_fast_reason = NULL; /* Set if we should retry frequently */ + const time_t slow_cutoff = now - FORCE_REGENERATE_DESCRIPTOR_INTERVAL; + const time_t fast_cutoff = now - FAST_RETRY_DESCRIPTOR_INTERVAL; + + /* If it's already dirty, don't mark it. */ + if (! desc_clean_since) + return; + + /* If it's older than FORCE_REGENERATE_DESCRIPTOR_INTERVAL, it's always + * time to rebuild it. */ + if (desc_clean_since < slow_cutoff) { mark_my_descriptor_dirty("time for new descriptor"); + return; + } + /* Now we see whether we want to be retrying frequently or no. The + * rule here is that we'll retry frequently if we aren't listed in the + * live consensus we have, or if the publication time of the + * descriptor listed for us in the consensus is very old. */ + ns = networkstatus_get_live_consensus(now); + if (ns) { + rs = networkstatus_vote_find_entry(ns, server_identitykey_digest); + if (rs == NULL) + retry_fast_reason = "not listed in consensus"; + else if (rs->published_on < slow_cutoff) + retry_fast_reason = "version listed in consensus is quite old"; + } + + if (retry_fast_reason && desc_clean_since < fast_cutoff) + mark_my_descriptor_dirty(retry_fast_reason); } /** Call when the current descriptor is out of date. */ void mark_my_descriptor_dirty(const char *reason) { + const or_options_t *options = get_options(); + if (server_mode(options) && options->_PublishServerDescriptor) + log_info(LD_OR, "Decided to publish new relay descriptor: %s", reason); desc_clean_since = 0; - log_info(LD_OR, "Decided to publish new relay descriptor: %s", reason); + if (!desc_dirty_reason) + desc_dirty_reason = reason; } /** How frequently will we republish our descriptor because of large (factor @@ -1654,21 +1821,20 @@ check_descriptor_bandwidth_changed(time_t now) /** Note at log level severity that our best guess of address has changed from * <b>prev</b> to <b>cur</b>. */ static void -log_addr_has_changed(int severity, uint32_t prev, uint32_t cur, +log_addr_has_changed(int severity, + const tor_addr_t *prev, + const tor_addr_t *cur, const char *source) { - char addrbuf_prev[INET_NTOA_BUF_LEN]; - char addrbuf_cur[INET_NTOA_BUF_LEN]; - struct in_addr in_prev; - struct in_addr in_cur; - - in_prev.s_addr = htonl(prev); - tor_inet_ntoa(&in_prev, addrbuf_prev, sizeof(addrbuf_prev)); + char addrbuf_prev[TOR_ADDR_BUF_LEN]; + char addrbuf_cur[TOR_ADDR_BUF_LEN]; - in_cur.s_addr = htonl(cur); - tor_inet_ntoa(&in_cur, addrbuf_cur, sizeof(addrbuf_cur)); + if (tor_addr_to_str(addrbuf_prev, prev, sizeof(addrbuf_prev), 1) == NULL) + strlcpy(addrbuf_prev, "???", TOR_ADDR_BUF_LEN); + if (tor_addr_to_str(addrbuf_cur, cur, sizeof(addrbuf_cur), 1) == NULL) + strlcpy(addrbuf_cur, "???", TOR_ADDR_BUF_LEN); - if (prev) + if (!tor_addr_is_null(prev)) log_fn(severity, LD_GENERAL, "Our IP Address has changed from %s to %s; " "rebuilding descriptor (source: %s).", @@ -1686,12 +1852,13 @@ void check_descriptor_ipaddress_changed(time_t now) { uint32_t prev, cur; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); (void) now; if (!desc_routerinfo) return; + /* XXXX ipv6 */ prev = desc_routerinfo->addr; if (resolve_my_address(LOG_INFO, options, &cur, NULL) < 0) { log_info(LD_CONFIG,"options->Address didn't resolve into an IP."); @@ -1699,14 +1866,17 @@ check_descriptor_ipaddress_changed(time_t now) } if (prev != cur) { - log_addr_has_changed(LOG_NOTICE, prev, cur, "resolve"); + tor_addr_t tmp_prev, tmp_cur; + tor_addr_from_ipv4h(&tmp_prev, prev); + tor_addr_from_ipv4h(&tmp_cur, cur); + log_addr_has_changed(LOG_NOTICE, &tmp_prev, &tmp_cur, "resolve"); ip_address_changed(0); } } /** The most recently guessed value of our IP address, based on directory * headers. */ -static uint32_t last_guessed_ip = 0; +static tor_addr_t last_guessed_ip = TOR_ADDR_NULL; /** A directory server <b>d_conn</b> told us our IP address is * <b>suggestion</b>. @@ -1716,35 +1886,36 @@ void router_new_address_suggestion(const char *suggestion, const dir_connection_t *d_conn) { - uint32_t addr, cur = 0; - struct in_addr in; - or_options_t *options = get_options(); + tor_addr_t addr; + uint32_t cur = 0; /* Current IPv4 address. */ + const or_options_t *options = get_options(); /* first, learn what the IP address actually is */ - if (!tor_inet_aton(suggestion, &in)) { + if (tor_addr_parse(&addr, suggestion) == -1) { log_debug(LD_DIR, "Malformed X-Your-Address-Is header %s. Ignoring.", escaped(suggestion)); return; } - addr = ntohl(in.s_addr); log_debug(LD_DIR, "Got X-Your-Address-Is: %s.", suggestion); if (!server_mode(options)) { - last_guessed_ip = addr; /* store it in case we need it later */ + tor_addr_copy(&last_guessed_ip, &addr); return; } + /* XXXX ipv6 */ if (resolve_my_address(LOG_INFO, options, &cur, NULL) >= 0) { /* We're all set -- we already know our address. Great. */ - last_guessed_ip = cur; /* store it in case we need it later */ + tor_addr_from_ipv4h(&last_guessed_ip, cur); /* store it in case we + need it later */ return; } - if (is_internal_IP(addr, 0)) { + if (tor_addr_is_internal(&addr, 0)) { /* Don't believe anybody who says our IP is, say, 127.0.0.1. */ return; } - if (tor_addr_eq_ipv4h(&d_conn->_base.addr, addr)) { + if (tor_addr_eq(&d_conn->_base.addr, &addr)) { /* Don't believe anybody who says our IP is their IP. */ log_debug(LD_DIR, "A directory server told us our IP address is %s, " "but he's just reporting his own IP address. Ignoring.", @@ -1755,14 +1926,15 @@ router_new_address_suggestion(const char *suggestion, /* Okay. We can't resolve our own address, and X-Your-Address-Is is giving * us an answer different from what we had the last time we managed to * resolve it. */ - if (last_guessed_ip != addr) { + if (!tor_addr_eq(&last_guessed_ip, &addr)) { control_event_server_status(LOG_NOTICE, "EXTERNAL_ADDRESS ADDRESS=%s METHOD=DIRSERV", suggestion); - log_addr_has_changed(LOG_NOTICE, last_guessed_ip, addr, + log_addr_has_changed(LOG_NOTICE, &last_guessed_ip, &addr, d_conn->_base.address); ip_address_changed(0); - last_guessed_ip = addr; /* router_rebuild_descriptor() will fetch it */ + tor_addr_copy(&last_guessed_ip, &addr); /* router_rebuild_descriptor() + will fetch it */ } } @@ -1773,8 +1945,8 @@ router_new_address_suggestion(const char *suggestion, static int router_guess_address_from_dir_headers(uint32_t *guess) { - if (last_guessed_ip) { - *guess = last_guessed_ip; + if (!tor_addr_is_null(&last_guessed_ip)) { + *guess = tor_addr_to_ipv4h(&last_guessed_ip); return 0; } return -1; @@ -1787,7 +1959,8 @@ router_guess_address_from_dir_headers(uint32_t *guess) void get_platform_str(char *platform, size_t len) { - tor_snprintf(platform, len, "Tor %s on %s", get_version(), get_uname()); + tor_snprintf(platform, len, "Tor %s on %s", + get_short_version(), get_uname()); } /* XXX need to audit this thing and count fenceposts. maybe @@ -1803,7 +1976,7 @@ get_platform_str(char *platform, size_t len) */ int router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, - crypto_pk_env_t *ident_key) + crypto_pk_t *ident_key) { char *onion_pkey; /* Onion key, PEM-encoded. */ char *identity_pkey; /* Identity key, PEM-encoded. */ @@ -1817,7 +1990,8 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, int result=0; addr_policy_t *tmpe; char *family_line; - or_options_t *options = get_options(); + char *extra_or_address = NULL; + const or_options_t *options = get_options(); /* Make sure the identity key matches the one in the routerinfo. */ if (crypto_pk_cmp_keys(ident_key, router->identity_pkey)) { @@ -1851,11 +2025,9 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, format_iso_time(published, router->cache_info.published_on); if (router->declared_family && smartlist_len(router->declared_family)) { - size_t n; - char *family = smartlist_join_strings(router->declared_family, " ", 0, &n); - n += strlen("family ") + 2; /* 1 for \n, 1 for \0. */ - family_line = tor_malloc(n); - tor_snprintf(family_line, n, "family %s\n", family); + char *family = smartlist_join_strings(router->declared_family, + " ", 0, NULL); + tor_asprintf(&family_line, "family %s\n", family); tor_free(family); } else { family_line = tor_strdup(""); @@ -1869,9 +2041,22 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, router->cache_info.extra_info_digest, DIGEST_LEN); } + if (router->ipv6_orport && + tor_addr_family(&router->ipv6_addr) == AF_INET6) { + char addr[TOR_ADDR_BUF_LEN]; + const char *a; + a = tor_addr_to_str(addr, &router->ipv6_addr, sizeof(addr), 1); + if (a) { + tor_asprintf(&extra_or_address, + "or-address %s:%d\n", a, router->ipv6_orport); + log_debug(LD_OR, "My or-address line is <%s>", extra_or_address); + } + } + /* Generate the easy portion of the router descriptor. */ result = tor_snprintf(s, maxlen, "router %s %s %d 0 %d\n" + "%s" "platform %s\n" "opt protocols Link 1 2 Circuit 1\n" "published %s\n" @@ -1886,6 +2071,7 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, router->address, router->or_port, decide_to_advertise_dirport(options, router->dir_port), + extra_or_address ? extra_or_address : "", router->platform, published, fingerprint, @@ -1906,6 +2092,7 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, tor_free(family_line); tor_free(onion_pkey); tor_free(identity_pkey); + tor_free(extra_or_address); if (result < 0) { log_warn(LD_BUG,"descriptor snprintf #1 ran out of room!"); @@ -2001,6 +2188,52 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, return (int)written+1; } +/** Copy the primary (IPv4) OR port (IP address and TCP port) for + * <b>router</b> into *<b>ap_out</b>. */ +void +router_get_prim_orport(const routerinfo_t *router, tor_addr_port_t *ap_out) +{ + tor_assert(ap_out != NULL); + tor_addr_from_ipv4h(&ap_out->addr, router->addr); + ap_out->port = router->or_port; +} + +/** Return 1 if we prefer the IPv6 address and OR TCP port of + * <b>router</b>, else 0. + * + * We prefer the IPv6 address if the router has one and + * i) the routerinfo_t says so + * or + * ii) the router has no IPv4 address. */ +int +router_ipv6_preferred(const routerinfo_t *router) +{ + return (!tor_addr_is_null(&router->ipv6_addr) + && (router->ipv6_preferred || router->addr == 0)); +} + +/** Copy the preferred OR port (IP address and TCP port) for + * <b>router</b> into *<b>addr_out</b>. */ +void +router_get_pref_orport(const routerinfo_t *router, tor_addr_port_t *ap_out) +{ + if (router_ipv6_preferred(router)) + router_get_pref_ipv6_orport(router, ap_out); + else + router_get_prim_orport(router, ap_out); +} + +/** Copy the preferred IPv6 OR port (IP address and TCP port) for + * <b>router</b> into *<b>ap_out</b>. */ +void +router_get_pref_ipv6_orport(const routerinfo_t *router, + tor_addr_port_t *ap_out) +{ + tor_assert(ap_out != NULL); + tor_addr_copy(&ap_out->addr, &router->ipv6_addr); + ap_out->port = router->ipv6_orport; +} + /** Load the contents of <b>filename</b>, find the last line starting with * <b>end_line</b>, ensure that its timestamp is not more than 25 hours in * the past or more than 1 hour in the future with respect to <b>now</b>, @@ -2057,9 +2290,9 @@ load_stats_file(const char *filename, const char *end_line, time_t now, * success, negative on failure. */ int extrainfo_dump_to_string(char **s_out, extrainfo_t *extrainfo, - crypto_pk_env_t *ident_key) + crypto_pk_t *ident_key) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); char identity[HEX_DIGEST_LEN+1]; char published[ISO_TIME_LEN+1]; char digest[DIGEST_LEN]; @@ -2069,7 +2302,7 @@ extrainfo_dump_to_string(char **s_out, extrainfo_t *extrainfo, char sig[DIROBJ_MAX_SIG_LEN+1]; char *s, *pre, *contents, *cp, *s_dup = NULL; time_t now = time(NULL); - smartlist_t *chunks = smartlist_create(); + smartlist_t *chunks = smartlist_new(); extrainfo_t *ei_tmp = NULL; base16_encode(identity, sizeof(identity), @@ -2083,6 +2316,10 @@ extrainfo_dump_to_string(char **s_out, extrainfo_t *extrainfo, tor_free(bandwidth_usage); smartlist_add(chunks, pre); + if (geoip_is_loaded()) { + smartlist_add_asprintf(chunks, "geoip-db-digest %s\n", geoip_db_digest()); + } + if (options->ExtraInfoStatistics && write_stats_to_extrainfo) { log_info(LD_GENERAL, "Adding stats to extra-info descriptor."); if (options->DirReqStatistics && @@ -2105,6 +2342,11 @@ extrainfo_dump_to_string(char **s_out, extrainfo_t *extrainfo, "exit-stats-end", now, &contents) > 0) { smartlist_add(chunks, contents); } + if (options->ConnDirectionStatistics && + load_stats_file("stats"PATH_SEPARATOR"conn-stats", + "conn-bi-direct", now, &contents) > 0) { + smartlist_add(chunks, contents); + } } if (should_record_bridge_info(options) && write_stats_to_extrainfo) { @@ -2142,7 +2384,7 @@ extrainfo_dump_to_string(char **s_out, extrainfo_t *extrainfo, } memset(sig, 0, sizeof(sig)); - if (router_get_extrainfo_hash(s, digest) < 0 || + if (router_get_extrainfo_hash(s, strlen(s), digest) < 0 || router_append_dirobj_signature(sig, sizeof(sig), digest, DIGEST_LEN, ident_key) < 0) { log_warn(LD_BUG, "Could not append signature to extra-info " @@ -2291,13 +2533,45 @@ router_get_description(char *buf, const routerinfo_t *ri) return "<null>"; return format_node_description(buf, ri->cache_info.identity_digest, - ri->is_named, + router_is_named(ri), ri->nickname, NULL, ri->addr); } /** Use <b>buf</b> (which must be at least NODE_DESC_BUF_LEN bytes long) to + * hold a human-readable description of <b>node</b>. + * + * Return a pointer to the front of <b>buf</b>. + */ +const char * +node_get_description(char *buf, const node_t *node) +{ + const char *nickname = NULL; + uint32_t addr32h = 0; + int is_named = 0; + + if (!node) + return "<null>"; + + if (node->rs) { + nickname = node->rs->nickname; + is_named = node->rs->is_named; + addr32h = node->rs->addr; + } else if (node->ri) { + nickname = node->ri->nickname; + addr32h = node->ri->addr; + } + + return format_node_description(buf, + node->identity, + is_named, + nickname, + NULL, + addr32h); +} + +/** Use <b>buf</b> (which must be at least NODE_DESC_BUF_LEN bytes long) to * hold a human-readable description of <b>rs</b>. * * Return a pointer to the front of <b>buf</b>. @@ -2345,6 +2619,18 @@ router_describe(const routerinfo_t *ri) return router_get_description(buf, ri); } +/** Return a human-readable description of the node_t <b>node</b>. + * + * This function is not thread-safe. Each call to this function invalidates + * previous values returned by this function. + */ +const char * +node_describe(const node_t *node) +{ + static char buf[NODE_DESC_BUF_LEN]; + return node_get_description(buf, node); +} + /** Return a human-readable description of the routerstatus_t <b>rs</b>. * * This function is not thread-safe. Each call to this function invalidates @@ -2379,10 +2665,15 @@ extend_info_describe(const extend_info_t *ei) void router_get_verbose_nickname(char *buf, const routerinfo_t *router) { + const char *good_digest = networkstatus_get_router_digest_by_nickname( + router->nickname); + int is_named = good_digest && tor_memeq(good_digest, + router->cache_info.identity_digest, + DIGEST_LEN); buf[0] = '$'; base16_encode(buf+1, HEX_DIGEST_LEN+1, router->cache_info.identity_digest, DIGEST_LEN); - buf[1+HEX_DIGEST_LEN] = router->is_named ? '=' : '~'; + buf[1+HEX_DIGEST_LEN] = is_named ? '=' : '~'; strlcpy(buf+1+HEX_DIGEST_LEN+1, router->nickname, MAX_NICKNAME_LEN+1); } @@ -2449,16 +2740,16 @@ router_purpose_from_string(const char *s) void router_free_all(void) { - crypto_free_pk_env(onionkey); - crypto_free_pk_env(lastonionkey); - crypto_free_pk_env(server_identitykey); - crypto_free_pk_env(client_identitykey); + crypto_pk_free(onionkey); + crypto_pk_free(lastonionkey); + crypto_pk_free(server_identitykey); + crypto_pk_free(client_identitykey); tor_mutex_free(key_lock); routerinfo_free(desc_routerinfo); extrainfo_free(desc_extrainfo); - crypto_free_pk_env(authority_signing_key); + crypto_pk_free(authority_signing_key); authority_cert_free(authority_key_certificate); - crypto_free_pk_env(legacy_signing_key); + crypto_pk_free(legacy_signing_key); authority_cert_free(legacy_key_certificate); if (warned_nonexistent_family) { diff --git a/src/or/router.h b/src/or/router.h index 3733099f93..69805d6f2d 100644 --- a/src/or/router.h +++ b/src/or/router.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -12,24 +12,25 @@ #ifndef _TOR_ROUTER_H #define _TOR_ROUTER_H -crypto_pk_env_t *get_onion_key(void); +crypto_pk_t *get_onion_key(void); time_t get_onion_key_set_at(void); -void set_server_identity_key(crypto_pk_env_t *k); -crypto_pk_env_t *get_server_identity_key(void); +void set_server_identity_key(crypto_pk_t *k); +crypto_pk_t *get_server_identity_key(void); int server_identity_key_is_set(void); -void set_client_identity_key(crypto_pk_env_t *k); -crypto_pk_env_t *get_tlsclient_identity_key(void); +void set_client_identity_key(crypto_pk_t *k); +crypto_pk_t *get_tlsclient_identity_key(void); int client_identity_key_is_set(void); authority_cert_t *get_my_v3_authority_cert(void); -crypto_pk_env_t *get_my_v3_authority_signing_key(void); +crypto_pk_t *get_my_v3_authority_signing_key(void); authority_cert_t *get_my_v3_legacy_cert(void); -crypto_pk_env_t *get_my_v3_legacy_signing_key(void); -void dup_onion_keys(crypto_pk_env_t **key, crypto_pk_env_t **last); +crypto_pk_t *get_my_v3_legacy_signing_key(void); +void dup_onion_keys(crypto_pk_t **key, crypto_pk_t **last); void rotate_onion_key(void); -crypto_pk_env_t *init_key_from_file(const char *fname, int generate, +crypto_pk_t *init_key_from_file(const char *fname, int generate, int severity); void v3_authority_check_key_expiry(void); +int router_initialize_tls_context(void); int init_keys(void); int check_whether_orport_reachable(void); @@ -39,30 +40,33 @@ void router_orport_found_reachable(void); void router_dirport_found_reachable(void); void router_perform_bandwidth_test(int num_circs, time_t now); -int authdir_mode(or_options_t *options); -int authdir_mode_v1(or_options_t *options); -int authdir_mode_v2(or_options_t *options); -int authdir_mode_v3(or_options_t *options); -int authdir_mode_any_main(or_options_t *options); -int authdir_mode_any_nonhidserv(or_options_t *options); -int authdir_mode_handles_descs(or_options_t *options, int purpose); -int authdir_mode_publishes_statuses(or_options_t *options); -int authdir_mode_tests_reachability(or_options_t *options); -int authdir_mode_bridge(or_options_t *options); - -uint16_t router_get_advertised_or_port(or_options_t *options); -uint16_t router_get_advertised_dir_port(or_options_t *options, +int net_is_disabled(void); + +int authdir_mode(const or_options_t *options); +int authdir_mode_v1(const or_options_t *options); +int authdir_mode_v2(const or_options_t *options); +int authdir_mode_v3(const or_options_t *options); +int authdir_mode_any_main(const or_options_t *options); +int authdir_mode_any_nonhidserv(const or_options_t *options); +int authdir_mode_handles_descs(const or_options_t *options, int purpose); +int authdir_mode_publishes_statuses(const or_options_t *options); +int authdir_mode_tests_reachability(const or_options_t *options); +int authdir_mode_bridge(const or_options_t *options); + +uint16_t router_get_active_listener_port_by_type(int listener_type); +uint16_t router_get_advertised_or_port(const or_options_t *options); +uint16_t router_get_advertised_dir_port(const or_options_t *options, uint16_t dirport); -int server_mode(or_options_t *options); -int public_server_mode(or_options_t *options); +int server_mode(const or_options_t *options); +int public_server_mode(const or_options_t *options); int advertised_server_mode(void); -int proxy_mode(or_options_t *options); +int proxy_mode(const or_options_t *options); void consider_publishable_server(int force); -int should_refuse_unknown_exits(or_options_t *options); +int should_refuse_unknown_exits(const or_options_t *options); void router_upload_dir_desc_to_dirservers(int force); -void mark_my_descriptor_dirty_if_older_than(time_t when); +void mark_my_descriptor_dirty_if_too_old(time_t now); void mark_my_descriptor_dirty(const char *reason); void check_descriptor_bandwidth_changed(time_t now); void check_descriptor_ipaddress_changed(time_t now); @@ -70,19 +74,27 @@ void router_new_address_suggestion(const char *suggestion, const dir_connection_t *d_conn); int router_compare_to_my_exit_policy(edge_connection_t *conn); int router_my_exit_policy_is_reject_star(void); -routerinfo_t *router_get_my_routerinfo(void); +const routerinfo_t *router_get_my_routerinfo(void); extrainfo_t *router_get_my_extrainfo(void); const char *router_get_my_descriptor(void); +const char *router_get_descriptor_gen_reason(void); int router_digest_is_me(const char *digest); int router_extrainfo_digest_is_me(const char *digest); -int router_is_me(routerinfo_t *router); +int router_is_me(const routerinfo_t *router); int router_fingerprint_is_me(const char *fp); -int router_pick_published_address(or_options_t *options, uint32_t *addr); +int router_pick_published_address(const or_options_t *options, uint32_t *addr); int router_rebuild_descriptor(int force); int router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, - crypto_pk_env_t *ident_key); + crypto_pk_t *ident_key); +void router_get_prim_orport(const routerinfo_t *router, + tor_addr_port_t *addr_port_out); +void router_get_pref_orport(const routerinfo_t *router, + tor_addr_port_t *addr_port_out); +void router_get_pref_ipv6_orport(const routerinfo_t *router, + tor_addr_port_t *addr_port_out); +int router_ipv6_preferred(const routerinfo_t *router); int extrainfo_dump_to_string(char **s, extrainfo_t *extrainfo, - crypto_pk_env_t *ident_key); + crypto_pk_t *ident_key); int is_legal_nickname(const char *s); int is_legal_nickname_or_hexdigest(const char *s); int is_legal_hexdigest(const char *s); @@ -102,9 +114,11 @@ const char *format_node_description(char *buf, const tor_addr_t *addr, uint32_t addr32h); const char *router_get_description(char *buf, const routerinfo_t *ri); +const char *node_get_description(char *buf, const node_t *node); const char *routerstatus_get_description(char *buf, const routerstatus_t *rs); const char *extend_info_get_description(char *buf, const extend_info_t *ei); const char *router_describe(const routerinfo_t *ri); +const char *node_describe(const node_t *node); const char *routerstatus_describe(const routerstatus_t *ri); const char *extend_info_describe(const extend_info_t *ei); diff --git a/src/or/routerlist.c b/src/or/routerlist.c index 30c20bf6e6..3c39e362df 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -22,7 +22,9 @@ #include "geoip.h" #include "hibernate.h" #include "main.h" +#include "microdesc.h" #include "networkstatus.h" +#include "nodelist.h" #include "policies.h" #include "reasons.h" #include "rendcommon.h" @@ -37,22 +39,25 @@ /****************************************************************************/ /* static function prototypes */ -static routerstatus_t *router_pick_directory_server_impl( - authority_type_t auth, int flags); -static routerstatus_t *router_pick_trusteddirserver_impl( - authority_type_t auth, int flags, int *n_busy_out); +static const routerstatus_t *router_pick_directory_server_impl( + dirinfo_type_t auth, int flags); +static const routerstatus_t *router_pick_trusteddirserver_impl( + dirinfo_type_t auth, int flags, int *n_busy_out); static void mark_all_trusteddirservers_up(void); -static int router_nickname_matches(routerinfo_t *router, const char *nickname); +static int router_nickname_matches(const routerinfo_t *router, + const char *nickname); +static int node_nickname_matches(const node_t *router, + const char *nickname); static void trusted_dir_server_free(trusted_dir_server_t *ds); -static void launch_router_descriptor_downloads(smartlist_t *downloadable, - routerstatus_t *source, - time_t now); static int signed_desc_digest_is_recognized(signed_descriptor_t *desc); static void update_router_have_minimum_dir_info(void); -static const char *signed_descriptor_get_body_impl(signed_descriptor_t *desc, - int with_annotations); +static const char *signed_descriptor_get_body_impl( + const signed_descriptor_t *desc, + int with_annotations); static void list_pending_downloads(digestmap_t *result, int purpose, const char *prefix); +static void launch_dummy_descriptor_download_as_needed(time_t now, + const or_options_t *options); DECLARE_TYPED_DIGESTMAP_FNS(sdmap_, digest_sd_map_t, signed_descriptor_t) DECLARE_TYPED_DIGESTMAP_FNS(rimap_, digest_ri_map_t, routerinfo_t) @@ -94,19 +99,22 @@ static smartlist_t *warned_nicknames = NULL; /** The last time we tried to download any routerdesc, or 0 for "never". We * use this to rate-limit download attempts when the number of routerdescs to * download is low. */ -static time_t last_routerdesc_download_attempted = 0; +static time_t last_descriptor_download_attempted = 0; /** When we last computed the weights to use for bandwidths on directory * requests, what were the total weighted bandwidth, and our share of that * bandwidth? Used to determine what fraction of directory requests we should - * expect to see. */ + * expect to see. + * + * @{ */ static uint64_t sl_last_total_weighted_bw = 0, sl_last_weighted_bw_of_me = 0; +/**@}*/ /** Return the number of directory authorities whose type matches some bit set * in <b>type</b> */ int -get_n_authorities(authority_type_t type) +get_n_authorities(dirinfo_type_t type) { int n = 0; if (!trusted_dir_servers) @@ -117,7 +125,7 @@ get_n_authorities(authority_type_t type) return n; } -#define get_n_v2_authorities() get_n_authorities(V2_AUTHORITY) +#define get_n_v2_authorities() get_n_authorities(V2_DIRINFO) /** Helper: Return the cert_list_t for an authority whose authority ID is * <b>id_digest</b>, allocating a new list if necessary. */ @@ -131,7 +139,7 @@ get_cert_list(const char *id_digest) if (!cl) { cl = tor_malloc_zero(sizeof(cert_list_t)); cl->dl_status.schedule = DL_SCHED_CONSENSUS; - cl->certs = smartlist_create(); + cl->certs = smartlist_new(); digestmap_set(trusted_dir_certs, id_digest, cl); } return cl; @@ -209,9 +217,18 @@ trusted_dirs_load_certs_from_string(const char *contents, int from_store, * probably means we wanted a different secret key or we are trying to * replace an expired cert that has not in fact been updated. */ if (!from_store) { - log_warn(LD_DIR, "Got a certificate for %s, but we already have it. " - "Maybe they haven't updated it. Waiting for a while.", - ds ? ds->nickname : "an old or new authority"); + if (authdir_mode(get_options())) { + log_warn(LD_DIR, + "Got a certificate for %s, but we already have it. " + "Maybe they haven't updated it. Waiting for a while.", + ds ? ds->nickname : "an old or new authority"); + } else { + log_info(LD_DIR, + "Got a certificate for %s, but we already have it. " + "Maybe they haven't updated it. Waiting for a while.", + ds ? ds->nickname : "an old or new authority"); + } + authority_cert_dl_failed(cert->cache_info.identity_digest, 404); } @@ -224,7 +241,7 @@ trusted_dirs_load_certs_from_string(const char *contents, int from_store, "signing key %s", from_store ? "cached" : "downloaded", ds->nickname, hex_str(cert->signing_key_digest,DIGEST_LEN)); } else { - int adding = directory_caches_dir_info(get_options()); + int adding = directory_caches_unknown_auth_certs(get_options()); log_info(LD_DIR, "%s %s certificate for unrecognized directory " "authority with signing key %s", adding ? "Adding" : "Not adding", @@ -280,7 +297,7 @@ trusted_dirs_flush_certs_to_disk(void) if (!trusted_dir_servers_certs_changed || !trusted_dir_certs) return; - chunks = smartlist_create(); + chunks = smartlist_new(); DIGESTMAP_FOREACH(trusted_dir_certs, key, cert_list_t *, cl) { SMARTLIST_FOREACH(cl->certs, authority_cert_t *, cert, { @@ -312,6 +329,7 @@ trusted_dirs_remove_old_certs(void) time_t now = time(NULL); #define DEAD_CERT_LIFETIME (2*24*60*60) #define OLD_CERT_LIFETIME (7*24*60*60) +#define CERT_EXPIRY_SKEW (60*60) if (!trusted_dir_certs) return; @@ -474,13 +492,13 @@ authority_certs_fetch_missing(networkstatus_t *status, time_t now) smartlist_t *missing_digests; char *resource = NULL; cert_list_t *cl; - const int cache = directory_caches_dir_info(get_options()); + const int cache = directory_caches_unknown_auth_certs(get_options()); if (should_delay_dir_fetches(get_options())) return; pending = digestmap_new(); - missing_digests = smartlist_create(); + missing_digests = smartlist_new(); list_pending_downloads(pending, DIR_PURPOSE_FETCH_CERTIFICATE, "fp/"); if (status) { @@ -514,7 +532,7 @@ authority_certs_fetch_missing(networkstatus_t *status, time_t now) } SMARTLIST_FOREACH_BEGIN(trusted_dir_servers, trusted_dir_server_t *, ds) { int found = 0; - if (!(ds->type & V3_AUTHORITY)) + if (!(ds->type & V3_DIRINFO)) continue; if (smartlist_digest_isin(missing_digests, ds->v3_identity_digest)) continue; @@ -540,7 +558,7 @@ authority_certs_fetch_missing(networkstatus_t *status, time_t now) if (!smartlist_len(missing_digests)) { goto done; } else { - smartlist_t *fps = smartlist_create(); + smartlist_t *fps = smartlist_new(); smartlist_add(fps, tor_strdup("fp/")); SMARTLIST_FOREACH(missing_digests, const char *, d, { char *fp; @@ -600,7 +618,7 @@ router_should_rebuild_store(desc_store_t *store) /** Return the desc_store_t in <b>rl</b> that should be used to store * <b>sd</b>. */ static INLINE desc_store_t * -desc_get_store(routerlist_t *rl, signed_descriptor_t *sd) +desc_get_store(routerlist_t *rl, const signed_descriptor_t *sd) { if (sd->is_extrainfo) return &rl->extrainfo_store; @@ -689,10 +707,10 @@ router_rebuild_store(int flags, desc_store_t *store) fname = get_datadir_fname(store->fname_base); fname_tmp = get_datadir_fname_suffix(store->fname_base, ".tmp"); - chunk_list = smartlist_create(); + chunk_list = smartlist_new(); /* We sort the routers by age to enhance locality on disk. */ - signed_descriptors = smartlist_create(); + signed_descriptors = smartlist_new(); if (store->type == EXTRAINFO_STORE) { eimap_iter_t *iter; for (iter = eimap_iter_init(routerlist->extra_info_map); @@ -713,8 +731,7 @@ router_rebuild_store(int flags, desc_store_t *store) smartlist_sort(signed_descriptors, _compare_signed_descriptors_by_age); /* Now, add the appropriate members to chunk_list */ - SMARTLIST_FOREACH(signed_descriptors, signed_descriptor_t *, sd, - { + SMARTLIST_FOREACH_BEGIN(signed_descriptors, signed_descriptor_t *, sd) { sized_chunk_t *c; const char *body = signed_descriptor_get_body_impl(sd, 1); if (!body) { @@ -730,7 +747,7 @@ router_rebuild_store(int flags, desc_store_t *store) c->len = sd->signed_descriptor_len + sd->annotations_len; total_expected_len += c->len; smartlist_add(chunk_list, c); - }); + } SMARTLIST_FOREACH_END(sd); if (write_chunks_to_file(fname_tmp, chunk_list, 1)<0) { log_warn(LD_FS, "Error writing router store to disk."); @@ -769,8 +786,7 @@ router_rebuild_store(int flags, desc_store_t *store) log_info(LD_DIR, "Reconstructing pointers into cache"); offset = 0; - SMARTLIST_FOREACH(signed_descriptors, signed_descriptor_t *, sd, - { + SMARTLIST_FOREACH_BEGIN(signed_descriptors, signed_descriptor_t *, sd) { if (sd->do_not_cache) continue; sd->saved_location = SAVED_IN_CACHE; @@ -780,7 +796,7 @@ router_rebuild_store(int flags, desc_store_t *store) } offset += sd->signed_descriptor_len + sd->annotations_len; signed_descriptor_get_body(sd); /* reconstruct and assert */ - }); + } SMARTLIST_FOREACH_END(sd); tor_free(fname); fname = get_datadir_fname_suffix(store->fname_base, ".new"); @@ -906,7 +922,7 @@ smartlist_t * router_get_trusted_dir_servers(void) { if (!trusted_dir_servers) - trusted_dir_servers = smartlist_create(); + trusted_dir_servers = smartlist_new(); return trusted_dir_servers; } @@ -926,10 +942,10 @@ router_get_trusted_dir_servers(void) * Don't pick an authority if any non-authority is viable; try to avoid using * servers that have returned 503 recently. */ -routerstatus_t * -router_pick_directory_server(authority_type_t type, int flags) +const routerstatus_t * +router_pick_directory_server(dirinfo_type_t type, int flags) { - routerstatus_t *choice; + const routerstatus_t *choice; if (get_options()->PreferTunneledDirConns) flags |= _PDS_PREFER_TUNNELED_DIR_CONNS; @@ -958,8 +974,8 @@ int router_get_my_share_of_directory_requests(double *v2_share_out, double *v3_share_out) { - routerinfo_t *me = router_get_my_routerinfo(); - routerstatus_t *rs; + const routerinfo_t *me = router_get_my_routerinfo(); + const routerstatus_t *rs; const int pds_flags = PDS_ALLOW_SELF|PDS_IGNORE_FASCISTFIREWALL; *v2_share_out = *v3_share_out = 0.0; if (!me) @@ -972,7 +988,7 @@ router_get_my_share_of_directory_requests(double *v2_share_out, /* XXXX This is a bit of a kludge */ if (rs->is_v2_dir) { sl_last_total_weighted_bw = 0; - router_pick_directory_server(V2_AUTHORITY, pds_flags); + router_pick_directory_server(V2_DIRINFO, pds_flags); if (sl_last_total_weighted_bw != 0) { *v2_share_out = U64_TO_DBL(sl_last_weighted_bw_of_me) / U64_TO_DBL(sl_last_total_weighted_bw); @@ -981,7 +997,7 @@ router_get_my_share_of_directory_requests(double *v2_share_out, if (rs->version_supports_v3_dir) { sl_last_total_weighted_bw = 0; - router_pick_directory_server(V3_AUTHORITY, pds_flags); + router_pick_directory_server(V3_DIRINFO, pds_flags); if (sl_last_total_weighted_bw != 0) { *v3_share_out = U64_TO_DBL(sl_last_weighted_bw_of_me) / U64_TO_DBL(sl_last_total_weighted_bw); @@ -1022,7 +1038,7 @@ trusteddirserver_get_by_v3_auth_digest(const char *digest) SMARTLIST_FOREACH(trusted_dir_servers, trusted_dir_server_t *, ds, { if (tor_memeq(ds->v3_identity_digest, digest, DIGEST_LEN) && - (ds->type & V3_AUTHORITY)) + (ds->type & V3_DIRINFO)) return ds; }); @@ -1032,10 +1048,10 @@ trusteddirserver_get_by_v3_auth_digest(const char *digest) /** Try to find a running trusted dirserver. Flags are as for * router_pick_directory_server. */ -routerstatus_t * -router_pick_trusteddirserver(authority_type_t type, int flags) +const routerstatus_t * +router_pick_trusteddirserver(dirinfo_type_t type, int flags) { - routerstatus_t *choice; + const routerstatus_t *choice; int busy = 0; if (get_options()->PreferTunneledDirConns) flags |= _PDS_PREFER_TUNNELED_DIR_CONNS; @@ -1047,7 +1063,8 @@ router_pick_trusteddirserver(authority_type_t type, int flags) /* If the reason that we got no server is that servers are "busy", * we must be excluding good servers because we already have serverdesc * fetches with them. Do not mark down servers up because of this. */ - tor_assert((flags & PDS_NO_EXISTING_SERVERDESC_FETCH)); + tor_assert((flags & (PDS_NO_EXISTING_SERVERDESC_FETCH| + PDS_NO_EXISTING_MICRODESC_FETCH))); return NULL; } @@ -1067,11 +1084,11 @@ router_pick_trusteddirserver(authority_type_t type, int flags) * If the _PDS_PREFER_TUNNELED_DIR_CONNS flag is set, prefer directory servers * that we can use with BEGINDIR. */ -static routerstatus_t * -router_pick_directory_server_impl(authority_type_t type, int flags) +static const routerstatus_t * +router_pick_directory_server_impl(dirinfo_type_t type, int flags) { - or_options_t *options = get_options(); - routerstatus_t *result; + const or_options_t *options = get_options(); + const node_t *result; smartlist_t *direct, *tunnel; smartlist_t *trusted_direct, *trusted_tunnel; smartlist_t *overloaded_direct, *overloaded_tunnel; @@ -1087,62 +1104,72 @@ router_pick_directory_server_impl(authority_type_t type, int flags) retry_without_exclude: - direct = smartlist_create(); - tunnel = smartlist_create(); - trusted_direct = smartlist_create(); - trusted_tunnel = smartlist_create(); - overloaded_direct = smartlist_create(); - overloaded_tunnel = smartlist_create(); + direct = smartlist_new(); + tunnel = smartlist_new(); + trusted_direct = smartlist_new(); + trusted_tunnel = smartlist_new(); + overloaded_direct = smartlist_new(); + overloaded_tunnel = smartlist_new(); /* Find all the running dirservers we know about. */ - SMARTLIST_FOREACH_BEGIN(consensus->routerstatus_list, routerstatus_t *, - status) { + SMARTLIST_FOREACH_BEGIN(nodelist_get_list(), const node_t *, node) { int is_trusted; - int is_overloaded = status->last_dir_503_at + DIR_503_TIMEOUT > now; + int is_overloaded; tor_addr_t addr; - if (!status->is_running || !status->dir_port || !status->is_valid) + const routerstatus_t *status = node->rs; + const country_t country = node->country; + if (!status) + continue; + + if (!node->is_running || !status->dir_port || !node->is_valid) continue; - if (status->is_bad_directory) + if (node->is_bad_directory) continue; - if (requireother && router_digest_is_me(status->identity_digest)) + if (requireother && router_digest_is_me(node->identity)) continue; - if (type & V3_AUTHORITY) { + if (type & V3_DIRINFO) { if (!(status->version_supports_v3_dir || - router_digest_is_trusted_dir_type(status->identity_digest, - V3_AUTHORITY))) + router_digest_is_trusted_dir_type(node->identity, + V3_DIRINFO))) continue; } - is_trusted = router_digest_is_trusted_dir(status->identity_digest); - if ((type & V2_AUTHORITY) && !(status->is_v2_dir || is_trusted)) + is_trusted = router_digest_is_trusted_dir(node->identity); + if ((type & V2_DIRINFO) && !(node->rs->is_v2_dir || is_trusted)) continue; - if ((type & EXTRAINFO_CACHE) && - !router_supports_extrainfo(status->identity_digest, 0)) + if ((type & EXTRAINFO_DIRINFO) && + !router_supports_extrainfo(node->identity, 0)) continue; - if (try_excluding && options->ExcludeNodes && - routerset_contains_routerstatus(options->ExcludeNodes, status)) { + if ((type & MICRODESC_DIRINFO) && !is_trusted && + !node->rs->version_supports_microdesc_cache) + continue; + if (try_excluding && + routerset_contains_routerstatus(options->ExcludeNodes, status, + country)) { ++n_excluded; continue; } /* XXXX IP6 proposal 118 */ - tor_addr_from_ipv4h(&addr, status->addr); + tor_addr_from_ipv4h(&addr, node->rs->addr); + + is_overloaded = status->last_dir_503_at + DIR_503_TIMEOUT > now; if (prefer_tunnel && status->version_supports_begindir && (!fascistfirewall || fascist_firewall_allows_address_or(&addr, status->or_port))) smartlist_add(is_trusted ? trusted_tunnel : - is_overloaded ? overloaded_tunnel : tunnel, status); + is_overloaded ? overloaded_tunnel : tunnel, (void*)node); else if (!fascistfirewall || fascist_firewall_allows_address_dir(&addr, status->dir_port)) smartlist_add(is_trusted ? trusted_direct : - is_overloaded ? overloaded_direct : direct, status); - } SMARTLIST_FOREACH_END(status); + is_overloaded ? overloaded_direct : direct, (void*)node); + } SMARTLIST_FOREACH_END(node); if (smartlist_len(tunnel)) { - result = routerstatus_sl_choose_by_bandwidth(tunnel, WEIGHT_FOR_DIR); + result = node_sl_choose_by_bandwidth(tunnel, WEIGHT_FOR_DIR); } else if (smartlist_len(overloaded_tunnel)) { - result = routerstatus_sl_choose_by_bandwidth(overloaded_tunnel, + result = node_sl_choose_by_bandwidth(overloaded_tunnel, WEIGHT_FOR_DIR); } else if (smartlist_len(trusted_tunnel)) { /* FFFF We don't distinguish between trusteds and overloaded trusteds @@ -1151,10 +1178,10 @@ router_pick_directory_server_impl(authority_type_t type, int flags) * is a feature, but it could easily be a bug. -RD */ result = smartlist_choose(trusted_tunnel); } else if (smartlist_len(direct)) { - result = routerstatus_sl_choose_by_bandwidth(direct, WEIGHT_FOR_DIR); + result = node_sl_choose_by_bandwidth(direct, WEIGHT_FOR_DIR); } else if (smartlist_len(overloaded_direct)) { - result = routerstatus_sl_choose_by_bandwidth(overloaded_direct, - WEIGHT_FOR_DIR); + result = node_sl_choose_by_bandwidth(overloaded_direct, + WEIGHT_FOR_DIR); } else { result = smartlist_choose(trusted_direct); } @@ -1173,26 +1200,27 @@ router_pick_directory_server_impl(authority_type_t type, int flags) goto retry_without_exclude; } - return result; + return result ? result->rs : NULL; } /** Choose randomly from among the trusted dirservers that are up. Flags * are as for router_pick_directory_server_impl(). */ -static routerstatus_t * -router_pick_trusteddirserver_impl(authority_type_t type, int flags, +static const routerstatus_t * +router_pick_trusteddirserver_impl(dirinfo_type_t type, int flags, int *n_busy_out) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); smartlist_t *direct, *tunnel; smartlist_t *overloaded_direct, *overloaded_tunnel; - routerinfo_t *me = router_get_my_routerinfo(); - routerstatus_t *result; + const routerinfo_t *me = router_get_my_routerinfo(); + const routerstatus_t *result; time_t now = time(NULL); const int requireother = ! (flags & PDS_ALLOW_SELF); const int fascistfirewall = ! (flags & PDS_IGNORE_FASCISTFIREWALL); const int prefer_tunnel = (flags & _PDS_PREFER_TUNNELED_DIR_CONNS); const int no_serverdesc_fetching =(flags & PDS_NO_EXISTING_SERVERDESC_FETCH); + const int no_microdesc_fetching =(flags & PDS_NO_EXISTING_MICRODESC_FETCH); int n_busy = 0; int try_excluding = 1, n_excluded = 0; @@ -1201,10 +1229,10 @@ router_pick_trusteddirserver_impl(authority_type_t type, int flags, retry_without_exclude: - direct = smartlist_create(); - tunnel = smartlist_create(); - overloaded_direct = smartlist_create(); - overloaded_tunnel = smartlist_create(); + direct = smartlist_new(); + tunnel = smartlist_new(); + overloaded_direct = smartlist_new(); + overloaded_tunnel = smartlist_new(); SMARTLIST_FOREACH_BEGIN(trusted_dir_servers, trusted_dir_server_t *, d) { @@ -1214,14 +1242,14 @@ router_pick_trusteddirserver_impl(authority_type_t type, int flags, if (!d->is_running) continue; if ((type & d->type) == 0) continue; - if ((type & EXTRAINFO_CACHE) && + if ((type & EXTRAINFO_DIRINFO) && !router_supports_extrainfo(d->digest, 1)) continue; if (requireother && me && router_digest_is_me(d->digest)) continue; - if (try_excluding && options->ExcludeNodes && + if (try_excluding && routerset_contains_routerstatus(options->ExcludeNodes, - &d->fake_status)) { + &d->fake_status, -1)) { ++n_excluded; continue; } @@ -1240,6 +1268,13 @@ router_pick_trusteddirserver_impl(authority_type_t type, int flags, continue; } } + if (no_microdesc_fetching) { + if (connection_get_by_type_addr_port_purpose( + CONN_TYPE_DIR, &addr, d->dir_port, DIR_PURPOSE_FETCH_MICRODESC)) { + ++n_busy; + continue; + } + } if (prefer_tunnel && d->or_port && @@ -1287,26 +1322,21 @@ router_pick_trusteddirserver_impl(authority_type_t type, int flags, static void mark_all_trusteddirservers_up(void) { - if (routerlist) { - SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, router, - if (router_digest_is_trusted_dir(router->cache_info.identity_digest) && - router->dir_port > 0) { - router->is_running = 1; - }); - } + SMARTLIST_FOREACH(nodelist_get_list(), node_t *, node, { + if (router_digest_is_trusted_dir(node->identity)) + node->is_running = 1; + }); if (trusted_dir_servers) { - SMARTLIST_FOREACH(trusted_dir_servers, trusted_dir_server_t *, dir, - { + SMARTLIST_FOREACH_BEGIN(trusted_dir_servers, trusted_dir_server_t *, dir) { routerstatus_t *rs; dir->is_running = 1; download_status_reset(&dir->v2_ns_dl_status); - rs = router_get_consensus_status_by_id(dir->digest); - if (rs && !rs->is_running) { - rs->is_running = 1; + rs = router_get_mutable_consensus_status_by_id(dir->digest); + if (rs) { rs->last_dir_503_at = 0; control_event_networkstatus_changed_single(rs); } - }); + } SMARTLIST_FOREACH_END(dir); } router_dir_info_changed(); } @@ -1326,148 +1356,160 @@ router_reset_status_download_failures(void) mark_all_trusteddirservers_up(); } -/** Return true iff router1 and router2 have the same /16 network. */ +/** Return true iff router1 and router2 have similar enough network addresses + * that we should treat them as being in the same family */ static INLINE int -routers_in_same_network_family(routerinfo_t *r1, routerinfo_t *r2) +addrs_in_same_network_family(const tor_addr_t *a1, + const tor_addr_t *a2) { - return (r1->addr & 0xffff0000) == (r2->addr & 0xffff0000); + /* XXXX MOVE ? */ + return 0 == tor_addr_compare_masked(a1, a2, 16, CMP_SEMANTIC); } -/** Look through the routerlist and identify routers that - * advertise the same /16 network address as <b>router</b>. - * Add each of them to <b>sl</b>. +/** + * Add all the family of <b>node</b>, including <b>node</b> itself, to + * the smartlist <b>sl</b>. + * + * This is used to make sure we don't pick siblings in a single path, or + * pick more than one relay from a family for our entry guard list. + * Note that a node may be added to <b>sl</b> more than once if it is + * part of <b>node</b>'s family for more than one reason. */ -static void -routerlist_add_network_family(smartlist_t *sl, routerinfo_t *router) +void +nodelist_add_node_and_family(smartlist_t *sl, const node_t *node) { - SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, r, + /* XXXX MOVE */ + const smartlist_t *all_nodes = nodelist_get_list(); + const smartlist_t *declared_family; + const or_options_t *options = get_options(); + + tor_assert(node); + + declared_family = node_get_declared_family(node); + + /* Let's make sure that we have the node itself, if it's a real node. */ { - if (router != r && routers_in_same_network_family(router, r)) - smartlist_add(sl, r); - }); -} + const node_t *real_node = node_get_by_id(node->identity); + if (real_node) + smartlist_add(sl, (node_t*)real_node); + } -/** Add all the family of <b>router</b> to the smartlist <b>sl</b>. - * This is used to make sure we don't pick siblings in a single path, - * or pick more than one relay from a family for our entry guard list. - */ -void -routerlist_add_family(smartlist_t *sl, routerinfo_t *router) -{ - routerinfo_t *r; - config_line_t *cl; - or_options_t *options = get_options(); + /* First, add any nodes with similar network addresses. */ + if (options->EnforceDistinctSubnets) { + tor_addr_t node_addr; + node_get_addr(node, &node_addr); - /* First, add any routers with similar network addresses. */ - if (options->EnforceDistinctSubnets) - routerlist_add_network_family(sl, router); + SMARTLIST_FOREACH_BEGIN(all_nodes, const node_t *, node2) { + tor_addr_t a; + node_get_addr(node2, &a); + if (addrs_in_same_network_family(&a, &node_addr)) + smartlist_add(sl, (void*)node2); + } SMARTLIST_FOREACH_END(node2); + } - if (router->declared_family) { - /* Add every r such that router declares familyness with r, and r + /* Now, add all nodes in the declared_family of this node, if they + * also declare this node to be in their family. */ + if (declared_family) { + /* Add every r such that router declares familyness with node, and node * declares familyhood with router. */ - SMARTLIST_FOREACH(router->declared_family, const char *, n, - { - if (!(r = router_get_by_nickname(n, 0))) - continue; - if (!r->declared_family) - continue; - SMARTLIST_FOREACH(r->declared_family, const char *, n2, - { - if (router_nickname_matches(router, n2)) - smartlist_add(sl, r); - }); - }); + SMARTLIST_FOREACH_BEGIN(declared_family, const char *, name) { + const node_t *node2; + const smartlist_t *family2; + if (!(node2 = node_get_by_nickname(name, 0))) + continue; + if (!(family2 = node_get_declared_family(node2))) + continue; + SMARTLIST_FOREACH_BEGIN(family2, const char *, name2) { + if (node_nickname_matches(node, name2)) { + smartlist_add(sl, (void*)node2); + break; + } + } SMARTLIST_FOREACH_END(name2); + } SMARTLIST_FOREACH_END(name); } /* If the user declared any families locally, honor those too. */ - for (cl = options->NodeFamilies; cl; cl = cl->next) { - if (router_nickname_is_in_list(router, cl->value)) { - add_nickname_list_to_smartlist(sl, cl->value, 0); - } + if (options->NodeFamilySets) { + SMARTLIST_FOREACH(options->NodeFamilySets, const routerset_t *, rs, { + if (routerset_contains_node(rs, node)) { + routerset_get_all_nodes(sl, rs, NULL, 0); + } + }); + } +} + +/** Given a <b>router</b>, add every node_t in its family (including the + * node itself!) to <b>sl</b>. + * + * Note the type mismatch: This function takes a routerinfo, but adds nodes + * to the smartlist! + */ +static void +routerlist_add_node_and_family(smartlist_t *sl, const routerinfo_t *router) +{ + /* XXXX MOVE ? */ + node_t fake_node; + const node_t *node = node_get_by_id(router->cache_info.identity_digest);; + if (node == NULL) { + memset(&fake_node, 0, sizeof(fake_node)); + fake_node.ri = (routerinfo_t *)router; + memcpy(fake_node.identity, router->cache_info.identity_digest, DIGEST_LEN); + node = &fake_node; } + nodelist_add_node_and_family(sl, node); } -/** Return true iff r is named by some nickname in <b>lst</b>. */ +/** Return true iff <b>node</b> is named by some nickname in <b>lst</b>. */ static INLINE int -router_in_nickname_smartlist(smartlist_t *lst, routerinfo_t *r) +node_in_nickname_smartlist(const smartlist_t *lst, const node_t *node) { + /* XXXX MOVE */ if (!lst) return 0; - SMARTLIST_FOREACH(lst, const char *, name, - if (router_nickname_matches(r, name)) - return 1;); + SMARTLIST_FOREACH(lst, const char *, name, { + if (node_nickname_matches(node, name)) + return 1; + }); return 0; } /** Return true iff r1 and r2 are in the same family, but not the same * router. */ int -routers_in_same_family(routerinfo_t *r1, routerinfo_t *r2) +nodes_in_same_family(const node_t *node1, const node_t *node2) { - or_options_t *options = get_options(); - config_line_t *cl; + /* XXXX MOVE */ + const or_options_t *options = get_options(); - if (options->EnforceDistinctSubnets && routers_in_same_network_family(r1,r2)) - return 1; - - if (router_in_nickname_smartlist(r1->declared_family, r2) && - router_in_nickname_smartlist(r2->declared_family, r1)) - return 1; - - for (cl = options->NodeFamilies; cl; cl = cl->next) { - if (router_nickname_is_in_list(r1, cl->value) && - router_nickname_is_in_list(r2, cl->value)) + /* Are they in the same family because of their addresses? */ + if (options->EnforceDistinctSubnets) { + tor_addr_t a1, a2; + node_get_addr(node1, &a1); + node_get_addr(node2, &a2); + if (addrs_in_same_network_family(&a1, &a2)) return 1; } - return 0; -} -/** Given a (possibly NULL) comma-and-whitespace separated list of nicknames, - * see which nicknames in <b>list</b> name routers in our routerlist, and add - * the routerinfos for those routers to <b>sl</b>. If <b>must_be_running</b>, - * only include routers that we think are running. - * Warn if any non-Named routers are specified by nickname. - */ -void -add_nickname_list_to_smartlist(smartlist_t *sl, const char *list, - int must_be_running) -{ - routerinfo_t *router; - smartlist_t *nickname_list; - int have_dir_info = router_have_minimum_dir_info(); - - if (!list) - return; /* nothing to do */ - tor_assert(sl); - - nickname_list = smartlist_create(); - if (!warned_nicknames) - warned_nicknames = smartlist_create(); + /* Are they in the same family because the agree they are? */ + { + const smartlist_t *f1, *f2; + f1 = node_get_declared_family(node1); + f2 = node_get_declared_family(node2); + if (f1 && f2 && + node_in_nickname_smartlist(f1, node2) && + node_in_nickname_smartlist(f2, node1)) + return 1; + } - smartlist_split_string(nickname_list, list, ",", - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + /* Are they in the same option because the user says they are? */ + if (options->NodeFamilySets) { + SMARTLIST_FOREACH(options->NodeFamilySets, const routerset_t *, rs, { + if (routerset_contains_node(rs, node1) && + routerset_contains_node(rs, node2)) + return 1; + }); + } - SMARTLIST_FOREACH(nickname_list, const char *, nick, { - int warned; - if (!is_legal_nickname_or_hexdigest(nick)) { - log_warn(LD_CONFIG, "Nickname '%s' is misformed; skipping", nick); - continue; - } - router = router_get_by_nickname(nick, 1); - warned = smartlist_string_isin(warned_nicknames, nick); - if (router) { - if (!must_be_running || router->is_running) { - smartlist_add(sl,router); - } - } else if (!router_get_consensus_status_by_nickname(nick,1)) { - if (!warned) { - log_fn(have_dir_info ? LOG_WARN : LOG_INFO, LD_CONFIG, - "Nickname list includes '%s' which isn't a known router.",nick); - smartlist_add(warned_nicknames, tor_strdup(nick)); - } - } - }); - SMARTLIST_FOREACH(nickname_list, char *, nick, tor_free(nick)); - smartlist_free(nickname_list); + return 0; } /** Return 1 iff any member of the (possibly NULL) comma-separated list @@ -1475,7 +1517,7 @@ add_nickname_list_to_smartlist(smartlist_t *sl, const char *list, * return 0. */ int -router_nickname_is_in_list(routerinfo_t *router, const char *list) +router_nickname_is_in_list(const routerinfo_t *router, const char *list) { smartlist_t *nickname_list; int v = 0; @@ -1484,7 +1526,7 @@ router_nickname_is_in_list(routerinfo_t *router, const char *list) return 0; /* definitely not */ tor_assert(router); - nickname_list = smartlist_create(); + nickname_list = smartlist_new(); smartlist_split_string(nickname_list, list, ",", SPLIT_SKIP_SPACE|SPLIT_STRIP_SPACE|SPLIT_IGNORE_BLANK, 0); SMARTLIST_FOREACH(nickname_list, const char *, cp, @@ -1494,34 +1536,32 @@ router_nickname_is_in_list(routerinfo_t *router, const char *list) return v; } -/** Add every suitable router from our routerlist to <b>sl</b>, so that +/** Add every suitable node from our nodelist to <b>sl</b>, so that * we can pick a node for a circuit. */ static void -router_add_running_routers_to_smartlist(smartlist_t *sl, int allow_invalid, - int need_uptime, int need_capacity, - int need_guard) -{ - if (!routerlist) - return; +router_add_running_nodes_to_smartlist(smartlist_t *sl, int allow_invalid, + int need_uptime, int need_capacity, + int need_guard, int need_desc) +{ /* XXXX MOVE */ + SMARTLIST_FOREACH_BEGIN(nodelist_get_list(), const node_t *, node) { + if (!node->is_running || + (!node->is_valid && !allow_invalid)) + continue; + if (need_desc && !(node->ri || (node->rs && node->md))) + continue; + if (node->ri && node->ri->purpose != ROUTER_PURPOSE_GENERAL) + continue; + if (node_is_unreliable(node, need_uptime, need_capacity, need_guard)) + continue; - SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, router, - { - if (router->is_running && - router->purpose == ROUTER_PURPOSE_GENERAL && - (router->is_valid || allow_invalid) && - !router_is_unreliable(router, need_uptime, - need_capacity, need_guard)) { - /* If it's running, and it's suitable according to the - * other flags we had in mind */ - smartlist_add(sl, router); - } - }); + smartlist_add(sl, (void *)node); + } SMARTLIST_FOREACH_END(node); } /** Look through the routerlist until we find a router that has my key. Return it. */ -routerinfo_t * +const routerinfo_t * routerlist_find_my_routerinfo(void) { if (!routerlist) @@ -1541,13 +1581,13 @@ routerlist_find_my_routerinfo(void) * Don't exit enclave to excluded relays -- it wouldn't actually * hurt anything, but this way there are fewer confused users. */ -routerinfo_t * +const node_t * router_find_exact_exit_enclave(const char *address, uint16_t port) -{ +{/*XXXX MOVE*/ uint32_t addr; struct in_addr in; tor_addr_t a; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (!tor_inet_aton(address, &in)) return NULL; /* it's not an IP already */ @@ -1555,14 +1595,13 @@ router_find_exact_exit_enclave(const char *address, uint16_t port) tor_addr_from_ipv4h(&a, addr); - SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, router, - { - if (router->addr == addr && - router->is_running && - compare_tor_addr_to_addr_policy(&a, port, router->exit_policy) == + SMARTLIST_FOREACH(nodelist_get_list(), const node_t *, node, { + if (node_get_addr_ipv4h(node) == addr && + node->is_running && + compare_tor_addr_to_node_policy(&a, port, node) == ADDR_POLICY_ACCEPTED && - !routerset_contains_router(options->_ExcludeExitNodesUnion, router)) - return router; + !routerset_contains_node(options->_ExcludeExitNodesUnion, node)) + return node; }); return NULL; } @@ -1574,14 +1613,14 @@ router_find_exact_exit_enclave(const char *address, uint16_t port) * If <b>need_guard</b>, we require that the router is a possible entry guard. */ int -router_is_unreliable(routerinfo_t *router, int need_uptime, - int need_capacity, int need_guard) +node_is_unreliable(const node_t *node, int need_uptime, + int need_capacity, int need_guard) { - if (need_uptime && !router->is_stable) + if (need_uptime && !node->is_stable) return 1; - if (need_capacity && !router->is_fast) + if (need_capacity && !node->is_fast) return 1; - if (need_guard && !router->is_possible_guard) + if (need_guard && !node->is_possible_guard) return 1; return 0; } @@ -1589,7 +1628,7 @@ router_is_unreliable(routerinfo_t *router, int need_uptime, /** Return the smaller of the router's configured BandwidthRate * and its advertised capacity. */ uint32_t -router_get_advertised_bandwidth(routerinfo_t *router) +router_get_advertised_bandwidth(const routerinfo_t *router) { if (router->bandwidthcapacity < router->bandwidthrate) return router->bandwidthcapacity; @@ -1603,7 +1642,7 @@ router_get_advertised_bandwidth(routerinfo_t *router) /** Return the smaller of the router's configured BandwidthRate * and its advertised capacity, capped by max-believe-bw. */ uint32_t -router_get_advertised_bandwidth_capped(routerinfo_t *router) +router_get_advertised_bandwidth_capped(const routerinfo_t *router) { uint32_t result = router->bandwidthcapacity; if (result > router->bandwidthrate) @@ -1645,13 +1684,10 @@ kb_to_bytes(uint32_t bw) } /** Helper function: - * choose a random element of smartlist <b>sl</b>, weighted by + * choose a random element of smartlist <b>sl</b> of nodes, weighted by * the advertised bandwidth of each element using the consensus * bandwidth weights. * - * If <b>statuses</b> is zero, then <b>sl</b> is a list of - * routerinfo_t's. Otherwise it's a list of routerstatus_t's. - * * If <b>rule</b>==WEIGHT_FOR_EXIT. we're picking an exit node: consider all * nodes' bandwidth equally regardless of their Exit status, since there may * be some in the list because they exit to obscure ports. If @@ -1661,16 +1697,15 @@ kb_to_bytes(uint32_t bw) * guard node: consider all guard's bandwidth equally. Otherwise, weight * guards proportionally less. */ -static void * -smartlist_choose_by_bandwidth_weights(smartlist_t *sl, - bandwidth_weight_rule_t rule, - int statuses) +static const node_t * +smartlist_choose_node_by_bandwidth_weights(smartlist_t *sl, + bandwidth_weight_rule_t rule) { int64_t weight_scale; int64_t rand_bw; double Wg = -1, Wm = -1, We = -1, Wd = -1; double Wgb = -1, Wmb = -1, Web = -1, Wdb = -1; - double weighted_bw = 0; + double weighted_bw = 0, unweighted_bw = 0; double *bandwidths; double tmp = 0; unsigned int i; @@ -1760,15 +1795,14 @@ smartlist_choose_by_bandwidth_weights(smartlist_t *sl, bandwidths = tor_malloc_zero(sizeof(double)*smartlist_len(sl)); // Cycle through smartlist and total the bandwidth. - for (i = 0; i < (unsigned)smartlist_len(sl); ++i) { + SMARTLIST_FOREACH_BEGIN(sl, const node_t *, node) { int is_exit = 0, is_guard = 0, is_dir = 0, this_bw = 0, is_me = 0; double weight = 1; - if (statuses) { - routerstatus_t *status = smartlist_get(sl, i); - is_exit = status->is_exit && !status->is_bad_exit; - is_guard = status->is_possible_guard; - is_dir = (status->dir_port != 0); - if (!status->has_bandwidth) { + is_exit = node->is_exit && ! node->is_bad_exit; + is_guard = node->is_possible_guard; + is_dir = node_is_dir(node); + if (node->rs) { + if (!node->rs->has_bandwidth) { tor_free(bandwidths); /* This should never happen, unless all the authorites downgrade * to 0.2.0 or rogue routerstatuses get inserted into our consensus. */ @@ -1777,26 +1811,17 @@ smartlist_choose_by_bandwidth_weights(smartlist_t *sl, "old router selection algorithm."); return NULL; } - this_bw = kb_to_bytes(status->bandwidth); - if (router_digest_is_me(status->identity_digest)) - is_me = 1; + this_bw = kb_to_bytes(node->rs->bandwidth); + } else if (node->ri) { + /* bridge or other descriptor not in our consensus */ + this_bw = bridge_get_advertised_bandwidth_bounded(node->ri); + have_unknown = 1; } else { - routerstatus_t *rs; - routerinfo_t *router = smartlist_get(sl, i); - rs = router_get_consensus_status_by_id( - router->cache_info.identity_digest); - is_exit = router->is_exit && !router->is_bad_exit; - is_guard = router->is_possible_guard; - is_dir = (router->dir_port != 0); - if (rs && rs->has_bandwidth) { - this_bw = kb_to_bytes(rs->bandwidth); - } else { /* bridge or other descriptor not in our consensus */ - this_bw = bridge_get_advertised_bandwidth_bounded(router); - have_unknown = 1; - } - if (router_digest_is_me(router->cache_info.identity_digest)) - is_me = 1; + /* We can't use this one. */ + continue; } + is_me = router_digest_is_me(node->identity); + if (is_guard && is_exit) { weight = (is_dir ? Wdb*Wd : Wd); } else if (is_guard) { @@ -1807,13 +1832,14 @@ smartlist_choose_by_bandwidth_weights(smartlist_t *sl, weight = (is_dir ? Wmb*Wm : Wm); } - bandwidths[i] = weight*this_bw; + bandwidths[node_sl_idx] = weight*this_bw; weighted_bw += weight*this_bw; + unweighted_bw += this_bw; if (is_me) sl_last_weighted_bw_of_me = weight*this_bw; - } + } SMARTLIST_FOREACH_END(node); - /* XXXX023 this is a kludge to expose these values. */ + /* XXXX this is a kludge to expose these values. */ sl_last_total_weighted_bw = weighted_bw; log_debug(LD_CIRC, "Choosing node for rule %s based on weights " @@ -1824,10 +1850,20 @@ smartlist_choose_by_bandwidth_weights(smartlist_t *sl, /* If there is no bandwidth, choose at random */ if (DBL_TO_U64(weighted_bw) == 0) { /* Don't warn when using bridges/relays not in the consensus */ - if (!have_unknown) - log_warn(LD_CIRC, - "Weighted bandwidth is %f in node selection for rule %s", - weighted_bw, bandwidth_weight_rule_to_string(rule)); + if (!have_unknown) { +#define ZERO_BANDWIDTH_WARNING_INTERVAL (15) + static ratelim_t zero_bandwidth_warning_limit = + RATELIM_INIT(ZERO_BANDWIDTH_WARNING_INTERVAL); + char *msg; + if ((msg = rate_limit_log(&zero_bandwidth_warning_limit, + approx_time()))) { + log_warn(LD_CIRC, + "Weighted bandwidth is %f in node selection for rule %s " + "(unweighted was %f) %s", + weighted_bw, bandwidth_weight_rule_to_string(rule), + unweighted_bw, msg); + } + } tor_free(bandwidths); return smartlist_choose(sl); } @@ -1864,12 +1900,9 @@ smartlist_choose_by_bandwidth_weights(smartlist_t *sl, } /** Helper function: - * choose a random element of smartlist <b>sl</b>, weighted by + * choose a random node_t element of smartlist <b>sl</b>, weighted by * the advertised bandwidth of each element. * - * If <b>statuses</b> is zero, then <b>sl</b> is a list of - * routerinfo_t's. Otherwise it's a list of routerstatus_t's. - * * If <b>rule</b>==WEIGHT_FOR_EXIT. we're picking an exit node: consider all * nodes' bandwidth equally regardless of their Exit status, since there may * be some in the list because they exit to obscure ports. If @@ -1879,15 +1912,13 @@ smartlist_choose_by_bandwidth_weights(smartlist_t *sl, * guard node: consider all guard's bandwidth equally. Otherwise, weight * guards proportionally less. */ -static void * -smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, - int statuses) +static const node_t * +smartlist_choose_node_by_bandwidth(smartlist_t *sl, + bandwidth_weight_rule_t rule) { unsigned int i; unsigned int i_chosen; unsigned int i_has_been_chosen; - routerinfo_t *router; - routerstatus_t *status=NULL; int32_t *bandwidths; int is_exit; int is_guard; @@ -1928,57 +1959,43 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, guard_bits = bitarray_init_zero(smartlist_len(sl)); /* Iterate over all the routerinfo_t or routerstatus_t, and */ - for (i = 0; i < (unsigned)smartlist_len(sl); ++i) { + SMARTLIST_FOREACH_BEGIN(sl, const node_t *, node) { /* first, learn what bandwidth we think i has */ int is_known = 1; int32_t flags = 0; uint32_t this_bw = 0; - if (statuses) { - status = smartlist_get(sl, i); - if (router_digest_is_me(status->identity_digest)) - me_idx = i; - router = router_get_by_digest(status->identity_digest); - is_exit = status->is_exit; - is_guard = status->is_possible_guard; - if (status->has_bandwidth) { - this_bw = kb_to_bytes(status->bandwidth); + i = node_sl_idx; + + if (router_digest_is_me(node->identity)) + me_idx = node_sl_idx; + + is_exit = node->is_exit; + is_guard = node->is_possible_guard; + if (node->rs) { + if (node->rs->has_bandwidth) { + this_bw = kb_to_bytes(node->rs->bandwidth); } else { /* guess */ - /* XXX023 once consensuses always list bandwidths, we can take + /* XXX024 once consensuses always list bandwidths, we can take * this guessing business out. -RD */ is_known = 0; - flags = status->is_fast ? 1 : 0; + flags = node->rs->is_fast ? 1 : 0; flags |= is_exit ? 2 : 0; flags |= is_guard ? 4 : 0; } - } else { - routerstatus_t *rs; - router = smartlist_get(sl, i); - rs = router_get_consensus_status_by_id( - router->cache_info.identity_digest); - if (router_digest_is_me(router->cache_info.identity_digest)) - me_idx = i; - is_exit = router->is_exit; - is_guard = router->is_possible_guard; - if (rs && rs->has_bandwidth) { - this_bw = kb_to_bytes(rs->bandwidth); - } else if (rs) { /* guess; don't trust the descriptor */ - /* XXX023 once consensuses always list bandwidths, we can take - * this guessing business out. -RD */ - is_known = 0; - flags = router->is_fast ? 1 : 0; - flags |= is_exit ? 2 : 0; - flags |= is_guard ? 4 : 0; - } else /* bridge or other descriptor not in our consensus */ - this_bw = bridge_get_advertised_bandwidth_bounded(router); + } else if (node->ri) { + /* Must be a bridge if we're willing to use it */ + this_bw = bridge_get_advertised_bandwidth_bounded(node->ri); } + if (is_exit) bitarray_set(exit_bits, i); if (is_guard) bitarray_set(guard_bits, i); if (is_known) { - bandwidths[i] = (int32_t) this_bw; // safe since MAX_BELIEVABLE<INT32_MAX - // XXX this is no longer true! We don't always cap the bw anymore. Can - // a consensus make us overflow?-sh + bandwidths[i] = (int32_t) this_bw; + /* Casting this_bw to int32_t is safe because both kb_to_bytes + and bridge_get_advertised_bandwidth_bounded limit it to below + INT32_MAX. */ tor_assert(bandwidths[i] >= 0); if (is_guard) total_guard_bw += this_bw; @@ -1990,9 +2007,9 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, total_nonexit_bw += this_bw; } else { ++n_unknown; - bandwidths[i] = -flags; + bandwidths[node_sl_idx] = -flags; } - } + } SMARTLIST_FOREACH_END(node); /* Now, fill in the unknown values. */ if (n_unknown) { @@ -2078,7 +2095,7 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, } } - /* XXXX023 this is a kludge to expose these values. */ + /* XXXX this is a kludge to expose these values. */ sl_last_total_weighted_bw = total_bw; log_debug(LD_CIRC, "Total weighted bw = "U64_FORMAT @@ -2139,40 +2156,23 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, return smartlist_get(sl, i); } -/** Choose a random element of router list <b>sl</b>, weighted by - * the advertised bandwidth of each router. - */ -routerinfo_t * -routerlist_sl_choose_by_bandwidth(smartlist_t *sl, - bandwidth_weight_rule_t rule) -{ - routerinfo_t *ret; - if ((ret = smartlist_choose_by_bandwidth_weights(sl, rule, 0))) { - return ret; - } else { - return smartlist_choose_by_bandwidth(sl, rule, 0); - } -} - /** Choose a random element of status list <b>sl</b>, weighted by - * the advertised bandwidth of each status. - */ -routerstatus_t * -routerstatus_sl_choose_by_bandwidth(smartlist_t *sl, - bandwidth_weight_rule_t rule) -{ - /* We are choosing neither exit nor guard here. Weight accordingly. */ - routerstatus_t *ret; - if ((ret = smartlist_choose_by_bandwidth_weights(sl, rule, 1))) { + * the advertised bandwidth of each node */ +const node_t * +node_sl_choose_by_bandwidth(smartlist_t *sl, + bandwidth_weight_rule_t rule) +{ /*XXXX MOVE */ + const node_t *ret; + if ((ret = smartlist_choose_node_by_bandwidth_weights(sl, rule))) { return ret; } else { - return smartlist_choose_by_bandwidth(sl, rule, 1); + return smartlist_choose_node_by_bandwidth(sl, rule); } } -/** Return a random running router from the routerlist. Never - * pick a node whose routerinfo is in - * <b>excludedsmartlist</b>, or whose routerinfo matches <b>excludedset</b>, +/** Return a random running node from the nodelist. Never + * pick a node that is in + * <b>excludedsmartlist</b>, or which matches <b>excludedset</b>, * even if they are the only nodes available. * If <b>CRN_NEED_UPTIME</b> is set in flags and any router has more than * a minimum uptime, return one of those. @@ -2184,21 +2184,26 @@ routerstatus_sl_choose_by_bandwidth(smartlist_t *sl, * If <b>CRN_WEIGHT_AS_EXIT</b> is set in flags, we weight bandwidths as if * picking an exit node, otherwise we weight bandwidths for picking a relay * node (that is, possibly discounting exit nodes). + * If <b>CRN_NEED_DESC</b> is set in flags, we only consider nodes that + * have a routerinfo or microdescriptor -- that is, enough info to be + * used to build a circuit. */ -routerinfo_t * +const node_t * router_choose_random_node(smartlist_t *excludedsmartlist, routerset_t *excludedset, router_crn_flags_t flags) -{ +{ /* XXXX MOVE */ const int need_uptime = (flags & CRN_NEED_UPTIME) != 0; const int need_capacity = (flags & CRN_NEED_CAPACITY) != 0; const int need_guard = (flags & CRN_NEED_GUARD) != 0; const int allow_invalid = (flags & CRN_ALLOW_INVALID) != 0; const int weight_for_exit = (flags & CRN_WEIGHT_AS_EXIT) != 0; + const int need_desc = (flags & CRN_NEED_DESC) != 0; - smartlist_t *sl=smartlist_create(), - *excludednodes=smartlist_create(); - routerinfo_t *choice = NULL, *r; + smartlist_t *sl=smartlist_new(), + *excludednodes=smartlist_new(); + const node_t *choice = NULL; + const routerinfo_t *r; bandwidth_weight_rule_t rule; tor_assert(!(weight_for_exit && need_guard)); @@ -2208,29 +2213,26 @@ router_choose_random_node(smartlist_t *excludedsmartlist, /* Exclude relays that allow single hop exit circuits, if the user * wants to (such relays might be risky) */ if (get_options()->ExcludeSingleHopRelays) { - routerlist_t *rl = router_get_routerlist(); - SMARTLIST_FOREACH(rl->routers, routerinfo_t *, r, - if (r->allow_single_hop_exits) { - smartlist_add(excludednodes, r); + SMARTLIST_FOREACH(nodelist_get_list(), node_t *, node, + if (node_allows_single_hop_exits(node)) { + smartlist_add(excludednodes, node); }); } - if ((r = routerlist_find_my_routerinfo())) { - smartlist_add(excludednodes, r); - routerlist_add_family(excludednodes, r); - } + if ((r = routerlist_find_my_routerinfo())) + routerlist_add_node_and_family(excludednodes, r); - router_add_running_routers_to_smartlist(sl, allow_invalid, - need_uptime, need_capacity, - need_guard); + router_add_running_nodes_to_smartlist(sl, allow_invalid, + need_uptime, need_capacity, + need_guard, need_desc); smartlist_subtract(sl,excludednodes); if (excludedsmartlist) smartlist_subtract(sl,excludedsmartlist); if (excludedset) - routerset_subtract_routers(sl,excludedset); + routerset_subtract_nodes(sl,excludedset); // Always weight by bandwidth - choice = routerlist_sl_choose_by_bandwidth(sl, rule); + choice = node_sl_choose_by_bandwidth(sl, rule); smartlist_free(sl); if (!choice && (need_uptime || need_capacity || need_guard)) { @@ -2253,35 +2255,90 @@ router_choose_random_node(smartlist_t *excludedsmartlist, return choice; } -/** Helper: Return true iff the <b>identity_digest</b> and <b>nickname</b> - * combination of a router, encoded in hexadecimal, matches <b>hexdigest</b> - * (which is optionally prefixed with a single dollar sign). Return false if - * <b>hexdigest</b> is malformed, or it doesn't match. */ -static INLINE int -hex_digest_matches(const char *hexdigest, const char *identity_digest, - const char *nickname, int is_named) +/** Helper: given an extended nickname in <b>hexdigest</b> try to decode it. + * Return 0 on success, -1 on failure. Store the result into the + * DIGEST_LEN-byte buffer at <b>digest_out</b>, the single character at + * <b>nickname_qualifier_char_out</b>, and the MAXNICKNAME_LEN+1-byte buffer + * at <b>nickname_out</b>. + * + * The recognized format is: + * HexName = Dollar? HexDigest NamePart? + * Dollar = '?' + * HexDigest = HexChar*20 + * HexChar = 'a'..'f' | 'A'..'F' | '0'..'9' + * NamePart = QualChar Name + * QualChar = '=' | '~' + * Name = NameChar*(1..MAX_NICKNAME_LEN) + * NameChar = Any ASCII alphanumeric character + */ +int +hex_digest_nickname_decode(const char *hexdigest, + char *digest_out, + char *nickname_qualifier_char_out, + char *nickname_out) { - char digest[DIGEST_LEN]; size_t len; + tor_assert(hexdigest); if (hexdigest[0] == '$') ++hexdigest; len = strlen(hexdigest); - if (len < HEX_DIGEST_LEN) + if (len < HEX_DIGEST_LEN) { + return -1; + } else if (len > HEX_DIGEST_LEN && (hexdigest[HEX_DIGEST_LEN] == '=' || + hexdigest[HEX_DIGEST_LEN] == '~') && + len <= HEX_DIGEST_LEN+1+MAX_NICKNAME_LEN) { + *nickname_qualifier_char_out = hexdigest[HEX_DIGEST_LEN]; + strlcpy(nickname_out, hexdigest+HEX_DIGEST_LEN+1 , MAX_NICKNAME_LEN+1); + } else if (len == HEX_DIGEST_LEN) { + ; + } else { + return -1; + } + + if (base16_decode(digest_out, DIGEST_LEN, hexdigest, HEX_DIGEST_LEN)<0) + return -1; + return 0; +} + +/** Helper: Return true iff the <b>identity_digest</b> and <b>nickname</b> + * combination of a router, encoded in hexadecimal, matches <b>hexdigest</b> + * (which is optionally prefixed with a single dollar sign). Return false if + * <b>hexdigest</b> is malformed, or it doesn't match. */ +static int +hex_digest_nickname_matches(const char *hexdigest, const char *identity_digest, + const char *nickname, int is_named) +{ + char digest[DIGEST_LEN]; + char nn_char='\0'; + char nn_buf[MAX_NICKNAME_LEN+1]; + + if (hex_digest_nickname_decode(hexdigest, digest, &nn_char, nn_buf) == -1) return 0; - else if (len > HEX_DIGEST_LEN && - (hexdigest[HEX_DIGEST_LEN] == '=' || - hexdigest[HEX_DIGEST_LEN] == '~')) { - if (strcasecmp(hexdigest+HEX_DIGEST_LEN+1, nickname)) + + if (nn_char == '=' || nn_char == '~') { + if (!nickname) return 0; - if (hexdigest[HEX_DIGEST_LEN] == '=' && !is_named) + if (strcasecmp(nn_buf, nickname)) + return 0; + if (nn_char == '=' && !is_named) return 0; } - if (base16_decode(digest, DIGEST_LEN, hexdigest, HEX_DIGEST_LEN)<0) - return 0; - return (tor_memeq(digest, identity_digest, DIGEST_LEN)); + return tor_memeq(digest, identity_digest, DIGEST_LEN); +} + +/** Return true iff <b>router</b> is listed as named in the current + * consensus. */ +int +router_is_named(const routerinfo_t *router) +{ + const char *digest = + networkstatus_get_router_digest_by_nickname(router->nickname); + + return (digest && + tor_memeq(digest, router->cache_info.identity_digest, DIGEST_LEN)); } /** Return true iff the digest of <b>router</b>'s identity key, @@ -2289,10 +2346,12 @@ hex_digest_matches(const char *hexdigest, const char *identity_digest, * optionally prefixed with a single dollar sign). Return false if * <b>hexdigest</b> is malformed, or it doesn't match. */ static INLINE int -router_hex_digest_matches(routerinfo_t *router, const char *hexdigest) +router_hex_digest_matches(const routerinfo_t *router, const char *hexdigest) { - return hex_digest_matches(hexdigest, router->cache_info.identity_digest, - router->nickname, router->is_named); + return hex_digest_nickname_matches(hexdigest, + router->cache_info.identity_digest, + router->nickname, + router_is_named(router)); } /** Return true if <b>router</b>'s nickname matches <b>nickname</b> @@ -2300,20 +2359,43 @@ router_hex_digest_matches(routerinfo_t *router, const char *hexdigest) * matches a hexadecimal value stored in <b>nickname</b>. Return * false otherwise. */ static int -router_nickname_matches(routerinfo_t *router, const char *nickname) +router_nickname_matches(const routerinfo_t *router, const char *nickname) { if (nickname[0]!='$' && !strcasecmp(router->nickname, nickname)) return 1; return router_hex_digest_matches(router, nickname); } +/** Return true if <b>node</b>'s nickname matches <b>nickname</b> + * (case-insensitive), or if <b>node's</b> identity key digest + * matches a hexadecimal value stored in <b>nickname</b>. Return + * false otherwise. */ +static int +node_nickname_matches(const node_t *node, const char *nickname) +{ + const char *n = node_get_nickname(node); + if (n && nickname[0]!='$' && !strcasecmp(n, nickname)) + return 1; + return hex_digest_nickname_matches(nickname, + node->identity, + n, + node_is_named(node)); +} + /** Return the router in our routerlist whose (case-insensitive) * nickname or (case-sensitive) hexadecimal key digest is * <b>nickname</b>. Return NULL if no such router is known. */ -routerinfo_t * +const routerinfo_t * router_get_by_nickname(const char *nickname, int warn_if_unnamed) { +#if 1 + const node_t *node = node_get_by_nickname(nickname, warn_if_unnamed); + if (node) + return node->ri; + else + return NULL; +#else int maybedigest; char digest[DIGEST_LEN]; routerinfo_t *best_match=NULL; @@ -2357,17 +2439,14 @@ router_get_by_nickname(const char *nickname, int warn_if_unnamed) if (best_match) { if (warn_if_unnamed && n_matches > 1) { - smartlist_t *fps = smartlist_create(); + smartlist_t *fps = smartlist_new(); int any_unwarned = 0; - SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, router, - { + SMARTLIST_FOREACH_BEGIN(routerlist->routers, routerinfo_t *, router) { routerstatus_t *rs; - char *desc; - size_t dlen; char fp[HEX_DIGEST_LEN+1]; if (strcasecmp(router->nickname, nickname)) continue; - rs = router_get_consensus_status_by_id( + rs = router_get_mutable_consensus_status_by_id( router->cache_info.identity_digest); if (rs && !rs->name_lookup_warned) { rs->name_lookup_warned = 1; @@ -2375,12 +2454,9 @@ router_get_by_nickname(const char *nickname, int warn_if_unnamed) } base16_encode(fp, sizeof(fp), router->cache_info.identity_digest, DIGEST_LEN); - dlen = 32 + HEX_DIGEST_LEN + strlen(router->address); - desc = tor_malloc(dlen); - tor_snprintf(desc, dlen, "\"$%s\" for the one at %s:%d", + smartlist_add_asprintf(fps, "\"$%s\" for the one at %s:%d", fp, router->address, router->or_port); - smartlist_add(fps, desc); - }); + } SMARTLIST_FOREACH_END(router); if (any_unwarned) { char *alternatives = smartlist_join_strings(fps, "; ",0,NULL); log_warn(LD_CONFIG, @@ -2393,7 +2469,7 @@ router_get_by_nickname(const char *nickname, int warn_if_unnamed) SMARTLIST_FOREACH(fps, char *, cp, tor_free(cp)); smartlist_free(fps); } else if (warn_if_unnamed) { - routerstatus_t *rs = router_get_consensus_status_by_id( + routerstatus_t *rs = router_get_mutable_consensus_status_by_id( best_match->cache_info.identity_digest); if (rs && !rs->name_lookup_warned) { char fp[HEX_DIGEST_LEN+1]; @@ -2409,27 +2485,15 @@ router_get_by_nickname(const char *nickname, int warn_if_unnamed) } return best_match; } - return NULL; -} - -/** Try to find a routerinfo for <b>digest</b>. If we don't have one, - * return 1. If we do, ask tor_version_as_new_as() for the answer. - */ -int -router_digest_version_as_new_as(const char *digest, const char *cutoff) -{ - routerinfo_t *router = router_get_by_digest(digest); - if (!router) - return 1; - return tor_version_as_new_as(router->platform, cutoff); +#endif } /** Return true iff <b>digest</b> is the digest of the identity key of a * trusted directory matching at least one bit of <b>type</b>. If <b>type</b> * is zero, any authority is okay. */ int -router_digest_is_trusted_dir_type(const char *digest, authority_type_t type) +router_digest_is_trusted_dir_type(const char *digest, dirinfo_type_t type) { if (!trusted_dir_servers) return 0; @@ -2473,44 +2537,20 @@ hexdigest_to_digest(const char *hexdigest, char *digest) /** Return the router in our routerlist whose hexadecimal key digest * is <b>hexdigest</b>. Return NULL if no such router is known. */ -routerinfo_t * +const routerinfo_t * router_get_by_hexdigest(const char *hexdigest) { - char digest[DIGEST_LEN]; - size_t len; - routerinfo_t *ri; - - tor_assert(hexdigest); - if (!routerlist) - return NULL; - if (hexdigest[0]=='$') - ++hexdigest; - len = strlen(hexdigest); - if (hexdigest_to_digest(hexdigest, digest) < 0) + if (is_legal_nickname(hexdigest)) return NULL; - ri = router_get_by_digest(digest); - - if (ri && len > HEX_DIGEST_LEN) { - if (hexdigest[HEX_DIGEST_LEN] == '=') { - if (strcasecmp(ri->nickname, hexdigest+HEX_DIGEST_LEN+1) || - !ri->is_named) - return NULL; - } else if (hexdigest[HEX_DIGEST_LEN] == '~') { - if (strcasecmp(ri->nickname, hexdigest+HEX_DIGEST_LEN+1)) - return NULL; - } else { - return NULL; - } - } - - return ri; + /* It's not a legal nickname, so it must be a hexdigest or nothing. */ + return router_get_by_nickname(hexdigest, 1); } -/** Return the router in our routerlist whose 20-byte key digest - * is <b>digest</b>. Return NULL if no such router is known. */ +/** As router_get_by_id_digest,but return a pointer that you're allowed to + * modify */ routerinfo_t * -router_get_by_digest(const char *digest) +router_get_mutable_by_digest(const char *digest) { tor_assert(digest); @@ -2521,6 +2561,14 @@ router_get_by_digest(const char *digest) return rimap_get(routerlist->identity_map, digest); } +/** Return the router in our routerlist whose 20-byte key digest + * is <b>digest</b>. Return NULL if no such router is known. */ +const routerinfo_t * +router_get_by_id_digest(const char *digest) +{ + return router_get_mutable_by_digest(digest); +} + /** Return the router in our routerlist whose 20-byte descriptor * is <b>digest</b>. Return NULL if no such router is known. */ signed_descriptor_t * @@ -2571,7 +2619,7 @@ extrainfo_get_by_descriptor_digest(const char *digest) * The caller must not free the string returned. */ static const char * -signed_descriptor_get_body_impl(signed_descriptor_t *desc, +signed_descriptor_get_body_impl(const signed_descriptor_t *desc, int with_annotations) { const char *r = NULL; @@ -2620,7 +2668,7 @@ signed_descriptor_get_body_impl(signed_descriptor_t *desc, * The caller must not free the string returned. */ const char * -signed_descriptor_get_body(signed_descriptor_t *desc) +signed_descriptor_get_body(const signed_descriptor_t *desc) { return signed_descriptor_get_body_impl(desc, 0); } @@ -2628,7 +2676,7 @@ signed_descriptor_get_body(signed_descriptor_t *desc) /** As signed_descriptor_get_body(), but points to the beginning of the * annotations section rather than the beginning of the descriptor. */ const char * -signed_descriptor_get_annotations(signed_descriptor_t *desc) +signed_descriptor_get_annotations(const signed_descriptor_t *desc) { return signed_descriptor_get_body_impl(desc, 1); } @@ -2639,8 +2687,8 @@ router_get_routerlist(void) { if (PREDICT_UNLIKELY(!routerlist)) { routerlist = tor_malloc_zero(sizeof(routerlist_t)); - routerlist->routers = smartlist_create(); - routerlist->old_routers = smartlist_create(); + routerlist->routers = smartlist_new(); + routerlist->old_routers = smartlist_new(); routerlist->identity_map = rimap_new(); routerlist->desc_digest_map = sdmap_new(); routerlist->desc_by_eid_map = sdmap_new(); @@ -2672,16 +2720,15 @@ routerinfo_free(routerinfo_t *router) tor_free(router->platform); tor_free(router->contact_info); if (router->onion_pkey) - crypto_free_pk_env(router->onion_pkey); + crypto_pk_free(router->onion_pkey); if (router->identity_pkey) - crypto_free_pk_env(router->identity_pkey); + crypto_pk_free(router->identity_pkey); if (router->declared_family) { SMARTLIST_FOREACH(router->declared_family, char *, s, tor_free(s)); smartlist_free(router->declared_family); } addr_policy_list_free(router->exit_policy); - /* XXXX Remove if this turns out to affect performance. */ memset(router, 77, sizeof(routerinfo_t)); tor_free(router); @@ -2696,7 +2743,6 @@ extrainfo_free(extrainfo_t *extrainfo) tor_free(extrainfo->cache_info.signed_descriptor_body); tor_free(extrainfo->pending_sig); - /* XXXX remove this if it turns out to slow us down. */ memset(extrainfo, 88, sizeof(extrainfo_t)); /* debug bad memory usage */ tor_free(extrainfo); } @@ -2710,7 +2756,6 @@ signed_descriptor_free(signed_descriptor_t *sd) tor_free(sd->signed_descriptor_body); - /* XXXX remove this once more bugs go away. */ memset(sd, 99, sizeof(signed_descriptor_t)); /* Debug bad mem usage */ tor_free(sd); } @@ -2816,8 +2861,7 @@ routerlist_insert(routerlist_t *rl, routerinfo_t *ri) routerinfo_t *ri_old; signed_descriptor_t *sd_old; { - /* XXXX Remove if this slows us down. */ - routerinfo_t *ri_generated = router_get_my_routerinfo(); + const routerinfo_t *ri_generated = router_get_my_routerinfo(); tor_assert(ri_generated != ri); } tor_assert(ri->cache_info.routerlist_index == -1); @@ -2846,6 +2890,7 @@ routerlist_insert(routerlist_t *rl, routerinfo_t *ri) &ri->cache_info); smartlist_add(rl->routers, ri); ri->cache_info.routerlist_index = smartlist_len(rl->routers) - 1; + nodelist_add_routerinfo(ri); router_dir_info_changed(); #ifdef DEBUG_ROUTERLIST routerlist_assert_ok(rl); @@ -2866,7 +2911,6 @@ extrainfo_insert(routerlist_t *rl, extrainfo_t *ei) extrainfo_t *ei_tmp; { - /* XXXX remove this code if it slows us down. */ extrainfo_t *ei_generated = router_get_my_extrainfo(); tor_assert(ei_generated != ei); } @@ -2912,8 +2956,7 @@ static void routerlist_insert_old(routerlist_t *rl, routerinfo_t *ri) { { - /* XXXX remove this code if it slows us down. */ - routerinfo_t *ri_generated = router_get_my_routerinfo(); + const routerinfo_t *ri_generated = router_get_my_routerinfo(); tor_assert(ri_generated != ri); } tor_assert(ri->cache_info.routerlist_index == -1); @@ -2953,6 +2996,8 @@ routerlist_remove(routerlist_t *rl, routerinfo_t *ri, int make_old, time_t now) tor_assert(0 <= idx && idx < smartlist_len(rl->routers)); tor_assert(smartlist_get(rl->routers, idx) == ri); + nodelist_remove_routerinfo(ri); + /* make sure the rephist module knows that it's not running */ rep_hist_note_router_unreachable(ri->cache_info.identity_digest, now); @@ -3064,8 +3109,7 @@ routerlist_replace(routerlist_t *rl, routerinfo_t *ri_old, routerinfo_t *ri_tmp; extrainfo_t *ei_tmp; { - /* XXXX Remove this if it turns out to slow us down. */ - routerinfo_t *ri_generated = router_get_my_routerinfo(); + const routerinfo_t *ri_generated = router_get_my_routerinfo(); tor_assert(ri_generated != ri_new); } tor_assert(ri_old != ri_new); @@ -3075,6 +3119,9 @@ routerlist_replace(routerlist_t *rl, routerinfo_t *ri_old, tor_assert(0 <= idx && idx < smartlist_len(rl->routers)); tor_assert(smartlist_get(rl->routers, idx) == ri_old); + nodelist_remove_routerinfo(ri_old); + nodelist_add_routerinfo(ri_new); + router_dir_info_changed(); if (idx >= 0) { smartlist_set(rl->routers, idx, ri_new); @@ -3209,7 +3256,7 @@ void routerlist_reset_warnings(void) { if (!warned_nicknames) - warned_nicknames = smartlist_create(); + warned_nicknames = smartlist_new(); SMARTLIST_FOREACH(warned_nicknames, char *, cp, tor_free(cp)); smartlist_clear(warned_nicknames); /* now the list is empty. */ @@ -3221,28 +3268,25 @@ routerlist_reset_warnings(void) void router_set_status(const char *digest, int up) { - routerinfo_t *router; - routerstatus_t *status; + node_t *node; tor_assert(digest); SMARTLIST_FOREACH(trusted_dir_servers, trusted_dir_server_t *, d, if (tor_memeq(d->digest, digest, DIGEST_LEN)) d->is_running = up); - router = router_get_by_digest(digest); - if (router) { + node = node_get_mutable_by_id(digest); + if (node) { +#if 0 log_debug(LD_DIR,"Marking router %s as %s.", - router_describe(router), up ? "up" : "down"); - if (!up && router_is_me(router) && !we_are_hibernating()) + node_describe(node), up ? "up" : "down"); +#endif + if (!up && node_is_me(node) && !net_is_disabled()) log_warn(LD_NET, "We just marked ourself as down. Are your external " "addresses reachable?"); - router->is_running = up; - } - status = router_get_consensus_status_by_id(digest); - if (status && status->is_running != up) { - status->is_running = up; - control_event_networkstatus_changed_single(status); + node->is_running = up; } + router_dir_info_changed(); } @@ -3271,11 +3315,12 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, int from_cache, int from_fetch) { const char *id_digest; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); int authdir = authdir_mode_handles_descs(options, router->purpose); int authdir_believes_valid = 0; routerinfo_t *old_router; - networkstatus_t *consensus = networkstatus_get_latest_consensus(); + networkstatus_t *consensus = + networkstatus_get_latest_consensus_by_flavor(FLAV_NS); const smartlist_t *networkstatus_v2_list = networkstatus_get_v2_list(); int in_consensus = 0; @@ -3286,7 +3331,7 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, id_digest = router->cache_info.identity_digest; - old_router = router_get_by_digest(id_digest); + old_router = router_get_mutable_by_digest(id_digest); /* Make sure that we haven't already got this exact descriptor. */ if (sdmap_get(routerlist->desc_digest_map, @@ -3318,12 +3363,12 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, if (authdir) { if (authdir_wants_to_reject_router(router, msg, - !from_cache && !from_fetch)) { + !from_cache && !from_fetch, + &authdir_believes_valid)) { tor_assert(*msg); routerinfo_free(router); return ROUTER_AUTHDIR_REJECTS; } - authdir_believes_valid = router->is_valid; } else if (from_fetch) { /* Only check the descriptor digest against the network statuses when * we are receiving in response to a fetch. */ @@ -3350,14 +3395,15 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, { routerstatus_t *rs = - networkstatus_v2_find_entry(ns, id_digest); + networkstatus_v2_find_mutable_entry(ns, id_digest); if (rs && tor_memeq(rs->descriptor_digest, router->cache_info.signed_descriptor_digest, DIGEST_LEN)) rs->need_to_mirror = 0; }); if (consensus) { - routerstatus_t *rs = networkstatus_vote_find_entry(consensus, id_digest); + routerstatus_t *rs = networkstatus_vote_find_mutable_entry( + consensus, id_digest); if (rs && tor_memeq(rs->descriptor_digest, router->cache_info.signed_descriptor_digest, DIGEST_LEN)) { @@ -3610,6 +3656,7 @@ routerlist_remove_old_routers(void) const networkstatus_t *consensus = networkstatus_get_latest_consensus(); const smartlist_t *networkstatus_v2_list = networkstatus_get_v2_list(); int have_enough_v2; + const or_options_t *options = get_options(); trusted_dirs_remove_old_certs(); @@ -3641,8 +3688,7 @@ routerlist_remove_old_routers(void) cutoff = now - OLD_ROUTER_DESC_MAX_AGE; /* Build a list of all the descriptors that _anybody_ lists. */ if (caches && networkstatus_v2_list) { - SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, - { + SMARTLIST_FOREACH_BEGIN(networkstatus_v2_list, networkstatus_v2_t *, ns) { /* XXXX The inner loop here gets pretty expensive, and actually shows up * on some profiles. It may be the reason digestmap_set shows up in * profiles too. If instead we kept a per-descriptor digest count of @@ -3651,10 +3697,11 @@ routerlist_remove_old_routers(void) * improvement, possibly 1-4% if it also removes digestmap_set from the * profile. Not worth it for 0.1.2.x, though. The new directory * system will obsolete this whole thing in 0.2.0.x. */ - SMARTLIST_FOREACH(ns->entries, routerstatus_t *, rs, + SMARTLIST_FOREACH_BEGIN(ns->entries, routerstatus_t *, rs) { if (rs->published_on >= cutoff) - digestset_add(retain, rs->descriptor_digest)); - }); + digestset_add(retain, rs->descriptor_digest); + } SMARTLIST_FOREACH_END(rs); + } SMARTLIST_FOREACH_END(ns); } /* Retain anything listed in the consensus. */ @@ -3671,6 +3718,7 @@ routerlist_remove_old_routers(void) * routers.) */ /* we set this to true iff we don't care about v2 info, or we have enough. */ have_enough_v2 = !caches || + !(authdir_mode_any_main(options) || options->FetchV2Networkstatus) || (networkstatus_v2_list && smartlist_len(networkstatus_v2_list) > get_n_v2_authorities() / 2); @@ -3817,7 +3865,7 @@ router_load_single_router(const char *s, uint8_t purpose, int cache, if (!cache) /* obey the preference of the controller */ ri->cache_info.do_not_cache = 1; - lst = smartlist_create(); + lst = smartlist_new(); smartlist_add(lst, ri); routers_update_status_from_consensus_networkstatus(lst, 0); @@ -3859,7 +3907,7 @@ router_load_routers_from_string(const char *s, const char *eos, int descriptor_digests, const char *prepend_annotations) { - smartlist_t *routers = smartlist_create(), *changed = smartlist_create(); + smartlist_t *routers = smartlist_new(), *changed = smartlist_new(); char fp[HEX_DIGEST_LEN+1]; const char *msg; int from_cache = (saved_location != SAVED_NOWHERE); @@ -3934,7 +3982,7 @@ router_load_extrainfo_from_string(const char *s, const char *eos, smartlist_t *requested_fingerprints, int descriptor_digests) { - smartlist_t *extrainfo_list = smartlist_create(); + smartlist_t *extrainfo_list = smartlist_new(); const char *msg; int from_cache = (saved_location != SAVED_NOWHERE); @@ -3943,7 +3991,7 @@ router_load_extrainfo_from_string(const char *s, const char *eos, log_info(LD_DIR, "%d elements to add", smartlist_len(extrainfo_list)); - SMARTLIST_FOREACH(extrainfo_list, extrainfo_t *, ei, { + SMARTLIST_FOREACH_BEGIN(extrainfo_list, extrainfo_t *, ei) { was_router_added_t added = router_add_extrainfo_to_routerlist(ei, &msg, from_cache, !from_cache); if (WRA_WAS_ADDED(added) && requested_fingerprints) { @@ -3958,7 +4006,7 @@ router_load_extrainfo_from_string(const char *s, const char *eos, * all the extrainfos we want, and we never actually act on them * inside Tor, this should be harmless. */ } - }); + } SMARTLIST_FOREACH_END(ei); routerlist_assert_ok(routerlist); router_rebuild_store(0, &router_get_routerlist()->extrainfo_store); @@ -3971,7 +4019,7 @@ router_load_extrainfo_from_string(const char *s, const char *eos, static int signed_desc_digest_is_recognized(signed_descriptor_t *desc) { - routerstatus_t *rs; + const routerstatus_t *rs; networkstatus_t *consensus = networkstatus_get_latest_consensus(); int caches = directory_caches_dir_info(get_options()); const smartlist_t *networkstatus_v2_list = networkstatus_get_v2_list(); @@ -3995,6 +4043,18 @@ signed_desc_digest_is_recognized(signed_descriptor_t *desc) return 0; } +/** Update downloads for router descriptors and/or microdescriptors as + * appropriate. */ +void +update_all_descriptor_downloads(time_t now) +{ + if (get_options()->DisableNetwork) + return; + update_router_descriptor_downloads(now); + update_microdesc_downloads(now); + launch_dummy_descriptor_download_as_needed(now, get_options()); +} + /** Clear all our timeouts for fetching v2 and v3 directory stuff, and then * give it all a try again. */ void @@ -4002,35 +4062,37 @@ routerlist_retry_directory_downloads(time_t now) { router_reset_status_download_failures(); router_reset_descriptor_download_failures(); + if (get_options()->DisableNetwork) + return; update_networkstatus_downloads(now); - update_router_descriptor_downloads(now); + update_all_descriptor_downloads(now); } -/** Return 1 if all running sufficiently-stable routers will reject +/** Return 1 if all running sufficiently-stable routers we can use will reject * addr:port, return 0 if any might accept it. */ int -router_exit_policy_all_routers_reject(uint32_t addr, uint16_t port, - int need_uptime) -{ +router_exit_policy_all_nodes_reject(const tor_addr_t *addr, uint16_t port, + int need_uptime) +{ /* XXXX MOVE */ addr_policy_result_t r; - if (!routerlist) return 1; - SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, router, - { - if (router->is_running && - !router_is_unreliable(router, need_uptime, 0, 0)) { - r = compare_addr_to_addr_policy(addr, port, router->exit_policy); + SMARTLIST_FOREACH_BEGIN(nodelist_get_list(), const node_t *, node) { + if (node->is_running && + !node_is_unreliable(node, need_uptime, 0, 0)) { + + r = compare_tor_addr_to_node_policy(addr, port, node); + if (r != ADDR_POLICY_REJECTED && r != ADDR_POLICY_PROBABLY_REJECTED) return 0; /* this one could be ok. good enough. */ } - }); + } SMARTLIST_FOREACH_END(node); return 1; /* all will reject. */ } /** Return true iff <b>router</b> does not permit exit streams. */ int -router_exit_policy_rejects_all(routerinfo_t *router) +router_exit_policy_rejects_all(const routerinfo_t *router) { return router->policy_is_reject_star; } @@ -4043,14 +4105,13 @@ trusted_dir_server_t * add_trusted_dir_server(const char *nickname, const char *address, uint16_t dir_port, uint16_t or_port, const char *digest, const char *v3_auth_digest, - authority_type_t type) + dirinfo_type_t type) { trusted_dir_server_t *ent; uint32_t a; char *hostname = NULL; - size_t dlen; if (!trusted_dir_servers) - trusted_dir_servers = smartlist_create(); + trusted_dir_servers = smartlist_new(); if (!address) { /* The address is us; we should guess. */ if (resolve_my_address(LOG_WARN, get_options(), &a, &hostname) < 0) { @@ -4078,16 +4139,14 @@ add_trusted_dir_server(const char *nickname, const char *address, ent->is_running = 1; ent->type = type; memcpy(ent->digest, digest, DIGEST_LEN); - if (v3_auth_digest && (type & V3_AUTHORITY)) + if (v3_auth_digest && (type & V3_DIRINFO)) memcpy(ent->v3_identity_digest, v3_auth_digest, DIGEST_LEN); - dlen = 64 + strlen(hostname) + (nickname?strlen(nickname):0); - ent->description = tor_malloc(dlen); if (nickname) - tor_snprintf(ent->description, dlen, "directory server \"%s\" at %s:%d", + tor_asprintf(&ent->description, "directory server \"%s\" at %s:%d", nickname, hostname, (int)dir_port); else - tor_snprintf(ent->description, dlen, "directory server at %s:%d", + tor_asprintf(&ent->description, "directory server at %s:%d", hostname, (int)dir_port); ent->fake_status.addr = ent->addr; @@ -4118,8 +4177,8 @@ authority_cert_free(authority_cert_t *cert) return; tor_free(cert->cache_info.signed_descriptor_body); - crypto_free_pk_env(cert->signing_key); - crypto_free_pk_env(cert->identity_key); + crypto_pk_free(cert->signing_key); + crypto_pk_free(cert->identity_key); tor_free(cert); } @@ -4146,7 +4205,7 @@ clear_trusted_dir_servers(void) trusted_dir_server_free(ent)); smartlist_clear(trusted_dir_servers); } else { - trusted_dir_servers = smartlist_create(); + trusted_dir_servers = smartlist_new(); } router_dir_info_changed(); } @@ -4157,36 +4216,41 @@ int any_trusted_dir_is_v1_authority(void) { if (trusted_dir_servers) - return get_n_authorities(V1_AUTHORITY) > 0; + return get_n_authorities(V1_DIRINFO) > 0; return 0; } /** For every current directory connection whose purpose is <b>purpose</b>, * and where the resource being downloaded begins with <b>prefix</b>, split - * rest of the resource into base16 fingerprints, decode them, and set the - * corresponding elements of <b>result</b> to a nonzero value. */ + * rest of the resource into base16 fingerprints (or base64 fingerprints if + * purpose==DIR_PURPPOSE_FETCH_MICRODESC), decode them, and set the + * corresponding elements of <b>result</b> to a nonzero value. + */ static void list_pending_downloads(digestmap_t *result, int purpose, const char *prefix) { const size_t p_len = strlen(prefix); - smartlist_t *tmp = smartlist_create(); + smartlist_t *tmp = smartlist_new(); smartlist_t *conns = get_connection_array(); + int flags = DSR_HEX; + if (purpose == DIR_PURPOSE_FETCH_MICRODESC) + flags = DSR_DIGEST256|DSR_BASE64; tor_assert(result); - SMARTLIST_FOREACH(conns, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { if (conn->type == CONN_TYPE_DIR && conn->purpose == purpose && !conn->marked_for_close) { const char *resource = TO_DIR_CONN(conn)->requested_resource; if (!strcmpstart(resource, prefix)) dir_split_resource_into_fingerprints(resource + p_len, - tmp, NULL, DSR_HEX); + tmp, NULL, flags); } - }); + } SMARTLIST_FOREACH_END(conn); + SMARTLIST_FOREACH(tmp, char *, d, { digestmap_set(result, d, (void*)1); @@ -4206,13 +4270,26 @@ list_pending_descriptor_downloads(digestmap_t *result, int extrainfo) list_pending_downloads(result, purpose, "d/"); } -/** Launch downloads for all the descriptors whose digests are listed - * as digests[i] for lo <= i < hi. (Lo and hi may be out of range.) - * If <b>source</b> is given, download from <b>source</b>; otherwise, - * download from an appropriate random directory server. +/** For every microdescriptor we are currently downloading by descriptor + * digest, set result[d] to (void*)1. (Note that microdescriptor digests + * are 256-bit, and digestmap_t only holds 160-bit digests, so we're only + * getting the first 20 bytes of each digest here.) + * + * XXXX Let there be a digestmap256_t, and use that instead. + */ +void +list_pending_microdesc_downloads(digestmap_t *result) +{ + list_pending_downloads(result, DIR_PURPOSE_FETCH_MICRODESC, "d/"); +} + +/** Launch downloads for all the descriptors whose digests or digests256 + * are listed as digests[i] for lo <= i < hi. (Lo and hi may be out of + * range.) If <b>source</b> is given, download from <b>source</b>; + * otherwise, download from an appropriate random directory server. */ static void -initiate_descriptor_downloads(routerstatus_t *source, +initiate_descriptor_downloads(const routerstatus_t *source, int purpose, smartlist_t *digests, int lo, int hi, int pds_flags) @@ -4220,6 +4297,20 @@ initiate_descriptor_downloads(routerstatus_t *source, int i, n = hi-lo; char *resource, *cp; size_t r_len; + + int digest_len = DIGEST_LEN, enc_digest_len = HEX_DIGEST_LEN; + char sep = '+'; + int b64_256 = 0; + + if (purpose == DIR_PURPOSE_FETCH_MICRODESC) { + /* Microdescriptors are downloaded by "-"-separated base64-encoded + * 256-bit digests. */ + digest_len = DIGEST256_LEN; + enc_digest_len = BASE64_DIGEST256_LEN; + sep = '-'; + b64_256 = 1; + } + if (n <= 0) return; if (lo < 0) @@ -4227,15 +4318,19 @@ initiate_descriptor_downloads(routerstatus_t *source, if (hi > smartlist_len(digests)) hi = smartlist_len(digests); - r_len = 8 + (HEX_DIGEST_LEN+1)*n; + r_len = 8 + (enc_digest_len+1)*n; cp = resource = tor_malloc(r_len); memcpy(cp, "d/", 2); cp += 2; for (i = lo; i < hi; ++i) { - base16_encode(cp, r_len-(cp-resource), - smartlist_get(digests,i), DIGEST_LEN); - cp += HEX_DIGEST_LEN; - *cp++ = '+'; + if (b64_256) { + digest256_to_base64(cp, smartlist_get(digests, i)); + } else { + base16_encode(cp, r_len-(cp-resource), + smartlist_get(digests,i), digest_len); + } + cp += enc_digest_len; + *cp++ = sep; } memcpy(cp-1, ".z", 3); @@ -4256,9 +4351,10 @@ initiate_descriptor_downloads(routerstatus_t *source, * running, or otherwise not a descriptor that we would make any * use of even if we had it. Else return 1. */ static INLINE int -client_would_use_router(routerstatus_t *rs, time_t now, or_options_t *options) +client_would_use_router(const routerstatus_t *rs, time_t now, + const or_options_t *options) { - if (!rs->is_running && !options->FetchUselessDescriptors) { + if (!rs->is_flagged_running && !options->FetchUselessDescriptors) { /* If we had this router descriptor, we wouldn't even bother using it. * But, if we want to have a complete list, fetch it anyway. */ return 0; @@ -4282,6 +4378,7 @@ client_would_use_router(routerstatus_t *rs, time_t now, or_options_t *options) * So use 96 because it's a nice number. */ #define MAX_DL_PER_REQUEST 96 +#define MAX_MICRODESC_DL_PER_REQUEST 92 /** Don't split our requests so finely that we are requesting fewer than * this number per server. */ #define MIN_DL_PER_REQUEST 4 @@ -4296,35 +4393,49 @@ client_would_use_router(routerstatus_t *rs, time_t now, or_options_t *options) * them until they have more, or until this amount of time has passed. */ #define MAX_CLIENT_INTERVAL_WITHOUT_REQUEST (10*60) -/** Given a list of router descriptor digests in <b>downloadable</b>, decide - * whether to delay fetching until we have more. If we don't want to delay, - * launch one or more requests to the appropriate directory authorities. */ -static void -launch_router_descriptor_downloads(smartlist_t *downloadable, - routerstatus_t *source, time_t now) +/** Given a <b>purpose</b> (FETCH_MICRODESC or FETCH_SERVERDESC) and a list of + * router descriptor digests or microdescriptor digest256s in + * <b>downloadable</b>, decide whether to delay fetching until we have more. + * If we don't want to delay, launch one or more requests to the appropriate + * directory authorities. + */ +void +launch_descriptor_downloads(int purpose, + smartlist_t *downloadable, + const routerstatus_t *source, time_t now) { int should_delay = 0, n_downloadable; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); + const char *descname; + + tor_assert(purpose == DIR_PURPOSE_FETCH_SERVERDESC || + purpose == DIR_PURPOSE_FETCH_MICRODESC); + + descname = (purpose == DIR_PURPOSE_FETCH_SERVERDESC) ? + "routerdesc" : "microdesc"; n_downloadable = smartlist_len(downloadable); if (!directory_fetches_dir_info_early(options)) { if (n_downloadable >= MAX_DL_TO_DELAY) { log_debug(LD_DIR, - "There are enough downloadable routerdescs to launch requests."); + "There are enough downloadable %ss to launch requests.", + descname); should_delay = 0; } else { - should_delay = (last_routerdesc_download_attempted + + should_delay = (last_descriptor_download_attempted + MAX_CLIENT_INTERVAL_WITHOUT_REQUEST) > now; if (!should_delay && n_downloadable) { - if (last_routerdesc_download_attempted) { + if (last_descriptor_download_attempted) { log_info(LD_DIR, - "There are not many downloadable routerdescs, but we've " + "There are not many downloadable %ss, but we've " "been waiting long enough (%d seconds). Downloading.", - (int)(now-last_routerdesc_download_attempted)); + descname, + (int)(now-last_descriptor_download_attempted)); } else { log_info(LD_DIR, - "There are not many downloadable routerdescs, but we haven't " - "tried downloading descriptors recently. Downloading."); + "There are not many downloadable %ss, but we haven't " + "tried downloading descriptors recently. Downloading.", + descname); } } } @@ -4351,12 +4462,19 @@ launch_router_descriptor_downloads(smartlist_t *downloadable, * update_router_descriptor_downloads() later on, once the connections * have succeeded or failed. */ - pds_flags |= PDS_NO_EXISTING_SERVERDESC_FETCH; + pds_flags |= (purpose == DIR_PURPOSE_FETCH_MICRODESC) ? + PDS_NO_EXISTING_MICRODESC_FETCH : + PDS_NO_EXISTING_SERVERDESC_FETCH; } n_per_request = CEIL_DIV(n_downloadable, MIN_REQUESTS); - if (n_per_request > MAX_DL_PER_REQUEST) - n_per_request = MAX_DL_PER_REQUEST; + if (purpose == DIR_PURPOSE_FETCH_MICRODESC) { + if (n_per_request > MAX_MICRODESC_DL_PER_REQUEST) + n_per_request = MAX_MICRODESC_DL_PER_REQUEST; + } else { + if (n_per_request > MAX_DL_PER_REQUEST) + n_per_request = MAX_DL_PER_REQUEST; + } if (n_per_request < MIN_DL_PER_REQUEST) n_per_request = MIN_DL_PER_REQUEST; @@ -4371,11 +4489,11 @@ launch_router_descriptor_downloads(smartlist_t *downloadable, req_plural, n_downloadable, rtr_plural, n_per_request); smartlist_sort_digests(downloadable); for (i=0; i < n_downloadable; i += n_per_request) { - initiate_descriptor_downloads(source, DIR_PURPOSE_FETCH_SERVERDESC, + initiate_descriptor_downloads(source, purpose, downloadable, i, i+n_per_request, pds_flags); } - last_routerdesc_download_attempted = now; + last_descriptor_download_attempted = now; } } @@ -4391,7 +4509,7 @@ update_router_descriptor_cache_downloads_v2(time_t now) digestmap_t *map; /* Which descs are in progress, or assigned? */ int i, j, n; int n_download; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); const smartlist_t *networkstatus_v2_list = networkstatus_get_v2_list(); if (! directory_fetches_dir_info_early(options)) { @@ -4418,12 +4536,11 @@ update_router_descriptor_cache_downloads_v2(time_t now) * descriptor from the corresponding authority. */ n_download = 0; - SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, - { + SMARTLIST_FOREACH_BEGIN(networkstatus_v2_list, networkstatus_v2_t *, ns) { trusted_dir_server_t *ds; smartlist_t *dl; - dl = downloadable[ns_sl_idx] = smartlist_create(); - download_from[ns_sl_idx] = smartlist_create(); + dl = downloadable[ns_sl_idx] = smartlist_new(); + download_from[ns_sl_idx] = smartlist_new(); if (ns->published_on + MAX_NETWORKSTATUS_AGE+10*60 < now) { /* Don't download if the networkstatus is almost ancient. */ /* Actually, I suspect what's happening here is that we ask @@ -4441,8 +4558,7 @@ update_router_descriptor_cache_downloads_v2(time_t now) if (ds && !ds->is_running) continue; - SMARTLIST_FOREACH(ns->entries, routerstatus_t * , rs, - { + SMARTLIST_FOREACH_BEGIN(ns->entries, routerstatus_t * , rs) { if (!rs->need_to_mirror) continue; if (router_get_by_descriptor_digest(rs->descriptor_digest)) { @@ -4463,8 +4579,8 @@ update_router_descriptor_cache_downloads_v2(time_t now) smartlist_add(dl, rs->descriptor_digest); ++n_download; } - }); - }); + } SMARTLIST_FOREACH_END(rs); + } SMARTLIST_FOREACH_END(ns); /* At random, assign descriptors to authorities such that: * - if d is a member of some downloadable[x], d is a member of some @@ -4533,10 +4649,10 @@ void update_consensus_router_descriptor_downloads(time_t now, int is_vote, networkstatus_t *consensus) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); digestmap_t *map = NULL; - smartlist_t *no_longer_old = smartlist_create(); - smartlist_t *downloadable = smartlist_create(); + smartlist_t *no_longer_old = smartlist_new(); + smartlist_t *downloadable = smartlist_new(); routerstatus_t *source = NULL; int authdir = authdir_mode(options); int n_delayed=0, n_have=0, n_would_reject=0, n_wouldnt_use=0, @@ -4561,15 +4677,14 @@ update_consensus_router_descriptor_downloads(time_t now, int is_vote, map = digestmap_new(); list_pending_descriptor_downloads(map, 0); - SMARTLIST_FOREACH(consensus->routerstatus_list, void *, rsp, - { + SMARTLIST_FOREACH_BEGIN(consensus->routerstatus_list, void *, rsp) { routerstatus_t *rs = is_vote ? &(((vote_routerstatus_t *)rsp)->status) : rsp; signed_descriptor_t *sd; if ((sd = router_get_by_descriptor_digest(rs->descriptor_digest))) { - routerinfo_t *ri; + const routerinfo_t *ri; ++n_have; - if (!(ri = router_get_by_digest(rs->identity_digest)) || + if (!(ri = router_get_by_id_digest(rs->identity_digest)) || tor_memneq(ri->cache_info.signed_descriptor_digest, sd->signed_descriptor_digest, DIGEST_LEN)) { /* We have a descriptor with this digest, but either there is no @@ -4602,7 +4717,8 @@ update_consensus_router_descriptor_downloads(time_t now, int is_vote, if (is_vote && source) { char time_bufnew[ISO_TIME_LEN+1]; char time_bufold[ISO_TIME_LEN+1]; - routerinfo_t *oldrouter = router_get_by_digest(rs->identity_digest); + const routerinfo_t *oldrouter; + oldrouter = router_get_by_id_digest(rs->identity_digest); format_iso_time(time_bufnew, rs->published_on); if (oldrouter) format_iso_time(time_bufold, oldrouter->cache_info.published_on); @@ -4613,7 +4729,7 @@ update_consensus_router_descriptor_downloads(time_t now, int is_vote, source->nickname, oldrouter ? "known" : "unknown"); } smartlist_add(downloadable, rs->descriptor_digest); - }); + } SMARTLIST_FOREACH_END(rsp); if (!authdir_mode_handles_descs(options, ROUTER_PURPOSE_GENERAL) && smartlist_len(no_longer_old)) { @@ -4621,7 +4737,7 @@ update_consensus_router_descriptor_downloads(time_t now, int is_vote, log_info(LD_DIR, "%d router descriptors listed in consensus are " "currently in old_routers; making them current.", smartlist_len(no_longer_old)); - SMARTLIST_FOREACH(no_longer_old, signed_descriptor_t *, sd, { + SMARTLIST_FOREACH_BEGIN(no_longer_old, signed_descriptor_t *, sd) { const char *msg; was_router_added_t r; routerinfo_t *ri = routerlist_reparse_old(rl, sd); @@ -4634,7 +4750,7 @@ update_consensus_router_descriptor_downloads(time_t now, int is_vote, log_warn(LD_DIR, "Couldn't add re-parsed router: %s", msg?msg:"???"); } - }); + } SMARTLIST_FOREACH_END(sd); routerlist_assert_ok(rl); } @@ -4645,7 +4761,8 @@ update_consensus_router_descriptor_downloads(time_t now, int is_vote, smartlist_len(downloadable), n_delayed, n_have, n_in_oldrouters, n_would_reject, n_wouldnt_use, n_inprogress); - launch_router_descriptor_downloads(downloadable, source, now); + launch_descriptor_downloads(DIR_PURPOSE_FETCH_SERVERDESC, + downloadable, source, now); digestmap_free(map, NULL); done: @@ -4655,31 +4772,24 @@ update_consensus_router_descriptor_downloads(time_t now, int is_vote, /** How often should we launch a server/authority request to be sure of getting * a guess for our IP? */ -/*XXXX023 this info should come from netinfo cells or something, or we should +/*XXXX024 this info should come from netinfo cells or something, or we should * do this only when we aren't seeing incoming data. see bug 652. */ #define DUMMY_DOWNLOAD_INTERVAL (20*60) -/** Launch downloads for router status as needed. */ -void -update_router_descriptor_downloads(time_t now) +/** As needed, launch a dummy router descriptor fetch to see if our + * address has changed. */ +static void +launch_dummy_descriptor_download_as_needed(time_t now, + const or_options_t *options) { - or_options_t *options = get_options(); static time_t last_dummy_download = 0; - if (should_delay_dir_fetches(options)) - return; - if (directory_fetches_dir_info_early(options)) { - update_router_descriptor_cache_downloads_v2(now); - } - update_consensus_router_descriptor_downloads(now, 0, - networkstatus_get_reasonably_live_consensus(now)); - - /* XXXX023 we could be smarter here; see notes on bug 652. */ + /* XXXX024 we could be smarter here; see notes on bug 652. */ /* If we're a server that doesn't have a configured address, we rely on * directory fetches to learn when our address changes. So if we haven't * tried to get any routerdescs in a long time, try a dummy fetch now. */ if (!options->Address && server_mode(options) && - last_routerdesc_download_attempted + DUMMY_DOWNLOAD_INTERVAL < now && + last_descriptor_download_attempted + DUMMY_DOWNLOAD_INTERVAL < now && last_dummy_download + DUMMY_DOWNLOAD_INTERVAL < now) { last_dummy_download = now; directory_get_from_dirserver(DIR_PURPOSE_FETCH_SERVERDESC, @@ -4688,11 +4798,28 @@ update_router_descriptor_downloads(time_t now) } } +/** Launch downloads for router status as needed. */ +void +update_router_descriptor_downloads(time_t now) +{ + const or_options_t *options = get_options(); + if (should_delay_dir_fetches(options)) + return; + if (!we_fetch_router_descriptors(options)) + return; + if (directory_fetches_dir_info_early(options)) { + update_router_descriptor_cache_downloads_v2(now); + } + + update_consensus_router_descriptor_downloads(now, 0, + networkstatus_get_reasonably_live_consensus(now, FLAV_NS)); +} + /** Launch extrainfo downloads as needed. */ void update_extrainfo_downloads(time_t now) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); routerlist_t *rl; smartlist_t *wanted; digestmap_t *pending; @@ -4708,7 +4835,7 @@ update_extrainfo_downloads(time_t now) pending = digestmap_new(); list_pending_descriptor_downloads(pending, 1); rl = router_get_routerlist(); - wanted = smartlist_create(); + wanted = smartlist_new(); for (old_routers = 0; old_routers < 2; ++old_routers) { smartlist_t *lst = old_routers ? rl->old_routers : rl->routers; for (i = 0; i < smartlist_len(lst); ++i) { @@ -4720,7 +4847,7 @@ update_extrainfo_downloads(time_t now) sd = &((routerinfo_t*)smartlist_get(lst, i))->cache_info; if (sd->is_extrainfo) continue; /* This should never happen. */ - if (old_routers && !router_get_by_digest(sd->identity_digest)) + if (old_routers && !router_get_by_id_digest(sd->identity_digest)) continue; /* Couldn't check the signature if we got it. */ if (sd->extrainfo_is_bogus) continue; @@ -4815,25 +4942,33 @@ get_dir_info_status_string(void) static void count_usable_descriptors(int *num_present, int *num_usable, const networkstatus_t *consensus, - or_options_t *options, time_t now, + const or_options_t *options, time_t now, routerset_t *in_set, int exit_only) { + const int md = (consensus->flavor == FLAV_MICRODESC); *num_present = 0, *num_usable=0; - SMARTLIST_FOREACH(consensus->routerstatus_list, routerstatus_t *, rs, - { + SMARTLIST_FOREACH_BEGIN(consensus->routerstatus_list, routerstatus_t *, rs) + { if (exit_only && ! rs->is_exit) continue; - if (in_set && ! routerset_contains_routerstatus(in_set, rs)) + if (in_set && ! routerset_contains_routerstatus(in_set, rs, -1)) continue; if (client_would_use_router(rs, now, options)) { + const char * const digest = rs->descriptor_digest; + int present; ++*num_usable; /* the consensus says we want it. */ - if (router_get_by_descriptor_digest(rs->descriptor_digest)) { + if (md) + present = NULL != microdesc_cache_lookup_by_digest256(NULL, digest); + else + present = NULL != router_get_by_descriptor_digest(digest); + if (present) { /* we have the descriptor listed in the consensus. */ ++*num_present; } } - }); + } + SMARTLIST_FOREACH_END(rs); log_debug(LD_DIR, "%d usable, %d present.", *num_usable, *num_present); } @@ -4847,7 +4982,7 @@ count_loading_descriptors_progress(void) int num_present = 0, num_usable=0; time_t now = time(NULL); const networkstatus_t *consensus = - networkstatus_get_reasonably_live_consensus(now); + networkstatus_get_reasonably_live_consensus(now,usable_consensus_flavor()); double fraction; if (!consensus) @@ -4876,16 +5011,17 @@ update_router_have_minimum_dir_info(void) int num_exit_present = 0, num_exit_usable = 0; time_t now = time(NULL); int res; - or_options_t *options = get_options(); + const or_options_t *options = get_options(); const networkstatus_t *consensus = - networkstatus_get_reasonably_live_consensus(now); + networkstatus_get_reasonably_live_consensus(now,usable_consensus_flavor()); + int using_md; if (!consensus) { if (!networkstatus_get_latest_consensus()) - strlcpy(dir_info_status, "We have no network-status consensus.", + strlcpy(dir_info_status, "We have no usable consensus.", sizeof(dir_info_status)); else - strlcpy(dir_info_status, "We have no recent network-status consensus.", + strlcpy(dir_info_status, "We have no recent usable consensus.", sizeof(dir_info_status)); res = 0; goto done; @@ -4899,24 +5035,34 @@ update_router_have_minimum_dir_info(void) goto done; } + using_md = consensus->flavor == FLAV_MICRODESC; + count_usable_descriptors(&num_present, &num_usable, consensus, options, now, NULL, 0); count_usable_descriptors(&num_exit_present, &num_exit_usable, consensus, options, now, options->ExitNodes, 1); - if (num_present < num_usable/4) { +/* What fraction of desired server descriptors do we need before we will + * build circuits? */ +#define FRAC_USABLE_NEEDED .75 +/* What fraction of desired _exit_ server descriptors do we need before we + * will build circuits? */ +#define FRAC_EXIT_USABLE_NEEDED .5 + + if (num_present < num_usable * FRAC_USABLE_NEEDED) { tor_snprintf(dir_info_status, sizeof(dir_info_status), - "We have only %d/%d usable descriptors.", num_present, num_usable); + "We have only %d/%d usable %sdescriptors.", + num_present, num_usable, using_md ? "micro" : ""); res = 0; control_event_bootstrap(BOOTSTRAP_STATUS_REQUESTING_DESCRIPTORS, 0); goto done; } else if (num_present < 2) { tor_snprintf(dir_info_status, sizeof(dir_info_status), - "Only %d descriptor%s here and believed reachable!", - num_present, num_present ? "" : "s"); + "Only %d %sdescriptor%s here and believed reachable!", + num_present, using_md ? "micro" : "", num_present ? "" : "s"); res = 0; goto done; - } else if (num_exit_present < num_exit_usable / 3) { + } else if (num_exit_present < num_exit_usable * FRAC_EXIT_USABLE_NEEDED) { tor_snprintf(dir_info_status, sizeof(dir_info_status), "We have only %d/%d usable exit node descriptors.", num_exit_present, num_exit_usable); @@ -4932,8 +5078,8 @@ update_router_have_minimum_dir_info(void) if (!num_usable || !num_present) { tor_snprintf(dir_info_status, sizeof(dir_info_status), - "We have only %d/%d usable entry node descriptors.", - num_present, num_usable); + "We have only %d/%d usable entry node %sdescriptors.", + num_present, num_usable, using_md?"micro":""); res = 0; goto done; } @@ -4973,7 +5119,7 @@ void router_reset_descriptor_download_failures(void) { networkstatus_reset_download_failures(); - last_routerdesc_download_attempted = 0; + last_descriptor_download_attempted = 0; if (!routerlist) return; SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, ri, @@ -4988,7 +5134,7 @@ router_reset_descriptor_download_failures(void) /** Any changes in a router descriptor's publication time larger than this are * automatically non-cosmetic. */ -#define ROUTER_MAX_COSMETIC_TIME_DIFFERENCE (12*60*60) +#define ROUTER_MAX_COSMETIC_TIME_DIFFERENCE (2*60*60) /** We allow uptime to vary from how much it ought to be by this much. */ #define ROUTER_ALLOW_UPTIME_DRIFT (6*60*60) @@ -4997,7 +5143,7 @@ router_reset_descriptor_download_failures(void) * would not cause a recent (post 0.1.1.6) dirserver to republish. */ int -router_differences_are_cosmetic(routerinfo_t *r1, routerinfo_t *r2) +router_differences_are_cosmetic(const routerinfo_t *r1, const routerinfo_t *r2) { time_t r1pub, r2pub; long time_difference; @@ -5005,7 +5151,7 @@ router_differences_are_cosmetic(routerinfo_t *r1, routerinfo_t *r2) /* r1 should be the one that was published first. */ if (r1->cache_info.published_on > r2->cache_info.published_on) { - routerinfo_t *ri_tmp = r2; + const routerinfo_t *ri_tmp = r2; r2 = r1; r1 = ri_tmp; } @@ -5014,6 +5160,8 @@ router_differences_are_cosmetic(routerinfo_t *r1, routerinfo_t *r2) if (strcasecmp(r1->address, r2->address) || strcasecmp(r1->nickname, r2->nickname) || r1->or_port != r2->or_port || + !tor_addr_eq(&r1->ipv6_addr, &r2->ipv6_addr) || + r1->ipv6_orport != r2->ipv6_orport || r1->dir_port != r2->dir_port || r1->purpose != r2->purpose || crypto_pk_cmp_keys(r1->onion_pkey, r2->onion_pkey) || @@ -5024,7 +5172,6 @@ router_differences_are_cosmetic(routerinfo_t *r1, routerinfo_t *r2) (r1->contact_info && r2->contact_info && strcasecmp(r1->contact_info, r2->contact_info)) || r1->is_hibernating != r2->is_hibernating || - r1->has_old_dnsworkers != r2->has_old_dnsworkers || cmp_addr_policies(r1->exit_policy, r2->exit_policy)) return 0; if ((r1->declared_family == NULL) != (r2->declared_family == NULL)) @@ -5079,7 +5226,8 @@ router_differences_are_cosmetic(routerinfo_t *r1, routerinfo_t *r2) * incompatibility (if any). **/ int -routerinfo_incompatible_with_extrainfo(routerinfo_t *ri, extrainfo_t *ei, +routerinfo_incompatible_with_extrainfo(const routerinfo_t *ri, + extrainfo_t *ei, signed_descriptor_t *sd, const char **msg) { @@ -5087,7 +5235,7 @@ routerinfo_incompatible_with_extrainfo(routerinfo_t *ri, extrainfo_t *ei, tor_assert(ri); tor_assert(ei); if (!sd) - sd = &ri->cache_info; + sd = (signed_descriptor_t*)&ri->cache_info; if (ei->bad_sig) { if (msg) *msg = "Extrainfo signature was bad, or signed with wrong key."; @@ -5152,14 +5300,13 @@ routerinfo_incompatible_with_extrainfo(routerinfo_t *ri, extrainfo_t *ei, /** Assert that the internal representation of <b>rl</b> is * self-consistent. */ void -routerlist_assert_ok(routerlist_t *rl) +routerlist_assert_ok(const routerlist_t *rl) { routerinfo_t *r2; signed_descriptor_t *sd2; if (!rl) return; - SMARTLIST_FOREACH(rl->routers, routerinfo_t *, r, - { + SMARTLIST_FOREACH_BEGIN(rl->routers, routerinfo_t *, r) { r2 = rimap_get(rl->identity_map, r->cache_info.identity_digest); tor_assert(r == r2); sd2 = sdmap_get(rl->desc_digest_map, @@ -5189,9 +5336,8 @@ routerlist_assert_ok(routerlist_t *rl) tor_assert(sd3 == &(r->cache_info)); } */ - }); - SMARTLIST_FOREACH(rl->old_routers, signed_descriptor_t *, sd, - { + } SMARTLIST_FOREACH_END(r); + SMARTLIST_FOREACH_BEGIN(rl->old_routers, signed_descriptor_t *, sd) { r2 = rimap_get(rl->identity_map, sd->identity_digest); tor_assert(sd != &(r2->cache_info)); sd2 = sdmap_get(rl->desc_digest_map, sd->signed_descriptor_digest); @@ -5204,7 +5350,7 @@ routerlist_assert_ok(routerlist_t *rl) tor_assert(sd3 == sd); } */ - }); + } SMARTLIST_FOREACH_END(sd); RIMAP_FOREACH(rl->identity_map, d, r) { tor_assert(tor_memeq(r->cache_info.identity_digest, d, DIGEST_LEN)); @@ -5242,11 +5388,10 @@ routerlist_assert_ok(routerlist_t *rl) * If <b>router</b> is NULL, it just frees its internal memory and returns. */ const char * -esc_router_info(routerinfo_t *router) +esc_router_info(const routerinfo_t *router) { static char *info=NULL; char *esc_contact, *esc_platform; - size_t len; tor_free(info); if (!router) @@ -5255,10 +5400,7 @@ esc_router_info(routerinfo_t *router) esc_contact = esc_for_log(router->contact_info); esc_platform = esc_for_log(router->platform); - len = strlen(esc_contact)+strlen(esc_platform)+32; - info = tor_malloc(len); - tor_snprintf(info, len, "Contact %s, Platform %s", esc_contact, - esc_platform); + tor_asprintf(&info, "Contact %s, Platform %s", esc_contact, esc_platform); tor_free(esc_contact); tor_free(esc_platform); @@ -5321,11 +5463,11 @@ routerset_t * routerset_new(void) { routerset_t *result = tor_malloc_zero(sizeof(routerset_t)); - result->list = smartlist_create(); + result->list = smartlist_new(); result->names = strmap_new(); result->digests = digestmap_new(); - result->policies = smartlist_create(); - result->country_names = smartlist_create(); + result->policies = smartlist_new(); + result->country_names = smartlist_new(); return result; } @@ -5344,38 +5486,6 @@ routerset_get_countryname(const char *c) return country; } -#if 0 -/** Add the GeoIP database's integer index (+1) of a valid two-character - * country code to the routerset's <b>countries</b> bitarray. Return the - * integer index if the country code is valid, -1 otherwise.*/ -static int -routerset_add_country(const char *c) -{ - char country[3]; - country_t cc; - - /* XXXX: Country codes must be of the form \{[a-z\?]{2}\} but this accepts - \{[.]{2}\}. Do we need to be strict? -RH */ - /* Nope; if the country code is bad, we'll get 0 when we look it up. */ - - if (!geoip_is_loaded()) { - log(LOG_WARN, LD_CONFIG, "GeoIP database not loaded: Cannot add country" - "entry %s, ignoring.", c); - return -1; - } - - memcpy(country, c+1, 2); - country[2] = '\0'; - tor_strlower(country); - - if ((cc=geoip_get_country(country))==-1) { - log(LOG_WARN, LD_CONFIG, "Country code '%s' is not valid, ignoring.", - country); - } - return cc; -} -#endif - /** Update the routerset's <b>countries</b> bitarray_t. Called whenever * the GeoIP database is reloaded. */ @@ -5418,7 +5528,7 @@ routerset_parse(routerset_t *target, const char *s, const char *description) int r = 0; int added_countries = 0; char *countryname; - smartlist_t *list = smartlist_create(); + smartlist_t *list = smartlist_new(); smartlist_split_string(list, s, ",", SPLIT_SKIP_SPACE | SPLIT_IGNORE_BLANK, 0); SMARTLIST_FOREACH_BEGIN(list, char *, nick) { @@ -5464,7 +5574,7 @@ routerset_parse(routerset_t *target, const char *s, const char *description) void refresh_all_country_info(void) { - or_options_t *options = get_options(); + const or_options_t *options = get_options(); if (options->EntryNodes) routerset_refresh_countries(options->EntryNodes); @@ -5477,7 +5587,7 @@ refresh_all_country_info(void) if (options->_ExcludeExitNodesUnion) routerset_refresh_countries(options->_ExcludeExitNodesUnion); - routerlist_refresh_countries(); + nodelist_refresh_countries(); } /** Add all members of the set <b>source</b> to <b>target</b>. */ @@ -5527,11 +5637,11 @@ routerset_is_empty(const routerset_t *set) static int routerset_contains(const routerset_t *set, const tor_addr_t *addr, uint16_t orport, - const char *nickname, const char *id_digest, int is_named, + const char *nickname, const char *id_digest, country_t country) { - if (!set || !set->list) return 0; - (void) is_named; /* not supported */ + if (!set || !set->list) + return 0; if (nickname && strmap_get_lc(set->names, nickname)) return 4; if (id_digest && digestmap_get(set->digests, id_digest)) @@ -5559,13 +5669,14 @@ routerset_contains_extendinfo(const routerset_t *set, const extend_info_t *ei) ei->port, ei->nickname, ei->identity_digest, - -1, /*is_named*/ -1 /*country*/); } -/** Return true iff <b>ri</b> is in <b>set</b>. */ +/** Return true iff <b>ri</b> is in <b>set</b>. If country is <b>-1</b>, we + * look up the country. */ int -routerset_contains_router(const routerset_t *set, routerinfo_t *ri) +routerset_contains_router(const routerset_t *set, const routerinfo_t *ri, + country_t country) { tor_addr_t addr; tor_addr_from_ipv4h(&addr, ri->addr); @@ -5574,13 +5685,15 @@ routerset_contains_router(const routerset_t *set, routerinfo_t *ri) ri->or_port, ri->nickname, ri->cache_info.identity_digest, - ri->is_named, - ri->country); + country); } -/** Return true iff <b>rs</b> is in <b>set</b>. */ +/** Return true iff <b>rs</b> is in <b>set</b>. If country is <b>-1</b>, we + * look up the country. */ int -routerset_contains_routerstatus(const routerset_t *set, routerstatus_t *rs) +routerset_contains_routerstatus(const routerset_t *set, + const routerstatus_t *rs, + country_t country) { tor_addr_t addr; tor_addr_from_ipv4h(&addr, rs->addr); @@ -5589,50 +5702,59 @@ routerset_contains_routerstatus(const routerset_t *set, routerstatus_t *rs) rs->or_port, rs->nickname, rs->identity_digest, - rs->is_named, - -1); + country); } -/** Add every known routerinfo_t that is a member of <b>routerset</b> to +/** Return true iff <b>node</b> is in <b>set</b>. */ +int +routerset_contains_node(const routerset_t *set, const node_t *node) +{ + if (node->rs) + return routerset_contains_routerstatus(set, node->rs, node->country); + else if (node->ri) + return routerset_contains_router(set, node->ri, node->country); + else + return 0; +} + +/** Add every known node_t that is a member of <b>routerset</b> to * <b>out</b>, but never add any that are part of <b>excludeset</b>. * If <b>running_only</b>, only add the running ones. */ void -routerset_get_all_routers(smartlist_t *out, const routerset_t *routerset, - const routerset_t *excludeset, int running_only) -{ +routerset_get_all_nodes(smartlist_t *out, const routerset_t *routerset, + const routerset_t *excludeset, int running_only) +{ /* XXXX MOVE */ tor_assert(out); if (!routerset || !routerset->list) return; - if (!warned_nicknames) - warned_nicknames = smartlist_create(); - if (routerset_is_list(routerset)) { + if (routerset_is_list(routerset)) { /* No routers are specified by type; all are given by name or digest. * we can do a lookup in O(len(routerset)). */ SMARTLIST_FOREACH(routerset->list, const char *, name, { - routerinfo_t *router = router_get_by_nickname(name, 1); - if (router) { - if (!running_only || router->is_running) - if (!routerset_contains_router(excludeset, router)) - smartlist_add(out, router); + const node_t *node = node_get_by_nickname(name, 1); + if (node) { + if (!running_only || node->is_running) + if (!routerset_contains_node(excludeset, node)) + smartlist_add(out, (void*)node); } }); } else { /* We need to iterate over the routerlist to get all the ones of the * right kind. */ - routerlist_t *rl = router_get_routerlist(); - SMARTLIST_FOREACH(rl->routers, routerinfo_t *, router, { - if (running_only && !router->is_running) + smartlist_t *nodes = nodelist_get_list(); + SMARTLIST_FOREACH(nodes, const node_t *, node, { + if (running_only && !node->is_running) continue; - if (routerset_contains_router(routerset, router) && - !routerset_contains_router(excludeset, router)) - smartlist_add(out, router); + if (routerset_contains_node(routerset, node) && + !routerset_contains_node(excludeset, node)) + smartlist_add(out, (void*)node); }); } } #if 0 -/** Add to <b>target</b> every routerinfo_t from <b>source</b> except: +/** Add to <b>target</b> every node_t from <b>source</b> except: * * 1) Don't add it if <b>include</b> is non-empty and the relay isn't in * <b>include</b>; and @@ -5641,40 +5763,40 @@ routerset_get_all_routers(smartlist_t *out, const routerset_t *routerset, * 3) If <b>running_only</b>, don't add non-running routers. */ void -routersets_get_disjunction(smartlist_t *target, +routersets_get_node_disjunction(smartlist_t *target, const smartlist_t *source, const routerset_t *include, const routerset_t *exclude, int running_only) { - SMARTLIST_FOREACH(source, routerinfo_t *, router, { + SMARTLIST_FOREACH(source, const node_t *, node, { int include_result; - if (running_only && !router->is_running) + if (running_only && !node->is_running) continue; if (!routerset_is_empty(include)) - include_result = routerset_contains_router(include, router); + include_result = routerset_contains_node(include, node); else include_result = 1; if (include_result) { - int exclude_result = routerset_contains_router(exclude, router); + int exclude_result = routerset_contains_node(exclude, node); if (include_result >= exclude_result) - smartlist_add(target, router); + smartlist_add(target, (void*)node); } }); } #endif -/** Remove every routerinfo_t from <b>lst</b> that is in <b>routerset</b>. */ +/** Remove every node_t from <b>lst</b> that is in <b>routerset</b>. */ void -routerset_subtract_routers(smartlist_t *lst, const routerset_t *routerset) -{ +routerset_subtract_nodes(smartlist_t *lst, const routerset_t *routerset) +{ /*XXXX MOVE ? */ tor_assert(lst); if (!routerset) return; - SMARTLIST_FOREACH(lst, routerinfo_t *, r, { - if (routerset_contains_router(routerset, r)) { + SMARTLIST_FOREACH(lst, const node_t *, node, { + if (routerset_contains_node(routerset, node)) { //log_debug(LD_DIR, "Subtracting %s",r->nickname); - SMARTLIST_DEL_CURRENT(lst, r); + SMARTLIST_DEL_CURRENT(lst, node); } }); } @@ -5740,18 +5862,23 @@ routerset_free(routerset_t *routerset) /** Refresh the country code of <b>ri</b>. This function MUST be called on * each router when the GeoIP database is reloaded, and on all new routers. */ void -routerinfo_set_country(routerinfo_t *ri) +node_set_country(node_t *node) { - ri->country = geoip_get_country_by_ip(ri->addr); + if (node->rs) + node->country = geoip_get_country_by_ip(node->rs->addr); + else if (node->ri) + node->country = geoip_get_country_by_ip(node->ri->addr); + else + node->country = -1; } /** Set the country code of all routers in the routerlist. */ void -routerlist_refresh_countries(void) +nodelist_refresh_countries(void) /* MOVE */ { - routerlist_t *rl = router_get_routerlist(); - SMARTLIST_FOREACH(rl->routers, routerinfo_t *, ri, - routerinfo_set_country(ri)); + smartlist_t *nodes = nodelist_get_list(); + SMARTLIST_FOREACH(nodes, node_t *, node, + node_set_country(node)); } /** Determine the routers that are responsible for <b>id</b> (binary) and @@ -5794,7 +5921,7 @@ hid_serv_get_responsible_directories(smartlist_t *responsible_dirs, int hid_serv_acting_as_directory(void) { - routerinfo_t *me = router_get_my_routerinfo(); + const routerinfo_t *me = router_get_my_routerinfo(); if (!me) return 0; if (!get_options()->HidServDirectoryV2) { @@ -5810,7 +5937,7 @@ hid_serv_acting_as_directory(void) int hid_serv_responsible_for_desc_id(const char *query) { - routerinfo_t *me; + const routerinfo_t *me; routerstatus_t *last_rs; const char *my_id, *last_id; int result; @@ -5820,7 +5947,7 @@ hid_serv_responsible_for_desc_id(const char *query) if (!(me = router_get_my_routerinfo())) return 0; /* This is redundant, but let's be paranoid. */ my_id = me->cache_info.identity_digest; - responsible = smartlist_create(); + responsible = smartlist_new(); if (hid_serv_get_responsible_directories(responsible, query) < 0) { smartlist_free(responsible); return 0; diff --git a/src/or/routerlist.h b/src/or/routerlist.h index fec18705b3..8dcc6eb026 100644 --- a/src/or/routerlist.h +++ b/src/or/routerlist.h @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -11,7 +11,7 @@ #ifndef _TOR_ROUTERLIST_H #define _TOR_ROUTERLIST_H -int get_n_authorities(authority_type_t type); +int get_n_authorities(dirinfo_type_t type); int trusted_dirs_reload_certs(void); int trusted_dirs_load_certs_from_string(const char *contents, int from_store, int flush); @@ -27,53 +27,50 @@ int router_reload_router_list(void); int authority_cert_dl_looks_uncertain(const char *id_digest); smartlist_t *router_get_trusted_dir_servers(void); -routerstatus_t *router_pick_directory_server(authority_type_t type, int flags); +const routerstatus_t *router_pick_directory_server(dirinfo_type_t type, + int flags); trusted_dir_server_t *router_get_trusteddirserver_by_digest(const char *d); trusted_dir_server_t *trusteddirserver_get_by_v3_auth_digest(const char *d); -routerstatus_t *router_pick_trusteddirserver(authority_type_t type, int flags); +const routerstatus_t *router_pick_trusteddirserver(dirinfo_type_t type, + int flags); int router_get_my_share_of_directory_requests(double *v2_share_out, double *v3_share_out); void router_reset_status_download_failures(void); -void routerlist_add_family(smartlist_t *sl, routerinfo_t *router); -int routers_in_same_family(routerinfo_t *r1, routerinfo_t *r2); int routers_have_same_or_addr(const routerinfo_t *r1, const routerinfo_t *r2); -void add_nickname_list_to_smartlist(smartlist_t *sl, const char *list, - int must_be_running); -int router_nickname_is_in_list(routerinfo_t *router, const char *list); -routerinfo_t *routerlist_find_my_routerinfo(void); -routerinfo_t *router_find_exact_exit_enclave(const char *address, +int router_nickname_is_in_list(const routerinfo_t *router, const char *list); +const routerinfo_t *routerlist_find_my_routerinfo(void); +const node_t *router_find_exact_exit_enclave(const char *address, uint16_t port); -int router_is_unreliable(routerinfo_t *router, int need_uptime, +int node_is_unreliable(const node_t *router, int need_uptime, int need_capacity, int need_guard); -uint32_t router_get_advertised_bandwidth(routerinfo_t *router); -uint32_t router_get_advertised_bandwidth_capped(routerinfo_t *router); +uint32_t router_get_advertised_bandwidth(const routerinfo_t *router); +uint32_t router_get_advertised_bandwidth_capped(const routerinfo_t *router); -routerinfo_t *routerlist_sl_choose_by_bandwidth(smartlist_t *sl, - bandwidth_weight_rule_t rule); -routerstatus_t *routerstatus_sl_choose_by_bandwidth(smartlist_t *sl, - bandwidth_weight_rule_t rule); +const node_t *node_sl_choose_by_bandwidth(smartlist_t *sl, + bandwidth_weight_rule_t rule); -routerinfo_t *router_choose_random_node(smartlist_t *excludedsmartlist, +const node_t *router_choose_random_node(smartlist_t *excludedsmartlist, struct routerset_t *excludedset, router_crn_flags_t flags); -routerinfo_t *router_get_by_nickname(const char *nickname, +const routerinfo_t *router_get_by_nickname(const char *nickname, int warn_if_unnamed); -int router_digest_version_as_new_as(const char *digest, const char *cutoff); +int router_is_named(const routerinfo_t *router); int router_digest_is_trusted_dir_type(const char *digest, - authority_type_t type); + dirinfo_type_t type); #define router_digest_is_trusted_dir(d) \ - router_digest_is_trusted_dir_type((d), NO_AUTHORITY) + router_digest_is_trusted_dir_type((d), NO_DIRINFO) int router_addr_is_trusted_dir(uint32_t addr); int hexdigest_to_digest(const char *hexdigest, char *digest); -routerinfo_t *router_get_by_hexdigest(const char *hexdigest); -routerinfo_t *router_get_by_digest(const char *digest); +const routerinfo_t *router_get_by_hexdigest(const char *hexdigest); +const routerinfo_t *router_get_by_id_digest(const char *digest); +routerinfo_t *router_get_mutable_by_digest(const char *digest); signed_descriptor_t *router_get_by_descriptor_digest(const char *digest); signed_descriptor_t *router_get_by_extrainfo_digest(const char *digest); signed_descriptor_t *extrainfo_get_by_descriptor_digest(const char *digest); -const char *signed_descriptor_get_body(signed_descriptor_t *desc); -const char *signed_descriptor_get_annotations(signed_descriptor_t *desc); +const char *signed_descriptor_get_body(const signed_descriptor_t *desc); +const char *signed_descriptor_get_annotations(const signed_descriptor_t *desc); routerlist_t *router_get_routerlist(void); void routerinfo_free(routerinfo_t *router); void extrainfo_free(extrainfo_t *extrainfo); @@ -88,14 +85,16 @@ void router_set_status(const char *digest, int up); static int WRA_WAS_ADDED(was_router_added_t s); static int WRA_WAS_OUTDATED(was_router_added_t s); static int WRA_WAS_REJECTED(was_router_added_t s); -/** Return true iff the descriptor was added. It might still be necessary to - * check whether the descriptor generator should be notified. +/** Return true iff the outcome code in <b>s</b> indicates that the descriptor + * was added. It might still be necessary to check whether the descriptor + * generator should be notified. */ static INLINE int WRA_WAS_ADDED(was_router_added_t s) { return s == ROUTER_ADDED_SUCCESSFULLY || s == ROUTER_ADDED_NOTIFY_GENERATOR; } -/** Return true iff the descriptor was not added because it was either: +/** Return true iff the outcome code in <b>s</b> indicates that the descriptor + * was not added because it was either: * - not in the consensus * - neither in the consensus nor in any networkstatus document * - it was outdated. @@ -106,6 +105,8 @@ static INLINE int WRA_WAS_OUTDATED(was_router_added_t s) s == ROUTER_NOT_IN_CONSENSUS || s == ROUTER_NOT_IN_CONSENSUS_OR_NETWORKSTATUS); } +/** Return true iff the outcome code in <b>s</b> indicates that the descriptor + * was flat-out rejected. */ static INLINE int WRA_WAS_REJECTED(was_router_added_t s) { return (s == ROUTER_AUTHDIR_REJECTS); @@ -132,63 +133,71 @@ void router_load_extrainfo_from_string(const char *s, const char *eos, int descriptor_digests); void routerlist_retry_directory_downloads(time_t now); -int router_exit_policy_all_routers_reject(uint32_t addr, uint16_t port, - int need_uptime); -int router_exit_policy_rejects_all(routerinfo_t *router); +int router_exit_policy_all_nodes_reject(const tor_addr_t *addr, uint16_t port, + int need_uptime); + +int router_exit_policy_rejects_all(const routerinfo_t *router); trusted_dir_server_t *add_trusted_dir_server(const char *nickname, const char *address, uint16_t dir_port, uint16_t or_port, const char *digest, const char *v3_auth_digest, - authority_type_t type); + dirinfo_type_t type); void authority_cert_free(authority_cert_t *cert); void clear_trusted_dir_servers(void); int any_trusted_dir_is_v1_authority(void); void update_consensus_router_descriptor_downloads(time_t now, int is_vote, networkstatus_t *consensus); void update_router_descriptor_downloads(time_t now); +void update_all_descriptor_downloads(time_t now); void update_extrainfo_downloads(time_t now); int router_have_minimum_dir_info(void); void router_dir_info_changed(void); const char *get_dir_info_status_string(void); int count_loading_descriptors_progress(void); void router_reset_descriptor_download_failures(void); -int router_differences_are_cosmetic(routerinfo_t *r1, routerinfo_t *r2); -int routerinfo_incompatible_with_extrainfo(routerinfo_t *ri, extrainfo_t *ei, +int router_differences_are_cosmetic(const routerinfo_t *r1, + const routerinfo_t *r2); +int routerinfo_incompatible_with_extrainfo(const routerinfo_t *ri, + extrainfo_t *ei, signed_descriptor_t *sd, const char **msg); -void routerlist_assert_ok(routerlist_t *rl); -const char *esc_router_info(routerinfo_t *router); +void routerlist_assert_ok(const routerlist_t *rl); +const char *esc_router_info(const routerinfo_t *router); void routers_sort_by_identity(smartlist_t *routers); routerset_t *routerset_new(void); +void routerset_refresh_countries(routerset_t *rs); int routerset_parse(routerset_t *target, const char *s, const char *description); void routerset_union(routerset_t *target, const routerset_t *source); int routerset_is_list(const routerset_t *set); int routerset_needs_geoip(const routerset_t *set); int routerset_is_empty(const routerset_t *set); -int routerset_contains_router(const routerset_t *set, routerinfo_t *ri); +int routerset_contains_router(const routerset_t *set, const routerinfo_t *ri, + country_t country); int routerset_contains_routerstatus(const routerset_t *set, - routerstatus_t *rs); + const routerstatus_t *rs, + country_t country); int routerset_contains_extendinfo(const routerset_t *set, const extend_info_t *ei); -void routerset_get_all_routers(smartlist_t *out, const routerset_t *routerset, - const routerset_t *excludeset, - int running_only); + +int routerset_contains_node(const routerset_t *set, const node_t *node); +void routerset_get_all_nodes(smartlist_t *out, const routerset_t *routerset, + const routerset_t *excludeset, + int running_only); #if 0 -void routersets_get_disjunction(smartlist_t *target, const smartlist_t *source, +void routersets_get_node_disjunction(smartlist_t *target, + const smartlist_t *source, const routerset_t *include, const routerset_t *exclude, int running_only); #endif -void routerset_subtract_routers(smartlist_t *out, +void routerset_subtract_nodes(smartlist_t *out, const routerset_t *routerset); + char *routerset_to_string(const routerset_t *routerset); -void routerset_refresh_countries(routerset_t *target); int routerset_equal(const routerset_t *old, const routerset_t *new); void routerset_free(routerset_t *routerset); -void routerinfo_set_country(routerinfo_t *ri); -void routerlist_refresh_countries(void); void refresh_all_country_info(void); int hid_serv_get_responsible_directories(smartlist_t *responsible_dirs, @@ -196,5 +205,16 @@ int hid_serv_get_responsible_directories(smartlist_t *responsible_dirs, int hid_serv_acting_as_directory(void); int hid_serv_responsible_for_desc_id(const char *id); +void list_pending_microdesc_downloads(digestmap_t *result); +void launch_descriptor_downloads(int purpose, + smartlist_t *downloadable, + const routerstatus_t *source, + time_t now); + +int hex_digest_nickname_decode(const char *hexdigest, + char *digest_out, + char *nickname_qualifier_out, + char *nickname_out); + #endif diff --git a/src/or/routerparse.c b/src/or/routerparse.c index 2ff546bb1d..299d07d376 100644 --- a/src/or/routerparse.c +++ b/src/or/routerparse.c @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -64,13 +64,13 @@ typedef enum { K_DIR_OPTIONS, K_CLIENT_VERSIONS, K_SERVER_VERSIONS, + K_OR_ADDRESS, K_P, K_R, K_S, K_V, K_W, K_M, - K_EVENTDNS, K_EXTRA_INFO, K_EXTRA_INFO_DIGEST, K_CACHES_EXTRA_INFO, @@ -178,7 +178,7 @@ typedef struct directory_token_t { size_t object_size; /**< Bytes in object_body */ char *object_body; /**< Contents of object, base64-decoded. */ - crypto_pk_env_t *key; /**< For public keys only. Heap-allocated. */ + crypto_pk_t *key; /**< For public keys only. Heap-allocated. */ char *error; /**< For _ERR tokens only. */ } directory_token_t; @@ -263,7 +263,7 @@ typedef struct token_rule_t { /* Argument multiplicity: exactly <b>n</b> arguments. */ #define EQ(n) n,n,0 -/** List of tokens allowable in router descriptors */ +/** List of tokens recognized in router descriptors */ static token_rule_t routerdesc_token_table[] = { T0N("reject", K_REJECT, ARGS, NO_OBJ ), T0N("accept", K_ACCEPT, ARGS, NO_OBJ ), @@ -287,7 +287,7 @@ static token_rule_t routerdesc_token_table[] = { T01("family", K_FAMILY, ARGS, NO_OBJ ), T01("caches-extra-info", K_CACHES_EXTRA_INFO, NO_ARGS, NO_OBJ ), - T01("eventdns", K_EVENTDNS, ARGS, NO_OBJ ), + T0N("or-address", K_OR_ADDRESS, GE(1), NO_OBJ ), T0N("opt", K_OPT, CONCAT_ARGS, OBJ_OK ), T1( "bandwidth", K_BANDWIDTH, GE(3), NO_OBJ ), @@ -296,7 +296,7 @@ static token_rule_t routerdesc_token_table[] = { END_OF_TABLE }; -/** List of tokens allowable in extra-info documents. */ +/** List of tokens recognized in extra-info documents. */ static token_rule_t extrainfo_token_table[] = { T1_END( "router-signature", K_ROUTER_SIGNATURE, NO_ARGS, NEED_OBJ ), T1( "published", K_PUBLISHED, CONCAT_ARGS, NO_OBJ ), @@ -333,7 +333,7 @@ static token_rule_t extrainfo_token_table[] = { END_OF_TABLE }; -/** List of tokens allowable in the body part of v2 and v3 networkstatus +/** List of tokens recognized in the body part of v2 and v3 networkstatus * documents. */ static token_rule_t rtrstatus_token_table[] = { T01("p", K_P, CONCAT_ARGS, NO_OBJ ), @@ -346,7 +346,7 @@ static token_rule_t rtrstatus_token_table[] = { END_OF_TABLE }; -/** List of tokens allowable in the header part of v2 networkstatus documents. +/** List of tokens recognized in the header part of v2 networkstatus documents. */ static token_rule_t netstatus_token_table[] = { T1( "published", K_PUBLISHED, CONCAT_ARGS, NO_OBJ ), @@ -364,14 +364,14 @@ static token_rule_t netstatus_token_table[] = { END_OF_TABLE }; -/** List of tokens allowable in the footer of v1/v2 directory/networkstatus +/** List of tokens recognized in the footer of v1/v2 directory/networkstatus * footers. */ static token_rule_t dir_footer_token_table[] = { T1("directory-signature", K_DIRECTORY_SIGNATURE, EQ(1), NEED_OBJ ), END_OF_TABLE }; -/** List of tokens allowable in v1 directory headers/footers. */ +/** List of tokens recognized in v1 directory headers/footers. */ static token_rule_t dir_token_table[] = { /* don't enforce counts; this is obsolete. */ T( "network-status", K_NETWORK_STATUS, NO_ARGS, NO_OBJ ), @@ -403,14 +403,14 @@ static token_rule_t dir_token_table[] = { NO_ARGS, NEED_OBJ), \ T01("dir-address", K_DIR_ADDRESS, GE(1), NO_OBJ), -/** List of tokens allowable in V3 authority certificates. */ +/** List of tokens recognized in V3 authority certificates. */ static token_rule_t dir_key_certificate_table[] = { CERTIFICATE_MEMBERS T1("fingerprint", K_FINGERPRINT, CONCAT_ARGS, NO_OBJ ), END_OF_TABLE }; -/** List of tokens allowable in rendezvous service descriptors */ +/** List of tokens recognized in rendezvous service descriptors */ static token_rule_t desc_token_table[] = { T1_START("rendezvous-service-descriptor", R_RENDEZVOUS_SERVICE_DESCRIPTOR, EQ(1), NO_OBJ), @@ -424,7 +424,7 @@ static token_rule_t desc_token_table[] = { END_OF_TABLE }; -/** List of tokens allowed in the (encrypted) list of introduction points of +/** List of tokens recognized in the (encrypted) list of introduction points of * rendezvous service descriptors */ static token_rule_t ipo_token_table[] = { T1_START("introduction-point", R_IPO_IDENTIFIER, EQ(1), NO_OBJ), @@ -435,7 +435,7 @@ static token_rule_t ipo_token_table[] = { END_OF_TABLE }; -/** List of tokens allowed in the (possibly encrypted) list of introduction +/** List of tokens recognized in the (possibly encrypted) list of introduction * points of rendezvous service descriptors */ static token_rule_t client_keys_token_table[] = { T1_START("client-name", C_CLIENT_NAME, CONCAT_ARGS, NO_OBJ), @@ -444,7 +444,7 @@ static token_rule_t client_keys_token_table[] = { END_OF_TABLE }; -/** List of tokens allowed in V3 networkstatus votes. */ +/** List of tokens recognized in V3 networkstatus votes. */ static token_rule_t networkstatus_token_table[] = { T1_START("network-status-version", K_NETWORK_STATUS_VERSION, GE(1), NO_OBJ ), @@ -472,7 +472,7 @@ static token_rule_t networkstatus_token_table[] = { END_OF_TABLE }; -/** List of tokens allowed in V3 networkstatus consensuses. */ +/** List of tokens recognized in V3 networkstatus consensuses. */ static token_rule_t networkstatus_consensus_token_table[] = { T1_START("network-status-version", K_NETWORK_STATUS_VERSION, GE(1), NO_OBJ ), @@ -498,7 +498,7 @@ static token_rule_t networkstatus_consensus_token_table[] = { END_OF_TABLE }; -/** List of tokens allowable in the footer of v1/v2 directory/networkstatus +/** List of tokens recognized in the footer of v1/v2 directory/networkstatus * footers. */ static token_rule_t networkstatus_vote_footer_token_table[] = { T01("directory-footer", K_DIRECTORY_FOOTER, NO_ARGS, NO_OBJ ), @@ -507,7 +507,7 @@ static token_rule_t networkstatus_vote_footer_token_table[] = { END_OF_TABLE }; -/** List of tokens allowable in detached networkstatus signature documents. */ +/** List of tokens recognized in detached networkstatus signature documents. */ static token_rule_t networkstatus_detached_signature_token_table[] = { T1_START("consensus-digest", K_CONSENSUS_DIGEST, GE(1), NO_OBJ ), T("additional-digest", K_ADDITIONAL_DIGEST,GE(3), NO_OBJ ), @@ -519,6 +519,7 @@ static token_rule_t networkstatus_detached_signature_token_table[] = { END_OF_TABLE }; +/** List of tokens recognized in microdescriptors */ static token_rule_t microdesc_token_table[] = { T1_START("onion-key", K_ONION_KEY, NO_ARGS, NEED_KEY_1024), T01("family", K_FAMILY, ARGS, NO_OBJ ), @@ -543,6 +544,7 @@ static int router_get_hashes_impl(const char *s, size_t s_len, const char *start_str, const char *end_str, char end_char); static void token_clear(directory_token_t *tok); +static smartlist_t *find_all_by_keyword(smartlist_t *s, directory_keyword k); static smartlist_t *find_all_exitpolicy(smartlist_t *s); static directory_token_t *_find_by_keyword(smartlist_t *s, directory_keyword keyword, @@ -568,10 +570,10 @@ static directory_token_t *get_next_token(memarea_t *area, static int check_signature_token(const char *digest, ssize_t digest_len, directory_token_t *tok, - crypto_pk_env_t *pkey, + crypto_pk_t *pkey, int flags, const char *doctype); -static crypto_pk_env_t *find_dir_signing_key(const char *str, const char *eos); +static crypto_pk_t *find_dir_signing_key(const char *str, const char *eos); #undef DEBUG_AREA_ALLOC @@ -681,12 +683,12 @@ router_get_networkstatus_v3_hash(const char *s, char *digest, ' ', alg); } -/** Set <b>digest</b> to the SHA-1 digest of the hash of the extrainfo - * string in <b>s</b>. Return 0 on success, -1 on failure. */ +/** Set <b>digest</b> to the SHA-1 digest of the hash of the <b>s_len</b>-byte + * extrainfo string at <b>s</b>. Return 0 on success, -1 on failure. */ int -router_get_extrainfo_hash(const char *s, char *digest) +router_get_extrainfo_hash(const char *s, size_t s_len, char *digest) { - return router_get_hash_impl(s, strlen(s), digest, "extra-info", + return router_get_hash_impl(s, s_len, digest, "extra-info", "\nrouter-signature",'\n', DIGEST_SHA1); } @@ -699,7 +701,7 @@ router_get_extrainfo_hash(const char *s, char *digest) */ int router_append_dirobj_signature(char *buf, size_t buf_len, const char *digest, - size_t digest_len, crypto_pk_env_t *private_key) + size_t digest_len, crypto_pk_t *private_key) { char *signature; size_t i, keysize; @@ -765,7 +767,7 @@ tor_version_is_obsolete(const char *myversion, const char *versionlist) log_err(LD_BUG,"I couldn't parse my own version (%s)", myversion); tor_assert(0); } - version_sl = smartlist_create(); + version_sl = smartlist_new(); smartlist_split_string(version_sl, versionlist, ",", SPLIT_SKIP_SPACE, 0); if (!strlen(versionlist)) { /* no authorities cared or agreed */ @@ -773,7 +775,7 @@ tor_version_is_obsolete(const char *myversion, const char *versionlist) goto done; } - SMARTLIST_FOREACH(version_sl, const char *, cp, { + SMARTLIST_FOREACH_BEGIN(version_sl, const char *, cp) { if (!strcmpstart(cp, "Tor ")) cp += 4; @@ -795,7 +797,7 @@ tor_version_is_obsolete(const char *myversion, const char *versionlist) found_older = 1; } } - }); + } SMARTLIST_FOREACH_END(cp); /* We didn't find the listed version. Is it new or old? */ if (found_any_in_series && !found_newer_in_series && found_newer) { @@ -826,7 +828,7 @@ router_parse_directory(const char *str) int r; const char *end, *cp, *str_dup = str; smartlist_t *tokens = NULL; - crypto_pk_env_t *declared_key = NULL; + crypto_pk_t *declared_key = NULL; memarea_t *area = memarea_new(); /* XXXX This could be simplified a lot, but it will all go away @@ -847,7 +849,7 @@ router_parse_directory(const char *str) log_warn(LD_DIR, "No signature found on directory."); goto err; } ++cp; - tokens = smartlist_create(); + tokens = smartlist_new(); if (tokenize_string(area,cp,strchr(cp,'\0'),tokens,dir_token_table,0)) { log_warn(LD_DIR, "Error tokenizing directory signature"); goto err; } @@ -900,7 +902,7 @@ router_parse_directory(const char *str) dump_desc(str_dup, "v1 directory"); r = -1; done: - if (declared_key) crypto_free_pk_env(declared_key); + if (declared_key) crypto_pk_free(declared_key); if (tokens) { SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); @@ -922,7 +924,7 @@ router_parse_runningrouters(const char *str) directory_token_t *tok; time_t published_on; int r = -1; - crypto_pk_env_t *declared_key = NULL; + crypto_pk_t *declared_key = NULL; smartlist_t *tokens = NULL; const char *eos = str + strlen(str), *str_dup = str; memarea_t *area = NULL; @@ -932,7 +934,7 @@ router_parse_runningrouters(const char *str) goto err; } area = memarea_new(); - tokens = smartlist_create(); + tokens = smartlist_new(); if (tokenize_string(area,str,eos,tokens,dir_token_table,0)) { log_warn(LD_DIR, "Error tokenizing running-routers"); goto err; } @@ -960,13 +962,13 @@ router_parse_runningrouters(const char *str) /* Now that we know the signature is okay, and we have a * publication time, cache the list. */ - if (get_options()->DirPort && !authdir_mode_v1(get_options())) + if (get_options()->DirPort_set && !authdir_mode_v1(get_options())) dirserv_set_cached_directory(str, published_on, 1); r = 0; err: dump_desc(str_dup, "v1 running-routers"); - if (declared_key) crypto_free_pk_env(declared_key); + if (declared_key) crypto_pk_free(declared_key); if (tokens) { SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); @@ -981,12 +983,12 @@ router_parse_runningrouters(const char *str) /** Given a directory or running-routers string in <b>str</b>, try to * find the its dir-signing-key token (if any). If this token is * present, extract and return the key. Return NULL on failure. */ -static crypto_pk_env_t * +static crypto_pk_t * find_dir_signing_key(const char *str, const char *eos) { const char *cp; directory_token_t *tok; - crypto_pk_env_t *key = NULL; + crypto_pk_t *key = NULL; memarea_t *area = NULL; tor_assert(str); tor_assert(eos); @@ -1029,7 +1031,7 @@ find_dir_signing_key(const char *str, const char *eos) /** Return true iff <b>key</b> is allowed to sign directories. */ static int -dir_signing_key_is_trusted(crypto_pk_env_t *key) +dir_signing_key_is_trusted(crypto_pk_t *key) { char digest[DIGEST_LEN]; if (!key) return 0; @@ -1056,7 +1058,7 @@ static int check_signature_token(const char *digest, ssize_t digest_len, directory_token_t *tok, - crypto_pk_env_t *pkey, + crypto_pk_t *pkey, int flags, const char *doctype) { @@ -1261,7 +1263,7 @@ dump_distinct_digest_count(int severity) * s through end into the signed_descriptor_body of the resulting * routerinfo_t. * - * If <b>end</b> is NULL, <b>s</b> must be properly NULL-terminated. + * If <b>end</b> is NULL, <b>s</b> must be properly NUL-terminated. * * If <b>allow_annotations</b>, it's okay to encounter annotations in <b>s</b> * before the router; if it's false, reject the router if it's annotated. If @@ -1297,7 +1299,7 @@ router_parse_entry_from_string(const char *s, const char *end, --end; area = memarea_new(); - tokens = smartlist_create(); + tokens = smartlist_new(); if (prepend_annotations) { if (tokenize_string(area,prepend_annotations,NULL,tokens, routerdesc_token_table,TS_NOCHECK)) { @@ -1357,7 +1359,6 @@ router_parse_entry_from_string(const char *s, const char *end, tor_assert(tok->n_args >= 5); router = tor_malloc_zero(sizeof(routerinfo_t)); - router->country = -1; router->cache_info.routerlist_index = -1; router->cache_info.annotations_len = s-start_of_annotations + prepend_len; router->cache_info.signed_descriptor_len = end-s; @@ -1503,19 +1504,33 @@ router_parse_entry_from_string(const char *s, const char *end, router->contact_info = tor_strdup(tok->args[0]); } - if ((tok = find_opt_by_keyword(tokens, K_EVENTDNS))) { - router->has_old_dnsworkers = tok->n_args && !strcmp(tok->args[0], "0"); - } else if (router->platform) { - if (! tor_version_as_new_as(router->platform, "0.1.2.2-alpha")) - router->has_old_dnsworkers = 1; - } - if (find_opt_by_keyword(tokens, K_REJECT6) || find_opt_by_keyword(tokens, K_ACCEPT6)) { log_warn(LD_DIR, "Rejecting router with reject6/accept6 line: they crash " "older Tors."); goto err; } + { + smartlist_t *or_addresses = find_all_by_keyword(tokens, K_OR_ADDRESS); + if (or_addresses) { + SMARTLIST_FOREACH_BEGIN(or_addresses, directory_token_t *, t) { + tor_addr_t a; + maskbits_t bits; + uint16_t port_min, port_max; + /* XXXX Prop186 the full spec allows much more than this. */ + if (tor_addr_parse_mask_ports(t->args[0], &a, &bits, &port_min, + &port_max) == AF_INET6 && + bits == 128 && + port_min == port_max) { + /* Okay, this is one we can understand. */ + tor_addr_copy(&router->ipv6_addr, &a); + router->ipv6_orport = port_min; + break; + } + } SMARTLIST_FOREACH_END(t); + smartlist_free(or_addresses); + } + } exit_policy_tokens = find_all_exitpolicy(tokens); if (!smartlist_len(exit_policy_tokens)) { log_warn(LD_DIR, "No exit policy tokens in descriptor."); @@ -1532,7 +1547,7 @@ router_parse_entry_from_string(const char *s, const char *end, if ((tok = find_opt_by_keyword(tokens, K_FAMILY)) && tok->n_args) { int i; - router->declared_family = smartlist_create(); + router->declared_family = smartlist_new(); for (i=0;i<tok->n_args;++i) { if (!is_legal_nickname_or_hexdigest(tok->args[i])) { log_warn(LD_DIR, "Illegal nickname %s in family line", @@ -1574,8 +1589,6 @@ router_parse_entry_from_string(const char *s, const char *end, "router descriptor") < 0) goto err; - routerinfo_set_country(router); - if (!router->or_port) { log_warn(LD_DIR,"or_port unreadable or 0. Failing."); goto err; @@ -1618,7 +1631,7 @@ extrainfo_parse_entry_from_string(const char *s, const char *end, char digest[128]; smartlist_t *tokens = NULL; directory_token_t *tok; - crypto_pk_env_t *key = NULL; + crypto_pk_t *key = NULL; routerinfo_t *router = NULL; memarea_t *area = NULL; const char *s_dup = s; @@ -1631,11 +1644,11 @@ extrainfo_parse_entry_from_string(const char *s, const char *end, while (end > s+2 && *(end-1) == '\n' && *(end-2) == '\n') --end; - if (router_get_extrainfo_hash(s, digest) < 0) { + if (router_get_extrainfo_hash(s, end-s, digest) < 0) { log_warn(LD_DIR, "Couldn't compute router hash."); goto err; } - tokens = smartlist_create(); + tokens = smartlist_new(); area = memarea_new(); if (tokenize_string(area,s,end,tokens,extrainfo_token_table,0)) { log_warn(LD_DIR, "Error tokenizing extra-info document."); @@ -1768,7 +1781,7 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) return NULL; } - tokens = smartlist_create(); + tokens = smartlist_new(); area = memarea_new(); if (tokenize_string(area,s, eos, tokens, dir_key_certificate_table, 0) < 0) { log_warn(LD_DIR, "Error tokenizing key certificate"); @@ -1823,9 +1836,9 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) struct in_addr in; char *address = NULL; tor_assert(tok->n_args); - /* XXX023 use tor_addr_port_parse() below instead. -RD */ - if (parse_addr_port(LOG_WARN, tok->args[0], &address, NULL, - &cert->dir_port)<0 || + /* XXX024 use some tor_addr parse function below instead. -RD */ + if (tor_addr_port_split(LOG_WARN, tok->args[0], &address, + &cert->dir_port) < 0 || tor_inet_aton(address, &in) == 0) { log_warn(LD_DIR, "Couldn't parse dir-address in certificate"); tor_free(address); @@ -1975,6 +1988,7 @@ routerstatus_parse_entry_from_string(memarea_t *area, if (!consensus_method) flav = FLAV_NS; + tor_assert(flav == FLAV_NS || flav == FLAV_MICRODESC); eos = find_start_of_next_routerstatus(*s); @@ -1987,15 +2001,16 @@ routerstatus_parse_entry_from_string(memarea_t *area, goto err; } tok = find_by_keyword(tokens, K_R); - tor_assert(tok->n_args >= 7); + tor_assert(tok->n_args >= 7); /* guaranteed by GE(7) in K_R setup */ if (flav == FLAV_NS) { if (tok->n_args < 8) { log_warn(LD_DIR, "Too few arguments to r"); goto err; } - } else { - offset = -1; + } else if (flav == FLAV_MICRODESC) { + offset = -1; /* There is no identity digest */ } + if (vote_rs) { rs = &vote_rs->status; } else { @@ -2069,7 +2084,7 @@ routerstatus_parse_entry_from_string(memarea_t *area, else if (!strcmp(tok->args[i], "Fast")) rs->is_fast = 1; else if (!strcmp(tok->args[i], "Running")) - rs->is_running = 1; + rs->is_flagged_running = 1; else if (!strcmp(tok->args[i], "Named")) rs->is_named = 1; else if (!strcmp(tok->args[i], "Valid")) @@ -2100,6 +2115,8 @@ routerstatus_parse_entry_from_string(memarea_t *area, rs->version_supports_begindir = 1; rs->version_supports_extrainfo_upload = 1; rs->version_supports_conditional_consensus = 1; + rs->version_supports_microdesc_cache = 1; + rs->version_supports_optimistic_data = 1; } else { rs->version_supports_begindir = tor_version_as_new_as(tok->args[0], "0.2.0.1-alpha"); @@ -2109,6 +2126,10 @@ routerstatus_parse_entry_from_string(memarea_t *area, tor_version_as_new_as(tok->args[0], "0.2.0.8-alpha"); rs->version_supports_conditional_consensus = tor_version_as_new_as(tok->args[0], "0.2.1.1-alpha"); + rs->version_supports_microdesc_cache = + tor_version_supports_microdescriptors(tok->args[0]); + rs->version_supports_optimistic_data = + tor_version_as_new_as(tok->args[0], "0.2.3.1-alpha"); } if (vote_rs) { vote_rs->version = tor_strdup(tok->args[0]); @@ -2171,6 +2192,21 @@ routerstatus_parse_entry_from_string(memarea_t *area, vote_rs->microdesc = line; } } SMARTLIST_FOREACH_END(t); + } else if (flav == FLAV_MICRODESC) { + tok = find_opt_by_keyword(tokens, K_M); + if (tok) { + tor_assert(tok->n_args); + if (digest256_from_base64(rs->descriptor_digest, tok->args[0])) { + log_warn(LD_DIR, "Error decoding microdescriptor digest %s", + escaped(tok->args[0])); + goto err; + } + } else { + log_info(LD_BUG, "Found an entry in networkstatus with no " + "microdescriptor digest. (Router %s=%s at %s:%d.)", + rs->nickname, hex_str(rs->identity_digest, DIGEST_LEN), + fmt_addr32(rs->addr), rs->or_port); + } } if (!strcasecmp(rs->nickname, UNNAMED_ROUTER_NICKNAME)) @@ -2221,8 +2257,8 @@ networkstatus_v2_t * networkstatus_v2_parse_from_string(const char *s) { const char *eos, *s_dup = s; - smartlist_t *tokens = smartlist_create(); - smartlist_t *footer_tokens = smartlist_create(); + smartlist_t *tokens = smartlist_new(); + smartlist_t *footer_tokens = smartlist_new(); networkstatus_v2_t *ns = NULL; char ns_digest[DIGEST_LEN]; char tmp_digest[DIGEST_LEN]; @@ -2332,7 +2368,7 @@ networkstatus_v2_parse_from_string(const char *s) goto err; } - ns->entries = smartlist_create(); + ns->entries = smartlist_new(); s = eos; SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_clear(tokens); @@ -2779,7 +2815,7 @@ networkstatus_t * networkstatus_parse_vote_from_string(const char *s, const char **eos_out, networkstatus_type_t ns_type) { - smartlist_t *tokens = smartlist_create(); + smartlist_t *tokens = smartlist_new(); smartlist_t *rs_tokens = NULL, *footer_tokens = NULL; networkstatus_voter_info_t *voter = NULL; networkstatus_t *ns = NULL; @@ -2791,6 +2827,7 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, int i, inorder, n_signatures = 0; memarea_t *area = NULL, *rs_area = NULL; consensus_flavor_t flav = FLAV_NS; + char *last_kwd=NULL; tor_assert(s); @@ -2864,7 +2901,7 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, if (parse_iso_time(tok->args[0], &ns->published)) goto err; - ns->supported_methods = smartlist_create(); + ns->supported_methods = smartlist_new(); tok = find_opt_by_keyword(tokens, K_CONSENSUS_METHODS); if (tok) { for (i=0; i < tok->n_args; ++i) @@ -2931,7 +2968,7 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, } tok = find_by_keyword(tokens, K_KNOWN_FLAGS); - ns->known_flags = smartlist_create(); + ns->known_flags = smartlist_new(); inorder = 1; for (i = 0; i < tok->n_args; ++i) { smartlist_add(ns->known_flags, tor_strdup(tok->args[i])); @@ -2947,15 +2984,18 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, tok = find_opt_by_keyword(tokens, K_PARAMS); if (tok) { + int any_dups = 0; inorder = 1; - ns->net_params = smartlist_create(); + ns->net_params = smartlist_new(); for (i = 0; i < tok->n_args; ++i) { int ok=0; char *eq = strchr(tok->args[i], '='); + size_t eq_pos; if (!eq) { log_warn(LD_DIR, "Bad element '%s' in params", escaped(tok->args[i])); goto err; } + eq_pos = eq-tok->args[i]; tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, NULL); if (!ok) { log_warn(LD_DIR, "Bad element '%s' in params", escaped(tok->args[i])); @@ -2965,15 +3005,27 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, log_warn(LD_DIR, "%s >= %s", tok->args[i-1], tok->args[i]); inorder = 0; } + if (last_kwd && eq_pos == strlen(last_kwd) && + fast_memeq(last_kwd, tok->args[i], eq_pos)) { + log_warn(LD_DIR, "Duplicate value for %s parameter", + escaped(tok->args[i])); + any_dups = 1; + } + tor_free(last_kwd); + last_kwd = tor_strndup(tok->args[i], eq_pos); smartlist_add(ns->net_params, tor_strdup(tok->args[i])); } if (!inorder) { log_warn(LD_DIR, "params not in order"); goto err; } + if (any_dups) { + log_warn(LD_DIR, "Duplicate in parameters"); + goto err; + } } - ns->voters = smartlist_create(); + ns->voters = smartlist_new(); SMARTLIST_FOREACH_BEGIN(tokens, directory_token_t *, _tok) { tok = _tok; @@ -2983,7 +3035,7 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, if (voter) smartlist_add(ns->voters, voter); voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t)); - voter->sigs = smartlist_create(); + voter->sigs = smartlist_new(); if (ns->type != NS_TYPE_CONSENSUS) memcpy(voter->vote_digest, ns_digests.d[DIGEST_SHA1], DIGEST_LEN); @@ -3068,10 +3120,10 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, } /* Parse routerstatus lines. */ - rs_tokens = smartlist_create(); + rs_tokens = smartlist_new(); rs_area = memarea_new(); s = end_of_header; - ns->routerstatus_list = smartlist_create(); + ns->routerstatus_list = smartlist_new(); while (!strcmpstart(s, "r ")) { if (ns->type != NS_TYPE_CONSENSUS) { @@ -3111,7 +3163,7 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, } /* Parse footer; check signature. */ - footer_tokens = smartlist_create(); + footer_tokens = smartlist_new(); if ((end_of_footer = strstr(s, "\nnetwork-status-version "))) ++end_of_footer; else @@ -3144,7 +3196,7 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, tok = find_opt_by_keyword(footer_tokens, K_BW_WEIGHTS); if (tok) { - ns->weight_params = smartlist_create(); + ns->weight_params = smartlist_new(); for (i = 0; i < tok->n_args; ++i) { int ok=0; char *eq = strchr(tok->args[i], '='); @@ -3309,6 +3361,7 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, } if (rs_area) memarea_drop_all(rs_area); + tor_free(last_kwd); return ns; } @@ -3336,7 +3389,7 @@ detached_get_signatures(ns_detached_signatures_t *sigs, { smartlist_t *sl = strmap_get(sigs->signatures, flavor_name); if (!sl) { - sl = smartlist_create(); + sl = smartlist_new(); strmap_set(sigs->signatures, flavor_name, sl); } return sl; @@ -3353,7 +3406,7 @@ networkstatus_parse_detached_signatures(const char *s, const char *eos) memarea_t *area = NULL; digests_t *digests; - smartlist_t *tokens = smartlist_create(); + smartlist_t *tokens = smartlist_new(); ns_detached_signatures_t *sigs = tor_malloc_zero(sizeof(ns_detached_signatures_t)); sigs->digests = strmap_new(); @@ -3502,10 +3555,10 @@ networkstatus_parse_detached_signatures(const char *s, const char *eos) siglist = detached_get_signatures(sigs, flavor); is_duplicate = 0; - SMARTLIST_FOREACH(siglist, document_signature_t *, s, { - if (s->alg == alg && - tor_memeq(id_digest, s->identity_digest, DIGEST_LEN) && - tor_memeq(sk_digest, s->signing_key_digest, DIGEST_LEN)) { + SMARTLIST_FOREACH(siglist, document_signature_t *, dsig, { + if (dsig->alg == alg && + tor_memeq(id_digest, dsig->identity_digest, DIGEST_LEN) && + tor_memeq(sk_digest, dsig->signing_key_digest, DIGEST_LEN)) { is_duplicate = 1; } }); @@ -3608,7 +3661,7 @@ router_add_exit_policy(routerinfo_t *router, directory_token_t *tok) if (!newe) return -1; if (! router->exit_policy) - router->exit_policy = smartlist_create(); + router->exit_policy = smartlist_new(); if (((tok->tp == K_ACCEPT6 || tok->tp == K_REJECT6) && tor_addr_family(&newe->addr) == AF_INET) @@ -3716,7 +3769,7 @@ static void token_clear(directory_token_t *tok) { if (tok->key) - crypto_free_pk_env(tok->key); + crypto_pk_free(tok->key); } #define ALLOC_ZERO(sz) memarea_alloc_zero(area,sz) @@ -3962,11 +4015,11 @@ get_next_token(memarea_t *area, RET_ERR("Couldn't parse object: missing footer or object much too big."); if (!strcmp(tok->object_type, "RSA PUBLIC KEY")) { /* If it's a public key */ - tok->key = crypto_new_pk_env(); + tok->key = crypto_pk_new(); if (crypto_pk_read_public_key_from_string(tok->key, obstart, eol-obstart)) RET_ERR("Couldn't parse public key."); } else if (!strcmp(tok->object_type, "RSA PRIVATE KEY")) { /* private key */ - tok->key = crypto_new_pk_env(); + tok->key = crypto_pk_new(); if (crypto_pk_read_private_key_from_string(tok->key, obstart, eol-obstart)) RET_ERR("Couldn't parse private key."); } else { /* If it's something else, try to base64-decode it */ @@ -4122,13 +4175,30 @@ _find_by_keyword(smartlist_t *s, directory_keyword keyword, return tok; } +/** If there are any directory_token_t entries in <b>s</b> whose keyword is + * <b>k</b>, return a newly allocated smartlist_t containing all such entries, + * in the same order in which they occur in <b>s</b>. Otherwise return + * NULL. */ +static smartlist_t * +find_all_by_keyword(smartlist_t *s, directory_keyword k) +{ + smartlist_t *out = NULL; + SMARTLIST_FOREACH(s, directory_token_t *, t, + if (t->tp == k) { + if (!out) + out = smartlist_new(); + smartlist_add(out, t); + }); + return out; +} + /** Return a newly allocated smartlist of all accept or reject tokens in * <b>s</b>. */ static smartlist_t * find_all_exitpolicy(smartlist_t *s) { - smartlist_t *out = smartlist_create(); + smartlist_t *out = smartlist_new(); SMARTLIST_FOREACH(s, directory_token_t *, t, if (t->tp == K_ACCEPT || t->tp == K_ACCEPT6 || t->tp == K_REJECT || t->tp == K_REJECT6) @@ -4136,6 +4206,13 @@ find_all_exitpolicy(smartlist_t *s) return out; } +/** Helper function for <b>router_get_hash_impl</b>: given <b>s</b>, + * <b>s_len</b>, <b>start_str</b>, <b>end_str</b>, and <b>end_c</b> with the + * same semantics as in that function, set *<b>start_out</b> (inclusive) and + * *<b>end_out</b> (exclusive) to the boundaries of the string to be hashed. + * + * Return 0 on success and -1 on failure. + */ static int router_get_hash_impl_helper(const char *s, size_t s_len, const char *start_str, @@ -4300,8 +4377,8 @@ microdescs_parse_from_string(const char *s, const char *eos, s = eat_whitespace_eos(s, eos); area = memarea_new(); - result = smartlist_create(); - tokens = smartlist_create(); + result = smartlist_new(); + tokens = smartlist_new(); while (s < eos) { start_of_next_microdesc = find_start_of_next_microdesc(s, eos); @@ -4346,7 +4423,7 @@ microdescs_parse_from_string(const char *s, const char *eos, if ((tok = find_opt_by_keyword(tokens, K_FAMILY))) { int i; - md->family = smartlist_create(); + md->family = smartlist_new(); for (i=0;i<tok->n_args;++i) { if (!is_legal_nickname_or_hexdigest(tok->args[i])) { log_warn(LD_DIR, "Illegal nickname %s in family line", @@ -4358,7 +4435,7 @@ microdescs_parse_from_string(const char *s, const char *eos, } if ((tok = find_opt_by_keyword(tokens, K_P))) { - md->exitsummary = tor_strdup(tok->args[0]); + md->exit_policy = parse_short_policy(tok->args[0]); } crypto_digest256(md->digest, md->body, md->bodylen, DIGEST_SHA256); @@ -4381,6 +4458,14 @@ microdescs_parse_from_string(const char *s, const char *eos, return result; } +/** Return true iff this Tor version can answer directory questions + * about microdescriptors. */ +int +tor_version_supports_microdescriptors(const char *platform) +{ + return tor_version_as_new_as(platform, "0.2.3.1-alpha"); +} + /** Parse the Tor version of the platform string <b>platform</b>, * and compare it to the version in <b>cutoff</b>. Return 1 if * the router is at least as new as the cutoff, else return 0. @@ -4511,7 +4596,7 @@ tor_version_parse(const char *s, tor_version_t *out) if (close_paren-cp > HEX_DIGEST_LEN) return -1; hexlen = (int)(close_paren-cp); - memset(digest, 0, sizeof(digest)); + memwipe(digest, 0, sizeof(digest)); if ( hexlen == 0 || (hexlen % 2) == 1) return -1; if (base16_decode(digest, hexlen/2, cp, hexlen)) @@ -4621,7 +4706,7 @@ rend_parse_v2_service_descriptor(rend_service_descriptor_t **parsed_out, tor_malloc_zero(sizeof(rend_service_descriptor_t)); char desc_hash[DIGEST_LEN]; const char *eos; - smartlist_t *tokens = smartlist_create(); + smartlist_t *tokens = smartlist_new(); directory_token_t *tok; char secret_id_part[DIGEST_LEN]; int i, version, num_ok=1; @@ -4730,7 +4815,7 @@ rend_parse_v2_service_descriptor(rend_service_descriptor_t **parsed_out, /* Parse protocol versions. */ tok = find_by_keyword(tokens, R_PROTOCOL_VERSIONS); tor_assert(tok->n_args == 1); - versions = smartlist_create(); + versions = smartlist_new(); smartlist_split_string(versions, tok->args[0], ",", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); for (i = 0; i < smartlist_len(versions); i++) { @@ -4738,6 +4823,9 @@ rend_parse_v2_service_descriptor(rend_service_descriptor_t **parsed_out, 10, 0, INT_MAX, &num_ok, NULL); if (!num_ok) /* It's a string; let's ignore it. */ continue; + if (version >= REND_PROTOCOL_VERSION_BITMASK_WIDTH) + /* Avoid undefined left-shift behaviour. */ + continue; result->protocols |= 1 << version; } SMARTLIST_FOREACH(versions, char *, cp, tor_free(cp)); @@ -4813,8 +4901,8 @@ rend_decrypt_introduction_points(char **ipos_decrypted, session_key[CIPHER_KEY_LEN], *dec; int declen, client_blocks; size_t pos = 0, len, client_entries_len; - crypto_digest_env_t *digest; - crypto_cipher_env_t *cipher; + crypto_digest_t *digest; + crypto_cipher_t *cipher; client_blocks = (int) ipos_encrypted[1]; client_entries_len = client_blocks * REND_BASIC_AUTH_CLIENT_MULTIPLE * REND_BASIC_AUTH_CLIENT_ENTRY_LEN; @@ -4824,33 +4912,33 @@ rend_decrypt_introduction_points(char **ipos_decrypted, return -1; } memcpy(iv, ipos_encrypted + 2 + client_entries_len, CIPHER_IV_LEN); - digest = crypto_new_digest_env(); + digest = crypto_digest_new(); crypto_digest_add_bytes(digest, descriptor_cookie, REND_DESC_COOKIE_LEN); crypto_digest_add_bytes(digest, iv, CIPHER_IV_LEN); crypto_digest_get_digest(digest, client_id, REND_BASIC_AUTH_CLIENT_ID_LEN); - crypto_free_digest_env(digest); + crypto_digest_free(digest); for (pos = 2; pos < 2 + client_entries_len; pos += REND_BASIC_AUTH_CLIENT_ENTRY_LEN) { if (tor_memeq(ipos_encrypted + pos, client_id, REND_BASIC_AUTH_CLIENT_ID_LEN)) { /* Attempt to decrypt introduction points. */ - cipher = crypto_create_init_cipher(descriptor_cookie, 0); + cipher = crypto_cipher_new(descriptor_cookie); if (crypto_cipher_decrypt(cipher, session_key, ipos_encrypted + pos + REND_BASIC_AUTH_CLIENT_ID_LEN, CIPHER_KEY_LEN) < 0) { log_warn(LD_REND, "Could not decrypt session key for client."); - crypto_free_cipher_env(cipher); + crypto_cipher_free(cipher); return -1; } - crypto_free_cipher_env(cipher); - cipher = crypto_create_init_cipher(session_key, 0); + crypto_cipher_free(cipher); + len = ipos_encrypted_size - 2 - client_entries_len - CIPHER_IV_LEN; dec = tor_malloc(len); - declen = crypto_cipher_decrypt_with_iv(cipher, dec, len, + declen = crypto_cipher_decrypt_with_iv(session_key, dec, len, ipos_encrypted + 2 + client_entries_len, ipos_encrypted_size - 2 - client_entries_len); - crypto_free_cipher_env(cipher); + if (declen < 0) { log_warn(LD_REND, "Could not decrypt introduction point string."); tor_free(dec); @@ -4871,7 +4959,6 @@ rend_decrypt_introduction_points(char **ipos_decrypted, "check your authorization for this service!"); return -1; } else if (ipos_encrypted[0] == (int)REND_STEALTH_AUTH) { - crypto_cipher_env_t *cipher; char *dec; int declen; if (ipos_encrypted_size < CIPHER_IV_LEN + 2) { @@ -4880,13 +4967,13 @@ rend_decrypt_introduction_points(char **ipos_decrypted, return -1; } dec = tor_malloc_zero(ipos_encrypted_size - CIPHER_IV_LEN - 1); - cipher = crypto_create_init_cipher(descriptor_cookie, 0); - declen = crypto_cipher_decrypt_with_iv(cipher, dec, + + declen = crypto_cipher_decrypt_with_iv(descriptor_cookie, dec, ipos_encrypted_size - CIPHER_IV_LEN - 1, ipos_encrypted + 1, ipos_encrypted_size - 1); - crypto_free_cipher_env(cipher); + if (declen < 0) { log_warn(LD_REND, "Decrypting introduction points failed!"); tor_free(dec); @@ -4926,8 +5013,8 @@ rend_parse_introduction_points(rend_service_descriptor_t *parsed, /* Consider one intro point after the other. */ current_ipo = intro_points_encoded; end_of_intro_points = intro_points_encoded + intro_points_encoded_size; - tokens = smartlist_create(); - parsed->intro_nodes = smartlist_create(); + tokens = smartlist_new(); + parsed->intro_nodes = smartlist_new(); area = memarea_new(); while (!fast_memcmpstart(current_ipo, end_of_intro_points-current_ipo, @@ -4974,7 +5061,7 @@ rend_parse_introduction_points(rend_service_descriptor_t *parsed, info->identity_digest, DIGEST_LEN); /* Parse IP address. */ tok = find_by_keyword(tokens, R_IPO_IP_ADDRESS); - if (tor_addr_from_str(&info->addr, tok->args[0])<0) { + if (tor_addr_parse(&info->addr, tok->args[0])<0) { log_warn(LD_REND, "Could not parse introduction point address."); rend_intro_point_free(intro); goto err; @@ -5048,7 +5135,7 @@ rend_parse_client_keys(strmap_t *parsed_clients, const char *ckstr) memarea_t *area = NULL; if (!ckstr || strlen(ckstr) == 0) return -1; - tokens = smartlist_create(); + tokens = smartlist_new(); /* Begin parsing with first entry, skipping comments or whitespace at the * beginning. */ area = memarea_new(); @@ -5120,9 +5207,9 @@ rend_parse_client_keys(strmap_t *parsed_clients, const char *ckstr) /* The size of descriptor_cookie_tmp needs to be REND_DESC_COOKIE_LEN+2, * because a base64 encoding of length 24 does not fit into 16 bytes in all * cases. */ - if ((base64_decode(descriptor_cookie_tmp, REND_DESC_COOKIE_LEN+2, - tok->args[0], REND_DESC_COOKIE_LEN_BASE64+2+1) - != REND_DESC_COOKIE_LEN)) { + if (base64_decode(descriptor_cookie_tmp, sizeof(descriptor_cookie_tmp), + tok->args[0], strlen(tok->args[0])) + != REND_DESC_COOKIE_LEN) { log_warn(LD_REND, "Descriptor cookie contains illegal characters: " "%s", escaped(tok->args[0])); goto err; diff --git a/src/or/routerparse.h b/src/or/routerparse.h index 527de5dc8b..c6382a7f6b 100644 --- a/src/or/routerparse.h +++ b/src/or/routerparse.h @@ -1,7 +1,7 @@ /* Copyright (c) 2001 Matej Pfajfar. * Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** @@ -19,12 +19,12 @@ int router_get_networkstatus_v2_hash(const char *s, char *digest); int router_get_networkstatus_v3_hash(const char *s, char *digest, digest_algorithm_t algorithm); int router_get_networkstatus_v3_hashes(const char *s, digests_t *digests); -int router_get_extrainfo_hash(const char *s, char *digest); +int router_get_extrainfo_hash(const char *s, size_t s_len, char *digest); #define DIROBJ_MAX_SIG_LEN 256 int router_append_dirobj_signature(char *buf, size_t buf_len, const char *digest, size_t digest_len, - crypto_pk_env_t *private_key); + crypto_pk_t *private_key); int router_parse_list_from_string(const char **s, const char *eos, smartlist_t *dest, saved_location_t saved_location, @@ -44,8 +44,9 @@ addr_policy_t *router_parse_addr_policy_item_from_string(const char *s, int assume_action); version_status_t tor_version_is_obsolete(const char *myversion, const char *versionlist); -int tor_version_parse(const char *s, tor_version_t *out); +int tor_version_supports_microdescriptors(const char *platform); int tor_version_as_new_as(const char *platform, const char *cutoff); +int tor_version_parse(const char *s, tor_version_t *out); int tor_version_compare(tor_version_t *a, tor_version_t *b); int tor_version_same_series(tor_version_t *a, tor_version_t *b); void sort_version_list(smartlist_t *lst, int remove_duplicates); diff --git a/src/or/status.c b/src/or/status.c new file mode 100644 index 0000000000..04cd96eed5 --- /dev/null +++ b/src/or/status.c @@ -0,0 +1,115 @@ +/* Copyright (c) 2010-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file status.c + * \brief Keep status information and log the heartbeat messages. + **/ + +#include "or.h" +#include "config.h" +#include "status.h" +#include "nodelist.h" +#include "router.h" +#include "circuitlist.h" +#include "main.h" + +/** Return the total number of circuits. */ +static int +count_circuits(void) +{ + circuit_t *circ; + int nr=0; + + for (circ = _circuit_get_global_list(); circ; circ = circ->next) + nr++; + + return nr; +} + +/** Take seconds <b>secs</b> and return a newly allocated human-readable + * uptime string */ +static char * +secs_to_uptime(long secs) +{ + long int days = secs / 86400; + int hours = (int)((secs - (days * 86400)) / 3600); + int minutes = (int)((secs - (days * 86400) - (hours * 3600)) / 60); + char *uptime_string = NULL; + + switch (days) { + case 0: + tor_asprintf(&uptime_string, "%d:%02d hours", hours, minutes); + break; + case 1: + tor_asprintf(&uptime_string, "%ld day %d:%02d hours", + days, hours, minutes); + break; + default: + tor_asprintf(&uptime_string, "%ld days %d:%02d hours", + days, hours, minutes); + break; + } + + return uptime_string; +} + +/** Take <b>bytes</b> and returns a newly allocated human-readable usage + * string. */ +static char * +bytes_to_usage(uint64_t bytes) +{ + char *bw_string = NULL; + + if (bytes < (1<<20)) { /* Less than a megabyte. */ + tor_asprintf(&bw_string, U64_FORMAT" kB", U64_PRINTF_ARG(bytes>>10)); + } else if (bytes < (1<<30)) { /* Megabytes. Let's add some precision. */ + double bw = U64_TO_DBL(bytes); + tor_asprintf(&bw_string, "%.2f MB", bw/(1<<20)); + } else { /* Gigabytes. */ + double bw = U64_TO_DBL(bytes); + tor_asprintf(&bw_string, "%.2f GB", bw/(1<<30)); + } + + return bw_string; +} + +/** Log a "heartbeat" message describing Tor's status and history so that the + * user can know that there is indeed a running Tor. Return 0 on success and + * -1 on failure. */ +int +log_heartbeat(time_t now) +{ + char *bw_sent = NULL; + char *bw_rcvd = NULL; + char *uptime = NULL; + const routerinfo_t *me; + + const or_options_t *options = get_options(); + (void)now; + + if (public_server_mode(options)) { + /* Let's check if we are in the current cached consensus. */ + if (!(me = router_get_my_routerinfo())) + return -1; /* Something stinks, we won't even attempt this. */ + else + if (!node_get_by_id(me->cache_info.identity_digest)) + log_fn(LOG_NOTICE, LD_HEARTBEAT, "Heartbeat: It seems like we are not " + "in the cached consensus."); + } + + uptime = secs_to_uptime(get_uptime()); + bw_rcvd = bytes_to_usage(get_bytes_read()); + bw_sent = bytes_to_usage(get_bytes_written()); + + log_fn(LOG_NOTICE, LD_HEARTBEAT, "Heartbeat: Tor's uptime is %s, with %d " + "circuits open. I've sent %s and received %s.", + uptime, count_circuits(),bw_sent,bw_rcvd); + + tor_free(uptime); + tor_free(bw_sent); + tor_free(bw_rcvd); + + return 0; +} + diff --git a/src/or/status.h b/src/or/status.h new file mode 100644 index 0000000000..189ac789e1 --- /dev/null +++ b/src/or/status.h @@ -0,0 +1,10 @@ +/* Copyright (c) 2010-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#ifndef _TOR_STATUS_H +#define _TOR_STATUS_H + +int log_heartbeat(time_t now); + +#endif + diff --git a/src/or/tor_main.c b/src/or/tor_main.c index 1ce14ab768..2f4922317d 100644 --- a/src/or/tor_main.c +++ b/src/or/tor_main.c @@ -1,6 +1,6 @@ /* Copyright 2001-2004 Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** String describing which Tor subversion repository version the source was diff --git a/src/or/transports.c b/src/or/transports.c new file mode 100644 index 0000000000..4ba239562a --- /dev/null +++ b/src/or/transports.c @@ -0,0 +1,1221 @@ +/* Copyright (c) 2011-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file transports.c + * \brief Pluggable Transports related code. + * + * \details + * Each managed proxy is represented by a <b>managed_proxy_t</b>. + * Each managed proxy can support multiple transports. + * Each managed proxy gets configured through a multistep process. + * + * ::managed_proxy_list contains all the managed proxies this tor + * instance is supporting. + * In the ::managed_proxy_list there are ::unconfigured_proxies_n + * managed proxies that are still unconfigured. + * + * In every run_scheduled_event() tick, we attempt to launch and then + * configure the unconfiged managed proxies, using the configuration + * protocol defined in the 180_pluggable_transport.txt proposal. A + * managed proxy might need several ticks to get fully configured. + * + * When a managed proxy is fully configured, we register all its + * transports to the circuitbuild.c subsystem. At that point the + * transports are owned by the circuitbuild.c subsystem. + * + * When a managed proxy fails to follow the 180 configuration + * protocol, it gets marked as broken and gets destroyed. + * + * <b>In a little more detail:</b> + * + * While we are serially parsing torrc, we store all the transports + * that a proxy should spawn in its <em>transports_to_launch</em> + * element. + * + * When we finish reading the torrc, we spawn the managed proxy and + * expect {S,C}METHOD lines from its output. We add transports + * described by METHOD lines to its <em>transports</em> element, as + * transport_t structs. + * + * When the managed proxy stops spitting METHOD lines (signified by a + * '{S,C}METHODS DONE' message) we register all the transports + * collected to the circuitbuild.c subsystem. At this point, the + * pointers to transport_t can be transformed into dangling pointers + * at any point by the circuitbuild.c subsystem, and so we replace all + * transport_t pointers with strings describing the transport names. + * We can still go from a transport name to a transport_t using the + * fact that each transport name uniquely identifies a transport_t. + * + * <b>In even more detail, this is what happens when a SIGHUP + * occurs:</b> + * + * We immediately destroy all unconfigured proxies (We shouldn't have + * unconfigured proxies in the first place, except when SIGHUP rings + * immediately after tor is launched.). + * + * We mark all managed proxies and transports to signify that they + * must be removed if they don't contribute by the new torrc + * (we mark using the <b>marked_for_removal</b> element). + * We also mark all managed proxies to signify that they might need to + * be restarted so that they end up supporting all the transports the + * new torrc wants them to support (using the <b>got_hup</b> element). + * We also clear their <b>transports_to_launch</b> list so that we can + * put there the transports we need to launch according to the new + * torrc. + * + * We then start parsing torrc again. + * + * Everytime we encounter a transport line using a known pre-SIGHUP + * managed proxy, we cleanse that proxy from the removal mark. + * We also mark it as unconfigured so that on the next scheduled + * events tick, we investigate whether we need to restart the proxy + * so that it also spawns the new transports. + * If the post-SIGHUP <b>transports_to_launch</b> list is identical to + * the pre-SIGHUP one, it means that no changes were introduced to + * this proxy during the SIGHUP and no restart has to take place. + * + * During the post-SIGHUP torrc parsing, we unmark all transports + * spawned by managed proxies that we find in our torrc. + * We do that so that if we don't need to restart a managed proxy, we + * can continue using its old transports normally. + * If we end up restarting the proxy, we destroy and unregister all + * old transports from the circuitbuild.c subsystem. + **/ + +#define PT_PRIVATE +#include "or.h" +#include "config.h" +#include "circuitbuild.h" +#include "transports.h" +#include "util.h" +#include "router.h" + +static process_environment_t * +create_managed_proxy_environment(const managed_proxy_t *mp); + +static INLINE int proxy_configuration_finished(const managed_proxy_t *mp); + +static void managed_proxy_destroy(managed_proxy_t *mp, + int also_terminate_process); + +static void handle_finished_proxy(managed_proxy_t *mp); +static void configure_proxy(managed_proxy_t *mp); + +static void parse_method_error(const char *line, int is_server_method); +#define parse_server_method_error(l) parse_method_error(l, 1) +#define parse_client_method_error(l) parse_method_error(l, 0) + +static INLINE void free_execve_args(char **arg); + +/** Managed proxy protocol strings */ +#define PROTO_ENV_ERROR "ENV-ERROR" +#define PROTO_NEG_SUCCESS "VERSION" +#define PROTO_NEG_FAIL "VERSION-ERROR no-version" +#define PROTO_CMETHOD "CMETHOD" +#define PROTO_SMETHOD "SMETHOD" +#define PROTO_CMETHOD_ERROR "CMETHOD-ERROR" +#define PROTO_SMETHOD_ERROR "SMETHOD-ERROR" +#define PROTO_CMETHODS_DONE "CMETHODS DONE" +#define PROTO_SMETHODS_DONE "SMETHODS DONE" + +/** Number of environment variables for managed proxy clients/servers. */ +#define ENVIRON_SIZE_CLIENT 3 +#define ENVIRON_SIZE_SERVER 7 /* XXX known to be too high, but that's ok */ + +/** The first and only supported - at the moment - configuration + protocol version. */ +#define PROTO_VERSION_ONE 1 + +/** List of unconfigured managed proxies. */ +static smartlist_t *managed_proxy_list = NULL; +/** Number of still unconfigured proxies. */ +static int unconfigured_proxies_n = 0; +/** Boolean: True iff we might need to restart some proxies. */ +static int check_if_restarts_needed = 0; + +/** Return true if there are still unconfigured managed proxies, or proxies + * that need restarting. */ +int +pt_proxies_configuration_pending(void) +{ + return unconfigured_proxies_n || check_if_restarts_needed; +} + +/** Assert that the unconfigured_proxies_n value correctly matches the number + * of proxies in a state other than PT_PROTO_COMPLETE. */ +static void +assert_unconfigured_count_ok(void) +{ + int n_completed = 0; + if (!managed_proxy_list) { + tor_assert(unconfigured_proxies_n == 0); + return; + } + + SMARTLIST_FOREACH(managed_proxy_list, managed_proxy_t *, mp, { + if (mp->conf_state == PT_PROTO_COMPLETED) + ++n_completed; + }); + + tor_assert(n_completed + unconfigured_proxies_n == + smartlist_len(managed_proxy_list)); +} + +/** Return true if <b>mp</b> has the same argv as <b>proxy_argv</b> */ +static int +managed_proxy_has_argv(const managed_proxy_t *mp, char **proxy_argv) +{ + char **tmp1=proxy_argv; + char **tmp2=mp->argv; + + tor_assert(tmp1); + tor_assert(tmp2); + + while (*tmp1 && *tmp2) { + if (strcmp(*tmp1++, *tmp2++)) + return 0; + } + + if (!*tmp1 && !*tmp2) + return 1; + + return 0; +} + +/** Return a managed proxy with the same argv as <b>proxy_argv</b>. + * If no such managed proxy exists, return NULL. */ +static managed_proxy_t * +get_managed_proxy_by_argv_and_type(char **proxy_argv, int is_server) +{ + if (!managed_proxy_list) + return NULL; + + SMARTLIST_FOREACH_BEGIN(managed_proxy_list, managed_proxy_t *, mp) { + if (managed_proxy_has_argv(mp, proxy_argv) && + mp->is_server == is_server) + return mp; + } SMARTLIST_FOREACH_END(mp); + + return NULL; +} + +/** Add <b>transport</b> to managed proxy <b>mp</b>. */ +static void +add_transport_to_proxy(const char *transport, managed_proxy_t *mp) +{ + tor_assert(mp->transports_to_launch); + if (!smartlist_string_isin(mp->transports_to_launch, transport)) + smartlist_add(mp->transports_to_launch, tor_strdup(transport)); +} + +/** Called when a SIGHUP occurs. Returns true if managed proxy + * <b>mp</b> needs to be restarted after the SIGHUP, based on the new + * torrc. */ +static int +proxy_needs_restart(const managed_proxy_t *mp) +{ + /* mp->transport_to_launch is populated with the names of the + transports that must be launched *after* the SIGHUP. + mp->transports is populated with the names of the transports that + were launched *before* the SIGHUP. + + If the two lists contain the same strings, we don't need to + restart the proxy, since it already does what we want. */ + + tor_assert(smartlist_len(mp->transports_to_launch) > 0); + tor_assert(mp->conf_state == PT_PROTO_COMPLETED); + + if (smartlist_len(mp->transports_to_launch) != smartlist_len(mp->transports)) + goto needs_restart; + + SMARTLIST_FOREACH_BEGIN(mp->transports_to_launch, char *, t_t_l) { + if (!smartlist_string_isin(mp->transports, t_t_l)) + goto needs_restart; + + } SMARTLIST_FOREACH_END(t_t_l); + + return 0; + + needs_restart: + return 1; +} + +/** Managed proxy <b>mp</b> must be restarted. Do all the necessary + * preparations and then flag its state so that it will be relaunched + * in the next tick. */ +static void +proxy_prepare_for_restart(managed_proxy_t *mp) +{ + transport_t *t_tmp = NULL; + + tor_assert(mp->conf_state == PT_PROTO_COMPLETED); + + /* destroy the process handle and terminate the process. */ + tor_process_handle_destroy(mp->process_handle, 1); + mp->process_handle = NULL; + + /* destroy all its old transports. we no longer use them. */ + SMARTLIST_FOREACH_BEGIN(mp->transports, const char *, t_name) { + t_tmp = transport_get_by_name(t_name); + if (t_tmp) + t_tmp->marked_for_removal = 1; + } SMARTLIST_FOREACH_END(t_name); + sweep_transport_list(); + + /* free the transport names in mp->transports */ + SMARTLIST_FOREACH(mp->transports, char *, t_name, tor_free(t_name)); + smartlist_clear(mp->transports); + + /* flag it as an infant proxy so that it gets launched on next tick */ + mp->conf_state = PT_PROTO_INFANT; + unconfigured_proxies_n++; +} + +/** Launch managed proxy <b>mp</b>. */ +static int +launch_managed_proxy(managed_proxy_t *mp) +{ + int retval; + + process_environment_t *env = create_managed_proxy_environment(mp); + +#ifdef _WIN32 + /* Passing NULL as lpApplicationName makes Windows search for the .exe */ + retval = tor_spawn_background(NULL, + (const char **)mp->argv, + env, + &mp->process_handle); +#else + retval = tor_spawn_background(mp->argv[0], + (const char **)mp->argv, + env, + &mp->process_handle); +#endif + + process_environment_free(env); + + if (retval == PROCESS_STATUS_ERROR) { + log_warn(LD_GENERAL, "Managed proxy at '%s' failed at launch.", + mp->argv[0]); + return -1; + } + + log_info(LD_CONFIG, "Managed proxy at '%s' has spawned with PID '%d'.", + mp->argv[0], tor_process_get_pid(mp->process_handle)); + + mp->conf_state = PT_PROTO_LAUNCHED; + + return 0; +} + +/** Check if any of the managed proxies we are currently trying to + * configure have anything new to say. This is called from + * run_scheduled_events(). */ +void +pt_configure_remaining_proxies(void) +{ + smartlist_t *tmp = smartlist_new(); + + log_debug(LD_CONFIG, "Configuring remaining managed proxies (%d)!", + unconfigured_proxies_n); + + /* Iterate over tmp, not managed_proxy_list, since configure_proxy can + * remove elements from managed_proxy_list. */ + smartlist_add_all(tmp, managed_proxy_list); + + assert_unconfigured_count_ok(); + + SMARTLIST_FOREACH_BEGIN(tmp, managed_proxy_t *, mp) { + tor_assert(mp->conf_state != PT_PROTO_BROKEN || + mp->conf_state != PT_PROTO_FAILED_LAUNCH); + + if (mp->got_hup) { + mp->got_hup = 0; + + /* This proxy is marked by a SIGHUP. Check whether we need to + restart it. */ + if (proxy_needs_restart(mp)) { + log_info(LD_GENERAL, "Preparing managed proxy '%s' for restart.", + mp->argv[0]); + proxy_prepare_for_restart(mp); + } else { /* it doesn't need to be restarted. */ + log_info(LD_GENERAL, "Nothing changed for managed proxy '%s' after " + "HUP: not restarting.", mp->argv[0]); + } + + continue; + } + + /* If the proxy is not fully configured, try to configure it + futher. */ + if (!proxy_configuration_finished(mp)) + configure_proxy(mp); + + } SMARTLIST_FOREACH_END(mp); + + smartlist_free(tmp); + check_if_restarts_needed = 0; + assert_unconfigured_count_ok(); +} + +#ifdef _WIN32 + +/** Attempt to continue configuring managed proxy <b>mp</b>. */ +static void +configure_proxy(managed_proxy_t *mp) +{ + int pos; + char stdout_buf[200]; + smartlist_t *lines = NULL; + + /* if we haven't launched the proxy yet, do it now */ + if (mp->conf_state == PT_PROTO_INFANT) { + if (launch_managed_proxy(mp) < 0) { /* launch fail */ + mp->conf_state = PT_PROTO_FAILED_LAUNCH; + handle_finished_proxy(mp); + } + return; + } + + tor_assert(mp->conf_state != PT_PROTO_INFANT); + tor_assert(mp->process_handle); + + pos = tor_read_all_handle(tor_process_get_stdout_pipe(mp->process_handle), + stdout_buf, sizeof(stdout_buf) - 1, NULL); + if (pos < 0) { + log_notice(LD_GENERAL, "Failed to read data from managed proxy '%s'.", + mp->argv[0]); + mp->conf_state = PT_PROTO_BROKEN; + goto done; + } + + if (pos == 0) /* proxy has nothing interesting to say. */ + return; + + /* End with a null even if there isn't a \r\n at the end */ + /* TODO: What if this is a partial line? */ + stdout_buf[pos] = '\0'; + + /* Split up the buffer */ + lines = smartlist_new(); + tor_split_lines(lines, stdout_buf, pos); + + /* Handle lines. */ + SMARTLIST_FOREACH_BEGIN(lines, const char *, line) { + handle_proxy_line(line, mp); + if (proxy_configuration_finished(mp)) + goto done; + } SMARTLIST_FOREACH_END(line); + + done: + /* if the proxy finished configuring, exit the loop. */ + if (proxy_configuration_finished(mp)) + handle_finished_proxy(mp); + + if (lines) + smartlist_free(lines); +} + +#else /* _WIN32 */ + +/** Attempt to continue configuring managed proxy <b>mp</b>. */ +static void +configure_proxy(managed_proxy_t *mp) +{ + enum stream_status r; + char stdout_buf[200]; + + /* if we haven't launched the proxy yet, do it now */ + if (mp->conf_state == PT_PROTO_INFANT) { + if (launch_managed_proxy(mp) < 0) { /* launch fail */ + mp->conf_state = PT_PROTO_FAILED_LAUNCH; + handle_finished_proxy(mp); + } + return; + } + + tor_assert(mp->conf_state != PT_PROTO_INFANT); + tor_assert(mp->process_handle); + + while (1) { + r = get_string_from_pipe(tor_process_get_stdout_pipe(mp->process_handle), + stdout_buf, sizeof(stdout_buf) - 1); + + if (r == IO_STREAM_OKAY) { /* got a line; handle it! */ + handle_proxy_line((const char *)stdout_buf, mp); + } else if (r == IO_STREAM_EAGAIN) { /* check back later */ + return; + } else if (r == IO_STREAM_CLOSED || r == IO_STREAM_TERM) { /* snap! */ + log_warn(LD_GENERAL, "Our communication channel with the managed proxy " + "'%s' closed. Most probably application stopped running.", + mp->argv[0]); + mp->conf_state = PT_PROTO_BROKEN; + } else { /* unknown stream status */ + log_warn(LD_BUG, "Unknown stream status '%d' while configuring managed " + "proxy '%s'.", (int)r, mp->argv[0]); + } + + /* if the proxy finished configuring, exit the loop. */ + if (proxy_configuration_finished(mp)) { + handle_finished_proxy(mp); + return; + } + } +} + +#endif /* _WIN32 */ + +/** Register server managed proxy <b>mp</b> transports to state */ +static void +register_server_proxy(managed_proxy_t *mp) +{ + /* After we register this proxy's transports, we switch its + mp->transports to a list containing strings of its transport + names. (See transports.h) */ + smartlist_t *sm_tmp = smartlist_new(); + + tor_assert(mp->conf_state != PT_PROTO_COMPLETED); + SMARTLIST_FOREACH_BEGIN(mp->transports, transport_t *, t) { + save_transport_to_state(t->name, &t->addr, t->port); + log_notice(LD_GENERAL, "Registered server transport '%s' at '%s:%d'", + t->name, fmt_addr(&t->addr), (int)t->port); + smartlist_add(sm_tmp, tor_strdup(t->name)); + } SMARTLIST_FOREACH_END(t); + + /* Since server proxies don't register their transports in the + circuitbuild.c subsystem, it's our duty to free them when we + switch mp->transports to strings. */ + SMARTLIST_FOREACH(mp->transports, transport_t *, t, transport_free(t)); + smartlist_free(mp->transports); + + mp->transports = sm_tmp; +} + +/** Register all the transports supported by client managed proxy + * <b>mp</b> to the bridge subsystem. */ +static void +register_client_proxy(managed_proxy_t *mp) +{ + int r; + /* After we register this proxy's transports, we switch its + mp->transports to a list containing strings of its transport + names. (See transports.h) */ + smartlist_t *sm_tmp = smartlist_new(); + + tor_assert(mp->conf_state != PT_PROTO_COMPLETED); + SMARTLIST_FOREACH_BEGIN(mp->transports, transport_t *, t) { + r = transport_add(t); + switch (r) { + case -1: + log_notice(LD_GENERAL, "Could not add transport %s. Skipping.", t->name); + transport_free(t); + break; + case 0: + log_info(LD_GENERAL, "Succesfully registered transport %s", t->name); + smartlist_add(sm_tmp, tor_strdup(t->name)); + break; + case 1: + log_info(LD_GENERAL, "Succesfully registered transport %s", t->name); + smartlist_add(sm_tmp, tor_strdup(t->name)); + transport_free(t); + break; + } + } SMARTLIST_FOREACH_END(t); + + smartlist_free(mp->transports); + mp->transports = sm_tmp; +} + +/** Register the transports of managed proxy <b>mp</b>. */ +static INLINE void +register_proxy(managed_proxy_t *mp) +{ + if (mp->is_server) + register_server_proxy(mp); + else + register_client_proxy(mp); +} + +/** Free memory allocated by managed proxy <b>mp</b>. */ +static void +managed_proxy_destroy(managed_proxy_t *mp, + int also_terminate_process) +{ + if (mp->conf_state != PT_PROTO_COMPLETED) + SMARTLIST_FOREACH(mp->transports, transport_t *, t, transport_free(t)); + else + SMARTLIST_FOREACH(mp->transports, char *, t_name, tor_free(t_name)); + + /* free the transports smartlist */ + smartlist_free(mp->transports); + + /* free the transports_to_launch smartlist */ + SMARTLIST_FOREACH(mp->transports_to_launch, char *, t, tor_free(t)); + smartlist_free(mp->transports_to_launch); + + /* remove it from the list of managed proxies */ + smartlist_remove(managed_proxy_list, mp); + + /* free the argv */ + free_execve_args(mp->argv); + + tor_process_handle_destroy(mp->process_handle, also_terminate_process); + mp->process_handle = NULL; + + tor_free(mp); +} + +/** Handle a configured or broken managed proxy <b>mp</b>. */ +static void +handle_finished_proxy(managed_proxy_t *mp) +{ + switch (mp->conf_state) { + case PT_PROTO_BROKEN: /* if broken: */ + managed_proxy_destroy(mp, 1); /* annihilate it. */ + break; + case PT_PROTO_FAILED_LAUNCH: /* if it failed before launching: */ + managed_proxy_destroy(mp, 0); /* destroy it but don't terminate */ + break; + case PT_PROTO_CONFIGURED: /* if configured correctly: */ + register_proxy(mp); /* register its transports */ + mp->conf_state = PT_PROTO_COMPLETED; /* and mark it as completed. */ + break; + case PT_PROTO_INFANT: + case PT_PROTO_LAUNCHED: + case PT_PROTO_ACCEPTING_METHODS: + case PT_PROTO_COMPLETED: + default: + log_warn(LD_CONFIG, "Unexpected state '%d' of managed proxy '%s'.", + (int)mp->conf_state, mp->argv[0]); + tor_assert(0); + } + + unconfigured_proxies_n--; + tor_assert(unconfigured_proxies_n >= 0); +} + +/** Return true if the configuration of the managed proxy <b>mp</b> is + finished. */ +static INLINE int +proxy_configuration_finished(const managed_proxy_t *mp) +{ + return (mp->conf_state == PT_PROTO_CONFIGURED || + mp->conf_state == PT_PROTO_BROKEN || + mp->conf_state == PT_PROTO_FAILED_LAUNCH); +} + +/** This function is called when a proxy sends an {S,C}METHODS DONE message. */ +static void +handle_methods_done(const managed_proxy_t *mp) +{ + tor_assert(mp->transports); + + if (smartlist_len(mp->transports) == 0) + log_notice(LD_GENERAL, "Managed proxy '%s' was spawned successfully, " + "but it didn't launch any pluggable transport listeners!", + mp->argv[0]); + + log_info(LD_CONFIG, "%s managed proxy '%s' configuration completed!", + mp->is_server ? "Server" : "Client", + mp->argv[0]); +} + +/** Handle a configuration protocol <b>line</b> received from a + * managed proxy <b>mp</b>. */ +void +handle_proxy_line(const char *line, managed_proxy_t *mp) +{ + log_info(LD_GENERAL, "Got a line from managed proxy '%s': (%s)", + mp->argv[0], line); + + if (!strcmpstart(line, PROTO_ENV_ERROR)) { + if (mp->conf_state != PT_PROTO_LAUNCHED) + goto err; + + parse_env_error(line); + goto err; + } else if (!strcmpstart(line, PROTO_NEG_FAIL)) { + if (mp->conf_state != PT_PROTO_LAUNCHED) + goto err; + + log_warn(LD_CONFIG, "Managed proxy could not pick a " + "configuration protocol version."); + goto err; + } else if (!strcmpstart(line, PROTO_NEG_SUCCESS)) { + if (mp->conf_state != PT_PROTO_LAUNCHED) + goto err; + + if (parse_version(line,mp) < 0) + goto err; + + tor_assert(mp->conf_protocol != 0); + mp->conf_state = PT_PROTO_ACCEPTING_METHODS; + return; + } else if (!strcmpstart(line, PROTO_CMETHODS_DONE)) { + if (mp->conf_state != PT_PROTO_ACCEPTING_METHODS) + goto err; + + handle_methods_done(mp); + + mp->conf_state = PT_PROTO_CONFIGURED; + return; + } else if (!strcmpstart(line, PROTO_SMETHODS_DONE)) { + if (mp->conf_state != PT_PROTO_ACCEPTING_METHODS) + goto err; + + handle_methods_done(mp); + + mp->conf_state = PT_PROTO_CONFIGURED; + return; + } else if (!strcmpstart(line, PROTO_CMETHOD_ERROR)) { + if (mp->conf_state != PT_PROTO_ACCEPTING_METHODS) + goto err; + + parse_client_method_error(line); + goto err; + } else if (!strcmpstart(line, PROTO_SMETHOD_ERROR)) { + if (mp->conf_state != PT_PROTO_ACCEPTING_METHODS) + goto err; + + parse_server_method_error(line); + goto err; + } else if (!strcmpstart(line, PROTO_CMETHOD)) { + if (mp->conf_state != PT_PROTO_ACCEPTING_METHODS) + goto err; + + if (parse_cmethod_line(line, mp) < 0) + goto err; + + return; + } else if (!strcmpstart(line, PROTO_SMETHOD)) { + if (mp->conf_state != PT_PROTO_ACCEPTING_METHODS) + goto err; + + if (parse_smethod_line(line, mp) < 0) + goto err; + + return; + } else if (!strcmpstart(line, SPAWN_ERROR_MESSAGE)) { + /* managed proxy launch failed: parse error message to learn why. */ + int retval, child_state, saved_errno; + retval = tor_sscanf(line, SPAWN_ERROR_MESSAGE "%x/%x", + &child_state, &saved_errno); + if (retval == 2) { + log_warn(LD_GENERAL, + "Could not launch managed proxy executable at '%s' ('%s').", + mp->argv[0], strerror(saved_errno)); + } else { /* failed to parse error message */ + log_warn(LD_GENERAL,"Could not launch managed proxy executable at '%s'.", + mp->argv[0]); + } + + mp->conf_state = PT_PROTO_FAILED_LAUNCH; + return; + } + + log_notice(LD_GENERAL, "Unknown line received by managed proxy (%s).", line); + return; + + err: + mp->conf_state = PT_PROTO_BROKEN; + log_warn(LD_CONFIG, "Managed proxy at '%s' failed the configuration protocol" + " and will be destroyed.", mp->argv[0]); +} + +/** Parses an ENV-ERROR <b>line</b> and warns the user accordingly. */ +void +parse_env_error(const char *line) +{ + /* (Length of the protocol string) plus (a space) and (the first char of + the error message) */ + if (strlen(line) < (strlen(PROTO_ENV_ERROR) + 2)) + log_notice(LD_CONFIG, "Managed proxy sent us an %s without an error " + "message.", PROTO_ENV_ERROR); + + log_warn(LD_CONFIG, "Managed proxy couldn't understand the " + "pluggable transport environment variables. (%s)", + line+strlen(PROTO_ENV_ERROR)+1); +} + +/** Handles a VERSION <b>line</b>. Updates the configuration protocol + * version in <b>mp</b>. */ +int +parse_version(const char *line, managed_proxy_t *mp) +{ + if (strlen(line) < (strlen(PROTO_NEG_SUCCESS) + 2)) { + log_warn(LD_CONFIG, "Managed proxy sent us malformed %s line.", + PROTO_NEG_SUCCESS); + return -1; + } + + if (strcmp("1", line+strlen(PROTO_NEG_SUCCESS)+1)) { /* hardcoded temp */ + log_warn(LD_CONFIG, "Managed proxy tried to negotiate on version '%s'. " + "We only support version '1'", line+strlen(PROTO_NEG_SUCCESS)+1); + return -1; + } + + mp->conf_protocol = PROTO_VERSION_ONE; /* temp. till more versions appear */ + return 0; +} + +/** Parses {C,S}METHOD-ERROR <b>line</b> and warns the user + * accordingly. If <b>is_server</b> it is an SMETHOD-ERROR, + * otherwise it is a CMETHOD-ERROR. */ +static void +parse_method_error(const char *line, int is_server) +{ + const char* error = is_server ? + PROTO_SMETHOD_ERROR : PROTO_CMETHOD_ERROR; + + /* (Length of the protocol string) plus (a space) and (the first char of + the error message) */ + if (strlen(line) < (strlen(error) + 2)) + log_warn(LD_CONFIG, "Managed proxy sent us an %s without an error " + "message.", error); + + log_warn(LD_CONFIG, "%s managed proxy encountered a method error. (%s)", + is_server ? "Server" : "Client", + line+strlen(error)+1); +} + +/** Parses an SMETHOD <b>line</b> and if well-formed it registers the + * new transport in <b>mp</b>. */ +int +parse_smethod_line(const char *line, managed_proxy_t *mp) +{ + int r; + smartlist_t *items = NULL; + + char *method_name=NULL; + + char *addrport=NULL; + tor_addr_t tor_addr; + char *address=NULL; + uint16_t port = 0; + + transport_t *transport=NULL; + + items = smartlist_new(); + smartlist_split_string(items, line, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); + if (smartlist_len(items) < 3) { + log_warn(LD_CONFIG, "Server managed proxy sent us a SMETHOD line " + "with too few arguments."); + goto err; + } + + tor_assert(!strcmp(smartlist_get(items,0),PROTO_SMETHOD)); + + method_name = smartlist_get(items,1); + if (!string_is_C_identifier(method_name)) { + log_warn(LD_CONFIG, "Transport name is not a C identifier (%s).", + method_name); + goto err; + } + + addrport = smartlist_get(items, 2); + if (tor_addr_port_split(LOG_PROTOCOL_WARN, addrport, &address, &port)<0) { + log_warn(LD_CONFIG, "Error parsing transport " + "address '%s'", addrport); + goto err; + } + + if (!port) { + log_warn(LD_CONFIG, + "Transport address '%s' has no port.", addrport); + goto err; + } + + if (tor_addr_parse(&tor_addr, address) < 0) { + log_warn(LD_CONFIG, "Error parsing transport address '%s'", address); + goto err; + } + + transport = transport_new(&tor_addr, port, method_name, PROXY_NONE); + if (!transport) + goto err; + + smartlist_add(mp->transports, transport); + + /* For now, notify the user so that he knows where the server + transport is listening. */ + log_info(LD_CONFIG, "Server transport %s at %s:%d.", + method_name, address, (int)port); + + r=0; + goto done; + + err: + r = -1; + + done: + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + tor_free(address); + return r; +} + +/** Parses a CMETHOD <b>line</b>, and if well-formed it registers + * the new transport in <b>mp</b>. */ +int +parse_cmethod_line(const char *line, managed_proxy_t *mp) +{ + int r; + smartlist_t *items = NULL; + + char *method_name=NULL; + + char *socks_ver_str=NULL; + int socks_ver=PROXY_NONE; + + char *addrport=NULL; + tor_addr_t tor_addr; + char *address=NULL; + uint16_t port = 0; + + transport_t *transport=NULL; + + items = smartlist_new(); + smartlist_split_string(items, line, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); + if (smartlist_len(items) < 4) { + log_warn(LD_CONFIG, "Client managed proxy sent us a CMETHOD line " + "with too few arguments."); + goto err; + } + + tor_assert(!strcmp(smartlist_get(items,0),PROTO_CMETHOD)); + + method_name = smartlist_get(items,1); + if (!string_is_C_identifier(method_name)) { + log_warn(LD_CONFIG, "Transport name is not a C identifier (%s).", + method_name); + goto err; + } + + socks_ver_str = smartlist_get(items,2); + + if (!strcmp(socks_ver_str,"socks4")) { + socks_ver = PROXY_SOCKS4; + } else if (!strcmp(socks_ver_str,"socks5")) { + socks_ver = PROXY_SOCKS5; + } else { + log_warn(LD_CONFIG, "Client managed proxy sent us a proxy protocol " + "we don't recognize. (%s)", socks_ver_str); + goto err; + } + + addrport = smartlist_get(items, 3); + if (tor_addr_port_split(LOG_PROTOCOL_WARN, addrport, &address, &port)<0) { + log_warn(LD_CONFIG, "Error parsing transport " + "address '%s'", addrport); + goto err; + } + + if (!port) { + log_warn(LD_CONFIG, + "Transport address '%s' has no port.", addrport); + goto err; + } + + if (tor_addr_parse(&tor_addr, address) < 0) { + log_warn(LD_CONFIG, "Error parsing transport address '%s'", address); + goto err; + } + + transport = transport_new(&tor_addr, port, method_name, socks_ver); + if (!transport) + goto err; + + smartlist_add(mp->transports, transport); + + log_info(LD_CONFIG, "Transport %s at %s:%d with SOCKS %d. " + "Attached to managed proxy.", + method_name, address, (int)port, socks_ver); + + r=0; + goto done; + + err: + r = -1; + + done: + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + tor_free(address); + return r; +} + +/** Return the string that tor should place in TOR_PT_SERVER_BINDADDR + * while configuring the server managed proxy in <b>mp</b>. The + * string is stored in the heap, and it's the the responsibility of + * the caller to deallocate it after its use. */ +static char * +get_bindaddr_for_server_proxy(const managed_proxy_t *mp) +{ + char *bindaddr_result = NULL; + char *bindaddr_tmp = NULL; + smartlist_t *string_tmp = smartlist_new(); + + tor_assert(mp->is_server); + + SMARTLIST_FOREACH_BEGIN(mp->transports_to_launch, char *, t) { + bindaddr_tmp = get_stored_bindaddr_for_server_transport(t); + + smartlist_add_asprintf(string_tmp, "%s-%s", t, bindaddr_tmp); + + tor_free(bindaddr_tmp); + } SMARTLIST_FOREACH_END(t); + + bindaddr_result = smartlist_join_strings(string_tmp, ",", 0, NULL); + + SMARTLIST_FOREACH(string_tmp, char *, t, tor_free(t)); + smartlist_free(string_tmp); + + return bindaddr_result; +} + +/** Return a newly allocated process_environment_t * for <b>mp</b>'s + * process. */ +static process_environment_t * +create_managed_proxy_environment(const managed_proxy_t *mp) +{ + /* Environment variables to be added to or set in mp's environment. */ + smartlist_t *envs = smartlist_new(); + /* XXXX The next time someone touches this code, shorten the name of + * set_environment_variable_in_smartlist, add a + * set_env_var_in_smartlist_asprintf function, and get rid of the + * silly extra envs smartlist. */ + + /* The final environment to be passed to mp. */ + smartlist_t *merged_env_vars = get_current_process_environment_variables(); + + process_environment_t *env; + + { + char *state_tmp = get_datadir_fname("pt_state/"); /* XXX temp */ + smartlist_add_asprintf(envs, "TOR_PT_STATE_LOCATION=%s", state_tmp); + tor_free(state_tmp); + } + + smartlist_add(envs, tor_strdup("TOR_PT_MANAGED_TRANSPORT_VER=1")); + + { + char *transports_to_launch = + smartlist_join_strings(mp->transports_to_launch, ",", 0, NULL); + + smartlist_add_asprintf(envs, + mp->is_server ? + "TOR_PT_SERVER_TRANSPORTS=%s" : + "TOR_PT_CLIENT_TRANSPORTS=%s", + transports_to_launch); + + tor_free(transports_to_launch); + } + + if (mp->is_server) { + { + char *orport_tmp = + get_first_listener_addrport_string(CONN_TYPE_OR_LISTENER); + smartlist_add_asprintf(envs, "TOR_PT_ORPORT=%s", orport_tmp); + tor_free(orport_tmp); + } + + { + char *bindaddr_tmp = get_bindaddr_for_server_proxy(mp); + smartlist_add_asprintf(envs, "TOR_PT_SERVER_BINDADDR=%s", bindaddr_tmp); + tor_free(bindaddr_tmp); + } + + /* XXX024 Remove the '=' here once versions of obfsproxy which + * assert that this env var exists are sufficiently dead. + * + * (If we remove this line entirely, some joker will stick this + * variable in Tor's environment and crash PTs that try to parse + * it even when not run in server mode.) */ + smartlist_add(envs, tor_strdup("TOR_PT_EXTENDED_SERVER_PORT=")); + } + + SMARTLIST_FOREACH_BEGIN(envs, const char *, env_var) { + set_environment_variable_in_smartlist(merged_env_vars, env_var, + _tor_free, 1); + } SMARTLIST_FOREACH_END(env_var); + + env = process_environment_make(merged_env_vars); + + smartlist_free(envs); + + SMARTLIST_FOREACH(merged_env_vars, void *, x, tor_free(x)); + smartlist_free(merged_env_vars); + + return env; +} + +/** Create and return a new managed proxy for <b>transport</b> using + * <b>proxy_argv</b>. Also, add it to the global managed proxy list. If + * <b>is_server</b> is true, it's a server managed proxy. Takes ownership of + * <b>proxy_argv</b>. + * + * Requires that proxy_argv have at least one element. */ +static managed_proxy_t * +managed_proxy_create(const smartlist_t *transport_list, + char **proxy_argv, int is_server) +{ + managed_proxy_t *mp = tor_malloc_zero(sizeof(managed_proxy_t)); + mp->conf_state = PT_PROTO_INFANT; + mp->is_server = is_server; + mp->argv = proxy_argv; + mp->transports = smartlist_new(); + + mp->transports_to_launch = smartlist_new(); + SMARTLIST_FOREACH(transport_list, const char *, transport, + add_transport_to_proxy(transport, mp)); + + /* register the managed proxy */ + if (!managed_proxy_list) + managed_proxy_list = smartlist_new(); + smartlist_add(managed_proxy_list, mp); + unconfigured_proxies_n++; + + assert_unconfigured_count_ok(); + + return mp; +} + +/** Register proxy with <b>proxy_argv</b>, supporting transports in + * <b>transport_list</b>, to the managed proxy subsystem. + * If <b>is_server</b> is true, then the proxy is a server proxy. + * + * Takes ownership of proxy_argv. + * + * Requires that proxy_argv be a NULL-terminated array of command-line + * elements, containing at least one element. + **/ +void +pt_kickstart_proxy(const smartlist_t *transport_list, + char **proxy_argv, int is_server) +{ + managed_proxy_t *mp=NULL; + transport_t *old_transport = NULL; + + if (!proxy_argv || !proxy_argv[0]) { + return; + } + + mp = get_managed_proxy_by_argv_and_type(proxy_argv, is_server); + + if (!mp) { /* we haven't seen this proxy before */ + managed_proxy_create(transport_list, proxy_argv, is_server); + + } else { /* known proxy. add its transport to its transport list */ + if (mp->got_hup) { + /* If the managed proxy we found is marked by a SIGHUP, it means + that it's not useless and should be kept. If it's marked for + removal, unmark it and increase the unconfigured proxies so + that we try to restart it if we need to. Afterwards, check if + a transport_t for 'transport' used to exist before the SIGHUP + and make sure it doesn't get deleted because we might reuse + it. */ + if (mp->marked_for_removal) { + mp->marked_for_removal = 0; + check_if_restarts_needed = 1; + } + + SMARTLIST_FOREACH_BEGIN(transport_list, const char *, transport) { + old_transport = transport_get_by_name(transport); + if (old_transport) + old_transport->marked_for_removal = 0; + } SMARTLIST_FOREACH_END(transport); + } + + SMARTLIST_FOREACH(transport_list, const char *, transport, + add_transport_to_proxy(transport, mp)); + free_execve_args(proxy_argv); + } +} + +/** Frees the array of pointers in <b>arg</b> used as arguments to + execve(2). */ +static INLINE void +free_execve_args(char **arg) +{ + char **tmp = arg; + while (*tmp) /* use the fact that the last element of the array is a + NULL pointer to know when to stop freeing */ + _tor_free(*tmp++); + + tor_free(arg); +} + +/** Tor will read its config. + * Prepare the managed proxy list so that proxies not used in the new + * config will shutdown, and proxies that need to spawn different + * transports will do so. */ +void +pt_prepare_proxy_list_for_config_read(void) +{ + if (!managed_proxy_list) + return; + + assert_unconfigured_count_ok(); + SMARTLIST_FOREACH_BEGIN(managed_proxy_list, managed_proxy_t *, mp) { + /* Destroy unconfigured proxies. */ + if (mp->conf_state != PT_PROTO_COMPLETED) { + SMARTLIST_DEL_CURRENT(managed_proxy_list, mp); + managed_proxy_destroy(mp, 1); + unconfigured_proxies_n--; + continue; + } + + tor_assert(mp->conf_state == PT_PROTO_COMPLETED); + + mp->marked_for_removal = 1; + mp->got_hup = 1; + SMARTLIST_FOREACH(mp->transports_to_launch, char *, t, tor_free(t)); + smartlist_clear(mp->transports_to_launch); + } SMARTLIST_FOREACH_END(mp); + + assert_unconfigured_count_ok(); + + tor_assert(unconfigured_proxies_n == 0); +} + +/** The tor config was read. + * Destroy all managed proxies that were marked by a previous call to + * prepare_proxy_list_for_config_read() and are not used by the new + * config. */ +void +sweep_proxy_list(void) +{ + if (!managed_proxy_list) + return; + assert_unconfigured_count_ok(); + SMARTLIST_FOREACH_BEGIN(managed_proxy_list, managed_proxy_t *, mp) { + if (mp->marked_for_removal) { + SMARTLIST_DEL_CURRENT(managed_proxy_list, mp); + managed_proxy_destroy(mp, 1); + } + } SMARTLIST_FOREACH_END(mp); + assert_unconfigured_count_ok(); +} + +/** Release all storage held by the pluggable transports subsystem. */ +void +pt_free_all(void) +{ + if (managed_proxy_list) { + /* If the proxy is in PT_PROTO_COMPLETED, it has registered its + transports and it's the duty of the circuitbuild.c subsystem to + free them. Otherwise, it hasn't registered its transports yet + and we should free them here. */ + SMARTLIST_FOREACH(managed_proxy_list, managed_proxy_t *, mp, { + SMARTLIST_DEL_CURRENT(managed_proxy_list, mp); + managed_proxy_destroy(mp, 1); + }); + + smartlist_free(managed_proxy_list); + managed_proxy_list=NULL; + } +} + diff --git a/src/or/transports.h b/src/or/transports.h new file mode 100644 index 0000000000..02f159a5d6 --- /dev/null +++ b/src/or/transports.h @@ -0,0 +1,106 @@ +/* Copyright (c) 2003-2004, Roger Dingledine + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file transports.h + * \brief Headers for transports.c + **/ + +#ifndef TOR_TRANSPORTS_H +#define TOR_TRANSPORTS_H + +void pt_kickstart_proxy(const smartlist_t *transport_list, char **proxy_argv, + int is_server); + +#define pt_kickstart_client_proxy(tl, pa) \ + pt_kickstart_proxy(tl, pa, 0) +#define pt_kickstart_server_proxy(tl, pa) \ + pt_kickstart_proxy(tl, pa, 1) + +void pt_configure_remaining_proxies(void); + +int pt_proxies_configuration_pending(void); + +void pt_free_all(void); + +void pt_prepare_proxy_list_for_config_read(void); +void sweep_proxy_list(void); + +#ifdef PT_PRIVATE +/** State of the managed proxy configuration protocol. */ +enum pt_proto_state { + PT_PROTO_INFANT, /* was just born */ + PT_PROTO_LAUNCHED, /* was just launched */ + PT_PROTO_ACCEPTING_METHODS, /* accepting methods */ + PT_PROTO_CONFIGURED, /* configured successfully */ + PT_PROTO_COMPLETED, /* configure and registered its transports */ + PT_PROTO_BROKEN, /* broke during the protocol */ + PT_PROTO_FAILED_LAUNCH /* failed while launching */ +}; + +/** Structure containing information of a managed proxy. */ +typedef struct { + enum pt_proto_state conf_state; /* the current configuration state */ + char **argv; /* the cli arguments of this proxy */ + int conf_protocol; /* the configuration protocol version used */ + + int is_server; /* is it a server proxy? */ + + /* A pointer to the process handle of this managed proxy. */ + process_handle_t *process_handle; + + int pid; /* The Process ID this managed proxy is using. */ + + /** Boolean: We are re-parsing our config, and we are going to + * remove this managed proxy if we don't find it any transport + * plugins that use it. */ + unsigned int marked_for_removal : 1; + + /** Boolean: We got a SIGHUP while this proxy was running. We use + * this flag to signify that this proxy might need to be restarted + * so that it can listen for other transports according to the new + * torrc. */ + unsigned int got_hup : 1; + + /* transports to-be-launched by this proxy */ + smartlist_t *transports_to_launch; + + /* The 'transports' list contains all the transports this proxy has + launched. + + Before a managed_proxy_t reaches the PT_PROTO_COMPLETED phase, + this smartlist contains a 'transport_t' for every transport it + has launched. + + When the managed_proxy_t reaches the PT_PROTO_COMPLETED phase, it + registers all its transports to the circuitbuild.c subsystem. At + that point the 'transport_t's are owned by the circuitbuild.c + subsystem. + + To avoid carrying dangling 'transport_t's in this smartlist, + right before the managed_proxy_t reaches the PT_PROTO_COMPLETED + phase we replace all 'transport_t's with strings of their + transport names. + + So, tl;dr: + When (conf_state != PT_PROTO_COMPLETED) this list carries + (transport_t *). + When (conf_state == PT_PROTO_COMPLETED) this list carries + (char *). + */ + smartlist_t *transports; +} managed_proxy_t; + +int parse_cmethod_line(const char *line, managed_proxy_t *mp); +int parse_smethod_line(const char *line, managed_proxy_t *mp); + +int parse_version(const char *line, managed_proxy_t *mp); +void parse_env_error(const char *line); +void handle_proxy_line(const char *line, managed_proxy_t *mp); + +#endif + +#endif + diff --git a/src/test/Makefile.am b/src/test/Makefile.am index 904719d94b..31a464ee7a 100644 --- a/src/test/Makefile.am +++ b/src/test/Makefile.am @@ -1,6 +1,6 @@ TESTS = test -noinst_PROGRAMS = test +noinst_PROGRAMS = test test-child bench AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ -DLOCALSTATEDIR="\"$(localstatedir)\"" \ @@ -12,19 +12,38 @@ AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ # matters a lot there, and is quite hard to debug if you forget to do it. test_SOURCES = \ - test_data.c \ test.c \ test_addr.c \ + test_containers.c \ test_crypto.c \ + test_data.c \ test_dir.c \ - test_containers.c \ + test_microdesc.c \ + test_pt.c \ test_util.c \ + test_config.c \ tinytest.c +bench_SOURCES = \ + bench.c + test_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ @TOR_LDFLAGS_libevent@ test_LDADD = ../or/libtor.a ../common/libor.a ../common/libor-crypto.a \ ../common/libor-event.a \ - @TOR_ZLIB_LIBS@ -lm @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ + @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +bench_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ + @TOR_LDFLAGS_libevent@ +bench_LDADD = ../or/libtor.a ../common/libor.a ../common/libor-crypto.a \ + ../common/libor-event.a \ + @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ + @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +noinst_HEADERS = \ + tinytest.h \ + tinytest_macros.h \ + test.h + -noinst_HEADERS = tinytest.h tinytest_macros.h test.h diff --git a/src/test/Makefile.nmake b/src/test/Makefile.nmake new file mode 100644 index 0000000000..aec477cf99 --- /dev/null +++ b/src/test/Makefile.nmake @@ -0,0 +1,20 @@ +all: test.exe + +CFLAGS = /I ..\win32 /I ..\..\..\build-alpha\include /I ..\common /I ..\or + +LIBS = ..\..\..\build-alpha\lib\libevent.lib \ + ..\..\..\build-alpha\lib\libcrypto.lib \ + ..\..\..\build-alpha\lib\libssl.lib \ + ..\..\..\build-alpha\lib\libz.lib \ + ..\or\libtor.lib \ + ws2_32.lib advapi32.lib shell32.lib + +TEST_OBJECTS = test.obj test_addr.obj test_containers.obj \ + test_crypto.obj test_data.obj test_dir.obj test_microdesc.obj \ + test_pt.obj test_util.obj test_config.obj tinytest.obj + +test.exe: $(TEST_OBJECTS) + $(CC) $(CFLAGS) $(LIBS) ..\common\*.lib $(TEST_OBJECTS) + +clean: + del $(TEST_OBJECTS) *.lib test.exe diff --git a/src/test/bench.c b/src/test/bench.c new file mode 100644 index 0000000000..3eae532d30 --- /dev/null +++ b/src/test/bench.c @@ -0,0 +1,320 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/* Ordinarily defined in tor_main.c; this bit is just here to provide one + * since we're not linking to tor_main.c */ +const char tor_git_revision[] = ""; + +/** + * \file bench.c + * \brief Benchmarks for lower level Tor modules. + **/ + +#include "orconfig.h" + +#define RELAY_PRIVATE + +#include "or.h" +#include "relay.h" + +#if defined(HAVE_CLOCK_GETTIME) && defined(CLOCK_PROCESS_CPUTIME_ID) +static uint64_t nanostart; +static inline uint64_t +timespec_to_nsec(const struct timespec *ts) +{ + return ((uint64_t)ts->tv_sec)*1000000000 + ts->tv_nsec; +} + +static void +reset_perftime(void) +{ + struct timespec ts; + int r; + r = clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &ts); + tor_assert(r == 0); + nanostart = timespec_to_nsec(&ts); +} + +static uint64_t +perftime(void) +{ + struct timespec ts; + int r; + r = clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &ts); + tor_assert(r == 0); + return timespec_to_nsec(&ts) - nanostart; +} + +#else +static struct timeval tv_start = { 0, 0 }; +static void +reset_perftime(void) +{ + tor_gettimeofday(&tv_start); +} +static uint64_t +perftime(void) +{ + struct timeval now, out; + tor_gettimeofday(&now); + timersub(&now, &tv_start, &out); + return ((uint64_t)out.tv_sec)*1000000000 + out.tv_usec*1000; +} +#endif + +#define NANOCOUNT(start,end,iters) \ + ( ((double)((end)-(start))) / (iters) ) + +/** Run AES performance benchmarks. */ +static void +bench_aes(void) +{ + int len, i; + char *b1, *b2; + crypto_cipher_t *c; + uint64_t start, end; + const int bytes_per_iter = (1<<24); + reset_perftime(); + c = crypto_cipher_new(NULL); + + for (len = 1; len <= 8192; len *= 2) { + int iters = bytes_per_iter / len; + b1 = tor_malloc_zero(len); + b2 = tor_malloc_zero(len); + start = perftime(); + for (i = 0; i < iters; ++i) { + crypto_cipher_encrypt(c, b1, b2, len); + } + end = perftime(); + tor_free(b1); + tor_free(b2); + printf("%d bytes: %.2f nsec per byte\n", len, + NANOCOUNT(start, end, iters*len)); + } + crypto_cipher_free(c); +} + +static void +bench_cell_aes(void) +{ + uint64_t start, end; + const int len = 509; + const int iters = (1<<16); + const int max_misalign = 15; + char *b = tor_malloc(len+max_misalign); + crypto_cipher_t *c; + int i, misalign; + + c = crypto_cipher_new(NULL); + + reset_perftime(); + for (misalign = 0; misalign <= max_misalign; ++misalign) { + start = perftime(); + for (i = 0; i < iters; ++i) { + crypto_cipher_crypt_inplace(c, b+misalign, len); + } + end = perftime(); + printf("%d bytes, misaligned by %d: %.2f nsec per byte\n", len, misalign, + NANOCOUNT(start, end, iters*len)); + } + + crypto_cipher_free(c); + tor_free(b); +} + +/** Run digestmap_t performance benchmarks. */ +static void +bench_dmap(void) +{ + smartlist_t *sl = smartlist_new(); + smartlist_t *sl2 = smartlist_new(); + uint64_t start, end, pt2, pt3, pt4; + int iters = 8192; + const int elts = 4000; + const int fpostests = 100000; + char d[20]; + int i,n=0, fp = 0; + digestmap_t *dm = digestmap_new(); + digestset_t *ds = digestset_new(elts); + + for (i = 0; i < elts; ++i) { + crypto_rand(d, 20); + smartlist_add(sl, tor_memdup(d, 20)); + } + for (i = 0; i < elts; ++i) { + crypto_rand(d, 20); + smartlist_add(sl2, tor_memdup(d, 20)); + } + printf("nbits=%d\n", ds->mask+1); + + reset_perftime(); + + start = perftime(); + for (i = 0; i < iters; ++i) { + SMARTLIST_FOREACH(sl, const char *, cp, digestmap_set(dm, cp, (void*)1)); + } + pt2 = perftime(); + printf("digestmap_set: %.2f ns per element\n", + NANOCOUNT(start, pt2, iters*elts)); + + for (i = 0; i < iters; ++i) { + SMARTLIST_FOREACH(sl, const char *, cp, digestmap_get(dm, cp)); + SMARTLIST_FOREACH(sl2, const char *, cp, digestmap_get(dm, cp)); + } + pt3 = perftime(); + printf("digestmap_get: %.2f ns per element\n", + NANOCOUNT(pt2, pt3, iters*elts*2)); + + for (i = 0; i < iters; ++i) { + SMARTLIST_FOREACH(sl, const char *, cp, digestset_add(ds, cp)); + } + pt4 = perftime(); + printf("digestset_add: %.2f ns per element\n", + NANOCOUNT(pt3, pt4, iters*elts)); + + for (i = 0; i < iters; ++i) { + SMARTLIST_FOREACH(sl, const char *, cp, n += digestset_isin(ds, cp)); + SMARTLIST_FOREACH(sl2, const char *, cp, n += digestset_isin(ds, cp)); + } + end = perftime(); + printf("digestset_isin: %.2f ns per element.\n", + NANOCOUNT(pt4, end, iters*elts*2)); + /* We need to use this, or else the whole loop gets optimized out. */ + printf("Hits == %d\n", n); + + for (i = 0; i < fpostests; ++i) { + crypto_rand(d, 20); + if (digestset_isin(ds, d)) ++fp; + } + printf("False positive rate on digestset: %.2f%%\n", + (fp/(double)fpostests)*100); + + digestmap_free(dm, NULL); + digestset_free(ds); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + SMARTLIST_FOREACH(sl2, char *, cp, tor_free(cp)); + smartlist_free(sl); + smartlist_free(sl2); +} + +static void +bench_cell_ops(void) +{ + const int iters = 1<<16; + int i; + + /* benchmarks for cell ops at relay. */ + or_circuit_t *or_circ = tor_malloc_zero(sizeof(or_circuit_t)); + cell_t *cell = tor_malloc(sizeof(cell_t)); + int outbound; + uint64_t start, end; + + crypto_rand((char*)cell->payload, sizeof(cell->payload)); + + /* Mock-up or_circuit_t */ + or_circ->_base.magic = OR_CIRCUIT_MAGIC; + or_circ->_base.purpose = CIRCUIT_PURPOSE_OR; + + /* Initialize crypto */ + or_circ->p_crypto = crypto_cipher_new(NULL); + or_circ->n_crypto = crypto_cipher_new(NULL); + or_circ->p_digest = crypto_digest_new(); + or_circ->n_digest = crypto_digest_new(); + + reset_perftime(); + + for (outbound = 0; outbound <= 1; ++outbound) { + cell_direction_t d = outbound ? CELL_DIRECTION_OUT : CELL_DIRECTION_IN; + start = perftime(); + for (i = 0; i < iters; ++i) { + char recognized = 0; + crypt_path_t *layer_hint = NULL; + relay_crypt(TO_CIRCUIT(or_circ), cell, d, &layer_hint, &recognized); + } + end = perftime(); + printf("%sbound cells: %.2f ns per cell. (%.2f ns per byte of payload)\n", + outbound?"Out":" In", + NANOCOUNT(start,end,iters), + NANOCOUNT(start,end,iters*CELL_PAYLOAD_SIZE)); + } + + crypto_digest_free(or_circ->p_digest); + crypto_digest_free(or_circ->n_digest); + crypto_cipher_free(or_circ->p_crypto); + crypto_cipher_free(or_circ->n_crypto); + tor_free(or_circ); + tor_free(cell); +} + +typedef void (*bench_fn)(void); + +typedef struct benchmark_t { + const char *name; + bench_fn fn; + int enabled; +} benchmark_t; + +#define ENT(s) { #s , bench_##s, 0 } + +static struct benchmark_t benchmarks[] = { + ENT(dmap), + ENT(aes), + ENT(cell_aes), + ENT(cell_ops), + {NULL,NULL,0} +}; + +static benchmark_t * +find_benchmark(const char *name) +{ + benchmark_t *b; + for (b = benchmarks; b->name; ++b) { + if (!strcmp(name, b->name)) { + return b; + } + } + return NULL; +} + +/** Main entry point for benchmark code: parse the command line, and run + * some benchmarks. */ +int +main(int argc, const char **argv) +{ + int i; + int list=0, n_enabled=0; + benchmark_t *b; + + tor_threads_init(); + + for (i = 1; i < argc; ++i) { + if (!strcmp(argv[i], "--list")) { + list = 1; + } else { + benchmark_t *b = find_benchmark(argv[i]); + ++n_enabled; + if (b) { + b->enabled = 1; + } else { + printf("No such benchmark as %s\n", argv[i]); + } + } + } + + reset_perftime(); + + crypto_seed_rng(1); + + for (b = benchmarks; b->name; ++b) { + if (b->enabled || n_enabled == 0) { + printf("===== %s =====\n", b->name); + if (!list) + b->fn(); + } + } + + return 0; +} + diff --git a/src/test/test-child.c b/src/test/test-child.c new file mode 100644 index 0000000000..c5725f1c5a --- /dev/null +++ b/src/test/test-child.c @@ -0,0 +1,43 @@ +/* Copyright (c) 2011-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include <stdio.h> +#include "orconfig.h" +#ifdef _WIN32 +#define WINDOWS_LEAN_AND_MEAN +#include <windows.h> +#else +#include <unistd.h> +#endif + +/** Trivial test program which prints out its command line arguments so we can + * check if tor_spawn_background() works */ +int +main(int argc, char **argv) +{ + int i; + + fprintf(stdout, "OUT\n"); + fprintf(stderr, "ERR\n"); + for (i = 1; i < argc; i++) + fprintf(stdout, "%s\n", argv[i]); + fprintf(stdout, "SLEEPING\n"); + /* We need to flush stdout so that test_util_spawn_background_partial_read() + succeed. Otherwise ReadFile() will get the entire output in one */ + // XXX: Can we make stdio flush on newline? + fflush(stdout); +#ifdef _WIN32 + Sleep(1000); +#else + sleep(1); +#endif + fprintf(stdout, "DONE\n"); +#ifdef _WIN32 + Sleep(1000); +#else + sleep(1); +#endif + + return 0; +} + diff --git a/src/test/test.c b/src/test/test.c index b5b744eba7..ae423948ec 100644 --- a/src/test/test.c +++ b/src/test/test.c @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /* Ordinarily defined in tor_main.c; this bit is just here to provide one @@ -19,7 +19,7 @@ const char tor_git_revision[] = ""; #include <fcntl.h> #endif -#ifdef MS_WINDOWS +#ifdef _WIN32 /* For mkdir() */ #include <direct.h> #else @@ -71,6 +71,9 @@ int have_failed = 0; /** Temporary directory (set up by setup_directory) under which we store all * our files during testing. */ static char temp_dir[256]; +#ifdef _WIN32 +#define pid_t int +#endif static pid_t temp_dir_setup_in_pid = 0; /** Select and create the temporary directory we'll use to run our unit tests. @@ -83,12 +86,12 @@ setup_directory(void) int r; if (is_setup) return; -#ifdef MS_WINDOWS +#ifdef _WIN32 { char buf[MAX_PATH]; const char *tmp = buf; /* If this fails, we're probably screwed anyway */ - if (!GetTempPath(sizeof(buf),buf)) + if (!GetTempPathA(sizeof(buf),buf)) tmp = "c:\\windows\\temp"; tor_snprintf(temp_dir, sizeof(temp_dir), "%s\\tor_test_%d", tmp, (int)getpid()); @@ -119,56 +122,71 @@ get_fname(const char *name) return buf; } +/* Remove a directory and all of its subdirectories */ +static void +rm_rf(const char *dir) +{ + struct stat st; + smartlist_t *elements; + + elements = tor_listdir(dir); + if (elements) { + SMARTLIST_FOREACH_BEGIN(elements, const char *, cp) { + char *tmp = NULL; + tor_asprintf(&tmp, "%s"PATH_SEPARATOR"%s", dir, cp); + if (0 == stat(tmp,&st) && (st.st_mode & S_IFDIR)) { + rm_rf(tmp); + } else { + if (unlink(tmp)) { + fprintf(stderr, "Error removing %s: %s\n", tmp, strerror(errno)); + } + } + tor_free(tmp); + } SMARTLIST_FOREACH_END(cp); + SMARTLIST_FOREACH(elements, char *, cp, tor_free(cp)); + smartlist_free(elements); + } + if (rmdir(dir)) + fprintf(stderr, "Error removing directory %s: %s\n", dir, strerror(errno)); +} + /** Remove all files stored under the temporary directory, and the directory * itself. Called by atexit(). */ static void remove_directory(void) { - smartlist_t *elements; if (getpid() != temp_dir_setup_in_pid) { /* Only clean out the tempdir when the main process is exiting. */ return; } - elements = tor_listdir(temp_dir); - if (elements) { - SMARTLIST_FOREACH(elements, const char *, cp, - { - size_t len = strlen(cp)+strlen(temp_dir)+16; - char *tmp = tor_malloc(len); - tor_snprintf(tmp, len, "%s"PATH_SEPARATOR"%s", temp_dir, cp); - unlink(tmp); - tor_free(tmp); - }); - SMARTLIST_FOREACH(elements, char *, cp, tor_free(cp)); - smartlist_free(elements); - } - rmdir(temp_dir); + + rm_rf(temp_dir); } /** Define this if unit tests spend too much time generating public keys*/ #undef CACHE_GENERATED_KEYS -static crypto_pk_env_t *pregen_keys[5] = {NULL, NULL, NULL, NULL, NULL}; +static crypto_pk_t *pregen_keys[5] = {NULL, NULL, NULL, NULL, NULL}; #define N_PREGEN_KEYS ((int)(sizeof(pregen_keys)/sizeof(pregen_keys[0]))) /** Generate and return a new keypair for use in unit tests. If we're using * the key cache optimization, we might reuse keys: we only guarantee that * keys made with distinct values for <b>idx</b> are different. The value of * <b>idx</b> must be at least 0, and less than N_PREGEN_KEYS. */ -crypto_pk_env_t * +crypto_pk_t * pk_generate(int idx) { #ifdef CACHE_GENERATED_KEYS tor_assert(idx < N_PREGEN_KEYS); if (! pregen_keys[idx]) { - pregen_keys[idx] = crypto_new_pk_env(); + pregen_keys[idx] = crypto_pk_new(); tor_assert(!crypto_pk_generate_key(pregen_keys[idx])); } return crypto_pk_dup_key(pregen_keys[idx]); #else - crypto_pk_env_t *result; + crypto_pk_t *result; (void) idx; - result = crypto_new_pk_env(); + result = crypto_pk_new(); tor_assert(!crypto_pk_generate_key(result)); return result; #endif @@ -181,12 +199,443 @@ free_pregenerated_keys(void) unsigned idx; for (idx = 0; idx < N_PREGEN_KEYS; ++idx) { if (pregen_keys[idx]) { - crypto_free_pk_env(pregen_keys[idx]); + crypto_pk_free(pregen_keys[idx]); pregen_keys[idx] = NULL; } } } +typedef struct socks_test_data_t { + socks_request_t *req; + buf_t *buf; +} socks_test_data_t; + +static void * +socks_test_setup(const struct testcase_t *testcase) +{ + socks_test_data_t *data = tor_malloc(sizeof(socks_test_data_t)); + (void)testcase; + data->buf = buf_new_with_capacity(256); + data->req = socks_request_new(); + config_register_addressmaps(get_options()); + return data; +} +static int +socks_test_cleanup(const struct testcase_t *testcase, void *ptr) +{ + socks_test_data_t *data = ptr; + (void)testcase; + buf_free(data->buf); + socks_request_free(data->req); + tor_free(data); + return 1; +} + +const struct testcase_setup_t socks_setup = { + socks_test_setup, socks_test_cleanup +}; + +#define SOCKS_TEST_INIT() \ + socks_test_data_t *testdata = ptr; \ + buf_t *buf = testdata->buf; \ + socks_request_t *socks = testdata->req; +#define ADD_DATA(buf, s) \ + write_to_buf(s, sizeof(s)-1, buf) + +static void +socks_request_clear(socks_request_t *socks) +{ + tor_free(socks->username); + tor_free(socks->password); + memset(socks, 0, sizeof(socks_request_t)); +} + +/** Perform unsupported SOCKS 4 commands */ +static void +test_socks_4_unsupported_commands(void *ptr) +{ + SOCKS_TEST_INIT(); + + /* SOCKS 4 Send BIND [02] to IP address 2.2.2.2:4369 */ + ADD_DATA(buf, "\x04\x02\x11\x11\x02\x02\x02\x02\x00"); + test_assert(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks) == -1); + test_eq(4, socks->socks_version); + test_eq(0, socks->replylen); /* XXX: shouldn't tor reply? */ + + done: + ; +} + +/** Perform supported SOCKS 4 commands */ +static void +test_socks_4_supported_commands(void *ptr) +{ + SOCKS_TEST_INIT(); + + test_eq(0, buf_datalen(buf)); + + /* SOCKS 4 Send CONNECT [01] to IP address 2.2.2.2:4370 */ + ADD_DATA(buf, "\x04\x01\x11\x12\x02\x02\x02\x03\x00"); + test_assert(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks) == 1); + test_eq(4, socks->socks_version); + test_eq(0, socks->replylen); /* XXX: shouldn't tor reply? */ + test_eq(SOCKS_COMMAND_CONNECT, socks->command); + test_streq("2.2.2.3", socks->address); + test_eq(4370, socks->port); + test_assert(socks->got_auth == 0); + test_assert(! socks->username); + + test_eq(0, buf_datalen(buf)); + socks_request_clear(socks); + + /* SOCKS 4 Send CONNECT [01] to IP address 2.2.2.2:4369 with userid*/ + ADD_DATA(buf, "\x04\x01\x11\x12\x02\x02\x02\x04me\x00"); + test_assert(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks) == 1); + test_eq(4, socks->socks_version); + test_eq(0, socks->replylen); /* XXX: shouldn't tor reply? */ + test_eq(SOCKS_COMMAND_CONNECT, socks->command); + test_streq("2.2.2.4", socks->address); + test_eq(4370, socks->port); + test_assert(socks->got_auth == 1); + test_assert(socks->username); + test_eq(2, socks->usernamelen); + test_memeq("me", socks->username, 2); + + test_eq(0, buf_datalen(buf)); + socks_request_clear(socks); + + /* SOCKS 4a Send RESOLVE [F0] request for torproject.org */ + ADD_DATA(buf, "\x04\xF0\x01\x01\x00\x00\x00\x02me\x00torproject.org\x00"); + test_assert(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks) == 1); + test_eq(4, socks->socks_version); + test_eq(0, socks->replylen); /* XXX: shouldn't tor reply? */ + test_streq("torproject.org", socks->address); + + test_eq(0, buf_datalen(buf)); + + done: + ; +} + +/** Perform unsupported SOCKS 5 commands */ +static void +test_socks_5_unsupported_commands(void *ptr) +{ + SOCKS_TEST_INIT(); + + /* SOCKS 5 Send unsupported BIND [02] command */ + ADD_DATA(buf, "\x05\x02\x00\x01"); + + test_eq(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks), 0); + test_eq(0, buf_datalen(buf)); + test_eq(5, socks->socks_version); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(0, socks->reply[1]); + ADD_DATA(buf, "\x05\x02\x00\x01\x02\x02\x02\x01\x01\x01"); + test_eq(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks), -1); + /* XXX: shouldn't tor reply 'command not supported' [07]? */ + + buf_clear(buf); + socks_request_clear(socks); + + /* SOCKS 5 Send unsupported UDP_ASSOCIATE [03] command */ + ADD_DATA(buf, "\x05\x03\x00\x01\x02"); + test_eq(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks), 0); + test_eq(5, socks->socks_version); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(0, socks->reply[1]); + ADD_DATA(buf, "\x05\x03\x00\x01\x02\x02\x02\x01\x01\x01"); + test_eq(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks), -1); + /* XXX: shouldn't tor reply 'command not supported' [07]? */ + + done: + ; +} + +/** Perform supported SOCKS 5 commands */ +static void +test_socks_5_supported_commands(void *ptr) +{ + SOCKS_TEST_INIT(); + + /* SOCKS 5 Send CONNECT [01] to IP address 2.2.2.2:4369 */ + ADD_DATA(buf, "\x05\x01\x00"); + test_eq(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks), 0); + test_eq(5, socks->socks_version); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(0, socks->reply[1]); + + ADD_DATA(buf, "\x05\x01\x00\x01\x02\x02\x02\x02\x11\x11"); + test_eq(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks), 1); + test_streq("2.2.2.2", socks->address); + test_eq(4369, socks->port); + + test_eq(0, buf_datalen(buf)); + socks_request_clear(socks); + + /* SOCKS 5 Send CONNECT [01] to FQDN torproject.org:4369 */ + ADD_DATA(buf, "\x05\x01\x00"); + ADD_DATA(buf, "\x05\x01\x00\x03\x0Etorproject.org\x11\x11"); + test_eq(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks), 1); + + test_eq(5, socks->socks_version); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(0, socks->reply[1]); + test_streq("torproject.org", socks->address); + test_eq(4369, socks->port); + + test_eq(0, buf_datalen(buf)); + socks_request_clear(socks); + + /* SOCKS 5 Send RESOLVE [F0] request for torproject.org:4369 */ + ADD_DATA(buf, "\x05\x01\x00"); + ADD_DATA(buf, "\x05\xF0\x00\x03\x0Etorproject.org\x01\x02"); + test_assert(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks) == 1); + test_eq(5, socks->socks_version); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(0, socks->reply[1]); + test_streq("torproject.org", socks->address); + + test_eq(0, buf_datalen(buf)); + socks_request_clear(socks); + + /* SOCKS 5 Send RESOLVE_PTR [F1] for IP address 2.2.2.5 */ + ADD_DATA(buf, "\x05\x01\x00"); + ADD_DATA(buf, "\x05\xF1\x00\x01\x02\x02\x02\x05\x01\x03"); + test_assert(fetch_from_buf_socks(buf, socks, get_options()->TestSocks, + get_options()->SafeSocks) == 1); + test_eq(5, socks->socks_version); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(0, socks->reply[1]); + test_streq("2.2.2.5", socks->address); + + test_eq(0, buf_datalen(buf)); + + done: + ; +} + +/** Perform SOCKS 5 authentication */ +static void +test_socks_5_no_authenticate(void *ptr) +{ + SOCKS_TEST_INIT(); + + /*SOCKS 5 No Authentication */ + ADD_DATA(buf,"\x05\x01\x00"); + test_assert(!fetch_from_buf_socks(buf, socks, + get_options()->TestSocks, + get_options()->SafeSocks)); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(SOCKS_NO_AUTH, socks->reply[1]); + + test_eq(0, buf_datalen(buf)); + + /*SOCKS 5 Send username/password anyway - pretend to be broken */ + ADD_DATA(buf,"\x01\x02\x01\x01\x02\x01\x01"); + test_assert(!fetch_from_buf_socks(buf, socks, + get_options()->TestSocks, + get_options()->SafeSocks)); + test_eq(5, socks->socks_version); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(0, socks->reply[1]); + + test_eq(2, socks->usernamelen); + test_eq(2, socks->passwordlen); + + test_memeq("\x01\x01", socks->username, 2); + test_memeq("\x01\x01", socks->password, 2); + + done: + ; +} + +/** Perform SOCKS 5 authentication */ +static void +test_socks_5_authenticate(void *ptr) +{ + SOCKS_TEST_INIT(); + + /* SOCKS 5 Negotiate username/password authentication */ + ADD_DATA(buf, "\x05\x01\x02"); + + test_assert(!fetch_from_buf_socks(buf, socks, + get_options()->TestSocks, + get_options()->SafeSocks)); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(SOCKS_USER_PASS, socks->reply[1]); + test_eq(5, socks->socks_version); + + test_eq(0, buf_datalen(buf)); + + /* SOCKS 5 Send username/password */ + ADD_DATA(buf, "\x01\x02me\x08mypasswd"); + test_assert(!fetch_from_buf_socks(buf, socks, + get_options()->TestSocks, + get_options()->SafeSocks)); + test_eq(5, socks->socks_version); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(0, socks->reply[1]); + + test_eq(2, socks->usernamelen); + test_eq(8, socks->passwordlen); + + test_memeq("me", socks->username, 2); + test_memeq("mypasswd", socks->password, 8); + + done: + ; +} + +/** Perform SOCKS 5 authentication and send data all in one go */ +static void +test_socks_5_authenticate_with_data(void *ptr) +{ + SOCKS_TEST_INIT(); + + /* SOCKS 5 Negotiate username/password authentication */ + ADD_DATA(buf, "\x05\x01\x02"); + + test_assert(!fetch_from_buf_socks(buf, socks, + get_options()->TestSocks, + get_options()->SafeSocks)); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(SOCKS_USER_PASS, socks->reply[1]); + test_eq(5, socks->socks_version); + + test_eq(0, buf_datalen(buf)); + + /* SOCKS 5 Send username/password */ + /* SOCKS 5 Send CONNECT [01] to IP address 2.2.2.2:4369 */ + ADD_DATA(buf, "\x01\x02me\x03you\x05\x01\x00\x01\x02\x02\x02\x02\x11\x11"); + test_assert(fetch_from_buf_socks(buf, socks, + get_options()->TestSocks, + get_options()->SafeSocks) == 1); + test_eq(5, socks->socks_version); + test_eq(2, socks->replylen); + test_eq(5, socks->reply[0]); + test_eq(0, socks->reply[1]); + + test_streq("2.2.2.2", socks->address); + test_eq(4369, socks->port); + + test_eq(2, socks->usernamelen); + test_eq(3, socks->passwordlen); + test_memeq("me", socks->username, 2); + test_memeq("you", socks->password, 3); + + done: + ; +} + +/** Perform SOCKS 5 authentication before method negotiated */ +static void +test_socks_5_auth_before_negotiation(void *ptr) +{ + SOCKS_TEST_INIT(); + + /* SOCKS 5 Send username/password */ + ADD_DATA(buf, "\x01\x02me\x02me"); + test_assert(fetch_from_buf_socks(buf, socks, + get_options()->TestSocks, + get_options()->SafeSocks) == -1); + test_eq(0, socks->socks_version); + test_eq(0, socks->replylen); + test_eq(0, socks->reply[0]); + test_eq(0, socks->reply[1]); + + done: + ; +} + +static void +test_buffer_copy(void *arg) +{ + generic_buffer_t *buf=NULL, *buf2=NULL; + const char *s; + size_t len; + char b[256]; + int i; + (void)arg; + + buf = generic_buffer_new(); + tt_assert(buf); + + /* Copy an empty buffer. */ + tt_int_op(0, ==, generic_buffer_set_to_copy(&buf2, buf)); + tt_assert(buf2); + tt_int_op(0, ==, generic_buffer_len(buf2)); + + /* Now try with a short buffer. */ + s = "And now comes an act of enormous enormance!"; + len = strlen(s); + generic_buffer_add(buf, s, len); + tt_int_op(len, ==, generic_buffer_len(buf)); + /* Add junk to buf2 so we can test replacing.*/ + generic_buffer_add(buf2, "BLARG", 5); + tt_int_op(0, ==, generic_buffer_set_to_copy(&buf2, buf)); + tt_int_op(len, ==, generic_buffer_len(buf2)); + generic_buffer_get(buf2, b, len); + test_mem_op(b, ==, s, len); + /* Now free buf2 and retry so we can test allocating */ + generic_buffer_free(buf2); + buf2 = NULL; + tt_int_op(0, ==, generic_buffer_set_to_copy(&buf2, buf)); + tt_int_op(len, ==, generic_buffer_len(buf2)); + generic_buffer_get(buf2, b, len); + test_mem_op(b, ==, s, len); + /* Clear buf for next test */ + generic_buffer_get(buf, b, len); + tt_int_op(generic_buffer_len(buf),==,0); + + /* Okay, now let's try a bigger buffer. */ + s = "Quis autem vel eum iure reprehenderit qui in ea voluptate velit " + "esse quam nihil molestiae consequatur, vel illum qui dolorem eum " + "fugiat quo voluptas nulla pariatur?"; + len = strlen(s); + for (i = 0; i < 256; ++i) { + b[0]=i; + generic_buffer_add(buf, b, 1); + generic_buffer_add(buf, s, len); + } + tt_int_op(0, ==, generic_buffer_set_to_copy(&buf2, buf)); + tt_int_op(generic_buffer_len(buf2), ==, generic_buffer_len(buf)); + for (i = 0; i < 256; ++i) { + generic_buffer_get(buf2, b, len+1); + tt_int_op((unsigned char)b[0],==,i); + test_mem_op(b+1, ==, s, len); + } + + done: + if (buf) + generic_buffer_free(buf); + if (buf2) + generic_buffer_free(buf2); +} + /** Run unit tests for buffers.c */ static void test_buffers(void) @@ -353,6 +802,18 @@ test_buffers(void) buf_free(buf); buf = NULL; + /* Try adding a string too long for any freelist. */ + { + char *cp = tor_malloc_zero(65536); + buf = buf_new(); + write_to_buf(cp, 65536, buf); + tor_free(cp); + + tt_int_op(buf_datalen(buf), ==, 65536); + buf_free(buf); + buf = NULL; + } + done: if (buf) buf_free(buf); @@ -365,7 +826,7 @@ static void test_onion_handshake(void) { /* client-side */ - crypto_dh_env_t *c_dh = NULL; + crypto_dh_t *c_dh = NULL; char c_buf[ONIONSKIN_CHALLENGE_LEN]; char c_keys[40]; @@ -374,7 +835,7 @@ test_onion_handshake(void) char s_keys[40]; /* shared */ - crypto_pk_env_t *pk = NULL; + crypto_pk_t *pk = NULL; pk = pk_generate(0); @@ -404,7 +865,7 @@ test_onion_handshake(void) if (c_dh) crypto_dh_free(c_dh); if (pk) - crypto_free_pk_env(pk); + crypto_pk_free(pk); } static void @@ -555,6 +1016,28 @@ test_circuit_timeout(void) return; } +/* Helper: assert that short_policy parses and writes back out as itself, + or as <b>expected</b> if that's provided. */ +static void +test_short_policy_parse(const char *input, + const char *expected) +{ + short_policy_t *short_policy = NULL; + char *out = NULL; + + if (expected == NULL) + expected = input; + + short_policy = parse_short_policy(input); + tt_assert(short_policy); + out = write_short_policy(short_policy); + tt_str_op(out, ==, expected); + + done: + tor_free(out); + short_policy_free(short_policy); +} + /** Helper: Parse the exit policy string in <b>policy_str</b>, and make sure * that policies_summarize() produces the string <b>expected_summary</b> from * it. */ @@ -563,9 +1046,11 @@ test_policy_summary_helper(const char *policy_str, const char *expected_summary) { config_line_t line; - smartlist_t *policy = smartlist_create(); + smartlist_t *policy = smartlist_new(); char *summary = NULL; + char *summary_after = NULL; int r; + short_policy_t *short_policy = NULL; line.key = (char*)"foo"; line.value = (char *)policy_str; @@ -578,10 +1063,17 @@ test_policy_summary_helper(const char *policy_str, test_assert(summary != NULL); test_streq(summary, expected_summary); + short_policy = parse_short_policy(summary); + tt_assert(short_policy); + summary_after = write_short_policy(short_policy); + test_streq(summary, summary_after); + done: + tor_free(summary_after); tor_free(summary); if (policy) addr_policy_list_free(policy); + short_policy_free(short_policy); } /** Run unit tests for generating summary lines of exit policies */ @@ -598,7 +1090,7 @@ test_policies(void) smartlist_t *sm = NULL; char *policy_str = NULL; - policy = smartlist_create(); + policy = smartlist_new(); p = router_parse_addr_policy_item_from_string("reject 192.168.0.0/16:*",-1); test_assert(p != NULL); @@ -611,17 +1103,20 @@ test_policies(void) smartlist_add(policy, p); + tor_addr_from_ipv4h(&tar, 0x01020304u); test_assert(ADDR_POLICY_ACCEPTED == - compare_addr_to_addr_policy(0x01020304u, 2, policy)); + compare_tor_addr_to_addr_policy(&tar, 2, policy)); + tor_addr_make_unspec(&tar); test_assert(ADDR_POLICY_PROBABLY_ACCEPTED == - compare_addr_to_addr_policy(0, 2, policy)); + compare_tor_addr_to_addr_policy(&tar, 2, policy)); + tor_addr_from_ipv4h(&tar, 0xc0a80102); test_assert(ADDR_POLICY_REJECTED == - compare_addr_to_addr_policy(0xc0a80102, 2, policy)); + compare_tor_addr_to_addr_policy(&tar, 2, policy)); test_assert(0 == policies_parse_exit_policy(NULL, &policy2, 1, NULL, 1)); test_assert(policy2); - policy3 = smartlist_create(); + policy3 = smartlist_new(); p = router_parse_addr_policy_item_from_string("reject *:*",-1); test_assert(p != NULL); smartlist_add(policy3, p); @@ -629,7 +1124,7 @@ test_policies(void) test_assert(p != NULL); smartlist_add(policy3, p); - policy4 = smartlist_create(); + policy4 = smartlist_new(); p = router_parse_addr_policy_item_from_string("accept *:443",-1); test_assert(p != NULL); smartlist_add(policy4, p); @@ -637,7 +1132,7 @@ test_policies(void) test_assert(p != NULL); smartlist_add(policy4, p); - policy5 = smartlist_create(); + policy5 = smartlist_new(); p = router_parse_addr_policy_item_from_string("reject 0.0.0.0/8:*",-1); test_assert(p != NULL); smartlist_add(policy5, p); @@ -669,12 +1164,12 @@ test_policies(void) test_assert(p != NULL); smartlist_add(policy5, p); - policy6 = smartlist_create(); + policy6 = smartlist_new(); p = router_parse_addr_policy_item_from_string("accept 43.3.0.0/9:*",-1); test_assert(p != NULL); smartlist_add(policy6, p); - policy7 = smartlist_create(); + policy7 = smartlist_new(); p = router_parse_addr_policy_item_from_string("accept 0.0.0.0/8:*",-1); test_assert(p != NULL); smartlist_add(policy7, p); @@ -770,8 +1265,48 @@ test_policies(void) "accept *:*", "reject 1,3,5,7"); + /* Short policies with unrecognized formats should get accepted. */ + test_short_policy_parse("accept fred,2,3-5", "accept 2,3-5"); + test_short_policy_parse("accept 2,fred,3", "accept 2,3"); + test_short_policy_parse("accept 2,fred,3,bob", "accept 2,3"); + test_short_policy_parse("accept 2,-3,500-600", "accept 2,500-600"); + /* Short policies with nil entries are accepted too. */ + test_short_policy_parse("accept 1,,3", "accept 1,3"); + test_short_policy_parse("accept 100-200,,", "accept 100-200"); + test_short_policy_parse("reject ,1-10,,,,30-40", "reject 1-10,30-40"); + + /* Try parsing various broken short policies */ + tt_ptr_op(NULL, ==, parse_short_policy("accept 200-199")); + tt_ptr_op(NULL, ==, parse_short_policy("")); + tt_ptr_op(NULL, ==, parse_short_policy("rejekt 1,2,3")); + tt_ptr_op(NULL, ==, parse_short_policy("reject ")); + tt_ptr_op(NULL, ==, parse_short_policy("reject")); + tt_ptr_op(NULL, ==, parse_short_policy("rej")); + tt_ptr_op(NULL, ==, parse_short_policy("accept 2,3,100000")); + tt_ptr_op(NULL, ==, parse_short_policy("accept 2,3x,4")); + tt_ptr_op(NULL, ==, parse_short_policy("accept 2,3x,4")); + tt_ptr_op(NULL, ==, parse_short_policy("accept 2-")); + tt_ptr_op(NULL, ==, parse_short_policy("accept 2-x")); + tt_ptr_op(NULL, ==, parse_short_policy("accept 1-,3")); + tt_ptr_op(NULL, ==, parse_short_policy("accept 1-,3")); + /* Test a too-long policy. */ + { + int i; + char *policy = NULL; + smartlist_t *chunks = smartlist_new(); + smartlist_add(chunks, tor_strdup("accept ")); + for (i=1; i<10000; ++i) + smartlist_add_asprintf(chunks, "%d,", i); + smartlist_add(chunks, tor_strdup("20000")); + policy = smartlist_join_strings(chunks, "", 0, NULL); + SMARTLIST_FOREACH(chunks, char *, ch, tor_free(ch)); + smartlist_free(chunks); + tt_ptr_op(NULL, ==, parse_short_policy(policy));/* shouldn't be accepted */ + tor_free(policy); /* could leak. */ + } + /* truncation ports */ - sm = smartlist_create(); + sm = smartlist_new(); for (i=1; i<2000; i+=2) { char buf[POLICY_BUF_LEN]; tor_snprintf(buf, sizeof(buf), "reject *:%d", i); @@ -810,102 +1345,6 @@ test_policies(void) } } -/** Run AES performance benchmarks. */ -static void -bench_aes(void) -{ - int len, i; - char *b1, *b2; - crypto_cipher_env_t *c; - struct timeval start, end; - const int iters = 100000; - uint64_t nsec; - c = crypto_new_cipher_env(); - crypto_cipher_generate_key(c); - crypto_cipher_encrypt_init_cipher(c); - for (len = 1; len <= 8192; len *= 2) { - b1 = tor_malloc_zero(len); - b2 = tor_malloc_zero(len); - tor_gettimeofday(&start); - for (i = 0; i < iters; ++i) { - crypto_cipher_encrypt(c, b1, b2, len); - } - tor_gettimeofday(&end); - tor_free(b1); - tor_free(b2); - nsec = (uint64_t) tv_udiff(&start,&end); - nsec *= 1000; - nsec /= (iters*len); - printf("%d bytes: "U64_FORMAT" nsec per byte\n", len, - U64_PRINTF_ARG(nsec)); - } - crypto_free_cipher_env(c); -} - -/** Run digestmap_t performance benchmarks. */ -static void -bench_dmap(void) -{ - smartlist_t *sl = smartlist_create(); - smartlist_t *sl2 = smartlist_create(); - struct timeval start, end, pt2, pt3, pt4; - const int iters = 10000; - const int elts = 4000; - const int fpostests = 1000000; - char d[20]; - int i,n=0, fp = 0; - digestmap_t *dm = digestmap_new(); - digestset_t *ds = digestset_new(elts); - - for (i = 0; i < elts; ++i) { - crypto_rand(d, 20); - smartlist_add(sl, tor_memdup(d, 20)); - } - for (i = 0; i < elts; ++i) { - crypto_rand(d, 20); - smartlist_add(sl2, tor_memdup(d, 20)); - } - printf("nbits=%d\n", ds->mask+1); - - tor_gettimeofday(&start); - for (i = 0; i < iters; ++i) { - SMARTLIST_FOREACH(sl, const char *, cp, digestmap_set(dm, cp, (void*)1)); - } - tor_gettimeofday(&pt2); - for (i = 0; i < iters; ++i) { - SMARTLIST_FOREACH(sl, const char *, cp, digestmap_get(dm, cp)); - SMARTLIST_FOREACH(sl2, const char *, cp, digestmap_get(dm, cp)); - } - tor_gettimeofday(&pt3); - for (i = 0; i < iters; ++i) { - SMARTLIST_FOREACH(sl, const char *, cp, digestset_add(ds, cp)); - } - tor_gettimeofday(&pt4); - for (i = 0; i < iters; ++i) { - SMARTLIST_FOREACH(sl, const char *, cp, n += digestset_isin(ds, cp)); - SMARTLIST_FOREACH(sl2, const char *, cp, n += digestset_isin(ds, cp)); - } - tor_gettimeofday(&end); - - for (i = 0; i < fpostests; ++i) { - crypto_rand(d, 20); - if (digestset_isin(ds, d)) ++fp; - } - - printf("%ld\n",(unsigned long)tv_udiff(&start, &pt2)); - printf("%ld\n",(unsigned long)tv_udiff(&pt2, &pt3)); - printf("%ld\n",(unsigned long)tv_udiff(&pt3, &pt4)); - printf("%ld\n",(unsigned long)tv_udiff(&pt4, &end)); - printf("-- %d\n", n); - printf("++ %f\n", fp/(double)fpostests); - digestmap_free(dm, NULL); - digestset_free(ds); - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - SMARTLIST_FOREACH(sl2, char *, cp, tor_free(cp)); - smartlist_free(sl); - smartlist_free(sl2); -} - /** Test encoding and parsing of rendezvous service descriptors. */ static void test_rend_fns(void) @@ -914,10 +1353,10 @@ test_rend_fns(void) char service_id[DIGEST_LEN]; char service_id_base32[REND_SERVICE_ID_LEN_BASE32+1]; const char *next_desc; - smartlist_t *descs = smartlist_create(); + smartlist_t *descs = smartlist_new(); char computed_desc_id[DIGEST_LEN]; char parsed_desc_id[DIGEST_LEN]; - crypto_pk_env_t *pk1 = NULL, *pk2 = NULL; + crypto_pk_t *pk1 = NULL, *pk2 = NULL; time_t now; char *intro_points_encrypted = NULL; size_t intro_points_size; @@ -928,10 +1367,10 @@ test_rend_fns(void) char address3[] = "fooaddress.exit"; char address4[] = "www.torproject.org"; - test_assert(BAD_HOSTNAME == parse_extended_hostname(address1, 1)); - test_assert(ONION_HOSTNAME == parse_extended_hostname(address2, 1)); - test_assert(EXIT_HOSTNAME == parse_extended_hostname(address3, 1)); - test_assert(NORMAL_HOSTNAME == parse_extended_hostname(address4, 1)); + test_assert(BAD_HOSTNAME == parse_extended_hostname(address1)); + test_assert(ONION_HOSTNAME == parse_extended_hostname(address2)); + test_assert(EXIT_HOSTNAME == parse_extended_hostname(address3)); + test_assert(NORMAL_HOSTNAME == parse_extended_hostname(address4)); pk1 = pk_generate(0); pk2 = pk_generate(1); @@ -944,11 +1383,11 @@ test_rend_fns(void) generated->timestamp = now; generated->version = 2; generated->protocols = 42; - generated->intro_nodes = smartlist_create(); + generated->intro_nodes = smartlist_new(); for (i = 0; i < 3; i++) { rend_intro_point_t *intro = tor_malloc_zero(sizeof(rend_intro_point_t)); - crypto_pk_env_t *okey = pk_generate(2 + i); + crypto_pk_t *okey = pk_generate(2 + i); intro->extend_info = tor_malloc_zero(sizeof(extend_info_t)); intro->extend_info->onion_key = okey; crypto_pk_get_digest(intro->extend_info->onion_key, @@ -1015,9 +1454,9 @@ test_rend_fns(void) if (generated) rend_service_descriptor_free(generated); if (pk1) - crypto_free_pk_env(pk1); + crypto_pk_free(pk1); if (pk2) - crypto_free_pk_env(pk2); + crypto_pk_free(pk2); tor_free(intro_points_encrypted); } @@ -1026,8 +1465,74 @@ static void test_geoip(void) { int i, j; - time_t now = time(NULL); + time_t now = 1281533250; /* 2010-08-11 13:27:30 UTC */ char *s = NULL; + const char *bridge_stats_1 = + "bridge-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "bridge-ips zz=24,xy=8\n", + *dirreq_stats_1 = + "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "dirreq-v3-ips ab=8\n" + "dirreq-v2-ips \n" + "dirreq-v3-reqs ab=8\n" + "dirreq-v2-reqs \n" + "dirreq-v3-resp ok=0,not-enough-sigs=0,unavailable=0,not-found=0," + "not-modified=0,busy=0\n" + "dirreq-v2-resp ok=0,unavailable=0,not-found=0,not-modified=0," + "busy=0\n" + "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v2-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v3-tunneled-dl complete=0,timeout=0,running=0\n" + "dirreq-v2-tunneled-dl complete=0,timeout=0,running=0\n", + *dirreq_stats_2 = + "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "dirreq-v3-ips \n" + "dirreq-v2-ips \n" + "dirreq-v3-reqs \n" + "dirreq-v2-reqs \n" + "dirreq-v3-resp ok=0,not-enough-sigs=0,unavailable=0,not-found=0," + "not-modified=0,busy=0\n" + "dirreq-v2-resp ok=0,unavailable=0,not-found=0,not-modified=0," + "busy=0\n" + "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v2-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v3-tunneled-dl complete=0,timeout=0,running=0\n" + "dirreq-v2-tunneled-dl complete=0,timeout=0,running=0\n", + *dirreq_stats_3 = + "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "dirreq-v3-ips \n" + "dirreq-v2-ips \n" + "dirreq-v3-reqs \n" + "dirreq-v2-reqs \n" + "dirreq-v3-resp ok=8,not-enough-sigs=0,unavailable=0,not-found=0," + "not-modified=0,busy=0\n" + "dirreq-v2-resp ok=0,unavailable=0,not-found=0,not-modified=0," + "busy=0\n" + "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v2-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v3-tunneled-dl complete=0,timeout=0,running=0\n" + "dirreq-v2-tunneled-dl complete=0,timeout=0,running=0\n", + *dirreq_stats_4 = + "dirreq-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "dirreq-v3-ips \n" + "dirreq-v2-ips \n" + "dirreq-v3-reqs \n" + "dirreq-v2-reqs \n" + "dirreq-v3-resp ok=8,not-enough-sigs=0,unavailable=0,not-found=0," + "not-modified=0,busy=0\n" + "dirreq-v2-resp ok=0,unavailable=0,not-found=0,not-modified=0," + "busy=0\n" + "dirreq-v3-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v2-direct-dl complete=0,timeout=0,running=0\n" + "dirreq-v3-tunneled-dl complete=0,timeout=0,running=4\n" + "dirreq-v2-tunneled-dl complete=0,timeout=0,running=0\n", + *entry_stats_1 = + "entry-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "entry-ips ab=8\n", + *entry_stats_2 = + "entry-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "entry-ips \n"; + tor_addr_t addr; /* Populate the DB a bit. Add these in order, since we can't do the final * 'sort' step. These aren't very good IP addresses, but they're perfectly @@ -1053,19 +1558,26 @@ test_geoip(void) test_streq("??", NAMEFOR(2000)); #undef NAMEFOR - get_options()->BridgeRelay = 1; - get_options()->BridgeRecordUsageByCountry = 1; + get_options_mutable()->BridgeRelay = 1; + get_options_mutable()->BridgeRecordUsageByCountry = 1; /* Put 9 observations in AB... */ - for (i=32; i < 40; ++i) - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, i, now-7200); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, 225, now-7200); + for (i=32; i < 40; ++i) { + tor_addr_from_ipv4h(&addr, (uint32_t) i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, now-7200); + } + tor_addr_from_ipv4h(&addr, (uint32_t) 225); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, now-7200); /* and 3 observations in XY, several times. */ for (j=0; j < 10; ++j) - for (i=52; i < 55; ++i) - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, i, now-3600); + for (i=52; i < 55; ++i) { + tor_addr_from_ipv4h(&addr, (uint32_t) i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, now-3600); + } /* and 17 observations in ZZ... */ - for (i=110; i < 127; ++i) - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, i, now); + for (i=110; i < 127; ++i) { + tor_addr_from_ipv4h(&addr, (uint32_t) i); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, now); + } s = geoip_get_client_history(GEOIP_CLIENT_CONNECT); test_assert(s); test_streq("zz=24,ab=16,xy=8", s); @@ -1077,6 +1589,122 @@ test_geoip(void) test_assert(s); test_streq("zz=24,xy=8", s); + /* Start testing bridge statistics by making sure that we don't output + * bridge stats without initializing them. */ + s = geoip_format_bridge_stats(now + 86400); + test_assert(!s); + + /* Initialize stats and generate the bridge-stats history string out of + * the connecting clients added above. */ + geoip_bridge_stats_init(now); + s = geoip_format_bridge_stats(now + 86400); + test_streq(bridge_stats_1, s); + tor_free(s); + + /* Stop collecting bridge stats and make sure we don't write a history + * string anymore. */ + geoip_bridge_stats_term(); + s = geoip_format_bridge_stats(now + 86400); + test_assert(!s); + + /* Stop being a bridge and start being a directory mirror that gathers + * directory request statistics. */ + geoip_bridge_stats_term(); + get_options_mutable()->BridgeRelay = 0; + get_options_mutable()->BridgeRecordUsageByCountry = 0; + get_options_mutable()->DirReqStatistics = 1; + + /* Start testing dirreq statistics by making sure that we don't collect + * dirreq stats without initializing them. */ + tor_addr_from_ipv4h(&addr, (uint32_t) 100); + geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, now); + s = geoip_format_dirreq_stats(now + 86400); + test_assert(!s); + + /* Initialize stats, note one connecting client, and generate the + * dirreq-stats history string. */ + geoip_dirreq_stats_init(now); + tor_addr_from_ipv4h(&addr, (uint32_t) 100); + geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, now); + s = geoip_format_dirreq_stats(now + 86400); + test_streq(dirreq_stats_1, s); + tor_free(s); + + /* Stop collecting stats, add another connecting client, and ensure we + * don't generate a history string. */ + geoip_dirreq_stats_term(); + tor_addr_from_ipv4h(&addr, (uint32_t) 101); + geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, now); + s = geoip_format_dirreq_stats(now + 86400); + test_assert(!s); + + /* Re-start stats, add a connecting client, reset stats, and make sure + * that we get an all empty history string. */ + geoip_dirreq_stats_init(now); + tor_addr_from_ipv4h(&addr, (uint32_t) 100); + geoip_note_client_seen(GEOIP_CLIENT_NETWORKSTATUS, &addr, now); + geoip_reset_dirreq_stats(now); + s = geoip_format_dirreq_stats(now + 86400); + test_streq(dirreq_stats_2, s); + tor_free(s); + + /* Note a successful network status response and make sure that it + * appears in the history string. */ + geoip_note_ns_response(GEOIP_CLIENT_NETWORKSTATUS, GEOIP_SUCCESS); + s = geoip_format_dirreq_stats(now + 86400); + test_streq(dirreq_stats_3, s); + tor_free(s); + + /* Start a tunneled directory request. */ + geoip_start_dirreq((uint64_t) 1, 1024, GEOIP_CLIENT_NETWORKSTATUS, + DIRREQ_TUNNELED); + s = geoip_format_dirreq_stats(now + 86400); + test_streq(dirreq_stats_4, s); + + /* Stop collecting directory request statistics and start gathering + * entry stats. */ + geoip_dirreq_stats_term(); + get_options_mutable()->DirReqStatistics = 0; + get_options_mutable()->EntryStatistics = 1; + + /* Start testing entry statistics by making sure that we don't collect + * anything without initializing entry stats. */ + tor_addr_from_ipv4h(&addr, (uint32_t) 100); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, now); + s = geoip_format_entry_stats(now + 86400); + test_assert(!s); + + /* Initialize stats, note one connecting client, and generate the + * entry-stats history string. */ + geoip_entry_stats_init(now); + tor_addr_from_ipv4h(&addr, (uint32_t) 100); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, now); + s = geoip_format_entry_stats(now + 86400); + test_streq(entry_stats_1, s); + tor_free(s); + + /* Stop collecting stats, add another connecting client, and ensure we + * don't generate a history string. */ + geoip_entry_stats_term(); + tor_addr_from_ipv4h(&addr, (uint32_t) 101); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, now); + s = geoip_format_entry_stats(now + 86400); + test_assert(!s); + + /* Re-start stats, add a connecting client, reset stats, and make sure + * that we get an all empty history string. */ + geoip_entry_stats_init(now); + tor_addr_from_ipv4h(&addr, (uint32_t) 100); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, &addr, now); + geoip_reset_entry_stats(now); + s = geoip_format_entry_stats(now + 86400); + test_streq(entry_stats_2, s); + tor_free(s); + + /* Stop collecting entry statistics. */ + geoip_entry_stats_term(); + get_options_mutable()->EntryStatistics = 0; + done: tor_free(s); } @@ -1089,7 +1717,8 @@ test_stats(void) char *s = NULL; int i; - /* We shouldn't collect exit stats without initializing them. */ + /* Start with testing exit port statistics; we shouldn't collect exit + * stats without initializing them. */ rep_hist_note_exit_stream_opened(80); rep_hist_note_exit_bytes(80, 100, 10000); s = rep_hist_format_exit_stats(now + 86400); @@ -1134,7 +1763,7 @@ test_stats(void) test_assert(!s); /* Re-start stats, add some bytes, reset stats, and see what history we - * get when observing no streams or bytes at all. */ + * get when observing no streams or bytes at all. */ rep_hist_exit_stats_init(now); rep_hist_note_exit_stream_opened(80); rep_hist_note_exit_bytes(80, 100, 10000); @@ -1144,6 +1773,96 @@ test_stats(void) "exit-kibibytes-written other=0\n" "exit-kibibytes-read other=0\n" "exit-streams-opened other=0\n", s); + tor_free(s); + + /* Continue with testing connection statistics; we shouldn't collect + * conn stats without initializing them. */ + rep_hist_note_or_conn_bytes(1, 20, 400, now); + s = rep_hist_format_conn_stats(now + 86400); + test_assert(!s); + + /* Initialize stats, note bytes, and generate history string. */ + rep_hist_conn_stats_init(now); + rep_hist_note_or_conn_bytes(1, 30000, 400000, now); + rep_hist_note_or_conn_bytes(1, 30000, 400000, now + 5); + rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 10); + rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 15); + s = rep_hist_format_conn_stats(now + 86400); + test_streq("conn-bi-direct 2010-08-12 13:27:30 (86400 s) 0,0,1,0\n", s); + tor_free(s); + + /* Stop collecting stats, add some bytes, and ensure we don't generate + * a history string. */ + rep_hist_conn_stats_term(); + rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 15); + s = rep_hist_format_conn_stats(now + 86400); + test_assert(!s); + + /* Re-start stats, add some bytes, reset stats, and see what history we + * get when observing no bytes at all. */ + rep_hist_conn_stats_init(now); + rep_hist_note_or_conn_bytes(1, 30000, 400000, now); + rep_hist_note_or_conn_bytes(1, 30000, 400000, now + 5); + rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 10); + rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 15); + rep_hist_reset_conn_stats(now); + s = rep_hist_format_conn_stats(now + 86400); + test_streq("conn-bi-direct 2010-08-12 13:27:30 (86400 s) 0,0,0,0\n", s); + tor_free(s); + + /* Continue with testing buffer statistics; we shouldn't collect buffer + * stats without initializing them. */ + rep_hist_add_buffer_stats(2.0, 2.0, 20); + s = rep_hist_format_buffer_stats(now + 86400); + test_assert(!s); + + /* Initialize stats, add statistics for a single circuit, and generate + * the history string. */ + rep_hist_buffer_stats_init(now); + rep_hist_add_buffer_stats(2.0, 2.0, 20); + s = rep_hist_format_buffer_stats(now + 86400); + test_streq("cell-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "cell-processed-cells 20,0,0,0,0,0,0,0,0,0\n" + "cell-queued-cells 2.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00," + "0.00,0.00\n" + "cell-time-in-queue 2,0,0,0,0,0,0,0,0,0\n" + "cell-circuits-per-decile 1\n", s); + tor_free(s); + + /* Add nineteen more circuit statistics to the one that's already in the + * history to see that the math works correctly. */ + for (i = 21; i < 30; i++) + rep_hist_add_buffer_stats(2.0, 2.0, i); + for (i = 20; i < 30; i++) + rep_hist_add_buffer_stats(3.5, 3.5, i); + s = rep_hist_format_buffer_stats(now + 86400); + test_streq("cell-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "cell-processed-cells 29,28,27,26,25,24,23,22,21,20\n" + "cell-queued-cells 2.75,2.75,2.75,2.75,2.75,2.75,2.75,2.75," + "2.75,2.75\n" + "cell-time-in-queue 3,3,3,3,3,3,3,3,3,3\n" + "cell-circuits-per-decile 2\n", s); + tor_free(s); + + /* Stop collecting stats, add statistics for one circuit, and ensure we + * don't generate a history string. */ + rep_hist_buffer_stats_term(); + rep_hist_add_buffer_stats(2.0, 2.0, 20); + s = rep_hist_format_buffer_stats(now + 86400); + test_assert(!s); + + /* Re-start stats, add statistics for one circuit, reset stats, and make + * sure that the history has all zeros. */ + rep_hist_buffer_stats_init(now); + rep_hist_add_buffer_stats(2.0, 2.0, 20); + rep_hist_reset_buffer_stats(now); + s = rep_hist_format_buffer_stats(now + 86400); + test_streq("cell-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "cell-processed-cells 0,0,0,0,0,0,0,0,0,0\n" + "cell-queued-cells 0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00," + "0.00,0.00\n" + "cell-time-in-queue 0,0,0,0,0,0,0,0,0,0\n" + "cell-circuits-per-decile 0\n", s); done: tor_free(s); @@ -1180,12 +1899,13 @@ const struct testcase_setup_t legacy_setup = { { #group "_" #name, legacy_test_helper, 0, &legacy_setup, \ test_ ## group ## _ ## name } #define DISABLED(name) \ - { #name, legacy_test_helper, TT_SKIP, &legacy_setup, name } + { #name, legacy_test_helper, TT_SKIP, &legacy_setup, test_ ## name } #define FORK(name) \ { #name, legacy_test_helper, TT_FORK, &legacy_setup, test_ ## name } static struct testcase_t test_array[] = { ENT(buffers), + { "buffer_copy", test_buffer_copy, 0, NULL, NULL }, ENT(onion_handshake), ENT(circuit_timeout), ENT(policies), @@ -1193,8 +1913,23 @@ static struct testcase_t test_array[] = { ENT(geoip), FORK(stats), - DISABLED(bench_aes), - DISABLED(bench_dmap), + END_OF_TESTCASES +}; + +#define SOCKSENT(name) \ + { #name, test_socks_##name, TT_FORK, &socks_setup, NULL } + +static struct testcase_t socks_tests[] = { + SOCKSENT(4_unsupported_commands), + SOCKSENT(4_supported_commands), + + SOCKSENT(5_unsupported_commands), + SOCKSENT(5_supported_commands), + SOCKSENT(5_no_authenticate), + SOCKSENT(5_auth_before_negotiation), + SOCKSENT(5_authenticate), + SOCKSENT(5_authenticate_with_data), + END_OF_TESTCASES }; @@ -1203,14 +1938,21 @@ extern struct testcase_t crypto_tests[]; extern struct testcase_t container_tests[]; extern struct testcase_t util_tests[]; extern struct testcase_t dir_tests[]; +extern struct testcase_t microdesc_tests[]; +extern struct testcase_t pt_tests[]; +extern struct testcase_t config_tests[]; static struct testgroup_t testgroups[] = { { "", test_array }, + { "socks/", socks_tests }, { "addr/", addr_tests }, { "crypto/", crypto_tests }, { "container/", container_tests }, { "util/", util_tests }, { "dir/", dir_tests }, + { "dir/md/", microdesc_tests }, + { "pt/", pt_tests }, + { "config/", config_tests }, END_OF_GROUPS }; @@ -1259,7 +2001,11 @@ main(int c, const char **v) } options->command = CMD_RUN_UNITTESTS; - crypto_global_init(0, NULL, NULL); + if (crypto_global_init(0, NULL, NULL)) { + printf("Can't initialize crypto subsystem; exiting.\n"); + return 1; + } + crypto_set_tls_dh_prime(NULL); rep_hist_init(); network_init(); setup_directory(); diff --git a/src/test/test.h b/src/test/test.h index f7ae46ce6d..0b6e6c60cb 100644 --- a/src/test/test.h +++ b/src/test/test.h @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2003, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #ifndef _TOR_TEST_H @@ -34,18 +34,18 @@ #define test_neq_ptr(expr1, expr2) tt_ptr_op((expr1), !=, (expr2)) #define test_streq(expr1, expr2) tt_str_op((expr1), ==, (expr2)) #define test_strneq(expr1, expr2) tt_str_op((expr1), !=, (expr2)) -#define test_streq(expr1, expr2) tt_str_op((expr1), ==, (expr2)) #define test_mem_op(expr1, op, expr2, len) \ tt_assert_test_fmt_type(expr1,expr2,#expr1" "#op" "#expr2, \ const char *, \ - (memcmp(_val1, _val2, len) op 0), \ + (memcmp(val1_, val2_, len) op 0), \ char *, "%s", \ { size_t printlen = (len)*2+1; \ - _print = tor_malloc(printlen); \ - base16_encode(_print, printlen, _value, \ + print_ = tor_malloc(printlen); \ + base16_encode(print_, printlen, value_, \ (len)); }, \ - { tor_free(_print); } \ + { tor_free(print_); }, \ + TT_EXIT_TEST_FUNCTION \ ); #define test_memeq(expr1, expr2, len) test_mem_op((expr1), ==, (expr2), len) @@ -66,7 +66,7 @@ #define test_memeq_hex(expr1, hex) test_mem_op_hex(expr1, ==, hex) const char *get_fname(const char *name); -crypto_pk_env_t *pk_generate(int idx); +crypto_pk_t *pk_generate(int idx); void legacy_test_helper(void *data); extern const struct testcase_setup_t legacy_setup; diff --git a/src/test/test_addr.c b/src/test/test_addr.c index 1dab0e0112..e3f38073ec 100644 --- a/src/test/test_addr.c +++ b/src/test/test_addr.c @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #include "orconfig.h" @@ -14,30 +14,30 @@ test_addr_basic(void) uint16_t u16; char *cp; - /* Test parse_addr_port */ + /* Test addr_port_lookup */ cp = NULL; u32 = 3; u16 = 3; - test_assert(!parse_addr_port(LOG_WARN, "1.2.3.4", &cp, &u32, &u16)); + test_assert(!addr_port_lookup(LOG_WARN, "1.2.3.4", &cp, &u32, &u16)); test_streq(cp, "1.2.3.4"); test_eq(u32, 0x01020304u); test_eq(u16, 0); tor_free(cp); - test_assert(!parse_addr_port(LOG_WARN, "4.3.2.1:99", &cp, &u32, &u16)); + test_assert(!addr_port_lookup(LOG_WARN, "4.3.2.1:99", &cp, &u32, &u16)); test_streq(cp, "4.3.2.1"); test_eq(u32, 0x04030201u); test_eq(u16, 99); tor_free(cp); - test_assert(!parse_addr_port(LOG_WARN, "nonexistent.address:4040", + test_assert(!addr_port_lookup(LOG_WARN, "nonexistent.address:4040", &cp, NULL, &u16)); test_streq(cp, "nonexistent.address"); test_eq(u16, 4040); tor_free(cp); - test_assert(!parse_addr_port(LOG_WARN, "localhost:9999", &cp, &u32, &u16)); + test_assert(!addr_port_lookup(LOG_WARN, "localhost:9999", &cp, &u32, &u16)); test_streq(cp, "localhost"); test_eq(u32, 0x7f000001u); test_eq(u16, 9999); tor_free(cp); u32 = 3; - test_assert(!parse_addr_port(LOG_WARN, "localhost", NULL, &u32, &u16)); + test_assert(!addr_port_lookup(LOG_WARN, "localhost", NULL, &u32, &u16)); test_eq(cp, NULL); test_eq(u32, 0x7f000001u); test_eq(u16, 0); @@ -53,9 +53,17 @@ test_addr_basic(void) char tmpbuf[TOR_ADDR_BUF_LEN]; const char *ip = "176.192.208.224"; struct in_addr in; - tor_inet_pton(AF_INET, ip, &in); - tor_inet_ntop(AF_INET, &in, tmpbuf, sizeof(tmpbuf)); + + /* good round trip */ + test_eq(tor_inet_pton(AF_INET, ip, &in), 1); + test_eq_ptr(tor_inet_ntop(AF_INET, &in, tmpbuf, sizeof(tmpbuf)), &tmpbuf); test_streq(tmpbuf, ip); + + /* just enough buffer length */ + test_streq(tor_inet_ntop(AF_INET, &in, tmpbuf, strlen(ip) + 1), ip); + + /* too short buffer */ + test_eq_ptr(tor_inet_ntop(AF_INET, &in, tmpbuf, strlen(ip)), NULL); } done: @@ -65,16 +73,18 @@ test_addr_basic(void) #define _test_op_ip6(a,op,b,e1,e2) \ STMT_BEGIN \ tt_assert_test_fmt_type(a,b,e1" "#op" "e2,struct in6_addr*, \ - (memcmp(_val1->s6_addr, _val2->s6_addr, 16) op 0), \ + (memcmp(val1_->s6_addr, val2_->s6_addr, 16) op 0), \ char *, "%s", \ { int i; char *cp; \ - cp = _print = tor_malloc(64); \ + cp = print_ = tor_malloc(64); \ for (i=0;i<16;++i) { \ - tor_snprintf(cp, 3,"%02x", (unsigned)_value->s6_addr[i]);\ + tor_snprintf(cp, 3,"%02x", (unsigned)value_->s6_addr[i]);\ cp += 2; \ if (i != 15) *cp++ = ':'; \ } \ - }, { tor_free(_print); } \ + }, \ + { tor_free(print_); }, \ + TT_EXIT_TEST_FUNCTION \ ); \ STMT_END @@ -165,6 +175,7 @@ static void test_addr_ip6_helpers(void) { char buf[TOR_ADDR_BUF_LEN], bug[TOR_ADDR_BUF_LEN]; + char rbuf[REVERSE_LOOKUP_NAME_BUF_LEN]; struct in6_addr a1, a2; tor_addr_t t1, t2; int r, i; @@ -175,8 +186,30 @@ test_addr_ip6_helpers(void) struct sockaddr_in *sin; struct sockaddr_in6 *sin6; - // struct in_addr b1, b2; /* Test tor_inet_ntop and tor_inet_pton: IPv6 */ + { + const char *ip = "2001::1234"; + const char *ip_ffff = "::ffff:192.168.1.2"; + + /* good round trip */ + test_eq(tor_inet_pton(AF_INET6, ip, &a1), 1); + test_eq_ptr(tor_inet_ntop(AF_INET6, &a1, buf, sizeof(buf)), &buf); + test_streq(buf, ip); + + /* good round trip - ::ffff:0:0 style */ + test_eq(tor_inet_pton(AF_INET6, ip_ffff, &a2), 1); + test_eq_ptr(tor_inet_ntop(AF_INET6, &a2, buf, sizeof(buf)), &buf); + test_streq(buf, ip_ffff); + + /* just long enough buffer (remember \0) */ + test_streq(tor_inet_ntop(AF_INET6, &a1, buf, strlen(ip)+1), ip); + test_streq(tor_inet_ntop(AF_INET6, &a2, buf, strlen(ip_ffff)+1), + ip_ffff); + + /* too short buffer (remember \0) */ + test_eq_ptr(tor_inet_ntop(AF_INET6, &a1, buf, strlen(ip)), NULL); + test_eq_ptr(tor_inet_ntop(AF_INET6, &a2, buf, strlen(ip_ffff)), NULL); + } /* ==== Converting to and from sockaddr_t. */ sin = (struct sockaddr_in *)&sa_storage; @@ -268,12 +301,23 @@ test_addr_ip6_helpers(void) test_ntop6_reduces("1000:0001:0000:0007:0000:0000:0000:0000", "1000:1:0:7::"); + /* Bad af param */ + test_eq(tor_inet_pton(AF_UNSPEC, 0, 0), -1); + /* === Test pton: invalid in6. */ test_pton6_bad("foobar."); + test_pton6_bad("-1::"); + test_pton6_bad("00001::"); + test_pton6_bad("10000::"); + test_pton6_bad("::10000"); test_pton6_bad("55555::"); test_pton6_bad("9:-60::"); + test_pton6_bad("9:+60::"); + test_pton6_bad("9|60::"); + test_pton6_bad("0x60::"); + test_pton6_bad("::0x60"); + test_pton6_bad("9:0x60::"); test_pton6_bad("1:2:33333:4:0002:3::"); - //test_pton6_bad("1:2:3333:4:00002:3::");// BAD, but glibc doesn't say so. test_pton6_bad("1:2:3333:4:fish:3::"); test_pton6_bad("1:2:3:4:5:6:7:8:9"); test_pton6_bad("1:2:3:4:5:6:7"); @@ -281,8 +325,14 @@ test_addr_ip6_helpers(void) test_pton6_bad("1:2:3:4:5:6:1.2.3"); test_pton6_bad("::1.2.3"); test_pton6_bad("::1.2.3.4.5"); + test_pton6_bad("::ffff:0xff.0.0.0"); + test_pton6_bad("::ffff:ff.0.0.0"); + test_pton6_bad("::ffff:256.0.0.0"); + test_pton6_bad("::ffff:-1.0.0.0"); test_pton6_bad("99"); test_pton6_bad(""); + test_pton6_bad("."); + test_pton6_bad(":"); test_pton6_bad("1::2::3:4"); test_pton6_bad("a:::b:c"); test_pton6_bad(":::a:b:c"); @@ -291,6 +341,9 @@ test_addr_ip6_helpers(void) /* test internal checking */ test_external_ip("fbff:ffff::2:7", 0); test_internal_ip("fc01::2:7", 0); + test_internal_ip("fc01::02:7", 0); + test_internal_ip("fc01::002:7", 0); + test_internal_ip("fc01::0002:7", 0); test_internal_ip("fdff:ffff::f:f", 0); test_external_ip("fe00::3:f", 0); @@ -361,33 +414,67 @@ test_addr_ip6_helpers(void) test_addr_compare_masked("0::2:2:1", <, "0::8000:2:1", 81); test_addr_compare_masked("0::2:2:1", ==, "0::8000:2:1", 80); - /* Test decorated addr_to_string. */ - test_eq(AF_INET6, tor_addr_from_str(&t1, "[123:45:6789::5005:11]")); + /* Test undecorated tor_addr_to_str */ + test_eq(AF_INET6, tor_addr_parse(&t1, "[123:45:6789::5005:11]")); + p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 0); + test_streq(p1, "123:45:6789::5005:11"); + test_eq(AF_INET, tor_addr_parse(&t1, "18.0.0.1")); + p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 0); + test_streq(p1, "18.0.0.1"); + + /* Test decorated tor_addr_to_str */ + test_eq(AF_INET6, tor_addr_parse(&t1, "[123:45:6789::5005:11]")); p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); test_streq(p1, "[123:45:6789::5005:11]"); - test_eq(AF_INET, tor_addr_from_str(&t1, "18.0.0.1")); + test_eq(AF_INET, tor_addr_parse(&t1, "18.0.0.1")); p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); test_streq(p1, "18.0.0.1"); - /* Test tor_addr_parse_reverse_lookup_name */ - i = tor_addr_parse_reverse_lookup_name(&t1, "Foobar.baz", AF_UNSPEC, 0); + /* Test buffer bounds checking of tor_addr_to_str */ + test_eq(AF_INET6, tor_addr_parse(&t1, "::")); /* 2 + \0 */ + test_eq_ptr(tor_addr_to_str(buf, &t1, 2, 0), NULL); /* too short buf */ + test_streq(tor_addr_to_str(buf, &t1, 3, 0), "::"); + test_eq_ptr(tor_addr_to_str(buf, &t1, 4, 1), NULL); /* too short buf */ + test_streq(tor_addr_to_str(buf, &t1, 5, 1), "[::]"); + + test_eq(AF_INET6, tor_addr_parse(&t1, "2000::1337")); /* 10 + \0 */ + test_eq_ptr(tor_addr_to_str(buf, &t1, 10, 0), NULL); /* too short buf */ + test_streq(tor_addr_to_str(buf, &t1, 11, 0), "2000::1337"); + test_eq_ptr(tor_addr_to_str(buf, &t1, 12, 1), NULL); /* too short buf */ + test_streq(tor_addr_to_str(buf, &t1, 13, 1), "[2000::1337]"); + + test_eq(AF_INET, tor_addr_parse(&t1, "1.2.3.4")); /* 7 + \0 */ + test_eq_ptr(tor_addr_to_str(buf, &t1, 7, 0), NULL); /* too short buf */ + test_streq(tor_addr_to_str(buf, &t1, 8, 0), "1.2.3.4"); + + test_eq(AF_INET, tor_addr_parse(&t1, "255.255.255.255")); /* 15 + \0 */ + test_eq_ptr(tor_addr_to_str(buf, &t1, 15, 0), NULL); /* too short buf */ + test_streq(tor_addr_to_str(buf, &t1, 16, 0), "255.255.255.255"); + test_eq_ptr(tor_addr_to_str(buf, &t1, 15, 1), NULL); /* too short buf */ + test_streq(tor_addr_to_str(buf, &t1, 16, 1), "255.255.255.255"); + + t1.family = AF_UNSPEC; + test_eq_ptr(tor_addr_to_str(buf, &t1, sizeof(buf), 0), NULL); + + /* Test tor_addr_parse_PTR_name */ + i = tor_addr_parse_PTR_name(&t1, "Foobar.baz", AF_UNSPEC, 0); test_eq(0, i); - i = tor_addr_parse_reverse_lookup_name(&t1, "Foobar.baz", AF_UNSPEC, 1); + i = tor_addr_parse_PTR_name(&t1, "Foobar.baz", AF_UNSPEC, 1); test_eq(0, i); - i = tor_addr_parse_reverse_lookup_name(&t1, "1.0.168.192.in-addr.arpa", + i = tor_addr_parse_PTR_name(&t1, "1.0.168.192.in-addr.arpa", AF_UNSPEC, 1); test_eq(1, i); test_eq(tor_addr_family(&t1), AF_INET); p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); test_streq(p1, "192.168.0.1"); - i = tor_addr_parse_reverse_lookup_name(&t1, "192.168.0.99", AF_UNSPEC, 0); + i = tor_addr_parse_PTR_name(&t1, "192.168.0.99", AF_UNSPEC, 0); test_eq(0, i); - i = tor_addr_parse_reverse_lookup_name(&t1, "192.168.0.99", AF_UNSPEC, 1); + i = tor_addr_parse_PTR_name(&t1, "192.168.0.99", AF_UNSPEC, 1); test_eq(1, i); p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); test_streq(p1, "192.168.0.99"); memset(&t1, 0, sizeof(t1)); - i = tor_addr_parse_reverse_lookup_name(&t1, + i = tor_addr_parse_PTR_name(&t1, "0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f." "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." "ip6.ARPA", @@ -396,43 +483,91 @@ test_addr_ip6_helpers(void) p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); test_streq(p1, "[9dee:effe:ebe1:beef:fedc:ba98:7654:3210]"); /* Failing cases. */ - i = tor_addr_parse_reverse_lookup_name(&t1, + i = tor_addr_parse_PTR_name(&t1, "6.7.8.9.a.b.c.d.e.f." "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." "ip6.ARPA", AF_UNSPEC, 0); test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, + i = tor_addr_parse_PTR_name(&t1, "6.7.8.9.a.b.c.d.e.f.a.b.c.d.e.f.0." "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." "ip6.ARPA", AF_UNSPEC, 0); test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, + i = tor_addr_parse_PTR_name(&t1, "6.7.8.9.a.b.c.d.e.f.X.0.0.0.0.9." "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." "ip6.ARPA", AF_UNSPEC, 0); test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, "32.1.1.in-addr.arpa", + i = tor_addr_parse_PTR_name(&t1, "32.1.1.in-addr.arpa", AF_UNSPEC, 0); test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, ".in-addr.arpa", + i = tor_addr_parse_PTR_name(&t1, ".in-addr.arpa", AF_UNSPEC, 0); test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, "1.2.3.4.5.in-addr.arpa", + i = tor_addr_parse_PTR_name(&t1, "1.2.3.4.5.in-addr.arpa", AF_UNSPEC, 0); test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, "1.2.3.4.5.in-addr.arpa", + i = tor_addr_parse_PTR_name(&t1, "1.2.3.4.5.in-addr.arpa", AF_INET6, 0); test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, + i = tor_addr_parse_PTR_name(&t1, "6.7.8.9.a.b.c.d.e.f.a.b.c.d.e.0." "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." "ip6.ARPA", AF_INET, 0); test_eq(i, -1); + /* === Test tor_addr_to_PTR_name */ + + /* Stage IPv4 addr */ + memset(&sa_storage, 0, sizeof(sa_storage)); + sin = (struct sockaddr_in *)&sa_storage; + sin->sin_family = AF_INET; + sin->sin_addr.s_addr = htonl(0x7f010203); /* 127.1.2.3 */ + tor_addr_from_sockaddr(&t1, (struct sockaddr *)sin, NULL); + + /* Check IPv4 PTR - too short buffer */ + test_eq(tor_addr_to_PTR_name(rbuf, 1, &t1), -1); + test_eq(tor_addr_to_PTR_name(rbuf, + strlen("3.2.1.127.in-addr.arpa") - 1, + &t1), -1); + + /* Check IPv4 PTR - valid addr */ + test_eq(tor_addr_to_PTR_name(rbuf, sizeof(rbuf), &t1), + strlen("3.2.1.127.in-addr.arpa")); + test_streq(rbuf, "3.2.1.127.in-addr.arpa"); + + /* Invalid addr family */ + t1.family = AF_UNSPEC; + test_eq(tor_addr_to_PTR_name(rbuf, sizeof(rbuf), &t1), -1); + + /* Stage IPv6 addr */ + memset(&sa_storage, 0, sizeof(sa_storage)); + sin6 = (struct sockaddr_in6 *)&sa_storage; + sin6->sin6_family = AF_INET6; + sin6->sin6_addr.s6_addr[0] = 0x80; /* 8000::abcd */ + sin6->sin6_addr.s6_addr[14] = 0xab; + sin6->sin6_addr.s6_addr[15] = 0xcd; + + tor_addr_from_sockaddr(&t1, (struct sockaddr *)sin6, NULL); + + { + const char* addr_PTR = "d.c.b.a.0.0.0.0.0.0.0.0.0.0.0.0." + "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.ip6.arpa"; + + /* Check IPv6 PTR - too short buffer */ + test_eq(tor_addr_to_PTR_name(rbuf, 0, &t1), -1); + test_eq(tor_addr_to_PTR_name(rbuf, strlen(addr_PTR) - 1, &t1), -1); + + /* Check IPv6 PTR - valid addr */ + test_eq(tor_addr_to_PTR_name(rbuf, sizeof(rbuf), &t1), + strlen(addr_PTR)); + test_streq(rbuf, addr_PTR); + } + /* test tor_addr_parse_mask_ports */ test_addr_mask_ports_parse("[::f]/17:47-95", AF_INET6, 0, 0, 0, 0x0000000f, 17, 47, 95); @@ -478,12 +613,46 @@ test_addr_ip6_helpers(void) /* get interface addresses */ r = get_interface_address6(LOG_DEBUG, AF_INET, &t1); i = get_interface_address6(LOG_DEBUG, AF_INET6, &t2); -#if 0 - tor_inet_ntop(AF_INET, &t1.sa.sin_addr, buf, sizeof(buf)); - printf("\nv4 address: %s (family=%d)", buf, IN_FAMILY(&t1)); - tor_inet_ntop(AF_INET6, &t2.sa6.sin6_addr, buf, sizeof(buf)); - printf("\nv6 address: %s (family=%d)", buf, IN_FAMILY(&t2)); -#endif + + TT_BLATHER(("v4 address: %s (family=%d)", fmt_addr(&t1), + tor_addr_family(&t1))); + TT_BLATHER(("v6 address: %s (family=%d)", fmt_addr(&t2), + tor_addr_family(&t2))); + + done: + ; +} + +static void +test_addr_is_loopback(void *data) +{ + static const struct loopback_item { + const char *name; + int is_loopback; + } loopback_items[] = { + { "::1", 1 }, + { "127.0.0.1", 1 }, + { "127.99.100.101", 1 }, + { "128.99.100.101", 0 }, + { "8.8.8.8", 0 }, + { "0.0.0.0", 0 }, + { "::2", 0 }, + { "::", 0 }, + { "::1.0.0.0", 0 }, + { NULL, 0 } + }; + + int i; + tor_addr_t addr; + (void)data; + + for (i=0; loopback_items[i].name; ++i) { + tt_int_op(tor_addr_parse(&addr, loopback_items[i].name), >=, 0); + tt_int_op(tor_addr_is_loopback(&addr), ==, loopback_items[i].is_loopback); + } + + tor_addr_make_unspec(&addr); + tt_int_op(tor_addr_is_loopback(&addr), ==, 0); done: ; @@ -495,6 +664,7 @@ test_addr_ip6_helpers(void) struct testcase_t addr_tests[] = { ADDR_LEGACY(basic), ADDR_LEGACY(ip6_helpers), + { "is_loopback", test_addr_is_loopback, 0, NULL, NULL }, END_OF_TESTCASES }; diff --git a/src/test/test_config.c b/src/test/test_config.c new file mode 100644 index 0000000000..ff251a24d8 --- /dev/null +++ b/src/test/test_config.c @@ -0,0 +1,170 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#include "or.h" +#include "config.h" +#include "connection_edge.h" +#include "test.h" + +static void +test_config_addressmap(void *arg) +{ + char buf[1024]; + char address[256]; + time_t expires = TIME_MAX; + (void)arg; + + strlcpy(buf, "MapAddress .invalidwildcard.com *.torserver.exit\n" // invalid + "MapAddress *invalidasterisk.com *.torserver.exit\n" // invalid + "MapAddress *.google.com *.torserver.exit\n" + "MapAddress *.yahoo.com *.google.com.torserver.exit\n" + "MapAddress *.cn.com www.cnn.com\n" + "MapAddress *.cnn.com www.cnn.com\n" + "MapAddress ex.com www.cnn.com\n" + "MapAddress ey.com *.cnn.com\n" + "MapAddress www.torproject.org 1.1.1.1\n" + "MapAddress other.torproject.org " + "this.torproject.org.otherserver.exit\n" + "MapAddress test.torproject.org 2.2.2.2\n" + "MapAddress www.google.com 3.3.3.3\n" + "MapAddress www.example.org 4.4.4.4\n" + "MapAddress 4.4.4.4 7.7.7.7\n" + "MapAddress 4.4.4.4 5.5.5.5\n" + "MapAddress www.infiniteloop.org 6.6.6.6\n" + "MapAddress 6.6.6.6 www.infiniteloop.org\n" + , sizeof(buf)); + + config_get_lines(buf, &(get_options_mutable()->AddressMap), 0); + config_register_addressmaps(get_options()); + + /* MapAddress .invalidwildcard.com .torserver.exit - no match */ + strlcpy(address, "www.invalidwildcard.com", sizeof(address)); + test_assert(!addressmap_rewrite(address, sizeof(address), &expires, NULL)); + + /* MapAddress *invalidasterisk.com .torserver.exit - no match */ + strlcpy(address, "www.invalidasterisk.com", sizeof(address)); + test_assert(!addressmap_rewrite(address, sizeof(address), &expires, NULL)); + + /* Where no mapping for FQDN match on top-level domain */ + /* MapAddress .google.com .torserver.exit */ + strlcpy(address, "reader.google.com", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "reader.torserver.exit"); + + /* MapAddress *.yahoo.com *.google.com.torserver.exit */ + strlcpy(address, "reader.yahoo.com", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "reader.google.com.torserver.exit"); + + /*MapAddress *.cnn.com www.cnn.com */ + strlcpy(address, "cnn.com", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "www.cnn.com"); + + /* MapAddress .cn.com www.cnn.com */ + strlcpy(address, "www.cn.com", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "www.cnn.com"); + + /* MapAddress ex.com www.cnn.com - no match */ + strlcpy(address, "www.ex.com", sizeof(address)); + test_assert(!addressmap_rewrite(address, sizeof(address), &expires, NULL)); + + /* MapAddress ey.com *.cnn.com - invalid expression */ + strlcpy(address, "ey.com", sizeof(address)); + test_assert(!addressmap_rewrite(address, sizeof(address), &expires, NULL)); + + /* Where mapping for FQDN match on FQDN */ + strlcpy(address, "www.google.com", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "3.3.3.3"); + + strlcpy(address, "www.torproject.org", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "1.1.1.1"); + + strlcpy(address, "other.torproject.org", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "this.torproject.org.otherserver.exit"); + + strlcpy(address, "test.torproject.org", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "2.2.2.2"); + + /* Test a chain of address mappings and the order in which they were added: + "MapAddress www.example.org 4.4.4.4" + "MapAddress 4.4.4.4 7.7.7.7" + "MapAddress 4.4.4.4 5.5.5.5" + */ + strlcpy(address, "www.example.org", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "5.5.5.5"); + + /* Test infinite address mapping results in no change */ + strlcpy(address, "www.infiniteloop.org", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "www.infiniteloop.org"); + + /* Test we don't find false positives */ + strlcpy(address, "www.example.com", sizeof(address)); + test_assert(!addressmap_rewrite(address, sizeof(address), &expires, NULL)); + + /* Test top-level-domain matching a bit harder */ + addressmap_clear_configured(); + strlcpy(buf, "MapAddress *.com *.torserver.exit\n" + "MapAddress *.torproject.org 1.1.1.1\n" + "MapAddress *.net 2.2.2.2\n" + , sizeof(buf)); + config_get_lines(buf, &(get_options_mutable()->AddressMap), 0); + config_register_addressmaps(get_options()); + + strlcpy(address, "www.abc.com", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "www.abc.torserver.exit"); + + strlcpy(address, "www.def.com", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "www.def.torserver.exit"); + + strlcpy(address, "www.torproject.org", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "1.1.1.1"); + + strlcpy(address, "test.torproject.org", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "1.1.1.1"); + + strlcpy(address, "torproject.net", sizeof(address)); + test_assert(addressmap_rewrite(address, sizeof(address), &expires, NULL)); + test_streq(address, "2.2.2.2"); + + /* We don't support '*' as a mapping directive */ + addressmap_clear_configured(); + strlcpy(buf, "MapAddress * *.torserver.exit\n", sizeof(buf)); + config_get_lines(buf, &(get_options_mutable()->AddressMap), 0); + config_register_addressmaps(get_options()); + + strlcpy(address, "www.abc.com", sizeof(address)); + test_assert(!addressmap_rewrite(address, sizeof(address), &expires, NULL)); + + strlcpy(address, "www.def.net", sizeof(address)); + test_assert(!addressmap_rewrite(address, sizeof(address), &expires, NULL)); + + strlcpy(address, "www.torproject.org", sizeof(address)); + test_assert(!addressmap_rewrite(address, sizeof(address), &expires, NULL)); + + done: + ; +} + +#define CONFIG_TEST(name, flags) \ + { #name, test_config_ ## name, flags, NULL, NULL } + +struct testcase_t config_tests[] = { + CONFIG_TEST(addressmap, 0), + END_OF_TESTCASES +}; + diff --git a/src/test/test_containers.c b/src/test/test_containers.c index 46cc82d6c3..45898df4eb 100644 --- a/src/test/test_containers.c +++ b/src/test/test_containers.c @@ -1,6 +1,6 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #include "orconfig.h" @@ -43,7 +43,7 @@ test_container_smartlist_basic(void) /* XXXX test sort_digests, uniq_strings, uniq_digests */ /* Test smartlist add, del_keeporder, insert, get. */ - sl = smartlist_create(); + sl = smartlist_new(); smartlist_add(sl, (void*)1); smartlist_add(sl, (void*)2); smartlist_add(sl, (void*)3); @@ -77,7 +77,7 @@ test_container_smartlist_basic(void) static void test_container_smartlist_strings(void) { - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); char *cp=NULL, *cp_alloc=NULL; size_t sz; @@ -229,7 +229,7 @@ test_container_smartlist_strings(void) test_eq(f, 0); /* Test trivial cases for list of length 0 or 1 */ - tmp = smartlist_create(); + tmp = smartlist_new(); test_eq(0, smartlist_bsearch_idx(tmp, "foo", compare_strs_for_bsearch_, &f)); test_eq(f, 0); @@ -334,11 +334,11 @@ test_container_smartlist_strings(void) static void test_container_smartlist_overlap(void) { - smartlist_t *sl = smartlist_create(); - smartlist_t *ints = smartlist_create(); - smartlist_t *odds = smartlist_create(); - smartlist_t *evens = smartlist_create(); - smartlist_t *primes = smartlist_create(); + smartlist_t *sl = smartlist_new(); + smartlist_t *ints = smartlist_new(); + smartlist_t *odds = smartlist_new(); + smartlist_t *evens = smartlist_new(); + smartlist_t *primes = smartlist_new(); int i; for (i=1; i < 10; i += 2) smartlist_add(odds, (void*)(uintptr_t)i); @@ -387,7 +387,7 @@ test_container_smartlist_overlap(void) static void test_container_smartlist_digests(void) { - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); /* digest_isin. */ smartlist_add(sl, tor_memdup("AAAAAAAAAAAAAAAAAAAA", DIGEST_LEN)); @@ -420,9 +420,9 @@ test_container_smartlist_digests(void) static void test_container_smartlist_join(void) { - smartlist_t *sl = smartlist_create(); - smartlist_t *sl2 = smartlist_create(), *sl3 = smartlist_create(), - *sl4 = smartlist_create(); + smartlist_t *sl = smartlist_new(); + smartlist_t *sl2 = smartlist_new(), *sl3 = smartlist_new(), + *sl4 = smartlist_new(); char *joined=NULL; /* unique, sorted. */ smartlist_split_string(sl, @@ -515,7 +515,7 @@ test_container_bitarray(void) static void test_container_digestset(void) { - smartlist_t *included = smartlist_create(); + smartlist_t *included = smartlist_new(); char d[DIGEST_LEN]; int i; int ok = 1; @@ -568,7 +568,7 @@ _compare_strings_for_pqueue(const void *p1, const void *p2) static void test_container_pqueue(void) { - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); int (*cmp)(const void *, const void*); const int offset = STRUCT_OFFSET(pq_entry_t, idx); #define ENTRY(s) pq_entry_t s = { #s, -1 } @@ -705,7 +705,7 @@ test_container_strmap(void) /* Test iterator. */ iter = strmap_iter_init(map); - found_keys = smartlist_create(); + found_keys = smartlist_new(); while (!strmap_iter_done(iter)) { strmap_iter_get(iter,&k,&v); smartlist_add(found_keys, tor_strdup(k)); diff --git a/src/test/test_crypto.c b/src/test/test_crypto.c index 85a4e929a4..7f4347a41c 100644 --- a/src/test/test_crypto.c +++ b/src/test/test_crypto.c @@ -1,19 +1,20 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #include "orconfig.h" #define CRYPTO_PRIVATE #include "or.h" #include "test.h" +#include "aes.h" /** Run unit tests for Diffie-Hellman functionality. */ static void test_crypto_dh(void) { - crypto_dh_env_t *dh1 = crypto_dh_new(DH_TYPE_CIRCUIT); - crypto_dh_env_t *dh2 = crypto_dh_new(DH_TYPE_CIRCUIT); + crypto_dh_t *dh1 = crypto_dh_new(DH_TYPE_CIRCUIT); + crypto_dh_t *dh2 = crypto_dh_new(DH_TYPE_CIRCUIT); char p1[DH_BYTES]; char p2[DH_BYTES]; char s1[DH_BYTES]; @@ -95,13 +96,17 @@ test_crypto_rng(void) /** Run unit tests for our AES functionality */ static void -test_crypto_aes(void) +test_crypto_aes(void *arg) { char *data1 = NULL, *data2 = NULL, *data3 = NULL; - crypto_cipher_env_t *env1 = NULL, *env2 = NULL; + crypto_cipher_t *env1 = NULL, *env2 = NULL; int i, j; char *mem_op_hex_tmp=NULL; + int use_evp = !strcmp(arg,"evp"); + evaluate_evp_for_aes(use_evp); + evaluate_ctr_for_aes(); + data1 = tor_malloc(1024); data2 = tor_malloc(1024); data3 = tor_malloc(1024); @@ -113,14 +118,10 @@ test_crypto_aes(void) memset(data2, 0, 1024); memset(data3, 0, 1024); - env1 = crypto_new_cipher_env(); + env1 = crypto_cipher_new(NULL); test_neq(env1, 0); - env2 = crypto_new_cipher_env(); + env2 = crypto_cipher_new(crypto_cipher_get_key(env1)); test_neq(env2, 0); - j = crypto_cipher_generate_key(env1); - crypto_cipher_set_key(env2, crypto_cipher_get_key(env1)); - crypto_cipher_encrypt_init_cipher(env1); - crypto_cipher_decrypt_init_cipher(env2); /* Try encrypting 512 chars. */ crypto_cipher_encrypt(env1, data2, data1, 512); @@ -146,14 +147,12 @@ test_crypto_aes(void) test_memeq(data1, data3, 1024-5); /* Now make sure that when we encrypt with different chunk sizes, we get the same results. */ - crypto_free_cipher_env(env2); + crypto_cipher_free(env2); env2 = NULL; memset(data3, 0, 1024); - env2 = crypto_new_cipher_env(); + env2 = crypto_cipher_new(crypto_cipher_get_key(env1)); test_neq(env2, 0); - crypto_cipher_set_key(env2, crypto_cipher_get_key(env1)); - crypto_cipher_encrypt_init_cipher(env2); for (j = 0; j < 1024-16; j += 17) { crypto_cipher_encrypt(env2, data3+j, data1+j, 17); } @@ -163,16 +162,15 @@ test_crypto_aes(void) } } test_memeq(data2, data3, 1024-16); - crypto_free_cipher_env(env1); + crypto_cipher_free(env1); env1 = NULL; - crypto_free_cipher_env(env2); + crypto_cipher_free(env2); env2 = NULL; /* NIST test vector for aes. */ - env1 = crypto_new_cipher_env(); /* IV starts at 0 */ - crypto_cipher_set_key(env1, "\x80\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00"); - crypto_cipher_encrypt_init_cipher(env1); + /* IV starts at 0 */ + env1 = crypto_cipher_new("\x80\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00"); crypto_cipher_encrypt(env1, data1, "\x00\x00\x00\x00\x00\x00\x00\x00" "\x00\x00\x00\x00\x00\x00\x00\x00", 16); @@ -180,46 +178,64 @@ test_crypto_aes(void) /* Now test rollover. All these values are originally from a python * script. */ - crypto_cipher_set_iv(env1, "\x00\x00\x00\x00\x00\x00\x00\x00" - "\xff\xff\xff\xff\xff\xff\xff\xff"); + crypto_cipher_free(env1); + env1 = crypto_cipher_new_with_iv( + "\x80\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00", + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\xff\xff\xff\xff\xff\xff\xff\xff"); memset(data2, 0, 1024); crypto_cipher_encrypt(env1, data1, data2, 32); test_memeq_hex(data1, "335fe6da56f843199066c14a00a40231" "cdd0b917dbc7186908a6bfb5ffd574d3"); - - crypto_cipher_set_iv(env1, "\x00\x00\x00\x00\xff\xff\xff\xff" - "\xff\xff\xff\xff\xff\xff\xff\xff"); + crypto_cipher_free(env1); + env1 = crypto_cipher_new_with_iv( + "\x80\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00", + "\x00\x00\x00\x00\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff"); memset(data2, 0, 1024); crypto_cipher_encrypt(env1, data1, data2, 32); test_memeq_hex(data1, "e627c6423fa2d77832a02b2794094b73" "3e63c721df790d2c6469cc1953a3ffac"); - - crypto_cipher_set_iv(env1, "\xff\xff\xff\xff\xff\xff\xff\xff" - "\xff\xff\xff\xff\xff\xff\xff\xff"); + crypto_cipher_free(env1); + env1 = crypto_cipher_new_with_iv( + "\x80\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00", + "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff"); memset(data2, 0, 1024); crypto_cipher_encrypt(env1, data1, data2, 32); test_memeq_hex(data1, "2aed2bff0de54f9328efd070bf48f70a" "0EDD33D3C621E546455BD8BA1418BEC8"); /* Now check rollover on inplace cipher. */ - crypto_cipher_set_iv(env1, "\xff\xff\xff\xff\xff\xff\xff\xff" - "\xff\xff\xff\xff\xff\xff\xff\xff"); + crypto_cipher_free(env1); + env1 = crypto_cipher_new_with_iv( + "\x80\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00", + "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff"); crypto_cipher_crypt_inplace(env1, data2, 64); test_memeq_hex(data2, "2aed2bff0de54f9328efd070bf48f70a" "0EDD33D3C621E546455BD8BA1418BEC8" "93e2c5243d6839eac58503919192f7ae" "1908e67cafa08d508816659c2e693191"); - crypto_cipher_set_iv(env1, "\xff\xff\xff\xff\xff\xff\xff\xff" - "\xff\xff\xff\xff\xff\xff\xff\xff"); + crypto_cipher_free(env1); + env1 = crypto_cipher_new_with_iv( + "\x80\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00", + "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff"); crypto_cipher_crypt_inplace(env1, data2, 64); test_assert(tor_mem_is_zero(data2, 64)); done: tor_free(mem_op_hex_tmp); if (env1) - crypto_free_cipher_env(env1); + crypto_cipher_free(env1); if (env2) - crypto_free_cipher_env(env2); + crypto_cipher_free(env2); tor_free(data1); tor_free(data2); tor_free(data3); @@ -229,7 +245,7 @@ test_crypto_aes(void) static void test_crypto_sha(void) { - crypto_digest_env_t *d1 = NULL, *d2 = NULL; + crypto_digest_t *d1 = NULL, *d2 = NULL; int i; char key[160]; char digest[32]; @@ -346,7 +362,7 @@ test_crypto_sha(void) "bfdc63644f0713938a7f51535c3a35e2"); /* Incremental digest code. */ - d1 = crypto_new_digest_env(); + d1 = crypto_digest_new(); test_assert(d1); crypto_digest_add_bytes(d1, "abcdef", 6); d2 = crypto_digest_dup(d1); @@ -363,11 +379,11 @@ test_crypto_sha(void) crypto_digest_get_digest(d1, d_out1, sizeof(d_out1)); crypto_digest(d_out2, "abcdef", 6); test_memeq(d_out1, d_out2, DIGEST_LEN); - crypto_free_digest_env(d1); - crypto_free_digest_env(d2); + crypto_digest_free(d1); + crypto_digest_free(d2); /* Incremental digest code with sha256 */ - d1 = crypto_new_digest256_env(DIGEST_SHA256); + d1 = crypto_digest256_new(DIGEST_SHA256); test_assert(d1); crypto_digest_add_bytes(d1, "abcdef", 6); d2 = crypto_digest_dup(d1); @@ -387,9 +403,9 @@ test_crypto_sha(void) done: if (d1) - crypto_free_digest_env(d1); + crypto_digest_free(d1); if (d2) - crypto_free_digest_env(d2); + crypto_digest_free(d2); tor_free(mem_op_hex_tmp); } @@ -397,7 +413,7 @@ test_crypto_sha(void) static void test_crypto_pk(void) { - crypto_pk_env_t *pk1 = NULL, *pk2 = NULL; + crypto_pk_t *pk1 = NULL, *pk2 = NULL; char *encoded = NULL; char data1[1024], data2[1024], data3[1024]; size_t size; @@ -405,7 +421,7 @@ test_crypto_pk(void) /* Public-key ciphers */ pk1 = pk_generate(0); - pk2 = crypto_new_pk_env(); + pk2 = crypto_pk_new(); test_assert(pk1 && pk2); test_assert(! crypto_pk_write_public_key_to_string(pk1, &encoded, &size)); test_assert(! crypto_pk_read_public_key_from_string(pk2, encoded, size)); @@ -471,7 +487,7 @@ test_crypto_pk(void) /*XXXX test failed signing*/ /* Try encoding */ - crypto_free_pk_env(pk2); + crypto_pk_free(pk2); pk2 = NULL; i = crypto_pk_asn1_encode(pk1, data1, 1024); test_assert(i>0); @@ -480,14 +496,11 @@ test_crypto_pk(void) /* Try with hybrid encryption wrappers. */ crypto_rand(data1, 1024); - for (i = 0; i < 3; ++i) { + for (i = 0; i < 2; ++i) { for (j = 85; j < 140; ++j) { memset(data2,0,1024); memset(data3,0,1024); - if (i == 0 && j < 129) - continue; - p = (i==0)?PK_NO_PADDING: - (i==1)?PK_PKCS1_PADDING:PK_PKCS1_OAEP_PADDING; + p = (i==0)?PK_PKCS1_PADDING:PK_PKCS1_OAEP_PADDING; len = crypto_pk_public_hybrid_encrypt(pk1,data2,sizeof(data2), data1,j,p,0); test_assert(len>=0); @@ -499,7 +512,7 @@ test_crypto_pk(void) } /* Try copy_full */ - crypto_free_pk_env(pk2); + crypto_pk_free(pk2); pk2 = crypto_pk_copy_full(pk1); test_assert(pk2 != NULL); test_neq_ptr(pk1, pk2); @@ -507,9 +520,9 @@ test_crypto_pk(void) done: if (pk1) - crypto_free_pk_env(pk1); + crypto_pk_free(pk1); if (pk2) - crypto_free_pk_env(pk2); + crypto_pk_free(pk2); tor_free(encoded); } @@ -670,14 +683,16 @@ test_crypto_s2k(void) /** Test AES-CTR encryption and decryption with IV. */ static void -test_crypto_aes_iv(void) +test_crypto_aes_iv(void *arg) { - crypto_cipher_env_t *cipher; char *plain, *encrypted1, *encrypted2, *decrypted1, *decrypted2; char plain_1[1], plain_15[15], plain_16[16], plain_17[17]; char key1[16], key2[16]; ssize_t encrypted_size, decrypted_size; + int use_evp = !strcmp(arg,"evp"); + evaluate_evp_for_aes(use_evp); + plain = tor_malloc(4095); encrypted1 = tor_malloc(4095 + 1 + 16); encrypted2 = tor_malloc(4095 + 1 + 16); @@ -693,116 +708,79 @@ test_crypto_aes_iv(void) crypto_rand(plain_17, 17); key1[0] = key2[0] + 128; /* Make sure that contents are different. */ /* Encrypt and decrypt with the same key. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 4095, + encrypted_size = crypto_cipher_encrypt_with_iv(key1, encrypted1, 16 + 4095, plain, 4095); - crypto_free_cipher_env(cipher); - cipher = NULL; + test_eq(encrypted_size, 16 + 4095); - tor_assert(encrypted_size > 0); /* This is obviously true, since 4111 is + tt_assert(encrypted_size > 0); /* This is obviously true, since 4111 is * greater than 0, but its truth is not * obvious to all analysis tools. */ - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 4095, + decrypted_size = crypto_cipher_decrypt_with_iv(key1, decrypted1, 4095, encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; + test_eq(decrypted_size, 4095); - tor_assert(decrypted_size > 0); + tt_assert(decrypted_size > 0); test_memeq(plain, decrypted1, 4095); /* Encrypt a second time (with a new random initialization vector). */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted2, 16 + 4095, + encrypted_size = crypto_cipher_encrypt_with_iv(key1, encrypted2, 16 + 4095, plain, 4095); - crypto_free_cipher_env(cipher); - cipher = NULL; + test_eq(encrypted_size, 16 + 4095); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted2, 4095, + tt_assert(encrypted_size > 0); + decrypted_size = crypto_cipher_decrypt_with_iv(key1, decrypted2, 4095, encrypted2, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; test_eq(decrypted_size, 4095); - tor_assert(decrypted_size > 0); + tt_assert(decrypted_size > 0); test_memeq(plain, decrypted2, 4095); test_memneq(encrypted1, encrypted2, encrypted_size); /* Decrypt with the wrong key. */ - cipher = crypto_create_init_cipher(key2, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted2, 4095, + decrypted_size = crypto_cipher_decrypt_with_iv(key2, decrypted2, 4095, encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; test_memneq(plain, decrypted2, encrypted_size); /* Alter the initialization vector. */ encrypted1[0] += 42; - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 4095, + decrypted_size = crypto_cipher_decrypt_with_iv(key1, decrypted1, 4095, encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; test_memneq(plain, decrypted2, 4095); /* Special length case: 1. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 1, + encrypted_size = crypto_cipher_encrypt_with_iv(key1, encrypted1, 16 + 1, plain_1, 1); - crypto_free_cipher_env(cipher); - cipher = NULL; test_eq(encrypted_size, 16 + 1); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 1, + tt_assert(encrypted_size > 0); + decrypted_size = crypto_cipher_decrypt_with_iv(key1, decrypted1, 1, encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; test_eq(decrypted_size, 1); - tor_assert(decrypted_size > 0); + tt_assert(decrypted_size > 0); test_memeq(plain_1, decrypted1, 1); /* Special length case: 15. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 15, + encrypted_size = crypto_cipher_encrypt_with_iv(key1, encrypted1, 16 + 15, plain_15, 15); - crypto_free_cipher_env(cipher); - cipher = NULL; test_eq(encrypted_size, 16 + 15); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 15, + tt_assert(encrypted_size > 0); + decrypted_size = crypto_cipher_decrypt_with_iv(key1, decrypted1, 15, encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; test_eq(decrypted_size, 15); - tor_assert(decrypted_size > 0); + tt_assert(decrypted_size > 0); test_memeq(plain_15, decrypted1, 15); /* Special length case: 16. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 16, + encrypted_size = crypto_cipher_encrypt_with_iv(key1, encrypted1, 16 + 16, plain_16, 16); - crypto_free_cipher_env(cipher); - cipher = NULL; test_eq(encrypted_size, 16 + 16); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 16, + tt_assert(encrypted_size > 0); + decrypted_size = crypto_cipher_decrypt_with_iv(key1, decrypted1, 16, encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; test_eq(decrypted_size, 16); - tor_assert(decrypted_size > 0); + tt_assert(decrypted_size > 0); test_memeq(plain_16, decrypted1, 16); /* Special length case: 17. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 17, + encrypted_size = crypto_cipher_encrypt_with_iv(key1, encrypted1, 16 + 17, plain_17, 17); - crypto_free_cipher_env(cipher); - cipher = NULL; test_eq(encrypted_size, 16 + 17); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 17, + tt_assert(encrypted_size > 0); + decrypted_size = crypto_cipher_decrypt_with_iv(key1, decrypted1, 17, encrypted1, encrypted_size); test_eq(decrypted_size, 17); - tor_assert(decrypted_size > 0); + tt_assert(decrypted_size > 0); test_memeq(plain_17, decrypted1, 17); done: @@ -812,8 +790,6 @@ test_crypto_aes_iv(void) tor_free(encrypted2); tor_free(decrypted1); tor_free(decrypted2); - if (cipher) - crypto_free_cipher_env(cipher); } /** Test base32 decoding. */ @@ -851,18 +827,36 @@ test_crypto_base32_decode(void) ; } +static void * +pass_data_setup_fn(const struct testcase_t *testcase) +{ + return testcase->setup_data; +} +static int +pass_data_cleanup_fn(const struct testcase_t *testcase, void *ptr) +{ + (void)ptr; + (void)testcase; + return 1; +} +static const struct testcase_setup_t pass_data = { + pass_data_setup_fn, pass_data_cleanup_fn +}; + #define CRYPTO_LEGACY(name) \ { #name, legacy_test_helper, 0, &legacy_setup, test_crypto_ ## name } struct testcase_t crypto_tests[] = { CRYPTO_LEGACY(formats), CRYPTO_LEGACY(rng), - CRYPTO_LEGACY(aes), + { "aes_AES", test_crypto_aes, TT_FORK, &pass_data, (void*)"aes" }, + { "aes_EVP", test_crypto_aes, TT_FORK, &pass_data, (void*)"evp" }, CRYPTO_LEGACY(sha), CRYPTO_LEGACY(pk), CRYPTO_LEGACY(dh), CRYPTO_LEGACY(s2k), - CRYPTO_LEGACY(aes_iv), + { "aes_iv_AES", test_crypto_aes_iv, TT_FORK, &pass_data, (void*)"aes" }, + { "aes_iv_EVP", test_crypto_aes_iv, TT_FORK, &pass_data, (void*)"evp" }, CRYPTO_LEGACY(base32_decode), END_OF_TESTCASES }; diff --git a/src/test/test_data.c b/src/test/test_data.c index fc85857615..de2f9f58eb 100644 --- a/src/test/test_data.c +++ b/src/test/test_data.c @@ -1,6 +1,6 @@ /* Copyright 2001-2004 Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ /** First of 3 example authority certificates for unit testing. */ diff --git a/src/test/test_dir.c b/src/test/test_dir.c index 873d761a99..83c612045b 100644 --- a/src/test/test_dir.c +++ b/src/test/test_dir.c @@ -1,16 +1,18 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #include "orconfig.h" #define DIRSERV_PRIVATE #define DIRVOTE_PRIVATE #define ROUTER_PRIVATE +#define HIBERNATE_PRIVATE #include "or.h" #include "directory.h" #include "dirserv.h" #include "dirvote.h" +#include "hibernate.h" #include "networkstatus.h" #include "router.h" #include "routerlist.h" @@ -74,7 +76,7 @@ test_dir_formats(void) char *pk1_str = NULL, *pk2_str = NULL, *pk3_str = NULL, *cp; size_t pk1_str_len, pk2_str_len, pk3_str_len; routerinfo_t *r1=NULL, *r2=NULL; - crypto_pk_env_t *pk1 = NULL, *pk2 = NULL, *pk3 = NULL; + crypto_pk_t *pk1 = NULL, *pk2 = NULL, *pk3 = NULL; routerinfo_t *rp1 = NULL; addr_policy_t *ex1, *ex2; routerlist_t *dir1 = NULL, *dir2 = NULL; @@ -85,6 +87,8 @@ test_dir_formats(void) test_assert(pk1 && pk2 && pk3); + hibernate_set_state_for_testing_(HIBERNATE_STATE_LIVE); + get_platform_str(platform, sizeof(platform)); r1 = tor_malloc_zero(sizeof(routerinfo_t)); r1->address = tor_strdup("18.244.0.1"); @@ -92,6 +96,8 @@ test_dir_formats(void) r1->cache_info.published_on = 0; r1->or_port = 9000; r1->dir_port = 9003; + tor_addr_parse(&r1->ipv6_addr, "1:2:3:4::"); + r1->ipv6_orport = 9999; r1->onion_pkey = crypto_pk_dup_key(pk1); r1->identity_pkey = crypto_pk_dup_key(pk2); r1->bandwidthrate = 1000; @@ -121,7 +127,7 @@ test_dir_formats(void) r2->onion_pkey = crypto_pk_dup_key(pk2); r2->identity_pkey = crypto_pk_dup_key(pk1); r2->bandwidthrate = r2->bandwidthburst = r2->bandwidthcapacity = 3000; - r2->exit_policy = smartlist_create(); + r2->exit_policy = smartlist_new(); smartlist_add(r2->exit_policy, ex2); smartlist_add(r2->exit_policy, ex1); r2->nickname = tor_strdup("Fred"); @@ -137,6 +143,7 @@ test_dir_formats(void) test_assert(router_dump_router_to_string(buf, 2048, r1, pk2)>0); strlcpy(buf2, "router Magri 18.244.0.1 9000 0 9003\n" + "or-address [1:2:3:4::]:9999\n" "platform Tor "VERSION" on ", sizeof(buf2)); strlcat(buf2, get_uname(), sizeof(buf2)); strlcat(buf2, "\n" @@ -206,7 +213,7 @@ test_dir_formats(void) /* Okay, now for the directories. */ { - fingerprint_list = smartlist_create(); + fingerprint_list = smartlist_new(); crypto_pk_get_fingerprint(pk2, buf, 1); add_fingerprint_to_dir("Magri", buf, fingerprint_list); crypto_pk_get_fingerprint(pk1, buf, 1); @@ -243,9 +250,9 @@ test_dir_formats(void) tor_free(pk1_str); tor_free(pk2_str); tor_free(pk3_str); - if (pk1) crypto_free_pk_env(pk1); - if (pk2) crypto_free_pk_env(pk2); - if (pk3) crypto_free_pk_env(pk3); + if (pk1) crypto_pk_free(pk1); + if (pk2) crypto_pk_free(pk2); + if (pk3) crypto_pk_free(pk3); if (rp1) routerinfo_free(rp1); tor_free(dir1); /* XXXX And more !*/ tor_free(dir2); /* And more !*/ @@ -298,7 +305,7 @@ test_dir_versions(void) #define tt_versionstatus_op(vs1, op, vs2) \ tt_assert_test_type(vs1,vs2,#vs1" "#op" "#vs2,version_status_t, \ - (_val1 op _val2),"%d") + (val1_ op val2_),"%d",TT_EXIT_TEST_FUNCTION) #define test_v_i_o(val, ver, lst) \ tt_versionstatus_op(val, ==, tor_version_is_obsolete(ver, lst)) @@ -371,7 +378,7 @@ test_dir_versions(void) static void test_dir_fp_pairs(void) { - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); fp_pair_t *pair; dir_split_resource_into_fingerprint_pairs( @@ -399,7 +406,7 @@ test_dir_fp_pairs(void) static void test_dir_split_fps(void *testdata) { - smartlist_t *sl = smartlist_create(); + smartlist_t *sl = smartlist_new(); char *mem_op_hex_tmp = NULL; (void)testdata; @@ -587,7 +594,7 @@ static void test_dir_param_voting(void) { networkstatus_t vote1, vote2, vote3, vote4; - smartlist_t *votes = smartlist_create(); + smartlist_t *votes = smartlist_new(); char *res = NULL; /* dirvote_compute_params only looks at the net_params field of the votes, @@ -597,10 +604,10 @@ test_dir_param_voting(void) memset(&vote2, 0, sizeof(vote2)); memset(&vote3, 0, sizeof(vote3)); memset(&vote4, 0, sizeof(vote4)); - vote1.net_params = smartlist_create(); - vote2.net_params = smartlist_create(); - vote3.net_params = smartlist_create(); - vote4.net_params = smartlist_create(); + vote1.net_params = smartlist_new(); + vote2.net_params = smartlist_new(); + vote3.net_params = smartlist_new(); + vote4.net_params = smartlist_new(); smartlist_split_string(vote1.net_params, "ab=90 abcd=20 cw=50 x-yz=-99", NULL, 0, 0); smartlist_split_string(vote2.net_params, @@ -616,13 +623,81 @@ test_dir_param_voting(void) test_eq(0, networkstatus_get_param(&vote4, "foobar", 0, -100, 8)); smartlist_add(votes, &vote1); + + /* Do the first tests without adding all the other votes, for + * networks without many dirauths. */ + + res = dirvote_compute_params(votes, 11, 6); + test_streq(res, "ab=90 abcd=20 cw=50 x-yz=-99"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 2); + test_streq(res, ""); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 1); + test_streq(res, "ab=90 abcd=20 cw=50 x-yz=-99"); + tor_free(res); + smartlist_add(votes, &vote2); + + res = dirvote_compute_params(votes, 11, 2); + test_streq(res, "ab=27 abcd=20 cw=5 x-yz=-99"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 2); + test_streq(res, "ab=27 cw=5 x-yz=-99"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 3); + test_streq(res, "ab=27 cw=5 x-yz=-99"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 6); + test_streq(res, ""); + tor_free(res); + smartlist_add(votes, &vote3); + + res = dirvote_compute_params(votes, 11, 3); + test_streq(res, "ab=27 abcd=20 c=60 cw=50 x-yz=-9 zzzzz=101"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 3); + test_streq(res, "ab=27 abcd=20 cw=50 x-yz=-9"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 5); + test_streq(res, "cw=50 x-yz=-9"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 9); + test_streq(res, "cw=50 x-yz=-9"); + tor_free(res); + smartlist_add(votes, &vote4); - res = dirvote_compute_params(votes); - test_streq(res, - "ab=90 abcd=20 c=1 cw=50 x-yz=-9 zzzzz=101"); + res = dirvote_compute_params(votes, 11, 4); + test_streq(res, "ab=90 abcd=20 c=1 cw=50 x-yz=-9 zzzzz=101"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 4); + test_streq(res, "ab=90 abcd=20 cw=50 x-yz=-9"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 5); + test_streq(res, "ab=90 abcd=20 cw=50 x-yz=-9"); + tor_free(res); + + /* Test that the special-cased "at least three dirauths voted for + * this param" logic works as expected. */ + res = dirvote_compute_params(votes, 12, 6); + test_streq(res, "ab=90 abcd=20 cw=50 x-yz=-9"); + tor_free(res); + + res = dirvote_compute_params(votes, 12, 10); + test_streq(res, "ab=90 abcd=20 cw=50 x-yz=-9"); + tor_free(res); done: tor_free(res); @@ -684,7 +759,7 @@ generate_ri_from_rs(const vote_routerstatus_t *vrs) tor_strdup("123456789012345678901234567890123"); r->cache_info.signed_descriptor_len = strlen(r->cache_info.signed_descriptor_body); - r->exit_policy = smartlist_create(); + r->exit_policy = smartlist_new(); r->cache_info.published_on = ++published + time(NULL); return r; } @@ -697,7 +772,7 @@ get_detached_sigs(networkstatus_t *ns, networkstatus_t *ns2) char *r; smartlist_t *sl; tor_assert(ns && ns->flavor == FLAV_NS); - sl = smartlist_create(); + sl = smartlist_new(); smartlist_add(sl,ns); if (ns2) smartlist_add(sl,ns2); @@ -712,8 +787,8 @@ static void test_dir_v3_networkstatus(void) { authority_cert_t *cert1=NULL, *cert2=NULL, *cert3=NULL; - crypto_pk_env_t *sign_skey_1=NULL, *sign_skey_2=NULL, *sign_skey_3=NULL; - crypto_pk_env_t *sign_skey_leg1=NULL; + crypto_pk_t *sign_skey_1=NULL, *sign_skey_2=NULL, *sign_skey_3=NULL; + crypto_pk_t *sign_skey_leg1=NULL; const char *msg=NULL; time_t now = time(NULL); @@ -724,7 +799,7 @@ test_dir_v3_networkstatus(void) vote_routerstatus_t *vrs; routerstatus_t *rs; char *v1_text=NULL, *v2_text=NULL, *v3_text=NULL, *consensus_text=NULL, *cp; - smartlist_t *votes = smartlist_create(); + smartlist_t *votes = smartlist_new(); /* For generating the two other consensuses. */ char *detached_text1=NULL, *detached_text2=NULL; @@ -742,9 +817,9 @@ test_dir_v3_networkstatus(void) test_assert(cert2); cert3 = authority_cert_parse_from_string(AUTHORITY_CERT_3, NULL); test_assert(cert3); - sign_skey_1 = crypto_new_pk_env(); - sign_skey_2 = crypto_new_pk_env(); - sign_skey_3 = crypto_new_pk_env(); + sign_skey_1 = crypto_pk_new(); + sign_skey_2 = crypto_pk_new(); + sign_skey_3 = crypto_pk_new(); sign_skey_leg1 = pk_generate(4); test_assert(!crypto_pk_read_private_key_from_string(sign_skey_1, @@ -768,15 +843,15 @@ test_dir_v3_networkstatus(void) vote->valid_until = now+3000; vote->vote_seconds = 100; vote->dist_seconds = 200; - vote->supported_methods = smartlist_create(); + vote->supported_methods = smartlist_new(); smartlist_split_string(vote->supported_methods, "1 2 3", NULL, 0, -1); vote->client_versions = tor_strdup("0.1.2.14,0.1.2.15"); vote->server_versions = tor_strdup("0.1.2.14,0.1.2.15,0.1.2.16"); - vote->known_flags = smartlist_create(); + vote->known_flags = smartlist_new(); smartlist_split_string(vote->known_flags, "Authority Exit Fast Guard Running Stable V2Dir Valid", 0, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - vote->voters = smartlist_create(); + vote->voters = smartlist_new(); voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t)); voter->nickname = tor_strdup("Voter1"); voter->address = tor_strdup("1.2.3.4"); @@ -787,10 +862,10 @@ test_dir_v3_networkstatus(void) crypto_pk_get_digest(cert1->identity_key, voter->identity_digest); smartlist_add(vote->voters, voter); vote->cert = authority_cert_dup(cert1); - vote->net_params = smartlist_create(); + vote->net_params = smartlist_new(); smartlist_split_string(vote->net_params, "circuitwindow=101 foo=990", NULL, 0, 0); - vote->routerstatus_list = smartlist_create(); + vote->routerstatus_list = smartlist_new(); /* add the first routerstatus. */ vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; @@ -803,7 +878,7 @@ test_dir_v3_networkstatus(void) rs->or_port = 443; rs->dir_port = 8000; /* all flags but running cleared */ - rs->is_running = 1; + rs->is_flagged_running = 1; smartlist_add(vote->routerstatus_list, vrs); test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); @@ -818,7 +893,7 @@ test_dir_v3_networkstatus(void) rs->addr = 0x99009901; rs->or_port = 443; rs->dir_port = 0; - rs->is_exit = rs->is_stable = rs->is_fast = rs->is_running = + rs->is_exit = rs->is_stable = rs->is_fast = rs->is_flagged_running = rs->is_valid = rs->is_v2_dir = rs->is_possible_guard = 1; smartlist_add(vote->routerstatus_list, vrs); test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); @@ -835,7 +910,8 @@ test_dir_v3_networkstatus(void) rs->or_port = 400; rs->dir_port = 9999; rs->is_authority = rs->is_exit = rs->is_stable = rs->is_fast = - rs->is_running = rs->is_valid = rs->is_v2_dir = rs->is_possible_guard = 1; + rs->is_flagged_running = rs->is_valid = rs->is_v2_dir = + rs->is_possible_guard = 1; smartlist_add(vote->routerstatus_list, vrs); test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); @@ -931,7 +1007,7 @@ test_dir_v3_networkstatus(void) vote->dist_seconds = 300; authority_cert_free(vote->cert); vote->cert = authority_cert_dup(cert2); - vote->net_params = smartlist_create(); + vote->net_params = smartlist_new(); smartlist_split_string(vote->net_params, "bar=2000000000 circuitwindow=20", NULL, 0, 0); tor_free(vote->client_versions); @@ -972,7 +1048,7 @@ test_dir_v3_networkstatus(void) vote->dist_seconds = 250; authority_cert_free(vote->cert); vote->cert = authority_cert_dup(cert3); - vote->net_params = smartlist_create(); + vote->net_params = smartlist_new(); smartlist_split_string(vote->net_params, "circuitwindow=80 foo=660", NULL, 0, 0); smartlist_add(vote->supported_methods, tor_strdup("4")); @@ -1044,7 +1120,7 @@ test_dir_v3_networkstatus(void) "Running:Stable:V2Dir:Valid"); tor_free(cp); cp = smartlist_join_strings(con->net_params, ":", 0, NULL); - test_streq(cp, "bar=2000000000:circuitwindow=80:foo=660"); + test_streq(cp, "circuitwindow=80:foo=660"); tor_free(cp); test_eq(4, smartlist_len(con->voters)); /*3 voters, 1 legacy key.*/ @@ -1075,7 +1151,8 @@ test_dir_v3_networkstatus(void) test_assert(!rs->is_fast); test_assert(!rs->is_possible_guard); test_assert(!rs->is_stable); - test_assert(rs->is_running); /* If it wasn't running it wouldn't be here */ + /* (If it wasn't running it wouldn't be here) */ + test_assert(rs->is_flagged_running); test_assert(!rs->is_v2_dir); test_assert(!rs->is_valid); test_assert(!rs->is_named); @@ -1097,7 +1174,7 @@ test_dir_v3_networkstatus(void) test_assert(rs->is_fast); test_assert(rs->is_possible_guard); test_assert(rs->is_stable); - test_assert(rs->is_running); + test_assert(rs->is_flagged_running); test_assert(rs->is_v2_dir); test_assert(rs->is_valid); test_assert(!rs->is_named); @@ -1167,10 +1244,10 @@ test_dir_v3_networkstatus(void) /* Extract a detached signature from con3. */ detached_text1 = get_detached_sigs(con3, con_md3); - tor_assert(detached_text1); + tt_assert(detached_text1); /* Try to parse it. */ dsig1 = networkstatus_parse_detached_signatures(detached_text1, NULL); - tor_assert(dsig1); + tt_assert(dsig1); /* Are parsed values as expected? */ test_eq(dsig1->valid_after, con3->valid_after); @@ -1270,13 +1347,13 @@ test_dir_v3_networkstatus(void) if (con_md) networkstatus_vote_free(con_md); if (sign_skey_1) - crypto_free_pk_env(sign_skey_1); + crypto_pk_free(sign_skey_1); if (sign_skey_2) - crypto_free_pk_env(sign_skey_2); + crypto_pk_free(sign_skey_2); if (sign_skey_3) - crypto_free_pk_env(sign_skey_3); + crypto_pk_free(sign_skey_3); if (sign_skey_leg1) - crypto_free_pk_env(sign_skey_leg1); + crypto_pk_free(sign_skey_leg1); if (cert1) authority_cert_free(cert1); if (cert2) @@ -1305,7 +1382,7 @@ test_dir_v3_networkstatus(void) } #define DIR_LEGACY(name) \ - { #name, legacy_test_helper, 0, &legacy_setup, test_dir_ ## name } + { #name, legacy_test_helper, TT_FORK, &legacy_setup, test_dir_ ## name } #define DIR(name) \ { #name, test_dir_##name, 0, NULL, NULL } diff --git a/src/test/test_microdesc.c b/src/test/test_microdesc.c new file mode 100644 index 0000000000..89c578f4a7 --- /dev/null +++ b/src/test/test_microdesc.c @@ -0,0 +1,233 @@ +/* Copyright (c) 2010-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#include "or.h" + +#include "config.h" +#include "microdesc.h" + +#include "test.h" + +#ifdef _WIN32 +/* For mkdir() */ +#include <direct.h> +#else +#include <dirent.h> +#endif + +static const char test_md1[] = + "onion-key\n" + "-----BEGIN RSA PUBLIC KEY-----\n" + "MIGJAoGBAMjlHH/daN43cSVRaHBwgUfnszzAhg98EvivJ9Qxfv51mvQUxPjQ07es\n" + "gV/3n8fyh3Kqr/ehi9jxkdgSRfSnmF7giaHL1SLZ29kA7KtST+pBvmTpDtHa3ykX\n" + "Xorc7hJvIyTZoc1HU+5XSynj3gsBE5IGK1ZRzrNS688LnuZMVp1tAgMBAAE=\n" + "-----END RSA PUBLIC KEY-----\n"; + +static const char test_md2[] = + "onion-key\n" + "-----BEGIN RSA PUBLIC KEY-----\n" + "MIGJAoGBAMIixIowh2DyPmDNMDwBX2DHcYcqdcH1zdIQJZkyV6c6rQHnvbcaDoSg\n" + "jgFSLJKpnGmh71FVRqep+yVB0zI1JY43kuEnXry2HbZCD9UDo3d3n7t015X5S7ON\n" + "bSSYtQGPwOr6Epf96IF6DoQxy4iDnPUAlejuhAG51s1y6/rZQ3zxAgMBAAE=\n" + "-----END RSA PUBLIC KEY-----\n"; + +static const char test_md3[] = + "@last-listed 2009-06-22\n" + "onion-key\n" + "-----BEGIN RSA PUBLIC KEY-----\n" + "MIGJAoGBAMH3340d4ENNGrqx7UxT+lB7x6DNUKOdPEOn4teceE11xlMyZ9TPv41c\n" + "qj2fRZzfxlc88G/tmiaHshmdtEpklZ740OFqaaJVj4LjPMKFNE+J7Xc1142BE9Ci\n" + "KgsbjGYe2RY261aADRWLetJ8T9QDMm+JngL4288hc8pq1uB/3TAbAgMBAAE=\n" + "-----END RSA PUBLIC KEY-----\n" + "p accept 1-700,800-1000\n" + "family nodeX nodeY nodeZ\n"; + +static void +test_md_cache(void *data) +{ + or_options_t *options = NULL; + microdesc_cache_t *mc = NULL ; + smartlist_t *added = NULL, *wanted = NULL; + microdesc_t *md1, *md2, *md3; + char d1[DIGEST256_LEN], d2[DIGEST256_LEN], d3[DIGEST256_LEN]; + const char *test_md3_noannotation = strchr(test_md3, '\n')+1; + time_t time1, time2, time3; + char *fn = NULL, *s = NULL; + (void)data; + + options = get_options_mutable(); + tt_assert(options); + + time1 = time(NULL); + time2 = time(NULL) - 2*24*60*60; + time3 = time(NULL) - 15*24*60*60; + + /* Possibly, turn this into a test setup/cleanup pair */ + tor_free(options->DataDirectory); + options->DataDirectory = tor_strdup(get_fname("md_datadir_test")); +#ifdef _WIN32 + tt_int_op(0, ==, mkdir(options->DataDirectory)); +#else + tt_int_op(0, ==, mkdir(options->DataDirectory, 0700)); +#endif + + tt_assert(!strcmpstart(test_md3_noannotation, "onion-key")); + + crypto_digest256(d1, test_md1, strlen(test_md1), DIGEST_SHA256); + crypto_digest256(d2, test_md2, strlen(test_md1), DIGEST_SHA256); + crypto_digest256(d3, test_md3_noannotation, strlen(test_md3_noannotation), + DIGEST_SHA256); + + mc = get_microdesc_cache(); + + added = microdescs_add_to_cache(mc, test_md1, NULL, SAVED_NOWHERE, 0, + time1, NULL); + tt_int_op(1, ==, smartlist_len(added)); + md1 = smartlist_get(added, 0); + smartlist_free(added); + added = NULL; + + wanted = smartlist_new(); + added = microdescs_add_to_cache(mc, test_md2, NULL, SAVED_NOWHERE, 0, + time2, wanted); + /* Should fail, since we didn't list test_md2's digest in wanted */ + tt_int_op(0, ==, smartlist_len(added)); + smartlist_free(added); + added = NULL; + + smartlist_add(wanted, tor_memdup(d2, DIGEST256_LEN)); + smartlist_add(wanted, tor_memdup(d3, DIGEST256_LEN)); + added = microdescs_add_to_cache(mc, test_md2, NULL, SAVED_NOWHERE, 0, + time2, wanted); + /* Now it can work. md2 should have been added */ + tt_int_op(1, ==, smartlist_len(added)); + md2 = smartlist_get(added, 0); + /* And it should have gotten removed from 'wanted' */ + tt_int_op(smartlist_len(wanted), ==, 1); + test_mem_op(smartlist_get(wanted, 0), ==, d3, DIGEST256_LEN); + smartlist_free(added); + added = NULL; + + added = microdescs_add_to_cache(mc, test_md3, NULL, + SAVED_NOWHERE, 0, -1, NULL); + /* Must fail, since SAVED_NOWHERE precludes annotations */ + tt_int_op(0, ==, smartlist_len(added)); + smartlist_free(added); + added = NULL; + + added = microdescs_add_to_cache(mc, test_md3_noannotation, NULL, + SAVED_NOWHERE, 0, time3, NULL); + /* Now it can work */ + tt_int_op(1, ==, smartlist_len(added)); + md3 = smartlist_get(added, 0); + smartlist_free(added); + added = NULL; + + /* Okay. We added 1...3. Let's poke them to see how they look, and make + * sure they're really in the journal. */ + tt_ptr_op(md1, ==, microdesc_cache_lookup_by_digest256(mc, d1)); + tt_ptr_op(md2, ==, microdesc_cache_lookup_by_digest256(mc, d2)); + tt_ptr_op(md3, ==, microdesc_cache_lookup_by_digest256(mc, d3)); + + tt_int_op(md1->last_listed, ==, time1); + tt_int_op(md2->last_listed, ==, time2); + tt_int_op(md3->last_listed, ==, time3); + + tt_int_op(md1->saved_location, ==, SAVED_IN_JOURNAL); + tt_int_op(md2->saved_location, ==, SAVED_IN_JOURNAL); + tt_int_op(md3->saved_location, ==, SAVED_IN_JOURNAL); + + tt_int_op(md1->bodylen, ==, strlen(test_md1)); + tt_int_op(md2->bodylen, ==, strlen(test_md2)); + tt_int_op(md3->bodylen, ==, strlen(test_md3_noannotation)); + test_mem_op(md1->body, ==, test_md1, strlen(test_md1)); + test_mem_op(md2->body, ==, test_md2, strlen(test_md2)); + test_mem_op(md3->body, ==, test_md3_noannotation, + strlen(test_md3_noannotation)); + + tor_asprintf(&fn, "%s"PATH_SEPARATOR"cached-microdescs.new", + options->DataDirectory); + s = read_file_to_str(fn, RFTS_BIN, NULL); + tt_assert(s); + test_mem_op(md1->body, ==, s + md1->off, md1->bodylen); + test_mem_op(md2->body, ==, s + md2->off, md2->bodylen); + test_mem_op(md3->body, ==, s + md3->off, md3->bodylen); + + tt_ptr_op(md1->family, ==, NULL); + tt_ptr_op(md3->family, !=, NULL); + tt_int_op(smartlist_len(md3->family), ==, 3); + tt_str_op(smartlist_get(md3->family, 0), ==, "nodeX"); + + /* Now rebuild the cache! */ + tt_int_op(microdesc_cache_rebuild(mc, 1), ==, 0); + + tt_int_op(md1->saved_location, ==, SAVED_IN_CACHE); + tt_int_op(md2->saved_location, ==, SAVED_IN_CACHE); + tt_int_op(md3->saved_location, ==, SAVED_IN_CACHE); + + /* The journal should be empty now */ + tor_free(s); + s = read_file_to_str(fn, RFTS_BIN, NULL); + tt_str_op(s, ==, ""); + tor_free(s); + tor_free(fn); + + /* read the cache. */ + tor_asprintf(&fn, "%s"PATH_SEPARATOR"cached-microdescs", + options->DataDirectory); + s = read_file_to_str(fn, RFTS_BIN, NULL); + test_mem_op(md1->body, ==, s + md1->off, strlen(test_md1)); + test_mem_op(md2->body, ==, s + md2->off, strlen(test_md2)); + test_mem_op(md3->body, ==, s + md3->off, strlen(test_md3_noannotation)); + + /* Okay, now we are going to forget about the cache entirely, and reload it + * from the disk. */ + microdesc_free_all(); + mc = get_microdesc_cache(); + md1 = microdesc_cache_lookup_by_digest256(mc, d1); + md2 = microdesc_cache_lookup_by_digest256(mc, d2); + md3 = microdesc_cache_lookup_by_digest256(mc, d3); + test_assert(md1); + test_assert(md2); + test_assert(md3); + test_mem_op(md1->body, ==, s + md1->off, strlen(test_md1)); + test_mem_op(md2->body, ==, s + md2->off, strlen(test_md2)); + test_mem_op(md3->body, ==, s + md3->off, strlen(test_md3_noannotation)); + + tt_int_op(md1->last_listed, ==, time1); + tt_int_op(md2->last_listed, ==, time2); + tt_int_op(md3->last_listed, ==, time3); + + /* Okay, now we are going to clear out everything older than a week old. + * In practice, that means md3 */ + microdesc_cache_clean(mc, time(NULL)-7*24*60*60, 1/*force*/); + tt_ptr_op(md1, ==, microdesc_cache_lookup_by_digest256(mc, d1)); + tt_ptr_op(md2, ==, microdesc_cache_lookup_by_digest256(mc, d2)); + tt_ptr_op(NULL, ==, microdesc_cache_lookup_by_digest256(mc, d3)); + md3 = NULL; /* it's history now! */ + + /* rebuild again, make sure it stays gone. */ + microdesc_cache_rebuild(mc, 1); + tt_ptr_op(md1, ==, microdesc_cache_lookup_by_digest256(mc, d1)); + tt_ptr_op(md2, ==, microdesc_cache_lookup_by_digest256(mc, d2)); + tt_ptr_op(NULL, ==, microdesc_cache_lookup_by_digest256(mc, d3)); + + done: + if (options) + tor_free(options->DataDirectory); + microdesc_free_all(); + + smartlist_free(added); + if (wanted) + SMARTLIST_FOREACH(wanted, char *, cp, tor_free(cp)); + smartlist_free(wanted); + tor_free(s); + tor_free(fn); +} + +struct testcase_t microdesc_tests[] = { + { "cache", test_md_cache, TT_FORK, NULL, NULL }, + END_OF_TESTCASES +}; + diff --git a/src/test/test_pt.c b/src/test/test_pt.c new file mode 100644 index 0000000000..d3dadb9bfa --- /dev/null +++ b/src/test/test_pt.c @@ -0,0 +1,143 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#define PT_PRIVATE +#include "or.h" +#include "transports.h" +#include "circuitbuild.h" +#include "test.h" + +static void +reset_mp(managed_proxy_t *mp) +{ + mp->conf_state = PT_PROTO_LAUNCHED; + SMARTLIST_FOREACH(mp->transports, transport_t *, t, transport_free(t)); + smartlist_clear(mp->transports); +} + +static void +test_pt_parsing(void) +{ + char line[200]; + + managed_proxy_t *mp = tor_malloc(sizeof(managed_proxy_t)); + mp->conf_state = PT_PROTO_INFANT; + mp->transports = smartlist_new(); + + /* incomplete cmethod */ + strcpy(line,"CMETHOD trebuchet"); + test_assert(parse_cmethod_line(line, mp) < 0); + + reset_mp(mp); + + /* wrong proxy type */ + strcpy(line,"CMETHOD trebuchet dog 127.0.0.1:1999"); + test_assert(parse_cmethod_line(line, mp) < 0); + + reset_mp(mp); + + /* wrong addrport */ + strcpy(line,"CMETHOD trebuchet socks4 abcd"); + test_assert(parse_cmethod_line(line, mp) < 0); + + reset_mp(mp); + + /* correct line */ + strcpy(line,"CMETHOD trebuchet socks5 127.0.0.1:1999"); + test_assert(parse_cmethod_line(line, mp) == 0); + test_assert(smartlist_len(mp->transports)); + + reset_mp(mp); + + /* incomplete smethod */ + strcpy(line,"SMETHOD trebuchet"); + test_assert(parse_smethod_line(line, mp) < 0); + + reset_mp(mp); + + /* wrong addr type */ + strcpy(line,"SMETHOD trebuchet abcd"); + test_assert(parse_smethod_line(line, mp) < 0); + + reset_mp(mp); + + /* cowwect */ + strcpy(line,"SMETHOD trebuchy 127.0.0.1:1999"); + test_assert(parse_smethod_line(line, mp) == 0); + + reset_mp(mp); + + /* unsupported version */ + strcpy(line,"VERSION 666"); + test_assert(parse_version(line, mp) < 0); + + /* incomplete VERSION */ + strcpy(line,"VERSION "); + test_assert(parse_version(line, mp) < 0); + + /* correct VERSION */ + strcpy(line,"VERSION 1"); + test_assert(parse_version(line, mp) == 0); + + done: + tor_free(mp); +} + +static void +test_pt_protocol(void) +{ + char line[200]; + + managed_proxy_t *mp = tor_malloc_zero(sizeof(managed_proxy_t)); + mp->conf_state = PT_PROTO_LAUNCHED; + mp->transports = smartlist_new(); + mp->argv = tor_malloc_zero(sizeof(char*)*2); + mp->argv[0] = tor_strdup("<testcase>"); + + /* various wrong protocol runs: */ + + strcpy(line,"VERSION 1"); + handle_proxy_line(line, mp); + test_assert(mp->conf_state == PT_PROTO_ACCEPTING_METHODS); + + strcpy(line,"VERSION 1"); + handle_proxy_line(line, mp); + test_assert(mp->conf_state == PT_PROTO_BROKEN); + + reset_mp(mp); + + strcpy(line,"CMETHOD trebuchet socks5 127.0.0.1:1999"); + handle_proxy_line(line, mp); + test_assert(mp->conf_state == PT_PROTO_BROKEN); + + reset_mp(mp); + + /* correct protocol run: */ + strcpy(line,"VERSION 1"); + handle_proxy_line(line, mp); + test_assert(mp->conf_state == PT_PROTO_ACCEPTING_METHODS); + + strcpy(line,"CMETHOD trebuchet socks5 127.0.0.1:1999"); + handle_proxy_line(line, mp); + test_assert(mp->conf_state == PT_PROTO_ACCEPTING_METHODS); + + strcpy(line,"CMETHODS DONE"); + handle_proxy_line(line, mp); + test_assert(mp->conf_state == PT_PROTO_CONFIGURED); + + done: + tor_free(mp); +} + +#define PT_LEGACY(name) \ + { #name, legacy_test_helper, 0, &legacy_setup, test_pt_ ## name } + +struct testcase_t pt_tests[] = { + PT_LEGACY(parsing), + PT_LEGACY(protocol), + END_OF_TESTCASES +}; + diff --git a/src/test/test_util.c b/src/test/test_util.c index 12e3fa9793..4f9eb73e03 100644 --- a/src/test/test_util.c +++ b/src/test/test_util.c @@ -1,11 +1,12 @@ /* Copyright (c) 2001-2004, Roger Dingledine. * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #include "orconfig.h" #define CONTROL_PRIVATE #define MEMPOOL_PRIVATE +#define UTIL_PRIVATE #include "or.h" #include "config.h" #include "control.h" @@ -13,6 +14,10 @@ #include "mempool.h" #include "memarea.h" +#ifdef _WIN32 +#include <tchar.h> +#endif + /* XXXX this is a minimal wrapper to make the unit tests compile with the * changed tor_timegm interface. */ static time_t @@ -31,9 +36,12 @@ test_util_time(void) { struct timeval start, end; struct tm a_time; - char timestr[RFC1123_TIME_LEN+1]; + char timestr[128]; time_t t_res; int i; + struct timeval tv; + + /* Test tv_udiff */ start.tv_sec = 5; start.tv_usec = 5000; @@ -59,9 +67,7 @@ test_util_time(void) test_eq(-1005000L, tv_udiff(&start, &end)); - end.tv_usec = 999990; - start.tv_sec = 1; - start.tv_usec = 500; + /* Test tor_timegm */ /* The test values here are confirmed to be correct on a platform * with a working timegm. */ @@ -77,6 +83,17 @@ test_util_time(void) a_time.tm_mon = 1; /* Try a leap year, in feb. */ a_time.tm_mday = 10; test_eq((time_t) 1076393695UL, tor_timegm(&a_time)); + a_time.tm_mon = 0; + a_time.tm_mday = 10; + test_eq((time_t) 1073715295UL, tor_timegm(&a_time)); + a_time.tm_mon = 12; /* Wrong month, it's 0-based */ + a_time.tm_mday = 10; + test_eq((time_t) -1, tor_timegm(&a_time)); + a_time.tm_mon = -1; /* Wrong month */ + a_time.tm_mday = 10; + test_eq((time_t) -1, tor_timegm(&a_time)); + + /* Test {format,parse}_rfc1123_time */ format_rfc1123_time(timestr, 0); test_streq("Thu, 01 Jan 1970 00:00:00 GMT", timestr); @@ -85,9 +102,64 @@ test_util_time(void) t_res = 0; i = parse_rfc1123_time(timestr, &t_res); - test_eq(i,0); + test_eq(0,i); + test_eq(t_res, (time_t)1091580502UL); + /* The timezone doesn't matter */ + t_res = 0; + test_eq(0, parse_rfc1123_time("Wed, 04 Aug 2004 00:48:22 ZUL", &t_res)); test_eq(t_res, (time_t)1091580502UL); test_eq(-1, parse_rfc1123_time("Wed, zz Aug 2004 99-99x99 GMT", &t_res)); + test_eq(-1, parse_rfc1123_time("Wed, 32 Mar 2011 00:00:00 GMT", &t_res)); + test_eq(-1, parse_rfc1123_time("Wed, 30 Mar 2011 24:00:00 GMT", &t_res)); + test_eq(-1, parse_rfc1123_time("Wed, 30 Mar 2011 23:60:00 GMT", &t_res)); + test_eq(-1, parse_rfc1123_time("Wed, 30 Mar 2011 23:59:62 GMT", &t_res)); + test_eq(-1, parse_rfc1123_time("Wed, 30 Mar 1969 23:59:59 GMT", &t_res)); + test_eq(-1, parse_rfc1123_time("Wed, 30 Ene 2011 23:59:59 GMT", &t_res)); + test_eq(-1, parse_rfc1123_time("Wed, 30 Mar 2011 23:59:59 GM", &t_res)); + +#if 0 + /* This fails, I imagine it's important and should be fixed? */ + test_eq(-1, parse_rfc1123_time("Wed, 29 Feb 2011 16:00:00 GMT", &t_res)); + /* Why is this string valid (ie. the test fails because it doesn't + return -1)? */ + test_eq(-1, parse_rfc1123_time("Wed, 30 Mar 2011 23:59:61 GMT", &t_res)); +#endif + + /* Test parse_iso_time */ + + t_res = 0; + i = parse_iso_time("", &t_res); + test_eq(-1, i); + t_res = 0; + i = parse_iso_time("2004-08-32 00:48:22", &t_res); + test_eq(-1, i); + t_res = 0; + i = parse_iso_time("1969-08-03 00:48:22", &t_res); + test_eq(-1, i); + + t_res = 0; + i = parse_iso_time("2004-08-04 00:48:22", &t_res); + test_eq(0,i); + test_eq(t_res, (time_t)1091580502UL); + t_res = 0; + i = parse_iso_time("2004-8-4 0:48:22", &t_res); + test_eq(0, i); + test_eq(t_res, (time_t)1091580502UL); + test_eq(-1, parse_iso_time("2004-08-zz 99-99x99 GMT", &t_res)); + test_eq(-1, parse_iso_time("2011-03-32 00:00:00 GMT", &t_res)); + test_eq(-1, parse_iso_time("2011-03-30 24:00:00 GMT", &t_res)); + test_eq(-1, parse_iso_time("2011-03-30 23:60:00 GMT", &t_res)); + test_eq(-1, parse_iso_time("2011-03-30 23:59:62 GMT", &t_res)); + test_eq(-1, parse_iso_time("1969-03-30 23:59:59 GMT", &t_res)); + test_eq(-1, parse_iso_time("2011-00-30 23:59:59 GMT", &t_res)); + test_eq(-1, parse_iso_time("2011-03-30 23:59", &t_res)); + + /* Test tor_gettimeofday */ + + end.tv_sec = 4; + end.tv_usec = 999990; + start.tv_sec = 1; + start.tv_usec = 500; tor_gettimeofday(&start); /* now make sure time works. */ @@ -95,6 +167,25 @@ test_util_time(void) /* We might've timewarped a little. */ tt_int_op(tv_udiff(&start, &end), >=, -5000); + /* Test format_iso_time */ + + tv.tv_sec = (time_t)1326296338; + tv.tv_usec = 3060; + format_iso_time(timestr, tv.tv_sec); + test_streq("2012-01-11 15:38:58", timestr); + /* The output of format_local_iso_time will vary by timezone, and setting + our timezone for testing purposes would be a nontrivial flaky pain. + Skip this test for now. + format_local_iso_time(timestr, tv.tv_sec); + test_streq("2012-01-11 10:38:58", timestr); + */ + format_iso_time_nospace(timestr, tv.tv_sec); + test_streq("2012-01-11T15:38:58", timestr); + test_eq(strlen(timestr), ISO_TIME_LEN); + format_iso_time_nospace_usec(timestr, &tv); + test_streq("2012-01-11T15:38:58.003060", timestr); + test_eq(strlen(timestr), ISO_TIME_USEC_LEN); + done: ; } @@ -306,6 +397,334 @@ test_util_config_line(void) tor_free(v); } +static void +test_util_config_line_quotes(void) +{ + char buf1[1024]; + char buf2[128]; + char buf3[128]; + char buf4[128]; + char *k=NULL, *v=NULL; + const char *str; + + /* Test parse_config_line_from_str */ + strlcpy(buf1, "kTrailingSpace \"quoted value\" \n" + "kTrailingGarbage \"quoted value\"trailing garbage\n" + , sizeof(buf1)); + strlcpy(buf2, "kTrailingSpaceAndGarbage \"quoted value\" trailing space+g\n" + , sizeof(buf2)); + strlcpy(buf3, "kMultilineTrailingSpace \"mline\\ \nvalue w/ trailing sp\"\n" + , sizeof(buf3)); + strlcpy(buf4, "kMultilineNoTrailingBackslash \"naked multiline\nvalue\"\n" + , sizeof(buf4)); + str = buf1; + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "kTrailingSpace"); + test_streq(v, "quoted value"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_eq_ptr(str, NULL); + tor_free(k); tor_free(v); + + str = buf2; + + str = parse_config_line_from_str(str, &k, &v); + test_eq_ptr(str, NULL); + tor_free(k); tor_free(v); + + str = buf3; + + str = parse_config_line_from_str(str, &k, &v); + test_eq_ptr(str, NULL); + tor_free(k); tor_free(v); + + str = buf4; + + str = parse_config_line_from_str(str, &k, &v); + test_eq_ptr(str, NULL); + tor_free(k); tor_free(v); + + done: + tor_free(k); + tor_free(v); +} + +static void +test_util_config_line_comment_character(void) +{ + char buf[1024]; + char *k=NULL, *v=NULL; + const char *str; + + /* Test parse_config_line_from_str */ + strlcpy(buf, "k1 \"# in quotes\"\n" + "k2 some value # some comment\n" + "k3 /home/user/myTorNetwork#2\n" /* Testcase for #1323 */ + , sizeof(buf)); + str = buf; + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k1"); + test_streq(v, "# in quotes"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k2"); + test_streq(v, "some value"); + tor_free(k); tor_free(v); + + test_streq(str, "k3 /home/user/myTorNetwork#2\n"); + +#if 0 + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k3"); + test_streq(v, "/home/user/myTorNetwork#2"); + tor_free(k); tor_free(v); + + test_streq(str, ""); +#endif + + done: + tor_free(k); + tor_free(v); +} + +static void +test_util_config_line_escaped_content(void) +{ + char buf1[1024]; + char buf2[128]; + char buf3[128]; + char buf4[128]; + char buf5[128]; + char buf6[128]; + char *k=NULL, *v=NULL; + const char *str; + + /* Test parse_config_line_from_str */ + strlcpy(buf1, "HexadecimalLower \"\\x2a\"\n" + "HexadecimalUpper \"\\x2A\"\n" + "HexadecimalUpperX \"\\X2A\"\n" + "Octal \"\\52\"\n" + "Newline \"\\n\"\n" + "Tab \"\\t\"\n" + "CarriageReturn \"\\r\"\n" + "DoubleQuote \"\\\"\"\n" + "SimpleQuote \"\\'\"\n" + "Backslash \"\\\\\"\n" + "Mix \"This is a \\\"star\\\":\\t\\'\\x2a\\'\\nAnd second line\"\n" + , sizeof(buf1)); + + strlcpy(buf2, "BrokenEscapedContent \"\\a\"\n" + , sizeof(buf2)); + + strlcpy(buf3, "BrokenEscapedContent \"\\x\"\n" + , sizeof(buf3)); + + strlcpy(buf4, "BrokenOctal \"\\8\"\n" + , sizeof(buf4)); + + strlcpy(buf5, "BrokenHex \"\\xg4\"\n" + , sizeof(buf5)); + + strlcpy(buf6, "BrokenEscape \"\\" + , sizeof(buf6)); + + str = buf1; + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "HexadecimalLower"); + test_streq(v, "*"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "HexadecimalUpper"); + test_streq(v, "*"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "HexadecimalUpperX"); + test_streq(v, "*"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "Octal"); + test_streq(v, "*"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "Newline"); + test_streq(v, "\n"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "Tab"); + test_streq(v, "\t"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "CarriageReturn"); + test_streq(v, "\r"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "DoubleQuote"); + test_streq(v, "\""); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "SimpleQuote"); + test_streq(v, "'"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "Backslash"); + test_streq(v, "\\"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "Mix"); + test_streq(v, "This is a \"star\":\t'*'\nAnd second line"); + tor_free(k); tor_free(v); + test_streq(str, ""); + + str = buf2; + + str = parse_config_line_from_str(str, &k, &v); + test_eq_ptr(str, NULL); + tor_free(k); tor_free(v); + + str = buf3; + + str = parse_config_line_from_str(str, &k, &v); + test_eq_ptr(str, NULL); + tor_free(k); tor_free(v); + + str = buf4; + + str = parse_config_line_from_str(str, &k, &v); + test_eq_ptr(str, NULL); + tor_free(k); tor_free(v); + +#if 0 + str = buf5; + + str = parse_config_line_from_str(str, &k, &v); + test_eq_ptr(str, NULL); + tor_free(k); tor_free(v); +#endif + + str = buf6; + + str = parse_config_line_from_str(str, &k, &v); + test_eq_ptr(str, NULL); + tor_free(k); tor_free(v); + + done: + tor_free(k); + tor_free(v); +} + +#ifndef _WIN32 +static void +test_util_expand_filename(void) +{ + char *str; + + setenv("HOME", "/home/itv", 1); /* For "internal test value" */ + + str = expand_filename(""); + test_streq("", str); + tor_free(str); + + str = expand_filename("/normal/path"); + test_streq("/normal/path", str); + tor_free(str); + + str = expand_filename("/normal/trailing/path/"); + test_streq("/normal/trailing/path/", str); + tor_free(str); + + str = expand_filename("~"); + test_streq("/home/itv/", str); + tor_free(str); + + str = expand_filename("$HOME/nodice"); + test_streq("$HOME/nodice", str); + tor_free(str); + + str = expand_filename("~/"); + test_streq("/home/itv/", str); + tor_free(str); + + str = expand_filename("~/foobarqux"); + test_streq("/home/itv/foobarqux", str); + tor_free(str); + + str = expand_filename("~/../../etc/passwd"); + test_streq("/home/itv/../../etc/passwd", str); + tor_free(str); + + str = expand_filename("~/trailing/"); + test_streq("/home/itv/trailing/", str); + tor_free(str); + /* Ideally we'd test ~anotheruser, but that's shady to test (we'd + have to somehow inject/fake the get_user_homedir call) */ + + /* $HOME ending in a trailing slash */ + setenv("HOME", "/home/itv/", 1); + + str = expand_filename("~"); + test_streq("/home/itv/", str); + tor_free(str); + + str = expand_filename("~/"); + test_streq("/home/itv/", str); + tor_free(str); + + str = expand_filename("~/foo"); + test_streq("/home/itv/foo", str); + tor_free(str); + + /* Try with empty $HOME */ + + setenv("HOME", "", 1); + + str = expand_filename("~"); + test_streq("/", str); + tor_free(str); + + str = expand_filename("~/"); + test_streq("/", str); + tor_free(str); + + str = expand_filename("~/foobar"); + test_streq("/foobar", str); + tor_free(str); + + /* Try with $HOME unset */ + + unsetenv("HOME"); + + str = expand_filename("~"); + test_streq("/", str); + tor_free(str); + + str = expand_filename("~/"); + test_streq("/", str); + tor_free(str); + + str = expand_filename("~/foobar"); + test_streq("/foobar", str); + tor_free(str); + + done: + tor_free(str); +} +#endif + /** Test basic string functionality. */ static void test_util_strmisc(void) @@ -314,56 +733,113 @@ test_util_strmisc(void) int i; char *cp; - /* Tests for corner cases of strl operations */ + /* Test strl operations */ test_eq(5, strlcpy(buf, "Hello", 0)); + test_eq(5, strlcpy(buf, "Hello", 10)); + test_streq(buf, "Hello"); + test_eq(5, strlcpy(buf, "Hello", 6)); + test_streq(buf, "Hello"); + test_eq(5, strlcpy(buf, "Hello", 5)); + test_streq(buf, "Hell"); strlcpy(buf, "Hello", sizeof(buf)); test_eq(10, strlcat(buf, "Hello", 5)); - /* Test tor_strstrip() */ + /* Test strstrip() */ strlcpy(buf, "Testing 1 2 3", sizeof(buf)); tor_strstrip(buf, ",!"); test_streq(buf, "Testing 1 2 3"); strlcpy(buf, "!Testing 1 2 3?", sizeof(buf)); tor_strstrip(buf, "!? "); test_streq(buf, "Testing123"); + strlcpy(buf, "!!!Testing 1 2 3??", sizeof(buf)); + tor_strstrip(buf, "!? "); + test_streq(buf, "Testing123"); - /* Test tor_parse_long. */ - test_eq(10L, tor_parse_long("10",10,0,100,NULL,NULL)); - test_eq(0L, tor_parse_long("10",10,50,100,NULL,NULL)); - test_eq(-50L, tor_parse_long("-50",10,-100,100,NULL,NULL)); - - /* Test tor_parse_ulong */ + /* Test parse_long */ + /* Empty/zero input */ + test_eq(0L, tor_parse_long("",10,0,100,&i,NULL)); + test_eq(0, i); + test_eq(0L, tor_parse_long("0",10,0,100,&i,NULL)); + test_eq(1, i); + /* Normal cases */ + test_eq(10L, tor_parse_long("10",10,0,100,&i,NULL)); + test_eq(1, i); + test_eq(10L, tor_parse_long("10",10,0,10,&i,NULL)); + test_eq(1, i); + test_eq(10L, tor_parse_long("10",10,10,100,&i,NULL)); + test_eq(1, i); + test_eq(-50L, tor_parse_long("-50",10,-100,100,&i,NULL)); + test_eq(1, i); + test_eq(-50L, tor_parse_long("-50",10,-100,0,&i,NULL)); + test_eq(1, i); + test_eq(-50L, tor_parse_long("-50",10,-50,0,&i,NULL)); + test_eq(1, i); + /* Extra garbage */ + test_eq(0L, tor_parse_long("10m",10,0,100,&i,NULL)); + test_eq(0, i); + test_eq(0L, tor_parse_long("-50 plus garbage",10,-100,100,&i,NULL)); + test_eq(0, i); + test_eq(10L, tor_parse_long("10m",10,0,100,&i,&cp)); + test_eq(1, i); + test_streq(cp, "m"); + test_eq(-50L, tor_parse_long("-50 plus garbage",10,-100,100,&i,&cp)); + test_eq(1, i); + test_streq(cp, " plus garbage"); + /* Out of bounds */ + test_eq(0L, tor_parse_long("10",10,50,100,&i,NULL)); + test_eq(0, i); + test_eq(0L, tor_parse_long("-50",10,0,100,&i,NULL)); + test_eq(0, i); + /* Base different than 10 */ + test_eq(2L, tor_parse_long("10",2,0,100,NULL,NULL)); + test_eq(0L, tor_parse_long("2",2,0,100,NULL,NULL)); + test_eq(0L, tor_parse_long("10",-2,0,100,NULL,NULL)); + test_eq(68284L, tor_parse_long("10abc",16,0,70000,NULL,NULL)); + test_eq(68284L, tor_parse_long("10ABC",16,0,70000,NULL,NULL)); + + /* Test parse_ulong */ + test_eq(0UL, tor_parse_ulong("",10,0,100,NULL,NULL)); + test_eq(0UL, tor_parse_ulong("0",10,0,100,NULL,NULL)); test_eq(10UL, tor_parse_ulong("10",10,0,100,NULL,NULL)); test_eq(0UL, tor_parse_ulong("10",10,50,100,NULL,NULL)); + test_eq(10UL, tor_parse_ulong("10",10,0,10,NULL,NULL)); + test_eq(10UL, tor_parse_ulong("10",10,10,100,NULL,NULL)); + test_eq(0UL, tor_parse_ulong("8",8,0,100,NULL,NULL)); + test_eq(50UL, tor_parse_ulong("50",10,50,100,NULL,NULL)); + test_eq(0UL, tor_parse_ulong("-50",10,-100,100,NULL,NULL)); - /* Test tor_parse_uint64. */ + /* Test parse_uint64 */ test_assert(U64_LITERAL(10) == tor_parse_uint64("10 x",10,0,100, &i, &cp)); - test_assert(i == 1); + test_eq(1, i); test_streq(cp, " x"); test_assert(U64_LITERAL(12345678901) == tor_parse_uint64("12345678901",10,0,UINT64_MAX, &i, &cp)); - test_assert(i == 1); + test_eq(1, i); test_streq(cp, ""); test_assert(U64_LITERAL(0) == tor_parse_uint64("12345678901",10,500,INT32_MAX, &i, &cp)); - test_assert(i == 0); + test_eq(0, i); { - /* Test tor_parse_double. */ + /* Test parse_double */ double d = tor_parse_double("10", 0, UINT64_MAX,&i,NULL); - test_assert(i == 1); + test_eq(1, i); test_assert(DBL_TO_U64(d) == 10); d = tor_parse_double("0", 0, UINT64_MAX,&i,NULL); - test_assert(i == 1); + test_eq(1, i); test_assert(DBL_TO_U64(d) == 0); d = tor_parse_double(" ", 0, UINT64_MAX,&i,NULL); - test_assert(i == 0); + test_eq(0, i); d = tor_parse_double(".0a", 0, UINT64_MAX,&i,NULL); - test_assert(i == 0); + test_eq(0, i); d = tor_parse_double(".0a", 0, UINT64_MAX,&i,&cp); - test_assert(i == 1); + test_eq(1, i); d = tor_parse_double("-.0", 0, UINT64_MAX,&i,NULL); - test_assert(i == 1); + test_eq(1, i); + test_assert(DBL_TO_U64(d) == 0); + d = tor_parse_double("-10", -100.0, 100.0,&i,NULL); + test_eq(1, i); + test_eq(-10.0, d); } { @@ -381,38 +857,60 @@ test_util_strmisc(void) test_eq(i, 0); } - /* Test failing snprintf cases */ + /* Test snprintf */ + /* Returning -1 when there's not enough room in the output buffer */ test_eq(-1, tor_snprintf(buf, 0, "Foo")); test_eq(-1, tor_snprintf(buf, 2, "Foo")); - - /* Test printf with uint64 */ + test_eq(-1, tor_snprintf(buf, 3, "Foo")); + test_neq(-1, tor_snprintf(buf, 4, "Foo")); + /* Always NUL-terminate the output */ + tor_snprintf(buf, 5, "abcdef"); + test_eq(0, buf[4]); + tor_snprintf(buf, 10, "abcdef"); + test_eq(0, buf[6]); + /* uint64 */ tor_snprintf(buf, sizeof(buf), "x!"U64_FORMAT"!x", U64_PRINTF_ARG(U64_LITERAL(12345678901))); - test_streq(buf, "x!12345678901!x"); + test_streq("x!12345678901!x", buf); - /* Test for strcmpstart and strcmpend. */ + /* Test str{,case}cmpstart */ test_assert(strcmpstart("abcdef", "abcdef")==0); test_assert(strcmpstart("abcdef", "abc")==0); test_assert(strcmpstart("abcdef", "abd")<0); test_assert(strcmpstart("abcdef", "abb")>0); test_assert(strcmpstart("ab", "abb")<0); - + test_assert(strcmpstart("ab", "")==0); + test_assert(strcmpstart("ab", "ab ")<0); + test_assert(strcasecmpstart("abcdef", "abCdEF")==0); + test_assert(strcasecmpstart("abcDeF", "abc")==0); + test_assert(strcasecmpstart("abcdef", "Abd")<0); + test_assert(strcasecmpstart("Abcdef", "abb")>0); + test_assert(strcasecmpstart("ab", "Abb")<0); + test_assert(strcasecmpstart("ab", "")==0); + test_assert(strcasecmpstart("ab", "ab ")<0); + + /* Test str{,case}cmpend */ test_assert(strcmpend("abcdef", "abcdef")==0); test_assert(strcmpend("abcdef", "def")==0); test_assert(strcmpend("abcdef", "deg")<0); test_assert(strcmpend("abcdef", "dee")>0); - test_assert(strcmpend("ab", "abb")<0); - + test_assert(strcmpend("ab", "aab")>0); test_assert(strcasecmpend("AbcDEF", "abcdef")==0); test_assert(strcasecmpend("abcdef", "dEF")==0); - test_assert(strcasecmpend("abcDEf", "deg")<0); - test_assert(strcasecmpend("abcdef", "DEE")>0); - test_assert(strcasecmpend("ab", "abB")<0); + test_assert(strcasecmpend("abcdef", "Deg")<0); + test_assert(strcasecmpend("abcDef", "dee")>0); + test_assert(strcasecmpend("AB", "abb")<0); + + /* Test digest_is_zero */ + memset(buf,0,20); + buf[20] = 'x'; + test_assert(tor_digest_is_zero(buf)); + buf[19] = 'x'; + test_assert(!tor_digest_is_zero(buf)); /* Test mem_is_zero */ memset(buf,0,128); buf[128] = 'x'; - test_assert(tor_digest_is_zero(buf)); test_assert(tor_mem_is_zero(buf, 10)); test_assert(tor_mem_is_zero(buf, 20)); test_assert(tor_mem_is_zero(buf, 128)); @@ -423,11 +921,15 @@ test_util_strmisc(void) test_assert(!tor_mem_is_zero(buf, 10)); /* Test 'escaped' */ + test_assert(NULL == escaped(NULL)); test_streq("\"\"", escaped("")); test_streq("\"abcd\"", escaped("abcd")); - test_streq("\"\\\\\\n\\r\\t\\\"\\'\"", escaped("\\\n\r\t\"\'")); - test_streq("\"z\\001abc\\277d\"", escaped("z\001abc\277d")); - test_assert(NULL == escaped(NULL)); + test_streq("\"\\\\ \\n\\r\\t\\\"\\'\"", escaped("\\ \n\r\t\"'")); + test_streq("\"unnecessary \\'backslashes\\'\"", + escaped("unnecessary \'backslashes\'")); + /* Non-printable characters appear as octal */ + test_streq("\"z\\001abc\\277d\"", escaped("z\001abc\277d")); + test_streq("\"z\\336\\255 ;foo\"", escaped("z\xde\xad\x20;foo")); /* Test strndup and memdup */ { @@ -455,42 +957,43 @@ test_util_strmisc(void) test_assert(!tor_strisnonupper(cp)); tor_strupper(cp); test_streq(cp, "ABCDEF"); + tor_strlower(cp); + test_streq(cp, "abcdef"); + test_assert(tor_strisnonupper(cp)); test_assert(tor_strisprint(cp)); cp[3] = 3; test_assert(!tor_strisprint(cp)); tor_free(cp); - /* Test eat_whitespace. */ - { - const char *s = " \n a"; - test_eq_ptr(eat_whitespace(s), s+4); - s = "abcd"; - test_eq_ptr(eat_whitespace(s), s); - s = "#xyz\nab"; - test_eq_ptr(eat_whitespace(s), s+5); - } - /* Test memmem and memstr */ { const char *haystack = "abcde"; - tor_assert(!tor_memmem(haystack, 5, "ef", 2)); + test_assert(!tor_memmem(haystack, 5, "ef", 2)); test_eq_ptr(tor_memmem(haystack, 5, "cd", 2), haystack + 2); test_eq_ptr(tor_memmem(haystack, 5, "cde", 3), haystack + 2); + test_assert(!tor_memmem(haystack, 4, "cde", 3)); haystack = "ababcad"; test_eq_ptr(tor_memmem(haystack, 7, "abc", 3), haystack + 2); + /* memstr */ test_eq_ptr(tor_memstr(haystack, 7, "abc"), haystack + 2); + test_eq_ptr(tor_memstr(haystack, 7, "cad"), haystack + 4); + test_assert(!tor_memstr(haystack, 6, "cad")); + test_assert(!tor_memstr(haystack, 7, "cadd")); test_assert(!tor_memstr(haystack, 7, "fe")); - test_assert(!tor_memstr(haystack, 7, "longerthantheoriginal")); + test_assert(!tor_memstr(haystack, 7, "ababcade")); } /* Test wrap_string */ { - smartlist_t *sl = smartlist_create(); - wrap_string(sl, "This is a test of string wrapping functionality: woot.", + smartlist_t *sl = smartlist_new(); + wrap_string(sl, + "This is a test of string wrapping functionality: woot. " + "a functionality? w00t w00t...!", 10, "", ""); cp = smartlist_join_strings(sl, "", 0, NULL); test_streq(cp, - "This is a\ntest of\nstring\nwrapping\nfunctional\nity: woot.\n"); + "This is a\ntest of\nstring\nwrapping\nfunctional\nity: woot.\n" + "a\nfunctional\nity? w00t\nw00t...!\n"); tor_free(cp); SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); smartlist_clear(sl); @@ -501,11 +1004,100 @@ test_util_strmisc(void) test_streq(cp, "### This is a\n# test of string\n# wrapping\n# functionality:\n" "# woot.\n"); + tor_free(cp); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + wrap_string(sl, "A test of string wrapping...", 6, "### ", "# "); + cp = smartlist_join_strings(sl, "", 0, NULL); + test_streq(cp, + "### A\n# test\n# of\n# stri\n# ng\n# wrap\n# ping\n# ...\n"); + tor_free(cp); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + wrap_string(sl, "Wrapping test", 6, "#### ", "# "); + cp = smartlist_join_strings(sl, "", 0, NULL); + test_streq(cp, "#### W\n# rapp\n# ing\n# test\n"); + tor_free(cp); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + wrap_string(sl, "Small test", 6, "### ", "#### "); + cp = smartlist_join_strings(sl, "", 0, NULL); + test_streq(cp, "### Sm\n#### a\n#### l\n#### l\n#### t\n#### e" + "\n#### s\n#### t\n"); + tor_free(cp); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + wrap_string(sl, "First null", 6, NULL, "> "); + cp = smartlist_join_strings(sl, "", 0, NULL); + test_streq(cp, "First\n> null\n"); + tor_free(cp); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + wrap_string(sl, "Second null", 6, "> ", NULL); + cp = smartlist_join_strings(sl, "", 0, NULL); + test_streq(cp, "> Seco\nnd\nnull\n"); + tor_free(cp); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + wrap_string(sl, "Both null", 6, NULL, NULL); + cp = smartlist_join_strings(sl, "", 0, NULL); + test_streq(cp, "Both\nnull\n"); tor_free(cp); SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); smartlist_free(sl); + + /* Can't test prefixes that have the same length as the line width, because + the function has an assert */ } + + /* Test hex_str */ + { + char binary_data[68]; + size_t i; + for (i = 0; i < sizeof(binary_data); ++i) + binary_data[i] = i; + test_streq(hex_str(binary_data, 0), ""); + test_streq(hex_str(binary_data, 1), "00"); + test_streq(hex_str(binary_data, 17), "000102030405060708090A0B0C0D0E0F10"); + test_streq(hex_str(binary_data, 32), + "000102030405060708090A0B0C0D0E0F" + "101112131415161718191A1B1C1D1E1F"); + test_streq(hex_str(binary_data, 34), + "000102030405060708090A0B0C0D0E0F" + "101112131415161718191A1B1C1D1E1F"); + /* Repeat these tests for shorter strings after longer strings + have been tried, to make sure we're correctly terminating strings */ + test_streq(hex_str(binary_data, 1), "00"); + test_streq(hex_str(binary_data, 0), ""); + } + + /* Test strcmp_opt */ + tt_int_op(strcmp_opt("", "foo"), <, 0); + tt_int_op(strcmp_opt("", ""), ==, 0); + tt_int_op(strcmp_opt("foo", ""), >, 0); + + tt_int_op(strcmp_opt(NULL, ""), <, 0); + tt_int_op(strcmp_opt(NULL, NULL), ==, 0); + tt_int_op(strcmp_opt("", NULL), >, 0); + + tt_int_op(strcmp_opt(NULL, "foo"), <, 0); + tt_int_op(strcmp_opt("foo", NULL), >, 0); + + /* Test strcmp_len */ + tt_int_op(strcmp_len("foo", "bar", 3), >, 0); + tt_int_op(strcmp_len("foo", "bar", 2), <, 0); /* First len, then lexical */ + tt_int_op(strcmp_len("foo2", "foo1", 4), >, 0); + tt_int_op(strcmp_len("foo2", "foo1", 3), <, 0); /* Really stop at len */ + tt_int_op(strcmp_len("foo2", "foo", 3), ==, 0); /* Really stop at len */ + tt_int_op(strcmp_len("blah", "", 4), >, 0); + tt_int_op(strcmp_len("blah", "", 0), ==, 0); + done: ; } @@ -604,10 +1196,10 @@ test_util_threads(void) char *s1 = NULL, *s2 = NULL; int done = 0, timedout = 0; time_t started; -#ifndef MS_WINDOWS +#ifndef _WIN32 struct timeval tv; tv.tv_sec=0; - tv.tv_usec=10; + tv.tv_usec=100*1000; #endif #ifndef TOR_IS_MULTITHREADED /* Skip this test if we aren't threading. We should be threading most @@ -634,11 +1226,11 @@ test_util_threads(void) if (strmap_get(_thread_test_strmap, "thread 1") && strmap_get(_thread_test_strmap, "thread 2")) { done = 1; - } else if (time(NULL) > started + 25) { + } else if (time(NULL) > started + 150) { timedout = done = 1; } tor_mutex_release(_thread_test_mutex); -#ifndef MS_WINDOWS +#ifndef _WIN32 /* Prevent the main thread from starving the worker threads. */ select(0, NULL, NULL, NULL, &tv); #endif @@ -693,13 +1285,14 @@ test_util_gzip(void) test_assert(!tor_gzip_compress(&buf2, &len1, buf1, strlen(buf1)+1, GZIP_METHOD)); test_assert(buf2); - test_assert(!memcmp(buf2, "\037\213", 2)); /* Gzip magic. */ + test_assert(len1 < strlen(buf1)); test_assert(detect_compression_method(buf2, len1) == GZIP_METHOD); test_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1, GZIP_METHOD, 1, LOG_INFO)); test_assert(buf3); - test_streq(buf1,buf3); + test_eq(strlen(buf1) + 1, len2); + test_streq(buf1, buf3); tor_free(buf2); tor_free(buf3); @@ -708,13 +1301,13 @@ test_util_gzip(void) test_assert(!tor_gzip_compress(&buf2, &len1, buf1, strlen(buf1)+1, ZLIB_METHOD)); test_assert(buf2); - test_assert(!memcmp(buf2, "\x78\xDA", 2)); /* deflate magic. */ test_assert(detect_compression_method(buf2, len1) == ZLIB_METHOD); test_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1, ZLIB_METHOD, 1, LOG_INFO)); test_assert(buf3); - test_streq(buf1,buf3); + test_eq(strlen(buf1) + 1, len2); + test_streq(buf1, buf3); /* Check whether we can uncompress concatenated, compressed strings. */ tor_free(buf3); @@ -722,7 +1315,7 @@ test_util_gzip(void) memcpy(buf2+len1, buf2, len1); test_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1*2, ZLIB_METHOD, 1, LOG_INFO)); - test_eq(len2, (strlen(buf1)+1)*2); + test_eq((strlen(buf1)+1)*2, len2); test_memeq(buf3, "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAAAAZ\0" "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAAAAZ\0", @@ -737,45 +1330,46 @@ test_util_gzip(void) tor_strdup("String with low redundancy that won't be compressed much."); test_assert(!tor_gzip_compress(&buf2, &len1, buf1, strlen(buf1)+1, ZLIB_METHOD)); - tor_assert(len1>16); + tt_assert(len1>16); /* when we allow an incomplete string, we should succeed.*/ - tor_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1-16, + tt_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1-16, ZLIB_METHOD, 0, LOG_INFO)); + tt_assert(len2 > 5); buf3[len2]='\0'; - tor_assert(len2 > 5); - tor_assert(!strcmpstart(buf1, buf3)); + tt_assert(!strcmpstart(buf1, buf3)); /* when we demand a complete string, this must fail. */ tor_free(buf3); - tor_assert(tor_gzip_uncompress(&buf3, &len2, buf2, len1-16, + tt_assert(tor_gzip_uncompress(&buf3, &len2, buf2, len1-16, ZLIB_METHOD, 1, LOG_INFO)); - tor_assert(!buf3); + tt_assert(!buf3); /* Now, try streaming compression. */ tor_free(buf1); tor_free(buf2); tor_free(buf3); state = tor_zlib_new(1, ZLIB_METHOD); - tor_assert(state); + tt_assert(state); cp1 = buf1 = tor_malloc(1024); len1 = 1024; ccp2 = "ABCDEFGHIJABCDEFGHIJ"; len2 = 21; test_assert(tor_zlib_process(state, &cp1, &len1, &ccp2, &len2, 0) == TOR_ZLIB_OK); - test_eq(len2, 0); /* Make sure we compressed it all. */ + test_eq(0, len2); /* Make sure we compressed it all. */ test_assert(cp1 > buf1); len2 = 0; cp2 = cp1; test_assert(tor_zlib_process(state, &cp1, &len1, &ccp2, &len2, 1) == TOR_ZLIB_DONE); - test_eq(len2, 0); + test_eq(0, len2); test_assert(cp1 > cp2); /* Make sure we really added something. */ - tor_assert(!tor_gzip_uncompress(&buf3, &len2, buf1, 1024-len1, + tt_assert(!tor_gzip_uncompress(&buf3, &len2, buf1, 1024-len1, ZLIB_METHOD, 1, LOG_WARN)); test_streq(buf3, "ABCDEFGHIJABCDEFGHIJ"); /*Make sure it compressed right.*/ + test_eq(21, len2); done: if (state) @@ -802,14 +1396,12 @@ test_util_mmap(void) test_assert(! mapping); write_str_to_file(fname1, "Short file.", 1); - write_bytes_to_file(fname2, buf, buflen, 1); - write_bytes_to_file(fname3, buf, 16384, 1); mapping = tor_mmap_file(fname1); test_assert(mapping); test_eq(mapping->size, strlen("Short file.")); test_streq(mapping->data, "Short file."); -#ifdef MS_WINDOWS +#ifdef _WIN32 tor_munmap_file(mapping); mapping = NULL; test_assert(unlink(fname1) == 0); @@ -830,9 +1422,10 @@ test_util_mmap(void) /* Make sure that we fail to map a no-longer-existent file. */ mapping = tor_mmap_file(fname1); - test_assert(mapping == NULL); + test_assert(! mapping); /* Now try a big file that stretches across a few pages and isn't aligned */ + write_bytes_to_file(fname2, buf, buflen, 1); mapping = tor_mmap_file(fname2); test_assert(mapping); test_eq(mapping->size, buflen); @@ -841,6 +1434,7 @@ test_util_mmap(void) mapping = NULL; /* Now try a big aligned file. */ + write_bytes_to_file(fname3, buf, 16384, 1); mapping = tor_mmap_file(fname3); test_assert(mapping); test_eq(mapping->size, 16384); @@ -868,12 +1462,12 @@ test_util_control_formats(void) { char *out = NULL; const char *inp = - "..This is a test\r\nof the emergency \nbroadcast\r\n..system.\r\nZ.\r\n"; + "..This is a test\r\n.of the emergency \n..system.\r\n\rZ.\r\n"; size_t sz; sz = read_escaped_data(inp, strlen(inp), &out); test_streq(out, - ".This is a test\nof the emergency \nbroadcast\n.system.\nZ.\n"); + ".This is a test\nof the emergency \n.system.\n\rZ.\n"); test_eq(sz, strlen(out)); done: @@ -884,85 +1478,169 @@ static void test_util_sscanf(void) { unsigned u1, u2, u3; - char s1[10], s2[10], s3[10], ch; + char s1[20], s2[10], s3[10], ch; int r; - r = tor_sscanf("hello world", "hello world"); /* String match: success */ - test_eq(r, 0); - r = tor_sscanf("hello world 3", "hello worlb %u", &u1); /* String fail */ - test_eq(r, 0); - r = tor_sscanf("12345", "%u", &u1); /* Simple number */ - test_eq(r, 1); - test_eq(u1, 12345u); - r = tor_sscanf("", "%u", &u1); /* absent number */ - test_eq(r, 0); - r = tor_sscanf("A", "%u", &u1); /* bogus number */ - test_eq(r, 0); - r = tor_sscanf("4294967295", "%u", &u1); /* UINT32_MAX should work. */ - test_eq(r, 1); - test_eq(u1, 4294967295u); - r = tor_sscanf("4294967296", "%u", &u1); /* Always say -1 at 32 bits. */ - test_eq(r, 0); - r = tor_sscanf("123456", "%2u%u", &u1, &u2); /* Width */ - test_eq(r, 2); - test_eq(u1, 12u); - test_eq(u2, 3456u); - r = tor_sscanf("!12:3:456", "!%2u:%2u:%3u", &u1, &u2, &u3); /* separators */ - test_eq(r, 3); - test_eq(u1, 12u); - test_eq(u2, 3u); - test_eq(u3, 456u); - r = tor_sscanf("12:3:045", "%2u:%2u:%3u", &u1, &u2, &u3); /* 0s */ - test_eq(r, 3); - test_eq(u1, 12u); - test_eq(u2, 3u); - test_eq(u3, 45u); + /* Simple tests (malformed patterns, literal matching, ...) */ + test_eq(-1, tor_sscanf("123", "%i", &r)); /* %i is not supported */ + test_eq(-1, tor_sscanf("wrong", "%5c", s1)); /* %c cannot have a number. */ + test_eq(-1, tor_sscanf("hello", "%s", s1)); /* %s needs a number. */ + test_eq(-1, tor_sscanf("prettylongstring", "%999999s", s1)); +#if 0 + /* GCC thinks these two are illegal. */ + test_eq(-1, tor_sscanf("prettylongstring", "%0s", s1)); + test_eq(0, tor_sscanf("prettylongstring", "%10s", NULL)); +#endif + /* No '%'-strings: always "success" */ + test_eq(0, tor_sscanf("hello world", "hello world")); + test_eq(0, tor_sscanf("hello world", "good bye")); + /* Excess data */ + test_eq(0, tor_sscanf("hello 3", "%u", &u1)); /* have to match the start */ + test_eq(0, tor_sscanf(" 3 hello", "%u", &u1)); + test_eq(0, tor_sscanf(" 3 hello", "%2u", &u1)); /* not even in this case */ + test_eq(1, tor_sscanf("3 hello", "%u", &u1)); /* but trailing is alright */ + + /* Numbers (ie. %u) */ + test_eq(0, tor_sscanf("hello world 3", "hello worlb %u", &u1)); /* d vs b */ + test_eq(1, tor_sscanf("12345", "%u", &u1)); + test_eq(12345u, u1); + test_eq(1, tor_sscanf("12346 ", "%u", &u1)); + test_eq(12346u, u1); + test_eq(0, tor_sscanf(" 12347", "%u", &u1)); + test_eq(1, tor_sscanf(" 12348", " %u", &u1)); + test_eq(12348u, u1); + test_eq(1, tor_sscanf("0", "%u", &u1)); + test_eq(0u, u1); + test_eq(1, tor_sscanf("0000", "%u", &u2)); + test_eq(0u, u2); + test_eq(0, tor_sscanf("", "%u", &u1)); /* absent number */ + test_eq(0, tor_sscanf("A", "%u", &u1)); /* bogus number */ + test_eq(0, tor_sscanf("-1", "%u", &u1)); /* negative number */ + test_eq(1, tor_sscanf("4294967295", "%u", &u1)); /* UINT32_MAX should work */ + test_eq(4294967295u, u1); + test_eq(0, tor_sscanf("4294967296", "%u", &u1)); /* But not at 32 bits */ + test_eq(1, tor_sscanf("4294967296", "%9u", &u1)); /* but parsing only 9... */ + test_eq(429496729u, u1); + + /* Numbers with size (eg. %2u) */ + test_eq(0, tor_sscanf("-1", "%2u", &u1)); + test_eq(2, tor_sscanf("123456", "%2u%u", &u1, &u2)); + test_eq(12u, u1); + test_eq(3456u, u2); + test_eq(1, tor_sscanf("123456", "%8u", &u1)); + test_eq(123456u, u1); + test_eq(1, tor_sscanf("123457 ", "%8u", &u1)); + test_eq(123457u, u1); + test_eq(0, tor_sscanf(" 123456", "%8u", &u1)); + test_eq(3, tor_sscanf("!12:3:456", "!%2u:%2u:%3u", &u1, &u2, &u3)); + test_eq(12u, u1); + test_eq(3u, u2); + test_eq(456u, u3); + test_eq(3, tor_sscanf("67:8:099", "%2u:%2u:%3u", &u1, &u2, &u3)); /* 0s */ + test_eq(67u, u1); + test_eq(8u, u2); + test_eq(99u, u3); /* %u does not match space.*/ - r = tor_sscanf("12:3: 45", "%2u:%2u:%3u", &u1, &u2, &u3); - test_eq(r, 2); + test_eq(2, tor_sscanf("12:3: 45", "%2u:%2u:%3u", &u1, &u2, &u3)); + test_eq(12u, u1); + test_eq(3u, u2); /* %u does not match negative numbers. */ - r = tor_sscanf("12:3:-4", "%2u:%2u:%3u", &u1, &u2, &u3); - test_eq(r, 2); + test_eq(2, tor_sscanf("67:8:-9", "%2u:%2u:%3u", &u1, &u2, &u3)); + test_eq(67u, u1); + test_eq(8u, u2); /* Arbitrary amounts of 0-padding are okay */ - r = tor_sscanf("12:03:000000000000000099", "%2u:%2u:%u", &u1, &u2, &u3); - test_eq(r, 3); - test_eq(u1, 12u); - test_eq(u2, 3u); - test_eq(u3, 99u); - - r = tor_sscanf("99% fresh", "%3u%% fresh", &u1); /* percents are scannable.*/ - test_eq(r, 1); - test_eq(u1, 99); - - r = tor_sscanf("hello", "%s", s1); /* %s needs a number. */ - test_eq(r, -1); - - r = tor_sscanf("hello", "%3s%7s", s1, s2); /* %s matches characters. */ - test_eq(r, 2); + test_eq(3, tor_sscanf("12:03:000000000000000099", "%2u:%2u:%u", + &u1, &u2, &u3)); + test_eq(12u, u1); + test_eq(3u, u2); + test_eq(99u, u3); + + /* Hex (ie. %x) */ + test_eq(3, tor_sscanf("1234 02aBcdEf ff", "%x %x %x", &u1, &u2, &u3)); + test_eq(0x1234, u1); + test_eq(0x2ABCDEF, u2); + test_eq(0xFF, u3); + /* Width works on %x */ + test_eq(3, tor_sscanf("f00dcafe444", "%4x%4x%u", &u1, &u2, &u3)); + test_eq(0xf00d, u1); + test_eq(0xcafe, u2); + test_eq(444, u3); + + /* Literal '%' (ie. '%%') */ + test_eq(1, tor_sscanf("99% fresh", "%3u%% fresh", &u1)); + test_eq(99, u1); + test_eq(0, tor_sscanf("99 fresh", "%% %3u %s", &u1, s1)); + test_eq(1, tor_sscanf("99 fresh", "%3u%% %s", &u1, s1)); + test_eq(2, tor_sscanf("99 fresh", "%3u %5s %%", &u1, s1)); + test_eq(99, u1); + test_streq(s1, "fresh"); + test_eq(1, tor_sscanf("% boo", "%% %3s", s1)); + test_streq("boo", s1); + + /* Strings (ie. %s) */ + test_eq(2, tor_sscanf("hello", "%3s%7s", s1, s2)); test_streq(s1, "hel"); test_streq(s2, "lo"); - r = tor_sscanf("WD40", "%2s%u", s3, &u1); /* %s%u */ - test_eq(r, 2); + test_eq(2, tor_sscanf("WD40", "%2s%u", s3, &u1)); /* %s%u */ test_streq(s3, "WD"); - test_eq(u1, 40); - r = tor_sscanf("76trombones", "%6u%9s", &u1, s1); /* %u%s */ - test_eq(r, 2); - test_eq(u1, 76); + test_eq(40, u1); + test_eq(2, tor_sscanf("WD40", "%3s%u", s3, &u1)); /* %s%u */ + test_streq(s3, "WD4"); + test_eq(0, u1); + test_eq(2, tor_sscanf("76trombones", "%6u%9s", &u1, s1)); /* %u%s */ + test_eq(76, u1); test_streq(s1, "trombones"); - r = tor_sscanf("hello world", "%9s %9s", s1, s2); /* %s doesn't eat space. */ - test_eq(r, 2); + test_eq(1, tor_sscanf("prettylongstring", "%999s", s1)); + test_streq(s1, "prettylongstring"); + /* %s doesn't eat spaces */ + test_eq(2, tor_sscanf("hello world", "%9s %9s", s1, s2)); test_streq(s1, "hello"); test_streq(s2, "world"); - r = tor_sscanf("hi", "%9s%9s%3s", s1, s2, s3); /* %s can be empty. */ - test_eq(r, 3); + test_eq(2, tor_sscanf("bye world?", "%9s %9s", s1, s2)); + test_streq(s1, "bye"); + test_streq(s2, ""); + test_eq(3, tor_sscanf("hi", "%9s%9s%3s", s1, s2, s3)); /* %s can be empty. */ test_streq(s1, "hi"); test_streq(s2, ""); test_streq(s3, ""); - r = tor_sscanf("1.2.3", "%u.%u.%u%c", &u1, &u2, &u3, &ch); - test_eq(r, 3); - r = tor_sscanf("1.2.3 foobar", "%u.%u.%u%c", &u1, &u2, &u3, &ch); - test_eq(r, 4); + test_eq(3, tor_sscanf("1.2.3", "%u.%u.%u%c", &u1, &u2, &u3, &ch)); + test_eq(4, tor_sscanf("1.2.3 foobar", "%u.%u.%u%c", &u1, &u2, &u3, &ch)); + test_eq(' ', ch); + + done: + ; +} + +static void +test_util_path_is_relative(void) +{ + /* OS-independent tests */ + test_eq(1, path_is_relative("")); + test_eq(1, path_is_relative("dir")); + test_eq(1, path_is_relative("dir/")); + test_eq(1, path_is_relative("./dir")); + test_eq(1, path_is_relative("../dir")); + + test_eq(0, path_is_relative("/")); + test_eq(0, path_is_relative("/dir")); + test_eq(0, path_is_relative("/dir/")); + + /* Windows */ +#ifdef _WIN32 + /* I don't have Windows so I can't test this, hence the "#ifdef + 0". These are tests that look useful, so please try to get them + running and uncomment if it all works as it should */ + test_eq(1, path_is_relative("dir")); + test_eq(1, path_is_relative("dir\\")); + test_eq(1, path_is_relative("dir\\a:")); + test_eq(1, path_is_relative("dir\\a:\\")); + test_eq(1, path_is_relative("http:\\dir")); + + test_eq(0, path_is_relative("\\dir")); + test_eq(0, path_is_relative("a:\\dir")); + test_eq(0, path_is_relative("z:\\dir")); +#endif done: ; @@ -990,7 +1668,7 @@ test_util_mempool(void) test_eq(pool->item_alloc_size & 0x03, 0); test_assert(pool->new_chunk_capacity < 60); - allocated = smartlist_create(); + allocated = smartlist_new(); for (i = 0; i < 20000; ++i) { if (smartlist_len(allocated) < 20 || crypto_rand_int(2)) { void *m = mp_pool_get(pool); @@ -1161,30 +1839,71 @@ test_util_strtok(void) { char buf[128]; char buf2[128]; + int i; char *cp1, *cp2; - strlcpy(buf, "Graved on the dark in gestures of descent", sizeof(buf)); - strlcpy(buf2, "they.seemed;their!own;most.perfect;monument", sizeof(buf2)); - /* -- "Year's End", Richard Wilbur */ - test_streq("Graved", tor_strtok_r_impl(buf, " ", &cp1)); - test_streq("they", tor_strtok_r_impl(buf2, ".!..;!", &cp2)); + for (i = 0; i < 3; i++) { + const char *pad1="", *pad2=""; + switch (i) { + case 0: + break; + case 1: + pad1 = " "; + pad2 = "!"; + break; + case 2: + pad1 = " "; + pad2 = ";!"; + break; + } + tor_snprintf(buf, sizeof(buf), "%s", pad1); + tor_snprintf(buf2, sizeof(buf2), "%s", pad2); + test_assert(NULL == tor_strtok_r_impl(buf, " ", &cp1)); + test_assert(NULL == tor_strtok_r_impl(buf2, ".!..;!", &cp2)); + + tor_snprintf(buf, sizeof(buf), + "%sGraved on the dark in gestures of descent%s", pad1, pad1); + tor_snprintf(buf2, sizeof(buf2), + "%sthey.seemed;;their!.own;most.perfect;monument%s",pad2,pad2); + /* -- "Year's End", Richard Wilbur */ + + test_streq("Graved", tor_strtok_r_impl(buf, " ", &cp1)); + test_streq("they", tor_strtok_r_impl(buf2, ".!..;!", &cp2)); #define S1() tor_strtok_r_impl(NULL, " ", &cp1) #define S2() tor_strtok_r_impl(NULL, ".!..;!", &cp2) - test_streq("on", S1()); - test_streq("the", S1()); - test_streq("dark", S1()); - test_streq("seemed", S2()); - test_streq("their", S2()); - test_streq("own", S2()); - test_streq("in", S1()); - test_streq("gestures", S1()); - test_streq("of", S1()); - test_streq("most", S2()); - test_streq("perfect", S2()); - test_streq("descent", S1()); - test_streq("monument", S2()); - test_assert(NULL == S1()); - test_assert(NULL == S2()); + test_streq("on", S1()); + test_streq("the", S1()); + test_streq("dark", S1()); + test_streq("seemed", S2()); + test_streq("their", S2()); + test_streq("own", S2()); + test_streq("in", S1()); + test_streq("gestures", S1()); + test_streq("of", S1()); + test_streq("most", S2()); + test_streq("perfect", S2()); + test_streq("descent", S1()); + test_streq("monument", S2()); + test_eq_ptr(NULL, S1()); + test_eq_ptr(NULL, S2()); + } + + buf[0] = 0; + test_eq_ptr(NULL, tor_strtok_r_impl(buf, " ", &cp1)); + test_eq_ptr(NULL, tor_strtok_r_impl(buf, "!", &cp1)); + + strlcpy(buf, "Howdy!", sizeof(buf)); + test_streq("Howdy", tor_strtok_r_impl(buf, "!", &cp1)); + test_eq_ptr(NULL, tor_strtok_r_impl(NULL, "!", &cp1)); + + strlcpy(buf, " ", sizeof(buf)); + test_eq_ptr(NULL, tor_strtok_r_impl(buf, " ", &cp1)); + strlcpy(buf, " ", sizeof(buf)); + test_eq_ptr(NULL, tor_strtok_r_impl(buf, " ", &cp1)); + + strlcpy(buf, "something ", sizeof(buf)); + test_streq("something", tor_strtok_r_impl(buf, " ", &cp1)); + test_eq_ptr(NULL, tor_strtok_r_impl(NULL, ";", &cp1)); done: ; } @@ -1193,24 +1912,64 @@ static void test_util_find_str_at_start_of_line(void *ptr) { const char *long_string = - "hello world. hello world. hello hello. howdy.\n" - "hello hello world\n"; + "howdy world. how are you? i hope it's fine.\n" + "hello kitty\n" + "third line"; + char *line2 = strchr(long_string,'\n')+1; + char *line3 = strchr(line2,'\n')+1; + const char *short_string = "hello kitty\n" + "second line\n"; + char *short_line2 = strchr(short_string,'\n')+1; (void)ptr; - /* not-found case. */ - tt_assert(! find_str_at_start_of_line(long_string, "fred")); + test_eq_ptr(long_string, find_str_at_start_of_line(long_string, "")); + test_eq_ptr(NULL, find_str_at_start_of_line(short_string, "nonsense")); + test_eq_ptr(NULL, find_str_at_start_of_line(long_string, "nonsense")); + test_eq_ptr(NULL, find_str_at_start_of_line(long_string, "\n")); + test_eq_ptr(NULL, find_str_at_start_of_line(long_string, "how ")); + test_eq_ptr(NULL, find_str_at_start_of_line(long_string, "kitty")); + test_eq_ptr(long_string, find_str_at_start_of_line(long_string, "h")); + test_eq_ptr(long_string, find_str_at_start_of_line(long_string, "how")); + test_eq_ptr(line2, find_str_at_start_of_line(long_string, "he")); + test_eq_ptr(line2, find_str_at_start_of_line(long_string, "hell")); + test_eq_ptr(line2, find_str_at_start_of_line(long_string, "hello k")); + test_eq_ptr(line2, find_str_at_start_of_line(long_string, "hello kitty\n")); + test_eq_ptr(line2, find_str_at_start_of_line(long_string, "hello kitty\nt")); + test_eq_ptr(line3, find_str_at_start_of_line(long_string, "third")); + test_eq_ptr(line3, find_str_at_start_of_line(long_string, "third line")); + test_eq_ptr(NULL, find_str_at_start_of_line(long_string, "third line\n")); + test_eq_ptr(short_line2, find_str_at_start_of_line(short_string, + "second line\n")); + done: + ; +} - /* not-found case where haystack doesn't end with \n */ - tt_assert(! find_str_at_start_of_line("foobar\nbaz", "fred")); +static void +test_util_string_is_C_identifier(void *ptr) +{ + (void)ptr; - /* start-of-string case */ - tt_assert(long_string == - find_str_at_start_of_line(long_string, "hello world.")); + test_eq(1, string_is_C_identifier("string_is_C_identifier")); + test_eq(1, string_is_C_identifier("_string_is_C_identifier")); + test_eq(1, string_is_C_identifier("_")); + test_eq(1, string_is_C_identifier("i")); + test_eq(1, string_is_C_identifier("_____")); + test_eq(1, string_is_C_identifier("__00__")); + test_eq(1, string_is_C_identifier("__init__")); + test_eq(1, string_is_C_identifier("_0")); + test_eq(1, string_is_C_identifier("_0string_is_C_identifier")); + test_eq(1, string_is_C_identifier("_0")); + + test_eq(0, string_is_C_identifier("0_string_is_C_identifier")); + test_eq(0, string_is_C_identifier("0")); + test_eq(0, string_is_C_identifier("")); + test_eq(0, string_is_C_identifier(";")); + test_eq(0, string_is_C_identifier("i;")); + test_eq(0, string_is_C_identifier("_;")); + test_eq(0, string_is_C_identifier("Ã")); + test_eq(0, string_is_C_identifier("ñ")); - /* start-of-line case */ - tt_assert(strchr(long_string,'\n')+1 == - find_str_at_start_of_line(long_string, "hello hello")); done: ; } @@ -1224,36 +1983,47 @@ test_util_asprintf(void *ptr) int r; (void)ptr; - /* empty string. */ + /* simple string */ + r = tor_asprintf(&cp, "simple string 100%% safe"); + test_assert(cp); + test_streq("simple string 100% safe", cp); + test_eq(strlen(cp), r); + + /* empty string */ r = tor_asprintf(&cp, "%s", ""); - tt_assert(cp); - tt_int_op(r, ==, strlen(cp)); - tt_str_op(cp, ==, ""); + test_assert(cp); + test_streq("", cp); + test_eq(strlen(cp), r); + + /* numbers (%i) */ + r = tor_asprintf(&cp, "I like numbers-%2i, %i, etc.", -1, 2); + test_assert(cp); + test_streq("I like numbers--1, 2, etc.", cp); + test_eq(strlen(cp), r); - /* Short string with some printing in it. */ + /* numbers (%d) */ r = tor_asprintf(&cp2, "First=%d, Second=%d", 101, 202); - tt_assert(cp2); - tt_int_op(r, ==, strlen(cp2)); - tt_str_op(cp2, ==, "First=101, Second=202"); - tt_assert(cp != cp2); + test_assert(cp2); + test_eq(strlen(cp2), r); + test_streq("First=101, Second=202", cp2); + test_assert(cp != cp2); tor_free(cp); tor_free(cp2); /* Glass-box test: a string exactly 128 characters long. */ r = tor_asprintf(&cp, "Lorem1: %sLorem2: %s", LOREMIPSUM, LOREMIPSUM); - tt_assert(cp); - tt_int_op(r, ==, 128); - tt_assert(cp[128] == '\0'); - tt_str_op(cp, ==, - "Lorem1: "LOREMIPSUM"Lorem2: "LOREMIPSUM); + test_assert(cp); + test_eq(128, r); + test_assert(cp[128] == '\0'); + test_streq("Lorem1: "LOREMIPSUM"Lorem2: "LOREMIPSUM, cp); tor_free(cp); /* String longer than 128 characters */ r = tor_asprintf(&cp, "1: %s 2: %s 3: %s", LOREMIPSUM, LOREMIPSUM, LOREMIPSUM); - tt_assert(cp); - tt_int_op(r, ==, strlen(cp)); - tt_str_op(cp, ==, "1: "LOREMIPSUM" 2: "LOREMIPSUM" 3: "LOREMIPSUM); + test_assert(cp); + test_eq(strlen(cp), r); + test_streq("1: "LOREMIPSUM" 2: "LOREMIPSUM" 3: "LOREMIPSUM, cp); done: tor_free(cp); @@ -1264,24 +2034,40 @@ static void test_util_listdir(void *ptr) { smartlist_t *dir_contents = NULL; - char *fname1=NULL, *fname2=NULL, *dirname=NULL; + char *fname1=NULL, *fname2=NULL, *fname3=NULL, *dir1=NULL, *dirname=NULL; + int r; (void)ptr; fname1 = tor_strdup(get_fname("hopscotch")); fname2 = tor_strdup(get_fname("mumblety-peg")); + fname3 = tor_strdup(get_fname(".hidden-file")); + dir1 = tor_strdup(get_fname("some-directory")); dirname = tor_strdup(get_fname(NULL)); - tt_int_op(write_str_to_file(fname1, "X\n", 0), ==, 0); - tt_int_op(write_str_to_file(fname2, "Y\n", 0), ==, 0); + test_eq(0, write_str_to_file(fname1, "X\n", 0)); + test_eq(0, write_str_to_file(fname2, "Y\n", 0)); + test_eq(0, write_str_to_file(fname3, "Z\n", 0)); +#ifdef _WIN32 + r = mkdir(dir1); +#else + r = mkdir(dir1, 0700); +#endif + if (r) { + fprintf(stderr, "Can't create directory %s:", dir1); + perror(""); + exit(1); + } dir_contents = tor_listdir(dirname); - tt_assert(dir_contents); + test_assert(dir_contents); /* make sure that each filename is listed. */ - tt_assert(smartlist_string_isin_case(dir_contents, "hopscotch")); - tt_assert(smartlist_string_isin_case(dir_contents, "mumblety-peg")); + test_assert(smartlist_string_isin_case(dir_contents, "hopscotch")); + test_assert(smartlist_string_isin_case(dir_contents, "mumblety-peg")); + test_assert(smartlist_string_isin_case(dir_contents, ".hidden-file")); + test_assert(smartlist_string_isin_case(dir_contents, "some-directory")); - tt_assert(!smartlist_string_isin(dir_contents, ".")); - tt_assert(!smartlist_string_isin(dir_contents, "..")); + test_assert(!smartlist_string_isin(dir_contents, ".")); + test_assert(!smartlist_string_isin(dir_contents, "..")); done: tor_free(fname1); @@ -1299,34 +2085,52 @@ test_util_parent_dir(void *ptr) char *cp; (void)ptr; -#define T(input,expect_ok,output) \ +#define T(output,expect_ok,input) \ do { \ int ok; \ cp = tor_strdup(input); \ ok = get_parent_directory(cp); \ - tt_int_op(ok, ==, expect_ok); \ + tt_int_op(expect_ok, ==, ok); \ if (ok==0) \ - tt_str_op(cp, ==, output); \ + tt_str_op(output, ==, cp); \ tor_free(cp); \ } while (0); - T("/home/wombat/knish", 0, "/home/wombat"); - T("/home/wombat/knish/", 0, "/home/wombat"); - T("./home/wombat/knish/", 0, "./home/wombat"); - T("./wombat", 0, "."); - T("", -1, ""); - T("/", -1, ""); - T("////", -1, ""); + T("/home/wombat", 0, "/home/wombat/knish"); + T("/home/wombat", 0, "/home/wombat/knish/"); + T("/home/wombat", 0, "/home/wombat/knish///"); + T("./home/wombat", 0, "./home/wombat/knish/"); + T("/", 0, "/home"); + T("/", 0, "/home//"); + T(".", 0, "./wombat"); + T(".", 0, "./wombat/"); + T(".", 0, "./wombat//"); + T("wombat", 0, "wombat/foo"); + T("wombat/..", 0, "wombat/../foo"); + T("wombat/../", 0, "wombat/..//foo"); /* Is this correct? */ + T("wombat/.", 0, "wombat/./foo"); + T("wombat/./", 0, "wombat/.//foo"); /* Is this correct? */ + T("wombat", 0, "wombat/..//"); + T("wombat", 0, "wombat/foo/"); + T("wombat", 0, "wombat/.foo"); + T("wombat", 0, "wombat/.foo/"); + + T("wombat", -1, ""); + T("w", -1, ""); + T("wombat", 0, "wombat/knish"); + + T("/", 0, "/"); + T("/", 0, "////"); done: tor_free(cp); } -#ifdef MS_WINDOWS +#ifdef _WIN32 static void test_util_load_win_lib(void *ptr) { - HANDLE h = load_windows_system_library("advapi32.dll"); + HANDLE h = load_windows_system_library(_T("advapi32.dll")); (void) ptr; tt_assert(h); @@ -1337,6 +2141,506 @@ test_util_load_win_lib(void *ptr) #endif static void +clear_hex_errno(char *hex_errno) +{ + memset(hex_errno, '\0', HEX_ERRNO_SIZE + 1); +} + +static void +test_util_exit_status(void *ptr) +{ + /* Leave an extra byte for a \0 so we can do string comparison */ + char hex_errno[HEX_ERRNO_SIZE + 1]; + int n; + + (void)ptr; + + clear_hex_errno(hex_errno); + n = format_helper_exit_status(0, 0, hex_errno); + test_streq("0/0\n", hex_errno); + test_eq(n, strlen(hex_errno)); + + clear_hex_errno(hex_errno); + n = format_helper_exit_status(0, 0x7FFFFFFF, hex_errno); + test_streq("0/7FFFFFFF\n", hex_errno); + test_eq(n, strlen(hex_errno)); + + clear_hex_errno(hex_errno); + n = format_helper_exit_status(0xFF, -0x80000000, hex_errno); + test_streq("FF/-80000000\n", hex_errno); + test_eq(n, strlen(hex_errno)); + + clear_hex_errno(hex_errno); + n = format_helper_exit_status(0x7F, 0, hex_errno); + test_streq("7F/0\n", hex_errno); + test_eq(n, strlen(hex_errno)); + + clear_hex_errno(hex_errno); + n = format_helper_exit_status(0x08, -0x242, hex_errno); + test_streq("8/-242\n", hex_errno); + test_eq(n, strlen(hex_errno)); + + done: + ; +} + +#ifndef _WIN32 +/** Check that fgets waits until a full line, and not return a partial line, on + * a EAGAIN with a non-blocking pipe */ +static void +test_util_fgets_eagain(void *ptr) +{ + int test_pipe[2] = {-1, -1}; + int retval; + ssize_t retlen; + char *retptr; + FILE *test_stream = NULL; + char buf[10]; + + (void)ptr; + + /* Set up a pipe to test on */ + retval = pipe(test_pipe); + tt_int_op(retval, >=, 0); + + /* Set up the read-end to be non-blocking */ + retval = fcntl(test_pipe[0], F_SETFL, O_NONBLOCK); + tt_int_op(retval, >=, 0); + + /* Open it as a stdio stream */ + test_stream = fdopen(test_pipe[0], "r"); + tt_ptr_op(test_stream, !=, NULL); + + /* Send in a partial line */ + retlen = write(test_pipe[1], "A", 1); + tt_int_op(retlen, ==, 1); + retptr = fgets(buf, sizeof(buf), test_stream); + tt_want(retptr == NULL); + tt_int_op(errno, ==, EAGAIN); + + /* Send in the rest */ + retlen = write(test_pipe[1], "B\n", 2); + tt_int_op(retlen, ==, 2); + retptr = fgets(buf, sizeof(buf), test_stream); + tt_ptr_op(retptr, ==, buf); + tt_str_op(buf, ==, "AB\n"); + + /* Send in a full line */ + retlen = write(test_pipe[1], "CD\n", 3); + tt_int_op(retlen, ==, 3); + retptr = fgets(buf, sizeof(buf), test_stream); + tt_ptr_op(retptr, ==, buf); + tt_str_op(buf, ==, "CD\n"); + + /* Send in a partial line */ + retlen = write(test_pipe[1], "E", 1); + tt_int_op(retlen, ==, 1); + retptr = fgets(buf, sizeof(buf), test_stream); + tt_ptr_op(retptr, ==, NULL); + tt_int_op(errno, ==, EAGAIN); + + /* Send in the rest */ + retlen = write(test_pipe[1], "F\n", 2); + tt_int_op(retlen, ==, 2); + retptr = fgets(buf, sizeof(buf), test_stream); + tt_ptr_op(retptr, ==, buf); + tt_str_op(buf, ==, "EF\n"); + + /* Send in a full line and close */ + retlen = write(test_pipe[1], "GH", 2); + tt_int_op(retlen, ==, 2); + retval = close(test_pipe[1]); + test_pipe[1] = -1; + tt_int_op(retval, ==, 0); + retptr = fgets(buf, sizeof(buf), test_stream); + tt_ptr_op(retptr, ==, buf); + tt_str_op(buf, ==, "GH"); + + /* Check for EOF */ + retptr = fgets(buf, sizeof(buf), test_stream); + tt_ptr_op(retptr, ==, NULL); + tt_int_op(feof(test_stream), >, 0); + + done: + if (test_stream != NULL) + fclose(test_stream); + if (test_pipe[0] != -1) + close(test_pipe[0]); + if (test_pipe[1] != -1) + close(test_pipe[1]); +} +#endif + +/** Helper function for testing tor_spawn_background */ +static void +run_util_spawn_background(const char *argv[], const char *expected_out, + const char *expected_err, int expected_exit, + int expected_status) +{ + int retval, exit_code; + ssize_t pos; + process_handle_t *process_handle=NULL; + char stdout_buf[100], stderr_buf[100]; + int status; + + /* Start the program */ +#ifdef _WIN32 + status = tor_spawn_background(NULL, argv, NULL, &process_handle); +#else + status = tor_spawn_background(argv[0], argv, NULL, &process_handle); +#endif + + test_eq(expected_status, status); + if (status == PROCESS_STATUS_ERROR) + return; + + test_assert(process_handle != NULL); + test_eq(expected_status, process_handle->status); + +#ifdef _WIN32 + test_assert(process_handle->stdout_pipe != INVALID_HANDLE_VALUE); + test_assert(process_handle->stderr_pipe != INVALID_HANDLE_VALUE); +#else + test_assert(process_handle->stdout_pipe > 0); + test_assert(process_handle->stderr_pipe > 0); +#endif + + /* Check stdout */ + pos = tor_read_all_from_process_stdout(process_handle, stdout_buf, + sizeof(stdout_buf) - 1); + tt_assert(pos >= 0); + stdout_buf[pos] = '\0'; + test_eq(strlen(expected_out), pos); + test_streq(expected_out, stdout_buf); + + /* Check it terminated correctly */ + retval = tor_get_exit_code(process_handle, 1, &exit_code); + test_eq(PROCESS_EXIT_EXITED, retval); + test_eq(expected_exit, exit_code); + // TODO: Make test-child exit with something other than 0 + + /* Check stderr */ + pos = tor_read_all_from_process_stderr(process_handle, stderr_buf, + sizeof(stderr_buf) - 1); + test_assert(pos >= 0); + stderr_buf[pos] = '\0'; + test_streq(expected_err, stderr_buf); + test_eq(strlen(expected_err), pos); + + done: + if (process_handle) + tor_process_handle_destroy(process_handle, 1); +} + +/** Check that we can launch a process and read the output */ +static void +test_util_spawn_background_ok(void *ptr) +{ +#ifdef _WIN32 + const char *argv[] = {"test-child.exe", "--test", NULL}; + const char *expected_out = "OUT\r\n--test\r\nSLEEPING\r\nDONE\r\n"; + const char *expected_err = "ERR\r\n"; +#else + const char *argv[] = {BUILDDIR "/src/test/test-child", "--test", NULL}; + const char *expected_out = "OUT\n--test\nSLEEPING\nDONE\n"; + const char *expected_err = "ERR\n"; +#endif + + (void)ptr; + + run_util_spawn_background(argv, expected_out, expected_err, 0, + PROCESS_STATUS_RUNNING); +} + +/** Check that failing to find the executable works as expected */ +static void +test_util_spawn_background_fail(void *ptr) +{ +#ifndef BUILDDIR +#define BUILDDIR "." +#endif + const char *argv[] = {BUILDDIR "/src/test/no-such-file", "--test", NULL}; + const char *expected_err = ""; + char expected_out[1024]; + char code[32]; +#ifdef _WIN32 + const int expected_status = PROCESS_STATUS_ERROR; +#else + /* TODO: Once we can signal failure to exec, set this to be + * PROCESS_STATUS_ERROR */ + const int expected_status = PROCESS_STATUS_RUNNING; +#endif + + (void)ptr; + + tor_snprintf(code, sizeof(code), "%x/%x", + 9 /* CHILD_STATE_FAILEXEC */ , ENOENT); + tor_snprintf(expected_out, sizeof(expected_out), + "ERR: Failed to spawn background process - code %s\n", code); + + run_util_spawn_background(argv, expected_out, expected_err, 255, + expected_status); +} + +/** Test that reading from a handle returns a partial read rather than + * blocking */ +static void +test_util_spawn_background_partial_read(void *ptr) +{ + const int expected_exit = 0; + const int expected_status = PROCESS_STATUS_RUNNING; + + int retval, exit_code; + ssize_t pos = -1; + process_handle_t *process_handle=NULL; + int status; + char stdout_buf[100], stderr_buf[100]; +#ifdef _WIN32 + const char *argv[] = {"test-child.exe", "--test", NULL}; + const char *expected_out[] = { "OUT\r\n--test\r\nSLEEPING\r\n", + "DONE\r\n", + NULL }; + const char *expected_err = "ERR\r\n"; +#else + const char *argv[] = {BUILDDIR "/src/test/test-child", "--test", NULL}; + const char *expected_out[] = { "OUT\n--test\nSLEEPING\n", + "DONE\n", + NULL }; + const char *expected_err = "ERR\n"; + int eof = 0; +#endif + int expected_out_ctr; + (void)ptr; + + /* Start the program */ +#ifdef _WIN32 + status = tor_spawn_background(NULL, argv, NULL, &process_handle); +#else + status = tor_spawn_background(argv[0], argv, NULL, &process_handle); +#endif + test_eq(expected_status, status); + test_assert(process_handle); + test_eq(expected_status, process_handle->status); + + /* Check stdout */ + for (expected_out_ctr = 0; expected_out[expected_out_ctr] != NULL;) { +#ifdef _WIN32 + pos = tor_read_all_handle(process_handle->stdout_pipe, stdout_buf, + sizeof(stdout_buf) - 1, NULL); +#else + /* Check that we didn't read the end of file last time */ + test_assert(!eof); + pos = tor_read_all_handle(process_handle->stdout_handle, stdout_buf, + sizeof(stdout_buf) - 1, NULL, &eof); +#endif + log_info(LD_GENERAL, "tor_read_all_handle() returned %d", (int)pos); + + /* We would have blocked, keep on trying */ + if (0 == pos) + continue; + + test_assert(pos > 0); + stdout_buf[pos] = '\0'; + test_streq(expected_out[expected_out_ctr], stdout_buf); + test_eq(strlen(expected_out[expected_out_ctr]), pos); + expected_out_ctr++; + } + + /* The process should have exited without writing more */ +#ifdef _WIN32 + pos = tor_read_all_handle(process_handle->stdout_pipe, stdout_buf, + sizeof(stdout_buf) - 1, + process_handle); + test_eq(0, pos); +#else + if (!eof) { + /* We should have got all the data, but maybe not the EOF flag */ + pos = tor_read_all_handle(process_handle->stdout_handle, stdout_buf, + sizeof(stdout_buf) - 1, + process_handle, &eof); + test_eq(0, pos); + test_assert(eof); + } + /* Otherwise, we got the EOF on the last read */ +#endif + + /* Check it terminated correctly */ + retval = tor_get_exit_code(process_handle, 1, &exit_code); + test_eq(PROCESS_EXIT_EXITED, retval); + test_eq(expected_exit, exit_code); + + // TODO: Make test-child exit with something other than 0 + + /* Check stderr */ + pos = tor_read_all_from_process_stderr(process_handle, stderr_buf, + sizeof(stderr_buf) - 1); + test_assert(pos >= 0); + stderr_buf[pos] = '\0'; + test_streq(expected_err, stderr_buf); + test_eq(strlen(expected_err), pos); + + done: + tor_process_handle_destroy(process_handle, 1); +} + +/** + * Test for format_hex_number_for_helper_exit_status() + */ + +static void +test_util_format_hex_number(void *ptr) +{ + int i, len; + char buf[HEX_ERRNO_SIZE + 1]; + const struct { + const char *str; + unsigned int x; + } test_data[] = { + {"0", 0}, + {"1", 1}, + {"273A", 0x273a}, + {"FFFF", 0xffff}, +#if UINT_MAX >= 0xffffffff + {"31BC421D", 0x31bc421d}, + {"FFFFFFFF", 0xffffffff}, +#endif + {NULL, 0} + }; + + (void)ptr; + + for (i = 0; test_data[i].str != NULL; ++i) { + len = format_hex_number_for_helper_exit_status(test_data[i].x, + buf, HEX_ERRNO_SIZE); + test_neq(len, 0); + buf[len] = '\0'; + test_streq(buf, test_data[i].str); + } + + done: + return; +} + +/** + * Test that we can properly format q Windows command line + */ +static void +test_util_join_win_cmdline(void *ptr) +{ + /* Based on some test cases from "Parsing C++ Command-Line Arguments" in + * MSDN but we don't exercise all quoting rules because tor_join_win_cmdline + * will try to only generate simple cases for the child process to parse; + * i.e. we never embed quoted strings in arguments. */ + + const char *argvs[][4] = { + {"a", "bb", "CCC", NULL}, // Normal + {NULL, NULL, NULL, NULL}, // Empty argument list + {"", NULL, NULL, NULL}, // Empty argument + {"\"a", "b\"b", "CCC\"", NULL}, // Quotes + {"a\tbc", "dd dd", "E", NULL}, // Whitespace + {"a\\\\\\b", "de fg", "H", NULL}, // Backslashes + {"a\\\"b", "\\c", "D\\", NULL}, // Backslashes before quote + {"a\\\\b c", "d", "E", NULL}, // Backslashes not before quote + { NULL } // Terminator + }; + + const char *cmdlines[] = { + "a bb CCC", + "", + "\"\"", + "\\\"a b\\\"b CCC\\\"", + "\"a\tbc\" \"dd dd\" E", + "a\\\\\\b \"de fg\" H", + "a\\\\\\\"b \\c D\\", + "\"a\\\\b c\" d E", + NULL // Terminator + }; + + int i; + char *joined_argv; + + (void)ptr; + + for (i=0; cmdlines[i]!=NULL; i++) { + log_info(LD_GENERAL, "Joining argvs[%d], expecting <%s>", i, cmdlines[i]); + joined_argv = tor_join_win_cmdline(argvs[i]); + test_streq(cmdlines[i], joined_argv); + tor_free(joined_argv); + } + + done: + ; +} + +#define MAX_SPLIT_LINE_COUNT 4 +struct split_lines_test_t { + const char *orig_line; // Line to be split (may contain \0's) + int orig_length; // Length of orig_line + const char *split_line[MAX_SPLIT_LINE_COUNT]; // Split lines +}; + +/** + * Test that we properly split a buffer into lines + */ +static void +test_util_split_lines(void *ptr) +{ + /* Test cases. orig_line of last test case must be NULL. + * The last element of split_line[i] must be NULL. */ + struct split_lines_test_t tests[] = { + {"", 0, {NULL}}, + {"foo", 3, {"foo", NULL}}, + {"\n\rfoo\n\rbar\r\n", 12, {"foo", "bar", NULL}}, + {"fo o\r\nb\tar", 10, {"fo o", "b.ar", NULL}}, + {"\x0f""f\0o\0\n\x01""b\0r\0\r", 12, {".f.o.", ".b.r.", NULL}}, + {"line 1\r\nline 2", 14, {"line 1", "line 2", NULL}}, + {"line 1\r\n\r\nline 2", 16, {"line 1", "line 2", NULL}}, + {"line 1\r\n\r\r\r\nline 2", 18, {"line 1", "line 2", NULL}}, + {"line 1\r\n\n\n\n\rline 2", 18, {"line 1", "line 2", NULL}}, + {"line 1\r\n\r\t\r\nline 3", 18, {"line 1", ".", "line 3", NULL}}, + {"\n\t\r\t\nline 3", 11, {".", ".", "line 3", NULL}}, + {NULL, 0, { NULL }} + }; + + int i, j; + char *orig_line=NULL; + smartlist_t *sl=NULL; + + (void)ptr; + + for (i=0; tests[i].orig_line; i++) { + sl = smartlist_new(); + /* Allocate space for string and trailing NULL */ + orig_line = tor_memdup(tests[i].orig_line, tests[i].orig_length + 1); + tor_split_lines(sl, orig_line, tests[i].orig_length); + + j = 0; + log_info(LD_GENERAL, "Splitting test %d of length %d", + i, tests[i].orig_length); + SMARTLIST_FOREACH_BEGIN(sl, const char *, line) { + /* Check we have not got too many lines */ + test_assert(j < MAX_SPLIT_LINE_COUNT); + /* Check that there actually should be a line here */ + test_assert(tests[i].split_line[j] != NULL); + log_info(LD_GENERAL, "Line %d of test %d, should be <%s>", + j, i, tests[i].split_line[j]); + /* Check that the line is as expected */ + test_streq(line, tests[i].split_line[j]); + j++; + } SMARTLIST_FOREACH_END(line); + /* Check that we didn't miss some lines */ + test_eq_ptr(NULL, tests[i].split_line[j]); + tor_free(orig_line); + smartlist_free(sl); + sl = NULL; + } + + done: + tor_free(orig_line); + smartlist_free(sl); +} + +static void test_util_di_ops(void) { #define LT -1 @@ -1389,6 +2693,389 @@ test_util_di_ops(void) ; } +/** + * Test counting high bits + */ +static void +test_util_n_bits_set(void *ptr) +{ + (void)ptr; + test_eq(0, n_bits_set_u8(0)); + test_eq(1, n_bits_set_u8(1)); + test_eq(3, n_bits_set_u8(7)); + test_eq(1, n_bits_set_u8(8)); + test_eq(2, n_bits_set_u8(129)); + test_eq(8, n_bits_set_u8(255)); + done: + ; +} + +/** + * Test LHS whitespace (and comment) eater + */ +static void +test_util_eat_whitespace(void *ptr) +{ + const char ws[] = { ' ', '\t', '\r' }; /* Except NL */ + char str[80]; + size_t i; + + (void)ptr; + + /* Try one leading ws */ + strcpy(str, "fuubaar"); + for (i = 0; i < sizeof(ws); ++i) { + str[0] = ws[i]; + test_eq_ptr(str + 1, eat_whitespace(str)); + test_eq_ptr(str + 1, eat_whitespace_eos(str, str + strlen(str))); + test_eq_ptr(str + 1, eat_whitespace_no_nl(str)); + test_eq_ptr(str + 1, eat_whitespace_eos_no_nl(str, str + strlen(str))); + } + str[0] = '\n'; + test_eq_ptr(str + 1, eat_whitespace(str)); + test_eq_ptr(str + 1, eat_whitespace_eos(str, str + strlen(str))); + test_eq_ptr(str, eat_whitespace_no_nl(str)); + test_eq_ptr(str, eat_whitespace_eos_no_nl(str, str + strlen(str))); + + /* Empty string */ + strcpy(str, ""); + test_eq_ptr(str, eat_whitespace(str)); + test_eq_ptr(str, eat_whitespace_eos(str, str)); + test_eq_ptr(str, eat_whitespace_no_nl(str)); + test_eq_ptr(str, eat_whitespace_eos_no_nl(str, str)); + + /* Only ws */ + strcpy(str, " \t\r\n"); + test_eq_ptr(str + strlen(str), eat_whitespace(str)); + test_eq_ptr(str + strlen(str), eat_whitespace_eos(str, str + strlen(str))); + test_eq_ptr(str + strlen(str) - 1, + eat_whitespace_no_nl(str)); + test_eq_ptr(str + strlen(str) - 1, + eat_whitespace_eos_no_nl(str, str + strlen(str))); + + strcpy(str, " \t\r "); + test_eq_ptr(str + strlen(str), eat_whitespace(str)); + test_eq_ptr(str + strlen(str), + eat_whitespace_eos(str, str + strlen(str))); + test_eq_ptr(str + strlen(str), eat_whitespace_no_nl(str)); + test_eq_ptr(str + strlen(str), + eat_whitespace_eos_no_nl(str, str + strlen(str))); + + /* Multiple ws */ + strcpy(str, "fuubaar"); + for (i = 0; i < sizeof(ws); ++i) + str[i] = ws[i]; + test_eq_ptr(str + sizeof(ws), eat_whitespace(str)); + test_eq_ptr(str + sizeof(ws), eat_whitespace_eos(str, str + strlen(str))); + test_eq_ptr(str + sizeof(ws), eat_whitespace_no_nl(str)); + test_eq_ptr(str + sizeof(ws), + eat_whitespace_eos_no_nl(str, str + strlen(str))); + + /* Eat comment */ + strcpy(str, "# Comment \n No Comment"); + test_streq("No Comment", eat_whitespace(str)); + test_streq("No Comment", eat_whitespace_eos(str, str + strlen(str))); + test_eq_ptr(str, eat_whitespace_no_nl(str)); + test_eq_ptr(str, eat_whitespace_eos_no_nl(str, str + strlen(str))); + + /* Eat comment & ws mix */ + strcpy(str, " # \t Comment \n\t\nNo Comment"); + test_streq("No Comment", eat_whitespace(str)); + test_streq("No Comment", eat_whitespace_eos(str, str + strlen(str))); + test_eq_ptr(str + 1, eat_whitespace_no_nl(str)); + test_eq_ptr(str + 1, eat_whitespace_eos_no_nl(str, str + strlen(str))); + + /* Eat entire comment */ + strcpy(str, "#Comment"); + test_eq_ptr(str + strlen(str), eat_whitespace(str)); + test_eq_ptr(str + strlen(str), eat_whitespace_eos(str, str + strlen(str))); + test_eq_ptr(str, eat_whitespace_no_nl(str)); + test_eq_ptr(str, eat_whitespace_eos_no_nl(str, str + strlen(str))); + + /* Blank line, then comment */ + strcpy(str, " \t\n # Comment"); + test_eq_ptr(str + strlen(str), eat_whitespace(str)); + test_eq_ptr(str + strlen(str), eat_whitespace_eos(str, str + strlen(str))); + test_eq_ptr(str + 2, eat_whitespace_no_nl(str)); + test_eq_ptr(str + 2, eat_whitespace_eos_no_nl(str, str + strlen(str))); + + done: + ; +} + +/** Return a newly allocated smartlist containing the lines of text in + * <b>lines</b>. The returned strings are heap-allocated, and must be + * freed by the caller. + * + * XXXX? Move to container.[hc] ? */ +static smartlist_t * +smartlist_new_from_text_lines(const char *lines) +{ + smartlist_t *sl = smartlist_new(); + char *last_line; + + smartlist_split_string(sl, lines, "\n", 0, 0); + + last_line = smartlist_pop_last(sl); + if (last_line != NULL && *last_line != '\0') { + smartlist_add(sl, last_line); + } + + return sl; +} + +/** Test smartlist_new_from_text_lines */ +static void +test_util_sl_new_from_text_lines(void *ptr) +{ + (void)ptr; + + { /* Normal usage */ + smartlist_t *sl = smartlist_new_from_text_lines("foo\nbar\nbaz\n"); + int sl_len = smartlist_len(sl); + + tt_want_int_op(sl_len, ==, 3); + + if (sl_len > 0) tt_want_str_op(smartlist_get(sl, 0), ==, "foo"); + if (sl_len > 1) tt_want_str_op(smartlist_get(sl, 1), ==, "bar"); + if (sl_len > 2) tt_want_str_op(smartlist_get(sl, 2), ==, "baz"); + + SMARTLIST_FOREACH(sl, void *, x, tor_free(x)); + smartlist_free(sl); + } + + { /* No final newline */ + smartlist_t *sl = smartlist_new_from_text_lines("foo\nbar\nbaz"); + int sl_len = smartlist_len(sl); + + tt_want_int_op(sl_len, ==, 3); + + if (sl_len > 0) tt_want_str_op(smartlist_get(sl, 0), ==, "foo"); + if (sl_len > 1) tt_want_str_op(smartlist_get(sl, 1), ==, "bar"); + if (sl_len > 2) tt_want_str_op(smartlist_get(sl, 2), ==, "baz"); + + SMARTLIST_FOREACH(sl, void *, x, tor_free(x)); + smartlist_free(sl); + } + + { /* No newlines */ + smartlist_t *sl = smartlist_new_from_text_lines("foo"); + int sl_len = smartlist_len(sl); + + tt_want_int_op(sl_len, ==, 1); + + if (sl_len > 0) tt_want_str_op(smartlist_get(sl, 0), ==, "foo"); + + SMARTLIST_FOREACH(sl, void *, x, tor_free(x)); + smartlist_free(sl); + } + + { /* No text at all */ + smartlist_t *sl = smartlist_new_from_text_lines(""); + int sl_len = smartlist_len(sl); + + tt_want_int_op(sl_len, ==, 0); + + SMARTLIST_FOREACH(sl, void *, x, tor_free(x)); + smartlist_free(sl); + } +} + +static void +test_util_envnames(void *ptr) +{ + (void) ptr; + + tt_assert(environment_variable_names_equal("abc", "abc")); + tt_assert(environment_variable_names_equal("abc", "abc=")); + tt_assert(environment_variable_names_equal("abc", "abc=def")); + tt_assert(environment_variable_names_equal("abc=def", "abc")); + tt_assert(environment_variable_names_equal("abc=def", "abc=ghi")); + + tt_assert(environment_variable_names_equal("abc", "abc")); + tt_assert(environment_variable_names_equal("abc", "abc=")); + tt_assert(environment_variable_names_equal("abc", "abc=def")); + tt_assert(environment_variable_names_equal("abc=def", "abc")); + tt_assert(environment_variable_names_equal("abc=def", "abc=ghi")); + + tt_assert(!environment_variable_names_equal("abc", "abcd")); + tt_assert(!environment_variable_names_equal("abc=", "abcd")); + tt_assert(!environment_variable_names_equal("abc=", "abcd")); + tt_assert(!environment_variable_names_equal("abc=", "def")); + tt_assert(!environment_variable_names_equal("abc=", "def=")); + tt_assert(!environment_variable_names_equal("abc=x", "def=x")); + + tt_assert(!environment_variable_names_equal("", "a=def")); + /* A bit surprising. */ + tt_assert(environment_variable_names_equal("", "=def")); + tt_assert(environment_variable_names_equal("=y", "=x")); + + done: + ; +} + +/** Test process_environment_make */ +static void +test_util_make_environment(void *ptr) +{ + const char *env_vars_string = + "PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/bin\n" + "HOME=/home/foozer\n"; + const char expected_windows_env_block[] = + "HOME=/home/foozer\000" + "PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/bin\000" + "\000"; + size_t expected_windows_env_block_len = + sizeof(expected_windows_env_block) - 1; + + smartlist_t *env_vars = smartlist_new_from_text_lines(env_vars_string); + smartlist_t *env_vars_sorted = smartlist_new(); + smartlist_t *env_vars_in_unixoid_env_block_sorted = smartlist_new(); + + process_environment_t *env; + + (void)ptr; + + env = process_environment_make(env_vars); + + /* Check that the Windows environment block is correct. */ + tt_want(tor_memeq(expected_windows_env_block, env->windows_environment_block, + expected_windows_env_block_len)); + + /* Now for the Unixoid environment block. We don't care which order + * these environment variables are in, so we sort both lists first. */ + + smartlist_add_all(env_vars_sorted, env_vars); + + { + char **v; + for (v = env->unixoid_environment_block; *v; ++v) { + smartlist_add(env_vars_in_unixoid_env_block_sorted, *v); + } + } + + smartlist_sort_strings(env_vars_sorted); + smartlist_sort_strings(env_vars_in_unixoid_env_block_sorted); + + tt_want_int_op(smartlist_len(env_vars_sorted), ==, + smartlist_len(env_vars_in_unixoid_env_block_sorted)); + { + int len = smartlist_len(env_vars_sorted); + int i; + + if (smartlist_len(env_vars_in_unixoid_env_block_sorted) < len) { + len = smartlist_len(env_vars_in_unixoid_env_block_sorted); + } + + for (i = 0; i < len; ++i) { + tt_want_str_op(smartlist_get(env_vars_sorted, i), ==, + smartlist_get(env_vars_in_unixoid_env_block_sorted, i)); + } + } + + /* Clean up. */ + smartlist_free(env_vars_in_unixoid_env_block_sorted); + smartlist_free(env_vars_sorted); + + SMARTLIST_FOREACH(env_vars, char *, x, tor_free(x)); + smartlist_free(env_vars); + + process_environment_free(env); +} + +/** Test set_environment_variable_in_smartlist */ +static void +test_util_set_env_var_in_sl(void *ptr) +{ + /* The environment variables in these strings are in arbitrary + * order; we sort the resulting lists before comparing them. + * + * (They *will not* end up in the order shown in + * expected_resulting_env_vars_string.) */ + + const char *base_env_vars_string = + "PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/bin\n" + "HOME=/home/foozer\n" + "TERM=xterm\n" + "SHELL=/bin/ksh\n" + "USER=foozer\n" + "LOGNAME=foozer\n" + "USERNAME=foozer\n" + "LANG=en_US.utf8\n" + ; + + const char *new_env_vars_string = + "TERM=putty\n" + "DISPLAY=:18.0\n" + ; + + const char *expected_resulting_env_vars_string = + "PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/bin\n" + "HOME=/home/foozer\n" + "TERM=putty\n" + "SHELL=/bin/ksh\n" + "USER=foozer\n" + "LOGNAME=foozer\n" + "USERNAME=foozer\n" + "LANG=en_US.utf8\n" + "DISPLAY=:18.0\n" + ; + + smartlist_t *merged_env_vars = + smartlist_new_from_text_lines(base_env_vars_string); + smartlist_t *new_env_vars = + smartlist_new_from_text_lines(new_env_vars_string); + smartlist_t *expected_resulting_env_vars = + smartlist_new_from_text_lines(expected_resulting_env_vars_string); + + /* Elements of merged_env_vars are heap-allocated, and must be + * freed. Some of them are (or should) be freed by + * set_environment_variable_in_smartlist. + * + * Elements of new_env_vars are heap-allocated, but are copied into + * merged_env_vars, so they are not freed separately at the end of + * the function. + * + * Elements of expected_resulting_env_vars are heap-allocated, and + * must be freed. */ + + (void)ptr; + + SMARTLIST_FOREACH(new_env_vars, char *, env_var, + set_environment_variable_in_smartlist(merged_env_vars, + env_var, + _tor_free, + 1)); + + smartlist_sort_strings(merged_env_vars); + smartlist_sort_strings(expected_resulting_env_vars); + + tt_want_int_op(smartlist_len(merged_env_vars), ==, + smartlist_len(expected_resulting_env_vars)); + { + int len = smartlist_len(merged_env_vars); + int i; + + if (smartlist_len(expected_resulting_env_vars) < len) { + len = smartlist_len(expected_resulting_env_vars); + } + + for (i = 0; i < len; ++i) { + tt_want_str_op(smartlist_get(merged_env_vars, i), ==, + smartlist_get(expected_resulting_env_vars, i)); + } + } + + /* Clean up. */ + SMARTLIST_FOREACH(merged_env_vars, char *, x, tor_free(x)); + smartlist_free(merged_env_vars); + + smartlist_free(new_env_vars); + + SMARTLIST_FOREACH(expected_resulting_env_vars, char *, x, tor_free(x)); + smartlist_free(expected_resulting_env_vars); +} + #define UTIL_LEGACY(name) \ { #name, legacy_test_helper, 0, &legacy_setup, test_util_ ## name } @@ -1399,6 +3086,12 @@ struct testcase_t util_tests[] = { UTIL_LEGACY(time), UTIL_TEST(parse_http_time, 0), UTIL_LEGACY(config_line), + UTIL_LEGACY(config_line_quotes), + UTIL_LEGACY(config_line_comment_character), + UTIL_LEGACY(config_line_escaped_content), +#ifndef _WIN32 + UTIL_LEGACY(expand_filename), +#endif UTIL_LEGACY(strmisc), UTIL_LEGACY(pow2), UTIL_LEGACY(gzip), @@ -1409,15 +3102,33 @@ struct testcase_t util_tests[] = { UTIL_LEGACY(mmap), UTIL_LEGACY(threads), UTIL_LEGACY(sscanf), + UTIL_LEGACY(path_is_relative), UTIL_LEGACY(strtok), UTIL_LEGACY(di_ops), UTIL_TEST(find_str_at_start_of_line, 0), + UTIL_TEST(string_is_C_identifier, 0), UTIL_TEST(asprintf, 0), UTIL_TEST(listdir, 0), UTIL_TEST(parent_dir, 0), -#ifdef MS_WINDOWS +#ifdef _WIN32 UTIL_TEST(load_win_lib, 0), #endif + UTIL_TEST(exit_status, 0), +#ifndef _WIN32 + UTIL_TEST(fgets_eagain, TT_SKIP), +#endif + UTIL_TEST(spawn_background_ok, 0), + UTIL_TEST(spawn_background_fail, 0), + UTIL_TEST(spawn_background_partial_read, 0), + UTIL_TEST(format_hex_number, 0), + UTIL_TEST(join_win_cmdline, 0), + UTIL_TEST(split_lines, 0), + UTIL_TEST(n_bits_set, 0), + UTIL_TEST(eat_whitespace, 0), + UTIL_TEST(sl_new_from_text_lines, 0), + UTIL_TEST(envnames, 0), + UTIL_TEST(make_environment, 0), + UTIL_TEST(set_env_var_in_sl, 0), END_OF_TESTCASES }; diff --git a/src/test/tinytest.c b/src/test/tinytest.c index 11ffc2fe56..4d9afacce4 100644 --- a/src/test/tinytest.c +++ b/src/test/tinytest.c @@ -1,4 +1,4 @@ -/* tinytest.c -- Copyright 2009-2010 Nick Mathewson +/* tinytest.c -- Copyright 2009-2012 Nick Mathewson * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -22,13 +22,16 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#ifdef TINYTEST_LOCAL +#include "tinytest_local.h" +#endif #include <stdio.h> #include <stdlib.h> #include <string.h> #include <assert.h> -#ifdef WIN32 +#ifdef _WIN32 #include <windows.h> #else #include <sys/types.h> @@ -40,9 +43,6 @@ #define __attribute__(x) #endif -#ifdef TINYTEST_LOCAL -#include "tinytest_local.h" -#endif #include "tinytest.h" #include "tinytest_macros.h" @@ -64,16 +64,16 @@ const char *cur_test_prefix = NULL; /**< prefix of the current test group */ /** Name of the current test, if we haven't logged is yet. Used for --quiet */ const char *cur_test_name = NULL; -#ifdef WIN32 -/** Pointer to argv[0] for win32. */ -static const char *commandname = NULL; +#ifdef _WIN32 +/* Copy of argv[0] for win32. */ +static char commandname[MAX_PATH+1]; #endif static void usage(struct testgroup_t *groups, int list_groups) __attribute__((noreturn)); static enum outcome -_testcase_run_bare(const struct testcase_t *testcase) +testcase_run_bare_(const struct testcase_t *testcase) { void *env = NULL; int outcome; @@ -100,10 +100,10 @@ _testcase_run_bare(const struct testcase_t *testcase) #define MAGIC_EXITCODE 42 static enum outcome -_testcase_run_forked(const struct testgroup_t *group, +testcase_run_forked_(const struct testgroup_t *group, const struct testcase_t *testcase) { -#ifdef WIN32 +#ifdef _WIN32 /* Fork? On Win32? How primitive! We'll do what the smart kids do: we'll invoke our own exe (whose name we recall from the command line) with a command line that tells it to run just the test we @@ -119,7 +119,7 @@ _testcase_run_forked(const struct testgroup_t *group, DWORD exitcode; if (!in_tinytest_main) { - printf("\nERROR. On Windows, _testcase_run_forked must be" + printf("\nERROR. On Windows, testcase_run_forked_ must be" " called from within tinytest_main.\n"); abort(); } @@ -165,7 +165,7 @@ _testcase_run_forked(const struct testgroup_t *group, int test_r, write_r; char b[1]; close(outcome_pipe[0]); - test_r = _testcase_run_bare(testcase); + test_r = testcase_run_bare_(testcase); assert(0<=(int)test_r && (int)test_r<=2); b[0] = "NYS"[test_r]; write_r = (int)write(outcome_pipe[1], b, 1); @@ -174,6 +174,7 @@ _testcase_run_forked(const struct testgroup_t *group, exit(1); } exit(0); + return FAIL; /* unreachable */ } else { /* parent */ int status, r; @@ -218,9 +219,9 @@ testcase_run_one(const struct testgroup_t *group, } if ((testcase->flags & TT_FORK) && !(opt_forked||opt_nofork)) { - outcome = _testcase_run_forked(group, testcase); + outcome = testcase_run_forked_(group, testcase); } else { - outcome = _testcase_run_bare(testcase); + outcome = testcase_run_bare_(testcase); } if (outcome == OK) { @@ -239,13 +240,14 @@ testcase_run_one(const struct testgroup_t *group, if (opt_forked) { exit(outcome==OK ? 0 : (outcome==SKIP?MAGIC_EXITCODE : 1)); + return 1; /* unreachable */ } else { return (int)outcome; } } int -_tinytest_set_flag(struct testgroup_t *groups, const char *arg, unsigned long flag) +tinytest_set_flag_(struct testgroup_t *groups, const char *arg, unsigned long flag) { int i, j; size_t length = LONGEST_TEST_NAME; @@ -277,7 +279,7 @@ usage(struct testgroup_t *groups, int list_groups) puts(" Use --list-tests for a list of tests."); if (list_groups) { puts("Known tests are:"); - _tinytest_set_flag(groups, "..", 0); + tinytest_set_flag_(groups, "..", 0); } exit(0); } @@ -287,8 +289,13 @@ tinytest_main(int c, const char **v, struct testgroup_t *groups) { int i, j, n=0; -#ifdef WIN32 - commandname = v[0]; +#ifdef _WIN32 + const char *sp = strrchr(v[0], '.'); + const char *extension = ""; + if (!sp || stricmp(sp, ".exe")) + extension = ".exe"; /* Add an exe so CreateProcess will work */ + snprintf(commandname, sizeof(commandname), "%s%s", v[0], extension); + commandname[MAX_PATH]='\0'; #endif for (i=1; i<c; ++i) { if (v[i][0] == '-') { @@ -315,28 +322,28 @@ tinytest_main(int c, const char **v, struct testgroup_t *groups) } } else { const char *test = v[i]; - int flag = _TT_ENABLED; + int flag = TT_ENABLED_; if (test[0] == ':') { ++test; flag = TT_SKIP; } else { ++n; } - if (!_tinytest_set_flag(groups, test, flag)) { + if (!tinytest_set_flag_(groups, test, flag)) { printf("No such test as %s!\n", v[i]); return -1; } } } if (!n) - _tinytest_set_flag(groups, "..", _TT_ENABLED); + tinytest_set_flag_(groups, "..", TT_ENABLED_); setvbuf(stdout, NULL, _IONBF, 0); ++in_tinytest_main; for (i=0; groups[i].prefix; ++i) for (j=0; groups[i].cases[j].name; ++j) - if (groups[i].cases[j].flags & _TT_ENABLED) + if (groups[i].cases[j].flags & TT_ENABLED_) testcase_run_one(&groups[i], &groups[i].cases[j]); @@ -355,13 +362,13 @@ tinytest_main(int c, const char **v, struct testgroup_t *groups) } int -_tinytest_get_verbosity(void) +tinytest_get_verbosity_(void) { return opt_verbosity; } void -_tinytest_set_test_failed(void) +tinytest_set_test_failed_(void) { if (opt_verbosity <= 0 && cur_test_name) { if (opt_verbosity==0) puts(""); @@ -372,7 +379,7 @@ _tinytest_set_test_failed(void) } void -_tinytest_set_test_skipped(void) +tinytest_set_test_skipped_(void) { if (cur_test_outcome==OK) cur_test_outcome = SKIP; diff --git a/src/test/tinytest.h b/src/test/tinytest.h index cbe28b7f51..bcac9f079c 100644 --- a/src/test/tinytest.h +++ b/src/test/tinytest.h @@ -1,4 +1,4 @@ -/* tinytest.h -- Copyright 2009-2010 Nick Mathewson +/* tinytest.h -- Copyright 2009-2012 Nick Mathewson * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -23,15 +23,15 @@ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ -#ifndef _TINYTEST_H -#define _TINYTEST_H +#ifndef TINYTEST_H_INCLUDED_ +#define TINYTEST_H_INCLUDED_ /** Flag for a test that needs to run in a subprocess. */ #define TT_FORK (1<<0) /** Runtime flag for a test we've decided to skip. */ #define TT_SKIP (1<<1) /** Internal runtime flag for a test we've decided to run. */ -#define _TT_ENABLED (1<<2) +#define TT_ENABLED_ (1<<2) /** If you add your own flags, make them start at this point. */ #define TT_FIRST_USER_FLAG (1<<3) @@ -65,18 +65,18 @@ struct testgroup_t { #define END_OF_GROUPS { NULL, NULL} /** Implementation: called from a test to indicate failure, before logging. */ -void _tinytest_set_test_failed(void); +void tinytest_set_test_failed_(void); /** Implementation: called from a test to indicate that we're skipping. */ -void _tinytest_set_test_skipped(void); +void tinytest_set_test_skipped_(void); /** Implementation: return 0 for quiet, 1 for normal, 2 for loud. */ -int _tinytest_get_verbosity(void); +int tinytest_get_verbosity_(void); /** Implementation: Set a flag on tests matching a name; returns number * of tests that matched. */ -int _tinytest_set_flag(struct testgroup_t *, const char *, unsigned long); +int tinytest_set_flag_(struct testgroup_t *, const char *, unsigned long); /** Set all tests in 'groups' matching the name 'named' to be skipped. */ #define tinytest_skip(groups, named) \ - _tinytest_set_flag(groups, named, TT_SKIP) + tinytest_set_flag_(groups, named, TT_SKIP) /** Run a single testcase in a single group. */ int testcase_run_one(const struct testgroup_t *,const struct testcase_t *); diff --git a/src/test/tinytest_demo.c b/src/test/tinytest_demo.c index 4d2f588435..be95ce4c1d 100644 --- a/src/test/tinytest_demo.c +++ b/src/test/tinytest_demo.c @@ -1,4 +1,4 @@ -/* tinytest_demo.c -- Copyright 2009 Nick Mathewson +/* tinytest_demo.c -- Copyright 2009-2012 Nick Mathewson * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -53,7 +53,7 @@ test_strcmp(void *data) } /* Pretty often, calling tt_abort_msg to indicate failure is more - heavy-weight than you want. Instead, just say: */ + heavy-weight than you want. Instead, just say: */ tt_assert(strcmp("testcase", "testcase") == 0); /* Occasionally, you don't want to stop the current testcase just @@ -91,7 +91,7 @@ test_strcmp(void *data) /* First you declare a type to hold the environment info, and functions to set it up and tear it down. */ struct data_buffer { - /* We're just going to have couple of character buffer. Using + /* We're just going to have couple of character buffer. Using setup/teardown functions is probably overkill for this case. You could also do file descriptors, complicated handles, temporary @@ -164,7 +164,7 @@ test_memcpy(void *ptr) /* ============================================================ */ -/* Now we need to make sure that our tests get invoked. First, you take +/* Now we need to make sure that our tests get invoked. First, you take a bunch of related tests and put them into an array of struct testcase_t. */ @@ -189,15 +189,15 @@ struct testgroup_t groups[] = { /* Every group has a 'prefix', and an array of tests. That's it. */ { "demo/", demo_tests }, - END_OF_GROUPS + END_OF_GROUPS }; int main(int c, const char **v) { - /* Finally, just call tinytest_main(). It lets you specify verbose - or quiet output with --verbose and --quiet. You can list + /* Finally, just call tinytest_main(). It lets you specify verbose + or quiet output with --verbose and --quiet. You can list specific tests: tinytest-demo demo/memcpy diff --git a/src/test/tinytest_macros.h b/src/test/tinytest_macros.h index a7fa64a824..9ff69b1d50 100644 --- a/src/test/tinytest_macros.h +++ b/src/test/tinytest_macros.h @@ -1,4 +1,4 @@ -/* tinytest_macros.h -- Copyright 2009-2010 Nick Mathewson +/* tinytest_macros.h -- Copyright 2009-2012 Nick Mathewson * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -23,8 +23,8 @@ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ -#ifndef _TINYTEST_MACROS_H -#define _TINYTEST_MACROS_H +#ifndef TINYTEST_MACROS_H_INCLUDED_ +#define TINYTEST_MACROS_H_INCLUDED_ /* Helpers for defining statement-like macros */ #define TT_STMT_BEGIN do { @@ -51,19 +51,19 @@ /* Announce a non-failure if we're verbose. */ #define TT_BLATHER(args) \ TT_STMT_BEGIN \ - if (_tinytest_get_verbosity()>1) TT_DECLARE(" OK", args); \ + if (tinytest_get_verbosity_()>1) TT_DECLARE(" OK", args); \ TT_STMT_END #define TT_DIE(args) \ TT_STMT_BEGIN \ - _tinytest_set_test_failed(); \ + tinytest_set_test_failed_(); \ TT_GRIPE(args); \ TT_EXIT_TEST_FUNCTION; \ TT_STMT_END #define TT_FAIL(args) \ TT_STMT_BEGIN \ - _tinytest_set_test_failed(); \ + tinytest_set_test_failed_(); \ TT_GRIPE(args); \ TT_STMT_END @@ -74,7 +74,7 @@ #define tt_abort() TT_DIE(("%s", "(Failed.)")) /* Fail but do not abort the current test for the reason in msg. */ -#define tt_fail_printf(msg) TT_FAIL(msg) +#define tt_failprint_f(msg) TT_FAIL(msg) #define tt_fail_perror(op) TT_FAIL(("%s: %s [%d]",(op),strerror(errno), errno)) #define tt_fail_msg(msg) TT_FAIL(("%s", msg)) #define tt_fail() TT_FAIL(("%s", "(Failed.)")) @@ -82,28 +82,28 @@ /* End the current test, and indicate we are skipping it. */ #define tt_skip() \ TT_STMT_BEGIN \ - _tinytest_set_test_skipped(); \ + tinytest_set_test_skipped_(); \ TT_EXIT_TEST_FUNCTION; \ TT_STMT_END -#define _tt_want(b, msg, fail) \ +#define tt_want_(b, msg, fail) \ TT_STMT_BEGIN \ if (!(b)) { \ - _tinytest_set_test_failed(); \ - TT_GRIPE((msg)); \ + tinytest_set_test_failed_(); \ + TT_GRIPE(("%s",msg)); \ fail; \ } else { \ - TT_BLATHER((msg)); \ + TT_BLATHER(("%s",msg)); \ } \ TT_STMT_END /* Assert b, but do not stop the test if b fails. Log msg on failure. */ #define tt_want_msg(b, msg) \ - _tt_want(b, msg, ); + tt_want_(b, msg, ); /* Assert b and stop the test if b fails. Log msg on failure. */ #define tt_assert_msg(b, msg) \ - _tt_want(b, msg, TT_EXIT_TEST_FUNCTION); + tt_want_(b, msg, TT_EXIT_TEST_FUNCTION); /* Assert b, but do not stop the test if b fails. */ #define tt_want(b) tt_want_msg( (b), "want("#b")") @@ -111,57 +111,74 @@ #define tt_assert(b) tt_assert_msg((b), "assert("#b")") #define tt_assert_test_fmt_type(a,b,str_test,type,test,printf_type,printf_fmt, \ - setup_block,cleanup_block) \ + setup_block,cleanup_block,die_on_fail) \ TT_STMT_BEGIN \ - type _val1 = (type)(a); \ - type _val2 = (type)(b); \ - int _tt_status = (test); \ - if (!_tt_status || _tinytest_get_verbosity()>1) { \ - printf_type _print; \ - printf_type _print1; \ - printf_type _print2; \ - type _value = _val1; \ + type val1_ = (type)(a); \ + type val2_ = (type)(b); \ + int tt_status_ = (test); \ + if (!tt_status_ || tinytest_get_verbosity_()>1) { \ + printf_type print_; \ + printf_type print1_; \ + printf_type print2_; \ + type value_ = val1_; \ setup_block; \ - _print1 = _print; \ - _value = _val2; \ + print1_ = print_; \ + value_ = val2_; \ setup_block; \ - _print2 = _print; \ - TT_DECLARE(_tt_status?" OK":"FAIL", \ + print2_ = print_; \ + TT_DECLARE(tt_status_?" OK":"FAIL", \ ("assert(%s): "printf_fmt" vs "printf_fmt, \ - str_test, _print1, _print2)); \ - _print = _print1; \ + str_test, print1_, print2_)); \ + print_ = print1_; \ cleanup_block; \ - _print = _print2; \ + print_ = print2_; \ cleanup_block; \ - if (!_tt_status) { \ - _tinytest_set_test_failed(); \ - TT_EXIT_TEST_FUNCTION; \ + if (!tt_status_) { \ + tinytest_set_test_failed_(); \ + die_on_fail ; \ } \ } \ TT_STMT_END -#define tt_assert_test_type(a,b,str_test,type,test,fmt) \ +#define tt_assert_test_type(a,b,str_test,type,test,fmt,die_on_fail) \ tt_assert_test_fmt_type(a,b,str_test,type,test,type,fmt, \ - {_print=_value;},{}) + {print_=value_;},{},die_on_fail) /* Helper: assert that a op b, when cast to type. Format the values with * printf format fmt on failure. */ #define tt_assert_op_type(a,op,b,type,fmt) \ - tt_assert_test_type(a,b,#a" "#op" "#b,type,(_val1 op _val2),fmt) + tt_assert_test_type(a,b,#a" "#op" "#b,type,(val1_ op val2_),fmt, \ + TT_EXIT_TEST_FUNCTION) #define tt_int_op(a,op,b) \ - tt_assert_test_type(a,b,#a" "#op" "#b,long,(_val1 op _val2),"%ld") + tt_assert_test_type(a,b,#a" "#op" "#b,long,(val1_ op val2_), \ + "%ld",TT_EXIT_TEST_FUNCTION) #define tt_uint_op(a,op,b) \ tt_assert_test_type(a,b,#a" "#op" "#b,unsigned long, \ - (_val1 op _val2),"%lu") + (val1_ op val2_),"%lu",TT_EXIT_TEST_FUNCTION) #define tt_ptr_op(a,op,b) \ tt_assert_test_type(a,b,#a" "#op" "#b,void*, \ - (_val1 op _val2),"%p") + (val1_ op val2_),"%p",TT_EXIT_TEST_FUNCTION) #define tt_str_op(a,op,b) \ tt_assert_test_type(a,b,#a" "#op" "#b,const char *, \ - (strcmp(_val1,_val2) op 0),"<%s>") + (strcmp(val1_,val2_) op 0),"<%s>",TT_EXIT_TEST_FUNCTION) + +#define tt_want_int_op(a,op,b) \ + tt_assert_test_type(a,b,#a" "#op" "#b,long,(val1_ op val2_),"%ld",(void)0) + +#define tt_want_uint_op(a,op,b) \ + tt_assert_test_type(a,b,#a" "#op" "#b,unsigned long, \ + (val1_ op val2_),"%lu",(void)0) + +#define tt_want_ptr_op(a,op,b) \ + tt_assert_test_type(a,b,#a" "#op" "#b,void*, \ + (val1_ op val2_),"%p",(void)0) + +#define tt_want_str_op(a,op,b) \ + tt_assert_test_type(a,b,#a" "#op" "#b,const char *, \ + (strcmp(val1_,val2_) op 0),"<%s>",(void)0) #endif diff --git a/src/tools/Makefile.am b/src/tools/Makefile.am index 1bb5076849..35b0a41f53 100644 --- a/src/tools/Makefile.am +++ b/src/tools/Makefile.am @@ -2,17 +2,21 @@ bin_PROGRAMS = tor-resolve tor-gencert noinst_PROGRAMS = tor-checkkey tor_resolve_SOURCES = tor-resolve.c -tor_resolve_LDFLAGS = @TOR_LDFLAGS_libevent@ -tor_resolve_LDADD = ../common/libor.a -lm @TOR_LIBEVENT_LIBS@ @TOR_LIB_WS32@ +tor_resolve_LDFLAGS = +tor_resolve_LDADD = ../common/libor.a @TOR_LIB_MATH@ @TOR_LIB_WS32@ tor_gencert_SOURCES = tor-gencert.c -tor_gencert_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ - @TOR_LDFLAGS_libevent@ +tor_gencert_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ tor_gencert_LDADD = ../common/libor.a ../common/libor-crypto.a \ - -lm @TOR_ZLIB_LIBS@ @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + @TOR_LIB_MATH@ @TOR_ZLIB_LIBS@ @TOR_OPENSSL_LIBS@ \ + @TOR_LIB_WS32@ @TOR_LIB_GDI@ tor_checkkey_SOURCES = tor-checkkey.c -tor_checkkey_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ - @TOR_LDFLAGS_libevent@ +tor_checkkey_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ tor_checkkey_LDADD = ../common/libor.a ../common/libor-crypto.a \ - -lm @TOR_ZLIB_LIBS@ @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + @TOR_LIB_MATH@ @TOR_ZLIB_LIBS@ @TOR_OPENSSL_LIBS@ \ + @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +SUBDIRS = tor-fw-helper +DIST_SUBDIRS = tor-fw-helper + diff --git a/src/tools/Makefile.nmake b/src/tools/Makefile.nmake new file mode 100644 index 0000000000..a30a28b2e1 --- /dev/null +++ b/src/tools/Makefile.nmake @@ -0,0 +1,19 @@ +all: tor-resolve.exe tor-gencert.exe + +CFLAGS = /I ..\win32 /I ..\..\..\build-alpha\include /I ..\common /I ..\or + +LIBS = ..\..\..\build-alpha\lib\libevent.lib \ + ..\..\..\build-alpha\lib\libcrypto.lib \ + ..\..\..\build-alpha\lib\libssl.lib \ + ..\..\..\build-alpha\lib\libz.lib \ + ws2_32.lib advapi32.lib shell32.lib + + +tor-gencert.exe: tor-gencert.obj + $(CC) $(CFLAGS) $(LIBS) ..\common\*.lib tor-gencert.obj + +tor-resolve.exe: tor-resolve.obj + $(CC) $(CFLAGS) $(LIBS) ..\common\*.lib tor-resolve.obj + +clean: + del *.obj *.lib *.exe diff --git a/src/tools/tor-checkkey.c b/src/tools/tor-checkkey.c index 94c8cbd44c..10d13d8371 100644 --- a/src/tools/tor-checkkey.c +++ b/src/tools/tor-checkkey.c @@ -1,3 +1,5 @@ +/* Copyright (c) 2008-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ #define CRYPTO_PRIVATE @@ -15,7 +17,7 @@ int main(int c, char **v) { - crypto_pk_env_t *env; + crypto_pk_t *env; char *str; RSA *rsa; int wantdigest=0; @@ -56,7 +58,7 @@ main(int c, char **v) return 1; } - env = crypto_new_pk_env(); + env = crypto_pk_new(); if (crypto_pk_read_public_key_from_string(env, str, strlen(str))<0) { fprintf(stderr, "Couldn't parse key.\n"); return 1; @@ -69,7 +71,7 @@ main(int c, char **v) return 1; printf("%s\n",digest); } else { - rsa = _crypto_pk_env_get_rsa(env); + rsa = _crypto_pk_get_rsa(env); str = BN_bn2hex(rsa->n); printf("%s\n", str); diff --git a/src/tools/tor-fw-helper/Makefile.am b/src/tools/tor-fw-helper/Makefile.am new file mode 100644 index 0000000000..393562db03 --- /dev/null +++ b/src/tools/tor-fw-helper/Makefile.am @@ -0,0 +1,38 @@ +if USE_FW_HELPER +bin_PROGRAMS = tor-fw-helper +else +bin_PROGRAMS = +endif + +tor_fw_helper_SOURCES = \ + tor-fw-helper.c \ + tor-fw-helper-natpmp.c \ + tor-fw-helper-upnp.c +noinst_HEADERS = \ + tor-fw-helper.h \ + tor-fw-helper-natpmp.h \ + tor-fw-helper-upnp.h + +if NAT_PMP +nat_pmp_ldflags = @TOR_LDFLAGS_libnatpmp@ +nat_pmp_ldadd = -lnatpmp @TOR_LIB_IPHLPAPI@ +nat_pmp_cppflags = @TOR_CPPFLAGS_libnatpmp@ +else +nat_pmp_ldflags = +nat_pmp_ldadd = +nat_pmp_cppflags = +endif + +if MINIUPNPC +miniupnpc_ldflags = @TOR_LDFLAGS_libminiupnpc@ +miniupnpc_ldadd = -lminiupnpc -lm @TOR_LIB_IPHLPAPI@ +miniupnpc_cppflags = @TOR_CPPFLAGS_libminiupnpc@ +else +miniupnpc_ldflags = +miniupnpc_ldadd = +miniupnpc_cppflags = +endif + +tor_fw_helper_LDFLAGS = $(nat_pmp_ldflags) $(miniupnpc_ldflags) +tor_fw_helper_LDADD = ../../common/libor.a $(nat_pmp_ldadd) $(miniupnpc_ldadd) @TOR_LIB_WS32@ +tor_fw_helper_CPPFLAGS = $(nat_pmp_cppflags) $(miniupnpc_cppflags) diff --git a/src/tools/tor-fw-helper/tor-fw-helper-natpmp.c b/src/tools/tor-fw-helper/tor-fw-helper-natpmp.c new file mode 100644 index 0000000000..0e0b385f9b --- /dev/null +++ b/src/tools/tor-fw-helper/tor-fw-helper-natpmp.c @@ -0,0 +1,240 @@ +/* Copyright (c) 2010, Jacob Appelbaum, Steven J. Murdoch. + * Copyright (c) 2010-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file tor-fw-helper-natpmp.c + * \brief The implementation of our NAT-PMP firewall helper. + **/ + +#include "orconfig.h" +#ifdef NAT_PMP +#ifdef _WIN32 +#define STATICLIB +#endif +#include <stdint.h> +#include <stdio.h> +#include <string.h> +#include <errno.h> +#ifndef _WIN32 +#include <arpa/inet.h> +#endif + +// debugging stuff +#include <assert.h> + +#include "compat.h" + +#include "tor-fw-helper.h" +#include "tor-fw-helper-natpmp.h" + +/** This hooks NAT-PMP into our multi-backend API. */ +static tor_fw_backend_t tor_natpmp_backend = { + "natpmp", + sizeof(struct natpmp_state_t), + tor_natpmp_init, + tor_natpmp_cleanup, + tor_natpmp_fetch_public_ip, + tor_natpmp_add_tcp_mapping +}; + +/** Return the backend for NAT-PMP. */ +const tor_fw_backend_t * +tor_fw_get_natpmp_backend(void) +{ + return &tor_natpmp_backend; +} + +/** Initialize the NAT-PMP backend and store the results in + * <b>backend_state</b>.*/ +int +tor_natpmp_init(tor_fw_options_t *tor_fw_options, void *backend_state) +{ + natpmp_state_t *state = (natpmp_state_t *) backend_state; + int r = 0; + + memset(&(state->natpmp), 0, sizeof(natpmp_t)); + memset(&(state->response), 0, sizeof(natpmpresp_t)); + state->init = 0; + state->protocol = NATPMP_PROTOCOL_TCP; + state->lease = NATPMP_DEFAULT_LEASE; + + if (tor_fw_options->verbose) + fprintf(stdout, "V: natpmp init...\n"); + + r = initnatpmp(&(state->natpmp), 0, 0); + if (r == 0) { + state->init = 1; + fprintf(stdout, "tor-fw-helper: natpmp initialized...\n"); + return r; + } else { + fprintf(stderr, "tor-fw-helper: natpmp failed to initialize...\n"); + return r; + } +} + +/** Tear down the NAT-PMP connection stored in <b>backend_state</b>.*/ +int +tor_natpmp_cleanup(tor_fw_options_t *tor_fw_options, void *backend_state) +{ + natpmp_state_t *state = (natpmp_state_t *) backend_state; + int r = 0; + if (tor_fw_options->verbose) + fprintf(stdout, "V: natpmp cleanup...\n"); + r = closenatpmp(&(state->natpmp)); + if (tor_fw_options->verbose) + fprintf(stdout, "V: closing natpmp socket: %d\n", r); + return r; +} + +/** Use select() to wait until we can read on fd. */ +static int +wait_until_fd_readable(tor_socket_t fd, struct timeval *timeout) +{ + int r; + fd_set fds; + if (fd >= FD_SETSIZE) { + fprintf(stderr, "E: NAT-PMP FD_SETSIZE error %d\n", fd); + return -1; + } + FD_ZERO(&fds); + FD_SET(fd, &fds); + r = select(fd+1, &fds, NULL, NULL, timeout); + if (r == -1) { + fprintf(stdout, "V: select failed in wait_until_fd_readable: %s\n", + strerror(errno)); + return -1; + } + /* XXXX we should really check to see whether fd was readable, or we timed + out. */ + return 0; +} + +/** Add a TCP port mapping for a single port stored in <b>tor_fw_options</b> + * using the <b>natpmp_t</b> stored in <b>backend_state</b>. */ +int +tor_natpmp_add_tcp_mapping(tor_fw_options_t *tor_fw_options, + void *backend_state) +{ + natpmp_state_t *state = (natpmp_state_t *) backend_state; + int r = 0; + int x = 0; + int sav_errno; + + struct timeval timeout; + + if (tor_fw_options->verbose) + fprintf(stdout, "V: sending natpmp portmapping request...\n"); + r = sendnewportmappingrequest(&(state->natpmp), state->protocol, + tor_fw_options->internal_port, + tor_fw_options->external_port, + state->lease); + if (tor_fw_options->verbose) + fprintf(stdout, "tor-fw-helper: NAT-PMP sendnewportmappingrequest " + "returned %d (%s)\n", r, r==12?"SUCCESS":"FAILED"); + + do { + getnatpmprequesttimeout(&(state->natpmp), &timeout); + x = wait_until_fd_readable(state->natpmp.s, &timeout); + if (x == -1) + return -1; + + if (tor_fw_options->verbose) + fprintf(stdout, "V: attempting to readnatpmpreponseorretry...\n"); + r = readnatpmpresponseorretry(&(state->natpmp), &(state->response)); + sav_errno = errno; + + if (r<0 && r!=NATPMP_TRYAGAIN) { + fprintf(stderr, "E: readnatpmpresponseorretry failed %d\n", r); + fprintf(stderr, "E: errno=%d '%s'\n", sav_errno, + strerror(sav_errno)); + } + + } while (r == NATPMP_TRYAGAIN); + + if (r != 0) { + /* XXX TODO: NATPMP_* should be formatted into useful error strings */ + fprintf(stderr, "E: NAT-PMP It appears that something went wrong:" + " %d\n", r); + if (r == -51) + fprintf(stderr, "E: NAT-PMP It appears that the request was " + "unauthorized\n"); + return r; + } + + if (r == NATPMP_SUCCESS) { + fprintf(stdout, "tor-fw-helper: NAT-PMP mapped public port %hu to" + " localport %hu liftime %u\n", + (state->response).pnu.newportmapping.mappedpublicport, + (state->response).pnu.newportmapping.privateport, + (state->response).pnu.newportmapping.lifetime); + } + + tor_fw_options->nat_pmp_status = 1; + + return r; +} + +/** Fetch our likely public IP from our upstream NAT-PMP enabled NAT device. + * Use the connection context stored in <b>backend_state</b>. */ +int +tor_natpmp_fetch_public_ip(tor_fw_options_t *tor_fw_options, + void *backend_state) +{ + int r = 0; + int x = 0; + int sav_errno; + natpmp_state_t *state = (natpmp_state_t *) backend_state; + + struct timeval timeout; + + r = sendpublicaddressrequest(&(state->natpmp)); + fprintf(stdout, "tor-fw-helper: NAT-PMP sendpublicaddressrequest returned" + " %d (%s)\n", r, r==2?"SUCCESS":"FAILED"); + + do { + getnatpmprequesttimeout(&(state->natpmp), &timeout); + + x = wait_until_fd_readable(state->natpmp.s, &timeout); + if (x == -1) + return -1; + + if (tor_fw_options->verbose) + fprintf(stdout, "V: NAT-PMP attempting to read reponse...\n"); + r = readnatpmpresponseorretry(&(state->natpmp), &(state->response)); + sav_errno = errno; + + if (tor_fw_options->verbose) + fprintf(stdout, "V: NAT-PMP readnatpmpresponseorretry returned" + " %d\n", r); + + if ( r < 0 && r != NATPMP_TRYAGAIN) { + fprintf(stderr, "E: NAT-PMP readnatpmpresponseorretry failed %d\n", + r); + fprintf(stderr, "E: NAT-PMP errno=%d '%s'\n", sav_errno, + strerror(sav_errno)); + } + + } while (r == NATPMP_TRYAGAIN ); + + if (r != 0) { + fprintf(stderr, "E: NAT-PMP It appears that something went wrong:" + " %d\n", r); + return r; + } + + fprintf(stdout, "tor-fw-helper: ExternalIPAddress = %s\n", + inet_ntoa((state->response).pnu.publicaddress.addr)); + tor_fw_options->public_ip_status = 1; + + if (tor_fw_options->verbose) { + fprintf(stdout, "V: result = %u\n", r); + fprintf(stdout, "V: type = %u\n", (state->response).type); + fprintf(stdout, "V: resultcode = %u\n", (state->response).resultcode); + fprintf(stdout, "V: epoch = %u\n", (state->response).epoch); + } + + return r; +} +#endif + diff --git a/src/tools/tor-fw-helper/tor-fw-helper-natpmp.h b/src/tools/tor-fw-helper/tor-fw-helper-natpmp.h new file mode 100644 index 0000000000..54f541bcf4 --- /dev/null +++ b/src/tools/tor-fw-helper/tor-fw-helper-natpmp.h @@ -0,0 +1,47 @@ +/* Copyright (c) 2010, Jacob Appelbaum, Steven J. Murdoch. + * Copyright (c) 2010-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file tor-fw-helper-natpmp.h + **/ + +#ifdef NAT_PMP +#ifndef _TOR_FW_HELPER_NATPMP_H +#define _TOR_FW_HELPER_NATPMP_H + +#include <natpmp.h> + +/** This is the default NAT-PMP lease time in seconds. */ +#define NATPMP_DEFAULT_LEASE 3600 +/** NAT-PMP has many codes for success; this is one of them. */ +#define NATPMP_SUCCESS 0 + +/** This is our NAT-PMP meta structure - it holds our request data, responses, + * various NAT-PMP parameters, and of course the status of the motion in the + * NAT-PMP ocean. */ +typedef struct natpmp_state_t { + natpmp_t natpmp; + natpmpresp_t response; + int fetch_public_ip; + int status; + int init; /**< Have we been initialized? */ + int protocol; /**< This will only be TCP. */ + int lease; +} natpmp_state_t; + +const tor_fw_backend_t *tor_fw_get_natpmp_backend(void); + +int tor_natpmp_init(tor_fw_options_t *tor_fw_options, void *backend_state); + +int tor_natpmp_cleanup(tor_fw_options_t *tor_fw_options, void *backend_state); + +int tor_natpmp_add_tcp_mapping(tor_fw_options_t *tor_fw_options, + void *backend_state); + +int tor_natpmp_fetch_public_ip(tor_fw_options_t *tor_fw_options, + void *backend_state); + +#endif +#endif + diff --git a/src/tools/tor-fw-helper/tor-fw-helper-upnp.c b/src/tools/tor-fw-helper/tor-fw-helper-upnp.c new file mode 100644 index 0000000000..7c104f11cd --- /dev/null +++ b/src/tools/tor-fw-helper/tor-fw-helper-upnp.c @@ -0,0 +1,197 @@ +/* Copyright (c) 2010, Jacob Appelbaum, Steven J. Murdoch. + * Copyright (c) 2010-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file tor-fw-helper-upnp.c + * \brief The implementation of our UPnP firewall helper. + **/ + +#include "orconfig.h" +#ifdef MINIUPNPC +#ifdef _WIN32 +#define STATICLIB +#endif +#include <stdint.h> +#include <string.h> +#include <stdio.h> + +#include <assert.h> + +#include "compat.h" +#include "tor-fw-helper.h" +#include "tor-fw-helper-upnp.h" + +/** UPnP timeout value. */ +#define UPNP_DISCOVER_TIMEOUT 2000 +/** Description of the port mapping in the UPnP table. */ +#define UPNP_DESC "Tor relay" + +/* XXX TODO: We should print these as a useful user string when we return the + * number to a user */ +/** Magic numbers as miniupnpc return codes. */ +#define UPNP_ERR_SUCCESS 0 +#define UPNP_ERR_NODEVICESFOUND 1 +#define UPNP_ERR_NOIGDFOUND 2 +#define UPNP_ERR_ADDPORTMAPPING 3 +#define UPNP_ERR_GETPORTMAPPING 4 +#define UPNP_ERR_DELPORTMAPPING 5 +#define UPNP_ERR_GETEXTERNALIP 6 +#define UPNP_ERR_INVAL 7 +#define UPNP_ERR_OTHER 8 +#define UPNP_SUCCESS 1 + +/** This hooks miniupnpc into our multi-backend API. */ +static tor_fw_backend_t tor_miniupnp_backend = { + "miniupnp", + sizeof(struct miniupnpc_state_t), + tor_upnp_init, + tor_upnp_cleanup, + tor_upnp_fetch_public_ip, + tor_upnp_add_tcp_mapping +}; + +/** Return the backend for miniupnp. */ +const tor_fw_backend_t * +tor_fw_get_miniupnp_backend(void) +{ + return &tor_miniupnp_backend; +} + +/** Initialize the UPnP backend and store the results in + * <b>backend_state</b>.*/ +int +tor_upnp_init(tor_fw_options_t *options, void *backend_state) +{ + /* + This leaks the user agent from the client to the router - perhaps we don't + want to do that? eg: + + User-Agent: Ubuntu/10.04, UPnP/1.0, MiniUPnPc/1.4 + + */ + miniupnpc_state_t *state = (miniupnpc_state_t *) backend_state; + struct UPNPDev *devlist; + int r; + + memset(&(state->urls), 0, sizeof(struct UPNPUrls)); + memset(&(state->data), 0, sizeof(struct IGDdatas)); + state->init = 0; + +#ifdef MINIUPNPC15 + devlist = upnpDiscover(UPNP_DISCOVER_TIMEOUT, NULL, NULL, 0); +#else + devlist = upnpDiscover(UPNP_DISCOVER_TIMEOUT, NULL, NULL, 0, 0, NULL); +#endif + if (NULL == devlist) { + fprintf(stderr, "E: upnpDiscover returned: NULL\n"); + return UPNP_ERR_NODEVICESFOUND; + } + + assert(options); + r = UPNP_GetValidIGD(devlist, &(state->urls), &(state->data), + state->lanaddr, UPNP_LANADDR_SZ); + fprintf(stdout, "tor-fw-helper: UPnP GetValidIGD returned: %d (%s)\n", r, + r==UPNP_SUCCESS?"SUCCESS":"FAILED"); + + freeUPNPDevlist(devlist); + + if (r != 1 && r != 2) + return UPNP_ERR_NOIGDFOUND; + + state->init = 1; + return UPNP_ERR_SUCCESS; +} + +/** Tear down the UPnP connection stored in <b>backend_state</b>.*/ +int +tor_upnp_cleanup(tor_fw_options_t *options, void *backend_state) +{ + + miniupnpc_state_t *state = (miniupnpc_state_t *) backend_state; + assert(options); + + if (state->init) + FreeUPNPUrls(&(state->urls)); + state->init = 0; + + return UPNP_ERR_SUCCESS; +} + +/** Fetch our likely public IP from our upstream UPnP IGD enabled NAT device. + * Use the connection context stored in <b>backend_state</b>. */ +int +tor_upnp_fetch_public_ip(tor_fw_options_t *options, void *backend_state) +{ + miniupnpc_state_t *state = (miniupnpc_state_t *) backend_state; + int r; + char externalIPAddress[16]; + + if (!state->init) { + r = tor_upnp_init(options, state); + if (r != UPNP_ERR_SUCCESS) + return r; + } + + r = UPNP_GetExternalIPAddress(state->urls.controlURL, + state->data.first.servicetype, + externalIPAddress); + + if (r != UPNPCOMMAND_SUCCESS) + goto err; + + if (externalIPAddress[0]) { + fprintf(stdout, "tor-fw-helper: ExternalIPAddress = %s\n", + externalIPAddress); tor_upnp_cleanup(options, state); + options->public_ip_status = 1; + return UPNP_ERR_SUCCESS; + } else { + goto err; + } + + err: + tor_upnp_cleanup(options, state); + return UPNP_ERR_GETEXTERNALIP; +} + +/** Add a TCP port mapping for a single port stored in <b>tor_fw_options</b> + * and store the results in <b>backend_state</b>. */ +int +tor_upnp_add_tcp_mapping(tor_fw_options_t *options, void *backend_state) +{ + miniupnpc_state_t *state = (miniupnpc_state_t *) backend_state; + int r; + char internal_port_str[6]; + char external_port_str[6]; + + if (!state->init) { + r = tor_upnp_init(options, state); + if (r != UPNP_ERR_SUCCESS) + return r; + } + + if (options->verbose) + fprintf(stdout, "V: internal port: %d, external port: %d\n", + (int)options->internal_port, (int)options->external_port); + + tor_snprintf(internal_port_str, sizeof(internal_port_str), + "%d", (int)options->internal_port); + tor_snprintf(external_port_str, sizeof(external_port_str), + "%d", (int)options->external_port); + + r = UPNP_AddPortMapping(state->urls.controlURL, + state->data.first.servicetype, + external_port_str, internal_port_str, +#ifdef MINIUPNPC15 + state->lanaddr, UPNP_DESC, "TCP", 0); +#else + state->lanaddr, UPNP_DESC, "TCP", 0, 0); +#endif + if (r != UPNPCOMMAND_SUCCESS) + return UPNP_ERR_ADDPORTMAPPING; + + options->upnp_status = 1; + return UPNP_ERR_SUCCESS; +} +#endif + diff --git a/src/tools/tor-fw-helper/tor-fw-helper-upnp.h b/src/tools/tor-fw-helper/tor-fw-helper-upnp.h new file mode 100644 index 0000000000..f037c75bab --- /dev/null +++ b/src/tools/tor-fw-helper/tor-fw-helper-upnp.h @@ -0,0 +1,43 @@ +/* Copyright (c) 2010, Jacob Appelbaum, Steven J. Murdoch. + * Copyright (c) 2010-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file tor-fw-helper-upnp.h + * \brief The main header for our firewall helper. + **/ + +#ifdef MINIUPNPC +#ifndef _TOR_FW_HELPER_UPNP_H +#define _TOR_FW_HELPER_UPNP_H + +#include <miniupnpc/miniwget.h> +#include <miniupnpc/miniupnpc.h> +#include <miniupnpc/upnpcommands.h> +#include <miniupnpc/upnperrors.h> + +/** This is a magic number for miniupnpc lan address size. */ +#define UPNP_LANADDR_SZ 64 + +/** This is our miniupnpc meta structure - it holds our request data, + * responses, and various miniupnpc parameters. */ +typedef struct miniupnpc_state_t { + struct UPNPUrls urls; + struct IGDdatas data; + char lanaddr[UPNP_LANADDR_SZ]; + int init; +} miniupnpc_state_t; + +const tor_fw_backend_t *tor_fw_get_miniupnp_backend(void); + +int tor_upnp_init(tor_fw_options_t *options, void *backend_state); + +int tor_upnp_cleanup(tor_fw_options_t *options, void *backend_state); + +int tor_upnp_fetch_public_ip(tor_fw_options_t *options, void *backend_state); + +int tor_upnp_add_tcp_mapping(tor_fw_options_t *options, void *backend_state); + +#endif +#endif + diff --git a/src/tools/tor-fw-helper/tor-fw-helper.c b/src/tools/tor-fw-helper/tor-fw-helper.c new file mode 100644 index 0000000000..0510e65d11 --- /dev/null +++ b/src/tools/tor-fw-helper/tor-fw-helper.c @@ -0,0 +1,396 @@ +/* Copyright (c) 2010, Jacob Appelbaum, Steven J. Murdoch. + * Copyright (c) 2010-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file tor-fw-helper.c + * \brief The main wrapper around our firewall helper logic. + **/ + +/* + * tor-fw-helper is a tool for opening firewalls with NAT-PMP and UPnP; this + * tool is designed to be called by hand or by Tor by way of a exec() at a + * later date. + */ + +#include "orconfig.h" +#include <stdio.h> +#include <stdint.h> +#include <stdlib.h> +#include <getopt.h> +#include <time.h> +#include <string.h> + +#ifdef _WIN32 +#include <winsock2.h> +#endif + +#include "tor-fw-helper.h" +#ifdef NAT_PMP +#include "tor-fw-helper-natpmp.h" +#endif +#ifdef MINIUPNPC +#include "tor-fw-helper-upnp.h" +#endif + +/** This is our meta storage type - it holds information about each helper + including the total number of helper backends, function pointers, and helper + state. */ +typedef struct backends_t { + /** The total number of backends */ + int n_backends; + /** The backend functions as an array */ + tor_fw_backend_t backend_ops[MAX_BACKENDS]; + /** The internal backend state */ + void *backend_state[MAX_BACKENDS]; +} backends_t; + +/** Initalize each backend helper with the user input stored in <b>options</b> + * and put the results in the <b>backends</b> struct. */ +static int +init_backends(tor_fw_options_t *options, backends_t *backends) +{ + int n_available = 0; + int i, r, n; + tor_fw_backend_t *backend_ops_list[MAX_BACKENDS]; + void *data = NULL; + /* First, build a list of the working backends. */ + n = 0; +#ifdef MINIUPNPC + backend_ops_list[n++] = (tor_fw_backend_t *) tor_fw_get_miniupnp_backend(); +#endif +#ifdef NAT_PMP + backend_ops_list[n++] = (tor_fw_backend_t *) tor_fw_get_natpmp_backend(); +#endif + n_available = n; + + /* Now, for each backend that might work, try to initialize it. + * That's how we roll, initialized. + */ + n = 0; + for (i=0; i<n_available; ++i) { + data = calloc(1, backend_ops_list[i]->state_len); + if (!data) { + perror("calloc"); + exit(1); + } + r = backend_ops_list[i]->init(options, data); + if (r == 0) { + backends->backend_ops[n] = *backend_ops_list[i]; + backends->backend_state[n] = data; + n++; + } else { + free(data); + } + } + backends->n_backends = n; + + return n; +} + +/** Return the proper commandline switches when the user needs information. */ +static void +usage(void) +{ + fprintf(stderr, "tor-fw-helper usage:\n" + " [-h|--help]\n" + " [-T|--Test]\n" + " [-v|--verbose]\n" + " [-g|--fetch-public-ip]\n" + " -i|--internal-or-port [TCP port]\n" + " [-e|--external-or-port [TCP port]]\n" + " [-d|--internal-dir-port [TCP port]\n" + " [-p|--external-dir-port [TCP port]]]\n"); +} + +/** Log commandline options to a hardcoded file <b>tor-fw-helper.log</b> in the + * current working directory. */ +static int +log_commandline_options(int argc, char **argv) +{ + int i, retval; + FILE *logfile; + time_t now; + + /* Open the log file */ + logfile = fopen("tor-fw-helper.log", "a"); + if (NULL == logfile) + return -1; + + /* Send all commandline arguments to the file */ + now = time(NULL); + retval = fprintf(logfile, "START: %s\n", ctime(&now)); + for (i = 0; i < argc; i++) { + retval = fprintf(logfile, "ARG: %d: %s\n", i, argv[i]); + if (retval < 0) + goto error; + + retval = fprintf(stdout, "ARG: %d: %s\n", i, argv[i]); + if (retval < 0) + goto error; + } + now = time(NULL); + retval = fprintf(logfile, "END: %s\n", ctime(&now)); + + /* Close and clean up */ + retval = fclose(logfile); + return retval; + + /* If there was an error during writing */ + error: + fclose(logfile); + return -1; +} + +/** Iterate over over each of the supported <b>backends</b> and attempt to + * fetch the public ip. */ +static void +tor_fw_fetch_public_ip(tor_fw_options_t *tor_fw_options, + backends_t *backends) +{ + int i; + int r = 0; + + if (tor_fw_options->verbose) + fprintf(stdout, "V: tor_fw_fetch_public_ip\n"); + + for (i=0; i<backends->n_backends; ++i) { + if (tor_fw_options->verbose) { + fprintf(stdout, "V: running backend_state now: %i\n", i); + fprintf(stdout, "V: size of backend state: %u\n", + (int)(backends->backend_ops)[i].state_len); + fprintf(stdout, "V: backend state name: %s\n", + (char *)(backends->backend_ops)[i].name); + } + r = backends->backend_ops[i].fetch_public_ip(tor_fw_options, + backends->backend_state[i]); + fprintf(stdout, "tor-fw-helper: tor_fw_fetch_public_ip backend %s " + " returned: %i\n", (char *)(backends->backend_ops)[i].name, r); + } +} + +/** Iterate over each of the supported <b>backends</b> and attempt to add a + * port forward for the OR port stored in <b>tor_fw_options</b>. */ +static void +tor_fw_add_or_port(tor_fw_options_t *tor_fw_options, + backends_t *backends) +{ + int i; + int r = 0; + + if (tor_fw_options->verbose) + fprintf(stdout, "V: tor_fw_add_or_port\n"); + + for (i=0; i<backends->n_backends; ++i) { + if (tor_fw_options->verbose) { + fprintf(stdout, "V: running backend_state now: %i\n", i); + fprintf(stdout, "V: size of backend state: %u\n", + (int)(backends->backend_ops)[i].state_len); + fprintf(stdout, "V: backend state name: %s\n", + (const char *) backends->backend_ops[i].name); + } + r = backends->backend_ops[i].add_tcp_mapping(tor_fw_options, + backends->backend_state[i]); + fprintf(stdout, "tor-fw-helper: tor_fw_add_or_port backend %s " + "returned: %i\n", (const char *) backends->backend_ops[i].name, r); + } +} + +/** Iterate over each of the supported <b>backends</b> and attempt to add a + * port forward for the Dir port stored in <b>tor_fw_options</b>. */ +static void +tor_fw_add_dir_port(tor_fw_options_t *tor_fw_options, + backends_t *backends) +{ + int i; + int r = 0; + + if (tor_fw_options->verbose) + fprintf(stdout, "V: tor_fw_add_dir_port\n"); + + for (i=0; i<backends->n_backends; ++i) { + if (tor_fw_options->verbose) { + fprintf(stdout, "V: running backend_state now: %i\n", i); + fprintf(stdout, "V: size of backend state: %u\n", + (int)(backends->backend_ops)[i].state_len); + fprintf(stdout, "V: backend state name: %s\n", + (char *)(backends->backend_ops)[i].name); + } + r = backends->backend_ops[i].add_tcp_mapping(tor_fw_options, + backends->backend_state[i]); + fprintf(stdout, "tor-fw-helper: tor_fw_add_dir_port backend %s " + "returned: %i\n", (const char *)backends->backend_ops[i].name, r); + } +} + +/** Called before we make any calls to network-related functions. + * (Some operating systems require their network libraries to be + * initialized.) (from common/compat.c) */ +static int +network_init(void) +{ +#ifdef _WIN32 + /* This silly exercise is necessary before windows will allow + * gethostbyname to work. */ + WSADATA WSAData; + int r; + r = WSAStartup(0x101, &WSAData); + if (r) { + fprintf(stderr, "E: Error initializing Windows network layer " + "- code was %d", r); + return -1; + } + /* WSAData.iMaxSockets might show the max sockets we're allowed to use. + * We might use it to complain if we're trying to be a server but have + * too few sockets available. */ +#endif + return 0; +} + +int +main(int argc, char **argv) +{ + int r = 0; + int c = 0; + + tor_fw_options_t tor_fw_options; + backends_t backend_state; + + memset(&tor_fw_options, 0, sizeof(tor_fw_options)); + memset(&backend_state, 0, sizeof(backend_state)); + + while (1) { + int option_index = 0; + static struct option long_options[] = + { + {"verbose", 0, 0, 'v'}, + {"help", 0, 0, 'h'}, + {"internal-or-port", 1, 0, 'i'}, + {"external-or-port", 1, 0, 'e'}, + {"internal-dir-port", 1, 0, 'd'}, + {"external-dir-port", 1, 0, 'p'}, + {"fetch-public-ip", 0, 0, 'g'}, + {"test-commandline", 0, 0, 'T'}, + {0, 0, 0, 0} + }; + + c = getopt_long(argc, argv, "vhi:e:d:p:gT", + long_options, &option_index); + if (c == -1) + break; + + switch (c) { + case 'v': tor_fw_options.verbose = 1; break; + case 'h': tor_fw_options.help = 1; usage(); exit(1); break; + case 'i': sscanf(optarg, "%hu", &tor_fw_options.private_or_port); + break; + case 'e': sscanf(optarg, "%hu", &tor_fw_options.public_or_port); + break; + case 'd': sscanf(optarg, "%hu", &tor_fw_options.private_dir_port); + break; + case 'p': sscanf(optarg, "%hu", &tor_fw_options.public_dir_port); + break; + case 'g': tor_fw_options.fetch_public_ip = 1; break; + case 'T': tor_fw_options.test_commandline = 1; break; + case '?': break; + default : fprintf(stderr, "Unknown option!\n"); usage(); exit(1); + } + } + + if (tor_fw_options.verbose) { + fprintf(stderr, "V: tor-fw-helper version %s\n" + "V: We were called with the following arguments:\n" + "V: verbose = %d, help = %d, pub or port = %u, " + "priv or port = %u\n" + "V: pub dir port = %u, priv dir port = %u\n" + "V: fetch_public_ip = %u\n", + tor_fw_version, tor_fw_options.verbose, tor_fw_options.help, + tor_fw_options.private_or_port, tor_fw_options.public_or_port, + tor_fw_options.private_dir_port, tor_fw_options.public_dir_port, + tor_fw_options.fetch_public_ip); + } + + if (tor_fw_options.test_commandline) { + return log_commandline_options(argc, argv); + } + + /* At the very least, we require an ORPort; + Given a private ORPort, we can ask for a mapping that matches the port + externally. + */ + if (!tor_fw_options.private_or_port && !tor_fw_options.fetch_public_ip) { + fprintf(stderr, "E: We require an ORPort or fetch_public_ip" + " request!\n"); + usage(); + exit(1); + } else { + /* When we only have one ORPort, internal/external are + set to be the same.*/ + if (!tor_fw_options.public_or_port && tor_fw_options.private_or_port) { + if (tor_fw_options.verbose) + fprintf(stdout, "V: We're setting public_or_port = " + "private_or_port.\n"); + tor_fw_options.public_or_port = tor_fw_options.private_or_port; + } + } + if (!tor_fw_options.private_dir_port) { + if (tor_fw_options.verbose) + fprintf(stdout, "V: We have no DirPort; no hole punching for " + "DirPorts\n"); + + } else { + /* When we only have one DirPort, internal/external are + set to be the same.*/ + if (!tor_fw_options.public_dir_port && tor_fw_options.private_dir_port) { + if (tor_fw_options.verbose) + fprintf(stdout, "V: We're setting public_or_port = " + "private_or_port.\n"); + + tor_fw_options.public_dir_port = tor_fw_options.private_dir_port; + } + } + + if (tor_fw_options.verbose) { + fprintf(stdout, "V: pub or port = %u, priv or port = %u\n" + "V: pub dir port = %u, priv dir port = %u\n", + tor_fw_options.private_or_port, tor_fw_options.public_or_port, + tor_fw_options.private_dir_port, + tor_fw_options.public_dir_port); + } + + // Initialize networking + if (network_init()) + exit(1); + + // Initalize the various fw-helper backend helpers + r = init_backends(&tor_fw_options, &backend_state); + if (r) + printf("tor-fw-helper: %i NAT traversal helper(s) loaded\n", r); + + if (tor_fw_options.fetch_public_ip) { + tor_fw_fetch_public_ip(&tor_fw_options, &backend_state); + } + + if (tor_fw_options.private_or_port) { + tor_fw_options.internal_port = tor_fw_options.private_or_port; + tor_fw_options.external_port = tor_fw_options.private_or_port; + tor_fw_add_or_port(&tor_fw_options, &backend_state); + } + + if (tor_fw_options.private_dir_port) { + tor_fw_options.internal_port = tor_fw_options.private_dir_port; + tor_fw_options.external_port = tor_fw_options.private_dir_port; + tor_fw_add_dir_port(&tor_fw_options, &backend_state); + } + + r = (((tor_fw_options.nat_pmp_status | tor_fw_options.upnp_status) + |tor_fw_options.public_ip_status)); + if (r > 0) { + fprintf(stdout, "tor-fw-helper: SUCCESS\n"); + } else { + fprintf(stderr, "tor-fw-helper: FAILURE\n"); + } + + exit(r); +} + diff --git a/src/tools/tor-fw-helper/tor-fw-helper.h b/src/tools/tor-fw-helper/tor-fw-helper.h new file mode 100644 index 0000000000..058afc4e09 --- /dev/null +++ b/src/tools/tor-fw-helper/tor-fw-helper.h @@ -0,0 +1,57 @@ +/* Copyright (c) 2010, Jacob Appelbaum, Steven J. Murdoch. + * Copyright (c) 2010-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file tor-fw-helper.h + * \brief The main header for our firewall helper. + **/ + +#ifndef _TOR_FW_HELPER_H +#define _TOR_FW_HELPER_H + +#include <stdint.h> +#include <stdio.h> +#include <stdlib.h> +#include <getopt.h> +#include <time.h> + +/** The current version of tor-fw-helper. */ +#define tor_fw_version "0.1" + +/** This is an arbitrary hard limit - We currently have two (NAT-PMP and UPnP). + We're likely going to add the Intel UPnP library but nothing else comes to + mind at the moment. */ +#define MAX_BACKENDS 23 + +/** This is where we store parsed commandline options. */ +typedef struct { + int verbose; + int help; + int test_commandline; + uint16_t private_dir_port; + uint16_t private_or_port; + uint16_t public_dir_port; + uint16_t public_or_port; + uint16_t internal_port; + uint16_t external_port; + int fetch_public_ip; + int nat_pmp_status; + int upnp_status; + int public_ip_status; +} tor_fw_options_t; + +/** This is our main structure that defines our backend helper API; each helper + * must conform to these public methods if it expects to be handled in a + * non-special way. */ +typedef struct tor_fw_backend_t { + const char *name; + size_t state_len; + int (*init)(tor_fw_options_t *options, void *backend_state); + int (*cleanup)(tor_fw_options_t *options, void *backend_state); + int (*fetch_public_ip)(tor_fw_options_t *options, void *backend_state); + int (*add_tcp_mapping)(tor_fw_options_t *options, void *backend_state); +} tor_fw_backend_t; + +#endif + diff --git a/src/tools/tor-gencert.c b/src/tools/tor-gencert.c index 7a516b4571..c7ab8dc615 100644 --- a/src/tools/tor-gencert.c +++ b/src/tools/tor-gencert.c @@ -1,4 +1,4 @@ -/* Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ #include "orconfig.h" @@ -9,7 +9,9 @@ #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> +#ifdef HAVE_UNISTD_H #include <unistd.h> +#endif #include <openssl/evp.h> #include <openssl/pem.h> @@ -103,7 +105,7 @@ load_passphrase(void) cp = memchr(buf, '\n', n); passphrase_len = cp-buf; passphrase = tor_strndup(buf, passphrase_len); - memset(buf, 0, sizeof(buf)); + memwipe(buf, 0, sizeof(buf)); return 0; } @@ -111,7 +113,7 @@ static void clear_passphrase(void) { if (passphrase) { - memset(passphrase, 0, passphrase_len); + memwipe(passphrase, 0, passphrase_len); tor_free(passphrase); } } @@ -153,7 +155,7 @@ parse_commandline(int argc, char **argv) } months_lifetime = atoi(argv[++i]); if (months_lifetime > 24 || months_lifetime < 0) { - fprintf(stderr, "Lifetime (in months) was out of range."); + fprintf(stderr, "Lifetime (in months) was out of range.\n"); return 1; } } else if (!strcmp(argv[i], "-r") || !strcmp(argv[i], "--reuse")) { @@ -169,7 +171,7 @@ parse_commandline(int argc, char **argv) fprintf(stderr, "No argument to -a\n"); return 1; } - if (parse_addr_port(LOG_ERR, argv[++i], NULL, &addr, &port)<0) + if (addr_port_lookup(LOG_ERR, argv[++i], NULL, &addr, &port)<0) return 1; in.s_addr = htonl(addr); tor_inet_ntoa(&in, b, sizeof(b)); @@ -189,7 +191,7 @@ parse_commandline(int argc, char **argv) } } - memset(&s, 0, sizeof(s)); + memwipe(&s, 0, sizeof(s)); if (verbose) set_log_severity_config(LOG_DEBUG, LOG_ERR, &s); else @@ -222,13 +224,13 @@ static RSA * generate_key(int bits) { RSA *rsa = NULL; - crypto_pk_env_t *env = crypto_new_pk_env(); + crypto_pk_t *env = crypto_pk_new(); if (crypto_pk_generate_key_with_bits(env,bits)<0) goto done; - rsa = _crypto_pk_env_get_rsa(env); + rsa = _crypto_pk_get_rsa(env); rsa = RSAPrivateKey_dup(rsa); done: - crypto_free_pk_env(env); + crypto_pk_free(env); return rsa; } @@ -399,10 +401,10 @@ static int get_fingerprint(EVP_PKEY *pkey, char *out) { int r = 1; - crypto_pk_env_t *pk = _crypto_new_pk_env_rsa(EVP_PKEY_get1_RSA(pkey)); + crypto_pk_t *pk = _crypto_new_pk_from_rsa(EVP_PKEY_get1_RSA(pkey)); if (pk) { r = crypto_pk_get_fingerprint(pk, out, 0); - crypto_free_pk_env(pk); + crypto_pk_free(pk); } return r; } @@ -412,10 +414,10 @@ static int get_digest(EVP_PKEY *pkey, char *out) { int r = 1; - crypto_pk_env_t *pk = _crypto_new_pk_env_rsa(EVP_PKEY_get1_RSA(pkey)); + crypto_pk_t *pk = _crypto_new_pk_from_rsa(EVP_PKEY_get1_RSA(pkey)); if (pk) { r = crypto_pk_get_digest(pk, out); - crypto_free_pk_env(pk); + crypto_pk_free(pk); } return r; } @@ -495,7 +497,12 @@ generate_certificate(void) return 1; } - fputs(buf, f); + if (fputs(buf, f) < 0) { + log_err(LD_GENERAL, "Couldn't write to %s: %s", + certificate_file, strerror(errno)); + fclose(f); + return 1; + } fclose(f); return 0; } diff --git a/src/tools/tor-resolve.c b/src/tools/tor-resolve.c index 8c4d3f6483..4ef84f491c 100644 --- a/src/tools/tor-resolve.c +++ b/src/tools/tor-resolve.c @@ -1,5 +1,5 @@ /* Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson - * Copyright (c) 2007-2011, The Tor Project, Inc. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ /* See LICENSE for licensing information */ @@ -32,7 +32,7 @@ #include <errno.h> #endif -#ifdef MS_WINDOWS +#ifdef _WIN32 #if defined(_MSC_VER) && (_MSC_VER <= 1300) #include <winsock.h> #else @@ -393,7 +393,7 @@ main(int argc, char **argv) socksport = 9050; /* 9050 */ } } else if (n_args == 2) { - if (parse_addr_port(LOG_WARN, arg[1], NULL, &sockshost, &socksport)<0) { + if (addr_port_lookup(LOG_WARN, arg[1], NULL, &sockshost, &socksport)<0) { fprintf(stderr, "Couldn't parse/resolve address %s", arg[1]); return 1; } diff --git a/src/win32/orconfig.h b/src/win32/orconfig.h index 238b3f7ac9..d780d5d73d 100644 --- a/src/win32/orconfig.h +++ b/src/win32/orconfig.h @@ -3,8 +3,6 @@ */ /* Windows-only defines. */ -#define MS_WINDOWS -#define MS_WIN32 #define CONFDIR "" /* Define to 1 if you have the <arpa/inet.h> header file. */ @@ -122,6 +120,7 @@ /* Define to 1 if you have the <sys/socket.h> header file. */ #undef HAVE_SYS_SOCKET_H + /* Define to 1 if you have the <sys/stat.h> header file. */ #define HAVE_SYS_STAT_H @@ -233,5 +232,13 @@ #define USING_TWOS_COMPLEMENT /* Version number of package */ -#define VERSION "0.2.2.34-dev" +#define VERSION "0.2.3.18-rc-dev" + + +#define HAVE_STRUCT_SOCKADDR_IN6 +#define HAVE_STRUCT_IN6_ADDR +#define RSHIFT_DOES_SIGN_EXTEND +#define FLEXIBLE_ARRAY_MEMBER 0 +#define HAVE_EVENT2_EVENT_H +#define SHARE_DATADIR "" diff --git a/tor.spec.in b/tor.spec.in index 25557b73e2..b452c9bb35 100644 --- a/tor.spec.in +++ b/tor.spec.in @@ -6,7 +6,7 @@ # This should be incremented whenever the spec file changes, but # can drop back to zero at a new Tor version -%define specver 0 +%define specver 1 ## Things users may want to change # @@ -108,17 +108,14 @@ License: 3-clause BSD Vendor: The Tor Project (https://torproject.org) Packager: Erinn Clark <erinn@torproject.org> -%if %{is_suse} -Requires: openssl >= 0.9.7 -BuildRequires: openssl-devel >= 0.9.7, rpm >= 4.0, zlib-devel -%else -Requires: openssl >= 0.9.7 -BuildRequires: openssl-devel >= 0.9.7 -%endif -%if %{is_fc} -BuildRequires: rpm-build >= 4.0 +Requires: openssl >= 0.9.7, libevent >= 1.4.13 +BuildRequires: openssl-devel >= 0.9.7, libevent-devel >= 1.4.13, asciidoc + +# Fedora 16 and RHEL 5 have following conflicting packages according to rpm search +%if %{is_rh} Conflicts: tor-core, tor-lsb, tor-upstart %endif + Requires(pre): /usr/bin/id, /bin/date, /bin/sh Requires(pre): %{_sbindir}/useradd, %{_sbindir}/groupadd @@ -213,7 +210,7 @@ if [ ! -n "`/usr/bin/id -u %{toruser} 2>/dev/null`" ]; then # One would also like to default the UID, but doing that properly would # also require thought. if [ -x %{_sbindir}/nologin ]; then - %{_sbindir}/useradd -r -g %{torgroup} -d% {_localstatedir}/lib/%{name} -s %{_sbindir}/nologin %{toruser} 2> /dev/null + %{_sbindir}/useradd -r -g %{torgroup} -d %{_localstatedir}/lib/%{name} -s %{_sbindir}/nologin %{toruser} 2> /dev/null else %{_sbindir}/useradd -r -g %{torgroup} -d %{_localstatedir}/lib/%{name} -s /bin/false %{toruser} 2> /dev/null fi @@ -287,6 +284,14 @@ exit 0 %attr(0750,%{toruser},%{torgroup}) %dir %{_localstatedir}/log/%{name} %changelog +* Thu Jun 21 2012 Ondrej Mikle <ondrej.mikle@gmail.com> +- fixed to work with both rpmbuild and mock on RHEL/Fedora +- removed unnecessary files from rpm such as .git repo +- fixed build dependencies and package conflicts +- fixed creating _tor user on Fedora 17 (ancient typo) +- added/updated build instructions for RPM creation +- confirmed to build and run on EL5, EL6, Fedora 16/17, OpenSuse 12.1 + * Fri Aug 20 2010 Erinn Clark <erinn@torproject.org> - add conflicts for Fedora packages - add logic for SuSE since it requires special doc handling |