From cd0dd3e7238aeb9e0a83c2eba50ebfc6c4927788 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Sun, 3 Dec 2023 17:43:24 -0800 Subject: Work in progress on application-driven design goals for UDP --- proposals/339-udp-over-tor.md | 383 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 367 insertions(+), 16 deletions(-) diff --git a/proposals/339-udp-over-tor.md b/proposals/339-udp-over-tor.md index 12de0c6..289e962 100644 --- a/proposals/339-udp-over-tor.md +++ b/proposals/339-udp-over-tor.md @@ -11,7 +11,8 @@ Status: Accepted Tor currently only supports delivering two kinds of traffic to the internet: TCP data streams, and a certain limited subset of DNS requests. This proposal describes a plan to extend the Tor protocol so -that exit relays can also relay UDP traffic to the network?. +that exit relays can also relay some additional types of UDP traffic +to the network. Why would we want to do this? There are important protocols that use UDP, and in order to support users that rely on these protocols, we'll @@ -31,7 +32,16 @@ implemented some version of (relay fragment cells) so that we can transmit relay messages larger than 498 bytes. -# Overview +With the above in mind, the goal of this proposal is specifically to enable compatibility with more applications. +We will analyze some of those applications below, in order to choose an appropriate subset of UDP to implement which achieves our compatibility goals. +These compatibility goals will need to be weighed against any anonymity hazards or opportunities for abuse. + +We expect that work to improve round-trip time and jitter characteristics, such as Conflux, will benefit both TCP and UDP applications. +Many applications already support both transport protocols, and will perform similarly with or without this proposal. + +# UDP Traffic Models + +## User Datagram Protocol (RFC768) UDP is a datagram protocol; it allows messages of up to 65536 bytes, though in practice most protocols will use smaller messages in order to @@ -41,21 +51,342 @@ UDP messages can be dropped or re-ordered. There is no authentication or encryption baked into UDP, though it can be added by higher-level protocols like DTLS or QUIC. -When an application opens a UDP socket, the OS assigns it a 16-bit port -on some IP address of a local interface. The application may send -datagrams from that address:port combination, and will receive datagrams -sent to that address:port. +The "User Interface" suggested by RFC768 for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. + +Despite UDP's simplicity as an application of IP, we do need to be aware of IP features that are typically hidden by TCP's abstraction. +UDP applications typically try to obtain an awareness of the path MTU, using some type of path MTU discovery (PMTUD) algorithm. +On IPv4, this requires sending packets with the "Don't Fragment" flag set, and measuring when those packets are lost or when ICMP "Fragmentation Needed" replies are seen. + +Note that many applications have their own requirements for path MTU. For example, QUIC and common implementations of WebRTC require an MTU no smaller than 1200 bytes, but they can discover larger MTUs when available. + +## Socket Layer + +In practice the straightforward "User Interface" from RFC768, capable of arbitrary local address, is only available to privileged users. + +BSD-style sockets support UDP via `SOCK_DGRAM`. +UDP is a stateless protocol, but sockets do have state. +Each socket is bound, either explicitly with 'bind()' or automatically, to a source IP and port. + +At the API level, a socket is said to be _connected_ to a remote address:port if that address is the default destination. +A _connected_ socket will also filter out incoming packets with source addresses different from this default destination. + +A socket is considered _unconnected_ if 'connect()' has not been called. +These sockets have no default destination, and they accept datagrams from any source. + +There does not need to be any particular mapping between the lifetime of these application sockets and any higher-level "connection" the application establishes. +It's better to think of one socket as one allocated source port. +A typical application may allocate only a single port (one socket) for talking to many peers. +Every datagram sent or received on the socket may have a different peer address. + +## Network Address Translation (NAT) + +Much of the real-world complexity in UDP applications comes from their strategies to detect and overcome the effects of NAT. + +Many RFCs have been written on NAT behavior and NAT traversal strategies. +RFC4787 establishes best practices for NAT. +RFC8445 describes the Interactive Connectivity Establishment (ICE) protocol, which has become a very common application-level technique for building peer to peer applications that work through NAT. + +There are multiple fundamental technical issues that NAT presents: + +1. NAT must be stateful in order to route replies back to the correct source. + This directly conflicts with the stateless nature of UDP itself. + The NAT's mapping lifetime, determined by a timer, will not necessarily match the lifetime of the application-level connection. + This necessitates keep-alive packets in some protocols. + Protocols that allow their binding to expire may be open to a NAT rebinding attack, when a different party acquires access to the NAT's port allocation. +2. Applications no longer know an address they can be reached at without outside help. + Chosen port numbers may or may not be used by the NAT. + The effective IP address and port are not knowable without observing from an outside peer. +3. Filtering and mapping approaches both vary, and it's not generally possible to establish a connection without testing several alternatives and choosing the one that works. + This is the reason ICE exists, but it's also a possible anonymity hazard. + +We can use the constraints of NAT both to understand application behavior and to model Tor's behavior as a type of NAT. +In fact, Tor's many exit nodes already share similarity with some types of carrier-grade NAT. +Applications will need to assume very little about the IP address their outbound UDP originates on, and we can use that to our advantage in implementing UDP for Tor. + +TODO: More specifics here, more references. + +TODO: Maybe a whole section on the anonymity hazards from ICE. Not just the IP leak, but also the simultaneous sends on different interfaces. + +## Common Protocols + +Applications that want to use UDP are increasingly making use of higher-level protocols to avoid creating bespoke solutions for problems like NAT traversal, connection establishment, and reliable delivery. + +We will analyze how these protocols affect Tor's UDP traffic requirements. + +### QUIC + +RFC9000 defines QUIC, a multiplexed secure point-to-point protocol which supports reliable and unreliable delivery. The most common use is as an optional HTTP replacement, especially among Google services. + +QUIC does not normally try to traverse NAT; as an HTTP replacement, the server is expected to have a routable address. + +TODO: Look closer at the NAT rebinding attack described in the RFC, check how applicable that is for us. + +TODO: Is it a problem that NAT traversal plus QUIC effectively lets folks run a clearnet QUIC server on an exit node's IP? + +### WebRTC + +WebRTC is a large collection of protocols tuned to work together for media transport and NAT traversal. +It is increasingly common, both for browser-based telephony and for peer to peer data transfer. +Non-browser-based apps often implement WebRTC or have components in common with WebRTC. -With most (all?) IP stacks, a UDP socket can either be _connected_ to a -remote address:port (in which case all messages will be sent to that -address:port, and only messages from that address will be passed to the -application), or _unconnected_ (in which case outgoing messages can be -sent to any address:port, and incoming messages from any address:port -will be accepted). +See RFC7734, RFC8825, RFC8445, RFC5389, others. -In this version of the protocol, we support only _connected_ UDP -sockets, though we provide extension points for someday adding -_unconnected_ socket support. +TODO: Organize and expand references here. + +Of particular importance to us, WebRTC uses the Interactive Connection Establishment (ICE, RFC8445, RFC8838) protocol to establish a bidirectional channel between endpoints that may or may not be behind a NAT with unknown configuration. + +TODO: More about the specifics of how ICE effects us. More about privacy concerns. + +## Common Applications + +Some applications we've analyzed, in alphabetical order. + +### BitTorrent + +TODO: Analyze this (Incompatibility, abuse potential) + +### BigBlueButton + +TODO: Need to test. (probably in a similar boat to Jitsi) + +### Discord + +Discord is a popular chat app. Here I was testing a voice and video call directly between two users. + +The protocol is proprietary. UDP appears to be mandatory. Without UDP, the "RTC Connecting" status turns into "No Route" and the call fails. + +The communication I've observed is point to point UDP traffic to one server. +A server-reflexive IP address is visible as a string, suggesting behavior similar to STUN. + +We expect that UDP compatibility in Tor will likely make Discord usable. + +TODO: More testing could give us some confidence. Try running Discord over a UDP-capable tunnel that has latency/jitter characteristics similar to Tor. + +TODO: organize notes + +``` + - First try, UDP and P2P unblocked. I'm only seeing point to point UDP with port 50007 on a server. + - STUN-like messages early on, reflexive IP address appears as a text string + - After some time, still seeing no other checks and no p2p traffic. Just talking to that single addr and port 50007. + - Enabling UDP filter. Status at the top of "Connecting..." screen goes from "RTC Connecting" to "No Route" + - Retrying. Still "No Route". Call signaling goes through and I can join on the other side. Both sides fail to connect their real-time stream to the server. + - Disabling UDP filter, and discord immediately connects. +``` + +### DNS + +We would like to support arbitrary DNS queries. + +TODO: More detail here. NAT traversal isn't an issue, but we have others: + +- How this interacts with MTUs +- How we want to write exit policies for this +- Potential for abuse + +### FaceTime + +TODO: organize notes + +``` + - Trying to test the Android side specifically, with the remote end on macOS Monterey + - Link is of the form "https://facetime.apple.com/join#v=1&p=<>&k=<># where <> is a string possibly base64 + - Tried mobile firefox. Rejected by browser sniffing. Asks specifically for chrome or edge. + - I'm not getting direct P2P traffic in this setup, presumably because of strict firewall on the other end. This end is WebRTC of course. +Wireshark can see the STUN, DTLS, and SRTCP. Actual call traffic is point to point with apple's server, max size ~1170. Single peer. + - Filtering UDP and trying again. + - Now it's speaking TCP with the same IP address from earlier. TCP/80 specifically. Looks like TURN-over-TCP. + - Connection is good. + - Plaintext portion of STUN/TURN packets include text reference to WebRTC + - Retrying with Orbot on phone side. Mac is still behind the same restrictive NAT. + - Seems stable. Running ~30 mins fine, using 15-25 kB/sec +``` + +### Google Meet + +TODO: organize notes + +``` + - With UDP available, call very quickly sets up peer-to-peer UDP over the wifi. + - Even with local call data over wifi, uses UDP to send short messages over TURN to an external STUN/TURN server. (rtt pings?) + - Signaling over QUIC. Uses STUN/TURN extensively. Uses TURN even when the p2p channel is available. + - Connectivity checks start early in calling UI flow + - With UDP filtered, uses plaintext TURN over TCP (no SSL) + - Trying Orbot. Looks fine. Quality is smoothly adapting down. High latency but stable. Tested for >10min, no deterioration. +``` + +### Jitsi Meet + +Jitsi is an open source audio and video conferencing app, based on web technologies. +It uses WebRTC, with support for STUN and TURN, including UDP, TCP and TLS transports for TURN. + +The centrally hosted server (meet.jit.si) is configured with TCP support. +On a network with UDP blocked, it still functions. +However over Tor it has compatibility problems that seem unrelated to UDP. (HTTP 403 reported by the XMPP client) + +Self-hosted Jitsi instances often do not have TURN support. +Deploying a TURN server at all requires extra steps, and securing it with time-limited credentials requires additional customization at this time. + +We expect that UDP compatibility in Tor would allow many self-hosted Jitsi instances to host calls over Tor when they previously could not. We don't expect any change in the behavior at the centralized instance. + +TODO: organize notes + +``` + - First try, UDP and P2P unblocked. It's using both TURN and P2P (local wifi) traffic, seemingly for different streams. + - meet-jit-si-turnrelay.jitsi.net UDP/443. Messages in STUN extensions tell us this is Coturn-4.6.2 + - Retry with UDP filtered. Same server, switches to TCP/433 TURN-over-SSL for server connection. + - It notices that only the NAT'ed UDP is filtered, and starts doing UDP directly over the wifi for some of the traffic. The other portion keeps using TURN-over-SSL. + - Connection seems okay but didn't test extensively. Noticed 'bad signal' icon on one end. + - Trying with Orbot, expect to avoid local udp and to add tor latency. + - Over orbot, I get a "You have been disconnected" message suspiciously quickly. Switching to one-side-orbot to make this easier to debug. + - This is weird, wondering if something else is going on like possibly an intentional block. Look closer at this. + - When I have one side on orbot and one not, the non-orbot side reaches "You are the only one in the meeting" and it can hang out there. At this stage there's no stream traffic. + - Noticing some traffic to rtcstats-server.jitsi.net in the working trace at around the point where the orbot'ed side fails. This is all TCP/443 stuff though. + - Trying a new identity in orbot. + - Still immediate "You have been disconnected". Either tor is blocked or this is a problem with the stream that was going over local wifi before. + - Trying no orbot, UDP dropped, call hosted on phone and joined with desktop browser (chrome) + - This works! Smooth video call between desktop chrome jitsi webapp (unfiltered) and the UDP-filtered phone. Uses TURN-over-TLS for the entire call. Audio and video both work and stable. + - Can I add tor to this setup? Orbot and UDP filter phone hosting the call, browser joining? + - Same immediate "You have been disconnected". + - Ok, looks a lot like their TURN server has tor blocked. Need a different monitoring position to verify for sure. + + - Trying again with UDP blocked, and the TURN server is fine and the call is fine. Need to get access to trace in-between jitsi and tor. + - Can we try this using a transparent tor proxy on the openwrt side? + https://openwrt.org/docs/guide-user/services/tor/client + - Hmm. Trying to ignore the noise from other apps, and the re-connection attempt in jitsi that fails so quickly does not try to establish +any new connections, it's just a message exchanged with the main tcp/443 channel. + - Theory: the app is blocking based on IP address, either the fake addr used by the transproxy or the tor exit + - Can I repro that in another environment? + - Is there anything on logcat? + - Yes. JitsiMeetSDK messages are useful, as is filtering by PID. + - Is there anything useful in the public source code? + - lots here, would like to know where to start + +11-29 14:54:48.307 5948 5990 E JitsiMeetSDK: [modules/xmpp/strophe.util.js] Strophe: request id 258.1 error 403 happened +11-29 14:54:48.307 5948 5990 W JitsiMeetSDK: [modules/xmpp/strophe.util.js] Strophe: request errored, status: 403, number of errors: 1 +11-29 14:54:48.307 5948 5990 I JitsiMeetSDK: [modules/xmpp/xmpp.js] (TIME) Strophe disconnecting: 4123159.764245 +11-29 14:54:48.320 5948 5990 I JitsiMeetSDK: [modules/xmpp/xmpp.js] (TIME) Strophe disconnected: 4123163.070091 +11-29 14:54:48.321 5948 5990 I JitsiMeetSDK: [modules/xmpp/xmpp.js] {"environment":"meet-jit-si","envType":"prod","releaseNumber":"4517","shard":"meet-jit-si-us-ashburn-1-s6","region":"us-east-1","userRegion":"us-east-1","crossRegion":0,"id":"deployment_info"} +11-29 14:54:48.321 5948 5990 E JitsiMeetSDK: [modules/xmpp/xmpp.js] XMPP connection dropped! +11-29 14:54:48.321 5948 5990 I JitsiMeetSDK: [modules/statistics/statistics.js] {"type":"operational","action":"connection.failed","attributes":{"error_type":"connection.droppedError","error_message":"connection-dropped-error","shard_changed":true,"suspend_time":0,"time_since_last_success":null}} + +That 403 error looks important. Can we find that on the server side? Any indications of exactly what triggered this? +Just after this line which is fine but may indicate call preparatory info that could include IP addrs: + +11-29 14:59:52.934 5948 5990 D JitsiMeetSDK: [modules/RTC/RTCUtils.js] Available devices: [ { kind: 'videoinput', + +Can I check whether the behavior is the same using chrome? firefox? I need this to be easier to debug. +Uh. In browser (chrome + firefox) i'm getting a cloudflare popup that is not immediately succeeding. +Several retries with cloudflare... +And it's eventually working. audio is coming through. TURN over TLS. +Video still all black. Will it adapt down? +Trying again with chrome and video works. + +- Jitsi with local server (docker-compose) + + - Default setup uses centralized STUN server, doesn't support TURN + - I'm seeing all traffic going to the video bridge on udp/10000. no p2p traffic even on same wifi network. + - Needs further debug +``` + +### Signal + +Signal is a popular chat, voice, and video calling app with end to end encryption. +It is open source. +The protocol is based on WebRTC, implemented using a fork of Google's libwebsocket. + +Signal has good support for TCP over TURN. +Video and voice calls can be placed over Tor already, although it may take multiple attempts to establish a stable call. + +We expect that UDP compatibility in Tor would not have much effect on Signal. +We might improve latency slightly by removing the TURN-related hops, but the latency savings is expected to be small compared to the overall latency and jitter in Tor. + +### SIP + +TODO: Lots more detail here. + +- Overview/history of SIP +- Which configurations won't work +- Which configurations will +- How does this relate to Tor + +I'm still researching this but it looks like modern mobile-friendly SIP clients behave a lot like WebRTC. Historical SIP can require long lived port bindings with paired allocations, but I think any apps that require this would be broken for other reasons too. + +Looking for specific apps to test, I tried Linphone. It uses SIP over TLS for signaling, and it uses STUN and TURN for NAT traversal. However, there's no TCP support. Calls establish quickly, proceed with no audio for about 30 seconds while the app sends UDP packets that don't arrive, and then the call quietly ends. + +Expected that UDP support for Tor would likely allow some level of functionality from Linphone and other modern SIP softphones. +Any app that requires long-lived IP assignment or can't demultiplex RTP/RTCP on the same port will have trouble unless we take extraordinary measures to support these apps. + +### Skype + +TODO: organize notes + +``` + - Trying without any filtering. Quickly establishes direct p2p connection over wifi. + - Seems to be using STUN along with custom parts. Wireshark decodes some things and fails elsewhere. + - Peers: One local port talking to multiple peers, for NAT punching and for connectivity checks. (at least two stun-like servers in use) + - Trying with UDP filtered. + - Noticed STUN is happening while call is ringing, before it's been answered. + - With external UDP filtered, clients still find the local wifi p2p route. + - Turning on wifi client isolation. + - Now it's using TLS over TCP/443. Each call peer is talking to a different remote server. + - Seems like TURN-over-TLS. Starts this quickly after the STUN over UDP fails. + - Let's try over Orbot. + - Taking a while to establish a connection, quality is bad, but video and audio do come through. + - Trying again + - Quality still pretty bad, only using 10 kB/sec according to orbot. Might be an issue with skype's flow control? + - Letting it run to test stability. Running fine over orbot for 1hr. 20-40 kB/sec +``` + +### WhatsApp + +TODO: organize notes + +``` + - Already works with Orbot. Quality is pretty good. + - network traces show STUN, including STUN over TCP. unclear whether it uses ICE or WebRTC. + - This analysis claims it's not webrtc but SIP, also uses STUN: https://webrtchacks.com/whats-up-with-whatsapp-and-webrtc/ + - it's usage of STUN queries many servers in parallel using one local port. +``` + +### WiFi Calling + +TODO: I don't expect this to work, but we should make sure it's not breaking too badly. These are usually IPsec tunnels, not UDP. + +TODO: organize notes + +``` + - Noticed that these phones I'm testing (Verizon/Tracfone/BLU) are trying to do VPN traffic to wo.vzwwo.com which seems to be related to wifi calling. I don't have the accounts to test this, but if it continues to fail this may be a problem. +``` + +### Zoom + +TODO: organize notes + +``` + - Video call uses UDP if it can, but works with UDP firewalled off. + - Doesn't immediately look like STUN/TURN, but a custom thing with similar properties + - Not sure if it's trying to do NAT hole punching? Seems to be point-to-point UDP, it isn't even trying multiple servers. + - Disabling wifi client isolation, rebooting, and trying again to encourage a p2p conn. + - Does not appear to attempt a peer-to-peer connection at first. Simple 1:1 UDP with its server. + - zoompaoal220mmr.pao.zoom.us UDP/8801, TCP/443 + - AH. Eventually does switch to peer-to-peer, asynchronously. Two high UDP ports, using wlan local IP addrs. + - Handoff is managed using a custom protocol, signaling over their main TCP/443 channel, UDP connectivity checks with an all-caps UUID string visible inside. Stream handoff happens after connectivity checks finish, without interrupting stream. + - Various plaintext elements visible in UDP packets: UUID, strings like "detection_pkt_seq_no". Plaintext sequence numbers in the padding. Lots of padding, despite this padding being unencrypted and highly compressible. + - Video call goes over TCP/443 (and SSL) + - Trying Orbot. Quality is good! High latency but consistent video quality and audio is fine. + - Orbot bandwidth shows 80-200 kB/s each way. + - Connection seems stable, tested for 20 mins with no prob +``` + +## Malicious traffic + +TODO: Various kinds of traffic we want to avoid + +- Amplification attacks against arbitrary target (protocol contains reply addr, like SIP) +- Amplification attacks against exit relay (similar to attacks against NAT) +- Malicious fragmented traffic +- Excessive sends to a host that has never replied (DoS) +- Excessive number of peers (makes port scanning too much easier) # Tor protocol specification @@ -63,6 +394,8 @@ _unconnected_ socket support. ## Overview +TODO + We reserve three new relay commands: `CONNECT_UDP`, `CONNECTED_UDP` and `DATAGRAM`. @@ -115,6 +448,8 @@ can be mixed on the same circuit, but not on the same stream. ## Discussion on "too full" +TODO + (To be determined! We need an algorithm here before we implement, though our choice of algorithm doesn't need to be the same on all exits or for all clients, IIUC.) @@ -188,6 +523,8 @@ cell. ### CONNECT_UDP +TODO: Source port cookie, if we choose to have multiple streams per source port. (Under discussion still; see below) Flags for transmit/receive allowed. + ``` /* Tells an exit to connect a UDP port for connecting to a new target address. The stream ID is chosen by the client, and is part of @@ -220,6 +557,8 @@ cell behave with regards to the hostname given. ### CONNECTED_UDP +TODO: Do we need this at all? If a NAT is involved the info won't be correct. Even if it's correct, existing apps (connected via a VPN) wouldn't be able to make use of it. They would still talk to a STUN server. Do we care about UDP apps written directly to the Arti API? Are there non-malicious uses for this? + A CONNECTED_UDP cell sent in response to a CONNECT_UDP cell has the following format. @@ -242,6 +581,8 @@ the cell is considered malformed. ### DATAGRAM +TODO: This would need an additional byte or two of 'peer ID' if we choose to implement streams as 1:1 with local port allocations. This is still being discussed and a conclusion hasn't been reached, see below. + ``` struct datagram_body { /* The datagram body is the entire body of the message. @@ -256,8 +597,14 @@ We explicitly allow all END reasons from the existing Tor protocol. We may wish to add more as we gain experience with this protocol. +TODO: What does it mean for a stream to end here? This depends on how we choose to do port allocations, which needs more discussion. Should the stream lifetime and port allocation lifetime match, or should there be a separate timer system? + ### Extensions for unconnected sockets +TODO: This is flawed, the "connected/unconnected" idea is just a UNIX'ism and we should be thinking about streams and about how we allocate source ports. Also, limiting connections to a single peer breaks 100% of P2P connections, since they require using STUN (or similar) first. + +TODO: Right now beth and mike have different ideas for how this could work, and we need to discuss more. + Because of security concerns I don't suggest that we support unconnected sockets in the first version of this protocol. But _if we did_, here's how I'd suggest we do it. @@ -307,6 +654,8 @@ their exit policies did not say so. # MTU notes and issues +TODO: Revise this section. Nearly everything assumes the MTU is larger, and we are driving this design based on app compatibility. + Internet time. I might have this wrong. The "maximum safe IPv4 UDP payload" is "well known" to be only 508 bytes @@ -324,13 +673,13 @@ restrict their datagram size to fit into a transport this small. we'll probably be breaking a bunch of stuff, and creating a great deal of overhead.) - # Integration issues I do not know how applications should tell Tor that they want to use this feature. Any ideas? We should probably integrate with their MTU discovery systems too if we can. (TODO: write about some alternatives) +TODO: "Application" here being the VPN or the end-user app? We've discussed having a per-app opt-in for UDP, enforced at the VPN layer. # Resource management issues @@ -343,6 +692,8 @@ idea? # Security issues +TODO: update this + - Are there any major DoS or amplification attack vectors that this enables? I *think* no, because we don't allow spoofing the IP header. But maybe some wacky protocol out there lets you specify a -- cgit v1.2.3-54-g00ecf From 06b1aa5ecc0ac261472270b3d0a76ed031279195 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 7 Dec 2023 16:02:14 -0800 Subject: Some more on NAT behavior, some application notes --- proposals/339-udp-over-tor.md | 86 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 77 insertions(+), 9 deletions(-) diff --git a/proposals/339-udp-over-tor.md b/proposals/339-udp-over-tor.md index 289e962..3415890 100644 --- a/proposals/339-udp-over-tor.md +++ b/proposals/339-udp-over-tor.md @@ -83,8 +83,9 @@ Every datagram sent or received on the socket may have a different peer address. Much of the real-world complexity in UDP applications comes from their strategies to detect and overcome the effects of NAT. Many RFCs have been written on NAT behavior and NAT traversal strategies. -RFC4787 establishes best practices for NAT. -RFC8445 describes the Interactive Connectivity Establishment (ICE) protocol, which has become a very common application-level technique for building peer to peer applications that work through NAT. +RFC4787 and later RFC7857 offer best practices for implementing NAT. +Carrier-grade NAT requirements are addressed by RFC6888. +RFC8445 describes the Interactive Connectivity Establishment (ICE) protocol, which has become a common and recommended application-level technique for building peer to peer applications that work through NAT. There are multiple fundamental technical issues that NAT presents: @@ -99,13 +100,78 @@ There are multiple fundamental technical issues that NAT presents: 3. Filtering and mapping approaches both vary, and it's not generally possible to establish a connection without testing several alternatives and choosing the one that works. This is the reason ICE exists, but it's also a possible anonymity hazard. -We can use the constraints of NAT both to understand application behavior and to model Tor's behavior as a type of NAT. +We can use the constraints of NAT both to understand application behavior and as an opportunity to model Tor's behavior as a type of NAT. In fact, Tor's many exit nodes already share similarity with some types of carrier-grade NAT. Applications will need to assume very little about the IP address their outbound UDP originates on, and we can use that to our advantage in implementing UDP for Tor. -TODO: More specifics here, more references. - -TODO: Maybe a whole section on the anonymity hazards from ICE. Not just the IP leak, but also the simultaneous sends on different interfaces. +This body of IETF work is invaluable for understanding the scope of the problem and for defining common terminology. +We must take inspiration from these documents while also keeping in mind that the analogy between Tor and a NAT is imperfect. +For example, in analyzing Tor as a type of carrier-grade NAT, we may consider the "pooling behavior" defined in RFC4787: the choice of which external addresses map to an internal address. +Tor by necessity must carefully limit how predictable these mappings can ever be, to preserve its anonymity properties. +A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well as the various per-subscriber limiting requirements. + +Some edge cases must be carefully considered. +RFC4787 defines a framework for understanding the behavior of NAT by analyzing both its "mapping" and "filtering" behavior separately. +Mappings are the NAT's unit of state tracking. +Filters are layered on top of mappings, potentially rejecting incoming datagrams that don't match an already-expected address. +Both RFC4787 and the demands of peer to peer applications make a good case for always using an "Endpoint-Independent Mapping". + +Choice of filtering strategy is left open by the BEHAVE-WG recommendations. +RFC4787 defines three types with different properties, and does not make one single recommendation for all circumstances. +We can gain some additional insight by looking at requirements that come from outside RFC4787. + +- **Endpoint-Independent Filtering** allows incoming datagrams from any peer once a mapping has been established. + + RFC4787 recommends this approach, with the concession that it may not be ideal for all security requirements. + + In the context of Tor, we can likely rule out this technique entirely. + It makes traffic injection attacks possible from any source address, provided you can guess the UDP port number used at an exit. + It also makes possible clearnet hosting of UDP servers using an exit node's IP, which may have undesirable abuse properties. + + It precludes "Port overlapping" behavior as defined in RFC7857 section 3, which may be necessary in order to achieve sufficient utilization of local port numbers on exit nodes. + + It is still common for present-day applications to *prefer* endpoint-independent filtering, as it allows incoming connections from peers which cannot use STUN or a similar address fixing protocol. + Choosing endpoint-independent filtering would have *some* compatibility benefit, but among modern protocols which use ICE and STUN there would be no improvement. + The cost, on the other hand, would be an uncomfortably low-cost traffic injection attack and additional risks toward exit nodes. + +- **Address-Dependent Filtering** + + This is a permitted alternative according to RFC4787, in which incoming datagrams are allowed from only IP addresses we have previously sent to, but any port on that IP may be the sender. + + The intended benefits of this approach would be: + + - To support a class of applications that rely on, for a single local port, multiple remote ports achieving filter acceptance status when only one of those ports has been sent a datagram. + We are currently lacking examples of applications in this category. + Any application using ICE will be outside this category, as each port would have its own connectivity check datagrams exchanged in each direction. + + - To support peers using firewalls that are not compliant with RFC4787. This claim from the spec: + + ``` + However, if NAT-A uses Endpoint-Independent Filtering + or Address-Dependent Filtering, ICE will ultimately find + connectivity without requiring a UDP relay. + ``` + + TODO: Is this actually true? I haven't worked out how port-independent filtering is what makes this succeed. + + One security hazard of address-dependent and non-port-dependent filtering, identified in RFC4787, is that a peer on a NAT effectively negates the security benefits of this host filtering. + In fact, this should raise additional red flags as applied to either Tor or carrier grade NAT. + If we are intending to support peer to peer applications, it should be commonplace to establish UDP flows between two tor exit nodes. + When this takes place, non-port-dependent filtering would then allow anyone on Tor to connect via those same nodes and perform traffic injection. + The resulting security properties really become uncomfortably similar to endpoint-independent filtering. + +- **Address and Port-Dependent Filtering** + + This is the strictest variety of filtering, and it is an allowed alternative under RFC4787. + It provides opportunities for increased security and opportunities for reduced compatibility, both of which in practice may depend on other factors. + + For every application we've analyzed so far, port-dependent filtering is not a problem. + Usage of ICE will open all required filters during the connectivity check phase. + + This is the only type of filtering that provides any barrier at all between cross-circuit traffic injection when the communicating parties are known. + +RFC4787 recommends that filtering style be configurable. +We would like to implement that advice, but we are also looking for opportunities to make design decisions that give us the best network and end-user behaviors. ## Common Protocols @@ -121,8 +187,6 @@ QUIC does not normally try to traverse NAT; as an HTTP replacement, the server i TODO: Look closer at the NAT rebinding attack described in the RFC, check how applicable that is for us. -TODO: Is it a problem that NAT traversal plus QUIC effectively lets folks run a clearnet QUIC server on an exit node's IP? - ### WebRTC WebRTC is a large collection of protocols tuned to work together for media transport and NAT traversal. @@ -147,7 +211,11 @@ TODO: Analyze this (Incompatibility, abuse potential) ### BigBlueButton -TODO: Need to test. (probably in a similar boat to Jitsi) +BigBlueButton supports TURN over UDP and over TLS in the default configuration as of version 2.6. + +https://docs.bigbluebutton.org/administration/turn-server + +Confirmed this myself, and verified that BBB works over Tor already. ### Discord -- cgit v1.2.3-54-g00ecf From 094e0071c7fa7d23ac8a32cf3615b7b4b416d079 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 7 Dec 2023 17:18:29 -0800 Subject: Reduce application notes to a table --- proposals/339-udp-over-tor.md | 266 ++++-------------------------------------- 1 file changed, 24 insertions(+), 242 deletions(-) diff --git a/proposals/339-udp-over-tor.md b/proposals/339-udp-over-tor.md index 3415890..3395b0b 100644 --- a/proposals/339-udp-over-tor.md +++ b/proposals/339-udp-over-tor.md @@ -203,248 +203,30 @@ TODO: More about the specifics of how ICE effects us. More about privacy concern ## Common Applications -Some applications we've analyzed, in alphabetical order. - -### BitTorrent - -TODO: Analyze this (Incompatibility, abuse potential) - -### BigBlueButton - -BigBlueButton supports TURN over UDP and over TLS in the default configuration as of version 2.6. - -https://docs.bigbluebutton.org/administration/turn-server - -Confirmed this myself, and verified that BBB works over Tor already. - -### Discord - -Discord is a popular chat app. Here I was testing a voice and video call directly between two users. - -The protocol is proprietary. UDP appears to be mandatory. Without UDP, the "RTC Connecting" status turns into "No Route" and the call fails. - -The communication I've observed is point to point UDP traffic to one server. -A server-reflexive IP address is visible as a string, suggesting behavior similar to STUN. - -We expect that UDP compatibility in Tor will likely make Discord usable. - -TODO: More testing could give us some confidence. Try running Discord over a UDP-capable tunnel that has latency/jitter characteristics similar to Tor. - -TODO: organize notes - -``` - - First try, UDP and P2P unblocked. I'm only seeing point to point UDP with port 50007 on a server. - - STUN-like messages early on, reflexive IP address appears as a text string - - After some time, still seeing no other checks and no p2p traffic. Just talking to that single addr and port 50007. - - Enabling UDP filter. Status at the top of "Connecting..." screen goes from "RTC Connecting" to "No Route" - - Retrying. Still "No Route". Call signaling goes through and I can join on the other side. Both sides fail to connect their real-time stream to the server. - - Disabling UDP filter, and discord immediately connects. -``` - -### DNS - -We would like to support arbitrary DNS queries. - -TODO: More detail here. NAT traversal isn't an issue, but we have others: - -- How this interacts with MTUs -- How we want to write exit policies for this -- Potential for abuse - -### FaceTime - -TODO: organize notes - -``` - - Trying to test the Android side specifically, with the remote end on macOS Monterey - - Link is of the form "https://facetime.apple.com/join#v=1&p=<>&k=<># where <> is a string possibly base64 - - Tried mobile firefox. Rejected by browser sniffing. Asks specifically for chrome or edge. - - I'm not getting direct P2P traffic in this setup, presumably because of strict firewall on the other end. This end is WebRTC of course. -Wireshark can see the STUN, DTLS, and SRTCP. Actual call traffic is point to point with apple's server, max size ~1170. Single peer. - - Filtering UDP and trying again. - - Now it's speaking TCP with the same IP address from earlier. TCP/80 specifically. Looks like TURN-over-TCP. - - Connection is good. - - Plaintext portion of STUN/TURN packets include text reference to WebRTC - - Retrying with Orbot on phone side. Mac is still behind the same restrictive NAT. - - Seems stable. Running ~30 mins fine, using 15-25 kB/sec -``` - -### Google Meet - -TODO: organize notes - -``` - - With UDP available, call very quickly sets up peer-to-peer UDP over the wifi. - - Even with local call data over wifi, uses UDP to send short messages over TURN to an external STUN/TURN server. (rtt pings?) - - Signaling over QUIC. Uses STUN/TURN extensively. Uses TURN even when the p2p channel is available. - - Connectivity checks start early in calling UI flow - - With UDP filtered, uses plaintext TURN over TCP (no SSL) - - Trying Orbot. Looks fine. Quality is smoothly adapting down. High latency but stable. Tested for >10min, no deterioration. -``` - -### Jitsi Meet - -Jitsi is an open source audio and video conferencing app, based on web technologies. -It uses WebRTC, with support for STUN and TURN, including UDP, TCP and TLS transports for TURN. - -The centrally hosted server (meet.jit.si) is configured with TCP support. -On a network with UDP blocked, it still functions. -However over Tor it has compatibility problems that seem unrelated to UDP. (HTTP 403 reported by the XMPP client) - -Self-hosted Jitsi instances often do not have TURN support. -Deploying a TURN server at all requires extra steps, and securing it with time-limited credentials requires additional customization at this time. - -We expect that UDP compatibility in Tor would allow many self-hosted Jitsi instances to host calls over Tor when they previously could not. We don't expect any change in the behavior at the centralized instance. - -TODO: organize notes - -``` - - First try, UDP and P2P unblocked. It's using both TURN and P2P (local wifi) traffic, seemingly for different streams. - - meet-jit-si-turnrelay.jitsi.net UDP/443. Messages in STUN extensions tell us this is Coturn-4.6.2 - - Retry with UDP filtered. Same server, switches to TCP/433 TURN-over-SSL for server connection. - - It notices that only the NAT'ed UDP is filtered, and starts doing UDP directly over the wifi for some of the traffic. The other portion keeps using TURN-over-SSL. - - Connection seems okay but didn't test extensively. Noticed 'bad signal' icon on one end. - - Trying with Orbot, expect to avoid local udp and to add tor latency. - - Over orbot, I get a "You have been disconnected" message suspiciously quickly. Switching to one-side-orbot to make this easier to debug. - - This is weird, wondering if something else is going on like possibly an intentional block. Look closer at this. - - When I have one side on orbot and one not, the non-orbot side reaches "You are the only one in the meeting" and it can hang out there. At this stage there's no stream traffic. - - Noticing some traffic to rtcstats-server.jitsi.net in the working trace at around the point where the orbot'ed side fails. This is all TCP/443 stuff though. - - Trying a new identity in orbot. - - Still immediate "You have been disconnected". Either tor is blocked or this is a problem with the stream that was going over local wifi before. - - Trying no orbot, UDP dropped, call hosted on phone and joined with desktop browser (chrome) - - This works! Smooth video call between desktop chrome jitsi webapp (unfiltered) and the UDP-filtered phone. Uses TURN-over-TLS for the entire call. Audio and video both work and stable. - - Can I add tor to this setup? Orbot and UDP filter phone hosting the call, browser joining? - - Same immediate "You have been disconnected". - - Ok, looks a lot like their TURN server has tor blocked. Need a different monitoring position to verify for sure. - - - Trying again with UDP blocked, and the TURN server is fine and the call is fine. Need to get access to trace in-between jitsi and tor. - - Can we try this using a transparent tor proxy on the openwrt side? - https://openwrt.org/docs/guide-user/services/tor/client - - Hmm. Trying to ignore the noise from other apps, and the re-connection attempt in jitsi that fails so quickly does not try to establish -any new connections, it's just a message exchanged with the main tcp/443 channel. - - Theory: the app is blocking based on IP address, either the fake addr used by the transproxy or the tor exit - - Can I repro that in another environment? - - Is there anything on logcat? - - Yes. JitsiMeetSDK messages are useful, as is filtering by PID. - - Is there anything useful in the public source code? - - lots here, would like to know where to start - -11-29 14:54:48.307 5948 5990 E JitsiMeetSDK: [modules/xmpp/strophe.util.js] Strophe: request id 258.1 error 403 happened -11-29 14:54:48.307 5948 5990 W JitsiMeetSDK: [modules/xmpp/strophe.util.js] Strophe: request errored, status: 403, number of errors: 1 -11-29 14:54:48.307 5948 5990 I JitsiMeetSDK: [modules/xmpp/xmpp.js] (TIME) Strophe disconnecting: 4123159.764245 -11-29 14:54:48.320 5948 5990 I JitsiMeetSDK: [modules/xmpp/xmpp.js] (TIME) Strophe disconnected: 4123163.070091 -11-29 14:54:48.321 5948 5990 I JitsiMeetSDK: [modules/xmpp/xmpp.js] {"environment":"meet-jit-si","envType":"prod","releaseNumber":"4517","shard":"meet-jit-si-us-ashburn-1-s6","region":"us-east-1","userRegion":"us-east-1","crossRegion":0,"id":"deployment_info"} -11-29 14:54:48.321 5948 5990 E JitsiMeetSDK: [modules/xmpp/xmpp.js] XMPP connection dropped! -11-29 14:54:48.321 5948 5990 I JitsiMeetSDK: [modules/statistics/statistics.js] {"type":"operational","action":"connection.failed","attributes":{"error_type":"connection.droppedError","error_message":"connection-dropped-error","shard_changed":true,"suspend_time":0,"time_since_last_success":null}} - -That 403 error looks important. Can we find that on the server side? Any indications of exactly what triggered this? -Just after this line which is fine but may indicate call preparatory info that could include IP addrs: - -11-29 14:59:52.934 5948 5990 D JitsiMeetSDK: [modules/RTC/RTCUtils.js] Available devices: [ { kind: 'videoinput', - -Can I check whether the behavior is the same using chrome? firefox? I need this to be easier to debug. -Uh. In browser (chrome + firefox) i'm getting a cloudflare popup that is not immediately succeeding. -Several retries with cloudflare... -And it's eventually working. audio is coming through. TURN over TLS. -Video still all black. Will it adapt down? -Trying again with chrome and video works. - -- Jitsi with local server (docker-compose) - - - Default setup uses centralized STUN server, doesn't support TURN - - I'm seeing all traffic going to the video bridge on udp/10000. no p2p traffic even on same wifi network. - - Needs further debug -``` - -### Signal - -Signal is a popular chat, voice, and video calling app with end to end encryption. -It is open source. -The protocol is based on WebRTC, implemented using a fork of Google's libwebsocket. - -Signal has good support for TCP over TURN. -Video and voice calls can be placed over Tor already, although it may take multiple attempts to establish a stable call. - -We expect that UDP compatibility in Tor would not have much effect on Signal. -We might improve latency slightly by removing the TURN-related hops, but the latency savings is expected to be small compared to the overall latency and jitter in Tor. - -### SIP - -TODO: Lots more detail here. - -- Overview/history of SIP -- Which configurations won't work -- Which configurations will -- How does this relate to Tor - -I'm still researching this but it looks like modern mobile-friendly SIP clients behave a lot like WebRTC. Historical SIP can require long lived port bindings with paired allocations, but I think any apps that require this would be broken for other reasons too. - -Looking for specific apps to test, I tried Linphone. It uses SIP over TLS for signaling, and it uses STUN and TURN for NAT traversal. However, there's no TCP support. Calls establish quickly, proceed with no audio for about 30 seconds while the app sends UDP packets that don't arrive, and then the call quietly ends. - -Expected that UDP support for Tor would likely allow some level of functionality from Linphone and other modern SIP softphones. -Any app that requires long-lived IP assignment or can't demultiplex RTP/RTCP on the same port will have trouble unless we take extraordinary measures to support these apps. - -### Skype - -TODO: organize notes - -``` - - Trying without any filtering. Quickly establishes direct p2p connection over wifi. - - Seems to be using STUN along with custom parts. Wireshark decodes some things and fails elsewhere. - - Peers: One local port talking to multiple peers, for NAT punching and for connectivity checks. (at least two stun-like servers in use) - - Trying with UDP filtered. - - Noticed STUN is happening while call is ringing, before it's been answered. - - With external UDP filtered, clients still find the local wifi p2p route. - - Turning on wifi client isolation. - - Now it's using TLS over TCP/443. Each call peer is talking to a different remote server. - - Seems like TURN-over-TLS. Starts this quickly after the STUN over UDP fails. - - Let's try over Orbot. - - Taking a while to establish a connection, quality is bad, but video and audio do come through. - - Trying again - - Quality still pretty bad, only using 10 kB/sec according to orbot. Might be an issue with skype's flow control? - - Letting it run to test stability. Running fine over orbot for 1hr. 20-40 kB/sec -``` - -### WhatsApp - -TODO: organize notes - -``` - - Already works with Orbot. Quality is pretty good. - - network traces show STUN, including STUN over TCP. unclear whether it uses ICE or WebRTC. - - This analysis claims it's not webrtc but SIP, also uses STUN: https://webrtchacks.com/whats-up-with-whatsapp-and-webrtc/ - - it's usage of STUN queries many servers in parallel using one local port. -``` - -### WiFi Calling - -TODO: I don't expect this to work, but we should make sure it's not breaking too badly. These are usually IPsec tunnels, not UDP. - -TODO: organize notes - -``` - - Noticed that these phones I'm testing (Verizon/Tracfone/BLU) are trying to do VPN traffic to wo.vzwwo.com which seems to be related to wifi calling. I don't have the accounts to test this, but if it continues to fail this may be a problem. -``` - -### Zoom - -TODO: organize notes - -``` - - Video call uses UDP if it can, but works with UDP firewalled off. - - Doesn't immediately look like STUN/TURN, but a custom thing with similar properties - - Not sure if it's trying to do NAT hole punching? Seems to be point-to-point UDP, it isn't even trying multiple servers. - - Disabling wifi client isolation, rebooting, and trying again to encourage a p2p conn. - - Does not appear to attempt a peer-to-peer connection at first. Simple 1:1 UDP with its server. - - zoompaoal220mmr.pao.zoom.us UDP/8801, TCP/443 - - AH. Eventually does switch to peer-to-peer, asynchronously. Two high UDP ports, using wlan local IP addrs. - - Handoff is managed using a custom protocol, signaling over their main TCP/443 channel, UDP connectivity checks with an all-caps UUID string visible inside. Stream handoff happens after connectivity checks finish, without interrupting stream. - - Various plaintext elements visible in UDP packets: UUID, strings like "detection_pkt_seq_no". Plaintext sequence numbers in the padding. Lots of padding, despite this padding being unencrypted and highly compressible. - - Video call goes over TCP/443 (and SSL) - - Trying Orbot. Quality is good! High latency but consistent video quality and audio is fine. - - Orbot bandwidth shows 80-200 kB/s each way. - - Connection seems stable, tested for 20 mins with no prob -``` +With applications exhibiting such a wide variety of behaviors, how do we know what to expect from a good implementation? +How do we know which compatibility decisions will be most important to users? +For this it's helpful to look at specific application behaviors. +This is a best-effort analysis conducted at a point in time. +It's not meant to be a definitive reference, think of it as a site survey taken before we plan a building. + +In alphabetical order: + +| Application | Type | Protocol features | Current behavior | Expected outcome | +| ---------------------- | -------------- | --------------------------------- | ------------------ | -------------------------------------------------- | +| BitTorrent | File sharing | Many peers per local port | Fails without UDP | Works, new source of nuisance traffic | +| BigBlueButton | Telecom | WebRTC, TURN, TURN-over-TLS | Works | Slight latency improvement | +| Discord | Telecom | Proprietary, client/server | Fails without UDP | Starts working | +| DNS | Infrastructure | Might want IP fragmentation | Limited | Full DNS support, for better and worse | +| FaceTime | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | +| Google Meet | Telecom | STUN/TURN, TURN-over-TCP | Works | Slight latency improvement | +| Jitsi (meet.jit.si) | Telecom | WebRTC, TURN-over-TLS, Cloudflare | Fails on Tor | No change, problem does not appear UDP-related | +| Jitsi (docker-compose) | Telecom | WebRTC, centralized STUN only | Fails without UDP | Starts working | +| Linphone | Telecom (SIP) | SIP-over-TLS, STUN, TURN | Fails without UDP | Starts working | +| Signal | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | +| Skype | Telecom | P2P, STUN, TURN-over-TLS | Works | Slight latency improvement | +| WhatsApp | Telecom | STUN, TURN-over-TCP. Multi server | Works | Slight latency improvement | +| WiFi Calling | Telecom | IPsec tunnel | Out of scope | Still out of scope | +| Zoom | Telecom | client/server or P2P, UDP/TCP | Works | Slight latency improvement | ## Malicious traffic -- cgit v1.2.3-54-g00ecf From c1c238f4fa5bf318517091eab8695ba09dc3368c Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 7 Dec 2023 21:20:02 -0800 Subject: Thoughts on stream allocation pitfalls, and a suggestion --- proposals/339-udp-over-tor.md | 71 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 68 insertions(+), 3 deletions(-) diff --git a/proposals/339-udp-over-tor.md b/proposals/339-udp-over-tor.md index 3395b0b..2db0341 100644 --- a/proposals/339-udp-over-tor.md +++ b/proposals/339-udp-over-tor.md @@ -110,11 +110,10 @@ For example, in analyzing Tor as a type of carrier-grade NAT, we may consider th Tor by necessity must carefully limit how predictable these mappings can ever be, to preserve its anonymity properties. A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well as the various per-subscriber limiting requirements. -Some edge cases must be carefully considered. RFC4787 defines a framework for understanding the behavior of NAT by analyzing both its "mapping" and "filtering" behavior separately. Mappings are the NAT's unit of state tracking. Filters are layered on top of mappings, potentially rejecting incoming datagrams that don't match an already-expected address. -Both RFC4787 and the demands of peer to peer applications make a good case for always using an "Endpoint-Independent Mapping". +Both RFC4787 and the demands of peer to peer applications make a good case for always using an **Endpoint-Independent Mapping**. Choice of filtering strategy is left open by the BEHAVE-WG recommendations. RFC4787 defines three types with different properties, and does not make one single recommendation for all circumstances. @@ -239,8 +238,74 @@ TODO: Various kinds of traffic we want to avoid - Excessive number of peers (makes port scanning too much easier) -# Tor protocol specification +# Tor protocol design + +Using the specification- and application-based goals above, here we will briefly discuss the design constraints as they relate to Tor's protocol. + +## Stream usage + +An early design juncture in this project is the particular choice of scope for one *stream* in the existing Tor protocol. + +- One stream **per socket** was the approach suggested in an earlier version of this proposal. + + Each stream would match the lifetime of a source port allocation. + There would be a single peer address/port allowed per allocation. + This matches the usage of BSD-style sockets on which `connect()` has completed. + It's incompatible with many of the applications analyzed. + Multiple peers are typically needed for a variety of reasons, like connectivity checks or multi-region servers. + + This approach would be simplest to implement and specify. + It also unfortunately has very limited compatibility, and no clear path toward incremental upgrades if we wish to improve compatibility later. + +- One stream **per flow** has also been suggested. This would assign a stream ID to the combination of an allocated source port and a remote peer address/port. The flows may contain additional flags, like transmit and receive filtering, IPv4/v6, and *Don't Fragment*. + + This has advantages in keeping the datagram cells simple, with no additional IDs beyond the existing circuit ID. + It may also have advantages in DoS-prevention and in privacy analysis. + + By decoupling protocol features from the lifetime of sockets on the exit side, we facilitate implementing the desirable "Port overlapping" NAT behavior as mentioned above. + + Stream lifetimes, in this case, would not have any specific meaning other than the lifetime of the ID itself. + The bundle of flows associated with one source port would still all be limited to the lifetime of a Tor circuit, by scoping the source port identifier to be contained within the lifetime of its circuit. + + It would be necessary to allocate a new stream ID any time a new set of parameters (source port, remote address, remote port) is seen. + This would most commonly happen as a result of a first datagram sent to a new peer, coinciding with the establishment of a NAT-style mapping and the possible allocation of a socket on the exit. + A less common case needs to be considered too: what if the parameter tuple first occurs on the exit side? + We don't yet have a way to allocate stream IDs from either end of a circuit. + This would need to be considered, and the simplest solution might just be to partition the stream ID space into a half that can be allocated by each side. + This leaves a quite unfair split in the common case where streams are almost always allocated by the OP side. + + When is this exit-originated circuit ID allocation potentially needed? + It is clearly needed when using **address-dependent filtering**. + An incoming datagram from a previously-unseen peer port is expected to be deliverable, and the exit would need to allocate an ID for it. + Even with the stricter **address and port-dependent filtering** we may still be exposed to exit-originated circuit IDs if there are mismatches in the lifetime of the filter and the stream. + + This approach thus requires some attention to either correctly allocating stream IDs on both sides of the circuit, or choosing a filtering strategy and filter/mapping lifetime that does not ever leave stream IDs undefined when expecting incoming datagrams. + +- One stream **per mapping** is an alternative which attemps to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. + + A mapping would always be allocated from the OP side. + It could explicitly specify a filtering style, if we wish to allow applications to request non-port-dependent filtering for compatibility. + Each datagram within the stream would still need to be tagged with a peer address/port in some way. + + This puts us in a very similar design space to TURN, RFC8656. + In TURN, "allocations" are made explicitly on request, and assigned a random relayed port. + TURN also uses its "allocations" as an opportunity to support a *Don't Fragment* flag. + + Within the scope of one allocation, datagrams each still may have an arbitrary peer address. This takes up additional space in every datagram. + +- Hybrid approach, one stream **per mapping** and **per flow** at the same time + + We could try to combine the strengths of both approaches, using the lifetime of one stream to define a mapping and to carry otherwise-unbundled traffic while also allowing additional streams to bundle datagrams that would otherwise have repetetive headers. + + This avoids the space overhead of a purely **per mapping** approach and avoids the ID allocation and lifetime complexity introduced with **per flow**. + + This takes more inspiration from TURN, where commonly used peers will be defined as a "channel" with an especially short header. + In TURN, a "channel" is only ever allocated from the originating side of the connection. + Incoming datagrams with no channel can always be represented in the long form, so TURN never has to allocate channels unexpectedly. + + +# Tor protocol specification ## Overview -- cgit v1.2.3-54-g00ecf From aee32ef87eb76b1a0b67d2c32701c1f77772830c Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Mon, 11 Dec 2023 14:00:15 -0800 Subject: udp: quick notes on a few more areas to investigate --- proposals/339-udp-over-tor.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/proposals/339-udp-over-tor.md b/proposals/339-udp-over-tor.md index 2db0341..813531e 100644 --- a/proposals/339-udp-over-tor.md +++ b/proposals/339-udp-over-tor.md @@ -237,6 +237,19 @@ TODO: Various kinds of traffic we want to avoid - Excessive sends to a host that has never replied (DoS) - Excessive number of peers (makes port scanning too much easier) +See also RFC 7675, on the concept of "Send consent". + +# Anonymity risks + +TODO: ICE connectivity checks, as mentioned elsewhere. + +TODO: Are there plaintext identifiers in these telecom apps? + +TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? + +# Alternative designs + +TODO: Comparison vs. an entirely out-of-protocol and potentially out-of-process TURN server. Is this complexity warranted? # Tor protocol design -- cgit v1.2.3-54-g00ecf From 0dd88e6e89e620ab015fc402718bc6cb9ebaf4c1 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Wed, 20 Dec 2023 15:07:07 -0800 Subject: Revert proposal 339 to Nick's accepted version, split out new text The new file is now an unedited copy of the lines added in my diff. Future commits will clean this up. --- proposals/339-udp-over-tor.md | 313 +++------------------------------------ proposals/XXX-udp-app-support.md | 304 +++++++++++++++++++++++++++++++++++++ 2 files changed, 321 insertions(+), 296 deletions(-) create mode 100644 proposals/XXX-udp-app-support.md diff --git a/proposals/339-udp-over-tor.md b/proposals/339-udp-over-tor.md index 813531e..12de0c6 100644 --- a/proposals/339-udp-over-tor.md +++ b/proposals/339-udp-over-tor.md @@ -11,8 +11,7 @@ Status: Accepted Tor currently only supports delivering two kinds of traffic to the internet: TCP data streams, and a certain limited subset of DNS requests. This proposal describes a plan to extend the Tor protocol so -that exit relays can also relay some additional types of UDP traffic -to the network. +that exit relays can also relay UDP traffic to the network?. Why would we want to do this? There are important protocols that use UDP, and in order to support users that rely on these protocols, we'll @@ -32,16 +31,7 @@ implemented some version of (relay fragment cells) so that we can transmit relay messages larger than 498 bytes. -With the above in mind, the goal of this proposal is specifically to enable compatibility with more applications. -We will analyze some of those applications below, in order to choose an appropriate subset of UDP to implement which achieves our compatibility goals. -These compatibility goals will need to be weighed against any anonymity hazards or opportunities for abuse. - -We expect that work to improve round-trip time and jitter characteristics, such as Conflux, will benefit both TCP and UDP applications. -Many applications already support both transport protocols, and will perform similarly with or without this proposal. - -# UDP Traffic Models - -## User Datagram Protocol (RFC768) +# Overview UDP is a datagram protocol; it allows messages of up to 65536 bytes, though in practice most protocols will use smaller messages in order to @@ -51,278 +41,27 @@ UDP messages can be dropped or re-ordered. There is no authentication or encryption baked into UDP, though it can be added by higher-level protocols like DTLS or QUIC. -The "User Interface" suggested by RFC768 for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. - -Despite UDP's simplicity as an application of IP, we do need to be aware of IP features that are typically hidden by TCP's abstraction. -UDP applications typically try to obtain an awareness of the path MTU, using some type of path MTU discovery (PMTUD) algorithm. -On IPv4, this requires sending packets with the "Don't Fragment" flag set, and measuring when those packets are lost or when ICMP "Fragmentation Needed" replies are seen. - -Note that many applications have their own requirements for path MTU. For example, QUIC and common implementations of WebRTC require an MTU no smaller than 1200 bytes, but they can discover larger MTUs when available. - -## Socket Layer - -In practice the straightforward "User Interface" from RFC768, capable of arbitrary local address, is only available to privileged users. - -BSD-style sockets support UDP via `SOCK_DGRAM`. -UDP is a stateless protocol, but sockets do have state. -Each socket is bound, either explicitly with 'bind()' or automatically, to a source IP and port. - -At the API level, a socket is said to be _connected_ to a remote address:port if that address is the default destination. -A _connected_ socket will also filter out incoming packets with source addresses different from this default destination. - -A socket is considered _unconnected_ if 'connect()' has not been called. -These sockets have no default destination, and they accept datagrams from any source. - -There does not need to be any particular mapping between the lifetime of these application sockets and any higher-level "connection" the application establishes. -It's better to think of one socket as one allocated source port. -A typical application may allocate only a single port (one socket) for talking to many peers. -Every datagram sent or received on the socket may have a different peer address. - -## Network Address Translation (NAT) - -Much of the real-world complexity in UDP applications comes from their strategies to detect and overcome the effects of NAT. - -Many RFCs have been written on NAT behavior and NAT traversal strategies. -RFC4787 and later RFC7857 offer best practices for implementing NAT. -Carrier-grade NAT requirements are addressed by RFC6888. -RFC8445 describes the Interactive Connectivity Establishment (ICE) protocol, which has become a common and recommended application-level technique for building peer to peer applications that work through NAT. - -There are multiple fundamental technical issues that NAT presents: - -1. NAT must be stateful in order to route replies back to the correct source. - This directly conflicts with the stateless nature of UDP itself. - The NAT's mapping lifetime, determined by a timer, will not necessarily match the lifetime of the application-level connection. - This necessitates keep-alive packets in some protocols. - Protocols that allow their binding to expire may be open to a NAT rebinding attack, when a different party acquires access to the NAT's port allocation. -2. Applications no longer know an address they can be reached at without outside help. - Chosen port numbers may or may not be used by the NAT. - The effective IP address and port are not knowable without observing from an outside peer. -3. Filtering and mapping approaches both vary, and it's not generally possible to establish a connection without testing several alternatives and choosing the one that works. - This is the reason ICE exists, but it's also a possible anonymity hazard. - -We can use the constraints of NAT both to understand application behavior and as an opportunity to model Tor's behavior as a type of NAT. -In fact, Tor's many exit nodes already share similarity with some types of carrier-grade NAT. -Applications will need to assume very little about the IP address their outbound UDP originates on, and we can use that to our advantage in implementing UDP for Tor. - -This body of IETF work is invaluable for understanding the scope of the problem and for defining common terminology. -We must take inspiration from these documents while also keeping in mind that the analogy between Tor and a NAT is imperfect. -For example, in analyzing Tor as a type of carrier-grade NAT, we may consider the "pooling behavior" defined in RFC4787: the choice of which external addresses map to an internal address. -Tor by necessity must carefully limit how predictable these mappings can ever be, to preserve its anonymity properties. -A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well as the various per-subscriber limiting requirements. - -RFC4787 defines a framework for understanding the behavior of NAT by analyzing both its "mapping" and "filtering" behavior separately. -Mappings are the NAT's unit of state tracking. -Filters are layered on top of mappings, potentially rejecting incoming datagrams that don't match an already-expected address. -Both RFC4787 and the demands of peer to peer applications make a good case for always using an **Endpoint-Independent Mapping**. - -Choice of filtering strategy is left open by the BEHAVE-WG recommendations. -RFC4787 defines three types with different properties, and does not make one single recommendation for all circumstances. -We can gain some additional insight by looking at requirements that come from outside RFC4787. - -- **Endpoint-Independent Filtering** allows incoming datagrams from any peer once a mapping has been established. - - RFC4787 recommends this approach, with the concession that it may not be ideal for all security requirements. - - In the context of Tor, we can likely rule out this technique entirely. - It makes traffic injection attacks possible from any source address, provided you can guess the UDP port number used at an exit. - It also makes possible clearnet hosting of UDP servers using an exit node's IP, which may have undesirable abuse properties. - - It precludes "Port overlapping" behavior as defined in RFC7857 section 3, which may be necessary in order to achieve sufficient utilization of local port numbers on exit nodes. - - It is still common for present-day applications to *prefer* endpoint-independent filtering, as it allows incoming connections from peers which cannot use STUN or a similar address fixing protocol. - Choosing endpoint-independent filtering would have *some* compatibility benefit, but among modern protocols which use ICE and STUN there would be no improvement. - The cost, on the other hand, would be an uncomfortably low-cost traffic injection attack and additional risks toward exit nodes. - -- **Address-Dependent Filtering** - - This is a permitted alternative according to RFC4787, in which incoming datagrams are allowed from only IP addresses we have previously sent to, but any port on that IP may be the sender. - - The intended benefits of this approach would be: - - - To support a class of applications that rely on, for a single local port, multiple remote ports achieving filter acceptance status when only one of those ports has been sent a datagram. - We are currently lacking examples of applications in this category. - Any application using ICE will be outside this category, as each port would have its own connectivity check datagrams exchanged in each direction. - - - To support peers using firewalls that are not compliant with RFC4787. This claim from the spec: - - ``` - However, if NAT-A uses Endpoint-Independent Filtering - or Address-Dependent Filtering, ICE will ultimately find - connectivity without requiring a UDP relay. - ``` - - TODO: Is this actually true? I haven't worked out how port-independent filtering is what makes this succeed. - - One security hazard of address-dependent and non-port-dependent filtering, identified in RFC4787, is that a peer on a NAT effectively negates the security benefits of this host filtering. - In fact, this should raise additional red flags as applied to either Tor or carrier grade NAT. - If we are intending to support peer to peer applications, it should be commonplace to establish UDP flows between two tor exit nodes. - When this takes place, non-port-dependent filtering would then allow anyone on Tor to connect via those same nodes and perform traffic injection. - The resulting security properties really become uncomfortably similar to endpoint-independent filtering. - -- **Address and Port-Dependent Filtering** - - This is the strictest variety of filtering, and it is an allowed alternative under RFC4787. - It provides opportunities for increased security and opportunities for reduced compatibility, both of which in practice may depend on other factors. - - For every application we've analyzed so far, port-dependent filtering is not a problem. - Usage of ICE will open all required filters during the connectivity check phase. - - This is the only type of filtering that provides any barrier at all between cross-circuit traffic injection when the communicating parties are known. - -RFC4787 recommends that filtering style be configurable. -We would like to implement that advice, but we are also looking for opportunities to make design decisions that give us the best network and end-user behaviors. - -## Common Protocols - -Applications that want to use UDP are increasingly making use of higher-level protocols to avoid creating bespoke solutions for problems like NAT traversal, connection establishment, and reliable delivery. - -We will analyze how these protocols affect Tor's UDP traffic requirements. - -### QUIC - -RFC9000 defines QUIC, a multiplexed secure point-to-point protocol which supports reliable and unreliable delivery. The most common use is as an optional HTTP replacement, especially among Google services. - -QUIC does not normally try to traverse NAT; as an HTTP replacement, the server is expected to have a routable address. - -TODO: Look closer at the NAT rebinding attack described in the RFC, check how applicable that is for us. - -### WebRTC - -WebRTC is a large collection of protocols tuned to work together for media transport and NAT traversal. -It is increasingly common, both for browser-based telephony and for peer to peer data transfer. -Non-browser-based apps often implement WebRTC or have components in common with WebRTC. - -See RFC7734, RFC8825, RFC8445, RFC5389, others. - -TODO: Organize and expand references here. - -Of particular importance to us, WebRTC uses the Interactive Connection Establishment (ICE, RFC8445, RFC8838) protocol to establish a bidirectional channel between endpoints that may or may not be behind a NAT with unknown configuration. - -TODO: More about the specifics of how ICE effects us. More about privacy concerns. - -## Common Applications +When an application opens a UDP socket, the OS assigns it a 16-bit port +on some IP address of a local interface. The application may send +datagrams from that address:port combination, and will receive datagrams +sent to that address:port. -With applications exhibiting such a wide variety of behaviors, how do we know what to expect from a good implementation? -How do we know which compatibility decisions will be most important to users? -For this it's helpful to look at specific application behaviors. -This is a best-effort analysis conducted at a point in time. -It's not meant to be a definitive reference, think of it as a site survey taken before we plan a building. +With most (all?) IP stacks, a UDP socket can either be _connected_ to a +remote address:port (in which case all messages will be sent to that +address:port, and only messages from that address will be passed to the +application), or _unconnected_ (in which case outgoing messages can be +sent to any address:port, and incoming messages from any address:port +will be accepted). -In alphabetical order: - -| Application | Type | Protocol features | Current behavior | Expected outcome | -| ---------------------- | -------------- | --------------------------------- | ------------------ | -------------------------------------------------- | -| BitTorrent | File sharing | Many peers per local port | Fails without UDP | Works, new source of nuisance traffic | -| BigBlueButton | Telecom | WebRTC, TURN, TURN-over-TLS | Works | Slight latency improvement | -| Discord | Telecom | Proprietary, client/server | Fails without UDP | Starts working | -| DNS | Infrastructure | Might want IP fragmentation | Limited | Full DNS support, for better and worse | -| FaceTime | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | -| Google Meet | Telecom | STUN/TURN, TURN-over-TCP | Works | Slight latency improvement | -| Jitsi (meet.jit.si) | Telecom | WebRTC, TURN-over-TLS, Cloudflare | Fails on Tor | No change, problem does not appear UDP-related | -| Jitsi (docker-compose) | Telecom | WebRTC, centralized STUN only | Fails without UDP | Starts working | -| Linphone | Telecom (SIP) | SIP-over-TLS, STUN, TURN | Fails without UDP | Starts working | -| Signal | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | -| Skype | Telecom | P2P, STUN, TURN-over-TLS | Works | Slight latency improvement | -| WhatsApp | Telecom | STUN, TURN-over-TCP. Multi server | Works | Slight latency improvement | -| WiFi Calling | Telecom | IPsec tunnel | Out of scope | Still out of scope | -| Zoom | Telecom | client/server or P2P, UDP/TCP | Works | Slight latency improvement | - -## Malicious traffic - -TODO: Various kinds of traffic we want to avoid - -- Amplification attacks against arbitrary target (protocol contains reply addr, like SIP) -- Amplification attacks against exit relay (similar to attacks against NAT) -- Malicious fragmented traffic -- Excessive sends to a host that has never replied (DoS) -- Excessive number of peers (makes port scanning too much easier) - -See also RFC 7675, on the concept of "Send consent". - -# Anonymity risks - -TODO: ICE connectivity checks, as mentioned elsewhere. - -TODO: Are there plaintext identifiers in these telecom apps? - -TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? - -# Alternative designs - -TODO: Comparison vs. an entirely out-of-protocol and potentially out-of-process TURN server. Is this complexity warranted? - -# Tor protocol design - -Using the specification- and application-based goals above, here we will briefly discuss the design constraints as they relate to Tor's protocol. - -## Stream usage - -An early design juncture in this project is the particular choice of scope for one *stream* in the existing Tor protocol. - -- One stream **per socket** was the approach suggested in an earlier version of this proposal. - - Each stream would match the lifetime of a source port allocation. - There would be a single peer address/port allowed per allocation. - This matches the usage of BSD-style sockets on which `connect()` has completed. - It's incompatible with many of the applications analyzed. - Multiple peers are typically needed for a variety of reasons, like connectivity checks or multi-region servers. - - This approach would be simplest to implement and specify. - It also unfortunately has very limited compatibility, and no clear path toward incremental upgrades if we wish to improve compatibility later. - -- One stream **per flow** has also been suggested. This would assign a stream ID to the combination of an allocated source port and a remote peer address/port. The flows may contain additional flags, like transmit and receive filtering, IPv4/v6, and *Don't Fragment*. - - This has advantages in keeping the datagram cells simple, with no additional IDs beyond the existing circuit ID. - It may also have advantages in DoS-prevention and in privacy analysis. - - By decoupling protocol features from the lifetime of sockets on the exit side, we facilitate implementing the desirable "Port overlapping" NAT behavior as mentioned above. - - Stream lifetimes, in this case, would not have any specific meaning other than the lifetime of the ID itself. - The bundle of flows associated with one source port would still all be limited to the lifetime of a Tor circuit, by scoping the source port identifier to be contained within the lifetime of its circuit. - - It would be necessary to allocate a new stream ID any time a new set of parameters (source port, remote address, remote port) is seen. - This would most commonly happen as a result of a first datagram sent to a new peer, coinciding with the establishment of a NAT-style mapping and the possible allocation of a socket on the exit. - A less common case needs to be considered too: what if the parameter tuple first occurs on the exit side? - We don't yet have a way to allocate stream IDs from either end of a circuit. - This would need to be considered, and the simplest solution might just be to partition the stream ID space into a half that can be allocated by each side. - This leaves a quite unfair split in the common case where streams are almost always allocated by the OP side. - - When is this exit-originated circuit ID allocation potentially needed? - It is clearly needed when using **address-dependent filtering**. - An incoming datagram from a previously-unseen peer port is expected to be deliverable, and the exit would need to allocate an ID for it. - - Even with the stricter **address and port-dependent filtering** we may still be exposed to exit-originated circuit IDs if there are mismatches in the lifetime of the filter and the stream. - - This approach thus requires some attention to either correctly allocating stream IDs on both sides of the circuit, or choosing a filtering strategy and filter/mapping lifetime that does not ever leave stream IDs undefined when expecting incoming datagrams. - -- One stream **per mapping** is an alternative which attemps to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. - - A mapping would always be allocated from the OP side. - It could explicitly specify a filtering style, if we wish to allow applications to request non-port-dependent filtering for compatibility. - Each datagram within the stream would still need to be tagged with a peer address/port in some way. - - This puts us in a very similar design space to TURN, RFC8656. - In TURN, "allocations" are made explicitly on request, and assigned a random relayed port. - TURN also uses its "allocations" as an opportunity to support a *Don't Fragment* flag. - - Within the scope of one allocation, datagrams each still may have an arbitrary peer address. This takes up additional space in every datagram. - -- Hybrid approach, one stream **per mapping** and **per flow** at the same time - - We could try to combine the strengths of both approaches, using the lifetime of one stream to define a mapping and to carry otherwise-unbundled traffic while also allowing additional streams to bundle datagrams that would otherwise have repetetive headers. - - This avoids the space overhead of a purely **per mapping** approach and avoids the ID allocation and lifetime complexity introduced with **per flow**. - - This takes more inspiration from TURN, where commonly used peers will be defined as a "channel" with an especially short header. - In TURN, a "channel" is only ever allocated from the originating side of the connection. - Incoming datagrams with no channel can always be represented in the long form, so TURN never has to allocate channels unexpectedly. +In this version of the protocol, we support only _connected_ UDP +sockets, though we provide extension points for someday adding +_unconnected_ socket support. # Tor protocol specification -## Overview -TODO +## Overview We reserve three new relay commands: `CONNECT_UDP`, `CONNECTED_UDP` and `DATAGRAM`. @@ -376,8 +115,6 @@ can be mixed on the same circuit, but not on the same stream. ## Discussion on "too full" -TODO - (To be determined! We need an algorithm here before we implement, though our choice of algorithm doesn't need to be the same on all exits or for all clients, IIUC.) @@ -451,8 +188,6 @@ cell. ### CONNECT_UDP -TODO: Source port cookie, if we choose to have multiple streams per source port. (Under discussion still; see below) Flags for transmit/receive allowed. - ``` /* Tells an exit to connect a UDP port for connecting to a new target address. The stream ID is chosen by the client, and is part of @@ -485,8 +220,6 @@ cell behave with regards to the hostname given. ### CONNECTED_UDP -TODO: Do we need this at all? If a NAT is involved the info won't be correct. Even if it's correct, existing apps (connected via a VPN) wouldn't be able to make use of it. They would still talk to a STUN server. Do we care about UDP apps written directly to the Arti API? Are there non-malicious uses for this? - A CONNECTED_UDP cell sent in response to a CONNECT_UDP cell has the following format. @@ -509,8 +242,6 @@ the cell is considered malformed. ### DATAGRAM -TODO: This would need an additional byte or two of 'peer ID' if we choose to implement streams as 1:1 with local port allocations. This is still being discussed and a conclusion hasn't been reached, see below. - ``` struct datagram_body { /* The datagram body is the entire body of the message. @@ -525,14 +256,8 @@ We explicitly allow all END reasons from the existing Tor protocol. We may wish to add more as we gain experience with this protocol. -TODO: What does it mean for a stream to end here? This depends on how we choose to do port allocations, which needs more discussion. Should the stream lifetime and port allocation lifetime match, or should there be a separate timer system? - ### Extensions for unconnected sockets -TODO: This is flawed, the "connected/unconnected" idea is just a UNIX'ism and we should be thinking about streams and about how we allocate source ports. Also, limiting connections to a single peer breaks 100% of P2P connections, since they require using STUN (or similar) first. - -TODO: Right now beth and mike have different ideas for how this could work, and we need to discuss more. - Because of security concerns I don't suggest that we support unconnected sockets in the first version of this protocol. But _if we did_, here's how I'd suggest we do it. @@ -582,8 +307,6 @@ their exit policies did not say so. # MTU notes and issues -TODO: Revise this section. Nearly everything assumes the MTU is larger, and we are driving this design based on app compatibility. - Internet time. I might have this wrong. The "maximum safe IPv4 UDP payload" is "well known" to be only 508 bytes @@ -601,13 +324,13 @@ restrict their datagram size to fit into a transport this small. we'll probably be breaking a bunch of stuff, and creating a great deal of overhead.) + # Integration issues I do not know how applications should tell Tor that they want to use this feature. Any ideas? We should probably integrate with their MTU discovery systems too if we can. (TODO: write about some alternatives) -TODO: "Application" here being the VPN or the end-user app? We've discussed having a per-app opt-in for UDP, enforced at the VPN layer. # Resource management issues @@ -620,8 +343,6 @@ idea? # Security issues -TODO: update this - - Are there any major DoS or amplification attack vectors that this enables? I *think* no, because we don't allow spoofing the IP header. But maybe some wacky protocol out there lets you specify a diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md new file mode 100644 index 0000000..3a08706 --- /dev/null +++ b/proposals/XXX-udp-app-support.md @@ -0,0 +1,304 @@ +``` +Filename: XXX-udp-app-support.md +Title: Support for UDP Applications over Tor +Author: Micah Elizabeth Scott +Created: December 2023 +Status: Draft +``` + +that exit relays can also relay some additional types of UDP traffic +to the network. +With the above in mind, the goal of this proposal is specifically to enable compatibility with more applications. +We will analyze some of those applications below, in order to choose an appropriate subset of UDP to implement which achieves our compatibility goals. +These compatibility goals will need to be weighed against any anonymity hazards or opportunities for abuse. + +We expect that work to improve round-trip time and jitter characteristics, such as Conflux, will benefit both TCP and UDP applications. +Many applications already support both transport protocols, and will perform similarly with or without this proposal. + +# UDP Traffic Models + +## User Datagram Protocol (RFC768) +The "User Interface" suggested by RFC768 for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. +Despite UDP's simplicity as an application of IP, we do need to be aware of IP features that are typically hidden by TCP's abstraction. +UDP applications typically try to obtain an awareness of the path MTU, using some type of path MTU discovery (PMTUD) algorithm. +On IPv4, this requires sending packets with the "Don't Fragment" flag set, and measuring when those packets are lost or when ICMP "Fragmentation Needed" replies are seen. + +Note that many applications have their own requirements for path MTU. For example, QUIC and common implementations of WebRTC require an MTU no smaller than 1200 bytes, but they can discover larger MTUs when available. + +## Socket Layer +In practice the straightforward "User Interface" from RFC768, capable of arbitrary local address, is only available to privileged users. + +BSD-style sockets support UDP via `SOCK_DGRAM`. +UDP is a stateless protocol, but sockets do have state. +Each socket is bound, either explicitly with 'bind()' or automatically, to a source IP and port. +At the API level, a socket is said to be _connected_ to a remote address:port if that address is the default destination. +A _connected_ socket will also filter out incoming packets with source addresses different from this default destination. +A socket is considered _unconnected_ if 'connect()' has not been called. +These sockets have no default destination, and they accept datagrams from any source. + +There does not need to be any particular mapping between the lifetime of these application sockets and any higher-level "connection" the application establishes. +It's better to think of one socket as one allocated source port. +A typical application may allocate only a single port (one socket) for talking to many peers. +Every datagram sent or received on the socket may have a different peer address. + +## Network Address Translation (NAT) + +Much of the real-world complexity in UDP applications comes from their strategies to detect and overcome the effects of NAT. + +Many RFCs have been written on NAT behavior and NAT traversal strategies. +RFC4787 and later RFC7857 offer best practices for implementing NAT. +Carrier-grade NAT requirements are addressed by RFC6888. +RFC8445 describes the Interactive Connectivity Establishment (ICE) protocol, which has become a common and recommended application-level technique for building peer to peer applications that work through NAT. + +There are multiple fundamental technical issues that NAT presents: + +1. NAT must be stateful in order to route replies back to the correct source. + This directly conflicts with the stateless nature of UDP itself. + The NAT's mapping lifetime, determined by a timer, will not necessarily match the lifetime of the application-level connection. + This necessitates keep-alive packets in some protocols. + Protocols that allow their binding to expire may be open to a NAT rebinding attack, when a different party acquires access to the NAT's port allocation. +2. Applications no longer know an address they can be reached at without outside help. + Chosen port numbers may or may not be used by the NAT. + The effective IP address and port are not knowable without observing from an outside peer. +3. Filtering and mapping approaches both vary, and it's not generally possible to establish a connection without testing several alternatives and choosing the one that works. + This is the reason ICE exists, but it's also a possible anonymity hazard. + +We can use the constraints of NAT both to understand application behavior and as an opportunity to model Tor's behavior as a type of NAT. +In fact, Tor's many exit nodes already share similarity with some types of carrier-grade NAT. +Applications will need to assume very little about the IP address their outbound UDP originates on, and we can use that to our advantage in implementing UDP for Tor. + +This body of IETF work is invaluable for understanding the scope of the problem and for defining common terminology. +We must take inspiration from these documents while also keeping in mind that the analogy between Tor and a NAT is imperfect. +For example, in analyzing Tor as a type of carrier-grade NAT, we may consider the "pooling behavior" defined in RFC4787: the choice of which external addresses map to an internal address. +Tor by necessity must carefully limit how predictable these mappings can ever be, to preserve its anonymity properties. +A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well as the various per-subscriber limiting requirements. + +RFC4787 defines a framework for understanding the behavior of NAT by analyzing both its "mapping" and "filtering" behavior separately. +Mappings are the NAT's unit of state tracking. +Filters are layered on top of mappings, potentially rejecting incoming datagrams that don't match an already-expected address. +Both RFC4787 and the demands of peer to peer applications make a good case for always using an **Endpoint-Independent Mapping**. + +Choice of filtering strategy is left open by the BEHAVE-WG recommendations. +RFC4787 defines three types with different properties, and does not make one single recommendation for all circumstances. +We can gain some additional insight by looking at requirements that come from outside RFC4787. + +- **Endpoint-Independent Filtering** allows incoming datagrams from any peer once a mapping has been established. + + RFC4787 recommends this approach, with the concession that it may not be ideal for all security requirements. + + In the context of Tor, we can likely rule out this technique entirely. + It makes traffic injection attacks possible from any source address, provided you can guess the UDP port number used at an exit. + It also makes possible clearnet hosting of UDP servers using an exit node's IP, which may have undesirable abuse properties. + + It precludes "Port overlapping" behavior as defined in RFC7857 section 3, which may be necessary in order to achieve sufficient utilization of local port numbers on exit nodes. + + It is still common for present-day applications to *prefer* endpoint-independent filtering, as it allows incoming connections from peers which cannot use STUN or a similar address fixing protocol. + Choosing endpoint-independent filtering would have *some* compatibility benefit, but among modern protocols which use ICE and STUN there would be no improvement. + The cost, on the other hand, would be an uncomfortably low-cost traffic injection attack and additional risks toward exit nodes. + +- **Address-Dependent Filtering** + + This is a permitted alternative according to RFC4787, in which incoming datagrams are allowed from only IP addresses we have previously sent to, but any port on that IP may be the sender. + + The intended benefits of this approach would be: + + - To support a class of applications that rely on, for a single local port, multiple remote ports achieving filter acceptance status when only one of those ports has been sent a datagram. + We are currently lacking examples of applications in this category. + Any application using ICE will be outside this category, as each port would have its own connectivity check datagrams exchanged in each direction. + + - To support peers using firewalls that are not compliant with RFC4787. This claim from the spec: + + ``` + However, if NAT-A uses Endpoint-Independent Filtering + or Address-Dependent Filtering, ICE will ultimately find + connectivity without requiring a UDP relay. + ``` + + TODO: Is this actually true? I haven't worked out how port-independent filtering is what makes this succeed. + + One security hazard of address-dependent and non-port-dependent filtering, identified in RFC4787, is that a peer on a NAT effectively negates the security benefits of this host filtering. + In fact, this should raise additional red flags as applied to either Tor or carrier grade NAT. + If we are intending to support peer to peer applications, it should be commonplace to establish UDP flows between two tor exit nodes. + When this takes place, non-port-dependent filtering would then allow anyone on Tor to connect via those same nodes and perform traffic injection. + The resulting security properties really become uncomfortably similar to endpoint-independent filtering. + +- **Address and Port-Dependent Filtering** + + This is the strictest variety of filtering, and it is an allowed alternative under RFC4787. + It provides opportunities for increased security and opportunities for reduced compatibility, both of which in practice may depend on other factors. + + For every application we've analyzed so far, port-dependent filtering is not a problem. + Usage of ICE will open all required filters during the connectivity check phase. + + This is the only type of filtering that provides any barrier at all between cross-circuit traffic injection when the communicating parties are known. + +RFC4787 recommends that filtering style be configurable. +We would like to implement that advice, but we are also looking for opportunities to make design decisions that give us the best network and end-user behaviors. + +## Common Protocols + +Applications that want to use UDP are increasingly making use of higher-level protocols to avoid creating bespoke solutions for problems like NAT traversal, connection establishment, and reliable delivery. + +We will analyze how these protocols affect Tor's UDP traffic requirements. + +### QUIC + +RFC9000 defines QUIC, a multiplexed secure point-to-point protocol which supports reliable and unreliable delivery. The most common use is as an optional HTTP replacement, especially among Google services. + +QUIC does not normally try to traverse NAT; as an HTTP replacement, the server is expected to have a routable address. + +TODO: Look closer at the NAT rebinding attack described in the RFC, check how applicable that is for us. + +### WebRTC + +WebRTC is a large collection of protocols tuned to work together for media transport and NAT traversal. +It is increasingly common, both for browser-based telephony and for peer to peer data transfer. +Non-browser-based apps often implement WebRTC or have components in common with WebRTC. + +See RFC7734, RFC8825, RFC8445, RFC5389, others. + +TODO: Organize and expand references here. + +Of particular importance to us, WebRTC uses the Interactive Connection Establishment (ICE, RFC8445, RFC8838) protocol to establish a bidirectional channel between endpoints that may or may not be behind a NAT with unknown configuration. + +TODO: More about the specifics of how ICE effects us. More about privacy concerns. + +## Common Applications + +With applications exhibiting such a wide variety of behaviors, how do we know what to expect from a good implementation? +How do we know which compatibility decisions will be most important to users? +For this it's helpful to look at specific application behaviors. +This is a best-effort analysis conducted at a point in time. +It's not meant to be a definitive reference, think of it as a site survey taken before we plan a building. + +In alphabetical order: + +| Application | Type | Protocol features | Current behavior | Expected outcome | +| ---------------------- | -------------- | --------------------------------- | ------------------ | -------------------------------------------------- | +| BitTorrent | File sharing | Many peers per local port | Fails without UDP | Works, new source of nuisance traffic | +| BigBlueButton | Telecom | WebRTC, TURN, TURN-over-TLS | Works | Slight latency improvement | +| Discord | Telecom | Proprietary, client/server | Fails without UDP | Starts working | +| DNS | Infrastructure | Might want IP fragmentation | Limited | Full DNS support, for better and worse | +| FaceTime | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | +| Google Meet | Telecom | STUN/TURN, TURN-over-TCP | Works | Slight latency improvement | +| Jitsi (meet.jit.si) | Telecom | WebRTC, TURN-over-TLS, Cloudflare | Fails on Tor | No change, problem does not appear UDP-related | +| Jitsi (docker-compose) | Telecom | WebRTC, centralized STUN only | Fails without UDP | Starts working | +| Linphone | Telecom (SIP) | SIP-over-TLS, STUN, TURN | Fails without UDP | Starts working | +| Signal | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | +| Skype | Telecom | P2P, STUN, TURN-over-TLS | Works | Slight latency improvement | +| WhatsApp | Telecom | STUN, TURN-over-TCP. Multi server | Works | Slight latency improvement | +| WiFi Calling | Telecom | IPsec tunnel | Out of scope | Still out of scope | +| Zoom | Telecom | client/server or P2P, UDP/TCP | Works | Slight latency improvement | +## Malicious traffic + +TODO: Various kinds of traffic we want to avoid + +- Amplification attacks against arbitrary target (protocol contains reply addr, like SIP) +- Amplification attacks against exit relay (similar to attacks against NAT) +- Malicious fragmented traffic +- Excessive sends to a host that has never replied (DoS) +- Excessive number of peers (makes port scanning too much easier) + +See also RFC 7675, on the concept of "Send consent". + +# Anonymity risks + +TODO: ICE connectivity checks, as mentioned elsewhere. + +TODO: Are there plaintext identifiers in these telecom apps? + +TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? + +# Alternative designs + +TODO: Comparison vs. an entirely out-of-protocol and potentially out-of-process TURN server. Is this complexity warranted? + +# Tor protocol design + +Using the specification- and application-based goals above, here we will briefly discuss the design constraints as they relate to Tor's protocol. + +## Stream usage + +An early design juncture in this project is the particular choice of scope for one *stream* in the existing Tor protocol. + +- One stream **per socket** was the approach suggested in an earlier version of this proposal. + + Each stream would match the lifetime of a source port allocation. + There would be a single peer address/port allowed per allocation. + This matches the usage of BSD-style sockets on which `connect()` has completed. + It's incompatible with many of the applications analyzed. + Multiple peers are typically needed for a variety of reasons, like connectivity checks or multi-region servers. + + This approach would be simplest to implement and specify. + It also unfortunately has very limited compatibility, and no clear path toward incremental upgrades if we wish to improve compatibility later. + +- One stream **per flow** has also been suggested. This would assign a stream ID to the combination of an allocated source port and a remote peer address/port. The flows may contain additional flags, like transmit and receive filtering, IPv4/v6, and *Don't Fragment*. + + This has advantages in keeping the datagram cells simple, with no additional IDs beyond the existing circuit ID. + It may also have advantages in DoS-prevention and in privacy analysis. + + By decoupling protocol features from the lifetime of sockets on the exit side, we facilitate implementing the desirable "Port overlapping" NAT behavior as mentioned above. + + Stream lifetimes, in this case, would not have any specific meaning other than the lifetime of the ID itself. + The bundle of flows associated with one source port would still all be limited to the lifetime of a Tor circuit, by scoping the source port identifier to be contained within the lifetime of its circuit. + + It would be necessary to allocate a new stream ID any time a new set of parameters (source port, remote address, remote port) is seen. + This would most commonly happen as a result of a first datagram sent to a new peer, coinciding with the establishment of a NAT-style mapping and the possible allocation of a socket on the exit. + A less common case needs to be considered too: what if the parameter tuple first occurs on the exit side? + We don't yet have a way to allocate stream IDs from either end of a circuit. + This would need to be considered, and the simplest solution might just be to partition the stream ID space into a half that can be allocated by each side. + This leaves a quite unfair split in the common case where streams are almost always allocated by the OP side. + + When is this exit-originated circuit ID allocation potentially needed? + It is clearly needed when using **address-dependent filtering**. + An incoming datagram from a previously-unseen peer port is expected to be deliverable, and the exit would need to allocate an ID for it. + + Even with the stricter **address and port-dependent filtering** we may still be exposed to exit-originated circuit IDs if there are mismatches in the lifetime of the filter and the stream. + + This approach thus requires some attention to either correctly allocating stream IDs on both sides of the circuit, or choosing a filtering strategy and filter/mapping lifetime that does not ever leave stream IDs undefined when expecting incoming datagrams. + +- One stream **per mapping** is an alternative which attemps to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. + + A mapping would always be allocated from the OP side. + It could explicitly specify a filtering style, if we wish to allow applications to request non-port-dependent filtering for compatibility. + Each datagram within the stream would still need to be tagged with a peer address/port in some way. + + This puts us in a very similar design space to TURN, RFC8656. + In TURN, "allocations" are made explicitly on request, and assigned a random relayed port. + TURN also uses its "allocations" as an opportunity to support a *Don't Fragment* flag. + + Within the scope of one allocation, datagrams each still may have an arbitrary peer address. This takes up additional space in every datagram. + +- Hybrid approach, one stream **per mapping** and **per flow** at the same time + + We could try to combine the strengths of both approaches, using the lifetime of one stream to define a mapping and to carry otherwise-unbundled traffic while also allowing additional streams to bundle datagrams that would otherwise have repetetive headers. + + This avoids the space overhead of a purely **per mapping** approach and avoids the ID allocation and lifetime complexity introduced with **per flow**. + + This takes more inspiration from TURN, where commonly used peers will be defined as a "channel" with an especially short header. + In TURN, a "channel" is only ever allocated from the originating side of the connection. + Incoming datagrams with no channel can always be represented in the long form, so TURN never has to allocate channels unexpectedly. + + +# Tor protocol specification +TODO + +TODO + +TODO: Source port cookie, if we choose to have multiple streams per source port. (Under discussion still; see below) Flags for transmit/receive allowed. + +TODO: Do we need this at all? If a NAT is involved the info won't be correct. Even if it's correct, existing apps (connected via a VPN) wouldn't be able to make use of it. They would still talk to a STUN server. Do we care about UDP apps written directly to the Arti API? Are there non-malicious uses for this? + +TODO: This would need an additional byte or two of 'peer ID' if we choose to implement streams as 1:1 with local port allocations. This is still being discussed and a conclusion hasn't been reached, see below. + +TODO: What does it mean for a stream to end here? This depends on how we choose to do port allocations, which needs more discussion. Should the stream lifetime and port allocation lifetime match, or should there be a separate timer system? + +TODO: This is flawed, the "connected/unconnected" idea is just a UNIX'ism and we should be thinking about streams and about how we allocate source ports. Also, limiting connections to a single peer breaks 100% of P2P connections, since they require using STUN (or similar) first. + +TODO: Right now beth and mike have different ideas for how this could work, and we need to discuss more. + +TODO: Revise this section. Nearly everything assumes the MTU is larger, and we are driving this design based on app compatibility. + +TODO: "Application" here being the VPN or the end-user app? We've discussed having a per-app opt-in for UDP, enforced at the VPN layer. +TODO: update this + -- cgit v1.2.3-54-g00ecf From 9b886c681a35a9251bd63bb3c622fe1ca8c7cef5 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Wed, 20 Dec 2023 19:10:12 -0800 Subject: More writing and outlining Intro material is somewhat more there now, and I put in a very very rough sketch of the protocol-level recommendations at the very end. --- proposals/XXX-udp-app-support.md | 132 ++++++++++++++++++++++++++++++++------- 1 file changed, 108 insertions(+), 24 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index 3a08706..ce30bda 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -6,18 +6,60 @@ Created: December 2023 Status: Draft ``` -that exit relays can also relay some additional types of UDP traffic -to the network. -With the above in mind, the goal of this proposal is specifically to enable compatibility with more applications. -We will analyze some of those applications below, in order to choose an appropriate subset of UDP to implement which achieves our compatibility goals. -These compatibility goals will need to be weighed against any anonymity hazards or opportunities for abuse. +# Introduction -We expect that work to improve round-trip time and jitter characteristics, such as Conflux, will benefit both TCP and UDP applications. -Many applications already support both transport protocols, and will perform similarly with or without this proposal. +This proposal takes a fresh look at the problem of implementing support in Tor for applications which require UDP/IP communication. + +We start out by defining how this proposal compares to previous work, and the specific problem space we are addressing. +This leads into an analysis that references appropriate standards and proposes some specific solutions with properties we can compare. + +## History + +There have been multiple attempts within Tor to define some type of UDP support. + +Proposal 100 by Marc Liberatore in 2006 suggested a way to "add support for tunneling unreliable datagrams through tor with as few modifications to the protocol as possible." This proposal extends the existing protocol with a DTLS link mode alongside the existing TLS+TCP. The focus of this work was on a potential way to support unreliable traffic, not necessarily on UDP itself or on UDP applications. + +In proposal 100, a Tor "Stream" is used for one pairing of local and remote address and port, copying the technique used by Tor for TCP. This works for some types of UDP applications, as we detail below, but for many peer-to-peer use cases it's unhelpful to allocate local ports that can only be used with a single remote peer. + +Furthermore, no additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to a one Tor cell, a value that we will see is unusably small for most applications. + +It's possible these UDP protocol details would have been elaborated during design, but this proposal hit a snag elsewhere: there was no agreement on a way to avoid causing new attacks against anonymity. + +In 2018, Nick Mathewson and Mike Perry wrote this summary of the general issues with unreliable transports for Tor: + +https://research.torproject.org/techreports/side-channel-analysis-2018-11-27.pdf + +TODO: Look closer at this, some of the attacks described might overlap with non-datagram-transport ways of encapsulating UDP. (i.e. a malicious exit may always cause drops/injections regardless of what the rest of the anonymity network does.) + +TODO: There has got to be a section here for new risks with malicious exits. Some protocols may see increased risk vs. their existing TCP fallbacks. + +Proposal 339 by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor circuit using a new type of Tor message. As a prerequisite, it depends on proposal 319 to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. + +In proposal 339 the property of binding a stream both to a local port and to a remote peer is described in UNIX-style terminology as a "connected socket". We dive deeper into this idea below and supply an alternate formulation based on RFC4787, NAT behavior requirements for UDP. This single-peer behavior would be referred to as an "endpoint-dependent mapping" in RFC4787. It works fine for client/server apps but precludes the use of peer-to-peer techniques. + +## Scope + +This proposal aims to allow Tor applications and Tor-based VPNs to provide compatibility with applications that require UDP/IP communications. + +We don't have a specific list of applications that must be supported, but we are currently aiming for broad support of popular applications while still respecting and referencing all applicable Internet standards documents. + +Changes to the structure of the Tor network are out of scope, as are specific performance optimizations. We expect to rely on common optimizations to the performance of Tor circuits, rather than looking to make specific changes to optimize for unreliable datagram transmission. + +We will discuss the design implications of UDP on onion services below. +It's worth planning for this as a way to evaluate the design space, but in practice we are not aiming for UDP to onion services yet. +This will require changes to most applications that want to use it, as it implies that any media negotiations will need to understand onion addressing in addition to IPv4 and IPv6. + +We do not rigidly define the subset of UDP traffic that will be allowed. +There are several options discussed below using the framework of RFC4787 NAT recommendations. + +We do expect support for voice/video telecommunications apps. Even without an underlying transport that supports unreliable datagrams, we expect a tunnel to provide a usable level of compatibility. This design space is very similar to the TURN (RFC8656) specification, already used very widely for compatibility with networks that filter UDP. See the analysis of specific applications below. + +We do expect support for DNS clients. Tor currently only supports a limited subset of DNS queries, and it's desirable to support more. This will be analyzed in detail as an application below. DNS is one of very few applications that still rely on fragmented UDP datagrams, though this may not be relevant for us since only servers typically need to control the production of fragments. # UDP Traffic Models ## User Datagram Protocol (RFC768) + The "User Interface" suggested by RFC768 for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. Despite UDP's simplicity as an application of IP, we do need to be aware of IP features that are typically hidden by TCP's abstraction. UDP applications typically try to obtain an awareness of the path MTU, using some type of path MTU discovery (PMTUD) algorithm. @@ -26,6 +68,7 @@ On IPv4, this requires sending packets with the "Don't Fragment" flag set, and m Note that many applications have their own requirements for path MTU. For example, QUIC and common implementations of WebRTC require an MTU no smaller than 1200 bytes, but they can discover larger MTUs when available. ## Socket Layer + In practice the straightforward "User Interface" from RFC768, capable of arbitrary local address, is only available to privileged users. BSD-style sockets support UDP via `SOCK_DGRAM`. @@ -189,6 +232,7 @@ In alphabetical order: | WhatsApp | Telecom | STUN, TURN-over-TCP. Multi server | Works | Slight latency improvement | | WiFi Calling | Telecom | IPsec tunnel | Out of scope | Still out of scope | | Zoom | Telecom | client/server or P2P, UDP/TCP | Works | Slight latency improvement | + ## Malicious traffic TODO: Various kinds of traffic we want to avoid @@ -209,19 +253,35 @@ TODO: Are there plaintext identifiers in these telecom apps? TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? -# Alternative designs +# Design approaches + +## Datagram routing + +### 3rd party implementations + +### Hybrid approaches + +### Making room for a future Tor implementation + +## Tunneling + +### Using TURN (RFC8656) encapsulated in a Tor stream TODO: Comparison vs. an entirely out-of-protocol and potentially out-of-process TURN server. Is this complexity warranted? -# Tor protocol design +### Using Tor streams to an exit + +TODO: Bulk of the design goes here, this is what we likely start with -Using the specification- and application-based goals above, here we will briefly discuss the design constraints as they relate to Tor's protocol. +### Using Tor streams to a rendezvous point -## Stream usage +TODO: possible generalization of above to support onion UDP. big caveat that apps need extensions to support onion addressing. + +#### Stream usage An early design juncture in this project is the particular choice of scope for one *stream* in the existing Tor protocol. -- One stream **per socket** was the approach suggested in an earlier version of this proposal. +- One stream **per socket** was the approach suggested in Proposal 339. Each stream would match the lifetime of a source port allocation. There would be a single peer address/port allowed per allocation. @@ -279,26 +339,50 @@ An early design juncture in this project is the particular choice of scope for o In TURN, a "channel" is only ever allocated from the originating side of the connection. Incoming datagrams with no channel can always be represented in the long form, so TURN never has to allocate channels unexpectedly. +# Protocol Designs + +Currently we have two design candidates, and both are described here. + +TODO: Exit selection is likely common to both of these approaches. Talk about how we specify UDP exit policy, how we provide a smart tradeoff between ability to filter harmful traffic while avoiding an explosion in exit policy complexity. + +## Tor UDP Mapping Protocol + +This is *tunneled* approach, using Tor's existing protocol objects where possible. Exits advertise support for specific UDP policies. Clients choose an exit where they would like to allocate a UDP local port. They create one *stream* with a lifetime matching the lifetime of that mapping. Optional additional *streams* may be used as fast paths for common peers. This is the *hybrid approach* to stream usage, from above. + +- `NEW_UDP_MAPPING` -# Tor protocol specification -TODO + - always client-to-exit + - stream ID for mapping + - no address given + - "don't fragment" flag + - port-specific filtering flag + - no reply? early data always ok. + - ended on circuit teardown or by `END` -TODO +- `UDP_MAPPING_DATAGRAM` -TODO: Source port cookie, if we choose to have multiple streams per source port. (Under discussion still; see below) Flags for transmit/receive allowed. + - conveys one datagram on a stream defined by ALLOCATE_UDP_MAPPING. + - Includes peer address (ipv4/ipv6) as well as datagram content -TODO: Do we need this at all? If a NAT is involved the info won't be correct. Even if it's correct, existing apps (connected via a VPN) wouldn't be able to make use of it. They would still talk to a STUN server. Do we care about UDP apps written directly to the Arti API? Are there non-malicious uses for this? +- `NEW_UDP_FLOW` -TODO: This would need an additional byte or two of 'peer ID' if we choose to implement streams as 1:1 with local port allocations. This is still being discussed and a conclusion hasn't been reached, see below. + - Lifetime is <= lifetime of UDP mapping. + - stream ID for parent mapping, and new flow within that mapping + - ended on mapping end or by explicit `END` + - Includes peer address (ipv4/ipv6) -TODO: What does it mean for a stream to end here? This depends on how we choose to do port allocations, which needs more discussion. Should the stream lifetime and port allocation lifetime match, or should there be a separate timer system? +- `UDP_FLOW_DATAGRAM` -TODO: This is flawed, the "connected/unconnected" idea is just a UNIX'ism and we should be thinking about streams and about how we allocate source ports. Also, limiting connections to a single peer breaks 100% of P2P connections, since they require using STUN (or similar) first. + - Datagram contents without address, for flow streams. + - If a flow stream exists, the exit must use `UDP_FLOW_DATAGRAM` instead of `UDP_MAPPING_DATAGRAM`. + - Mappings that do not request port-specific filtering may always get unexpected UDP_MAPPING_DATAGRAMs. Mappings that do use port-specific filtering could make a flow for their only expected peers, then expect to never see `UDP_MAPPING_DATAGRAM`. -TODO: Right now beth and mike have different ideas for how this could work, and we need to discuss more. +## TURN over Tor -TODO: Revise this section. Nearly everything assumes the MTU is larger, and we are driving this design based on app compatibility. +In this approach, a single Tor stream is used for all UDP traffic. -TODO: "Application" here being the VPN or the end-user app? We've discussed having a per-app opt-in for UDP, enforced at the VPN layer. -TODO: update this +- `CONNECT_TURN` + - Establish a stream as a connection to the exit relay's built-in (or configured) TURN server. + - RFC8656 requires authentication before data can be relayed, which is a good default best practice for the internet perhaps but is the opposite of what Tor is trying to do. We would either modify the specification to relax this auth requirement, or we would provide a way for clients to discover credentials: either by fixing them ahead of time or by including them in the relay descriptor. + \ No newline at end of file -- cgit v1.2.3-54-g00ecf From 98d96e42ff300d6d0a4a0573a1f19bcb2afa904a Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 21 Dec 2023 16:45:30 -0800 Subject: Bit more writing --- proposals/XXX-udp-app-support.md | 119 +++++++++++++++++++++++++++------------ 1 file changed, 82 insertions(+), 37 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index ce30bda..350f91b 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -10,32 +10,37 @@ Status: Draft This proposal takes a fresh look at the problem of implementing support in Tor for applications which require UDP/IP communication. +This work is being done with the sponsorship and goals of the [Tor VPN Client for Android project](https://gitlab.torproject.org/groups/tpo/-/milestones/32). + We start out by defining how this proposal compares to previous work, and the specific problem space we are addressing. This leads into an analysis that references appropriate standards and proposes some specific solutions with properties we can compare. ## History -There have been multiple attempts within Tor to define some type of UDP support. - -Proposal 100 by Marc Liberatore in 2006 suggested a way to "add support for tunneling unreliable datagrams through tor with as few modifications to the protocol as possible." This proposal extends the existing protocol with a DTLS link mode alongside the existing TLS+TCP. The focus of this work was on a potential way to support unreliable traffic, not necessarily on UDP itself or on UDP applications. +There have been multiple attempts already within Tor to define some type of UDP support. -In proposal 100, a Tor "Stream" is used for one pairing of local and remote address and port, copying the technique used by Tor for TCP. This works for some types of UDP applications, as we detail below, but for many peer-to-peer use cases it's unhelpful to allocate local ports that can only be used with a single remote peer. +[Proposal 100](https://spec.torproject.org/proposals/100-tor-spec-udp.html) by Marc Liberatore in 2006 suggested a way to "add support for tunneling unreliable datagrams through tor with as few modifications to the protocol as possible." +This proposal suggested extending the existing TLS+TCP protocol with a new DTLS+UDP link mode. +The focus of this work was on a potential way to support unreliable traffic, not necessarily on UDP itself or on UDP applications. -Furthermore, no additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to a one Tor cell, a value that we will see is unusably small for most applications. +In proposal 100, a Tor *stream* is used for one pairing of local and remote address and port, copying the technique used by Tor for TCP. This works for some types of UDP applications, as we detail below, but for many peer-to-peer use cases it's unhelpful to allocate local ports that can only be used with a single remote peer. -It's possible these UDP protocol details would have been elaborated during design, but this proposal hit a snag elsewhere: there was no agreement on a way to avoid causing new attacks against anonymity. +Furthermore, no additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to a one Tor cell. This value we will see is unusably small for most applications. -In 2018, Nick Mathewson and Mike Perry wrote this summary of the general issues with unreliable transports for Tor: +It's possible these UDP protocol details would have been elaborated during design, but the proposal hit a snag elsewhere: +there was no agreement on a way to avoid facilitating new attacks against anonymity. -https://research.torproject.org/techreports/side-channel-analysis-2018-11-27.pdf +In 2018, Nick Mathewson and Mike Perry wrote a +[summary of the side-channel issues with unreliable transports for Tor](https://research.torproject.org/techreports/side-channel-analysis-2018-11-27.pdf). TODO: Look closer at this, some of the attacks described might overlap with non-datagram-transport ways of encapsulating UDP. (i.e. a malicious exit may always cause drops/injections regardless of what the rest of the anonymity network does.) -TODO: There has got to be a section here for new risks with malicious exits. Some protocols may see increased risk vs. their existing TCP fallbacks. +[Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor circuit using a new type of Tor message. +As a prerequisite, it depends on [proposal 319](https://spec.torproject.org/proposals/319-wide-everything.html) to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. -Proposal 339 by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor circuit using a new type of Tor message. As a prerequisite, it depends on proposal 319 to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. - -In proposal 339 the property of binding a stream both to a local port and to a remote peer is described in UNIX-style terminology as a "connected socket". We dive deeper into this idea below and supply an alternate formulation based on RFC4787, NAT behavior requirements for UDP. This single-peer behavior would be referred to as an "endpoint-dependent mapping" in RFC4787. It works fine for client/server apps but precludes the use of peer-to-peer techniques. +In proposal 339 the property of binding a stream both to a local port and to a remote peer is described in UNIX-style terminology as a *connected socket*. We dive deeper into this idea below and supply an alternate formulation based on [RFC4787](https://www.rfc-editor.org/rfc/rfc4787), *NAT behavior requirements for UDP*. +The single-peer *connected socket* behavior would be referred to as an *endpoint-dependent mapping* in RFC4787. +This type works fine for client/server apps but precludes the use of NAT traversal for peer-to-peer transfer. ## Scope @@ -43,21 +48,32 @@ This proposal aims to allow Tor applications and Tor-based VPNs to provide compa We don't have a specific list of applications that must be supported, but we are currently aiming for broad support of popular applications while still respecting and referencing all applicable Internet standards documents. -Changes to the structure of the Tor network are out of scope, as are specific performance optimizations. We expect to rely on common optimizations to the performance of Tor circuits, rather than looking to make specific changes to optimize for unreliable datagram transmission. +Changes to the structure of the Tor network are out of scope, as are most performance optimizations. We expect to rely on common optimizations to the performance of Tor circuits, rather than looking to make specific changes that optimize for unreliable datagram transmission. We will discuss the design implications of UDP on onion services below. It's worth planning for this as a way to evaluate the design space, but in practice we are not aiming for UDP to onion services yet. This will require changes to most applications that want to use it, as it implies that any media negotiations will need to understand onion addressing in addition to IPv4 and IPv6. We do not rigidly define the subset of UDP traffic that will be allowed. -There are several options discussed below using the framework of RFC4787 NAT recommendations. +There are several options discussed below using the RFC4787 framework. + +We require support for DNS clients. Tor currently only supports a limited subset of DNS queries, and it's desirable to support more. This will be analyzed in detail as an application below. DNS is one of very few applications that still rely on fragmented UDP datagrams, though this may not be relevant for us since only servers typically need to control the production of fragments. -We do expect support for voice/video telecommunications apps. Even without an underlying transport that supports unreliable datagrams, we expect a tunnel to provide a usable level of compatibility. This design space is very similar to the TURN (RFC8656) specification, already used very widely for compatibility with networks that filter UDP. See the analysis of specific applications below. +We require support for voice/video telecommunications apps. Even without an underlying transport that supports unreliable datagrams, we expect a tunnel to provide a usable level of compatibility. This design space is very similar to the TURN ([RFC8656](https://www.rfc-editor.org/rfc/rfc8656)) specification, already used very widely for compatibility with networks that filter UDP. See the analysis of specific applications below. -We do expect support for DNS clients. Tor currently only supports a limited subset of DNS queries, and it's desirable to support more. This will be analyzed in detail as an application below. DNS is one of very few applications that still rely on fragmented UDP datagrams, though this may not be relevant for us since only servers typically need to control the production of fragments. +We require support for peer-to-peer UDP transfer without additional relaying, in apps that use ICE ([RFC8445](https://www.rfc-editor.org/rfc/rfc8445)) or similar connection establishment techniques. +Video calls between two Tor users should transit directly between two exit nodes. +This requires that allocated UDP ports can each communicate with multiple peers: +*endpoint-independent mapping* as described by RFC4787. + +We do not plan to support applications which accept arbitrary incoming datagrams, for example a DNS server hosted via Tor. +RFC4787 calls this *endpoint-independent filtering*. +It's unnecessary for running peer-to-peer apps, and it facilitates an extremely easy traffic injection attack. # UDP Traffic Models +To better specify the role of a UDP extension for Tor, we will look at a few frameworks for describing UDP applications. + ## User Datagram Protocol (RFC768) The "User Interface" suggested by RFC768 for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. @@ -233,36 +249,26 @@ In alphabetical order: | WiFi Calling | Telecom | IPsec tunnel | Out of scope | Still out of scope | | Zoom | Telecom | client/server or P2P, UDP/TCP | Works | Slight latency improvement | -## Malicious traffic - -TODO: Various kinds of traffic we want to avoid - -- Amplification attacks against arbitrary target (protocol contains reply addr, like SIP) -- Amplification attacks against exit relay (similar to attacks against NAT) -- Malicious fragmented traffic -- Excessive sends to a host that has never replied (DoS) -- Excessive number of peers (makes port scanning too much easier) - -See also RFC 7675, on the concept of "Send consent". - -# Anonymity risks - -TODO: ICE connectivity checks, as mentioned elsewhere. - -TODO: Are there plaintext identifiers in these telecom apps? - -TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? - # Design approaches +TODO + ## Datagram routing +TODO + ### 3rd party implementations +TODO + ### Hybrid approaches +TODO: Idea about splitting UDP and TCP traffic. Applicable to clearnet (Tor/3rd-party split) and to onion (Exit/Rend split) + ### Making room for a future Tor implementation +TODO: Consider whether datagram message types could realistically carry over to a future Tor protocol that supports unrealiable delivery. Would we be building a foundation for real or is it not useful yet? + ## Tunneling ### Using TURN (RFC8656) encapsulated in a Tor stream @@ -385,4 +391,43 @@ In this approach, a single Tor stream is used for all UDP traffic. - Establish a stream as a connection to the exit relay's built-in (or configured) TURN server. - RFC8656 requires authentication before data can be relayed, which is a good default best practice for the internet perhaps but is the opposite of what Tor is trying to do. We would either modify the specification to relax this auth requirement, or we would provide a way for clients to discover credentials: either by fixing them ahead of time or by including them in the relay descriptor. - \ No newline at end of file + +# Risks + +## Performance and compatibility regressions + +TODO: By implementing UDP at all, we open ourselves up to bugs and performance regressions vs any existing TCP compatibility support present in applications. + +## Additional malicious traffic + +TODO: Various kinds of traffic we want to avoid + +- Amplification attacks against arbitrary target (protocol contains reply addr, like SIP) +- Amplification attacks against exit relay (similar to attacks against NAT) +- Malicious fragmented traffic +- Excessive sends to a host that has never replied (DoS) +- Excessive number of peers (makes port scanning too much easier) + +See also RFC 7675, on the concept of "Send consent". + +## Additional high-bandwidth application traffic + +TODO: Analyze bittorrent and friends + +## Additional exposure to malicious exit nodes + +TODO: There may be new cases where an exit node could modify or inspect UDP-tunneled traffic that would otherwise have been unavailable, e.g. by bypassing tor or by using a secure TURN-over-TLS tunnel. + +## Additional risks to anonymity + +TODO: ICE connectivity checks, as mentioned elsewhere. + +TODO: Are there plaintext identifiers in these telecom apps? + +TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? + +## Port allocation risks + +TODO: Without a way to limit port allocations, exits can quickly run out. + +TODO: Incorporate a mitigation in the form of Port Overlapping -- cgit v1.2.3-54-g00ecf From 827d46eea3c4ab447ea56d9bacf107854fed5c54 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 21 Dec 2023 17:13:46 -0800 Subject: Maybe that's all for today There are some really big TODOs that I need to crack open but maybe that's a job for after break. Happy new year! --- proposals/XXX-udp-app-support.md | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index 350f91b..bdea42b 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -76,8 +76,9 @@ To better specify the role of a UDP extension for Tor, we will look at a few fra ## User Datagram Protocol (RFC768) -The "User Interface" suggested by RFC768 for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. +The "User Interface" suggested by [RFC768](https://www.rfc-editor.org/rfc/rfc768) for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. Despite UDP's simplicity as an application of IP, we do need to be aware of IP features that are typically hidden by TCP's abstraction. + UDP applications typically try to obtain an awareness of the path MTU, using some type of path MTU discovery (PMTUD) algorithm. On IPv4, this requires sending packets with the "Don't Fragment" flag set, and measuring when those packets are lost or when ICMP "Fragmentation Needed" replies are seen. @@ -89,10 +90,11 @@ In practice the straightforward "User Interface" from RFC768, capable of arbitra BSD-style sockets support UDP via `SOCK_DGRAM`. UDP is a stateless protocol, but sockets do have state. -Each socket is bound, either explicitly with 'bind()' or automatically, to a source IP and port. -At the API level, a socket is said to be _connected_ to a remote address:port if that address is the default destination. -A _connected_ socket will also filter out incoming packets with source addresses different from this default destination. -A socket is considered _unconnected_ if 'connect()' has not been called. +Each socket is bound, either explicitly with `bind()` or automatically, to a source IP and port. + +At the API level, a socket is said to be *connected* to a remote address:port if that address is the default destination. +A *connected* socket will also filter out incoming packets with source addresses different from this default destination. +A socket is considered *unconnected* if `connect()` has not been called. These sockets have no default destination, and they accept datagrams from any source. There does not need to be any particular mapping between the lifetime of these application sockets and any higher-level "connection" the application establishes. @@ -208,6 +210,11 @@ QUIC does not normally try to traverse NAT; as an HTTP replacement, the server i TODO: Look closer at the NAT rebinding attack described in the RFC, check how applicable that is for us. +When QUIC is used as an optional upgrade path, we must compare any proposed UDP support against the baseline of a non-upgraded original connection. +In these cases we are not looking for any specific compatibility enhancement, simply an avoidance of regression. + +In cases where QUIC is used as a primary protocol without TCP fallback, we expect UDP support to be vital. These applications are currently niche but we expect they may rise in popularity. + ### WebRTC WebRTC is a large collection of protocols tuned to work together for media transport and NAT traversal. @@ -251,19 +258,24 @@ In alphabetical order: # Design approaches -TODO +Now that we've defined some categories of UDP traffic we are interested in handling, this section starts to examine different implementation techniques. + +We can broadly split these into *datagram routing* and *tunneling*. ## Datagram routing -TODO +These approaches seek to use an anonymity network that can directly route datagrams from place to place. +These approaches are the most obviously suitable for implementing UDP, but they also form the widest departure from classic Tor. ### 3rd party implementations -TODO +TODO: What examples do we have of 3rd party mixnets that already have these desirable datagram properties? i2p would be a good reference point here, but only for the mixnet portion itself not for any kind of exit bridge. ### Hybrid approaches -TODO: Idea about splitting UDP and TCP traffic. Applicable to clearnet (Tor/3rd-party split) and to onion (Exit/Rend split) +TODO: If we do have a good story for some potential datagram mixnet above, examine the possibility of splitting traffic between Tor and that network. + +TODO: Also discuss splitting traffic between exit and rend for onions? This is technically more likely to be useful for tunneling below, so maybe we don't go into it here. ### Making room for a future Tor implementation @@ -353,7 +365,7 @@ TODO: Exit selection is likely common to both of these approaches. Talk about ho ## Tor UDP Mapping Protocol -This is *tunneled* approach, using Tor's existing protocol objects where possible. Exits advertise support for specific UDP policies. Clients choose an exit where they would like to allocate a UDP local port. They create one *stream* with a lifetime matching the lifetime of that mapping. Optional additional *streams* may be used as fast paths for common peers. This is the *hybrid approach* to stream usage, from above. +This is a *tunneled* approach, using Tor's existing protocol objects where possible. Exits advertise support for specific UDP policies. Clients choose an exit where they would like to allocate a UDP local port. They create one *stream* with a lifetime matching the lifetime of that mapping. Optional additional *streams* may be used as fast paths for common peers. This is the *hybrid approach* to stream usage, from above. - `NEW_UDP_MAPPING` -- cgit v1.2.3-54-g00ecf From 5fed47868f7d88b3c9a078ba202b7a698ec8d5df Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Tue, 9 Jan 2024 19:22:55 -0800 Subject: Some more thoughts on possible attacks --- proposals/XXX-udp-app-support.md | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index bdea42b..b7dc616 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -35,7 +35,7 @@ In 2018, Nick Mathewson and Mike Perry wrote a TODO: Look closer at this, some of the attacks described might overlap with non-datagram-transport ways of encapsulating UDP. (i.e. a malicious exit may always cause drops/injections regardless of what the rest of the anonymity network does.) -[Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor circuit using a new type of Tor message. +[Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor stream using a new type of Tor message. As a prerequisite, it depends on [proposal 319](https://spec.torproject.org/proposals/319-wide-everything.html) to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. In proposal 339 the property of binding a stream both to a local port and to a remote peer is described in UNIX-style terminology as a *connected socket*. We dive deeper into this idea below and supply an alternate formulation based on [RFC4787](https://www.rfc-editor.org/rfc/rfc4787), *NAT behavior requirements for UDP*. @@ -77,6 +77,7 @@ To better specify the role of a UDP extension for Tor, we will look at a few fra ## User Datagram Protocol (RFC768) The "User Interface" suggested by [RFC768](https://www.rfc-editor.org/rfc/rfc768) for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. + Despite UDP's simplicity as an application of IP, we do need to be aware of IP features that are typically hidden by TCP's abstraction. UDP applications typically try to obtain an awareness of the path MTU, using some type of path MTU discovery (PMTUD) algorithm. @@ -438,6 +439,28 @@ TODO: Are there plaintext identifiers in these telecom apps? TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? +### Traffic injection attacks + +Some forms of UDP support would have obvious and severe traffic injection vulnerabilities. For example, the very permissive *endpoint-independent filtering* strategy would allow any host on the internet to send datagrams in bulk to all available local ports on a Tor exit in order to map that traffic's effect on any guards they control. + +Any vehicle for malicious parties to mark traffic can be abused to deanonymize users. Even if there is a more restrictive filtering policy, UDP's lack of sequence numbers make header spoofing attacks considerably easier than in TCP. Third parties capable of routing datagrams to an exit with a spoofed source address could bypass filtering when the communicating parties are known or can be guessed. For example, a malicious superuser at an ISP without egress filtering could send packets with the source IPs set to various common DNS servers and application STUN/TURN servers. If a specific application is being targeted, only the exit node and local port numbers need to be guessed by brute force. + +In case of malicious exit relays, whole datagrams can be inserted and dropped, and datagrams may be padded with additional data, both without any specific knowledge of the application protocol. With specific protocol insights, a malicious relay may make arbitrary edits to plaintext data. + +Of particular interest is the plaintext STUN, TURN, and ICE traffic used by most WebRTC apps. These applications rely on higher-level protocols (SRTP, DTLS) to provide end-to-end encryption and authentication. A compromise at the connection establishment layer would not violate application-level end-to-end security requirements, making it outside the threat model of WebRTC but very much still a concern for Tor. + +These attacks are not fully unique to the proposed UDP support, but UDP may increase exposure. In cases where the application already has a fallback using TURN-over-TLS, the proposal is a clear regression over previous behaviors. Even when we are comparing plaintext to plaintext, there may be a serious downside to centralizing all connection establishment traffic through a small number of exit IPs. Depending on your threat model, it could very well be more private to allow the UDP traffic to bypass Tor entirely. + +### Peer to peer deanonymization attacks + +One of our goals was to achieve the compatibility and perhaps performance benefits of allowing "peer to peer" (in our case really exit-to-exit) UDP connections. We expect this to enable the subset of applications that lack a fallback path which loops traffic through an app-provided server. + +This goal may be at odds with our privacy requirements. At minimum, a pool of malicious exit nodes could passively collect metadata about these connections as a noisy proxy for call metadata. + +TODO: Seems likely applications do often leak enough information through the plaintext portions of their UDP traffic in order to facilitate fingerprinting, I should look closer at this and confirm or deny. + +Even if the application traffic itself is fingerprint-resistant, this is easily combined with the above traffic injection attacks in order to mark specific communicating peers. + ## Port allocation risks TODO: Without a way to limit port allocations, exits can quickly run out. -- cgit v1.2.3-54-g00ecf From 301c694623a3264133bf5e6bf7fbde4783bdacc2 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Wed, 10 Jan 2024 09:53:30 -0800 Subject: Additional links and headings --- proposals/XXX-udp-app-support.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index b7dc616..386037f 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -25,7 +25,7 @@ The focus of this work was on a potential way to support unreliable traffic, not In proposal 100, a Tor *stream* is used for one pairing of local and remote address and port, copying the technique used by Tor for TCP. This works for some types of UDP applications, as we detail below, but for many peer-to-peer use cases it's unhelpful to allocate local ports that can only be used with a single remote peer. -Furthermore, no additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to a one Tor cell. This value we will see is unusably small for most applications. +Furthermore, no additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to what fits in a single Tor cell. This value we will see is unusably small for most applications. It's possible these UDP protocol details would have been elaborated during design, but the proposal hit a snag elsewhere: there was no agreement on a way to avoid facilitating new attacks against anonymity. @@ -108,9 +108,10 @@ Every datagram sent or received on the socket may have a different peer address. Much of the real-world complexity in UDP applications comes from their strategies to detect and overcome the effects of NAT. Many RFCs have been written on NAT behavior and NAT traversal strategies. -RFC4787 and later RFC7857 offer best practices for implementing NAT. -Carrier-grade NAT requirements are addressed by RFC6888. -RFC8445 describes the Interactive Connectivity Establishment (ICE) protocol, which has become a common and recommended application-level technique for building peer to peer applications that work through NAT. +[RFC4787](https://www.rfc-editor.org/rfc/rfc4787.html) and later [RFC7857](https://www.rfc-editor.org/rfc/rfc7857.html) offer best practices for implementing NAT. These are sometimes referred to as the [BEHAVE-WG](https://datatracker.ietf.org/wg/behave/about/) recommendations, based on the "Behavior Engineering for Hindrance Avoidance" working group behind them. +Carrier-grade NAT requirements are addressed by [RFC6888](https://www.rfc-editor.org/rfc/rfc6888.html). + +[RFC8445](https://www.rfc-editor.org/rfc/rfc8445.html) describes the Interactive Connectivity Establishment (ICE) protocol, which has become a common and recommended application-level technique for building peer-to-peer applications that work through NAT. There are multiple fundamental technical issues that NAT presents: @@ -135,6 +136,8 @@ For example, in analyzing Tor as a type of carrier-grade NAT, we may consider th Tor by necessity must carefully limit how predictable these mappings can ever be, to preserve its anonymity properties. A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well as the various per-subscriber limiting requirements. +### Mapping and Filtering Behaviors + RFC4787 defines a framework for understanding the behavior of NAT by analyzing both its "mapping" and "filtering" behavior separately. Mappings are the NAT's unit of state tracking. Filters are layered on top of mappings, potentially rejecting incoming datagrams that don't match an already-expected address. -- cgit v1.2.3-54-g00ecf From 759f97875b3789469d64f07aadf7b50e0dc32283 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Wed, 10 Jan 2024 10:52:22 -0800 Subject: Resolve TODO related to REQ-8 in RFC4787 I looked at this a bit more and I'm still not sure what specific circumstance the claim is alluding to. I replaced the TODO with a more permanent description of the claim as well as my skeptical take. --- proposals/XXX-udp-app-support.md | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index 386037f..54b0e41 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -32,8 +32,8 @@ there was no agreement on a way to avoid facilitating new attacks against anonym In 2018, Nick Mathewson and Mike Perry wrote a [summary of the side-channel issues with unreliable transports for Tor](https://research.torproject.org/techreports/side-channel-analysis-2018-11-27.pdf). - -TODO: Look closer at this, some of the attacks described might overlap with non-datagram-transport ways of encapsulating UDP. (i.e. a malicious exit may always cause drops/injections regardless of what the rest of the anonymity network does.) +The focus of this document is on the link between Tor relays, but there is considerable overlap between the attack space explored here and the potential risks of any application-level UDP support. +Attacks that are described here, such as drops and injections, may be applied by malicious exits or some types of third parties even in an implementation using only present-day reliable Tor transports. [Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor stream using a new type of Tor message. As a prerequisite, it depends on [proposal 319](https://spec.torproject.org/proposals/319-wide-everything.html) to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. @@ -165,21 +165,13 @@ We can gain some additional insight by looking at requirements that come from ou This is a permitted alternative according to RFC4787, in which incoming datagrams are allowed from only IP addresses we have previously sent to, but any port on that IP may be the sender. - The intended benefits of this approach would be: + The intended benefits of this approach versus the port-dependent filtering below are unclear, and may no longer be relevant. In theory they would be: - To support a class of applications that rely on, for a single local port, multiple remote ports achieving filter acceptance status when only one of those ports has been sent a datagram. We are currently lacking examples of applications in this category. - Any application using ICE will be outside this category, as each port would have its own connectivity check datagrams exchanged in each direction. - - - To support peers using firewalls that are not compliant with RFC4787. This claim from the spec: - - ``` - However, if NAT-A uses Endpoint-Independent Filtering - or Address-Dependent Filtering, ICE will ultimately find - connectivity without requiring a UDP relay. - ``` + Any application using ICE should be outside this category, as each port would have its own connectivity check datagrams exchanged in each direction. - TODO: Is this actually true? I haven't worked out how port-independent filtering is what makes this succeed. + - REQ-8 in RFC4787 claims the existence of a scenario in which this approach facilitates ICE connections with a remote peer that disregards REQ-1 (the peer does not use **Endpoint-Independent Mapping**). It is not clear that this claim is still relevant. One security hazard of address-dependent and non-port-dependent filtering, identified in RFC4787, is that a peer on a NAT effectively negates the security benefits of this host filtering. In fact, this should raise additional red flags as applied to either Tor or carrier grade NAT. -- cgit v1.2.3-54-g00ecf From 7a7177dee689a87c0b3506f323c586511d206351 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Wed, 10 Jan 2024 12:44:25 -0800 Subject: A few more RFC refs, answer the QUIC TODO --- proposals/XXX-udp-app-support.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index 54b0e41..b9bda00 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -200,11 +200,13 @@ We will analyze how these protocols affect Tor's UDP traffic requirements. ### QUIC -RFC9000 defines QUIC, a multiplexed secure point-to-point protocol which supports reliable and unreliable delivery. The most common use is as an optional HTTP replacement, especially among Google services. +[RFC9000](https://www.rfc-editor.org/rfc/rfc9000.html) defines QUIC, a multiplexed secure point-to-point protocol which supports reliable and unreliable delivery. The most common use is as an optional HTTP replacement, especially among Google services. QUIC does not normally try to traverse NAT; as an HTTP replacement, the server is expected to have a routable address. -TODO: Look closer at the NAT rebinding attack described in the RFC, check how applicable that is for us. +QUIC provides its own flexible connection lifetimes which may outlive individual network links or NAT mappings. +The intention is to provide transparent roaming as mobile users change networks. +This automated path discovery opens additional opportunities for malicious traffic, for which the RFC also offers mitigations. See *path validation* in section `8.2`, and the additional mitigations from section `9.3`. When QUIC is used as an optional upgrade path, we must compare any proposed UDP support against the baseline of a non-upgraded original connection. In these cases we are not looking for any specific compatibility enhancement, simply an avoidance of regression. @@ -217,13 +219,16 @@ WebRTC is a large collection of protocols tuned to work together for media trans It is increasingly common, both for browser-based telephony and for peer to peer data transfer. Non-browser-based apps often implement WebRTC or have components in common with WebRTC. -See RFC7734, RFC8825, RFC8445, RFC5389, others. +Of particular importance to us, WebRTC uses the Interactive Connection Establishment (ICE) protocol to establish a bidirectional channel between endpoints that may or may not be behind a NAT with unknown configuration. -TODO: Organize and expand references here. +Any generalized solution to connection establishment, like ICE, will require sending connectivity test probes. These have an inherent hazard to anonymity: assuming no delays are inserted intentionally, the result is a broadcast of similar traffic across all available network interfaces. This could form a convenient correlation beacon for an attacker attempting to deanonymize users who use WebRTC over a Tor VPN. -Of particular importance to us, WebRTC uses the Interactive Connection Establishment (ICE, RFC8445, RFC8838) protocol to establish a bidirectional channel between endpoints that may or may not be behind a NAT with unknown configuration. - -TODO: More about the specifics of how ICE effects us. More about privacy concerns. +See +[RFC8825](https://www.rfc-editor.org/rfc/rfc8825.html) _Overview: Real-Time Protocols for Browser-Based Applications_, +[RFC8445](https://www.rfc-editor.org/rfc/rfc8445.html) _Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal_, +[RFC8838](https://www.rfc-editor.org/rfc/rfc8838.html) _Trickle ICE: Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (ICE) Protocol_, +[RFC5389](https://www.rfc-editor.org/rfc/rfc5389.html) _Session Traversal Utilities for NAT (STUN)_, +and others. ## Common Applications -- cgit v1.2.3-54-g00ecf From 25feebf9d0a0de3e36fcf3331e41e0f9e9d16afa Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Wed, 10 Jan 2024 18:22:22 -0800 Subject: Continued work on structure, formatting, TODOs --- proposals/XXX-udp-app-support.md | 350 ++++++++++++++++++++++++++------------- 1 file changed, 238 insertions(+), 112 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index b9bda00..1acc2ef 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -17,24 +17,34 @@ This leads into an analysis that references appropriate standards and proposes s ## History -There have been multiple attempts already within Tor to define some type of UDP support. +There have already been multiple attempts over Tor's history to define some type of UDP extension. -[Proposal 100](https://spec.torproject.org/proposals/100-tor-spec-udp.html) by Marc Liberatore in 2006 suggested a way to "add support for tunneling unreliable datagrams through tor with as few modifications to the protocol as possible." +### 2006 + +[Proposal 100](https://spec.torproject.org/proposals/100-tor-spec-udp.html) by Marc Liberatore in 2006 suggested a way to "add support for tunneling unreliable datagrams through tor with as few modifications to the protocol as possible." This proposal suggested extending the existing TLS+TCP protocol with a new DTLS+UDP link mode. The focus of this work was on a potential way to support unreliable traffic, not necessarily on UDP itself or on UDP applications. -In proposal 100, a Tor *stream* is used for one pairing of local and remote address and port, copying the technique used by Tor for TCP. This works for some types of UDP applications, as we detail below, but for many peer-to-peer use cases it's unhelpful to allocate local ports that can only be used with a single remote peer. +In proposal 100, a Tor *stream* is used for one pairing of local and remote address and port, copying the technique used by Tor for TCP. +This works for some types of UDP applications, but it's broken by common behaviors like ICE connectivity checks, NAT traversal attempts, or using multiple servers via the same socket. +We go into more detail about application behavior below. -Furthermore, no additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to what fits in a single Tor cell. This value we will see is unusably small for most applications. +No additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to what fits in a single Tor cell. +This value we will see is much too small for most applications. It's possible these UDP protocol details would have been elaborated during design, but the proposal hit a snag elsewhere: there was no agreement on a way to avoid facilitating new attacks against anonymity. -In 2018, Nick Mathewson and Mike Perry wrote a +### 2018 + +In 2018, Nick Mathewson and Mike Perry wrote a [summary of the side-channel issues with unreliable transports for Tor](https://research.torproject.org/techreports/side-channel-analysis-2018-11-27.pdf). -The focus of this document is on the link between Tor relays, but there is considerable overlap between the attack space explored here and the potential risks of any application-level UDP support. + +The focus of this document is on the communication between Tor relays, but there is considerable overlap between the attack space explored here and the potential risks of any application-level UDP support. Attacks that are described here, such as drops and injections, may be applied by malicious exits or some types of third parties even in an implementation using only present-day reliable Tor transports. +### 2020 + [Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor stream using a new type of Tor message. As a prerequisite, it depends on [proposal 319](https://spec.torproject.org/proposals/319-wide-everything.html) to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. @@ -93,7 +103,7 @@ BSD-style sockets support UDP via `SOCK_DGRAM`. UDP is a stateless protocol, but sockets do have state. Each socket is bound, either explicitly with `bind()` or automatically, to a source IP and port. -At the API level, a socket is said to be *connected* to a remote address:port if that address is the default destination. +At the API level, a socket is said to be *connected* to a remote `(address, port)` if that address is the default destination. A *connected* socket will also filter out incoming packets with source addresses different from this default destination. A socket is considered *unconnected* if `connect()` has not been called. These sockets have no default destination, and they accept datagrams from any source. @@ -148,15 +158,15 @@ RFC4787 defines three types with different properties, and does not make one sin We can gain some additional insight by looking at requirements that come from outside RFC4787. - **Endpoint-Independent Filtering** allows incoming datagrams from any peer once a mapping has been established. - + RFC4787 recommends this approach, with the concession that it may not be ideal for all security requirements. - + In the context of Tor, we can likely rule out this technique entirely. It makes traffic injection attacks possible from any source address, provided you can guess the UDP port number used at an exit. It also makes possible clearnet hosting of UDP servers using an exit node's IP, which may have undesirable abuse properties. - + It precludes "Port overlapping" behavior as defined in RFC7857 section 3, which may be necessary in order to achieve sufficient utilization of local port numbers on exit nodes. - + It is still common for present-day applications to *prefer* endpoint-independent filtering, as it allows incoming connections from peers which cannot use STUN or a similar address fixing protocol. Choosing endpoint-independent filtering would have *some* compatibility benefit, but among modern protocols which use ICE and STUN there would be no improvement. The cost, on the other hand, would be an uncomfortably low-cost traffic injection attack and additional risks toward exit nodes. @@ -166,7 +176,7 @@ We can gain some additional insight by looking at requirements that come from ou This is a permitted alternative according to RFC4787, in which incoming datagrams are allowed from only IP addresses we have previously sent to, but any port on that IP may be the sender. The intended benefits of this approach versus the port-dependent filtering below are unclear, and may no longer be relevant. In theory they would be: - + - To support a class of applications that rely on, for a single local port, multiple remote ports achieving filter acceptance status when only one of those ports has been sent a datagram. We are currently lacking examples of applications in this category. Any application using ICE should be outside this category, as each port would have its own connectivity check datagrams exchanged in each direction. @@ -257,179 +267,301 @@ In alphabetical order: | WiFi Calling | Telecom | IPsec tunnel | Out of scope | Still out of scope | | Zoom | Telecom | client/server or P2P, UDP/TCP | Works | Slight latency improvement | -# Design approaches +# High level approaches -Now that we've defined some categories of UDP traffic we are interested in handling, this section starts to examine different implementation techniques. +Now that we've defined some categories of UDP traffic we are interested in handling, this section starts to examine different high-level implementation techniques we could adopt. We can broadly split these into *datagram routing* and *tunneling*. +Ideally we would be choosing a design that solves problems we have in the near-term while also providing a solid foundation for future enhancements to Tor, including changes which may add full support for unreliable delivery of datagrams. If we proceed down that path with insufficient understanding of the long-term goal, there's a risk that we will choose to adopt complexity in service of future goals while failing to serve them adequately when the time comes. + ## Datagram routing -These approaches seek to use an anonymity network that can directly route datagrams from place to place. -These approaches are the most obviously suitable for implementing UDP, but they also form the widest departure from classic Tor. +These approaches seek to use a network that can directly route datagrams from place to place. These approaches are the most obviously suitable for implementing UDP, but they also form the widest departure from classic Tor. + +### Intentional UDP leak + +The simplest approach would be to allow UDP traffic to bypass the anonymity layer. This is an unacceptable loss of anonymity in many cases, given that the client's real IP address is made visible to web application providers. + +In other cases, this is an acceptable or even preferable approach. For example, VPN users may be more concerned with achieving censorship-resistant network connectivity than hiding personal identifiers from application vendors. + +In threat models where application vendors are more trustworthy than the least trustworthy Tor exits, it may be more appropriate to allow direct peer-to-peer connections than to trust Tor exits with unencrypted connection establishment traffic. ### 3rd party implementations -TODO: What examples do we have of 3rd party mixnets that already have these desirable datagram properties? i2p would be a good reference point here, but only for the mixnet portion itself not for any kind of exit bridge. +Another option would be to use an unrelated anonymizer system for datagram traffic. It's not clear that a suitable system already exists. I2P provides a technical solution for routing anonymized datagrams, but not a Tor-style infrastructure of exit node operators. -### Hybrid approaches +This points to the key weakness of relying on a separate network for UDP: Tor has an especially well-developed community of volunteers running relays. Any UDP solution that is inconvenient for relay operators has little chance of adoption. -TODO: If we do have a good story for some potential datagram mixnet above, examine the possibility of splitting traffic between Tor and that network. +### Future proofing -TODO: Also discuss splitting traffic between exit and rend for onions? This is technically more likely to be useful for tunneling below, so maybe we don't go into it here. +This is likely where we would seek to expand Tor's design in order to add end-to-end support for unreliable delivery in the future. +A specific design is out of the scope of this document. -### Making room for a future Tor implementation +It is worth thinking early about how we can facilitate combinations of approaches. +We may find a need for an abstraction similar to a network routing table, allowing multiple UDP providers to coexist. -TODO: Consider whether datagram message types could realistically carry over to a future Tor protocol that supports unrealiable delivery. Would we be building a foundation for real or is it not useful yet? +Even without bringing any new network configurations to Tor, achieving interoperable support for both exit nodes and onion services in a Tor UDP implementation requires some attention to how multiple UDP providers can coexist. ## Tunneling -### Using TURN (RFC8656) encapsulated in a Tor stream +The approaches in this section add a new construct which does not exist in UDP itself: a point to point tunnel between clients and some other location at which they establish the capability to send and receive UDP datagrams. + +Any tunneling approach requires some way to discover tunnel endpoints. +We would like this to come as an extension of Tor's existing process for distributing consensus and representing exit policy. + +We expect exit policies for UDP to have limited practical amounts of diversity. +VPN implementations will need to know ahead of time which tunnel circuits to build, or they will suffer a significant spike in latency for the first outgoing datagram to a new peer. +Additionally, it's common for UDP port numbers to be randomly assigned. +This makes highly specific exit policies even less useful and even higher overhead than they are with TCP. + +### Using TURN encapsulated in a Tor stream + +The scope of this tunnel is quite similar to the existing TURN relays, used commonly by WebRTC applications to implement fallbacks for clients who cannot find a more direct connection path. + +TURN is defined by [RFC8656](https://www.rfc-editor.org/rfc/rfc8656) as a set of extensions built on the framework from STUN in [RFC8489](https://www.rfc-editor.org/rfc/rfc8489.html). The capabilities are a good match for our needs, offering clients the ability to encapsulate UDP datagrams within a TCP stream, and to allocate local port mappings on the server. -TODO: Comparison vs. an entirely out-of-protocol and potentially out-of-process TURN server. Is this complexity warranted? +TURN was designed to be a set of modular and extensible pieces, which may be too far opposed to Tor's design philosophy of providing single canonical representations. Any adoption of TURN will need to consider the potential for malicious implementations to mark traffic, facilitating deanonymization attacks. + +TURN has a popular embeddable C-language implementation, [coturn](https://github.com/coturn/coturn), which may be suitable for including alongside or inside C tor. ### Using Tor streams to an exit -TODO: Bulk of the design goes here, this is what we likely start with +Most of the discussion on UDP implementation in Tor so far has assumed this approach. Essentially it's the same strategy as TCP exits, but for UDP. When the OP initializes support for UDP, it pre-builds circuits to exits that support required UDP exit policies. These pre-built circuits can then be used as tunnels for UDP datagrams. + +Within this overall approach, there are various ways we could choose to assign Tor *streams* for the UDP traffic. This will be considered below. ### Using Tor streams to a rendezvous point -TODO: possible generalization of above to support onion UDP. big caveat that apps need extensions to support onion addressing. +To implement onion services which advertise UDP, we may consider using multiple simultaneous tunnels. +In addition to exit nodes, clients could establish the ability to allocate virtual UDP ports on a rendezvous node of some kind. + +The most immediate challenge in UDP rendezvous would then become application support. Protocols like STUN and ICE deal directly with IPv4 and IPv6 formats in order to advertise a routable address to their peer. Supporting onion services in WebRTC would require protocol extensions and software modifications for STUN, TURN, ICE, and SDP at minimum. + +UDP-like rendezvous extensions would have limited meaning unless they form part of a long-term strategy to forward datagrams in some new way for enhanced performance or compatibility. Otherwise, application authors might as well stick with Tor's existing TCP-like rendezvous functionality. + +# Specific designs using Tor streams + +Let's look more closely at Tor *streams*, the multiplexing layer right below circuits. + +Streams have a 16-bit identifier, allocated arbitrarily by clients. Stream lifetimes are subject to some ambiguity still in the Tor spec. They are allocated by clients, but may be destroyed by either peer. + +## Stream per tunnel + +The fewest new streams would be a single stream for all of UDP. This is what we get if we choose an off-the-shelf protocol like TURN as our UDP proxy. + +This approach would require only a single new Tor message type: + +- `CONNECT_TURN` + + - Establish a stream as a connection to the exit relay's built-in (or configured) TURN server. + +Note that RFC8656 requires authentication before data can be relayed, which is a good default best practice for the internet perhaps but is the opposite of what Tor is trying to do. We would either deviate from the specification to relax this auth requirement, or we would provide a way for clients to discover credentials: perhaps by fixing them ahead of time or by including them in the relay descriptor. + +## Stream per socket + +One stream **per socket** was the approach suggested in [Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020. + +In proposal 339, there would be one new type of stream and three new message types: `CONNECT_UDP`, `CONNECTED_UDP`, and `DATAGRAM`. + +Each stream's lifetime would match the lifetime of a source port allocation. +There would be a single peer `(remote address, remote port)` allowed per `local port` allocation. +This matches the usage of BSD-style sockets on which `connect()` has completed. +It's incompatible with many of the applications analyzed. +Multiple peers are typically needed for a variety of reasons, like connectivity checks or multi-region servers. -#### Stream usage +This approach would be simplest to implement and specify, especially in the existing C tor implementation. -An early design juncture in this project is the particular choice of scope for one *stream* in the existing Tor protocol. +It also unfortunately has very limited compatibility, and no clear path toward incremental upgrades if we wish to improve compatibility later. -- One stream **per socket** was the approach suggested in Proposal 339. +## Stream per flow - Each stream would match the lifetime of a source port allocation. - There would be a single peer address/port allowed per allocation. - This matches the usage of BSD-style sockets on which `connect()` has completed. - It's incompatible with many of the applications analyzed. - Multiple peers are typically needed for a variety of reasons, like connectivity checks or multi-region servers. +One stream **per flow** has also been suggested. +In particular, Mike Perry brought up this approach during our conversations about UDP earlier and we spent some time analyzing it. - This approach would be simplest to implement and specify. - It also unfortunately has very limited compatibility, and no clear path toward incremental upgrades if we wish to improve compatibility later. - -- One stream **per flow** has also been suggested. This would assign a stream ID to the combination of an allocated source port and a remote peer address/port. The flows may contain additional flags, like transmit and receive filtering, IPv4/v6, and *Don't Fragment*. +This would assign a stream ID to the tuple consisting of at least `(local port, remote address, remote port)`. Additional flags may be included for features like transmit and receive filtering, IPv4/v6 choice, and IP *Don't Fragment*. - This has advantages in keeping the datagram cells simple, with no additional IDs beyond the existing circuit ID. - It may also have advantages in DoS-prevention and in privacy analysis. - - By decoupling protocol features from the lifetime of sockets on the exit side, we facilitate implementing the desirable "Port overlapping" NAT behavior as mentioned above. +This has advantages in keeping the datagram cells simple, with no additional IDs beyond the existing circuit ID. +It may also have advantages in DoS-prevention and in privacy analysis. - Stream lifetimes, in this case, would not have any specific meaning other than the lifetime of the ID itself. - The bundle of flows associated with one source port would still all be limited to the lifetime of a Tor circuit, by scoping the source port identifier to be contained within the lifetime of its circuit. +By decoupling protocol features from the lifetime of sockets on the exit side, we facilitate implementing the desirable "Port overlapping" NAT behavior mentioned above. - It would be necessary to allocate a new stream ID any time a new set of parameters (source port, remote address, remote port) is seen. - This would most commonly happen as a result of a first datagram sent to a new peer, coinciding with the establishment of a NAT-style mapping and the possible allocation of a socket on the exit. - A less common case needs to be considered too: what if the parameter tuple first occurs on the exit side? - We don't yet have a way to allocate stream IDs from either end of a circuit. - This would need to be considered, and the simplest solution might just be to partition the stream ID space into a half that can be allocated by each side. - This leaves a quite unfair split in the common case where streams are almost always allocated by the OP side. +Stream lifetimes, in this case, would not have any specific meaning other than the lifetime of the ID itself. +The bundle of flows associated with one source port would still all be limited to the lifetime of a Tor circuit, by scoping the source port identifier to be contained within the lifetime of its circuit. - When is this exit-originated circuit ID allocation potentially needed? - It is clearly needed when using **address-dependent filtering**. - An incoming datagram from a previously-unseen peer port is expected to be deliverable, and the exit would need to allocate an ID for it. +It would be necessary to allocate a new stream ID any time a new `(local port, remote address, remote port)` tuple is seen. +This would most commonly happen as a result of a first datagram sent to a new peer, coinciding with the establishment of a NAT-style mapping and the possible allocation of a socket on the exit. - Even with the stricter **address and port-dependent filtering** we may still be exposed to exit-originated circuit IDs if there are mismatches in the lifetime of the filter and the stream. +A less common case needs to be considered too: what if the parameter tuple first occurs on the exit side? +We don't yet have a way to allocate stream IDs from either end of a circuit. +This would need to be considered. +One simple solution would be to statically partition the stream ID space into a portion that can be independently allocated by each side. - This approach thus requires some attention to either correctly allocating stream IDs on both sides of the circuit, or choosing a filtering strategy and filter/mapping lifetime that does not ever leave stream IDs undefined when expecting incoming datagrams. +When is this exit-originated circuit ID allocation potentially needed? +It is clearly needed when using **address-dependent filtering**. +An incoming datagram from a previously-unseen peer port is expected to be deliverable, and the exit would need to allocate an ID for it. -- One stream **per mapping** is an alternative which attemps to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. +Even with the stricter **address and port-dependent filtering** we may still be exposed to exit-originated circuit IDs if there are mismatches in the lifetime of the filter and the stream. - A mapping would always be allocated from the OP side. - It could explicitly specify a filtering style, if we wish to allow applications to request non-port-dependent filtering for compatibility. - Each datagram within the stream would still need to be tagged with a peer address/port in some way. +This approach thus requires some attention to either correctly allocating stream IDs on both sides of the circuit, or choosing a filtering strategy and filter/mapping lifetime that does not ever leave stream IDs undefined when expecting incoming datagrams. - This puts us in a very similar design space to TURN, RFC8656. - In TURN, "allocations" are made explicitly on request, and assigned a random relayed port. - TURN also uses its "allocations" as an opportunity to support a *Don't Fragment* flag. - - Within the scope of one allocation, datagrams each still may have an arbitrary peer address. This takes up additional space in every datagram. +## Stream per mapping -- Hybrid approach, one stream **per mapping** and **per flow** at the same time +One stream **per mapping** is an alternative which attemps to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. - We could try to combine the strengths of both approaches, using the lifetime of one stream to define a mapping and to carry otherwise-unbundled traffic while also allowing additional streams to bundle datagrams that would otherwise have repetetive headers. +A mapping would always be allocated from the OP side. +It could explicitly specify a filtering style, if we wish to allow applications to request non-port-dependent filtering for compatibility. +Each datagram within the stream would still need to be tagged with a peer address/port in some way. - This avoids the space overhead of a purely **per mapping** approach and avoids the ID allocation and lifetime complexity introduced with **per flow**. - - This takes more inspiration from TURN, where commonly used peers will be defined as a "channel" with an especially short header. - In TURN, a "channel" is only ever allocated from the originating side of the connection. - Incoming datagrams with no channel can always be represented in the long form, so TURN never has to allocate channels unexpectedly. +This approach would involve a single new type of stream, two new messages that pertain to these *flow* streams: -# Protocol Designs +- `NEW_UDP_MAPPING` + + - Always client-to-exit + - Creates a new mapping, with a specified stream ID + - Succeeds instantly; no reply is expected, early data is ok. + - Externally-visible local port number is arbitrary, and must be determined through interaction with other endpoints + - Might contain an IP "don't fragment" flag + - Might contain a requested filtering mode + - Lifetime is until circuit teardown or `END` message + +- `UDP_MAPPING_DATAGRAM` + + - conveys one datagram on a stream defined by `NEW_UDP_MAPPING`. + - Includes peer address (IPv4/IPv6) as well as datagram content + +This puts us in a very similar design space to TURN, RFC8656. +In TURN, "allocations" are made explicitly on request, and assigned a random relayed port. +TURN also uses its "allocations" as an opportunity to support a *Don't Fragment* flag. + +The principal disadvantage of this approach is in space overhead, especially the proportional overhead on small datagrams which must still carry a full-size address. + +## Hybrid stream appoach + +We can extend the approach above with an optimization that addresses the undesiarable space overhead from redundant address headers. + +This uses two new types of stream, in order to have streams **per mapping** and **per flow** at the same time. -Currently we have two design candidates, and both are described here. +The per-mapping stream remains the sole interface for managing the lifetime of a mapped UDP port. Mappings are created explicitly by the client. As an optimization, within the lifetime of a mapping there may exist some number of *flows*, each assigned their own ID. -TODO: Exit selection is likely common to both of these approaches. Talk about how we specify UDP exit policy, how we provide a smart tradeoff between ability to filter harmful traffic while avoiding an explosion in exit policy complexity. +This tries to combine the strengths of both approaches, using the lifetime of one stream to define a mapping and to carry otherwise-unbundled traffic while also allowing additional streams to bundle datagrams that would otherwise have repetetive headers. +It avoids the space overhead of a purely **per mapping** approach and avoids the ID allocation and lifetime complexity introduced with **per flow**. -## Tor UDP Mapping Protocol +This approach takes some inspiration from TURN, where commonly used peers will be defined as a "channel" with an especially short header. +Incoming datagrams with no channel can always be represented in the long form, so TURN never has to allocate channels unexpectedly. -This is a *tunneled* approach, using Tor's existing protocol objects where possible. Exits advertise support for specific UDP policies. Clients choose an exit where they would like to allocate a UDP local port. They create one *stream* with a lifetime matching the lifetime of that mapping. Optional additional *streams* may be used as fast paths for common peers. This is the *hybrid approach* to stream usage, from above. +The implementation here could be a strict superset of the **per mapping** implementation, adding new commands for flows while retaining existing behavior for mappings. There would be a total of four new message types: - `NEW_UDP_MAPPING` - - always client-to-exit - - stream ID for mapping - - no address given - - "don't fragment" flag - - port-specific filtering flag - - no reply? early data always ok. - - ended on circuit teardown or by `END` + - Same as above - `UDP_MAPPING_DATAGRAM` - - conveys one datagram on a stream defined by ALLOCATE_UDP_MAPPING. - - Includes peer address (ipv4/ipv6) as well as datagram content + - Same as above - `NEW_UDP_FLOW` - Lifetime is <= lifetime of UDP mapping. - stream ID for parent mapping, and new flow within that mapping - ended on mapping end or by explicit `END` - - Includes peer address (ipv4/ipv6) + - Includes peer address (IPv4/IPv6) - `UDP_FLOW_DATAGRAM` - Datagram contents without address, for flow streams. - - If a flow stream exists, the exit must use `UDP_FLOW_DATAGRAM` instead of `UDP_MAPPING_DATAGRAM`. - - Mappings that do not request port-specific filtering may always get unexpected UDP_MAPPING_DATAGRAMs. Mappings that do use port-specific filtering could make a flow for their only expected peers, then expect to never see `UDP_MAPPING_DATAGRAM`. -## TURN over Tor +We must consider the traffic marking opportunities we open when allowing an exit to represent one incoming datagram as either a *flow* or *mapping* datagram. -In this approach, a single Tor stream is used for all UDP traffic. +It's possible this traffic injection potential is not worse than the baseline amount of injection potential than every UDP protocol presents. See more on risks below. For this hybrid stream approach specifically, there's a limited mitigation we can use to allow exits only a bounded amount of leaked information per UDP peer: -- `CONNECT_TURN` +We would like to state that exits may not choose to send a `UDP_MAPPING_DATAGRAM` when they could have sent a `UDP_FLOW_DATAGRAM`. +Sometimes it is genuinely unclear though: an exit may have received this datagram in-between processing `NEW_UDP_MAPPING` and `NEW_UDP_FLOW`. +We could choose to terminate circuits which send a `UDP_MAPPING_DATAGRAM` for a peer that has already been referenced in a `UDP_FLOW_DATAGRAM`, giving exits a one-way gate to let them switch a peer from *mapping* datagram to *flow* datagram but not the reverse. - - Establish a stream as a connection to the exit relay's built-in (or configured) TURN server. - - RFC8656 requires authentication before data can be relayed, which is a good default best practice for the internet perhaps but is the opposite of what Tor is trying to do. We would either modify the specification to relax this auth requirement, or we would provide a way for clients to discover credentials: either by fixing them ahead of time or by including them in the relay descriptor. +Mappings that do not request port-specific filtering may always get unexpected `UDP_MAPPING_DATAGRAM`s. Mappings that do use port-specific filtering could make a flow for their only expected peers, then expect to never see `UDP_MAPPING_DATAGRAM`. + +We may wish for `NEW_UDP_MAPPING` to have an option requiring that only `UDP_FLOW_DATAGRAM` is to be used, never `UDP_MAPPING_DATAGRAM`. +This would remove the potential for ambiguity, but costs in compatibility as it's no longer possible to implement non-port-specific filtering. # Risks -## Performance and compatibility regressions +Any proposed UDP support involves significant risks to user privacy and software maintainability. +We will try to elaborate some of these risks here, so they can be compared against the expected benefits. + +## Behavior regressions + +In some applications it is possible that Tor's implementation of a UDP compatibility layer will cause a regression in the ultimate level of performance or security. + +Performance regressions can occur accidentally due to bugs or compatibility glitches. +They may also occur for more fundamental reasons of protocol layering. +For example, the redundant error correction layers when tunneling QUIC over TCP. +These performance degradations are expected to be minor, but there's some unavoidable risk. + +We may mitigate the risk of severe performance or compatibility regressions by giving users a way to toggle UDP support per-application. + +Privacy and security regressions have more severe consequences and they can be much harder to detect. +There are straightforward downgrades, like WebRTC apps that give up TURN-over-TLS for plaintext TURN-over-UDP. +More subtly, the act of centralizing connection establishment traffic in Tor exit nodes can make users an easier target for other attacks. + +## Bandwidth usage -TODO: By implementing UDP at all, we open ourselves up to bugs and performance regressions vs any existing TCP compatibility support present in applications. +We expect an increase in overall exit bandwidth requirements due to peer-to-peer file sharing applications. -## Additional malicious traffic +Current users attempting to use BitTorrent over Tor are hampered by the lack of UDP compatibility. Interoperability with common file-sharing peers would make Tor more appealing to users with a large and sustained appetite for anonymized bandwidth. -TODO: Various kinds of traffic we want to avoid +## Malicious traffic + +We expect UDP compatibility in Tor will give malicious actors additional opportunities to transmit unwanted traffic. + +- Amplification attacks against arbitrary targets + + These are possible only in limited circumstances where the protocol allows an arbitrary reply address, like SIP. + The peer is often at fault for having an overly permissive configuration. + Nevertheless, any of these *easy* amplification targets can be exploited from Tor with little consequence, creating a nuisance for the ultimate target and for exit operators. + +- Amplification attacks against exit relay + + An amplification peer which doesn't allow arbitrary destinations can still be used to attack the exit relay itself or other users of that relay. + This is essentially the same attack that is possible against any NAT the attacker is behind. -- Amplification attacks against arbitrary target (protocol contains reply addr, like SIP) -- Amplification attacks against exit relay (similar to attacks against NAT) - Malicious fragmented traffic -- Excessive sends to a host that has never replied (DoS) -- Excessive number of peers (makes port scanning too much easier) -See also RFC 7675, on the concept of "Send consent". + If we allow sending large UDP datagrams over IPv4 without the *Don't Fragment* flag set, we allow attackers to generate fragmented IP datagrams. + This is not itself a problem, but it has historically been a common source of inconsistencies in firewall behavior. + +- Excessive sends to an uninterested peer + + Whereas TCP mandates a successful handshake, UDP will happily send unlimited amounts of traffic to a peer that has never responded. + To prevent denial of service attacks we have an opportunity and perhaps a responsibility to define our supported subset of UDP to include true bidirectional traffic but exclude continued sends to peers who do not respond. + + See also [RFC7675](https://www.rfc-editor.org/rfc/rfc7675.html) and STUN's concept of "Send consent". + +- Excessive number of peers + + We may want to place conservative limits on the maximum number of peers per mapping or per circuit, in order to make bulk scanning of UDP port space less convenient. -## Additional high-bandwidth application traffic + The limit does need to be on peers, not stream IDs as we presently do for TCP. -TODO: Analyze bittorrent and friends + In this proposal stream IDs are not necessarily meaningful except as a representational choice made by clients. + Strategies like the *per-mapping* stream assignment like we have for TCP. -## Additional exposure to malicious exit nodes +## Local port usage -TODO: There may be new cases where an exit node could modify or inspect UDP-tunneled traffic that would otherwise have been unavailable, e.g. by bypassing tor or by using a secure TURN-over-TLS tunnel. +Exit routers will have a limited number of local UDP ports. In the most constrained scenario, an exit may have a single IP with 16384 or fewer ephemeral ports available. These ports could each be allocated by one client for an unbounded amount of exclusive use. + +In order to enforce high levels of isolation between different subsequent users of the same local UDP port, we may wish to enforce a delay between allocations during which nobody may own the port. Effective isolation requires this timer to be greater than any timer we expect to encounter on a peer or a NAT. In RFC4787's recommendations a NAT's mapping timer must be longer than 2 minutes. Our timer should ideally be *much* longer than 2 minutes. + +An attacker who allocates ports for only this minimum duration of 2 minutes would need to send 136.5 requests per second to achieve sustained use of all available ports. With multiple simultaneous clients this could easily be done while bypassing per-circuit rate limiting. + +The expanded definition of "Port overlapping" from [RFC7857 section 3](https://datatracker.ietf.org/doc/html/rfc7857#section-3), may form at least a partial mitigation: + + This document clarifies that this port overlapping behavior may be extended to connections originating from different internal source IP addresses and ports as long as their destinations are different. + +This gives us an opportunity for a vast reduction in the number of required ports and file descriptors. Practically, though, it does require us to make a guess about which potential peers one source port may communicate with. + +Our UDP implementation will need to choose a port assignment based on knowledge of only the first peer the app is sending to. +Heuristically, we can make this work. The first peer in practice will be less unique than subsequent peers. Applications will contact centralized services before contacting peers. This ordering is necessary in the general case of ICE-like connection establishment. ## Additional risks to anonymity @@ -460,9 +592,3 @@ This goal may be at odds with our privacy requirements. At minimum, a pool of ma TODO: Seems likely applications do often leak enough information through the plaintext portions of their UDP traffic in order to facilitate fingerprinting, I should look closer at this and confirm or deny. Even if the application traffic itself is fingerprint-resistant, this is easily combined with the above traffic injection attacks in order to mark specific communicating peers. - -## Port allocation risks - -TODO: Without a way to limit port allocations, exits can quickly run out. - -TODO: Incorporate a mitigation in the form of Port Overlapping -- cgit v1.2.3-54-g00ecf From 679138c337affa080ce1ebb47da43e8fdc69d3ce Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 11 Jan 2024 09:13:52 -0800 Subject: Table of contents, spell check, additional fixes --- proposals/XXX-udp-app-support.md | 253 +++++++++++++++++++++++---------------- 1 file changed, 147 insertions(+), 106 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index 1acc2ef..c338f17 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -1,12 +1,52 @@ -``` +# UDP Application Support in Tor + +```text Filename: XXX-udp-app-support.md -Title: Support for UDP Applications over Tor +Title: UDP Application Support in Tor Author: Micah Elizabeth Scott Created: December 2023 Status: Draft ``` -# Introduction +## Table of Contents + +- [Introduction](#introduction) + - [History](#history) + - [Scope](#scope) +- [UDP Traffic Models](#udp-traffic-models) + - [User Datagram Protocol (RFC768)](#user-datagram-protocol-rfc768) + - [Socket Layer](#socket-layer) + - [Network Address Translation (NAT)](#network-address-translation-nat) + - [Mapping and Filtering Behaviors](#mapping-and-filtering-behaviors) + - [Common Protocols](#common-protocols) + - [QUIC](#quic) + - [WebRTC](#webrtc) + - [Common Applications](#common-applications) +- [Overview of Possible Solutions](#overview-of-possible-solutions) + - [Datagram Routing](#datagram-routing) + - [Intentional UDP Leak](#intentional-udp-leak) + - [3rd Party Implementations](#3rd-party-implementations) + - [Future Work on Tor](#future-work-on-tor) + - [Tunneling](#tunneling) + - [TURN Encapsulated in a Tor Stream](#turn-encapsulated-in-a-tor-stream) + - [Tor Stream Tunnel to an Exit](#tor-stream-tunnel-to-an-exit) + - [Tor Stream Tunnel to a Rendezvous Point](#tor-stream-tunnel-to-a-rendezvous-point) +- [Specific Designs Using Tor Streams](#specific-designs-using-tor-streams) + - [One Stream per Tunnel](#one-stream-per-tunnel) + - [One Stream per Socket](#one-stream-per-socket) + - [One Stream per Flow](#one-stream-per-flow) + - [One Stream per Mapping](#one-stream-per-socket) + - [Hybrid Mapping and Flow Approach](#hybrid-mapping-and-flow-approach) +- [Risks](#risks) + - [Behavior Regressions](#behavior-regressions) + - [Bandwidth Usage](#bandwidth-usage) + - [Malicious Traffic](#malicious-traffic) + - [Local Port Usage](#local-port-usage) + - [Traffic Injection](#traffic-injection) + - [Peer-to-Peer Deanonymization](#peer-to-peer-deanonymization) + - [Additional Risks to Anonymity](#additional-risks-to-anonymity) + +## Introduction This proposal takes a fresh look at the problem of implementing support in Tor for applications which require UDP/IP communication. @@ -15,11 +55,11 @@ This work is being done with the sponsorship and goals of the [Tor VPN Client fo We start out by defining how this proposal compares to previous work, and the specific problem space we are addressing. This leads into an analysis that references appropriate standards and proposes some specific solutions with properties we can compare. -## History +### History There have already been multiple attempts over Tor's history to define some type of UDP extension. -### 2006 +#### 2006 [Proposal 100](https://spec.torproject.org/proposals/100-tor-spec-udp.html) by Marc Liberatore in 2006 suggested a way to "add support for tunneling unreliable datagrams through tor with as few modifications to the protocol as possible." This proposal suggested extending the existing TLS+TCP protocol with a new DTLS+UDP link mode. @@ -35,7 +75,7 @@ This value we will see is much too small for most applications. It's possible these UDP protocol details would have been elaborated during design, but the proposal hit a snag elsewhere: there was no agreement on a way to avoid facilitating new attacks against anonymity. -### 2018 +#### 2018 In 2018, Nick Mathewson and Mike Perry wrote a [summary of the side-channel issues with unreliable transports for Tor](https://research.torproject.org/techreports/side-channel-analysis-2018-11-27.pdf). @@ -43,7 +83,7 @@ In 2018, Nick Mathewson and Mike Perry wrote a The focus of this document is on the communication between Tor relays, but there is considerable overlap between the attack space explored here and the potential risks of any application-level UDP support. Attacks that are described here, such as drops and injections, may be applied by malicious exits or some types of third parties even in an implementation using only present-day reliable Tor transports. -### 2020 +#### 2020 [Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor stream using a new type of Tor message. As a prerequisite, it depends on [proposal 319](https://spec.torproject.org/proposals/319-wide-everything.html) to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. @@ -52,7 +92,7 @@ In proposal 339 the property of binding a stream both to a local port and to a r The single-peer *connected socket* behavior would be referred to as an *endpoint-dependent mapping* in RFC4787. This type works fine for client/server apps but precludes the use of NAT traversal for peer-to-peer transfer. -## Scope +### Scope This proposal aims to allow Tor applications and Tor-based VPNs to provide compatibility with applications that require UDP/IP communications. @@ -80,11 +120,11 @@ We do not plan to support applications which accept arbitrary incoming datagrams RFC4787 calls this *endpoint-independent filtering*. It's unnecessary for running peer-to-peer apps, and it facilitates an extremely easy traffic injection attack. -# UDP Traffic Models +## UDP Traffic Models To better specify the role of a UDP extension for Tor, we will look at a few frameworks for describing UDP applications. -## User Datagram Protocol (RFC768) +### User Datagram Protocol (RFC768) The "User Interface" suggested by [RFC768](https://www.rfc-editor.org/rfc/rfc768) for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. @@ -95,7 +135,7 @@ On IPv4, this requires sending packets with the "Don't Fragment" flag set, and m Note that many applications have their own requirements for path MTU. For example, QUIC and common implementations of WebRTC require an MTU no smaller than 1200 bytes, but they can discover larger MTUs when available. -## Socket Layer +### Socket Layer In practice the straightforward "User Interface" from RFC768, capable of arbitrary local address, is only available to privileged users. @@ -113,7 +153,7 @@ It's better to think of one socket as one allocated source port. A typical application may allocate only a single port (one socket) for talking to many peers. Every datagram sent or received on the socket may have a different peer address. -## Network Address Translation (NAT) +### Network Address Translation (NAT) Much of the real-world complexity in UDP applications comes from their strategies to detect and overcome the effects of NAT. @@ -146,7 +186,7 @@ For example, in analyzing Tor as a type of carrier-grade NAT, we may consider th Tor by necessity must carefully limit how predictable these mappings can ever be, to preserve its anonymity properties. A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well as the various per-subscriber limiting requirements. -### Mapping and Filtering Behaviors +#### Mapping and Filtering Behaviors RFC4787 defines a framework for understanding the behavior of NAT by analyzing both its "mapping" and "filtering" behavior separately. Mappings are the NAT's unit of state tracking. @@ -163,7 +203,7 @@ We can gain some additional insight by looking at requirements that come from ou In the context of Tor, we can likely rule out this technique entirely. It makes traffic injection attacks possible from any source address, provided you can guess the UDP port number used at an exit. - It also makes possible clearnet hosting of UDP servers using an exit node's IP, which may have undesirable abuse properties. + It also makes possible clear-net hosting of UDP servers using an exit node's IP, which may have undesirable abuse properties. It precludes "Port overlapping" behavior as defined in RFC7857 section 3, which may be necessary in order to achieve sufficient utilization of local port numbers on exit nodes. @@ -202,17 +242,17 @@ We can gain some additional insight by looking at requirements that come from ou RFC4787 recommends that filtering style be configurable. We would like to implement that advice, but we are also looking for opportunities to make design decisions that give us the best network and end-user behaviors. -## Common Protocols +### Common Protocols Applications that want to use UDP are increasingly making use of higher-level protocols to avoid creating bespoke solutions for problems like NAT traversal, connection establishment, and reliable delivery. We will analyze how these protocols affect Tor's UDP traffic requirements. -### QUIC +#### QUIC [RFC9000](https://www.rfc-editor.org/rfc/rfc9000.html) defines QUIC, a multiplexed secure point-to-point protocol which supports reliable and unreliable delivery. The most common use is as an optional HTTP replacement, especially among Google services. -QUIC does not normally try to traverse NAT; as an HTTP replacement, the server is expected to have a routable address. +QUIC does not normally try to traverse NAT; as an HTTP replacement, the server is expected to have an address reachable without any prior connection setup. QUIC provides its own flexible connection lifetimes which may outlive individual network links or NAT mappings. The intention is to provide transparent roaming as mobile users change networks. @@ -223,7 +263,7 @@ In these cases we are not looking for any specific compatibility enhancement, si In cases where QUIC is used as a primary protocol without TCP fallback, we expect UDP support to be vital. These applications are currently niche but we expect they may rise in popularity. -### WebRTC +#### WebRTC WebRTC is a large collection of protocols tuned to work together for media transport and NAT traversal. It is increasingly common, both for browser-based telephony and for peer to peer data transfer. @@ -234,13 +274,13 @@ Of particular importance to us, WebRTC uses the Interactive Connection Establish Any generalized solution to connection establishment, like ICE, will require sending connectivity test probes. These have an inherent hazard to anonymity: assuming no delays are inserted intentionally, the result is a broadcast of similar traffic across all available network interfaces. This could form a convenient correlation beacon for an attacker attempting to deanonymize users who use WebRTC over a Tor VPN. See -[RFC8825](https://www.rfc-editor.org/rfc/rfc8825.html) _Overview: Real-Time Protocols for Browser-Based Applications_, -[RFC8445](https://www.rfc-editor.org/rfc/rfc8445.html) _Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal_, -[RFC8838](https://www.rfc-editor.org/rfc/rfc8838.html) _Trickle ICE: Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (ICE) Protocol_, -[RFC5389](https://www.rfc-editor.org/rfc/rfc5389.html) _Session Traversal Utilities for NAT (STUN)_, +[RFC8825](https://www.rfc-editor.org/rfc/rfc8825.html) *Overview: Real-Time Protocols for Browser-Based Applications*, +[RFC8445](https://www.rfc-editor.org/rfc/rfc8445.html) *Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal*, +[RFC8838](https://www.rfc-editor.org/rfc/rfc8838.html) *Trickle ICE: Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (ICE) Protocol*, +[RFC5389](https://www.rfc-editor.org/rfc/rfc5389.html) *Session Traversal Utilities for NAT (STUN)*, and others. -## Common Applications +### Common Applications With applications exhibiting such a wide variety of behaviors, how do we know what to expect from a good implementation? How do we know which compatibility decisions will be most important to users? @@ -267,7 +307,7 @@ In alphabetical order: | WiFi Calling | Telecom | IPsec tunnel | Out of scope | Still out of scope | | Zoom | Telecom | client/server or P2P, UDP/TCP | Works | Slight latency improvement | -# High level approaches +## Overview of Possible Solutions Now that we've defined some categories of UDP traffic we are interested in handling, this section starts to examine different high-level implementation techniques we could adopt. @@ -275,11 +315,11 @@ We can broadly split these into *datagram routing* and *tunneling*. Ideally we would be choosing a design that solves problems we have in the near-term while also providing a solid foundation for future enhancements to Tor, including changes which may add full support for unreliable delivery of datagrams. If we proceed down that path with insufficient understanding of the long-term goal, there's a risk that we will choose to adopt complexity in service of future goals while failing to serve them adequately when the time comes. -## Datagram routing +### Datagram Routing These approaches seek to use a network that can directly route datagrams from place to place. These approaches are the most obviously suitable for implementing UDP, but they also form the widest departure from classic Tor. -### Intentional UDP leak +#### Intentional UDP Leak The simplest approach would be to allow UDP traffic to bypass the anonymity layer. This is an unacceptable loss of anonymity in many cases, given that the client's real IP address is made visible to web application providers. @@ -287,13 +327,13 @@ In other cases, this is an acceptable or even preferable approach. For example, In threat models where application vendors are more trustworthy than the least trustworthy Tor exits, it may be more appropriate to allow direct peer-to-peer connections than to trust Tor exits with unencrypted connection establishment traffic. -### 3rd party implementations +#### 3rd Party Implementations Another option would be to use an unrelated anonymizer system for datagram traffic. It's not clear that a suitable system already exists. I2P provides a technical solution for routing anonymized datagrams, but not a Tor-style infrastructure of exit node operators. This points to the key weakness of relying on a separate network for UDP: Tor has an especially well-developed community of volunteers running relays. Any UDP solution that is inconvenient for relay operators has little chance of adoption. -### Future proofing +#### Future Work on Tor This is likely where we would seek to expand Tor's design in order to add end-to-end support for unreliable delivery in the future. A specific design is out of the scope of this document. @@ -303,7 +343,7 @@ We may find a need for an abstraction similar to a network routing table, allowi Even without bringing any new network configurations to Tor, achieving interoperable support for both exit nodes and onion services in a Tor UDP implementation requires some attention to how multiple UDP providers can coexist. -## Tunneling +### Tunneling The approaches in this section add a new construct which does not exist in UDP itself: a point to point tunnel between clients and some other location at which they establish the capability to send and receive UDP datagrams. @@ -313,9 +353,9 @@ We would like this to come as an extension of Tor's existing process for distrib We expect exit policies for UDP to have limited practical amounts of diversity. VPN implementations will need to know ahead of time which tunnel circuits to build, or they will suffer a significant spike in latency for the first outgoing datagram to a new peer. Additionally, it's common for UDP port numbers to be randomly assigned. -This makes highly specific exit policies even less useful and even higher overhead than they are with TCP. +This would make highly specific Tor exit policies even less useful and even higher overhead than they are with TCP. -### Using TURN encapsulated in a Tor stream +#### TURN Encapsulated in a Tor Stream The scope of this tunnel is quite similar to the existing TURN relays, used commonly by WebRTC applications to implement fallbacks for clients who cannot find a more direct connection path. @@ -325,28 +365,30 @@ TURN was designed to be a set of modular and extensible pieces, which may be too TURN has a popular embeddable C-language implementation, [coturn](https://github.com/coturn/coturn), which may be suitable for including alongside or inside C tor. -### Using Tor streams to an exit +#### Tor Stream Tunnel to an Exit Most of the discussion on UDP implementation in Tor so far has assumed this approach. Essentially it's the same strategy as TCP exits, but for UDP. When the OP initializes support for UDP, it pre-builds circuits to exits that support required UDP exit policies. These pre-built circuits can then be used as tunnels for UDP datagrams. Within this overall approach, there are various ways we could choose to assign Tor *streams* for the UDP traffic. This will be considered below. -### Using Tor streams to a rendezvous point +#### Tor Stream Tunnel to a Rendezvous Point To implement onion services which advertise UDP, we may consider using multiple simultaneous tunnels. In addition to exit nodes, clients could establish the ability to allocate virtual UDP ports on a rendezvous node of some kind. -The most immediate challenge in UDP rendezvous would then become application support. Protocols like STUN and ICE deal directly with IPv4 and IPv6 formats in order to advertise a routable address to their peer. Supporting onion services in WebRTC would require protocol extensions and software modifications for STUN, TURN, ICE, and SDP at minimum. +The most immediate challenge in UDP rendezvous would then become application support. Protocols like STUN and ICE deal directly with IPv4 and IPv6 formats in order to advertise a reachable address to their peer. Supporting onion services in WebRTC would require protocol extensions and software modifications for STUN, TURN, ICE, and SDP at minimum. UDP-like rendezvous extensions would have limited meaning unless they form part of a long-term strategy to forward datagrams in some new way for enhanced performance or compatibility. Otherwise, application authors might as well stick with Tor's existing TCP-like rendezvous functionality. -# Specific designs using Tor streams +## Specific Designs Using Tor Streams Let's look more closely at Tor *streams*, the multiplexing layer right below circuits. Streams have a 16-bit identifier, allocated arbitrarily by clients. Stream lifetimes are subject to some ambiguity still in the Tor spec. They are allocated by clients, but may be destroyed by either peer. -## Stream per tunnel +We have an opportunity to use this additional existing multiplexing layer to serve a useful function in the new protocol, or we can opt to interact with streams as little as possible in order to keep the protocol features more orthogonal. + +### One Stream per Tunnel The fewest new streams would be a single stream for all of UDP. This is what we get if we choose an off-the-shelf protocol like TURN as our UDP proxy. @@ -358,34 +400,34 @@ This approach would require only a single new Tor message type: Note that RFC8656 requires authentication before data can be relayed, which is a good default best practice for the internet perhaps but is the opposite of what Tor is trying to do. We would either deviate from the specification to relax this auth requirement, or we would provide a way for clients to discover credentials: perhaps by fixing them ahead of time or by including them in the relay descriptor. -## Stream per socket +### One Stream per Socket One stream **per socket** was the approach suggested in [Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020. In proposal 339, there would be one new type of stream and three new message types: `CONNECT_UDP`, `CONNECTED_UDP`, and `DATAGRAM`. -Each stream's lifetime would match the lifetime of a source port allocation. -There would be a single peer `(remote address, remote port)` allowed per `local port` allocation. +Each stream's lifetime would match the lifetime of a local port allocation. +There would be a single peer `(remote address, remote port)` allowed per `local port`. This matches the usage of BSD-style sockets on which `connect()` has completed. It's incompatible with many of the applications analyzed. Multiple peers are typically needed for a variety of reasons, like connectivity checks or multi-region servers. This approach would be simplest to implement and specify, especially in the existing C tor implementation. - It also unfortunately has very limited compatibility, and no clear path toward incremental upgrades if we wish to improve compatibility later. -## Stream per flow +A simple one-to-one mapping between streams and sockets would preclude the optimizations necessary to address [local port usage](#local-port-usage) risks below. Solutions under this design are possible, but only by decoupling logical protocol-level sockets from the ultimate implementation-level sockets and reintroducing much of the complexity that we attempted to avoid by choosing this design. + +### One Stream per Flow One stream **per flow** has also been suggested. -In particular, Mike Perry brought up this approach during our conversations about UDP earlier and we spent some time analyzing it. +Specifically, Mike Perry brought this up during our conversations about UDP recently and we spent some time analyzing it from a RFC4787 perspective. +We will see below it has interesting properties but also some hidden complexity. This would assign a stream ID to the tuple consisting of at least `(local port, remote address, remote port)`. Additional flags may be included for features like transmit and receive filtering, IPv4/v6 choice, and IP *Don't Fragment*. This has advantages in keeping the datagram cells simple, with no additional IDs beyond the existing circuit ID. It may also have advantages in DoS-prevention and in privacy analysis. -By decoupling protocol features from the lifetime of sockets on the exit side, we facilitate implementing the desirable "Port overlapping" NAT behavior mentioned above. - Stream lifetimes, in this case, would not have any specific meaning other than the lifetime of the ID itself. The bundle of flows associated with one source port would still all be limited to the lifetime of a Tor circuit, by scoping the source port identifier to be contained within the lifetime of its circuit. @@ -405,46 +447,45 @@ Even with the stricter **address and port-dependent filtering** we may still be This approach thus requires some attention to either correctly allocating stream IDs on both sides of the circuit, or choosing a filtering strategy and filter/mapping lifetime that does not ever leave stream IDs undefined when expecting incoming datagrams. -## Stream per mapping +### One Stream per Mapping -One stream **per mapping** is an alternative which attemps to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. +One stream **per mapping** is an alternative which attempts to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. A mapping would always be allocated from the OP side. It could explicitly specify a filtering style, if we wish to allow applications to request non-port-dependent filtering for compatibility. Each datagram within the stream would still need to be tagged with a peer address/port in some way. -This approach would involve a single new type of stream, two new messages that pertain to these *flow* streams: +This approach would involve a single new type of stream, two new messages that pertain to these *mapping* streams: - `NEW_UDP_MAPPING` - - Always client-to-exit - - Creates a new mapping, with a specified stream ID + - Always client-to-exit. + - Creates a new mapping, with a specified stream ID. - Succeeds instantly; no reply is expected, early data is ok. - - Externally-visible local port number is arbitrary, and must be determined through interaction with other endpoints - - Might contain an IP "don't fragment" flag - - Might contain a requested filtering mode - - Lifetime is until circuit teardown or `END` message + - Externally-visible local port number is arbitrary, and must be determined through interaction with other endpoints. + - Might contain an IP "don't fragment" flag. + - Might contain a requested filtering mode. + - Lifetime is until circuit teardown or `END` message. - `UDP_MAPPING_DATAGRAM` - - conveys one datagram on a stream defined by `NEW_UDP_MAPPING`. - - Includes peer address (IPv4/IPv6) as well as datagram content + - Conveys one datagram on a stream previously defined by `NEW_UDP_MAPPING`. + - Includes peer address (IPv4/IPv6) as well as datagram content. -This puts us in a very similar design space to TURN, RFC8656. -In TURN, "allocations" are made explicitly on request, and assigned a random relayed port. -TURN also uses its "allocations" as an opportunity to support a *Don't Fragment* flag. +This puts us in a very similar design space to TURN (RFC8656). +In that protocol, "allocations" are made explicitly on request, and assigned a random relayed port. +TURN also uses its allocations as an opportunity to support a *Don't Fragment* flag. -The principal disadvantage of this approach is in space overhead, especially the proportional overhead on small datagrams which must still carry a full-size address. +The principal disadvantage of this approach is in space overhead, especially the proportional overhead on small datagrams which must each carry a full-size address. -## Hybrid stream appoach - -We can extend the approach above with an optimization that addresses the undesiarable space overhead from redundant address headers. +### Hybrid Mapping and Flow Approach +We can extend the approach above with an optimization that addresses the undesirable space overhead from redundant address headers. This uses two new types of stream, in order to have streams **per mapping** and **per flow** at the same time. The per-mapping stream remains the sole interface for managing the lifetime of a mapped UDP port. Mappings are created explicitly by the client. As an optimization, within the lifetime of a mapping there may exist some number of *flows*, each assigned their own ID. -This tries to combine the strengths of both approaches, using the lifetime of one stream to define a mapping and to carry otherwise-unbundled traffic while also allowing additional streams to bundle datagrams that would otherwise have repetetive headers. +This tries to combine the strengths of both approaches, using the lifetime of one stream to define a mapping and to carry otherwise-unbundled traffic while also allowing additional streams to bundle datagrams that would otherwise have repetitive headers. It avoids the space overhead of a purely **per mapping** approach and avoids the ID allocation and lifetime complexity introduced with **per flow**. This approach takes some inspiration from TURN, where commonly used peers will be defined as a "channel" with an especially short header. @@ -454,26 +495,26 @@ The implementation here could be a strict superset of the **per mapping** implem - `NEW_UDP_MAPPING` - - Same as above + - Same as above. - `UDP_MAPPING_DATAGRAM` - - Same as above + - Same as above. - `NEW_UDP_FLOW` - - Lifetime is <= lifetime of UDP mapping. - - stream ID for parent mapping, and new flow within that mapping - - ended on mapping end or by explicit `END` - - Includes peer address (IPv4/IPv6) + - Allocates a stream ID as a *flow*, given the ID to be allocated and the ID of its parent *mapping* stream. + - Includes a peer address (IPv4/IPv6). + - The *flow* has a lifetime strictly bounded by the outer *mapping*. It is deleted by an explicit `END` or when the mapping is de-allocated for any reason. - `UDP_FLOW_DATAGRAM` - - Datagram contents without address, for flow streams. + - Datagram contents only, without address. + - Only appears on *flow* streams. We must consider the traffic marking opportunities we open when allowing an exit to represent one incoming datagram as either a *flow* or *mapping* datagram. -It's possible this traffic injection potential is not worse than the baseline amount of injection potential than every UDP protocol presents. See more on risks below. For this hybrid stream approach specifically, there's a limited mitigation we can use to allow exits only a bounded amount of leaked information per UDP peer: +It's possible this traffic injection potential is not worse than the baseline amount of injection potential than every UDP protocol presents. See more on [risks](#risks) below. For this hybrid stream approach specifically, there's a limited mitigation we can use to allow exits only a bounded amount of leaked information per UDP peer: We would like to state that exits may not choose to send a `UDP_MAPPING_DATAGRAM` when they could have sent a `UDP_FLOW_DATAGRAM`. Sometimes it is genuinely unclear though: an exit may have received this datagram in-between processing `NEW_UDP_MAPPING` and `NEW_UDP_FLOW`. @@ -484,12 +525,12 @@ Mappings that do not request port-specific filtering may always get unexpected ` We may wish for `NEW_UDP_MAPPING` to have an option requiring that only `UDP_FLOW_DATAGRAM` is to be used, never `UDP_MAPPING_DATAGRAM`. This would remove the potential for ambiguity, but costs in compatibility as it's no longer possible to implement non-port-specific filtering. -# Risks +## Risks Any proposed UDP support involves significant risks to user privacy and software maintainability. We will try to elaborate some of these risks here, so they can be compared against the expected benefits. -## Behavior regressions +### Behavior Regressions In some applications it is possible that Tor's implementation of a UDP compatibility layer will cause a regression in the ultimate level of performance or security. @@ -504,49 +545,49 @@ Privacy and security regressions have more severe consequences and they can be m There are straightforward downgrades, like WebRTC apps that give up TURN-over-TLS for plaintext TURN-over-UDP. More subtly, the act of centralizing connection establishment traffic in Tor exit nodes can make users an easier target for other attacks. -## Bandwidth usage +### Bandwidth Usage We expect an increase in overall exit bandwidth requirements due to peer-to-peer file sharing applications. Current users attempting to use BitTorrent over Tor are hampered by the lack of UDP compatibility. Interoperability with common file-sharing peers would make Tor more appealing to users with a large and sustained appetite for anonymized bandwidth. -## Malicious traffic +### Malicious Traffic We expect UDP compatibility in Tor will give malicious actors additional opportunities to transmit unwanted traffic. -- Amplification attacks against arbitrary targets +#### Amplification attacks against arbitrary targets - These are possible only in limited circumstances where the protocol allows an arbitrary reply address, like SIP. - The peer is often at fault for having an overly permissive configuration. - Nevertheless, any of these *easy* amplification targets can be exploited from Tor with little consequence, creating a nuisance for the ultimate target and for exit operators. +These are possible only in limited circumstances where the protocol allows an arbitrary reply address, like SIP. +The peer is often at fault for having an overly permissive configuration. +Nevertheless, any of these *easy* amplification targets can be exploited from Tor with little consequence, creating a nuisance for the ultimate target and for exit operators. -- Amplification attacks against exit relay +#### Amplification attacks against exit relay - An amplification peer which doesn't allow arbitrary destinations can still be used to attack the exit relay itself or other users of that relay. - This is essentially the same attack that is possible against any NAT the attacker is behind. +An amplification peer which doesn't allow arbitrary destinations can still be used to attack the exit relay itself or other users of that relay. +This is essentially the same attack that is possible against any NAT the attacker is behind. -- Malicious fragmented traffic +#### Malicious fragmented traffic - If we allow sending large UDP datagrams over IPv4 without the *Don't Fragment* flag set, we allow attackers to generate fragmented IP datagrams. - This is not itself a problem, but it has historically been a common source of inconsistencies in firewall behavior. +If we allow sending large UDP datagrams over IPv4 without the *Don't Fragment* flag set, we allow attackers to generate fragmented IP datagrams. +This is not itself a problem, but it has historically been a common source of inconsistencies in firewall behavior. -- Excessive sends to an uninterested peer +#### Excessive sends to an uninterested peer - Whereas TCP mandates a successful handshake, UDP will happily send unlimited amounts of traffic to a peer that has never responded. - To prevent denial of service attacks we have an opportunity and perhaps a responsibility to define our supported subset of UDP to include true bidirectional traffic but exclude continued sends to peers who do not respond. +Whereas TCP mandates a successful handshake, UDP will happily send unlimited amounts of traffic to a peer that has never responded. +To prevent denial of service attacks we have an opportunity and perhaps a responsibility to define our supported subset of UDP to include true bidirectional traffic but exclude continued sends to peers who do not respond. - See also [RFC7675](https://www.rfc-editor.org/rfc/rfc7675.html) and STUN's concept of "Send consent". +See also [RFC7675](https://www.rfc-editor.org/rfc/rfc7675.html) and STUN's concept of "Send consent". -- Excessive number of peers +#### Excessive number of peers - We may want to place conservative limits on the maximum number of peers per mapping or per circuit, in order to make bulk scanning of UDP port space less convenient. +We may want to place conservative limits on the maximum number of peers per mapping or per circuit, in order to make bulk scanning of UDP port space less convenient. - The limit does need to be on peers, not stream IDs as we presently do for TCP. +The limit does need to be on peers, not stream IDs as we presently do for TCP. - In this proposal stream IDs are not necessarily meaningful except as a representational choice made by clients. - Strategies like the *per-mapping* stream assignment like we have for TCP. +In this proposal stream IDs are not necessarily meaningful except as a representational choice made by clients. +Strategies like the *per-mapping* stream assignment like we have for TCP. -## Local port usage +### Local Port Usage Exit routers will have a limited number of local UDP ports. In the most constrained scenario, an exit may have a single IP with 16384 or fewer ephemeral ports available. These ports could each be allocated by one client for an unbounded amount of exclusive use. @@ -556,22 +597,14 @@ An attacker who allocates ports for only this minimum duration of 2 minutes woul The expanded definition of "Port overlapping" from [RFC7857 section 3](https://datatracker.ietf.org/doc/html/rfc7857#section-3), may form at least a partial mitigation: - This document clarifies that this port overlapping behavior may be extended to connections originating from different internal source IP addresses and ports as long as their destinations are different. +> This document clarifies that this port overlapping behavior may be extended to connections originating from different internal source IP addresses and ports as long as their destinations are different. This gives us an opportunity for a vast reduction in the number of required ports and file descriptors. Practically, though, it does require us to make a guess about which potential peers one source port may communicate with. Our UDP implementation will need to choose a port assignment based on knowledge of only the first peer the app is sending to. Heuristically, we can make this work. The first peer in practice will be less unique than subsequent peers. Applications will contact centralized services before contacting peers. This ordering is necessary in the general case of ICE-like connection establishment. -## Additional risks to anonymity - -TODO: ICE connectivity checks, as mentioned elsewhere. - -TODO: Are there plaintext identifiers in these telecom apps? - -TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? - -### Traffic injection attacks +#### Traffic Injection Some forms of UDP support would have obvious and severe traffic injection vulnerabilities. For example, the very permissive *endpoint-independent filtering* strategy would allow any host on the internet to send datagrams in bulk to all available local ports on a Tor exit in order to map that traffic's effect on any guards they control. @@ -583,7 +616,7 @@ Of particular interest is the plaintext STUN, TURN, and ICE traffic used by most These attacks are not fully unique to the proposed UDP support, but UDP may increase exposure. In cases where the application already has a fallback using TURN-over-TLS, the proposal is a clear regression over previous behaviors. Even when we are comparing plaintext to plaintext, there may be a serious downside to centralizing all connection establishment traffic through a small number of exit IPs. Depending on your threat model, it could very well be more private to allow the UDP traffic to bypass Tor entirely. -### Peer to peer deanonymization attacks +#### Peer-to-Peer Deanonymization One of our goals was to achieve the compatibility and perhaps performance benefits of allowing "peer to peer" (in our case really exit-to-exit) UDP connections. We expect this to enable the subset of applications that lack a fallback path which loops traffic through an app-provided server. @@ -592,3 +625,11 @@ This goal may be at odds with our privacy requirements. At minimum, a pool of ma TODO: Seems likely applications do often leak enough information through the plaintext portions of their UDP traffic in order to facilitate fingerprinting, I should look closer at this and confirm or deny. Even if the application traffic itself is fingerprint-resistant, this is easily combined with the above traffic injection attacks in order to mark specific communicating peers. + +### Additional Risks to Anonymity + +TODO: ICE connectivity checks, as mentioned elsewhere. + +TODO: Are there plaintext identifiers in these telecom apps? + +TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? -- cgit v1.2.3-54-g00ecf From 130c4f56a6ede67090cec6bbf9f46d747179c466 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 11 Jan 2024 11:47:12 -0800 Subject: More work on Risks section, new section on next steps --- proposals/XXX-udp-app-support.md | 125 +++++++++++++++++++++++++-------------- 1 file changed, 81 insertions(+), 44 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index c338f17..bf9b6af 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -40,11 +40,15 @@ Status: Draft - [Risks](#risks) - [Behavior Regressions](#behavior-regressions) - [Bandwidth Usage](#bandwidth-usage) - - [Malicious Traffic](#malicious-traffic) - - [Local Port Usage](#local-port-usage) + - [Local Port Exhaustion](#local-port-exhaustion) + - [Application Fingerprinting](#application-fingerprinting) + - [Peer-to-Peer Metadata Collection](#peer-to-peer-metadata-collection) + - [Interaction with Other Networks](#interaction-with-other-networks) - [Traffic Injection](#traffic-injection) - - [Peer-to-Peer Deanonymization](#peer-to-peer-deanonymization) - - [Additional Risks to Anonymity](#additional-risks-to-anonymity) + - [Malicious Outgoing Traffic](#malicious-outgoing-traffic) +- [Next Steps](#next-steps) + - [Requiring a Long-Term Datagram Plan](#requiring-a-long-term-datagram-plan) + - [Alternatively, Modular Application-Level Support](#alternatively-modular-application-level-support) ## Introduction @@ -415,7 +419,7 @@ Multiple peers are typically needed for a variety of reasons, like connectivity This approach would be simplest to implement and specify, especially in the existing C tor implementation. It also unfortunately has very limited compatibility, and no clear path toward incremental upgrades if we wish to improve compatibility later. -A simple one-to-one mapping between streams and sockets would preclude the optimizations necessary to address [local port usage](#local-port-usage) risks below. Solutions under this design are possible, but only by decoupling logical protocol-level sockets from the ultimate implementation-level sockets and reintroducing much of the complexity that we attempted to avoid by choosing this design. +A simple one-to-one mapping between streams and sockets would preclude the optimizations necessary to address [local port exhaustion](#local-port-exhaustion) risks below. Solutions under this design are possible, but only by decoupling logical protocol-level sockets from the ultimate implementation-level sockets and reintroducing much of the complexity that we attempted to avoid by choosing this design. ### One Stream per Flow @@ -551,7 +555,63 @@ We expect an increase in overall exit bandwidth requirements due to peer-to-peer Current users attempting to use BitTorrent over Tor are hampered by the lack of UDP compatibility. Interoperability with common file-sharing peers would make Tor more appealing to users with a large and sustained appetite for anonymized bandwidth. -### Malicious Traffic +### Local Port Exhaustion + +Exit routers will have a limited number of local UDP ports. In the most constrained scenario, an exit may have a single IP with 16384 or fewer ephemeral ports available. These ports could each be allocated by one client for an unbounded amount of exclusive use. + +In order to enforce high levels of isolation between different subsequent users of the same local UDP port, we may wish to enforce a delay between allocations during which nobody may own the port. Effective isolation requires this timer to be greater than any timer we expect to encounter on a peer or a NAT. In RFC4787's recommendations a NAT's mapping timer must be longer than 2 minutes. Our timer should ideally be *much* longer than 2 minutes. + +An attacker who allocates ports for only this minimum duration of 2 minutes would need to send 136.5 requests per second to achieve sustained use of all available ports. With multiple simultaneous clients this could easily be done while bypassing per-circuit rate limiting. + +The expanded definition of "Port overlapping" from [RFC7857 section 3](https://datatracker.ietf.org/doc/html/rfc7857#section-3), may form at least a partial mitigation: + +> This document clarifies that this port overlapping behavior may be extended to connections originating from different internal source IP addresses and ports as long as their destinations are different. + +This gives us an opportunity for a vast reduction in the number of required ports and file descriptors. Practically, though, it does require us to make a guess about which potential peers one source port may communicate with. + +Our UDP implementation will need to choose a port assignment based on knowledge of only the first peer the app is sending to. +Heuristically, we can make this work. The first peer in practice will be less unique than subsequent peers. Applications will contact centralized services before contacting peers. This ordering is necessary in the general case of ICE-like connection establishment. + +### Application Fingerprinting + +UDP applications present an increased surface of plaintext data that may be available for user fingerprinting by malicious exits. + +Exposed values can include short-lived identifiers like STUN usernames. +Typically it will also be possible to determine what type of software is in use, and maybe what version of that software. + +Short-lived identifiers are still quite valuable to attackers, because they can reliably track application sessions across changes to the Tor exit. If longer-lived identifiers exist for any reason, that of course provides a powerful tool for call metadata gathering. + +### Peer-to-Peer Metadata Collection + +One of our goals was to achieve the compatibility and perhaps performance benefits of allowing "peer-to-peer" (in our case really exit-to-exit) UDP connections. We expect this to enable the subset of applications that lack a fallback path which loops traffic through an app-provided server. + +This goal may be at odds with our privacy requirements. At minimum, a pool of malicious exit nodes could passively collect metadata about these connections as a noisy proxy for call metadata. + +Any additional signals like [application fingerprints](#application-fingerprinting) or [injected](#traffic-injection) markers may be used to enrich this metadata graph, possibly tracking users across sessions or across changes to the tunnel endpoint. + +### Interaction with Other Networks + +Any application using an ICE-like interactive connection establishment scheme will easily leak information across network boundaries if it ever has access to multiple networks at once. + +In applications that are not privacy conscious this is often desired behavior. For example, a video call to someone in your household may typically transit directly over Wifi, decreasing service costs and improving latency. This implies that some type of local identifier accompanies the call signaling info, allowing the devices to find each other's LAN address. + +Privacy-preserving solutions for this use case are still an active area of standardization effort. The [IETF draft on mDNS ICE candidates](https://www.ietf.org/archive/id/draft-ietf-mmusic-mdns-ice-candidates-02.html) proposes one way to accomplish this by generating short-lived unique IDs which are only useful to peers with physical access to the same mDNS services. + +Without special attention to privacy, the typical implementation is to share all available IP addresses and to initiate simultaneous connectivity tests using any IP pairs which cannot be trivially discarded. This applies to ICE as specified, but also to any proprietary protocol which operates in the same design space as ICE. This has multiple issues in a privacy-conscious environment: The IP address disclosures alone can be fatal to anonymity under common threat models. Even if meaningful IP addresses are not disclosed, the timing correlation from connectivity checks can provide confirmation beacons that alert an attacker to some connection between a LAN user and a Tor exit. + +### Traffic Injection + +Some forms of UDP support would have obvious and severe traffic injection vulnerabilities. For example, the very permissive *endpoint-independent filtering* strategy would allow any host on the internet to send datagrams in bulk to all available local ports on a Tor exit in order to map that traffic's effect on any guards they control. + +Any vehicle for malicious parties to mark traffic can be abused to deanonymize users. Even if there is a more restrictive filtering policy, UDP's lack of sequence numbers make header spoofing attacks considerably easier than in TCP. Third parties capable of routing datagrams to an exit with a spoofed source address could bypass filtering when the communicating parties are known or can be guessed. For example, a malicious superuser at an ISP without egress filtering could send packets with the source IPs set to various common DNS servers and application STUN/TURN servers. If a specific application is being targeted, only the exit node and local port numbers need to be guessed by brute force. + +In case of malicious exit relays, whole datagrams can be inserted and dropped, and datagrams may be padded with additional data, both without any specific knowledge of the application protocol. With specific protocol insights, a malicious relay may make arbitrary edits to plaintext data. + +Of particular interest is the plaintext STUN, TURN, and ICE traffic used by most WebRTC apps. These applications rely on higher-level protocols (SRTP, DTLS) to provide end-to-end encryption and authentication. A compromise at the connection establishment layer would not violate application-level end-to-end security requirements, making it outside the threat model of WebRTC but very much still a concern for Tor. + +These attacks are not fully unique to the proposed UDP support, but UDP may increase exposure. In cases where the application already has a fallback using TURN-over-TLS, the proposal is a clear regression over previous behaviors. Even when we are comparing plaintext to plaintext, there may be a serious downside to centralizing all connection establishment traffic through a small number of exit IPs. Depending on your threat model, it could very well be more private to allow the UDP traffic to bypass Tor entirely. + +### Malicious Outgoing Traffic We expect UDP compatibility in Tor will give malicious actors additional opportunities to transmit unwanted traffic. @@ -561,7 +621,7 @@ These are possible only in limited circumstances where the protocol allows an ar The peer is often at fault for having an overly permissive configuration. Nevertheless, any of these *easy* amplification targets can be exploited from Tor with little consequence, creating a nuisance for the ultimate target and for exit operators. -#### Amplification attacks against exit relay +#### Amplification attacks against an exit relay An amplification peer which doesn't allow arbitrary destinations can still be used to attack the exit relay itself or other users of that relay. This is essentially the same attack that is possible against any NAT the attacker is behind. @@ -587,49 +647,26 @@ The limit does need to be on peers, not stream IDs as we presently do for TCP. In this proposal stream IDs are not necessarily meaningful except as a representational choice made by clients. Strategies like the *per-mapping* stream assignment like we have for TCP. -### Local Port Usage - -Exit routers will have a limited number of local UDP ports. In the most constrained scenario, an exit may have a single IP with 16384 or fewer ephemeral ports available. These ports could each be allocated by one client for an unbounded amount of exclusive use. - -In order to enforce high levels of isolation between different subsequent users of the same local UDP port, we may wish to enforce a delay between allocations during which nobody may own the port. Effective isolation requires this timer to be greater than any timer we expect to encounter on a peer or a NAT. In RFC4787's recommendations a NAT's mapping timer must be longer than 2 minutes. Our timer should ideally be *much* longer than 2 minutes. - -An attacker who allocates ports for only this minimum duration of 2 minutes would need to send 136.5 requests per second to achieve sustained use of all available ports. With multiple simultaneous clients this could easily be done while bypassing per-circuit rate limiting. - -The expanded definition of "Port overlapping" from [RFC7857 section 3](https://datatracker.ietf.org/doc/html/rfc7857#section-3), may form at least a partial mitigation: - -> This document clarifies that this port overlapping behavior may be extended to connections originating from different internal source IP addresses and ports as long as their destinations are different. - -This gives us an opportunity for a vast reduction in the number of required ports and file descriptors. Practically, though, it does require us to make a guess about which potential peers one source port may communicate with. +## Next Steps -Our UDP implementation will need to choose a port assignment based on knowledge of only the first peer the app is sending to. -Heuristically, we can make this work. The first peer in practice will be less unique than subsequent peers. Applications will contact centralized services before contacting peers. This ordering is necessary in the general case of ICE-like connection establishment. +At this point we have perhaps too many possibilities for how to proceed. We could integrate UDP quite closely with Tor itself or not at all. -#### Traffic Injection +Choosing a route forward is both a risk/benefit tradeoff and a guess about the future of Tor. If we expect that some future version of Tor will provide its own datagram transport, we should plot a course aiming in that direction. If not, we might be better served by keeping compatibility features confined to separate protocol layers when that's practical. -Some forms of UDP support would have obvious and severe traffic injection vulnerabilities. For example, the very permissive *endpoint-independent filtering* strategy would allow any host on the internet to send datagrams in bulk to all available local ports on a Tor exit in order to map that traffic's effect on any guards they control. - -Any vehicle for malicious parties to mark traffic can be abused to deanonymize users. Even if there is a more restrictive filtering policy, UDP's lack of sequence numbers make header spoofing attacks considerably easier than in TCP. Third parties capable of routing datagrams to an exit with a spoofed source address could bypass filtering when the communicating parties are known or can be guessed. For example, a malicious superuser at an ISP without egress filtering could send packets with the source IPs set to various common DNS servers and application STUN/TURN servers. If a specific application is being targeted, only the exit node and local port numbers need to be guessed by brute force. - -In case of malicious exit relays, whole datagrams can be inserted and dropped, and datagrams may be padded with additional data, both without any specific knowledge of the application protocol. With specific protocol insights, a malicious relay may make arbitrary edits to plaintext data. - -Of particular interest is the plaintext STUN, TURN, and ICE traffic used by most WebRTC apps. These applications rely on higher-level protocols (SRTP, DTLS) to provide end-to-end encryption and authentication. A compromise at the connection establishment layer would not violate application-level end-to-end security requirements, making it outside the threat model of WebRTC but very much still a concern for Tor. - -These attacks are not fully unique to the proposed UDP support, but UDP may increase exposure. In cases where the application already has a fallback using TURN-over-TLS, the proposal is a clear regression over previous behaviors. Even when we are comparing plaintext to plaintext, there may be a serious downside to centralizing all connection establishment traffic through a small number of exit IPs. Depending on your threat model, it could very well be more private to allow the UDP traffic to bypass Tor entirely. - -#### Peer-to-Peer Deanonymization - -One of our goals was to achieve the compatibility and perhaps performance benefits of allowing "peer to peer" (in our case really exit-to-exit) UDP connections. We expect this to enable the subset of applications that lack a fallback path which loops traffic through an app-provided server. - -This goal may be at odds with our privacy requirements. At minimum, a pool of malicious exit nodes could passively collect metadata about these connections as a noisy proxy for call metadata. +### Requiring a Long-Term Datagram Plan -TODO: Seems likely applications do often leak enough information through the plaintext portions of their UDP traffic in order to facilitate fingerprinting, I should look closer at this and confirm or deny. +If we choose to add new long-term maintenance burdens to our protocol stack, we should ensure they serve our long-term goals for UDP adoption as well as these shorter-term application compatibility goals. -Even if the application traffic itself is fingerprint-resistant, this is easily combined with the above traffic injection attacks in order to mark specific communicating peers. +This work so far has been done with the assumption that end-to-end datagram support is out of scope. If we intend to proceed down any path which encodes a datagram-specific protocol into Tor proper, we should prioritize additional protocol research and standardization work. -### Additional Risks to Anonymity +### Alternatively, Modular Application-Level Support -TODO: ICE connectivity checks, as mentioned elsewhere. +Without a clear way to implement fully generic UDP support, we're left with application-level goals. +Different applications may have contrasting needs, and we can only achieve high levels of both compatibility and privacy by delegating some choices to app authors or per-app VPN settings. -TODO: Are there plaintext identifiers in these telecom apps? +This points to an alternative in which UDP support is excluded from Tor as much as possible while still supporting application requirements. +For example, this could motivate the [TURN Encapsulated in a Tor Stream](#turn-encapsulated-in-a-tor-stream) design, or even simpler designs where TURN servers are maintained independently from the Tor exit infrastructure. -TODO: Is there any chance we make the anonymity risk worse by providing UDP exits than it would be with an application-provided TCP relay server? +The next prototyping step we would need at this stage is a version of `onionmasq` extended to support NAT-style UDP mappings using TURN allocations provided through TURN-over-TCP or TURN-over-TLS. +This can be a platform for further application compatibility experiments. +It could potentially become a low-cost minimal implementation of UDP application compatibility, serving to assess which remaining user needs are still unmet. -- cgit v1.2.3-54-g00ecf From 0e662b4e0e40bd63e0c7216a6ca3a2ccdb44fea4 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 11 Jan 2024 12:02:51 -0800 Subject: Minor fixes: spell check, dangling sentence fragment --- proposals/XXX-udp-app-support.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index bf9b6af..6adb598 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -275,7 +275,7 @@ Non-browser-based apps often implement WebRTC or have components in common with Of particular importance to us, WebRTC uses the Interactive Connection Establishment (ICE) protocol to establish a bidirectional channel between endpoints that may or may not be behind a NAT with unknown configuration. -Any generalized solution to connection establishment, like ICE, will require sending connectivity test probes. These have an inherent hazard to anonymity: assuming no delays are inserted intentionally, the result is a broadcast of similar traffic across all available network interfaces. This could form a convenient correlation beacon for an attacker attempting to deanonymize users who use WebRTC over a Tor VPN. +Any generalized solution to connection establishment, like ICE, will require sending connectivity test probes. These have an inherent hazard to anonymity: assuming no delays are inserted intentionally, the result is a broadcast of similar traffic across all available network interfaces. This could form a convenient correlation beacon for an attacker attempting to de-anonymize users who use WebRTC over a Tor VPN. See [RFC8825](https://www.rfc-editor.org/rfc/rfc8825.html) *Overview: Real-Time Protocols for Browser-Based Applications*, @@ -365,7 +365,7 @@ The scope of this tunnel is quite similar to the existing TURN relays, used comm TURN is defined by [RFC8656](https://www.rfc-editor.org/rfc/rfc8656) as a set of extensions built on the framework from STUN in [RFC8489](https://www.rfc-editor.org/rfc/rfc8489.html). The capabilities are a good match for our needs, offering clients the ability to encapsulate UDP datagrams within a TCP stream, and to allocate local port mappings on the server. -TURN was designed to be a set of modular and extensible pieces, which may be too far opposed to Tor's design philosophy of providing single canonical representations. Any adoption of TURN will need to consider the potential for malicious implementations to mark traffic, facilitating deanonymization attacks. +TURN was designed to be a set of modular and extensible pieces, which may be too far opposed to Tor's design philosophy of providing single canonical representations. Any adoption of TURN will need to consider the potential for malicious implementations to mark traffic, facilitating de-anonymization attacks. TURN has a popular embeddable C-language implementation, [coturn](https://github.com/coturn/coturn), which may be suitable for including alongside or inside C tor. @@ -603,7 +603,7 @@ Without special attention to privacy, the typical implementation is to share all Some forms of UDP support would have obvious and severe traffic injection vulnerabilities. For example, the very permissive *endpoint-independent filtering* strategy would allow any host on the internet to send datagrams in bulk to all available local ports on a Tor exit in order to map that traffic's effect on any guards they control. -Any vehicle for malicious parties to mark traffic can be abused to deanonymize users. Even if there is a more restrictive filtering policy, UDP's lack of sequence numbers make header spoofing attacks considerably easier than in TCP. Third parties capable of routing datagrams to an exit with a spoofed source address could bypass filtering when the communicating parties are known or can be guessed. For example, a malicious superuser at an ISP without egress filtering could send packets with the source IPs set to various common DNS servers and application STUN/TURN servers. If a specific application is being targeted, only the exit node and local port numbers need to be guessed by brute force. +Any vehicle for malicious parties to mark traffic can be abused to de-anonymize users. Even if there is a more restrictive filtering policy, UDP's lack of sequence numbers make header spoofing attacks considerably easier than in TCP. Third parties capable of routing datagrams to an exit with a spoofed source address could bypass filtering when the communicating parties are known or can be guessed. For example, a malicious superuser at an ISP without egress filtering could send packets with the source IPs set to various common DNS servers and application STUN/TURN servers. If a specific application is being targeted, only the exit node and local port numbers need to be guessed by brute force. In case of malicious exit relays, whole datagrams can be inserted and dropped, and datagrams may be padded with additional data, both without any specific knowledge of the application protocol. With specific protocol insights, a malicious relay may make arbitrary edits to plaintext data. @@ -643,9 +643,7 @@ See also [RFC7675](https://www.rfc-editor.org/rfc/rfc7675.html) and STUN's conce We may want to place conservative limits on the maximum number of peers per mapping or per circuit, in order to make bulk scanning of UDP port space less convenient. The limit does need to be on peers, not stream IDs as we presently do for TCP. - In this proposal stream IDs are not necessarily meaningful except as a representational choice made by clients. -Strategies like the *per-mapping* stream assignment like we have for TCP. ## Next Steps -- cgit v1.2.3-54-g00ecf From 5b2d5866414fb205172ca735ce84d911feae948f Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 11 Jan 2024 19:26:54 -0800 Subject: More editing, and an expanded analysis of port overlapping --- proposals/XXX-udp-app-support.md | 215 +++++++++++++++++++++++---------------- 1 file changed, 130 insertions(+), 85 deletions(-) diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md index 6adb598..55f2192 100644 --- a/proposals/XXX-udp-app-support.md +++ b/proposals/XXX-udp-app-support.md @@ -56,8 +56,8 @@ This proposal takes a fresh look at the problem of implementing support in Tor f This work is being done with the sponsorship and goals of the [Tor VPN Client for Android project](https://gitlab.torproject.org/groups/tpo/-/milestones/32). -We start out by defining how this proposal compares to previous work, and the specific problem space we are addressing. -This leads into an analysis that references appropriate standards and proposes some specific solutions with properties we can compare. +The proposal begins with a summary of previous work and the specific problem space being addressed. +This leads into an analysis of possible solutions, and finally some possible conclusions about the available development opportunities. ### History @@ -71,10 +71,9 @@ The focus of this work was on a potential way to support unreliable traffic, not In proposal 100, a Tor *stream* is used for one pairing of local and remote address and port, copying the technique used by Tor for TCP. This works for some types of UDP applications, but it's broken by common behaviors like ICE connectivity checks, NAT traversal attempts, or using multiple servers via the same socket. -We go into more detail about application behavior below. No additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to what fits in a single Tor cell. -This value we will see is much too small for most applications. +This value is much too small for most applications. It's possible these UDP protocol details would have been elaborated during design, but the proposal hit a snag elsewhere: there was no agreement on a way to avoid facilitating new attacks against anonymity. @@ -92,7 +91,7 @@ Attacks that are described here, such as drops and injections, may be applied by [Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor stream using a new type of Tor message. As a prerequisite, it depends on [proposal 319](https://spec.torproject.org/proposals/319-wide-everything.html) to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. -In proposal 339 the property of binding a stream both to a local port and to a remote peer is described in UNIX-style terminology as a *connected socket*. We dive deeper into this idea below and supply an alternate formulation based on [RFC4787](https://www.rfc-editor.org/rfc/rfc4787), *NAT behavior requirements for UDP*. +In proposal 339 the property of binding a stream both to a local port and to a remote peer is described in UNIX-style terminology as a *connected socket*. This idea is explored below using alternate terminology from [RFC4787](https://www.rfc-editor.org/rfc/rfc4787), *NAT behavior requirements for UDP*. The single-peer *connected socket* behavior would be referred to as an *endpoint-dependent mapping* in RFC4787. This type works fine for client/server apps but precludes the use of NAT traversal for peer-to-peer transfer. @@ -104,11 +103,11 @@ We don't have a specific list of applications that must be supported, but we are Changes to the structure of the Tor network are out of scope, as are most performance optimizations. We expect to rely on common optimizations to the performance of Tor circuits, rather than looking to make specific changes that optimize for unreliable datagram transmission. -We will discuss the design implications of UDP on onion services below. -It's worth planning for this as a way to evaluate the design space, but in practice we are not aiming for UDP to onion services yet. +This document will briefly discuss UDP for onion services below. +It's worth planning for this as a way to evaluate the future design space, but in practice we are not aiming for UDP onion services yet. This will require changes to most applications that want to use it, as it implies that any media negotiations will need to understand onion addressing in addition to IPv4 and IPv6. -We do not rigidly define the subset of UDP traffic that will be allowed. +The allowed subset of UDP traffic is not subject to a single rigid definition. There are several options discussed below using the RFC4787 framework. We require support for DNS clients. Tor currently only supports a limited subset of DNS queries, and it's desirable to support more. This will be analyzed in detail as an application below. DNS is one of very few applications that still rely on fragmented UDP datagrams, though this may not be relevant for us since only servers typically need to control the production of fragments. @@ -120,13 +119,13 @@ Video calls between two Tor users should transit directly between two exit nodes This requires that allocated UDP ports can each communicate with multiple peers: *endpoint-independent mapping* as described by RFC4787. -We do not plan to support applications which accept arbitrary incoming datagrams, for example a DNS server hosted via Tor. +We do not plan to support applications which accept incoming datagrams from previously-unknown peers, for example a DNS server hosted via Tor. RFC4787 calls this *endpoint-independent filtering*. -It's unnecessary for running peer-to-peer apps, and it facilitates an extremely easy traffic injection attack. +It's unnecessary for running peer-to-peer apps, and it facilitates an extremely easy [traffic injection](#traffic-injection) attack. ## UDP Traffic Models -To better specify the role of a UDP extension for Tor, we will look at a few frameworks for describing UDP applications. +To better specify the role of a UDP extension for Tor, this section explores a few frameworks for describing noteworthy subsets of UDP traffic. ### User Datagram Protocol (RFC768) @@ -159,33 +158,40 @@ Every datagram sent or received on the socket may have a different peer address. ### Network Address Translation (NAT) -Much of the real-world complexity in UDP applications comes from their strategies to detect and overcome the effects of NAT. +Much of the real-world complexity in applying UDP comes from defining strategies to detect and overcome the effects of NAT. +As a result, an intimidating quantity of IETF documentation has been written on NAT behavior and on strategies for NAT traversal. -Many RFCs have been written on NAT behavior and NAT traversal strategies. [RFC4787](https://www.rfc-editor.org/rfc/rfc4787.html) and later [RFC7857](https://www.rfc-editor.org/rfc/rfc7857.html) offer best practices for implementing NAT. These are sometimes referred to as the [BEHAVE-WG](https://datatracker.ietf.org/wg/behave/about/) recommendations, based on the "Behavior Engineering for Hindrance Avoidance" working group behind them. -Carrier-grade NAT requirements are addressed by [RFC6888](https://www.rfc-editor.org/rfc/rfc6888.html). + +[RFC6888](https://www.rfc-editor.org/rfc/rfc6888.html) makes additional recommendations for "carrier grade" NAT systems, where small pools of IP addresses are shared among a much larger number of subscribers. [RFC8445](https://www.rfc-editor.org/rfc/rfc8445.html) describes the Interactive Connectivity Establishment (ICE) protocol, which has become a common and recommended application-level technique for building peer-to-peer applications that work through NAT. There are multiple fundamental technical issues that NAT presents: 1. NAT must be stateful in order to route replies back to the correct source. + This directly conflicts with the stateless nature of UDP itself. The NAT's mapping lifetime, determined by a timer, will not necessarily match the lifetime of the application-level connection. This necessitates keep-alive packets in some protocols. Protocols that allow their binding to expire may be open to a NAT rebinding attack, when a different party acquires access to the NAT's port allocation. + 2. Applications no longer know an address they can be reached at without outside help. + Chosen port numbers may or may not be used by the NAT. The effective IP address and port are not knowable without observing from an outside peer. -3. Filtering and mapping approaches both vary, and it's not generally possible to establish a connection without testing several alternatives and choosing the one that works. + +3. Filtering and mapping approaches both vary, and it's not generally possible to establish a connection without interactive probing. + This is the reason ICE exists, but it's also a possible anonymity hazard. + This risk is explored a bit further below in the context of [interaction with other networks](#interaction-with-other-networks). We can use the constraints of NAT both to understand application behavior and as an opportunity to model Tor's behavior as a type of NAT. In fact, Tor's many exit nodes already share similarity with some types of carrier-grade NAT. Applications will need to assume very little about the IP address their outbound UDP originates on, and we can use that to our advantage in implementing UDP for Tor. -This body of IETF work is invaluable for understanding the scope of the problem and for defining common terminology. -We must take inspiration from these documents while also keeping in mind that the analogy between Tor and a NAT is imperfect. +This body of work is invaluable for understanding the scope of the problem and for defining common terminology. +Let's take inspiration from these documents while also keeping in mind that the analogy between Tor and a NAT is imperfect. For example, in analyzing Tor as a type of carrier-grade NAT, we may consider the "pooling behavior" defined in RFC4787: the choice of which external addresses map to an internal address. Tor by necessity must carefully limit how predictable these mappings can ever be, to preserve its anonymity properties. A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well as the various per-subscriber limiting requirements. @@ -195,21 +201,21 @@ A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well RFC4787 defines a framework for understanding the behavior of NAT by analyzing both its "mapping" and "filtering" behavior separately. Mappings are the NAT's unit of state tracking. Filters are layered on top of mappings, potentially rejecting incoming datagrams that don't match an already-expected address. -Both RFC4787 and the demands of peer to peer applications make a good case for always using an **Endpoint-Independent Mapping**. +Both RFC4787 and the demands of peer-to-peer applications make a good case for always using an **Endpoint-Independent Mapping**. Choice of filtering strategy is left open by the BEHAVE-WG recommendations. -RFC4787 defines three types with different properties, and does not make one single recommendation for all circumstances. -We can gain some additional insight by looking at requirements that come from outside RFC4787. +RFC4787 does not make one single recommendation for all circumstances, +instead it defines three behavior options with different properties: - **Endpoint-Independent Filtering** allows incoming datagrams from any peer once a mapping has been established. RFC4787 recommends this approach, with the concession that it may not be ideal for all security requirements. - In the context of Tor, we can likely rule out this technique entirely. - It makes traffic injection attacks possible from any source address, provided you can guess the UDP port number used at an exit. + This technique cannot be safely applied in the context of Tor. + It makes [traffic injection](#traffic-injection) attacks possible from any source address, provided you can guess the UDP port number used at an exit. It also makes possible clear-net hosting of UDP servers using an exit node's IP, which may have undesirable abuse properties. - It precludes "Port overlapping" behavior as defined in RFC7857 section 3, which may be necessary in order to achieve sufficient utilization of local port numbers on exit nodes. + This permissive filter is also incompatible with our proposed mitigation to [local port exhaustion](#local-port-exhaustion) on exit relays. Even with per-circuit rate limiting, an attacker could trivially overwhelm the local port capacity of all combined UDP-capable Tor exits. It is still common for present-day applications to *prefer* endpoint-independent filtering, as it allows incoming connections from peers which cannot use STUN or a similar address fixing protocol. Choosing endpoint-independent filtering would have *some* compatibility benefit, but among modern protocols which use ICE and STUN there would be no improvement. @@ -229,11 +235,11 @@ We can gain some additional insight by looking at requirements that come from ou One security hazard of address-dependent and non-port-dependent filtering, identified in RFC4787, is that a peer on a NAT effectively negates the security benefits of this host filtering. In fact, this should raise additional red flags as applied to either Tor or carrier grade NAT. - If we are intending to support peer to peer applications, it should be commonplace to establish UDP flows between two tor exit nodes. + If supporting peer-to-peer applications, it should be commonplace to establish UDP flows between two Tor exit nodes. When this takes place, non-port-dependent filtering would then allow anyone on Tor to connect via those same nodes and perform traffic injection. The resulting security properties really become uncomfortably similar to endpoint-independent filtering. -- **Address and Port-Dependent Filtering** +- **Address- and Port-Dependent Filtering** This is the strictest variety of filtering, and it is an allowed alternative under RFC4787. It provides opportunities for increased security and opportunities for reduced compatibility, both of which in practice may depend on other factors. @@ -244,13 +250,15 @@ We can gain some additional insight by looking at requirements that come from ou This is the only type of filtering that provides any barrier at all between cross-circuit traffic injection when the communicating parties are known. RFC4787 recommends that filtering style be configurable. -We would like to implement that advice, but we are also looking for opportunities to make design decisions that give us the best network and end-user behaviors. +We would like to implement that advice, but only to the extent it can be done safely and meaningfully in the context of an anonymity system. +When possible, it would provide additional compatibility at no mandatory cost to allow applications to optionally request **Address-Dependent Filtering**. +Otherwise, **Address- and Port-Dependent Filtering** is the most appropriate default setting. ### Common Protocols Applications that want to use UDP are increasingly making use of higher-level protocols to avoid creating bespoke solutions for problems like NAT traversal, connection establishment, and reliable delivery. -We will analyze how these protocols affect Tor's UDP traffic requirements. +This section looks at how these protocols affect Tor's UDP traffic requirements. #### QUIC @@ -263,19 +271,21 @@ The intention is to provide transparent roaming as mobile users change networks. This automated path discovery opens additional opportunities for malicious traffic, for which the RFC also offers mitigations. See *path validation* in section `8.2`, and the additional mitigations from section `9.3`. When QUIC is used as an optional upgrade path, we must compare any proposed UDP support against the baseline of a non-upgraded original connection. -In these cases we are not looking for any specific compatibility enhancement, simply an avoidance of regression. +In these cases the goal is not a compatibility enhancement but an avoidance of regression. -In cases where QUIC is used as a primary protocol without TCP fallback, we expect UDP support to be vital. These applications are currently niche but we expect they may rise in popularity. +In cases where QUIC is used as a primary protocol without TCP fallback, UDP compatibility will be vital. +These applications are currently niche but we expect they may rise in popularity. #### WebRTC WebRTC is a large collection of protocols tuned to work together for media transport and NAT traversal. -It is increasingly common, both for browser-based telephony and for peer to peer data transfer. -Non-browser-based apps often implement WebRTC or have components in common with WebRTC. +It is increasingly common, both for browser-based telephony and for peer-to-peer data transfer. +Non-browser apps often implement WebRTC as well, for example using [`libwebrtc`](https://github.com/webrtc-sdk/libwebrtc). +Even non-WebRTC apps sometimes have significant overlaps in their technology stacks, due to the independent history of ICE, RTP, and SDP adoption. Of particular importance to us, WebRTC uses the Interactive Connection Establishment (ICE) protocol to establish a bidirectional channel between endpoints that may or may not be behind a NAT with unknown configuration. -Any generalized solution to connection establishment, like ICE, will require sending connectivity test probes. These have an inherent hazard to anonymity: assuming no delays are inserted intentionally, the result is a broadcast of similar traffic across all available network interfaces. This could form a convenient correlation beacon for an attacker attempting to de-anonymize users who use WebRTC over a Tor VPN. +Any generalized solution to connection establishment, like ICE, will require sending connectivity test probes. These have an inherent hazard to anonymity: assuming no delays are inserted intentionally, the result is a broadcast of similar traffic across all available network interfaces. This could form a convenient correlation beacon for an attacker attempting to de-anonymize users who use WebRTC over a Tor VPN. This is the risk enumerated below as [*interaction with other networks*](#interaction-with-other-networks). See [RFC8825](https://www.rfc-editor.org/rfc/rfc8825.html) *Overview: Real-Time Protocols for Browser-Based Applications*, @@ -290,7 +300,7 @@ With applications exhibiting such a wide variety of behaviors, how do we know wh How do we know which compatibility decisions will be most important to users? For this it's helpful to look at specific application behaviors. This is a best-effort analysis conducted at a point in time. -It's not meant to be a definitive reference, think of it as a site survey taken before we plan a building. +It's not meant to be a definitive reference, think of it as a site survey taken before planning a building. In alphabetical order: @@ -313,11 +323,8 @@ In alphabetical order: ## Overview of Possible Solutions -Now that we've defined some categories of UDP traffic we are interested in handling, this section starts to examine different high-level implementation techniques we could adopt. - -We can broadly split these into *datagram routing* and *tunneling*. - -Ideally we would be choosing a design that solves problems we have in the near-term while also providing a solid foundation for future enhancements to Tor, including changes which may add full support for unreliable delivery of datagrams. If we proceed down that path with insufficient understanding of the long-term goal, there's a risk that we will choose to adopt complexity in service of future goals while failing to serve them adequately when the time comes. +This section starts to examine different high-level implementation techniques we could adopt. +Broadly they can be split into *datagram routing* and *tunneling*. ### Datagram Routing @@ -339,22 +346,22 @@ This points to the key weakness of relying on a separate network for UDP: Tor ha #### Future Work on Tor -This is likely where we would seek to expand Tor's design in order to add end-to-end support for unreliable delivery in the future. +There may be room for future changes to Tor which allow it to somehow transfer and route datagrams directly, without a separate process of establishing circuits and tunnels. +If this is practical it may prove to be the simplest and highest performance route to achieving high quality UDP support in the long term. A specific design is out of the scope of this document. It is worth thinking early about how we can facilitate combinations of approaches. -We may find a need for an abstraction similar to a network routing table, allowing multiple UDP providers to coexist. - -Even without bringing any new network configurations to Tor, achieving interoperable support for both exit nodes and onion services in a Tor UDP implementation requires some attention to how multiple UDP providers can coexist. +Even without bringing any new network configurations to Tor, achieving interoperable support for both exit nodes and onion services in a Tor UDP implementation requires some attention to how multiple UDP providers can share protocol responsibilities. +This may warrant the introduction of some additional routing layer. ### Tunneling -The approaches in this section add a new construct which does not exist in UDP itself: a point to point tunnel between clients and some other location at which they establish the capability to send and receive UDP datagrams. +The approaches in this section add a new construct which does not exist in UDP itself: a point-to-point tunnel between clients and some other location at which they establish the capability to send and receive UDP datagrams. Any tunneling approach requires some way to discover tunnel endpoints. -We would like this to come as an extension of Tor's existing process for distributing consensus and representing exit policy. +For the best usability and adoption this should come as an extension of Tor's existing process for distributing consensus and representing exit policy. -We expect exit policies for UDP to have limited practical amounts of diversity. +In practice, exit policies for UDP will have limited practical amounts of diversity. VPN implementations will need to know ahead of time which tunnel circuits to build, or they will suffer a significant spike in latency for the first outgoing datagram to a new peer. Additionally, it's common for UDP port numbers to be randomly assigned. This would make highly specific Tor exit policies even less useful and even higher overhead than they are with TCP. @@ -365,7 +372,7 @@ The scope of this tunnel is quite similar to the existing TURN relays, used comm TURN is defined by [RFC8656](https://www.rfc-editor.org/rfc/rfc8656) as a set of extensions built on the framework from STUN in [RFC8489](https://www.rfc-editor.org/rfc/rfc8489.html). The capabilities are a good match for our needs, offering clients the ability to encapsulate UDP datagrams within a TCP stream, and to allocate local port mappings on the server. -TURN was designed to be a set of modular and extensible pieces, which may be too far opposed to Tor's design philosophy of providing single canonical representations. Any adoption of TURN will need to consider the potential for malicious implementations to mark traffic, facilitating de-anonymization attacks. +TURN was designed to be a set of modular and extensible pieces, which might be too distant from Tor's design philosophy of providing single canonical representations. Any adoption of TURN will need to consider the potential for malicious implementations to mark traffic, facilitating de-anonymization attacks. TURN has a popular embeddable C-language implementation, [coturn](https://github.com/coturn/coturn), which may be suitable for including alongside or inside C tor. @@ -373,28 +380,32 @@ TURN has a popular embeddable C-language implementation, [coturn](https://github Most of the discussion on UDP implementation in Tor so far has assumed this approach. Essentially it's the same strategy as TCP exits, but for UDP. When the OP initializes support for UDP, it pre-builds circuits to exits that support required UDP exit policies. These pre-built circuits can then be used as tunnels for UDP datagrams. -Within this overall approach, there are various ways we could choose to assign Tor *streams* for the UDP traffic. This will be considered below. +Within this overall approach, there are various ways to assign Tor *streams* for the UDP traffic. This will be considered below. #### Tor Stream Tunnel to a Rendezvous Point -To implement onion services which advertise UDP, we may consider using multiple simultaneous tunnels. -In addition to exit nodes, clients could establish the ability to allocate virtual UDP ports on a rendezvous node of some kind. +To implement onion services which advertise UDP ports, we can use additional tunnels. +A new type of tunnel could end at a rendezvous point rather than an exit node. +Clients could establish the ability to allocate a temporary virtual datagram mailbox at these rendezvous nodes. +This leaves more open questions about how outgoing traffic is routed, and which addressing format would be used for the datagram mailbox. The most immediate challenge in UDP rendezvous would then become application support. Protocols like STUN and ICE deal directly with IPv4 and IPv6 formats in order to advertise a reachable address to their peer. Supporting onion services in WebRTC would require protocol extensions and software modifications for STUN, TURN, ICE, and SDP at minimum. -UDP-like rendezvous extensions would have limited meaning unless they form part of a long-term strategy to forward datagrams in some new way for enhanced performance or compatibility. Otherwise, application authors might as well stick with Tor's existing TCP-like rendezvous functionality. +UDP-like rendezvous extensions would have limited meaning unless they form part of a long-term strategy to forward datagrams in some new way for enhanced performance or compatibility. Otherwise, application authors might as well stick with Tor's existing reliable circuit rendezvous functionality. ## Specific Designs Using Tor Streams Let's look more closely at Tor *streams*, the multiplexing layer right below circuits. -Streams have a 16-bit identifier, allocated arbitrarily by clients. Stream lifetimes are subject to some ambiguity still in the Tor spec. They are allocated by clients, but may be destroyed by either peer. +Streams have an opaque 16-bit identifier, allocated from the onion proxy (OP) endpoint. +Stream lifetimes are subject to some slight ambiguity still in the Tor spec. +They are always allocated from the OP end but may be destroyed asynchronously by either circuit endpoint. We have an opportunity to use this additional existing multiplexing layer to serve a useful function in the new protocol, or we can opt to interact with streams as little as possible in order to keep the protocol features more orthogonal. ### One Stream per Tunnel -The fewest new streams would be a single stream for all of UDP. This is what we get if we choose an off-the-shelf protocol like TURN as our UDP proxy. +The fewest new streams would be a single stream for all of UDP. This is the result when using an off-the-shelf stream oriented protocol like TURN-over-TCP as our UDP proxy. This approach would require only a single new Tor message type: @@ -402,7 +413,9 @@ This approach would require only a single new Tor message type: - Establish a stream as a connection to the exit relay's built-in (or configured) TURN server. -Note that RFC8656 requires authentication before data can be relayed, which is a good default best practice for the internet perhaps but is the opposite of what Tor is trying to do. We would either deviate from the specification to relax this auth requirement, or we would provide a way for clients to discover credentials: perhaps by fixing them ahead of time or by including them in the relay descriptor. + This would logically be a TURN-over-TCP connection, though it does not need to correspond to any real TCP socket if the TURN server is implemented in-process with tor. + +Note that RFC8656 requires authentication before data can be relayed, which is a good default best practice for the internet perhaps but is the opposite of what Tor is trying to do. We would either deviate from the specification to relax this auth requirement, or provide a way for clients to discover credentials: perhaps by fixing them ahead of time or by including them in the relay descriptor. ### One Stream per Socket @@ -425,7 +438,7 @@ A simple one-to-one mapping between streams and sockets would preclude the optim One stream **per flow** has also been suggested. Specifically, Mike Perry brought this up during our conversations about UDP recently and we spent some time analyzing it from a RFC4787 perspective. -We will see below it has interesting properties but also some hidden complexity. +This approach has some interesting properties but also hidden complexity that may ultimately make other options more easily applicable. This would assign a stream ID to the tuple consisting of at least `(local port, remote address, remote port)`. Additional flags may be included for features like transmit and receive filtering, IPv4/v6 choice, and IP *Don't Fragment*. @@ -447,7 +460,7 @@ When is this exit-originated circuit ID allocation potentially needed? It is clearly needed when using **address-dependent filtering**. An incoming datagram from a previously-unseen peer port is expected to be deliverable, and the exit would need to allocate an ID for it. -Even with the stricter **address and port-dependent filtering** we may still be exposed to exit-originated circuit IDs if there are mismatches in the lifetime of the filter and the stream. +Even with the stricter **address and port-dependent filtering** clients may still be exposed to exit-originated circuit IDs if there are mismatches in the lifetime of the filter and the stream. This approach thus requires some attention to either correctly allocating stream IDs on both sides of the circuit, or choosing a filtering strategy and filter/mapping lifetime that does not ever leave stream IDs undefined when expecting incoming datagrams. @@ -455,11 +468,11 @@ This approach thus requires some attention to either correctly allocating stream One stream **per mapping** is an alternative which attempts to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. -A mapping would always be allocated from the OP side. +A mapping would always be allocated from the OP (client) side. It could explicitly specify a filtering style, if we wish to allow applications to request non-port-dependent filtering for compatibility. Each datagram within the stream would still need to be tagged with a peer address/port in some way. -This approach would involve a single new type of stream, two new messages that pertain to these *mapping* streams: +This approach would involve a single new type of stream, and two new messages that pertain to these *mapping* streams: - `NEW_UDP_MAPPING` @@ -516,23 +529,24 @@ The implementation here could be a strict superset of the **per mapping** implem - Datagram contents only, without address. - Only appears on *flow* streams. -We must consider the traffic marking opportunities we open when allowing an exit to represent one incoming datagram as either a *flow* or *mapping* datagram. +We must consider the traffic marking opportunities provided when allowing an exit to represent one incoming datagram as either a *flow* or *mapping* datagram. -It's possible this traffic injection potential is not worse than the baseline amount of injection potential than every UDP protocol presents. See more on [risks](#risks) below. For this hybrid stream approach specifically, there's a limited mitigation we can use to allow exits only a bounded amount of leaked information per UDP peer: +It's possible this traffic injection potential is not worse than the baseline amount of injection potential than every UDP protocol presents. See more on [risks](#risks) below. For this hybrid stream approach specifically, there's a limited mitigation available which allows exits only a bounded amount of leaked information per UDP peer: -We would like to state that exits may not choose to send a `UDP_MAPPING_DATAGRAM` when they could have sent a `UDP_FLOW_DATAGRAM`. +Ideally exits may not choose to send a `UDP_MAPPING_DATAGRAM` when they could have sent a `UDP_FLOW_DATAGRAM`. Sometimes it is genuinely unclear though: an exit may have received this datagram in-between processing `NEW_UDP_MAPPING` and `NEW_UDP_FLOW`. -We could choose to terminate circuits which send a `UDP_MAPPING_DATAGRAM` for a peer that has already been referenced in a `UDP_FLOW_DATAGRAM`, giving exits a one-way gate to let them switch a peer from *mapping* datagram to *flow* datagram but not the reverse. +A partial mitigation would terminate circuits which send a `UDP_MAPPING_DATAGRAM` for a peer that has already been referenced in a `UDP_FLOW_DATAGRAM`. +The exit is thereby given a one-way gate allowing it to switch from using *mapping* datagrams to using *flow* datagrams at some point, but not to switch back and forth repeatedly. Mappings that do not request port-specific filtering may always get unexpected `UDP_MAPPING_DATAGRAM`s. Mappings that do use port-specific filtering could make a flow for their only expected peers, then expect to never see `UDP_MAPPING_DATAGRAM`. -We may wish for `NEW_UDP_MAPPING` to have an option requiring that only `UDP_FLOW_DATAGRAM` is to be used, never `UDP_MAPPING_DATAGRAM`. +`NEW_UDP_MAPPING` could have an option requiring that only `UDP_FLOW_DATAGRAM` is to be used, never `UDP_MAPPING_DATAGRAM`. This would remove the potential for ambiguity, but costs in compatibility as it's no longer possible to implement non-port-specific filtering. ## Risks Any proposed UDP support involves significant risks to user privacy and software maintainability. -We will try to elaborate some of these risks here, so they can be compared against the expected benefits. +This section elaborates some of these risks, so they can be compared against expected benefits. ### Behavior Regressions @@ -543,7 +557,7 @@ They may also occur for more fundamental reasons of protocol layering. For example, the redundant error correction layers when tunneling QUIC over TCP. These performance degradations are expected to be minor, but there's some unavoidable risk. -We may mitigate the risk of severe performance or compatibility regressions by giving users a way to toggle UDP support per-application. +The risk of severe performance or compatibility regressions may be mitigated by giving users a way to toggle UDP support per-application. Privacy and security regressions have more severe consequences and they can be much harder to detect. There are straightforward downgrades, like WebRTC apps that give up TURN-over-TLS for plaintext TURN-over-UDP. @@ -551,7 +565,7 @@ More subtly, the act of centralizing connection establishment traffic in Tor exi ### Bandwidth Usage -We expect an increase in overall exit bandwidth requirements due to peer-to-peer file sharing applications. +We should expect an increase in overall exit bandwidth requirements due to peer-to-peer file sharing applications. Current users attempting to use BitTorrent over Tor are hampered by the lack of UDP compatibility. Interoperability with common file-sharing peers would make Tor more appealing to users with a large and sustained appetite for anonymized bandwidth. @@ -559,7 +573,7 @@ Current users attempting to use BitTorrent over Tor are hampered by the lack of Exit routers will have a limited number of local UDP ports. In the most constrained scenario, an exit may have a single IP with 16384 or fewer ephemeral ports available. These ports could each be allocated by one client for an unbounded amount of exclusive use. -In order to enforce high levels of isolation between different subsequent users of the same local UDP port, we may wish to enforce a delay between allocations during which nobody may own the port. Effective isolation requires this timer to be greater than any timer we expect to encounter on a peer or a NAT. In RFC4787's recommendations a NAT's mapping timer must be longer than 2 minutes. Our timer should ideally be *much* longer than 2 minutes. +In order to enforce high levels of isolation between different subsequent users of the same local UDP port, we may wish to enforce an extended delay between allocations during which nobody may own the port. Effective isolation requires this timer duration to be greater than any timer encountered on a peer or a NAT. In RFC4787's recommendations a NAT's mapping timer must be longer than 2 minutes. Our timer should ideally be *much* longer than 2 minutes. An attacker who allocates ports for only this minimum duration of 2 minutes would need to send 136.5 requests per second to achieve sustained use of all available ports. With multiple simultaneous clients this could easily be done while bypassing per-circuit rate limiting. @@ -567,10 +581,28 @@ The expanded definition of "Port overlapping" from [RFC7857 section 3](https://d > This document clarifies that this port overlapping behavior may be extended to connections originating from different internal source IP addresses and ports as long as their destinations are different. -This gives us an opportunity for a vast reduction in the number of required ports and file descriptors. Practically, though, it does require us to make a guess about which potential peers one source port may communicate with. +This gives us an opportunity for a vast reduction in the number of required ports and file descriptors. +Exit routers can automatically allocate local ports for use with a specific peer when that peer is first added to the client's filter. + +Due to the general requirements of NAT traversal, UDP applications with any NAT support will always need to communicate with a relatively well known server prior to any attempts at peer-to-peer communication. +This early peer could be an entire application server, or it could be a STUN endpoint. +In any case, the identity of this first peer gives us a hint about the set of all potential peers. + +Within the exit router, each local port will track a separate mapping owner for each peer. +When processing that first outgoing datagram, the exit may choose any local port where the specific peer is not taken. +Subsequent outgoing datagrams on the same port may communicate with a different peer, and there's no guarantee all these future peers will be claimed successfully. + +When is this a problem? +An un-claimable peer represents a case where the exact `(local ip, local port, remote ip, remote port)` tuple is in use already for a different mapping in some other Tor stream. +For example, imagine two clients are running different types of telecom apps which are nevertheless inter-compatible and capable of both calling the same peers. +Alternatively, consider the same app but with servers in several regions. +The two apps will begin by communicating with different sets of peers, due to different application servers and different bundled STUN servers. +This is our hint that it's likely appropriate to overlap their local port allocations. +At this point, both of these applications may be successfully sharing a `(local ip, local port)` tuple on the exit. +As soon as one of these apps calls a peer with some `(remote ip, remote port)`, the other app will be unable to contact that specific peer. -Our UDP implementation will need to choose a port assignment based on knowledge of only the first peer the app is sending to. -Heuristically, we can make this work. The first peer in practice will be less unique than subsequent peers. Applications will contact centralized services before contacting peers. This ordering is necessary in the general case of ICE-like connection establishment. +The lack of connectivity may seem like a niche inconvenience, and perhaps that is the extent of the issue. +It seems likely this heuristic could result in a problematic information disclosure under some circumstances, and it deserves closer study. ### Application Fingerprinting @@ -579,11 +611,12 @@ UDP applications present an increased surface of plaintext data that may be avai Exposed values can include short-lived identifiers like STUN usernames. Typically it will also be possible to determine what type of software is in use, and maybe what version of that software. -Short-lived identifiers are still quite valuable to attackers, because they can reliably track application sessions across changes to the Tor exit. If longer-lived identifiers exist for any reason, that of course provides a powerful tool for call metadata gathering. +Short-lived identifiers are still quite valuable to attackers, because they may reliably track application sessions across changes to the Tor exit. If longer-lived identifiers exist for any reason, that of course provides a powerful tool for call metadata gathering. ### Peer-to-Peer Metadata Collection -One of our goals was to achieve the compatibility and perhaps performance benefits of allowing "peer-to-peer" (in our case really exit-to-exit) UDP connections. We expect this to enable the subset of applications that lack a fallback path which loops traffic through an app-provided server. +One of our goals was to achieve the compatibility and perhaps performance benefits of allowing "peer-to-peer" (in our case really exit-to-exit) UDP connections. +We expect this to enable the subset of applications that lack a fallback path which loops traffic through an app-provided server. This goal may be at odds with our privacy requirements. At minimum, a pool of malicious exit nodes could passively collect metadata about these connections as a noisy proxy for call metadata. @@ -609,40 +642,52 @@ In case of malicious exit relays, whole datagrams can be inserted and dropped, a Of particular interest is the plaintext STUN, TURN, and ICE traffic used by most WebRTC apps. These applications rely on higher-level protocols (SRTP, DTLS) to provide end-to-end encryption and authentication. A compromise at the connection establishment layer would not violate application-level end-to-end security requirements, making it outside the threat model of WebRTC but very much still a concern for Tor. -These attacks are not fully unique to the proposed UDP support, but UDP may increase exposure. In cases where the application already has a fallback using TURN-over-TLS, the proposal is a clear regression over previous behaviors. Even when we are comparing plaintext to plaintext, there may be a serious downside to centralizing all connection establishment traffic through a small number of exit IPs. Depending on your threat model, it could very well be more private to allow the UDP traffic to bypass Tor entirely. +These attacks are not fully unique to the proposed UDP support, but UDP may increase exposure. In cases where the application already has a fallback using TURN-over-TLS, the proposal is a clear regression over previous behaviors. Even when comparing plaintext to plaintext, there may be a serious downside to centralizing all connection establishment traffic through a small number of exit IPs. Depending on your threat model, it could very well be more private to allow the UDP traffic to bypass Tor entirely. ### Malicious Outgoing Traffic -We expect UDP compatibility in Tor will give malicious actors additional opportunities to transmit unwanted traffic. +We can expect UDP compatibility in Tor will give malicious actors additional opportunities to transmit unwanted traffic. + +In general, exit abuse will need to be filtered administratively somehow. +This is not unique to UDP support, and exit relay administration typically involves some type of filtering response tooling that falls outside the scope of Tor itself. + +Exit administrators may choose to modify their exit policy, or to silently drop problematic traffic. +Silent dropping is discouraged in most cases, as Tor prioritizes the accuracy of an exit's advertised policy. +Detailed exit policies have a significant space overhead in the overall Tor consensus document, but it's still seen as a valuable resource for clients during circuit establishment. + +Exit policy filtering may be less useful in UDP than with TCP due to the inconvenient latency spike when establishing a new [tunnel](#tunneling). +Applications that are sensitive to RTT measurements made during connection establishment may fail entirely when the tunnel cannot be pre-built. + +This section lists a few potential hazards, but the real-world impact may be hard to predict owing to a diversity of custom UDP protocols implemented across the internet. -#### Amplification attacks against arbitrary targets +- Amplification attacks against arbitrary targets -These are possible only in limited circumstances where the protocol allows an arbitrary reply address, like SIP. + These are possible only in limited circumstances where the protocol allows an arbitrary reply address, like SIP. The peer is often at fault for having an overly permissive configuration. Nevertheless, any of these *easy* amplification targets can be exploited from Tor with little consequence, creating a nuisance for the ultimate target and for exit operators. -#### Amplification attacks against an exit relay +- Amplification attacks against an exit relay -An amplification peer which doesn't allow arbitrary destinations can still be used to attack the exit relay itself or other users of that relay. + An amplification peer which doesn't allow arbitrary destinations can still be used to attack the exit relay itself or other users of that relay. This is essentially the same attack that is possible against any NAT the attacker is behind. -#### Malicious fragmented traffic +- Malicious fragmented traffic -If we allow sending large UDP datagrams over IPv4 without the *Don't Fragment* flag set, we allow attackers to generate fragmented IP datagrams. + If we allow sending large UDP datagrams over IPv4 without the *Don't Fragment* flag set, we allow attackers to generate fragmented IP datagrams. This is not itself a problem, but it has historically been a common source of inconsistencies in firewall behavior. -#### Excessive sends to an uninterested peer +- Excessive sends to an uninterested peer -Whereas TCP mandates a successful handshake, UDP will happily send unlimited amounts of traffic to a peer that has never responded. + Whereas TCP mandates a successful handshake, UDP will happily send unlimited amounts of traffic to a peer that has never responded. To prevent denial of service attacks we have an opportunity and perhaps a responsibility to define our supported subset of UDP to include true bidirectional traffic but exclude continued sends to peers who do not respond. -See also [RFC7675](https://www.rfc-editor.org/rfc/rfc7675.html) and STUN's concept of "Send consent". + See also [RFC7675](https://www.rfc-editor.org/rfc/rfc7675.html) and STUN's concept of "Send consent". -#### Excessive number of peers +- Excessive number of peers -We may want to place conservative limits on the maximum number of peers per mapping or per circuit, in order to make bulk scanning of UDP port space less convenient. + We may want to place conservative limits on the maximum number of peers per mapping or per circuit, in order to make bulk scanning of UDP port space less convenient. -The limit does need to be on peers, not stream IDs as we presently do for TCP. + The limit would need to be on peers, not stream IDs as we presently do for TCP. In this proposal stream IDs are not necessarily meaningful except as a representational choice made by clients. ## Next Steps -- cgit v1.2.3-54-g00ecf From 0de94cc8ed7b4d475a304c839358a03384be09b0 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 25 Jan 2024 08:58:24 -0800 Subject: Assign number to proposals/348-udp-app-support.md --- proposals/348-udp-app-support.md | 715 +++++++++++++++++++++++++++++++++++++++ proposals/XXX-udp-app-support.md | 715 --------------------------------------- 2 files changed, 715 insertions(+), 715 deletions(-) create mode 100644 proposals/348-udp-app-support.md delete mode 100644 proposals/XXX-udp-app-support.md diff --git a/proposals/348-udp-app-support.md b/proposals/348-udp-app-support.md new file mode 100644 index 0000000..55f2192 --- /dev/null +++ b/proposals/348-udp-app-support.md @@ -0,0 +1,715 @@ +# UDP Application Support in Tor + +```text +Filename: XXX-udp-app-support.md +Title: UDP Application Support in Tor +Author: Micah Elizabeth Scott +Created: December 2023 +Status: Draft +``` + +## Table of Contents + +- [Introduction](#introduction) + - [History](#history) + - [Scope](#scope) +- [UDP Traffic Models](#udp-traffic-models) + - [User Datagram Protocol (RFC768)](#user-datagram-protocol-rfc768) + - [Socket Layer](#socket-layer) + - [Network Address Translation (NAT)](#network-address-translation-nat) + - [Mapping and Filtering Behaviors](#mapping-and-filtering-behaviors) + - [Common Protocols](#common-protocols) + - [QUIC](#quic) + - [WebRTC](#webrtc) + - [Common Applications](#common-applications) +- [Overview of Possible Solutions](#overview-of-possible-solutions) + - [Datagram Routing](#datagram-routing) + - [Intentional UDP Leak](#intentional-udp-leak) + - [3rd Party Implementations](#3rd-party-implementations) + - [Future Work on Tor](#future-work-on-tor) + - [Tunneling](#tunneling) + - [TURN Encapsulated in a Tor Stream](#turn-encapsulated-in-a-tor-stream) + - [Tor Stream Tunnel to an Exit](#tor-stream-tunnel-to-an-exit) + - [Tor Stream Tunnel to a Rendezvous Point](#tor-stream-tunnel-to-a-rendezvous-point) +- [Specific Designs Using Tor Streams](#specific-designs-using-tor-streams) + - [One Stream per Tunnel](#one-stream-per-tunnel) + - [One Stream per Socket](#one-stream-per-socket) + - [One Stream per Flow](#one-stream-per-flow) + - [One Stream per Mapping](#one-stream-per-socket) + - [Hybrid Mapping and Flow Approach](#hybrid-mapping-and-flow-approach) +- [Risks](#risks) + - [Behavior Regressions](#behavior-regressions) + - [Bandwidth Usage](#bandwidth-usage) + - [Local Port Exhaustion](#local-port-exhaustion) + - [Application Fingerprinting](#application-fingerprinting) + - [Peer-to-Peer Metadata Collection](#peer-to-peer-metadata-collection) + - [Interaction with Other Networks](#interaction-with-other-networks) + - [Traffic Injection](#traffic-injection) + - [Malicious Outgoing Traffic](#malicious-outgoing-traffic) +- [Next Steps](#next-steps) + - [Requiring a Long-Term Datagram Plan](#requiring-a-long-term-datagram-plan) + - [Alternatively, Modular Application-Level Support](#alternatively-modular-application-level-support) + +## Introduction + +This proposal takes a fresh look at the problem of implementing support in Tor for applications which require UDP/IP communication. + +This work is being done with the sponsorship and goals of the [Tor VPN Client for Android project](https://gitlab.torproject.org/groups/tpo/-/milestones/32). + +The proposal begins with a summary of previous work and the specific problem space being addressed. +This leads into an analysis of possible solutions, and finally some possible conclusions about the available development opportunities. + +### History + +There have already been multiple attempts over Tor's history to define some type of UDP extension. + +#### 2006 + +[Proposal 100](https://spec.torproject.org/proposals/100-tor-spec-udp.html) by Marc Liberatore in 2006 suggested a way to "add support for tunneling unreliable datagrams through tor with as few modifications to the protocol as possible." +This proposal suggested extending the existing TLS+TCP protocol with a new DTLS+UDP link mode. +The focus of this work was on a potential way to support unreliable traffic, not necessarily on UDP itself or on UDP applications. + +In proposal 100, a Tor *stream* is used for one pairing of local and remote address and port, copying the technique used by Tor for TCP. +This works for some types of UDP applications, but it's broken by common behaviors like ICE connectivity checks, NAT traversal attempts, or using multiple servers via the same socket. + +No additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to what fits in a single Tor cell. +This value is much too small for most applications. + +It's possible these UDP protocol details would have been elaborated during design, but the proposal hit a snag elsewhere: +there was no agreement on a way to avoid facilitating new attacks against anonymity. + +#### 2018 + +In 2018, Nick Mathewson and Mike Perry wrote a +[summary of the side-channel issues with unreliable transports for Tor](https://research.torproject.org/techreports/side-channel-analysis-2018-11-27.pdf). + +The focus of this document is on the communication between Tor relays, but there is considerable overlap between the attack space explored here and the potential risks of any application-level UDP support. +Attacks that are described here, such as drops and injections, may be applied by malicious exits or some types of third parties even in an implementation using only present-day reliable Tor transports. + +#### 2020 + +[Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor stream using a new type of Tor message. +As a prerequisite, it depends on [proposal 319](https://spec.torproject.org/proposals/319-wide-everything.html) to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. + +In proposal 339 the property of binding a stream both to a local port and to a remote peer is described in UNIX-style terminology as a *connected socket*. This idea is explored below using alternate terminology from [RFC4787](https://www.rfc-editor.org/rfc/rfc4787), *NAT behavior requirements for UDP*. +The single-peer *connected socket* behavior would be referred to as an *endpoint-dependent mapping* in RFC4787. +This type works fine for client/server apps but precludes the use of NAT traversal for peer-to-peer transfer. + +### Scope + +This proposal aims to allow Tor applications and Tor-based VPNs to provide compatibility with applications that require UDP/IP communications. + +We don't have a specific list of applications that must be supported, but we are currently aiming for broad support of popular applications while still respecting and referencing all applicable Internet standards documents. + +Changes to the structure of the Tor network are out of scope, as are most performance optimizations. We expect to rely on common optimizations to the performance of Tor circuits, rather than looking to make specific changes that optimize for unreliable datagram transmission. + +This document will briefly discuss UDP for onion services below. +It's worth planning for this as a way to evaluate the future design space, but in practice we are not aiming for UDP onion services yet. +This will require changes to most applications that want to use it, as it implies that any media negotiations will need to understand onion addressing in addition to IPv4 and IPv6. + +The allowed subset of UDP traffic is not subject to a single rigid definition. +There are several options discussed below using the RFC4787 framework. + +We require support for DNS clients. Tor currently only supports a limited subset of DNS queries, and it's desirable to support more. This will be analyzed in detail as an application below. DNS is one of very few applications that still rely on fragmented UDP datagrams, though this may not be relevant for us since only servers typically need to control the production of fragments. + +We require support for voice/video telecommunications apps. Even without an underlying transport that supports unreliable datagrams, we expect a tunnel to provide a usable level of compatibility. This design space is very similar to the TURN ([RFC8656](https://www.rfc-editor.org/rfc/rfc8656)) specification, already used very widely for compatibility with networks that filter UDP. See the analysis of specific applications below. + +We require support for peer-to-peer UDP transfer without additional relaying, in apps that use ICE ([RFC8445](https://www.rfc-editor.org/rfc/rfc8445)) or similar connection establishment techniques. +Video calls between two Tor users should transit directly between two exit nodes. +This requires that allocated UDP ports can each communicate with multiple peers: +*endpoint-independent mapping* as described by RFC4787. + +We do not plan to support applications which accept incoming datagrams from previously-unknown peers, for example a DNS server hosted via Tor. +RFC4787 calls this *endpoint-independent filtering*. +It's unnecessary for running peer-to-peer apps, and it facilitates an extremely easy [traffic injection](#traffic-injection) attack. + +## UDP Traffic Models + +To better specify the role of a UDP extension for Tor, this section explores a few frameworks for describing noteworthy subsets of UDP traffic. + +### User Datagram Protocol (RFC768) + +The "User Interface" suggested by [RFC768](https://www.rfc-editor.org/rfc/rfc768) for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. + +Despite UDP's simplicity as an application of IP, we do need to be aware of IP features that are typically hidden by TCP's abstraction. + +UDP applications typically try to obtain an awareness of the path MTU, using some type of path MTU discovery (PMTUD) algorithm. +On IPv4, this requires sending packets with the "Don't Fragment" flag set, and measuring when those packets are lost or when ICMP "Fragmentation Needed" replies are seen. + +Note that many applications have their own requirements for path MTU. For example, QUIC and common implementations of WebRTC require an MTU no smaller than 1200 bytes, but they can discover larger MTUs when available. + +### Socket Layer + +In practice the straightforward "User Interface" from RFC768, capable of arbitrary local address, is only available to privileged users. + +BSD-style sockets support UDP via `SOCK_DGRAM`. +UDP is a stateless protocol, but sockets do have state. +Each socket is bound, either explicitly with `bind()` or automatically, to a source IP and port. + +At the API level, a socket is said to be *connected* to a remote `(address, port)` if that address is the default destination. +A *connected* socket will also filter out incoming packets with source addresses different from this default destination. +A socket is considered *unconnected* if `connect()` has not been called. +These sockets have no default destination, and they accept datagrams from any source. + +There does not need to be any particular mapping between the lifetime of these application sockets and any higher-level "connection" the application establishes. +It's better to think of one socket as one allocated source port. +A typical application may allocate only a single port (one socket) for talking to many peers. +Every datagram sent or received on the socket may have a different peer address. + +### Network Address Translation (NAT) + +Much of the real-world complexity in applying UDP comes from defining strategies to detect and overcome the effects of NAT. +As a result, an intimidating quantity of IETF documentation has been written on NAT behavior and on strategies for NAT traversal. + +[RFC4787](https://www.rfc-editor.org/rfc/rfc4787.html) and later [RFC7857](https://www.rfc-editor.org/rfc/rfc7857.html) offer best practices for implementing NAT. These are sometimes referred to as the [BEHAVE-WG](https://datatracker.ietf.org/wg/behave/about/) recommendations, based on the "Behavior Engineering for Hindrance Avoidance" working group behind them. + +[RFC6888](https://www.rfc-editor.org/rfc/rfc6888.html) makes additional recommendations for "carrier grade" NAT systems, where small pools of IP addresses are shared among a much larger number of subscribers. + +[RFC8445](https://www.rfc-editor.org/rfc/rfc8445.html) describes the Interactive Connectivity Establishment (ICE) protocol, which has become a common and recommended application-level technique for building peer-to-peer applications that work through NAT. + +There are multiple fundamental technical issues that NAT presents: + +1. NAT must be stateful in order to route replies back to the correct source. + + This directly conflicts with the stateless nature of UDP itself. + The NAT's mapping lifetime, determined by a timer, will not necessarily match the lifetime of the application-level connection. + This necessitates keep-alive packets in some protocols. + Protocols that allow their binding to expire may be open to a NAT rebinding attack, when a different party acquires access to the NAT's port allocation. + +2. Applications no longer know an address they can be reached at without outside help. + + Chosen port numbers may or may not be used by the NAT. + The effective IP address and port are not knowable without observing from an outside peer. + +3. Filtering and mapping approaches both vary, and it's not generally possible to establish a connection without interactive probing. + + This is the reason ICE exists, but it's also a possible anonymity hazard. + This risk is explored a bit further below in the context of [interaction with other networks](#interaction-with-other-networks). + +We can use the constraints of NAT both to understand application behavior and as an opportunity to model Tor's behavior as a type of NAT. +In fact, Tor's many exit nodes already share similarity with some types of carrier-grade NAT. +Applications will need to assume very little about the IP address their outbound UDP originates on, and we can use that to our advantage in implementing UDP for Tor. + +This body of work is invaluable for understanding the scope of the problem and for defining common terminology. +Let's take inspiration from these documents while also keeping in mind that the analogy between Tor and a NAT is imperfect. +For example, in analyzing Tor as a type of carrier-grade NAT, we may consider the "pooling behavior" defined in RFC4787: the choice of which external addresses map to an internal address. +Tor by necessity must carefully limit how predictable these mappings can ever be, to preserve its anonymity properties. +A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well as the various per-subscriber limiting requirements. + +#### Mapping and Filtering Behaviors + +RFC4787 defines a framework for understanding the behavior of NAT by analyzing both its "mapping" and "filtering" behavior separately. +Mappings are the NAT's unit of state tracking. +Filters are layered on top of mappings, potentially rejecting incoming datagrams that don't match an already-expected address. +Both RFC4787 and the demands of peer-to-peer applications make a good case for always using an **Endpoint-Independent Mapping**. + +Choice of filtering strategy is left open by the BEHAVE-WG recommendations. +RFC4787 does not make one single recommendation for all circumstances, +instead it defines three behavior options with different properties: + +- **Endpoint-Independent Filtering** allows incoming datagrams from any peer once a mapping has been established. + + RFC4787 recommends this approach, with the concession that it may not be ideal for all security requirements. + + This technique cannot be safely applied in the context of Tor. + It makes [traffic injection](#traffic-injection) attacks possible from any source address, provided you can guess the UDP port number used at an exit. + It also makes possible clear-net hosting of UDP servers using an exit node's IP, which may have undesirable abuse properties. + + This permissive filter is also incompatible with our proposed mitigation to [local port exhaustion](#local-port-exhaustion) on exit relays. Even with per-circuit rate limiting, an attacker could trivially overwhelm the local port capacity of all combined UDP-capable Tor exits. + + It is still common for present-day applications to *prefer* endpoint-independent filtering, as it allows incoming connections from peers which cannot use STUN or a similar address fixing protocol. + Choosing endpoint-independent filtering would have *some* compatibility benefit, but among modern protocols which use ICE and STUN there would be no improvement. + The cost, on the other hand, would be an uncomfortably low-cost traffic injection attack and additional risks toward exit nodes. + +- **Address-Dependent Filtering** + + This is a permitted alternative according to RFC4787, in which incoming datagrams are allowed from only IP addresses we have previously sent to, but any port on that IP may be the sender. + + The intended benefits of this approach versus the port-dependent filtering below are unclear, and may no longer be relevant. In theory they would be: + + - To support a class of applications that rely on, for a single local port, multiple remote ports achieving filter acceptance status when only one of those ports has been sent a datagram. + We are currently lacking examples of applications in this category. + Any application using ICE should be outside this category, as each port would have its own connectivity check datagrams exchanged in each direction. + + - REQ-8 in RFC4787 claims the existence of a scenario in which this approach facilitates ICE connections with a remote peer that disregards REQ-1 (the peer does not use **Endpoint-Independent Mapping**). It is not clear that this claim is still relevant. + + One security hazard of address-dependent and non-port-dependent filtering, identified in RFC4787, is that a peer on a NAT effectively negates the security benefits of this host filtering. + In fact, this should raise additional red flags as applied to either Tor or carrier grade NAT. + If supporting peer-to-peer applications, it should be commonplace to establish UDP flows between two Tor exit nodes. + When this takes place, non-port-dependent filtering would then allow anyone on Tor to connect via those same nodes and perform traffic injection. + The resulting security properties really become uncomfortably similar to endpoint-independent filtering. + +- **Address- and Port-Dependent Filtering** + + This is the strictest variety of filtering, and it is an allowed alternative under RFC4787. + It provides opportunities for increased security and opportunities for reduced compatibility, both of which in practice may depend on other factors. + + For every application we've analyzed so far, port-dependent filtering is not a problem. + Usage of ICE will open all required filters during the connectivity check phase. + + This is the only type of filtering that provides any barrier at all between cross-circuit traffic injection when the communicating parties are known. + +RFC4787 recommends that filtering style be configurable. +We would like to implement that advice, but only to the extent it can be done safely and meaningfully in the context of an anonymity system. +When possible, it would provide additional compatibility at no mandatory cost to allow applications to optionally request **Address-Dependent Filtering**. +Otherwise, **Address- and Port-Dependent Filtering** is the most appropriate default setting. + +### Common Protocols + +Applications that want to use UDP are increasingly making use of higher-level protocols to avoid creating bespoke solutions for problems like NAT traversal, connection establishment, and reliable delivery. + +This section looks at how these protocols affect Tor's UDP traffic requirements. + +#### QUIC + +[RFC9000](https://www.rfc-editor.org/rfc/rfc9000.html) defines QUIC, a multiplexed secure point-to-point protocol which supports reliable and unreliable delivery. The most common use is as an optional HTTP replacement, especially among Google services. + +QUIC does not normally try to traverse NAT; as an HTTP replacement, the server is expected to have an address reachable without any prior connection setup. + +QUIC provides its own flexible connection lifetimes which may outlive individual network links or NAT mappings. +The intention is to provide transparent roaming as mobile users change networks. +This automated path discovery opens additional opportunities for malicious traffic, for which the RFC also offers mitigations. See *path validation* in section `8.2`, and the additional mitigations from section `9.3`. + +When QUIC is used as an optional upgrade path, we must compare any proposed UDP support against the baseline of a non-upgraded original connection. +In these cases the goal is not a compatibility enhancement but an avoidance of regression. + +In cases where QUIC is used as a primary protocol without TCP fallback, UDP compatibility will be vital. +These applications are currently niche but we expect they may rise in popularity. + +#### WebRTC + +WebRTC is a large collection of protocols tuned to work together for media transport and NAT traversal. +It is increasingly common, both for browser-based telephony and for peer-to-peer data transfer. +Non-browser apps often implement WebRTC as well, for example using [`libwebrtc`](https://github.com/webrtc-sdk/libwebrtc). +Even non-WebRTC apps sometimes have significant overlaps in their technology stacks, due to the independent history of ICE, RTP, and SDP adoption. + +Of particular importance to us, WebRTC uses the Interactive Connection Establishment (ICE) protocol to establish a bidirectional channel between endpoints that may or may not be behind a NAT with unknown configuration. + +Any generalized solution to connection establishment, like ICE, will require sending connectivity test probes. These have an inherent hazard to anonymity: assuming no delays are inserted intentionally, the result is a broadcast of similar traffic across all available network interfaces. This could form a convenient correlation beacon for an attacker attempting to de-anonymize users who use WebRTC over a Tor VPN. This is the risk enumerated below as [*interaction with other networks*](#interaction-with-other-networks). + +See +[RFC8825](https://www.rfc-editor.org/rfc/rfc8825.html) *Overview: Real-Time Protocols for Browser-Based Applications*, +[RFC8445](https://www.rfc-editor.org/rfc/rfc8445.html) *Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal*, +[RFC8838](https://www.rfc-editor.org/rfc/rfc8838.html) *Trickle ICE: Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (ICE) Protocol*, +[RFC5389](https://www.rfc-editor.org/rfc/rfc5389.html) *Session Traversal Utilities for NAT (STUN)*, +and others. + +### Common Applications + +With applications exhibiting such a wide variety of behaviors, how do we know what to expect from a good implementation? +How do we know which compatibility decisions will be most important to users? +For this it's helpful to look at specific application behaviors. +This is a best-effort analysis conducted at a point in time. +It's not meant to be a definitive reference, think of it as a site survey taken before planning a building. + +In alphabetical order: + +| Application | Type | Protocol features | Current behavior | Expected outcome | +| ---------------------- | -------------- | --------------------------------- | ------------------ | -------------------------------------------------- | +| BitTorrent | File sharing | Many peers per local port | Fails without UDP | Works, new source of nuisance traffic | +| BigBlueButton | Telecom | WebRTC, TURN, TURN-over-TLS | Works | Slight latency improvement | +| Discord | Telecom | Proprietary, client/server | Fails without UDP | Starts working | +| DNS | Infrastructure | Might want IP fragmentation | Limited | Full DNS support, for better and worse | +| FaceTime | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | +| Google Meet | Telecom | STUN/TURN, TURN-over-TCP | Works | Slight latency improvement | +| Jitsi (meet.jit.si) | Telecom | WebRTC, TURN-over-TLS, Cloudflare | Fails on Tor | No change, problem does not appear UDP-related | +| Jitsi (docker-compose) | Telecom | WebRTC, centralized STUN only | Fails without UDP | Starts working | +| Linphone | Telecom (SIP) | SIP-over-TLS, STUN, TURN | Fails without UDP | Starts working | +| Signal | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | +| Skype | Telecom | P2P, STUN, TURN-over-TLS | Works | Slight latency improvement | +| WhatsApp | Telecom | STUN, TURN-over-TCP. Multi server | Works | Slight latency improvement | +| WiFi Calling | Telecom | IPsec tunnel | Out of scope | Still out of scope | +| Zoom | Telecom | client/server or P2P, UDP/TCP | Works | Slight latency improvement | + +## Overview of Possible Solutions + +This section starts to examine different high-level implementation techniques we could adopt. +Broadly they can be split into *datagram routing* and *tunneling*. + +### Datagram Routing + +These approaches seek to use a network that can directly route datagrams from place to place. These approaches are the most obviously suitable for implementing UDP, but they also form the widest departure from classic Tor. + +#### Intentional UDP Leak + +The simplest approach would be to allow UDP traffic to bypass the anonymity layer. This is an unacceptable loss of anonymity in many cases, given that the client's real IP address is made visible to web application providers. + +In other cases, this is an acceptable or even preferable approach. For example, VPN users may be more concerned with achieving censorship-resistant network connectivity than hiding personal identifiers from application vendors. + +In threat models where application vendors are more trustworthy than the least trustworthy Tor exits, it may be more appropriate to allow direct peer-to-peer connections than to trust Tor exits with unencrypted connection establishment traffic. + +#### 3rd Party Implementations + +Another option would be to use an unrelated anonymizer system for datagram traffic. It's not clear that a suitable system already exists. I2P provides a technical solution for routing anonymized datagrams, but not a Tor-style infrastructure of exit node operators. + +This points to the key weakness of relying on a separate network for UDP: Tor has an especially well-developed community of volunteers running relays. Any UDP solution that is inconvenient for relay operators has little chance of adoption. + +#### Future Work on Tor + +There may be room for future changes to Tor which allow it to somehow transfer and route datagrams directly, without a separate process of establishing circuits and tunnels. +If this is practical it may prove to be the simplest and highest performance route to achieving high quality UDP support in the long term. +A specific design is out of the scope of this document. + +It is worth thinking early about how we can facilitate combinations of approaches. +Even without bringing any new network configurations to Tor, achieving interoperable support for both exit nodes and onion services in a Tor UDP implementation requires some attention to how multiple UDP providers can share protocol responsibilities. +This may warrant the introduction of some additional routing layer. + +### Tunneling + +The approaches in this section add a new construct which does not exist in UDP itself: a point-to-point tunnel between clients and some other location at which they establish the capability to send and receive UDP datagrams. + +Any tunneling approach requires some way to discover tunnel endpoints. +For the best usability and adoption this should come as an extension of Tor's existing process for distributing consensus and representing exit policy. + +In practice, exit policies for UDP will have limited practical amounts of diversity. +VPN implementations will need to know ahead of time which tunnel circuits to build, or they will suffer a significant spike in latency for the first outgoing datagram to a new peer. +Additionally, it's common for UDP port numbers to be randomly assigned. +This would make highly specific Tor exit policies even less useful and even higher overhead than they are with TCP. + +#### TURN Encapsulated in a Tor Stream + +The scope of this tunnel is quite similar to the existing TURN relays, used commonly by WebRTC applications to implement fallbacks for clients who cannot find a more direct connection path. + +TURN is defined by [RFC8656](https://www.rfc-editor.org/rfc/rfc8656) as a set of extensions built on the framework from STUN in [RFC8489](https://www.rfc-editor.org/rfc/rfc8489.html). The capabilities are a good match for our needs, offering clients the ability to encapsulate UDP datagrams within a TCP stream, and to allocate local port mappings on the server. + +TURN was designed to be a set of modular and extensible pieces, which might be too distant from Tor's design philosophy of providing single canonical representations. Any adoption of TURN will need to consider the potential for malicious implementations to mark traffic, facilitating de-anonymization attacks. + +TURN has a popular embeddable C-language implementation, [coturn](https://github.com/coturn/coturn), which may be suitable for including alongside or inside C tor. + +#### Tor Stream Tunnel to an Exit + +Most of the discussion on UDP implementation in Tor so far has assumed this approach. Essentially it's the same strategy as TCP exits, but for UDP. When the OP initializes support for UDP, it pre-builds circuits to exits that support required UDP exit policies. These pre-built circuits can then be used as tunnels for UDP datagrams. + +Within this overall approach, there are various ways to assign Tor *streams* for the UDP traffic. This will be considered below. + +#### Tor Stream Tunnel to a Rendezvous Point + +To implement onion services which advertise UDP ports, we can use additional tunnels. +A new type of tunnel could end at a rendezvous point rather than an exit node. +Clients could establish the ability to allocate a temporary virtual datagram mailbox at these rendezvous nodes. + +This leaves more open questions about how outgoing traffic is routed, and which addressing format would be used for the datagram mailbox. +The most immediate challenge in UDP rendezvous would then become application support. Protocols like STUN and ICE deal directly with IPv4 and IPv6 formats in order to advertise a reachable address to their peer. Supporting onion services in WebRTC would require protocol extensions and software modifications for STUN, TURN, ICE, and SDP at minimum. + +UDP-like rendezvous extensions would have limited meaning unless they form part of a long-term strategy to forward datagrams in some new way for enhanced performance or compatibility. Otherwise, application authors might as well stick with Tor's existing reliable circuit rendezvous functionality. + +## Specific Designs Using Tor Streams + +Let's look more closely at Tor *streams*, the multiplexing layer right below circuits. + +Streams have an opaque 16-bit identifier, allocated from the onion proxy (OP) endpoint. +Stream lifetimes are subject to some slight ambiguity still in the Tor spec. +They are always allocated from the OP end but may be destroyed asynchronously by either circuit endpoint. + +We have an opportunity to use this additional existing multiplexing layer to serve a useful function in the new protocol, or we can opt to interact with streams as little as possible in order to keep the protocol features more orthogonal. + +### One Stream per Tunnel + +The fewest new streams would be a single stream for all of UDP. This is the result when using an off-the-shelf stream oriented protocol like TURN-over-TCP as our UDP proxy. + +This approach would require only a single new Tor message type: + +- `CONNECT_TURN` + + - Establish a stream as a connection to the exit relay's built-in (or configured) TURN server. + + This would logically be a TURN-over-TCP connection, though it does not need to correspond to any real TCP socket if the TURN server is implemented in-process with tor. + +Note that RFC8656 requires authentication before data can be relayed, which is a good default best practice for the internet perhaps but is the opposite of what Tor is trying to do. We would either deviate from the specification to relax this auth requirement, or provide a way for clients to discover credentials: perhaps by fixing them ahead of time or by including them in the relay descriptor. + +### One Stream per Socket + +One stream **per socket** was the approach suggested in [Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020. + +In proposal 339, there would be one new type of stream and three new message types: `CONNECT_UDP`, `CONNECTED_UDP`, and `DATAGRAM`. + +Each stream's lifetime would match the lifetime of a local port allocation. +There would be a single peer `(remote address, remote port)` allowed per `local port`. +This matches the usage of BSD-style sockets on which `connect()` has completed. +It's incompatible with many of the applications analyzed. +Multiple peers are typically needed for a variety of reasons, like connectivity checks or multi-region servers. + +This approach would be simplest to implement and specify, especially in the existing C tor implementation. +It also unfortunately has very limited compatibility, and no clear path toward incremental upgrades if we wish to improve compatibility later. + +A simple one-to-one mapping between streams and sockets would preclude the optimizations necessary to address [local port exhaustion](#local-port-exhaustion) risks below. Solutions under this design are possible, but only by decoupling logical protocol-level sockets from the ultimate implementation-level sockets and reintroducing much of the complexity that we attempted to avoid by choosing this design. + +### One Stream per Flow + +One stream **per flow** has also been suggested. +Specifically, Mike Perry brought this up during our conversations about UDP recently and we spent some time analyzing it from a RFC4787 perspective. +This approach has some interesting properties but also hidden complexity that may ultimately make other options more easily applicable. + +This would assign a stream ID to the tuple consisting of at least `(local port, remote address, remote port)`. Additional flags may be included for features like transmit and receive filtering, IPv4/v6 choice, and IP *Don't Fragment*. + +This has advantages in keeping the datagram cells simple, with no additional IDs beyond the existing circuit ID. +It may also have advantages in DoS-prevention and in privacy analysis. + +Stream lifetimes, in this case, would not have any specific meaning other than the lifetime of the ID itself. +The bundle of flows associated with one source port would still all be limited to the lifetime of a Tor circuit, by scoping the source port identifier to be contained within the lifetime of its circuit. + +It would be necessary to allocate a new stream ID any time a new `(local port, remote address, remote port)` tuple is seen. +This would most commonly happen as a result of a first datagram sent to a new peer, coinciding with the establishment of a NAT-style mapping and the possible allocation of a socket on the exit. + +A less common case needs to be considered too: what if the parameter tuple first occurs on the exit side? +We don't yet have a way to allocate stream IDs from either end of a circuit. +This would need to be considered. +One simple solution would be to statically partition the stream ID space into a portion that can be independently allocated by each side. + +When is this exit-originated circuit ID allocation potentially needed? +It is clearly needed when using **address-dependent filtering**. +An incoming datagram from a previously-unseen peer port is expected to be deliverable, and the exit would need to allocate an ID for it. + +Even with the stricter **address and port-dependent filtering** clients may still be exposed to exit-originated circuit IDs if there are mismatches in the lifetime of the filter and the stream. + +This approach thus requires some attention to either correctly allocating stream IDs on both sides of the circuit, or choosing a filtering strategy and filter/mapping lifetime that does not ever leave stream IDs undefined when expecting incoming datagrams. + +### One Stream per Mapping + +One stream **per mapping** is an alternative which attempts to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. + +A mapping would always be allocated from the OP (client) side. +It could explicitly specify a filtering style, if we wish to allow applications to request non-port-dependent filtering for compatibility. +Each datagram within the stream would still need to be tagged with a peer address/port in some way. + +This approach would involve a single new type of stream, and two new messages that pertain to these *mapping* streams: + +- `NEW_UDP_MAPPING` + + - Always client-to-exit. + - Creates a new mapping, with a specified stream ID. + - Succeeds instantly; no reply is expected, early data is ok. + - Externally-visible local port number is arbitrary, and must be determined through interaction with other endpoints. + - Might contain an IP "don't fragment" flag. + - Might contain a requested filtering mode. + - Lifetime is until circuit teardown or `END` message. + +- `UDP_MAPPING_DATAGRAM` + + - Conveys one datagram on a stream previously defined by `NEW_UDP_MAPPING`. + - Includes peer address (IPv4/IPv6) as well as datagram content. + +This puts us in a very similar design space to TURN (RFC8656). +In that protocol, "allocations" are made explicitly on request, and assigned a random relayed port. +TURN also uses its allocations as an opportunity to support a *Don't Fragment* flag. + +The principal disadvantage of this approach is in space overhead, especially the proportional overhead on small datagrams which must each carry a full-size address. + +### Hybrid Mapping and Flow Approach + +We can extend the approach above with an optimization that addresses the undesirable space overhead from redundant address headers. +This uses two new types of stream, in order to have streams **per mapping** and **per flow** at the same time. + +The per-mapping stream remains the sole interface for managing the lifetime of a mapped UDP port. Mappings are created explicitly by the client. As an optimization, within the lifetime of a mapping there may exist some number of *flows*, each assigned their own ID. + +This tries to combine the strengths of both approaches, using the lifetime of one stream to define a mapping and to carry otherwise-unbundled traffic while also allowing additional streams to bundle datagrams that would otherwise have repetitive headers. +It avoids the space overhead of a purely **per mapping** approach and avoids the ID allocation and lifetime complexity introduced with **per flow**. + +This approach takes some inspiration from TURN, where commonly used peers will be defined as a "channel" with an especially short header. +Incoming datagrams with no channel can always be represented in the long form, so TURN never has to allocate channels unexpectedly. + +The implementation here could be a strict superset of the **per mapping** implementation, adding new commands for flows while retaining existing behavior for mappings. There would be a total of four new message types: + +- `NEW_UDP_MAPPING` + + - Same as above. + +- `UDP_MAPPING_DATAGRAM` + + - Same as above. + +- `NEW_UDP_FLOW` + + - Allocates a stream ID as a *flow*, given the ID to be allocated and the ID of its parent *mapping* stream. + - Includes a peer address (IPv4/IPv6). + - The *flow* has a lifetime strictly bounded by the outer *mapping*. It is deleted by an explicit `END` or when the mapping is de-allocated for any reason. + +- `UDP_FLOW_DATAGRAM` + + - Datagram contents only, without address. + - Only appears on *flow* streams. + +We must consider the traffic marking opportunities provided when allowing an exit to represent one incoming datagram as either a *flow* or *mapping* datagram. + +It's possible this traffic injection potential is not worse than the baseline amount of injection potential than every UDP protocol presents. See more on [risks](#risks) below. For this hybrid stream approach specifically, there's a limited mitigation available which allows exits only a bounded amount of leaked information per UDP peer: + +Ideally exits may not choose to send a `UDP_MAPPING_DATAGRAM` when they could have sent a `UDP_FLOW_DATAGRAM`. +Sometimes it is genuinely unclear though: an exit may have received this datagram in-between processing `NEW_UDP_MAPPING` and `NEW_UDP_FLOW`. +A partial mitigation would terminate circuits which send a `UDP_MAPPING_DATAGRAM` for a peer that has already been referenced in a `UDP_FLOW_DATAGRAM`. +The exit is thereby given a one-way gate allowing it to switch from using *mapping* datagrams to using *flow* datagrams at some point, but not to switch back and forth repeatedly. + +Mappings that do not request port-specific filtering may always get unexpected `UDP_MAPPING_DATAGRAM`s. Mappings that do use port-specific filtering could make a flow for their only expected peers, then expect to never see `UDP_MAPPING_DATAGRAM`. + +`NEW_UDP_MAPPING` could have an option requiring that only `UDP_FLOW_DATAGRAM` is to be used, never `UDP_MAPPING_DATAGRAM`. +This would remove the potential for ambiguity, but costs in compatibility as it's no longer possible to implement non-port-specific filtering. + +## Risks + +Any proposed UDP support involves significant risks to user privacy and software maintainability. +This section elaborates some of these risks, so they can be compared against expected benefits. + +### Behavior Regressions + +In some applications it is possible that Tor's implementation of a UDP compatibility layer will cause a regression in the ultimate level of performance or security. + +Performance regressions can occur accidentally due to bugs or compatibility glitches. +They may also occur for more fundamental reasons of protocol layering. +For example, the redundant error correction layers when tunneling QUIC over TCP. +These performance degradations are expected to be minor, but there's some unavoidable risk. + +The risk of severe performance or compatibility regressions may be mitigated by giving users a way to toggle UDP support per-application. + +Privacy and security regressions have more severe consequences and they can be much harder to detect. +There are straightforward downgrades, like WebRTC apps that give up TURN-over-TLS for plaintext TURN-over-UDP. +More subtly, the act of centralizing connection establishment traffic in Tor exit nodes can make users an easier target for other attacks. + +### Bandwidth Usage + +We should expect an increase in overall exit bandwidth requirements due to peer-to-peer file sharing applications. + +Current users attempting to use BitTorrent over Tor are hampered by the lack of UDP compatibility. Interoperability with common file-sharing peers would make Tor more appealing to users with a large and sustained appetite for anonymized bandwidth. + +### Local Port Exhaustion + +Exit routers will have a limited number of local UDP ports. In the most constrained scenario, an exit may have a single IP with 16384 or fewer ephemeral ports available. These ports could each be allocated by one client for an unbounded amount of exclusive use. + +In order to enforce high levels of isolation between different subsequent users of the same local UDP port, we may wish to enforce an extended delay between allocations during which nobody may own the port. Effective isolation requires this timer duration to be greater than any timer encountered on a peer or a NAT. In RFC4787's recommendations a NAT's mapping timer must be longer than 2 minutes. Our timer should ideally be *much* longer than 2 minutes. + +An attacker who allocates ports for only this minimum duration of 2 minutes would need to send 136.5 requests per second to achieve sustained use of all available ports. With multiple simultaneous clients this could easily be done while bypassing per-circuit rate limiting. + +The expanded definition of "Port overlapping" from [RFC7857 section 3](https://datatracker.ietf.org/doc/html/rfc7857#section-3), may form at least a partial mitigation: + +> This document clarifies that this port overlapping behavior may be extended to connections originating from different internal source IP addresses and ports as long as their destinations are different. + +This gives us an opportunity for a vast reduction in the number of required ports and file descriptors. +Exit routers can automatically allocate local ports for use with a specific peer when that peer is first added to the client's filter. + +Due to the general requirements of NAT traversal, UDP applications with any NAT support will always need to communicate with a relatively well known server prior to any attempts at peer-to-peer communication. +This early peer could be an entire application server, or it could be a STUN endpoint. +In any case, the identity of this first peer gives us a hint about the set of all potential peers. + +Within the exit router, each local port will track a separate mapping owner for each peer. +When processing that first outgoing datagram, the exit may choose any local port where the specific peer is not taken. +Subsequent outgoing datagrams on the same port may communicate with a different peer, and there's no guarantee all these future peers will be claimed successfully. + +When is this a problem? +An un-claimable peer represents a case where the exact `(local ip, local port, remote ip, remote port)` tuple is in use already for a different mapping in some other Tor stream. +For example, imagine two clients are running different types of telecom apps which are nevertheless inter-compatible and capable of both calling the same peers. +Alternatively, consider the same app but with servers in several regions. +The two apps will begin by communicating with different sets of peers, due to different application servers and different bundled STUN servers. +This is our hint that it's likely appropriate to overlap their local port allocations. +At this point, both of these applications may be successfully sharing a `(local ip, local port)` tuple on the exit. +As soon as one of these apps calls a peer with some `(remote ip, remote port)`, the other app will be unable to contact that specific peer. + +The lack of connectivity may seem like a niche inconvenience, and perhaps that is the extent of the issue. +It seems likely this heuristic could result in a problematic information disclosure under some circumstances, and it deserves closer study. + +### Application Fingerprinting + +UDP applications present an increased surface of plaintext data that may be available for user fingerprinting by malicious exits. + +Exposed values can include short-lived identifiers like STUN usernames. +Typically it will also be possible to determine what type of software is in use, and maybe what version of that software. + +Short-lived identifiers are still quite valuable to attackers, because they may reliably track application sessions across changes to the Tor exit. If longer-lived identifiers exist for any reason, that of course provides a powerful tool for call metadata gathering. + +### Peer-to-Peer Metadata Collection + +One of our goals was to achieve the compatibility and perhaps performance benefits of allowing "peer-to-peer" (in our case really exit-to-exit) UDP connections. +We expect this to enable the subset of applications that lack a fallback path which loops traffic through an app-provided server. + +This goal may be at odds with our privacy requirements. At minimum, a pool of malicious exit nodes could passively collect metadata about these connections as a noisy proxy for call metadata. + +Any additional signals like [application fingerprints](#application-fingerprinting) or [injected](#traffic-injection) markers may be used to enrich this metadata graph, possibly tracking users across sessions or across changes to the tunnel endpoint. + +### Interaction with Other Networks + +Any application using an ICE-like interactive connection establishment scheme will easily leak information across network boundaries if it ever has access to multiple networks at once. + +In applications that are not privacy conscious this is often desired behavior. For example, a video call to someone in your household may typically transit directly over Wifi, decreasing service costs and improving latency. This implies that some type of local identifier accompanies the call signaling info, allowing the devices to find each other's LAN address. + +Privacy-preserving solutions for this use case are still an active area of standardization effort. The [IETF draft on mDNS ICE candidates](https://www.ietf.org/archive/id/draft-ietf-mmusic-mdns-ice-candidates-02.html) proposes one way to accomplish this by generating short-lived unique IDs which are only useful to peers with physical access to the same mDNS services. + +Without special attention to privacy, the typical implementation is to share all available IP addresses and to initiate simultaneous connectivity tests using any IP pairs which cannot be trivially discarded. This applies to ICE as specified, but also to any proprietary protocol which operates in the same design space as ICE. This has multiple issues in a privacy-conscious environment: The IP address disclosures alone can be fatal to anonymity under common threat models. Even if meaningful IP addresses are not disclosed, the timing correlation from connectivity checks can provide confirmation beacons that alert an attacker to some connection between a LAN user and a Tor exit. + +### Traffic Injection + +Some forms of UDP support would have obvious and severe traffic injection vulnerabilities. For example, the very permissive *endpoint-independent filtering* strategy would allow any host on the internet to send datagrams in bulk to all available local ports on a Tor exit in order to map that traffic's effect on any guards they control. + +Any vehicle for malicious parties to mark traffic can be abused to de-anonymize users. Even if there is a more restrictive filtering policy, UDP's lack of sequence numbers make header spoofing attacks considerably easier than in TCP. Third parties capable of routing datagrams to an exit with a spoofed source address could bypass filtering when the communicating parties are known or can be guessed. For example, a malicious superuser at an ISP without egress filtering could send packets with the source IPs set to various common DNS servers and application STUN/TURN servers. If a specific application is being targeted, only the exit node and local port numbers need to be guessed by brute force. + +In case of malicious exit relays, whole datagrams can be inserted and dropped, and datagrams may be padded with additional data, both without any specific knowledge of the application protocol. With specific protocol insights, a malicious relay may make arbitrary edits to plaintext data. + +Of particular interest is the plaintext STUN, TURN, and ICE traffic used by most WebRTC apps. These applications rely on higher-level protocols (SRTP, DTLS) to provide end-to-end encryption and authentication. A compromise at the connection establishment layer would not violate application-level end-to-end security requirements, making it outside the threat model of WebRTC but very much still a concern for Tor. + +These attacks are not fully unique to the proposed UDP support, but UDP may increase exposure. In cases where the application already has a fallback using TURN-over-TLS, the proposal is a clear regression over previous behaviors. Even when comparing plaintext to plaintext, there may be a serious downside to centralizing all connection establishment traffic through a small number of exit IPs. Depending on your threat model, it could very well be more private to allow the UDP traffic to bypass Tor entirely. + +### Malicious Outgoing Traffic + +We can expect UDP compatibility in Tor will give malicious actors additional opportunities to transmit unwanted traffic. + +In general, exit abuse will need to be filtered administratively somehow. +This is not unique to UDP support, and exit relay administration typically involves some type of filtering response tooling that falls outside the scope of Tor itself. + +Exit administrators may choose to modify their exit policy, or to silently drop problematic traffic. +Silent dropping is discouraged in most cases, as Tor prioritizes the accuracy of an exit's advertised policy. +Detailed exit policies have a significant space overhead in the overall Tor consensus document, but it's still seen as a valuable resource for clients during circuit establishment. + +Exit policy filtering may be less useful in UDP than with TCP due to the inconvenient latency spike when establishing a new [tunnel](#tunneling). +Applications that are sensitive to RTT measurements made during connection establishment may fail entirely when the tunnel cannot be pre-built. + +This section lists a few potential hazards, but the real-world impact may be hard to predict owing to a diversity of custom UDP protocols implemented across the internet. + +- Amplification attacks against arbitrary targets + + These are possible only in limited circumstances where the protocol allows an arbitrary reply address, like SIP. +The peer is often at fault for having an overly permissive configuration. +Nevertheless, any of these *easy* amplification targets can be exploited from Tor with little consequence, creating a nuisance for the ultimate target and for exit operators. + +- Amplification attacks against an exit relay + + An amplification peer which doesn't allow arbitrary destinations can still be used to attack the exit relay itself or other users of that relay. +This is essentially the same attack that is possible against any NAT the attacker is behind. + +- Malicious fragmented traffic + + If we allow sending large UDP datagrams over IPv4 without the *Don't Fragment* flag set, we allow attackers to generate fragmented IP datagrams. +This is not itself a problem, but it has historically been a common source of inconsistencies in firewall behavior. + +- Excessive sends to an uninterested peer + + Whereas TCP mandates a successful handshake, UDP will happily send unlimited amounts of traffic to a peer that has never responded. +To prevent denial of service attacks we have an opportunity and perhaps a responsibility to define our supported subset of UDP to include true bidirectional traffic but exclude continued sends to peers who do not respond. + + See also [RFC7675](https://www.rfc-editor.org/rfc/rfc7675.html) and STUN's concept of "Send consent". + +- Excessive number of peers + + We may want to place conservative limits on the maximum number of peers per mapping or per circuit, in order to make bulk scanning of UDP port space less convenient. + + The limit would need to be on peers, not stream IDs as we presently do for TCP. +In this proposal stream IDs are not necessarily meaningful except as a representational choice made by clients. + +## Next Steps + +At this point we have perhaps too many possibilities for how to proceed. We could integrate UDP quite closely with Tor itself or not at all. + +Choosing a route forward is both a risk/benefit tradeoff and a guess about the future of Tor. If we expect that some future version of Tor will provide its own datagram transport, we should plot a course aiming in that direction. If not, we might be better served by keeping compatibility features confined to separate protocol layers when that's practical. + +### Requiring a Long-Term Datagram Plan + +If we choose to add new long-term maintenance burdens to our protocol stack, we should ensure they serve our long-term goals for UDP adoption as well as these shorter-term application compatibility goals. + +This work so far has been done with the assumption that end-to-end datagram support is out of scope. If we intend to proceed down any path which encodes a datagram-specific protocol into Tor proper, we should prioritize additional protocol research and standardization work. + +### Alternatively, Modular Application-Level Support + +Without a clear way to implement fully generic UDP support, we're left with application-level goals. +Different applications may have contrasting needs, and we can only achieve high levels of both compatibility and privacy by delegating some choices to app authors or per-app VPN settings. + +This points to an alternative in which UDP support is excluded from Tor as much as possible while still supporting application requirements. +For example, this could motivate the [TURN Encapsulated in a Tor Stream](#turn-encapsulated-in-a-tor-stream) design, or even simpler designs where TURN servers are maintained independently from the Tor exit infrastructure. + +The next prototyping step we would need at this stage is a version of `onionmasq` extended to support NAT-style UDP mappings using TURN allocations provided through TURN-over-TCP or TURN-over-TLS. +This can be a platform for further application compatibility experiments. +It could potentially become a low-cost minimal implementation of UDP application compatibility, serving to assess which remaining user needs are still unmet. diff --git a/proposals/XXX-udp-app-support.md b/proposals/XXX-udp-app-support.md deleted file mode 100644 index 55f2192..0000000 --- a/proposals/XXX-udp-app-support.md +++ /dev/null @@ -1,715 +0,0 @@ -# UDP Application Support in Tor - -```text -Filename: XXX-udp-app-support.md -Title: UDP Application Support in Tor -Author: Micah Elizabeth Scott -Created: December 2023 -Status: Draft -``` - -## Table of Contents - -- [Introduction](#introduction) - - [History](#history) - - [Scope](#scope) -- [UDP Traffic Models](#udp-traffic-models) - - [User Datagram Protocol (RFC768)](#user-datagram-protocol-rfc768) - - [Socket Layer](#socket-layer) - - [Network Address Translation (NAT)](#network-address-translation-nat) - - [Mapping and Filtering Behaviors](#mapping-and-filtering-behaviors) - - [Common Protocols](#common-protocols) - - [QUIC](#quic) - - [WebRTC](#webrtc) - - [Common Applications](#common-applications) -- [Overview of Possible Solutions](#overview-of-possible-solutions) - - [Datagram Routing](#datagram-routing) - - [Intentional UDP Leak](#intentional-udp-leak) - - [3rd Party Implementations](#3rd-party-implementations) - - [Future Work on Tor](#future-work-on-tor) - - [Tunneling](#tunneling) - - [TURN Encapsulated in a Tor Stream](#turn-encapsulated-in-a-tor-stream) - - [Tor Stream Tunnel to an Exit](#tor-stream-tunnel-to-an-exit) - - [Tor Stream Tunnel to a Rendezvous Point](#tor-stream-tunnel-to-a-rendezvous-point) -- [Specific Designs Using Tor Streams](#specific-designs-using-tor-streams) - - [One Stream per Tunnel](#one-stream-per-tunnel) - - [One Stream per Socket](#one-stream-per-socket) - - [One Stream per Flow](#one-stream-per-flow) - - [One Stream per Mapping](#one-stream-per-socket) - - [Hybrid Mapping and Flow Approach](#hybrid-mapping-and-flow-approach) -- [Risks](#risks) - - [Behavior Regressions](#behavior-regressions) - - [Bandwidth Usage](#bandwidth-usage) - - [Local Port Exhaustion](#local-port-exhaustion) - - [Application Fingerprinting](#application-fingerprinting) - - [Peer-to-Peer Metadata Collection](#peer-to-peer-metadata-collection) - - [Interaction with Other Networks](#interaction-with-other-networks) - - [Traffic Injection](#traffic-injection) - - [Malicious Outgoing Traffic](#malicious-outgoing-traffic) -- [Next Steps](#next-steps) - - [Requiring a Long-Term Datagram Plan](#requiring-a-long-term-datagram-plan) - - [Alternatively, Modular Application-Level Support](#alternatively-modular-application-level-support) - -## Introduction - -This proposal takes a fresh look at the problem of implementing support in Tor for applications which require UDP/IP communication. - -This work is being done with the sponsorship and goals of the [Tor VPN Client for Android project](https://gitlab.torproject.org/groups/tpo/-/milestones/32). - -The proposal begins with a summary of previous work and the specific problem space being addressed. -This leads into an analysis of possible solutions, and finally some possible conclusions about the available development opportunities. - -### History - -There have already been multiple attempts over Tor's history to define some type of UDP extension. - -#### 2006 - -[Proposal 100](https://spec.torproject.org/proposals/100-tor-spec-udp.html) by Marc Liberatore in 2006 suggested a way to "add support for tunneling unreliable datagrams through tor with as few modifications to the protocol as possible." -This proposal suggested extending the existing TLS+TCP protocol with a new DTLS+UDP link mode. -The focus of this work was on a potential way to support unreliable traffic, not necessarily on UDP itself or on UDP applications. - -In proposal 100, a Tor *stream* is used for one pairing of local and remote address and port, copying the technique used by Tor for TCP. -This works for some types of UDP applications, but it's broken by common behaviors like ICE connectivity checks, NAT traversal attempts, or using multiple servers via the same socket. - -No additional large-message fragmentation protocol is defined, so the MTU in proposal 100 is limited to what fits in a single Tor cell. -This value is much too small for most applications. - -It's possible these UDP protocol details would have been elaborated during design, but the proposal hit a snag elsewhere: -there was no agreement on a way to avoid facilitating new attacks against anonymity. - -#### 2018 - -In 2018, Nick Mathewson and Mike Perry wrote a -[summary of the side-channel issues with unreliable transports for Tor](https://research.torproject.org/techreports/side-channel-analysis-2018-11-27.pdf). - -The focus of this document is on the communication between Tor relays, but there is considerable overlap between the attack space explored here and the potential risks of any application-level UDP support. -Attacks that are described here, such as drops and injections, may be applied by malicious exits or some types of third parties even in an implementation using only present-day reliable Tor transports. - -#### 2020 - -[Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020 introduced a simpler UDP encapsulation design which had similar stream mapping properties as in proposal 100, but with the unreliable transport omitted. Datagrams are tunneled over a new type of Tor stream using a new type of Tor message. -As a prerequisite, it depends on [proposal 319](https://spec.torproject.org/proposals/319-wide-everything.html) to support messages that may be larger than a cell, extending the MTU to support arbitrarily large UDP datagrams. - -In proposal 339 the property of binding a stream both to a local port and to a remote peer is described in UNIX-style terminology as a *connected socket*. This idea is explored below using alternate terminology from [RFC4787](https://www.rfc-editor.org/rfc/rfc4787), *NAT behavior requirements for UDP*. -The single-peer *connected socket* behavior would be referred to as an *endpoint-dependent mapping* in RFC4787. -This type works fine for client/server apps but precludes the use of NAT traversal for peer-to-peer transfer. - -### Scope - -This proposal aims to allow Tor applications and Tor-based VPNs to provide compatibility with applications that require UDP/IP communications. - -We don't have a specific list of applications that must be supported, but we are currently aiming for broad support of popular applications while still respecting and referencing all applicable Internet standards documents. - -Changes to the structure of the Tor network are out of scope, as are most performance optimizations. We expect to rely on common optimizations to the performance of Tor circuits, rather than looking to make specific changes that optimize for unreliable datagram transmission. - -This document will briefly discuss UDP for onion services below. -It's worth planning for this as a way to evaluate the future design space, but in practice we are not aiming for UDP onion services yet. -This will require changes to most applications that want to use it, as it implies that any media negotiations will need to understand onion addressing in addition to IPv4 and IPv6. - -The allowed subset of UDP traffic is not subject to a single rigid definition. -There are several options discussed below using the RFC4787 framework. - -We require support for DNS clients. Tor currently only supports a limited subset of DNS queries, and it's desirable to support more. This will be analyzed in detail as an application below. DNS is one of very few applications that still rely on fragmented UDP datagrams, though this may not be relevant for us since only servers typically need to control the production of fragments. - -We require support for voice/video telecommunications apps. Even without an underlying transport that supports unreliable datagrams, we expect a tunnel to provide a usable level of compatibility. This design space is very similar to the TURN ([RFC8656](https://www.rfc-editor.org/rfc/rfc8656)) specification, already used very widely for compatibility with networks that filter UDP. See the analysis of specific applications below. - -We require support for peer-to-peer UDP transfer without additional relaying, in apps that use ICE ([RFC8445](https://www.rfc-editor.org/rfc/rfc8445)) or similar connection establishment techniques. -Video calls between two Tor users should transit directly between two exit nodes. -This requires that allocated UDP ports can each communicate with multiple peers: -*endpoint-independent mapping* as described by RFC4787. - -We do not plan to support applications which accept incoming datagrams from previously-unknown peers, for example a DNS server hosted via Tor. -RFC4787 calls this *endpoint-independent filtering*. -It's unnecessary for running peer-to-peer apps, and it facilitates an extremely easy [traffic injection](#traffic-injection) attack. - -## UDP Traffic Models - -To better specify the role of a UDP extension for Tor, this section explores a few frameworks for describing noteworthy subsets of UDP traffic. - -### User Datagram Protocol (RFC768) - -The "User Interface" suggested by [RFC768](https://www.rfc-editor.org/rfc/rfc768) for the protocol is a rough sketch, suggesting that applications have some way to allocate a local port for receiving datagrams and to transmit datagrams with arbitrary headers. - -Despite UDP's simplicity as an application of IP, we do need to be aware of IP features that are typically hidden by TCP's abstraction. - -UDP applications typically try to obtain an awareness of the path MTU, using some type of path MTU discovery (PMTUD) algorithm. -On IPv4, this requires sending packets with the "Don't Fragment" flag set, and measuring when those packets are lost or when ICMP "Fragmentation Needed" replies are seen. - -Note that many applications have their own requirements for path MTU. For example, QUIC and common implementations of WebRTC require an MTU no smaller than 1200 bytes, but they can discover larger MTUs when available. - -### Socket Layer - -In practice the straightforward "User Interface" from RFC768, capable of arbitrary local address, is only available to privileged users. - -BSD-style sockets support UDP via `SOCK_DGRAM`. -UDP is a stateless protocol, but sockets do have state. -Each socket is bound, either explicitly with `bind()` or automatically, to a source IP and port. - -At the API level, a socket is said to be *connected* to a remote `(address, port)` if that address is the default destination. -A *connected* socket will also filter out incoming packets with source addresses different from this default destination. -A socket is considered *unconnected* if `connect()` has not been called. -These sockets have no default destination, and they accept datagrams from any source. - -There does not need to be any particular mapping between the lifetime of these application sockets and any higher-level "connection" the application establishes. -It's better to think of one socket as one allocated source port. -A typical application may allocate only a single port (one socket) for talking to many peers. -Every datagram sent or received on the socket may have a different peer address. - -### Network Address Translation (NAT) - -Much of the real-world complexity in applying UDP comes from defining strategies to detect and overcome the effects of NAT. -As a result, an intimidating quantity of IETF documentation has been written on NAT behavior and on strategies for NAT traversal. - -[RFC4787](https://www.rfc-editor.org/rfc/rfc4787.html) and later [RFC7857](https://www.rfc-editor.org/rfc/rfc7857.html) offer best practices for implementing NAT. These are sometimes referred to as the [BEHAVE-WG](https://datatracker.ietf.org/wg/behave/about/) recommendations, based on the "Behavior Engineering for Hindrance Avoidance" working group behind them. - -[RFC6888](https://www.rfc-editor.org/rfc/rfc6888.html) makes additional recommendations for "carrier grade" NAT systems, where small pools of IP addresses are shared among a much larger number of subscribers. - -[RFC8445](https://www.rfc-editor.org/rfc/rfc8445.html) describes the Interactive Connectivity Establishment (ICE) protocol, which has become a common and recommended application-level technique for building peer-to-peer applications that work through NAT. - -There are multiple fundamental technical issues that NAT presents: - -1. NAT must be stateful in order to route replies back to the correct source. - - This directly conflicts with the stateless nature of UDP itself. - The NAT's mapping lifetime, determined by a timer, will not necessarily match the lifetime of the application-level connection. - This necessitates keep-alive packets in some protocols. - Protocols that allow their binding to expire may be open to a NAT rebinding attack, when a different party acquires access to the NAT's port allocation. - -2. Applications no longer know an address they can be reached at without outside help. - - Chosen port numbers may or may not be used by the NAT. - The effective IP address and port are not knowable without observing from an outside peer. - -3. Filtering and mapping approaches both vary, and it's not generally possible to establish a connection without interactive probing. - - This is the reason ICE exists, but it's also a possible anonymity hazard. - This risk is explored a bit further below in the context of [interaction with other networks](#interaction-with-other-networks). - -We can use the constraints of NAT both to understand application behavior and as an opportunity to model Tor's behavior as a type of NAT. -In fact, Tor's many exit nodes already share similarity with some types of carrier-grade NAT. -Applications will need to assume very little about the IP address their outbound UDP originates on, and we can use that to our advantage in implementing UDP for Tor. - -This body of work is invaluable for understanding the scope of the problem and for defining common terminology. -Let's take inspiration from these documents while also keeping in mind that the analogy between Tor and a NAT is imperfect. -For example, in analyzing Tor as a type of carrier-grade NAT, we may consider the "pooling behavior" defined in RFC4787: the choice of which external addresses map to an internal address. -Tor by necessity must carefully limit how predictable these mappings can ever be, to preserve its anonymity properties. -A literal application of RFC6888 would find trouble in REQ-2 and REQ-9, as well as the various per-subscriber limiting requirements. - -#### Mapping and Filtering Behaviors - -RFC4787 defines a framework for understanding the behavior of NAT by analyzing both its "mapping" and "filtering" behavior separately. -Mappings are the NAT's unit of state tracking. -Filters are layered on top of mappings, potentially rejecting incoming datagrams that don't match an already-expected address. -Both RFC4787 and the demands of peer-to-peer applications make a good case for always using an **Endpoint-Independent Mapping**. - -Choice of filtering strategy is left open by the BEHAVE-WG recommendations. -RFC4787 does not make one single recommendation for all circumstances, -instead it defines three behavior options with different properties: - -- **Endpoint-Independent Filtering** allows incoming datagrams from any peer once a mapping has been established. - - RFC4787 recommends this approach, with the concession that it may not be ideal for all security requirements. - - This technique cannot be safely applied in the context of Tor. - It makes [traffic injection](#traffic-injection) attacks possible from any source address, provided you can guess the UDP port number used at an exit. - It also makes possible clear-net hosting of UDP servers using an exit node's IP, which may have undesirable abuse properties. - - This permissive filter is also incompatible with our proposed mitigation to [local port exhaustion](#local-port-exhaustion) on exit relays. Even with per-circuit rate limiting, an attacker could trivially overwhelm the local port capacity of all combined UDP-capable Tor exits. - - It is still common for present-day applications to *prefer* endpoint-independent filtering, as it allows incoming connections from peers which cannot use STUN or a similar address fixing protocol. - Choosing endpoint-independent filtering would have *some* compatibility benefit, but among modern protocols which use ICE and STUN there would be no improvement. - The cost, on the other hand, would be an uncomfortably low-cost traffic injection attack and additional risks toward exit nodes. - -- **Address-Dependent Filtering** - - This is a permitted alternative according to RFC4787, in which incoming datagrams are allowed from only IP addresses we have previously sent to, but any port on that IP may be the sender. - - The intended benefits of this approach versus the port-dependent filtering below are unclear, and may no longer be relevant. In theory they would be: - - - To support a class of applications that rely on, for a single local port, multiple remote ports achieving filter acceptance status when only one of those ports has been sent a datagram. - We are currently lacking examples of applications in this category. - Any application using ICE should be outside this category, as each port would have its own connectivity check datagrams exchanged in each direction. - - - REQ-8 in RFC4787 claims the existence of a scenario in which this approach facilitates ICE connections with a remote peer that disregards REQ-1 (the peer does not use **Endpoint-Independent Mapping**). It is not clear that this claim is still relevant. - - One security hazard of address-dependent and non-port-dependent filtering, identified in RFC4787, is that a peer on a NAT effectively negates the security benefits of this host filtering. - In fact, this should raise additional red flags as applied to either Tor or carrier grade NAT. - If supporting peer-to-peer applications, it should be commonplace to establish UDP flows between two Tor exit nodes. - When this takes place, non-port-dependent filtering would then allow anyone on Tor to connect via those same nodes and perform traffic injection. - The resulting security properties really become uncomfortably similar to endpoint-independent filtering. - -- **Address- and Port-Dependent Filtering** - - This is the strictest variety of filtering, and it is an allowed alternative under RFC4787. - It provides opportunities for increased security and opportunities for reduced compatibility, both of which in practice may depend on other factors. - - For every application we've analyzed so far, port-dependent filtering is not a problem. - Usage of ICE will open all required filters during the connectivity check phase. - - This is the only type of filtering that provides any barrier at all between cross-circuit traffic injection when the communicating parties are known. - -RFC4787 recommends that filtering style be configurable. -We would like to implement that advice, but only to the extent it can be done safely and meaningfully in the context of an anonymity system. -When possible, it would provide additional compatibility at no mandatory cost to allow applications to optionally request **Address-Dependent Filtering**. -Otherwise, **Address- and Port-Dependent Filtering** is the most appropriate default setting. - -### Common Protocols - -Applications that want to use UDP are increasingly making use of higher-level protocols to avoid creating bespoke solutions for problems like NAT traversal, connection establishment, and reliable delivery. - -This section looks at how these protocols affect Tor's UDP traffic requirements. - -#### QUIC - -[RFC9000](https://www.rfc-editor.org/rfc/rfc9000.html) defines QUIC, a multiplexed secure point-to-point protocol which supports reliable and unreliable delivery. The most common use is as an optional HTTP replacement, especially among Google services. - -QUIC does not normally try to traverse NAT; as an HTTP replacement, the server is expected to have an address reachable without any prior connection setup. - -QUIC provides its own flexible connection lifetimes which may outlive individual network links or NAT mappings. -The intention is to provide transparent roaming as mobile users change networks. -This automated path discovery opens additional opportunities for malicious traffic, for which the RFC also offers mitigations. See *path validation* in section `8.2`, and the additional mitigations from section `9.3`. - -When QUIC is used as an optional upgrade path, we must compare any proposed UDP support against the baseline of a non-upgraded original connection. -In these cases the goal is not a compatibility enhancement but an avoidance of regression. - -In cases where QUIC is used as a primary protocol without TCP fallback, UDP compatibility will be vital. -These applications are currently niche but we expect they may rise in popularity. - -#### WebRTC - -WebRTC is a large collection of protocols tuned to work together for media transport and NAT traversal. -It is increasingly common, both for browser-based telephony and for peer-to-peer data transfer. -Non-browser apps often implement WebRTC as well, for example using [`libwebrtc`](https://github.com/webrtc-sdk/libwebrtc). -Even non-WebRTC apps sometimes have significant overlaps in their technology stacks, due to the independent history of ICE, RTP, and SDP adoption. - -Of particular importance to us, WebRTC uses the Interactive Connection Establishment (ICE) protocol to establish a bidirectional channel between endpoints that may or may not be behind a NAT with unknown configuration. - -Any generalized solution to connection establishment, like ICE, will require sending connectivity test probes. These have an inherent hazard to anonymity: assuming no delays are inserted intentionally, the result is a broadcast of similar traffic across all available network interfaces. This could form a convenient correlation beacon for an attacker attempting to de-anonymize users who use WebRTC over a Tor VPN. This is the risk enumerated below as [*interaction with other networks*](#interaction-with-other-networks). - -See -[RFC8825](https://www.rfc-editor.org/rfc/rfc8825.html) *Overview: Real-Time Protocols for Browser-Based Applications*, -[RFC8445](https://www.rfc-editor.org/rfc/rfc8445.html) *Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal*, -[RFC8838](https://www.rfc-editor.org/rfc/rfc8838.html) *Trickle ICE: Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (ICE) Protocol*, -[RFC5389](https://www.rfc-editor.org/rfc/rfc5389.html) *Session Traversal Utilities for NAT (STUN)*, -and others. - -### Common Applications - -With applications exhibiting such a wide variety of behaviors, how do we know what to expect from a good implementation? -How do we know which compatibility decisions will be most important to users? -For this it's helpful to look at specific application behaviors. -This is a best-effort analysis conducted at a point in time. -It's not meant to be a definitive reference, think of it as a site survey taken before planning a building. - -In alphabetical order: - -| Application | Type | Protocol features | Current behavior | Expected outcome | -| ---------------------- | -------------- | --------------------------------- | ------------------ | -------------------------------------------------- | -| BitTorrent | File sharing | Many peers per local port | Fails without UDP | Works, new source of nuisance traffic | -| BigBlueButton | Telecom | WebRTC, TURN, TURN-over-TLS | Works | Slight latency improvement | -| Discord | Telecom | Proprietary, client/server | Fails without UDP | Starts working | -| DNS | Infrastructure | Might want IP fragmentation | Limited | Full DNS support, for better and worse | -| FaceTime | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | -| Google Meet | Telecom | STUN/TURN, TURN-over-TCP | Works | Slight latency improvement | -| Jitsi (meet.jit.si) | Telecom | WebRTC, TURN-over-TLS, Cloudflare | Fails on Tor | No change, problem does not appear UDP-related | -| Jitsi (docker-compose) | Telecom | WebRTC, centralized STUN only | Fails without UDP | Starts working | -| Linphone | Telecom (SIP) | SIP-over-TLS, STUN, TURN | Fails without UDP | Starts working | -| Signal | Telecom | WebRTC, TURN, TURN-over-TCP | Works | Slight latency improvement | -| Skype | Telecom | P2P, STUN, TURN-over-TLS | Works | Slight latency improvement | -| WhatsApp | Telecom | STUN, TURN-over-TCP. Multi server | Works | Slight latency improvement | -| WiFi Calling | Telecom | IPsec tunnel | Out of scope | Still out of scope | -| Zoom | Telecom | client/server or P2P, UDP/TCP | Works | Slight latency improvement | - -## Overview of Possible Solutions - -This section starts to examine different high-level implementation techniques we could adopt. -Broadly they can be split into *datagram routing* and *tunneling*. - -### Datagram Routing - -These approaches seek to use a network that can directly route datagrams from place to place. These approaches are the most obviously suitable for implementing UDP, but they also form the widest departure from classic Tor. - -#### Intentional UDP Leak - -The simplest approach would be to allow UDP traffic to bypass the anonymity layer. This is an unacceptable loss of anonymity in many cases, given that the client's real IP address is made visible to web application providers. - -In other cases, this is an acceptable or even preferable approach. For example, VPN users may be more concerned with achieving censorship-resistant network connectivity than hiding personal identifiers from application vendors. - -In threat models where application vendors are more trustworthy than the least trustworthy Tor exits, it may be more appropriate to allow direct peer-to-peer connections than to trust Tor exits with unencrypted connection establishment traffic. - -#### 3rd Party Implementations - -Another option would be to use an unrelated anonymizer system for datagram traffic. It's not clear that a suitable system already exists. I2P provides a technical solution for routing anonymized datagrams, but not a Tor-style infrastructure of exit node operators. - -This points to the key weakness of relying on a separate network for UDP: Tor has an especially well-developed community of volunteers running relays. Any UDP solution that is inconvenient for relay operators has little chance of adoption. - -#### Future Work on Tor - -There may be room for future changes to Tor which allow it to somehow transfer and route datagrams directly, without a separate process of establishing circuits and tunnels. -If this is practical it may prove to be the simplest and highest performance route to achieving high quality UDP support in the long term. -A specific design is out of the scope of this document. - -It is worth thinking early about how we can facilitate combinations of approaches. -Even without bringing any new network configurations to Tor, achieving interoperable support for both exit nodes and onion services in a Tor UDP implementation requires some attention to how multiple UDP providers can share protocol responsibilities. -This may warrant the introduction of some additional routing layer. - -### Tunneling - -The approaches in this section add a new construct which does not exist in UDP itself: a point-to-point tunnel between clients and some other location at which they establish the capability to send and receive UDP datagrams. - -Any tunneling approach requires some way to discover tunnel endpoints. -For the best usability and adoption this should come as an extension of Tor's existing process for distributing consensus and representing exit policy. - -In practice, exit policies for UDP will have limited practical amounts of diversity. -VPN implementations will need to know ahead of time which tunnel circuits to build, or they will suffer a significant spike in latency for the first outgoing datagram to a new peer. -Additionally, it's common for UDP port numbers to be randomly assigned. -This would make highly specific Tor exit policies even less useful and even higher overhead than they are with TCP. - -#### TURN Encapsulated in a Tor Stream - -The scope of this tunnel is quite similar to the existing TURN relays, used commonly by WebRTC applications to implement fallbacks for clients who cannot find a more direct connection path. - -TURN is defined by [RFC8656](https://www.rfc-editor.org/rfc/rfc8656) as a set of extensions built on the framework from STUN in [RFC8489](https://www.rfc-editor.org/rfc/rfc8489.html). The capabilities are a good match for our needs, offering clients the ability to encapsulate UDP datagrams within a TCP stream, and to allocate local port mappings on the server. - -TURN was designed to be a set of modular and extensible pieces, which might be too distant from Tor's design philosophy of providing single canonical representations. Any adoption of TURN will need to consider the potential for malicious implementations to mark traffic, facilitating de-anonymization attacks. - -TURN has a popular embeddable C-language implementation, [coturn](https://github.com/coturn/coturn), which may be suitable for including alongside or inside C tor. - -#### Tor Stream Tunnel to an Exit - -Most of the discussion on UDP implementation in Tor so far has assumed this approach. Essentially it's the same strategy as TCP exits, but for UDP. When the OP initializes support for UDP, it pre-builds circuits to exits that support required UDP exit policies. These pre-built circuits can then be used as tunnels for UDP datagrams. - -Within this overall approach, there are various ways to assign Tor *streams* for the UDP traffic. This will be considered below. - -#### Tor Stream Tunnel to a Rendezvous Point - -To implement onion services which advertise UDP ports, we can use additional tunnels. -A new type of tunnel could end at a rendezvous point rather than an exit node. -Clients could establish the ability to allocate a temporary virtual datagram mailbox at these rendezvous nodes. - -This leaves more open questions about how outgoing traffic is routed, and which addressing format would be used for the datagram mailbox. -The most immediate challenge in UDP rendezvous would then become application support. Protocols like STUN and ICE deal directly with IPv4 and IPv6 formats in order to advertise a reachable address to their peer. Supporting onion services in WebRTC would require protocol extensions and software modifications for STUN, TURN, ICE, and SDP at minimum. - -UDP-like rendezvous extensions would have limited meaning unless they form part of a long-term strategy to forward datagrams in some new way for enhanced performance or compatibility. Otherwise, application authors might as well stick with Tor's existing reliable circuit rendezvous functionality. - -## Specific Designs Using Tor Streams - -Let's look more closely at Tor *streams*, the multiplexing layer right below circuits. - -Streams have an opaque 16-bit identifier, allocated from the onion proxy (OP) endpoint. -Stream lifetimes are subject to some slight ambiguity still in the Tor spec. -They are always allocated from the OP end but may be destroyed asynchronously by either circuit endpoint. - -We have an opportunity to use this additional existing multiplexing layer to serve a useful function in the new protocol, or we can opt to interact with streams as little as possible in order to keep the protocol features more orthogonal. - -### One Stream per Tunnel - -The fewest new streams would be a single stream for all of UDP. This is the result when using an off-the-shelf stream oriented protocol like TURN-over-TCP as our UDP proxy. - -This approach would require only a single new Tor message type: - -- `CONNECT_TURN` - - - Establish a stream as a connection to the exit relay's built-in (or configured) TURN server. - - This would logically be a TURN-over-TCP connection, though it does not need to correspond to any real TCP socket if the TURN server is implemented in-process with tor. - -Note that RFC8656 requires authentication before data can be relayed, which is a good default best practice for the internet perhaps but is the opposite of what Tor is trying to do. We would either deviate from the specification to relax this auth requirement, or provide a way for clients to discover credentials: perhaps by fixing them ahead of time or by including them in the relay descriptor. - -### One Stream per Socket - -One stream **per socket** was the approach suggested in [Proposal 339](https://spec.torproject.org/proposals/339-udp-over-tor.html) by Nick Mathewson in 2020. - -In proposal 339, there would be one new type of stream and three new message types: `CONNECT_UDP`, `CONNECTED_UDP`, and `DATAGRAM`. - -Each stream's lifetime would match the lifetime of a local port allocation. -There would be a single peer `(remote address, remote port)` allowed per `local port`. -This matches the usage of BSD-style sockets on which `connect()` has completed. -It's incompatible with many of the applications analyzed. -Multiple peers are typically needed for a variety of reasons, like connectivity checks or multi-region servers. - -This approach would be simplest to implement and specify, especially in the existing C tor implementation. -It also unfortunately has very limited compatibility, and no clear path toward incremental upgrades if we wish to improve compatibility later. - -A simple one-to-one mapping between streams and sockets would preclude the optimizations necessary to address [local port exhaustion](#local-port-exhaustion) risks below. Solutions under this design are possible, but only by decoupling logical protocol-level sockets from the ultimate implementation-level sockets and reintroducing much of the complexity that we attempted to avoid by choosing this design. - -### One Stream per Flow - -One stream **per flow** has also been suggested. -Specifically, Mike Perry brought this up during our conversations about UDP recently and we spent some time analyzing it from a RFC4787 perspective. -This approach has some interesting properties but also hidden complexity that may ultimately make other options more easily applicable. - -This would assign a stream ID to the tuple consisting of at least `(local port, remote address, remote port)`. Additional flags may be included for features like transmit and receive filtering, IPv4/v6 choice, and IP *Don't Fragment*. - -This has advantages in keeping the datagram cells simple, with no additional IDs beyond the existing circuit ID. -It may also have advantages in DoS-prevention and in privacy analysis. - -Stream lifetimes, in this case, would not have any specific meaning other than the lifetime of the ID itself. -The bundle of flows associated with one source port would still all be limited to the lifetime of a Tor circuit, by scoping the source port identifier to be contained within the lifetime of its circuit. - -It would be necessary to allocate a new stream ID any time a new `(local port, remote address, remote port)` tuple is seen. -This would most commonly happen as a result of a first datagram sent to a new peer, coinciding with the establishment of a NAT-style mapping and the possible allocation of a socket on the exit. - -A less common case needs to be considered too: what if the parameter tuple first occurs on the exit side? -We don't yet have a way to allocate stream IDs from either end of a circuit. -This would need to be considered. -One simple solution would be to statically partition the stream ID space into a portion that can be independently allocated by each side. - -When is this exit-originated circuit ID allocation potentially needed? -It is clearly needed when using **address-dependent filtering**. -An incoming datagram from a previously-unseen peer port is expected to be deliverable, and the exit would need to allocate an ID for it. - -Even with the stricter **address and port-dependent filtering** clients may still be exposed to exit-originated circuit IDs if there are mismatches in the lifetime of the filter and the stream. - -This approach thus requires some attention to either correctly allocating stream IDs on both sides of the circuit, or choosing a filtering strategy and filter/mapping lifetime that does not ever leave stream IDs undefined when expecting incoming datagrams. - -### One Stream per Mapping - -One stream **per mapping** is an alternative which attempts to reduce the number of edge cases by merging the lifetimes of one stream and one **endpoint-independent mapping**. - -A mapping would always be allocated from the OP (client) side. -It could explicitly specify a filtering style, if we wish to allow applications to request non-port-dependent filtering for compatibility. -Each datagram within the stream would still need to be tagged with a peer address/port in some way. - -This approach would involve a single new type of stream, and two new messages that pertain to these *mapping* streams: - -- `NEW_UDP_MAPPING` - - - Always client-to-exit. - - Creates a new mapping, with a specified stream ID. - - Succeeds instantly; no reply is expected, early data is ok. - - Externally-visible local port number is arbitrary, and must be determined through interaction with other endpoints. - - Might contain an IP "don't fragment" flag. - - Might contain a requested filtering mode. - - Lifetime is until circuit teardown or `END` message. - -- `UDP_MAPPING_DATAGRAM` - - - Conveys one datagram on a stream previously defined by `NEW_UDP_MAPPING`. - - Includes peer address (IPv4/IPv6) as well as datagram content. - -This puts us in a very similar design space to TURN (RFC8656). -In that protocol, "allocations" are made explicitly on request, and assigned a random relayed port. -TURN also uses its allocations as an opportunity to support a *Don't Fragment* flag. - -The principal disadvantage of this approach is in space overhead, especially the proportional overhead on small datagrams which must each carry a full-size address. - -### Hybrid Mapping and Flow Approach - -We can extend the approach above with an optimization that addresses the undesirable space overhead from redundant address headers. -This uses two new types of stream, in order to have streams **per mapping** and **per flow** at the same time. - -The per-mapping stream remains the sole interface for managing the lifetime of a mapped UDP port. Mappings are created explicitly by the client. As an optimization, within the lifetime of a mapping there may exist some number of *flows*, each assigned their own ID. - -This tries to combine the strengths of both approaches, using the lifetime of one stream to define a mapping and to carry otherwise-unbundled traffic while also allowing additional streams to bundle datagrams that would otherwise have repetitive headers. -It avoids the space overhead of a purely **per mapping** approach and avoids the ID allocation and lifetime complexity introduced with **per flow**. - -This approach takes some inspiration from TURN, where commonly used peers will be defined as a "channel" with an especially short header. -Incoming datagrams with no channel can always be represented in the long form, so TURN never has to allocate channels unexpectedly. - -The implementation here could be a strict superset of the **per mapping** implementation, adding new commands for flows while retaining existing behavior for mappings. There would be a total of four new message types: - -- `NEW_UDP_MAPPING` - - - Same as above. - -- `UDP_MAPPING_DATAGRAM` - - - Same as above. - -- `NEW_UDP_FLOW` - - - Allocates a stream ID as a *flow*, given the ID to be allocated and the ID of its parent *mapping* stream. - - Includes a peer address (IPv4/IPv6). - - The *flow* has a lifetime strictly bounded by the outer *mapping*. It is deleted by an explicit `END` or when the mapping is de-allocated for any reason. - -- `UDP_FLOW_DATAGRAM` - - - Datagram contents only, without address. - - Only appears on *flow* streams. - -We must consider the traffic marking opportunities provided when allowing an exit to represent one incoming datagram as either a *flow* or *mapping* datagram. - -It's possible this traffic injection potential is not worse than the baseline amount of injection potential than every UDP protocol presents. See more on [risks](#risks) below. For this hybrid stream approach specifically, there's a limited mitigation available which allows exits only a bounded amount of leaked information per UDP peer: - -Ideally exits may not choose to send a `UDP_MAPPING_DATAGRAM` when they could have sent a `UDP_FLOW_DATAGRAM`. -Sometimes it is genuinely unclear though: an exit may have received this datagram in-between processing `NEW_UDP_MAPPING` and `NEW_UDP_FLOW`. -A partial mitigation would terminate circuits which send a `UDP_MAPPING_DATAGRAM` for a peer that has already been referenced in a `UDP_FLOW_DATAGRAM`. -The exit is thereby given a one-way gate allowing it to switch from using *mapping* datagrams to using *flow* datagrams at some point, but not to switch back and forth repeatedly. - -Mappings that do not request port-specific filtering may always get unexpected `UDP_MAPPING_DATAGRAM`s. Mappings that do use port-specific filtering could make a flow for their only expected peers, then expect to never see `UDP_MAPPING_DATAGRAM`. - -`NEW_UDP_MAPPING` could have an option requiring that only `UDP_FLOW_DATAGRAM` is to be used, never `UDP_MAPPING_DATAGRAM`. -This would remove the potential for ambiguity, but costs in compatibility as it's no longer possible to implement non-port-specific filtering. - -## Risks - -Any proposed UDP support involves significant risks to user privacy and software maintainability. -This section elaborates some of these risks, so they can be compared against expected benefits. - -### Behavior Regressions - -In some applications it is possible that Tor's implementation of a UDP compatibility layer will cause a regression in the ultimate level of performance or security. - -Performance regressions can occur accidentally due to bugs or compatibility glitches. -They may also occur for more fundamental reasons of protocol layering. -For example, the redundant error correction layers when tunneling QUIC over TCP. -These performance degradations are expected to be minor, but there's some unavoidable risk. - -The risk of severe performance or compatibility regressions may be mitigated by giving users a way to toggle UDP support per-application. - -Privacy and security regressions have more severe consequences and they can be much harder to detect. -There are straightforward downgrades, like WebRTC apps that give up TURN-over-TLS for plaintext TURN-over-UDP. -More subtly, the act of centralizing connection establishment traffic in Tor exit nodes can make users an easier target for other attacks. - -### Bandwidth Usage - -We should expect an increase in overall exit bandwidth requirements due to peer-to-peer file sharing applications. - -Current users attempting to use BitTorrent over Tor are hampered by the lack of UDP compatibility. Interoperability with common file-sharing peers would make Tor more appealing to users with a large and sustained appetite for anonymized bandwidth. - -### Local Port Exhaustion - -Exit routers will have a limited number of local UDP ports. In the most constrained scenario, an exit may have a single IP with 16384 or fewer ephemeral ports available. These ports could each be allocated by one client for an unbounded amount of exclusive use. - -In order to enforce high levels of isolation between different subsequent users of the same local UDP port, we may wish to enforce an extended delay between allocations during which nobody may own the port. Effective isolation requires this timer duration to be greater than any timer encountered on a peer or a NAT. In RFC4787's recommendations a NAT's mapping timer must be longer than 2 minutes. Our timer should ideally be *much* longer than 2 minutes. - -An attacker who allocates ports for only this minimum duration of 2 minutes would need to send 136.5 requests per second to achieve sustained use of all available ports. With multiple simultaneous clients this could easily be done while bypassing per-circuit rate limiting. - -The expanded definition of "Port overlapping" from [RFC7857 section 3](https://datatracker.ietf.org/doc/html/rfc7857#section-3), may form at least a partial mitigation: - -> This document clarifies that this port overlapping behavior may be extended to connections originating from different internal source IP addresses and ports as long as their destinations are different. - -This gives us an opportunity for a vast reduction in the number of required ports and file descriptors. -Exit routers can automatically allocate local ports for use with a specific peer when that peer is first added to the client's filter. - -Due to the general requirements of NAT traversal, UDP applications with any NAT support will always need to communicate with a relatively well known server prior to any attempts at peer-to-peer communication. -This early peer could be an entire application server, or it could be a STUN endpoint. -In any case, the identity of this first peer gives us a hint about the set of all potential peers. - -Within the exit router, each local port will track a separate mapping owner for each peer. -When processing that first outgoing datagram, the exit may choose any local port where the specific peer is not taken. -Subsequent outgoing datagrams on the same port may communicate with a different peer, and there's no guarantee all these future peers will be claimed successfully. - -When is this a problem? -An un-claimable peer represents a case where the exact `(local ip, local port, remote ip, remote port)` tuple is in use already for a different mapping in some other Tor stream. -For example, imagine two clients are running different types of telecom apps which are nevertheless inter-compatible and capable of both calling the same peers. -Alternatively, consider the same app but with servers in several regions. -The two apps will begin by communicating with different sets of peers, due to different application servers and different bundled STUN servers. -This is our hint that it's likely appropriate to overlap their local port allocations. -At this point, both of these applications may be successfully sharing a `(local ip, local port)` tuple on the exit. -As soon as one of these apps calls a peer with some `(remote ip, remote port)`, the other app will be unable to contact that specific peer. - -The lack of connectivity may seem like a niche inconvenience, and perhaps that is the extent of the issue. -It seems likely this heuristic could result in a problematic information disclosure under some circumstances, and it deserves closer study. - -### Application Fingerprinting - -UDP applications present an increased surface of plaintext data that may be available for user fingerprinting by malicious exits. - -Exposed values can include short-lived identifiers like STUN usernames. -Typically it will also be possible to determine what type of software is in use, and maybe what version of that software. - -Short-lived identifiers are still quite valuable to attackers, because they may reliably track application sessions across changes to the Tor exit. If longer-lived identifiers exist for any reason, that of course provides a powerful tool for call metadata gathering. - -### Peer-to-Peer Metadata Collection - -One of our goals was to achieve the compatibility and perhaps performance benefits of allowing "peer-to-peer" (in our case really exit-to-exit) UDP connections. -We expect this to enable the subset of applications that lack a fallback path which loops traffic through an app-provided server. - -This goal may be at odds with our privacy requirements. At minimum, a pool of malicious exit nodes could passively collect metadata about these connections as a noisy proxy for call metadata. - -Any additional signals like [application fingerprints](#application-fingerprinting) or [injected](#traffic-injection) markers may be used to enrich this metadata graph, possibly tracking users across sessions or across changes to the tunnel endpoint. - -### Interaction with Other Networks - -Any application using an ICE-like interactive connection establishment scheme will easily leak information across network boundaries if it ever has access to multiple networks at once. - -In applications that are not privacy conscious this is often desired behavior. For example, a video call to someone in your household may typically transit directly over Wifi, decreasing service costs and improving latency. This implies that some type of local identifier accompanies the call signaling info, allowing the devices to find each other's LAN address. - -Privacy-preserving solutions for this use case are still an active area of standardization effort. The [IETF draft on mDNS ICE candidates](https://www.ietf.org/archive/id/draft-ietf-mmusic-mdns-ice-candidates-02.html) proposes one way to accomplish this by generating short-lived unique IDs which are only useful to peers with physical access to the same mDNS services. - -Without special attention to privacy, the typical implementation is to share all available IP addresses and to initiate simultaneous connectivity tests using any IP pairs which cannot be trivially discarded. This applies to ICE as specified, but also to any proprietary protocol which operates in the same design space as ICE. This has multiple issues in a privacy-conscious environment: The IP address disclosures alone can be fatal to anonymity under common threat models. Even if meaningful IP addresses are not disclosed, the timing correlation from connectivity checks can provide confirmation beacons that alert an attacker to some connection between a LAN user and a Tor exit. - -### Traffic Injection - -Some forms of UDP support would have obvious and severe traffic injection vulnerabilities. For example, the very permissive *endpoint-independent filtering* strategy would allow any host on the internet to send datagrams in bulk to all available local ports on a Tor exit in order to map that traffic's effect on any guards they control. - -Any vehicle for malicious parties to mark traffic can be abused to de-anonymize users. Even if there is a more restrictive filtering policy, UDP's lack of sequence numbers make header spoofing attacks considerably easier than in TCP. Third parties capable of routing datagrams to an exit with a spoofed source address could bypass filtering when the communicating parties are known or can be guessed. For example, a malicious superuser at an ISP without egress filtering could send packets with the source IPs set to various common DNS servers and application STUN/TURN servers. If a specific application is being targeted, only the exit node and local port numbers need to be guessed by brute force. - -In case of malicious exit relays, whole datagrams can be inserted and dropped, and datagrams may be padded with additional data, both without any specific knowledge of the application protocol. With specific protocol insights, a malicious relay may make arbitrary edits to plaintext data. - -Of particular interest is the plaintext STUN, TURN, and ICE traffic used by most WebRTC apps. These applications rely on higher-level protocols (SRTP, DTLS) to provide end-to-end encryption and authentication. A compromise at the connection establishment layer would not violate application-level end-to-end security requirements, making it outside the threat model of WebRTC but very much still a concern for Tor. - -These attacks are not fully unique to the proposed UDP support, but UDP may increase exposure. In cases where the application already has a fallback using TURN-over-TLS, the proposal is a clear regression over previous behaviors. Even when comparing plaintext to plaintext, there may be a serious downside to centralizing all connection establishment traffic through a small number of exit IPs. Depending on your threat model, it could very well be more private to allow the UDP traffic to bypass Tor entirely. - -### Malicious Outgoing Traffic - -We can expect UDP compatibility in Tor will give malicious actors additional opportunities to transmit unwanted traffic. - -In general, exit abuse will need to be filtered administratively somehow. -This is not unique to UDP support, and exit relay administration typically involves some type of filtering response tooling that falls outside the scope of Tor itself. - -Exit administrators may choose to modify their exit policy, or to silently drop problematic traffic. -Silent dropping is discouraged in most cases, as Tor prioritizes the accuracy of an exit's advertised policy. -Detailed exit policies have a significant space overhead in the overall Tor consensus document, but it's still seen as a valuable resource for clients during circuit establishment. - -Exit policy filtering may be less useful in UDP than with TCP due to the inconvenient latency spike when establishing a new [tunnel](#tunneling). -Applications that are sensitive to RTT measurements made during connection establishment may fail entirely when the tunnel cannot be pre-built. - -This section lists a few potential hazards, but the real-world impact may be hard to predict owing to a diversity of custom UDP protocols implemented across the internet. - -- Amplification attacks against arbitrary targets - - These are possible only in limited circumstances where the protocol allows an arbitrary reply address, like SIP. -The peer is often at fault for having an overly permissive configuration. -Nevertheless, any of these *easy* amplification targets can be exploited from Tor with little consequence, creating a nuisance for the ultimate target and for exit operators. - -- Amplification attacks against an exit relay - - An amplification peer which doesn't allow arbitrary destinations can still be used to attack the exit relay itself or other users of that relay. -This is essentially the same attack that is possible against any NAT the attacker is behind. - -- Malicious fragmented traffic - - If we allow sending large UDP datagrams over IPv4 without the *Don't Fragment* flag set, we allow attackers to generate fragmented IP datagrams. -This is not itself a problem, but it has historically been a common source of inconsistencies in firewall behavior. - -- Excessive sends to an uninterested peer - - Whereas TCP mandates a successful handshake, UDP will happily send unlimited amounts of traffic to a peer that has never responded. -To prevent denial of service attacks we have an opportunity and perhaps a responsibility to define our supported subset of UDP to include true bidirectional traffic but exclude continued sends to peers who do not respond. - - See also [RFC7675](https://www.rfc-editor.org/rfc/rfc7675.html) and STUN's concept of "Send consent". - -- Excessive number of peers - - We may want to place conservative limits on the maximum number of peers per mapping or per circuit, in order to make bulk scanning of UDP port space less convenient. - - The limit would need to be on peers, not stream IDs as we presently do for TCP. -In this proposal stream IDs are not necessarily meaningful except as a representational choice made by clients. - -## Next Steps - -At this point we have perhaps too many possibilities for how to proceed. We could integrate UDP quite closely with Tor itself or not at all. - -Choosing a route forward is both a risk/benefit tradeoff and a guess about the future of Tor. If we expect that some future version of Tor will provide its own datagram transport, we should plot a course aiming in that direction. If not, we might be better served by keeping compatibility features confined to separate protocol layers when that's practical. - -### Requiring a Long-Term Datagram Plan - -If we choose to add new long-term maintenance burdens to our protocol stack, we should ensure they serve our long-term goals for UDP adoption as well as these shorter-term application compatibility goals. - -This work so far has been done with the assumption that end-to-end datagram support is out of scope. If we intend to proceed down any path which encodes a datagram-specific protocol into Tor proper, we should prioritize additional protocol research and standardization work. - -### Alternatively, Modular Application-Level Support - -Without a clear way to implement fully generic UDP support, we're left with application-level goals. -Different applications may have contrasting needs, and we can only achieve high levels of both compatibility and privacy by delegating some choices to app authors or per-app VPN settings. - -This points to an alternative in which UDP support is excluded from Tor as much as possible while still supporting application requirements. -For example, this could motivate the [TURN Encapsulated in a Tor Stream](#turn-encapsulated-in-a-tor-stream) design, or even simpler designs where TURN servers are maintained independently from the Tor exit infrastructure. - -The next prototyping step we would need at this stage is a version of `onionmasq` extended to support NAT-style UDP mappings using TURN allocations provided through TURN-over-TCP or TURN-over-TLS. -This can be a platform for further application compatibility experiments. -It could potentially become a low-cost minimal implementation of UDP application compatibility, serving to assess which remaining user needs are still unmet. -- cgit v1.2.3-54-g00ecf From 8adff0474bbbb5244baf916f3f85224658efa40f Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 25 Jan 2024 09:02:37 -0800 Subject: Update proposal 348 header I had added the 'text' format and put the heading first during earlier editing to satisfy a markdown lint tool, but now that I'm running the "reindex" script it has its own requirements. This patch fixes the indexer. --- proposals/348-udp-app-support.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/348-udp-app-support.md b/proposals/348-udp-app-support.md index 55f2192..eb67ff5 100644 --- a/proposals/348-udp-app-support.md +++ b/proposals/348-udp-app-support.md @@ -1,13 +1,13 @@ -# UDP Application Support in Tor - -```text -Filename: XXX-udp-app-support.md +``` +Filename: 348-udp-app-support.md Title: UDP Application Support in Tor Author: Micah Elizabeth Scott Created: December 2023 Status: Draft ``` +# UDP Application Support in Tor + ## Table of Contents - [Introduction](#introduction) -- cgit v1.2.3-54-g00ecf From 55983d3db9a1b7b330baecd24ec055d21fb02559 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 25 Jan 2024 09:04:50 -0800 Subject: Proposal 348 status: draft -> open --- proposals/348-udp-app-support.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/348-udp-app-support.md b/proposals/348-udp-app-support.md index eb67ff5..ac83fed 100644 --- a/proposals/348-udp-app-support.md +++ b/proposals/348-udp-app-support.md @@ -3,7 +3,7 @@ Filename: 348-udp-app-support.md Title: UDP Application Support in Tor Author: Micah Elizabeth Scott Created: December 2023 -Status: Draft +Status: Open ``` # UDP Application Support in Tor -- cgit v1.2.3-54-g00ecf From 00fd93229bf61e3044247af94b2070193ec09545 Mon Sep 17 00:00:00 2001 From: Micah Elizabeth Scott Date: Thu, 25 Jan 2024 09:05:09 -0800 Subject: Re-index proposals --- proposals/000-index.txt | 2 ++ proposals/BY_INDEX.md | 1 + proposals/BY_STATUS.md | 1 + proposals/SUMMARY.md | 1 + 4 files changed, 5 insertions(+) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index a71b35d..8c9668f 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -269,6 +269,7 @@ Proposals by number: 345 Migrating the tor specifications to mdbook [CLOSED] 346 Clarifying and extending the use of protocol versioning [OPEN] 347 Domain separation for certificate signing keys [OPEN] +348 UDP Application Support in Tor [OPEN] Proposals by status: @@ -307,6 +308,7 @@ Proposals by status: 344 Prioritizing Protocol Information Leaks in Tor 346 Clarifying and extending the use of protocol versioning 347 Domain separation for certificate signing keys + 348 UDP Application Support in Tor ACCEPTED: 282 Remove "Named" and "Unnamed" handling from consensus voting [for arti-dirauth] 285 Directory documents should be standardized as UTF-8 [for arti-dirauth] diff --git a/proposals/BY_INDEX.md b/proposals/BY_INDEX.md index 7c318fa..a8f1ac4 100644 --- a/proposals/BY_INDEX.md +++ b/proposals/BY_INDEX.md @@ -265,4 +265,5 @@ Below are a list of proposals sorted by their proposal number. See * [`345-specs-in-mdbook.md`](/proposals/345-specs-in-mdbook.md): Migrating the tor specifications to mdbook \[CLOSED\] * [`346-protovers-again.md`](/proposals/346-protovers-again.md): Clarifying and extending the use of protocol versioning \[OPEN\] * [`347-domain-separation.md`](/proposals/347-domain-separation.md): Domain separation for certificate signing keys \[OPEN\] +* [`348-udp-app-support.md`](/proposals/348-udp-app-support.md): UDP Application Support in Tor \[OPEN\] diff --git a/proposals/BY_STATUS.md b/proposals/BY_STATUS.md index 5a103ac..0c265c2 100644 --- a/proposals/BY_STATUS.md +++ b/proposals/BY_STATUS.md @@ -40,6 +40,7 @@ for discussion. * [`344-protocol-info-leaks.txt`](/proposals/344-protocol-info-leaks.txt): Prioritizing Protocol Information Leaks in Tor * [`346-protovers-again.md`](/proposals/346-protovers-again.md): Clarifying and extending the use of protocol versioning * [`347-domain-separation.md`](/proposals/347-domain-separation.md): Domain separation for certificate signing keys +* [`348-udp-app-support.md`](/proposals/348-udp-app-support.md): UDP Application Support in Tor ## ACCEPTED proposals: slated for implementation diff --git a/proposals/SUMMARY.md b/proposals/SUMMARY.md index f66d166..df02273 100644 --- a/proposals/SUMMARY.md +++ b/proposals/SUMMARY.md @@ -258,6 +258,7 @@ - [`345-specs-in-mdbook`](./345-specs-in-mdbook.md): Migrating the tor specifications to mdbook (CLOSED) - [`346-protovers-again`](./346-protovers-again.md): Clarifying and extending the use of protocol versioning (OPEN) - [`347-domain-separation`](./347-domain-separation.md): Domain separation for certificate signing keys (OPEN) + - [`348-udp-app-support`](./348-udp-app-support.md): UDP Application Support in Tor (OPEN) -- cgit v1.2.3-54-g00ecf