```
Filename: 346-protovers-again.md
Title: Clarifying and extending the use of protocol versioning
Author: Nick Mathewson
Created: 19 Oct 2023
Status: Open
```

# Introduction

In proposal 264, we introduced "subprotocol versions" as a way to
independently version different pieces of the Tor protocols, and
communicate which parts of the Tor protocols are supported,
recommended, and required.

Here we clarify the semantics of individual subprotocol versions, and
describe more ways to use and negotiate them.

# Semantics: Protocol versions are feature flags

One issue we left unclarified previously is the relationship between
two different versions of the same subprotocol.  That is, if we know
the semantics of (say) `Handshake=7`, can we infer anything about a
relay that supports `Handshake=8`?  In particular, can we infer that
it supports all of the same features implied by `Handshake=7`?  If we
want to know "does this relay support some feature supported by
`Handshake=7`", must we check whether it supports `Handshake=7`, or
should we check `Handshake=x for any x≥7`?

In this proposal, we settle the question as follows: subprotocol
versions are flags.  They do not have any necessary semantic
relationship between them.

We reject the `≥` interpretation for several reasons:
  * It's tricky to implement.
  * It prevents us from ever removing a feature.
  * It requires us to implement features in the same order across
    all Tor versions.

## ...but sometimes a flag _is_ a version!

There _are_ some places in our protocol (notably: directory authority
consensus methods, and channel protocol versions) where there _is_ a
semantic relationship between version numbers.  Specifically: "higher
numbers are already better".  When parties need to pick a one of
_these_ versions, they always pick the highest version number
supported by enough of them.

When this kind of _real_ version intersects with the "subprotocol
versions" system, we use the same numbers:

 * `Link` subprotocols correspond one-to-one with the version numbers
   sent in a VERSIONS cell.
 * `Microdesc` and `Cons` subprotocols correspond to a _subset_ of
   the version numbers of consensus methods.

## How to document subprotocol versions

When describing a subprotocol, we should be clear what relationship,
if any, exists between its versions and any versions negotiated
elsewhere in the specifications.

Unless otherwise documented, all versions can be in use _at the same
time_: if only one can exist at once (on a single circuit, a single
document, etc), this must be documented.

> Implication: This means that we must say (for example) that you
> can't use Link=4 and Link=5 on the same channel.

# Negotiating protocol versions in circuit handshakes.

Here we describe a way for a client to opt into features as part of
its circuit handshake, in order to avoid proliferating negotiating
extensions.

## Binary-encoding protocol versions.

We assign a one-byte encoding for each protocol version number,
ordered in the same way as in tor-spec.

| Protocol | Id  |
| ---      | --- |
| Link     | 0  |
| LinkAuth | 1  |
| Relay    | 2  |
| DirCache | 3  |
| HSDir    | 4  |
| HSIntro  | 5  |
| HSRend   | 6  |
| Desc     | 7  |
| MicroDesc| 8  |
| Cons     | 9  |
| Padding  | 10 |
| FlowCtrl | 11 |
| Conflux  | 12 |
| RelayCell| 13 |
| Datagram | TBD|

> Note: This is the same encoding used in [walking onions proposal][prop323].
> It takes its order from the ordering of protocol versions in
> [tor-spec][subprotocol-versioning] and matches up with the values defined in
> for `protocol_type_t` in C tor's `protover.h`.

## Requesting an opt-in circuit feature

When a client wants to request a given set of features, it sends an
`ntor_v3` extension containing:

```
struct subproto_request {
  struct req[..]; // up to end of extension
}

struct req {
  u8 protocol_id;
  u8 protovol_version;
}
```

> Note 1: The above format does not include any parameters for each
> req.  Thus, if we're negotiating an extension that requires a client-
> supplied parameter, it may not be appropriate to use this
> request format.
>
> Note 2: This proposal does not include any relay extension
> acknowledging support.  In the case of individual subprotocols, we could
> later say "If this subprotocol is in use, the relay MUST also send
> extension foo".
>
> Note 3: The existence of this extension does not preclude the later
> addition of other extensions to negotiate featuress differently, or
> to do anything else.

Each `req` entry corresponds to a single subprotocol version. A client
MUST NOT send any `req` entry unless:
  * That subprotocol version is advertised by the relay,
  * OR that subprotocol version is listed as required for relays in the
    current consensus, using `required-relay-protocols`.

> Note: We say above that a client may request a _required_ subprotocol
> even if the relay does not advertise it.  This is what allows
> clients to send a `req` extension to introduction points and
> rendezvous points, even when we do not recognize the relay from the
> consensus.
>
> Note 2: If the intro/rend point does not support a _required_ protocol,
> it should not be on the network, and the client/service should not have
> selected it.

If a relay receives a `subproto_request` extension for any subprotocol
version that it does not support, it MUST reject the circuit with a
DESTROY cell.

> Alternatives: we _could_ give the relay the option to
> decline to support an extension, and we _could_ require the
> relay to acknowledge which extensions it is providing.
> We aren't doing that, in the name of simplicity.

Only certain subprotocol versions need to be negotiated in this way;
they will be explicitly listed as such in our specifications, with
language like "This extension is negotiated as part of the circuit
extension handshake".  Other subprotocol versions MUST NOT be listed
in this extension; if they are, the relay SHOULD reject the circuit.

> Alternative: We _could_ allow the client to list other subprotocols
> that the relay supports which are nonetheless irrelevant to
> the circuit protocol, like `Microdesc`, or ones that don't currently need
> to be negotiated, like `HsRend`.
>
> This is not something we plan to do.

Currently specified subprotocol versions which can be negotiated using
this extension are:

   * FlowCtrl=2 (congestion control)
   * RelayCell=1 (proposal 340)

The availability of the `subproto_request` extension itself
will be indicated by a new Relay=X flag.  When used, it will supplant
several older `ntor_v3` extensions, including:

   * (TODO: list these here, if we find any. I think FlowCtrl has an
     extension?)

That is, if using `subproto_request`, there is no need to send the
(TODO) extensions.



## Making features that can be disabled.

Sometimes, we will want the ability to make features that can be
enabled or disabled from the consensus.  But if we were to make a single
flag that can turn the feature on and off, we'd run into trouble:
after the feature was turned off, every relay would stop providing it
right away, but there would be a delay before clients realized that
the relays had stopped advertising the feature.  During this interval,
clients would try to enable the feature, and the relays would reject
their circuits.

To solve this problem, we need to make features like these controlled
by a _pair_ of consensus parameters: one to disable advertising the
feature, and one to disable the feature itself.  To disable a feature,
first the authorities would tell relays to stop advertising it, and
only later tell the relays to stop supporting it.  (If we want to
_enable_ a previously disabled feature, we can turn on advertisement
and support at the same time.)

These parameters would be specified something like this (for a
hypthetical `Relay=33` feature).

 * `support-relay-33`: if set to 1, relays that can provide
   `Relay=33` should do so.
 * `advertise-relay-33`: if set to 1, relays that are
   providing `Relay=33` should include it in their advertised
   protocol versions.

Note: as a safety measure, relays MUST NOT advertise any feature that
they do not support.  This is reflected in the descriptions of the
parameters above.

When we add a new feature of this kind, we should have the
`advertise-*` flag parameter be `1` by default, and probably we should
have `support-*` be `1` by default oo.

# Subprotocol versions in onion services

Here we describe how to expand the onion service protocols in order
to better accomodate subprotocol versions.

## Advertising an onion service's subprotocols

In its encrypted descriptor (the innermost layer), the onion service
adds a new entry:

  * `"protocols"` - A list of supported subprotocol versions, in the
    same format as those listed in a microdescriptor or descriptor.

Note that this is NOT a complete list of all the subprotocol versions
actually supported by the onion service.  Instead, onion services only
advertise a subprotocol version if they support it, _and_ it is
documented in the specs as being supported by onion services.

> Alternative: I had considered having a mask that would be put in the
> consensus document, telling the onion services which subprotocols
> to advertise.  I don't think that's a great idea, however.

Right now, the following protocols should be advertised:

 - FlowCtrl
 - Conflux (?? Doesn't this take parameters? TODO)
 - Pow (??? Doesn't this take parameters? If we do it, we need to
   allocate a subprotocol for it. TODO)


## Negotiating subprotocols with an onion service.

In the `hs_ntor` handshake sent by the client, we add an encrypted
`subproto_request` extension of the same format, with the same
semantics, as used in the `ntor-v3` handshake.

This supplants the following:

 - (Congestion Control; Pow? TODO)


## Advertising other relays' subprotocols?

> Alternative: I had previously considered a design where the introduction points
> in the onion service descriptor would be listed along with their
> subprotocols, and the `hs_ntor` handshake would contain the
> subprotocols of the rendezvous point.
>
> I'm rejecting this design for now because it gives the onion service
> and the client too much freedom to lie about relays.  In the future,
> the walking onions design would solve this, since the contact
> information for intro and rend points would be authenticated.

# Appendix

New numbers to reserve:

  * An extension ID for the `ntor_v3` handshake `subproto_request`
    extension.
  * An extension ID for the `hs_ntor` handshake `subproto_request`
    extension.
  * A Relay= subprotocol indicating support for the ntor-v3 and
    hs_ntor extensions.
  * The numeric encoding of each existing subprotocol, in the table
    above.

# Acknowledgments

Thanks to David Goulet and Mike Perry for their feedback on earlier
versions of this proposal!

[prop323]: https://spec.torproject.org/proposals/323-walking-onions-full.html
[subprotocol-versioning]: https://spec.torproject.org/tor-spec/subprotocol-versioning.html