Filename: 346-protovers-again.md
Title: Clarifying and extending the use of protocol versioning
Author: Nick Mathewson
Created: 19 Oct 2023
Status: Open

Introduction

In proposal 264, we introduced "subprotocol versions" as a way to independently version different pieces of the Tor protocols, and communicate which parts of the Tor protocols are supported, recommended, and required.

Here we clarify the semantics of individual subprotocols, and describe more ways to use and negotiate them.

Semantics: Protocol versions are feature flags

One issue we left unclarified previously is the relationship between two different "versions" of the same subprotocol. That is, if we know the semantics of (say) Handshake=7, can we infer anything about a relay that supports Handshake=8? In particular, can we infer that it supports all of the same features implied by Handshake=7? If we want to know "does this relay support some feature supported by Handshake=7", must we check whether it supports Handshake=7, or should we check Handshake=x for any x≥7?

In this proposal, we settle the question as follows: subprotocol "versions" are flags. They do not have any necessary semantic relationship between them.

We reject the ≥ interpretation for several reasons:

• It's tricky to implement.
• It prevents us from ever removing a feature.
• It requires us to implement features in the same order across all Tor versions.

Because of these semantics, we will henceforth avoid referring to a Proto=N item as a "subprotocol version". Instead we will call them "subprotocols" or "subprotocol flags". We will still refer to "subprotocol versioning" when we are talking about the use of subprotocols to handle version differenes in the Tor protocol.

But what about when a flag is a version!

There are some places in our protocol (notably: directory authority consensus methods, and channel protocol versions) where there is a semantic relationship between version numbers. Specifically: "higher numbers are already better". When parties need to pick a one of these versions, they always pick the highest version number supported by enough of them.

When this kind of real version intersects with the "subprotocol versions" system, we use the same numbers:

• Link subprotocols correspond one-to-one with the version numbers sent in a VERSIONS cell.
• Microdesc and Cons subprotocols correspond to a subset of the version numbers of consensus methods.

Note however that nothing in this section requires us to adopt a "≥" interpretation for subprotocols: even though Link=3 does indicate an earlier protocol than Link=4, we will still never assume that a feature provided by Link=4 is automatically available if the relay has Link=5.

How to document subprotocols

When describing a subprotocol, we should be clear what relationship, if any, exists between its flags, and any featurees negotiated elsewhere in the specifications.

Unless otherwise documented, all subprotocol flags can be in use at the same time: if only one can exist at once (on a single circuit, a single document, etc), this must be documented.

Implication: This means that we must say (for example) that you can't use Link=4 and Link=5 on the same channel.

Negotiating subprotocol flags in circuit handshakes.

Here we describe a way for a client to opt into features as part of its circuit handshake, in order to avoid proliferating negotiating extensions.

Binary-encoding subprotocol flags.

We assign a one-byte encoding for each subprotocol, ordered in the same way as in tor-spec.

Note: This is the same encoding used in walking onions proposal. It takes its order from the ordering of subprotocols in tor-spec and matches up with the values defined in for protocol_type_t in C tor's protover.h.

Requesting an opt-in circuit feature

When a client wants to request a given set of features, it sends an ntor_v3 extension containing:

struct subproto_request {
  struct req[..]; // up to end of extension
}

struct req {
  u8 protocol_id;
  u8 protocol_flag_number; // (formerly called "version")
}

Within each subproto_request, the req entries SHOULD be sorted in ascending order by protocol_id, then by protocol_flag_number. (For example, 01 01 comes before 01 02, which comes before 02 01.)

Note 1: The above format does not include any parameters for each req. Thus, if we're negotiating an extension that requires a client- supplied parameter, it may not be appropriate to use this request format.

Note 2: This proposal does not include any relay extension acknowledging support. In the case of individual subprotocols, we could later say "If this subprotocol is in use, the relay MUST also send extension foo".

Note 3: The existence of this extension does not preclude the later addition of other extensions to negotiate featuress differently, or to do anything else.

Each req entry corresponds to a single subprotocol flag. A client MUST NOT send any req entry unless:

• That subprotocol flag is advertised by the relay,
• OR that subprotocol flag is listed as required for relays in the current consensus, using required-relay-protocols.

Note: We say above that a client may request a required subprotocol even if the relay does not advertise it. This is what allows clients to send a req extension to introduction points and rendezvous points, even when we do not recognize the relay from the consensus.

Note 2: If the intro/rend point does not support a required protocol, it should not be on the network, and the client/service should not have selected it.

If a relay receives a subproto_request extension for any subprotocol flag that it does not support, it MUST reject the circuit with a DESTROY cell.

Alternatives: we could give the relay the option to decline to support an extension, and we could require the relay to acknowledge which extensions it is providing. We aren't doing that, in the name of simplicity.

Only certain subprotocol flags need to be negotiated in this way; they will be explicitly listed as such in our specifications, with language like "This extension is negotiated as part of the circuit extension handshake". Other subprotocol flags MUST NOT be listed in this extension; if they are, the relay SHOULD reject the circuit.

Alternative: We could allow the client to list other subprotocols that the relay supports which are nonetheless irrelevant to the circuit protocol, like Microdesc, or ones that don't currently need to be negotiated, like HsRend.

This is not something we plan to do.

The availability of the subproto_request extension will be indicated by a new Relay=5 flag.

Migration from old extensions to subproto_request is not specified here.

Making features that can be disabled.

Sometimes, we will want the ability to make features that can be enabled or disabled from the consensus. But if we were to make a single flag that can turn the feature on and off, we'd run into trouble: after the feature was turned off, every relay would stop providing it right away, but there would be a delay before clients realized that the relays had stopped advertising the feature. During this interval, clients would try to enable the feature, and the relays would reject their circuits.

To solve this problem, we need to make features like these controlled by a pair of consensus parameters: one to disable advertising the feature, and one to disable the feature itself. To disable a feature, first the authorities would tell relays to stop advertising it, and only later tell the relays to stop supporting it. (If we want to enable a previously disabled feature, we can turn on advertisement and support at the same time.)

These parameters would be specified something like this (for a hypthetical Relay=33 feature).

• support-relay-33: if set to 1, relays that can provide Relay=33 should do so.
• advertise-relay-33: if set to 1, relays that are providing Relay=33 should include it in their advertised subprotocol flags.

Similarly, we might define a set of parameters for a hidden service feature:

• hs-support-relay-33: if set to 1, services that can provide Relay=33 should do so.
• hs-advertise-relay-33: if set to 1, services that provide Relay=33 should include it in their advertised flags.

Note: as a safety measure, Tor instances MUST NOT advertise any feature that they do not support. This is reflected in the descriptions of the parameters above.

Note also: these parameters will not automatically exist for every new protocol feature; we should only specify them when their presence is warranted.

When we add a new feature of this kind, we should have the advertise-* flag parameter be 1 by default, and probably we should have support-* be 1 by default oo.

Subprotocol flags in onion services

Here we describe how to expand the onion service protocols in order to better accomodate subprotocol flags.

Advertising an onion service's subprotocols

In its encrypted descriptor (the innermost layer), the onion service adds a new entry:

• "proto" - A list of supported subprotocol flags, in the same format as those listed in a microdescriptor or descriptor.

Note that this is NOT a complete list of all the subprotocol flags actually supported by the onion service. Instead, onion services only advertise a subprotocol flags if they support it, and it is documented in the specs as being supported by onion services.

Alternative: I had considered having a mask that would be put in the consensus document, telling the onion services which subprotocols to advertise. I don't think that's a great idea, however.

Migration from existing flags to the proto item is not specified here.

Negotiating subprotocols with an onion service.

In the hs_ntor handshake sent by the client, we add an encrypted subproto_request extension of the same format, with the same semantics, as used in the ntor-v3 handshake.

Advertising other relays' subprotocols?

Alternative: I had previously considered a design where the introduction points in the onion service descriptor would be listed along with their subprotocols, and the hs_ntor handshake would contain the subprotocols of the rendezvous point.

I'm rejecting this design for now because it gives the onion service and the client too much freedom to lie about relays. In the future, the walking onions design would solve this, since the contact information for intro and rend points would be authenticated.

Appendix

New reserved numbers:

• An extension ID for the ntorv3/hs_ntor handshake subproto_request extension. (This is "3". See proposal 358 for why only one extension ID is needed.)
• A Relay= subprotocol indicating support for the ntor-v3 and hs_ntor extensions. (Relay=5 is reserved for this.)
• The numeric encoding of each existing subprotocol, in the table above. (This is merged into tor-spec.)

Acknowledgments

Thanks to David Goulet and Mike Perry for their feedback on earlier versions of this proposal!