diff options
Diffstat (limited to 'spec/bandwidth-file-spec/implementation-details.md')
| -rw-r--r-- | spec/bandwidth-file-spec/implementation-details.md | 398 |
1 files changed, 398 insertions, 0 deletions
diff --git a/spec/bandwidth-file-spec/implementation-details.md b/spec/bandwidth-file-spec/implementation-details.md new file mode 100644 index 0000000..dbc02cc --- /dev/null +++ b/spec/bandwidth-file-spec/implementation-details.md @@ -0,0 +1,398 @@ +<a id="bandwidth-file-spec.txt-2.4"></a> + +# Implementation details + +<a id="bandwidth-file-spec.txt-2.4.1"></a> + +## Writing bandwidth files atomically { #write-atomically } + +To avoid inconsistent reads, implementations SHOULD write bandwidth files +atomically. If the file is transferred from another host, it SHOULD be +written to a temporary path, then renamed to the V3BandwidthsFile path. + +sbws versions 0.7.0 and later write the bandwidth file to an archival +location, create a temporary symlink to that location, then atomically rename +the symlink +to the configured V3BandwidthsFile path. + +Torflow does not write bandwidth files atomically. + +<a id="bandwidth-file-spec.txt-2.4.2"></a> + +## Additional KeyValue pair definitions { #key-value-pairs } + +KeyValue pairs in RelayLines that current implementations generate. + +<a id="bandwidth-file-spec.txt-2.4.2.1"></a> + +### Simple Bandwidth Scanner { #sbws } + +sbws RelayLines contain these keys: + +"node_id" hexdigest + +As above. + +"bw" Bandwidth + +As above. + +"nick" nickname + +\[Exactly once.\] + +The relay nickname. + +Torflow also has a "nick" KeyValue. + +"rtt" Int + +\[Zero or one time.\] + +The Round Trip Time in milliseconds to obtain 1 byte of data. + +This KeyValue was added in version 1.1.0 of this specification. +It became optional in version 1.3.0 or 1.4.0 of this specification. + +"time" DateTime + +\[Exactly once.\] + +The date and time timestamp in ISO 8601 format and UTC time zone +when the last bandwidth was obtained. + +This KeyValue was added in version 1.1.0 of this specification. +The Torflow equivalent is "measured_at". + +"success" Int + +\[Zero or one time.\] + +The number of times that the bandwidth measurements for this relay were +successful. + +This KeyValue was added in version 1.1.0 of this specification. + +"error_circ" Int + +\[Zero or one time.\] + +The number of times that the bandwidth measurements for this relay +failed because of circuit failures. + +This KeyValue was added in version 1.1.0 of this specification. +The Torflow equivalent is "circ_fail". + +"error_stream" Int + +\[Zero or one time.\] + +The number of times that the bandwidth measurements for this relay +failed because of stream failures. + +This KeyValue was added in version 1.1.0 of this specification. + +"error_destination" Int + +\[Zero or one time.\] + +The number of times that the bandwidth measurements for this relay +failed because the destination Web server was not available. + +This KeyValue was added in version 1.4.0 of this specification. + +"error_second_relay" Int + +\[Zero or one time.\] + +The number of times that the bandwidth measurements for this relay +failed because sbws could not find a second relay for the test circuit. + +This KeyValue was added in version 1.4.0 of this specification. + +"error_misc" Int + +\[Zero or one time.\] + +The number of times that the bandwidth measurements for this relay +failed because of other reasons. + +This KeyValue was added in version 1.1.0 of this specification. + +"bw_mean" Int + +\[Zero or one time.\] + +The measured bandwidth mean for this relay in bytes per second. + +This KeyValue was added in version 1.2.0 of this specification. + +"bw_median" Int + +\[Zero or one time.\] + +The measured bandwidth median for this relay in bytes per second. + +This KeyValue was added in version 1.2.0 of this specification. + +"desc_bw_avg" Int + +\[Zero or one time.\] + +The descriptor average bandwidth for this relay in bytes per second. + +This KeyValue was added in version 1.2.0 of this specification. + +"desc_bw_obs_last" Int + +\[Zero or one time.\] + +The last descriptor observed bandwidth for this relay in bytes per +second. + +This KeyValue was added in version 1.2.0 of this specification. + +"desc_bw_obs_mean" Int + +\[Zero or one time.\] + +The descriptor observed bandwidth mean for this relay in bytes per +second. + +This KeyValue was added in version 1.2.0 of this specification. + +"desc_bw_bur" Int + +\[Zero or one time.\] + +The descriptor burst bandwidth for this relay in bytes per +second. + +This KeyValue was added in version 1.2.0 of this specification. + +"consensus_bandwidth" Int + +\[Zero or one time.\] + +The consensus bandwidth for this relay in bytes per second. + +This KeyValue was added in version 1.2.0 of this specification. + +"consensus_bandwidth_is_unmeasured" Bool + +\[Zero or one time.\] + +If the consensus bandwidth for this relay was not obtained from +three or more bandwidth authorities, this KeyValue is True or +False otherwise. + +This KeyValue was added in version 1.2.0 of this specification. + +"relay_in_recent_consensus_count" Int + +\[Zero or one time.\] + +The number of times this relay was found in a consensus in the +last data_period days. (Unless otherwise stated, data_period is +5 by default.) + +This KeyValue was added in version 1.4.0 of this specification. + +"relay_recent_priority_list_count" Int + +\[Zero or one time.\] + +The number of times this relay has been prioritized to be measured +in the last data_period days. + +This KeyValue was added in version 1.4.0 of this specification. + +"relay_recent_measurement_attempt_count" Int + +\[Zero or one time.\] + +The number of times this relay was tried to be measured in the +last data_period days. + +This KeyValue was added in version 1.4.0 of this specification. + +"relay_recent_measurement_failure_count" Int + +\[Zero or one time.\] + +The number of times this relay was tried to be measured in the +last data_period days, but it was not possible to obtain a +measurement. + +This KeyValue was added in version 1.4.0 of this specification. + +"relay_recent_measurements_excluded_error_count" Int + +\[Zero or one time.\] + +The number of recent relay measurement attempts that failed. +Measurements are recent if they are in the last data_period days +(5 by default). + +(See the note in section 1.4, version 1.4.0, about excluded relays.) + +This KeyValue was added in version 1.4.0 of this specification. + +"relay_recent_measurements_excluded_near_count" Int + +\[Zero or one time.\] + +When all of a relay's recent successful measurements were performed in +a period of time that was too short (by default 1 day), the relay is +excluded. This KeyValue contains the number of recent successful +measurements for the relay that were ignored for this reason. + +(See the note in section 1.4, version 1.4.0, about excluded relays.) + +This KeyValue was added in version 1.4.0 of this specification. + +"relay_recent_measurements_excluded_old_count" Int + +\[Zero or one time.\] + +The number of successful measurements for this relay that are too old +(more than data_period days, 5 by default). + +Excludes measurements that are already counted in +relay_recent_measurements_excluded_near_count. + +(See the note in section 1.4, version 1.4.0, about excluded relays.) + +This KeyValue was added in version 1.4.0 of this specification. + +"relay_recent_measurements_excluded_few_count" Int + +\[Zero or one time.\] + +The number of successful measurements for this relay that were ignored +because the relay did not have enough successful measurements (fewer +than 2, by default). + +Excludes measurements that are already counted in +relay_recent_measurements_excluded_near_count or +relay_recent_measurements_excluded_old_count. + +(See the note in section 1.4, version 1.4.0, about excluded relays.) + +This KeyValue was added in version 1.4.0 of this specification. + +"under_min_report" bool + +\[Zero or one time.\] + +If the value is 1, there are not enough eligible relays in the +bandwidth file, and Tor bandwidth authorities MAY NOT vote on this +relay. (Current Tor versions do not change their behaviour based on +the "under_min_report" key.) + +If the value is 0 or the KeyValue is not present, there are enough +relays in the bandwidth file. + +Because Tor versions released before April 2019 (see section 1.4. for +the full list of versions) ignore "vote=0", generator implementations +MUST NOT change the bandwidths for under_min_report relays. Using the +same bw value makes authorities that do not understand "vote=0" +or "under_min_report=1" produce votes that don't change relay weights +too much. It also avoids flapping when the reporting threshold is +reached. + +This KeyValue was added in version 1.4.0 of this specification. + +"unmeasured" bool + +\[Zero or one time.\] + +If the value is 1, this relay was not successfully measured and +Tor bandwidth authorities MAY NOT vote on this relay. +(Current Tor versions do not change their behaviour based on +the "unmeasured" key.) + +If the value is 0 or the KeyValue is not present, this relay +was successfully measured. + +Because Tor versions released before April 2019 (see section 1.4. for +the full list of versions) ignore "vote=0", generator implementations +MUST set "bw=1" for unmeasured relays. Using the minimum bw value +makes authorities that do not understand "vote=0" or "unmeasured=1" +produce votes that don't change relay weights too much. + +This KeyValue was added in version 1.4.0 of this specification. + +"vote" bool + +\[Zero or one time.\] + +If the value is 0, Tor directory authorities SHOULD ignore the relay's +entry in the bandwidth file. They SHOULD vote for the relay the same +way they would vote for a relay that is not present in the file. + +This MAY be the case when this relay was not successfully measured but +it is included in the Bandwidth File, to diagnose why they were not +measured. + +If the value is 1 or the KeyValue is not present, Tor directory +authorities MUST use the relay's bw value in any votes for that relay. + +Implementations MUST also set "bw=1" for unmeasured relays. +But they MUST NOT change the bw for under_min_report relays. +(See the explanations under "unmeasured" and "under_min_report" +for more details.) + +This KeyValue was added in version 1.4.0 of this specification. + +"xoff_recv" Int + +\[Zero or one time.\] + +The number of times this relay received `XOFF_RECV` stream events while +being measured in the last data_period days. + +This KeyValue was added in version 1.6.0 of this specification. + +"xoff_sent" Int + +\[Zero or one time.\] + +The number of times this relay received `XOFF_SENT` stream events while +being measured in the last data_period days. + +This KeyValue was added in version 1.6.0 of this specification. + +"r_strm" Float + +\[Zero or one time.\] + +The stream ratio of this relay calculated as explained in B4.3. + +This KeyValue was added in version 1.7.0 of this specification. + +"r_strm_filt" Float + +\[Zero or one time.\] + +The filtered stream ratio of this relay calculated as explained in B4.3. + +This KeyValue was added in version 1.7.0 of this specification. + +<a id="bandwidth-file-spec.txt-2.4.2.2"></a> + +### Torflow + +Torflow RelayLines include node_id and bw, and other KeyValue pairs \[2\]. + +References: + + +1. <https://gitlab.torproject.org/tpo/network-health/torflow> +2. <https://gitlab.torproject.org/tpo/network-health/torflow/-/blob/main/NetworkScanners/BwAuthority/README.spec.txt?ref_type=heads#L332> + The Torflow specification is outdated, and does not match the current + implementation. See section A.1. for the format produced by Torflow. +3. [The Tor Directory Protocol](../dir-spec) +4. [How Tor Version Numbers Work In Tor](../version-spec.md) +5. <https://semver.org/> +``` |