summaryrefslogtreecommitdiff
path: root/scripts/maint/updateFallbackDirs.py
AgeCommit message (Collapse)Author
2017-05-16Update fallback stability requirementsteor
Decrease the minimum number of fallbacks to 100. Final fix to 20913.
2017-05-16Merge branch 'fallbacks-201702-v3' into fallbacks-2017-05teor
2017-05-16Make the usage example in updateFallbackDirs.py actually workteor
And explain what it does. Fixes bug 22270 in 0.3.0.3-alpha.
2017-02-27Modify fallback stability requirementsteor
Increase the fallback stability requirement to 30 days. When this was at 7 days, we chose far too many unstable fallbacks. Decrease the guard flag requirement to 0.8. When this was at 0.9, we lost too many fallbacks due to version upgrades. (The running and v2dir flags ensure DirPorts are available to clients.) Partial fixes to #20913.
2017-01-25Add the output file to the usageSuphanat Chunhapanya
2017-01-25Add usage for check_existingSuphanat Chunhapanya
2017-01-25Logging existing fallbacks at warning levelSuphanat Chunhapanya
2017-01-25Automate checking existing fallbacksSuphanat Chunhapanya
2016-12-19Avoid an error in the fallback script when a fallback doesn't have any uptimeteor
Sometimes, the fallback generation script doesn't add attributes to the fallbacks in the list. If this happens, log an error, and avoid selecting that fallback. This is a rare issue: it should not change selection behaviour. Fixes issue #20945.
2016-12-19Allow fallbacks serving consensuses that expired less than 24 hours agoteor
This works around #20909, where relays serve stale consensuses for a short time, and then recover. Update to the fix for #20539.
2016-12-19Avoid checking fallback candidates' DirPorts if they are down in OnionOOteor
Exclude relays that have been down for 1 or more days from the fallback candidate list. When a relay operator has multiple relays, this prioritises relays that are up over relays that are down. Fixes issue #20926.
2016-12-19Be more tolerant of a decreased number of fallbacksteor
2016-12-19Reduce fallback bandwidth requirement to 1 MByte/steor
2016-12-19Allow 3 fallbacks per operatorteor
This is safe now we are choosing 200 fallbacks. Closes ticket 20912.
2016-12-19Require fallbacks to have 90% Running, V2Dir, and Guard flagsteor
This allows 73% of clients to bootstrap in the first 5 seconds without contacting an authority. Part of #18828.
2016-12-19Require fallback directories to have the same address and port for 7 daysteor
7 days is a tradeoff between the expected time between major Tor releases, which is 6 months, and the number of relays with enough stability. Relays whose OnionOO stability timer is reset on restart by bug #18050 should upgrade to Tor 0.2.8.7 or later, which has a fix for this issue. Closes ticket #20880; maintains short-term fix in e220214 in tor-0.2.8.2-alpha.
2016-12-19Display the fingerprint when downloading consensuses from fallbacksteor
2016-12-19Exclude relays that deliver an expired consensus from the fallback listteor
Part of #20539, based on #20501.
2016-12-19Exclude relay versions affected by #20499 from the fallback listteor
Part of #20539, based on #20509.
2016-12-19Make fallback sort order configurableteor
Closes issue #20882.
2016-12-19Select 200 fallback directories by default for each releaseteor
Closes ticket #20881.
2016-12-19Provide bandwidth and consensus weight for each candidate fallbackteor
And make it clear that updateFallbackDirs.py outputs bandwidth in megabytes per second. Closes #20878.
2016-12-19Add a missed return to fallbackdir_comment() in updateFallbackDirs.pyteor
Closes ticket #20877.
2016-06-28Comment-out fallbacks in a way the stem fallback parser understandsteor (Tim Wilson-Brown)
If we manually remove fallbacks in C by adding '/*' and '*/' on separate lines, stem still parses them as being present, because it only looks at the start of a line. Add a comment to this effect in the generated source code.
2016-06-22Avoid errors in updateFallbackDirs.py when there are no fallbacksteor (Tim Wilson-Brown)
2016-06-22Document how to test the hard-coded fallback listteor (Tim Wilson-Brown)
2016-04-29Simplify string cleansing in fallback update scriptteor (Tim Wilson-Brown)
No behavioural change
2016-04-29Improve logging in fallback update scriptteor (Tim Wilson-Brown)
Log a notice just before the script is about to perform a potentially time-consuming operation Clarify the warning when py2-ipaddress isn't found Make log levels more consistent No behavioural change (just logging)
2016-04-29Improve comments in fallback update scriptteor (Tim Wilson-Brown)
2016-04-26Report fallback directory detail changes when rebuilding listteor (Tim Wilson-Brown)
As well as the existing reports of IPv6 address additions or removals, the script now warns when keys change but IPv4:ORPort or IPv6:IPv6ORPort remain the same. Existing checks for other whitelist detail changes have also been re-worded and upgraded to warnings. This makes it easier for changes to be identified so operators can be contacted to confirm whether the change is stable.
2016-04-15Limit fallbacks from the same operatorteor (Tim Wilson-Brown)
Use IP address, effective family, and contact info to discover and limit fallbacks to one per operator. Also analyse netblock, ports, IP version, and Exit flag, and print the results. Don't exclude any fallbacks from the list because of netblocks, ports, IP version, or Exit flag.
2016-04-15Simplify fallback list outputteor (Tim Wilson-Brown)
When creating the list of fallbacks for a release: * don't output fallback name and contact * sort fallbacks by fingerprint
2016-04-15Select fallbacks by bandwidth rather than consensus weightteor (Tim Wilson-Brown)
But as advertised bandwidth is controlled by relays, use consensus weight and median weight to bandwidth ratio to approximate measured bandwidth. Includes minor comment changes and parameter reordering.
2016-04-15Only count active fallbacks in updateFallbackDirs.pyteor (Tim Wilson-Brown)
Previously, we would cut the list down to 100 fallbacks, then check if they could serve a consensus, and comment them out if they couldn't. This would leave us with fewer than 100 active fallbacks. Now, we stop when there are 100 active fallbacks. Also count fallbacks with identical contact info. Also fix minor logging issues.
2016-04-15Remove fallback directory weightsteor (Tim Wilson-Brown)
Give each fallback a set weight of 10.0 for client selection. Fallbacks must have at least 3000 consensus weight. This is (nominally) 100 times the expected extra load of 20 kilobytes per second (50 GB per month). Fixes issue #17905.
2016-04-15Improve fallback selection and outputteor (Tim Wilson-Brown)
Improve the download test: * Allow IPv4 DirPort checks to be turned off. * Add a timeout to stem's consensus download. * Actually check for download errors, rather than ignoring them. * Simplify the timeout and download error checking logic. Tweak whitelist/blacklist checks to be more robust. Improve logging, make it warn by default. Cleanse fallback comments more thoroughly: * non-printables (yes, ContactInfo can have these) * // comments (don't rely on newlines to prevent // */ escapes)
2016-02-27Update default fallback directories for 0.2.8.2-alpha (Feb 2016)teor (Tim Wilson-Brown)
Allow fallback directories which have been stable for 7 days to work around #18050, which causes relays to submit descriptors with 0 DirPorts when restarted. (Particularly during Tor version upgrades.) Ignore low fallback directory count in alpha builds. Set the target count to 50.
2016-02-27Fix minor fallback directory script issues with time zonesteor (Tim Wilson-Brown)
2016-02-27Check if fallback directories can serve the consensus fast enoughteor (Tim Wilson-Brown)
2016-01-18Fix a display issue in updateFallbackDirs.py outputteor (Tim Wilson-Brown)
2016-01-12Merge remote-tracking branch 'teor/bug17888'Nick Mathewson
2016-01-12Merge remote-tracking branch 'teor/bug17887-v2'Nick Mathewson
2016-01-12Don't limit fallback directory candidates to 500teor (Tim Wilson-Brown)
Only impose the limit when selecting fallback directories for inclusion in tor.
2016-01-12Fix various issues with fallback directory data handlingteor (Tim Wilson-Brown)
* support maximum history age in _avg_generic_history() * fix division-by-zero trap in _avg_generic_history() * skip missing (i.e. null/None) intervals in _avg_generic_history() * Python timedelta.total_seconds() function not available in 2.6; replace with equivalent expression * set DEBUG logging level to make relay exclusion reasons visible * move CUTOFF_GUARD test to end in order to expose more exclusion reasons Patch by "starlight", merge modifications by "teor".
2016-01-12Don't reduce the weight of exits selected as fallback directoriesteor (Tim Wilson-Brown)
When selecting exits as fallback directories, don't reduce their weights. Closes ticket #17888.
2016-01-12Allow fallback directory selection to use day-old datateor (Tim Wilson-Brown)
Allow cached or outdated Onionoo data to be used to choose fallback directories, as long as it's less than a day old. Modify last modified date checks in preparation for Onionoo change
2015-12-16Add Fallback Directory Candidate Selection Scriptteor
"Tor has included a feature to fetch the initial consensus from nodes other than the authorities for a while now. We just haven't shipped a list of alternate locations for clients to go to yet. Reasons why we might want to ship tor with a list of additional places where clients can find the consensus is that it makes authority reachability and BW less important. We want them to have been around and using their current key, address, and port for a while now (120 days), and have been running, a guard, and a v2 directory mirror for most of that time." Features: * whitelist and blacklist for an opt-in/opt-out trial. * excludes BadExits, tor versions that aren't recommended, and low consensus weight directory mirrors. * reduces the weighting of Exits to avoid overloading them. * places limits on the weight of any one fallback. * includes an IPv6 address and orport for each FallbackDir, as implemented in #17327. (Tor won't bootstrap using IPv6 fallbacks until #17840 is merged.) * generated output includes timestamps & Onionoo URL for traceability. * unit test ensures that we successfully load all included default fallback directories. Closes ticket #15775. Patch by "teor". OnionOO script by "weasel", "teor", "gsathya", and "karsten".