diff options
| -rw-r--r-- | README | 2 | ||||
| -rw-r--r-- | doc/HACKING/ReleasingTor.md | 261 | ||||
| -rw-r--r-- | doc/HACKING/ReleasingTor.md.old | 255 |
3 files changed, 325 insertions, 193 deletions
@@ -1,3 +1,5 @@ +[](https://gitlab.torproject.org/tpo/core/tor/-/commits/main) + Tor protects your privacy on the internet by hiding the connection between your Internet address and the services you use. We believe Tor is reasonably secure, but please ensure you read the instructions and diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md index 490c100fcb..12f5d1b16b 100644 --- a/doc/HACKING/ReleasingTor.md +++ b/doc/HACKING/ReleasingTor.md @@ -1,239 +1,114 @@ -# CHECKLIST - -Here's a summary checklist, with the things that Nick messes up most often. - -Did you: - - * [ ] Copy the ChangeLog to the ReleaseNotes? - * [ ] Check that the new versions got approved? - * [ ] Check the release date in the ChangeLog? - * [ ] Update the GeoIP file? - -# Putting out a new release +# How to Release Tor Here are the steps that the maintainer should take when putting out a -new Tor release: - -## 0. Preliminaries - -1. Get at least three of weasel/arma/Sebastian/Sina to put the new - version number in their approved versions list. Give them a few - days to do this if you can. - -2. If this is going to be an important security release, give the packagers - advance warning, via `tor-packagers@lists.torproject.org`. - - -3. Given the release date for Tor, ask the TB team about the likely release - date of a TB that contains it. See note below in "commit, upload, - announce". - -## I. Make sure it works - -1. Make sure that CI passes: have a look at the branches on gitlab. - - _Optionally_, have a look at Travis - (https://travis-ci.org/torproject/tor/branches), Appveyor - (https://ci.appveyor.com/project/torproject/tor/history), and - Jenkins (https://jenkins.torproject.org/view/tor/). - Make sure you're looking at the right branches. - - If there are any unexplained failures, try to fix them or figure them - out. - -2. Verify that there are no big outstanding issues. You might find such - issues -- - - * On Gitlab - - * On coverity scan - - * On OSS-Fuzz - -## II. Write a changelog - - -1a. (Alpha release variant) - - Gather the `changes/*` files into a changelog entry, rewriting many - of them and reordering to focus on what users and funders would find - interesting and understandable. - - To do this, run `./scripts/maint/sortChanges.py changes/* > changelog.in` - to combine headings and sort the entries. Copy the changelog.in file into - the ChangeLog. Run `format_changelog.py --inplace` (see below) to clean up - the line breaks. - - Remove the `changes/*` files that you just merged into the ChangeLog. - - After that, it's time to hand-edit and fix the issues that - lintChanges can't find: - - 1. Within each section, sort by "version it's a bugfix on", else by - numerical ticket order. - - 2. Clean them up: - - Make stuff very terse - - Describe the user-visible problem right away - - Mention relevant config options by name. If they're rare or unusual, - remind people what they're for - - Avoid starting lines with open-paren - - Present and imperative tense: not past. - - "Relays", not "servers" or "nodes" or "Tor relays". - - "Onion services", not "hidden services". +new Tor release. It is split in 3 stages and coupled with our Tor CI Release +pipeline. - "Stop FOOing", not "Fix a bug where we would FOO". +Before we begin, first rule is to make sure: - Try not to let any given section be longer than about a page. Break up - long sections into subsections by some sort of common subtopic. This - guideline is especially important when organizing Release Notes for - new stable releases. + - Our CI pass for each version to release + - Coverity has no new alerts - If a given changes stanza showed up in a different release (e.g. - maint-0.2.1), be sure to make the stanzas identical (so people can - distinguish if these are the same change). +## 0. Security Release - 3. Clean everything one last time. +To start with, if you are doing a security release, this must be done few days +prior to the release: - 4. Run `./scripts/maint/format_changelog.py --inplace` to make it prettier + 1. If this is going to be an important security release, give the packagers + advance warning, via `tor-packagers@lists.torproject.org`. -1b. (old-stable release variant) - For stable releases that backport things from later, we try to compose - their releases, we try to make sure that we keep the changelog entries - identical to their original versions, with a "backport from 0.x.y.z" - note added to each section. So in this case, once you have the items - from the changes files copied together, don't use them to build a new - changelog: instead, look up the corrected versions that were merged - into ChangeLog in the main branch, and use those. +## 1. Preliminaries - Add "backport from X.Y.Z" in the section header for these entries. +The following must be done 2 days at the very least prior to the release: -2. Compose a short release blurb to highlight the user-facing - changes. Insert said release blurb into the ChangeLog stanza. If it's - a stable release, add it to the ReleaseNotes file too. If we're adding - to a release-* branch, manually commit the changelogs to the later - git branches too. + 1. Add the version(s) in the dirauth-conf git repository as the + RecommendedVersion and RequiredVersion so they can be approved by the + authorities and be in the consensus before the release. -3. If there are changes that require or suggest operator intervention - before or during the update, mail operators (either dirauth or relays - list) with a headline that indicates that an action is required or - appreciated. + 2. Send a pre-release announcement to `tor-project@lists.torproject.org` in + order to inform every teams in Tor of the upcoming release. This is so + we can avoid creating release surprises and sync with other teams. -4. If you're doing the first stable release in a series, you need to - create a ReleaseNotes for the series as a whole. To get started - there, copy all of the Changelog entries from the series into a new - file, and run `./scripts/maint/sortChanges.py` on it. That will - group them by category. Then kill every bugfix entry for fixing - bugs that were introduced within that release series; those aren't - relevant changes since the last series. At that point, it's time - to start sorting and condensing entries. (Generally, we don't edit the - text of existing entries, though.) + 3. Ask the network-team to review the `changes/` files in all versions we + are about to release. This step is encouraged but not mandatory. -## III. Making the source release. -1. In `maint-0.?.x`, bump the version number in `configure.ac` and run - `./scripts/main/update_versions.py` to update version numbers in other - places, and commit. Then merge `maint-0.?.x` into `release-0.?.x`. +## 2. Tarballs - When you merge the maint branch forward to the next maint branch, or into - main, merge it with `-s ours` to avoid conflict with the version - bump. +To build the tarballs to release, we need to launch the CI release pipeline: -2. In `release-0.?.x`, run `make distcheck`, put the tarball up in somewhere - (how about your homedir on people.torproject.org?) , and tell `#tor-dev` - about it. + https://gitlab.torproject.org/tpo/core/tor-ci-release - If you want, wait until at least one person has built it - successfully. (We used to say "wait for others to test it", but our - CI has successfully caught these kinds of errors for the last several - years.) +The `versions.yml` needs to be modified with the Tor versions you want to +release. Once done, git commit and push to trigger the release pipeline. -3. Make sure that the new version is recommended in the latest consensus. - (Otherwise, users will get confused when it complains to them - about its status.) +The first two stages (Preliminary and Patches) will be run automatically. The +Build stage needs to be triggered manually once all generated patches have +been merged upstream. - If it is not, you'll need to poke Roger, Weasel, Sebastian, and Sina - again: see the note at the start of the document. + 1. Download the generated patches from the `Patches` stage. -## IV. Commit, upload, announce + 2. For the ChangeLog and ReleaseNotes, you need to write a blurb at the top + explaining a bit the release. -1. Sign the tarball, then sign and push the git tag: + 3. Review, modify if needed, and merged them upstream. -```console -$ gpg -ba <the_tarball> -$ git tag -s tor-0.4.x.y-<status> -$ git push origin tag tor-0.4.x.y-<status> -``` + 4. Manually trigger the `maintained` job in the `Build` stage so the CI can + build the tarballs without errors. - (You must do this before you update the website: the website scripts - rely on finding the version by tag.) +Once this is done, each selected developers need to build the tarballs in a +reproducible way using: - (If your default PGP key is not the one you want to sign with, then say - "-u <keyid>" instead of "-s".) + https://gitlab.torproject.org/tpo/core/tor-ci-reproducible -2. scp the tarball and its sig to the dist website, i.e. - `/srv/dist-master.torproject.org/htdocs/` on dist-master. Run - "static-update-component dist.torproject.org" on dist-master. +Simply run the `./build.sh` which will commit interactively the signature for +you. You then only need to git push. - In the `project/web/tpo.git` repository, update `databags/versions.ini` - to note the new version. Push these changes to `master`. +Once all signatures have been committed: - (NOTE: Due to #17805, there can only be one stable version listed at - once. Nonetheless, do not call your version "alpha" if it is stable, - or people will get confused.) + 1. Manually trigger the `signature` job in the `Post-process` stage of the + CI release pipeline. - (NOTE: It will take a while for the website update scripts to update - the website.) + 2. If it passes, the tarball(s) and signature(s) will be available as + artifacts and should be used for the release. -3. Email the tor-packagers@lists.torproject.org mailing list to tell them - about the new release. + 3. Put them on `dist.torproject.org`: - Also, email tor-packagers@lists.torproject.org. + Upload the tarball and its sig to the dist website, i.e. + `/srv/dist-master.torproject.org/htdocs/` on dist-master. Run + "static-update-component dist.torproject.org" on dist-master. - Mention where to download the tarball (`https://dist.torproject.org/`). + In the `project/web/tpo.git` repository, update `databags/versions.ini` + to note the new version. Push these changes to `master`. - Include a link to the changelog. + (NOTE: Due to #17805, there can only be one stable version listed at once. + Nonetheless, do not call your version "alpha" if it is stable, or people + will get confused.) -4. Wait for the download page to be updated. (If you don't do this before you - announce, people will be confused.) + (NOTE: It will take a while for the website update scripts to update the + website.) -5. Mail the release blurb and ChangeLog to tor-talk (development release) or - tor-announce (stable). - Post the changelog on the blog as well. You can generate a - blog-formatted version of the changelog with - `./scripts/maint/format_changelog.py -B` +## 3. Post Process - When you post, include an estimate of when the next TorBrowser - releases will come out that include this Tor release. This will - usually track https://wiki.mozilla.org/RapidRelease/Calendar , but it - can vary. +Once the tarballs have been uploaded and are ready to be announced, we need to +do the following: - For templates to use when announcing, see: - https://gitlab.torproject.org/tpo/core/team/-/wikis/NetworkTeam/AnnouncementTemplates + 1. Merge upstream the artifacts from the `patches` job in the + `Post-process` stage of the CI release pipeline. -## V. Aftermath and cleanup + 2. Write and post the release announcement for the `forum.torproject.net` + in the `News -> Tor Release Announcement` category. -1. If it's a stable release, bump the version number in the - `maint-x.y.z` branch to "newversion-dev", and do a `merge -s ours` - merge to avoid taking that change into main. + Mention in which Tor Browser version (with dates) the release will be + in. This usually only applies to the latest stable. -2. If there is a new `maint-x.y.z` branch, create a Travis CI cron job that - builds the release every week. (It's ok to skip the weekly build if the - branch was updated in the last 24 hours.) +### New Stable -3. Forward-port the ChangeLog (and ReleaseNotes if appropriate) to the - main branch. + 1. Create the `maint-x.y.z` and `release-x.y.z` branches and update the + `./scripts/git/git-list-tor-branches.sh` with the new version. -4. Keep an eye on the blog post, to moderate comments and answer questions. ## Appendix: An alternative means to notify packagers diff --git a/doc/HACKING/ReleasingTor.md.old b/doc/HACKING/ReleasingTor.md.old new file mode 100644 index 0000000000..490c100fcb --- /dev/null +++ b/doc/HACKING/ReleasingTor.md.old @@ -0,0 +1,255 @@ +# CHECKLIST + +Here's a summary checklist, with the things that Nick messes up most often. + +Did you: + + * [ ] Copy the ChangeLog to the ReleaseNotes? + * [ ] Check that the new versions got approved? + * [ ] Check the release date in the ChangeLog? + * [ ] Update the GeoIP file? + +# Putting out a new release + +Here are the steps that the maintainer should take when putting out a +new Tor release: + +## 0. Preliminaries + +1. Get at least three of weasel/arma/Sebastian/Sina to put the new + version number in their approved versions list. Give them a few + days to do this if you can. + +2. If this is going to be an important security release, give the packagers + advance warning, via `tor-packagers@lists.torproject.org`. + + +3. Given the release date for Tor, ask the TB team about the likely release + date of a TB that contains it. See note below in "commit, upload, + announce". + +## I. Make sure it works + +1. Make sure that CI passes: have a look at the branches on gitlab. + + _Optionally_, have a look at Travis + (https://travis-ci.org/torproject/tor/branches), Appveyor + (https://ci.appveyor.com/project/torproject/tor/history), and + Jenkins (https://jenkins.torproject.org/view/tor/). + Make sure you're looking at the right branches. + + If there are any unexplained failures, try to fix them or figure them + out. + +2. Verify that there are no big outstanding issues. You might find such + issues -- + + * On Gitlab + + * On coverity scan + + * On OSS-Fuzz + +## II. Write a changelog + + +1a. (Alpha release variant) + + Gather the `changes/*` files into a changelog entry, rewriting many + of them and reordering to focus on what users and funders would find + interesting and understandable. + + To do this, run `./scripts/maint/sortChanges.py changes/* > changelog.in` + to combine headings and sort the entries. Copy the changelog.in file into + the ChangeLog. Run `format_changelog.py --inplace` (see below) to clean up + the line breaks. + + Remove the `changes/*` files that you just merged into the ChangeLog. + + After that, it's time to hand-edit and fix the issues that + lintChanges can't find: + + 1. Within each section, sort by "version it's a bugfix on", else by + numerical ticket order. + + 2. Clean them up: + + Make stuff very terse + + Describe the user-visible problem right away + + Mention relevant config options by name. If they're rare or unusual, + remind people what they're for + + Avoid starting lines with open-paren + + Present and imperative tense: not past. + + "Relays", not "servers" or "nodes" or "Tor relays". + + "Onion services", not "hidden services". + + "Stop FOOing", not "Fix a bug where we would FOO". + + Try not to let any given section be longer than about a page. Break up + long sections into subsections by some sort of common subtopic. This + guideline is especially important when organizing Release Notes for + new stable releases. + + If a given changes stanza showed up in a different release (e.g. + maint-0.2.1), be sure to make the stanzas identical (so people can + distinguish if these are the same change). + + 3. Clean everything one last time. + + 4. Run `./scripts/maint/format_changelog.py --inplace` to make it prettier + +1b. (old-stable release variant) + + For stable releases that backport things from later, we try to compose + their releases, we try to make sure that we keep the changelog entries + identical to their original versions, with a "backport from 0.x.y.z" + note added to each section. So in this case, once you have the items + from the changes files copied together, don't use them to build a new + changelog: instead, look up the corrected versions that were merged + into ChangeLog in the main branch, and use those. + + Add "backport from X.Y.Z" in the section header for these entries. + +2. Compose a short release blurb to highlight the user-facing + changes. Insert said release blurb into the ChangeLog stanza. If it's + a stable release, add it to the ReleaseNotes file too. If we're adding + to a release-* branch, manually commit the changelogs to the later + git branches too. + +3. If there are changes that require or suggest operator intervention + before or during the update, mail operators (either dirauth or relays + list) with a headline that indicates that an action is required or + appreciated. + +4. If you're doing the first stable release in a series, you need to + create a ReleaseNotes for the series as a whole. To get started + there, copy all of the Changelog entries from the series into a new + file, and run `./scripts/maint/sortChanges.py` on it. That will + group them by category. Then kill every bugfix entry for fixing + bugs that were introduced within that release series; those aren't + relevant changes since the last series. At that point, it's time + to start sorting and condensing entries. (Generally, we don't edit the + text of existing entries, though.) + +## III. Making the source release. + +1. In `maint-0.?.x`, bump the version number in `configure.ac` and run + `./scripts/main/update_versions.py` to update version numbers in other + places, and commit. Then merge `maint-0.?.x` into `release-0.?.x`. + + When you merge the maint branch forward to the next maint branch, or into + main, merge it with `-s ours` to avoid conflict with the version + bump. + +2. In `release-0.?.x`, run `make distcheck`, put the tarball up in somewhere + (how about your homedir on people.torproject.org?) , and tell `#tor-dev` + about it. + + If you want, wait until at least one person has built it + successfully. (We used to say "wait for others to test it", but our + CI has successfully caught these kinds of errors for the last several + years.) + +3. Make sure that the new version is recommended in the latest consensus. + (Otherwise, users will get confused when it complains to them + about its status.) + + If it is not, you'll need to poke Roger, Weasel, Sebastian, and Sina + again: see the note at the start of the document. + +## IV. Commit, upload, announce + +1. Sign the tarball, then sign and push the git tag: + +```console +$ gpg -ba <the_tarball> +$ git tag -s tor-0.4.x.y-<status> +$ git push origin tag tor-0.4.x.y-<status> +``` + + (You must do this before you update the website: the website scripts + rely on finding the version by tag.) + + (If your default PGP key is not the one you want to sign with, then say + "-u <keyid>" instead of "-s".) + +2. scp the tarball and its sig to the dist website, i.e. + `/srv/dist-master.torproject.org/htdocs/` on dist-master. Run + "static-update-component dist.torproject.org" on dist-master. + + In the `project/web/tpo.git` repository, update `databags/versions.ini` + to note the new version. Push these changes to `master`. + + (NOTE: Due to #17805, there can only be one stable version listed at + once. Nonetheless, do not call your version "alpha" if it is stable, + or people will get confused.) + + (NOTE: It will take a while for the website update scripts to update + the website.) + +3. Email the tor-packagers@lists.torproject.org mailing list to tell them + about the new release. + + Also, email tor-packagers@lists.torproject.org. + + Mention where to download the tarball (`https://dist.torproject.org/`). + + Include a link to the changelog. + +4. Wait for the download page to be updated. (If you don't do this before you + announce, people will be confused.) + +5. Mail the release blurb and ChangeLog to tor-talk (development release) or + tor-announce (stable). + + Post the changelog on the blog as well. You can generate a + blog-formatted version of the changelog with + `./scripts/maint/format_changelog.py -B` + + When you post, include an estimate of when the next TorBrowser + releases will come out that include this Tor release. This will + usually track https://wiki.mozilla.org/RapidRelease/Calendar , but it + can vary. + + For templates to use when announcing, see: + https://gitlab.torproject.org/tpo/core/team/-/wikis/NetworkTeam/AnnouncementTemplates + +## V. Aftermath and cleanup + +1. If it's a stable release, bump the version number in the + `maint-x.y.z` branch to "newversion-dev", and do a `merge -s ours` + merge to avoid taking that change into main. + +2. If there is a new `maint-x.y.z` branch, create a Travis CI cron job that + builds the release every week. (It's ok to skip the weekly build if the + branch was updated in the last 24 hours.) + +3. Forward-port the ChangeLog (and ReleaseNotes if appropriate) to the + main branch. + +4. Keep an eye on the blog post, to moderate comments and answer questions. + +## Appendix: An alternative means to notify packagers + +If for some reason you need to contact a bunch of packagers without +using the publicly archived tor-packagers list, you can try these +people: + + - {weasel,sysrqb,mikeperry} at torproject dot org + - {blueness} at gentoo dot org + - {paul} at invizbox dot io + - {vincent} at invizbox dot com + - {lfleischer} at archlinux dot org + - {Nathan} at freitas dot net + - {mike} at tig dot as + - {tails-rm} at boum dot org + - {simon} at sdeziel.info + - {yuri} at freebsd.org + - {mh+tor} at scrit.ch + - {security} at brave.com |