Add proposal 314 (markdown) and 315 (required fields.)

Also tweak reindex.py under the assumption that we will be accepting
proposal 314.

4 files changed, 158 insertions, 3 deletions

diff --git a/proposals/000-index.txt b/proposals/000-index.txt
index 7706b2a..de45e48 100644
--- a/proposals/000-index.txt
+++ b/proposals/000-index.txt
@@ -234,6 +234,8 @@ Proposals by number:
 311  Tor Relay IPv6 Reachability [DRAFT]
 312  Tor Relay Automatic IPv6 Address Discovery [DRAFT]
 313  Tor Relay IPv6 Statistics [DRAFT]
+314  Allow Markdown for proposal format [OPEN]
+315  Updating the list of fields required in directory documents [OPEN]
 
 
 Proposals by status:
@@ -274,6 +276,8 @@ Proposals by status:
    299  Preferring IPv4 or IPv6 based on IP Version Failure Count
    306  A Tor Implementation of IPv6 Happy Eyeballs
    310  Towards load-balancing in Prop 271
+   314  Allow Markdown for proposal format
+   315  Updating the list of fields required in directory documents
  ACCEPTED:
    188  Bridge Guards and other anti-enumeration defenses
    249  Allow CREATE cells with >505 bytes of handshake data
diff --git a/proposals/314-allow-markdown-proposals.md b/proposals/314-allow-markdown-proposals.md
new file mode 100644
index 0000000..2d2f653
--- /dev/null
+++ b/proposals/314-allow-markdown-proposals.md
@@ -0,0 +1,38 @@
+```
+Filename: 314-allow-markdown-proposals.md
+Title: Allow Markdown for proposal format.
+Author: Nick Mathewson
+Created: 23 April 2020
+Status: Open
+```
+
+# Introduction
+
+This document proposes a change in our proposal format: to allow
+Markdown.
+
+## Motivation
+
+Many people, particularly researchers, have found it difficult to
+write text in the format that we prefer.  Moreover, we have often
+wanted to add more formatting in proposals, and found it nontrivial
+to do so.
+
+Markdown is an emerging "standard" (albeit not actually a
+standardized one), and we're using it in several other places.  It
+seems like a natural fit for our purposes here.
+
+# Details
+
+We should pick a particular Markdown dialect.  "CommonMark" seems like a
+good choice, since it's the basis of what github and gitlab use.
+
+We should also pick a particular tool to use for validating Markdown
+proposals.
+
+We should continue to allow text proposals.
+
+We should continue to require headers for our proposals, and do so
+using the format at the head of this document: wrapping the headers
+inside triple backticks.
+
diff --git a/proposals/315-update-dir-required-fields.txt b/proposals/315-update-dir-required-fields.txt
new file mode 100644
index 0000000..1ee2e54
--- /dev/null
+++ b/proposals/315-update-dir-required-fields.txt
@@ -0,0 +1,111 @@
+Filename: 315-update-dir-required-fields.txt
+Title: Updating the list of fields required in directory documents
+Author: Nick Mathewson
+Created: 23 April 2020
+Status: Open
+
+1. Introduction
+
+   When we add a new field to a directory document, we must at first
+   describe it as "optional", since older Tor implementations will
+   not generate it.  When those implementations are obsolete and
+   unsupported, however, we can safely describe those fields as
+   "required", since they are always included in practice.
+
+   Making fields required is not just a matter of bookkeeping: it
+   helps prevent bugs in two ways.  First, it simplifies our code.
+   Second, it makes our code's requirements match our assumptions
+   about the network.
+
+   Here I'll describe a general policy for making fields required
+   when LTS versions become unsupported, and include a list of
+   fields that should become required today.
+
+   This document does not require to us to make all optional fields
+   required -- only those which we intend that all Tor instances
+   should always generate and expect.
+
+   When we speak of making a field "required", we are talking about
+   describing it as "required" in dir-spec.txt, so that any document
+   missing that field is no longer considered well-formed.
+
+2. When fields should become required
+
+   We have three relevant kinds of directory documents: those
+   generated by relays, those generated by authorities, and those
+   generated by onion services.
+
+   Relays generate extrainfo documents and routerdesc documents.
+   For these, we can safely make a field required when it is always
+   generated by all relay versions that the authorities allow to
+   join the network.  To avoid partitioning, authorities should
+   start requiring the field before any relays or clients do.
+
+   (If a relay field indicates the presence of a now-required
+   feature, then instead of making the field mandatory, we may
+   change the semantics so that the field is assumed to be
+   present. Later we can remove the option.)
+
+   Authorities generate authority certificates, votes, consensus
+   documents, and microdescriptors.  For these, we can safely make a
+   field required once all authorities are generating it, and we are
+   confident that we do not plan to downgrade those authorities.
+
+   Onion services generate service descriptors.  Because of the risk
+   of partitioning attacks, we should not make features in service
+   descriptors required without a phased process, described in the
+   following section.
+
+2.1. Phased addition of onion service descriptor changes
+
+   Phase one: we add client and service support for the new field,
+   but have this support disabled by default. By default, services
+   should not generate the new field, and clients should not parse
+   it when it is present.  This behavior is controlled by a pair of
+   network parameters.  (If the feature is at all complex, the
+   network parameters should describe a _minimum version_ that
+   should enable the feature, so that we can later enable it only in
+   the versions where the feature is not buggy.)
+
+   During this phase, we can manually override the defaults on
+   particular clients and services to test the new field.
+
+   Phase two: authorities use the network parameters to enable the
+   client support and the service support.  They should only do this
+   once enough clients and services have upgraded to a version that
+   supports the feature.
+
+   Phase three: once all versions that support the feature are
+   obsolete and unsupported, the feature may be marked as required
+   in the specifications, and the network parameters ignored.
+
+   Phase four: once all versions that used the network parameters
+   are obsolete and unsupported, authorities may stop including
+   those parameters in their votes.
+
+3. Directory fields that should become required.
+
+   These fields in router descriptors should become required:
+      * identity-ed25519
+      * master-key-ed25519
+      * onion-key-crosscert
+      * ntor-onion-key
+      * ntor-onion-key-crosscert
+      * router-sig-ed25519
+      * proto
+
+   These fields in router descriptors should become "assumed present":
+      * hidden-service-dir
+
+   These fields in extra-info documents should become required:
+      * identity-ed25519
+      * router-sig-ed25519
+
+   The following fields in microdescriptors should become
+   required:
+      * ntor-onion-key
+
+   The following fields in votes and consensus documents should
+   become required:
+      * pr
+
diff --git a/proposals/reindex.py b/proposals/reindex.py
index 1f52466..361ea16 100755
--- a/proposals/reindex.py
+++ b/proposals/reindex.py
@@ -37,10 +37,12 @@ def readProposal(fn):
                 return fields
             if line[0].isspace():
                 fields[lastField] += " %s"%(line.strip())
+            elif line == "```":
+                pass
             else:
                 parts = line.split(":", 1)
                 if len(parts) != 2:
-                    raise Error("%s:%s:  Neither field nor continuation"%
+                    raise Error("%s:%s:  Neither field, continuation, nor ```."%
                                 (fn,lineno))
                 else:
                     fields[parts[0]] = parts[1].strip()
@@ -93,8 +95,8 @@ def readProposals():
     for fn in os.listdir(DIR):
         m = FNAME_RE.match(fn)
         if not m: continue
-        if not fn.endswith(".txt"):
-            raise Error("%s doesn't end with .txt"%fn)
+        if not (fn.endswith(".txt") or fn.endswith(".md")):
+            raise Error("%s doesn't end with .txt or .md"%fn)
         num = m.group(1)
         fields = readProposal(fn)
         checkProposal(fn, fields)