Merge branch 'writing_tests'

1 files changed, 167 insertions, 0 deletions

diff --git a/doc/WritingTests.txt b/doc/WritingTests.txt
new file mode 100644
index 0000000000..ab6d084ff9
--- /dev/null
+++ b/doc/WritingTests.txt
@@ -0,0 +1,167 @@
+
+Writing tests for Tor: an incomplete guide
+==========================================
+
+Tor uses a variety of testing frameworks and methodologies to try to
+keep from introducing bugs.  The major ones are:
+
+   1. Unit tests written in C and shipped with the Tor distribution.
+
+   2. Integration tests written in Python and shipped with the Tor
+      distribution.
+
+   3. Integration tests written in Python and shipped with the Stem
+      library.  Some of these use the Tor controller protocol.
+
+   4. System tests written in Python and SH, and shipped with the
+      Chutney package.  These work by running many instances of Tor
+      locally, and sending traffic through them.
+
+   5. The Shadow network simulator.
+
+How to run these tests
+----------------------
+
+=== The easy version
+
+To run all the tests that come bundled with Tor, run "make check"
+
+To run the Stem tests as well, fetch stem from the git repository,
+set STEM_SOURCE_DIR to the checkout, and run "make test-stem".
+
+To run the Chutney tests as well, fetch chutney from the git repository,
+set CHUTNEY_PATH to the checkout, and run "make test-network".
+
+=== Running particular subtests
+
+XXXX WRITEME
+
+=== Finding test coverage
+
+When you configure Tor with the --enable-coverage option, it should
+build with support for coverage in the unit tests, and in a special
+"tor-cov" binary.  If you launch
+
+XXXX "make test-network" doesn't know about "tor-cov"; you don't get
+XXXX coverage from that yet, unless you do "cp src/or/tor-cov
+XXXX src/or/tor" before you run it.
+
+What kinds of test should I write?
+----------------------------------
+
+XXXX writeme.
+
+
+Unit and regression tests: Does this function do what it's supposed to?
+-----------------------------------------------------------------------
+
+Most of Tor's unit tests are made using the "tinytest" testing framework.
+You can see a guide to using it in the tinytest manual at
+
+   https://github.com/nmathewson/tinytest/blob/master/tinytest-manual.md
+
+To add a new test of this kind, either edit an existing C file in src/test/,
+or create a new C file there.  Each test is a single function that must
+be indexed in the table at the end of the file.  We use the label "done:" as
+a cleanup point for all test functions.
+
+(Make sure you read tinytest-manual.md before proceeding.)
+
+I use the term "unit test" and "regression tests" very sloppily here.
+
+=== A simple example
+
+Here's an example of a test function for a simple function in util.c:
+
+    static void
+    test_util_writepid(void *arg)
+    {
+      (void) arg;
+
+      char *contents = NULL;
+      const char *fname = get_fname("tmp_pid");
+      unsigned long pid;
+      char c;
+
+      write_pidfile(fname);
+
+      contents = read_file_to_str(fname, 0, NULL);
+      tt_assert(contents);
+
+      int n = sscanf(contents, "%lu\n%c", &pid, &c);
+      tt_int_op(n, OP_EQ, 1);
+      tt_int_op(pid, OP_EQ, getpid());
+
+    done:
+      tor_free(contents);
+    }
+
+This should look pretty familier to you if you've read the tinytest
+manual.  One thing to note here is that we use the testing-specific
+function "get_fname" to generate a file with respect to a temporary
+directory that the tests use.  You don't need to delete the file;
+it will get removed when the tests are done.
+
+Also note our use of OP_EQ instead of == in the tt_int_op() calls.
+We define OP_* macros to use instead of the binary comparison
+operators so that analysis tools can more easily parse our code.
+(Coccinelle really hates to see == used as a macro argument.)
+
+Finally, remember that by convention, all *_free() functions that
+Tor defines are defined to accept NULL harmlessly.  Thus, you don't
+need to say "if (contents)" in the cleanup block.
+
+=== Exposing static functions for testing
+
+Sometimes you need to test a function, but you don't want to expose
+it outside its usual module.
+
+To support this, Tor's build system compiles a testing version of
+teach module, with extra identifiers exposed.  If you want to
+declare a function as static but available for testing, use the
+macro "STATIC" instead of "static."  Then, make sure there's a
+macro-protected declaration of the function in the module's header.
+
+For example, crypto_curve25519.h contains:
+
+#ifdef CRYPTO_CURVE25519_PRIVATE
+STATIC int curve25519_impl(uint8_t *output, const uint8_t *secret,
+                           const uint8_t *basepoint);
+#endif
+
+The crypto_curve25519.c file and the test_crypto.c file both define
+CRYPTO_CURVE25519_PRIVATE, so they can see this declaration.
+
+=== Mock functions for testing in isolation
+
+Often we want to test that a function works right, but the function depends
+on other functions whose behavior is hard to observe, or whose 
+
+XXXX WRITEME
+
+=== Advanced techniques: Namespaces
+
+
+XXXX write this.  danah boyd made us some really awesome stuff here.
+
+
+Integration tests: Calling Tor from the outside
+-----------------------------------------------
+
+XXXX WRITEME
+
+Writing integration tests with Stem
+-----------------------------------
+
+XXXX WRITEME
+
+System testing with Chutney
+---------------------------
+
+XXXX WRITEME
+
+Who knows what evil lurks in the timings of networks? The Shadow knows!
+-----------------------------------------------------------------------
+
+XXXX WRITEME
+