Kinbiko

Testing Tips for Beginner Gophers

February 22, 2020

Some technical knowledge assumed.

Let’s have a go at writing tests in Go. Puns aside, testing in Go is baked into the language. Assuming you’re comfortable with the Go basics, let’s get started with writing tests. First of all, the way you run all tests in the current package is:

$ go test

However, unless you’ve got tests in your project nothing is going to execute. Go tests have to follow these three rules:

  • The filename of all your test files must end with _test.go (including the underscore _).
  • The test function names must start with the Test prefix.
  • The test functions must have exactly 1 parameter, of type *testing.T (conventionally called t).

Thus a file called something_test.go containing:

package mypkg

import "testing"

func TestSomething(t *testing.T) {
}

is a minimal piece of code that counts as a test, and can be verified as such by running go test in the same directory.

As positively exhilarating as the above test was, it is useless, as it can never fail. To make the test fail, we use the *testing.T parameter.

func TestSomething(t *testing.T) {
    t.Errorf("test failed")
}

Now we have a test that’s always failing. This isn’t very useful, but it’s a start. If we run this we get the following output:

--- FAIL: TestSomething (0.00s)
    something_test.go:6: test failed
    FAIL
    exit status 1
    FAIL    _/Users/kinbiko/repos/mypkg     0.005s

The built-in assertions are all you need.

There are two key methods on t you need to know:

  • t.Errorf marks your test as failing, and prints the given message.
  • t.Fatalf does the same as t.Errorf, but will stop the execution of the test at the line it’s called.

As the programmer, it’s your responsibility to identify whether or not a test should fail. This means putting ifs in your test code, which might be surprising to you, especially if you’re coming from a language like Java, where the test framework includes various forms of equality checks for most assertions.

func TestFizzBuzz(t *testing.T) {
    if fizzbuzz(3) != "Fizz" {
        t.Errorf("expected the 3rd FizzBuzz value to be 'Fizz'")
    }
}

Personally, I prefer this more explicit way of writing test assertions, and you’ll see why in a sec, but if you’re set on avoing ifs, there’s a very popular testing library called Testify which exposes this XUnit-like testing syntax.

Again, the built-in testing framework assumes you’ll write somewhat non-trivial code to make the right assertions. This might sound like a bad thing, but it turns out breaking the convention of ‘dumb’ tests allows for some pretty valuable techniques…

Table-driven tests

Table-driven tests are a very common testing pattern in Go. The general idea is that you define a “test table”: a slice of ‘test case’ structs, that you iterate over, executing the same bit of test code over and over again for all the test cases. It’s perhaps easier to show how this works with an example:

func TestFizzBuzz(t *testing.T) {
    // Define the test table: an anonymous struct type that
    // we define and initialise a slice of immediately.
    tt := []struct {
        in  int
        exp string
    }{
        {in: 1, exp: "1"},
        {in: 2, exp: "2"},
        {in: 3, exp: "Fizz"},
        {in: 4, exp: "4"},
        {in: 5, exp: "Buzz"},
        {in: 6, exp: "Fizz"},
        {in: 7, exp: "7"},
        {in: 10, exp: "Buzz"},
        {in: 15, exp: "FizzBuzz"},
        {in: 30, exp: "FizzBuzz"},
    }

    // Iterate over all test cases
    for _, tc := range tt {
        if got := fizzbuzz(tc.in); got != tc.exp {
            // A good human readable assertion message makes understanding test failures easy!
            t.Errorf("expected the FizzBuzz value number %d to be '%s' but was '%s'", tc.in, tc.exp, got)
        }
    }
}

Notice how t.Errorf forces us to think about and write our own assertion message. This is a good thing. You get to choose a human-readable message including exactly the information you care about, instead of relying on a testing framework’s default message.

The authors of Testify have of course thought about this, and allows for custom assertion messages, but it doesn’t force you to write these. In my experience, often when using Testify these assertion messages are not written, and even when they are included it feels like a chore, which in turn leads to less thought being put into these assertion messages, losing their merit. This is a large part of why I prefer the standard library’s "testing" framework out of the box. This is of course personal (or team) preference, and I’ll let you make up your own mind.

Sub-tests

For simple programs like FizzBuzz, the test cases are very simple: One input, one output. More commonly in ‘real’ apps, you’re testing more complex business logic. In these cases you probably want more “context” around what exactly you’re testing. This is where sub-tests come in.

Sub-tests are effectively inner tests within outer tests, which allow you to give your inner tests a name, and share some common setup. Let’s examine how sub-tests are useful by writing some tests for a fictional app that allows users to ‘like’ posts, similar to Facebook or Instagram. Here’s a test for a certain feature of this app, try and see if you can understand what the likesExcerpt function does by reading the tests.

func TestLikesExcerpt(t *testing.T) {
	var (
		jeffords  = user{id: 0, name: "Jeffords"}
		diaz      = user{id: 1, name: "Diaz"}
		scully    = user{id: 2, name: "Scully"}
		peralta   = user{id: 3, name: "Peralta"}
		boyle     = user{id: 4, name: "Boyle"}
		holt      = user{id: 5, name: "Holt"}
		santiago  = user{id: 6, name: "Santiago"}
		hitchcock = user{id: 7, name: "Hitchcock"}
		linetti   = user{id: 8, name: "Linetti"}
	)

	db := setUpDB()
	db.Insert([]user{jeffords, diaz, scully, peralta, boyle, santiago, hitchcock, holt, linetti})
	s := service{db: db}

	// Note: it's not uncommon to define the for loop and the test table at the same time.
	for _, tc := range []struct {
		name string
		in   []int
		exp  string
	}{
		{
			name: "given no likes then invite user to like",
			in:   []int{},
			exp:  "Be the first to like this",
		},
		{
			name: "given one like then name the user",
			in:   []int{scully.id},
			exp:  "Scully likes this",
		},
		{
			name: "given two likes then name both users",
			in:   []int{jeffords.id, diaz.id},
			exp:  "Jeffords and Diaz like this",
		},
		{
			name: "given three likes then name the first two users and count the last user (singular)",
			in:   []int{jeffords.id, diaz.id, scully.id},
			exp:  "Jeffords, Diaz, and 1 more person likes this",
		},
		{
			name: "given more than three users then name the first two users and count the rest (plural)",
			in:   []int{jeffords.id, diaz.id, scully.id, boyle.id, peralta.id, linetti.id},
			exp:  "Jeffords, Diaz, and 4 others like this",
		},
	} {
		// Note: We're intentionally shadowing the t parameter in the subtest,
		// by defining a parameter with the same name.
		t.Run(tc.name, func(t *testing.T) {
			got, err := s.likesExcerpt(tc.in...)
			if err != nil {
				// Note: even though we're using Fatalf here, we only interrupt the
				// innermost subtest; the other test cases will still run.
				t.Fatalf("didn't expect an error but got '%s'", err.Error())
			}
			if got != tc.exp {
				t.Errorf("expected excerpt '%s' but got '%s'", tc.exp, got)
			}
		})
	}

	t.Run("errors out on unrecognised IDs", func(t *testing.T) {
		if _, err := s.likesExcerpt(scully.id, 123, boyle.id, 456); err == nil {
			t.Errorf("expected an error relating to unrecognised IDs but didn't get one")
		}
	})
}

The above creates a subtest per test case, using the test case name as the name of the subtest. This makes it easier to understand which test case failed when you have multiple subtests. This is because in the case of a test failure, it will print the name of the test that failed prior to the assertion message. For example, if the "given three likes then name the first two users and count the last user (singular)" test case failed, we would get the following message:

--- FAIL: TestLikesExcerpt (0.00s)
    --- FAIL: TestLikesExcerpt/given_three_likes_then_name_the_first_two_users_and_count_the_last_user_(singular) (0.00s)
            likes_test.go:87: expected excerpt 'Jeffords, Diaz, and 1 more person likes this' but got 'Jeffords, Diaz, and 1 others like this'

which explicitly names the subtest that failed, enabling us to quickly identify what might be wrong.

Also notice the second single subtest that checks an error case. Because we did all the hard work of defining the users and setting up the service and db in the outer test, we can use them in both sub tests.

Integration tests vs unit tests; the _test package.

For any package, e.g. storage, you can optionally use a storage_test package in the same directory to use as a package for just test logic. The idea behind the storage_test package is to test only the types, funcs, methods, and variables that are public in the storage package. If you’ve written code in other languages such as Java or Ruby, you might be used to seeing these types of tests in different directories altogether. I like Go’s choice of allowing a _test package in the same directory, as it makes it easier to find the production code for a given test, and vice versa.

Writing tests like this is optional, in that you’re not required to use this package for your tests. There’s nothing preventing you from writing exactly the same tests within the storage package directly, but there are some advantages to using the _test package:

  • You’re testing the code from the same point of view that your users are using it, giving you a better feel for how your package is to use.
  • You’re not relying on implementation details of your package, and your tests are less rigid and fragile.
  • You’re likely to find a necessary and sufficient level of dependency injection required for your package, helping you find the correct interfaces to demand in your package.
  • Compilation errors in these tests probably indicate a breaking change in your API, informing you that you might need to do a major version bump.

If we call the storage_test package tests ‘Integration tests’ and the storage tests ‘unit tests’, then it might be tempting to believe that all tests should be integration tests. I reject this idea, and believe you get the most value out of your code and time by picking your battles around when to write unit tests and when to write integration tests. In general, I default to writing integration tests, and introduce unit tests for complex and “hard to hit” logic.

Let’s look at an example of what the files of a storage package might look like:

cache.go
cache_test.go <-- `package storage`: unit test of the cache implementation details because caching logic is hard.
storage.go
storage_test.go <-- `package storage_test`: integration test of storage, which uses the logic of cache.go internally.

Test with confidence and future steps

You now know the core techniques and ideas behind testing your Go applications and you no longer have any excuses to not write tests for your go packages. This is the first out of 3 articles I’ve been meaning to write, and I’ve got drafts for the followup articles “Testing Tips for Intermediate Gophers” and “Testing Tips for Advanced Gophers” as well.

The intermediate article would include: Test coverage, limiting test durations with timeouts, benchmarking, using the race detector, mocking, (executable) documentation examples, testing HTTP servers, and verifying JSON data.

The advanced article would cover: Parallelisation, setup and teardown tricks, fast vs slow tests, support functions for tests, failsafe tests, random data, killing an executing test and parsing the panic output, speeding up tests, and some TDD tricks.

These are more involved articles and would require more effort from me personally. I will probably only publish these if this one does well so… If you liked this article, please share it on your favourite social media platform (links for sharing are at the top of the article)!


Roger Guldbrandsen

Written by Roger Guldbrandsen.