Tests should be independent of each other and use different data structures (no shared stuff). Shared data structures could potential include artifacts of previously run tests
One behaviour should be tested per test case
Fast
Slow tests slow down build pipelines
Speed up tests by:
Using mocks or stubs to replace slow components
Redesign source code so slower pieces of code can be tested separately from fast pieces of code
Move slower tests to a different suite that runs less often (adhoc rather than in CI)
Independence
A test should test a single behaviour
Tests should not depend on other tests in any way
Tests should clean up their messes (don’t reply on data that is lying around / artifacts)
Existence
Tests should have a reason to exist otherwise they add time for no reason
Tests should find bugs, document behaviour, etc
Repeatable
Tests should not be flaky and idempotent
These are rarely due to differing OS’s
Strong Assertions
These should be as strong as possible to fully validate behaviour and break at the sign of any change in output
Break on Behaviour Change
Tests should tell you when you break expected behaviour even if that change was intended
[TDD] helps here
Single Failure Reasons
This is related to tests only testing a single behaviour
If a test fails, we should immediately know what broke to fix the bug
Make sure that naming and assertions are clear
Easy to Write
There should be no friction to write tests
Basically good tooling and infrastructure investments into QA are worth it since developers will actually write tests
Easy to Read
Test suites will scale as code bases grow and tests won’t be read until something breaks but when they are read, it needs to be obvious what is wrong
Easy to Change and Evolve
Make sure changing test code is not too painful
Your tests are inherently coupled to production code, but the more white box your tests are, the harder they are to change.