Effective Software Testing MEAP V04
Copyright
Welcome
Brief contents
Chapter 1: Effective and systematic software testing
1.1 Developers that test versus developers that do not
1.2 Effective software testing for developers
1.2.1 Effective testing in the development process
1.2.2 Effective testing as an iterative process
1.2.3 Focusing on development and then on testing
1.2.4 The myth of "correctness by design"
1.2.5 The cost of testing
1.2.6 The meaning of effective and systematic
1.2.7 The role of test automation
1.3 Principles of software testing (or why testing is so hard)
1.3.1 Exhaustive testing is impossible
1.3.2 Knowing when to stop testing
1.3.3 Variability is important ("the pesticide paradox")
1.3.4 Bugs happen in some places more than others
1.3.5 Whatever testing you do, it will never be perfect or enough
1.3.6 Context is king
1.3.7 Verification is not validation
1.4 The testing pyramid and where we should focus
1.4.1 Unit testing
1.4.2 Integration testing
1.4.3 System testing
1.4.4 When to use each of the test levels?
1.4.5 Why do I favor unit tests?
1.4.6 What if I disagree with the testing pyramid?
1.4.7 Is what I will learn from this book enough for me to find all the bugs?
1.5 Exercises
1.6 Summary
1.7 References
Chapter 2: Specification-based testing
2.1 The requirements say it all
2.1.1 Step 1: Understanding the requirements, the inputs, and the outputs
2.1.2 Step 2: Just explore what the program does for a bunch of inputs
2.1.3 Step 3: Judiciously explore the possible inputs and outputs and identify the partitions
2.1.4 Step 4: Analyze the boundaries
2.1.5 Step 5: Devise test cases
2.1.6 Step 6: Automating the test cases
2.1.7 Step 7: Augment the test suite with creativity and experience
2.2 Specification-based testing in a nutshell
2.3 Finding bugs with specification testing
2.4 Specification-based testing in the real world
2.4.1 The process should be iterative and not sequential
2.4.2 How far should I go in doing specification testing?
2.4.3 Partition or boundary? It does not matter!
2.4.4 On and off points are enough but feel free to throw in some in and out points too
2.4.5 Use variations of the same input to facilitate understanding
2.4.6 When the number of combinations explodes, be pragmatic
2.4.7 When in doubt, go for the simplest input
2.4.8 Pick reasonable values for inputs you do not care about
2.4.9 Test for nulls and exceptional cases but only when it makes sense.
2.4.10 Go for parameterized tests when tests have the same skeleton. Little duplication is never a problem but a lot of duplication is.
2.4.11 It works with requirements of any granularity or tests other than unit tests
2.4.12 How does this work when we have classes and state?
2.4.13 The role of experience and creativity
2.5 Exercises
2.6 Summary
2.7 References
Chapter 3: Structural testing and code coverage
3.1 Code coverage, the right way
3.2 Structural testing in a nutshell
3.3 The different coverage criteria
3.3.1 Line coverage
3.3.2 Branch coverage
3.3.3 Condition + branch coverage
3.3.4 Path coverage
3.4 Complex conditions and the MC/DC coverage criteria
3.4.1 An abstract example
3.4.2 Creating a test suite that achieves MC/DC
3.5 Handling loops and similar constructs
3.6 Criteria subsumption and how to choose one
3.7 Specification-based and structural testing, a running example
3.8 Boundary testing and structural testing
3.9 Structural testing alone is often not enough!
3.10 Structural testing in the real world
3.10.1 Why do some people hate code coverage?
3.10.2 What does it mean to have achieved 100% coverage?
3.10.3 What coverage criteria should I use?
3.10.4 MC/DC for when expressions are too complex and cannot be simplified
3.10.5 Are there other coverage criteria I should know about?
3.10.6 What should not be covered?
3.11 Mutation testing
3.12 Exercises
3.13 Summary
3.14 References
Chapter 4: Designing contracts
4.1 Pre-conditions and post-conditions
4.1.1 The assertion keyword
4.1.2 Strong and weak pre- and post-conditions
4.2 Invariants
4.3 Changing contracts and the Liskov Substitution Principle
4.3.1 Inheritance and contracts
4.4 How is design-by-contract related to testing?
4.5 Design-by-contract in the real world
4.5.1 Weak or strong pre-conditions?
4.5.2 Input validation or contracts? Or both?
4.5.3 Asserts or exceptions: when to use one and the other?
4.5.4 Exception or soft return values?
4.5.5 When not to use design-by-contract?
4.5.6 Should I write tests for my pre-conditions, post-conditions, and invariants?
4.5.7 Tooling support
4.6 Exercises
4.7 Summary
4.8 References
Chapter 5: Property-based testing
5.1 Example 1: The passing grade program
5.2 Example 2: Testing the unique method
5.3 Example 3: Testing the indexOf method
5.4 Example 4: Testing the Basket class
5.5 Example 5: Creating complex domain objects
5.6 Property-based testing in the real world
5.6.1 Example-based testing vs property-based testing
5.6.2 Careful with expensive or impossible parameter generation, poorly distributed data generation, and edge cases
5.6.3 Creativity is key
5.7 Exercises
5.8 Summary
5.9 References
Chapter 6: Test doubles and mocks
6.1 Dummies, fakes, stubs, spies, and mocks
6.1.1 Dummy objects
6.1.2 Fake objects
6.1.3 Stubs
6.1.4 Mocks
6.1.5 Spies
6.2 An Introduction to mocking frameworks
6.2.1 Stubbing dependencies
6.2.2 Mocks and expectations
6.2.3 Capturing arguments
6.2.4 Simulating exceptions
6.3 Mocks in the real world
6.3.1 The disadvantages of mocking
6.3.2 What to mock and what not to mock?
6.3.3 Date and time wrappers
6.3.4 Mocking types you do not own
6.3.5 What do others say about mocking?
6.4 Exercises
6.5 Summary
6.6 References
Chapter 7: Designing for testability
7.1 Separate the infrastructure from the domain code
7.2 Dependency injection and controllability
7.3 Make your classes and methods observable
7.3.1 Example 1: Introducing methods to facilitate assertions
7.3.2 Example 2: Observing the behavior of void methods
7.4 Dependency via class constructor or value via method parameter?
7.5 Design for testability in the real world
7.5.1 The cohesion of the class under test
7.5.2 The coupling of the class under test
7.5.3 Complex conditions and testability
7.5.4 Private methods and testability
7.5.5 Static methods, singletons, and testability
7.5.6 The hexagonal architecture and mocks as a design technique
7.5.7 Where to read more about design for testability?
7.6 Exercises
7.7 Summary
7.8 References
Chapter 8: Test-driven development
8.1 Our first TDD session
8.2 Reflecting about our first TDD experience
8.3 TDD in the real world
8.3.1 To TDD or to not TDD?
8.3.2 TDD 100% of the time?
8.3.3 Does TDD work for all types of applications and domains?
8.3.4 What does research say about TDD?
8.3.5 Baby steps in a productive way
8.3.6 Other schools of TDD
8.3.7 TDD and proper testing
8.3.8 The history behind TDD
8.4 Summary
8.5 Exercises
8.6 References
Chapter 9: Larger tests
9.1 Going for larger tests
9.1.1 Testing larger components
9.1.2 Testing larger components that go beyond our own code base
9.2 Database and SQL testing
9.2.1 What to test in an SQL query?
9.2.2 Writing automated tests for SQL queries
9.2.3 Setting up infrastructure for SQL tests
9.2.4 Best practices
9.3 System tests
9.3.1 An introduction to Selenium
9.3.2 Designing Page Objects
9.3.3 Patterns and best practices
9.4 Final notes on larger tests
9.4.1 Specification-based testing, boundary testing, structural testing, property-based testing: how does this all fit here?
9.4.2 Perform cost/benefit analysis
9.4.3 Be careful with methods that are covered but not tested
9.4.4 A proper code infrastructure is key
9.4.5 DSLs and tools for stakeholders to write tests
9.4.6 What about Web API testing? What about testing responsive web apps? What about testing apps that use some modern UI framework?
9.5 Summary
9.6 Exercises
9.7 References
Chapter 10: Test code quality
10.1 Principles of maintainable test code
10.1.1 Tests should be fast
10.1.2 Tests should be cohesive, independent, and isolated
10.1.3 Tests should have a reason to exist
10.1.4 Tests should be repeatable and not flaky
10.1.5 Tests should have strong assertions
10.1.6 Tests should break in case the behavior changes
10.1.7 Tests should have a single and clear reason to fail
10.1.8 Tests should be easy to write
10.1.9 Tests should be easy to read
10.1.10 Tests should be easy to change and evolve
10.2 Test smells
10.2.1 Excessive duplication
10.2.2 Unclear assertions
10.2.3 Bad handling of complex or external resources
10.2.4 Fixtures that are too general
10.2.5 Sensitive assertions
10.3 Exercises
10.4 Summary
10.5 References
Chapter 11: Wrapping up the book
11.1 Although the model looks linear, iterations are fundamental
11.2 A "bug-free" software development process: reality or myth?
11.3 Involve your final user
11.4 Unit testing is hard in practice
11.5 Invest in monitoring
11.6 What is next?
Chapter A: Answers
A.1 Chapter 1
A.2 Chapter 2
A.3 Chapter 3
A.4 Chapter 4
A.5 Chapter 5
A.6 Chapter 6
A.7 Chapter 7
A.8 Chapter 8
A.9 Chapter 9
A.10 Chapter 10