Slow and Brittle: Replacing End-to-End Testing

room: Grand Ballroom D North — time: Thursday 16:00-16:45, Thursday 16:45-17:30
Level: Expert

End-to-end tests appear everywhere: test-driven development, story-test-driven development, acceptance testing, functional testing, and system testing. They’re also slow, brittle, and expensive. In this expert-level workshop, we will discuss why end-to-end testing is used, examine where and why it breaks down, and generate more effective solutions. We will spark ideas for participants to explore further on their own.

We will not be debating the premise (that end-to-end tests are problematic). This is an expert-level workshop and attendees will be expected to participate fully.

Process/Mechanics

Our goal for this workshop is to generate serious, practical alternatives to end-to-end testing. We’ve listed Garret the Guru as the primary persona for this session because he seemed to match our needs most closely. We’re looking for experienced practitioners, although not necessarily as experienced as Garret.

(If the program committee would like a “Practicing” level session rather than an “Expert” level session, we can easily modify the session. In a “Practicing” level session, we would spend more time explaining the problem and providing example solutions.)

We will start by summarizing the problems of end-to-end testing. We will cover a variety of scenarios where we have seen high cost, low return results: from test-driven development; to storytest-driven development; to functional and acceptance testing. We will highlight the commonalities and then turn the discussion over to the group.

  • Introductions (~5 min)
  • Talk: The Case Against End-to-End Testing (~20 min)
  • Activity: The Case for End-to-End Testing (~10 min)
  • Discussion: Alternatives I (~15 min)
  • Lightning Round I (10 min)
  • Discussion: Alternatives II (~15 min)
  • Lightning Round II (10 min)
  • Wrap-up (~5 min)

The “alternatives” discussions are performed in table groups with two or three groups presenting in each lightning round. The second discussion builds on the ideas generated in the first.

Within two months of the workshop, we will publicly post the group’s conclusions and announce them on jamesshore.com.

Learning outcomes
  • the problems with end-to-end testing
  • the needs end-to-end testing fulfills
  • alternatives to end-to-end testing
  • new testing ideas and approaches
Featured participants
Primary target persona