Introduction
Developer Testing is a trend in software development that aims to improve delivery speed and reduce costs. It’s important to note that there isn’t an exact definition of what Developer Testing is; it is going to vary from organization to organization, with some having Developers take over more of the testing tasks and others have Developers take over all of the testing tasks. It could range from more unit testing by Developers to testing full end-to-end workflows. No matter the case for your organization, to have success in moving testing tasks to Developers is reliant on a culture that supports this—not just implementing a new tool or saying, “Okay, Developers, now you get to test!”
Practitioners of Developer Testing claim that it delivers several cost-cutting and quality benefits, including:
- Shorter time-to-market with consistent, higher end-product quality.
- Finding and addressing issues in the earliest stages.
- Higher efficiency and ownership.
- Reduced handoff and waiting.
This paradigm shift presents its own set of problems—if it was easy, everyone would have done it a long time ago! A major bottleneck in many organizations’ Developer Testing initiative is a simple question: How much testing is really a Developer’s responsibility? The days of “testing” belonging to Test or QA Teams are gone. Test Teams have had a specific testing function. Every organization is now relying much more on formal, measurable, repeatable Developer unit testing. This we already know, but how much testing Development Teams will take over is the issue. Like with much process change, there is a right way and a wrong way.
No matter the SDLC, job titles, or testing processes, a successful Developer Testing initiative is reliant on one thing: Culture. It’s easy to tell when something in an organization or team’s culture is off, but it’s harder to know what steps to take to fix the cultural issues. In many years of consulting and training for various organizations, I have seen many cultural issues, problems, and, quite honestly, disasters. So, I wanted to write this piece specific to culture for organizations looking to improve their testing, work environment, productivity, retention, and product quality—specifically surrounding Developer Testing. This will be a 2-part piece, as this is an in-depth topic. Part 1 will first compare and contrast “culture” and “practices,” as well as their implications and place in your organization; it will then examine the origins of the notion of “Developer Testing,” and answer the big question: “Is testing really a Developer’s responsibility?” Part 2 will then delve into the importance and impact that culture has on a team, and then examine cultural practices from various SDLC ideologies and how they affect the success of a Developer Testing initiative. So, let’s dive into part 1.
Culture vs. Practice: What’s the Difference?
What’s the difference between culture and practice? I throw around these words a lot because, in my consulting work, they are the foundation to everything, from getting a product done and delivering value to the customer, to a team simply getting along better. You may have a good understanding of the difference here, but let’s take a step back and define some differences before using them more liberally later in this article.
An example of the difference between culture and practice is the Agile idea on measurement. The Agile Manifesto was signed in 2001—immediately after the Dot-Com boom in the late 1990s. Before this, I remember half of my Friday work time was coming up with a weekly dashboard of a dozen different numbers that were marginally actionable, usually interesting, and misunderstood by half of the people at the weekly project meeting; but, the main problem here was that it took too long to generate! So, the cultural idea in Agile is to stop with a giant dashboard—it’s overhead. This derived from the Agile Principle: Working software is the primary measure of progress.
In contrast, Scrum has a practice of measuring burndown. You take that number and get velocity—that’s it. No dashboards and little overhead. You just measure burndown. That’s the Scrum practice. As a result:
- Everyone on the team knew what that number meant.
- My Friday afternoons were liberated to actually do real work and not overhead.
- The burndown/velocity number is actionable; the goal in the sprint is to get your burndown (from a total number of story points) to zero. Velocity is the basis of project planning (how many story points can your team get done in a sprint) for the next sprint.
Practice is more related to process, like measuring burndown and velocity. Other examples include ensuring you have a Sprint retrospective, practicing test-driven development (TDD), or measuring Developer Unit Testing. Culture is ideas or agreements that create a team dynamic that significantly impacts—hopefully, increases—delivering customer value. Examples here would include: Quality at Every Step (Lean), Group Ownership of Quality (Scrum), Fail Fast (Lean), and Respect (XP).
Why is this distinction important? Practices come and go, but culture endures. A practice is having Sprint “Ceremonies,” like a daily standup and release planning meeting. Culture is collaboration. A culture of collaboration will stay when processes change, evolve, or get dropped forever.
Let’s say there is a build process, and you have to do steps 1, 2, and 3 in order (a practice). But—“just this time”—we’re going to change one thing and not go through the whole build process. How is your team going to respond? It can range from an open discussion to bullying. But, with the right culture in place—like full and open communication, collaboration, and group ownership of quality—we can avoid the latter. Culture is how teams come to good decisions; good culture will have teams getting along year after year, and not break down from one process lapse—like changing the build process and not discussing the risk because “it’s ok, and just this one time.”
Developer Testing is a New Buzz Word, But Not a New Idea
Although increased Developer Testing seems to currently be breaking news in the Software Testing world, it’s not; multiple development methodologies over the years have called for increased ownership of quality (and therefore testing) across the entire team. In traditional software development, testing and development were 2 separate functions. During the waterfall era, Developers were expected to implement “everything” and then completely rely on a QA Team for testing once the code was done. This led to a culture of handoff and waiting and sometimes a vicious back-and-forth bug-and-fix cycle, causing delays in releases. This process and practice very often lead to a sour culture and lack of collaboration. It is also famous for herculean efforts to save “quality.”
The widespread adoption of Agile with Scrum brought a paradigm shift and drastically changed the way Engineers work by putting Developers and Testers on the same cross-functional Scrum Team, thus fading the distinction between Developers and Testers. There were joint practices for everyone on the team regardless of tasks performed: Sprint planning, estimating, daily stand-ups… various practices that stemmed from a collaborative culture. Teams now started to focus on iteratively rolling out working software rapidly by adopting and embracing changes.
Agile Teams needed a culture that promoted quality as a team responsibility, instead of quality being solely owned by a QA Team. This idea has its roots in the Lean Principle of Quality at Every Step, but the culture changed for no other reason other than sprints really sped up delivery and the idea of having a final gatekeeper for quality was outdated. Developers now had more responsibility for quality tasks. This cultural idea is called shift-left, known for shifting testing tasks earlier in the process and distributing some testing tasks to Developers. This redistribution of testing tasks focused Developers on validating individual acceptance criteria from user stories, primarily with unit tests for faster functional bug finding and fixing. Simultaneously, this was supposed to (hopefully) refocus Testers on user story-level tests, performing bigger integration, end-to-end workflows, and full transactions on the variety of supported platforms for higher customer experience and satisfaction. This redistribution of tasks based on the culture ideas of Quality at Every Step, group ownership of quality, and shift-left has been successful in many teams, while others are still struggling with embracing new culture changes.
Obviously, this cultural change was inspired by various practices. In this set of practices, particularly coming from XP (eXtreme Programming), Developers had more and usually formalized responsibility for TDD (or at least measurable unit testing). This new practice brought up this idea: If Developers are testing more, let’s cut QA Team testing a little or a lot.
So, Developer Testing is not a new idea, but rather it is rapidly evolving. The question now is how much and how far Developers take testing tasks, as well as what tasks are left for Test Teams. And, how do we plan testing not just to be rushed through at the last minute, but plan testing in a way that supports a culture of quality to deliver value to your customers?
Is Testing Really a Developer’s Responsibility?
Development methodologies aside, there was always a question about how much of the testing responsibility should be handed to Developers. Should the Developers limit themselves to unit testing? Or, should they go beyond and own integration testing through End-to-End Testing? The effectiveness of Developer Testing is often debated due to various reasons, including:
- We need to minimize or eliminate handoff and waiting.
- Developing and testing are 2 different areas of expertise. It does not mean that all Developers are bad Testers; rather, they are simply not specialized in testing and might not be as proficient or effective as a traditional Tester.
- Common Software Team culture is Developers validate, Testers find bugs. There are very different mindsets.
- Some Developers dislike testing as they feel it is boring and repetitive.
- Developers often lack the objectivity to critique their own code and may be biased in their choice of tests to run.
- Developers may lack the understanding of users or modeling tests based on how users may behave and the errors that may happen in the normal course of use. The focus on an end-user perspective has always been the responsibility of a QA Team.
- There may be some old culture ideas to battle such as: You pay me to write code, not test it.
The Lean principle and cultural idea of Quality at Every Step may be the primary cultural idea to firmly implant into the team’s culture. The evolution of CI/CD as a DevOps practice and rise of Continuous Training practices and DevOps cultural shifts are essential to cultivating the Developer Testing mindset at an organizational level. Quality at Every Step has many applications. From everyone sharing responsibility for quality user stories and quality environments, to the CI/CD practice of “do one thing and test it, then do another thing and test it.” In part 2, we’ll look at the cultural changes inspired by numerous development methodologies that will help your organization to foster Developer Testing and ease its adoption.
Part 1 Summary
We’ve covered a lot so far, but the main idea I want you to take away from this piece is any success in a Developer Testing initiative—no matter how much testing responsibility you’re delegating to your Developers—is reliant on one thing: A good team culture that supports quality goals and the skill-building and estimates to test and assess the product delivering real value to your customers. Your team’s culture is what endures when new practices develop or current practices evolve. Additionally, no matter what other reasons you have for shifting testing tasks to Developers, your main driver for this cultural shift should be for the sake of quality—not cutting costs, not scrambling to finish testing on a new feature, but because you want to improve quality within your team, your SDLC, and for your end-user.
Part 2 of this piece, which will be coming out next week, will look at various development methodologies, their practices, and their effects on a team’s culture. It’ll then offer advice for leaders and managers looking to successfully implement these culture ideas (whether it’s the first time, or you’re attempting to get it right again).
Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), available in English, Chinese and Japanese, and Global Software Test Automation (HappyAbout Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries."