Agile has been all the rage in software development for many years now. Its popularity exists mostly among smaller teams that iterate quickly and release…
What are the Agile Engineering Skills, Scrum Developer Practices, or Software Craftsmanship practices? These are three (of many) common names for a set of practices…
What are the Agile Engineering Skills, Scrum Developer Practices, or Software Craftsmanship practices?These are three (of many) common names for a set of practices assembled in the mid-1990s to support and improve Agile software development efforts. They are a collection of practices that help teams work in an incremental, iterative, and simultaneous way (i.e., all functional departments working towards a clear goal on a continuous basis, and not waiting on each other at explicit or implicit process gates).
They were first combined as a complete collection of practices within Extreme Programming, another popular Agile method, around 1996.
What I've observed repeatedly on numerous Scrum teams is a slowing of value delivery, sometimes after a mere 5 or 6 sprints (even on greenfield projects). Developers find it more and more challenging to provide new releasable-quality features, and the burden of testing every feature--preferably every single one since the inception of the product--also increases, and usually falls heaviest upon testers.
I call this the Agilist’s Dilemma: The very thing that helps us deliver highest value continuously can also have a systemic “back-pressure” that negatively impacts the quality and maintainability of the product. This further reduces the ability to deliver quickly. If left unchecked, progress could cease entirely.
It’s neither because of a shortcoming in Agile methods, nor is it a great mystery. The very act of software development, itself, must change alongside our development methodologies. Software development teams that use Scrum or Kanban get the most benefit out of their Agile methods when they replace older, gated development activities with practices designed to work within shorter feedback loops.
The Agilist’s Dilemma is due to the nature of software: It becomes more and more difficult to add new features onto old without modifying the older code, or creating a lot of unmaintainable duplication and unneeded complexity in the system.
We need ways to keep our code truly well-designed. In other words: Easily changeable.
In order to adapt to changing market forces, Agile teams cannot try to plan or predict all features up-front. In order to add new features to code designed “for the moment” by an Agile team, we must be able to reshape the design so that it supports the new functionality.
Thus, we must have the ability to refactor. Over the years, I've come to see refactoring as the central Agile engineering skill: This is the practice that helps whole teams innovate. All other development practices support refactoring in some way.
Test-Driven Development (TDD) is a novel way to write code incrementally. This practice creates a safety-net of automated, comprehensive tests that allow for rapid, fearless refactoring; just the sort of thing that's needed to maintain and support innovative features, without compromising quality.
Behavior-Driven Development is a similar practice for writing business-facing tests. It can also help build confidence in the quality of the code, but its true power lies in helping the Agile team clearly circumscribe a feature for the understanding and agreement of all Agile team members. The conversations that occur, and the ubiquitous language that develops via BDD are at least as important as the automated testing, itself.
Whereas either practice would suffice to create a safety net needed to support refactoring, the difference in emphasis is important: BDD encourages critical conversations about features and scope, whereas TDD is a core developer practice used to craft, change, and design the product's code with greater confidence and far fewer defects.
Complex and interdependent business models and business rules are best supported by both BDD practices together, starting as early in the development process as possible. Within a complex business domain, TDD can be used to break up dependencies between business rules and their objects, and thus completely avoid the need for numerous slow, combinatorial test scenarios. Teams that use both find a sweet spot: Rather than relying solely on an explosion of BDD examples, the team can drop into a TDD cycle to more easily cover all permutations, or to build a supportive substructure of the code that has no immediate mapping to the customer-facing ubiquitous language.
Simpler business domains and smaller products (e.g., an early version of Twitter) would be fine with one practice or the other. If your company plans to grow this area of the business in the future, plan to try both in concert with each other.
Teams larger than two people, when building software in a rapidly-changing, ever-growing code-base, also need to provide their changes to others on the team; developers and testers.
This is particularly important when teams “swarm” on the one or two highest priority Product Backlog Items (i.e., PBIs, e.g. "user stories") in the sprint backlog.
This is done via Continuous Integration, the practice of fully integrating development changes into source-control and build systems, many times per day. This provides a single unified snapshot of the direction, quality, and fit-for-purpose of the product. This is also a key “DevOps" practice, and Continuous Delivery and Continuous Deployment both rely on this core developer practice.
We ask two people to work together—truly together at one computer whenever possible—on the product's programming tasks. In my experience, this practice provides far more than any academic study has been able to ascertain. Interestingly, this--certainly the most contentious Agile engineering skill--is the one that best facilitates the adoption and mastery of all other engineering skills and practices.
And there are quite a few other Agile engineering skills, some typically adopted more contextually than the sample of core practices described above.
Yes, but not all have experience doing so in a highly iterative and incremental multi-person environment.
I’ve encountered perhaps one in a hundred developers who have experienced TDD or pair-programming in college. Most university curricula still tend to focus on data structures, algorithms, compiler theory, automata theory, and other (very fascinating!) topics. Students are often expected to work independently on labs. Formal software testing is—at best—done by the professor or TA while grading the assignment.
Then, when folks graduate, they discover that most software development does not involve the invention and design of fresh new algorithms, or even the implementation of good old algorithms. Many key algorithms are built into the libraries of our modern programming languages.
Whereas we’d all like to be the next Zuckerberg, the reality is that even the most talented developers can expect to spend many years implementing business rules and customer features for apps or APIs. These are often interesting, innovative, and lucrative products, but they rarely push the boundaries of computer science, itself.
Though our projects, products, and programs are often very interesting work, if developers are seen as fungible resources and unconsciously incentivized to do counter-productive work (e.g., copy/paste/modify), or increasingly repetitive tasks; morale will suffer, performance will suffer, craftsmanship and quality will suffer, and your product will suffer. Everyone, including you, will be miserable.
The craftsmanship skills require self-discipline, and some time to investigate. Many teams try to learn on their own. Some developers will have success with this approach, but they don’t always have the influence over the whole team to sustain a viable team practice. Professional developers typically do not have much experience with effective training techniques. One of the important skills I convey in my technical courses is how to establish a disciplined practice.
I like to think of our technical courses as pragmatic, intensive, in-depth, challenging, and immensely fun post-grad courses.
It can take a team one or two months after training in these practices to get used to them and to see some clear, measurable benefit. That first month or two will slow the team down quite a bit. Firstly, be prepared for that.
Unfortunately, most teams, and their leaders, give up before then. When teams abandon the practices, it’s usually because they don’t feel they have the time or the autonomy necessary to learn and explore these practices. Please trust your team to act professionally.
Scrum already asks us to work towards transparency, empiricism, honesty, and trust. Continue to foster those values in your teams and at all levels of leadership. All important values grow in both directions, as relationships between people.
As an Agile leader, you likely see more of the "big picture" than the team has time to investigate.
Many of these practices help shorten critical feedback loops, often from months to days, or even minutes. Even without formal metrics, you should be able to perceive marked improvements in value delivered, quality, and team morale.
Of course, there are also costs. TDD, for example, generates unit-testing code at a ratio of about 2/1 or even 3/1 compared to shippable implementation code. Pair programming clearly appears to use two people to do a single task, so how could the benefits outweigh the costs?
Both TDD and pair programming greatly reduce rework. TDD has been shown to reduce defect density by 40-90%, with about 15% more time to complete tasks, initially. That much reduction in defects avoids broken features (and unhappy customers), time spent searching for the defect, fixing, and then re-testing the defective area (some developers tell me this is what they do half the time!).
I've found through experience that TDD is only costly at the start of the learning curve. After developers get used to it, TDD becomes their de facto tool for building code, and the accelerative capabilities outpace the costs.
Pair Programming is a practice that’s a bit more subtle. It, too, has been known to reduce defects by almost the same amount as TDD. If you’re already doing TDD, would you need both?
It’s a valid question.
Academic studies of pair programming miss the longer-term systemic benefits of having an idea, idiom, design, or clever technique spread like wildfire through the team. Teams develop real on-the-job proficiency from each other; commitment and support in the other practices; and pragmatic, work-related, team-building social interaction.
If you manage a team of people in cubes, you've probably noticed it's never easy to tell whether folks are working, or mentally "checked-out.” When you observe a team who routinely pairs, you can tell the difference. You will see a marked improvement in team communication, camaraderie, and morale. A happy team is a productive team is a happy team.
Also, as I suggest to the developers: Try it for a month or two. Then ask yourself, "Did that experiment really cost me twice as much?" Probably not.
Often the benefits of Agile Engineering practices are gained by the reduction or replacement of some other practice that doesn’t fit quite as well in an iterative, incremental, and parallel-work methodology like Scrum.
For example, let's examine the practice of holding code reviews. This is one of the best tried-and-true ways to find defects in a heavily-gated or hierarchical methodology. I used it for the ~15 years prior to my introduction to Agile methods. I resisted giving up such a powerful, collaborative practice! (We used to do code reviews in a room of about 4-6 people.)
Yet many teams discover their constraint is in the code reviews, particularly if all reviews must involve the architect or tech lead. If only one or two people must review all code before a PBI is complete, then they will find themselves in the unpleasant position of being the limiting factor of team productivity. Also, they must never take a sick day or vacation! If they did, productivity would stop.
Instead, I recommend that tech leads pair-program with developers, use TDD and BDD practices, and coach the team’s developers on the tools, architectural direction, idioms, and expectations for the product.
If a team pairs up on all production code (that is, the code that actually goes into the product), and they adopt Collective Responsibility (also known as "collective code ownership"), then traditional code reviews will become less and less necessary, because traditional defects and standards-violations will become extinct.
Primarily, the Agile Engineering skills reduce the cost-of-delay of lingering defects and delayed time-to-market. They all support refactoring, which allows Agile teams to adapt their existing design to fit new, sometimes unexpectedly innovative features into the existing product, without breaking any existing functionality. This ability to rapidly implement an innovative change is critical to the success of any modern software development project, program, or product.
You may notice that most of the benefits described above are product/organization -oriented. Those can impact developers in a positive way, but there’s often a significant lag. Fortunately, all of these practices have a variety of immediate benefits for developers, as well. (And, of course, these developer benefits are often the cause of the organizational benefits.)
I've noticed that some of the best-designed, easiest-to-maintain code I’ve ever seen (or written, myself) has been as a result of continuous, just-in-time, refactoring. This is often called out as its own skill: Emergent Design.
By “deferring” certain design decisions until they’re needed, you’re not doing as much up-front design that may or may not be needed.
This skill can seem really weird at first, but it’s a very logical discipline. We developers ask: What new thing does the code need to do? What inputs and circumstances will give what outputs? How do I expect client code to call this object or function?
You design the call to the new code by asking those questions first, and creating a small engineering specification (it’s not really a “test” until it passes) for the new functionality.
This practice gives a lot of instantaneous feedback, catching tiny logic errors right away, so we don’t have to over-design (or argue over) the implementation. And we avoid having to return to the implementation months later to find the defect that was found by the customers.
Also, the implementation changes in response to one simple request at a time, rather than having us juggle multiple concerns while writing or changing code. (That juggling was possible with the simple code I wrote in the 80s and early 90s. But software has become far more complex, particularly when there's more than one person writing code.)
It’s eventually much easier to simply “think in tests” rather than labor over all possible contingencies without any trace of executable examples (the old way); and now you end up building that fast safety-net of tests that will allow for more confident refactorings later on.
Just those two practices together have saved days per week in time spent searching for defects, tracing them in the debugger, discovering the defect in code we wrote months ago and can barely remember why we wrote it that way, and the re-work of that code. All of this debugging is a significant waste of your time, plus it's not all that much fun, is it?
BDD gives the whole team the opportunity to collaboratively clarify business requests. Fewer debates over scope-creep, "works as designed," and "that's not what I meant" occur on a team that starts building software with given/when/then examples.
“I can’t focus with all that noise,” “What if the other person slows me down?” “I’m an introvert…” Yep, I've heard it all.
Truth is, most developers find topical conversation—i.e., “talking it out”—helps them focus, helps them see alternative solutions, helps them deal with various interruptions, and helps them keep up their energy and enthusiasm for the work.
I suggest trying it for a month. Most people see the benefits, and even enjoy it. You may be an introvert, but regarding pairing, that means you don’t want to be sitting in a noisy crowd. If everyone on your team pairs, no one seems to notice the noise.
And you’ll be able to recuperate from a fun but exhausting day at the office using this next practice...
We work until our focus starts to fade, then we leave the office, both physically and mentally. This avoids defects, burn-out, health problems, family troubles, and an ever-increasing pressure to sling more and more lines of code.
There may be rare occasions for overtime or a few work-from-home hours, but on well-running and successful Agile teams, they are rare. One client I worked with recently reported that they haven't had a single incident requiring a developer to work in the evening or on a weekend since 2004. More than a dozen years without a “developer emergency.” And that’s on an application that helps surgeons make life-and-death triage decisions.
This 3-day course covers the core Scrum Developer Practices and as many peripheral Agile Engineering practices as time permits. Participants form small Scrum teams and work on a simple product backlog, using the Scrum Developer practices in very short (2 hour) sprints to create working software. Based on their experiences, they help decide what needs to be covered before the next sprint.
The course covers a broad selection of the practices, and goes as deep as we need in order to improve flow. The audience is primarily developers, though often managers and testers, who can write code in the selected programming language, will participate in order to experience these practices and give their cross-functional perspective.
This course takes a deep dive into the skills needed to begin, and master, a Test-Driven discipline. A much different format than CSD: Topics are pre-determined, and pairs of developers immerse themselves into labs that have been designed to hone particular critical skills. You will experience refactoring, TDD, test-doubles, and legacy-code characterization testing. This last practice is critical for teams trying to jump-start a TDD practice but who already have significant amounts of complex, untested code. When I last checked, an actual characterization-testing lab makes our TDD course unique in the industry, world-wide.
The audience is developers. Often testers and managers participate, but they act as developers during the labs, and must know how to write code in the programming language(s) chosen for the course.
This whole-team practice is the perfect starting-point when taking a story into a sprint to be developed and completed. The results are clear, readable, repeatable executable-specifications. Modern tools allow these tests to be instantly automated without much additional effort. One of the most powerful benefits of this practice is early and improved conversations around scope and priorities. This practice is a powerful bridge between Product (business) and Development (technology).
We strongly recommend that whole teams, including developers, testers/QAs/QEs, and Product/BAs, attend this course together.
Further questions? Contact us, or feel free to leave a comment!
There was a problem reporting this post.
Please confirm you want to block this member.
You will no longer be able to:
Please allow a few minutes for this process to complete.
