Blog

A New Kind of Testing

Posted on Sep 17, 2014 | 0 comments

A New Kind of Testing

Let’s get straight to the point… Software testers and QA analysts should not be finding bugs in software. As a software tester, your job should revolve around preventing bugs from ever finding their way into a software product in the first place; but in the current corporate landscape where developers and testers are segregated, that may seem like an impossibility. I want to talk about how we can bring more awareness to this problem and open our eyes to a whole new kind of testing, one that happens before any software is written. If we successfully apply this new kind of testing, then we may dramatically reduce the cost of bugs while at the same time improving job satisfaction for software testers and QA analysts. The Agile Testing Mindset Radical change in behavior requires a radical change in thinking, so we will start by looking at how differently we must think about testing in order to truly improve it. First, the quality of a software product should not be the sole responsibility of testers. I too commonly see organizations that place all the burden of software quality on the testing team, instead of the development team writing the product code. This burden should be shared equally among all individuals that are working on a product, from project managers and business analysts to developers and testers. Next, our goal should be to test first, not after. That is, we should give consideration to quality up front, not as an afterthought. Great companies further understand that quality is not a verification technique, it is a design practice. Great design leads to great quality, when the two practices come together intentionally. Lastly, a huge focus and commitment to test automation can greatly reduce waste in the quality assurance efforts. The following table compares the traditional thinking with our “new” thinking. Traditional Thinking “New” Thinking Burn the toast, scrape it off Testers verify behavior Testers find bugs Testing starts after development is done If more bugs are found, QA is doing well Testers are a separate team from development Don’t burn the toast! Testers define behavior Testers prevent bugs Development starts after tests are ready If more bugs are found in testing, QA is failing Developers and testers are on the same team The burnt toast analogy used above explains the way that traditional teams create software products by quickly creating a terrible initial product and then relying very heavily on QA efforts to clean things up by “scraping off the burnt parts.” This analogy is not my own, but unfortunately I cannot find the original author of this concept, so if you know the original author, please let me know in the comments. A New Role for Testers In this new kind of testing, we need a new role for our testers and QA analysts. In order to adhere to the mindset above, our testers will need to work closely with a Product Owner (in Scrum) or Product Manager of some sort to understand and help document and communicate desired system behaviors. This can sometimes be a huge amount of critically important work that directly impacts the overall quality of a product. Great communication about system behavior can reduce a number of communication related software defects by eliminating confusion. Traditional testers may be very familiar with writing test cases, and these are one form of documentation of desired system behavior that testers should write before development begins. Writing the test cases first means that developers have more reference material that will help them to do their job better. Lastly, testers have a technically focused role, and should take advantage of the...

Read More

Choosing the Right Measure of Progress

Posted on Aug 9, 2014 | 0 comments

Choosing the Right Measure of Progress

What is Progress? Defining progress will help you to better understand the difference between being busy and productive. Unfortunately, I often see very intelligent and productive people falling into the trap of doing busywork instead of making progress. In order to effectively combat this problem, it helps to define progress much in the same way we define a vision: envision the end state. Attempt to describe in detail the properties and aspects of the ideal end state for your current task or project. For example, if I was trying to build an office chair, I may describe the end state as follows: “I can sit comfortably in the chair for several hours,” “The chair is elevated from the floor,” or “I can rotate the seat of the chair easily.” Each of these properties can be easily measured, whether objectively or not, and represent true progress toward completing my goal. Tasks do not represent progress. Continuing with my example from earlier, there may be many things that I need to do in order to build a great office chair. Perhaps I need to buy materials such as wood, steel, and fabric. Maybe I need to sketch designs, cut out upholstery, and assemble arm rests. All of these things represent actions that I can and probably should do in order to reach my end goal of having a lovely new office chair, but none of these things represent true progress. Tasks act as hypotheses to be tested, not the results of an experiment. For each task you decide to act on, you make an implicit statement: “If I complete this task, then I will be closer to achieving my goal.” Experience probably tells you that even the most successful and accomplished people regularly learn otherwise. At the end of each completed task, and often before, we learn about new tasks that must be completed. For example, before you can buy wood you need to find a store that sells the wood. Perhaps the store is closed and you now need to find another store or different wood, let’s say you chose to change to a different kind of wood. Now, the store that sells the new wood is open, but when you get there, they have run out and recommend another store. You don’t have enough gas to get to the other store, so you start off toward the gas station. We now have a situation where in order to make progress on my chair, I must go to the gas station. If I measure productivity by the tasks completed I may find that I am immensely productive, checking off task after task after task. However, without any measurable and tangible progress being made on the chair itself, I am failing to achieve my goal. How you measure progress directly impacts whether you feel most productive doing busywork or accomplishing your goals.  Measure Real Progress The real challenge with choosing a measure of progress is recognizing incremental steps toward a goal that represent true progress and not the busywork of tasks that may or may not contribute to the goal. This can sometimes be more art than science, but focusing on realized benefits as a measure of progress will almost always deliver results. Since user stories are the primary measure of progress on most Agile projects, it is important that they represent realizable benefits as opposed to tasks. Using our existing example to tie everything together here, the following benefits would be great measures of progress on our office chair. The chair has a designated seat. The seat is elevated from the floor. The elevated seat can hold...

Read More

The Argument for Emergent Design

Posted on Jul 30, 2014 | 0 comments

The Argument for Emergent Design

The Opposition Argument Let’s start off this post by exploring the arguments against emergent design. I often hear many different arguments and reasons to oppose emergent design, so I will share the few that I hear most often. First, I often hear that emergent design leads to wasted effort because building a product that only supports a small initial set of customer needs will later require the product development team to rebuild the entire product (or a substantial piece) in order to support more advanced customer needs. The wasted effort comes from the rework that the development team needs to perform in order to build the advanced features on top of the naively simple system. It follows that if the development team had asked about the advanced features before starting work, then the product could have been built from the beginning to support both the simple and more complex cases. Almost every experienced developer has several horror stories that they can very quickly relate about a poorly planned project where they endured the hardship of throwing away hours of work because the project direction (though, seemingly not the customer’s needs) changed frequently and dramatically. In addition to this, I also often hear that emergent design leads to poor system architecture because developers do not align their current efforts with a greater picture at the outset. Without steadfast direction, developers create code that works, but lacks cohesion and tends to feel more complex than necessary. After a long enough timeline of working without a big picture, the overall system architecture suffers. Almost every experienced developer has several horror stories that they can very quickly relate about a poorly designed system that was impossible to update or maintain because the system lacked a cohesive overall system architecture. While I have definitely heard many more arguments against emergent design, I have heard these two far more often than any of the others, so I want to explore these problems a little bit deeper. Product Rework and Bad Design I ask you to set aside any preconceptions for a brief moment while we try to discover the root causes of bad design and product rework. In my experience, product teams typically blame product rework and bad design on poor or inadequate initial planning. That is, if we spend more time on planning, or plan in a more skillful, careful, or comprehensive manner, then the outcome of bad design and rework can be avoided. This seems logical at first glance, so very few people give this a second thought, and attempt to solve these problems by bolstering planning activities. Improved planning generally leads to better outcomes, which gives us the impression that we have solved the problems, until they reoccur. But they do reoccur so frequently in software development that it has led some teams into a state we sometimes refer to as “analysis paralysis.” That means that the product development team cannot make any progress on the product because all of their time is spent in planning activities. The problem of bad design and rework cannot be completely solved by better planning, but both proponents and opponents of emergent design agree on one thing: The best designs tend to emerge from product development teams that have a very deep and complete understanding of a problem and its solution. So, if better planning does not completely prepare us for success, then what else must we do? True Emergent Design Great planning will only take a team so far because it is only one source of information and learning. In particular, planning typically revolves around certain constrained activities such as lengthy conversations with colleagues and customers, documenting decisions, drawing diagrams,...

Read More

Predicting Scope, Schedule, and Cost

Posted on Jul 16, 2014 | 0 comments

Predicting Scope, Schedule, and Cost

Every project has three key variables… The three key variables that project managers must manage are typically scope, schedule, and cost. Businesses tend to create project plans that assign constant values for these variables at the project outset, therefore creating the illusion that projects have fixed scope, schedule, and cost. So before we continue, allow me to dispel a myth so we can start down the road to improvement. Fixed scope, schedule, and cost is an insidious falsehood that undermines the entire project management institution and community. Whether or not you openly admit it… This is a true statement; and I must assume you understand and have seen the truth in your experiences on real projects. We propagate this lie despite the deleterious consequences. However, I believe that there are important sociocultural reasons why we do this, so don’t beat yourself up if you still live the lie. What is more important is the acknowledgement that in the real world scope, schedule, and cost are not fixed, and we must become excellent at predicting these things for business planning and management purposes, no matter what process we use. Agile projects have a unique set of challenges in predicting scope, schedule, and cost that differ from traditional projects. This is because Agile projects are managed differently… Different in key ways that impact scope, schedule, and cost. Below are some excerpts of the relevant principles of the Agile manifesto. Scope: “Welcome changing requirements, even late in development…” Schedule: “Deliver working software frequently, from a couple of weeks to a couple of months…” With constantly changing requirements… We need a way to measure and predict the final scope of a project that not only allows, but welcomes change. Furthermore, we must deliver a working product on a shorter timescale than the 12+ months that is sometimes typical in the IT industry. Lastly, if our customers and executive sponsors are accustomed to infrequent product revisions, then it is quite likely that despite any incremental releases, we will still need to predict and communicate the future delivery dates of major product features that are in development for several months (or even years) at a time. Okay, enough build up, how do we predict successfully? I will share with you some tried and tested techniques that I have been applying very successfully at my clients for some time now. However, I must warn you in advance, these are advanced techniques that will take some time and patience to understand and apply, so be prepared to run experiments and learn from your mistakes. Create a predictive model. I have found that the best way to model Agile software product development is by first assigning a relative point scale to user stories, which we refer to as “Story Points.” Numerically speaking, using a fibonacci scale seems to yield the best predictions, so we will use that scale. It’s important that this step is done correctly, and that the story points are relative to other user stories, not a point of reference, like “1 point is equal to 1 day of development.” Instead, a 1-point story must be roughly equivalent in effort to any other 1-point story, and a third less effort than a 3-point story. On Agile projects it is wasteful to break down long-term projects into fully defined user story backlogs because we expect detailed requirements to change, so we must also assign relative sizes to larger work items (I will use the term “feature”) that we haven’t broken into user stories yet. We can use the same fibonacci scale, but to avoid confusion, I recommend...

Read More

Agile Jenga

Posted on Feb 8, 2014 | 0 comments

Agile Jenga

After several years… Working across a variety of industries trying to adopt Agile development practices, I can confidently say that Agile Jenga is the single most popularly played game among Agile adopters.  If you are part of a large organization that is adopting Agile after years of doing things the traditional way, then you already know the rules.  Here are the rules for those who are unfamiliar with the game…   Much like the party game Jenga… We need to start by creating a structure or model of our desired software development process as a set of building blocks.  In order to achieve this first step, most organizations will pick a popular existing structure like “Scrum” or “SAFe.” These structures are already set up for us so that we do not have to go through the tedious process of learning how to build such a structure, which is typically seen by most large organizations as a waste of time. Now, as the name of the game suggests, the goal is to create a new structure based on the original that stands much taller and more statuesque to reflect the uniqueness and complexity of our company by moving pieces around.  We carefully remove the building blocks from the foundation of the existing structure and defer them for later adoption by placing them at the top of the “todo” list. As the game progresses… It will start to become more and more difficult to continue deferring foundational blocks without everything crashing down around us, so we need to get some structural experts to help keep the structure steady.  We refer to these “experts” as Agile coaches, and they have years of experience helping organizations play Agile Jenga.  However, we need to be careful when choosing an Agile coach, because there are many coaches out there that want to revert all our hard work by taking the pieces off the top and putting them back into the foundation! We refer to those unproductive coaches as “Agile zealots,” and usually can keep them out of our organization by interviewing coaches to make sure they understand the rules of the game beforehand. Sometimes, despite our best efforts and expert support… The structure may become unsteady and people may start to become nervous that everything is going to collapse.  There are many different ways we can address this, and just like any game, trends in the ways to address it have emerged and shifted over time.  Five or more years ago, when Agile was a young concept, the most common way to address the unsteady structure was to knock it over on purpose and play a different game out of frustration. As organizations matured in their Agile Jenga playing, new trends emerged, like leaving some of the important foundational pieces alone, and deferring other pieces that were less load-bearing in the structure.  Good Agile coaches could help us to better understand which pieces were more load-bearing, and which ones could easily be deferred, which allowed us to get much taller and more complex structures without full collapse.  We hired good coaches earlier so that we didn’t make weak structures prematurely.  However, even with this more advanced technique, we still eventually discovered that the structures would become unstable. Let’s skip forward to the trends of today’s savvy Agile Jenga players. We now have access to many case studies from large companies that have built monumental and impressive structures that have, to date, entirely avoided collapse. We look at these in awe of the marvel of engineering that created them and hope that one day our organization can also...

Read More

Let ‘Form Follow Function’

Posted on Jan 23, 2014 | 0 comments

Let ‘Form Follow Function’

There is a reason that your frisbee is shaped like a disc… And it isn’t just because the manufacturer sells more frisbees that way!  In this post we will explore some novel ways to apply this concept to building a software product that will not only save you time and energy during product development but also lead to better designs for your products.   Certainly, you have heard the phrase “form follows function,” right? Well, if you haven’t, the concept is very straightforward: the form or design of an object is largely influenced by the function or purpose it is meant to serve.  There are numerous ways to define what we mean by “form” or by “function” in a software context, so let’s start out with some common definitions for these things.  Generally speaking, the form of a software product can be defined as its user interface.  The function of a software product, therefore, may be defined as the benefits or features that it offers to its users.  Despite these definitions, the point I’m trying to make is different and more subtle than “let your user interface be dictated by your software product features”, so bear with me.  The subtlety lies in the definition we use for “follow,” among other things. Before we jump to the definition of “follow”, however, let’s talk for a moment about the way that typical software products are conceived.  In organizations large and small, software products often begin with the user interface or a description of the user interface.  Sketches and mock ups of interfaces usually help to create a better visual picture of what the software will do, and in particular, how its users will interact with it.  This is logical because people who use software are also the ones defining new products, and their window into the product is through its interface.  Just prior to creating UI sketches, we tend to talk about what a software product does.  “What does (or will) it do?” is usually the first question people will ask our software product visionaries.  The question we should be asking first and foremost is: “what problem does (or will) the software solve?” Until we can let go of the question “what does it do?” we will never be able to shift away from conceiving of software products apart from their user interfaces, but there are big benefits to this type of lateral thinking. Let’s return now to the definition of the term “follow.”  When originally conceived, the “form follows function” phrase was meant to define “follows” as “influenced by” or perhaps even “dictated by.”  This is not the definition that I want to use in the software product context.  Rather, let’s use the first definition we find in the dictionary: “to come after.”  This is more of a temporal definition that has to do with the logical ordering of things in time.  So, in our software product context… Let the user interface come after the benefits and features have been realized. This may either seem like nonsense to you or totally obvious, but it’s neither an impossible challenge nor a suggestion to build N-tier enterprise software starting with the data model layer and finishing with the UI layer.  Allow me to clarify with examples. Ultimately, your product is built by a software development team We need a way to define the product so that it can be built by the team.  If we were trying to answer the question “what does it do?” then we might define our product as a set of software “requirements,” such as “the software shall display a modal...

Read More