Noise factors vs. control factors

On the latest Circuit Breaker podcast, Bob and Greg talked about causal structures. This is when you’re trying to improve a system and, in order to get the effect you want, you first try to understand the mechanism of how the system works.

At 9:00 Bob gave an example of making dish soap:

Some people put a fragrance into a dish soap. But depending on the type of fragrance, it could be oil based. And so all of a sudden the surfactant is used to actually attack the fragrance. So you start to realize that there’s interdependence between these systems. Ultimately what we want to be able to do is design systems to be as independent as possible. But that interdependence then becomes a noise factor — something we have to know about, but we also have to then figure out how to design around.

When I heard that, I had to hit pause and listen again. It’s so dense. “That interdependence becomes a noise factor” —what?— “that we have to then figure out how to design around” — huh? Let me unpack this a bit and talk about how it applies to software.

In my 20+ years making software, I never heard about “noise factors“ from software people. I learned about it from the world of physical engineering and manufacturing. But it’s one of those things that, once you know it, it applies to everything.

Whenever we’re designing a mechanism, there are things that we can control and things we can’t control.

For example, suppose we’re trying to add a new section to our marketing website to highlight customer reviews. We see a place where we can insert some highlighted quotes from customers in between the other content blocks that are already there. Now in theory, we could design these additions any way we want. But if we don’t want to also redesign the whole marketing site, we need to constrain our changes so that they fit into current design. There are maximums for the width and the height of our new section. And concerning the interior of our new block, there are also constraints. We won’t just use any font or color we like, because those things should match the rest of the page.

In this example, the control factors are all the things we get to decide: Which quotes to include, whether the block has some header text or not, the language of the header text, the position of the name of each reviewer relative to their quote, whether to use just  first name or first and last name, whether to include their title, etc.

The noise factors are all the things that we can’t control. That’s the width of the container that our block fits into, the maximum height, the colors and typography of the existing site, etc. There can also be noise factors in the implementation. For example, the CSS framework used for styling the elements, or the content management system used to build the page, or the Javascript framework available, etc.

Note how this is all relative to a specific effort. It could very well be that, in the future, we want to completely redesign the marketing site. At that time, all those things that were formerly noise factors would become control factors. We could change the whole color scheme and layout if we want.

This is easy to see when we talk about a marketing page. The constraints are right there on the surface. It can be trickier when we talk about backend projects. When it comes to changing functionality, in theory we could control everything. It’s all just software! But if we take that attitude, the scope runs out of control, and the ripple effects and interdependencies make projects run longer and longer. You set out to change one thing, and before you know it you’re fighting with some totally different part of the system or waiting on another team.

When projects involve software changes, we need to explicitly decide which things are control factors and which things are noise factors, again relative to that specific project. That’s how we get to a definition of “done” that is achievable for a small number of people in a fixed time.

What does that look like? Suppose we have a system that sends one-off invoices. And now we’re doing a new project to allow setting up recurring invoices. One of the designers on the project adds a tiny little change to the invoice template: it should show a little notice that indicates when the invoice is a recurring one. In theory, this should be no problem to change. So everyone agrees it’s “in scope.” But as time passes, and the team moves from the third to the fourth to the fifth week of the project, they suddenly find out that invoices are sometimes rendered as PDFs. And the tooling that renders the PDFs is … let’s say … “legacy”. Making changes and testing with the PDF tooling is turning out to be way harder than it should be.

This is an unexpected interdependence between the new recurring invoice functionality and the PDF generation code. If we treat everything across this interdependence as a control factor — meaning all of it is in scope — that leads down the path of growing the scope to include fixing or replacing the PDF tooling. That means delaying the project, piling up unfinished work, and so on. (This kind of power-through-it response is more common than you might expect!)

The alternative is straight from Bob’s quote: to treat the interdependence as a noise factor. That means breaking the chain by deciding that the PDF generation isn’t going to change as part of this project. But remember that wasn’t the end of the quote. The next part is “…and designing around it.”

If changing the PDF tooling is off limits — for this project, in this time window — what are our options? The first option is to just leave out the notice on the PDF, which leads to an inconsistency between the PDF and other views of the invoice. Someone may catch that in the future, file it as a bug, and now we’ve created future work we don’t want.

Another option is to drop the notice entirely from all displays of the invoice, so the PDF remains consistent. This might be fine, but perhaps the designer had a good reason for wanting the notice? Simply dropping a requirement because it’s hard can accumulate as low UX quality over time.

Very often, there’s a third way. In this case, the team can declare the invoice template in all its forms to be a noise factor. At the same time, they could decide that the email where the invoice is delivered is now a control factor. They could redesign the solution slightly by mentioning in the text of the email that this is a recurring invoice, above where the existing invoice template is rendered.

This is what designing around the noise factor means — taking the noise factor as a constraint, stepping back, questioning assumptions, and finding a new design that still meets the overall goals with the control factors.

Note that declaring something a noise factor doesn’t mean leaving it behind forever. In this example, deciding the PDF tooling is a noise factor actually opens up that work for parallelization. The PDF tooling and the recurring invoices become independent efforts, that can be worked on by different people at different times. The invoice team doesn’t have to wait to deploy their new functionality, and meanwhile — or in the future — another team can replace what happens behind the PDF generation API to use better tooling.

These decisions compound project after project. They make the difference between regularly shipping vs. long projects that spiral out of control. This is what Shape Up called scope hammering. It’s not just kicking things out of the requirements. We don’t have to shrink scope by eliminating outcomes. We can shrink scope by choosing not to touch certain things and then designing a way to get the outcomes we want with the control factors we have left.