Unnecessary complexity, why does it happen?

In software industry, there’s a strange disease called complexity. Especially complexity that exists for no good reason. I don’t know why it happens exactly but I know it’s widespread from experience. Software development is a complex process and the problems we deal with are somewhat complex as well. But these problems are hardly rocket science. They are nothing compared to space exploration or cancer research, for example.

Yet, every day, I see unnecessary complexity scattered around software projects. It’s usually a hassle to read someone else’s code. Trying to use a new framework is a steep learning curve. A new programming language? That’s even worse. Something that intuitively feels like it should be simple, usually ends up being quite complex to implement. Why is that? I don’t know the exact answer but I have some ideas.

First has to do with the people involved in software development. These are usually smart, motivated people who like challenges and puzzles. Otherwise, they wouldn’t be in the business of writing thousands and thousands of lines of code in-front of a screen, day in and day out. Smart people have the tendency to over think, over engineer, over analyze. While regular people have difficulty in grasping complexity, these people have difficulty in appreciating simplicity. When you have so many smart people around working on the same problem, complexity arises naturally and simple solution are often forgotten or ignored.

Second has to do with human nature. People are social animals with egos and they like to assert their dominance. In animals, dominance is often asserted with raw physical power whereas in humans, it’s more about intellectual power, at least in software development. In a software project, whoever shows the mastery of complex problems and solutions is often assumed to be a competent person. In design meetings, everyone loves to yell out complex ideas and buzz words to assert their competence within the group. While mastery of complex concepts is definetely a good attribute, a better attribute is to have the ability to find the simplest solution to a given problem but the latter is often ignored because it doesn’t help in asserting dominance.

Third has to do with the overall lack of discipline in software design process. While designing software systems, there are a lot of moving parts, a lot of people involved and there’s limited time and patience. The whole process is often rushed due to time and patience constraints. In that environment, it’s easier to find a “good-enough” solution and move on. That works for part of the problem but when you add many “good-enough” solutions, you usually end up with an over-engineered, over-complex system overall. There’s most definitely a simpler and more elegant solution but it requires a more disciplined and more rigorous design process that hardly exists in a typical software shop.

Given all this, simple problems that should have simple solutions end up with unnecessarily complex solutions. Complexity is like a virus. Once you have it in an inner layer of the system, all the outer layers eventually suffer from the same problem. Pretty soon, you have a overly complex system that nobody knows how to maintain, every change causes a ton of other changes. Unnecessary complexity turns the optimistic, positive, creative process of software development into a fearful, dull and frustrating process both for creators, testers and users.

All of this happens because we didn’t take the time to find a simple solution and we didn’t resist the waves of complexity during software design. Next time you design a piece of software, make sure you spend some time to find a simple solution. A solution that is easy to grasb, easy to explain, easy to implement and easy to test. Such a solution exists, you just need to spend the time to look for it and you need to have to courage and will to resist the evil of unnecessary complexity. The success of your projects depends on this.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s