Author Archives: Mete

Unnecessary complexity, why does it happen?

In software industry, there’s a strange disease called complexity. Especially complexity that exists for no good reason. I don’t know why it happens exactly but I know it’s widespread from experience. Software development is a complex process and the problems we deal with are somewhat complex as well. But these problems are hardly rocket science. They are nothing compared to space exploration or cancer research, for example.

Yet, every day, I see unnecessary complexity scattered around software projects. It’s usually a hassle to read someone else’s code. Trying to use a new framework is a steep learning curve. A new programming language? That’s even worse. Something that intuitively feels like it should be simple, usually ends up being quite complex to implement. Why is that? I don’t know the exact answer but I have some ideas.

First has to do with the people involved in software development. These are usually smart, motivated people who like challenges and puzzles. Otherwise, they wouldn’t be in the business of writing thousands and thousands of lines of code in-front of a screen, day in and day out. Smart people have the tendency to over think, over engineer, over analyze. While regular people have difficulty in grasping complexity, these people have difficulty in appreciating simplicity. When you have so many smart people around working on the same problem, complexity arises naturally and simple solution are often forgotten or ignored.

Second has to do with human nature. People are social animals with egos and they like to assert their dominance. In animals, dominance is often asserted with raw physical power whereas in humans, it’s more about intellectual power, at least in software development. In a software project, whoever shows the mastery of complex problems and solutions is often assumed to be a competent person. In design meetings, everyone loves to yell out complex ideas and buzz words to assert their competence within the group. While mastery of complex concepts is definetely a good attribute, a better attribute is to have the ability to find the simplest solution to a given problem but the latter is often ignored because it doesn’t help in asserting dominance.

Third has to do with the overall lack of discipline in software design process. While designing software systems, there are a lot of moving parts, a lot of people involved and there’s limited time and patience. The whole process is often rushed due to time and patience constraints. In that environment, it’s easier to find a “good-enough” solution and move on. That works for part of the problem but when you add many “good-enough” solutions, you usually end up with an over-engineered, over-complex system overall. There’s most definitely a simpler and more elegant solution but it requires a more disciplined and more rigorous design process that hardly exists in a typical software shop.

Given all this, simple problems that should have simple solutions end up with unnecessarily complex solutions. Complexity is like a virus. Once you have it in an inner layer of the system, all the outer layers eventually suffer from the same problem. Pretty soon, you have a overly complex system that nobody knows how to maintain, every change causes a ton of other changes. Unnecessary complexity turns the optimistic, positive, creative process of software development into a fearful, dull and frustrating process both for creators, testers and users.

All of this happens because we didn’t take the time to find a simple solution and we didn’t resist the waves of complexity during software design. Next time you design a piece of software, make sure you spend some time to find a simple solution. A solution that is easy to grasb, easy to explain, easy to implement and easy to test. Such a solution exists, you just need to spend the time to look for it and you need to have to courage and will to resist the evil of unnecessary complexity. The success of your projects depends on this.

Leave a comment

Posted by on December 25, 2014 in software industry


Accountability and Sense of Ownership in Software Development

One of the most overlooked concepts in software development is accountability. Accountability means a team is totally responsible for successful implementation and execution of a piece of software or a service from the beginning to the end. From the time the software/service is envisioned and designed, to the time the software/service implemented and gets used by the end users, a single group of people is involved in the whole process and once it goes out of the door, that team is held accountable for the successes and failures for the end result.

This is the only way to make sure there’s a sense of ownership in a software project. Sense of ownership ensures that people in a project feel some kind of emotional attachment and responsibility towards the end result. Without a healthy dose of sense of ownership, it’s very easy for teams to develop an “ain’t my problem” kind of attitude and that’s not healthy to have in any project.

In big software companies, priorities often change and as a result, some teams are asked to stop working on a certain project, transition the project to some other team and start another new project. To upper level management, this is not a big deal, they’re just moving “resources” around from one project to another. But anyone who has done some kind of software development knows that this is much more than simply moving resources around.

By switching teams and projects around, accountability and sense of ownership in a software project are ruined. The team leaving the project will not care about the project or held accountable for it anymore because they will have a new project to work on. The team inheriting the project cannot be held accountable or feel ownership for the project either because they were not involved from the initial design. In the end, customers suffer due to poor quality. After a few project shuffles, people in the company gradually stops caring about accountability or sense of ownership because they expect that sooner or later their current project will be reshuffled and they are not going to be accountable for that nasty production performance bug.

To sum up, if someone asks you how to make sure a software project fails? Quite easy. Assign it to one team, then reassign it to another team in the middle. In the end, you’ll have an orphan piece of software that nobody feels responsible or accountable for and you planted the seeds of a non-caring unaccountable culture. Sure way of killing current and future software projects.

Leave a comment

Posted by on November 19, 2014 in software industry


Team formation: another broken process in software development

In my previous post, I talked about why I think the technical interview process is totally broken. In this post, I want to talk about yet another broken process in software industry: how teams are assembled (or more like misassembled!) for a software project.

In a typical software company, a person is hired by a team to work on a particular project within that team. This makes total sense. However, after some time, things change. The actual time frame varies from place to place, it could be as soon as 1 month or something longer like 1 year or more. Either way, change is inevitable, the project either ends or completely changes or the team gets a completely new assignment. This is the point where it gets really weird. Now, the person who was hired to work on a particular project is asked to work on a completely new project. The hired person is basically stuck with the team he/she was hired into to work on a completely different project that he/she did not sign up for.

If the project and the team sound interesting, he/she can carry on and all is good. However, the problem arises when either the project is not interesting or does not align with the person’s goals or the team is not what the person hoped to be. At this point, the hired person has a few choices but none of them are good. He/she can try to get out of the project/team but that means he/she needs to find something else to do within the company. In big companies, there’s a lot bureaucracy around switching to a new group. You usually need to get your manager’s approval (which is quite awkward as it is) and then go through the process of making yourself accepted in the new group. It’s just not straightforward as it should be.

The other choice is to quit the company and join to some other company which is even a bigger deal with more interviews and process. Either way, you end up spending a lot of time not doing any real work. So, what usually happens is that, many people go on, accept their faith and continue working on uninteresting projects with uninspiring people because it’s easier for the time being. This naturally results in uninspired mediocre work and it goes on until either the company or the person realizes how ridiculous the whole situation is. The person either quits or gets fired.

I’ve seen this happen a few times to solid people and it is quite sad. It does not happen because person is incompetent but it rather happens because there is a misalignment between the person’s abilities and his/her place in the company. And there isn’t a good and open process to fix that alignment within the company. This is especially true in big companies.

I think the solution is not that difficult though. In big companies, there should not be strict hierarchies with solid lines between managers and developers. The weird notion that a single manager effectively owning a software engineer within a particular group should change. Instead, software engineers should be treated more like free agents. They should be assigned to projects for a period of time and when that time elapses, there should be an opportunity for an open discussion among the employee, manager and possibly someone from HR to figure out what makes sense as the next step. Software engineers should be presented with project choices within the company at that point in time and they should have the real option of staying in the current group with the current project or move to another project with little or no process. The company hierarchy should not dictate what people do and where. Rather, the business needs should dictate what the projects should be and then people should be able to gather around those projects freely, kind of like open source project model but within an organization.

Big companies will tell you that they already encourage people to move around within the company but in reality, this is far from true. First, the amount of process and bureaucracy involved is so high that many people do not even bother. Second, the culture at these companies does not encourage moving around. The very existence of a strict hierarchy dictates the boundaries that an engineer needs to adhere to and it is quite difficult to get out of those boundaries. Third, a software engineer is not presented with other choices once he/she is hired into a role and there is hardly ever an open communication about the employee’s next step. Employees end up sneaking around the company hierarchy to find the right alignment and it just should not be like that.

Open source software is so successful mainly due to people involved in those projects. Most of open source people put time and effort into those projects because they really care about them, not because of a random hierarchy. The same should happen in software companies as well. The company that figures out a way to let people gather around projects they care about voluntarily will create the winning innovative culture that is so lacking in big companies nowadays.

Leave a comment

Posted by on September 5, 2014 in software industry


Why the technical interview process is so broken in software industry?

I’m convinced that the technical interview process, the process that a software team uses to select good candidates to add to a project, is totally broken in our industry. It’s broken mainly because most technical interviews try to answer one aspect of a good software developer (Is he/she a good coder?), whereas software development requires skills well beyond just coding and those skills are completely ignored in most interviews.

First of all, we need to talk a little about what some characteristics of a good software developer are. The list might vary a little from person to person but overall I think most developers will agree with me with the following list:

  • Great coder. Writes maintainable, testable code for others to read and understand.
  • Passionate. Cares about the work and tries to produce the best work possible.
  • Team player. Cares about team members, seeks help and provides help to the team as needed.
  • Independent. Pulls his/her own weight and gets stuff done for the team.
  • Takes initiative. Does not just try to cruise along.
  • Great communicator. Communicates clearly and concisely both verbally and written.
  • Fun to be around. He/she is someone you would like to hang out with after work.

Over the years, I found that rockstar developers have most, if not all, of the characteristics above. What’s alarming is that only the first point (great coder) is traditionally measured in technical interviews. A candidate goes through hours and hours of hard technical questions (that sometimes the interviewer does not know the answer to, happens more often than you think!) and while coding skills eat majority of the time, the rest non-technical skills are either completely ignored or at best guessed through a series of informal chatty questions.

But why is that? Why don’t interviews test non-technical skills that are just as important? The main answer I can give from my experience is the lack of time and effort. In a typical interview setting, one has 30 mins to 1 hour max to test out a candidate. That’s barely enough time to get to know the person and gauge his/her technical background. It’s hard to predict if someone is passionate or a team player in 30 minutes, there are no questions that can answer “Is this an independent person?”. The other part is that it’s much easier for developers to throw couple technical questions to a candidate than to really try to get to know the candidate. Coming up with good technical and behavioral questions is hard and most developers do not have time or willingness to put their day work aside (where they get assessed at the end of the year for bonus) and spend time for interview preparations (where they have no incentive for a bonus). As a result, short technical interviews are bound to fail because they can only test whether the person is a great coder or not and nothing else.

So, are technical interviews doomed? Not really, if we put more time and effort into the process. In one of the groups I was part of at Adobe, after the initial interview, we used to give candidates a 1-2 day programming exercise to complete at home. The exercise was deliberately vague in order to force the candidate to ask questions and communicate. After the exercise was complete, we’d go through the code, asking him/her to explain code. This was very valuable because we got to see the coding style, we got a sense of the design style, communication style and got a more thorough view, away from the pressures of a regular interview process.

The same idea could be extended even further. Why not bring candidates for a trial period for a week or two weeks to your project and ask them to fix something about your project? They get paid, of course, you get to work with them as if they are coworkers and let them show themselves in a natural setting rather than the interview setting. At the end of the trial period, the team decides whether they want to work with that person or not. There are practical limitations (what if the candidate is currently employed?) and this is definitely much harder and time consuming for the team than the traditional interview process but it could definitely be made to work and it’s much better than hiring the wrong candidate and having him/her quit in the middle of the project.

The success and failure of a project depends a lot on the people who are part of the project. If we don’t have a good process to select good candidates for our project, what chance do we have to create the best software we can possibly create as a team?

Leave a comment

Posted by on August 26, 2014 in Interview


Thoughts on performance

Performance can make or break a piece of software, this is clear; nobody puts up with an unresponsive client UI or a slow back-end server in today’s age of software abundance and choice. Despite this, performance is often overlooked until late in the release cycle and doesn’t get the proper attention it deserves. This might not be a big deal in one release cycle, but after a few release cycles,  you can end up with a slow-moving giant that nobody knows how to fix instead of a lean fast machine that you used to have.  At this point, you either accept what you have or you take the hit and go through the painful process of profiling, analyzing, fixing, and in some cases redesigning. How does software end up like this in the first place? I can think of four reasons.

First, it’s not easy to sell performance. New features, especially visual ones where people can see and play, often are much easier to market and sell then subtle yet more important features like performance. Two more features on the release cycle looks better than 20% increase in throughput for example, so performance is not treated as a proper feature but rather seen as a thing to check at the end of the release cycle. As a result, performance does not get the time and resources it needs.

Second, it’s not easy to reason about performance. You need to define what metrics are being measured in the name of performance, define what qualifies as acceptable performance, define use cases where performance is important. This requires through understanding of the software and the use cases around it. It’s hard to get the scope of performance work right, it’s either too broad to implement or too narrow to produce anything useful.

Third, performance work is hard, sometimes harder than implementing the software. It usually needs additional tools/software outside of the software itself in order to write tests to simulate the agreed upon use cases and track some numbers around those cases. In most places, there is simply not enough time left over outside the feature development to build those tools. Even if you have all these tools, you need time to run the complicated performance scenarios and if numbers don’t look right, you need time to find out why; it can be anywhere in the code. You also need to do this all over again in every release cycle or get some time to implement automated performance tests that can track performance for you. This is a lot of work.

Fourth, performance is usually not tightly integrated with the overall feature development. When a new feature is being developed, there is a lot of focus on the new capabilities that the new feature brings from Engineering, QA, Product Management but not as much focus on two things: 1. How does this feature perform by itself? 2. How does this feature affect the overall performance? The result of ignoring #1 is that a new feature gets designed and developed without performance in mind, and by ignoring #2 the overall performance of existing system gets worse which is even worse.

Despite all this, professional software developers have the obligation to design and implement performant software, no matter what the realities of the workplace is and I think with some effort, performance can be saved and maintained over the release cycles with a few guidelines that I hope to share in a future post.

Leave a comment

Posted by on June 16, 2014 in Performance, Programming


Hybrid apps, not as straightforward as you think


I recently had the chance to go out of my comfort zone (server side development) at Skype and work on some client side mobile features. One of the topics that came up a number of times is whether a certain feature should be implemented in pure native code or whether it can be written as a hybrid feature using a mix of native and HTML/JavaScript code.

Developers are very religious about this topic. (It’s interesting how developers feel more religious about topics that they can fully understand, compared to topics that are hard to understand, but this can be another blog post). While some want to implement features in native code fully, others prefer to abstract away as much as possible using HTML/JavaScript as a common language and use technologies like PhoneGap/Cordova to figure out the details on how to run that code on various platforms.

As someone who used to work at Adobe, this reminds me of the discussions around Flash on mobile. At some point, Flash provided a plausible implementation of “Write once, run everywhere” myth on mobile phones. Writing code once and letting a common runtime like Flash to figure out how to run it on various mobile platforms was indeed a nice idea in theory. In reality, every mobile platform is different and they simply could not be abstracted away in a common runtime like Flash. That’s what I learned and that’s why I now believe that “Write once, run everywhere” is a myth.

Even if you don’t believe me that “Write once, run everywhere” is a myth, and even if we assume that technologies like Flash and PhoneGap/Cordova provide excellent frameworks and performance across all mobile platforms, I still think that developers should still be worried about hybrid apps.

Mixing different languages is hard and it increases the complexity of a project by tenfold. When you have a pure native app, you know exactly what IDE to use, what language to learn, what tools to use. You know exactly how to debug and how to test your application. Every native mobile platform has its known and accepted tools for the job.

When you start mixing native and HTML/JavaScript, everything goes out of the window. All of a sudden, you need to worry about this trojan horse sitting within your native code. Testing becomes difficult, debugging becomes difficult, profiling becomes difficult, code coverage becomes difficult…With pure native code, you’re constrained by the boundaries of that environment and that’s a good thing to have, so you can focus on your application. With a hybrid app, you end up spending a lot of time going back and forth between two programming models. It’s not impossible to navigate through but you need to ask yourself: Is it really worth the effort?

Don’t get me wrong. If you need a certain feature and if the most feasible way of doing it is in HTML/JavaScript, then by all means, go ahead and do that. I’m not against the idea of hybrid apps per say. What terrifies me though is how lightly the hybrid vs. native decision is made sometimes. It is not supposed to be an easy decision. While mixing things up, you need to be aware of the complexity you’re introducing on developers and testers. If the complexity outweighs the feature you’re introducing, you need to think again. If you’re going to create a hybrid app, you really need to consider that decision carefully and make sure you’re getting enough back from your hybrid feature for all the complexity it’s introducing.

1 Comment

Posted by on June 10, 2014 in HTML, Native, Programming


Specialization, an excuse for laziness and indifference?


In my previous post, I talked about specialized vs. generalist programmers and I concluded that there’s no clear answer when it comes to whether someone should be a generalist or a specialized programmer. While I advocated a balanced specialized vs. generalist approach, I think it all boils down to the personality of the developer. What is clear to me though is that specialization can be used as a good excuse for laziness and that’s not a good attribute to have in a high performing team.

As programmers, I’m sure we all heard quotes like “Oh, I’m a server side developer, I can’t do UI”, or “I’m an object-oriented Java guy, I can’t deal with JavaScript” or “I’m a UX guy, I don’t know how to work with databases” or worse “I’m not a tester, don’t ask me write tests”. These are unfortunate statements from people who claim to be programmers. They are basically saying “I’m lazy, I want to cruise along my comfort zone and don’t ask me to do something new where I might need to challenge myself”. Programmers like these bring down not only themselves but the whole team as well.

It’s very hard to be a rock star server side developer and also have great UX design skills, so I’m not advocating that everyone should be great in everything, that’s impossible. What I’m advocating though is that every developer should be comfortable with every part of the software development process. Sure, you can be an expert in database design but if the UX team needs help, you should have no problem in jumping in and helping with some UI work. At some point in development cycle if QA engineers fall behind, you should have no problem in writing and automating good tests for them.

Software development is a team activity. Users do not care who built the software or how the software was built or who was the frontend or backend developer. All they care is whether the software works as a whole and whether it solves a problem they care about. Given that, good programmers need to have the mindset to work on the software as a whole, end-to-end. They need to chip in and help out in any way they can. Usually, they will help with their specialized knowledge and skills in their specialized corner but more often than not, they will need to take a generalist approach and help in areas that they might not be comfortable with.

Another unfortunate side affect of specialization is that it breeds indifference. When a server side developer realizes that the UI of the product is a little off, he usually does not care because in his specialized mindset, it’s the UX designer’s job to get that right. Worse, even when he cares and he raises his concern about the awful UI, he usually gets the usual “This is UX’s job, don’t worry about it” kind of response. Either way, you end up with mediocre sub-parts of a product that make up a mediocre product and in the ends, users see a mediocre product as a whole and everybody loses.

Next time you hear a programmer say “Sorry I’m not X, so I can’t do Y”, tell them to stop being lazy and help the team out. Don’t let them use their specialized title as an excuse for laziness because their laziness will eventually affect the team and the last thing you want to be part of is a lazy bunch of developers that accomplish nothing. And never ever let someone tell you that “Quality/UX/performance is my job, don’t worry about it”. It’s team’s responsibility to get a good product out as a whole and every person is equally responsible of the every part of the product.

1 Comment

Posted by on June 3, 2014 in Programming

%d bloggers like this: