On Great Engineers, Part 1


Part 1 of a 3 part series...

The importance of hiring great engineers can not be overstated.

The engineer is the lifeblood of today's corporation. If your company is dependent on revenue that is generated in whole or in part on systems developed in-house, then you are running a software company. It doesn't matter if you are also a marketing company, a retail company, or a services company, if you rely on the success of your technology, then you need to build your engineering team as you would at a software company.

Technology can make or break a company. And today there are very few large corporations that are not in some way counting on their software development teams to innovate, or at the very least, consistently deliver quality product. At most modern corporations, downtime in computer systems directly corresponds to lost revenue. Even short-term outages in fulfillment systems, customer management systems, or online points of sale can cause an catastrophic halt in business processes.

Many of these systems can, and frequently should be, outsourced to true software development companies. If there are third-party vendors that can handle a particular task, then unless you are prepared to run the in-house software development team as it needs to be run, you should consider handing it off. Running a software company is challenging and expensive -- and short-cuts and compromises made here will have disastrous and long-term consequences. Of course, all of the standard caveats apply when selecting a third-party software vendor. You are only as good as your supplier, and as much care must be taken in overseeing that relationship as is required to manage it in-house. Still, unless your company is prepared for the commitment to doing it right, going outside is frequently the best option.

However, if your business is dependent on innovation in technology, in getting a new product to market quickly, or making a vision of new or better services a reality, then your company has no choice but to develop the software itself. (Outsourcing one's core intellectual property is one of the most well understood, and confusingly, most frequently repeated blunders in business.) And once you've committed to building the software, you've also committed to building a software development team.

There is no substitute for great engineers. No matter how substantial the genius of the business management team, a mediocre software development team can never produce the quality output of a great engineering team. Neither adding more mediocre engineers nor increasing the total number of hours a mediocre team works on a project will produce equivalent results. Far too often management teams believe that they can make due with lesser programmers. They feel that they can either simply hire a larger, if individually weaker, team. Or that they can just stretch development time-lines to compensate for slower developers.

But the fact of the matter is that lesser programmers don't simply code slower -- they also write code that can be devastatingly costly. That cost comes in the form of longer development cycles, more frequent downtime, higher bug incident rates, and less code re-use. Study after study has shown that the later in a development cycle a bug is found dramatically increases the cost of fixing the issue. Mediocre developers, almost by definition, write buggier code, and catch those bugs later in the cycle. Moreover, less able developers are not writing code that future teams will want to extend later on -- leading to complete rewrites of systems where simple modifications should have been possible.

Another downside to hiring average developers is that the company soon adopts a culture of mediocrity. The programmers won't trust each other to do a good job and will want to share less and re-use less of other people's work. The best engineers will spend the bulk of their time trying to get their ideas across -- sometimes an altogether wasted effort -- and will spend less of their own time writing good code.

Some signs to look out for as a management team in determining the overall skill level of the engineers are as follows: do your projects frequently come in late? Are the time and resource estimates being done by project managers, rather than the developers themselves? Do you tend to distrust your developers when they raise issues? Do often hear how you need more project managers and more QA resources? Do your developers grumble and complain? Is there high voluntary turnover among the best engineers? Are the developers frequently saying that they need to rebuild parts of the system rather than extend it? Do your systems crash or have downtime? Do you hear from team leads that they need engineers with a different skill set than the one they have available?

If those things are true, then you might have a problem with the quality of your engineering team. But how does one end up with a mediocre team in the first place?

The first and most obvious problem is simply hiring bad engineers. This can happen when the hiring manager just don't know what makes a good engineer to begin with, or when the team does not do a proper tech interview. (These two points are covered in detail in parts 2 and 3 of this series.)

But it can also happen when the business consciously decides to save money by hiring weaker engineers. It's true -- the best engineers cost more in salary. But a few thousand dollars here or there is nothing compared to the lost revenue of downtime, or having to double the size of a team to get a simple job done, in having projects delayed, or in having to rewrite entire systems. A good engineer, who may cost 50% more than a mediocre one, pay back the premium within months by writing much, much better code. (If I could explain just one thing to every CEO in America, it would be that cutting corners with your engineers is cutting directly into the bottom line, and the math never lines up in the company's favor.)

Another classic way in which a company ends up with bad engineers is by acquisitions or mergers. These engineers weren't directly hired by the company itself, but rather came along when different teams were brought together. The median skill level for one team many be different than another, and those differences in talent can bring down the overall quality for a combined team. And in some extreme cases, questionable management decisions can be a contributing factor. (I recall one situation -- and I wish I were making this up -- in which one division of a corporation was only allowed to hire from a pool of people being let go by another division of the corporation. Needless to say, it wasn't the great engineers who were being let go.)

The best way to build a team of great engineers is to do it while growing. Budget for the best, and hire fewer of them if you have to. And much as it pains me to say it, if you have to let a few of the under-performing developers go in order to make room for one good engineer, it is in the best interest of your company to do so. But once you've built that team, try hard to keep them together. Be loyal to the good engineers, and they will be loyal to you. A team just gets better over time -- and you will be astounded by what can be done with a small group of highly motivated, highly talented developers. (I know I'm amazed each and every day by the team I work with.)

And though it is a bit of an aside, once you've made the investment in a great engineering team, go out of your way to help them do their jobs. Like any professional, software developers need certain specialized tools to do their work. For software engineers, those tools are computers (and lots of them), but also things like good chairs, desks, private and quiet offices or cubicles, conference rooms, white-boards, snacks and coffee and drinks, books and magazine subscriptions, software licenses, good lighting, etc., etc. Be sure that you give the developers whatever they need. The developers will tell you themselves what is working and what isn't. Don't think that just because one thing works for a business manager it will work for an engineer. Denying these things is like buying a 300 hp sports car and putting 87 octane fuel in it and wondering why the performance is so average. Great engineers will do the best they can with the tools they have available, but a small investment in infrastructure will result in significantly more productivity.

These ideas are common sense to a software development company. Say these things around the halls of Microsoft, Google, or Amazon and they will react like you just told them that the sky is blue. But what far too many corporations haven't understood yet is that they too are software development companies. And that they need to act like it or the software they are building, and depending on, will be second rate. Depending on second rate software holds the entire company back -- and I've never met a CEO, or a share-holder, that wants to run a second-rate corporation.

(Continue reading part 2 for more on what makes a great engineer, and part 3 on how to hire great engineers...)