ContentThere has been no shortage of media coverage regarding the trend of offshoring software development efforts to India, China, etc. However, most of those articles tend to be written from a political or economic viewpoint, and few really touch on the day-to-day reality of the business and technical implications of offshoring. Over the past year or so I’ve spent a great deal of time and energy analyzing the practicalities of this trend, both at a high level and directly managing offshore teams.
First, I should explain that I am fundamentally pro-globalization, and a firm believer in open borders, free trade, and promoting a worldwide culture of interchange of ideas, goods, and people. The ideal society is one in which people participate by choice, as determined by their free will. Progress, as I would define it, is contingent upon the dynamic interplay of disparate points of view. As even a cursory study of complexity theory or cellular automata reveals, to maximize the production of information you must induce the “edge-of-chaos,” or the state between order and chaos where the phase transition occurs. As modern, industrialized societies tend to gravitate naturally toward order rather than chaos (order being better suited to meet the basic human needs of safety, reproductive potential, and sustenance), the challenge is to provide and adequate seeding of chaos. Closed societies simply become stagnant and politically and technologically inbred, whereas open societies provide fertile soil for innovation.
Thus on a macro level, the concept of offshoring is sound — embracing the unique perspectives of developers from other countries is a opportunity to enrich the development culture and increase the total dynamic potential of engineering teams. In practice, however, this is found to be a rather Pollyanna-esque perspective, as a number of forces are aligned against this potential.
The dominant corporate culture at most companies is structured not to embrace dynamism and edge-of-chaos development, but rather to impose order above all else. Hierarchical organizational and managerial models, the need for predictable revenue forecasting (and predictable revenue growth) as demanded by market investors, and the general individual need for security in a capitalist society with few safety nets, all work against the potential for optimal innovation. In this specific instance, offshore teams will never be encouraged to innovate or bring fresh perspectives to the table. On the contrary, offshore and outsourced teams are often chosen specifically for their ability to follow specifications and adhere to engineering guidelines authored at the corporate headquarters.
These problems exist with onshore development as well, but they are magnified significantly when the principle challenge in managing offshore resources becomes one of control rather than one of empowerment. The net innovation at a corporation will actually decline during the offshoring process, as a greater percentage of onshore resources will be dedicated to controlling and managing the offshore effort (rather than innovating), and few, if any, of the new resources will be presented with the opportunity to do anything other than follow rigid instructions.
This being the case, then the offshore value proposition is reduced to that of seating lower-cost developers at the keyboards. To someone focusing only on bottom and top line concerns, this may seem like a perfectly valid equation. If one looks at an engineering group with 100 developers and hears that they can they can lower the total salary expenditure by 35% simply by shifting those developers offshore, then the logic in straightforward and reasonable.
But what this does not take into account is the actual mechanics of software development. In the actual engineering process, by far the most important characteristic is communication. Communication among developers, communication between the business and the developers, between quality assurance teams and the business and the developers, with technical analysts and information architects, with operations teams, with marketing teams — the spiderweb of connections that need to be formed and maintained is exponential. In every development effort, onshore or offshore, the most critical factor in the success of the project is how well information is propagated throughout the team. Tools to disseminate information range from hallway conversation to meetings in conference rooms to conference calls to email to mailing lists to web sites to chat rooms to instance messaging to video conferences. Successful projects employ any and all of them in abundance, and successful companies spare no expense in preparing and constantly improving the communication infrastructure.
Entire software development methodologies have been formed simply to improve the environment for communication. The broad group of agile processes are fundamentally seeking nothing more than to better the flow of information between the people writing the code, the people testing the code, and the people that need the code.
Yet communication becomes a challenge on an entirely different level when multiple timezones and disconnected infrastructures are added to the mix. In practice, I found that teams separated by 12 hours can manage to fit in only about 1 hour on average of “face-to-face” communication per work day. Questions, and in a good development environment there will be plenty of them, can sit unanswered for hours while the other half of the globe is in bed. The lost half-days add up quickly, and most common response to this is to mitigate the loss by adding more bodies, which has the two-fold effect of a) increasing costs, and b) further complicating the flow of information. So while the general (and correct) industry trend is to promote the back-and-forth exchange of information between team members, the physical limitations inherent in intra-timezone collaboration work strongly against that trend.
Moreover, offshore developers should never be considered “drop-in” replacements for onshore resources. Any company that decides to supplement an existing team with one or two offshore resources because it looks cheaper on paper doesn’t stand a chance of seeing a benefit (and in fact, will almost certainly lose time due to increased communication overhead). The amount of training necessary for one developer effectively cancels out 1/3rd of their first-year production capability. Thus planning for offshore projects must be done well in advance — particularly as the requirements for accurately defining the product to be built are substantially increased to mitigate communication challenges. (Note that this runs entirely counter to agile doctrine, and effectively pushes split onshore/offshore development practices back a generation toward waterfall methodologies.)
And it is only fair to address realistically the question of cost savings with offshore development. Yes, salaries in India and China are far lower than American salaries. And right now, they are even cheaper, as offshore contract firms are eating a huge percentage of the costs to entice American businesses. But long-term, even elementary economic theory dictates that those salaries will rise (though clearly not to equal levels), and that once market saturation is reached, the fees associated with the staffing companies will rise dramatically. And while there are still vast untapped pools of educated developers offshore, the number of qualified offshore contracting companies prepared to provide quality service is very limited, and growth there is much, much slower. These factors will gradually (or, as I suspect, not even that gradually) erode the baseline cost savings.
Yet salary considerations are only part of the total cost. In order to even begin tackling the communication challenge, a massive up-front investment must be made in building out a communication infrastructure. Video conferencing systems and telephone bridge lines must be put in place and paid for. While VOIP technologies will ultimately drive the prices down, they are presently are very non-negligible costs. Database licenses must be procured, database instances must be installed and maintained, and the data itself must be replicated. Shared services must be migrated to a DMZ, and security policies and processes must be established.
Developer retention is another consideration — just as domestically, the best offshore talent will be subject to moving between companies, and this problem has traditionally been magnified because there is no company loyalty when the firm signing your paycheck isn’t the one you are writing software for, but is instead a third-party outsourcing firm.
And perhaps the most important cost consideration is that projects will simply take longer and be harder to complete when the development team is spread around the world. Costs will increase because of the additional development managers and project managers required to keep everything running smoothly. Domestic productivity does decrease as an environment of fear sets in over whether your job might be next to move. Developers and managers must be housed and flown into the US or overseas for training. All of these factors tend to actually increase the cost of offshore software development, rather than decrease it. But because so many of these issues are not accounted for directly against the offshoring initiative (such as decreased productivity), it is sometimes hard for those making budgets to calculate the total cost.
Quite frankly, those third-party research and analysis companies that a producing the “objective” studies saying that offshoring is cheaper are speaking only what the corporate management and boardrooms want to hear. You don’t sell a lot of reports by saying there is not a silver bullet that makes software development cheap and easy. Quality software development is neither cheap nor easy. And companies that lie to themselves about that fact find themselves with cheap, mediocre software. There is no substitute for talent, just as there is no long-term substitute for innovating in the marketplace.
So, is it possible to make offshore software development cost-effective and long-term viable? I believe the answer is yes, but it will not work if the offshore resources are treated as interchangeable with onshore resources. To do it correctly, the corporation needs to a) own, not outsource, it’s offshore development (so as to be able to properly incentivize and retain talent), b) accept that the total costs are going to be comparable in the long-run, not lower, c) be willing move entire sections of the company, including business owners, marketing, etc., offshore as well (this is necessary to run fully agile teams offshore). But if the value is not in cost savings, why offshore in the first place? The answer is to recruit global talent and to promote innovation. India, China, Russia, etc., all have an abundance of brilliant developers, and recruiting in even the most tech-rich cities in America (Seattle, Boston, etc) is so competitive that there are not enough good developers to go around.
In summary, I am not at all opposed to the concept of offshoring. In fact, I very much believe in the possibility that greater collaboration between developers around the world can not help but to produce dynamic and innovative results. However, the structure of the modern American publicly-traded corporation and the development processes they have adopted, unless radically changed, serve to not only negate the potential cost benefits of offshoring, but also to cause a appreciable decline in output and innovation. Changes in corporate cultures and advances in communication technology can likely overcome these limitations. The best and brightest of companies will recognize this and adapt rapidly and reap the rewards. However, companies that are not willing or able to make radical adjustments (and this unfortunately includes not just many, but most, companies) will quickly find that their offshoring plans that made so much sense on paper ultimately cost the company it’s most important asset — innovation.