Team Augmentation: A More Sophisticated Approach to Outsourcing
Why are You Doing This?
You have a vision to execute. You have strategic goals and milestones to reach. You have limited resources. Your success depends on people—developers—sometimes very specific developers. Everyone needs developers. Other employers are courting your people right now.
The best talent is in short supply. Key technical staff may be planning to leave or have already left your organization. Keeping a bunch of talented and well-compensated developers happy, aligned and on task is a challenge we all know too well.
Facing these challenges, you have almost certainly considered outsourcing at least some portion of your development work. Why can’t some of this be someone else’s problem?
Suppose you have made the decision. The CEO or the board says this must be done. It’s up to you to find a vendor and solicit proposals. Now what? How can you keep from getting burned?
How can you be sure you aren’t going to trade one set of headaches for another?
Like you, many executives facing these choices already have some experience with outsourcing and know the benefits and drawbacks as they experienced them in one or more engagements.
Anyone who has been around the block knows outsourcing doesn’t automatically provide a “free lunch.” But there are so many success cases. You might even know someone who has done it successfully. The potential benefits—at least as presented by the vendors—are tantalizing.
You are going to do it. You need to succeed.
Self-knowledge is key to success. To really understand why we outsource, we need to dig deeper into our motivation.
Frustration Building an Effective Team
Many organizations seek to outsource software development due to the frustration they experience building and sustaining an effective technology team. This frustration can hobble an otherwise sound business with significant growth prospects. Whatever the causes, inability to effectively execute The Vision results in competitive disadvantage.
Sometimes changes can be made to relieve this frustration. However, many organizations will find some changes challenging to implement. In some cases it might even require the company to undergo a “change of identity” in terms of how they think about themselves. This is likely a more serious transformation than most want to contemplate.
If you have been in business for half a decade or more and have dozens or hundreds or even thousands of employees, big adjustments like this take strong leadership, commitment at the board level, and plenty of time. If business is growing, and the company is otherwise in good shape, why tinker with something as fundamental as your company culture or identity? You simply need to execute!
Some form of outsourcing will figure strongly in your plans.
You May Be a Software Company (and Don’t Know It Yet!)
Software is eating the world. Every business is becoming a software business. It may seem like an understatement but think about it for a minute. You use tons of software in your business, but are you a software company?
If you are a business of any size, you probably have CRM, HRM, ERP, Websites, customer service systems, accounting systems, and more. You are reading this because you have developers producing something or plan to in the near future. What are they really doing?
Lots of companies know they are software companies—that is, their business is made possible by specific software they design or produce. If you are producing software of a strategic nature—software that is key to the future of your enterprise—then you are already a software company.
You may not sell software as a service or sell apps in app stores but you are probably selling a service that is mediated or delivered by software or a product that contains software to function and deliver value. Software is strategic to your business and the IP represented in that software is part of what makes your company valuable. It may be time for your organization to recognize it.
The signs above point toward some changes you might make to address some of the frustration you experience in building an effective software team. Creating a CTO position, adopting a technical career ladder, improving the social status of software development staff in your company, are all good things a company increasingly reliant on its own software should do.
However, they may not be sufficient and they generally take time. Sometimes these changes are simply not possible.
Challenges Faced by Software Companies
Software companies deliver software as a service, distribute apps of some kind, or sell primarily online. Even when companies have all the typical attributes of a software company in place, they often still find frustration building an effective team.
Most software companies were startups once. Maybe your company is still a startup. Or maybe your company has graduated from startup phase but is trying to hang on to that coveted “startup culture.” Maybe you have a legacy business and a new business that is more reliant on tech. Situations vary, but some of the following scenarios will be familiar to you.
Stability, Turnover & Recruiting
“Boss, do you have a minute?”
That look on their face tells you everything. You smile through you teeth, but your eyes are not happy. “Sure, come in.” Another developer is checking out. Backfilling the last departure was hard enough. It took weeks of your and your team’s time. Sifting through candidates, evaluating, hiring, on-boarding, training. You planned to do a round of hiring in the second half of the year but now you have to do it again. You didn’t account for this on your schedule.
Your company may be located in one of the hottest markets for technical talent. We call this a Tier-1 Market. It makes sense to locate where the talent pool is largest. However, Tier-1 Markets are the most expensive areas in which to operate. Average job tenure for technical staff is shorter. The opportunities for developers are much more numerous and the culture of job-hopping is well ingrained. Equity is expected but not sufficient. How do you get someone to stay on past their first or last vesting period?
To keep high-performers, it is not enough just to provide “a good job” and some equity, you may need to have “a mission” that aligns with their personal values, the business growth must track expectations, and the work conditions must be up to par with others offering similar attractions. Employers feel relentless pressure on the staffing side. Turnover is a continual tax on productivity.
Alternatively, your company may be located in a second- or third-Tier market for developers. Some startups locate or start in places where the competition for technical staff is not so great. Maybe it is where the founders prefer to live. Maybe there is a greater strategic reason to locate there.
Typically this also means the talent pool is not very large. A moderately high-growth company in a Tier-3 market can quickly exhaust the pool of local talent and struggle to execute as skilled developers leave to Tier-1 markets with greater opportunity and higher compensation.
You have the task of executing the vision and getting all the necessary work done. Staffing issues are a continual thorn in your side.
Developers seek intrinsic pleasure in their work. One thing that keeps talented developers on the job is knowing that their work is sufficiently important and challenging. They need to feel that they are growing professionally and that they working on high-profile projects. One problem is that not all the work you need to get done is “sexy” or high-status enough to retain the interest of your key people. Some of this just requires competent professional execution. Naturally, you may look to outsource these parts and keep those staff you do have working on the most compelling parts of your system.
Team stability is key to consistent, predictable velocity. You need this to make your commitments to the board, your CEO, the product team, etc. As we will detail later, some outsourcing models can help you achieve sustainable, predictable velocity that give you greater confidence in meeting your commitments.
The Budget Squeeze
“We just don’t have the money to keep doing it this way.”
You don’t want to hear that but you know what it means. You have built something substantial but the company is still finding its feet in the market after a launch, transition or pivot. Your engineering team is a big line item on the budget. The company needs some breathing room to get to profitability or the next round of financing. You still have to execute but you have to do it with less.
Or perhaps you have been asked to do more but Finance will not let you scale up the number of developers you need to meet performance expectations. You can’t just ask people to work more hours. You know there is declining marginal utility in hours worked. You are borrowing from the future when you push people to an unsustainable pace and the impact on morale and turnover can be severe.
This is a natural time when companies look toward outsourcing with the promise of greater cost efficiency and sustainability.
Quality is Job None
“We have another regression in production and had to rollback. We’re currently investigating the impact on a number of user accounts.”
Great. How did that one slip through? You’ve spoken to your developers time and again on the need to manually and thoroughly test their own work. They set up a Continuous Deployment pipeline with fancy red-green indicators. It shoots code to production in a jiffy. They review each others’ code and sign off on merge requests. “But we have automated tests!” they say. Right. Not enough apparently. And when the developers encounter problems like this, the answer is always to write “more tests.”
“Give us some time to write more tests!”
You are suspicious. It sounds reasonable but it feels like sandbagging. Tests produce nothing tangible. It’s not a line item on your budget. “Automated Testing” is not on your roadmap. Users can’t see it. The CEO doesn’t care. Product just wants it to work. What are you supposed to do? Pad the schedule even more?
You know there is No Silver Bullet. Testing is a discipline and an art. You know about Test-Driven Development. Maybe your developers already practice this particularly rigorous discipline. Are you certain your developers understand where the value truly exists in their code? Are they making the right tradeoff decisions? Are they balancing the value in the tests against the time taken to create them, maintain them, and the additional friction they cause down the line? We seek a middle way of “just enough testing” or “just the right testing.” Who’s to say what that is?
When problems like this latest one slip into production, you feel there is no replacement for manual testing and good QA. However, QA needs to fit properly into your agile workflow and delivery pipeline. You’ve worked with QA teams before. They’re often not very agile.
When the testing workload looks enormous—preparing for a big release or product launch—companies often consider staffing up QA teams. How many do you need? Is it worth hiring full time? Should you use contractors? Sometimes QA looks like a good candidate for outsourcing. You need to bring up the Quality.
Everything is Taking Too Long
“Our last sprint came in at half of our projected output and a quarter of what Product wants.”
What is going on? Your team keeps missing milestones. Expectations seem reasonable. Maybe your team needs a cultural reboot. Hiring new people into the current team hasn’t really produced the boost you expected. You might consider outsourcing if your team is just not performing and upon serious reflection, you feel it is beyond salvation. Whatever the cause, you need to pick up the Velocity.
We Want to Do Things Differently This Time
“Our IT people are always standing in the way. I’ve had it with them!
Your operations team has all kinds of standards and restrictions about what can be done. They’re acting like gatekeepers, slowing everything down. Your development staff is too accustomed to working with them. They don’t want to alienate their friends and drinking buddies by upsetting the existing order.
You want to try new things. New technology. More agile approaches. Better practices. New infrastructure. Competition is driving you to move at a pace you just know your organization can’t currently match. You need some flexibility.
Time to think about outsourcing. With the right vendor selection, you can run your project outside of all the usual roadblocks and sticks-in-the-mud that seem more interested in their bureaucratic roles and job security than in the success of the company.
We Need to Work on The Legacy System
“I need a couple of you to go back on core for a while…”
A few years ago you embarked on a “big rewrite” or refactoring of your application. Now you have a new codebase or collection of codebases implementing the latest architectural patterns and cutting edge best practices. It has a fresh UI. Half your customers have moved over to the new system and more are transitioning every month. Development is full steam ahead.
However, some key customers cannot be moved over until next year when the new system matures to the point it can support them. You have a legacy system to maintain. Those key customers are demanding new integrations and features to keep them afloat.
You have a choice:
- Take a few of your experienced and productive developers off the new system to dive into the old.
- Hire new developers to work on the old system.
Neither of these choices sounds attractive. Each comes with a cost. You can’t give up the velocity on the new system. You know your current team hates the old system. Forcing them to work on it risks morale and possibly, departure of key engineers.
New staff don’t want to work on the “old system” either. If you recruit some developers talented enough to get up to speed quickly, they will likely know the score pretty quickly and want in on the new project as well. Who wants to work on a dead end project while everyone across the room is doing new, fun stuff?
Sometimes outsourcing maintenance and enhancement can end-run these political staffing problems for a legacy system. The vendor’s staff will be less concerned with their tasking—it is all just “client work” that needs to get done. Certain outsourcing models can work very well for this and similar scenarios.
The Stalled Project
You have a nice, well defined project but no one in the organization has the interest, time or expertise to execute. Consider outsourcing.
The Secret Project
“We need to keep this low profile.”
You need to test out a business or product idea outside of an existing product team. You do not need the production-grade, bullet-proof stuff your A-team is used to creating. You don’t even know how the potential users will react. You’re trying to be Lean but your team has not fully embraced this philosophy. Conducting a high-profile experiment risks public failure. Outsourcing promises to get it done quickly, inexpensively, and potentially out of view of all the usual nosy suspects.
Considerations
All of these scenarios involve one or more of the following key considerations: Velocity, Quality, Sustainability, Special Expertise, and Politics.
Velocity
‟What one programmer can do in one month, two programmers can do in two months.”
– Frederick P. Brooks
You know you are not moving “fast enough” but what exactly is fast enough? We refer to the speed of development as Velocity. In Agile projects, we measure Velocity in terms of points assigned to units of user-visible functionality called Story Points. Other systems for assigning points exist such as Function Point Analysis. Whatever scoring system you use, points estimate or measure the amount of complexity and the size of code changes required to deliver a unit of functionality. If you are only measuring hours or time spent, you are not measuring Velocity.
Over time, we observe a rough correspondence between points and time spent within a given team and project. However all projects change in Velocity during different phases of maturity. Time spent per point can fluctuate with team capability, morale, external factors and the growing complexity of the system. Projects early in their life cycle may experience high velocity. Later in the life cycle, process maturity tends to slow progress. As the size of a project grows, the effort required to make changes can grow as well. I use a rule of thumb:
For every four person-years of development work, you will spend one similarly capable person-year per year just to maintain your system.
If your team size is constant, this means both apparent and actual progress will inevitably slow over time. This illustrative rule-of-thumb is not meant to be exact or universal for every project.
According to Fred Brooks, adding people doesn’t make things go faster. This is often true. It is also a little hyperbolic. The effect of adding people to a project or team really depends on how much communication must take place between the different team members. This is an extremely important consideration when contemplating outsourcing.
In a well structured system, we can open independent “fronts of development” where the interdependencies are minimal. Each front imposes a cognitive and time load on staff in terms of planning and accepting the work product. How many fronts can you handle? Is your system structured in such a way that these fronts are obvious?
To obtain a Velocity boost from outsourcing, if you do not plan to outsource the entire project, you must have a clear idea where these fronts can be opened and which are most amenable to outsourcing.
Measuring Velocity
We calculate Velocity by dividing our calendar into cycles or sprints and totaling all the points delivered and accepted in the previous cycle. Frequently, we average this figure over several previous cycles to smooth out variation.
Agile teams often use project tracking systems which do this automatically.
Velocity vs Value
Velocity is deceptive in one respect. Point complexity and value are not the same. Clients and stakeholders care about the pace of value delivery. However, delivery of complexity is all we usually are capable of measuring. Systems for assigning business value to individual features are often not well developed. We rely on the judgement of the Product Manager or Business Analyst to determine value.
Complexity measures like story points serve as rough measures of time and cost. They do help managers plan work streams and eliminate or postpone items whose value is marginal compared to the effort involved.
Velocity Boosts from Outsourcing
When you have a front (or multiple fronts) of development that do not require deep domain expertise or lots of coordination with other teams, you likely have a candidate for boosting your velocity via outsourcing. Examples would include:
- Integrating with external partner APIs
- Upgrading framework versions
- Updating UX or “re-skinning” an existing application
- Adding new payment methods
- On-boarding new customers
All of these activities are possibly of vital business importance but they may represent a significant amount of labor that key internal staff cannot be pressed to do.
Another great opportunity for outsourcing is when you have a standalone app or application that is not part of your “core” system. Outsource it!
One important source of Velocity problems is the environment in which developers work.
Developers perform “deep work” that requires concentration. Production happens when they are “in the zone” — something I describe in detail in my book, Level Up: How to Become a Great Professional Software Developer.
The work environment and schedule should be designed to maximally support periods of unbroken concentration. Most developers need 3 hour segments of time to accomplish significant chunks of work. In a typical 8 hour work day, we may expect two such segments. If your developers are interrupted with mandatory meetings, or are expected to respond to email throughout the day, handle production issues, or experience other interruptions and distractions such as a noisy, open office environment, we can expect their production to suffer.
If you are experiencing low Velocity and any of these situations apply to your development team, it may be appropriate to address them first if at all possible before attempting to outsource any portion of your development.
Consider making these adjustments:
- Schedule all meetings at the beginning or end of the day or right after lunch.
- Keep meetings short unless you have good reason (eg. Spring Planning or Retrospectives)
- Move developers to a quiet area of the office away from noisy employees or frequent foot traffic.
- Give teams of 4-5 people working on related work a room with a door that closes.
- Seat people according to their project so that coordination does not require getting up to walk around.
- Provide meeting space nearby so that impromptu discussions can move to an area where they will not disturb the others who are working.
- Eliminate as many distractions in the workplace as possible. (Smart phones, social media, and noisy online chat rooms are huge culprits.)
- Provide an area for casual conversation and relaxation that is removed from the production area.
- Make other accommodations such as you can afford. (eg. quality chairs and tables, natural light, large screens that are low-eyestrain, etc.)
If you still experience Velocity problems after making these adjustments, or your organization will not support making these adjustments, consider outsourcing to boost your Velocity.
Quality
Quality problems are expensive. They arise for many reasons, including:
- Requirements Defects
- Process Deficiencies
- Insufficient Capability and Expertise
- Less Than Ideal Technical or Architectural Choices
- Poor Discipline and Morale
- Unsustainable Pace
In all cases quality problems require greater testing discipline, either on the part of the developers themselves, or on the part of a QA team, or both. If your organization does not possess sufficient QA talent or staff, augmenting your capability with an outsourced team of QA experts and specialists may be an option.
Measuring Quality
We measure quality problems in terms of the number and severity of bugs or defects.
Software product teams usually evolve some system for categorizing bugs: Critical, Urgent, Low Priority, etc. Reducing this sort of quality problem to a single score seems deceptively simple. We can easily multiply defects by their severity level and sum them. One problem is that we presume that all defects have been found and categorized accurately.
You also have a subjective aspect to software Quality which belongs more under the heading of what I call “Polish.” In addition to an absence of defects, Polish encompasses design and user experience considerations that contribute to the overall “feel” of using the software. It is more difficult to measure. I use a Polish scale of 0-5. With this scale, we can put a quantitative indicator on perceived and desired Quality. I strongly suspect that the effort required to achieve a given Polish level is not linear — that is producing a Polish level of 5 is not 5 times harder than a Polish level of 1 but probably more like 10 times harder.
Many Apple products have a subjective Polish level of 5. A 0 might be a very rough prototype with little thought given to appearance or “feel.” Most systems for internal audiences will require a 1 or 2 Polish level. Systems for customers will require a 2 or 3. Highly competitive markets may require a 3 or even a 4. Consumers may expect a 4 or 5 but they will tolerate a lower level if they are early adopters or if the product is sufficiently compelling in other ways.
Consumer tech experiences with products from companies like Apple have raised expectations. Business products that can approach these levels of Polish will stand out in the marketplace.
Defect Tracking
Bug or defect tracking is integrated into almost all software project management systems. Bug tracking is so critical that some organizations use glorified bug trackers to manage the entire software development work flow. We think this gets the cart before the horse but whatever gets the job done is often good enough. Sometimes a poor choice of tool for software project management can contribute to quality problems.
If you are working in a collaborative manner, it is important that your outsourcing partner use the same or similar tools to manage software development work flows. Sometimes an outsourcing partner will have greater expertise in this area and can expose your organization to superior techniques or tools.
Requirements Defects
Requirements defects are some of the most pernicious. A requirements defect in the form of an incorrect or incomplete specification moves through the pipeline and may not be surfaced by developers until acceptance and even after code has moved to a production build.
Developers who are not well trained, experienced or assertive may “fill in the blanks” by making assumptions that seem reasonable to them. When defects like this are repeatedly uncovered, most developers will complain about the quality of the requirements they have received. Be wary however, as some developers are timid and may not lay the blame where it belongs.
I preach that Quality is everyone’s job. Even though you have a QA person or team doing testing, developers must take responsibility even for poor requirements. They should demand complete information or clarity and avoid making unwarranted assumptions. You want to welcome and encourage constructive pushback and input from developers to get them more engaged in the overall process. Isolation and tight boundaries of function can hinder efforts to improve Quality.
A good QA team should be involved early in the development process. They will want to groom the backlog, participate in sprint planning, and improve requirements to remove defects as early as possible in the development cycle.
Process Deficiencies
Process concerns not related to requirements defects consist of things like code reviews, processes for utilizing staging, testing and review environments, approval and acceptance processes, code merge procedures, as well as Continuous Integration and deployment.
Advanced techniques may include feature flipping and limited rollouts of new capabilities. All such processes are supported by tooling. Process Deficiencies consist of the lack of such practices or the improper application of tooling and techniques.
Insufficient Capability and Expertise
Occasionally teams simply lack the expertise in producing high-quality software. They may not properly utilize various automated functional and unit testing techniques. They may be unaware of the quality expectations. They may lack discipline around code reviews and merging of code. They may not have a grasp of the business implications and severity of defects that are encountered.
Architecture and Technical Choices
Quality problems often arise from poor architecture or technical choices. Every project has what I call an “Experimentation Budget” — that is the amount of uncertainty or problems the project can tolerate from experimental or unproven technology.
If you are integrating Open Source modules into your project, chances are a few of them might be a little raw. For safety’s sake, on web projects, I generally allow only one major unproven piece of tech. Often the problem isn’t in the module itself but in your team’s poor understanding of how to use it and lack of awareness about the problems it may bring.
Sometimes an especially strong (usually young) but perhaps overconfident developer introduces a new technique or library to your project. Pretty soon he or she is rewriting all the code to take advantage of the new technique. The problem is, the rest of the team hasn’t had time to digest it and fully embrace it.
Your over-confident developer likely overestimates his or her ability to handle all the consequences of the choice. In other cases, staff engage in Resume (or CV) Driven Development where they feel compelled to stack all the latest goodies on a project in order to ensure their skills are the most marketable and up to date.
Architecture often follows team structure. I have seen systems carved into two “services” along apparently arbitrary lines because presumably one developer did not want to play in another developers’ code base. That would mean they would have to coordinate too much.
This bifurcation added 5,000 lines of unnecessary communications code which slowed things down, made debugging difficult, made deploying code trickier and was a continual source of problems.
Poor Discipline and Morale
When development teams lose discipline and morale takes a nosedive, quality problems are a sure to follow. Leadership is required to keep people engaged. Toxic individuals may need to be removed from the team. The team may need to be sheltered from organizational turbulence and politics. Find the weakest links and remove them. Add someone strong and fearless. Get a champion in place to protect the team and lead them to a better place. Or outsource.
Unsustainable Pace
When your motto is “move fast and break things” expect things to be broken. I suspect that Facebook’s early slogan is meant to counter the bureaucratic attitude that can metastasize and hinder progress. I would not take it literally.
An unsustainable pace does not allow for time to clean up, refactor, or re-evaluate past work. Running full steam ahead for months or years will leave large problems in wait and the bug tracker will be full of issues nobody wants to touch. If the only bugs that get fixed are ones that are marked “critical” — you might be operating unsustainably. Things can get so bad that developers would rather leave and work on a different system than slog through all the technical debt.
Some outsourcing partners can help by taking on these chores that never seem to get done with your core team. Outsourcing can be a way of maintaining forward progress while not digging yourself into a hole it will be impossible to escape.
Sustainability
Sustainability is critically important in software development. Very rarely do we build a successful system and then stop working on it completely. Most systems will require ongoing development at some level for the entire lifecycle of the system. Software systems slowly rot and deteriorate even when little feature development is being done.
For many web applications, continuous attention is required just to maintain position in a competitive market and a dynamic technical environment.
We must find a way to sustain a continuous level of development capability and attention. Cost is one aspect of Sustainability. If your current approach is unaffordable, it cannot be sustained. Some types of outsourcing can help with the affordability of keeping a team in place.
If you are having trouble keeping a team in place due to turnover, something about your approach may be unsustainable. Some stability is required in order to achieve sustainability. Continually recruiting, onboarding and training new people is a tax on sustainability. Outsourcing can help.
Special Expertise
You might have need for specialized expertise that are simply not available in your organization or for which you do not wish to hire a permanent staff member. Hiring a contractor or outsourcing to an organization that possesses the required expertise is the likely solution.
Organizational Politics
“We’re just not good at this sort of thing.”
This is a touchy subject but there are many reasons that companies hire vendors to do things they might otherwise be able to do themselves. Companies have traditionally hired consultants to tell them things they found it difficult to broadcast directly from management.
Sometimes companies hire vendors because it is quick way for a manager to gain budget and authority without staffing up. It is a way to get around a hiring freeze.
Perhaps your project is risky and likely to fail. It is better if the failure can blamed on a vendor. Perhaps your company culture looks down on IT people. Better to hire a vendor. Some companies are even pathological in their relationships with vendors—withholding payments, threatening legal action, etc., can seem like a strange sport to some people.
Believe me, I’ve seen it. It leaves you wondering, why exactly did they seek to outsource in the first place?
Outsourcing Locations
Outsourcing Doesn’t Mean Offshore
You may have come here thinking “outsourcing” means hiring a bunch of people with strange accents on the other side of the world for ten cents on the dollar. That is what most people think about when they hear the word.
Just today at lunch I received a cold call from someone with a thick South Asian accent attempting to sell me QA services. I provide QA services!
Stop, please.
The truth is, outsourcing simply consists of hiring a vendor to execute a project or perform work as an alternative to allocating the work to your own staff. It has the result of making someone else responsible for the human resources and possibly the performance side of things.
The hiring and firing of developers or related staff is not your problem. The vendor is offering a concrete work product or a level of resourcing and capability that is independent of the individuals involved in performing the work.
Outsourced team members could be in the same building, city or town, in a nearby state or province, on the other side of the continent, or across an ocean. They could be working from a Class A office tower in a central business district of a Tier 1 metro or from a spare room in a Tennessee farm house. The most intrepid may even employ digital nomads.
There is no telling where they will be working from next month! How exciting! (They’re probably in Chiang Mai, Thailand…unless it’s March or April when air quality is abysmal.)
You have to know what is right for you and your organization. Some organizations want a vendor to place people at desks right in their office. Some are only comfortable dealing with a firm they can go visit over a lunch hour. Your work may require US Citizenship or the work to be performed in the United States.
Maybe you simply require the most talented people available, wherever they may be. Some projects need a lot of “overlap” in working hours across timezones. Your project may work with just a couple hours of overlap. Completely asynchronous work hours for distributed teams require special methods and techniques which many have yet to master.
Set Up Your Own Shop?
If you are planning to leverage international labor arbitrage, then outsourcing to an offshore vendor must be compared against the costs and hassles of setting up your own operation in a target jurisdiction.
Do you really want to wade into dealing with foreign governments, landlords, building contractors, shifting foreign employment markets and a completely different set of employment laws and tax codes? This is not for the feint of heart.
Someone has to go do all that stuff and then stick around to manage and oversee the operation. Unless your company employs someone with the necessary capabilities and temperament who also has the bug to leave the country and go live in your target country, this is probably not an option.
Even if you do have someone, this person is likely going to demand a full expat package. Add another US$100k to their salary and benefits. More if they have family in tow. Top International Schools are as expensive as any elite private school back home. Think $20k/year per child. Those teachers are paid well!
Companies planning to hire hundreds or thousands of employees typically will want to set up their own operation with a manager or two from their home country. Companies in need of dozens of staff are on the threshold. If you need only a dozen or fewer, you are almost certainly better off working with a vendor who is already set up and running.
Sometimes a company just getting its feet wet in an overseas employment market will work with a vendor or several for a period of time, until they get a feel for just how big a footprint they want to keep overseas.
I have helped firms through this process.
Choosing a Target Location
When you decide to look outside your local market to source a team for a project or long term engagement, you have many considerations:
- Culture
- Language
- Time zone overlap
- Engineering practices
- Sustainability
Culture is important. I have had clients tell me they really appreciate little things like when they crack a joke from the Mike Judd movie “Office Space” during a meeting and our offshore team laughs because they’ve all seen it.
If you’re in a long term relationship with a vendor, these things make a difference. If your humor is lost, expect much else to be lost in communication as well.
Different countries have different levels of cultural alignment. We found the Philippines offers a high degree of cultural affinity with the United States. During the “American period” in the Philippines (1898-1944), Americans set up the education system, helped shape many government and civil society institutions, and introduced nearly universal English language education.
College educated Filipinos have routinely consumed American film, television and sports entertainment ever since and this provides a deep store of common understanding to draw upon in communication.
The United Kingdom had a similarly strong influence on many countries, most notably India. Language and culture are intertwined. Is English a second language where you are sourcing? Is it widely taught? Do staff discuss work in English amongst themselves or do they use another language? Is their primary language similarly expressive?
One advantage of English is its precision, broad vocabulary, and wide International adoption.
This is incredibly important in technical communication and particularly in software development where misunderstanding drives up costs and delays projects. In fact, for agile teams, this may be the single most important factor in choosing where to have your work performed. How well do the people actually performing the work understand you and vice versa?
In locations and countries where English is not universal or a primary language, a manager or lead developer particularly skilled in English will typically be the face of the team and will have to interpret specifications and be the conduit for questions from the developers.
This can lead to a game of “telephone” and generally has a deleterious effect on timelines and quality.
Culture entails other nuances which affect suitability and sourcing outcomes. Good software developers in our home countries have a reputation for being a bit disagreeable.
On the plus side, this means they will surface issues with designs or specifications or the work of teammates in such a way that the issues will get addressed early. Often this takes the form of constructive “pushback” where requests are a bit unreasonable or impractical.
Sometimes they offer up ideas for delivering 80% of the value with 20% of the effort—always a win—even though this might feel like not getting exactly what you want. This is valuable and can save time and money and possibly frustration down the line.
On the other hand, sometimes this goes too far and the appearance of hostility or not-being-a-team-player can sour what ought to be a productive relationship.
Developers from a highly agreeable culture, where politeness reigns and deference to authority is the norm, will not push back as much. This can feel pleasant but it can lead to delays and increased costs.
I have assembled a simplistic chart comparing target sourcing countries and some of their key distinguishing features. There is considerable variance within countries and some of these attributes will amount to “stereotypes” that are not representative of all developers or firms in those markets.
For example, there are agile development firms in India with excellent communication skills. However, you may have trouble finding one.
Work Hours Overlap
Daily windows for communication with your team are almost essential. Experienced, asynchronous, distributed teams can sometimes do without this practice but most teams will require at least a couple hours during which short “stand-ups”, or longer conversations can take place. This dictates which locations are suitable outsourcing destinations.
Having a long overlap is convenient but it is not necessary. Some offset can allow for concentration time without interruption. Frequent interruption is terrible for developer productivity. To achieve their highest sustainable productivity, developers should start each day with a clear idea of what they will be trying to accomplish and they should have all the necessary information required to complete their work at their disposal.
If the daily meeting takes place at the beginning of their work day, this can be sorted out easily and developers can get to work. If the daily stand-up takes place at the end of the developers’ day, the discussion will revolve around the plan for the subsequent day.
Long overlap allows for a certain sloppiness. Developers and teams will be tempted to start tasks that are less well defined because they know they will be able to hunt down any missing information at any time during the day. These information hunts can be extremely time consuming since they typically require synchronous communication with a person on the client side who may not be immediately available even during work hours.
Some outsourcing vendors work swing shifts or odd hours to accommodate the need for a greater overlap with their clients. While some developers and teams appreciate non-standard working hours, this can be stressful for more experienced staff with families.
Consequently, the teams will skew younger. We know one firm in Manila that works from 2 PM to 12PM in order to deal with the US East coast. Would you work those hours? Almost all their developers are in their early 20s.
Here is a table of suitable outsourcing destinations based on location and standard working hours:
Vendor Selection
Selecting an offshore vendor is a tricky business. There can be a big difference between working with a firm that is locally owned and managed and one that has an owner or manager from your home country or another developed, English-speaking country. If you are venturing abroad in your search, you may find an apparent cost advantage in firms that are locally-owned as their owners and management may have lower cost structures and different income expectations.
However, savings may evaporate quickly when misunderstandings or conflict arises. Trust is an important factor regardless of who you work with. Be honest with yourself about any attitudes and capabilities you bring to the table that will affect your ability to communicate and cooperate with the owner or top management of the vendors you short-list.
Whether you are selecting an offshore or domestic vendor, different vendors exhibit traits in keeping with their “personality.” In his book, Outsource It!, Nick Kryn describes five types of organizations:
- Process-Focused
- Customer-Focused
- Technology-Focused
- Ideals-Focused
- Hybrid
Different types of projects and clients will be best matched with different types of firms.
At AELOGICA, I run on a hybrid Technology and Process-focused firm, attempting to bring the strengths of both types to bear on solving our clients’ problems. Technology-focused firms are typically started and at least partly managed by one or more technologists like me.
I am first and foremost a software developer and entrepreneur. This gives me a superior ability to recognize, recruit and develop engineering talent, to identify leading technology platforms and build innovative solutions
However, I am also a systematizer. I copy systems and processes that work and innovate where necessary to create processes for reliable delivery of high-quality software.
I seek to impart my philosophy to my staff (as laid out in my book, LevelUp) and transmit this message through my marketing.
I am not a high-touch, customer-focused, always-on-the-phone, salesperson or account manager type. While I am creative and I understand values and branding, I am not primarily driven by idealistic concerns and I tend to hire pragmatic problem-solvers with solid professional skills.
I prefer to work on business strategy where it intersects with technology. I enjoy creating value through the assembly of people, processes and technology to deliver long-term solutions.
I have built AELOGICA as the vehicle for my work. Others will build firms in a different mold according to their different strengths.
Know what you’re looking for in a vendor to make sure you hook up with a firm that aligns with your needs. If your organization is constantly in flux and your requirements change day-to-day and week-to-week, you might want a Customer-Focused vendor that is highly responsive to your dynamic nature.
If your firm is a non-profit or a strongly mission-driven organization, or you need highly-creative output, you might want an Ideals-Focused vendor. For government or large enterprise projects, you might want more of a pure Process-Focused vendor who can conform to your rigorous process requirements and who shares your tempo.
Learn to recognize and identify different types of firms to help you narrow your focus in selection.
Engineering Practices
If you have a strong engineering culture, you will probably have strict requirements on not only the technology platform or stack you want to use but also testing and delivery practices.
Finding a firm whose practices mirrors your own will put you well on the road to a happy sourcing outcome. Assemble a list of the competencies and practices you regard as essential and make sure your prospective vendor can satisfy them.
Be upfront about coding standards or test coverage targets you may want to employ. Some projects will require 100% test coverage. This means the automated test suite your team will create along with the application code will exercise every line of application code in the system.
Your ratio of test code to application code may be 1:1 or greater. Not every firm is capable of delivering this. It can expensive and time consuming but it is important for mature businesses that value stability, predictability and uptime.
Complete test coverage is not appropriate to all projects. Prototypes and minimum viable products produced on a shoestring may have little or no requirement for such a complete test suite.
Every stack, whether you are using Microsoft, Enterprise Java, Ruby on Rails, Node.js, MEAN, Meteor, Phoenix, a PHP framework, or something else, will have a collection of associated best practices and safe technology choices. Make sure your vendor has experience with the best practices for your selected technology stack.
Engagement Models
American companies have been outsourcing software development for decades. Engagement models vary. Time & materials, seats, fixed price, cost plus, fixed margin, all describe various methods of billing for software work or development capacity. Newer models offer flexible “cloud teams”, co-located or remote teams, onshore, near shore, and yes, offshore.
What is all this? Over my decades as an independent consultant and owner of a software development firm, I sought ways to bring the incentives on the Vendor and Client side of each engagement into maximal alignment.
As much as possible, I like to be on the same side as my Clients. I do not like to experience conflict of interest. It leads to tense conversations and unpleasantness that I do not enjoy. This aim for maximal alignment has led me to innovate and to experiment with different billing models.
Traditional Outsourcing
Traditional engagements will fall generally into some variation of hourly-billing or fixed-bid with change management. I call these “Traditional Outsourcing” models. Both models have advantages for different scenarios but they also present challenges that can lead to suboptimal outcomes.
Hourly Billing
Hourly billing is probably the most common and familiar. Upfront estimation and project requirements definition are not as thorough or rigorous. It is relatively easy to just get started and rapidly scale up. Vendors like it because they know they earn a margin for every hour worked.
Clients like it because hourly rates make it easy to “compare” vendors. All you have to do is call around, get the hourly rates, assemble a spreadsheet and choose the firm with best rate that you think will deliver. Easy, peasy, right? You can hand this off to a b-school intern, right? Not!
Hourly-billing incentivizes the vendor to delay, over-staff or under-skill a project. Under-skilling can produce substandard work that requires too much correction. This model is especially likely to produce cost overruns. Even with the best intentions, work will likely expand to consume your entire budget and then some. Everything takes longer than expected.
Not all developer-hours are equal. Some developers can produce at a rate of 2x, 3x, or even 10x compared to others of a similar age and years of experience. However, highly capable but less industrious developers, can be prone to sandbagging or wasting time. You don’t want to pay for this behavior. Some dishonest vendors even will pad hours or look the other way when employees pad in an attempt to maximize billing. Not good.
Most successful small vendors will have a handful of high-performers, perhaps even one or two “10xers.”
If you beat them up on the rate, you are unlikely to get the best talent they have to offer. Most vendors will quote a “blended rate” reflecting a mix of skill levels since this gives them some necessary flexibility in staffing. Some will differentiate between roles or seniority and charge different rates for different individuals. It is very difficult to compare these rates across vendors.
Some vendors will assign a “Senior” title and billing category to someone with just 3 years of experience! While it is true that rare talents emerge with great capability in just 3 years (I’ve seen it!) this is very rare—a real 1% phenomena. In my experience, even strong developers generally take 5-10 years of full time work to mature into what I call a Senior Developer.
In 3 or 4 cases out of 5, I submit that hourly-billing will lead to a situation where you exhaust available funds for the project before you get what you want or you will terminate the contract once your frustration level exceeds the switching cost of finding another vendor.
After two or three of these experiences, you may give up on outsourcing entirely. I have dealt with a number of clients who have been burned in this way, some so badly that they acted like abuse victims in terms of the fear and mistrust they brought to negotiations.
Hourly billing is not impossible to make work. You can learn to manage it. You may need to vet each developer the vendor adds to the team. You will need to track progress closely, measure productivity and be quick to let them know when they are not performing to expectations.
This sounds a lot like managing employees which is one of the headaches we are trying to avoid with outsourcing…
So, maybe hourly billing is not such a great idea. What about fixed-price?
Fixed-Price/Fixed-Bid
Fixed-price or fixed-bid contracts sound like just the ticket to avoid this trap. You get clarity over the total expenditure up front. If the vendor screws up, it’s on them. Who cares how they staff the project? It’s deliver or die. You get to hold payments back until they meet critical milestones.
Usually you keep a nice juicy payment for the end to make sure they deliver. This final payment may comprise the vendor’s entire profit on the deal (if such exists) and then some. If you’re ruthless and have an adversarial mindset, you can even make the vendor do extra out-of-scope work at the end before collecting to eat up whatever margin they might scrape away with.
Vendors approach these engagements with great caution. Developers know this engagement model can lead to a dreaded “death march” with unpaid overtime and lots of stress. Detailed upfront analysis is necessary to completely capture the scope of the project. All aspects of delivery must be under the vendor’s control. The vendor must have excellent understanding of their production rates and ability, to yield sufficient accuracy in estimation.
Changes after the project scope is fixed must go through a defined change control process that is spelled out in the contract. This is Waterfall (the antithesis of Agile) development all the way.
The ways this type of engagement can go wrong are numerous. Most obviously, if anything goes too far off the rails, the vendor can end up losing money on the project. Faced with the prospect of losing substantial money or losing the client, most vendors will choose to walk away. You have increased delivery risk.
The best candidates for this model are projects that you have defined completely before you go out to bid. If you have already done the analysis and design work and have a sufficiently detailed set of requirements, complete designs and a detailed scope document, you can proceed to accept bids and compare them.
What is sufficient in terms of detail? If you have never done this before, you probably do not have a clear idea. As much as 20% of a Waterfallstyle project budget may be spent on requirements gathering, design and analysis up front. If you expect the vendor to help you in this phase, prepare to pay for this assistance.
Some less ethical vendors will look at an incomplete or rough scope and pull a figure out of their hat.
“That looks like a quarter million, give or take.”
“Great but everyone is going to say that. Bid 125k and they will come up with the rest eventually.”
If you think you’re getting a fixed quote with a rough estimate like that, you are in for some pain. Deep down you probably know they are underbidding to get the work. They know it.
You’re just baking in the uncomfortable conversation you will have 6 or 12 months down the line when it becomes clear the vendor will not deliver as promised for the original contracted amount.
An experienced vendor will seek to have complete control. Anything that is outside their control will be out-of-scope for a fixed-bid. An example would include: deploying the application in your datacenter or a private cloud managed by your IT team. From the vendor’s perspective, there is no telling how long this will take or how much coordination will be involved.
Uncooperative or bureaucratic IT staff, poor work-hours overlap, and obsolete, obscure or unreliable infrastructure—factors outside the vendor’s control—all could play a role in blowing out the time estimates. Yikes!
If you need the vendor to help you deploy the app, let that part be hourly or let them do it their way.
While hourly and fixed-bid engagement models are far from only variations in Traditional Outsourcing, most other models are just refinements to these two. The variations represent band-aids and patches applied to hourly or fixed-bid to address particular concerns.
None of them seemed particularly satisfactory to me as I built my company.
Team Augmentation
At AELOGICA, I introduced a different and very successful model called Team Augmentation.
Team Augmentation is capacity-oriented instead of project-oriented. In essence, we provide a team of a stable capacity for long-duration engagements, potentially indefinitely, for a fixed monthly fee.
This works for organizations that know they need to develop and maintain a longterm capacity to develop, support or maintain a system cost-effectively. Team members will rotate over time. Team size can be adjusted from time to time with a minimum of disruption.
The effect of this model is that it aligns the vendor with the client’s long term interest. We want to provide the highest sustainable capability to our client to keep them happy and continue the relationship. Long term recurring revenue for us has priority over short-term profits. We offer a reasonable cancellation period which protects both sides in the event something isn’t working out.
Team Augmentation is especially well suited to established and growing businesses with large code bases or multiple applications to maintain. Significant knowledge is built up and retained within the team and performance tends to increase over time. Careful but infrequent rotation allows us to retain experienced and knowledgable team members within our company even if they tire of the project and desire something new.
At times I have had clients tell me, “You have 8 people with experience on our project even though we only have a team of four. There is no way we could afford to retain that knowledge and experience here.”
In an emergency or when particularly challenging work comes up, it helps to be able to draw upon the experience of past team members. Rotation allows different people an opportunity to advance and enjoy leadership roles, training and experience.
With Team Augmentation, our team becomes a critical part of your organization. We meet daily for “standups” via video conference with your product manager and other stakeholders just as you would with an internal team.
I solicit feedback weekly on all our teams through a simple system that allows the client representative to tell me when the team is doing well and call my attention and that of our management immediately if something is going awry. I love sharing positive feedback with our teams.
And they love it too! I also love knowing that I will hear about concerns quickly so that I can do something to address them before they become a problem that jeopardizes the relationship. Clients like knowing they have a direct line to the top without having to schedule calls or meetings or send long emails detailing a history of complaints.
Our team sizes vary from as small as 2 to as many as 8. Teams can be comprised entirely of developers, or they can be a mix of development, design and QA as needed. Teams can have dedicated as well as adjunct members.
This gives flexibility to access skill sets that are not needed every day, such as analytics or machine learning, and provides for less disruptive rotation of team members when rotation is necessary.
Team Augmentation works best when stability, quality and sustainability are primary concerns. Team Augmentation is not appropriate if you are concerned you do not have “enough work” to justify a team.
Almost every established system has a long list of maintenance chores and technical debt to be paid. Our Augmented Teams can help you power through these backlogs when their primary work stream is light.
Team Augmentation is attractive to talented engineering staff who prioritize a stable work environment, good relationships and a steady income.
Value Points
Although Team Augmentation can be used to develop new systems using an Agile methodology, for new system development we prefer a slightly different model.
The Value Points model does a better job of aligning incentives for entrepreneurs and other clients that are developing new systems from the beginning using an Agile approach.
Value Points gives the agile system developer the ultimate in control over budget and the pace of development.
In Value Points, you only pay for the work product you accept and approve. You decide during each cycle what will get done and you know what you will pay.
Because it focuses on output, Value Points eliminates most concerns over skill level or different hourly rates for different workers. All projects benefit from a range of skill levels. You do not want to pay your Executive Chef to peel potatoes. (He doesn’t really want to peel potatoes either.)
But every project needs some potato-peelers.
Value Points appeals to the most highly productive developers who also want to be compensated according to their production instead of on a fixed income. If they produce more, they earn more. It also appeals to less skilled developers who can contribute and are willing to be compensated on the value they produce.
Value Points focuses the team like a laser on delivering business value.
When to Choose Traditional Outsourcing
Traditional Outsourcing models work well for some clients and projects. When you have a complete and very detailed spec, solicit fixed-bids. When your projects are on the small side and you have a lot of confidence in the vendor, hourly can work just fine. It gives you flexibility to adjust the team and project scope easily.
Best cases for traditional outsourcing:
- Short duration projects
- Large projects with complete specification
Examples:
- A mobile app that talks to an API you have exposed on your system where the design is already complete down to the last detail.
- A large enterprise data entry application and workflow to replace a legacy app where all features have been thoroughly analyzed and captured in requirements documents.
- A mock up, test or prototype of a system for demonstration purposes.
When to Choose Team Augmentation
Team Augmentation is best when you are looking for a sustainable, long-term capability. If you know you will have an endless stream of work to maintain, support and grow your system,
Then Team Augmentation is the way to go.
Some good candidates for Team Augmentation:
- You have to integrate with many 3rd party systems, particularly as new customers are onboarded. This work can be done by an Augmented Team while your core team stay focused.
- You have no internal development capacity but require continual support, upgrade and improvement of your application.
- When a lead, founder or very capable developer has left and you need to increase your bus-count and decrease risk from turnover.
When to Choose Value Points
Paying for points is really geared toward seed-stage funded startups. You don’t want to commit to a substantial monthly expenditure forever. You want to pay for work product and not time spent. You want to move fast and need the sharpest developers to execute on your aggressive schedule.
Value Points is the way to go. You will negotiate a price for each feature point delivered/accepted. You will do some upfront estimation and rough guesswork to figure out how many feature points will likely be required and how many you can afford.
New ideas will arise during development and you will want to implement them. Feature points force you through a budgeting process so that you understand just exactly how additional features and ideas will impact your design, budget and timeline.
Conclusion
I hope this has helped to introduce you to different outsourcing motivations and engagement models. Hopefully I have helped you clarify your thinking around what you seek. If you need Team Augmentation or Value Points style development in Asia/Pac, North America or Europe, please contact me, Steven Smith, steven@aelogica.com, and feel free to try my Team Augmentation Calculator and other tools at tools.aelogica.com.