20 February 2010

Why does new product/feature development take so long?

This is a post for non-technical readers (particularly non-technical product and high level feature owners) to explain why technology is "so slow" to deliver your new products and features.

Comparative baseline.  You need to ask yourself, "Why do I think something is taking too long?" What's my baseline, what am I comparing to when I think "slow"?  More often than not, I find that people are just displacing their general frustration in not having something they want (much like a child does), and take it out on the deliverer (the parent).

That, or you're just the type of person that moans about most things generally, so please stop.

I find that some stakeholders either delight in or don't realize they're only selectively comparing one organization to another.  Why can Company X deliver a new feature in a week when it takes our tech organization 3 months?  They rarely seek to understand why, they just want to selectively pick comparison points to criticize the delivery team.

Of course what they may not seek to understand is that Company X:
  • May have more and better technologists (perhaps at a higher cost base)
  • Invested in a technology delivery system that is much more efficient to extend, scale, and maintain (i.e., relatively less technical debt accumulated)
  • Enables a significantly different approach to IT due to a different revenue and cost structure (e.g., can afford better kit, replaced more often, serviced by better more expensive support channels)
True switching costs.  Perhaps you're the person that was frustrated with the rate of in-house delivery, and sought out a big supplier to deliver what you're looking for.  Hey, cheaper right?  And you got to teach those in-house slackers a lesson.  You get bigger economies of scale, and a lot more people to deploy to build your solutions.  But then it turns out your knowledge domain is new to the supplier.  Congratulations!  You just paid for the privilege of bringing a bunch of new and external people up to speed that didn't know your domain from a hole in the ground.  And then the supplier takes that knowledge out to other customers, finally creating that economy of scale you were sold on in the first place (Supplier: "Hey, thanks very much for giving me a new area to expand my business and dilute your priorities and control!").

A really good technologist or really thorough domain knowledge, especially both together, is rarely a commodity.  A "Java programmer" is a commodity.  A "really good Java programmer who understands our customer's requirements and has several year's experience developing our products within our business culture" is not.

You're a "get what you want", "bulldog", "win-lose" negotiator.  You might be the type of stakeholder that always demands things be done more quickly than what's been presented to you.  You might think that technical trade-off discussions are just technobabble to justify a "heavily padded", "low risk", "plenty of surfing time" schedule.  Perhaps you think your a tough negotiator and your stripping out the "fluff" the technologists inserted into the plan to save the shareholder's money.  Either way, that means you think you understand better how long things should take as compared to one or more people that just spent days or weeks thinking about the problem.

Unfortunately, you really need to understand and accept four laws of software development physics:
  • Good technologists rarely over-estimate.  Most either want to please you by giving you an aggressive schedule, they think their team is better than they actually are (they take their own personal estimations and extrapolate it for their entire team, one aspect of the Dunning-Kruger Effect), and/or don't think about whole solution delivery timings.
  • A qualified and professional team of people who spend time thinking about a solution will most likely know more about how long it will take to construct the solution than you do
  • Technical debt can be ratcheted up and quality down to deliver speed
  • Assuming the technology team is reasonably competent, dedicated, and professional, the only way to reduce the schedule is to alter some other project dimension
Here's the thing, technology teams can sometimes seemingly magically strip time out of a schedule.  However, if you take the time to understand the trade-offs, it's not magic.  Technologists can generally remove quality, stability, best practices and "this is how it should be done" dimensions from a project and deliver more quickly.  However, by doing so they're increasing technical debt and/or business risk.

As the stakeholder, you'd be wise to track technical debt just like you track project budgets.  Unless of course you plan to hand the technical debt over to the next management team and move on to another project.  Say, aren't you clever getting that big bonus for an on-time delivery.  Too bad the new administration didn't know about all that debt they're acquiring...

Servicing the debt.  If you don't manage the product and technical debt, if you don't seek to understand the tradeoffs that are being made when you pressurize delivery schedules, things will start to move even slower than they did before.  It will appear to you that your team is getting "worse" over time, their deliveries slower.

This won't initially make sense to you if you don't understand the trade-offs that have been made.  The team's technical, domain, and cultural knowledge should be getting better and better so why is everything taking longer now to deliver?  When the technologists try to explain why, it's very complicated.  In fact, it sounds like the same old technobabble they were trying to use to con you into a heavily padded delivery schedule.  Probably best to continue to ignore it like you've always done.  Your approach has served you and your delivery schedule well so far.  The team is probably just getting burned out and it might be time to switch suppliers!  And a new job has opened up elsewhere you'd be perfect for, let someone else struggle with this declining, unmotivated delivery team...

Excuses, Excuses

Let's face it, some individuals and teams are really really bad at what they're supposed to be doing.  Maybe, just maybe, you have been an attentive, detail oriented, engaged stakeholder, you do understand the tradeoffs that have been made, and the team you work with simply isn't delivering.  One or several of the following may have happened.

First, maybe the team really is bad (as compared to other similar teams).  How can that be?
  • Unaligned expectations - you really can't teach a pig to sing; you've hired someone that while good in their own right, can't possibly be good at what you've hired them to do
  • Burnout - people really have burned out (guess what, managing burnout is just another technical debt to manage)
  • Bad hiring - some people really are just incompetent and/or lazy; these types also tend to be good at lying as well
  • People change - they've just tuned out, perhaps due to personal issues; maybe their worked well in your business' previous culture and context a few years ago but don't in the current one
  • Bad alignment - maybe someone else in the business is poaching their time
  • Different priorities - they're delivering just enough not to get fired while they work on their own business or day trade
  • Poor management and leadership - people don't know what to do and/or aren't enabled to do it; their manager simply isn't managing, the organization isn't enabling them
  • Sewing seeds of discontent - a few people are really just negative, nasty, and unpleasant; they spread FUD to create discontent in the team and then take pleasure in the results
Second, I have to admit it, a lot of good technologists put architecture, future-proofing, tools, process efficiency, frameworks, scalability, extensibility, and maintainability against actually delivering a single feature.  They want to build a double-super-awesome application to dominate all other applications, and it takes a lot of architectural work to do that.  The key here is a technology leader that pushes for incremental improvement and delivery along all dimensions (product and feature delivery first and foremost) and makes technical debt levels transparent to stakeholders.

Third, you really have been screwed by a delivery team or supplier.  They're farming their alleged 100% allocated team out to three different customers like you.  Your SLAs have no teeth.  You've been sold using Ruby on Rails for development and even now a team of 30 people in several other countries are reading Ruby for Dummies and wondering why they've been hired to evaluate precious gems and lay railroad tracks but not write software.  50% of the time and budget are gone and you're too heavily invested to change.

Perhaps you really are in one or more of these situations.  If you are, you're probably justified in making drastic changes.  However, you have to ask yourself how things got that way, and what role you played in it.  Because if you're the one that created the bad situation in the first place, are you really qualified to fix or replace it?  Do yourself and your shareholders a favor and get some help.

What's the right way?

So what is the formula to speed up delivery?  Assuming that you prioritize speed over cost (see The Trinity Extended for more on this):
  • Acquire a good technology leader.   Find someone with credentials and references you trust, and then let them get on with it.  If you made a bad choice, then really, that's your or maybe your Boss' fault, isn't it?
  • Acquire good technologists.  Unfortunately, they're rarely cheap, because they know what they're worth.  Also unfortunately there are a bunch out there that will take advantage of you so you can also easily end up not getting what you paid for.
  • Domain knowledge, and even better, interest.  Acquire technologists that understand what you want built.  They should "get it" when you give a high level overview on what you want.  Ideally, you'll find people that have an actual personal interest in what you want done.  They want to build software that they would use themselves.
  • Delivered something similar.  Acquire technologists that have a proven track record delivering something similar to what you want to do.  It will certainly help with scheduling and anticipating risks.
  • Cultural alignment.  Acquire people that live and breathe within your target market.  It's much more likely they'll have an implicit understanding of your customers and what's required right at the start.  Also hire people that have worked in companies like yours:
    • Start-ups vs big companies
    • New development versus support and extend existing, inherited, purchased products
    • New development versus integration and middleware
    • Internal versus external customer facing
    • b2b versus b2c
    • Dedicated resourcing (generally apolitical) versus programme "beg-borrow-steal" persuasion/shared model of resourcing (generally political)
    • Static (not much change, driving down costs) versus dynamic (fast moving market, environment; lots of change)
    • Reactive (operationally oriented; opportunity led) versus Proactive (project oriented; strategy led)
  • Knowingly manage technical debt.  Sure, you can accelerate now and pay later.  Commercial realities may dictate this behavior.  Just make sure you knowingly stay aware of how much technical debt you're creating as you go.  Set budget, strategy, constraints, horizon expectations clearly and have your tech team explain where debt is being accumulated and likely impacts that will result.
There are many other and similar views on how to deliver fast and well.  The whole family of Agile, XP, Scrum, DSDM, FDD, Kanban and other similar methodologies all take views on this.  They are all interesting and worthwhile to understand and use as appropriate.  But to me most of them don't really emphasize enough the human component, the true difference a really excellent technology individual and/or team can make.  Instead if closely followed they tend to treat people equally, reward mediocrity, and put process ahead of people.  All technologists are definitely not created equal, and yes, they're not machines they're people.

If you are unable to effectively make a judgment about the situation (good for you, at least you recognize this in yourself), bring in an external IT consultant you trust or with a good reputation to perform an IT audit.  Have the consultant take a view on what is and isn't working.  And whatever you do, don't bring in a consultant who is a stealth sales implant for a large professional services arm at the same company.  Make it very clear that you would use someone else to do any follow-up remediation.


It is possible that you have a poor, slow speed delivery team.  Perhaps you do need to significantly alter how you deliver with new people, new suppliers.

However, before you do anything radical, do your business and shareholders a favor.  Sit back and think for a minute where the "slow" designation comes from and how objective it really is.  If the only common denominator between the in-house and out-sourced speed assessment is you, perhaps it's your judgment of "slow" may be flawed and/or that you really aren't managing your technical debt.  And if you're really not sure, get a project/IT audit done by a trusted resource to give you an outside view.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.