Agile is a double-edged sword!
On the one hand, following a period of intense change, budget spend and organisational upheaval… your department will have quicker cycle times to market, be more predictable operationally, produce higher quality software, have better working relationships with its customers and will be more adaptive to changes of business objectives and requirements.
On the other hand, all the department’s problems that you know currently exist will become very visible and transparent very quickly. All those skeletons in closets you’ve been dampening down, or battles you’ve been fighting to secure funding will rise to the top of agendas. If you’re looking for an easy ride or at least some calm seas, don’t start agile.
Agile methodologies expose the weaknesses in your value stream of delivering software, it exposes poor infrastructure, asset management, governance, processes, delivery, and most fundamentally, it will expose weak people…
Agile not only highlights these problems, it rubs your face in it by reporting all the dirty linen as impediments and waste, and directly attributes slow delivery to the specific problem. These could include lack of testing environments, or poor VM performance, or production support interruptions, or no automated tests, or poor architecture etc, etc, etc.
Not only that, but if you walk this road, your department will start to judge you and the success of the agile initiative by your personal ability to secure funding and remove organisational impediments i.e. your executive skills.
So before you start down this road I think the first question for a CIO should be “Should building software be a core competency of this company?”
If the answer is no, then maybe the answer lies elsewhere in out-sourcing your software development to a good agile house. However, if there is a need for a strong software development capability because technology is fundamental to your business strategy, and the board of directors are in agreement, then agile is the best approach. This is because Agile development, specifically Extreme Programming, is one of the the most disciplined approaches to software development in the world.
So, let’s assume you’re a pro-active CIO/IT Director, with a supportive board behind you, ready for the challenge… Where do you start?
Fundamentally, you’re initiating a change programme to completely overhaul the way your company builds software. Don’t fool yourself into thinking you’re doing anything less!
The change will affect everyone involved, your consumers, your internal customers, finance, marketing, HR, Operations and your own area.
Forget the Hype – Moving to Agile is a large organisational change programme that will overhaul your approach to software delivery.
Make no mistake, it’s about people, its about change and therefore we all know it won’t be easy!
The good thing is that each change is incremental, there’s no big bang, you just have to keep removing barriers, impediments and keep the budgets going.
It’s continuous improvement, not revolution.
You’re going to need help!
There’s plenty of agile consultancies out there at the moment flooding the market, but fundamentally I think you need to look for experience and named resources to cover 3 fundamental areas; the organisational change, the delivery process and the engineering practices:
- Experience of managing organisational and cultural change including the stakeholder communications, business case creation, and internal marketing. And be able to demonstrate the energy it requires. (regardless of if its agile or not).
- Delivery management expertise, commonly Scrum is the predominant process adopted by agile teams. So look for Scrum Practitioners or Coaches with at least 5 years experience and references to back it up.
- Engineering Practices – Extreme Programming expertise, automated tools usage, and deployment specialists… basically strong agile engineering experience.
With competence in these 3 areas, you won’t go too far off track but do not underestimate the need for the Engineering Practices as it is fundamental to the quality of product and reducing cycles times to market.
And as with all change programmes… this will take months not weeks.
“The propensity to exploit Agile delivery to achieve business success is directly proportional to the capacity of a company to change and continuously improve.”
I guess I’m pretty much at a loss every time I read one of these Agile raves. I’ve been developing software for 34 years and I continue to be amazed at how almost every article on software development not only misses the mark but actually leads people down fruitless paths.
The key to success in creating great software is GREAT DESIGN. The same is true in building houses, cars, whatever. The Japanese demolished the American car industry by creating cars that could go 200,000 miles without a major repair, while American car makers were still selling us vehicles with 5-digit odometers. This didn’t happen by accident and it certainly didn’t happen by being “Agile”. If you want to understand how a handful of companies from a tiny island country managed to obliterate America’s automotive titans on their own turf, I suggest you read some books about a guy name W. Edwards Deming. And let me tell you ahead of time, it’s all about DESIGN.
I have seen projects claiming to adhere to all sorts of methodologies (which for the most part can be reduced to scheduling techniques), and I can tell you that almost all of them are failures. The reason is simple: you can’t build a great car, a great house, or a great anything unless you start with a great design. And what great design requires is a great designer. Nobody is much interested in the methodology that was used to build the various Frank Lloyd Wright houses. Rather they are interested in Frank Lloyd Wright’s designs, which are ultimately what define his houses. And with good reason: the design defines the house. All the methodology in the world won’t produce a livable house (let alone a functional work of art) unless great design is at the core.
Are you patting yourself on the back because your Agile team “made” every one of its last several two-week “sprints”? What if most of your people’s time was spent circumventing problems that should never have existed in the first place? What if your competition, using a better design, was developing twice your capability during those same weeks because they weren’t suffocating under a cumbersome construction?
Agile has become a religion of sorts, against which no one may speak without risking his career. Unfortunately there seems to be very little truth at the bottom of it. By default your software team will build you a pile of crap using Agile, Waterfall, or any other methodology. And in fact I’d say they’ll do it more easily using Agile, which tends to emphasize almost everything EXCEPT design, under the belief that virtually any group of software writers will eventually somehow cogitate themselves toward a product that works well. Nobody in his right mind would design a house or a car that way, and with good reason: the results would be painfully and visibly catastrophic.
The only reason that software development managers can claim success using today’s largely design-less procedures is that software is invisible to both them and their customers. After all, you can’t open the hood of a web site and comprehend the rat’s nest that’s underneath, right?
Sorry, there are no magic bullets. Without great design, your project and your budget are both toast.
Hello Dave,
Thanks very much for your comments, however, I think you have missed the point the blog was making about the Japanese car industry, and the Machine that Changed the World. You are correct that they created vehicles that didn’t require major repairs, but the point is ‘how’ they did this. Jones & Womack spent years analyzing this and it was a combination of design, implementation and testing in an overall manufacturing value stream involving their suppliers, focusing on creating flow through eliminating waste, that enabled these companies to produce better products.
They could have come up with the best design in the world but without changing the way they worked , they couldn’t have built it because they didn’t have the resources (capital, time, assets) – they had to change how they worked to achieve their success, and this is the crux of the matter. Change.
Turning to software development, we all know that intrinsically, building a car and building software are fundamentally different things. Cars are manufactured from a prototype which has been designed and specified, tested and commissioned long before it hits the factory line and takes years of effort. So for the process of implementing or building a car the inputs are known, specified, defined. The process is known, and therefore with defined inputs and process the outputs are predictable.
Software development is invariably the modeling of the real world and the world is constantly changing. The requirement from 6 weeks ago has changed and new models / software / processes are needed right now to make money for the business.
Now… there is an argument that for certain types of software implementation a prototype is required. Personally I’d be in favour of it where cycle times to market allow for it, where lives are at stake, where requirements are static… However, anyone working in a consumer facing service industry probably does not have that luxury/constraint.
Software projects need to deliver within quarters, to meet market opportunities or fend off threats and years of production are not in the equation.
You’ve stated that the “Key to success in creating great software is GREAT DESIGN”, so it leads me to the question “What about implementing the design?”
“Are you able to implement your design yourself or do you rely on others?”
If you rely on others, is it 3 others, 30 others, 300 others, 3000 others?
And then is the design implemented as you specified? How do you know?
So what’s wrong with designing the solution a piece at a time, specifying an example and writing an automated test to ensure the design is implemented correctly?
By building in 2 week sprints we’ll only ever be 2 weeks off target, reducing our risk, and with full automated test suites we’ll have confidence to make changes because we can see the impact.
How long does your design take to implement? X business needs a new market offering within 6 weeks, designed, built, tested, and live, how long will your design take?
Does your customer know the end point? Can your customer expound their requirements?
Your customer knows the market opportunity, know they need a product to exploit the opportunity within a short time period, so should we wait until they can articulate the entire solution so we can complete our design or should we start with what they know, specify by example, automate these tests, incrementally review the product fortnightly and keep moving forward until we meet the needs of the business?
Many managers do not have the luxury of good designers, or budgets to attract the best in the market, or train the potential they have. Many managers have to follow corporate policy of off-shore development, out-sourced infrastructure, and executives that do not want great software, just something that works that makes money.
For these people, stating that great design is the key to success does not move them or their organization forward. However, articulating the need for change, introducing iterative processes, disciplined engineering, and creating closer business relationships helps an organization move forward and demonstrate value (or at least an improvement) to their customers.
I think you’re right that Agile has become hyped, misunderstood, and misused to the point of being amorphous or nebulous.To some people Agile has become a religion, but I was confused about your reference to my blog as an agile rave, I think it is quite the opposite. It states that agile is about changing the entire organization to develop software differently, it says it will cost time and money, and puts extreme programming at the heart of the argument, and it states that these problems you cite in your comments need to be fixed.
In my opinion the “Key to success in creating great software is GREAT PEOPLE”. I’d rather have some great people using no process, than some poor people using Scrum… because the great people will find a way to build something valuable for the business. Great people will create a great design, implement it well and will produce it at high quality.
Steve: Thanks for your reply. Unfortunately this subject is far too large to properly discuss on a blog, but I feel compelled to try.
First, I am not trying to make an improper analogy between writing software and manufacturing cars. My analogy is rather with the design process that takes place before the first car comes off the line. You can have all the JIT and great vendor relationships and scheduling methodologies in the world, but if the tranny you’re building is destined to shake itself apart at the 75,000-mark because of its poor design, none of those other things will help you. And that really is the crux of the matter here.
I work under managers and with programmers who don’t seem to understand design at all. The managers don’t even look at what their programmers are doing and so they are clearly operating under the implicit idea that any design must be as good as any other design. Most programmers with whom I work have absolutely no conception of the long-term implications of doing things one way vs. another way. The most popular mentality that I see is one of “let me just do it however I want and don’t bother me with any kind of overall guidance”.
The result is that almost every project on which I’ve worked over the past several years is basically a Rube Goldberg device, strung together with a bunch of logic that breaks the moment you touch it. Assumptions made in one part of the project by one programmer are violated by the code that was written in another part by another programmer. Making modifications to such projects is an eternal nightmare, with each change likely to produce unintended consequences leading to yet more dangerous changes and so on.
I agree with many of the things you’re saying about the limitations of real-world development, but this doesn’t change the long-term realities. We can take note of the fact that many consumers won’t spend the money to buy a quality home appliance and so resign ourselves to making and selling only cheap appliances, but this doesn’t change the fact that in the long run everyone will spend much more repairing the cheap appliance than buying the quality one, and will make his life miserable with all the unscheduled down-time and time lost making repairs.
Clients who specify 6-week deliveries are obviously going to get a piece of throw-away garbage that will probably be kept on for years at a fantastic maintenance expense that nobody will understand was unnecessary. So the client spent very little upfront and got a cheap lousy product out the door in 6 weeks. And now this client will spend maybe 50% of their development cost for the rest of the project’s lifetime just to control its inherent instability. What will be the cost of that? What will be the effect on the client’s competitiveness when it takes twice as long to modify the product (due to its inherent design flaws) as would otherwise be necessary, with nobody realizing it?
One example might speak more than 1000 generalities. My current project is a web site that deals with money. In order to allow me to centrally control how the site manipulates money, I created a centralized “money” object that is used to store money throughout the site. I did this over the objections of other staff, who felt we’d be better off just using the same primitive data types that all our other calculations use, so that “money” calculations would be indistinguishable from all other calculations. Recently our BAs gave us a new requirement: we had to change the rounding algorithm we were using in all our monetary computations. With my centralized “money” object, I changed one line of code in under 5 minutes – and our entire site reliably changed its money computations throughout. Had I implemented money calculations using the preferred method of my peers, this change would have taken me at least a day, it would have been very dangerous, and when I was done I would still be unsure that I had caught every instance of money-rounding. My management, no doubt, would have congratulated themselves heartily that we had “made” the “sprint” that contained this unnecessary day’s worth of work.
I did not cherry-pick one unusual example here. This is the typical result that you get with great design vs. not-great design. It seems to me that the “great design” approach much more closely meets the Agile ideal of responding well to continuous change.
In the absence of great or even good design (which seems to be the norm), then I suppose that a project with Agile is better off than a project without it. After all if programming is nothing more than herding a bunch of cattle who are inclined to wander off in random directions and who have no idea where they are going, then I guess you are better off keeping constant tabs on them and having frequent “standups” to ensure everyone is heading the same way. It has only occurred to me very recently that perhaps the skill levels and the expectations of most real-world projects are so unfathomably low that the imposition of Agile actually causes an improvement.