It has been interesting watching the SOA (service oriented architecture) vs Web 2.0 (read/write web) discussions this past week, John Hagel has perhaps the best summary. Whilst I love the overall objective of SOA, my interest has waned as the ‘experts’ have got in on the act. They seem intent on designing the perfect SOA world in theory first and that approach usually leads down a dark path to being expensive and complicated to adopt, with questionable return on investment. Web 2.0, on the other hand, feels much more real in that the applications are out there to work and play with. But the low cost and large number of similar applications makes it almost too easy to start adopting different niche solutions with little concern for their longevity or value, and that approach can stumble across a chasm between consumer/silos and corporate/integrated adoption.
Microsoft hosted an event called Spark in March to look at the connections between Web 2.0, Service Oriented Architecture (SOA) and Software as a Services (SaaS). David Hill has posted his notes. Mike Platt, one of the organisers for Spark, has documented some of the initial architectures that were brainstormed during the event and there is a blog tracking progress. It’s a good start to marrying up Web 2.0 and SOA. Jeff Schneider has created a very good image ‘Competing Interests‘ to visualise the clash between organisations and individuals, business and I.T, and the challenges they face.
The common problem shared by SOA and Web 2.0 is that neither approach enamours IT to business. The Internet is changing how people interact with organisations, speeding up the pace of communications, decisions and transactions. There is frustration at the slow pace of change inside organisations compared to outside, and IT is too often part of the problem instead of the solution. Regardless of whether the preference is for the theoretical and controlled SOA approach or the practical but chaotic Web 2.0 approach, there needs to be closer alignment between IT activities and business needs.
Jack Vinson woke me up to the Theory of Constraints (ToC) and I think SOA and Web 2.0 would make a lot of progress within organisations when accompanied by a ToC statement. The short version of ToC: “Technology is beneficial if, and only if, it diminishes a limitation.” 4 questions to ask:
- What is the power of the technology?
- What limitation(s) will it diminish
- What rules were flawed because of the limitation
- What new rules should be followed
For example, there’s not much point implementing wiki software to improve document collaboration if the traditional method for reviewing reports is a monthly meeting with handouts to review. You have to be prepared to change behaviour as much as the method and to do that requires everyone to be involved, business units as well as I.T.
Once you’ve pass the ToC justification, the solution still needs to be implemented. Too much theory and top-down design risks creating a complicated and/or sterile solution that nobody wants to use. Too much chaos and bottom-up adoption risks causing confusion and duplication, wasting time and resources.
In the software development world, a new method called Scrum is gaining support and I believe the method could be applied to most I.T. projects. It attempts to mesh theory with practice, and align I.T. solutions with evolving business needs. The idea behind Scrum is that when customers define requirements for a project, they are based on an ‘envisioned system’. But that envisioned system is not sufficient for predicting costs because the initial set of requirements sound great in theory but can fail in practice. I have often seen this happen when building prototypes for customers: once the requirements are translated into a working model the customer realises that they want something different, or that their requirements are going to be too expensive to ever achieve a return on investment. The end result is the same – the need to redefine the requirements. But projects are often signed off without any attempt to prototype and costs are based on that initial set of ideal requirements . The implementer (be it internal I.T. or external consultants) is then committed to delivering those requirements come what may, costs and project plans can spiral out of control and, for the biggest (i.e. government) projects, we hear all about it in the news.
The Scrum method assumes from the start that there will be changes to the requirements and uses an iterative process for implementing a new solution. It recognises that whilst the ‘envisioned system’ is needed as the start point for a project, the ‘essential system’ is what will be built. The definitions:
- Envisioned system: a system as initially foreseen by customers to deliver the needed business value
- Essential system: a system with the minimum set of functionality, architecture and design that delivers the envisioned system’s business value
The Scrum method rejects the traditional approach of predicting costs and resources at the start of the project and then holding the project team accountable for delivery of those predictions (and thus encouraging the project team to make it extremely difficult for the business to change requirements, no matter how compelling the reason to do so). Instead, Scrum is a collaborative process that encourages the business to change requirements as needed throughout the project by not requiring I.T. to be rigid in their predicted costs (time, resource, money) up front. By making the essential system the baseline for the project, the business is free to adjust resources required for the project – more funds can help deliver the project sooner, if there is clear business benefit to do so; delaying key milestones can reduce risk when theory doesn’t translate into practice as well as initially expected.
I think the combination of ToC and Scrum can create a framework for better aligning I.T. projects with business needs. Whether those systems are based on SOA, Web 2.0, SaaS or whatever then doesn’t really matter as long as the solution delivers the goods. If the experts defining the theories include alignment with the Theory of Constraints in their models and a Scrummy approach to adoption, it will be far easier to justify and prove their purpose, benefit and value…
Maybe we need to follow the scientists lead – we need a unified theory for doing I.T. 🙂
Related posts:
Additional References:
- Theory of Constraints (Wikipedia)
- Scrum software development (Wikipedia)
- List of SOA vs Web 2.0 posts (Mike Platt)
Joining Dots 
I've been thinking the same thing about Scrum – that it can be used as a technique for managing IT projects in general, not just software development efforts. ( Here is my post on the subject). But the advantage of scrum is also its drawback – being inherently short-range, it doesn't help a department determine its budget. I think the "unified theory" you propose would need to offer a long-range planing methodology that can be used in conjunction with the short-term planning of Scrum. –michael
Yes, we perhaps need 'son of Scrum' that acts more as a framework for aligning IT with business in general (especially to get organisations away from this 'fixed cost up front' approach that leads to some of the more spectacular project failures). I'm pottering away mashing up bits of Scrum with ToC, systems thinking and network theory to try and find a way to modernise I.T. adoption, and will post an update if anything useful comes out of the process. Thanks for the link – great post summarising the actual Scrum process.