Not "yet another SOA blog post"

I had a job opportunity recently to work as an SOA architect, on SOA projects, in the SOA team of a consulting company. As many companies still make it their marketing, at OCTO we get annoyed with the negative effect of SOA term usage in lots of situations. Such situations include confusing marketing purposes, global IT strategy of a company, or SOA compliant architectures of many kinds. These approximations tend to hide weaknesses behind this term. We rather prefer to discuss on concrete solutions with respect to customer needs, and SOA is all but concrete with its abstractions.

Most discussions about SOA are sterile, because nobody is giving the same meaning to this acronym. Talking about it generates more confusion and misunderstanding than bringing real life problems to light. The goal of this article is not to propose one more SOA definition, but to analyze recurrent perceptions of what SOA could be. These perceptions are indifferently observed at various actors, as IT end users, software vendors and consulting companies. Do not feel offended if you recognize yourself in one of the given cases. We all forge our perception of what SOA really means, as a mix of all these points of view, and more.

SOA as a prerequisite

Gartner: "To go to cloud, [...] you cannot bypass services. You have to build on SOA strategy."

We often hear that SOA is a prerequisite for other architectural paradigms such as Complex Event Processing, mobility or Cloud Computing. Nevertheless many business units buy a SaaS solution themselves: this represents the major part of SaaS purchases. These business units reasonably can't have an SOA strategy as they bypass the IT department.

Moreover it is really easy to expose a service (i.e. a unit action based on a single record: a command, an employee, etc.) from a cloud solution. On the contrary, it is not as easy to consume services from it:

• As any software packages, SaaS solutions are rather passive against other parts of the Information System. To consume an external service, we would have to plug on a hook (or make a trigger to fire) at the right place (a transaction, an update, etc.), which is not trivial with software packages. Even if you can plug in at the right place, not all software packages offer a web services toolbox to handle serialization, headers, security, etc.
• PaaS and IaaS solutions permit to consume remote services without restrictions. But the problem is somewhere else. It is technically hard to expose on premise services to the cloud, because of difficulties associated with IPV4 NAT, firewalls and other security issues. Paradoxically we want to open systems that have been willingly built as fortresses. The key to the Cloud is not an SOA strategy; it is Tunneling (i.e. network tunnels that remain open between on premise and cloud systems). Tunneling techniques differ between cloud providers, but the idea is the same.

Of course having cross-applications exchanges based on a unit logic is a good enabler for cloud integration. But in most cases, services providers are hosted on the cloud, and services consumers are on-premise applications. As a result on premise service-orientation is not a prerequisite to take advantage of cloud computing.

The everything-is-a-service syndrome

Another perception of SOA is to consider everything as a service in any architecture layer: business services, logical services, IS services, technical services, web services, infrastructure services. Improper use of the word service is all the most disturbing, that it is already used to define many aspects of IT: IT services, IT transformation services, outsourcing services, consulting services, etc. Many software vendors and consulting companies define their SOA reference architecture similar to the following:

Some software vendors make everything a service, and give the illusion to reduce the complexity with more abstraction. With their so-called SOA infrastructures, they largely participate in this confusion :

• Concepts that have been existing for a long time are simply re-branded, including ESBs internal components
• The extension of ESB - outside of its role of bus - with services containers capabilities is very confusing. After years these containers are marginally used. The business logic is still in application servers. On the contrary, instead of having ESB embedding applications, the trend is for applications to embed their ESB...
• Inside of the ESB, what was an integration flow is now often referred to as service that contains integration logic. The attempt of defining new SOA patterns is no more than a revamping of well known integration patterns, with ... more complexity.

SOA as a set of technologies only standardized the transportation layer (SOAP, HTTP, security on it, etc.). Although this is crucial, transportation is just one of the features provided by an integration middleware (routing, transformation, orchestration, etc.). Even in a world of well defined services, we would still need a middleware in many cases.

The one-size-fits-all architecture solution

Some says SOA is an architectural paradigm. Some even says SOA is a philosophy. SOA is often perceived as a ultimate architecture methodology that embraces all aspects of the enterprise. What was IT governance is now SOA governance, but the purpose is the same: manage lifecycles to align IT and business sustainably.

Many organizations, especially consulting companies, have renamed thousands of architects as SOA architects. Does it mean such people are not able to design a batch anymore? Probably not. Architects have always been architects regardless of IT trends. The same goes with the organization: a transverse SOA team is unsuitable, as everything is not simply a well-grained service. Counter-examples are : a batch, a software package, a mainframe, an appliance, etc.

Most architectures defined as SOA are based on web services, that are synchronous by definition. But the more dependencies on external web services your application has, the less available, robust and testable it is. Remember to use asynchronous messaging as possible.

SOA can't be a finality: we have to compose Information Systems with heterogeneous technologies and various paradigms. Use the right solution as the right time to get the best of each architecture.

The mythical reuse rate

A great promise of SOA definitions is to reduce IT costs by making more reuse of business services. In real life, there is a big difference in reuse regarding the nature of the service:

• Services that are largely reused by nature. This is often the case for referential services as products, customers, providers, etc. We also find some highly reusable business services as order management, stock management, accounting transaction, etc. I have seen such services reused more than ten times.
• Services that are so specific that chances of reuse are very low: core-business services (e.g. digital right management), B2B and partners interactions (e.g. market data, carrier), domain-specific services (e.g. recruitment), public services (e.g. specific law), etc.

The myth around services reuse lead to various problems, in which:

• Making any service reusable prior to its effective reuse is over-engineering. In order to make reusable services, we have to create usable services first. Who has never faced such unusable services? tens of input parameters, complex data structures, performance bottleneck, insufficient availability, etc. are common characteristics of an unusable service)
• Inappropriate reuse is worst than no reuse at all: it creates a bad-coupling. A well-known anti-pattern is to reuse many services from within a batch...
• Trying to extend a service beyond its role results in a bad granularity

Do not reuse services if it does not appear natural in the first place. If you spend to much time finding out how to reuse a service, it will probably end badly. An alternative to reuse services at all cost is to reuse patterns: a same way to do, but different implementations and different deployments.

The top-down holism

A common vision of SOA starts with higher level business processes. These processes rely on business services, themselves implemented through IT components. And finally IT components are hosted on technical infrastructures. All these concepts are considered commutable boxes. In this holistic approach, run-time operational constraints are not given enough weight: constraints that make however foundations of our Information Systems (complexity, productivity, maintainability, performance, etc.)

Many speeches pretend that as a global top-down approach, SOA tied with BPM promises to reach business agility. This is not so simple:

• On one hand, we have predictable processes with a relatively slow variation (e.g. HR and supply chain, when not core-business). IT organizational agility is sufficient enough here: changes can fit in normal application lifecycles, without any particular abstraction or decoupling mechanism.
• On the other hand, we have rather unstructured processes. Neither SOA nor BPM brings an out-of-the-box solution for that kind of processes that relies on knowledge workers (e.g. CRM, risk management, personal services, healthcare, etc.). These processes can be tooled up with new-generation Case Management, collaboration workflows, or can be executed manually. The most important is that the end-user keeps control on what happens.

I don't think that Enterprise wide - long running - business processes are fast changing. On the contrary, the more operational a process is, the faster he can change. If SOA is fostered to change a process more easily in - let's say - five years, this is over-engineering. The entire paradox is here: bringing more complexity to reduce complexity. Oracle says to start the SOA revolution with a corporate-wide center of excellence, when IBM says to start with infrastructure foundations. Both approaches lead to even more IT centralization, and are certainly not bringing agility.

SOA initiatives often hide a lack of agility, by using technological abstraction as an organizational workaround. Are you activating the right lever to gain agility ?

Service-Orientation as a software design paradigm

Forrester: "SOA refers to a software design approach where systems are composed of various and sometimes shared services. The point is better interoperability and the ability to save time and money through the reuse of code." Building application as a heap of services is a theoretical vision. Everything is not so isolated in applications. Totally separating parts of a same application as silos leads to maintenance difficulties, development overhead and performance issues.

Gartner: "By 2015, SOA will be used in more than 80% of applications". Actually it doesn't matter to have SOA in applications. We already have Object-Orientation as an effective software design paradigm. What could make the difference is having Service-Orientation between / around application, as an integration paradigm**.** Geeks apart, who really cares about SAP or Salesforce.com internal architectures? What matters is what services they expose, not what services they are built with. That's why systems are generally not built as a heap of services.

An interesting exception is to resort to SOA for denormalization and performance reasons (LinkedIn architecture for example). Each service is finally an application, with a certain amount of processing and its own storage system. LinkedIn chose to brand its particular architecture SOA, and that's another perception of SOA again. This architecture orientation brings a lot of complexity and difficulties regarding: storage, testing, errors management, consistence, transaction management, availability, etc.

As it addresses a major performance requirement, this choice of denormalization should not be made a priori. Such an architecture style is particularly relevant when an organization wants to foster independent feature teams.

The SOAP-only shortcut

For many people, SOA and SOAP/REST are equivalent. This shortcut especially concerns some developers, as SOAP is more tangible as SOA. They are no ambiguity around SOAP and this term permits concrete technical debates. However there is often confusions between various combinations of transportation protocols (HTTP, JMS, pure TCP...) and representation formats (SOAP, JSON, pure XML...). UDDI failed to standardize services registry around the single SOAP protocol. It illustrates that there will never be a single protocol that unifies systems. At the moment something is normalized, there are growing initiatives to challenge it.

We can name it a message, an event, a request-response, an API call or a service call : it remains a flow between systems. That's why components can be exposed and consumed through technologies such as RPC, messaging protocol (e.g. pure JMS, AMQP), proprietary protocol (e.g. BAPI/iDoc, SWIFT), streaming API, all with their pros and cons. The most important is that exchanges are not necessarily synchronous.

What are we talking about exactly?

I didn't say that these perceptions are wrong as it would assume there is an SOA reality. My own vision of SOA has been a mix of some of these perceptions. Until I realized SOA has too many different meanings according to one's pains and interests. As a result of all these different opinions, I give no particular meaning to the term SOA. Actually, I am more and more convinced that there is no way to align on a consensual definition of SOA.

The SOA subject seems a little outdated, but believe me, it's here and crawling everywhere. Standardization organizations and other IT consortiums are still working hard on writing down SOA reference documents (see recent examples of OASIS and Open Group). Many software vendors and consulting companies are also pushing it intensely.

When dealing with a new consulting position, I carefully avoid the direct debate about SOA. Each time, I have to make an effort to understand what SOA really means in context. Projects often involve people from different companies, and different cultures as well. However nothing should be implicit about SOA and architecture. My last recommendation is to drill-down into the debate as quickly as possible: use precise words, talk about precise notions as integration patterns, component granularity, errors management, deployment typologies, etc. Can't we turn the page once for all?