A recent article by Phil Windley, Teaming up for SOA discussed keeping SOA governance collaborative rather than chiseling the "Ten Commandments".  The article can be summarized thusly, SOA governance is about managing the desired behavior of an organization in order to ensure successful SOA adoption and efforts by applying the appropriate level of governance, using the correct mechanisms, at the appropriate time in the project lifecycle and allowing for collaborative feedback to evolve the governance process.

The topics that Phil covers in his article are the same sort of topics that I have discussed here on several occasions. SOA projects are not done in a vacuum nor are they remote islands in a far off place.  While must organizations begin their SOA journey with a pilot project, they soon move down the path towards initiatives that span multiple departments or business organizations.  As new players, tribes, become engaged in the SOA journey they bring their own cultures which is quite different from a pilot effort where there is only a single culture.  Now the organization is whole new game because everything changes.

SOA governance requires their be extreme levels of collaboration.  As I have stated numerous times, the technical aspects of SOA aren’t the most difficult, but rather hard part is managing the culture, politics, and economics, better known as governance.  As Phil states, some have deemed these as Layers 8 and 9 of the OSI seven-layer model: economics and politics.

SOA governance addresses collaboration patterns, acceptable standards, and establishing communication channels.  A good governance framework will also align the incentives within an organization with the goals of SOA and establishes SOA support structures.

Most organizations have some sense of IT governance, which is defined as "specifying the decision rights and accountability framework to encourage desirable behavior in the use of IT" by Peter Weill and Jeanne Ross, and augmenting that to align with the SOA aspirations of the organization should not be detrimental, mind-numbing, or difficult.

Some organizations like a step-by-step guide for performing tasks, a "cookbook".  IMHO, when is comes to SOA, in particular governance, this is to prescriptive and assumes that the level of governance and the same processes will be required for each SOA effort.  SOA is an evolutionary mindset, a culture change, and is different for each organization, even each individual component with an organization.  Thus not every SOA effort will need the same processes and governance.

A checklist of considerations seems more practical to me and would seem to add more value to organizations.  There have been several best practices and recommendations made by people from lessons learned when it comes to governance considerations.

Governance considerations:

• SOA Competency Center

• success criteria

• compliance policies

• performance policies

• development policies

• security policies

• process improvement mechanism

• incentive plans

• enabling technologies

SOA Competency Center:  I have discussed the establishment of an SOA Competency Center, or similar type organization, several times here.  As SOA adoption moves from a single pilot project into multiple projects and ultimately throughout an organization to necessity of an SOA CC type organization becomes more evident.

Success criteria:  It is important for all parties involved in the SOA effort to understand the vision and the criteria for determining success.  Success criteria are needed for each stage of adoption as well as the end-state.

Compliance policies:  Policies related to compliance ensure that organizations adhere to regulatory compliance (i.e. Sarbanes-Oxley), interoperability standards (i.e. WS-I Basic Profile), or other organizational policies.  These are typically design-time sorts of policies.  By that I mean that these policies are enforced while the service component is being developed and its interface being published to a registry repository.

Performance policies:  Policies related to performance  ensure that a service consumer is receiving an agreed upon level of service (i.e. SLA), message routing occurs properly (i.e. dynamic routing), or mediation occurs properly (i.e. content or protocol mediation).  When developing an SLA a common template seems to work quite well.  People have asked me, especially recently, if such a thing exists and I point them to the legal industry because SLAs are a legal binding agreement and you better to consult than attorney.  SLAs are not new to SOA so start by reviewing existing SLA examples that don’t involve service experiences and SOA specific attributes.

Development policies:  Development policies ensure that service components are design and developed with reuse, collaboration, and participation in mind from the outset.  BEA as a mechanism that allows an organization to prescribe certain service, or service types, for a project to use, a kind a prescriptive reuse. 

Service components need to be design and developed for collaboration with other service components, through open standard interfaces, and for as part of a larger canvas, not going to deploy single service or SOA.  An enterprise SOA is going to be made up of  smaller SOA deployments from many groups, departments, or business units, so service components need to participate in the larger execution context and not simply in a localized one.

Security policies: Security policies ensure that only authorized and authenticated parties, human or machine, are granted access and visibility to service artifacts and data.  Security policies need to address service artifact access and usage and authentication and authorization of service consumers.  There are several standards that assist with these concerns such as Security Assertion Markup Language (SAML), Public Key Infrastructure (PKI), eXtensible Access Control Markup Language (XACML), and Web Service Secure Exchange (WSS).

Process improvement:  Organizations need to establish a way to receive feedback on the governance approach, both successes and failures, in order to continuously evolve and hopefully improve the approach.  Organizations must be in extreme collaboration in order for SOA adoption to succeed.

Incentive plans:  SOA is a mindset that requires a culture change and in order to get an organization to change its culture there needs to be a reason to do so.  One can’t say, "Oh SOA is a good thing" and expect people to simply buy into it.  The culture has to be nourished, cultivated, and incetivized in order for it to change.  One incentive plan I heard of was based on a point system.  Each time someone performed a best practice or complied with established policy the received points, points were deducted for the opposite.  Furthermore, if someone went beyond best practices or compliance they received bonus points.  Each point was worth $1.00.  Points could be cashed in for various rewards (i.e. iPods, gift cards, etc.)

Enabling Technologies:  There are several technologies for enforcing SOA governance.  First and foremost is the SOA registry repository.  The SOA repository manages the lifecycle of a governance policy.  Some of the more popular PePs include products from IBM Datapower, Layer 7, Cisco, Radiant Mercury, and Forum Systems.

An ESB is also a type of intermediary that handles the messaging portion of an SOA.  The policies that it enforces are concerned primarily with routing and mediation.  While it can handle security policy enforcement it wasn’t necessarily designed for such and organizations may encounter significant performance issues.  It is best practice to off load things that an ESB may not have been intended to address to dedicated devices for such reasons.

Web Service Management tools provide runtime enforcement of policies such as SLAs, Quality of Service (QoS), and other performance policies. 

Policies may be authored in the repository, but is not required to be.  Intermediaries can also be policy authoring points but the policy should be managed in a centralized, logically speaking, location. Either way organizations need bi-directional communication between the registry repository and the intermediaries in order to ensure the collection and dissemination of metrics to gage the health of the SOA.  These metrics will allow organizations to improve the governance approach, evolve their SOA ecosystem to meet demand and usage, and improve service component development over time.

Conclusion

I believe that if organizations attempt to follow a "recipe" for implementing governance, especially at the tactical level, they are simply building a "recipe" for failure.  Granted a repeatable process is warranted but organizations shouldn’t consult Betty Crocker for their SOA activities.

Thus far in this series of entries on SOA Governance, I have defined SOA Governance and provided a deep dive into SOA governance in this final entry I will discuss on the implementation aspects of SOA governance.

Implementing SOA Governance

Full SOA Lifecycle Management is required for an enterprise to realize the promised benefits of SOA.  Without SOA Lifecycle Management the potential exists for business policies to be violated, which may result in manners of behavior that can prove to be detrimental to the enterprise, and technical policies to be, which may results in noncompliant inefficient services.

Steps to SOA Governance

1. Define enterprise goals, objectives, and values

2. Form a cross-organization SOA Governance Council to oversee SOA implementation.  Define roles and responsibilities

3. Decide funding model

4. Perform analysis of existing/potential SOA artifacts and capabilities

5. Define SOA principles, standards, and best practices to be followed

6. Define SOA governance processes

7. Define project management and development approaches

8. Define SOA program and service level metrics

9. Execute SOA governance plan throughout lifecycle

Governance processes

• Business control

• SOA maturity level improvement

• Architectural compliance

• SOA implementation compliance

• Conflict resolution and escalations

• Service policy management and compliance (SLAs)

• Evaluating and selecting SOA tools, products, and technology stacks

• Service re-use

• SOA change management

• Service Access and security

• Service promotion and provisioning

• SOA skill set training and information dissemination

Full SOA Lifecycle Management

SOA Lifecycle Management involves defining, implementing, and enforcing policies and process across the entire SOA lifecycle.  Below are some of the processes and requirements for SOA Lifecycle Management.

Design Time

• Process to identify, find, categorize, and reuse services

• Process for developing services

      Adherence to:

• Architecture principles

• Business policies / contracts / service level agreements including design time verification of service’s ability to meet QoS

• Design best practices

• • Top-down versus Bottom-up approach

  • Amount of Coupling

  • Autonomy

  • Statelessness

  • Discoverability

  • Location Transparency    

    Verification and Test

• Process to conduct design reviews

• Process to conduct compliance audits

• Testing

• • white box — from the service provider perspective

  • black box — from the service consumer perspective

Run Time

 Management of:

·         Facilitation of definition and tracking of policies and SLAs

·         Policy, SLA administration / provisioning as per requirement

·         Policy enforcement

·         Logging and monitoring

·         Generating notifications, alerts

·         Facilitation of root cause analysis

·         Automatic deployment and upgrades

·         Reports generation – Service Usage, Billing, Failure Analysis

Change Time

• Process for service change control

• Process for modification notifications

• Process for service versioning

• Process for service metadata control

I will be revisiting various aspects of SOA governance from time to time.  So please don’t think this is the FINAL word on SOA governance and come back often to hear the ramblings to the SOA Jedi on the subject.

In the first entry of this series, SOA Governance Part 1: A Definition, I provided an introduction to SOA governance. In this entry I intended to begin describing SOA governance in more detail by explaining the constructs that make-up SOA governance and discussing some of it impacts on existing enterprise processes and relationships.

SOA Governance Impacts

SOA must be driven by the enterprise and align with enterprise goals, objective, process, and values. As such, there will be impacts to existing IT organizational structures and processes. One example of this is the Enterprise Architecture (EA) model of an enterprise. The elements of Enterprise Architecture (disciplines, practices, roles, and solutions) must align and support SOA concepts and SOA governance is the insurance mechanism. IT Governance will also be impacted by SOA Governance. IT Governance oversees areas such as governance processes, budgeting, and project prioritization. It also handles business oversight and the management offices.

SOA Governance Requirements

SOA governance must address:

• determining the decisions that must be made

• determining who should make the decisions along with their rights, roles, and responsibilities

• determine the mechanism(s) to be employed in order to make, manage, monitor, and measure the decisions

By constructing and implementing SOA governance strategy that addresses these areas an enterprise can gain "full SOA Lifecycle Management". SOA Governance Justification SOA has many moving parts, building blocks that must be maintained by different organizations within an enterprise and in the extended enterprise, other enterprises that one does business with. Shared services must be managed for compliance with a Service-Level Agreement (SLA) in order to function efficiently.

SOA Governance Constructs

SOA governance has three main domains:

1. Strategic

2. Organizational

3. Enablement

Strategic SOA Governance The strategic domain of SOA governance is concerned with "What?" decisions must be made in order for an enterprise to effectively manage and utilize its IT resources. These decisions are generally concerned with:

• Decisions of Principle

• • IT statements of its utility of the enterprise

• Decisions of Architecture

• • Strategy statements on organizing the necessary IT resources in order to achieve business and technical standardization and integration

• Decisions of Investment

• • Prioritization statements for projects in order to justify IT investment

• Decisions of Ecosystem

• • Statements defining the structure, relationships, and control of shared IT resources.

• Decisions of Need

• • Statements justifying the business goals and needs for purchasing or developing applications

Organizational SOA Governance The organizational domain of SOA governance is concerned with "Who?" will make decisions and what is their role, responsibility, and relationship to the other enterprise participants. It is an SOA best practice to construct an SOA Center of Excellence (COE), an SOA Competency Center, or equivalent cross-functional organization that has executive support and power to make the required decisions. It is the role of the SOA Governance Council to establish the governance processes and policies, as well as define the roles and responsibilities, creating new roles where necessary. A subcomponent of organizational SOA governance is behavioral governance. Behavioral governance is concerned with the cultural and sociological areas of SOA and promoting desirable behavior and its effectiveness by establishing an incentive strategy for SOA and compliance with established SOA governance policies, principles, and processes. One avenue is to develop SOA Heroes, those people who not only comply with all SOA governance policies and processes but take to extra step to help promote those policies and processes throughout the enterprise. The incentive strategy should provide guidance on how to conform to the desired behavior in the form of such things as best practices and lessons learned. Not only does the incentive strategy award for desired behavior but it must address non-conformance behavior as well. Behavioral governance must also address how to measure the effectiveness of SOA adoption and the SOA governance policies and processes. One mechanism of measurement is through SLA, others include determining return on investment (ROI) through factors such as re-use and lifecycle agility. Enablement Governance The enablement domain of SOA governance is concerned with "How?" decisions will be made and managed. As mentioned earlier one mechanism often used to implement SOA governance decision making is through the establishment of an SOA Governance Council, which is comprised of representation from across the enterprise. The SOA Governance Council is typically organized into sub-councils with each having specific focus areas and decision making jurisdiction. The SOA Governance council at large as final says in all key decisions. Some of the sub-councils include:

• An architecture council

• A development council

• A business development council

Other mechanisms for implementing SOA governance include tracking the IT projects and resources reused through consumption and Service-Level Agreements (SLAs). Enablement governance must ensure that the appropriate governance policies and processes are defined and enforced in all governance domains.

What to govern?

Although we’ve describe the necessary constructs for SOA governance and some of the components involved, there is still one question left unanswered, "What SOA artifacts should be governed?" In order to answer this question I will focus on implementing SOA governance for the services lifecycle. The services lifecycle is comprised of three phases:

1. design-time

2. runtime

3. change-time

To demonstrate these phases I will direct the focus to the SOA Registry Repository. An SOA Registry Repository is an authoritative enterprise catalog, system of record (SOR), that manages the metadata of services including, service location, purpose, business rules, and usage plans to enable service lifecycle management. A registry repository is typically an integrated tool that is centrally managed. While an enterprise may have multiple policy authoring points within its SOA ecosystem, those policies should be managed by the SOA registry repository for lifecycle management purposes. Areas and Artifacts to govern include:

• service interfaces (WSDLs)

• service metadata (versions, description, etc.)

• service contracts (relationships/consumer provider pairings)

• taxonomies and XML schemas

• ontologies

• security

• quality of service (QoS through SLAs)

• standards and policies (WS-I compliance)

• protocols, and technologies (SOAP, WS-*, etc.)

• service development (change management, alters, and notifications)

There has been an enormous amount of information been bestowed on the world regarding the various aspects of Service Oriented Architecture (SOA), I want to discuss one particular aspect that being SOA governance. I will attempt to provide some clarification by discussing, in a multi-entry series, this subject called SOA Governance.

 If ten people were surveyed on the question "What is SOA governance?” there would most likely be ten unique responses.  Some responses seem to be of the camp that SOA governance is only the governing of SOA artifacts such as WSDLs, service descriptions, metadata, and contracts.  While others take a more holistic where they include the more traditional IT governance aspects as well.  In order for an organization to realize significant benefit from an SOA imitative there must be a working governance plan.

The SOA governance and lifecycle management consist of:

• Portfolio Management

• Architecture Management

• Development Management

• Operational Management

SOA governance must address:

• The decisions that are required for effective utilization of investment and assets

• The organizational constructs and relationships required for making decisions

• How decisions will be made, enforced, and monitored

The maturity of an organization’s Governance capabilities and the architecture has direct effect on the benefits that the organization realizes from the SOA initiative.  The more mature the SOA architecture and the stronger and more mature the Governance capabilities the higher the return-on-investment and assets (ROI/ROA) due to the realization of greater benefits from the initiative.

So What is Governance?

Governance is both an art and a science.  The "art" deals with sensitive subject of organizational structures, relationships, and responsibilities.  The old saying "keep your friends close and your enemies even closer" can be applied to organizational constructs.  People tend to keep their resources close to them and fear releasing them or the control of them to other parts of the organization.  In order for the organization to benefit from its SOA initiative(s) the right balance must be struck and governance assists in ensuring that.  While the "science" deals with the actual enforcement of the decisions that have been made.  Enforcement can come through the utilization of policies and procedures both manual and automated. Governance is more behavior that technology.

Governance: - the policies, processes, metrics, and structured relationships required for and organization to operate and meet its enterprise capabilities and needs.

How is SOA governance different?

There is no essential difference between SOA governance and traditional governance, already discussed.  There are unique sets of relationships, procedures, and policies required for SOA governance.  SOA has special policy needs especially in the areas of creation, communication, enforcement, and maintenance of automated policies throughout the SOA lifecycle (design-time, run time, and change time).

SOA Governance: - application of governance with the addition of special relationships, policies, and processes to address the unique SOA aspects and artifacts.

Governance must begin at an enterprise level and have decision points, organizational roles and responsibilities, and mechanisms of enforcement throughout the SOA lifecycle.