Background / Introduction

The intention of this series of articles isn’t to delve into the lowest level of detail within any single Enterprise Architecture framework, but rather to provide a high-level overview (as I’ve done more than once before) demonstrating that nearly everything is an evolution of one or more things that came before it.    Most importantly, is to walk away with a clear understanding of the fundamental concepts and principles, being capable of Thinking on your feet in the “real” world (not memorization of data and terminology that can be easily referenced or changes from version to version.. just to get a trending certification and not remember what it means next week).

Experience comes in time, but you’ll find that understanding the following concepts, terminology, and “use-cases” will be applicable in many areas for years to come (good ideas are resurrected, transformed / extended, and re-applied.. even as paradigms shift / Methodologies/ Frameworks, and products come and go).   Enterprise Architecture (EA) aligns the Business process, requirements, and strategic objectives with the appropriate underlying Technical Architecture. 

EA has grown from several disciplines, borrowing and extending principals and process from several areas : Software Development (Waterfall, SDLC/RUP, to Agile), traditional Project Management and Business Planning principles (1970’s IBM BSP), as well as borrowing from Total Quality Management characteristics (optimally mapping capabilities and iteratively improving toward future objectives), and IT Service Management.   Additionally, most of this ~50 year evolution (as with many areas in technology) originated from Government funded research (DOD, NIST Enterprise Architecture Model, etc).

While today’s variations were further cultivated by individuals tirelessly working in the trenches to solve REAL-world business problems, we also have to thank a few key organizations (The Open Group ~ TOGAF) and institutions for harvesting and unifying this process to ensure longevity through adaptation and wide-spread adoption.

This article (Part 1) provides a chronological overview of WHAT came before (and later became leveraged, or adopted within) formal Enterprise Architecture :

• SW Development Methodologies (Waterfall, SDLC, Agile -> DevOps -> DevSecOps)
• Project & Quality Management Methodologies (PMP, Lean, TQM, SixSigma, RUP, Kanban, Scrum)
• IT Service Management (ITIL, COBIT..)
• Site Reliability Engineering (SRE’s)
• Modeling Languages to support Architecture Frameworks (UML, SysML, SOAML, UDPM, UAF)

Part 2 of this article, continues forward exploring the chronology of formal Enterprise Architecture Frameworks, and their evolution toward today’s agile needs  :

• Enterprise Architecture Frameworks (PRISM, NIST, TOGAF, OEAF,..)
• Challenges of implementing EA in an Agile / Cloud-First World
• Futures & Evolution of Enterprise Architecture

Part 3, focuses on the real-world challenges, & realities of implementing EA   :

• Challenges of implementing EA in an Agile / Cloud-First World
• What’s different between Enterprise class (business/ mission-critical) workloads vs. general purpose ?
• Managing Complexity through Standards & Governance
• Potential pitfalls of Agile/DevOps left unmanaged
• Cloud Native Architectural “ramifications”
• Futures & long-term Evolution of Enterprise Architecture

SW Development Methodologies

(Waterfall, SDLC, AdaptiveSWDev, RapidAppDev, Agile -> DevOps)

Waterflow Model :

WHAT was MISSING before the ~1990’s+ ?

(noting that Silo’d deployments and dedicated HW was best practice)

• No Wide-spread access to the Internet via HTML/Web Browser – Which wasn’t available until after the 1994 NCSA (National Center for SuperComputer Applications) release of Mosaic, 1994’s Netscape release of Mozilla, and 1995’s Microsoft release of Internet Explorer.
• No/little Re-Use of assets via large public repositories (Apache, Github, javascript/ npm, Container/ VM images, etc) to leverage elsewhere for Development.
• Minimal FOSS (Free / Open Source Software) w/o SW Licensing limitations/restrictions (GNU, .. pre-Apache).    Don’t forget.. that the term “Open Source” wasn’t even coined until 1998 !!
• Many enabling technologies were missing (such as Java’s cross-platform portability) along with a variety of tools to offer low-cost, Rapid/Agile capabilities (Polyglot dev w Python/ Javascript/ Node.js/ JSON.., Virtualization/ OS Containers, etc) restricted and constrained SW Development within “Silo’d” environments bounded within platforms.  (for more details see my article Cloud Architecture 101)
• Continuous Improvement wasn’t yet in vogue (see my article regarding Enterprise Architecture).
• Automation of Testing and promotion from Dev -> Test -> Prod .. didn’t exist beyond scripting and ~C/ “makefiles”.

1987 – Carnegie Mellon CMM (Capability Maturity Model) and CMMI (CMM Integration)

At the same time that computing and the Internet was in it’s very infancy, the CMM was created to characterize and measure an organization’s process “Maturity” Levels (later applied more holistically to configurations and more integrated Enterprise Architectures as CMMI).

As per Wikipedia :

The project consisted of members of industry, government and the Carnegie Mellon Software Engineering Institute (SEI). The main sponsors included the Office of the Secretary of Defense (OSD) and the National Defense Industrial Association.

CMMI is the successor of the capability maturity model (CMM) or Software CMM. The CMM was developed from 1987 until 1997. In 2002, version 1.1 was released, version 1.2 followed in August 2006, and version 1.3 in November 2010. Some major changes in CMMI V1.3 ^[4] are the support of agile software development,^[5] improvements to high maturity practices^[6] and alignment of the representation (staged and continuous).^[7]

According to the Software Engineering Institute (SEI, 2008), CMMI helps “integrate traditionally separate organizational functions, set process improvement goals and priorities, provide guidance for quality processes, and provide a point of reference for appraising current processes.”^[8]

In March 2016, the CMMI Institute was acquired by ISACA.

NOTE:  You will find MANY variations today of the following “Maturity Model” across many disciplines (originating here  first as the CMM) :

** Part 2 of this series will explore how the Maturity Model has evolved as a tool within multiple frameworks, and examine it’s implementation within TOGAF.

Fast Forward.. to today’s SW Development craze that has grown beyond SW, and into one of today’s most adopted business movements :

“Agility” & Continuous Improvement (aka SCRUM, DevOps, etc..)

2001 – The “Agile Manifesto”, aka the “Manifesto for Agile SW Development“

As described in the History of Agile, the Spring of 2000 initiated talks at a ski resort ( ), where several in the SW development community gathered and started to come to a shared emphasis to promote/progress ideas in “Light methodologies, such as Extreme Programming, Adaptive Software Development, Crystal, and SCRUM“.   While this initial meeting didn’t result in one formal consensus, it did result in several articles that started conversations.

.. Then on February 11, 2001 at the lodge in Snowbird (ski resort), seventeen members from this extended community did align their objectives to become the “Agile Alliance”, and ultimately together wrote the Agile Manifesto and the 12 principles of Agile SW Development.

Note that the 12 principles of the Agile Manifesto came before and is distinctly different from the 12 Factor Application methodology that several Heroku developers contributed to and published in 2011 by the Heroku Co-Founder Adam Wiggins.    The latter (12 Factor App) lends itself toward more prescriptive SW development as a service or “stateless” mi microservice guidelines for Application Architecture (stateless, don’t hard-code config info but store locally, isolate dependencies, scale horizontally, and build-deploy-run in an iterative model..).   The 12FactorApp principles come more into context when discussing the post-DevOps (post 2008) movement where “Cloud Native” development and concepts flourished (see the next section).

As you can see from the above example, the founders of Agile came from the SW development world, but adopted and incorporated MANY ideas and leveraged the best of several models (from several disciplines beyond SW development) to develop Agile methodology (Continuous Improvement, Small teams, customer directed discovery/req’s, end-to-end ownership of SW, iterative/regular readouts on progress, etc).    Ironically, several years prior (1994), I published my MBA dissertation on SW Development through Continuous Improvement and TQM that incorporated most of these concepts, though never published beyond my University (since as part of required course materials).

Taking a step back and being fair, Agile(more holistically than only within the SW development specific context) is neither a Methodology, nor a Framework, but more so a collection of principles and a philosophy of deploying and reducing wasted cycles.. offering more rapid continuous value.

**As such, we need to acknowledge that Agile & Continuous Improvement themes came from and built upon movements that we will discuss later in this article. **

The SDLC (Software Development LifeCycle) with Agile Methods :

SDLC with Agile is a more traditional “Enterprise” SW development methodology applying “Agile” methods :

Agile “DevOps” & Cloud Native Development (aka “Cloud First”)

When Patrick DuBois and Andrew Clay Schafer tried to connect at the  2008 Agile Conference about “Agile Infrastructure”, the connection to DevOps was born. Patrick later coined the term “DevOps”, to represent the merging of Development and Operations.

As a contrast to more traditional Development models, today’s DevOps is intended to bypass the serial nature of Development and disjointed Operations.. Integrating them together and focusing around “Automation” of Testing / Integration, decreasing the time to Develop-> Deploy.

While not the topic here, the another Architectural trend being advocated by cloud providers is to develop using a “Cloud Native” or “Cloud First” approach, such as described in the section above relating to The 12Factor App.   This centers around breaking an Application/ Workload into it’s underlying SW components, aka “Services”, and then further into their constituent modules, aka “Micro-Services” for optimal scalability and light-weight (rapid) development cycles.

With Agile DevOps, developers as part of “2 pizza“ teams will typically “Own” the entire life-cycle for specific components/modules and/or “MicroServices“, becoming experts and gatekeepers from Development -> Testing -> (Delivery)  and potentially encompassing automated CI/CD Integration/ Staging -> Prod Deployment.

Hence, extending ownership and capabilities beyond traditional Development, to “Operational” aspects ~= Dev-Ops.

Many of the popular tools of today for DevOps center around Docker Containers / Images (and repositories, many times including public resources such as bitnami ).

With the additional task and complexity in managing many Containers (& Docker “pods”), Kubernetes has filled the void and become the standard for Container mgmt and Orchestration.

Containers are a technology that isn’t new, but is today a perfect fit for Cloud-Native development/testing and Micro-Service based deployments.   The first “Production Ready” Containers actually were released in 2004 by Sun Microsystems as part of Solaris 10 (and later rebranded as “Zones”), but were a technology before their time, with little marketed use-cases (and no DevOps community movement at the time to promote them).   From 2004-2010 Sun Microsystems Open Sourced it’s Solaris OS code, exposing it’s Containers, Dtrace, and ZFS capabilities for other Linux variants to decipher and reverse-engineer within their subsequent OS versions.

*Note that Linux containers weren’t released until 2008, and Docker Containers until 2013.

Today’s Containers (Docker has won that battle as the standard) offer light-weight workload (Non-VM) virtualization within a multi-tenant environment (sharing the same underlying OS).    This offers running many environments with very little hardware (nor VM overhead), and is well suited for cloud-based or locally isolated Development/ Testing & DevOps (where enterprise requirements such as high-performance, high availability, security, and fine-grained production workload isolation of a VM or even bare-metal system aren’t as stringent).   For this reason, today’s DevOps world is in reality becoming more of a DevSecOps world, with the realization that Security needs to be factored in at all levels/ layers, especially when dealing with Enterprise class SLA’s/ SLC’s.

*See our article on DevOps & Cloud Native Development for a more comprehensive review.

Note that Agile DevOps can be in conflict with the methods, process, and culture of traditional Enterprise SW development (and Enterprise Architecture Principles / methods/ reference architectures) that historically have generated large monolithic SOA/ J2EE/ ESB (JAR’s, WAR’s, -> EAR’s) code-bases and libraries which are very slow to extend/test/deploy.

Additionally, many Enterprise Apps (J2EE) run within Java Web/ Application Servers with shared JVMs are diverging from many of today’s application designs and runtime alternatives (eg.  within “Polyglot” multi-language development, and/or Microservice / CNA-Container Native App space, not to mention the rapid growth within Serverless / FaaS-Functions as a Service deployments and architectures).

For Java application environments, many have already moved from JEE to more agile frameworks such as Spring (Opensource open framework), in addition to the many branches of polyglot development (Javascript/ node.js for web, python for rapid development & AI/ datascience, etc).

In terms of the contrast between a traditional “Waterfall” development model vs. Agile SW Dev, see the diagram below :

DevOps with Continuous Integration, Delivery, and Deployment tools

Today’s “Holy Grail” of DevOps is to Automate the entire  process, integrated end-to-end so that each developer/team can readily control/manage the process from via “Pipelines” from Development -> Automated Testing -> Automated Integration & Deployment to Production

** See our other article for more on SW Development & DevOps & CI/CD **

Project & Quality Management Methodologies

 (BSP, PMP, DSDM, Lean Manufacturing, TQM, RUP / UML, Kanban, Scrum)

Note that we’re intentionally leaving out the basic “Project Management” history and disciplines to focus here on the more recent THEMES and Objectives that we find all around us today :   Continuous Improvement, Agile/ Rapid methodologies, ..to Scrum teams

Even though little thought, nor mention is given today, the history of Quality and Continuous Improvement Methodologies is very interesting.

At the core of today’s Total Quality Management (TQM) and “Continuous Improvement” movements, brings us to ~1950’s Japan where the United States had sent it’s best team of Quality Control & Quality Management guru’s to assist in the rebuilding of Japan’s economy and manufacturing sectors.

Notably :

• Joseph M. Juran
  Mr. Juran taught managing for quality.  The first edition of Juran’s Quality Control Handbook in 1951 attracted the attention of the Japanese Union of Scientists and Engineers (JUSE), which invited him to Japan in 1952.   When he finally arrived in Japan in 1954, Juran met with executives from ten manufacturing companies, and to this day has made a tremendous impact.

• W. Edwards Deming
  Deming taught America’s cutting edge Statistical Analysis (from Western Electric’s advancements with Juran), fathering much of what is today’s TQM mindset, and as Wikipedia notes :

Deming as one of the inspirations for what has become known as the Japanese post-war economic miracle of 1950 to 1960, when Japan rose from the ashes of war on the road to becoming the second-largest economy in the world through processes largely influenced by the ideas Deming (& others) taught during this post-war “rebuilding” period :^[4]

1. 1. Better design of products to improve service
   2. Higher level of uniform product quality
   3. Improvement of product testing in the workplace and in research centers
   4. Greater sales through side [global] markets

Deming’s message to Japan’s chief executives was that improving quality would reduce expenses, while increasing productivity and market share (in an iterative way).^[4]

The Deming Cycle (Plan, Do, Study, Act)  is the basis of six sigma’s DMAIC cycle.

• Kaoru Ishikawa

Best know for Japan’s Quality movements (Quality Circles, etc) is Kaoru Ishikawa, who’s leadership is world known.   He translated, integrated and expanded the management concepts of W. Edwards Deming and Joseph M. Juran into the Japanese system.

His Japanese nation-wide movement “Company Wide Quality Control (CWQC)” became famous for a bottom-up process, that was initiated and led from the Top.. which later became much of the Six Sigma mantra.

 In 1960, Ishikawa introduced the concept of quality circles (1962) in conjunction with the Japanese Union of Scientists and Engineers (JUSE) quality control research group.

When you see a “Fishbone” diagram .. think twice, they are actually called “Ishikawa diagrams” :

So when you ask.. ? Why have Japanese automobiles and electronics manufacturing been 2nd to none ? .. remember Deming, Juran, and Ishikawa, .. and the little known fact that prior to their assistance, Japan was actually well known for poor quality in manufacturing.

Until Ford reached out for Deming’s assistance in 1981, American companies denied the need.. and resisted change (after 40 years of American TQM leadership globally), continuing to operate inefficiently until loosing Billion$ during the 1980’s.. when Japan had already eaten their lunch (only then did Ford’s motto change to : “Quality is job 1”).

LEAN Manufacturing & the Toyota Production System (TPS)

LEAN & TPS also brought us several tools/ approaches that have since been applied world-wide (in addition to a few of the best-selling business books) :   

Kanban diagrams (and pull systems)

Just-in-time (TPS: JIT Manufacturing)

JIT, or “flow”, has ruled the world of Manufacturing Inventory Control  and Supply Chain Management ever since defining that in a perfect manufacturing process that “pulls” and “flows”, there will be NO over-purchased inventory.

“Autonomation” (TPS: smart automation)

Lean Manufacturing has the five principles :

Lean Manufacturing also identifies seven wastes that can exist in any manufacturing process :

Jumping forward a decade or 2, skipping over the 2000’s craze around TQM and SixSigma.. brings us to :

SCRUM – “, aka the implementation of “Agile Development” for Project Management

Summarized by the Scrum Alliance :

Scrum is an Agile framework for completing complex projects. Scrum originally was formalized for software development projects, but it works well for any complex, innovative scope of work. The possibilities are endless. The Scrum framework is deceptively simple.

ITSM – IT Service Management Methodologies (ITIL & COBIT)

ITSM frameworks, such as ITIL and COBIT provide a set of interrelated “Best Practices” that provide guidance for developing, delivering, and managing enterprise IT services.

While in the 1990’s and 2000’s, IT Service Management Strategy was the “rage”, today’s rapid shift toward Cloud-enabled Services & Deployment Models has placed ITSM on the sideline, with much of it’s process and best practices being incorporated within Enterprise Architecture & embedded as part of the latest generation of automated/ integrated Infrastructure as Operational & platform Management tools (Dell’s HCI – Hyper Converged Infrastructure, VMware’s SDDC– the Software Defined DataCenter, Oracle’s Engineered Systems, HW “Appliances“, and/or within Cloud IaaS/ PaaS, all of which abstract the control plane management capabilities and interfaces within a SPOG – Single Pane of Glass Management interface).   

For formal Governance and Detailed ITSM process/methodology, Enterprise Architecture can be overlapped with ITSM (eg. ITIL), as depicted at the tail end of the EA/ TOGAF section, but again caution related to process “agility” and business/ cultural impacts need to be considered.

1989 – ITIL—short for Information Technology Infrastructure Library

In 1989, GTIMM (Government Infrastructure Management Method) was renamed to ITIL.    ITIL is a framework for implementing ITSM most predominantly centered around managing Enterprise IT Services and Data/ Information systems with business/LOB’s.  It’s best known by it’s library of best practices and abundance of practitioners.

ITIL version 2 (2000) :

In 2000, ITIL v 2 rearanged the 30 volumes into 9 categories.    ITIL also saw it’s adoption grow with Microsoft adopting it as the foundation for their MOF (Microsoft Operations Framework)

1996 – COBIT(Control Objectives for Information and Related Technology)

In 1996, COBIT was released by the ISACA (Information Systems Audit and Control Association) for IT governance and management.   While it won’t be discussed here in detail, but is best suited where IT Governance is the central focus being short feedback loops for quality, control, risk and reliability management :

Site Reliabilty Engineering (2003)

In 2003, Benjamin Treynor @Google coined the term “SRE”, reflecting titles of production Engineers (blending administration, operations, systems engineering, and Production support) who’s first priority was in Production Reliability (aka Up-Time).

Since that point in time this unique role has grown to become a well known and recognized title, but definitely is still a staple within Google.

While the SRE doesn’t have a formal methodology nor specific set of processes (external to Google), expect this position to be treated as such (a role), and one set of roles and responsibilities that need to be woven into the fabric of Enterprise Architecture where applicable.

Areas and disciplines related include the following foundation of activities and knowledge :

Below, Google depicts various aspects of an SRE bootstrapping (getting up to speed) in preparation for the realities of supporting live “on-call” activities :

While a unique role, most organizations handle these tasks within other more traditional job descriptions such as : Systems Engineering, Systems Administration, Operations, back-line Support, On-Site Engineers .. or as our discussions above reflected, the growing emphasis on self-sufficient DevOps teams, where the addition of an SRE for business critical environments would be recommended.

The team at Google have written 2 books that will give you a window into their world of ensuring production reliability.   Another interesting read on this topic is an O’Reilly Ebook titled “Case Studies in Infrastructure Change Management How Google Rebuilds the Jet While Flying It“, where Google’s SRE’s also define and describe their use of ICM – Infrastructure Change Management as significant.

Continue the Conversation..

Let us know what you think and please Comment and/or Retweet Below with any Q’s, thoughts, or requests for future content.

Visit our Contact page for further ways to reach out..  or engage and participate by becoming a contributor 🙂