Over the course of the past 2 decades, technology has advanced dramatically to offer the following capabilities, which were required PRIOR to wide-spread, general-purpose adoption of cloud computing (each a pre-req built upon the technology that preceded it). While this is far from a complete list, it gives you a high-level grasp of how rapidly technology has evolved, and is still evolving. :
The main culprits of this latency are found in :
*Enterprise Type-1 VM’s do offer much better “encapsulated” security attributes such as disabling services/ports/RBAC.. (vs. Containers). However, it does need to be considered that that since every VM has it’s own local COPY of an entire OS Kernel (eg. RH Linux, Win10, ..), and the trade-off is the additional required footprints in capacity overhead (storage space, and more notably, CPU/Memory), NTM the additional performance overhead of running many OK kernels on a shared multi-tenant system with many VM’s.
SDN de-couples the network functions from the physical devices/network, ~analogous to de-coupling virtual servers (VMs) from physical servers, re-creating the traditional network constructs in a virtual space including NICs/ ports, switches, routers, firewalls, etc. (eg. VMware’s NSX/NSX-T)
From VMware’s (vSAN) page :
SDS software abstracts the underlying storage through a virtual data plane; this makes the VM (and the application) the fundamental unit of storage provisioning and management. By creating a flexible separation between applications and available resources, the pervasive hypervisor can balance all IT resources—compute, memory, storage, and networking—needed by an application.
Software-defined storage (SDS) is a storage data center architecture that separates the management and provisioning of storage from the underlying physical hardware, similar to server and network virtualization. SDS replaces static and inefficient hardware with dynamic, agile, and automated solutions. VMware’s simple-yet-intelligent SDS technologies deliver a more scalable, flexible, and automated storage solution for today’s business and application demands.
With software-defined storage, applications are dynamically provisioned with the precise mix of capacity, performance, and protection needed—no more, no less.
This storage solution allows businesses to eliminate inefficient, purpose-built hardware and use more agile, transparent, and automated systems instead. It brings to storage the simplicity, efficiency, and cost-savings that server virtualization brought to compute.
Leading examples of SDS within the industry (both from Dell-EMC/ VMware) include :
NOTE: Today’s Cloud-Native movement toward “MicroServices” advocates (via 12 Factor SW Architecture) decoupling individual SW functions/services to run individually/ separately in containers (as horizontally scalable, distributed, lightweight, and if possible stateless application components). For this reason, Production container deployments with MicroServices can result in thousands of containers that need to speak with one-another, and address cascading failures (via curcuit breaker patterns), the task can quickly become a SIGNIFICANT feat to architect/design/code/configure/test & most importantly support according to SLA’s.
*Since containers are little more than isolated regions within a shared OS Kernel (only needing the app, & related libraries), they are extremely light-weight and extremely fast to spin up. However, it needs to be realized that since root can gain access to all underlying containers/resources, Container isolation and security can be a matter of great concern and require significant configuration & oversight (as compared to VM’s, which fully encapsulate the entire OS and all applications within). For this reason, today many Public Cloud providers actually encapsulate container environments within VM’s, especially when in multi-tenant (shared HW) environments.
At the same time that technology was rapidly advancing, we need to look back and recognize the economic down-turn that simultaneously hit all with Global Recessions, that we still feel the remnants of today. We need to realize that in part, this down-turn in Corporate spending led to many of today’s movements toward :
This economic down-turn, coupled with a decade+ of FOSS (Free and Open Source Software) being promoted/ developed within Universities resulted in rapid adoption and deployments within business Production environments. While this did initially reduce up-front CAPEX “acquisition Costs”, it brought forward much of today’s COMPLEXITY, that has since ballooned across the following areas of IT management :
As the computer industry embraced these paradigm shifts, they also witnessed their customers both cutting budgets, while many in the long-run (after several years of FOSS & VM sprawl) ultimately struggle with TCO increases, and as significantly (if not more-so) escalating complexity to manage/operate their environments.
This reality is what led most vendors to utilize the technology listed above to develop and promote “Converged Infrastructure” (Cisco UCS.. / VBlock, ..) and/or “Engineered Systems” (Oracle’s Hyper-converged) platforms, to REDUCE both COSTS and COMPLEXITY in On-premises IT management.
Most converged solutions require and leverage SPOG – Single Pane of Glass Management suites (Oracle’s is OEM 13c Cloud Control, while many competing x86 platforms leverage VMware ESX/i).
Together, these “Single-Stack” platforms do unify HW, OS, VM, & Patch Management.. ultimately REDUCING :
Many of today’s organizations still have dominant footprints running On-Premises in Private-Cloud Converged/ Hyper-Converged/ SDN Pod architectures.
** See our other articles for a more detailed examination of today’s HYPER-Converged Infrastructure offerings from today’s leading vendors such as Dell Technologies (Dell-EMC, VMware, Pivotal, Bitfusion..+) VxRack/VxRail offerings that offer VMware’s SDDC (Software Defined DataCenter – VMware vSphere/ESXi, vSAN, vCenter/vRA/vRO, +) and even can fully integrate VMware’s VCF (VMware Cloud Foundation) stack within a HW solution, tested IN the factory.
At the same time that Converged Infrastructures were being adopted, Amazon (AWS), Microsoft (Azure), Google, Oracle (OCI), and IBM have rapidly shifted their focus and R&D efforts on Public Cloud IaaS / PaaS / SaaS offerings …
Upon the arrival of tools like Chef and Puppet, the concept of Infrastructure as Code was born (Configuration Management automation). Together these tools enabled developers / sysadmins to script & REUSE the former manual command-line driven process of maintaining/configuring infrastructure. Upon use and re-use/extensions of these tools / techniques, the PROCESS of building & maintaining systems and infrastructure (deployment environments) began to closely resemble the way that SW developers build/ maintain their applications and source code.
At the same time that IaC was gaining traction, several advancements and many competing efforts, tools and frameworks to automate and centralizing the Management of these many virtualized commodity environments grew rapidly.
Many ad-hoc tools began to fill the void in the Application Container space (today’s Docker and competing CoreOS rkt) offering a lighter-weight way to virtualize, image, and rapidly deploy services without full-fledged heavy weight VM’s/ Hypervisors.
.. which needed an Orchestration engine, that Kubernetes has taken hold of (for now).
** For a deeper examination re: IaC Tools (Ansible, Chef, Puppet, Terraform w/ Docker/Kubernetes) See our article DevOps 101 : .. to Agile Cloud Native Development (w CI/CD).
The ultimate solution for managing any environment is to offer SPOG (Single Pane of Glass) management, unifying all monitoring, provisioning, and lifecycle management capabilities through ONE centralized interface.
*While the following solutions were initially created for private cloud management, most IaaS management solutions today are extensible (beyond their own proprietary API’s), offering open REST api’s (Representational State Transfer), extending Private-Cloud IaaS / PaaS to Hybrid Cloud models.
Also NOTE that each and every Public Cloud provider has developed or extended similar solutions which allow for operating, management, and monitoring of their respective cloud platforms (eg. Dreamhost compute cloud for example is built upon OpenStack).
The best know of the full-stack IaaS Cloud Management Platforms (CMP’s) are :
Openstack is a combination of open source tools (known as projects) that use pooled virtual resources to build and manage private and public clouds. Six of these projects handle the core cloud-computing services of compute, networking, storage, identity, and image services.
The comprehensive nature of OpenStack is both it’s strength and weakness.
While OpenStack had become the dominant FOSS player (beating out CloudStack), it recently has lost momentum due to the complex installation & integration required, along with today’s rapid shift to utilize and explore the growing landscape of Public Cloud services (AWS, GCP, OCI, Azure..), where both IaaS & PaaS platform services (DBaaS, JaaS, BDaaS, IoT,..) can be rapidly provisioned and centrally managed remotely via HTTPS at the click of a mouse.
On the positive side, OpenStack today has also embraced and extended it’s capabilities beyond Bare Metal (BM) and Virtualized environments (VMs) to also include containers and Kubernetes. It’s extensibility can also be noted to offer many variations for deployment, such as incorporating (underneath or in place of it’s own SWIFT ObjStore) the Ceph distributed object, block, and file storage platform (which has become a very popular alternative distributed Object Storage platform).
The VMware vRealize Suite (formerly called vCenter Operations Management Suite), is a software platform designed to help IT administrators build and manage heterogeneous, hybrid clouds. In addition, VMware has done the best job of integrating Hybrid solution frameworks with nearly all cloud providers, as depicted below.
Beyond the Public Cloud providers (and as eluded to above), several other targeted solutions/platforms and frameworks exist for less comprehensive management of Application Workloads in a PaaS model (vs. inclusive of all Infrastructure/Storage/Networking/Virtualization – IaaS, as described in the sections above and explained/contrasted further in the next section).
Most of these platforms today leverage the proliferation of OpenSource components (eg. Devel- Spring, Languages- Java, Python, OS- Linux, Containers- Docker, Sched/Orchestration- Kubernetes, etc) embedded together, extending prior PaaS solutions.
Again, without offering a comprehensive list, but touching on a few of the biggest names in this space are :
As outlined on Wikipedia : Cloud Foundry :
The Cloud Foundry platform is available from either the Cloud Foundry Foundation as open-source software or from a variety of commercial providers as either a software product or delivered as a service. Cloud Foundry is open-source software, and hence, it is available to anyone. Deploying Cloud Foundry involves interfacing with the underlying infrastructure using the Cloud Foundry BOSH deployment system, another open-source tool governed by the Cloud Foundry Foundation. Note: In 2009, VMware initiated the original development, before being transferred to Pivotal. In April 2013, Pivotal was created from EMC and VMware, to market assets including Cloud Foundry, RabbitMQ and Spring.[7] In 2019, Pivotal was fully acquired by VMware.
Cloud Foundry is promoted for continuous delivery as it supports the full application development lifecycle, from initial development through all testing stages to deployment. Cloud Foundry’s container-based architecture runs apps in any programming language over a variety of cloud service providers. This multi-cloud environment allows developers to use the cloud platform that suits specific application workloads and move those workloads as necessary within minutes with no changes to the application.
Applications deployed to Cloud Foundry access external resources via an Open Service Broker API, which was launched in December 2016.[15]
In a platform, all external dependencies such as databases, messaging systems, file systems and so on are considered services. Cloud Foundry allows administrators to create a marketplace of services, from which users can provision these services on-demand. When an application is pushed to Cloud Foundry the services it needs may also be specified. This process puts the credentials in an environment variable.
OpenShift Container Platform (formerly known as OpenShift Enterprise[12]) is Red Hat’s on-premises private platform as a service product, built around a core of application containers powered by Docker, with orchestration and management provided by Kubernetes, on a foundation of Red Hat Enterprise Linux and Red Hat Enterprise Linux CoreOS (RHCOS)[13].
The main differentiator between OpenShift and vanilla Kubernetes is the notion of build related artifacts being first class Kubernetes resources upon which standard Kubernetes operations can apply. The OpenShift client program is “oc” – which offers a superset of capabilities offered by the “kubectl” client program of Kubernetes[11]. Using this client, one can directly interact with the build related resources using sub-commands (such as “new-build” or “start-build”). In addition to this, an OpenShift-native pod build technology called Source-to-Image (S2I) is available out of the box. For the OpenShift platform, this provides capabilities equivalent to what Jenkins can do.
Another difference when OpenShift compared to Kubernetes:
On September, 2011 (only 6 years ago), the National Institute of Standards and Technology’s (NIST) working definition of cloud computing, the 16th and final definition has been published as The NIST Definition of Cloud Computing (NIST Special Publication 800-145).
This defined “cloud computing” as including the following (taken and paraphrased from the link above) :
Before your organization boils the ocean and does detailed Discovery & Requirements Analysis, the 4 Key areas in the chart below highlight the Key Questions that you need to first explore more thoroughly in order to determine cloud “viability”, let alone select the appropriate Cloud Service (IaaS, PaaS, SaaS) & Deployment Models (Private, Hybrid, Public) :
One critical area that needs to be planned for in advance is how Cloud-enablement and Digital Transformation ultimately shifts the Business Operating Model of IT : supporting End-users, Lines of Business, AppDev, Admin & Operations teams (See my other article regarding how this relates to the 4 key areas : Control, Security, Latency, & Cost).
The transition to Cloud requires a Business Transformation of IT to a Cloud Operating Model and typically a radical shift in IT Operations. For companies that are closing entire datacenters and migrating the majority of Infrastructure + Application Environments to Public Cloud providers, this transition can be VERY disruptive. The list below specifically reflects impact to areas/environments migrated to Public Cloud providers, and not necessarily for remaining On-Premises Private/Hybrid Cloud deployments :
Today we’re blessed with a wide depth and breadth of both On-premises Private Cloud HW/Solution options as well as an incredible pace of expanding (and very robust) Public Cloud IaaS -> PaaS -> SaaS offerings with mature (and secure) capabilities. While the trends uniformly point to rapid current/future Cloud adoption, stay tuned for future updates as vendors and Cloud providers gain/loose traction and jockey positions as the market dictates it’s supply & demand for various cloud services and deployment models.
NOTE that the following data is only valid for ~1 Qtr, given that 3 months in Cloud-Years is like a normal Calendar Year (things are changing rapidly, to say the least). Check online for the latest capabilities and vendor announcements.
Several of the following charts come from a Forbes article , IDC data, as well as RightScale survey data.. aggregating this data from various studies and demonstrating a clear #Trend :
The following Charts & Data are from : RightScale 2018 & RightScale 2017 State of the Cloud Reports
2017 Spending |
2017 Growth (%) |
2018 Spending |
2018 Growth (%) |
2019 Spending |
2019 Growth (%) |
|
Data Center Systems |
181 |
6.3 |
188 |
3.7 |
190 |
1.1 |
Enterprise Software |
352 |
8.8 |
391 |
11.1 |
424 |
8.4 |
Devices |
663 |
5.1 |
706 |
6.6 |
715 |
1.3 |
IT Services |
933 |
4.4 |
1,003 |
7.4 |
1,048 |
4.6 |
Communications Services |
1,392 |
1.3 |
1,452 |
4.3 |
1,468 |
1.1 |
Overall IT |
3,521 |
3.8 |
3,740 |
6.2 |
3,846 |
2.8 |
** In (past) future articles we will be delving into some of the topics covered here in more detail, including :
Let us know what you think and please Comment and/or Retweet Below with any Q’s, thoughts, or requests for future content.
Visit our Contact page for further ways to reach out.. or engage and participate by becoming a contributor 🙂
To contact the author directly :
All content and comments made are that of the author, and not of current nor past employers.