Hybrid Cloud Computing Pattern
Abstract
Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. The Hybrid Cloud Computing (HCC) capability pattern provides a practical, pragmatic guide for development of cloud computing capability focused on interoperability and design for affordability. This capability pattern seeks to balance the cost, speed, and agility afforded cloud computing consumers with the required security, privacy and confidentiality. The HCC seeks to use hybrid cloud capabilities to balance the cost, speed, and agility afforded cloud computing consumers with the required security, privacy and confidentiality. This cloud computing capability pattern can be used to make Government more agile and adaptive with a focus on collaboration, openness and transparency
Description
Problem Statement & Context
Problem Statement
In the past year, cloud computing has morphed from an obscure concept into the driving force behind Federal information technology. Although its merits are still hotly debated, this new approach promises speed, agility and low cost with greatly improved security, privacy and confidentiality. When unveiled last year, Federal CIO Vivek Kundra highlighted three key elements of this new initiative. The first major element was the Apps.gov site, a clearinghouse for business, social media, and productivity applications, as well as cloud IT services. According to Mr. Kundra, the second element of the effort will be budgeting. For fiscal year 2010, cloud computing pilot projects will be pushed hard followed in 2011 by the issuance of more definitive guidance to agencies throughout government. The final key element will include policy planning and architecture consisting of centralized certifications, target architectures and formal security, privacy, and procurement processes. While many still honestly debate the value of cloud computing, deployment of this grand vision will also depend on the realization of cloud interoperability (integration of cloud capabilities) and portability (the ability to move from one cloud service / provider to another one). In a world of many and varied cloud computing solutions, government users must be able to freely move from one cloud vendor to another. This is not only an important technical requirement, but is also critical to protecting federal procurement integrity. This pattern applies to various across Government domains: Joint Ground, Air, Maritime operations, Organization and Nations combinations, Civil and Military coordination.
Context
Definition
Cloud Computing: Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. (This definition is from the latest draft of the NIST Working Definition of Cloud Computing published by the U.S. Government's National Institute of Standards and Technology1.)
What is cloud computing?
Definition
Definition: Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.
Three Delivery models
Software-as-a-Service
The consumer uses an application, but does not control the operating system, hardware or network infrastructure on which it's running.
Example
Salesforce.com
Platform-as-a-Service
ii. Platform as a Service (PaaS): The consumer uses a hosting environment for their applications. The consumer controls the applications that run in the environment (and possibly has some control over the hosting environment), but does not control the operating system, hardware or network infrastructure on which they are running. The platform is typically an application framework.
Example
Google AppEngine
Force.com
Infrastructure-as-a-Service
iii. Infrastructure as a Service (IaaS): The consumer uses "fundamental computing resources" such as processing power, storage, networking components or middleware. The consumer can control the operating system, storage, deployed applications and possibly networking components such as firewalls and load balancers, but not the cloud infrastructure beneath them.
Example
Amazon Web Service
Amazon played a key role in the development of cloud computing by modernizing their data centers after the dot-com bubble, which, like most computer networks, were using as little as 10% of their capacity at any one time just to leave room for occasional spikes. Having found that the new cloud architecture resulted in significant internal efficiency improvements whereby small, fast-moving "two-pizza teams" could add new features faster and easier, Amazon started providing access to their systems through Amazon Web Services on a utility computing basis in 2005.[22] This characterization of the genesis of Amazon Web Services has been characterized as an extreme over-simplification by a technical contributor to the Amazon Web Services project.[23]
Unisys
EMC Atmos
Loudcloud
Loudcloud, founded in 1999 by Marc Andreessen, was one of the first to attempt to commercialize cloud computing with an Infrastructure as a Service model.[20] By the turn of the 21st century, the term "cloud computing" began to appear more widely,[21] although most of the focus at that time was limited to SaaS, called "ASP's" or Application Service Providers, under the terminology of the day.
Services
Compute
Physical Machines
Virtual Machines
OS-level virtualization
Network
Storage
Other as-a-service
Rational for adopting NIST definition
Consideration for other delivery models
DEsktop-as-a-service
Data-as-a-Service
Grid-as-a-service
Utility COmputing
Four Deployment Models
Public Cloud
i. Public Cloud: In simple terms, public cloud services are characterized as being available to clients from a third party service provider via the Internet. The term “public” does not always mean free, even though it can be free or fairly inexpensive to use. A public cloud does not mean that a user’s data is publically visible; public cloud vendors typically provide an access control mechanism for their users. Public clouds provide an elastic, cost effective means to deploy solutions.
Private Cloud
ii. Private Cloud: A private cloud offers many of the benefits of a public cloud computing environment, such as being elastic and service based. The difference between a private cloud and a public cloud is that in a private cloud-based service, data and processes are managed within the organization without the restrictions of network bandwidth, security exposures and legal requirements that using public cloud services might entail. In addition, private cloud services offer the provider and the user greater control of the cloud infrastructure, improving security and resiliency because user access and the networks used are restricted and designated.
Community Cloud
iii. Community Cloud: A community cloud is controlled and used by a group of organizations that have shared interests, such as specific security requirements or a common mission. The members of the community share access to the data and applications in the cloud.
Hybrid Cloud
iv. Hybrid Cloud: A hybrid cloud is a combination of a public and private cloud that interoperates. In this model users typically outsource nonbusiness- critical information and processing to the public cloud, while keeping business-critical services and data in their control.
Five essential characteristics
Rapid Elasticity
i. Rapid Elasticity: Elasticity is defined as the ability to scale resources both up and down as needed. To the consumer, the cloud appears to be infinite, and the consumer can purchase as much or as little computing power as they need. This is one of the essential characteristics of cloud computing in the NIST definition.
Measured Service
ii. Measured Service: In a measured service, aspects of the cloud service are controlled and monitored by the cloud provider. This is crucial for billing, access control, resource optimization, capacity planning and other tasks.
On-Demand Self-Service
iii. On-Demand Self-Service: The on-demand and self-service aspects of cloud computing mean that a consumer can use cloud services as needed without any human interaction with the cloud provider.
Ubiquitous Network Access
iv. Ubiquitous Network Access: Ubiquitous network access means that the cloud provider’s capabilities are available over the network and can be accessed through standard mechanisms by both thick and thin clients.
Resource Pooling
v. Resource Pooling: Resource pooling allows a cloud provider to serve its consumers via a multi-tenant model. Physical and virtual resources are assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter).
Two Domains
Enterprise
i. Enterprise – "replacement for static reconfuration", usually physical organization leveraging the Internet or intranet for global connectivity with centralized governance and management (need better difiniton than static)
Tactical/DEployable/Adhoc (Need new name)
ii. Tactical – dynamic, virtual organization typically short-lived. Leverages wireless networking technology to leverage internetworking technologies to form private or public, interoperable systems in order to accomplish a time or goal specified activity.
Cloud computing is not.
Grid Computing
a. Grid computing — "a form of distributed computing, whereby a 'super and virtual computer' is composed of a cluster of networked, loosely coupled computers acting in concert to perform very large tasks"
Utility Computing
b. Utility computing — the "packaging of computing resources, such as computation and storage, as a metered service similar to a traditional public utility, such as electricity"
Autonomic Computing
c. Autonomic computing — "computer systems capable of self-management"
Government Cloud Computing Examples
United States
Federal Chief Information Officers Council Data.gov & IT Dashboard Defense Information Systems Agency (DISA) Rapid Access Computing Environment (RACE) US Department of Energy (DOE) Magellan General Services Administration (GSA) Apps.gov Department of the Interior National Business Center (NBC) Cloud Computing NASA Nebula National Institute of Standards and Technology (NIST)
European Union
Resources and Services Virtualization without Barriers Project (RESERVOIR)
Canada
Canada Cloud Computing Cloud Computing and the Canadian Environment
United Kingdom
G-Cloud
Japan
The Digital Japan Creation Project (ICT Hatoyama Plan) The Kasumigaseki Cloud
Architectural TEnants
1. Maintain security and control of personal identifiable information and data on premise (I.e., from cyber attack, data compromise, and personal identity theft). 2. Obtain agility and cost benefits from the use public cloud capabilities 3. Develop cloud bursting capability to right sized private cloud 4. Extend architectural capabilities to mobile and handheld devices
Pre-Conditions
There should be an agreed upon enterprise security, privacy and confidentiality policy, with a complete understanding of data location, access rights and security methods. There should be means of communication (with sufficient bandwidth and secured as appropriate) between systems willing to share. Right to share: legal issues and other possible obstacles to information sharing must be solved for getting the full benefit of the pattern. Will to share: organisational, cultural and other human issues may create obstacles to information sharing and should be solved for getting the full benefit of the pattern Policies & Contracts, including Service Level Agreements (SLAs) MUST be referenced.
Participants
Taxonomy
Service Consumer
The service consumer is the end user or enterprise that actually uses the service, whether it is Software, Platform or Infrastructure as a Service. Depending on the type of service and their role, the consumer works with different user interfaces and programming interfaces. Some user interfaces look like any other application; the consumer does not need to know about cloud computing as they use the application. Other user interfaces provide administrative functions such as starting and stopping virtual machines or managing cloud storage. Consumers writing application code use different programming interfaces depending on the application they are writing. Consumers work with SLAs and contracts as well. Typically these are negotiated via human intervention between the consumer and the provider. The expectations of the consumer and the reputation of the provider are a key part of those negotiations.
Service Provider
The service provider delivers the service to the consumer. The actual task of the provider varies depending on the type of service: • For Software as a Service, the provider installs, manages and maintains the software. The provider does not necessarily own the physical infrastructure in which the software is running. Regardless, the consumer does not have access to the infrastructure; they can access only the application. • For Platform as a Service, the provider manages the cloud infrastructure for the platform, typically a framework for a particular type of application. The consumer’s application cannot access the infrastructure underneath the platform. • For Infrastructure as a Service, the provider maintains the storage, database, message queue or other middleware, or the hosting environment for virtual machines. The consumer uses that service as if it were a disk drive, database, message queue, or machine, but they cannot access the infrastructure that hosts it. In the service provider diagram, the lowest layer of the stack is the firmware and hardware on which everything else is based. Above that is the software kernel, either the operating system or virtual machine manager that hosts the infrastructure beneath the cloud. The virtualized resources and images include the basic cloud computing services such as processing power, storage and middleware. The virtual images controlled by the VM manager include both the images themselves and the metadata required to manage them. Crucial to the service provider’s operations is the management layer. At a low level, management requires metering to determine who uses the services and to what extent, provisioning to determine how resources are allocated to consumers, and monitoring to track the status of the system and its resources. At a higher level, management involves billing to recover costs, capacity planning to ensure that consumer demands will be met, SLA management to ensure that the terms of service agreed to by the provider and consumer are adhered to, and reporting for administrators. Security applies to all aspects of the service provider’s operations. (The many levels of security requirements are beyond the scope of this paper.) Open standards apply to the provider’s operations as well. A well-rounded set of standards simplify operations within the provider and interoperability with other providers.
Service Developer
The service developer creates, publishes and monitors the cloud service. These are typically "line-of-business" applications that are delivered directly to end users via the SaaS model. Applications written at the IaaS and PaaS levels will subsequently be used by SaaS developers and cloud providers. Development environments for service creation vary. If developers are creating a SaaS application, they are most likely writing code for an environment hosted by a cloud provider. In this case, publishing the service is deploying it to the cloud provider’s infrastructure. During service creation, analytics involve remote debugging to test the service before it is published to consumers. Once the service is published, analytics allow developers to monitor the performance of their service and make changes as necessary.
Structure
Critical Elements
• Development of robust security, privacy and confidentiality • Acquisition of the the speed, lower cost and agility capabilities • Extension of enterprise capabilities to end user hand held devices
HCC On-Premise Environment
The enterprise on-premise environment is the foundation of the HCC pattern. Integration and utilization of the on-premise environment is critical as it is generally the source of enterprise secure data. It provides Government agencies a known secure enterprise environment for development of enterprise data and applications, and by using it as part of the cloud computing capability it reduces the concern for lack of control, security, privacy and confidentiality of data when developing cloud solutions
HCC Public and Private Clouds
Public and private clouds offer speed of acquisition and use, utility based pricing and more agility especially when integrated with the enterprise on-premise environment. Public cloud providers offer infinitely elastic automatic scale and virtual private cloud capabilities that lower cost and seek to manage security risk. Private clouds offer right sized cloud bursting capabilities for applications that extend beyond traditional short lived public cloud needs or for applications that need to run when bandwidth or survivability issues exist. In order to make use of cloud services part of an agency enterprise solution, interoperability issues, cloud integration, and portability issues need to be resolved. The HCC pattern with it integration of on-premise enterprise environments with public and private cloud capabilities balances benefits and risks associated with cloud computing adoption.
HCC Mobile and Hand Held Devices
This design pattern provides for the extension of enterprise capabilities to multiple mobile and hand held devices.
Use Case Scnarios
Enterprise
End User to Cloud
Applications running on the cloud and accessed by end users
Cloud-based collaborative environment
Enterprise to CLoud to End User
Applications running in the public cloud and accessed by employees and customers
Enterprise to Cloud
Cloud applications integrated with internal IT capabilities
Cloud Bursting
Enterprise to Cloud to Enterprise
Cloud applications running in the public cloud and interoperating with partner applications (supply chain)
Virtually binding shipboard & Land vehicle IT infrastrcutures
Private Cloud
A cloud hosted by an organization inside that organization’s firewall.
Community Cloud
Subset of Hybrid CLoud. Intranet instead of internet
Changing Cloud Vendors
An organization using cloud services decides to switch cloud providers or work with additional providers.
Hybrid Cloud
Multiple clouds work together, coordinated by a cloud broker that federates data, applications, user identity, security and other details.
Tactical/Deployable (Additional Requirements)
Limited/Intermittent Connectivity
Network Connection Authentication
Redundant Compute/Storage Processes
Autonomic Capabilities
Post Conditions
The HCC pattern offers the ability to provide increased adoption of cloud computing. It provides increased interoperability and portability of cloud based solutions and supports nascent cloud computing standards.
Implementation Guidance
Requirements
Security Requirements
Regulations
Regulations are not technical issues, but they must be addressed. Laws and regulations will determine security requirements that take priority over functional requirements.
Security Controls
Although a given consumer might need all of these security controls, consumers should be wary of any cloud provider that makes security-related claims and reassurances without an infrastructure capable of delivering all of them.
Asset Management
It must be possible to manage all of the hardware, network and software assets (physical or virtual) that make up the cloud infrastructure. This includes being able to account for any physical- or network-based access of an asset for audit and compliance purposes.
Cryptography: Key and Certificate Managemnt
Any secure system needs an infrastructure for employing and managing cryptographic keys and certificates. This includes employing standards-based cryptographic functions and services to support information security at rest and in motion.
Data/Storage Security
It must be possible to store data in an encrypted format. In addition, some consumers will need their data to be stored separately from other consumers' data.
Endpoint Security
Consumers must be able to secure the endpoints to their cloud resources. This includes the ability to restrict endpoints by network protocol and device type.
Event Auditing and Reporting
Consumers must be able to access data about events that happen in the cloud, especially system failures and security breaches. Access to events includes the ability to learn about past events and reporting of new events as they occur. Cloud providers cause significant damage to their reputations when they fail to report events in a timely manner.
Identity, Roles, Access Control and Attributes
It must be possible to define the identity, roles, entitlements and any other attributes of individuals and services in a consistent, machine-readable way in order to effectively implement access control and enforce security policy against cloud-based resources.
Network Security
It must be possible to secure network traffic at the switch, router and packet level. The IP stack itself should be secure as well.
Security Policies
It must be possible to define policies, resolve, and enforce security policies in support of access control, resource allocation and any other decisions in a consistent, machinereadable way. The method for defining policies should be robust enough that SLAs and licenses can be enforced automatically.
Service Automation
There must be an automated way to manage and analyze security control flows and processes in support of security compliance audits. This also includes reporting any events that violate any security policies or customer licensing agreements.
Workload and Service Management
It must be possible to configure, deploy and monitor services in accordance with defined security policies and customer licensing agreements.
Security Federation Patterns
To implement these security controls, several federation patterns are needed. Cloud providers should deliver these patterns through existing security standards.
Trust
The ability for two parties to define a trust relationship with an authentication authority. That authentication authority is capable of exchanging credentials (typically X.509 certificates), and then using those credentials to secure messages and create signed security tokens (typically SAML). Federated trust is the foundation upon which all the other secure federation patterns are based.
Identity Management
The ability to define an identity provider that accepts a user’s credentials (a user ID and password, a certificate, etc.) and returns a signed security token that identifies that user. Service providers that trust the identity provider can use that token to grant appropriate access to the user, even though the service provider has no knowledge of the user.
Access Management
The ability to write policies (typically in XACML) that examine security tokens to manage access to cloud resources. Access to resources can be controlled by more than one factor. For example, access to a resource could be limited to users in a particular role, but only across certain protocols and only at certain times of the day.
Single Sign-on / Sign-Off
The ability to federate logins based on credentials from a trusted authority. Given an authenticated user with a particular role, federated single sign-on allows a user to login to one application and access other applications that trust the same authority. Federated single sign-off is part of this pattern as well; in many situations it will be vital that a user logging out of one application is logged out of all the others. The Single Sign-On pattern is enabled by the Identity Management pattern.
Audit and Compliance
The ability to collect audit and compliance data spread across multiple domains, including hybrid clouds. Federated audits are necessary to ensure and document compliance with SLAs and regulatory requirements.
Configuration Management
The ability to federate configuration data for services, applications and virtual machines. This data can include access policies and licensing information across multiple domains.
Developer Requirements
Caching
A programming toolkit that supports caching for cloud resources would boost the performance of cloud applications. Developers need an API to flush the cache, and might need an API to specifically put an object or resource into the cache.
Centralized Logging
Logging is a common requirement for all developers, regardless of their role or the type of application they are writing. APIs should support writing logs, examining entries, creating logs and opening and closing log files.
Database
Developers need a way to access cloud databases. Cloud databases vary widely in their design (some are schema-driven and relational, many are neither), so developers writing cloud applications will choose a cloud database provider based on the needs of each application. The database API must support the basic CRUD operations.
Identity Management
Developers need a way to manage identities. In the simplest case, this requires a way to authenticate a given user's credentials. In cases where an application works with multiple data sources and applications, a developer needs a way to federate that user's identities. A federated identity management system can produce credentials to allow a given user to access a particular service even if the service and user have no knowledge of each other. APIs for identity management should cache and delete credentials as appropriate.
Messaging-Point-to-Point
Developers need an API to post messages to a queue and to consume those messages. An API must also allow the developer to peek the message (examine the message’s contents without consuming it).
Messaging-Pub-Sub
Developers need an API to work with topics in a message queuing system. The API should allow developers to post messages to a topic and retrieve messages from a topic.
Raw Compute / Job Processing
Developers need an API to work with large processing jobs such as Hadoop-style data mining. The API should allow developers to start, stop, monitor and pause processing jobs.
Session Management
The ability to manage user sessions is crucial, particularly in a cloud environment. The infrastructure of a cloud is redundant and resilient in the face of machine failures, so sessions must be maintained even when a particular cloud node goes down. The session API must make it easy to access or manipulate the current state of the user session.
Service Discovery
Developers need a way to discover which cloud services are available. Cloud services should be searchable by type of service, with the API providing additional function appropriate to each service type.
SLAs
Developers using service discovery need an automated way to determine the policies of the services their code discovers. With such an API, developers can write applications that interrogate cloud services and select the one that best meets the application’s SLA criteria.
Storage
Developers need a way to access cloud storage services. The API must provide the ability to store and retrieve both data and metadata.
Operational Requirements
End User to Cloud
Identity
The cloud service must authenticate the end user.
Open Client
Access to the cloud service should not require a particular platform or technology.
Security
SLA
Although service level agreements for end users will usually be much simpler than those for enterprises, cloud vendors must be clear about what guarantees of service they provide.
Enterprise to Cloud to End User
Indentity
The cloud service must authenticate the end user.
Open CLient
Access to the cloud service should not require a particular platform or technology.
Federated Identity
In addition to basic the identity needed by an end user, an enterprise user is likely to have an identity with the enterprise. The ideal is that the enterprise user manages a single ID, with an infrastructure federating other identities that might be required by cloud services.
Location Awareness
Depending on the kind of data the enterprise is managing on the user's behalf, there might be legal restrictions on the location of the physical server where the data is stored. Although this violates the cloud computing ideal that the user should not have to know details of the physical infrastructure, this requirement is essential. Many applications cannot be moved to the cloud until cloud vendors provide an API for determining the location of the physical hardware that delivers the cloud service.
Metering and Monitoring
All cloud services must be metered and monitored for cost control, chargebacks and provisioning.
Management and Governance
Public cloud providers make it very easy to open an account and begin using cloud services; that ease of use creates the risk that individuals in an enterprise will use cloud services on their own initiative. Management of VMs and of cloud services such as storage, databases and message queues is needed to track what services are used. Governance is crucial to ensure that policies and government regulations are followed wherever cloud computing is used. Other governance requirements will be industry- and geography-specific.
Security
Any use case involving an enterprise will have more sophisticated security requirements than one involving a single end user. Similarly, the more advanced enterprise use cases to follow will have equally more advanced security requirements.
Common File Format for VMs
A VM created for one cloud vendor’s platform should be portable to another vendor’s platform. Any solution to this requirement must account for differences in the ways cloud vendors attach storage to virtual machines.
Common APIs for Cloud Storage and Middleware
The enterprise use cases require common APIs for access to cloud storage services, cloud databases, and other cloud middleware services such as message queues. Writing custom code that works only for a particular vendor’s cloud service locks the enterprise into that vendor’s system and eliminates some of the financial benefits and flexibility that cloud computing provides.
Data and Application Federation
Enterprise applications need to combine data from multiple cloud-based sources, and they need to coordinate the activities of applications running in different clouds.
SLAs and Benchmarks
In addition to the basic SLAs required by end users, enterprises who sign contracts based on SLAs will need a standard way of benchmarking performance. There must be an unambiguous way of defining what a cloud provider will deliver, and there must be an unambiguous way of measuring what was actually delivered.
Lifecycle Management
Enterprises must be able to manage the lifecycle of applications and documents. This requirement includes versioning of applications and the retention and destruction of data. Discovery is a major issue for many organizations. There are substantial legal liabilities if certain data is no longer available. In addition to data retention, in some cases an enterprise will want to make sure data is destroyed at some point.
Enterprise to Cloud
Federated Identity
In addition to basic the identity needed by an end user, an enterprise user is likely to have an identity with the enterprise. The ideal is that the enterprise user manages a single ID, with an infrastructure federating other identities that might be required by cloud services.
Open CLient
Access to the cloud service should not require a particular platform or technology.
Location Awareness
Depending on the kind of data the enterprise is managing on the user's behalf, there might be legal restrictions on the location of the physical server where the data is stored. Although this violates the cloud computing ideal that the user should not have to know details of the physical infrastructure, this requirement is essential. Many applications cannot be moved to the cloud until cloud vendors provide an API for determining the location of the physical hardware that delivers the cloud service.
Indentity
The cloud service must authenticate the end user.
Metering and Monitoring
All cloud services must be metered and monitored for cost control, chargebacks and provisioning.
Management and Governance
Public cloud providers make it very easy to open an account and begin using cloud services; that ease of use creates the risk that individuals in an enterprise will use cloud services on their own initiative. Management of VMs and of cloud services such as storage, databases and message queues is needed to track what services are used. Governance is crucial to ensure that policies and government regulations are followed wherever cloud computing is used. Other governance requirements will be industry- and geography-specific.
Security
Any use case involving an enterprise will have more sophisticated security requirements than one involving a single end user. Similarly, the more advanced enterprise use cases to follow will have equally more advanced security requirements.
Common File Format for VMs
A VM created for one cloud vendor’s platform should be portable to another vendor’s platform. Any solution to this requirement must account for differences in the ways cloud vendors attach storage to virtual machines.
Common APIs for Cloud Storage and Middleware
The enterprise use cases require common APIs for access to cloud storage services, cloud databases, and other cloud middleware services such as message queues. Writing custom code that works only for a particular vendor’s cloud service locks the enterprise into that vendor’s system and eliminates some of the financial benefits and flexibility that cloud computing provides.
Data and Application Federation
Enterprise applications need to combine data from multiple cloud-based sources, and they need to coordinate the activities of applications running in different clouds.
SLAs and Benchmarks
In addition to the basic SLAs required by end users, enterprises who sign contracts based on SLAs will need a standard way of benchmarking performance. There must be an unambiguous way of defining what a cloud provider will deliver, and there must be an unambiguous way of measuring what was actually delivered.
Lifecycle Management
Enterprises must be able to manage the lifecycle of applications and documents. This requirement includes versioning of applications and the retention and destruction of data. Discovery is a major issue for many organizations. There are substantial legal liabilities if certain data is no longer available. In addition to data retention, in some cases an enterprise will want to make sure data is destroyed at some point.
Deployment
It should be simple to build a VM image and deploy it to the cloud as necessary. When that VM image is built, it should be possible to move that image from one cloud provider to another, compensating for the different mechanisms vendors have for attaching storage to VMs. Deployment of applications to the cloud should be straightforward as well.
Industry-specific standards and protocols
Many cloud computing solutions between enterprises will use existing standards such as RosettaNet or OAGIS. The applicable standards will vary from one application to the next and from one industry to the next.
Enterprise to Cloud to Enterprise
Federated Identity
In addition to basic the identity needed by an end user, an enterprise user is likely to have an identity with the enterprise. The ideal is that the enterprise user manages a single ID, with an infrastructure federating other identities that might be required by cloud services.
Open CLient
Access to the cloud service should not require a particular platform or technology.
Location Awareness
Depending on the kind of data the enterprise is managing on the user's behalf, there might be legal restrictions on the location of the physical server where the data is stored. Although this violates the cloud computing ideal that the user should not have to know details of the physical infrastructure, this requirement is essential. Many applications cannot be moved to the cloud until cloud vendors provide an API for determining the location of the physical hardware that delivers the cloud service.
Indentity
The cloud service must authenticate the end user.
Metering and Monitoring
All cloud services must be metered and monitored for cost control, chargebacks and provisioning.
Management and Governance
Public cloud providers make it very easy to open an account and begin using cloud services; that ease of use creates the risk that individuals in an enterprise will use cloud services on their own initiative. Management of VMs and of cloud services such as storage, databases and message queues is needed to track what services are used. Governance is crucial to ensure that policies and government regulations are followed wherever cloud computing is used. Other governance requirements will be industry- and geography-specific.
Security
Any use case involving an enterprise will have more sophisticated security requirements than one involving a single end user. Similarly, the more advanced enterprise use cases to follow will have equally more advanced security requirements.
Common File Format for VMs
A VM created for one cloud vendor’s platform should be portable to another vendor’s platform. Any solution to this requirement must account for differences in the ways cloud vendors attach storage to virtual machines.
Common APIs for Cloud Storage and Middleware
The enterprise use cases require common APIs for access to cloud storage services, cloud databases, and other cloud middleware services such as message queues. Writing custom code that works only for a particular vendor’s cloud service locks the enterprise into that vendor’s system and eliminates some of the financial benefits and flexibility that cloud computing provides.
Data and Application Federation
Enterprise applications need to combine data from multiple cloud-based sources, and they need to coordinate the activities of applications running in different clouds.
SLAs and Benchmarks
In addition to the basic SLAs required by end users, enterprises who sign contracts based on SLAs will need a standard way of benchmarking performance. There must be an unambiguous way of defining what a cloud provider will deliver, and there must be an unambiguous way of measuring what was actually delivered.
Lifecycle Management
Enterprises must be able to manage the lifecycle of applications and documents. This requirement includes versioning of applications and the retention and destruction of data. Discovery is a major issue for many organizations. There are substantial legal liabilities if certain data is no longer available. In addition to data retention, in some cases an enterprise will want to make sure data is destroyed at some point.
Deployment
It should be simple to build a VM image and deploy it to the cloud as necessary. When that VM image is built, it should be possible to move that image from one cloud provider to another, compensating for the different mechanisms vendors have for attaching storage to VMs. Deployment of applications to the cloud should be straightforward as well.
Industry-specific standards and protocols
Many cloud computing solutions between enterprises will use existing standards such as RosettaNet or OAGIS. The applicable standards will vary from one application to the next and from one industry to the next.
Transaction Concurrency
For applications and data shared by different enterprises, transactions and concurrency are vital. If two enterprises are using the same cloud-hosted application, VM, middleware or storage, it's important that any changes made by either enterprise are done reliably.
Interoperability
Because more than one enterprise is involved, interoperability between the enterprises is essential.
Private Cloud
Open Client
Metering & Monitoring
Management & Governance
Security
Deployment
Interoperability
Common Vm Format
SLAs
Changing Cloud Vendors
Open Client
Location Awareness
Security
SLAs
Common VM file format
Common CLoud Storage API
Common Cloud Middleware API
This includes all of the operations supported by today’s cloud services, including cloud databases, cloud message queues and other middleware. APIs for connecting to, creating and dropping databases and tables. Cloud database vendors have enforced certain restrictions to make their products more elastic and to limit the possibility of queries against large data sets taking significant resources to process. For example, some cloud databases don't allow joins across tables, and some don't support a true database schema. Those restrictions are a major challenge to moving between cloud database vendors, especially for applications built on a true relational model. Other middleware services such as message queues are more similar, so finding common ground among them should be simpler.
SaaS Vendor
Industry-specific standards
Changing Middleware VEndors
Industry-specific standards
Common Cloud Middleware APIs
Changing Cloud Storage VEndors
Common CLoud Storage API
Changing VM host
Common VM Format
Hybrid Cloud
Federated Identity
In addition to basic the identity needed by an end user, an enterprise user is likely to have an identity with the enterprise. The ideal is that the enterprise user manages a single ID, with an infrastructure federating other identities that might be required by cloud services.
Open CLient
Access to the cloud service should not require a particular platform or technology.
Location Awareness
Depending on the kind of data the enterprise is managing on the user's behalf, there might be legal restrictions on the location of the physical server where the data is stored. Although this violates the cloud computing ideal that the user should not have to know details of the physical infrastructure, this requirement is essential. Many applications cannot be moved to the cloud until cloud vendors provide an API for determining the location of the physical hardware that delivers the cloud service.
Indentity
The cloud service must authenticate the end user.
Metering and Monitoring
All cloud services must be metered and monitored for cost control, chargebacks and provisioning.
Management and Governance
Public cloud providers make it very easy to open an account and begin using cloud services; that ease of use creates the risk that individuals in an enterprise will use cloud services on their own initiative. Management of VMs and of cloud services such as storage, databases and message queues is needed to track what services are used. Governance is crucial to ensure that policies and government regulations are followed wherever cloud computing is used. Other governance requirements will be industry- and geography-specific.
Security
Any use case involving an enterprise will have more sophisticated security requirements than one involving a single end user. Similarly, the more advanced enterprise use cases to follow will have equally more advanced security requirements.
Common File Format for VMs
A VM created for one cloud vendor’s platform should be portable to another vendor’s platform. Any solution to this requirement must account for differences in the ways cloud vendors attach storage to virtual machines.
Common APIs for Cloud Storage and Middleware
The enterprise use cases require common APIs for access to cloud storage services, cloud databases, and other cloud middleware services such as message queues. Writing custom code that works only for a particular vendor’s cloud service locks the enterprise into that vendor’s system and eliminates some of the financial benefits and flexibility that cloud computing provides.
Data and Application Federation
Enterprise applications need to combine data from multiple cloud-based sources, and they need to coordinate the activities of applications running in different clouds.
SLAs and Benchmarks
In addition to the basic SLAs required by end users, enterprises who sign contracts based on SLAs will need a standard way of benchmarking performance. There must be an unambiguous way of defining what a cloud provider will deliver, and there must be an unambiguous way of measuring what was actually delivered.
Lifecycle Management
Enterprises must be able to manage the lifecycle of applications and documents. This requirement includes versioning of applications and the retention and destruction of data. Discovery is a major issue for many organizations. There are substantial legal liabilities if certain data is no longer available. In addition to data retention, in some cases an enterprise will want to make sure data is destroyed at some point.
Deployment
It should be simple to build a VM image and deploy it to the cloud as necessary. When that VM image is built, it should be possible to move that image from one cloud provider to another, compensating for the different mechanisms vendors have for attaching storage to VMs. Deployment of applications to the cloud should be straightforward as well.
Industry-specific standards and protocols
Many cloud computing solutions between enterprises will use existing standards such as RosettaNet or OAGIS. The applicable standards will vary from one application to the next and from one industry to the next.
Interoperability
Because more than one enterprise is involved, interoperability between the enterprises is essential.
Standards
Taxonomy
Across Cloud Services
As cloud computing becomes more common, applications will likely use different types of cloud services. An application might use a cloud storage service, a cloud message queue, and manage (start/stop/monitor) virtual machines running in the cloud. Standards to define how these different services work together should provide value.
Within Cloud Services
As cloud computing becomes more common, applications will likely use different types of cloud services. An application might use a cloud storage service, a cloud message queue, and manage (start/stop/monitor) virtual machines running in the cloud. Standards to define how these different services work together should provide value.
Between the Cloud and Enterprise
Even as cloud computing emerges, enterprise architectures such as Java EE are not going away. Standards that define how an enterprise application communicates with resources such as a cloud database or a cloud message queue would enable those applications to use cloud services with little or no changes. Figuring out how to integrate cloud computing with existing architectures and development paradigms will be a major challenge for this group.
Within an Enterprise
Standards within an enterprise will be determined by requirements such as interoperability, auditability, security and management, and will build upon the standards that apply between enterprises and the cloud. The enterprise will interact with some combination of private, public and hybrid clouds.
SOA
Cloud Computing
Enterprise Security
Mobile handheld application development
Virtualization
API
Levels
The Wire
At this level, the developer writes directly to the wire format of the request. If the service is REST-based, the developer creates the appropriate HTTP headers, creates the payload for the request, and opens an HTTP connection to the service. The REST service returns data with an accompanying HTTP response code. Because of the straightforward nature of many REST services, it is possible to be relatively efficient while writing code at this level. If the service is SOAP-based, the developer creates the SOAP envelope, adds the appropriate SOAP headers, and fills the body of the SOAP envelope with the data payload. The SOAP service responds with a SOAP envelope that contains the results of the request. Working with SOAP services requires parsing the XML content of the envelopes; for that reason, most SOAP services are invoked with a higher-level API.
Language-specific Toolkits
Developers at this level use a languagespecific toolkit to work with SOAP or REST requests. Although developers are still focused on the format and structure of the data going across the wire, many of the details (handling response codes and calculating signatures, for example) are handled by the toolkit.
Service-specific Toolkit
The developer uses a higher-level toolkit to work with a particular service. Working at this level, the developer is able to focus on business objects and business processes. A developer can be far more productive when focusing on the data and processes that matter to the organization instead of focusing on the wire protocol.
Service-neutral Toolkit
This is the highest level of API. A developer working at this level uses a common interface to multiple cloud computing providers. As with Level 3, the developer focuses on business objects and business processes. Unlike Level 3, a developer working at Level 4 does not have to worry about which cloud service they are working with. An application written with a service-neutral toolkit should be able to use a different cloud vendor (see Changing Cloud Vendors on page 27) with minimal changes to the code, if any.
Categories
Ordinary PRogramming
The usual application programming interfaces in C#, PHP, Java, etc. There is nothing cloud-specific in this category.
Deployment
Programming interfaces to deploy applications to the cloud. In addition to any cloud-specific packaging technologies, this includes traditional packaging mechanisms such as .Net assemblies and EAR/WAR files.
Cloud Services
Programming interfaces that work with cloud services. As discussed in the previous section, cloud service APIs can be either service-specific or service-neutral. These APIs are divided into subcategories for cloud storage services, cloud databases, cloud message queues, and other cloud services. A developer writing code using cloud services APIs is aware that they are using the cloud.
Image and Infrastructure Management
Programming interfaces to manage virtual machine images and infrastructure details. APIs for images support uploading, deploying starting, stopping, restarting, and deleting images. Infrastructure management APIs control details such as firewalls, node management, network management and load balancing.
Internal Interfaces
Programming interfaces for the internal interfaces between the different parts of a cloud infrastructure. These are the APIs you would use if you wanted to change vendors for the storage layer in your cloud architecture.
Security Controls
Cryptography: Key and Certificate Managemnt
KMIP, the OASIS Key Management Interoperability Protocol (http://www.oasis-open.org/committees/kmip/)
Data/Storage Security
It must be possible to store data in an encrypted format. In addition, some consumers will need their data to be stored separately from other consumers' data.
Identity, Roles, Access Control and Attributes
SAML, the OASIS Security Assertion Markup Language (http://saml.xml.org/) X.509 Certificates, part of the ITU Public Key and Attribute Certificate Frameworks Recommendations (http://www.itu.int/rec/T-REC-X.509)
Security Policies
XACML, the OASIS eXtensible Access Control Markup Language (http://www.oasis-open.org/committees/xacml/)
Workload and Service Management
SPML, the OASIS Service Provisioning Markup Language (http://www.oasis-open.org/committees/provision/)
Interoperability
http://code.google.com/p/cloud-interop/
Storage
ThriftStore
PySector
SectorJNI
Compute
Sector File System for Hadoop
PySphere
Additional Guidance
NIF Guidance
Specialized Frameworks Guidance
Application capability
Community of Interest capability
Integration capability
Communications capability
Information capability
Information assurance capability
System and network control capability
Expert advice
Lessons Learned
SOA is foundational
Match cloud deployment & delivery model
Standards are nacent & evolving
Mobile devices are critical application entry point
Appropriate cloud service identification required for sucessful business model
Cloud Computing not appropriate for all applications
Pay attention to business and culture in security model
Standard enterprise architecture, security and implementation still required
Constraints
Real-time access to data and cloud services
Real-time dynamic brokers
Certification and accreditqation
Federated cloud architectures
Legal & Cultural issures
Related Patterns
Published
Legacy Services Capability Pattern
Disconnected, Intermittent, Limited (DIL) Communications Management Pattern
Design Phase Service Integration
All Hazards Alert and Warning
Secure Formatted Information Exchange Gateway
Core Network Access
Space, Air, Ground, Maritime Mobile Networks
Information Dissemination and Shared Database
Proposed/Required/Under Discussion
Known Uses
Potential Capability
References
• Federal Cloud Computing Initiative wiki: http://federalcloudcomputing.wik.is/ • Software as a Service, Using Cloud Computing to Achieve Business Agility : 2009-05-01, ISBN: 0595382479, Greer, Melvin B • NCOIC Cloud Computing Working Group: https://www.ncoic.org/home/ Cloud Computing Use Cases A white paper produced by the Cloud Computing Use Case Discussion Group Version 3.0 2 February 2010
Verification
Terms
Approaches
1. A certificate under the hand of a responsible officer of the certifying organization that claims verification for all requirements (as a whole) 2. A certificate under the hand of a responsible officer of the certifying organization that it has verification with each requirement 3. Objective Evidence that confirms that the certifying organization has validated verification with each requirement. This may be a freeform report or preferably one in the approved NCOIC format 4. A test specification describing how verification will be tested for each specified behaviors 5. A full and approved Verification & Validation Program involving tracing of requirements from inception to delivery through, design, manufacture and testing 6. An approved test – an exchange with a Majjic participant with an official report from the participant confirming the success of the experiment. – an already certified building block with an official report from the participant confirming the success of the experiment. – former and next Majjic experiments participation with an official Majjic/NATO or nation report confirming the success of the experiment.
Meeting Requirements
The requirements fall into three categories 1. Those requiring the presence of a function. 2. Those requiring conformance to a standard called out within this pattern. 3. Those requiring conformance to a measurable behaviour specified within this pattern. In the case of 2) and 3) there will be provided either in the standard or within this pattern • A definition of an externally observable metric or indicator for the behaviour. • An unambiguous specification of what the range of values of the metric or indicator must be to be considered conformant. Conformance can only be claimed if each and every requirement imposed or selected above in Paragraph _____ has been addressed, either by examination or inspection. It is noted that conformance evidenced at a higher level of the steps specified in Paragraph 3.2 will create more confidence in customers than that at a lower level. NCOIC reserve the right to demand that the level of conformance be declared publicly, whenever conformance is so declared.
Building Block Conformance Requirements
Topic
Floating Topic
Hybrid Cloud Computing Pattern
Added: 2010-03-03 12:36:02
From: (Joined 2010-02-14 20:10:07)
318 views |110 downloads
Hybrid Cloud Computing Pattern
More From:
