OpenStack platform, страница 2

Nova is such an important component of the OpenStack, that Nova developer teams meets weekly, they have their own ecosystem and gives a lot of significance to the code-review, saying - “Code reviews are the life blood of the Nova developer community”. They are also very fond of new developers, which are getting involved into the Nova project and tries to help them with everything they can.

1. Image Service (Glance)

Glance provides discovery, registration, and delivery services for disk and server images. Stored images can be used as a template for starting new instances (operating systems or containers with services) in future. In addition, templates can be created beforehand by administrators and been provided to the users. Glance can also be used to store and catalog an unlimited number of backups. The Image Service can store disk and server images in a variety of back-ends, including OpenStack Object Storage. Templates may have wide variety of formats: raw, vhd (hyper-V), VDI (Virtual Box (Oracle)), qcow2 (QEMU/KVM), VMDK (VMWare), OVF. The Image Service API providing a standard REST interface lets clients streaming the images between locations and servers.

Glance adds many enhancements to existing legacy infrastructures. For example, if integrated with VMware, Glance introduces advanced features to the vSphere family such as, vMotion. This feature is the live migration of a running VM, from one physical server to another, without service interruption. Thus, it enables a dynamic and automated self-optimizing datacenter

OpenStack's image is an operating system installed on a virtual machine (VM). If a developer adds a variation to an image (as a configuration job), the result is an instance of that image. Subsequently, that instance is an image that developers can add more variations to.

Glance - is a compute module, it does not store images, but holds their metadata from Swift (Object Storage) or a storage backend datastore. Other modules must communicate with the images metadata through Glance. Nova can change some information about the images, but all its commands will be addressed to Glance, Glance is the only module that can add, delete, share, or duplicate images.

1. Object Storage (Swift)

Object Storage is a scalable redundant storage system, a lot of storage data can be stored and retrieved with a simple API. Objects and files are written to multiple disk drives spread throughout servers in the data center, with the OpenStack software responsible for ensuring data replication and integrity across the cluster. Usually there is three copies of the data in each datacenter, but this value is configurable. It must be clear, that object storage is not an ordinary file system, but a storage having no central "brain" or master point.

Storage clusters scale horizontally simply by adding new servers. Should a server or hard drive fail, OpenStack replicates its content from other active nodes to new locations in the cluster. Because OpenStack uses software logic to ensure data replication and distribution across different devices, inexpensive unreliable hard drives and servers can be used. Cost efficiency is a very valuable parameter in datacenters.

Swift is built to be scalable and optimized for durability, availability and concurrency. Its main usage is for storing an unstructured data.

In August 2009, Rackspace started the development of the predecessor to Swift, as a complete replacement for the Cloud Files product. The initial development team consisted of nine developers. SwiftStack, an object storage software company, is currently the leading developer for Swift. This is a classic example of benefit gained from modular architecture and well-designed API.

1. Dashboard (Horizon)

OpenStack Dashboard provides administrators and users a graphical interface to access, manage, and automate cloud-based resources. The design permits third party products and services, such as billing, monitoring, and additional management tools.

The dashboard is one of several ways users can interact with OpenStack resources. Developers can automate access or build tools to manage resources using the native OpenStack API or the EC2 compatibility API (An API used by Amazon – the biggest hosting company in the world).

Horizon consists of three central dashboards, a “User Dashboard”, a “System Dashboard”, and a “Settings” dashboard. Between these three they cover the core OpenStack applications.

1. Identity Service (Keystone)

Keystone is an authentication system for the datacenter users and regulate user’s rights for different OpenStack services they can access. It acts as a common authentication system and can integrate with different backend services like LDAP for keeping its metadata. It supports different types of authentication: username and password credentials, token-based systems, ssh keys and Amazon Web Services logins (AWS-style logins).

Keystone can provide list of different services allowed to the user, so third-party programs can determine which services is allowed for them to use.

For administrator Keystone enables to configure users and systems policies, create users and grant them permissions relative to different OpenStack services. Users is allowed to manage resources to their discretion.

1. Networking (Neutron)

OpenStack Networking (Neutron) is a system for managing networks and IP addresses, in general, it provides networking as a service between interfaces, and enables other OpenStack components to configure itself. It is very unusual for old IT world to freely enable networking configuration, but it is as it is, in new IT innovations, it became perfectly normal. Neutron main mission is to implement network services and to provide scalable and technology-agnostic network abstraction.

OpenStack Networking enables to create different network topologies and provide different services for various applications or user groups. It is not based on VLANs as standard models, so is it easily configurable. OpenStack Networking manages IP addresses, allowing sticking static IP address to the machine or to assign dynamic IP address via DHCP. Dynamic IP addresses let traffic to be dynamically rerouted to any resources in the IT infrastructure.

Users can create their own networks, control traffic, connect servers and devices to one or more networks and configure advanced network polices. OpenStack Networking provides an extension framework that can be used to manage additional services such as intrusion detection systems (IDS), firewalls, virtual private networks (VPN), load balancing, etc.

Administrators can use software defined networking (SDN) technology like OpenFlow to support high levels of multi-tenancy and massive scale. OpenStack Neutron, with its plugin architecture, provides the ability to integrate SDN controllers into the OpenStack. This integration of SDN controllers into Neutron using plugins provides centralized management, and facilitates the network programmability. With this integration of OpenStack Neutron and SDN controllers, the changes to the network and network elements triggered by the OpenStack user, are translated into Neutron APIs, and handled by neutron plugins, applying the changes to interfaces.

The main benefit of the OpenStack Networking is that it is pluggable, scalable and API-driven, both: administrators and users can use it. Scalability is very important in datacenters in our days, because the number of IP addresses, routing configurations and security rules can quickly grow into the millions.

1. Block Storage (Cinder)

OpenStack Block Storage provides persistent block-level storage devices for use with OpenStack compute instances. The block storage system manages the creation, attaching and detaching of the block devices to different servers by means of different hypervisors. Block storage volumes are fully integrated into other OpenStack services allowing cloud users to manage their own storage needs by means of graphical interface or using console commands. In addition to local Linux server storage (root file system of the operating system, which is usually small in datacenters), it can use different storage platforms like Ceph, NetApp, Nexenta, SolidFire, xZadara, EMC (ScaleIO, VMAX and VNX), IBM Storage, SAN Volume Controller, Pure Storage, etc. Block storage is appropriate for performance sensitive operations, such as database storage or providing a server with access to raw block level storage. Snapshot management provides functionality for backing up data stored on block storage volumes and restore them.

Cinder has some main features, which makes it very useful in context of OpenStack:

1. Redundancy makes it very reliable, and in case of some breakage, the system will restore the data. User can configure the replication parameter. Therefore, it makes no need for special RAID 0-7 systems to be used any more.

2. No limits for the size of the storage. Several hard drives can be represented to the clients system as one single block device.

3. Absence of central database, leads to the scalability benefits.

4. Detection of drive failures, preventing data corruption.

5. Enable of direct access to the data from the browser.

6. Real-time monitoring of the client activities concerned the block storage.

7. The access to the block storage can be limited for different users.

In total, Cinder virtualizes pools of block storage devices and provides them to the end users without requiring any knowledge of where their storage is actually deployed or on what type of device.

1. Orchestration (Heat)

To understand the main idea of the Heat, firstly we must discus what is “orchestration”. Orchestration is the concept of unified coordination, management and handling exceptions of the complex computer system as a whole. It is something like implicitly autonomic control of the system. However, in reality, after taking into account technical details, orchestration is only the effect achieved by automation of the computer system processes. In cloud computing, orchestration consists of three parts:

1. Composing of architecture, tools and processes used to deliver a defined service

2. Stitching of software and hardware component to deliver a defined service

3. Connecting and automating of workflows to deliver a defined service

Heat is a service to orchestrate many different cloud applications using templates in form of the text files, through an OpenStack native REST API and a CloudFormation compatible Query API. To set up Heat you must describe the infrastructure in a text file, its resources and relationships between them (this will enable Heat to create the entire infrastructure in the correct order). As Heat always knows the infrastructure, it is capable of updating it in a correct way after changing the template. As an additional feature Heat configuration files is compatible with configuration management tools such as Puppet and Chef. It enables to write the unified configuration files for setting up the infrastructure and to set up applications inside infrastructure. Through integration with the Ceilometer (telemetry service), the orchestration engine can also perform auto-scaling of certain infrastructure elements.

Heat mission consist of creation a human and machine accessible service for managing the entire lifecycle of infrastructure and services within the cloud.

1. Telemetry (Ceilometer)

The OpenStack Telemetry Service (Ceilometer) aggregates usage and performance data across the services settled up in an OpenStack cloud. This enables monitoring the usage of the cloud across the OpenStack services and allows cloud operators to view metrics globally or by individual deployed resources.

This service is very important for the hosting companies, because it is used to establish customer billing. As this is directly connected to the money, it is very important for the telemetry to be reliable and faultless. Its purpose is to measure, monitor and alarm in case of fire (because nobody will read and notice this, and I will be a little bit surprised if anyone does). Ceilometer readouts are logged and can be viewed if necessary or if the customer will request the statistics of the resource usage, he must pay for.

1. Database (Trove)

Trove is a database as a service for OpenStack. It has two diagonal types of engines: relational and non-relational database. The service goal is to permit users quickly utilize database without necessity to handle complex administrative tasks. Of course, Trove focuses on high performance, while performing administration tasks: deployment, configuration, backups, restores and monitoring. Trove is still actively developed, and people are focused on improving its internal mechanisms.

1. Elastic Map Reduce (Sahara)

In this section, first of all, I would like to introduce you to the Hadoop clusters. Hadoop is a Map-Reduce distributed database system. It is based on the concept of two operations: map and reduce. This concept became very popular for the task of processing huge amount of data in an undertime. This concept widely used by such IT giants as Google, Yandex, and some other less known companies. Hadoop is one of the realization of the concept, brought into world by Apache. The Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly available service on top of a cluster of computers, each of which may be prone to failures.

As it is formulated in the main project: “Sahara aims to provide users with simple means to provision Hadoop clusters by specifying several parameters like Hadoop version, cluster topology, nodes hardware details and a few more”. After a user fills all the parameters, Sahara deploys the cluster in a few minutes; it also is capable of adding and removing worker nodes on demand.

Sahara’s architecture consists of “Vendor Plugins” – mechanism responsible for configuring and launching data processing units, Elastic Data Processing (EDP) – mechanism responsible for scheduling and managing data processing jobs and other Hadoop parts that are responsible for communication with other OpenStack components, such as Horizon, Keystone, Glance, etc.

1. Bare Metal Provisioning (Ironic)

Ironic is an integrated OpenStack project, which aims to use bare metal machines instead of virtual machines. It is best thought of as a bare metal hypervisor API and a set of plugins, which interact with the bare metal hypervisors. By default, it will use PXE (preboot execution environment (on the client side it requires only a PXE-capable network interface controller)) and IPMI (intelligent platform management interface (set of computer interface specifications for an autonomous management and monitoring independent from bare computer resources, such as CPU or firmware)) in concert to provision and turn on/off machines. Ironic also supports vendor-specific plugins, which may implement additional functionality.

In real world this service is usually not used, because of the virtualization technology that is adjusted nearly everywhere. Virtualization gives some valuable benefits like scalability and that is why usually preferable.

There is an interesting thing about the Ironic. Because of the service purpose “bare metal”, the mascot “bear metal” was created, you can find the reference to the picture at the end ^[12].

1. Multiple Tenant Cloud Messaging (Zaqar)

Zaqar (used to be known as Marconi) is a multi-targeted cloud messaging service for web developers. It combines the ideas pioneered by Amazon's SQS (Simple Queue Service) product with additional semantics to support event broadcasting apart from the other services aspects like reliable, simple, scalable, secure and inexpensive firewall-friendly messaging from one pier to another. Underlying the most important properties: security and scalability is very important for web applications, because it must obtain agility and hacker resistance. All these requests lead to next design principles: component-based architecture (enables to add new abilities operatively), availability and scalability, fault tolerance, recoverability (failures is easy to debug and repair), open standards (important step for security).