Cloud Driver

2019独角兽企业重金招聘Python工程师标准>>>

Cloudify cloud driver 是基于云环境的Cloufify抽像层。为Cloudify提供云基础设施接口。为Cloudify运行应用按需提供计算资源。

1.计算资源的提供

Cloudify使用agentless-installation方式安装Cloudify，即在安装过程中不需要在分配的机器中预装任何的Cloudify软件（如，agent）。Cloudify controller只要通过SSH与新分配的机器简单的建立连接，再安装需要的Cloudify组件并连接到它的Cloudify集群。

因此，除了满足最小需求的镜像，不需要特别的机器镜像。可能使用任何的*nix或Windows镜像，可以是虚拟机也可以是物理机。另外提供者也不一定要是真是的云，可以使用任意的一些非虚拟化的主机，创建BYON(Bring Your Own Node)或“local cloud”环境。

2.Cloud Driver什么时候使用

Cloud driver被以下组件加载和使用：

1. Cloudify Shell在使用以下命令运行或撤消，以供应或取消供应machine时，加载一个cloudify driver的实例。

• bootstrap-cloud, teardown-cloud (management machines)
• install-application, uninstall-application (application machines)

2. ESM（Elastic Service Manager）会在每次服务部署时加载一个cloud driver实例。每个实例负责对特定服务进行横向扩展或收缩。在集群中有且只有一个ESM实例，该ESM运行0个或多个cloud driver实例。

注：只有在bootstrapping之前修改cloud driver 才能生效。

3.Cloud Driver如何工作

cloud driver是一个通过文件配置的Java POJO对象。当定义一个cloud的配置文件时，就必须在其中的cloud driver配置文件中定义它的类名。(<cloudifyRoot>/tools/cli/plugins/esc/<cloudName>/<cloudName>-cloud.groovy).Cloudify会根据需要实例化这个类（使用无参构造访求），并且委派它来实现横向扩展或收缩的请求。

• 模向扩展（scaling-out）：cloud driver负责分配machine，验证SSH是否打开，并向Cloudify返回machine的明细。明细包括访问machine需要的主机IP地址和登录认证信息。
• 横向收缩（scaling-in）：cloud driver负责关闭machine，并将其释放到可用machines池中。在虚拟环境中（包括大部分Iaas clouds），这意味着直接关闭虚拟机本身。相反，在物理机环境下，这意味着要么关闭机器本身，或是简单的在该机器上停止Cloudify agent的运行。

4.供应错误处理

云提供者和APIs并不完美。在一个大规模分布式环境中，可能会发生错误。重要的是要记住cloud driver基础设施并不是“事务性”的。cloud driver实现的是负责对来自云环境的错误进行妥善处理。最重要的是，在供应一个machine时发生了错误，cloud driver需负责正确地释放资源。例如，如果cloud driver请求了一个machine，然后等待machine变到可用状态的过程中，这个machine可能会花过久的时间来启动，这就可能发生请求超时。这种情况下，cloud driver需要在cloud driver抛出TimeoutException之前关闭该machine。注：当TimeoutException发生时，Cloudify会重新发出启动一个新machine的请求。

5.Cloud Driver API

cloud driver实现类需要实现org.cloudifysource.esc.driver.provisioning.ProvisioningDriver接口。

1 package org.cloudifysource.esc.driver.provisioning; 2 3 4 public interface ProvisioningDriver { 5 6 /** ************ 7 * Adds a new ProvisioningDriverListner to the ProvisioningDriver. 8 * 9 * @param pdl A class that implements ProvisioningDriverListner. 10 */ 11 void addListener(ProvisioningDriverListener pdl); 12 /** ************ 13 * Passes a configuration map for all setting defined for this cloud. 14 * @param cloudTemplate 15 * @param cloud 16 * 17 * @param config The configuration settings. 18 */ 19 void setConfig(Cloud cloud, String cloudTemplate, boolean management); 20 21 /** ************ 22 * Passes an Admin API object that can be used to query the current cluster state. 23 * IMPORTANT: do not perform any blocking operations on this Admin instance, 24 * 25 * @param config The configuration settings. 26 */ 27 void setAdmin(Admin admin); 28 29 30 /** ************* 31 * Starts an additional machine on the cloud to scale out this specific service. 32 * 33 * @param duration Time duration to wait for the instance. 34 * @param unit Time unit to wait for the instance. 35 * @return The details of the started instance. 36 * @throws TimeoutException In case the instance was not started in the allotted time. 37 * @throws CloudProvisioningException If a problem was encountered while starting the machine. 38 */ 39 MachineDetails startMachine( long duration, TimeUnit unit) throws TimeoutException, CloudProvisioningException; 40 41 /** **************** 42 * Start the management machines for this cluster. 43 * 44 * @param duration timeout duration. 45 * @param unit timeout unit. 46 * @return The created machine details. 47 * @throws TimeoutException If creating the new machines exceeded the given timeout. 48 * @throws CloudProvisioningException If the machines needed for management could not be provisioned. 49 */ 50 MachineDetails[] startManagementMachines( long duration, TimeUnit unit) throws TimeoutException, CloudProvisioningException; 51 52 53 /** ************** 54 * Stops a specific machine for scaling in or shutting down a specific service. 55 * @throws CloudProvisioningException 56 */ 57 boolean stopMachine( final String machineIp, final long duration, final TimeUnit unit) throws InterruptedException, TimeoutException, CloudProvisioningException; 58 59 /** *********** 60 * Stops the management machines. 61 * 62 * @throws TimeoutException in case the stop operation exceeded the given timeout. 63 * @throws CloudProvisioningException If the stop operation failed. 64 */ 65 void stopManagementMachines() throws TimeoutException, CloudProvisioningException; 66 67 /** ********** 68 * Returns the name of this cloud. 69 * @return the name of the cloud. 70 */ 71 String getCloudName(); 72 73 /** *********** 74 * Called when the service that this provisioning implementation is responsible for scaling 75 * is undeployed. The implementation is expected to release/close all relevant resources, 76 * such as thread pools, sockets, files, etc. 77 */ 78 void close(); 79 }

6.Cloud Driver配置文件

cloud driver的实现类，及任何必需的云配置信息，通过在基于Groovy DSL的配置文件中指定。这些信息用于填充对应实体类org.cloudifysource.dsl.cloud.Cloud的实例对象。这个对象用于调用cloud driver实例的setConfig()方法传参。

配置文件示例：

1 cloud { 2 // Mandatory. The name of the cloud, as it will appear in the Cloudify UI. 3 name = " ec2 " 4 5 /** ****** 6 * General configuration information about the cloud driver implementation. 7 */ 8 configuration { 9 // Optional. The cloud implementation class. Defaults to the built in jclouds-based provisioning driver. 10 className " org.cloudifysource.esc.driver.provisioning.jclouds.DefaultProvisioningDriver " 11 // Optional. The template name for the management machines. Defaults to the first template in the templates section below. 12 managementMachineTemplate " SMALL_LINUX_32 " 13 // Optional. Indicates whether internal cluster communications should use the machine private IP. Defaults to true. 14 connectToPrivateIp true 15 } 16 17 /** *********** 18 * Provider specific information. 19 */ 20 provider { 21 // Mandatory. The name of the cloud provider. 22 // When using the default cloud driver, maps to the Compute Service Context provider name. 23 provider " aws-ec2 " 24 25 // Mandatory. All files from this LOCAL directory will be copied to the remote machine directory. 26 localDirectory " tools/cli/plugins/esc/ec2/upload " 27 // Mandatory. Files from the local directory will be copied to this directory on the remote machine. 28 remoteDirectory " /home/ec2-user/gs-files " 29 // Mandatory. The HTTP/S URL where cloudify can be downloaded from by newly started machines. 30 // Mandatory. The prefix for new machines started for services. 31 machineNamePrefix " cloudify_agent_ " 32 // Optional. Defaults to true. Specifies whether cloudify should try to deploy services on the management machine. 33 // Do not change this unless you know EXACTLY what you are doing. 34 dedicatedManagementMachines true 35 36 // 37 managementOnlyFiles ([]) 38 39 // Optional. Logging level for the internal cloud provider logger. Defaults to INFO. 40 sshLoggingLevel " WARNING " 41 42 // Mandatory. Name of the new machine/s started as cloudify management machines. 43 managementGroup " cloudify_manager " 44 // Mandatory. Number of management machines to start on bootstrap-cloud. In production, should be 2. Can be 1 for dev. 45 numberOfManagementMachines 1 46 zones ([ " agent " ]) 47 48 /* The estimated amount of RAM used 49 by the operating system and the GSA running on the machine */ 50 reservedMemoryCapacityPerMachineInMB 1024 51 52 } 53 54 /** *********** 55 * Cloud authentication information 56 */ 57 user { 58 // Optional. Identity used to access cloud. 59 // When used with the default driver, maps to the identity used to create the ComputeServiceContext. 60 user " ENTER_USER " 61 62 // Optional. Key used to access cloud. 63 // When used with the default driver, maps to the credential used to create the ComputeServiceContext. 64 apiKey " ENTER_API_KEY " 65 66 67 68 } 69 70 71 /** ********* 72 * Cloud machine templates available with this cloud. 73 */ 74 templates ([ 75 // Mandatory. Template Name. 76 SMALL_LINUX_32 : template{ 77 // Mandatory. Image ID. 78 imageId " us-east-1/ami-76f0061f " 79 // Mandatory. Amount of RAM available to a machine. 80 machineMemoryMB 1600 81 // Mandatory. Hardware ID. 82 hardwareId " m1.small " 83 // Optional. Location ID. 84 locationId " us-east-1 " 85 localDirectory " upload " 86 remoteDirectory " /home/ec2-user/gs-files " 87 username " REPLACE_WITH_THE_SSH_USER_NAME " 88 password " REPLACE_WITH_THE_SSH_USER_PASSWORD " 89 90 // Additional template options. 91 // When used with the default driver, the options names are considered 92 // method names invoked on the TemplateOptions object with the value as the parameter. 93 keyFile " cloud-demo.pem " 94 options ([ 95 " securityGroups " : [ " default " ] as String[], 96 " keyPair " : " cloud-demo " 97 ]) 98 99 // Optional. Overrides to default cloud driver behavior. 100 // When used with the default driver, maps to the overriding properties passed to the ComputeServiceContext a 101 overrides ([:]) 102 103 privileged true 104 105 /* 106 The following optional attribute contains the path from which the JDK will be downloaded. 107 Default Linux values are : 108 32 bit 109 http://repository.cloudifysource.org/com/oracle/java/1.6.0_32/jdk-6u32-linux-i586.bin 110 64 bit 111 http://repository.cloudifysource.org/com/oracle/java/1.6.0_32/jdk-6u32-linux-x64.bin 112 But you can set it manually as follows : 113 (If you want to rely on the pre-installed JDK, set javaUrl to "NO_INSTALL") 114 */ 115 javaUrl " http://my-pc:8080/java.bin " 116 } 117 ]) 118 119 120 /** *************** 121 * Optional. Custom properties used to extend existing drivers or create new ones. 122 */ 123 custom ([:]) 124 }

6.1设置 Java download path

在默认情况下，Cloudify会从以下地方下载JAVA:

• http://repository.cloudifysource.org/com/oracle/java/1.6.0_32/jdk-6u32-linux-i586.bin (32 bit)
• http://repository.cloudifysource.org/com/oracle/java/1.6.0_32/jdk-6u32-linux-x64.bin (64 bit).

然而，如果已经在VM中预装了JDK，则可以设置javaUrl属性值为”NO_INSTALL“。也可指定到其它下载位置。如下示例：

templates ([

SMALL_LINUX_32 : template{

...

javaUrl "NO_INSTALL"

//javaUrl "http://192.168.0.4:8090/cloudify/jdk-6u32-linux-i586.bin"

...

}

...

])

6.2设置VM的系统变量

在templates的env区域，可指定任何系统的环境变量，例如：

SMALL_LINUX : template {

...

/* The following environment variables (ANT_OPTS and JAVA_OPTS)

will seep into Cloudify's processes

and will be known to every script, Java process, ant process

and life cycle event handler. */

env ([

"ANT_OPTS" : "-Dhttp.proxyHost=proxy -Dhttp.proxyPort=8080" ,

"JAVA_OPTS" : "-Xmx512m"

])

...

}

6.3CloudDriver 配置文件的参数

可在cloud driver DSL中应用Groovy GString表达式。也就是说，可以通过一个单的命名约定好的properties文件定义变量来设定DSL中的值。例如：

# Set the following values in ec2-cloud.properties

 

vmPrefix="REPLACE_WITH_VM_PREFIX"

userName="REPLACE_WITH_SSH_USERNAME"

apiKeyStr="REPLACE_WITH_API_KEY"

keyPairStr="REPLACE_WITH_KEYPAIR_NAME"

 

ec2-cloud.groovy:

cloud {

...

provider {

...

/* If vmPrefix , does NOT exist in ec2-cloud.properties,

then an empty string will be used.*/

machineNamePrefix vmPrefix + "_cloudify_agent_"

...

}

user {

/* If userName, does NOT exist in ec2-cloud.properties,

then an empty string will be used.*/

user userName

/* If apiKeyStr, does NOT exist in ec2-cloud.properties,

then an empty string will be used.*/

apiKey apiKeyStr

}

templates ([

SMALL_LINUX : template{

...

/* If keyPairStr, does NOT exist in ec2-cloud.properties,

then an empty string will be used.*/

keyFile keyPairStr + ".pem"

...

options ([

"securityGroups" : ["default"] as String[],

/* If keyPairStr, does NOT exist in ec2-cloud.properties,

then an empty string will be used.*/

"keyPair" : keyPairStr

])

...

},

...

])

...

}

注：这里的properties文件和groovy需要同名。

6.4定制脚本或代码

可以直接在template中initilizationCommand中编写代码，或指向上传的代码进行执行。

SMALL_LINUX : template{

...

/* The following command will be executed after Java and Cloudify are installed,

but before Cloudify is started.

*/

initializationCommand "echo this is an init command"

}

 

/* example #2 */

SMALL_LINUX : template{

...

/* The following command will be executed after Java and Cloudify are installed,

but before Cloudify is started.

In the following example, myscript.sh should be placed in the same folder

as the bootstrap-management.sh - (usually the upload folder).

*/

initializationCommand "chmod +x myscript.sh; ./myscript.sh"

}

6.5添加 templates

在Bootstrapping process完成后，可能需要在云中添加更多的templates。

通过CLI命令：add-templates [directory or file path]

• 如果path是一个groovy文件，那么这个文件应该命名为*-template.groovy，并且在它的路径中有且只有一个该后缀文件。这个路径中的文件都将随该groovy文件一起拷贝到management machine中。
• 如果path是一个文件夹或Zip文件，那么这个文件夹（或Zip解压出来的文件夹）应当至少有一个template文件（一个以-template.groovy为后缀的文件）和template需要的所有文件（e.g. the upload folder）。

所有的template froovy文件不是直接使用template命名，而是将其作为前缀：<template name>-template.groovy。例如，”linux-template.groovy“将变成”SMALL_LINUX-template.groovy“。properities和overrides文件也将采用相同的命名方式重命名。

示例：

[

SMALL_LINUX : template {

imageId "OpenLogic__OpenLogic-CentOS-62-20120531-en-us-30GB.vhd"

machineMemoryMB 1600

hardwareId "Small"

localDirectory "upload"

username "SMALL_LINUX"

password "ENTER_PASSWORD"

remoteDirectory "/home/ENTER_USER_NAME/gs-files"

}

]

注: 1.template必在中括号之中，将被转换成Map。

2.一个groovy文件中只允许一个template,如果有多个会出现异常，并且template不会被添加。

3.如果template已存在，不会被覆盖。

6.6Reserved Memory

reservedMemoryCapacityPerMachineInMB：在cloud driver的provider区域可设置此属性，为操作系统、运行在machine上的Cloudify agent和其它非Cloudify的应用程序（那些预装应用，并不是Cloudify或当前配方中的应用）估计RAM使用量。

machineMemoryMB：在cloud driver的templates区域，机器可用的RAM总量。

• 专用模式（dedicated mode, isolationSLA —— 每个machine对应一个Service实例):
  reservedMemoryCapacityPerMachineInMB值必须比template中定义machineMemoryMB值小。Cloudify将会使用剩下的内存容量满足服务实例计划容量。从而能够在新分配的machine安装实例。
• 多租用模式（Multi Tenancy modes）:
  服务实例的内存容量需求是由用户定义 (via isolationSLA), 因此用户必须给instanceMemoryMB 属性指定一个有效的数值，需同时考虑到reservedMemoryCapacityPerMachineInMB和machineMemoryMB，否则连安装单一实例的内存都不会有足够。

注：只有在多租用模式下才会使用reservedMemoryCapacityPerMachineInMB属性，因为在专用模式下，计划容量是由Cloudify计算出来的。

7.默认的Cloud Driver

Cloudify内置一个基于流行的jclouds框架的cloud driver实现。这个cloud driver支持了较多的cloud providers和APIs上部署Cloudify 服务。参考：支持列表

这个默认driver可以直接使用，或者基于此展以适用于需求。

转载于:https://my.oschina.net/hjswust/blog/121914