Python APIs: The best-kept secret of OpenStack

Writing OpenStack automation scripts with Python bindings

Lorin Hochstein (, Lead Architect, Cloud Services, Nimbis Services

Summary:  As an OpenStack user or administrator, you often need to write scripts to automate common tasks. In addition to the REST and command-line interfaces, OpenStack exposes native Python API bindings. Learn how to use these Python bindings to greatly simplify the process of writing OpenStack automation scripts.

OpenStack is an increasingly popular open source solution for deploying Infrastructure as a Service (IaaS) clouds. OpenStack ships with a dashboard web app that works well for performing manual tasks, such as launching a single virtual machine (VM) instance, but if you want to automate your cloud-based tasks, you'll need to write scripts that can drive OpenStack.

Services in OpenStack

The term service in OpenStack is overloaded. It is used to refer to:

  • An OpenStack project (for example, Compute Service (nova), Identity Service (keystone)
  • An entry in the Identity Service catalog (for example, image, compute, volume)
  • A Linux® daemon (for example, nova-api,quantum-l3-agent)

An OpenStack project is associated with one or more entries in the Identity Service catalog and is implemented by one or more Linux daemons. In the context of this article, I use service to mean OpenStack project.

Many users write either automation scripts directly against the OpenStack Representational State Transfer (REST) application programming interface (API) or shell scripts that invoke the command-line tools (for example, keystone or nova). But a better way exists to write OpenStack automation scripts in Python. All of the OpenStack services expose native Python APIs that expose the same feature set as the command-line tools. Unfortunately, not much documentation is available to describe how to use these APIs.

If you're a Python programmer, the Python APIs are much simpler to work with than command-line tools or the REST API. In this article, I demonstrate how you can use the native OpenStack Python APIs to automate common user and administrative tasks.

OpenStack projects and code names

The term OpenStack doesn't refer to a single application. Rather, it's a collection of services that work in concert to implement an IaaS cloud. (See the sidebar Services in OpenStack for what service means here.) Each OpenStack service has an official name and a code name, as shown in Table 1, and every OpenStack service exposes its own Python API.

Table 1. OpenStack services and code names
Official name Code name
Identity Service keystone
Image Service glance
Compute Service nova
Networking Service quantum
Block Storage Service cinder
Object Storage Service swift

Installing the Python bindings

The Python bindings are bundled with the command-line tools for each service. In fact, each command-line tool is implemented using the corresponding Python API. You can install each tool from the Python Package Index (PyPi—seeResources for a link) by using pip, a Python package installer. The pip package names are:

  • python-keystoneclient
  • python-glanceclient
  • python-novaclient
  • python-quantumclient
  • python-cinderclient
  • python-swiftclient

For example, to install the keystone client, run the following command:

$ pip install python-keystoneclient

You can install these packages into a Python virtualenv or into your system-wide Python packages, if you have root privileges on your local machine.

All of the OpenStack APIs are versioned, and the Python bindings support multiple API versions to maintain backwards compatibility. As a result, it's safe to download the latest version of these packages, because they will work with all older versions of OpenStack services.

In this article, I focus on Python API examples from the following services:

  • OpenStack Identity Service (keystone)
  • OpenStack Image Service (glance)
  • OpenStack Compute Service (nova)

Setting up a test environment

To get the most out of this article, I recommend that you have access to an OpenStack cloud with administrator privileges so that you can try the code snippets. If you don't currently have admin access to an OpenStack cloud, the simplest thing to do is deploy OpenStack inside a VM. The DevStack project (see Resources for a link) was designed to makes it simple to create a development-oriented deployment of OpenStack on a single machine. Paired with a virtualization tool like VirtualBox, you can bring up an OpenStack cloud on your laptop—even on Mac or Windows®.

You can also get a free account on TryStack, the community-maintained OpenStack sandbox (see Resources). Note that you can only get user-level privileges on TryStack, not admin-level privileges, so you won't be able to use TryStack to test any scripts that require admin privileges.

OpenStack Identity (keystone)

A client makes requests against the Identity (keystone) API by instantiating the appropriate keystone client Python object and calling its methods. Because the APIs are versioned, the Python clients are always associated with a specific version of the API.

Listing 1 shows an example of using version 2.0 of the keystone client to add the Image Service to the service catalog.

Listing 1. Creating an admin role with keystone
import keystoneclient.v2_0.client as ksclient
# Replace the method arguments with the ones from your local config
keystone = ksclient.Client(auth_url="",
glance_service ="glance",
                            description="OpenStack Image Service")


You must supply credentials when instantiating a keystoneclient.v2_0.client.Client object. A keystone endpoint accepts two types of credentials: a token or a user name and password. If you are an administrator, you can use the admin token, which is a special token that has administrator privileges and never expires. You define this token with the admin_tokenconfiguration option in the /etc/keystone/keystone.conf file on the machine that runs the keystone service (see Listing 2).

Listing 2. Authenticating with the auth token
import keystoneclient.v2_0.client as ksclient

# Replace the values below with the ones from your local config
endpoint = ""
admin_token = "devstack"

keystone = ksclient.Client(endpoint=endpoint, token=admin_token)

Using the admin token is generally frowned upon for security reasons. Instead, the OpenStack Identity developers recommend that you always use a user name and password for authentication after you have created a user who has admin privileges (see Listing 3).

Listing 3. Authenticating with a user name and password
import keystoneclient.v2_0.client as ksclient

# Replace the values below  the ones from your local config,
auth_url = ""
username = "admin"
password = "devstack"
tenant_name = "demo"

keystone = ksclient.Client(auth_url=auth_url, username=username,
                           password=password, tenant_name=tenant_name)

Loading an openrc file

To simplify authentication, I recommend that you create an openrc file that exports the credentials to environment variables. Doing so allows you to avoid hard-coding login information into your scripts. Listing 4 shows an example openrc file.

Listing 4. Loading credentials from environment variables
export OS_USERNAME="myname"
export OS_PASSWORD="mypassword"
export OS_TENANT_NAME="mytenant"
export OS_AUTH_URL=""

The environment variables OS_USERNAME, OS_PASSWORD, OS_TENANT_NAME, and OS_AUTH_URL are standardized across all of the Python command-line tools. If these environment variables are set, the command-line tools (keystonenova) will use them to authenticate against their API endpoints.

Load these environment variables into your current shell with the Bash source built-in command. If you use Bash as your standard shell, you may want to add this line to your .profile so that the environment variables are automatically set each time you log in:

$ source openrc

When the openrc file has been sourced, Python scripts can retrieve the credentials from the environment. Let's create a Python file called, as shown in Listing 5, that extracts the login information from the environment. Note thatkeystone and nova use slightly different variable names in their client initializer methods, so I define separate functions for each.

Listing 5.
#!/usr/bin/env python
import os

def get_keystone_creds():
    d = {}
    d['username'] = os.environ['OS_USERNAME']
    d['password'] = os.environ['OS_PASSWORD']
    d['auth_url'] = os.environ['OS_AUTH_URL']
    d['tenant_name'] = os.environ['OS_TENANT_NAME']
    return d

def get_nova_creds():
    d = {}
    d['username'] = os.environ['OS_USERNAME']
    d['api_key'] = os.environ['OS_PASSWORD']
    d['auth_url'] = os.environ['OS_AUTH_URL']
    d['project_id'] = os.environ['OS_TENANT_NAME']
    return d

Authentication tokens

If the client initializer returns without throwing an exception, it has successfully authenticated against the endpoint. You can access the keystone token that you were just issued through the auth_token attribute of the returned object, as shown in Listing 6. When authenticating against the glance API, you will need to explicitly pass a keystone authentication token as an argument to the initializer, as discussed later.

Listing 6. Successfully authenticating against a keystone endpoint in an interactive Python session 
>>> import keystoneclient.v2_0.client as ksclient
>>> from credentials import get_keystone_creds
>>> creds = get_keystone_creds()
>>> keystone = ksclient.Client(**creds)
>>> keystone.auth_token

Note: The Grizzly release of OpenStack Identity uses Public Key Infrastructure tokens by default, which are much longer than the universally unique identifier tokens (for example, 7d9fde355f09458f8e97986a5a652bfe) used in previous releases of OpenStack.

CRUD operations

The keystone API is essentially a create, read, update, delete (CRUD) interface: Most interactions with the keystone API either read from the keystone back-end database or modify it. Most interactions with the API happen by making calls onManager objects. A Manager represents a collection of objects of the same type. For example, a UserManager manipulateskeystone users, a TenantManager manipulates tenants, a RoleManager manipulates roles, and so on. The managers support operations such as create (create a new object), get (retrieve an object by ID), list (retrieve all of the objects), anddelete.

Creating users, tenants, and roles

Typically, one of the first tasks you perform when deploying OpenStack is to create a keystone tenant, and then akeystone user with administrative privileges. Listing 7 shows an example of how to automate this process by using the Python API. The script performs the following tasks:

  • Create a user role (Client.roles.create).
  • Create an admin role (Client.roles.create).
  • Create a tenant named acme (Client.tenants.create).
  • Create a user named admin (Client.users.create).
  • Assign the admin user the admin role in the acme tenant (Client.roles.add_user_role).

This is a good scenario for using the admin token, because the Identity Service does not yet contain any users with administrative privileges.

Listing 7. Creating a user, tenant, and role
import keystoneclient.v2_0.client as ksclient
endpoint = ""
admin_token = "devstack"

keystone = ksclient.Client(endpoint=endpoint, token=admin_token)
user_role = keystone.roles.create("user")
admin_role = keystone.roles.create("admin")
acme_tenant = keystone.tenants.create(tenant_name="Acme",
                        description="Employees of Acme Corp.",
admin_user = keystone.users.create(name="admin",
keystone.roles.add_user_role(admin_user, admin_role, acme_tenant)

Creating services and endpoints

Typically, the next task in deploying the Identity Service in an OpenStack cloud is populating keystone with the services in the cloud and the endpoints. Listing 8 shows an example of how to add a service and an endpoint for the Identity Service using the and Client.endpoints.create methods.

Listing 8. Creating a service and endpoint
import keystoneclient.v2_0.client as ksclient
creds = get_keystone_creds() # See xxx
keystone = ksclient.Client(**creds)
service ="keystone",
                                   description="OpenStack Identity Service")

keystone_publicurl = ""
keystone_adminurl = ""

Accessing the service catalog

One of keystone's primary functions is to serve as a service catalog. Clients can use keystone to look up the endpoint URL of an OpenStack service. The API exposes this functionality through thekeystoneclient.v2_0.client.Client.service_catalog.url_for method. This method allows you to look up a service endpoint by its type (for example, image, volume, compute, network) and its endpoint type (publicURLinternalURL,adminURL).

Listing 9 demonstrates how to use the url_for method to retrieve the endpoint of the OpenStack Image (glance) Service.

Listing 9. Querying for the glance endpoint in an interactive Python session 
>>> import keystoneclient.v2_0.client as ksclient
>>> creds = get_keystone_creds() # See <a href="openrc-creds" />
>>> keystone = ksclient.Client(**creds)
>>> glance_endpoint = keystone.service_catalog.url_for(service_type='image',
>>> glance_endpoint

OpenStack Compute (nova)

Version 1.1 vs. version 2 of the nova API

Version 1.1 and version 2 of the nova API are identical. You can pass "2" instead of "1.1" as the first argument of the novaclient.client.Client initializer, but no novaclient.v2 module exists—only a novaclient.v1_1 module.

The OpenStack Compute (nova) Python API works similarly to the OpenStack Identity API. I use version 1.1 of the nova API here, so the classes in version 1.1 of the nova Python bindings I use in this article are in the novaclient.v1_1 Python namespace.

Authenticating against the nova-api endpoint

You make requests against the nova-api endpoint by instantiating a novaclient.v1_1.client.Client object and making calls against it. You can retrieve a client that communicates with version 1.1 of the API in two ways. Listing 10 demonstrates how to retrieve the appropriate client by passing the version string as an arguments.

Listing 10. Pass version as argument
from novaclient import client as novaclient
from credentials import get_nova_creds
creds = get_nova_creds()
nova = novaclient.Client("1.1", **creds)

Listing 11 demonstrates how to retrieve the appropriate client by importing the version 1.1 module explicitly.

Listing 11. Import version directly
import novaclient.v1_1.client as nvclient
from credentials import get_nova_creds
creds = get_nova_creds()
nova = nvclient.Client(**creds)

Listing instances

Use the Client.servers.list method to list the current VM instances, as shown in Listing 12.

Listing 12. Retrieve a list of VM instances in an interactive Python session 
>>> import novaclient.v1_1.client as nvclient
>>> from credentials import get_nova_creds
>>> creds = get_nova_creds()
>>> nova = nvclient.Client(**creds)
>>> nova.servers.list()
[<Server: cirros>]

Retrieving an instance by name and shutting it down

If you don't know the ID of an instance but you know its name, you can use the Server.find method. Listing 13 shows how to find an instance by name and terminate it using the Server.delete method.

Listing 13. Terminate the "my-vm" instance
import novaclient.v1_1.client as nvclient
from credentials import get_nova_creds
creds = get_nova_creds()
nova = nvclient.Client(**creds)

server = nova.servers.find(name="my-vm")

Booting an instance and checking status

To start up a new instance, you use the Client.servers.create method, as shown in Listing 14. Note that you must pass an image object and flavor object, not the names of the image and flavor. The example also uses theClient.keypairs.create method to upload the Secure Shell (SSH) public key at ~/.ssh/ and names the key pair mykey, assuming that key pair does not yet exist. Finally, it uses the Client.servers.get method to retrieve the current state of the instance, which it uses to poll for status.

Listing 14. Booting a new instance
import os
import time
import novaclient.v1_1.client as nvclient
from credentials import get_nova_creds
creds = get_nova_creds()
nova = nvclient.Client(**creds)
if not nova.keypairs.findall(name="mykey"):
    with open(os.path.expanduser('~/.ssh/')) as fpubkey:
image = nova.images.find(name="cirros")
flavor = nova.flavors.find(name="m1.tiny")
instance = nova.servers.create(name="test", image=image, flavor=flavor, key_name="mykey")

# Poll at 5 second intervals, until the status is no longer 'BUILD'
status = instance.status
while status == 'BUILD':
    # Retrieve the instance again so the status field updates
    instance = nova.servers.get(
    status = instance.status
print "status: %s" % status

Attaching a floating IP address

To attach a floating IP address, you must first verify that OpenStack has an available floating IP address. Use theClient.floating_ips.list method to retrieve a list of available floating IP addresses. If the resulting list is empty, allocate a new floating IP address by using the Client.floating_ips.create method, then assign it to the instance using theServer.add_floating_ip method, as shown in Listing 15.

Listing 15. Creating a floating IP address
>>> nova.floating_ips.list()
>>> floating_ip = nova.floating_ips.create()
<FloatingIP fixed_ip=None, id=1, instance_id=None, ip=, pool=public>
>>> instance = nova.servers.find(name="test")
>>> instance.add_floating_ip(floating_ip)

Changing a security group

Use the Client.security_group_rules.create method to add rules to a security group. Listing 16 shows an example that modifies the default security group to allow SSH (which runs on port 22) as well as all Internet Control Message Protocol (ICMP) traffic. To do so, I use the Client.security_groups.find method to retrieve the security group named default.

Listing 16. Allowing port 22 and ICMP in the default security group 
import novaclient.v1_1.client as nvclient
from credentials import get_nova_creds
creds = get_nova_creds()
nova = nvclient.Client(**creds)

secgroup = nova.security_groups.find(name="default")

Retrieving the console log

The Server.get_console_output method retrieves the text sent to the console when the VM boots up. You can parse the console output to work around a common issue with changing host keys.

One of the drawbacks of an IaaS cloud such as OpenStack is that it doesn't interoperate well with SSH host key checking. If you are logging in to an instance at, say,, and that IP address had been previously used by another instance that you had logged in to in the past, you'll get an error that looks like Listing 17.

Listing 17. Error when host ID has changed
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
Please contact your system administrator.
Add correct host key in /home/mylogin/.ssh/known_hosts to get rid of this message.
Offending key in /home/mylogin/.ssh/known_hosts:1
RSA host key for has changed and you have requested strict checking.
Host key verification failed.

If your VM image has the cloud-init package installed (see Resources), then it will output the host key to the console, as shown in Listing 18.

Listing 18. Example SSH key output in console
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDciNMyzj0osyPOM+
SdBJAwIoisRFPxyroNGiVw= root@my-name
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDU854+fNdcKMZTLCUejMOZllQmmphr6V5Aaz1F2+x2jXql5rqKQ
7EjwLqhIH/TUuzGQiWmFGvyz root@my-name

Listing 19 shows a script that uses the Server.get_console_output API method to extract the SSH host key from the console and update the ~/.ssh/known_hosts file so you don't get that SSH warning when you use SSH to access the floating IP instance the first time.

Listing 19. Extracting the SSH host key from the console
import os
import subprocess
import novaclient.v1_1.client as nvclient
from credentials import get_nova_creds

def get_server(creds, servername):
    nova = nvclient.Client(**creds)
    return nova.servers.find(name=servername)

def remove_hostkey(ip):["ssh-keygen", "-R", ip])

def get_hostkey_from_console(text):
    lines = text.split('\n')
    start = '-----BEGIN SSH HOST KEY KEYS-----\r'
    end = '-----END SSH HOST KEY KEYS-----\r'
    start_ind = lines.index(start)
    end_ind = lines.index(end)
    for i in range(start_ind+1, end_ind):
        key = lines[i].rstrip()
        if key.startswith('ssh-rsa'):
            return key
    raise KeyError("ssh host key not found")

def main():
    server = get_server(get_nova_creds(), "my-server")
    netname = "my-network"
    (fixed_ip, floating_ip) = server.networks[netname]
    # Remove existing key, if any
    output = server.get_console_output()
    key = get_hostkey_from_console(output)
    with open(os.path.expanduser("~/.ssh/known_hosts"), 'a') as f:
        f.write("{0} {1}\n".format(floating_ip, key))

if __name__ == '__main__':

OpenStack Image (glance)

The OpenStack Image Service (glance) is responsible for managing a catalog of VM images that the Compute Service uses.

Authenticating against the glance endpoint

The OpenStack Image (glance) Python API has some minor differences to the Compute API when doing the initial authentication. The glance API depends on information that it must obtain from the keystone API:

  • The glance endpoint URL
  • keystone authentication token

Like the nova API, with the glance API, you can pass the API version as an argument or import the module directly. Listing 20 shows an example of how to authenticate against a glance endpoint with version 2 of the API.

Listing 20. Authenticating with the glance API
import keystoneclient.v2_0.client as ksclient
import glanceclient
creds = get_keystone_creds()
keystone = ksclient.Client(**creds)
glance_endpoint = keystone.service_catalog.url_for(service_type='image',
glance = glanceclient.Client('2',glance_endpoint, token=keystone.auth_token)

Listing 21 shows an example in which the relevant glance module is imported directly.

Listing 21. Importing the glance module directly
import keystoneclient.v2_0.client as ksclient
import glanceclient.v2.client as glclient
creds = get_keystone_creds()
keystone = ksclient.Client(**creds)
glance_endpoint = keystone.service_catalog.url_for(service_type='image',
glance = glclient.Client(glance_endpoint, token=keystone.auth_token)

List available images

Use the Client.images.list method to list the current images, as shown in Listing 22. Note that this method returns a generator, where the list methods in the nova API return list objects.

Listing 22. Retrieve a list of VM images
>>> import keystoneclient.v2_0.client as ksclient
>>> import glanceclient.v2.client as glclient
>>> creds = get_keystone_creds()
>>> keystone = ksclient.Client(**creds)
>>> glance_endpoint = keystone.service_catalog.url_for(service_type='image',
...                                                    endpoint_type='publicURL')
>>> glance = glclient.Client(glance_endpoint, token=keystone.auth_token)
>>> images = glance.images.list()
>>> images
<generator object list at 0x10c8efd70>
{u'status': u'active', u'tags': [], u'kernel_id':
u'8ab02091-21ea-434c-9b7b-9b4e2ae49591', u'container_format': u'ami', u'min_ram': 0,
u'ramdisk_id': u'd36267b5-7cae-4dec-b5bc-6d2de5c89c64', u'updated_at':
u'2013-05-28T00:44:21Z', u'visibility': u'public', u'file':
u'/v2/images/cac50405-f4d4-4715-b1f6-7f00ff5030e6/file', u'min_disk': 0,
u'id': u'cac50405-f4d4-4715-b1f6-7f00ff5030e6', u'size': 25165824, u'name':
u'cirros-0.3.1-x86_64-uec', u'checksum': u'f8a2eeee2dc65b3d9b6e63678955bd83',
u'created_at': u'2013-05-28T00:44:21Z', u'disk_format': u'ami', u'protected':
False, u'schema': u'/v2/schemas/image'}

Uploading an image to glance

Listing 23 shows an example of how to use the glance API to upload a file. You need to use version 1 of the API to create an image, because the Python API bindings have not implemented the create method for version 2.

Listing 23. Uploading an image to glance
import keystoneclient.v2_0.client as ksclient
import glanceclient
creds = get_keystone_creds()
keystone = ksclient.Client(**creds)
glance_endpoint = keystone.service_catalog.url_for(service_type='image',
glance = glanceclient.Client('1',glance_endpoint, token=keystone.auth_token)
with open('/tmp/cirros-0.3.0-x86_64-disk.img') as fimage:
    glance.images.create(name="cirros", is_public=True, disk_format="qcow2",
                         container_format="bare", data=fimage)

Next steps

This article provides only a brief overview of the functionality the OpenStack Python APIs expose. Here are a few ways you can learn more about how the APIs work.

The official API documentation

The OpenStack project maintains documentation for all of OpenStack Python APIs (see Resources). All of the APIs have auto-generated documentation for each module, class, and method. Some of the APIs have usage examples in the docs, and others don't.

Introspect the API

One of the best ways to learn about the APIs is to use them inside an interactive command-line Python interpreter. The bpython interpreter as an enhanced Python interpreter that displays valid method names as you type and automatically shows you the docstring of a function (see Figure 1).

Figure 1. The bpython automatic help display 
bpython showing the help for the add_floating_ip method 

Examine the CLI source code

One of Python's strengths is its readability, and there's no better way to learn the API than to read the source code. The packages are all hosted on github in the openstack group. For example, to get a copy of the nova API source code, run the following command:

git clone

Because the command-line clients are implemented using the API, each package conveniently ships with an example application.

For the novaclient API, the files of most interest are those in the novaclient/v1_1 directory, which contains the Python classes that form the API. The command-line commands on the shell are implemented as do_* methods in novaclient/v1_1/ For example, nova flavor-list is implemented as the do_flavor_list method, which ultimately calls the Client.flavors.list API method.

Examine other apps that use the Python APIs

Several other applications use the OpenStack Python APIs. The OpenStack Dashboard (see Resources for a link) communicates with the various OpenStack services entirely using the Python APIs. It's a great example of an application that uses the Python APIs. In particular, look at the openstack_dashboard/api directory to see how the dashboard uses the Python API.

The OpenStack Client (see Resources) is an effort to unify the functionality across the existing clients into a single command-line interface. It uses the Python APIs from all of the other projects.

The Heat project is an orchestration layer designed to work with OpenStack and uses the APIs. In particular, look at the heat/engine/ file.

Ansible is a Python-based configuration management tool that has several OpenStack modules that use the Python APIs. In particular, look at the library/cloud directory, which contains the Ansible OpenStack modules.

Once you learn how the Python APIs work, it's difficult to imagine going back to using the REST API or the command-line tools to build your OpenStack automation scripts.



Get products and technologies

  • python-keystoneclient: the OpenStack Identity API and keystone command-line tool. 

  • python-novaclient: the OpenStack Compute API and nova command-line tool. 

  • python-glanceclient: the OpenStack Image API and glance command-line tool. 

  • DevStack: Bash scripts for doing a quick, development-oriented OpenStack deployment on a single server. 

  • Download the bpython enhanced Python interpreter. 


About the author

Lorin Hochstein photo

Lorin Hochstein is an academic-turned-software developer/operator. He is the lead architect for Cloud Services at Nimbis Services, where he deploys OpenStack for technical computing applications. Lorin is a regular contributor to the OpenStack documentation project. He tweets as @lhochstein and blogs

  • 0
  • 0
  • 0


  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
©️2022 CSDN 皮肤主题:大白 设计师:CSDN官方博客 返回首页
钱包余额 0