Skip to main content

Raymii.org Logo (IEC resistor symbol)logo

Quis custodiet ipsos custodes?
Home | About | All pages | RSS Feed | Gopher

Automating Openstack with cloud init run a script on VM's first boot

Published: 11-03-2015 | Author: Remy van Elst | Text only version of this article


Table of Contents


openstack

This tutorial will show you how to create a VM in Openstack and execute a scriptat the first boot using cloud-init's user-data feature. This way you caneliminate some more manual labor and keep a small base image, instead ofrequiring all kinds of specific images for specific tasks.

This tutorial will also give you a few example scripts to use with cloud-initand to create Openstack virtual machines from the command line.

You can see all my Openstack related articles here. For example, how tobuild an automated High Available website cluster with Ansible andOpenstack.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get $100 credit for 60 days). (referral link)

For this tutorial I've used the CloudVPS Openstack platform. It will alsowork with any other openstack or cloud provider/software that supports cloud-init and the specific user_data feature. If you build your own images thenmake sure they have cloud-init enabled to support this.

Note that this article is not sponsored nor endorsed by CloudVPS, nor am Ispeaking for or as CloudVPS.

cloud-init

Instead of manually creating a VM, logging in to it and executing a few commandsto set it up, you can automate all these steps. The creation of the VM and thestuff you do when it is up can all be scripted. You can use a programminglanguage like Python to do this, but simple Bash scripts also work just as well.

cloud-init is a piece of software created to help with initializing virtualmachines on multiple different cloud software platforms. It is a collection ofPython scripts that run on a VM's first boot.

It understands and talks to different data providers like Amazons or theOpenstack metadata service.

It uses that information to, for example, set a root password, grow the rootfilesystem, setup an SSH key, do a callback to an URL when a VM is finishedbooting or execute commands at boot. All those things and many more are providedby so called cloud-init modules. Therefor it can be extended easily.

The metadata provided by the cloud provided can contain things like the VM'sname, its IP addres(es), a root password or an SSH key. You can also provideyour own metadata using the so called user_data.

cloud-init has a nifty feature that allows us to place a script in theuser_data which it will execute at the end of the first boot of the machine.It can be a bash script, or any other script as long as it starts with #!.

This tutorial was tested with cloud-init versions 0.7.4 up to 0.7.7. The/etc/cloud/cloud.cfg config file needs the following enabled:

cloud_final_modules:  - scripts-user

Providing user_data to a new VM in Openstack

To provide the user_data script to a new VM you need to place your user_datascript in a file, in this example user_data.file. See below for an examplescript

Make sure you have the Openstack Command Line Tools installed. Forconvinience, also create a computerc file which holds your credentials andsource it in your shell.

The parameter to supply the user data is --user-data $filename. To boot up asmall Ubuntu machine at CloudVPS with our user_data file we can use thiscommand:

nova boot --image "CloudVPS Ubuntu 14.04"  --key-name $ssh_key --flavor "Standard 1" --availability-zone NL1 --user-data user_data.file "Example VPS 1"

If you have the console of the machine open (nova get-vnc-console $UUID novnc)then you should see your script executed at the end of the cloud-init run atboot.

Example user_data cloud-init script

This is an example bash script you can push via the user_data. It gives you ageneric idea of what can be done. You could install and setup your configurationmanagement framework like Puppet or Chef, or just use plain commands. Thisexample uses Ansible to deploy the imaginary Example App for your company atfirst boot:

#!/bin/bash# Example script to run at first boot via Openstack# using the user_data and cloud-init.# This example installs Ansible and deploys your # org's example App.echo "userdata running on hostname: $(uname -n)"echo "Using pip to install Ansible"pip2 install --upgrade ansible 2>&1echo "Cloning repo with example code"git clone https://gitlab.mycompany.org/ansible/example-app.git /tmp/apppushd /tmp/appansible-playbook ./our-app.ymlpopdexit 0

You can also use Python, Ruby or any of your favorite language. As long as theuser_data starts with #! cloud-init will see it as a script and not asspecific cloud-init modules. You do need to make sure that your base image hasthe interpreter installed (Python, Ruby etc.) or bootstrap that via the script.

Here is another script that installs Wordpress on CentOS, including nginx, php-fpm and mysql:

#!/bin/bash# Example script to run at first boot via Openstack using the user_data and cloud-init. This example installs Wordpress, nginx, MySQL and PHP-FPM.# Author: Remy van Elst, https://raymii.org; License: GNU GPLv3printf "\033c" #clear screenVERSION="$(grep -Eo "[0-9]\.[0-9]" /etc/redhat-release | cut -d . -f 1)"echo "Installing EPEL"rpm -Uvh http://cdn.duplicity.so/utils/epel-release-${VERSION}.noarch.rpm 2>&1echo "Installing Ansible and Git"yum -y install ansible git gmp 2>&1echo "Cloning repo with Wordpress Playbook"git clone https://github.com/RaymiiOrg/ansible-examples.git /tmp/app 2>&1echo "Creating Ansible inventory file"echo -e "[wordpress-server]\n127.0.0.1" > /tmp/app/wordpress-nginx/inventoryecho "Starting playbook"cd /tmp/app/wordpress-nginxansible-playbook -i inventory ./site.yml 2>&1exit 0

The repository was forked from Ansible's example repo and changed so that thesite.yml playbook includes the connection: local line. That way we don't useSSH to run the playbook. It also randomly generates the database passwordinstead of using a variable.

Re-execute or debugging

The script only runs at first boot of the machine via cloud-init. If youexecute the cloud-init command again it will not execute the script because italready did it. Testing and debugging the script can be quite intensive if youneed to boot up a machine every time.

We can however fool cloud-init by letting it think the machine did a freshfirst boot. We need to remove the following two files:

/var/lib/cloud/instances/$UUID/boot-finished/var/lib/cloud/instances/$UUID/sem/config_scripts_user

Replace $UUID by your instance's UUID.

Execute the following command to run the cloud-init final module again:

cloud-init modules --mode final

The final module will execute our user_data script again. Before every newtest run you need to remove the two files listed above.

Keep in mind as well that if you for example touch a file and run the scriptagain, the file will still be there. Changes are persistent, build your codeidempotent so that it handles that.

If you've by accident deleted to much cloud-init data you can re-initialize itwith the following command:

cloud-init init

Command Line script to create VM's

Here is an example script you can use to create an amount of VM's using thecommand line. It will wait until the VM is active before creating the next one,and it passes through a user_data file. You can use this, for example, toeasily start up 20 servers and set them up as Apache webservers to scale up whenyour site gets a lot of traffic and needs to scale up.

You do need to place a credentials file named computerc in your home folder.

#!/bin/bashKEY="SSH Key Name"BOOTIMG="IMAGE UUID"ZONE="NL1"FLAVOR="Standard 1"source ~/computerc for RUN in {1..20}; do    echo "Creating VM ${RUN}""    VMUUID=$(nova boot \        --image "${BOOTIMG}" \        --flavor "${FLAVOR}" \        --availability-zone "${ZONE}" \        --nic net-id=00000000-0000-0000-0000-000000000000 \        --key-name "${KEY}" \        --user-data user_data.file \        "VPS-${RUN}-${ZONE}" | awk '/id/ {print $4}' | head -n 1);    until [[ "$(nova show ${VMUUID} | awk '/status/ {print $4}')" == "ACTIVE" ]]; do        :    done    echo "VM ${RUN} (${VMUUID}) is active."done
Tags: bash, cloud, cloud-init, nova, openstack, python, tutorials, user-data