Skip to main content Logo (IEC resistor symbol)logo

Quis custodiet ipsos custodes?
Home | About | All pages | RSS Feed | Gopher

Build a FreeBSD 10.3-release Openstack Image with bsd-cloudinit

Published: 27-04-2016 | Author: Remy van Elst | Text only version of this article

Table of Contents

We are going to prepare a FreeBSD image for Openstack deployment. We do this bycreating a FreeBSD 10.3-RELEASE instance, installing it and converting it usingbsd-cloudinit. We'll use the CloudVPS public Openstack cloud for this.Create an account there and install the Openstack command line tools, likenova, cinder and glance.

You can see all my Openstack related articles here. For example, how to useDuplicity to create Encrypted backups to the Openstack Swift Object Store

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get $100 credit for 60 days). (referral link)

This tutorial is available for:

Command Line tools

Make sure you have the Openstack command line tools installed. Follow theofficial openstack guide here. If you have pip installed you can usethat to install the tools:

pip install python-novaclientpip install python-cinderclientpip install python-glanceclientpip install python-keystoneclientpip install python-neutronclientpip install python-swiftclient

Save yourself some time and create a file named computerc with the belowcontents:

export OS_AUTH_URL=""export OS_TENANT_NAME="<tenant name>"export OS_USERNAME="<username>"export OS_PASSWORD="<password>"export OS_TENANT_ID="<tenant id>"

When you are going to do stuff with the Openstack command line clients, loadthis file:

source computerc

That way, your authentication data are loaded and you don't have to giveparameters like -os-username and such.

Openstack Overview

Openstack is a datacenter virtualization plaform consisting out of manydifferent tools and services. For this tutorial it is important to know thefollowing ones.

Compute (Nova)

This is the virtualization service. It works with a hypervisor to create andmanage virtual machines. You can create a VM based on a specific "Flavour",which is just a definition of specs like disk, cpu and ram.

Block Storage (Cinder)

This is the service which makes block devices (volumes) available to services. Aflavour can not be changed, just resized. If you want extra storage you need tocreate a volume, attach it to the VM and mount it there for use.

Images (Glance)

This is the service which holds all the images. Images can be used to boot a VMfrom. Images can be prepared with tools like cloud-init to make them behavebetter in a cloud environment, for example, setting an SSH key or password atboot.

ISO Installation

Booting from an ISO with a disk attached is quite tricky in Openstack. ThisFreeBSD tutorial can be used for any ISO which needs to be installed. Theprocess is as following:

You then have an image with your own installation available to boot new vm'sfrom.

Lets get started.

Upload the ISO to glance

Start by uploading the FreeBSD 10.3-RELEASE ISO to Openstack using the Glanceclient:

glance image-create --name "FreeBSD-10.3-RELEASE-amd64-dvd1.iso" --disk-format iso --container-format bare --copy-from ""

If your Openstack provider does not support the --copy-from parameter you willhave to download the ISO yourself:


And upload it as an image. That can take a while:

glance image-create --file Downloads/FreeBSD-10.3-RELEASE-amd64-dvd1.iso --name "FreeBSD-10.3-RELEASE-amd64-dvd1.iso" --disk-format iso --container-format bare --progress

The result of the glance command should be something like below:

+------------------+--------------------------------------+| Property         | Value                                |+------------------+--------------------------------------+| checksum         | None                                 || container_format | bare                                 || created_at       | 2014-11-16T10:18:39                  || deleted          | False                                || deleted_at       | None                                 || disk_format      | iso                                  || id               | 5b74bee7-c05e-4102-9dd2-349dec4adee2 || is_public        | False                                || min_disk         | 0                                    || min_ram          | 0                                    || name             | FreeBSD-10.3-RELEASE-amd64-dvd1.iso  || owner            | 0                                    || protected        | False                                || size             | 2585028608                           || status           | queued                               || updated_at       | 2014-11-16T10:18:39                  || virtual_size     | None                                 |+------------------+--------------------------------------+

Note down the ISO ID, in our case 5b74bee7-c05e-4102-9dd2-349dec4adee2.

Create the volume

Create the root volume for our FreeBSD preparation install:

cinder create --display-name "FreeBSD-10.3-root" --availability-zone=NL1 16

This creates an 16 GB volume, on which we will install FreeBSD. The result ofthe command is like below:

+---------------------+--------------------------------------+|       Property      |                Value                 |+---------------------+--------------------------------------+|     attachments     |                  []                  ||  availability_zone  |                 NL1                  ||       bootable      |                false                 ||      created_at     |      2014-11-16T10:23:00.573804      || display_description |                 None                 ||     display_name    |          FreeBSD-10.3-root           ||      encrypted      |                False                 ||          id         | f00fdea0-37ea-4a0e-9a1d-403d39ce8320 ||       metadata      |                  {}                  ||         size        |                  16                  ||     snapshot_id     |                 None                 ||     source_volid    |                 None                 ||        status       |               creating               ||     volume_type     |                 None                 |+---------------------+--------------------------------------+

Again, note down the ID, in our case f00fdea0-37ea-4a0e-9a1d-403d39ce8320.

Boot a VM with an ISO and extra volume

Boot a new instance from the ISO, attaching the volume we just created as well:

nova boot --image <freebsd iso image id> --key-name <ssh key> --poll --flavor "Standard 1" --availability-zone NL1 --nic net-id=00000000-0000-0000-0000-000000000000 --block-device-mapping "vdb=<volume freebsd-10-root id>:::0" "FreeBSD-10.3-RELEASE-install"

Your output will be like below:

+--------------------------------------+-----------------------------------------------------------------------------+| Property                             | Value                                                                       |+--------------------------------------+-----------------------------------------------------------------------------+| OS-DCF:diskConfig                    | MANUAL                                                                      || OS-EXT-AZ:availability_zone          | NL1                                                                         || OS-EXT-STS:power_state               | 0                                                                           || OS-EXT-STS:task_state                | scheduling                                                                  || OS-EXT-STS:vm_state                  | building                                                                    || OS-SRV-USG:launched_at               | -                                                                           || OS-SRV-USG:terminated_at             | -                                                                           || accessIPv4                           |                                                                             || accessIPv6                           |                                                                             || adminPass                            |                                                                             || config_drive                         |                                                                             || created                              | 2014-11-16T10:28:12Z                                                        || flavor                               | Standard 1 (111)                                                            || hostId                               |                                                                             || id                                   | dad95c7d-f921-4e89-8ea7-99e1654d65fe                                        || image                                | FreeBSD-10.3-RELEASE-amd64-dvd1.iso (5b74bee7-c05e-4102-9dd2-349dec4adee2)   || key_name                             | -                                                                           || metadata                             | {}                                                                          || name                                 | FreeBSD-10.3-RELEASE-install                                                || os-extended-volumes:volumes_attached | [{"id": "f00fdea0-37ea-4a0e-9a1d-403d39ce8320"}]                            || progress                             | 0                                                                           || security_groups                      | default                                                                     || status                               | BUILD                                                                       || tenant_id                            |                                                                             || updated                              | 2014-11-16T10:28:12Z                                                        || user_id                              |                                                                             |+--------------------------------------+-----------------------------------------------------------------------------+

Note down the ID like usual, in our case:dad95c7d-f921-4e89-8ea7-99e1654d65fe.

Install FreeBSD 10.3

Do a standard install of FreeBSD 10. Use the Openstack console to do that.

These are my simple install settings:

Stop the install VM

Shut the instance from FreeBSD using shutdown -p now and after that via nova:

nova stop <install vm id>

Detach the volume:

nova volume-detach <install vm id> <install root volume id>

When the volume is detached you can delete the installation VM. We don't need itanymore:

nova delete <install vm id>

The volume will not be destroyed when the VM is deleted.

Boot a new VM with the volume as root disk

Boot a new instance with the volume freebsd was installed on as the root disk:

nova boot --block-device source=volume,id=<root volume id>,dest=volume,shutdown=preserve,bootindex=0 --poll --flavor "Standard 1" --availability-zone NL1 --nic net-id=00000000-0000-0000-0000-000000000000 --key-name <ssh key> FreeBSD-10.3-RELEASE-configure

Here we use --block-device to specify that the only disk attached to theinstance should be the volume source=volume,id=<volume id>. This can also bean image as source. The destination is also a volume, this can also be local.

If you have more block devices specified here you should make sure there is only1 with the bootindex 0, that is the disk the VM will try to boot from.

Prepare FreeBSD for Cloud Init

If you get a mountroot error from the bootloader, enter the following:


Change /etc/fstab after the fact, change ada0 to vtbd0 for VirtIO support.

Bootstrap the packaging system (pkg) so we can install some stuff required forbsd-cloudinit:


Enter y.

Install vim and py27-setuptools (for bsd-cloudinit):

pkg install vim-lite py27-setuptools ca_root_nss

The ca root nss is required by fetch to do certificate validation.

Login Users

By default cloudinit will create a user named freebsd which has sudoprivileges without password. However, if you want to enable root login, so notwith a freebsd user, you need to add your SSH key, enable root login and addsome extra configuration later on for cloud-init.

If you want root login, add your SSH key:

mkdir /root/.sshchmod 700 /root/.sshecho "ssh-rsa AAAA[...]" > /root/.ssh/authorized_keyschmod 600 /root/.ssh/authorized_keys

Enable root login via ssh:

vim /etc/ssh/sshd_configPermitRootLogin yes

Expand root filesystem

Since FreeBSD 10.2-RELEASE there is an rc script which, when the file/firstboot exists, expands the root filesystem to the full disk. While bsd-cloudinit does this as well, if you don't need the whole cloudinit stack,(when you use a static ssh key for example), you can touch that file to makesure the disk is expanded at the first boot:

touch /firstboot

bsd-cloudinit install

Install python modules for bsd-cloudinit:

rehasheasy_install eventleteasy_install iso8601

Add the following to /boot/loader.conf to make sure the console works:


This sets console output to go to the serial console, which is displayed by novaconsolelog, and the video console for sites with VNC or Spice configured.

Now do any other customizations you want to have in your image.

Do the bsd-cloudinit install:

fetch +x

The default username will be freebsd, bsd-cloudinit will create it if it doesnot exists. The freebsd user will also be able to sudo without a password bydefault, bsd-cloudinit will handle that by default.

Before you start the installer, make sure you do any other configuration youwant. After the installer has started and you reboot, cloudinit will prepare thevm for use.

Start the installer:


Delete all history:

set history = 0history -c

Zero out all the free space:

dd if=/dev/zero of=/blarm /bla

We do that so that the image compresses better later.

Turn the machine off:

shutdown -p now

Terminate the machine, otherwise you cannot detach the volume (ERROR: Can'tdetach root device volume (HTTP 403)):

nova delete <id of freebsd-configure vm>

Convert the volume to an image:

cinder upload-to-image <freebsd-10-root volume id> FreeBSD-10.3-RELEASE-CloudInit

This might take a while.

Set the min-disk and min-ram requirements, plus some more properties for theimage:

glance image-update --min-disk 8 --min-ram 1024 --property architecture=x86_64 --property image_supports_keypair=true --property image_supports_password=true --property supported=false <id from the converted volume image>

If needed, make it public:

glance --name "UNSUPPORTED: FreeBSD-10.3-RELEASE" --is-public True <id from the converted volume image>

That's it. You are done and have a good workable freebsd image.

Boot a new instance from your newly created image:

nova boot --image <id from the converted volume image> --flavor "Standard 4" --availability-zone NL1 --nic net-id=00000000-0000-0000-0000-000000000000 --key-name <your ssh key> FreeBSD-10.3-RELEASE-cloudinit

When the instance has spawned you can login as the freebsd user,freebsd@ipaddress.

Compress the image

If your Openstack provider does not compress the images converted from volumesyou might need to do that yourself. You want to do that because storing 475 MBcosts you less qouta/money than 16 GB.

First, use glance to download the image:

glance image-download --file bsd.raw <freebsd image uuid>

Convert and compress the image to qcow2 with qemu-img:

qemu-img convert -c -f raw -O qcow2 bsd.raw bsd.qcow2

You will get the best compression if you've zero'd out the image as listedabove.

Upload this new smaller image to Openstack:

glance image-create --name FreeBSD-10.3 --disk-format qcow2 --container-format bare --min-disk 16 --min-ram 1024  --property architecture=x86_64 --property image_supports_keypair=true --property image_supports_password=true --property supported=false --file ./bsd.qcow2 

In my case the converted volume was 16 GB and the compressed image was 475MB.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10credit). (referral link)

Tags: cloud, cloudinit, compute, freebsd, image, openstack, python, tutorials