A Practical Introduction to OpenNebula on the Edge with Docker

Issue

miniONE is OpenNebula’s simple evaluation tool for installing an all-in-one, single-node instance based on KVM, LXD system containers, or Firecracker microVMs. The latest version of miniONE brings two new options that allow users to deploy Docker images as Firecracker microVMs thanks to OpenNebula's seamless integration with Docker Hub. By using those new mininONE features together with OpenNebula’s Edge Computing features you can easily deploy containerized applications on cloud resources at the edge from bare-metal infrastructure providers such as AWS and Packet. 

Requirements

We are going to use Packet resources for this tutorial, so you will need a Packet account, and in particular an API token and project ID that will be used to provision resources on Packet. Please follow the Packet getting started guide to create and retrieve them by using the Packet dashboard.

Solution

Step 1: OpenNebula Frontend Installation

You will need one host for the OpenNebula frontend. It can be a physical host in your rack, your own VM, or even a VM running in the public cloud. Choose either the RHEL/CentOS or Ubuntu/Debian family, just ensure it is a relatively fresh and updated system.

For this tutorial we have decided to run the frontend on a bare-metal server on Packet. You can create a server from the Packet dashboard, by choosing different plans, facilities and operating systems. We have chosen the c1.small.x86 plan, the ams1 facility in Amsterdam (Europe), and Ubuntu 18.04 as the operating system.

It will take some minutes before the packet host is ready. Once it is ready, you can connect via SSH to the frontend host and download the miniONE tool:

wget 'https://github.com/OpenNebula/minione/releases/latest/download/minione'

and then execute miniONE to deploy OpenNebula's frontend on the host:

sudo bash minione --frontend --yes

While miniONE is doing its job, you should get an output like this:

### Checks & detection
Checking augeas is installed  SKIP will try to install
Checking for present ssh key  SKIP

### Main deployment steps:
Install OpenNebula frontend version 5.12
Install  augeas-tools

Do you agree? [yes/no]:

### Installation
Updating APT cache  OK
Install  augeas-tools  OK
Configuring repositories  OK
Updating APT cache  OK
Installing OpenNebula packages  OK

### Configuration
Switching OneGate endpoint in oned.conf  OK
Switching OneGate endpoint in onegate-server.conf  OK
Switching keep_empty_bridge on in OpenNebulaNetwork.conf  OK
Switching scheduler interval in oned.conf  OK
Switching to QEMU emulation  OK
Setting initial password for current user and oneadmin  OK
Changing WebUI to listen on port 80  OK
Starting OpenNebula services  OK
Enabling OpenNebula services  OK
Add ssh key to oneadmin user  OK
Update ssh configs to allow VM addresses reusig  OK
Ensure own hostname is resolvable  OK
Checking OpenNebula is working  OK

and finally end up with a successful report similar to this one:

### Report
OpenNebula 5.12 was installed
Sunstone [the webui] is running on:
  http://147.75.100.229/
Use following to login:
  user: oneadmin
  password: ah6Z7TvUNG

Step 2: OpenNebula Firecracker Edge Installation

Now you can proceed to provision the Packet edge nodes with the Firecracker hypervisors that will be used for the deployment of containerized application on the edge. miniONE gives you the possibility to extend an OpenNebula environment by adding hypervisor nodes at the edge, by using the option --node to the deployment command.

From the frontend host, we will provision a resource on Packet. In order to run the command, some basic information has to be provided: the API token, the project ID, the edge facility (in our example is sjc1 in Sunnyvale, CA).

sudo bash minione --node --firecracker --edge packet --edge-packet-token $TOKEN --edge-packet-project $PROJECT --edge-packet-facility sjc1 --yes

The provisioning will take 5-10 minutes to complete.

While miniONE is doing its job, you should get an output like this:

### Checks & detection
Checking python-pip is installed  SKIP will try to install
Checking ansible  SKIP will try to install

### Main deployment steps:
Install ONEProvision
Configure IPAM Packet, alias IP mapping driver, VM hooks
Trigger oneprovision
Export appliance and update VM template
Install  python-pip

Do you agree? [yes/no]:

### Installation
Updating APT cache  OK
Install  python-pip  OK
Install from PyPI 'ansible≥2.8.0,<2.9.0'  OK
Configuring repositories  OK
Create docker packages repository  OK
Updating APT cache  OK
Installing opennebula-provision package   OK
Install docker  OK
Start docker service  OK
Enable docker service  OK

### Configuration
Applying packet changes to oned.conf  OK
Update ssh configs to accessing Packet hosts  OK
Add oneadmin to docker group  OK
Update network hooks  OK
Checking OpenNebula is working  OK
Configuring packet hooks  OK
Restarting OpenNebula  OK
Prepare packet template  OK
Checking packet template [/tmp/tmp.7y4oeKrM6n]  OK
Running oneprovision
...
Exporting [alpine] from dockerhub to local datastore  OK
Exporting [Kernel 5.4 x86_64 - Firecracker] to local datastore  OK
Waiting until the image is ready  OK
Updating VM template  OK

and finally end up with something similar to this:

### Packet provisioned
  ID NAME              CLUSTER    TVM      ALLOCATED_CPU      ALLOCATED_MEM STAT
   0 147.75.101.67     PacketClus   0       0 / 350 (0%)     0K / 7.8G (0%) on  

To extend the setup by additional hypervisor on Packet run following command:
./minione --edge packet --node --edge-packet-token [<token>] --edge-packet-project [<project>]

To cleanup (delete resources in OpenNebula and Packet) run:
oneprovision delete 7670025d-5e6a-43ba-89e5-6b10576d9b0e --cleanup

Step 3: Run Containerized Applications on the Edge using the Docker Hub Marketplace

At this point you have a ready-to-use OpenNebula frontend connected with the Firecracker hypervisor on the edge and you are ready to deploy your containerized application. You can do this by using Sunstone, OpenNebula's WebUI.

We are going to deploy an Nginx application from the Docker Hub marketplace that is already configured on miniONE.

Just select Nginx from the Apps tab.

Now download it into the datastore related to the edge-node (i.e. packet-100-default):

When Nginx is imported from Docker Hub, a VM template is also created. You'll have to update the template.

You can do that by adding the virtual networks. Add first the local network (i.e. packet-100-host-only):

and then the public network (i.e. packet-100-public) as an alias of NIC0:

Don't forget also the kernel image (already imported by miniONE in the edge node datastore - packet-100-files):

And add the start script for starting the Nginx application at the microVM boot time:

Once the VM template is updated you can instantiate it to create a microVM:

By using the public IP of the microVM (alias in the VM info tab)...

...you can use your web browser to access the Nginx application straightaway!

Step 4: Run Non-Official Docker Hub Images on the Edge 

To show how to run non-official Docker Hub images on the edge, we are going to deploy MinIO, which is a High Performance Object Storage with a S3 compatible API, by using the command line interface (CLI). We are going to show also how to prepare a persistent datablock image to be used as persistent volume for the application.

First we need to register the MinIO Docker image in the edge image datastore (i.e. packet-100-default) by using the following command:

oneimage create -d packet-100-default --path 'docker://minio/minio?size=256&format=raw&filesystem=ext4' --name minio

Then, with the following command we will create a persistent datablock that will be used as a persistent volume for the application:

oneimage create -d packet-100-default --type DATABLOCK --size 2048 --persistent --name minio-data

and then you can format the datablock image with:

img=$(oneimage show minio-data | grep SOURCE | awk -F ' +' '{print $3}')
sudo virt-format --partition=none --filesystem=ext4 --format=raw -a $img

Now you can create a template "minio.tpl" of the microVM that will use the Docker image registered in the edge datastore:

NAME="minio"
CPU="1"
MEMORY="1024"
DISK=[
  IMAGE="minio",
  TARGET="vda" ]
DISK=[
  IMAGE="minio-data",
  TARGE="vdb" ]
GRAPHICS=[
  LISTEN="0.0.0.0",
  TYPE="VNC" ]
NIC=[
  NETWORK="packet-100-host-only" ]
NIC_ALIAS=[
  NETWORK="packet-100-public",
  PARENT="NIC0" ]
OS=[
  KERNEL_CMD="console=ttyS0 reboot=k panic=1 pci=off",
  KERNEL_DS="$FILE[IMAGE=\"kernel\"]" ]
CONTEXT=[
  NETWORK="YES",
  SET_HOSTNAME="minio",
  PASSWORD="root",
  SSH_PUBLIC_KEY="$USER[SSH_PUBLIC_KEY]",
  FILES = "/usr/share/minio/init.sh"
]

The template can be created by using:

onetemplate create minio.tpl

In the template, you can specify the CPU, MEMORY, the registered docker image, the persistent volume and the virtual networks. It is mandatory to specify in the OS section, the "kernel" image and the kernel boot parameters; as a kernel image you can use the one that is imported by miniONE during the installation. In order to start the minIO application, an init script (/usr/share/minio/init.sh) has been defined:

#!/bin/bash
mount /dev/vdb /data
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
export MINIO_ACCESS_KEY=ACCESSKEY
export MINIO_SECRET_KEY=SECRETKEY
export MINIO_UPDATE=off
nohup /usr/bin/docker-entrypoint.sh server /data > /tmp/minio.log 2>&1 &

Once the template has been created, the microVM can be instantiated with this command:

onetemplate instantiate minio --name minio

To test the MinIO application that has been deployed, we can use the "mc" MinIO client:

You can use the "mc" client to add the MinIO server just deployed.

mc config host add minio http://[micro_vm_public_ip]:9000 ACCESSKEY SECRETKEY --api S3v4

Then you can create a bucket:

mc mb minio/datasets

and add one file to the bucket:

mc cp <file> minio/datasets

You can manage the lifecycle of your application by using the onevm command to, for example, poweroff, resume and terminate the microVM.

Then you can access the MinIO GUI by using http://[micro_vm_public_ip]:9000/minio and use the access key and secret key used in the init script. Voilà!

Step 5: Cleanup

Before you finish this evaluation, don’t forget to clean up your Packet hosts! To delete the Packet hosts you can use the oneprovision command:

oneprovision delete [ID] --cleanup

miniONE is a tool to really help newcomers get “up and running” with an OpenNebula environment. And now you’ve seen how easy it is to launch your own edge deployment or eventually extend your current setup with on-demand bare-metal resources at the edge and how to easily deploy containerized application on the edge using OpenNebula's new native integration with Docker Hub. Enjoy!