CoreOS, etcd, systemd and Fleet in 5 minutes
Introduction
This article is a simplified guide which guides you through doing infrastrcture setup for a CoreOS cluster (currently via Amazon EC2) and examples of using etcd, systemd and Fleet.
The CoreOS setup will have etcd and fleet services running, for a cluster of 3 servers.
Setup guide
Most of the information is following the CoreOS EC2 guide for performing manual steps.
Simple service configruation, usage and testing is followed from the quick start guide leading onto complete guides for services.
1) Initial instance creation
Create a vanilla CoreOS instance on Amazon EC2. Create only a single instances. We will then do the necessary configuration and then clone 2 additional instances.
2) Security group setup
For your existing security group (via the Security groups page), you will need to open inbound ports 4001 and 7001 via a “custom TCP Rule” option, providing a soure type of “Custom IP” and entering your existing/current security group ID.
3) Cloud-Config setup
Cloud-config is the CoreOS configuration file, which you can declaratively configure and customize various OS level items. This config file gets loaded every time the server is booted. It is here one can configure using services such as Fleet, etcd and cluster discovery. You can read up more about cloud-config
The cloud-cofing file to be used is:
You can only edit the cloud-config when the instance is stopped and not running. In the “Instances” page, If the instance is running then stop the instance first. Then on the instance right-click and go to “Instance Settings” -> “View/Change User Data”. You can then copy and past the above into the text area that appears.
Note: replace the <token>
with a token generated after visiting the URL https://discovery.etcd.io/new?size=x where x is the number of servers in the cluster.
4) Create multiple instances
In the instances section, right click on the server and select “Launch More Like This”. On the following review page, make sure you alter the storage size (as it defaults to 8 GB) and change the Tag value for the “name” property. This will then create a new server with the same cloud-config file and security groups, therefore allowing the cluster to grow and enabling the instances to discover each other.
5) Test service discovery through etcd
From the quick start guide you can view a complete ectd guide, but for the purposes of ensuring service discovery exists, a simple example is illustrated here. Note: it is recommended to start using etcd2 rather than etcd.
In one of the machines (login using SSH
as user core
), set a key message
with value Hello
like so:
Log into another one of the machines in the cluster and read the value of the message:
After this validation, you can change the current value:
Log into another machine in the cluster and read the value of the message:
Service discovery and etcd have been setup succesfully now.
6) Unit files and systemd
systemd is an init system that provides features for starting, stopping and managing processes. systemd is run through Unit files that reside in /etc/systemd/system
, that have the extension .service
.
You can view the systemd guide for more details.
Create a hello.service
file in the /etc/systemd/system
folder with this content:
To start a new unit, systemd needs to create the symlink and then start the file:
To verify the unit started, you can see the list of containers running with command docker ps
:
and read the unit’s output with journalctl:
You can check the status of the service:
You can stop the service by running the stop
command:
7) Fleet
Fleet is a cluster manager that controls systemd at the cluster level. Fleet works by receiving systemd unit files and scheduling them onto machines in the cluster based on declared conflicts and other preferences encoded in the unit file. Two types of units can be run in your cluster — standard and global units. Standard units are long-running processes that are scheduled onto a single machine. If that machine goes offline, the unit will be migrated onto a new machine and started. Global units will be run on all machines in the cluster.
You can see which machines are under fleet’s control by running the command:
To illustrate an example of a standard unit:
create a myapp.service
file in /etc/systemd/system
which conatins:
Run the start command to start up the container on the cluster:
The unit should have been scheduled to a machine in your cluster:
To stop the container, run:
To illustrate an example of a global unit:
on the bottom of the myapp.service
file add an additional tag and value
Run the start command to start up the container on all clusters:
To read further on Fleet and its commands, visit the Fleet client page and launching containers with fleet