New Year Resolutions

Almost everyone makes them, but almost no one can stick to them. Myself included, for the last years i have been consistently making new years resolutions, some have turned out great and some have failed miserably.

The biggest mistake is that people tend to make overblown goals that require a complete change of their habits. Habits which were obtained over many years are not that easy to break, just with the beginning of a new year. For example, someone who never exercises makes the resolution to go to the gym 4 times a week. The drastic change increases the likelihood of failure extensively. The trick is to lower the level of success so the goal is achievable which leads to better adherence in the long term.

My recommendation about creating successful new year resolutions is to start small and try to modify an already existing habit like eating food when you are hungry. Instead of eating a cookie, you eat something healthy like some nuts. Habits are based on a trigger signal. If you can change the Action to the trigger you can rewire your brain and therefore change a habit.

There is one big exception to this. And that’s when you are dealing with addiction. I used to smoke cigarettes for more than 5 years consistently. At the peak when I was in the military I smoked more than 20 cigarettes a day. I made it my new year resolution to stop 2 consecutive years in a row and it never worked out.

2015

In 2015 I again made the effort to try to stop smoking. Knowing of my past failures I decided to not stop on the first of January but a month earlier on the first of December. In the process of quitting I read the book “The Easyway” by Allen Carr. I highly recommend this book to anyone who is struggling with smoking. Since December 1. 2015, I am officially a non-smoker.

2016

In 2016 I made the resolution to read more. I used to hate reading books of any kind. I bought a kindle and started small, I set the barrier to success to 1 hour a week. Now one year later, I have built the habit of reading every night before I go to sleep and it has become my favorite activity to relax. The books I have read so far all have changed my mindset for the better and I’m excited to read many more perspective changing books in the future.

2017

2017 is the year in which I’m writing this Post. My main resolution for this year is to write more. That’s also the reason why I started this Blog. My English writing needs a ton of improvement that I’m looking to get with more practice over the coming years of my Blogging journey.

Stateless WordPress Docker Container

In this Post, I will show you how you can build a stateless WordPress Setup in a Docker container. The container can later be used in production or as a reproducible development environment.

Motivation

My main motivation in building a stateless Docker container was to be able to deploy and scale the container without having to set up a clustered filesystem.
For this to work, we have to make some modifications and give up some flexibility, in favor of much better maintainability and easier scalability.

The following setup can also be used as a local development environment.

All the Code can be found on Github here wordpress-stateless.

Core Concepts of the Setup

The main difference from a traditional Docker WordPress container setup is that we do not mount Docker volumes into the container. Neither for the WordPress Installation itself nor for the wp-content directory. We install all our plugins and themes on docker build via the Dockerfile.

This way the “docker build” command creates a self-contained image with all plugins and themes that we can tag via “docker tag” to have it assigned a specific build version. Now we can distribute the container to multiple Hosts and don’t have to worry about the filesystem or any kind of volume mounts.

As said before we lose some flexibility. The simplicity of installing plugins via the WordPress Admin is no longer a valid option. Because we have no docker volume mounts, a container restart causes the filesystem to be reset to the initial state of the tagged docker image layer.

But how should we handle media uploads? I thought you might ask.
For this, we use a CDN ( Content Delivery Network) service in conjunction with a WordPress plugin that supports moving the uploaded files to the CDN and rewriting all media links to point to the CDN.

For my setup, I choose Google Cloud Storage and the WP-Stateless WordPress Plugin.

Security

Security with WordPress should always be taken seriously. If an attacker manages to get access to your container and embeds malware in your local files to serve to your users, you can just restart your container and reset it back to the build state.
I have described more on the update of WordPress itself in the Production section later on.

Docker images

I have built multiple Dockerfiles, some of them are depending on each other. It may sound complex but it actually makes things easier when working with the images. Let’s first build them via the build script found in the root of the Github repo.

./build-images.sh 4.7.2 

We have to provide a version for the images, I’m using the WordPress Core Version that is defined in the Base Image Dockerfile but you could provide a different version here.

Base container

FROM php:7.1-fpm

# install the PHP extensions we need
RUN apt-get update && apt-get install -y sudo wget unzip vim mysql-client libpng12-dev libjpeg-dev && rm -rf /var/lib/apt/lists/* \
	&& docker-php-ext-configure gd --with-png-dir=/usr --with-jpeg-dir=/usr \
	&& docker-php-ext-install gd mysqli opcache


# set recommended PHP.ini settings
# see https://secure.php.net/manual/en/opcache.installation.php
RUN { \
		echo 'opcache.memory_consumption=128'; \
		echo 'opcache.interned_strings_buffer=8'; \
		echo 'opcache.max_accelerated_files=4000'; \
		echo 'opcache.revalidate_freq=60'; \
		echo 'opcache.fast_shutdown=1'; \
		echo 'opcache.enable_cli=1'; \
	} > /usr/local/etc/php/conf.d/opcache-recommended.ini

# wordpress version from : https://github.com/docker-library/wordpress/blob/master/php7.0/fpm/Dockerfile
ENV WORDPRESS_VERSION 4.8.2
ENV WORDPRESS_SHA1 a99115b3b6d6d7a1eb6c5617d4e8e704ed50f450

# upstream tarballs include ./wordpress/ so this gives us /usr/src/wordpress
RUN curl -o wordpress.tar.gz -SL https://wordpress.org/wordpress-${WORDPRESS_VERSION}.tar.gz \
	&& echo "$WORDPRESS_SHA1 *wordpress.tar.gz" | sha1sum -c - \
	&& tar -xzf wordpress.tar.gz -C /usr/src/ \
	&& rm wordpress.tar.gz \
	&& chown -R www-data:www-data /usr/src/wordpress


##############################################################################################
# WORDPRESS CUSTOM SETUP
##############################################################################################

# extract wordpress on build
RUN tar cf - --one-file-system -C /usr/src/wordpress . | tar xf -

# add custom scripts
ADD vars.sh /vars.sh
ADD entrypoint.sh /entrypoint.sh
ADD plugins.sh /plugins.sh
RUN chmod +x /entrypoint.sh /vars.sh /plugins.sh


# execute custom entrypoint script
CMD ["/entrypoint.sh"]

The base container is largely based on the official WordPress Docker container. The Dockerfile downloads the WordPress installation via a version defined in the Dockerfile and verifies the code with the “sha1sum” command. If you want to install a different Version of WordPress you have to change the two ENV variables WORDPRESS_VERSION/WORDPRESS_SHA1.

I based the container on the latest PHP-FPM 7.x base image.

CLI Container

FROM wp-stateless-base:wp-4.8.2

##############################################################################################
# WORDPRESS CLI SETUP
##############################################################################################

# install less for wp-cli support , and xterm for terminal support
RUN apt-get update && apt-get install -y less
ENV TERM=xterm

# install wp-cli
RUN curl -o /usr/local/bin/wpcli https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar \
		&& chmod +x /usr/local/bin/wpcli

# add wpcli wrapper
ADD wpcli.sh /usr/local/bin/wp
RUN chmod +x /usr/local/bin/wp

# add tab completion
ADD wp-completion.bash /wp-completion.bash
RUN echo "source /wp-completion.bash" >> ~/.bashrc

##############################################################################################
# CUSTOM ENTRYPOINT
##############################################################################################
ADD entrypoint.sh /entrypoint_cli.sh
RUN chmod +x /entrypoint_cli.sh

ENTRYPOINT ["/entrypoint_cli.sh"]

As you can see, the CLI container is based on the Base container that you just saw. The CLI container adds the WP-CLI Command line utility. We can use this container to setup fresh WordPress installations or make some DB operations via the WP-CLI commands. The image also supports SEARCH REPLACE in the database via ENV variables. This is useful if you want to download your production database and replace the URLs with some local domain.

NGINX

FROM wp-stateless-cli:wp-4.8.2

# install nginx
RUN apt-get update && apt-get install -y nginx && rm -rf /var/lib/apt/lists/*


##############################################################################################
# NGINX SETUP
##############################################################################################
RUN rm -r /etc/nginx/sites-enabled/*
ADD default.conf /etc/nginx/sites-enabled/default.conf
ADD wordpress.conf /etc/nginx/global/wordpress.conf
ADD restrictions.conf /etc/nginx/global/restrictions.conf


##############################################################################################
# CUSTOM ENTRYPOINT
##############################################################################################
ADD entrypoint.sh /entrypoint_nginx.sh
RUN chmod +x /entrypoint_nginx.sh

# reset entrypoint from parent cli
ENTRYPOINT []
CMD ["/entrypoint_nginx.sh"]

Last but not least, I have built an image that includes Nginx as a Webserver to execute the PHP Files via FastCGI. The container depends on the Base image, which means that the web server and PHP-fpm are running inside of the same container. I like this setup because it is easier to deploy. I intentionally didn’t include Nginx in the base image, so you can decide yourself on how to setup your web server. If you want, you can run Nginx in a separate container and call PHP-FPM via sockets.

Local Site Setup

The setup folder in the repo is an example on how to use the CLI image to initialize a new WordPress Database.


# Wordpress Stateless Setup

Used to initialize a fresh Database and generate a wp-config.php file.

- Adjust settings in wp-cli.yml
- Run Container with mapped output folder to save wp-config.php
- Execute setup.sh via the CMD parameter

> docker run --name wp_stateless_setup --rm  --interactive \
-v $(pwd)/output:/var/config \
-v $(pwd)/wp-cli.yml:/var/www/html/wp-cli.yml \
-v $(pwd)/setup.sh:/var/www/html/setup.sh \
wp-stateless-cli:wp-4.7.5 /var/www/html/setup.sh

Let’s initialize a new WordPress database together. If you don’t have an existing MySQL instance available you can start a local database via the docker-compose file in sample/docker-comose.yml.

cd sample
docker-compose up -d db

Now we have to modify the wp-cli.yml config file with our Database connection details. Attention, since we do not link to the database container we cannot use “0.0.0.0” as the host address, instead, we use the docker0 bridge IP address to connect to the database. You can find the bridge IP via “ifconfig docker0”. In my case, the IP of the Bridge is “172.17.0.1”.

Let’s execute the setup.sh script inside the CLI container.

cd ../setup
docker run --name wp_stateless_setup --rm --interactive \
-v $(pwd)/output:/var/config \
-v $(pwd)/wp-cli.yml:/var/www/html/wp-cli.yml \
-v $(pwd)/setup.sh:/var/www/html/setup.sh \
wp-statless-cli:wp-4.7.2 /var/www/html/setup.sh

After the script has finished executing and no errors where raised, there should be a wp-config.php file created in the output folder.

Now we can stop the Database container.

cd ../sample/
docker-compose down

Sample Dockerfile

First, we have to move the generated wp-config.php file into the sample/wordpress directory.

mv ../setup/output/wp-config.php ./wordpress/

The provided Dockerfile in the sample folder shows you how to use the previously built images and customize it with your own plugins and themes.

FROM wp-stateless-nginx:wp-4.7.5

##############################################################################################
# CUSTOM PHP CONFIG
##############################################################################################
RUN { \
  		echo 'upload_max_filesize=10M'; \
  		echo 'post_max_size=10M'; \
  	} > /usr/local/etc/php/conf.d/upload.ini

##############################################################################################
# WORDPRESS Config
##############################################################################################
ADD ./wordpress/wp-config.php /var/www/html/wp-config.php
# chown wp-config.php to root
RUN chown root:root /var/www/html/wp-config.php

##############################################################################################
# WORDPRESS Plugins Setup
##############################################################################################
RUN mkdir /plugins

# Add All Plugin Files but
ADD ./wordpress/plugins/ /plugins

# Execute each on its own for better caching support
RUN /plugins.sh /plugins/base
RUN /plugins.sh /plugins/security

# Delete Plugins script and folder
RUN rm /plugins.sh && rm /plugins -r

# ADD OWN CUSTOM PLUGINS
ADD ./plugins/my-plugin /var/www/html/wp-content/plugins/my-plugin

##############################################################################################
# WORDPRESS Themes Setup
##############################################################################################
ADD ./themes/my-theme /var/www/html/wp-content/themes/my-theme

We are baseing the image of the WordPress container that includes the Nginx web server.
The interesting part starts at line number 20. We add the sample/wordpress/plugin folder into the container and execute a script called /plugins.sh. This script is provided in the base container that we have built before. What this script does is, it parses a file which is provided via the first argument and downloads the plugins from the central WordPress plugin directory and stores them in wp-content/plugins inside of the container.
The syntax of the Plugin file is:

# WordPress plugin name ( downloads latest version )
pluginname
# Specify plugin with a specific Version
pluginname version
# Plugin ZIP file via URL Download
pluginfileurl

At the end of the Dockerfile we add our own local plugins and themes. I provided one sample plugin and theme.

Let’s build our final Docker image. We use the docker-compose command to simply the parameters used for building the image.

# execute in sample folder
docker-compose build

After the image was built successfully we can start the full docker-compose setup with the database.

# execute in sample folder
docker-compose up -d
# verify containers running
docker-compose ps

Now we can visit the URL you provided in setup/wp-cli.yml. The sample URL is “www.mywordpress.local”. We can add a local resolution for this domain via the following command.

sudo /bin/su -c "echo '127.0.0.1 www.mywordpress.local' >> /etc/hosts"
cat /etc/hosts

The admin can be found at http://www.mywordpress.local/wp-admin/.
Default login is “root” / “root”.
In the Admin UI > Plugins, our installed plugins are listed and can be activated. The local plugin “My-Plugin” should also be listed there.
The local theme can be enabled in the Network Admin > Themes.

Docker Compose

The sample folder contains a docker-compose.yml file for local development. The file also simplifies the build process of the container.

I suggest that you copy the contents of sample folder into its own directory and version control it separately.

Plugin development

volumes:
- ./plugins/my-plugin:/var/www/html/wp-content/plugins/my-plugin # Plugin development
- ./themes/my-theme:/var/www/html/wp-content/themes/my-theme # Theme development

I have added a local mount for the Plugin “my-plugin” to the docker-compose.yml config. This way you can edit the Plugin Code and see the changes instantly when you are running the setup locally.
The mound overwrites the files which were  added on build time via the Dockerfile.

Theme development

The same concept applies to Themes. Just mount your local themes when you want to make changes and see them instantly reflected in the Browser.

Production

To run your own image in production, you can start the container and configure it to point to a production database. To change the connection details you can either mount your wp-config.php into the container or you can overwrite some settings via ENV variables.

The base image contains a script vars.sh that filters the wp-config.php on the start of the container to replace some variables.

You can see for yourself which variables are supported and if you need some other variables you can easily modify the script to support them too.

The ENV variable “WORDPRESS_DEV” should either be set to “false” or not provided at all. The variable controls global Error Output and enables/disables the OPCode cache.

Session Sync

As soon as you start to deploy you WordPress container to multiple hosts and you start useing a load balancer without sticky session, you have to setup a session store to sync your users session data.

WordPress Core Update

Each WordPress update requires a rebuild of the base docker image. And all images depending on the base image. It is wise to test your site locally with a new version and see if everyting is working before you deploy your container to production.

Automatic updates

By default minor updates (4.5 to 4.5.1, 4.5.1 to 4.5.2, etc) are automated as they often contain security fixes. This means if we do not change the default config our instances get the security updates too. But if your production server is restarted the security update is lost. For this reason, it is important to check your Mail for available security updates and rebuild the container as soon as possible to avoid any vulnerabilities.

Closing

So there you have it. My special setup that Im using to run this Blog. The way the container is setup i now have alot of flexability in terms of chosing a hosting provider. I deployed this Blog on Kuberenetes in Google Container Engine, but i could easily switch to some other providers like Digital Ocean or Amazon EC2 Container Service.

If you have any questions or improvments, Im happy to hear from you in the comments below.

 

Blogception – How this Blog is setup

Welcome to my first Post on my personal Blog.
In this Post, I’m going to outline how I built this Blog and what technologies and hosting setup I’m using.

You maybe already guessed on which platform this Blog is built on, just by looking at the current Theme or checking the source of this page.
The Blog is running on WordPress.

WordPress is easy to setup and really extensible. In the past, I’ve tried and even built different Blog engines but I came back to WordPress.
After all, at the time of writing this WordPress is powering 25% of the Internet.

The setup

WordPress is the most popular CMS on the Internet, this also makes it lucrative for hackers to find security vulnerabilities to mass target and attack WordPress installations to distribute malware.
Therefore security is a big priority in my setup.

The first step to more security is installing WordPress inside a Docker Container. There are many WordPress Dockerfiles available on DockerHub ready to download. There even is an official Docker Image provided, you can check it out here. I decided to build my own image for reasons I will explain later on. This post will simply describe the architecture of my setup and not every detailed configuration.
I will write a follow-up post with a detailed explanation of the configurations and Dockerfiles and post them to my Github page.

Requirments

Stateless

An important requirement for me was to have a “stateless setup”. The setup in a container enables me to scale and replicate to multiple nodes. To achieve this had to make some adjustments to how the docker image is setup.

WordPress by itself is heavily stateful. The WordPress code is saved on the filesystem ( which also represents a state in some way ). For instance, your wp-config.php is saved on the filesystem which contains important configuration details. The installed themes and plugins are saved in the wp-content directory and of course, the Database contains all the Posts and configuration about your site. Updates of WordPress itself also need to update the PHP code stored on the filesystem.

The Database component will never be truly stateless and we will deploy Mysql as a separate instance in a container. The scaling of the Database itself should also be handled independently.

In a typical WordPress setup, we would install WordPress directly on the filesystem and modify the configuration via a text editor. We would be adding our own themes directly into the installation. This approach is hard to maintain and if you want to run your WordPress Blog on 2 separate nodes you have to make a copy of all files. Updating WordPress via the UI would also not work with multiple nodes.

Support for different Environments

I also wanted to be able to setup different environments to develop locally and later deploy to test, staging and production servers.

Easy way to handle WordPress Updates

WordPress updates should be able to be tested and applied to production nodes without to much manual work.

Steps to support all our Requirments

Eliminating Filesystem state

So first I needed to get rid of the filesystem as the main state representation. There are different ways to solve this Problem.

Version Control WordPress

One possibility would be to use Git to version control the WordPress code itself including your own themes and plugins codes. This way one could write scripts that execute a “git pull” on your production nodes to update all production servers with the newest version of your code. This would also enable us to have different environments. Any updates or plugin changes done via the WordPress Admin UI have to be manually checked in to Git and distributed across all nodes.

Use Docker to build a WordPress Version

Docker images can be tagged and stored in a central registry. Docker images are built via Dockerfiles. In a Dockerfile, we can download the WordPress Code and make our own modifications to it in a reproducible way. The official WordPress Docker image does exactly that, but it also creates a Docker Volume for the installation itself. When we run the official container, a separate Directory for this Docker Volume is created on our local filesystem. This enables the user to log into the WordPress Admin and install plugins and themes and after restarting the container the data is still available. This makes the Filesystem stateful and we do not want that.

To solve this I built my own Dockerfile and skipped the creation of a Docker Volume for the WordPress installation. I added all my own themes and plugins via the Dockerfile while building the image with “docker build”. This provides a big advantage, that the state is now represented in the Dockerfile and the filesystem is now stateless, anytime the docker container is restarted the filesystem is reset to the state in the Dockerfile. The drawback is that any manual changes introduced via the WordPress Admin are lost during a restart of the container. But I can take this drawback for a stateless filesystem and much better security.

Other ways

I’m sure there are other ways to deploy and scale a WordPress installation. But for this Setup, I have used the approach via the Dockerfile.

WordPress Media Files

The Media Library also defines local state with all the Files stored in wp-upload. To fix this we can offload our media files to a CDN provider like Google Cloud Storage or Amazon S3. I have used a WordPress Plugin for that.

Multiple Environments

To support multiple environments we can either add some entry point Scripts in the Docker container that replaces WordPress configuration variables or fully map a wp-config.php file into the container via Docker Volume mappings.
This in conjunction with Docker-Compose we are able to run a local Dev environment and use the same Docker image to run in production.

Frontend Server

This Blog is running on the Web, which means it is accessible via the HTTP Protocol. This means that we need a Webserver which is able to execute the WordPress PHP code every time an HTTP Request is made by the browser. I have used use Nginx together with PHP-fpm. Nginx executes the PHP code via FastCGI.

I added Nginx directly into the Dockerfile that includes the WordPress installation.

Google Cloud

There are many hosting providers that allow you to deploy a WordPress installation. Some are fully managed and others are more flexible. For my setup, I choose the Google Cloud because it enables fine grade resource control and the scaling to multiple nodes. There also is extensive Docker support via a hosted Container Registry and Google Container Engine.

CloudSQL

For the Database, I decided to use the Google Managed Mysql Service CloudSQL. I choose to use the managed Service because it frees me from managing yet another piece of software. Monitoring the Database and applying updates or managing data with backups scripts is a very resource intensive task. For this reason, I’m willing to pay a little more than I otherwise would have to, if I was running a Mysql instance myself.

Kubernetes

For the actual deployment of the Docker Container that includes WordPress, I’m using the open source Cluster Manager Kubernetes.

Kubernetes is developed by Google engineers together with the open source community. The concepts of Kubernetes are derived from a Google internal Cluster Manager called Borg. Kubernetes makes it easy to deploy Docker containers onto a Cluster of compute instances.

We could setup Kuberenetes on any Hosting provider. This again requires the need for maintenance to monitor and update all the Services needed to run a Kubernetes cluster. Luckily Google also provides a service for that, called Google Container Engine. Container Engine offers a fully managed Master Node, this master node is automatically updated. The slave nodes show up as normal compute instances that are running a special Image which includes all the services that are required to run Kuberentes.

Summary

So that’s it for now. To summarize in one Sentence; This Blog is operating on WordPress, versioned in a Docker Image, served via Nginx and deployed via Kubernetes, managed by Google Container Engine that is running on Google Compute Engine.
More detailed posts will follow and I will open source all of my Dockerfiles needed for this setup.