It can be a pain for developers to setup a Solr instance for local development. Beyond the setup, there's the possibility for differences in configurations and versions. It's also entirely possible a developer could have multiple projects requiring different Solr versions and configurations. Whether you're a developer or devops engineer, adding Docker to your toolkit can ease this pain.
A recent project I'm working on is hosted on Pantheon. We're using their multidev configuration so each developer has their own branch and remote environment. Some developers work locally, and others work directly on the remote multidev environment.
For those not familiar with Pantheon, this would be the equivalent of an integration environment configured to automatically update with commits to a Git master branch, with each developer having their own branch to work on and merge into master.
The Problem:
Pantheon provides Solr instances and access to them from the multidev environments (environments hosted on their servers), but not remote access for those working on a local environment.
The Solution:
This is a great use case for Docker!
If you have a team of developers working locally, you could setup a separate Solr instance or each developer on a single development server. If you are a developer needing a Solr instance, having Docker running somewhere would be handy to easily spin up one or more Solr instances as they're needed.
The steps below are meant to demonstrate the process of building an image and running one or more containers. I'll also explain a little of what's going on behind the scenes to take away some of the "magic" factor. The `drupal-solr` build is not meant for production. it should however show how you can create a build for any service you would like to provide with Docker.
1) Install Docker
Installation is beyond the scope of this article, but head over to http://docs.docker.io/en/latest/ to learn how to install. It's easy to get up and running on Ubuntu 12+, but because of the kernel requirements, you need to run it on a VM if you want to run it on OSX.
2) Get the Docker files
https://github.com/mikemilano/dockerfiles (The drupal-solr is the file we're interested in for this post)
cd /usr/local/src git clone [email protected]:mikemilano/dockerfiles.git
3) Customize Solr (optional)
The drupal-solr build is configured with the Drupal schema already and setup to copy everything in drupal-solr/files to the root file system of the image. Solr itself will live in /srv/solr, so you can navigate to drupal-solr/files/srv/solr to make any modifications. There is no security setup by default.
4) Build the Docker image
This will take a while the first time because it is downloading the base ubuntu image.
You can easily see what a build does by looking at the "Dockerfile" in this directory. Docker caches the steps so if you make changes, it's only re-building from the line that changed on down.
The last line of the Docker file is the command that is run by default when you launch a container.
CMD ["/etc/docker/drupal-solr-start"]
If you would like to modify the start command, update: drupal-solr/files/etc/docker/drupal-solr-start.
Docker uses git to store and tag images. Containers are launched based on the tag name of an image. The Docker convention is to namespace the image with something like "
Here's how to build the image: (remember, the 1st time will take a while because you are downloading an entire OS)
cd /usr/local/src/dockerfiles/drupal-solr/ docker build -t="mmilano/drupal-solr" .
5) Launch your Solr instance(s)
This is where the beauty of containers comes into play. You can easily launch one or more Solr instances that only use the resources necessary for the service itself. They will be up and running in seconds.
Here are a couple examples:
Launch Solr allowing Docker to assign the external IP
docker run -d mmilano/drupal-solr
The -d flag tells Docker to run the container detached. If you do not use that, you will see the output of Solr as if you ran the startup command.
Docker creates an internal network and assigns an internal IP to each container. Each container running this image will run Solr on port 8983 of the internal IP. Docker then maps a port on the host, to the internal ip:port the image exposes. You access the service exposed by the container with the host ip:port. If you do not provide a port option, Docker will automatically assign an avaialble host port. You can see active containers by running `docker ps`.
To see what port the container is mapped to, you can run `docker ps`.
# docker ps ID IMAGE COMMAND CREATED STATUS PORTS 1ac4bca9ba12 mmilano/drupal-solr:latest /etc/docker/drupal-s 2 seconds ago Up 1 seconds 49154->8983
In this case, Docker mapped the container's internal port, `8983', to the host's port, `49154'.
Launch Solr with explicitly defined port
docker run -p 4001:8983 -d mmilano/drupal-solr
By default, Solr is setup to run on port 8983. You can explicitly define the port you would like this container to run on by using the -p flag. This configuration will route the container's local port, 8983, to the Docker host's port, 4001. i.e. if the host's IP is 1.2.3.4, it would be accessible at 1.2.3.4:4001.
Launch 3 more instances on different ports
docker run -p 4001:8983 -d mmilano/drupal-solr docker run -p 4002:8983 -d mmilano/drupal-solr docker run -p 4003:8983 -d mmilano/drupal-solr
You get the idea!
Take a look at the docker run documentation for more options.
A few more notes about images, containers and the host file system
I'm not sure it's documented anywhere, so I wanted to remove some of the mystery there may be around physical files on the Docker host.
Docker stores the images and containers in `/var/lib/docker` by default. Image data is stored in `/var/lib/docker/graph` and container specific files are stored in `/var/lib/docker/containers/ <container id>`. Image data is only stored once regardless of how many containers you launch. For example, the container directory for a new drupal-solr instance is only 56K.
Modifications to the file system of a container will be stored in the container's directory. That file system will continue to exist after you kill a container. You can run docker start <container id> to start a container after it has been killed.
To complete remove a container, including it's file system, you need to run docker rm <container id>.
Here's a few more helpful commands:
List images
docker images
List active containers
docker ps
List active and inactive containers
docker ps -a
Launch an interactive container
This is great for exploring or modifying an image. You need to re-tag the image after making changes and existing though. I recommend having all your changes in the build files though so you can re-create your image at any time. Typing `exit` will exit and kill the container. `ctrl+d` will detach the container from your shell, and keep it running. `docker attach <container id>` will re-attach the container.
docker run -i -t mmilano/drupal-solr /bin/bash
List active processes inside a container
docker top
Remove a container
docker rm
Remove all containers (sometimes helpful when expirmenting)
docker ps -a | awk '{print $1}' | xargs docker rm
In conclusion
One thing that excites me is that this custom Solr build can live in the version control system of the project that uses it. Any developer can checkout the code and build/run the instance on the platform and configuration it was meant to run with. This can be applied to an entire infrastructure if it is one that is a good match for containerization.
Docker, in its infancy, is an incredible tool for developers & sysadmins to have in their chest. It will be exciting to build with, see what others build, and watch Docker mature in the years to come.