These days I try to run as many applications in Docker as I can. Docker containers are much easier to maintain and upgrade. They are also extremely portable. Graylog is no exception.
I’ve been running Graylog for years to centralize my logs. However, my installation has grown long in the tooth. Here is how I upgraded to a Graylog Docker container.
What is Graylog?
Graylog is a suite of tools that allows you to collect, aggregate and process log files from servers, network appliances, applications and more. It supports many popular formats, like syslog, out of the box. If an application or service logs in a text format, Graylog can process it.
Why would I want to use Graylog?
- Receive notifications of break-in attempts anywhere on your network
- Receive notifications of machines connecting to your network
- Create security dashboards to monitor key metrics
It’s also good for maintenance and troubleshooting. You can receive alerts when certain events occur or search through log history to try and pinpoint issues.
What you need to get started
You don’t need too much to get started, especially if you are already familiar with Docker. Here’s what you’ll need:
- A Docker host. You can follow how I set mine up here. It won’t take long.
- A decent amount of disk space. How much you need really depends on how many logs you’ll be centralizing, and how long you plan on keeping them. For a home lab, 100GB is probably good enough for most people.
- Systems to send log files. In my case, I have all my servers sending syslog outputs. I also have Docker containers sending outputs via Logspout in addition to my pfSense router, and my Omada controller for my wireless access points.
- Probably 15 to 30 minutes of time.
Note these instructions are on an Ubuntu 18.04 Docker installation, but can be easily ported to other OSs.
Step 1: Install Docker Compose
I decided to use docker compose because Graylog really is a combination of applications, and thus we need a combination of MongoDB, Elasticsearch, and Graylog containers.
On your Docker host you’ll want to use the latest docker-compose version, which you can get from Github:
sudo curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
Then make sure docker-compose has execute permissions:
sudo chmod +x /usr/local/bin/docker-compose
Just like that, you’re ready to use Docker compose
Step 2: Create persistent storage locations
It’s a good practice to separate the data storage from the container image for Docker containers. This allows you to recreate and upgrade containers without fear of losing configuration settings or data. I have a drive setup specifically for storage that I use for systems with frequent writes (like my security cameras) that I also use for centralized logging.
You’ll probably want persistant data locations for:
- The Mongo database
- The Elasticsearch data
- The Graylog journal and configuration directories
Step 3: Create your Docker Compose file
You’ll want to create a directory that will contain your docker-compose.yml file:
mkdir graylog cd graylog nano docker-compose.yml
Start by placing the following code in your docker-compose.yml file. This file is based on the official Graylog Docker documentation. They haven’t updated it for the latest Docker Compose version, but I was able to make it work for me. You will need to make changes to suit your needs as we’ll discuss later, but let’s start with this file:
version: '2' services: # MongoDB: https://hub.docker.com/_/mongo/ mongodb: image: mongo:3 #DB in share for persistence volumes: - /video/graylog/mongodb:/data/db # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/6.6/docker.html elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.6.1 #data folder in share for persistence volumes: - /video/graylog/elasticsearchdata:/usr/share/elasticsearch/data environment: - http.host=0.0.0.0 - transport.host=localhost - network.host=0.0.0.0 - node.max_local_storage_nodes=4 - "ES_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 mem_limit: 1g # Graylog: https://hub.docker.com/r/graylog/graylog/ graylog: image: graylog/graylog:3.0 #journal and config directories in local NFS share for persistence volumes: - /video/graylog/journal:/usr/share/graylog/data/journal - /video/graylog/config:/usr/share/graylog/data/config environment: # CHANGE ME (must be at least 16 characters)! - GRAYLOG_PASSWORD_SECRET=forpasswordencryption # Password: admin - GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918 - GRAYLOG_HTTP_BIND_ADDRESS=0.0.0.0:7555 - GRAYLOG_HTTP_EXTERNAL_URI=http://192.168.10.152:7555/ - GRAYLOG_TRANSPORT_EMAIL_WEB_INTERFACE_URL=http://192.168.10.152:7555 - GRAYLOG_TRANSPORT_EMAIL_HOSTNAME=smtp.example.com - GRAYLOG_TRANSPORT_EMAIL_ENABLED=true - GRAYLOG_TRANSPORT_EMAIL_PORT=25 - GRAYLOG_TRANSPORT_EMAIL_USE_AUTH=false - GRAYLOG_TRANSPORT_EMAIL_USE_TLS=false - GRAYLOG_TRANSPORT_EMAIL_USE_SSL=false - GRAYLOG_TRANSPORT_FROM_EMAILfirstname.lastname@example.org - GRAYLOG_TRANSPORT_SUBJECT_PREFIX=[graylog] links: - mongodb:mongo - elasticsearch depends_on: - mongodb - elasticsearch ports: # Graylog web interface and REST API - 7555:7555 # Syslog TCP - 8514:8514 # Syslog UDP - 8514:8514/udp # GELF TCP - 12201:12201 # GELF UDP - 12201:12201/udp
If you aren’t familiar with YAML, I want to make sure that you pay special attention to the indentations of each line. They need to be exact. Use spaces, not tabs. Now let’s walk through this file
Graylog Docker Compose file overview
This file creates 3 containers:
These are the components needed for a basic Graylog installation. The links (line 49) and depends_on (line 52) indicate to Docker that these containers are interconnected and necessary for the container to work.
Each container specifies a few options which we’ll cover next.
Specifying your volumes
Earlier I mentioned that the advantage of creating persistent storage. Storing configuration files and data outside of the container allows for easy upgrade and recreation of containers without worrying about losing data. In order to do this, each container uses a volume option.
Lines 8, 14, 31, and 32 specify volumes for persistent storage for each of the containers. You should modify these lines for your configuration.
Graylog environment options
There are a few environment options for the Graylog container that you should understand and modify for your needs:
- GRAYLOG_PASSWORD_SECRET – This field is used to encrypt Graylog passwords.
- GRAYLOG_ROOT_PASSWORD_SHA2 – This is a SHA2 hash of the password for the admin user (above, the hash is for the password “admin”). You can generate your own password hash with the following command:
echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
- GRAYLOG_HTTP_BIND_ADDRESS – This specifies the network interface used by the Graylog web frontend. If you only have on network interface it is recommended you use 0.0.0.0. Whatever port you specify here must also be in the “ports:” section.
- GRAYLOG_HTTP_EXTERNAL_URI – The public URI of Graylog which will be used by the web interface to communicate with the Graylog REST API.
- GRAYLOG_TRANSPORT options – These options specify the settings for the mail relay Graylog will use for sending notifications. You need to set these options appropriately if you want your Graylog instance to be able to email you with alerts and notifications.
Here you need to specify the ports that will be used by the Graylog container. This includes the port for the web interface, and any ports you plan to use for log collection.
Step 4: Putting it all together
Now that you’ve finalized your docker-compose.yml file you are ready to start up your containers:
sudo docker-compose up -d
Run this command in the directory containing your docker-compose.yml file and the containers will be brought to life. You should then be able to go to the URI you specified and start setting up your Graylog instance.
Using a Graylog as a centralized logging tool is a great option for monitoring your network, applications, machines and security. Graylog can be somewhat of a complicated install, but Docker, and Docker compose, make it much easier. All you have to do is modify the docker-compose.yml file above to your liking and you’ll have a working Graylog instance.
Interested in supporting HomeTechHacker?
Have you found the content on this site useful? If so, are you interested in supporting me and this site? There’s no obligation of course, but I would really appreciate any support you can give. Below are a few ways you can show support:
- Share this site with your friends and on social media (use the sharing links at the end of this page for your convenience)
- Subscribe to this site
- Purchase my book, The Smart Home Manual, for yourself or as a gift.
- Put a link to HomeTechHacker on a site you have access to. Be sure to let me know about it!
- Reach out to me via my contact page or Twitter and let me know something I should write about
- Shop at Amazon through my affiliate links and ads on these pages. See my disclosures for more details about affiliate links. You can also just shop from the link below:
- US Amazon link: https://amzn.to/2H917ud
Thank you! I really appreciate it!