No More Secrets: Logging Made Easy Through Graylog Part 1

Logging Made Easy Through Graylog Part 1
Logging is an important piece of an organization’s security posture. Logging without organization, searchability, or reporting leads to data being missed. This is the start of a long series that VDA Labs is writing on Graylog. This will be a multi-part series covering a variety of topics including the following items:

• Installation, securing, and optimizing the setup part 1
• Installation, securing, and optimizing the setup part 2
• Domain Controller/DHCP log collection and alerts
• File/print server log collection and alerts
• Exchange server log collection
• IIS log collection
• Firewall log collection
• Setting up Threat Intelligence
• SQL Server

Knowledge is power

Although many pieces of software provide logging, that logging can be modified or deleted from a system before it can be used for troubleshooting or post incident analysis. Having an external source of logging from all systems and services can help decrease time spent troubleshooting or correlating logs from separate systems. External logging can also provide insight into what an attacker may have done on systems even if they deleted or modified the local logs.

Graylog provides an answer to this problem by providing a way to export logs and put them in a separate system. That machine can then be used for: alerting, dashboards, reporting, and incident response.

Sizing Your System

For this blog we will be using the following system specs.

• 6 cores
• 24GB of memory
• 500GB of SSD Storage

Graylog is resource intensive, but with the following specs administrators can expect to process a few thousand messages per second. If administrators need more performance, they should start looking at a multi-node configuration. This process is outside the scope of this blog.

DNS

I CANNOT STRESS THIS ENOUGH. YOU MUST HAVE A VALID FQDN WITH A DNS RECORD.

If a valid DNS record does not exist and you use a IP for your configuration files within Graylog, you will have failures due to missing or invalid Subject Alternative Names (SAN) on your certificates.

Installing Prerequisites

Lets begin by installing and configuring the prerequisites for Graylog functionality.This installation will be done in an Ubuntu 18.04LTS environment. The following packages need to be installed before continuing:

sudo apt-get install apt-transport-https openjdk-8-jre-headless uuid-runtime pwgen

MongoDB

Next lets install MongoDB.

After the installation, make sure to configure MongoDB to start at boot:

sudo systemctl daemon-reload sudo systemctl enable mongod.service sudo systemctl restart mongod.service

Elasticsearch

Graylog 3.x requires Elasticsearch 6.x. Using the following guide, lets install and configure Elasticsearch to work with the Graylog install. Starting with the install, lets run the following commands to install Elasticsearch:

use graylog;
db.createUser(
{
user: "mongo_admin",
pwd: "password123",
roles: [ { role: "root", db: "admin" } ]
}
)
exit