No More Secrets: Logging Made Easy Through Graylog Part 6

Logging is a important but often overlooked part of an organization’s security posture. Logging without organization, searchability, or reporting leads to data being missed. This a continuation of a longer series that VDA Labs is writing on Graylog. This is part 6 of a multi-part series covering a variety of topics, including the following items:

1. Installation, securing, and optimizing the setup part 1
2. Installation, securing, and optimizing the setup part 2
3. Domain Controller/DHCP log collection and alerts
4. File/print server log collection and alerts
5. Exchange server log collection
6. IIS log collection
7. Firewall log collection
8. Setting up Threat Intelligence
9. SQL Server

This week focuses on gathering logs from from Microsoft IIS. These logs can help track and monitor users and greatly enhance troubleshooting and incident response event due to the correlation that can be completed.

Note: The Microsoft Server 2019 is being used. Some steps or logs may be different depending on the version of Microsoft Server being are used.

Prerequisites

Installing our Log Shipping Agent

To ship the logs from our server, lets use NXlog Community Edition. This is a tool which can be configured to read just about any file or system input. There is also an enterprise support option available if required by your organization.

The Windows .MSI install will be used for this guide. Both the GraylogWeb.pem and GraylogWeb.key from part 2 of this series and the graylogca.pem from part one will be needed.

Installation

Installing NXLog is as simple as running through our download .msi installer. After installation is finished, the configuration needs to be written.

Configuration

The first step of this configuration is to copy the GraylogWeb.pem, GraylogWeb.key, and graylogca.pem to C:\Program Files(x86)\nxlog\certs. Without these files, the SSL connection to the Graylog input that was created in the last step will fail. With those files moved over, let’s start going over the configuration we need to create. Here is the complete configuration file, located in C:\Program Files(x86)\nxlog\conf\nxlog.conf:

## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/docs/

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
Module xm_gelf
ShortMessageLength 3000
</Extension>

<Extension w3c>
Module xm_csv
Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs(User-Agent), $cs(Referer), $sc-status, $sc-substatus, $sc-win32-status, $time-taken
FieldTypes string, string, string, string, string, string, integer, string, string, string, string, string, string, string, string
Delimiter ' '
QuoteChar '"'
EscapeControl FALSE
UndefValue -
</Extension>

<Input iis_in>

Module im_file
File "C:\\W3SVC1\\u_ex*"
SavePos TRUE
Exec if $raw_event =~ /^#/ drop(); \
else \
{ \
w3c->parse_csv(); \
$EventTime = parsedate($date + " " + $time); \
$EventTime = strftime($EventTime, "%Y-%m-%dT%H:%M:%SZ"); \
$SourceName = "IIS"; \
}
</Input>


<Input EventLog_In>
# Use 'im_mseventlog' for Windows XP, 2000 and 2003
Module im_msvistalog
# Uncomment the following to collect specific event logs only
Query <QueryList>\
<Query Id="0">\
<Select Path="Application">*</Select>\
<Select Path="System">*</Select>\
<Select Path="Security">*</Select>\
</Query>\
</QueryList>
</Input>


<Output iis_out>
Module om_ssl
OutputType GELF_TCP
Host <FQDN>
CertFile C:\Program Files (x86)\nxlog\cert\GraylogWeb.pem
CertKeyFile C:\Program Files (x86)\nxlog\cert\GraylogWeb.key
CAFile C:\Program Files (x86)\nxlog\cert\graylogca.pem
Port 12201
</Output>

<Output EventLog_Out>
Module om_ssl
OutputType GELF_TCP
Host <FQDN>
CertFile C:\Program Files (x86)\nxlog\cert\GraylogWeb.pem
CertKeyFile C:\Program Files (x86)\nxlog\cert\GraylogWeb.key
CAFile C:\Program Files (x86)\nxlog\cert\graylogca.pem
Port 12201
</Output>

<Route IIS>
Path iis_in => iis_out
</Route>

<Route EventLog>
Path EventLog_In => EventLog_Out
</Route>

Breaking Down The Configuration File

<Extension w3c>
   Module xm_csv
   Fields $date, $time, $s-sitename, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs(User-Agent), $cs(Referer), $sc-status, $sc-substatus, $sc-win32-status, $sc-bytes, $cs-bytes, $time-taken
   FieldTypes string, string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer, string, string
   Delimiter ' '
   QuoteChar '"'
   EscapeControl FALSE
   UndefValue -
</Extension>

All logs created by IIS are in a CSV format with fields defined in the begging of the document. The fields need to each be listed under the “Fields” sectiong with a “$” put in front and a “,” separating each value. The type of field also needs to be defined in the section “FieldTypes” with one type for every field defined. NXlog is also going to parse our date and time and put them into a format that Graylog understands.

<Input iis_in>

   Module      im_file
   File        "C:\\W3SVC1\\u_ex*"
   SavePos     TRUE
     Exec            if $raw_event =~ /^#/ drop();                    \
               else                                             \
               {                                                \
                   w3c->parse_csv();                            \
                   $EventTime = parsedate($date + " " + $time); \
                   $EventTime = strftime($EventTime, "%Y-%m-%dT%H:%M:%SZ"); \
                   $SourceName = "IIS";                                         \
               }
</Input>

With the fields and field types defined a input for IIS logs needs to be created. A IIS server may have multiple IIS instances running on it. Each instance will have its own log file and a new input will need to be created for each log file. The “Input iis_in” can be copied and appended with a number to create a new input that points to a different log location.

With the configuration file and SSL certs copied over, restart the NXlog service and browse to C:\Program Files (x86)\nxlog\data\nxlog.log. The service should restart without errors.

Organizing our Data

Creating IIS Log Indexes

Browsing to the Graylog server, IIS logs are now coming into the system. To ensure the system stays healthy and easy to organize it is important to create a new index to route the IIS logs into.

Browsing to the Graylog server, the IIS logs are now coming in. Before doing anything else create a new IIS index for the IIS logs. For more detailed instructions on how to create indexes please see the instructions in part 3 of this blog series.

Key in on the SourceModuleName of iis_in as a field starting with iis_in was created for both types of Exchange logs when NXLog was configured.

Graylog has a built in parser for IIS logs so if everything worked our logs should be in the web server index with individual fields created.

With IIS logs being parsed it is possible to create alerts and dashboards based on the business needs and specific application capabilities.

Stay tuned for next week where VDA will go over the basics of collecting logs from a firewall and some basic alerts for firewalls.