Deploy Graylog using SaltStack and supporting formulas I ’ve written a formula for deploying Graylog with additional Salt formulas for Elasticsearch and MongoDB, which support a Graylog install. Currently, this has only been deployed on CentOS 7 so the Salt states are pretty specific to CentOS and RHEL based distros. There are plans to expand support to Debian distros in the future though. Both the Elasticsearch and MongoDB formulas can be used on their own, while the Graylog formula can be used in conjuncture with or deployed after the other requirements have been satisfied.

Content Packs for Graylog Lately, I have been working with Graylog a lot so I decided to update a few items on github and update their entries on the Graylog marketplace website. BRO content pack for Graylog The BRO IDS content pack contains pipeline rules, a stream, a dashboard displaying interesting activity, and a syslog tcp input to capture and index BRO logs coming from a Security Onion sensor. Fortinet content pack for Graylog The Fortigate UTM content pack contains extractors, a stream, a dashboard displaying the last 24 hours of activity, and a syslog tcp input.

Overview For anyone that doesn’t know, Security Onion is a custom Linux distribution running on Ubuntu that can be used as a Network Intrusion Detection System (NIDS). Security Onion integrates several configurable apps like BRO IDS, Snort, Suricata, and OSSEC to name a few. By default, there is an integrated ELSA Stack that can be configured, which makes SO a pretty interesting one-stop shop for getting your feet wet with IDS technology.

Tuning Graylog, Elasticsearch, and MongoDB for optimized cluster performance This has been an article a long time in the making. One problem with making changes to a complex clustered environment is that you may have to wait long periods of time to gather data that either shows an improvement or shows a negative impact. Some other considerations just make total sense, if you can afford them. Running on SSDs is going to perform far better than spinning disks.

Setup HA Cluster Services on CentOS 7 Install HA Cluster components Install pacemaker and the cluster control software on both nodes that will be part of the cluster(corosync is pulled in as a dependency) yum install pacemaker pcs Enable and start the cluster management service systemctl enable pcsd.service systemctl start pcsd.service Enable corosync and pacemaker to start on boot on all nodes systemctl enable corosync.service systemctl enable pacemaker.service Set the hacluster user’s password

Setup for Logging Setup rsyslog aggregator nodes (Optional) Setup Note: As part of the overall design, an HA cluster allows aggregating logs to the Central Log Repository with as little loss of logs as possible due to downtime or maintenance. Below are steps for building an HA cluster and setting up rsyslog for CENTOS 7. Install/upgrade to the latest rsyslog yum update rsyslog Create an rsyslog spool directory (this will be needed later)

Setup for Logging Setup rsyslog node Install/upgrade to the latest rsyslog yum update rsyslog Create an rsyslog spool directory (this will be needed later) mkdir /var/lib/rsyslog Setup Note: A custom rsyslog.conf is available for the CLR node that allows receiving logs on tcp port 514 by default. Copy the content from the appendixes into the appropriate files. Path: /etc/rsyslog.conf rsyslog.conf for CLR server Edit the rsyslog config vi /etc/rsyslog.conf Uncomment the lines for the action and change server.

Additional Setup for master node Setup Graylog Web UI on master node Setup Note: newer versions of graylog do not require a separate install for the web interface anymore so we can make a few firewall rule changes and be good. Configure Graylog WebUI firewalld rules Let ’s make some firewall rule changes specifically to allow web traffic. If for some reason you aren’t using a firewall then you can skip this.

Additional setup for master node Setup graylog-server on master node Install instructions from http://docs.graylog.org/en/2.2/pages/installation.html Setup Note: This deployment is not using a prebuilt rpm package, many of the next steps will be moving files, creating directories, creating additional files, and setting up the proper permissions on the linux command line. An rpm package is available but because when this guide for first written, the RPM only had support for openjdk v1.

Additional Setup for master node Install mongodb on master node Install instructions from https://docs.mongodb.com/manual/administration/install-on-linux/ Create repo file for mongodb vi /etc/yum.repos.d/MongoDB-3.4.repo Insert this text [mongodb-org-3.4] name=MongoDB Repository baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.4/x86_64/ gpgcheck=1 enabled=1 gpgkey=https://www.mongodb.org/static/pgp/server-3.4.asc</pre> Install mongodb yum install mongodb-org Set mongod to start on boot systemctl enable mongod .service Allow mongodb to use port 27017 for communication semanage port -a -t mongod_port_t -p tcp 27017 Start mongodb systemctl start mongod Setup Note: By default, mongodb does not enforce any kind of user authentication.