Anticipate cyber-threats with PatrOwl, manage them with TheHive

TL;DR: How to quickly setup a comprehensive SOC infrastructure based on open-source tools PatrOwl and TheHive.

This article is mostly inspired from a document written in french by Nicolas Schmitz, operational CISO at the ENS Lyon, a French public graduate school. See the original LinkedIn article here

The Tools

  • TheHive is a scalable 4-in-1 open source and free Security Incident Response Platform designed to make life easier for SOCs, CSIRTs, CERTs and any information security practitioner dealing with security incidents that need to be investigated and acted upon swiftly. It is the perfect companion for MISP. Thanks to Cortex, our powerful free and open source analysis engine, you can analyze (and triage) observables at scale using more than 100 analyzers.Officially supported by the StrangeBee team.

Introduction

Once the vulnerability is in the process, the security expert must follow the effective resolution of the incident, even though these circuits are often unsuitable for cyber-security incident management. This could be messy and hard to manage effectively in day to day operations. But that was before :)

In this article, we will install a set of open-source security tools which will work together thanks to an orchestrator which will bring them real added value.

3 scanners will be installed:

  • OpenVAS is an excellent general-purpose opensource scanner ideal for scanning entire subnets, performing authenticated scans for an extended vulnerability scanning. This product is used as a core system by many commercial software on the market.
  • Nmap will scan the infrastructure, and compare the versions of the applications with the vulnerability database. You’ll quickly find out if applications listening to a port on your network are vulnerable.
  • Arachni will be our web vulnerability scanner (SQL injections, XSS, file inclusion ..). Note that Nikto and Zap are also available.

This document covers only few use cases of the products capacities. It mostly focus on the technical aspect of the installation. A video is available to quickly show you a use case, once the platform is installed.

Prerequisites and system requirements

  • In the following parts, the VM will use the IP address ‘192.168.0.154’. Of course, it have to be replaced by the IP of your own VM.
  • Some of the operations described below must be performed as root.
  • This documentation does not cover changing the different default passwords, SSL/TLS encryption & certificates, as many things that will of course be necessary before going into production.

PatrowlManager installation

PatrowlManager dashboard

The official documentation is available here. We will use the Docker installation with a persistant storage of the DB.

apt-get install docker-composemkdir -p /data/patrowl_pg_data
mkdir -p /sources; cd /sources/
git clone https://github.com/Patrowl/PatrowlManager.git
cd PatrowlManager/

The Dockerfile has to be edited for enabling the persistant storage. Open the ‘docker-compose.yml’ file and add the following line (in bold):

...db:    
image: postgres:11-alpine
restart: always
container_name: patrowl-postgres
environment:
- POSTGRES_DB=patrowl_db
- POSTGRES_USER=PATROWL_DB_USER
- POSTGRES_PASSWORD=PATROWL_DB_PASSWD_TO_CHANGE
expose:
- '5432'
volumes:
- ./var/db/init_db.sql:/docker-entrypoint-initdb.d/init_db.sql
- /data/patrowl_pg_data:/var/lib/postgresql/data/
...

Now build the docker image from source and run the container :

docker-compose build — force-rm
docker-compose up -d

Tip: It is also possible to use the official Docker images available here on Docker Hub

You are now able to connect to the PatrowlManager Web GUI at http://192.168.0.154:8083 with the default credentials : admin/Bonjour1!

By default, PatrowlManager is like a general without an army. So we will provide him with a few soldiers, called here PatrowlEngines or simply “engines”.

PatrowlEngines installation

cd /sources/ ;
git clone https://github.com/Patrowl/PatrowlEngines.git

Each engine will be available in ‘/sources/PatrowlEngines/engines’ dir.

OpenVAS installation

mkdir -p /data/openvas/
docker run --rm -d -p 4443:443 -p 9390:9390 -e PUBLIC_HOSTNAME=192.168.0.154 -v /data/openvas/:/var/lib/openvas/mgr/ -name openvas mikesplain/openvas:9

Please note: OpenVAS takes several minutes to initialize (it takes a long time to retrieve signatures). Do not hesitate to go for a coffee! You can check the progress in logs or with a ‘top’ :

docker exec -it openvas top

Hint: As long as the CPU is at 100% you have to wait :-)

You can now connect to https://192.168.0.154:4443 with default credentials admin/admin and access a blank and old-fashioned OpenVAS console:

OpenVAS console — 90’s style

No further configuration is needed here ! PatrowlManager and its engine will be in charge to configure and start the scans.

OpenVAS Engine installation

cd /sources/PatrowlEngines/engines/openvas/
cp openvas.json.sample openvas.json
vi openvas.json

Update the gmp_host parameter in the openvas.json file to set you IP address:

...
gmp_host : 192.168.0.154

Build and run the OpenVAS engine:

cd /sources/PatrowlEngines/engines/openvas/
docker build — tag "patrowl-openvas" .
docker run -d --rm -p 5116:5016 -v /sources/PatrowlEngines/engines/openvas/openvas.json:/opt/patrowl-engines/openvas/openvas.json --name="openvas-engine" patrowl-openvas

Configuration of OpenVAS engine in PatrowlManager

Hint #1: Do not forget the final slash ‘/’

Hint #2: Consider using the IP address when you work with Docker containers running in separate contexts. Avoid using localhost or 0.0.0.0

The engine must then appear in the `Engines/List Engines` menu, with Oper.State at “Ready”.

Nmap Engine installation

cd /sources/PatrowlEngines/engines/nmap
docker build -tag "patrowl-nmap" .
docker run -d --rm -p 5101:5001 --name="nmap-engine" patrowl-nmap

That’s all ? Yes.

Configuration of Nmap engine in PatrowlManager

http://192.168.0.154:5101/engines/nmap/

Then ensure the Operational State (Oper.State) is set to ‘Ready’.

Arachni Engine installation

cd /sources/PatrowlEngines/engines/arachni
cp arachmi.json.sample arachmi.json
docker build --tag "patrowl-arachni" .
docker run -d --rm -p 5105:5005 -v /sources/PatrowlEngines/engines/arachni/arachni.json:/opt/patrowl-engines/arachni/arachni.json --name="arachni-engine" patrowl-arachni

Configuration of Arachni engine in PatrowlManager

http://192.168.0.154:5105/engines/arachni/

Then ensure the Operational State (Oper.State) is set to ‘Ready’.

Note: Arachni scanner consumes a lots of CPU/RAM resources (F*ck Ruby). Don’t panic if your fan is over-running.

TheHive installation — The quick and easy way

Here is a (quick and dirty) simple way to get started:

mkdir -p /data/elasticsearch /data/thehiveconf /sources/thehive
chmod 770 /data/elasticsearch
wget https://gist.github.com/MaKyOtOx/f2e727a4d9d2f7d9a3b6f1f4c7bd15ae -O /sources/thehive/docker-compose.yml
wget https://raw.githubusercontent.com/TheHive-Project/TheHive/master/conf/application.sample -O /data/thehiveconf/application.conf

Then edit the /data/thehiveconf/application.conf and update:

  • play.http.secret.key’ with a random value
  • uri’ with the value “http://192.168.0.154:9200/” (ElasticSearch)

Now it’s time to download and run TheHive:

cd /sources/thehive/
docker-compose up -d

Note: Cortex is not installed in this tutorial. But feel free to install, use it massively and contribute with your own analyzers.

After a few moments, you can access http://192.168.0.154:9000/ and click on the “Update Database” button. After initializing its Elastic databases, TheHive will offer you the creation of an admin account then you will be able to discover the interface.

Make PatrOwl and TheHive working together

Create an API Key on TheHive

Add new user ‘api’ in TheHive

Note: Do not forget to check “Allow alerts creation”

Once the user is registered, click on his “Create API Key” button, then “Reveal:” copy the displayed value.

Configure TheHive alerts on PatrowlManager

  • In the ‘Key’ field, indicate: alerts.endpoint.thehive.apikey
  • In the ‘Value’ field, paste the value of your API Key generated in TheHive

Repeat the operation for the url, with the following values:

Repeat the operation to use it, with the following values:

  • In the ‘Key’ field, enter alerts.endpoint.thehive.user
  • In the ‘Value’ field, indicate api

Configure alerting rules on PatrowlManager

It is necessary to repeat these steps for low or medium alerts if needed. It is a one-time operation ! You got it.

Oh, that’s all :) If all commands worked like a charm until now, you should be able to start scans and receive alerts.

Conclusion

During the migration phase, it will be necessary to redefine in Patrowl the set of scan targets that were previously scattered in different tools. CSV import modules are available and if you change the scanner tomorrow, you will no longer have to do this :)

These projects are open-source, the community is very active worldwide and already adopted by medium and large organizations, MSSP and global SOC and CERT/CSIRT teams.

If you do not have enough manpower in system administration, do not hesitate to be accompanied by the Patrowl team which offer help for the deployment! A SaaS edition is also available now.

Several interesting use cases that should be presented too:

  • Usage of the Cortex analyzer for PatrOwl (get asset or scan reports)
  • Usage of Cortex analyzers directly from Patrowl using the engine
  • MISP integration for sharing findings and assets

Bonus: the All-in-One script

WTF man !?! Why didn’t you start there !!

Find below an example of a script for starting all components on startup.

#!/bin/shecho "Starting OpenVAS scanner"/usr/bin/docker run -d --rm -p 4443:443 -p 9390:9390 -e PUBLIC_HOSTNAME=192.168.0.154 -v /data/openvas/:/var/lib/openvas/mgr/ --name openvas mikesplain/openvas:9echo "Starting OpenVAS engine"/usr/bin/docker run -d --rm -p 5116:5016 -v /sources/PatrowlEngines/engines/openvas/openvas.json:/opt/patrowl-engines/openvas/openvas.json --name="openvas-engine" patrowl-openvasecho "Starting Nmap engine"/usr/bin/docker run -d --rm -p 5101:5001 --name="nmap-engine" patrowl-nmapecho "Starting Arachni engine"/usr/bin/docker run -d --rm -p 5105:5005 -v /sources/PatrowlEngines/engines/arachni/arachni.json:/opt/patrowl-engines/arachni/arachni.json --name="arachni-engine" patrowl-arachniecho "Starting TheHive"/usr/bin/docker-compose -f /sources/thehive/docker-compose.yml up -decho "Starting PatrowlManager"/usr/bin/docker-compose -f /sources/PatrowlManager/docker-compose.yml up -d

Final Greetz

  • Nabil Adouani, Thomas Franco and Jérôme Léonard, the founders of StrangeBee and co-creators of TheHive and Cortex
  • My awesome team at Patrowl and GreenLock Advisory, working hard to build useful security products and deliver high-quality services
  • Our customers and contributors ❤️
  • Nicolas Schmitz for the original document and his contributions on the project :)

PatrOwl is also available on SaaS. Looking for official Support or Professional Services ? Contact getsupport@patrowl.io or visit https://patrowl.io for more information.

— Nicolas MATTIOCCO, CEO of PatrOwl and GreenLock Advisory

This is not an Owl

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store