Anticipate cyber-threats with PatrOwl, manage them with TheHive

The Tools

  • PatrOwl is a scalable, smart and open-source solution for orchestrating Security Operations. Written in Python, PatrOwl is composed of a Front-end application PatrowlManager communicating with one or multiple PatrowlEngines micro-applications. These probes perform the scans, analyze the results and format them in a normalized way. It remains incredibly easy to customize all components. Several connectors are already publicly available, including Nessus, OpenVAS, Nmap, SSLScan, Arachni, DroopScan, Sublist3r, SonarQube, CertStream, … Officially supported by the PatrOwl team;
  • TheHive is a scalable 4-in-1 open source and free Security Incident Response Platform designed to make life easier for SOCs, CSIRTs, CERTs and any information security practitioner dealing with security incidents that need to be investigated and acted upon swiftly. It is the perfect companion for MISP. Thanks to Cortex, our powerful free and open source analysis engine, you can analyze (and triage) observables at scale using more than 100 analyzers.Officially supported by the StrangeBee team.

Introduction

This document is intended for operational CISO and security engineers who must juggle several “vulnerability scanners”, each with their own capacities, their own repository of assets to scan, their own reports, etc. Once the scan policies have been defined and implemented in each tool, the security expert must still, often manually, inject the vulnerabilities found in a circuit allowing remediation: ticketing platform, emails or the famous shared .xls files.

  • OpenVAS is an excellent general-purpose opensource scanner ideal for scanning entire subnets, performing authenticated scans for an extended vulnerability scanning. This product is used as a core system by many commercial software on the market.
  • Nmap will scan the infrastructure, and compare the versions of the applications with the vulnerability database. You’ll quickly find out if applications listening to a port on your network are vulnerable.
  • Arachni will be our web vulnerability scanner (SQL injections, XSS, file inclusion ..). Note that Nikto and Zap are also available.

Prerequisites and system requirements

  • We will install the entire solution on a single virtual machine in Debian Buster. The setup is comfortable with at least 4 CPUs, 16GB of RAM, and 64GB of SSD disk space. Production installations should be designed regarding the quantity of elements scanned and your resilience needs.
  • In the following parts, the VM will use the IP address ‘192.168.0.154’. Of course, it have to be replaced by the IP of your own VM.
  • Some of the operations described below must be performed as root.
  • This documentation does not cover changing the different default passwords, SSL/TLS encryption & certificates, as many things that will of course be necessary before going into production.

PatrowlManager installation

PatrowlManager dashboard
apt-get install docker-composemkdir -p /data/patrowl_pg_data
mkdir -p /sources; cd /sources/
git clone https://github.com/Patrowl/PatrowlManager.git
cd PatrowlManager/
...db:    
image: postgres:11-alpine
restart: always
container_name: patrowl-postgres
environment:
- POSTGRES_DB=patrowl_db
- POSTGRES_USER=PATROWL_DB_USER
- POSTGRES_PASSWORD=PATROWL_DB_PASSWD_TO_CHANGE
expose:
- '5432'
volumes:
- ./var/db/init_db.sql:/docker-entrypoint-initdb.d/init_db.sql
- /data/patrowl_pg_data:/var/lib/postgresql/data/
...
docker-compose build — force-rm
docker-compose up -d

PatrowlEngines installation

The official documentation is available here. We will use the Docker installation as well.

cd /sources/ ;
git clone https://github.com/Patrowl/PatrowlEngines.git

OpenVAS installation

We use the mikesplain/openvas Docker image with the options that are good for being able to connect to it and have data persistence:

mkdir -p /data/openvas/
docker run --rm -d -p 4443:443 -p 9390:9390 -e PUBLIC_HOSTNAME=192.168.0.154 -v /data/openvas/:/var/lib/openvas/mgr/ -name openvas mikesplain/openvas:9
docker exec -it openvas top
OpenVAS console — 90’s style

OpenVAS Engine installation

cd /sources/PatrowlEngines/engines/openvas/
cp openvas.json.sample openvas.json
vi openvas.json
...
gmp_host : 192.168.0.154
cd /sources/PatrowlEngines/engines/openvas/
docker build — tag "patrowl-openvas" .
docker run -d --rm -p 5116:5016 -v /sources/PatrowlEngines/engines/openvas/openvas.json:/opt/patrowl-engines/openvas/openvas.json --name="openvas-engine" patrowl-openvas

Configuration of OpenVAS engine in PatrowlManager

Go back to http://192.168.0.154:8083/, menu `Engines/+Add scan engine instance` and set the API URL of the fresh Docker container :

Nmap Engine installation

No need to install Nmap, it’s embedded within the engine

cd /sources/PatrowlEngines/engines/nmap
docker build -tag "patrowl-nmap" .
docker run -d --rm -p 5101:5001 --name="nmap-engine" patrowl-nmap

Configuration of Nmap engine in PatrowlManager

Exactly the same as the OpenVAS engine with the following API URL :

http://192.168.0.154:5101/engines/nmap/

Arachni Engine installation

The web application scanner Arachni is embedded in the Docker image too. Build and run the Arachni engine :

cd /sources/PatrowlEngines/engines/arachni
cp arachmi.json.sample arachmi.json
docker build --tag "patrowl-arachni" .
docker run -d --rm -p 5105:5005 -v /sources/PatrowlEngines/engines/arachni/arachni.json:/opt/patrowl-engines/arachni/arachni.json --name="arachni-engine" patrowl-arachni

Configuration of Arachni engine in PatrowlManager

Exactly the same as the other engines with the following API URL :

http://192.168.0.154:5105/engines/arachni/

TheHive installation — The quick and easy way

A lot of very good and comprehensive articles already talk about TheHive & Cortex installation (see here or here).

mkdir -p /data/elasticsearch /data/thehiveconf /sources/thehive
chmod 770 /data/elasticsearch
wget https://gist.github.com/MaKyOtOx/f2e727a4d9d2f7d9a3b6f1f4c7bd15ae -O /sources/thehive/docker-compose.yml
wget https://raw.githubusercontent.com/TheHive-Project/TheHive/master/conf/application.sample -O /data/thehiveconf/application.conf
  • play.http.secret.key’ with a random value
  • uri’ with the value “http://192.168.0.154:9200/” (ElasticSearch)
cd /sources/thehive/
docker-compose up -d

Make PatrOwl and TheHive working together

Here is a basic use case: PatrOwl continuously run security scans on assets. If a new vulnerability with a high severity is found, send an alert to TheHive. The security analyst will be aware of a new security incident and start his investigation. Let’s make this happen.

Create an API Key on TheHive

In order for Patrowl to “push” alerts into TheHive, you must first generate and retrieve an API key there. In your TheHive instance, go to the “Admin” / “Users” menu. Click on “Add User” then indicate the following:

Add new user ‘api’ in TheHive

Configure TheHive alerts on PatrowlManager

Now, in PatrowlManager, go to the “Admin” / “Settings” menu then click on “Add new setting”:

  • In the ‘Key’ field, indicate: alerts.endpoint.thehive.apikey
  • In the ‘Value’ field, paste the value of your API Key generated in TheHive
  • In the ‘Key’ field, enter alerts.endpoint.thehive.user
  • In the ‘Value’ field, indicate api

Configure alerting rules on PatrowlManager

Still in PatrowlManager, we will have to establish rules for the export of findings to TheHive: Go to the “Rules” / “List rules” menu then fill as below:

Conclusion

Such a platform may seem like a Health Robinson machine. It should rather be seen as an assembly of dedicated tools in the pure KISS philosophy, scalable and adjusted to your needs and IT ecosystem. The maintenance workload, updates, etc. should not be underestimated, but these operations are greatly simplified thanks to the use of Docker containers.

  • Usage of the Cortex analyzer for PatrOwl (get asset or scan reports)
  • Usage of Cortex analyzers directly from Patrowl using the engine
  • MISP integration for sharing findings and assets

Bonus: the All-in-One script

#!/bin/shecho "Starting OpenVAS scanner"/usr/bin/docker run -d --rm -p 4443:443 -p 9390:9390 -e PUBLIC_HOSTNAME=192.168.0.154 -v /data/openvas/:/var/lib/openvas/mgr/ --name openvas mikesplain/openvas:9echo "Starting OpenVAS engine"/usr/bin/docker run -d --rm -p 5116:5016 -v /sources/PatrowlEngines/engines/openvas/openvas.json:/opt/patrowl-engines/openvas/openvas.json --name="openvas-engine" patrowl-openvasecho "Starting Nmap engine"/usr/bin/docker run -d --rm -p 5101:5001 --name="nmap-engine" patrowl-nmapecho "Starting Arachni engine"/usr/bin/docker run -d --rm -p 5105:5005 -v /sources/PatrowlEngines/engines/arachni/arachni.json:/opt/patrowl-engines/arachni/arachni.json --name="arachni-engine" patrowl-arachniecho "Starting TheHive"/usr/bin/docker-compose -f /sources/thehive/docker-compose.yml up -decho "Starting PatrowlManager"/usr/bin/docker-compose -f /sources/PatrowlManager/docker-compose.yml up -d

Final Greetz

Thanks again to:

  • Nabil Adouani, Thomas Franco and Jérôme Léonard, the founders of StrangeBee and co-creators of TheHive and Cortex
  • My awesome team at Patrowl and GreenLock Advisory, working hard to build useful security products and deliver high-quality services
  • Our customers and contributors ❤️
  • Nicolas Schmitz for the original document and his contributions on the project :)
This is not an Owl

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store