Install PatrowlHears — Vulnerability Intelligence Center
The Patrowl.io community provides scalable, free and open-source solutions for orchestrating Security Operations and providing Threat Intelligence feeds. PatrowlHears provides a unified source of vulnerability, exploit and threat Intelligence feeds. Users accesses a comprehensive and continuously updated vulnerability database scored and enriched with exploit and threat news information. These metadata are collected from public OSINT and private feeds. As today, it’s one of the most extended database.
See full article on medium here.
Hardware minimum requirements
PatrowlHears uses PosgreSQL to store data and RabbitMQ to process queues. We recommend using a virtual machine with at least 2vCPU, 8 GB of RAM and 50 GB of storage disk. You can also use a physical machine with similar specifications.
Installation steps
The very easy way: Use Docker
- Install Docker and Docker-compose
- Run the docker stack:
git clone https://github.com/Patrowl/PatrowlHears
cd PatrowlHears
docker-compose up
- Open your browser on http://localhost:8383
- Load initial DB records and latest updates (ETA: ~2h):
docker-compose exec patrowlhears bash -c 'cd backend_app && ./load_init_data.sh'
docker-compose exec patrowlhears bash -c 'cd backend_app && ./import_data_updates.sh'
- Regularly update DB with command:
docker-compose exec patrowlhears bash -c 'cd backend_app && ./import_data_updates.sh'
The easy way: Use installation script
- Install Git and Python3 on your server and un the commands:
git clone https://github.com/Patrowl/PatrowlHears
cd PatrowlHears
./install.sh
- Open your browser on http://localhost:8383
- Regularly update DB with command:
cd backend_app && ./import_data_updates.sh
The DevOps way: Install and deploy from Ansible playbook
- Go to the playbook location
cd deploy/ansible/playbooks
- Copy and update the sample file
ansible/vars.yml.sample
toansible/vars.yml
- Run the Ansible playbook:
ansible-playbook patrowlhears.yml -t patrowlhears-install -i myhost,
Note 1: Do not forget to update Ansible vars and default ansible.cfg options. Note 2: Do not forget the comma ‘,’ after the hostname/ip (well-known Ansible inventory trick).
The Nerd way: Install and deploy from sources
The following section contains a step-by-step guide to build PatrowlHears from its sources.
1. Install system pre-requisites
The following software are required to download and run PatrowlHears:
We strongly recommend to use the system packages. To install the requirements and run PatrowlHears from sources, please follow the instructions below depending on your operating system.
1.1. MacOS/X with Brew
brew update
brew install postgres python3 rabbitmq npm
python -m ensurepip
pip install virtualenv
1.2. Ubuntu 16.04/18.04/20.04 LTS with APT
sudo apt update
sudo apt upgrade -y
sudo apt install -y build-essential python3 python3-dev git curl rabbitmq-server postgresql postgresql-client nodejs libpq-dev nginx
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python3 get-pip.py
rm get-pip.py
sudo pip3 install virtualenv
1.3. CentOS/RHEL with YUM
sudo yum install -y git python3 python3-pip python3-virtualenv rabbitmq-server postgresql postgresql-client npm postgresql-devel
2. Download PatrowlHears from GitHub
git clone https://github.com/Patrowl/PatrowlHears
3. Build the frontend application (VueJS) — Optional
cd PatrowlHears/frontend
npm install
npm run build
4. Install python dependencies within the virtualenv
cd ../backend_app
python3 -m virtualenv env
source env/bin/activate
pip install -r requirements.txt
Note 1: If
python3 -m virtualenv env
does not work, please consider the commandvirtualenv env
but ensure that Python3 is selected.
Note 2: Be careful, next commands MUST be launched within the python3 virtual environment. The prefix(env3)
should appear in the command prompt. Ex:
(env) GreenLock@GL02:PatrowlHears$ ls
If you opened another terminal, please ensure you use the virtualenv with the command source env3/bin/activate
. If you want to exit the virtual environment, use the command deactivate
. If not, do nothing and stop asking.
Note 3: for MacOs users, install pythons modules from
requirements.macos.txt
file.
5. Create the PostgreSQL database
- Edit file the
var/db/create_user_and_db.sql
and update the user and password values (default values are: 'patrowlhears
' and 'patrowlhears
').
Note: You should consider to set a stronger password in Production.
5.1. MacOS
- Execute the SQL script:
psql < var/db/create_user_and_db.sql
5.2. Ubuntu 16.04/18.04/20.04 and CentOS/RHEL
- Execute the SQL script:
sudo -u postgres psql < var/db/create_user_and_db.sql
Note: By default, the script create the database ‘patrowlhears_db’ with the user/role ‘patrowlhears’. The default password is ‘patrowlhears’. If you change these settings, do not forget to update the
backend_app/settings.py
configuration file with your updates, or pass it it using environment variables.
6. Configure PatrowlHears (Django backend) application
- Copy
backend_app/settings.py.sample
tobackend_app/settings.py
and update at least following options:
Application settings ALLOWED_HOSTS
, LOGGING_LEVEL
, PROXIES
, SECRET_KEY
DB settings (service location and credentials): DATABASES
,
RabbitMQ settings (service location and credentials): BROKER_URL
(default values are guest/guest
),
Email settings (alerting): EMAIL_HOST
, EMAIL_HOST_USER
, EMAIL_HOST_PASSWORD
, EMAIL_HOST_PORT
- Create the db schema using the Django commands:
python manage.py makemigrations
python manage.py migrate
- Collect static files (production mode — all static files copied to /staticfiles/ and will be served by NGinx):
python manage.py collectstatic --noinput
- Create the Django superuser with all privileges, more than Batman but without a cape:
python manage.py shell < var/bin/create_default_admin.py
Note 1: Default login is
admin
and password isBonjour1!
.
Note 2: You are in charge to renew the password once the application started. Please keep these credentials in a safe place. This account will be used for the first login on the PatrowlHears application.
- Create the default organization:
python manage.py shell < var/bin/create_default_organization.py
7. Start the Django backend server7.1 Testing environment
- Start Supervisord (Celery workers consuming the tasks enqueued RabbitMQ — Yes, that’s how asynchronus tasks work here):
supervisord -c var/etc/supervisord.conf
Note: The Supervisor daemon will be listening on port TCP/9002. Update this in the configuration file if you are not agree with that arbitrary choice. Who really cares ?
- Check every celery workers are in state
RUNNING
:
supervisorctl -s http://127.0.0.1:9002 status all
- Then, the Django application (development only):
python manage.py runserver 127.0.0.1:8000
- or, using Gunicorn (recommended in production):
gunicorn backend_app.wsgi:application -b 127.0.0.1:8000 --access-logfile -
7.2 Production environment (Nginx serving static files)
- Open the
backend_app/settings.py
file and set the variableDEBUG=False
. - Follow the same steps for starting the development environment (see #7.1)
- Customize the
nginx.conf
file provided. Then start it:
[sudo] nginx -p .
Note: By default the WEB pages is exposed from port TCP/8383
8. Load initial DB fixtures and latest updates
- Run following commands (in path
backend_app
):
./load_init_data.sh
./import_data_updates.sh
Need help ? Stuck somewhere ?
Don’t panic! The community could help you as soon as you double-checked your issue and its undoubtedly related to PatrowlHears installation:
- Contact us at
getsupport@patrowl.io
, or - Chat with us on Gitter
- Star us on Github
- Follow us on Twitter and LinkedIn
Cheers !