A Short Story of Getting Work Done: TheHive 3.3.0

Some of our die-hard fans noticed that we silently released TheHive 3.3.0 a few days ago, after six release candidates. Well. Silently won’t be the right word to use in this case as we are drowning under work and feature requests and we sometimes postpone communication in favour of getting true real work done.

So, without any further ado, we are happy to announce the official availability of our latest (and of course greatest) release of the most-advanced, next-gen, HI (Human Intelligence), gluten-free, (add here any keyword that you fancy to help us get the Gartner attention and land in the Magic Quadrant™), free and open source Security Incident Response Platform Security Orchestration Automation & Response Platform.

As stated earlier, TheHive 3.3.0 went through the largest number of release candidates to date in order to ensure it contains more features than bugs (or unexpected functionality as our dear Nabil call them sometimes).

Since RC5 which we have blogged about on Feb 26, 2019, below is an outline of the changes we made. Check TheHive Installation Guide for installation instructions.

The new Related Alerts tab in Case View introduced in 3.3.0-RC5

Fixed Issues

  • #899: fix a crashing issue encountered with Firefox
  • #907: dynamic (auto-refresh) of cases was broken in 3.3.0-RC5
  • #930: merging cases by CaseID was broken

Implemented Enhancements

  • #666: add support for filtering tags by prefix and wildcard search
  • #901: remove the possibility of creating cases from scratch (i.e. empty cases) when explicitly disabled by an admin
  • #908: add support for text-based widgets to the dashboards
  • #912: responders can now add tags to alerts when triggered thanks to the new AddTagToAlert operation

Looking for Help?

Something does not work as expected? Then please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org.

Since You are Here

TheHive and Cortex are a huge success. According to our estimates, there are about a hundred organisations of different sizes and locations using or testing them. And as the number of users grows, so does the number of features, professional service and support requests. 

We tried finding a solution to keep TheHive Project as healthy as possible. So we created Creative Source, a nonprofit organisation (NPO), in the hope that we could leverage it to hire more developers thanks to the generous donations of our large user community. Sadly, not everyone in this world is generous and altruistic. At the end, all but one company (yes, exactly one) trusted us enough to make a donation and get tailored services for its needs in return. Most of the others either did not reply to our proposals or explained that their procurement process does not accommodate working with NPOs.

As we informed you a few weeks ago, some members of our core team are finalising an alternative option to ensure not only the viability of TheHive and Cortex as FOSS products on the long run but the ability to provide professional training, support, and services without making highly bureaucratic, think-in-the-box-but-never-outside procurement departments freak out.

Expect to hear from us soon…

On Spring, Bees and Cortex-Analyzers 1.16.0

Spring is here and your favorite bees are busy buzzing flowers to prepare you the most palatable honey ever. In the meantime, we are pleased to announce the immediate availability of Cortex-Analyzers 1.16.0, which adds a new responder and three new analyzers to complete an already hefty collection, bringing the total to 117 analyzers and 3 responders!

Release Overview

One responder has been added:

  • FalconCustomIOC, contributed by Michael (#421). We don’t know the last name of Michael. That could be Jordan. Who knows?

Three analyzers have been added:

We could not duly test some of these additions due to lack of access to the associated services or to our legendary laziness. So we would really appreciate it if you could test them and let us know whether they work or not.

FalconCustomIOC

The FalconCustomIOC responder let you submit observables from alerts or cases to Crowdstrike Falcon Custom IOC service.

Crowdstrike Falcon is a paid service. An account and an API key are required to configure and run this responder.

AbuseIPDB

AbuseIPDB analyzer let you determine wether an IP has been reported as malicious or not to the AbuseIPDB web service.

An account and an API key is needed to configure and use this analyzer.

TheHive displays the analyzer results as follows:

AbuseIPDB: short report
AbuseIPDB: long report

BackscatterIO

The BackscatterIO analyzer lets you query the Backscatter.io service for IPs, networks or autonomous systems (AS). It comes in two flavors:

  • BackscatterIO_GetObservations: determine whether an observables has a known scanning activity
  • BackscatterIO_Enrichment: enrich your observables with additional information

TheHive displays this analyzer results as follow:

Backscatter.io:short report
Backscatter.io GetObservations: long report
Bascatter.io GetObservations: long report

SoltraEdge

SoltraEdge analyzer lets you query any observable against theSoltra Edge platform.

To configure and use this analyzer, an account, a token key and the base URL of a SoltraEdge server are needed.

TheHive displays this analyzers result as follow:

SoltraEdge: short report
SoltraEdge: long report

Get It While Supply Lasts!

Each analyzer and responder comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.16.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Cortex-Analyzers 1.15.3 get ready for URLhaus and Cuckoo

Few improvements have been introduced in this version :

  • Proofpoint analyzer has been updated to use python3 (#417)
  • Long report of Cuckoo Sandbox analyzer has been improved to be able to display Cuckoo v. 2.0.6 reports (#418)
  • URLhaus analyzer has also been updated to use the recently introduced API (#431)

On your Cortex server, update your analyzers with the following command:

$ cd /opt/Cortex-Analyzers && git pull
$ for I in analyzers/*/requirements.txt; do sudo -H /usr/bin/python2 -m pip install -r $I
$ for I in analyzers/*/requirements.txt; do sudo -H /usr/bin/python3 -m pip install -r $I

Updating analyzers should be followed by also updating report templates. Download new report templates and add the archive in TheHive report templates.

Detailed information

Cuckoo Sandbox

Thanks to @nicpenning Cuckoo analyzer is now able to display reports from version  2.0.6 of Cuckoo. With this version, remote connections part of the report has been fixed and is now well displayed

The new cuckoo analyzer template

Notice: This fix has been reviewed by our core team, unfortunately, we have not been able to test it.

URLhaus

Abuse.ch, the operator of URLhaus recently introduced a new API for their service in order to handle bulk queries and reduce costs for their main page that uses CDNs for delivery. Because of that, thanks to Nils Kuhnert, the URLhaus analyzer has been rewritten from scratch. Also, the templates changed because there are three different API endpoints used:

The short reports now display the threat type – which currently is always malware_download and, for the hash observable type, the malware family – if given by URLhaus:

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Here comes the Sun (and TheHive 3.3-RC5)

On February 10, 2019, we released TheHive 3.3-RC2. It contained new features such as bulk alert merging, alert sorting, observable tag autocompletion, exporting case tags to MISP & more. Since then your favourite French code Chefs have been beesy refining TheHive 3.3 through new release candidates while getting Cortex 3 ready for prime time.

Over the weekend, Nabil decided he was not working enough already during the week. So he drained his batteries to the very last drop to release TheHive 3.3-RC5 before he crashed headfirst into his bed for a long, reparative sleep. Cumulatively since RC2, we added several features and squashed 10 bugs as described below.

Note that release candidates are beta software. You can get TheHive 3.3-RC5 from the pre-release, beta repositories. As usual, we encourage you to test it and report any bugs or issues you spot so we can address them before the final release. 

Check TheHive Installation guide for further details.

New Features

  • #485: disable clickable widgets when editing a dashboard.
  • #820: provide direct access to MISP events for those alerts that have been generated from such events.
Direct access to the MISP event from which this alert has been generated from
  • #829: improve password-protected ZIP imports.
  • #831: add auto-completion to case and observable tags in their respective Details pages.
  • #838: when you add an observable that already exists in another case, indicate whether the existing observable is an IOC or not.
  • #870: provide a link to copy the alert ID to the clipboard which allows you to easily manipulate an alert through TheHive4py.
Now the Alert ID can be directly copied to the clipboard without resorting to your browser dev mode
  • #884: add a Related Alerts tab to the Case view.
The new Related Alerts tab in Case View
  • #888: add a new UI configuration admin section. One of the first use cases of this section consist in disabling creating empty cases (i.e. cases not associated with a template). It will be gradually improved with new use cases so speak your mind!
  • #893: disable the case template selection when trying to merge multiple alerts for which no case template exists.
  • #897: mouseover text for alert preview.

Fixed Bugs

  • #856: various fixes related to alert updates and, when applicable, associated cases.
  • #869: use the observable TLP instead of the case TLP to decide whether a Cortex responder could be executed or not.
  • #871: fix log messages for MISP synchronisation.
  • #874: fix typo in alert labels (credits: Zachary Priddy).
  • #877: fix Akka version mismatch (credits: Zachary Priddy).
  • #890: the Hide Empty Case button in the new UI admin section was broken.
  • #894: when clicking on a donut or a widget in the dashboards, this should take you to the search page with predefined filters, without generating errors.
  • #895: exclude responder jobs from search results.
  • #896: dashboard clicks are not correctly translated into tag filters.

Looking for Help?

Something does not work as expected? Then please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org.

The Dockerization Will not be Televised

Do you know what the following set of commands achieve?

$ cd /opt/Cortex-Analyzers
$ sudo git pull
$ for I in $(find /opt/Cortex-Analyzers -name 'requirements.txt'); do sudo -H pip2 install -U -r $I; done \
&& for I in $(find /opt/Cortex-Analyzers -name 'requirements.txt'); do sudo -H pip3 install -U \
-r $I || true; done

The answer is obvious Doctor Watson, right? These highly readable commands (pun intended) allow you to update your Cortex analyzers and responders to the latest stable versions, downloading new ones in the process, going over all the Python 2 and Python 3 dependencies to install the missing ones and upgrade the old ones to make sure they work correctly. These operations take quite a long time and cause some headaches in the process (Hello, I have Python 3.X and this dependency is no longer required, or Hi, I have an old version of Python 2 and it seems I need this other dependency).

And if you are lucky enough to get it running smoothly, you are still not done as you need to log in to the Cortex UI as an organisation administrator (unlike TheHive, Cortex supports multi-tenancy), click on the Refresh analyzers button under Organization > Analyzers then go to Organization > Responders and click on Refresh responders.

So while the answer to the opening question might be simple, updating analyzers and responders is far from being straightforward, to say the least, even if we forget the ugly fact that both are stored in a repository “conveniently” named Cortex-Analyzers*:

thehive@thehive-training:/opt/Cortex-Analyzers$ ls -d a* r*
analyzers  responders

Unnecessary Complexity Must Die

Your lovely, hard-working bees hate unnecessary complexity. Our project’s front page blatantly states our mission to bring Security Incident Response to the masses. And we have to stand by our words even if TheHive and Cortex are free, open source solutions and we do not gain anything from them save for the huge satisfaction of helping our fellow incident handlers level the fight against cybercriminals & all kinds of other animals of the APT (Advanced Persistent Troll Threat) bestiary.

There is only one possible solution: simplify the installation and update process of the current, official 115 analyzers and responders we have as of this writing, the future ones and any private or unofficial ones written in other programming languages such as those developed in Go by Rosetelecom-CERT.

Docker all the Things!

Starting from Cortex 3.0, the next major release of your favourite analysis and active response engine, all analyzers and responders will be dockerized. It will no longer be necessary to install them along with their various dependencies. They will be dowloaded from our cortexengine Docker organisation. Sysadmins might also configure automatic updates.

As a side advantage of using Docker, analyzers, and responders will also be isolated from each other which gives more flexibility and possibilities.

© Steve Simson. This artwork and the title of this blog post are inspired by The Revolution will not be televised, a song from the late and great Gil Scott-Heron.

For those users who have private, custom analyzers and responders that they don’t want or can’t share with the community, several options will be available:

  • Continue managing their analyzers and responders in the same way as currently supported by Cortex 2 (i.e. launch them as processes, with no isolation whatsoever).
  • Dockerize them and store them locally on their Cortex instance.
  • Dockerize them and publish them on a Docker registry, either the official one or a private registry.

A Docker image of Cortex 3 will still be provided. It will contain a Docker engine to launch dockerized analyzers and responders using DIND (Docker in Docker).

It won’t be necessary to modify the code of the current, official analyzers and responders. A drone job will monitor the analyzer and responder repository and automatically build docker images when it detects changes.

The Cortex Web interface will be slightly modified to accommodate the whole process and allow adding in-house/private Certificate Authorities to allow Cortex to smoothly perform updates in those corporate environments where TLS/SSL inspection is enabled.

Nice Movie Trailer. When is it Coming to a Theatre near me?

We are working hard to get Cortex 3 out of the oven in Q1 (of this year, yes). We will reach out to you, dear reader, in due time, to help us test it and refine it before putting it on the digital shelf for free, as usual. We will provide a smooth migration path in order to move safely your current analyzers and responders and their configuration to Cortex 3.

So to paraphrase the late and great Gil Scott-Heron:

The dockerization will not be televised

The dockerization will not be televised

The dockerization will be live.

Since you are here

The success of TheHive and Cortex continue to grow, far more than we initially foresaw. As far as we know, there are about a hundred organisations of different sizes and locations using or testing them. And as the number of users grows, so does the number of features, professional service and support requests.

We have tried addressing these requests through Creative Source, a nonprofit organisation (NPO). All but one company trusted us enough to make a donation and get tailored services for its needs in return. Most of the others either did not reply to our proposals or explained that their procurement process does not accommodate working with NPOs.

Some members of our core team are actively working on alternative options to ensure not only the viability of TheHive and Cortex as FOSS products on the long run but the ability to provide professional training, support, and services without freaking out highly bureaucratic, think-in-the-box-but-never-outside procurement departments.

Stay tuned 🐝


(*) When the idea behind Cortex was born into our hive mind, we did not initially think about active response capabilities. So we naturally called the repository which was supposed to contain analyzers Cortex-Analyzers . When, at a later stage, we added responders, we put them in the same repository for obvious laziness pretences  ¯\_(ツ)_/¯.

Correction: February 15, 2019
Typographical errors have been corrected. Some rewording has been made for the sake of clarity.

UnshortenLink, SSRF and Cortex-Analyzers 1.15.2

We urge our fellow fighters of cybercrime and funny and not-so-funny animals-in-disguise, to update their Cortex analyzers to the latest 1.15.2 version which plugs a serious vulnerability in the Unshortenlink analyzer and fixes several bugs.

Unshortenlink Vulnerability

On Jan 24, Alexandre Basquin discovered a SSRF vulnerability in the Unshortenlink 1.0 analyzer. By exploiting it, an attacker which has access to a Cortex instance with an analyze role can scan the ports of localhost and possibly of all the hosts reachable by Cortex. This could be automated through Cortex4py by making repetitive calls to the API and thus scanning multiple ports & hosts. In essence, the attacker can perform reconnaissance thanks to Unshortenlink and gain knowledge on which ports are open and which aren’t.

Version 1.1 of Unshortenlink, included in Cortex-Analyzers 1.15.2, plugs this vulnerability by disabling submissions containing IPs & ports. Only URLs are now accepted.

Bug Fixes

Additionally, Cortex-Analyzers 1.15.2 corrects the following bugs:

  • #416: the Mailer responder now supports UTF-8 encoding.
  • #410: remove wrapping of the results produced by crt.sh as a list which ends up breaking the output of the report template. Contributed by Thomas Kastner.
  • #409: enum is not required for Python 3.4+ to make the MISP Search analyzer work.
  • #408: FileInfo’s Manalyze plugin did not work after Manalyze renamed plugin_btcaddress to plugin_cryptoaddress.
  • #406: fix a broken link in the Cymon_Check_IP report, submitted by Manabu Niseki.
  • #313: fix wrong file handling in OTXQuery.

How to Update your Analyzers?

To update your analyzers to version 1.15.2, please read the Updating section of the Cortex installation guide.

Since the version numbers of few analyzers have changed, you’ll need to connect as an orgadmin to Cortex to refresh the analyzer list, disabling old ones and enabling the new versions.

TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

Wait a Minute! Where’s the Blog Post about 1.15.1?

Good catch! There was no blog post about Cortex-Analyzers 1.15.1. Call us lazy but there was nothing Earth-shattering, pole-shifting in that release. Nonetheless, here are a list of fixes included in that release and from which you’ll inherit automatically if you update 1.15.0 to 1.15.2 directly:

  • #402: Malwares analyzer code relied on functionality that’s only available in Python 3.7+. It has been fixed to work with 3.4 and up thanks to the work of Arcuri Davide.
  • #404: fixes a bad folder renaming in the HIBP (Have I Been Pwned) analyzer.
  • #398: MISP Search analyzer wouldn’t run without the enum dependency. Contributed by Mars Huang. Later improved in 1.15.2 (see above).

No Luck Luke?

If something does not work as expected, keep calm and ask on our user forum, reach out to fellow users on Gitter, or send us an email at support@thehive-project.org.

TheHive 3.3-RC2, Hot out of the Oven

TheHive Project’s code Chefs, dressed in their outright haute cuisine outfit, including the traditional toque blanche, have been quite busy lately, working on dockerizing all the Cortex analyzers (more on this later in an upcoming post), and doing tedious work to prepare the replacement of Elasticsearch by a GraphDB which will help us finally release much-awaited features such as multi-tenancy, delayed for way too many months (yeah, yeah, don’t chastise them but feel free to help them). In the meantime, they found enough bandwidth to release a new major version of TheHive.

Version 3.3, currently a release candidate includes several bug fixes and many new features as outlined below. Please note that TheHive 3.3-RC2 is beta software. As all our other release candidates, you can grab it from the pre-release, beta repositories. As usual, we would truly appreciate your help making it a great stable release by testing it as thoroughly as possible and reporting back any bugs or issues you encounter so we can address them before the final release.

Check TheHive Installation guide for further details.

Wait! Where’s RC1?

TheHive 3.3-RC1 was very short-lived. Few hours after its release, and thanks to Chris (a.k.a. crackytsi on GitHub), Thomas Franco, our back-end mastermind, discovered an issue with the Debian 8 and Debian 9 packages.

New Features

  • #836: add a new exportCaseTags parameter to the MISP configuration section. If set to true, all the tags associated with a case will be exported along with it to MISP.
  • #861: add support for Java higher than 8, such as OpenJDK 11.
  • #271: bulk merge alerts into a case. Select multiple alerts at once and create a single case out of them or merge them into an existing case using its ID.
  • #824: add ability to sort alerts by reference, status, type, source…
  • #826: when previewing an alert, there are sometimes no overlap with an existing case. However, an analyst might already know, thanks to HI (Human Intelligence), that the alert should be merged into a specific case. This is now possible thanks to a new button.
  • #769: improve case template selection for case creation. If you have defined a large set of case templates, you will be able to sort/filter to find the case you want to use when creating a New Case.
New case template selector
  • #657: add observable tags auto-completion. Contributed by Tyler Chong (Thanks!).
Observable tag auto-completion

Fixed Bugs

  • #864: do not return a session cookie when making an API call.
  • #856: there was a bug where after a followed alert PATCH, if the alert has already been promoted to a case, the case is not updated. Now, if the alert has follow=true, if it gets updated, its status is set to Updated and the related case is updated too.
  • #845: assigned but unstarted tasks were not showing up in My Tasks.
  • #844: enable user account locking through the Delete API endpoint.

Stuck?

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.