Elasticsearch 7.11.x is now supported !

After the first feedback received from the community on Discord and Github, we have put the coming 4.1.0 release of TheHive on hold to focus on what can potentially break Cortex v3.1.0 and TheHive v3.5.0 with the upgrade to Elasticsearch v7.11.x.
Several questions appeared on our side:

  • How to fix the installation process ?
  • How to fix the upgrade process ?
  • As the update of an existing Elasticsearch database to 7.11.x makes the downgrade impossible after that, will our fix repair it ?

Good news: this is fixed with TheHive v3.5.1 and Cortex 3.1.1, and these versions will repair databases that have already been updated to ES 7.11.x.

The update to these new versions introduces new indexes, for Cortex (cortex_6), as well as for TheHive (the_hive_17).

  • If you plan to run a new installation for Cortex, just follow the installation guide;
  • If you want to update your setup, we recommend to follow this process:
    • Stop TheHive or Cortex
    • Stop Elasticsearch
    • Update Elasticsearch package
    • Restart Elasticsearch and ensure this is going well
    • Install TheHive 3.5.1 or Cortex 3.1.1 package
  • If you already updated Elasticsearch to version 7.11.x and faced potential breaks with Cortex or TheHive, follow the process described previously and you should regain access to your data.

Once updated, you should be invited to update the database:

Update Database after installing TheHive 3.5.1 or Cortex 3.1.1

Click on “Update Database” and you should then be invited to login. Everything should work fine after that.

As usual, we recommend making backups or a snapshot of the database before running the upgrade.

How to report issues

Please open an issue on GitHub for Cortex of the TheHive with the dedicated template for TheHive 3.x. We will monitor them closely and respond accordingly. 

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Discord, or send us an email at support@thehive-project.org. We will be more than happy to help!

Compatibility issues with Elasticsearch update

Dear TheHive and Cortex users,

If you are running TheHive v3.5.0 and / or Cortex v3.1.0, the underlying database is Elasticsearch v7.x.

Elastic recently released two new versions: v7.11.0 and v7.11.1. After some initial feedback and investigations, we found that the new releases introduce changes that break the compatibility with our products – TheHive 3.5.0 and Cortex 3.1.0.

Therefore, please DO NOT upgrade your current database to Elasticsearch v7.11.x as no rollback is possible. Elasticsearch v7.11.x breaks the installation process as well as the update process.

If you are in the process of installing or updating to Cortex v3.1.0 or TheHive v3.5.0, you need to specify the exact working version of Elasticsearch to use:

  • For Debian packages: “apt install elasticsearch=7.10.2”
  • FOR RPM packages: “yum install elasticsearch-7.10.2-1”

We are currently running deeper investigations and are planning to release updated versions as soon as possible for Cortex v3.1.0 and for TheHive 3.5.0.

We will keep you informed, stay tuned!

Cortex-Analyzers 1.15.3 get ready for URLhaus and Cuckoo

Few improvements have been introduced in this version :

  • Proofpoint analyzer has been updated to use python3 (#417)
  • Long report of Cuckoo Sandbox analyzer has been improved to be able to display Cuckoo v. 2.0.6 reports (#418)
  • URLhaus analyzer has also been updated to use the recently introduced API (#431)

On your Cortex server, update your analyzers with the following command:

$ cd /opt/Cortex-Analyzers && git pull
$ for I in analyzers/*/requirements.txt; do sudo -H /usr/bin/python2 -m pip install -r $I
$ for I in analyzers/*/requirements.txt; do sudo -H /usr/bin/python3 -m pip install -r $I

Updating analyzers should be followed by also updating report templates. Download new report templates and add the archive in TheHive report templates.

Detailed information

Cuckoo Sandbox

Thanks to @nicpenning Cuckoo analyzer is now able to display reports from version  2.0.6 of Cuckoo. With this version, remote connections part of the report has been fixed and is now well displayed

The new cuckoo analyzer template

Notice: This fix has been reviewed by our core team, unfortunately, we have not been able to test it.

URLhaus

Abuse.ch, the operator of URLhaus recently introduced a new API for their service in order to handle bulk queries and reduce costs for their main page that uses CDNs for delivery. Because of that, thanks to Nils Kuhnert, the URLhaus analyzer has been rewritten from scratch. Also, the templates changed because there are three different API endpoints used:

The short reports now display the threat type – which currently is always malware_download and, for the hash observable type, the malware family – if given by URLhaus:

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Cortex 1.1.2 Released

We are glad to announce a new version of your favorite observable analysis engine which corrects bugs introduced by version 1.1.1 and adds a few enhancements. As a reminder, TheHive, our Security Incident Response Platform, can interact with one or several Cortex instances. Moreover, starting from version 1.1.1, Cortex has a two-way integration with MISP.

We highly advise you to upgrade your Cortex in to instance to 1.1.2.

Screen Shot 2017-05-24 at 11.51.54.png
Cortex 1.1.2 – Job Report Example with CERT-SG’s Abuse Finder

Fixed Issues

  • #27: fixed the daunting error 500 that many users of  TheHive encountered when a job is submitted to Cortex.
  • #29: the MISP expansion modules are now disabled by default to avoid another error 500.
  • #31: the web interface was displaying SNAPSHOT (oops!) for the Cortex version.  It now displays the correct version.

Enhancements

  • #28: when you enable the MISP expansion modules, Cortex will not be slowed down and starts without delay.
  • #30: add a page loader mask similar to TheHive’s.

Download & Get Down to Work

To update your current Cortex installation, follow the instructions of the installation guide. Before doing so, you may want to save the job reports that were not executed via TheHive. Cortex 1 has no persistence and restarting the service will wipe out any existing reports.

Please note that you can install Cortex using an RPM or DEB package, deploy it using an Ansible script, use Docker, install it from a binary or build it from sources.

Support

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Mellifera 1: Bugfixes, Enhancements and Documentation

Last week, we have released Mellifera (TheHive 2.11.0), a major version of your favorite (or soon to be favorite) Security Incident Response Platform. Sadly, some annoying bugs have slipped past our QA (n’est-ce pas Thomas ?).

We are happy to announce the availability of Mellifera 1 (TheHive 2.11.1) which corrects those bugs and adds a few enhancements detailed below.

Issues Corrected

  • #204: update case templates created with previous versions of TheHive.
  • #205: remove duplicate tags associated to an observable present in two cases upon a case merging operation.
  • #206: apply case templates when an alert is converted into a case.

Enhancements

We also took the opportunity of this hotfix to add the following enhancements:

  • #180: merge duplicate tasks during a case merge operation. Starting from this release, if you have waiting tasks (i.e. not assigned) with the same name in cases you’d like to merge, the new merged case will have only one task instead of two.
  • #211: show the number of available analyzer reports for each observable. If an observable has not been analyzed yet, say so.

Documentation

Please note that we have moved all the documentation of TheHive in a new repository. If you are not using TheHive4py 1.2.0 (or future versions), you can send alerts to Mellifera using the API as documented.

Download & Get Down to Work

If you have an existing TheHive installation, please follow the new migration guide.

If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, deploy it using an Ansible script, use Docker, install it from a binary or build it from sources.

Support

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Buckfast 1 and Cortex All-in-one Package

When you use TheHive, running an analyzer on an observable through Cortex will generate a long report and, in most cases, a short report as well.

Let’s see how this works in practice through an example. Assume we are trying to assess whether the 636a4249104acaaf6d76d7409dc3cb2d MD5 hash is malicious or not:

Screen Shot 2017-03-26 at 22.21.10.png

We start by clicking on it, which will open a new tab:Screen Shot 2017-03-26 at 22.21.22.pngThis TLP:WHITE hash was imported from a MISP event published by our good friends at CIRCL.lu sometimes ago. As you can see from the screenshot above, no analyzer was executed on it.  Let’s check if it is known to VirusTotal (VT). To do so, we just need to click on the fire icon located at the right side of the VirusTotal_GetReport_2_0 row.

Screen Shot 2017-03-26 at 22.21.53

A blink of an eye later, the job has finished successfully as we can tell from the green checkmark. Clicking on the date will let us see the long report, presented according to a report template that we freely provide with most analyzers to the exception of PassiveTotal (but in a few days, PT will also get its own nifty templates).Screen Shot 2017-03-26 at 22.22.11.png

Since we are checking whether VT knows a hash or not, it will give us the results if any corresponding to the last time the associated file was scanned on the service. In our case, this dates back to Dec 2, 2016.

When the analyzer was executed, it also produced a short report which TheHive displays below the observable:Screen Shot 2017-03-26 at 22.21.45.pngShort reports come in 4 colors. Red means danger (what else?). Orange means suspicious. Green means innocuous. And blue is informational. OK but what does this have to do with the title of this post?

A few days ago, while working on a new set of analyzers, Nils Kuhnert reported an issue in Buckfast 1 (2.10.1) pertaining to short reports on observables. When he ran some analyzers that should have produced short reports, he didn’t get any. When he reverted to Buckfast 0 (2.10.0), it worked. We tracked down the problem and found that our build process was the culprit. The all-in-one  binary package which was supposed to contain Buckfast 1 and Cortex was in fact a 2.10.0 TheHive snapshot that had a regression. We have uploaded a fresh all-in-one binary package with Buckfast 1 instead of the development snapshot.

If you have grabbed the binary all-in-one package, please download it again and update your instance. If you are using a docker version or built Buckfast 1 from sources, you are fine. To make sure you are running the right version, click on your username once you are logged in then on About TheHive. You should see the following information:Screen Shot 2017-03-26 at 22.56.09.png

We are going to review our release process from the ground up to ascertain such errors never occur again. We expect it to be ready for Mellifera, our next major release of TheHive. Please note that starting from that release, we will no longer provide all-in-one binary and docker packages. Instead, we’ll have separate packages for TheHive and Cortex. TheHive4py and the upcoming Cortex4py will be made available through PIP.