Rejoice folks! You can now play with TheHive & Cortex thanks to the test VM we created. It includes Mellifera 12, the latest major version of TheHive, Cortex 1.1.3, the latest Cortex analyzers with all dependencies and ElasticSearch installed on top of Ubuntu 16.04 with Oracle JRE 8.
The test VM is intended to be used… well… for testing or training purposes. We strongly encourage you to refrain from using it for production.
Get It
You can download the VM from the following location:
click on Import templates button and select the downloaded package
Plug it with MISP
The test VM does not contain a MISP instance and none is configured in TheHive’s configuration file. To play with MISP, you may want to use the VM our good friends at CIRCL provide. Once you’ve downloaded it or if you have an existing instance, edit /etc/thehive/application.conf and follow the configuration guide.
Restart or Go Mad
After each modification of /etc/thehive/application.conf do not forget to restart the service:
$ sudo service thehive restart
Troubles?
TheHive service logs are located in /var/log/thehive/application.log.
Configure Cortex
All available analyzers are installed with their dependencies, but none is configured. To configure analyzers, edit /etc/cortex/application.conf and follow the configuration guide.
Restart or Go Mad
After each modification of /etc/cortex/application.conf do not forget to restart the service:
$ sudo service cortex restart
Troubles?
Cortex service logs are located in /var/log/cortex/application.log.
Need Help?
Something does not work as expected? No worries, we got you covered. Please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Correction: July 8, 2017
An earlier version of this post offered to download the VM from Dropbox but they suspended the associated link due to seemingly heavy traffic. The post was updated to replace the Dropbox link with a Google Drive one.
It’s a sunny week in Paris, France (not Texas) barring the tropical rain that washed out the city earlier this morning. And when there’s sun in France, there’s happiness and… coding of course (what else?). The French Chefs of TheHive Project seem to be in a good mood (n’est-ce pas Jérôme ?), thanks to the vitamin D extra charge they got for free from the big star up above.
Version 1.2.2 of the Python API client for TheHive is now available. It mainly fixes issues related to missing Python dependencies and adds support for creating alerts containing files for Python 3.
To update your existing package:
$ sudo pip install thehive4py --upgrade
If you are just getting started with TheHive4py, you can forgo the --upgrade at the end of the command above.
Houston? Are you There?
Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. As usual, we’ll be more than happy to help!
TheHive Project French Chefs are very happy to announce Mellifera 12, a brand new, all shiny, major version of TheHive.
This MAGA (Make Analysis Great Again) edition of your (soon to be?) favorite Security Incident Response Platform (SIRP) contains an awful lot of changes and improvements. While you can read the full changelog while waiting in line for your burger at HopDoddy or for your turn for the latest Disney attraction, we’d like to concentrate on a few features that would make you enjoy Digital Forensics & Incident Response like never before 🙂
Go Short or Go North
The Observables tab can now display the short (a.k.a. mini) reports produced by analyzers. Whenever you run an analysis (or many), the mini-reports will be shown as soon as the corresponding analyzer jobs have successfully finished. That way, you no longer have to click on each observable to access the short report.
Short Reports shown on the Observables Tab
In fact, you don’t even have to click on the observable to access the long reports. You must simply click on the corresponding short report and the long one will be displayed on top of the observables tab as shown in the following screencast. Ain’t that nifty?
A Single Click on the Short Report Shows the Long One
The short reports have been also improved to follow a taxonomy. To get to this stage, we had to review all 24 analyzers and their flavors, add new functionality to the CortexUtils Python library and improve the analyzers to add a summary section to their JSON output which Mellifera 12 interprets and displays according to a color code as described in our previous post. Please make sure to read it as it contains important information on how to update your cortexutils version and the analyzers as well as the report templates.
Is This Alert New or What?
Mellifera 12 introduces an important feature pertaining to alerts. To put it simply, whenever you receive a new alert from MISP, email, SIEM or any other source that you have connected with TheHive, the alert preview page will tell you if there are similarities with existing cases and if so, Mellifera 12 will let you import the new alert in the existing case and any updates made to that alert (think of an ongoing MISP event) will be automatically added to the case.
Alert Preview Page with the new Similar cases Section
Template this, Template that
In addition to the ‘similarity’ feature outlined above, Mellifera 12 lets you choose the case template to use when importing a new alert instead of having to use only a specific case template per alert type/source.
Choose the Template You’d Like to Use to Import an Alert
Custom Fields
We heard our community and implemented a feature that was requested by several users: custom fields.
So you’d like to add a business impact to a specific type of cases? Or a set of TTPs? Or a Threat Actor? Or specify a Business Unit? No problem! Ask an admin to create a custom field, associate it with a case template and there you go.
Add a Custom Field
Unlike metrics, custom fields must not be filled to close a case. You can also supercharge a case with custom fields that have not been associated to a case template. We currently support four types of custom fields: strings, numbers, booleans and dates. And you can create lists of acceptable values to limit your analysts’ choices to legitimate data.
If you have an existing installation of TheHive, please follow the migration guide.
If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
TheHive, Cortex and MISP work nicely together and if you’ve read our June-Dec 17 roadmap post, the integration of our products with the de facto threat sharing platform will get better in a few months.
During the FIRST conference presentation we gave last week, we displayed a picture that we will use here to try to explain how these three open source and free products integrate with one another.
A Picture is Worth a Thousand Words…
TheHive
TheHive is a Security Incident Response Platform (SIRP). It can receive alerts from different sources (SIEM, IDS, email. etc.) via its REST API. This is where alert feeders come into play.
Alert Feeders
Think of an alert feeder as a specialized program which consumes a security event (SIEM alert, email report, IDS alert, and so on), parses it and outputs an alert that its sends to TheHive through TheHive4py, the Python library we provide to interact with TheHive’s REST API.
We do not supply such feeders but developing them should be straightforward. If not, let us know and we’ll do our best to help you out.
Alerts
Any alert sent to TheHive will show up in its Alerts pane. In addition to the sources mentioned above, new or updated MISP events will show up as well in that area if you configured TheHive to connect to one or several MISP instances. If so, TheHive will poll those MISP instance(s) at every interval looking for new or updated events. If there are any, TheHive will generate an alert which will end up in the Alerts pane.
The Alerts Pane
Alerts can be ignored, mark as read, previewed and imported. When an alert is imported, it becomes a case that needs to be investigated.
Cases
The Workflow that is at the Heart of TheHive
A case can be generated from an alert or created from scratch. It is subdivided into tasks (think identification, containment, eradication, check proxy logs, and so on) and observables (IP addresses, hashes, email addresses, domain names, URLs…). When analysts are working on tasks, they add logs as they go. In TheHive’s terminology, logs are text entries which may contain attachments to help analysts record what they have been doing. Logs can be written using Markdown or a rich-text editor.
Case Templates
You don’t need to add the same tasks over and over when working on cases belonging to a given category (DDoS, Malspam, APT, …). You can create custom templates to which you add tasks as shown below. This is very useful when you are dealing with alerts so that when you import them, you can select which case template you’d like to apply and there you go!
A Sample Case Template
Observables
Observables can be tagged, flagged as IOCs, and analyzed. When the investigation is well in progress or completed, you may want to share the resulting IOCs or a subset of those with partners and peers. TheHive will support the ability to export that data to MISP in September 2017. Until then, you can still export your IOCs as text, CSV or as a MISP-compatible format that you can use to add them to your MISP instance using the freetext editor. TheHive can export IOCs/observables in protected (hxxps://www[.]somewhere[.]com/) or unprotected mode.
Every observable must have a TLP (Traffic Light Protocol) level. By default, any added observable is considered TLP:AMBER. Please note that the TLP is taken into account by some analyzers. Wait! Analyzers?
Cortex
Cortex is our standalone analysis engine and a perfect companion for TheHive and MISP. Analysts can use it to analyze observables using its Web UI, in which case they can be submitted only one at a time. The Web UI should really be limited to quick assessments of observables before creating a case in TheHive (or in an alternate SIRP). The power of Cortex really comes into play when you use its REST API. TheHive speaks natively to Cortex (as MISP does). Moreover, TheHive can leverage one or several Cortex servers.
Observable Page and List of Analyzers
Analyzers
As of this writing, Cortex has 23 analyzers which come in a total of 39 flavors and more will be available soon.
An analyzer can be written in any programming language supported by Linux though all of our current analyzers are written in Python. This is because we provide a Python library called Cortexutils which contains a set of utility classes that make it easier to write an analyzer in Python.
Flavors
Analyzers such as VirusTotal, PassiveTotal or DomainTools can provide different analysis services. Let’s take VirusTotal as an example. You can scan a file or URL. That’s one flavor. You can also obtain the latest available report on VirusTotal.com for a file, hash, domain or IP address. That’s a second flavor. So the VirusTotal analyzer has two flavors.
How about PassiveTotal? It has 8 flavors: unique resolutions lookup, SSL certificate history lookup, malware lookup, passive DNS lookup, data enrichment lookup, SSL certificate details lookup, OSINT lookup and WHOIS data lookup.
The MISP Search Analyzer
At this point, we need to mention a special analyzer that may create some confusion if not understood correctly: the MISP Search analyzer. Thanks to it, Cortex has the ability to search observables within a MISP instance as represented by the arrow that goes from the Analyzers to MISP.
Search for MISP Events Containing a Given Observable
When an observable is found in an event, Cortex will return the number of records found (i.e. the number of events where the observable has been found) and a list of links to those events with additional data.
Searching for a Hash Using the MISP Search Analyzer from the Cortex Web UIThe Same Search Conducted from TheHive: Long ReportMini-Report
The current version of the MISP Search analyzer can only search within a single MISP instance but in the near future, it will be able to support multiple ones.
MISP Expansion Modules
Besides its own analyzers (which include MISP Search described above), Cortex can also invoke MISP expansion modules. These are normally used by MISP to enrich attributes within events but Cortex can also take advantage of them to analyze observables.
There is some overlap between the native Cortex analyzers and MISP expansion modules. When choosing between a native analyzer or an expansion module, we highly recommend you select the former. The expansion modules are deactivated in the default Cortex configuration.
Jobs
When you submit an observable for analysis, Cortex will create a job and, if successful, it will generate an analysis report in JSON format. TheHive has the ability to parse those results and present them in a human-friendly fashion thanks to report templates we offer for free. So when you’ll submit an observable to Cortex from TheHive, you’ll get back a short (or mini) report and a long one. The first can be thought of as a really tiny Exec Analyst Summary while the second provides more insight and details.
Calling Cortex from MISP
In addition to the expansion modules we have just mentioned, MISP 2.4.73 and up can enrich attributes using Cortex analyzers. The configuration is pretty straightforward. So if all you are concerned about is threat intelligence and sharing, you may augment your visibility into a given threat represented as a MISP event by leveraging all current 23 Cortex analyzers and any future ones.
Conclusion
TheHive, Cortex and MISP are three open source and free products that can highly aid you combat threats and keep the ‘monsters’ at bay.
TheHive, as a SIRP, allows you to investigate security incident swiftly in a collaborative manner. Several analysts can work simultaneously on tasks & cases . While cases can be created from scratch, TheHive can receive alerts from different sources thanks to alert feeders which consume security events generated by multiple sources and feed them into TheHive using TheHive4py Python library. TheHive can also sync to one or several MISP instances to receive new and updated events which will appear in the alert pane with all the other alerts generated by other sources. Analysts can then preview new alerts to decide whether they need to be acted upon. If so, they can transform them into investigation cases using templates.
To analyze the observables collected in the course of an investigation and/or imported from a MISP event, TheHive can rely on one or several Cortex analysis engines. Cortex is another standalone product that we have developed which sole purpose is to allow you to analyze observables at scale thanks to its large number of analyzers, MISP expansion modules and any analyzer you might have developed on the side. Cortex has a REST API that can be used to empower other security products such as ‘analytics’ software, alternate SIRPs or MISP.
The highly popular threat sharing platform can indeed enrich attributes thanks to Cortex as it has a native integration with it. And in a few months, you will also be able to export cases from TheHive as MISP events that you can share with peers and partners.
If you do share, you do care about our collective mission to defend thedigital assets that are under our watch from harm. So let us fight together as one.
A few days ago, we have been made aware of a bug in the way we pulled new or updated MISP events to inject them within Mellifera’s alerting panel. As a result, some events did not show up as intended. So you might have missed some of the action shared by peers and partners through MISP.
As true Frenchmen who care a lot about cuisine, TheHive Project’s Chefs went back to their code kitchen and figured out a more palatable recipe to make sure you won’t be left under the impression that you were seeing all new or updated MISP events while in fact you didn’t (we don’t want you to go too easy & lazy n’est-ce pas ?). Mellifera 11.3 (TheHive 2.11.3), a hotfix version has been released to that end and should fix the issue. Please note that you must use MISP 2.4.73 or better.
In addition, this new version of your favorite (or soon to be favorite) Security Incident Response Platform can be installed from a deb package on Ubuntu 16.04 without having to fiddle with OpenJDK. We have repackaged the software to avoid grabbing OpenJDK 9 (which TheHive does not support) and force the installation of version 8.
Finally, if an admin creates an empty case template, users can add tasks to it while previously this wasn’t possible.
Download & Get Down to Work
If you have an existing installation of TheHive, please follow the new migration guide.
If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, deploy it using an Ansible script, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
A new version of TheHive will be released by the end of June. We will take this opportunity to review our release naming and numbering from the ground up.
Months ago, we started giving ‘major’ versions (2.10, 2.11, …) the name of honey bee varieties. 2.10 was called Buckfast. 2.11, the current version, is called Mellifera. And we were supposed to give 2.12 yet another name. However, and after the few hiccups we’ve encountered with our QA as of late, we have decided to change things around in order to make sure new releases are as stable and well-maintained as you should expect them to be.
Starting from the next release (2.12), we will abide by the following numbering scheme:
Only major versions will have corresponding honey bee names. So long as we stay with v2, we’ll keep calling all the minor versions Mellifera N (2.12.0 = Mellifera 12). Version 3 will be called Cerana.
Mellifera 12 – June 29, 2017 (planned date)
Mellifera 12 (v 2.12) will succeed to Mellifera 2 (the current version) to comply with the new naming scheme. It will allow you to see how similar new alerts are to existing cases so you can decide whether you import them into an existing case, create a new one or ignore them altogether. Mellifera 12 will show you the status of all the related cases (#229) to the one you are working on. Finally, you’ll have the ability to change the default case template before importing an alert.
M12 will also support custom fields (#12), a feature that has been requested by numerous users. This version will also add mini-reports to the Observables tab. That way, once a Cortex analysis has been completed, analysts will be able to view part or all the resulting short report in that tab instead of having to navigate to the page of each observable to read the short report.
Mellifera 13 – September 14, 2017
TheHive 2.13 should be the last Mellifera version. It will complete TheHive’s integration with MISP by adding the ability to export all observables or a subset of them to a MISP instance. Please note that TheHive allowed you from the start to import events from multiple MISP instances but since sharing is caring, we wanted to add the ability to export to this very popular threat sharing platform from your Security Incident Response Platform (SIRP). We do not want to rush it though.
Cerana – October 12, 2017
Cerana or TheHive 3.0.0 will bring a complete UI overhaul to make it even easier to work on cases, perform analysis and get your job done, after the interface refreshments Mellifera brought. It will lay the ground for some nifty features we have in mind.
Cerana 1 – November 15, 2017
TheHive 3.1.0 will include dynamic dashboards: the ability to work with the statistics and metrics the way you want and generate customized dashboards to help you drive your activities.
Keep an eye on TheHive’s milestones on GitHub. There are other features and enhancements that we might add as we progress and we will reflect them on that page.
Correction: June 12, 2017
An earlier version mentioned GitHub issue #36 as pertaining to custom fields while it is a request for globally-defined tags that an analyst can choose from.
The Chefs behind TheHive Project’s delicious code are happy to announce the availability of Mellifera 2 (TheHive v2.11.2), the scalable, free and open source Security Incident Response Platform. This minor version fixes two irking issues related to MISP and adds a few enhancements detailed below.
Mellifera – The New Alerting Panel
Fixed Issues
#220: alerts related to MISP events are not properly updated.
#221: in some edge cases, alerts related to MISP events are created with no attribute.
Enhancements
#188: display the case severity in the My tasks and the Waiting tasks pages to let analysts prioritize their work.
#218: show the description of an alert in the alerting panel.
#224: visually distinguish between analyzed and non-analyzed observables.
Download & Get Down to Work
If you have an existing TheHive installation, please follow the new migration guide.
If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, deploy it using an Ansible script, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Correction: May 26, 2017
A copy/paste error from a previous blog post was fixed.
We are glad to announce a new version of your favorite observable analysis engine which corrects bugs introduced by version 1.1.1 and adds a few enhancements. As a reminder, TheHive, our Security Incident Response Platform, can interact with one or several Cortex instances. Moreover, starting from version 1.1.1, Cortex has a two-way integration with MISP.
We highly advise you to upgrade your Cortex in to instance to 1.1.2.
Cortex 1.1.2 – Job Report Example with CERT-SG’s Abuse Finder
Fixed Issues
#27: fixed the daunting error 500 that many users of TheHive encountered when a job is submitted to Cortex.
#29: the MISP expansion modules are now disabled by default to avoid another error 500.
#31: the web interface was displaying SNAPSHOT (oops!) for the Cortex version. It now displays the correct version.
Enhancements
#28: when you enable the MISP expansion modules, Cortex will not be slowed down and starts without delay.
To update your current Cortex installation, follow the instructions of the installation guide. Before doing so, you may want to save the job reports that were not executed via TheHive. Cortex 1 has no persistence and restarting the service will wipe out any existing reports.
Please note that you can install Cortex using an RPM or DEB package, deploy it using an Ansible script, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
We are pleased to announce the availability of 2 new Cortex analyzers and an update to 2 existing ones:
New: VMRay and FireHOL
Updated: Joe Sandbox and Fortiguard URL Category
We would like to thank Nils Kuhnert from CERT-BUND, CERT-BDF and Eric Capuano for their precious contributions.
To install the new analyzers, grab the Cortex-Analyzers repository and unpack its content (or git pull the master branch) in your existing /path/to/cortex-analyzers. Then follow the Cortex analyzers guide.
To import the new report templates in your instance of TheHive:
click on Import templates button and select the downloaded package
VMRay
The VMRay analyzer has been submitted by Nils Kunhert from CERT-BUND. It lets you run a file in a local or remote (cloud) VMRay sandbox. The analyzer also lets you check existing analysis reports.
The analyzer accepts files and hashes as input. VMRay is a commercial service and you need an API key to run the analyzer. To make it work, install the requestsPython library. It should already have been installed since it is used by other analyzers as well.
To use the analyzer, add the following section to the Cortex configuration file (application.conf):
VMRay {
url = ""
key = ""
certpath = ""
}
When called from TheHive, the following output is produced:
TheHive: VMRay Analyzer – Short and Long Report Samples
Important note: an analysis on VMRay, like on any other sandbox, can take a long time. That is why the analyzer tries to fetch the report until it is ready.
FireHOL
The FireHOL analyzer has been submitted by Nils Kuhnert from CERT-BUND. It lets you use the lists maintained by FireHOL project and check if an IP resides in one of them. FireHOL is an open source project. The analyzer reports the block lists in which an IP resides with the latest updated ones displayed first. To make it work, you’ll need to download the lists in a directory first (and it would be wise to do it on a regular fashion using a cron entry for example):
The ignoreolderthandays parameter lets you tell the analyzer to ignore matches found in lists that have not been refreshed in <int> days where <int> is an integer.
When called from TheHive, the following output is produced:
TheHive: FireHOL Analyzer – Short and Long Report Samples
Joe Sandbox
Thanks to CERT-BDF, the Joe Sandbox analyzer has been updated to support Joe Sandbox Cloud service beside the on-premises version (Ultimate). Like with other Joe Sandbox services, you need to add the following section to the Cortex configuration file (application.conf):
JoeSandbox {
url = ""
apikey = ""
}
Fortiguard URL Category
Thanks to Eric Capuano, the Fortiguard URL Category analyzer is working again. Eric has modified it to handle the changes made by Fortiguard to their free online API.
Correction: May 23, 2017
An earlier version of this post used ignoredays instead of ignoreolderthandays for the FireHOL Blocklists analyzer. This parameter has also been described.
TheHive Project’s Chefs are thrilled to announce the immediate availability of Cortex 1.1.1.
Starting from this version, Cortex can be integrated in two ways with MISP as described below. We would like to thank Alexandre Dulaunoy for inviting us to the Open Source Security Software Hackathon which took place in Luxembourg during two days (May 2-3, 2017). Andras Iklody worked with us during the event in order to make this two way integration a reality. Merci !
Now in addition to TheHive, our Security Incident Response Platform which can connect to multiple MISP instances to receive new or updated events, let analysts preview then import them if they deem them worth investigating, Cortex can query MISP modules or be invoked from MISP to let an instance’s users leverage the power of its 21 analyzers. And in the near future, TheHive will also gain the ability to export observables to MISP.
Obviously, there are some overlap between Cortex native analyzers and MISP expansion modules. For example, you could query the CIRCL’s Passive DNS service using a native Cortex analyzer or a MISP expansion module. When there’s overlap, we highly recommend you rely on the Cortex analyzer. That way, we will be able to better help you in case you encounter issues or need help to make it work.
In order to invoke MISP expansion modules within Cortex, they need to be installed on the same host that Cortex runs on. Please read the MISP Integration guide.
Invoke Cortex Analyzers within MISP
Starting from version 2.4.73, a MISP instance can invoke Cortex analyzers. To do so, connect to the MISP Web UI with sufficient privileges, then go to Administration > Server settings > Plugin settings. Edit the Cortex section as follows:
set Plugin.Cortex_services_enable to true
set Plugin.Cortex_services_url to http://ip_address(replace ip_address with the IP address of Cortex)
set Plugin.Plugin.Cortex_services_port to port (replace port with the port on which Cortex is listening: 9000 by default)
Once this operation is completed, the Cortex analyzer list should appear in MISP’s Cortex section. The analyzers must be enabled to make them available to the instance users.
Documentation
Please note that we have moved all the documentation of Cortex to a new repository.
Download & Get Down to Work
To update your current Cortex installation, follow the instructions of the installation guide. Before doing so, you may want to save the job reports that were not executed via TheHive. Cortex 1 has no persistence and restarting the service will wipe out any existing reports.
Please note that you can install Cortex using an RPM or DEB package, deploy it using an Ansible script, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Correction: May 18, 2017
An earlier version of this post contained a few typos which were corrected.
TheHive Project 