It’s a sunny week in Paris, France (not Texas) barring the tropical rain that washed out the city earlier this morning. And when there’s sun in France, there’s happiness and… coding of course (what else?). The French Chefs of TheHive Project seem to be in a good mood (n’est-ce pas Jérôme ?), thanks to the vitamin D extra charge they got for free from the big star up above.
Version 1.2.2 of the Python API client for TheHive is now available. It mainly fixes issues related to missing Python dependencies and adds support for creating alerts containing files for Python 3.
To update your existing package:
$ sudo pip install thehive4py --upgrade
If you are just getting started with TheHive4py, you can forgo the --upgrade at the end of the command above.
Houston? Are you There?
Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. As usual, we’ll be more than happy to help!
TheHive, Cortex and MISP work nicely together and if you’ve read our June-Dec 17 roadmap post, the integration of our products with the de facto threat sharing platform will get better in a few months.
During the FIRST conference presentation we gave last week, we displayed a picture that we will use here to try to explain how these three open source and free products integrate with one another.
A Picture is Worth a Thousand Words…
TheHive
TheHive is a Security Incident Response Platform (SIRP). It can receive alerts from different sources (SIEM, IDS, email. etc.) via its REST API. This is where alert feeders come into play.
Alert Feeders
Think of an alert feeder as a specialized program which consumes a security event (SIEM alert, email report, IDS alert, and so on), parses it and outputs an alert that its sends to TheHive through TheHive4py, the Python library we provide to interact with TheHive’s REST API.
We do not supply such feeders but developing them should be straightforward. If not, let us know and we’ll do our best to help you out.
Alerts
Any alert sent to TheHive will show up in its Alerts pane. In addition to the sources mentioned above, new or updated MISP events will show up as well in that area if you configured TheHive to connect to one or several MISP instances. If so, TheHive will poll those MISP instance(s) at every interval looking for new or updated events. If there are any, TheHive will generate an alert which will end up in the Alerts pane.
The Alerts Pane
Alerts can be ignored, mark as read, previewed and imported. When an alert is imported, it becomes a case that needs to be investigated.
Cases
The Workflow that is at the Heart of TheHive
A case can be generated from an alert or created from scratch. It is subdivided into tasks (think identification, containment, eradication, check proxy logs, and so on) and observables (IP addresses, hashes, email addresses, domain names, URLs…). When analysts are working on tasks, they add logs as they go. In TheHive’s terminology, logs are text entries which may contain attachments to help analysts record what they have been doing. Logs can be written using Markdown or a rich-text editor.
Case Templates
You don’t need to add the same tasks over and over when working on cases belonging to a given category (DDoS, Malspam, APT, …). You can create custom templates to which you add tasks as shown below. This is very useful when you are dealing with alerts so that when you import them, you can select which case template you’d like to apply and there you go!
A Sample Case Template
Observables
Observables can be tagged, flagged as IOCs, and analyzed. When the investigation is well in progress or completed, you may want to share the resulting IOCs or a subset of those with partners and peers. TheHive will support the ability to export that data to MISP in September 2017. Until then, you can still export your IOCs as text, CSV or as a MISP-compatible format that you can use to add them to your MISP instance using the freetext editor. TheHive can export IOCs/observables in protected (hxxps://www[.]somewhere[.]com/) or unprotected mode.
Every observable must have a TLP (Traffic Light Protocol) level. By default, any added observable is considered TLP:AMBER. Please note that the TLP is taken into account by some analyzers. Wait! Analyzers?
Cortex
Cortex is our standalone analysis engine and a perfect companion for TheHive and MISP. Analysts can use it to analyze observables using its Web UI, in which case they can be submitted only one at a time. The Web UI should really be limited to quick assessments of observables before creating a case in TheHive (or in an alternate SIRP). The power of Cortex really comes into play when you use its REST API. TheHive speaks natively to Cortex (as MISP does). Moreover, TheHive can leverage one or several Cortex servers.
Observable Page and List of Analyzers
Analyzers
As of this writing, Cortex has 23 analyzers which come in a total of 39 flavors and more will be available soon.
An analyzer can be written in any programming language supported by Linux though all of our current analyzers are written in Python. This is because we provide a Python library called Cortexutils which contains a set of utility classes that make it easier to write an analyzer in Python.
Flavors
Analyzers such as VirusTotal, PassiveTotal or DomainTools can provide different analysis services. Let’s take VirusTotal as an example. You can scan a file or URL. That’s one flavor. You can also obtain the latest available report on VirusTotal.com for a file, hash, domain or IP address. That’s a second flavor. So the VirusTotal analyzer has two flavors.
How about PassiveTotal? It has 8 flavors: unique resolutions lookup, SSL certificate history lookup, malware lookup, passive DNS lookup, data enrichment lookup, SSL certificate details lookup, OSINT lookup and WHOIS data lookup.
The MISP Search Analyzer
At this point, we need to mention a special analyzer that may create some confusion if not understood correctly: the MISP Search analyzer. Thanks to it, Cortex has the ability to search observables within a MISP instance as represented by the arrow that goes from the Analyzers to MISP.
Search for MISP Events Containing a Given Observable
When an observable is found in an event, Cortex will return the number of records found (i.e. the number of events where the observable has been found) and a list of links to those events with additional data.
Searching for a Hash Using the MISP Search Analyzer from the Cortex Web UIThe Same Search Conducted from TheHive: Long ReportMini-Report
The current version of the MISP Search analyzer can only search within a single MISP instance but in the near future, it will be able to support multiple ones.
MISP Expansion Modules
Besides its own analyzers (which include MISP Search described above), Cortex can also invoke MISP expansion modules. These are normally used by MISP to enrich attributes within events but Cortex can also take advantage of them to analyze observables.
There is some overlap between the native Cortex analyzers and MISP expansion modules. When choosing between a native analyzer or an expansion module, we highly recommend you select the former. The expansion modules are deactivated in the default Cortex configuration.
Jobs
When you submit an observable for analysis, Cortex will create a job and, if successful, it will generate an analysis report in JSON format. TheHive has the ability to parse those results and present them in a human-friendly fashion thanks to report templates we offer for free. So when you’ll submit an observable to Cortex from TheHive, you’ll get back a short (or mini) report and a long one. The first can be thought of as a really tiny Exec Analyst Summary while the second provides more insight and details.
Calling Cortex from MISP
In addition to the expansion modules we have just mentioned, MISP 2.4.73 and up can enrich attributes using Cortex analyzers. The configuration is pretty straightforward. So if all you are concerned about is threat intelligence and sharing, you may augment your visibility into a given threat represented as a MISP event by leveraging all current 23 Cortex analyzers and any future ones.
Conclusion
TheHive, Cortex and MISP are three open source and free products that can highly aid you combat threats and keep the ‘monsters’ at bay.
TheHive, as a SIRP, allows you to investigate security incident swiftly in a collaborative manner. Several analysts can work simultaneously on tasks & cases . While cases can be created from scratch, TheHive can receive alerts from different sources thanks to alert feeders which consume security events generated by multiple sources and feed them into TheHive using TheHive4py Python library. TheHive can also sync to one or several MISP instances to receive new and updated events which will appear in the alert pane with all the other alerts generated by other sources. Analysts can then preview new alerts to decide whether they need to be acted upon. If so, they can transform them into investigation cases using templates.
To analyze the observables collected in the course of an investigation and/or imported from a MISP event, TheHive can rely on one or several Cortex analysis engines. Cortex is another standalone product that we have developed which sole purpose is to allow you to analyze observables at scale thanks to its large number of analyzers, MISP expansion modules and any analyzer you might have developed on the side. Cortex has a REST API that can be used to empower other security products such as ‘analytics’ software, alternate SIRPs or MISP.
The highly popular threat sharing platform can indeed enrich attributes thanks to Cortex as it has a native integration with it. And in a few months, you will also be able to export cases from TheHive as MISP events that you can share with peers and partners.
If you do share, you do care about our collective mission to defend thedigital assets that are under our watch from harm. So let us fight together as one.
Lo and behold, The Chefs behind TheHive Project have been pretty busy at the code kitchen, continuously improving some of your favorite recipes. After releasing Mellifera 11.3 and Cortex 1.1.3, here comes the turn of Cortexutils, the Python library containing utility classes for Cortex analyzers.
Version 1.1.1 of the library solves an encoding issue, in fact a regression introduced by the previous release. It also corrects a situation where error reports wouldn’t be generated in case the analyzer did not require configuration (we have a handful of such creatures in store).
You can grab the new Cortexutils release through PIP. To update your existing installation, please run the following command:
sudo pip install cortexutils --upgrade
Running Into Trouble?
Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!
Moments ago, we have announced the release of Mellifera 11.3. And since we don’t want to leave you wanting for more update er fun time, you may want to schedule as well a Cortex update shall you need it 😉
Cortex 1.1.3 is now available. This hotfix corrects the deb package to make it compatible with Ubuntu 16.04 without having to fiddle with OpenJDK. As we did with TheHive, we have repackaged the software to avoid grabbing OpenJDK 9 (which Cortex does not support) and force the installation of version 8. This version also corrects a cryptic error that might be thrown out by Cortex as a result of an improper interpretation of an analyzer failure.
Download & Get Down to Work
To update your current Cortex installation, follow the instructions of the installation guide. Before doing so, you may want to save the job reports that were not executed via TheHive. Cortex 1 has no persistence and restarting the service will wipe out any existing reports.
Please note that you can install Cortex using an RPM or DEB package, deploy it using an Ansible script, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
A new version of TheHive will be released by the end of June. We will take this opportunity to review our release naming and numbering from the ground up.
Months ago, we started giving ‘major’ versions (2.10, 2.11, …) the name of honey bee varieties. 2.10 was called Buckfast. 2.11, the current version, is called Mellifera. And we were supposed to give 2.12 yet another name. However, and after the few hiccups we’ve encountered with our QA as of late, we have decided to change things around in order to make sure new releases are as stable and well-maintained as you should expect them to be.
Starting from the next release (2.12), we will abide by the following numbering scheme:
Only major versions will have corresponding honey bee names. So long as we stay with v2, we’ll keep calling all the minor versions Mellifera N (2.12.0 = Mellifera 12). Version 3 will be called Cerana.
Mellifera 12 – June 29, 2017 (planned date)
Mellifera 12 (v 2.12) will succeed to Mellifera 2 (the current version) to comply with the new naming scheme. It will allow you to see how similar new alerts are to existing cases so you can decide whether you import them into an existing case, create a new one or ignore them altogether. Mellifera 12 will show you the status of all the related cases (#229) to the one you are working on. Finally, you’ll have the ability to change the default case template before importing an alert.
M12 will also support custom fields (#12), a feature that has been requested by numerous users. This version will also add mini-reports to the Observables tab. That way, once a Cortex analysis has been completed, analysts will be able to view part or all the resulting short report in that tab instead of having to navigate to the page of each observable to read the short report.
Mellifera 13 – September 14, 2017
TheHive 2.13 should be the last Mellifera version. It will complete TheHive’s integration with MISP by adding the ability to export all observables or a subset of them to a MISP instance. Please note that TheHive allowed you from the start to import events from multiple MISP instances but since sharing is caring, we wanted to add the ability to export to this very popular threat sharing platform from your Security Incident Response Platform (SIRP). We do not want to rush it though.
Cerana – October 12, 2017
Cerana or TheHive 3.0.0 will bring a complete UI overhaul to make it even easier to work on cases, perform analysis and get your job done, after the interface refreshments Mellifera brought. It will lay the ground for some nifty features we have in mind.
Cerana 1 – November 15, 2017
TheHive 3.1.0 will include dynamic dashboards: the ability to work with the statistics and metrics the way you want and generate customized dashboards to help you drive your activities.
Keep an eye on TheHive’s milestones on GitHub. There are other features and enhancements that we might add as we progress and we will reflect them on that page.
Correction: June 12, 2017
An earlier version mentioned GitHub issue #36 as pertaining to custom fields while it is a request for globally-defined tags that an analyst can choose from.
The Chefs behind TheHive Project’s delicious code are happy to announce the availability of Mellifera 2 (TheHive v2.11.2), the scalable, free and open source Security Incident Response Platform. This minor version fixes two irking issues related to MISP and adds a few enhancements detailed below.
Mellifera – The New Alerting Panel
Fixed Issues
#220: alerts related to MISP events are not properly updated.
#221: in some edge cases, alerts related to MISP events are created with no attribute.
Enhancements
#188: display the case severity in the My tasks and the Waiting tasks pages to let analysts prioritize their work.
#218: show the description of an alert in the alerting panel.
#224: visually distinguish between analyzed and non-analyzed observables.
Download & Get Down to Work
If you have an existing TheHive installation, please follow the new migration guide.
If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, deploy it using an Ansible script, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Correction: May 26, 2017
A copy/paste error from a previous blog post was fixed.
We are glad to announce a new version of your favorite observable analysis engine which corrects bugs introduced by version 1.1.1 and adds a few enhancements. As a reminder, TheHive, our Security Incident Response Platform, can interact with one or several Cortex instances. Moreover, starting from version 1.1.1, Cortex has a two-way integration with MISP.
We highly advise you to upgrade your Cortex in to instance to 1.1.2.
Cortex 1.1.2 – Job Report Example with CERT-SG’s Abuse Finder
Fixed Issues
#27: fixed the daunting error 500 that many users of TheHive encountered when a job is submitted to Cortex.
#29: the MISP expansion modules are now disabled by default to avoid another error 500.
#31: the web interface was displaying SNAPSHOT (oops!) for the Cortex version. It now displays the correct version.
Enhancements
#28: when you enable the MISP expansion modules, Cortex will not be slowed down and starts without delay.
To update your current Cortex installation, follow the instructions of the installation guide. Before doing so, you may want to save the job reports that were not executed via TheHive. Cortex 1 has no persistence and restarting the service will wipe out any existing reports.
Please note that you can install Cortex using an RPM or DEB package, deploy it using an Ansible script, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
As we have mentioned in the previous post, Randorisec reported to us that Cortex 1.0.0 is affected by the same Reflected XSS vulnerability as Buckfast 0 and 1 (respectively TheHive 2.10.0 and 2.10.1). This is due to the fact that both use the same angular-ui-notification service. The issue also affects Cortex 1.0.1.
We are happy to announce the immediate availability of Cortex 1.0.2 which fixes the vulnerability referenced as AP2 in Randorisec’s report. Moreover, this new release fixes 3 bugs:
Issue #11: jobs list API doesn’t take into account the limit parameter.
Issue #13: global section in configuration file is ignored.
Issue #16: redirect to jobs list when a job is not found.
To update your current Cortex installation, follow the instructions of the Installation Guide. Before doing so, you may want to save the job reports that were not executed via TheHive. Cortex 1 has no persistence and restarting the service will wipe out any existing reports.
We would like to thank again Davy Douhine, Randorisec’s CEO, ArtsSEC, Frédéric Cikala, Nicolas Mattiocco, Florent Montel and Mohamed Mrabah for devoting time and efforts for making our products more secure.
Correction: April 21, 2017
An earlier version of this article mentioned Maximilano Soler among the professionals who joined Randorisec to perform a pentest on our products. At his request, we have removed his name and replaced it by ArtsSEC.
Our good friends at Randorisec, joined by other pentesting professionals (see below), performed a fully fledged pentest of Buckfast 0 (TheHive 2.10.0) and Cortex 1.0.0 during 4 man-days spanning several weeks, starting from February 9, 2017 and ending on March 21, 2017.
They have identified several security issues detailed in their report which they privately shared with us prior to publication. As a result, we are happy to announce the immediate availability of Buckfast 2 (TheHive 2.10.2) which fixes the following vulnerabilities:
Stored XSS (ref. AP1 in the report) and Reflected XSS (AP2): malicious JavaScript code can be injected. It will be then executed on the victim’s browser. See issue #159 for more details.
Vertical privilege escalation (AP3): an authenticated simple user can have access to some admin menus. See issue #160 and issue #161.
CSRF (AP8): As no anti-CSRF tokens are used, TheHive is vulnerable to CSRF attacks. See issue #158.
Cortex 1.0.0 and 1.0.1 are also affected by AP2. A new Cortex version will be released very shortly to fix it.
Additionally, Buckfast 2 fixes the following bugs:
Issue #152: pagination does not work with 100 results per page.
Issue #169: error when importing some MISP events due to their unexpected JSON format. This has also been fixed in MISP v2.4.71.
We have also added the following features:
Issue #157: add persistence for task viewing options.
Issue #174: run all analyzers on multiple observables from the observables view.
Randorisec identified 4 more security issues rated low which aren’t fixed by this release:
Concurrent sessions allowed (AP4): we do not deem this a security vulnerability and hence we won’t fix it unless our user community request a patch.
No account lockout policy (AP5): if you use the local authentication system, it can be brute-forced. We are going to fix this in Mellifera 1 (TheHive 2.11.1) due at the end of May 2017. In the meantime, you can use LDAP, Active Directory or both and configure a password policy on those systems.
No password policy (AP6): as no password policy is enforced when using the local database for storing user credentials, users can set weak passwords (e.g.: containing only one character). We are going to fix this in Mellifera 1 (TheHive 2.11.1) due at the end of May 2017. In the meantime, you can use LDAP, Active Directory or both and configure a password policy on those systems.
Information leakage (AP7): information such as installed software versions (TheHive, ElasticSearch) is publicly available. TheHive should be not be publicly accessible and access should be filtered by a firewall or a similar device for authorized IP addresses only.
If you are running Buckfast 1 or a previous version, please follow the updating instructions to update to Buckfast 2. It is actually an extremely simple operation. If you are doing a fresh installation, we have you covered as well.
Please note that Randorisec and the pentesting professionals that joined it for this pentest have no contract with TheHive Project and did not receive any compensation of any sort to perform this work. They worked on their free time as a way to contribute to the security of Free, Open Source Software projects. We’d like to wholeheartedly thank Davy Douhine, Randorisec’s CEO, ArtsSEC, Frédéric Cikala, Nicolas Mattiocco, Florent Montel and Mohamed Mrabah for their invaluable contribution.
Correction: April 21, 2017
An earlier version of this article mentioned Maximilano Soler among the professionals who joined Randorisec to perform a pentest on our products. At his request, we have removed his name and replaced it by ArtsSEC.