TheHive 3.2.0-RC1: The MISP Love Edition

Guess what? Our integration with MISP, the de facto standard for threat sharing, has just gotten better with our latest beta release: TheHive 3.2.0-RC1.

While you could synchronize TheHive with one or multiple MISP instances in earlier versions and select events using filters like their age, the number of attributes they contain or exclude those which are created by specific organisations or contain one or several black-listed tags, 3.2.0-RC1 adds the ability to whitelist tags, thus limiting the events that would show up in TheHive’s Alerts pane to only those which have been tagged with labels your SOC/CSIRT/CERT needs to act on. This can be very useful for example if your Cyber Threat Intelligence analysts pre-select or create events in MISP and tag for SOC consumption those that need to be acted on.

The Hive - Logo - Schéma - V1_Plan de travail 1.png
TheHive, Cortex, MISP: The Power DFIR & CTI Trio

To use this feature, use the whitelist.tags parameter in the MISP section of TheHive’s application.conf as described in the documentation.

This new version also adds the ability to create dashboards out of responder actions, log responder operations, and offers a confirmation dialog before running a responder to avoid noob over-clicks and errors made by seasoned incident handlers running low on caffeine.

TheHive 3.2.0-RC1 will also show you the description of an observable if any while hovering over one in the Observables tab. You can also see observable tags when previewing an alert in the Alerts pane.

Last but not least, some users reported severe problems when they enabled TLS/SSL directly on TheHive without resorting to a reverse proxy such as NGINX. Blame that on the crappy TLS support in Play framework ;-). So we highly recommend using a reverse proxy for that purpose, and delegate authentication to it if you are relying on X.509 authentication, as TheHive 3.2.0-RC1 allows you to. Please check the Single Sign-On on TheHive with X.509 Certificates guide for further information.

For additional details on this release, please check the full changelog.

Warning Capt’n Robinson!

The RC in 3.2.0-RC1 stands for Release Candidate. Please help us make a great stable release out of it by testing it as thoroughly as possible and reporting back any bugs or issues you encounter so we can address them before the final release. You’ll find this release candidate in the pre-release, beta repositories.

Please check TheHive Installation guide for further details.

You got a problem?

Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Cortex4py 2 is Out!

Cortex, a free, open source software allows security analysts and threat hunters to analyze and enrich observables (IP addresses, hashes, domains, …) collected in the course of an investigation or received from third parties, for example through MISP, the de facto standard for threat sharing.

On March 29, 2018, we released Cortex 2, a major improvement over the previous version which brought, among other cool features, authentication, caching, multi-tenancy (RBAC) and rate limiting. Instead of deploying several Cortex 1 instances behind reverse proxies which would implement authentification, administrators can deploy a single Cortex 2, create multiple organizations and serve the needs of various information security populations while enjoying extra features.

On May 31, 2018, we published a brand new API guide so that developers can take advantage of the powerful REST API of the product. Sadly, Cortex4py, the FOSS Python library we provide to interact with the API was not compatible with Cortex 2. Until today.

Thanks to the hard work of our dear Nabil Adouani, we are happy to announce the immediate availability of Cortex4py 2.0.0, a complete rewrite of the library in Python 3. Cortex4py 2.0.0 is fully compatible with Cortex 2. However, it doesn’t work with Cortex 1.

While TheHive, the highly popular free and open source Security Incident Response Platform (SIRP) we develop has native support for many Cortex 2 instances, Python developers can leverage Cortex4py to interact with Cortex 2, manage organizations, users, analyzer configurations and analyze observables at scale from alternative SIRPs, SIEMs or custom scripts thanks to the 83 analyzers Cortex 2 has as of June 18, 2018.

Screen Shot 2018-06-18 at 20.01.27.png
Cortex 2: there is more than one way to interact with it

Use It

To install Cortex4py, use PIP3:

$ sudo -H pip3 install cortex4py

If you are using Python on a Windows operating system, please forgo the sudo command.

Usage

Cortex4py 2 comes with a usage guide which includes many examples. For example, if you want to fetch the last 10 successful jobs that have been executed against domain names and display the result summaries of those 10 jobs you could write something like:

Screen Shot 2018-06-18 at 19.58.45.png
Sample Python3 code to retrieve Cortex analyzer results

Migrating from Cortex4py 1

If you have already written scripts using Cortex4py 1.x (for Cortex 1), we tried to keep the already available methods. However, we recommend you adapt your code to leverage the new Cortex4py 2 classes and methods as soon as feasible. Moreover, the existing scripts must be updated to support authentication if you intend to use them with Cortex 2. Please read the Cortex4py 2 usage guide for more information.

Support

Cortex 2.0.0 is brand new software. As such, it might contain bugs and limitations. If you find any or encounter problems, please ask on our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Unveiling Cortex 2

TheHive Project’s Master Chefs are extremely happy to share, for free their latest recipe with the Cyber Threat Intelligence, Digital Forensics and Incident Response communities: Cortex 2.

As its predecessor, Cortex 2 is published under an AGPL v3 license and it introduces many important features that we brushed upon in a Dec 2017 post.

Screen Shot 2017-12-15 at 17.16.06
Cortex 2 — Architecture

Update: Cortex 2.0.1 was released since this post went live. It corrects a few bugs we uncovered in 2.0.0 as described in the changelog. Please install Cortex 2.0.1 instead of 2.0.0.

Authentication

Cortex 2 supports all the authentication methods that TheHive supports: LDAP, Active Directory, local accounts, API Keys, and X.509 SSO.

To connect your favorite Security Incident Response Platform with Cortex 2, you will need to update TheHive to Cerana 0.7 (TheHive 3.0.7) which was released today as well. This version fixes a regression pertaining to case templates introduced by Cerana 0.6 and is the first version to fully support Cortex 2’s API changes and authentication.

To make TheHive 3.0.7 analyze observables at scale through Cortex 2, you have to create an account on Cortex 2 with the read and analyze roles (see the next section) and generate the associated API Key. Next, feed the key in TheHive’s /etc/thehive/application.conf as described in the documentation et voilà !

TheHive 3.0.7 remains compatible with Cortex 1 and you can connect it to a mixed set of Cortex 1 and/or Cortex 2 instances with no issues.

Organizations, Analyzers and Rate Limiting

Cortex 2 introduces multi-tenancy through organizations and each organization can have its own set of users, with different roles, its own set of analyzers and, if necessary, rate limits that will prevent analysts from burning quotas.

Multi-tenancy has several interesting use cases. For instance, if you are the CSIRT or CERT of a large multinational organization with several regional teams, you can create an organization for each region within your constituency and enable the analyzers that they may need to use. Let’s assume that you bought a VirusTotal subscription that limits you to 5000 requests per month. You can configure the corresponding analyzers to give each region a fair share of that quota and keeping some requests for your own use.

In case you are a commercial CSIRT or an MSSP, you could do the same for your customers by installing only one Cortex 2 instance and creating an organization for each customer.

Screen Shot 2018-03-29 at 16.27.05.png
Configure an analyzer graphically and impose rate limits if necessary

User Roles

By default, Cortex 2 is shipped with the default cortex organization which sole purpose is to create other ones and manage the users within each organization and their associated powers. The cortex organization hosts all users with the superAdmin role and it cannot be used to configure or run analyzers.

As described in the new Quick Start Guide, after installing Cortex 2, updating its database and creating the first user who will have super admin powers, you’ll have to create your first organization and at least one user within that organization with orgAdmin rights.

Screen Shot 2018-03-29 at 16.33.02
Create an organization

You can then log out and log in using the orgAdmin account to create further users within that organization, enable and configure analyzers etc. Please note that no analyzer is enabled by default and you need at least v 1.9.0 of the cortex-analyzers repository. To update your set of analyzers to 1.9.0, please run git pull.

Screen Shot 2018-03-29 at 16.28.47
Manage users within an organization

Besides the superAdmin and orgAdmin roles, Cortex 2 introduces the read role which allows users to access analyzer reports and read them but not execute analyzers. For that, users need the analyze role (which implies the read role). orgAdmin users can also run analyzers. superAdmin users are limited to the default cortex organization. While they can create organizations and manage users within them, they cannot access analyzer configurations such as confidential API keys or job reports.

Screen Shot 2018-03-29 at 16.31.28
Job reports

Report Persistence and Caching

Cortex 2 relies on Elasticsearch 5.x to store many configuration items but also all the analyzer reports that have been generated. Unlike its predecessor, you won’t lose your existing reports should you need to restart the service or the host it is running on.

Cortex 2 also introduces report caching. By default the cache.job parameter is set to 10 minutes in /etc/cortex/application.conf. That means that if an analysis on a given observable with a defined TLP is requested and that a report has been previously generated in the last 10 minutes, Cortex 2 will serve that report instead of running a new analysis. This feature can help prevent soliciting analyzers, particularly those which require a subscription or have quotas, when there is no need to do so. Please note that this parameter is global to all the analyzers and all the organizations that are configured in the Cortex 2 instance. We do have plans to make it more granular in future versions.

Migrating from Cortex 1

If you are migrating from Cortex 1.x, we recommend that you:

  1. Save the configuration of your analyzers (which ones are enabled and what their configuration items are, such as users/passwords or API keys).
  2. Install Cortex 2.
  3. Edit /etc/cortex/application.conf to add the secret key as shown in Step 1 of the Quick Start Guide and point Cortex to the location of the analyzers.
  4. Follow the remaining steps of the Quick Start Guide to enable the analyzers you need and reinject their configuration.

What’s Next?

In the upcoming weeks, we will release a new version of Cortex4py in order to make it compatible with Cortex 2, continue the work we started with our MISP Project friends to support MISP attribute enrichment through Cortex 2 (MISP currently only supports enrichment using Cortex 1), and perform a long-overdue overhaul of our documentation.

Feeling Generous? Donate!

As you know, we are a FOSS project and donations are always welcome to make our products even better for the community.

All donations go to Creative Source, the non-profit organization we have created, and we will use them to improve TheHive, Cortex & Hippocampe but also to develop (even better) integrations with other FOSS solutions such as MISP.

So if you are feeling generous, please contact us at support@thehive-project.org.

Creative Source can also provide so-called professional, entreprise-grade support, help integrating the products, train your analysts before they drain or assist you in specific areas such as developing in-house analyzers for Cortex.

Support

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Correction: March 30, 2018
Instructions on how to update the cortex-analyzers have been added. Also, Cortex 2.0.1 was released to correct a few bugs in the previous version since this post went live.

Cerana 0.2: X-Pack Auth, Multi-source Dashboards

TheHive Chefs are happy to announce the immediate availability of their latest recipe: Cerana 0.2 (a.k.a. TheHive 3.0.2).

While TheHive 3.0.0 brought you dynamic dashboards among other niceties, the latest (and, of course, the greatest) version of your Mom’s favourite Security Incident Response Platform fixes a bug spotted by our longtime supporter Megan Roddie (merci !). Indeed, Nabil was running low on coffee so he didn’t make the necessary changes to support the new sighted toggle introduced by Cerana for file observables.

Cerana 0.2 also adds X-Pack authentication for Elasticsearch, a feature contributed by srilumpa. Thanks! To enable this functionality, and assuming the X-Pack plugin for Elasticsearch is installed, add the following section to /etc/thehive/application.conf:

search.username = "jessica"​​​​​

​search.password = "drink-beat-repeat"

Last but not least, we decided to make dynamic dashboards even more powerful. You can now create new graphs that support multiple series from multiple entities (or sources).

multiline2
Multi-line Dashboards Example — Number of IOCs imported from MISP vs. those imported from other sources

As Christmas is approaching, go ahead and play with dynamic dashboards to impress your management as soon as 2018 rears its head or truly drive your CTI and DFIR activities and plan well ahead how you should improve automation or collaboration (or beg for additional headcount).

multiline1.png
Multi-line Dashboards Example — How to create one

Ain’t that nifty? Who said bees aren’t nice? Joyeux Noël !

 

Feeling Generous? Donate!

As you know, we are a FOSS project and donations are always welcome to make our products even better for the community.

All donations go to Creative Source, the non-profit organization we have created, and we will use them to improve TheHive, Cortex & Hippocampe but also to develop (even better) integrations with other FOSS solutions such as MISP.

So if you are feeling generous, contact us at support@thehive-project.org. Of course the funds may also be used to keep Nabil happy by providing a steady flow of caffeine. 😉

Creative Source can also provide so-called professional, entreprise-grade support, help integrating the products, train your analysts before they drain or assist you in specific areas such as developing in-house analyzers for Cortex.

Download & Get Down to Work

If you have an existing installation of TheHive, please follow the migration guide.

If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, use Docker, install it from a binary or build it from sources.

Support

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Cortex 2: a Sneak Peek

Unless you’ve been living in a cave with no Internet connection during the last year or so, you certainly know a thing or two about Cortex, TheHive’s perfect sidekick, which allows you to analyze observables, at scale, using its 30+ analyzers.

As of this writing, the latest version of Cortex is 1.1.4. Cortex can be queried using its Web UI for quick assessment of an observable. But the true power of Cortex is unleashed when the engine is queried through its REST API, either from TheHive (which can leverage multiple Cortex instances), from alternative SIRPs (Security Incident Response Platforms), Threat Intelligence Platforms and programs thanks to Cortex4py. Indeed, when Cortex is called through the API, it can analyze large sets of observables. Each analysis generates a job. Jobs are queued on first-created, first-executed basis.

However, Cortex 1 has three limitations:

  1. It does not support authentication. If you install it and don’t shield it from abuse (using a firewall for example), anyone can submit analysis jobs and consume your query quotas for subscription-based, commercial services, for example. Non-CSIRT/CERT/SOC personnel or threat actors can also view all the jobs you’ve executed (what observables you have analyzed, using which analyzers and what the associated results were).
  2. It does not support rate-limiting. All it takes to ruin your quotas is an unexperienced analyst who’d create a case in TheHive from a MISP event containing thousands of attributes, select them all from the newly created case, and run them through various Cortex analyzers.
  3. It has no persistence. If you restart the Cortex service or the host it runs on, all your analysis results will disappear. Please note that if you query Cortex from TheHive, the latter will keep a copy of all the reports generated by the analyzers.

Moreover, analyzer configuration is not as easy as we’d like it to be. Enters Cortex 2.

Authentication, Organizations, Configuration and Rate Limiting

Cortex 2, due for release in February 2018, almost a year after the release of the first version, will support all the authentication methods TheHive supports: LDAP, Active Directory, local accounts, API keys and/or SSO using X.509 certificates (an experimental feature as of this writing).

Once created, users will be associated to an organization. Each organization has its own configuration: which analyzers are enabled, associated API keys and/or authentication credentials for services (VirusTotal, PassiveTotal, MISP, …) and a query quota.

For example, if you have an overall quota on VT for 10,000 queries/month, you can limit the number of queries to 5000 for org A, 3000 for org B and leave 2000 for other uses. Rate limits can be configured per month or per day.

Screen Shot 2017-12-15 at 17.16.06
Cortex 2 — Architecture

More on Organizations

Organizations will be ideal for multi-tenant Cortex instances deployed, for a example, by the central CSIRT of a large company. They can then create orgs for their regional SOCs. Commercial teams such as MSSPs will also be able to use a single instance to serve all their customers.

Graphical Interface Enhancements

Administrators will not have to edit /etc/cortex/application.conf by hand to enable and configure analyzers per org. They will be able to do so from the Web UI. The Web UI will also allow them to manage users, orgs and authentication tokens when applicable.

Report Persistence and Freshness

Cortex 2 will use ES 5 for storage, like TheHive. That way, you will no longer lose your existing jobs when you reboot the Cortex host or restart the service. You will also be able to query historical results to monitor changes and so on. We will also add an optional parameter to make Cortex 2 to serve the latest report generated by an analyzer if it is called again, on the same observable in the last X seconds or minutes. That way, we’ll avoid running the same queries again and again for the same observable and thus consuming quotas and CPU and storage resources.

Pricing

Cortex 2 is a significant development over Cortex 1 … but it’ll still cost you nothing as it will remain free and open source. We could feel you itching when you started reading this paragraph. Chill out! But if you are willing to support the project, you can donate to Creative Source, the non-profit organization we have created to sustain TheHive, Cortex and Hippocampe in the long run. Interested? Contact us at support@thehive-project.org then.