Cortex 2.1.0: The Response Edition

We released Cortex 2.1.0 as a release candidate back in July 31, 2018 along with TheHive 3.1.0-RC1. By then, the power duo which makes digital forensics, incident response and, to an extent, cyber threat intelligence, better, faster, happier, regular exercising gained the ability to perform active response.

We ate our own dog food for a couple of months. We found bugs. We added enhancements and we listened to the early adopters of these new major versions. And today we are thrilled to announce the availability of the stable release of Cortex 2.1.0 along with TheHive 3.1.0.

Cortex 2.1.0 restores the ability to query the analysis and response engine from MISP for enrichment purposes. A new version of the de facto standard for threat sharing should be released shortly as there are also some API-related issues on its side to make the integration fully working again.

mighty-morphin-power-rangers.jpg
Source : Consequence of Sound

Cortex 2.1.0 also gives you the ability to see the PAP (Permissible Actions Protocol) values for each analyzer as well as any custom cache values you might have configured.

You can check out the full changelog and we highly encourage you to install this new version and let us know what you think of it.

Troubles?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

 

 

TheHive 3.1.0-RC2 is Out!

After a nice summer break which allowed us to rest and enjoy Real Life™ (mostly) away from keyboards, screens and constant distractions and interruptions, we set to take into account the bug reports and feedback on TheHive 3.1.0-RC1 which we released a day or so before packing up for the mountains and elsewhere.

We are pleased to announce the immediate availability of Release Candidate 2 for TheHive 3.1.0. It contains numerous bug fixes and enhancements. You can read the full change log if you have nothing better to do with your life.

Among the changes we introduce in this new RC, we would like to highlight the following:

  • #652: the ability to set custom fields as mandatory.
  • #685: the quick case search box on the top navigation bar has been restored as it is highly useful for quick lookups without having to resort to the revamped search page.
  • #667: use alternative authentication methods when certificate authentification is enabled and the client does not present a certificate.

We would also like to thank Marc-André Doll for contributing a nice UX enhancement and GitHub user secdecompiled for adding the ability to view alerts and cases with large descriptions in an easy way.

We encourage you to take TheHive 3.1.0-RC2 for a spin as quickly as you can and report any bug or issue so we can address them for the final release, scheduled in a couple of weeks. Cortex 2.1.0 is still at RC1 and we should be able to make a stable release at the same date as TheHive 3.1.0.

Important Note

We would like to remind you that starting from these versions, we have two release channels: a stable one that should be used for production systems and a pre-release channel that should be used to try the release candidates such as TheHive 3.1-RC2 and Cortex 2.1-RC1, and help us iron out bugs before adding them to the stable channel. Those who love living on the bleeding edge may be tempted by running the release candidates on their production environment given all the candy and icing we added. They are at liberty of doing so but we don’t want to hear anyone one whining about an RC that broke everything and beyond.

We also moved our package and binary repositories to https://bintray.com/thehive-project and Docker images are now under thehiveproject.

Please check TheHive Installation and Cortex Installation guides for further details.

Running into Troubles?

Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

There are More than 80 Ways to Analyze Them

TheHive Project Chefs are pleased to announce the immediate availability of Cortex-Analyzers 1.10.1. To install this new release and benefit from 11 new analyzers and some fixes:

$ cd /path/to/Cortex-Analyzers
$ sudo git pull
$ for I in analyzers/*/requirements.txt; do sudo -H pip2 install -r $I; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -r $I || true; done

After running these commands, read the Analyzer Requirements Guide,  log into the Cortex 2 Web UI as an orgAdmin, click on the Refresh Analyzers button in the Cortex Web UI, configure the new analyzers and enjoy!

click.png
Cortex: click on Refresh Analyzers after updating to the latest Cortex-analyzers version

If you are using TheHive, get the last version of  the report templates and import them into TheHive.

New Analyzers

We have added 11 analyzers to this release, bringing the total to 53 (83 if we count all the flavors):

  1. Crtsh: contributed by crackytsi
  2. Cybercrime-Tracker: contributed by ph34tur3
  3. FireEye iSIGHT: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  4. GreyNoise: contributed by Nclose
  5. IBM X-Force: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  6. Malwares: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  7. MnemonicPDNS: contributed by Michael Stensrud from the Nordic Financial CERT
  8. StaxxSearch: contributed by Robert Nixon
  9. StopForumSpam: contributed by Marc-André Doll from STARC (by EXAPROBE)
  10. ThreatCrowd: contributed by Rémi Allain from Cyberprotect
  11. Unshortenlink: contributed by Rémi Pointel from CERT-BDF

Crtsh

Get Crt.sh certificate transparency lists associated with a domain name. Crt.sh is an online service operated by the Comodo Certificate Authority.

The analyzer comes in only one flavor. No configuration is required. It can be used out of the box.

Screen Shot 2018-06-05 at 16.46.22.png

Screen Shot 2018-06-05 at 16.46.31.png
TheHive: Crtsh — Short and Long Report Samples

Cybercrime-Tracker

Use the Cybercrime-tracker.net service to assess whether an IP address, URL, domain, or FQDN has a C2 (Command & Control) entry in its database.

This analyzer comes in only one flavor. No configuration is required. It can be used out of the box.

Screenshot from 2018-06-05 14-43-54.png

Screenshot from 2018-06-05 14-43-37.png
TheHive: CyberCrime-Tracker — Short and Long Report Samples

FireEye iSIGHT

Leverage FireEye iSIGHT Threat Intelligence to qualify domains, IP addresses, hashes and URLs.

This analyzer comes in only one flavor. You need a valid FireEye iSIGHT Threat Intelligence subscription to use the analyzer. Retrieve the API key associated with your account and provide it as a value to the key parameter. Obtain the password associated with the API key and provide it as a value to the pwd parameter.

Fireeye.png
TheHive: FireEye iSIGHT — Long Report Sample (courtesy of Andrea Garavaglia)

GreyNoise

Determine whether an IP has known scanning activity using GreyNoise.

This analyzer comes in only one flavor. It can be used out of the box without configuration. However, if you make many requests, you need to obtain an API key. Please contact GreyNoise to ask for one. Once you get the API key, provide it as the value of the key parameter.

Screenshot from 2018-06-05 14-48-02.png

Screenshot from 2018-06-05 14-47-42.png
TheHive: GreyNoise — Short and Long Report Samples

IBM X-Force

Query domains, IPs, hashes and URLs against IBM X-Force Threat Intelligence sharing platform.

This analyzer comes in only one flavor. Access to IBM X-Force Threat Exchange requires an IBM ID. Once you have access to the service, supply the URL of the service as value for the url parameter, the API key associated with your account as value for the key parameter and the associated password as the value of the pwd parameter.

IBM-XForce.PNG
TheHive: IBM X-Force — Long Report Sample (courtesy of Andrea Garavaglia)

Malwares

Query Malwares.com and get reports on files, hashes, domain names and IP addresses.

The analyzer comes in two flavors:
– Malwares_pDNS_GetReport: get the latest Malwares report for a file,
hash, domain or an IP address.
– Malwares_pDNS_Scan: scan a file or URL.

You need to sign up for a Malwares.com account. An API key to use the service’s API should be associated with your account. Supply it as the value of the key parameter.

Screen Shot 2018-06-05 at 16.56.38.png

Screen Shot 2018-06-05 at 16.56.57.png
TheHive: Malwares_GetReport — Short and Long Report Samples

MnemonicPDNS

Query IP addresses and domain names against Mnemonic Passive DNS service.

This analyzer comes in two flavors:

  • Mnemonic_pDNS_Public: query Mnemonic’s public service.
  • Mnemonic_pDNS_Closed: query Mnemonic’s closed service.

When using the public service, the analyzer can be used out of the box with no further configuration. When using the closed service, you need to contact Mnemonic to get an API key which you’ll need to supply as the value of the key parameter.

Screenshot from 2018-06-05 14-46-40.png

Screenshot from 2018-06-05 14-46-19.png
TheHive: Mnemonic PDNS — Short and Long Report Samples

StaxxSearch

Fetch observable details from an Anomali STAXX instance.

This analyzer comes in only one flavor. You need to install an Anomali STAXX instance or to have access to one to use the analyzer. Supply the following parameters to the analyzer in order to use it:

  • auth_url: URL of the authentication endpoint.
  • query_url: URL of the intelligence endpoint.
  • username: the STAXX user name.
  • password: the STAXX password.
  • cert_check: boolean indicating whether the certificate of the endpoint must be checked or not.
  • cert_path: path to the CA on the system to validate the endpoint’s certificate if cert_check is true.

image0010.png

image002.png
TheHive: StaxxSearch — Short and Long Report Samples (courtesy of Robert Nixon)

StopForumSpam

Query StopForumSpam to check if an IP or email address is a known spammer.

You need to define the thresholds above which the analyzed observable should be marked as suspicious or malicious.

Screen Shot 2018-06-05 at 18.32.08.png

Screen Shot 2018-06-05 at 18.32.21.png
TheHive: StopForumSpam — Short and Long Report Samples

ThreatCrowd

Look up domains, mail and IP addresses on [ThreatCrowd(https://www.threatcrowd.org/), a service powered by AlienVault.

This analyzer comes in only one flavor. No configuration is needed. It can be used out of the box.

Screenshot from 2018-06-05 14-45-29.png

Screenshot from 2018-06-05 14-45-11.png
TheHive: Threatcrowd — Short and Long Report Samples

Unshortenlink

Follow redirects of shortened URLs to reveal the real ones.

This analyzer comes in only one flavor.

No configuration is required. It can be used out of the box.

Warning: using this analyzer without extra caution might lead to unexpected consequences. For example, if the URL you are seeking to unshorten is an attacker-controlled one, you may end up leaving undesired traces in the threat actor’s infrastructure logs. The TLP values Cortex allows you to configure to prevent the use of an analyzer if the TLP associated with an observable is above the authorized level won’t be of much help since Unshortenlink have to access the shortened URL. Please do not activate this analyzer unless you (and your fellow analysts) know what they are doing.

Pasted image at 2018_06_05 09_07 PM.png

Screen Shot 2018-06-05 at 21.05.45.png
TheHive: Unshortenlink — Short and Long Report Samples

Additional Enhancements

  • YARA analyzer had a bug which was fixed in version 1.9.7 of the Cortex-analyzers repository. If you install 1.10, you’ll obviously benefit from the fix 😉
  • A permission problem that prevented using the Cuckoo Sandbox analyzer was corrected (thanks Felix Bauer!)

Support

Something does not work as expected? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

CorrectionJune 6, 2018
An earlier version of this post mentioned that GreyNoise needs an API key. That’s only necessary if a certain level of requests are made. Also, to configure analyzers, you need to be orgAdmin.

TheHive 3.0.10 Released

Rejoice DFIR fighters! Your favorite Security Incident Response Platform has just gotten better. TheHive 3.0.10 (codename Cerana 0.10) is just out of the oven. Thomas and Nabil, as the true code Chefs they are, have fixed a number of issues and added a few enhancements in this release. Read on, update and defend your constituency!

Fixes

  • #446: keep observable reports when performing a case merge operation
  • #592: fix the error message when an observable is created with invalid data
  • #583: cases can be merged by using their case ID
  • #584: correct error messages when performing invalid searches
  • #586: a regression was introduced that prevented analyzer names from being reflected in the modal view of mini-reports
  • #512: there were situations were mini-reports won’t display upon job success
  • #522: under certain circumstances, TheHive failed to synchronize MISP events correctly
  • #555: private dashboards must not be searchable
  • #557: fix an error when a merged case is deleted

Enhancements

  • #579: rotate application.log to keep it from becoming huge
  • #564: when submitting a job to Cortex, send the case ID as well
  • #563: poll the status of connectors (MISP, Cortex) every minute and update the UX display accordingly
  • #548: sort related cases by the amount of shared observables
  • #546: display start and end times for closed tasks

Support

Something does not work as expected? You have troubles installing or upgrading? You have spotted new bugs? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Unveiling Cortex 2

TheHive Project’s Master Chefs are extremely happy to share, for free their latest recipe with the Cyber Threat Intelligence, Digital Forensics and Incident Response communities: Cortex 2.

As its predecessor, Cortex 2 is published under an AGPL v3 license and it introduces many important features that we brushed upon in a Dec 2017 post.

Screen Shot 2017-12-15 at 17.16.06
Cortex 2 — Architecture

Update: Cortex 2.0.1 was released since this post went live. It corrects a few bugs we uncovered in 2.0.0 as described in the changelog. Please install Cortex 2.0.1 instead of 2.0.0.

Authentication

Cortex 2 supports all the authentication methods that TheHive supports: LDAP, Active Directory, local accounts, API Keys, and X.509 SSO.

To connect your favorite Security Incident Response Platform with Cortex 2, you will need to update TheHive to Cerana 0.7 (TheHive 3.0.7) which was released today as well. This version fixes a regression pertaining to case templates introduced by Cerana 0.6 and is the first version to fully support Cortex 2’s API changes and authentication.

To make TheHive 3.0.7 analyze observables at scale through Cortex 2, you have to create an account on Cortex 2 with the read and analyze roles (see the next section) and generate the associated API Key. Next, feed the key in TheHive’s /etc/thehive/application.conf as described in the documentation et voilà !

TheHive 3.0.7 remains compatible with Cortex 1 and you can connect it to a mixed set of Cortex 1 and/or Cortex 2 instances with no issues.

Organizations, Analyzers and Rate Limiting

Cortex 2 introduces multi-tenancy through organizations and each organization can have its own set of users, with different roles, its own set of analyzers and, if necessary, rate limits that will prevent analysts from burning quotas.

Multi-tenancy has several interesting use cases. For instance, if you are the CSIRT or CERT of a large multinational organization with several regional teams, you can create an organization for each region within your constituency and enable the analyzers that they may need to use. Let’s assume that you bought a VirusTotal subscription that limits you to 5000 requests per month. You can configure the corresponding analyzers to give each region a fair share of that quota and keeping some requests for your own use.

In case you are a commercial CSIRT or an MSSP, you could do the same for your customers by installing only one Cortex 2 instance and creating an organization for each customer.

Screen Shot 2018-03-29 at 16.27.05.png
Configure an analyzer graphically and impose rate limits if necessary

User Roles

By default, Cortex 2 is shipped with the default cortex organization which sole purpose is to create other ones and manage the users within each organization and their associated powers. The cortex organization hosts all users with the superAdmin role and it cannot be used to configure or run analyzers.

As described in the new Quick Start Guide, after installing Cortex 2, updating its database and creating the first user who will have super admin powers, you’ll have to create your first organization and at least one user within that organization with orgAdmin rights.

Screen Shot 2018-03-29 at 16.33.02
Create an organization

You can then log out and log in using the orgAdmin account to create further users within that organization, enable and configure analyzers etc. Please note that no analyzer is enabled by default and you need at least v 1.9.0 of the cortex-analyzers repository. To update your set of analyzers to 1.9.0, please run git pull.

Screen Shot 2018-03-29 at 16.28.47
Manage users within an organization

Besides the superAdmin and orgAdmin roles, Cortex 2 introduces the read role which allows users to access analyzer reports and read them but not execute analyzers. For that, users need the analyze role (which implies the read role). orgAdmin users can also run analyzers. superAdmin users are limited to the default cortex organization. While they can create organizations and manage users within them, they cannot access analyzer configurations such as confidential API keys or job reports.

Screen Shot 2018-03-29 at 16.31.28
Job reports

Report Persistence and Caching

Cortex 2 relies on Elasticsearch 5.x to store many configuration items but also all the analyzer reports that have been generated. Unlike its predecessor, you won’t lose your existing reports should you need to restart the service or the host it is running on.

Cortex 2 also introduces report caching. By default the cache.job parameter is set to 10 minutes in /etc/cortex/application.conf. That means that if an analysis on a given observable with a defined TLP is requested and that a report has been previously generated in the last 10 minutes, Cortex 2 will serve that report instead of running a new analysis. This feature can help prevent soliciting analyzers, particularly those which require a subscription or have quotas, when there is no need to do so. Please note that this parameter is global to all the analyzers and all the organizations that are configured in the Cortex 2 instance. We do have plans to make it more granular in future versions.

Migrating from Cortex 1

If you are migrating from Cortex 1.x, we recommend that you:

  1. Save the configuration of your analyzers (which ones are enabled and what their configuration items are, such as users/passwords or API keys).
  2. Install Cortex 2.
  3. Edit /etc/cortex/application.conf to add the secret key as shown in Step 1 of the Quick Start Guide and point Cortex to the location of the analyzers.
  4. Follow the remaining steps of the Quick Start Guide to enable the analyzers you need and reinject their configuration.

What’s Next?

In the upcoming weeks, we will release a new version of Cortex4py in order to make it compatible with Cortex 2, continue the work we started with our MISP Project friends to support MISP attribute enrichment through Cortex 2 (MISP currently only supports enrichment using Cortex 1), and perform a long-overdue overhaul of our documentation.

Feeling Generous? Donate!

As you know, we are a FOSS project and donations are always welcome to make our products even better for the community.

All donations go to Creative Source, the non-profit organization we have created, and we will use them to improve TheHive, Cortex & Hippocampe but also to develop (even better) integrations with other FOSS solutions such as MISP.

So if you are feeling generous, please contact us at support@thehive-project.org.

Creative Source can also provide so-called professional, entreprise-grade support, help integrating the products, train your analysts before they drain or assist you in specific areas such as developing in-house analyzers for Cortex.

Support

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Correction: March 30, 2018
Instructions on how to update the cortex-analyzers have been added. Also, Cortex 2.0.1 was released to correct a few bugs in the previous version since this post went live.

Cerana 0.2: X-Pack Auth, Multi-source Dashboards

TheHive Chefs are happy to announce the immediate availability of their latest recipe: Cerana 0.2 (a.k.a. TheHive 3.0.2).

While TheHive 3.0.0 brought you dynamic dashboards among other niceties, the latest (and, of course, the greatest) version of your Mom’s favourite Security Incident Response Platform fixes a bug spotted by our longtime supporter Megan Roddie (merci !). Indeed, Nabil was running low on coffee so he didn’t make the necessary changes to support the new sighted toggle introduced by Cerana for file observables.

Cerana 0.2 also adds X-Pack authentication for Elasticsearch, a feature contributed by srilumpa. Thanks! To enable this functionality, and assuming the X-Pack plugin for Elasticsearch is installed, add the following section to /etc/thehive/application.conf:

search.username = "jessica"​​​​​

​search.password = "drink-beat-repeat"

Last but not least, we decided to make dynamic dashboards even more powerful. You can now create new graphs that support multiple series from multiple entities (or sources).

multiline2
Multi-line Dashboards Example — Number of IOCs imported from MISP vs. those imported from other sources

As Christmas is approaching, go ahead and play with dynamic dashboards to impress your management as soon as 2018 rears its head or truly drive your CTI and DFIR activities and plan well ahead how you should improve automation or collaboration (or beg for additional headcount).

multiline1.png
Multi-line Dashboards Example — How to create one

Ain’t that nifty? Who said bees aren’t nice? Joyeux Noël !

 

Feeling Generous? Donate!

As you know, we are a FOSS project and donations are always welcome to make our products even better for the community.

All donations go to Creative Source, the non-profit organization we have created, and we will use them to improve TheHive, Cortex & Hippocampe but also to develop (even better) integrations with other FOSS solutions such as MISP.

So if you are feeling generous, contact us at support@thehive-project.org. Of course the funds may also be used to keep Nabil happy by providing a steady flow of caffeine. 😉

Creative Source can also provide so-called professional, entreprise-grade support, help integrating the products, train your analysts before they drain or assist you in specific areas such as developing in-house analyzers for Cortex.

Download & Get Down to Work

If you have an existing installation of TheHive, please follow the migration guide.

If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, use Docker, install it from a binary or build it from sources.

Support

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Cortex 2: a Sneak Peek

Unless you’ve been living in a cave with no Internet connection during the last year or so, you certainly know a thing or two about Cortex, TheHive’s perfect sidekick, which allows you to analyze observables, at scale, using its 30+ analyzers.

As of this writing, the latest version of Cortex is 1.1.4. Cortex can be queried using its Web UI for quick assessment of an observable. But the true power of Cortex is unleashed when the engine is queried through its REST API, either from TheHive (which can leverage multiple Cortex instances), from alternative SIRPs (Security Incident Response Platforms), Threat Intelligence Platforms and programs thanks to Cortex4py. Indeed, when Cortex is called through the API, it can analyze large sets of observables. Each analysis generates a job. Jobs are queued on first-created, first-executed basis.

However, Cortex 1 has three limitations:

  1. It does not support authentication. If you install it and don’t shield it from abuse (using a firewall for example), anyone can submit analysis jobs and consume your query quotas for subscription-based, commercial services, for example. Non-CSIRT/CERT/SOC personnel or threat actors can also view all the jobs you’ve executed (what observables you have analyzed, using which analyzers and what the associated results were).
  2. It does not support rate-limiting. All it takes to ruin your quotas is an unexperienced analyst who’d create a case in TheHive from a MISP event containing thousands of attributes, select them all from the newly created case, and run them through various Cortex analyzers.
  3. It has no persistence. If you restart the Cortex service or the host it runs on, all your analysis results will disappear. Please note that if you query Cortex from TheHive, the latter will keep a copy of all the reports generated by the analyzers.

Moreover, analyzer configuration is not as easy as we’d like it to be. Enters Cortex 2.

Authentication, Organizations, Configuration and Rate Limiting

Cortex 2, due for release in February 2018, almost a year after the release of the first version, will support all the authentication methods TheHive supports: LDAP, Active Directory, local accounts, API keys and/or SSO using X.509 certificates (an experimental feature as of this writing).

Once created, users will be associated to an organization. Each organization has its own configuration: which analyzers are enabled, associated API keys and/or authentication credentials for services (VirusTotal, PassiveTotal, MISP, …) and a query quota.

For example, if you have an overall quota on VT for 10,000 queries/month, you can limit the number of queries to 5000 for org A, 3000 for org B and leave 2000 for other uses. Rate limits can be configured per month or per day.

Screen Shot 2017-12-15 at 17.16.06
Cortex 2 — Architecture

More on Organizations

Organizations will be ideal for multi-tenant Cortex instances deployed, for a example, by the central CSIRT of a large company. They can then create orgs for their regional SOCs. Commercial teams such as MSSPs will also be able to use a single instance to serve all their customers.

Graphical Interface Enhancements

Administrators will not have to edit /etc/cortex/application.conf by hand to enable and configure analyzers per org. They will be able to do so from the Web UI. The Web UI will also allow them to manage users, orgs and authentication tokens when applicable.

Report Persistence and Freshness

Cortex 2 will use ES 5 for storage, like TheHive. That way, you will no longer lose your existing jobs when you reboot the Cortex host or restart the service. You will also be able to query historical results to monitor changes and so on. We will also add an optional parameter to make Cortex 2 to serve the latest report generated by an analyzer if it is called again, on the same observable in the last X seconds or minutes. That way, we’ll avoid running the same queries again and again for the same observable and thus consuming quotas and CPU and storage resources.

Pricing

Cortex 2 is a significant development over Cortex 1 … but it’ll still cost you nothing as it will remain free and open source. We could feel you itching when you started reading this paragraph. Chill out! But if you are willing to support the project, you can donate to Creative Source, the non-profit organization we have created to sustain TheHive, Cortex and Hippocampe in the long run. Interested? Contact us at support@thehive-project.org then.