We could not leave for the week-end without issuing a minor release or two so here we go.
Starting from TheHive 3.0.1, an administrator has the ability to configure Cortex job polling by defining the time between two polls thanks to the cortex.refreshDelay parameter as well as the number of consecutive failures before giving up (via cortex.MaxRetryOnError). However, these settings prevent the service from starting correctly. TheHive 3.1.2 corrects this issue.
When running a job in Cortex with the exact same details, the function findSimilarJob is called. It should return results from any previous jobs, but in the latest versions (2.1.0, 2.1.1) it does not because of a change that went past our QA.
In a similar fashion, the GUI search function was broken. Cortex 2.1.2 fixes both issues.
Excuse my French but I Need Help
Keep calm. We speak French. So if you encounter any difficulty to update TheHive or Cortex, please join our user forum, contact us on Gitter, or send us an email at firstname.lastname@example.org. We are always ready to help as does our user community.
We released Cortex 2.1.0 as a release candidate back in July 31, 2018 along with TheHive 3.1.0-RC1. By then, the power duo which makes digital forensics, incident response and, to an extent, cyber threat intelligence, better, faster, happier, regular exercising gained the ability to perform active response.
We ate our own dog food for a couple of months. We found bugs. We added enhancements and we listened to the early adopters of these new major versions. And today we are thrilled to announce the availability of the stable release of Cortex 2.1.0 along with TheHive 3.1.0.
Cortex 2.1.0 restores the ability to query the analysis and response engine from MISP for enrichment purposes. A new version of the de facto standard for threat sharing should be released shortly as there are also some API-related issues on its side to make the integration fully working again.
Cortex 2.1.0 also gives you the ability to see the PAP (Permissible Actions Protocol) values for each analyzer as well as any custom cache values you might have configured.
After a nice summer break which allowed us to rest and enjoy Real Life™ (mostly) away from keyboards, screens and constant distractions and interruptions, we set to take into account the bug reports and feedback on TheHive 3.1.0-RC1 which we released a day or so before packing up for the mountains and elsewhere.
We are pleased to announce the immediate availability of Release Candidate 2 for TheHive 3.1.0. It contains numerous bug fixes and enhancements. You can read the full change log if you have nothing better to do with your life.
Among the changes we introduce in this new RC, we would like to highlight the following:
#652: the ability to set custom fields as mandatory.
#685: the quick case search box on the top navigation bar has been restored as it is highly useful for quick lookups without having to resort to the revamped search page.
#667: use alternative authentication methods when certificate authentification is enabled and the client does not present a certificate.
We encourage you to take TheHive 3.1.0-RC2 for a spin as quickly as you can and report any bug or issue so we can address them for the final release, scheduled in a couple of weeks. Cortex 2.1.0 is still at RC1 and we should be able to make a stable release at the same date as TheHive 3.1.0.
We would like to remind you that starting from these versions, we have two release channels: a stable one that should be used for production systems and a pre-release channel that should be used to try the release candidates such as TheHive 3.1-RC2 and Cortex 2.1-RC1, and help us iron out bugs before adding them to the stable channel. Those who love living on the bleeding edge may be tempted by running the release candidates on their production environment given all the candy and icing we added. They are at liberty of doing so but we don’t want to hear anyone one whining about an RC that broke everything and beyond.
Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at email@example.com. We are here to help.
Indeed, in the RBAC world, the conversion of any search queries submitted to TheHive into an Elasticsearch one is fully dependent on the user context. The user view must be kept within the boundaries of the group or groups to which they belong. Each search filter, each search parameter, must return only the results that the user can view.
The data scope needs to be clearly identified at the case level. To perform a search against task logs for example, TheHive will need to identify the parent task log, then identify the parent case and only then verify the scope. This is no small undertaking.
Similarities across cases or alerts, such as the Related Cases feature or the relationships between a given alert and existing cases, would need additional work that has not been clearly identified at this stage. But the difficulties do not stop there. Any element that has no clear relationship with case entities will have to be singled out and specific code would need to be added to limit access according to the RBAC rules. This will be clearly the case for the audit trail. Also, what should TheHive display when an analyst group is working on a case that shares observables with another one belonging to a different group? Shall it allow a limited view without any details so that groups may request from a super administrator to authorize both groups to collaborate on the investigation, something that distributed CERTs or SOCs in a large corporation may desire? Or shall it keep the data completely isolated as MSSPs which serve multiple customers with a single instance will require? We know the answer: make it configurable. But take a step back and think of the implications at the code (and security) level.
Contrary to the feature we added to Cortex 2, which allow multiple organizations to use a single Cortex instance, multi-tenancy in TheHive is a much more complex feature to implement and which is expected to have a significant impact on the platform’s performance. It will also need extreme caution to avoid blind spots that attackers (and not so innocent tenants) may exploit to circumvent scope limitations and extend their view to data they are not supposed to access. That’s why we had to delay it to Cerana 2 (TheHive 3.2), currently planned for the end of October 2018.
If you are well versed in Elasticsearch and Scala and willing to help implement this feature, please contact us at firstname.lastname@example.org.
TheHive Project Chefs are pleased to announce the immediate availability of Cortex-Analyzers 1.10.1. To install this new release and benefit from 11 new analyzers and some fixes:
$ cd /path/to/Cortex-Analyzers
$ sudo git pull
$ for I in analyzers/*/requirements.txt; do sudo -H pip2 install -r $I; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -r $I || true; done
After running these commands, read the Analyzer Requirements Guide, log into the Cortex 2 Web UI as an orgAdmin, click on the Refresh Analyzers button in the Cortex Web UI, configure the new analyzers and enjoy!
This analyzer comes in only one flavor. You need a valid FireEye iSIGHT Threat Intelligence subscription to use the analyzer. Retrieve the API key associated with your account and provide it as a value to the key parameter. Obtain the password associated with the API key and provide it as a value to the pwd parameter.
Determine whether an IP has known scanning activity using GreyNoise.
This analyzer comes in only one flavor. It can be used out of the box without configuration. However, if you make many requests, you need to obtain an API key. Please contact GreyNoise to ask for one. Once you get the API key, provide it as the value of the key parameter.
Query domains, IPs, hashes and URLs against IBM X-Force Threat Intelligence sharing platform.
This analyzer comes in only one flavor. Access to IBM X-Force Threat Exchange requires an IBM ID. Once you have access to the service, supply the URL of the service as value for the url parameter, the API key associated with your account as value for the key parameter and the associated password as the value of the pwd parameter.
Query Malwares.com and get reports on files, hashes, domain names and IP addresses.
The analyzer comes in two flavors:
– Malwares_pDNS_GetReport: get the latest Malwares report for a file,
hash, domain or an IP address.
– Malwares_pDNS_Scan: scan a file or URL.
You need to sign up for a Malwares.com account. An API key to use the service’s API should be associated with your account. Supply it as the value of the key parameter.
Query IP addresses and domain names against Mnemonic Passive DNS service.
This analyzer comes in two flavors:
Mnemonic_pDNS_Public: query Mnemonic’s public service.
When using the public service, the analyzer can be used out of the box with no further configuration. When using the closed service, you need to contact Mnemonic to get an API key which you’ll need to supply as the value of the key parameter.
This analyzer comes in only one flavor. No configuration is needed. It can be used out of the box.
Follow redirects of shortened URLs to reveal the real ones.
This analyzer comes in only one flavor.
No configuration is required. It can be used out of the box.
Warning: using this analyzer without extra caution might lead to unexpected consequences. For example, if the URL you are seeking to unshorten is an attacker-controlled one, you may end up leaving undesired traces in the threat actor’s infrastructure logs. The TLP values Cortex allows you to configure to prevent the use of an analyzer if the TLP associated with an observable is above the authorized level won’t be of much help since Unshortenlink have to access the shortened URL. Please do not activate this analyzer unless you (and your fellow analysts) know what they are doing.
YARA analyzer had a bug which was fixed in version 1.9.7 of the Cortex-analyzers repository. If you install 1.10, you’ll obviously benefit from the fix 😉
A permission problem that prevented using the Cuckoo Sandbox analyzer was corrected (thanks Felix Bauer!)
Correction: June 6, 2018 An earlier version of this post mentioned that GreyNoise needs an API key. That’s only necessary if a certain level of requests are made. Also, to configure analyzers, you need to be orgAdmin.
Rejoice DFIR fighters! Your favorite Security Incident Response Platform has just gotten better. TheHive 3.0.10 (codename Cerana 0.10) is just out of the oven. Thomas and Nabil, as the true code Chefs they are, have fixed a number of issues and added a few enhancements in this release. Read on, update and defend your constituency!
#446: keep observable reports when performing a case merge operation
#592: fix the error message when an observable is created with invalid data
#579: rotate application.log to keep it from becoming huge
#564: when submitting a job to Cortex, send the case ID as well
#563: poll the status of connectors (MISP, Cortex) every minute and update the UX display accordingly
#548: sort related cases by the amount of shared observables
#546: display start and end times for closed tasks
Something does not work as expected? You have troubles installing or upgrading? You have spotted new bugs? No worries, please join our user forum, contact us on Gitter, or send us an email at email@example.com. We are here to help.
Update: Cortex 2.0.1 was released since this post went live. It corrects a few bugs we uncovered in 2.0.0 as described in the changelog. Please install Cortex 2.0.1 instead of 2.0.0.
Cortex 2 supports all the authentication methods that TheHive supports: LDAP, Active Directory, local accounts, API Keys, and X.509 SSO.
To connect your favorite Security Incident Response Platform with Cortex 2, you will need to update TheHive to Cerana 0.7 (TheHive 3.0.7) which was released today as well. This version fixes a regression pertaining to case templates introduced by Cerana 0.6 and is the first version to fully support Cortex 2’s API changes and authentication.
To make TheHive 3.0.7 analyze observables at scale through Cortex 2, you have to create an account on Cortex 2 with the read and analyze roles (see the next section) and generate the associated API Key. Next, feed the key in TheHive’s /etc/thehive/application.confas described in the documentationet voilà !
TheHive 3.0.7 remains compatible with Cortex 1 and you can connect it to a mixed set of Cortex 1 and/or Cortex 2 instances with no issues.
Organizations, Analyzers and Rate Limiting
Cortex 2 introduces multi-tenancy through organizations and each organization can have its own set of users, with different roles, its own set of analyzers and, if necessary, rate limits that will prevent analysts from burning quotas.
Multi-tenancy has several interesting use cases. For instance, if you are the CSIRT or CERT of a large multinational organization with several regional teams, you can create an organization for each region within your constituency and enable the analyzers that they may need to use. Let’s assume that you bought a VirusTotal subscription that limits you to 5000 requests per month. You can configure the corresponding analyzers to give each region a fair share of that quota and keeping some requests for your own use.
In case you are a commercial CSIRT or an MSSP, you could do the same for your customers by installing only one Cortex 2 instance and creating an organization for each customer.
By default, Cortex 2 is shipped with the default cortex organization which sole purpose is to create other ones and manage the users within each organization and their associated powers. The cortex organization hosts all users with the superAdmin role and it cannot be used to configure or run analyzers.
As described in the new Quick Start Guide, after installing Cortex 2, updating its database and creating the first user who will have super admin powers, you’ll have to create your first organization and at least one user within that organization with orgAdmin rights.
You can then log out and log in using the orgAdmin account to create further users within that organization, enable and configure analyzers etc. Please note that no analyzer is enabled by default and you need at least v 1.9.0 of the cortex-analyzers repository. To update your set of analyzers to 1.9.0, please run git pull.
Besides the superAdmin and orgAdmin roles, Cortex 2 introduces the read role which allows users to access analyzer reports and read them but not execute analyzers. For that, users need the analyze role (which implies the read role). orgAdmin users can also run analyzers. superAdmin users are limited to the default cortex organization. While they can create organizations and manage users within them, they cannot access analyzer configurations such as confidential API keys or job reports.
Report Persistence and Caching
Cortex 2 relies on Elasticsearch 5.x to store many configuration items but also all the analyzer reports that have been generated. Unlike its predecessor, you won’t lose your existing reports should you need to restart the service or the host it is running on.
Cortex 2 also introduces report caching. By default the cache.job parameter is set to 10 minutes in /etc/cortex/application.conf. That means that if an analysis on a given observable with a defined TLP is requested and that a report has been previously generated in the last 10 minutes, Cortex 2 will serve that report instead of running a new analysis. This feature can help prevent soliciting analyzers, particularly those which require a subscription or have quotas, when there is no need to do so. Please note that this parameter is global to all the analyzers and all the organizations that are configured in the Cortex 2 instance. We do have plans to make it more granular in future versions.
Migrating from Cortex 1
If you are migrating from Cortex 1.x, we recommend that you:
Save the configuration of your analyzers (which ones are enabled and what their configuration items are, such as users/passwords or API keys).
Follow the remaining steps of the Quick Start Guide to enable the analyzers you need and reinject their configuration.
In the upcoming weeks, we will release a new version of Cortex4py in order to make it compatible with Cortex 2, continue the work we started with our MISP Project friends to support MISP attribute enrichment through Cortex 2 (MISP currently only supports enrichment using Cortex 1), and perform a long-overdue overhaul of our documentation.
Feeling Generous? Donate!
As you know, we are a FOSS project and donations are always welcome to make our products even better for the community.
All donations go to Creative Source, the non-profit organization we have created, and we will use them to improve TheHive, Cortex & Hippocampe but also to develop (even better) integrations with other FOSS solutions such as MISP.
Creative Source can also provide so-called professional, entreprise-grade support, help integrating the products, train your analysts before they drain or assist you in specific areas such as developing in-house analyzers for Cortex.