On February 10, 2019, we released TheHive 3.3-RC2. It contained new features such as bulk alert merging, alert sorting, observable tag autocompletion, exporting case tags to MISP & more. Since then your favourite French code Chefs have been beesy refining TheHive 3.3 through new release candidates while getting Cortex 3 ready for prime time.
Over the weekend, Nabil decided he was not working enough already during the week. So he drained his batteries to the very last drop to release TheHive 3.3-RC5 before he crashed headfirst into his bed for a long, reparative sleep. Cumulatively since RC2, we added several features and squashed 10 bugs as described below.
Note that release candidates are beta software. You can get TheHive 3.3-RC5 from the pre-release, beta repositories. As usual, we encourage you to test it and report any bugs or issues you spot so we can address them before the final release.
#888: add a new UI configuration admin section. One of the first use cases of this section consist in disabling creating empty cases (i.e. cases not associated with a template). It will be gradually improved with new use cases so speak your mind!
#893: disable the case template selection when trying to merge multiple alerts for which no case template exists.
Do you know what the following set of commands achieve?
$ cd /opt/Cortex-Analyzers
$ sudo git pull
$ for I in $(find /opt/Cortex-Analyzers -name 'requirements.txt'); do sudo -H pip2 install -U -r $I; done \
&& for I in $(find /opt/Cortex-Analyzers -name 'requirements.txt'); do sudo -H pip3 install -U \
-r $I || true; done
The answer is obvious Doctor Watson, right? These highly readable commands (pun intended) allow you to update your Cortexanalyzers and responders to the latest stable versions, downloading new ones in the process, going over all the Python 2 and Python 3 dependencies to install the missing ones and upgrade the old ones to make sure they work correctly. These operations take quite a long time and cause some headaches in the process (Hello, I have Python 3.X and this dependency is no longer required, or Hi, I have an old version of Python 2 and it seems I need this other dependency).
And if you are lucky enough to get it running smoothly, you are still not done as you need to log in to the Cortex UI as an organisation administrator (unlike TheHive, Cortex supports multi-tenancy), click on the Refresh analyzers button under Organization > Analyzers then go to Organization > Responders and click on Refresh responders.
So while the answer to the opening question might be simple, updating analyzers and responders is far from being straightforward, to say the least, even if we forget the ugly fact that both are stored in a repository “conveniently” named Cortex-Analyzers*:
thehive@thehive-training:/opt/Cortex-Analyzers$ ls -d a* r*
Unnecessary Complexity Must Die
Your lovely, hard-working bees hate unnecessary complexity. Our project’s front page blatantly states our mission to bring Security Incident Response to the masses. And we have to stand by our words even if TheHive and Cortex are free, open source solutions and we do not gain anything from them save for the huge satisfaction of helping our fellow incident handlers level the fight against cybercriminals & all kinds of other animals of the APT (Advanced Persistent Troll Threat) bestiary.
There is only one possible solution: simplify the installation and update process of the current, official 115 analyzers and responders we have as of this writing, the future ones and any private or unofficial ones written in other programming languages such as those developed in Go by Rosetelecom-CERT.
Docker all the Things!
Starting from Cortex 3.0, the next major release of your favourite analysis and active response engine, all analyzers and responders will be dockerized. It will no longer be necessary to install them along with their various dependencies. They will be dowloaded from our cortexengine Docker organisation. Sysadmins might also configure automatic updates.
As a side advantage of using Docker, analyzers, and responders will also be isolated from each other which gives more flexibility and possibilities.
For those users who have private, custom analyzers and responders that they don’t want or can’t share with the community, several options will be available:
Continue managing their analyzers and responders in the same way as currently supported by Cortex 2 (i.e. launch them as processes, with no isolation whatsoever).
Dockerize them and store them locally on their Cortex instance.
Dockerize them and publish them on a Docker registry, either the official one or a private registry.
A Docker image of Cortex 3 will still be provided. It will contain a Docker engine to launch dockerized analyzers and responders using DIND (Docker in Docker).
It won’t be necessary to modify the code of the current, official analyzers and responders. A drone job will monitor the analyzer and responder repository and automatically build docker images when it detects changes.
The Cortex Web interface will be slightly modified to accommodate the whole process and allow adding in-house/private Certificate Authorities to allow Cortex to smoothly perform updates in those corporate environments where TLS/SSL inspection is enabled.
Nice Movie Trailer. When is it Coming to a Theatre near me?
We are working hard to get Cortex 3 out of the oven in Q1 (of this year, yes). We will reach out to you, dear reader, in due time, to help us test it and refine it before putting it on the digital shelf for free, as usual. We will provide a smooth migration path in order to move safely your current analyzers and responders and their configuration to Cortex 3.
The success of TheHive and Cortex continue to grow, far more than we initially foresaw. As far as we know, there are about a hundred organisations of different sizes and locations using or testing them. And as the number of users grows, so does the number of features, professional service and support requests.
We have tried addressing these requests through Creative Source, a nonprofit organisation (NPO). All but one company trusted us enough to make a donation and get tailored services for its needs in return. Most of the others either did not reply to our proposals or explained that their procurement process does not accommodate working with NPOs.
Some members of our core team are actively working on alternative options to ensure not only the viability of TheHive and Cortex as FOSS products on the long run but the ability to provide professional training, support, and services without freaking out highly bureaucratic, think-in-the-box-but-never-outside procurement departments.
Stay tuned 🐝
— (*) When the idea behind Cortex was born into our hive mind, we did not initially think about active response capabilities. So we naturally called the repository which was supposed to contain analyzers Cortex-Analyzers . When, at a later stage, we added responders, we put them in the same repository for obvious laziness pretences ¯\_(ツ)_/¯.
Correction: February 15, 2019 Typographical errors have been corrected. Some rewording has been made for the sake of clarity.
We urge our fellow fighters of cybercrime and funny and not-so-funny animals-in-disguise, to update their Cortex analyzers to the latest 1.15.2 version which plugs a serious vulnerability in the Unshortenlink analyzer and fixes several bugs.
On Jan 24, Alexandre Basquin discovered a SSRF vulnerability in the Unshortenlink 1.0 analyzer. By exploiting it, an attacker which has access to a Cortex instance with an analyze role can scan the ports of localhost and possibly of all the hosts reachable by Cortex. This could be automated through Cortex4py by making repetitive calls to the API and thus scanning multiple ports & hosts. In essence, the attacker can perform reconnaissance thanks to Unshortenlink and gain knowledge on which ports are open and which aren’t.
Version 1.1 of Unshortenlink, included in Cortex-Analyzers 1.15.2, plugs this vulnerability by disabling submissions containing IPs & ports. Only URLs are now accepted.
Additionally, Cortex-Analyzers 1.15.2 corrects the following bugs:
#416: the Mailer responder now supports UTF-8 encoding.
#410: remove wrapping of the results produced by crt.sh as a list which ends up breaking the output of the report template. Contributed by Thomas Kastner.
#409: enum is not required for Python 3.4+ to make the MISP Search analyzer work.
#408: FileInfo’s Manalyze plugin did not work after Manalyze renamed plugin_btcaddress to plugin_cryptoaddress.
#406: fix a broken link in the Cymon_Check_IP report, submitted by Manabu Niseki.
click on Import templates button and select the downloaded package
Wait a Minute! Where’s the Blog Post about 1.15.1?
Good catch! There was no blog post about Cortex-Analyzers 1.15.1. Call us lazy but there was nothing Earth-shattering, pole-shifting in that release. Nonetheless, here are a list of fixes included in that release and from which you’ll inherit automatically if you update 1.15.0 to 1.15.2 directly:
#402: Malwares analyzer code relied on functionality that’s only available in Python 3.7+. It has been fixed to work with 3.4 and up thanks to the work of Arcuri Davide.
#404: fixes a bad folder renaming in the HIBP (Have I Been Pwned) analyzer.
#398: MISP Search analyzer wouldn’t run without the enum dependency. Contributed by Mars Huang. Later improved in 1.15.2 (see above).
Dear fellow incident handlers and cybercrime fighters around the world, the galaxy, the known and the unknown universe, first and foremost, all TheHive Project’s team would like to wish a wonderful new year 2019 to you and to your cherished relatives. We truly hope that eagles, pandas, kittens, babars, bears and all sorts of animals will stay out of the way. And remember that you don’t need to go bankrupt by purchasing so-called Next Gen™ magical solutions that work only when there’s a full moon and the page number of the book you are currently reading is 42 to investigate threats 😉
We would like to begin the year by introducing version 1.15.0 of Cortex analyzers, bringing the total number of analyzers to a whopping 113! And thanks to Kyle Parrish, this release improves the Mailer responder to allow you to specify a custom port number for your SMTP server and adds a new one to blacklist observables on Cisco Umbrella utilizing the Enforcement API. The Cisco Umbrella Blacklister responder will then add the tag Umbrella:blockedto the observable.
Cortex-Analyzers 1.15.0 also include fixes and enhancements for Eml_Parser, IBM X-Force, Fortiguard, and Shodan. Most of these modifications were contributed by our continuously growing user community. Thanks to all of those who help us in our mission to provide free and open source security incident response tools to the masses!
Please read the relevant sections in the Cortex installation guide to install or update your analyzers and responders in order to benefit from all this sweet & tasty honey.
3 new flavors for DomainTools, thanks again to ANSSI
This analyzer lets you query the Cyberprotect ThreatScore service for domains and IP addresses. No configuration is needed and it can be used out of the box.
TheHive displays the analyzer results as follows:
Have I Been Pwned
The HIBP_Query analyzer lets you check email addresses on Have I Been Pwned. You can use an optional parameter to include unverified breaches in the search results. Otherwise, it can be used without any additional configuration.
When called from TheHive, results would display as such:
As it name states, The Patrowl_GetReport analyzer will let you get the current PatrOwl report for a FQDN, a domain name or an IP address. You need a running PatrOwl instance or to have access to one to use the analyzer.
If you fire it from TheHive, it would display results as follows:
This analyzer comes in two flavors in order to get Whois data and Passive DNS details using SecurityTrails. To use both flavors, you will need an account for the service to retrieve the associated API key, which you need to configure the analyzers.
SecurityTrails_Passive_DNS displays results in TheHive as follows:
In addition to Shodan_Host and Shodan_Search, which allow you to obtain Shodan information on a host and the search results for a domain name, now you can get domain resolutions (Shodan_DNSResolve), obtain scan history results for an IP address (Shodan_Host_History), get information on a domain (Shodan_InfoDomain) and the reverse DNS resolutions for an IP address (Shodan_ReverseDNS).
The following DomainTools flavors were added to this release:
DomainTools_HostingHistory: get a list of historical registrant, name servers and IP addresses for a domain.
DomainTools_ReverseIPWhois: get a list of IP addresses which share the same registrant information. It applies to a mail, IP, or domain.
Moreover, please note that DomainTools_WhoisLookup now handles IP addresses in addition to domains and provides parsed results. DomainTools_WhoisLookup_IP is thus not needed anymore. Instead, DomainTools_WhoisLookupUnparsed has been added to do the same as DomainTools_WhoisLookup, except that the output results are unparsed.
Guess what? Our integration with MISP, the de facto standard for threat sharing, has just gotten better with our latest beta release: TheHive 3.2.0-RC1.
While you could synchronize TheHive with one or multiple MISP instances in earlier versions and select events using filters like their age, the number of attributes they contain or exclude those which are created by specific organisations or contain one or several black-listed tags, 3.2.0-RC1 adds the ability to whitelist tags, thus limiting the events that would show up in TheHive’s Alerts pane to only those which have been tagged with labels your SOC/CSIRT/CERT needs to act on. This can be very useful for example if your Cyber Threat Intelligence analysts pre-select or create events in MISP and tag for SOC consumption those that need to be acted on.
To use this feature, use the whitelist.tags parameter in the MISP section of TheHive’s application.conf as described in the documentation.
TheHive 3.2.0-RC1 will also show you the description of an observable if any while hovering over one in the Observables tab. You can also see observable tags when previewing an alert in the Alerts pane.
Last but not least, some users reported severe problems when they enabled TLS/SSL directly on TheHive without resorting to a reverse proxy such as NGINX. Blame that on the crappy TLS support in Play framework ;-). So we highly recommend using a reverse proxy for that purpose, and delegate authentication to it if you are relying on X.509 authentication, as TheHive 3.2.0-RC1 allows you to. Please check the Single Sign-On on TheHive with X.509 Certificates guide for further information.
For additional details on this release, please check the full changelog.
Warning Capt’n Robinson!
The RC in 3.2.0-RC1 stands for Release Candidate. Please help us make a great stable release out of it by testing it as thoroughly as possible and reporting back any bugs or issues you encounter so we can address them before the final release. You’ll find this release candidate in the pre-release, beta repositories.
Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at email@example.com. We are here to help.