Lo and behold, we aren’t dead & TheHive Project ain’t toast! So, foremost, Happy New Year folks (we are still in January, right?)! We have some nice gifts coming up for you, gifts that have required very heavy-duty work. Of course, you might complain that we haven’t been responsive as of late but hey, there’s only so much we can do, right?
We’ll talk about those gifts in the upcoming weeks. In the meantime, there’s a new Cortex version in town and we urge you to upgrade to it, particularly if you consider deploying several Cortex nodes as a cluster. Indeed, Cortex 3.0.1 fixes a missing dependency that is required to set up such an architecture. Additionally (and this is the part where you should be paying attention), this version fixes the display of error messages pertaining to analyzer and responder operations, and also ensure that old responders and analyzers no longer show up once you clicked on that Refresh button.
Fixes and Enhancements
#244 Prevent the Play secret key from being displayed in the logs at startup. Nonetheless, you can still display it (for troubleshooting purposes or to make things easier for attackers that might have access to the logs and be interested in such a world-changing secret) by using the --show-secret option when starting Cortex
#243 fixes the display of error messages when analyzers & responders fail
This analyzer lets you check if an IP address has been registered in your DNS sinkhole. TheHive displays the analyzer results as follows:
This analyzer lets you determine whether an IP address has been reported as a threat on Cisco Talos Intelligence service. No special access to the service is required to run the analyzer.
TheHive displays the analyzer results as follows:
This analyzer has been enriched to display SHA-1 fingerprints. The long report format has been updated to reflect this new information.
FileInfo has been updated and is now able to parse PDF files and extract IOCs such as URLs, hosts, domains, IPs, hashes and many more.The analyzer does also support the last version of the extract-msg library.
VirusTotal and Python3
The VirusTotal analyzer, including all its flavours, now uses Python3 and an updated virustotal-api library.
Yeti API key
An optional API key can now be configured and used by the Yeti analyzer.
A hash computation has been fixed in this analyzer.
A first fix has been introduced to avoid this analyzer to crash when there is no content-description in content_header, and a second has been added to correct a header display issue.
IBM XForce Lookup
The analyzer has been improved to allow users to add a trailing / at the end of the API URL without breaking everything.
Updating your Analyzers in Cortex 2.x
Each analyzer and responder comes with its own, pip compatible requirements.txt file. Run the following commands to update your Cortex analyzers to the latest version:
cd path/to/Cortex-Analyzers git pull for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \ for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done
If you want to use dockerised analyzers and responders, ensure that the URL of the catalog.json file corresponding to the Cortex-Analyzers repository is registered inapplication.conf. Please note that this won’t work if you are tracking the stable catalog.
After doing so, do not forget to login to Cortex as an orgadmin, click on the Refresh Analyzers button, then Disable and Enable again each analyzer and responder. Analyzer (and responder) updates should occur automatically as long as docker.autoUpdate is set to true in application.conf (this is the default setting).
Update TheHive Report Templates
If you are using TheHive, you must import the new report templates in your instance as follows:
Do you know what the following set of commands achieve?
$ cd /opt/Cortex-Analyzers
$ sudo git pull
$ for I in $(find /opt/Cortex-Analyzers -name 'requirements.txt'); do sudo -H pip2 install -U -r $I; done \
&& for I in $(find /opt/Cortex-Analyzers -name 'requirements.txt'); do sudo -H pip3 install -U \
-r $I || true; done
The answer is obvious Doctor Watson, right? These highly readable commands (pun intended) allow you to update your Cortexanalyzers and responders to the latest stable versions, downloading new ones in the process, going over all the Python 2 and Python 3 dependencies to install the missing ones and upgrade the old ones to make sure they work correctly. These operations take quite a long time and cause some headaches in the process (Hello, I have Python 3.X and this dependency is no longer required, or Hi, I have an old version of Python 2 and it seems I need this other dependency).
And if you are lucky enough to get it running smoothly, you are still not done as you need to log in to the Cortex UI as an organisation administrator (unlike TheHive, Cortex supports multi-tenancy), click on the Refresh analyzers button under Organization > Analyzers then go to Organization > Responders and click on Refresh responders.
So while the answer to the opening question might be simple, updating analyzers and responders is far from being straightforward, to say the least, even if we forget the ugly fact that both are stored in a repository “conveniently” named Cortex-Analyzers*:
thehive@thehive-training:/opt/Cortex-Analyzers$ ls -d a* r*
Unnecessary Complexity Must Die
Your lovely, hard-working bees hate unnecessary complexity. Our project’s front page blatantly states our mission to bring Security Incident Response to the masses. And we have to stand by our words even if TheHive and Cortex are free, open source solutions and we do not gain anything from them save for the huge satisfaction of helping our fellow incident handlers level the fight against cybercriminals & all kinds of other animals of the APT (Advanced Persistent Troll Threat) bestiary.
There is only one possible solution: simplify the installation and update process of the current, official 115 analyzers and responders we have as of this writing, the future ones and any private or unofficial ones written in other programming languages such as those developed in Go by Rosetelecom-CERT.
Docker all the Things!
Starting from Cortex 3.0, the next major release of your favourite analysis and active response engine, all analyzers and responders will be dockerized. It will no longer be necessary to install them along with their various dependencies. They will be dowloaded from our cortexengine Docker organisation. Sysadmins might also configure automatic updates.
As a side advantage of using Docker, analyzers, and responders will also be isolated from each other which gives more flexibility and possibilities.
For those users who have private, custom analyzers and responders that they don’t want or can’t share with the community, several options will be available:
Continue managing their analyzers and responders in the same way as currently supported by Cortex 2 (i.e. launch them as processes, with no isolation whatsoever).
Dockerize them and store them locally on their Cortex instance.
Dockerize them and publish them on a Docker registry, either the official one or a private registry.
A Docker image of Cortex 3 will still be provided. It will contain a Docker engine to launch dockerized analyzers and responders using DIND (Docker in Docker).
It won’t be necessary to modify the code of the current, official analyzers and responders. A drone job will monitor the analyzer and responder repository and automatically build docker images when it detects changes.
The Cortex Web interface will be slightly modified to accommodate the whole process and allow adding in-house/private Certificate Authorities to allow Cortex to smoothly perform updates in those corporate environments where TLS/SSL inspection is enabled.
Nice Movie Trailer. When is it Coming to a Theatre near me?
We are working hard to get Cortex 3 out of the oven in Q1 (of this year, yes). We will reach out to you, dear reader, in due time, to help us test it and refine it before putting it on the digital shelf for free, as usual. We will provide a smooth migration path in order to move safely your current analyzers and responders and their configuration to Cortex 3.
The success of TheHive and Cortex continue to grow, far more than we initially foresaw. As far as we know, there are about a hundred organisations of different sizes and locations using or testing them. And as the number of users grows, so does the number of features, professional service and support requests.
We have tried addressing these requests through Creative Source, a nonprofit organisation (NPO). All but one company trusted us enough to make a donation and get tailored services for its needs in return. Most of the others either did not reply to our proposals or explained that their procurement process does not accommodate working with NPOs.
Some members of our core team are actively working on alternative options to ensure not only the viability of TheHive and Cortex as FOSS products on the long run but the ability to provide professional training, support, and services without freaking out highly bureaucratic, think-in-the-box-but-never-outside procurement departments.
Stay tuned 🐝
— (*) When the idea behind Cortex was born into our hive mind, we did not initially think about active response capabilities. So we naturally called the repository which was supposed to contain analyzers Cortex-Analyzers . When, at a later stage, we added responders, we put them in the same repository for obvious laziness pretences ¯\_(ツ)_/¯.
Correction: February 15, 2019 Typographical errors have been corrected. Some rewording has been made for the sake of clarity.