Last month (that should be… April… we are kinda losing track of time during the confinement), we made silently 2 patch releases for TheHive 3.4, our current stable version even if we have our hands full of soap and bleach as we are working on the eagerly awaited TheHive 4.0.0 final release: 3.4.1, shortly followed by 3.4.2. Your lovely bees are truly committed at keeping TheHive 3 branch buzzing well after 4.0.0 is out.
As usual, we’d like to start by thank the community for bringing the issues they discover to our attention. This is definitely one of the best contributions that we can get from you!
Released on April 25, 2020, 3.4.1 mainly fixed some docker-related issues as well as problems with OAuth2 and MISP integration, in addition to a few bugs, as described in the changelog.
Docker: TheHive fails to connect to Elasticsearch (NoNodeAvailableException) #854
Improved support for OpenID connect and OAuth2 #1110
TheHive’s Docker entrypoint logs the Play secret key at startup (… looking elsewhere hoping not to attract too much attention on this one) #1177
Configure TheHive’s first run using Docker Compose #1199
TheHive’s docker containers should be orchestration-ready #1204
MISP synchronisation: any attribute having the to_idsflag will be imported as ioc by TheHive. In the same way, when you export a case to MISP, observables which have the iocflag on will become MISP attributes for which to_ids is true #1273
We have been speaking about it for almost two years.
We have been making it for more than twelve months.
And the day finally came for TheHive 4, our latest and greatest version, to be unleashed! The Chefs behind TheHive Project’s Code Kitchen are very happy to announce the immediate availability of TheHive 4.0, Release Candidate 1 (or 4.0-RC1 or the cool geeks call it).
That’s your right but please don’t complain that, once released, the stable version is so buggy that it crashed your entire SOC operation and drove down the valuation of cryptocurrencies.
OK, OK… You Convinced Me. Where Should I Start?
Good! Well first things first. At this time, we produced documentation in kind of a rush while minding bazillion other things at the same time. We still need to proof-read it and enhance it.
If you are a seasoned TheHive user/contributor and you know what you are doing, please start with the installation guides for Debian or RedHat like operating systems. Then read the Quick start guide.
Noob warning: if you are completely new to TheHive, please use the latest stable version (3.4). TheHive 4.0-RC1 adds non-negligible complexity to accommodate advanced features such as RBAC and multi-tenancy and we will be very busy taking feedback from the intermediate/advanced users of our platform to make sure the stable version is rock-solid before we can recommend it to beginners.
Nice catch Eagle Eye! Indeed webhooks have not been integrated in RC1. They will make a reappearance in a future RC, before the stable release. We have integrated them into a new notification system that is almost finished but still needs some elbow grease.
But Are you Going to Maintain TheHive 3.4.x when 4.0 will be Released?
You should know that bees will never let you down unless you gas them with pesticides (i.e. non-constructive feedback) and exigences (don’t forget that this is FOSS and we try to do the best we can, right?). So TheHive 3.4.x is scheduled to be maintained around two years after the release of 4.0 as a stable version, unless Elasticsearch 6.x is EOL’ed before that. In which case, we will have no choice but phase out 3.4.x (moving to ES 7+ will require a lot of work that we can put elsewhere).
Help!!! TheHive 4.0-RC1 Does not Work!
Please open an issue on GitHub using the template made for TheHive4 if you’d like to report a bug on this version. We will monitor those closely and respond accordingly.
Correction: March 3, 2020 A new section regarding webhooks was added. In addition, a few typos were corrected.
Guess what? TheHive Project is still alive and well, as Saâd already mentioned in a previous blog post.
We’ve been certainly very busy lately, preparing the upcoming release of TheHive 4 and doing many other things beside working on our FOSS project. As a result, it took us a rather long time to merge several community contributions and reduce the sizeable pile of pull requests.
We would like to thank our contributors for their patience and we hope the cyberdefenders out there will enjoy the brand new Cortex-Analyzers 2.4.0 release, with many new analyzers, responders and some bug fixes & improvements, bringing the total to a whooping 138 analyzers (counting all flavors) and 10 responders!
Additionally, with this release, all analyzers are now using Python 3. No more Python 2 technodebt!
The EmailRep analyzer checks the reputation of an email address against the emailrep.io database.
This analyzer accesses IP-centric features provided by ipinfo.io. While the EmailRep API can be used without a token for limited usage, the ipinfo.io analyzer requires the configuration of an API token before use.
This analyzer lets you query the free Maltiverse Threat Intelligence platform for enrichment information about a particular hash, domain, ip or url.
TheHive displays the analyzer results as follows:
Andrea Garavaglia contributed this one a long time ago and we finally merged it into the Cortex-Analyzers repository. Andrea gave a talk about the background of this analyzer at the fourth MISP summit. You can watch it here.
In order to use the analyzer, you need to point it to a Neo4j server (you need to supply the host, port, login & password).
This analyzer lets you leverage PaloAlto Autofocus services. Provided you are an Autofocus customer and you have access to their API, you need to configure the analyzer with your username and a token key.
The analyzer comes with 3 flavors:
AUTOFOCUS_GetSampleAnalysis lets you request a full report for a given hash.
AUTOFOCUS_SearchIOC lets you research for samples linked to specific IoCs with datatypes like domain, fqdn, user-agent, imphash, ip, mutex, tag and url. Please note that mutex and tag are not default datatypes in TheHive. You need to create them in TheHive before you can leverage them.
AUTOFOCUS_SearchJSON lets you research for samples based on a complex JSON query.
Important: TheHive has no templates corresponding to this analyzer have been published yet. They will be provided in the near future.
This analyzer queries Team Cymru’s Malware Hash Registry for known malware hashes (MD5 or SHA-1). If it is malware and known by the service, it returns the last time it has been seen along with an approximate anti-virus detection percentage.
Overview of the New Responders
This responder allows the integration between TheHive/Cortex and KnowBe4’s User Events API. If a mail observable is tagged with a specified tag, corresponding to the responder’s configuration (e.g. phished), then the associated user will have a custom event added to their profile in KnowBe4.
A valid account on KnowBe4 and an API key are required to run this responder.
This responder sends observables you select to a Palo Alto Minemeld instance.
This responder performs actions on Wazuh, the open source security monitoring platform. It currently supports ad-hoc firewall blocking of ip observables.
New PassiveTotal flavors
Thanks to Brandon Dixon, the PassiveTotal analyzer gains 3 new flavors, bringing the total to 11:
PassiveTotal_Trackerslet you make tracker lookups on observables of type domain, fqdn and ip.
PassiveTotal_Host_Pairs let you make host pair lookups on observables of type domain, fqdn and ip.
PassiveTotal_Componentslets you make components lookup on observables of type domain, fqdn and ip.
They come with their own report templates.
The analyzer has been updated to support GreyNoise API v2, thanks to the contribution of Whitney Champion (#562).
New Data Types Supported by Some Analyzers
VirusTotal_GetReporthas been updated to allow requests for observables of type fqdn.
Threatcrowd has been updated to allow requests for observables of type domain.
Shodan has been updated to allow requests for observables of type fqdn.
[#602] The MISP analyzer was bumped to version 2.1 and is ready to use PyMISP 2.4.120.
Get It While Supply Lasts!
If you are using the dockerized analyzers & responders, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button.
If you are still using the old-style way of installing analyzers and responders, run the following commands:
cd path/to/Cortex-Analyzers git pull for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done
Once done, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button. Refer to the online Cortex documentation for further details.
Update TheHive Report Templates
If you are using TheHive, you must import the new report templates in your instance as follows:
TheHive Project’s Code Chefs, sweating under their toques, are working hard to deliver TheHive 4 as soon as feasible. The current target release date for the 1st release candidate (4.0-RC1) is Friday Feb 28, 2020.
While TheHive 4 will be the first release to support graph databases, multi-tenancy and Role-Based Access Control (RBAC), it will also have a nifty feature that can simplify the incident response and digital forensics workflows of our fellow cyberdefenders: TheHiveFS.
What is TheHiveFS?
Starting from TheHive 4, TheHive can be ‘mounted’ as a remote, WebDAV filesystem. The filesystem can be securely mounted if SSL/TLS is enabled.
Thanks to TheHiveFS, you can quickly access all files stored in TheHive directly from your investigation machine. This can speed up the time needed to triage and analyse evidence.
What Types of Files Can I Access through TheHiveFS?
You can access, in read-only mode, all files attached to task logs and all observables which datatype is file, as long as you are allowed to do so. Indeed, TheHive 4 comes with RBAC so if, for example, you are not allowed to view a case or some file observables in a case, you won’t be able to access them using TheHiveFS, the same way as if you are using the WebUI.
How Can I Mount TheHiveFS?
Assuming you have a WebDAV client, such as davfs2, use the following command line:
$ sudo mount -t davfs -o noexec https://myhiveinstance:9001/fs /mnt/dav/
You can also point your graphical file manager to:
You will need to authenticate using your username and password as if you were connecting to TheHive’s WebUI.
Mom, I’ve Just Stepped on a Landmine
Beware folks. When you download a file observable using TheHive’s WebUI, it will conveniently create a password-protected ZIP archive before handing you the file. This way, we avoid accidental double clicks that may lead to the infection and compromise of your workstation, which might reflect bad on you or force you to offer breakfast the next morning to all your fellow teammates.
There is no such protection if you use TheHiveFS. Let us repeat this so it sinks: there is no such protection if you use TheHiveFS.
If you mount TheHive’s filesystem and open by accident or by a great deal of will, as a true, hardcore fan of Russian roulette, a file observable that is in fact malware courtesy of your favourite bear, kitten, panda or eagle, you can’t blame your friendly bees. But we will empathise (and our empathy level is directly correlated to the amount of pains au chocolat you send our way).
You’ve been warned.
That Sounds Awesome! When Can I Try It?
As written above, you will be able to try TheHiveFS as soon as TheHive 4.0-RC1 is released and that’s currently planned for the end of February 2020.
You can cry, beg, try to bribe us with VC money, make the line at 3:00 AM in front of TheHive Store (there ain’t no such store, we are not Apple), this will not make us work any faster. But you can always cheer us up, hug us or just thank us. This means a lot to us and to the free, open source software flame we carry deep within our souls.
One More Thing…
While we aren’t Apple, we can mimic Steve to share one more information that will make TheHiveFS even more interesting by Q3-Q4 2020. We plan to add support for large file management in TheHive 4.1, the next major version after 4.0 as would Captain Obvious say. Thanks to this feature, you will be able to upload memory and disk images to TheHive and if your Internet line breaks, the upload will resume automatically.
One of the big improvements you’ll notice in Cortex 3 is the support for dockerized analyzers. And amongst some of their benefits, the installation process has been significantly simplified. So let’s assume you do not want to bang your head against Python, or other library dependencies. Then read one for a way to set up analyzers and run them quickly.
The following instructions have been tested on Ubuntu 18.04. If you already have a Cortex instance up and running, you can jump directly to the docker installation section below.
Install System Packages
Ensure your system contains the required packages:
Once Cortex is configured, restart the service with the following command, wait a few seconds and you should be able to connect to Cortex on https://<cortex_host>:9001 et voilà!
sudo service restart cortex
Important Note: The catalog analyzers.json contains information regarding versions of analyzers we consider stable and that are updated with bug fixes. This is typically synchronised with our master branch on Github. When you are using this catalog, you are de facto benefiting from the latest analyzer updates without needing to refresh anything in Cortex or setup again the configuration to get the latest version.
We also provide two additional catalogs:
analyzers-stable.json which strictly follows versions of analyzers if you do not want any uncontrolled updates. What does that mean in practice? You will have to click on the Refresh button in Cortex to update your analyzers, disable old ones and enable new versions. Moreover, you will also have to setup again their configuration. Typically, if you installed and setup Cortex with this catalog and the current version of FileInfo analyzers is 6.0, you won’t benefit from the next version, let’s say 6.1, unless you refresh Cortex.
analyzers-devel.jsonwhich contains information about new analyzers or version of analyzers that contains code that has been reviewed but not tested enough (or even not tested at all at times) to be deemed ready for production environments. This is typically synchronized with the develop branch of our Github repository.
Same goes for responders. All available catalogs for Cortex are published on bintray so you can choose the one that better fits your needs (or your risk/gambling profile :p).
If that sounds still complicated, worry not! We also wrote a little program that helps you prepare the environment and install everything. We ensured that it works well on Ubuntu 18.04. The program uses two environment variables to set up everything: FEEDERS_SYSACCOUNT and FEEDERS_HOMEDIR :
There are also sane, default settings in case you did not set any value. DigitalShadows2TH’s home directory will be set to /opt/thehive_feeders/DigitalShadows2TH. To use the script, run the following command line and follow the instructions:
Previous versions of DigitalShadows2TH allowed only one case template to be associated with alerts created by the feeder in TheHive. Starting from DigitalShadows2TH 2.4.0, you can define a case template for each type of incidents raised by DigitalShadows in the configuration file.
The configuration pertaining to TheHive looks as follows:
A template can be defined for all the following DigitalShadows incident types:
A default template can be defined for DigitalShadows incidents. If no template is found for a specific incident type, the feeder looks for the default template. if no default template is found, an empty case will be created by when importing the alert.
Update or Install
If you are not using docker, just pull the repository and update your configuration file with the new templates part for TheHive.
Update your Repository
$ cd /opt/TheHive_feeders/DigitalShadows2TH/
$ git pull
The configuration file has changed, so you need to update yours before running the program. A new templates section has been added for TheHive and the path has changed. It is now in the config/ directory of the project.
Install and Use via the Code Repository
$ cd /opt/TheHive_feeders
$ git clone https://github.com/TheHive-Project/DigitalShadows2TH.git
After that, follow the prerequisites and edit the configuration file. In /opt/TheHive_feeders/DigitalShadows2TH/config/ copy config.py.template to config.py and modify it.
Use cases and detailed configuration instructions can be found in the README file in the repository.
Some of our die-hard fans noticed that we silently released TheHive 3.3.0 a few days ago, after six release candidates. Well. Silently won’t be the right word to use in this case as we are drowning under work and feature requests and we sometimes postpone communication in favour of getting true real work done.
So, without any further ado, we are happy to announce the official availability of our latest (and of course greatest) release of the most-advanced, next-gen, HI (Human Intelligence), gluten-free, (add here any keyword that you fancy to help us get the Gartner attention and land in the Magic Quadrant™), free and open source Security Incident Response Platform Security Orchestration Automation & Response Platform.
As stated earlier, TheHive 3.3.0 went through the largest number of release candidates to date in order to ensure it contains more features than bugs (or unexpected functionality as our dear Nabil call them sometimes).
TheHive and Cortex are a huge success. According to our estimates, there are about a hundred organisations of different sizes and locations using or testing them. And as the number of users grows, so does the number of features, professional service and support requests.
We tried finding a solution to keep TheHive Project as healthy as possible. So we created Creative Source, a nonprofit organisation (NPO), in the hope that we could leverage it to hire more developers thanks to the generous donations of our large user community. Sadly, not everyone in this world is generous and altruistic. At the end, all but one company (yes, exactly one) trusted us enough to make a donation and get tailored services for its needs in return. Most of the others either did not reply to our proposals or explained that their procurement process does not accommodate working with NPOs.
As we informed you a few weeks ago, some members of our core team are finalising an alternative option to ensure not only the viability of TheHive and Cortex as FOSS products on the long run but the ability to provide professional training, support, and services without making highly bureaucratic, think-in-the-box-but-never-outside procurement departments freak out.
Spring is here and your favorite bees are busy buzzing flowers to prepare you the most palatable honey ever. In the meantime, we are pleased to announce the immediate availability of Cortex-Analyzers 1.16.0, which adds a new responder and three new analyzers to complete an already hefty collection, bringing the total to 117 analyzers and 3 responders!
One responder has been added:
FalconCustomIOC, contributed by Michael (#421). We don’t know the last name of Michael. That could be Jordan. Who knows?
We could not duly test some of these additions due to lack of access to the associated services or to our legendary laziness. So we would really appreciate it if you could test them and let us know whether they work or not.
The FalconCustomIOC responder let you submit observables from alerts or cases to Crowdstrike Falcon Custom IOC service.
Crowdstrike Falcon is a paid service. An account and an API key are required to configure and run this responder.
AbuseIPDB analyzer let you determine wether an IP has been reported as malicious or not to the AbuseIPDB web service.
An account and an API key is needed to configure and use this analyzer.
TheHive displays the analyzer results as follows:
The BackscatterIO analyzer lets you query the Backscatter.io service for IPs, networks or autonomous systems (AS). It comes in two flavors:
BackscatterIO_GetObservations: determine whether an observables has a known scanning activity
BackscatterIO_Enrichment: enrich your observables with additional information
TheHive displays this analyzer results as follow:
SoltraEdge analyzer lets you query any observable against theSoltra Edge platform.
To configure and use this analyzer, an account, a token key and the base URL of a SoltraEdge server are needed.
TheHive displays this analyzers result as follow:
Get It While Supply Lasts!
Each analyzer and responder comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.16.0, run the following commands:
cd path/to/Cortex-Analyzers git pull for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \ for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done
On February 10, 2019, we released TheHive 3.3-RC2. It contained new features such as bulk alert merging, alert sorting, observable tag autocompletion, exporting case tags to MISP & more. Since then your favourite French code Chefs have been beesy refining TheHive 3.3 through new release candidates while getting Cortex 3 ready for prime time.
Over the weekend, Nabil decided he was not working enough already during the week. So he drained his batteries to the very last drop to release TheHive 3.3-RC5 before he crashed headfirst into his bed for a long, reparative sleep. Cumulatively since RC2, we added several features and squashed 10 bugs as described below.
Note that release candidates are beta software. You can get TheHive 3.3-RC5 from the pre-release, beta repositories. As usual, we encourage you to test it and report any bugs or issues you spot so we can address them before the final release.
#888: add a new UI configuration admin section. One of the first use cases of this section consist in disabling creating empty cases (i.e. cases not associated with a template). It will be gradually improved with new use cases so speak your mind!
#893: disable the case template selection when trying to merge multiple alerts for which no case template exists.
Dear fellow incident handlers and cybercrime fighters around the world, the galaxy, the known and the unknown universe, first and foremost, all TheHive Project’s team would like to wish a wonderful new year 2019 to you and to your cherished relatives. We truly hope that eagles, pandas, kittens, babars, bears and all sorts of animals will stay out of the way. And remember that you don’t need to go bankrupt by purchasing so-called Next Gen™ magical solutions that work only when there’s a full moon and the page number of the book you are currently reading is 42 to investigate threats 😉
We would like to begin the year by introducing version 1.15.0 of Cortex analyzers, bringing the total number of analyzers to a whopping 113! And thanks to Kyle Parrish, this release improves the Mailer responder to allow you to specify a custom port number for your SMTP server and adds a new one to blacklist observables on Cisco Umbrella utilizing the Enforcement API. The Cisco Umbrella Blacklister responder will then add the tag Umbrella:blockedto the observable.
Cortex-Analyzers 1.15.0 also include fixes and enhancements for Eml_Parser, IBM X-Force, Fortiguard, and Shodan. Most of these modifications were contributed by our continuously growing user community. Thanks to all of those who help us in our mission to provide free and open source security incident response tools to the masses!
Please read the relevant sections in the Cortex installation guide to install or update your analyzers and responders in order to benefit from all this sweet & tasty honey.
3 new flavors for DomainTools, thanks again to ANSSI
This analyzer lets you query the Cyberprotect ThreatScore service for domains and IP addresses. No configuration is needed and it can be used out of the box.
TheHive displays the analyzer results as follows:
Have I Been Pwned
The HIBP_Query analyzer lets you check email addresses on Have I Been Pwned. You can use an optional parameter to include unverified breaches in the search results. Otherwise, it can be used without any additional configuration.
When called from TheHive, results would display as such:
As it name states, The Patrowl_GetReport analyzer will let you get the current PatrOwl report for a FQDN, a domain name or an IP address. You need a running PatrOwl instance or to have access to one to use the analyzer.
If you fire it from TheHive, it would display results as follows:
This analyzer comes in two flavors in order to get Whois data and Passive DNS details using SecurityTrails. To use both flavors, you will need an account for the service to retrieve the associated API key, which you need to configure the analyzers.
SecurityTrails_Passive_DNS displays results in TheHive as follows:
In addition to Shodan_Host and Shodan_Search, which allow you to obtain Shodan information on a host and the search results for a domain name, now you can get domain resolutions (Shodan_DNSResolve), obtain scan history results for an IP address (Shodan_Host_History), get information on a domain (Shodan_InfoDomain) and the reverse DNS resolutions for an IP address (Shodan_ReverseDNS).
The following DomainTools flavors were added to this release:
DomainTools_HostingHistory: get a list of historical registrant, name servers and IP addresses for a domain.
DomainTools_ReverseIPWhois: get a list of IP addresses which share the same registrant information. It applies to a mail, IP, or domain.
Moreover, please note that DomainTools_WhoisLookup now handles IP addresses in addition to domains and provides parsed results. DomainTools_WhoisLookup_IP is thus not needed anymore. Instead, DomainTools_WhoisLookupUnparsed has been added to do the same as DomainTools_WhoisLookup, except that the output results are unparsed.