On Spring, Bees and Cortex-Analyzers 1.16.0

Spring is here and your favorite bees are busy buzzing flowers to prepare you the most palatable honey ever. In the meantime, we are pleased to announce the immediate availability of Cortex-Analyzers 1.16.0, which adds a new responder and three new analyzers to complete an already hefty collection, bringing the total to 117 analyzers and 3 responders!

Release Overview

One responder has been added:

  • FalconCustomIOC, contributed by Michael (#421). We don’t know the last name of Michael. That could be Jordan. Who knows?

Three analyzers have been added:

We could not duly test some of these additions due to lack of access to the associated services or to our legendary laziness. So we would really appreciate it if you could test them and let us know whether they work or not.

FalconCustomIOC

The FalconCustomIOC responder let you submit observables from alerts or cases to Crowdstrike Falcon Custom IOC service.

Crowdstrike Falcon is a paid service. An account and an API key are required to configure and run this responder.

AbuseIPDB

AbuseIPDB analyzer let you determine wether an IP has been reported as malicious or not to the AbuseIPDB web service.

An account and an API key is needed to configure and use this analyzer.

TheHive displays the analyzer results as follows:

AbuseIPDB: short report
AbuseIPDB: long report

BackscatterIO

The BackscatterIO analyzer lets you query the Backscatter.io service for IPs, networks or autonomous systems (AS). It comes in two flavors:

  • BackscatterIO_GetObservations: determine whether an observables has a known scanning activity
  • BackscatterIO_Enrichment: enrich your observables with additional information

TheHive displays this analyzer results as follow:

Backscatter.io:short report
Backscatter.io GetObservations: long report
Bascatter.io GetObservations: long report

SoltraEdge

SoltraEdge analyzer lets you query any observable against theSoltra Edge platform.

To configure and use this analyzer, an account, a token key and the base URL of a SoltraEdge server are needed.

TheHive displays this analyzers result as follow:

SoltraEdge: short report
SoltraEdge: long report

Get It While Supply Lasts!

Each analyzer and responder comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.16.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Here comes the Sun (and TheHive 3.3-RC5)

On February 10, 2019, we released TheHive 3.3-RC2. It contained new features such as bulk alert merging, alert sorting, observable tag autocompletion, exporting case tags to MISP & more. Since then your favourite French code Chefs have been beesy refining TheHive 3.3 through new release candidates while getting Cortex 3 ready for prime time.

Over the weekend, Nabil decided he was not working enough already during the week. So he drained his batteries to the very last drop to release TheHive 3.3-RC5 before he crashed headfirst into his bed for a long, reparative sleep. Cumulatively since RC2, we added several features and squashed 10 bugs as described below.

Note that release candidates are beta software. You can get TheHive 3.3-RC5 from the pre-release, beta repositories. As usual, we encourage you to test it and report any bugs or issues you spot so we can address them before the final release. 

Check TheHive Installation guide for further details.

New Features

  • #485: disable clickable widgets when editing a dashboard.
  • #820: provide direct access to MISP events for those alerts that have been generated from such events.
Direct access to the MISP event from which this alert has been generated from
  • #829: improve password-protected ZIP imports.
  • #831: add auto-completion to case and observable tags in their respective Details pages.
  • #838: when you add an observable that already exists in another case, indicate whether the existing observable is an IOC or not.
  • #870: provide a link to copy the alert ID to the clipboard which allows you to easily manipulate an alert through TheHive4py.
Now the Alert ID can be directly copied to the clipboard without resorting to your browser dev mode
  • #884: add a Related Alerts tab to the Case view.
The new Related Alerts tab in Case View
  • #888: add a new UI configuration admin section. One of the first use cases of this section consist in disabling creating empty cases (i.e. cases not associated with a template). It will be gradually improved with new use cases so speak your mind!
  • #893: disable the case template selection when trying to merge multiple alerts for which no case template exists.
  • #897: mouseover text for alert preview.

Fixed Bugs

  • #856: various fixes related to alert updates and, when applicable, associated cases.
  • #869: use the observable TLP instead of the case TLP to decide whether a Cortex responder could be executed or not.
  • #871: fix log messages for MISP synchronisation.
  • #874: fix typo in alert labels (credits: Zachary Priddy).
  • #877: fix Akka version mismatch (credits: Zachary Priddy).
  • #890: the Hide Empty Case button in the new UI admin section was broken.
  • #894: when clicking on a donut or a widget in the dashboards, this should take you to the search page with predefined filters, without generating errors.
  • #895: exclude responder jobs from search results.
  • #896: dashboard clicks are not correctly translated into tag filters.

Looking for Help?

Something does not work as expected? Then please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org.

TheHive 3.3-RC2, Hot out of the Oven

TheHive Project’s code Chefs, dressed in their outright haute cuisine outfit, including the traditional toque blanche, have been quite busy lately, working on dockerizing all the Cortex analyzers (more on this later in an upcoming post), and doing tedious work to prepare the replacement of Elasticsearch by a GraphDB which will help us finally release much-awaited features such as multi-tenancy, delayed for way too many months (yeah, yeah, don’t chastise them but feel free to help them). In the meantime, they found enough bandwidth to release a new major version of TheHive.

Version 3.3, currently a release candidate includes several bug fixes and many new features as outlined below. Please note that TheHive 3.3-RC2 is beta software. As all our other release candidates, you can grab it from the pre-release, beta repositories. As usual, we would truly appreciate your help making it a great stable release by testing it as thoroughly as possible and reporting back any bugs or issues you encounter so we can address them before the final release.

Check TheHive Installation guide for further details.

Wait! Where’s RC1?

TheHive 3.3-RC1 was very short-lived. Few hours after its release, and thanks to Chris (a.k.a. crackytsi on GitHub), Thomas Franco, our back-end mastermind, discovered an issue with the Debian 8 and Debian 9 packages.

New Features

  • #836: add a new exportCaseTags parameter to the MISP configuration section. If set to true, all the tags associated with a case will be exported along with it to MISP.
  • #861: add support for Java higher than 8, such as OpenJDK 11.
  • #271: bulk merge alerts into a case. Select multiple alerts at once and create a single case out of them or merge them into an existing case using its ID.
  • #824: add ability to sort alerts by reference, status, type, source…
  • #826: when previewing an alert, there are sometimes no overlap with an existing case. However, an analyst might already know, thanks to HI (Human Intelligence), that the alert should be merged into a specific case. This is now possible thanks to a new button.
  • #769: improve case template selection for case creation. If you have defined a large set of case templates, you will be able to sort/filter to find the case you want to use when creating a New Case.
New case template selector
  • #657: add observable tags auto-completion. Contributed by Tyler Chong (Thanks!).
Observable tag auto-completion

Fixed Bugs

  • #864: do not return a session cookie when making an API call.
  • #856: there was a bug where after a followed alert PATCH, if the alert has already been promoted to a case, the case is not updated. Now, if the alert has follow=true, if it gets updated, its status is set to Updated and the related case is updated too.
  • #845: assigned but unstarted tasks were not showing up in My Tasks.
  • #844: enable user account locking through the Delete API endpoint.

Stuck?

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

New Year, New Analyzers

Dear fellow incident handlers and cybercrime fighters around the world, the galaxy, the known and the unknown universe, first and foremost, all TheHive Project’s team would like to wish a wonderful new year 2019 to you and to your cherished relatives. We truly hope that eagles, pandas, kittens, babars, bears and all sorts of animals will stay out of the way. And remember that you don’t need to go bankrupt by purchasing so-called Next Gen™ magical solutions that work only when there’s a full moon and the page number of the book you are currently reading is 42 to investigate threats 😉

We would like to begin the year by introducing version 1.15.0 of Cortex analyzers, bringing the total number of analyzers to a whopping 113! And thanks to Kyle Parrish, this release improves the Mailer responder to allow you to specify a custom port number for your SMTP server and adds a new one to blacklist observables on Cisco Umbrella utilizing the Enforcement API. The Cisco Umbrella Blacklister responder will then add the tag Umbrella:blockedto the observable.

Cortex-Analyzers 1.15.0 also include fixes and enhancements for Eml_Parser, IBM X-Force, Fortiguard, and Shodan. Most of these modifications were contributed by our continuously growing user community. Thanks to all of those who help us in our mission to provide free and open source security incident response tools to the masses!

Please read the relevant sections in the Cortex installation guide to install or update your analyzers and responders in order to benefit from all this sweet & tasty honey.

New Analyzers

The following analyzers have been added:

Cyberprotect

This analyzer lets you query the Cyberprotect ThreatScore service for domains and IP addresses. No configuration is needed and it can be used out of the box.

TheHive displays the analyzer results as follows:

Have I Been Pwned

The HIBP_Query analyzer lets you check email addresses on Have I Been Pwned. You can use an optional parameter to include unverified breaches in the search results. Otherwise, it can be used without any additional configuration.

When called from TheHive, results would display as such:

PatrOwl

As it name states, The Patrowl_GetReport analyzer will let you get the current PatrOwl report for a FQDN, a domain name or an IP address. You need a running PatrOwl instance or to have access to one to use the analyzer.

If you fire it from TheHive, it would display results as follows:

SecurityTrails

This analyzer comes in two flavors in order to get Whois data and Passive DNS details using SecurityTrails. To use both flavors, you will need an account for the service to retrieve the associated API key, which you need to configure the analyzers.

SecurityTrails_Passive_DNS displays results in TheHive as follows:

The Whois variant produces reports such as:

Cisco Umbrella

In addition to Cisco Umbrella Investigate, you can now query the Umbrella Reporting API for recent DNS queries and their status for a domain name using the new Umbrella_Report analyzer.

New Shodan Flavors

In addition to Shodan_Host and Shodan_Search, which allow you to obtain Shodan information on a host and the search results for a domain name, now you can get domain resolutions (Shodan_DNSResolve), obtain scan history results for an IP address (Shodan_Host_History), get information on a domain (Shodan_InfoDomain) and the reverse DNS resolutions for an IP address (Shodan_ReverseDNS).

DomainTools

The following DomainTools flavors were added to this release:

  • DomainTools_HostingHistory: get a list of historical registrant, name servers and IP addresses for a domain.
  • DomainTools_ReverseIPWhois: get a list of IP addresses which share the same registrant information. It applies to a mail, IP, or domain.

Moreover, please note that DomainTools_WhoisLookup now handles IP addresses in addition to domains and provides parsed results. DomainTools_WhoisLookup_IP is thus not needed anymore. Instead, DomainTools_WhoisLookupUnparsed has been added to do the same as DomainTools_WhoisLookup, except that the output results are unparsed.

Synapse 1.1.0 Released

A few months ago, we released Synapse, our first “meta” alert feeder for TheHive in order to lower the burden of highly repetitive tasks on incident handlers. Thanks to a scalable and modular design, Synapse aims to make incident response easier by automating some of its tedious parts.

The first step towards this challenge was based on the integration of Microsoft Exchange with TheHive in Synapse 1.0.0. This way, user notifications pertaining to suspicious emails can be easily consumed and acted upon in TheHive.

Today, we are releasing version 1.1.0 which goes further by adding support for Microsoft Exchange O365 and the IBM QRadar SIEM.

big-picture

Exchange O365

Theoretically, Exchange O365 was supposed to be functional in Synapse 1.0.0.
However, since we did not have an O365 account we could not fully test that feature. Thankfully with the help of one of our users we managed to solve a bug and finally validate the Exchange O365 integration.

For more details about Exchange and TheHive, have a look here.

IBM QRadar SIEM

Members of TheHive’s Core Team have practical experience with QRadar and we decided to make good use of it to the benefit of our fellow analysts.

With the Community Edition of QRadar in one hand and an instance of TheHive in the other, we managed to create alerts in TheHive out of QRadar offenses. Furthermore, when a case or alert related to a QRadar offense is closed in TheHive, it also closes it in QRadar automatically.

For more details about QRadar and TheHive, have a look here. Alternatively, you may also want to consider Pierre Barlet’s qradar2thehive script.

Send your Ideas our Way

With this 1.1.0 release, the list of integrated products with TheHive goes up to three: Exchange, Exchange O365 and QRadar. However, we don’t really have a plan regarding the next candidate for integration so tweet us at @TheHive_Project and tell us what you want!

 

Cortex 101: Dissecting Observables a Hundred Ways

TheHive Project’s Code Chefs are glad to announce that, thanks to the precious contributions of the user community, Cortex has broken the one hundred analyzer mark.

Cortex-Analyzers version 1.14.0 is out and includes new analyzers, some improvements and some bug fixes.

New Analyzers

New and enhanced analyzers, described below, are:

  • Cisco Investigate by Cisco Umbrella Research @opendns
  • Proofpoint Forensics Lookup by Emmanuel Torquato
  • Proofpoint Threat Insight Forensics Analyzer by Emmanuel Torquato
  • RecordedFuture by jojoob
  • urlscan.io search by ninoseki
  • Google DNS over HTTP by 0xswitch
  • RTF files support in FileInfo by Nils Kuhnert
  • Datascan and Inetnum flavors in Onyphe analyzer by Pierre Baudry and Adrien Barchapt

Again, huge thanks for the awesome work that has been performed by all our contributors!

Investigate

Cisco Umbrella Investigate provides threat intelligence about domains and IP addresses accross the Internet. The analyzer can be used to query the Cisco Umbrella (formerly OpenDNS) API and get information about an IP or a domain name. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-investigate-short

sc-investigate-long
Cisco Investigate: short and long reports

Proofpoint Forensics Lookup

According to Proofpoint’s website, the Forensics API allows insight in detailed forensic evidences about individual threats or compaigns. The analyzer can be used to check observables against given indicators of compromise stored in the ProofPoint service.

Unfortunately, there are currently no sample report screenshots available, because TheHive’s Core Team does not have access to Proofpoint services. Also, due to the same reason, this analyzer could not be tested by us. If you have access to the service and can test the analyzer and/or provide report screenshots, please let us know.

RecordedFuture

This analyzer lets you get the latest risk data from RecordedFuture for a hash, domain or an IP address. It can be used to query the API and get information. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-recordedfuture-short

sc-recordedfuture-long
RecordedFuture: short and long reports

Urlscan.io search

Urlscan.io is a service that scans and analyzes websites. Submitted pages will be browsed like a regular user would do and every activity gets recorded. The analyzer submitted by ninoseki queries urlscan without initiating a scan which would be publicly visible on the website. Accepted datatypes for this analyzer are URL, domain, hash and IP.

The templates which display the results of the analyzer look like the following screenshots:

Screenshot from 2018-10-25 09-55-34

Screenshot from 2018-10-25 09-48-58
Urlscan.io: short and long reports

Google DNS over HTTP

This analyzer provides DNS information for an IP, a domain or a FQDN by making calls to Google DNS-over-HTTP (DoH). No API key is required.

Results are displayed in TheHive in the following manner:sc-googleDNS-short

sc-googleDNS-long
Google DNS: short and long reports

RTF files support in FileInfo

The FileInfo meta analyzer has been improved and now leverages the rtfobj tool provided in the Oletools suite by Decalage.

Results are displayed in TheHive in the following manner:

sc-fileinfo_RTF-short

sc-rtfobj-long
FileInfo with rtfobj: short and long reports

Datascan and Inetnum flavors in Onyphe analyzer

The Onyphe analyzer has been enhanced with two new flavors. Datascan provides information about known open ports on a specific IP, and Inetnum enumerates all known network information about the analyzed IP address.

An API key is required to use the analyzer and can be obtained by creating an account on the Onyphe website.

Results are displayed in TheHive in the following manner:

sc-onyphe-inetnum-short

sc-onyphe_inetnum-long
Onyphe  Inetnum: short and long reports

sc-onyphe_datascan-short

sc-onyphe_datascan-long
Onyphe Datascan: long and short reports

Bug fixes and enhancements

  • #248: Improve error msg when VT Get Report does not have an entry for
  • #323: Fix an issue with HybridAnalysis analyzer filenames handler
  • #329: Enhance PassiveTotal Passive DNS report
  • #348: Fix a typo in URLhaus’s long.html
  • #356 : Force the use of Python3 in MISP analyzer
  • #362: Fix file not found issue and empty result set in CERT.at passive DNS analyzer

Get It While Supply Lasts!

Each analyzer comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.14.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, get the latest version of  the report templates and import them into TheHive.

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

EmlParser: a New Cortex Analyzer for EML Files

The new EmlParser analyzer which we included in Cortex-Analyzers 1.12.0 leverages the eml_parser python library written by GOVCERT-LU. It parses EML email,  a MIME RFC 822 standard format, and extract all the information to help the analyst triage and investigate. EmlParser will prove very useful when analyzing observables imported from Synapse alerts.

You might notice that the analyzer’s requirements.txt installs the eml_parser library from one of our repositories. The original library dependencies contains file_magic library which brokes other analyzers that use python-magic. GOVCERT-LU is addressing this situation in their code but the installation process still considers file-magic as a mandatory library. We decided to consider it as an extra requirement.

Screen Shot 2018-07-26 at 08.19.11.png

Screen Shot 2018-07-26 at 08.19.31.png
EmlParser: short and long report samples

Get It While Supply Lasts!

To update your Cortex analyzers to 1.12.0, run the following commands:

cd path/to/Cortex-Analyzers

git pull

for I in analyzers/*/requirements.txt; do sudo -H pip2 install -r $I; done && \

for I in analyzers/*/requirements.txt; do sudo -H pip3 install -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button.

Update TheHive Report Templates

If you are using TheHive, get the latest version of  the report templates and import them into TheHive.

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!