Cortex Passes the 20 Analyzers Mark

Thanks to the invaluable contributions of our growing and thriving user community, Cortex has now 6 more analyzers, bringing the total to 21. The new analyzers, released under our usual AGPL v3 license, are:

  • CIRCLPassiveDNS
  • CIRCLPassiveSSL
  • GoogleSafebrowsing
  • Nessus
  • Virusshare
  • Yara

All but one have been submitted by Nils Kuhnert of CERT-Bund. The Nessus analyzer has been contributed by our long-time friend Guillaume Rousse.

Cortexutils 1.1.0

While reviewing the submissions, we realized that a new version of the Cortexutils library was needed in order to support both Python 2 and 3. Hence we released version 1.1.0. You can grab it through PIP. To update your existing installation, please run the following command:

 sudo pip install cortexutils --upgrade

Note that Cortexutils 1.1.0 is required to benefit from these analyzers and future ones. If you are performing a fresh Cortex installation, follow the guide.

Installation

To install the new analyzers, grab the Cortex-Analyzers repository and unpack its content (or git pull the master  branch) in your existing /path/to/cortex-analyzers. Then follow the Cortex analyzers guide.

New Short and Long Report Templates for TheHive

Short and long reports for TheHive were also created to parse and display the results produced by the new analyzers. We also bundled in the new package a URL analysis template for Joe Sandbox which was missing and improved some of the older short templates in order to follow a taxonomy.

To import the new report templates in your instance of TheHive:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

CIRCLPassiveDNS

The CIRCLPassiveDNS analyzer lets you check the CIRCL’s Passive DNS service for a given domain. It takes domains and URLs as input. Access to the service is allowed to trusted partners in Luxembourg and abroad. If you think you qualify, please contact the good CIRCL folks. To make it work, you’ll need the pypdns Python library.

In order to take advantage of CIRCLPassiveDNS, you need to add the following section to the Cortex configuration file (application.conf):

CIRCLPassiveDNS {
     user=""
     password=""
}

When called from TheHive, the following output is produced:

sc-short-CIRCLPassiveDNS.png

sc-long-CIRCLPassiveDNS.png
TheHive: CIRCLPassiveDNS Analyzer – Short and Long Report Samples

CIRCLPassiveSSL

The CIRCLPassiveSSL analyzer lets you check CIRCL’s Passive SSL service for a given IP address or certificate hash. Access to the service is restricted to partners and security researchers worldwide. If you think you qualify, please contact the good CIRCL folks. This analyzer needs the pypssl Python library to work properly.

To use it, please add the following section to the Cortex configuration file (application.conf):

CIRCLPassiveSSL {
     user=""
     password=""
}

When called from TheHive, the following output is produced:

sc-short-CIRCLPassiveSSL.png

sc-long-CIRCLPassiveSSL.png
TheHive: CIRCLPassiveSSL Analyzer – Short and Long Report Samples

GoogleSafebrowsing

This analyzer lets you check URLs and domains against Google Safe Browsing. You need an API key to use it.

To leverage GoogleSafebrowsing, add the following section to Cortex’ configuration file:

GoogleSafebrowsing {
  key="" 
}

When you run the analyzer fromTheHive, you should see output similar to the samples below:

sc-short-safebrowsing.png

sc-long-safebrowsing.png
TheHive: GoogleSafebrowsing Analyzer — Short and Long Report Samples

Nessus

The Nessus analyzer lets you leverage Tenable’s Nessus Vulnerability Scanner to scan an IP address or a FQDN. Of course, you must not scan assets that do not belong to you, unless you really know what you are doing. That’s why safeguards were built in the analyzer’s configuration:

Nessus {
   url="<https://url.to.scanner>"
   login=""
   password=""
   policy=""
   ca_bundle=""
   allowed_networks=[ 'x.y.z.t/8', 'a.b.c.d/24', ... ]

The nessrest Python library is needed to make REST API calls to Nessus. Analysts would use the analyzer to assess the vulnerabilities of potentially compromised machines or new, unknown assets that have been plugged into one of their constituency’s networks. Of course, penetration testers conducting large-scale reconnaissance can also benefit from this analyzer.

sc-short-nessus.png

sc-long-nessus.png
TheHive: Nessus Analyzer — Short and Long Report Samples

Virusshare

The Virusshare analyzer lets you verify whether a file or hash is available on VirusShare.com. It requires the progressbar2 Python library besides requests (which should be already installed if you have an existing Cortex installation). As stated by Nils:

This analyzer enables searching for md5 hashes in Virusshare.com hash list. It does not download samples for you nor links directly to the sample – the author of virusshare prohibits the automatic download/site scraping and I respect that. It provides a button to start the virusshare search, though, but you need an account for that. You can request an invitation to the platform through contacting the admin via mail, directly.

To use it, add the following section to your Cortex application.conf:

Virusshare {
   path="/path/to/download/directory"
}

Quoting Nils again, in order to download the newest available hash lists from virusshare.com, you can run the download_hashes.py script that comes with the analyzer.

./download_hashes.py /path/to/your/download/directory

Upon running the analyzer from TheHive, the report will contain a link to the corresponding Virusshare page if a match is found as shown below.

sc-long-virusshare.png
TheHive: Virusshare Analyzer — Long Report Sample

Yara

Last but not least, the Yara analyzer can check files against YARA rules using yara-python. To use it, add the following to your Cortex configuration file:

Yara {
    rules=["/path/a", "/path/b", "/path/my/rules.yar"]
}

You can specify path to directories and files. If you supply a directory, the analyzer expects to find an index.yar or index.yas file. The index file can include other rule files. An example can be found in the Yara-rules repository.

sc-short-yara.png

sc-long-yara.png
TheHive: Yara Analyzer — Short and Long Report Samples

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

TheHive4py API Client 1.1.1 Released

The French chefs at TheHive Project’s code kitchen have released version 1.1.1 of TheHive4py. The API client for our Security Incident Response Platform has been updated to comply with Buckfast 2 (TheHive 2.10.2).

In April 20, 2017, Buckfast 2 was released to plug a number of vulnerabilities identified by our friends at Randorisec. Among other changes, Buckfast 2 implements a protection against CSRF attacks. As a result, API calls made by TheHive4py have been modified in order to support that protection. In essence, TheHive4py 1.1.1 submits authentication credentials for each call instead of a per-session authentication.

To update your existing package, please use PIP. Shall you encounter any difficulties, do not hesitate to ask on our user forum or contact us at support@thehive-project.org. You can also join our Gitter channel and have a chat with us.

A new, major version of TheHive4py (1.2.0) will be released in the upcoming days to be compatible with Mellifera, our next major release of TheHive, which will feature a brand new alerting framework.

Cortex 1.0.2 Released

As we have mentioned in the previous post, Randorisec reported to us that Cortex 1.0.0 is affected by the same Reflected XSS vulnerability as Buckfast 0 and 1 (respectively TheHive 2.10.0 and 2.10.1). This is due to the fact that both use the same angular-ui-notification service. The issue also affects Cortex 1.0.1.

randori_logo_original

We are happy to announce the immediate availability of Cortex 1.0.2 which fixes the vulnerability referenced as AP2 in Randorisec’s report. Moreover, this new release fixes 3 bugs:

  • Issue #11: jobs list API doesn’t take into account the limit parameter.
  • Issue #13: global section in configuration file is ignored.
  • Issue #16: redirect to jobs list when a job is not found.

To update your current Cortex installation, follow the instructions of the Installation Guide. Before doing so, you may want to save the job reports that were not executed via TheHive. Cortex 1 has no persistence and restarting the service will wipe out any existing reports.

Should you encounter any difficulties, please do not hesitate to read the FAQ, ask questions on the user forum or on Gitter, or contact us directly at support@thehive-project.org.

We would like to thank again Davy Douhine, Randorisec’s CEO, ArtsSEC, Frédéric Cikala, Nicolas Mattiocco, Florent Montel and Mohamed Mrabah for devoting time and efforts for making our products more secure.

Correction: April 21, 2017
An earlier version of this article mentioned Maximilano Soler among the professionals who joined Randorisec to perform a pentest on our products. At his request, we have removed his name and replaced it by ArtsSEC.

Buckfast 2 Release and Randorisec’s Pentest Report

Our good friends at Randorisec, joined by other pentesting professionals (see below), performed a fully fledged pentest of Buckfast 0 (TheHive 2.10.0) and Cortex 1.0.0 during 4 man-days spanning several weeks, starting from February 9, 2017 and ending on March 21, 2017.

randori_logo_original.png

They have identified several security issues detailed in their report which they privately shared with us prior to publication. As a result, we are happy to announce the immediate availability of Buckfast 2 (TheHive 2.10.2) which fixes the following vulnerabilities:

  • Stored XSS (ref. AP1 in the report) and Reflected XSS (AP2): malicious JavaScript code can be injected. It will be then executed on the victim’s browser. See issue #159 for more details.
  • Vertical privilege escalation (AP3): an authenticated simple user can have access to some admin menus. See issue #160 and issue #161.
  • CSRF (AP8): As no anti-CSRF tokens are used, TheHive is vulnerable to CSRF attacks. See issue #158.

Cortex 1.0.0 and 1.0.1 are also affected by AP2. A new Cortex version will be released very shortly to fix it.

Additionally, Buckfast 2 fixes the following bugs:

  • Issue #152: pagination does not work with 100 results per page.
  • Issue #169: error when importing some MISP events due to their unexpected JSON format. This has also been fixed in MISP v2.4.71.

We have also added the following features:

  • Issue #157: add persistence for task viewing options.
  • Issue #174: run all analyzers on multiple observables from the observables view.

Randorisec identified 4 more security issues rated low which aren’t fixed by this release:

  • Concurrent sessions allowed (AP4): we do not deem this a security vulnerability and hence we won’t fix it unless our user community request a patch.
  • No account lockout policy (AP5): if you use the local authentication system, it can be brute-forced. We are going to fix this in Mellifera 1 (TheHive 2.11.1) due at the end of May 2017. In the meantime, you can use LDAP, Active Directory or both and configure a password policy on those systems.
  • No password policy (AP6): as no password policy is enforced when using the local database for storing user credentials, users can set weak passwords (e.g.: containing only one character). We are going to fix this in Mellifera 1 (TheHive 2.11.1) due at the end of May 2017. In the meantime, you can use LDAP, Active Directory or both and configure a password policy on those systems.
  • Information leakage (AP7): information such as installed software versions (TheHive, ElasticSearch) is publicly available. TheHive should be not be publicly accessible and access should be filtered by a firewall or a similar device for authorized IP addresses only.

If you are running Buckfast 1 or a previous version, please follow the updating instructions to update to Buckfast 2. It is actually an extremely simple operation. If you are doing a fresh installation, we have you covered as well.

Should you encounter any difficulties, please do not hesitate to read the FAQ, ask questions on the user forum or on Gitter, or contact us directly at support@thehive-project.org.

Please note that Randorisec and the pentesting professionals that joined it for this pentest have no contract with TheHive Project and did not receive any compensation of any sort to perform this work. They worked on their free time as a way to contribute to the security of Free, Open Source Software projects. We’d like to wholeheartedly thank Davy Douhine, Randorisec’s CEO, ArtsSEC, Frédéric Cikala, Nicolas Mattiocco, Florent Montel and Mohamed Mrabah for their invaluable contribution.

Correction: April 21, 2017
An earlier version of this article mentioned Maximilano Soler among the professionals who joined Randorisec to perform a pentest on our products. At his request, we have removed his name and replaced it by ArtsSEC.

Joe Sandbox, MISP Search and Report Improvements

We are thrilled to announce that Cortex has two new analyzers: Joe Sandbox and MISP Search. Moreover, we have produced new analyzer report templates for TheHive and improved existing ones.

Joe Sandbox

List JSB Cortex.png
Cortex: New Joe Sandbox Analyzer

Joe Sandbox, by Joe Security LLC, is a very powerful malware analysis platform that has been around for many years and comes in two flavors: cloud and on-premises. The Joe Sandbox Cortex analyzer has been tested using an on-prem Joe Sandbox Ultimate version and can process URLs and files. The analyzer can process files with or without Internet access.

To use the analyzer, you must provide the API key of your Joe Sandbox instance. You must log in to Joe Sandbox, click on your account name, then on Settings and on the API Key tab.

report JSB Cortex.png
Cortex: Joe Sandbox Output Example

We have produced a report template for the Joe Sandbox analyzer output resulting from file analysis. The URL analysis report template is not yet available but it should be in a few days.

JSB TH short report

JSB_THEHIVE.png
TheHive: Joe Sandbox Analyzer – Short and Long Report Samples

MISP Search

Screen Shot 2017-03-31 at 15.25.07.png
Cortex: New MISP Search Analyzer

It is no longer necessary to present MISP, the de facto standard of threat sharing. The new MISP Search analyzer will let you search events containing the observable you provide as an input. It applies to a lot of observable types as you can see in the screenshot above.

To use it, you’ll need to supply the API key available in the MISP UI interface.

result_MISP_Cortex.jpg
Cortex: MISP Analyzer Output Sample

Nils Kuhnert created an alternate MISP Search analyzer which has the ability to query multiple MISP instances. We are currently reviewing his submission along with several other analyzers he contributed before improving the newly released MISP Search analyzer.

PassiveTotal Report Templates

PT Whois short report.pngPT UniqueRes TH short report.png

While we published the PassiveTotal analyzer weeks ago, TheHive didn’t have report templates for it at the time. We have now new, shiny short and long report templates for most of the services provided by the PT analyzer.

PT PDNS long report.png
TheHive: PassiveTotal PassiveDNS – Long Report Sample

DomainTools Whois Lookup Report Template

DT Whois TH short report.png

The short report templates of the DomainTools Whois Lookup analyzer has been improved. We now use a taxonomy to provide more context and differentiate between the DomainTools and PassiveTotal Whois results.

VirusTotal Get Report and VirusTotal Scan Report Templates

VT TH short report.png
VT and JSB TH short report.png

The short report templates for both services have also been improved to use a taxonomy to provide additional context and distinguish their results from the PassiveTotal Malware service.

Get the new analyzers

To install the new analyzers, grab the Cortex-Analyzers repository and unpack its content (or git pull the master  branch) in your existing /path/to/cortex-analyzers.

The Joe Sandbox analyzer does not need any additional Python library if you have already installed Cortex and the analyzers following the guide we provide.  To use it, edit your Cortex configuration file (/path/to/cortex/application.conf) and add the following lines in the analyzer section:

 JoeSandbox {
     apikey="..."
     url="..."
 }

By default, Joe Sandbox will time out the analysis after 30*60 seconds (30 minutes). Additionally, the analyzer will wait for the Joe Sandbox server to respond within 30 seconds. If no response is received within this period, it will time out. If you want to override these values, you’ll need to add the following lines in the analyzer section:

JoeSandbox {
     apikey="..."
     url="..."
     analysistimeout=<NEW VALUE> # optional
     networktimeout=<NEW VALUE> # optional
}

The MISP Search analyzer requires pymisp. Use the following command line to install the required library:

sudo pip install pymisp

Then edit your Cortex configuration file (/path/to/cortex/application.conf) and add the following lines in the analyzer section:

MISP {
     api_key="..."
     url="..."
}

Please note that you must restart Cortex to take the changes into account. The current version has no persistence so you’ll lose all your existing jobs.

You can find the full installation requirements for Cortex and Cortex-Analyzers on the Cortex wiki pages.

Use the New Report Templates

To import the new report templates in your instance of TheHive:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help you!

Jigsaw Falling Into Place

While we released TheHive as a free, open source product in November 2016, it must not be chalked off quickly as a young, immature solution.

v1.0.0 was put into production in our environment in October 2014. Yes, October 2014. And we’ve been using it every day and refining it since then. Once we deemed it good enough, we decided to share it with the community under an AGPL license to help incident responders in their mission.

Make no mistake. TheHive is a field-tested, mature Security Incident Response Platform (SIRP) built by people who are passionate about Digital Forensics and Incident Response.

A few months after the first public release (v 2.9.0), we adopted bee-related codenames for new major versions and published Buckfast (v 2.10.0).Cortex, the analysis engine that allowed TheHive to analyze and assess observables at scale was shipped as a separate product.  Buckfast can interface with one or several Cortex instances depending on your performance and OPSEC needs. For example, you may want to install a separate Cortex on your investigation, air-gapped network to interact with your sandbox as you don’t want to be firing those malicious samples on your corporate network.

Buckfast can also create cases out of MISP events. You can configure it to import them from a single or many MISP instances. And to prepare for the next major version, Mellifera, due in early May 2017, we have released TheHive4py, a Python API client for TheHive.

TheHive4py will be improved to fully support Mellifera’s alerting framework. To put it simply, Mellifera will not only let you preview MISP events and import them but also receive SIEM alerts, email incident reports and different other types of alerts depending on your environment thanks to TheHive4py. And if an analyst discards an alert by mistake in Mellifera’s notification area, they can go back to a ‘trash bin’ and fix their error.  Mellifera will also allow you to export cases as MISP events to share IOCs with other teams.

TLP-WHITE-Jigsaw_Falling_Into_Place-2017-03.001.png
Jigsaw Falling Into Place

Now lets’ get back to TheHive’s perfect companion: Cortex. As of this writing, Cortex features 13 analyzers. These analyzers can perform one type of analysis (such as Abuse Finder) or several (such as DomainTools which can do 6). In the very near future, we plan to add at least 10 more analyzers which are shown in the boxes with dotted borders in the picture above. All upcoming analyzers are contributed by our user community whom we wholeheartedly thank. One of the analyzers will allow you to check observables from TheHive against a MISP instance to search for events that may contain them.

We have also begun work on a Python API client for Cortex dubbed… Cortex4py (how creative wink wink). This will allow people who are not using TheHive to summon the power of Cortex from their SIRP, scripts or any other DFIR tool that can import or interact with Python code.

So in the few months since our project was born to the Internet, we have released a solid collaborative SIRP, a simple yet powerful analysis engine to analyze observables and aid teams in their investigations as well as a Python API client for our SIRP. We also have rather ambitious plans to make them even much more useful.

Oh and one more thing! We have released another piece of software around the same time as the first version of TheHive and on which we haven’t said much so far: Hippocampe. Hippocampe can regularly download feeds and exposes a REST API to let you query them from Cortex (or from other tools). You submit an observable and it’ll tell you if it appears in one or several feeds along with a score. The score takes into account the trust you put in the feed sources (which can be adjusted over time) and the number of sources which contain the observable. We’ll cover Hippocampe in more details in an upcoming post.

Before you run away from us
Before you’re lost between the notes
The beat goes round and round
Jigsaw is falling into place
So there is nothing to explain

Buckfast 1 and Cortex All-in-one Package

When you use TheHive, running an analyzer on an observable through Cortex will generate a long report and, in most cases, a short report as well.

Let’s see how this works in practice through an example. Assume we are trying to assess whether the 636a4249104acaaf6d76d7409dc3cb2d MD5 hash is malicious or not:

Screen Shot 2017-03-26 at 22.21.10.png

We start by clicking on it, which will open a new tab:Screen Shot 2017-03-26 at 22.21.22.pngThis TLP:WHITE hash was imported from a MISP event published by our good friends at CIRCL.lu sometimes ago. As you can see from the screenshot above, no analyzer was executed on it.  Let’s check if it is known to VirusTotal (VT). To do so, we just need to click on the fire icon located at the right side of the VirusTotal_GetReport_2_0 row.

Screen Shot 2017-03-26 at 22.21.53

A blink of an eye later, the job has finished successfully as we can tell from the green checkmark. Clicking on the date will let us see the long report, presented according to a report template that we freely provide with most analyzers to the exception of PassiveTotal (but in a few days, PT will also get its own nifty templates).Screen Shot 2017-03-26 at 22.22.11.png

Since we are checking whether VT knows a hash or not, it will give us the results if any corresponding to the last time the associated file was scanned on the service. In our case, this dates back to Dec 2, 2016.

When the analyzer was executed, it also produced a short report which TheHive displays below the observable:Screen Shot 2017-03-26 at 22.21.45.pngShort reports come in 4 colors. Red means danger (what else?). Orange means suspicious. Green means innocuous. And blue is informational. OK but what does this have to do with the title of this post?

A few days ago, while working on a new set of analyzers, Nils Kuhnert reported an issue in Buckfast 1 (2.10.1) pertaining to short reports on observables. When he ran some analyzers that should have produced short reports, he didn’t get any. When he reverted to Buckfast 0 (2.10.0), it worked. We tracked down the problem and found that our build process was the culprit. The all-in-one  binary package which was supposed to contain Buckfast 1 and Cortex was in fact a 2.10.0 TheHive snapshot that had a regression. We have uploaded a fresh all-in-one binary package with Buckfast 1 instead of the development snapshot.

If you have grabbed the binary all-in-one package, please download it again and update your instance. If you are using a docker version or built Buckfast 1 from sources, you are fine. To make sure you are running the right version, click on your username once you are logged in then on About TheHive. You should see the following information:Screen Shot 2017-03-26 at 22.56.09.png

We are going to review our release process from the ground up to ascertain such errors never occur again. We expect it to be ready for Mellifera, our next major release of TheHive. Please note that starting from that release, we will no longer provide all-in-one binary and docker packages. Instead, we’ll have separate packages for TheHive and Cortex. TheHive4py and the upcoming Cortex4py will be made available through PIP.

Cortex API Documentation

TheHive can interact with one or several Cortex instances to analyze observables and aid you in your investigation in the best possible way while taking into account your OPSEC needs. But you don’t need TheHive to unleash the power of Cortex.

Cortex can be used as a standalone product. You can run analyzers on observables you supply using its simple yet useful Web UI. And if you are not using TheHive, you can bridge your SIRP (Security Incident Response Platform) or any other tool with Cortex thanks to its REST API. To do so, please read the API documentation that we have published. And in a very few days, things will get easier as we will release Cortex4py, a Python API client for Cortex.

Let us know if you have any questions or problems on our user forum, on Gitter, or by sending us an email at support@thehive-project.org. We will be more than happy to help you!

TheHive4py 1.1.0 Released

The French chefs at TheHive code kitchen have just updated TheHive4py, our Python API client for TheHive. Version 1.1.0 lets you add an attachment to a task log but also observables, including files, to a case.

Moreover, TheHive4py 1.1.0 supports searching and you can look up cases by using string keywords.

To update your existing package, please use PIP. Shall you encounter any difficulties, do not hesitate to ask on our user forum or contact us at support@thehive-project.org. You can also join our Gitter channel and have a chat with us.

We will continue improving the client to cover most, if not all API calls. TheHive4py will also undergo significant changes in the upcoming weeks to be compatible with Mellifera, our next major release of TheHive, which will feature a brand new alerting framework.

 

Buckfast 1 Released

The French chefs of TheHive Project code kitchen have been pretty busy as of late. After updating most Cortex analyzers and adding PassiveTotal, we have released TheHive4py 1.0.0 last week, shortly followed by a minor update to this new Python API client for you favorite Security Incident Response Platform. Last week, we have also published Cortex 1.0.1.  And here comes Buckfast 1 (TheHive 2.10.1).

This new release fixes a regression introduced by the previous one where the Flow, our Twitter-like live stream feature, won’t open in a new window. Buckfast 1 also fixes an issue related to newly added observable datatypes that non-admins can’t use unless TheHive service is restarted.

Moreover, the OTXQuery analyzer will now display an error in the long report in case the job fails. Pagination buttons were also introduced at the top of a case’s task list to make it easier for analysts to sift through tasks.  We have also removed the Run all analyzers button next to each observable in the observables tab. We deemed this change necessary to avoid cases where analysts would hit it without really thinking about what they are doing. For certain datatypes this could run all of the current 13 analyzers, some of which need a subscription and may eat the team’s query quota pretty fast.

Buckfast 1 can be run from any URL path and not just from the root directory. It will also close all open tasks in merged cases as they are absorbed in the new case resulting from the merge operation. We have fixed other things and added some additional minor features. Please consult the full changelog.

If you are running Buckfast 0 or a previous version, please follow the updating instructions. It is actually extremely simple to update TheHive. If you are doing a fresh installation, we have you covered as well.

Should you encounter any difficulties, please do not hesitate to read the FAQ, ask questions on the user forum or contact us directly at support@thehive-project.org.

Bon appétit !