It wasn't so long back when I blogged about our first announcement of Threat Intelligence capabilities at Report URI, and I said back then that we'd be announcing more over the coming months. Well, it's time for the next set of new features!

Threat Intelligence

My previous blog post announced two pretty big features.

First, we were looking for "Indicators of Compromise" in your CSP reports. A good example of an IoC might be that you're loading JavaScript from a site that is known to host malware, or your page is sending XHRs to a domain known to be involved with data theft. If we spot something that we know is involved in criminal or malicious activity, it will now be flagged up on the report in your account.

Second, we were looking for "Domain Generation Algorithm" domains. A DGA is precisely what it sounds like and can be used by malware to produce a sequence of random domains to communicate with in the future. Use of domains on your site that look like they were generated by a DGA to load assets from or send data to can be a cause for concern.

If you want more details on our IoC and DGA features, please do check out the blog post announcing them. For now, onto the new stuff!

Domain Reputation

As mentioned in the IoC and DGA blog post, we are now using various external sources of Threat Intelligence data to enrich our own telemetry. One feed of data that we've been testing for a while is a 'Domain Reputation' score, an external API that we can query with a domain to see its reputation. Domains with a long standing, good reputation and no ties to known criminal or malicious activity will have a high score, while domains with known ties to criminal or malicious activity, or other unsavoury behaviour will have a low score. It's very simple in essence, but our problem was our scale! It's now very common for us to process well in excess of 500,000,000 CSP reports per day and as you can imagine, we couldn't find many API providers willing to support that kind of volume!

I'll delve into the deep technical details in a future blog post, but for now, we've managed to find a stable implementation that significantly reduces our load on the external API providers, whilst still allowing us to process reports at the volume and efficiency that we need.

With this all setup and tested, we've finally pushed the feature live and you can now expect to see a 'Low Reputation' badge alongside blocked domains that have a low reputation!

You can also set your own custom value for the Domain Reputation if you just want to filter out reports for domains that are equal to or below a particular score. The range is 0 - 100 inclusive and the filter applies a <= to the value you specify. Any query with a Domain Reputation value specified will exclude domains for which we have no reputation data.

Domain Registered Recently

One thing that's useful to know when analysing CSP reports is when the blocked domain was registered. We've consistently observed throughout various attacks in the past that criminal actors will be using new domains for their attack. Manually checking when the domain was registered is something I've done many times myself both for my own reports and when customers reach out with something that they're suspicious about but can't quite figure out. We now check the registration date of blocked domains and if a domain was registered in the last 90 days, we'll add a prominent 'Recently Registered' badge to the report. We've also added a "New Domain Filter" so that you can see only reports where the blocked domain is a new domain that was registered recently.

I had some real examples of this in our production data so depending on when you're reading this, you can manually verify that those domains in the screenshot were registered in the last 90 days!

Implementing this feature also came with the same constraints as the Domain Reputation above. We place a high demand on the external API providers for this data given the volume of domains we encounter, and even for the data we cache locally, we can hit peaks of ~10,000 reports per second being processed so it results in a lot of overhead.

It's live, and there's more coming

Both of these features are now live and available for all customers with a suitable plan that includes our Threat Intelligence features. We have more announcements planned for the future to bring you more Threat Intelligence capabilities. If you'd like more details about our threat intel capabilities, we have a dedicated Product Page that should help.