As Report URI has continued to grow, we're constantly hearing about new things that our users want. Alongside maintaining the site, fixing bugs and constantly scaling, we're always listening to feedback to guide us on feature development too. Based on what we've heard, we got to work on the next set of features and we're now ready to share them with you!

Report URI

Regular readers will know about Report URI, the product and company I started work on back in 2014! It gathers telemetry from browsers on various configurable security mechanisms, like Content Security Policy, and can help you quickly become aware of problems on your site like hostile JavaScript. Whilst we started out as a simple service to gather and observe this telemetry, our capabilities have evolved over the years and they continue to do so. Now it's time for the latest round of new features and hints into where we're going in the future.

The CSP Wizard

As a CSP reporting service, our goal was simple, to allow people to collect reports! The problem we quickly found though was that it was quite difficult for website operators to get to a point where they had a CSP they could deploy and actually, they needed help creating the CSP first, before they could send reports. This was what drove the creation of the CSP Wizard back in 2018.

Since its inception, the CSP Wizard remains one of the most used features outside of our core service offering of gathering reports and has helped countless websites quickly and easily get most of the groundwork done for creating a viable CSP. As use of the Wizard has grown, and as it is used by larger and more complex organisations, a common request has been for more flexibility in how the Wizard can be used, which is our first new feature!

You've always been able to have multiple CSPs in the 'My Policies' section of your account, but you could only ever work on one of them at a time in the Wizard. Our idea was that you would do the bulk of the work for a policy on a particular site and once ready, you could finalise the policy for deployment and move onto the next site. What we've found over time is that organisations that operate a large number of sites, or a few sites that are very complex, may sometimes want to switch and change between the policy they're currently working on. Our Teams feature does allow this, because each individual team can work on their own CSP for their own site, but the team could still only work on a single policy at once. No more!

When switching between the policy that is currently being used with the Wizard, no progress will be lost in the Wizard. Previously, when switching between the policy that is active with the Wizard, our view was that work on this policy was 'complete' and the Wizard items would be cleared, leaving you with your final CSP. It turns out this wasn't the case and required a little bit of re-architecting on our side for users or teams to now be able to dynamically switch between policies being used in the Wizard at any time you like!

The CSP Builder

This tool has been present on Report URI since day one and allows you to create a Content Security Policy using a GUI-based tool. It's been really popular since launch in 2015 and has received various updates ([1][2][3]) since. The most commonly requested feature for the CSP Builder was a tighter integration between the builder and the policies users have in the My Policies section of their account, which we're now happy to announce!

The CSPs in the My Policies section of your account can now be manually edited in the CSP Builder with a single click of the new 'Edit' button.

Once opened in the CSP Builder you have all of the powerful editing features of the CSP Builder available and can make any manual changes that you might require, or even build an entire policy from scratch!

What's even better is that you can build most of a policy using the CSP Wizard and then just tweak it with those final changes using the CSP Builder if you need to. This will make the process of building a CSP even easier and you can now go from a blank policy, all the way to a fully functioning CSP in our UI without the need for any other tools or processes!

Domain Generation Algorithms

Often used by malware and bad actors, a DGA does exactly what it sounds like; it's an algorithm used to generate domains! You can read more on the DGA Wikipedia page, but this paragraph gives a great overview:

The technique was popularized by the family of worms Conficker.a and .b which, at first generated 250 domain names per day. Starting with Conficker.C, the malware would generate 50,000 domain names every day of which it would attempt to contact 500, giving an infected machine a 1% possibility of being updated every day if the malware controllers registered only one domain per day. To prevent infected computers from updating their malware, law enforcement would have needed to pre-register 50,000 new domain names every day. From the point of view of botnet owner, they only have to register one or a few domains out of the several domains that each bot would query every day.

The DGA follows a particular pattern when it comes to the format of the domain it generates, but various families of malware and other similar threats can create a wide variety of formats, making it difficult to identify them.

I first started digging into DGAs when looking at our own internal data feeds and I started to notice frequent and odd domains. One thing we track as a service is the very first time we've seen an asset blocked from a particular domain. Whilst we do process around 700,000,000+ CSP reports per day, I've talked about our use of probabilistic data structures like Bloom Filters and how we can do this kind of analysis with relative ease. If I were to give you a sample of the domains that caught my eye, you could probably guess why they did.

'inyo4y.com',
'svn0czn.com',
'umnb7r9.com',
'aayp1.com',
'a6rm7n.com',
'6uyqy3.com',
'0137mw.com',

To you and I, these domains "just don't look right". If I spotted any of those domains loading assets on my site, I'd be taking an immediate look at what was going on! This is exactly the opportunity we want to give to our customers, but without the requirement to observe all of your CSP reports to look for these suspicious domains and identify them manually. To do this, we're now using a Markov Chain to analyse the registrable domain of inbound CSP reports for either blocked JavaScript (script-src, script-src-attr, script-src-elem) or blocked outbound communications (form-action, connect-src), and look for anomalies. We've built the chain from a set of ~7,000,000 known good domains and are now using it to identify DGAs, with the small selection above being some that we've identified in production data.

I plan to write a technical blog post on how we've built and used a Markov Chain to implement this new feature, but for now I want to announce that it's in open beta for all users, paid and free. At some point in the future this, and other Threat Intelligence based features, will likely be restricted to paid-only plans, but for now, whilst we continue to improve and work on them, they're available for everyone to use.

To use it, we currently have a hidden query string parameter on the Reports page. It's been there for a while as I've been testing it with various enterprise customers and we're now ready to share it with everyone. Once we're happy that we've made enough progress it will be added to the UI, but for now, simply add the dga=on query string parameter either on its own, or along with other query filters, to turn on the DGA filter.

https://report-uri.com/account/reports/csp/?dga=on
https://report-uri.com/account/reports/csp/?calendar=Jul%202022&unit=months&dga=on

Indicator Of Compromise

An IOC can be quite literally anything depending on the context you're operating in. For our customers, people operating websites, an IOC might be something like loading JavaScript from a domain that is known to host malware. This would be an Indicator Of Compromise because only a compromised website would load JS from a known malware domain! Maybe an attacker has found an XSS bug and got some JS into your site that shouldn't be there, or a dependency you use has been compromised, but we can say for sure that if you're loading JS from a known malware domain, it's not good.

As a service processing all of your CSP reports, we're in a great position to be able to look out for IOCs in your reports. We already have powerful features like Script Watch, which audits your JS dependencies on an ongoing basis, and Data Watch, which audits the locations your site is sending data to on an ongoing basis, but we wanted to take that a little further. Right now these features will only tell you about new scripts or new locations you're sending data to, which is great because you can immediately become aware of them and check they're expected, but we wanted to start going further.

The difficulty came in taking the step between saying "hey, you have a new JS dependency" and saying "hey, you have a new JS dependency and it's super bad!!!". To begin this process, we've now started ingesting various sources of Threat Intelligence data that we can use to enrich our analysis of inbound reports. Just like the DGA analysis above, this feature has been present for a short while now and I've been experimenting with it alongside some of our enterprise customers. It's present on the CSP Reports page and activated in a similar fashion to the DGA feature above, with an undocumented query string parameter. 

https://report-uri.com/account/reports/csp/?ioc=on
https://report-uri.com/account/reports/csp/?calendar=Jul%202022&unit=months&ioc=on

You should of course always hope that any query with the ioc=on parameter returns no reports, but if it does, it could be something worth looking at. As we continue to improve this capability, the IOC and DGA markers will become visible in other features like Script Watch and Data Watch, along with configurable alerts that can be sent immediately upon us detecting either one of these issues.

Future Improvements

The IOC and DGA features above are just the start of what we're planning and we have a lot more coming. We're currently trialling other sources of threat intelligence data to enrich our own analysis and once we've made a little more progress, they'll be announced in open beta too. In addition we're also looking to include WHOIS data to look at the age of domain registrations because brand new domains might require more immediate attention and identifying associated domains to see if we can link a domain to other IOC data. This is just the start of some really big ideas we have and I'm excited to continue developing our capabilities in this space!