Defender ATP Advanced hunting with TI from URLhaus

Posted onAuthorAlex Verboon11 Comments

Hello everyone, in today’s article we are going to take look at how we can use Threat Intelligence (TI) data from URLhaus with Microsoft Defender ATP advanced hunting.

URLhaus

URLhaus is a project from abuse.ch with the goal of sharing malicious URLs that are being used for malware distribution. https://urlhaus.abuse.ch/ The project provides several ways to find and retrieve information about malware URLs.

You can browse the URL database interactively through https://urlhaus.abuse.ch/browse/

You can also download the database in various formats such as a CSV file that contains the following information:

  • ID
  • Dateadded (UTC)
  • URL
  • URL status
  • Threat
  • Associated tags
  • Link to URLhaus entry
  • Reporter

Or just as plain text file with URLs only. Here there are several downloads available:

  • Plain text URL list with all malware URLs known to URLhaus
  • Plain text URL list – most recent additions from the past 30 days
  • Plain text URL list – online – containing only online (active) malware URLs

Advanced Hunting and the externaldata operator

Advanced hunting in Microsoft Defender ATP is based on the Kusto query language. The externaldata operator allows us to read data from an external storage such as a file hosted as a feed or stored as a blob in Azure blog storage.

Let me show two examples using two data sources from URLhaus. First we are going to retrieve the URLhaus detailed database information containing online URLs.

https://urlhaus.abuse.ch/downloads/csv_online/

In the following example, we use the online URLs only list.

https://urlhaus.abuse.ch/downloads/text_online/

And how about looking at malicious all URLs from the URLhaus database whose domain name resolve to an IP address associated with a particular geo IP location (country code)? (To see data related to your country, simply change the country code i.e. NL, US etc. )

Now that we are at it anyway, let pull the list of domain names that are associated with malware URLs.

Advanced hunting finding matches based on TI from URLhaus

Now that we know how to retrieve external data from URLhaus using advanced hunting, let us use this data for with our hunting queries in Microsoft Defender ATP or Microsoft Threat Protection.

Below is query where we can identify any DeviceNetWorkEvents associated with malware URLs.

Let’s change the query a bit and let us look at DeviceEvents

Now instead of just looking at teach Defender table separately, let us search across various tables.

Finally, if you’re interested at looking up data from URLhaus through PowerShell, take a look at my PowerShell module PSURLhaus

I would like to credit @Pawp81 who’s query included in the AdvancedHuntingCheatCheet inspired me to look further into the use of externaldata in advanced hunting queries.

You can find all the KQL queries mentioned in this blog post here; https://github.com/alexverboon/MDATP/tree/master/AdvancedHunting/URLHaus

Well, that is it for today, hope you enjoyed this article.

Alex

Related posts:

  1. Generating Advanced hunting queries with PowerShell
  2. How to generate a monthly Defender ATP Threat and Vulnerability Report
  3. Microsoft Defender ATP Advanced Hunting – Who’s logging on with local admin rights?
  4. Advance your Microsoft Defender ATP hunting skills using the Atomic execution framework
  5. Hunting for Local Group Membership changes

11 Replies to “Defender ATP Advanced hunting with TI from URLhaus”

  1. Hello Alex, great post with a lot of new opportunities! When looking at the screenshot with the country code example the URL appears to be missing in the output. I see similar behavior in our environment also when exporting the results the URL is not in the output. Do you have any idea why the URL is not visible?

  2. Hi,

    How will it work in an enterprise environment having a webproxy?

    Because all network connections routed via proxy will have destination address of the proxy logged in the EDR network events.

  4. See: When network protection is turned on, you’ll see that on a device’s timeline the IP address will keep representing the proxy, while the real target address shows up.

  5. Hello and sorry for the late response. note the very last line of the below query, i have removed the reference to the url attribute, you should now see the url info as well.

    let urlhaus_feed_CH = (externaldata(url: string ) [@”https://urlhaus.abuse.ch/feeds/country/CH/”]
    with (format=”txt”))
    | where url !startswith “#”
    | project url;
    urlhaus_feed_CH
    // # Dateadded (UTC),URL,URL_status,Threat,Host,IPaddress,ASnumber,Country
    | extend data = parse_csv(url)
    | extend dateadded = data[0]
    | extend url = data[1]
    | extend url_status = data[2]
    | extend threat = data[3]
    | extend host = data[4]
    | extend IPaddress = data[5]
    | extend ASnumber = data[6]
    | extend Country = data[7]
    | project-away data

  6. Hi, thanks for this guide, it helped me hunt thousands of IOC’s very quickly by pasting them all in a single txt-file.
    However, I can’t work out how to hunt in a CSV file. Does someone have an example please?

  7. Dear proffesional,

    Is it possible to get the queries that you have used in the section ” Advanced hunting finding matches based on TI from URLhaus”.

    I can not read the images content.

    Thank you for this good knowledge

  9. Dear professional,
    Thank you very much.
    It is very, very useful.
    Thanks again
    //Damián

  10. Awesome reference, thanks so much! You’ve matched the IP list with results from DeviceNetworkEvents, DeviceFileEvents, and DeviceEvents. But do you have a way to match a column in the csv with results from those tables? I’m also wondering if there’s a way to add columns from the csv into the final results (e.g. adding the tags and the urlhaus_link columns).

    James

Leave a Reply