The following is provided from Microsoft Security and Compliance blogs at TechCommunity:

While well-funded and highly organized security operations teams often have the most sophisticated detection mechanisms in place, these teams still need experts that can run guided investigations to locate and stop certain threats. For example, sophisticated attackers often live off the land, taking advantage of normal system functionality that leaves almost no identifiable traces. While behavior-based detection algorithms powered by machine learning and AI can learn and respond quickly, human experts remain extremely valuable, especially if they know the network and are familiar with how attacks might play out.

 

What is threat hunting?

Cyberthreat hunting or simply threat hunting is a proactive cybersecurity activity that aims to find threats that are either buried under massive quantities of security signals and alert data or are simply not flagged by security products. It is generally a manual process, although great tools that we will describe in this article can make the process much less tedious and time-consuming.

 

During threat hunting, SecOps practitioners apply threat intelligence takeaways, whether from their own internal research or external research, and devise ingenious ways to determine the existence of an otherwise undetected threat. To do that, they need efficient access to comprehensive data about events and entities in their network as well as a good, quantifiable understanding of normal states or baselines.

 

Threat hunting lets analysts work with established baselines and highlight behavior that might be interesting. With the right tools, analysts can tailor their threat hunting activities to their environments and the threats that they will likely encounter. For instance, they can hunt for unusual behavior—like unexpected network connections—that might indicate that an in-house app or an account has been compromised.

 

The process of establishing the baselines themselves can also be part of threat hunting. To be able to do this, analysts need tools that can look backwards and forwards in time quickly, providing data that is sufficiently granular for defining normal states.

 

Effective threat hunting relies on:

 

	Comprehensive, well-structured, and retrievable event and system data
	Threat intelligence: knowledge about threat actors or actor infrastructure, methodologies, and indicators
	Granular baseline information that represents normal activity and states

 

Threat hunting example

 

 Let’s look at what Jessica, our fictional but awesome SecOps person, might go through:

 

1. Jessica, who works for Contoso Health Services, finds out about a new vulnerability that affects one of the product suites in her environment. In this case, the attacks are against a known web content management system (CMS).
2. After doing some more research online, she determines that, because the release of this vulnerability is so recent, it is unclear how attackers might be able to leverage it in her environment. She also knows that a patch to remedy the issue is not yet available.
3. She creates a query for behaviors tied to the processes involved in this vulnerability to determine existing baseline and normal behavior. She then modifies queries to return only what would not be expected.
4. Jessica also creates rules so that the queries run regularly and send her notifications whenever there are matches.
5. Because Jessica did her research and constructed her queries very well—carefully considering the possibility that some unaffected machines might exhibit threat-like behavior—each match to her query constitutes a viable threat-hunting find. These matches include unusual process activity that might very well be actual attempts to abuse the vulnerable CMS.

Clearly, Jessica’s finds can benefit Contoso Health Services by proactively locating exploitation attempts against an unpatched vulnerability. Likewise, her ability to efficiently design and deploy proactive defenses highlights her own capabilities as a defender.

 

Data is key

With cloud-based storage and compute solutions, we can now easily collect massive quantities of data. But as we store larger data sets, there is a growing need to be able to efficiently manipulate and make sense of them.

 

Microsoft Threat Protection itself is made possible by the power of the Azure cloud coupled with insights from the Intelligent Security Graph. In the background, massive amounts of threat intelligence and security data from across Microsoft’s portfolio are crunched and matched against indicators, expert human rules, and machine learning (ML) algorithms in Microsoft AI. This process generates meaningful alerts, identifying threat components and activities that automated investigation and response (AIR) capabilities remediate.

 

For example, Microsoft Threat Protection distinguishes between malicious and normal attempts to write to the registry by looking at millions of examples of registry writes and their contexts: the files or processes involved, file pedigrees, whatever was written to the registry, the time the writes were performed, and so on. With this much baseline info, the AI can confidently raise alerts and start performing remediation activities, rapidly placing harmful registry modifications and associated files in quarantine.

 

While AI and other automated systems are particularly effective at finding threats, human intuition and flexibility can still beat them when dealing with highly specialized or unusual scenarios. What human analysts need, however, are tools that let them:

 

• Effectively access and handle large sets of data — while the interface is easy-to-use, the tool also needs to be responsive and must label and organize data well. At the same time, log storage must be as straightforward as any competitive cloud-based storage and compute solution that can be deployed and scaled without professional system integrators and other specialists.
• Automate monitoring of interesting matches to new data — going back to Jessica’s fictional investigation, the tools should let her monitor new activities for matches to the attack activities she has modeled. Without this automation, Jessica will be tied to her chair, constantly looking for matches as new data comes in.

 

Cross-product advanced hunting with Microsoft Threat Protection

With advanced hunting in Microsoft Threat Protection—available in the Microsoft 365 security center with a valid license (go here to get started)—you can deep dive and hunt across data from various workspaces in your Microsoft 365 environment. Advanced hunting initially covers both your endpoints and your Office 365 email. By the end of March 2020, we will expand the schema to cover identity- and app-related signals from Azure ATP and Microsoft Cloud App Security.

 

You can work with Kusto queries, plus you have the convenience of switching to richer views made possible by the various integrated solutions. For example, you can drilldown from a query to dedicated pages with comprehensive contextual information about specific alerts, devices, users, domains, IP addresses, and even software vulnerabilities.

 

The specialized data set is organized in a manageable schema covering security-sensitive event and entity information, such as device info, network configuration info, process events, registry events, logon events, file events, and email events.

 

Microsoft will continually incorporate more information into this schema. Here are a few examples of the sophisticated threat hunting activities you can perform with the current coverage.

 

Identify vulnerable devices

With advanced hunting, you can access software inventory information from Threat & Vulnerability Management. Imagine being able to write queries that check for possible exploitation behavior on devices running vulnerable software.

 

The following sample query locates machines affected by the RDP vulnerability CVE-2019-0708—popularly known as “BlueKeep”—and checks for actual RDP connections initiated by unexpected executables:

 

let BlueKeepVulnerableMachines = DeviceTvmSoftwareInventoryVulnerabilities 
| where CveId == "CVE-2019-0708"
| distinct DeviceId;
// Find unusual processes on Windows 7 or Windows Server 2008 machines with
// outbound connections to TCP port 3389 
let listMachines = DeviceInfo
| where OSVersion == "6.1" //Win7 and Srv2008
| distinct DeviceId;
DeviceNetworkEvents
| where DeviceId in(BlueKeepVulnerableMachines)
| where RemotePort == 3389
| where Protocol == "Tcp" and ActionType == "ConnectionSuccess"
| where InitiatingProcessFileName !in~ //Removing expected programs
("mstsc.exe","RTSApp.exe", "RTS2App.exe","RDCMan.exe","ws_TunnelService.exe", 
"RSSensor.exe","RemoteDesktopManagerFree.exe","RemoteDesktopManager.exe", 
"RemoteDesktopManager64.exe","mRemoteNG.exe","mRemote.exe","Terminals.exe", 
"spiceworks-finder.exe","FSDiscovery.exe","FSAssessment.exe", "chrome.exe", 
"microsodeedgecp.exe", "LTSVC.exe", "Hyper-RemoteDesktop.exe", "", "RetinaEngine.exe",
"AuvikService.exe", "AuvikAgentService.exe", "CollectGuestLogs.exe", 
"NetworkWatcherAgent.exe", "MobaRTE.exe", "java.exe", "mscorsvw.exe", "MultiDesk.exe", 
"Microsoft Remote Desktop", "javaw.exe", "ASGRD.exe", "MultiDesk64.exe", "Passwordstate.exe")
| join listMachines on DeviceId
| project Timestamp, DeviceId, DeviceName, RemoteIP, InitiatingProcessFileName, 
InitiatingProcessFolderPath, InitiatingProcessSHA1
| summarize conn=count() by DeviceId, InitiatingProcessFileName, bin(Timestamp, 1d)

 

Hunt for threats that land by email and impact devices

You can also run queries that track threats that might have arrived through email and then traversed your endpoints. For example, this simple query checks for files from a known malicious email sender:

 

//Get prevalence of files sent by a malicious sender in your organization
EmailAttachmentInfo
| where SenderFromAddress =~ "MaliciousSender@example.com"
| where isnotempty(SHA256)
| join (
DeviceFileEvents
| project FileName, SHA256
) on SHA256

Read more about hunting on devices and email

 

Start simple, learn fast

There’s no need to get intimidated by the query interface as the Kusto Query Language is straightforward. It has very powerful data manipulation capabilities that can be learned with more experience, but it takes only a few minutes to begin running simple queries, like locating a file SHA mentioned in the Twitter feed of your favorite security researcher.

 

Once you are there, you can easily look deeper into an instance of the SHA on a specific device or grab a list of all the devices with that SHA and look for commonalities between those devices. Again, it does not hurt that you have other Microsoft Threat Protection features, such as file and machine profile pages, at your disposal.

 

Advanced hunting is backed by a strong community of experienced security practitioners and Kusto Query Language users who are ready to share expertise so that you can easily learn a new syntax. You will find many blog posts in the Microsoft Defender ATP Tech Community discussing various query techniques. You could also explore the Microsoft Threat Protection repository or the Microsoft Defender ATP repository for queries covering various known threat campaigns and techniques.

Soon enough, you’ll be creating custom detection rules—available by the end of March 2020 with Microsoft Threat Protection—from your hunting queries. These detection rules automatically check for and respond to various events and system states, including suspected breach activity and misconfigured machines.

 

Try it yourself

It’s time to try advanced hunting for yourself! If you believe PowerShell download activity in your network is likely suspicious, give the query below a try.

 

// Finds PowerShell execution events that could involve a download
union DeviceProcessEvents, DeviceNetworkEvents
| where Timestamp > ago(7d)
// Pivoting on PowerShell processes
| where FileName in~ ("powershell.exe", "powershell_ise.exe")
// Suspicious commands
| where ProcessCommandLine has_any("WebClient",
"DownloadFile",
"DownloadData",
"DownloadString",
"WebRequest",
"Shellcode",
"http",
"https")
| project Timestamp, DeviceName, InitiatingProcessFileName, InitiatingProcessCommandLine, 
FileName, ProcessCommandLine, RemoteIP, RemoteUrl, RemotePort, RemoteIPType
| top 100 by Timestamp

 

Want to explore this query further and understand how it might catch malicious activity? Learn how this query works

 

Hunt across your entire environment with Azure Sentinel

The advanced hunting capabilities in Microsoft Threat Protection enable you to find threats across your users, endpoints, email and productivity tools, and apps. You can integrate the data from Microsoft Threat Protection into Azure Sentinel and then expand that dataset to include data from Azure Security Center and third-party security products to find threats that span your entire environment.

 

Azure Sentinel provides cloud-native SIEM capabilities, including AI that fuses multiple alerts to a complete attack chain. For example, it can take an alert from Microsoft Threat Protection and combine that with an alert from a third-party firewall. You can then visualize that attack chain or use Kusto Query Language to query across the full set of security data and then remediate the issue and put in place an automated solution with Azure Logic Apps.

 

Louie Mayor & Justin Carroll

Microsoft Threat Protection team

The above was provided from Microsoft Security and Compliance blogs at TechCommunity