microsoft/msticpy

microsoft/msticpy

Description: Microsoft Threat Intelligence Security Tools

Language: Python

License: NOASSERTION

Stars: 1970

Forks: 332

Open issues: 11

Created: 2019-02-21T17:30:03Z

Pushed: 2026-06-08T13:03:14Z

Default branch: main

Fork: no

Archived: no

README:

MSTIC Jupyter and Python Security Tools

!GitHub Actions build  

Microsoft Threat Intelligence Python Security Tools.

msticpy is a library for InfoSec investigation and hunting in Jupyter Notebooks. It includes functionality to:

• query log data from multiple sources
• enrich the data with Threat Intelligence, geolocations and Azure

resource data

• extract Indicators of Activity (IoA) from logs and unpack encoded data
• perform sophisticated analysis such as anomalous session detection and

time series decomposition

• visualize data using interactive timelines, process trees and

multi-dimensional Morph Charts

It also includes some time-saving notebook tools such as widgets to set query time boundaries, select and display items from lists, and configure the notebook environment.

The msticpy package was initially developed to support Jupyter Notebooks authoring for Microsoft Sentinel. While Azure Sentinel is still a big focus of our work, we are extending the data query/acquisition components to pull log data from other sources (currently Splunk, Microsoft Defender for Endpoint and Microsoft Graph are supported but we are actively working on support for data from other SIEM platforms). Most of the components can also be used with data from any source. Pandas DataFrames are used as the ubiquitous input and output format of almost all components. There is also a data provider to make it easy to and process data from local CSV files and pickled DataFrames.

The package addresses three central needs for security investigators and hunters:

• Acquiring and enriching data
• Analyzing data
• Visualizing data

We welcome feedback, bug reports, suggestions for new features and contributions.

Installing

For core install:

pip install msticpy

or for the latest dev build

pip install git+https://github.com/microsoft/msticpy

Upgrading

To upgrade msticpy to the latest public non-beta release, run:

pip install --upgrade msticpy

Note it is good practice to copy your msticpyconfig.yaml and store it on your disk but outside of your msticpy folder, referencing it in an environment variable. This prevents you from losing your configurations every time you update your msticpy installation.

Documentation

Full documentation is at ReadTheDocs

Sample notebooks for many of the modules are in the docs/notebooks folder and accompanying notebooks.

You can also browse through the sample notebooks referenced at the end of this document to see some of the functionality used in context. You can play with some of the package functions in this interactive demo on mybinder.org.


---

Log Data Acquisition

QueryProvider is an extensible query library targeting Microsoft Sentinel/Log Analytics, Microsoft XDR, Splunk, OData and other log data sources. It also has special support for Mordor data sets and using local data.

Built-in parameterized queries allow complex queries to be run from a single function call. Add your own queries using a simple YAML schema.

Data Queries Notebook

Data Enrichment

Threat Intelligence providers

The TILookup class can lookup IoCs across multiple TI providers. built-in providers include AlienVault OTX, IBM XForce, VirusTotal and Azure Sentinel.

The input can be a single IoC observable or a pandas DataFrame containing multiple observables. Depending on the provider, you may require an account and an API key. Some providers also enforce throttling (especially for free tiers), which might affect performing bulk lookups.

TIProviders and TILookup Usage Notebook

GeoLocation Data

The GeoIP lookup classes allow you to match the geo-locations of IP addresses using either:

• GeoLiteLookup - Maxmind Geolite (see )
• IPStackLookup - IPStack (see )

GeoIP Lookup and GeoIP Notebook

Azure Resource Data, Storage and Azure Sentinel API

The AzureData module contains functionality for enriching data regarding Azure host details with additional host details exposed via the Azure API. The AzureSentinel module allows you to query incidents, retrieve detector and hunting queries. AzureBlogStorage lets you read and write data from blob storage.

Azure Resource APIs, Azure Sentinel APIs, Azure Storage

Security Analysis

This subpackage contains several modules helpful for working on security investigations and hunting:

Anomalous Sequence Detection

Detect unusual sequences of events in your Office, Active Directory or other log data. You can extract sessions (e.g. activity initiated by the same account) and identify and visualize unusual sequences of activity. For example, detecting an attacker setting a mail forwarding rule on someone's mailbox.…