microsoft/SynapseML v1.1.0

SynapseML v1.1.0

Repository: microsoft/SynapseML

Tag: v1.1.0

Published: 2025-11-20T03:51:26Z

Prerelease: no

Release notes:

##### We are excited to announce the release of SynapseML v1.1 marking a host of powerful new features introduced since the initial v1.0 release. SynapseML is an open-source library that aims to streamline the development of massively scalable machine learning pipelines. It unifies several existing ML Frameworks and new Microsoft algorithms in a single, scalable API that is usable across Python, R, Scala, and Java. SynapseML is usable from any Apache Spark platform with first class enterprise support on Microsoft Fabric.

Highlights

| | | | |:--:|:--:|:--:| |Microsoft Fabric | AI Functions | OneLake | | Build and operationalize distributed ML with SynapseML in Fabric | Apply Pandas and Spark LLM transformations with one line of code | Automatically derive AI insights for unstructured data in OneLake | | Build Your First Model | Explore AI Functions | Learn More |

| | | |:--:|:--:| |Hugging Face | Azure AI Foundry | | Use open source models hosted on Hugging Face | Run Azure AI Foundry models in your notebook | | Try an Example | View Notebook |

More Hightlights

Spark 3.5 Support – In this version we transitioned to Spark 3.5 as our main Spark platform.

OpenAI Ecosystem – Comprehensive improvements including global parameter defaults, GPT-4 enablement, custom endpoints/headers, GPU-accelerated embeddings with KNN, and fine-grained control over model parameters (top_p, seed, responseFormat, temperature).

ML Innovation – HuggingFaceCausalLM transformer for distributed language model evaluation, custom embedder support, and synthetic difference-in-differences causal inference module.

Platform features – Spark Native OneLake support; MSI for Azure Storage; OpenAITranslate transformer.

AI Functions in Data Wrangler on Fabric – AI Functions built into Data Wrangler in Fabric allow you to apply LLM-powered operations to your dataframe without writing a single line of code.

New Features

Documentation 📚

• AI Functions
• AI Powered Transforms in OneLake
• Azure OpenAI for Big Data in SynapseML
• AI Functions in Data Wrangler
• AI Foundry
• Hugging Face

AI Functions ⚡

• Added support for AI Functions in Pandas (#1579613, #1585011, #1596611, #1509195, #1501185, #1494610, #1494951)
• Added support for AI Functions in PySpark (#1460790, #1572928, #1599735, #1439858, #1463533)
• Added support for async AI Functions execution (#1529058, #1523727)
• JSON response support & improved language validation. (#1551823, #1566189)
• Seed param for reproducible chat/completions (API 2024-10-21). (#1551883)
• Fuzzy case-insensitive matching for Classify. (#1515064)
• Add AI Functions Operations to Data Wrangler (#1590130, #1638257, #1718967, #1725101, #1730446)

Azure OpenAI 🌸

• Enhanced Model Parameters – Added top_p, seed, responseFormat, temperature, and subscription key support (#2410, #2329,...