microsoft/Document-Knowledge-Mining-Solution-Accelerator

microsoft/Document-Knowledge-Mining-Solution-Accelerator

Description: Solution accelerator built on Azure OpenAI Service and Azure AI Document Intelligence to process and extract summaries, entities, and metadata from unstructured, multi-modal documents and enable searching and chatting over this data.

Language: C#

License: MIT

Stars: 221

Forks: 118

Open issues: 18

Created: 2024-09-19T19:02:38Z

Pushed: 2026-06-08T05:48:16Z

Default branch: main

Fork: no

Archived: no

README:

Document knowledge mining solution accelerator

Ingest, extract, and classify content from a high volume of assets to gain deeper insights and generate relevant suggestions for quick and easy reasoning. This enables the ability to conduct chat-based insight discovery, analysis, and receive suggested prompt guidance to further explore your data.

Solution overview

This solution leverages Azure OpenAI and Azure AI Document Intelligence in a hybrid approach by combining Optical Character Recognition (OCR) and multi-modal Large Language Model (LLM) to extract information from documents to provide insights without pre-training including text documents, handwritten text, charts, graphs, tables, and form fields.

Solution architecture

|| |---|

How to customize

If you'd like to customize the solution accelerator, here are some common areas to start:

[Technical architecture](./docs/TechnicalArchitecture.md)

[Content and data processing workflow](./docs/DataProcessing.md)

Key features

Click to learn more about the key features this solution enables

• Ingest and extract real-world entities

Process and extract information unique to your ingested data pipeline such as people, products, events, places, or behaviors. Used to populate filters.

• Chat-based insights discovery

Choose to chat with all indexed assets, a single asset, select a set of assets, or chat with a generated list of assets from a based on a user-led keyword search.

• Text and document data analysis

Analyze, compare, and synthesize materials into deep insights, making content accessible through natural language prompting.

• Prompt ​suggestion ​guidance

Suggest a next best set of questions based on the prompt inquiry. Include referenced materials to guide deeper avenues of user-led discovery.​

• ​Multi-modal information processing

Ingest and extract knowledge from multiple content types and various format types. Enhance with scanned images, handwritten forms, and text-based tables.​

Quick deploy

How to install or deploy

Follow the quick deploy steps on the deployment guide to deploy this solution to your own Azure subscription.

> Note: This solution accelerator requires Azure Developer CLI (azd) version 1.18.0 or higher. Please ensure you have the latest version installed before proceeding with deployment. Download azd here.

> Note: This solution accelerator also requires Bicep CLI version 0.33.0 or higher for compiling infrastructure templates. Install Bicep.

[Click here to launch the deployment guide](./docs/DeploymentGuide.md)

> Note: Some tenants may have additional security restrictions that run periodically and could impact the application (e.g., blocking public network access). If you experience issues or the application stops working, check if these restrictions are the cause. In such cases, consider deploying the WAF-supported version to ensure compliance. To configure, [Click here](./docs/DeploymentGuide.md#31-choose-deployment-type-optional).

> ⚠️ Important: Check Azure OpenAI Quota Availability

To ensure sufficient quota is available in your subscription, please follow [quota check instructions guide](./docs/QuotaCheck.md) before you deploy the solution.

Prerequisites and costs

To deploy this solution accelerator, ensure you have access to an Azure subscription with the necessary permissions to create resource groups, resources, app registrations, and assign roles at the resource group level. This should include Contributor role at the subscription level and Role Based Access Control role on the subscription and/or resource group level. Follow the steps in [Azure Account Set Up](./docs/AzureAccountSetUp.md).

*Note: Due to model availability within various data center regions, the following services have been hard-coded to specific regions:*

• Azure Open AI (GPT 4o mini):

The solution relies on GPT-4o mini and text-embedding-3-large models which are all currently available in the 'WestUS3', 'EastUS', 'EastUS2', 'SwedenCentral' region. Please check the model summary table and region availability if needed.

• Azure AI Document Intelligence (East US):

The solution relies on a 2023-10-31-preview or later that is currently available in East US region. The deployment region for this model is fixed in 'East US'

Check the Azure Products by Region page and select a region where the following services are available.

Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. The majority of the Azure resources used in this infrastructure are on usage-based pricing tiers. However, Azure Container Registry has a fixed cost per registry per day.

Use the Azure pricing calculator to calculate the cost of this solution in your subscription.

Review a sample pricing sheet in the event you want to customize and scale usage.

_Note: This is not meant to outline all costs as selected SKUs, scaled use, customizations, and integrations into your own tenant can affect the total consumption of this sample solution. The sample pricing sheet is meant to give you a starting point to customize the estimate for your specific needs._

| Product | Description | Cost | |---|---|---| | Azure OpenAI Service | Used for chat experience/RAG in web app, data processing workflow for…