RepoSnowflake (Arctic)Snowflake (Arctic)published Nov 5, 2025seen 5d

Snowflake-Labs/sfguide-declarative-pipelines-dynamic-tables

Python

Open original ↗

Captured source

source ↗
GH
GitHub/github.com/Snowflake-Labs/sfguide-declarative-pipelines-dynamic-tables
Snowflake-Labs/sfguide-declarative-pipelines-dynamic-tables repository metadata
Source ↗
published Nov 5, 2025seen 5dcaptured 8hhttp 200method plain

Snowflake-Labs/sfguide-declarative-pipelines-dynamic-tables

Language: Python

Stars: 26

Forks: 29

Open issues: 0

Created: 2025-11-05T03:30:21Z

Pushed: 2026-05-27T10:57:41Z

Default branch: main

Fork: no

Archived: no

README:

Build Autonomous SQL Pipelines with Cortex Code & Dynamic Tables

This lab demonstrates building a declarative data pipeline using Snowflake Dynamic Tables — driven entirely by natural language prompts to Cortex Code inside Snowsight Workspaces.

Instead of manually writing SQL, you describe your pipeline intent to Cortex Code and it generates, validates, and executes the SQL for you.

Prerequisites

  • Snowflake account with ACCOUNTADMIN access (free trial)
  • Cortex Code enabled (setup guide)
  • Basic familiarity with data engineering concepts

How It Works

Each SQL file in this repo follows a prompt-first pattern:

/*
================================================================================
CORTEX CODE PROMPT
================================================================================

================================================================================
EXPECTED OUTPUT

================================================================================
*/

-- The expected SQL follows below...

Workflow: 1. Create a Snowsight Workspace from this repository 2. Open the Cortex Code panel (Cmd+L) 3. Open a SQL file — copy the prompt at the top into CoCo 4. Review CoCo's generated SQL against the expected output below 5. Execute when satisfied

The SQL files remain fully runnable on their own for anyone who prefers the traditional approach.

Files

| File | Purpose | CoCo Approach | |:-----|:--------|:--------------| | 00_setup_environment.sql | Role, DB, warehouse, tables, data load | Direct execution | | 01_dynamic_tables.sql | 3-tier pipeline (5 dynamic tables) | Generate-then-confirm | | 02_sproc.sql | Stored procedure for synthetic test data | Generate-then-confirm | | 03_incremental_refresh.sql | Test incremental refresh capabilities | Sequential prompts | | 04_monitoring.sql | Pipeline monitoring queries | Direct execution | | 05_semantic_view_agent.sql | Semantic view + Cortex Agent creation | Generate-then-confirm | | 06_cleanup.sql | Drop all lab resources | Direct execution |

Branches

| Branch | Contents | Purpose | |:-------|:---------|:--------| | main | Prompt-only SQL files | Create your Workspace from this branch — CoCo sees prompts, not answers | | solutions | Full SQL implementation | Reference / answer key — check here after completing each step |

1. Navigate to Projects > Workspaces in Snowsight 2. Click Create > From Git repository 3. Enter: https://github.com/Snowflake-Labs/sfguide-declarative-pipelines-dynamic-tables 4. Select Public repository 5. Open Cortex Code (Cmd+L) and start with 00_setup_environment.sql

What You'll Build

  • A three-tier declarative data pipeline processing ~1B order records
  • Stored procedures for generating test data
  • Incremental refresh validation
  • Monitoring queries for pipeline observability
  • A semantic view for natural language querying
  • A Cortex Agent for conversational data exploration

Related Resources

License

Apache 2.0

Notability

notability 3.0/10

Routine repo, low traction (26 stars)