Snowflake-Labs/Summit26-InteractiveLab

Snowflake-Labs/Summit26-InteractiveLab

Description: Interactive lab for Summit 2026

Language: Python

License: Apache-2.0

Stars: 2

Forks: 15

Open issues: 0

Created: 2026-03-02T21:33:09Z

Pushed: 2026-06-02T17:43:30Z

Default branch: main

Fork: no

Archived: no

README:

Summit 2026 Interactive Lab

Real-Time Arcade Score Streaming with Snowpipe Streaming + Interactive Tables

Stream thousands of arcade game scores from across the globe into Snowflake in real time, materialise them into an Interactive Table, and query with an Interactive Warehouse — experiencing sub-second latency at scale.

---

Architecture

Python Generator
─────────────────
Arcade Score Events Snowpipe Streaming SDK
(unlimited rows/sec) ──────► StreamingIngestClient
20 games └─ Channel 0 ──► ARCADE_SCORES
45 cities (Interactive Table)
500 players CLUSTER BY (GAME_ENDED_AT)
│
▼
SUMMIT_INT_WH
(Interactive Warehouse, XS)
Always-on · sub-second queries

Snowpipe Streaming uses the channel API, not SQL DML, so it writes rows directly into the Interactive Table — no intermediate landing table needed.

Warehouse design

| Warehouse | Type | Purpose | |---|---|---| | SUMMIT_TRAD_WH | Standard XS | Traditional warehouse for comparison benchmarks | | SUMMIT_INT_WH | Interactive XS | All lab queries; always-on for instant low-latency responses |

---

Prerequisites

| Requirement | Details | |---|---| | Python | 3.9 – 3.13 | | Snowflake account | In a supported region for Interactive Tables/Warehouses | | Role | ACCOUNTADMIN (or CREATE WAREHOUSE + CREATE DATABASE privileges) | | OpenSSL | For RSA key-pair generation | | JMeter | For concurrency testing (optional) |

Supported regions (Interactive Tables GA)

AWS: us-east-1, us-west-2, us-east-2, ca-central-1, ap-northeast-1, ap-southeast-2, eu-central-1, eu-west-1, eu-west-2 GCP: us-central1, us-east4, europe-west2/3/4, australia-southeast2 Azure: All Azure regions

---

Lab Setup

1 — Run the Snowflake setup script

Open `sql/01_setup.sql` in Snowsight and run it using a standard warehouse session.

The script provisions (in order): 1. ARCADE_STREAMING_ROLE + ARCADE_STREAMING_USER + RSA keypair auth policy 2. ARCADE_DB database + PUBLIC schema + SUMMIT_TRAD_WH standard warehouse 3. ARCADE_SCORES Interactive Table (CLUSTER BY (GAME_ENDED_AT), initially empty) 4. SUMMIT_INT_WH Interactive Warehouse (XS, always-on) 5. ARCADE_REPORTING_POOL compute pool (XS, for the optional Streamlit dashboard) 6. Grants for ARCADE_STREAMING_ROLE and ARCADE_LAB_READER

2 — Additional setup: Register RSA public key and configure network policy

bash sql/02_service_auth.sh

Generates rsa_key.p8 / rsa_key.pub if they don't exist, then prints additional SQL statements to complete the setup process. This includes an ALTER USER statement to register the RSA public key with the service account user.

IMPORTANT: You must copy and paste all generated SQL statements into Snowsight and run as ACCOUNTADMIN to complete the setup process.

> rsa_key.p8 is in .gitignore and must never be committed.

3 — Create profile.json

cp profile.json.example profile.json

Edit with your account identifier and the full path to rsa_key.p8 (the script from Step 2 prints the exact path):

{
"user": "ARCADE_STREAMING_USER",
"account": "YOUR_ORG-YOUR_ACCOUNT",
"url": "https://YOUR_ORG-YOUR_ACCOUNT.snowflakecomputing.com:443",
"private_key_file": "/full/path/to/rsa_key.p8",
"role": "ARCADE_STREAMING_ROLE"
}

For convenience, the output of the final SQL statement from Step 2 contains a fully populated JSON string which can be used direclty in profile.json.

4 — Install Python dependencies

python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt

5 — Start the arcade streamer

cd python
python arcade_streamer.py

============================================================
Summit 2026 – Arcade Scores Snowpipe Streamer
============================================================
Account : YOUR_ORG-YOUR_ACCOUNT
Database : ARCADE_DB.PUBLIC
Pipe : ARCADE_SCORES-STREAMING
Channels : 1
Target : unlimited rows/sec
============================================================

[14:22:05] rows: 512 | 512.0 rows/sec | errors: 0 | elapsed: 1s
[14:22:05] [latency] ARCADE_CHANNEL_0_A3F2B1C4: 540 ms avg
[14:22:10] rows: 1,024 | 512.0 rows/sec | errors: 0 | elapsed: 6s
[14:22:10] [latency] ARCADE_CHANNEL_0_A3F2B1C4: 512 ms avg

6 — Wait for cache warm-up

The SUMMIT_INT_WH Interactive Warehouse starts warming its local SSD cache as soon as it resumes. Wait 2–3 minutes after the streamer starts before running interactive queries — the first few queries after resume will be slower while the cache populates.

---

Lab Exercises

Open `sql/03_lab_queries.sql` in Snowsight while the streamer is running.

> Tip: Use SUMMIT_INT_WH for exercises marked ⚡ and SUMMIT_TRAD_WH > for exercises marked 🔧. Each USE WAREHOUSE statement is already in the > query file.

⚡ Exercise 1 — Pipeline throughput

Watch row counts grow in real time and measure live ingest throughput (rows/sec). The SDK-reported avg processing latency per channel is also printed to the streamer console.

⚡ Exercise 2 — Data freshness

Each row carries GAME_ENDED_AT (when the score was generated and sent, in UTC). The query measures freshness: how many seconds ago was the most recently committed row? The streamer's console also logs Snowflake-reported avg processing latency per channel via the SDK's get_channel_statuses() API.

⚡ Exercise 3 — Global leaderboard (Interactive Warehouse)

Top 20 scores of the last 24 hours. Notice how the CLUSTER BY (GAME_ENDED_AT) clustering key allows the Interactive Warehouse to skip irrelevant partitions and return results in under a second.

⚡ Exercise 4 — Per-game top 5 (window function)

Uses QUALIFY ROW_NUMBER() scoped to the last hour. Stays well within the interactive warehouse 5-second query limit.

⚡ Exercise 5 — Country heat map

Which countries are playing most right now? Japan and South Korea should dominate — the data generator weights cities by real gaming culture.

⚡ Exercise 6 — Game popularity

Pac-Man and…