Snowflake-Labs/decision-models-in-snowflake-model-registry

Snowflake-Labs/decision-models-in-snowflake-model-registry

Description: Run an open-weight decision model (decider-2b) from the Snowflake Model Registry on SPCS and score it on JevBench

Language: TypeScript

License: Apache-2.0

Stars: 0

Forks: 0

Open issues: 0

Created: 2026-10-01T13:38:02Z

Pushed: 2026-10-05T18:09:23Z

Default branch: main

Fork: no

Archived: no

README:

Run Jev-class decision models on Snowflake

Code for the Snowflake Developers Blog post Decisions Are All You Need: Run Jev-Class Decision Models on Snowflake.

It runs decider-2b, an open-weight decision model, from the Snowflake Model Registry as a batch inference job (run_batch) on one NVIDIA A10G. JevBench's own harness scores the answers on JevBench's 231 public decisions.

The same model also runs as a real-time inference service, called over its REST endpoint and from SQL. Both paths scale across GPUs. A Snowflake App Runtime app in app/triage-board/ uses the real-time service to triage support tickets side by side with a frontier LLM.

Throughput comes from query history and client-side timers. Cost comes from Snowflake's metering views.

decider-2b is an independent open model. It isn't TypeSafe AI's Jev, and JevBench isn't affiliated with TypeSafe AI. Nothing here is a JevBench Score or a leaderboard entry.

What it costs

The GPU pool bills for every minute its node is up, including the five minutes before auto-suspend. Run sql/99_teardown.sql when you're done.

Prices below are Standard edition list prices in AWS US West (Oregon).

• Batch job. GPU time while scoring the 231 decisions is $0.0057, or $0.025 per 1,000 decisions. Each job also holds the node for about 6 minutes to start, load and warm up the model and write its output (about $0.11). The first job builds the image (about 7 minutes, $0.13).
• Service. The image builds on a CPU pool (about $0.05). The GPU node bills for every hour it stays up, busy or not: 0.57 credits, or $1.14, an hour.
• REST. The 231 decisions took 17.7 s from 8 concurrent clients ($0.024 per 1,000 in GPU node time) and 37.4 s from one ($0.051 per 1,000).
• Storage. A model version (3.78 GB) is about $0.09 a month.
• Triage board. The app keeps its own decider-2b service on one GPU_NV_S node ($1.14 an hour while it's up) and runs AI_COMPLETE on an XS warehouse. In our 25-ticket run with one request in flight per side, decider-2b cost $0.049 per 1,000 tickets in GPU time and claude-sonnet-5 cost $4.98 per 1,000 in tokens and warehouse time. Its README has the teardown.

Requirements

• Python 3.12 and the Snowflake CLI with a connection configured.
• A region with GPU_NV_S compute pools.
• A role that can:
• create a database, a warehouse and a compute pool (CREATE DATABASE, CREATE WAREHOUSE and CREATE COMPUTE POOL on the account), plus an external access integration for the service path (CREATE INTEGRATION);
• create an image repository, a model and services in the schema (batch jobs are services), plus a network rule for the service path;
• read SNOWFLAKE.ACCOUNT_USAGE for the cost queries.

The scripts don't set a role. They use your connection's default.

• For the triage board: Node.js and npm, Snowflake CLI 3.26 or later (for snow app deploy), Snowflake App Runtime, and AI_COMPLETE access to claude-sonnet-5.

Layout

| Path | What it does | |---|---| | python/decider_model.py | The Model Registry CustomModel wrapper. Its one method, system_one, takes JevBench's request and returns decider's response | | python/log_model_batch.py | Registers the weights for batch jobs as version BATCH: the same wrapper, with version ranges for the helper libraries the batch base image needs | | python/bench_batch.py | Runs a version over the 231 items with run_batch on the GPU pool, saves the job's output and scores it with JevBench's harness | | python/log_model.py, python/create_service.py | The service path: registers version SERVICE with every package pinned, builds the image on a CPU pool and starts the service | | python/create_service_http.py, python/bench_http.py | Real-time inference over REST. Starts the same version as DECIDER_2B_HTTP with a public endpoint, then times the 231 items one request at a time, from 8 concurrent clients, and as 20 single-row SQL calls. Scores the answers with JevBench's harness and prices the GPU time | | python/bench_scaling.py | Batch throughput as you add model copies per GPU (num_workers) or nodes (replicas). Repeats the 231 items and runs one job per config. Records scoring time, decisions per second, when each worker joined, and cost at list price | | python/bench_http_sweep.py | Real-time inference under load. Against a service with several instances, runs 1, 4, 16 and 64 closed-loop REST clients for 30 seconds each. Records decisions per second, p50/p95/p99 latency, errors, how requests spread across instances, and cost at list price | | python/decider_model_batched.py, python/replay_batch_plans.py | The batched serving path I also tried (slower on these items), and an offline replay of its batch plans | | python/load_jevbench.py | Loads the 231 public items into JEVBENCH_ITEMS | | python/smoke_test_local.py | Runs the wrapper in-process on six items and checks that JevBench parses the output, before any GPU starts | | python/score_with_jevbench.py | Scores the service run's stored answers with JevBench's harness at commit bb05a335 | | python/bench_config.py, python/snowpark_session.py | The database and schema for the Python scripts, and a Snowpark session from your CLI connection | | python/requirements.txt | The local Python environment | | sql/00–09 | Setup, prices, the GPU pool (02_compute), the service's build pool and egress (02_service_build), views, the service pre-flight, loading a batch job's output (05_batch_load), the service run, throughput and cost | | sql/99_teardown.sql | Drops the pools, warehouse, services and integration | | results/ | The scored outputs and summaries behind the post | | app/triage-board/ | A Snowflake App Runtime (Next.js) app. It triages the same support tickets live with decider-2b's...