Run Snowflake Arctic with an API

Run Snowflake Arctic with an API – Replicate blog

Replicate Blog

Run Snowflake Arctic with an API

Posted April 23, 2024 by cbh123

Snowflake Arctic is a new open-source language model from Snowflake. Arctic is on-par or better than both Llama 3 8B and Llama 2 70B on all metrics while using less than half of the training compute budget. It’s massive. At 480B, Arctic is the biggest open-source model to date. As expected from a model from Snowflake, it’s good at SQL and other coding tasks, and it has a liberal Apache 2.0 license. With Replicate, you can run Arctic in the cloud with one line of code.

Try Arctic in our API playground

Before you dive in, try Arctic in our API playground.

Try tweaking the prompt and see how Arctic responds. Most models on Replicate have an interactive API playground like this, available on the model page: https://replicate.com/snowflake/snowflake-arctic-instruct

The API playground is a great way to get a feel for what a model can do, and provides copyable code snippets in a variety of languages to help you get started.

Running Arctic with JavaScript

You can run Arctic with our official JavaScript client :

Install Replicate’s Node.js client library

Copy

npm install replicate

Set the REPLICATE_API_TOKEN environment variable

Copy

export REPLICATE_API_TOKEN = r8_9wm **********************************

(You can generate an API token in your account. Keep it to yourself.)

Import and set up the client

Copy

import Replicate from "replicate" ;

const replicate = new Replicate ({ auth: process.env. REPLICATE_API_TOKEN , });

Run snowflake/snowflake-arctic-instruct using Replicate’s API. Check out the model’s schema for an overview of inputs and outputs.

Copy

const input = {};

for await ( const event of replicate. stream ( "snowflake/snowflake-arctic-instruct" , { input })) { process.stdout. write (event. toString ()); };

To learn more, take a look at the guide on getting started with Node.js .

Running Arctic with Python

You can run Arctic with our official Python client :

Install Replicate’s Python client library

Copy

pip install replicate

Set the REPLICATE_API_TOKEN environment variable

Copy

export REPLICATE_API_TOKEN = r8_9wm **********************************

(You can generate an API token in your account. Keep it to yourself.)

Import the client

Copy

import replicate

Run snowflake/snowflake-arctic-instruct using Replicate’s API. Check out the model’s schema for an overview of inputs and outputs.

Copy

The snowflake/snowflake-arctic-instruct model can stream output as it's running.

for event in replicate.stream( "snowflake/snowflake-arctic-instruct" , input = {}, ): print ( str (event), end = "" )

To learn more, take a look at the guide on getting started with Python .

Running Arctic with cURL

Your can call the HTTP API directly with tools like cURL:

Set the REPLICATE_API_TOKEN environment variable

Copy

export REPLICATE_API_TOKEN = r8_9wm **********************************

(You can generate an API token in your account. Keep it to yourself.)

Run snowflake/snowflake-arctic-instruct using Replicate’s API. Check out the model’s schema for an overview of inputs and outputs.

Copy

curl -s -X POST \ -H "Authorization: Bearer $REPLICATE_API_TOKEN " \ -H "Content-Type: application/json" \ -H "Prefer: wait" \ -d $'{ "input": {} }' \ https://api.replicate.com/v1/models/snowflake/snowflake-arctic-instruct/predictions

To learn more, take a look at Replicate’s HTTP API reference docs .

You can also run Arctic using other Replicate client libraries for Go, Swift, and others .

Snowflake Arctic Chatbot

If you want a place to start, Streamlit have built a demo app with Replicate :

Try it out here  or  fork it on GitHub .

Keep up to speed

Sign up for Snowflake’s Arctic Hackathon!

Chat with Arctic.

Follow us on  Twitter  X to get the latest from the Arctic.

Hop in our Discord to talk Arctic.

Happy hacking! 🦙

Next: Run Meta Llama 3 with an API