deepinfra/deepinfra-chat

deepinfra/deepinfra-chat

Description: Sample Next.js ai chat app using Deep Infra inference and Vercel ai sdk

Language: TypeScript

Stars: 1

Forks: 2

Open issues: 0

Created: 2025-03-17T09:01:46Z

Pushed: 2025-03-17T13:57:12Z

Default branch: main

Fork: no

Archived: no

README: This is a sample Next.js ai chat application that uses Deep Infra models for inference and Vercel AI SDK.

Table of Contents

• [Deploy](#deploy)
• [Getting Started](#getting-started): Use the Deep Infra Vercel integration to quickly setup and run this sample app
• [Manual Setup](#manual-setup): Detailed instruction for local development
• [Experiment](#experiment): Try diffenet models and inference options

Deploy

![Deploy with Vercel](https://vercel.com/new/clone?repository-url=https://github.com/deepinfra/deepinfra-chat&products=[{"type":"integration","integrationSlug":"deepinfra","productSlug":"api-token","protocol":"ai"}])

Getting Started

This section assumes you have set up a Deep Infra account and project using the Vercel Integration (press deploy button above).

Step 1. Pull environment variables

You'll need a Deep Infra API key in your environment variables to connect to the model. Run the following command to pull them from Vercel:

vercel env pull

Step 2. Run the app

Run npm run dev. You can start chatting with the ai model immediately.

Manual Setup

Step 1. Deep Infra account

Create a Deep Infra account either through the Vercel marketplace integration or by directly registering at Deep Infra

Step 2. Clone the sample app

git clone git@github.com:deepinfra/deepinfra-chat.git

cd deepinfra-chat

npm install

Step 3. Set up environment variables

Copy the .env.local.example file in this directory to .env.local (which will be ignored by Git):

cp .env.local.example .env.local

From the api keys page in your Deep Infra dashboard, create a new token or use an existing one. Use that token to set the DEEPINFRA_API_KEY variable in .env.local

Experiment

By default the sample app uses model meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo specified in app/page.tsx:

const DI_MODEL = "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo";

The actual inference calls to the model happen in app/api/chat/route.ts:

export async function POST(req: Request) {
const { messages, model } = await req.json();

const result = streamText({
model: deepinfra(model),
system: "Be a helpful assistant.",
messages,
});

return result.toDataStreamResponse();
}

You can experiment with different Deep Infra models, prompts and options. See the Deep Infra docs, Vercel AI SDK docs and Deep Infra AI SDK Provider docs.