Rag With Astro Fastapi Surrealdb Tailwind

Building a RAG with Astro, FastAPI, SurrealDB and Llama 3.1

GLM 5.2 is live! Opus-level intelligence at open-source rates. Pay per token on serverless. Try it today.

Blog

RAG With Astro Fastapi Surrealdb Tailwind Building a RAG with Astro, FastAPI, SurrealDB and Llama 3.1

PUBLISHED 8/14/2024

Table of Contents Prerequisites Tech Stack High-Level Data Flow and Operations Step 1: Setup SurrealDB Server Step 2: Generate Step 3: Create a new FastAPI application

Install Dependencies

Define Data Models using Pydantic

Use Fireworks API Key

Use Fireworks Nomic AI Embeddings Model

Define SurrealDB Vector Store

Initialize FastAPI App

Create a Knowledge Update API endpoint

Create a Chat API endpoint

Run FastAPI App Locally Create a new Astro application

Add Tailwind CSS to the application

Integrate React in your Astro project

Install an AI SDK and Axios

Build Conversation User Interface

Build User Interface to Update Application’s Knowledge

Run Astro Application Locally Conclusion

Table of Contents

Large Language Models have revolutionized how we retrieve information or build search systems. Retrieval-augmented generation (RAG) methodology has become a common way to access or extract information. This guide teaches you how to build a Retrieval-Augmented Generation application using SurrealDB, Fireworks, FastAPI, and Astro. By the end of this guide, you will be able to update the chatbot’s knowledge visually and obtain the latest and personalized responses to your queries. Prerequisites

You'll need the following: • Node.js 18 or later • A Fireworks account

Tech Stack

The following technologies are used in creating our RAG application: Technology Type Description FastAPI Framework A high performance framework to build APIs with Python 3.8+. Astro Framework Framework for building fast, modern websites with serverless backend support. TailwindCSS Framework CSS framework for building custom designs. SurrealDB Platform A multi-model database platform. Fireworks Platform Lightning-fast Inference platform to run generative AI models.

High-Level Data Flow and Operations

This is a high-level architecture of how data is flowing and operations that take place 👇🏻

• When a user asks a question, relevant vectors to the latest user question are queried from SurrealDB. Further, they are combined with the user messages to create a system context. The response is then streamed to the user from Fireworks hosted Llama 3.1 405B Instruct Model. • When a user updates the existing knowledge other system, vector embeddings with metadata are created for the particular information, and then pushed to SurrealDB

Step 1: Setup SurrealDB Server

You can find various methods to install and run the SurrealDB server in the documentation . Let's opt for installing SurrealDB using its dedicated install script for our scenario. In your terminal window, execute the following command: 1 2 curl -- proto '=https' -- tlsv1 . 2 - sSf https : / / install . surrealdb . com | sh

The above command attempts to install the latest version of SurrealDB (per your platform and CPU type) into the /usr/local/bin  folder in your system. Once that is done, execute the following command in your terminal window: 1 2 surreal start -- log trace -- user root -- pass root -- bind 0.0 .0 .0 : 4304 file : mydatabase . db

The above command does the following: • Starts the SurrealDB server at 0.0.0.0:4304 network address. • Enables trace level logging producing verbose logs in your terminal window. • Sets the user and password of the default database as root . • Creates the file mydatabase.db to persist data on your filesystem.

Step 2: Generate Fireworks AI API Key

Model inference requests to the Fireworks API require an API Key. To generate this API key, log in to your Fireworks account and navigate to API Keys . Enter a name for your API key and click the  Create Key  button to generate a new API key. Copy and securely store this token for later use as  FIREWORKS_API_KEY  environment variable. Locally, set and export the  FIREWORKS_API_KEY  environment variable by executing the following command: 1 2 export FIREWORKS_API_KEY = ""

Step 3: Create a new FastAPI application

First, let's start by creating a new project. You can create a new directory by executing the following command in your terminal window: 1 2 3 4

Create and move to the new directory

mkdir chat-streaming cd chat-streaming

Install Dependencies

Next, you can install the required dependencies by executing the following command in your terminal window: 1 2 3 4 5 pip install surrealdb pip install fireworks-ai pip install langchain langchain-community langchain_fireworks pip install fastapi "uvicorn[standard]"

The above command installs the required libraries to run ASGI Server, FastAPI, Fireworks AI, SurrealDB and LangChain in your Python project. Next, create a file main.py with the following code: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 import uuid, os from typing import List

FastAPI

from fastapi import FastAPI from pydantic import BaseModel

Streaming Response utility

from fastapi.responses import StreamingResponse

Enable CORS utility

from fastapi.middleware.cors import CORSMiddleware

Fireworks SDK

import fireworks.client

SurrealDB Vector Store SDK for LangChain

from langchain_community.vectorstores import SurrealDBStore

Fireworks Embeddings Integration via LangChain

from langchain_fireworks import FireworksEmbeddings

The above code imports the following: • os module to use the environment variable you’ve set earlier. • List to denote a list of elements of specific type. • BaseModel class to define models of the request body FastAPI endpoints. • StreamingResponse class to generate streaming responses from FastAPI endpoints. • CORSMiddleware FastAPI middleware to enable Cross Origin Resource Sharing of FastAPI endpoints. • fireworks.client SDK for conveniently accessing Fireworks supported LLMs. • SurrealDBStore class by LangChain to use SurrealDB as vector store. • FireworksEmbeddings class via LangChain Fireworks integration to use Nomic AI Embeddings Model.

Define Data Models using Pydantic

To create the data types of request body in your FastAPI endpoints, append the following code in main.py file: 1 2 3 4 5 6 7 8 9 10 11 12 13

Class representing the string of messages to be searched and embedded as system context.

class LearningMessages(BaseModel): messages: str

Class...