friendliai/friendli-python

friendliai/friendli-python

Description: Friendli Suite python SDK

Language: Python

License: Apache-2.0

Stars: 1

Forks: 2

Open issues: 1

Created: 2024-09-03T05:29:17Z

Pushed: 2026-01-05T03:03:43Z

Default branch: main

Fork: no

Archived: no

README:

Friendli Python SDK

Supercharge Generative AI Serving with Friendli 🚀

Token Setup

When using Friendli Python SDK, you need to provide a Friendli Token for authentication and authorization purposes. A Friendli Token serves as an alternative method of authorization to signing in with an email and a password. You can generate a new Friendli Token through the Friendli Suite, at your "Personal settings" page by following the steps below.

1. Go to the Friendli Suite and sign in with your account. 2. Click the profile icon at the top-right corner of the page. 3. Click "Personal settings" menu. 4. Go to the "Tokens" tab on the navigation bar. 5. Create a new Friendli Token by clicking the "Create token" button. 6. Copy the token and save it in a safe place. You will not be able to see this token again once the page is refreshed.

Table of Contents

• [Friendli Python SDK](#friendli-python-sdk)
• [Token Setup](#token-setup)
• [SDK Installation](#sdk-installation)
• [SDK Example Usage](#sdk-example-usage)
• [Authentication](#authentication)
• [Available Resources and Operations](#available-resources-and-operations)
• [Server-sent event streaming](#server-sent-event-streaming)
• [File uploads](#file-uploads)
• [Retries](#retries)
• [Error Handling](#error-handling)
• [Server Selection](#server-selection)
• [Custom HTTP Client](#custom-http-client)
• [Resource Management](#resource-management)
• [Debugging](#debugging)
• [IDE Support](#ide-support)

SDK Installation

> [!NOTE] > Python version upgrade policy > > Once a Python version reaches its official end of life date, a 3-month grace period is provided for users to upgrade. Following this grace period, the minimum python version supported in the SDK will be updated.

The SDK can be installed with *uv*, *pip*, or *poetry* package managers.

uv

*uv* is a fast Python package installer and resolver, designed as a drop-in replacement for pip and pip-tools. It's recommended for its speed and modern Python tooling capabilities.

uv add friendli

PIP

*PIP* is the default package installer for Python, enabling easy installation and management of packages from PyPI via the command line.

pip install friendli

Poetry

*Poetry* is a modern tool that simplifies dependency management and package publishing by using a single pyproject.toml file to handle project metadata and dependencies.

poetry add friendli

Shell and script usage with uv

You can use this SDK in a Python shell with uv and the uvx command that comes with it like so:

uvx --from friendli python

It's also possible to write a standalone Python script without needing to set up a whole project like so:

#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.9"
# dependencies = [
# "friendli",
# ]
# ///

from friendli import SyncFriendli

sdk = SyncFriendli(
# SDK arguments
)

# Rest of script here...

Once that is saved to a file, you can run it with uv run script.py where script.py can be replaced with the actual file name.

SDK Example Usage

Chat completions

Given a list of messages forming a conversation, the model generates a response.

# Synchronous Example

import os

from friendli import SyncFriendli

with SyncFriendli(
token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
res = friendli.serverless.chat.complete(
model="meta-llama-3.1-8b-instruct",
messages=[
{
"role": "system",
"content": "You are a helpful assistant.",
},
{
"role": "user",
"content": "Hello!",
},
],
max_tokens=200,
)

# Handle response
print(res)

The same SDK client can also be used to make asynchronous requests by importing asyncio.

# Asynchronous Example
import asyncio
import os

from friendli import AsyncFriendli

async def main():
async with AsyncFriendli(
token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
res = await friendli.serverless.chat.complete(
model="meta-llama-3.1-8b-instruct",
messages=[
{
"role": "system",
"content": "You are a helpful assistant.",
},
{
"role": "user",
"content": "Hello!",
},
],
max_tokens=200,
)

# Handle response
print(res)

asyncio.run(main())

Tool assisted chat completions

Given a list of messages forming a conversation, the model generates a response. Additionally, the model can utilize built-in tools for tool calls, enhancing its capability to provide more comprehensive and actionable responses.

# Synchronous Example

import os

from friendli import SyncFriendli

with SyncFriendli(
token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
res = friendli.serverless.tool_assisted_chat.complete(
model="meta-llama-3.1-8b-instruct",
messages=[
{
"role": "user",
"content": "What is 3 + 6?",
},
],
max_tokens=200,
stream=False,
tools=[
{
"type": "math:calculator",
},
],
)

# Handle response
print(res)

The same SDK client can also be used to make asynchronous requests by importing asyncio.

# Asynchronous Example
import asyncio
import os

from friendli import AsyncFriendli

async def main():
async with AsyncFriendli(
token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
res = await friendli.serverless.tool_assisted_chat.complete(
model="meta-llama-3.1-8b-instruct",
messages=[
{
"role": "user",
"content": "What is 3 + 6?",
},
],
max_tokens=200,
stream=False,
tools=[
{
"type": "math:calculator",
},
],
)

# Handle response
print(res)

asyncio.run(main())

Authentication

Per-Client Security Schemes

This SDK supports the following security scheme globally:

| Name | Type | Scheme | Environment Variable | | ------- | ---- | ----------- | -------------------- | | token | http | HTTP Bearer | FRIENDLI_TOKEN |

To authenticate with the API the token parameter must be set when initializing the SDK client instance. For example:

import os

from friendli import SyncFriendli

with SyncFriendli(
token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
res = friendli.container.chat.complete(
messages=[
{
"role": "system",
"content": "You are a helpful assistant.",
},
{
"role": "user",…