NousResearch/kaida

NousResearch/kaida

Description: A prototype Kotlin LLM library.

Language: Kotlin

License: MIT

Stars: 9

Forks: 3

Open issues: 0

Created: 2025-03-18T21:19:02Z

Pushed: 2025-03-28T23:40:18Z

Default branch: main

Fork: no

Archived: no

README:

Kaida

Kaida is a prototype Kotlin library designed to simplify and standardize interactions with multiple large language model (LLM) APIs including OpenAI-compatible completion and chat endpoints as well as Anthropic. It also features a strongly typed Directed Acyclic Graph (DAG) pipeline for structuring complex multi-step workflows.

Designed for extremely rapid iteration, Kaida prioritizes:

• Unified LLM abstractions that fully model the union set of all supported features using sum types.
• Swap models (e.g. chat to completion, different APIs, sampler settings) without changing your application code at all.
• Get compile time errors or fail fast at runtime - never silently do the wrong thing.
• Write flexible code that can account for a variety of different features.
• Fully asynchronous streaming API built on Kotlin's asynchronous flows that supports canceling requests and parallelism
• Automatic serialization of all intermediate states as well as outputs using kotlinx.serialization
• Observable and debuggable: a pipeline's internal state is easily inspected at any point during execution
• Decoupled design that lends itself well to dynamically driven UI
• Every level of tool: low level LLM completion API, high level template and pipeline API built on top

This document introduces Kaida incrementally through practical examples.

Table of Contents

• [Installation](#installation)
• [Configuration](#configuration)
• [Example Project](#example-project)
• [Motivation: Why Use Kaida?](#motivation-why-use-kaida)
• [Typed Feature Flags and Fail-Fast](#typed-feature-flags-and-fail-fast)
• [Templates for Structured Prompts](#templates-for-structured-prompts)
• [Building Complex Workflows: The Pipeline DSL](#building-complex-workflows-the-pipeline-dsl)
• [Retry Policies](#retry-policies)
• [Serialization and Reloading Pipelines](#serialization-and-reloading-pipelines)
• [Simplifying Serialization](#simplifying-serialization)
• [Contributing](#contributing)
• [License](#license)

Installation

Using Gradle, add:

dependencies {
implementation("org.nous:kaida:")
}

Replace `` with the current release version.

Configuration

Kaida uses YAML to configure model details and authentication separately:

`config/auth.yaml`

- name: "openai"
auth:
type: "api_key"
api_key: "your_api_key_here"
- name: "anthropic"
auth:
type: "api_key"
api_key: "your_api_key_here"

`config/models.yaml`

- name: "gpt4o-mini"
auth: "openai"
model:
type: "openai"
model: "gpt-4o-mini"
- name: "claude3-opus"
auth: "anthropic"
model:
type: "anthropic"
model: "claude-3-opus-20240229"
temperature: 1.2
top_p: 0.95

Example Project

If you want to get a quick start, you can find a practical example using Kaida here!

Motivation: Why Use Kaida?

Interacting directly with multiple LLM providers often results in fragmented and redundant code. Each API provider tends to have distinct capabilities, request structures, and behaviors. Kaida eliminates this complexity by providing a single consistent interface:

val model = ModelRegistry.get("claude3-opus")!!

val request = model.multiTurn(
listOf(
Message("hi what's up"),
Message("the sky????", MessageRole.Assistant),
),
)

request.collect {
when(it) {
is TextCompletion -> print(it)
else -> TODO()
}
}

This uniformity simplifies integrating new models or changing providers without refactoring business logic.

Typed Feature Flags and Fail-Fast

Different models have varying capabilities (e.g. logprobs, system prompts, logit bias, etc). Kaida makes such features explicit:

val model = ModelRegistry.get("claude3-opus")!!

// note: only a chat completion endpoint will accept multiple messages
val chat = listOf(
Message("hi what's up"),
Message("the sky???", MessageRole.Assistant),
)

// this request wants logprobs, and we have a system prompt
val inferenceParams = InferenceParameters(
logProbs=true,
// we also can indicate here we'd prefer that this system prompt be cached if caching is available/supported
systemPrompt=listOf(TextBlock("whatever", cacheHint=CacheHint.Cache))
)

/*
we infer that logprobs and system prompt are required for this request,
which means we would throw if either was unsupported by the underlying model
*/
val response = model.multiTurn(chat, inferenceParams) {
// however, we're actually okay with text completion, we'd just prefer logprobs if it's supported
logProbs(InferenceFlag.Preferred)
}

/*
now we consume the cold flow (which begins the request)

we could get either logprob completions or text completions
here depending on whether the underlying model supports logprobs
*/
response.collect { result ->
when (result) {
is LogProbCompletion -> {
result.tokens.forEach { token ->
val probability = "%.2f".format(token.selected.logprob)
val tokenText = token.selected.token
val options = token.options.joinToString(", ") { it.token.replace("\n", "\\n") }

println("$probability ${tokenText.padEnd(20)} [$options]")
}
}
is TextCompletion -> println(result.content)
}
}

If the requested model doesn't support a required feature, Kaida immediately throws an error preventing unexpected runtime behavior.

Templates for Structured Prompts

Kaida provides a straightforward template system:

Please convert the following text to Shakespearean prose, preserving the emotional tone, theme, narrative, prose as closely as possible.

Do NOT include any other text besides the prose, which should be placed within a code block.


$TEXT

val model = ModelRegistry.get("claude3-opus")!!

val completion = model.templateStream(Path("instruct", "shakespeare").toString()) {
variable("TEXT", """
Hi Bob,

I hope you're doing well! It's been a while since our last conversation, and I wanted to touch base. I've been reflecting on our recent project discussions, and I have some new ideas that I think could really take things to the next level.

Best regards,
Alice
""".trimIndent().trim())
}

// use a helper to fetch the contents as a string, regardless of completion type
println(completion.asString())

// OR collect the completion stream…