replicate/cog-runtime

replicate/cog-runtime

Description: Cog runtime implementation

Language: Go

Stars: 5

Forks: 2

Open issues: 13

Created: 2024-11-15T00:12:14Z

Pushed: 2026-04-02T00:47:10Z

Default branch: main

Fork: no

Archived: yes

README:

Cog Runtime

COG RUNTIME IS BEING MERGED INTO MAINLINE COG, cog-runtime is being archived as we continue that work

Work will continue COG

Historical Information

New implementation for Cog's production runtime component, which is responsible for:

• Cog HTTP server
• Input and output schema validation
• Model execution

Cog HTTP server is rewritten in Go for better performance, reliability, concurrency and isolation.

Schema validation and model execution were rewritten in pure Python for simplicity, better error handling and reduced risk of dependency conflicts.

Cog HTTP server

This is the Go HTTP server that:

• Manages the Python model runner process
• Handles HTTP requests
• Downloads input files and uploads output files
• Manages async and concurrency predictions
• Makes webhook callbacks
• Logging and health check of Python runner process
• Communicates with the Python runner via a mix of Unix signals, HTTP, and JSON files

coglet

Python model runner that:

• Is source compatible with existing Cog API
• Has zero Python dependency to minimize risk of interfering with model code
• Inspects Python predictor code for input and output schema
• Invokes setup() and predict() methods