deepinfra/deepctl

deepinfra/deepctl

Description: Command line tool for Deep Infra cloud ML inference service

Language: Rust

License: Apache-2.0

Stars: 36

Forks: 3

Open issues: 2

Created: 2022-12-07T22:25:03Z

Pushed: 2024-06-10T16:32:04Z

Default branch: main

Fork: no

Archived: no

README:

deepctl

Command line tool for Deep Infra cloud ML inference service.

Installation

Linux, MacOS

curl https://deepinfra.com/get.sh | sh

Download from Github

You can also download deepctl from Releases.

Usage

Sign in

You need a Github account to use deepctl. This command will open your Github Sign-on page in your browser. Follow instructions there.

deepctl auth login

Auth token

You can view your Deep Infra API auth token to use it in your applications using the following command or in the dashboard.

deepctl auth token

List supported models

deepctl model list

Create a model deployment

deepctl deploy create -m openai/whisper-small

Inference

To get information about model, it's inference arguments, response fields and ways to call endpoint, use the following command:

deepctl model info -m openai/whisper-small

Output:

model: openai/whisper-small
type: automatic-speech-recognition
CURL invocation:

curl -X POST \
-H "Authorization: bearer $AUTH_TOKEN" \
-F audio=@my_voice.mp3 \
'https://api.deepinfra.com/v1/inference/openai/whisper-small'

deepctl invocation:

deepctl infer \
-m 'openai/whisper-small' \
-i audio=@my_voice.mp3
...

When using HTTP inference api make sure to pass your AUTH_TOKEN header.

curl -X POST \
-H "Authorization: bearer $AUTH_TOKEN" \
-F audio=@/path/to/hello_world.mp3 \
'https://api.deepinfra.com/v1/inference/openai/whisper-small'

Output:

{
"text": "Hello World",
"segments": [
...
],
"language": "en"
}

List your deployments

deepctl deploy list

Output:

[
{
"created_at": "2023-01-26T19:33:23",
"deploy_id": "DpM4BkrjEspUwmTa",
"fail_reason": "",
"model_name": "openai/whisper-small",
"status": "running",
"task": "automatic-speech-recognition",
"updated_at": "2023-01-26T19:33:23"
}
]

Monitor Logs

Use deploy_id from the output above.

deepctl log query -f DpM4BkrjEspUwmTa

Delete deployment

Use deploy_id from the output above to delete the deployment.

deepctl deploy delete DpM4BkrjEspUwmTa

More information

You can always use

deepctl help

to view more information on any command.

Check version and update

deepctl version check
deepctl version update