NVIDIA/GUSLI

NVIDIA/GUSLI

Description: User space Block device access API

Language: C++

License: Apache-2.0

Stars: 13

Forks: 8

Open issues: 1

Created: 2025-09-08T16:16:42Z

Pushed: 2026-08-25T11:27:25Z

Default branch: main

Fork: no

Archived: no

README:

Gusli

Description

Gusli (G3+ User Space Access Library) is a C++ library encapsulating IO (Read/Write) to any local block device (or a file). It has a client library (.so) for IO submitting apps and server library (.so) for execution of the io's via backend

• Both client and server processes run on the same machine in user space. No Linux Kernel components involved.
• Zero copy kernel bypass io, using shared memory between processes
• 1 client connects to multiple servers. Each server represents a block device.
• Supported servers: SPDK based apps with standard bdevs, Any other process based on server library examples.
• IO (read/Write) can be blocking or asynchronous. Async-io can use callback functions and/or be polled for completion and/or canceled.
• The reason for separation of client and server is to avoid compile/link dependency between io submitting app and block device backend. Example: Backend might be spdk block device but client application does not have to be spdk application.
• Client library ensures io is valid & properly throttled before issuing it to the server

Additional Documentation and description: Here

Installation & Usage

make all

• For production make clean all BUILD_RELEASE=1 BUILD_FOR_UNITEST=0 VERBOSE=1 ALLOW_USE_URING=0; ls /usr/lib/libg*; ls /usr/include/gus*;
• executables of unit-tests will be created in project root dir (same as makefile)
• use the command above to build and install production libraries into /usr (/usr/lib, /usr/include)
• make file does installation of binaries, see its output:
• Deliverables: .so/.hpp files pair for server side and .so/.hpp files pair for client
• More info: type make help To see more info of how to compile
• Compilation: Both g++ and clang++ are supported. clang example: make USE_CLANG=1 all

Dependencies

• Mandatory: None. Plain C++ code (libc) code.
• Third party components are not auto downloaded/installed. Please do the following:

| OS | Install | Remarks | |---------|-----------------------------------------|----------------| | Ubuntu | apt install g++ make cmake pkg-config | ✅ | | Fedora | dnf install gcc-c++ cmake pkg-config | 🚧 Not tested |

• Optional:
• For IO's with large scatter gathers (> 64 ranges), uring API to local block devices may help. So consider: sudo apt install liburing-dev
• For developers: useful utils: sudo apt install -y build-essential gdb meld ncdu tree valgrind
• You can see the actual dependencies of this dynamic library by doing ldd /usr/lib/libgusli_*.so

Monitor IO environment using Gusli

Bash script which allows you to monitor the io buffers, stacks, threads, etc of both client and server processes, as well as force kill them. 1. source gusli/80scripts/service.sh; 2. GUSLI; To get help about about monitoring client and server

Examples (C++):

1. Client: [write-read-verify io](./07examples/client/sample_client.hpp)

• function which connects to Gusli server issues some basic ios to it
• class unitest_io shows how io can be launched in many modes (with/without async callback, with/without polling, blocking mode), with verification.

2. Server: [A few server classes](./07examples/server/) like ram based block device, always-fail-io, spdk-bdev, etc 3. Testing executables:

• SPDK Both: Executable which forks itself, runs spdk server in 1 process, client basic io tests in another process [spdk_app](./unitest/sample_app_spdk_both.cpp)
• ./unitest/run for running various Gusli unit-tests

4. Gusli clients connect to servers via a config file. Example of config code can be found in a function client_simple_test_of_server() 5. See unit-test directory to learn more how to use the library and how it is tests. 6. How to link? See examples above

How to integrate with Gusli

1. First decide what does your app need. Serve io (like user space block device) or Issue io (then you are a client). 2. Use the supplied examples above to start from sample server/client application and extend as needed. 3. Application can be both server and client, this works but not recommended. See examples of self forking app. 4. If your app is a issuing io to standard linux block devices, then you can use only Gusli client, without server. 5. If your app is spdk based block device do the following steps: Download spdk, and install it. Compile Gusli example server app which loads Gusli server. Change spdk configuration file to match your block device 6. If your application is for production, please make sure you have a monitoring service/script which coordinates client/server apps

• Generates proper configuration which block devices should be visible to client via servers
• Restart server, client after crash
• Note: Gusli is just 2 libraries, not services.

Linkage type

• We strongly suggest to use dynamic linking to be able to enjoy latest Gusli features without recompiling your executable.
• Static linking is generally used for debugging and unit-testing only.
• You can use static linking (server/client) *.a files or dynamic linking *.so files.

Integrations with other open source projects

• Integrating client with NIXL, see [install script](./80scripts/integration_with_nixl.sh)
• Integrating server with SPDK, see [install script](./80scripts/integration_with_spdkbdev.sh)

> ⚠️ Gusli team assisted with the above integrations for testing purposes. It is not responsible for it and integration can break if any of the above open source projects introduce breaking changes.

Performance

Concurrency

• Gusli Server is single threaded (1 cpu core for each block device). This is by design as multiple block devices may be used.
• This thread receives client io, submits it to backend and sends completions to client
• Gusli client allocates 1 thread for for each block device.
• This thead submits io to server and listens for completions from server
• You can submit and wait for IO from multiple threads. For more info read code documentation of client api header file

Numbers

• Client is able to push about 1.5 - 2.5 Million ios per second...