microsoft/superbenchmark v0.11.0

Release SuperBench v0.11.0

Repository: microsoft/superbenchmark

Tag: v0.11.0

Published: 2024-10-08T06:18:01Z

Prerelease: no

Release notes:

SuperBench 0.11.0 Release Notes

SuperBench Improvements

• Add CUDA 12.4 dockerfile.
• Upgrade nccl version to v2.23.4 and install ucx v1.16.0 in cuda 12.4 dockefile.
• Fix MSCCL build error in CUDA12.4 docker build pipeline.
• Add ROCm6.2 dockerfile.
• Update hpcx link in cuda11.1 dockerfile to fix docker build failure.
• Improve document (Fix metrics name and typos in user tutorial, add BibTeX in README and repo).
• Limit protobuf version to be 3.20.x to fix onnxruntime dependency error.
• Update omegaconf version to 2.3.0 and fix issues caused by omegaconf version update.
• Fix MSCCL build error in CUDA12.4 docker build pipeline.
• Update Docker Exec Command for Persistent HPCX Environment.
• Fix cuda 12.2 dockerfile LD_LIBRARY_PATH issue.
• Use types-setuptools to replace types-pkg_resources.
• Add configuration for NDv5 H100 and AMD MI300x.

Micro-benchmark Improvements

• Add hipblasLt tuning to dist-inference cpp implementation.
• Add support for NVIDIA L4/L40/L40s GPUs in gemm-flops.
• Upgrade mlc to v3.11.

Model-benchmark Improvements

• Support FP8 transformer model training in ROCm6.2 dockerfile.

Result Analysis

• Fix bug of failure test and warning of pandas in data diagnosis.