NVIDIA/garak v0.14.0

v0.14.0

Repository: NVIDIA/garak

Tag: v0.14.0

Published: 2026-02-04T17:31:04Z

Prerelease: no

Release notes:

What's Changed

New features

• feature: support JSON config as well as YAML by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1490
• feature: remove --generate_autodan CLI option by @sinhaabhiraj0 in https://github.com/NVIDIA/garak/pull/1521
• expand items in report entry_type:eval by @leondz in https://github.com/NVIDIA/garak/pull/1547
• reporting: tier biased security aggregate by @leondz in https://github.com/NVIDIA/garak/pull/1329
• feature: Report Generator refactor by @otavionvidia in https://github.com/NVIDIA/garak/pull/1573
• feature: Add detector evaluation doc and benchmark results by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1597
• task: calibration data updates winter 2026 by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1599

Notice: Breaking changes in this release

• Modified format of eval and digest entires in JSONL report

See "Run analysis" documentation for more details of compatibility expectations.

• Fully redesigned HTML reports
• Removal of --generate_autodan cli option

Improved plugins

• Fix atkgen verbose output displaying incorrect conversation turns by @snehalvartak in https://github.com/NVIDIA/garak/pull/1542
• generator: relax constraint to specify valid openai model name by @leondz in https://github.com/NVIDIA/garak/pull/1557
• generators: check message count requested matches message count generated by @leondz in https://github.com/NVIDIA/garak/pull/1558
• fix: JavaScript detector regex improvement by @erickgalinkin in https://github.com/NVIDIA/garak/pull/1588

Documentation

• docs: add guardion ai / promptintel to project by @leondz in https://github.com/NVIDIA/garak/pull/1530
• docs: add brief stubs for garak.analyze content by @leondz in https://github.com/NVIDIA/garak/pull/1569

Tuning & fixes

• Make GoogleTranslator multiprocessing-safe by @motlaharsh0909-lgtm in https://github.com/NVIDIA/garak/pull/1543
• Fix: improved unsafe pickle strategy by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1545
• track which detector is reporting invalid pkg entry; only report once by @leondz in https://github.com/NVIDIA/garak/pull/1556
• arch: Remove many builtin configs by @leondz in https://github.com/NVIDIA/garak/pull/1555
• update: rm deprecated gpt-3.5 model from defaults by @leondz in https://github.com/NVIDIA/garak/pull/1552
• task: update openai library to 2.x by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1550
• test: isolate _config in all tests by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1549
• update LangChain generator for 1.x library support by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1551
• add license to docs footer by @leondz in https://github.com/NVIDIA/garak/pull/1572
• guard for None content as response from target by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1574
• fix: 100% pass rate should not give 'elevated risk' for relative scores by @leondz in https://github.com/NVIDIA/garak/pull/1579
• Move generation output cardinality check into Generator.generate() by @leondz in https://github.com/NVIDIA/garak/pull/1578
• calling as_dict() on default instantiated Attempt now doesn't raise TypeError by @leondz in https://github.com/NVIDIA/garak/pull/1590
• task: remove fschat by @erickgalinkin in https://github.com/NVIDIA/garak/pull/1567
• task: report test tweaks and dependency lock updates by @otavionvidia in https://github.com/NVIDIA/garak/pull/1593
• Task: transformers 5 support by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1582
• task: exclude files not needed in distributed package by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1601
• reporting: factor up defcon calculation by @leondz in https://github.com/NVIDIA/garak/pull/1581
• rm jinja2 direct dependency requirement by @leondz in https://github.com/NVIDIA/garak/pull/1596
• task: update huggingface config pattern by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1602
• reporting: stabilise tbsa hashes by @leondz in https://github.com/NVIDIA/garak/pull/1603

New Contributors

• @snehalvartak made their first contribution in https://github.com/NVIDIA/garak/pull/1542
• @sinhaabhiraj0 made their first contribution in https://github.com/NVIDIA/garak/pull/1521
• @motlaharsh0909-lgtm made their first contribution in https://github.com/NVIDIA/garak/pull/1543
• @otavionvidia made their first contribution in https://github.com/NVIDIA/garak/pull/1573

Full Changelog: https://github.com/NVIDIA/garak/compare/v0.13.3...v0.14.0