google-deepmind/clrs v2.0.1

v2.0.1 CLRS-Text Algorithmic Reasoning Language Benchmark

Repository: google-deepmind/clrs

Tag: v2.0.1

Published: 2024-07-18T19:03:48Z

Prerelease: no

Release notes:

Main Changes

Added the CLRS-Text Algorithmic Reasoning Language Benchmark to the codebase

CLRS-Text is a textual version of the traces generated by thirty algorithms selected from the third edition of the standard "Introduction to Algorithms" textbook by Cormen, Leiserson, Rivest, and Stein. It serves to consolidate and unify previous lines of research in this direction and offers a robust test bed for evaluating language models' out-of-distribution reasoning capabilities.

CLRS-Text is Hugging Face compatible and can generate data in many formats, such as JSON.

For more details, refer to the [README.md].

Base CLRS script update Revamped the base script incorporating modeling and data improvements from the A Generalist Neural Algorithmic Learner paper.

What's Changed

• Add ml_collections to requirements.txt. by @copybara-service in https://github.com/google-deepmind/clrs/pull/139
• Added script to generate json for all algorithms. by @copybara-service in https://github.com/google-deepmind/clrs/pull/140
• Adding huggingface generators for clrs text by @mcleish7 in https://github.com/google-deepmind/clrs/pull/135
• Add clrs_text to __init__.py by @copybara-service in https://github.com/google-deepmind/clrs/pull/144
• Roll forward PR #104 by @RerRayne in https://github.com/google-deepmind/clrs/pull/146
• Add CLRS-Text details in the README files. by @copybara-service in https://github.com/google-deepmind/clrs/pull/148

New Contributors

• @sigeisler made their first contribution in https://github.com/google-deepmind/clrs/pull/104
• @mcleish7 made their first contribution in https://github.com/google-deepmind/clrs/pull/135
• @RerRayne made their first contribution in https://github.com/google-deepmind/clrs/pull/146
• @gurux13 made their first contribution in https://github.com/google-deepmind/clrs/pull/147

Full Changelog: https://github.com/google-deepmind/clrs/compare/v1.0.0...v2.0.0