togethercomputer/SMiR

togethercomputer/SMiR

Description: synthetic data pipeline for multi-image reasoning

Language: Python

License: Apache-2.0

Stars: 3

Forks: 0

Open issues: 0

Created: 2024-10-16T22:18:48Z

Pushed: 2025-03-04T21:45:48Z

Default branch: main

Fork: no

Archived: no

README:

SMiR

Synthetic data pipeline for multi-image reasoning

Overview

This repository contains the official implementation of our paper: Efficient Synthetic Data Pipeline to Improve Multi-Image Reasoning.

🏆 Credits

We would like to acknowledge the following resources that were instrumental in the development of SMIR:

• Meta Llama 3.1: We utilized the Llama 3.1 model as our foundational language model via "Together AI".

• SigLIP: We utilized a SigLIP model as our embedding model from Google.

• CLIP: We utilized MetaCLIP, Meta's implementation of CLIP, as our embedding model.

• We used training and evaluation code from the following repositories:
• MANTIS: Interleaved Multi-Image Instruction Tuning
• From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline

📚 BibTeX

@misc{li2025smirefficientsyntheticdata,
title={SMIR: Efficient Synthetic Data Pipeline To Improve Multi-Image Reasoning},
author={Andrew Li and Rahul Thapa and Rahul Chalamala and Qingyang Wu and Kezhen Chen and James Zou},
year={2025},
eprint={2501.03675},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2501.03675},
}