Benchmark dataset
GS-QA2 questions
3,300 questions: 2,800 vector-only and 500 raster-only or raster–vector. Pin the dataset commit used for every run.
Open dataset →GS-QA Leaderboard
Transparent evaluation across 3,300 questions and 53 vector, raster, and raster-vector task templates.
Published benchmark
Zhuocheng Shang, Shahd Elmahallawy, Zabir Al Nazi, Vagelis Hristidis, and Ahmed Eldawy. 2026. GS-QA2: A Benchmark for Question Answering over Raster–Vector Data. In Proceedings of the 34th ACM International Conference on Advances in Geographic Information Systems (SIGSPATIAL '26). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3841645.3843431
@inproceedings{shang2026gsqa2,
author = {Zhuocheng Shang and Shahd Elmahallawy and Zabir Al Nazi and
Vagelis Hristidis and Ahmed Eldawy},
title = {{GS-QA2}: A Benchmark for Question Answering over Raster--Vector Data},
booktitle = {Proceedings of the 34th ACM International Conference on Advances
in Geographic Information Systems},
series = {SIGSPATIAL '26},
year = {2026},
month = nov,
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
doi = {10.1145/3841645.3843431},
isbn = {979-8-4007-2950-8}
}
Data and artifacts
Questions live in a versioned Hugging Face dataset. The leaderboard's paper-reported result rows are available here as JSON and CSV.
Benchmark dataset
3,300 questions: 2,800 vector-only and 500 raster-only or raster–vector. Pin the dataset commit used for every run.
Open dataset →Evaluation tracks
V1–V28 are vector-only, R1–R11 are raster-only, and VR1–VR14 combine vector and raster reasoning.
Explore templates →Leaderboard artifacts
Reuse the normalized, paper-reported scores that power the tables on this page.
Paper-reported evaluation
Select a vector answer type or a raster map-algebra class. Each group is ranked only with its own scientifically appropriate metric.
| Rank | Model / method | Verification | Benchmark questions | Metric | Score |
|---|
Inspect the benchmark
Compare vector-only (V1–V28), raster-only (R1–R11), and raster–vector (VR1–VR14) templates on the same questions.
| Model / method | Template ID | Track and template | Benchmark questions | Primary metric | Score |
|---|
No results match the selected filters.
Evaluation request
Complete one form for a self-service result or a maintainer-managed database, raster, or full evaluation.
Required information
Required fields are marked with *. Do not include API keys or gated-model tokens.