Benchmarking the benchmarking models

G Anand, R Kodali - Benchmarking: An international journal, 2008 - emerald.com
… scheme of benchmarking and thereby the unique benchmarking models that are … for
each type of benchmarking. Further it aims to propose a universal benchmarking model, …

Superb: Speech processing universal performance benchmark

S Yang, PH Chi, YS Chuang, CIJ Lai… - arXiv preprint arXiv …, 2021 - arxiv.org
Universal PERformance Benchmark We establish and release Speech processing Universal
PERformance Benchmark … labeled data to effectively benchmark the generalizability of …

Audiobench: A universal benchmark for audio large language models

B Wang, X Zou, G Lin, S Sun, Z Liu, W Zhang… - arXiv preprint arXiv …, 2024 - arxiv.org
We introduce AudioBench, a universal benchmark designed to evaluate Audio Large Language
Models (AudioLLMs). It encompasses 8 distinct tasks and 26 datasets, among which, 7 …

A universal protocol to benchmark camera calibration for sports

F Magera, T Hoyoux, O Barnich… - Proceedings of the …, 2024 - openaccess.thecvf.com
… , we designed a new benchmarking protocol, … benchmarking protocol provides fairer
evaluations of camera calibration methods. By defining our requirements for proper benchmarking, …

Unicorn on rainbow: A universal commonsense reasoning model on a new multitask benchmark

N Lourie, R Le Bras, C Bhagavatula… - Proceedings of the AAAI …, 2021 - ojs.aaai.org
… , recently introduced benchmarks. First, we propose a new multitask benchmark, RAINBOW,
… Last but not least, we introduce a new universal commonsense reasoning model, UNICORN…

Wgb: Towards a universal graph benchmark

K Ammar, MT Özsu - … Series on Big Data Benchmarking, WBDB. cn, Xi'an …, 2014 - Springer
… Our benchmark is a domain specific big data benchmark focusing on graph data only. We
need a flexible generator because we aim to assess graph systems using all potential graph …

Marble: Music audio representation benchmark for universal evaluation

R Yuan, Y Ma, Y Li, G Zhang, X Chen… - Advances in …, 2023 - proceedings.neurips.cc
… , and the absence of a universal and community-driven benchmark. To address this issue, …
Benchmark for universaL Evaluation, termed MARBLE. It aims to provide a benchmark for …

UniMod1K: Towards a More Universal Large-Scale Dataset and Benchmark for Multi-modal Learning

XF Zhu, T Xu, Z Liu, Z Tang, XJ Wu, J Kittler - International Journal of …, 2024 - Springer
The emergence of large-scale high-quality datasets has stimulated the rapid development
of deep learning in recent years. However, most computer vision tasks focus on the visual …

A universal benchmarking method for probabilistic solar irradiance forecasting

D Yang - Solar Energy, 2019 - Elsevier
… Firstly, it is desirable for the universal benchmarking method to have the highest reliability.
To ensure a complete coverage, the natural bounds, ± ∞ , can be employed. However, such …

Light field salient object detection: A review and benchmark

K Fu, Y Jiang, GP Ji, T Zhou, Q Zhao… - Computational Visual …, 2022 - Springer
… comprehensive review and a benchmark for light field SOD, … Secondly, we benchmark nine
representative light field SOD … Our supplemental data make a universal benchmark possible…