Skip to content

fix: respect ratio_list sampling in MultiScaleDataSet - #18345

Open
lzy494177-byte wants to merge 1 commit into
PaddlePaddle:mainfrom
lzy494177-byte:fix/multiscale-ratio-sampling
Open

lzy494177-byte wants to merge 1 commit into
PaddlePaddle:mainfrom
lzy494177-byte:fix/multiscale-ratio-sampling

Conversation

@lzy494177-byte

Copy link
Copy Markdown

Problem

MultiScaleDataSet does not correctly respect the _index_map
generated by SimpleDataSet when ratio_list contains values below 1.0.

For ds_width=False, sample indices are resolved through
data_idx_order_list instead of _index_map.

For ds_width=True, wh_ratio_sort is built from the full dataset
instead of the ratio-sampled subset.

After reset_data_lines() changes the sampled subset, the cached
width-aware state in MultiScaleSampler can also become stale.

Example

Given two label files:

  • Dataset A: 100 samples
  • Dataset B: 100 samples
  • ratio_list = [0.5, 0.5]

Expected:

  • 50 samples from A
  • 50 samples from B

Although _index_map contains the correct sampled indices,
MultiScaleDataSet may access samples outside that subset.

For width-separated samples, this can result in accessing
100 samples from A and 0 samples from B.

Fix

  • Resolve MultiScaleDataSet indices through _index_map.
  • Build width-aware ordering from the current sampled subset.
  • Refresh width-aware ordering after ratio resampling.
  • Refresh cached MultiScaleSampler batches after dataset reset.
  • Add regression tests for both ds_width=False and ds_width=True,
    including epoch reset behavior.

Tests

python -m pytest -q tests/ppocr/test_simple_dataset.py

Result:

3 passed

All modified files also pass pre-commit.

@CLAassistant

CLAassistant commented Sep 4, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@lzy494177-byte
lzy494177-byte force-pushed the fix/multiscale-ratio-sampling branch from f58d73a to 47f5175 Compare September 4, 2026 16:55
@lzy494177-byte
lzy494177-byte force-pushed the fix/multiscale-ratio-sampling branch from 47f5175 to dcc033d Compare September 4, 2026 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants