Skip to content

Validate ClassLabel values during table casts - #8631

Open
Anakintano wants to merge 1 commit into
huggingface:mainfrom
Anakintano:fix/classlabel-ingestion-validation
Open

Anakintano wants to merge 1 commit into
huggingface:mainfrom
Anakintano:fix/classlabel-ingestion-validation

Conversation

@Anakintano

Copy link
Copy Markdown

Fixes #8596.

Validate ClassLabel values during table casts while keeping the metadata-only path for other features.

Comment thread src/datasets/table.py
return contains_class_label(feature.feature)
return False

features = Features.from_arrow_schema(schema)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validation looks right, and I could not break the recursion: [ClassLabel(...)], Sequence, and a ClassLabel inside a nested dict all normalize to List/dict before they get here, so all three raise as intended at 03482510.

The cost lands on the branch this else used to be, though. Features.from_arrow_schema(schema) runs on every call where the schemas are already equal and the metadata carries huggingface, including datasets with no ClassLabel anywhere, and it rebuilds the features' arrow schema to compare field by field, so it scales with column count. Clean python:3.12-slim container, pip install -e ., same table and same script at 03482510 and at merge base d336dcb8:

                              base d336dcb8     head 03482510
 1 column,  no ClassLabel          1.5 us/call      80.1 us/call
20 columns, no ClassLabel          1.4 us/call    1730.1 us/call

Base is flat in column count; head is not. This is per batch, not per dataset: table_cast is what _cast_table calls in every packaged module, plus arrow_writer.py:795 and iterable_dataset.py:125.

A substring test on the raw metadata before line 2398 keeps the fast path and cannot produce a false negative, since _type is serialized as the literal ClassLabel. Measured on the same schemas at 1.00 and 2.61 us/call:

if b"ClassLabel" not in schema.metadata[b"huggingface"]:
    if table.schema.metadata != schema.metadata:
        return table.replace_schema_metadata(schema.metadata)
    return table

Checked against the four cases above: True for ClassLabel, List(ClassLabel) and the nested dict, False when there is none.

Not checked: I timed table_cast directly rather than a full load_dataset, so I have not put a number on the end to end effect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

load_dataset silently accepts out-of-range ClassLabel indices when the source dtype already matches

2 participants