Real3D-AD reproduction: corrected AU-PRO is much lower than reported, and release protocol seems inconsistent #4

Hi, thanks for releasing the BTP code.

I tried to reproduce the Real3D-AD zero-shot results using the released code and found several issues that may need clarification.

Environment / setup

• Dataset: Real3D-AD
• Device: GPU 4090 24GB
• Input points: 2048, FPS preprocessing with utils/processing_real3d.py
• Encoder: ULIP-2-PointBERT-10k-xyzrgb-pc-vit_g-objaverse_shapenet-pretrained.pt
• Training split: test, as shown in the README
• Epochs: 100
• Seed: 111
• Evaluation: test_standard_aupro.py
• Protocol: one-class source training, evaluate on other categories

Main reproduction result

When training all 12 Real3D-AD source categories and averaging over the 132 source-target pairs, I obtained:

The object-level metrics are close to the paper/README range, but the corrected standard AU-PRO is very low compared with the paper-reported 81.9.

AU-PRO issue

The README says that the original AU-PRO implementation was corrected and that test_standard_aupro.py should be used for standard AU-PRO reporting.

However, with test_standard_aupro.py, the reproduced Real3D-AD AU-PRO is only:

• 50.15 on sampled 2048-point evaluation
• 48.62 on full-resolution evaluation

For comparison, evaluating the same checkpoints with legacy test.py gives much higher values:

This suggests that the paper-reported 81.9 may correspond to the old point-coverage-style metric rather than the corrected standard region-level AU-PRO. Could you clarify which AU-PRO implementation was used for the updated arXiv/table results?

Protocol inconsistency

The paper describes training on each Real3D-AD category and evaluating on the remaining categories. Table 2 also lists all 12 source categories.

However, the released run_train_test.sh only uses:

REAL3D_CLASSES=(car chicken gemstone)

When I aggregate only these three source classes, the result becomes:

The 3-source setting is much closer to the paper P-AUROC/P-AP, but the standard AU-PRO is still far below 81.9. Could you clarify whether the reported Real3D-AD results use all 12 source categories or only selected source categories?

Possible prompt-learning issue in released code
I also noticed that the released training/evaluation path appears to ignore the learnable prompt embeddings:

tokenized_prompts = prompt_learner()
text_normal_embed, text_anomaly_embed = encoder.encode_text_from_tokens(tokenized_prompts)

This seems to encode only token IDs and does not pass the learnable prompt embeddings through the text tower. I changed this locally to encode the actual prompt embeddings, and verified that prompt_learner.ctx receives gradients. Could you confirm whether this is intended, or whether the released code is missing the prompt-embedding path used in the paper?

Questions：
Which AU-PRO implementation should reproduce the Real3D-AD 81.9 result?
Were the reported Real3D-AD results computed with all 12 source categories or only selected source categories?
Is the released prompt-learning path correct, or should the learnable prompt embeddings be passed into the text encoder?

Thanks!