Replies: 2 comments 6 replies
-
|
👋 Hello @mwelliott, thanks for sharing these detailed results and for using Ultralytics 🚀. This is an automated response to help you get support quickly — an Ultralytics engineer will also assist you here soon. We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered. If this is a 🐛 Bug Report regarding validation discrepancies, please provide a minimum reproducible example (MRE) to help us debug it. For this case, please include:
If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results. Join the Ultralytics community where it suits you best. For real-time chat, head to Discord 🎧. Prefer in-depth discussions? Check out Discourse. Or dive into threads on our Subreddit to share knowledge with the community. UpgradeUpgrade to the latest pip install -U ultralyticsEnvironmentsYOLO may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):
StatusIf this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLO Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit. |
Beta Was this translation helpful? Give feedback.
-
|
Can you post your training command and the exact name of the model you trained? |
Beta Was this translation helpful? Give feedback.
-
|
It's possible some results got corrupted during validation, so it resulted in lower mAP. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you I really appreciate it - would you say that it is generally good practice to always re-run validation manually at the end of training to ensure we get the right the proper validation results? |
Beta Was this translation helpful? Give feedback.
-
|
Good question—no, you shouldn’t need to re-run validation; the end-of-training val and a standalone from ultralytics import YOLO
model = YOLO("runs/detect/16_55k_cleaned_upper_body_garments_with_sleeves_train/weights/best.pt")
metrics = model.val() # reuses dataset/imgsz/rect from training
print(metrics.box.map, metrics.box.map50, metrics.box.map75)If numbers still diverge, please share the output of |
Beta Was this translation helpful? Give feedback.
-
|
Thank you gentlemen! I have a new training run in progress, so once this completes I will give your recommendations a shot. Also I had the python environment wrong, it was indeed 3.13 and not 3.11 so the environments were the same (literally ran the validation on the same box/environment after training completed). For my validation command I did not specify the batch size either, I just used: So I will simplify the validation command to what you recommended, ensure the environment is indeed the same, and we'll see how it goes! stay tuned, might be a couple days :) |
Beta Was this translation helpful? Give feedback.
-
|
Sounds good—one extra check while you’re at it: printing the checkpoint’s stored overrides will confirm you’re truly using the same from ultralytics import YOLO
m = YOLO("runs/detect/16_55k_cleaned_upper_body_garments_with_sleeves_train/weights/best.pt")
print({k: m.overrides.get(k) for k in ("data", "imgsz", "batch", "rect", "conf", "iou")})
metrics = m.val() # uses the saved training overrides; no extra kwargs
print(metrics.box.map, metrics.box.map50, metrics.box.map75)If results still diverge after matching those, share the new logs and we’ll dig deeper. For how overrides are applied and the available args, see the YOLO.val() reference and validation settings: YOLO.val() method reference and Validation settings. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Howdy! I just had a 200 epoch object detection training session complete - this consisted of around 55k images and 130k~ bounding boxes.
As you know, once training completes we get some validation metrics. Here are the last few lines of the training, showing the last 2 epochs and then the validation output:
I noticed that some of the classes had very low precision and map50-95 in the validation results, which doesn't seem to line up with the overall metrics we see via the individual epochs output. It also doesn't line up with real world usage, as these classes are very accurate (like left_chest and right_chest). So, I then ran the validation again manually using the same config, dataset, and environment immediately after the training completed. Essentially just immediately thought - hey, something is strange, let me run validation again. Here are the results of that:
Interestingly- now the precision, recall, and mAP numbers are a lot higher. And they seem to align much better with what the individual epoch runs were showing in their rolled up metrics. Is this a bug, or am I just not understanding how these validation metrics work after a training run completes? I'd love to know - thank you so much for any knowledge or insights! :)
Quick edit - I am using Ultralytics 8.3.179 and this is a yolo11x model. Python version is 3.11.9
Beta Was this translation helpful? Give feedback.
All reactions