Thanks for this interesting initiative. Potentially it can contribute in harmonizing tools.

We'll first look into the ID topic. We understand any ID shall be globally unique, across categories of simulation entities. So far esmini have reused any specified ID in the OpenDRIVE objects, e.g. stationary objects. This traceability from OSI to scenario components is important. We'll now look into whether we can utilize ExternalReference to achieve this instead.

@thomassedlmayer We have an experimental branch which aims to address the issue of overlapping ID's in OSI. Would it be possible for you to verify that branch's functionality against the ASAM OSI Quality Checker? It would be great to get your feedback in case there's still some issues.

The branch in question: feature/global_osi_ids

We have now made a release 2.57.0 which hopefully complies with ASAM OSI Quality Checker. This is a first step for this PR.

Hi @slundel6 and @eknabevcc,
thank you for integrating this so quickly! The new id setup appears to work great for the minimal example of our current test case. I don't see any remaining overlaps between lanes/lane boundaries and moving objects. I didn't test it with more complex scenarios yet but I will have a more detailed look soon.

Though, you probably noticed that the validation step still fails. There are four remaining issues of which 3 will be fixed by some updates on the side of osc-validation. I will update this PR once this is solved on our side.
Optional explanation: During initial development, osc-validation only supported OSI SensorView as input format, so we had to build a workaround to convert ESMini's GroundTruth output to SensorView. This currently generates some unnecessary QC issues because of unset fields. We will refactor the ESMini integration in osc-validation so that it uses the direct output of ESMini as osc-validation now supports GroundTruth as input as well. Then these issues should be gone.

But there is one remaining issue which I unfortunately overlooked to mention in my initial request: OSI's rules require the host_vehicle_id to be set in the GroundTruth message (see OSI docs). ESMini currently does not set the host vehicle id in the GroundTruth message even though one entity in the scenario file is called "Ego". Would this be something that could be adapted, too?

Once this is solved, this should allow more focus on actually validating the OpenSCENARIO part of ESMini (which is the actual goal of osc-validation!

Thank you again for your cooperation and your help regarding the OSI output of ESMini!

We added the host_vehicle_id (seems like a great thing to have available!) to a patch-release v2.57.1

The new id setup appears to work great for the minimal example of our current test case. I don't see any remaining overlaps between lanes/lane boundaries and moving objects.

I accidentally stumbled across this MR while I was having issues with not globally unique IDs in the OSI quality checker. I still don't understand this rule at all, but since it is being enforced in esmini now, I found another issue with the IDs.

Check out this test scenario: xodr_stationary_objects_example.zip

The IDs of traffic signs don't seem to be handled.

Field 'id' value 9 in message 'osi3.TrafficSign' is not globally unique, already used by different message 'osi3.LaneBoundary

Good catch @ClemensLinnhoff , it was indeed missed. Thanks for the included scenario-files, it made it easy to spot.

A new commit 81ec9e62 has been pushed to dev that hopefully fixes it.

Hi @thomassedlmayer,

I've been looking into moving this PR forward. Here's a status summary of where things stand:

Resolved on esmini side (all merged to dev):

• Global OSI IDs — v2.57.0
• host_vehicle_id in GroundTruth — v2.57.1
• Traffic sign ID overlaps — commit 81ec9e6

Resolved on osc-validation side:

• GroundTruth input support appears to have landed in PMSFIT/osc-validation (README now lists "OSI GroundTruth/SensorView output"). This was the main blocker you mentioned in December (the SensorView→GroundTruth refactor).

Remaining work to get this mergeable:

1. Rebase onto current dev — the PR has merge conflicts
2. Update the osc-validation commit ref in action.yml (currently pinned to 44c6a2f) to pick up the GroundTruth support
3. Remove the OSMP_FMU/CMakeLists.txt workaround (as you noted in the PR description)
4. Re-run CI to verify everything passes end-to-end

I'd be happy to help with the rebase and cleanup if you can grant me push access to your fork. Let me know!

I updated the osc-validation reference to the newest state including some new test cases to validate start trigger behavior of some exemplary conditions. Good news: esmini passes all of them so far! (see test (ubuntu-22.04, Release) / Run osc-validation-step; There you can download the artifacts incl. overall html report.)

I recently noticed that esmini has the --traj_filter defaulted to 0.1 which somehow merges close points on a trajectory and thereby slightly manipulates the trajectory. On certain, especially real-world captured trajectories where vehicles are in "almost standstill" this leads to significant increases of the trajectory similarity metric as compared to the input trajectory. For the osc-validation call of esmini I now set this to 0.0. At least for me esmini's output due to the default setting was quite confusing. For the strict follow mode the default setting kind of disregards the OSC spec.

Also, I had to fix some CI issues that occurred out of nowhere because the github actions windows runners seem to report cygwin instead of msys now, causing PYTHON to be unset and smoke_test.py to fail. I don't know if this is just happening on my fork?

@eknabevcc @slundel6 Do you have a timeline for osc-validation to be integrated? Did you have a look if you could agree on the general setup of the github actions for osc-validation? Let me know, if I need to make any changes to comply with your requirements.

Btw, we asked to present the osc-validation project in the OpenSCENARIO CCB, so we get more traction/support also from other scenario engine providers. The discussion will probably take place on May 22.

Side note: I temporarily disabled the nightly runs because I keep getting notifications due to a missing dev branch on my fork. Maybe these could be set up more fork-friendly in general but I didn't look more into that for now.

Thanks @thomassedlmayer for pushing this forward. We'll have a look.

I can right now confirm I had the same issue with sudden cygwin OSTYPE reported for windows images, but only sometimes. We had to update script (commit d144794) to support both "msys" and "cygwin". To my understanding it's just a matter of how the bash environment is launched, it should be no functional difference.