Wow, nice what can come out from relaxing hobby work 🤩

Yes, I can confirm, it works perfectly according to your instructions in Chrome on Windows.

For our understanding, can you super-briefly explain the flow wrt graphics. E.g: How is graphics elements (e.g. road surface, roadmarkings, road objects) created? Can the engine reuse .osgb files?

Here a short architecture summary:

• Esmini C++ core libraries are compiled into a WASM executable served to the browser via esmini.js and index.html
• File loading is implemented using the file system access API of the browser to load all required files from the root directory and its subdirectories, this tree will then be mapped to an equivalent inside the browser cache. This does not work in Firefox since Firefox only allows single files to be loaded. It is possible to work around this limitation but so far I haven't found a very user friendly way so I am leaving that task for later.
• Controls are implemented using HTML, CSS and Javascript which interact with the WASM esmini core
• The viewer/visualization is rewritten using three.js (a very well established library for 3D web apps). It queries the road geometry via the WASM bindings and then implements the procedural rendering on the Javascript side. I can provide more detailed description about this part if needed.
• Similar to the desktop visualization, it is possible to either use the procedural build in road and entity renderer, which is building all meshes and lines with three.js primitives, or include glTF assets as part of the scenario directory tree which three.js can load and render natively. The glTF assets will override the procedural renderer. To answer your question about this, OpenSceneGraph assets are not supported, since the library it just doesn't integrate well with the modern web 3D ecosystem. glTF is currently best for this and is natively supported by tools such as RoadRunner, Blender, three.js etc. Theoretically we might be able to implement some kind of loader/converter for .osgb files later but that is not as straightforward as using .glb files and instead I would rather like to transition to the much more modern standard which is also supported by ASAM OpenMATERIAL 3D (see supported file formats).

My suggested plan for moving forward, of course open for different suggestions:

1. Merge this PR as it is now structurally after a few smaller iterations, i.e., leave everything in the EnvironmentSimulator/Libraries/esminiJS subdirectory to absolutely minimize the initial blast radius.
2. In a follow up PR we figure out how to make it from just a demo/example into a release artifact, maybe move partially to EnvironmentSimulator/Applications/esmini-wasm and then see how to run a release workflow that publishes to something like esmini-app.github.io or esmini-wasm.github.io (or maybe even buy a domain of your choice for it, like esmini.xyz).

@eknabe Complementary to the architectural summary above, here is an example scenario using .glb assets that you should be able to load in the web app to see the difference between the procedural and the glTF based rendering:
bdsc_example.zip

Ok. I think the procedural creation of road surface, markings and various object types is an important part of esmini functionality, yet optional. The portability* and off-screen rendering capabilities provided by OSG are core values, I think (although for full-stack setups typically OSG viewer is replaced by something fancier).

So my initial thought is that OSG will stay as the core scenegraph engine for some time, at least until we have/see another solution covering same capabilities and bringing additional value.

As a compromise strategy, I could see a middle ground approach where a second rendering engine could re-use the low level graphics from OSG scenegraph. E.g. via OSG lib reading back the created geometries. In such setup there would be no need for two separate 3D content generators, which otherwise I think will be hard to maintain (keeping exact same output in esmini standalone and web. E.g. we don't want the different viewer to show different output, at least not in terms of lacking features).

But let us review further, maybe see how the procedural generator in your branch handles various OpenDRIVE content. And we can discuss further from there. We like to keep all doors open, even replacing OSG with something else (still portable and supporting headless/off-screen rendering).

• I know several users using esmini on Mac. And I also check sometimes, always positively surprised how good it works. So it could be interesting to hear about your issue.

Is there any documentation on how to build the desktop app for modern MacOS, i.e. M processor family with ARM architecture? Maybe your known users are still on Intel Macs? If there is something documented please provide a link. I would like to avoid going off-topic though in this PR, I can open a separate issue for this, I think there already is one but for Linux not for MacOS: #660

That being said, here my comments on your strategic considerations:

• I can understand the idea to try to minimize and reuse rendering code as much as possible. I am a bit sceptical though whether this can work based on the existing APIs on the WASM side since there is a considerable "impedance mismatch" between the OSG and three.js. My understanding is if you want to enforce this you end up with implementing an additional compatibility layer might be more work to maintain then what I have implemented. I think at the moment the desktop application rendering code is too tailor made for OpenSceneGraph. We would need two code paths in C++ on top of the abstraction layer, one for osg and one for glTF. We can investigate this a bit if you want. But please first think about this a bit because this is something that we can also do later if needed. Personally I don't have so much "fear" in this area because we already have OpenDRIVE as the common ground API. So while there might be slight visual differences between the desktop and the web app we can never drift much apart. Only the last part of the stack is different for the two rendering libraries.
• For the discussion we are having now it is useful to consider the last ~10 years of history with attempts to bring osg to the web and its multiple failure. I believe any attempt of bringing osg C++ code to WASM is either doomed to fail or brings in a huge amount of extra maintenance work. Supporting a little bit of three.js rendering code is a piece of cake compared to that tbh. It would be good if you could roughly review that part to come to your own conclusion.
• To also state this explicit: I am not suggesting to delete the desktop app anytime soon. My suggestion is to deliver a lightweight web app alongside the desktop app and then see how it goes. Keep the desktop app as the main supported version until further notice. At least until all details of the web based solution have been figured out but most probably the will exist in parallel for a very long time.
• One more comment: I believe if we limit the WASM C++ code to mostly stupid wrapper/middleware code and the web based viewer code to a single vanilla js file like I am proposing here now, it is IMHO super maintainable with agentic coding. That way you can very easily and cleanly separate the esmini classic code base and the web based part. If you start intertwining them more than necessary to have the pixel perfect viewer equivalent on the web you most probably end up with more work than if you go for a clean separation. Please think about it. Consider that I have basically vibe coded the viewer part of this and I am not afraid of any negative impacts of that approach because the blast radius is perfectly contained to viewer.js with the current architecture. That means adding a missing feature from the desktop app or making a small adjustment is just a prompt away while the valuable hand crafted esmini base libs are not impacted at all.

I am next going to add the OSI like visualization part that can be activated as an overlay like on the desktop app to be more feature complete.

Please also give me a small list of missing and blocking must have features if there is anything you have in mind.

esmini builds are universal and hence runs on both Apple Silicon and Intel based systems. You can grab from release.

Or build yourself. For details see CI build script

But essentially something like this should work

mkdir build
cd build
cmake ..
cmake --build . --config Release --target install -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64"

We do not (or never did) intend or aim to port any OSG code to web.

I just want to set the expectations right. Although I really appreciate the initiative I also realize that our resources are limited. We'll not be able to prioritize maintaining a web solution unless all core parts are common.

Regarding strategy, there are some various ideas. But it will all depend on sharp need and use cases. So from my point, it's too early to engage in deeper (although fun) discussions.

Maybe your POC would be better off in a separate repo, more focus and it would be more decoupled from esmini - expectations on visual output similarities as well. On the other hand, I can also consider adding it to esmini main repo. But that needs to be a mutual understanding that it doesn't come with maintenance commitment.

For the review, I tried add/replace scenario. At least OpenDRIVE failed, original is sticky. Is it expected? Or how do you replace it?

Sidenote:

mkdir -p build && cd build && cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_OSX_ARCHITECTURES=arm64 && cmake --build . --config Release --target install -j 2

worked for me.

I pushed an update, here is the latest version with overlay rendering:
esmini-wasm.zip

At least OpenDRIVE failed, original is sticky. Is it expected? Or how do you replace it?

I did not quite get this question, can you describe your issue a bit more?

The basic idea is that there is one (maybe later multiple) "baked in" example scenario which gets loaded at the beginning when the website is opened so people can instantly play with it. Similar to odrviewer.io The .xodr and .xosc files for that are just hosted on the same webserver as the application code. Then if desired, the user should press the "Load scenario directory" button to load one of his own scenarios from his local filesystem. Remember this final step is not supported in Firefox or Safari at the moment, try with Chrome or Edge for example.

Regarding maintaining the code base I really don't have much expectations at all. I think though, even or especially on a best effort basis it is much easier to maintain it in the same repo, exactly like you have been doing for the JS lib until now. For releasing, I imagine something like with CARLA simulator, they create a new release on a best effort basis when they have time between Christmas and New Year's Eve or something like that.
Maybe for now to really build a close to zero commitment workflow, a good middle ground could be to run the release workflow for the web version only manually and when it breaks it just breaks for some time until someone finds the time to fix it and make a new release. So basically we would end up with web release with a lower frequency compared to the desktop version which would be totally fine with me. I can offer to prototype a manual Github pages release workflow in my @johschmitz fork version or in a repo that only handles the Github pages part until it works and then hand it over to you so you have a one click solution ready that you just point to a release tag.

I would maybe permanently host my own private version on Github pages to push to a Github pages page but I feel that would be a bit of an "unfriendly fork" and I would only consider doing this if you would absolutely not want to have it under one of your repos @eknabe / @eknabevcc and you would give me green light for it. Honestly though, I would prefer to not run my own repo and rather contribute to one centralized version from time to time. You know like Linux kernel etc.

Sticky OpenDRIVE
Solved. The problem was catalog not found, resulting in error loading the scenario, hence it looked like it loaded the same OpenDRIVE... After fixing the path to vehicle catalog it worked as expected. Nice!

Hosting
We have no problem continue hosting the java track. So far it's been there for anyone interested to contribute, it has not been part of our core tools. And not actively maintained by core esmini team. Of course that might change, I mean maybe we'll start working on it as well, depending on future prioritizations. But if we agree and start with current low expectations it's OK with us. Decision is yours.

Okay cool, my suggestion then is to merge this version. I will spend some more time on it when I get time again at some point but for now there will be a first working version in that esminiJS directory which I believe is a good step forward compared to what was there before.

Definitely agree this is a major step from previous state. Really good work!

Now merged on dev branch. Will be merged to master by next esmini release.

@johschmitz, @eknabevcc,

I'm not sure if this is the right place to say it, but I wanted to leave a quick thank you here.

I've been building drawtonomy, and I was able to integrate the esmini-WASM build from this PR into it!
Thank you so much for this amazing work. Really!

@kosuke55 very nice to hear, thank you for the testamony!

Just the most recent build for reference:
esmini-wasm.zip

@johschmitz Nice! Does this latest build reflect the latest (committed) code (or is it based on local changes)?

Should be what is merged, I just zipped the local files to be sure it is in this thread since I wasn't sure what exact state I uploaded a couple of messages above. No updates for the moment.