Problem description

thank you for the great work on RLinf.

I have a setup with both simulation and real-world data. I first trained a SmolVLA model with LeRobot, and it achieves about a 50% success rate in the real environment.

Then I used RLinf for post-training. I built a ManiSkill-based simulation environment under envs, trying to align it as closely as possible with the real setup, and I also implemented the corresponding SmolVLA model code under models.

However, when I run the evaluation script, the success rate is 0. I suspect this may be caused by a sim2real gap.

Could you please advise on the following?

Best practices for adding a custom environment and model to RLinf.
Any documentation or examples for integrating custom envs and models.
Guidance for reducing sim2real gaps when using RLinf for post-training.

Any suggestions would be greatly appreciated. Thank you!

Before submitting a new issue...

• Have you checked relevant issues, FAQs (https://rlinf.readthedocs.io/en/latest/rst_source/faq.html), or asked the chatbot at the top right corner of the documentation page.