🚀 Feature request

Add support for the Parallel composition block when using ModernBERT through the AdapterHub plugin interface.

Currently, models integrated via the plugin interface (like ModernBERT) cannot use certain features, including Parallel composition. It would be great to extend support for this, enabling richer adapter configurations for plugin-based models.

Documentation reference: https://docs.adapterhub.ml/plugin_interface.html

Motivation

I’d like to use multiple adapters simultaneously during inference with ModernBERT. The Parallel composition block is ideal for this, but it’s currently unsupported for plugin-based models. This limits the ability to efficiently reuse pretrained adapters without needing to re-merge or re-train them.

Your contribution

I'm not familiar with the internal architecture enough to contribute a PR at this time, but I'm happy to test and provide feedback if needed.