MJKAN (Modulation Joint KAN) is a novel neural network layer designed to bridge the gap between Kolmogorov-Arnold Networks (KANs) and Multilayer Perceptrons (MLPs).

While KANs offer superior theoretical expressiveness via learnable activation functions, they often suffer from high computational costs and optimization difficulties. MJKAN overcomes these challenges by integrating FiLM (Feature-wise Linear Modulation) with Radial Basis Function (RBF) activations.

Key Features:

• Hybrid Architecture: Combines the non-linear expressive power of KANs with the efficiency of MLPs.
• Tunable Complexity: The number of basis functions ($k$) acts as a direct dial for model complexity.
• Efficiency: Significantly faster inference and lower resource usage compared to standard B-spline KANs.
• Versatility: Demonstrated effectiveness in Function Regression, PDE solving, Image Classification, and NLP.

The MJKAN layer is inspired by the Kolmogorov–Arnold representation theorem. Unlike standard KANs that use B-splines, MJKAN uses a FiLM-modulated RBF decomposition.

Given an input vector $\mathbf{x}$, the layer output $y$ is calculated as:

$$ y = \sum_{i=1}^{d_{\text{in}}} \text{FiLM}_i(x_i) + \text{Base}(x) $$

Where:

1. RBF Expansion: Each input $x_i$ is expanded into $K$ Gaussian basis functions: $$\phi_{ij}(x_i) = \exp\left( -\frac{(x_i - c_j)^2}{2\sigma^2} \right)$$
2. FiLM Modulation: The expansion allows for learnable scaling ($\gamma$) and shifting ($\beta$): $$\text{FiLM}_i(x_i) = \gamma_i x_i + \beta_i$$

Note: With trivial modulation, MJKAN behaves like a KAN. With identity RBFs, it reduces to a linear MLP.

MJKAN demonstrates superior approximation capabilities compared to MLPs. As the number of basis functions ($K$) increases, the Root Mean Square Error (RMSE) decreases significantly, especially for complex compositional functions.

Consistent with literature suggesting KANs excel at scientific computing, MJKAN outperforms MLP in solving the 1D viscous Burgers' equation.

On standard vision datasets, MJKAN is competitive with MLP.

• Observation: Smaller basis sizes ($k=5$) generally generalize better for classification.
• Trade-off: Larger basis sizes increase expressiveness but can lead to overfitting on sparse data (e.g., CIFAR-100).

The cost of MJKAN scales linearly with the number of basis functions.

Paper: Bridging KAN and MLP: MJKAN, a Hybrid Architecture with Both Efficiency and Expressiveness