Robust Dual-Model Collaborative Random Vector Functional Link Network

When training neural networks on real‑world data, noisy labels and outliers can quickly erode performance, especially for models that rely on simple least‑squares fitting. A new approach described in the arXiv pre‑print “Robust Dual‑Model Collaborative Random Vector Functional Link Network” tackles this problem by combining the speed of Random Vector Functional Link (RVFL) networks with a robust loss function derived from the kernel risk‑sensitive mean p‑power (KRP) criterion. The result is a model that retains the lightweight training of RVFL while automatically down‑weighting corrupted samples.

What You Need to Know

The paper introduces the Kernel Risk‑Sensitive Mean p‑Power RVFL (KRPRVFL). Instead of minimizing the usual squared‑error loss, KRPRVFL optimizes a KRP‑based objective that is less sensitive to large residuals. This loss can be interpreted as a smooth approximation of a truncated quadratic, which naturally reduces the impact of points far from the model’s prediction.

In addition to the robust loss, the authors propose a dual‑model collaborative scheme. One RVFL network is trained on the full dataset using the KRP loss, while a second, lighter RVFL is trained only on the samples identified as reliable by the first network. The predictions of the two models are then averaged, yielding a final estimator that benefits from both robustness and the complementary information each model captures.

Empirical tests on benchmark classification and regression datasets with synthetic label noise show that KRPRVFL consistently outperforms standard RVFL and several recent robust alternatives, achieving lower test error while keeping training times comparable to the original RVFL implementation.

Why It Matters

Many real‑world applications—such as medical diagnosis, fraud detection, or sensor‑fusion systems—suffer from imperfect annotations. A model that can maintain accuracy without extensive data cleaning reduces both the labor cost and the risk of propagating errors into downstream decisions. KRPRVFL offers a plug‑in replacement for existing RVFL pipelines, requiring only a change in the loss function and the addition of a lightweight secondary model.

Because RVFL networks are already known for their fast training (closed‑form solution for the output weights) and low memory footprint, the robust variant preserves these advantages. Practitioners can therefore gain robustness without sacrificing the speed that makes RVFL attractive for edge devices or rapid prototyping scenarios.

Key Details

• Loss function: Replaces least‑squares with the kernel risk‑sensitive mean p‑power criterion, which applies a diminishing weight to large residuals.
• Dual‑model collaboration: Primary RVFL learns from all data using the robust loss; secondary RVFL refines predictions using only samples deemed reliable.
• Prediction fusion: Final output is the average of the two models’ predictions, balancing robustness and representational capacity.
• Computational cost: Training remains O(N·D) for N samples and D hidden nodes, identical to standard RVFL, thanks to the closed‑form solution for output weights.
• Empirical validation: Tested on UCI regression and classification sets with noise levels up to 40%; KRPRVFL reduced error by 5‑15% points over baseline RVFL.

What’s Next

The authors suggest extending the framework to deep RVFL architectures and exploring adaptive selection of the p‑parameter in the KRP loss to further tailor robustness to specific noise distributions. They also propose investigating theoretical guarantees for the dual‑model collaboration, which could clarify under what conditions the ensemble outperforms any single robust RVFL.

📌 Source: Arxiv Ml