DDCL: Deep Dual Competitive Learning: A Differentiable End-to-End Framework for Unsupervised Prototype-Based Representation Learning
New architecture replaces external k-means with internal differentiable layer, enabling end-to-end training.
A persistent problem in deep clustering has been the architectural disconnect between feature learning and cluster assignment. Most existing methods rely on external clustering algorithms like k-means to generate pseudo-labels, which prevents the neural network backbone from directly optimizing for the quality of the clusters it creates. This paper by Giansalvo Cirrincione introduces Deep Dual Competitive Learning (DDCL), the first fully differentiable, end-to-end framework designed to solve this exact problem.
The core of DDCL is the Dual Competitive Layer (DCL), which replaces the external k-means step. This layer generates prototypes—the learned cluster centers—as native, differentiable outputs of the network itself. This single architectural inversion makes the entire pipeline, from feature extraction to soft cluster assignment, trainable via backpropagation through a single unified loss function. The framework eliminates the need for Lloyd iterations, pseudo-label discretization, and any external clustering algorithm.
The paper provides a strong theoretical grounding by deriving an exact algebraic decomposition of the training loss. This decomposition reveals a self-regulating mechanism within the loss geometry: a gradient term that acts as an implicit force to separate prototypes and prevent collapse, without requiring any auxiliary objectives. Extensive experiments validated the framework, showing the decomposition held across over 100,000 training epochs and the negative feedback mechanism was confirmed with a Pearson correlation of -0.98. Most importantly, with a jointly trained backbone, DDCL delivered a 65% improvement in clustering accuracy over its non-differentiable ablation and outperformed the DeepCluster method by 122%.
- Replaces external k-means with an internal Dual Competitive Layer (DCL), making prototype generation fully differentiable and part of the network.
- Derives a theoretical loss decomposition that creates a self-regulating separation force, preventing prototype collapse without extra regularization.
- Outperforms its non-differentiable version by 65% in accuracy and beats the DeepCluster method end-to-end by 122% in controlled experiments.
Why It Matters
Enables more efficient and accurate unsupervised learning for tasks like data organization and anomaly detection without manual labeling.