A Block-Based Regularization Proposal for Neural Networks
Exploring Localized Weight Groupings as a Way to Control Overfitting
Introduction
I’m not an expert in machine learning. I've been studying the field out of curiosity and an almost irrational drive to understand if some things could be done differently. I ended up thinking about a simple idea — which might already exist in more sophisticated forms — but I thought it was worth sharing: a regularization strategy based on weight blocks.
The idea came as an attempt to simplify regularization processes. What if we grouped weights into trios forming structural subsets (blocks), and applied a smoothing average?
Core Idea: Block-Based Regularization
- Apply regularization using local contrast smoothing, promoting continuity —
... (read 294 more words →)