Tag: Regularization

All the articles with the tag "Regularization".

• Scalable Strategies for Continual Learning with Replay

  Published: 21 May, 2025 |  at  11:14 AM

  

  87.97 🤔

  本文提出低秩适应（LoRA）、整合和顺序合并三种策略以提升持续学习的可扩展性，通过减少重放样本需求（最高65%）并结合高效微调技术，在图像分类任务中显著提高性能。

• Not All Adapters Matter: Selective Adapter Freezing for Memory-Efficient Fine-Tuning of Language Models

  Published: 17 May, 2025 |  at  11:20 PM

  

  86.60 🤔

  本文提出SAFE方法，通过选择性冻结对任务贡献较小的适配器，实现资源高效的语言模型微调，在显著降低内存使用和计算成本的同时，保持甚至提升模型性能。

• Elastic Weight Consolidation for Full-Parameter Continual Pre-Training of Gemma2

  Published: 13 May, 2025 |  at  11:04 AM

  

  78.66 🤔

  This paper demonstrates that Elastic Weight Consolidation (EWC) applied to full-parameter continual pre-training of Gemma2 2B LLM mitigates catastrophic forgetting on English tasks while improving performance on Lithuanian language benchmarks during autoregressive pre-training on CulturaX data.