Tag: Efficiency
All the articles with the tag "Efficiency".
-
Skywork Open Reasoner 1 Technical Report
Published: at 11:44 AM
88.60 🤔Skywork-OR1通过提出MAGIC框架，利用多阶段训练和自适应熵控制的强化学习方法，显著提升了长链式推理模型在数学和编码任务上的性能，并在AIME24和AIME25基准上超越了DeepSeek-R1和Qwen3-32B。
-
Hide & Seek: Transformer Symmetries Obscure Sharpness & Riemannian Geometry Finds It
Published: at 11:09 AM 88.35 🤔This paper introduces geodesic sharpness, a novel measure using Riemannian geometry to account for transformer symmetries on a quotient manifold, demonstrating stronger correlations with generalization across diagonal networks, vision transformers, and language models compared to traditional adaptive sharpness.
-
LoLA: Low-Rank Linear Attention With Sparse Caching
Published: at 11:40 AM 88.31 🤔LoLA通过结合线性注意力、滑动窗口和稀疏缓存三种内存形式，在推理时有效缓解记忆冲突，显著提升线性注意力模型在长上下文关联回忆和语言建模任务上的性能，同时保持高效内存使用。
-
SLOT: Sample-specific Language Model Optimization at Test-time
Published: at 11:22 AM 88.26 🤔本文提出SLOT方法，通过测试时对每个输入提示优化一个轻量级样本特定参数向量δ，显著提升大型语言模型在推理任务上的性能，如Qwen2.5-7B在GSM8K上提升8.65%。
-
Unifying Multimodal Large Language Model Capabilities and Modalities via Model Merging
Published: at 11:19 AM 88.24 🤔本文提出一个多模态大语言模型（MLLM）融合基准和改进的任务向量优化方法（WUDI v2），通过低秩近似去除噪声并优化合并向量，在多任务和跨模态融合实验中取得平均2.48%的性能提升，展现了无需数据训练即可构建高性能MLLMs的潜力。