Tag: Multimodal Systems
All the articles with the tag "Multimodal Systems".
-
Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks
Published: at 11:07 AM
87.75 🤔ASTRA introduces an efficient defense for Vision Language Models by adaptively steering activations away from adversarial directions using image attribution, achieving state-of-the-art performance in mitigating jailbreak attacks with minimal impact on benign utility and high inference efficiency.
-
Zero-Shot Vision Encoder Grafting via LLM Surrogates
Published: at 11:23 AM 87.49 🤔本文提出通过构建小型代理模型训练视觉编码器并零样本嫁接至大型LLM（如Llama-70B），在保持视觉理解能力的同时将VLM训练成本降低约45%。
-
RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoning
Published: at 11:26 AM 87.36 🤔本文提出 RaaS 算法，通过识别推理任务中的里程碑令牌并采用 LRU 缓存策略管理 KV 向量，在保持高准确性的同时实现了 O(L) 的时间和内存复杂度，显著优于现有方法如 Quest 的内存效率。
-
No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces
Published: at 11:25 AM 87.18 🤔本文提出了一种等向性模型合并框架，通过展平任务矩阵奇异值谱并结合公共与任务特定子空间，显著提升了多任务模型的性能，在视觉和语言任务上达到了最先进的合并效果。
-
Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering
Published: at 11:22 AM 86.89 🤔本文通过将GRPO算法应用于Qwen2-Audio-7B-Instruct模型，在音频问答任务中取得了64.5%的最佳准确率，证明强化学习在小规模数据集上优于监督微调，但显式推理过程未显著提升性能，且与人类水平仍有差距。