LLM & SLM 研究日报
算法·训练·推理 —— 大语言模型与小语言模型的前沿研究
生成时间: 2026/7/19 09:00:08
📊 今日概况
| 方向 | 论文数 |
|---|---|
| 🧮 算法与架构 | 12 |
| 🏋️ 训练方法 | 3 |
| ⚡ 推理优化 | 4 |
| 总计扫描 | 50 |
📝 论文列表
🧮 算法与架构 (12 篇)
1. Beyond the Leaderboard: Design Lessons for Trustworthy Multimodal VQA
- arXiv: 2607.15241
- 摘要: multimodal,trustworthy,leaderboard,vqa,healthcare,evidence,medico,mediaeval,lessons,beyond
- 关键词: multimodal,trustworthy,leaderboard,vqa,healthcare,evidence,medico,mediaeval,lessons,beyond
2. TikStance: A Multimodal and Hierarchical Dataset for Multi-target Stance Analysis in TikTok Political Conversations
- arXiv: 2607.15240
- 摘要: stance,political,tikstance,tiktok,conversations,target,biden,multimodal,trump,audiovisual
- 关键词: stance,political,tikstance,tiktok,conversations,target,biden,multimodal,trump,audiovisual
3. T^2MLR: Transformer with Temporal Middle-Layer Recurrence
- arXiv: 2607.15178
- 摘要: middle,recurrence,layer,reasoning,t2mlr,2mlr,transformer,token,transformers,portantly
- 关键词: middle,recurrence,layer,reasoning,t2mlr,2mlr,transformer,token,transformers,portantly
4. Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents
- arXiv: 2607.15095
- 摘要: manifesto,coalition,party,negotiation,ideological,partisan,formateur,dpo,lineage,rag
- 关键词: manifesto,coalition,party,negotiation,ideological,partisan,formateur,dpo,lineage,rag
5. Show Me How You Reason and I'll Tell You Who You Are: Reasoning Graphs for Robust LLM Authorship Attribution
- arXiv: 2607.14905
- 摘要: llm,authorship,paraphrasing,reasoning,attribution,obfuscation,percentage,tell,imaginable,generated
- 关键词: llm,authorship,paraphrasing,reasoning,attribution,obfuscation,percentage,tell,imaginable,generated
6. CoTu at EXACT 2026: Neuro-Symbolic Reasoning for Transparent Educational QA
- arXiv: 2607.14735
- 摘要: cotu,2026,reasoning,symbolic,neuro,answer,educational,emits,deduction,transparent
- 关键词: cotu,2026,reasoning,symbolic,neuro,answer,educational,emits,deduction,transparent
7. Gold-Guided Programmatic Distillation for Financial Reasoning over Hybrid Tables and Text
- arXiv: 2607.14709
- 摘要: reasoning,programmatic,teacher,distillation,gold,financial,tat,rationales,student,verified
- 关键词: reasoning,programmatic,teacher,distillation,gold,financial,tat,rationales,student,verified
8. D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding
- arXiv: 2607.14647
- 摘要: cut,decoding,draft,speculative,verification,pruning,tokens,drafts,request,drafting
- 关键词: cut,decoding,draft,speculative,verification,pruning,tokens,drafts,request,drafting
9. On-Policy Delta Distillation
- arXiv: 2607.15161
- 摘要: distillation,policy,delta,reasoning,opd,teacher,signal,reward,naver,post
- 关键词: distillation,policy,delta,reasoning,opd,teacher,signal,reward,naver,post
10. Multimodal Semantic-Aware Contrastive Learning For False Negative Mitigation in 3D Medical Imaging
- arXiv: 2607.14995
- 摘要: multimodal,contrastive,false,semantic,pediatric,samples,negatives,radiology,negative,semantically
- 关键词: multimodal,contrastive,false,semantic,pediatric,samples,negatives,radiology,negative,semantically
11. Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation
- arXiv: 2607.14895
- 摘要: rlm,reasoning,rlms,domains,tuning,instruction,verifiable,performance,coding,unverifiable
- 关键词: rlm,reasoning,rlms,domains,tuning,instruction,verifiable,performance,coding,unverifiable
12. GAttNHP: Group Attention Neural Hawkes Process for Extrapolation Reasoning in Temporal Knowledge Graphs
- arXiv: 2607.14733
- 摘要: gattnhp,hawkes,tkg,attention,chains,group,temporal,tailed,quantile,arrival
- 关键词: gattnhp,hawkes,tkg,attention,chains,group,temporal,tailed,quantile,arrival
🏋️ 训练方法 (3 篇)
1. Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents
- arXiv: 2607.15095
- 摘要: manifesto,coalition,party,negotiation,ideological,partisan,formateur,dpo,lineage,rag
- 关键词: manifesto,coalition,party,negotiation,ideological,partisan,formateur,dpo,lineage,rag
2. Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation
- arXiv: 2607.14895
- 摘要: rlm,reasoning,rlms,domains,tuning,instruction,verifiable,performance,coding,unverifiable
- 关键词: rlm,reasoning,rlms,domains,tuning,instruction,verifiable,performance,coding,unverifiable
3. Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs
- arXiv: 2607.14888
- 摘要: ideological,finetuning,generalisation,innocuous,seeming,prompting,ideology,sycophantic,finetuned,shifts
- 关键词: ideological,finetuning,generalisation,innocuous,seeming,prompting,ideology,sycophantic,finetuned,shifts
⚡ 推理优化 (4 篇)
1. D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding
- arXiv: 2607.14647
- 摘要: cut,decoding,draft,speculative,verification,pruning,tokens,drafts,request,drafting
- 关键词: cut,decoding,draft,speculative,verification,pruning,tokens,drafts,request,drafting
2. Routing Ceilings Are Domain-Independent: Structural Prior Injection in Code Security Vulnerability Detection
- arXiv: 2607.14628
- 摘要: cwe,sair,vulnerability,vudenc,cheatsheets,cheatsheet,collapse,cve,structural,code
- 关键词: cwe,sair,vulnerability,vudenc,cheatsheets,cheatsheet,collapse,cve,structural,code
3. Kernel weighted importance sampling for off-policy evaluation in contextual bandits
- arXiv: 2607.15067
- 摘要: wis,importance,vanilla,sampling,kernel,weighted,policy,bandits,contextual,evaluation
- 关键词: wis,importance,vanilla,sampling,kernel,weighted,policy,bandits,contextual,evaluation
4. Causal Inference for Sequential Settings under Interference and Latent Confounding
- arXiv: 2607.14940
- 摘要: causal,settings,latent,confounders,sequential,interference,confounding,outcome,outcomes,units
- 关键词: causal,settings,latent,confounders,sequential,interference,confounding,outcome,outcomes,units
今日技术热点
今日扫描到 算法与架构 12 篇、训练方法 3 篇、推理优化 4 篇。
算法与架构趋势
当前 LLM 架构正从纯 Transformer 向混合架构演进：SSM (Mamba) 和线性注意力在长序列场景展现优势，MoE 在推理成本可控的前提下持续扩展参数规模。小模型架构注重蒸馏和紧凑设计。
训练方法趋势
DPO 和直接偏好优化正在成为 RLHF 的高效替代方案。合成数据质量成为新的研究焦点。LoRA/QLoRA 已成为高效微调的事实标准。
推理优化趋势
INT4 量化 (GPTQ/AWQ) 已成熟，GGUF 格式让端侧部署成为可能。Speculative decoding 在线推理中逐步普及。KV cache 压缩是降低长上下文推理成本的关键。
关键洞察
- 架构多元化: Transformer 不再是唯一选择，SSM 和混合架构值得持续关注
- 对齐轻量化: DPO 系列方法降低了高质量对齐的门槛
- 推理即服务: 推理优化的研究热度反映了部署需求的爆发
- 小模型逆袭: 端侧 SLM 的设计思路与大模型差异显著，需要专门的技术栈
- 数据 > 算法: 训练数据质量对模型能力的影响被重新审视
学习建议
- 重点关注 Mamba/Mamba-2 和混合架构的最新论文
- 实践 DPO 训练流程，对比 RLHF 的效果差异
- 尝试 vLLM + 量化模型的端到端推理优化
注：GLM-5 API 未调用，此为备用分析
📚 附录
筛选关键词
算法: attention mechanism, mixture of experts, MoE, sparse attention, flash attention, rotary position, RoPE, grouped query, GQA, KV cache …
训练: pre-training, pretraining, post-training, fine-tuning, finetuning, supervised fine-tuning, SFT, alignment, RLHF, DPO …
推理: inference, serving, latency, throughput, speculative decoding, batching, continuous batching, PagedAttention, vLLM, quantization …
本报告由 OpenClaw 自动生成 | LLM & SLM Research Daily