LLM & SLM 研究日报
算法·训练·推理 —— 大语言模型与小语言模型的前沿研究
生成时间: 2026/7/7 21:32:28
📊 今日概况
| 方向 | 论文数 |
|---|---|
| 🧮 算法与架构 | 6 |
| 🏋️ 训练方法 | 3 |
| ⚡ 推理优化 | 6 |
| 总计扫描 | 50 |
📝 论文列表
🧮 算法与架构 (6 篇)
1. EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments
- arXiv: 2607.05155
- 摘要: edgebench,world,134,real,agent,scaling,laws,environments,roughly,hours
- 关键词: edgebench,world,134,real,agent,scaling,laws,environments,roughly,hours
2. Multi-Large Language Model Orchestrated Severity Assessment of Clinical Records (MOSAIC)
- arXiv: 2607.05032
- 摘要: mosaic,agentic,severity,kappa,dcsi,tiers,weight,t2d,dissco,llm
- 关键词: mosaic,agentic,severity,kappa,dcsi,tiers,weight,t2d,dissco,llm
3. You Frame It: How Conceptual Representations Shape LLM Detection and Reasoning about Antisemitism
- arXiv: 2607.04945
- 摘要: antisemitism,conceptual,taxonomic,llms,representations,justificatory,reasoning,holocaust,ideologically,definitional
- 关键词: antisemitism,conceptual,taxonomic,llms,representations,justificatory,reasoning,holocaust,ideologically,definitional
4. LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure
- arXiv: 2607.04733
- 摘要: sft,tuning,entropy,token,supervised,fine,structure,multimodal,preservation,local
- 关键词: sft,tuning,entropy,token,supervised,fine,structure,multimodal,preservation,local
5. Air Quality Downscaling with Station-Guided Pseudo-Supervision
- arXiv: 2607.05292
- 摘要: cams,downscaling,station,spatial,europe,guided,coarse,atmospheric,fields,observations
- 关键词: cams,downscaling,station,spatial,europe,guided,coarse,atmospheric,fields,observations
6. FUSE: FK-Steered Multi-Modal Flow Matching for Efficient Simulation-Based Posterior Estimation
- arXiv: 2607.05252
- 摘要: fuse,steered,sbi,estimation,posterior,simulation,modal,matching,inference,multimodal
- 关键词: fuse,steered,sbi,estimation,posterior,simulation,modal,matching,inference,multimodal
🏋️ 训练方法 (3 篇)
1. LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure
- arXiv: 2607.04733
- 摘要: sft,tuning,entropy,token,supervised,fine,structure,multimodal,preservation,local
- 关键词: sft,tuning,entropy,token,supervised,fine,structure,multimodal,preservation,local
2. Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment
- arXiv: 2607.04728
- 摘要: policy,sis,token,importance,tokens,plug,rejection,sampling,level,turning
- 关键词: policy,sis,token,importance,tokens,plug,rejection,sampling,level,turning
3. Learning Only What Valid Adapters Can Express: Subspace-Constrained Adaptation Against Fine-Tuning Poisoning
- arXiv: 2607.05300
- 摘要: pool,subspace,lora,percent,adapters,constrained,adaptation,tuning,fine,blocked
- 关键词: pool,subspace,lora,percent,adapters,constrained,adaptation,tuning,fine,blocked
⚡ 推理优化 (6 篇)
1. LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure
- arXiv: 2607.04733
- 摘要: sft,tuning,entropy,token,supervised,fine,structure,multimodal,preservation,local
- 关键词: sft,tuning,entropy,token,supervised,fine,structure,multimodal,preservation,local
2. Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment
- arXiv: 2607.04728
- 摘要: policy,sis,token,importance,tokens,plug,rejection,sampling,level,turning
- 关键词: policy,sis,token,importance,tokens,plug,rejection,sampling,level,turning
3. Biologically Informed Deep Neural Networks for Multi-Omic Integration, Pathway Activity Inference and Risk Stratification in Cancer
- arXiv: 2607.05306
- 摘要: omic,omics,pathway,integration,biologically,activity,multi,representational,informed,cancer
- 关键词: omic,omics,pathway,integration,biologically,activity,multi,representational,informed,cancer
4. Adaptive Inference Batching using Policy Gradients
- arXiv: 2607.05272
- 摘要: batching,policy,gpu,bursty,policies,routing,arrivals,heterogeneous,traffic,burstgpt
- 关键词: batching,policy,gpu,bursty,policies,routing,arrivals,heterogeneous,traffic,burstgpt
5. FUSE: FK-Steered Multi-Modal Flow Matching for Efficient Simulation-Based Posterior Estimation
- arXiv: 2607.05252
- 摘要: fuse,steered,sbi,estimation,posterior,simulation,modal,matching,inference,multimodal
- 关键词: fuse,steered,sbi,estimation,posterior,simulation,modal,matching,inference,multimodal
6. PDEFlow: Autonomous Agentic PDE Pipelines for Neural Operator Learning and Solver-Free Inference
- arXiv: 2607.05134
- 摘要: pdeflow,solver,agentic,pde,operator,pipelines,backed,inference,ode,specifications
- 关键词: pdeflow,solver,agentic,pde,operator,pipelines,backed,inference,ode,specifications
今日技术热点
今日扫描到 算法与架构 6 篇、训练方法 3 篇、推理优化 6 篇。
算法与架构趋势
当前 LLM 架构正从纯 Transformer 向混合架构演进：SSM (Mamba) 和线性注意力在长序列场景展现优势，MoE 在推理成本可控的前提下持续扩展参数规模。小模型架构注重蒸馏和紧凑设计。
训练方法趋势
DPO 和直接偏好优化正在成为 RLHF 的高效替代方案。合成数据质量成为新的研究焦点。LoRA/QLoRA 已成为高效微调的事实标准。
推理优化趋势
INT4 量化 (GPTQ/AWQ) 已成熟，GGUF 格式让端侧部署成为可能。Speculative decoding 在线推理中逐步普及。KV cache 压缩是降低长上下文推理成本的关键。
关键洞察
- 架构多元化: Transformer 不再是唯一选择，SSM 和混合架构值得持续关注
- 对齐轻量化: DPO 系列方法降低了高质量对齐的门槛
- 推理即服务: 推理优化的研究热度反映了部署需求的爆发
- 小模型逆袭: 端侧 SLM 的设计思路与大模型差异显著，需要专门的技术栈
- 数据 > 算法: 训练数据质量对模型能力的影响被重新审视
学习建议
- 重点关注 Mamba/Mamba-2 和混合架构的最新论文
- 实践 DPO 训练流程，对比 RLHF 的效果差异
- 尝试 vLLM + 量化模型的端到端推理优化
注：GLM-5 API 未调用，此为备用分析
📚 附录
筛选关键词
算法: attention mechanism, mixture of experts, MoE, sparse attention, flash attention, rotary position, RoPE, grouped query, GQA, KV cache …
训练: pre-training, pretraining, post-training, fine-tuning, finetuning, supervised fine-tuning, SFT, alignment, RLHF, DPO …
推理: inference, serving, latency, throughput, speculative decoding, batching, continuous batching, PagedAttention, vLLM, quantization …
本报告由 OpenClaw 自动生成 | LLM & SLM Research Daily