DPO

Timeline

1. Oct 19, 2026 · Product / feature launch · 1 source
   Amazon SageMaker JumpStart: 2026 year-to-date in review
   Amazon SageMaker JumpStart is a machine learning (ML) hub within Amazon SageMaker AI that provides access to a curated catalog of AI models, solution templates, and example notebooks.
2. Oct 16, 2026 · Research paper · 1 source
   Analyzing Occupational Gender Bias across the LLM Training Lifecycle
   We find that when gender is stated, post-training increases semantic diversity for male-coded profiles, whereas for female-coded profiles the increase is smaller after SFT and not significant after DPO and RLVR.
3. Oct 16, 2026 · Open-source release · 1 source
   axolotl-ai-cloud/axolotl v0.21.0
   This release is headlined by Reflection's Beam, a 501B-parameter MoE you can QLoRA-tune on a single node.
4. Oct 16, 2026 · Research paper · 1 source
   SCALE: Improving Guidance-Free Visual Generation via Self-Contrastive Alignment for Likelihood Estimation
   In this paper, we propose self-contrastive guidance (SCG), which contrasts the model's prediction on real and self-generated samples to obtain a direct, model-based signal that pushes generation toward the true data manifold.
5. Oct 16, 2026 · Research paper · 1 source
   Compression Precedes, Content Decides: Emergent Misalignment in Reward Models
   Emergent Misalignment - the out of domain generalization of narrow harmful fine-tuning first shown by Betley et al. [2025] has been widely studied in language models but not in reward models that supervise them.
6. Oct 15, 2026 · Open-source release · 1 source
   unslothai/unsloth v0.1.908-beta: NVIDIA Nemotron 3.5 + Meta Muse Spark Open
   Run NVIDIA's new Nemotron 3.5 models and Meta's new Muse Spark Open locally in Unsloth.
7. Oct 15, 2026 · Open-source release · 1 source
   huggingface/trl v1.16.0
   🛠️ TRL now trains on AWS Trainium
8. Oct 15, 2026 · Research paper · 1 source
   PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents
   We propose PROBE (Policy Rule Observation via Behavioral Exploration), an algorithm for training LLM-based auditors that interact with deployed agents as realistic users and probe them for policy violations through multi-turn, tool-mediated dialogue.
9. Oct 15, 2026 · Model release · 3 sources
   Apertus 1.5 Release Notes
   Apertus 1.5 is a continued pretraining of the first generation model: adding 4 trillion tokens of text and multimodal training data to the 8B model and 2 trillion tokens to the 70B model, for a total of 19T and 17T, respectively.
10. Oct 15, 2026 · Research paper · 1 source
    All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents
    To address this, we propose All-in-One Memory (AiO-Mem), which organizes these operations as transitions over a shared evidence state and directly evaluates the semantic quality of each action in a unified reward space.
11. Oct 15, 2026 · Research paper · 1 source
    Bellman-Consistent Trajectory RL for LLM Arithmetic Reasoning
    To address sparse terminal rewards in LLM arithmetic reasoning, we propose a Bellman-consistent trajectory RL method.
12. Oct 14, 2026 · Opinion / analysis · 1 source
    Tracing where LLM "slop" style comes from in the model and in training, and cutting it out [R]
    I wanted to find where the recognizable assistant style (em dashes, "it's not X, it's Y", lists of three, buzzwords) lives in a model and which training stage installs it.
13. Oct 14, 2026 · Research paper · 1 source
    Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge
    We present Manacá-1B-Instruct-2, the second-generation post-training alignment of Manacá-1B (1.72B parameters, trained from scratch for PT-BR), with an SFT corpus of 291,223 examples.
14. Oct 14, 2026 · Research paper · 1 source
    DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents
    To reduce these loops, we propose DeLoop, a self-supervised training method.
15. Oct 13, 2026 · Research paper · 1 source
    DPOison-VL: Clean Preference Poisoning in Vision-Language Direct Preference Optimization
    We introduce DPOison-VL, the first clean preference poisoning attack against VLM-DPO, which manipulates model alignment solely through imperceptible perturbations on a small fraction of training images, while leaving all prompts, responses, and preference annotations semantically intact.
16. Oct 13, 2026 · Research paper · 1 source
    Do Factuality Methods Make LLMs More Factual? Reproducibility and Fair Comparison
    In this paper, we present the first independent reproduction of five factuality alignment methods, which span on-policy RL (PPO, GRPO, and Dr. GRPO) and offline preference optimization (DPO and KTO), and compare them on a common benchmark with common evaluation metrics.
17. Oct 13, 2026 · Research paper · 1 source
    VARI: Robust Text-to-Video Generation via Group-Structured Intent Alignment
    We introduce VARI (Verified Alignment over Representations of Intent), a group-structured framework that treats automatically verified formulations of the same visual intent as a shared supervision unit.
18. Oct 12, 2026 · Research paper · 1 source
    EPOCH: Coverage-Aware Evolution with Experiential Preference Optimization for LLM-based Automated Heuristic Design
    LLM-based automated heuristic design (AHD) discovers heuristics for combinatorial optimization problems (COPs), where an LLM generates candidates in an evolutionary search.
19. Oct 12, 2026 · Research paper · 2 sources
    LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators
    Generative discovery in science requires coupling high-level scientific reasoning with the ability to generate samples respecting the structure of a design space.
20. Oct 12, 2026 · Research paper · 1 source
    SFT Leaves a Reproducible Signature in Activation Geometry: Calibrating Post-Training Interpretability Against Run-to-Run Noise
    SFT leaves a large, reproducible signature: base $\to$ SFT displaces the graph by 17.8 pairwise-distance SDs beyond the mean distance between four independent LoRA-SFT runs, and the pre-registered estimator replicates on Llama-3.1-8B with Llama-Scope.
21. Oct 12, 2026 · Research paper · 1 source
    AD4EDU: Benchmarking University Lecture Audio Description across Accessibility Standards, Metrics, and Learner Preferences
    Audio description (AD) aims to enhance the accessibility of visual media for blind and low-vision (BLV) persons.
22. Oct 12, 2026 · Research paper · 1 source
    WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search
    To build an effective rewriter that remains faithful to user intent, we propose WeFair, Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in WeChat short-video search.
23. Oct 12, 2026 · Research paper · 1 source
    Davinci-Tool: Scaling Agentic Tool-Use Supervision with Verifiable Executable Environments
    To address this challenge, we introduce Davinci-Tool, a 30B tool-use model, and Davinci-ToolData, an execution-grounded dataset designed to scale environment coverage and outcome verification together.
24. Oct 12, 2026 · Opinion / analysis · 1 source
    QLoRA on ExLlamaV3 Quants - Train @ 2-8bpw
    A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without transformers.
25. Oct 12, 2026 · Research paper · 1 source
    WildTrace: Evaluating Training Data Detection in Realistic Large-Scale LLM Post-Training
    To this end, we introduce \method, a reliable benchmark designed to evaluate detection methods in realistic, large-scale post-training settings.
26. Oct 11, 2026 · Benchmark result · 1 source
    Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI — Fireworks AI
    Jetashree Ravi, who leads part of the applied machine learning team at Fireworks AI, explains how teams move from closed models to open ones without losing quality.
27. Oct 10, 2026 · Research paper · 1 source
    Japanese Handwriting Generation Based on Human Preference Feedback Using Direct Preference Optimization
    We propose a Japanese handwriting generation framework that integrates human preference feedback using Direct Preference Optimization (DPO) to align generative models with human cognitive and stylistic preferences.
28. Oct 10, 2026 · Research paper · 1 source
    PsyMemBehav: Psychology-Informed and Memory-Grounded Two-Level Identity-to-Action Reasoning for Social-Media Behavior Simulation
    We introduce PsyMemBehav, which maps psychology-informed profiles and retrieved memories to target-first platform behavior through two-level identity-to-action reasoning.
29. Oct 9, 2026 · Research paper · 1 source
    Golden Anchor for Video Generation
    State-of-the-art systems achieve controllability primarily through image-to-video (I2V) generation, in which an injected anchor image determines the identity, composition, and style of the resulting video.
30. Oct 9, 2026 · Research paper · 1 source
    Evolving Creatives with Clicks: CTR-Driven Advertising image Editing via Attribute Disentanglement and Preference Alignment
    In this work, we propose Evolving Creatives with Clicks (ECC), which reformulates creative optimization as fine-grained editing rather than generation from scratch.

Often appears with