Issues

• Test real objects instead of hand-built stand-ins

  huggingface/trl#7565 · Filter by author albertvillanova opened on Oct 7, 2026
  

• Some modules fail to import when an optional dependency is missing

  huggingface/trl#7560 · Filter by author albertvillanova opened on Oct 7, 2026
  

• Remote-code models with tied embeddings fail to load with transformers 5.3.0: NotImplementedError: Cannot copy out of meta tensor

  huggingface/trl#7558 · Filter by author albertvillanova opened on Oct 7, 2026
  1

  

• SFT with the default chunked_nll loss fails on a model split across devices

  huggingface/trl#7551 · Filter by author albertvillanova opened on Oct 7, 2026

• VLM SFT collator passes processor kwargs loose to apply_chat_template, triggering a processor_kwargs warning

  huggingface/trl#7546 · Filter by author qgallouedec opened on Oct 6, 2026

• IWOPDTrainer can skip the mandatory vLLM weight sync when resuming from a checkpoint

  huggingface/trl#7544 · Filter by author sahil-mangla opened on Oct 6, 2026
  1

• BEMACallback crashes at on_train_begin with a PEFT (LoRA) model

  huggingface/trl#7543 · Filter by author sahil-mangla opened on Oct 6, 2026
  1

• SFT with assistant_only_loss crashes on Cohere checkpoints whose chat template has named variants

  huggingface/trl#7529 · Filter by author albertvillanova opened on Oct 5, 2026
  

• SFTTrainer and RewardTrainer modify the caller's peft_config when chat_template_path adds tokens

  huggingface/trl#7526 · Filter by author Vedant-Agarwal opened on Oct 5, 2026
  1

• Add score centering to GRPO for vLLM training-inference mismatch

  huggingface/trl#7520 · Filter by author vladbataev opened on Oct 4, 2026

• OnlineDPOTrainer documents processing_class as optional but rejects omission

  huggingface/trl#7513 · Filter by author DaoyuanLi2816 opened on Oct 3, 2026

• Clarify raw held-out dataset evaluation for CPO and ORPO

  huggingface/trl#7512 · Filter by author DaoyuanLi2816 opened on Oct 3, 2026
  1

• BEMACallback fails at train start with ordinary PEFT LoRA

  huggingface/trl#7511 · Filter by author DaoyuanLi2816 opened on Oct 3, 2026
  4

• AsyncGRPOTrainer: support eval_dataset for validation reward

  ✨ enhancement
  huggingface/trl#7509 · Filter by author lewtun opened on Oct 2, 2026
  3

  

• Async GRPO does not train Gemma 4's tool-calling turns

  huggingface/trl#7503 · Filter by author albertvillanova opened on Oct 2, 2026
  

• Make the upkeep of stored chat templates more robust

  huggingface/trl#7501 · Filter by author albertvillanova opened on Oct 2, 2026
  

• Revisit how the tiny models track chat template revisions

  huggingface/trl#7496 · Filter by author albertvillanova opened on Oct 2, 2026
  

• Supported models are rejected when their chat template differs from TRL's stored copy

  huggingface/trl#7492 · Filter by author albertvillanova opened on Oct 2, 2026
  

• Remove the slow tests

  huggingface/trl#7484 · Filter by author albertvillanova opened on Oct 2, 2026

• AsyncGRPOTrainer and AsyncDistillationTrainer log the wrong epoch with the epoch-driven stop

  🐛 bug
  huggingface/trl#7482 · Filter by author lewtun opened on Oct 2, 2026
  5

  

• BEMACallback breaks when any parameter is frozen

  huggingface/trl#7480 · Filter by author Shr3yash opened on Oct 1, 2026
  3

• cast_lm_head_to_fp32 is ignored during GRPO generations (both BF16 autocast and vLLM)

  huggingface/trl#7430 · Filter by author ingambe opened on Sep 28, 2026

• Perf Issue: Colocated vLLM sleeps and reloads weights between the tool turns of one rollout

  Feature

  huggingface/trl#7428 · Filter by author ingambe opened on Sep 28, 2026

• Merged-LoRA export changes frozen base weights

  huggingface/trl#7423 · Filter by author ingambe opened on Sep 28, 2026

• [Feature request] C-GRPO: conformal adaptive rollout budgets for GRPO (NeurIPS 2026)

  huggingface/trl#7402 · Filter by author AryaFayyazi opened on Sep 25, 2026