SFTTrainer using quantized model + peft does not work

[osanseviero]

Example code from the official docs fails due to this

from datasets import load_dataset
from transformers import AutoModelForCausalLM
from trl import SFTTrainer

dataset = load_dataset("timdettmers/openassistant-guanaco", split="train")

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)

model = AutoModelForCausalLM.from_pretrained(
    "EleutherAI/gpt-neo-125m",
    load_in_8bit=True,
    device_map="auto",
)

trainer = SFTTrainer(
    model,
    train_dataset=dataset,
    dataset_text_field="text",
    peft_config=peft_config,
)

Error logs

AttributeError                            Traceback (most recent call last)
[<ipython-input-3-ffcd404fb112>](https://localhost:8080/#) in <cell line: 18>()
     16 )
     17 
---> 18 trainer = SFTTrainer(
     19     model,
     20     train_dataset=dataset,

[/usr/local/lib/python3.10/dist-packages/trl/trainer/sft_trainer.py](https://localhost:8080/#) in __init__(self, model, args, data_collator, train_dataset, eval_dataset, tokenizer, model_init, compute_metrics, callbacks, optimizers, preprocess_logits_for_metrics, peft_config, dataset_text_field, packing, formatting_func, max_seq_length, infinite, num_of_sequences, chars_per_token, dataset_num_proc, dataset_batch_size, neftune_noise_alpha, model_init_kwargs)
    169                     )
    170 
--> 171                     preprare_model_kwargs = {"use_gradient_checkpointing": args.gradient_checkpointing}
    172 
    173                     if _support_gc_kwargs:

AttributeError: 'NoneType' object has no attribute 'gradient_checkpointing'

Versions

transformers==4.35.2
trl==0.7.5.dev0 (install from source; I also tried from last release with same error).

Additional Info

If I pass my own TrainingArguments with gradient_checkpointing=True then all is fine.
Specifically, this line https://github.com/huggingface/trl/blob/main/trl/trainer/sft_trainer.py#L171 won't work if args is None.

[elliotttruestate]

What is the specific source of that example code?

The example script here https://github.com/huggingface/trl/blob/main/examples/scripts/sft.py provides an expected input and code structure.

[osanseviero]

https://huggingface.co/docs/trl/sft_trainer#training-adapters-with-base-8-bit-models

[lvwerra]

cc @younesbelkada