DOC: Extend modules_to_save doc with pooler example (#2175)

See #2171 --------- Co-authored-by: Steven Liu <[email protected]>
huggingface · Oct 25, 2024 · 8d545c6 · 8d545c6
1 parent 0041434
commit 8d545c6
Showing 1 changed file with 6 additions and 0 deletions.
diff --git a/docs/source/developer_guides/troubleshooting.md b/docs/source/developer_guides/troubleshooting.md
@@ -118,6 +118,12 @@ You should probably TRAIN this model on a down-stream task to be able to use it
 
 The mentioned layers should be added to `modules_to_save` in the config to avoid the described problem.
 
+<Tip>
+
+As an example, when loading a model that is using the DeBERTa architecture for sequence classification, you'll see a warning that the following weights are newly initialized: `['classifier.bias', 'classifier.weight', 'pooler.dense.bias', 'pooler.dense.weight']`. From this, it follows that the `classifier` and `pooler` layers should be added to: `modules_to_save=["classifier", "pooler"]`.
+
+</Tip>
+
 ### Extending the vocabulary
 
 For many language fine-tuning tasks, extending the model's vocabulary is necessary since new tokens are being introduced. This requires extending the embedding layer to account for the new tokens and also storing the embedding layer in addition to the adapter weights when saving the adapter.