Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LLM] update code with paddlenlp #2557

Merged
merged 1 commit into from
Nov 18, 2024
Merged

[LLM] update code with paddlenlp #2557

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions llm/server/server/data/processor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from abc import ABC, abstractmethod

from paddlenlp.transformers import Llama3Tokenizer, LlamaTokenizer
from paddlenlp.utils.llm_utils import get_eos_token_id
from paddlenlp.trl.llm_utils import get_eos_token_id
from server.engine.config import Config
from server.utils import data_processor_logger

Expand Down Expand Up @@ -282,7 +282,7 @@ def _load_tokenizer(self):
"""
if self.config.use_hf_tokenizer:
from transformers import AutoTokenizer
return AutoTokenizer.from_pretrained(self.config.model_dir, use_fast=False)
return AutoTokenizer.from_pretrained(self.config.model_dir, use_fast=False, vocab_file=os.path.join(self.config.model_dir, "sentencepiece.bpe.model"))
else:
from paddlenlp.transformers import AutoTokenizer
return AutoTokenizer.from_pretrained(self.config.model_dir)
Expand Down
2 changes: 1 addition & 1 deletion llm/server/server/engine/infer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
import paddle
import paddle.distributed as dist
import paddle.distributed.fleet as fleet
from paddlenlp.utils.llm_utils import get_rotary_position_embedding
from paddlenlp.trl.llm_utils import get_rotary_position_embedding
from paddlenlp_ops import step_paddle
from server.data.processor import DataProcessor
from server.engine.config import Config
Expand Down
Loading