使用甄嬛传剧本数据以LoRA方式训练BLOOMZ模型,实现以甄嬛口吻对不同人物进行不同回答的Bot
![image](https://private-user-images.githubusercontent.com/133947013/239542057-a1ccacf5-93cb-4a5e-b879-d20d14c58be5.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxMTAwNzcsIm5iZiI6MTczOTEwOTc3NywicGF0aCI6Ii8xMzM5NDcwMTMvMjM5NTQyMDU3LWExY2NhY2Y1LTkzY2ItNGE1ZS1iODc5LWQyMGQxNGM1OGJlNS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA5JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOVQxNDAyNTdaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1kYmMwOTkxMjNjMDFmYzZjZTUyODJiMzNlNGM3MzA4NTY4NDRlNGZjOGZmMDhjMWEwZmVhMjM0NTUzYmIwMDBhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.TrNjXfmS1Ke-DQN3nCHX7LNfrfVHa_oOL3NxawnaNyc)
![image](https://private-user-images.githubusercontent.com/133947013/239542156-030b596e-cd86-4129-abf7-4ca1d44e22c7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxMTAwNzcsIm5iZiI6MTczOTEwOTc3NywicGF0aCI6Ii8xMzM5NDcwMTMvMjM5NTQyMTU2LTAzMGI1OTZlLWNkODYtNDEyOS1hYmY3LTRjYTFkNDRlMjJjNy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA5JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOVQxNDAyNTdaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT04MmRjNTA2NjUxMGEyYjVjYWY3Zjk1MzliYjQ5Y2UxNTNiM2ZmYmY0ZjAxNmUyNzA5OThjYTUzOTFkNmVlMTU2JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.-vyLTE8BInyBwddJ_wBRTEbFf1ry1amn6bwqnw34Q5Y)
基础模型使用BLOOMZ-7B,请从Hugging Face下载后放入base_model目录
数据来源于甄嬛传76集剧本(data_dir/scripts_data),对剧本数据进行了清洗
- 过滤出甄嬛回答的问答对
- 保留提问者身份,去除甄嬛回答身份
- 清洗内容(滤除短的无意义的问话)
- 去除对话次数少于10次的人的数据
- 人工过滤(1小时)
采用3块 32G V100 进行训练
依赖已经在requirements.txt中写明