Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

order_enum 类型打开超大枚举文件的问题 #109

Open
liiiiiang opened this issue Mar 16, 2023 · 3 comments
Open

order_enum 类型打开超大枚举文件的问题 #109

liiiiiang opened this issue Mar 16, 2023 · 3 comments

Comments

@liiiiiang
Copy link

往mysql造测试数据时,某几个字段需要按顺序读取本地的枚举文件,其中有个枚举文件比较大,接近10G。
目前生成数据的速度实在是有点慢,想请问下是否有解决方法,还是说不支持这样方式造数据。
感谢

@gangly
Copy link
Owner

gangly commented Mar 18, 2023

目前的enum实现方法是把整个文件读进内存按行遍历,估计10G占用内存太大,能把meta文件和运行机器的配置发我看看么

@liiiiiang
Copy link
Author

您好,以下是我的meta文件

user_id||varchar(255)||user_id[:inc(user_id,1)]
app_id||varchar(32)||app_id[:order_enum(file://appId.txt)]
unique_id||varchar(32)||unique_id[:inc(unique_id,330128200102783006)]
index||int||index[:random_int(1,10)]
feature||varchar(1000)||feature[:order_enum(file://feature_0.txt)]
index_feature||varchar(6000)||index_feature[:order_enum(file://indexFeature_0.txt)]
length||int||length[:order_enum(file://indexFeatureLength_0.txt)]
url||varchar(255)||url[:image_url]
data_type||char(1)||type[:random_int(0,1)]
status||int||status[:random_int(0,1)]
create_time||datetime||create_time[:date_this_month]
update_time||datetime||update_time[:date_this_month]
remark||varchar(255)||remark[:string(16,128)]
create_by||varchar(64)||create_by[:name]
update_by||varchar(64)||update_by[:name]
sort||int||sort[:random_int(0,9)]```

@liiiiiang
Copy link
Author

liiiiiang commented Mar 22, 2023

我使用的设备的配置信息

    Language: Chinese (Simplified Han) (Regional Setting: Chinese (Simplified Han))
    Processor: 12th Gen Intel(R) Core(TM) i5-12400 (12 CPUs), ~2.5GHz
    Memory: 32768MB RAM
    Available OS Memory: 32452MB RAM
    Page File: 33359MB used, 6185MB available```

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants