You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello. Thank you very much for sharing your experience.
I also want to pretrain a bert with google v3-8, and I want to ask about how to choose the VM Type and Disk size to ensure a bert training.
I noticed that one example said that with 4GBs training corpus, in the create_pretraining_data step, there may cause about 400MB RAM. Is the Virtual Machine and Disk size need to bigger than the RAM maty take? (my corpus is about 1GB)
best wishes.
The text was updated successfully, but these errors were encountered:
I solved this problem by using TPU VMs. And after furth check, if using VM instance, just choose a fitable CPU or GPU , and change the Disk size will be okay.
Hello. Thank you very much for sharing your experience.
I also want to pretrain a bert with google v3-8, and I want to ask about how to choose the VM Type and Disk size to ensure a bert training.
I noticed that one example said that with 4GBs training corpus, in the create_pretraining_data step, there may cause about 400MB RAM. Is the Virtual Machine and Disk size need to bigger than the RAM maty take? (my corpus is about 1GB)
best wishes.
The text was updated successfully, but these errors were encountered: