HILBERT
HILBERT is the BERT-Large model for Hungarian Trained on the 4 BN NYTI-BERT corpus. One of the pioneers of the revolutionary transformer models, BERT has caused a sweeping success in the field of neural NLP.
HIL-RoBERTa
Case based RoBERTa model trained on Hungarian Wikipedia published as part of the Webcorpus 2.0 dataset.
Pretraining was done in 1.25 million steps, with a batch size of 32, using a BPE encoded vocabulary of 30000 subwords.
Using a learning rate of 1e-4, the training went on for five epochs. On a configuration consisting of 4 GTX 1080Ti GPU cards with a total of 44 GB vRAM, the training took 219 hours.
For further details see: this page »
To DOWNLOAD the models, please fill out the registration form: » REGISTRATION FORM «