HILANCO

Hungarian Intelligent Language Applications Consortium

« Back to Home Page

HIL-ELECTRA

ELECTRA is based on the GAN (Generative adversarial network) method. Two ELECTRA models were trained:

  • ELECTRA wiki: Trained on Hungarian Wikipedia. Training time: ~5 days.
  • ELECTRA NYTI-BERT: Trained on NYTI-BERT v1 corpus (contains the Hungarian Wikipedia). Training time: ~7 days.
Both models were trained on 1 single GeForce RTX 2080 Ti type video card. Both pretraining was done in 1 million steps, with a batch size of 80. The vocabulary size was 64.000.
For further details see this page »

To DOWNLOAD the models, please fill out the registration form: » REGISTRATION FORM «

References

More Language Models