Hong Kong Machine Learning Season 3 Episode 12
21.07.2021 - Hong Kong Machine Learning - ~2 Minutes
When?
- Wednesday, July 21, 2021 from 7:00 PM to 9:00 PM (Hong Kong Time)
Where?
- At your home, on zoom. All meetups will be online as long as this COVID-19 crisis is not over.
Programme:
Talk 1: PAGnol: A French extreme-scale model
Extreme-scale models with billions of parameters (GPT-3, T5, etc.) have garnered increased interest over the past year, demonstrating unique few-shot learning abilities. We introduce PAGnol, a collection of French GPT models. Using scaling laws, we efficiently train PAGnol-XL (1.5B parameters) with the same computational budget as CamemBERT, a model 13 times smaller. PAGnol-XL is the largest model trained to date for the French language. We also explore some of the motivations behind extreme-scale models: from the surprising finding of scaling laws that larger models are more efficient, to recent research on prompt-tuning questioning the established fine-tuning paradigm.
Talk 2: Bayesian Modeling without the Math
Alexandre Andorra, Principal Data Scientist & Co-founder, PyMC Labs
Video Recording of the HKML Meetup on YouTube
- YouTube videos:
Talk 1:
HKML S3E12 - PAGnol: A French extreme-scale model by Julien Launay
Talk 2:
HKML S3E12 - Bayesian Modeling without the Math by Alexandre Andorra
Thanks to our patrons for supporting the meetup!
Check the patreon page to join the current list: