Google's New Switch Transformer Model Achieves 1.6 Trillion Parameters, Efficiency Gains

The Googlers built the Switch Transformers on the back of its own T5 models (introduced in 2019), powered them with 32 of Google's in-house Tensor ...
http://dlvr.it/RsHQr4