Microsoft’s open source Dion optimizer aims to make huge AI models faster and cheaper to train
Microsoft has unveiled Dion, a new open-source optimizer that could make massive AI models faster and cheaper to train by using low rank orthonormal updates.