Factored One-sided Adam-Moment for Practical and Scalable SOAP
FOAM simplifies SOAP with input-only curvature and factored moments, aiming to deliver second-order optimization quality with near-AdamW cost.
ls ./papers | grep "#LLM pre-training"
Trending AI research papers from Hugging Face, summarized in plain language: what each paper does, why it matters and where to read more.
1 paper · #LLM pre-training
FOAM simplifies SOAP with input-only curvature and factored moments, aiming to deliver second-order optimization quality with near-AdamW cost.