On the Design of Hyper-Connections: Simplification, Stability, and Scalability
SimpleHC removes residual mixing from Hyper-Connections and groups memory access, matching mHC quality while cutting reported training overhead and improving stability.
ls ./papers | grep "#mixture-of-experts models"
Trending AI research papers from Hugging Face, summarized in plain language: what each paper does, why it matters and where to read more.
2 papers · #mixture-of-experts models
SimpleHC removes residual mixing from Hyper-Connections and groups memory access, matching mHC quality while cutting reported training overhead and improving stability.
DepthGraft treats cross-layer KV sharing as both a cache optimization and a training regularizer, reporting stronger benchmarks and faster prefill with little decoding overhead.