SAE Interprets MoE: Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization
A new SAE-based analysis finds that MoE experts contribute to distinct latent features, revealing fine-grained roles such as Java, web, geometry, and arithmetic specialization.