---
title: "MOEReview Pan: Dense Training Sparse Inference"
source: https://www.jemoka.com/posts/kbhmoereview_pan_dense_training_sparse_inference/
---

Train experts densely, and then during inference keep only topk
