huihui-ai

/

DeepSeek-R1-Pruned-Coder-411B

like 4

Hi, I'm interested in trying my hand at pruning MoE models, with the goal of eventually pruning Kimi-K2. Do you have any code samples for how you are deciding which experts to keep and then actually removing experts from the model? I plan to start with some tiny models first.