Russian language is missing
#12
by
Kosh69
- opened
Tested for generation in Russian. Unlike the original model (30b), where the model responded in good Russian - in this model generation in Russian does not work at all. Although understanding of Russian is present.
This issue is likely another side effect of the pruning process. Since the model was pruned based on expert activation frequencies from a specific calibration set, it’s probable that the experts specializing in Russian (or any underrepresented languages) were pruned away. The model may still "understand" Russian because the foundational language capabilities are intact, but without the Russian-specific experts, it can't generate coherent text in the language. 😊