Potentially still broken?
Hi guys, after using it heavily for the past few days, I noticed this model is consistently less intelligent than the 30B MoE model, even when thinking. This does not align with people's experiences using other quants or other model formats. I only tried the Q4_K_XL, but this may affect other sizes too.
As an example, consider the promptYou have six horses and want to race them to see which is fastest. What is the best way to do this?
The 30B MoE model correctly answers that the fastest way to do it is simply one race, sometimes suggesting more races if you are constrained by the tracks.
However, this 32B model consistently gets this wrong, and keeps over-engineering the solution and hallucinating assumptions.
Is there an issue with quants for this specific model perhaps?
What is your context length set to? We heard people had issues with looping because the context length was set to 2,048. Have you tried increasing your context length?