More context ?
#1
by
LLMleech
- opened
How can we get more context than 2k, given that gemma 3 27b handles 131k?
Thanks for the question! Although the base Gemma 3–27B can handle up to ~131 K tokens, our fine-tuned model was trained with a 2 048-token window. So while you can still feed in longer text, quality will probably drop off past 2 048 tokens.