File size: 625 Bytes
15795ad 3a23197 4603656 15795ad 6b4c86c fc45f11 10da14f fc45f11 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
---
license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
tags:
- gemma
- gguf
---
# Gemma 2B Instruct GGUF
Contains Q4 & Q8 quantized GGUFs for [google/gemma](https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b)
## Perf
| Variant | Device | Perf |
| - | - | - |
| Q4 | M1 Pro 10-core GPU | 90 tok/s |
| | Snapdragon 778G CPU | 10 tok/s |
| | RTX 2070S | 40 tok/s |
| Q8 | M1 Pro 10-core GPU | 54 tok/s |
| | Snapdragon 778G CPU | 6 tok/s |
| | RTX 2070S | 25 tok/s |
| F16 | M1 Pro 10-core GPU | 30 tok/s|
| | Snapdragon 778G CPU | <1 tok/s | |