requesting agricultural / AgriLLM converts
Currently no LMStudio converts available. It would be cool if both would work with 24GB VRAM on a GPU.
AgriLLM-Qwen3-30B
AgriLLM-Llama-70B
both see https://huggingface.co/AI71ai
Thanks.
They are booth queued!
Thanks for lot for requesting them. I'm surprised we missed them as based on the model card they probably should have been queued as highly anticipated models on release day.
You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#agrillm-Qwen3-30B-A3B-GGUF and https://hf.tst.eu/model#Llama-agrillm-3.3-70B-GGUF for quants to appear.
@nicoboss
No chance to come below the 24GB wall with the Llama-agrillm-3.3-70B-i1-GGUF ?
Or maybe the published trainig data at https://huggingface.co/AI71ai could be used to create a smaller model of this important llm?
Regards Christian
Quants are still being generated but we already have uploaded Llama-agrillm-3.3-70B.i1-IQ1_M.gguf which is 16.8 GB in size and so should fit on a 24 GB GPU. Please keep in mind that while such very low bits per wight quants while fit they inevitably lost quite some quality compared to the source model so you will probably get a better experience by using agrillm-Qwen3-30B-A3B.i1-IQ4_XS.gguf instead as Qwen3 is a more recent base model and has the perfect size to run on a 24 GB GPU.
Or maybe the published trainig data at https://huggingface.co/AI71ai could be used to create a smaller model of this important llm?
Yes you could easily create your own version of this model by using axolotl to finetune any model you like using their dataset. For Llama-agrillm-3.3-70B they used 8xH100 for 7 hours which when considering H100 RunPod prices plus other resources needed will be around $250 to train it. If you intend on commercially using this model this is probably worth it. They were nice enough to provide all their finetuning settings inside the model card. If you have a lot of RAM (around 300 GB for 70B) and two 24 GB GPUs you could create your own version up to 70B using your own hardware by using FSDP and offloading all layers to RAM but doing so will take quite a while to train. For smaller models I can also help you finetune. 8B I could easily finetune on Richard’s supercomputer using 4x A100 40G.
To make the AgriLLM project truly impactful for global resilience, I suggest we move away from the dense 70B IQ1 path. To be practically useful in the field, we need a Vision-Reasoning model that can solve complex diagnostic tasks on consumer hardware.
Instead of a heavily degraded 70B, I propose focusing on a Reasoning-distilled 27–30B base that runs stably at Q4. My concrete recommendations are:
Option A: google/gemma-3-27b-it (The VLM Choice): This is the current gold standard for Vision-Reasoning. Its native multimodal architecture allows it to 'think' through visual data. A farmer needs a model that doesn't just describe a leaf, but reasons through the symptoms (e.g., 'Yellowing edges suggest potassium deficiency, but the necrotic spots point to early blight').
Option B: Qwen/Qwen3-VL-30B-A3B-Thinking: This is the 'Thinking' version of Qwen3. It is specifically optimized to perform step-by-step reasoning on visual inputs. Its MoE architecture makes it incredibly fast on a single RTX 3090, which is vital for NGOs and localized advisory.
The Goal: > We should aim for a model that explains its visual diagnosis through Chain-of-Thought (CoT). By using datasets like AgMMU or MIRAGE, we can build a tool that provides reliable, logical, and visual agricultural support.
A Q4_K_M quant of such a 27-30B Vision-Reasoning model would be far more robust and accessible than a 'lobotomized' 70B IQ1. What are your thoughts on building a dedicated Agri-Vision-Reasoning model?
Just a suggestion from a hardware-accessibility point of view – I don’t have the necessary hardware myself, but wanted to share the idea to make the world a bit better.
Regards,
Christian
reformulated