fast field test with your models
same test on each model - one question, receive same 20 snippet (a 1024 chracters) from embedded model
16k contextsize
600pages PDF embedded
grag-nemo-12b-sft-hessian-ai - 80%
grag-llama-3.1-8b-sft-hessian-ai - 70%
grag-phi-3.5-mini-4b-sft-hessian-ai - no use
answers in comparison to
SauerkrautLM-Nemo-12b-Instruct - 90%
SauerkrautLM-Llama-3.1-8b-Instruct - 90%
no model can handle real context of 16k token .. try it !
but keep working, the German community is very quiet ;)
Hi
@kalle07
,
these models are meant for synthetic data generation and especially the SFT-Models are strictly bound to the Prompt Templates that were trained in this stage.
If you would like to test a more generialized model you could test the merged llama model or even look into the example where we have merged with hermes.
https://huggingface.co/avemio/German-RAG-LLAMA-3.1-8B-MERGED-HESSIAN-AI
https://huggingface.co/avemio/German-RAG-x-HERMES-LLAMA-MERGED-EXP
The decision to work with Base-Models was in the end resulting into too narrow models that only accept instruction tasks trained in the different stages. Was better for comparison but not better for the performance in the end.
Hope that helps until we will prepare the next round!