The config.json seems incorrect
#7
by
shoubing35
- opened
The config.json seems incorrect.
"model_type": "llama" should be "falcon"
"architectures": ["LlamaForCausalLM"] should be "architectures": ["FalconForCausalLM"]
shoubing35
changed discussion status to
closed
shoubing35
changed discussion status to
open
weights seem to be llama weights as well:
model.embed_tokens.weight
@shoubing35 - Falcon3 Architecture is compatible with llama architecture. We leverage modelling_llama.py of Hugging face for Falcon3 models. Hope it addresses your concern.