Model initialization
#22
by
Homin
- opened
Hi,
This model looks impressive, and I’m interested in better understanding the details of how it was initialized. I reviewed the training information provided in the model card and am particularly curious about how the wav2vec2 SSL checkpoint was used. Since the wav2vec2 architecture differs from the ConformerEncoder used in this model, it doesn’t seem possible to directly load the wav2vec2 weights into the encoder. Could you share more details on how the this model was initialized from the wav2vec2 model?
https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2#training
'Initialized from a wav2vec SSL checkpoint pretrained on the LibriLight dataset'
