NbAiLab

/

nb-whisper-large-distil-turbo-beta

like 3

Follow

Nasjonalbiblioteket AI Lab 212

I am getting this error when running the model.

Traceback (most recent call last):
  File "/home/frederik/production/bifrost/scripts/benchmark_asr/benchmark_asr.py", line 1034, in <module>
    main()
  File "/home/frederik/production/bifrost/scripts/benchmark_asr/benchmark_asr.py", line 1030, in main
    benchmark.run()
  File "/home/frederik/production/bifrost/scripts/benchmark_asr/benchmark_asr.py", line 550, in run
    dataset_info = self.inference()
  File "/home/frederik/production/bifrost/scripts/benchmark_asr/benchmark_asr.py", line 654, in inference
    hyp = model.transcribe(audio, sr).lower()
  File "/home/frederik/production/bifrost/scripts/benchmark_asr/benchmark_asr.py", line 349, in transcribe
    output = self.model(audio, **kwargs)
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/pipelines/automatic_speech_recognition.py", line 280, in __call__
    return super().__call__(inputs, **kwargs)
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1450, in __call__
    return next(
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 124, in __next__
    item = next(self.iterator)
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 269, in __next__
    processed = self.infer(next(self.iterator), **self.params)
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1365, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/pipelines/automatic_speech_recognition.py", line 535, in _forward
    tokens = self.model.generate(**generate_kwargs)
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/models/whisper/generation_whisper.py", line 832, in generate
    ) = self.generate_with_fallback(
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/models/whisper/generation_whisper.py", line 1019, in generate_with_fallback
    seek_sequences, seek_outputs = self._postprocess_outputs(
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/models/whisper/generation_whisper.py", line 1129, in _postprocess_outputs
    seek_outputs["token_timestamps"] = self._extract_token_timestamps(
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/models/whisper/generation_whisper.py", line 263, in _extract_token_timestamps
    weights = torch.stack([cross_attentions[l][:, h] for l, h in alignment_heads])
  File "/home/frederik/.cache/uv/environments-v2/benchmark-asr-e5822204b311a382/lib/python3.10/site-packages/transformers/models/whisper/generation_whisper.py", line 263, in <listcomp>
    weights = torch.stack([cross_attentions[l][:, h] for l, h in alignment_heads])
IndexError: list index out of range

When I run the exact same code using the NbAiLab/nb-whisper-large model, it runs without issue.