Update

README.md

@@ -9,26 +9,110 @@ datasets:
 license: cc-by-4.0
 ---
 
-## ESPnet2 LID model
+## ESPnet2 Spoken Language Identification (LID) model
 
 ### `espnet/geolid_vl107only_independent_frozen`
 
-This model was trained by Qingzheng-Wang using geolid recipe in [espnet](https://github.com/espnet/espnet/).
+This geolocation-aware language identification (LID) model is developed using the [ESPnet](https://github.com/espnet/espnet/) toolkit. It integrates the powerful pretrained [MMS-1B](https://huggingface.co/facebook/mms-1b) as the encoder and employs [ECAPA-TDNN](https://arxiv.org/pdf/2005.07143) as the embedding extractor to achieve robust spoken language identification.
 
-### Demo: How to use in ESPnet2
+The main innovations of this model are:
+1. Incorporating geolocation prediction as an auxiliary task during training.
+2. Conditioning the intermediate representations of the self-supervised learning (SSL) encoder on intermediate-layer information.
+This geolocation-aware strategy greatly improves robustness, especially for dialects and accented variations.
 
-Follow the [ESPnet installation instructions](https://espnet.github.io/espnet/installation.html)
-if you haven't done that already.
+For further details on the geolocation-aware LID methodology, please refer to our paper: *Geolocation-Aware Robust Spoken Language Identification* (arXiv link to be added).
+
+### Usage Guide: How to use in ESPnet2
+
+#### Prerequisites
+First, ensure you have ESPnet installed. If not, follow the [ESPnet installation instructions](https://espnet.github.io/espnet/installation.html).
+
+#### Quick Start
+Run the following commands to set up and use the pre-trained model:
 
 ```bash
 cd espnet
-git checkout 77e4293952083b9e32bc19a5ddc19efe45e70e4a
+
 pip install -e .
 cd egs2/geolid/lid1
-./run.sh --skip_data_prep false --skip_train true --download_model espnet/geolid_vl107only_independent_frozen
+
+# Download the exp_combined to egs2/geolid/lid1
+hf download espnet/geolid_vl107only_independent_frozen --local-dir . --exclude "README.md" "meta.yaml" ".gitattributes"
+
+./run_voxlingua107_only.sh --skip_data_prep false --skip_train true --lid_config conf/voxlingua107_only/mms_ecapa_upcon_32_44_it0.4_independent_frozen.yaml
 ```
 
+This will download the pre-trained model and run inference using the VoxLingua107 test data.
 
+### Train and Evaluation Datasets
+
+The training used only the VoxLingua107 dataset, comprising 6,628 hours of speech across 107 languages from YouTube.
+
+| Dataset       | Domain      | #Langs. Train/Test | Dialect | Training Setup (VL107-only) |
+| ------------- | ----------- | ------------------ | ------- | --------------------------- |
+| [VoxLingua107](https://cs.taltech.ee/staff/tanel.alumae/data/voxlingua107/)  | YouTube     | 107/33             | No      | Seen                        |
+| [Babel](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=31a13cefb42647e924e0d2778d341decc44c40e9)         | Telephone   | 25/25              | No      | Unseen                      |
+| [FLEURS](https://huggingface.co/datasets/google/xtreme_s)        | Read speech | 102/102            | No      | Unseen                      |
+| [ML-SUPERB 2.0](https://huggingface.co/datasets/espnet/ml_superb_hf) | Mixed       | 137/(137, 8)       | Yes     | Unseen                      |
+| [VoxPopuli](https://huggingface.co/datasets/facebook/voxpopuli)     | Parliament  | 16/16              | No      | Unseen                      |
+
+### Results
+
+**Accuracy (%) on In-domain and Out-of-domain Test Sets**
+
+<style>
+.hf-model-cell {
+    max-width: 120px;
+    overflow-x: auto;
+    white-space: nowrap;
+    scrollbar-width: thin;
+    scrollbar-color: #888 #f1f1f1;
+}
+
+.config-cell {
+    max-width: 100px;
+    overflow-x: auto;
+    white-space: nowrap;
+    scrollbar-width: thin;
+    scrollbar-color: #888 #f1f1f1;
+}
+
+.hf-model-cell::-webkit-scrollbar,
+.config-cell::-webkit-scrollbar {
+    height: 6px;
+}
+
+.hf-model-cell::-webkit-scrollbar-track,
+.config-cell::-webkit-scrollbar-track {
+    background: #f1f1f1;
+    border-radius: 3px;
+}
+
+.hf-model-cell::-webkit-scrollbar-thumb,
+.config-cell::-webkit-scrollbar-thumb {
+    background: #888;
+    border-radius: 3px;
+}
+
+.hf-model-cell::-webkit-scrollbar-thumb:hover,
+.config-cell::-webkit-scrollbar-thumb:hover {
+    background: #555;
+}
+</style>
+
+<div style="overflow-x: auto;">
+
+| ESPnet Recipe                    | Config | VoxLingua107 | Babel | FLEURS | ML-SUPERB2.0 Dev | ML-SUPERB2.0 Dialect | VoxPopuli | Macro Avg. |
+| ------------------------- | ----------- | ------------ | ----- | ------ | ---------------- | -------------------- | --------- | ---------- |
+| <div class="hf-model-cell">[egs2/geolid/lid1](https://github.com/espnet/espnet/tree/master/egs2/geolid/lid1)</div> | <div class="config-cell">`conf/voxlingua107_only/mms_ecapa_upcon_32_44_it0.4_independent_frozen.yaml`</div> | 94.2         | 87.1  | 95.0   | 89.0             | 77.2                 | 90.4      | 88.8       |
+
+</div>
+
+For more detailed inference results, please refer to the `exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/inference` directory in this repository.
+
+> **Note (2025-08-18):**  
+> The corresponding GitHub recipe [egs2/geolid/lid1](https://github.com/espnet/espnet/tree/master/egs2/geolid/lid1) has not yet been merged into the ESPnet master branch.  
+> See TODO: add PR link for the latest updates.
 
 ## LID config
 
@@ -276,9 +360,16 @@ distributed: false
 
 
 
-### Citing ESPnet
+### Citation
 
 ```BibTex
+@inproceedings{wang2025geolid,
+  author={Qingzheng Wang, Hye-jin Shim, Jiancheng Sun, and Shinji Watanabe},
+  title={Geolocation-Aware Robust Spoken Language Identification},
+  year={2025},
+  booktitle={Procedings of ASRU},
+}
+
 @inproceedings{watanabe2018espnet,
   author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},
   title={{ESPnet}: End-to-End Speech Processing Toolkit},
@@ -288,23 +379,4 @@ distributed: false
   doi={10.21437/Interspeech.2018-1456},
   url={http://dx.doi.org/10.21437/Interspeech.2018-1456}
 }
-
-
-
-
-
-
-```
-
-or arXiv:
-
-```bibtex
-@misc{watanabe2018espnet,
-  title={ESPnet: End-to-End Speech Processing Toolkit},
-  author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},
-  year={2018},
-  eprint={1804.00015},
-  archivePrefix={arXiv},
-  primaryClass={cs.CL}
-}
 ```

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_voxlingua107_lang/lid_inference_test.log

@@ -0,0 +1,290 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Tue May 27 16:46:05 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue08] 2025-05-27 16:46:20,609 (abs_task:2406) INFO: config file: exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue08] 2025-05-27 16:46:34,331 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamLang2VecConditionModel(
+  (frontend): S3prlFrontendLang2VecCondition(
+    (upstream): S3PRLUpstreamLang2VecCondition(
+      (upstream): UpstreamExpertLang2VecCondition(
+        (model): Wav2Vec2ModelLang2VecCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderLang2VecCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (conditioning_projs): ModuleDict(
+              (32): Linear(in_features=299, out_features=1280, bias=True)
+              (36): Linear(in_features=299, out_features=1280, bias=True)
+              (40): Linear(in_features=299, out_features=1280, bias=True)
+              (44): Linear(in_features=299, out_features=1280, bias=True)
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamLang2VecConditionModel
+    Total Number of model parameters: 978.26 M
+    Number of trainable parameters: 978.26 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue08] 2025-05-27 16:46:34,705 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_voxlingua107_lang/lids0
+[gpue08] 2025-05-27 16:47:26,844 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue08] 2025-05-27 16:48:11,546 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue08] 2025-05-27 16:49:00,081 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue08] 2025-05-27 16:49:43,568 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue08] 2025-05-27 16:50:28,280 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue08] 2025-05-27 16:51:12,200 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue08] 2025-05-27 16:51:52,954 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue08] 2025-05-27 16:52:36,802 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7
+[gpue08] 2025-05-27 16:53:21,771 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8
+[gpue08] 2025-05-27 16:54:15,927 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9
+[gpue08] 2025-05-27 16:55:04,249 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10
+[gpue08] 2025-05-27 16:55:53,390 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11
+[gpue08] 2025-05-27 16:56:40,872 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12
+[gpue08] 2025-05-27 16:57:29,245 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13
+[gpue08] 2025-05-27 16:58:21,026 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14
+[gpue08] 2025-05-27 16:59:01,689 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15
+[gpue08] 2025-05-27 16:59:03,702 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True
+[gpue08] 2025-05-27 16:59:03,703 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=779 threads=1
+# Ended (code 0) at Tue May 27 16:59:04 CDT 2025, elapsed time 779 seconds

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_voxlingua107_lang/lid_inference_test.log

@@ -0,0 +1,281 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Fri May 16 16:43:45 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue02] 2025-05-16 16:44:01,140 (abs_task:2341) INFO: config file: exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2364: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue02] 2025-05-16 16:44:11,133 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamLang2VecConditionModel(
+  (frontend): S3prlFrontendLang2VecCondition(
+    (upstream): S3PRLUpstreamLang2VecCondition(
+      (upstream): UpstreamExpertLang2VecCondition(
+        (model): Wav2Vec2ModelLang2VecCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderLang2VecCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (conditioning_projs): ModuleDict(
+              (32): Linear(in_features=299, out_features=1280, bias=True)
+              (36): Linear(in_features=299, out_features=1280, bias=True)
+              (40): Linear(in_features=299, out_features=1280, bias=True)
+              (44): Linear(in_features=299, out_features=1280, bias=True)
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamLang2VecConditionModel
+    Total Number of model parameters: 978.26 M
+    Number of trainable parameters: 978.26 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue02] 2025-05-16 16:44:11,480 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_voxlingua107_lang/lids0
+[gpue02] 2025-05-16 16:44:41,063 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue02] 2025-05-16 16:45:04,619 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue02] 2025-05-16 16:45:26,674 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue02] 2025-05-16 16:45:48,892 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue02] 2025-05-16 16:46:12,926 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue02] 2025-05-16 16:46:57,918 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue02] 2025-05-16 16:47:28,796 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue02] 2025-05-16 16:47:31,224 (lid_inference_dist:200) INFO: args.save_embd_per_utt: False
+[gpue02] 2025-05-16 16:47:31,225 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=227 threads=1
+# Ended (code 0) at Fri May 16 16:47:32 CDT 2025, elapsed time 227 seconds

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_voxlingua107_lang/results

@@ -0,0 +1,1628 @@
+Accuracy: 77.21%
+Macro Accuracy: 85.60%
+Accuracy per Language:
+ara: 70.37%
+guj: 98.95%
+spa: 97.81%
+ell: 71.37%
+tam: 99.08%
+deu: 73.48%
+tel: 99.01%
+eng: 74.73%
+Key: ara_sada_acw_000000, Target: ara, Predicted: tur
+Key: ara_sada_acw_000001, Target: ara, Predicted: hau
+Key: ara_sada_acw_000004, Target: ara, Predicted: tuk
+Key: ara_sada_acw_000005, Target: ara, Predicted: grn
+Key: ara_sada_acw_000006, Target: ara, Predicted: ceb
+Key: ara_sada_acw_000008, Target: ara, Predicted: hau
+Key: ara_sada_acw_000009, Target: ara, Predicted: bos
+Key: ara_sada_acw_000011, Target: ara, Predicted: ben
+Key: ara_sada_acw_000014, Target: ara, Predicted: amh
+Key: ara_sada_acw_000023, Target: ara, Predicted: jav
+Key: ara_sada_acw_000025, Target: ara, Predicted: som
+Key: ara_sada_acw_000029, Target: ara, Predicted: mar
+Key: ara_sada_acw_000033, Target: ara, Predicted: som
+Key: ara_sada_acw_000038, Target: ara, Predicted: eng
+Key: ara_sada_acw_000044, Target: ara, Predicted: hau
+Key: ara_sada_acw_000045, Target: ara, Predicted: fas
+Key: ara_sada_acw_000048, Target: ara, Predicted: eng
+Key: ara_sada_acw_000056, Target: ara, Predicted: hat
+Key: ara_sada_acw_000057, Target: ara, Predicted: hau
+Key: ara_sada_acw_000063, Target: ara, Predicted: amh
+Key: ara_sada_acw_000065, Target: ara, Predicted: tgl
+Key: ara_sada_acw_000067, Target: ara, Predicted: ben
+Key: ara_sada_acw_000069, Target: ara, Predicted: hat
+Key: ara_sada_acw_000071, Target: ara, Predicted: snd
+Key: ara_sada_acw_000075, Target: ara, Predicted: cmn
+Key: ara_sada_acw_000077, Target: ara, Predicted: amh
+Key: ara_sada_acw_000078, Target: ara, Predicted: amh
+Key: ara_sada_acw_000079, Target: ara, Predicted: aze
+Key: ara_sada_acw_000080, Target: ara, Predicted: hau
+Key: ara_sada_acw_000083, Target: ara, Predicted: jav
+Key: ara_sada_acw_000084, Target: ara, Predicted: amh
+Key: ara_sada_acw_000087, Target: ara, Predicted: ceb
+Key: ara_sada_acw_000094, Target: ara, Predicted: tat
+Key: ara_sada_acw_000096, Target: ara, Predicted: hau
+Key: ara_sada_acw_000099, Target: ara, Predicted: amh
+Key: ara_sada_acw_000101, Target: ara, Predicted: cmn
+Key: ara_sada_acw_000103, Target: ara, Predicted: hau
+Key: ara_sada_acw_000106, Target: ara, Predicted: aze
+Key: ara_sada_acw_000110, Target: ara, Predicted: lin
+Key: ara_sada_acw_000116, Target: ara, Predicted: fas
+Key: ara_sada_acw_000117, Target: ara, Predicted: ben
+Key: ara_sada_acw_000118, Target: ara, Predicted: hun
+Key: ara_sada_acw_000123, Target: ara, Predicted: som
+Key: ara_sada_acw_000124, Target: ara, Predicted: hau
+Key: ara_sada_acw_000125, Target: ara, Predicted: heb
+Key: ara_sada_acw_000127, Target: ara, Predicted: nno
+Key: ara_sada_acw_000130, Target: ara, Predicted: swa
+Key: ara_sada_acw_000131, Target: ara, Predicted: mya
+Key: ara_sada_acw_000132, Target: ara, Predicted: jav
+Key: ara_sada_acw_000139, Target: ara, Predicted: jav
+Key: ara_sada_acw_000140, Target: ara, Predicted: ces
+Key: ara_sada_acw_000147, Target: ara, Predicted: mlt
+Key: ara_sada_acw_000149, Target: ara, Predicted: aze
+Key: ara_sada_acw_000153, Target: ara, Predicted: mlg
+Key: ara_sada_acw_000161, Target: ara, Predicted: som
+Key: ara_sada_acw_000162, Target: ara, Predicted: ell
+Key: ara_sada_acw_000163, Target: ara, Predicted: msa
+Key: ara_sada_acw_000164, Target: ara, Predicted: mlt
+Key: ara_sada_acw_000171, Target: ara, Predicted: urd
+Key: ara_sada_acw_000173, Target: ara, Predicted: lav
+Key: ara_sada_afb_000002, Target: ara, Predicted: tat
+Key: ara_sada_afb_000003, Target: ara, Predicted: kan
+Key: ara_sada_afb_000007, Target: ara, Predicted: pus
+Key: ara_sada_afb_000009, Target: ara, Predicted: snd
+Key: ara_sada_afb_000010, Target: ara, Predicted: mlt
+Key: ara_sada_afb_000011, Target: ara, Predicted: guj
+Key: ara_sada_afb_000016, Target: ara, Predicted: amh
+Key: ara_sada_afb_000018, Target: ara, Predicted: amh
+Key: ara_sada_afb_000020, Target: ara, Predicted: som
+Key: ara_sada_afb_000021, Target: ara, Predicted: ell
+Key: ara_sada_afb_000030, Target: ara, Predicted: amh
+Key: ara_sada_afb_000032, Target: ara, Predicted: kaz
+Key: ara_sada_afb_000042, Target: ara, Predicted: yid
+Key: ara_sada_afb_000043, Target: ara, Predicted: amh
+Key: ara_sada_afb_000046, Target: ara, Predicted: urd
+Key: ara_sada_afb_000047, Target: ara, Predicted: hau
+Key: ara_sada_afb_000050, Target: ara, Predicted: som
+Key: ara_sada_afb_000051, Target: ara, Predicted: snd
+Key: ara_sada_afb_000053, Target: ara, Predicted: som
+Key: ara_sada_afb_000057, Target: ara, Predicted: bod
+Key: ara_sada_afb_000058, Target: ara, Predicted: hau
+Key: ara_sada_afb_000071, Target: ara, Predicted: heb
+Key: ara_sada_afb_000072, Target: ara, Predicted: jav
+Key: ara_sada_afb_000073, Target: ara, Predicted: mya
+Key: ara_sada_afb_000080, Target: ara, Predicted: asm
+Key: ara_sada_afb_000084, Target: ara, Predicted: msa
+Key: ara_sada_afb_000085, Target: ara, Predicted: amh
+Key: ara_sada_afb_000088, Target: ara, Predicted: swa
+Key: ara_sada_afb_000094, Target: ara, Predicted: nno
+Key: ara_sada_afb_000099, Target: ara, Predicted: afr
+Key: ara_sada_afb_000104, Target: ara, Predicted: hau
+Key: ara_sada_afb_000106, Target: ara, Predicted: heb
+Key: ara_sada_afb_000108, Target: ara, Predicted: ben
+Key: ara_sada_afb_000117, Target: ara, Predicted: bod
+Key: ara_sada_afb_000118, Target: ara, Predicted: mya
+Key: ara_sada_afb_000119, Target: ara, Predicted: hrv
+Key: ara_sada_afb_000121, Target: ara, Predicted: ell
+Key: ara_sada_afb_000124, Target: ara, Predicted: khm
+Key: ara_sada_afb_000129, Target: ara, Predicted: hau
+Key: ara_sada_afb_000132, Target: ara, Predicted: yid
+Key: ara_sada_afb_000136, Target: ara, Predicted: heb
+Key: ara_sada_afb_000138, Target: ara, Predicted: tat
+Key: ara_sada_afb_000140, Target: ara, Predicted: amh
+Key: ara_sada_afb_000141, Target: ara, Predicted: heb
+Key: ara_sada_afb_000143, Target: ara, Predicted: bod
+Key: ara_sada_afb_000147, Target: ara, Predicted: fas
+Key: ara_sada_afb_000148, Target: ara, Predicted: tur
+Key: ara_sada_afb_000149, Target: ara, Predicted: som
+Key: ara_sada_afb_000151, Target: ara, Predicted: hun
+Key: ara_sada_afb_000153, Target: ara, Predicted: mlt
+Key: ara_sada_afb_000155, Target: ara, Predicted: grn
+Key: ara_sada_afb_000156, Target: ara, Predicted: amh
+Key: ara_sada_afb_000159, Target: ara, Predicted: fas
+Key: ara_sada_afb_000161, Target: ara, Predicted: som
+Key: ara_sada_afb_000162, Target: ara, Predicted: tat
+Key: ara_sada_afb_000164, Target: ara, Predicted: amh
+Key: ara_sada_afb_000166, Target: ara, Predicted: fas
+Key: ara_sada_afb_000167, Target: ara, Predicted: heb
+Key: ara_sada_afb_000168, Target: ara, Predicted: nno
+Key: ara_sada_afb_000173, Target: ara, Predicted: amh
+Key: ara_sada_afb_000181, Target: ara, Predicted: urd
+Key: ara_sada_afb_000183, Target: ara, Predicted: sqi
+Key: ara_sada_afb_000184, Target: ara, Predicted: hau
+Key: ara_sada_afb_000188, Target: ara, Predicted: hin
+Key: ara_sada_afb_000190, Target: ara, Predicted: heb
+Key: ara_sada_afb_000192, Target: ara, Predicted: amh
+Key: ara_sada_afb_000193, Target: ara, Predicted: bos
+Key: ara_sada_afb_000194, Target: ara, Predicted: tat
+Key: ara_sada_afb_000196, Target: ara, Predicted: aze
+Key: ara_sada_afb_000197, Target: ara, Predicted: hau
+Key: ara_sada_afb_000203, Target: ara, Predicted: som
+Key: ara_sada_afb_000208, Target: ara, Predicted: snd
+Key: ara_sada_afb_000212, Target: ara, Predicted: heb
+Key: ara_sada_afb_000220, Target: ara, Predicted: heb
+Key: ara_sada_afb_000221, Target: ara, Predicted: som
+Key: ara_sada_arb_000005, Target: ara, Predicted: heb
+Key: ara_sada_arb_000024, Target: ara, Predicted: amh
+Key: ara_sada_arb_000031, Target: ara, Predicted: yor
+Key: ara_sada_arb_000033, Target: ara, Predicted: yor
+Key: ara_sada_arb_000041, Target: ara, Predicted: mlg
+Key: ara_sada_ars_000007, Target: ara, Predicted: amh
+Key: ara_sada_ars_000008, Target: ara, Predicted: amh
+Key: ara_sada_ars_000016, Target: ara, Predicted: ell
+Key: ara_sada_ars_000018, Target: ara, Predicted: hau
+Key: ara_sada_ars_000019, Target: ara, Predicted: bos
+Key: ara_sada_ars_000030, Target: ara, Predicted: amh
+Key: ara_sada_ars_000031, Target: ara, Predicted: snd
+Key: ara_sada_ars_000032, Target: ara, Predicted: bak
+Key: ara_sada_ars_000033, Target: ara, Predicted: hau
+Key: ara_sada_ars_000038, Target: ara, Predicted: lav
+Key: ara_sada_ars_000039, Target: ara, Predicted: msa
+Key: ara_sada_ars_000042, Target: ara, Predicted: slk
+Key: ara_sada_ars_000043, Target: ara, Predicted: dan
+Key: ara_sada_ars_000044, Target: ara, Predicted: lav
+Key: ara_sada_ars_000053, Target: ara, Predicted: slv
+Key: ara_sada_ars_000055, Target: ara, Predicted: eng
+Key: ara_sada_ars_000058, Target: ara, Predicted: mya
+Key: ara_sada_ars_000061, Target: ara, Predicted: isl
+Key: ara_sada_ars_000069, Target: ara, Predicted: sqi
+Key: ara_sada_ars_000072, Target: ara, Predicted: snd
+Key: ara_sada_ars_000075, Target: ara, Predicted: lao
+Key: ara_sada_ars_000082, Target: ara, Predicted: san
+Key: ara_sada_ars_000090, Target: ara, Predicted: guj
+Key: ara_sada_ars_000091, Target: ara, Predicted: heb
+Key: ara_sada_ars_000092, Target: ara, Predicted: yor
+Key: ara_sada_ars_000098, Target: ara, Predicted: sna
+Key: ara_sada_ars_000105, Target: ara, Predicted: cmn
+Key: ara_sada_ars_000110, Target: ara, Predicted: amh
+Key: ara_sada_ars_000115, Target: ara, Predicted: tat
+Key: ara_sada_ars_000121, Target: ara, Predicted: fra
+Key: ara_sada_ars_000123, Target: ara, Predicted: isl
+Key: ara_sada_ars_000129, Target: ara, Predicted: urd
+Key: ara_sada_ars_000136, Target: ara, Predicted: nno
+Key: ara_sada_ars_000140, Target: ara, Predicted: som
+Key: ara_sada_ars_000146, Target: ara, Predicted: ita
+Key: ara_sada_ars_000152, Target: ara, Predicted: amh
+Key: deu_swissdial_ag_000003, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000008, Target: deu, Predicted: tat
+Key: deu_swissdial_ag_000013, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000014, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000015, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000016, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000023, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000025, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000026, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000031, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000032, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000035, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000037, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000038, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000040, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000042, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000047, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000055, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000059, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000062, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000064, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000073, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000076, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000084, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000085, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000086, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000088, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000089, Target: deu, Predicted: yid
+Key: deu_swissdial_ag_000091, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000092, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000093, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000095, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000096, Target: deu, Predicted: slk
+Key: deu_swissdial_ag_000100, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000107, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000108, Target: deu, Predicted: ltz
+Key: deu_swissdial_ag_000110, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000114, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000117, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000118, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000120, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000121, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000122, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000124, Target: deu, Predicted: hun
+Key: deu_swissdial_ag_000126, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000127, Target: deu, Predicted: hun
+Key: deu_swissdial_ag_000128, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000134, Target: deu, Predicted: sqi
+Key: deu_swissdial_ag_000135, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000138, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000139, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000141, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000148, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000150, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000151, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000159, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000163, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000004, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000008, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000013, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000016, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000018, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000023, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000024, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000025, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000030, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000031, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000033, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000034, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000035, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000036, Target: deu, Predicted: slv
+Key: deu_swissdial_be_000039, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000040, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000042, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000043, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000048, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000054, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000059, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000060, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000066, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000076, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000084, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000087, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000090, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000091, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000092, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000096, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000103, Target: deu, Predicted: est
+Key: deu_swissdial_be_000107, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000108, Target: deu, Predicted: ltz
+Key: deu_swissdial_be_000112, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000113, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000114, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000115, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000117, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000124, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000125, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000129, Target: deu, Predicted: afr
+Key: deu_swissdial_bs_000003, Target: deu, Predicted: ltz
+Key: deu_swissdial_bs_000008, Target: deu, Predicted: dan
+Key: deu_swissdial_bs_000019, Target: deu, Predicted: dan
+Key: deu_swissdial_bs_000031, Target: deu, Predicted: lat
+Key: deu_swissdial_bs_000036, Target: deu, Predicted: ltz
+Key: deu_swissdial_bs_000039, Target: deu, Predicted: dan
+Key: deu_swissdial_bs_000079, Target: deu, Predicted: ltz
+Key: deu_swissdial_bs_000082, Target: deu, Predicted: ltz
+Key: deu_swissdial_bs_000088, Target: deu, Predicted: ltz
+Key: deu_swissdial_bs_000093, Target: deu, Predicted: nld
+Key: deu_swissdial_bs_000097, Target: deu, Predicted: nno
+Key: deu_swissdial_bs_000114, Target: deu, Predicted: afr
+Key: deu_swissdial_bs_000133, Target: deu, Predicted: nld
+Key: deu_swissdial_bs_000139, Target: deu, Predicted: afr
+Key: deu_swissdial_gr_000010, Target: deu, Predicted: afr
+Key: deu_swissdial_gr_000040, Target: deu, Predicted: afr
+Key: deu_swissdial_gr_000059, Target: deu, Predicted: slv
+Key: deu_swissdial_gr_000063, Target: deu, Predicted: slv
+Key: deu_swissdial_gr_000064, Target: deu, Predicted: slv
+Key: deu_swissdial_gr_000081, Target: deu, Predicted: afr
+Key: deu_swissdial_gr_000088, Target: deu, Predicted: nld
+Key: deu_swissdial_gr_000105, Target: deu, Predicted: afr
+Key: deu_swissdial_gr_000114, Target: deu, Predicted: slv
+Key: deu_swissdial_gr_000115, Target: deu, Predicted: dan
+Key: deu_swissdial_gr_000123, Target: deu, Predicted: slk
+Key: deu_swissdial_gr_000133, Target: deu, Predicted: slv
+Key: deu_swissdial_lu_000002, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000003, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000004, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000006, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000007, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000009, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000011, Target: deu, Predicted: hun
+Key: deu_swissdial_lu_000012, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000014, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000016, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000017, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000019, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000021, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000022, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000023, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000024, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000026, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000027, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000029, Target: deu, Predicted: slk
+Key: deu_swissdial_lu_000031, Target: deu, Predicted: yid
+Key: deu_swissdial_lu_000033, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000034, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000036, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000037, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000038, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000040, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000042, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000043, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000044, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000046, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000047, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000051, Target: deu, Predicted: cym
+Key: deu_swissdial_lu_000052, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000053, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000054, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000055, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000056, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000057, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000058, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000065, Target: deu, Predicted: hun
+Key: deu_swissdial_lu_000066, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000067, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000069, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000071, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000072, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000073, Target: deu, Predicted: isl
+Key: deu_swissdial_lu_000075, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000077, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000079, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000080, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000081, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000085, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000086, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000087, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000090, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000095, Target: deu, Predicted: isl
+Key: deu_swissdial_lu_000096, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000097, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000099, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000102, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000103, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000104, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000106, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000107, Target: deu, Predicted: est
+Key: deu_swissdial_lu_000108, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000112, Target: deu, Predicted: yid
+Key: deu_swissdial_lu_000113, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000116, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000117, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000120, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000121, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000123, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000125, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000126, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000129, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000130, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000132, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000133, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000134, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000135, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000136, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000137, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000138, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000140, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000144, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000146, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000150, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000151, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000153, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000155, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000157, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000158, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000159, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000162, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000163, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000164, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000166, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000167, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000168, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000170, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000172, Target: deu, Predicted: nor
+Key: deu_swissdial_lu_000173, Target: deu, Predicted: afr
+Key: deu_swissdial_sg_000010, Target: deu, Predicted: nld
+Key: deu_swissdial_sg_000046, Target: deu, Predicted: dan
+Key: deu_swissdial_sg_000078, Target: deu, Predicted: nld
+Key: deu_swissdial_sg_000104, Target: deu, Predicted: est
+Key: deu_swissdial_sg_000109, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000003, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000004, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000005, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000009, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000010, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000016, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000020, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000021, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000029, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000030, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000032, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000034, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000038, Target: deu, Predicted: yid
+Key: deu_swissdial_vs_000042, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000044, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000045, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000047, Target: deu, Predicted: dan
+Key: deu_swissdial_vs_000048, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000053, Target: deu, Predicted: dan
+Key: deu_swissdial_vs_000055, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000059, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000060, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000064, Target: deu, Predicted: nno
+Key: deu_swissdial_vs_000073, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000075, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000076, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000078, Target: deu, Predicted: cym
+Key: deu_swissdial_vs_000082, Target: deu, Predicted: dan
+Key: deu_swissdial_vs_000083, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000088, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000090, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000091, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000096, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000102, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000105, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000108, Target: deu, Predicted: slv
+Key: deu_swissdial_vs_000114, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000116, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000120, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000124, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000127, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000129, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000132, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000136, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000137, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000000, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000006, Target: deu, Predicted: ltz
+Key: deu_swissdial_zh_000007, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000015, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000017, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000029, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000032, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000050, Target: deu, Predicted: ltz
+Key: deu_swissdial_zh_000060, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000066, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000072, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000078, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000080, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000081, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000083, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000086, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000088, Target: deu, Predicted: slk
+Key: deu_swissdial_zh_000091, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000094, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000097, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000101, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000103, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000105, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000115, Target: deu, Predicted: ltz
+Key: deu_swissdial_zh_000116, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000120, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000127, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000133, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000134, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000141, Target: deu, Predicted: afr
+Key: ell_cretan_cre_000002, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000003, Target: ell, Predicted: tgl
+Key: ell_cretan_cre_000006, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000007, Target: ell, Predicted: srp
+Key: ell_cretan_cre_000011, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000013, Target: ell, Predicted: snd
+Key: ell_cretan_cre_000014, Target: ell, Predicted: bos
+Key: ell_cretan_cre_000017, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000019, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000020, Target: ell, Predicted: ind
+Key: ell_cretan_cre_000031, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000033, Target: ell, Predicted: lin
+Key: ell_cretan_cre_000034, Target: ell, Predicted: bel
+Key: ell_cretan_cre_000036, Target: ell, Predicted: rus
+Key: ell_cretan_cre_000037, Target: ell, Predicted: bul
+Key: ell_cretan_cre_000039, Target: ell, Predicted: tel
+Key: ell_cretan_cre_000040, Target: ell, Predicted: spa
+Key: ell_cretan_cre_000041, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000048, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000050, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000052, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000053, Target: ell, Predicted: lao
+Key: ell_cretan_cre_000054, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000055, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000059, Target: ell, Predicted: mlg
+Key: ell_cretan_cre_000063, Target: ell, Predicted: mlg
+Key: ell_cretan_cre_000064, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000065, Target: ell, Predicted: mya
+Key: ell_cretan_cre_000066, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000067, Target: ell, Predicted: cat
+Key: ell_cretan_cre_000071, Target: ell, Predicted: tel
+Key: ell_cretan_cre_000078, Target: ell, Predicted: por
+Key: ell_cretan_cre_000081, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000084, Target: ell, Predicted: nno
+Key: ell_cretan_cre_000088, Target: ell, Predicted: bos
+Key: ell_cretan_cre_000091, Target: ell, Predicted: mlg
+Key: ell_cretan_cre_000092, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000094, Target: ell, Predicted: grn
+Key: ell_cretan_cre_000096, Target: ell, Predicted: mon
+Key: ell_cretan_cre_000097, Target: ell, Predicted: aze
+Key: ell_cretan_cre_000098, Target: ell, Predicted: pus
+Key: ell_cretan_cre_000099, Target: ell, Predicted: vie
+Key: ell_cretan_cre_000100, Target: ell, Predicted: mlg
+Key: ell_cretan_cre_000101, Target: ell, Predicted: grn
+Key: ell_cretan_cre_000103, Target: ell, Predicted: lat
+Key: ell_cretan_cre_000107, Target: ell, Predicted: tha
+Key: ell_cretan_cre_000110, Target: ell, Predicted: bul
+Key: ell_cretan_cre_000112, Target: ell, Predicted: guj
+Key: ell_cretan_cre_000120, Target: ell, Predicted: por
+Key: ell_cretan_cre_000126, Target: ell, Predicted: mlg
+Key: ell_cretan_cre_000128, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000138, Target: ell, Predicted: glg
+Key: ell_cretan_cre_000139, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000140, Target: ell, Predicted: mya
+Key: ell_cretan_cre_000147, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000149, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000150, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000151, Target: ell, Predicted: sun
+Key: ell_cretan_cre_000153, Target: ell, Predicted: lit
+Key: ell_cretan_cre_000156, Target: ell, Predicted: lao
+Key: ell_cretan_cre_000157, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000160, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000161, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000162, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000163, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000167, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000168, Target: ell, Predicted: hin
+Key: ell_cretan_cre_000170, Target: ell, Predicted: mri
+Key: ell_cretan_cre_000171, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000172, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000173, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000174, Target: ell, Predicted: bel
+Key: ell_cretan_cre_000177, Target: ell, Predicted: lit
+Key: ell_cretan_cre_000185, Target: ell, Predicted: bod
+Key: ell_cretan_cre_000187, Target: ell, Predicted: por
+Key: ell_cretan_cre_000188, Target: ell, Predicted: lav
+Key: ell_cretan_cre_000189, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000195, Target: ell, Predicted: por
+Key: ell_cretan_cre_000196, Target: ell, Predicted: tel
+Key: ell_cretan_cre_000197, Target: ell, Predicted: por
+Key: ell_cretan_cre_000203, Target: ell, Predicted: kan
+Key: ell_cretan_cre_000206, Target: ell, Predicted: aze
+Key: ell_cretan_cre_000210, Target: ell, Predicted: bos
+Key: ell_cretan_cre_000216, Target: ell, Predicted: lit
+Key: ell_cretan_cre_000219, Target: ell, Predicted: hat
+Key: ell_cretan_cre_000221, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000225, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000226, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000228, Target: ell, Predicted: bel
+Key: ell_cretan_cre_000230, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000232, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000233, Target: ell, Predicted: fin
+Key: ell_cretan_cre_000235, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000237, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000239, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000240, Target: ell, Predicted: lit
+Key: ell_cretan_cre_000241, Target: ell, Predicted: pus
+Key: ell_cretan_cre_000242, Target: ell, Predicted: pus
+Key: ell_cretan_cre_000245, Target: ell, Predicted: por
+Key: ell_cretan_cre_000248, Target: ell, Predicted: amh
+Key: ell_cretan_cre_000249, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000250, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000251, Target: ell, Predicted: tgl
+Key: ell_cretan_cre_000252, Target: ell, Predicted: lin
+Key: ell_cretan_cre_000254, Target: ell, Predicted: lit
+Key: ell_cretan_cre_000261, Target: ell, Predicted: asm
+Key: ell_cretan_cre_000264, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000265, Target: ell, Predicted: ceb
+Key: ell_cretan_cre_000266, Target: ell, Predicted: bel
+Key: ell_cretan_cre_000267, Target: ell, Predicted: spa
+Key: ell_cretan_cre_000268, Target: ell, Predicted: bod
+Key: ell_cretan_cre_000270, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000274, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000275, Target: ell, Predicted: bel
+Key: ell_cretan_cre_000281, Target: ell, Predicted: ben
+Key: ell_cretan_cre_000282, Target: ell, Predicted: mlg
+Key: ell_cretan_cre_000283, Target: ell, Predicted: lit
+Key: ell_cretan_cre_000285, Target: ell, Predicted: mar
+Key: ell_cretan_cre_000286, Target: ell, Predicted: khm
+Key: ell_cretan_cre_000288, Target: ell, Predicted: mlg
+Key: ell_messenian_mes_000009, Target: ell, Predicted: eus
+Key: ell_messenian_mes_000011, Target: ell, Predicted: afr
+Key: ell_messenian_mes_000012, Target: ell, Predicted: tgl
+Key: ell_messenian_mes_000043, Target: ell, Predicted: slv
+Key: ell_messenian_mes_000056, Target: ell, Predicted: nno
+Key: ell_messenian_mes_000058, Target: ell, Predicted: asm
+Key: ell_messenian_mes_000077, Target: ell, Predicted: ces
+Key: ell_messenian_mes_000079, Target: ell, Predicted: afr
+Key: ell_messenian_mes_000085, Target: ell, Predicted: ita
+Key: ell_messenian_mes_000099, Target: ell, Predicted: mkd
+Key: ell_messenian_mes_000110, Target: ell, Predicted: ind
+Key: ell_messenian_mes_000136, Target: ell, Predicted: nno
+Key: ell_messenian_mes_000139, Target: ell, Predicted: slv
+Key: ell_messenian_mes_000141, Target: ell, Predicted: mkd
+Key: ell_messenian_mes_000155, Target: ell, Predicted: mri
+Key: ell_messenian_mes_000161, Target: ell, Predicted: hrv
+Key: ell_messenian_mes_000164, Target: ell, Predicted: cat
+Key: ell_messenian_mes_000182, Target: ell, Predicted: tgl
+Key: eng_globe_aus_000015, Target: eng, Predicted: fra
+Key: eng_globe_aus_000039, Target: eng, Predicted: ceb
+Key: eng_globe_aus_000054, Target: eng, Predicted: bul
+Key: eng_globe_aus_000082, Target: eng, Predicted: bod
+Key: eng_globe_aus_000118, Target: eng, Predicted: slv
+Key: eng_globe_bre_000034, Target: eng, Predicted: hun
+Key: eng_globe_bre_000050, Target: eng, Predicted: glv
+Key: eng_globe_bre_000052, Target: eng, Predicted: cym
+Key: eng_globe_bre_000054, Target: eng, Predicted: swe
+Key: eng_globe_bre_000089, Target: eng, Predicted: cym
+Key: eng_globe_bre_000099, Target: eng, Predicted: deu
+Key: eng_globe_bre_000100, Target: eng, Predicted: cym
+Key: eng_globe_bre_000103, Target: eng, Predicted: deu
+Key: eng_globe_bre_000107, Target: eng, Predicted: lit
+Key: eng_globe_bre_000116, Target: eng, Predicted: cym
+Key: eng_globe_bre_000130, Target: eng, Predicted: cym
+Key: eng_globe_bre_000133, Target: eng, Predicted: nor
+Key: eng_globe_bre_000144, Target: eng, Predicted: dan
+Key: eng_globe_bre_000152, Target: eng, Predicted: tgl
+Key: eng_globe_can_000033, Target: eng, Predicted: ell
+Key: eng_globe_can_000063, Target: eng, Predicted: cmn
+Key: eng_globe_can_000069, Target: eng, Predicted: mya
+Key: eng_globe_can_000112, Target: eng, Predicted: glv
+Key: eng_globe_can_000134, Target: eng, Predicted: cym
+Key: eng_globe_fil_000011, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000014, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000016, Target: eng, Predicted: khm
+Key: eng_globe_fil_000018, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000019, Target: eng, Predicted: kor
+Key: eng_globe_fil_000021, Target: eng, Predicted: war
+Key: eng_globe_fil_000026, Target: eng, Predicted: ces
+Key: eng_globe_fil_000027, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000032, Target: eng, Predicted: slk
+Key: eng_globe_fil_000033, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000036, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000042, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000043, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000045, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000064, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000070, Target: eng, Predicted: glv
+Key: eng_globe_fil_000077, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000080, Target: eng, Predicted: mya
+Key: eng_globe_fil_000087, Target: eng, Predicted: ceb
+Key: eng_globe_fil_000088, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000089, Target: eng, Predicted: cym
+Key: eng_globe_fil_000096, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000104, Target: eng, Predicted: fra
+Key: eng_globe_fil_000106, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000113, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000117, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000123, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000130, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000132, Target: eng, Predicted: ceb
+Key: eng_globe_fil_000133, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000146, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000152, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000155, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000156, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000158, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000159, Target: eng, Predicted: tgl
+Key: eng_globe_fil_000163, Target: eng, Predicted: tgl
+Key: eng_globe_gle_000007, Target: eng, Predicted: cym
+Key: eng_globe_gle_000020, Target: eng, Predicted: nld
+Key: eng_globe_gle_000022, Target: eng, Predicted: glv
+Key: eng_globe_gle_000045, Target: eng, Predicted: glv
+Key: eng_globe_gle_000050, Target: eng, Predicted: glv
+Key: eng_globe_gle_000069, Target: eng, Predicted: glv
+Key: eng_globe_gle_000079, Target: eng, Predicted: ita
+Key: eng_globe_gle_000087, Target: eng, Predicted: cym
+Key: eng_globe_gle_000150, Target: eng, Predicted: glv
+Key: eng_globe_gle_000154, Target: eng, Predicted: pan
+Key: eng_globe_gle_000159, Target: eng, Predicted: mri
+Key: eng_globe_gle_000162, Target: eng, Predicted: glv
+Key: eng_globe_gle_000167, Target: eng, Predicted: hin
+Key: eng_globe_nze_000004, Target: eng, Predicted: glv
+Key: eng_globe_nze_000020, Target: eng, Predicted: cym
+Key: eng_globe_nze_000029, Target: eng, Predicted: asm
+Key: eng_globe_nze_000094, Target: eng, Predicted: nld
+Key: eng_globe_nze_000108, Target: eng, Predicted: snd
+Key: eng_globe_nze_000118, Target: eng, Predicted: bul
+Key: eng_globe_nze_000127, Target: eng, Predicted: cym
+Key: eng_globe_nze_000150, Target: eng, Predicted: cym
+Key: eng_globe_sae_000007, Target: eng, Predicted: ron
+Key: eng_globe_sae_000008, Target: eng, Predicted: asm
+Key: eng_globe_sae_000011, Target: eng, Predicted: tam
+Key: eng_globe_sae_000015, Target: eng, Predicted: pan
+Key: eng_globe_sae_000035, Target: eng, Predicted: afr
+Key: eng_globe_sae_000045, Target: eng, Predicted: afr
+Key: eng_globe_sae_000047, Target: eng, Predicted: asm
+Key: eng_globe_sae_000050, Target: eng, Predicted: kan
+Key: eng_globe_sae_000058, Target: eng, Predicted: hin
+Key: eng_globe_sae_000062, Target: eng, Predicted: ben
+Key: eng_globe_sae_000063, Target: eng, Predicted: fra
+Key: eng_globe_sae_000066, Target: eng, Predicted: cym
+Key: eng_globe_sae_000079, Target: eng, Predicted: pan
+Key: eng_globe_sae_000090, Target: eng, Predicted: epo
+Key: eng_globe_sae_000100, Target: eng, Predicted: hin
+Key: eng_globe_sae_000102, Target: eng, Predicted: hun
+Key: eng_globe_sae_000104, Target: eng, Predicted: tgl
+Key: eng_globe_sae_000109, Target: eng, Predicted: glv
+Key: eng_globe_sae_000110, Target: eng, Predicted: afr
+Key: eng_globe_sae_000115, Target: eng, Predicted: pan
+Key: eng_globe_sae_000119, Target: eng, Predicted: mri
+Key: eng_globe_sae_000120, Target: eng, Predicted: cym
+Key: eng_globe_sae_000134, Target: eng, Predicted: tgl
+Key: eng_globe_sae_000140, Target: eng, Predicted: tuk
+Key: eng_globe_sae_000143, Target: eng, Predicted: slv
+Key: eng_globe_sae_000144, Target: eng, Predicted: sna
+Key: eng_globe_sae_000160, Target: eng, Predicted: fra
+Key: eng_globe_sae_000161, Target: eng, Predicted: tel
+Key: eng_globe_sae_000165, Target: eng, Predicted: hin
+Key: eng_globe_sae_000168, Target: eng, Predicted: ina
+Key: eng_globe_sae_000169, Target: eng, Predicted: tam
+Key: eng_globe_sco_000002, Target: eng, Predicted: cym
+Key: eng_globe_sco_000009, Target: eng, Predicted: cym
+Key: eng_globe_sco_000011, Target: eng, Predicted: glv
+Key: eng_globe_sco_000017, Target: eng, Predicted: glv
+Key: eng_globe_sco_000035, Target: eng, Predicted: glv
+Key: eng_globe_sco_000036, Target: eng, Predicted: cym
+Key: eng_globe_sco_000038, Target: eng, Predicted: sna
+Key: eng_globe_sco_000047, Target: eng, Predicted: cym
+Key: eng_globe_sco_000048, Target: eng, Predicted: glv
+Key: eng_globe_sco_000052, Target: eng, Predicted: cym
+Key: eng_globe_sco_000062, Target: eng, Predicted: cym
+Key: eng_globe_sco_000065, Target: eng, Predicted: glv
+Key: eng_globe_sco_000067, Target: eng, Predicted: lat
+Key: eng_globe_sco_000072, Target: eng, Predicted: glv
+Key: eng_globe_sco_000078, Target: eng, Predicted: cym
+Key: eng_globe_sco_000090, Target: eng, Predicted: glv
+Key: eng_globe_sco_000093, Target: eng, Predicted: glv
+Key: eng_globe_sco_000099, Target: eng, Predicted: cym
+Key: eng_globe_sco_000106, Target: eng, Predicted: cym
+Key: eng_globe_sco_000108, Target: eng, Predicted: glv
+Key: eng_globe_sco_000109, Target: eng, Predicted: glv
+Key: eng_globe_sco_000110, Target: eng, Predicted: glv
+Key: eng_globe_sco_000112, Target: eng, Predicted: glv
+Key: eng_globe_sco_000116, Target: eng, Predicted: cym
+Key: eng_globe_sco_000118, Target: eng, Predicted: isl
+Key: eng_globe_sco_000126, Target: eng, Predicted: cym
+Key: eng_globe_sco_000129, Target: eng, Predicted: glv
+Key: eng_globe_sco_000141, Target: eng, Predicted: glv
+Key: eng_globe_sco_000142, Target: eng, Predicted: glv
+Key: eng_globe_sco_000144, Target: eng, Predicted: nld
+Key: eng_globe_sco_000150, Target: eng, Predicted: por
+Key: eng_globe_use_000004, Target: eng, Predicted: glv
+Key: eng_globe_use_000023, Target: eng, Predicted: msa
+Key: eng_globe_use_000038, Target: eng, Predicted: dan
+Key: eng_globe_use_000057, Target: eng, Predicted: mri
+Key: eng_globe_use_000059, Target: eng, Predicted: afr
+Key: eng_globe_use_000061, Target: eng, Predicted: hau
+Key: eng_globe_use_000076, Target: eng, Predicted: khm
+Key: eng_globe_use_000098, Target: eng, Predicted: tel
+Key: eng_globe_use_000102, Target: eng, Predicted: deu
+Key: eng_globe_use_000115, Target: eng, Predicted: jpn
+Key: eng_globe_use_000164, Target: eng, Predicted: glv
+Key: eng_globe_use_000175, Target: eng, Predicted: tgl
+Key: eng_l2arctic_ara_000001, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000002, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000003, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000007, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000011, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000013, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000015, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000018, Target: eng, Predicted: tuk
+Key: eng_l2arctic_ara_000021, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000023, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000028, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000029, Target: eng, Predicted: ron
+Key: eng_l2arctic_ara_000030, Target: eng, Predicted: kan
+Key: eng_l2arctic_ara_000031, Target: eng, Predicted: ell
+Key: eng_l2arctic_ara_000032, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000037, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000038, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000039, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000042, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000044, Target: eng, Predicted: ina
+Key: eng_l2arctic_ara_000045, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000046, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000051, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000061, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000068, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000069, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000070, Target: eng, Predicted: nor
+Key: eng_l2arctic_ara_000072, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000074, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000077, Target: eng, Predicted: pol
+Key: eng_l2arctic_ara_000079, Target: eng, Predicted: fra
+Key: eng_l2arctic_ara_000080, Target: eng, Predicted: hun
+Key: eng_l2arctic_ara_000083, Target: eng, Predicted: tgk
+Key: eng_l2arctic_ara_000087, Target: eng, Predicted: amh
+Key: eng_l2arctic_ara_000088, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000092, Target: eng, Predicted: cym
+Key: eng_l2arctic_ara_000096, Target: eng, Predicted: hau
+Key: eng_l2arctic_ara_000097, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000100, Target: eng, Predicted: tgk
+Key: eng_l2arctic_ara_000105, Target: eng, Predicted: nld
+Key: eng_l2arctic_ara_000109, Target: eng, Predicted: fra
+Key: eng_l2arctic_ara_000115, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000119, Target: eng, Predicted: tuk
+Key: eng_l2arctic_ara_000120, Target: eng, Predicted: pol
+Key: eng_l2arctic_ara_000121, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000122, Target: eng, Predicted: hye
+Key: eng_l2arctic_ara_000123, Target: eng, Predicted: kat
+Key: eng_l2arctic_ara_000127, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000129, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000130, Target: eng, Predicted: ben
+Key: eng_l2arctic_ara_000133, Target: eng, Predicted: hye
+Key: eng_l2arctic_ara_000140, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000146, Target: eng, Predicted: pus
+Key: eng_l2arctic_ara_000155, Target: eng, Predicted: pus
+Key: eng_l2arctic_ara_000164, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000166, Target: eng, Predicted: tgk
+Key: eng_l2arctic_ara_000167, Target: eng, Predicted: ara
+Key: eng_l2arctic_cmn_000010, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000011, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000019, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000027, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000042, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000047, Target: eng, Predicted: khm
+Key: eng_l2arctic_cmn_000052, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000058, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000060, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000063, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000071, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000075, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000076, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000077, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000078, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000080, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000087, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000088, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000091, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000094, Target: eng, Predicted: ben
+Key: eng_l2arctic_cmn_000097, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000098, Target: eng, Predicted: mya
+Key: eng_l2arctic_cmn_000100, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000102, Target: eng, Predicted: khm
+Key: eng_l2arctic_cmn_000106, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000107, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000108, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000113, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000114, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000115, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000116, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000120, Target: eng, Predicted: kor
+Key: eng_l2arctic_cmn_000125, Target: eng, Predicted: mya
+Key: eng_l2arctic_cmn_000126, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000127, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000128, Target: eng, Predicted: kor
+Key: eng_l2arctic_cmn_000130, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000132, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000133, Target: eng, Predicted: fas
+Key: eng_l2arctic_cmn_000134, Target: eng, Predicted: hau
+Key: eng_l2arctic_cmn_000135, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000137, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000138, Target: eng, Predicted: mya
+Key: eng_l2arctic_cmn_000139, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000141, Target: eng, Predicted: lao
+Key: eng_l2arctic_cmn_000142, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000143, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000146, Target: eng, Predicted: cmn
+Key: eng_l2arctic_hin_000005, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000007, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000010, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000014, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000016, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000019, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000020, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000022, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000024, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000025, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000033, Target: eng, Predicted: guj
+Key: eng_l2arctic_hin_000040, Target: eng, Predicted: snd
+Key: eng_l2arctic_hin_000054, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000055, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000057, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000061, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000064, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000065, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000069, Target: eng, Predicted: haw
+Key: eng_l2arctic_hin_000078, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000080, Target: eng, Predicted: pan
+Key: eng_l2arctic_hin_000085, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000096, Target: eng, Predicted: snd
+Key: eng_l2arctic_hin_000101, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000102, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000103, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000104, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000106, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000107, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000108, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000110, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000113, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000114, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000116, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000117, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000118, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000119, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000122, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000123, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000125, Target: eng, Predicted: ara
+Key: eng_l2arctic_hin_000126, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000127, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000128, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000129, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000130, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000131, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000132, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000133, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000138, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000140, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000141, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000142, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000143, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000144, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000146, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000147, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000149, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000152, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000153, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000154, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000155, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000156, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000157, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000158, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000159, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000161, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000162, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000163, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000164, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000165, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000166, Target: eng, Predicted: hrv
+Key: eng_l2arctic_hin_000167, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000168, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000169, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000170, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000171, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000173, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000175, Target: eng, Predicted: pan
+Key: eng_l2arctic_hin_000176, Target: eng, Predicted: glv
+Key: eng_l2arctic_hin_000180, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000182, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000183, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000186, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000187, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000188, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000189, Target: eng, Predicted: tuk
+Key: eng_l2arctic_hin_000191, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000193, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000194, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000195, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000196, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000197, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000199, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000200, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000201, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000203, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000204, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000205, Target: eng, Predicted: kan
+Key: eng_l2arctic_kor_000022, Target: eng, Predicted: ltz
+Key: eng_l2arctic_kor_000069, Target: eng, Predicted: dan
+Key: eng_l2arctic_kor_000112, Target: eng, Predicted: lat
+Key: eng_l2arctic_kor_000118, Target: eng, Predicted: vie
+Key: eng_l2arctic_kor_000137, Target: eng, Predicted: lao
+Key: eng_l2arctic_kor_000157, Target: eng, Predicted: hun
+Key: eng_l2arctic_kor_000165, Target: eng, Predicted: slv
+Key: eng_l2arctic_spa_000014, Target: eng, Predicted: heb
+Key: eng_l2arctic_spa_000027, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000033, Target: eng, Predicted: pol
+Key: eng_l2arctic_spa_000035, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000036, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000045, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000047, Target: eng, Predicted: fra
+Key: eng_l2arctic_spa_000052, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000055, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000062, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000063, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000067, Target: eng, Predicted: cym
+Key: eng_l2arctic_spa_000068, Target: eng, Predicted: tgk
+Key: eng_l2arctic_spa_000070, Target: eng, Predicted: sqi
+Key: eng_l2arctic_spa_000071, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000076, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000078, Target: eng, Predicted: spa
+Key: eng_l2arctic_spa_000102, Target: eng, Predicted: mar
+Key: eng_l2arctic_spa_000119, Target: eng, Predicted: hun
+Key: eng_l2arctic_spa_000149, Target: eng, Predicted: fin
+Key: eng_l2arctic_vie_000002, Target: eng, Predicted: bod
+Key: eng_l2arctic_vie_000004, Target: eng, Predicted: war
+Key: eng_l2arctic_vie_000005, Target: eng, Predicted: bod
+Key: eng_l2arctic_vie_000007, Target: eng, Predicted: lao
+Key: eng_l2arctic_vie_000015, Target: eng, Predicted: lao
+Key: eng_l2arctic_vie_000016, Target: eng, Predicted: lat
+Key: eng_l2arctic_vie_000019, Target: eng, Predicted: bod
+Key: eng_l2arctic_vie_000022, Target: eng, Predicted: guj
+Key: eng_l2arctic_vie_000026, Target: eng, Predicted: afr
+Key: eng_l2arctic_vie_000028, Target: eng, Predicted: mya
+Key: eng_l2arctic_vie_000042, Target: eng, Predicted: lao
+Key: eng_l2arctic_vie_000045, Target: eng, Predicted: tgl
+Key: eng_l2arctic_vie_000047, Target: eng, Predicted: khm
+Key: eng_l2arctic_vie_000053, Target: eng, Predicted: nno
+Key: eng_l2arctic_vie_000055, Target: eng, Predicted: lao
+Key: eng_l2arctic_vie_000062, Target: eng, Predicted: mya
+Key: eng_l2arctic_vie_000064, Target: eng, Predicted: lao
+Key: eng_l2arctic_vie_000066, Target: eng, Predicted: kor
+Key: eng_l2arctic_vie_000069, Target: eng, Predicted: mri
+Key: eng_l2arctic_vie_000074, Target: eng, Predicted: bod
+Key: eng_l2arctic_vie_000088, Target: eng, Predicted: mya
+Key: eng_l2arctic_vie_000092, Target: eng, Predicted: vie
+Key: eng_l2arctic_vie_000106, Target: eng, Predicted: msa
+Key: eng_l2arctic_vie_000107, Target: eng, Predicted: lao
+Key: eng_l2arctic_vie_000109, Target: eng, Predicted: cym
+Key: eng_l2arctic_vie_000120, Target: eng, Predicted: mya
+Key: eng_l2arctic_vie_000125, Target: eng, Predicted: cym
+Key: eng_l2arctic_vie_000131, Target: eng, Predicted: bod
+Key: eng_l2arctic_vie_000135, Target: eng, Predicted: mya
+Key: eng_l2arctic_vie_000145, Target: eng, Predicted: cym
+Key: eng_l2arctic_vie_000148, Target: eng, Predicted: vie
+Key: eng_l2arctic_vie_000151, Target: eng, Predicted: vie
+Key: eng_l2arctic_vie_000153, Target: eng, Predicted: lao
+Key: eng_l2arctic_vie_000158, Target: eng, Predicted: vie
+Key: eng_l2arctic_vie_000163, Target: eng, Predicted: vie
+Key: eng_openslr83_nor_000017, Target: eng, Predicted: glv
+Key: eng_openslr83_sco_000000, Target: eng, Predicted: cym
+Key: eng_openslr83_sco_000005, Target: eng, Predicted: cym
+Key: eng_openslr83_sco_000007, Target: eng, Predicted: cym
+Key: eng_openslr83_sco_000016, Target: eng, Predicted: cym
+Key: eng_openslr83_sco_000034, Target: eng, Predicted: cym
+Key: eng_openslr83_sou_000065, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000015, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000032, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000037, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000040, Target: eng, Predicted: glv
+Key: eng_openslr83_wel_000053, Target: eng, Predicted: glv
+Key: eng_openslr83_wel_000055, Target: eng, Predicted: glv
+Key: eng_openslr83_wel_000065, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000073, Target: eng, Predicted: glv
+Key: eng_openslr83_wel_000074, Target: eng, Predicted: glv
+Key: eng_openslr83_wel_000076, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000079, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000081, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000084, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000085, Target: eng, Predicted: glv
+Key: eng_openslr83_wel_000086, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000087, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000092, Target: eng, Predicted: cym
+Key: eng_voxpopuli_ces_000000, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000001, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000002, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000003, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000004, Target: eng, Predicted: fin
+Key: eng_voxpopuli_ces_000005, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000006, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000007, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000008, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000009, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000010, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000011, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000012, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000013, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000014, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000015, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000016, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000017, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000018, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000019, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000020, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000022, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000023, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000024, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000025, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000026, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000027, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000028, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000029, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000030, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000031, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000032, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000033, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000034, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000035, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000036, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000038, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000039, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000040, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000041, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000042, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000043, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000044, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000045, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000046, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000047, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000048, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000049, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000050, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000051, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000052, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000053, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000054, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ces_000055, Target: eng, Predicted: ces
+Key: eng_voxpopuli_deu_000001, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000002, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000004, Target: eng, Predicted: cym
+Key: eng_voxpopuli_deu_000013, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000016, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000020, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000030, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000034, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000037, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000039, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000041, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000043, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000045, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000050, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000061, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000063, Target: eng, Predicted: deu
+Key: eng_voxpopuli_deu_000065, Target: eng, Predicted: deu
+Key: eng_voxpopuli_est_000000, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000001, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000002, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000004, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000005, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000006, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000007, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000008, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000009, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000010, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000011, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000012, Target: eng, Predicted: lav
+Key: eng_voxpopuli_est_000013, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000014, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000015, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000016, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000017, Target: eng, Predicted: deu
+Key: eng_voxpopuli_est_000019, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000020, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000021, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000022, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000023, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000024, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000025, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000026, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000027, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000028, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000031, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000032, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000034, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000035, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000036, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000037, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000038, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000039, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000040, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000041, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000042, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000043, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000044, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000045, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000046, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000047, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000048, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000049, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000050, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000051, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000052, Target: eng, Predicted: est
+Key: eng_voxpopuli_est_000053, Target: eng, Predicted: est
+Key: eng_voxpopuli_fin_000000, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000001, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000002, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000003, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000005, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000006, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000007, Target: eng, Predicted: lav
+Key: eng_voxpopuli_fin_000008, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000009, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000010, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000011, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000012, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000013, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000014, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000015, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000016, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000017, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000018, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000019, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000020, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000021, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000022, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000023, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000024, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000025, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000026, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000027, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000028, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000029, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000030, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000031, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000032, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000033, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000034, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000035, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000036, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000037, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000038, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000039, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000040, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000041, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000042, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000043, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000044, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000045, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000046, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000047, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000049, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000050, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000051, Target: eng, Predicted: mri
+Key: eng_voxpopuli_fin_000052, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000053, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000054, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000055, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000056, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000057, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000058, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000059, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fin_000060, Target: eng, Predicted: fin
+Key: eng_voxpopuli_fra_000000, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000002, Target: eng, Predicted: ell
+Key: eng_voxpopuli_fra_000003, Target: eng, Predicted: lav
+Key: eng_voxpopuli_fra_000004, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000005, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000006, Target: eng, Predicted: por
+Key: eng_voxpopuli_fra_000007, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000008, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000009, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000010, Target: eng, Predicted: ell
+Key: eng_voxpopuli_fra_000011, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000012, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000013, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000014, Target: eng, Predicted: mya
+Key: eng_voxpopuli_fra_000015, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000016, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000017, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000018, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000019, Target: eng, Predicted: slk
+Key: eng_voxpopuli_fra_000021, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000022, Target: eng, Predicted: por
+Key: eng_voxpopuli_fra_000023, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000024, Target: eng, Predicted: ell
+Key: eng_voxpopuli_fra_000025, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000026, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000027, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000028, Target: eng, Predicted: por
+Key: eng_voxpopuli_fra_000029, Target: eng, Predicted: cym
+Key: eng_voxpopuli_fra_000030, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000032, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000033, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000034, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000035, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000036, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000037, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000038, Target: eng, Predicted: por
+Key: eng_voxpopuli_fra_000039, Target: eng, Predicted: cym
+Key: eng_voxpopuli_fra_000040, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000041, Target: eng, Predicted: lav
+Key: eng_voxpopuli_fra_000042, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000043, Target: eng, Predicted: slv
+Key: eng_voxpopuli_fra_000044, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000045, Target: eng, Predicted: fra
+Key: eng_voxpopuli_fra_000046, Target: eng, Predicted: por
+Key: eng_voxpopuli_fra_000047, Target: eng, Predicted: fra
+Key: eng_voxpopuli_fra_000048, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000049, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000050, Target: eng, Predicted: por
+Key: eng_voxpopuli_fra_000051, Target: eng, Predicted: por
+Key: eng_voxpopuli_fra_000053, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000055, Target: eng, Predicted: por
+Key: eng_voxpopuli_fra_000056, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000057, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000058, Target: eng, Predicted: nld
+Key: eng_voxpopuli_fra_000059, Target: eng, Predicted: ell
+Key: eng_voxpopuli_hun_000000, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000001, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000004, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000005, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000006, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000007, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000008, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000009, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000012, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000013, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000014, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000015, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000018, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000019, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000020, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000021, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000022, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000023, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000025, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000026, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000027, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000029, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000030, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000031, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000032, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000033, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000034, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000036, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000037, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000038, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000039, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000040, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000042, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000043, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000044, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000046, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000048, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000049, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000052, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000054, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000056, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000057, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000058, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000059, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000060, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000062, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000065, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000066, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000067, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000069, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000070, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000071, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000072, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000073, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000074, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000075, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000076, Target: eng, Predicted: hun
+Key: eng_voxpopuli_ita_000000, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000001, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000002, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000003, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000004, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000006, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000007, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000008, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000009, Target: eng, Predicted: slv
+Key: eng_voxpopuli_ita_000011, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000012, Target: eng, Predicted: cym
+Key: eng_voxpopuli_ita_000014, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000015, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000016, Target: eng, Predicted: slk
+Key: eng_voxpopuli_ita_000017, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000019, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000020, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000021, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000022, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000023, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000024, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000025, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000026, Target: eng, Predicted: hrv
+Key: eng_voxpopuli_ita_000028, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000029, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000030, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000031, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000032, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000033, Target: eng, Predicted: ces
+Key: eng_voxpopuli_ita_000035, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000036, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000037, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000038, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000039, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000040, Target: eng, Predicted: hrv
+Key: eng_voxpopuli_ita_000041, Target: eng, Predicted: nld
+Key: eng_voxpopuli_ita_000042, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000043, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000044, Target: eng, Predicted: hrv
+Key: eng_voxpopuli_ita_000045, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000046, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000047, Target: eng, Predicted: hrv
+Key: eng_voxpopuli_ita_000048, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000049, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000050, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000051, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000052, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000053, Target: eng, Predicted: ara
+Key: eng_voxpopuli_ita_000054, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000055, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000056, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000057, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000058, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000059, Target: eng, Predicted: ita
+Key: eng_voxpopuli_ita_000060, Target: eng, Predicted: ita
+Key: eng_voxpopuli_nld_000009, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000010, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000013, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000015, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000017, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000022, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000023, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000024, Target: eng, Predicted: mlt
+Key: eng_voxpopuli_nld_000026, Target: eng, Predicted: hau
+Key: eng_voxpopuli_nld_000029, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000031, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000034, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000036, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000038, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000040, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000041, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000042, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000047, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000051, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000057, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000060, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000062, Target: eng, Predicted: nld
+Key: eng_voxpopuli_nld_000064, Target: eng, Predicted: nld
+Key: eng_voxpopuli_pol_000001, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000002, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000006, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000007, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000008, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000009, Target: eng, Predicted: deu
+Key: eng_voxpopuli_pol_000011, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000012, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000013, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000015, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000017, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000019, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000020, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000021, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000022, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000023, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000024, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000025, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000026, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000027, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000028, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000029, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000030, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000031, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000032, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000034, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000037, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000038, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000040, Target: eng, Predicted: hun
+Key: eng_voxpopuli_pol_000041, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000042, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000044, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000045, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000046, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000047, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000048, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000050, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000052, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000053, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000058, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000059, Target: eng, Predicted: pol
+Key: eng_voxpopuli_ron_000000, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000002, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000003, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000005, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000006, Target: eng, Predicted: hun
+Key: eng_voxpopuli_ron_000007, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000008, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000012, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000013, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000014, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000015, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000016, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000017, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000018, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000019, Target: eng, Predicted: hun
+Key: eng_voxpopuli_ron_000020, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000021, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000024, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000025, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000026, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000029, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000030, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000031, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000032, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000033, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000034, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000035, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000036, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000037, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000038, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000039, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000040, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000041, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000044, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000045, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000046, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000047, Target: eng, Predicted: ell
+Key: eng_voxpopuli_ron_000048, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000049, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000050, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000051, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000052, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000053, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000054, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000055, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000056, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000057, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000060, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000063, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000065, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000066, Target: eng, Predicted: ron
+Key: eng_voxpopuli_slk_000000, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000001, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000005, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000006, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000007, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000009, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000010, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000011, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000012, Target: eng, Predicted: hrv
+Key: eng_voxpopuli_slk_000015, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000016, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000017, Target: eng, Predicted: ces
+Key: eng_voxpopuli_slk_000019, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000020, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000022, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000023, Target: eng, Predicted: ces
+Key: eng_voxpopuli_slk_000026, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000027, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000028, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000029, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000032, Target: eng, Predicted: pol
+Key: eng_voxpopuli_slk_000034, Target: eng, Predicted: ces
+Key: eng_voxpopuli_slk_000035, Target: eng, Predicted: nld
+Key: eng_voxpopuli_slk_000037, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000039, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000040, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000041, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000042, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000043, Target: eng, Predicted: ces
+Key: eng_voxpopuli_slk_000044, Target: eng, Predicted: ces
+Key: eng_voxpopuli_slk_000045, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000047, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000048, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000049, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000050, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000051, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000052, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000054, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000055, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000057, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000059, Target: eng, Predicted: slk
+Key: eng_voxpopuli_slk_000060, Target: eng, Predicted: ara
+Key: eng_voxpopuli_slk_000062, Target: eng, Predicted: ces
+Key: eng_voxpopuli_slk_000063, Target: eng, Predicted: pol
+Key: eng_voxpopuli_slk_000064, Target: eng, Predicted: slk
+Key: eng_voxpopuli_spa_000000, Target: eng, Predicted: spa
+Key: eng_voxpopuli_spa_000003, Target: eng, Predicted: spa
+Key: eng_voxpopuli_spa_000007, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000010, Target: eng, Predicted: spa
+Key: eng_voxpopuli_spa_000017, Target: eng, Predicted: glg
+Key: eng_voxpopuli_spa_000020, Target: eng, Predicted: cat
+Key: eng_voxpopuli_spa_000021, Target: eng, Predicted: spa
+Key: eng_voxpopuli_spa_000023, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000027, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000029, Target: eng, Predicted: spa
+Key: eng_voxpopuli_spa_000030, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000032, Target: eng, Predicted: spa
+Key: eng_voxpopuli_spa_000033, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000035, Target: eng, Predicted: glg
+Key: eng_voxpopuli_spa_000036, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000037, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000040, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000043, Target: eng, Predicted: hau
+Key: eng_voxpopuli_spa_000047, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000050, Target: eng, Predicted: ell
+Key: eng_voxpopuli_spa_000052, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000053, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000058, Target: eng, Predicted: ell
+Key: eng_voxpopuli_spa_000059, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000060, Target: eng, Predicted: ara
+Key: eng_voxpopuli_spa_000064, Target: eng, Predicted: cym
+Key: eng_voxpopuli_spa_000067, Target: eng, Predicted: spa
+Key: eng_voxpopuli_spa_000068, Target: eng, Predicted: spa
+Key: guj_ms_speech_guj_000038, Target: guj, Predicted: mar
+Key: spa_openslr_spa_arg_000006, Target: spa, Predicted: ita
+Key: spa_openslr_spa_arg_000015, Target: spa, Predicted: por
+Key: spa_openslr_spa_arg_000022, Target: spa, Predicted: dan
+Key: spa_openslr_spa_arg_000062, Target: spa, Predicted: eus
+Key: spa_openslr_spa_arg_000114, Target: spa, Predicted: eus
+Key: spa_openslr_spa_chi_000022, Target: spa, Predicted: grn
+Key: spa_openslr_spa_col_000007, Target: spa, Predicted: epo
+Key: spa_openslr_spa_col_000044, Target: spa, Predicted: por
+Key: spa_openslr_spa_col_000056, Target: spa, Predicted: ita
+Key: spa_openslr_spa_col_000095, Target: spa, Predicted: fas
+Key: spa_openslr_spa_per_000015, Target: spa, Predicted: eus
+Key: spa_openslr_spa_per_000035, Target: spa, Predicted: ita
+Key: spa_openslr_spa_pue_000014, Target: spa, Predicted: grn
+Key: spa_openslr_spa_pue_000071, Target: spa, Predicted: ita
+Key: spa_openslr_spa_ven_000112, Target: spa, Predicted: eus
+Key: tam_ms_speech_tam_000102, Target: tam, Predicted: tel
+Key: tel_ms_speech_tel_000014, Target: tel, Predicted: mal

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_voxlingua107_lang/lid_inference_test.log

@@ -0,0 +1,291 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Fri May 16 16:33:03 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue02] 2025-05-16 16:33:20,000 (abs_task:2341) INFO: config file: exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2364: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue02] 2025-05-16 16:33:30,412 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamLang2VecConditionModel(
+  (frontend): S3prlFrontendLang2VecCondition(
+    (upstream): S3PRLUpstreamLang2VecCondition(
+      (upstream): UpstreamExpertLang2VecCondition(
+        (model): Wav2Vec2ModelLang2VecCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderLang2VecCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (conditioning_projs): ModuleDict(
+              (32): Linear(in_features=299, out_features=1280, bias=True)
+              (36): Linear(in_features=299, out_features=1280, bias=True)
+              (40): Linear(in_features=299, out_features=1280, bias=True)
+              (44): Linear(in_features=299, out_features=1280, bias=True)
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamLang2VecConditionModel
+    Total Number of model parameters: 978.26 M
+    Number of trainable parameters: 978.26 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue02] 2025-05-16 16:33:30,762 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_voxlingua107_lang/lids0
+[gpue02] 2025-05-16 16:34:05,333 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue02] 2025-05-16 16:34:36,939 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue02] 2025-05-16 16:35:13,502 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue02] 2025-05-16 16:35:46,461 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue02] 2025-05-16 16:36:14,706 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue02] 2025-05-16 16:36:51,347 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue02] 2025-05-16 16:37:26,344 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue02] 2025-05-16 16:38:02,825 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7
+[gpue02] 2025-05-16 16:38:41,474 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8
+[gpue02] 2025-05-16 16:39:17,680 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9
+[gpue02] 2025-05-16 16:39:53,999 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10
+[gpue02] 2025-05-16 16:40:24,699 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11
+[gpue02] 2025-05-16 16:41:04,097 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12
+[gpue02] 2025-05-16 16:41:36,434 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13
+[gpue02] 2025-05-16 16:42:15,847 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14
+[gpue02] 2025-05-16 16:42:47,941 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15
+[gpue02] 2025-05-16 16:43:22,603 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16
+[gpue02] 2025-05-16 16:43:43,333 (lid_inference_dist:200) INFO: args.save_embd_per_utt: False
+[gpue02] 2025-05-16 16:43:43,334 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=641 threads=1
+# Ended (code 0) at Fri May 16 16:43:44 CDT 2025, elapsed time 641 seconds

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/inference/valid.accuracy.best/dev_voxlingua107_lang/lid_inference_test.log

@@ -0,0 +1,275 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Fri May 16 15:09:41 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue02] 2025-05-16 15:10:03,018 (abs_task:2341) INFO: config file: exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2364: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue02] 2025-05-16 15:10:16,727 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamLang2VecConditionModel(
+  (frontend): S3prlFrontendLang2VecCondition(
+    (upstream): S3PRLUpstreamLang2VecCondition(
+      (upstream): UpstreamExpertLang2VecCondition(
+        (model): Wav2Vec2ModelLang2VecCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderLang2VecCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (conditioning_projs): ModuleDict(
+              (32): Linear(in_features=299, out_features=1280, bias=True)
+              (36): Linear(in_features=299, out_features=1280, bias=True)
+              (40): Linear(in_features=299, out_features=1280, bias=True)
+              (44): Linear(in_features=299, out_features=1280, bias=True)
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamLang2VecConditionModel
+    Total Number of model parameters: 978.26 M
+    Number of trainable parameters: 978.26 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue02] 2025-05-16 15:10:17,080 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/dev_voxlingua107_lang/lids0
+[gpue02] 2025-05-16 15:11:10,973 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue02] 2025-05-16 15:11:43,358 (lid_inference_dist:200) INFO: args.save_embd_per_utt: False
+[gpue02] 2025-05-16 15:11:43,358 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=123 threads=1
+# Ended (code 0) at Fri May 16 15:11:44 CDT 2025, elapsed time 123 seconds

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/inference/valid.accuracy.best/dev_voxlingua107_lang/results

@@ -0,0 +1,129 @@
+Accuracy: 94.22%
+Macro Accuracy: 90.26%
+Accuracy per Language:
+isl: 100.00%
+hrv: 50.00%
+swe: 94.00%
+ell: 80.00%
+lav: 100.00%
+cmn: 95.65%
+tur: 90.28%
+mkd: 100.00%
+jpn: 100.00%
+hye: 96.00%
+dan: 95.96%
+fra: 97.00%
+fin: 96.77%
+pol: 100.00%
+nor: 62.50%
+rus: 96.55%
+urd: 76.92%
+ara: 96.00%
+aze: 94.12%
+ita: 100.00%
+eng: 93.75%
+lit: 84.62%
+spa: 98.18%
+hun: 100.00%
+srp: 57.14%
+ukr: 88.24%
+est: 95.29%
+fas: 92.00%
+nld: 97.00%
+por: 100.00%
+slv: 88.89%
+deu: 95.12%
+nno: 66.67%
+Key: ara_AfS6C1PXAdQ__U__S20---0104.730-0111.410.wav, Target: ara, Predicted: hau
+Key: ara_TPWwuy20K_c__U__S70---0466.380-0472.600.wav, Target: ara, Predicted: hau
+Key: ara_cwvnYGInNNg__U__S229---0661.830-0670.560.wav, Target: ara, Predicted: heb
+Key: ara_tl39W93P0r4__U__S32---0282.970-0286.530.wav, Target: ara, Predicted: fra
+Key: aze_3UUShvAQxQY__U__S199---1315.800-1322.250.wav, Target: aze, Predicted: tur
+Key: aze_bYKK1m78ecE__U__S91---0592.500-0596.130.wav, Target: aze, Predicted: isl
+Key: cmn_ZUzq_TIfYL4__U__S39---0442.690-0454.380.wav, Target: cmn, Predicted: tha
+Key: dan_E3vuA0Mqk1Q__U__S13---0072.140-0083.530.wav, Target: dan, Predicted: nno
+Key: dan_Nyl6CuW6Qfk__U__S26---0557.690-0560.120.wav, Target: dan, Predicted: nor
+Key: dan_ZZD1qu4ScPg__U__S14---0166.700-0176.010.wav, Target: dan, Predicted: haw
+Key: dan_qCg43KdKRr0__U__S0---0843.270-0847.580.wav, Target: dan, Predicted: bod
+Key: deu_8L3k8XNTtNA__U__S100---2689.380-2692.180.wav, Target: deu, Predicted: nno
+Key: deu_9O2haSYzftE__U__S0---0000.000-0004.200.wav, Target: deu, Predicted: heb
+Key: deu_eyZqRcgGkiY__U__S126---1155.890-1162.390.wav, Target: deu, Predicted: nld
+Key: deu_cMZO2zXTBv8__U__S100---0341.910-0344.350.wav, Target: deu, Predicted: yid
+Key: ell_bw_mDLVdgtY__U__S18---0119.750-0127.200.wav, Target: ell, Predicted: bre
+Key: eng_K977aQQpAVk__U__S106---0393.230-0397.100.wav, Target: eng, Predicted: cym
+Key: eng_4y7p9R2No-4__U__S12---0266.390-0268.460.wav, Target: eng, Predicted: lat
+Key: eng_eQXHc-tJMXM__U__S11---1066.230-1077.360.wav, Target: eng, Predicted: cym
+Key: est_5gWpxiFOouQ__U__S2---1635.950-1646.620.wav, Target: est, Predicted: khm
+Key: eng_xuM9TP8ETJI__U__S1---0063.150-0077.230.wav, Target: eng, Predicted: mar
+Key: eng_yhU8sj0X-yM__U__S25---0146.220-0152.200.wav, Target: eng, Predicted: nno
+Key: est_E05LlgvSMg0__U__S156---1171.030-1172.780.wav, Target: est, Predicted: fin
+Key: est_7vZIuc9qumg__U__S21---0145.690-0153.320.wav, Target: est, Predicted: nep
+Key: est_gTl2GSJBxNw__U__S0---0000.000-0008.420.wav, Target: est, Predicted: mlt
+Key: fas_9k1oVW4Ynyw__U__S15---0097.430-0101.630.wav, Target: fas, Predicted: slv
+Key: fas_EjSRRddYuc4__U__S58---0355.980-0359.590.wav, Target: fas, Predicted: lat
+Key: fas_SMcjja_krx4__U__S2---0012.190-0021.730.wav, Target: fas, Predicted: tgk
+Key: fas_gLoBPMrad3E__U__S14---0097.650-0102.010.wav, Target: fas, Predicted: yid
+Key: fas_XUGZwtXgvRA__U__S154---0993.540-0997.340.wav, Target: fas, Predicted: san
+Key: fas_x_Di4cq4ixM__U__S100---1353.580-1358.390.wav, Target: fas, Predicted: pus
+Key: fas_nPts67VQKRQ__U__S250---1629.010-1632.750.wav, Target: fas, Predicted: hat
+Key: fas_zZCjOs-WwKo__U__S195---1357.430-1377.010.wav, Target: fas, Predicted: aze
+Key: fin_C4H2GlJRkNU__U__S100---1604.910-1610.210.wav, Target: fin, Predicted: est
+Key: fin_S_VWbBtBey4__U__S0---0308.380-0310.650.wav, Target: fin, Predicted: khm
+Key: fin_i6m8DKfYNUM__U__S145---0562.860-0573.500.wav, Target: fin, Predicted: est
+Key: fra_Lo_JX-8KHEw__U__S151---0284.430-0299.020.wav, Target: fra, Predicted: lin
+Key: fra_SLfpp704KI8__U__S57---0368.470-0372.910.wav, Target: fra, Predicted: afr
+Key: fra_pLwARlYGwS0__U__S2---0046.320-0052.700.wav, Target: fra, Predicted: lin
+Key: hye_DoR3ptr3ahw__U__S28---0216.960-0223.190.wav, Target: hye, Predicted: urd
+Key: hrv_Jntmbw5_vOI__U__S291---0379.300-0383.970.wav, Target: hrv, Predicted: bos
+Key: hrv_hUgTLHiQJ60__U__S3---0046.320-0060.040.wav, Target: hrv, Predicted: bos
+Key: hye_Qmo3P38Ytek__U__S32---0245.460-0249.320.wav, Target: hye, Predicted: snd
+Key: hye_XM7EQuJEG38__U__S267---1644.620-1648.650.wav, Target: hye, Predicted: amh
+Key: hye_um6xT5Gjgus__U__S194---1224.460-1234.130.wav, Target: hye, Predicted: lat
+Key: lit_3svAywrL0_I__U__S149---0461.370-0464.980.wav, Target: lit, Predicted: bre
+Key: lit_cTaKSWT6ds0__U__S49---0492.750-0499.290.wav, Target: lit, Predicted: slv
+Key: nld_2C5HehL-Fx0__U__S101---1125.890-1131.720.wav, Target: nld, Predicted: ltz
+Key: nld_0LhAXOxz-JU__U__S32---0243.280-0247.880.wav, Target: nld, Predicted: afr
+Key: nld_0LhAXOxz-JU__U__S396---2475.670-2488.950.wav, Target: nld, Predicted: afr
+Key: nno_iLv9Mp7Z3SE__U__S50---0290.270-0300.430.wav, Target: nno, Predicted: nor
+Key: nor_97e9pEtHAxg__U__S32---0201.830-0210.250.wav, Target: nor, Predicted: nno
+Key: nor_HW_49WuFloM__U__S106---0621.590-0626.130.wav, Target: nor, Predicted: nno
+Key: nor_I1vUI8va8Yc__U__S49---0294.940-0302.560.wav, Target: nor, Predicted: afr
+Key: nor_hEA8NB6Gojk__U__S1---1704.010-1709.570.wav, Target: nor, Predicted: nno
+Key: nor_UxHL_uql05E__U__S118---0587.340-0598.790.wav, Target: nor, Predicted: nno
+Key: nor_tV3Le8SUz_0__U__S276---1831.870-1841.550.wav, Target: nor, Predicted: nno
+Key: nor_xVNA15ifyIw__U__S494---0311.220-0317.160.wav, Target: nor, Predicted: nno
+Key: nor_bOnuTgAOkl8__U__S1---0180.450-0187.220.wav, Target: nor, Predicted: nno
+Key: nor_ySVkmT8SgNM__U__S345---2245.790-2255.930.wav, Target: nor, Predicted: nno
+Key: slv_Hu-33c4xcwU__U__S100---0534.990-0540.460.wav, Target: slv, Predicted: hrv
+Key: rus_iUdCazc2qEs__U__S170---0648.590-0665.280.wav, Target: rus, Predicted: bel
+Key: spa_UYBcNrx8kvQ__U__S186---2292.670-2299.590.wav, Target: spa, Predicted: kor
+Key: swe_0gbhN4C2JSg__U__S0---0310.630-0319.010.wav, Target: swe, Predicted: nno
+Key: srp_8dvIaAOLlGA__U__S216---1326.410-1335.490.wav, Target: srp, Predicted: hrv
+Key: srp_TrZPwrrrC8k__U__S172---1130.440-1138.910.wav, Target: srp, Predicted: hrv
+Key: swe_0x4xb4AaTy0__U__S0---0301.450-0319.780.wav, Target: swe, Predicted: nno
+Key: srp_rkQhxxO5Qt4__U__S109---0820.610-0826.190.wav, Target: srp, Predicted: bos
+Key: swe_PjMuio7mLss__U__S193---0310.690-0316.160.wav, Target: swe, Predicted: nor
+Key: swe_i7zSYkRMXas__U__S0---1020.080-1026.030.wav, Target: swe, Predicted: nno
+Key: swe_wMAAiJhj0VA__U__S100---0564.840-0568.420.wav, Target: swe, Predicted: nno
+Key: tur_-H4m-34Aeoc__U__S60---0434.240-0442.760.wav, Target: tur, Predicted: heb
+Key: swe_ilhngbAuxvs__U__S14---2441.740-2445.720.wav, Target: swe, Predicted: ltz
+Key: tur_4C-efpD-DlM__U__S7---0050.890-0055.080.wav, Target: tur, Predicted: ben
+Key: tur_KfJTU69HRWI__U__S11---0048.380-0056.300.wav, Target: tur, Predicted: aze
+Key: tur_5IW3HlTJdMc__U__S115---0705.150-0710.380.wav, Target: tur, Predicted: aze
+Key: tur_a1MRbrMI8_I__U__S145---2439.720-2445.680.wav, Target: tur, Predicted: aze
+Key: tur_kLouhYz_EQk__U__S13---0102.530-0106.300.wav, Target: tur, Predicted: kor
+Key: tur_SjS8CCqZxTk__U__S106---0354.470-0358.660.wav, Target: tur, Predicted: nor
+Key: urd_4tO6ayZxs5s__U__S1---0194.310-0211.150.wav, Target: urd, Predicted: hin
+Key: ukr_Oa0IlRCP7w0__U__S138---1702.720-1708.290.wav, Target: ukr, Predicted: bos
+Key: ukr_cSDxFl4xZ0M__U__S275---1824.900-1830.730.wav, Target: ukr, Predicted: rus
+Key: urd_Ax5M8gcEdCg__U__S1---0105.190-0115.110.wav, Target: urd, Predicted: hin
+Key: urd_N59t4A1mxfA__U__S101---0715.390-0720.460.wav, Target: urd, Predicted: snd
+Key: urd_eTfyAm6CFB0__U__S25---0439.860-0446.680.wav, Target: urd, Predicted: hin
+Key: urd_Tj2pngm_vuA__U__S1---0070.400-0089.660.wav, Target: urd, Predicted: hin
+Key: urd_J7RizO2mvm4__U__S3---0042.600-0051.180.wav, Target: urd, Predicted: san
+Key: urd_U_h8Bgywxrc__U__S0---0222.420-0227.610.wav, Target: urd, Predicted: hin
+Key: urd_n3l7PavcOFk__U__S0---0379.380-0397.930.wav, Target: urd, Predicted: hin
+Key: urd_o3awRytwrUY__U__S1---0122.040-0129.980.wav, Target: urd, Predicted: fas
+Key: urd_ySjOb5uaA-U__U__S107---0336.690-0353.110.wav, Target: urd, Predicted: cym
+Key: urd_o3awRytwrUY__U__S1---0290.850-0306.430.wav, Target: urd, Predicted: fas
+Key: urd_o3awRytwrUY__U__S1---0632.460-0642.400.wav, Target: urd, Predicted: fas

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_voxlingua107_lang/lid_inference_test.log

@@ -0,0 +1,336 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Fri May 16 15:29:06 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue02] 2025-05-16 15:29:22,203 (abs_task:2341) INFO: config file: exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2364: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue02] 2025-05-16 15:29:32,382 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamLang2VecConditionModel(
+  (frontend): S3prlFrontendLang2VecCondition(
+    (upstream): S3PRLUpstreamLang2VecCondition(
+      (upstream): UpstreamExpertLang2VecCondition(
+        (model): Wav2Vec2ModelLang2VecCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderLang2VecCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (conditioning_projs): ModuleDict(
+              (32): Linear(in_features=299, out_features=1280, bias=True)
+              (36): Linear(in_features=299, out_features=1280, bias=True)
+              (40): Linear(in_features=299, out_features=1280, bias=True)
+              (44): Linear(in_features=299, out_features=1280, bias=True)
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamLang2VecConditionModel
+    Total Number of model parameters: 978.26 M
+    Number of trainable parameters: 978.26 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue02] 2025-05-16 15:29:32,731 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_voxlingua107_lang/lids0
+[gpue02] 2025-05-16 15:30:28,250 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue02] 2025-05-16 15:31:29,388 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue02] 2025-05-16 15:32:30,025 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue02] 2025-05-16 15:33:41,554 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue02] 2025-05-16 15:34:45,848 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue02] 2025-05-16 15:35:40,662 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue02] 2025-05-16 15:36:31,911 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue02] 2025-05-16 15:37:35,086 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7
+[gpue02] 2025-05-16 15:38:34,369 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8
+[gpue02] 2025-05-16 15:39:35,470 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9
+[gpue02] 2025-05-16 15:40:45,143 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10
+[gpue02] 2025-05-16 15:41:44,725 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11
+[gpue02] 2025-05-16 15:42:45,976 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12
+[gpue02] 2025-05-16 15:43:35,253 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13
+[gpue02] 2025-05-16 15:44:35,428 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14
+[gpue02] 2025-05-16 15:45:41,733 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15
+[gpue02] 2025-05-16 15:46:37,468 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16
+[gpue02] 2025-05-16 15:47:25,258 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 17
+[gpue02] 2025-05-16 15:48:14,261 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 18
+[gpue02] 2025-05-16 15:49:37,718 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 19
+[gpue02] 2025-05-16 15:50:28,832 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 20
+[gpue02] 2025-05-16 15:51:17,795 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 21
+[gpue02] 2025-05-16 15:52:13,077 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 22
+[gpue02] 2025-05-16 15:53:09,422 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 23
+[gpue02] 2025-05-16 15:54:14,113 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 24
+[gpue02] 2025-05-16 15:55:23,133 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 25
+[gpue02] 2025-05-16 15:56:25,007 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 26
+[gpue02] 2025-05-16 15:57:22,498 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 27
+[gpue02] 2025-05-16 15:58:28,784 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 28
+[gpue02] 2025-05-16 15:59:41,942 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 29
+[gpue02] 2025-05-16 16:00:45,243 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 30
+[gpue02] 2025-05-16 16:01:50,443 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 31
+[gpue02] 2025-05-16 16:02:57,175 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 32
+[gpue02] 2025-05-16 16:03:49,857 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 33
+[gpue02] 2025-05-16 16:04:58,821 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 34
+[gpue02] 2025-05-16 16:06:05,739 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 35
+[gpue02] 2025-05-16 16:07:04,501 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 36
+[gpue02] 2025-05-16 16:08:06,650 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 37
+[gpue02] 2025-05-16 16:08:56,869 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 38
+[gpue02] 2025-05-16 16:10:42,668 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 39
+[gpue02] 2025-05-16 16:11:39,638 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 40
+[gpue02] 2025-05-16 16:12:44,805 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 41
+[gpue02] 2025-05-16 16:13:44,970 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 42
+[gpue02] 2025-05-16 16:14:59,216 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 43
+[gpue02] 2025-05-16 16:15:47,866 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 44
+[gpue02] 2025-05-16 16:16:46,568 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 45
+[gpue02] 2025-05-16 16:17:40,642 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 46
+[gpue02] 2025-05-16 16:18:33,323 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 47
+[gpue02] 2025-05-16 16:19:25,947 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 48
+[gpue02] 2025-05-16 16:20:18,394 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 49
+[gpue02] 2025-05-16 16:21:23,138 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 50
+[gpue02] 2025-05-16 16:22:24,219 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 51
+[gpue02] 2025-05-16 16:23:28,259 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 52
+[gpue02] 2025-05-16 16:24:22,109 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 53
+[gpue02] 2025-05-16 16:25:19,894 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 54
+[gpue02] 2025-05-16 16:26:20,392 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 55
+[gpue02] 2025-05-16 16:27:20,976 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 56
+[gpue02] 2025-05-16 16:28:22,402 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 57
+[gpue02] 2025-05-16 16:29:23,094 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 58
+[gpue02] 2025-05-16 16:30:12,825 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 59
+[gpue02] 2025-05-16 16:31:12,064 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 60
+[gpue02] 2025-05-16 16:32:16,256 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 61
+[gpue02] 2025-05-16 16:33:01,500 (lid_inference_dist:200) INFO: args.save_embd_per_utt: False
+[gpue02] 2025-05-16 16:33:01,500 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=3836 threads=1
+# Ended (code 0) at Fri May 16 16:33:02 CDT 2025, elapsed time 3836 seconds

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_voxlingua107_lang/lid_inference_test.log

@@ -0,0 +1,290 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Fri May 16 15:11:45 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_voxlingua107_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_voxlingua107_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_voxlingua107_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml --lid_model_file exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 8 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt false --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue02] 2025-05-16 15:12:01,248 (abs_task:2341) INFO: config file: exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2364: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue02] 2025-05-16 15:12:11,661 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamLang2VecConditionModel(
+  (frontend): S3prlFrontendLang2VecCondition(
+    (upstream): S3PRLUpstreamLang2VecCondition(
+      (upstream): UpstreamExpertLang2VecCondition(
+        (model): Wav2Vec2ModelLang2VecCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderLang2VecCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (conditioning_projs): ModuleDict(
+              (32): Linear(in_features=299, out_features=1280, bias=True)
+              (36): Linear(in_features=299, out_features=1280, bias=True)
+              (40): Linear(in_features=299, out_features=1280, bias=True)
+              (44): Linear(in_features=299, out_features=1280, bias=True)
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamLang2VecConditionModel
+    Total Number of model parameters: 978.26 M
+    Number of trainable parameters: 978.26 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue02] 2025-05-16 15:12:12,017 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_voxlingua107_raw/spk_mms_ecapa_upcon_32_44_it0.4_independent_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_voxlingua107_lang/lids0
+[gpue02] 2025-05-16 15:13:09,809 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue02] 2025-05-16 15:14:05,884 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue02] 2025-05-16 15:14:58,617 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue02] 2025-05-16 15:15:55,618 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue02] 2025-05-16 15:16:53,845 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue02] 2025-05-16 15:17:51,064 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue02] 2025-05-16 15:18:50,990 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue02] 2025-05-16 15:19:55,904 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7
+[gpue02] 2025-05-16 15:20:53,867 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8
+[gpue02] 2025-05-16 15:22:09,543 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9
+[gpue02] 2025-05-16 15:23:04,788 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10
+[gpue02] 2025-05-16 15:23:58,362 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11
+[gpue02] 2025-05-16 15:24:57,466 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12
+[gpue02] 2025-05-16 15:25:59,579 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13
+[gpue02] 2025-05-16 15:26:57,875 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14
+[gpue02] 2025-05-16 15:28:01,852 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15
+[gpue02] 2025-05-16 15:29:04,351 (lid_inference_dist:200) INFO: args.save_embd_per_utt: False
+[gpue02] 2025-05-16 15:29:04,352 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=1040 threads=1
+# Ended (code 0) at Fri May 16 15:29:05 CDT 2025, elapsed time 1040 seconds

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/tensorboard/train/events.out.tfevents.1747382406.gpue02.delta.ncsa.illinois.edu.4061979.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e94aec9eafec16372ab32b8fa94f7820342abeaa4ee3e8bf76c0d2c41e0714c
+size 333926

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/tensorboard/train/events.out.tfevents.1755588660.gpue01.delta.ncsa.illinois.edu.672803.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f64f0249df23805707ffb386715a27500e7ada827cb65ff452d4efcb2ad91f8
+size 88

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/tensorboard/train/events.out.tfevents.1755588993.gpue01.delta.ncsa.illinois.edu.682794.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ded6556895974f83d53ca564bf7581a90cb3d709499aa60e53b605343ef00696
+size 88

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/tensorboard/valid/events.out.tfevents.1747382406.gpue02.delta.ncsa.illinois.edu.4061979.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ae69aabebb88e4d9c2df7c4454477e7d3dd775f13cce22977dee7d22179f641
+size 18542

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/tensorboard/valid/events.out.tfevents.1755588660.gpue01.delta.ncsa.illinois.edu.672803.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f0f68b28b509fbd3899c81e179eff342efe5486ea135701cc8765b4bbe5456d
+size 88

exp_voxlingua107_only/lid_mms_ecapa_upcon_32_44_it0.4_independent_frozen_raw/tensorboard/valid/events.out.tfevents.1755588993.gpue01.delta.ncsa.illinois.edu.682794.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da635e135ca27fed48691df83327ebf27d61a169b18d7f370b1f2cd0e0284d71
+size 88