File size: 11,668 Bytes
5d8c911 2c4c266 c81eede 377983e 8989e4c 1003015 998e66d 24d8107 cd141b4 a1039d6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 |
2022-05-04 13:30:45 INFO Running runs: []
2022-05-04 13:30:45 INFO Agent received command: run
2022-05-04 13:30:45 INFO Agent starting run with config:
eval_split_name: test
eval_steps: 500
evaluation_strategy: steps
generation_max_length: 40
generation_num_beams: 1
gradient_accumulation_steps: 8
greater_is_better: True
hidden_dropout: 0.17305159310134854
language: fr.en
learning_rate: 0.00012335092351490598
logging_steps: 1
max_duration_in_seconds: 20
metric_for_best_model: bleu
model_name_or_path: ./
num_train_epochs: 3
output_dir: ./
per_device_eval_batch_size: 8
per_device_train_batch_size: 8
save_steps: 500
task: covost2
warmup_steps: 500
2022-05-04 13:30:45 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.17305159310134854 --language=fr.en --learning_rate=0.00012335092351490598 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-04 13:30:50 INFO Running runs: ['w4rlzz90']
2022-05-05 09:24:36 ERROR 500 response executing GraphQL.
2022-05-05 09:24:36 ERROR {"errors":[{"message":"context deadline exceeded"}]}
2022-05-05 09:25:32 ERROR 500 response executing GraphQL.
2022-05-05 09:25:32 ERROR {"errors":[{"message":"context deadline exceeded"}]}
2022-05-05 16:55:14 INFO Cleaning up finished run: w4rlzz90
2022-05-05 16:55:15 INFO Agent received command: run
2022-05-05 16:55:15 INFO Agent starting run with config:
eval_split_name: test
eval_steps: 500
evaluation_strategy: steps
generation_max_length: 40
generation_num_beams: 1
gradient_accumulation_steps: 8
greater_is_better: True
hidden_dropout: 0.0565117244178473
language: fr.en
learning_rate: 0.0004100464310609422
logging_steps: 1
max_duration_in_seconds: 20
metric_for_best_model: bleu
model_name_or_path: ./
num_train_epochs: 3
output_dir: ./
per_device_eval_batch_size: 8
per_device_train_batch_size: 8
save_steps: 500
task: covost2
warmup_steps: 500
2022-05-05 16:55:15 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.0565117244178473 --language=fr.en --learning_rate=0.0004100464310609422 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 16:55:20 INFO Running runs: ['npwlqyuz']
2022-05-05 16:59:27 INFO Cleaning up finished run: npwlqyuz
2022-05-05 16:59:28 INFO Agent received command: run
2022-05-05 16:59:28 INFO Agent starting run with config:
eval_split_name: test
eval_steps: 500
evaluation_strategy: steps
generation_max_length: 40
generation_num_beams: 1
gradient_accumulation_steps: 8
greater_is_better: True
hidden_dropout: 0.09631139674620667
language: fr.en
learning_rate: 0.0001294077232279209
logging_steps: 1
max_duration_in_seconds: 20
metric_for_best_model: bleu
model_name_or_path: ./
num_train_epochs: 3
output_dir: ./
per_device_eval_batch_size: 8
per_device_train_batch_size: 8
save_steps: 500
task: covost2
warmup_steps: 500
2022-05-05 16:59:28 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.09631139674620667 --language=fr.en --learning_rate=0.0001294077232279209 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 16:59:33 INFO Running runs: ['58ade1nh']
2022-05-05 17:02:30 INFO Cleaning up finished run: 58ade1nh
2022-05-05 17:02:31 INFO Agent received command: run
2022-05-05 17:02:31 INFO Agent starting run with config:
eval_split_name: test
eval_steps: 500
evaluation_strategy: steps
generation_max_length: 40
generation_num_beams: 1
gradient_accumulation_steps: 8
greater_is_better: True
hidden_dropout: 0.055807655334937395
language: fr.en
learning_rate: 0.0002523256445833689
logging_steps: 1
max_duration_in_seconds: 20
metric_for_best_model: bleu
model_name_or_path: ./
num_train_epochs: 3
output_dir: ./
per_device_eval_batch_size: 8
per_device_train_batch_size: 8
save_steps: 500
task: covost2
warmup_steps: 500
2022-05-05 17:02:31 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.055807655334937395 --language=fr.en --learning_rate=0.0002523256445833689 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 17:02:36 INFO Running runs: ['2zmxvr9t']
2022-05-05 17:05:37 INFO Cleaning up finished run: 2zmxvr9t
2022-05-05 17:05:38 INFO Agent received command: run
2022-05-05 17:05:38 INFO Agent starting run with config:
eval_split_name: test
eval_steps: 500
evaluation_strategy: steps
generation_max_length: 40
generation_num_beams: 1
gradient_accumulation_steps: 8
greater_is_better: True
hidden_dropout: 0.08376731063580722
language: fr.en
learning_rate: 0.00023509256443134124
logging_steps: 1
max_duration_in_seconds: 20
metric_for_best_model: bleu
model_name_or_path: ./
num_train_epochs: 3
output_dir: ./
per_device_eval_batch_size: 8
per_device_train_batch_size: 8
save_steps: 500
task: covost2
warmup_steps: 500
2022-05-05 17:05:38 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.08376731063580722 --language=fr.en --learning_rate=0.00023509256443134124 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 17:05:43 INFO Running runs: ['s79deysf']
2022-05-05 17:08:40 ERROR Detected 5 failed runs in a row, shutting down.
2022-05-05 17:08:40 INFO To change this value set WANDB_AGENT_MAX_INITIAL_FAILURES=val
2022-05-05 17:31:30 INFO Running runs: []
2022-05-05 17:31:30 INFO Agent received command: run
2022-05-05 17:31:30 INFO Agent starting run with config:
eval_split_name: test
eval_steps: 500
evaluation_strategy: steps
generation_max_length: 40
generation_num_beams: 1
gradient_accumulation_steps: 16
greater_is_better: True
hidden_dropout: 0.2
language: fr.en
learning_rate: 0.0006039131612823546
logging_steps: 1
max_duration_in_seconds: 20
metric_for_best_model: bleu
model_name_or_path: ./
num_train_epochs: 3
output_dir: ./
per_device_eval_batch_size: 8
per_device_train_batch_size: 8
save_steps: 500
task: covost2
warmup_steps: 500
2022-05-05 17:31:30 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=16 --greater_is_better=True --hidden_dropout=0.2 --language=fr.en --learning_rate=0.0006039131612823546 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 17:31:35 INFO Running runs: ['3sxltsfz']
2022-05-05 17:37:18 INFO Running runs: []
2022-05-05 17:37:19 INFO Agent received command: run
2022-05-05 17:37:19 INFO Agent starting run with config:
eval_split_name: test
eval_steps: 500
evaluation_strategy: steps
generation_max_length: 40
generation_num_beams: 1
gradient_accumulation_steps: 16
greater_is_better: True
hidden_dropout: 0.2
language: fr.en
learning_rate: 0.0003
logging_steps: 1
max_duration_in_seconds: 20
metric_for_best_model: bleu
model_name_or_path: ./
num_train_epochs: 3
output_dir: ./
per_device_eval_batch_size: 8
per_device_train_batch_size: 8
save_steps: 500
task: covost2
warmup_steps: 500
2022-05-05 17:37:19 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=16 --greater_is_better=True --hidden_dropout=0.2 --language=fr.en --learning_rate=0.0003 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 17:37:24 INFO Running runs: ['i9acyhfo']
2022-05-06 01:57:37 ERROR 500 response executing GraphQL.
2022-05-06 01:57:37 ERROR {"error":"driver: bad connection"}
2022-05-06 07:05:53 ERROR 500 response executing GraphQL.
2022-05-06 07:05:53 ERROR {"error":"driver: bad connection"}
2022-05-06 07:06:49 ERROR 500 response executing GraphQL.
2022-05-06 07:06:49 ERROR {"errors":[{"message":"context deadline exceeded"}]}
2022-05-06 09:14:19 ERROR 500 response executing GraphQL.
2022-05-06 09:14:19 ERROR {"errors":[{"message":"context deadline exceeded"}]}
2022-05-06 09:15:15 ERROR 500 response executing GraphQL.
2022-05-06 09:15:15 ERROR {"errors":[{"message":"context deadline exceeded"}]}
|