ctheodoris

/

Geneformer

like 244

I am following the cell_classification.ipynb , and I meet
_________________________________________________
Some weights of BertForSequenceClassification were not initialized from the model checkpoint at /disk2/cai045/Geneformer/gf-12L-30M-i2048/ and are newly initialized: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'classifier.bias', 'classifier.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

TypeError Traceback (most recent call last)
Cell In[5], line 6
1 train_valid_id_split_dict = {"attr_key": "individual",
2 "train": train_ids,
3 "eval": eval_ids}
5 # Example 6 layer 30M Geneformer model: https://huggingface.co/ctheodoris/Geneformer/blob/main/gf-6L-30M-i2048/model.safetensors
----> 6 all_metrics = cc.validate(model_directory="/disk2/cai045/Geneformer/gf-12L-30M-i2048/",
7 prepared_input_data_file=f"{output_dir}/{output_prefix}_labeled_train.dataset",
8 id_class_dict_file=f"{output_dir}/{output_prefix}_id_class_dict.pkl",
9 output_directory=output_dir,
10 output_prefix=output_prefix,
11 split_id_dict=train_valid_id_split_dict)
12 # to optimize hyperparameters, set n_hyperopt_trials=100 (or alternative desired # of trials)

File ~/anaconda3/envs/geneformer/lib/python3.10/site-packages/geneformer/classifier.py:791, in Classifier.validate(self, model_directory, prepared_input_data_file, id_class_dict_file, output_directory, output_prefix, split_id_dict, attr_to_split, attr_to_balance, gene_balance, max_trials, pval_threshold, save_eval_output, predict_eval, predict_trainer, n_hyperopt_trials, save_gene_split_datasets, debug_gene_split_datasets)
789 train_data = data.select(train_indices)
790 if n_hyperopt_trials == 0:
--> 791 trainer = self.train_classifier(
792 model_directory,
793 num_classes,
794 train_data,
795 eval_data,
796 ksplit_output_dir,
797 predict_trainer,
798 )
799 else:
800 trainer = self.hyperopt_classifier(
801 model_directory,
802 num_classes,
(...)
806 n_trials=n_hyperopt_trials,
807 )

File ~/anaconda3/envs/geneformer/lib/python3.10/site-packages/geneformer/classifier.py:1236, in Classifier.train_classifier(self, model_directory, num_classes, train_data, eval_data, output_directory, predict)
1234 def_training_args["evaluation_strategy"] = "no"
1235 def_training_args["load_best_model_at_end"] = False
-> 1236 training_args_init = TrainingArguments(**def_training_args)
1238 if self.freeze_layers is not None:
1239 def_freeze_layers = self.freeze_layers

TypeError: TrainingArguments.init() got an unexpected keyword argument 'evaluation_strategy'

My transformers version is 4.51.3