hotchpotch
/

fineweb-2-japanese-text-cleaner

Model card Files Files and versions Community

fineweb-2-japanese-text-cleaner

Ctrl+K

Ctrl+K

1 contributor

History: 5 commits

hotchpotch's picture

Update README.md

a54bbe1 verified 3 months ago

scripts
Upload 2 files 3 months ago
.gitattributes

1.57 kB

Upload folder using huggingface_hub 3 months ago
README.md

3.57 kB

Update README.md 3 months ago
config.json

705 Bytes

Upload folder using huggingface_hub 3 months ago
model.safetensors

427 MB
LFS

Upload folder using huggingface_hub 3 months ago
sentencepiece.bpe.model

5.07 MB
LFS

Upload folder using huggingface_hub 3 months ago
special_tokens_map.json

964 Bytes

Upload folder using huggingface_hub 3 months ago
tokenizer.json

17.1 MB
LFS

Upload folder using huggingface_hub 3 months ago
tokenizer_config.json

1.2 kB

Upload folder using huggingface_hub 3 months ago
training_args.bin
Detected Pickle imports (10)
- "transformers.training_args.TrainingArguments",
- "torch.device",
- "transformers.trainer_pt_utils.AcceleratorConfig",
- "transformers.training_args.OptimizerNames",
- "transformers.trainer_utils.HubStrategy",
- "accelerate.utils.dataclasses.DistributedType",
- "transformers.trainer_utils.SaveStrategy",
- "transformers.trainer_utils.SchedulerType",
- "transformers.trainer_utils.IntervalStrategy",
- "accelerate.state.PartialState"
How to fix it?
5.3 kB
LFS

Upload folder using huggingface_hub 3 months ago