update README on how to upload to huggingface
Browse files
README.md
CHANGED
|
@@ -29,3 +29,15 @@ The test file is tab-separated, the first column is additional positions you wan
|
|
| 29 |
|
| 30 |
### Example output
|
| 31 |

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
### Example output
|
| 31 |

|
| 32 |
+
|
| 33 |
+
### Upload to huggingface
|
| 34 |
+
This repo is hosted on [Github](https://github.com/approach0/azbert), and only mirrored at [huggingface](https://huggingface.co/castorini/azbert-base).
|
| 35 |
+
|
| 36 |
+
To upload to huggingface, use the `upload2hgf.sh` script.
|
| 37 |
+
Before runnig this script, be sure to check:
|
| 38 |
+
* check points for model and tokenizer are created under `./ckpt` folder
|
| 39 |
+
* model contains all the files needed: `config.json` and `pytorch_model.bin`
|
| 40 |
+
* tokenizer contains all the files needed: `added_tokens.json`, `special_tokens_map.json`, `tokenizer_config.json`, `vocab.txt` and `tokenizer.json`
|
| 41 |
+
* no `tokenizer_file` field in `tokenizer_config.json` (sometimes it is located locally at `~/.cache`)
|
| 42 |
+
* `git-lfs` is installed
|
| 43 |
+
* having git-remote named `hgf` reference to `https://huggingface.co/castorini/azbert-base`
|