Working locally, TBD HF space

Removing CUDA deps

Not sure what happened, more deps

Adding again

Removing gcc and gnu deps from req file

Init for HF space

Visualisation working on CPU via CUDA_VISIBLE_DEVICE=-1 python demo_patcher.py 'Daenerys Targaryen is in Game of Thrones, a fantasy epic by George R.R. Martin.'

Improve HF integration (#98)

Open source weights! (#97)

Cast int sample id to str (#96)

Init distributed when loading model (#94)

Fix eval mask (#93)

remove selective activation checkpointing (#92)

update (#91)

Get generation working for BLT (#86)

Fix in-place addition of patch_embds (#85)

Some fixes for entropy model predictions (#83)

Update ppl evals to work with blt model, in addition to entropy model (#82)

Update iterate_data (#81)

Add way to call consolidate (#80)

When merging configs, do not merge data sources (#79)

Get evals working again. (#46)

Reduce per file resources arrow uses (#77)

Let process start before yielding preloaded prefetch buffer, avoid needlessly losing buffer in edge cases (#75)

Add approximate state persistence (#73)

Fix rsync to not preserve original permissions, instead use destination (#76)

Correctly reset batch iterator at each arrow create_iter call. (#74)

Pass mask in packing_iterator, correctly handle last batch, fix masking (#65)

Initialize rope embeddings properly for the entropy model (#72)

Remove byte tokenizer and add config args to switch between byte/patch packing (#68)

Add vocab and seq len abstract fields (#66)

Fix: Correct model_args usage in parallelize_model call (#69)

Update iterator inheritance, pass file format args, limit iterator (#63)

Make apex logs less noisy (#60)

Make it possible to specify multiple config files (#54)

fix(README): correct typo in quickstart instructions (#62)

using apex rmsnorm (#57)

Update README.md (#58)

Fix multiprocessing dataloader checkpointing and use it in the train script (#50)

Test first batch matches (#53)

disable reshard after forward (#56)

make sure max_encoder_seq_length matches (#55)

fix save and reload model state (#49)

Add bpb and n_bytes to metric logging (#41)

Fix init and repro (#48)

Allow ArrowIterator to read from json (#45)

Update checkpointing to use fsspec (#39)

Add rope fp32 (#43)

fix stool (#44)

Fix wandb logging (#42)