Spaces:

---

aisheets

/

sheets

like 325

Running

Should be really easy to add! cc @frascuchon

It's done, @julien-c

nice!!! will use it

@kszucs @lhoestq is there something special to check here in AISheets to make sure the exported Parquet will chunk/deduplicate nicely?

Not sure what the app uses for the Parquet export. It's likely in JS and therefore unlikely to use the Parquet writer for Xet from Arrow

Exactly, all is TS code.

Current export is straightforward/simplistic.

1. Data is stored on DuckDB.
2. We export the file locally

COPY (SELECT ...) TO 'file.parquet' (FORMAT PARQUET)

3. We push the file to the hub with the hub js library

Here's an example of the result: https://huggingface.co/datasets/dvilasuero/hands_playing_instruments

Thanks! @lhoestq . Does the datasets library make any special preparations before sending Parquet files? Or is enough with using huggingface_hub + hf_xet?