liming's picture

liming

largegpt
ยท

AI & ML interests

None yet

Recent Activity

Organizations

None yet

largegpt's activity

view reply

If we don't store every chunk hash to block hash, how can we download a model file. For example if a model does not have a key chunk, the shard info only contains "file hash -> list of chunks", the CAS only has "block hash -> block content", how can we know which block is needed ?

view reply

How can we find the chunk content using chunk hash? The CAS system only stores "block_hash -> block_content", Where does the map of chunk to block?

what does the shards store? Is it "file_name, shard_id, chunk_hash, block_hash"?