Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper
•
2203.05482
•
Published
•
6
Papers about model merging
referenced in the mergekit repo: https://github.com/cg123/mergekit
Note Linear merging
Editing Models with Task Arithmetic
Paper • 2212.04089 • Published • 6Note Task Arithmetic
Resolving Interference When Merging Models
Paper • 2306.01708 • Published • 13Note TIES
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
Paper • 2311.03099 • Published • 28Note DARE