Text Generation
Transformers
PyTorch
Safetensors
English
llama
finance
text-generation-inference
Inference Endpoints
AdaptLLM commited on
Commit
4106840
·
1 Parent(s): 9a9528c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -4,6 +4,10 @@ This repo contains the model for our paper [Adapting Large Language Models via R
4
 
5
  We explore **continued pre-training on domain-specific corpora** for large language models. While this approach enriches LLMs with domain knowledge, it significantly hurts their prompting ability for question answering. Inspired by human learning via reading comprehension, we propose a simple method to **transform large-scale pre-training corpora into reading comprehension texts**, consistently improving prompting performance across tasks in **biomedicine, finance, and law domains**. Our 7B model competes with much larger domain-specific models like BloombergGPT-50B. Moreover, our domain-specific reading comprehension texts enhance model performance even on general benchmarks, indicating potential for developing a general LLM across more domains.
6
 
 
 
 
 
7
  ## GitHub repo:
8
  https://github.com/microsoft/LMOps
9
 
@@ -11,7 +15,7 @@ https://github.com/microsoft/LMOps
11
  Our models of different domains are now available in Huggingface: [Biomedicine-LLM](https://huggingface.co/AdaptLLM/medicine-LLM), [Finance-LLM](https://huggingface.co/AdaptLLM/finance-LLM) and [Law-LLM](https://huggingface.co/AdaptLLM/law-LLM), the performances of our AdaptLLM compared to other domain-specific LLMs are:
12
 
13
  <p align='center'>
14
- <img src="./comparison.png" width="700">
15
  </p>
16
 
17
  ## Domain-specific Tasks:
 
4
 
5
  We explore **continued pre-training on domain-specific corpora** for large language models. While this approach enriches LLMs with domain knowledge, it significantly hurts their prompting ability for question answering. Inspired by human learning via reading comprehension, we propose a simple method to **transform large-scale pre-training corpora into reading comprehension texts**, consistently improving prompting performance across tasks in **biomedicine, finance, and law domains**. Our 7B model competes with much larger domain-specific models like BloombergGPT-50B. Moreover, our domain-specific reading comprehension texts enhance model performance even on general benchmarks, indicating potential for developing a general LLM across more domains.
6
 
7
+ <p align='center'>
8
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/650801ced5578ef7e20b33d4/Qi89o8zcwqbLtdhtNbMvV.png" width="700">
9
+ </p>
10
+
11
  ## GitHub repo:
12
  https://github.com/microsoft/LMOps
13
 
 
15
  Our models of different domains are now available in Huggingface: [Biomedicine-LLM](https://huggingface.co/AdaptLLM/medicine-LLM), [Finance-LLM](https://huggingface.co/AdaptLLM/finance-LLM) and [Law-LLM](https://huggingface.co/AdaptLLM/law-LLM), the performances of our AdaptLLM compared to other domain-specific LLMs are:
16
 
17
  <p align='center'>
18
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/650801ced5578ef7e20b33d4/6efPwitFgy-pLTzvccdcP.png" width="700">
19
  </p>
20
 
21
  ## Domain-specific Tasks: