Update README.md

README.md

@@ -11,41 +11,71 @@ tags:
 - falcon-7b
 - custom_code
 - text-generation-inference
+- endpoints-template
 metrics:
 - exact_match
 - f1
 pipeline_tag: text-generation
 inference: false
 ---
+# 🚀 falcon-7b-finetuned-QA-MRC-4-bit
+
+Falcon-7b-finetuned-QA-MRC-4-bit is a model for Machine Reading Comprehension (MRC) with Question Answering (QA). It was built by fine-tuning [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) on the [SQuAD2.0](https://huggingface.co/datasets/squad_v2) dataset. This repo only includes the LoRA adapters from fine-tuning with 🤗's [peft](https://github.com/huggingface/peft) package. 
+
+## Model Summary
+
+- **Model Type:** Causal decoder-only
+- **Language(s):** English
+- **Base Model:** [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) (License: [Apache 2.0](https://huggingface.co/tiiuae/falcon-7b#license))
+- **Dataset:** [SQuAD2.0](https://huggingface.co/datasets/squad_v2) (License: cc-by-sa-4.0)
+- **License(s):** Apache 2.0 inherited from "Base Model" and cc-by-sa-4.0 inherited from "Dataset"
+
+## Model Details
+
+The model was fine-tuned in 4-bit precision using 🤗 `peft` adapters, `transformers`, and `bitsandbytes`. Training relied on a method called "Low Rank Adapters" ([LoRA](https://arxiv.org/pdf/2106.09685.pdf)), specifically the [QLoRA](https://arxiv.org/abs/2305.14314) variant. The run took **approximately 5.08 hours** and was executed on a workstation with **a single A100-SXM NVIDIA GPU** with 37 GB of available memory.
+
+### Model Date
+
+August 08, 2023
+
 ## Usage
+
 ### Prompt
+
 The model was trained on the following kind of prompt:
+
 ```python
-def format_prompt(question, context):
-  return f"""Answer the question based on the context below. If the question cannot be answered using the information provided answer with 'No answer'. Stop response if end.
+"""Answer the question based on the context below. If the question cannot be answered using the information provided answer with 'No answer'. Stop response if end.
 >>TITLE<<: Flawless answer.
 >>CONTEXT<<: {context}
 >>QUESTION<<: {question}
 >>ANSWER<<:
-""".strip()
+"""
 ```
-### Example
-[Notebook](https://colab.research.google.com/drive/1d2WP-MimF34NN72wGU0gX0uSUTHirN8A?usp=sharing)
+
+### Inference
+
+You will need **at least 6GB of memory** to swiftly run inference.
+
+[Colab Notebook](https://colab.research.google.com/drive/1d2WP-MimF34NN72wGU0gX0uSUTHirN8A?usp=sharing)
+
+#### Example 1:
 ```python
 context = '''The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain "Amazonas" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species.'''
 question = '''Which name is also used to describe the Amazon rainforest in English?'''
 
->>> Amazonia or the Amazon Jungle
+>>> 'Amazonia or the Amazon Jungle'
 ```
 
+#### Example 2 (No answer):
 ```python
-context1 = '''The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain "Amazonas" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species.'''
-question1 = '''What is 2 + 2?'''
+context = '''The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain "Amazonas" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species.'''
+question = '''What is 2 + 2?'''
 
->>> No answer
+>>> 'No answer'
 ```
-## Training procedure
 
+## Training procedure
 
 The following `bitsandbytes` quantization config was used during training:
 - load_in_8bit: False
@@ -57,26 +87,30 @@ The following `bitsandbytes` quantization config was used during training:
 - bnb_4bit_quant_type: nf4
 - bnb_4bit_use_double_quant: True
 - bnb_4bit_compute_dtype: float16
+
 ### Performance
+
 Evaluated on the SQuAD 2.0 dev set with the [Metrics](https://huggingface.co/docs/datasets/v2.14.4/en/loading#metrics)
 
-```
-'exact': 69.83684838838042
-'f1': 74.4130429770687
-'total': 2513
-
-'HasAns_exact': 66.5625
-'HasAns_f1': 75.546857032323
-'HasAns_total': 1280
-'NoAns_exact': 73.2360097323601
-'NoAns_f1': 73.2360097323601
-'NoAns_total': 1233
-'best_exact': 69.8766414643852
+```python
+'exact': 70.1779158429822
+'f1': 75.06205246831128
+'total': 3541
+
+'HasAns_exact': 65.49104720564297
+'HasAns_f1': 74.87505577335338
+'HasAns_total': 1843
+'NoAns_exact': 75.26501766784452
+'NoAns_f1': 75.26501766784452
+'NoAns_total': 1698
+'best_exact': 70.20615645297939
 'best_exact_thresh': 0.0
-'best_f1': 74.4528360530736
+'best_f1': 75.09029307830835
 'best_f1_thresh': 0.0
 ```
+
 ### Framework versions
+
 - PEFT 0.5.0.dev0
 - Transformers 4.31.0
 - Datasets 2.14.4