Update README.md

README.md

@@ -40,6 +40,34 @@ import pytest
 
 import pandas as pd<N
 ```
+---
+
+## The Journey
+The model took 6 major steps which are:
+
+1. Data Collection
+2. Raw Data Cleaning
+3. Data Preprocessing
+4. Building & Training the Tokenizer
+5. Testing the Model on Large Dataset
+6. Deploying the Final Model on HuggingFace
+
+#### Data Collection
+The data was collected from python github repositories using web scraping techniques, It took nearly a day to gather 200GB worth of data.
+
+#### Raw Data Cleaning
+200GB of python code?? sounds ridiculous! that's why we needed to clean the downloaded repositories from any non-python files such as PDF,idx..etc
+
+#### Data Preprocessing
+I tried splitting the lines of code for each repository then merged them all under one single text file named **python_text_data.txt**
+
+#### Building & Training the Tokenizer
+For this step I have used **ByteLevelBPETokenizer** and trained it then saved the model on the desktop
+
+#### Testing the Model on Large Dataset
+After training the tokenizer on a large dataset, It was time for some tests to see how good is the model before proceeding.
+
+---
 
 ## Considerations: