Update README.md
Browse files
README.md
CHANGED
|
@@ -2,7 +2,7 @@ Method and Apparatus for Dynamically Reducing Logit Computation in Large Languag
|
|
| 2 |
|
| 3 |
Detailed Description of the Invention:
|
| 4 |
|
| 5 |
-
This invention relates to a method and apparatus for dynamically reducing the computational cost of text generation in large language models (LLMs), particularly those with large vocabularies. The invention achieves this by selectively deactivating tokens and thereby reducing the computations required for generating logits in the output layer of the LLM.
|
| 6 |
|
| 7 |
Background:
|
| 8 |
|
|
|
|
| 2 |
|
| 3 |
Detailed Description of the Invention:
|
| 4 |
|
| 5 |
+
This invention relates to a method and apparatus for dynamically reducing the computational cost of text generation in large language models (LLMs), particularly those with large (token) vocabularies. The invention achieves this by selectively/dynamically deactivating tokens in the output head of the LLM (before computing their logits) and thereby reducing the computations required for generating logits in the output layer of the LLM.
|
| 6 |
|
| 7 |
Background:
|
| 8 |
|