CryptGPT: Privacy-Preserving Language Models Using Vigenere Cipher (Part 1)
•
5
Diwank Tomer
diwank
AI & ML interests
None yet
Recent Activity
liked
a model
about 11 hours ago
nvidia/AceInstruct-72B
updated
a collection
1 day ago
Vision
liked
a model
1 day ago
NimVideo/mochi-1-transformer-42
Articles
Organizations
Posts
2
Post
2212
Just published "CryptGPT: A Simple Approach to Privacy-Preserving Language Models Using the Vigenere Cipher".
https://huggingface.co/blog/diwank/cryptgpt-part1
tl;dr - we pretrained a gpt-2 tokenizer and model from scratch on a dataset encrypted with Vigenere cipher and it performs as well as regular gpt-2. Except in order to use it, you need to know the encryption key.
links:
https://github.com/creatorrr/cryptgpt
diwank/cryptgpt
diwank/cryptgpt-large
https://huggingface.co/blog/diwank/cryptgpt-part1
tl;dr - we pretrained a gpt-2 tokenizer and model from scratch on a dataset encrypted with Vigenere cipher and it performs as well as regular gpt-2. Except in order to use it, you need to know the encryption key.
links:
https://github.com/creatorrr/cryptgpt
diwank/cryptgpt
diwank/cryptgpt-large
Post
1706
Really excited to read about Kolmogorov Arnold Networks as a novel alternatives to Multi Layer Perceptrons.
Excerpt:
> Kolmogorov-Arnold Networks (KANs) are promising alternatives of Multi-Layer Perceptrons (MLPs). KANs have strong mathematical foundations just like MLPs: MLPs are based on the universal approximation theorem, while KANs are based on Kolmogorov-Arnold representation theorem. KANs and MLPs are dual: KANs have activation functions on edges, while MLPs have activation functions on nodes. This simple change makes KANs better (sometimes much better!) than MLPs in terms of both model accuracy and interpretability.
https://github.com/KindXiaoming/pykan
Excerpt:
> Kolmogorov-Arnold Networks (KANs) are promising alternatives of Multi-Layer Perceptrons (MLPs). KANs have strong mathematical foundations just like MLPs: MLPs are based on the universal approximation theorem, while KANs are based on Kolmogorov-Arnold representation theorem. KANs and MLPs are dual: KANs have activation functions on edges, while MLPs have activation functions on nodes. This simple change makes KANs better (sometimes much better!) than MLPs in terms of both model accuracy and interpretability.
https://github.com/KindXiaoming/pykan
models
12
diwank/hn-upvote-classifier
Text Classification
•
Updated
•
15
diwank/cryptgpt-large
Text Generation
•
Updated
•
125
diwank/cryptgpt
Text Generation
•
Updated
•
21
diwank/dfe-large-en-2
Sentence Similarity
•
Updated
•
7
diwank/dfe-base-en-1
Sentence Similarity
•
Updated
•
51
diwank/dfe-base-en
Updated
diwank/dial-flan-silicone
Text2Text Generation
•
Updated
•
3
diwank/dial-t0-silicone
Text2Text Generation
•
Updated
•
5
diwank/bartner
Text2Text Generation
•
Updated
•
8
diwank/silicone-deberta-pair
Text Classification
•
Updated
•
15
datasets
69
diwank/hn-upvote-data
Viewer
•
Updated
•
9.66k
•
46
diwank/chessllm-processed
Viewer
•
Updated
•
19.9k
•
33
diwank/fineweb-edu-2024-10-1M_tokenized-qwen2_1024
Viewer
•
Updated
•
1.03M
•
74
diwank/conceptnet-with-descriptions
Viewer
•
Updated
•
3.15M
•
30
diwank/llmlingua-compressed-text
Viewer
•
Updated
•
222k
•
36
•
2
diwank/python-code-execution-chatml
Viewer
•
Updated
•
3.61k
•
17
diwank/python-code-execution-output
Viewer
•
Updated
•
3.61k
•
36
•
1
diwank/slimorca-corrections-dpo-chatml
Viewer
•
Updated
•
10.3k
•
16
diwank/slimorca-corrected-chatml
Viewer
•
Updated
•
359k
•
57
diwank/slimorca-autoj-corrected
Viewer
•
Updated
•
359k
•
16