Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI Perspective
Paper
β’
2503.01933
β’
Published
β’
10
Rakshit Aralimatti PRO
RakshitAralimatti
AI & ML interests
Poor GPU Guy
Recent Activity
upvoted
a
paper
4 days ago
Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI
Perspective
published
a model
about 1 month ago
RakshitAralimatti/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M-GGUF
Organizations
RakshitAralimatti's activity
upvoted
a
paper
4 days ago
reacted to
hexgrad's
post with π₯
28 days ago
Post
6869
Wanted: Peak Data. I'm collecting audio data to train another TTS model:
+ AVM data: ChatGPT Advanced Voice Mode audio & text from source
+ Professional audio: Permissive (CC0, Apache, MIT, CC-BY)
This audio should *impress* most native speakers, not just barely pass their audio Turing tests. Professional-caliber means S or A-tier, not your average bloke off the street. Traditional TTS may not make the cut. Absolutely no low-fi microphone recordings like Common Voice.
The bar is much higher than last time, so there are no timelines yet and I expect it may take longer to collect such mythical data. Raising the bar means evicting quite a bit of old data, and voice/language availability may decrease. The theme is *quality* over quantity. I would rather have 1 hour of A/S-tier than 100 hours of mid data.
I have nothing to offer but the north star of a future Apache 2.0 TTS model, so prefer data that you *already have* and costs you *nothing extra* to send. Additionally, *all* the new data may be used to construct public, Apache 2.0 voicepacks, and if that arrangement doesn't work for you, no need to send any audio.
Last time I asked for horses; now I'm asking for unicorns. As of writing this post, I've currently got a few English & Chinese unicorns, but there is plenty of room in the stable. Find me over on Discord at
+ AVM data: ChatGPT Advanced Voice Mode audio & text from source
+ Professional audio: Permissive (CC0, Apache, MIT, CC-BY)
This audio should *impress* most native speakers, not just barely pass their audio Turing tests. Professional-caliber means S or A-tier, not your average bloke off the street. Traditional TTS may not make the cut. Absolutely no low-fi microphone recordings like Common Voice.
The bar is much higher than last time, so there are no timelines yet and I expect it may take longer to collect such mythical data. Raising the bar means evicting quite a bit of old data, and voice/language availability may decrease. The theme is *quality* over quantity. I would rather have 1 hour of A/S-tier than 100 hours of mid data.
I have nothing to offer but the north star of a future Apache 2.0 TTS model, so prefer data that you *already have* and costs you *nothing extra* to send. Additionally, *all* the new data may be used to construct public, Apache 2.0 voicepacks, and if that arrangement doesn't work for you, no need to send any audio.
Last time I asked for horses; now I'm asking for unicorns. As of writing this post, I've currently got a few English & Chinese unicorns, but there is plenty of room in the stable. Find me over on Discord at
rzvzn: https://discord.gg/QuGxSWBfQy
- 4 replies
published
a
model
about 1 month ago
updated
a
model
about 1 month ago
GGUF Models
#1 opened 2 months ago
by
RakshitAralimatti
updated
2
models
3 months ago
Model Selection dout
3
#3 opened 3 months ago
by
RakshitAralimatti
updated
2
models
3 months ago
Adding `safetensors` variant of this model
1
#1 opened 3 months ago
by
SFconvertbot
updated
a
model
3 months ago
Getting Error
3
#1 opened 6 months ago
by
RakshitAralimatti
Size of the Dataset?
6
#9 opened 4 months ago
by
RakshitAralimatti
Total Size of dataset ?
#14 opened 4 months ago
by
RakshitAralimatti
upvoted
a
paper
5 months ago
authored
a
paper
5 months ago
Dataset Used for Fine-tuning
#1 opened 5 months ago
by
RakshitAralimatti
reacted to
bartowski's
post with β€οΈ
7 months ago
Post
10083
So turns out I've been spreading a bit of misinformation when it comes to imatrix in llama.cpp
It starts true; imatrix runs the model against a corpus of text and tracks the activation of weights to determine which are most important
However what the quantization then does with that information is where I was wrong.
I think I made the accidental connection between imatrix and exllamav2's measuring, where ExLlamaV2 decides how many bits to assign to which weight depending on the goal BPW
Instead, what llama.cpp with imatrix does is it attempts to select a scale for a quantization block that most accurately returns the important weights to their original values, ie minimizing the dequantization error based on the importance of activations
The mildly surprising part is that it actually just does a relatively brute force search, it picks a bunch of scales and tries each and sees which one results in the minimum error for weights deemed important in the group
But yeah, turns out, the quantization scheme is always the same, it's just that the scaling has a bit more logic to it when you use imatrix
Huge shoutout to @compilade for helping me wrap my head around it - feel free to add/correct as well if I've messed something up
It starts true; imatrix runs the model against a corpus of text and tracks the activation of weights to determine which are most important
However what the quantization then does with that information is where I was wrong.
I think I made the accidental connection between imatrix and exllamav2's measuring, where ExLlamaV2 decides how many bits to assign to which weight depending on the goal BPW
Instead, what llama.cpp with imatrix does is it attempts to select a scale for a quantization block that most accurately returns the important weights to their original values, ie minimizing the dequantization error based on the importance of activations
The mildly surprising part is that it actually just does a relatively brute force search, it picks a bunch of scales and tries each and sees which one results in the minimum error for weights deemed important in the group
But yeah, turns out, the quantization scheme is always the same, it's just that the scaling has a bit more logic to it when you use imatrix
Huge shoutout to @compilade for helping me wrap my head around it - feel free to add/correct as well if I've messed something up
- 5 replies
liked
a
model
7 months ago
