turboderp/Llama-3.3-Nemotron-Super-49B-v1-exl3
Updated
•
20
•
6
Mindkrypted
mindkrypted
AI & ML interests
*.*
Recent Activity
liked
a model
1 day ago
turboderp/Llama-3.3-Nemotron-Super-49B-v1-exl3
liked
a model
6 days ago
turboderp/Mistral-Large-Instruct-2411-exl3
Organizations
mindkrypted's activity
liked
a
model
1 day ago
liked
a
model
6 days ago
reacted to
sr-rai's
post with 🤗
8 days ago
Post
2591
ExLlamaV3 is out. And it introduces EXL3 - a new SOTA quantization format!
"The conversion process is designed to be simple and efficient and requires only an input model (in HF format) and a target bitrate. By computing Hessians on the fly and thanks to a fused Viterbi kernel, the quantizer can convert a model in a single step, taking a couple of minutes for smaller models, up to a few hours for larger ones (70B+) (on a single RTX 4090 or equivalent GPU.)"
Repo: https://github.com/turboderp-org/exllamav3
"The conversion process is designed to be simple and efficient and requires only an input model (in HF format) and a target bitrate. By computing Hessians on the fly and thanks to a fused Viterbi kernel, the quantizer can convert a model in a single step, taking a couple of minutes for smaller models, up to a few hours for larger ones (70B+) (on a single RTX 4090 or equivalent GPU.)"
Repo: https://github.com/turboderp-org/exllamav3
- 1 reply
upvoted
a
collection
9 days ago
liked
a
model
26 days ago
Fix required for config.json >> BOS Token ID - Incoherent outputs
1
#8 opened 3 months ago
by
mindkrypted
liked
a
model
about 2 months ago
liked
a
Space
2 months ago
liked
a
model
2 months ago
liked
a
model
3 months ago
French voice (FF Alpha) has more of a Quebec French accent than a French French accent
4
#4 opened 3 months ago
by
stduhpf
reacted to
bartowski's
post with 👍
3 months ago
Post
73027
Switching to
I posted a poll on twitter, and others have mentioned the interest in me using the convention of including the author name in the model path when I upload.
It has a couple advantages, first and foremost of course is ensuring clarity of who uploaded the original model (did Qwen upload Qwen2.6? Or did someone fine tune Qwen2.5 and named it 2.6 for fun?)
The second thing is that it avoids collisions, so if multiple people upload the same model and I try to quant them both, I would normally end up colliding and being unable to upload both
I'll be implementing the change next week, there are just two final details I'm unsure about:
First, should the files also inherit the author's name?
Second, what to do in the case that the author name + model name pushes us past the character limit?
Haven't yet decided how to handle either case, so feedback is welcome, but also just providing this as a "heads up"
author_model-nameI posted a poll on twitter, and others have mentioned the interest in me using the convention of including the author name in the model path when I upload.
It has a couple advantages, first and foremost of course is ensuring clarity of who uploaded the original model (did Qwen upload Qwen2.6? Or did someone fine tune Qwen2.5 and named it 2.6 for fun?)
The second thing is that it avoids collisions, so if multiple people upload the same model and I try to quant them both, I would normally end up colliding and being unable to upload both
I'll be implementing the change next week, there are just two final details I'm unsure about:
First, should the files also inherit the author's name?
Second, what to do in the case that the author name + model name pushes us past the character limit?
Haven't yet decided how to handle either case, so feedback is welcome, but also just providing this as a "heads up"
- 5 replies
liked
a
model
4 months ago
liked
a
Space
4 months ago