Timm ❤️ Transformers: Use any timm model with transformers
•
25
Ross Wightman
rwightman
AI & ML interests
Computer vision, transfer learning, semi/self supervised learning, robotics.
Recent Activity
upvoted
an
article
about 11 hours ago
MiniMax-01 is Now Open-Source: Scaling Lightning Attention for the AI Agent Era
posted
an
update
about 15 hours ago
I re-worked the JuptyerLab Space template recently. It's optimized for `timm` use, but will work great with `transformers` and other libs. Updated the base image, Python 3.12, Pillow-SIMD before better CPU use with image preprocessing, and made a number of other tweaks. From the Jupyter launcher you can run the terminal and setup a `timm` environment in moments with `setup_timm_dev` or `setup_timm_scripts` helpers. Give it a try, https://huggingface.co/spaces/timm/jupyterlab-timm
updated
a model
1 day ago
timm/ese_vovnet57b.ra4_e3600_r256_in1k
Articles
Organizations
rwightman's activity
upvoted
an
article
about 11 hours ago
Article
MiniMax-01 is Now Open-Source: Scaling Lightning Attention for the AI Agent Era
By
•
•
32
posted
an
update
about 15 hours ago
Post
363
I re-worked the JuptyerLab Space template recently. It's optimized for
timm use, but will work great with transformers and other libs. Updated the base image, Python 3.12, Pillow-SIMD before better CPU use with image preprocessing, and made a number of other tweaks. From the Jupyter launcher you can run the terminal and setup a timm environment in moments with setup_timm_dev or setup_timm_scripts helpers. Give it a try,
timm/jupyterlab-timm
updated
a
model
1 day ago
published
a
model
1 day ago
updated
2
models
1 day ago
published
2
models
1 day ago
Adding `safetensors` variant of this model
#1 opened 2 days ago
by
SFconvertbot
upvoted
an
article
1 day ago
Article
Timm ❤️ Transformers: Use any timm model with transformers
•
25
Adding `safetensors` variant of this model
#1 opened 3 days ago
by
SFconvertbot
Adding `safetensors` variant of this model
#2 opened 3 days ago
by
SFconvertbot
Adding `safetensors` variant of this model
#1 opened 3 days ago
by
SFconvertbot
Adding `safetensors` variant of this model
#1 opened 3 days ago
by
SFconvertbot
reacted to
ariG23498's
post with 🚀
1 day ago
Post
1466
Timm ❤️ Transformers
Wtih the latest version of transformers you can now use any timm model with the familiar transformers API.
Blog Post: https://huggingface.co/blog/timm-transformers
Repository with examples: https://github.com/ariG23498/timm-wrapper-examples
Collection: ariG23498/timmwrapper-6777b85f1e8d085d3f1374a1
Wtih the latest version of transformers you can now use any timm model with the familiar transformers API.
Blog Post: https://huggingface.co/blog/timm-transformers
Repository with examples: https://github.com/ariG23498/timm-wrapper-examples
Collection: ariG23498/timmwrapper-6777b85f1e8d085d3f1374a1
liked
a
model
8 days ago
reacted to
merve's
post with 🔥
8 days ago
Post
1746
ByteDance just dropped SA2VA: a new family of vision LMs combining Qwen2VL/InternVL and SAM2 with MIT license 💗
ByteDance/sa2va-model-zoo-677e3084d71b5f108d00e093
> The models are capable of tasks involving vision-language understanding and visual referrals (referring segmentation) both for images and videos ⏯️
> The models come in 1B, 4B and 8B and are based on InternVL2.5 for base architecture and Qwen2, Qwen2.5 and InternLM2 for language model part (depending on the checkpoint)
> The model is very interesting, it has different encoders for different modalities each (visual prompt, text prompt, image and video) then it concatenates these to feed into LLM 💬
the output segmentation tokens are passed to SAM2, to sort of match text (captions or semantic classes) to masks ⤵️
> Their annotation pipeline is also interesting, they seems to use two open large vision LMs to refine the annotations, and have different levels of descriptions to provide consistency.
> The models are capable of tasks involving vision-language understanding and visual referrals (referring segmentation) both for images and videos ⏯️
> The models come in 1B, 4B and 8B and are based on InternVL2.5 for base architecture and Qwen2, Qwen2.5 and InternLM2 for language model part (depending on the checkpoint)
> The model is very interesting, it has different encoders for different modalities each (visual prompt, text prompt, image and video) then it concatenates these to feed into LLM 💬
the output segmentation tokens are passed to SAM2, to sort of match text (captions or semantic classes) to masks ⤵️
> Their annotation pipeline is also interesting, they seems to use two open large vision LMs to refine the annotations, and have different levels of descriptions to provide consistency.
- 1 reply
posted
an
update
8 days ago
Post
1082
New
* PaliGemma2 encoders (ported from google/paligemma-2-release-67500e1e1dbfdd4dee27ba48)
* AIMv-2 encoders (ported from apple/aimv2-6720fe1558d94c7805f7688c)
A few higher resolution 384x384 ConvNeXt-Nano ImageNet-12k pretrain & finetunes. See other changes here: https://github.com/huggingface/pytorch-image-models/releases/tag/v1.0.13
And support added in both
* DFN CLIP L/14 w/ 39B samples seen - apple/DFN2B-CLIP-ViT-L-14-39B, timm/vit_large_patch14_clip_224.dfn2b_s39b
* MetaCLIP H/14 (altogether) - timm/vit_huge_patch14_clip_224.metaclip_altogether
And last, ~70-80 models that were relying on
timm 1.0.13 and OpenCLIP 2.30.0 releases to start the year. Both modest but worthwhile updates.timm added a number of new model weights, supporting loading of: * PaliGemma2 encoders (ported from google/paligemma-2-release-67500e1e1dbfdd4dee27ba48)
* AIMv-2 encoders (ported from apple/aimv2-6720fe1558d94c7805f7688c)
A few higher resolution 384x384 ConvNeXt-Nano ImageNet-12k pretrain & finetunes. See other changes here: https://github.com/huggingface/pytorch-image-models/releases/tag/v1.0.13
And support added in both
OpenCLIP and timm for two CLIP models that were missed. The DFN L/14 is 🔥* DFN CLIP L/14 w/ 39B samples seen - apple/DFN2B-CLIP-ViT-L-14-39B, timm/vit_large_patch14_clip_224.dfn2b_s39b
* MetaCLIP H/14 (altogether) - timm/vit_huge_patch14_clip_224.metaclip_altogether
And last, ~70-80 models that were relying on
timm remapping from OpenCLIP got their own timm hub instances to allow use with the upcoming Transformers TimmWrapperModel
liked
a
model
14 days ago
liked
a
model
15 days ago