Hugging Face's logo
tim-lawson 's Collections
Learning to Skip the Middle Layers of Transformers
Multi-Layer SAEs
Multi-Layer SAEs with Tuned Lens
Multi-Layer SAEs with Transformers
Multi-Layer SAEs with Tuned Lens and Transformers
Single-Layer SAEs
Single-Layer SAEs with Transformers

Learning to Skip the Middle Layers of Transformers

Transformers with a novel gating mechanism that skips layers from the middle outward: https://arxiv.org/pdf/2506.21103

Upvote
-