Nuclear Norm Regularization for Deep Learning
Paper
•
2405.14544
•
Published
•
1
Representation & Optimization
Understanding about representation sheds light on optimization
Note CS inequality for matrix allows penalizing element-wise Frobenius norm to encourage low-rank representations.
Token embeddings violate the manifold hypothesis
Paper • 2504.01002 • Published • 1Note Some token have more synonyms than others.
-
Approximate Nullspace Augmented Finetuning for Robust Vision Transformers
Paper • 2403.10476 • Published • 1 -
ElaLoRA: Elastic & Learnable Low-Rank Adaptation for Efficient Model Fine-Tuning
Paper • 2504.00254 • Published • 1
Flex Attention: A Programming Model for Generating Optimized Attention Kernels
Paper • 2412.05496 • Published • 1Note Customize attention mask with optimized performance comparable with Flashattention
-
Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad
Paper • 2503.21934 • Published
Value Residual Learning For Alleviating Attention Concentration In Transformers
Paper • 2410.17897 • Published • 9Note Halve KV cache via sharing value embedding across attention blocks
-
Hogwild! Inference: Parallel LLM Generation via Concurrent Attention
Paper • 2504.06261 • Published • 104 -
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
Paper • 2503.01840 • Published • 5 -
Is the Reversal Curse a Binding Problem? Uncovering Limitations of Transformers from a Basic Generalization Failure
Paper • 2504.01928 • Published • 1 -
Gradient Surgery for Multi-Task Learning
Paper • 2001.06782 • Published • 1 -
SelfCP: Compressing Long Prompt to 1/12 Using the Frozen Large Language Model Itself
Paper • 2405.17052 • Published • 2 -
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
Paper • 2403.19647 • Published • 4 -
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
Paper • 2504.13837 • Published • 106 -
It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization
Paper • 2504.13173 • Published • 17 -
Representation Learning with Contrastive Predictive Coding
Paper • 1807.03748 • Published • 1 -
Training Large Language Models to Reason in a Continuous Latent Space
Paper • 2412.06769 • Published • 84 -
SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference
Paper • 2502.18137 • Published • 57 -
Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Light
Paper • 2504.16922 • Published • 1 -
Interpreting Emergent Planning in Model-Free Reinforcement Learning
Paper • 2504.01871 • Published • 12