Brad's picture

2 1 7

Brad

Firepal3D

AI & ML interests

None yet

Recent Activity

reacted to grimjim's post with 🔥 25 days ago

I recently have been looking at a paper titled "Why Warmup the Learning Rate? Underlying Mechanisms and Improvements", by Dayal Singh Kalra and Maissam Barkeshli, and was struck by "warmup" being analogous to simulated annealing. https://arxiv.org/abs/2406.09405 Taking the physical analogy further, the "warmup" is a stochastic process to knock the system out of current local minima, allowing easier transition toward newer minima. It works because it reduces "fit" and therefore "friction".

liked a dataset about 1 month ago

inclinedadarsh/nl-to-regex

View all activity

Organizations

None yet

models 0

None public yet

datasets 0

None public yet