Attn-Only-2L512W-Shortformer-6B-big-lr
NeelNanda/Attn-Only-2L512W-Shortformer-6B-big-lr
transformers
Tracked by OpenModelStats since August 24, 2026View on Hugging Face ↗
- Downloads 30DRolling recent-window download count as reported by the source platform.
- 1,480
- Total downloadsCumulative all-time download counter reported by the source platform.
- 27.1K
- Gained 7DIncrease in cumulative total downloads between OpenModelStats' own observations seven days apart.
- —
- Gained 30DIncrease in cumulative total downloads between OpenModelStats' own observations roughly thirty days apart.
- —
- LikesCommunity likes reported by the source platform.
- 0
- Rank · tracked modelsPosition among all eligible models tracked by OpenModelStats, ordered by recent downloads.
- #20,710
- –this week
- Parameters
- —
- Last source update
- Oct 29, 2022
Downloads over time
Daily gain between consecutive observationsNot enough tracked history yet for a chart.
OpenModelStats began tracking this model on 2026-08-24. Charts appear as daily observations accumulate.
Likes over time
Not enough tracked history yet for a chart.
OpenModelStats began tracking this model on 2026-08-24. Charts appear as daily observations accumulate.
Rank among tracked models
Lower is betterNot enough tracked history yet for a chart.
OpenModelStats began tracking this model on 2026-08-24. Charts appear as daily observations accumulate.
Overview
Attn-Only-2L512W-Shortformer-6B-big-lr is an AI model published by NeelNanda. OpenModelStats has tracked the model since Aug 24, 2026, most recently observing it 2h ago.
- Published
- Oct 19, 2022
- Last updated
- Oct 29, 2022
- Library
- transformers
- License
- —
- Task
- —
- Parameters
- —
- Spaces
- —
- Derivative models
- —
- Velocity 7D/day
- —
transformersendpoints_compatibleregion:us
Publisher
NeelNandaView publisher statistics →Similar models
| Model | DL 30D |
|---|---|
| all-MiniLM-L6-v2sentence-transformers | 255.8M |
| bert-base-uncasedgoogle-bert | 95.4M |
| ms-marco-MiniLM-L6-v2cross-encoder | 87.6M |
| bge-small-en-v1.5BAAI | 71.6M |
| electra-base-discriminatorgoogle | 54.7M |
| paraphrase-multilingual-MiniLM-L12-v2sentence-transformers | 54.5M |