README.md

metadata

license: apache-2.0
tags:
  - reward-model
  - rlhf
  - sparse-autoencoder
  - interpretability

SARM: Interpretable Reward Model via Sparse Autoencoder

• Authors (* indicates equal contribution)

  Shuyi Zhang*, Wei Shi*, Sihang Li*, Jiayi Liao, Tao Liang, Hengxing Cai, Xiang Wang

• Paper: Interpretable Reward Model via Sparse Autoencoder

• Model: schrieffer/SARM-4B

  • Finetuned from model: Llama-3.1-8B-Instruct

• Code Repository: https://github.com/schrieffer-z/sarm

• Demo: Try SARM Demo in Huggingface Space