Rho-1: Not All Tokens Are What You Need

Figure 1: Rho-1 is pre-trained with Selective Language Modeling (SLM). SLM improves average few-shot accuracy on GSM8k and MATH by over 16%, achieving the baseline performance 5-10x faster.

🔥 News

• [2024/04/12] 🔥🔥🔥 Rho-Math-v0.1 models released at 🤗 HuggingFace!
  • Rho-Math-1B and Rho-Math-7B achieve 15.6% and 31.0% few-shot accuracy on MATH dataset, respectively — matching DeepSeekMath with only 3% of the pretraining tokens.
  • Rho-Math-1B-Interpreter is the first 1B LLM that achieves over 40% accuracy on MATH.
  • Rho-Math-7B-Interpreter achieves 52% on MATH dataset, using only 69k samples for fine-tuning.
• [2024/04/11] Rho-1 paper and repo released.

💡 Introduction

Selective Lanugage Modeling (SLM)

Figure 2: Upper: Even an extensively filtered pretraining corpus contains token-level noise. Left: Previous Causal Language Modeling (CLM) trains on all tokens. Right: Our proposed Selective Language Modeling (SLM) selectively applies loss on those useful and clean tokens.

Figure 3: The pipeline of Selective Language Modeling. SLM optimizes language model performance by concentrating on valuable, clean tokens during pre-training. It involves three steps: (Step 1) Initially, train a reference model on high-quality data. (Step 2) Then, score each token's loss in a corpus using the reference model. (Step 3) Finally, train the language model selectively on tokens that show higher excess loss compared to the reference loss.

Evaluation Results

🚀 Quick Start

Evaluation

git clone git@github.com:microsoft/rho.git
cd rho-1/math-evaluation-harness

bash scripts/run_eval.sh cot microsoft/rho-math-7b-v0.1

bash scripts/run_eval.sh tora microsoft/rho-math-7b-interpreter-v0.1

☕️ Citation

@misc{lin2024rho1,
      title={Rho-1: Not All Tokens Are What You Need}, 
      author={Zhenghao Lin and Zhibin Gou and Yeyun Gong and Xiao Liu and Yelong Shen and Ruochen Xu and Chen Lin and Yujiu Yang and Jian Jiao and Nan Duan and Weizhu Chen},
      year={2024},
      eprint={2404.07965},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

• Example Notebook
• Original Model Card
• Text Generation Inference Documentation

Send Request

curl <AZUREML_ENDPOINT_URL> \
    -X POST \
    -d '{"inputs":"What is Deep Learning?"}' \
    -H "Authorization: Bearer <AZUREML_TOKEN>" \
    -H "Content-Type: application/json"

Supported Parameters

• inputs (string): Input prompt.
• parameters (object):
  • best_of (integer): Generate best_of sequences and return the one if the highest token logprobs.
  • decoder_input_details (boolean): Whether to return decoder input token logprobs and ids.
  • details (boolean): Whether to return generation details.
  • do_sample (boolean): Activate logits sampling.
  • frequency_penalty (float): The parameter for frequency penalty. 1.0 means no penalty Penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim.
  • grammar (object): One of the following
    • #1 (object):
      • type (enum): Possible values: json.
      • value (string): A string that represents a JSON Schema. JSON Schema is a declarative language that allows to annotate JSON documents with types and descriptions.
    • #2 (object):
      • type (enum): Possible values: regex.
      • value (string): The regular expression.
    • #3 (object):
      • type (enum): Possible values: json_schema.
      • value (object):
        • name (string): Optional name identifier for the schema
        • schema (object): The actual JSON schema definition
  • max_new_tokens (integer): Maximum number of tokens to generate.
  • repetition_penalty (float): The parameter for repetition penalty. 1.0 means no penalty. See this paper for more details.
  • return_full_text (boolean): Whether to prepend the prompt to the generated text
  • seed (integer): Random sampling seed.
  • stop (string[]): Stop generating tokens if a member of stop is generated.
  • temperature (float): The value used to module the logits distribution.
  • top_k (integer): The number of highest probability vocabulary tokens to keep for top-k-filtering.
  • top_n_tokens (integer): The number of highest probability vocabulary tokens to keep for top-n-filtering.
  • top_p (float): Top-p value for nucleus sampling.
  • truncate (integer): Truncate inputs tokens to the given size.
  • typical_p (float): Typical Decoding mass See Typical Decoding for Natural Language Generation for more information.
  • watermark (boolean): Watermarking with A Watermark for Large Language Models.
  • stream (boolean): Whether to stream the output tokens or not. Defaults to false.

Example payload

{
  "inputs": "What is Deep Learning?",
  "parameters": {
    "do_sample": true,
    "top_p": 0.95,
    "temperature": 0.2,
    "top_k": 50,
    "max_new_tokens": 256,
    "repetition_penalty": 1.03,
    "stop": ["\nUser:", "<|endoftext|>", "</s>"]
  }
}

OpenAI Chat Completion API compatibility