Code Llama is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. CodeLlama-70b model is designed for general code synthesis and understanding.

Ethical Considerations and Limitations

Code Llama and its variants are a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Code Llama’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate or objectionable responses to user prompts. Therefore, before deploying any applications of Code Llama, developers should perform safety testing and tuning tailored to their specific applications of the model.

Inference samples

Inference type	Python sample (Notebook)	CLI with YAML
Real time	text-generation-online-endpoint.ipynb	text-generation-online-endpoint.sh
Batch	text-generation-batch-endpoint.ipynb	coming soon

Finetuning samples

Task	Use case	Dataset	Python sample (Notebook)	CLI with YAML
Text Generation	Summarization	Samsum	summarization_with_text_gen.ipynb	text-generation.sh

Evaluation samples

Task	Use case	Dataset	Python sample (Notebook)	CLI with YAML
Text generation	Text generation	cnn_dailymail	evaluate-model-text-generation.ipynb	evaluate-model-text-generation.yml

Sample input and output

Sample input

{
    "input_data": {
        "input_string": [
            "def fibonacci("
        ],
        "parameters": {
            "top_p": 0.9,
            "temperature": 0.2,
            "do_sample": true,
            "max_new_tokens": 200
        }
    }
}

Sample output

[
  {
    "0": "def fibonacci(n):\n    if n == 0:\n        return 0\n    elif n == 1:\n        return 1\n    else:\n        return fibonacci(n-1) + fibonacci(n-2)\n\n\ndef main():\n    print(fibonacci(5))\n\n\nif __name__ == \"__main__\":\n    main()\n"
  }
]