IBM Granite 13b foundation model version 1 model card

Granite 13 Billion Model (granite.13b) Details

IBM Generative AI Large Language Foundation Models are Enterprise-level English-language models trained on enterprise-relevant datasets from five domains – internet, academic, code, legal and finance – all of which have been curated for business use by IBM. All data sets were scrutinized to exclude objectionable content and benchmarked against internal and external models to enable responsible deployment and address key issues including governance, risk assessment, privacy concerns and bias mitigation. The Granite Base 13 Billion (granite.13b.base) V1.0 model has been trained using over 1T tokens. This is the base model, from which other variants were fine-tuned to target downstream tasks. The Granite family of models will support all 5 language tasks (Q&A, Generate, Extract, Summarize, and Classify).

Besides the base variant (granite.13b.base), there are two other models that have been fine-tuned. The first variant (granite.13b.instruct) is an instruction-tuned Supervised Fine-Tuning (SFT) model, which was further tuned using a combination of the Flan Collection, 15k samples from Dolly, Anthropic's human preference data about helpfulness and harmlessness, Instructv3, and internal synthetic datasets specifically designed for summarization and dialog tasks (~700K samples). The second variant (granite.13b.chat) is a Contrastive fine-tuning (CFT) variant, which uses a new objective called unlikelihood training, which penalizes unlikely generations by assigning a lower probability value.

The table below lists current official variants released by IBM Research. The Massive Multitask Language Understanding (MMLU) benchmark is used to show the performance of each variant.

• Person or organization developing the model:
  • Granite (13B) was developed by IBM Research.
• Model release date and version:
  • Granite (13B) version 1.0 was released on 09/21/2023.
• Model type:
  • Granite (13B) V1.0 is a large decoder-only transformer model.
  • The following features were used in the design of the model:
    1. Decoder-only model
    2. Multi-Query Attention
    3. 50K GPT-NeoX tokenizer
    4. Flash Attention
    5. 8k context length
    6. Absolute (learnt) position embeddings
• Information about training algorithms, parameters, fairness constraints or other applied approaches, and features:
  • Model was trained using 4x Tensor Parallel + 4x Pipeline Parallel + 32x ZeRO-1 + Sequence Parallel + Flash Attention using a fork of Megatron-LM.
  • Cluster: CCC
  • GPUs: 256x A100 80GB
  • Interconnect: 200 gigabit Infiniband
  • Dataset streamed over GPFS
• Paper or other resource for more information:
  • https://www.ibm.com/downloads/cas/X9W4O6BM
• License:
  • Available only through IBM products and offerings. Contact IBM for licensing terms.

Intended Use

• Primary intended uses:

  • .chat / .instruct : The Granite series of models are a family of IBM-trained decoder-only models used for text generation, summarization, question and answer, classification, and extraction.
  • base : The base model will be primarily used to fine-tune downstream language tasks.

• Primary intended users:

  • The primary users are IBM Enterprise clients looking to bolster their portfolios with Enterprise-level generative AI models.

• Out-of-scope use cases:

  • The granite.13b models are not designed, tested, or supported, for code use cases of any kind.

Factors

• Relevant factors: Models work with proper English text. All datasets have been cleansed of any type of tagging (e.g., HTML), and all media has been removed as well.
• Evaluation factors: Evaluation datasets have to be proper English, and are limited to text only.

Metrics

IBM has built a comprehensive test framework FM eval that is used throughout the model's life-cycle. This can be used both to evaluate IBM's own models and those already trained by third-parties allowing models to be measured against a variety of benchmarks. The evaluation framework runs on an Openshift cluster with GPU support and uses various AI model evaluation frameworks: Eleuther AI's Language Model Evaluation Harness lm-eval, Stanford's Holistic Evaluation of Language Models (HELM), Beyond the Imitation Game Benchmark (BIG-bench), as well as IBM-internal datasets.

Data, Limitations, and Recommendations

• Data selection for training:
  • The Granite Base (13B) V1.0 model was trained using IBM's curated pre-training dataset. A breakdown of the sampling data used for training is shown in the table below.