ai/granite-4.0-nano repository overview

⁠Granite 4.0 Nano

⁠Description

Granite-4.0-350M is a lightweight instruct model finetuned from Granite-4.0-350M-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques including supervised finetuning, reinforcement learning, and model merging.

⁠Characteristics

⁠Intended use

Intended use: Granite 4.0 Nano instruct models feature strong instruction following capabilities bringing advanced AI capabilities within reach for on-device deployments and research use cases. Additionally, their compact size makes them well-suited for fine-tuning on specialized domains without requiring massive compute resources.

⁠Available model variants

¹: VRAM estimated based on model characteristics.

⁠Use this AI model with Docker Model Runner

docker model run ai/granite-4.0-nano

⁠Considerations

• Optimized for instruction following, tool/function calling, and long-context (up to 128K tokens) scenarios.
• Strong generalist capabilities: summarization, classification, extraction, QA/RAG, coding, function-calling, and multilingual dialogue.
• Multilingual: best performance in English; a few-shot approach or light finetuning can help close gaps for other languages.
• Safety & reliability: despite alignment, the model can still produce inaccurate or biased outputs—apply domain-specific evaluation and guardrails.
• Infrastructure note: trained on NVIDIA GB200 NVL72 at CoreWeave; use acceleration libraries (e.g., accelerate, optimized attention/KV cache settings) for efficient inference.

⁠Benchmark performance

⁠Links

• https://www.ibm.com/granite⁠
• https://www.ibm.com/granite/docs/⁠
• https://ibm.biz/granite-learning-resources⁠