Ollama model for documentation as ai assistant

Would be fine if somebody could train an llm on the blender documentation. You would than have a perfect assistant for training and learning Blender much faster! to run locally on ollama would be perfect....

Perhaps a smaller model would help people with smaller gpus.
Size 3b to 8b....

Recommended Base Models:

Llama 3.2 3B - Excellent choice for 8GB VRAM, good instruction following
Phi-3.5 Mini (3.8B) - Microsoft's efficient model, great for technical content
Qwen2.5 3B - Strong reasoning abilities, good for documentation-based tasks
Gemma 2 2B - Very memory efficient, decent performance

Training Approach:

Fine-tuning over full training - Much more feasible with your hardware constraints
Use QLoRA (Quantized LoRA) - Reduces memory usage significantly while maintaining quality
4-bit quantization - Essential for fitting larger context windows

Technical Setup:

Framework: Use unsloth or axolotl for efficient fine-tuning
Context length: 4096-8192 tokens (balance between performance and memory)
Batch size: Start with 1-2, use gradient accumulation
Learning rate: 2e-4 to 5e-4 for LoRA fine-tuning

Data Preparation:

Convert Blender docs to instruction-response pairs
Include code examples, UI explanations, and workflow guides
Aim for 10k-50k high-quality examples
Format consistently (ChatML or Alpaca format)

Memory Optimization:

Enable gradient checkpointing
Use mixed precision (fp16/bf16)
Consider DeepSpeed ZeRO if using multiple techniques

The Llama 3.2 3B with QLoRA fine-tuning would be my top recommendation - it strikes the best balance of capability and efficiency for your use case while being well-supported in the Ollama ecosystem.