QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning
A 4B-parameter model fine-tuned with QLoRA beats commercial giants at specialized legal reasoning.
A research team from QU-NLP, led by Mohammad AL-Smadi, has published a novel method for fine-tuning large language models to perform complex Arabic Islamic inheritance reasoning. Their approach uses a two-stage Quantized Low-Rank Adaptation (QLoRA) process on the compact Qwen3-4B model. First, the model undergoes domain adaptation using 3,166 Islamic legal consultation (fatwa) records to learn specialized terminology and jurisprudential reasoning patterns. This is followed by task-specific training on a dataset of 12,000 structured inheritance cases, teaching the model to generate precise, JSON-formatted outputs that detail heirs, blocking rules, and fractional shares.
The technical implementation is highly efficient, utilizing 4-bit NormalFloat (NF4) quantization and rank-128 LoRA adapters, which drastically reduces computational requirements compared to full model fine-tuning. Despite its small size, the resulting model achieved a 90% score on the Mawarith Inheritance Reasoning Evaluation (MIR-E) benchmark, a specialized test for this legal domain. The paper notes this performance is competitive with larger commercial systems like Gemini-2.5-flash, proving that targeted, resource-efficient fine-tuning can enable smaller open-source models to master niche, high-stakes reasoning tasks that involve multi-step analysis, rule application, and exact calculation.
- Uses a two-stage QLoRA fine-tuning strategy on the Qwen3-4B model, first for domain knowledge (3,166 fatwas) then for task execution (12,000 cases).
- Achieved a 90% MIR-E score using efficient 4-bit NF4 quantization, demonstrating high accuracy with minimal computational footprint.
- Shows small, open-source models can rival commercial giants like Gemini-2.5-flash in specialized, rule-based legal reasoning when properly adapted.
Why It Matters
This demonstrates a cost-effective blueprint for creating highly accurate, specialized AI assistants for complex legal, financial, and regulatory domains.