First time fine-tuning, need a sanity check — 3B or 7B for multi-task reasoning? [D]

A self-taught developer, after a year of working with LLMs via APIs, is venturing into fine-tuning for the first time. They aim to train a model on three related reasoning tasks: reading underlying subtext in questions (e.g., 'should I quit my job' often masks identity or fear), holding multiple perspectives without premature collapse, and identifying the load-bearing thread in messy, multi-problem inputs. The core debate is whether a 3B parameter model (Phi-4-mini) or a 7B model (Qwen 2.5) is sufficient for this multi-task reasoning, given 40-60k training examples generated from philosophy, psychology case studies, and strategy literature. The developer runs an M4 Mac with 24GB unified memory, where 3B fits comfortably with LoRA, and 7B is tight but doable.

The developer's primary concerns include whether a 3B model can hold three related reasoning modes without confusing them on out-of-distribution data, whether the tasks' similarity makes training harder than if they were entirely separate, and what unknown pitfalls might arise. They seek insights from anyone who has attempted multi-task training on reasoning data at this scale, asking for specific experiences and papers rather than generic advice. This scenario highlights the practical challenges of fine-tuning for nuanced cognitive tasks with limited hardware, a common bottleneck for self-taught practitioners pushing beyond prompt engineering.