Bug Fix Makes Free AI Chatbot Answers More Accurate
This quiet fix stops AI from giving wrong answers after long conversations.
A fix in ggml-cpu addresses a bug where GGML_OP_SOFT_MAX_BACK could produce silently wrong output when dst aliases src1. Because the op is listed in ggml_op_can_inplace, the graph allocator may assign dst to alias either src0 (dy) or src1 (y).
When dst aliases src1, the first step overwrites y and the third step then reads the overwritten values, so the output is silently wrong. Aliasing dst with src0 is unaffected, and the CUDA kernel completes its reduction before writing and is already safe.
The fix replaces the sequence with a single fused loop that reads both sources before writing, which is correct under either aliasing. A regression test marks dy as a graph output so the allocator is forced to alias dst with y, asserts that the alias actually happened, and compares against values computed on the host.
- A bug in free AI software caused wrong answers on regular computers, not on high-end graphics cards.
- Developers fixed it by changing how the AI handles memory, making it more reliable.
- This means free AI tools like chatbots and coding helpers will be more accurate for everyday users.
Why It Matters
More accurate free AI tools mean fewer frustrating errors in your daily chats and tasks.