Probabilistic concept modeling framework rethinks AI reasoning

Gretta Duleba, building on work by John Wentworth and David Lorell, introduces a probabilistic and Bayesian framework for modeling how concepts are represented inside minds—from simple biological agents to advanced AI systems. The framework occupies a middle ground between low-level, hardware-specific descriptions (neural circuits, biochemical processes) and high-level, philosophical abstractions. Rather than claiming agents literally perform purely Bayesian updates, the author argues that probabilistic models are a powerful lens: even an E. coli bacterium can be usefully modeled as having a goal and a world model (an 'intentional stance'), because evolution optimized it under selection pressure.

For more advanced agents—especially those with greater resources and more selection pressure—the Bayesian approximation becomes increasingly accurate. The 'embedded' nature of these models means the underlying implementation (e.g., neural network weights, brain activity) won't obviously look like a causal DAG with probability tables, but the behavior can be 'reverse-compiled' into an equivalent Bayesian representation. This perspective suggests that as AI systems become more capable, their internal concept representations will naturally align with probabilistic causal models, which has implications for interpretability and alignment. The article highlights that 'latent' variables in such models are not just mathematical conveniences but reflect compression of real-world structure.