Research & Papers

Bogdan & de Valois-Franklin's Machine Psychometrics measures AI with 8-dimension Mindprint

A 45-page paper proposes evaluating AI's psychological structure, not just benchmark scores.

Deep Dive

A new paper from researchers Alex Bogdan and Adrian de Valois-Franklin proposes Machine Psychometrics as a disciplined alternative to current AI evaluation methods. The authors argue that current capability benchmarks only scratch the surface, ignoring the psychological structure behind an agent's behavior. They identify two symmetrical errors: Artificial Mind Blindness (denying any psychological organization in non-biological systems) and Artificial Mind Projection (assuming human-like inner life from fluent output). To circumvent this philosophical deadlock, they introduce a measurement layer that doesn't require resolving the consciousness question. Drawing on Michael Levin's continuum view of cognition and tools from mathematical psychology—Item Response Theory, Signal Detection Theory, Bayesian cognitive modeling, calibration analysis, and cognitive-bias batteries—the paper builds a methodology to assess latent behavioral, metacognitive, communicative, and self-modeling dispositions.

The operational core of Machine Psychometrics is the Machine Mindprint, a multidimensional, domain-bounded, versioned profile spanning eight specific traits: calibration, source integrity, suggestibility resistance, context stability, expressive alignment, tool integrity, drift monitoring, and distributional grounding. A companion Trust Protocol turns these Mindprints into actionable deployment decisions through probe batteries, perturbation testing, reliability/validity analysis, and longitudinal monitoring across high-stakes domains. The philosophical stance, Artificial Mind Discipline, neither anthropomorphizes nor dismisses AI—it seeks to understand artificial agents precisely because they are not human, prioritizing measurement before judgment. The paper is 45 pages with 11 figures and positioned at the intersection of AI, computational linguistics, and neuroscience.

Key Points
  • Introduces Machine Psychometrics as a new measurement science for AI's latent psychological dispositions, using tools like Item Response Theory and Signal Detection Theory.
  • Defines the Machine Mindprint—an 8-dimensional profile covering calibration, source integrity, suggestibility resistance, context stability, expressive alignment, tool integrity, drift monitoring, and distributional grounding.
  • Proposes a Trust Protocol for deployment decisions that includes probe batteries, perturbation testing, and longitudinal reliability/validity analysis.

Why It Matters

Offers a systematic, non-anthropomorphic framework to audit AI trustworthiness for high-stakes deployments beyond simple accuracy scores.