Posture suffices to classify activities, but motion needs temporal dynamics

A new preprint from Farahmandi and Blohm explores a fundamental dissociation in human movement analysis: classifying an activity requires only static body posture, but faithfully reconstructing that movement demands full temporal dynamics. Using videos of 16 everyday activities from the MoVi dataset, the team compared three representation methods—Temporal Movement Primitives (TMPs), Legendre polynomial coefficients, and autoencoder latent embeddings. Legendre coefficients and TMPs achieved the highest classification accuracy, highlighting that the average spatial configuration (the overall posture) is the most discriminative feature. They also identified nine critical joints that are most informative for telling activities apart.

Strikingly, good classification did not translate to good motion generation. While TMPs preserved temporal dynamics and produced perceptually natural reconstructions, Legendre polynomials—despite excellent classification—yielded frozen, unnatural motions that captured only the average pose. This reveals that the visual system may rely on static posture for rapid action recognition, but generating realistic movements requires understanding how joint angles change over time. The findings suggest efficient clinical screening could be done using postural features alone, while any application involving movement generation (e.g., animation, rehabilitation) must incorporate complete motion dynamics.