An Open Problem

Automated judgment of generated visuals almost always works on a single still frame. A model produces an image or a piece of an interface, another model or a scoring function looks at one captured frame, and it returns a verdict: this looks right, this looks wrong, this matches the request. For a still image that is the whole object, and the approach is sound. For anything that moves, it quietly measures the wrong thing, and the gap it leaves is real and currently unsolved.

The problem is that a still frame carries almost no information about motion. Motion is defined by how a thing changes across time. A single frame is one instant with the time dimension removed, so the very thing that distinguishes one animation from another is the thing a frame cannot contain. Judging motion from a frame is like judging a piece of music from one sampled moment of sound. The instant might be pleasant or unpleasant, but it says nothing about tempo, rhythm, or whether the notes arrive in an order that makes sense.

The failure is sharpest with animations that differ only in timing. Take two versions of the same set of elements coming onto a screen. In the first they arrive in a staggered sequence, one after another with a slight offset, and the effect reads as deliberate and correct. In the second they all arrive at once, or in an order that fights the layout, and the effect reads as broken. These two animations pass through nearly identical frames. At the start every element is absent in both. At the end every element is settled in its final place in both, so the last frame is the same. In the middle the frames differ only by small offsets in position and opacity that any single snapshot renders as a minor, ambiguous difference. A frame-based score, sampling at any one instant, sees two nearly identical pictures and returns nearly identical verdicts. It cannot tell the correct animation from the broken one, because the information that separates them was never in any of the frames it looked at.

The same holds for the other timing properties. Easing, the way a movement accelerates and decelerates, is entirely a property of the path through time; two movements with the same start and end and different easing occupy the same positions at the endpoints and differ only in how they travel between them. Speed is the same. Sample any of these at a single instant and the differences collapse into small positional offsets that carry no clear signal about whether the motion is right. The endpoints are identical by construction, and the middle is where all the meaning lives and where a snapshot is least informative.

The honest conclusion is that judging motion needs a measurement taken over time, not a snapshot. The score has to see the trajectory: how each element moves across the whole duration, when it starts relative to the others, how it accelerates, when it comes to rest. That means capturing the motion as a sequence and reasoning about the sequence as a whole, which is a fundamentally heavier thing to do than reading one frame. A frame is cheap to capture and cheap to score. A trajectory has to be recorded across the full length of the animation, aligned in time, and evaluated as a shape in time rather than a picture, and there is no established, inexpensive instrument that does this well.

This is what makes it a genuinely open problem rather than a solved one waiting to be applied. For still images the automated critics are capable and improving quickly. For motion the field is still mostly extending the still-image approach onto a medium it does not fit, scoring frames and hoping the result generalizes to movement, which it does not for exactly the reasons above. A reliable, affordable way to measure whether an animation reads as correct, one that captures timing and responds to stagger and easing and speed the way a person does, does not yet exist in any standard form. It is a real gap. Naming it precisely, that the instrument has to measure over time and no cheap one does, is most of what can be said honestly about it today.