---
tags: [prism, animation, research, motion, sam]
---
[← Dev Index](../../../INDEX.md)

# PRISM Motion Upgrade — Research Report (2026-08-18)

Commissioned by Joshua ("lights good, other effects terrible — research
better techniques; maybe nest SAM calls intelligently"). Dynamic fable
flight; filed by raven-alpha. Constraints verified on this VM: CPU-only,
numpy 2.5.1 + PIL 12.1.1, NO cv2/scipy/torch installed, SAM vit_b local,
per-frame GIF render.

## Verdict on the nested-SAM hypothesis: SUPPORTED, with literature

- SAM natively returns whole/part/subpart masks per point prompt
  (`multimask_output=True`, Kirillov et al. 2023) — level one of
  "SAM within SAM" is free against ONE cached image embedding.
- Meta's **Animated Drawings** (SIGGRAPH 2023, MIT license) is the
  canonical pipeline: segment → joints → triangulated mesh → **ARAP
  deformation**. Its ARAP solver is pure Python — CPU-proven.
- ChildlikeSHAPES (arXiv 2504.08022, 2025): hierarchical figure →
  head/torso/limbs parsing for animation — direct academic validation.
- Cost note: SAM's expense is the encoder (seconds, once per scene);
  mask decoding per prompt is milliseconds. Nesting is preprocessing,
  not per-frame.

## Why translate/bob reads as sticker-sliding

Uniform translation gives every pixel the same vector; real bodies have
a motion GRADIENT (zero at the anchor — feet, hips, trunk — rising to
the free end) plus a stationary contact edge. Fixes, ascending cost:

1. **Anchored amplitude ramp** — per-pixel displacement weighted by
   distance-from-anchor; pure numpy; the 80% fix, nearly free.
2. **MLS-rigid deformation** (Schaefer et al., SIGGRAPH 2006; numpy impl
   github.com/Jarvis73/Moving-Least-Squares): Puppet-Warp-style control
   pins, rigid variant avoids rubber. Weights precompute once per
   scene → ~50-200ms/frame at 640px.
3. ARAP mesh (best, +1-2 days for marginal gain at these amplitudes —
   skip initially). TPS (needs skimage install, rubberier — skip).

## Texture motion (water/smoke/grass) — the CPU path exists

- Reference: Holynski et al., Eulerian Motion Fields (CVPR 2021).
- CPU approximation = **flow maps** (Vlachos, Portal 2, SIGGRAPH 2010):
  advect texture along a flow field, two copies half a period out of
  phase, crossfade the resets. Two numpy warps + a lerp per frame.
- **Phase shimmer** (Freeman et al. 1991, "Motion Without Movement"):
  Gabor-pair phase spin = perceived motion with ZERO displacement — no
  reveal gap, no inpainting. Right for flames, sparkle, heat haze.
- Endless Loops (SIGGRAPH 2021) ran classical displacement cinemagraphs
  in <1s on mobile — this class is CPU-cheap.

## The inpainting gap

Dilate each moving mask by max displacement, inpaint that band ONCE per
scene, composite = plate → warped part (feathered 2-4px).
- No-install: numpy dilate-and-average fill, or pyheal (pure-Python
  Telea; slow but once-per-scene).
- With approval: `opencv-python-headless` → cv2.inpaint(TELEA) in ms,
  plus remap/blur speedups everywhere. THE ONE INSTALL ASK.

## Amplitude law (why "rubber person" happened)

Cinemagraph practice: small, slow, repetitive, isolated. Defaults:
tip displacement 1-3% of part length, zero at anchor, sinusoidal ease,
period 2-6s, per-part phase offsets. Rubber = uniform translation +
amplitude big enough to expose the shear boundary; the ramp kills both.

## Top 3 by looks-better-per-build-day

1. **Nested SAM + MLS-rigid anchored part warp** — replaces
   translate/bob/skew entirely (~1-2 days, no installs).
2. **Pre-computed inpainted plate** behind every moving mask
   (~0.5-1 day; cv2 nice-to-have, numpy fallback fine).
3. **Flow-map crossfade for textures + phase shimmer for glow**
   (~1 day, no installs; perfect GIF loops by construction).

Full source list in session transcript (Segment Anything 2023, Animated
Drawings 2023, ChildlikeSHAPES 2025, Schaefer MLS 2006, Holynski 2021,
Vlachos 2010, Freeman 1991, Wadhwa 2013, Endless Loops 2021, OpenCV
inpainting docs, pyheal, PyPatchMatch, cinemagraph practice guides).