Datamoshing in COPs

There is something about seeing a video fail, revealing the system underneath. Since Datamoshing originates from encoding errors, which we don’t have access to in COPs, this implementation is trying to replicate the effect as closely as possible. Optical Flow or Block Matching is used to extract Motion Vectors from any input video. These Motion Vectors are then used to advect the frozen frame’s pixels based on difference masks, causing the characteristic distortion.

The strength of this approach lies in its flexibility. We can use rendered mtion vectors to distort any video, or just noise, customs masks allow for only certain regions to be moshed, or based on depth. At last free from the whimiscal nature of the codec!

Grab it here on gumroad as part of the Pixel Pandemonium Pack

What even is datamoshing?

Traditional datamoshing relies on video encoding artifacts. Video streams are split into I-frames (Intra-coded), which display full images without any outside data, and P-frames (Predicted) and B-frames (Bi-predictive). Instead of storing full frames, P- and B-frames only contain data describing changes from the preceding frame, like Motion Vectors for pixel movement or color transforms. The classic datamoshing effect happens when you remove the I-frames, forcing the P- and B-frame motion data to transform the wrong pixels.

Since we don’t have access to encoding algorithms in COPs this tries to emulate what’s happening behind the scenes as closely as possible. I implemented block-matching (the actual algorithm used during compression to measure motion) to extract motion from the video. The video data is then advected to create the distortion effect, while custom masks based on frame differences and motion magnitude mix the new and old frames together. One of my main goals was maximum user control since the og effects is not that controlable. This supports custom masks, external Motion Vectors and switching to a custom optical flow implementation for smoother motion.