Granular video synthesis
Pixi is an iOS app in development, a collab between Coldcut and Paul Miller who coded it. Paul also created the awesome and free TXmodular synth for Mac & PC.
Pixi could be termed a ‘granular video synthesiser’. It was first prototyped from round 2011 in Quartz Composer as Granul8…making it a qtz meant it could be loaded into fab modular VJ software VDMX and used for VJing. We then realised it would make a great touch controlled app.
In audio, granular synthesis involves using a small part of a Audio sample, the ‘grain’ as the basis for synthesis. The part of the sample used for the grain can be shuttled up and down the waveform and can be of varying size. Pixi is termed Granular Video Synthesis by analogy: in Pixi , video synthesis is based on a small part of an IMAGE, typically a photo, and this source sample can be taken from different parts of the image, and change size and aspect ratio. The rectangular sample is combined with a mask image (e.g. circle, star) to yield a Brush. The Brush is then used to draw patterns. If the sample is the size of the entire source picture, then Pixi can function as a basic picture processing/fx app.
Here are screenshots of the control panel of the Pixi prototype :-
But there are many ways in which Pixi can produce surprising and interesting visual results from manipulations of the Brush. There is also the ability to use Particle systems as multiple Brushes, an enhancement by Paul to the program which was a bit like going from a mono to a polyphonic synthesiser. Pixi has a sophisticated Patch Matrix like a modular synth in which Control params e.g. Acceloromoter, LFO, touch control, Perlin noise are assigned to Synth params e.g. brush size, position, alpha, feedback. This enable an infinite number of patches to be constructed, and favourite ones saved.
Work is continuing on Pixi with Amaury Hazen from BCN now in pole coding position. We envisage 2 versions, a Master Synth version for iPad for designing presets and a Player version for iPhone which just loads presets and applies them to user images. As the core is cross platform Open Frameworks, it should be possible to do e.g. Android versions also. This will have to wait till we see how the iOS version does.
We would like to work with visual artists to take their work as images into Pixi and then design presets based on these images, as a ‘Visual remixing’ process. These could be inapp purchases and so visual artists could earn from Pixi too.
Here are some pictures made using Pixi. More examples can be seen in the Gallery.
“We would like to work with visual artists to take their work as images into Pixi and then design presets based on these images, as a ‘Visual remixing’ process. These could be inapp purchases and so visual artists could earn from Pixi too..”