Tools
An overview of some new creative tools that use ML.
- Teachable Machine
- RunwayML
- EbSynth
- Photoshop Neural Filters
Teachable Machine
Google’s Teachable Machine tool allows you to train a (neural-network-based) image recognition system, in the browser, without code. Note that its models can also be used with p5.js.
I recently learned that as a chronic face toucher, I might be more likely to get Coronavirus.?So I made the ANTI-FACE-TOUCHING MACHINE™ to train myself to touch my face less. (sound on) Made with https://t.co/KGM4yT8eXP #CoronaOutbreak pic.twitter.com/yS8l0Zxqrl
— Isaac Blankensmith (@Blankensmith) March 2, 2020
Live demo! I show a version I got working with p5.js here. It was easy!
Some of the people contributing to the project include recent CMU architecture alumna Irene Alvarado, and undergrad Design alums, Lucas Ochoa and Gautam Bose. Gautam and and Lucas have made an absurd cereal sorter:
Here’s another project breakdown by Gautham and Lucas, on making a system that tells you when your pancakes are ready to flip:
Shiffman has made a Coding Train tutorial:
Need more? Here’s a tutorial for a Head Tilting Detector.
RunwayML
RunwayML is a startup that packages recent ML research into automatable workflows that artists can use with no coding. It has dozens of offerings, including the following:
RunwayML’s MaskRCNN – Detects objects in an image while simultaneously generating a high-quality segmentation mask:
RunwayML’s AttnGAN – Turn text descriptions of scenes into synthesized images:
RunwayML’s SPADE-landscapes – Generate realistic images of landscapes from sketches and doodles:
RunwayML’s BigBiGAN – Generate similar images based on the input image:
RunwayML’s Background Matting – Real-Time High-Resolution Background Matting:
RunwayML’s HiFill – High resolution image inpainting with HiFill:
Lots of Runway help information here.
EbSynth
Easy style transfer for video. “Teach” the system how it should look by painting over one frame, and it carries the texture forward over time.