About

The goal of Doodler (alt title: KeithRNN) was to train a machine learning javascript program on an artist's work so that a user could collaborate with that artist in real time. With Google Brain resident David Ha work on recurrent neural networks I saw an opportunity to realize a version of this with 2d drawings. 

 

Originally, I started exploring WriteRNN, a machine learning model that predicted cursive handwriting in javascript. Through that project I learned about Davis Ha's SketchRNN - a machine learning model developed to produce fake Kanji characters. This was a perfect jumping off point to work with 2d artist drawings. I decided to create a dataset of Keith Haring drawings for my training data since (aside from him being all around incredible) the style of the artwork (as simple, single line images)  translated well to the ML model. 

 

After compiling a large enough collection of Keith Haring drawings, I converted them to SVG files so the line paths could be interpreted by TensorFlow. After the initial learning curves (this was my first deeper dive in TF), I was able to train the model on sample drawing datasets (boomerangs, kangaroos, etc) and finally on Keith Haring drawings and generate fake images in python. At the time of this project, David Ha was completing a system to export those weights to JS, to generate the images in realtime. I'm still continuing with this idea and am now working on creating a JS verison of this that works in 3D space. 


Screen-Shot-2017-05-02-at-6.53.59-PM.png
Screen-Shot-2017-04-30-at-1.14.48-PM.png
boomerang.jpg
Screen-Shot-2017-04-25-at-2.08.10-PM-768x244.png
31973E4A1FC20EB9.jpg
Screen-Shot-2017-04-30-at-4.36.20-PM.png
Screen Shot 2017-04-30 at 4.53.43 PM.png

RNN sequence length: 300, number of epochs: 300, min_num_stroke: 2, max_num_stroke: 8

RNN sequence length: 500, number of epochs: 500, min_num_stroke: 4, max_num_stroke: 16