Population: Infinite Week 5 (Final)

For the final assignment, I wanted to train a style transfer model on my previous coding experiments. I utilized Spell and ml5.js in order to complete this and followed Yining’s guide in order complete this. With this, Spell was used in order to train the fast transfer style model and ml5 was used to run the model. It took some time for me to set up the environment and gather the reference images that I wanted to use. I later realized that with the fast transfer style model utilized only one image in order to transfer the style, so I did not need to gather so many reference images. I ran into errors with committing the images to Github and also with naming of files and variables and such. But eventually I was able to run the style transfer model on three images (mainly because that is all that I had time for). While running the first model, I utilized Spell’s K80 GPU machine and let it run for around 8 hours. After running it for 8 hours, the model only completed it one epoch, but I was unable to let it run for longer since I had to go to class and bring my laptop causing the connection with Spell to break and the model stopped running. Then I ran the other two models using Spell’s V100 GPU machine. I let the second model run for around 3 hours, and it didn’t complete even one epoch. I let it run for 3 hours because I wanted to see how well it would be able to transfer the style with such little computing time. I was also trying to maximize the amount of credits I used before I had to start paying for it out of my own pocket. I let the third model run for around 4 hours and it finished running within that time.

1st model

1st model

2nd model

2nd model

3rd model

3rd model

Then I used ml5.js to convert and run each of the models. Based of off Yining’s already existing template, I just utilized webcam input in order to see the style transfer models being I also like fact that utilizing an input with motion causes an output with motion and most of my sketches have motion as well.

The image above is the sketch that I used to get the image that was used to train the first model. The video below is the result of the style transfer model being implemented with a webcam input and the image next to the video is the image used to train the model.

ezgif.com-video-to-gif-6.gif
ezgif.com-video-to-gif.gif

The image above is the sketch that I used to get the image that was used to train the second model. The video below is the result of the style transfer model being implemented with a webcam input and the image next to the video is the image used to train the model.

ezgif.com-video-to-gif-5.gif
ezgif.com-video-to-gif-3.gif

The image above is the sketch that I used to get the image that was used to train the third model. The video below is the result of the style transfer model being implemented with a webcam input and the image next to the video is the image used to train the model.

ezgif.com-video-to-gif-4.gif

Overall, I learned that over time my now personal style changes and although I think I have a particular style, it changes over time and goes back to certain elements. I also learned that running a model takes a lot of time and computing power. I wish I had just waited a little longer for the first two models to run, cause I stopped them when they were not too far from finishing (this I didn’t realize until after I had run the last model). Also using a half trained model, gave me results that I kind of liked although they did not accurately combine the image used to train and the webcam input together. Surprisingly the image that I liked the least, ended up producing the best result in my opinion. Unfortunately I wasn’t able to train a model on the entirety of my images, so any model I ran does not accurately depict the style that I have. I think this would be really interesting cause over time the things that I code vary, so I’d like to see how my entire style would be depicted. I also enjoyed having a chance to look at all the things that I’ve coded since the end of December because I don’t normally look at my previously done work and I tend to forget about it rather quickly, so this was a nice opportunity to reflect on all that I’ve done. It also gave me the opportunity to note things that I liked and would like to go back and explore more and see how I can improve going forward.