I made a survelience ramen bot that takes a picture when it sees someone take instant ramen out of the pantry, and tweets the image on twitter. I thought it would be interesting to document me and my housemates' instant ramen eating habits, since our pantry is always stocked with it.
I worked backwards, starting with the twitterbot. I used Twit API, and node.js. (Most of the work was from setting up the twitter account and learning about command prompt.) Then I added the image capture to the Feature Extractor template. I struggled with connecting the two programs, since one runs on pj5s (client-based?) and the other on node (local computer?). I tried to call the twitterbot code in the feature extractor code(trying out different modules and programs), but I couldn't get it to work. I opted to make the twitterbot run continously once I call it in the command prompt; it calls a function every 5 seconds to check if there is a new image to post.
I made the twitter account header/profile look like a food blog/food channel account. I thought it would be a fun visual contrast between the tweets.
It would be better if I finished this earlier, so that there would be more realistic twitter documentation of me and my housemates. none of my housemates were avaliable/awake by the time I finished
find a better camera location, so it looks less performative
I should of done samples of holding food that wasn't instant ramen
this can only be run locally from my computer, maybe upload to heroku?
Even compared to all the other projects, I spent a very long time troubleshooting and down-scoping my idea for this project! My first idea was to train a model to recognize its physical form -- a model interpreting footage of the laptop the code was running on, or webcam footage reflecting the camera (the model's 'eyes') back at it. However, training for such specific situations with so much variability would have required thousands of training data.
Next, I waffled between several other ideas, especially using a two-dimensional regressor. I was feeling pretty bad about the whole project because none of my ideas expressed interesting ideas in a simple but conceptually sophisticated way. I endeavored to get the 2D regressor working (which was its own bag of fun monkeys,) and make the program track the point of my pen as I drew.
Luckily, Golan showed me an awesome USB microscope camera! The first thing I noticed when experimenting with this camera was how gross my skin was. There were tiny hairs and dust particles all over my fingers, and a hangnail which I tried to pull off, causing my finger to bleed. Though the bleeding healed within a few hours, it inspired a project about a deceptively cute vampiric bacterium who is a big fan of fingers.
This project makes use of two regressors (determining the x and y location of the fingertip) and a classifier (to determine whether a finger is present and if it is bloody.) I did not show the training process in my video because it takes a while. If I had more time, I think there is lots of potential for compelling interactions with the Bacterium. I wanted him to provoke some pity and disgust in the viewer, while also being very cute.
In conclusion, I spent many hours on this project and tried hard. I really like Machine Learning, so I wanted my piece to be 'better' and 'more'. But I learnt a lot and made an amusing thing so I don't feel unfulfilled by it.
Space Invader controlled by hand posture - collaborated with Sanjay
This is an exploration of training our own model for posture detection to apply it to game. We were inspired by the rock paper scissor detection and wanted to something that also detects gestures but in a different game scOur first step is to detect both the rotation of the hand and whether you pull the trigger. Using regression template we were able to create two axises that separately detect the two features. However, as we combined the detection with Space Invader, we faced a very low frame rate because of the expensive calculation. Currently this is a very crude exploration, and we could be more creative with the application of the shooting gestures. A further development of this game could be optimizing the calculation time by moving offline and try to save and preload the model. Following are some other game scenarios this gesture can be developed:
We were really inspired by Google's drawing machine learning, and the ability to play around with the different types of applications that machine learning has in with drawing. In order to most quickly and accurately iterate over and over again, we started our explorations by playing around with the whiteboard. We started off playing around with the program to see if machine learning was able to detect the difference between written text and drawings. From this, we were also thinking of maybe incorporating mathematical graphs and/or equations as a possible third scope; an example that lives between text and drawing.
From our experiments, we saw that computer could usually detect between drawings and text, presumptuously mostly dependent on the text. The diversity of drawings was differed widely, as we literally began to draw just about everything that first came to mind, whereas text was definitely more limited in terms of aesthetic, and was visually more uniform. However, we came upon an interesting discovery when drawing stars, but in a linear form. Despite being drawings, it was detected as text, because of its linear nature. This propelled us into thinking about the possible implications for using machine learning to detect the differences between languages.
The stars that sparked off our stars.
Our final exploration dealt with exploring the detecting the difference between western and eastern languages; western being more letter-based, and eastern being more pictorial-based characters.
Western Languages
Eastern Languages
Training our model with white background, western text, and eastern text.
We decided to map the result out visually through three colors:
White indicates that there is no hand written text being shown. (We fed a series of white background images to train this part)
Blue indicates that it is more-so western text. (We fed a series of handwritten phrases and words in English, Spanish, French, and Latin to train this part)
Red indicates that is more-so eastern text. (We fed a series of handwritten phrases and words in Chinese, Japanese, and Korean to train this part)
LanguageML_Test1
From our results, we've discovered a couple things.
The program is relatively good at detecting a "blank background" though a couple times, when our paper was shifted, the program recognized it as "western".
But most importantly, the program was very accurate in detecting western text, but significantly less so with eastern text.
This observation has led us to a couple hypotheses:
Our data is lacking. We took about 100 photos for western and eastern each, but this may have been not enough for the machine learning to generative a conclusive enough database.
The photos that we took could also have been of not high enough quality.
In our sample data, we only wrote horizontally for western text, where as eastern had both horizontal and vertical.
Future thoughts...
To test the machine learning program to see if could simply tell the difference between western and eastern languages, we could do away with the "varied handwriting" completely and use very strict criteria (for handwriting style) in writing our sample text. When we tested the learned program, we could continue to write in that same style between the eastern and western texts. This could help isolate our variables to test out our above hypothesis.
We were inspired by Google's drawing machine learning, and the ability to play around with the different types of applications that machine learning has in with drawing. In order to most quickly and accurately iterate over and over again, we started our explorations by playing around with the whiteboard. We started off playing around with the program to see if machine learning was able to detect the difference between written text and drawings. From this, we were also thinking of maybe incorporating mathematical graphs and/or equations as a possible third scope; an example that lives between text and drawing.
From our experiments, we saw that computer could usually detect between drawings and text, presumptuously mostly dependent on the text. The diversity of drawings was differed widely, as we literally began to draw just about everything that first came to mind, whereas text was definitely more limited in terms of aesthetic, and was visually more uniform. However, we came upon an interesting discovery when drawing stars, but in a linear form. When only one star was drawn, it was detected as a drawing, but as we continued to draw stars next to each other, in a linear fashion, it was detected as text. This propelled us into thinking about the possible implications for using machine learning to detect the differences between languages.
The stars that sparked off our stars.
Our final exploration dealt with exploring the detecting the difference between western and eastern languages; western being more letter-based, and eastern being more pictorial-based characters.
We decided to map the result out visually through three colors:
White indicates that there is no hand written text being shown. (We fed a series of white background images to train this part)
Blue indicates that it is more-so western text. (We fed a series of handwritten phrases and words in English, Spanish, French, and Latin to train this part)
Red indicates that is more-so eastern text. (We fed a series of handwritten phrases and words in Chinese, Japanese, and Korean to train this part)
LanguageML_Test1
From our results, we've discovered a couple things.
The program is relatively good at detecting a "blank background" though a couple times, when our paper was shifted, the program recognized it as "western".
But most importantly, the program was very accurate in detecting western text, but significantly less so with eastern text.
This observation has led us to a couple hypotheses:
Our data is lacking. We took about 100 photos for western and eastern each, but this may have been not enough for the machine learning to generative a conclusive enough database.
The photos that we took could also have been of not high enough quality.
In our sample data, we only wrote horizontally for western text, where as eastern had both horizontal and vertical.
Future thoughts...
To test the machine learning program to see if could simply tell the difference between western and eastern languages, we could do away with the "varied handwriting" completely and use very strict criteria (for handwriting style) in writing our sample text. When we tested the learned program, we could continue to write in that same style between the eastern and western texts. This could help isolate our variables to test out our above hypothesis.
I tried a bunch of different things with the cat one. I like the eye of Sauron one. The emojis are cool too. I found that you really can learn the language of it and totally use it to draw, even if it doesn't always do what you expect.
I was most interested for facades about how the background changed when you changed one element. The rest of the house got a lot fancier when you added windowsills and shutters and trim to the windows. The first one with just windows was much grayer and even dirty looking.
In GANBreeder, I tried to optimize for mouth size. I learned that you lose points on other factors like realism if you just maximize one trait. That was interesting. I tried to find a balance.
I also spent an embarrassingly long time trying to make myself, but I think it's interesting to see if the algorithm can make anyone. I wonder if it's possible to make any arbitrary person given enough iterations. Or any arbitrary image in general. My hunch would be definitely not, but maybe I'm wrong.
GANPaint
Before:
After:
Thoughts: needs more dome.
Text To Transformer
Everybody wants to be happy. But what most people don't know is that that's the hardest thing to get to be happy."
"I'm just doing my best out here. I'm just doing what I can do. ... It's been hard."
"It's just really rough to play here." -- Drew Bledsoe
So far, Bledsoe says, the NBA hasn't impacted him. He's never made more than $4 million playing in the NBA. But it's the job as a starting point guard that's been his biggest challenge.
"I just came to the NBA because I'm hungry," Bledsoe said. "I didn't make the NBA because I worked hard. I made the NBA because I play hard. At that position, playing with these guys and being together, I just tried to play my hardest and do whatever I can to keep this thing going."
Bledsoe says his teammates don't seem to mind his absence, which makes it easier for him to be a leader. Last year, the Celtics lost Brandon Bass and Al Horford to...
I think it's interesting that it started talking about the NBA even though that wasn't in the prompt at all.
It's clearly time for a revolution. As a young man, I knew this for sure, or so I thought. But once we got there, the revolution never happened.
There's a very long way to go, but there's light at the end of the tunnel. And that's my point.
The more you talk about it in any detail, the more you realize that we're not fighting a single war -- we're fighting a series of wars, and they're all part of a much, much larger, much grander scheme, designed to do to me what I think they do to anyone who supports me.
I am fighting to preserve the culture of this country, the history and the legacy of America. I am fighting to ensure the integrity of the family, the home, the community, the job, and all the other pillars of our democracy.
I am fighting to preserve the soul and promise of this amazing country, the values that are the foundation upon which it has been built, and the freedoms that have been the bedrock upon which all Americans of every race, every political bent...
Inspiring.
Google AI Experiments
I messed around with the Infinite Drum Machine, and I think the visualization is really cool. It actually makes it super easy to find similar-sounding samples, but each one is still unique and interesting. Even without the machine learning it's a cool idea, but I also really like the visualization of the algorithm's grouping.
This process was very relaxing and gave me a feeling of control.
The blending between different textures when they were added was very aesthetic.
Mixing faces was a lot more fun than I thought it would be; it was like creating a new person from scratch.
The infinite tiling program didn't take any of my images without raising errors, so I didn't really have a custom output image for this part.
At the end of the day, the sky will still come down."
There are a lot of signs all around the city and on the outskirts that people are ready for rain this Saturday.
Many are holding blankets to protect the windows and cars.
The biggest threat this weekend is rain.
Sometimes I leave my homework for a bit and come back to it, and I think that's the first step to change.
So that's how I started this class. I just wanted to make it better for the next generation. And then a couple of months ago I was inspired to do this again after seeing that this happened all over the United States: students of a certain age who had never been exposed to the art scene at their school were really impressed by the artistry of the pieces I presented.
So after two months of showing art to people, I was like, "Yes, this actually works." I knew that people were getting their first taste of art history. And also, it was fun. But I really really didn't think that I'd be able to create this art class in this way, with that kind of energy. I think that's why there have been people like me who go to school where the art class isn't as successful, where the art classes aren't as well-rounded.
I experimented with the Google Quick Draw game. It was fun to play a sort of speed-pictionary where both the guesser and the artist are under pressure to guess or draw the given prompt.
It interesting to see how edges2cats understood unconventional angles of cats.
Bonus crocs:
B. GANPaintStudio
I think it was interesting how the network interpreted "erasing" a semantic. The door brush in particular seems to un-doorify doors by making them windows.
C. ArtBreeder
This tool is much better at creating uncanny and discomforting images than a human is. I find that interesting because fear is such a difficult emotion to inspire in an image.
D. Infinite Patterns by Alexander Mordvintsev
I used a sketch I made recently as the input image. The CPPN tool seemed to struggle with the simple color palette but the DeepDream tool was able to create a psychedelic pattern.
I am Groot. The only thing you did wrong was get in my way."
"What did you just say?"
"I said, 'I am Groot.'"
"That has got to sting."
"It does."
"But don't you feel like you can let yourself go?"
"Yes, I do."
"I will let you go if it hurts."
"Then I'll tell you everything. So go on."
She was so sweet. She'd never met a man with such a beautiful face. She had her own power. "I don't want to talk about it," she said. "Let's forget it." Then she started on her story.
"And why the tree there?" he asked. "I thought it was just for fun. It was nothing bad."
"And the flower?"
"Well, I don't remember, but it's my favorite. I wanted to bring flowers to you when you found out I can't fly."
"
Then Comes a Mist and a Weeping Rain by Faith Wood (faithwood)
Fandoms: Harry Potter - J. K. Rowling
Tags
No Archive Warnings Apply Draco Malfoy/Harry Potter Harry Potter Draco Malfoy Gregory Goyle Millicent Bulstrode Ernie Macmillan Hermione Granger Ron Weasley Humor Romance Hogwarts Eighth Year HP: EWE Post - Deathly Hallows Complete
Summary
It always rains for Draco Malfoy. Metaphorically. And literally. Ever since he had accidentally Conjured a cloud. A cloud that's ever so cross.
"You shouldn't have conjured one at first," Goyle said.
"I didn't know the clouds could get so big," Hermione said dryly.
"Then why did you do it again?"
"I had to," Hermione said.
"I suppose the weather's going to kill one of us eventually," Goyle said. "But at least we won't have to face those clouds every time we meet."
- E
F. Google AI Experiments
I'm interested in how a lot of these experiments incorporate AI with visual or aural input and output from the user. I tried shadow art, but I found that the entropy introduced by video capture made it difficult to use in the conditions I was in. Additionally, the interaction itself wasn't very interesting. The second experiment I tried, semi-conductor, was more compelling to me because rather than having just an on-screen reaction, the interaction actually felt like I was influencing an orchestra in multiple dimensions.
This is such a playful and fun tool to play around with.
B. GAN Paint
Original:
Resulting:
This image synthesizing system is really interesting to work with, particularly when the resulting image produces something completely unexpected (for example - drawing 'domes' in the above, resulted in bright blue streaks across the image). This seems like it could be a really powerful tool in generative image making.
C. Art Breeder
What I like most about this program is the control that the user has over the image mixes and adjustments. The resulting images can be really beautiful.
D. Infinite Patterns
Image:
Source:
I've been tinkering around this for a bit now, but still don't entirely understand how the sliding menus change the image (in consistent reproducible ways) however, the resulting images tend to be beautiful.
E.GPT-2
This was one of my favorite readymades in this list. It seems that sometimes the completed text would go off in an entirely unrelated direction, and other times a smooth continuation of what I was expecting. Either way the results tended to be really funny.
F. Google AI
One of the experiments I played with was Quick,Draw! in which a network works to guess what you are drawing. It was really fun to hear the in-between guesses that occurred as the images were being drawn.
