Work

szh-SituatedEye

We were inspired by Google's drawing machine learning, and the ability to play around with the different types of applications that machine learning has in with drawing. In order to most quickly and accurately iterate over and over again, we started our explorations by playing around with the whiteboard. We started off playing around with the program to see if machine learning was able to detect the difference between written text and drawings. From this, we were also thinking of maybe incorporating mathematical graphs and/or equations as a possible third scope; an example that lives between text and drawing.

From our experiments, we saw that computer could usually detect between drawings and text, presumptuously mostly dependent on the text. The diversity of drawings was differed widely, as we literally began to draw just about everything that first came to mind, whereas text was definitely more limited in terms of aesthetic, and was visually more uniform. However, we came upon an interesting discovery when drawing stars, but in a linear form. When only one star was drawn, it was detected as a drawing, but as we continued to draw stars next to each other, in a linear fashion, it was detected as text. This propelled us into thinking about the possible implications for using machine learning to detect the differences between languages.

The stars that sparked off our stars.

Our final exploration dealt with exploring the detecting the difference between western and eastern languages; western being more letter-based, and eastern being more pictorial-based characters.

We decided to map the result out visually through three colors:

  • White indicates that there is no hand written text being shown. (We fed a series of white background images to train this part)
  • Blue indicates that it is more-so western text. (We fed a series of handwritten phrases and words in English, Spanish, French, and Latin to train this part)
  • Red indicates that is more-so eastern text. (We fed a series of handwritten phrases and words in Chinese, Japanese, and Korean to train this part)

From our results, we've discovered a couple things.

The program is relatively good at detecting a "blank background" though a couple times, when our paper was shifted, the program recognized it as "western".

But most importantly, the program was very accurate in detecting western text, but significantly less so with eastern text.

This observation has led us to a couple hypotheses:

  • Our data is lacking. We took about 100 photos for western and eastern each, but this may have been not enough for the machine learning to generative a conclusive enough database.
  • The photos that we took could also have been of not high enough quality.
  • In our sample data, we only wrote horizontally for western text, where as eastern had both horizontal and vertical.

Future thoughts...

To test the machine learning program to see if could simply tell the difference between western and eastern languages, we could do away with the "varied handwriting" completely and use very strict criteria (for handwriting style) in writing our sample text. When we tested the learned program, we could continue to write in that same style between the eastern and western texts. This could help isolate our variables to test out our above hypothesis.

gray-MachineLearning

Pix2Pix:

I tried a bunch of different things with the cat one. I like the eye of Sauron one. The emojis are cool too. I found that you really can learn the language of it and totally use it to draw, even if it doesn't always do what you expect.

I was most interested for facades about how the background changed when you changed one element. The rest of the house got a lot fancier when you added windowsills and shutters and trim to the windows. The first one with just windows was much grayer and even dirty looking.

In GANBreeder, I tried to optimize for mouth size. I learned that you lose points on other factors like realism if you just maximize one trait. That was interesting. I tried to find a balance.

I also spent an embarrassingly long time trying to make myself, but I think it's interesting to see if the algorithm can make anyone. I wonder if it's possible to make any arbitrary person given enough iterations. Or any arbitrary image in general. My hunch would be definitely not, but maybe I'm wrong.

GANPaint

Before:

After:

Thoughts: needs more dome.

Text To Transformer

Everybody wants to be happy. But what most people don't know is that that's the hardest thing to get to be happy."

"I'm just doing my best out here. I'm just doing what I can do. ... It's been hard."

"It's just really rough to play here." -- Drew Bledsoe

So far, Bledsoe says, the NBA hasn't impacted him. He's never made more than $4 million playing in the NBA. But it's the job as a starting point guard that's been his biggest challenge.

"I just came to the NBA because I'm hungry," Bledsoe said. "I didn't make the NBA because I worked hard. I made the NBA because I play hard. At that position, playing with these guys and being together, I just tried to play my hardest and do whatever I can to keep this thing going."

Bledsoe says his teammates don't seem to mind his absence, which makes it easier for him to be a leader. Last year, the Celtics lost Brandon Bass and Al Horford to...

I think it's interesting that it started talking about the NBA even though that wasn't in the prompt at all.

It's clearly time for a revolution.  As a young man, I knew this for sure, or so I thought. But once we got there, the revolution never happened.
There's a very long way to go, but there's light at the end of the tunnel. And that's my point.
The more you talk about it in any detail, the more you realize that we're not fighting a single war -- we're fighting a series of wars, and they're all part of a much, much larger, much grander scheme, designed to do to me what I think they do to anyone who supports me.
I am fighting to preserve the culture of this country, the history and the legacy of America. I am fighting to ensure the integrity of the family, the home, the community, the job, and all the other pillars of our democracy.
I am fighting to preserve the soul and promise of this amazing country, the values that are the foundation upon which it has been built, and the freedoms that have been the bedrock upon which all Americans of every race, every political bent...

Inspiring.

Google AI Experiments

I messed around with the Infinite Drum Machine, and I think the visualization is really cool. It actually makes it super easy to find similar-sounding samples, but each one is still unique and interesting. Even without the machine learning it's a cool idea, but I also really like the visualization of the algorithm's grouping.

 

gray-Body

I made an annoying robot that withholds a cute picture from you as long as it can. It's based in a terminal.

I wanted it to be a little unclear how much this robot knew, and at one point it asks you to do something and then doesn't even check if you've done it at all, it just moves on regardless. But I also just wanted to see how much you can make somebody work if you just slowly add more and more tasks. It's like the sunk-cost fallacy; they've come this far so they can't stop now. It's just a prototype, and I'd like it polished up and longer, but it captures the idea, I think.

Here are some videos of people playing it:

Here's the link to the project: https://editor.p5js.org/gray/sketches/XQDS1U6gE

sansal – ML Toe Dipping

This process was very relaxing and gave me a feeling of control.

The blending between different textures when they were added was very aesthetic.

Mixing faces was a lot more fun than I thought it would be; it was like creating a new person from scratch.

The infinite tiling program didn't take any of my images without raising errors, so I didn't really have a custom output image for this part.

At the end of the day, the sky will still come down."

There are a lot of signs all around the city and on the outskirts that people are ready for rain this Saturday.

Many are holding blankets to protect the windows and cars.

The biggest threat this weekend is rain.

Sometimes I leave my homework for a bit and come back to it, and I think that's the first step to change.

So that's how I started this class. I just wanted to make it better for the next generation. And then a couple of months ago I was inspired to do this again after seeing that this happened all over the United States: students of a certain age who had never been exposed to the art scene at their school were really impressed by the artistry of the pieces I presented.

So after two months of showing art to people, I was like, "Yes, this actually works." I knew that people were getting their first taste of art history. And also, it was fun. But I really really didn't think that I'd be able to create this art class in this way, with that kind of energy. I think that's why there have been people like me who go to school where the art class isn't as successful, where the art classes aren't as well-rounded.

 

I experimented with the Google Quick Draw game. It was fun to play a sort of speed-pictionary where both the guesser and the artist are under pressure to guess or draw the given prompt.

 

 

tli-MachineLearning

A. Pix2Pix

It interesting to see how edges2cats understood unconventional angles of cats.

Bonus crocs:

B. GANPaintStudio

I think it was interesting how the network interpreted "erasing" a semantic. The door brush in particular seems to un-doorify doors by making them windows.

C. ArtBreeder

This tool is much better at creating uncanny and discomforting images than a human is. I find that interesting because fear is such a difficult emotion to inspire in an image.

D. Infinite Patterns by Alexander Mordvintsev

I used a sketch I made recently as the input image. The CPPN tool seemed to struggle with the simple color palette but the DeepDream tool was able to create a psychedelic pattern.

E. GPT-2

I first tried the title of the most popular fanfic in Archive of Our Own.

I am Groot. The only thing you did wrong was get in my way."

"What did you just say?"

"I said, 'I am Groot.'"

"That has got to sting."

"It does."

"But don't you feel like you can let yourself go?"

"Yes, I do."

"I will let you go if it hurts."

"Then I'll tell you everything. So go on."

She was so sweet. She'd never met a man with such a beautiful face. She had her own power. "I don't want to talk about it," she said. "Let's forget it." Then she started on her story.

"And why the tree there?" he asked. "I thought it was just for fun. It was nothing bad."

"And the flower?"

"Well, I don't remember, but it's my favorite. I wanted to bring flowers to you when you found out I can't fly."

"
Then I tried entering the description of the most popular Harry Potter fanfic on the same website.
Then Comes a Mist and a Weeping Rain by Faith Wood (faithwood)
Fandoms: Harry Potter - J. K. Rowling 

Tags

No Archive Warnings Apply Draco Malfoy/Harry Potter Harry Potter Draco Malfoy Gregory Goyle Millicent Bulstrode Ernie Macmillan Hermione Granger Ron Weasley Humor Romance Hogwarts Eighth Year HP: EWE Post - Deathly Hallows Complete 

Summary

It always rains for Draco Malfoy. Metaphorically. And literally. Ever since he had accidentally Conjured a cloud. A cloud that's ever so cross.

"You shouldn't have conjured one at first," Goyle said.


"I didn't know the clouds could get so big," Hermione said dryly.


"Then why did you do it again?"

"I had to," Hermione said.

"I suppose the weather's going to kill one of us eventually," Goyle said. "But at least we won't have to face those clouds every time we meet."

- E

F. Google AI Experiments

I'm interested in how a lot of these experiments incorporate AI with visual or aural input and output from the user. I tried shadow art, but I found that the entropy introduced by video capture made it difficult to use in the conditions I was in. Additionally, the interaction itself wasn't very interesting. The second experiment I tried, semi-conductor, was more compelling to me because rather than having just an on-screen reaction, the interaction actually felt like I was influencing an orchestra in multiple dimensions.

lubar – machinelearning

A. Pix2Pix

This is such a playful and fun tool to play around with.

B. GAN Paint

Original:

Resulting:

This image synthesizing system is really interesting to work with, particularly when the resulting image produces something completely unexpected (for example - drawing 'domes' in the above, resulted in bright blue streaks across the image). This seems like it could be a really powerful tool in generative image making.

C. Art Breeder

What I like most about this program is the control that the user has over the image mixes and adjustments. The resulting images can be really beautiful.

D. Infinite Patterns

Image:

Source:

I've been tinkering around this for a bit now, but still don't entirely understand how the sliding menus change the image (in consistent reproducible ways) however, the resulting images tend to be beautiful.

E.GPT-2

 

This was one of my favorite readymades in this list. It seems that sometimes the completed text would go off in an entirely unrelated direction, and other times a smooth continuation of what I was expecting. Either way the results tended to be really funny.

F. Google AI

One of the experiments I played with was Quick,Draw! in which a network works to guess what you are drawing. It was really fun to hear the in-between guesses that occurred as the images were being drawn.

vingu -ML toe dipping

APix2Pix I played around to see how it would work with non-cat drawings, and how it would detect circles and ellipses.

B GANPaint StudioI found it interesting that it used parts of the image, such as turning the red bus into a red door. (rather than pasting on a door)

 

C Art Breeder

I manipulated the genes so that the original genes could not be recongized. I also combined all the members of BTS for fun.

D Infinite Patterns

 

E GPT 2

lsh-MachineLearningA

Pix2Pix

The edges2cats filter was interesting to play with in regards to how it dealt with line and scale. Large open spaces were filled with texture, while finer, smaller detail led to a blurry mess.

The facades felt more ambiguous in output oddly enough. Maybe due to the Mondrian style of partitioning the UI requires.

GAN Paint

source: Carnegie Mellon University College of Fine Arts Facebook

When working with the default image, the effects were immediate no matter what tool I used. When I uploaded my own image, the effects became much harder to tease out.

Artbreeder

Artbreeder is always a lot of fun, given the sheer amount of content one can generate within the tool. I do occasionally consider it through the following lens though:

Infinite Patterns

GPT-2

Do you know who I am?

You see, this is where things get a little confusing. I'm a lot more than I let on as a person.

Let me explain.

First, I'm not a monster.

Second - and this is important because it has implications for my next step - I'm the kind of person who is able to change.

Change is not the same thing as changing a person. A person can make a life-long change in such a short time that the change is invisible - it's not part of the world they know or understand or know how to interact or have compassion for.

Changing a person is hard; you're making the change for the sake of yourself and others.

And that's hard.

The best way I've found to change was in what I didn't know I was capable of.

It wasn't that I wanted to change, it was that I knew I couldn't change, and so I made a choice - a good, thoughtful choice - to not learn the lessons I could

What's love got to do with it? Well, like it's a question that's been answered many times over, it's an all-encompassing question that goes a number of ways and there is no right or wrong answer.

But for those who want to know, the answer starts with a simple premise:

"You should love everyone you meet."

And why would you want to say something so obvious and easy to say? Well, perhaps you need some extra help when it comes to saying it or you fear other people don't really know what they're doing when it comes to the most basic human needs... or maybe you can't quite make the case that it's not a simple idea, even though you know it to be so. But the good news is that once you say it, it's there for everyone to understand and respect, so there's really no reason to fear or deny saying it.

I'd argue that this simple premise applies to any relationship for both men and women so even if you're not ready for it right away, there can be no better idea

I usually don't think of AI as being used for a stylistic decision like font making, so I was curious to see what pairings the system suggested.

iSob-MachineLearningA

A. Pix2Pix

I spent some time on Edges2Cats making people's fursonas.

This tool is lots of fun, but frustrating to use because of the discrepancies between how humans interpret edges and how an edge detector does. Connie suggested that I look at the original inputs (edge detections of cat photos) to see what arrangements of edges produced what output. This technique was necessary to understand how a cat nose/mouth was generated. As such, all my drawings have a very pronounced upper lip area to force the cat mouth to appear.

Here are facades of windows within doors, and an image of a dark future where humans are the handbags of omnipotent AIs.

B. GANpaint studio

GANpaint was probably the most frustrating tool in this assignment, but I think it has a lot of potential! Currently, the lack of layers you can 'paint' with and the very low resolution output limit what you can create. I assume that as the encoding of semantic features becomes better understood, more features will be available to manipulate in later demos.

C. Artbreeder

I have spent so much time on Artbreeder that I don't even know what to say about it! The new Portraits category is very frustrating (I can't understand what the sliders actually encode) but in a way that makes me want to spend more time with it. I tried to make myself, but it didn't turn out very well. Other people have made amazing Danny Devitos and Elon Musks.

Here is my Artbreeder profile if you want to see all my creations.

D. Infinite Patterns

 

E. GPT-2

That was the funnest hour of my life so far. All original text came from humorous online posts, it is not mine.

F. Google AI Experiments

I played Semantris for a couple of rounds. Perhaps it was not the most creative use of my time, but I enjoyed it a lot. It made me wonder if I was contributing training data to something. The most obvious answer would be a semantic mapping algorithm. It would be interesting if destroying multiple connected blocks required you to relate the two words in a clue.

zapra – 08a – machinelearning

Pix2Pix

This was a really fun way of seeing the different opportunities to interact with machine learning from the user's perspective. I love all my cats.

GAN Paint

This reminded me a lot of the stamp tool in Photoshop. The examples given are all architectural, but I think it would be interesting to see this being used on people or other living things.

Artbreeder

I had a lot of fun creating my own mix of genes (see a chicken-catamaran hybrid below).

Infinite Patterns

I created this using the seal + stone wall + African Grey Parrot crossbreed I had made in ArtBreeder.

Talk to Transformer

This was my favorite one to play with by far. My first several attempts were too vague to generate anything of interest, but its funny to see how many specifics you need to feed the transformer to get a satisfactory amount of coherent nonsense.

  1. I had this crazy dream about zombies last night where we were all in the same room together." He smiled. "I don't remember if I was the one who said 'we' that time or whether it was the other guests or the other staff, but we were all feeling weird when we woke up to discover that we would soon be fighting for their lives against these horrible creatures."

"Yes, I remember," her father replied with a small smile. "It was a good dream." He reached for a drink of water, but hesitated when he heard the sound of water dripping down the hallway below, then he looked down the hallway to see that three of the four staircases had collapsed on their own.

Harry sighed. With one hand pressed to his forehead, he reached for the drink in the bowl and took a sip. "What Can I tell you a secret about why I am afraid of birds? I am afraid of birds because, like all things that live in the sky, they have eyes. And I fear that their eyes are in the sky and their eye-screens are on and I can't tell them anything except what their own eyes can see. And so I fear their eyes like a mother afraid of a baby. And I know they will hurt me.

"What if one day, one of them looks up and sees me, I am so afraid of being hurt so I put a glass of water on the stove and sit there, so that they can't reach me. They are too smart, too clever or there is some mistake. So, even if they know every part about me that has not been invented, they will never hurt me. And sometimes they seem to be watching the moon."

He paused and then added: "And sometimes they are thinking how to tell this to their offspring. They are already thinking too much, but I am only a child, and they have the power of children."

The woman laughed and said happened?" he asked.

"When we arrived at the conference, we found out that the hospital was closing down until further notice while the owner decided to leave the building entirely." Her father replied as he sipped from the cup. "And the next time you arrive

----------------------------------------------------------

2. Can I tell you a secret about why I am afraid of birds? I am afraid of birds because, like all things that live in the sky, they have eyes. And I fear that their eyes are in the sky and their eye-screens are on and I can't tell them anything except what their own eyes can see. And so I fear their eyes like a mother afraid of a baby. And I know they will hurt me.

"What if one day, one of them looks up and sees me, I am so afraid of being hurt so I put a glass of water on the stove and sit there, so that they can't reach me. They are too smart, too clever or there is some mistake. So, even if they know every part about me that has not been invented, they will never hurt me. And sometimes they seem to be watching the moon."

He paused and then added: "And sometimes they are thinking how to tell this to their offspring. They are already thinking too much, but I am only a child, and they have the power of children."

The woman laughed and said

Google AI: Magic Sketchpad

Here's a sketchpad on glitch that will try to finish your drawings for you. It's cool when its accurate and endearing when it scribbles haphazardly.