Technical Presentation
We will be using the excellent Glitch.com as our platform for creating multi-user networked web applications. Our TA, Charlotte Stiles, has created an account on Glitch, with a number of template projects for you:
- Simple Shared Canvas: app • code
- Simple Chatroom: app • code
- Simple Agar.io Clone: app • code
- Bare socket.io example: code
Additional template projects for immersive shared 2D bodies:
- p5.js + Glitch socket.io + clmTracker face tracker: app • code
Allows people to communicate their face data in a shared virtual space.
- p5.js + Glitch socket.io + ml5.js PoseNet body tracker: app • code
Allows people to communicate their body skeleton data in a shared virtual space. (Built on an example by Lingdong Huang)
Telematic Art (Art Across Distances)
This lecture includes many contributions from NYU Prof. Tega Brain, friend and collaborator.
Today, we participate in synchronous video calls with Skype and FaceTime, collaboratively edit documents in Google Docs, and build multiperson virtual environments in MineCraft. These systems and processes are part of the fabric of everyday life, but it hasn't always been so, and it's worth stepping back and thinking about the use of networked telecommunications in media arts. As is often the case, some media artworks from the past presaged common tools today... and others represent intriguing dead-ends in the history of attempts to devise new modes of communication.
In the projects below, questions how each project resolves issues of:
- Critical consideration of the nature of networked communication
- One to many / many to one / many to many / one to one
- Asynchronous vs. synchronous communication
- Equal roles vs. complementary roles
- Distances of time and/or space
- Which senses are engaged
- Anonymity and intimacy
Across deep time: the Chauvet Cave paintings (40,000 - 30,000 BCE)
The Cave of Forgotten Dreams by Werner Herzog (we will watch from 29:58 - 32:45). Regarding two artists, 5000 years apart, who 'collaborated' on the same drawing, Herzog remarks: "We are locked in history, and they were not."
Cairns
Cairns are human-made piles of stones. Cairns have been and are used for a broad variety of purposes, from prehistoric times to the present, often as landmarks, memorials and trail markers. It is a collaboratively created structure, built one stone at a time, often over the course of centuries.
nVidia Holodeck
So how did we get from there, to ...here?
and, ...here?
Agar.io
Agar.io and the related Slither.io are compelling multiplayer games that are astonishingly well-tuned. Very different strategies are required depending on one's current size/score.
Telephone Picture (László Moholy-Nagy, 1923)
The first use of telecommunications as an artistic medium occurred in 1923 when the Hungarian constructivist artist László Moholy-Nagy made Telephone Pictures. These works questioned the idea of the isolated individual artist and the unique art object.
"In April 1923, Moholy had Construction in Enamel 2 and 3 made at a local enamel factory. He would later claim to have ordered them by describing them over the telephone, exaggerating both his distance from the manufacturing process that produced them and the degree of technological mediation involved. In doing so Moholy presented the artist in the modern age as producer of ideas rather than things. While sharing the same abstract geometric composition, the works use a mathematical progression to change its scale, highlighting the conception of the image as transferable data." (MoMA)
VideoPlace (~1972-1989)
Myron Krueger's landmark VideoPlace (~1972-1989) was a 2D artificial reality that enveloped its users, and responded to their movements and actions, without the use of encumbering goggles or gloves. One of the first interactive computer artworks, it allowed participants in different locations to share a common virtual space -- even, at different scales.
Hole in Space (1980)
Hole in Space (1980) was a "communication sculpture" by Kit Galloway and Sherrie Rabinowitz, which used live two-way video to connect street-level audiences in Los Angeles and New York City.
"On a November evening in 1980 the unsuspecting public walking past the Lincoln Center for the Performing Arts in New York City, and «The Broadway» department store located in the open air Shopping Center in Century City (Los Angeles), had a surprising counter with each other. Suddenly head-to-toe, life-sized, television images of the people on the opposite coast appeared. They could now see, hear, and speak with each other as if encountering each other on the same sidewalk. No signs, sponsor logos, or credits were posted--no explanation at all was offered. No self-view video monitors to distract from the phenomena of this life-size encounter. [...] «Hole-In-Space» suddenly severed the distance between both cities and created an outrageous pedestrian intersection. There was the evening of discovery, followed by the evening of intentional word-of-mouth rendezvous, followed by a mass migration of families and trans-continental loved ones, some of which had not seen each other for over twenty years."
Poietic Generator (1986+) and R/Place (2017)
The Poietic Generator is a social-network game designed by Olivier Auber in 1986. The game takes place within a two-dimensional matrix in the tradition of board games and its principle is similar to both Conway's Game of Life and the surrealists' Exquisite corpse.
Million Dollar Homepage (2005) is a website "conceived by Alex Tew, a student from Wiltshire, England, to raise money for his university education. The home page consisted of a million pixels arranged in a 1000 �-- 1000 pixel grid; the image-based links on it were sold for $1 per pixel in 10 �-- 10 blocks. The purchasers of these pixel blocks provided tiny images to be displayed on them, a URL to which the images were linked, and a slogan to be displayed when hovering a cursor over the link. The aim of the website was to sell all of the pixels in the image, thus generating a million dollars of income for the creator." (Wikipedia)
R/Place "was a collaborative project and social experiment hosted on the social networking site Reddit that began on April Fools' Day 2017. The experiment involved an online canvas of one million (1000x1000) pixel squares, located at a subreddit called /r/place, which registered users could edit by changing the color of a single pixel from a 16-colour palette. After each pixel was placed, a timer prevented the user from placing any pixels for a period of time varying from 5 to 20 minutes." (Wikipedia)
ClearBoard (1990)
ClearBoard, a work of HCI design research by Hiroshi Ishii and Minoru Kobayashi, "is designed to integrate interpersonal space and shared workspace seamlessly. A design goal of ClearBoard is to allow a pair of users to shift easily between interpersonal space and shared workspace using familiar everyday cues such as the partner's gestures, head movements, eye contact, and gaze direction. The key metaphor of ClearBoard design is 'talking through and drawing on a big transparent glass board.' ClearBoard requires less eye and head movement to switch focus between the drawing surface and the partner's face than is needed in either the whiteboard or the desktop environment. We found ClearBoard provides the capability we call "gaze awareness": the ability to monitor the direction of a partner's gaze and thus his or her focus of attention. A ClearBoard user can tell which screen objects the partner is gazing at during a conversation more easily and precisely than is possible in an ordinary meeting environment with a whiteboard." (MIT Tangible Media Group)
Telematic Dreaming (1992-1993)
Paul Sermon's Telematic Dreaming (1992-1993) "is an installation that exists within the ISDN digital telephone network. Two separate interfaces are located in separate locations, these interfaces in themselves are dynamic installations that function as customized video-conferencing systems. A double bed is located within both locations, one in a blacked out space and the other in an illuminated space. The bed in the light location has a camera situated directly above it, sending a live video image of the bed, and a person ("A") lying on it, to a video projector located above the other bed in the blacked out location. The live video image is projected down on to the bed with another person ("B") on it. A second camera, next to the video projector, sends a live video image of the projection of person "A" with person "B" back to a series of monitors that surround the bed and person "A" in the illuminated location. The telepresent image functions like a mirror that reflects one person within another person's reflection." (Paul Sermon)
The Trace (1995)
In The Trace (1995), an interactive installation by Rafael Lozano-Hemmer, a participant encounters the moving, ghostlike "presence" of another person, who is located in an identical but separate room. Their presence is conveyed by a glowing spot created by a pair of intersecting beams of light.
Motion Phone (1995)
Motion Phone by Scott Snibbe (1995; iOS version 2012) "is a new form of communication and a creative game. It lets people make abstract animations together over a network in a unique combination of conversation and creation. In MotionPhone's shared infinite canvas, people can create unlimited variations of movement by recording the gestures from their fingertips. When connected to other players, MotionPhone lets you create, comment, play, and communicate visually with others." (Scott Snibbe)
Telegarden (1995)
Ken Goldberg & Joseph Santarromana's TeleGarden is an art installation that allows web users to view and interact with a remote garden filled with living plants. Members can plant, water, and monitor the progress of seedlings via the tender movements of an industrial robot arm. The Telegarden was developed at the University of Southern California and went online in June 1995. In its first year, over 9000 members helped cultivate. In September 1996, the Telegarden was moved to the lobby of the Ars Electronica Center in Austria, where it remained online until August 2004.
inTouch (1998)
inTouch (Scott Brave & Andy Dahley, 1998) -- an experiment in telepresent haptics, allowed two people to communicate through shared movements of a set of wooden rollers.
BumpList (2003)
Jonah Brucker-Cohen's BumpList (2003) is "an email community for the determined". BumpList "functions like a standard, public listserv, but adds the constraints of limited membership (only 6 people could subscribe at once) and the urgency of forcing people to re-subscribe if they get bumped and want to continue the conversations, discussions, arguments in which they are engaged." (Brucker-Cohen)
Image Labeling Game (2004)
In 2004, Luis von Ahn and Laura Dabbish developed the "ESP Game", which was later (2006) developed into the "Google Image Labeler". The foundational concept of gamifying the elicitation of intersubjectively valid labels for images, eventually would underpin a wide range of technologies, like CAPTCHA, for labeling images and (more importantly) creating terascale datasets for training machine learning algorithms.
Can We Talk? (2011)
Zach Gage's Can We Talk? (2011) "is a chat program for having serious conversations. It attempts to disambiguate the reasons for silence in online conversation, while preserving the qualities that make text chatting a unique platform for communication." (Gage)
Do Not Touch (2013)
Studio Moniker's Do Not Touch (2013) [live version here] is a music video created from the crowdsourced movements of its viewers' cursors.
Social Turkers (2013)
Lauren McCarthy's Social Turkers (2013) asks: "What if we could receive real-time feedback on our social interactions? Would unbiased third party monitors be better suited to interpret situations and make decisions for the parties involved? How might augmenting our experience help us become more aware in our relationships, shift us out of normal patterns, and open us to unexpected possibilities? I am developing a system like this for myself using Amazon Mechanical Turk. During a series of dates with new people I meet on the internet, I will stream the interaction to the web using an iPhone app. Turk workers will be paid to watch the stream, interpret what is happening, and offer feedback as to what I should do or say next. This feedback will be communicated to me via text message." (McCarthy)
the space between us (2015)
the space between us (2015) by David Horvitz is an iOS app which connects two people's phones. Once connected, the app displays the distance and direction to the other person.
Die With Me (2018)
Dries Depoorter and David Surprenant developed Die With Me, a chat app you can only use when you have less than 5% battery. "Die together in a chatroom on your way to offline peace."
Exhausting a Crowd (2015)
Kyle McDonald's Exhausting a Crowd (2015) is "a crowdsourced description of 12 hours in Piccadilly Circus, London, inspired by the classic 60-page piece of experimental literature from Georges Perec, 'An Attempt at Exhausting a Place in Paris', written from a bench over three days in 1974." (McDonald)
Real Me
Jingwen Zhu's Real Me (2015) "is a chatting app with BLE connected muscle sensor, pulse sensor and vibration sensor. It tracks the user's emotion while chatting and using system messages to tell the 'real emotion' of the user. Sensor readings also draw out geometric motion graphics as the background of the chatting room. The user's heart rate is mapped to the color of the speech bubble, their smile triggers the particles of the window, and leg shaking triggers the shaking of speech bubbles." (Zhu)
Aram Bartholl, Dead Drops (2010)
Other collaborative drawing tools
- Communimage (Johannes Gees, 1999)
- Net.Flag (Mark Napier, 2002)
- Drawball (2005)
Experimental chat spaces by Maddy Varner (CMU BFA '16)
- Poopchat Pro - A chatroom in which participants communicate with small poops
- Pasties - A chatroom in which participants communicate exclusively by copy-pasting.
If time permits, we'll also look at...
- Bilal, Wafaa. Domestic Tension (2007). Viewers log onto the internet to contact the artist, or to shoot him with a paintball gun.
- Max Hawkins, Call in the Night (2013), "An experimental radio show and telephone network documenting the nighttime experience. Anyone with a phone can participate for free."
- Soledad Penadés, Chat.meatspac.es. (2014), allows 'chatting' with small webcam GIFs.
- Exonemo. The Internet Bedroom (2015). A sleepover in Google Hangouts.