ROBBIE TILTON

Time Dilation with Unity3D

rtilton1 — Wed, 22 Feb 2012 02:16:22 +0000

Using Unity3D I teamed up with Jackson Snellings to create an interactive environment where a user can take control of two joysticks to navigate and control time within the scene. As a user pulls on one joystick time begins to fast forward. You can see this in the audio manipulation and time distortion. As time moves forward the environment eventually melts down and becomes flooded with water. We created this piece as our second assignment for Creating 3D Worlds with James George. The piece implies the juxtaposition between a god-like ability to alter time and the powerlessness we have toward the destruction of the earth

AGI Scripting with Ruby

rtilton1 — Tue, 21 Feb 2012 16:06:58 +0000

This week we learned how to script with AGI using ruby. AGI stands for Asterisk Gateway Interface and allows one to make asterisk commands through a proper programming language (ruby or PHP).

Within your asterisk extensions.conf file all you need is below:

exten => s,1,Answer();
exten => s,n,AGI(/home/rtt233/asterisk_agi/weather.rb);

Then, in weather.rb you can use the ruby-agi to call your dialplan with conditional logic. Below is an example that allows you to pull the weather from www.weather.gov.

#!/usr/bin/ruby
 
require 'rubygems'
require 'ruby-agi'		#for Asterisk AGI
require 'net/http'		#for http connections
require 'rexml/document'	#for parsing XML
 
agi = AGI.new
 
# for a complete list of US cities, go to
# http://www.weather.gov/xml/current_obs/
weatherURL = []
weatherURL << "http://www.weather.gov/xml/current_obs/KNYC.xml" #NYC
weatherURL << "http://www.weather.gov/xml/current_obs/KJFK.xml" #JFK
weatherURL << "http://www.weather.gov/xml/current_obs/KART.xml" #Watertown, NY
weatherURL << "http://www.weather.gov/xml/current_obs/KBGM.xml" #Binghamton
weatherURL << "http://www.weather.gov/xml/current_obs/KBUF.xml" #Buffalo
weatherURL << "http://www.weather.gov/xml/current_obs/KDKK.xml" #Chautauqua County
weatherURL << "http://www.weather.gov/xml/current_obs/KDSV.xml" #Dansville
weatherURL << "http://www.weather.gov/xml/current_obs/KELM.xml" #Corning, NY
weatherURL << "http://www.weather.gov/xml/current_obs/KHPN.xml" #Westchester
weatherURL << "http://www.weather.gov/xml/current_obs/KFRG.xml" #Farmingdale, NY
 
while true #loop forever
	agi.stream_file("vm-extension")
	result = agi.wait_for_digit(-1)  # -1 = wait forever
	digit = result.digit
	if (!digit)
		agi.noop("no button received.  quitting AGI script.")
		exit #quit ruby
	end
	url = weatherURL[digit.to_i]
	# get the XML data as a string
	xml_data = Net::HTTP.get_response(URI.parse(url)).body
	# extract event information
	doc = REXML::Document.new(xml_data)
	temp_f = doc.get_text('current_observation/temp_f')
 
	if temp_f
		current_temp = temp_f
	else
		agi.noop("couldn't find temp in xml. quitting.")
		continue = false
		exit #quit ruby
	end
	#say the temp and ask for another digit
	agi.say_number(current_temp)
end

3D Cinemagraph Morphing

rtilton1 — Fri, 17 Feb 2012 02:12:57 +0000

Using the Kinect I created several sequenced 3D videos that morph into each other through a particle explosion/retraction. When building this I hoped to capture some of the short subtle animations that I love in gif cinemagraphs. Although this is a bit different than a classic cinemagraph I hope to work on new ways to take/render 3D images.

In order to create this application I used openFrameworks with C++ (ofxKinect library). To take the 3D photo sequences I used Kyle McDonald’s code which takes an RGBA image, but writes the depth values into the A variable of the image file. His code can be found on github here. The project also stems from some inspiration from the Janus Machine which Kyle made in 2011. My modified code can be found on my Github here.

Soundtrack feat Fatboy Slim – Gangster Tripping

Neural Computation

rtilton1 — Thu, 16 Feb 2012 19:37:36 +0000

There are three types of computing: digital computing, analog computing, and neural computing. Digital computing is known for being really good and efficient at computing bits and can process billions of bits at a time. Our modern day personal computing all use digital computing. When researching emotions and the ability to program human feeling into a computer I’m constantly directed to study neural computing (when adapted to a digital computer it is often referred to as Affective Computing) – which is the way our minds work. Marvin Minsky breaks down the process of the mind with six levels of computation. The image on the left illustrates his six process model. He refers to Platos republic when description how each of the six levels are independent and their communication is limited (similar to the prisoners in the cave who could only view shadows of what was between them and the fire). Through understanding the way the mind works (which is still not fully known) through Marvin Minsky’s The Emotional Machine, one realizes how complex each level is and how our neural capabilities are highly complex and run in parallel formations where comprehension can be derived all at once. Computers tend to be more linear, but I believe to properly reproduce the human cognitive experience one would need to use massively parallel machines and need a custom computer(s) that are capable of less linear memory storage and processing. For example, Chris Wood from the New Mexican states, “Modern computers have a central clock with rates in the range of 1GHz, which corresponds to a cycle time of one billionth of a second. In contrast, the maximum rate a neuron can fire is about 1KHz, which corresponds to a tortoise-like one thousandth of a second. Digital computers are serial in the sense that all computation is performed by a single CPU, although the current direction in computer design is toward 2-, 4- and 8-processor designs, and the most powerful supercomputers have thousands of processors. In contrast, the brain has no single clock, but it computes with 100 billion neurons and 100 trillion synapses simultaneously.” The manner in which the mind works is pretty incredible, but it’s interesting how contrasting it is to the machines we use. To unlock the mysteries of the mind, many argue that we can prove theories of our brains processes if we can make a computer capable of thinking/acting like us. Although this would be an incredible accomplishment, for my project I plan to only focus on one particular aspect of a specific level – self-conscious emotions. Perhaps if each level is explored and perfected a greater digital machine can be born where all parts of the mind work together in a coherent and human-like way.

Dialing (SIP/IAX) and DB with Asterisk

rtilton1 — Sat, 11 Feb 2012 03:47:00 +0000

Wowzers! A lot of new material this week in the Asterisk world. First of all, I learned how to use SIP and IAX2 to make calls. For this I started with a SoftPhone (Zoiper Classic) – using it to call the Asterisk server. Below is my sip.conf

[rtt233]
type=friend
username=rtt233_sip
secret=XXXXXXX
host=dynamic
dtmfmode=rfc2833
disallow=all
allow=gsm
nat=yes
qualify=yes
context=rtt233_sip

Below is my dialplan for SIP:

[rtt233_sip]
exten => _X,1,GoTo(rtt233_sip_answer,s,1)
 
[rtt233_sip_answer]
exten => s,1,Playback(hello-world)
	same => n,Playback(auth-thankyou)
	same => n,Hangup()

Below is my iax.conf:

[rtt233_iax]
type=friend
host=dynamic
username=rtt233_iax
secret=XXXXXXXX
auth=md5
requirecalltoken=no
context=rtt233_iax

Below is my dialplan for IAX:

[rtt233_iax]
exten => _X,1,GoTo(rtt233_iax_answer,s,1)
 
[rtt233_iax_answer]
exten => s,1,Playback(hello-world)
	same => n,Playback(auth-thankyou)
	same => n,Hangup()

My dialplan and setup files here are pretty plain, but it was great to see the connection being made from my machines softphone to the Asterisk server. After this I learned about dialing to a real phone using SIP:

exten => s,1,Dial(SIP/itp_jnctn/19175551234,30,r) ;SIP call

An alternative to this – one can let the user choose a number to dial with:

exten => _1NXXNXXXXXX,1,Dial(SIP/itp_jnctn/${EXTEN},30,r) ;dials any valid North American phone.

As well as calling a machine through the dial plan IAX2:

	same => n,Set(CALLERID(num)=12125551234) ;set caller ID
	same => n,Dial(IAX2/rtt233_iax,30,r)

Alright…So we have simple dialing down…how can we save variables so that they can be reused over multiple calls? With Asterisk we can save variables to a Database(DB). Check out the below DB calls that can be used in any dialplan:

	same => n,Set(numOfCalls=${DB(rtt233/calls)}) ;set # in database
	same => n,Set(numOfCalls=$[1 + ${numOfCalls}]) ;add 1 to it
	same => n,Set(DB(rtt233/calls)=${numOfCalls}) ;take # from database
	same => n,SayDigits(${numOfCalls}) ;say # of calls

Face Puppets

rtilton1 — Thu, 09 Feb 2012 22:05:12 +0000

For Appropriating New Technologies I used Jason Saragih’s C++ Face Tracker code alongside with Kyle McDonald’s ofxFaceTracker library found on his github here. In essence I’m using the face tracking code to get a 1 mask on a static image and another mask from my webcam stream. I then use the TextureMesh from the static image to map it onto my webcam mask. Lastly, I position/translate my mask (with the static image textured onto it) into the same position that if found the static mask so that it looks like it is part of the static image. This allows me to puppeteer the image. My code can be downloaded on my Github here.

The video I made is an art piece to puppeteer what many american’s feel is the ultimate puppet – US Presidents. With puppeteering I’m trying to pull out the inner child in famous presidents who are forced into a very public and very serious role. My goal of this is to have babies and young children control these serious figures. I wasn’t able to find a child to control the puppets with this week, but plan to redo this piece with the proper pupet master.

Lastly, I’d like to give another huge applaud to Jason Saragih, Kyle McDonald, and Arturo for making this technology open source and accessible for newb inventors like myself. It’s really amazing that we can all use this type of face tracking and I can’t even imagine what we will be using in 20 years…

soundrack feat Jay-Z – Forever Young

Emotional Intelligence Alogorithms

rtilton1 — Wed, 08 Feb 2012 17:47:39 +0000

I’ve always been fascinated by Emotional Intelligence (EQ) and feel it often takes a backseat to peoples fascination in IQ and making things quantifiably intelligent. As with computing, the machine has been built to be able to perform highly data intensive procedures at extremely high speeds, but we still haven’t uncovered emotional computing. For my Research Algorithms class I’ll be making a computer that feels emotions.

There are two types emotions that drive our life and our actions: 1) the way in which we understand the emotions around us (example – facial expressions, gestures, tone of voice) and 2) the way in which we feel emotions for ourselves. Through this external and internal process we are able to listen to emotions, think about emotions, and react to emotions.

There have been a lot of really great examples in the field of understanding emotions. The two videos presented to the left use robotics (from MIT Media Lab and Cambridge) to mimic human emotions. Although they seek realistic in terms of movement and understanding and are extremely impressive accomplishment, the ability to think and feel for oneself is missing.

In 2004, the Washington Post asked its readers, “If computers are to have emotional components, what role would they play in everyday life? Do human beings really want an emotional relationship with a mechanical mind?” Through media we’ve seen many instances of computers that can feel however the consequence of these machines existence is often portrayed as a threat to the human kind. Hal in 2001 Space Odyssey attempts to kill all the humans on a spaceship, in I-Robot the ability to feel leads to murder and suicide, and in A.I. Artificial Intelligence, David follows a path similar to Pinocchio with a heartbreaking conclusion where he relizes he will never be a real boy and that the mother he loves has to die. Although Hollywood portrays machines with emotions as an evolutionary disaster, I believe there is a deeper meanings that can be derived from creating a computer that feels.

Beatbox with Asterisk

rtilton1 — Tue, 07 Feb 2012 00:19:18 +0000

This week I used Asterisk to create a beatbox where a user is given a continuous beat. As the beat loops a user can break out of it and contribute to the music by pressing two on their keypad. This initiates a record application that records a users audio input at a lower pitch and then places it back within the stream of the beatbox. Right now the beat and the recorded sound play back-to-back. In future iterations I would like to use Sox to combine the audio files on the fly to create a seamless sound integration.

Below is my code

[rtt233]
exten => s,1,Wait(1)
exten => s,n,Playback(/home/rtt233/asterisk_sounds/intro)
exten => s,n,Goto(rtt233_beat,s,1)
 
[rtt233_beat]
exten => s,1,Playback(/home/rtt233/asterisk_sounds/recordstart)
	same => n,Background(/home/rtt233/asterisk_sounds/beat, m)
	same => n,Goto(rtt233_beat_withoutRecorded,s,1)
	same => n,WaitExten(30);
 
[rtt233_beat_withoutRecorded]
exten => s,1,Background(/home/rtt233/asterisk_sounds/beat, m)
	same => n,Goto(rtt233_beat_withoutRecorded,s,1)
	same => n,WaitExten(30);
 
exten => 2,1, Playback(/home/rtt233/asterisk_sounds/recordend)
	same => n,Set(PITCH_SHIFT(rx)=0.7)
	same => n,Record(asterisk-recording%d:ulaw);
	same => n,Goto(rtt233_beat_withRecorded,s,1)
	same => n,WaitExten(30);
 
 
[rtt233_beat_withRecorded]
exten => s,1,Background(/home/rtt233/asterisk_sounds/beat, m)
	same => n,Playback(${RECORDED_FILE})
	same => n,Goto(rtt233_beat_withRecorded,s,1)
	same => n,WaitExten(30);

Voicemail with Asterisk

rtilton1 — Mon, 06 Feb 2012 19:13:44 +0000

Setting up voicemail on Asterisk is surprisingly simple! After I wrote the dialplan I was shocked it actually worked. In my extensions.conf file I used the below commands:

[rtt233]
exten => s,1,Wait(1)
exten => s,n,Playback(/home/rtt233/asterisk_sounds/sound_tester_hello)
exten => s,n,Goto(rtt233_easy_voicemail,s,1)
 
[rtt233_easy_voicemail]
exten => s,1,Voicemail(10@voicemail_rtt233, u)
exten => s,n,Hangup();
;voicemail will go the a extension if * is hit during voicemail app
exten => a,1,VoiceMailMain(10@voicemail_rtt233)
exten => a,n,Hangup();

In my voicemail.conf file I used the below commands:

[voicemail_rtt233]
 
;extension_number => voicemail_password,user_name,user_email_address,user_pager_email_address,user_option(s)
 
10 => 10,Robbie Tilton,rtt233,rtilton1@gmail.com,,attach=yes|serveremail=rtt233@nyu.edu|tz=eastern|saycid=yes|callback=frmvm}operator=no}envelope=yes

You’ll also notice that I called an audio file in the Playback() application noted in context rtt233. This audio file was created using Audacity with the file settings: 8000Hz, mono, and 16-bit. I exported this file as a .wav and uploaded it to my asterisk_sounds folder in which asterisk read it and converted it into a ULAW or GSM audio file. With a compressor I could have also done this conversion on my own, but with small files this is not necessary.

Chat Roulette – Webcam Hacking

rtilton1 — Fri, 03 Feb 2012 14:28:10 +0000

In Kyle McDonalds appropriating new technologies class we were asked to use face tracking to mess with chat roulette webcam streams. For my project I create a Jack Nickleson mask that overlays on top of any face recognized by the webcam. I’m using the openCV HaarFinder openFrameworks library to calculate the face tracking and am using CamTwist to insert my application into my webcam stream. In order to capture the users webcam video and perform the face tracking and mask addition I used code zach Liebermans posted that allows for the oF app window to read whatever is underneath it. The orginal source can be found here: http://forum.openframeworks.cc/index.php/topic,2946.0.html

It was a lot of fun to see a product of mine out in the wild with real-time video, audio, and text. People in groups often had a stronger reaction to the masked face than individuals alone. In the future I’d like to come up with a stronger conceptual concept to work with this technology.

I’m not going to share all my code on this post, but you can find it on my Gitub if you want to download it: https://github.com/rtilton1/AppropriatingNewTechnologies. Below is the core of the app in my draw loop.

    imageCopy.draw(0,0);
 
    for(int i = 0; i < finder.blobs.size(); i++) {
        ofPushStyle();
        ofFill();//make color fill
        //ofSetColor(255, 0, 0);//make red
        face.draw(finder.blobs[i].boundingRect.x-13, finder.blobs[i].boundingRect.y-23, finder.blobs[i].boundingRect.width+35, finder.blobs[i].boundingRect.height+30);
		//ofRect(finder.blobs[i].boundingRect);
        ofPopStyle();
    }