The visuals and audio distortion were done with openFrameworks/C++. We used 7 Plasma screens and 7 mac minis daisy-chained together through audio-in and audio-out cables. We used the ofxAUPlugin to create an auditory delay to our audio-out jack and Kyle McDonald’s cylinder mesh screen-cast to inform the 3D wave-form. Throughout this process Johann and I learned a lot about particle systems, wave-forms, and communication between multiple machines.

Traditional conference rooms often come with frustrating user experiences where many callers can’t seem to get in, missed the invitation, or can’t seem to find the e-mail where the call details were provided. With Rehuddle, users are given several super simple ways to share: 1) click to automatically copy the call details to your clipboard 2) click to auto generate an e-mail with the call details 3) text message someone to join the room 4) share the room via facebook and 5) generate a call that places a caller directly in the room (no extension needed).

The site is built with Ruby, Asterisk, Javascript, Node.js, and MySQL. Each user that joins a “huddle” is provided an avatar based on the last four digits of their phone-number. This allows each user to be uniquely identified and understand who is on the call. The avatars are currently being generated using Node.js and Underscore.js. Node is being used to ping our mySQL server every second and pipe the results into a socket that the client can read by parsing JSON. Underscore.js is being used to compare the current list of callers with a previous list of callers. This allows the client to know which callers to add for each browser and which callers to remove. Each conference room also gets a unique URL. If a user creates a room called “Sony Meeting” – Rehuddle will auto generate a URL Rehuddle.com/sony-meeting where the call can live. This url generation is being done with Ruby utilizing Sinatra and Datamapper/MySQL. Within the room our copy to clipboard share feature is using a JQuery library called zClip. The SMS messages are being routed through a Twilio’s Ruby API, the Facebook share call-to-action is using the standard Facebook Developers share button, and the call generator is happening in Asterisk, but is routed through Flowroute through a SIP registry.

In future iterations Phil and I are looking to add document sharing features, editable name tags for each avatar, and potentially unique phone numbers for each room. We both feel that there’s a huge gap in the market for conference systems that are simple, easy to use, easy to share, and provide an informative and enjoyable web interface. We both look forward to more user testing to see where this product takes us. A big thank you goes to Chris Kairalla for teaching us Asterisk and helping us debug throughout the build!

Speech recognition for Asterisk can be found here. Once flac and lib-perlwww are installed on your server all you need to do is grab his gitHub files and place speech-recog.agi in your share/asterisk/agi-bin folder. This will allow you to run commands like the ones below:

exten => s,1,agi(speech-recog.agi,en-US)
exten => s,n,Verbose(1,The text you just said is: ${utterance})
exten => s,n,Verbose(1,The probability to be right is: ${confidence})
exten => s,n,Hangup()

For speech synthesis – a very similar process is required. With flac and libwww-perl already installed on your server you’ll just have to grab his github files and place the speech-recog.agi in your share/asterisk/agi-bin. This will then allow you to run Asterisk commands like the ones below:

exten => s,1,AGI(googletts.agi,"This is a simple google text to speech test in english.",en)
exten => s,n,AGI(googletts.agi,"hello\, I am a talking computer",en)
exten => s,n,wait(1)
exten => s,n,Hangup()

This week I started reading Wired for Speech by Clifford Nass and Scott Brave. They discuss the human-computer relationship in regards to speech and have conducted many interesting tests to prove theories. Based on their studies humans have affiliations with computers that can speak which directly affects how they judge them and feel about them. For instance, a computer with a feminine voice who is teaching a lesson is more likely to be thought of as emotional and less strict than a computer with a male voice (who teaches the exact same lesson). Personalities are also generated based on how feminine or masculine a computers voice is even when synthetic.

Regarding the reciprocation of love to a computer – in December 2009 – a human for the first time attempted to marry a fictitious character from a video game. The character married was from a Nintendo DS game called “Love Plus.” Sal, the human, stated “I love this character, not a machine”…”I understand 100 percent that this is a game.” But does this love come from an addiction and are these feelings as ‘authentic’ as the love one feels for a human? Hiroshi Ashizaki, an author on internet and game addiction thinks that Sal’s relation is healthy and that the youth is feeling more and more comfortable expressing their true selves through virtual worlds rather than the real world.

Overall, there are a lot of mixed perceptions on this topic. With an experiment of my own I’m looking to help shed some more light on the topic (experiment to be announced when results are collected).

The node code is as follows:

var agi_net = require('net');
var remote_net = require('net');
 
var REMOTE_PORT=12002;
 
var AGI_HOST = '127.0.0.1';
var AGI_PORT = 12001;
 
/**
 * AGI will only send key presses, and that's it.
 * There's no identifying characteristics, just raw press events.
*/
 
/** Remote clients send no info to this server.
 * all they do is receive byte representations of digits.
 * (0-9, #, *) on the phone's keypad as ascii bytes*/
 
//remote screens or physical objects
var remoteClients = [];
remote_net.createServer(function(sock){
  console.log('CONNECTED REMOTE CLIENT: ' + sock.remoteAddress +':'+ sock.remotePort);
  remoteClients.push(sock); //add client
      // Add a 'close' event handler to this instance of socket
    sock.on('close', function() {
        for(var i = 0; i < remoteClients.length; i++) {
            if(remoteClients[i] == sock) { //remove client
                remoteClients.splice(i,1);
                break;
            }
        }
        console.log('DISCONNECTED REMOTE CLIENT: ' + sock.remoteAddress +':'+ sock.remotePort);
    });
 
    // Add a 'data' event handler to this instance of socket
    sock.on('data', function(data) {
        for ( var i = 0; i < data.length; i++){
            handleByte(data[i]);
        }
        function handleByte(buf){
         console.log(buf);   
        }
 
    });
}).listen(REMOTE_PORT);
console.log('Server listening for remote connections on ' + REMOTE_PORT);
 
agi_net.createServer(function(sock){
  console.log('CONNECTED AGI CLIENT: ' + sock.remoteAddress +':'+ sock.remotePort);
      // Add a 'close' event handler to this instance of socket
    sock.on('close', function(data) {
        console.log('CLOSED AGI CLIENT: ' + sock.remoteAddress +':'+ sock.remotePort);
    });
 
    // Add a 'data' event handler to this instance of socket
    sock.on('data', function(data) {
        console.log(data);
        //broadcast data to all remote clients
        for (var i = 0; i < remoteClients.length; i++){
            var client = remoteClients[i];
            client.write(data);
        }
    });
}).listen(AGI_PORT, AGI_HOST);
 
console.log('Server listening for AGI connections on ' + AGI_HOST +':'+ AGI_PORT);

The AGI script is as follows:

#!/usr/bin/ruby
 
#very simple implementation of asterisk -> server -> client stack
#only sends the digit and the caller id.
#designed for very simple control, like arduino or web page
 
require 'rubygems'
require 'ruby-agi'
require 'socket' 
 
agi = AGI.new
host = 'localhost'
port = 12001
looping = true
s = TCPSocket.open(host, port)
agi.stream_file("vm-extension")
while looping
    result = agi.wait_for_digit(-1) # wait forever
	if result.digit
        s.write "#{result.digit}"
	else #hangup broke the pending AGI request
        looping = false
    end
end

The asterisk dialplan is as follows:

[rtt233_socketIO_arduino]
exten => 1,1,Answer();
exten => 1,n,AGI(/home/rtilton1/asterisk_agi/socketIO_arduino.rb);
exten => 1,n,Hangup();

All code examples and knowledge are curtesy to our awesome professor Chris Kairalla! I’m excited to continue on node and download the free book/pdf found here: http://visionmedia.github.com/masteringnode/

My Arduino code is noted below:

//state constants
#define HTTP_ACTIVE 10
#define HTTP_IDLE 20
#include <SPI.h>
#include <Ethernet.h>fa
 
//setup          70    A5    DA    00    85    70
byte mac[] = { 0x70, 0xA5, 0xDA, 0x00, 0x85, 0x70 };
IPAddress server (XXX,XXX,XXX,XX);  // my server
//client stuff
char httpCommand[] = "GET /~rtilton1/sinatra/arduinocall/?callnumber=19174365310 HTTP/1.0";
EthernetClient client;
byte httpState = HTTP_IDLE;
int httpDelay = 5000; //wait at least this long in millis before sending another request
unsigned long connectTime = 0;
 
//button stuff
const int buttonPin = 2;
int buttonState = 0;
 
void setup() {
    Serial.begin(9600);
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    for(;;)
      ;
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("ready.");
}
 
void loop() {
  //read input from server, if any.
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }
  //check button, only read if there are no active http connections and we're past minimum delay
  if (httpState == HTTP_IDLE && pastHttpDelay()) {
    buttonState = digitalRead(buttonPin);
    if (buttonState == HIGH) {
      serverConnect();
    }
 
  }
  //close client if disconnected
  if (!client.connected() && httpState == HTTP_ACTIVE) {
    client.stop();
    httpState = HTTP_IDLE;
    Serial.println();
    Serial.println("disconnected.");
  }
}
 
boolean pastHttpDelay(){
  if ((millis() - connectTime) > httpDelay) {
    return true;
  }
  else {
    return false;
  }
}
 
void serverConnect(){
  Serial.println("connecting to stu.itp.nyu.edu");
  if (client.connect(server, 80)) {
    httpState = HTTP_ACTIVE;
    connectTime = millis(); //when did we connect?
    Serial.println("connected");
    client.println(httpCommand);
    client.println();
  }
  else {
    httpState = HTTP_IDLE;
    Serial.println("connection failed");
  }
}

With the software which you can download on Github here- it comes with several standard applications – one of which is a keyboard that allows users to type. For people like graffiti artist Temp, this is a truly amazing way to communicate to the outside world which once was not thought possible. When viewing this keyboard (shown on the left – second image from the top) I initially felt there were several aspects of the keyboard that could be improved upon:
1) The rollover/click state is indicated by the button fading into a saturated green. This fading affect doesn’t give the user much of understanding to the amount of time required or remaining for the click to initiate.
2) The window is jam packed with buttons and thus doesn’t give the user much space to rest there eyes after they typed or are waiting for a response. Because of this there is also a limited amount of space where text is displayed. I wanted to create a text area where more sentences (or even paragraphs) could be typed.
3) The IA felt like there could potentially be some consolidating of features or even use of non screen real-estate. Potentially numbers, shift, and delete functions could be removed from the main key area and placed on the external limits of the screen.
4) I felt that typing required an increased up and down motion of the eyes to check what has been typed when it could potentially be displayed infront of a users eyes at all time.

With these problems in mind I created a design with the large black typing area and a green chalkboard keyboard selection area. Overall I felt this redesigned keyboard organized the buttons in a more intuitive manner and allowed for clearer click state (with a black character filling effect), a cleaner work space with a single row navigation, and a reorganized UI with non key letter buttons (shift, numbers, delete) placed outside the screens width. With this Apple Dock appropriated design finished – I seeked to create an app that would fulfill this image. As it proved, the Apple Dock lens effect was more difficult to code than I anticipated. After struggling to get the dock to function as I wanted, I decided that I needed to get something functional in code to begin user testing to determine if this interface actually worked or not.

When I began my tests I found that the standard keyboard that comes with the eyeTracking software performed fairly well even with it’s minimal design and button filled screen. In comparison to a linear alphabet (similar to the one proposed in my design) – I could not type as fast. The results from my tests are below:
Standard Keyboard
Font-family: Helvetica Neue Med
Font-size: 16
Button Dimensions: 100×100
Type speed (“HELLO”): 17 seconds
Errors: 4

Single Line keyboard
Font-family: Hand Times
Font-size: 48
Button Dimensions: 50×50
Type speed (“HELLO”): 57.5 seconds
Errors: 5

Font-family: Hand Times
Font-size: 68
Button Dimensions: 70×70
Type speed (“HELLO”): 49 seconds
Errors: 4

Font-family: Hand Times
Font-size: 90
Button Dimensions: 90×90
Type speed (“HELLO”): 1:12 seconds
Errors: 1

After reviewing my results I found that at a button size comprable to the standard keyboard I was able to achieve the same results with fewer errors, but it took me much longer to achieve this. So is this a viable keyboard or not? For now I think it is not the correct solution, but I do believe the error reduction is something that can be learned from in my next iteration. Without a rowed approach it is more difficult to hit the wrong key. I’m also interested in the Dasher (bottom image on the left) software that Kyle and Greg showed me which claims to achieve 29 words per minute with eyetracking software. Perhaps the predictive text route is the way to go…

Lastly, I’d like to thank everyone who helped me throughout this project. ITP is truly a wonderful place where magic happens, but the support of everyone on the floor helps to keep everything flowing. I plan to continue this research to one day have a better eyeTracking keyboard for the physically handicap.

Audio was added in post-production.