Progress work of AdShades (Visual Simplification System)

The project idea for AdShades had been floating around my head for a while. I have always wanted to do augmented reality projects that deal with the control of advertising. In order to do my project I had to do a lot of planning and some experimentation to finally get to the final product. The project started with the research for the correct programming language. I was looking at Jitter and Processing and after looking at libraries, I found that JMyron would be great for my project. I read in the reference that there was a function called getGloBoxes() as well as trackNotColor(), and there I had found the exact thing I needed.

The second phase of the project involved the coding and driver issues of the camera. Again after more research, I found that the ps3 eye camera is great for computer vision, but the drivers are created by some guy and are not official. After a lot of playing around(experimenting on 4pcs and a mac), I found my project would only run on windows xp 32bit(with the inclusion of exta .dll's from the internet). This is why I borrowed by friends laptop that I then broke. I was very active on online threads talking with a guy named liudr a lot. The main thread is at http://processing.org/discourse/yabb2/YaBB.pl?num=1259097615 He helped me get the video running at 640 x 480 at 30fps.

The programming is relatively simple based on the great yet old library JMyron.
import JMyron.*;

JMyron m;//a camera object
int[][] globArray;

void setup(){
size(640,480,P2D);//P2D allows the sketch to run at 30fps
m = new JMyron();//make a new instance of the object
m.start(width,height);//starts the capture
m.findGlobs(1);//glob finding on
m.minDensity(10); //the required density of ads
loadPixels();
}

void draw(){
color gal = m.average(0,0,width,height);//a function that averages colors
m.trackNotColor(int(red(gal)),int(green(gal)),int(blue(gal)),150);//trackNotColor finds everyhting that is not the surrounding color
m.update();//update the camera view
m.imageCopy(pixels);//draw image to stage
int[][] b = m.globBoxes();//get the center points
updatePixels();

noStroke();
globArray=m.globBoxes();
for(int i=0;i< .75*width && (b[i][2] > 15)){
int[] boxArray=globArray[i];
int currColor = m.average(
boxArray[0],
boxArray[1],
boxArray[0] + boxArray[2],
boxArray[1] + boxArray[3]);
fill(red(currColor), green(currColor), blue(currColor));//All of this fills each shape with its average color.

rect( b[i][0] , b[i][1] , b[i][2] , b[i][3] );// the rectangle boxes
}
}
println(frameRate);
}

public void stop(){
m.stop();//stop the object
super.stop();
}

The next step of the project was to build the mount and aquire the video glasses. There was a lot of RMA issues as you know. The mount for the video camera was made with plexi and the laser cutter to make a pretty professional mount. I also bought multiple video adapters and ended up having to break a cable apart and create a proprietary s video to composite cable. I also had to program a mouse with hotkeys so I could start the animation, start a screen capture, and stop the capture. I however need to find a much better screen caputre program because I could only get 15 frames per second in an with an old codec.


The performance of the piece was amazing. I went to the oaks Mall and was very surprised by the results. It was almost perfect. The screen capture program does not show how great it looked and the feeling of having your vision actively changed. I wish the screen caps were better. Here are some of my favorite scenes.

From left to right: Oaks Mall Entrance, Sale Poster
Sleep Number Store, Victoria's Secret.

I was really happy with the reading in the difficulty of blocking advertising, and that you are more aware of it if you have to think about blocking it. It was also fun to talk to some people about the project, which I would like to integrate into the future performances of the piece.

For the future of the piece, it has been accepted into the Creativity of the Arts and Sciences, so I will be working on it a lot more. I need to figure out an easy way to have a wearable computer with a processing sketch and screen capture running. I am currently trying to buy a laptop that can easily run the program and have good TV out.