Semester Proposal - Alberto Semper

https://prezi.com/2o13mm2rmty_/semester-project/

Final Semester Project

As I had suspected, the "final" presentation of this work still felt very much in-progress. Most of my worktime was spent wrestling the circuit itself, working with Charlie, Jason Huang, and eventually Luis Cruz just to get a usable signal. For the time, this signal was simply not going to work REM eye movement -- the signal was too fast and small for this circuit which is why labs have the nice $3,000 amps, I suppose.

Working so long on getting a signal left me choked for time programming in the game engine to create an .exe to interface with the signal. The current build I had of the labyrinth just didn't really show much with the signal I had so I did a quick makeover to a simple object based programming that moved sprites left or right according to the signal.

For the presentation of the project I took a more performative approach. I hooked up James in the computer lab to the electrodes and gave him instructions to follow my finger with his eyes, shined a light in one eye at a time, etc. At this time the circuit did not work. There are many possible reasons, but the most prominent were the potential additional noise present in the electronics of the computer lab, James was actually sweating a little and that can have a big impact, or some other unforseen change between my testing at home and then running it in the lab. As a side note, the circuit works best with very strong eye movement in one direction or the other, and works the most clearly when you close one eye or the other. James was biologically unable to close his left eye without closing his right eye, so that didn't help either. It's okay James; maybe your children won't inherit that gene so I can experiment on them.

Running it as a performance, shoddy as it was in the setting and context of time, definitely gave me some great ideas on how to tweak it as I lock down the signal.

Overall, dissatisfied with the current state of the project as a work of art, but as a progression toward something greater I am very happy to have a lot of the groundwork laid out. With the elements I have it is just a matter of spending more time calibrating the circuit, adding filters, and working on the executable.

Here are some photos taken by Kelsey while I subject James to weirdness:

Attached to this post is the .exe
EOG.exe


Arduino Leonardo Code:
//read electrode signal on pin A0
int analogpin={A0};
int threshold=500;

void setup() {
  // initialize serial communication at 9600 bits per second:
  Serial.begin(9600);
  //start keyboard library
  Keyboard.begin();
}

void loop() {
  // read the input on analog pin 0:
  int sensorValue = analogRead(A0);
{
//If the electrode signal has a high voltage, the wearer is looking Right.
if (analogRead(analogpin)>threshold)
{
  Serial.println("RIGHT");
  Keyboard.write(0xD7);
}
//If the electrode signal has a low voltage, the wearer is looking Left.
if (analogRead(analogpin)<threshold)
{
  Serial.println("LEFT");
  Keyboard.write(0xD8);
}
//Otherwise play static animation; this will likely never really happen in this instance.
else
Serial.println("STATIC");
}
delay(500);
}

process work

So last night, we ran into a big problem with our project. It worked how we wanted it to Sunday night, but while doing some little last minuet touch ups to the installation and a final run through test, the code would not up load to the boards. It's really bizarre. We even tried uploading the basic Blink code and it would not work. It kept giving us this same error code:

I googled this error code and everything I found suggested reseting the bootloader and reseting the driver. I tried following tutorials on how to do this, but it is pretty far out of our knowledge and skill base.

We changed every variable we could thing of. We changed usb cables, uploaded from different computers, used a PC instead of our macs, used fresh new arduino boards, and used different codes with the new boards. But I think we have determined it is our code. We figured this out by taking Ivette's arduino board and uploading the simple Blink code to it, and then trying to upload our code and we got this same error code. We have no idea how our code could have gotten corrupted with in the past 24 hours since this is the same code we have been using for weeks. We have looked over our code and retyped it into a fresh sketch window and tried uploading it and it still produces this error. Right now we are trying to update the arduino program, type up the code in a new sketch window, and upload it to a fresh board. We shall see if this works.

UPDATE:  It worked. We had to update the Arduino program on our laptops to the latest version which was just added on the 15th. (What a coincidence.) So we updated it on both of our laptops, and had to retype the code into a new window and resave it and THEN we could upload it to the arduino board. And we also had to use brand new fresh boards. I think the boards that had our old code without the update were corrupted by the code somehow.

Here is a video explaining our project a little more with both sensors and all motors working.

Process work

Did not work :(  even when plugging the 9 volt to it. We even checked it with a multi meter and it was giving 4 volts which we concluded should be enough for each motor. This circuit only has two motors soldered to it in to save ourselves a little stress and time. The circuit is soldered to hold 12 motors in total

But we took it to a different outlet to see if that was the issue. But I burnt my finger on the board instead.

EOG Amplification Circuit Fritzing Diagram

Ultimately, the Notch and OpAmp circuits never worked correctly. For this semester I will be working with the EOG circuit amplified through the Instrumental Amp IC INA118P. This signal is a bit noisey but does still distinguish left eye movement from right eye movement.

This circuit is not sensitive enough to detect something as small as REM, but is a great starting point for exploring EOG Arduino interface because IT WORKS.

The power supply should be 12V 1A for the L7805s I am using, but the only adapter I have that works with the circuit correctly is 2A. Ideally, the L7805s would also be a different variation of the same regulator, LM7805. Theoretically there is no difference, but in reality I do get an output voltage of 8.6V when it is supposed to give me 5V from one regulator, and a 3.3V from the other. Something here isn't working correctly, but I've studied the circuit intensely and following the instructions for it soundly so my suspicion is either the 12V 2A adapter is effecting it, or the difference between LM7805 and L7805.

The L7805 circuit takes the 12V power supply and regulates it down to a smooth 5V. The reason we are using two supplies is to drive the differential amplification between the two electrode inputs. Thus a power supply ground is created from the 12V supply, and a second, virtual ground, is made from the output voltage of the second regulator.

The INA118P takes the signal input and amplifies it with a gain of +500 (giving us a net gain of +501 to the signal). We use a 100ohm resistor to set the gain. There is a chart on the INA118P's datasheet that shows how to set the gain with different resistors.

When we move our eyes the electrical signal from the skin polarizes the corresponding electrode and raises or lowers the voltage. A GND electrode is connected to the forehead and to the virtual ground of the 7805 series. This signal is output as a single voltage to the Arduino's Analog 0 input and the ATMEGA328P converts it into a digital value we can then use in programming (ADC).

FINAL PROJECT: Mixed Signals - Jen Herrera

-MIXED SIGNALS-

Photos:

       

Video Documentation:

Description:
This piece is about interspecies communication/human-animal communication. The interface is a handmade cat plush with an LED matrix head. Inside the head is where the arduino/protoshield is housed, and a long-life battery is housed in the body in a hidden zipper pouch. Embroidered into the plush's back are rows of conductive thread that serve as one big capacitive touch sensor to trigger the matrix. At first I thought of programming the matrix with 'faces' or recognizable symbols like letters or numbers, but then decided it might be too cliche and too embedded in human communication to be animal communication. I chose to instead create abstract compositions that would be displayed when the cat was touched, the thinking being that they would invite interpretation by the viewer, just as animal communication invites interpretation. In critique, one of the first things my classmates began discussing was whether one reaction was a 'good' reaction or a 'bad' reaction, so I was really pleased with that! This lends to my concept being that whatever the animal's intention behind it's communication, we only take away our human interpretation of it.

Process Posts:
[1] [2] [3] [4] [5] [6] [7] [8]

Code (through 'read more'):

Semester Project Log

Circuit Iterations 1, 2, and 3 do not give desired results. Nothing useable attained.

Met with Jason Huang, Biomedical engineer undergraduate. He lended me an Analog Discovery device and I am using Waveform to view my signal with its oscilloscope function.

Documentation of components and voltage as I build the circuit once again from scratch, checking each component's voltage as I move along.

1. 12 V wall-plug transformer to barrel plug power supply.
   Transforms household 120v to 12v.
   Multimeter is actually reading at 18.46v.
   Barrel plug reads 18.46v.
   Changed to a different "12v" 1A adapter aaaannnnd.....reading at 6.07v. This won't work either...goes to show checking your equipment with a multimeter is extremely valuable.
   Luckily...had a third "12v" AC/DC transformer and finally 12(ish)v!
2.  7805 Voltage regulator (2)
   Both regulators read out at 0.054v from 18.46v input.
   After switching to the actual 12v adapter I get 0.23v which is a very significant difference.
3. All gold-cup electrodes have about 1.5ohm resistance from cup to terminal.
   Left and Right electrodes tested positive for continuity to pins 2 and 3 to INA118.
   Through Analog Discovery + Waveforms successfully got a signal but currently is too noisey -- which is the point of the filters. At this point I only have the 12v supply running through the regulators and into the INA118. I get a readout that fluctuates between just under +5v and a little bit over -5v which is pretty ideal from what I understand.
4. Found a bad resistor in the Notch Filter circuit. a 3.3k ohm resistor was reading at nearly nothing...0.03 ohm. Signal read on WaveForms from LM358 filter are low and inconsistent. Uncertain if this is "normal" before the LM741 IC integration.
5. Signal unusable after LM741 integration. Reverting back to sampling the INA118 with the rest of the circuit built also yields significantly different results at before.
6. Arduino Paul Uno is no longer responding to any serial input/output after swapping things around trying to get a sensible value. RIP Paul.

5th Attempt:
I feel I understand the circuit significantly better now, and as such I am attempting the 5th build from this schematic:
http://onloop.net/hairyplotter/images/final_eog_ckt.png

This schematic is more intended for Arduino Uno. I didn't understand this person's schematic at first so I avoided it but now it makes more sense to me having built the very similar one presented in the MAKE Magazine article.

1. Power Supply to Regulators
   12v DC power confirmed from power supply. 7805 regulators confirmed connected by the right gates,
   7805 (a) 1 --- connected to 7805 (b) confirmed
   7805 (a) 2 --- to Power Supply GND confirmed
   7805 (a) 3 --- to 7805 (b) 2 (GND) confirmed
   7805 (b) 1 --- connected to 7805 (a) confirmed
   7805 (b) 2 --- to 7805(a) 3 confirmed
   7805 (b) 3 --- V+ out to INA118 pin7 confirmed
2. INA118 InstrumentationAmp (+500 gain)
   11.2v between pin4(GND) and pin7(V+ input)
   pin1 to pin8 connected by 100ohm resistor (+500 gain)
   pin7 V+ output to LM358 pin8 Notch Filter
   pin6 passes through parallel cut-off circuit to filter out high frequencies; this parallel reconnects to pins 2 and 3 of LM358 as positive and negative voltages.
3. LM358
   Filters noise from the amplified V+, V- signal and outputs it through pin6 toward the final amplification circuit LM741
   Cut-off circuit connection test:
   R2 connected to C1, C2, C3, and R3
   R3 connected to C1, C2, C4, and R2
   R4 connected C1, C2, C3, C4
   pin2 of LM358 connected C1, C2, and R4
   pin3 connected to R2, R3, C3, and C4
   pin6 to resistor, capcitor(to GND), and onward to LM741 pin3 as V+
4. LM741 final amplification IC
   V+ signal from pin6 of LM358 with a difference of the virtual GND
   Output voltage follows the input by connecting V- (pin2) to Vo giving the analog signal.
   
   When I hook up the Arduino Analog 0 to pin 6 of 741 I get the same arbitrary drifting of values as before.
   
   Received an e-mail from Luis Cruz himself this morning, giving me access to the Google Drive Eyeboard documents. After several more builds from this guide I came to a temporary solution. I bypassed the notch and opamp, getting the signal from the instrumental amp (118P) alone and I have a useable signal for the purposes of this project. It is still cluttered and takes a strong eye movement to detect but the values do indeed rise with right eye movement, drop with left eye movement, and stay relatively steady when stationary.

12/11 - Semester Project Process post #8 - Jen Herrera

The interface(plush) is currently being built! It's about 60% there I would say. The overall project I would estimate at 80% finished, with a few days left to go.

The housing for the LED matrix and the arduino:

The hidden zipper compartment for the extended life battery:

Embroidered 'touch' area made of conductive thread:
(seen from behind)

As for the content of it, I decided to follow through with my last concept iteration, in which I decided that the matrix should not display any recognizable words or symbols, but should be made of abstract compositions that might invite interpretation. In this way, even if the viewer gathers meaning from the 'cat''s reaction, it is merely the human interpretation of animal thoughts, just like in real life.

Since I wrote about that concept, I have put together 4 compositions that will be displayed on the matrix, one each time that the cat's back is pet.

Compositions:

I think I'm doing well with the time that's left. I just have to keep on sewing and get this thing assembled! :)

12/3 - Semester Project Process Post #7 - Jen Herrera

I spent almost three hours soldering this protoshield for myself. My LED matrix previously had 16 jumper wires dangling out of it, and it was causing problems when they came loose. SO now that I have this thing all set up, it's basically plug-and-play. It kinda limits my arduino to this project only when it's on, but that's alright. (shouldn't have put headers in all the pins I wasn't using or should have soldered more slots for wires...oh well. )

Anyways, it works!

One thing I had a huge problem with today was trying to figure out how to make the images on my matrix randomized when the sensor is touched... I really want to make 3-5 compositions, and have the code randomly select one to show when the plush is touched. I'm still currently struggling with this as my biggest obstacle. I'm hoping to have the plush body interface finished this weekend, so then all I should have to do is figure out the random images.

11/30 - Semester Project Process post #6 - Jen Herrera

At this point in time, I'm into the making of my project.
The plush's body is a work-in-progress; the body pattern is drafted and the head pattern will be finished soon.

More important than the physical is the electrical; I have all the pieces I need now. My usb power supply arrived today:

It's pretty large, so I need to make sure that I leave room for it in the final design. I'm looking forward to the fact that my project will be self-contained though! For the matrix, I just need to solder up a protoshield with its various connections so that it will be a plug-and-play sort of function. I have the protoshield in my mailbox and found a tutorial on soldering it for the matrix:
https://learn.adafruit.com/32x16-32x32-rgb-led-matrix/connecting-using-a-proto-shield

For my concept, the content of the matrix when the plush is touched is the most important thing. Last mini-crit, it was suggested that I look into animal communication research and I thought that maybe I would find some imagery to use in there. After researching, I found that the most interesting part of the articles were sections that said 'The cat was very concerned with the owner's emotions, researcher said.' In the end, we can observe interspecies communications all day long, but we are still making conclusions about them as humans, and are never able to fully understand how another animal is interpreting the situation.

Using this idea, I decided I'm going to simplify things and use the mechanics of the matrix to my advantage. It draws shapes very much like processing does, so it will be easy for me to draw various compositions of line and shape. Like abstract painting, it will be up to the viewer to try and interpret the shapes; or otherwise not try to interpret them. These composed reactions will be randomized in the order they show up, how long they last, and whether or not they blink. The 'emotions' of the 'animal' will remain a mystery, even if the viewer thinks that they have figured it out. 

11/18 Semester Project Process post #5 - Jen Herrera

After our prototype crit the general consensus was that my plush should be more realistic to better communicate interspecies communication. For this, I ordered some faux fur fabric to construct the toy out of. My main concern with using long fur was that a patch of conductive fabric, like I meant to use with short plush fur, would look like an ugly bald spot in the middle of the plush.

That's why for my second mini project I was investigating using the conductive thread for capacitive touch. My theory was that if I could sew rows into the fur, parting it where I sewed and then brushing the fur back over it, that I could get capacitive touch action that was nearly undetectable to the eye.

My fur came in this week and I got to work testing; good news, with a very low threshold (to make up for the lack of direct contact due to the fur) it works!

Video:

I'm super thrilled about this. I think it will look SO much better than the short pile fabric with patches of rough, unpleasant conductive thread. Concept wise, I love that the signal comes from just touching the fur of the 'animal' with no visible interface. It'll be so much closer to petting a cat and getting a reaction out of it!

I also found a free-to-use cat pattern in the general shape I was going to design one in. So I'm going to save myself a little time and just tweak that pattern since it's nearly an exact image of what I want.

Final Project 2 Process

Here are the supplies we have for our final project. Peg board, DC motors, and plastic tubing.

We have two different sizes of tubes.

The thicker tube with be placed in the black spaces of the board to fill space and give volume to the piece. The smaller tubing will be placed on the DC motor like so.

Every hole will have a DC motor placed through it like this. the wiring and Arduino boards can rest on top of the peg board. We will get a tiny prototype built soon.

11/9 - Semester project Process #4 with Protoype

My first go at a prototype:

In this test I got both the LED Matrix to work, as well as a conductive fabric capacitive touch sensor. It took a lot of research just to hook up the fabric, and once it was done I had to mash a bunch of code together. I ended up finding some code on how to make a capacitive touch piano out of tin foil. I also took the code to draw on the matrix from the library that I downloaded for it (It's looking surprisingly like drawing shapes in processing!). I cut up the two codes and mashed them together so that instead of playing a note when the 'key' is hit, it draws to the matrix.

Some concerns of mine are that this is definitely an arduino uno project and there are a ton of wires flopping around, sometimes getting loose and being unpredictable. I believe I'll have to put together a proto shield or something like that to make running the matrix more stable. As for the fabric sensors, I still want them to be sewn in with conductive thread, so I will have to experiment with securely fastening resistors into the thread and a way to turn the thread into a 'wire' at the end... maybe some sort of tie-in plug or pin for when it comes to attaching the arduino.

There's a lot of problem solving to do yet, but I'm happy with my progress so far.
I still need to ponder further the meaning or content of what will show up on the matrix, but I just wanted to get it working first!

11/8 - Semester Project Process post #3- Jen Herrera

I got my LED Matrix up and running! It definitely has to be driven off of the Arduino Uno, so I'm re-thinking my setup for my project. I really want to use the lilypad for the conductive fabric and thread, but now I'm not sure how the two arduinos will be able to talk to one another. I might have to look into making the fabric work with the Uno instead.

Either way, I want the project to be self-contained in the plush toy, which leads me to the power source realization. For my smaller lilypad projects, I was using the 3V battery that came with the protosnap set. The LED matrix definitely needs a 5v power supply that can draw up to 2A. There are regular battery packs that can accomodate this, but I'm worried about the life of them. My 3v battery died surprisingly fast with just my small blink LEDs for mini project 1. I found a battery pack meant for powering tablets and pis that I think will do the job.

Now to finish up my prototype!

Process: Real Time Clock

For my project I am creating a synthesizer...visual synthesizer

In order to do this one of the components is creating a real time clock.
Which I have done and documented below....

Research Day - Enterface: somnwhere poster

10-21 Semester Project Development

This week I received a hot/cold sleep gel mask :

This one.

Also I received the gold-cup electrodes for reading electrical signals from the eyes. In order to fully implement these sensors I still need to solve the amplification issue, but now that I have them in my hands testing can begin.

Our work on serial controls with Arduino has been enormously helpful towards my semester project as the Arduino Uno will be serving as an interface between the mask and the visual software in development.

On the software side I am currently working with a few different programming languages (STUPID) to solve what I would like to be the final delivery method for the visual interface. Unity is the most practical model but I am unfamiliar with its toolset so Unreal Engine V or Game Maker Language are more viable options. I currently have a working prototype in GML so that's my safety net on the software side.

The other option I am looking into is 6502 Assembly which is an old programming language used in NES programming (Apple II, Commodore 64). It's an 8-bit hex based language. Difficult and out of date language, but I found some good information on loading the programming into donor boards of NES carts giving the software its own circuit board to live on.

The bit shift makes me wonder if there is another route entirely to visualizing eye movement. The directions can be assigned to physical outputs using the Binary code...this would eliminate a need for external software which is appealing.

This simple iteration of a first-person labyrinth is fascinating in its slow-update time. For clarity purposes I also am attracted to the black and white aesthetic. Things to consider. 

10/25

Just found this awesome DIY Electrode guide. Happy to be using the real stuff a la Katerie Gladdys and our awesome lab fees being put to use, but thought this was really interesting.

http://eeghacker.blogspot.com/2013/11/homemade-passive-electrodes.html

Pretty important PDF detailing EEG, EOG electrode placement.

http://www.n2sleep.com/Portals/408/Skins/PB-CYU/files/ELECTRODE%20PLACEMENT%20SYSTEM%20FOR%20SLEEP.pdf

https://www.youtube.com/watch?v=pZBk3zQtVB0

Amazing polysomnograph placement text:

https://books.google.com/books?id=CdXFG_tKmoEC&pg=PA317&lpg=PA317&dq=EOG+placement&source=bl&ots=SowITKwboW&sig=rJGw0ll6nLiZUTHRUrUYV7RINAo&hl=en&sa=X&ved=0CB0Q6AEwADgKahUKEwjw4-n-4uHIAhUH7R4KHR3SCt8#v=onepage&q=EOG%20placementeog&f=false

Nice paper on EOG amplification:

http://www.ijimt.org/papers/449-T129.pdf

EMG guide

http://www.instructables.com/id/EMG-Biofeedback/

Uh, wow. Reading through this now, but this is an incredible find for tuning EOG signals into output signals from Arduino.

https://engineering.purdue.edu/ece477/Archive/2012/Fall/F12-Grp02/nb/Matt/Human%20Electrooculography%20Interface.pdf

PDF guide to make Brion Gysin's Dream Machine

http://magazine.seymourprojects.com/wp-content/uploads/2011/09/dream-machine-plans.pdf

10/21 - Semester Project Process #2 - Jen Herrera

Since proposing my semester project, I've been focused on gathering a list of the materials I need to construct it. My first mini-project was a good exercise in working with conductive thread. I made plenty of mistakes and was able to fix them in time, so I hope that learning from those mistakes will save me trouble when I work on the final plush. I still have a lot of bits to work out, and I'm especially nervous about the LED Matrix and hooking that up. I don't think coding that will be difficult, but I have no idea how wiring that will go. I want to get a matrix in the mail before the 2nd mini project so that I can start working for it.

At the moment I need to:
1. Order supplies
2. Figure out how each component will work seperately
3. Make a mock-up pattern for the plush interface.

Conceptually, I need to tighten up which direction I want to go. When you touch the sensors on the plush, will it work predictably? Be frustrating and random? Or be unexpected but systematic?

Semester Project Process

Use bright signs to connect to projectors

Connecting the projectors to the bright signs will allow the arduino to control the images or videos being displayed

http://support.brightsign.biz/entries/21137931-BrightSign-meets-Arduino-TTL-Serial-Communication-

Semester Project