VR Manifesto

VR is a complex multi-faceted medium. I don’t think I can possibly organize all my thoughts about it, and this manifesto will probably change many times as I learn more about it. I don’t really feel qualified to write a manifesto at all, but it’s cool to record my current opinions about VR – what it is, what it isn’t, and what it can do for me (right now).

1. VR is a powerful tool with many applications. I’m starting with this point to simply highlight what we all know. VR is still a relatively new medium and is growing and ever changing. What makes it so powerful is its ability to transform time and space, whether for storytelling, simulation, or world building. It can be applied to any context and discipline, but I want to focus on how VR changes the traditional framework and boundaries for creatives. Visual artists used to a 2D space, now have a 3D canvas. Filmmakers that imagine their stories on a rectangular screen can now give an audience 360 degree views. Architects accustomed to designing functional space can use their knowledge in a new context that, by choice, abandons or adheres to physics. And users can have first-hand experiences in far away places in the physical comfort of their living rooms.

2. VR emphasizes presence by hijacking our perceptual abilities. We started this class with discussions of presence and immersion. I believe that VR is truly an exercise of these ideas. I personally find VR to be incredibly immersive. I give over to my presence immediately, and it doesn’t take much for me to disassociate from the real world. I place myself fully in the VR space. However, I acknowledge (and I think it’s important that VR creators, in general, also acknowledge this) that the hijacking of perception is only one aspect of presence. The other aspect is the ability to suspend belief, the mental part, the dealing with this complicated human thing of consciousness. I believe it’s easy to convince the brain that what it is seeing is “real”, but it’s a lot harder to convince the mind.

3. Because perception in VR is mainly visual, this is both and advantage and a disadvantage. As a visual person, VR blows my mind every time, because it makes me “see” differently. Humans are certainly visual creatures, but we have other senses. I definitely think that while visuals and sounds are powerful in VR, it never feels quite “whole” without smell, taste, and touch. When you limit the senses, you emphasize the virtuality. Ultimately, it frustrates me that VR’s heavy visual component makes it not completely accessible.

4. VR can make you feel things. I always thought of VR as having the capability of creating empathy, a conclusion I had come to more from just imagining uses for VR rather than from doing VR experiences or looking at research. After these readings this week, I’m not too sure how to think – empathy might be too strong of a word. So instead I’d like to say that because of its immersive quality, VR makes you feel. These emotions can be anything from a quick scare to more profound lasting effects, which leads me to point…

5. We have tools to elicit certain kinds of feelings, but reactions to a VR experience are unpredictable…and

6. Unpredictability is what makes VR awesome. We have built a toolkit as creatives, as storytellers, as people, of what sorts of things elicit emotion. We know that color, space, sound, etc. are elements that in combination or in absence can be provocative, but just because we intend for something doesn’t mean it will turn out that way. That’s one of the things I like most about VR. Since it enhances first-person experience, certain emotions and perceptions might feel heightened, but the ultimate impact is subjective.

7. VR is temporary. When thinking about creating for a VR experience, I think it’s important to consider that the experience is fleeting. You must create interest for your user while they are in the headset and whether or not that experience sticks with them after is probably more up to them than up to you. For a new user the novelty of the technology may be enough to enthrall them, but for the VR veteran, what makes them stay? Make for the moment.

8. VR is virtual, but believability and immersion come from being grounded in reality. I think the most successful VR experiences for me were ones that still borrowed principles from reality, meaning our world or something that resembles things we already know. However, if you create rules for your world to abide by, then users are more likely to buy into it. It can be the most fantastical place in the world, where the most unrealistic things happen, but if it’s meant to happen that way, I know that it belongs there.

9. VR spaces can become places. I have enjoyed thinking about the distinction between these words, and have gravitated to the word “personal” as being the key difference. Space is an abstract concept, place is somewhere meaningful to the individual. I think that whether a VR experience is modeled from an more abstract space that feels general or obscure or it is modeled from a personal childhood memory or somewhere familiar to the creator or the user, the experience itself can become a place for the user. No one will experience a VR experience the same way, but once they have experienced it, that world is now a place for them.

10. VR is art. Even with the limited experience that I had as a user of VR before Poetics of Space, I always admired its capabilities and its aesthetics. I’ve discovered now that, as a medium, it melds well with my interdisciplinary nature and I’ve gotten to explore it with my cognitive science lens. However, what I am enjoying most is that it makes me to tap into my artistry. It allows me to revisit principles of visual language, think about concepts and meaning, and delve into ethics. VR doesn’t feel like an unapproachable technology anymore, it feels like something I can mold and manipulate with my hands, and that is really exciting.

Music Interaction Design: Post Midterm Progress

After presenting my midterm, I felt I was on the right track. I received positive feedback for the overall concept and aesthetic, but was encourage to take it further. In addition, I realized that the FFT() and getCentroid() functions I were using didn’t quite give me the accuracy I was looking for. Although it was suggested before to use ml5, I think I stayed away from it because I was too nervous to delve into another coding library. I stuck with what was familiar but I ended up applying the wrong tool to my task.

Since the midterm my main focus has been transferring my current progress over to ml5 for improved accuracy. However, it almost feels like a step back because it took me a while just to understand ml5 and get the pitchDetection examples running (after starting a server, I can only run the pitchDetection examples in FireFox, no other browsers! grrr why??).

Coding for me is a quite painstaking activity. It is kind of ironic that I’ve chosen a final project that is primarily coding-based, so it is slow going, but it is going.

I wish I had more to show, but I feel good about my current understanding of ml5, the example pitchDetection programs, and how to modify them for what I want to do.

Soft Robotics: Big Box Store & Cable Control Experiments

Big Box Store Visit

When it’s a rainy day, visit Michael’s!

BEESWAX: A soft material in that it is has a specific melting point. This version attracted me because it is in small pellets to later become a larger vat of material.

UNICORN: This was a soft plastic material. Not rigid like a hard plastic, it had give to it and could be squished. Also hilarious that it’s called a unicorn but it has no horn.

RUBBER STAMP: Rubber in general is a very malleable material, but at the same time it retains its shape really well.

STAMP PAD: I was attracted to the ink pad as well as the stamp itself because it is essentially a sponge. I like the aspect of how it can retain a liquid, which could be interesting to explore further.

CORD: Ropes and cords are very flexible. They can be manipulated themselves or used to manipulate other things. This one had a special glow in the dark property.

WOODEN SNAKE: Although wood is not traditionally considered “soft”, it was interesting to see how altering it in this way, gives it articulation and flexibility like a soft material.

PLAY DOH: An ultimate throwback for a soft material. Its squishy quality is the most attractive, but I always remember it drying out. I wonder how this will effect things.

FOAM CREATURE: This form of foam was awesome to touch. It was very squishy and could hold shape briefly if you applied enough force. It was most fun to watch it slowly return to its original shape.

BRUSHES: This attracted me as a soft material because its a collection of one thing (a bristle) to make a larger surface area.

CHOCOLATE MOLDS: Literally soft robots! It is a silky textured plastic I think, kind of like a silicone. It retains shape, but has a floppy quality.

VR Final Treatment

For this project, I am working with Dylan Dawkins. Our goal is to create an immersive experience of a mass extinction event. Linked is a slide deck outlining our plans.

Schematic Package

Once our ideas were finalized and approved from the conceptual package, it was time to shift gears to creating a schematic package.

This would provide a more detailed plan of execution including:

• User-flow diagrams

• Systems diagrams

• Drawings for fabrication

• Schedule of work broken down for each team member

• Cost summary

• Visual design

SCHEMATIC PACKAGE DECK

Working calendar, timeline, task sheet, bill of materials, and resources are compiled here.

We decided to have our team members play to their strengths, so the division of labor became as follows:

Dylan - AR

Mingna - Sound/Video

Emily - Graphic Design

Maya - Fabrication

VR Midterm Post-Mortem

Some reflection of the VR Viewmaster project:

I am happy that, for the most part, the idea of “child’s perspective” came through strongly in the piece to those who tried it. I was intrigued by the responses to the three environments as a series. I purposefully mixed up the order in which they were presented because two me the environments were more like memory snapshots. It’s interesting how maybe the number “three” or the presentation of something one after another made my classmates search for an underlying narrative. I always thought of viewfinders as a transportive device that took you to many disconnected places, or in other words, the more different the pictures were, the more fun it was to keep flipping through. I wanted to carry that over into my own work and connect them more by theme. In the future, I will try to provide some on-boarding before people start an experience. 

I had many technical problems. A lot of titled cameras, sometimes exaggerated on purpose but sometimes by accident. Moving forward I need to test in the headset regularly throughout my process of making. My biggest frustration which my classmates didn’t actually seem to mind, was that the car was see-through. I had really wanted to make that space enclosed, but it ends up having kind of the opposite effect. I think the transparency has to do with the type of model I imported. I’d like to know more about models and what properties allow for a successful transfer to Unity. 

Overall, I am happy that I was successful in creating 3 separate environments that elicit a similar theme of feeling. I definitely want to challenge myself to create more provocative emotion and present with technical polish. 

Music Interaction Design: Midterm

Concept.

My project is a music learning tool for singers. Inspired by the phenomenon of synesthesia, the project combines color with notes as a way to provide a visual to sound. For an untrained ear, one may not be able to hear if she is singing incorrectly, but perhaps she could more easily adjust if she sees it.

I believe it is pretty entertaining. For the initial prototypes it certainly feels game like and participants become determined to see how their voice can change things. I think it is a good entry point to music learning because its simply exploring your own ability to listen to and produce sound.

Technology. Briefly, how does it work?

The project mainly uses the getCentroid function of FFT in p5 sound to identify pitch. A recording (with tuning tones) are analyzed and parsed for pitch. Live microphone input, to capture voice pitch, is analyzed the same way. Voice input is mapped in certain ways to give the user a better sense of what’s happening.

In action.

PROTOTYPE 1: This first attempted tried to map pitch to the width or height of the voice circle. Meaning that a flat note would squish the circle making is squat and stretched horizontally. A sharp note would become tall and narrow. Thus, the user is meant to aim for a perfectly round circle.

However, with all the ranges/bins of centroid values, it ends up producing more than 2 circles. This is confusing. My goal is to have one circle that’s static to represent the given pitch and an ever changing circle for voice input.

PROTOTYPE 2: Gone away with mapping pitch to width and height of the ellipse, and instead, this version asks the user to match the diameter of their voice circle to the diameter of the pitch circle. This is most similar to my initial modular prototypes. I also added a “confidence” mapping. This links volume to voice input. The louder the user sings, they create a bolder color.

PROTOTYPE 3: I still feel compelled to illustrate the nuances of pitch. So in this version, the circle travels vertically depending on whether the voice note is flat or sharp. This metaphor works well, I think, because you can think about producing sound as going below or above the target. Volume mapping is included in this version as well and as an exciting artifact, the color mixing for harmony is especially prominent.

Next steps.

• Although tones are associated with unique frequency values hertz…getting p5 to recognize them in the same way is almost impossible….in addition, something like the human voice, has way more idiosyncrasies than a computerized singular tone. Thus, my main battle is with noise. I tried different filtering techniques but still have a weird strobing effect that I don’t like. My next step is to really figure this out so that the aesthetics can be purer.

• I would like to continue building the interface to figure out the best way to help people learn to sing.

• title?

On another note, this project has really encouraged me to play with Ableton. I don’t think it relates to my project yet…but I wanted to also share my 2nd real attempt at a song:

Conceptual Package Post-Mordem

Our presentation went well and validated our current concepts! There was some concern that we were being too ambitious, but we feel up for the challenge. Our biggest decision was to focus mainly on the AR rock wall for this course. (The VR could be a possible side project for Dylan and I in our other VR-based class)

Below was some feedback we received:

Re: Conceptual Package Presentation 3/7/2019

General feedback for all groups: 

• Think about spatial design, creating transformative moments (soundscape)

• Creating a narrative journey/cumulative installations vs. installations that stand alone

• How do you onboard museum- goers?

FOR US: 

• Bring in the scope

• Can you make it more tangible?

• Include Rampino’s voice?

• Think about lighting and sound design (big bang planetarium project dome using theatrical lighting  → digital vs. analog mechanisms)

• Focus on your metaphors: what is the tablet exactly for AR, is it a “lens”?

• Interesting to talk about the atmosphere, exploring the “invisible” thing

• Do you want to provide a physical takeaway?

• Liked rockwall the most 

DECISION: AR rock wall is the focus!

VR viewmaster

For this assignment, we were asked to create 3 scenes in Unity to serve as a VR version of a viewmaster. I chose to visualize different snapshots of childhood memories. I purposefully played with scale to make the viewer embody a child’s perspective.

The first view is a fantastical Christmas scene. I remember loving to lie beneath the tree and looking up into the lights.

The second view represents car rides to our country home. I wanted to play with altering the skybox and give the illusion of motion as if truly on the road.

The third view is an interpretation of an imaginary place I created as a child. A particular section of a backyard brook, I deemed “Crocodile Island”. The trees, large rocks, and running water existed in real life, but the crocodiles and adventures I had there were made up.

Conceptual Package Presentation

In class we presented our final conceptual package to our professor, Brett, the class, and some guest critics involved in museum exhibition design. We narrowed our idea down to an interactive AR rock wall paired with a VR experience.

FINAL CONCEPTUAL PACKAGE DECK