Technical Progress: FLORA Board to Arduino Leornado

After few round of testing on Adafruit FLORA board, I still faced problem connecting the circuit and coding the keyboard control.

Therefore, Yee Siang recommend me to explore Arduino Leornado Board, an Arduino board that has built-in keyboard integration. I can connect the circuit like most of the Arduino board, no extra library needed. I agreed with the suggestion too. Since I am considering investing one Arduino board that allow easy flash integration, this is a stable choice.

I ordered one Arduino Leornado online from a local seller and give it a try so that I can meet the deadline of Alpha test on week 8 (after the New Year holiday).

Once I received the board, it took me few days to understand the board by testing out some of the codes shared online.

Example:
Keyboard Message - Arduino Leornado
Keyboard Modifiers - Arduino Leornado

My main challenge is to get Arduino Leornado to pass sensor info to computer and turn them into keyboard input. The two links above helps me to grab the concept, then adapt them into pressure sensor analog data condition.

Along the way of testing, I tested the input with 2 or more pressure points to see how I can allow more variety of user control.

But due to different sensor point has a different "minimum number" and "maximum number" at it's before and after pressure. Some times when I plug the input pin at the wrong place, it will trigger a infinite keyboard press on my computer, and it got my workstation hang for many times.

After few times of such "disaster", I learnt my lesson and carefully label all of the wiring so that no more "computer hang" incident will ever happened again.

Technical Progress: Velostat Pressure Sensor - Multiple Input

The first trial of DIY pressure sensor successfully detect the analog output from the changes of voltage when input weight is added onto the sensor. To create the pressure sensor, the pressure pad is separated into 2 part, the top part connected to the 5 volt pin on the Arduino board and the bottom part connected to the Ground pin. If there's contact between the top part and the bottom part, it'll be a complete circuit. No voltage loss.

For pressure sensor, it means there's voltage changes when force is added upon the sensor. A layer or 2 of velostat is added to avoid direct contact of the top and bottom of the pressure sensor. Once the velostat layer is pressed, it will trigger analog voltage changes. This set of pressure sensor will need to connect with a resistor of atleast 1K ohm.

To work on multiple velostat detection like my reference Joyslipper 2, I need to find example online that has same circuit connection with multiple analog output and resistors connections on the bread board.

At last, I found this circuit diagram that connects three potentiometers together, fits my requirement pretty well.

It took me quite a while to find this. But it's really helpful. I used this circuit to work on my latest slippers connection.

Technical Progress: Explore Arduino Adafruit Flora Board

Making velostat pressure sensor wasn't too difficult. As long as I got the top and bottom layer concept clear, it's quite easy to produce many of them.

Now, the challenge is how to connect the velostat pressure sensor with computer for input control.

I remember my classmate did a hand glove that allow user to generate input through the hashtag gesture. I decided to ask my friend about the Arduino logic and how I can turn the analog data from the pressure sensor into controlling flash-based interaction. 

From there, I was introduced Adafruit FLORA board. 

One good thing about Adafruit is that, they provide very resourceful learning system for their customer to expand the capabilities of their boards. FLORA board adapt Arduino codes, provided that developer has to download Adafruit library in order to demonstrate its features in Arduino environment. Keyboard library is available in Adafruit FLORA board. In this case, I can turn every pressure input into a keyboard control to activate my flash based content. 

Besides that, my main Joyslipper reference in the previous post is also using Adafruit FLORA board. I think this is the best start for the coding experiment. 

Instead of Using Arduino Application to code, I will have to download Adafruit version of Arduino Application.

With Adafruit version of Arduino, there are library specially dedicated for the Neopixel shoes, using velostat pressure sensor to control LED lights.

Digitize Storyboard

Although I haven't fully confirmed the visual idea I am suppose to create, I carry on transfer my storyboard sketches into digital look. During the visual transferring stage, I realized that if I am going to use vector graphic to make the complete video, it's going to be very time consuming. 

Since I haven't come into a clear conclusion of my visual idea, I thought of combining vector and realistic video footages to solve the technical issue of time consuming visual production. The other concern I have is the difficulty of creating the right angle for the vector world. As my project aim to create a visual illusion art work that engage user's attention, the visual has to be believable. Shooting real video footages and turn them into monochromatic mode could allow me to produce believable angle in a shorter time frame. But the challenge for this solution will be, to scout for location to film the video, the equipment required, to get approval for shooting video at the public place and etc.

While having these thoughts in my mind, I still need to test out the visual experience for my storyboard. So below are the first draft of the storyboard. 

Scene 1

Scene 2

Scene 3

Scene 4

From Scene 1 to Scene 4, I manage to make the graphic all in vector format. The challenge will be to get the perspective right. And the next step is to consider how these scenes are going to look like during the animation stage. Some graphic elements need to be able to flexibly adjusted for animation purposes.

Scene 5

Scene 6

Scene 7

From Scene 5 to Scene 7, I faced the challenge to create woods environment from such wide variety of angles. Therefore I decided to choose some of the high res photos from the internet to use as my footage sample.

Scene 8

Since 8 suddenly turn back to a very vector style visual. Personally I wasn't very happy with the result. But due to my struggle of time management issue during this Christmas festive season, I compromised for the sake of meeting my minimum production timeline.

Finalize Storyboard

I've been struggling a lot to settle the storyline and how illusion art could play a role in the storytelling, so here's the story flow!

Cinematography Reference: Wilderness Downtown (pop-up window, parallel sequence)

Sound: A Series of Unfortunate Event (opening title sequence music)

Technical Progress: DIY Pressure Sensor Test 1

I waited a month before the velostat delivered to me. Feel so excited trying out the DIY pressure sensor. For the time being, I have no Conductive Fabric to connect the cables, so use Aluminium foil to replace as a temporary solution.

Multimeter at the top left corner helped me to measure the resistor that works for my velostat as different thickness and materials will have different value to electric resistance.  

DIY a crocodile head cable

 Detail explanation in the video below

Info from > https://www.sparkfun.com/tutorials/389

Testing Arduino Connectivity & Coding

Additional Reference: http://www.kobakant.at/DIY/?tag=velostat

Perspective Tool + Art Style Testing

I am still quite slow in my visual development so far. My proposed art style was an experimental art style that I'm not familiar with. Therefore, I took very long time to decide the visual approach for the project. 

Throughout this process, I realised that I am a very safe designer. Lacking of guts to experiment in my final year project. The unpredictable feeling drags me on and on. I keep looking at references, and some time lost in references too.

Although I have such struggle, I still have to meet the timeline proposed. So I thought of trying out the style using Adobe Illustrator to create something. At least something that keeps me going.

The image above is one of my visual testing, playing with alphabet and trees in a perspective mode. This work is created to test on user perspective while walking in the forest in first person point of view.

Parallel Sequence, Multiple Screen Narrative Sample Analysis

Example 1: The Wilderness Downtown

Wilderness Downtown project is an experimental interactive narrative project. By using the latest html5 web video feature from Google Chrome, this project uses pop-up windows to engage viewer attention. Instead of watching an ordinary film on the browser, there will be browser window pop up and close simultaneously to composite the entire viewing experience.

Other than pop-up and close browser windows in different timeline, some of the pop-up windows (the bird flying pop-up window) will move towards the center window and then the center window will have the same bird flying into its video. This gives viewer a "magical feeling" as if the birds are flying across the browser windows. (refer to Screenshot 2)

The other feature of this project is the integration of Google Street View into the music video. By typing in a US address, viewer can see their address street view appeared in the video from top view and street view. Computer generated graphics will also overlay on the video to make it as part of the story.


Below are more analysis and observation made on some of the narrative scenes:

[Screenshot 1]

A layout composition made of browser window engage viewer to pay attention the different scenario at the same time. Compared to watching traditional film in cinema or on TV, this is a different experience all together. 

[Screenshot 2]

(explained on the 2nd paragraph above)

[Screenshot 3]

The main character that leads the viewer attention starts from running. And on which ground/environment he's running on, the pop-up windows explain that further. In this scene, while the main character stop and turn around, the top left window of the chosen location that presented from a bird-eye view responding to the character's action and turn as well. 

[Screenshot 4]

Similar narrative with Screenshot 3, the continuous scene of it.

[Screenshot 5]

This part shows the other interactive feature of the project. Viewer are invited to key in the message he or she wants to talk to the younger self to remind them to care for the environment. The 3 blue-ish box at the bottom left corner are mini video of machines response to viewer's mouse click and keyboard activity. View can use his/ her mouse to draw branches. Once viewer's input the completed, computer generated graphic bird will rest on the viewer generated branches. Then these birds will also flying across the pop-up window into the city top view.

[Screenshot 6]

The main character remain running. But this time, it's from a lower angle to review the growing tree behind him. These tree are also simultaneously growing in viewer's chosen city on top view angle and street view angle. 

[Conclusion]

I find this reference helpful because it has the "illusion" and "magical" element in the viewing experience. Instead of choosing optical illusion art style, this video uses realism technique. Blurring the boundary of real and fact to create a make believe narrative the makes viewer's environment part of the narrative content.

Besides that, the choice of media and technology is also very creative. It effectively create an engaging viewing experience by using pop-up windows, viewer choice of location, viewer's personalized messages being responded with the animated birds... and etc.

Creating Op Art using A.I.

Technical Progress: Purchase Arduino Kit

To create a DIY pressure sensor like my finding in Joyslippers version 2, I need to gather material as stated in the tutorial. Among all materials stated. There's one material that I do not know. It's called "velostat". After research online for it a little more, I realised that it's a material which it's electrical resistance decreases when pressured. My main ingredient for the DIY pressure sensor.

To have a better understanding in how it works, I found this site very helpful. And it makes the DIY shoes achievable.

https://learn.adafruit.com/firewalker-led-sneakers/make-velostat-step-sensors

The next thing I need to do, is to purchase it and try it!

Art Style Development

After some good rest in the holiday, I think it's time to rearrange my plans and thoughts of the design development.

I've tried some C4D approach of creating some depth of view angle, explored how Adobe After Effect helps generate fake 3D using vector graphics to do animation. I've also found some tutorial that teach me how to create Optical illusion effect in Adobe Illustrator. So, the conclusion is to mix around techniques and tool to create the final artwork.

For the time being, I haven't really come out with much visual testing and design.

Plan for the coming week -> Draw full scenery design x 3 different art style approach

I started today with a reading of the thoughts behind one of my favourite reference. Daehyun Kim's artwork has very interesting style that other black and white art style don't have.

http://www.moonassi.com/

Who are these people? Who's face?

The face with no expression is borrowed from old Buddhist painting which has always fascinated me since university. I thought that the face is perfect to conceal their feelings because I don’t want to show directly if these characters are good or bad through their faces. The simple black suit which looks like underwear has been chosen in order to make you only focus on their gestures. I purposely don’t show the time, place or gender.

Above quote gave me even more in depth consideration for my work. In this case, what do I want people to focus in the woods. Based on that, I can then set the hierarchy of details to build a effective visual for the environment.

Getting to Know C4D

To start working on C4D, I need to first install the software onto my work station. But I realized that my work station does not has the compatible specs to support C4D, so I have to work on alternative solution. After many rounds of testing and finding, I finally manage to get a work station that has C4D installed. The work station is shared with another person, so the con of the solution is that they will be times that I might not be able to work on the project.

Since the holiday is 2 months long, starts learning and working on the visual should be fine. Even if I can't work on the project in some of the days, it should be manageable still.

Learning the software
I browsed through some Youtube video to help me familiarize with the working environment inside C4D. And particularly, some tutorials that create 3D out of low polygons seems very much a good start for me to pick up some relevant skills. So I compile some of them here as the skill set learning documentation.

Pitch Presentation Self Reflection

In the progress of re-compiling the Project Idea, Creative Solution, Media Technology, and User Experience Design, I reminded of the entire project scope. After organizing my thoughts, turning the idea into visual makes me reconsider the solution approach.

My plan of digital design production is to use Adobe Illustrator (A.I.), to create a fake 3D optical illusion environment. After making the first prototype, I realized that although A.I. has the perspective grid feature, it's still very difficult for me to construct a 3D environment from scratch without a 3D environment based software. 

After deep thoughts, I consulted Zui, tutor that specialized in 3D based software, to find out the possibility of constructing my project in 3D environment. There're 2 software that I'm aware of - 3D Max and Cinema 4D. Zui gave me a quick intro to the basic usage of the 2 software, and I decided to use C4D as my design development platform. The reason for choosing C4D is because the software comes with timeline. So I can model the 3D visual and plan the animation all in one software, then render into the interactive environment. 3D Max is more specialize in modelling, no timeline tools available, so that's the major reason I drop the software option.

Black and white Animation References

playing with masking of shape can create different object in the op art environment

the blurred shape at the beginning gives room for imagination! Good idea!

interesting animation with sound

Baby View Classical Music- Soothe and relax your baby - High Contrast - Infant Visual Stimulation

Fantasy woods mood board research

Before I can draft my optical illusion woods design that looks so attractive and inviting, that user will want to walk into the illusionary world.

Perspective Drawing in Adobe Illustrator [totorial]

Ambience sound reseach

Target Audience Analysis [Research collection]

Extension Marketing: Campaign Planning and Audience Analysis

App marketing: competitor and target audience analysis

Generation Z: The new millennial

The power of the Net Generation

The Millennials: Confident. Connected. Open to Change.

DIY Pressure Sensor Research

How to Make a Ridiculously Cheap Analog Pressure Sensor

DIY Tech Footwear Round-Up

Capacitive Sensing Library (Arduino)

Joy Slippers

Joy Slippers Version 2

Buying conductive thread for the wearable tech

Three Monitor Connection

After many round of discussion and considering the visual experience I intended to create for the project, I finally decided to choose a panel of 3 monitor screens as the display setup. One of the main reason for choosing 3 monitor display instead of projection mapping is because of the difference in visual quality. Projection mapping, with the projector specs in the college, cannot deliver very high definition of optical illusion visual. As my visual content is mostly black and white, the quality of the visual is very crucial in order to create a realistic illusion visual experience. Besides that, I'm also concern about the availability of the projectors, location for the projectors and the distance between projection and the user. These are all the issues that cause me to change my display platform from projection to monitor screen. 

In this post, I've collected some of the background knowledge of how to connect more than 2 monitor screens to a CPU. 

My thoughts of MAGICAL storytelling in film - Part 1

I've watched some film that has the theme of magic. 

Film list: Now You See Me, The Illusionist, Oz the Great and Powerful, and Prestige.

Animation: The Fantastic Flying Books of Mr Morris Lessmore

There are some of the consistency in use of scene element and I'm going to list them down as my analysis to help me develop my storyboard.

#1 Spiral/ Circular Swirl

Swirl and Spiral are commonly used element that creates the illusion of falling into hole/ tunnel. It suggest direction, a path from one location to another location, potentially twist of dimension.

Oz the Great and Powerful - Tornado + hot air baloon

Oz the Great and Powerful - Opening Title Sequence

A Series of Unfortunate Events - Opening Title
(hot air baloon + spiral elements)

The Fantastic Flying Books of Mr Morris Lessmore - Tornado

Now you see me - Poster

Swirl & Spiral in Op Art 

The vector op art below are collection that helps me to develop my storyboard in creating dimension, transitioning in the adventures.

Monument Valley Game

The Making Video

I shared this project in my 5 4 3 2 1 poster before. But back then I haven't personally played the game. I've got a chance to play the actual iPad game today. It's mind blowing to me. Although I have watched how it works in youtube video demo, personal experience as a user is still amazing.

It puts together the visual theory of facet and perspective with good colour combination to create impossible architecture as game environment. Then it also has very good interactive graphic management to allow interactive feedbacks in the game. The smaller cube in the architecture of the building can be rotate to allow player to avoid the obstacles and take alternative path to the goal.

The game uses very creative condition checking to avoid confusion of architecture control for player to navigate in the game. The twist and turn of the building reveal extensive levels challenge at the same time create surprises to the player.

Monument Valley Game UI

This gif animation explains the features of rotating the architecture element in
the game environment to create alternative path to allow player to reach
the intented destination. By rotating the pink colour cube with little circle
emboss from the surface, will then change some part of the architecture in the game.  

This game uses simplicity art style. Simple shapes, with different brightness to create the 3 dimensional environment and then became the condition of character's movement.

This ui shows the multi-dimension of the ground for characters to move from one place to another.

With the ability to rotate some part of the architecture of the building, it helps the main character to continue her journey by shifting the enemy's walking plane to a opposite dimension that it'll never block main character's way. Some times, player can also shift the plane of the architecture so that the main character can walk on the continuing path when the architecture of the building is connected in certain point of view.