CONCEPT: Melding the physical and digital worlds with tangible interaction
Our concept for this project was to recreate a physical flashlight that instead creates light in a virtual scene. This acts as a USB HID device by controlling the mouse on a computer screen.
PROGRAMMING PROCESS
Git repo here.
Moving the mouse
Using the Mouse library and the Arduino Nano 33 iot’s built-in accelerometer, Dylan wrote a program to reads the gyroscope values of the board and move the computer mouse accordingly.
void readGyro() { float x, y, z; if (IMU.gyroscopeAvailable()) { IMU.readGyroscope(x, y, z); //recalibrate incoming data; int x_avg = round(x) + offset_gx; int y_avg = round(y) + offset_gy; int z_avg = round(z) + offset_gz; if (hasMouseBegun) { moveMouse(x_avg, y_avg, z_avg); } }}
Bluetooth
Since flashlights are usually battery operated and incredibly portable, it felt important to try to mimic this ability. To make the flashlight untethered, we decided to use bluetooth (Bluetooth BLE library). I focused my efforts on this, programming the Arduino inside the flashlight to act as a peripheral (powered by battery) that connects to a second central Arduino that plugs into the computer via USB. Video of it the peripheral sending gyroscope data to the central below:
COVID
With changes brought by Coronavirus, Dylan and I truly had to divide and conquer to make this work. I handed the Arduinos back to Dylan, so that he could work on the “fabrication”.
DEVICE DESIGN
BOM here.
Fortunately, we were very intentional about our device design deliberately maintaining the intuitive nature of a flashlight. We chose to use an existing flashlight enclosure and didn’t have to make anything new. Dylan worked on rewiring the switch, changing the light to a neopixel, and connecting the battery.
He also added more functionalities like LEDs to flash it is in pairing mode and change to statically lit when it’s connected. This is a feature that we felt would make it feel familiar as a bluetooth device.
Central
Peripheral
UNITY SCENE
Dylan created a dark forest scene in Unity and handed that off to me finalize. We felt this is a setting that would work well with the flashlight devices. To create magic, I’m adding elements that will react to the light.
