The act of creating useful tools that I know will make people's lives easier, and fun games that I know people will enjoy, is pure bliss. Even when I'm just messing around to learn a new library or figure out how a new system works, programming gives a sense of adventure unlike anything else.

Here are some fun projects I've built in my own spare time and as coursework:

Machine Learning Classifier for Determining Film Trailer Genres (with John Fuini and Noel Kimber)

Research paper | Poster | Github repo

This is a pipeline for calculating a feature set, based on video and audio content, from a given movie trailer, and then classifying automatically by genre. We constructed our feature set from a combination of computer vision techniques and volume/frequency analysis. After this, we applied Support Vector Machines as well as Binary Decision Trees as classifiers. Our cross-validation testing showed an accuracy of better than 90% for some of the more common film genres.

Lunar Rover Ground Station

"Moon Shot" documentary feature (showing me driving the rover, but no actual shots of software)

Over the course of six months, I worked with the Google Lunar XPRIZE team Hakuto in Sendai, Japan, to build a fully-featured ground station for their "Moonraker" lunar rover, in preparation for a 2017 mission to the Moon. My ground station is equipped with a number of both standard and advanced features, including a robust fault detection and alarm system, 3D environment viewing, and several innovative types of live telemetry analysis and visualization. This system, built entirely by myself from scratch, is still in use and has already been tested in 100+ hours of laboratory and field tests.

Telemetry Display and Fault Diagnosis Tool (with Nick Reiter)

Frontend Explanation

Backend Explanation

Github repo | Explanatory poster

This data visualization tool is an experimental testbed for a number of techniques to display detailed telemetry data from an aerospace system, towards the goals of achieving a quick understanding of the current state of the system and troubleshooting any complex faults that may occur during system operation. Various techniques, backed by research and driving factors related to the needs of aerospace fault analysis, are employed within this interface, including a degree-of-interest tree modeling the data channel hierarchy, plots for real-time data display, and a correlation matrix showing relationships between various telemetry channels.

The client side of this application was written in JavaScript, using the D3.js library for various visualization-related components. In addition, data is generated dynamically via a program written in Python, and can be loaded from a number of data sources and distributed to multiple clients at once.

Ball Tracker and Predictor for Juggling

Github repo

Research paper

This is a Python application that uses the OpenCV library (version 2) to track the movement of balls during a video. It tracks the balls with a combination of color filtering and k-means clustering. Once the balls' positions are determined, their velocities are estimated by comparing consecutive frames. Then numerical integration (Euler's method) is used to find a predicted trajectory through space (subject to gravity). Ultimately, these trajectories are updated with new data from the image processing side, using a weighted filter to combine the predicted trajectory with the observed position and velocity.

A Brief History of Rocket Launches (with Sonja Khan and Nick Reiter)

Live demo | Github repo

This is a data visualization made using JavaScript and the D3.js data visualization library. Data for all of the recorded orbital rocket launches from 1957 to early 2015 is shown in an interactive, animated fashion, and colors allow you to easily differentiate between successful launches and failed ones.

3D Telepresence Interface for Robotic Manipulation (NASA JPL project)

In this project, I led development of an intuitive and innovative 3D interface for robotic calibration and manipulation. This interface permits extremely fast calibration of sensors and manipulator position (on the order of ~30 seconds), and allows precise manipulation in dynamic environments with an easily expandable library of objects and interaction methods. The tool was written in C#, and used the Unity game engine to generate the 3D environment.

Magnetic Ball Levitation System

Report of Results

This is a system that uses an electromagnet, driven by a controller written in Simulink and running under xPC target, to stably and robustly levitate a metal ball in mid-air. Starting with mostly-assembled hardware, a colleague and I ran various tests to determine system parameters, then designed a full-state feedback controller, using gains calculated with LQR, to apply a dynamically changing electromagnetic force to balance the force of gravity on the ball. This resulted in the stable behavior you see in the video above. Done as part of a university control systems course.

PID Controller for Line-Following Robot

GitHub Repo | Accompanying Presentation

For this project, I built and tuned a PID controller to turn an already-assembled, motorized vehicle into a line-following robot. The robot carries an Arduino Uno-equivalent board, and the code for this project was written in C++. (Some supplementary code for parameter analysis, when determining PID gains, was written in Matlab.)

Arduino Simon Says

Github repo | Schematic

An re-implementation of "Simon Says" built from scratch and programmed on Arduino, with LEDs and a piezo speaker to provide audiovisual feedback. Also includes a sound engine that I wrote to play music borrowed from Super Mario Bros. Controlled with an SNES controller. Hardware switches enable multiple modes and difficulty settings.

Markov Decision Process Solver for Inverted Pendulum Problem

Matlab project (7 KB)

This application models the approximated, discretized dynamics of an inverted pendulum balancing problem as a Markov Decision Process, and then uses Bellman value iteration to calculate an optimal state-feedback control law for the pendulum. This control law is simulated (and animated), as shown to the left.

3D Animation Engine and Animated Short

Binary and source (C++ with OpenGL) (3.7 MB)

Full animation engine, built on top of skeleton UI-only code, as a final project for an introductory computer graphics class. I wrote all of the 3D rendering code, particle effect simulation, and spline interpolation code for animation curves. This code makes use of two different particle effects; burn marks from the rover's laser and Martian soil clumps. Intersection code determines where the burn marks appear based on the laser orientation. Physical effects are modeled for particles; the soil kicked up by the rover is subject to pressure drag, gravity and kinetic friction, as well as an initial velocity whose direction that depends on the rover's current orientation.

The video above is an animated short that I created with this engine. Rover model geometry and textures are my own (but were not a priority for this project). Further details on texture and audio sources can be found in the YouTube description.

Arduino PID Motor Controller

Source (3 KB)

An implementation of an angular velocity controller for a motor using a photoelectric sensor for feedback. Uses PID control (with gains determined by Ziegler-Nichols) to drive a motor at a specified angular velocity profile. Written in C++.

At Heart's a Stereo: HTML5 Data Visualization

Live demo | Github repo

This is a data visualization that I wrote from scratch over the course of a few days, with JavaScript using HTML5 Canvas. Data is visualized in the style of a 2D stereo equalizer, where bar height and color can represent two variables. Text is dynamically sized and animations let you see how data changes over time. The custom data format allows users to add their own data easily. The concept for this visualization was created by Chia-Yi Hou.

Panorama Image Stitcher

Windows Executable (6.2 MB) | Source (2.9 MB)

This is an application that takes two images as panorama input and stitches them together by computing a homography between the images and constructing an equirectangular image projection to align and stitch the images. Overlapping pixels are blended together. For this project I built a Harris corner detector as well as a RANSAC-based alignment algorithm.

This project was written in C++, using the Qt framework. Larry Zitnick at Microsoft Research wrote the majority of the UI code and the high-level architecture, and I wrote the majority of the actual stitching code.

Face Detector

Windows Executable (33.9 MB) | Source (31.3 MB)

This is an application that takes an input image and identifies human faces within the image. The application relies on a training dataset of faces and non-faces, and uses this data to compute Haar-like features as described by Viola and Jones in this paper. After these features have been computed, the application uses AdaBoost to compute a classifier which is then applied to the test image to detect faces.

This project was written in C++, using the Qt framework. Larry Zitnick at Microsoft Research wrote the majority of the UI code and the high-level architecture, and I wrote the majority of the machine learning and detection code.

Blinded by the Beat

Windows Executable (4.5 MB) | Source (4.7 MB)

An experimental application that I made for SEAHOP, this is a puzzle game which uses no graphics. Players must rely entirely on audio to solve the puzzle. Once solved, an answer is revealed graphically.

Give it a try--stereo headphones help a lot. Controls are WASD or arrow keys, and space.

This project was written in C++, using the SDL library for multimedia. It requires the .NET framework to run.

T-Rex Vision Motion Detector

Xcode 4 Project (Mac OS X) (37 KB)

This is a simple application that takes a video stream (e.g., from a webcam) as input and outputs any pixels that have changed significantly over the past several frames, resulting in a sort of "T-Rex vision"-style motion detector (as inspired by Jurassic Park).

Image data is stored in a ring buffer, and when the standard deviation for historical intensity values for a pixel exceeds the draw threshold, that pixel is assumed to have changed significantly and is drawn. The ring buffer size and draw threshold are adjustable.

This project was written in C++, using the OpenCV library. This project requires the OpenCV library to run.

SEAHOP Puzzle Competition Puzzle Webpages

For the SEAHOP puzzle-solving competition, I wrote a dynamic website where teams can log in, and only certain puzzles are visible to them, determined by which puzzles they've already solved in the competition. This makes it so that all teams follow the same general order of puzzles, which allows the puzzle designer (me) to ramp up difficulty in a fun way.

I also built an administrative console for event staff, to make updating the puzzles each team has solved easy and fast.

This project was written in PHP, interacting with a MySQL database. See the SEAHOP page on this site for more information about SEAHOP.

Small Projects

I've also built the following projects for work, the details of which are mostly confidential:

I'm always open to learning new languages, libraries, and areas of programming. Learning new things is half the fun of programming!