Josh's CS 527 Website

For our third and final project, "Koosbane Karaoke", we worked in groups. The goal was to use motion capture data to animate a non-humanoid character to music, much like the way Jim Henson's earliest muppets could believably dance even without the expected human limbs.

We knew one of the most important parts by far was to select a good song, so our first step was to consider many different songs and pick one that we felt would work best. I'm not quite sure how it happened, but "Hamster Dance" was chosen (even over Macarena) and a country hoe-down theme became the basis of our animation. I edited the original song down to a version that included the main important parts and wasn't much longer than 60 seconds.

I set off right away to make a hamster 'cause I figured that would be the easiest (and cutest) thing to animate, but after asking Andy if that would be too 'humanoid' he basically confirmed that since our hamster would still be dancing somewhat like a human it wasn't acceptable. Foiled again by Andy! Luckily, we could still use him in a supporting role.

After making a last minute decision to change our main character from a hamster to a wheelbarrow and pitchfork, we were exceptionally grateful that our two real dancer friends were still willing to help out. (Thank you so much Christine and Stefanie!!!!)

We all worked together to figure out how best to 'dance like a wheelbarrow and pitchfork', but inevitably the motion capture session was long and tedious simply because we wanted enough data that we'd have a full 60 seconds of GOOD data. We walked out of the cave with almost 10 MEGABYTES of tracker data.

Next up - time to start assembling all the elements of the scene. I took point on creating all the models and smoothing the data, Jason took point on creating the tools in electro we'd need to create the final animation, and Sai dove right into the heavy math involved with translating information from the motion data and adapting it to the models we created.

Noticable in these two pictures here is the fact that I was probably gunning for a little too much in our scene.. but even so, I tried to make a couple different complexities of the scene and ran into some rather humorous scaling issues using models downloaded from Google's 3D warehouse.

Jason made a ton of utilities both separate and in the code itself to aid Sai and me. By far the coolest and most useful tool though was the sequencer. This gem allowed us to map four arbitrary models to any set of motion data we wanted to visualize, and then play back the file at varying speeds and exaggerations of movement. It also had the ability to set entrance and exit points to a data file, so that precise timing going from one clip of data to another could be timed easily. When all the parameters were in good shape, a write_out function would create a new data file with the trimmed size, time-scaled keyframes, and any changes made to the scale of movements all preserved.

I used excel to graph the data to visually find any rough spots, and wrote a couple simple macros to pull extraneous values back into usable ranges and just generally smooth the tracker data as well. The joint code in electro failed us all pretty badly, so we spent the vast majority of our final work on the project tweaking custom translation functions that brought the motion data into the appropriate values for our characters' extremeties. One saving grace however was the reliability of the orientation values of our main 'body' sensor - using this greatly eased the process of figuring out the other sensors positions in relative object space for the wheelbarrow.

With decent data and decent code for actually using that data, the last steps were just a marathon of building out each piece and sequencing it all together. Jason's tools made it much nicer though, so mad props to him for making the rest of the work go that much quicker.. we never would've finished in time without his tools or his work on translating the motion capture data into the correct object space.

I built the scene itself and all the models in sketchup, but we used numerous other modeling tools for tweaking the models until just about everything about them was fit for our use. Getting the lighting right on the wheelbarrow was especially difficult. We tried lighting inside for the barn itself, but it never looked as good as flat lighting on the barn and dynamic lighting on the wheelbarrow, so we stuck to that.

Also worthy of mention were the functions Jason built that allowed us to create our own key-frames for more absolute model positioning and orientation. Once we got the hang of using these functions I keyed animations for the wheelbarrow's entrance and the motion of the camera itself. Jason and Sai used these to animate the 'bigger' motions of the pitchfork, including its hops and tilts.

When it was all said and done we really just didn't have enough time for all of our ideas. This wasn't really anyone's fault, it's just the way it happened. We're still pleased with the end result, but we had some even cooler ideas we are kinda bummed we didn't have time to implement. Some points about the animation itself:

Download the final project HERE