How did my “We are Broken” animation come to be? What goes into an MMD animation? What’s it like to make an MMD video for a whole song?

My MMD videos are always different. In general, I like to create videos that elicit emotions. Two of my previous videos feature “endearing friendship” and “a wild party.” With my latest full-length MMD video, I decided to try a much more complicated idea. Instead of aiming to elicit an emotion, I decided to portray a state of being.

The Making of my “We Are Broken”
Full Length Animation

The song was three minutes long. If I only had one and a half minutes, it might be more difficult to portray a range of emotions without making the animation feel rushed. There was more time to show everything I wanted to show if I made a full-length MMD video.

This is the result of my experiment.

Inspiration

Sometimes, I hear a song and immediately start planning a video. Other times, it takes a while for the ideas to come. This was one of the latter projects. I was listening to the song We Are Broken by Paramore and thinking of what kind of events would relate to it.

That thinking brought up some memories from when I was in the hospital for a long time. I wanted to capture the emotions that came with those hospital trips. I wanted to portray all of the emotions, from the hopelessness of being stuck in bed to the joy of winning a Wizard of Oz VHS in a bingo game. (Yeah, this was so long ago that VHS was the way to watch movies at home. I’m better now.)

Making a Model

In order to make this video, I needed a model of someone in a hospital gown. Unfortunately, nobody had made a model like that. Maybe I would need to make my own, but I kept looking. I found Simple Gown. It is a hospital gown, complete with ties on the back.

I used many LearnMMD tutorials to help get the look I wanted, modifying the model’s appearance until it made me happy. I published a test using Bleachless by NightWalkerDOF7. After that, I noticed there were a few more problems, so I kept modifying it.

I edited the textures as well as the mesh. The skin textures were made paler and slightly yellower. The hair lost some of its contrast. After that, I made the texture transparent in areas where it was causing me trouble.

The last thing I added was a hospital band. I decided to use a Japanese hospital band because Miku is Japanese. After some research, I came up with this design.

Fun fact: The patient ID number is the same as the UPC for Miku Hatsune’s original release. The barcode itself is a public domain image that I found on Pixabay.

Choosing the Stages

Once I had the model, I needed to get stages. I already had a model with the outside of a hospital, which was part of amiamy111’s City Stage. It would work for the first part of the song, where the singer states that she is outside.

I wanted another model that could show the inside of a hospital room. For that, I chose o-DSV-o’s Hospital Room. It was more realistic than the first model, but I didn’t mind. To me, it looked perfect.

Initially, I was under the impression that I would have to switch between the stages by rendering two separate files, one with each stage. This was not true. I was able to switch the stages using the “display” function on the panel where you load models. The stages change without you ever changing the file. It’s so convenient!

Talking Time

This animation uses all five tracks in Mogg’s Face and Lips. From top to bottom, they are lip sync, lip sync overflow and mouth expressions, eyebrows, eyes, and anything else.

We Are Broken uses Mogg’s Face and Lips extension for most of the facial animation and all of the lip sync. We have an entire category dedicated to using that program, so please go check it out.

I did not expect to be using Face and Lips for this project. In the past, I had a difficult time making sure that the lip sync was accurate. It turns out that, for some reason, Face and Lips uses a time-based system to figure out how far to move the play head even though MMD is only able to add keyframes at 30 frames a second. As far as I know, there is no way to switch it to a frame by frame animation system. Setting the program’s Zoom (Time) to x4 makes the play head move one frame at a time, fixing my problem.

Once I figured out that little trick, I was able to animate the lip sync. I listened carefully to the song and chose the mouth shape that best represented the sound, not the vowel in the word. Japanese phonemes don’t always match up with the English ones.

At this point, I noticed something. The music in the intro was actually Hayley Williams singing. What was she saying? Try as I might, I could not get the sounds to make sense. I would just have to ignore it and hope nobody else minded.

Once More, with Feelings

I moved on to the expressions once I finished the lip sync. It was fun to play around with the expressions sliders to figure out exactly what emotions I could make Miku convey. I’m proudest of her “worried” expression.

The end of the Face and Lips file contains another interesting touch: a “brave face”. The song is a downer. I thought showing Miku putting on a smile in the last moments would be clever. In the final version of the motion, this turned into a Graduate-style ending where her smile slowly fades away. I’m not sure which version I like more.

Getting A Move On

Once I’d finished all of the animation in Face and Lips, I ported that file over to MMD and started making Miku move. In general, I already knew what I wanted her to do. We Are Broken is a relatively calm song, so I made most of her movements simple and smooth. In the intro, I just had her breathing. She begins to move with more purpose as the actual music starts. Most of these moves are simple and relatively contained. She doesn’t even move her feet until she turns around at the end of the first chorus.

The second verse has more energy to it, so I had her move more. She rushes towards the door, turns around multiple times, and bends into a ball. There are no lines where she is still.

The bridge of the song is more calm, so I made her movements slower again. Single movements stretch over whole musical phrases. The last line of the bridge raises the energy levels. The stronger, more emphatic moves come back.

The camera is always moving. In all of the shots except for one, it’s focused on Miku. I used that shot because I wanted to easily move the model between locations. She ended up clipping through a wall. I liked the movement more than I liked the camera being on her.

When Miku had a perfect motion, I chose my effects and rendered the animation.

In Conclusion

Making We Are Broken was interesting. I wouldn’t necessarily call it “fun” to make, but I liked making it. There were times when I became frustrated or confused. Making a full-length animation is a lot of work, but, in my opinion, it’s worth it. I’m proud of the final product. I hope you like it. Maybe it will inspire you to create something.

In fact, you can download the model and the motion to create your own animations now!

The TDA Hospital Miku has a complete MMD model. You can edit it, but can’t take parts from it unless you ask me. Please don’t redistribute it or use it for R18 animations.

We Are Broken has files for the motion, the camera and lights, and a file with the lip sync on its own. You can edit it in any way you like. Ask me if you want to distribute your edited motion. Please only use it with the provided music file. Do not make money with this motion.

I hope you enjoy these assets as much as I do. Remember to Keep The Faith and credit all of your resources.

– SEE BELOW for MORE MMD TUTORIALS…