Creating MMD Diva Performance Animations – An Overview

How do I create a Diva Performance MMD animation? How do I make a professional-looking MMD music video? What software do I need to make top-end, high quality MMD videos? What is the actual process for creating quality MMD animations from scratch?

Creating MMD
Diva Performance Animations
… An Overview

In the final analysis, MMD is a program made for creating animations, yet a lot of people have little idea on how to get started in creating custom motions for making their dream videos or what really is involved in the process. My experience with many beginners that I have communicated with in the past is that they come up with all sorts of ideas on what they want to do and then set themselves unrealistic goals which invariably leads to frustration and disappointment. Unfortunately we live in an age of instant gratification and automation so to a lot of people many things in life seem to require very little effort to obtain or achieve, but in reality this of course just is not true. This short article will give you a brief overview of the work required and other considerations that are required to successfully finish a custom made animated video production of your own.

A steep learning curve…

We will not be discussing the steps on ‘how to’ animate as that will take a book and there are plenty of good tutorials on the subject around on MMD animation techniques. Suffice to say, that computer animation does have a steep

Editor’s Note:

learning curve and is very work intensive. But for anyone who has never done something like this, it is helpful to have a broad overview of exactly what is involved and the level of commitment required.

Computer animation is not something for everyone; you really have to have the desire to learn it in order to get into it. By the same token, you do not have to be a computer animator to enjoy MMD. There are plenty of activities associated with MMD such as making pictures or editing models for use in MMD productions, or for that matter, just watching the videos that others have made is an important aspect of the hobby.

Specialize in a particular genre…

It is also important to realize that successful MMD computer animators choose specializations. The considerations, knowledge and skill sets necessary to successfully program a dance performance is likely to be very different from those required to program a song performance.

And if you think about this, this makes a lot of sense because of the time required to program an animated performance is no light matter. If you are stuck doing something that you do not like, it will be a real chore. This is one of the reasons why hardly anyone in the MMDC will entertain requests for making animations.

Motions… Emotions: The Diva Performance.

I specialize in a particular type of animation, too. I call these ‘diva performances’ and what basically this is, is to program specific MMD models to perform emotive presentations which in the real world are associated with real life diva performers of the type like Celine Dion and other singers in her genre. But I don’t work with real stars, just 3D models of anime style girls and the approach I adopt is not just to mimic real life performers but to translate what real performers do to be ‘diva-like” to MMD performing models. In reality, my motions are a form of caricature and there is a really simple reason for this: MMD performing models are not human.

MMD models are not human…

There are a lot of things that humans can do that 3D models of the type available to MMD cannot do. Most of these things are very subtle because of the way we actually communicate with each other and this includes a lot of subtle signals in terms of facial expressions and body language that MMD models just do not have the capability to mimic. But they don’t need to have these as they have certain features and assets that allows credible and effective personal performances to be created with them.

However, I do not want to end up writing a book on this subject, suffice to say, a lot of thought has to go into a production even before a single model is loaded into MMD. Instead, I will introduce the tools that I normally use and outline my basic game plan and thought processes used when creating a video production of the type that I specialize in.

Recommended Software

You will normally need the following software and thorough knowledge on how to use them in order to complete your MMD animation projects:

1. MOGG Face and Lips: This program is used to create the lip sync and facial data. It has much better control than the tools in MMD for creating this type of performance, especially since with this type of performance the camera is usually up close and personal. Some people do use MMD to do the lip sync, but for more detailed results and fine control over expressions and morphs, the MOGG Projects application is much, much better.

2. MMD (or MMM): This is used to create the performance motion itself as well as to render the final production. In particular you really need to know how to use the frames manipulation display as a lot of work needs to be done here. MMD is actually a cut down version of much more sophisticated professional grade animation software packages. If you are familiar with Blender, for example, you will know exactly what I mean. Stripped down though it might be, it is more than adequate for the purpose for which the program was written. It also means that you will not need to reference a thick manual to use the program which is great since an actual written manual does not exist for MMD.

You will also have to have a full understanding of what key frames are and how interpolation works plus how to moderate movements using interpolation curves.

Some motion makers makes the claim that MMM is actually the better of the two applications when it comes to making motions, and this is actually true in terms of the features that MMM offers. That said, I personally prefer the MMD GUI as it is less cluttered. Use whichever program you personally prefer as in the final analysis, it is the final product that matters and both programs are capable of delivering equally high quality results.

3. PMD/PMX Editor: This is used to customize the MMD performing model to work the way you want them to work. Almost all MMD models used in serious animated productions are hardly ever used ‘out of the box’. All serious motion makers tend to make changes to their copy of a model so that they work the way they need them to… and actually this was one of the reasons why the PMD Editor was created. Also, for those of us who do not use MMD in its native Japanese language, it is mandatory to edit the model’s labels so that they read in English or whatever language your native tongue happens to be. But never change the Japanese labels as MMD internally references bones and facials using their Japanese labels only.

4. Audacity: This is used to edit the music or other audio clips used as required.

5. AviUtil (or similar): This is used for video editing and to create the production version of the video.

6. NotePad: You will also need at least a basic word processor like NotePad to take notes and edit things like effect files or sometimes even model files themselves. I use NotePad a lot to jot down ideas and concepts which are used in the final project. You can only remember so much in your head unless you are a genius which most definitely I am not.

If you want to build your own accessories, stages and sets you will need a 3D modeling program and an advanced graphics editor too, but for this discussion, it is not necessary to go into these.

For your reference I personally use SketchUp, Blender and Photoshop. So if you really do want to get into creating polished videos, you will need skills in using quite a few programs to get things done. Then again, there are plenty of really nicely made stages and accessories made by others who specialize in making these things available for use in MMD. You do not have to build everything from scratch. Most people cannot, anyway, and this is one of the reasons why MMD has been so successful for so long. It has a huge high quality resource base and that is being added to almost on a daily basis.

The Game Plan

The following, in a nutshell is the ‘game plan’ I normally use to complete an original animation project:

1. Choose a song that you like, really like, as you will be hearing that song hundreds of times. I like ballads so I specialize in those. Once the song is chosen I listen to it enough times to learn the lyrics well enough to recite most of them in my head.

2. Determine the length of your project. This is very important especially when you first start creating animations. Most songs are 3 to 4 minutes long. At 30 frames per second (fps) this works out for a 3 minute long song to be 5400 frames. This is a lot of frames and could literally take months to program something this long.

Fortunately most songs can be edited to extract key selections that measure around 1:30 mins long and most anime theme songs are made to this exact length. This translates to 2700 frames which can be comfortably handled working on a project part time within a month or even a week or so if you just do it in the evenings and weekends like I do.

What I am trying to say here is: do not set yourself up for disappointment by biting off too much to do when you first start programming motions.

3. Edit the music if necessary. I tend to cut recordings short or to change their tempo or pitch to make them sound more like they’re being sung by a young performer. Also, a lot of songs have ‘fillers’ which can sometimes be edited out.

4. Choose a performer for your song. Individual MMD models are as unique as their human counterparts. So as a crude example, you do not normally want IA to be singing a song sung by Miku. Your final choice for your production should be a model you really like and that is best suitable for the selected song. Apart from anything else, you will be looking at this same model for hours on end.

Another important consideration when choosing a performing model are the mouth shapes available to the model. Different model makers have different interpretations of the basic mouth shapes, so for example, a lip sync animation made for a Tda base model could look terrible on an Animasa base model. So choose a model with the right set of mouth shapes for the song being rendered.

5. Once the performing model is chosen, edit it as necessary so that it will work the way you work and move in the way you want. At the very least, anglicize its labels. On my models, I tend to add some of the more advanced bones like those found on Tda style models if the model does not already have these. I also add a more complex hand system as I hate posing fingers and I have created a system where you can create motions for hands without posing a single finger. The alternative approach to hands is to have a library of hand poses available which can be loaded on to the model as needed.

6. Create a concept or theme. You did memorize the song right? What I do is to take my dog for long walks and think through the entire sequence so that I have a good idea of what the performance might look like by the end of my walks. Wherein it is true that sometimes I get strange looks from passer bys when doing this as I might move my hands in strange ways while working through the motions, the method works for me. Professional animators call this process ‘barn storming‘ and this is usually followed by creating a ‘story board’ where they actually draw out roughly all the key frame poses on a special chart. I just choose to do all of this in my head and use NotePad to keep key concepts that I do not want to forget. Use whatever method you prefer.

7. Create the lip sync. You want to do this in a locked room and in a quiet environment. Do not try to do the whole thing in one sitting either. When you feel that you have done enough, stop. Save your work and continue later. Trying to force it is going to lead to bad results. It is tedious but well worth the effort to do well and for a diva performance it is very important to ‘get it right’ as the camera is usually very close to the face and people do catch flaws if you get it wrong.

8. Once the lip sync is done create the facial expressions. If you have a ‘story board’ this is really easy to do. This is the part where you create the illusion that the MMD model is actually singing the song. I cannot emphasize enough how important creating the correct set of expressions are to the success of your model’s performance. Expressions are key in engaging your audience and for creating a credible persona for your performer. Always remember that MMD models are not humans and that they have large heads and huge eyes. Use, especially the latter to advantage. Never use the random blink function for this type of animation. The model only blinks when you want it to, and anything else that you want it to do.

This brings us to another point. A lot of MMD motion makers create motions that is intended to be shared and most of these are excellent, but by the same token they tend to be rather generic in the sense that they will work on almost any MMD model and in this process, there is generally hardly any provision for individualization and character building. However, a motion made specifically for a Miku performance will generally work well on most Miku models and portrays her character as understood generally by the her fan base in a recognizable manner. But try to use that same motion with Gumi and you might end up with a lot of dislikes or people switching to the next video after a second or so.

What you have to do is to decide whether or not you will be sharing the data because this decision does effect what you do and the way you work. My normal practice, after learning the hard way, is generally not to share my data simply because it makes no sense to do so. This is a practice common to a lot of motion makers who make the type of motions that I make. Some of the best song performance motions seen on NicoNicoDouga, for example, are never publicly shared simply because there is no reason to do so.

Not sharing your data also means that your videos will tend to be unique and if you are a serious channel builder, this is an important consideration as it helps to brand your productions.

9. Once the lip sync and facials are done, load up the model, add the data to it and pose it. Set up at least some type of staging, add MMEs, load the music, and create a ‘proof of concept’ test video. Then watch this video carefully a couple of times as you will be using it to catch errors in the work done so far as well as to use it to solidify your initial concept. I always do this as this is an invaluable tool. It works kind of like an artist’s sketch pad.

One more thing about the ‘test of concept’ video. It is also used to make a determination on whether a project is worth pursuing. Less than 50% of my ‘bright ideas’ ever get beyond this stage. In the ‘cold harsh morning light’ some ideas are just not worth the further intensive investment in time and effort to finish.

10. If the project is worth finishing, the test video is also used to allow the model to tell me how he or she wants to move. If I have done the expressions correctly, the model should ‘come to life’ and if I watch carefully the model will indicate to me how it should be moved to best express the meaning of the song and to help reinforce the model’s unique character. This is not as crazy as it may sound and this is exactly what I do. I actually have internal dialogues with the performing model balanced against my original story board / concept and the actual movements of the model are evolved from this to best interpret the meaning of the lyrics and to flesh out the model’s character. Do not forget also that in a 1:30 second long video, you need to establish the model’s character very quickly or else you will end up with a very generic looking performance.

11. Next is the hard part, to actually program the model’s movements. Personally, I do this sort of work late at night when no one is around to disturb me. Also, I do not do it all in a single sitting. It may take me a week or two to finish an entire production just working on it a little at a time whenever I feel like it. The point being is that it is partly a mechanical process but at the same time a creative one. You just need to be in the mood to do it or you can do this the way professional animators do it, which is a more disciplined approach. That is to lock themselves away, put up a ‘do not disturb’ sign and do as much of it as they can in one sitting. But it is very intensive work to get it all to work and look right so the process is tedious and mentally exhausting. So even in this type of scenario, most people won’t try to do everything in a single sitting.

In real life, take for example some of the animated cinematic sequence that are often used in video games; some of these can take up to six months or more to create working full time. So do not be surprised if your 1:30 second home made MMD production can take a whole month or more to compete just working on it in the evenings. That said, I have actually done some motions over a single weekend but then I had to skip doing otherwise important things like eating and sleeping or else survive by sleeping at my console and sustaining myself on snacks, copious amounts of coffee and nicotine.

12. Once the motion is done, make another test video and watch it a couple of times. This time around though, you should have also created at least a rough of any camera motion needed so that you will have a very good idea of what the final production will look like. With this test video, its greatest function is to ensure that the timing is correct and the model is moving naturally and that its movements have a contextual sense. The performing model should only move when it makes sense for it to do so.

Note also that unlike human performers, MMD models do need to move to some degree when performing otherwise you will end up with a very bland performance. Human singers can pretty much stand almost perfectly still and sing and create a wonderful and intensive rendition of a song. This is because they can communicate, in addition to their voice, with the audience through subtle signals in terms of facial expressions, body language and sometimes very slight hand movements. Actually, hands are a very important part of our communications suite, and this fact does translate well to MMD models.

Also, humans need to breathe when they sing and it is more efficient for the singing if they are not moving around much. This is pure biology. The high energy song and dance routines that you see in videos where performers are singing and moving around like mad is generally done using a pre-recorded track for the actual singing and the performer is actually only lip syncing when they appear to be singing.

MMD models do not have the mesh or facial sets to duplicate much of the subtleness of human communications. To get around this we need to create a caricature type of interpretation and how you do this depends on your style of making animation and video making. Animated characters just don’t look interesting just standing still and singing. If you watch the better made MMD productions and especially anime cartoons where the characters are singing, you can see all sorts of tricks employed to overcome this hurdle. The key to employing these techniques successfully to create a truly emotive performance relies totally on balancing the techniques used in a manner that creates a natural and immersive viewer experience.

In other words, if you don’t get this right, the end result is going to look fake. I often end up changing major segments of the motion after watching a test render as in the ‘cold light of morning’ what I previously thought was a good idea does not work. And each time afterwards I make another test render. For some projects I have made as many as a half dozen test renders until I was finally satisfied with the results. Something worth doing is worth doing well, no one wants to watch a poorly made production and it’s a huge disservice to my subscribers to make them watch crap.

13. Once you have got everything right, you can set up MMD to make the final render. This is where you add in the full MME suite that you have designed for the render and tweak the lighting and physics settings if these are needed. Incidentally, you will always almost never use the default light settings as these are very boring.

Make your final render when this is all done. There are two ways to go around this. One way is to render in high definition and the other way is to render raw. Either way works and the only difference is that it will impact the final production copy. Since in almost all instances the production copy will be hosted on a streaming video site, I tend to just render in high definition. This suffices for most viewing experiences as mostly these types of videos will never be shown on a movie screen. Also, I render without sound. The reason for this will be apparent in the final step.

14. Once I have my production render I load this into AviUtil and it is here where I add back the sound track, and where necessary, this is also the software that is used to add other sound effects if this is needed. MMM can take multiple audio tracks and compile them in a single render, MMD cannot.

AviUtil is also used to add titles and credits. These give a professional polish to the end product and is worth doing. Once I am satisfied with the video editing, I recompile one more time. This usually does mean that the video is recompressed again but since I rendered originally in high definition, any loss in visual quality is negligible. Still I want to create a high quality end product as once the video is uploaded to the streaming host, they will recompress the video yet again to match their technical requirements and allowances do need to be made for this. Additionally, with services like YouTube, audio tracks are converted on their servers to mp3 format. Which is why I tend to do my renders without audio and export the audio track from AviUtil in mp3 format since I trust my own conversion more than I trust theirs.

15. Finally, once your video is done, watch it until you get sick of it before uploading it. You will be surprised how often you might have missed an important detail and have to redo even entire segments.

When you are finally satisfied and only then, upload the video to your video hosting service and then announce it to all your friends and social network so that people will come to see all your hard work. This is actually rather important as people are not hanging around just waiting on your next production. If you want to get more views, you do have to tell people that you have something new for them to watch.

But even that is not the end of it! Get feed back, so that you will know what you did right and especially what you did wrong so that the next time you can do even better. I cannot emphasize enough how important feedback is to the whole process; this is where I personally learn a lot about how to get things ‘right’.

A test-render and a finished video…

I apologize for the fact that this article does not come with a lot of pictures, but what I will do instead is to show you an actual conceptual test render video that I have made for a project that I am currently working on to show you that I actually do, do what I am telling you that you might want to do. In this video, the lip sync and facial expressions have been done, but the model does not have motion made for it yet so it does not move and the stage setting is purely for conceptual purposes; additionally, I am also using this video to test out my initial mood and lighting settings so some MMEs were thrown into the mix. The video is rather basic but it is watchable… and it does show those of you who are more insightful the value of making this type of video. The other value of doing these types of videos is to use them to communicate with others that are working with you in a ‘collab’ so that everyone has a point of reference and it makes it easier for everyone to be on the same page moving forward.

Furthermore, if you do watch it carefully and engage mentally with the performing model, Miku in this example, you should be able to understand exactly what I am trying to say when the model itself will tell you how it wants to move in order to best express the meanings of the words being sung.

I hope you have found some useful ideas here. Also do not be afraid to experiment. MMD does have the capability of making the ‘impossible’ possible, so just let your imagination run free and give full wings to your creativity potentials. Personally, I have found making ‘off the beaten track’ animated videos the most rewarding of my MMD related works. The final video clip is an example of this kind of work. Thanks for reading.

– SEE BELOW for MORE MMD TUTORIALS…

— — —

– _ — –

– SEE BELOW for MORE MMD TUTORIALS…

— — —

– _ — –