Ray-MMD Toon Shading Basics

How can I set up Raycast/Ray-MMD toon-shading? How can I use toon-shading with Raycast? What is one method of toon shading?

Ray-MMD Toon Shading Basics

Hi, folks! I’m MuffinMo, and this is my first article, so please treat me kindly. Today, I’m going to teach you how to do toon shading with Raycast-MMD. This is the technique that I used in my Ai Kotoba III video, and it can look very good if used correctly.

Please be warned though, I’m only going over how to set up the code for Raycast toon-shading. I will happily answer any questions you might have about my settings and/or personal techniques.

Quick side note: toon shading is only available in ray-mmd-1.5.0 and later.

My “Ai Kotoba III” featuring Ray-MMD Toon Shading…

–

First Step: get Notepad++ to edit text files

You’ll need a source code editor. I recommend Notepad++. It’s free, and works really well. You can get it here:

https://notepad-plus-plus.org/

Configuring the Start

Before you open MMD, you’ll need to open the Raycast configuration file. I’m using the master version of Raycast. Once you go into the ray.conf file, scroll down to where it says “Toon-shading material.” You’ll need to set the value below it to 2.

You’ll also need to scroll down to “Screen Space Subsurface Scattering” and set that value to 0.

You can turn on the outlines, but that’s optional.

Basic Set Up

Now you can open MMD. Go ahead and set up your video or picture.

Don’t set up your lighting just yet, though. You’ll want to do that later.

Once you’ve set up your video/picture, load in a skybox of your choice. I’ve loaded Time of Night.

Now load in ray.x, and apply main.fx to your stage and model. If you’re using time of day or time of night, you’ll have to apply material_skybox.fx to the skybox under the MaterialMap tab manually, same thing with the EnvLight tab. Instead of adding the lighting effect inside the individual folder, add the one titled “skylighting_none.fx” inside the skybox folder. This will get rid of any weird spots on the model. Also be sure to disable MMD’s anti-aliasing, ground shadow, and the coordinate axis.

Editing the Material

At first, things aren’t going to look so great.

(I’ve had to adjust my lighting due to the nature of the stage I’m using, but usually I wait until I’m finished material mapping to adjust the lighting.)

This is where things are going to get a bit more complicated.

Go ahead and open up material_2.0.fx in Notepad++. Scroll down to the CustomA and CustomB slots. Make a copy of the file before you proceed.

Now set the “CUSTOM_ENABLE” value to 8 and copy the following settings:

“#define CUSTOM_A_MAP_FROM 0

#define CUSTOM_A_MAP_UV_FLIP 0

#define CUSTOM_A_MAP_COLOR_FLIP 0

#define CUSTOM_A_MAP_SWIZZLE 0

#define CUSTOM_A_MAP_APPLY_SCALE 0

#define CUSTOM_A_MAP_FILE “custom.png”

const float customA = 0.62;

const float customAMapLoopNum = 1.0;

#define CUSTOM_B_MAP_FROM 0

#define CUSTOM_B_MAP_UV_FLIP 0

#define CUSTOM_B_MAP_COLOR_FLIP 0

#define CUSTOM_B_MAP_APPLY_SCALE 0

#define CUSTOM_B_MAP_FILE “custom.png”

const float3 customB = float3(0.61, 0.55, 0.548);

const float2 customBMapLoopNum = 1.0;”

Save the file, and your screen should look like this:

Congratulations! You’ve officially created a toon shader!

Final Words…

These settings are not set in stone; feel free to experiment! Try playing with different colors, try normal mapping, try all sorts of things! Like I said before, any questions that you have, feel free to ask! I will do my very best to get back to you in a timely manner.

Thanks for reading, and remember to take care of yourself!

Credits

TDA Miku by Xoriu

Unknown Mother Goose Motion by ゲッツ

Facials by しのざき

Lip motion by kaz

Hand motion by すぱにぃ

IK motion by kemo

Camera by 鷹羽はるひ

椛暗式-遇见_暮ver1.0 by 椛暗/bananasteamedstuffs

Raycast by rui