What you'll need:

  1. Rigged model of virtual character
  2. Vtuber software
  3. Computer and modern smartphone (webcams are also supported by some software)

  1. Rigged model:

    • What is "rigged"? It means that 2d image of a character got split into multiple parts which correlate to real body parts and their movement (like eyes, mouth, arms and so on), and those parts got multiple animation states made for each (like smiling, blinking or waving arms).
    • Vtubing software usually comes with free sample rigged models.
    • Eventually you will have to make rigged model for your own character. You can do rigging yourself or commission it to someone.
    • Rigging for 2d characters is usually done with Live2D Cubism Editor.
    • Free rigged models can also be found on Live2D website: https://www.live2d.com/en/download/sample-data/).
  2. Vtuber software:

    • In this guide we will use "Vtube Studio", it is free to download and try.
    • There are many other vtuber apps available, some are listed here.
  3. Computer and modern smartphone:

    • Different vtuber apps have different requirements, some can work without phone, but all apps will require camera to capture your movements.
    • "Vtube Studio" requires at least Iphone X (or webcam), and a computer that can run simple Unity engine apps.
    • Principle is the same for all apps, so follow along even if you are using different software and hardware.

How it works:

  1. Connect your phone and computer to the same network via WiFi (or ethernet)
  2. Launch vtuber software on your phone and on your computer
  3. Connect phone and computer vtuber software to each other ("start server" on computer, and "connect to server" on phone)
  4. At this point, your phone is constantly sending data of your face movement to computer
  5. On computer you load your rigged character model in the vtuber software
  6. Vtuber software makes character model to mirror your face movement in real time
  7. Finally, you capture vtuber software footage in OBS as "Game Capture" and stream!

Detailed Steps:

  1. Download Vtube Studio to your computer and to your phone.
  2. Connect your computer and phone to the same local network (for example to the same wi-fi).
  3. On computer:
    • Start Vtube Studio app
    • Go into app setting (gear icon on the left of the screen)
    • Turn on "Start Server" switch (it's on the first screen of settings, in the panel "Connection Settings").
  4. On phone:
    • Start Vtube Studio app
    • Go into app settings
    • Tap "Find Server" button (in a few second IP address should appear)
    • Tap "Use this IP"
    • Turn on "Connect to PC" switch
  5. Great, your phone camera now captures your face and body movement data and sends it to computer. Let's load virtual character to move with that data.
  6. On computer:
    • Close settings menu (cross icon on top right)
    • Show icons on the left by double-clicking on empty space anywhere
    • Open virtual character selection menu (person icon on the left)
    • Select sample model like Hiyori or Tororo
  7. Done! Your face and body movements now translated into virtual character movements. Try opening mouth, shaking your head, and making wide eyes to see how it works.
  8. Now you can use Vtube Studio output in your streaming software. For example in OBS Studio you can add another Game Capture source with Vtube Studio window target.
  9. Reminder: If you're using free version of Vtube Studio, every minute it will stop connection from phone to computer, so you will have to turn on "Connect to PC" switch on your phone again.

Tip 1: It is best to position camera on your eye-level, about 50cm away from you (or at the same distance as you computer screen).

Tip 2: Great way to hold your phone in place is to use "selfie tripod" with adjustable height.

Tip 3: Lighting conditions are important, so if you notice flickering of some body parts (like eyes or mouth), turn on more lights and make your room brighter.