Hi,
I'm making game animations (a character with a two-handed sword: idle, run, attacks) by generating video and capturing the motion from it. It works, but the motion in the video often is quite stiff. I know its most likely not even possible to reach blender/mata quality yet, but i want to try pushing the quality as much as i can.
What I do now:
- Pose a mannequin in 3D for the key poses (e.g. idle and mid-run), render them, and turn them into images of one consistent performer: plain clothes, grey studio, floor lines.
- MiniMax H3 in ComfyUI, first + last frame pinned, about 5 s (124 frames) at 1344x768, 20 steps, with a motion booster LoRA. I also tried text-only, which gives freer motion but no control over the poses.
- Run a video-to-3D pose estimator (FlowHMR) on the clip.
- Retarget to a Mixamo rig, then to the Unreal mannequin.
- Clean up in scripts: lock the hands to the sword, fix knees and feet, remove lean, and add hip bounce back etc.
I'm looking for some opinions on how to squeeze h3 vid quality and much as possible to make it more fluid and dynamic.
Prompt that i used to generate the video
How the reference pictures align with the target video ā Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot 1) aligns with the 5.17-second mark of the target video.
integrated_multimodal_description: [Shot 1] Live-action, real-time speed, one continuous take in a plain light-grey studio with a light-grey floor marked with thin dark lines every metre, as in Picture 1 and Picture 2. The barefoot man in navy clothes begins in the low wide stance and framing of Picture 1, seen from the front at a three-quarter angle from the side his sword points to, his head turned toward the left of the picture, the long thin wooden sabre held level at chin height, its blade reaching toward the left of the picture and toward the camera; the hand nearer the blade grips the black handle just behind the guard beside his chin, and the hand at the end of the handle grips its end, his elbow out toward the right of the picture. The camera holds a perfectly static shot from a tripod while he stands and starts. As he drives off, the camera begins to travel on a straight line parallel to his path, toward the left of the picture and back, level, at a constant height, keeping the same direction and focal length and the same distance to him, quickly matching his running speed, so that from about 1.5 seconds he stays in the same place and the same size in the frame as in Picture 2 while the thin dark floor lines slide steadily past. At about 0.3 seconds he shifts his weight onto the leg on the right of the picture and dips a little, and in the same flowing, accelerating movement his two fists slide straight down the front of his chest and belly toward his near hip, his elbows bent and close to his ribs the whole way, the fists close to his body; as the fists drop, the sabre tips over so its point swings down in a diagonal toward the floor ahead of him, the point only ever moving downward from where it started; at the same moment his body pitches forward toward the left of the picture and his first long step drives off. By about 0.8 seconds his fists are low beside his near hip and the blade points diagonally down at the floor ahead of him, in front of his legs. His two fists stay closed around the black handle exactly where they are the whole time, the hand nearer the blade just behind the guard and the hand at the end of the handle at its end. From about 0.8 seconds he sprints toward the left of the picture in long, reaching strides: the far foot lands, then the near foot, then the far foot, then the near foot, and so on, about three foot strikes every second, the front leg reaching far ahead before each heel lands, the rear foot kicking up high behind him to hip height, both feet leaving the floor between strikes, his torso leaning far forward. As he runs, his shoulders and chest twist against his hips with every stride, and his two hands and the sabre swing with them as one piece, rocking forward and back and rising and falling with each step, while his torso bobs with each landing; his whole upper body works with the run, the hands keeping hold of the handle low in front of him and the blade pointing down at the floor ahead of his leading foot. The sabre stays one rigid piece of light wood of the same length the whole time. Every frame is sharp, crisp and evenly exposed, with his legs, arms and hands clearly visible against the light-grey background. He keeps sprinting at full speed right up to the last frame, his legs and arms still moving, and at 5.17 seconds he passes through the stride, position and framing of Picture 2.
overall_soundscape: Bare feet slap the studio floor in a quick, accelerating rhythm, with his sharp breathing and the swish of the wooden sabre. The studio is otherwise quiet.
non_diegetic_music: N/A