From Character art to Animation Frame Sequences
On this page
The goal is not to generate a video that "looks like it's moving," but to obtain a set of animation resources that can be stably addressed, switched, and reproduced by the game runtime.
This method uses a complete character concept art as the identity baseline, generates short action videos using Gemini Omni Flash, and then compresses the video into a transparent WebP sprite sheet with a fixed canvas, fixed anchor, and fixed frame count. The actual workload lies not in calling the model, but in pre-generation constraints, action window selection, keying, frame sampling, scale normalization, and per-frame acceptance.
The final pipeline is as follows:
Define Applicability Boundaries First
This approach is suitable for:
- 2D characters displayed at small or medium sizes in-game;
- Short actions such as idle, attack, hit, and death;
- Performances where the action returns to a static idle state or holds on the final frame after completion;
- Projects where the concept art style is more important than skeletal precision, and the budget is insufficient for hand-drawn frame-by-frame animation.
It is not suitable for directly handling:
- Close-up hero performances and shots requiring frame-by-face scrutiny;
- Long loops that require pixel-perfect consistency between the first and last frames;
- Actions where weapons must hit specific coordinates along precise trajectories;
- Complex occlusion, grappling, two-person brawls, or intricate cloth interactions;
- Projects with strict artistic retouching requirements for the silhouette and anatomy of every frame.
The key realization is that video models provide a "motion asset" with temporal continuity, not an animation ready for ingestion. The final product comes from subsequent filtering and normalization.
Why Choose Image-to-Video
The most intuitive approach is to list key poses and generate each frame separately. The actual results usually suffer from three problems:
- Each frame reinterprets the character, causing slight variations in face, clothing layers, armor plates, weapon length, and body proportions;
- Individual frames look fine when zoomed in, but exhibit noticeable flickering when played continuously;
- Each frame interpolation introduces new identity drift, ultimately requiring extensive manual retouching to maintain consistency.
The advantage of image-to-video is not that individual frames are prettier, but that adjacent frames within the same generation share temporal context. It still redraws details, but the drift is typically smaller than independent image generation, and you can extract only the most stable 0.5–1.5 seconds from a longer output.
Therefore, this workflow adheres to two principles:
- One video handles only one primary action;
- Do not strive for perfection across the entire video; instead, look for usable windows with stable identity and clear motion.
Another fully validated route involves converting the character into a Meshy 3D model first, then batch-rendering frames in Blender. This pipeline works for modeling, rigging, animation, and sprite sheet production, but the final rendering did not pass the 2D artistic quality gate. To avoid interrupting the main workflow, the full experimental review is placed in the following section.
Prepare the Character Identity Card
Concept Art Requirements
The character concept art should ideally meet the following conditions:
- Complete from head to toe, with soles, weapons, and long hems not cropped;
- Facing a unified direction agreed upon by the game, e.g., all facing right;
- Clear face, hairstyle, clothing layers, and signature props;
- Pose close to a neutral guard stance, with limbs not heavily occluding each other;
- Safe space around the character, especially horizontal space reserved for long weapons and death animations;
- No background, shadows, particles, text, or UI.
If you only have a bust portrait, first complete it into a full identity master image. Letting the video model invent legs, shoes, or weapon tips during the action will inevitably cause identity drift.
Why Use a Solid Color Background
Most video models do not output a stable Alpha channel ready for use. Rather than performing generic background removal on complex scenes afterwards, it is better to bake the keying conditions into the input and prompt.
A high-saturation blue #005AF9 was used in validation because the character subject is primarily warm browns, black-grays, and dark reds, providing a large distance from blue. This color is not a standard answer. If the character itself has blue clothing, cool lighting effects, or cyan edges, switch to green, magenta, or another key color far from the subject's palette.
The identity card should use an opaque canvas of 1280×720:
- Save the original RGBA concept art without modifying colors directly on the master;
- Composite the character onto the selected key color;
- Export as a fully opaque PNG;
- Re-read the file to confirm that canvas size, Alpha, and edge colors have not changed.
Do not hand a transparent PNG directly to the model and assume the transparent areas will be preserved. The model may interpret transparent areas as black, checkerboard, white borders, or a new scene.
Composition
The identity card only handles character constraints, not the final sprite sheet dimensions. The character should not fill the entire card: attacks extend weapons, hits cause leaning back, and death transitions from a vertical silhouette to a horizontal one. Leaving space during the input phase is far more reliable than cropping hands and feet and fixing issues after generation.
It is recommended to define three invariants first:
- Unified source facing direction;
- Unified approximate foot position;
- Unified visible height range for standing characters.
These three values will become part of the sprite sheet manifest during post-processing.
Generate Action Source Clips with Gemini Omni Flash
The model used in this article is Gemini Omni Flash, with model code gemini-omni-flash-preview. It provides image-to-video capabilities via the official Gemini Interactions API. The model is still in Preview; for formal production, record the model code and generation date. Do not just write "Gemini."
Install the official SDK:
Place the key in the environment variable GEMINI_API_KEY. Do not write it into scripts, prompts, or generation logs. Below is a minimal official API call skeleton:
=
=
=
For large outputs, switch to URI delivery as per the official documentation and download locally immediately after generation. Temporary download links are not an asset preservation solution.
Prompts Are Not Action Names, But Constraint Lists
Writing "let the character attack" is far from enough. The model will independently add camera movement, enemies, particles, a second attack, or clothing changes. An effective prompt must contain at least six layers of information:
- Identity Lock: Face, hairstyle, body proportions, clothing colors and layers, signature props;
- Style Lock: Continue using the 2D painting language of the input image; do not convert to live action, 3D, or another rendering style;
- Frame Lock: Single shot, fixed camera, full body, unified facing, and safe zone;
- Background Lock: Key color remains uniform from start to finish; no ground, shadows, scenery, or VFX;
- Timeline: Hold, wind-up, action, end pose, recovery, or hold on final frame;
- Failure Zones: Extra limbs, second weapon, turning around, costume changes, blur, out-of-frame, and repeated actions.
You can start with the following sanitized template:
10-second, 16:9, single continuous shot for a 2D game character action.
Treat the input image as an immutable character model sheet and color key.
Keep exactly the same face, hairstyle, body proportions, garment colors,
garment layers, footwear, and signature prop in every frame. Preserve the
same hand-painted 2D illustration style. Never turn the character into live
action, 3D, cel animation, or a different costume.
The full body and the complete prop remain inside the central safe area.
The character faces screen-right for the entire clip. The camera is locked:
no pan, tilt, zoom, shake, lens change, perspective change, or scene cut.
The background remains a perfectly uniform solid key color. No floor,
shadow, particles, blood, text, logo, watermark, attacker, or projectile.
[0.0-1.5s] Hold the input guard pose with only subtle breathing.
[1.5-2.0s] Prepare one clear action while keeping both feet controlled.
[2.0-2.7s] Perform exactly one readable action toward screen-right.
[2.7-3.3s] Hold the completed action pose briefly.
[3.3-4.8s] Recover along a physically plausible reverse path.
[4.8-10.0s] Return as closely as possible to the input guard pose and hold.
Use sharp outlines and a high-shutter look. No motion blur, depth of field,
compression smearing, extra limbs, duplicated props, costume recoloring,
face redesign, body-scale change, background flicker, or second action.
No dialogue, voice, music, or sound effects.
Attacks, hits, and deaths cannot be handled by simply replacing a verb. Each action type must redefine body force, prop state, and ending behavior:
- Attack: Only one active action occurs; specify direction, weapon, force path, and recovery;
- Hit: Attack comes from off-screen; no attacker or projectile appears; the character does not counter-attack and returns to idle;
- Death: Specify losing balance, sinking down, and finally stopping; the character must not stand up or return to guard in the final segment;
- Idle: Only allow small movements like breathing, hair tips, and hem swaying; no walking, turning, or camera drift.
Gemini Omni Flash currently lacks independent negative prompts, system instructions, temperature, or top_p controls. All prohibitions must be written in the standard prompt. The official documentation also recommends explicitly stating single-shot and no-cut rules, otherwise the model tends to organize multiple shots on its own.
One Video Per One-Time Action
Do not let the same video sequentially handle idle, attack, hit, and death. While this seems to save API calls, it amplifies three problems:
- The model must maintain identity for a longer duration;
- Instant cuts, turning around, or second costumes may appear between actions;
- Failure in any one action drags down the entire video.
A more robust unit is "one character × one one-time action." Generate attacks, hits, and deaths separately. Derive idle animations preferentially from stable hold segments of source clips that have already passed identity review.
Extract Action Frames from Video
Select Action Windows Before Processing All Frames
Upon receiving the MP4, do not immediately keyframe at the full frame rate. First, generate a contact sheet at 2–4 FPS to quickly check the entire video:
- Which frame does the action truly start?
- Does a second attack or extra recovery appear?
- Where does the face, clothing, or prop change?
- Does it generate an off-screen attacker, VFX, double, or wrong projectile?
- Does the character stand up after falling, or suddenly switch to a different death pose?
- Which segment is clearest at in-engine size?
When selecting windows, prefer short over long. Do not keep useless frames just to fill the model's output duration. An attack in-game usually requires less than one second; the long hold segments provided by the model are merely to help find stable entry and exit points.
For example, after selecting the action window, extract candidate frames:
Identity Drift Warrants Discarding the Whole Clip
If any of the following items change significantly within the usable window, regenerate:
- Face, age appearance, or hairstyle;
- Main clothing color, collar, cuffs, and clothing layers;
- Armor plate arrangement, shoulder guard structure, and cloak pattern;
- Weapon length, container count, ropes, or signature hanging props;
- Body proportions, finger count, or limb connection relationships.
Do not take half from two videos with slightly different identities and stitch them into one action. A static contact sheet might not reveal the issue, but continuous playback will form jumps more obvious than slight drift in a single clip.
Blue Screen Keying and Despill
Simply deleting pixels "close to blue" will leave blue edges on hair, hems, and weapons, and will also swallow semi-transparent brushstrokes. A more robust process involves two steps.
Step 1: Generate soft Alpha using color distance.
Let the current pixel be C and the estimated background color be B:
d = ||C - B||
t = clamp((d - low) / (high - low), 0, 1)
alpha = smoothstep(t)
The background color can be the median of clean frame edge pixels. low and high determine the transition width between fully transparent and fully opaque areas. Using soft Alpha, rather than a single threshold, preserves hair strands and painting edges.
Step 2: Reverse-solve background contamination in the edges.
Anti-aliased edges are already a mix of foreground and blue screen. You can approximately reverse-solve using the compositing formula:
observed = alpha * foreground + (1 - alpha) * background
foreground ≈
(observed - (1 - alpha) * background) / max(alpha, epsilon)
After reverse-solving, suppress residual blue dominance, otherwise blue edges in high-speed actions will look like extra VFX. This must be combined with the character's color palette: if the character inherently contains a lot of key color, automatic despill will harm the subject. In that case, change the key color before generation rather than stacking thresholds afterwards.
Frame Sampling by Motion Amount
Uniform frame sampling wastes slots on static segments while leaving only a few vastly different poses for the fastest attack phases. A better approach is to allocate frames based on cumulative motion.
A practical implementation is:
- Extract candidate frames at
12–24 FPS; - Downsample each frame into a low-resolution RGBA signature;
- Multiply by Alpha first to prevent the transparent background from participating in difference calculations;
- Calculate the mean absolute difference between adjacent signatures;
- Add a small base weight to each adjacent interval to prevent static segments from losing all frames;
- Cumulate the weights, then select target frames at equal intervals of cumulative motion;
- Force-keep the first and last frames of the action window.
Pseudocode:
energy[i] = base + mean(abs(signature[i] - signature[i - 1]))
cumulative = prefix_sum(energy)
targets = linspace(0, cumulative.last, wanted_frame_count)
selected = nearest_indices(cumulative, targets)
This way, fast swings get more frames, short pauses are still preserved, and the final playback is smoother than uniform sampling with the same frame count.
The following specifications can serve as a starting point, not a universal standard:
| Action | Final Frame Count | Playback FPS | Duration | End Behavior |
|---|---|---|---|---|
| Idle | 18 | 10 FPS | 1.8 sec | Loop |
| Attack | 24 | 30 FPS | 0.8 sec | Return to Idle |
| Hit | 18 | 24 FPS | 0.75 sec | Return to Idle |
| Death | 30 | 24 FPS | 1.25 sec | Hold on Final Frame |
Frame counts should serve action readability and runtime pacing, not mechanically increase for "smoothness." Source clips already slightly redraw details over time; overly dense sampling increases volume without necessarily improving perception.
Construct Idle Loops from Stable Windows
Video models can usually maintain an approximately static segment at the beginning or end, but the start and end of the entire video are not the same image. Connecting the start and end directly will cause noticeable jumps.
A low-cost method is to extract ten stable micro-motion frames from the beginning of an already approved source clip, and then create a ping-pong loop:
forward = [0, 1, 2, ..., 9]
backward = [8, 7, 6, ..., 1]
idle = forward + backward
Do not repeat the two turning endpoints, or the endpoints will hold for two beats. This ping-pong is only suitable for reversible micro-motions like breathing, hair tips, and hem swaying; actions with clear temporal direction like walking, swinging, or cloth falling cannot be reversed into loops.
Idle animations should preferably come from the same accepted source clip as the attack, reusing the same crop, scale, and foot baseline. Regenerating a separate idle video might yield a different face or costume.
Unify Canvas and Runtime Coordinates
Unified Crop, Scale, and Foot Baseline
The most underestimated problem after keying transparent frames is canvas jitter. If each frame is centered and scaled according to its own bounding box, the character will appear to grow and shrink in place, and the body center will drift left and right.
The correct approach is to treat the "action" as the normalization unit:
- Calculate the Alpha bounding box for all selected frames;
- Find the joint range of these bounding boxes;
- Calculate the crop and scale only once;
- All frames use the same crop, scale, and horizontal position;
- Each frame only performs limited vertical alignment based on foot position;
- Paste the result into transparent cells of the same size.
Validation used 1024×1024 cells and a unified foot line. Specific pixel values can vary, but a single project must have one clear convention.
Standing Actions Share the Same Scale Tier
Attacks usually provide the most complete standing motion range. Save its crop, scale, and baseline as the character's standing scale tier. Idle and hit actions reuse this tier to avoid sudden height changes when switching between the three actions.
Do not "fill the canvas" by re-scaling for each action segment. A sprite sheet looking larger in isolation does not mean it is more correct in a combat scene.
Death Actions Use Joint Cropping
Death transitions from a vertical silhouette to a horizontal one. Forcing a standing crop usually cuts off the head, feet, cloak, or weapon. Death actions should use their own joint bounding box.
Joint cropping may make the standing start appear slightly smaller. The runtime can provide bounded display.scale and offset to restore a visible height close to the standing action; this compensation only solves layout differences caused by joint cropping and cannot mask severed limbs, truncated weapons, or wrong composition.
Foot Baseline Is Not Necessarily the Lowest Alpha Point
For characters with short hems and normal weapons, the bottom of the Alpha bounding box can be approximated as the foot base. Long spears, floor-length cloaks, tails, or hanging containers may extend lower than the feet. In such cases, additional foot masks or manual anchors are needed. Wrong anchors will cause the character to bounce up and down to align with props.
Fix Source Facing, Decide Mirroring at Runtime
All source frames should adopt the same facing, e.g., uniformly facing screen-right. Each action manifest explicitly records sourceFacing, and the runtime decides whether to mirror based on character positioning.
Do not have some actions facing left and others facing right during generation, then rely on filenames or eyesight to guess. Mixing attack directions, weapon grips, and hit leans makes runtime double-flipping difficult to debug.
Package Sprite Sheet and Action Manifest
Use standard static WebP sprite sheets at runtime, not Animated WebP. Static sprite sheets are easier to preload, address frame-by-frame, pause on specific frames, and share resources between normal and reduced motion modes.
A simple directory structure is:
animations/
└── actor-a/
├── idle/
│ ├── manifest.json
│ └── sheet.webp
├── attack/
│ ├── manifest.json
│ └── sheet.webp
├── hit/
│ ├── manifest.json
│ └── sheet.webp
└── death/
├── manifest.json
└── sheet.webp
The action manifest should at least contain:
cues are more important than "how many milliseconds this animation lasts." Combat logic should align damage, hit VFX, hit reactions, and sound effects with the contact frame, rather than guessing the action midpoint. When animation speed is adjusted later, only the relationship between frames and time needs re-interpretation; combat results do not need to change.
Three generation-period artifacts should also be kept alongside the sprite sheet:
- Contact Sheet: View all frames at once to review identity and clothing drift;
- In-Engine Preview: View under real background, real display size, and real playback speed;
- Machine Report: Record frame sampling timestamps, crop, scale, baseline, metrics, and output hash.
These are production evidence and do not necessarily all enter the runtime resource directory.
Quality Acceptance
Automated Checks Are Suitable for Discovering Structural Errors
- Original video duration, resolution, frame rate, and decode status;
- Whether the target frame count is complete;
- Whether each cell's canvas size matches the sprite sheet rows and columns;
- Whether the Alpha bounding box touches the canvas edges;
- Foot baseline drift after normalization;
- Mean, P95, and maximum displacement of character centroids between adjacent frames;
- IoU of adjacent silhouettes;
- Average sharpness of the action window and loss relative to the original art;
- Color drift of the key color background over time;
- Silhouette and color difference between start and end frames to judge loopability;
- Whether WebP can be truly decoded by the target browser.
Foot baseline drift can initially use ≤2 px as the target for 1024² cells, then calibrate based on character size and artistic style. Centroid displacement and silhouette IoU should not have a universal threshold across actions: fast swings, restrained hits, and deaths inherently have different motion distributions and are better compared against baseline actions of the same type.
Artistic Quality Still Requires Manual Per-Frame Review
- Are the facial features, hairstyle, and age appearance still the same character?
- Have the main clothing color, collar, cuffs, and layers changed?
- Have armor plates, cloak tears, patterns, and accessories been redrawn?
- Have props like weapons, containers, and ropes been added/removed, bent, or switched hands?
- Have fingers, joints, and limb connections collapsed in contact frames?
- Is there only one action, and do the direction and force align with gameplay?
- Are there extra attackers, projectiles, particles, or doubles?
- Are the wind-up, contact, and end poses clearly readable at actual in-game size?
The review order should be: contact sheet first, then loop playback, and finally placing it in a real combat scene. Looking only at large transparent images amplifies details that do not affect the in-engine experience and misses scale jumps and timing issues that truly impact gameplay.
Common Failures and Handling
The model performed two attacks in a row.
The prompt explicitly states "only one action in the entire clip," but the model may still append a second attack to fill the duration. Do not force-use the entire clip; cut off the second action. If the first action is incomplete, regenerate.
An attacker or energy ball appears out of nowhere during a hit.
The prompt must specify that damage comes from off-screen and prohibit attackers, weapons, projectiles, and VFX. If the hallucination appears only before or after the action window, shorten the time window; if it covers the contact frame, discard the whole clip.
Clothing color or armor structure changes during the action.
Change the identity description from "same character" to an itemized clothing list, and declare that the input image serves as both an immutable character design sheet and color palette. If drift persists, regenerate; do not patch a few frames from a bad clip.
Blue halos appear on edges.
Process both soft Alpha and despill. The prompt must also prohibit VFX, motion arcs, and background gradients, as the model might paint blue motion trails as part of the subject.
The character grows and shrinks after per-frame cropping.
Switch to using a joint crop and single scale for the entire action segment. Per-frame, only allow limited anchor translation; do not allow re-scaling.
Long weapons or mounted props extend beyond the standing crop.
Prioritize expanding the identity card safe zone or action cells. If the character scale is correct but only a few frames of props slightly exceed the boundary, perform bounded smooth displacement on the entire frame while keeping the character scale unchanged; do not shrink the entire character to accommodate a weapon.
The character stands up at the end of the death animation.
The death prompt must explicitly state that the final segment remains held, and post-processing must cut off before "standing up." Set the last frame to hold at runtime; do not loop the entire death segment.
The idle loop jumps at the start and end.
Do not directly connect the start and end of the generated video. Select an internal stable hold segment and use a ping-pong loop without repeating endpoints; if the action has irreversible motion, discard this asset and use a static idle or skeletal micro-motion instead.
Slight redrawing is visible when zoomed in, but normal in-engine.
First confirm the delivery size. Small-sized combat actors allow local brushstroke variations that do not affect identity recognition, but the face, clothing silhouette, and signature props must not drift. Quality gates should revolve around the usage scenario, not strive for every frame to independently serve as character concept art.
Meshy + Blender 3D Intermediate State Review
To confirm whether the 3D route could provide more stable frame sequences, the following production chain was fully validated:
Character art
→ Standard T-pose Multi-view
→ Meshy Generates Textured 3D Character
→ Auto-rig Standard Biped Skeleton
→ Generate or Reuse Actions and Mount Weapons Separately
→ Blender Fixed Camera Batch Render
→ Transparent Frames, WebP Sprite Sheet, and Action Manifest
This validation did not stop at model previews. The trial actually produced a bindable model, idle, attack, hit, and death animations,
and completed two-handed weapon mounting, orthographic camera auto-framing, transparent background rendering, and browser preview. The four sprite sheets totaled 31 frames,
all entered into a unified 1280×720 canvas, and passed safe margin checks. Actions were recognizable, frame-to-frame structure was stable, and offline re-rendering
was reproducible. From an engineering perspective, this pipeline has run successfully.
These frames were ultimately not adopted because they did not pass the artistic quality gate.
Pipeline Passed, Final Product Failed
| Acceptance Layer | Actual Result | Judgment |
|---|---|---|
| Model & Textures | Multi-view input successfully generated usable mesh, with complete front, side, and back structures | Pass |
| Skeleton & Animation | All four action types playable, attack actions and two-handed weapon relationships recognizable | Pass |
| Rendering & Packaging | Fixed camera, transparent frames, unified anchors, sprite sheet, and action manifest all produced | Pass |
| In-Engine Size Comparison | Character identity, silhouette temperament, and painting texture significantly weaker than concept art | Fail |
This result is easily masked by "the animation actually moves." Looking at the 3D candidates alone, they are complete game characters; the gap only becomes concentrated when placed in the same scene, same foot line, and same display size as the concept art assets. Concept art features a wide, low combat silhouette, dark shading, and tattered, feathery, irregular edges on clothing and armor; 3D candidates are more upright, more symmetrical, with neat clothing layers, continuous silhouettes, and more uniform lighting and materials. The action semantics are preserved, but the visual identity of the character is not.
The First Quality Loss Occurs Before Modeling
To improve the success rate of auto-modeling and rigging, inputs must first be organized into standard bipedal characters: T-pose or A-pose, with clear limbs, consistent front, side, and back views, and weapons, cloaks, ribbons, and complex occlusions removed. This is a reasonable 3D production input, but not necessarily a faithful one to the concept art.
The most recognizable parts of the concept art are often precisely these "non-standard" elements: skewed center of gravity, composition stretched by weapons, tattered armor, overlapping plates, exaggerated hand/foot proportions, and ink blocks that only exist in the current perspective. When organizing the character into a clean neutral stance, what is deleted first is not noise, but silhouette language. Even if multi-views are supplemented, the added back and side views can only ensure view consistency, not automatically recover the design that has been normalized away.
This also explains why "providing more views" did not solve the final problem. Multi-views reduce 3D reconstruction ambiguity but do not guarantee that the new settings retain the expressiveness of the concept art.
The Second Quality Loss Occurs in the 2D-to-3D Translation
Comparing the trial's multi-view inputs and Meshy models shows that clothing categories, armor positions, and character proportions are largely preserved, but details are continuously averaged: faces become closer to generic templates, armor and clothing folds lose layers, edges change from broken brushstrokes to continuous surfaces, and stains and local color differences are compressed into more uniform textures. The mesh is very stable during animation, but it is stable for a character that has become more generic.
This is caused by the different goals of 3D reconstruction and 2D painting. Auto-modeling prioritizes closed, continuous, texture-mappable, and riggable surfaces; concept art can rely on gaps, soft edges, large dark areas, and perspective-limited exaggerations to exist. The former interprets the latter's painting techniques as geometry or texture, and normalizes the undetermined parts. Increasing frame-to-frame stability does not make up for this information loss.
The Third Quality Loss Is Amplified at In-Engine Size
Blender rendering already uses transparent backgrounds, fixed orthographic cameras, tone compression, and unified downsampling, but these processes can only unify output specifications; they cannot turn 3D surfaces back into concept art brushstrokes. When scaled down to actual in-game sizes, faces and fine textures become almost invisible; what truly determines perception is the outer silhouette, light/shadow blocks, and action center of gravity. Concept art has clear designs in these three aspects, while 3D candidates present a clean, continuous, and standard game-model visual language, making them look like they come from a different asset library than the 2D characters on screen.
Outlines, paper textures, color grading, or NPR materials can change the rendering surface, but cannot recover the cloak shapes deleted in the input stage, nor automatically reconstruct the broken, irregular edges, asymmetric color blocks, and facial expressions of the concept art. To achieve the same quality, one must return to the character level to retopologize, repaint textures, design custom materials, and manually retouch keyframes. This can become an effective 3D-to-2D art pipeline, but it is no longer the "hand concept art to Meshy, then frame in Blender" automatic solution.
What This Route Is Still Suitable For
In this trial, the action masters, weapon trajectories, cameras, and sprite sheet tools all retain value for continued use. 3D results are suitable for:
- Action rhythm and center of gravity reference;
- Two-handed weapon grip point and trajectory verification;
- Camera, placeholder, and collision range pre-visualization;
- Projects where the final art style already accepts 3D, 2.5D, or unified NPR.
However, for the final pixels based on hand-drawn 2D concept art as the identity baseline, the acceptance order should be "look like the character first, then check if the action is stable." These 3D candidates are exactly the opposite: actions and engineering specs are qualified, but the visual quality of the character itself has significantly declined. Therefore, they are retained as action research and did not enter formal assets. This failure also determined the acceptance priority for subsequent workflows: maintain character identity and 2D art style first, then handle frame-to-frame stability.
Production Records and Sanitization
To enable future reproduction, it is recommended to save:
- Model code, generation date, and official SDK version;
- Sanitized prompt templates and action differences;
- Content hashes of input identity cards and original videos;
- Reasons for adoption and rejection;
- Final action time windows and candidate sampling rates;
- Key color, keying thresholds, and despill versions;
- Crop, scale, foot baseline, and source facing;
- Frame counts, playback FPS, action cues, and sprite sheet hashes;
- Manual review conclusions and target runtime screenshots.
Input concept art should only use self-owned works or materials with clear authorization. Officially generated videos carry invisible SynthID origin markers; internal production records should continue to retain model and input sources, not using "already sampled into frames" as a reason to discard source information.
Public experience documents do not need to save:
- API keys, authorization headers, and temporary download links;
- External task IDs, internal project IDs, and supply chain routes;
- Real character names, world-building terms, and unreleased asset paths;
- Single costs, account information, and private storage addresses;
- Complete prompts and original images that can reverse-engineer unreleased content.
Project-internal records can retain full production logs; public documents only extract methods, boundaries, and transferable parameters.
Minimum Acceptance Checklist
Before Generation:
- Concept art is complete, with no cropping of feet, hems, or props;
- Key color has sufficient distance from the character's main palette;
- Identity card is fully opaque, with the character in the safe zone;
- Prompts lock identity, style, camera, background, timeline, and failure zones separately;
- Each video is assigned only one primary action.
During Processing:
- Review low-FPS contact sheet first, then determine action windows;
- Discard entire windows with identity or clothing drift;
- Use soft color keying and despill, do not perform hard-threshold keying;
- Sample frames based on cumulative motion;
- Calculate crop and scale only once per action;
- Standing actions share the scale tier; death actions use joint cropping;
- Source facing, foot baseline, and end behavior are written to the manifest.
Before Ingestion:
- Contact sheet reviewed frame-by-frame for face, clothes, armor, and props;
- Sprite sheet frame count, grid, and manifest are consistent;
- No severed limbs, truncations, blue edges, or transparent dirt;
- Playback passes under real background and real display size;
- Attack contact frames, hit reactions, and death hold frames align with runtime pacing;
- Browser can decode WebP, with static fallback frames on failure.
Conclusion: The Final Product Is an Animation Contract
Video models solve "making the same 2D character produce continuous motion"; game production must also solve "which frames are usable, how to align, how to play, when to hit, and how to degrade on failure."
The truly reusable result is not a single generated video, but this set of stable constraints:
- Same identity card;
- One source clip per action;
- Clean time windows;
- Adaptive frame sampling;
- Unified scale and anchors;
- Explicit playback manifest;
- Automated metrics plus manual identity review.
As long as this contract is stable, video models can be upgraded, action counts can increase, and the runtime does not need to be rewritten.
References
- Gemini Omni Flash: Official Guide to Generating and Editing Videos
- Gemini API: Overview of Video Generation Models
Keywords: Game Development, Character Animation, Action Frames, Sprite Sheet, Image-to-Video, Gemini Omni Flash, Chroma Key, Adaptive Frame Sampling, Foot Baseline, Sprite Sheet, WebP