The authors tackle "Customized Multimodal Role-Play" (CMRP)—making a unified model generate both text and images that consistently match a specific character's persona, dialogue style, and visual appearance. They built RoleScape-20, a dataset of 20 characters with persona descriptions, style cues, and text-image interactions. Their method, UniCharacter, uses two-stage training: supervised finetuning on unified multimodal data, then character-specific reinforcement learning to optimize cross-modal consistency. With just 10 images plus interaction examples, the model learns the target character in about 100 GPU hours.
Main takeaways:
- New task: jointly customizing persona, dialogue style, and visual identity across text and image generation
- RoleScape-20 dataset with 20 characters including training/eval data for persona, style, and visual cues
- UniCharacter uses Unified-SFT followed by Character-GRPO (group relative policy optimization)
- Requires only 10 images plus interaction examples, takes ~100 GPU hours
- Substantially outperforms prior approaches on cross-modal consistency