AI-assisted Chinese Font Generation Process and Technology Optimization

This research project, conducted in NEXT Lab in collaboration with Apple Inc., aimed to significantly improve the efficiency of Chinese typeface design through AI-assisted tools rooted in human-computer interaction principles. The proposed workflow consists of the following phases:

1. Manual annotation of strokes for a curated set of ~710 Chinese characters.
2. Construction of a stroke database, incorporating annotated data, Ideographic Description Sequences (IDS), and normalized central moments for graphical and structural features.
3. Training semantic segmentation networks on labeled data for accurate stroke segmentation.
4. For the remaining 5,000–10,000 characters:
   • A diffusion model is used to generate high-quality glyph images.
   • The trained segmentation network segments strokes from the generated images.
5. Each segmented stroke is matched to database candidates using normalized central moments, enabling vectorized reconstruction of glyphs.
6. Professional designers then perform rapid adjustments to finalize the typeface design.

Through various user studies, we demonstrateed that this pipeline can improve design efficiency by over 30% without compromising quality. Our manuscript An intelligent font generation system based on stroke inference, mitigating production labor and enhancing design experience has been published on Expert Systems with Applications (ESWA) , in which I am the third author (second student author).

In this project, I was primarily responsible for:

1. Manual stroke annotation
2. Construction the stroke database using labeled data, which includes graphical, structural, and semantic features.
3. Design and training of semantic segmentation models
4. Development of stroke-matching algorithms based on normalized central moments to reconstruct vectorized Chinese characters
5. Collaboration in completing the manuscript.

On top of this research, I then conducted:

1. ControlNet for stroke intersection detection and segmentation, effectively removing the need for manual stroke annotation, which was submitted as a Chinese patent A Chinese character stroke extraction method and device based on ControlNet (一种基于ControlNet的汉字笔画提取方法及装置) and was under reviewed.
2. Refinement of the stroke database to include more graphical and structural information
3. Process refinement based on in-depth interviews with multiple professional Chinese font designers
4. Performing large-scale comparative and aesthetic evaluations

These works, under the supervision of Professor Kejun Zhang, served as the foundation for my master’s thesis at Zhejiang University: Research on Chinese Type Intelligent Design Based on Stroke Interaction