Figure 1. A few sample frames of animations from our method. The high-resolution images can be downloaded [here].
Painterly Animation with Video Content Extraction
Liang Lin, Kun Zeng, Han Lv, Yizhou Wang, Yingqing Xu, Song-Chun Zhu
Int'l Symposium on Non-Photorealistic Animation and Rendering (NPAR) Annecy, France, 2010.
[PDF]
Note: this work is done at Lotus Hill Research Institute ,
mirror page at Liang Lin's website
[Introduction]
We present an interactive video stylization system for transforming an input video into a painterly animation. Although
similar oil-painting effects can be generated manually by the
paint-on-glass technique, such animation production is not only very
laborious, but also requires considerable artistic skills. For example,
the Oscar-winning animation Man and Sea took
artists years to accomplish. In comparison, our interactive system allows
amateur players to produce painterly animations from real-life video
clips with far less time and efforts.
Our better results on painterly animation benefit from: a) advanced techniques in computer vision for video parsing, and b) advanced
painterly rendering techniques based on image parsing.
[Overview]
The
system consists of two phases: A Content Extraction phase to obtain
semantic objects in the video and establish dense feature
correspondence; and a Painterly Rendering phase to select, place and
propagate brush strokes for stylized animations based on the semantic
content and object motion derived from the first phase. Our system has
the following contributions: (1) It renders artistic style animation
using a diverse set of example-based brush strokes, and these strokes
are automatically selected according to the object classes. (2) It
sticks the strokes tightly to the object surface in the animations by
warping and shifting strokes in accordance with the transformation of
the object with dense feature correspondence in both textured and
textureless areas. (3) It reduces the scintillation effects by several
techniques: (i) confining the strokes inside each object; (ii) the
deferred rendering and backward completion for newly birth strokes, and
(iii) a damped system to stabilize strokes in space and time.
[Representative Results]
Figure 1. A few sample frames of animations from our method. The high-resolution images can be downloaded [here].
[Stylized Animations Demos]
DEMO1 [avi 19.5M] (from the Cartoon Movie: Princess Mononoke)
DEMO2 [avi 12.2M] (from the Movie: the Lord of the Rings)
DEMO3 [avi 17.6M] (from the Movie: the Lord of the Rings)
DEMO4 [avi 34.9M] (The lady sequence by a hand-held camera)
DEMO5 [avi 29.2M] (The LENA sequence originally proposed by Wang et al. 2004)
