Projects(Backup)
R&D Engineer @ Media Lab, Next Media Animation
- the Facial expression system, aHEAD
- Deferred Rendering system
- Binary space partition tree system for scene managment
- the In-house Game Engine development
- Scene tool for game design of the next generation engine
- Real-time toon & silhouette shader for cartoon rendering in 3ds Max (V-Ray, Scanline, Quicksilver)
- Augmented Reality System for TV News
- Mobile application development (iOS/Objective-C)
|
|
Latest News... I am currently developing the application in Objective-C. I used LiveDriver SDK for tracking facial express to drive the cartoon avatar. For recording the performance on iPhone screen, the EVERYPLAY SDK is used to solve it. FMOD SDK is also integrated in the application for recording voice via microphone. Jack @ 2013/06/01
|
As a R&D programmer since October 2009, I have developed the facial expression system, aHEAD, to compute the weights of the facial blend shape efficiently and generate stable animation for different facial models from a facial motion capture data stream. The system has been applied to production pipeline of next generation animated news in Next Media Animation.
|
Our facial animation work had already applied to the Japanese movie, "TRAVELERS". (2013/04/13) Click the following links to know the detail information. [ Movie Website ] [ BackScene Clip ] (0:30~0:40) |
I am also interested in real-time rendering and game programming. I have implemented the deferred rendering system for real-time news animation tools of NMA. For rendering efficiency, I have implemented binary space partition tree system for scene management of the in-house game engine, TheFly, we used in NMA. I am one of the team members of developing the game engine and my major work is to improve the rendering efficiency and quality using the next generation game development technologies.
I have experiences in programming with Maya and 3ds Max, including script programming and plugins development. Recently, I have implemented a real-time silhouette shader for cartoon rendering which is based on the V-Ray renderer. I implemented the shader with HLSL but integrated to V-Ray renderer in 3ds Max to b the boost rendering performance and keeping the quality.
For the better news performance on Next TV, We also implemented Augmented Reality system for real-time interaction between the news anchor and the animation. By the pattern recognition and feature tracking, the news anchor can show the different patterns to display animation to interact with it.
Constraint-based 3D Model Deformation
2007/07 - 2008/06 M.S. thesis (advisor: Prof. Ming Ouhyoung)
We proposed a 3d model deformation technique, which is based on rigging by building the skeleton of the model by user. The approach not only preserves the shape details, but also derives more natural deformation by adding several high-level constraints for more natural results of mesh deformation. The above constraints include preserving the length of bones, limitation of joint angle, and rigidity. Users can modify the pose of a model by direct manipulation on skeleton, and get the realistic deformed mesh. We optimize a given energy function according to the position and weight of skeletons. Our results show the following advantages: convenience of manipulation on rigging and deformation, high-level constraints and more natural and realistic deformed mesh. Our research is inspired by a research paper done by Kun Zhou et al. in Microsoft Research Asia.
(presented in CGW 2008)
We proposed a 3d model deformation technique, which is based on rigging by building the skeleton of the model by user. The approach not only preserves the shape details, but also derives more natural deformation by adding several high-level constraints for more natural results of mesh deformation. The above constraints include preserving the length of bones, limitation of joint angle, and rigidity. Users can modify the pose of a model by direct manipulation on skeleton, and get the realistic deformed mesh. We optimize a given energy function according to the position and weight of skeletons. Our results show the following advantages: convenience of manipulation on rigging and deformation, high-level constraints and more natural and realistic deformed mesh. Our research is inspired by a research paper done by Kun Zhou et al. in Microsoft Research Asia.
(presented in CGW 2008)
The Detection of the Image Orientation
2006/09 - 2007/07 Industry-University Cooperation
A solution to correct the orientation of massive photo according to some important features which could be the orientation reference, like faces, sky, grass...,etc.. Face Detection by OpenCV and Skin Detection by our lab are also used in the work. we train lots photos by SVN to get the better results on detection and correction of the image orientation.
A solution to correct the orientation of massive photo according to some important features which could be the orientation reference, like faces, sky, grass...,etc.. Face Detection by OpenCV and Skin Detection by our lab are also used in the work. we train lots photos by SVN to get the better results on detection and correction of the image orientation.
Flag Flag Revolution (self-made Wii game)
2007/05 - 2007/06 Term project of Virtual Reality
We developed a video game with Wii remote controllers. Players use the remote controllers as the red and white flags and let the red and white flags to the right position when hearing the command. At the commencement of the game, the command will be given one at a time with the initial speed and it will become faster and faster later on in the game. Players should do the correct response to either raise up or put down their hands with controllers according to the commands are given.
We developed a video game with Wii remote controllers. Players use the remote controllers as the red and white flags and let the red and white flags to the right position when hearing the command. At the commencement of the game, the command will be given one at a time with the initial speed and it will become faster and faster later on in the game. Players should do the correct response to either raise up or put down their hands with controllers according to the commands are given.
2D Image Morphing Implementation
2006/10 - 2006/11 Term project of Computer Graphics
Morphing is implemented by warping and cross-dissloving. We use three types of warping method to process, mesh warping, field warping, point warping.
Morphing is implemented by warping and cross-dissloving. We use three types of warping method to process, mesh warping, field warping, point warping.
A Re-convertible Color Transfer and Its Quality Measuring with Statistic Approach
2005/06 - 2006/06 B.S. thesis
It combines the color transfer process with information hiding techniques. We embed crucial image attributes, such as the mean and the variance, into the resultant image, after the color transfer process to ensure the reversibility. the algorithm has three advantages : memory saving, repetition color transfer, and fast computation.
(presented in NCS 2005)
It combines the color transfer process with information hiding techniques. We embed crucial image attributes, such as the mean and the variance, into the resultant image, after the color transfer process to ensure the reversibility. the algorithm has three advantages : memory saving, repetition color transfer, and fast computation.
(presented in NCS 2005)
Drawing Together (route tracing by GPS device)
2008/08 Night market Workshop by Media Lab, M.I.T
We implemented in python to trace the users' route by cell phones with GPS and projected it as a big picture on the building to have lots of interesting interactions with strangers.
We implemented in python to trace the users' route by cell phones with GPS and projected it as a big picture on the building to have lots of interesting interactions with strangers.
