Immersive Visual Effects / Project 1

on
23.04.2025 - 28.05.2025 (Week 1 - Week 6)
Ng Kar Yee / 0367743
Bachelor of Design (Honours) in Creative Media
Immersive Visual Effects / Project 1

LECTURES

Week 1⇒ Introduction to Visual Effects

On our first day, Mr Fauzi gave us a brief introduction on the module and told us what to expect for the current semester. He gave us a brief introduction on the module information booklet and discussed about the weekly lesson plan. Besides, assignment, assessment and weightage were also discussed.



The History Of VFX: How The Morphing Effect Evolved 
  • The first visual effects had absolutely nothing to do with the intricacies of CGI, those were simple in-camera effects that did not allow for a lot of crazy ideas. 
  • In the late 19th-early 20th century, VFX were mostly created using jump-cuts and stop-motion. Because there was essentially no video editing software, filmmakers had to create the little visual tricks while filming. The effect of disappearing was one of the easiest to do because all it required was a good jump-cut.
  • Film directors and FX artists also used lighting effects and played with colored lights to make SFX makeup on actors’ faces appear or disappear as the lighting setup changed.
The Future of VFX
  • Morphing VFX you can see in the 2010s and early 2020s movies already seem to be the absolute peak of photorealism in CGI and frankly, it’s hard to imagine how much further the artists can push believability in VFX.
  • The number of tools the artists can use now is absolutely dizzying and the teams always keep creating something new and exclusive to go beyond the possibilities of the existing applications.
  • It’s been only 40 years since ILM’s first attempt at photorealistic CGI face morphing in Star Trek 4: The Voyage Home and now the industry is already at insane Venom-level CG transformations.

Week 2⇒ Camera Tracking 

For week 2, we had a tutorial on camera tracking. Mr Fauzi shares with us a tutorial video on how to move the camera of an object around to make it more interesting. 


After studying the tutorial video, Mr Fauzi instructed us to apply what we learn on an exercise. He provided us a template to help us understand how does it work. Then. he tasked us on recreating the template and add in our own short animation. The only requirement is to do at least a 10 second video and add in any background music to match the animation. After understanding the task and requirements, I proceed on doing the exercises.

FINAL EXERCISE


Youtube Link

Week 3⇒ Zenith And Nadir

For week 3, we had a lecture on Zenith and Nadir.



Figure 1.0 Zenith & Nadir

In astronomy, the Celestial Sphere is an imaginary sphere surrounding the Earth onto which all celestial bodies (like stars and planets) appear to be projected.

Zenith:

  • The Zenith is the point in the sky that is directly above an observer.
  • It is located 90° from all points on the horizon.
  • If you stand outside and look straight up, you’re looking at your zenith.
  • It changes based on your location on Earth.
Nadir:
  • The Nadir is the point that is directly beneath the observer, on the opposite side of the zenith.
  • It is also 90° below the horizon.
  • Though we can't see it (since it lies beneath the ground), it’s an important reference point in celestial coordinate systems.
Relationship with the Horizon:
  • The Horizon separates the visible half of the celestial sphere from the invisible half.
  • The Zenith lies at the highest point in the sky relative to the horizon.
  • The Nadir lies at the lowest, directly downward from the observer.
Zenith and Nadir camera angles

Zenith

  • The point on the Celestial Sphere directly above the observer.
  • The zenith is diametrically opposite to the nadir. In general, the highest state or point.
  • Originally derived from the Arabic semt ar-ras meaning way over the head: ME senyth; ML cenit.
Nadir
  • The point on the Celestial Sphere directly below the observer.
  • The nadir is diametrically opposite to the zenith. In general, the lowest state or point.
  • Originally derived from the Arabic nazir meaning the corresponding opposite.

Week 4 ⇒ Compositing Toolkit 2

What is a HDRI files?
  • The HDRI file is a digital image format. HDRIs are often created in photo editing software to enhance digital photographs that can then be converted to smaller files to share and upload.
  • HDRI is short for High Dynamic Range Image. Digital cameras only have a limited dynamic range — that’s why some areas of a photo appear darker than they do in real life. HDRIs give photo editors a chance to brighten the corners and create an image that looks more natural.
Pros and cons of HDRI files.

Advantages of HDRI files.

  • HDRIs can help you to customise and enhance your images to get the best results.
  • You can open and view them in a range of different photo editing programmes.
  • You can easily create HDRI files with Photoshop.

Disadvantages of HDRI files.

  • 32-bit HDRI files are too large to use the full range of Photoshop editing features and sharing options.
  • To use Photoshop you’ll first need to save a copy of your original, then convert it to a 16 or 8-bit image.


Mr Fauzi also shared a video tutorial on how to creating HDRI with the HDReye app in phone. Then we proceed to try it in class where we film the classroom area and an outside area beside the university campus lake. It was quite interesting as it was my first time recording a HDRI file. However sadly there were some limitations. Apparently the app can only be use in Apple IOS and we couldn't download the outcome in the free trial version. :(

Figure 1.1 Mr Fauzi And Sue Zhiun Recording The HDRI

Week 5 ⇒ Animation Techniques

For week 5, we were introduced to the Rotoscoping in After Effects. Mr Fauzi provided a few tutorial videos from Youtube. Rotoscoping is a visual effects technique used in video editing and animation where artists manually trace over motion picture footage, frame by frame, to create a matte or mask for an element, typically to isolate it from the background or to add effects.

Key Purposes of Rotoscoping:

1. Object/Subject Isolation 
  • Remove or separate a person or object from the background.
2. Replacing Backgrounds
  • Used when green screen wasn't available; helps replace sky, environment, etc.
3. Adding Visual Effects (VFX)
  • Add glowing effects, lightning, or other elements to moving objects.
4. Color Correction or Blurring
  • Apply changes to specific parts of a frame (e.g., blur someone’s face or change shirt color).
5. AR / Motion Tracking Prep
  • Prepares footage for tracking moving elements in 3D space.




Week 6 ⇒ Keying Overview

For week 6, we were introduced to the Keying Overview where we were taught how to remove a green screen in Adobe After Effects. We found out that the keying effect can cleanly remove the green background and make the subject blend smoothly into a new background. Mr Fauzi provided a tutorial videos from Youtube.

INTRODUCTION


PROJECT 1 ⇒ PRE PRODUCTION

For our first project, Mr Fauzi introduced Pre Production to us where we need to propose a series of idea and brainstorming into a slide presentation. For this project, we would be will be exposed to the history of visual effects techniques and master the concepts, processes, and tools to build a strong understanding of the building blocks of visual effects production as it applies to 360-degree or volumetric videos and Augmented Reality.

For this project, we are required to,

Instruction:
Students have to propose a series of idea and brainstorming into a slide presentation.

Students are to use information for the idea execution.
• create mood-board that reflects their concept/idea
• storyboard
• floor plan

Students will start with thumbnail sketches and test a wide range of conceptual ideas through digitization using the appropriate software. Students conclude by selecting and presenting their proposed channel bumper interpretation in the instructed format/s.

RESEARCH

To have a head start, I started doing some research on what can we do for this project.




IDEATION

After researching, I found out that projection mapping are very fascinating as it can blend in imaginations and captivates audiences because it transforms ordinary physical surfaces like buildings, models, sculptures, or even people into dynamic, animated canvases.

At our ideation process, Mr Fauzi came in and suggested that we could do either projection mapping or augmented reality (AR) for our project. He shows us a few projection mapping examples and also introduced AR to us as well as shows us websites like 8th wall where we can experiences AR games and projects. He stated that 8th wall can be a platform for us to build our project as well as we could bring in 3D assets or generate them in the website.  Then, he shows us a few personal projects as an example and inspiration of how projection mapping and AR are as Sue Zhuin and I was very new to these topics. 

After discussing with Mr Fauzi and getting a senses of how these two works, I thought to myself why should we just choose one... let's do both instead!! The reason behind choosing this bold and creative approach came from the examples and research we looked into. I found that projection mapping is really eye-catching and can grab the viewer’s attention, but it mostly offers a passive experience as people just watch without doing much. On the other hand, AR lets people interact, but it often depends on the environment around them, which might be plain or not visually interesting. Therefore I felt that by combining both projection mapping and AR, we can create something that’s both visually exciting and interactive, giving the viewers a chance to enjoy and engage with the experience at the same time.

With that in mind, I proposed 2 ideas, 

1. “Grow” (Projection Mapping + AR)

My first idea involved creating a real-life model of a vase, where projection mapping would be used to animate roots and leaves growing out of the vase. On top of that, an AR element would allow viewers to tap the screen to grow flowers, making the experience feel more alive and interactive. I wanted to name this concept “Grow” to reflect the idea of nurturing and visualizing growth, allowing the audience to experience the organic development of a living form through both projection and AR.

2. Interactive Touch-Based Model

The second idea was inspired by a video I came across (LightFarm in Research Section), where projection mapping was combined with Arduino sensors to allow physical touch interaction on a model. Instead of using a phone for interaction like in AR, viewers could physically touch the model to trigger projected responses. I was interested in exploring this kind of setup to create a more tactile and immediate interaction, merging technology and physical engagement in a more hands-on way.

Sue Zhiun on the other hand proposed a virtual try-on feature using AR to help viewers decide their OOTD (Outfit of the Day). After proposing our ideas to Mr Fauzi, he feedbacked that he found my ideas interesting and would like to further look into in. I also voice out that I was very interested in adding in Arduino sensors to our project if we had the opportunity XD however we quickly realised that we had a few limitations in skills and facilities wise.  

Limitations

  1. We don't have the skills in building 3D assets
  2. We don't have projectors for projection mapping
  3. We don't have the facility nor capability in using Arduino sensors
  4. Hardware limitations
Mr Fauzi tried to help us on borrowing a projector personally from his friends however for the Arduino idea, it was way out of our current capability. Therefore, we conclude on doing a projection mapping + AR idea project. He also suggested that Sue Zhiun and I can form a group if we both agree on doing the same idea. After discussing with Sue Zhiun, both of us agreed on grouping and started to brainstorm our idea. Mr Fauzi also recommended that we could explore topics on current issues like global warming to our project as it would be easily related to everyone.

Sue Zhiun and I decided to focus our project on the theme of global warming, using projection mapping to show three possible outcomes by visualizing the past, present, and two different future scenarios. Our goal was to raise awareness about environmental change and encourage viewers to reflect on how human actions impact the Earth over time.

Past

  • This scene represents a time when the world was still untouched by modern development—lush, green, full of flora and fauna, and clean air. It symbolizes the natural beauty and balance of the Earth before major human impact.
Present
  • In the present-day mapping, we show signs of urban development, pollution, and industrial activity. Smoke, haze, and the reduction of green spaces represent the ongoing environmental challenges caused by human progress.

Two Future Outcomes

1. Cyberpunk Future
  • A visually striking futuristic world that still has advanced technology but is actively taking action to protect and restore the environment. This outcome represents hope—what the future could look like if we make sustainable choices today.
2. Zombie Apocalypse Future
  • A dark and dystopian version of the future where the world has been neglected, leading to environmental collapse and lifelessness. This scenario warns of the consequences of doing nothing and letting climate issues worsen.

PROPOSAL WRITING

After having an idea of what we wanted to do we start to draft out our proposal in Canva.

Project Introduction

Climate change is one of the most pressing issues of our time. As the Earth continues to experience the consequences of global warming, it is vital to communicate this crisis in innovative and impactful ways. "Afterlight" is an immersive storytelling experience that combines projection mapping and augmented reality (AR) to visualize the journey of our planet from its lush past to its uncertain future. Through this installation, we aim to evoke emotional responses, foster awareness, and inspire action.

Problem Statement

Despite the widespread of scientific consensus and increasing environmental campaigns, public understanding and engagement with the issue of global warming still remains limited. Traditional methods of communication often fail to convey the urgency and complexity of climate change in ways that resonate with broader audiences. Therefore, there is a critical need for more engaging, immersive, and emotionally impactful mediums that can bring the realities of climate change to life and encourage proactive behavioral changes. Hence, this project seeks to address this gap by using emerging media technologies to create an evocative and educational experience.

Project Overview

"Afterlight" is a multi-phase narrative projection and AR experience that will be displayed on a sculpture shaped like a cluster of buildings. The sculpture serves as a physical canvas to illustrate the transformation of Earth over time. Projection mapping will bring the sculpture to life, while AR will add a magical and interactive layer, allowing the audience to explore the environment more deeply through mobile devices.

After finishing our proposal writing, we presented to Mr Fauzi and consulted his advices. He felt that the idea was interesting and just needed some minor changes. He suggest to find a never used project name as our current one Re-Earth (Original Name) felt widely used. Therefore we change from Re-Earth (Original Name) to Afterlight (Current Name). He also suggested to add in an artistic statement to our project.

Artistic Statement

This is the message behind Afterlight—an immersive art project that tells the story of Earth’s past, present, and possible futures. We created this work to help people feel the urgency of climate change in a more personal and emotional way. Our goal is not only to show facts but to touch hearts and minds, reminding everyone that the future of the planet depends on the choices we make today.

The idea started with a BBQ kettle, a symbol of global warming and Earth’s rising temperature. We use projection mapping to display scenes of Earth’s natural beauty in the past, pollution in the present, and two different futures: a hopeful cyberpunk world with clean energy, or a dark, burning future filled with chaos and destruction. With AR (augmented reality), visitors can use their phones to explore more—seeing flying cars, ruined buildings, or choosing between different endings to the story.

We made this project using recyclable materials, combining real textures with digital art, sound, and simple interactions. Our visual style is inspired by both nature and cyberpunk culture. Afterlight is a space for reflection and imagination—a place to think about where we came from, where we are now, and where we could be heading. Through this work, we hope to spark conversations and inspire people to take better care of our shared future.

FINAL PRE PRODUCTION

IVFX Proposal by BONG SUE ZHIUN

FEEDBACK

WEEK 3

  • Ideation is interesting and can look further into it

WEEK 5

  • Proposal overall is good
  • Focus on project topic as main title in presentation
  • Presentation skills can be improve
  • Don't state project weakness in project proposal
  • Start on basic idea first, don't start too big first
  • Need to refine the project name, try to not clash names with other projects

REFLECTION

Working on Pre Production has been a fun ride where I got to enhance my skills and knowledge in ideation and creative writing. Besides, it was new experience as it was my first time getting to know more about projection mapping and AR. Moreover, we have the opportunity to analyze and explore different type of related projects and see how profesional share their knowledge in the specific field. This project was challenging yet interesting for me as I need to combine creativity and creative writing skills however with the guidance I got from Mr Fauzi and my friends, I managed to finish the tasks.

Comments

Post a Comment