Nilsletter #5 | AI Tracking, Web 3D magic, and Unreal's Cinematic Leap 🚀

Issue #05 | Date: September 22, 2023

Hello 3D enthusiast,

Welcome to the fifth round of the 3D Nilsletter! After a refreshing two-week holiday, I'm back and eager to share the latest and greatest from the 3D universe over the past 7 days. This includes fascinating insights and inspiration for your own projects. I trust you'll discover something truly exciting in this edition.

And now to this weeks of new stuff in the 3D space.

AI: Cotracking

CoTracker is a powerful point tracker for videos developed by Meta AI. It can track points accurately and robustly in challenging conditions, even when they are partially occluded or moving quickly. CoTracker can track points in real time and has the potential to be used in a variety of applications, such as virtual reality and augmented reality, video surveillance, autonomous driving, sports analytics, and medical imaging. You grab the open source code here.

WEB 3D: Lusion

There's this amazing website from a creative production studio based in the UK that's packed with cutting-edge web 3D tech. It's seamless, stunning, and truly showcases the potential of today's web technology. Could this be a glimpse into a future where websites transform into immersive 3D experiences, breaking free from the constraints of traditional 2D? It certainly seems like a thrilling possibility on the horizon.

Unreal Engine: Blue Dot - Behind the scenes

Three months ago, Unreal Engine unveiled "Blue Dot," a stunning photorealistic short film showcasing next-gen real-time cinematic human rendering. Recently, they shared insights into the creation process, addressing the challenge of the Uncanny Valley. This behind-the-scenes glimpse provides valuable details on their approach and the project's hurdles. Metahuman technology, initially a game-changer for in-game cinematic quality, is now getting ready for the big screen. It offers film directors unparalleled control, allowing adjustments and refinements months after the initial motion capture work. The future of cinematic creation is evolving rapidly.

NeRF: NeRF in Unreal together with VDB

Since the release of Unreal Engine 5.3, importing volumetric assets (VDB) has become a possibility. Notably, prior to this update, Unreal Engine already supported the import of NeRF captures. The convergence of these features opens up exciting creative possibilities, enabling the development of compelling and photorealistic scenes in real-time. Jason Key's recent demonstration serves as a compelling example of the remarkable quality achievable by integrating VDBs within a NeRF capture, all running smoothly in Unreal Engine.

I hope you enjoyed this week's news roundup, and I'm looking forward to sharing more exciting developments in the 3D space in the upcoming editions of this 3D Nilsletter.

That’s all for now!

Regards,
Nils Bakker

If you enjoyed this newsletter, forward it to a friend and invite them to subscribe!

^{Get in touch:
For inquiries, suggestions, submissions or just to say hi reach out to me at
[email protected]
instagram.com/nils_bakker
nilsbakker.nl}