Notes on How A.I. Will Change the 3D Industry

procedural-generation

notes

My notes from Andrew Price’s talk at Blender Conference 2018 on how A.I. will change the 3D industry.

Author

Christian Mills

Published

December 9, 2021

Overview
Assets are unreasonably expensive
Machine Creep
Not in Presentation

Overview

I went back and watched the talk Andrew Price (Blender Guru) gave at Blender Conference 2018 on how A.I. will change the 3D industry. This time, I decided to take some notes.

Question to consider: What is not going to change in the next 10 years?

Assets are unreasonably expensive

Creating a building Asset:

Modeling: 12 hours
Texturing: 10 hours
First Pass total: 22 hours
Revisions: x2-3

Problem: static workflows

Solution: procedural workflows

Practical Procedural Generation for Everyone (GDC 2017)

Practical Procedural Generation for Everyone
Procedural Modeling Example: Procedural Lake Village by Anastasia Opera

Procedural Lake Village

Houdini Procedural Lake Houses Complete
Procedural Texturing Example: Poliigon Substance Designer

poliigon

poliigon generators
Procedural Texturing Example: Substance Painter
- Bake → Smart Materials → Smart Masks
substance3d
Procedural Level Design: Houdini

Procedural World Generation of Ubisoft’s Far Cry 5
- Create an ecosystem
  - Set rules to define where certain trees and plants would live
  - Other factors
    - Occlusion
    - Flow
    - Slope
    - Curvature
    - Illumination
    - Altitude
    - Latitude
    - Longitude
    - Wind
  - Tools for customization like roads and buildings

Machine Creep

Traditional Software: input (e.g. photo) → action (filter) → output

Machine Learning: input (e.g. photo) → assess → appropriate action → compare → is it good? → output

Needs huge datasets and fast hardware

Machine Learning Use Cases

De-noising
Super resolution
Motion Capture
- Densepose
Animation
- Mode-adaptive Neural Networks for motion control

Machine-Assisted Creativity

Problem: experimentation takes up 50%-70% of time
BicycleGAN
- Model input: outline of an object in an image and the ground truth image
- Model output: generate variations of image
- GitHub Repository
- Toward Multi-modal Image-to-image translation
DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network
- GitHub Repository
Sketch to Image
- pix2pix
  
  Image-to-Image Demo - Affine Layer
  - GitHub Repository
StyleGAN2
- GitHub Repository
- GitHub Repository
Progressive Growing of GANS (PGAN)
- GitHub Repository
PyTorch
Text to Image
- StackGAN V2 (2017)
- text2image (April 2021)
- TediGAN (March 2021)
- DF-GAN
Style Transfer
- A Style-Aware Content Loss for Real-time HD Style Transfer
  - Adaptive Style Transfer (TensorFlow 2018)
  - color-transform (PyTorch 2019)