Notes on How A.I. Will Change the 3D Industry

Overview
Assets are unreasonably expensive
Machine Creep
Not in Presentation

Overview

I went back and watched the talk Andrew Price (Blender Guru) gave at Blender Conference 2018 on how A.I. will change the 3D industry. This time, I decided to take some notes.

Question to consider: What is not going to change in the next 10 years?

Assets are unreasonably expensive

Creating a building Asset:

Modeling: 12 hours
Texturing: 10 hours
First Pass total: 22 hours
Revisions: x2-3

Problem: static workflows

Solution: procedural workflows

Practical Procedural Generation for Everyone (GDC 2017)

Practical Procedural Generation for Everyone
Procedural Modeling Example: Procedural Lake Village by Anastasia Opera

Procedural Lake Village

Houdini Procedural Lake Houses Complete
Procedural Texturing Example: Poliigon Substance Designer

poliigon

poliigon generators
Procedural Texturing Example: Substance Painter
- Bake → Smart Materials → Smart Masks
substance3d
Procedural Level Design: Houdini

Procedural World Generation of Ubisoft’s Far Cry 5
- Create an ecosystem
  - Set rules to define where certain trees and plants would live
  - Other factors
    - Occlusion
    - Flow
    - Slope
    - Curvature
    - Illumination
    - Altitude
    - Latitude
    - Longitude
    - Wind
  - Tools for customization like roads and buildings

Machine Creep

Traditional Software: input (e.g. photo) → action (filter) → output

Machine Learning: input (e.g. photo) → assess → appropriate action → compare → is it good? → output

Needs huge datasets and fast hardware

Machine Learning Use Cases

De-noising
Super resolution
Motion Capture
- Densepose
Animation
- Mode-adaptive Neural Networks for motion control

Machine-Assisted Creativity

Problem: experimentation takes up 50%-70% of time
BicycleGAN
- Model input: outline of an object in an image and the ground truth image
- Model output: generate variations of image
- GitHub Repository
- Toward Multi-modal Image-to-image translation
DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network
- GitHub Repository
Sketch to Image
- pix2pix
  
  Image-to-Image Demo - Affine Layer
  - GitHub Repository
StyleGAN2
- GitHub Repository
- GitHub Repository
Progressive Growing of GANS (PGAN)
- GitHub Repository
PyTorch
Text to Image
- StackGAN V2 (2017)
- text2image (April 2021)
- TediGAN (March 2021)
- DF-GAN
Style Transfer
- A Style-Aware Content Loss for Real-time HD Style Transfer
  - Adaptive Style Transfer (TensorFlow 2018)
  - color-transform (PyTorch 2019)