# Differentiable Time-Gated Rendering
### [Project](https://shuangz.com/projects/dtgr-sa21/) | [Paper](https://shuangz.com/projects/dtgr-sa21/dtgr-sa21.pdf) | [Supplement](https://shuangz.com/projects/dtgr-sa21/dtgr-sa21_supp.zip) 

Differentiable Time-Gated Rendering  
Lifan Wu*, Guangyan Cai*, Ravi Ramamoorthi, and Shuang Zhao (* equal contribution)  
ACM Transactions on Graphics (SIGGRAPH Asia 2021), 40(6), December 2021

## Abstract
The continued advancements of time-of-flight imaging devices have enabled new imaging pipelines with numerous applications. Consequently, several forward rendering techniques capable of accurately and efficiently simulating these devices have been introduced. However, general-purpose differentiable rendering techniques that estimate derivatives of time-of-flight images are still lacking. In this paper, we introduce a new theory of differentiable time-gated rendering that enjoys the generality of differentiating with respect to arbitrary scene parameters. Our theory also allows the design of advanced Monte Carlo estimators capable of handling cameras with near-delta or discontinuous time gates.

We validate our theory by comparing derivatives generated with our technique and finite differences. Further, we demonstrate the usefulness of our technique using a few proof-of-concept inverse-rendering examples that simulate several time-of-flight imaging scenarios.

## System Requirements
Currently, our implementation only supports Linux. We have tested our implementation on Ubuntu 20.04 LTS. Note that since our implementaiton relies on [embree](https://www.embree.org/), a ray tracing framework from Intel, it only works with Intel's CPUs. 

## Dependencies
We assume Python 3 is already installed. Additional dependencies can be installed via the following shell command (assuming we are using Ubuntu or other Debian-based Linux distributions): 
```bash
source install.sh
```
After installing, make sure to set the path via the following line: 
```bash
source setpath.sh
```

## Compilation
Use the following lines to compile our code:
```bash
cd vredner
mkdir build 
cd build
cmake ..
sudo make install 
```

## Running Experiments
### Overview
We provide all the scenes and parameters for gernerating the paper results in the `paper_results` directory. Each scene is stored in a separate subdirectory and the scene name is the same as the subdirectory name. `opt_settings.py` contains the settings for every scene. The most important setting to pay attention to is the integrator. We provide two integrators: `standard` and `ellip_nee` (see our paper for their details). The settings of a scene typically look like this: 
```python
'[scene name]': {
    'render': {
        '[integrator name]': { ... }
        ...
    }
    ...
}
```
For instance, 
```python
# Tree (Fig. 17)
'plant_light': {
    'render': {
        'ellip_nee': { ... },
        'standard': { ... },
    },
    ...
},
```
In this case, the scene name is `plant_light` and the integrator can be either `standard` or `ellip_nee`. Notice that not every scene contains the setting for both integrators. 

### Time-gated Rendering
We can render time-gated images using dtgr_render.py:
```bash
python dtgr_render.py [scene name] [integrator name]
```
The results will be generated at `[scene name]/results_[integrator name]`.  

For instance:
```bash
python dtgr_render.py plant_light ellip_nee
```
will generate time-gated rendering at `plant_light/results_ellip_nee`.

### Inverse Rendering 
To do inverse rendering as shown in the paper, run:
```bash
python optimize.py [scene name] [integrator name]
```

### Switch Between Scenes
Different scenes might have different numbers of parameters that we want to optimize and under our current implementation, we need to change that number manually in `vredner/include/config.h` and recompile. 

Uncomment one of the lines: 
```c++
// vredner/include/config.h

// Uncomment one of the following lines and recompile to run the corresponding scene
constexpr int nder = 1; // validation, cube, branches, tree
// constexpr int nder = 2; // teaser, corridor, bunny 
// constexpr int nder = 121; // height field
// constexpr int nder = 59; // sofa
```
and recompile: 
```bash
cd vredner/build
sudo make install 
```

## Citation
```
@article{Wu:2021:DTGR,
  title={Differentiable Time-Gated Rendering},
  author={Wu, Lifan and Cai, Guangyan and Ramamoorthi, Ravi and Zhao, Shuang},
  journal={ACM Trans. Graph.},
  volume={40},
  number={6},
  year={2021},
  pages={287:1--287:16}
}
```
