In the Prog-GAN Paper have you ever figured out what c is?

[jakubLangr]

Hi, thanks so much for all your great work! AMAZING stuff.

In this paper summary (https://github.com/aleju/papers/blob/master/neural-nets/Progressive_Growing_of_GANs.md) there's this todo:
"(TODO exact formula for c?)"
and I was wondering whether you ever figured that out. This is the last bit of the paper I have trouble understanding.

[aleju]

Hi jakub, thanks for the praise.
Haven't really found out about c. I know that I got the same question by somebody else before who linked this ProGAN implementation. I wrote that the implementation looked to me like they went with something similar to c=sqrt(2/fan_in), though I have trouble seeing that right in the code, so not sure.

[twcmchang]

Hi @aleju, thanks for your great ProGAN summarization. I have some findings of c. After tracing the author's TensorFlow implementation, I found that the definition of c is shown in the function of get_weight (in network.py line 22).

The authors used the notation w_hat=w/c in the paper, but the implementation is w_hat=w*wscale, where wscale=gain/sqrt(fan_in) and the default value gain=sqrt(2). In sum, the default setting of c is c=sqrt(fan_in/2).

Last, it is quite interesting that in the dense layer of the generator the gain is manually set at np.sqrt(2)/4 (in network.py line 184). It seems to match the setting in the original Theano implementation but I have no idea now.