Question about multi-scaling

[xilongzhou]

Hi Ivan,

You work is fantastic and I like the interesting idea very much!

I am implementing your idea to my project but got some issues. Sometime I got some pixelation in part of the images (256x256 images) when I tried your code on my dataset, here is just an example (not what I obtained) about what I mean for pixelation:

I kinda feel like this issue is caused by multi-scale, I use default code setting for multiscaling (start from 32x32 and gradually increase to 256x256). I notice that in the table 1 of paper, you make a comparison between with and w/o multi-scaling part, and in terms of the quality, with multi-scaling produces better results. Here are my questions:

1. have you even come across this pixelation issues when you run experiments?
2. how good are output images if you do not use multi-scaling? Do you think it worth trying?
3. If I want to turn off multi-scaling, is there a way to implement this quickly based on your code?

Thank you very much for your time! I appreciate your help!

[universome]

Hi, thanks! I suppose that this pixelation issue is mainly due to the spatial independence between pixels. The spatial independence between pixels makes it difficult for the model to make neighboring pixels coherent with each other, which means that the transitions might become sharper. The best way to solve this issue is to introduce some spatial dependence between pixels, like using bilinear/upfirdn2d interpolation, convolutional instead of linear layers in G, maybe adding correlated spatial noise, using instance norm, etc. If you inspect samples from CIPS closer, you might notice similar artifacts.

For INR-GAN, using bilinear interpolation + instance-norm (see the difference between inr-gan.yaml and inr-gan-bil.yaml configs) makes the samples more smooth on FFHQ (and other datasets as well):