Poor output quality when using GoogleNet and CaffeNet

[dniku]

Command:

python2 style.py -c "$ROOT_DIR/johannesburg.jpg" -s "$ROOT_DIR/starry_night.jpg" -o "$ROOT_DIR/starry_johannesburg.jpg" --model googlenet

Command:

python2 style.py -c "$ROOT_DIR/johannesburg.jpg" -s "$ROOT_DIR/starry_night.jpg" -o "$ROOT_DIR/starry_johannesburg.jpg" --model caffenet

Is this normal? I'm running Gentoo with a GeForce 750 Ti 2Gb, driver version 361.28, CUDA 7.0.28, Caffe built from git today. I'm getting out of memory errors when I try to run with the default neural network.

[fzliu]

Caffenet unfortunately doesn't work very well for style transfer. The Inception model will look better if you remove the LRN layers and fine-tune the network on Imagenet. This is something that I've gotten around to in the past, but never got around to hosting it somewhere appropriate.

[dniku]

I am more concerned about the "grid" pattern which shows on both images. This looks a lot like a graphical artifact, as if there was something wrong with my video card. However, I've tried other implementations (https://github.com/anishathalye/neural-style on GPU and https://github.com/jcjohnson/neural-style on CPU) and they don't have this problem.

[austingg]

@Pastafarianist This "grid" pattern may caused by the aggressive big stride of the early conv layer in caffenet and googlenet, which lead to a lot of information loss. So vgg like network has no "grid" pattern.

[dniku]

Could someone with better hardware than mine run the same commands on the same images (from the repository) and make sure that the output is similar? If it is, then this issue should be closed.

[crocodoyle]

I'm getting the same grid artifacts using caffenet, to a lesser extent with googlenet.

[fzliu]

This version of the Inception model should produce fewer artifacts: https://www.dropbox.com/s/tdaowz2au059iqi/googlenet_style.caffemodel?dl=0

[austingg]

hi @fzliu ,
What's the difference between this version and the original inception model ?

[fzliu]

This model is fine-tuned from the original Inception model after removing local response normalization (LRN).