Close
By Lucy Clerkin, UCL
Deep Art applies Deep Learning to artistic style [1], and is pretty good fun. A Convolution Neural Network (CNN) separates and recombines the content and style of an image, providing a neural algorithm for the creation of artistic images.
Each layer of the CNN can be thought of as a collection of image filters, each extracting a particular feature from the input image, and producing a ‘feature map’. When CNN are trained on object recognition, higher layers capture the high-level content in terms of objects and their arrangement, rather exact pixel values. This is the ‘Content representation’ (see fig. 1, from Gatys et al.).
Now for style: a representation of the style of the input image is built from the correlations between different filter responses (i.e. features). This gives a multi-scale representation of the input image that captures its texture info but not the global arrangement. Such ‘styles’ can then be applied to another image, as in fig 2 (from Gatys et al.)
But you can use any image you like for the ‘style’, why stick to paintings? Here are some versions of Ofer’s 1985 Cambridge photo in the style of: the 2DF galaxy redshift map; the Horsehead Nebula; the Orion Nebula, Dwingeloo 1 (and neighbours), and a pair of interacting galaxies (Arp 273). Some Ofer related, some just because they look good!
We can also flip things and use Ofer Lahav as our ‘artistic style’, and render e.g. the 2df map in the style of Ofer Lahav, or the Orion Nebula in the style of Ofer Lahav.
Or we can get meta… and use as our ‘style’ a collage of Ofer Lahavs in the style of various astronomical images. I could go on, but I have a PhD to finish... ;)
[1] A Neural Algorithm of Artistic Style, Leon A. Gatys, Alexander S. Ecker, Matthias Bethge, 2015 (https://arxiv.org/abs/1508.06576)
