After a chat with David Ha about this idea I was convinced that it was worth giving it a shot. It is a simplified implementation of Ken Stanley's Compositional Pattern Producing Network (CPPN) with a twist. Ken's CPPNs are similar to Artificial Neural Networks, but they differ in the non-homogeneous ways they use both inputs and activation functions, and in the fact that they typically use coordinates as inputs and output a single value (pixel intensity, or pixel color if you output 3 values) for the input coordinates. Because of this mapping from coordinates to intensities and colors, they are used for generating generative art, as in PicBreeder.
When I say non-homogeneous, I mean that neurons can have any number of inputs (including the outputs of other neurons) and can apply any sort of function to them (including sine and cosine, or RBF for instance). Because of this generality, one of the problems CPPNs face is that there could be cycles in the connectivity path from inputs to outputs. There are several possible strategies in the literature to resolve this.
David Ha implemented a simplified version of CPPNs in TensorFlow in the form of a classic layered Artificial Neural Network architecture that takes coordinates as inputs, projects them linearly to obtain a hidden representation, and then uses them as inputs to a regular neural network. The output is still pixel intensity or color, as in a CPPN.
I made several improvements to his code, adding a video recording feature, porting it to Python 3 and to a newer version of TensorFlow, along with some other cleanup. The main twist, however, concerns one of the assumptions of the CPPN. As mentioned above, loops are an issue for CPPNs, but because of the fixed nature of the architecture and the lack of repetition in the unfolding of the layers in David Ha's implementation, the model cannot produce recursive patterns. So I added an additional input parameter to the model: the number of times the model should feed its own output back as its input (it is actually a metaparameter). By controlling the depth of the recursion and constructing the network architecture and its computational graph dynamically, the model regains the ability to produce recursive and fractal-like patterns. I named this model Recursive Pattern Producing Network (RPPN), and it can generate beautiful high-resolution abstract images like the first two in the gallery, as well as fractal-like patterns like the third one.