New article in Communications in Nonlinear Science and Numerical Simulations
Our colleague Sebastian Raubitzek, researcher at SBA Research and a member of the Security and Privacy Research Group at the University of Vienna, has published a journal article titled “Fractals in Neural Networks: Introducing a New Class of Activation Functions” in Communications in Nonlinear Science and Numerical Simulations in collaboration with the CD lab AsTra.
© Niklas Schnaubelt
We developed fractal activation functions to increase the expressivity of neural networks. These new paradigm of activation functions also introduces additional complexity into neural networks.
Abstract
We introduce a class of activation functions for neural networks based on the Blancmange curve and Weierstrass–Mandelbrot series. These functions are designed to integrate fractal, self-affine, and multi-scale structure into standard feed-forward architectures while remaining compatible with common training procedures. In contrast to smooth or piece-wise linear activations such as tanh and ReLU, the proposed activations are intentionally non-smooth, with the aim of increasing expressivity without modifying network depth or width. We evaluate these activation functions on standard tabular classification benchmarks using shallow neural networks. Several fractal variants achieve competitive or improved performance relative to established baselines. Geometric diagnostics based on trajectory deformation indicate increased expressive capacity, while additional analyses highlight trade-offs related to computational cost and gradient stability. Overall, the results suggest that fractal activation functions provide a viable and flexible extension of existing activation designs and motivate further study in larger and deeper architectures.
Authors: Sebastian Raubitzek, Tobias Kietreiber, Sebastian Eresheim, Alexander Buchelt, Kevin Mallinger