We apply a recently developed measure of multiscale complexity to the Gaussian model consisting of continuous spins with bilinear interactions for a variety of interaction matrix structures. We find two universal behaviors of the complexity profile. For systems with variables that are not frustrated, an exponential decay of multiscale complexity in the disordered regime shows the presence of small-scale fluctuations and a logarithmically diverging profile of fixed shape near the critical point describes the spectrum of collective modes. For frustrated variables, oscillations in complexity indicate the presence of global or local constraints. These observations show that the multiscale complexity may be a useful tool for interpreting the underlying structure of systems for which pair correlations can be measured.