Umeå University's logo

In this paper we develop a geometric framework for equivariant manifold neural ordinary differential equations (NODEs), and use it to analyse their modelling capabilities for symmetric data. First, we consider the action of a Lie group G on a smooth manifold M and establish the equivalence between equivariance of vector fields, symmetries of the corresponding Cauchy problems, and equivariance of the associated NODEs. We also propose a novel formulation of the equivariant NODEs in terms of the differential invariants of the action of G on M, based on Lie theory for symmetries of differential equations, which provides an efficient parameterisation of the space of equivariant vector fields in a way that is agnostic to both the manifold M and the symmetry group G. Second, we construct augmented manifold NODEs through embeddings into equivariant flows, and show that they are universal approximators of equivariant diffeomorphisms on any connected M. Furthermore, we show that the augmented NODEs can be incorporated in the geometric framework and parametrised using higher order differential invariants. Finally, we consider the induced action of G on different fields on M and show how it generalises previous work, e.g., continuous normalizing flows, to equivariant models in any geometry.

We investigate the optimization of neural networks on symmetric data, and compare the strategy of constraining the architecture to be equivariant to that of using data augmentation. Our analysis reveals that the relative geometry of the admissible and the equivariant layers, respectively, plays a key role. Under natural assumptions on the data, network, loss, and group of symmetries, we show that compatibility of the spaces of admissible layers and equivariant layers, in the sense that the corresponding orthogonal projections commute, implies that the sets of equivariant stationary points are identical for the two strategies. If the linear layers of the network also are given a unitary parametrization, the set of equivariant layers is even invariant under the gradient flow for augmented models. Our analysis however also reveals that even in the latter situation, stationary points may be unstable for augmented training although they are stable for the manifestly equivariant models.

High-resolution wide-angle fisheye images are becoming more and more important for robotics applications such as autonomous driving. However, using ordinary convolutional neural networks or vision transformers on this data is problematic due to projection and distortion losses introduced when projecting to a rectangular grid on the plane. We introduce the HEAL-SWIN transformer, which combines the highly uniform Hierarchi-cal Equal Area iso-Latitude Pixelation (HEALPix) grid used in astrophysics and cosmology with the Hierarchical Shifted-Window (SWIN) transformer to yield an efficient and flexible model capable of training on high-resolution, distortion-free spherical data. In HEAL-SWIN, the nested structure of the HEALPix grid is used to perform the patching and windowing operations of the SWIN transformer, enabling the network to process spherical representations with minimal computational overhead. We demonstrate the superior performance of our model on both synthetic and real automotive datasets, as well as a selection of other image datasets, for semantic segmentation, depth regression and classification tasks.

We consider the relationship between symmetries of two-dimensional autonomous dynamical systems in two common formulations; as a set of differential equations for the derivative of each state with respect to time, and a single differential equation in the phase plane representing the dynamics restricted to the state space of the system. Both representations can be analysed with respect to their symmetries, and we establish the correspondence between the set of infinitesimal generators of the respective formulations. We show that every generator of a symmetry of the autonomous system induces a well-defined vector field generating a symmetry in the phase plane and, conversely, that every symmetry generator in the phase plane can be lifted to a generator of a symmetry of the original system, which is unique up to constant translations in time. We exemplify the lift of symmetries in two cases; a mass conserved linear model and a non-linear oscillator.

We study symmetries in the phase plane for separable, autonomous two-state systems of ordinary differential equations (ODEs). We prove two main theoretical results concerning the existence and non-triviality of two orthogonal symmetries for such systems. In particular, we show that these symmetries correspond to translations in the internal energy of the system, and describe their action on solution trajectories in the phase plane. In addition, we apply recent results establishing how phase plane symmetries can be extended to incorporate temporal dynamics to these energy translation symmetries. Subsequently, we apply our theoretical results to the analysis of three models from the field of mathematical biology: a canonical biological oscillator model, the Lotka–Volterra (LV) model describing predator–prey dynamics, and the SIR model describing the spread of a disease in a population. We describe the energy translation symmetries in detail, including their action on biological observables of the models, derive analytic expressions for the extensions to the time domain, and discuss their action on solution trajectories.

We survey the mathematical foundations of geometric deep learning, focusing on group equivariant and gauge equivariant neural networks. We develop gauge equivariant convolutional neural networks on arbitrary manifolds M using principal bundles with structure group K and equivariant maps between sections of associated vector bundles. We also discuss group equivariant neural networks for homogeneous spaces M= G/ K , which are instead equivariant with respect to the global symmetry G on M . Group equivariant layers can be interpreted as intertwiners between induced representations of G, and we show their relation to gauge equivariant convolutional layers. We analyze several applications of this formalism, including semantic segmentation and object detection networks. We also discuss the case of spherical networks in great detail, corresponding to the case M= S²= SO (3) / SO (2) . Here we emphasize the use of Fourier analysis involving Wigner matrices, spherical harmonics and Clebsch–Gordan coefficients for G= SO (3) , illustrating the power of representation theory for deep learning.

We analyze the role of rotational equivariance in convolutional neural networks (CNNs) applied to spherical images. We compare the performance of the group equivariant networks known as S2CNNs and standard non-equivariant CNNs trained with an increasing amount of data augmentation. The chosen architectures can be considered baseline references for the respective design paradigms. Our models are trained and evaluated on single or multiple items from the MNIST- or FashionMNIST dataset projected onto the sphere. For the task of image classification, which is inherently rotationally invariant, we find that by considerably increasing the amount of data augmentation and the size of the networks, it is possible for the standard CNNs to reach at least the same performance as the equivariant network. In contrast, for the inherently equivariant task of semantic segmentation, the non-equivariant networks are consistently outperformed by the equivariant networks with significantly fewer parameters. We also analyze and compare the inference latency and training times of the different networks, enabling detailed tradeoff considerations between equivariant architectures and data augmentation for practical problems.

In this paper we present a novel concept to efficiently harvest vibrational energy at low frequencies and very small displacement. We describe and evaluate an electromagnetic energy harvester which generates power from a magnetic circuit with motion induced variations of an air gap. External vibrations induce oscillations of the gap length around an equilibrium point, due to a linear spring counteracting the magnetic force. The relative position of the spring can be adjusted to optimize the harvester output for excitation amplitude and frequency. A simulation model is built in COMSOL and verified by comparison with lab measurements. The simulation model is used to determine the potential performance of the proposed concept under both harmonic and non-harmonic excitation. Under harmonic excitation, we achieve a simulated RMS load power of 26.5 μW at 22 Hz and 0.028 g acceleration amplitude. From a set of comparable EH we achieve the highest theoretical power metric of 1712.2 µW/cm3/g2 while maintaining the largest relative bandwidth of 81.8%. Using measured non-harmonic vibration data, with a mean acceleration of 0.039 g, resulted in a mean power of 52 μW. Moreover, the simplicity and robustness of our design makes it a competitive alternative for use in practical situations.

In this paper, we extend the optimization analysis found in the current literature for single-degree-of-freedom vibrational energy harvesters. We numerically derive and analyze the optimization conditions based on unified expressions for piezoelectric and electromagnetic energy harvesters. Our contribution lies in the detailed analysis and comparison of both resonant and anti-resonant states while fully including the effect of intrinsic resistance. We include both the case of excitation by inertial load and prescribed displacement, as the latter has not been elaborated on in the previous literature and provides new insights. We perform a general analysis but also consider typical values of applied piezoelectric and electromagnetic energy harvesters. Our results improve upon previous similar comparative studies by providing new and useful insights regarding optimal load, load power and power input to output efficiency. Our analysis shows an exponential increase in the critical mechanical quality factor due to the resistive loss coefficient. We find that the ratio of mechanical quality factor to resistive loss coefficient, at resonance, increases drastically close to the theoretical maximum for load power. Under the same optimization conditions, an equivalent conclusion can be drawn regarding efficiency. We find that the efficiency at anti-resonance behaves differently and is equal to or larger than the efficiency at resonance. We also show that the optimal load coefficient at resonance has a significant dependence on the mechanical quality factor only when the resistive loss coefficient is large. Our comparison of excitation types supports the previous literature, in a simple and intuitive way, regarding optimal load by impedance matching and power output efficiency. Our modeling and exploration of new parameter spaces provide an improved tool to aid the development of new harvester prototypes.

Objective: To determine to what extent accelerometer-based arm, leg and trunk activity is associated with sensorimotor impairments, walking capacity and other factors in subacute stroke.

Design: Cross-sectional study.

Patients: Twenty-six individuals with stroke (mean age 55.4 years, severe to mild motor impairment).

Methods: Data on daytime activity were collected over a period of 4 days from accelerometers placed on the wrists, ankles and trunk. A forward stepwise linear regression was used to determine associations between free-living activity, clinical and demographic variables.

Results: Arm motor impairment (Fugl-Meyer Assessment) and walking speed explained more than 60% of the variance in daytime activity of the more-affected arm, while walking speed alone explained 60% of the more-affected leg activity. Activity of the less-affected arm and leg was associated with arm motor impairment (R2=0.40) and independence in walking (R2=0.59). Arm activity ratio was associated with arm impairment (R2=0.63) and leg activity ratio with leg impairment (R2=0.38) and walking speed (R2=0.27). Walking-related variables explained approximately 30% of the variance in trunk activity.

Conclusion: Accelerometer-based free-living activity is dependent on motor impairment and walking capacity. The most relevant activity data were obtained from more-affected limbs. Motor impairment and walking speed can provide some information about real-life daytime activity levels.