Sparse Transformer-Based Numerical Solvers for Nonlinear Partial Differential Equations

Abstract

Nonlinear partial differential equations (PDEs) are essential in modeling complex physical phenomena in fluid dynamics, heat transfer, and engineering systems. Traditional numerical approaches such as finite difference and finite element methods tend to be slow, computationally intensive and unable to scale to high-dimensional nonlinear equations. More recently, Physics-Informed Neural Networks (PINNs) have become popular as an alternative; they have slow convergence rates, and cannot learn long-range dependencies, but are considered promising. The paper presents a Sparse Transformer-Based Numerical Solver (STNS) to solve the nonlinear PDEs effectively. The proposed framework uses a sparse mechanism of self-attention to drastically cut down the computational costs with the ability to maintain interactions of global contexts in the solution space. To make predictions that are physics-consistent without large labelled datasets, a physics-informed loss is included to impose PDE constraints in training. The performance of the proposed model is tested on benchmark problems, such as the Burgers equation, and the Navier-Stokes equations. Observations during experiments show that STNS demonstrates better accuracy, shorter training duration and convergence behavior as opposed to PINNs and traditional numerical solvers. The results underscore the promise of sparse transformer architectures as scalable and efficient to solve complex nonlinear PDEs in scientific computing.