introduction.tex

%! TEX root = main.tex

In recent years, research interest in using Physics-Informed Neural Networks (PINNs) has surged.
The idea of using neural networks to represent solutions of ordinary and partial differential equations goes back to the 1990s \cite{dissanayake_neural-network-based_1994,lagaris_artificial_1998}, but upon the term PINN being coined about five years ago, the field exploded. 
Partly, it reflects the immense popularity of all things machine learning and artificial intelligence (ML/AI). 
It also seems very attractive to be able to solve differential equations without meshing the domain, and without having to discretize the equations in space and time. 
PINN methods incorporate the differential equations as constraints in the loss function, and obtain the solution by minimizing the loss function using standard ML techniques.
They are easily implemented in a few lines of code, taking advantage of the ML frameworks that have become available in recent years, such as PyTorch. 
In contrast, traditional numerical solvers for PDEs such as the Navier-Stokes equations can require years of expertise and thousands of lines of code to develop, test and maintain. 
The general optimism in this field has perhaps held back critical examinations of the limitations of PINNs, and the challenges of using them in practical applications. 
This is compounded by the well-known fact that the academic literature is biased to positive results, and negative results are rarely published. 
We agree with a recent perspective article that calls for a view of ``cautious optimism'' in these emerging methods \cite{vinuesa_emerging_2022}, for which discussion in the published literature of both successes and failures is needed.

In this paper, we examine the solution of Navier-Stokes equations using PINNs in flows with instabilities, particularly vortex shedding. 
Fluid dynamic instabilities are ubiquitous in nature and engineering applications, and any method competing with traditional CFD should be able to handle them. 
In a previous conference paper, we already reported on our observations of the limitations of PINNs in this context \cite{chuang_experience_2022}. 
Although the solution of a laminar flow with vorticity, the classical Taylor-Green vortex, was well represented by a PINN solver, the same network architecture failed to give the expected solution in a flow with vortex shedding. 
The PINN solver accurately represented the steady solution at a lower Reynolds number of $Re=40$, but reverted to the steady state solution in two-dimensional flow past a circular cylinder at $Re=200$, which is known to exhibit vortex shedding. 
Here, we investigate this failure in more detail, comparing with a traditional CFD solver and with a data-driven PINN that receives as training data the solution of the CFD solver. 
We look at various fluid diagnostics, and also use dynamic mode decomposition (DMD) to analyze the flow and help explain the difficulty of the PINN solver to capture oscillatory solutions.

Other works have called attention to possible failure modes for PINN methods. Krishnapriyan el al. \cite{krishnapriyan_failure_2021} studied PINN models of simple problems of convection, reaction, and reaction-diffusion, and found that the PINN method only works for the simplest, slowyly varying problems.
They suggested that the neural network architecture is expressive enough to represent a good solution, but the landscape of the loss function is too complex for the optimization to find it. 
Fuks and Tchelepi \cite{fuks_limitations_2020} studied the limitations of PINNs in solving the Buckley-Leverett equation, a nonlinear hyperbolic equation that models two-phase flow in porous media. 
They found that the neural network model was unable to represent the solution of the 1D hyperbolic PDE when shocks were present, and also concluded that the problem was the optimization process, or the loss function.
The failure to capture the vortex shedding of cylinder flow is also highlighted in a recent work by Rohrhofer et al. \cite{rohrhofer_fixedpoints_2023}, who cite our previous conference paper. 

Our PINN solvers were built using the NVIDIA \emph{Modulus} toolkit,\footnote{\url{https://developer.nvidia.com/modulus}} a high-level package built on PyTorch for building, training, and fine-tuning physics-informed machine learning models.
For the traditional CFD solver, we used our own code, \emph{PetIBM}, which is open-source and available on GitHub, and has also been peer reviewed \cite{chuang_petibm_2018}. 
A Reproducibility Statement gives more details regarding all the open research objects to accompany the paper, and how the interested reader can reuse them.