The project logs, technical runbooks, and problem-solving notes behind Ben’s engineering builds.
This document outlines the implementation and application of Physics-Informed Neural Networks (PINNs).
Traditional deep learning models rely strictly on empirical data-driven pattern recognition. Consequently, they often fail to generalize outside their training distribution and can violate fundamental conservation laws. PINNs mitigate these vulnerabilities by embedding known governing physical laws directly into the neural network’s Loss Function.
PINNs leverage neural networks as universal function approximators while enforcing physical consistency. By utilizing the automatic differentiation capabilities native to modern deep learning frameworks (such as PyTorch’s autograd), the network evaluates differential equations and initial/boundary conditions without requiring traditional numerical discretization (e.g., finite differences or finite elements).
The neural network acts as a continuous model, mapping independent variables (such as time, t, and space, x) to the physical state variables of the system.
Partial derivatives are computed using the chain rule via the framework’s computational graph. This enables the calculation of physical residuals at arbitrary points in the domain, bypassing mesh-generation constraints.
To restrict the optimization search space to physically consistent states, the network’s objective function is formulated as a composite loss:
Total Loss = Loss(data) + Loss(physics) + Loss(boundary)
This regularization dramatically improves data efficiency and ensures robustness in data-scarce regimes.
To evaluate PINN performance, we model a classical mass-spring-damper system without external excitation. This system is governed by a second-order linear ordinary differential equation (ODE) derived from Newton’s Second Law:
m * (d²x/dt²) + μ * (dx/dt) + k * x = 0
Substituting these physical parameters into the governing equation yields:
d²x/dt² + 4 * (dx/dt) + 400 * x = 0
This configuration describes an underdamped oscillator, which exhibits decaying harmonic motion which provides a more meaningful curve for our neural network to fit.
To establish a baseline for verification, we calculate the exact analytical solution. Details on derivation of the analytical solution can be found in this link. The characteristic equation is:
r² + 4*r + 400 = 0
Solving for the roots (r):
r = -2 ± i * sqrt(396)
r ≈ -2 ± 19.900 * i
This yields the attenuation factor (alpha) and damped natural frequency (omega_d):
alpha = 2.0
omega_d ≈ 19.900 rad/s
The general solution for an underdamped system is:
x(t) = e^(-alpha * t) * [A * cos(omega_d * t) + B * sin(omega_d * t)]
Applying the initial conditions:
At t = 0, x(0) = 1.0:
x(0) = e^(0) * [A * cos(0) + B * sin(0)] = A
A = 1.0
At t = 0, dx/dt(0) = 0.0:
Differentiating x(t):
dx/dt = -alpha * e^(-alpha * t) * [A * cos(omega_d * t) + B * sin(omega_d * t)] + e^(-alpha * t) * [-A * omega_d * sin(omega_d * t) + B * omega_d * cos(omega_d * t)]
Evaluating at t = 0:
0 = -alpha * A + B * omega_d
B = (alpha * A) / omega_d
B = (2.0 * 1.0) / 19.900 ≈ 0.10050
Expressing this in amplitude-phase form:
x(t) = C * e^(-alpha * t) * cos(omega_d * t - phi)
Where:
This gives the final, exact analytical solution used for evaluating PINN accuracy:
x(t) = 1.0050 * e^(-2 * t) * cos(19.900 * t - 0.10017)
The python scripts that I used to explore PINNs can be found here.
1_Forward_Euler.py: Simulates the system using a state-space representation solved via the Forward Euler method. Compares numerical results directly against the analytical solution to check for accuracy. This script was written mainly to check how fast is a direct simulation method would be and for a simple 1D system like the mass-spring-damper system. This method solves much faster than a PINN and for a forward solution of a simple system, such a method should be used instead of a PINN.
requires_grad=True is set as we need to propagate the loss from these variables.3_pinn_data.py: Here we generate a noisy dataset which only spans the first 0.5s to check if the physics loss actually helps the model to generalise to the time period where there is no data. We consider 2 loss terms, the data loss and the physics loss. The data loss is calculated as the mean squared error between the model’s predictions and the noisy data. The total loss is calculated as 5000 * Data Loss + Physics Loss. 5000 is a hyperparameter that needs to be tuned to prevent the trivial solution of zero everywhere, which satisfies the physics loss to be obtained.