In this work, the transport of unsteady, three-dimensional turbulent flows is the subject of main consideration. In all cases, the flow is assumed to be Newtonian and incompressible with a constant density. In the formulation, the space coordinate is identified by the vector \({\bf x}\equiv {x,\ y,\ z}\), and the time is denoted by t. The velocity field is denoted by \({\bf V} ({\bf x},t)\), with its three components, \({u ({\bf x},t) ,\ v({\bf x},t),\ w({\bf x},t)}\), along the three flow directions \({\ x,\ y,\ z}\), respectively. The pressure, the density, and the dynamic viscosity are denoted by \(p ({\bf x},t)\), \(\rho ({\bf x},t)\), and \(\nu\), respectively. The latter two are assumed constant. The (dummy) parameters \({\bf Q} ({\bf x},t)\) (as a vector), and/or Q (as a scalar) are used to denote a transport variable.
All of the flows considered are statistically homogeneous. High-resolution DNS and lower-resolution LES data are considered on
\(N_x \times N_y \times N_z\), and
\(M_x \times M_y \times M_z\) grid points, respectively. A box filter [
27] is employed to create the LES data from the original DNS. All of the statistical averages, including the Reynolds-averaged values are obtained by data ensembled over the entire domain. In this way, the ensemble averages, denoted by an over-bar are defined by
suitable for homogeneous flows. In the training process, the available DNS data are at a regular time interval
\(\delta\), as
\({\bf Q}^d =\lbrace {\bf Q}^d(t)\rbrace\) within the time
\(\lbrace t_0,t_0+\delta ,\ \dots , t_0+K\delta \rbrace\). The objective is to predict high-resolution DNS data after the historical data, at time
\(\lbrace t_0+(K+1)\delta , \ldots , t_0+M\delta \rbrace\). The variable
\({\bf Q}^l(x,y,z,t)\) represents the low-resolution LES data at timestep
t. Since the LES data can be created at a lower computational cost, they are used for both training and testing periods and at a higher frequency. The variable
\({\bf Q}^l = \lbrace {\bf Q}^l(t)\rbrace\) denotes LES data within the time range
\([t_0,t_0+M\delta ]\).
3.1 Runge–Kutta Transition Unit
The datasets
\({\bf Q}\) pertaining to turbulent flows consist of the transport variables that interact with each other and evolve temporally and spatially. The traditional temporal models, e.g., LSTM [
24], rely on large and consecutive training samples to capture the underlying patterns over time. However, the amount of high-fidelity DNS data is often limited. The RKTU structure is developed for reconstructing flow variables over a long period, given an initial DNS sample
\({\bf Q}^d\) at
t, and frequent low-resolution LES data samples
\({\bf Q}^l\). The prediction follows an auto-regressive process in which the predicted DNS
\(\hat{{\bf Q}}^d(x,y,z,{t})\) at time
t, and frequent LES data
\({\bf Q}^l\) from the current time to the next interval [
\(t,t+\delta\)] are used to predict the DNS at next timestep
\(\hat{{\bf Q}}^d(x,y,z,{t+\delta })\).
The RKTU is based on the
Runge–Kutta (RK) discretization method [
6]. The principal idea is to leverage the continuous physical relationship described by the underlying PDE to bridge the gap between the discrete data samples and the continuous flow dynamics. The scheme can be applied to any dynamical systems governed by deterministic PDEs. Consider the PDE of the target variables
\({\bf Q}\) as expressed by
where
\({\bf Q}_t\) denotes the temporal derivative of
\({\bf Q}\), and
\({{\bf f}}(t, {\bf Q};\theta)\) is a non-linear function (parameterized by coefficient
\(\theta\)) that summarizes the current value of
\({\bf Q}\) and its spatial variations. The turbulence data follow the Navier–Stokes equation for an incompressible flow. Thus, for
\({\bf Q} \equiv {\bf V}({\bf x},t)\),
The term
\(\nabla\) denotes the gradient operator and
\(\Delta =\nabla \cdot \nabla\) on each of the components of the velocity. The independent variable
t is omitted in the function
\({{\bf f}}(\cdot)\), because
\({{\bf f}}({\bf Q})\) in the Navier–Stokes equation is for a specific time
t (same with
t in
\({\bf Q}_t\)). Figure
2 shows the overall structure of the method and involves a series of intermediate states
\(\lbrace {\bf Q}(t,0),{\bf Q}(t,1),{\bf Q}(t,2),\dots ,{\bf Q}(t,N)\rbrace\). The temporal gradients are estimated at these states
\(\lbrace {\bf Q}_{t,0},{\bf Q}_{t,1},{\bf Q}_{t,2},\dots ,{\bf Q}_{t,N}\rbrace\). Starting from
\({\bf Q}(t,0)={\bf Q}(t)\), the RKTU estimates the temporal gradient as
\({\bf Q}_{t,0}\) and then moves
\({\bf Q}(t)\) toward the gradient direction to create the next intermediate state
\({\bf Q}(t,1)\). The process is repeated for
N intermediate states. For the fourth-order RK method, as employed here,
\(N=3\).
For the starting data point
\({\bf Q}(t)\), an augmentation mechanism is adopted by combining the DNS and LES data,
\({\bf Q}(t) = W^d {\bf Q}^d (t) + W^l{\bf Q}^l (t)\), where
\(W^d\) and
\(W^l\) are trainable model parameters, and
\({\bf Q}^l(t)\) is the up-sampled LES data with the same resolution as DNS. The RKTU estimates the first temporal gradient
\({\bf Q}_{t,0}={\bf f}({\bf Q}(t))\) using the Navier–Stokes equation and computes the next intermediate state variable
\({\bf Q}(t,1)\) by moving the flow data
\({\bf Q}(t)\) along the direction of temporal derivatives. Given frequent LES data, the intermediate states
\({\bf Q}(t,n)\) are also augmented by using LES data
\({\bf Q}^l(t,n)\), as
\({\bf Q}(t,n) = W^d {\bf Q}(t,n) + W^l {\bf Q}^l(t,n)\), and they follow the same process to move
\({\bf Q}(t)\) along the estimated gradient
\({\bf Q}_{t,n}\) to compute the next intermediate states
\({\bf Q}(t,n+1)\),
The temporal derivative
\({\bf Q}_{t,3}\) is then computed from the last intermediate state by
\({\bf f}({\bf Q}(t,{3}))\). According to Equation (
4), the intermediate LES data
\({\bf Q}^l(t,n)\) are selected as
\({\bf Q}^l(t,1)={\bf Q}^l(t+\delta /2)\),
\({\bf Q}^l(t,2)={\bf Q}^l(t+\delta /2)\), and
\({\bf Q}^l(t,3)={\bf Q}^l(t+\delta)\). Finally, RKTU combines all the intermediate temporal derivatives as a composite gradient to calculate the final prediction of next step flow data
\(\hat{{\bf Q}}_\text{RKTU}(t+\delta)\),
where
\(\lbrace w_n\rbrace _{n=1}^N\) are the trainable model parameters.
The RKTU requires the temporal derivatives in the Navier–Stokes equation. The RKTU estimates the temporal derivatives through the function
\({{\bf f}}(\cdot)\). According to Equation (
3), the evaluation of
\({{\bf f}}(\cdot)\) requires explicitly estimation of the first-order and second-order spatial derivatives. One of the most popular approaches for evaluating spatial derivatives is through
finite difference methods (FDMs) [
54]. However, the discretization in FDMs can cause larger errors for locations with complex dynamics. The RKTU structure, as depicted in Figure (
2), utilizes CNN layers to replace the FDMs. The CNNs have the inherent capability to learn additional non-linear relationships from data and capture the spatial derivatives required in the Navier–Stokes equation. After estimating the first-order and second-order spatial derivatives, they are used in Equation (
3) to obtain the temporal derivative
\({\bf Q}_{t,n}\).
The padding strategies for CNNs also need to be considered. Standard padding strategies (e.g., zero padding) do not satisfy the spatial boundary conditions of the flows considered here. These conditions describe how the flow data interact with the external environment. With the assumption of homogeneous turbulence, periodic boundary conditions are imposed on all three flow directions. Thus, periodic data augmentation is made for each of the six faces (of the 3D cubic data) with an additional two layers of data before feeding it to the model.
3.2 Temporally Enhancing Layer
The RKTU can capture the data in the spatial and temporal field between a pair of consecutive data points, but it may cause large reconstruction errors in the long-time prediction if the time interval
\(\delta\) is large. Temporal models, such as LSTM [
24], and
temporal convolutional network (TCN) [
31] are widely used to capture the long-term dependencies in time-series prediction. In this case, the LSTM model is incorporated in a TEL to further enhance the RKTU to capture long-term temporal dependencies. This TEL structure can be replaced by other existing temporal models such as TCN. Figure
3 shows two different approaches for integrating the TEL structure with the RKTU structure. In the first enhancing method shown in Figure
3(a), the RKTU output flow data
\(\hat{{\bf Q}}_\text{RKTU}\) are fed to the TEL structure, which is essentially an LSTM layer. After further processing through the TEL structure, the model produces the reconstructed flow data
\(\hat{{\bf Q}}^\text{d}(t)\). Given the true DNS data
\({\bf Q}^d (t)\) in the training set, the reconstructed loss
\(\mathcal {L}_\text{recon}\) can be expressed using the
mean squared error (MSE) loss,
The second method uses the TEL structure to complement the output of the RKTU structure, i.e., learning the residual of the RKTU output, as shown in Figure
3(b). In the training process, both true DNS data
\({\bf Q}^d\) at time
\(\lbrace t,\dots t+(K-1)\delta \rbrace\) and RKTU output
\(\hat{{\bf Q}}_\text{RKTU}\) are used to produce the corresponding temporal output feature
\(\hat{{\bf Q}}_{\text{TEL}}\) at time
\(\lbrace t+\delta , \dots , t+K\delta \rbrace\). Then in the testing process, this method uses only the initial true DNS data
\({\bf Q}^d\) in time
\(t+K\delta\) and the next series of predicted DNS data
\(\hat{{\bf Q}}^d\) as the DNS input to generate
\(\hat{{\bf Q}}_{\text{TEL}}\). Finally, this method adopts a linear combination to combine the RKTU output
\(\hat{{\bf Q}}_\text{RKTU}\) and corresponding TEL output
\(\hat{{\bf Q}}_{\text{TEL}}\) to obtain the final reconstructed output
\(\hat{{\bf Q}}^d\), which can be represented as
where
\(w_{r}^t\) and
\(w_{t}^t\) are trainable parameters. Finally, the reconstructed loss
\(\mathcal {L}_\text{recon}\) can also be represented by Equation (
6).