Part II — Intelligent Engineering Search: Surrogate Models, Uncertainty Quantification, and Active Learning

Introduction: From Simulation Cost to Intelligent Search

The previous section established the central limitation of modern computational engineering.

High-fidelity simulation provides the most accurate representation of physical behaviour, but the computational cost of repeatedly evaluating complex models prevents exhaustive exploration of large design spaces.

The challenge is therefore no longer simply solving the governing equations.

It is deciding which solutions are worth calculating.

The engineering design problem can be represented as:

$$\mathbf{y} = \mathbf{f}(\mathbf{x})$$

where:

• $\mathbf{x}$ represents the design variables,
• $\mathbf{y}$ represents the resulting system response,
• $\mathbf{f}$ represents the underlying physics model.

For a pulse jet system, the design vector may contain:

$$\mathbf{x} = \begin{bmatrix} L_c & D_c & L_t & D_t & \theta & \dot m_f & A_i \end{bmatrix}^{T}$$

where:

• $L_c$ represents combustion chamber length,
• $D_c$ represents combustion chamber diameter,
• $L_t$ represents exhaust tube length,
• $D_t$ represents exhaust tube diameter,
• $\theta$ represents geometric parameters,
• $\dot m_f$ represents fuel flow rate,
• $A_i$ represents inlet area.

The response vector may include:

$$\mathbf{y} = \begin{bmatrix} F & \eta & T_{wall} & \Delta P & f_r & \sigma_{max} \end{bmatrix}^{T}$$

where:

• $F$ represents thrust,
• $\eta$ represents efficiency,
• $T_{wall}$ represents wall temperature,
• $\Delta P$ represents pressure variation,
• $f_r$ represents resonant frequency,
• $\sigma_{max}$ represents maximum stress.

The optimisation objective is therefore:

$$\mathbf{x}^{*} = \arg\max_{\mathbf{x}} f(\mathbf{x})$$

subject to:

$$g_i(\mathbf{x})<0$$

where $g_i(\mathbf{x})$ represents constraints such as:

• maximum allowable temperature,
• structural stress limits,
• manufacturing restrictions,
• operating boundaries.

However, unlike conventional mathematical optimisation problems, evaluating:

$$\mathbf{f}(\mathbf{x})$$

requires executing an expensive physical simulation.

A high-fidelity pulse jet simulation may involve:

• compressible flow equations,
• turbulence modelling,
• combustion chemistry,
• thermal transport,
• structural coupling.

The optimisation algorithm itself is therefore not the computational bottleneck.

The dominant cost is repeatedly obtaining new evaluations of the engineering response.

A conventional simulation-driven optimisation workflow follows:

$$\text{Design} \rightarrow \text{High-Fidelity Simulation} \rightarrow \text{Performance Evaluation} \rightarrow \text{Next Design}$$

Every optimisation decision requires another expensive physics evaluation.

Figure 1. Conventional simulation-driven optimisation. Every optimisation decision requires another expensive physics evaluation.

For simple engineering problems, this approach is effective.

However, modern engineering systems often contain thousands or millions of possible design combinations.

The challenge is therefore not simply finding an optimum.

The challenge is obtaining enough information about the design space for the optimisation process to make intelligent decisions.

---

Surrogate Models: Learning the Design Space

The previous optimisation approaches share the same fundamental limitation.

They require direct access to the true engineering response.

For expensive simulations, this means every optimisation decision requires another high-fidelity calculation.

Surrogate modelling changes the problem.

Instead of repeatedly evaluating:

$$\mathbf{y} = \mathbf{f}(\mathbf{x})$$

the engineering workflow first learns an approximation:

$$\hat{\mathbf{y}} = \hat{\mathbf{f}}(\mathbf{x})$$

where:

• $\mathbf{f}(\mathbf{x})$ represents the original physics-based simulation,
• $\hat{\mathbf{f}}(\mathbf{x})$ represents an approximation learned from previous simulations.

The surrogate model becomes a computationally inexpensive representation of the engineering design space.

The purpose of a surrogate is not to replace the governing physics.

The high-fidelity solver remains the source of truth.

Instead, the surrogate provides a fast approximation of the relationship between design variables and engineering performance.

Rather than storing every simulated design individually, the surrogate learns the underlying structure of the design space from previously evaluated examples.

This changes the structure of the optimisation process.

Instead of:

$$\text{Design} \rightarrow \text{High-Fidelity Simulation} \rightarrow \text{Performance}$$

being repeated thousands of times, the process becomes:

$$\text{Design} \rightarrow \text{Surrogate} \rightarrow \text{Predicted Performance}$$

with only selected designs being returned to the high-fidelity solver.

Figure 2. Surrogate-assisted optimisation. High-fidelity simulation is reserved for designs that provide valuable information to improve the model.

The computational advantage is significant.

A transient reacting CFD simulation may require:

$$t_{CFD} \approx \text{hours}$$

whereas evaluating a trained surrogate model may require:

$$t_{surrogate} \approx \text{milliseconds}$$

This difference allows optimisation algorithms to explore regions of the design space that would otherwise be inaccessible.

Instead of spending computational resources evaluating every possible design, the system can use the surrogate to identify promising regions and selectively perform expensive simulations where they provide the greatest value.

---

Constructing Physics-Guided Surrogate Models

A surrogate model is constructed from a database of previously evaluated engineering designs.

Each simulation produces a training pair:

$$(\mathbf{x}_i,\mathbf{y}_i)$$

where:

$$\mathbf{x}_i$$

contains the design parameters and:

$$\mathbf{y}_i$$

contains the resulting engineering response.

The complete training dataset becomes:

$$D = \{ (\mathbf{x}_1,\mathbf{y}_1), (\mathbf{x}_2,\mathbf{y}_2), ... (\mathbf{x}_n,\mathbf{y}_n) \}$$

The surrogate model attempts to approximate the unknown mapping:

$$\mathbf{f}: \mathbb{R}^{n} \rightarrow \mathbb{R}^{m}$$

using only available simulation data.

For the pulse jet example, the training data may contain:

Input variables:

• combustion chamber length,
• combustion chamber diameter,
• exhaust tube length,
• exhaust tube diameter,
• valve geometry,
• fuel flow rate,
• inlet area.

Output variables:

• average thrust,
• specific impulse,
• pressure oscillation frequency,
• maximum wall temperature,
• thermal stress,
• acoustic response.

After training, the surrogate becomes a computational model of the design space.

The optimiser can then explore thousands or millions of candidate designs using the surrogate rather than repeatedly executing the full physics simulation.

However, the quality of this approximation depends strongly on the information provided during training.

A surrogate trained only on sparse or poorly distributed data may produce inaccurate predictions, particularly in regions of the design space that have not been explored.

This creates an important engineering requirement:

The surrogate must learn from information that captures both the global behaviour of the system and the detailed physics that determine performance.

This motivates the use of physics-guided and multi-fidelity approaches.

Multi-Fidelity Information

Surrogate models do not necessarily need to learn directly from the highest-fidelity simulation.

In many engineering applications, the most effective approach is to combine information from multiple levels of modelling accuracy.

The simulation hierarchy established in Part I provides a natural structure:

$$\text{Analytical Model} \downarrow \text{0D Thermodynamic Model} \downarrow \text{Quasi-1D Model} \downarrow \text{3D CFD} \downarrow \text{Experimental Validation}$$

Each modelling level provides different information at a different computational cost.

Low-fidelity models provide large quantities of inexpensive data.

For example, a simplified pulse jet model may rapidly evaluate:

• approximate thrust trends,
• combustion frequency,
• pressure oscillation behaviour.

However, these models cannot accurately represent:

• boundary layer losses,
• turbulent mixing,
• three-dimensional flow structures,
• local thermal effects.

High-fidelity CFD captures these effects but requires significantly more computational resources.

The purpose of a surrogate model is therefore not simply to imitate the most expensive model.

Instead, it is to combine information across the entire modelling hierarchy.

A useful interpretation is:

Low-fidelity models define the broad structure of the design space.

High-fidelity models provide corrections where accuracy matters.

Figure 3. Multi-fidelity surrogate modelling combines inexpensive approximate models with high-fidelity simulations to efficiently represent complex engineering behaviour.

By combining these information sources, the surrogate can achieve high predictive accuracy without requiring an impractical number of expensive simulations.

---

Physics-Guided Surrogate Modelling

A common misconception is that surrogate models are purely data-driven replacements for physics.

In engineering applications, this is rarely desirable.

The objective is not to remove physical understanding from the modelling process.

Instead, effective engineering surrogate models incorporate existing knowledge wherever possible.

For example, a pulse jet surrogate may already reflect known physical relationships:

• increasing combustion energy increases exhaust velocity,
• excessive temperatures increase thermal constraints,
• certain geometries produce increased acoustic losses,
• pressure oscillations are influenced by chamber dimensions.

The machine learning model is not discovering these relationships from nothing.

Instead, it is learning the complex nonlinear interactions between variables that are difficult to derive analytically.

This distinction separates engineering surrogate modelling from general-purpose prediction.

The objective is not:

Predict the output of a system.

The objective is:

Construct the cheapest possible representation of engineering behaviour while preserving the relationships required for design decisions.

It is important to distinguish this approach from Physics-Informed Neural Networks (PINNs).

PINNs directly incorporate governing equations into the optimisation loss function by penalising violations of physical laws.

The methods discussed here instead incorporate physical knowledge through:

• multi-fidelity data,
• physical constraints,
• reduced-order models,
• residual learning.

This approach is therefore better described as physics-guided surrogate modelling.

Physics-guided surrogate modelling can incorporate information through several mechanisms:

• using reduced-order physics models as additional training data,
• enforcing known constraints during optimisation,
• learning corrections between low- and high-fidelity simulations,
• embedding physical relationships within the model structure.

The result is not a replacement for simulation.

It is a computational representation of simulation behaviour that can accelerate engineering search.

---

Common Surrogate Architectures

Different surrogate models provide different balances between computational efficiency, flexibility, interpretability, and predictive uncertainty estimation.

No single surrogate architecture is optimal for every engineering problem.

The appropriate choice depends on:

• the size of the available dataset,
• the dimensionality of the design space,
• the complexity of the physics,
• whether predictive uncertainty is required.

Figure 4. Comparison of engineering surrogate model architectures. Different approaches provide different balances between flexibility, interpretability, and uncertainty estimation.

---

Polynomial Response Surfaces

Polynomial response surfaces are among the simplest surrogate approaches.

The engineering response is approximated using a mathematical expression:

$$\hat{y} = a_0 + a_1x_1 + a_2x_2 + a_3x_1x_2 + ...$$

where the coefficients are determined from existing simulation data.

These models are effective for:

• low-dimensional design spaces,
• smooth engineering behaviour,
• rapid sensitivity analysis.

Their advantages include:

• simple implementation,
• low computational cost,
• high interpretability.

However, polynomial models become limited when the engineering response contains:

• strong nonlinearities,
• discontinuities,
• complex variable interactions.

For highly nonlinear systems, more flexible approaches are required.

---

Neural Network Surrogates

Neural networks provide a flexible method for approximating complex nonlinear relationships.

Their popularity in engineering optimisation comes from a simple observation:

Modern engineering systems are often governed by highly nonlinear interactions between many variables.

Explicitly deriving these relationships analytically is often impossible, while storing every simulated design is computationally infeasible.

Instead, a neural network learns the underlying mapping directly from simulation data.

Given a collection of simulation results:

$$D = \{ (\mathbf{x}_i,\mathbf{y}_i) \}_{i=1}^{N}$$

the objective is to determine a parameterised function:

$$\hat{\mathbf{f}}(\mathbf{x};\theta)$$

such that:

$$\hat{\mathbf{f}}(\mathbf{x};\theta) \approx \mathbf{f}(\mathbf{x})$$

where $\theta$ represents the trainable parameters of the network.

Rather than storing the outcome of every simulation individually, the network compresses the behaviour of the design space into a finite set of numerical parameters.

During optimisation, evaluating the surrogate therefore becomes a sequence of matrix operations rather than solving coupled governing equations.

---

Learning the Design Space

For the pulse jet example:

$$\mathbf{x} = \begin{bmatrix} L_c & D_c & L_t & D_t & \theta & \dot m_f & A_i \end{bmatrix}^{T}$$

is supplied as the network input.

The output may consist of:

$$\hat{\mathbf{y}} = \begin{bmatrix} F & \eta & T_{wall} & f_r & \sigma_{max} \end{bmatrix}^{T}$$

The internal structure of the network progressively transforms these design variables into increasingly abstract representations.

For a fully connected feed-forward network, each layer performs:

$$\mathbf{h}^{(l+1)} = \phi \left( W^{(l)} \mathbf{h}^{(l)} + \mathbf{b}^{(l)} \right)$$

where:

• $W^{(l)}$ is the weight matrix,
• $\mathbf{b}^{(l)}$ is the bias vector,
• $\phi(\cdot)$ is a nonlinear activation function.

Although each transformation is mathematically simple, stacking many layers allows the network to approximate highly nonlinear functions.

Universal approximation theory demonstrates that sufficiently expressive neural networks can approximate a broad class of continuous functions to arbitrary accuracy, provided sufficient training data and network capacity are available.

From an engineering perspective, this means the network can learn relationships that would be difficult to derive analytically.

For example, the influence of exhaust diameter on thrust is not independent of combustion temperature, pressure oscillation frequency, or chamber geometry.

The response emerges from the coupled behaviour of the entire system.

Rather than modelling these interactions explicitly, the network infers them from simulation data.

Figure 5. Neural networks approximate complex nonlinear engineering relationships by learning transformations between design variables and system responses.

Advantages for Engineering Optimisation

Neural networks possess several characteristics that make them particularly well suited to engineering design exploration.

First, they scale effectively with increasing dimensionality.

Engineering optimisation problems frequently involve dozens or hundreds of design variables, including geometry, operating conditions, material properties, and control parameters.

Traditional interpolation methods deteriorate rapidly as dimensionality increases, whereas neural networks remain computationally efficient once trained.

Second, they naturally represent highly nonlinear relationships.

Engineering systems are rarely linear.

Combustion, turbulence, heat transfer, structural deformation, and acoustic resonance all introduce nonlinear coupling between variables.

Neural networks can represent these interactions without requiring explicit mathematical derivations.

Third, evaluation is extremely inexpensive.

Once trained, prediction requires only a sequence of matrix multiplications.

Consequently:

$$t_{prediction} \ll t_{simulation}$$

allowing millions of candidate designs to be evaluated within seconds.

This capability fundamentally changes the optimisation workflow.

Rather than repeatedly solving expensive multiphysics models, the optimiser can first perform extensive exploration using the surrogate before requesting only a small number of additional high-fidelity simulations.

---

The Problem of Extrapolation

Despite these advantages, neural networks possess a significant limitation that prevents them from being used as autonomous engineering models.

They always produce an answer.

Suppose the network has been trained using pulse jet geometries with exhaust diameters between:

$$20\ \text{mm} \leq D_t \leq 80\ \text{mm}$$

During optimisation, the algorithm proposes:

$$D_t = 2\ \text{mm}$$

This geometry lies well outside the region represented by the training data.

Nevertheless, the neural network will still evaluate:

$$\hat{\mathbf{f}}(\mathbf{x})$$

and may confidently predict:

$$F = 520\ \text{N}$$

The prediction appears numerically identical to those produced within the training domain.

However, the network provides no indication that this result is based entirely upon extrapolation.

From the perspective of the optimiser, this design may appear highly attractive.

From the perspective of the underlying physics, the prediction may be meaningless.

This behaviour is not unique to pulse jets.

Every engineering optimisation problem eventually encounters regions where little or no simulation data exists.

Since optimisation naturally seeks extreme solutions, these unexplored regions often coincide with the predicted optimum.

The consequence is that the optimiser becomes increasingly dependent upon the least reliable predictions made by the surrogate.

Figure 6. Neural networks can accurately approximate known regions of the design space but cannot inherently determine when predictions are based on extrapolation.

---

Gaussian Process Models

Gaussian Processes provide a fundamentally different approach.

Instead of learning only a deterministic function approximation, Gaussian Processes treat the prediction itself as probabilistic.

The engineering response is represented as:

$$f(\mathbf{x}) \sim \mathcal{GP} ( m(\mathbf{x}), k(\mathbf{x},\mathbf{x}') )$$

where:

• $m(\mathbf{x})$ represents the mean function,
• $k(\mathbf{x},\mathbf{x}')$ represents the covariance function describing relationships between points.

Unlike neural networks, which learn a fixed mapping between inputs and outputs, a Gaussian Process defines a probability distribution over possible functions that are consistent with the available data.

The covariance kernel describes how strongly two designs are expected to be related.

For example, if two pulse jet geometries differ only slightly in chamber diameter, it is reasonable to expect their performance to be similar.

Conversely, two radically different geometries should exhibit weaker correlation.

The kernel therefore provides a mathematical measure of similarity throughout the design space.

After observing training data, the Gaussian Process updates its belief regarding the unknown function and produces two outputs for every design point:

The predicted response:

$$\mu(\mathbf{x})$$

and the predictive uncertainty:

$$\sigma^2(\mathbf{x})$$

The posterior prediction becomes:

$$f(\mathbf{x})|\mathcal{D} \sim \mathcal{N} ( \mu(\mathbf{x}), \sigma^2(\mathbf{x}) )$$

where:

• $\mu(\mathbf{x})$ represents the expected engineering response,
• $\sigma^2(\mathbf{x})$ represents uncertainty in that prediction.

This allows the model to distinguish between:

• regions densely supported by simulation data,
• regions where predictions rely on interpolation,
• unexplored regions where additional simulations may provide valuable information.

For example, instead of predicting:

$$F = 420\text{ N}$$

the model may predict:

$$F = 420 \pm 8\text{ N}$$

or:

$$F = 610 \pm 220\text{ N}$$

The second prediction is not necessarily incorrect.

Rather, it indicates that the model possesses insufficient information to make a confident estimate.

The uncertainty becomes an engineering quantity in its own right.

Figure 7. Gaussian Processes provide both predicted performance and confidence estimation through the predictive mean and variance.

---

The Scalability Problem

Although Gaussian Processes provide uncertainty estimates naturally, they introduce a different computational limitation.

Training requires operations on an $N \times N$ covariance matrix, where $N$ is the number of training samples.

The computational complexity therefore scales approximately as:

$$\mathcal{O}(N^3)$$

while memory requirements scale as:

$$\mathcal{O}(N^2)$$

For small datasets, this is entirely manageable.

However, engineering optimisation often generates tens or hundreds of thousands of simulations.

Training a conventional Gaussian Process on datasets of this size rapidly becomes impractical.

Ironically, the surrogate model intended to reduce computational cost may itself become computationally expensive.

Consequently, Gaussian Processes are rarely used as standalone global surrogate models for large engineering problems.

Instead, their uncertainty estimation capability is most valuable when combined with models that scale effectively to large datasets.

This motivates hybrid surrogate architectures.

---

Hybrid Neural Network–Gaussian Process Surrogate Models

Engineering optimisation requires two capabilities from a surrogate model.

First, it must accurately approximate the relationship between design variables and system performance.

Second, it must quantify the uncertainty associated with those predictions.

These requirements create a fundamental trade-off between neural networks and Gaussian Processes.

Neural networks provide the flexibility required to capture complex engineering relationships:

$$\hat{\mathbf{y}} = NN(\mathbf{x};\theta)$$

However, they do not naturally determine whether a prediction is supported by sufficient information.

Gaussian Processes provide uncertainty estimation:

$$f(\mathbf{x}) \sim \mathcal{GP} ( m(\mathbf{x}), k(\mathbf{x},\mathbf{x}') )$$

but become computationally expensive as the number of training samples increases.

A practical engineering solution is therefore to combine both approaches.

The objective is not to choose between prediction accuracy and uncertainty estimation.

It is to construct a surrogate architecture capable of providing both.

Residual Learning Architecture

A hybrid Neural Network–Gaussian Process surrogate combines the strengths of both approaches.

The neural network learns the dominant nonlinear structure of the engineering system.

The Gaussian Process then models the remaining prediction error.

Instead of directly approximating:

$$\mathbf{y} = \mathbf{f}(\mathbf{x})$$

the system separates the response into:

$$\mathbf{y} = \hat{\mathbf{y}}_{NN} + \mathbf{r}$$

where:

• $\hat{\mathbf{y}}_{NN}$ represents the neural network prediction,
• $\mathbf{r}$ represents the remaining modelling error.

The residual error is defined as:

$$\mathbf{r}(\mathbf{x}) = \mathbf{f}(\mathbf{x}) - \hat{\mathbf{f}}_{NN}(\mathbf{x})$$

The Gaussian Process then learns this residual field:

$$\mathbf{r}(\mathbf{x}) \sim \mathcal{GP} ( m(\mathbf{x}), k(\mathbf{x},\mathbf{x}') )$$

The final surrogate prediction becomes:

$$\hat{\mathbf{f}}(\mathbf{x}) = \hat{\mathbf{f}}_{NN}(\mathbf{x}) + \hat{\mathbf{r}}_{GP}(\mathbf{x})$$

The final predictive mean therefore becomes:

$$\mu(\mathbf{x}) = NN(\mathbf{x}) + \mu_{GP}(\mathbf{x})$$

while the predictive uncertainty is inherited from the Gaussian Process:

$$\sigma^2(\mathbf{x}) = \sigma^2_{GP}(\mathbf{x})$$

This architecture significantly reduces the complexity of the Gaussian Process.

Rather than approximating the complete nonlinear behaviour of the engineering system, it is only required to model the comparatively small discrepancies remaining after the neural network has captured the dominant trends.

The resulting hybrid surrogate combines:

• the scalability of neural networks,
• the probabilistic uncertainty estimation of Gaussian Processes,
• improved local prediction accuracy through residual correction.

The neural network captures:

• global nonlinear trends,
• complex variable interactions,
• high-dimensional relationships.

The Gaussian Process captures:

• local correction behaviour,
• model discrepancy,
• predictive uncertainty.

Figure 8. Hybrid Neural Network–Gaussian Process surrogate. The neural network captures global system behaviour while the Gaussian Process learns residual errors and predictive uncertainty.

This structure is particularly valuable for engineering systems where the majority of the design space follows predictable physical trends, but certain regions contain complex effects that are difficult to model.

For the pulse jet example:

The neural network may learn the overall relationship between:

• chamber geometry,
• fuel flow,
• exhaust dimensions,

and:

• thrust,
• efficiency,
• operating frequency.

The Gaussian Process can then identify where the prediction becomes unreliable due to:

• limited simulation data,
• unusual geometry,
• nonlinear flow effects,
• extrapolation beyond the training region.

The resulting surrogate provides both:

A predicted response:

$$\mu(\mathbf{x})$$

and an associated uncertainty estimate:

$$\sigma(\mathbf{x})$$

This combination changes the role of the surrogate.

It is no longer simply a fast approximation of the physics model.

It becomes an intelligent engineering model that understands both its prediction and the limitations of that prediction.

---

The Surrogate Accuracy Problem

The effectiveness of an optimisation framework depends entirely on the quality of the surrogate model.

If:

$$\hat{f}(\mathbf{x}) \approx f(\mathbf{x})$$

then millions of candidate designs can be explored efficiently.

However, if the approximation is incorrect, the optimiser may converge toward designs that only exist within the mathematical representation of the system.

The surrogate introduces a new engineering problem:

How can the system determine whether its own predictions are reliable?

A conventional regression model provides only a predicted value:

$$\hat{y} = 500N$$

It does not indicate whether this prediction comes from:

• a well-understood region of the design space,
• interpolation between known simulations,
• extreme extrapolation beyond available data.

For autonomous optimisation, this limitation is critical.

The optimiser naturally searches for regions where predicted performance is highest.

Without knowledge of predictive uncertainty, it may exploit regions where the surrogate is simply incorrect.

A design predicted to produce exceptional performance may represent:

• a genuine improvement,
• an unexplored region requiring further investigation,
• a failure of the approximation.

Therefore, an effective engineering surrogate must provide not only a prediction.

Figure 9. Hybrid surrogate models provide both predicted performance and predictive uncertainty, enabling informed engineering decisions.

It must also provide an estimate of uncertainty.

The purpose of a surrogate model is not simply to predict performance.

An optimisation system must also know when those predictions are trustworthy.

Consider two possible pulse jet geometries.

Design A:

$$F = 405 \pm 5 \text{ N}$$

Design B:

$$F = 520 \pm 180 \text{ N}$$

A conventional optimiser selects Design B because the predicted thrust is higher.

However, the uncertainty indicates that the model has little knowledge of this region of the design space.

Design B may represent a genuine improvement, but it may also represent a failure of the approximation.

This distinction fundamentally changes how optimisation should proceed.

Rather than always selecting the design with the highest predicted performance, the optimisation framework must also consider where additional simulations would most improve its understanding of the underlying physics.

Consequently, while neural networks provide an effective global approximation of the design space, uncertainty estimation becomes essential for intelligent engineering search.

---

From Prediction to Decision

The surrogate model consequently changes the role of simulation.

Previously, simulation was used only to evaluate a candidate design.

In an intelligent engineering workflow, each simulation becomes a source of information that improves the model of the design space.

The objective is no longer:

$$\text{Find the best design immediately}$$

Instead, the objective becomes:

$$\text{Select the next simulation that provides the greatest value}$$

This represents a fundamental change in computational engineering.

The optimisation process is no longer simply searching for better designs.

It is actively learning the structure of the design space.

This transition from prediction to information-driven search introduces active learning, where the optimisation algorithm determines which simulations should be performed next.

---

The surrogate model therefore transforms simulation from a purely evaluative process into an information-generating process.

However, prediction alone is insufficient.

An intelligent optimisation system must determine:

• which regions require additional simulation,
• which designs provide maximum information,
• when low-fidelity models are sufficient,
• when high-fidelity validation is necessary.

This requires a decision mechanism that combines predicted performance with predictive uncertainty.

This requires a decision mechanism that combines predicted performance with predictive uncertainty.

The next simulation is therefore selected by maximising an acquisition function:

$$\mathbf{x}_{next} = \arg\max_{\mathbf{x}} A(\mathbf{x})$$

where:

• $A(\mathbf{x})$ represents the acquisition function,
• $\mathbf{x}$ represents a candidate design point,
• $\mathbf{x}_{next}$ represents the next design selected for high-fidelity evaluation.

The acquisition function evaluates the expected value of performing a new simulation at a particular location within the design space.

Rather than considering only predicted performance, the acquisition function combines multiple sources of information:

• predicted engineering performance,
• predictive uncertainty,
• expected information gain.

A simple example is:

$$A(\mathbf{x}) = \alpha \mu(\mathbf{x}) + \beta \sigma(\mathbf{x})$$

where:

• $\mu(\mathbf{x})$ represents the predicted performance from the surrogate model,
• $\sigma(\mathbf{x})$ represents predictive uncertainty,
• $\alpha$ and $\beta$ control the balance between exploitation and exploration.

The first term encourages the optimisation process to evaluate designs that are predicted to perform well.

The second term encourages evaluation of uncertain regions where additional simulations may significantly improve the surrogate model.

This creates a fundamental trade-off:

• Exploitation: evaluate designs expected to provide high performance.
• Exploration: evaluate designs where the model has limited knowledge.

The challenge is therefore no longer constructing a surrogate model alone.

The challenge is determining how to use surrogate predictions and uncertainty estimates to decide which simulation provides the greatest value.

This is the foundation of active learning.

The next stage is therefore active learning: allowing the optimisation process to intelligently select the next simulation.