Kernel functions

Here we provide a description of the different classes used to build and evaluate the RBF and the GRBF surrogate models. Three different kernel functions \(\phi(r)\) are defined in the code, the Exponential kernel (RBF_Exponential, GRBF_Exponential), the Matérn kernel (RBF_Matern, GRBF_Matern) and the Cubic kernel (RBF_Cubic, GRBF_Cubic).

The fit method of these classes solves the system \(A s = F\). In the case of RBF interpolants, the vector \(F\) contains the values of the function where the points have been evaluated and the matrix \(A\) is defined as:

\[\begin{split}A = \begin{bmatrix} \Phi & P \\ P^T & 0. \end{bmatrix}\end{split}\]

In the case of GRBF interpolants, the vector \(F\) contains both the values of the function and its gradient and the matrix \(A\) is defined as:

\[\begin{split}A = \begin{bmatrix} \Phi & \Phi_\mathrm{d} \\ -\Phi_\mathrm{d}^T & \Phi_\mathrm{dd}, \end{bmatrix}\end{split}\]

where \(\Phi\) is the RBF kernel matrix, \(P\) is a matrix with linear polynomial terms, \(\Phi_\mathrm{d}\) is the first derivative of the RBF kernel matrix \(\Phi\) and \(\Phi_\mathrm{dd}\) is the second derivative 1, 2.

The matrix \(\Phi\) is defined as:

\[\Phi_{i,j} = \phi(r_{i,j}),\]

where \(r_{i,j} = \left \| x^i-x^j \right\|\) is the Euclidean distance between the points \(x^i\) and \(x^j\). Tha matrices \(\Phi_\mathrm{d}\) and \(\Phi_\mathrm{dd}\) are defined as:

\[\Phi_{\mathrm{d}_{i,j,k}} = \dfrac{\partial \Phi_{i,j}}{\partial x_k^i} = \dfrac{\partial \Phi_{i,j}}{\partial r_{i,j}} \dfrac{\partial r_{i,j}}{\partial x_k^i},\]

\[ \begin{align}\begin{aligned}\Phi_{\mathrm{dd}_{i,j,k,l}} = \dfrac{\partial \Phi_{\mathrm{d}_{i,j,k}}}{\partial x_l^i} = \dfrac{\partial^2 \Phi_{i,j}}{\partial r_{i,j}^2} \dfrac{\partial r_{i,j}}{\partial x_l^i} \dfrac{\partial r_{i,j}}{\partial x_k^i}\\+ \dfrac{\partial \Phi_{i,j}}{\partial r_{i,j}} \dfrac{\partial^2 r_{i,j}}{\partial (x_k^i) \partial (x_l^i)}.\end{aligned}\end{align} \]

where \(x_k^i\) is the kth component of the ith point in the vector of evaluated points x.

Once the fit method has been used to build the surrogate model, it is possible to evaluate points using the method evaluate. If needed, the internal parameters can be updated using the update method.

1

Regis, R G and C A Shoemaker. 2013. Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Engineering Optimization 45 (5): 529-555.

2

Laurent, L, R Le Riche, B Soulier and P A Boucard. 2019. An Overview of Gradient-Enhanced Metamodels with Applications. Archives of Computational Methods in Engineering 26 (1). 61-106.

Exponential Kernel

The Exponential Kernel is defined as follows:

\[\Phi(r) = \exp \left( -\dfrac{r^2}{2 l^2} \right),\]

where \(l>0\) and \(r=\|y-x\|\), where \(\|\cdot\|\) is the Euclidean norm.

The first derivative of the Exponential Kernel with respect to the distance is defined as:

\[\Phi_{,r}(r) = - \dfrac{r}{l^2} \exp \left( -\dfrac{r^2}{2 l^2} \right)\]

The second derivative is defined as:

\[\Phi_{,rr}(r) = - \dfrac{1}{l^2} \exp \left( -\dfrac{r^2}{2 l^2} \right) + \dfrac{r^2}{l^4} \exp \left( -\dfrac{r^2}{2 l^2} \right)\]

class DyCors.kernels.exponential.GRBF_Exponential(l=1.0)

GRBF Exponential kernel class.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter. Width of the kernel.

Attributes

lfloat or ndarray, shape (d,)

Internal parameter. Width of the kernel.

sndarray, shape(m*(d+1),)

GRBF coefficients.

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

dfndarray, shape (m*d,)

Array of function gradient values at x.

evaluate(y)

Evaluate surrogate model at given points.

Parameters

yndarray, shape (n,d,)

Array of points where we want to evaluate the surrogate model.

Returns

fndarray, shape(n,)

Array of interpolated values.

fit(x, f, df)

Build surrogate model.

Parameters

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

dfndarray, shape (m*d,)

Array of function gradient values at x.

Returns

Phindarray, shape(m,m,)

RBF matrix.

Andarray, shape(m*(d+1),m*(d+1),)

GRBF matrix with gradient terms.

optimize()

Optimize internal parameters as the inverse of the average gradient.

update(l=1.0)

Update internal parameters of the kernel.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter. Width of the kernel.

class DyCors.kernels.exponential.RBF_Exponential(l=1.0)

RBF Exponential kernel class.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter. Width of the kernel.

Attributes

lfloat or ndarray, shape (d,)

Internal parameter. Width of the kernel.

sndarray, shape(m+d+1,)

RBF coefficients.

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

constr(l)

Constraint on the condition number of the kernel matrix.

eloo(l)

Compute leave-one-one error.

evaluate(y)

Evaluate surrogate model at given points.

Parameters

yndarray, shape (n,d,)

Array of points where we want to evaluate the surrogate model.

Returns

fndarray, shape(n,)

Array of interpolated values.

fit(x, f)

Build surrogate model.

Parameters

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

Returns

Phindarray, shape(m,m,)

RBF matrix.

Andarray, shape(m*(d+1),m*(d+1),)

RBF matrix with linear polynomial terms.

optimize()

Optimize internal parameters based on the Leave-One-Out error using a differential evolution algorithm [1], [2]. A constraint is applied to the condition number of the kernel matrix to ensure the smoothness of the function.

References

1

Rippa, S. 1999. An algorithm for selecting a good value for the parameter c in radial basis function interpolation. Advances in Computational Mathematics 11 (2). 193-210.

2

Bompard, M, J Peter and J A Desideri. 2010. Surrogate models based on function and derivative values for aerodynamic global optimization. V European Conference on Computational Fluid Dynamics ECCOMAS CFD 2010, ECCOMAS. Lisbon, Portugal.

update(l=1.0)

Update internal parameters of the kernel.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter. Width of the kernel.

Matérn Kernel

The half integer simplification of the Matérn Kernel is defined as follows:

\[\Phi(r) = \exp \left( -\sqrt{2p+1} \dfrac{r}{l} \right) \dfrac{p!}{(2p)!} \sum_{i=0}^{p} \dfrac{(p+i)!}{i!(p-i)!} \left( 2 \sqrt{2p+1} \dfrac{r}{l} \right)^{p-i},\]

where \(p=\left \lfloor{\nu - 1/2}\right \rfloor , l>0, \nu>=1/2\) and \(r=\|y-x\|\), where \(\|\cdot\|\) is the Euclidean norm.

The first derivative of the Matérn Kernel with respect to the distance is defined as:

\[\begin{split}\Phi_{,r}(r) = - \dfrac{\sqrt{2p+1}}{l} \exp \left( -\sqrt{2p+1} \dfrac{r}{l} \right) \dfrac{p!}{(2p)!} \sum_{i=0}^{p} \dfrac{(p+i)!}{i!(p-i)!} \left( 2 \sqrt{2p+1} \dfrac{r}{l} \right)^{p-i} \\ + \dfrac{2 \sqrt{2p+1}}{l} \exp \left( -\sqrt{2p+1} \dfrac{r}{l} \right) \dfrac{p!}{(2p)!} \sum_{i=0}^{p-1} \dfrac{(p+i)!}{i!(p-i)!} \left( 2 \sqrt{2p+1} \dfrac{r}{l} \right)^{p-i-1}\end{split}\]

The second derivative is defined as:

\[\begin{split}\Phi_{,rr}(r) = \dfrac{2p+1}{l^2} \exp \left( -\sqrt{2p+1} \dfrac{r}{l} \right) \dfrac{p!}{(2p)!} \sum_{i=0}^{p} \dfrac{(p+i)!}{i!(p-i)!} \left( 2 \sqrt{2p+1} \dfrac{r}{l} \right)^{p-i} \\ - \dfrac{4 (2p+1)}{l^2} \exp \left( -\sqrt{2p+1} \dfrac{r}{l} \right) \dfrac{p!}{(2p)!} \sum_{i=0}^{p-1} \dfrac{(p+i)!}{i!(p-i)!} \left( 2 \sqrt{2p+1} \dfrac{r}{l} \right)^{p-i-1} \\ + \dfrac{4 (2p+1)}{l^2} \exp \left( -\sqrt{2p+1} \dfrac{r}{l} \right) \dfrac{p!}{(2p)!} \sum_{i=0}^{p-2} \dfrac{(p+i)!}{i!(p-i)!} \left( 2 \sqrt{2p+1} \dfrac{r}{l} \right)^{p-i-2}\end{split}\]

class DyCors.kernels.matern.GRBF_Matern(l=1.0, nu=2.5)

GRBF Matérn kernel class.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter of the kernel.

nufloat, optional

Internal parameter. Order of Bessel function of the kernel.

Attributes

lfloat or ndarray, shape (d,)

Internal parameter of the kernel.

nufloat

Internal parameter. Order of Bessel function of the kernel.

sndarray, shape(m*(d+1),)

GRBF coefficients.

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

dfndarray, shape (m*d,)

Array of function gradient values at x.

evaluate(y)

Evaluate surrogate model at given points.

Parameters

yndarray, shape (n,d,)

Array of points where we want to evaluate the surrogate model.

Returns

fndarray, shape(n,)

Array of interpolated values.

fit(x, f, df)

Build surrogate model.

Parameters

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

dfndarray, shape (m*d,)

Array of function gradient values at x.

Returns

Phindarray, shape(m,m,)

RBF matrix.

Andarray, shape(m*(d+1),m*(d+1),)

GRBF matrix with gradient terms.

optimize()

Optimize internal parameters as the inverse of the average gradient.

update(l=1.0, nu=2.5)

Update internal parameters of the kernel.

Parameters

lfloat or ndarray, shape (d,)

Internal parameter of the kernel.

nufloat, optional

Internal parameter. Order of Bessel function of the kernel.

class DyCors.kernels.matern.RBF_Matern(l=1.0, nu=2.5)

RBF Matérn kernel class.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter. Width of the kernel.

nufloat, optional

Internal parameter. Order of Bessel function of the kernel.

Attributes

lfloat or ndarray, shape (d,)

Internal parameter. Width of the kernel.

nufloat

Internal parameter. Order of Bessel function of the kernel.

sndarray, shape(m+d+1,)

RBF coefficients.

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

constr(ip)

Constraint on the condition number of the kernel matrix.

eloo(ip)

Compute leave-one-one error.

evaluate(y)

Evaluate surrogate model at given points.

Parameters

yndarray, shape (n,d,)

Array of points where we want to evaluate the surrogate model.

Returns

fndarray, shape(n,)

Array of interpolated values.

fit(x, f)

Build surrogate model.

Parameters

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

Returns

Phindarray, shape(m,m,)

RBF matrix.

Andarray, shape(m*(d+1),m*(d+1),)

RBF matrix with linear polynomial terms.

optimize()

Optimize internal parameters based on the Leave-One-Out error using a differential evolution algorithm [1], [2]. A constraint is applied to the condition number of the kernel matrix to ensure the smoothness of the function.

References

1

Rippa, S. 1999. An algorithm for selecting a good value for the parameter c in radial basis function interpolation. Advances in Computational Mathematics 11 (2). 193-210.

2

Bompard, M, J Peter and J A Desideri. 2010. Surrogate models based on function and derivative values for aerodynamic global optimization. V European Conference on Computational Fluid Dynamics ECCOMAS CFD 2010, ECCOMAS. Lisbon, Portugal.

update(l=1.0, nu=2.5)

Update internal parameters of the kernel.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter of the kernel.

nufloat, optional

Internal parameter. Order of Bessel function of the kernel.

Cubic Kernel

The Cubic Kernel is defined as follows:

\[\Phi(r) = r^3/l^3\]

where \(l>0\), \(r=\|y-x\|\), where \(\|\cdot\|\) is the Euclidean norm.

The first derivative of the Cubic Kernel with respect to the distance is defined as:

\[\Phi_{,r}(r) = 3 r^2/l^3\]

The second derivative is defined as:

\[\Phi_{,rr}(r) = 6 r/l^3\]

class DyCors.kernels.cubic.GRBF_Cubic(l=1.0)

GRBF Cubic kernel class.

Parameters

lfloat or ndarray, shape (d,)

Internal parameter. Width of the kernel.

Attributes

lfloat or ndarray, shape (d,)

Internal parameter. Width of the kernel.

sndarray, shape(m*(d+1),)

GRBF coefficients.

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

dfndarray, shape (m*d,)

Array of function gradient values at x.

evaluate(y)

Evaluate surrogate model at given points.

Parameters

yndarray, shape (n,d,)

Array of points where we want to evaluate the surrogate model.

Returns

fndarray, shape(n,)

Array of interpolated values.

fit(x, f, df)

Build surrogate model.

Parameters

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

dfndarray, shape (m*d,)

Array of function gradient values at x.

Returns

Phindarray, shape(m,m,)

RBF matrix.

Andarray, shape(m*(d+1),m*(d+1),)

GRBF matrix with gradient terms.

optimize()

Optimize internal parameters as the inverse of the average gradient.

update(l=1.0)

Update internal parameters of the kernel.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter. Width of the kernel.

class DyCors.kernels.cubic.RBF_Cubic(l=1.0)

RBF Cubic kernel class.

Parameters

lfloat or ndarray, shape (d,)

Internal parameter. Width of the kernel.

Attributes

lfloat or ndarray, shape (d,)

Internal parameter. Width of the kernel.

sndarray, shape(m+d+1,)

RBF coefficients.

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

constr(l)

Constraint on the condition number of the kernel matrix.

eloo(l)

Compute leave-one-one error.

evaluate(y)

Evaluate surrogate model at given points.

Parameters

yndarray, shape (n,d,)

Array of points where we want to evaluate the surrogate model.

Returns

fndarray, shape(n,)

Array of interpolated values.

fit(x, f)

Build surrogate model.

Parameters

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,)

Array of function values at x.

Returns

Phindarray, shape(m,m,)

RBF matrix.

Andarray, shape(m*(d+1),m*(d+1),)

RBF matrix with linear polynomial terms.

optimize()

Optimize internal parameters based on the Leave-One-Out error using a differential evolution algorithm [1], [2]. A constraint is applied to the condition number of the kernel matrix to ensure the smoothness of the function.

References

1

Rippa, S. 1999. An algorithm for selecting a good value for the parameter c in radial basis function interpolation. Advances in Computational Mathematics 11 (2). 193-210.

2

Bompard, M, J Peter and J A Desideri. 2010. Surrogate models based on function and derivative values for aerodynamic global optimization. V European Conference on Computational Fluid Dynamics ECCOMAS CFD 2010, ECCOMAS. Lisbon, Portugal.

update(l=1.0)

Update internal parameters of the kernel.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter. Width of the kernel.