
Need to fit some data to a surface (function of two independent variables) in Octave?
Matlab has a function to do it directly but Octave needs some help.
In this part we consider an example which demonstrates a gradient descent least squares method. This doesn’t require the use of an additional packages in Octave.
# Perform surface fitting to data in octave using a batch adjusted,
# gradient decent, least squares method without using any additional
# octave packages
# In principle I can see no reason why this technique will not work for a
# higher number of dimensions of observed data but the script will need
# to be adjusted to permit this extended use.
# Example runs in Octave 5.2.0
# J E Green, Bear Instruments LTD © 2020.
# www.bearinstruments.co.uk
# Power Electronics, Embedded Systems & Instrumentation
clear
# for demonstration only, use the communications package to add noise to
# the contrived data
pkg load communications
# tolerance at which the fit is said to be 'good enough'.
tol = 1e-3
# maximum number of iterations we'll accept.
iterations = 100
# independent variables (one is row matrix the other is a column matrix)
x = [-5:5];
y = [-5:5]';
# observed results (I have contrived these for the purposes of the example)
z = [ -24 -22 -20 -18 -16 -14 -12 -10 -8 -6 -4
-21 -19 -17 -15 -13 -11 -9 -7 -5 -3 -1
-18 -16 -14 -12 -10 -8 -6 -4 -2 0 2
-15 -13 -11 -9 -7 -5 -3 -1 1 3 5
-12 -10 -8 -6 -4 -2 0 2 4 6 8
-9 -7 -5 -3 -1 1 3 5 7 9 11
-6 -4 -2 0 2 4 6 8 10 12 14
-3 -1 1 3 5 7 9 11 13 15 17
0 2 4 6 8 10 12 14 16 18 20
3 5 7 9 11 13 15 17 19 21 23
6 8 10 12 14 16 18 20 22 24 26 ];
# add noise to the "observations" to give a signal to noise ratio of 0 dB.
z = awgn(z,0);
p_new(1) = 0;
p_new(2) = 0;
p_new(3) = 0;
# coefficient of error carried into the next iteration
# smaller --> slower convergence
# too large --> oscillatory behaviour and failure to converge
c = 0.0004;
#predicted target
z_pred = zeros(size(x)(2),size(y));
#error at each step
error = zeros(size(x)(2),size(y));
# mean squared error
MSE(1:iterations) = 0;
for itter = 1:iterations
p(1) = p_new(1);
p(2) = p_new(2);
p(3) = p_new(3);
# The function we intend to fit (a bi-variate version of y = mx + c
# z = mx + ny + c where m, n and c are to be found)
z_pred = p(1) + p(2)*x + p(3)*y;
error = z_pred - z;
error_x = error.*x;
error_y = error.*y;
MSE_val = mean(mean((error.^2)));
MSE(itter) = MSE_val;
p_new(1) = p(1) - c*sum(sum(error));
p_new(2) = p(2) - c*sum(sum(error_x));
p_new(3) = p(3) - c*sum(sum(error_y));
if MSE_val < tol
break;
endif
endfor
# Transpose the MSE vector from column to row (because it looks nicer when
# it's printed in the console.
MSE = MSE';
# find any zero values in the MSE (this is probably because we didn't need
# all the iterations)
[r c] = find(MSE == 0);
# If there are no zeros in r (i.e. if we used all the iterations)
if isempty(r)
# plot the mean squared error in the case that the maximum number of
# iterations was used
figure
semilogy(MSE)
title ("Convergence");
# un ; to see the mean squared error with iteration data in the console
MSE;
# If we didn't use all the iterations
else
# plot the mean squared error in the case that less than the maximum number
# of iterations was used. First clean the data up to remove the trailing
# zeros due to the unused iterations. These don't plot on log paper and
# cause an error in the console.
MSEx(1:(r(1)-1)) = [1:(r(1)-1)];
MSEy(1:(r(1)-1)) = MSE(1:(r(1)-1));
figure
semilogy(MSEx,MSEy)
title ("Convergence");
# un ; this next line to see the mean squared error with iteration data
# in the console
MSEy';
endif
# plot the residuals
figure
surf(error);
title ("Residuals");
# plot the fitted data
figure
surf(z_pred);
title ("Fitted Data");
# plot the observed data
figure
surf(z);
title ("Observed Data");
# total sum of squares
SST = sum(sum((z - mean(mean(z))).^2));
# sum of squared residuals
SSR = sum(sum(error.^2));
# R^2
Rsq = (1 - (SSR./SST))
# List the fitted coefficients
p
