Fitting more data to a surface in Octave

Need to fit some data to a surface (function of two independent variables) in Octave?

Matlab has a function to do it directly but Octave needs some help.

In this part we consider an example which demonstrates a gradient descent least squares method. This doesn’t require the use of an additional packages in Octave.

# Perform surface fitting to data in octave using a batch adjusted, 
# gradient decent, least squares method without using any additional 
# octave packages

# In principle I can see no reason why this technique will not work for a 
# higher number of dimensions of observed data but the script will need 
# to be adjusted to permit this extended use.

# Example runs in Octave 5.2.0

# J E Green, Bear Instruments LTD © 2020.
# www.bearinstruments.co.uk
# Power Electronics, Embedded Systems & Instrumentation

clear

# for demonstration only, use the communications package to add noise to 
# the contrived data
pkg load communications

# tolerance at which the fit is said to be 'good enough'.
tol = 1e-3

# maximum number of iterations we'll accept.
iterations = 100

# independent variables (one is row matrix the other is a column matrix)
x = [-5:5];
y = [-5:5]';

# observed results (I have contrived these for the purposes of the example)
z = [ -24  -22  -20  -18  -16  -14  -12  -10   -8   -6   -4
  -21  -19  -17  -15  -13  -11   -9   -7   -5   -3   -1
  -18  -16  -14  -12  -10   -8   -6   -4   -2    0    2
  -15  -13  -11   -9   -7   -5   -3   -1    1    3    5
  -12  -10   -8   -6   -4   -2    0    2    4    6    8
   -9   -7   -5   -3   -1    1    3    5    7    9   11
   -6   -4   -2    0    2    4    6    8   10   12   14
   -3   -1    1    3    5    7    9   11   13   15   17
    0    2    4    6    8   10   12   14   16   18   20
    3    5    7    9   11   13   15   17   19   21   23
    6    8   10   12   14   16   18   20   22   24   26 ];

# add noise to the "observations" to give a signal to noise ratio of 0 dB.
z = awgn(z,0);   
    
p_new(1) = 0;
p_new(2) = 0;
p_new(3) = 0;

# coefficient of error carried into the next iteration 
# smaller --> slower convergence
# too large --> oscillatory behaviour and failure to converge
c = 0.0004;

#predicted target
z_pred = zeros(size(x)(2),size(y));

#error at each step
error = zeros(size(x)(2),size(y));

# mean squared error
MSE(1:iterations) = 0;

for itter = 1:iterations
  p(1) = p_new(1);
  p(2) = p_new(2);
  p(3) = p_new(3);
  
  # The function we intend to fit (a bi-variate version of y = mx + c
  # z = mx + ny + c where m, n and c are to be found)
  z_pred = p(1) + p(2)*x + p(3)*y;
   
  error = z_pred - z;
  error_x = error.*x;
  error_y = error.*y;
  
  MSE_val = mean(mean((error.^2)));
  MSE(itter) = MSE_val;
 
  p_new(1) = p(1) - c*sum(sum(error));
  p_new(2) = p(2) - c*sum(sum(error_x));
  p_new(3) = p(3) - c*sum(sum(error_y));
  
  if MSE_val < tol
    break;
  endif
 endfor 

# Transpose the MSE vector from column to row (because it looks nicer when
# it's printed in the console.
MSE = MSE';

# find any zero values in the MSE (this is probably because we didn't need
# all the iterations)
[r c] = find(MSE == 0);

# If there are no zeros in r (i.e. if we used all the iterations)
if isempty(r)
  # plot the mean squared error in the case that the maximum number of 
  # iterations was used
  figure
  semilogy(MSE)
  title ("Convergence");
  # un ; to see the mean squared error with iteration data in the console
  MSE;

# If we didn't use all the iterations
else
  # plot the mean squared error in the case that less than the maximum number
  # of iterations was used. First clean the data up to remove the trailing
  # zeros due to the unused iterations. These don't plot on log paper and
  # cause an error in the console.
  MSEx(1:(r(1)-1)) = [1:(r(1)-1)];
  MSEy(1:(r(1)-1)) = MSE(1:(r(1)-1));
  figure
  semilogy(MSEx,MSEy)
  title ("Convergence");
  # un ; this next line to see the mean squared error with iteration data 
  # in the console
  MSEy';
endif

# plot the residuals
figure
surf(error);
title ("Residuals");
 
# plot the fitted data
figure
surf(z_pred);
title ("Fitted Data");

# plot the observed data 
figure
surf(z);
title ("Observed Data");

# total sum of squares
SST = sum(sum((z - mean(mean(z))).^2));

# sum of squared residuals
SSR = sum(sum(error.^2));

# R^2
Rsq = (1 - (SSR./SST))

# List the fitted coefficients
p