I have a set of points in 3D space represented by the following NumPy array:
import numpy as nppoints = np.array([4.5403791090466825, -6.474122845743137, 1.720865155131852 4.544710813420152, -6.224218513611945, 1.7088861199877527 4.537233620491136, -5.98484530658863, 1.699488873354222 4.521937968342778, -5.721674150789189, 1.6849114580360904 4.4999241714099405, -5.4938430240669724, 1.679752942910385 4.830182546259025, -5.657936979701614, 1.6888307295378522 5.121468843368871, -5.803097135994744, 1.6954809688529893 5.439088364842268, -5.965512825953125, 1.6981638740242355 5.704276912211997, -6.100013441509505, 1.69575210534024 5.674604213881624, -6.316464211693753, 1.696360866592035 5.6375373276155125, -6.568875993817544, 1.7173100480278745 5.601324312047108, -6.791416283459123, 1.7222351265798983 5.556256817028301, -7.025669295257478, 1.7240530742321507 5.3173007670429975, -6.898939280431394, 1.7278595480033725 5.046628449981028, -6.753703318879904, 1.725804884872875 4.803244289601083, -6.620716409453152, 1.726377963879765 4.822340852273528, -6.379078541501187, 1.7032476446943 4.830532452723338, -6.144069893351172, 1.7047976672875338 4.834760563421453, -5.884305796074045, 1.6876475718597206 5.110719812888194, -6.028142616316405, 1.687525680260671 5.417740311618597, -6.184558989903391, 1.7124527178431916 5.385270875729216, -6.438008203433672, 1.712366908533943 5.355507693509876, -6.661093026054448, 1.729038450873722 5.07348022798482, -6.513010696655368, 1.726279452092121 5.090180621637709, -6.283446799878667, 1.7091940453643182])A plot of these points (and their labels) in 3D space is shown in the image below.
In this image, certain points are circled in red, referred to as the 4-connected points to the point of interest. However, my attempt to solve this problem by finding the closest 4 points (euclidian distance) to the point of interest doesn't work in all cases (points drawn in blue).
It should be noted that the 3D points are contained in a plane or a curved surface. However, the presence of noise in the data readings means that while we can approximate the points to be on or near a surface, they are not precisely on top of it.
Additionally, the grid of points can sometimes be slightly deformed, not forming straight lines, as shown in this cloud of points (raw data).
Do you have any suggestions on how to determine these 4-connected points in 3D space?
Edit 04/04/2024:
Two possible solutions that look very similar in terms of relative angles when the grid is very skewed:


