Distance

Dist. Squared

Point A

Point B

Point C

Point D

Sum of Dist^2

Line Minimizing X-Distance

Line Minimizing Y-Distance

When the points are lined up vertically it seems that
the Y approximation lines fits it well. For the Y values there 
is difference between them but only vertically 
so the line can go straight up and down hitting every 
point. If I had to pick one, I would say the Y minimizing
line fits the vertical set of points better. 

Solve[{dm == 0, db == 0}, {m, b}]
{{m -> (-3 y[A]^2 + 2 y[A] y[B] - 3 y[B]^2 + 2 y[A] y[C] + 
      2 y[B] y[C] - 3 y[C]^2 + 2 y[A] y[D] + 2 y[B] y[D] + 
      2 y[C] y[D] - 3 y[D]^2)/(-3 x[A] y[A] + x[B] y[A] + x[C] y[A] + 
      x[D] y[A] + x[A] y[B] - 3 x[B] y[B] + x[C] y[B] + x[D] y[B] + 
      x[A] y[C] + x[B] y[C] - 3 x[C] y[C] + x[D] y[C] + x[A] y[D] + 
      x[B] y[D] + x[C] y[D] - 3 x[D] y[D]), 
  b -> (-x[B] y[A]^2 - x[C] y[A]^2 - x[D] y[A]^2 + x[A] y[A] y[B] + 
      x[B] y[A] y[B] - x[A] y[B]^2 - x[C] y[B]^2 - x[D] y[B]^2 + 
      x[A] y[A] y[C] + x[C] y[A] y[C] + x[B] y[B] y[C] + 
      x[C] y[B] y[C] - x[A] y[C]^2 - x[B] y[C]^2 - x[D] y[C]^2 + 
      x[A] y[A] y[D] + x[D] y[A] y[D] + x[B] y[B] y[D] + 
      x[D] y[B] y[D] + x[C] y[C] y[D] + x[D] y[C] y[D] - x[A] y[D]^2 -
       x[B] y[D]^2 - x[C] y[D]^2)/(3 x[A] y[A] - x[B] y[A] - 
      x[C] y[A] - x[D] y[A] - x[A] y[B] + 3 x[B] y[B] - x[C] y[B] - 
      x[D] y[B] - x[A] y[C] - x[B] y[C] + 3 x[C] y[C] - x[D] y[C] - 
      x[A] y[D] - x[B] y[D] - x[C] y[D] + 3 x[D] y[D])}}
Min[{dm},{db}]
This is shown here by the line x=$\frac{y-z}{n}

A = (4.22, 2.28)
B = (3.84, 3.56)
C = (0.86, 1.7)
D = (1.146, 4.082)
A1: x(A)
B1: y(A)
A2: x(B)
A3: x(C)
A4: x(D)
B2: y(B)
B3: y(C)
B4: y(D)
m = -4.8 (slider from -15 to 5, step 0.1)
b = 15 (slider from -10 to 15, step 0.1)
a: y = (m * x) + b
c = Line(A, xAxis)
E = Intersect(a, c)
d = Segment(A, E)
e = Line(C, xAxis)
f = Line(B, xAxis)
g = Line(D, xAxis)
F = Intersect(a, e)
h = Segment(F, C)
G = Intersect(a, f)
i = Segment(B, G)
H = Intersect(a, g)
j = Segment(D, H)
A7 = Length(h)
A8 = Length(i)
A9 = Length(j)
A6 = Length(d)
B6: Length[d]^(2)
B7: Length[h]^(2)
B8: Length[j]^(2)
B9: Length[i]^(2)
B11 = Sum(B6:B9)
n: ((-3 * y(A)^(2)) + ((2 * y(A)) * y(B)) - (3 * y(B)^(2)) + ((2 * y(A)) * y(C)) + ((2 * y(B)) * y(C)) - (3 * y(C)^(2)) + ((2 * y(A)) * y(D)) + ((2 * y(B)) * y(D)) + ((2 * y(C)) * y(D)) - (3 * y(D)^(2))) / (((-3 * x(A)) * y(A)) + (x(B) * y(A)) + (x(C) * y(A))) + (x(D) * y(A)) + (x(A) * y(B)) - ((3 * x(B)) * y(B)) + (x(C) * y(B)) + (x(D) * y(B)) + (x(A) * y(C)) + (x(B) * y(C)) - ((3 * x(C)) * y(C)) + (x(D) * y(C)) + (x(A) * y(D)) + (x(B) * y(D)) + (x(C) * y(D)) - ((3 * x(D)) * y(D))
z: (((-x(B)) * y(A)^(2)) - (x(C) * y(A)^(2)) - (x(D) * y(A)^(2)) + ((x(A) * y(A)) * y(B)) + ((x(B) * y(A)) * y(B)) - (x(A) * y(B)^(2)) - (x(C) * y(B)^(2)) - (x(D) * y(B)^(2)) + ((x(A) * y(A)) * y(C)) + ((x(C) * y(A)) * y(C)) + ((x(B) * y(B)) * y(C)) + ((x(C) * y(B)) * y(C)) - (x(A) * y(C)^(2)) - (x(B) * y(C)^(2)) - (x(D) * y(C)^(2)) + ((x(A) * y(A)) * y(D)) + ((x(D) * y(A)) * y(D)) + ((x(B) * y(B)) * y(D)) + ((x(D) * y(B)) * y(D)) + ((x(C) * y(C)) * y(D)) + ((x(D) * y(C)) * y(D)) - (x(A) * y(D)^(2)) - (x(B) * y(D)^(2)) - (x(C) * y(D)^(2))) / (((3 * x(A)) * y(A)) - (x(B) * y(A)) - (x(C) * y(A)) - (x(D) * y(A)) - (x(A) * y(B)) + ((3 * x(B)) * y(B)) - (x(C) * y(B)) - (x(D) * y(B)) - (x(A) * y(C)) - (x(B) * y(C)) + ((3 * x(C)) * y(C)) - (x(D) * y(C)) - (x(A) * y(D)) - (x(B) * y(D)) - (x(C) * y(D)) + ((3 * x(D)) * y(D)))
k: y = (n * x) + z
I: (A1, B1)
J: (A2, B2)
K: (A3, B3)
L: (A4, B4)
list1: {I, J, K, L}
l = FitLineY(list1)
p: ((-(((-2 * x(A)) * y(A)) + (x(B) * y(A)) + (x(C) * y(A)) + (x(A) * y(B)) - ((2 * x(B)) * y(B)) + (x(C) * y(B)) + (x(A) * y(C)) + (x(B) * y(C)) - ((2 * x(C)) * y(C))))) / ((2 * (x(A)^(2) - (x(A) * x(B)) + x(B)^(2) - (x(A) * x(C)) - (x(B) * x(C)) + x(C)^(2))))
β: (1 / 3 * (y(A) + y(B) + y(C))) + (((x(A) / 3 + x(B) / 3 + x(C) / 3) * (((-2 * x(A)) * y(A)) + (x(B) * y(A)) + (x(C) * y(A)) + (x(A) * y(B)) - ((2 * x(B)) * y(B)) + (x(C) * y(B)) + (x(A) * y(C)) + (x(B) * y(C)) - ((2 * x(C)) * y(C))))) / ((2 * (x(A)^(2) - (x(A) * x(B)) + x(B)^(2) - (x(A) * x(C)) - (x(B) * x(C)) + x(C)^(2))))
q: y = (p * x) + β
r: x = (y - z) / n