Csdrhrt

If you have 3 labeled points on a surface of a paper. Like

    1





2       3

This makes a perfect equilateral triangle.

From this perspective I can say that the camera is on top of the paper looking down. We can say the camera is at coordinate $(0,0,100)$. Which is 0 degree rotation in Z axis and 90 degree rotation in Y axis.

Then, I move the camera to some arbitrary spot. Now, the points are like

         1



2                 3

This looks like the camera is farther back from the paper and was lowered, say at location $(0, -100, 50)$. Which is about -90 degree rotation in Z-axis, and 45 degree rotation in Y axis.

So my question is basically, given the $(x_1,y_1), (x_2,y_2), (x_3,y_3)$. Is there some formula that can take these arbitrary points, compare it with the original 3, to know how much of a X,Y,Z rotation it is of the camera?

I can also rotate on angles like this. For example, I can take the second example from above, then I can rotate my head clockwise, making the numbers flip like

2

    1

3

I think getting a normal vector from the center might be better to find.

Assumption:

I am assuming that the camera is orthographic, since for a perspective projection you would need at least 4 points to determine the position/rotation of the camera.

General Overview of the Method:

Read about the camera transformation here -> https://en.wikipedia.org/wiki/3D_projection#Perspective_projection .

Let the camera transform matrix be $C$. Let the original point be $p$ and the projected point be $b$. Then the orthographic projection is simply $(b_x, b_y)$ where
$$ begin{bmatrix}
b_x\
b_y\
b_z
end{bmatrix}=Ccdot p $$

Let $v_{1,2,3}$ be the vertices of the original triangle. Let $b_{1,2,3}$ be the vertices of the triangle in the transformed coordinate space.
$$Ccdot [v_1 v_2 v_3]=[b_1 b_2 b_3]$$
Thus
$$C=[b_1 b_2 b_3]cdot [v_1 v_2 v_3]^{-1}$$

If you fill in values for $v_{1,2,3}$ and only the $x$ and $y$ coordinates of $b_{1,2,3}$, this should leave you with a matrix with some unknowns ($b_{1z}$, $b_{2z}$, and $b_{3z}$), but from there you can deconstruct the camera transform into rotation (and translation), if you make the assumption that the camera is looking at the origin, and that the camera is a fixed distance away.

Good luck. This should be enough to get you started, but I will expand this answer if you need it.

Let, as a model, we use a camera point $K$ and a camera screen $k$, where $k$ is a plane, and let the orthogonal projection $O$ of point $K$ on the plane $k$ be the origin of the two dimensional coordinate system $Oe_1e_2$ on the screen $k$. Attach a three dimensional coordinate system $Ke_1e_2e_3$ to the point $K$ where $e_1$ and $e_2$ are the unit vectors of $Oe_1e_2$ and let $e_3$ be the unit vector orthogonal to $k$ and pointing from $K$ to $k$. Denote by $nu$ the distance from $K$ to $k$. We may or may not know it.

At first let $A_0B_0C_0$ be an equilateral triangle in a plane $p$ parallel to $k$. Let $G_0 in p$ be the center of $A_0B_0C_0$ so that $G_0$ lies on the $e_3$ axis, i.e. $G_0$ is the orthogonal projection of $K$ onto $p$. Then we project $A_0B_0C_0$ onto the plane $k$ by drawing the lines $KA_0, , KB_0, , KC_0$ and possibly $KG_0$ and denoting by $A, B, C$ and $O$ their intersection points with $k$. Thus we obtain the projected equilateral triangle $ABC$ with center $O$.

Next we move the camera $K$ together with the screen $k$ to a different position and again we project the triangle $A_0B_0C_0$ with center $G_0$ onto the plane $k$ (which is now positioned differently with respect to $p$ and is this possibly not parallel to it) the same way we did before. We draw the lines between point $K$ and the points $A_0, B_0, C_0$ and $G_0$ and look at their respective intersection points $A, B, C$ and $G$ with plane $k$. The resulting triangle $ABC$ is most likely not equilateral.

Think of the coordinate system $Ke_1e_2e_3$, which moved together with the camera and the screen, and recall that $nu$ is the distance between $K$ and $k$, the orthogonal projection of $K$ onto $k$ is $O$, and $Oe_1e_2$ is the coordinate system on the screen $k$ as before.

If we take a plane $q$ parallel to $p$ passing through point say $C$, then the intersection points
$$A_q = KA cap q, ,, B_q = KB cap q, ,, C_q = KC cap q , text{ and } , G_q = KG cap q$$
form an equilateral triangle $A_qB_qC$ with center $G_q$ whose edges are parallel to the edges of $A_0B_0C_0$ (in other words triangle $A_qB_qC$ with center $G_q$ is homothetic image of triangle $A_0B_0C_0$ with center $G_0$ with respect $K$ )

Now, you know the locations of the points $A, , B, , C$ and say $G$ with respect to the screen's $k$ coordinate system $Oe_1e_2$ which means you know the vectors $$vec{OA}, , vec{OB}, , vec{OC}, ,, text{ and } ,, vec{OG}$$ so thus, we can form the vectors
begin{align*}
vec{KA} &= vec{KO} + vec{OA} = vec{OA} + vec{KO} = vec{OA} + nu , e_3\
vec{KB} &= vec{KO} + vec{OB} = vec{OB} + vec{KO} = vec{OB} + nu , e_3\
vec{KC} &= vec{KO} + vec{OC} = vec{OC} + vec{KO} = vec{OC} + nu , e_3\
vec{KG} &= vec{KO} + vec{OG} = vec{OG} + vec{KO} = vec{OG} + nu , e_3\
end{align*}

Thus, there exists constants $lambda, , mu$ and $nu$, if you do not know $nu$
begin{align*}
vec{KA_q} &= lambda , vec{KA}= lambda , vec{OA} + lambda , nu , e_3\
vec{KB_q} &= mu , vec{KB}= mu , vec{OA} + mu , nu , e_3\
vec{KG_q} &= frac{1}{3} big(lambda , vec{KA} + mu , vec{KB} + vec{KC}big)\ &= frac{1}{3} big( lambda , vec{OA} + mu , vec{OA} + vec{OC} + (lambda + mu + 1) , nu , e_3 big)
end{align*}
The restrictions that lead to the equations for the unknowns $lambda, , mu$ and possibly $nu$ are
begin{align*}
|vec{C_qA_q}| &= |vec{A_qB_q}|\
|vec{C_qB_q}| &= |vec{A_qB_q}|\
vec{KG_q} & , |, | , vec{KG}\
end{align*} where the first two say that the edges of equilateral triangle $A_qB_qC$ are equal while the last one represents the fact that the points $K, , G_q $ and $G$ are collinear. As equations, these three conditions are

begin{align*}
|lambda , vec{KA} - vec{KC}|^2 &= |lambda , vec{KA} - mu , vec{KB}|^2 \
|lambda , vec{KA} - vec{KC}|^2 &= |lambda , vec{KA} - mu , vec{KB}|^2 \
|vec{KG_q}times vec{KG}|^2 &= 0\
end{align*}
which written in terms of $lambda, , mu, , nu$ unknown variables turn into

begin{align*}
|lambda , vec{OA} - vec{OC} + (lambda , nu , - nu) ,
e_3 |^2 &= |lambda , vec{OA} - mu , vec{OB} + (lambda ,
nu , - mu , nu) ,
e_3 |^2 \
|mu , vec{OB} - vec{OC} + (mu , nu , - nu) , e_3 |^2 &=
|lambda , vec{OA} - mu , vec{OB} + (lambda , nu , - mu
, nu) ,
e_3 |^2 \
|big(frac{1}{3} big( lambda , vec{OA} + mu ,
vec{OA} + vec{OC} + (lambda + mu + 1) , nu , e_3 big)big)
times big(vec{OG} + nu , e_3 big) |^2 &= 0\
end{align*}

If you know $nu$ then only the first two equations are relevant

begin{align*}
|lambda , vec{OA} - vec{OC} + (lambda , nu , - nu) ,
e_3 |^2 &= |lambda , vec{OA} - mu , vec{OB} + (lambda ,
nu , - mu , nu) ,
e_3 |^2 \
|mu , vec{OB} - vec{OC} + (mu , nu , - nu) , e_3 |^2 &=
|lambda , vec{OA} - mu , vec{OB} + (lambda , nu , - mu
, nu) ,
e_3 |^2
end{align*}

All these systems of equations may have several solutions though.

I haven't checked for typos and mistakes and I do not guarantee this is the best solution.