Csdrhrt

I currently try to flatten a triangle in 3d space while maintaining the edge lengths. The triangle consists of 3 vertices, all with x,y,z coordinates and is drawn clockwise. The second vertex yields 1 for the z value, the other two vertices are aligned to the x-axis and yield 0 for z.
Directly setting the z value would violate the edge length constraint. The target is to transform the 3d triangle so it could be completely projected in 2d.

I tried to calculate the angle between a vector lying on the ground and an edge vector to get a rotation matrix. To flatten the triangle I would have to rotate the triangle with exactly this angle.
This however is error-prone under real life conditions. I'm currently looking for a way to transform the triangle directly without the need of a rotation.

Let's say the three vertices of the triangle are
$$
vec{v}_1 = left [ begin{matrix} x_1 \ y_1 \ z_1 end{matrix} right ],
quad
vec{v}_2 = left [ begin{matrix} x_2 \ y_2 \ z_2 end{matrix} right ],
quad
vec{v}_3 = left [ begin{matrix} x_3 \ y_3 \ z_3 end{matrix} right ]
$$
You can construct a two-dimensional coordinate system, where $vec{v}_1$ is at origin, $vec{v}_2$ is on the positive $x$ axis, and $vec{v}_3$ is somewhere above the $x$ axis (i.e., has a positive $y$ coordinate).

If we use
$$vec{V}_1 = left [ begin{matrix} 0 \ 0 end{matrix} right ], quad
vec{V}_2 = left [ begin{matrix} h \ 0 end{matrix} right ], quad
vec{V}_3 = left [ begin{matrix} i \ j end{matrix} right ]$$
then the rules to keep the edge lengths intact are
$$leftlbracebegin{aligned}
h^2 &= lVert vec{v}_2 - vec{v}_1 rVert^2 \
i^2 + j^2 &= lVert vec{v}_3 - vec{v}_1 rVert^2 \
(h-i)^2 + j^2 &= lVert vec{v}_3 - vec{v}_2 rVert^2
end{aligned}right.$$
i.e.
$$leftlbracebegin{aligned}
h^2 &= (x_2 - x_1)^2 + (y_2 - y_1)^2 + (z_2 - z_1)^2 \
i^2 + j^2 &= (x_3 - x_1)^2 + (y_3 - y_1)^2 + (z_3 - z_1)^2 \
(h-i)^2 + j^2 &= (x_3 - x_2)^2 + (y_3 - y_2)^2 + (z_3 - z_2)^2
end{aligned}right.$$
If the three points are all separate and not on the same line, there is a solution:
$$begin{aligned}
h &= sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2 + (z_2 - z_1)^2} \
i &= frac{(x_3 - x_1)(x_2 - x_1) + (y_3 - y_1)(y_2 - y_1) + (z_3 - z_1)(z_2 - z_1)}{h} \
j &= sqrt{(x_3 - x_1)^2 + (y_3 - y_1)^2 + (z_3 - z_1)^2 - i^2}
end{aligned}$$
This means that in the triangle, if we drop a line from the third vertex, perpendicular to the line between the first two vertices, the two lines intersect at distance $i$ from the first vertex towards the second vertex. (But note that $i$ can be negative, meaning the opposite direction.) The distance between the third vertex and the intersection is $j$.

OP has a triangle with $z_1 = z_2 = 0$, and wants to find out where the third vertex would be, if the triangle is rotated around the first edge to bring the third vertex to the $xy$ plane.

First, we need two unit vectors. The first unit vector, $hat{u}$, is from $vec{v}_1$ towards $vec{v}_2$:
$$hat{u} = frac{vec{v}_2 - vec{v}_1}{lVertvec{v}_2 - vec{v}_1rVert} = left [ begin{matrix} frac{x_2 - x_1}{h} \ frac{y_2 - y_1}{h} \ 0 end{matrix} right ]$$
The second vector is perpendicular to the first, but also on the $xy$ plane. There are two options:
$$hat{v}_{+} = left [ begin{matrix} frac{y_2 - y_1}{h} \ frac{x_1 - x_2}{h} \ 0 end{matrix} right ] quad text{or} quad
hat{v}_{-} = left [ begin{matrix} frac{y_1 - y_2}{h} \ frac{x_2 - x_1}{h} \ 0 end{matrix} right ]$$
Typically, you pick the one that is in the same halfspace as $vec{v}_3$, i.e. has the larger (positive) dot product; this corresponds to the smaller rotation angle:
$$hat{v} = begin{cases}
hat{v}_{+}, & hat{v}_{+} cdot vec{v}_3 ge hat{v}_{-} cdot vec{v}_3 \
hat{v}_{-}, & hat{v}_{+} cdot vec{v}_3 lt hat{v}_{-} cdot vec{v}_3 end{cases}$$

Then, the location of the third vertex on the $xy$ plane is $vec{v}_3^prime$,
$$vec{v}_3^prime = vec{v}_1 + i hat{u} + j hat{v}$$

This is the exact same location you get, if you rotate the triangle around the edge between vertices $vec{v}_1$ and $vec{v}_2$, bringing the third vertex also to the $xy$ plane. The two options, $hat{v}_{+}$ and $hat{v}_{-}$ correspond to rotations that differ by 180°.

Given three points $p_1$, $p_2$ and $p_3$.

Define two circles $C_1$ with $p_1$ as center and $l_1$ as radius and $C_2$ with $p_2$ as center and $l_2$ as radius.

Where $l_1 = |p_1 - p_3|$ and $l_2 = |p_2 - p_3|$ .

The intersection of circles $C_1$ and $C_2$ define two points, one of which can be identified with the point $p_3$.

Now, given any arbitrary plane with normal $n$ containing the points $p_1$ and $p_2$ you can compute the point $p_3$ as one of the intersection points of the above two circles both defined with normal $n$.

See: circle circle intersection + cordinates + 3d + normal plane