Csdrhrt

The full question: Prove or disprove that the set of all continuous relations mappings $ phicolon [0;1]to [0;1]$, for which $phi(0)=0$, $phi(1)=1$ and $x<y implies phi(x)< phi(y)$ forms a group with respect to the operation of superposition/composition.

I was able to prove closure by noting that the intersection of their domains is simply the interval $[0;1]$ itself thus the composition will also be a relation mapping from $[0;1]$ to $[0;1]$.

At this point, I decided to prove associativity. I first considered this well-known proof:
$$hcirc(gcirc f)(x)=h(gcirc f(x))= h(g(f(x))=hcirc g(f(x))=(hcirc g)circ f(x)$$
However, this is a proof for function composition whereas the question is about relations mappings. So, since the question asked about relations mappings and not just functions, is this proof of associativity still valid? After all, it seems to me that all the continuous increasing relations mappings from $[0;1]$ to $[0;1]$ are indeed functions. (I'm guessing this statement would require it's own proof, which i'm trying to draft as well.)

Stuck on associativity, I decided to see if the other group axioms would be easier to work with. I verified than an identity function $i(x)=x$ exists, satisfying the conditions of the problem. Then I had more trouble with proving that the inverses will also be continious and within the interval so I'm inclined to conclude that the set is not a group. Any hints on how to proceed with this proof or disproof?

$require{AMScd}$
So first of all the associativity follows from the general fact about functions: if we have

$$begin{CD}
A @>{f}>> B @>{g}>> C @>{h}>> D
end{CD} $$

then $hcirc(gcirc f)=(hcirc g)circ f$ regardless of what those functions are. This can be checked by evaluating each side at $xin A$ as you've done yourself.

Now the set of all bijections (not necessarily continuous) from a set to itself is a group together with $circ$ as a group multiplication. Denote this group by $mathcal{F}(X)$.

So now it is enough to show that the set you are looking at is a subgroup of $mathcal{F}(X)$.

(1) So first of all if $f:[0,1]to[0,1]$ is such that $f(0)=0$, $f(1)=1$ and $f$ is continuous and monotonic then $f$ is a bijection. This follows from the intermediate value theorem.

Denote by $mathcal{M}([0,1])$ the set of all those functions.

(2) The set $mathcal{M}([0,1])$ is closed under $circ$. This you claim you've shown. The proof is quite simple though. If $x<y$ then $f(x)<f(y)$ and thus so is $g(f(x))<g(f(y))$. Meaning $gcirc finmathcal{M}([0,1])$ if both $f,ginmathcal{M}([0,1])$.

(3) If $fin mathcal{M}([0,1])$ then $f^{-1}inmathcal{M}([0,1])$. So obviously $f^{-1}(0)=0$ and $f^{-1}(1)=1$. Now we will show that $f^{-1}$ is monotonic. Indeed, assume that $x<y$. Then $x=f(a)$ and $y=f(b)$ since $f$ is bijective. But since $f$ is monotonic then case when $a=b$ or $a>b$ are impossible. It follows that $a<b$ and thus $f^{-1}(x)<f^{-1}(y)$.

Finally $f^{-1}$ is continuous simply because $[0,1]$ is compact and so $f$ is closed.