Csdrhrt

I'm trying to implement a Markov Chain class in Python to test some theoretical results with actual iterations over the matrix itself. What I've implemented up to now seems to work and output the expected result, but I'm at a loss on how to calculate the variance on the expected number of steps needed to get to an absorbing state.

I'm pulling the formula used for this calculation from this article on Wikipedia, namely:

$$
(2N-I_t)t-t_{sq}
$$

where:

$$
N: fundamental matrix\
I_t: Identity matrix\
t: Expected no. of steps (N1)\
t_{sq}: Hadamard product of t with itself
$$

The code I use to compute this is:

t = self._es

twoN = np.multiply(2, self._N)

sub = np.subtract(twoN, self._It)

dot = np.dot(sub, t)

variance = np.subtract(dot, np.multiply(t, t))

std = [np.sqrt(el) for el in variance]

The matrix I'm using as an example is the following (HTH coin toss sequence / substring generation):

$$
begin{bmatrix}
1/2& 1/2& 0& 0\
0& 1/2& 1/2& 0\
1/2& 0& 0& 1/2\
0& 0& 0& 1
end{bmatrix}
$$

It follows that the Fundamental matrix is:

$$
begin{bmatrix}
4& 4& 2\
2& 4& 2\
2& 2& 2
end{bmatrix}
$$

The expected number of steps to reach the absorbing state are then:

$$
begin{bmatrix}
10\
8\
6
end{bmatrix}
$$

Thus, following my code, twoN, sub, dot and the variance would be:

$$
twoN:
begin{bmatrix}
8& 8& 4\
4& 8& 4\
4& 4& 4
end{bmatrix}
$$
$$
sub:
begin{bmatrix}
7& 8& 4\
4& 7& 4\
4& 4& 3
end{bmatrix}
$$
$$
dot:
begin{bmatrix}
158\
120\
90
end{bmatrix}
$$
$$
variance:
begin{bmatrix}
58\
56\
54\
end{bmatrix}
$$
$$
std:
begin{bmatrix}
sim7.6\
sim7.5\
sim7.3
end{bmatrix}
$$

However these numbers seem way off to me, and I am not sure if I am messing up the computation somewhere or if the formula I'm using is wrong (or using a notation I don't understand).

For reference, if I run the MC 10.000 times I get a mean value on the number of steps to reach the absorbing state very close to the predicted ones, with the highest deviation I've seen at around 0.15 from the prediction starting from that state.

Any help would be greatly appreciated!

P.S.: I have an additional doubt for which I'd like to not open another question, so if anyone is willing to answer this I'd be really grateful.

Can we say that a MC is absorbing if the absorbing state is reached from any other state? Or does the state need to be reachable from all other states (in one or more steps)?

How I understood it at first was that being reachable from any one state would have been enough, but today I was told that that's not the case and the definition is stronger (namely, reachable from all other states which are not absorbing themselves). Thinking about it, I came up with this matrix that seems to prove my first intuition was wrong, but I'd still like confirmation:

$$
begin{bmatrix}
*& *& 0& 0\
*& *& 0& 0\
0& 0& 1/2& 1/2\
0& 0& 0& 1
end{bmatrix}
$$